Olivier Allain

EFFECTIVE DEMAND: SECURING THE FOUNDATIONS

 and Allain (2009).

The production period

It is now clear to me that Keynes's theory rests on production periods of different durations, each one comprising several days: 'on any day in any firm a number of production periods of differing lengths overlap' (Keynes, 1973, p. 180). Today employment decision then partly depends on what is expected for next days, hence the definitions of D as depending on entrepreneurs' short-term expectations where short-term is not restricted to today but takes into account the future days of the production period. More precisely, D depends on some receipts which spread over several days. Eventually, today effective demand is the intersection between today Z and D functions.

In addition, because of overlapping production periods, there is 'no definite relationship between aggregate effective demand at one time and aggregate income at some other later time' (Keynes, 1973, p. 179-180) as MH shows himself concluding that 'Y and D* are quite different ' (p. 62).

In short, I must thank MH for having made it much clearer. However, let us notice that neither consumption nor investment have been mentioned here.

Employers and dealers

MH introduces further a distinction between employers and dealers which I first found odd and cumbersome before realising it makes things more comprehensible. Accordingly, MH claims that entrepreneurs' expectations are almost always fulfilled: 'since production is undertaken only against forward orders, the employer's expectations are fulfilled by contract, and can be disappointed only if the dealer defaults upon delivery (this would represent a windfall loss for the employer, which does not affect present or future employment decisions). Whether the dealers' expectations are fulfilled is literally a question for another day and depends on whether they have correctly judged future demand at the time of delivery in a given state of expectation…' (p. 67). As a result, if I well understood, one can say that unforeseen changes in demand are mainly bore by dealers, through windfall profits or losses. These unforeseen changes affect employers only indirectly, through the impact of market prices on forward prices. In other words, the multiplier has an effect on current market prices before it affects short-term expectations, and then tomorrow employment.

These developments are very stimulating and will fuel my further reflection. However, it seems excessive to pretend that Keynes suggests this distinction between employers and dealers while quotations in footnote no 15 do not clearly establish that point. Consequently, I think MH departs too much from Keynes's analysis. Indeed, what does remain of entrepreneurs' expectations if these 'expectations are fulfilled by contract' (p. 67)? My opinion is that Keynes is here more traditional than MH in the sense that he does not make the same distinction between market and forward prices. In addition, he clearly assumes that firms' expectations may be unfulfilled (not at the aggregate level but at the disaggregated one): 'when one is dealing with aggregates, aggregate effective demand at time A has no corresponding aggregate income at time B. All one can compare is the expected and actual income resulting to an entrepreneur from a particular decision' (Keynes, 1973, p. 180). Also, 'I now feel that if I were writing the book again I should begin by setting forth my theory on the assumption that short-period expectations were always fulfilled; and then have a subsequent chapter showing what difference it makes when short-period expectations are disappointed' (Keynes, 1973, p. 181).

The Aggregate Supply Function

My own interpretation is that Z is built as an aggregate function from the entrepreneurs' point of view: assuming m firms, Z is the sum of the m individual supply functions provided that, according to Keynes's adhesion to the first classical postulate, the profit maximisation condition is fulfilled (i.e. marginal cost equal price). Consequently, labour (N) influences Z through its impact on marginal cost as well as on production. Z does not include given prices but a condition on prices: in order to hire N, price may equal w/Q'(N).

JH's analysis is broadly the same as mine, for instance when he explains that 'unit supply price will grow with employment under conditions of decreasing marginal returns to labour' (p. 730).

For his part, MH does not make explicit Z specification and properties. It may be read as an aggregate function from the entrepreneurs' point of view in appendix A (eq. A2a, p. 73), but the composition of the "forward offer prices" vector (p s ) raises questions: does it include functions (as in JH and OA specifications where the supply price depends on N) or rather scalars? In the second case, there should be a problem because profit maximising firms have no degree of freedom to choose n, and Z(N) reduces to one point.

The third appendix does not give any answer, even if p s depends on Y (eq. C4, p. 78), because MH focuses here on market prices for goods which have been already produced, then N is not at stake.

To sum up, I regret that MH did not clarify Z construction, especially because he puts the emphasis on producers' behaviour in his article.

What do entrepreneurs expect when taking their hiring decisions? The issue about one or two functions on the demand side

This question has its importance in connection with Keynes's definition of D as 'the proceeds which entrepreneurs expect to receive from the employment of N men' (GT, p. 25).

According to the conventional approach, expectations are about consumption (D 1 ) and investment (D 2 ). On the contrary, my article (among others) rests on the idea that a distinction between aggregate demand and global expenditures is necessary, just because it is not possible to specify D = f(N) = D 1 + D 2 as an aggregate function from the entrepreneurs' point of view.

Indeed, the only way to specify D 1 + D 2 as an aggregate function is to aggregate households' consumption behaviour on the one hand, and firms' investment behaviour on the other hand (or households' behaviour if they hold capital goods). For an entrepreneur who is taking his hiring decision, D 1 + D 2 may have at best an existence at the global level, for the economy as a whole, before to have some reality at the disaggregated level. Adopting such approach means however that one assumes omniscient entrepreneurs who compute the global expenditure first, and then deduce the share of this global expenditure they carry out. I think it does not match Keynes's entrepreneur picture.

On the contrary, according to the second strand of interpretation, entrepreneurs should not be omniscient. The individual firm's demand function results from an entrepreneur wondering how many workers to hire for a given or expected price, say p'. For p', each entrepreneur can compute his receipts depending on how many workers are hired 2 . The aggregation of these individual functions results in D which fits very well with D definition, just as GT's following paragraph ('Now if for a given value of N...', GT, p. 25) fits well with firms comparing their proceeds with a maximisation profit condition in order to determine N. Several comments must be done here.

a. Let me first underline that the comparison between Z and this definition of D results in the volume of employment at the aggregate as well as at the disaggregate level (because each entrepreneur can build his own Z and D curves 3 ). At the opposite, a comparison between Z and D 1 + D 2 results in employment at the aggregate but not at the disaggregate level. On this subject, the former approach is better for articulating firms' decisions and macroeconomic outcomes.

b. As Section I of Chapter 3 focuses on firms' employment behaviour, and also because Keynes clearly defines D from the entrepreneurs' point of view, I still think that the former 2 Let us recall that a firm can produce and sell any quantity of goods at the market price in a competitive economy.

3 See OA, pp. 8-9.

approach is more relevant than the second, provided that prices expectations relates to goods which have different delivery dates. In the other case, it would mean that Keynes refers to a hidden function. This is not my interpretation. Indeed, let us recall that Keynes does not mention consumption or investment in Section I of Chapter 3. To my knowledge, he never relates entrepreneurs' expectations to consumption and investment expenditures.

Besides, entrepreneur's decision rests on his realised and expected sale-proceeds (GT, p. 51). Or, when changes were unforeseen, the entrepreneur takes price or stock variations into account (GT, p. 123). To sum up, entrepreneurs' decision rest on their own business.

c. It seems to me that MH takes the same way: 'production and employment decisions are reserved to entrepreneurs, by definition, based on their price expectations' (p. 56). And further: 'each day firms must decide, in a short-period equilibrium process that Keynes calls 'the principle of effective demand', how much employment to offer today based on their expectations of the market prices they will receive for the heterogeneous finished output that will emerge at the end of the various production periods' (p. 60). Employment does not depend on entrepreneurs' expectations about propensity to consume or investment. On the contrary, changes in consumption or investment entail changes in market and forward prices, and then changes in entrepreneurs' behaviour. To sum up, what MH calls the aggregate demand function (i.e. D 1 + D 2 ) does not refer to 'the proceeds which entrepreneurs expect to receive from the employment of N men'. On the contrary, it seems that MH makes an implicit use of 'my' specification of D.

d. I again ask MH how he defines p s . In the event of a scalars vector, I wonder if MH did not merge Z and 'my' specification of D in the same function: p s being given, firms choose n in order to maximise their profit and Z(N) reduces to a point which is the effective demand.

e. For his part, JH seems to share my interpretation when he writes, without any mention to the propensity to consume, that 'if an entrepreneur takes the demand price as exogenously given, total expected sales proceeds will be a linear function of the quantity produced, and hence a concave function of the level of employment if decreasing returns to labour are assumed. Aggregation of the individual entrepreneur's expected demand curves leads to an aggregate demand curve (D) ...'. But, the continuing citation is really puzzling as he asserts that '[D] is concave as long as the marginal propensity to consume is smaller than one' (p. 733). From my point of view, JH makes confusion between the two distinct functions because the propensity to consume has nothing to do here. Moreover, one can wonder why he questions the concavity of D while he gives a clear explanation two lines above. In some sense, in is interpretation, the D function is over determined.

Then, JH omits to mention that Casorosa's critics do not concern some inconsistency in the construction of D, but the conventional interpretations which view 'the expected demand function as the entrepreneurs' expectation of the expenditure function' (Casarosa, 1981, p. 192). On the contrary, Casarosa pleads on an accurate distinction between the two functions, as I do in my article.

The closure of the system, temporary equilibriums and convergence

Assuming that individual firms do not directly expect D 1 + D 2 raises an important interrogation about the closure of the system: it should be proved that these firms adjust their production in order to respond to exogenous changes in consumption or investment. My feeling is that Keynes did not give a formal proof but was sure that firms respond properly4 . It was the main goal of my article to make explicit such formal proof while Keynes just gives some indications in Chapters 5 and 10 (Section IV).

MH presentation about overlapping production period shows the complexity of the issue: 'aggregate effective demand and income are not commensurable because they do not have the same dimensions in time ' (p. 62). This probably explains why Chapter 3 does not include any consideration about system closure or balance accounting.

I admit confusion in my article because I wrongly assumed that each production period lasts only one day. But, because GT is a general theory, it has to correctly work under this simplifying assumption as well as in a more complex framework. My analysis should then not be considerate as wrong, just as incomplete.

As a result, the temporary equilibrium analysis in my article (macroeconomic changes induce today prices or stocks adjustments which affect tomorrow firms' behaviour) remains relevant.

The analysis is also consistent with several quotations from GT. For instance, in the case of unforeseen expansion in the capital goods industries, 'the efforts of those newly employed in [these] industries to consume a proportion of their increased incomes will raise the prices of consumption-goods until a temporary equilibrium between demand and supply has been brought about partly by the high prices causing a postponement of consumption, partly by a redistribution of income in favour of the saving classes as an effect of the increased profits resulting from the higher prices, and partly by the higher prices causing a depletion of stocks' (GT,.

This approach seems also consistent with MH's analysis of the multiplier: 'the multiplier is a market period (ex post) relationship between the realised values of consumption and investment output, which may well influence the state of expectation, but is not directly itself a causal element of the principle of effective demand' (p. 70).

In short, I think that, in the 'theoretical world', an unforeseen change in investment results in the following outcomes.

a. The principle of effective demand holds provided that employment is determined by today entrepreneurs' expectations.

b. As change in investment is unforeseen, it results in market adjustments on prices and/or stock of inventories (temporary equilibrium).

c. Entrepreneurs take these market adjustments into account when formulating their expectations tomorrow morning, hence the convergence toward an equilibrium where expectations are fulfilled5 .

It seems to me that JH agrees with this interpretation, for instance as he writes that: 'the quantity reactions do not take place within the production period but -if at all -in the transition from one period to the next' (p. 736). On the contrary, MH's approach diverges at least in two ways.

• MH restricts the principle of effective demand to the first item (a), while I include (maybe wrongly) the three items.

• According to MH, the causation runs from, say, yesterday evening results to today expectations. Then, these expectations will be fulfilled today evening because production is undertaken forward orders. To my standpoint, today expectations may be unfulfilled this evening because production is not undertaken forward orders.

Eventually, I would like to ask MH if he thinks that, in GT (if not in the only Chapter 3), there is a theoretical step by step adjustment of the type I described above. Moreover, I think his distinction between employers and dealers allows him to drop artificially the adjustment problem out of the story.

The specification of the consumption function and the two demand functions once again

Considerations about temporary equilibriums lead to questions about the consumption function specification. Indeed, N being given, the value of firms' receipts (R) may change because of market prices adjustment. For convenience, let us assume that the whole receipts are distributed to households (through wages and profits) and that consumption behaviour only rests on a given marginal propensity to consume (c). Under these assumptions, the value of consumption expenditures (D 1 ) depends on the price level as well as on employment:

D 1 = c.R = c.p.Q(N)
In other words, for any given N, an increase of the current market price entails a rise of R and then of D 1 , although consumption remains unchanged in real terms. Many important implications should be underlined. b. More precisely, one can still think that prices are set on the market. But, at the macroeconomic level, buyers' price is not independent from sellers' price. Everything happens as if sellers propose their own price. If it is too low, the closure of the system entails an excess demand (which may be solved by an increase of the price or by a drawing from the stocks of goods)6 .

c. As a crucial result, D 1 (in nominal terms) does not exist before a price has been proposed to households. Then the function (D 1 + D 2 ) cannot be drawn on the N/Z diagram. Consequently, there is no possibility for a unique intersection point between (D 1 + D 2 ) and Z. Effective demand cannot be defined in the way which has been taken by JH and MH among many others.

The main critic I address to MH and JH articles may be that they do not take this closure difficulty into account. How do they specify the demand price p d ? Do they think that D 1 can be defined independently of Z, and how? d. One can eventually argue Keynes specifies D 1 as a function of N rather than R, that is D 1 = χ(N). Does it mean that Keynes made a mistake? I do not think so. My interpretation is that the two first sections of Chapter 3 have different goals • In Section I, Keynes focuses on firms' production behaviour. He attempts to prove that his theory will not depart from competitiveness and profit maximisation hypotheses even if its analysis refers to the whole economy. In other words, he explains the mechanisms at stake in the first classical postulate. • In Section II, he summarises the macroeconomic theory which will essentially be developed in Books III and IV.

Indeed, Keynes gives the impression that issues about competitiveness and profit maximisation are definitely solved at the end of Section I. That allows him to adopt another point of view in Section II which begins as follow: 'A brief summary of the theory of employment to be worked out in the course of the following chapters...' (GT, p. 27). Above all, that allows him to put p' (resulting from Section I) in the consumption function which therefore only depends on employment level: D 1 = c.p'.Q(N) = χ(N).

Conclusion

My actual state of mind may be summarized in the next points.

a. I am still convinced that GT faces a closure difficulty between the macroeconomic outcomes and the microeconomic behaviour of firms. In short, individual firms' behaviour could not directly depend on consumption and investment expenditures because they do not have any reality at first at their level, hence the necessity to make a distinction between the global expenditure and the aggregate demand functions.

b. Did Keynes have the same concern? Nobody knows. But two points must be underlined in the event of a negative answer:

• A function such as D = D 1 + D 2 cannot be built as an aggregate from the entrepreneurs' point of view. • Owing to a problem of price determination, D 1 cannot be determined independently from Z. One can perhaps skip 'my' interpretation of D, but only if the link between D 1 and Z is made explicit.

c. I still think that employment is always at equilibrium, provided that equilibrium refers to the intersection between Z and D.

d. However, resulting from a closure necessity, expectations may be unfulfilled (at the end of the day as well as later). The topics of temporary equilibrium and convergence toward fulfilled expectations then remain relevant.

Allain O. ( 2009) "Effective Demand and Short-term Adjustments in the General Theory", Review of Political Economy, 21(1), pp. 1-22. Casarosa, C. (1981) The microfoundations of Keynes's aggregate supply and expected demand analysis, Economic Journal,91,

  a. Households are indifferent to the price level when they take their consumption decision.They just take the market price as given, without any effect on their real consumption. This outcome echoes the interpretation of many Keynesians who claim that prices are set (or proposed) by entrepreneurs. It reinforces also my own interpretation about the D function.Accordingly, I think that JH and MH are wrong when they put a demand price in their demand function (p d in MH's model, p. 73).

Correspondence address : Olivier Allain, Centre d'Economie de la Sorbonne, 106-112 Boulevard de l'Hôpital, 75013 Paris, France. E-mail : olivier.allain@univ-paris5.fr.

th "Dijon" Post-Keynesian Conference -Roskilde University -13th -14th May 2011

Someone told me that Joan Robinson said she had to be compared to a horse (laboriously digging her furrow) while Keynes was a tiger (brilliant but not really interested in details). If someone knows the reference, I would be happy to have it.

Things are of course more complex in the 'real world' where entrepreneurs have to take into account many parameters and changes at the same time.5 th "Dijon" Post-Keynesian Conference -RoskildeUniversity -13th -14th May 2011

At the microeconomic level, the rules remain those of handbooks: the income and general level of price being given, relative prices determine the composition of the basket of consumption goods, and changes in relative prices entail substitutions between goods.5 th "Dijon" Post-Keynesian Conference -RoskildeUniversity -13th -14th May 2011


Pierre Fleckinger
email: pierre.fleckinger@shs.polytechnique.fr.

Informed Principal and Countervailing Incentives

Keywords: Informed Principal, Countervailing Incentives, Risk Neutrality JEL Classification: D82

It has been shown by Maskin and Tirole (1990, proposition 11) that with quasi-linear preferences and private values, an informed principal neither gains nor loses if her private information is revealed before contracting takes place. The note shows that this result may not hold when the agent faces countervailing incentives.

Introduction

In a well-known contribution, [START_REF] Maskin | The Principal-Agent Relationship with an Informed Principal: The Case of Private Values[END_REF] study the case of bilateral adverse selection in contracting with private values. Their last result (Maskin and Tirole, 1990, proposition 11) has an important implication for the theory of adverse selection: they show that when the agent (he) is risk-neutral, the principal (she) can not benefit from her private information. This amounts to say that all the results obtained in the standard quasi-linear setting with an uninformed principal translate directly to the informed principal case, up to a parameterization. The aim of this note is to show that this result is no more true when one removes the assumption that one type of agent is always more efficient than the other. Relatedly, [START_REF] Cella | Informed Principal with Correlation[END_REF] shows that if the types are correlated, the principal is better off if her information is not revealed.

Following [START_REF] Lewis | Countervailing Incentives in Agency Problems[END_REF], we consider the [START_REF] Baron | Regulating a Monopoly with Unknown Costs[END_REF] framework with only two types, and allow for countervailing incentives that arise when fixed and marginal costs are negatively related. Then, depending on the level of the characteristic (say, quantity) traded, one type of agent or the other may be more efficient. When the agent is not sure about the principal's type, he does not know what quantity will be traded, and he may not know whether he is the efficient type or not. This sometimes softens the incentive constraints, and therefore makes pooling desirable for the principal. We illustrate this idea in a setting where such effect may even allow to implement the first-best allocation.

Setting

The notations are borrowed from [START_REF] Maskin | The Principal-Agent Relationship with an Informed Principal: The Case of Private Values[END_REF] whenever possible. The principal and the agent exchange some good with contractible attribute y ∈ R + (say, quantity or observable quality) and a monetary transfer t ∈ R + . The agent has type θ i with probability p i , i = 1, 2, such that p 1 + p 2 = 1, and a type-independent reservation utility equal to 0. The principal has two possible types α j , j = 1, 2, with probability π j such that π 1 + π 2 = 1. The utility function of the principal is

V (y, t, α) = αv(y) -t
where v is increasing and strictly concave, and without loss of generality, α 1 < α 2 . The utility of the agent is U(y, t, θ) = t -(F (θ) + θy)

In the following, the notations are shortened with indices pertaining to types: Subscripts refer to the agent and superscripts to the principal. We assume the following regarding fixed costs F (θ) and marginal costs θ:

Assumption 1 θ 1 < θ 2 and F (θ 1 ) = F 1 > F 2 = F (θ 2 ).
In addition, we assume that the principal's preferred levels if he knew the agent's type,

y j * i = (v ′ ) -1 θ i α j , satisfy:
Assumption 2 y j * i > 0 for all i, j.

Given assumption 1, technology 1 performs better for high levels of y than technology 2, while the converse is true for low levels. Indeed, let ŷ = F 1 -F 2 θ 2 -θ 1 , then the following holds:

F 1 + θ 1 y ≤ F 2 + θ 2 y if and only if y ≥ ŷ (1)
This is the source of countervailing incentives in this model: for a given (y, t), one or the other type of agent may have an incentive to misrepresent his type, but which one depends on y. In turn, the level that the principal wants to implement is also a function of her own private valuation 1 . We know from [START_REF] Myerson | Mechanism Design by an Informed Principal[END_REF] that we can restrict attention to mechanisms where first the principal offers a full contingent contract, then the agent and the principal simultaneously announce their type, under incentive and interim participation constraints, and finally the corresponding allocation is implemented. The informed principal must thus solve simultaneously the program (P j ) for each type α j , which is Max

{y j i ,t j i } p 1 V j (y j 1 , t j 1 ) + p 2 V j (y j 2 , t j 2 ) π 1 U i (y 1 i , t 1 i ) + π 2 U i (y 2 i , t 2 i ) ≥ 0 (IR i ) 2 j=1 π j U 1 (y j 1 , t j 1 ) ≥ 2 j=1 π j U 1 (y j 2 , t j 2 ) (IC 1 ) (P j ) 2 j=1 π j U 2 (y j 2 , t j 2 ) ≥ 2 j=1 π j U 2 (y j 1 , t j 1 ) (IC 2 ) 2 i=1 p i V 1 (y 1 i , t 1 i ) ≥ 2 i=1 p i V 1 (y 2 i , t 2 i ) (IC 1 ) 2 i=1 p i V 2 (y 2 i , t 2 i ) ≥ 2 i=1 p i V 2 (y 1 i , t 1 i ) (IC 2 )
1 Maskin and Tirole (1990) assume that U (y, t, θ 1 ) > U (y, t, θ 2 ) for all (y, t), and a Spence-Mirrlees property for some results. Here, the first assumption is relaxed, as (1) illustrates, and that is the essential point. A Spence-Mirrlees condition holds here (that boils down to c 1 < c 2 ), but is not important in itself. The same results obtain with the following specification, that entails no fixed cost and no single-crossing:

U i (y, t) = t-c i (y), with c i (0) = c i (0 + ) = 0, 0 < c ′ 2 (0) < c ′ 1 (0) and 0 ≤ c ′′ 1 < c ′′ 2 .
A property analogous to (1) would then hold, namely, there is some ŷ such that c 1 (y) ≤ c 2 (y) if and only if y ≥ ŷ. Similarly, a type-dependent reservation utility -as in [START_REF] Maggi | On Countervailing Incentives[END_REF] and [START_REF] Jullien | Participation Constraints in Adverse Selection Models[END_REF] -could also generate the same results, since reservation utility would play the same role as fixed cost.

where the first two incentive constraints concern the agent and the last two concern the principal. Depending on the parameters, the solutions to this program have (very) different structures. The next section focuses on the case in which the principal may benefit from countervailing incentives.

Results

An important implication of countervailing incentives for the principal is the following:

Proposition 1 The informed principal can achieve her first-best payoff if and only if:

π 1 y 1 * 2 + π 2 y 2 * 2 ≤ ŷ ≤ π 1 y 1 * 1 + π 2 y 2 * 1 (2)
Proof. Sufficiency. Assume that (2) holds. We want to show that the allocation {y j * i , t j * i = F i + θ i y j * i } is implementable. a. The participation constraint of the agent is binding.

b. The incentive constraints of the principal are necessarily satisfied: by definition, {y j * i } are her preferred levels. c. Consider now the incentive constraint of agent 1; because (IR 1 ) is binding, it writes:

2 j=1 π j U 1 (y j 2 , t j
2 ) ≤ 0 or:

F 1 + θ 1 (π 1 y 1 * 2 + π 2 y 2 * 2 ) ≥ π 1 t 1 * 2 + π 2 t 2 * 2 = F 2 + θ 2 (π 1 y 1 * 2 + π 2 y 2 * 2 )
But since π 1 y 1 * 2 + π 2 y 2 * 2 ≤ ŷ, we know from (1) that the inequality is satisfied.

d. The case of agent 2 is symmetric.

Necessity. Now, assume that her preferred outcome is implementable by the principal. Then it must be the case that the incentive constraints are satisfied. It is straightforward to see by reverting the preceding calculations that this implies (2). Under some circumstances, the principal can implement her first-best choice. However, this may simply come from the countervailing effect, and not be a consequence of the principal's private information. Indeed, if for α j one has y j * 2 < ŷ < y j * 1 then, by applying proposition 1 to π j = 1, we know the principal α j can implement the first-best. If both α 1 and α 2 satisfy the inequality, both types can implement the first-best independently and the fact that the principal is privately informed has no consequence. As the next corollary shows, however, this is true in broader circumstances.

ŷ y 1 * 2 y 1 * 1 y 2 * 2 y 2 * 1 F 2 F 1 α 1 v(y) α 2 v(y) y
Corollary 1 If y 1 * 1 < ŷ < y 2 * 2 , no principal could implement the first-best if the agent knew her type. However, if v ′′′ ≤ 0, there always exist beliefs (π 1 , π 2 ) of the agent such that both types of principal can implement their preferred allocation.

Proof. Consider y * (α, θ), the mapping from both types to the principal's preferred solution. By the first-order condition, we have v ′ (y * ) = θ α for any α, θ. It is clear that ∂y * ∂α > 0 and ∂y * ∂θ < 0. Since α 1 < α 2 , under the given condition, we have y 1 * 2 ≤ y 1 * 1 < ŷ < y 2 * 2 ≤ y 2 * 1 . Assume the agents knows α, then π j = 0 or 1, and condition (2) is violated. Thus no principal can implement the first-best here if α is common knowledge. Now we want to find (π 1 , π 2 ) that verify condition (2). Satisfying it is equivalent to be able to find a π 1 such that

y 2 * 2 - ŷ y 2 * 2 -y 1 * 2 ≤ π 1 ≤ y 2 * 1 - ŷ y 2 * 1 -y 1 * 1 (3)
which is feasible when the corresponding interval is non empty (it is included in (0, 1) in any case). From the study of y * , we already have y 2 * 2ŷ < y 2 * 1ŷ. We now show that y * (α, θ) has increasing differences when v ′′′ ≤ 0 to complete the proof. We can differentiate twice the first-order condition because it holds for any α, θ, so that:

∂ 2 y * ∂α∂θ v ′′ (y * ) = - 1 α 2 - ∂y * ∂α ∂y * ∂θ v ′′′ (y * )
The second term of the RHS is negative when v ′′′ ≤ 0; also, v ′′ < 0 given strict concavity. Thus necessarily ∂ 2 y * ∂α∂θ ≥ 0, which means y * has increasing differences, implying y 2 * 2y 1 * 2 ≥ y 2 * 1y 1 * 1 , so we can conclude that there always exists a π 1 satisfying (3).

The corollary is illustrated in figure 1. We know from Lewis and Sappington (1989, Section 4) that a principal may want to create countervailing incentives. When the principal is informed, it may be strictly desirable for her to preserve Bayesian countervailing incentives (by keeping private her information as long as possible), or even attempt to manipulate beliefs to create them.

Figure 1 :

 1 Figure 1: A setting satisfying the conditions of corollary 1


Raphaël Bailly
email: raphael.bailly@lif.univ-mrs.fr

Amaury Habrard
email: amaury.habrard@lif.univ-mrs.fr

Franc ¸ois Denis
email: francois.denis@lif.univ-mrs.fr

A Spectral Approach for Probabilistic Grammatical Inference on Trees ⋆

We focus on the estimation of a probability distribution over a set of trees. We consider here the class of distributions computed by weighted automata -a strict generalization of probabilistic tree automata. This class of distributions (called rational distributions, or rational stochastic tree languages -RSTL) has an algebraic characterization: All the residuals (conditional) of such distributions lie in a finite-dimensional vector subspace. We propose a methodology based on Principal Components Analysis to identify this vector subspace. We provide an algorithm that computes an estimate of the target residuals vector subspace and builds a model which computes an estimate of the target distribution.

Introduction

In this article, we focus on the problem of learning probability distributions over trees. This problem is motivated by the high need in XML applications or natural language processing to represent large tree sets by probabilistic models. From a machine learning standpoint, this problem can be formulated as follows. Given a sample of trees independently drawn according to an unknown distribution p, a classical problem is to infer an estimate of p in some class of probabilistic models [START_REF] Carrasco | Stochastic inference of regular tree languages[END_REF]. This is a classical problem in grammatical inference and the objective here is to find a good estimate of the model's parameters. A usual class of models is the class of probabilistic tree automata (PTA) where the parameters lie in [0, 1].

Recent approaches propose using a larger class of representation: the class of rational distributions (also called rational stochastic tree languages, or RSTL) that can be computed by weighted tree automata -with parameters in R, hence with weights that can be negative and without any per state normalisation condition. This class has two interesting properties: It has a high level of expressiveness since it strictly includes the class of PTA and it admits a canonical form with a minimal number of parameters (see [START_REF] Denis | On rational stochastic languages[END_REF] for an illustration in the string case). It has notably the characterization that the residuals of a rational distribution (a special kind of conditional distributions) lie in a finite-dimensional subspace. This set of residuals spans a vector subspace W of the vector space of real values functions over trees. W is finite dimensional and its dimension corresponds to the minimal number of states needed by a weighted tree automaton to compute p. Thus, a goal of an inference algorithm might be to identify this subspace W . This was illustrated by the algorithm DEES [START_REF] Denis | Learning rational stochastic tree languages[END_REF][START_REF] Denis | Relevant representations for the inference of rational stochastic tree languages[END_REF] which builds iteratively a weighted automaton computing an estimate of p. However, the iterative approach presented before suffers from the drawback to rely on statistical tests that are done on fewer and fewer examples when the structure grows.

In order to overcome this drawback, in this paper we investigate the possibility of using Principal Component Analysis (PCA) to identify the target vector subspace spanned by the residuals of a rational distribution, and then to build a representation from this subspace. PCA has already been used in grammatical inference for learning rational string distributions in [START_REF] Bailly | Grammatical inference as a principal component analysis problem[END_REF], and in another framework in [START_REF] Clark | Languages as hyperplanes: grammatical inference with string kernels[END_REF]. Another spectral approach was proposed in [START_REF] Hsu | A spectral algorithm for learning hidden markov models[END_REF][START_REF] Hsu | A spectral algorithm for learning hidden markov models[END_REF] for learning a class of Hidden Markov Models (HMM) over sequences. In this paper, we show that considering the class of rational distributions offers a natural framework for applying PCA to identify the target residuals subspace. Moreover, we obtain a high gain of expressiveness since we are able to infer classes of distributions that can not be computed by PTA. This gain in expressiveness has unfortunately two main drawbacks: the class of rational distributions is not recursively enumerable and it is not decidable if a rational series defines a distribution [START_REF] Denis | Learning classes of probabilistic automata[END_REF]. In spite of these strong constraints, we give some asymptotic error bounds and provide pointwise convergence result.

The paper is organized as follows. Section 2 gives the preliminaries on trees and rational tree series. Section 3 is devoted to our algorithm, while the convergence properties are presented in Section 4. Some experiments are provided in the last section.

Preliminaries

In this section, we introduce the objects that will be used all along in the paper. We mainly follow notations and definitions from [START_REF] Comon | Tree automata techniques and applications[END_REF] about trees. Formal power tree series have been introduced in [START_REF] Berstel | Recognizable formal power series on trees[END_REF] where the main results appear. Some notations about norms and matrices terminate this section.

Trees and Contexts

Unranked Trees Let F be an unranked alphabet. The set of unranked trees over F is the smallest set T F satisfying F ⊆ T F , and for any f ∈ F , and

t 1 , . . . , t m ∈ T F , f (t 1 , . . . , t m ) ∈ T F .
Any tree defined over an unranked alphabet F can be represented over a ranked alphabet F @ = F @ 2 ∪ F @ 0 with only one binary symbol @, i.e. F @ 2 = {@(•, •)} where @ ∈ F and constants that comprise all symbols in F : F @ 0 = F . Figure 1(d) shows such a representation (called curryfication) of the tree of Figure 1(c). Curryfication can be formally defined by induction:

-curry(f (t 1 , . . . , t n )) = @(curry(f (t 1 , . . . , t n-1 )), curry(t n )) -curry(f (t)) = @(f, curry(t)) -curry(a) = a for a ∈ F
This particular class of ranked alphabet is in bijection with the set of unranked trees [START_REF] Comon | Tree automata techniques and applications[END_REF], i.e. labeled trees in which any node may have an unbounded number of children. Weighted automata on unranked trees are defined in [START_REF] Högberg | Bisimulation minimisation of weighted automata on unranked trees[END_REF], where it is proved that weighted unranked tree automata on F are equivalent to weighted tree automata on F @ . As ranked trees are a particular case of unranked trees and weighted ranked tree automata can be seen as a particular case of weighted unranked tree automata, the results still hold for any ranked alphabet.

Hence, without loss of generality, and in all the rest of the paper, we will only consider a ranked alphabet equipped with constant symbols and with only one binary symbol in the following of the paper. For convenience, we will use f for denoting the binary symbol instead of @. Contexts Contexts are element c of C F ⊂ T F ∪{$} where $ is a variable that appears exactly once as a leaf in c ($ is a constant and $ ∈ F ). Given a context c ∈ C F and a tree t ∈ T F , one can build a tree c[t] ∈ T F by replacing the (unique) occurrence of $ in c by the tree t. Definition 1. The length of a tree or a context is the number of functional symbols used to define it, including the special symbol $.

T k F (resp. T ≥k F ) will denote the set of trees of length k (resp. length greater or equal than k).

Tree Series

A (formal power) tree series on T F is a mapping r : T F → R. The vector space of all tree series on T F is denoted by R[T F ]. We denote by ℓ 2 (T F ) the vector subspace of R[T F ] of tree series r such that t∈T F r(t) 2 < ∞. This vector subspace is equipped with a dot product (r, s) = t∈T F r(t)s(t).

Given

r ∈ R[T F ], a residual of r is a series s ∈ R[T F ] such that s(t) = r(c[t]
) for some c ∈ C F . This series is denoted ċr : t → r(c[t]). One defines the set of residuals of r by { ċr|c ∈ C F }. Let c i be an enumeration of C F , and t j an enumeration of T F . Given r ∈ R[T F ], one defines the (infinite) observation matrix X of r by: (X) i,j = r(c i [t j ]).

      r(c 1 [t 1 ]) . . . r(c 1 [t j ]) . . . . . . . . . r(c i [t 1 ]) . . . r(c i [t j ]) . . . . . . . . .      
Rational series (series computed by weighted automata) are the series with a finite rank observation matrix. The rank of the observation matrix is the rank of the rational series (i.e. the state number of a minimal automaton computing the series). From this observation, one can define a canonical linear representation of a rational tree series as introduced in [START_REF] Denis | Learning rational stochastic tree languages[END_REF]. We give here a simpler definition: Definition 2. The linear representation of a rational tree series over T F is given by:

the rank d of the series, and {q 1 , . . .

q d } a basis of R d . -τ ∈ R d . -for each a ∈ F 0 , a vector a ∈ R d . -for f ∈ F 2 , a bilinear mapping f ∈ L(R d , R d ; R d ).
The linear representation is denoted by (F, d, , τ ).

The mapping can be inductively extended to a mapping :

T F → R d that satisfies f (t 1 , t 2 ) = f (t 1 , t 2 ) for any t 1 , t 2 ∈ T F .
Finally, the value of r(t) is given by: r(t) = t ⊤ τ where ⊤ denotes the transpose operator.

Example 2. Let F = {a, f (•, •)} a ranked alphabet, consider the linear representation (F, 2, , τ ) of the series r such that {e 1 , e 2 } is a basis of R 2 , τ = (1, 0) and defined by the following expressions:

a = 2e1 3 + e2 3 , f (e 1 , e 2 ) = e1 3 + 2e2 3 , f (e i , e j ) = 0 f or (i, j) = (1, 2).
One has:

r(f (a, a)) = f (a, a) ⊤ τ = f (a, a) ⊤ τ = f ( 2e1 3 + e2 3 , 2e1 3 + e2 3 ) ⊤ τ = ( 2 3 2 3 f (e1, e1) + 2 3 1 3 f (e1, e2) + 1 3 2 3 f (e2, e1) + 1 3 1 3 f (e2, e2)) ⊤ τ = ( 2 3 2 3 (0, 0) + 2 3 1 3 ( 1 3 , 2 3 ) + 1 3 2 3 (0, 0) + 1 3 1 3 (0, 0)) • τ = 2 3 3 .
Rational tree series can be equivalently represented by weighted tree automata where the number of states of the automata corresponds to the dimension of the linear representations. Indeed, a tree automaton is a tuple (Q, F, τ, δ) where Q, τ and δ are respectively the set of states, the terminal vector and the transition function. Let (F, 2, , τ ) be a linear representation and let (e 1 , . . . , e d ) be a basis of R d . Let Q = {e 1 , . . . , e d }. Any linear relation of the form f (e i , e j ) = k α k i,j e k yields to d transition rules of the form f (e i , e j )

α k i,j
--→ e k and τ (e i ) is set to τ ⊤ e i . See [START_REF] Denis | Relevant representations for the inference of rational stochastic tree languages[END_REF][START_REF] Borchardt | The Theory of Recognizable Tree Series[END_REF] for more details.

Example 3. A weighted automaton computing the series in example 2 would be Q = {q 1 , q 2 }, F = {a, f }, δ defined by: a

2/3 --→ q 1 , a 1/3 --→ q 2 , f (q 1 , q 2 ) 1/3 --→ q 1 , f (q 1 , q 2 ) 2/3 --→ q 2 and τ (q 1 ) = 1, τ (q 2 ) = 0. Let R[C F ] be the set of mappings s : C F → R. For r ∈ R[T F ] and t ∈ T F , one can define tr ∈ R[C F ] by: tr(c) = ċr(t) = r(c[t]).
The ċr correspond to the rows of the observation matrix X and the tr to its columns, and one has the following equivalent properties:

1. r ∈ R[T F ] has an observation matrix X with finite rank d.

The vector subspace of R[T

F ] spanned by { ċr|c ∈ C F } has dimension d. 3. The vector subspace of R[C F ] spanned by { tr|t ∈ T F } has dimension d.
Let us denote by C n the set of contexts of length lower than n, and let ∼ Cn be the equivalence relation over

R[C F ] defined by f ∼ Cn g iff ∀c ∈ C n , f (c) = g(c). One defines R[C n ] as the quotient vector space R[C F ]/ ∼ Cn , equipped with the regular dot product (f, g) = c∈Cn f (c)g(c).

Rational Distribution and Strong Consistency

Definition 3. A rational distribution (or rational stochastic tree language, RSTL) over T F is a rational series computing a probability distribution.

In other words, a RSTL is a probability distribution that can be computed by a weighted automaton (or that admits a linear representation). It can be shown that there exists some rational distributions that cannot be computed by any probabilistic tree automaton. It is undecidable to know whether a rational series given by a linear representation defines a probability distribution (see [START_REF] Denis | On rational stochastic languages[END_REF] for an illustration in the string case). Definition 4. A strongly consistent stochastic tree languages (or strongly consistent distribution) over T F is a probability distribution over T F having a bounded average tree size i.e. t∈T F p(t)|t| < ∞.

It can be shown (see [START_REF] Denis | Relevant representations for the inference of rational stochastic tree languages[END_REF]) that, if p is a rational distribution having a bounded average tree size, there exists some constants 0 < C and 0 < ρ < 1 such that:

t∈T ≥k F p(t) ≤ Cρ k .

Principle of the Algorithm

Let p be rational distribution on T F (strongly consistent or not). We give first a general algorithm that takes a sample i.i.d. according to p as input. For this purpose, let C n be the set of contexts of length lower that n and let us make the assumption that { ċp|c ∈ C n } and { ċp|c ∈ C F } span the same vector subspace of R[T F ]. In other words, we suppose that considering the set C n is sufficient to get the whole space of residuals.

Let V be the finite dimensional subspace of ℓ 2 (T F ) spanned by the set { ċp|c ∈ C n }. V * will denote the set { tp| Cn , t ∈ T F } ⊂ R[C n ] -for convenience we still denote by tp the mapping tp| Cn . Π V denotes the orthogonal projection over V relatively to the dot product inherited from ℓ 2 (T F ), and Π V * denotes the orthogonal projection over V * relatively to the dot product inherited from R[C n ]. Let S be a sample of N trees independently and identically drawn according to p and let p S be the empirical distribution on T F defined from S. V S denotes the vector subspace of ℓ 2 (T F ) spanned by { ċp S |c ∈ C n }, and V * S the subspace of R[C n ] spanned by { tp S |t ∈ T F }. We first build from S an estimate V * S,d of V * and then we show that V * S,d can be used to build a linear representation such that its associated rational series approximate the target p. In this section, we implicitly suppose that the dimension d of V * is known. We will show in the next section how it can be estimated from the data.

Estimating the Target Space

Let d > 0 be an integer. The first step consists in finding the d-dimensional vector subspace V * S,d of V * S that minimizes the distance to { tp S |t ∈ T F }:

V * S,d = arg min dim(W * )=d,W * ⊆V * S t∈T F tp S -Π W * ( tp S ) 2 .
V * S,d can be computed using principal component analysis.

Let {t j } be an enumeration of T F , and {c i } be an enumeration of C n . Let X S the empirical mean matrix defined by: (X S ) i,j = p S (c i [t j ]), and let X be the expectation matrix defined by: (X) i,j = p(c i [t j ]).

V * S,d corresponds to the vector subspace spanned by the d first (normalized) eigenvectors (corresponding to d largest eigenvalues) of the matrix M S = X S X ⊤ S . N S will denote the matrix X ⊤ S X S which corresponds to the dual problem of the PCA. We will denote by W * = {w * 1 , . . . , w * d } the set of eigenvectors (ordered by decreasing eigenvalues) of M S , and by W = {w 1 , . . . , w d } the corresponding eigenvectors of N S . W * is the matrix with the vectors {w * 1 , . . . , w * d } as columns -this matrix corresponds to the projection operator Π V * , while W is the matrix with vectors {w 1 , . . . , w d } as columns, because both W and W * are orthonormal.

Let λ 1 , . . . , λ d be the associated singular values; they also are the square roots of the eigenvalues of M S .

We recall here the relationships between the w i and w * i eigenvectors:

X S w i = λ i w * i and X ⊤ S w * i = λ i w i .
In particular,

M S w * i = X S X ⊤ S w * i = λ i X S w i = λ 2 i w * i .

Building the Linear Representation From the Dual Space

The eigenvectors found in the previous section form the basis of the residual space.

In order to complete the linear representation, we now need to define, in the basis {w * 1 , . . . , w * d }, the terminal vector τ , the mapping for the constant symbols a and the bi-linear operator f .

The idea is to identify, for any tree t, the mapping tp S to its projection on the space spanned by W * , that is W * W * ⊤ tp S . We shall see in next section that this identification leads to a bounded error, decreasing as the size of the sample grows.

-The vector space is the space spanned by W * .

-For each a ∈ F 0 , a = W * ⊤ āp S .

In order to define f , we use a known relation between eigenvectors of the standard and dual PCA:

w * i = k (wi) k
λi tk p S . We use the bilinearity of f to obtain:

-f (w * i , w * j ) = 1≤k,l≤d (wi) k (wj ) l λiλj W * ⊤ f (t k , t l )p S . -τ i = w * i ($)
, corresponding to the terminal weight of a tree in a bottom-up process. -Finally, r(t) = t ⊤ .τ .

The different steps of the algorithm are described in Algorithm 1.

Consistency

Let us consider the observation matrix X defined by: X ij = p(c i [t j ]), where c i and t j are respectively contexts and trees. Let S be a sample of size N i.i.d. from p. X S is defined as the empirical observation matrix built from the empirical distribution p S . In Data: A sample S of trees in TF i.i.d. according to a distribution p, a dimension d and a set of contexts Cn. Result: A linear representation A of a tree series (F, d, , τ ). Let X the matrix defined by X[i, j] = pS(ci[tj]); M = XX ⊤ /* variance-covariance matrix */; (λi, w * i , wi) ← square roots of eigenvalues of M in decreasing order and corresponding eigenvectors, and eigenvectors in the dual; Let w * 1 , . . . , w * d be the eigenvectors corresponding to the d largest eigenvalues and let W * = [w * 1 , . . . , w * d ] be the matrix having the vectors w * i as columns Let be the operator defined by:

foreach f ∈ F do if a ∈ F0 then a = W * ⊤ āpS; if f ∈ F2 then f (w * i , w * j ) = P 1≤k,l≤d (w i ) k (w j ) l λ i λ j W * ⊤ f (t k , t l )pS; end (τ )i = w * i ($) ; return A = (F, d, , τ );
Algorithm 1: Building a linear representation corresponding to a sample S and a dimension d.

this section, we will bound the difference between those two matrices, and show how it induces a bound for the convergence of the singular values and on the distance between the estimate and the target distribution for a tree t.

First, here is a simple result straightforward from the properties of empirical mean:

Lemma 1. Let p a probability distribution over T F , and p S its empirical estimate from a sample of size N drawn i.i.d. from p, one has

E( p S -p 2 2 ) = t∈T F E((p S (t) -p(t)) 2 ) = t∈T F p(t)(p(t) -1) N ≤ 1 N .
Let C n be the set of contexts of length lower or equal than n, it can easily be shown that:

Lemma 2. Let t be a tree. There is at most n contexts in C n such that t = c[t ′ ] for some tree t ′ .

The previous lemma helps us to bound the occurrence number of a tree in the matrix X, which will allow us to use some concentration inequality to bound the error over X. One denotes F as the Frobenius norm on matrices, and ∆ X = X -X S F . Lemma 3. Let X be a probability observation matrix restricted to the contexts belonging to C n . Let X S the empirical estimator of X from a sample S of size N . Then, one has with probability at least 1 -δ (δ > 0):

∆ X = X -X S F ≤ n N 1 + log( 1 δ ) .
Proof. This proof uses a construction similar to the proof of Proposition 19 in [START_REF] Hsu | A spectral algorithm for learning hidden markov models[END_REF]. Let z be a discrete random variable that takes values in T F . Let X be a probability observation matrix built from a set C n of contexts as lines and trees from T F as columns. One estimates X from N i.i.d. copies of z i of z (i = 1, . . . , N ). One associates to each variable z i a matrix X i indexed by contexts of C n and trees of T F such that

X i [j, k] = 1 if z i = c j [t k
] and 0 otherwise.

From Lemma 2, X i has at most n non null entries. The empirical estimate of X is

X S = 1 N N i=1 X i . Our objective is to bound X S -X F .
Let S ′ be a sample that differs from S on at most one example z ′ k . Then,

| X S -X F -X S ′ -X F | ≤ X S -X S ′ F ≤ 2n N .
From McDiarmid inequality [START_REF] Mcdiarmid | On the method of bounded differences[END_REF], one obtains:

P r( X S -X F ≥ E( X S -X F ) + ǫ) ≤ e -N n ǫ 2 .
By Lemma 1 and Lemma 2 and by using Jensen's inequality, it can be proved that E( X S -X F ) ≤ n N . By fixing δ = e -N n ǫ 2 , one gets the result. ⊓ ⊔

Singular Values Convergence

We use the previous result to show how one can assess the correct dimension of the target space. We will first recall some known result. Given an observation matrix X of rank d in the target space, and given its empirical estimate X S , we can rewrite X S as a sum X + E where E models the sampling error. We have the following result from [START_REF] Stewart | Matrix Perturbation Theory[END_REF]. Lemma 4. (Theorem 4.11 in [15]). Let X ∈ R m×n with m ≥ n, and let X S = X + E.

If the singular values of X and X S are (λ 1 > . . . > λ n ) and (λ S,1 > . . . > λ S,n ) respectively, then |λ S,i -

λ i | ≤ E 2 , i = 1, . . . , n.
Applied to our situation, this provides a valid way to assess the target dimension: let d be the rank of the target rational series, X S be the observation matrix deduced from a sample S, |S| = N . Proof. The condition N > 4n

λ 2 d (1 + log( 1 δ )) 2 implies that s < λ d 2
, thus the corresponding singular value λ S,d from X S satisfies λ S,d > 2s -X -X S 2 . This quantity is greater than s with probability at least 1 -δ.

⊓ ⊔

Bounds for the Estimation Error

We suppose here that the correct dimension has been found. We will not provide exact bounds, but only asymptotic bounds, and we will often use the equivalence between norms of vectors and matrices -since the vector spaces considered are finitedimensional. Let us first introduce some notations corresponding to errors over the objects handled by our algorithm:

-∆ x = max x-x S 2 with x (resp. x S ) a row or a column of the observation matrix X (resp. X S ). -∆ v = max w -w S 2 with w (resp. w S ) a left singular vector of the observation matrix X (resp. X S ). -∆ λ = max λ -λ S 2 with λ (resp. λ S ) a singular value of the observation matrix X (resp. X S ).

-∆ Π = W W T -W S W T
S F with W (resp. W S ) the d first singular vectors of the observation matrix X (resp. X S ).

Lemma 5. ∆ λ < ∆ X and ∆ x < ∆ X .
Proof. Straightforward from Lemma 4 and the norm relation 2 ≤ F for the first inequality, and the definition of ∆ x and ∆ X for the second.

⊓ ⊔

The following corollary gives an asymptotic bound on the error of the covariance matrix used to compute the eigenvectors.

Corollary 1. ∆ M = M -M S F . One has M -M S F ≤ O(∆ X ) Proof. One has M -M S F ≤ XX ⊤ -XX ⊤ S + XX ⊤ S -X S X ⊤ S F ≤ ( X F + X ⊤ S F )∆ X . Thus: ∆ M ≤ ∆ X (2 X F + ∆ X ).

⊓ ⊔

In order to provide asymptotic bounds on the other errors, we need to introduce some known results about eigenvectors and PCA from [START_REF] Zwald | On the convergence of eigenspaces in kernel principal component analysis[END_REF]. Let A be a symmetric positive Hilbert-Schmidt operator with positive eigenvalues 1

λ 2 1 > • • • > λ 2 d > 0. δ r = 1 2 (λ 2 r -λ 2 r+1
), and let δr = inf(δ r , δ r-1 ). Let B be a symmetric positive Hilbert-Schmidt operator such that B F < δr /2 and that B F < δ d /2. The results from [START_REF] Zwald | On the convergence of eigenspaces in kernel principal component analysis[END_REF] provide error bounds on projection operators and eigenvectors. In our framework, A corresponds to the covariance matrix M and A + B to the empirical one M S . Let W (resp. W S ) be the matrix of the d first eigenvectors of A (resp. A + B ), and w r (resp. w S,r ) the corresponding r-th eigenvector, we have the following results. Lemma 6. (Theorem 2 -remark of [START_REF] Zwald | On the convergence of eigenspaces in kernel principal component analysis[END_REF]

) w r -w S,r 2 ≤ 2 B F δr . Theorem 3. (Theorem 3 of [16]) W W ⊤ -W S W ⊤ S F ≤ B F δ d .
We are now able to provide asymptotic bounds for the two remaining errors.

Lemma 7. One has ∆ v = O(∆ X ) and ∆ Π = O(∆ X ).
Proof. By using respectively Lemma 6 and Theorem 3, and from Corollary 1, one has

∆ v ≤ 4∆ X (2 X F + ∆ X ) δd = O(∆ X ) and ∆ Π ≤ ∆ X (2 X F + ∆ X ) δd = O(∆ X ).

⊓ ⊔

Let us denote λ = inf 1...d λ i . We will now study some errors on the parameters of the linear representation built by our algorithm. Let p = (F, d, , τ ) be the target linear representation equipped with the basis {w * 1 , . . . , w * d } and let r S = (F, d, S , τ S ) the linear representation equipped with the basis {w * S,1 , . . . , w * S,d } found by our algorithm from a sample S. Let us define the following error bounds on the coefficients:

-∆ τ = sup i (τ -τ S ) i , -∆ a = sup i (a -a S ) i for a ∈ F 0 , -∆ f = sup i,j,k (f (w * j , w * k ) -f S (w * S,j , w * S,k )) i .
One can check the following lemma.

Lemma 8.

∆ τ ≤ dO(∆ v ) ≤ 2d X F O(∆ X ), ∆ a ≤ 2O(∆ X ), ∆ f ≤ O(∆ X ) λ w i w ⊤ j F (2 + 2 X F δ + 8 X F δ ( w i 1 + w j 1 )) .
All the mappings considered through the algorithm are continuous, thus the mapping deduced from the algorithm converges pointwisely towards the target distribution as ∆ X tends to zero. We provide then the following result which gives a bound for the estimation error. Theorem 4. Let p be a rational distribution with rank d. Let n be the maximum length of a context used in the algorithm. If S is a sample i.i.d. from p, let r S be the mapping deduced from the algorithm. There exists C such that for any 0 < δ < 1, for any tree t of length k, if S is an i.i.d. sample of size N then, with confidence at least 1 -δ, one has:

|r s (t) -p(t)| < Ckd 2k n N log( 1 δ ).
Proof. Let us prove the statement by induction on k. Let us denote ∆ k a bound for the error made for the estimate the coefficient of t for a tree t of height k. One has:

∆ 1 = ∆ a = O(∆ X ) Using f (u, v) = 1≤i≤d 1≤i≤d (u) i (v) j f (w * i , w * j ), with |u| + |v| = k -1, one has: ∆ k = O(d 2 (|u|d 2|u| ∆ X +|v|d 2|v| ∆ X +∆ X )) ≤ O(kd 2k ∆ X ) = O(kd 2k n N log( 1 δ )).
Then, since p(t) = t T .τ and r s (t) = t T S .τ S , one has the conclusion. ⊓ ⊔

Strongly Convergent Case

In the case of strongly consistent distribution, one does not need to consider a finite set of contexts to perform the algorithm: one has, with confidence greater than 1 -δ, |t| < log(2C/δ) log(1/ρ) . One can provide a bound result for this special case.

Theorem 5. Let p be a strongly consistent rational distribution with rank d. Let S be a sample i.i.d. from p, let r S be the mapping deduced from the algorithm. There exists C such that for any 0 < δ < 1, for any tree t of length k, if S is an i.i.d. sample of size N then, with confidence at least 1 -δ, one has:

|r s (t) -p(t)| < Ckd 2k log(N ) N log 2 ( 1 δ ).
Proof. One bounds the length of a tree drawn by p: with a confidence greater than

1 -δ/2N , |t| < log(2N C/δ) log(1/ρ) .
Thus, with confidence 1 -δ/2, S has only trees of length lower than log(2N C/δ) log(1/ρ) . By replacing n in the previous result, one obtains the conclusion.

⊓ ⊔

Illustration

In order to illustrate the algorithm, we consider the distribution p defined by tree series of Example 2.

To study the behavior our algorithm, we consider an observation matrix 5 × 5, built on the set of trees T = {t 1 , t 2 , t 3 , t 4 , t 5 }, where t 1 = a, t 2 = f (a, a), t 3 = f (a, f (a, a)), t 4 = f (f (a, a), a), t 5 = f (f (a, a), f (a, a)) and the set of contexts

C = {$, f (a, $), f ($, a), f (f (a, a), $), f ($, f (a, a))}.
We generate i.i.d. samples from p of different sizes containing respectively 10 3 , 10 4 , 10 5 and 10 6 trees. On Figure 2, we show the different eigenvalues (square of singular values) in decreasing order obtained from the different samples.

We can observe that the convergence of the computed series towards the target value, and the convergence of singular values, is closely O( 1 √ |S| ) (in average values).

We also compared the average standard deviation of the probabilities of the trees in T obtained with our model, with rank 2 learned from the different learning samples, with the theoretical standard deviation of the classical probability (binomial) estimator. We have that p(t 1 ) ≃ 0.6666, p(t 2 ) ≃ 0.0741, p(t 3 ) ≃ 0.0329, p(t 4 ) ≃ 0.0082 and p(t 5 ) ≃ 0.0037. The values obtained are shown on Figure 3. The estimated standard deviation is, in majority (21/25), lower than the theoretical standard deviation of a the binomial estimator: The algorithm seems to work better than the simple frequency estimator for the task of density estimation. Now, we consider the problem of the dimension estimate. The second singular value λ 2 ∼ 7.74 • 10 -3 . Using the bound ∆ X to estimate the correct dimension (Theorem 2), we can estimate that:

-For N = 10 6 , the rank 2 is found with a parameter δ ∼ 0.59 (confidence 0.41).

-For N = 2 • 10 6 , the rank 2 is found with a parameter δ ∼ 0.12 (confidence 0.88).

-For N = 3 • 10 6 , the rank 2 is found with a parameter δ ∼ 0.02 (confidence 0.98).

Conclusion and Discussion

We have studied the problem of learning an unknown distribution p from finite independently and identically drawn samples. We have proposed a new approach for identifying rational distributions on trees, or rational stochastic tree languages. Most classical inference algorithms in probabilistic grammatical inference build an automaton or a grammar iteratively from a sample S. Starting from an automaton composed of only one state, then they have to decide whether a new state must be added to the structure. This iterative decision relies on a statistical test with a known drawback: as the structure grows, the test relies on fewer and fewer examples. Instead of this iterative approach, we tackle the problem globally and our algorithm computes in one step the space needed to build the output automaton. That is, we have reduced the problem set in the classical probabilistic grammatical inference framework to a classical optimization problem. This point offers the interesting opportunity to apply classical results in statistical machine learning theory to probabilistic grammatical inference.

We have provided three types of results. First, we have given a result for convergence of eigenvalues which can be used for the estimation of the dimension of the target vector space, which is a crucial point in probabilistic grammatical inference and may allow to avoid costly cross-validation procedures. Second, we have provided error bounds for the convergence of the parameters of a linear representation. We have finally obtained pointwise convergence results for the probability estimate of a tree.

One perspective would then to obtain an ℓ 1 -convergence, probably restricted to the case of strongly consistent stochastic tree languages, and to obtain tighter bounds. We finally need to experimentally study and compare our approach to existing ones on real data, this is a work in progress. Another perspective would consist in introducing non linearity via the kernel PCA technique developed in [START_REF] Shawe-Taylor | On the concentration of spectral properties[END_REF] and by the Hilbert space embedding of distributions proposed in [START_REF] Smola | A hilbert space embedding for distributions[END_REF][START_REF] Song | Hilbert space embeddings of Hidden Markov Models[END_REF].

Example 1 .

 1 Let F 0 = {a, b}, F 1 = {g(•)} and F 2 = {f (•, •)}. Then t = f (a, g(b)) ∈ T F (Figure 1(a)), c = f (a, $) ∈ C F (Figure 1(b)) and c[t] = f (f (a, g(b)), a) (Figure 1(c)).

Fig. 1 .

 1 Fig.1. An example of tree t = f (a, g(b)), context c = f ($, a) and their composition c[t] = f (f (a, g(b)), a), as defined in Example 1. On the right a representation of t over an alphabet with only one binary symbol @ and with the elements of F seen as constant symbols.

Theorem 1 .Theorem 2 .

 12 Let Λ be the set of singular values of X S . Let Λ s be the subset of singular values of X S greater than s. For a given confidence parameter δ, letd ′ = |Λ s | for s = n N (1 + log( 1 δ )). With probability greater than 1 -δ, one has d ≥ d ′ .Proof. Straightforward from Lemma 3 and Lemma 4: with probability greater than 1δ, the singular values in Λ s match non-zeros singular values from the target observation matrix X. ⊓ ⊔ Let λ d the smallest non-zero eigenvalue of X. Let Λ be the set of singular values of X S . Let Λ s be the subset of singular values of X S greater than s. For a given confidence parameter δ, let d ′ = |Λ s | for s = probability greater than 1 -δ, one has d = d ′ .

Fig. 2 .

 2 Fig. 2. Curves of eigenvaluessquare of singular values -(in logarithmic scale) for sample size of 10 3 , 10 4 , 10 5 and 10 6 trees (the lightest to the darkest).

t σ 10 2 σ 10 3 σ 10 4 σ 10 5 σ 10 6 t 1 3 . 5 Fig. 3 .

 6353 Fig. 3. Average standard deviation of trees in T measured from the 2-dimensional model learned on samples of size 10 2 , 10 3 , 10 4 , 10 5 and 10 6 . The standard deviation of the theoretical binomial estimator is indicated in italics.

Recall that according to our notation the λi denote singular values.

Acknowledgement

The authors wish to thank the anonymous reviewers for their comments and suggestions.

⋆ This work was partially supported by the ANR LAMPADA ANR-09-EMER-007 project and by the IST Programme of the European Community, under the PASCAL2 Network of Excellence, IST-2007-216886. This publication only reflects the authors' views.

Ranked Trees Let F =