Hippolyte D'albis
email: hippolyte.d-albis@univ-tlse1.fr

Emmanuelle Augeraud-Véron

Competitive Growth in a Life-cycle Model: Existence and Dynamics *

Keywords: JEL Classification: D50, D90 Overlapping-generations Models, Endogenous Growth, Functionnal Differential Equations of Mixed Type

In this paper, the dynamic behavior of the capital growth rate is analyzed using an overlapping-generations model with continuous trading and finitely lived agents. Assuming a technology satisfying constant social returns to capital, the equilibrium growth rate is piecewisedefined by functional differential equations with both delayed and advanced terms. The main result concerns the existence of a solution expressed as a series of exponentials, which is shown to crucially depend on the initial wealth distribution among cohorts. Upon existence, the dynamics of the capital growth rate has a saddle-point trajectory that converges to a unique steady-state. Along the transition path, the growth rate exhibits exponentially decreasing oscillations.

Introduction

As they deal with population or social security issues, applied studies have come to use settings that encompass an arbitrarily large number of overlapping generations of individuals to ease the confrontation with real world annual data. Among them, Auerbach and Kotlikoff [START_REF] Auerbach | Dynamic Fiscal Policy[END_REF] notably propose a deterministic framework and Rios-Rull [START_REF] Rìos-Rull | Life-cycle economies and aggregate fluctuations[END_REF] a stochastic one. Such models are unfortunately at odds with the benchmark structure of the theoretical literature based upon two periods overlapping generations of individuals. Recent contributions, such as Kehoe et al. [START_REF] Kehoe | Gross substitutability in large-square economies[END_REF], have thus been aimed at developing a fruitful analysis of large-square models and exhibited specific conditions on the dynamic properties of equilibrium paths. However, a limit to their use on a wide spread basis springs from the difficulties that emerge in the course of their analytical resolution. Our paper introduces a simple method that allows for solving models with a continuum of overlapping generations.

It also proposes a complete resolution in a specific case which provides new insights on the existence and convergence properties of equilibrium paths.

In models with many overlapping generations, the dynamics of endogenous variables depends on a finite number of their past and future realizations. This dependency creates the analytical difficulty of these models. In discrete time frameworks, the analysis of the dynamics requires the study of polynomials, whose order increases with the number of generations, and which will not be tractable in the general cases. We claim that this difficulty may however be circumvented using the continuous time framework developed by Cass and Yaari [START_REF] Cass | Individual saving, aggregate capital accumulation, and efficient growth, in "Essays on the Theory of Optimal Economic Growth[END_REF]; in a nutshell, it eases the mathematical resolution without affecting the qualitative analysis. However, in continuous time frameworks, the dynamics is characterized by a functional differential equation of mixed type (MFDE), i.e. the dynamics is affected by distributed delays and advances. We propose an integrated treatment of such a structure in a simple competitive equilibrium case considering an overlapping generations model with production in which the technology exhibits constant social returns to capital. Capital dynamics is hence described by an MFDE that happens to be linear, and allows for a global dynamics analysis.

We prove that, upon existence of a competitive path, the growth rate has a transitional dynamics and converges to a steady state through exponentially decreasing oscillations. The oscillatory behavior is now a well known output of dynamic systems with delays. As pointed out by Boucekkine, de la Croix and Licandro [START_REF] Boucekkine | Vintage human capital, demographic trends and endogenous growth[END_REF], they appear as resulting from the vintage human capital structure. Recently, Demichelis and Polemarchakis [START_REF] Demichelis | Frequency of trade and the determinacy of equilibrium paths: logarithmic economies of overlapping generations under certainty[END_REF] and d'Albis and Augeraud-Véron [START_REF] Albis | Balanced cycles in an OLG model with a continuum of finitely-lived individuals[END_REF] have also been find a related class of fluctuations.

However, as for the existence problem, there is to our knowledge, few papers that have studied it. Rustichini [START_REF] Rustichini | Functional differential equations of mixed type: the linear autonomous case[END_REF] has claimed that MFDE may fail to have a solution while Burke [START_REF] Burke | Equilibrium for overlapping generations in continuous time[END_REF] has proven that the existence of an equilibrium is not always ensured in continuous-time overlapping generations models even under rather standard and simple assumptions.

Our main result is a theorem of existence of an intertemporal equilibrium.

We proceed by studying the existence of a strictly positive and bounded solution to the MFDE. We show how to use the piecewise definition of the capital dynamics to define the set of initial distributions of wealth that ensure the existence of the equilibrium. This result points out the importance of ini-tial conditions: not all initial wealth distributions among the generations are compatible with the existence of an equilibrium. Some distributions may indeed generate short run fluctuations whose magnitude is so large that they would lead to negative aggregate assets. As a consequence, when macroeconomic variables oscillate, initial distributions matter.

The paper is organized as follows. In section 2, we present the basic framework of the model and determine the functional equations that characterize the capital dynamics at the equilibrium. In section 3, we analyze the global dynamics and propose an existence theorem based on the initial wealth distribution. We conclude in section 4.

The model

This section develops an overlapping-generations model with continuous trading and finitely-lived individuals. Assuming a technology with constant returns to capital, it characterizes the capital dynamics of such an economy.

Individual behavior

Time is continuous and has a starting point denoted τ ∈ R; let t denotes the time index such that t ≥ τ . Individuals live for an interval of time of length ω > 0. In what follows, the lifespan is considered as finite although the limit case of an infinite horizon (ω → +∞) can be computed for discussion purposes. It is assumed that individuals only derive utility from consumption and that they have isoelastic preferences and a non negative time discount denoted ρ. Let c (s, t) ≥ 0 denotes the real consumption of an individual who born at time s as of time t. Hence, the intertemporal utility of an individual who born at time s ∈ (τω, t], denoted u (s, t), satisfies:

u (s, t) = Z s+ω t e -ρ(z-t) v (c (s, z)) dz, (1)
where:

v (c) = ¯c1-1 σ -1 1-1 σ if σ 6 = 1, ln c if σ = 1, (2)
with σ > 0 standing for the elasticity of intertemporal substitution. During a lifetime, the labor supply is fixed and equal to 1 and w (t) , an ageindependent labor income is received. Individuals have access to competitive capital markets that yield the risk-free interest rate r (t). Let a (s, t) denotes the real wealth of an individual who born at time s as of time t. The instantaneous budget constraint is therefore for all t ≥ τ :

∂a (s, t) ∂t = r (t) a (s, t) + w (t) -c (s, t) . (3)
Individuals enter the economy with no financial assets except those who are alive at the initial date of the economy and which are endowed with a given financial wealth. Therefore, initial conditions write:

a (s, τ ) given if s ∈ (τ -ω, τ ) , a (s, s) = 0 if s ≥ τ. (4)
Finally, individuals cannot die indebted. Hence, for s > τω, terminal constraints write:

a (s, s + ω) ≥ 0 if ω ¿ +∞, lim z→+∞ e -z s r(u)du a (s, s + z) ≥ 0 if ω → +∞. (5)
Moreover, assume that a (s, t) and c (s, t) are C 1 ((τω, ∞) × (τ, ∞)) while r (t) and w (t) are continuous for all t ∈ (τ, ∞).

The individual program is to maximize (1) subject to (3), ( 4) and ( 5). The optimal consumption behavior is given in the following lemma.

Lemma 1 For (s, t) ∈ (τω, τ ) × (τ, s + ω), the optimal consumption satisfies:

c (s, t) = a (s, τ ) e t τ r(u)du + R s+ω τ w (z) e t z r(u)du dz R s+ω τ e t z [(1-σ)r(u)+σρ]du dz , ( 6
)
and for (s, t) ∈ (τ, ∞) × [s, s + ω): c (s, t) = R s+ω s w (z) e t z r(u)du dz R s+ω s e t z [(1-σ)r(u)+σρ]du dz . ( 7
)
Proof. See the Appendix.

As initial conditions (4) differ, it is necessary to distinguish individuals who were alive at time t = τ , and whose behavior is affected by the initial wealth distribution, from those who born at time t > τ. Moreover, equations [START_REF] Boucekkine | Vintage human capital, demographic trends and endogenous growth[END_REF] and [START_REF] Boucekkine | del Río, Vintage capital and the dynamics of the AK model[END_REF] show that the optimal consumption is a function of the past and anticipated factor prices all along the individual's life-cycle. Hence, in the case of an infinite lifespan obtained as ω → +∞, it is the entire path of prices starting at the birth date that affects the individual behavior.

Aggregate dynamics

The demographic structure is in overlapping generations and the population is stable.

N (t) = Z t t-ω βN (s) ds. (8)
The demographic growth rate, denoted n, is obtained by replacing N (t) = N (s) e n(t-s) in ( 8) and then solving the following equation:

Z ω 0 βe -nα dα = 1. (9)
The macroeconomic counterpart of any individual variable is obtained by integrating it on the birth date over an interval of length ω. Hence, the aggregate assets per head, denoted a (t), satisfy for all t ≥ τ :

a (t) = Z t t-ω βe -n(t-s) a (s, t) ds. (10)
Differentiating [START_REF] Cass | Individual saving, aggregate capital accumulation, and efficient growth, in "Essays on the Theory of Optimal Economic Growth[END_REF] with respect to time and using the individual budget constraints yields the following standard dynamics:

da (t) dt = [r (t) -n] a (t) + w (t) -c (t) , (11)
where c (t) denotes the aggregate consumption per head such that:

c (t) = Z t t-ω βe -n(t-s) c (s, t) ds. (12)
Remark that for t ∈ (τ, τ + ω), the population is composed of cohorts born before and after t = τ , while for t ∈ (τ + ω, ∞), it only remains cohorts born after the initial date τ . Using ( 6) and [START_REF] Boucekkine | del Río, Vintage capital and the dynamics of the AK model[END_REF], c (t) hence writes:

c (t) = Z τ t-ω βe -n(t-s) a (s, τ ) e t τ r(u)du + R s+ω τ w (z) e t z r(u)du dz R s+ω τ e t z [(1-σ)r(u)+σρ]du dz ds + Z t τ βe -n(t-s) R s+ω s w (z) e t z r(u)du dz R s+ω s e t z [(1-σ)r(u)+σρ]du dz ds, (13)
if t ∈ (τ, τ + ω) and:

c (t) = Z t t-ω βe -n(t-s) R s+ω s w (z) e t z r(u)du dz R s+ω s e t z [(1-σ)r(u)+σρ]du dz ds, (14)
if t ≥ τ + ω. As a consequence, the aggregate dynamics is generically piecewise-defined. Exceptions are obviously the limit case of an infinite lifespan (ω → +∞) where only (13) holds, and the limit case of no initial date for the economy (τ → -∞) where it is only ( 14) that characterizes the dynamics. Remark that the piecewise definition, which will be crucial for the existence result presented section 3, is not considered in related works on continuous-time OLG model such as Demichelis and Polemarchakis [START_REF] Demichelis | Frequency of trade and the determinacy of equilibrium paths: logarithmic economies of overlapping generations under certainty[END_REF] and d'Albis and Augeraud-Véron [START_REF] Albis | Balanced cycles in an OLG model with a continuum of finitely-lived individuals[END_REF] who focus on the asymptotic dynamics.

Moreover, equations ( 13) and [START_REF] Hupkes | Center manifold theory for functional differential equations of mixed type[END_REF] show that the aggregate consumption at time t is determined by the path of factor prices on the interval (τ, t + ω) when t belongs to (τ, τ + ω) and on the interval (tω, t + ω) when t ≥ τ +ω.

It is important to notice that the dynamics of the aggregate consumption at time t is also affected the same path of prices, which implies that the dynamics is characterized by functional differential equations. Indeed, differentiating [START_REF] Diekmann | Delay Equations[END_REF] with respect to time yields:

dc (t) dt = [σ (r (t) -ρ) -n] c (t) + βc (t, t) -βe -nω c (t -ω, t) , (15)
where individual consumptions at the beginning and the end of the life-cycle are immediately computed with ( 6) and ( 7) such as:

c (t, t) = R t+ω t w (z) e t z r(u)du dz R t+ω t e t z [(1-σ)r(u)+σρ]du dz , (16)
and:

c (t -ω, t) = ¯a(t-
if t ≥ τ + ω. (17)
The dynamics of c (t) given by [START_REF] Kehoe | Gross substitutability in large-square economies[END_REF] decompose itself in three terms. The first one is the usual difference between the individual consumption growth rate, σ (r (t)ρ), and the population growth rate, n. Aggregate consumption per head is likely to increase when the first is greater than the second. The second term, which was initially pointed out by Blanchard [START_REF] Blanchard | Debt, deficits and finite horizons[END_REF] and Weil [START_REF] Ph | Overlapping families of infinitely-lived agents[END_REF], is the initial consumption of the newborns, c (t, t). They enter the economy at rate β and make the aggregate consumption increase. By considering individuals with infinite horizons, the two authors obtain a consumption of the newborns that reduces to a function of variables characterized for time t. This is not anymore true when ω is finite. Differentiating ( 16) with respect to t yields a dynamics that depends on future prices:

dc (t, t) dt = ( σ [r (t) -ρ] + 1 -e t t+ω [(1-σ)r(u)+σρ]du R t+ω t e t z [(1-σ)r(u)+σρ]du dz ) c (t, t) + w (t + ω) e t t+ω r(u)du -w (t) R t+ω t e t z [(1-σ)r(u)+σρ]du dz , (18)
while the infinite horizon case reduce to:

dc (t, t) dt ¯ω→+∞ = σ [r (t) -ρ] c (t, t) + c (t, t) -w (t) R +∞ t e t z [(1-σ)r(u)+σρ]du dz , (19)
provided that r (z) > max {0, σρ/ (σ -1)} for z ∈ [t, +∞). Remark notably with [START_REF] Mallet-Paret | Exponential dichotomies and Wiener-Hopf factorizations for mixed-type functional differential equations[END_REF], that the wage at time t + ω, w (t + ω) , affects the consumption of the newborn of date t, while it does not influence the consumption of individuals born before date t. Finite horizons thus create a discrete advance in the aggregate dynamics.

The third term that influence the dynamics of c (t) given by ( 15) corresponds to the last consumption, c (tω, t), of those who leave the economy at rate βe -nω . This term is obviously linked to the assumption of finite lifespan and therefore does not appear in Blanchard [START_REF] Blanchard | Debt, deficits and finite horizons[END_REF] and Weil [START_REF] Ph | Overlapping families of infinitely-lived agents[END_REF]. The level and the dynamics of c (tω, t) depend on past prices. Nevertheless, it is important to remark with ( 17) that c (tω, t) can be rewritten for t ∈ (τ, τ + ω) as a function of variables characterized at time t. After that initial interval of time, the dynamics is affected by delayed variables. Indeed, one has:

dc (t -ω, t) dt = ( σ [r (t) -ρ] + e t t-ω [(1-σ)r(u)+σρ]du -1 R t t-ω e t z [(1-σ)r(u)+σρ]du dz ) c (t -ω, t) + w (t) -w (t -ω) e t t-ω r(u)du R t t-ω e t z [(1-σ)r(u)+σρ]du dz , (20)
if t ≥ τ + ω. The wage at time tω, w (tω) , affects the consumption of individual born at date tω, while it does not influence the consumption of individuals born after date t. The finite lifespan assumption hence creates a discrete delay in the aggregate dynamics as of time t ≥ τ + ω. Conversely, before date τ + ω, their are only cohorts born before date τ that die, and consequently, there is no past value of factor prices that becomes unrelevant for the aggregate dynamics. To summarize, the aggregate dynamics is influenced by discrete advances for t ∈ (τ, τ + ω) and by both advances and delays for t ≥ τ + ω.

Intertemporal equilibrium

Assume there is a unique material good, whose price is normalized to 1. It can be used for consumption or for adding to the capital stock. This good is produced by many competitive firms whose aggregate activity is described by a production function with labor L (t) and capital K (t) as inputs. The aggregate production is supposed to satisfy:

Y (t) = A (K (t)) α (e (t) L (t)) 1-α , (21)
where e (t) is an externality from the producer perspective and where A > 0 and α ∈ (0, 1). The capital depreciation rate is constant and denoted δ ≥ 0.

For all t ≥ τ , factor prices equal marginal products:

r (t) + δ = αA (K (t)) α-1 (e (t) L (t)) 1-α , (22)
w (t) = (1 -α) A (K (t)) α (e (t)) 1-α (L (t)) -α . ( 23
)
Define the aggregate capital per head as k (t) = K (t) /L (t). To close the model, assume that the externality is of the "learning-by-doing" type and thus satisfies at the equilibrium the following equality: e (t) = k (t).

Definition An equilibrium with perfect foresight is a function k (t) , t ≥ τ, that belongs to R + , is continuous and has bounded variations on (τ, ∞) such that (i) individuals maximize their utility subject to the budget constraints, (ii) firms maximize their profits, (iii) markets clear, and (iv) e (t) = k (t).

It is useful to define the following variables which represent various cumula-tive functions of past capital stock:

y (t) = Z t τ k (z) e -r(z-t) dz, (24)
v (t) = β (1 -α) A R t τ k (z) R t z e -γ(t-s) e -r(z-s) dsdz R ω 0 e -µz dz , (25)
ṽ (t) = β (1 -α) A Z t τ k (z) Z t z e -γ(t-s) e -r(z-s) R s τ e -µ(u-s) du dsdz. ( 26
)
where r = αAδ, γ = nσ (rρ) and µ = rσ (rρ). Then, the equilibrium is characterized in the next Lemma.

Lemma 2

The equilibrium is the solution of the following system:

dv (t) dt = -γv (t) + β (1 -α) A R ω 0 e -µu du y (t) , (27)
dṽ (t) dt = -γṽ (t) + β (1 -α) A R τ -t 0 e -µu du y (t) , (28)
dy (t) dt = ry (t) + k (t) , (29)
dk (t) dt = (A -δ -n) k (t) -z (t
) + e -nω ṽ (t)e -σ(r-ρ)ω e -γ(t-τ ) ṽ (τ + ω)

+v (t) -e -rω v (t + ω) + e -rω e -γ(t-τ ) v (τ + ω) , (30)
if t ∈ (τ, τ + ω) and of ( 27), (29) and:

dk (t) dt = (A -δ -n) k (t)+ ¡ 1 + e -(γ+r)ω ¢ v (t)-e -γω v (t -ω)-e -rω v (t + ω) , (31)
if t ≥ τ + ω. In addition, z (t) , t ∈ (τ, τ + ω) and k (τ ) > 0 are given.

Proof. See the Appendix.

As explained previously, the dynamics is piecewise-defined: as the economy is supposed to have a finite starting point, a first system of equations characterizes the capital dynamics as long as it exists individuals who were alive at the initial date; this system includes, in variable z (t), the initial distribution of wealth among generations which constitutes the initial condition of the system. The capital dynamics, given by (30), is characterized by a differential equation with discrete advances. When all the individuals who were alive at the initial date, have died, a second system of equations then characterizes the dynamics for an infinite future. The capital dynamics, then

given by (31), is characterized by a differential of mixed type (MFDE): delays and advances influence the dynamics. It is easy to check that the two system are the same for t = τ + ω.

Moreover, the bounded life-span assumption is crucial: we show in the next corollary, that when ω goes to infinity, the dynamics is define by a single system of ordinary differential equations.

Corollary 1 Suppose that n > 0 and µ > 0. When ω → +∞, the equilibrium is the solution of ( 27), (29) and:

dk (t) dt = (A -δ -n) k (t) -nµe -γ(t-τ ) k (τ ) + v (t) , (32)
for all t ≥ τ and with k (τ ) > 0 given.

Proof. See the Appendix.

It is important to notice that the linearity of the dynamics crucially depends on the assumption of constant social returns to capital. The dynamics would be non linear by replacing the learning-by-doing assumption by, for instance, an exogenous labor augmenting technical progress such as: e (t) = e (τ ) e g(t-τ ) , with g > 0. Linearizing on the neighborhood of a steadystate would give some informations on the asymptotic behavior. Provided there is no Hopf bifurcations, the dynamics is likely to be qualitatively similar to the one that is to be studied in the next section. On such issue, we refer to the pioneer article proposed by Cass and Yaari [START_REF] Cass | Individual saving, aggregate capital accumulation, and efficient growth, in "Essays on the Theory of Optimal Economic Growth[END_REF] who study a non linear dynamics in the case of a specific neoclassical production function, and to recent developments by d'Albis and Augeraud-Véron [START_REF] Albis | Balanced cycles in an OLG model with a continuum of finitely-lived individuals[END_REF] and [START_REF] Albis | Endogenous retirement and monetary cycles[END_REF] and Hupkes and Verduyn Lunel [START_REF] Hupkes | Center manifold theory for functional differential equations of mixed type[END_REF]. However, when the dynamics is non linear, the existence problem studied below, that relies on the dynamics behavior on the initial interval of time is still an open question.

The results

This section proposes a complete study of the global dynamics. In a first part, we use rather standard tools to analyze the dynamic behavior after some initial interval of time. The results obtained are then similar to those proposed by recent related contributions. The real novelty of the paper lies in the second part of the section, where we propose an existence theorem that uses the dynamics during the initial interval of time.

Capital dynamics for t ≥ τ + ω

Let us first ignore the initial dynamics and study the capital behavior for t ≥ τ + ω. We proceed by analyzing the roots of the corresponding characteristic equation.

Lemma 3 Let r = αAδ. The characteristic equation associated to the system of equations ( 27), ( 29) and (31) has:

1) a unique real root denoted ḡ, that satisfies ḡ < σ (rρ);

2) no complex roots with real part that belong to the closed strip:

[min {rn, ḡ} , max {rn, ḡ}] , except ḡ itself;

3) an infinity of complex roots with real parts greater than max {rn, ḡ};

4) an infinity of complex roots with real parts lower than min {rn, ḡ}.

Proof. See the Appendix.

The interpretation of these results lies in two corollaries. Let us first define g (t) the growth rate of capital per head at time t such that:

k (t) = k (τ ) e t τ g(u)du . (33)
Then,

Corollary 2 A balanced growth path exists along which the growth rate of capital per head is lower than the individual's consumption growth rate.

Proof. See the Appendix.

As usual, the "learning-by-doing" assumption yields some endogenous growth.

It exists a steady-state for such a growth rate whose value corresponds to the unique real root of the characteristic function studied in Lemma 3. Corollary 2 hence claims that the steady-state value for the endogenous growth rate has the exogenous individual's consumption growth rate for upper bound.

This condition is indeed necessary to obtain some positive aggregate assets in the economy. With [START_REF] Boucekkine | del Río, Vintage capital and the dynamics of the AK model[END_REF], compute the individual's initial consumption on the balanced growth path as follows:

c (s, s) = w (s) R ω 0 e -(r-ḡ)z dz R ω 0 e -[(1-σ)r+σρ]z dz . ( 34
)
The condition ḡ < σ (rρ) implies c (s, s) < w (s) and is then necessary to obtain a positive asset accumulation along the life-cycle. Were ḡ be higher than σ (rρ), individuals would be indebted all along their life and the aggregate assets would be negative. Remark also that the aggregate capital growth rate ḡ + n may be higher than the interest rate, which however, following the arguments in Saint-Paul [START_REF] Saint-Paul | Fiscal Policy in an Endogenous Growth Model[END_REF], does not implies that the equilibrium path is dynamically inefficient. Consider now the other roots of the characteristic equation.

Corollary 3

The capital growth rate, g (t), has a saddle-point trajectory.

Proof. See the Appendix.

Due to delays and advances, the characteristic equation associated to the capital dynamics is transcendental. There is an infinite number of roots that have been characterized in Lemma 2. Notably, complex roots with real part larger than ḡ create some instability: any solution including a complex root with a real part larger than ḡ would therefore diverge. Then, since there is no permanent cycle, the dynamics of the growth rate is saddle-point. Let us now characterize the dynamics on the stable manifold.

Lemma 4 There exists T > τ + ω such that:

k (t) = δ 1 e ḡ(t-τ ) + X p n +iq n ∈Γ S δ n e pn(t-τ ) cos (q n (t -τ )) , (35)
for all t ≥ T and where Γ S is the subset of complex roots of the characteristic equation with real part lower than ḡ, and where the δ n are residues computed in the proof.

Proof. See the Appendix.

In the proof of Lemma, it is shown that the MFDE that characterizes the dynamics for t ≥ τ + ω, rewrites, on the stable manifold, as a delay differential equation with only the complex roots whose real part are lower than ḡ. Following Mallet-Paret and Verduyn-Lunel [START_REF] Mallet-Paret | Exponential dichotomies and Wiener-Hopf factorizations for mixed-type functional differential equations[END_REF], this result is due to the absence of roots in a given strip, which has been proved in Lemma 3. Then, standard results for delay differential equations may be applied and notably the one that permit to write the solution as a series of exponential. As in Boucekkine and al. [START_REF] Boucekkine | Vintage human capital, demographic trends and endogenous growth[END_REF] and in the Vintage Capital literature, we therefore observe that the capital dynamics exhibits oscillations that decrease in magnitude and finally disappear. Concerning the endogenous growth rate, its solution can be written, using (35), as follows:

g (t) = δ 1 ḡ + P pn+iqn∈Γ S δ n e (p n -ḡ)(t-τ ) (p n cos (q n (t -τ )) -q n sin (q n (t -τ ))) δ 1 + P p n +iq n ∈Γ S δ n e (p n -ḡ)(t-τ ) cos (q n (t -τ )) , (36)
It has been said that the growth rate has a saddle-point trajectory. Moreover, on the stable manifold, it converges with exponentially decreasing oscillations to its steady-state. Since p n < ḡ in (36), we indeed have:

lim t→+∞ g (t) = ḡ. (37)
Hence, introducing a demographic structure in a simple one-sector model with constant returns to capital is sufficient to eliminate its most unpleasant conclusion, namely, the absence of transitional dynamics. Our results on equilibrium growth complement in this latter regards those of Boucekkine, Licandro, Puch and del Río [START_REF] Boucekkine | del Río, Vintage capital and the dynamics of the AK model[END_REF] on optimal growth. These oscillations that generically emerge with delay differential equations are now shown to be crucial for the existence of an equilibrium.

Existence of a competitive equilibrium

Rustichini [START_REF] Rustichini | Functional differential equations of mixed type: the linear autonomous case[END_REF] pointed out that the Cauchy problem for an MFDE is not well set up because of the advanced terms, and therefore that the differential equation may fail to have a solution. In our framework, we are going to show that the dynamic system always has a solution, but that this solution may not constitute a valid equilibrium. Our argument relies on the interaction between the oscillating dynamics and the initial wealth distribution.

The importance of the initial conditions can be apprehended by first fig-

uring out that if the initial wealth distribution were the stationary one, there will be obviously no dynamics out of the balanced growth path. A similar situation would occur in a economy with no initial conditions. Following Burke [START_REF] Burke | A benchmark for comparative dynamics and determinacy in overlapping-generation economies[END_REF] and Demichelis and Polemarchakis [START_REF] Demichelis | Frequency of trade and the determinacy of equilibrium paths: logarithmic economies of overlapping generations under certainty[END_REF] who have stressed the importance of the date at which the economy begins, consider the following result:

Lemma 5 If τ → -∞, the endogenous growth rate solves: g (t) = ḡ, for all t ∈ R.

Proof. See the Appendix.

Hence, if there is no initial condition, the economy is immediately on the balanced growth path. The key point of the result is that the solution k (t)

must have bounded variations on (τ, ∞). If time extends from an infinite past, the condition of bounded variations extends to R. This imposes to not only eliminate the exponentially increasing oscillating solutions as it has been done in Lemma 4, but also the exponentially decreasing oscillating solutions whose real part satisfy p n < ḡ. We conclude with Lemma 3, that the only valid solution has therefore the following form: k (t) = δ 1 e ḡt . Remark also that such a solution is the only one which would satisfy the non-negativity constraint on k (t). The existence result we now present relies on a similar reasoning.

Theorem 1 For any a (s, τ ) given, there exists a unique solution to the system of equations defined Lemma 2. Such a solution do not always constitute an equilibrium since it may imply k (t) ≤ 0 for some t ∈ [τ, ∞).

Proof. See the Appendix.

To prove the existence of a bounded solution we use two preliminary result developed above: from Lemmas 2 and 4, the dynamics is described by functional equations with discrete advances on [τ, τ + ω] and with discrete delays on [τ + ω, ∞). The delay differential equation may then be initialized for any set of initial conditions and there consequently exists a unique solution on [τ, ∞). This solution has a dynamic behavior which is similar to the one of (35): oscillations, whose magnitude decreases with time, characterize the dynamics. However, these oscillations may lead to a negative capital for some finite interval of time, even with a positive stock for times t = τ and t → ∞.

Such a solution is of course not a valid equilibrium.

It is the initial distribution of assets that play the crucial role. To understand this, suppose that an economy is on its balanced growth path and faces an unanticipated increase of the interest rate. Each individual has therefore to choose a new and lower level of consumption to be compatible with the exogenous increase of its consumption growth rate. This additional saving induces a strong increase in the capital growth rate that overshoots its new long-run value. Then, generations who born after the shock benefit from an higher human wealth. This relatively reduces their propensity to save and consequently reduces the capital growth rate. This process continues for a long time while it decreases in magnitude. It is however possible that a saving reduction is strong enough to lead to a negative aggregate asset accumulation. Optimal behaviors are then not compatible with an intertemporal equilibrium.

The presence of bounded life-span generates a transitional dynamics of the growth rate if the economy has a starting point. It indeed creates a persistent dependence of endogenous variables to initial conditions as it is the case in some models of vintage capital. This dependence is usually designated as a replacement echoes effect. If the magnitude of the echo is sufficiently important, the equilibrium does not exist. Conversely, were the individual's life-span unbounded, the oversaving from individuals contemporaneous to the shock would not be possible since it would last forever. The dynamics is then monotonic and, whatever the initial distribution of wealth, there exists an equilibrium.

We now present some numerical illustrations of the impact of the initial wealth distribution on the capital dynamics. Let us consider the simplest possible case such that the economy begins at time τ = 0 and that the length

Conclusion

In this paper, we have theoretically studied the intertemporal equilibrium path of an overlapping-generations model with finitely-lived individuals and constant returns to capital. We have shown that its existence lies on the initial wealth distribution and that its dynamics to a balanced growth path is governed by exponentially decreasing oscillations. Such oscillations may be incompatible with the existence of an equilibrium. Economies with neoclassical production functions shall exhibit the same properties. Their dynamics are however characterized by a non linear MFDE which limit the theoretical analysis to local dynamics and does not permit to solve analytically the initial conditions problem. Applied to exchange economies, the method we have presented in this paper could also be used to study the issue of the indeterminacy of equilibrium path following the recent contributions of Kehoe et al. [START_REF] Kehoe | Determinacy of equilibrium in large-scale economies[END_REF], Burke [START_REF] Burke | A benchmark for comparative dynamics and determinacy in overlapping-generation economies[END_REF], and Ghiglino and Olszak-Duquenne [START_REF] Ghiglino | On the impact of heterogeneity on indeterminacy[END_REF].

Appendix

Proof of Lemma 1. The optimal consumption path satisfies for all t ≥ τ :

∂c (s, t) ∂t = σ [r (t) -ρ] c (s, t) , (38)
and:

Z s+ω t c (s, z) e -z t r(u)du dz = a (s, t) + Z s+ω t w (z) e -z t r(u)du dz. (39)
Equation ( 38) is the standard Euler condition while condition (39) is the intertemporal budget constraint obtained integrating forward condition [START_REF] Auerbach | Dynamic Fiscal Policy[END_REF] and using the optimal terminal conditions on assets:

a (s, s + ω) = 0 if ω ¿ +∞, lim z→+∞ e -z s r(u)du a (s, s + z) = 0 if ω → +∞. (40)
Using ( 38) and (39) allow to rewrite the optimal consumption such that:

c (s, t) = a (s, t) + R s+ω t w (z) e -z t r(u)du dz R s+ω t e -z t [(1-σ)r(u)+σρ]du dz . (41)
For individuals born at time s ∈ (τω, τ ) and still alive at time t ∈ (τ, τ + ω), the optimal consumption path is obtained using (41) to first define c (s, τ ); then with (38) one obtains:

c (s, t) = a (s, τ ) + R s+ω τ w (z) e -z τ r(u)du dz R s+ω τ e -z τ [(1-σ)r(u)+σρ]du dz e σ t τ [r(u)-ρ]du , (42)
and then [START_REF] Boucekkine | Vintage human capital, demographic trends and endogenous growth[END_REF]. Conversely, for individuals born at time s > τ, the initial consumption c (s, s) is obtained using the second equation of ( 4) and (41); then, with (38), one has:

c (s, t) = R s+ω s w (z) e -z s r(u)du dz R s+ω s e -z s [(1-σ)r(u)+σρ]du dz e σ t s [r(u)-ρ]du , (43)
and then [START_REF] Boucekkine | del Río, Vintage capital and the dynamics of the AK model[END_REF]. ¤ Proof of Lemma 2. Replacing the assumption e (t) = k (t) in ( 22) and ( 23) yields factor prices: r (t) = αAδ and w (t) = (1α) Ak (t). Moreover, at the equilibrium of the capital market, one has: a (t) = k (t). Hence [START_REF] Demichelis | Frequency of trade and the determinacy of equilibrium paths: logarithmic economies of overlapping generations under certainty[END_REF] rewrites:

dk (t) dt = [A -δ -n] k (t) -c (t) , (44)
where c (t) is computed using (42), ( 43) and the factor prices to obtain:

c (t) = β Z τ t-ω a (s, τ ) e -n(t-s) e σ(αA-δ-τ )(t-τ ) R s+ω τ e -[(1-σ)(αA-δ)+σρ](z-τ ) dz ds +β (1 -α) A Z τ t-ω e -n(t-s) e σ(αA-δ-ρ)(t-τ ) R s+ω τ k (z) e -(αA-δ)(z-τ ) dz R s+ω τ e [(1-σ)(αA-δ)+σρ](t-z) dz ds + β (1 -α) A R t τ e [-n+σ(αA-δ-ρ)](t-s) R s+ω s k (z) e -(αA-δ)(z-s) dzds R ω 0 e -[(1-σ)(αA-δ)+σρ]z dz , (45)
if t ∈ (τ, τ + ω) and:

c (t) = β (1 -α) A R t t-ω e [-n+σ(αA-δ-ρ)](t-s) R s+ω s k (z) e -(αA-δ)(z-s) dzds R ω 0 e -[(1-σ)(αA-δ)+σρ]z dz , ( 46
) if t ≥ τ + ω. Let r = αA -δ, γ = n -σ (r -ρ) and µ = r -σ (r -ρ).
Then, applying Fubini's theorem and rearranging, (45) rewrites:

c (t) = z (t) -β (1 -α) Ae -nω Z t τ k (z) Z t z e -γ(t-s) e -r(z-s) R s τ e -µ(u-s) du dsdz +β (1 -α) Ae -σ(r-ρ)ω e -γ(t-τ ) Z τ +ω τ k (z) Z τ +ω z e -γ(τ +ω-s) e -r(z-s) R s τ e -µ(u-s) du dsdz - β (1 -α) A R t τ k (z) R t z e -γ(t-s) e -r(z-s) dsdz R ω 0 e -µz dz - β (1 -α) Ae -rω e -γ(t-τ ) R τ +ω τ k (z) R τ +ω z e -γ(τ +ω-s) e -r(z-s) dsdz R ω 0 e -µz dz + β (1 -α) Ae -rω R t+ω τ k (z) R t+ω z e -γ(t+ω-s) e -r(z-s) dsdz R ω 0 e -µz dz , (47)
where z (t) is a function of time and initial conditions such that:

z (t) = βe -γ(t-τ ) Z τ t-ω a (s, τ ) e -n(τ -s) R s+ω τ e -µ(z-τ ) dz ds. (48)
Using ( 25) and ( 26) yields (30). Concerning (46), apply Fubini's theorem and rearrange to obtain:

c (t) = - β (1 -α) A ¡ 1 + e -(γ+r)ω ¢ R ω 0 e -µz dz Z t τ k (z) µZ t z e -γ(t-s) e -r(z-s) ds ¶ dz + β (1 -α) Ae -γω R ω 0 e -µz dz Z t-ω τ k (z) µZ t-ω z e -γ(t-ω-s) e -r(z-s) ds ¶ dz + β (1 -α) Ae -rω R ω 0 e -µz dz Z t+ω τ k (z)
µZ t+ω z e -γ(t+ω-s) e -r(z-s) ds ¶ dz.( 49)

Using ( 25) yields (31). ¤

Proof of Corollary 1. For ω → +∞, it is only equation (30) which is relevant for the dynamics. Remark with (9) that lim ω→+∞ β = n if n > 0 and with (48) that lim ω→+∞ z (t) = nµe -γ(t-τ ) k (τ ) if µ > 0. Moreover, define φ (ω) = e -nω ṽ (t)e -σ(r-ρ)ω e -γ(t-τ ) ṽ (τ + ω)

-e -rω v (t + ω) + e -rω e -γ(t-τ ) v (τ + ω) , (50)
where v (.) and ṽ (.) are respectively given by ( 25) and (26). Then compute lim ω→+∞ φ (ω) = 0 to conclude. ¤ Proof of Lemma 3. For all t ≥ τ + ω, the dynamics of capital is given by ( 44) and ( 46). It will be useful to define the following variable:

x (t) = e -(αA-δ-n)t k (t) . (51)
Then:

dx (t) dt = (1 -α) Ax (t) -β (1 -α) A Z t t-ω R s+ω s x (z) e -n(z-s) dz R ω 0 e -µz dz e -µ(t-s) ds, (52)
where µ = (1σ) (αAδ) + σρ. The characteristic equation Q (λ) is obtained by replacing x (t) by e gt in (52). Hence:

Q (λ) = λ -(1 -α) A " 1 - R ω 0 e -µs R ω 0 e λ(z-s) e -nz dzds R ω 0 e -nz dz R ω 0 e -µz dz # . ( 53
)
At the exception of λ = 0, all roots of Q (λ) = 0 are relevant for the dynamics of x (t). Roots, denoted g, of the characteristic equation associated to the dynamics of k (t), denoted P (g), are then simply derived using g = λ + αA -

δ -n.
In what follows we characterize the roots of (53).

1) Real roots. Observe that Q (0) = 0, lim λ→+∞ Q (λ) = +∞ and:

Q 00 (λ) = (1 -α) A R ω 0 e -µs R ω 0 (z -s) 2 e λ(z-s) e -nz dzds R ω 0 e -nz dz R ω 0 e -µz dz > 0. (54)
Hence, apart from λ = 0, there exists a unique real root denoted λ. Moreover,

since Q (n -µ) = n -µ, we have λ < n -µ.
As a consequence, there exists a unique real root to P (g) denoted ḡ that satisfies ḡ < σ (αAδ)σρ.

Re (Q (p + iq)) = p-(1 -α) A " 1 - R ω 0 e -µs R ω 0 e p(z-s) cos (q (z -s)) e -nz dzds R ω 0 e -nz dz R ω 0 e -µz dz # . (55)
Then, we have Re (Q (p + iq)) ≤ Q (p) and we conclude using step 1 to state that Q (p) < 0 for any p ∈ ¡ min © 0, λª , max © 0, λª¢ . Consider now the complex root λ + iq. If it were a root of (53), it would satisfy:

λ -(1 -α) A " 1 - R ω 0 e -µs R ω 0 e λ(z-s) cos (q (z -s)) e -nz dzds R ω 0 e -nz dz R ω 0 e -µz dz # = 0, (56)
This would imply that:

(1 -α) A ≤ ¯λ ¯+(1 -α) A R ω 0 e -µs R ω 0 e λ(z-s) |cos (q (z -s))| e -nz dzds R ω 0 e -nz dz R ω 0 e -µz dz . (57)
Suppose |cos (q (zs))| < 1 for some (z, s) ∈ [0, ω] 2 , then

(1 -α) A < ¯λ ¯+ (1 -α) A R ω 0 e -µs R ω 0 e λ(z-s) e -nz dzds R ω 0 e -nz dz R ω 0 e -µz dz , (58)
which is not possible since Q ¡ λ¢ < 0. Suppose now |cos (q (zs))| = 1 for all (z, s) ∈ [0, ω] 2 ; it would imply q = 0. Apply the same procedure to show that the pure imaginary root iq is not a root of (53).

As a consequence, P (g) has no complex root with real part that belong to

[min {αA -δ -n, ḡ} , max {αA -δ -n, ḡ}] except ḡ itself.
3) and 4) Complex roots of large modulus. They can be localized using the geometrical method proposed by Bellman and Cooke [START_REF] Bellman | Differential Difference Equations[END_REF] p. 410. Their method is based on a distribution diagram. Compute equation (53) to obtain:

Q (λ) = λ -(1 -α) A " 1 - nµ ¡ 1 -e -(λ+µ)ω ¢ ¡ e (λ-n)ω -1 ¢ (λ -n) (λ + µ) (1 -e -nω ) (1 -e -µω ) # . (59)
Define Q (λ) = (λn) (λ + γ) e λω Q (λ) such that:

Q (λ) = λ (λ -n) (λ + µ) e λω -(1 -α) A (λ -n) (λ + µ) e λω + (1 -α) A nµ ¡ e λω -e -γω ¢ ¡ e (λ-n)ω -1 ¢ (1 -e -nω ) (1 -e -µω ) . ( 60
)
Hence Q (λ) is expressed as P 3 i=0 P 2 j=0 λ i e jλ . Then, plot the points x ij with coordinates ¡ e jλ , λ i ¢ such that: The zeros of L 1 are those of function:

λ 3 e λω + (1 -α) Anµe -µω (1 -e -nω ) (1 -e -µω ) = 0, (61)
which asymptotically have a negative real part. The zeros of L 2 are those of function:

λ 3 e -λω + (1 -α) Anµe -nω (1 -e -nω ) (1 -e -µω ) = 0, (62)
which asymptotically have a positive real part. ¤ Proof of Corollary 2. On the balanced growth path, the capital growth rate, defined as (dk (t) /dt) /k (t), is constant. It is therefore given by the pure real root of the characteristic equation, namely ḡ. ¤ Proof of Corollary 3. Let λ n ∈ C, be the roots of equation ( 53) and g n = λ n + rn, with r = αAδ. Following Rustichini [START_REF] Rustichini | Functional differential equations of mixed type: the linear autonomous case[END_REF], define the following subsets:

Γ S = {g n , Re(g n ) < min {r -n, ḡ}} , Γ U = {g n , Re(g n ) > max {r -n, ḡ}} , Γ C = {ḡ} .
Now define M S (resp. M U , M C ) the subset spanned by the roots that belong to Γ S (resp. Γ U , Γ C ). There exists three families of operators:

T S (t) : M S -→ M S for t > 0, T U (t) : M U -→ M U for t < 0, T C (t) : M C -→ M C for t ∈ (-∞, ∞) .
and there exists x > 0 such that for all φ ∈ M S , kT S (t)φk ≤ x kφk , for all t > 0. Therefore elements of M S ∪ M C define the solutions k n (t) that converge to the balanced growth path. To conclude on the dynamics of g (t)

use the second point of Lemma Proof of Claim 1. This is a direct consequence of the result established in the proof of Lemma 3 such that there is no complex roots in the closed strip

[min {αA -δ -n, ḡ} , max {αA -δ -n, ḡ}]. ¤ Claim 2.
Solutions that converge to the balanced growth path solve: and T P (-t) for all t ≥ 0. Applying then Theorems 5.1 and 5.3 we show that these semi group can be represented, up to a finite mode, by delay differential equations. We will write the delay equation as:

dk (t) dt = Z 0 -ω k (t + θ) dη -(θ) , (63)
dκ (t) dt = Z 0 -ω κ (t + θ) dη -(θ) . (64)
Then, use Claim 1 to obtain (63). ¤ Then: zζ -ζ = ϕ. Integrating this equation yields:

ζ (θ) = e zθ ζ (0) + Z 0 θ e z(θ-s) ϕ (s) ds. (65)
Using the following boundary condition:

zζ (0) - Z 0 -ω ζ (s) dη -(s) = ϕ (0) , (66)
equation ( 65) rewrites:

zζ (0) - Z 0 -ω e zs ζ (0) + Z 0 s e z(s-u) ϕ (u) dudη -(s) = ϕ (0) , (67)
and thus:

∆ -(z) ζ (0) = ϕ (0) + Z 0 -ω Z 0 s e z(s-u) ϕ (u) dudη -(s) . (68)
Conclude that the resolvant of A has the following representation:

¡ (zI -A) -1 ϕ ¢ (θ) = e zθ Ã ∆ -1 -(z) " ϕ (0) + R 0 -ω R 0 s e z(s-u) ϕ (u) dudη -(s) + R 0 θ e -zs ϕ (s) ds #! . (69)
Then, since δ n (λ) = res z=λ ¡ (zI -A) -1 ϕ ¢ , one has: We now illustrate this Theorem by computing the dynamics in a particular case. We will characterize the stationary distribution of wealth among generation and the solution for any wealth distribution. To avoid tedious calculus, let us solve the dynamics in the simplest case such that τ = 0, ρ = 0, σ = 1, ω = 1, δ = 0 and n = 0 (and consequently β = 1). Define:

δ n (λ) = e λ. µ H (λ) • ϕ (0) + Z 0 -ω Z 0 s e z(s-u) ϕ (u) dudη -(s) ¸ ¶ , (70)
x (t) = k (t) e -αAt . (74)
Then, equation (44) rewrites:

dx (t) dt = - Z 0 t-1
a (s, 0) (s + 1) ds + (1α) A µ ln (t)

Z t 0 x (z) dz - Z 1 t x (z) ln (z) dz ¶ -(1 -α) A µZ t 0 zx (z) dz - Z t+1 t x (z) (t -z + 1) dz ¶ , (75)
for t ∈ [0, 1] and:

dx (t) dt = (1 -α) A µ x (t) - Z t t-1
(1t + z) x (z) dz -

Z t+1 t (1 + t -z) x (z) dz ¶ , (76)
for t ≥ 1.

Let us first compute the steady-state distribution of wealth among the generations. Differentiating (76) two times yields:

d 3 x (t) dt 3 = (1 -α) A • d 2 x (t) dt 2 + 2x (t) -x (t + 1) -x (t -1) ¸, (77)
for t ≥ 1 and let x (t) = x (1) e (ḡ-αA)(t-1) where ḡ is the growth rate at steady-state. Replace x (t) in (75), to obtain:

x (θ) = e (ḡ-αA)(θ+1)

"

(ḡ -αA) 2 + 2 -e (ḡ-αA) - (ḡ -αA) 3 (1 -α) A # , (78)
for θ ∈ [0, 1] and :

a (t -1, 0) = tη £ (ḡ -αA) 2 + (1 -α) A (1 + t) ¤ e (ḡ-αA)(t+1) -t (1 -α) Aη
To characterize the dynamics, we use Lemma 5.7 of Mallet Parret and Verduyn Lunel [START_REF] Mallet-Paret | Exponential dichotomies and Wiener-Hopf factorizations for mixed-type functional differential equations[END_REF] to compute the following equation:

dx (t) dt = (1 -α) A x (t) 2 -(1 -α) A Z t t-1
x (z) (zt + 1) dz

+ (1 -α) A (λ -ḡ + αA) x (t) Z t t-1
(1t + s) Z t s e (ḡ-αA)(z-t) dzds -(1α) A (λḡ + αA)

Z t t-1
x (θ)

Z θ t-1
Z θ s e (ḡ-αA)(z-θ) dzdsdθ, (81) whose roots of the characteristic equation are very close to the roots with negative real parts of (76). Remark that λ is the real root of the characteristic equation of:

x (t) = Z t t-1
(1t + s)

Z t s x (z) dzds. (82)
Then compute ϕ and φ as described in the first part of the proof. It is then possible to find an initial distribution such that k (t) < 0 on the interval

[0, 2]. ¤

of individual lifespan are normalized: ω = 1 .

 1 Hence the initial dynamics cover the interval [0, 1] while the second dynamics hold for t ≥ 1. Moreover, assume that preferences are logarithmic (σ = 1) and that there is no time discount (ρ = 0); the population is stationary (n = 0 and consequently β = 1) and there is no capital depreciation (δ = 0). Parameter of the production function have been chosen to be A = 100 and α = 0.01 such that the interest rate is normalized (r = αA = 1). In figure1, we have plotted two initial distributions of wealth among generations at time τ = 0 that yield to the same initial aggregate capital per capita k (0) = 1. The solid line is the stationary distribution while the dashed line represents another initial distribution.

Figure 1 :

 1 Figure 1: Initial wealth distribution.

Figure 2 .

 2 Figure 2. Aggregate dynamics.

1 and V 2 ,

 12 Asymptotic roots are those of the convex envelop of the diagram. Here, the distribution diagram is composed of two segments L 1 and L 2 whose slopes are respectively positive and negative. To L 1 and L 2 correspond two locus V which belong to C and where lie asymptotically the zeros of Q (λ).

  for t ≥ i + ω where dη -(θ) is a Lebegues-Stieljes measure on [-ω, 0]. Proof of Claim 2. According to Theorems 3.1 and 3.2 of Mallet-Paret and Verduyn Lunel [18], we have the direct sum decomposition C ([-ω, ω] , C) = P ⊕ Q, and we can define (as in the proof of Lemma 3) semi groups T Q (t)

  Let A: D (A) → C ([-ω, 0]) be the generator of the semi group associated with (64). As in Diekman et al. [12] page 111, we give the explicit representation of the resolvant of A. Let ζ = (zI -A) -1 ϕ, where ϕ ∈ C [τ + ω, ∞).

  where H (λ) is defined by ∆ (z) -1 = H (z) / (zλ) with H analytic at z = λ.¤Proof of Lemma 5. If τ → -∞, the dynamic system is given by equations (27), (29) and (31) only, which can be rewritten as an MFDE. Then, following Mallet-Paret[START_REF] Mallet-Paret | The Fredholm alternative for functional differential equation of mixed type[END_REF], only pure real roots and pure imaginary roots of the characteristic equation are, if they exist, yield a solution which is bounded on R. Remark that the result can also be immediately derived from equation (36). ¤ Proof of Theorem 1. Consider the case such that ω and τ are finite. It has been shown in the proof of Lemmas 2 and 4 that the dynamics on the stable manifold writes as a functional equation with advances for t ∈ [τ, τ + ω] and with delays for t ≥ τ + ω. Let us consider the dynamics:dk (t) dt = Z t t-ω k (u) dη_ (tu) for t ∈ [τ + ω, τ + 2ω] . (71) If k (u) is given for u ∈ [τ, τ + ω], equation(71) is an ordinary differential equation which is non homogenous and linear. Let us denotes by k (u), the path k (u) for u ∈ [τ, τ + ω]. Applying the constant variation formula, we can compute k (t) for t ∈ [τ + ω, τ + 2ω]. Let us denotes the solution as follows:k (t) = ϕ ³ b k (u) ; u ∈ [τ, τ + ω] ´. (72)The dynamics of k (t) for t ∈ [τ, τ + ω] is given by the differential equation (44) with (47) and (48). If k (u) is given for u ∈ [τ + ω, τ + 2ω], equation(44)also becomes an ordinary differential equation which is non homogenous and linear. By simply applying the constant variation formula, we can computek (t) for t ∈ [τ, τ + ω].The solutions writes:k (t) = φ (ϕ (k (u) , u ∈ [τ, τ + ω]) , z (s) ; s ∈ [τω, τ ]) . (73)where z (s) is a function of the initial wealth distribution given by (48).Differentiating (73) with respect to time yield to an ordinary differential equation that can be solved. The equation has an unique solution which depends on the initial condition.

1 te 1 e

 11 (ḡ-αA)(z+1) dz ¶ -t (1α) A µZ t+1 (ḡ-αA)z dz ¶ ,(79)for t ∈ [0, 1] and with:η = " (ḡ -αA) 2 + 2e (ḡ-αA) -(ḡ -αA) 3 (1α) A # .


Luis Rodrigues

Didier Henrion

Mehdi Abedinpour Fallah

An Inverse Optimality Method to Solve a Class of Optimal Control Problems

Keywords: optimal control, inverse optimality, nonlinear systems, Hamilton-Jacobi-Bellman equation

This paper presents an inverse optimality method to solve the Hamilton-Jacobi-Bellman equation for a class of nonlinear problems for which the cost is quadratic and the dynamics are affine in the input. The method is inverse optimal because the running cost that renders the control input optimal is also explicitly determined. One special feature of this work, as compared to other methods in the literature, is the fact that the solution is obtained directly for the control input. The value function can also be obtained after one solves for the control input. Furthermore, a Lyapunov function that proves at least local stability of the controller is also obtained. In this regard the main contribution of this paper can be interpreted in two different ways: offering an analytical expression for Lyapunov functions for a class of nonlinear systems and obtaining an optimal controller for the same class of systems using a specific optimization functional. We also believe that an additional contribution of this paper is to identify explicit classes of systems and optimization functionals for which optimal control problems can be solved analytically. In particular, for second order systems three cases are identified: i) control input only as a function of the second state variable, ii) control input affine in the second state variable when the dynamics are affine in that variable and iii) control input affine in the first state variable when the dyamics are affine in that variable. The relevance of the proposed methodology is illustrated in several examples, including the Van der Pol oscillator, mass-spring systems and vehicle path following.

Introduction

Optimal control problems are hard to solve because the optimal controller is the solution of a partial differential equation called the Hamilton-Jacobi-Bellman (HJB) equation [START_REF] Bryson | Applied Optimal Control[END_REF]. However, when the cost is quadratic and the dynamics are affine in the input there is an explicit solution for the input as a function of the derivatives of the value function. This fact will be used to develop a method to solve the HJB equation for a class of nonlinear systems. The main motivation for this work comes from the controller designer's perspective. When designers are faced with a control engineering problem and want to formulate it in the optimal control framework, the first challenge is to choose the most appropriate cost that will yield a control solution with physical significance. Although this is a difficult choice, quite often the following three properties are required for the design:

1. The closed loop system should be asymptotically stable to a desired equilibrium point 2. The system should have enough damping so that the trajectories do not take too long to settle around the desired equilibrium point 3. The control energy should be penalized in the cost to avoid high control inputs that can saturate actuators

The particular functions involved in the cost are not usually pre-defined, except possibly the requirement on the control energy that is usually represented by a quadratic cost on the input. The work on this paper attempts to find a controller and a cost that together meet the requirements 1-3 and render the controller optimal relative to that cost. To that aim, the cost will be fixed to be quadratic in the input and have an unknown term in the state that shall be determined. The solution is therefore based on the concept of inverse optimality. One special feature of this method, as compared to other methods in the literature, is the fact that the solution is obtained directly for the control input without needing to assume or compute a value function first. Rather, the value function is obtained after one has solved for the control input. A Lyapunov function will also be constructed, at least locally. Work on optimal control and approximate solutions, such as inverse optimality, has started in the sixties (see for example [START_REF] Kalman | When is a linear control system optimal ?[END_REF], [START_REF] Lukes | Optimal Regulation of Nonlinear Dynamical Systems[END_REF] and references therein), concentrating mostly on linear quadratic problems driven by aerospace applications. Thirty years later, the concept of inverse optimality has been revisited by many authors to address nonlinear optimal control problems. In a pioneering paper, Lukes [START_REF] Lukes | Optimal Regulation of Nonlinear Dynamical Systems[END_REF] approximates the solution to an optimal control problem with analytic functions by a a Taylor series, starting with first order terms in the dynamics and second order terms in the cost. The resulting controller is therefore the sum of a Linear Quadratic Regulator (LQR) with higher order terms. Reference [START_REF] Doyle | Nonlinear Games: examples and counterexamples[END_REF] finds the dynamics that verify the HJB equation given the running cost and a value function. In [START_REF] Freeman | Inverse Optimality in Robust Stabilization[END_REF] an analytical expression for a stabilizing controller is obtained for feedback linearizable dynamics given the coordinate transformation that feedback linearizes the system, a control Lyapunov function obtained as the solution of the Riccatti equation for the linearized dynamics and a bound on the decay rate of the Lyapunov function. It is shown that the controller is optimal relative to a cost involving a control penalty bias. Reference [START_REF] Margaliot | Some nonlinear optimal control problems with closed-form solutions[END_REF] uses Young's inequality, which was used before in [START_REF] Krstić | Inverse optimal design of input-to-state stabilizing nonlinear controllers[END_REF] for the design of input-to-state stabilizing controllers, to find an analytical expression for the solution to a class of nonlinear optimal control problems. An expression for the cost that makes the controller optimal was also found. However, there is no indication as to what conditions must be satisfied such that the obtained cost is a sensible cost, namely, such that it is non-negative. This is shown on a case-by-case basis in the examples. Reference [START_REF] Krstić | Inverse optimal design of input-to-state stabilizing nonlinear controllers[END_REF] showed that both the inverse optimal gain assignment and H ∞ problems are solvable for the case where the system is in strict-feedback nonlinear form. For a similar strict-feedback nonlinear form, the work presented in [START_REF] Pan | Backstepping Design with Local Optimality Matching[END_REF] develops a recursive backstepping controller design procedure and the corresponding construction of the cost functional using nonlinear Cholesky factorization. It is shown that under the assumptions that the value function for the system has a Cholesky factorization and the running cost is convex, it is possible to construct globally stabilizing control laws to match the optimal H ∞ control law up to any desired order, and to be inverse optimal with respect to some computable cost function. In terms of applications, reference [START_REF] Krstic | Inverse optimal stabilization of a rigid spacecraft[END_REF] presents an inverse optimal control approach for regulation of a rotating rigid spacecraft by solving an HJB equation. The resulting design includes a penalty on the angular velocity, angular position, and the control torque. The weight in the penalty on the control depends on the current state and decreases for states away from the origin. Inverse optimal stabilization of a class of nonlinear systems is also investigated in [START_REF] Guojun | Inverse Optimal Stabilization of a Class of Nonlinear Systems[END_REF] resulting in a controller that is optimal with respect to a meaningful cost function. The inverse optimality approach used in [START_REF] Krstic | Inverse optimal stabilization of a rigid spacecraft[END_REF] and [START_REF] Guojun | Inverse Optimal Stabilization of a Class of Nonlinear Systems[END_REF] requires the knowledge of a control Lyapunov function and a stabilizing control law of a particular form. In [START_REF] Kanazawa | Inverse optimal control problem for bilinear systems: Application to the inverted pendulum with horizontal and vertical movement[END_REF] an optimal feedback controller for bilinear systems is designed to minimize a quadratic cost function. This inverse optimal control design is also applied to the problem of the stabiliz! ation of an inverted pendulum on a cart with horizontal and vertical movement.

Building on the concept of inverse optimality, but in contrast with previous approaches, the objective of this paper is to offer a solution method for a class of nonlinear systems that can determine at the same time a controller and a sensible non-negative cost that renders the controller optimal. Although limited to models up to third order, an important contribution of the work presented in this paper is the fact that the models considered here do not have to be in strict nonlinear feedback form. In fact, the derivative of state variable i does not necessarily have to be an affine function of state variable i + 1 for the models considered in this paper. Furthermore, the running cost is not assumed to be convex. In addition, the analytical solution for the control input is obtained directly, without needing to first assume or compute any coordinate transformation, value function, or Lyapunov function. The value function and a Lyapunov function can however be computed once! the optimal control input has been found. Finally, conditions are given such that the cost that makes the controller optimal is a sensible non-negative cost. The paper is organized as follows. First the optimal control problem is defined and solved for a class of second order systems, followed by its extension to a class of third order systems and conclusions. Several examples are presented throughout the paper. In the notation used in the paper V x i denotes the partial derivative of V with respect to x i and f ′ (x i ) denotes the derivative of function f with respect to its only argument x i .

Optimal Control Problem Definition and Solution: Second Order Systems

Consider the following optimal control problem

V (x 0 ) = inf ∞ 0 {Q(x) + ru 2 } dt s.t. ẋ1 (t) = f 1 (x 1 , x 2 ) ẋ2 (t) = f 2 (x 1 , x 2 ) + bu x(0) = x 0 , u ∈ U (1)
where it is assumed that

V is of class C 1 , Q(x) ≥ 0, Q(0) = 0, r > 0, b = 0, x(t) = [x 1 (t) x 2 (t)] T ∈ IR 2 .
The set U represents the allowable inputs, which are considered to be Lebesgue integrable functions. The functions f 1 , f 2 are not identically zero and are assumed to be continuous with f 1 (0) = f 2 (0) = 0. These functions will be further constrained in the theorems presented in the paper. The term

L(x 1 , x 2 , u) = Q(x) + ru 2 (2)
is called the running cost. When f 1 , f 2 are linear, from the LQR theory [START_REF] Bryson | Applied Optimal Control[END_REF], one knows that the optimal solution is a linear state feedback law u = -k 1 x 1 -k 2 x 2 . Inspired by this fact, for nonlinear f 1 , f 2 we will search for nonlinear additive state feedback laws of the form u = u 1 (x 1 ) + u 2 (x 2 ) with u 1 (0) = u 2 (0) = 0. The first problem to be solved is to find out for what forms of Q(x) such a control input exists. The second problem is to find a solution u = u 1 (x) + u 2 (x) given a Q(x) in the allowed form. We start by presenting necessary conditions that the value function V must verify for additive control solutions to exist.

Lemma 1 Assume that a control solution of the form

u(x) = u 1 (x 1 ) + u 2 (x 2 ) (3)
with u 1 (x 1 ) of class C 1 and u 2 (x 2 ) continuous, exists for problem (1) with u 1 (0) = u 2 (0) = 0. Furthermore, assume that a class C 1 function V exists that verifies the corresponding HJB equation inf

u H (x 1 , x 2 , u, V x 1 , V x 2 ) = 0 ( 4
)
where

H = Q(x) + ru 2 + V x 1 f 1 (x 1 , x 2 ) + V x 2 f 2 (x 1 , x 2 ) + V x 2 bu (5)
with boundary condition V (0) = 0. Then V must be of the form

V (x) = -2b -1 r (x 2 u 1 (x 1 ) + U 2 (x 2 )) + h(x 1 ) (6)
where u 1 (x 1 ), h(x 1 ) and U(x 2 ) are functions of class C 1 with

u 2 (x 2 ) = U ′ 2 (x 2 ), h(0) = 2b -1 rU 2 (0) (7)
Furthermore, u 1 and u 2 are solutions of the equation

Q -ru 2 1 -2ru 1 u 2 -ru 2 2 -2b -1 rx 2 u ′ 1 f 1 + h ′ f 1 -2b -1 ru 1 f 2 -2b -1 ru 2 f 2 = 0 (8)
where the arguments of the functions were omitted for simplicity.

Proof. Consider the HJB equation ( 4) associated with (1). The necessary condition on u to be a minimizer is

V x 2 = -2b -1 ru(x) (9)
and therefore

V (x) = -2b -1 r u(x)dx 2 + h(x 1 ) (10)
where h(x 1 ) is an arbitrary integration function of x 1 . Replacing (3) into ( 10) yields [START_REF] Margaliot | Some nonlinear optimal control problems with closed-form solutions[END_REF].

From the boundary condition V (0) = 0 one obtains the constraint [START_REF] Krstić | Inverse optimal design of input-to-state stabilizing nonlinear controllers[END_REF] taking into account that u 1 (0) = 0. Differentiating [START_REF] Guojun | Inverse Optimal Stabilization of a Class of Nonlinear Systems[END_REF] with respect to x 1 and using (3) yields

V x 1 = -2b -1 rx 2 u ′ 1 (x 1 ) + h ′ (x 1 ) (11)
Finally, replacing (3), [START_REF] Krstic | Inverse optimal stabilization of a rigid spacecraft[END_REF], and ( 11) in ( 4) yields [START_REF] Pan | Backstepping Design with Local Optimality Matching[END_REF] after rearranging. This finishes the proof.

Remark 1 It is important to note that assuming a control input of the form (3) allows one to transform the HJB equation into an ordinary differential equation instead of a partial differential equation. Furthermore, it is interesting to note that if the value function ( 6) does not have cross terms in x 1 and x 2 , from ( 9), the controller will only depend on x 2 .

Based on the form of ( 8), this equation will now be solved for three different cases: i) control input only as a function of x 2 , ii) control input affine in x 2 when the dynamics are affine in that variable and iii) control input affine in x 1 when the dyamics are affine in that variable.

Case I: Solutions depending only on x 2

For this case we first assume that f 2 is only a function of x 2 . The result is stated in Theorem 1.

Theorem 1 Assume that f 1 (x 1 , x 2 ) and f 2 (x 2 ) are continuous and such that

f 1 (0, 0) = 0, f 2 (0) = 0 ( 12
)
and f 1 is not identically zero. If Q(0, 0) = 0 and Q is of the form Q(x 1 , x 2 ) = -g(x 1 )f 1 (x 1 , x 2 ) + Q 2 (x 2 ) ( 13
)
where g is a function of class C 1 not identically zero, Q 2 = 0 and

Q 2 (x 2 ) ≥ 0 -g(x 1 )f 1 (x 1 , x 2 ) + Q 2 (x 2 ) ≥ 0 (14)
then the stabilizing control input u = u 2 (x 2 ) that is a solution of the quadratic equation

Q 2 (x 2 ) -ru 2 2 -2b -1 ru 2 f 2 (x 2 ) = 0 (15)
is an optimal solution of problem (1) if it is continuous and the corresponding value function is given by

V (x 1 , x 2 ) = -2b -1 r u 2 (x 2 )dx 2 + g(x 1 )dx 1 -2rb -1 U 2 (0) (16) Furthermore, if u 2 is of class C 1 and g ′ (x 1 ) > 0, x 1 = 0 u ′ 2 (x 2 ) < 0, x 2 = 0 ( 17
)
then V is positive definite and it is a local Lyapunov function. The function V is a global Lyapunov function if it is radially unbounded. Finally, the trajectories converge to one of the minimizers of L(x 1 , x 2 , u(x 1 , x 2 )), i.e, to a point (x 1 , x 2 ) such that L = 0. If L is convex, then the trajectories will converge to the origin for all initial conditions.

Proof. From the proof of Lemma 1 the HJB equation can be written as [START_REF] Pan | Backstepping Design with Local Optimality Matching[END_REF]. With u 1 = 0 this equation becomes

Q -ru 2 2 + h ′ f 1 -2b -1 ru 2 f 2 = 0 ( 18
)
where Q ≥ 0 under conditions (14). Making

h ′ (x 1 ) = g(x 1 ) (19)
using ( 13) and ( 15) yields 0 = 0, and therefore the HJB equation is satisfied. The HJB equation is a sufficient condition for the control input (3) with u 1 (x 1 ) = 0 to be a solution that minimizes the cost of problem (1) because the second derivative of the Hamiltonian (5) with respect to u is equal to 2r > 0. Using u 1 = 0 and replacing the integral of ( 19) in ( 6) yields the value function (16) taking into account [START_REF] Krstić | Inverse optimal design of input-to-state stabilizing nonlinear controllers[END_REF]. Observe that from the HJB equation ( 4) and from Q(x) ≥ 0, if u 2 is continuous we have

V = -L(x 1 , x 2 , u) ≤ 0 (20)
which makes V a local Lyapunov function for the system if u 2 is also of class C 1 because of the conditions (17) on the Hessian of V . If V is also radially unbounded it is a global Lyapunov function. Finally, since the optimal cost (16) is finite for all initial conditions, then the trajectories will converge to one of the minimizers of L(x 1 , x 2 , u(x 1 , x 2 )) because L ≥ 0 and lim t→∞ L = 0 for integrability. If L is convex, then the trajectories must converge to the origin because the origin is the only minimizer of L. This finishes the proof.

Remark 2 Note that equation (15) with Q 2 (x 2 ) ≥ 0 corresponds to the solution of an optimal control problem with running cost L = Q 2 (x 2 ) + ru 2 and first order dynamics ẋ2 = f 2 (x 2 ) + bu 2 . Therefore, the result of Theorem 1 reduces the solution of an optimal control problem for a second order system to the solution of an optimal control problem for a first order system.

Example 1 If f 1 (x 1 , x 2 ) = -x 3 1 -2x 1 x 2 , f 2 (x 2 ) = x 2 3 (1 + x 2 2 ) and Q(x 1 , x 2 ) = (x 2 1 + x 2 ) 2 +x 4 2 , b = r = 1 then using the result of Theorem 1 we get g(x 1 ) = x 1 , Q 2 (x 2 ) = x 2 2 + x 4 2 and u = u 2 (x 2 ) = -2 + √ 3 x 2 1 + x 2 2 .
We now assume that f 1 is only a function of x 2 .

Theorem 2 Assume that f 1 (x 2 ) and f 2 (x 1 , x 2 ) = f 21 (x 1 )+f 22 (x 1 , x 2 ) are continuous and such that f 1 (0) = 0, f 22 (0, 0) = 0 (21)

and f 1 , f 21 are not identically zero. If Q is of the form Q(x 1 , x 2 ) = rk 2 f 2 1 + 2b -1 rkf 1 f 22 (22) and b -1 kf 1 (x 2 )f 22 (x 1 , x 2 ) ≥ 0 (23) then the control input u = u 2 (x 2 ) = kf 1 (x 2
) is an optimal solution of problem (1) and the corresponding value function is given by

V (x 1 , x 2 ) = -2b -1 rk f 1 (x 2 )dx 2 -f 21 (x 1 )dx 1 (24) Furthermore, if f 1 , f 21 are of class C 1 and b -1 kf ′ 1 (x 2 ) < 0, x 2 = 0 b -1 kf ′ 21 (x 1 ) > 0, x 1 = 0 ( 25
)
then V is positive definite and it is a local Lyapunov function. The function V is a global Lyapunov function if it is radially unbounded. Finally, the trajectories converge to one of the minimizers of L. If L is convex, then the trajectories will converge to the origin for all initial conditions.

Proof. From the proof of Theorem 1 with u 1 = 0 the HJB equation can be written as

Q -ru 2 2 (x 2 ) + h ′ (x 1 )f 1 (x 2 ) -2b -1 ru 2 (x 2 ) [f 21 (x 1 ) + f 22 (x 1 , x 2 )] = 0 (26) Making u 2 (x 2 ) = kf 1 (x 2 ), h ′ (x 1 ) = 2b -1 rkf 21 (27)
and using (22) yields 0 = 0, and therefore the HJB equation is satisfied. Note that under assumption (23), the running cost L is non-negative. The rest of the proof follows the same reasoning of the proof of Theorem 1.

Example 2 If f 1 (x 2 ) = x 3 2 , f 2 (x 1 , x 2 ) = -x 3 1 -x 2 1 x 2 , Q(x 1 , x 2 ) = rk 2 x 6 2 -2b -1 rkx 2 1
x 4 2 then using the result of Theorem 2 we get that u = kx 3 2 is the optimal control with V (x 1 , x 2 ) = -2b -1 rk (x 4 2 /4 + x 4 1 /4) where k is chosen such that b -1 k < 0.

2.2 Case II: Solutions that are affine in x 2 and depend on both

x 1 , x 2
For this case we assume that both f 1 and f 2 are affine functions of x 2 . The main result is stated in Theorem 3.

Theorem 3 Assume that

f 1 (x 1 , x 2 ) = g 1 (x 1 ) + g 2 (x 1 )x 2 f 2 (x 1 , x 2 ) = g 3 (x 1 ) + g 4 (x 1 )x 2 ( 28
)
where g 2 (x 1 ) = 0, g 3 (x 1 ), g 4 (x 1 ) are continuous functions,

g 1 (x 1 ) is of class C 1 , g 1 (0) = g 2 (0) = g 3 (0) = g 4 (0) = 0. If given Q 1 (x 1 ) ≥ 0, q 2 > 0, the stabilizing solution u 1 of Q 1 (x 1 ) -ru 2 1 -2b -1 ru 1 g 3 (x 1 ) = 0 (29) is of class C 1 then the control input u = u 1 (x 1 ) -k 2 x 2 (30)
with

k 2 = ± q 2 r -1 , b -1 k 2 > 0 is a solution of the optimal control problem (1) when Q is of the form Q(x) = Q 1 (x 1 ) + q 2 x 2 2 + 2rb -1 (u ′ 1 g 2 -k 2 g 4 ) x 2 2 -h ′ g 1 ( 31
)
and k 2 2 + 2b -1 (u ′ 1 g 2 -k 2 g 4 ) ≥ 0, h ′ g 1 ≤ 0. ( 32
)
where h(x 1 ) is a function of class C 1 satisfying

h ′ g 2 = -2rk 2 u 1 + b -1 g 3 + 2rb -1 (u 1 g 4 + u ′ 1 g 1 ) (33)
The resulting value function is

V (x) = rb -1 -2u 1 x 2 + k 2 x 2 2 -2rk 2 g -1 2 u 1 + b -1 g 3 dx 1 +2rb -1 g -1 2 (u 1 g 4 + u ′ 1 g 1 ) dx 1 +c ( 34
)
where c is chosen such that the boundary condition V (0) = 0 is satisfied. The function V is also a local Lyapunov function provided it is positive definite in a region around the origin. If V is globally positive definite and radially unbounded then it is a Lyapunov function. If L is convex, then the trajectories will converge to the origin for all initial conditions. If L is not convex then the trajectories will converge to one of the minimizers of L.

Proof. Taking into account (29), (30) and (31), equation ( 8) becomes after rearranging

q-r k 2 2 + 2b -1 (u ′ 1 g 2 -k 2 g 4 ) - q 2 r x 2 2 +2rx 2 k 2 u 1 + b -1 g 3 -b -1 (u 1 g 4 + u ′ 1 g 1 ) + h ′ g 2 2r +h ′ g 1 = 0. ( 35
)
where

q = 2rb -1 (u ′ 1 g 2 -k 2 g 4 ) x 2 2 -h ′ g 1 .
Replacing k 2 = ± q 2 r -1 and h ′ given by (33) into (35), the HJB equation is satisfied. Note that this is a sufficient condition for optimality because the second derivative of the Hamiltonian with respect to u is equal to 2r > 0. For positivity of the running cost L one must have

Q 1 (x 1 ) + q 2 x 2 2 + 2rb -1 (u ′ 1 g 2 -k 2 g 4 ) x 2 2 -h ′ g 1 ≥ 0. ( 36
)
Note that this constraint is always satisfied if (32) holds. The value function V is obtained replacing the control inputs and h in [START_REF] Margaliot | Some nonlinear optimal control problems with closed-form solutions[END_REF]. Note that from the Hessian of V , b -1 k 2 is one of the sufficient conditions for V to be positive definite. The rest of the proof follows the same argument as in the proof of Theorem 1.

Remark 3 It is important to note that the equation ( 29) corresponds to the solution of an optimal control problem with running cost Q 1 (x 1 ) and first order dynamics ẋ1 = g 3 (x 1 ) + bu 1 . Therefore, the result of Theorem 3 reduces the solution of an optimal control problem for a second order system to the solution of an optimal control problem for a first order system plus the addition of a viscous damping term

u 2 (x 2 ) = -k 2 x 2 .
Remark 4 It is interesting to note that when g 1 (x 1 ) = 0, g 2 (x 1 ) = 1, Q 1 (x 1 ) = 0 and x 1 g 3 (x 1 ) < 0, x 1 = 0, meaning that ẋ1 = g 3 (x 1 ) is asymptotically stable, then the result of Theorem 3 coincides with the result of Theorem 2.

Example 3 Consider the mass-spring system with dynamics g 1 (x 1 ) = 0, g 2 (x 1 ) = 1, g 3 (x 1 ) = -x 3 1 , g 4 (x 1 ) = 0, b = 1, and assume Q 1 (x 1 ) = 0. Then, using the results of Theorem 3, from (29) we get u 1 = 0. Therefore, with the running cost L(x, u) = q 2 x 2 2 +ru 2 the solution is u = -q 2 r -1 x 2 with value function V (x) = √ q 2 r (x 2 2 + 0.5x 4 1 ), which is also a Lyapunov function for the closed loop system. Note that the control input is adding viscous damping to the mass-spring system to stabilize it to the origin, which makes perfect sense from a physical point of view.

Example 4 Consider the Van der Pol oscillator with dynamics given by b = 1, g 1 (x 1 ) = 0, g 2 (x 1 ) = 1, g 3 (x 1 ) = -x 1 , g 4 (x 1 ) = 0.5(1-x 2 1 ), and assume Q 1 (x 1 ) = 0, q 2 = 1, r = 1. Then, using the results of Theorem 3, from (29) we get u 1 = 0 and the optimal controller u = -x 2 with associated value function V (x) = x 2 1 +x 2 2 , which is also a Lyapunov function for the closed loop system. The running cost is L(x, u) = x 2 1 x 2 2 + u 2 . This controller makes perfect sense from a physical point of view because to damp out the oscillations and make the trajectories converge to the origin the input simply adds viscous damping.

Example 5 Let b = 1, g 1 (x 1 ) = -x 3 1 , g 2 (x 1 ) = 1, g 3 (x 1 ) = g 4 (x 1 ) = 0, Q 1 (x 1 ) = q 1 x 2
1 , r = 1, q 1 , q 2 > 0. This is a system in strict feedback form to which backstepping techniques can be applied. From the results of Theorem 3, solving (29) the resulting controller is u = -√ q 1 x 1 -√ q 2 x 2 . From (33) one gets

h ′ g 1 = -2 √ q 1 q 2 x 4 1 + √ q 1 x 6 1 ≤ 0 and from (31) Q(x 1 , x 2 ) = q 1 x 2 1 + 2 √ q 1 q 2 x 4 1 + 2 √ q 1 x 6 1 + (q 2 -2 √ q 1 ) x 2 2
The constraint (32) is q 2 ≥ 2 √ q 1 . Finally, from (34), the value function is

V (x) = 2 √ q 1 x 1 x 2 + √ q 2 x 2 2 + √ q 1 x 2 1 + 2 √ q 1 x 4 1 4
which is a Lyapunov function for the closed loop system.

2.3 Case III: Solutions that are affine in x 1 and depend on both

x 1 , x 2
For this case we assume that both f 1 and f 2 are affine in x 1 . The main result of this section is stated in the next theorem.

Theorem 4 Assume that there exist real scalars a, b, c, d such that

f 1 (x 1 , x 2 ) = ax 1 + f (x 2 ) f 2 (x 1 , x 2 ) = cx 1 + df (x 2 ) ( 37
)
where f is not identically zero and is assumed to be continuous with f (0) = 0 and with a locally positive definite anti-derivative F (x 2 ) such that F ′ (x 2 ) = f (x 2 ). Assume further that c(ad -c) ≥ 0 and that for some β > 0

βa 2 ≥ c 2 ≥ a 2 d 2 (38)
This implies that either a = c = 0 or a = 0, c = 0 or a = 0, c = 0, d = 0. Furthermore, assume that

Q(x) = q 1 x 2 1 + q 2 x 2 2 + q(x) (39)
where q 1 ≥ 0, q 2 > 0 and

q 1 = q 2 c 2 a -2 + 2rb -2 c (ad -c) , a = 0 q 1 ≥ q 2 d 2 , a = 0 (40)
Finally, let q(x) be chosen as

q = 2rk 1 k 2 x 1 x 2 + rb -2 k 2 1 k -2 2 -d 2 f 2 (41)
Then, there exist gains k 1 , k 2 , k verifying

k 2 = q 2 r ( 42
)
k 1 = -ca -1 k 2 , a = 0 q 1 r , a = 0 (43) k = b -1 d + k 1 k 2 (44)
such that the control input ( 3) is a solution of the HJB equation ( 4) associated with (1) with value function

V (x) = -rb -1 kcx 2 1 + rb -1 k 1 √ k 2 x 1 + √ k 2 x 2 2 +2rb -1 k (F (x 2 ) -F (0)) ( 45
)
The function V (x) is also a local Lyapunov function provided b > 0, kc ≤ 0 and the term kF (x 2 ) is locally positive definite, i.e, if for some class K function δ and positive γ one has kF

(x 2 ) ≥ δ ( x 2 ) , ∀x 2 ∈ Ω {x 2 : x 2 ≤ γ} ( 46
)
If V is globally positive definite and radially unbounded then it is a Lyapunov function.

Finally, the trajectories converge to one of the minimizers of L. If L is convex, then the trajectories will converge to the origin for all initial conditions.

Proof. The HJB equation ( 8) can be written as

0 = (q 1 -rp 1 ) x 2 1 + (q 2 -rk 2 2 ) x 2 2 + q +h ′ f + p 2 x 1 f -rk (k -2b -1 d) f 2 -2r [k 1 k 2 -b -1 (k 1 a + k 2 c)] x 1 x 2 + h ′ ax 1 +2r [b -1 (k 1 + k 2 d) -k 2 k] x 2 f (47)
where the arguments of the functions were omitted for simplicity and

p 1 (k 1 , k) = k 2 1 -2b -1 ck 1 p 2 (k 1 , k) = 2r [b -1 (k 1 d + kc) -k 1 k] ( 48
)
Since by assumption q 2 > 0, then (42) implies k 2 = 0 and (44) is well defined. Note that the term 47) vanishes because of (44). Note that if a = 0 then c = 0 because of inequalities (38). This observation together with (43) yields

[b -1 (k 1 + k 2 d) -k 2 k] x 2 f in (
k 1 a + k 2 c = 0
and therefore the term (k

1 a + k 2 c) x 1 x 2 in (47) vanishes. Making h ′ (x 1 ) = -p 2 (k 1 , k)x 1 (49)
the term h ′ f + p 2 x 1 f in (47) vanishes. Using (48), ( 40), ( 42), (43), and (44) for the case a = 0, and using (48) and (43) for the case a = 0 (which implies also c = 0), one finds that q 1 -rp 1 = ap 2 . We also see that the term (q 1 -rp 1 ) x 2 1 + h ′ ax 1 in (47) vanishes. The term (q 2 -rk 2

2 ) x 2 2 vanishes because of constraint (42). Using (41) and (44) the term q -2rk 1 k 2 x 1 x 2 -rk (k -2b -1 d) f 2 in (47) also vanishes. Since all terms in (47) vanish, the HJB equation is satisfied. This is a sufficient condition for the control input (3) to be a solution that minimizes the cost of problem (1) because the second derivative of the Hamiltonian (5) with respect to u is equal to 2r > 0. The running cost is a sensible cost because from ( 2) and ( 38)-(43) it is given by

L = r (k 1 x 1 + k 2 x 2 ) 2 + 2rc(ad -c)b -2 x 2 1 + rb -2 k 2 1 k -2 2 -d 2 f 2 + ru 2
and it is non-negative with a minimun at x 1 = x 2 = u = 0 under the assumptions c(ad -c) ≥ 0, (38), ( 40), ( 42), (43). Replacing the integral of (49) in (6), using (48) and (44) yields the value function (45). The boundary condition V (0) = 0 yields the term -2rb -1 kF (0), which is a constant of integration. The rest of the proof follows the same argument as in the proof of Theorem 1.

Remark 5 It is interesting that the square of the nonlinearity comes naturally as a term in the cost, although this would be difficult to predict based on a general tendency to always construct costs that have only quadratic terms on the state.

Example 6

For system (1) with f (x 2 ) = x 2 , a = c = d = 0 and b = 1 one obtains a double integrator. According to Theorem 4, the solution corresponding to

q 1 = q 2 = r = 1 is u = -x 1 -2x 2
and the running cost is

L(x 1 , x 2 , u) = (x 1 + x 2 ) 2 + x 2 2 + u 2
The closed loop system is critically damped and has a double pole at -1. The value function is

V = (x 1 + x 2 ) 2 + x 2 2
which can be rewritten as V = x T P x where

P = 1 1 1 2 Note that V = -(x 1 + x 2 ) 2 -x 2 2 -(x 1 + 2x 2 ) 2 < 0, ∀(x 1 ,
x 2 ) = (0, 0) Therefore, the value function is a global Lyapunov function.

Example 7

For system (1) with a = 0, d = ca -1 , q 1 = q 2 c 2 a -2 and irrespectively of f (x 2 ), q 1 , q 2 one has k 1 k -1 2 = -d, k = 0 and the solution is a linear controller

u = -k 2 (x 2 -ca -1 x 1 )
The running cost and the value function are respectively

L = rk 2 2 (x 2 -ca -1 x 1 ) 2 + ru 2 and V = rb -1 k 2 (x 2 -ca -1 x 1 ) 2 y z y x x B B Y Figure 1: Path Following of Unicycle
Note that in this case the two differential equations in (1) can be combined and the dynamics become ż = bu where z = x 2 -ca -1 x 1 . The controller is u = -k 2 z, which makes the trajectories of z converge to the origin exponentially. In fact, this all makes sense because according to Theorem 4, the trajectories are guaranteed to converge to the minimizers of L given by the points in the set {(x 1 , x 2 ) : x 2 = ca -1 x 1 }, for which the value function is zero. However, note that in this case the value function is not a Lyapunov function because k = 0 and there is no guarantee that the trajectories converge to the origin. It is however a Lyapunov function for the dynamics of z. If c = 0, which implies d = 0, then q 1 = k 1 = 0 and the trajectories will converge to the set of points {(x 1 , x 2 ) : x 2 = 0}. But for x 2 = 0 we have ẋ1 = ax 1 and x 1 therefore converges to zero if and only if a < 0. Example 8 For system (1) consider f (x 2 ) = x 3 2 , a = c = d = 0 and b = 1. According to Theorem 4, the optimal controller corresponding to q 1 = q 2 = r = 1 is

u = -x 1 -x 2 -x 3 2 the running cost is L(x 1 , x 2 , u) = (x 1 + x 2 ) 2 + x 6 2 + u 2 and the value function is V = (x 1 + x 2 ) 2 + 0.5x 4 2 Note that V = -(x 1 + x 2 ) 2 -x 6 2 -x 1 + x 2 + x 3 2 2 < 0, ∀(x 1 , x 2 ) = (0, 0)
Therefore, the value function is a global Lyapunov function. According to Theorem 4, if q 1 = q 2 = r = 1, the optimal controller is

u = -x 1 -x 2 -sin(x 2 ) the running cost is L(x 1 , x 2 , u) = (x 1 + x 2 ) 2 + sin 2 (x 2 ) + u 2
and the value function is

V = (x 1 + x 2 ) 2 + 2 -2 cos(x 2 )
The derivative of the value function is

V = -(x 1 + x 2 ) 2 -sin 2 (x 2 ) -[x 1 + x 2 + sin(x 2 )] 2 ≤ 0
Therefore, the value function is a local Lyapunov function, which proves local stability in the sense of Lyapunov. However, the Lyapunov function is not radially unbounded (it is zero for x 1 = -x 2 = 2nπ for n integer) and asymptotic stability to the origin cannot be proved. In fact, by LaSalle's Invariance Principle [START_REF] Khalil | Nonlinear Systems[END_REF], the trajectories are only guaranteed to converge to the largest invariant set contained in {(x 1 , x 2 ) : V (x 1 , x 2 ) = 0}, which is the set {(x 1 , x 2 ) : x 1 = -x 2 , x 2 = nπ} where n is an integer. Notice that this is also the set of minimizers of L, which is in accordance with Theorem 4. Furthermore, invoking the result of Theorem 4, one cannot guarantee convergence to the origin because L is not convex in this case. Figure 2 shows several trajectories of the unicycle for different initial conditions. Convergence to the desired path is clearly seen for the initial conditions shown in the figure .  where h x 1 is the derivative of h with respect to x 1 and h x 2 is the derivative of h with respect to x 2 . Replacing (51), (67), (68), and (57) in (52) yields after rearranging

0 = (q 1 -rk 2 1 ) x 2 1 + (q 2 -rk 2 2 ) x 2 2 + (q 3 -rk 2 3 ) x 2 3 + Q -2rk 1 k 2 x 1 x 2 -2rk 1 k 3 x 1 x 3 -2rk 2 k 3 x 2 x 3 -rk 2 4 f 2 -rk 2 5 g 2 -2r (k 4 f + k 5 g) (k 1 x 1 + k 2 x 2 + k 3 x 3 ) -2rk 4 k 5 f g +2rb -1 x 3 [k 4 f ′ (df + g) + f (k 1 + dk 2 ) + gk 2 ] + (h x 1 + dh x 2 ) f + h x 2 g (69)
Using ( 59)-( 62), (69) transforms to

0 = -2r (k 4 f + k 5 g) (k 1 x 1 + k 2 x 2 + k 3 x 3 ) -2rk 4 k 5 f g -2rdk 4 k 5 f 2 +2rb -1 x 3 [f (k 1 + dk 2 ) + gk 2 ] + (h x 1 + dh x 2 ) f + h x 2 g (70)
Making

h x 2 = 2rk 5 (k 1 x 1 + k 2 x 2 ) + 2rk 4 k 5 f (71)
yields by integration

h = 2rk 5 k 1 x 1 x 2 + rk 5 k 2 x 2 2 + 2rk 4 k 5 F (x 2 ) + w(x 1 ) (72)
where w is an arbitrary integration function of x 1 . Taking the derivative of (72) with respect to x 1 yields h

x 1 = 2rk 5 k 1 x 2 + w ′ (x 1 ) (73)
Replacing ( 71) and ( 73) into (70), making

w ′ (x 1 ) = 2rk 1 (k 4 -dk 5 ) x 1 (74)
and using (63)-(64) yields the identity 0 = 0 which proves that the HJB equation is satisfied. This is a sufficient condition for the control input (51) to be a solution that minimizes the cost of problem (50) because the second derivative of the Hamiltonian (53) with respect to u is equal to 2r > 0. Using (60)-(64) the running cost is given by

L = r (k 1 x 1 + k 2 x 2 + k 3 x 3 ) 2 + rb -2 k -2 3 k 2 1 -d 2 k 2 2 f 2 + rk 2 5 g 2 -2rb -1 k 4 f ′ x 3 (df + g) + ru 2
and it is non-negative with a minimun at x 1 = x 2 = x 3 = u = 0 under the assumption (66). Integrating (74), using (72), ( 63)-( 64) and the boundary condition V (0) = 0, (54) yields the value function (65). Since V = -L ≤ 0, the function V is also a local Lyapunov function for the system if it is positive definite in a neighbourhood of the origin. The rest of the proof follows the same argument as in the proof of Theorem 1.

Remark 6 It is interesting to note the similarity in the strucure of (45) and (65) for the case c = 0. It is also worth to mention that for d = 0 the results of Theorem 5 agree with the ones obtained in [START_REF] Omrani | An Inverse Optimality Method to Solve a Class of Third Order Optimal Control Problems[END_REF].

Example 10 Consider now the third order integrator extension of example 9. The dynamics are given by (50) with f (x 2 ) = sin(x 2 ), g(x 3 ) = x 3 , b = 1, d = 0. If q 1 = q 3 = r = 1 and q 2 = 4, then according to Theorem 5 the optimal controller is

u = -x 1 -2x 2 -3x 3 -sin(x 2 ) ( 75
)
the running cost is

L = (x 1 + 2x 2 + x 3 ) 2 + (4 -2cos(x 2 ))x 2 3 + sin 2 (x 2 ) + u 2 (76)
and the value function is

V (x) = (x 1 + 2x 2 + x 3 ) 2 + 2x 2 3 + 2x 3 sin(x 2 ) -4 cos(x 2 ) + 4 (77)
Computing the Hessian of V and approximating the sin and cos functions by their first order Taylor series around zero one finds that V is guaranteed to be positive definite in the set {(x 1 , x 2 , x 3 ) ∈ R 3 : |x 2 | < ǫ, |x 3 | < 1.5ǫ -1 } for small ǫ. Note that if one plots the functions that are the principal minors of the Hessian, one can actually find that ǫ can be as big as π/10 and the values of x 3 can still be obtained from the approximation above giving an accurate estimation of the region where V is positive definite. Moreover, the derivative of the value function is

V = -(4 -2cos(x 2 ))x 2 3 -(x 1 + 2x 2 + x 3 ) 2 -sin 2 (x 2 ) -(x 1 + 2x 2 + 3x 3 + sin(x 2 )) 2
(78)

and is negative definite for x 2 ∈ (-π, π). Therefore, the value function is a local Lyapunov function in the largest invariant set contained in (x 1 , x 2 , x 3

) ∈ R 3 | |x 2 | < π ∩ (x 1 , x 2 , x 3 ) ∈ R 3 | V > 0}
where > 0 stands for positive definite. Note that, as in the previous example, one cannot guarantee convergence to the origin from any initial condition because L is not convex.

Conclusions

This paper presented an inverse optimality method to solve a class of nonlinear optimal control problems. The method is inverse optimal because the running cost that renders the control input optimal is also explicitly determined. The resulting running cost was shown to be a sensible non-negative cost with a minimum at the origin.

There are two main advantages of this method. First, the analytical solution for the control input is obtained directly without needing to assume or compute a coordinate transformation, value function or Lyapunov function. The value function and a Lyapunov function can however be computed after the control input has been found. Another advantage is that it is capable of solving many examples of interest, inlcuding the Van der Pol oscillator, mass-spring systems and vehicle path following. The main drawback of the method is that it is restricted to a specific class of optimal control problems for which the dynamics are affine in the input and the cost is quadratic in the input.

Two interesting conclusions can be drawn from this work. First, the value function contains terms that are the negative integral of the control input. Regarding the control input as a force and the value function as potential energy, this integration leads to the usual expression for conservative forces, which is physically interesting. Second, this work emphasizes the importance of cross terms on the state to find a solution to some optimal control problems. This is not only true in the value function, where they are needed to make the input be a function of both state variables, but also in the cost. Furthermore, making the cost depend on the nonlinearity, potentially including nonquadratic terms on the state, seems to be an important feature of this method. This is in contrast to the traditional quadratic costs that have been used in a great percentage of the available literature in optimal control.

Example 9 Figure 2 :

 92 Figure 2: Unicycle Trajectories

Acknowledgments

The authors would like to acknowledge the Natural Sciences and Engineering Research Council of Canada (NSERC) for funding this research and would like to dedicate this paper to the memory of Dr. El-Kebir Boukas.

Optimal Control Problem Definition and Solution:

Third Order Systems

The results of the previous section are now extended to a class of third order systems for which a = c = 0. Consider the following optimal control problem

where it is assumed that q 1 ≥ 0, q 2 ≥ 0,

The set U represents the allowable inputs, which are considered to be Lebesgue integrable functions. The functions f, g are not identically zero and are assumed to be continuous with f (0) = g(0) = 0. The function g(x 3 ) is assumed to have a locally positive definite anti-derivative G(x 3 ) such that G ′ (x 3 ) = g(x 3 ).

As before, we start by presenting necessary conditions that the value function V must verify for a solution of the form (51) to exist.

Lemma 2 Assume that a control solution of the form

exists for problem (50) and that a class C 1 function V exists that verifies the corresponding HJB equation

where

and with boundary condition V (0) = 0. Then V must be of the form

where h and G are functions of class C 1 with

Proof. Consider the HJB equation (52) associated with (50). The necessary condition on u to be a minimizer is

and therefore

where h is an arbitrary integration function of x 1 and x 2 . Searching for a solution of the form (51), expression (58) becomes (54). From the boundary condition V (0) = 0 one obtains the constraint (56). This finishes the proof.

Theorem 5 Let Q(x) be chosen as

that the control input (51) is a solution of the HJB equation (52) associated with (50) with value function

where u is given by (51). The function V is also a local Lyapunov function for the system provided it is positive definite in a neighborhood of the origin and

If V is globally positive definite and radially unbounded then it is a global Lyapunov function. Finally, the trajectories converge to one of the minimizers of L. If L is convex, then the trajectories will converge to the origin for all initial conditions.

Proof. Differentiating (54) with respect to x 1 yields

and with respect to x 2 yields