Lisa Morhaim

Ayşegül Yıldız Ulus

On history-dependent optimization models: a unified framework to analyze models with habits, satiation and optimal growth

Keywords: History-dependent, Dynamic programming, Optimal growth, Habits, Satiation. JEL Classification: C61, D90

We provide first a framework for history-dependent utility models. We further consider discrete infinite horizon dynamic optimization programs in which the instantaneous payoff presents such history-dependence. An issue about models with habits is their lack of general framework. Our framework allows to study habit models that are either additive or multiplicative or neither, as well as satiation models. Moreover, with this unified setting one can treat the usual optimal growth models with or without habit formation and with or without satiation effects. As the way the history dependence is formalized allows us to use dynamic programming tools. We show that the value function is the unique fixed point of the Bellman operator. Such history-dependent modelizations have their motivations and applications in many areas among which decision theory, psychology, behavioral and environmental economics.

Introduction

"How terribly important habit is. It may largely determine the characteristics or the nature of the brain itself. Habits influence or perhaps can largely determine the choice of trains of thought in one's work." says Ulam [START_REF] Stanislaw | Adventures of a Mathematician[END_REF]. Whether it is habit and virtue as in Aristotle [START_REF] Ameriks | Aristotle: Nicomachean Ethics[END_REF] (Nicomachean Ethics, Book II, Chapter 1), whether it is called habits, customs, satiation, or any other ways, history-dependent behavior issues have a long history in diverse and broad areas, as philosophy (see Wright [START_REF] John | Ideas of habit and custom in early modern philosophy[END_REF], Carlisle [START_REF] Carlisle | On habit[END_REF]), physiology (see Wright [START_REF] John | Custom and habit in physiology and the science of human nature in the british enlightenment[END_REF]), sociology (see Camic 1986[4]), psychology (see Gardner and Rebar [START_REF] Gardner | Habit formation and behavior change[END_REF]), etc. In Economics, these questions have been raised from both theoretical as well as empirical viewpoints. As the work of Havranek et al. [START_REF] Havranek | Habit formation in consumption: A meta-analysis[END_REF] shows the literature about the estimates of habit formation with respect to data analysis techniques and models specifications is vast. Some authors (as Fuhrer [START_REF] Jeffrey | Habit formation in consumption and its implications for monetary-policy models[END_REF]) shows that habit formation models are crucial in order to fit the data and some others like Dynan [START_REF] Dynan | Habit formation in consumer preferences: Evidence from panel data[END_REF] finds no evidence of habits working with panel household data. Refer also to Tserenjigmid [START_REF] Tserenjigmid | On the characterization of linear habit formation[END_REF] and Wathieu [START_REF] Wathieu | Habits and the anomalies in intertemporal choice[END_REF]. Several optimal growth models with history-dependent utilities have been proposed in the literature since the fundamental Ryder and Heal [START_REF] Harl | Optimal growth with intertemporally dependent preferences[END_REF] paper, mostly taking habits into account, with various utility and habit stocks evolution specifications (see Carroll Overland and Weil [START_REF] Christopher D Carroll | Saving and growth with habit formation[END_REF], Wendner [START_REF] Wendner | Do habits raise consumption growth[END_REF], Hiraguchi [START_REF] Hiraguchi | Some foundations for multiplicative habits models[END_REF][START_REF] Hiraguchi | A two sector endogenous growth model with habit formation[END_REF]). We rely on axiomatic foundations on preferences as developed for habits in Rozen [START_REF] Rozen | Foundations of intrinsic habit formation[END_REF], and further in Rustichini and Siconolfi [START_REF] Rustichini | Dynamic theory of preferences: Habit formation and taste for variety[END_REF] and He Dyer and Butler [START_REF] He | On the axiomatization of the satiation and habit formation utility models[END_REF] for habits and satiation to propose a general framework for modelling history-dependent utilites in order to study optimal growth models, thus attempting clarifying existing models' differences and similarities. The paper is organized as follows. In Section 2, we introduce the general adjusted consumption framework as well as the general optimal growth model with adjusted consumption. In Section 3, we show that such frameworks allow to encompass the usual models in the habit and satiation literature as well as the usual optimal growth models with habit previously treated in the literature. Section 4 is reserved for the general results of the existence of a solution and caracterisation of optimal paths given by dynamic programming tools.

A general framework 2.1 Introduction to the general adjusted consumption framework

In this model, we consider a representative consumer consuming a single good on periods t = 0, 1, 2, .... with preference over the consumption c = (c 0 , c 1 , ....) from the set of consumption streams which is

a subset of l ∞ l ∞ + = {x = (x t ) ∞ t=0 ∈ (IR + ) IN , x ∞ := sup t∈IN x t < +∞}
The preference depends on her consumption history which is modelized in the following way. At time t = 0, she has a time-0 history1 , denoted h(0) := (h

(0) 0 , h (0) 1 , ....) = (h (0) j ) ∞ j=1 lying in l ∞ + .
Having consumed c 0 at date 0, she has then time-1 history equal to h(1) := (c 0 , h(0) ) at date t = 1. And so forth, for any date t ≥ 1, time-t history will be

∀t ≥ 1, h(t) = (h (t) j ) ∞ j=1 := (c t-1 , h(t-1) ) = (c t-1 , ...., c 0 , h(0) )
where h

(t) j is the consumption j periods prior to time t for j such that 1 ≤ j ≤ t, h

(t) j = c t-j and (h (t) j ) ∞ j=t+1 = h(0)
. The consumer knows that her date t-preferences are influenced both by her consumption at date t and by her time-t consumption history. Indeed, her date t-preferences change endogenously with respect to a reference point (called date t-adjustment level) that is generated from her time-t consumption history h(t) through an adjustment level function) denoted by ϕ, where ϕ is defined on l ∞ + with values on R = (IR + ) n (where n ≥ 1), i.e.

ϕ : l ∞ + → R Note that n = 1 is considered in order to recover the Ryder and Heal [START_REF] Harl | Optimal growth with intertemporally dependent preferences[END_REF] model and n = 2 when we wish to modelize habits and satiation effects as in He Dyer and Butler [START_REF] He | On the axiomatization of the satiation and habit formation utility models[END_REF] (see Section 3). Thus this framework allows to consider other models with different possible effects. The consumer then faces her adjusted consumption stream as follows

(c t , ϕ( h(t) )) ∞ t=0
The consumer's instantaneous utility function u is defined on a subset D u of IR + × R, i.e.

u : D u ⊆ (IR + × R) → IR
and for β ∈ (0, 1) the (given) discount factor, the intertemporal utility will be

∞ t=0 β t u(c t , ϕ( h(t) ))
We now discuss several interpretations and forms of ϕ that have been introduced and usually dealt with in the related literature. We refer more specifically to the papers by Rozen [START_REF] Rozen | Foundations of intrinsic habit formation[END_REF], Ryder and Heal [START_REF] Harl | Optimal growth with intertemporally dependent preferences[END_REF], He Dyer and Butler [START_REF] He | On the axiomatization of the satiation and habit formation utility models[END_REF], Rustichini and Siconolfi [START_REF] Rustichini | Dynamic theory of preferences: Habit formation and taste for variety[END_REF], Baucells and Sarin [START_REF] Baucells | Predicting utility under satiation and habit formation[END_REF].

The adjustment level function ϕ

In Rozen [START_REF] Rozen | Foundations of intrinsic habit formation[END_REF], consumption histories are habits' streams. Rozen [START_REF] Rozen | Foundations of intrinsic habit formation[END_REF] provided habit formation preference axioms that lead to a representation through the instantaneous utility u(c t -ϕ( h(t) ) so that ϕ is a linear function defined by a unique sequence of coefficients (α k ) k≥1 by

∀ h = (h k ) k≥1 , ϕ( h) = +∞ k=1 α k h k
with the coefficients satisfying ∀k ≥ 1, α k ∈ (0, 1).

We use the notation h(0) = (h 1 , h 2 , h 3 , ....) =: h instead of h (0) = (...., h 3 , h 2 , h 1 ) =: h in Rozen [START_REF] Rozen | Foundations of intrinsic habit formation[END_REF], and h(t) = (c t-1 , ...., c , c 0 , h) instead of h (t) = (h, c 0 , c 1 , ...., c t-1 ) in Rozen [START_REF] Rozen | Foundations of intrinsic habit formation[END_REF].

When initial history h is given, ϕ can be defined through a law of motion and if the consumption stream is c = (c t ) +∞ t=0 , for any t ≥ 1, h(t) being defined as before, i.e. h(t) = (c t-1 , c t-2 , ....., c 1 , c 0 , h), one has in our notations as follows: [START_REF] Rozen | Foundations of intrinsic habit formation[END_REF] distinguishes between two types of habits whether they have geometric coefficients with

∀t ≥ 1, ϕ( h(t+1) ) = λ 1 ϕ( h(t) ) + λ 2 c t for λ 1 , λ 2 ∈ [0, 1] such that λ 1 +λ 2 ≤ 1. Rozen
λ 1 + λ 2 = 1 or if λ 1 + λ 2 < 1.
Habit formation in continuous time is considered in the seminal paper of Ryder and Heal [START_REF] Harl | Optimal growth with intertemporally dependent preferences[END_REF]. Rewriting the discrete time version, c t denotes the consumption at date t and the habit variable z t is a weighted average of past consumption levels, so that the law of motion is given by

z t+1 = ρc t + (1 -ρ)z t , with ρ ∈ (0, 1)
This model can be written with our notations defining ϕ by the law of motion

∀t ≥ 0, ϕ( h(t+1) ) = ρc t + (1 -ρ)ϕ( h(t) )
so that the instantaneous utility u(c t , z t ) is u(c t , ϕ( h(t) )). In both previous models, the adjustment level function can be defined by a recurrence relation. In Rozen, ϕ is a linear function that can be defined by the following recurrence relation

∀t ≥ 0, ϕ( h(t+1) ) = G(c t , ϕ( h(t) )) where G(x, y) = λ 1 y + λ 2 x
Similarly, in Ryder and Heal [START_REF] Harl | Optimal growth with intertemporally dependent preferences[END_REF], as well as in many of the usual cases, ϕ is a linear function that can be defined by the following recurrence relation

∀t ≥ 0, ϕ( h(t+1) ) = G(c t , ϕ( h(t) ))
where

G(x, y) = ρx + (1 -ρ)y
Even though linear formation is important as widely used and prominent in applied literature, Rustichini Siconolfi [START_REF] Rustichini | Dynamic theory of preferences: Habit formation and taste for variety[END_REF] 2 underline that habit stock formation can be generalized to any form of G.

In order to consider such generalizations, we propose a framework that allows not only to cover all the linear cases but also to consider nonlinear dynamics as suggested in Rustichini and Siconolfi [START_REF] Rustichini | Dynamic theory of preferences: Habit formation and taste for variety[END_REF].

We allow the adjusted level function ϕ to be defined more generally by

∀t ≥ 0, ϕ( h(t+1) ) = G(c t , ϕ( h(t) ))
with the function G not being necessarily linear. Moreover, this framework allows to encompass models with both habits and satiation effects. Let us consider such a model as the one described in He Dyer and Butler [START_REF] He | On the axiomatization of the satiation and habit formation utility models[END_REF] (as well as in Beaucells and Sarin [START_REF] Baucells | Predicting utility under satiation and habit formation[END_REF]). Recall that in these models, the instantaneous utility function is given by [v(c t + s t -z t )v(s t )] with the habit level z t satisfying the law of motion (with λ ∈ (0, 1))

z t+1 = (1 -λ)c t + λz t
and the satiation level s t is recursively defined by (with γ ∈ (0, 1])

s t+1 = γ(s t + c t )
We can reformulate it in our framework by defining ϕ : l ∞ + → R, with R = IR 2 such that for any t, ϕ( h(t) ) = (s t , z t ), imposing that the function ϕ satisfies the following law of motion

ϕ( h(t+1) ) = (γ(c t + π 1 • ϕ( h(t) )), (1 -λ)c t + λπ 2 • ϕ( h(t) ))
where π i is the projection on the i-th coordinate (i = 1, 2). The instantaneous utility u :

(IR + × IR 2 ) → IR is defined for any c ∈ l ∞ + , any s ∈ IR and z ∈ IR by u(c, s, z) := v(c + s -z) -v(s) so that u(c t , ϕ( h(t) )) = v(c t + π 1 • ϕ( h(t) ) -π 2 • ϕ( h(t) )) -v(π 1 • ϕ( h(t) ))

On some utility u usual cases

We recall some of the main usual forms of utility. In particular, in the habit literature, the most common specifications are either additive or multiplicative. Rozen [START_REF] Rozen | Foundations of intrinsic habit formation[END_REF] considers an additive specification whereas Caroll Overland and Weil [START_REF] Christopher D Carroll | Saving and growth with habit formation[END_REF] and Hiraguchi [START_REF] Hiraguchi | Some foundations for multiplicative habits models[END_REF] consider a multiplicative specification.

For any c ∈ IR + and any history level φ ∈ R, let us define the adjusted instantaneous consumption a(c, φ) ∈ IR. Habits models correspond to the particular case in which ϕ is real-valued (ie R = IR) and the instantaneaous utility function u is defined by u(c, φ) := v(a(c, φ)) with v an utility function defined on IR. The additive habits models is the particular case (with the notation as in Havranek) in which a(c, φ) = c -γφ and multiplicative habits models is the particular case in which

a(c, φ) = cφ γ
Implications of the choice of additive or multiplicative formulations have been discussed in detail (see Wendner [START_REF] Wendner | Do habits raise consumption growth[END_REF]). Nevertheless, also neither additive nor multiplicative specifications need to be taken into account as Ryder and Heal [START_REF] Harl | Optimal growth with intertemporally dependent preferences[END_REF] remark. The examples given by Ryder and Heal [START_REF] Harl | Optimal growth with intertemporally dependent preferences[END_REF] are

u(c, φ) = -e (c-φ) -c -1 2 (then D u = IR * + × IR), u(c, φ) = c φ (then D u = IR + × IR * + ) and u(c, φ) = (-φ) 3 -c -1 2 (then D u = IR * + × IR)
. One goal of this paper is to propose a framework allowing to encompass all such utility forms: either additive, multiplicative or neither.

A framework for optimal growth models with adjusted consumptions

Let us consider an agent facing her utility as described in the previous section, i.e. her instantaneous utility function u is defined on a subset

D u of IR + × R, with R = (IR + ) n , n ≥ 1, u : D u ⊆ (IR + × R) → IR
and let a production function f be defined on IR + :

f : IR + → IR +
Let the discount factor β ∈ (0, 1) be fixed. For initial given capital stock k 0 > 0 and initial given time-0 history h(0) ∈ l ∞ + , the following general framework and optimization problem, with f the production function and k t the capital stock at date t, is given as follows:

P(k 0 , h(0) ) =                                Maximize +∞ t=0 β t u(c t , ϕ( h(t) )) s.t. ∀t ≥ 0, k t+1 = f (k t ) -c t ∀t ≥ 0, c t ≥ 0 and k t ≥ 0, where k 0 > 0 is given ∀t ≥ 1, h(t) = (c t-1 , ...., c 1 , c 0 , h(0) ) where h(0) = (h 1 , .....) ∈ l ∞ + is given ∀t ≥ 0, (c t , ϕ( h(t) )) ∈ D u ⊆ (IR + × R)
This framework allows to encompass many of the existing models and provides a general model allowing to treat several habit and satiation effects. We discuss on that more precisely in the next section.

Some models and examples

In this section, we will show that our framework allows to recover the models previously considered in the literature, both in the literature on history-dependent utility as well as in the literature on history-dependent optimal growth models.

The discrete time version of the Ryder and Heal[20] model

The original optimization problem in the seminal paper of Ryder and Heal [START_REF] Harl | Optimal growth with intertemporally dependent preferences[END_REF] is in continous time.

We can adapt it in discrete time as follows. The instantaneous consumption is denoted c t , z t is the habit level, and k t the capital stock, at date t. Given initial capital stock k 0 and initial habit level z 0 , the model is:

P(k 0 , z 0 ) =                Maximize +∞ t=0 β t u(c t , z t ) s.t. ∀t ≥ 0, k t+1 = f (k t ) -c t z t+1 = λz t + (1 -λ)c t c t , k t ≥ 0 k 0 > 0 and z 0 ≥ 0 are given
Initial habit level and capital stock (k 0 , z 0 ) being given, the Ryder and Heal optimization program can be written through the previously defined general framework by setting the initial history sequence h(0) := (z 0 , z 0 , ....., z 0 , ....) in l ∞ + and ϕ :

l ∞ + → R, with R = IR + such that    ∀x = (x t ) t ∈ l ∞ such that ∃x ∈ IR + , ∀t ∈ IN, x t = x, one has ϕ(x) = x ∀c ∈ IR + , ∀x ∈ l ∞ + , ϕ(c, x) = (1 -λ)c + λϕ(x)
This allows to recover the Ryder and Heal model. Note that any intial history h(0) ∈ l ∞ + can be considered, provided that ϕ( h(0) ) = z 0 and for all c ∈ IR + and x ∈ l ∞ + , ϕ(c, x) = (1 -λ)c + λϕ(x).

3.2 Some multiplicative habit models (Carroll Overland and Weil [START_REF] Christopher D Carroll | Saving and growth with habit formation[END_REF],

Hiraguchi [START_REF] Hiraguchi | Some foundations for multiplicative habits models[END_REF])

Similarly, the discrete time version of the Carroll Overland and Weil's as well as Hiraguchi's models can be derived from our framework. Carroll, Overland and Weil [START_REF] Christopher D Carroll | Saving and growth with habit formation[END_REF] and Hiraguchi [START_REF] Hiraguchi | Some foundations for multiplicative habits models[END_REF] consider a CRRA utility function of the form (where σ > 1 and γ ∈ [0, 1])

u(c, z) = 1 1 -σ ( c z γ ) 1-σ
The adaptation in our framework is straightforward, defining ϕ as in Section 3.1.

Environmental economics

History dependence is important in environmental economics' models. For example, habit is a crucial issue in these models. We discuss the specifications, which include nonlinear dynamics, proposed by Löfgren [START_REF] Löfgren | Habit formation in the environmental quality: dynamic optimal environmental taxation[END_REF]. Note that Ikefuji [START_REF] Ikefuji | Habit formation in an endogenous growth model with pollution abatement activities[END_REF] and the recent model by Safi and Ben Hassen [START_REF] Safi | Subtractive versus multiplicative habits in environmental economics[END_REF] can be reformulated in our framework.

Löfgren [START_REF] Löfgren | Habit formation in the environmental quality: dynamic optimal environmental taxation[END_REF] proposes a model with environmental quality habit formation and in which a consumption good moreover causes a negative external effect on the environment. The intertemporal utility of the social planner depends on consumption of two goods and the environment. The social planner maximizes the utility given the negative effect of the consumption good on the environment and taking into account that there is habit formation in environmental quality. The model is adapted in discrete time the following way. The instantaneous utility (which takes the particular quadratic form) depends on n t which is the environment that displays habit formation, x t the "dirty" consumption good (the environmental bad), z t the "clean" consumption good and s t the habit level related to the environment, i.e. ũ(n t , x t , z t , s t ) with (γ ∈ (0, 1), β ∈ (0, 1))

   n t = n -γx t z t = y -x t s t+1 = βn t + (1 -δ)s t
One can consider the model with the utility function u defined by u(x t , s t ) = ũ(n -γx t , x t , y -x t , s t ) and s t satisfies the recurrence relation

s t+1 = λ + λ 1 s t + λ 2 x t with λ = βn, λ 1 = (1 -δ), λ 2 = -βγ, since s t+1 = βn t + (1 -δ)s t = s t+1 = β(n -γx t ) + (1 -δ)s t = βn -βγx t + (1 -δ)s t
Note that this is a nonlinear dynamics case.

The model can be written in our notations with c t := x t in the same manner as in Section 3.1 where ϕ is defined through the following recurrence relation

∀c ∈ IR + , ∀ h ∈ l ∞ + , ϕ(c, h) = λ + λ 1 ϕ( h) + λ 2 c and u(c, s) = ũ(n -γc, c, y -c, s) so that u(c t , ϕ( h(t) )) = ũ(n -γc t , c t , y -c t , ϕ( h(t) ))
We now turn to the second aspect of our paper and study history-dependent optimal growth problems.

4 The optimal growth model with adjusted consumptions

In this section, we consider discrete infinite horizon dynamic optimization programs in which the instantaneous payoff presents history-dependence.

On the general model

Let us consider the general model with u, ϕ and β be given as introduced in Section 2.4.

P u,ϕ,β (k 0 , h(0) ) =                                Maximize +∞ t=0 β t u(c t , ϕ( h(t) )) s.t. ∀t ≥ 0, k t+1 = f (k t ) -c t ∀t ≥ 0, c t ≥ 0 and k t ≥ 0, where k 0 > 0 is given ∀t ≥ 1, h(t) = (c t-1 , ...., c 1 , c 0 , h(0) ) where h(0) = (h 1 , .....) ∈ l ∞ + is given ∀t ≥ 0, (c t , ϕ( h(t) )) ∈ D u ⊆ (IR + × R)
It is equivalent to

P u,ϕ,β (k 0 , h(0) ) =                                Maximize +∞ t=0 β t u(f (k t ) -k t+1 , ϕ( h(t) )) s.t. ∀t ≥ 0, k t+1 ∈ [0, f (k t )] ∀t ≥ 1, h(t) = (f (k t-1 ) -k t , f (k t-2 ) -k t-1 , ....., f (k 1 ) -k 2 , f (k 0 ) -k 1 , h(0) ) ∀t ≥ 0, (f (k t ) -k t+1 , ϕ( h(t) )) ∈ D u ⊆ (IR + × R) k 0 > 0 and h(0) = (h 1 , .....) ∈ l ∞ + given

Notations and feasible sets

Notations.

Given (k, h) ∈ IR + × l ∞ + , let us denote D u (k, h) = {k ∈ IR + , (f (k) -k , ϕ( h)) ∈ D u } Definition 4.1.
-For any k ∈ IR + and history h ∈ l ∞ + , the feasible correspondence Γ is given by

Γ(k, h) = {k ∈ [0, f (k)], (f (k) -k , ϕ( h)) ∈ D u } = [0, f (k)] ∩ D u (k, h)
For any given initial capital stock k 0 > 0, and initial time-0 history h(0) ∈ l ∞ + , the feasible set Π(k 0 , h(0) ) is defined by the set of capital sequences feasible from k 0 and h(0) , ie for any k 0 > 0, for any h(0) ∈ l ∞ + ,

Π(k 0 , h(0) ) = { k = (k t ) +∞ t=1 ∈ (IR + ) IN , ∀t ≥ 0, k t+1 ∈ Γ(k t , h(t) ), h(t+1) = (f (k t ) -k t+1 , h(t) )}

Assumptions

Let us give a list of assumptions.

The assumptions on the adjustment function ϕ:

(R1) ϕ : l ∞ → R is a continuous function (R2) ϕ is a linear function 3 .
3 This assumption ensures uniqueness of the problem solution. In the Ryder and Heal model, ϕ is linear on the subset of l ∞ of the sequences constant from a certain rank T , i.e. on the set l ∞

c := {x ∈ l ∞ , ∃T ∈ IN, ∃x ∈ IR, ∀t ≥ T, x t = x}. Indeed, let x ∈ l ∞ c , then ϕ(x) = ϕ(x 0 , x 1 , ...., x T , x T , ....) = λx 0 + (1 -λ)ϕ(x 1 , ...., x T , x T , ....) = λx 0 + λ(1 -λ)x 1 + (1 -λ) 2 ϕ(x 2 , ...., x T , x T , ....) = λ T -1 i=0 (1 -λ) i + λ(1 -λ) T x T
The assumptions on the utility u: (P0) u is continuous on D u (P1) u is increasing in the present consumption c, ie for any c ∈ IR + and y ∈ IR such that (c, y) ∈ int(D u ),u c (c, y) > 0 (P2) u is nonincreasing in any coordinate of the adjustment level y, ie for any c ∈ IR + and y ∈ R such that (c, y) ∈ int(D u ), u y i (c, y) ≤ 0 (P3) concave in (c, y) and strictly concave in c (P4

) (i) ∀y ∈ R, lim c→0 + u c (c, y) = M (y) ≤ +∞ and lim c→+∞ u c (c, y) < 1 (ii) ∀y ∈ R, u(0, y) = 0
The assumptions on the production function f : (f1) f : IR + → IR + is an increasing, continuous and concave function, f (0) = 0.

The objective is well-defined

Lemma 4.1.

-Assume e : IR + → IR + is an increasing continuous function that satisfies (i) or (ii).

Then there exists a, a ∈ IR + such that a = 1 and for any x > 0, e(x) ≤ ax + a , and hence for any x > 0, e + (x) = max{0, e(x)} ≤ ax + a . One obtains if y ∈ [0, e(x)] with x ∈ [0, 1], then y ≤ m. Now, assume y ∈ [0, e(x)] with x > 1. One has y ≤ e(x) and since e(0) = 0, using the fact that e is concave, since

1 x ∈]0, 1[ y ≤ e(x) 0 ≤ e(0) ⇒ 1 x y + (1 - 1 x )0 ≤ 1 x e(x) + (1 - 1 x )e(0) ≤ e( 1 x x + (1 - 1 x )0)
the last inequality coming from the concavity of e. Then 1 x y ≤ e(1) and one obtains if y ∈ [0, e(x)] with x > 1, then y ≤ xe(1). Finally, for any x ∈ R + , if y ∈ [0, e(x)] then y ≤ x(e(1) + m), ie one can define a = e(1) + m. QED Lemma 4.2.-Assume (f1), (P3) and either (P4)(i) or (P4)(ii). Let k 0 > 0 and h(0) ∈ l ∞ + be given. Then there exist a, a 1 (k 0 ), a 2 ∈ IR + such that for any feasible sequence k = (k t ) +∞ t=1 ∈ Π(k 0 , h(0) ) and its associated history h(t) , for any t ≥ 0,

u + (f (k t ) -k t+1 , ϕ( h(t) )) ≤ a 1 (k 0 )a t + a 2
Proof. By assumption (f1) and lemma 4.1, there exists a, a ∈ IR + such that a = 1 and ∀k ∈ IR + , f (k) ≤ ak + a One can check by induction that for any k 0 ∈ IR + and for all feasible sequence k = (k t ) t ie such that for all t, k t+1 ∈ Γ(k t , h(t) )

∀t, k t ≤ a t k 0 + 1 -a t 1 -a a = (k 0 - a 1 -a )a t + a 1 -a
For any sequence k = (k t ) t ∈ Π(k 0 , h), the associated histories are such that for all t, h(t) = (c t-1 , c t , ...., c 0 , h(0) ) such that for all j = 0, ...., t -1, 0 ≤ c j ≤ f (k j-1 ) ≤ (k 0 -a 1-a )a j + a 1-a , so that for all t, h(t) belongs to a compact set Π h of the product topology in l + ∞ . Since u and ϕ are continuous, there exists ϕ ∈ R such that

∀c ∈ IR + , ∀ h(t) ∈ Π h , u(c, ϕ( h(t) )) ≤ u(c, ϕ)
By assumption (P3) and either (P4)(i) or (P4)(ii), applying Lemma 4.1, there exist A, A ∈ IR + such that ∀c ∈ IR + , u + (c, ϕ) ≤ Ac + A so that along a feasible path k = (k t ) +∞ t=1 ∈ Π(k 0 , h), by x → u(x, y) is increasing, for any t ≥ 0,

u(f (k t ) -k t+1 , ϕ( h(t) )) ≤ u(f (k t ) -k t+1 , ϕ) ≤ u(f (k t ), ϕ) ≤ u(ak t + a , ϕ) ≤ u + (ak t + a , ϕ) ≤ A(ak t + a ) + A
and so for any t ≥ 0,

u(f (k t ) -k t+1 , ϕ( h(t) )) ≤ Aa(a t k 0 + 1-a t 1-a a ) + Aa + A ≤ Aa((k 0 -a 1-a )a t + a 1-a ) + Aa + A ie u(f (k t ) -k t+1 , ϕ( h(t) )) ≤ a 1 (k 0 )a t + a 2
with a 1 (k 0 ) = Aa(k 0 -a 1-a ) and a 2 = Aa a 1-a + Aa + A . QED Proposition 4.1.-Assume aβ < 1. Let k 0 ∈ IR + * and h(0) ∈ l ∞ + be given. Then for any feasible sequence k = (k t ) +∞ t=1 ∈ Π(k 0 , h), the limit 6 lim

T →+∞ T t=0 β t u(f (k t ) -k t+1 , ϕ( h(t) )), with ∀t ≥ 1, h(t) = (f (k t-1 ) -k t , f (k t-2 ) -k t-1 , ....., f (k 1 ) -k 2 , f (k 0 ) -k 1 , h(0) ), is well-defined.
6 ie the objective Proof. By Lemma 4.2, along a feasible path k = (k t ) +∞ t=1 ∈ Π(k 0 , h(0) ), with associated histories h

(t) = (f (k t-1 ) -k t , f (k t-2 ) -k t-1 , ....., f (k 1 ) -k 2 , f (k 0 ) -k 1 , h(0) ), ∀t, u(f (k t ) -k t+1 , ϕ( h(t) )) ≤ a 1 (k 0 )a t + a 2 Then for all T ∈ IN, T t=0 β t u(f (k t ) -k t+1 , ϕ(h (t) )) ≤ T t=0 β t (a 1 (k 0 )a t + a 2 ) = a 1 (k 0 ) T t=0 (aβ) t + a 2 T t=0 β t
since 0 < aβ < 1 and 0 < β < 1 the conclusion follows. QED For a sequence k = (k t ) +∞ t=1 ∈ Π(k 0 , h(0) ), we denote by U( k) the objective ie

U( k) = +∞ t=0 β t u(f (k t ) -k t+1 , ϕ( h(t) ))
Assumption on the objective not being equal to -∞ at the optimum (A) ∀k 0 > 0 and h(0

) = 0, ∃ k ∈ Π(k 0 , h(0) ) such that U( k) > -∞.

Existence and uniqueness of the solution

Proposition 4.2.-Assume (R1), (f1), (P0), (P3) and either (P4)(i) or (P4)(ii). Assume moreover that αβ ∈ (0, 1), Π(k 0 , h(0) ) = ∅ and D u is a closed subset of IR 2 . Then there exists an optimal solution. Moreover, if ϕ is linear, the solution is unique.

Proof Let us show that the objective is upper semi-continuous and the feasible sequence set is a compact set of the product topology. Recall that one can check by induction that for any given

k 0 ∈ IR + and h(0) ∈ l ∞ + , for all feasible sequence k = (k t ) t ∈ Π(k 0 , h(0) ), for all t, k t+1 ∈ [0, f (k t )], which implies ∀t, k t ≤ a t k 0 + 1 -a t 1 -a a = (k 0 - a 1 -a )a t + a 1 -a
The feasible set Π(k 0 , h(0) ) is included in a compact set for the product topoplogy. Moreover, it is closed (by the continuity of f and ϕ, and by D u being a closed subset7 ). So the feasible set Π(k 0 , h(0) ) is a compact set for the product topoplogy.

We next show that the objective U is upper semi-continuous.

Let us consider a sequence kn = {(k n t ) +∞ t=1 } n ⊂ Π(k 0 , h(0) ) that converges to k = (k t ) +∞ t=1 ∈ Π(k 0 ). Note that when n converges to +∞, the sequence of associated histories ∀t ≥ 1, h n (t) = (f (k n t-1 )k n t , f (k n t-2 ) -k n t-1 , ....., f (k n 1 ) -k n 2 , f (k 0 ) -k n 1 , h(0) ) converges to the associated history ∀t ≥ 1, h(t) = (f (k t-1 ) -k t , f (k t-2 ) -k t-1 , ....., f (k 1 ) -k 2 , f (k 0 ) -k 1 , h(0) ) and by the continuity of ϕ, for any t ≥ 1, the sequence (ϕ( h n (t)

)) n converges to ϕ( h(t) ).

Let us show that lim n→+∞ U( kn ) ≤ U( k).

For any t ≥ 0, by Lemma 4.2, for any k = (k t ) +∞ t=1 ∈ Π(k 0 , h(0) ) and with history h(t) associated to k u + (f (k t ) -k t+1 , ϕ( h(t) )) ≤ a 1 (k 0 )a t + a 2 and by 0 < aβ < 1, for any ε > 0, there exists T ε such that for any (k t ) +∞ t=1 ∈ Π(k 0 , h(0) ), and for any T ≥ T ε , +∞ t=T β t u + (f (k t ) -k t+1 , ϕ( h(t) )) ≤ ε So for any ε > 0, there exists T ε such that for any n ∈ IN and for any

β t u(f (k t ) -k t+1 , ϕ( h(t) ))
So U is upper semi-continuous on Π(k 0 , h(0) ).

By Weierstrass Theorem (see Aubin [START_REF] Aubin | Initiation à l'analyse appliquée[END_REF], Theorem 5.3.1), since U is upper semi-continuous and Π(k 0 , h(0) ) is a compact set for the product topology, there exists an optimal solution.

The assumption (P3) together with on one hand the fact that u is nondecreasing with respect to the first variable and f is concave, and on the other hand the function ϕ is linear, ensures that there is a unique optimal consumption sequence, hence there is a unique optimal capital sequence.

  (i) e is strictly concave, differentiable and lim x→0 + e (x) = M ≤ +∞ and lim k→+∞ e (x) < 1 (ii) e(0) = 0 and e is concave Proof. (i) 4 Assume e : IR + → IR + is an increasing, continuous and twice differentiable function and that e is strictly concave, lim x→0+ e (x) = M ≤ +∞ and lim x→+∞ e (x) < 1. If lim x→0+ e (x) ≤ 1, then by the concavity of e, the function e is decreasing, and for any x in IR + , one has e (x) ≤ 1 so one can find a, a ∈ IR + such that a = 1 and for any x > 0, e(x) ≤ ax + a . If lim x→0+ e (x) > 1, since lim x→+∞ e (x) < 1, the function e admits a fixed-point x (ie e(x) = x). If x > x, then e(x) < x ≤ (1 + ε)x for ε > 0, and if x ≤ x, since e is increasing, then e(x) ≤ e(x) = x. So for any x in IR + , e(x) ≤ (1 + ε)x + x and one can define a = 1 + ε, a = x (ii) 5 Assume e : IR + → IR + is an increasing, continuous and twice differentiable function and that e(0) = 0 and e is concave. Since e is continuous on IR + , e has a maximum on [0, 1]. Define m = max{y/y ∈ [0, e(x)], x ∈ [0, 1]}.

  T ≥ T ε , and for any n ∈ IN and for any T ≥ T ε , By taking n → +∞ (and using the continuity of u, f and ϕ in the part at the right), Since this is true for any ε > 0, by taking ε → 0,

							+∞	β t u + (f (k n t ) -k n t+1 , ϕ( h n (t)	)) ≤ ε
							t=T
	+∞	β t u(f (k n t ) -k n t+1 , ϕ( h n (t)	)) ≤	T	β t u(f (k n t ) -k n t+1 , ϕ( h n (t)	)) +	+∞	β t u + (f (k n t ) -k n t+1 , ϕ( h n (t)	))
	t=0						t=0		t=T
							≤	T	β t u(f (k n t ) -k n t+1 , ϕ( h n (t)	)) + ε
							t=0
		lim n→+∞	t=0 +∞	β t u(f (k n t ) -k n t+1 , ϕ( h n (t)	)) ≤
		lim n→+∞	+∞ t=0	β t u(f (k n t ) -k n t+1 , ϕ( h n (t)	)) ≤	+∞ t=0

T t=0 β t u(f (k t ) -k t+1 , ϕ( h(t) )) + ε Since this is true for any T ≥ T ε , by taking T → +∞, lim n→+∞ +∞ t=0 β t u(f (k n t ) -k n t+1 , ϕ( h n (t) )) ≤ +∞ t=0 β t u(f (k t ) -k t+1 , ϕ( h(t) )) + ε

We use the term time t-history where Rozen[START_REF] Rozen | Foundations of intrinsic habit formation[END_REF] uses time t-habit and the term reference point at time t (as Rozen[START_REF] Rozen | Foundations of intrinsic habit formation[END_REF]) where Ryder and Heal[START_REF] Harl | Optimal growth with intertemporally dependent preferences[END_REF] use the term customary level of consumption or expected level of consumption.

cf p.3

the proof is analogous as the proof of proposition 2.2.1 p.17 in Le Van and Dana[START_REF] Cuong | Dynamic programming in economics[END_REF]

the proof is analogous as the explanation in comment[START_REF] Aubin | Initiation à l'analyse appliquée[END_REF] on assumption (H2) p.161 in Le Van and Morhaim[START_REF] Cuong | Optimal growth models with bounded or unbounded returns: a unifying approach[END_REF]

Note that this is true in both usual cases with habits only, as either D u = (IR + ) 2 and D u = {(x, y) ∈ IR 2 , x-y ≥ 0}

Note that assumption (A) ensures that such a k ∈ Π(k 0 , h(0) ) with U( k) > -∞ exists.

The value function and Bellman equation

Definition 4.2.-The value function V is defined on IR + * × l ∞ + by for any (k 0 , h(0) )

-The value function V is upper semi-continuous.

Proof. A direct proof using lemma 4.1 can be done. Indeed 8 , let us consider a sequence (k n 0 , h n (0)

) n ⊂

) Let ε > 0. By Lemma 4.1 and as in Proposition 4.1's proof, there exist i 0 and T 0 such that for any i ≥ i 0 and for any T ≥ T 0 , and for optimal path ( kn

) and its associated history

) can be assumed to converge to some k in Π(k 0 , h(0) ). By the definition of the associated history, this implies that ( h n i (t)

) i converges to h(t) the history associated to k, and by the continuity of ϕ, that (ϕ(

8 see Theorem 1 (ii) p.7 in Le Van Morhaim [START_REF] Cuong | Optimal growth models with bounded or unbounded returns: a unifying approach[END_REF]), sauf qu'ici il faut s'assurer que avec la suite des associated histories ca marche aussi 9 by the compactness of Π(

) is satisfied and f is increasing, this assumption is satisfied. In particular, it is satisfied as soon as f is increasing and

Proof. (i) Let h(0) ∈ l ∞ + and 0 < k 0 < k 0 . Let (k t ) +∞ t=1 be a feasible path from (k 0 , h(0) ) and h(t) its associated habit sequence. Then such a path is also feasible from (k 0 , h(0) ) (since

Since the inequality is true for any feasible path from (k 0 , h), it is also true for the sup on the feasible set

, and let us show that

t=1 be any feasible path from (k 0 , h (0) ) and h (t) its associated habit sequence. Then (k t ) +∞ t=1 is a feasible path from (k 0 , h(0) ) and h(t) its associated habit sequence satisfies ∀t,

Since ϕ is nondecreasing and u is decreasing with respect to its second variable

), and let us show that V (k 0 , h (0) ) ≥ V (k 0 , h(0) ). Let (k t ) +∞ t=1 be any feasible path from (k 0 , h (0) ) and h (t) its associated habit sequence. Recall that in this case,

and for any h(0) , h (0) and c ∈ IR + , if ϕ( h(0) ) ≥ ϕ( h (0) ) then ϕ(c, h(0) ) ≥ ϕ(c, h (0) ). Then (k t ) +∞ t=1 is a feasible path from (k 0 , h(0) ) and h(t) its associated habit sequence satisfies ∀t, ϕ(

). Since u is decreasing with respect to its second variable

), one has (k 1 , ...., k T , k T +1 , ....) ∈ Π(k 0 , h(0) ), and

From (i) then lim T →+∞ β T +1 V (k t , h(T+1) ) = 0 QED A standard proof (see Theorem 4.4 p.75 Stokey Lucas and Prescott [START_REF] Stokey | Recursive methods in dynamic economics[END_REF]) allows to show the following result.

Proposition 4.5.-Assume (R1), (f1), (P0), (P3) and either (P4)(i) or (P4)(ii). Then k * is an optimal solution if and only if

Let B be the Bellman operator, ie B : Proof. The proof that V is a fixed-point of the Bellman operator is standard. Uniqueness of the fixed point is shown by contradiction. Indeed, suppose there exists another fixed-point of B in F(IR + × l ∞ + , IR) F b (IR + × l ∞ + , IR). One can easily check that W ≤ V . Let us show that V ≤ W .