Je tiens tout d'abord à remercier profondément mon directeur de thèse Anis Matoussi de m'avoir accueilli au sein de son équipe, qui a su me diriger tout au long de cette thèse avec patience, rigueur, enthousiasme et bonne humeur. Sa patience et ses approches toujours présent à la fois scientifique et pratique ainsi que ses qualités humaines m'ont énormément appris.

J'adresse toute ma gratitude envers mon co-directeur de thèse Mohamed Mnif qui a été présent par ses conseils et ses remarques pour accompagner mes premiers pas en recherche depuis le master.

Je remercie profondément Mohamed Mrad, mon co-encadrant de thèse, d'une part pour avoir accepté de faire partie de cette thèse et d'autre part pour ses conseils, ses remarques, sa générosité avec laquelle il partageait son savoir et sa grande patience.

Je tiens aussi à remercier Jean-Francois Chassagneux et Boualem Djehiche de m'avoir fait l'honneur de rapporter ma thèse. Je suis également très reconnaissante à l'encontre de Marina Kleptsyna, Amal Ben Abda et Caroline Hillairet pour avoir accepter de participer à ce jury en tant qu'examinatrices.

Je remercie également tous les membres du laboratoire Manceau de mathématiques ainsi que le département des mathématiques. Je suis reconnaissante pour leur accueil, leurs conseils et leur sympathie, sans oublier les

INTRODUCTION

This thesis is a cotutelle thesis between Le Mans University, LMM-IRA and Tunis El-Manar University (UTM), LAMSIN-ENIT. I realized during this thesis, a set of 3 works on the stochastic mean field control problem and dynamic progressive utility. The works that are part of this thesis are the following:

• A. Matoussi, M. Mnif and C. Ziri. Linear quadratic control problems for mean field stochastic differential equation with Jumps. Preprint (2021), submitted for publication [START_REF] Matoussi | Mean field control problems and application to production of an exhaustible resource[END_REF].

• S. Ben Aziza and C. Ziri. Numerical approximation for a class of mean field games. Forthcoming paper [START_REF] Ben Aziza | Numerical approach for mean field games[END_REF].

• M. Mrad and C. Ziri. Learning dynamic utilities. Forthcoming paper [START_REF] Mrad | Learning dynamic utilities[END_REF].

Mean field control problem and application to production of an exhaustible resource

The first part of this thesis is dedicated to the study of linear quadratic control problems for mean field stochastic differential equation with jumps (LQ-MF SDE in short) and application to the production of an exhaustible resource. This chapter is based on co-work with Matoussi and Mnif [START_REF] Matoussi | Mean field control problems and application to production of an exhaustible resource[END_REF].

An overview of mean field stochastic differential equations

Historically, the mean-field stochastic differential equations, also known under the name of McKean Vlasov SDE, can be traced back to the works by Kac [65] and McKean [START_REF] Mckean | A CLASS OF MARKOV PROCESSES ASSOCIATED WITH NONLINEAR[END_REF]. These equations are SDEs whose drift and diffusion coefficients depend not only on the state of the unknown process, but also on its law of probability. Recently, this type of equation has been the subject of renewed interest in the context of mean field game theory and mean field control problems. So, we will dedicate the following part to introduce the main results of mean field BSDEs in the framework with and without jumps.

Mean field backward stochastic differential equations: Continuous setting Recently in

2009, Buckdahn et al. [START_REF] Buckdahn | Mean-field backward stochastic differential equations: a limit approach[END_REF] have introduced a new type of backward stochastic differential equations and they have called them the mean field backward stochastic differential equations ( MF BSDEs in short). They showed that the BSDE can be obtained by an approximation involving N-particle systems with weak interaction. Afterwords, Buckdahn, Li and Peng [START_REF] Buckdahn | Mean-field backward stochastic differential equations and related partial differential equations[END_REF] generalized their previous results on mean field BSDEs in which the initial data (t, x) were frozen in the law variable of the coefficients, they investigated the associated nonlocal PDE.

More precisely, the authors [START_REF] Buckdahn | Mean-field backward stochastic differential equations and related partial differential equations[END_REF] assumed the following Assumptions: Assumption 1.1.1.

• There exists a constant C ≥ 0 such that, P-a.s., for all t ∈ [0, T ], y 1 , y 2 , y

• f (•, 0, 0, 0, 0) ∈ H 2 F (0, T ; IR).

The authors established the following existence result:

Theorem 1.1.1. ( Buckdahn, Li and Peng [START_REF] Buckdahn | Mean-field backward stochastic differential equations and related partial differential equations[END_REF])

Under Assumptions 1.1.1. For any random variable ξ ∈ L 2 (ω, F T , P), the mean field BSDE

Y t = ξ + T t E ′ [f (s, Y s , Y ′ s , Z s , Z ′ s )]ds + T t Z s dW s , (1.1.1)
where E ′ is an operator defined by

E ′ (Γ(., ω)) := Ω γ(ω ′ , ω)P(dω ′ ), ∀γ ∈ L 1 (Ω 1+1 , F ⊗ F, P ⊗ P),
has a unique solution (Y, Z) ∈ S 2 F (0, T : IR) × H 2 F (0, T : IR d ).

Y 1 t ≤ Y 2 t , t ∈ [0, T ], P-a.s.
In 2016, Li, sun and Xiong [START_REF] Li | Linear quadratic optimal control problems for mean-field backward stochastic differential equations[END_REF] was the first to study the stochastic linear quadratic problem for mean field BSDEs. They have established the result of existence and uniqueness for the following linear mean field BSDE which depends on the laws of the state Y and the control α:

where ξ is an F T -measurable random vector and A, Â, B, B, C, Ĉ are given deterministic matrixvalued functions that satisfied A, Â ∈ L 1 (0, T : IR n×n ), B, B ∈ L 2 (0, T : IR n×m ), C, Ĉ ∈ L 2 (0, T : IR n×n ).

Moreover, there exist a constant K > 0, independent of ξ and α, such that

E sup t≤s≤T |Y s | 2 + T t |Z s | 2 ds ≤ KE sup t≤s≤T |ξ| 2 + T t |α s | 2 ds .
Recently, Li [START_REF] Li | Reflected mean-field backward stochastic differential equations. approx-imation and associated nonlinear pdes[END_REF] introduced a class of mean field reflected BSDEs which makes the connection between the results of classical reflected BSDEs (RBSDEs in short) [START_REF] El | Reflected solutions of backward sde's, and related ob-stacle problems for pde's[END_REF] and those of MF BSDEs [START_REF] Buckdahn | Mean-field backward stochastic differential equations and related partial differential equations[END_REF][START_REF] Buckdahn | Mean-field backward stochastic differential equations: a limit approach[END_REF]. Later, Djehiche, Elie and Hamadéne [START_REF] Djehiche | Mean-field reflected backward stochastic differential equations[END_REF] deepen the mean field reflected forward backward SDEs by adding the dependence on the distribution of the component of the solution. Since MF BSDE has important application in Mathematical finance as optimal control, we refer the reader for example to [START_REF] Anderson | A maximum principle for sdes of mean-field type[END_REF][START_REF] Buckdahn | A mean-field stochastic control problem with partial observations[END_REF][START_REF] Djehiche | Mean-field reflected backward stochastic differential equations[END_REF][START_REF] Djehiche | Mean-field backward-forward stochastic differential equations and nonzero sum stochastic differential games[END_REF][START_REF] Huang | A linear-quadratic optimal control problem for mean-field stochastic differential equations in infinite horizon[END_REF][START_REF] Li | Reflected mean-field backward stochastic differential equations. approx-imation and associated nonlinear pdes[END_REF][START_REF] Li | General mean-field bsdes with continuous coefficients[END_REF][START_REF] Li | Linear quadratic optimal control problems for mean-field backward stochastic differential equations[END_REF][START_REF] Yong | Linear-quadratic optimal control problems for mean-field stochastic differential equations[END_REF][START_REF] Yong | Stochastic Controls: Hamiltonian Systems and HJB Equations[END_REF].

Mean field backward stochastic differential equations : Discontinuous setting One of the directions who attracted many researchers is the case involving a discontinuity in the dynamics of the state solution. In this setting, the mean field BSDE is no longer driven by Brownian motion, but also by a random jump measure. Roughly speaking, the mean field backward stochastic equation with jumps has the following form on a filtered probability space (Ω, F, F, P)

on which the filtration F = (F t ) t satisfies the usual conditions of completeness and right continuity    dY s = -f (s, Y s , Z s , R s , P (Ys,Zs,Rs) )ds + Z s dW s + χ R s (e)π(ds, de),

Y T = ξ, (1.1.3)
where

• W = (W t ) 0≤t≤T is a Brownian motion with respect to its natural filtration under P.

• π(dt, de) is a random jump measure defined on IR + × χ where χ ⊂ IR l \{0} is non-empty open set equipped with its Borel field B(χ).

To be more precisely, π is the compensated random measure of π defined as following π(dt, de) = π(dt, de) -η(dt, de).

where the associated compensator η is absolutely continuous with respect to λ ⊗ dt (i.e) η(dt, de) = λ(de)dt.

for a σ-finite measure λ on (χ, B(χ)) satisfying

E (1 ∧ |e| 2 )λ(de) < ∞.
In [START_REF] Li | Mean-field forward and backward SDEs with jumps and associated nonlocal quasilinear integral-PDEs[END_REF], Li were the first interested by a mean field BSDE driven by a Browian motion and an independent Poisson random measure. The author study the existence and uniqueness of MF BSDE with jumps under following assumptions Assumption 1.1.2.

(i) f (•, •, 0, 0, 0, δ 0 ) ∈ H 2 F (0, T ).

(ii) f is Lipschitz with respect to (y, z, r, µ) ∈ IR×IR 2 ×L 2 (χ, B(χ), λ)×P 2 (IR d+1+d ×L 2 (χ, B(χ), λ)),

uniformly with respect to (s, w).

(iii) ξ ∈ L 2 (F T ).

Theorem 1.1.2. ( Li [83] )

Under the Assumption 1.1.2, the mean field BSDE with jumps (1.1.3) has a unique solution

(Y, Z, R) ∈ S 2 F (0, T : IR) × H 2 F (0, T : IR d ) × H 2 ν (0, T : IR).
In 2016, Tang and Meng [START_REF] Tang | Linear-quadratic optimal control problems for mean-field backward stochastic differential equations with jumps[END_REF] study linear quadratic optimal control problem for mean field backward stochastic differential equations driven by a Poisson random measure and a

Brownian motion which depend on the law of the state X and the control α. They established the existence and uniqueness of linear quadratic BSDE with jumps in the following form In 2016, Min and Li [START_REF] Li | Controlled mean-field backward stochastic differential equations with jumps involving the value function[END_REF] considered a new type of BSDEs namely MF BSDEs with jumps coupled with the value function. The authors proved that this type of BSDEs admits a unique solution, and established a comparison theorem as well as a dynamic programming principal.

             dY t = A t Y t + Ât E[Y t ] + B t α t + Bt E[α t ] + C t Z t + Ĉt E[Z t ] + χ D t R t (

Mean field control approach

Stochastic optimal control has been widely studied in recent decades due to its applications to mathematical finance, insurance, economics, engineering, etc. Recently, stochastic optimal control problems for mean field stochastic differential equations ( in short MF SDEs) have attracted more attention. The control problems of mean field type are stochastic optimal control problems where the cost function and the parameters of the dynamics depend on the law of the controlled stochastic process. In general, such problems correspond to the control of a large number of agents by a global planner. This theory is referred to mean field control (e.g., by

Bensoussan, Frehse, and Yam [START_REF] Bensoussan | Mean field games and mean field type control theory[END_REF]) or control of McKean Vlasov dynamics (e.g., by Carmona and Delarue [START_REF] Carmona | The master equation for large population equilibriums[END_REF]). It has found applications such as risk management, portfolio management, insurance, economics, engineering, see [START_REF] Alasseur | An extended mean field game for storage in smart grids[END_REF][START_REF] Basei | A weak martingale approach to linear-quadratic mckean vlasov stochastic control problems[END_REF][START_REF] Carmona | Applications of mean field games in financial engineering and economic theory[END_REF][START_REF] Graber | Linear quadratic mean field type control and mean field games with common noise, with application to production of an exhaustible resource[END_REF][START_REF] Matoussi | Mean-field backward-forward sde with jumps and storage problem in smart grids[END_REF][START_REF] Miller | Linear-quadratic mckean-vlasov stochastic differential games[END_REF].

To be more precisely, let us consider the following mean field optimal control, with the state following equation    dX t = b t (X t , P Xt , α t , P αt )dt + σ t (X t , P Xt , α t , P αt )dW t ,

X 0 = x, (1.1.5)
and the following cost functional J(α) := E T 0 e -ρt f t (X t , P Xt , α t , P αt )dt + e -ρT g(X T , P Xt ) .

(1. 1.6) The main goal in the control problem is to minimise the cost functional J over the set of admissible controls A defined for ρ ≥ 0 as,

A := {α : Ω × [0, T ] → IR m s.t α is IF -adapted and T 0 e -ρt E[|α t | 2 ]dt < ∞}.
An admissible control α * ∈ A is said the optimal control if

J(α * ) = inf
Bensoussan [START_REF] Bensoussan | Lectures on stochastic control[END_REF], Peng [START_REF] Peng | A general stochastic maximum principle for optimal control problem[END_REF]. The original version of Pontryagin's maximum principle was first introduced for deterministic control problems in the 1960's by Pontryagin et al. as in classical calculus of variation. The basic idea is to perturb an optimal control and to use some sort of Taylor expansion of the state trajectory around the optimal control, by sending the perturbation to zero, one obtains some inequality, and by duality.

In 2010, Buckdahn, Djehiche and Li [START_REF] Buckdahn | A general stochastic maximum principle for sdes of mean-field type[END_REF] were the first to be interested in the optimal control for stochastic differential equations of mean field type. They introduce a problem in which the coefficients depend on the expected value of the state of the solution process. The authors were establish a Peng-type general stochastic maximum principle, specifying a necessary conditions for optimality, for a general action space, using the classical spike perturbation. This maximum principle differs from the classical one in the sense that here the first order adjoint equation turns out to be a linear mean-field backward SDE, however, the second order adjoint equation remains the same as in Peng's stochastic maximum principle.

Buckadahn et al. [START_REF] Buckdahn | A stochastic maximum principle for general mean-field systems[END_REF] extended the approach of Peng in the classic framework by considering the second order variational equations and the corresponding second order adjoint process in the mean field setting. Many authors have made contributions on optimal control problems for mean field SDEs, see for example the works of Anderson and Djehiche [START_REF] Anderson | A maximum principle for sdes of mean-field type[END_REF] and Carmona and Delarue [START_REF] Carmona | The master equation for large population equilibriums[END_REF] under some convexity assumptions. We can also refer the reader to Alasseur et al. [START_REF] Alasseur | An extended mean field game for storage in smart grids[END_REF] where the authors develop a model for the optimal management of energy storage and distribution in a smart grid system through an extended MFG.

Recently, motivated by economic examples, Graber [START_REF] Graber | Linear quadratic mean field type control and mean field games with common noise, with application to production of an exhaustible resource[END_REF] studied mean field control problem for a linear stochastic differential equation with common noise. The author characterize the solution both in the term of a stochastic maximum principle and a Riccati equation. He reformulated the Nash equilibrium for MFG as an optimal control problem in order to discuss the case where a mean field Nash equilibrium is also the solution to an optimal control problem.

Dynamic programming approach Other approach to solve the mean field control problem is the dynamic programming approach. The classical dynamic programming principle is a fundamental principle in the theory of stochastic control. The basic idea of the approach is to consider a family of control problems by varying the initial state values, and to derive some relations between the associated value functions. This principle, called the dynamic programming principle were initiated in the 1950s by Bellman. This approach yields a certain partial differential equation (PDE in short), of second order and nonlinear, called Hamilton-Jacobi-Bellman (HJB in short). When this PDE can be solved by the explicit or theoretical achievement of a smooth solution, the verification theorem validates the optimality of the candidate solution to the HJB equation. This method is a weak formulation of the standard martingale optimality principle, where the verification theorem for stochastic control problem is a crucial step ( see e.g EL Karoui [START_REF] El Karoui | Les aspects probabilistes du controle stochastique: In 9th saint flour probability summer school-1979[END_REF] and Pham [START_REF] Pham | Continuous-time stochastic control and optimization with financial applications[END_REF]).

For mean field control problems, stochastic dynamic programming requires adaptation. Optimality conditions for mean field control problem are derived either by stochastic calculus of variation ( see e.g [START_REF] Anderson | A maximum principle for sdes of mean-field type[END_REF]) or by stochastic dynamic programming in the quadratic case ( see e.g [START_REF] Bensoussan | Mean field games and mean field type control theory[END_REF]),

but not in the general case for the fundamental reason that Bellman's principle does not apply in its original form on the stochastic trajectories of say X t if those depend also on its law. Therefore, the idea which is proposed by several researchers (Lions [START_REF] Lions | Cours au college de france[END_REF], Bensoussan et al. [START_REF] Bensoussan | The master equation in mean field theory[END_REF]), is to formulate the problem without any such restriction if we work with the probability measure of X t and use the Fokker-Planck equation.

In 2014, Lauriere and Pironneau [START_REF] Lauriere | Dynamic programming for mean-field type control, comptes rendus mathematique[END_REF] under some density assumption on the probability law of the state process, show that Bellman's principle applies to the dynamic programming value function V (τ, ρ τ ), where the dependency on ρ τ is functional as in Lion's analysis of mean field games. Later, Pham and wei [START_REF] Pham | Dynamic programming for optimal control of stochastic mckeanvlasov dynamics[END_REF] derive also the Bellman equation for mean field stochastic control problem, and they prove a verification theorem in a McKean Vlasov framework. The authors give an explicit solutions to the Bellman equation for the linear quadratic mean field control problem, with applications to the mean-variance portfolio selection and a systemic risk model.

Recently, Pham et al. [START_REF] Basei | A weak martingale approach to linear-quadratic mckean vlasov stochastic control problems[END_REF][START_REF] Miller | Linear-quadratic mckean-vlasov stochastic differential games[END_REF] propose an other approach for solving linear quadratic mean field stochastic control problems. The authors introduce the martingale formulation for verification theorem for solving a linear quadratic mean field stochastic problems. The optimal control involves the solution to a system of Riccati ordinary differential equations and to a linear mean field backward stochastic differential equation.

The verification theorem, the key of this approach, is introduced in the following form :

Theorem 1.1.4. (Finite horizon Verification Theorem [START_REF] Basei | A weak martingale approach to linear-quadratic mckean vlasov stochastic control problems[END_REF])

Let {W α t , t ∈ [0, T ], α t ∈ A} be a family of F-adapted process in this form W α t = w t (X α t , E[X α t ]) for some F-adapted random field {w t (x, x), t ∈ [0, T ], x, x ∈ IR d } satisfying

w t (x, x) ≤ C(X t + |x| 2 + |x| 2 ), t ∈ [0, T ], x, x ∈ IR d ,
for some positive constant C, and non-negative process X with sup t∈[0,T ] E[|X t |] < ∞, and such that (i) w T (x, x) = g(x, x), x, x ∈ IR d .

Introduction (ii) The map t ∈ [0, T ] -→ E[S α t ],
with

S α t = e -ρt W α t + t 0 e -ρs f s (X α s , E[X α s ], α t , E[α t ])ds,
is non-decreasing for all α ∈ A.

(iii) The map t -→ E[S α * t ] is constant for some α * ∈ A.

Then, α * is an optimal control and E[w 0 (X 0 , E[X 0 ])] is the value of the LQ MF control problem

(1.1.6)

V 0 = E[w 0 (X 0 , E[X 0 ])] = J(α * ).
Moreover, any other optimal control satisfies the condition (iii).

In [START_REF] Basei | A weak martingale approach to linear-quadratic mckean vlasov stochastic control problems[END_REF], Basei and Pham interested to solve a linear quadratic control problem. Roughly speaking, they consider a quadratic cost functional of the following type, where α ∈ A,

J(α) := E T 0 e -ρt f t (X α t , E[X α t ], α t , E[α t ])dt + e -ρT g(X T , E[X T ]) , (1.1.7)
where for each t ∈ [0, T ], x, x ∈ IR d and a, ā ∈ IR m f t (x, x, a, ā) := (xx) t Q t (xx) + xT (Q t + Qt )x + 2a

(i) β, γ ∈ L 2 F (Ω × [0, T ], IR d ), (ii) A, Ã, C, C ∈ L ∞ ([0, T ], IR d×d ), B, B, D, D ∈ L ∞ ([0, T ], IR d×m ).
Assumption 1.1.5. The coefficients in equation (1.1.8)-(1.1.9) satisfy:

(i) Q, Q ∈ L ∞ ([0, T ], S d ), ,P, P ∈ S d , N, Ñ ∈ L ∞ ([0, T ], S m ), I, Ĩ ∈ L ∞ ([0, T ], IR m×d ), (ii) M ∈ L 2 F (Ω × [0, T ], IR d ), H ∈ L 2 F (Ω × [0, T ], IR m ), L ∈ L 2 F (IR d ),
(iii) There exists ξ > 0 such that, for each t ∈ [0, T ],

N t ≥ ξI m , P ≥ 0, Q t + I T t N -1 t I t ≥ 0.
(iv) There exists ξ > 0 such that, for each t ∈ [0, T ],

N t + Ñt ≥ ξI m , P + P ≥ 0, (Q t + Qt ) -(I t + Ĩt ) T (N t + Ñt ) -1 (I t + Ĩt ) ≥ 0.

Remark 1.1.5. The uniform positive definite assumption on N and N + Ñ is a standard and natural coercive condition when dealing with linear-quadratic control problems.

Thanks to the weak formulation of verification theorem (see, Theorem 1.1.6) and some techniques, Basei and Pham have proved the existence of an optimal control: Theorem 1.1.6. (Basei and Pham [START_REF] Basei | A weak martingale approach to linear-quadratic mckean vlasov stochastic control problems[END_REF] )

Under Assumptions 1.1.4-1.1.5, there exists an optimal control α * for LQMF problem (1. 1.6) giving by the following explicit form α * t = -S -1 t U t (X * t -X * t ) -S -1 t (ϵ t -εt ) -S -1 t (V t X * t + Θ t ), ∀t ∈ [0, T ] P -a.s.,

where X * = X α * is the state process where the α * is the optimal control and the deterministic coefficients S, Ŝ U and V and the stochastic coefficients ϵ t and Θ t , are defined by

                       S t := N t + D ⊤ t K t D t , Ŝt := Nt + D⊤ t K t Dt , U t := I t + D ⊤ t K t C t + B ⊤ t K t , V t := Ît + D⊤ t K t Ĉt + Bt Λ t , Θ t := Ht + D⊤ t K t γt + D ZY t .
(1.1.11)

And (K, Λ, Y, Z Y , φ) ∈ L ∞ ([0, T ], S d ) × L ∞ ([0, T ], S d ) × S 2 F (Ω × [0, T ], R d ) × L 2 F (Ω × [0, T ], R d ) × χ, IR d ) × L ∞ ([0, T ], S d
), is the unique solution to the decoupled system (1.1.20):

                 dK t = . K t dt, 0 ≤ t ≤ T, K T = P, dΛ t = . Λ t dt, 0 ≤ t ≤ T, Λ = P , dY t = . Y t dt + Z Y t dW t , 0 ≤ t ≤ T, Y T = L, dφ t = .
φ, 0 ≤ t ≤ T, φ T = 0.

(1.1.12)

In [START_REF] Miller | Linear-quadratic mckean-vlasov stochastic differential games[END_REF], Pham and Miller propose an approach based on weak martingale optimality principle together with a fixed point argument in the space of controls for solving a stochastic differential game with linear McKean-Vlasov dynamics and quadratic cost functional. They characterize the Nash equilibria by a terms of systems of Riccati ordinary differential equations and linear mean field backward stochastic differential equations.

Main results and contributions

Motivation In this work, we are motivated by an energy market to produce energy from an exhaustible resource such as oil. Roughly speaking, each producer optimizes production rate that drives her revenue, as well as exploration effort to replenish her reserve. This exploration activity is modeled by a jumps process that leads to stochastic increments to reserves level. This model, represented by a model of production of exhaustible resource with accumulating or maintaining a level of reserves, is inspired by a series of works extended from the Hotelling's model [START_REF] Hotelling | The economics of exhaustible resources[END_REF]. In the classic Hotelling's model, the dynamic market evolution is driven by the use of existing reserves of an exhaustible reserves to produce energy without possibility to exploration and/or discovery of new reserves. However, many studies have made it possible to ensure that there are still resource bases to be explored over time, that is to say that the reservation rate can be increased. Particularly we can see the paper of Pindyck [107] that considered a deterministic model of exploring for a non-renewable resource. The author assumed exploration to be incremental and represented as a deterministic reserve addition. A series of works extended Prindyck's model [107] for the case of stochastic discoveries such as: Deshmukh et al. [START_REF] Deshmukh | Optimal consumption and exploration of nonrenewable resources under uncertainty[END_REF],

Arrow and Chang [START_REF] Arrow | Optimal pricing, use, and exploration of uncertain resource stocks[END_REF] and Keller et al. [START_REF] Hagan | Optimal pricing, use and exploration of uncertain natural resources[END_REF]. This increase in reserve discoveries occurs stochastically via the Point process, it should be noted that this increase is smaller, and it is the reason that the resources always remain exhaustible. In this work, we are particularly motivated by a continuous model of the production of an exhaustible resource recently introduced by Guèant et al. [START_REF] Guéant | Mean field games and applications. parisprinceton lectures 30 on mathematical finance[END_REF] where they have considered a mean field version of a this model with a quadratic cost function. Roughly speaking, the authors consider a large number of oil producers, which can be viewed either as wells or from a more macro standpoint as oil companies. Each oil producers i initially has a reserve x i 0 where are distributed according to an initial distribution. These reserves X i will of course contribute to production α i such that, for any specific producer i, his reserve dynamic is given by the following stochastic differential equation

   dX i t = -α i t dt + σX i t dW i t , X i 0 = x i 0 ,
where W i is Brownian motion specific to each producer. Each production need to optimize a profit criterion of the following form

max α i E ∞ 0 e -r t α i t p t -C(α i t )) dt , s.t. α ≥ 0, X t ≥ 0, (1.1.13)
where

• p is the selling price for producer i follows. The price p of producer i is related to his production and also to the production of all other producers.

• C is the cost functions of production, defined as

C(α) := δα + β α 2 2 .
In this work, we introduce a model of production of exhaustible resource with accumulating or maintaining a level of reserves. Reserves level X i t of each producer decreases at a controlled production rate α i ≥ 0, and also has random discrete increment due to exploration. We use a Poisson point process πi to model the new discoveries. To be more precisely, πi has intensity λ(t)a i t , where is the effort of the exploration controlled by player i. So the reserve dynamic of each producer is given by the following stochastic differential equation

   dX i t = -α i t dt + σX i t dW i t + η X X i t πi (de, dt), X i 0 = x i 0 , (1.1.14)
where x i 0 is the initial reserves level and η is the unit amount of discoveries . Each player i wants to maximize his own profit (1.1.15)

J(α) := E ∞ 0 e -ρ t α i t p t (α i t ) -C p (α i t ) -C ex (α i t , X i t ) dt , (1.1.15)
where • P i t is the selling price for producer i. It follows a linear inverse demand rule, defined as:

P i t := P t (α i t ) = P 0 t -δ α i t -ε t 0 1 N
• C p is the cost functions of production, defined as:

C p (α i t ) := c 1 Var(α i t ).
• C ex is the cost functions of exploration defined as:

C ex (α i t , X i t ) := c 2 α i t ( x i 0 -X i t x i 0 ).
with c 1 and c 2 are positive constants and represent respectively the cost of production and the cost of extraction where are the same for the all producers. In the following, we will propose a explicit method to solve this problem.

Our main result : Our first contribution presented in this thesis to study the solvability of linear quadratic stochastic control problem for mean field stochastic differential equation with jumps in the both finite and infinite horizon. The mean field LQ optimal control problem was initially studied by Yong [START_REF] Yong | Linear-quadratic optimal control problems for mean-field stochastic differential equations[END_REF] and was generalized later by Huang, Li and Yong [START_REF] Huang | A linear-quadratic optimal control problem for mean-field stochastic differential equations in infinite horizon[END_REF], Li, Sun

and Xiong [START_REF] Li | Linear quadratic optimal control problems for mean-field backward stochastic differential equations[END_REF] to variant cases. Along with the development of mean field LQ optimal control problems, the LQ differential game for mean field SDEs have also attracted extensive research, among which, we would like to mention Bensoussan et al. [START_REF] Bensoussan | Mean field games and mean field type control theory[END_REF] and Graber [START_REF] Graber | Linear quadratic mean field type control and mean field games with common noise, with application to production of an exhaustible resource[END_REF].

Here, we propose an approach to solve the linear quadratic control problems for mean field stochastic differential equation with jump defined as :

V (α * ) = inf α J(α), (1.1.16)
where

J(α) :=E T 0 e -ρt (x -x) t Q t (x -x) + xT (Q t + Qt )x + 2a T I t (x -x) (1.1.17) + 2ā T (I t + Ĩt )x + (a -ā) T N t (a -ā) + āT (N t + Ñ )ā + 2M T t x + 2H T t a, dt + e -ρT (x -x) T P (x -x) + xT (P + P )x + 2L T x , subject to          dX α t = ( β t + A t x + Ãt x + B t a + Bt ā)dt + (γ t + C t x + Ct x + D t a + Dt ā)dW t + X (δ t + F t (e)x + Ft (e)x + G t (e)a + Gt (e)ā)π(de, dt), X 0 = x, ( 1
(i) δ ∈ L 2 F (Ω × [0, T ], IR d ), (ii) G(.), G(.) ∈ K 2 ([0, T ] × χ, IR d×d ) , F (.), F (.) ∈ K 2 ([0, T ] × χ, IR d×m ).
Our approach is a weak formulation of the standard martingale optimality principle used in the verification theorem for stochastic control problems. Roughly speaking, using the Verification

α * t = -S -1 t U t (X * t -X * t ) -S -1 t (ϵ t -εt ) -S -1 t (V t X * t + Θ t ), ∀t ∈ [0, T ] P -a.s.,
where X * = X α * is the state process where the α * is the optimal control and the deterministic coefficients S, Ŝ U and V and the stochastic coefficients ϵ t and Θ t , are defined by

                                       S t := N t + D T t K t D t + χ G T t (e)K t G t (e)λ(de), Ŝt := Nt + DT t K t Dt + χ ĜT t (e)K t Ĝt (e)λ(de), U t := I t + D T t K t C t + B T t K t + χ G T t (e)K t F t (e)λ(
ϵ t := H t + D T t K t γ t + B T t Y t + D T t Z Y t + χ G T t (e)K t δ t λ(de) + χ G T t (e)R Y t (e)λ(de). (1.1.19) And (K, Λ, Y, Z Y , R Y , φ) ∈ L ∞ ([0, T ], S d )×L ∞ ([0, T ], S d )×S 2 F (Ω×[0, T ], R d )×L 2 F (Ω×[0, T ], R d )× K 2 (Ω × [0, T ] × χ, IR d ) × L ∞ ([0, T ], S d ),
is the unique solution to the following decoupled system:

                   dK t = . K t dt, 0 ≤ t ≤ T, K T = P, dΛ t = . Λ t dt, 0 ≤ t ≤ T, Λ = P , dY t = . Y t dt + Z Y t dW t + χ R Y t (e)π(de, dt), 0 ≤ t ≤ T, Y T = L, dφ t = . φ, 0 ≤ t ≤ T, φ T = 0. (1.1.20)
By the theory developed above, we will solve a control problem for a model of production of exhaustible resource with accumulating or maintaining a level of reserves.

In our work, we assume that the common price P 0 t = p 0 , for all t ≥ 0 where p 0 is interpreted as a substitute price for the exhaustible resource. In the next proposition, we gives an explicit solution to the problem (1.1.15). Proposition 1.1.8. We assume that p 0 = c 2 +εx 0 for all t ≥ 0, x 0 is large enough and 1 > ρ 2 + 2ε δ . Then the solution of (1.1.15) is given by

α * t = K c 1 ,c 2 X * t -+ (Λ ε -K c 1 ,c 2 ) X * t -.
Finally, we discussed two different cases: The first one, when p 0 = c 2 + εx 0 i.e. p 0 coincides with c 2 + εx 0 which is the cost of extraction for the last unit of resource. The second case when

p 0 < c 2 + εx 0 (i.e.
) the Hotelling rent is negative. It is not obvious to check the positivity of the the state process and the optimal strategy. We study the stationary level of the reserve and the optimal production rate in mean.

Numerical approach for a class of mean field games

The second part of this thesis is dedicated to the numerical resolution of a mean field game problem. We will introduce an approach, called Markov chain approximation ( MCA in short)

where was developed by H. Kushner and P.G. Dupuis [START_REF] Kushner | Numerical methods for stochastic control problems in continuous time[END_REF]. This method applies a Markov chain approximation to continuous time and continuous space state control problem. This chapter is based on co-work with Ben Aziza [START_REF] Ben Aziza | Numerical approach for mean field games[END_REF].

Mean field game approach

The theory of mean field games( MFG in short) is a recent branch of dynamic games that aims at modeling and analyzing complex decision processes involving a large number of agents, which have individually a small influence on the overall system, and are influenced by the behavior of other agents. The MFG is presented by a control problem that approximate a large number of players in interaction. It's devoted to mathematically analyze a problem where there are large number of small players, who have similar references. The theory was introduced since 2006 in series of seminal papers, by Lasry and Lions [START_REF] Lasry | Jeux à champ moyen. i-le cas stationnaire[END_REF][START_REF] Lasry | Jeux à champ moyen. ii-horizon fini et contrôle optimal[END_REF][START_REF] Lasry | Mean field games[END_REF], Caines et al. [START_REF] Caines | Large-population cost-coupled lqg problems with nonuniform agents: individual-mass behavior and decentralized nash equilibria[END_REF]. Since its inception, the mean field games theory has expanded tremendously, and has become an important tool in the study of dynamical and equilibrium behavior of large systems.

Let us consider a game of a N players who act in terms of the strategies of the other players. The players are in interaction taking into account the global behaviors of the flow. Each player takes his decision in a strategic way considering the decisions of the others presented through the law distribution of the field.

Let X i be the state of player i, i = 1, ..., N , which satisfies the following stochastic differential

equation dX i t = -α i t dt + √ 2σdB i t , X i 0 = x i ,
where σ is a fixed positive constant, (B i t ) 1≤i≤N are N Brownian Motion independents and α i the control associated to the player i. Each individual player i will need to choose his strategy α i to minimize his cost functional J i over the period [0, T ], which is influenced by the state of other players:

J i (α i , (α j ) i̸ =j ) := E[ T 0 (L(s, X i s , α i s , m i s )ds + G(X i T , m i T )], (1.2.21)
where g is the terminal cost and L is measurable and locally bounded functions which satisfies the coercivity condition

lim |α|→∞ |L(t, X t , α t , m t )| |α| = +∞, with m i i = i̸ =j δ X j t
is the probability measure of other players and δ is the Dirac measure.

For a fixed number N of players, the player i solves his optimization problem assuming that the other players have already made their decisions, in order to find the Nash equilibrium.

Definition 1.2.1. (Nash equilibrium)

(α 1 , ..., α N ) is a Nash equilibrium of the system of N players if J(α i , (α j ) i̸ =j ) ≤ J(α, (α j ) i̸ =j ), for all i ∈ {1, ..., N } and α ∈ A.

The Nash equilibrium is thus such that no player regrets his choice in view of the choice of the others, he could not have done better. Often Nash equilibrium is presented as a situation where each player adopts the best answer "given" the choice of the others.

Let us introduce a Hamiltonian system of N players satisfy the following equation :

H(x, m, p, m) = sup α∈IR d {-pα -L(t, x, α, m)},
and define the Nash system of N players by the following equation

     -∂ t u i -ν∆u i + H(x i , m i , Du i ) = f (x i , m i t ), u(x i , T ) = ϕ(x i , m i T ). (1.2.22)
When a solution to the above system exists and is regular, a Nash equilibrium is given by the control functions

α i = -H i p (t, x i , ∇ x i u i , m i ).
Then, when players use these controls, u i is called the value function of player i, and represents the lowest average cost that player i can achieve between times t and T if its state is given by

x at time t. The system (1.2.22) is a system of N coupled Hamilton-Jacobi equations. We refer the reader to the work of Crandall and Lions, for a thorough study of the Hamilton-Jacobi equations and the framework in which they are well posed the theory of viscosity solutions and to the work of A. Bensoussan and J. Frehse [START_REF] Bensoussan | Control and nash games with mean field effect[END_REF][START_REF] Bensoussan | Mean field games and mean field type control theory[END_REF]. The main difficulty in the study of the system (1.2.22) consists in the fact that a player, when he changes his strategy, affects all the other players. Thus, the existence and uniqueness of solutions of (1.2.22) are in general complicated problems. However, when the number of players is large, any qualitative or quantitative study of the behavior of the solutions of (1.2.22) becomes impossible because of the number of equations and their strong coupling. Contrary to what the previous remark suggests, the structure of Nash equilibrium should simplify when the number of players becomes very large since the effect of each player on the system becomes negligible. This is the founding idea of the theory of mean field games, developed at the same time in the initial independent works of Lasry and

Lions [START_REF] Lasry | Jeux à champ moyen. i-le cas stationnaire[END_REF][START_REF] Lasry | Jeux à champ moyen. ii-horizon fini et contrôle optimal[END_REF][START_REF] Lions | Cours au college de france[END_REF] and of Huang, Caines and Malhamé [START_REF] Caines | Large-population cost-coupled lqg problems with nonuniform agents: individual-mass behavior and decentralized nash equilibria[END_REF].

The MFG problem is mathematically described through a system of two non linear partial differential equations introduced by a following system

                 -∂ t u -ν∆u + H(x, m, Du) = f (x, m), in T d × (0, T ), ∂ t m -ν∆m -div(D p H(x, m, Du)m) = 0, in T d × (0, T ), m(0) = m 0 , u(x, T ) = ϕ(x, m(T )), (1.2.23)
where ν is a non-negative parameter, and m 0 and m t are the probability or the density.

The Hamiltonian

H : (x, p) ∈ R d ×IR → H(x, p) ∈ IR is convex function with respect to p defined by H(x, m, p) = sup α {-pα -L(x, α)}.
This is a forward-backward system. The first equation is called a Hamilton-Jacobi equation where is associated with an optimal control problem. This equation describes the reaction of players to the mass. It presents the decisions based on where you want to be in the future. The second one is called a transport or Fokker-Planck equation for the distribution of the agents. The equation describes the aggregation of the action of all players. It presents where the population behavior actually ends up based on the initial distribution. The existence of solutions of HJB-FP system (1.2.23) is treated in many works [START_REF] Bensoussan | Control and nash games with mean field effect[END_REF][START_REF] Caines | Large-population cost-coupled lqg problems with nonuniform agents: individual-mass behavior and decentralized nash equilibria[END_REF][START_REF] Camilli | A semi-discrete approximation for a first order mean field game problem[END_REF][START_REF] Carmona | Applications of mean field games in financial engineering and economic theory[END_REF][START_REF] Carmona | Probabilistic Theory of Mean Field Games with Applications I-II[END_REF][START_REF] Guéant | New numerical method for mean field games with quadratic costs[END_REF][START_REF] Lasry | Jeux à champ moyen. i-le cas stationnaire[END_REF]. A uniqueness criterion for the solution of the field game system was presented by J.M. Lasry and P.L. Lions in their first work [START_REF] Lasry | Jeux à champ moyen. i-le cas stationnaire[END_REF] on the theory of mean field games called Lasry-Lions monotonicity condition defined as :

(f (x, m) -f (x, m ′ )d(m -m ′ )(x) ≥ 0 ∀m, m ′ ∈ P, and
(ϕ(x, m) -ϕ(x, m ′ )d(m -m ′ )(x) ≥ 0 ∀m, m ′ ∈ P.
The interpretation of the above Monotonicity conditions is that the players dislike congested regions and prefer configurations in which they are scattered.

Many works have treated this system in the periodic setting, for example [START_REF] Cardaliaguet | Notes on mean field games[END_REF][START_REF] Cardaliaguet | The master equation and the convergence problem in mean field games[END_REF][START_REF] Lasry | Jeux à champ moyen. i-le cas stationnaire[END_REF][START_REF] Lasry | Mean field games[END_REF][START_REF] Lions | Cours au college de france[END_REF].

They establish the existence result of classical solution under a wide range of sufficient conditions on H, f and ϕ, and the uniqueness result under the uniform convexity of the Hamiltonian and Lasry-Lions monotonicity condition. Here, we consider a mean field game model where any players controls its private state X t at time t, by taking an action α t in a closed convex subset A, where the dynamic of state is driven by the following the stochastic differential equation

dX t = α t dt + √ 2σdB t , (1.2.24)
where B t is a standard Brownian motion, σ is a fixed positive constant and A is the set of admissible control.

Each player will need to choose his strategy to minimize his cost functional (1.2.25) over the period [0, T ], which is influenced by the state of other players:

J(x, t, α, m) = E[ T t (L(X α s , α s ) + F (X α s , m s ))ds + Φ(X α T , m T )]. (1.2.25)
The problem of differential games with an infinite number of agents is articulated in the following form:

(i) for each fixed deterministic flow of probability measures m on IR d , solve the standard stochastic control problem:

V (x, t) := inf α∈A J(x, t, α, m). (1.2.26) (ii) Find a flow m such that L( Xt ) = m t for all t ∈ [0, T ],
where Xt is a solution to the above SDE (1.2.24).

We can see that the first step provides the best response of a given player interacting with the statistical distribution of the states of the other players if this distribution m is assumed to be given by t, when the second step solves a fixed point problem in the goal of the search for fixed points of the best response function. Once these two steps have been taken successfully, if the fixed-point optimal control α identified in step (ii) is in feedback form α * t = ϕ(t, X α t , m t ), for some deterministic function ϕ on [0, T ] × T d × P(IR d ). Then, if the players use α * t , the mean field equilibrium {m t } 0≤t≤T with value in P(K) should coincide with the law of X and satisfy the Fokker-Planck equation.

The literature MFG has grown considerably: many authors work on this subject, see : Cardaliaguet [START_REF] Cardaliaguet | Notes on mean field games[END_REF], Bensoussan et al. [START_REF] Bensoussan | Mean field games and mean field type control theory[END_REF], Carmona and Delarue [START_REF] Carmona | Probabilistic Theory of Mean Field Games with Applications I-II[END_REF], Gomes et al. [START_REF] Gomes | Mean field games models-a brief survey[END_REF][START_REF] Gomes | On the existence of classical solutions for stationary extended mean field games[END_REF][START_REF] Gomes | A-priori estimates for stationary mean-field games[END_REF]. A lot of MFG problems have explicit or semi-explicit solutions, but it's not always obvious or easy to solve a coupled MFG PDE system. Therefore, numerical simulations of MFGs play a crucial role to solve this models.

Numerical methods

Numerical resolution has been interested by many authors and there is still an active on going research dealing with the numerical approximation of different types of MFG systems most of them are based on the PDE system introduced by Lions and Lasry in [START_REF] Lasry | Mean field games[END_REF]. Finite difference methods has been the first method of approximation the PDE system whose proposed by Achdou et al. in [START_REF] Achdou | Mean field games: Numerical methods[END_REF]. They propose a finite difference scheme with monotone approximations of the Hamiltonian and a discrete weak formulation of the Fokker-Planck equation, both in finite and infinite horizon.

To introduce the approximation of HJB-FP system (1.2.23) proposed by Achdou et al. [START_REF] Achdou | Mean field games: Numerical methods[END_REF], let us consider two positive integers N T and N h and ∆t = T /N T and h = 1/N h and t n = n × ∆t,

x i = i × N h for (n, i) ∈ {0, ..., N T } × {0, ..., N h }.
We approximate u and m respectively by vectors U and M such that for each (n, i) ∈ {0, ..., N T } × {0, ..., N h },

u(t n , x i ) ≈ U h i,n , and, m(t n , x i ) ≈ M h i,n .
Remark 1.2.2. To simplify the notation, we present the scheme in the framework of a one dimension.

Finite difference operators :

Let us introduce the elementary finite difference operators:

D t U h i,n := U h i,n+1 -U h i,n ∆t , for 0 ≤ i ≤ N T -1, DU h i,n := U h i+1,n -U h i,n h , for 1 ≤ i ≤ N h -1, ∆ h U h i,n = - 2U h i,n -U h i+1,n -U h i-1,n h 2 , for 1 ≤ i ≤ N h -1, (1.2.27) ∇ h U h i,n := ((DU ) i , (DU ) i-1 ) T , for 0 ≤ k ≤ N h -1.
We consider H the discrete Hamiltonian and assume that it satisfy the following Assumptions: The continuous Hamiltonian H will be approximated by H at (x i , t k ). Then, the discret HJB equation will be in the following form:

-D t U h i,n -ν∆ h U h i,n + H(x i , ∇ h U h i,n ) = f h (x i , M h i,n+1 ), U h i,Nt = ϕ h (M h i,Nt ),
where the operators f h satisfies the following Assumption:

Assumption 1.2.2.
1. We assume that f h is continuous on K h .

2. The numerical cost f h is monotone in the following sense:

(f h (m k ) -f h (m k ), m k -m k ) 2 ≤ 0, ⇒ f h (m k ) = f h (m k ). (1.2.

28)

3. There exists a constant C independent of h such that for all times t and for all grid function

m ∈ P(K h ), ||f h [m]|| ∞ ≤ C, and |(f h [m]) i,j -(f h [m]) k,l | ≤ Cd T (x i,j , x k,l ),
where d T (x, y) is the distance between the two points x and y in the torus T 2 .

4. For all sequence m h , there exists a continuous and bounded function ω :

IR + → IR + such that ω(0) = 0 and ||f h [m] -f h [m h ]|| L ∞ (G h ) ≤ ω(||m -m h || L 1 (T 2 ) ).
The main idea in [START_REF] Achdou | Mean field games: Numerical methods[END_REF] is to discretise also the Fokker-Planck equation as the HJB equation based on a weak formulation. Let us consider a smooth test function Φ ∈ C ∞ ([0, T ]×T)" which involve the following expression

- T ∂ x (H p (x, ∂ x u(t, x))m(t, x))Φ(t, x)dx = T H p (x, ∂ x u(t, x)∂ x Φ(t, x)dx, (1.2.29)
where they used an integration by parts and the periodic boundary conditions. In view of that precedes, it is quite natural to propose the following discrete version of the right hand side of (1.2.29)

h N h -1 i=0 M h i,n+1 H(x i , ∇ h U h i,n ) Φ i+1,n -Φ i,n h + H(x i , ∇ h U h i,n ) Φ i,n -Φ i-1,n h . (1.2.30)
Performing a discrete integration by parts, we obtain

-h N h -1 i=0 T i (U h i,n , M h i,n+1 )Φ i,n ,
where T i is a discrete transport operator

T i (U h , M h ) = 1 h (M h i,n Hp 1 (x i , ∇ h U h i,n ) -M h i-1,n Hp 1 (x i-1 , ∇ h U h i-1,n ) + M h i+1,n Hp 2 (x i+1 , ∇ h U h i+1,n ) -M h i,n Hp 2 (x i , ∇ h U h i,n ).
Then, the Fokker-Planck equation can be approximated in a short way by:

-D t M h i,n -ν∆ h M h i,n+1 + T i (U h i,n , M h i, n + 1) = 0, M h i,0 = m0 (x i ).
Summarizing, the discrete problem is to look for two grid functions U and M on G h,t :

             -D t U h i,n -ν∆ h U h i,n + H(x i , ∇ h U h i,n ) = f h (x i , M h i,n+1 ), -D t M h i,n -ν(∆ h M h i,n+1 ) i + T i (U h i,n , M h i,n+1 ) = 0, U Nt i = ϕ h (M Nt i ), M h i,0 = m0 (x i ).
(1.2.31)

The uniqueness for the discretized problems can be obtained by similar arguments as those used in the continuous case, using Lasry-Lions monotonicity condition

The convergence of this scheme was discussed by Achdou et al. in [START_REF] Achdou | Mean field games: Numerical methods[END_REF] where they consider a particular case. They considered an Hamiltonian has the following form H(x, p) = H(x) + |p| β for β > 1 and H is continuous function. The first result of convergence theorems [START_REF] Achdou | Mean field games: convergence of a finite difference method[END_REF][START_REF] Achdou | Mean field games: Numerical methods[END_REF] make some assumption that the MFG system of PDEs has a unique classical solution and a strong versions of Lasry-Lions monotonicity assumptions. Under this assumptions, the solution of the discrete system converges towards the classical solution as the grid parameters tend to zero.

In [START_REF] Achdou | Convergence of a finite difference scheme to weak solutions of the system of partial differential equations arising in mean field games[END_REF], Achdou and Porretta showed that the solutions of the discrete system converges to a weak solution of the forward-backward system. In [START_REF] Lachapelle | Computation of mean field equilibria in economics[END_REF], Lachapelle, Salomon and Turinici provided an iterative scheme using a discrete Markov decision problem. They proved that as the grid steps tend to zero, the solution of the discretized MFG system con verges to a weak solution.

Recently, Chassagneux, Carmona, Delarue and the others [START_REF] Angiuli | Numerical Probabilistic Approach to MFG[END_REF] have introduced two algorithms for numerically solving FBSDEs of McKean Vlasov type, which can be used to formulate the solutions to mean field game problems. The first algorithm, which is based on the paper of Chassagneux, Crisan, and Delarue [START_REF] Chassagneux | Numerical method for fbsdes of mckean-vlasov type[END_REF], relies on a tree structure to represent the pathwise law of the solution. The second algorithm takes the algorithm presented in the paper of Delarue and Menozzi [START_REF] Delarue | A forward-backward stochastic algorithm for quasi-linear PDEs[END_REF] for solving FBSDEs and extends it to the mean field framework.

In recent work, Bayraktar, Budhiraja, and Cohen tebayraktar2017numerical used the Markov chain approximation method to construct an approximation for the MFG with reflecting barriers.

They formulate the MFG model in terms of a controlled reflected diffusion with a cost function and study the convergence of their numerical scheme is established for a small time horizon T.

The idea is that for a fixed measure on the path space, they define a Markov decision problem after a time and space discretization. The authors proved that the discrete numerical scheme is an almost contraction. Roughly speaking, the solution of the discrete MFG considered can be seen as the solution of a fixed point problem on the space of probability measures on certain path space. The Markov chain approximation is recently used by Ben Aziza and Toumi [START_REF] Ben Aziza | A probabilistic numerical method for a class of mean field games[END_REF] to solve the system based on the construction of a discrete mean field game where the controlled state is a Markov chain approximatting the controlled SDE. Our work can be seen as the generalization of [START_REF] Ben Aziza | A probabilistic numerical method for a class of mean field games[END_REF]. For more results on the numerical methods for mean field games, we refer the reader to [START_REF] Achdou | Mean field games: convergence of a finite difference method[END_REF][START_REF] Achdou | Mean field games: Numerical methods[END_REF][START_REF] Angiuli | Numerical Probabilistic Approach to MFG[END_REF][START_REF] Carlini | A fully discrete semi-lagrangian scheme for a first order mean field game problem[END_REF][START_REF] Carlini | A semi-lagrangian scheme for a degenerate second order mean field game system[END_REF][START_REF] Guéant | New numerical method for mean field games with quadratic costs[END_REF].

Main results and contributions

Let us consider a mean field game model where any players controls its private state X t at time t, by taking an action α t in a closed convex subset A, where the dynamic of state is driven by the following the stochastic differential equation

dX α t = b(t, X α t , α t )dt + σdB t , X α 0 ∼ m 0 , (1.2.32)
where B t is a standard Brownian motion, σ is a fixed positive constant and A is the set of admissible control control.

In this work, we introduce a Markov chain approximation method. This approach was inspired from the works of Kushner et al. [START_REF] Kushner | Numerical methods for stochastic control problems in continuous time[END_REF][START_REF] Kushner | Numerical methods for stochastic control in continuous time[END_REF]

p(ξ h k , η|α k ) = P(ξ h k+1 = η|ξ h j , α j , j ≤ k), (1.2.33)
for transitions of the Markov chain from stage ξ h k to the stage ξ h k+1 under control policy α k . These transition probabilities must satisfy the non-negativity p > 0 and conservation p = 1.

The Markov chain transition probabilities p(ξ h

k , η|α k ) are constructed from the finite difference approximation of HJB equation. Its law ν n at time n is defined by ν n := ν n+1 p. By iterating this equality, we obtain for any n ≥ 1,

ν n = P n ν 0 ,
where ν 0 is the initial distribution. Roughly speaking, we start by approximate the HJB equation using difference finite method as follows:

-( U i,k+1 -U i,k ∆t ) - 1 2 σ 2 ( U i+1,k -2U i,k + U i-1,k h 2 ) + H(x, U i+1,k -U i-1,k 2h ) = F (M h i,k , x i ), (1.2.34)
with

H(x i , U i+1,k -U i-1,k 2h ) = inf α {b(t k , x i , α i,k ).( U i+1,k -U i-1,k 2h ) + L(x i , α i,k )}, (1.2.35)
where we assume the following boundary conditions

U -1,k = U 0,k , U N h x +1,k = U N h x ,k , 1 ≤ i ≤ N h x . (1.2.36)
Therefore, it leads to the following approximation of HJB equation

               U i,k-1 = inf α∈A h L h (x i , α i,k ) + F h (x i , M h i,k ) dt + U i,k 1 -σ 2 ∆t h 2 +U i-1,k σ 2 2 ∆t h 2 + ∆t 2h b(x i , α i,k ) +U i+1,k σ 2 2 ∆t h 2 - ∆t 2h b(x i , α i,k ) , U i,N h t = Φ h (x i , M h i,N h t ), 0 ≤ i ≤ N h x .
(1.2.37)

Then, from the finite differential approximation (1.2.37), we define a transition probabilities such that at each time

t k , p h (t k , ν i,k , α i,k ; k, k + 1) = P h (k, α i,k
) is a matrix of transition probabilities for the approximated Markov Chain defined as follows:

P h i,i-1 (k, α i,k ) = σ 2 2 ∆t (h) 2 + ∆t 2h b(x i , α i,k ), 1 ≤ i ≤ N h x -1, (1.2

.38)

P h i,i (k, α i,k ) = 1 -σ 2 ∆t h 2 , 1 ≤ i ≤ N h x -1, P h i,i+1 (k, α i,k ) = σ 2 2 ∆t h 2 - ∆t 2h b(x i , α i,k ), 1 ≤ i ≤ N h x -1.
In this work, we define the law associated to discrete Markov chain problem M h from its transition probabilities as follows Then there exist a unique solution of the system (1.2.37)-(1.2.39).

M h * ,0 = π h 0 , M h * ,k = (P h (k, α * k )) t M h * ,k-1 , for 1 ≤ k ≤ N h t , ( 1
Based on the results of Billengsley [START_REF] Billingsley | Convergence of probability measures[END_REF], our second main result in this part is the following convergence Theorem 1.2.5:

Theorem 1.2.5. Let (M h ) h be the discrete MFG equilibrium and ξ h being its associated optimally controlled Markov chain. Let (m h ) h ∈ P(D[0, T ]) be the law of the continuous-time-space piecewise constant interpolated process ξh .

Then, the sequence (m h ) h≥0 converges weakly to some m in P(D[0, T ]) where m is an equilibrium of MFG problem.

To prove the convergence result, we need the results in Kushner and Dupuis [START_REF] Kushner | Numerical methods for stochastic control problems in continuous time[END_REF] for weak convergence of time-continuous controlled Markov chains on the space of càdlàg processes D[0; T ] which is a separable and complete for the Skorohod topology. We need to add some boundary condition, to force the state variable X to stay in the domain. In fact, with the local reflection direction, if the chain tries to leave the state space then it is returned immediately in space.

It will be also necessary to introduce a new class of control, the relaxed control which provide a very powerful tool in the study of the convergence properties of sequences of optimally controlled processes. The idea of relaxed control is to replace the A-valued process α t with a P (A)-valued process α R t , where P (A) is the space of probability measures equipped with the topology of weak convergence. According to the result of Kushner and Dupuis in [START_REF] Kushner | Numerical methods for stochastic control problems in continuous time[END_REF] and for any m ∈ P , we get

inf α R, * J(α R , m * ) = inf α J(α, m * ).
The last part of this work is dedicated to a comparison between an almost explicit simulated solution and the numerical solution corresponding to the convergent discrete MFG solution for a linear model.

Learning dynamic utilities

The third part considers the task of learning dynamic preferences of an agent from observed sequence of his decisions at discrete dates (possibly random) (τ i ) i . There are many settings in which this capability is very useful. In cooperative setting, we want to help the agent make good decisions. This is the principle of robo-advisors which are online investment management services that employ mathematical algorithms to provide financial advice with minimal human intervention. Based on online questionnaires to obtain information about the client's degree of risk-aversion, financial status, and desired return on investment, the robo-advisors use different algorithms to manage and allocate client assets in the most possible efficient way according to his preferences. In a more competitive setting, we want to predict the agent's actions in order to optimize better our own payoffs, this is the case of two-players A and B, where A is trying to learn the utility of B, to optimize his/her own payoff which also depends on the actions and utility of B. This chapter based on co-work with Mrad [START_REF] Mrad | Learning dynamic utilities[END_REF].

is the consumption process. Since then, the theory has grown in interest under the pressure of the economic reality, which created news incentives for different approaches, see Chambers & Echenique [START_REF] Chambers | Revealed preference theory[END_REF]. An example is the evolutionary economics by Arthur [START_REF] Arthur | Complexity and the economy[END_REF]. Economies are considered as complex evolutionary systems, where the agents try to predict the outcomes of their actions, and how the market would be modified by their decisions.

Beyond the fields of economics and finance, in the last few decades this forward-looking viewpoint has been also considered in many optimization problems as a consequence of the expansion of computer science and e-commerce. The forward modeling allows anticipations on the future values of observables. But, the agents also need to adjust their (random) preferences over time, following an "inverse thinking" approach as has been suggested by Gomez-Ramirez [START_REF] Gomez-Ramirez | Don't blame the economists. it is an inverse problem![END_REF].

In the field of machine learning, we refer for example to Friedman and Sandow [START_REF] Fürnkranz | Preference learning[END_REF], Chajewska, Koller, and Ormoneit [START_REF] Chajewska | Learning an agent's utility function by observing behavior[END_REF]: and Stahl [START_REF] Stahl | Approximation of utility functions by learning similarity measures[END_REF] and Hibbard [START_REF] Hibbard | Model-based utility functions[END_REF]. As well, reasoning with preferences has been recognized as a particularly promising research direction for artificial intelligence see Nielsen and Jensen. [START_REF] Nielsen | Learning a decision maker's utility function from (possibly) inconsistent behavior[END_REF] and Qi, Xu, and Lafferty. [START_REF] Qi | Learning high-dimensional concave utility functions for discrete choice models[END_REF]. For e-commerce, see [START_REF] William | Utility functions in autonomic systems[END_REF]: The principal idea is to build the utility function of a user from his search history and purchases on the Internet. This utility will then be used to target the user by proposing products or baskets of goods that we know are likely to be of interest to him and at specific prices evaluated from this utility function.

Before presenting our learning approach that allow the robo-advisor to predict the agent's utility, we first introduce the concept of dynamic utilities and recall some of the main results in this theory which we will use extensively in this work.

In the classical framework, the utilities are chosen deterministic independently of the the investment universe (very dynamic). In fact, in this classical portfolio optimization program, the investor set at time t = 0 his utility U for maturity T , he then optimizes his expected utility over all admissible strategies and find the associated optimal policy and then the optimal portfolio. The problem is that this optimal process is strongly dependent on the maturity T and it is found independently of the market evolution and do not take into account the new investment constraints imposed on this agent. Actually, this is not consistent with the periods of crisis that have been quite frequent in recent years and that can significantly influence the choices and preferences of any agent with regard to products that become much too risky or more profitable. Moreover, this optimal portfolio is not time consistent: if the investor decides to continue investing until a new maturity T ′ > T , the two portfolios obtained for two different maturities do not usually coincide at T when they should (because they represent the same preferences). In order to rectify this inconsistency, we have no choice: by the principle of dynamic programming, the utility must be a stochastic process to allow a flexibility to modify our strategy if it is necessary and to readjust the agent preferences over time.

In 2002, Musiela and Zariphopoulou [START_REF] Musiela | Backward and forward utilities and the associated pricing systems: The case study of the binomial model[END_REF] introduced the notion of "Forward utility" which are dynamic, progressive utilities, consistent with a given financial market if they satisfy some given conditions (see Definition 1.3.2 below) . A dynamic utility is a random field U (t, z, ω) adapted to the available information, and at each time t, it is a standard utility function. See [START_REF] Musiela | Backward and forward utilities and the associated pricing systems: The case study of the binomial model[END_REF][START_REF] Musiela | Portfolio choice under space-time monotone performance criteria[END_REF][START_REF] Musiela | Stochastic partial differential equations and portfolio choice[END_REF] for more detailed discussions and motivations. Aprogressive utility gives an adaptative way to include new information on environment evolution available to economic agents. Since these utilities are stochastic, time dependent and moving forward, we consider them as a family of semi-martingales depending on a parameter, the wealth of the agent in the economic context.

A dynamic utility should represent, possibly changing over time, individual preferences of an agent starting with a today's specification of his utility, U (0; z) = u(z). The preferences are affected over time by the available information represented by the filtration (F t ) t≥0 defined on the probability space (Ω, P, F). The filtered probability space (Ω, P, (F t ) t≥0 ) is assumed to satisfy usual conditions of right continuity and completeness. The filtration F 0 is not necessarily assumed to be trivial, so that the initial condition U (0, z) is not necessarily a deterministic function.

We start by introducing the definition of a progressive utility as a progressive random field with concavity property: Definition 1.3.1 (Progressive Utility). A progressive utility is a càdlàg progressive random field on , U = {U (t, z); t ≥ 0, z > 0} such that,

• UTILITY PROPERTY: U is strictly concave, strictly increasing, and non-negative.

• REGULARITY PROPERTY: U is a C 2 -random field, with continuous first and second derivatives random fields U z and U zz .

• INADA CONDITIONS: U goes to 0 when x goes to 0 and the derivative U z goes to ∞ when z goes to 0, and to 0 when z goes to ∞.

The notion of progressive utility is very general and should be specified so as to represent more realistically the dynamic evolution of the individual preferences of an investor in a given financial market. Once his consistent progressive utility is defined, an investor can then turn to a portfolio optimization problem in a larger financial market or to calculate indifference prices.

Musiela and Zariphopoulou [START_REF] Musiela | Investment and Valuation Under Backward and Forward Dynamic Exponential Utilities in a Stochastic Factor Model[END_REF][START_REF] Musiela | Stochastic partial differential equations and portfolio choice[END_REF] introduced the following definition of a consistent dynamic utility: Definition 1.3.2 (Consistent dynamic utility). A consistent dynamic utility U = {U (t, z); t ≥ 0, z > 0} is a progressive utility with the following additional properties:

Consistency with the test-class: For any admissible wealth process X,

E(U (t, X t )/F s ) ≤ U (s, X s ), ∀s ≤ t a.s.
Existence of optimal wealth: For any initial wealth x > 0, there exists an optimal wealth process X * such that X * 0 = x, and for all s ≤ t,

U (s, X * s ) = E(U (t, X * t )/F s ) ∀s ≤ t a.s.
In short for any admissible wealth X, U (., X . ) is a positive supermartingale and a martingale for the optimal-benchmark wealth X * .

Motivated by Zariphoupolou et al [START_REF] Musiela | Backward and forward utilities and the associated pricing systems: The case study of the binomial model[END_REF], El Karoui and Mrad [START_REF] El Karoui | An Exact connection between two Solvable SDEs and a Non Linear Utility Stochastic PDEs[END_REF] study and fully characterize the consistent dynamic utility in an Itô incomplete market. The key idea of [START_REF] El Karoui | An Exact connection between two Solvable SDEs and a Non Linear Utility Stochastic PDEs[END_REF] is to study in details the marginal utility U x (t, x) instead of U (t, x) and to consider at the same time the dual problem.

They establish the fully non-linear stochastic partial differential equation (SPDE) satisfied by the marginal utility and link it with the SDEs that satisfy the optimal primal and dual processes.

Under some regularity assumption, the authors give sufficient conditions ensuring concavity, monotonicity, differentiability both for random field and for solution of SDE. They establish a dynamics of consistent utilities and their convex conjugate and a fully characterization of the marginal utility. In 2020 [START_REF] El Karoui | Recover Dynamic Utility from Observable Process: Application to the economic equilibrium[END_REF], the authors study the problem of revealed utility from observed data and then give a consistent way to generate the utility of the agent from the informations given by observing his choices over time. The result of this last paper are established in a very general framework (no regularity assumptions on time and without reference to any market) which makes it applicable to a continuous or discrete framework for vast and varied field like e-commerce, Robo-Advisor, artificial intelligence...

In 2021, Matoussi and Mrad [START_REF] Matoussi | Dynamic Utility and related nonlinear SPDE driven by Lévy Noise[END_REF] study consistent dynamic utilities in an incomplete Itô's market driven by a multidimensional brownian motion and a general Lévy measure.

Let us briefly present the idea and different steps of the revealed dynamic utility problem as in [START_REF] El Karoui | Recover Dynamic Utility from Observable Process: Application to the economic equilibrium[END_REF]. By definition, the observable is a so-called dynamic positive characteristic process {X t (x)} considered for different values of its initial condition X 0 (x) = x > 0, and assumed to be increasing in x (to be coherent with the expected utility criterium). The goal is to recover, from a given initial utility function u, a stochastic dynamic utility U = {U (t, z, ω), z > 0}, "revealed optimally" in the sense that at any (stopping) time τ , the preference for the observable process is in mean equal to its value at time 0, E(U (τ, X τ (x))) = u(x): from the probabilistic dynamic view point, on a given filtered probability space, the performance process "U (t, X t (x))

is a martingale". Focusing on the concavity of the utility criterium U, tools of convex analysis play a key role, especially the invertible decreasing marginal utility U z (t, z). Its allows to define

the convex Fenchel-Legendre transform Ũ (t, y) of U (t, x), by U (t, z)-z U z (t, z) = Ũ (t, U z (t, z)),
but also to linearize the recovery problem, by using the one to one correspondence between the class of dynamic utilities U (revealed by X) and the adjoint processes Y candidate to play the role of U z (t, X t ), more precisely {Y t (u z (x)) = U z (t, X t (x))}. Since the characteristic process is invertible, the triplet {(u, X t (x), U (t, z))} is in one to one correspondence with the triplet { u, X t (x), Y t (u z (x)) }. In fact, denoting by {X t (z) := (X t ) -1 (z)} the inverse flow of X, the authors have the characterization

U z (t, z) = Y t (u z (X t (z)), and
U (t, z) = U (t, z 0 ) + z z 0 Y t (u z (X t (x))dz.
Note that in the continuous semimartingale [START_REF] El Karoui | An Exact connection between two Solvable SDEs and a Non Linear Utility Stochastic PDEs[END_REF] and semimartingale with jumps framework [START_REF] Matoussi | Dynamic Utility and related nonlinear SPDE driven by Lévy Noise[END_REF],

the authors have shown that if U is a regular dynamic utility, consistent with a given financial market, it is necessary in this form with X being the optimal portfolio and Y being the optimal state price density process.

To illustrate some of the main results of dynamic utilities [START_REF] Matoussi | Dynamic Utility and related nonlinear SPDE driven by Lévy Noise[END_REF], let us consider a filtered probability space (Ω, F, P) supports a q-dimensional standard Brownian motion B = (B 1 , . . . , B q ) on and an independent q ′ -dimensional Poisson random measure π on [0, ∞[×IR q ′ with constant time dependent intensity measure λdt × ν(de) defined on the filtered probability space

(Ω, F, F, P). λ is the intensity of jumps and ν is a finite positive measure on IR q ′ , satisfying standard integrability assumptions. We also denote by π the compensated version of π:

π(dt, de) = π(dt, de) -ν(de)λ(t)dt.
Let us consider two IR d -valued stochastic processes X and Y , solutions of the following stochastic differential equations (SDE for short)

dX t (x) = µ(t, X t (x))dt + q i=1 σ i (t, X t (x))dB i t + IR q ′ h(t, X t -(y), e)π(dt, de), X 0 (x) = x, ( 1
       |b(t, x) -b(t, y)| ≤ C Y |x -y|, |µ(t, 0)| ≤ C Y , |γ(t, x) -γ(t, y)| ≤ C Y |x -y|, |γ(t, 0)| ≤ C Y , |g(t, x, e) -g(t, y, e)| ≤ C Y (e)|x -y|, |g(t, 0, e)| ≤ C Y (e), (1.3.42)
where C Y (e) satisfies IR q ′ [C Y (e)] p ν(de) < ∞, ∀p ≥ 2.

• (HP2) b, γ and g are continuously space-differentiable functions s.t. ∇ y b, ∇ y γ and ∇ y g

satisfy (HP1) with (C Y,∇ , C Y,∇ (e)) in place of (C Y , C Y (e)).
• (HP3) b, γ and g are α Y -Hölder continuous in time, locally in space, The novelty of [START_REF] Matoussi | Dynamic Utility and related nonlinear SPDE driven by Lévy Noise[END_REF] is that the authors have succeeded in linking the solutions of these SPDE to those of two SDEs. They have shown that the marginal utility is represented as the compound of the solutions of two SDEs. This explains the need to study the composition of the approximations of the two SDEs instead of trying to discretise the SPDEs directly, which is not an easy task. To be clearer, the dynamic utility of an investor is a random map (t, z, ω) → U (t, z, ω), depending on time t and wealth. Let us assume that it is of type Itô driven by a Lévy process,

|b(t, x) -b(s, x)| + |γ(t, x) -γ(s, x)| ≤ C Y (1 + |x|)|t -s| α Y . |g(t, x, e) -g(s, x, e)| ≤ C Y (e)(1 + |x|)|t -s| α Y . ( 1
dU (t, z) = β(t, z)dt + γ(t, z)dB t + IR H(t, z, e)π(dt, de),
dU (t, z) = -zU z (t, z)r t + IR (U (t, z) + H(t, z, e))ν(de) -Q(t, z, κ * ) dt + γ(t, z)dW t + IR H(t, z, e)π(dt, de), (1.3.44)
where

Q(t, z, κ * ) := IR U + H t, z(1 + κ * t .h S (t, e)) ν(de) + 1 2 U zz (t, z)∥zκ * t + γ z (t, z) + U z (t, z)(η t -α t ) U zz (t, z) ∥ 2 + 1 2 U zz (t, z)∥ γ z (t, z) + U z (t, z)(η t -α t ) U zz (t, z) ∥ 2 .
With r and η denotes the interest rate and the risk premium of the market. And zκ * t (z) denotes the strategy of the investor defined as

zκ * t (z) = - γ R z (t, z) + U z (t, z)(η t -α t ) U zz (t, z) -IR (U z + H z ) t, z(1 + κ * t (z).h S (t, e) .h S (t, e)ν(de) U zz (t, z)
.

(

In this work, we wish to apply these results [START_REF] El Karoui | Recover Dynamic Utility from Observable Process: Application to the economic equilibrium[END_REF][START_REF] Matoussi | Dynamic Utility and related nonlinear SPDE driven by Lévy Noise[END_REF] in the field of machine learning for a financial model with defects, in other word a financial model with jumps. Therefore, we will dedicate the following section to introduce the main result of Matoussi and Mrad [START_REF] Matoussi | Dynamic Utility and related nonlinear SPDE driven by Lévy Noise[END_REF]:

Let us consider a stochastic process depending on a spatial parameter x. This parameter x, motivated by economic considerations, represents the initial wealth of an investor, taking no negative values in IR + . It can be viewed as a progressive random fields, in other word, it is a random variable measurable w.r.t. IF ∞ ⊗ B(IR + ) ⊗ B(IR 0 + ), which is a collection of progressive processes t → X(t, x). This approach is mainly based on stochastic flows technics. The idea in [START_REF] Matoussi | Dynamic Utility and related nonlinear SPDE driven by Lévy Noise[END_REF], is to established the forward dynamics of the inverse flow X of a regular semi-martingale X, monotone with respect to its initial condition. In [START_REF] Fujiwara | Stochastic differential equations of jump type and lévy processes in diffeomorphisms group[END_REF], Authors study the regularity of semimartingales random field from the regularity of its local characteristics (ν, σ, h) and conversely ( [48, Theorem 1.3]). They establish in particular the backward dynamic of the inverse flow X of a regular semimartingale X, monotonic with respect to its initial condition. In Matoussi and Mrad [START_REF] Matoussi | Dynamic Utility and related nonlinear SPDE driven by Lévy Noise[END_REF] under some sufficient conditions, the authors are concerned by the forward dynamic of the inverse flow of an Itô random field. This is a key result to establish the main results of the paper and particularly to show that the marginal utility is a compound of two stochastic flow satisfying a SDEs with jumps. So as in El Karoui and Mrad [START_REF] El Karoui | Recover Dynamic Utility from Observable Process: Application to the economic equilibrium[END_REF], the regularity of the coefficients of SDEs is a fundamental. Therefore, we start by introducing a certain notion of differentiable random fields called K m,δ locregular as follows:

(i) X is said to be K m,δ loc -regular (resp. K m,δ loc -regular) if X is C m,δ
-random fields such that for any compact K ⊂]0, +∞[, and any T ,

T 0 ∥ϕ∥ m,δ:K (t, ω) < ∞, (resp. T 0 ∥ψ∥ 2 m,δ:K (t, ω)dt < ∞).
(ii) X is said to be K m,δ loc,ν -regular if X is C m,δ -random fields such that for any compact K ⊂ ]0, +∞[, and any T ,

T 0 IR ∥ψ∥ 2 m,δ:K (t, e, ω)ν(de)dt < ∞. (iii) We use the notations K m b , K m b , K m,δ b , K m,δ b , K m b,ν and K m,δ b,ν
, when these different norms are well-defined on the whole space ]0, +∞[ where the derivatives (up to a certain order) are bounded in the spatial parameter, with integrable (resp. square integrable) in time random bound.

As we have previously mentioned, the key point in [START_REF] El Karoui | An Exact connection between two Solvable SDEs and a Non Linear Utility Stochastic PDEs[END_REF][START_REF] El Karoui | Recover Dynamic Utility from Observable Process: Application to the economic equilibrium[END_REF][START_REF] Matoussi | Dynamic Utility and related nonlinear SPDE driven by Lévy Noise[END_REF] is to show that the marginal utility as the compound maps of Y and the inverse flow X. So we need to find the forward dynamics of the inverse flow X of a regular semi-martingale X which is monotonic with respect to its initial condition.

Theorem 1.3.3. (Matoussi and Mrad [89])

Let X be a solution of the following SDE(µ, σ, h),

dX t (x) = µ(t, X t (x))dt + σ(t, X t (x))dW t + IR h(t, X t -(x), e)π(dt, de). (1.3.46) Let δ ∈ [0, 1[ and assume (µ, σ, h) ∈ K 2,δ b × K 2,δ b × K 2,δ b,ν .
(i) Then, the SDE(µ, σ, h) admits a unique strong solution X(z), starting from z at time t = 0.

(ii) If in addition, the maps z → z + h(t, z, e) are homeomorphic, the map z → X t (z) is strictly increasing with inverse flow X(t, x) satisfying the following second order SPDE,

dX(t, x) = X x (t, x) -µ(t, x) + IR h(t, x, e)ν(de) + 1 2 ∂ x X x (t, x)∥σ(t, x)∥ 2 -X(t, x) + IR ψ(t, x, e)ν(de) dt -X x (t, x)σ(t, x)dW t + IR -X(t -, x) + X(t -, ψ(t, x, e)) π(dt, de), (1.3.47)
where ψ(t, z, e) is the inverse map of z → z + h(t, z, e).

Remarks 1. For the proof in [START_REF] Matoussi | Dynamic Utility and related nonlinear SPDE driven by Lévy Noise[END_REF], the authors use an extension of Iôt-Ventzel's formula which was established by Oksendal and Zhang [START_REF] Oksendal | The itô-ventzell formula and forward stochastic differential equations driven by poisson random measures[END_REF].

As mentioned above, see details in [START_REF] El Karoui | An Exact connection between two Solvable SDEs and a Non Linear Utility Stochastic PDEs[END_REF][START_REF] El Karoui | Recover Dynamic Utility from Observable Process: Application to the economic equilibrium[END_REF][START_REF] Matoussi | Dynamic Utility and related nonlinear SPDE driven by Lévy Noise[END_REF], we study the marginal utility along the optimal portfolio U z (t, X t ) and show that U z (t, X t ) is a price density process solution of some specific SDE (the optimal SDE of the dual problem satisfied by the optimum Y ), regularity on the coefficients of this SDE show the existence and uniqueness and then leads to the main result:

U z (t, z) = Y • U z (X t (z)).
Using the above theorem and the Iôt-Ventzel's formula, Matoussi and Mrad [START_REF] Matoussi | Dynamic Utility and related nonlinear SPDE driven by Lévy Noise[END_REF] introduce the following results in order to gives the dynamics of the compound map Y • X, Corollary 1.3.4 (Matoussi and Mrad [89]). Let Y be a K 2,δ loc -semimartingale and X a K 3,δ locsemimartingale for some δ ∈]0, 1[, satisfying the dynamics

   dX t (x) = µ(t, X t (x))dt + σ(t, X t (x))dW t + IR h(t, X t -(x), e)π(dt, de), dY t (y) = b(t, Y t (y))dt + γ(t, Y t (y))dB i t + IR q ′ g(t, Y t -(y), e)π(dt, de). (1.3.48)
Assume X to be monotonic with respect to its initial condition and let X denotes its inverse flow.

Then the compound map Φ(t, x) := Y t (X(t, x)) satisfies the following second order SPDE

dΦ(t, x) = µ Y (t, Φ(t, x)) -Φ x (t, x) µ(t, x) + σ Y y (t, Φ(t, x))σ(t, x) + 1 2 ∂ x Φ x (t, x)∥σ(t, x)∥ 2 dt + IR H(t, x, e) -h Y (t, Φ(t, x), e) + Φ x (t, x)h(t, x, e) ν(de)dt + σ Y (t, Φ(t, x)) -Φ x (t, x)σ(t, x) dW t + IR H(t, x, e)π(dt, de), with H(t, x, e) := Φ t -, ψ(t, x, e) + h Y (t, Φ(t -, ψ(t, x, e)), e) -Φ(t -, x).
According to [START_REF] El Karoui | Recover Dynamic Utility from Observable Process: Application to the economic equilibrium[END_REF][START_REF] Matoussi | Dynamic Utility and related nonlinear SPDE driven by Lévy Noise[END_REF], the marginal utility is written as

U z = Y • u z (X t (z))
, under Monotonicity assumptions and the following necessary and sufficient condition:

Proposition 1.3.5. In the particular case where U (t, z) = zY t with

dY t = µ Y (t, Y t )dt + σ Y (t, Y t )dW t + IR h Y (t, Y t -, e)π(dt, de). (1.3.49)
The product XY follows the dynamics

d(X t Y t ) = X t µ Y (t, Y t ) + Y t µ X (t, X t ) + σ X (t, X t )σ Y (t, Y t ) dt + X t σ Y (t, Y t ) + Y t σ X (t, X t ) dW t + I R h Y (t, Y t , e)h X (t, X t , e)ν(de)dt (1.3.50) + I R X t -h Y (t, Y t -, e) + Y t -h X (t, X t -, e) + h Y (t, Y t -, e)h X (t, X t -, e) π(dt, de).
Therefore, the process (X t Y t ) t is a local martingale if and only if

X t µ Y (t, Y t ) + Y t µ X (t, X t ) + σ X (t, X t )σ Y (t, Y t ) + IR h Y (t, Y t , e)h X (t, X t , e)ν(de) = 0, dt ⊗ P.
Note that this result also characterizes a class of solutions of the SPIDE (1.3.44), making a connection with those of two SDEs (1.3.40) and (1.3.41). From a numerical point of view we can then propose a simple scheme to solve the SPIDE, by combining two Euler schemes to solve the optimal SDEs, as established in [START_REF] Gobet | Convergence rate of strong approximations of compound random maps, application to spdes[END_REF][START_REF] Mrad | Solving some stochastic partial differential equations driven by lévy noise using two sdes[END_REF].

On the field of forward utilities, the agent needs to adjust his preferences over time, then the idea is to use the Forward approach to anticipate the future value observable and apply the inverse problem using this prediction to deduct the value of the parameters that characterize the system and this leads us to a family of forward model solutions consistent with the data and not with a prediction.

As mentioned above, this forward thinking is suitable for the study of revealed preference problems in vast and varied field like E-commerce, Robo-Advisor, artificial intelligence. It is a learning approach based on the observation of the behavior of an agent. Roughly speaking, the objective is always to learn about the utility of an agent (player, investor, e-commerce customer) by observing its behavior in front of the important change of its fundamental parameters. So this work is a direct application of ideas developed in El Karoui and Mrad [START_REF] El Karoui | An Exact connection between two Solvable SDEs and a Non Linear Utility Stochastic PDEs[END_REF] to the framework of robo-advisors.

Main results and contributions

Motivation Automated investment managers or robo-advisors, were imposed these last years in the financial and economic world to replace or alternate with traditional human advisors. The robo-advisors are online platforms that provide financial advice or portfolio management to an investor or client. The performance of the robo-advisor strongly depends on its ability to accurately assess the investor's risk tolerance. They have many advantages over the investment services offered by traditional management consultants. It knows the investor better than a traditional asset manager. Because of this better knowledge, the robo-advisor may propose a more appropriate asset allocation. In addition, robo-advisor performs the task in a systematic way and implements an automated rebalancing process. From the client's perspective, the biggest advantage of robo-Advisors is that it provides a low-cost alternative to traditional investing.

The idea of this platforms is to develop a new investment framework in which the robo-advisor not only manages the client's portfolio, but also reacts regularly with him to obtain updated information on his performance over time. More precisely, the client communicates his preferences to the robo-advisor only at specific update dates (τ i ) i . So between two update times, the robo-advisor does not receive any information from the customer. Therefore, it develops a framework to learn the investor's preferences between two update times. In other words, by observing the data communicated by the client at interaction dates, robo-advisor approximates the preferences of the client to make investment decisions and it interacts repeatedly with the client to update its information by asking the investor his new data to avoid making decisions based on stale information. In general, the robo-advisor solicits this information through targeted online questionnaires, and transfers it to digital data.

We assume that we have observations for several different initial conditions (z j ) j . Such informations are not necessarily collected in the questionnaires of the Robo-Advisors, but maybe it is necessary because we know very well that the choices of an agent are strongly conditioned by its initial position (its initial wealth). Moreover, without these observations, we have a little chance to find the dynamic utility of the agent, see the papers [START_REF] El Karoui | Consistent utility of investment and consumption: a forward/backward spde viewpoint[END_REF][START_REF] El Karoui | An Exact connection between two Solvable SDEs and a Non Linear Utility Stochastic PDEs[END_REF][START_REF] El Karoui | Recover Dynamic Utility from Observable Process: Application to the economic equilibrium[END_REF][START_REF] Matoussi | Dynamic Utility and related nonlinear SPDE driven by Lévy Noise[END_REF].

Contribution

In this work, we are interested in learning the dynamic utility. In other words, even the robo-advisor builds the client's utility U at specific update dates τ i ( where is an increasing sequence of jumps times ), it wants to approximate it all the time t. Therefore, we set the following steps to achieve our goal:

1. Simulate the inverse flows X of the wealth process defined as a pure jumps process.

2. Build the price density process Y .

Build the dynamics of marginal utility

U z (t, z) = Y t (u z (X t (z))).
4. Build the client's utility U (τ i , .) at τ i update times.

5. Approximate utility U (t, .) at an intermediate date t ∈]τ i , τ i+1 [ to be able to make decisions for him.

In this work, we have two main parts, the first to simulate the utility of the client based on the results of the Mrad and its co-authors [START_REF] El Karoui | Recover Dynamic Utility from Observable Process: Application to the economic equilibrium[END_REF][START_REF] Matoussi | Dynamic Utility and related nonlinear SPDE driven by Lévy Noise[END_REF][START_REF] Mrad | Solving some stochastic partial differential equations driven by lévy noise using two sdes[END_REF] and the second one is dedicated to the utility learning between two update times. Therefore, in the following we propose algorithms to approximate the inverse flow of X and Y in order to construct the marginal utility.

Simulation of inverse flow of (1.3.40) as a backward in time SDE :

In 2018, Gobet and Mrad [START_REF] Gobet | Convergence rate of strong approximations of compound random maps, application to spdes[END_REF] have discussed different approaches to the inverse flow of SDE solution in the continuous framework (see Section 4 in [START_REF] Gobet | Convergence rate of strong approximations of compound random maps, application to spdes[END_REF]). Among these approaches, we are interested in algorithm of inverse flow as a backward in time SDE which it has recently been extended by Mrad [START_REF] Mrad | Solving some stochastic partial differential equations driven by lévy noise using two sdes[END_REF] to the discountinuous case ( see section 4 [START_REF] Mrad | Solving some stochastic partial differential equations driven by lévy noise using two sdes[END_REF]). In the following we introduce some scheme to approximate the inverse flow of X.

Let us consider X s,t (x) the solution, starting from x at time s, of the SDE with coefficients (µ, σ, h). Denote by X s,t the inverse map of Y s,t which can also be defined by X s,t (x) := X t (X s (x)). This approach asks to consider the dynamics of X s,t (x) in the variable s: doing so, we aim at computing the inverse of Y backward in time instead of forward in time.

Simulating the inverse flow as a solution of a stochastic differential equation backward in time was employed in [START_REF] Gobet | Convergence rate of strong approximations of compound random maps, application to spdes[END_REF] in a continuous case. With this result in hand, the approximation of X s,t is made possible simply using a standard Euler scheme like for Y n . Using similar notations as below:

Scheme 1.3.1. Based on this key result, the approximation of the inverse flow X is achieved by the following steps.

• Set X N t,t (z) = z. If t = θ k for some k ∈ 1, J , set X N t -,t (z) = z -k(t, z, E k ) else X N t -,t (z) = z. -For s ∈]τ t , t], set X N s,t (z) = X N t -,t -σ(t, X N t -,t (z)) • (B t -B s ) (1.3.51) -µ(t, X N t -,t (z)) -∂ z σ(t, X N t -,t (z)) • σ(t, X N t -,t (z)) - IR X(t, X N t -,t (z), e)ν(de) (t -s). • For l ∈ 0, N + J } satisfying t l < τ t , -if t l = θ k for some k ∈ 1, J , set X N t - l ,t (z) = X N t l ,t (z) -k(t l , X N t l ,t (z), E k ) else X N t - l ,t (z) = X N t l ,t (z) -and for s ∈ [t (l-1) , t l [ X N s,t (z) = X N t - l ,t (z) -σ(t l , X N t - l ,t (z)) • (B t l -B s ) (1.3.52) -σ(t l , X N t - l ,t (z)) -∂ z σ(t l , X N t - l ,t (z)) • σ(t l , X N t - l ,t (z)) - IR h(t l , X N t - l ,t (z), e)ν(de) (t l -s).

Simulation of the price density process (1.3.41) using Euler schema

Let us consider a sequence e 1 , e 2 , ... of independent random variables with common exponential distribution with parameter 1. We define

Λ(t) = t 0 λ(s)ds, t ∈ [0, T ].
To simplify, we take λ(t) = 1. The number of jumps of the random Poisson measure N (dt, de)

in an interval [0, t] is determined as

J(t) = max{k : k j=1 e j ≤ Λ(t)}.
We denotes by J = J(T ) the total number of jumps in [0, T ] and The jump times (τ k ) k of the Poisson measure defined by

τ k = Λ -1 ( k j=1 e j ), k ∈ {1, ..., J},
where Λ -1 is the right continuous inverse of Λ. The jump times can be computed recursively by

e k = τ k τ k-1 λ(s)ds, k ∈ {1, ..., J}.
Once the jump times are computed, we proceed to sample the marks {E k }, that, conditionally on the values of the jumps times, are independent random variables distributed respectively according to {ν(de)}. The random measure with intensity λ(t)dtÖν(de) can then be constructed as

π(dt, de) = J k=1 δ (τ k ,E k ) (dt, de).
Consequently, the stochastic intergal with respect to the Poisson random measure can be defined as

t 0 IR d h(s, X s -, e)π(dt, de) = J(t) k=1 h(τ k , X τ - k , E k ), t ∈ [0, T ].
Let N ≥ 1 and let us consider the discretization family

{ ti := i T N , i ∈ 0, N } of [0, T ].
We consider also the jump times {τ k , k ∈ 1, J } with corresponding marks {E k , k ∈ 1, J }. Then, we consider the argumented partition given by the union

P N = {t l , l ∈ 0, N + J } = ti := i T N , i ∈ 0, N ∪ τ k , k ∈ 1, J .
Thus, using Euler scheme, the discretization of Y (1.3.41) is given by the following steps Scheme 1.3.2. The Euler scheme to approximate the solution Y of the SDE (1.3.41) is given by the following steps

• Set Y N 0 (x) = x. • For k = 0, ...., N + J -1, if t ∈ (t k , t k+1 ], then Y N t -(x) = Y N t k (x) + b(t k , Y N t k (x))(t -t k ) + γ(t k , Y N t k (x))(B t -B t k ).
• If t k+1 = τ l for some l ∈ 1, J , then we introduce a correction due to jumps discontinuities

Y N t k +1 (x) = Y N t k (x) + g(t k+1 , Y N k+1 (x), E l ).
Simulation of dynamic utility U N : According to Proposition 1.3.5, we must ensure that the product XY remains a local martingale for the building of process X and Y . To ensure this constraint, we must take care that the parameters of the jumps V and W (which help us to build Y and X) are independent.

Once we have simulated X and Y , we can construct the marginal utility U z as follows

U N (t, z) = n i=1 Y N t (u z (X N t (z i )))(z i+1 -z i )
which converges to

U N (t, z) = z z 0 Y N t (u z (X N t (x)))dx. (1.3.53) Remark 1.3.6.
It should be mentioned that the approximated utility must be concave with respect to its wealth z.

This approach is based on the work of Mrad and his co-authors [START_REF] El Karoui | An Exact connection between two Solvable SDEs and a Non Linear Utility Stochastic PDEs[END_REF][START_REF] El Karoui | Recover Dynamic Utility from Observable Process: Application to the economic equilibrium[END_REF][START_REF] Matoussi | Dynamic Utility and related nonlinear SPDE driven by Lévy Noise[END_REF], it allows us to estimate the agent's utility at dates and interaction points. This is a very important step as until now there was no idea how to quantify the preferences of an agent from these choices. The goal then, starting from the values obtained (U (t i , z j )) i,j , is to estimate U (t, z) for all (t, z) and to predict its future decisions. For this we will use different machine learning algorithms and we will compare them.

Learning utilities method

In the second part, we propose different methods to learn client's utilities. Let's assume that the client has enough sufficient interaction with the machine. Can we approximate/learn his utility until the next interaction date so that the machine can make decisions for him while remaining consistent with his preferences and past decisions ?

The aim in the following is to propose an algorithm for learning the preferences of the agent.

As we do not have an access to a real data, we will simulate our own data, the client's utility from the observations (X) i,j and (Y ) i,j for (i, j) ∈ 1, J × 1, n using the above procedure.

The generated values U (τ i , z j ) will play the role of starting data for our learning algorithms.

This kind of algorithm could clearly be used on robo-advisor platforms, since it can "predict" (over a short period of time) the preferences of an agent and therefore can make decisions for him/her.

Machine Learning algorithms automatically learn to perform a task or make predictions from data and improve their performance over time. Once trained, the algorithm will be able to find patterns in new data. Depending on the information available during the learning phase, learning is qualified in different ways. If the data are labeled (i.e., the response to the task is known for these data), it is supervised learning. It is called classification if the labels are discrete, or regression if they are continuous. If the model is learned in an iterative way, according to a reward received by the program for each action taken, it is called reinforcement learning. In the most general case, without labels, we try to determine the underlying structure of the data (which can be a probability density) and it is then called unsupervised learning.

Here, we are interested in supervised learning. Our task is to predict a target value which is the value of utility at fixed time τ i , denoted by (U i j ) j∈ 1,n a vector of the matrix (U(τ i , z j )), for (i, j) ∈ 1, N × 1, n from different initial wealth values (z j ) 1,n and fixed value τ i :

U i j := (U(τ i , z j )) j∈ 1,n .
In Machine Learning, all starts from a Dataset which contains our data. In supervised learning, the Dataset contains the questions (Z j ) j∈ 1,n and answers (U i j ) j∈ 1,n to the problem that the machine must solve.

In this work, we have chosen to use Scikit-learn library of Python contains all the math-ematical functions which are necessary for learning problems. Scikit-learn is an open source machine learning library that supports supervised and unsupervised learning. It also provides various tools for model fitting, data preprocessing, model selection and evaluation and many other utilities. In a few lines, we can develop a learning models as follows:

• Create the Dataset. (Z in , U i ) such that -Z in are the input datas. It contains different initial values of wealth (z j ) j∈ 1,n , (i.e) (Z i j ) j∈ 1,n = (z j ) j∈ 1,n .
-U i are the target. It contains a vector of the approximated utility matrix (U(τ i , z j )) for

(i, j) ∈ 1, N × 1, n .
• Define our model: choose the best model to fit our data set among the predefined models in the package.

• Fitting SKlearn.fit(Z in , U i ) : The objective is fitted the model, (i.e), the machine must learn from the model.

1. The samples matrix Z in . The size of Z in is typically (n_samples, n_features), which means that samples are represented as lines and features are represented as columns.

2. The target values U i is usually 1d array where the i-th entry corresponds to the target of the i th sample of Z in .

• Predicting: SKlearn.predict(Z in ): Once the estimator is fitted, it can be used for predicting target values of new data. For this step we have as output :

U p,i := SKlearn.predict(Z in ),
such that U p represents the value predicted with our model.

• Evaluation: SKlearn.score(Z in , U i ): We can now observe the accuracy of our model using the function Model.score. This function returns the coefficient of determination R 2 of the prediction. The coefficient R 2 is defined as

R 2 = 1 - (U i -U p,i ) 2 (U i -Ū ) 2 , with Ū := 1 N N j=1 U i j represents the mean of (U i j ) j .
In this work, we propose to use two packages from Scikit-learn library create our model and we will compare the results:

• Support vector Machine : In machine learning, support-vector machines (SVM in short) are supervised learning models with associated learning algorithms that analyze data for classification and regression analysis, developed by Vladimir Vapnik et Alexey

Chervonenkis. The problem of Support Vector regression ( SVR) is to find a function that approximates mapping from an input domain of real numbers on the basis of a training sample. The objective, when we are moving on with SVR, is to basically consider the points that are within the decision boundary line. The best fit line is the hyperplane of the equation a j Z in j + b, ∀j ∈ 1, n that has a maximum number of points. Thus, to summarize, we are looking for a hyperplane such that most of train observation are inside the marge ϵ and we need to add a supplementary constraint

U i j -(a j z j + b) ≤ ϵ + ξ j , if the point is above the margin, (1.3.54)
and

(a i z j + b) -U i j ≤ ϵ + ξ * j , if the point is below the margin, (1.3.55)
where 1. Z in = (Z in j ) j∈ 1,n represents the input defined by different initial values of the wealth. 2. (a, b) represent the parameters of weigth (a i ) i and biais b: we choose it randomly at the beginning then we try to find the optimal values in the algorithm.

3. U i = (U i j ) j := (U(τ i , z j )) j represents the target: the value which we try to predict. We remain that the target is defined by a column vector of the matrix U(τ i , z j ) i∈ 1,N ,j∈ 1,n at fixed τ i . 4. ϵ is the width of the margin of hyperplan.

5. (ξ j ) j , (ξ * j ) j ≥ 0 represent the error of the observations outside the hyperplane.

This problem allow some observations to fall on the wrong side of the margin, but will penalized them by the parameter C. Finally, we can introduce our optimization problem:

min a∈IR n ,b∈IR    1 2 ||a|| 2 + C l j=1 (ξ j + ξ * j )    , (1.3.56)
where l is the number of the points outside the margin, under the constraint 1. Basically, the idea of SVR model is to minimize the sum of squared weights by taking into account the misclassified observations in order to maximize the margin.

The second idea of support vector machine is the kernel method. It is an extension to non-linear cases. It is quite simple and relies on the projection of the data in a higher dimensional space in which the problem becomes linear. The kernel method is a mathematical trick to solve this problem elegantly, by defining the transformations ϕ and scalar products via a kernel k(a, b) = ϕ(a) T ϕ(b) such that the equation of hyperplan is as follow

U p j := a j ϕ(Z in j ) + b.
Therefore, using the kernel method, the optimization problem (1.3.57)-(1.3.58) becomes

min a∈IR n ,b∈IR 1 2 ||a|| 2 + C l i=1 (ξ j + ξ * j ) , (1.3.57)
where l is the number of the points outside the margin and we remains that

ξ j = U i j - (a j ϕ(z j ) + b -ϵ) and ξ * j = (a j ϕ(z j ) + b + ϵ) -U i j , under the following constraint            U i j -(a j ϕ(Z in j ) + b) ≤ ϵ + ξ j , (a j ϕ(Z in j ) + b) -U i j ≤ ϵ + ξ * j , ξ j , ξ * j ≥ 0, ∀j. (1.3.58)
We also chose to use the NuSVR model of the SVM class: for this model, we add a new parameter ν which controls the number of support vectors and allows to compute automatically ϵ.

Then, for ν ∈ (0, 1], the optimization problem becomes for fixed time t:

min a∈IR n ,b∈IR 1 2 ||a|| 2 + C(ν ϵ + 1 l l i=1 (ξ i + ξ * i ) , (1.3.59)
where l is the number of the points outside the margin and we remains that The idea of Machine Learning is to let the machine find the parameters θ l,i k := (a, b) for the l-th and k-th node layer of the model that minimize the cost function J. In order to do this, we need to use a back-propagation learning algorithm.

ξ j = U i j - (a j ϕ(z j ) + b -ϵ) and ξ * j = (a j ϕ(z j ) + b + ϵ) -U i ,
Actually, the differences between these outputs U p and the desired outputs U t form errors that are corrected via back-propagation, the weights of the neural network are then changed. By applying this step several times, the error tends to decrease and the network offers a better prediction.

Then for a fixed neural network structure, number of layers, number of neurons in each layer and fixed activation functions, the program therefore amounts to determine the set of parameters such that

θ * := arg min θ J(θ).
In this work, we choose the LBFGS solver as Back-propagation algorithm of the optimization problem. LBFGS is named from the initials of the mathematicians Broyden, Fletcher, Goldfarb and Shanno, who discovered it independently in the late 1960s. This learning algorithm is based on Newton's method which fits the weights using additionally the Hessian matrix H gives the second derivatives of the cost function J with respect to the weights in the following way:

θ l,i k,j = θ l,i k,j -αH -1 ∂J(θ l,i k ) ∂θ l,i k,j
, j = 0, ..., n l + 1, for the l-yh hidden layer and the k-th node.

At the end of this work, we implemented these methods in order to predict the dynamic utility of an agent and compared these different learning methods implemented.

Finally, we can deduce that the neural network model is the best prediction model for our dataset. Beside the other models used in this work are not so bad. Unfortunately, there are no clear rules to follow to choose the model to use from the beginning. In fact, it is recommended to start with the easiest and simplest examples, if these models do not work then consider using a more sophisticated model. CHAPTER 2

MEAN FIELD CONTROL PROBLEM AND APPLICATION TO PRODUCTION OF AN EXHAUSTIBLE RESOURCE

Introduction

Our ain in this chapter is to study a linear quadratic optimal control for mean field stochastic differential equation with jumps. We will use a weak formulation of the standard martingale optimality principle to solve our control problem. This approach is used in the verification theorem for stochastic control problem which is a crucial step in the classical approach to dynamic programming ( see e.g [START_REF] El Karoui | Les aspects probabilistes du controle stochastique: In 9th saint flour probability summer school-1979[END_REF][START_REF] Korn | The martingale optimality principle : The best you can is good enough[END_REF]). We prove the existence of an unique optimal control on finite horizon, by using this method under some assumptions on the coefficients. The optimality of the control is derived by the stability of an decoupled system of Riccati equation and backward stochastic differential equation with jumps. We also consider the problem with common noise adding up some assumptions on the coefficients. On the other hand, we introduce also the linear quadratic problem on the infinite case. This work can be regarded as an extension of the work of Basei and Pham [START_REF] Basei | A weak martingale approach to linear-quadratic mckean vlasov stochastic control problems[END_REF] to the case with jumps.

The main motivation behind this work comes from stochastic model of production of an exhaustible resource, such as oil. It was inspired by Pindyck [107] from Hotelling's work [START_REF] Hotelling | The economics of exhaustible resources[END_REF] and it was adapted later on many works [START_REF] Chan | Bertrand and cournot mean field games[END_REF][START_REF] Chan | Fracking, renewables and mean field games[END_REF][START_REF] Deshmukh | Optimal consumption and exploration of nonrenewable resources under uncertainty[END_REF][START_REF] Ludkovski | Exploration and Exhaustibility in Dynamic Cournot Games[END_REF]]. In the model, the reserves of an exhaustible are a depleted resource, but there was a possibility of exploration and discovery of new reserves which ensure the accumulating or maintaining a level of this reserves. We modelled the new discoveries occur according to a jump process with intensity given by the exploration effort.

The outline of this chapter is a follows. After recalling some notation, we give in Section 2.3 a detailed description of the techniques used to show the existence of the unique optimal control, we then use the weak formulation of verification theorem introduced in previous section. In Section 2.4, we extend our results to the case where a common noise is present, so we add a new assumptions for the new parameters.

We adapt also our results to the infinite-horizon case, in Section 2.5. Here, we kept the similar steps as the case of the finite horizon to apply verification theorem on this case, except that we should look for the stability of decoupled system of Riccati equation on infinite horizon. Finally, in Section 2.6, we introduce an application of production of of exhaustible resource with accumulating or maintaining a level of reserves where this increments of reserves is represented via a Poisson process. This work is concertized in the preprint [START_REF] Matoussi | Mean field control problems and application to production of an exhaustible resource[END_REF].

Framework: Notations and setting

Let T > 0 be a given time horizon and (Ω, F, F := (F t ) t≤T , P) be a stochastic basis such that F 0 contains all the P -null sets of F, F t + = ∩ ε>0 F t+ε = F t , and we suppose that the filtration is generated by the two following mutually independent processes : (i) a standard real Brownian motion B := (B t ) 0≤t≤T and (ii) a Poisson random measure π on IR + × χ, where χ ⊂ IR \ {0} is equipped with its Borel field B(χ). Throughout this paper the measure λ(.) is assumed to be finite on (χ, B(χ))

i.e. χ λ(de) < ∞. Let η(dt, de) = λ(de)dt be its compensated process, i.e., {π([

0, t]×A) = (π -η)([0, t] × A)} t≤T is a martingale for every A ∈ B(χ).
Let ρ ≥ 0 be the discount factor and define A the set of admissible controls as follows:

A := {α : Ω × [0, T ] → IR m s.t α is IF -predictable and T 0 e -ρt E[|α t | 2 ]dt < ∞}.
Let S d be the set of symmetric matrices and (H, |.|) a normed space. We define the following sets:

• L ∞ ([0, T ], H) := ϕ : [0, T ] → H s.t. ϕ is measurable and sup 0≤s≤T |ϕ s | < ∞ , • K ∞ ([0, T ] × χ, H) := ϕ : [0, T ] → H s.t. ϕ is measurable and sup 0≤s≤T,e∈χ |ϕ s (e)| < ∞ , • L 2 ([0, T ], H) := {ϕ : [0, T ] → H s.t. ϕ is measurable and E T 0 e -ρs |ϕ s | 2 ds < ∞}, • L 2 F T (H) := ϕ : Ω → H s.t. ϕ is F T -measurable and E[|ϕ s | 2 ] < ∞ , • S 2 F (Ω × [0, T ], H) := {ϕ : Ω × [0, T ] → H s.t. ϕ is F-progressively measurable and E[ess sup s∈[0,T ] |ϕ s | 2 ] < ∞}, • L 2 F (Ω × [0, T ], H) := {ϕ : Ω × [0, T ] → H s.t. ϕ is F-progressively measurable and T 0 e -ρs E[|ϕ s | 2 ]ds < ∞}, • K 2 F (Ω × [0, T ] × χ, H) := {K : Ω × [0, T ] × χ → H s.t. K is P ⊗ B(χ)-measurable process and E[ T 0 χ |K s (e)| 2 λ(de)ds] < ∞},
where P denote the σ-field of F-predictable sets on Ω × [0, T ].

We define the controlled linear mean field stochastic differential equation in IR d , for a given F-measurable random variable X 0 and a control α ∈ A, by:

           dX α t = b t (X α t , E[X α t ], α t , E[α t ])dt + σ t (X α t , E[X α t ], α t , E[α t ])dW t + X R t (X α t -, E[X α t -], α t , E[α t ], e)π(de, dt), X α 0 = X 0 , (2.2.1)
where for each t ∈ [0, T ], x, x ∈ IR d , a, ā ∈ IR m and e ∈ χ, we set:

b t (x, x, a, ā) := β t + A t x + Ãt x + B t a + Bt ā, (2.2.2) σ t (x, x, a, ā) := γ t + C t x + Ct x + D t a + Dt ā, R t (x, x, a, ā, e) := R t (x, x, a, ā)(e) := δ t (e) + F t (e)x + Ft (e)x + G t (e)a + Gt (e)ā.
Here β, γ are vector-valued F-progressively processes, δ is vector-valued F-predictable pro- We consider a quadratic cost functional to be minimized over α ∈ A of the following type:

J(α) := E T 0 e -ρt f t (X α t , E[X α t ], α t , E[α t ])dt + e -ρT g(X T , E[X T ]) , (2.2.3) where for each t ∈ [0, T ], x, x ∈ IR d and a, ā ∈ IR m f t (x, x, a, ā) := (x -x) ⊤ Q t (x -x) + x⊤ (Q t + Qt )x + 2a ⊤ I t (x -x) (2.2.4) + 2ā ⊤ (I t + Ĩt )x + (a -ā) ⊤ N t (a -ā) + ā⊤ (N t + Ñ )ā + 2M ⊤ t x + 2H ⊤ t a,
and

g(x, x) := (x -x) ⊤ P (x -x) + x⊤ (P + P )x + 2L ⊤ x. (2.2.5)
The coefficients M, H, L defined in (2.2.4) and (2. (H1) The coefficients in equations (2.2.2) satisfy:

(i) β, γ ∈ L 2 F (Ω × [0, T ], IR d ) and δ ∈ K 2 F (Ω × [0, T ] × χ, IR d ), (ii) A, Ã, C, C ∈ L ∞ ([0, T ], IR d×d ), B, B, D, D ∈ L ∞ ([0, T ], IR d×m ), G(.), G(.) ∈ K ∞ ([0, T ] × χ, IR d×m ) , F (.), F (.) ∈ K ∞ ([0, T ] × χ, IR d×d ) .
(H2) The coefficients in equations (2.2.4)-(2.2.5) satisfy:

(i) Q, Q ∈ L ∞ ([0, T ], S d ), P, P ∈ S d , N, Ñ ∈ L ∞ ([0, T ], S m ), I, Ĩ ∈ L ∞ ([0, T ], IR m×d ), (ii) M ∈ L 2 F (Ω × [0, T ], IR d ), H ∈ L 2 F (Ω × [0, T ], IR m ), L ∈ L 2 F (IR d ),
(iii) there exists ξ > 0 such that, for each t ∈ [0, T ],

N t ≥ ξI m , P ≥ 0, Q t -I ⊤ t N -1 t I t ≥ 0, (iv)
there exists ξ > 0 such that, for each t ∈ [0, T ],

N t + Ñt ≥ ξI m , P + P ≥ 0, (Q t + Qt ) -(I t + Ĩt ) ⊤ (N t + Ñt ) -1 (I t + Ĩt ) ≥ 0.
In order to simplify, we denote X t for X α t , Xt for E[X α t ], ᾱ for E[α t ] and b t (x, x), σ t (x, x), R t (x, x)(e) for b t (x, x, α, ᾱ), σ t (x, x, α, ᾱ) and R t (x, x, α, ᾱ)(e). The following lemma gives an estimate for the solution of mean field SDE (2.2.2). Lemma 2.2.1. Let Assumption (H1) be satisfied. Then for any admissible control α, the equation (2.2.1) has a unique solution (X t ) t ∈ S 2 IF . Moreover, we have the following estimate:

E sup 0≤t≤T |X t | 2 ≤ C(1 + E |X 0 | 2 ), (2.2.6)
where C is a positive constant which depends on α.

Proof. We will use the fixed point argument of an appropriate map l defined from

L 2 F (Ω × [0, T ], IR d ) into itself such that (X t ) t := l((x t ) t )
where (X t ) t is solution of the SDE:

dX t = b t (x t , E[x t ])dt + σ t (x t , E[x t ])dW t + χ R t (x t , E[x t ])(e)π(de, dt), (2.2.7) Given x 1 , x 2 ∈ L 2 F (Ω × [0, T ], IR d ),
we define X 1 and X 2 by using equation (2.2.7). We asssume that

X 1 0 = X 2 0 . We define L = ||A|| ∞ ∨ || Ã|| ∞ ∨ ||C|| ∞ ∨ || C|| ∞ ∨ ||F || ∞ ∨ || F || ∞ . Then E[|X 1 t -X 2 t | 2 ] ≤ 2E[| t 0 b s (x 1 s , E[x 1 s ]) -b s (x 2 s , E[x 2 s ])ds| 2 ] + 2E[| t 0 σ s (x 1 s , E[x 1 s ]) -σ s (x 2 s , E[x 2 s ])dB s | 2 ] + 2E[| t 0 R s (x 1 s -, E[x 1 s -])(e) -R s (x 2 s -, E[x 2 s -])(e)π(de, ds)| 2 ].
From the definition of b s and using Cauchy Scharwz's inequality, we have:

|b s (x 1 s , E[x 1 s ]) -b s (x 2 s , E[x 2 s ])| 2 = |A s (x 1 s -x 2 s ) + Ãs (E[x 1 s ] -E[x 2 s ])| 2 ≤ 2L 2 (|x 1 s -x 2 s | 2 + E[|x 1 s -x 2 s | 2 ]).
Using again Cauchy Scharwz's inequality, we have:

E[| t 0 b s (x 1 s , E[x 1 s ]) -b s (x 2 s , E[x 2 s ])ds| 2 ] ≤ 4L 2 T t 0 E[|x 1 s -x 2 s | 2 ]ds. (2.2.8)
For the stochastic integral terms, we have:

E[| t 0 σ s (x 1 s , E[x 1 s ]) -σ s (x 2 s , E[x 2 s ])dB s | 2 ] ≤ 4L 2 t 0 E[|x 1 s -x 2 s |]ds (2.2.9) E[| t 0 R s (x 1 s -, E[x 1 s -])(e) -R s (x 2 s -, E[x 2 s -])(e)π(de, ds)| 2 ] ≤ 4L 2 λ(χ) t 0 E[|x 1 s -x 2 s | 2 ]ds. ( 2
E[|X 1 t -X 2 t | 2 ] ≤ 4L 2 (1 + T + λ(χ)) t 0 E[|x 1 s -x 2 s | 2 ]ds.
For a positive constant c, we define the norm ||x|| 2 c := E[ T 0 e -cs |x s | 2 ds]. Then, we have:

||X 1 -X 2 || 2 c := T 0 E[|X 1 t -X 2 t | 2 ]e -ct dt ≤ 4L 2 (1 + T + λ(χ)) T 0 t 0 E[|x 1 s -x 2 s | 2 ]dse -ct dt ≤ 4L 2 (1 + T + λ(χ)) T 0 E[|x 1 s -x 2 s | 2 ] T s e -ct dtds ≤ 4L 2 (1 + T + λ(χ)) c T 0 E[|x 1 s -x 2 s | 2 ]e -cs ds = 4L 2 (1 + T + λ(χ)) c ||x 1 -x 2 || 2 c .
For c large enough, we have

4L 2 (1+T +λ(χ)) c < 1. Therefore l is a contraction mapping on L 2 F (Ω × [0, T ], IR d ) into itself, and so l has a unique fixed point X ∈ L 2 F (Ω × [0, T ], IR d
) which is the unique solution to the mean field SDE (2.2.1).

We turn to prove estimate (2.2.6). For n ∈ IN , we define

τ n := inf{t ≥ 0, |X t -X 0 | ≥ n} and f n (t) := E[sup s≤t |X s∧τn -X 0 | 2 ]
. For all t ≤ T , we have:

sup t≤T |X t∧τn -X 0 | 2 ≤ 2(sup t≤T | t∧τn 0 b s (X s , E[X s ], α s , E[α s ])ds| 2 + sup t≤T | t∧τn 0 σ s (X s , E[X s ], α s , E[α s ])ds| 2 + sup t≤T | t∧τn 0 χ R s (X s -, E[X s -], α s , E[α s ], e)π(de, ds)| 2 ).
Using Cauchy Scharwz's inequality and Burkholder-Davis-Gundy Inequality, we have:

E[sup t≤T |X t∧τn -X 0 | 2 ] ≤ C(T E[ T 0 |b s∧τn (X s∧τn , E[X s∧τn ], α s∧τn , E[α s∧τn ])| 2 ds + E[ T 0 |σ s∧τn (X s∧τn , E[X s∧τn ], α s∧τn , E[α s∧τn ])| 2 ds] + E[ T 0 χ |R s∧τn (X s∧τn , E[X s∧τn ], α s∧τn , E[α s∧τn ], e)| 2 λ(de)ds], (2.2.11)
where C is a positive constant which could change from line to line. For α ∈ A and using the definition of the drift of the state process X (See Equation(2.2.1)), under (H1), we have:

E[ T 0 |b s∧τn (X s∧τn , E[X s∧τn ], α s∧τn , E[α s∧τn ])| 2 ds] ≤ T 0 CE[1 + |X s∧τn | 2 + E[|X s∧τn |] 2 ] ≤ T 0 C(1 + E[|X 0 | 2 ] + f n (s))ds (2.2.12)
Similarly, we have:

E[ T 0 |σ s∧τn (X s∧τn , E[X s∧τn ], α s∧τn , E[α s∧τn ])| 2 ds] + E[ T 0 χ |R s∧τn (X s∧τn , E[X s∧τn ], α s∧τn , E[α s∧τn ], e)| 2 λ(de)ds] ≤ t 0 C(1 + E[|X 0 | 2 ] + f n (s))ds. (2.2.13)
Plugging inequalities (2.2.12) and (2.2.13) into (2.2.11) and using Gronwall's lemma, we obtain

E[sup s≤T |X s∧τn -X 0 | 2 ] ≤ C(1 + E[|X 0 | 2 ]).
Sending n to infinity, by monotone convergence theorem, we obtain

E[sup s≤T |X s -X 0 | 2 ] ≤ C(1 + E[|X 0 | 2 ]
) and so the estimate (2.2.6) is obtained.

Linear quadratic mean field control problem on finite horizon

To solve a linear quadratic mean field control problem ( LQMF control problem in short), we have to find a strategy α * ∈ A, such that

V := J(α * ) = inf α∈A J(α), (2.3.14)
where the criterion J is defined by (2.2.3). By Assumption (H2) and the estimate (2.2.6), the LQMF control problem (2.3.14) is well defined. The aim of this section to solve this control problem on finite horizon by proving a suitable verification theorem.

We are going to use a weak formulation of the standard martingale optimality principle, see e.g [START_REF] El Karoui | Les aspects probabilistes du controle stochastique: In 9th saint flour probability summer school-1979[END_REF][START_REF] Korn | The martingale optimality principle : The best you can is good enough[END_REF].

Lemma 2.3.1. (Finite horizon Verification Theorem, Lemma 3.1 in [START_REF] Basei | A weak martingale approach to linear-quadratic mckean vlasov stochastic control problems[END_REF])

Let {W α t , t ∈ [0, T ], α t ∈ A} be a family of F-adapted process in this form W α t = w t (X α t , E[X α t ]) for some F-adapted random field {w t (x, x), t ∈ [0, T ], x, x ∈ IR d } satisfying w t (x, x) ≤ C(X t + |x| 2 + |x| 2 ), t ∈ [0, T ], x, x ∈ IR d , (2.3.15)
where C is a positive constant, X is a non-negative process satisfying

sup t∈[0,T ] E[|X t |] < ∞, and
(i) w T (x, x) = g(x, x), x, x ∈ IR d , (ii) the map t ∈ [0, T ] -→ E[S α t ], with S α t = e -ρt W α t + t 0 e -ρs f s (X α s , E[X α s ], α t , E[α t ])ds, is non-decreasing for all α ∈ A, (iii) the map t -→ E[S α * t ] is constant for some α * ∈ A.
Then, α * is an optimal control and E[w 0 (X 0 , E[X 0 ])] is the value of the LQMF control problem

(2.3.14) i.e. V 0 = E[w 0 (X 0 , E[X 0 ])] = J(α * ).
Moreover, any other optimal control satisfies the condition (iii).

Proof. From the growth condition (2.3.15) and the estimate (2.2.6), for all t ∈ [0, T ] and α ∈

A, E[S α t ] is well defined. From condition (i), we have E[e -ρT W α T ] = E[e -ρT g(X T , XT )], which implies E[S α T ] = J(α). From condition (ii), we have E[W α 0 (X 0 , X0 )] = E[S α 0 ] ≤ E[S α T ] = J(α).
In the other side, for α = α * and using condition (iii), we have

E[W α * 0 (X 0 , X0 )] = E[S α * 0 ] = E[S α * T ] = J(α * ),
which ensures the optimality of control α * .

Moreover, we consider an another optimal control α ∈ A, then we have

E[W α 0 (X 0 , X0 )] = E[S α 0 ] = E[S α * 0 ] = E[S α * T ] = J(α * ) = J(α).
Since the map t → E[S α t ] is non-decreasing, so it is constant, and we conclude the proof. We start by introducing the following notations:

 := A + Ã, B := B + B, Ĉ := C + C, D := D + D, F := F + F , Ĝ := G + G, Î := I + Ĩ, N := N + Ñ , Q := Q + Q, P := P + P .
Our aim is to find the expression of the random field w t (x, x) which satisfies the assumptions of Lemma (2.3.1).

From the quadratic form of the cost functional f t in (2.2.4) and the terminal cost g t in (2.2.5), we guess the quadratic form of random field {w t (x, x), t ∈ [0, T ], x, x ∈ IR d } i.e.

w t (x, x) = (x -x) ⊤ K t (x -x) + x⊤ Λ t x + 2Y ⊤ t x + φ t , (2.3.16)
where (K, Λ, Y, φ) are valued in S d × S d × IR d × IR and solution of the following system:

                   dK t = . K t dt, 0 ≤ t ≤ T, K T = P, dΛ t = . Λ t dt, 0 ≤ t ≤ T, Λ = P , dY t = . Y t dt + Z Y t dW t + χ R Y t (e)π(de, dt), 0 ≤ t ≤ T, Y T = L, dφ t = . φ t dt, 0 ≤ t ≤ T, φ T = 0.
(2.3.17)

The deterministic functions

( . K, . Λ, . φ) are valued in S d × S d × IR, the processes ( . Y , Z Y ) are F-adapted valued in IR d × IR d and R Y is F-predictable process valued in IR d .
For α ∈ A, we consider S α defined as:

S α t = e -ρt w t (X t , Xt ) + t 0 e -ρs f s (X s , Xs , α t , ᾱt )ds, t ∈ [0, T ]. (2.3.18)
Then E[S α t ] satifies the following ODE:

dE[S α t ] = e -ρt E[D α t ]dt,
where the drift E[D α t ] is defined as follows:

E[D α t ] = E -ρw t (X t , Xt ) + d dt E[w t (X t , Xt )] + f t (X t , Xt , α t , ᾱt ) .
From the dynamics of X t (See equation (2.2.1)), we have:

d Xt = [ βt + Ât Xt + Bt ᾱt ]dt, (2.3.19)
and

d(X t -Xt ) = [(β t -βt ) + A t (X t -Xt ) + B t (α -ᾱt )]dt (2.3.20) + [γ t + C t (X t -Xt ) + Ĉt Xt + D t (α t -ᾱt ) + Dt ᾱt ]dW t + χ [δ t + F t (e)(X t --Xt -) + Ft (e) Xt -+ G t (e)(α t -ᾱt ) + Ĝt (e) ᾱt ]π(de, dt).
We apply the Itô's formula to w t (X t , Xt ), we use the quadratic form of the running cost f t and the dynamics of equations (2.3.19), (2.3.20) and (2.2.1), we obtain :

E[D α t ] =E (X t -Xt ) ⊤ ( . K t + ϕ t )(X t -Xt ) + X⊤ t ( . Λ t + ψ t ) Xt (2.3.21) +2( . Y t + ∆ t ) ⊤ X t + . φ t -ρφ t + Γt + κ t (α) ,
where, for t ∈ [0, T ], Ft (e) ⊤ K t δt (e)λ(de)

                                                         ϕ t := -ρK t + K t A t + A ⊤ t K t + C ⊤ t K t C t + Q t + χ F ⊤ t (e)K t F t (e)λ(de) = ϕ t (K t ), ψ t := -ρΛ t + Λ t Ât + Â⊤ t Λ t + Ĉ⊤ t K t Ĉt + Qt + χ F ⊤ t (e)K t Ft (e)λ(de) = ψ t (K t , Λ t ), ∆ t := -ρY t + A ⊤ t (Y t -Ȳt ) + Â⊤ T Ȳt + C ⊤ t (Z Y t -Z Y t ) + Ĉ⊤ t Z Y t +K t (β t -βt ) + Λ t βt + C ⊤ t K t (γ t -γt ) + Ĉ⊤ t K t γt + M t + χ F ⊤ t (e)(
= ∆ t (K t , Λ t , Y t , Ȳt , Z Y t , Z Y t , R Y t , R Y t ), Γ t := γ ⊤ t K t γ t + 2β ⊤ t Y t + 2γ ⊤ t Z Y t + χ δ t (e) ⊤ K t δ t (e)λ(de) + 2 χ δ t (e) ⊤ R Y t (e)λ(de) = Γ t (K t , Y t , Z Y t , R Y t ), and
κ t (α) := (α t -ᾱt ) ⊤ S t (α -ᾱt ) + ᾱ⊤ t Ŝt ᾱt + 2(U t (X t -Xt ) + V t Xt + Θ t + ϵ t -εt )α t . (2.3.22)
Here S t , Ŝt , U t , V t , Θ t which appear in (2.3.22), are defined as follows and

                               S t := N t + D ⊤ t K t D t + χ G ⊤ t (
       ϵ t := H t + D ⊤ t K t γ t + B ⊤ t Y t + D ⊤ t Z Y t + χ G ⊤ t (e)K t δ t (e)λ(de) + χ G ⊤ t (e)R Y t (e)λ(de), εt := Ht + D ⊤ t K t γt + B ⊤ t Ȳt + D ⊤ t Z Y t + χ G ⊤ t (e)K t δt (e)λ(de) + χ G ⊤ t (e)R Y t (e)λ(de). (2.3.24)
We notice that the matrices S t and Ŝt are definite positive in S m . This follows from the nonnegativity of the matrix K, conditions (iii)-(iv) in (H2) and the non-negativity of the integral χ G ⊤ t (e) K t G t (e)π(dt, de). In this case, one could find a deterministic IR m×m -valued Υ such that for all t ∈ [0, T ]

Υ t S t Υ ⊤ t = Ŝt , for all t ∈ [0, T ].
This implies, that we can rewrite the expectation of κ t (α) as

E[κ t (α)] = E (α t -ᾱt + Υ ⊤ t ᾱt -η t ) ⊤ S t (α t -ᾱt + Υ ⊤ t ᾱt -η t ) -ζ t ,
where

η t = a 0 t (X t , Xt ) + Υ ⊤ t a 1 t ( Xt ) dt ⊗ dP a.e., (2.3.25)
with a 0 t (X t , Xt ) a centred random variable and a 1 t a deterministic function

a 0 t (x, x) := -S -1 t U t (x -x) -S -1 t (ϵ t -εt ), a 1 t (x) := -Ŝ-1 t (V t x + Θ t ), and
ζ t := (X t -Xt ) ⊤ (U ⊤ t S -1 t U t )(X t -Xt ) + X⊤ t (V t Ŝ-1 t V t ) Xt + 2(U ⊤ t S -1 t (ϵ t -εt ) + V ⊤ t Ŝ-1 t Θ t ) ⊤ X t + (ϵ t -εt ) ⊤ S -1 t (ϵ t -εt ) + Θ ⊤ t Ŝ-1 t Θ t .
It yields that one could write (2.3.21) in the following form :

E[D α t ] = E [ (X t -Xt ) ⊤ ( . K t + ϕ 0 t )(X t -Xt ) + X⊤ t ( . Λ t + ψ 0 t ) Xt + 2( . Y t + ∆ 0 t ) ⊤ X t + . φ t -ρφ t + Γ0 t + α t -a 0 t (X t , Xt ) -ᾱt + Υ ⊤ t (ᾱ t -a 1 t ( Xt )) ) ⊤ S t (α t -a 0 t (X t , Xt ) -ᾱt + Υ ⊤ t (ᾱ t -a 1 t ( Xt )) ,
where

                 ϕ 0 t := ϕ t -U ⊤ t S -1 t U t = ϕ 0 t (K t ), ψ 0 t := ψ t -V ⊤ t Ŝ-1 t V t = ψ 0 t (K t , Λ t ), ∆ 0 t := ∆ t -U ⊤ t S -1 t (ϵ t -εt ) -V ⊤ t Ŝ-1 t Θ t = ∆ 0 t (K t , Λ t , Y t , Ȳt , Z Y t , Zt Y , R Y t , RY t ), Γ 0 t := Γ t -(ϵ t -εt ) ⊤ S -1 t (ϵ t -εt ) -Θ ⊤ t Ŝ-1 t Θ t = Γ 0 t (K t , Y t , Z Y t , R Y t ).
(2.3.26)

By choosing (K, Λ, Y, φ) s.t. for all t ∈ [0, T ], we have

. K t + ϕ 0 t = 0, . Λ t + ψ 0 t = 0, . Y t + ∆ 0 t = 0, . φ t -ρφ t + Γ0 t = 0, (2.3.27)
we obtain

E[D α t ] = E [ (α t -a 0 t (X t , Xt ) -ᾱt +Υ ⊤ t (ᾱ t -a 1 t ( Xt ))) ⊤ S t (α t -a 0 t (X t , Xt ) -ᾱ t + Υ ⊤ t (ᾱ t -a 1 t ( Xt ))) , (2.3.28)
which is non-negative for all t ∈ [0, T ], α ∈ A. This shows that S α satisfies the condition (ii) of the verification theorem (See Lemma 2.3.1).

In the following lemma, we study the existence of a unique solution to the system of ODEs-BSDE (2.3.17) satisfying (2.3.27).

Lemma 2.3.2. The decoupled system

                       dK t = -ϕ 0 t (K t )dt, 0 ≤ t ≤ T, K T = P, dΛ t = -ψ 0 t (K t , Λ t )dt, 0 ≤ t ≤ T, Λ T = P , dY t = -∆ 0 t (K t , Λ t , Y t , Ȳt , Z Y t , Zt Y , R Y t , R Y t )dt + Z Y t dW t , + χ R Y t (e)π(de, dt), 0 ≤ t ≤ T, Y T = L, dφ t = (ρφ t -E[Γ 0 t (K t , Y t , Z Y t , R Y t )])dt, 0 ≤ t ≤ T, φ T = 0, (2.3.29)
where the processes ϕ 0 , ψ 0 , ∆ 0 and Γ 0 are defined in (2.3.26), admits a unique solution

(K, Λ, Y, Z Y , R Y , φ) in L ∞ ([0, T ], S d ) ×L ∞ ([0, T ], S d ) ×S 2 F (Ω × [0, T ], R d ) ×L 2 F (Ω × [0, T ], R d ) × K 2 F (Ω × [0, T ] × χ, IR d ) × L ∞ ([0, T ], S d ). Proof.
• We start by introducing the following multi-dimensional Riccati-type equations

                               . K t + Q t -ρK t + K t A t + A ⊤ t K t + C ⊤ t K t C t + χ F ⊤ t (e)K t F t (e)λ(de) -I t + D ⊤ t K t C t + B ⊤ t K t + χ G ⊤ t (e)K t F t (e)λ(de) ⊤ N t + D ⊤ t K t D t + χ G ⊤ t (e)K t G t (e)λ(de) -1 I t + D ⊤ t K t C t + B ⊤ t K t + χ G ⊤ t (e)K t F t (e)λ(de) = 0, K T = P, (2.3.30)
Where the unknown is the matrix K. It is known that the equations (2.3.30) are related to the linear quadratic stochastic control problem:

v t (x) := inf α∈A E T t e -ρs ( Xα,x s ) ⊤ Q s Xα,x s + 2α ⊤ s I Xα,x s + α ⊤ s N α s ds + e -ρT ( Xα,x T ) ⊤ P Xα,x T , 62
where for α ∈ A, the process X is solution of the following SDE: • Given K, we consider the following equation for Λ:

        
                               . Λ t + Qt -ρΛ t + Λ t Ât + Â⊤ t Λ t + Ĉ⊤ t K t Ĉt + χ F ⊤ t (e)K t Ft (e)λ(de) -Ît + D⊤ t K t Ĉt + Bt Λ t + χ Ĝ⊤ t (e)K t Ft (e)λ(de) ⊤ Nt + D⊤ t K t Dt + χ Ĝ⊤ t (e)K t Ĝt (e)λ(de) -1
Ît + D⊤ t K t Ĉt + Bt Λ t + χ Ĝ⊤ t (e)K t Ft (e)λ(de) = 0, Λ T = P .

(

We rewrite this multi-dimensional Riccati equations as follows:

   . Λ t + QK t -ρΛ t + Λ t Ât + Â⊤ t Λ t -( ÎK t + B⊤ t Λ t ) ⊤ ( N K t ) -1 ( ÎK t + B⊤ t Λ t ) = 0 Λ T = P , (2.3.33)
where QK t , ÎK t , N K t are defined by

               QK t := Qt + Ĉ⊤ t K t Ĉt + χ F ⊤ t (
= Qt -( Ît ) ⊤ ( Nt ) -1 Ît + ( Ĉt -Dt N -1 t Ît ) ⊤ K( Ĉt -Dt N -1 t Ît ) -( D⊤ t K( Ĉt -D⊤ t ( Nt ) -1 Ît )) ⊤ ( Nt + D⊤ t K Dt ) -1 ( D⊤ t K t ( Ĉt -D⊤ t ( Nt ) -1 Ît )) + X ( Ft (e) -Ĝt (e) N -1 t Ît ) ⊤ K( Ft (e) -Ĝt (e) N -1 t Ît )λ(de) - X ( Ĝt (e) ⊤ K t ( Ft (e) -Ĝ⊤ t (e)( Nt ) -1 Ît )) ⊤ λ(de)( N K t ) -1 X ( Ĝt (e) ⊤ K( Ft (e) -Ĝ⊤ t (e)( Nt ) -1 Ît ))λ(de) As K t ≥ 0, we have N K t ≥ Nt ≥ ξI m > 0, then ( D⊤ t K( Ĉt -D⊤ t ( Nt ) -1 Ît )) ⊤ ( N K t ) -1 ( D⊤ t K t ( Ĉt -D⊤ t ( Nt ) -1 Ît )) (2.3.35) ≤ ( D⊤ t K( Ĉt -D⊤ t ( Nt ) -1 Ît )) ⊤ ( D⊤ t K Dt ) -1 ( D⊤ t K t ( Ĉt -D⊤ t ( Nt ) -1 Ît )) As K ∈ L ∞ ([0, T ], S d )
QK t -( ÎK t ) ⊤ ( N K t ) -1 ÎK t ≥ Qt + Î⊤ t ( Nt ) -1 Ît ≥ 0
As in the first step and according to [START_REF] Tang | Linear-quadratic optimal control problems for mean-field backward stochastic differential equations with jumps[END_REF] and to Yong and Zhou [START_REF] Yong | Stochastic Controls: Hamiltonian Systems and HJB Equations[END_REF], we deduce that equation (2.3.33) admits a unique solution Λ ∈ L ∞ ([0, T ], S d ) with Λ t ≥ 0.

• Given (K, Λ), we consider the following mean field BSDE

     dY t = -∆ 0 t (K t , Λ t , Y t , Ȳt , Z Y t , Zt Y , R Y t , RY t )dt + Z Y t dW t + χ R Y t (e)π(de, dt) Y T = L.
(2.3.37)

It could be written in the following form

                     dY t = (c t + θ ⊤ t (Y t -E[Y t ]) + θ⊤ t E[Y t ] + ϑ ⊤ t (Z Y t -E[Z Y t ]) + θ⊤ t E[Z Y t ] + χ ϱ ⊤ t (e)(R Y t (e) -E[R Y t (e)])λ(de) + χ ρ⊤ t (e)E[R Y t (e)]λ(de) dt +Z Y t dW t + χ R Y t (e)π(de, dt), Y T = L, (2.3.38)
where the stochastic process

(c t ) t ∈ L 2 IF (Ω × [0, T ], IR d ) is defined by, ∀t ∈ [0, T ] P p.s c t := -M t -K t (β t -βt ) -Λ t βt -C ⊤ t K t (γ t -γt ) -Ĉ⊤ t K t γt - χ F ⊤ t (e)K t (δ t (e) -δt (e))λ(de) - χ F ⊤ t (e)K t δt (e)λ(de) + U ⊤ t S -1 t H t -Ht + D ⊤ t K t (γ t -γt ) + χ G t (e) ⊤ K t (δ t (e) -δt (e))λ(de) + V ⊤ t S -1 t ( Ht + D⊤ t K t γt + χ Ĝ⊤ t (e)K t δt (e)λ(de)),
and the deterministic coefficients θ, θ, ϑ, θ ∈ L ∞ ([0, T ], IR d×d ), and ϱ, ρ ∈ K ∞ ([0, T ] × χ, IR d×d ) are defined by

θ t := ρI d -A t + B t S -1 t U t , θt := ρI d -Ât + Bt Ŝ-1 t V t , ϑ t := -C t + D t S -1 t U t , θt := -Ĉt + Dt Ŝ-1 t V t , ϱ t (e) := -F t (e) + G t (e)S -1 t U t , ρt (e) := -Ft (e) + Ĝt (e) Ŝ-1 t V t .
By Tang and Meng [START_REF] Tang | Linear-quadratic optimal control problems for mean-field backward stochastic differential equations with jumps[END_REF] (see Lemma 2.2.), under (H1) the BSDE (2.3.37) admits a unique

solution (Y, Z Y , R Y ) ∈ S 2 F (Ω × [0, T ], R d ) ×L 2 F (Ω × [0, T ], R d ) × K 2 F (Ω × [0, T ] × χ, IR d ).
• Given (K, Λ, Y, Z Y , R Y ), the linear ordinary differential equation for φ:

dφ t = (ρφ t -E[Γ 0 t ])dt, φ T = 0, (2.3.39)
where Γ 0 t is defined by (2.3.26), admits a unique explicit solution given by:

φ t = ⊤ t e -ρ(s-t) l s ds.
The deterministic function l is defined, for all t ∈ [0, T ] by:

l t := -E γ ⊤ t K t γ t + 2β ⊤ t Y t + 2δ ⊤ t Z Y t + 2 χ δ ⊤ t R Y t (e)λ(de) + χ δ ⊤ t K t δ t λ(de) -ϵ ⊤ t S -1 t ϵ t -ε⊤ t S -1 t εt + Θ ⊤ t Ŝ-1 t Θ t .
The next theorem gives a connection between the solution to the decoupled system (2.

α * t = -S -1 t U t (X * t --X * t -) -S -1 t (ϵ t --εt -) -S -1 t (V t X * t -+ Θ t ), dt ⊗ P -a.e.
where X * = X α * is the state process, and α * is the optimal control. The corresponding value of the problem is

V 0 = J(α * ) = E (X 0 -X0 ) T K 0 (X 0 -X0 ) + XT 0 Λ 0 X0 + 2Y T 0 X 0 + φ 0 .
Proof. We proved in Lemma 2.

α * t -a 0 t (X * t , X * t ) -ᾱ * t + Υ ⊤ t ( ᾱ * t -a 1 t ( X * t )) = 0 dt ⊗ dP a.e. ( 2

.3.40)

Taking the expectation and using the fact that E[a 0 t (X * t , X * t )] = 0 and Υ t is an invertible matrix, we get ᾱ * t -a 1 t ( X * t ) = 0. Then we obtain

α * t = a 0 t (X * t , X * t ) + a 1 t ( X * t ).
As the strategy α * must be IF -predictable and the number of jumps of the state process X is finite a.s. over the time interval [0, T ], then the optimal strategy α * satisfies

α * t = -S -1 t U t (X * t --X * t -) -S -1 t (ϵ t --εt -) -S -1 t (V t X * t -+ Θ t ).
Under Assumptions (H1)-(H2), S -1 , Ŝ-1 , U , V are bounded and Θ, ϵ are square-integrable

respectively in L 2 ([0, T ], IR m ) and L 2 F (Ω × [0, T ], IR m ).
As X * satisfies the square integrability condition (See inequality (2.2.6)), then α * ∈ A. We proved that there exists a random field w t (x, x) that satisfies the assumptions of Lemma 2.3.2. Then, by the verification theorem, we conclude that α * is the optimal control for the LQMF problem (2.3.14).

Extensions of LQMF problem on the case of common noise

In this section, we extend the results of Theorem 2.3.3, to the case with a common noise. Let W and W 0 be two independent real Brownian motions and π be an Poisson random measure defined on the same filtered probability space (Ω, F t , (P, P 0 )) where IF = (F t ) t∈[0,T ] be the filtration generated by the (W, W 0 , π) and we denote by IF 0 = {F 0 t } t∈[0,T ] the filtration generated by W 0 . As in section (2.3), for any r.v. X 0 and α ∈ A, the controlled process X α t is defined by:

           dX α t = b t (X α t , E[X α t |F 0 t ], α t , E[α t |W 0 t ])dt + σ t (X α t , E[X α t |F 0 t ], α t , E[α t |F 0 t ])dW t + σ 0 t (X α t , E[X α t |F 0 t ], α t , E[α t |F 0 t ])dW 0 t + X R t (X α t -, E[X α t -|F 0 t ], α t , E[α t |F 0 t ])(e)π(de, dt), X α 0 = X 0 , (2.4.41)
where for each t ∈ [0, T ], x, x ∈ IR d and a, ā ∈ IR m we set:

b t (x, x, a, ā) := β t + A t x + Ãt x + B t a + Bt ā, (2.4.42)
σ t (x, x, a, ā) := γ t + C t x + Ct x + D t a + Dt ā, σ 0 t (x, x, a, ā) := γ 0 t + C 0 t x + C0 t x + D 0 t a + D0 t ā, R t (x,
x, a, ā)(e) := δ t (e) + F t (e)x + Ft (e)x + G t (e)a + Gt (e)ā.

For this case, we assume that

(H3) (i) A, Ã, B, B, C, C, D, D, C 0 , C0 , D 0 , D0 , F , F , G, G are essentially bounded IF 0 - adapted processes, (ii) β, γ, γ 0 are square-integrable IF -adapted processes and δ is square-integrable IF - predictable process.
The LQMF control problem is to find α * ∈ A s.t.

V 0 := J(α * ) = inf α∈A J(α), (2.4.43)
where As in the previous section, we guess a quadratic form for w t (x, x) i.e.

J(α) := E ⊤ 0 e -ρt f t (X α t , E[X α t |F 0 t ], α t , E[α t |F 0 t ])dt + e -ρT g(X α T , E[X α t |F 0 T ]) , ( 2
w t (x, x) = (x -x) T K t (x -x) + xT Λ t x + 2Y T t x + φ t ,
where the processes K, Λ, Y and φ are to be determined later. As the quadratic terms in f t and g are IF 0 -adapted, we guess that K and Λ are IF 0 -adapted. Since the affine coefficients in b t , σ t and σ 0 t and the linear coefficients in f t and g are IF -adapted, then Y is IF -adapted i.e. depends on W , W 0 and π. We look for processes

(K, Λ, Y, φ) valued in S d × S d × IR d × IR
and satisfy the following system:

                   dK t = . K t dt + Z K t dW 0 t , 0 ≤ t ≤ T, K T = P, dΛ t = . Λ t dt + Z Λ dW 0 t , 0 ≤ t ≤ T, Λ T = P , dY t = . Y t dt + Z Y t dW t + Z Y,0 t dW 0 t + χ R Y t (e)π(de, dt), 0 ≤ t ≤ T, Y T = L, dφ t = . φ t , 0 ≤ t ≤ T, φ T = 0, (2.4.45)
for some IF 0 -adapted processes φ valued in IR. We keep the notations of section 2.3 and we add the following notations:

Ĉ0 t = C 0 t + C0 t , D0 t = D 0 t + D0 t .
For α ∈ A and t ∈ [0, T ], we set S α in the following form:

S α t = e -ρt W α t + t 0 e -ρs f s (X α s , E[X α s |F 0 s ], α s , E[α s |F 0 s ])ds. (2.4.46)
As in Section 2.3, we compute

d E[S α t ] = e -ρt E[D α t ]dt,
where

E[D α t ] = E -ρw t (X α t , E[X α t |F 0 t ]) + d dt E[w t (X α t , E[X α t |F 0 t ])] + f t (X α t , E[X α t |F 0 t ], α t , E[α t |F 0 t ]) .
As in the previous section, we denote by Xt for E[X α t |F 0 t ] and ᾱt for E[α t |F 0 t ]. By applying Itô's formula to S α t and using the quadratic form of the cost f t and the dynamics of Xt and X t -Xt , we obtain:

d Xt = [β t + Ât Xt + Bt ᾱt ]dt, (2.4.47)
and

d(X t -Xt ) = [(β t -βt ) + A t (X t -Xt ) + B t (α t -ᾱt )]dt (2.4.48
)

+ [γ t + C t (X t -Xt ) + Ĉt Xt + D t (α t -ᾱt ) + Dt ᾱt ]dW t + [γ 0 t + C 0 t (X t -Xt ) + Ĉ0 t Xt + D 0 t (α t -ᾱt ) + D0 t ᾱt ]dW 0 t + χ [δ t + F t (e)(X t --Xt -) + Ft (e) Xt -+ G t (e)(α t -ᾱt ) + Ĝt (e) ᾱt ]π(de, dt).
Using the same arguments as in the previous section, we obtain:

E[D α t ] =E (X t -Xt ) T ( . K t + ϕ t )(X t -Xt ) + X⊤ t ( . Λ t + ψ t ) Xt (2.4.49) +2( . Y t + ∆ t )X t + . φ t -ρφ t + Γt + κ t (α) ,
where

                                                                               ϕ t := -ρK t + K t A t + A ⊤ t K t + Z K t C 0 t + (C 0 t ) ⊤ Z K t + C ⊤ t K t C t + (C 0 t ) ⊤ K t C 0 t + Q t + χ F ⊤ t (e)K t F t (e)λ(de) = ϕ t (K t , Z K t ), ψ t := -ρΛ t + Λ t Ât + Â⊤ t Λ t + Z Λ t C 0 t + (C 0 t ) ⊤ Z Λ t + Ĉ⊤ t K t Ĉt + (C 0 t ) ⊤ Γ t C 0 t + Qt + χ F ⊤ t (e)K t Ft (e)λ(de) = ψ t (K t , Λ t ), ∆ t := -ρY t + A ⊤ t Y t + Ã⊤ T Ȳt + C ⊤ t Z Y t + (C 0 t ) ⊤ Z Y,0 t + ( C0 t ) ⊤ Z Y,0 t + C⊤ t Z Y t +K t (β t -βt ) + Λ t βt + C ⊤ t K t (γ t -γt ) + Ĉ⊤ t K t γt + M t + Z K t (γ 0 t -γ0 t ) + Z Λ t γ0 t +(C 0 t ) ⊤ K t (γ 0 t -γ0 t ) + ( Ĉ0 t ) ⊤ Λ t γ0 t + χ F ⊤ t (e)(R Y t (e) -RY t (e))λ(de) + χ Ft (e) ⊤ RY t (e)λ(de) + χ F ⊤ t (e)(δ t (e) -δt (e))λ(de) + χ Ft (e) ⊤ δt (e)λ(de) = ∆ t (K t , Z K t , Λ t , Z Λ t , Y t , Ȳt , Z Y t , Z Y t , R Y t , R Y t ), Γ t := γ ⊤ t K t γ t + (γ 0 t -γ0 t ) ⊤ K t (γ 0 t -γ0 t ) + 2(δ t ) ⊤ Z Y,0 t + (γ 0 t ) ⊤ Λ t (γ 0 t ) + 2β ⊤ t Y t +2γ ⊤ t Z Y t + χ δ ⊤ t (e)K t δ t (e)λ(de) + 2 χ δ t (e) ⊤ R Y t (e)λ(de) = Γ t (K t , Y t , Z Y t , R Y t ), and
κ t (α) := (α t -ᾱt ) ⊤ S t (α t -ᾱt ) + ᾱ⊤ t Ŝt ᾱt + 2(U t (X t -Xt ) + V t Xt + Θ t + ϵ t -εt )α t . (2.4.50)
Here S t , Ŝt , U t , V t , Θ t are defined as follows:

                                     S t := N t + D ⊤ t K t D t + (D 0 t ) ⊤ K t D 0 t + χ G ⊤ t (e)K t G t (e)λ(de), Ŝt := Nt + D⊤ t K t Dt + ( D0 t ) ⊤ K t D0 t + χ Ĝ⊤ t (e)K t
Ĝt (e)λ(de),

U t := I t + D ⊤ t K t C t + B ⊤ t K t + (D 0 t ) ⊤ K t C 0 t + (D 0 t ) ⊤ Z K t + χ G ⊤ t (e)K t F t (e)λ(de), V t := Ît + D⊤ t K t Ĉt + Bt Λ t + (D 0 t ) ⊤ Z Λ t + (D 0 t ) ⊤ K t (C 0 t ) + χ Ĝt (
                   ϵ t := H t + D ⊤ t K t γ t + B ⊤ t Y t + D ⊤ t Z Y t + (D 0 t ) ⊤ Z Y,0 t + (D 0 t ) ⊤ K t (γ t ) + χ G ⊤ t (e)K t δ t λ(de) + χ G ⊤ t (e)R Y t (e)λ(de), εt := Ht + D ⊤ t K t γt + B ⊤ t Ȳt + D ⊤ t Z Y t + (D 0 t ) ⊤ Z Y,0 t + (D 0 t ) ⊤ K t (γ t ) + χ G ⊤ t (e)K t δt λ(de) + χ G ⊤ t (e)R Y t (e)λ(de).
(2.4.52)

Completing the square in the expression of E[D α t ] (See equation (2.4.49)) and getting rid of the terms which not depend in α, the computations lead to the following decoupled system,

                         dK t = -ϕ 0 t (K t , Z k )dt + Z K dW 0 t , 0 ≤ t ≤ T, K T = P, dΛ t = -ψ 0 t (K t , Λ t , Z Λ )dt + Z Λ dW 0 t , 0 ≤ t ≤ T, Λ T = P , dY t = -∆ 0 t (K t , Z K t , Λ t , Z Λ t , Y t , Ȳt , Z Y t , Zt Y , R Y t , RY t )dt + Z Y t dW t +Z Y,0 dW 0 t + χ R Y t (e)π(de, dt), 0 ≤ t ≤ T, Y T = L, dφ t = (ρφ t -E[Γ 0 t ])dt, 0 ≤ t ≤ T, φ T = 0, (2.4.53)
with

                 ϕ 0 t := ϕ t -U ⊤ t S -1 t U t , ψ 0 t := ψ t -V ⊤ t Ŝ-1 t V t , ∆ 0 t := ∆ t -V ⊤ t Ŝ-1 t Θ t -U ⊤ t S -1 t (ϵ t -εt ), Γ 0 t := Γ t -(ϵ t -εt ) ⊤ S -1 t (ϵ t -εt ) -Θ ⊤ t Ŝ-1 t Θ t .
(

α * t = -S -1 t U t (X * t --E[X * t -|F 0 t ]) -S -1 t (ϵ t --E[ϵ t -|F 0 t ]) -Ŝ-1 t (V t -E[X * t -|F 0 t ] + Θ t ).
Proof. The proof is similar to that of Theorem 2.3.3.

Linear quadratic mean field control problem on infinite horizon

Let us consider the infinite horizon case. For ρ > 0, we define the set of admissible controls as follows:

A := {α : Ω × IR + → IR m s.t α is IF -predictable and ∞ 0 e -ρt E[|α t | 2 ]dt < ∞}.
The controlled process is defined on [0, ∞) by:

         dX α t = b t (X α t , E[X α t ], α t , E[α t ])dt + σ t (X α t , E[X α t ], α t , E[α t ])dW t + X R t (X α t -, E[X α t -], α t , E[α t ])(e)π(de, dt), X α 0 = X 0 , ( 2.5.55)
where for each t ≥ 0 ,x, x ∈ IR d , we use the same formulation as in Section 2.3:

b t (x, x, a, ā) := β t + Ax + Ãx + Ba + Bā, (2.5
.56)

σ t (x, x, a, ā) := γ t + Cx + C x + Da + Dā, R t (x, x, a, ā)(e) := δ t (e) + F (e)x + F (e)x + G(e)a + G(e)ā.
For the infinite horizon case, the coefficients of the linear terms are constant vectors, and the coefficients β, γ and δ are stochastic processes. The control problem on infinite horizon is formulated as:

V 0 := inf α∈A J(α), (2.5.57)
where

J(α) := E ∞ 0 e -ρt f t (X α t , E[X α t ], α t , E[α t ])dt , (2.5.58)
and, for each t ≥ 0, x, x ∈ IR d and a, ā ∈ IR m , we define f as in Section 2.2 i.e.

f t (x, x, a, ā) :=(x -x) ⊤ Q(x -x) + x⊤ (Q + Q)x + 2a ⊤ I(x -x) (2.5.59) + 2ā ⊤ (I + Ĩ)x + (a -ā) ⊤ N (a -ā) + ā(N + Ñ )ā + 2M ⊤ t x + 2H ⊤ t a.
Here, we note that the coefficients of the quadratic terms are constant matrices, and the coefficients H and M may be stochastic processes.

Let (H, |.|) be a normed space, and ρ a positive constant. We define the new spaces:

• L ∞ (IR + , H) := ϕ : IR + → H s.t. ϕ is measurable and sup t≥0 |ϕ t | < ∞ a.e. , • K ∞ (χ, H) := {K : χ → H, K is B(χ)-measurable and sup e∈χ |K(e)| < ∞}, • L 2,ρ F (Ω × IR + , H) := { ϕ : Ω × IR + → H s.t. ϕ is F-progressively measurable and ∞ 0 e ρt E[|ϕ t | 2 ]dt < ∞} • K 2,ρ F (Ω×IR + ×χ, H) := {K : Ω×IR + ×χ → H, K is P⊗B(χ)-measurable processes and E ∞ 0 χ e ρt |K t (e)| 2 λ(de)dt < ∞},
where P denote the σ-field of F-predictable sets on Ω × IR + .

We consider the following assumptions on the coefficients of the problem in the infinite horizon case.

(H1

′ ) The coefficients in equation (2.5.55) satisfy:

(i) β, γ ∈ L 2,ρ F (Ω × IR + , IR d ) and δ ∈ K 2,ρ F (Ω × IR + × χ, IR d ), (ii) A, Ã, C, C ∈ IR d×d and B, B, D, D, J, J ∈ IR d×m , (iii) F, F ∈ K ∞ (χ, IR d×d ) and G, G ∈ K ∞ (χ, IR d×m ).

(H2

′ ) The coefficients in equation (2.5.59) satisfy:

(i) Q, Q ∈ S d , N, Ñ ∈ S m , I, Ĩ ∈ IR m×d , (ii) M ∈ L 2,ρ F (Ω × IR + , IR d ), H ∈ L 2,ρ F (Ω × IR + , IR m ), (iii) N > 0, Q + I T N -1 I ≥ 0, (iv) N + Ñ > 0, (Q + Q) -(I + Ĩ) T (N + Ñ ) -1 (I + Ĩ) ≥ 0. (H3 ′ ) ρ > 2 max{|A| + |C| 2 + χ |F (e)| 2 λ(de), | Â|}.
Proposition 2.5.1. Under (H1 ′ ) and (H3 ′ ), the following estimate holds for each square-integrable variable X 0 and α ∈ A,

∞ 0 e -ρt E[|X α t | 2 ]dt ≤ C α (1 + E[|X 0 | 2 ]), (2.5.60)
where C α is a positive constant.

Proof. By Itô's formula and Young's inequality, we have the following estimate

d dt E[e -ρt | Xt | 2 ] = e -ρt (-ρ| Xt | 2 + 2 b⊤ t Xt ) ≤ e -ρt (-ρ| Xt | 2 + 2(| βt || Xt | + | B||ᾱ t || Xt | + X⊤ t  Xt )) ≤ e -ρt [(-ρ + 2| Â| + ϵ)| Xt | 2 + c ϵ (| βt | 2 + |ᾱ t | 2 )],
where c ϵ is a positive constant. We define:

ζ ϵ := | X0 | 2 + c ϵ ∞ 0 e -ρt E[|β 2 t | + |α 2 t |], η ϵ := ρ -2| Â| -ϵ.
Under (H1 ′ ) and as α ∈ A, we have ζ ϵ < ∞ and for ϵ small enough, we have η ϵ > 0. By Gronwall's lemma, we get:

∞ 0 e -ρt | Xt | 2 dt ≤ c α,ϵ (1 + | X0 | 2 ), (2.5.61)
where c α,ϵ is positive constant. Using again Itô's formula and Young's inequality, we have the following estimate:

d dt E[e -ρt |X t -Xt | 2 ] = e -ρt E[-ρ|X t -Xt | 2 + 2(b t -bt ) ⊤ (X t -Xt ) + |σ t | 2 + χ |R t (e)| 2 λ(de)] ≤ e -ρt E[-ρ|X t -Xt | 2 + 2(|β t -βt ||X t -Xt | + |B||α t -ᾱt ||X t -Xt | + (X t -Xt ) ⊤ A(X t -Xt )) + 2(|γ t | 2 + |C| 2 |X t -Xt | 2 + | Ĉ| 2 | Xt | 2 + |D| 2 |α t | 2 + | D| 2 |ᾱ t | 2 ) + 2( χ |δ t (e)| 2 λ(de) + χ |F (e)| 2 λ(de)|X t -Xt | 2 + χ | F (e)| 2 λ(de)| Xt | 2 + χ |G(e)| 2 λ(de)|α t | 2 + χ | G(e)| 2 λ(de)|ᾱ t | 2 )] ≤ e -ρt E[(-ρ + 2|A| + 2|C| 2 + 2 χ |F (e)| 2 λ(de) + ϵ)|X t -Xt | 2 + c ′ ϵ (|β t | 2 + |γ t | 2 + χ |δ t (e)| 2 λ(de)| + |α t | 2 + | Xt | 2 )],
where c ϵ is a positive constant. We define:

ζ ′ ϵ := c ′ ϵ ∞ 0 e -ρt E[|β t | 2 + |γ t | 2 + χ |δ t (e)| 2 λ(de)| + |α t | 2 + | Xt | 2 ]dt, η ′ ϵ := ρ -2|A| -2|C| 2 -2 χ |F (e)| 2 λ(de) -ϵ.
Under (H1 ′ ), inequality (2.5.61) and as α ∈ A, we have ζ ′ ϵ < ∞ and for ϵ small enough, we have η ′ ϵ > 0. By Gronwall's lemma, we get

E[ ∞ 0 e -ρt |X t -Xt | 2 dt] ≤ c ′ α,ϵ (1 + | X0 | 2 ), (2.5
W α t = w t (X α t , E[X α t ]) for some F-adapted random field {w t (x, x), t ∈ [0, T ], x, x ∈ IR d } satisfying w t (x, x) ≤ C(χ t + |x| 2 + |x| 2 ), t ∈ [0, ∞), x, x ∈ IR d ,
for some positive constant C, and non-negative process χ s.t. e -ρt E[χ t ] converge to zero as t → ∞, and such that:

(i) the map t ∈ IR + -→ E[S α t ], with S α t = e -ct W α t + t 0 e -cs f s (X α s , E[X α s ], α t , E[α t ])ds, is non-decreasing for all α ∈ A, (ii) the map t ∈ IR + -→ E[S α * t ] is constant for some α * ∈ A.
Then, α * is an optimal control and E[w 0 (X 0 , E[X 0 ])] is the value of the LQMKV control problem (2.5.57) i.e.

V 0 = E[w 0 (X 0 , E[X 0 ])] = J(α * ).
Moreover, any other optimal control satisfies the condition (ii).

We extend the results in Theorem 2.3.3 to the infinite horizon case, where we kept the steps similar to the finite horizon case. We prove our result by applying the lemma 2.5.2. We should look for the stability of decoupled system on infinite horizon.

We adopt the same approach as in the finite time horizon. We cnsider a candidate for the random field ω t (x, x) in the form:

w t (x, x) = (x -x) T K t (x -x) + xT Λ t x + 2Y T t x + φ t ,
where (K, Λ, Y, φ) valued in S d × S d × IR d × IR satisfying the following system:

                   dK t = -ϕ 0 t dt, t ≥ 0, dΛ t = -ψ 0 t dt, t ≥ 0, dY t = -∆ 0 t dt + Z Y t dW t + χ R Y t (e)π(de, dt), t ≥ 0, dφ t = (ρφ t -E[Γ 0 t ])dt, t ≥ 0.
(2.5.63)

The maps ϕ 0 , ψ 0 , ∆ 0 , Γ 0 are defined by (2.3.26), where in this case the coefficients A, Ã, B, B,

C, C, D, D, Q, Q, N , Ñ , I, Ĩ, F (.
), F (.), G(.), G(.) are constant i.e. independent of time.

We note that there are no terminal conditions in the system, as we are in the infinite horizon case. We need to show the existence of a solution to the system (2.5.63).

Lemma 2.5.3. We assume (H1 ′ )-(H2 ′ ). Then, the system (2.5.63) admits a solution

(K, Λ, Y, Z Y , R Y , φ) ∈ L ∞ ([0, ∞], S d )×L ∞ ([0, ∞], S d )×L 2,ρ F (Ω×[0, ∞], R d )×L 2,ρ F (Ω×[0, ∞], R d )× K 2,ρ (Ω × [0, ∞] × χ, IR d ) × L ∞ ([0, ∞], S d ).
Proof. We prove the existence of a solution to the decoupled system (2.5.63).

• We introduce the following Riccati-type equation:

Q -ρK + KA + A ⊤ K + C ⊤ KC + χ F ⊤ (e)KF (e)λ(de) -I + D ⊤ KC + B ⊤ K + χ G ⊤ (e)KF (e)λ(de) ⊤ N + D ⊤ KD + χ G ⊤ (e)KG(e)λ(de) -1 I + D ⊤ KC + B ⊤ K + χ G ⊤ (e)KF (e)λ(de) = 0.
(2.5.64)

We prove the existence of a solution to (2.5.64) by relating it to a suitable infinite-horizon linear-quadratic control problem. For T ∈ IR + ∪{∞} and x ∈ IR d , we consider the following control problem:

V T (x) := inf α∈A T E T 0 e -ρt ( Xα,x t ) ⊤ Q Xα,x t + 2α ⊤ t I Xα,x t + α ⊤ t N α t dt ,
where A T is defined by:

A T := {α : Ω × IR + → IR m s.t α is IF predictable and T 0 e -ρt E[|α t | 2 ]dt < ∞},
and for α ∈ A T , the process X := Xα,x is solution of the following SDE:

d Xt = (A Xt +Bα t )dt+(C Xt +Dα t )dW t + X (F (e)
Xt +G(e)α t )π(de, dt), X0 = 0. (2.5.65)

Thanks to the integrability condition for α ∈ A T , we have

T 0 e -ρt E[| Xα,x t | 2
]dt < ∞, and so the problems V T are well-defined for any T ∈ R + ∪ {∞}. If T < ∞, as already recalled in the finite-horizon case, (H1 ′ )-(H2 ′ ) imply that there exists a unique symmetric solution

(K T t ) t∈[0,T ] to Riccati equations:                                d dt K T t + Q -ρK T t + K T t A + A ⊤ K T t + C ⊤ K T t C + χ F ⊤ (e)K T t F (e)λ(de) -I + D ⊤ K T t C + B ⊤ K T t + χ G ⊤ (e)K T t F (e)λ(de) ⊤ N + D ⊤ K T t D + χ G ⊤ (e)K T t G(e)λ(de) -1 I + D ⊤ K T t C + B ⊤ K T t + χ G ⊤ (e)K T t F (e)λ(de) = 0, K T T = 0, (2.5.66)
and that for every x ∈ IR d we have

V T (x) = x ⊤ K T 0 x. It is easy to check from the definition of V T that V T (x) → V ∞ (x)
as T goes to infinity, from which we deduce that

V ∞ (x) = lim T →∞ x ⊤ K T 0 x = x lim T →∞ K T 0 x, for all x ∈ IR d .
This implies the existence of the limit K = lim T →∞ K T 0 . By passing to the limit in T in ODE (2.5.66) at t = 0, we obtain by standard arguments (see Lemma 2.8 in [?]), that K satisfies (2.5.64). Moreover, K ∈ S d and K ≥ 0.

• Given K, we consider the following equation of Λ:

QK -ρΛ + Λ Â + Â⊤ Λ -( ÎK + B⊤ Λ) ⊤ ( N K ) -1 ( ÎK + B⊤ Λ) = 0, (2.5.67)
where Existence of a solution to (2.5.67) is obtained by the same arguments used for (2.5.64) under (H2 ′ ).

               QK := Q + Ĉ⊤ K Ĉ + χ F ⊤ (
• Given (K, Λ), we consider the following mean field BSDE with jumps on infinite horizon:

dY t = c t + (ρI d + θ) ⊤ (Y t -E[Y t ]) + (ρI d + θ) ⊤ E[Y t ] + ϑ ⊤ (Z Y t -E[Z Y t ]) + θ⊤ E[Z Y t ] + χ ϱ ⊤ (e)(R Y t (e) -E[R Y t (e)])λ(de) + χ ρ⊤ (e)E[R Y t (e)]λ(de) dt + Z Y t dW t + χ R Y t (e)π(de, dt), (2.5.68)
where the stochastic process c ∈ L 2,ρ

F (Ω × [0, ∞], R d
) is defined by:

c t := -M t -K (β t -βt ) -Λ βt -C ⊤ K (γ t -γt ) -Ĉ⊤ K γt - χ F ⊤ (e) K (δ t (e) -δt (e))λ(de) - χ F ⊤ (e) K δt (e)λ(de) + U ⊤ S -1 H t -Ht + D ⊤ K(γ t -γt ) + χ G(e) ⊤ K (δ t (e) -δt (e))λ(de) + V ⊤ Ŝ-1 Ht + D⊤ K γt + χ
Ĝ⊤ (e)K δt (e)λ(de) , the coefficients θ, θ, ϑ, θ are constant in IR d and are defined by:

θ := -A + B S -1 U, θ := -Â + B Ŝ-1 V, ϑ := -C + D S -1 U, θ := -Ĉ + D Ŝ-1 V,
and ϱ, ρ : χ -→ IR d are defined by: ϱ(e) := -F (e) + G(e)S -1 U, ∀e ∈ χ, ρ(e) := -F (e) + Ĝ(e) Ŝ-1 V, ∀e ∈ χ.

To simplify the notations let us denote:

-f (t, Y t , Z t , R t , E[Y t ], E[Z t ], E[R t ]) = c t + (ρI d + θ) ⊤ (Y t -E[Y t ]) + (ρI d + θ) ⊤ E[Y t ] + ϑ ⊤ (Z t -E[Z t ]) + θ⊤ E[Z t ] + χ ϱ ⊤ (e)(R t (e) -E[R t (e)])λ(de) + χ ρ⊤ (e)E[R t (e)]λ(de).
We prove that the following linear BSDE with jumps defined by: for t ≥ 0,

     dY t = -f (t, Y t , Z t , R t , E[Y t ], E[Z t ], E[R t ])dt + Z t dW t + χ R t (e)π(de, dt) lim t→∞ Y t = 0.
(2.5.69)

has a solution (Y, Z, R) in L 2,ρ F (Ω × IR + , R d ) × L 2,ρ F (Ω × IR + , R d ) × K 2,ρ (Ω × IR + × χ, IR d ),
where ρ is a positive constant which will be fixed later.

Existence: Let (Y n , Z n , R n ) be a solution on [0, n] of the following BSDE Y n t = n t f n (s, Y n s , Z n s , R n s , E[Y n s ], E[Z n s ], E[R n s ])ds - n t Z n s dB s - n t χ R n t (e)π(de, ds), t ∈ [0, n],
where

f n (t, Y n t , Z n t , R n t , E[Y n t ], E[Z n t ], E[R n t ]) = c t 1 [0,n] (t) + (ρI d + θ) ⊤ (Y n t -E[Y n t ]) + (ρI d + θ) ⊤ E[Y n t ] + ϑ ⊤ (Z n t -E[Z n t ]) + θ⊤ E[∆Z n t ] + χ ϱ ⊤ (e)(R n t (e) -E[R n t (e)])λ(de) + χ ρ⊤ (e)E[R n t (e)]λ(de),
and we take (Y n t , Z n t , R n t ) = (0, 0, 0) on (n, ∞). We fix m > n. Applying Itô's formula to e ρt |Y m t -Y n t | 2 , we get for all t ≥ 0,

e ρT |Y m T -Y n T | 2 -e ρt |Y m t -Y n t | 2 = T t e ρs ρ|Y m s -Y n s | 2 + |Z m s -Z n s | 2 + χ |(R m s -R n s )(e)| 2 λ(de) ds -2 T t e ρs (Y m s -Y n s ) ⊤ ∆ n,m f s ds + 2 T t e ρs (Y m s -Y n s ) ⊤ (Z m s -Z n s )dB s + χ e ρs (Y m s --Y n s -) ⊤ (R m s -R n s )(e)π(de, ds),
where

∆ n,m f s := f m (s, Y m s , Z m s , R m s , E[Y m s ], E[Z m s ], E[R m s ])-f n (s, Y n s , Z n s , R n s , E[Y n s ], E[Z n s ], E[R n s ]).
We focus on the dependence in Y , we obtain:

e ρT |Y m T -Y n T | 2 -e ρt |Y m t -Y n t | 2 = T t e ρs ρ|Y m s -Y n s | 2 + |Z m s -Z n s | 2 + χ |(R m s -R n s )(e)| 2 λ(de) ds -2 T t e ρs ((Y m s -Y n s ) -E[Y m s -Y n s ]) ⊤ (ρI d + θ)(Y m s -Y n s )ds -2 T t e ρs E[Y m s -Y n s ] ⊤ (ρI d + θ)(Y m s -Y n s )ds -2 T t e ρs (Y m s -Y n s ) ⊤ ∆ n,m f 0 s ds + 2 T t e ρs (Y m s -Y n s ) ⊤ (Z m s -Z n s )dB s + χ e ρs (Y m s --Y n s -) ⊤ (R m s -R n s )(e)π(de, ds).
where

∆ n,m f 0 s := f m (s, 0, Z m s , R m s , 0, E[Z m s ], E[R m s ]) -f n (s, 0, Z n s , R n s , 0, E[Z n s ], E[R n s ]).
Taking the expectation, the contribution of the stochastic integrals vanishes. Using the Young's inequality 2ab ≤ ϵa 2 + 1 ϵ b 2 , where ϵ > 0, and Cauchy Schwarz's inequality, we obtain:

2E T t e ρs ((Y m s -Y n s ) -E[Y m s -Y n s ]) ⊤ (ρI d + θ)(Y m s -Y n s )ds + 2E T t e ρs E[Y m s -Y n s ] ⊤ (ρI d + θ)(Y m s -Y n s )ds ≤ 2ρE T t e ρs |Y m s -Y n s | 2 ds + 2 T t e ρs |θ|E |Y m s -Y n s | 2 ds + 2 T t e ρs |θ|E [|Y m s -Y n s |] 2 ds + 2 T t e ρs | θ|E [|Y m s -Y n s |] 2 ≤ 2ρE T t e ρs |Y m s -Y n s | 2 ds + 2 T t e ρs (2|θ| + | θ|)E |Y m s -Y n s | 2 ds.
Similar calculus for the term T t e ρs (Y m s -Y n s ) ⊤ ∆ n,m f 0 s ds, shows that:

E T 0 e ρs ρ|Y m s -Y n s | 2 + |Z m s -Z n s | 2 + χ |(R m s -R n s )(e)| 2 λ(de) ds ≤ E[ e ρT |Y m T -Y n T | 2 ] + (δ + 1 ϵ + 2ρ + C) E T 0 e ρs |Y m s -Y n s | 2 ds + Cϵ E T 0 e ρs |Z m s -Z n s | 2 ds + E T 0 χ e ρs |(R m s -R n s )(e)| 2 λ(de)ds + 1 δ E T 0 e ρs |c s | 2 1 [n,m] (s)ds .
where

C := 2|θ| + | θ| + 2|ϑ| + | θ| + 2 χ |ϱ(e)| 2 λ(de) + χ |ρ(e)| 2 λ(de). Under (H1 ′ )(iii), C is finite. By choosing 0 < ϵ < 1 2C , ρ and δ > 0 s.t. ρ > δ + 1 ϵ + 2ρ + C + 1 2
, we deduce:

E T 0 e ρs |Y m s -Y n s | 2 + |Z m s -Z n s | 2 + χ |(R m s -R n s )(e)| 2 λ(de) ds ≤ 2 δ E T 0 e ρs |c s | 2 1 [n,m] (s)ds .
Sending T to infinity, by the monotone convergence theorem, we obtain:

E ∞ 0 e ρs |Y m s -Y n s | 2 + |Z m s -Z n s | 2 + χ |(R m s -R n s )(e)| 2 λ(de) ds ≤ 2 δ E ∞ 0 e ρs |c s | 2 1 [n,m] (s)ds . As c ∈ L 2,ρ F (Ω × [0, ∞], R d ), then |c s | 2 1 [n,m] (s) -→ 0, dt ⊗ dP a.e.
, s ≥ 0 when n goes to infinity. By using the dominated convergence theorem for the right hand side, we deduce that the sequence

(Y n , Z n , K n ) is a Cauchy sequence in L 2,ρ F (Ω × [0, ∞], R d ) × L 2,ρ F (Ω × [0, ∞], R d ) × K 2,ρ (Ω × [0, ∞] × χ, IR d
) and that the limit (Y, Z, K) is a solution of a MF BSDE with jumps (2.5.69).

• Given (K, Λ, Y, Z Y , R Y ), the linear ordinary differential equation for φ

dφ t = (ρφ t -E[Γ 0 t ])dt. (2.5.70)
where

Γ 0 t = Γ t -(ϵ t -εt ) ⊤ S -1 t (ϵ t -εt ) -Θ ⊤ t Ŝ-1 t Θ t is defined in (2.3.26
), admits a unique explicit solution given by: φ

t = ∞ t e -ρ(s-t) E[Γ 0 s ]ds.
Then, we proved the existence of solutions of the decoupled system (2.5.63).

The following theorem gives the structure of the optimal control for LQMF problem (2.5.57).

Theorem 2.5.4. Under Assumptions (H1 ′ )-(H2 ′ ), the optimal control for LQMF problem (2.5.57) is given by

α * t = -S -1 U (X * t --X * t -) -S -1 (ϵ t --εt -) -S -1 (V X * t -+ Θ t ),
where X * = X α * is the state process where the α * is the optimal control and the deterministic coefficients S, Ŝ, U and V and the stochastic coefficients ϵ t and Θ t are defined in Section 2.3.

Application to production of an exhaustible resource

In this section, we study a model of production of exhaustible resource with accumulating or maintaining a level of reserves, inspired by a series of works extended from the Hotelling's model [START_REF] Hotelling | The economics of exhaustible resources[END_REF]. In the classic Hotelling's model, the dynamics market's evolution is driven by the use of existing reserves of an exhaustible reserves to produce energy without possibility to exploration and/or discovery of new reserves. But many studies have made it possible to ensure that there are still resources to be explored over time, that is to say that the reservation rate can be increased. We can refer to the series of works extended from Prindyck's model [107],

Deshmukh et al. [START_REF] Deshmukh | Optimal consumption and exploration of nonrenewable resources under uncertainty[END_REF], Arrow and Chang [START_REF] Arrow | Optimal pricing, use, and exploration of uncertain resource stocks[END_REF], and Keller et al. [START_REF] Hagan | Optimal pricing, use and exploration of uncertain natural resources[END_REF]. The increase in reserve discoveries occurs stochastically via the Poisson process. It should be noted that this increase is smaller, and it is the reason that the resources always remain exhaustible.

We consider an energy market with N producers (players). Each producer uses exhaustible resources, such as oil, to produce energy. The quantity X i t represents the reserve's level of player i, at time t, i = 1, ..., N . It takes values in the set IR + . The reserve level X i t decreases at a controlled production rate α i t ≥ 0 dt ⊗ dP a.e., and also has random discrete increment due to exploration. We use N independent Poisson point process π i , i = 1...N to model the new discoveries and we denote by λ i (de)dt the associated compensator. We assume that the dynamics of the reserve has a noise which is proportional to the current level of the reserve. The reserve's dynamics of each producer i is given by the following stochastic differential equation:

   dX i t = -α i t dt + σX i t dW i t + η X X i t -πi (de, dt), X i 0 = x i 0 ,
(2.6.71)

where x i 0 is the initial reserve's level of player i, σ > 0, W i , i = 1...N are independent standard Brownian motion and independent of π i , i = 1...N , and η > 0 is the rate of new discoveries. The cost functional for producer i is given by:

J i (α 1 , ..., α n ) := E ∞ 0 e -ρ t -α i t P t (α i t ) + C p (α i t ) + C ex (α i t , X i t ) dt , (2.6.72)
where

• P i t is the selling price for producer i. It follows a linear inverse demand rule, defined as:

P i t := P t (α i t ) = P 0 t -δ α i t -ε t 0 1 N N j=1 α j s ds.
Here δ, ε are positive constants and P 0 is a deterministic function. It will be the same for all producers. The price P i of producer i is related to his production and also to the production of all other producers.

• C p is the cost functions of production, defined as:

C p (α i t ) := c 1 Var(α i t ).
• C ex is the cost functions of exploration defined as:

C ex (α i t , X i t ) := c 2 α i t ( x i 0 -X i t x i 0
).

The constants c 1 and c 2 are positive and represent respectively the cost of production and the cost of extraction. They are the same for the all producers. From the theory of propagation of chaos, the individual level of reserve X i and the price process P i , i = 1, ..., N , become independent and identically distributed, when N goes to infinity, with a common distribution given by the law of the solution (X, P ) to the stochastic Mckean-Vlasov equation

dX t = -α t dt + σX t dW t + η X X t -π(de, dt), X 0 = x 0 , ( 2
.6.73)

P t = P 0 t -δ α t -ε t 0 α s ds, (2.6.74)
where W is a Brownian motion and α t = a(t, X t ) ,t ≥ 0, for some measurable function a on IR + × IR. We reduce the problem to a representative producer with initial reserve x 0 > 0. The state process is given X (see (2.6.73)). The control processes are given by (α, P ), where P satisfies (2.6.74). The aim of the representative producer is to minimise the cost functional given by:

J(α) := E ∞ 0 e -ρ t α t (-P 0 t + δ α t + ε t 0 ᾱs ds) + c 1 Var(α t ) + c 2 α t ( x 0 -X t x 0 )dt , (2.6.75)
under the constraints that α t ≥ 0 and X t ≥ 0 P a.s. for all t ≥ 0. As Xt = x 0 -t 0 ᾱs ds, then

J(α) = E ∞ 0 e -ρ t α t (-P 0 t + δ α t + ε(x 0 -Xt )) + c 1 Var(α t ) + c 2 α t ( x 0 -X t x 0 )dt ,
and we are in the framework of Section 2.5 with d = m = 1 (one-dimensional state variable and control), the coefficients of the state process and the cost functional are given by

B = -1, C = σ, F (e) = η, for all e ∈ χ,
and

N + Ñ = δ, N = δ + c 1 , I + Ĩ = - c 2 + εx 0 2x 0 , I = - c 2 2x 0 , H t = c 2 + εx 0 -P 0 t 2 ,
where the other coefficient are equal to zero. We define λ(χ) := χ λ(de). Notice that under the assumption ρ > σ 2 + λ(χ)η 2 , (H1 ′ ) and (H3 ′ ) are satisfied. By following the approach developed in section 2.5, the optimal control is given explicitly. We have to solve the decoupled system of Ricatti equations and BSDEs with jumps (2.5.63). The Riccati equations (2.5.64) for K and Λ

(2.5.67) are given by:

K + c 2 2x 0 2 δ + c 1 + (ρ -σ 2 -λ(χ)η 2 )K = 0, (2.6.76) Λ + c 2 +εx 0 2x 0 2 δ + ρΛ -(σ 2 + λ(χ)η 2 )K = 0.
(2.6.77)

Let us also remark that the condition (H2 ′ ) is not satisfied, but we have the existence of a

solution (K, Λ) to (2.6) such that K c 1 ,c 2 := K + c 2 2x 0 δ + c 1 > 0 and Λ ε := Λ + c 2 +εx 0 2x 0 δ > 0,
and given by:

K c 1 ,c 2 = -(ρ -σ 2 -λ(χ)η 2 ) + (ρ -σ 2 -λ(χ)η 2 ) 2 + 2c 2 ρ-σ 2 -λ(χ)η 2 x 0 (δ+c 1 ) 2 > 0,
and

Λ ε = -ρ + ρ 2 + 2 ρ(c 2 +εx 0 )+2(σ 2 +λ(χ)η 2 )K δx 0 2 > 0.
Therefore, we can write the linear BSDE (2.5.69) with jumps as:

-dY t = Λ ε 2 (c 2 + εx 0 -P 0 t ) -(ρ + K c 1 ,c 2 )Y t + (K c 1 ,c 2 -Λ ε ) Ȳt (2.6.78) -σZ Y t + η χ R Y t (e)λ(de) dt -Z Y t dW t - χ R Y t (e)π(de, dt).
One could check that a solution of the BSDE (2.6.78) is given by:

(Y, Z Y , R Y ) = ( ∞ t e -(ρ+Λε)(s-t) Λ ε c 2 + εx 0 -P 0 s 2 ds, 0, 0) 0≤t≤T .
(2.6.79)

In the remaining part of the paper, we assume that P 0 t = p 0 for all t ≥ 0: p 0 is interpreted as a substitute price for the exhaustible resource. We study two cases. The first one, when p 0 = c 2 + εx 0 i.e. p 0 coincides with c 2 + εx 0 which is the cost of extraction for the last unit of resource. In other words, the Hotelling rent H r := p 0 -c 2 -εx 0 is equal to zero. The second case when p 0 < c 2 + εx 0 i.e. the Hotelling rent is negative. The next proposition gives an explicit solution to the problem (2.6.75) when H r = 0. Proposition 2.6.1. We assume that p 0 = c 2 +εx 0 for all t ≥ 0, x 0 is large enough and 1 > ρ 2 + 2ε δ . Then the solution of (2.6.75) is given by

α * t = K c 1 ,c 2 X * t -+ (Λ ε -K c 1 ,c 2 ) X * t -.
Proof. Since (H1 ′ ) and (H3 ′ ) are satisfied and the Riccati equations have a solution, then, by Theorem (2.5.4), the optimal control is then given by:

α * t = K c 1 ,c 2 (X * t --X * t -) + Λ ε X * t --1 2δ c 2 + εx 0 -P 0 t -Y t .
As P 0 t = p 0 = c 2 + εx 0 for all t ≥ 0, then the solution of the BSDE (2.6.79) satisfies Y t = 0 for all t ≥ 0 which yields:

α * t = K c 1 ,c 2 (X * t --X * t -) + Λ ε X * t -.
It remains to show that the optimal strategy satisfies the constraint α * t ≥ 0 P a.s. for all t ≥ 0. As x 0 is large, by using Taylor's formula, we have

2 (Λ ε -K c 1 ,c 2 ) = -ρ + ρ 2 + 2ε δ (1 + ρc 2 + 2(σ 2 + λ(χ)η 2 )K δx 0 (ρ 2 + 2ε δ ) ) - c 2 ρ x 0 (δ + c 1 ) + o( 1 x 0 ) = -ρ + ρ 2 + 2ε δ + c 2 ρ x 0 ( 1
δ ρ 2 + 2ε δ - 1 δ + c 1 ) + 2(σ 2 + λ(χ)η 2 )K δx 0 ρ 2 + 2ε δ + o( 1 x 0 ). As 1 > ρ 2 + 2ε δ , then Λ ε -K c 1 ,c 2 ≥ 0.
(2.6.80)

We define the stopping time τ * as follows:

τ * := inf{t ≥ 0 s.t. X * t ≤ 0}.
Then on the set {t < τ * }, from inequality (2.6.80), we have α * t ≥ 0 P a.s. On the set {t = τ * }, the state process X * τ * = 0, which implies α * τ * = 0. Since the drift, the diffusion and the jump terms of the state process are equal to zero, then the process X * remains at the level 0 for all t ≥ τ * and the optimal strategy α * is the null strategy for all t ≥ τ * .

In the second case, we assume that p 0 < c 2 + ϵx 0 . It is not obvious to check the positivity of the the state process and the optimal strategy. We study the stationary level of the reserve and the optimal production rate in mean. From the definition of X * , we have

X * t = x 0 - t 0 α * s ds = x 0 - t 0 Λ ε X * s ds + t 0 1 2δ c 2 + εx 0 -p 0 - ∞ s e (-(ρ+Λε)(u-s)) Λ ε c 2 + εx 0 -p 0 2 du ds which implies X * t = x 0 + (2ρ + Λ ε )(c 2 + ϵx 0 ) 2δ 1 -e -Λεt 2Λ ε (ρ + Λ ε ) - p 0 2δ t 0 e -Λε(t-s) (1 - ∞ s Λ ε 2 e (ρ+Λε)(u-s) du)ds. It yields that lim t→∞ X * t = (2ρ + Λ ε )(c 2 + ϵx 0 -p 0 ) 4δΛ ε (ρ + Λ ε )
. As the Hotelling rent H r := p 0 -c 2 -ϵx 0 is negative, then lim t→∞ X * t exists and is positive. As lim

t→∞ X * t = x 0 - ∞ 0 ᾱ * s ds, then lim t→∞ ᾱ * t = 0.
It means that when we switch to substitute good, there is a remaining resource and we stop the production of exhaustible resource.

CHAPTER 3 NUMERICAL APPROACH FOR A CLASS OF MEAN FIELD GAMES

Introduction

In this work, we want to apply this method for a MFG problem where the drift depends not only on α but also on X. Therefore, our mean field game problem is written in the following form:

inf α J(x, t, α, m) := inf α E T t f (X α s , α s , m s )ds + Φ(X α T , m T ) , (3.1.1) subject to dX α t = b(t, X α t , α t )dt + σdB t X α 0 ∼ m 0 .
In this work, we introduce the Markov chain approximation approach to solve a mean field game problem. This approach was inspired from the work of Kushner and Dupuis [START_REF] Kushner | Numerical methods for stochastic control problems in continuous time[END_REF]. It's based on the approximation of the stochastic control problem by a control problem for a discrete time and discrete space controlled Markov chain. In recent work, E. Bayraktar, A. Budhiraja, and A. Cohen [START_REF] Bayraktar | A numerical scheme for a mean field game in some queueing systems based on markov chain approximation method[END_REF] used the Markov chain approximation method to construct an approximation for the MFG with reflecting barriers. They formulate the MFG model in terms of a controlled reflected diffusion with a cost function and study the convergence of their numerical scheme is established for a small time horizon T. The idea is that for a fixed measure on the path space, we define a Markov decision problem after a time and space discretization. Roughly speaking, the solution of the discrete MFG considered can be seen as the solution of a fixed point problem on the space of probability measures on certain path space. Then the discrete scheme is interpreted as a formulation of a discrete mean field game where the controlled statevariable is described by a Markov chain chosen to be consistent with the controlled stochastic differential equation associated to the original problem, [START_REF] Kushner | Numerical methods for stochastic control problems in continuous time[END_REF]. The Markov chain corresponds to an optimally controlled one by some representative agent for which we define the discrete control problem. Existence and uniqueness of solution of the discrete MFG equilibrium is proved using an attractive probabilistic formulation. This work is concertized in the preprint [START_REF] Ben Aziza | Numerical approach for mean field games[END_REF].

Framework: Notations and setting

Let us consider a mean field game model where each players i controls its private state X i t at time t, by taking an action α t in a closed convex subset A, where the dynamic of state is driven by the following the stochastic differential equation

dX i t = b(t, X i t , α t )dt + σdB t , X i 0 ∼ m 0 , (3.2.2)
where B t is a standard Brownian motion, σ is a fixed positive constant and A is the set of admissible control,

A := α : Ω × [0, T ] → A s.t α is IF -adapted and t 0 |α t |dt < ∞ .
We consider K a non empty compact of IR d and let us assume that the function b satisfy the following assumptions 3.2.1 : Each player will need to choose his strategy to minimize his cost functional J over the period [0, T ], which is influenced by the state of other players. The problem of differential games with an infinite number of agents is articulated in the following form:

Assumption 3.2.1. The function b is defined on [0, T ] × K × A in IR as follows b : [0, T ] × K × A → IR (t, x, α) → b(t, x, α). ( 3
(i) for each fixed flow of probability measures m on IR d , solve the standard stochastic control problem:

u(x, t) := inf α∈A J(t, x, α, m), (3.2.4) subject to dX α t = b(t, X α t , α t )dt + σdB t X α 0 ∼ m 0 . (ii) Find m such that L(X α t ) = m t for all t ∈ [0, T ],
where X α t is a solution of the above stochastic differential equation.

We can see that the first step provides the best response u of a given player interacting with the stochastical distribution of the states of the other players. If this distribution m is assumed to be given, when the second step solves a fixed point problem in the goal of the search for fixed points m t of the best response function. Once these two steps have been taken successfully, if the fixed-point optimal control α * identified in step (i) is in feedback form α * t = ϕ(t, X α t , m t ) for some deterministic function ϕ on [0, T ] × T d × P(K). Then if the players use α * t , the mean field equilibrium {m t } 0≤t≤T with value in P(K) should coincide with the law of X and satisfy the Fokker Planck equation.

Definition 3.2.1. (MFG equilibrium )

We say that m * ∈ P(K) is an equilibrium for the MFG associated to the initial probability density m 0 if and only if there exist α * ∈ A such that

J(t, X α t , α * , m * ) = inf α∈A J(t, X α t , α, m * ), (3.2.5
)

and m * t = L(X α t ), ∀t ∈ [0, T ]. (3.2.6)
The objective is to minimize a cost function of the following form :

J(x, t, α, m) = E T t (f (X α s , α s , m s )ds + Φ(X α T , m T ) , (3.2.7)
where f is a measurable function defined in this form that we will adapt for the rest of the paper:

f (X α s , α s , m s ) := L(X α s , α s ) + F (X α s , m s ).
with F and L are a measurable functions satisfied assumptions (3.2.2)-(3.2.3) respectively introduced below:

We introduce first the Wasserstein distance W 1 as

W 1 (ν, ν ′ ) = min |x -y| dµ such that µ ∈ Π(ν, ν ′ ) ,
where

Π(ν, ν ′ ) := π ∈ P(K × K), π(A × K) = ν(A), π(K × A) = ν ′ (A) for eachA ∈ B(K) .
Assumption 3.2.2. The function F is defined on P × K in IR as:

F : P × K → IR (m, x) → F (m, x). (F h1 ) F is K F -Lipschitz and K F is independent of m, i.e. | F (m, x) -F (m, y) |≤ K F | x -y |, ∀m ∈ P and (x, y) ∈ K × K.
(F h2 ) F is Lipschitz from the space P endowed with W 1 to the space C(K), endowed with the uniform norm, i.e. there exist C F > 0, such that

∥F (m 1 , x) -F (m 2 , x)∥ ≤ C F W 1 (m 1 , m 2 ), ∀ m 1 , m 2 ∈ P(K) and x ∈ K.
(F h3 ) ∂ x F exists and is C ∂F -Lipschitz from the space P(K) endowed with W 1 to the space C(K), endowed with the uniform norm. i.e. there exist

L ∂F > 0, such that ∥∂ x F (m 1 , x)] -∂ x F (m 2 , x)∥ ≤ C ∂F W 1 (m 1 , m 2 ), ∀m 1 , m 2 ∈ P(K) and x ∈ K.
Assumption 3.2.3. The function L is defined on K × A in IR as follows

L : K × A → IR (x, α) → L(x, α). (L h1 ) L is K L -Lipschitz and K L is independent of α, i.e. | L(x 1 , α) -L(x 2 , α) |≤ K L | x 1 -x 2 |, ∀(x 1 , x 2 ) ∈ K × K and α ∈ A.
(L h2 ) L is strictly convex with respect to α, i.e.

∂ α L(x, α 1 )(α 2 -α 1 ) ≤ L(x, α 2 ) -L(x, α 1 ), ∀α 1 , α 2 ∈ A and x ∈ K. (L h3 ) For all x ∈ K, the function α → ∂ α L(x, α) is continuous function. Assumption 3.2.4.
The function Φ is defined on K × P in IR as:

Φ : P × K → IR (m, x) → Φ(m, x). (Φ h1 ) Φ is K Φ -Lipschitz and K Φ is independent of m, | Φ(x, m) -Φ(y, m) |≤ K Φ | x -y |, ∀m ∈ P(K) and (x, y) ∈ K × K.
(Φ h2 ) Φ is C ϕ Lipschitz from the space P endowed with W 1 to the space of continuous functions C(K), endowed with the uniform norm .i.e. there exists

C Φ > 0, such that ∀m 1 , m 2 ∈ P ∥Φ(x, m 1 ) -Φ(x, m 2 )∥ ≤ C Φ W 1 (m 1 , m 2 ). (Φ h3 ) ∂ x ϕ[m]
exists and is C ∂ϕ -Lipschitz on P space endowed with W 1 to the space C(K), endowed with the uniform norm .i.e. there exists C ∂ϕ > 0, such that

∥∂ x ϕ(x, m 2 ) -∂ x ϕ(x, m 2 )∥ ≤ C ∂ϕ W 1 (m 1 , m 2 ), ∀m 1 , m 2 ∈ P(K) and x ∈ K.
According to [START_REF] Cardaliaguet | Notes on mean field games[END_REF], the control problem is mathematically described through a system of two non linear partial differential equations introduced by a following system

                 -∂ t u -ν∆u + H(x, m, Du) = f (x, m), in T d × (0, T ), ∂ t m -ν∆m -div(D p H(x, m, Du)m) = 0, in T d × (0, T ), m(0) = m 0 , u(x, T ) = ϕ(x, m(T )), (3.2.8)
where σ is a non-negative given parameter, T > 0 is a fixed time horizon, K is a compact subset of IR, P is the space probability measures on K and m 0 is a measure on P(K). H : (x, p) ∈ K × IR → H(x, p) ∈ IR is the Hamiltonian, a convex function with respect to p defined by

H(x, m, p) = sup α {-pα -L(x, α)}.
Then we will introduce a numerical method to solve the coupled system of forward-backward partial differential equations, with variables u and m.

We will refer to the system forward-backward PDEs system (3. ) is treated in many works [START_REF] Bensoussan | Control and nash games with mean field effect[END_REF][START_REF] Caines | Large-population cost-coupled lqg problems with nonuniform agents: individual-mass behavior and decentralized nash equilibria[END_REF][START_REF] Camilli | A semi-discrete approximation for a first order mean field game problem[END_REF][START_REF] Carmona | Applications of mean field games in financial engineering and economic theory[END_REF][START_REF] Carmona | Probabilistic Theory of Mean Field Games with Applications I-II[END_REF][START_REF] Guéant | New numerical method for mean field games with quadratic costs[END_REF][START_REF] Lasry | Jeux à champ moyen. i-le cas stationnaire[END_REF]. A uniqueness criterion for the solution of the field game system was presented by J.M. Lasry and P.L. Lions in their pioneer work [START_REF] Lasry | Jeux à champ moyen. i-le cas stationnaire[END_REF] on the theory of mean field games called Lasry-Lions Monotonicity condition.

Numerical Approximation

In this section, we propose a numerical method to approximate mean field control problem. We introduce Markov chain Approximation developed by Kushner [START_REF] Kushner | Numerical methods for stochastic control problems in continuous time[END_REF]. The approximating Markov chain is chosen such that certain local properties of the approximating chain are "similar" to those of the original controlled process. A cost function for the Markov chain model which is an appropriate analogue to that for the original model is then found.

This method applies a Markov chain approximation to continuous time, continuous state stochastic control problems by renormalizing finite differences forms as proper Markov chain transition probabilities. These transition probabilities arise when deriving finite difference versions of the dynamic programming equation. An important advantage of this method is that the Markov chain approximation facilitates convergence proofs for the numerical methods in terms of probabilistic argument.

Approximating Markov chain

The idea behind this approach is to construct a discrete time space Markov chain ξ h and its state space. This chain admits a continuous interpolation which approximates the state process X t . Roughly speaking, we define a Markov chain ξ h , on the state space where his transition probabilities denoted by p(t, ν, α; x, x + h). These transition probabilities must be positive p > 0 and satisfies y∈{x±h} p(t, ν, α; x, y) = 1.

According to the construction of a Markov chain, its probability measure ν n at time n is defined by

ν n = ν n+1 p(t, ν, α; x, y).
By iterating this equality, we obtain for any n ≥ 1,

ν n = (p(t, ν, α; x, y)) n ν 0 ,
where (p) n is the transition matrix and ν 0 is the initial distribution.

Discretization setting

In this part, we consider T a terminal time T > 0, and K = [-R, R] with R > 0. We define

h = 2R N h x and ∆t = T N h t
where N h x and N h t as the mesh step and the time step for our discrete scheme, for two positives integers N h

x and N t . Denote x i := -R + ih, t k := k∆t, we consider the space grid G h and the time-space grid G h,dt defined by

G h := {x i , 0 ≤ i ≤ N h x } and G h,dt = {(x i , t k ), 0 ≤ i ≤ N h x , 0 ≤ k ≤ N h t }.
We denote the approximation of u and m respectively by vectors U and M such for each (i, k),

u(t k , x i ) ≈ U i,k , and M (t k , x i ) ≈ M h i,k .
Then for the discretization of the HJB equation, we begin by introducing the finite difference schemes. So for the approximation of ∂ x u at (x i , t k ) ∈ G h,dt we define the following discrete operators:

DU i,k := U i+1,k -U i-1,k 2h , for 1 ≤ i ≤ N h x -1, (3.3.9)
For the approximation of ∆u we will use the standard operator

∆ h U i,k = U i+1,k -2U i,k + U i-1,k h 2 , for 1 ≤ i ≤ N h x -1, (3.3.10)
and for approximate ∂ t u at (x i , t k ) ∈ G h,dt we introduce the following operator

DU i,k := U i,k -U i,k-1 ∆t , for 1 ≤ k ≤ N h t -1. (3.3.11)
Let us consider also L h , F h and Φ h be approximated operators respectively of L , F and Φ as:

L h : G h × A h → IR, F h : P(G h ) × G h → IR, Φ h : P(G h ) × G h → IR, where A h is the set of control α h . If ξ h k ′ = x i , ∀M h ∈ P(G h,dt ), ∀α h ∈ A h [L h (α h )] i,k ′ := L(x i , α h i,k ′ ), [F h (M h )] i,k ′ := F (x i , M h * k ′ ), [Φ h (M h )] i,k ′ := Φ(x i , M h t k ′ ).

Construction of Markov chain associated to MFG problem

Discret problem problem Using the implicit scheme (3.3.11) for ∂ t u, the explicit scheme (3.3.10) and the centered scheme (3.3.9), we discretize the HJB equation in (3.2.8) as

-( U i,k+1 -U i,k ∆t ) - 1 2 σ 2 ( U i+1,k -2U i,k + U i-1,k h 2 ) + H(x, U i+1,k -U i-1,k 2h ) = F (x i , M h i,k ), (3.3.12)
with

H(x i , U i+1,k -U i-1,k 2h ) = inf α {b(t k , x i , α i,k ).( U i+1,k -U i-1,k 2h ) + L(x i , α i,k )}. (3.3.13)
Therefore, it leads to the following discretized form of HJB equation for (x, t) ∈ G:

               U i,k-1 = inf α∈A h L h (x i , α i,k ) + F h (x i , M h i,k ) dt + U i,k 1 -σ 2 ∆t h 2 +U i-1,k σ 2 2 ∆t h 2 - ∆t 2h b(t k , x i , α i,k ) +U i+1,k σ 2 2 ∆t h 2 + ∆t 2h b(t k , x i , α i,k ) , U i,N h t = Φ h (x i , M h i,N h t ), 0 ≤ i ≤ N h x , (3.3.14)
and for x ∈ ∂(G × [0, T ]), discretized form of HJB equation satisfies the following boundary conditions:

                             U 0,k-1 = L(x 0 , α 0,k ) + F h (x 0 , M h i,k ) ∆t + U 0,k 1 - σ 2 2 ∆t h 2 + ∆t 2h b(t k , x 0 , α 0,k ) +U 1,k σ 2 2 ∆t h 2 - ∆t 2h b(t k , x 0 , α 0,k ) , U N h x ,k-1 = L(x N h x , α N h x ,k ) + F h (x N h x , M h i,k ) ∆t + U N h x ,k 1 - σ 2 2 ∆t h 2 + ∆t 2h b(t k , x N h x , α N h x ,k ) +U N h x -1,k 1 -σ 2 2 ∆t h 2 - ∆t 2h b(t k , x N h x , α N h x ,k ) . (3.3.15)
We assume also the following boundary conditions

U -1,k = U 0,k , U N h x +1,k = U N h x ,k , 1 ≤ i ≤ N h x . (3.3.16)
Transition probabilities for discrete Markov chain Once we discretize the HJB equation in this form (3.3.14), then, we define (ξ h k ) 0≤k≤N h x a controlled Markov chain characterized by its transition probabilities determined explicitly from the HJB approximation (3.3.14). In other words, from the finite differential approximation, we define the transition probabilities as follows

p(t k , ν i,k , α i,k ; k, k+1) = P h (k, α i,k ) such that at each time t k , (P h i,j (k, α i,k )) 0≤i,j≤N h
x is the matrix of transition probabilities for the controlled Markov Chain, defined by

P h i,i-1 (k, α i,k ) = σ 2 2 ∆t h 2 - ∆t 2h b(t k , x i , α i,k ), 1 ≤ i ≤ N h x -1, (3.3.17)
P h i,i (k, α i,k ) = 1 -σ 2 ∆t h 2 , 1 ≤ i ≤ N h x -1, P h i,i+1 (k, α i,k ) = σ 2 2 ∆t h 2 + ∆t 2h b(t k , x i , α i,k ), 1 ≤ i ≤ N h x -1.
With some transition probability with respect to boundary condition

P h 0,0 (k, α 0,k ) = 1 - σ 2 2 ∆t h 2 + ∆t 2h b(t k , x 0 , α 0,k ), (3.3.18)
P h 0,1 (k, α 0,k ) = σ 2 2 ∆t h 2 - ∆t 2h b(t k , x 0 , α 0,k )), P h N h x ,N h x (k, α N h x ,k ) = σ 2 2 ∆t h 2 - ∆t 2h b(t k , x N h x , α N h x ,k ), P h N h x ,N h x -1 (k, α N h x ,k ) = σ 2 2 ∆t h 2 + ∆t 2h b(t k , x N h x , α N h x ,k ).
The P h i,j (k, α i,k ) represents the conditional probability that the state of the chain ξ h at time t k+1 is x j given the state x i at time t k , and the control action α * k , given by P h ij (k, α i,k ). Therefore, the symmetric finite difference approximation HJB (3.3.14) can be written in the following form

U i,k = inf α i,k    (L(x i , α i,k ) + F (x i , M * k )) dt + N h x j=0 P h ij (k, α i,k )U j,k+1    . ( 3.3.19)
Let us denote by A h the set of all policies (α h , ξ h ) for the controlled Markov chain

(m h 0 , α h , {P h (k, α h * k ), 0 ≤ k ≤ N h t }), A h := A h m h 0 ; α; (P h (k, α * k )) 0≤k≤N h t . Remark 3.3.1.
An important advantage of this method is that the Markov chain approximation facilitates proofs of convergence of numerical methods in terms of probabilistic argument.

Roughly speaking, by approximating the state variable process by a Markov chain, we obtain the appropriate transition probabilities of the Markov chain, we will show that converges to the solution of the Fokker Planks equation.

According to Kushner and Dupuis [START_REF] Kushner | Numerical methods for stochastic control problems in continuous time[END_REF], the Markov chain approximation variation

∆ξ h k = ξ h k+1 - ξ h k with |∆ξ h k | = o(h)
must satisfy the local consistency conditions. This means that from a local point of view, the conditional mean and covariance of the changes in state of the chain are proportional to the local mean drift and covariance for the original process. Kushner and Dupuis, 2013, [70] ).

Definition 3.3.2. (Local consistency

We say that {P h (k, α * k ), 0 ≤ k ≤ N h t } is locally consistent with the state process (X t ) defined by (3.2.2) if for any admissible policy (µ h , ξ h ) the following local consistency condition holds

E α i,k [∆ ξ h k ] = b(t k , X t k , α t k )dt + o(h), (3.3.20)
and

E α i,k [|∆ξ h k -E α i,k [∆ξ h k ]| 2 ] = 2σ 2 dt + o(h), (3.3.21)
where

∆ξ h k := ξ h k+1 -ξ h k and E α i,k is the conditional expectation given ξ h k = x i .
. For all k, we choose h and

∆t such that ∆t ≤ h 2 σ 2 and h ≤ σ 2 sup i,k {b(t k , x i , α)} , ( 3.3.22)
then (P h (k, α * k )) 0≤k≤N h t
is a family of transition probabilities matrices.

Proof. We must check that 0 ≤ P h i,i ≤ 1. Then

0 ≤ 1 -σ 2 ∆t h 2 ≤ 1 ⇔ 0 ≤ σ 2 ∆t h 2 ≤ 1 ⇔ ∆t ≤ h 2 σ 2 .
And we check that 0

≤ P h i,i-1 ≤ 1. 0 ≤ σ 2 2 ∆t h 2 - ∆t 2h sup i,k (b(t k , x i , α i,k )) ≤ 1 ⇔ 0 ≤ ∆t σ 2 h 2 ( 1 2 -h sup i,k (b(t k , x i , α i,k )) 2σ 2 ) ≤ 1. Then 0 ≤ ( 1 2 -h sup i,k {b(t k , x i , α i,k )} 2σ 2 ) ≤ 1 ⇔ h ≤ σ 2 sup i,k {b(t k , x i , α i,k )} .
The law of the controlled Markov Chain is defined through the matrix of transition probabilities (3.3.17)- (3.3.18). The idea in this work is instead of discretizing the Fokker Planck equation, we define the probability measure of the controlled discrete Markov chain using its transition probability matrix,

M h * ,0 = m h 0 , M h * ,k = (P h (k, α * k )) t M h * ,k-1 , for 1 ≤ k ≤ N h t , (3.3.23)
where we denote by M h * ,k the vector (M h i,k ) 0≤i≤N h x , for all the i and fixed k, and by (P h (k, α * k )) t the transposed matrix of (P h (k, α * k )).

Remark 3.3.5. In [START_REF] Achdou | Mean field games: convergence of a finite difference method[END_REF], Achdou and Capuzzo-Dolcetta introduce a numerical method using the finite difference methods. They propose a finite difference scheme with monotone approximations of the Hamiltonian and also of the discrete formulation of the Fokker-Planck equation. For our approach, we do not look to discretize the Fokker-Planck equation. From finite difference approximation of HJB equation (3.3.14), we define the law m h of the Markov chain from its transition probabilities, and we will show that it converges weakly to the solution of the Fokker Planck equation. We say that M h ∈ P(G h,dt ) is an equilibrium for the discrete MFG control problem if there exists an admissible strategy (α h , ξ h ) such that

Discrete MFG equilibrium

(α h , ξ h ) = arg min (α,ξ)∈A h J h (i, k, α, ξ, M ), (3.3.24)
where

J h (i, k, α h , ξ h , M h ) = E k ′ ≥k L h (ξ h k ′ , α h * k ′ ) + F (ξ h k ′ , Mt k ′ ) dt + Φ h (ξ h N h t , MT ]) | ξ h ℓ , α h * ℓ , ℓ ≤ k; ξ h k = xi , (3.3.25)
and M h is the discrete distribution law of the discrete-time-space controlled Markov chain

(m h 0 , α h , {P h (k, α h * k ), 0 ≤ k ≤ N h t }) and satisfies    M h * 0 = m h 0 , M h * k = P h (k, α h * k ) t M h * k-1 = ℓ≤k-1 (P h (ℓ, α h * ℓ )) t m h 0 , 1 ≤ k ≤ N h t .
(3.3.26)

Existence and uniqueness results of discrete MFG solution

In this section, we will study the existence and uniqueness of discrete MFG equilibrium.

Existence result

To show the existence of a solution of discrete MFG problem, we use the result of Brouwer's fixed point theorem. The idea is to show that a function Γ h admits at least a fixed point using this theorem. Therefore, we define the function Γ h , for a given h > 0, as follows

Γ h : P(G h,dt ) → P(G h,dt ) m h → Γ h (m h ) = M h , ( 3.4.27
)

such that M h = Γ h (m h ) is the law of the discrete-time-space controlled Markov chain (m h 0 , α h , {P h (k, α h * k ), 0 ≤ k ≤ N h t }
) according to the following equation:

   M h * ,0 = m h 0 , M h * ,k = (P h (k, α h * k )) t M h * ,k-1 = ℓ≤k-1 (P h (ℓ, α h * ℓ )) t m h 0 , for 1 ≤ k ≤ N h t , (3.4.28)
with α h is associated to the optimal admissible policy (α h , ξ h ) = arg min

(ν h ,ξ h )∈A h J h (i, k, ν h , ξ h , M h ). ( 3

Let us consider that

M h,n = Γ(m h,n ) and M h = Γ(m h ),
where m h,n , m h ∈ P(G h,dt ), such that M h,n ( respectively M h ) satisfies equation (3.3.26) associated to α h,n ( respectively α h ).

In other hand, we can write the above equations in the following form

M h,n i,k+1 = P h i,i (k, α h,n i,k )M h,n i,k + P h i+1,i (k, α h,n i,k )M h,n i-1,k + P h i-1,i (k, α h,n i,k )M h,n i+1,k ,
and

M h i,k+1 = P h i,i (k, α h i,k )M h i,k + P h i+1,i (k, α h i,k )M h i-1,k + P h i-1,i (k, α h i,k )M h,n i+1,k ,
with P h are the probability transitions defined by (3.3.17)- (3.3.18). Then, we take the difference between M h,n i,k+1 and M h i,k+1 , for all

0 ≤ i ≤ N h t , M h,n i,k+1 -M h i,k+1 = 1 -σ 2 ∆t h 2 M h,n i,k -M h i,k + σ 2 2 ∆t h 2 M h,n i-1,k -M h i-1,k + σ 2 2 ∆t h 2 M h,n i+1,k -M h i+1,k M h i,k-1 + ∆t 2h b(t k , x i-1,k , α h,n i-1,k ) M h,n i+1,k + ∆t 2h b(t k , x i+1,k , α h i+1,k ) M h i-1,k - ∆t 2h b(t k , x i+1,k , α h,n i+1,k ) M h,n i-1,k - ∆t 2h b(t k , x i-1,k , α h i-1,k ) M h i+1,k .
We know that b is bounded, then we get

M h,n i,k+1 -M h i,k+1 ≤ 1 -σ 2 ∆t h 2 M h,n i,k -M h i,k + σ 2 2 ∆t h 2 M h,n i-1,k -M h i-1,k + σ 2 2 ∆t h 2 M h,n i+1,k -M h i+1,k + ∆t 2h C b ( M h,n i+1,k -M h i+1,k ) - ∆t 2h C b ( M h,n i-1,k -M h i-1,k ).
Then, we have

|M h,n i,k+1 -M h i,k+1 | ≤ 1 -σ 2 ∆t h 2 |M h,n i,k -M h i,k | + ( σ 2 2 ∆t h 2 + ∆t 2h C b )|M h,n i-1,k -M h i-1,k | + ( σ 2 2 ∆t h 2 + ∆t 2h C b )|M h,n i+1,k -M h i+1,k |.
We take the maximum on i in the both side, we obtain

max i |M h,n i,k+1 -M h i,k+1 | ≤ 1 -σ 2 ∆t h 2 max i M h,n i,k -M h i,k + ( σ 2 2 ∆t h 2 + ∆t 2h C b ) max i |M h,n i,k -M h i,k | + ( σ 2 2 ∆t h 2 + ∆t 2h C b ) max i |M h,n i,k -M h i,k | ≤ 1 + ∆t C b 2h 2 max i M h,n i,k -M h i,k From the discret Gronwall's lemme, if (v n ) satisfy v n+1 ≤ (1 + η)v n + k, then v n ≤ exp(nη)v 0 + k n-1 i=1 exp(iη),
Hence, we obtain

M h,n * ,k+1 -M h * ,k+1 ≤ C m h 0 -m h 0 = 0, where C = exp( ∆t C b 2h 2 ).
Finally, we proved the mapping Γ h , defined by (3.4.27), is continuous on a convex compact set P(G h,∆t ). Then from Brouwer's fixed point theorem, we get the existence of equilibrium of the discrete MFG control problem.

Uniqueness result

Using the same argument of Lions-Lasry Monotonicity method ( [74], Theorem 4.1), we will prove the uniqueness of discrete MFG equilibrium.

(F h (M * k ) -F h (M * k ), M * k -M * k ) 2 ≤ 0 IR F h (M * k ) = F h (M * k ). (Φ h (M * k ) -Φ h (M * k ), M * k -M * k ) 2 ≤ 0 ⇒ Φ h (M * k ) = Φ h (M * k ).
Then there exist at least one solution of the system (

Proof. Let M and M ∈ P(G h ) be two equilibrium for the discrete MFG control problem (α, ξ) ( resp. (α, ξ)) and U (resp. U ) be respectively the optimal policy and the discrete value function associated to M (resp. M ) satisfying the following equations respectively:

       U i,k = inf α i,k ∈A h    L h (x i , α i,k ) + F h (x i , M i,k ) dt + N h t j=0 P h ij (k, α i,k )U j,k+1    , U * ,N h t = Φ h (M * ,N h t ), (3.4.30)
and

       U i,k = inf α i,k ∈A h    L h (x i , α i,k ) + F h (x i , M i,k ) dt + N h t j=0 P h ij (k, α i,k )U j,k+1    , U * ,N h t = Φ h (M * ,N h t ), (3.4.31)
Taking the scalar product of the difference between the equations satisfied by U and U multiplied by the difference M -M , we obtain

N h t k=0 (U * ,k -U * ,k , M * ,k -M * ,k )2 = (U * ,N h t -U * ,N h t , M * ,N h t -M * ,N h t )2 + N h t -1 k=0 (P h (k, α * ,k )U * ,k+1 , M * ,k -M * ,k )2 - N h t -1 k=0 < P h (k, α * ,k )U * ,k+1 , M * ,k -M * ,k )2 (3.4.32)
+ ∆t

N h t -1 k=0 (L h (x, α * ,k ) -L h (x, α * ,k ), M * ,k -M * ,k )2 + ∆t N h t -1 k=0 (F h (x, M * ,k ) -F h (x, M * ,k ), M * ,k -M * ,k )2.
Using the fact that M and M are two solutions of (3.3.26), then satisfy the same initial condition, i.e.

M * 0 = m h 0 = M * 0 .
Therefore, we have the following equality

N h t k=0 (U * ,k -U * ,k , M * ,k -M * ,k ) 2 = N h t -1 k=0 (U * ,k+1 -U * ,k+1 , M * ,k+1 -M * ,k+1 ) 2 .
On the other hand, we know that

(P h (k, α * ,k )U * ,k+1 , M * ,k+1 ) 2 = (U * ,k+1 , P h (k, α * ,k ) t M * ,k+1 ) 2 = (U * ,k+1 , M * ,k+1 ) 2 .
Then, the equation (3.4.32) becomes

(U * ,N h t -U * ,N h t , M * ,N h t -M * ,N h t ) 2 + ∆t N h t -1 k=0 (F h (x, M * ,k ) -F h (x, M * ,k ), M * ,k -M * ,k ) 2 = ∆t N h t -1 k=0 (L h (x, α * ,k ) -L h (x, α * ,k ), M * ,k ) 2 -∆t N h t -1 k=0 (L h (x, α * ,k ) -L h (x, α * ,k ), M * ,k ) 2 + N h t -1 k=0
((P h (k, α * ,k ) -P h (k, α * ,k ))U * ,k+1 , M * ,k ) 2 - N h t -1 k=0
((P h (k, α * ,k ) -P h (k, α * ,k ))U * ,k+1 , M * ,k ) 2 .
Using the explicit expression of the transition probabilities, we calculate

(P h (k, α * ,k )-P h (k, α * ,k )),
for all 0 ≤ k ≤ N h t -1, we have :

P h 0,0 (k, α 0,k ) -P h 0,0 (k, α 0,k )) = - ∆t 2h (b(t k , x 0 , α 0,k ) -b(t k , x 0 , α 0,k )), P h 0,1 (k, α 0,k ) -P h 0,1 (k, α 0,k )) = ∆t 2h (b(t k , x 0 , α 0,k ) -b(t k , x 0 , α 0,k )), P h N h x ,N h x (k, α N h x ,k ) -P h N h x ,N h x (k, α N h x ,k )) = ∆t 2h (b(t k , x N h x , α N h x ,k ) -b(t k , x N h x , α N h x ,k )), P h N h x ,N h x -1 (k, α N h x ,k ) -P h N h x ,N h x -1 (k, α N h x ,k )) = - ∆t 2h (b(t k , x N h x , α N h x ,k ) -b(t k , x N h x , α N h x ,k )), and
P h i,i-1 (k, α i,k ) -P h i,i-1 (k, α N i,k )) = ∆t 2h (b(t k , x i , α i,k ) -b(t k , x i , α i,k )), P h i,i (k, α i,k ) -P h i,i (k, α i, )) = - ∆t 2h (b(t k , x i , α i,k ) -b(t k , x i , α i,k )), P h i,i+1 (k, α i,k ) -P h i,i+1 (k, α N i,k )) = ∆t 2h (b(t k , x i , α i,k ) -b(t k , x i , α i,k )).
Then by iterating, we get

(P h (k, α * ,k ) -P h (k, α * ,k ))U * ,k+1 ≤ ∆t h (b(x i , α i,k ) -b(x i , α i,k ))DU i,k+1 .
Then, using Assumption 3.2.1 and the fact that

∂ α L = -b 2 (t) DU , we get, (U * ,N h t -U * ,N h t , M * ,N h t -M * ,N h t )2 + ∆t N h t -1 k=0 < F h (x, M * ,k ) -F h (x, M * ,k ), M * ,k -M * ,k )2 ≤ -∆t N h t -1 k=0 (L h (x, α * ,k+1 ) -L h (x, α * ,k+1 ), M * ,k )2 + ∆t N h t -1 k=0 (L h (x, α * ,k+1 ) -L h (x, α * ,k+1 ), M * ,k )2 + C N h t -1 k=0 ((α i,k+1 -α i,k+1 ) ∂αL(x, α i,k+1 ), M * ,k )2 -C N h t -1 k=0 ((α i,k+1 -α i,k+1 ) ∂αL(x, α i,k+1 ), M * ,k )2.
Moreover, we have that L is strictly convex with respect to α, i.e.

∂ α L(x, α 1 )(α 2 -α 1 ) ≤ L(x, α 2 ) -L(x, α 1 ), ∀α 1 , α 2 ∈ A.
Therefore,

(U * ,N h t -U * ,N h t , M * ,N h t -M * ,N h t ) 2 + ∆t N h t -1 k=0 (F h (x, M * ,k ) -F h (x, M * ,k ), M * ,k -M * ,k ) 2 ≤ 0.
Then F h and Φ h satisfy the Monotony condition. Therefore, we obtain the uniqueness of the solution.

Convergence results

This Markov Chain Approximation approach was initiated by Dupuis and Kushner [START_REF] Kushner | Numerical methods for stochastic control problems in continuous time[END_REF] to construct numerical solution for the HJB equation. The authors prove the existence of approximating consistent Markov chain which converges to an stochastic process X.

In order to study the convergence of our discrete-MFG equilibrium (3.3.24) to the continuous

The idea of relaxed control is to replace the A-valued process α t with a P(A)-valued process α R t , where P(A) is the space of probability measures equipped with the topology of weak convergence.

Definition 3.5.2. An admissible relaxed control is a measure α R on B(A × [0, T ]) such that, for all t ∈ [0, T ] α R (A × [0, t]) = t. For B ∈ B(A), we define α R t (B) = lim δ→0 α R (B × [t -δ, t]) δ ,
then we can write

α R (du, dt) = α R t (du)dt.
Let us denote R the set of relaxed controls.

We consider then the relaxation of the stochastic control problem where the drift function b is

replaced by the function b R : [0, T ] × K × R defined as b R (t, x, α R ) := A b(s, x, u )α R s (du),
and the function L is replaced by the function L R : K × R defined as

L R (x, α R ) := A L(x, u)α R s (du)
For any relaxed control α R , we associate a cost function giving by

J R (α R t , m, t) = E[ T t [ A L(X α R s s , u)α R s (du) + F (X α R s s , m s )]ds + Φ(X α R t T , m T )], (3.5.36)
where

X α R t t
is solution of the following relaxed model We say that m * ∈ P [0, T ] is a relaxed equilibrium for the MFG associated to the initial probability density µ 0 ∈ P if and only if it exist α R, * ∈ R such that

dX α R t = A b(s, X α R s , u )α R s (du)ds + σdB t , X α R 0 ∼ m 0 . ( 3
J(t, xα R, * , m) = inf α R ∈R J(t, x, α R , m), and m * = L(X α R t ), ∀t ∈ [0, T ]. ( 3
inf α R ∈R J(t, x, α R , m) = inf α∈A J(t, x, α, m). (3.5.39)
Proof. To prove (3.5.39), see the result of Kushner et al. [START_REF] Kushner | Numerical methods for stochastic control problems in continuous time[END_REF], Section 4.6, for more details.

The idea is that, given a relaxed control α R ∈ R, there exists a sequence (α n ) n ∈ A of ordinary controls such that the random measure δ αn(t) (du)dt converges weakly to α R (du)dt P-a.s.

Remark 3.5.5. The class of relaxed controls will be of importance in the convergence proof only. They do not enter into the numerical algorithms.

In this section, we will give the main result on this section, using the intermediate tool introduced above.

Theorem 3.5.6. Let M h be the discrete MFG equilibrium (3.3.24) and ξ h being its associated optimally controlled Markov chain defined by (3.3.17). Let m h ∈ P(D[0, T ]) be the law of the continuous-time-space piecewise constant interpolated process ξh .

Then, the sequence m h h≥0 converges weakly to some m in P(D[0, T ]) where m is an equilibrium of the MFG (3.5.38).

Moreover, let α R,h be the relaxed control process associated to the time-space piecewise interpolated process α R,h , then α R,h converges weakly to some relaxed process α R such that

J R (t, Xt , α R, * , m) = min α R ∈R J R (t, Xt , α R , m) and m t = L( Xα R T ), ∀t ∈ [0, T ],
where the process X α R t is a weak solution of

d Xα R t = A b(s, Xα R s , u )α R s (du)dsdt + σdB t + dz(t), Xα R 0 ∼ m 0 .
(3.5.40) Remark 3.5.7. In this theorem, we had to change the frame and add a local reflection direction to force the process X to return to the space if it leaves it. where z being a reflecting term on K associated to reflecting directions r(∂K), such that there exist a process γ : [0, T ] → IR, γ(s) ∈ r(X α,ϵ s ) almost surely with respect to the random measure induced by | z(.) |:

| z | (.) = t 0 1 K ( Xα,ϵ s )d | z | (s), (3.5.41)
and

z(t) = t 0 γ(s)d | z | (s). (3.5.42)
In fact, for the discretization of our HJB equation, we have imposed boundary conditions ** ** to force the result to remain in the set, which can avoid the discretization of the reflection term.

Alternatively, we can also present a discretization of the HJB by taking the reflection term into account, by agreeing on Kushner's approximation [START_REF] Kushner | Numerical methods for stochastic control problems in continuous time[END_REF], Section 11 in the multidimensional case IR d , where they has to consider with 2 cases:

• if x ∈ K, u satisfy the same discrete equation

U i,k = inf α i,k    (L(x i , α i,k ) + F (x i , M * k )) dt + N h x j=0 P h ij (k, α i,k )U j,k+1    ,
• if x ∈ ∂K, they consider a bounded and differentiable C s.t. ∂ x C(x)γ > 0, for any γ ∈ r(x),

x ∈ ∂K, where u satisfy the following equation

U i,k = inf α i,k    N h x j=0 P h ij (k, α i,k )U j,k+1 + ∂ x C(x i )∆z h (x i )    . with the state ξ h n = x is in K as ∆z h (x) = E h n [∆ξ h n ].
It should be mentioned that, according to [START_REF] Kushner | Numerical methods for stochastic control problems in continuous time[END_REF], the Markov chain in this case is always locally consistent with the reflected diffusion (3. The proof of this theorem is done in many steps, we start by studying the convergence of the approximating Markov Chain to stochastic controlled process X and its law m converges in the space P(D[0, T ]) to some m. Then by injecting this limit X in the control problem (3.5.36)-(3.5.37), we show that L( X) associated to X converges to the same limit m.

Thurs, the first step is to show that the controlled Markov chain {ξ h , h > 0} converges to a Càdlàg process on D[0, T ] and the law m h of the chain converges to m = L(ξ h ) on P(D[0, T ]).

So we start by study the tightness of {ξ h , h > 0} which will give us the convergence of Markov chain. Define F n t to be the σ-algebra generated by x n , n ≤ t. Let T n T be the set of F n t -stopping times which are less than or equal to T with probability one, and assume for each Step 1: In fact, for x i ∈ G h , we have

T ∈ [O, ∞[ that lim δ→0 sup n sup τ ∈T n T E[1 ∧ |x n (τ + δ) -x n (τ )|]] = 0. ( 3
ξ h (t) -x = ξ h k -ξ 0 = N h t -1 i=0 ∆ξ h i+1 = N h t -1 i=0 E α i,k [∆ξ h k ] + ∆ξ h i+1 -E α i,k [∆ξ h k ] .
We apply Young's inequality , we get

ξ h (t) -x 2 = N h t -1 i=0 E α i,k [∆ξ h k ] + ∆ξ h i+1 -E α i,k [∆ξ h k ] 2 ≤ 2    N h t -1 i=0 E α i,k [∆ξ h k ]| 2 + | N h t -1 i=0 ∆ξ h i+1 -E α i,k [∆ξ h k ] 2    .
We take the conditional expectation E α i,k ,

E α i,k ξ h (t) -x 2 ≤ 2    E α i,k   N h t -1 i=0 E α i,k [∆ξ h k ] 2   E α i,k   N h t -1 i=0 ∆ξ h i+1 -E α i,k [∆ξ h k ] 2      .
From Remark 3.5.8, the chain {ξ h } h satisfies the local consistency conditions (3.3.20)- (3.3.21).

Then we obtain

E α i,k ξ h (t) -x 2 ≤ 2    E α i,k   N h t -1 i=0 b(ξ h k )∆t| 2   + E α i,k   N h t -1 i=0 |2σ∆t + o(∆t(ξ h k ))      ≤ 2K 2 t 2 + 2(K + o(h))t,
where K is a bound for |b(x)| ∨ |σ(x)| for all x. Finally, from Tchebyshev's inequality, the first condition (3.5.43) is hold.

Step 2: Verifying that

lim δ→0 sup n sup τ ∈T n T E[1 ∧ |x n (τ + δ) -x n (τ )|] = 0.
Thanks to the local consistency condition, we have

E[1 ∧ |ξ h (τ + δ) -ξ h (τ )|] ≤ E[|ξ h (τ + δ) -ξ h (τ )|] 1/2 ≤ 2K 2 δ 2 + 2(K + o(h) )δ 1/2 -→ δ→0 0
Finally, from the result 3.5.9 of Kushner and Dupuis [START_REF] Kushner | Numerical methods for stochastic control problems in continuous time[END_REF], we obtain that the chain {ξ h , h > 0} is tight.

Therefore, we have the following result: Lemma 3.5.11. The sequence (ξ h ) h converges to a Càdlàg process X in D[0, T ] such that L( X) corresponds to the limit of L(ξ h ).

Proof. By tightness proved on Proposition 3.5.10, we may extract a weakly convergent subsequence, again referred to as (ξ h , h > 0).

To prove that ξ h converges weakly to X ∈ D[0, T ] such that L( X) corresponds to the limit of L(ξ h ), we need to use the result of Billingsley, 2013, ( [19], theorem 13.4, page 142 ). Then we should prove the almost sure continuity of the limit process trajectories X.

So, thanks to the construction of ξ h , we get

sup 0<t<T |ξ h (t) -ξ h (t -)| = dx = h -→ h→0 0.
Then we conclude that ξ h converge to a Càdlàg process X in D[0, T ].

Based also on the tightness of m and the results of Billingsley [START_REF] Billingsley | Convergence of probability measures[END_REF], we get Lemma 3.5.12. Let (m h ) h the sequence of laws associated to ξ h in P(D[0, T ]), as mentioned in previous Theorem 3.5.6. Then, the sequence (m h ) h is tight and there exists an subsequence, still noted (m h ) h converging weakly to some m in P(D[0, T ]).

Proof. From remark 3.5.1, the space P(D[0, T ]) is a complete and separable space. According results of Billengsley ( [19]" Theorem 1.3) each probability measure on P(D[0, T ]) is tight. Thurs the sequence (m h ) h is tight then we can extract a sub-sequence that converge to some limit m in P(D[0, T ]).

And finally we conclude Lemma 3.5.13. For all (t i ) i a finite sequence in [0, T ], we have

L(ξ h (t 1 ), ... , ξ h (t k )) ≡ m h t 1 ⊗ .... ⊗ m h t k converges to mt 1 ⊗ .... ⊗ mt k when h → 0.
Proof. To prove this last point of the lemma, thanks again to the results of Billingsley ( [19], theorem 2.7, page 21) and the Lemma 3.5.11, we have the almost sure continuity of the limit process trajectories X. As it explained in section 13 of Billingsley [START_REF] Billingsley | Convergence of probability measures[END_REF] for tightness in D[0, T ],

we conclude that .

Thus we inject ( m, ξ h ) into the control problem (3.4.29) in order to show that the solution of Fokker-Planck and the solution of equation (3.3.26) converge to the same limit.

So let us denote by (ᾱ h , ξ h ) the optimal policy associated to the control problem

(ᾱ h , ξh ) = arg min (α h ,ξ h )∈A h Jh (i, k, α h , ξ h , m). (3.5.45)
Then, similarly to (3.3.19), we obtain the following equations for the associated discrete value

function U h Ū h i,k = inf α∈A h    L h (x i , α * ,k ) + F [ mt k ](x i ) dt + j P h ij (k, α i,k ) Ū h j,k+1 ]    , with terminal condition Ū h * ,N h t = Φ[ mT ]. Moreover, ∀ 0 ≤ k ≤ N h t -1 and ∀ 1 ≤ i ≤ N h
x , the optimal action ᾱ verifies

ᾱh i,k = arg min α∈A h   [L h (α)] i,k dt + j P h ij (k, α) Ū h j,k+1   .
Let us define mh as the law of the controlled Markov chain (m h 0 , ᾱh , {P h (k, ᾱh * k ), 0 ≤ k ≤ N h t }) with its piecewise constant interpolation density function mh (x, t).

   mh * k = P h (k, ᾱh * k ) T M h * k-1 = ℓ≤k-1 (P h (ℓ, ᾱh * ℓ )) T m h 0 , for 1 ≤ k ≤ N h t , mh * 0 = m h 0 ,
(3.5.46) Proposition 3.5.14. α h,R converges weakly to some relaxed process α R as h → 0 such that

J R (x, t, α R , m) = inf α R ∈R(A×[0,T ]) J R (x, t, α R , m), ∀t ∈ [0, T ], (3.5.47)
Then, the solution of a MFG discrete problem (3.3.24) converges weakly to a solution of a MFG continuous problem (3.2.5).

Numerical simulation

The theoretical study of the approach proposed in this thesis pushes the numerical validation.

Therefore in order to validate our approach, we study the case of Linear Quadratic Mean Field Games(LQ-MFG for short). First we present the class of LQ-MFG. Later we solve the associated MFG PDEs using the Markov chain approximation approach. To compare the results and certify it we applied for a model with explicit solution.

Linear quadratic model

We introduce a Linear Quadratic model of MFG where the cost functional is quadratic in all state variables, control variables and the mean field terms, while the controlled dynamics are linear. The Linear Quadratic model was well studied in Bensoussan et al. [START_REF] Bensoussan | Mean field games and mean field type control theory[END_REF]. Let us consider the following linear quadratic form for L, F, b and Φ:

L(x, α) := 1 2 (x * Qx + α * Rα) , F (m, x) := 1 2 x -S εm(ε)dε * Q x -S εm(ε)dε , b(t, x, α) := Ax + Bα, Φ(m, x) := 1 2 x * QT x + x -ST εm(ε)dε * QT x -ST εm(ε)dε ,
where A, B, R, Q, Q, S are given constants.

We remain that we look to solve the following coupled HJB-FP system of PDE

           -∂ t u(t, x) - 1 2 σ 2 ∆u(t, x) + H(x, ∂ x u(t, x)) = F [m(t, x)](x) ∂ t m(t, x) - 1 2 σ 2 ∆m(t, x) -∂ x (m(t, x)∂ p H(x, ∂ x u(t, x)) = 0 u(T, x) = Φ(m(T, x), x), m(0, x) = µ 0 (x).
(3.6.48) find a trajectory that has the desired boundary value.

In other words, by initializing z 0 , we solve the equation (3.6.52) by replacing z by z 0 . Then we inject the solution z 1 in (3.6.55) and solve it by initializing r 0 . We repeat these steps by injecting each time the solution found z k in the equations for the next step and so on. We repeat this procedure until we find c.

Algorithm 1 Shooting method

Initial (z 0 , r 0 ).

for i = 0 to N -1 do:

Find z k+1 solution of (3.6.52)

dz dt = BR -1 Br k t -A z k t + BR -1 Br k t ,
Find r k+1 (3.6.55)

dr dt = -Ar k t + 2P t BR -1 Br k t + Sz k+1 t Q, Return (z N , r N ) Repeat until found r k T = r T .
Therefore, the first step is to solve (3.6.54) and the second one is to solve the coupled system (3.6.52)-(3.6.55) for z and r using shooting method and we end up with find the solution s of (3.6.56). To simplify the calculation, we take the Gaussian probability density defined as follows

m(t, x) = 1 √ 2πv(t) exp (x -z(t)) 2 2v 2 (t) .
Moreover, we know that the optimal control α k is defined as follows

α k = min α J(x, t, α, m).
Thus, in this linear quadratic framework, we obtain this explicit form :

α k := - B R ∂ x u(t, x).
(3.6.57)

Numerical results

The numerical solution of the problem is based on a fixed point argument. The distribution is initialized and then an iteration method is applied to the optimal policies. At each iteration the optimal strategy is computed, then the value function and the new distribution until stability. By extending these intervals, we notice, according to the figures above, that the numerical solution of the value function is closer to the explicit solution. On the other hand, the numerical solution of the probability measure, for this test, is also closer to the explicit solution. On the other hand, for a larger value of A, we also lose the convergence of the solutions.

LEARNING DYNAMIC UTILITIES

Introduction

This chapter aim to propose an algorithm to learn the utility of an agent by observing her decisions at discrete (possibly random) dates (τ i ) i . There are many settings in which this capability is very useful. In cooperative setting, we may want to help the agent make good decisions. This is the principle of robo-advisors which are online investment management services that employ mathematical algorithms to provide financial advice with minimal human intervention. Based on online questionnaires to obtain information about the client's degree of risk-aversion, financial status, and desired return on investment, the robo-advisors use their algorithms to manage and allocate client assets in the most efficient way possible. In a more competitive setting, we may want to predict the agent's actions so as to better optimize our own payoffs, this is the case of two-player A and B where A is trying to learn the utility's of B, to optimize her own payoff which also depends on B's actions and utility.

From a theoretical and application point of view this problem of recovering the utility of an agent from observations is an old inverse problem: Same questions was addressed by the economist Samuelson in the 40's [START_REF] Samuelson | A note on the pure theory of consumer's behavior[END_REF][START_REF] Samuelson | Consumption theory in terms of revealed preference[END_REF], with the theory of "revealed preference" where the observable is the consumption process. Since then, the theory has been growing in interest under the pressure of the economic reality, which created news incentives for different approaches, see Chambers & Echenique [START_REF] Chambers | Revealed preference theory[END_REF]. An example is the evolutionary economics by Arthur [START_REF] Arthur | Complexity and the economy[END_REF].

Economies are considered as complex evolutionary systems, where the agents try to predict the outcomes of their actions, and how the market would be modified by their decisions.

Beyond the fields of economics and finance, in the last few decades this forward-looking viewpoint has been also considered in many optimization problems as a consequence of the expansion of computer science and e-commerce. The forward modeling allows anticipations on the future values of observations But, the agents also need to adjust their (random) preferences over time, following an "inverse thinking" approach as has been suggested by Gomez-Ramirez [START_REF] Gomez-Ramirez | Don't blame the economists. it is an inverse problem![END_REF].

In the field of machine learning, we refer for example to Friedman and Sandow [START_REF] Fürnkranz | Preference learning[END_REF], Chajewska, Koller, and Ormoneit [START_REF] Chajewska | Learning an agent's utility function by observing behavior[END_REF]: and Stahl [START_REF] Stahl | Approximation of utility functions by learning similarity measures[END_REF] and Hibbard [START_REF] Hibbard | Model-based utility functions[END_REF]. As well, reasoning with preferences has been recognized as a particularly promising research direction for artificial intelligence see 123

Nielsen and Jensen. [START_REF] Nielsen | Learning a decision maker's utility function from (possibly) inconsistent behavior[END_REF] and Qi, Xu, and Lafferty. [START_REF] Qi | Learning high-dimensional concave utility functions for discrete choice models[END_REF]. For e-commerce, see [START_REF] William | Utility functions in autonomic systems[END_REF]: The principal idea is to build the utility function of a user from his search history and purchases on the Internet. This utility will then be used to target the user by proposing products or baskets of goods that we know are likely to be of interest to him and at specific prices evaluated from this utility function.

Since the problem is posed forward in time and given the randomness of the universe, the utility that we seek to construct from the observations must be a dynamic utility coherent in time and consistent with this universe (in a sense that we specify further on). For these reasons, we are concerned, in this work by, the concept of "forward" dynamic utilities which was first introduced and considered by Musiela and Zariphopoulou [START_REF] Musiela | Backward and forward utilities and the associated pricing systems: The case study of the binomial model[END_REF][START_REF] Musiela | Portfolio choice under space-time monotone performance criteria[END_REF][START_REF] Musiela | Stochastic partial differential equations and portfolio choice[END_REF] to model the preferences of an agent in a continuous semimartingale financial market, see also Henderson and Hobson [START_REF] Henderson | Horizon-unbiased utility functions[END_REF], El Karoui and Mrad [START_REF] El Karoui | An Exact connection between two Solvable SDEs and a Non Linear Utility Stochastic PDEs[END_REF] and El Karoui & al [START_REF] El Karoui | Consistent utility of investment and consumption: a forward/backward spde viewpoint[END_REF] for a model with consumption. See also Matoussi and Mrad [89] for a semimartingale market with Jumps.

Recently in [START_REF] El Karoui | Recover Dynamic Utility from Observable Process: Application to the economic equilibrium[END_REF], the authors have addressed the problem of revealed utility in a general dynamic framework beyond finance and economics. They establish a necessary and sufficient condition for the existence of at least one solution to the "revealed" utility problem and given a simple way for recovering the utility of an agent provided that the observed process X(x)

is monotonic with respect to its initial condition x, which is an assumption that is satisfied in finance and economics, one does not invest more to earn less. In a more general framework, it also remains true as soon as we have some regularity (Lipschitz is enough) on the diffusion coefficients of X.

They are the results of this last paper that we implement in this work. Let us briefly present the idea and different steps of the revealed dynamic utility problem as in [START_REF] El Karoui | Recover Dynamic Utility from Observable Process: Application to the economic equilibrium[END_REF]. By definition, the observable is a so-called dynamic positive characteristic process {X t (x)} considered for different values of its initial condition X 0 (x) = x > 0, and assumed to be increasing in x (to be coherent with the expected utility criterium). The goal is to recover, from a given initial utility function u, a stochastic dynamic utility U = {U (t, z, ω), z > 0}, "revealed optimally" in the sense that at any (stopping) time τ , the preference for the observable process is in mean equal to its value at time 0, E(U (τ, X τ (x))) = u(x): from the probabilistic dynamic view point, on a given filtered probability space, the performance process "U (t, X t (x)) is a martingale". Focusing on the concavity of the utility criterium U, tools of convex analysis play a key role, especially the invertible decreasing marginal utility U z (t, z). Its allows to define the convex Fenchel-Legendre transform Ũ (t, y) of U (t, x), by U (t, z) -z U z (t, z) = Ũ (t, U z (t, z)), but also to linearize the recovery problem, by using the one to one correspondence between the class of dynamic utilities U (revealed by X) and the adjoint processes Y candidate to play the role of U z (t, X t ), more precisely {Y t (u z (x)) = U z (t, X t (x))}. Since the characteristic process is invertible, the triplet {(u, X t (x), U (t, z))} is in one to one correspondence with the triplet { u, X t (x), Y t (u z (x)) }. In fact, denoting by {X t (z) := (X t ) -1 (z)} the inverse flow of X, we have the characterization

U z (t, z) = Y t (u z (X t (z)), and
U (t, z) = U (t, z 0 ) + z z 0 Y t (u z (X t (x))dz.
Note that in the continuous semimartingale [START_REF] El Karoui | An Exact connection between two Solvable SDEs and a Non Linear Utility Stochastic PDEs[END_REF] and semimartingale with jumps framework [START_REF] Matoussi | Dynamic Utility and related nonlinear SPDE driven by Lévy Noise[END_REF],

the authors have shown that if U is a regular dynamic utility, consistent with a given financial market, it is necessarily in this form with X being the optimal portfolio and Y being the optimal state price density process.

The goal, in the revealed utility problem, is then to identify the class of the adjoint processes {Y t (y)} in bijection with a revealed dynamic utility U, such that U (t, X t (x)) is a martingale. A quasi-necessary condition on the triplet { u, X t (x), Y t (y) is "{X t (x)Y t (y)} is a supermartingale for any x, y and {X t (x)Y t (u z (x))} is a martingale for y = u z (x) "; this last condition guarantees that the conjugate dynamic process { Ũ (t, Y t (y))} read along the adjoint process {Y t (y)}, is a martingale.

Note that the revealed utility problem does not necessarily have a unique solution, there are as many utilities as there are processes Y satisfying this last condition. From a financial point of view, the observable X is the optimal portfolio of the investor and Y is his optimal state price (pricing kernel). If the market is complete, Y is unique and corresponds, up to a discount factor, to the risk neutral density. However, if the market is incomplete, several choices of Y are possible. This work is concertized in the preprint [START_REF] Mrad | Learning dynamic utilities[END_REF].

Preliminary results on the numerical approximation of dynamic utility

Automated investment managers or robo-advisors were imposed these last years in the financial and economic world to replace or alternate with traditional human advisors. The roboadvisors are online platforms that provide financial advice or portfolio management to an investor or a client. The performance of the robo-advisor strongly depends on its ability to accurately assess the investor's risk tolerance. They have many advantages over the investment services offered by traditional management consultants. Robo-advisor knows the investor better than a traditional asset manager. Because of this better knowledge, the robo-advisor may propose a more appropriate asset allocation. In addition, it performs the task in a systematic way and implements an automated rebalancing process. From the client's perspective, the biggest advantage of robo-Advisor is that it provides a low-cost alternative to traditional investing.

The idea of these platforms is to develop a new investment framework in which the robo-advisor not only manages the client's portfolio but also reacts regularly with him/her to obtain updated information on his/her performance during time. More precisely, the client is giving an information about his/her preferences to the robo-advisor only at specific update dates (τ i ) i . So, between two update dates, the robo-advisor does not receive any information from the customer. Therefore, it develops a framework to learn the investor's preferences between two update dates. In other words, by observing the data communicated by the client at interaction dates, robo-advisor approximates the preferences of the client to make investment decisions and it interacts repeatedly with the client to update its informations by asking from the investor his/her new data in order to avoid making decisions based on stale information. In general, the robo-advisor solicits this information through targeted online questionnaires, and transfers it to digital data. We assume that we have observations for several different initial conditions (z j ) j .

Such informations are not necessarily collected in the questionnaires of the robo-Advisors, but maybe it is necessary because we know very well that the choices of an agent are strongly conditioned by its initial position (e.g initial wealth). Moreover, without these observations, we have a little chance to find the dynamic utility of the agent, see the papers [START_REF] El Karoui | Consistent utility of investment and consumption: a forward/backward spde viewpoint[END_REF][START_REF] El Karoui | An Exact connection between two Solvable SDEs and a Non Linear Utility Stochastic PDEs[END_REF][START_REF] El Karoui | Recover Dynamic Utility from Observable Process: Application to the economic equilibrium[END_REF][START_REF] Matoussi | Dynamic Utility and related nonlinear SPDE driven by Lévy Noise[END_REF]. In this work the idea is the following

• Step 1: From the observations X τ i (z j ) and Y τ i (y j ) we construct the marginal utility at the points (τ i , z j ), using the characterization

U z (τ i , z j ) = Y τ i (u z (X τ i (z j )))
, where X t (z) = X -1 t (z) denotes the inverse maps of z → X t (z).

• Step 2: We integrate numerically with respect to the spatial parameter and get U (τ i , z j ).

• Step 3: This data is used as a starting point to learn the utility U at any point (t, z), using several artificial intelligence methods, specifically the deep learning method.

• Step 4: Finally we try to predict the utility at a reasonable horizon (to be able to make decisions in place of the client) and we conclude with a comparison of these different approaches.

Note, as we do not dispose of observations, we will generate them by ourselves and this by assuming that the processes X and Y evolve according to two stochastic differential equations with jumps.

The model

In this section, we suppose that the filtered probability space (Ω, F, P) supports a q-dimensional standard Brownian motion B = (B 1 , . . . , B q ) on and an independent q ′ -dimensional Poisson random measure N on [0, ∞[×IR q ′ with constant time dependent intensity measure λdt × ν(de) defined on the filtered probability space (Ω, F, F, P). λ is the intensity of jumps and ν is a finite positive measure on IR q ′ , satisfying standard integrability assumptions. We also denote by Ñ the compensated version of N :

Ñ (dt, de) = N (dt, de) -ν(de)λ(t)dt.

We are concerned by two IR d -valued stochastic processes X and Y , solutions of the following stochastic differential equations (SDE for short) Before giving the Euler scheme, we first expose how to simulate the integral with respect to the Poisson measure. We follow the presentation of [START_REF] Mordecki | Adaptive weak approximation of diffusions with jumps[END_REF].

dX t (x) = µ(t, X t (x))dt + q i=1 σ i (t, X t (x))dB i t + IR q ′ h(t, X t -(y), e) Ñ (dt, de), X 0 (x) = x, ( 4
|b(t, x) -b(s, x)| + |γ(t, x) -γ(s, x)| ≤ C Y (1 + |x|)|t -s| α Y . |g(t, x, e) -g(s, x, e)| ≤ C Y (e)(1 + |x|)|t -s| α Y . ( 4
Simulation of the integral with respect to the Poisson random measure Let us consider a sequence e 1 , e 2 , ... of independent random variables with common exponential distribution with parameter 1. We define

Λ(t) = t 0 λ(s)ds, t ∈ [0, T ].
To simplify, we take λ(t) = 1.

The number of jumps of the random Poisson measure N (dt, de) in an interval [0, t] is determined as

J(t) = max{k : k j=1 e j ≤ Λ(t)}.
We denotes by J = J(T ) the total number of jumps in [0, T ] and the jump times (τ k ) k of the Poisson measure defined by

τ k = Λ -1 ( k j=1 e j ), k ∈ 1, J ,
where Λ -1 is the right continuous inverse of Λ. The jump times can be computed recursively by

e k = τ k τ k-1 λ(s)ds, k ∈ 1, J .
Once the jump times are computed, we proceed to sample the marks {E k }, that, conditionally on the values of the jumps times, are independent random variables distributed respectively according to {ν(de)}. The random measure with intensity λ(t)dtÖν(de) can then be constructed as

N (dt, de) = J k=1 δ (τ k ,E k ) (dt, de).
Consequently, the stochastic integral with respect to the Poisson random measure can be defined as

t 0 IR d g(s, Y s -, e)N (dt, de) = J(t) k=1 G(τ k , Y τ - k , E k ), t ∈ [0, T ].

Euler schemes

Let N ≥ 1 and let us consider the discretization family

{ ti := i T N , i ∈ 0, N } of [0, T ].
We consider also the jump times {τ k , k ∈ 1, J } with corresponding marks {E k , k ∈ 1, J }. Consider the augmented partition given by the union )) converges to U z (., .) in any L p -norm, at the order β := min(α X , α Y , 1 2 ) w.r.t. N : For any p > 0, t ∈ [0, T ] and any z,

P N = {t l , l ∈ 0, N + J } = ti := i T N , i ∈ 0, N ∪ τ k , k ∈ 1,
|Y N 0,t (u z (X N 0,t (z))) -U z (t, z) | Lp = O(N -β ).
Thus with a simple integration, we can calculate an approximation U of the utility U at any time t i , by

U(t i , z j ) = j-1 k=0 Y N t i (u z (X N t i (z k ))(z k+1 -z k ),
since z 0 = 0 and we assume u(t, 0) = 0 a.s. for any t ≥ 0.

This approach allows us to generate the utility of the agent at the dates τ i of interaction with the robo-advisor on a grid (z j ) points. Here are some numerical results obtained by applying this process, scheme 1 for simulating Y and scheme 2 for the inverse of X. This approach is based on the work of Mrad and his co-authors [START_REF] El Karoui | An Exact connection between two Solvable SDEs and a Non Linear Utility Stochastic PDEs[END_REF][START_REF] El Karoui | Recover Dynamic Utility from Observable Process: Application to the economic equilibrium[END_REF][START_REF] Matoussi | Dynamic Utility and related nonlinear SPDE driven by Lévy Noise[END_REF], it allows us to estimate the agent's utility at dates and points of interaction . This is a very important step as until

(U i j ) j∈ 1,n := (U(τ i , z j )) j∈ 1,n .
• Choice of the model: choose the best model to fit our data set. Among the predefined models in SKlearn library, we can mention -Decision Tree -Support vector Machine ( SVM) -Neural Network -K nearest neighbors ( KNN).

• Fitting SKlearn.fit(Z in , U i ) : The objective is to fit the model, (i.e), the machine must learn from the model:

1. The samples matrix Z in . The size of Z in is typically (n_samples, n_features), which means that samples are represented as lines and features are represented as columns.

2. The target values U i is usually 1d array where the i th entry corresponds to the target of the i th sample of Z in .

• Predicting: SKlearn.predict(Z in ): Once the estimator is fitted, it can be used for predicting a future value U p,i using the following syntax ( we use the notation "p" for prediction)

U p,i := SKlearn.predict(Z in ).

• Evaluation: SKlearn.score(Z in , U i ): The accuracy of our model is obtained using the function Model.score. This function returns the coefficient of determination R 2 of the prediction. The coefficient R 2 is defined as

R 2 = 1 - (U i -U p,i ) 2 (U i -Ū ) 2 ,
with Ū := 1 n n j=1 U i j represents the mean of (U i j ) j . The best possible score is 1.0 and it can be negative (because the model can be arbitrarily worse).

Remarks 3. Train the model using all the Data, an also use it to test the result is not a good way to measure the performance of our model and may involve overfitting. So to avoid this, we choose to split the data, usually we make 80/20: Train set (80%) and Test set (20%).

To create a train set and test set, thanks to python again, we use the train_test_split function of Sklearn. SKlearn.train_test_split is a function in Sklearn model selection for splitting data arrays into two subsets: for training data and for testing data. With this function, we don't need to divide the data manually. As we said above, using the same dataset for both training and testing steps, increases the chances of inaccurate predictions.

In this section, we use the method described above to learn the utility of an investor. Firstly, we create the target, then we want to implement (separately) two packages from Scikit-learn library: Support vector Machine and Neural Network model to create the model and we compare the results. To recap, we are looking to implement the following algorithm 3 for SVM and Neural Network models We want to apply the learning methods described above to predict the utility of an investor according to the information observed by the Robo-Advisor, the preferences of the customer or the response of the customer to a questionnaire.

Algorithm 3 Implementation algorithm in Python

Support Vector Machine

SVM approach

In machine learning, support-vector machines 2. (a, b) represent the parameters of weight (a i ) i and biais b: we choose it randomly at the beginning then we try to find the optimal values in the algorithm.

3. U i = (U i j ) j := (U(τ i , z j )) j represents the target. We remain that the target is defined by a column vector of the matrix U(τ i , z j ) i∈ 1,N ,j∈ 1,n at fixed τ i . 4. ϵ is the width of the margin of hyperplan.

The idea is to build a plane of equation ax + b = 0 with a margin ϵ, in other words, we want to build a hyperplane of width 2ϵ (see figure 4 By constructing this hyperplane with a margin of tolerance , we can have points outside the This problem allow some observations to fall on the wrong side of the margin, but will penalized them by the parameter C. Finally, we can introduce our optimization problem:

min a∈IR n ,b∈IR    1 2 ||a|| 2 + C l j=1 (ξ j + ξ * j )    , ( 4.3.10)
under the following constraint

           U i j -(a j Z in j + b) ≤ ϵ + ξ j , (a j Z in j + b) -U i j ≤ ϵ + ξ * j , ξ j , ξ * j ≥ 0, ∀j, (4.3.11)
where l is the number of the points outside the margin. Basically, the idea of SVR model is to minimize the sum of squared weights by taking into account the misplaced observations in order to maximize the margin.

A second idea is the kernel method. It is an extension to non-linear cases. It is quite simple and relies on the projection of the data in a higher dimensional space in which the problem becomes linear. The kernel method is a mathematical trick to solve this problem elegantly, by defining the transformations ϕ and scalar products via a kernel k(a, b) = ϕ(a) T ϕ(b) such that the equation of hyperplan is as follow

U p j := a j ϕ(Z in j ) + b.
Actually, it is far from being easy to identify the transformation function ϕ, so the second problem is rather than choosing ϕ, we choose the kernel function k. A kernel is a function k capable of computing the scalar product ϕ(a) T ϕ(b) based only on the original vector a and b without having to compute the transformation ϕ. In practice, this function is used ϕ indirectly, without knowing it. The most used kernels are

• Linear : k(a, b) = a T b. • Polynomial : k(a, b) = (γa T b + r) d . • Gaussian RBF : k(a, b) = exp(-γ||a -b|| 2 ).
Nevertheless, there is no rule for choosing the most adapted kernel for a specific problem. This technique is based on distance minimization.

in different initial values of wealth and as target U i = (U i j ) j the utility value for a fixed value of time τ i and different values (z j ) j .

Then, under given parameters C > 0 and choosing Gaussian RBF as kernel k, defined by:

k(z i , z j ) = exp(-γ||z i -z j || 2 ).
This kernel function is typical and very widely used since it allows with an appropriate setting to represent any function. The Gaussian kernel represents a measure of similarity between samples (x i , x j ).

We assume that the equation of hyperplan is as follow U p j := a j ϕ(Z in j )+b, where ϕ is the implicit mapping embedded in the RBF kernel k(x, y), see [START_REF] Vert | A primer on kernel methods[END_REF].

Then, the equation of the decision boundary becomes

a j ϕ(Z in j ) + b = ϵ , a j ϕ(Z in j ) + b = -ϵ.
Actually, there is no rule to choose the regularization parameter C, we need to just test to find the most adjusted value to keep.

Solving the following optimization problem under the constraint (4.3.11):

min a∈IR n ,b∈IR 1 2 ||a|| 2 + C l i=1 (ξ i + ξ * i ) ,
we find the learned or predicted value denoted by U p . To evaluate SVR model, we visualize, on figure 4.6, the evolution of the Mean Squared Loss defined by

E i err,k := 1 2 N n j=1 (U i j -U p,k j ) 2
, which is the error at the k-th iteration. (4.3.14) We chose to evaluate the error of our learning model for the last vector of the utility matrix in order to have more visibility on the last step of the learning process. In order to perform some more tests with this learning model, we take, in this time, a column vector of the approximated utility matrix for different values of t and a fixed wealth value z as input data. Will it be possible to predict the dynamics of utility as a function of time t ?

In other words, we try to learn the dynamic of utility. We define our data (T j , U j ) in following way: as input T j := (t i ) i in different values of times between two update τ i and τ i+1 and as target

U j := (U(τ i , z j )) i∈ 1,J ,
where U j is the j-th vector-line of the generated utility matrix (U(τ i , z j )) (i,j)∈ 1,J × 1,n . Then, with a for loop from i = 1 to N , we run the following steps for different values of time τ i : (Z in j ) j = (z) j still the input, the target is (U i j ) j = (U(τ i , z j )) j . Then, we train the model with Fit((Z in j ) j ) and finally, we test it by predict((Z in j ) j ,(U i j ) j ). For this model, we have three important parameters to set

• Hidden layer sizes, to fix the number of neurons and number of hidden layer.

• Activation function for the hidden layer identity, useful to implement linear bottleneck, returns f (x) = x.

logistic: the logistic sigmoid function, returns f (x) = 1/(1 + exp(-x)).

tanh: the hyperbolic tan function, returns f (x) = tanh(x).

-Relu: the rectified linear unit function, returns f (x) = max(0, x)

• The solver for weight optimization:

lbfgs is an optimizer in the family of quasi-Newton methods.

sgd refers to stochastic gradient descent.

adam refers to a stochastic gradient-based optimizer proposed by Kingma, Diederik, and Jimmy Ba.

Example: We develop a simple example of a Neural Network. We introduce a model with three hidden layers and we set the activation function as Activation=tanh:

1. Input layer : it contains the data Z in = (Z in j ) j , j = 1, ...., n 0 .

2. First hidden layer: it contains n 1 perceptron having for input (Z in j ) j∈{1,...,n 0 } ∈ IR n 0 and for output Z 1,i ∈ IR n 1 defined as, for a 1,i ∈ IR n 1 ,n and b 1,i ∈ IR n 1 , Z where we take σ(x) = tanh(x).

(1) 1 3. Second hidden layer: it contains n 2 perceptron having for input (Z 1,i j ) j∈{1,...,n 1 } ∈ IR n 1 and for output Z 2,i ∈ IR n 2 given by , for a 2,i ∈ IR n 2 ,n 1 and b 2,i ∈ IR n 2 , Z 2,i k = σ( j=1,...,n 1 Z 1,i j a 2,i k,j + b 2,i k ), ∀k ∈ {1, ..., n 2 }, where we take σ(x) = tanh(x).

a 1,i 1,1 a 1,i 1,1 a 1,i 1,2 a 1,i 1,2 a 1,i 1,3 a 1,i 1,3 a 1,i 1,4 a 1,i 1,4 a 1,

4.

Third hidden layer: it contains n 3 perceptron having for input (Z 2,i j ) j∈{1,...,n 2 } ∈ IR n 2 and for output Z 3,i j ∈ IR n 3 defined as for a 3,i ∈ IR n 3 ,n 2 and b 3,i ∈ IR n 3 , Z 3,i j = σ( j=1,...,n 2 Z 2,i j a 3,i k,j + b 3,i k ), ∀k ∈ {1, ..., n 3 } where we take σ(x) = tanh(x). where activation function for the hidden layers is σ(x) = tanh(x). For the Neural Network, we start from the input Z (first layer) to reach the output U p (output layer), so that the information flows in a forward way. This is called forward propagation. During training, forward propagation can continue onward until it produces a cost J.

The idea of Machine learning is to let the machine find the parameters θ l,i k := (a, b) for the l-th and k-th node layer of the model that minimize the cost function J. In order to do this, we need to use a back-propagation learning algorithm: the most common example of such an algorithm is Gradient Descent. This back-propagation algorithm allows the information from the cost J to flow backward through the network in order to compute the gradient ∇J and to update the set of parameters θ l,i k = (a, b) as follows:

θ l,i k,j = θ l,i k,j -α ∂J(θ l,i k ) ∂θ l,i k,j , j = 0, ..., n k + 1, for the l-th hidden layer and the k-th node, where n l is the number of nodes for the l-th layer and α is a given learning rate.

Actually, the differences between these outputs U p and the target U form errors that are corrected via back-propagation, the weights of the Neural Network are then changed. By applying this step several times, the error tends to decrease and the network offers a better fitting or prediction.

Then for a fixed Neural Network structure, number of layers, number of neurons in each layer and fixed activation functions, the program therefore amounts to determine the set of parameters such that θ * := arg min θ J(θ).

In this work, we choose the LBFGS solver as Back-propagation algorithm of the optimization problem. LBFGS is named from the initials of the mathematicians Broyden, Fletcher, Goldfarb and Shanno, who discovered it independently in the late 1960s. This learning algorithm is based on Newton's method which fits the weights using additionally the Hessian matrix H gives the second derivatives of the error function with respect to the weights in the following way: θ l,i k,j = θ l,i k,j -αH -1 ∂J(θ l,i k ) ∂θ l,i k,j , j = 0, ..., n k + 1, for the l-th hidden layer and the k-th node.

For this algorithm, we choose to optimize the squared-loss function J defined as 2. Update the parameters θ l,i k,j to this form: θ l,i k,j = θ l,i k,j -αH -1 ∂J(θ l,i k ) ∂θ l,i k,j , j = 0, ..., n k + 1, for the l-th hidden layer and the k-th node.

where α is the learning rate and n k is the number of nodes for the (l -1)-th layer.

Numerical results of Neural Network prediction

In this section, as the other models, we want to learn client's utility at different points of time τ .

We run algorithm 3 for different values of wealth z and for different points of time τ in a for loop.

Therefore, we take a data set ((Z in j ) j , (U i j ) j ) such that input set is give by (Z in j ) j = (z) j and the target value (U i j ) j is given by (U i j ) j = (U(τ i , z j )) j , ∀j ∈ 1, n .

As the other models, machine succeeded the prediction of utility for different points of time t ( see Figure 4.17

U j = (U j i ) i = (U(τ i , z j )) i ,
where (U(τ i , z j )) i is a line vector with utility value for a fixed value of wealth z j and different points of time τ .

We can deduce that the Neural Network prediction is better than the other predictions with The prediction of this model was more accurate than the other models, with a determination score R 2 = 0.95. We can conclude that this is the most successful model for our data. According to hidden layers. This implies that a deep Neural Network with the same number of parameters as an SVM is always more complex than the lattest one. Support vector machines effectively use only a subset of a data set as training data. This is because they reliably identify the decision boundary based on only the support vectors. Therefore, for well-separated classes, the number of observations required to train an SVM is not large.

We will dedicate the Section 4.5 to compare the three models used in this work to predict the dynamic utility of an investor.

Learning with Neural Network model

We suppose that the robo-advisor has collected more information than the one communicated by the agent, can it still predict the utility of the client from this data. Roughly speaking, we take a vector of data that contain more samples than the initial one. Then we get the following result, using the same data (Z in , U t,i ),

Comparison of machine learning models

In this section, we aim to compare between different methods of machine learning considered in this chapter Thus, we implemented the three models for the same input values and the same values of targeted utility built the previous section. In the following table, we present some of the predicted values (at randomly selected times) obtained using the 3 methods on the test set: The mean of the values predicted by the neural network is very close to the mean of the target values, which reinforces and confirms our conclusions.

U t,

Conclusion and perspectives

We deduce that the Neural Network model is the best prediction model for our dataset. But the other models are not so bad as well. Unfortunately, there are no clear rules to follow to choose the model to use from the beginning. In fact, it is recommended to start with the easiest and simplest examples, if these models do not work then consider using a more sophisticated model. Abstract : This thesis is devoted to the study of stochastic controls and their applications. In the first chapter, we studied a linear quadratic mean field control problem in the framework with jump. We used an approach based on a weak formulation of the optimality principle of standard martingales to prove the existence of an optimal control. We have extended the result found for a model of exhaustible energy production with extraction. In the second chapter, we are interested in the numerical solution of a mean field game problem. The characterization of the optimal strategies of the interacting agents is obtained by studying a system of coupled nonlinear PDEs allowing the determination of the agent's characteristics as well as the distribution of these characteristics in the population. We have presented a numerical scheme to determine the couple (u,m) associated to the value function and the probability density, based on a fixed point argument of the coupled system of the Hamilton Jacobi Bellman equation and the Fokker Planck equation. We applied our approach for a linear quadratic example with an explicit solution in order to compare the two numerical and explicit solutions. In the third chapter, we introduce an algorithm to "predict" the dynamic utility of an agent by observing its decisions at discrete dates. This is the principle of Robo-advisors which are online investment management services that use mathematical algorithms to provide financial services with minimal human intervention. The idea of this work is to learn the dynamic utility of an investor in a financial market with defaults using different Machine Learning methods.

3 : 1 . 1 . 3 .Theorem 1 . 1 . 3 .

 3113113 e)λ(de) + χ Dt E[R t (e)]λ(de) dt + Z t dW t + χ R t (e)π(de, dt), Y T = ξ, (1.1.4) where ξ is an F T -measurable random variable and A, Â, B, B, C, Ĉ, D, D satisfy assumption 1.1.Assumption The matrix valued functions A, Â, B, B, C, Ĉ, D, D are uniformly bounded measurable functions Under the above assumptions, the existence result is stated as follows ( Tang and Meng [114]) Let Assumption 1.1.3 be satisfied, then for any (ξ, α) ∈ L 2 (F T ) × A, the state equation (1.1.4) has a unique solution (Y, Z, R) ∈ S 2 F (0, T : IR) × H 2 F (0, T : IR d ) × H 2 ν (0, T : IR).

Assumption 1 . 1 . 6 .

 116 .1.18) under Assumptions 1.1.4-1.1.5 by adding also the following one: The coefficients in equation (1.1.18) satisfy:

Assumption 1 . 3 . 2 .

 132 .3.43) • (HP4) ∇ y b, ∇ y γ and ∇ y g satisfy (HP3) with (C Y,∇ , C Y,∇ (e)) in place of (C Y , C Y (e)). (HP1) and (HP3) are satisfied for µ, σ and h (instead of b, γ and g) with coefficient C X , C X (e) and α X (instead of C Y , C Y (e) and α Y ).

  where B is a d-dimendional Brownian motion and π is the compensated version of independent Poisson random measure π on [0, ∞] × IR with intensity measure λ(t)dt × ν(de). The triplet (β, γ, H) denotes the local characteristics of U with values in IR × IR d × IR. Denotes by U z and U zz the first and the second derivative of U with respect to z. Then, the dynamic utility U , associated to the investor's portfolio optimization, is consistent if it solves the second-order fully nonlinear SPDE driven by Lévy noise and of HJB type

  cess, and A, Ã, B, B, C, C, D, D, F , F , G, G are deterministic matrix-valued functions such that A, Ã, C, C : [0, T ] → IR d×d , B, B, D, D : [0, T ] → IR d×m . In the other hand F, F : [0, T ]×Ω×χ → IR d×d and G, G : [0, T ]×χ → IR m×d are P⊗B(χ)-measurable process .

  2.5) are vector-valued F-progressively measurable processes such that M : [0, T ] × Ω → IR d , H : [0, T ] × Ω → IR m and L : Ω → IR d and the other coefficients Q, Q, P, P , N, Ñ , I, Ĩ, are deterministic matrix-valued functions such that Q, Q : [0, T ] → R d×d , P, P ∈ R d×d , N, Ñ : [0, T ] → R m×m and I, Ĩ : [0, T ] → R m×d . The symbol ⊤ denotes the transpose operator of any vector or matrix. Now, we assume the following conditions on the coefficients of the problem :

 d

  Xs = (A Xs + Bα s )ds + (C Xs + Dα s )dW s + X (F s (e) Xα,x s + G s (e)α s )π(de, ds), X0 = x. (2.3.31) A straightforward extention of Yong and Zhou ( [118], Theorem 7.2 p.320 ) to the jump diffusion case, and under the assumptions (H1) and (H2) (i)-(iii), there exists a unique solution K ∈ L ∞ ([0, T ], S d ) to the equation (2.3.30) and the solution satisfies K t ≥ 0.

  K , Z Λ valued in S d and some IF -adapted processes . Y , Z Y , Z Y,0 valued in IR d , an IF -predictable process R Y valued in IR d and a continuous function .

  e)K F (e)λ(de), ÎK := Î + D⊤ K Ĉ + χ Ĝ⊤ (e)K F (e)λ(de), N K := N + D⊤ K D + χ Ĝ⊤ (e)K Ĝ(e)λ(de).

.2. 3 )

 3 We assume that the function has the linear form : b(t, x, α) = b 1 (t, x) + b 2 (t)α, where (b h1 ) The mapping b 1 is measurable and bounded function on [0, T ] × K. (b h2 ) The mapping b 2 is measurable and bounded function .

2 . 8 )

 28 as the mean field game PDEs system (MF PDE in short). The first equation is backward in time. It is called a Hamilton-Jacobi equation where is associated with an optimal control problem. This equation describes the reaction of players to the mass. It presents the decisions based on where you want to be in the future. The second one is forward in time. This is called a transport or Fokker-Planck equation for the distribution of the agents. The equation describes the aggregation of the action of all players. It presents where the population behavior actually ends up based on the initial distribution. The existence of solutions of HJB-FP system (3.2.8

Remark 3 . 3 . 3 .Lemma 3 . 3 . 4 .

 333334 According to Kushner and Dupuis[START_REF] Kushner | Numerical methods for stochastic control problems in continuous time[END_REF], under a local consistency condition the minimal cost function associated to the approximated Markov chain converge to the minimal cost function associated to the original problem. Let (P h (k, α * k )) 0≤k≤N h t be given by (

  Now, we can define the MFG equilibrium for the Markov chain control problem (3.3.19) as follow Definition 3.3.6. (Discrete MFG equilibrium)

.5. 37 ) 3 . 5 . 3 .

 37353 Definition (Relaxed MFG Equilibrium)

.5. 38 ) 3 . 5 . 4 .

 38354 Proposition According to the result of Kushner and Dupuis in[START_REF] Kushner | Numerical methods for stochastic control problems in continuous time[END_REF] and for any m ∈ P , we get

  5.35)-(3.5.41)-(3.5.42) , if it is locally consistent with (3.2.2) in K and is also locally consistent with the reflection direction r(x).

Remark 3 . 5 . 8 .Theorem 3 . 5 . 9 .

 358359 By our construction, the Markov chain {ξ h } h>0 defined by (3.3.17)-(3.3.18) satisfies the local consistency conditions (3.3.20)-(3.3.21).First, we need to announced the following theorem: (Kushner and Dupuis [70]) Consider an arbitrary collection of processes {x n , n ∈ N} defined on the probability space (Ω, F, P) and taking values in D[0, ∞[. Assume that for each t ∈ [0, ∞[ and δ > 0 there exists compact K t,δ such that sup n P[x n (t) / ∈ K t,δ ] ≤ δ. (3.5.43)

.5. 44 )Proposition 3 . 5 . 10 .

 443510 Then {x n , n ∈ N} is tight.Now we can announced the following result:The collection {ξ h , h > 0} is tight.Proof. We will show that the conditions (3.5.43)-(3.5.44) hold for the process { ξh , h > 0} associated to the transition probabilities (3.3.18)-(3.3.17).

Figure 3 . 3 :Figure 3 . 5 :

 3335 Figure 3.3: The explicit probability measure Figure 3.4: The numerical probability measure

Figure 3 . 6 :Figure 3 . 7 :Figure 3 . 13 :Figure 3 . 14 : 2 Figure 3 . 15 :Test 4 :

 363731331423154 Figure 3.6: Explicit and numerical value function at T /2.

Figure 3 . 17 :Figure 3 . 18 : 2 Figure 3 . 19 :Figure 3 . 20 :Figure 3 . 21 : 2 Figure 3 . 22 :

 31731823193203212322 Figure 3.17: Explicit and Numerical value function t t 0 Figure 3.18: Explicit and Numerical value function at T /2 Figure 3.19: Explicit and Numerical value function at T

( 4 . 2 . 3 )

 423 where C Y (e) satisfies IR q ′ [C Y (e)] p ν(de) < ∞, ∀p ≥ 2. (HP2) b, γ and g are continuously space-differentiable functions s.t. ∇ y b, ∇ y γ and ∇ y g satisfy (HP1) with (C Y,∇ , C Y,∇ (e)) in place of (C Y , C Y (e)). (HP3) b, γ and g are α Y -Hölder continuous in time, locally in space,

.2. 4 )Assumption 4 . 2 . 2 (

 4422 (HP4) ∇ y b, ∇ y γ and ∇ y g satisfy (HP3) with (C Y,∇ , C Y,∇ (e)) in place of (C Y , C Y (e)). For X). (HP1) and (HP3) are satisfied for µ, σ and h (instead of b, γ and g) with coefficient C X , C X (e) and α X (instead of C Y , C Y (e) and α Y ).

Figure 4 . 1 :

 41 Figure 4.1: Simulation of utility at each time τ .

Figure 4 . 2 :

 42 Figure 4.2: Monotonocity and Concavity of approximated utilities U(t, z)

Figure 4 . 3 :

 43 Figure 4.3: Dynamic of approximated utilities U(t, z)

  .

  4) in order to have the maximum of points between the two decision bounds (4.3.8)-(4.3.9) (which is the black points on figure 4.4. The green points on the decision bounds are called the support vectors which play a main role in this problem).

Remarks 4 .

 4 The optimization problem (4.3.10)-(4.3.11) can be solved in a numerical way very efficiently by the SVR model described above.

Figure 4 . 6 :

 46 Figure 4.6: Evolution of SVR model's utility learning error at t i = t N fixed for 1000 iteration

Figure 4 . 7 :

 47 Figure 4.7: Prediction Utility with SVR model for fixed value of τ i .

Figure 4 . 9 :Figure 4 . 10 :

 49410 Figure 4.9: Evolution of the NuSVR error at t i = T fixed for n = 1000 iteration

Figure 4 .

 4 Figure 4.11: Utility's map: (τ, z) → U p (τ, z)

Figure 4 .

 4 Figure 4.13: Dynamics : t → U p (τ, z) for different values of t for n = 150.

4. 3 . 3

 33 Learning utility with Neural Network Neural Network ApproachIn this paragraph, we try to learn the investor's utility, using the Neural Network model from the SKlearn library of Python. The Neural Network is the most used technique in Machine learning.It is an artificial Neural Network with multiple layers between the input and output. There are different types of Neural Networks and they always consist of the same components: neurons, synapses, weights, biases, and functions. These components functioning similar to the human brains and can be trained like any other Machine Learning algorithm. In general, Neural Network consists of three types of layers: the first one is called the input layer, the last one is called the output layer and the remaining one are called hidden layers.For the computation implementation, we use the Multi-layer Perceptron classifier from the SKlearn library in Python. Multi-layer Perceptron (MLP) is a supervised learning algorithm that learns a function U by training on a dataset. Given a set of features X and a target y, it can learn a non-linear function approximator for either classification or regression. It is different from logistic regression, in that between the input and the output layer, there can be one or more non-linear layers, called hidden layers. MLPRegressor model implements a multi-layer perceptron (MLP) that trains using back-propagation with no activation function in the output layer, which can also be seen as using the identity function as activation function.

kFigure 4 . 15 :

 415 Figure 4.15: Graph illustrating the transition from the input layer to the first hidden layer.

5 . 3 Z 3 3 Z 3

 53333 Output layer : It contains the predict value U p ∈ IR n 4 given by, for a 4,i ∈ IR n 4 ,n 3 andb 4,i ∈ IR n 4 , U p k = σ( j=1,...,n ,i j a 4,i k,j + b 4,i k ) = j=1,...,n ,i j a 4,i k,j + b 4,i k , ∀k ∈ {1, ..., n 4 },with for output layer σ(x) = identity(x). This corresponds to

Figure 4 . 16 :

 416 Figure 4.16: Neural Network graph of a 5-layer perceptron with n input units and m output units and 3 hidden layers where n1 = 5, n2 = 3 and n3 = 5.

∂ 2 J(

 2 -(a z i + b)) 2 .in the following way 1. Calculate the gradient and the Hessian of the Cost Function,∂J(θ) ∂θ i and H(J) = ∂ 2 J(θ) ∂θ i ∂θ j i,j , with θ = (a, b) -(a z i + b))z i , U i -(a z i + b)), ∂ 2 J(a, b) ∂ 2 b = 1,

3 .

 3 Start the forward propagation again, i.e. we repeat the Neural Network algorithm with the new parameters θ and find a new prediction U p . 4. If we get |U p -U| ≤ ϵ, for a given ϵ, (4.3.18) we keep the output U p . Else we repeat steps 1, 2 and 3 until verifying the condition (4.3.18).

  ), we see that Neural Network model is better than the others (SVR and NuSVR models), it succeeded in predicting for values of wealth z ∈ [0, 0.5]. In fact, for the neural network model, we start with random values of weight a and bias b and work backwards, several times, to improve the output. The difference with the other methods tested in this work, that for the Neural Network model we have local predictions for each layer and each forward passage in the loop. Which allows to decrease the error each iteration and give a better prediction or fitting compared to SVR or NuSVR models.

Figure 4 . 17 :Figure 4 .Figure 4 . 19 :

 4174419 Figure 4.17: Predict utility (t, z) → U p (t, z)

  SVR and NuSVR models ( see Figure 4.20).

Figure 4 . 20 :

 420 Figure 4.20: Dynamic of Utility for fixed value of z.

Figure 4 . 21 :

 421 Figure 4.21: Dynamic of Utility for fixed value of z with missing data.

Figure 4 . 25 :

 425 Figure 4.25: Prediction of dynamic utility with missing data.

Figure 4 Figure 4 .

 44 Figure 4.26: Dataset with 60 samples, Implementation time 0.145 seconds

Figure 4 . 29 For

 429 Figure 4.29

Figure 4 . 30 Figure 4 . 31

 430431 Figure 4.30 Figure 4.31

Figure 4 . 32 : 4 :

 4324 Figure 4.32: Comparison of 3 methods for a prediction of a trajectory for τ i fixed.

Figure 4 . 33 : 5 :

 4335 Figure 4.33: Comparison of 3 methods for a prediction of a trajectory at fixed z j .

Figure 4 . 34 :

 434 Figure 4.34: Neural Networks prediction vs. target utility.

Figure 4 . 35 :

 435 Figure 4.35: Predict utility for τ i ∈ [0.4, 0.8] Figure 4.36: Predict utility for τ i ∈ [0.2, 0.6]

Titre:

  Contrôle stochastique et applications : types Champ moyen et utilités dynamiques Mots clés : Problème de contrôle de type mean field, énergies épuisables, jeux à champ moyen, utilités dynamiques, machine Learning Résumé : Cette thèse est consacrée à l'étude des contrôles stochastiques et leurs applications. Dans le premier chapitre, on a étudié un problème de contrôle linéaire quadratique type mean field dans le cadre avec saut. On a utilisé une approche basée sur une formulation faible du principe d'optimalité des martingales standard dans la théorie du contrôle stochastique afin de prouver l'existence d'un contrôle optimal. Nous avons étendu le résultat trouver pour un modèle du production d'énergies épuisables avec extraction. Dans le deuxième chapitre, on s'est intéressé à la résolution numérique d'un problème des jeux à champ moyen (Mean field Game). La caractérisation des stratégies optimales des agents en interaction est obtenue vie l'étude d'un système d'EDP non linéaires couplées permettant la détermination de la caractéristique de l'agent ainsi que la distribution de ces caractéristiques dans la population. Nous avons présenté un schéma numérique pour déterminer le couple (u,m) associé à la fonction de valeur et à la densité de probabilité, en se basant sur un argument de point fixe du système couplé de l'équation de Hamilton Jacobi Bellman et l'équation Fokker Planck. Nous avons appliqué notre approche pour un exemple linéaire quadratique avec une solution explicite afin de comparer les deux solutions numérique et explicite. Dans le troisième chapitre, on introduit un algorithme pour « prédire » l'utilité dynamique d'un agent en observant ses décisions à des dates discrètes (éventuellement aléatoires). C'est le principe des Robo-advisors qui sont des services de gestion d'investissement en ligne qui utilisent des algorithmes mathématiques pour fournir des services financiers avec un minimum d'intervention humaine. L'idée de ce travail est l'apprentissage de l'utilité dynamique d'un investisseur sur un marché financier avec défauts en utilisant des différentes méthodes de Machine Learning. Title : Stochastic control and applications : Mean field types and dynamic utilities Keywords : Mean field control problem, exhaustible energy production, mean field games, Markov chain approximation, dynamic utilities, machine learning

  Theorem 1.1.6 introduced in[START_REF] Miller | Linear-quadratic mckean-vlasov stochastic differential games[END_REF], we get an explicit form of the optimal control:

	Theorem 1.1.7. Under Assumptions (1.1.4)-(1.1.5)-(1.1.6), there exists an optimal control α *
	for LQMF problem (2.3.14) giving by the following explicit form

Theorem 1.2.4. Under some assumptions and if F and Φ satisfy the Monotonicity condition.

	.2.39)
	with (P h (k, α * k )) t is the transposed matrix of (P h (k, α * k )). Then, we will show that it
	converges weakly to the solution of the Fokker Planck equation.
	The first main result of this part is Theorem 1.2.4 given below, where we prove an existence and
	uniqueness results of the solution of our discrete version of mean field system (1.2.37)-(1.2.39):

  , γ i are deterministic functions from IR + × IR d into IR d and h, g are deterministic functions from IR + × IR d × IR q ′ into IR d×q ′ , globally Lipschitz in space to ensure the existence

	where µ, b, σ i of a unique strong solution under the following assumptions :
	Assumption 1.3.1.		• (HP1) There exist C Y and C Y (e) s.t. for any t ∈ [0, T ], x, y ∈ IR d and
	e ∈ IR q ′
				.3.40)
		q
	dY t (y) = b(t, Y t (y))dt +	i=1	γ i (t, Y t (y))dB i t +	IR q ′ g(t, Y t -(y), e)π(dt, de), Y 0 (y) = y, (1.3.41)

  The optimization problem (4.3.10)-(4.3.11) can be solved in a numerical way very efficiently by the SVR model described above.

3.54-1.3.55. Remarks 2.

  under the same constraint. called hidden layers. we use the Multi-layer Perceptron model of Neural Network class of SKlearn library. Multi-layer Perceptron (MLP) is a supervised learning algorithm that learns a function U i by training on a dataset. Given a set of features Z in and a target U i , it can learn a non-linear function approximator for either classification or regression. It is different from logistic regression, in that between the input and the output layer, there can be one or more non-linear layers, called hidden layers.

• Neural Network model : A neural network is an artificial neural network with multiple layers between the input and output layers. In general, it is made up of three types of layers: the first layer is called the input layer, the last layer is called the output layer and possi-bly other layers

  3.29) and the solution to the LQMF problem(2.3.14).

	Theorem 2.3.3. Under Assumptions (H1)-(H2), there exists an optimal control α * for LQMF
	problem (2.3.14) giving by the following explicit form

  It yields that Assumption (ii) of the verification in Lemma 2.3.2 is satisfied. Moreover, one could prove that E[D α

3.2 the existence of (K, Λ, Y, Z Y , R Y , φ) solution to the system

(2.3.29)

. We consider the candidate {w t (x, x)} given by

(2.3.16)

.As K, Λ, R are bounded and Y satisfies a square-integrability condition, then the growth condition of the random field {w t (x, x)} i.e condition (2.3.15)) in the verification theorem (see Lemma 2.3.2) is satisfied. Thanks to the choice of (K, Λ, Y, Z Y , R Y , φ), the terminal conditions satisfied {w T (x, x)} = g(x, x), and so Assumption (i) of Lemma 2.3.2 is satisfied. From equation (2.3.28), we deduce that E[D α t ] is non-negative for all α ∈ A. * t ] = 0 for some α = α * if and only if

  .4.44) and the coefficients f , g defined in (2.2.4)-(2.2.5), satisfy the following assumptions: (H4) (i) Q, Q, I, Ĩ, N, Ñ are essentially bounded IF 0 -adapted processes, (ii) P, P are essentially bounded F 0 T -measurable random variables, (iii) M, H are square-integrable IF -adapted processes, and L is a square-integrable F T -measurable random variables.

  .4.53). Given (K, Λ), we have also the existence of a unique solution (Y, Z Y , Z Y,0 , R Y ) of the mean field backward stochastic differential equation in(2.4.53). Then from Lemma 2.3.1, we have the following proposition which gives the structure of the optimal control α * .

	Proposition 2.4.1. Under Assumptions (H3)-(H4), the optimal control α * for optimal problem
	(2.4.43)-(2.4.44) is defined as follows

.4.54) One could prove that (K, Λ) is the unique solution to the Stochastic Backward Riccati Equation in (2

  So we need Brouwer's fixed point theorem to prove that the function Γ h is continuous function on P(G h,dt ) and in consequence prove the existence of an equilibrium of the discrete MFG control problem (3.3.19)-(3.3.26).

.4.29) Proposition 3.4.1. Under assumptions (3.2.1)-(3.2.3), there exists at least one solution of the system (3.3.19)-(3.3.26).

Proof. As we have that the set P(G h,dt ) being convex compact, indeed, according to Billengsley (

[START_REF] Billingsley | Convergence of probability measures[END_REF]

, Theorem 1.3), the space P(G h,dt ) is a compact set since G h,dt is also compact set.

  Proposition 3.4.2. Under assumptions (3.2.2)-(3.2.4) and if F h and Φ h satisfy the Monotony condition, (i.e.),

4.3.1 Machine Learning programs using Python

	Most of the time, it is not necessary to develop its own algorithm, thanks to PYTHON, they
	are already developed in open source libraries. For example, Scikit-learn library contains all
	the mathematical functions which are necessary for learning problems. Scikit-learn is an open
	source machine learning library that supports supervised and unsupervised learning. It also
	provides various tools for model fitting, data preprocessing, model selection and evaluation and
	many other utilities.
	With Scikit-learn, it is enough to write a few lines to develop models of linear regression and
	classification:

• Create the Dataset. (Z in , U i ) such that -Z in are the input data. It contains different initial values of wealth (z j ) j∈ 1,n , (i.e)

(Z in j ) j∈ 1,n = (z j ) j∈ 1,n .

-U i are the target. It contains a vector of the utility matrix (U(τ i , z j )) (i,j)∈ 1,N × 1,n approximated in Section 4.2.2. In other words, U i is utility values at a fixed time τ i and for different values of (z) j∈ 1,n :

  (SVM in short) are supervised learning models with associated learning algorithms that analyze data for classification and regression analysis, developed by Vladimir Vapnik et Alexey Chervonenkis. Support Vector Regression (SVR) uses the same principle as SVM, but for regression problems. The problem of regression is to find a function that approximates mapping from an input domain to real numbers on the basis of a training sample. The objective, when we are moving on with SVR, is to basically consider the points that are within the decision boundary line. The best fit line is the hyperplane of equationa j Z in j + b = 0, ∀j ∈ 1, n that has a maximum number of points between the two decision boundary lines. Thus, to summarize, we are looking for a hyperplane such that most of train observation are inside the marge ϵ satisfying the following constraints

	U i j -(a j Z in j + b) ≤ ϵ, ∀j ∈ 1, n , if the point is above the margin,	(4.3.8)
	and
	(a j Z in j + b) -U i j ≤ ϵ, ∀j ∈ 1, n , if the point is below the margin,	(4.3.9)
	where
	1. Z in = (Z in

j ) j∈ 1,n represents the input defined by different initial values of the wealth.

  + b 1,i k ), ∀k ∈ {1, ..., n 1 },

	1,i k = σ(	Z in j a 1,i k,j
	j=1,...,n 0

Table 4

 4 .3, we have a value very close to zero for the rate R which is a good indication, that MSE is very low compared to the mean value of the data. Even for making the test with hidden/unseen data the Neural Network model managed to predict better, see Figure 4.21.

	R 2	MSE	MSA	Rtest
	0.9	0.80	0.56	0.06

Table 4 .

 4 3: Performance of Neural Network modelThe Neural Network model provided good prediction capabilities with respect to the regression problem of SVR models. In fact, with increasing input size the svm model is having lineary increasing number of parameters, on the other hand, Neural network model is not having. Even if we are mainly interested in three hidden layer, Neural Network can have as more than three

Table 4 .

 4 6: A sample of the few predicted values via Target value on Test set for different models.

	Neural Network

j test U p NuSVR U p SVR U p

Remerciements

one, we need the results of Kushner and Dupuis [START_REF] Kushner | Numerical methods for stochastic control problems in continuous time[END_REF] for weak convergence of time-continuous controlled Markov chains on the space of Càdlàg processes D[0; T ] which is a separable and complete for the Skorohod topology. We will use the same arguments used by Ben Aziza et al.

in [START_REF] Ben Aziza | A probabilistic numerical method for a class of mean field games[END_REF].

Remark 3.5.1. From the results of Billingsley ( [19], Theorem 6.8 page 73), since the space of Càdlàg processes D[0; T ], then the space of probability measures for Càdlàg processes P(D[0; T ]) is also separable and complete. Moreover, the relative compactness of any family of probability measures is equivalent to the tightness ( [START_REF] Kushner | Numerical methods for stochastic control problems in continuous time[END_REF], Theorem 9.1.1)

Before announcing the main results of this work, we must introduce some mathematical notions and tools that will be useful to prove this convergence.

To force the state process X to stay in the domain, we need to add the following Neumann boundary condition

and

where r(x) is the reflection direction at x and ∂K is the boundary set of K.

In the following we propose a concise description of the limit Mean Field Game related to the system (3.2.8) with boundary conditions (3.5.33)- (3.5.34).

Let (Ω, F, P, IF = {F t }) be a filtered probability space, and let us consider a population of players where a representative player is described by a state variable X α,ϵ ∈ K, we recall that K is a compact set. dX α t = b(t, X α t , α t )dt + σdB t X α 0 ∼ µ 0 , (3.5.35) where B is a standard Brownian motion, α an admissible control process .

In this case, we must deal with the convergence of the controls, in addition to the convergence properties of the processes. Therefore, we need to define a new class of control, the relaxed control which provide a very powerful tool in the study of the convergence properties of sequences of optimally controlled processes. This is due to the fact that under general conditions, arbitrary sequences of relaxed controls have compact closure. This is not true of ordinary controls. The relaxed control is a class of control were first introduced by Yong to establish existence of a minimizing control for problem for the calculus of variations. This class were later used especially to prove the convergence properties of the approximation numerical problem.

(See [START_REF] Kushner | Numerical methods for stochastic control problems in continuous time[END_REF] chapter 9 for more details ).

where

with z is a reflecting process defined in (3.5.41)- (3.5.42).

The last step is to show that m h converges to the same limit as m h , using the following result of Bayraktar et al [START_REF] Bayraktar | A numerical scheme for a mean field game in some queueing systems based on markov chain approximation method[END_REF] : Lemma 3.5.15. There exist a T > 0, h 0 > 0 and q ∈ (0, 1), such that for every T ≤ T ,

Lemma 3.5.16. m h converges weakly to m, the weak limit of m h .

Proof.

To prove this convergence, we will use the Wassertein distance W 1 .

Using the following inequality

From Lemma 3.5.15, we have that W 2 1 (m h , m) -→ h→0 0. So we just need to prove that

We remind that mh and m h are respectively the law of the optimal controlled Markov chain

Moreover, m h and m h present the induced measures associated to m h and m respectively.

Consequently, we get that m h and m h converge weakly to the same limit m.

Concluding, the previous results collected together lead to the convergence result that we look to prove in this work. In one hand we have that the tow families of probability measures { mh , h > 0} and {m h , h > 0} converge weakly to the same limit law m. On the other hand, the limit law m is a equilibrium of the MFG continuous problem (3.2.5).

Explicit solution

In this section, we have to solve the system of HJB-FP equations introduced by 3.6.48 which reads as follows

(3.6.49)

We look for a solution u(t, x) of the following quadratic form

x * P (t)x + x * r(t) + s(t), (3.6.50) so we get

Then the FP equation be written as

We set z(t) = xm(t, x)dx the mean of m. It easy to see that z(t) satisfies the following equation

(3.6.52)

Using the quadratic form of u, we get the coupled system of differential equations:

) end while.

Here, we compare both explicit and numerical solutions for the same initial probability density m 0 = N(0, 0.5) for A = 0.01.

Let us start by setting the parameters chosen for the numerical experiments: [START_REF] Achdou | Convergence of a finite difference scheme to weak solutions of the system of partial differential equations arising in mean field games[END_REF][START_REF] Achdou | Convergence of a finite difference scheme to weak solutions of the system of partial differential equations arising in mean field games[END_REF].

Test 1 :

From figures 3.1 and 3.2, we can see that the probability densities are more and more similar when evolving in time. The evolution of the probability distributions seems to be toward a centered Gaussian one. The probability measure: FP solution lose the convergence of the explicit solution to the numerical solution.

Test 2 : For A = 0.5, we get the following numerical result: Test 3 : For A = 1, we get the following numerical result:

• If t k+1 = τ l for some l ∈ 1, J , then we introduce a correction due to jumps discontinuities the authors discussed these different approaches in detail (see [START_REF] Gobet | Convergence rate of strong approximations of compound random maps, application to spdes[END_REF]Section 4]) and they have exposed the main difficulties of each method. The simplest, most efficient and less expensive is the backward method, it allow us to approximate the inverse flow X using the Euler's scheme.

Inverse flow as a backward in time SDE

Without going into detail, this is the method we will adopt here.

Let (X s,t (x)) t≥s be the solution, starting from x at time s, of the SDE with coefficients (µ, σ, h).

Denote by X s,t the inverse map of X s,t which can also be defined by X s,t (x) := X t (X s (x))

where X t (x) ̸ = X 0,t (x), focuses on the dynamics of X s,t (x) in the variable s: doing so, we aim at computing the inverse of Y backward in time instead of forward in time. This approach relies on the following key result.

The notations ds and ← -B s are to remind that the integrals must be considered in a backward way 1 . With this result in hand, the approximation of X s,t is made possible simply using a standard Euler scheme like for Y n . Using similar notations as below, Scheme 4.2.2. Based on this key result, the approximation of the inverse flow X is achieved by the following steps.

It is well known that the Euler scheme, whether forward (as for Y N ) or backward (as for X N ), converges, under regularity assumptions (like Lipschitz-type), to the solution of the stochastic differential equation at the rate 1

. However, few results give us the rate of convergence of the compound of two approximation schemes. Thanks to the results of the paper [START_REF] Mrad | Solving some stochastic partial differential equations driven by lévy noise using two sdes[END_REF], we know that our compound process Y N 0,t (u z (X N 0,t (z))) t≥0 converges to the marginal utility U z (t, z) at the rate Then, for any concave function u with Lipschitz marginal utility u z , the compound Euler scheme

now there was no idea how to quantify the preferences of an agent from these choices. The goal then, starting from the values obtained (U (t i , z j )) i,j , is to estimate U (t, z) for all (t, z) and to predict its future decisions. For this we will use different machine learning algorithms and we will compare them.

The aim in the following is to propose an algorithm for learning the preferences of the agent.

As we do not have access to real data, we will simulate our own data X τ i (z j ) and Y τ i for (i, j) ∈ 1, J × 1, n . Once these values have been simulated we proceed to approximate the utility U at the points (t i , z j ) (using the above procedure). The generated values U (τ i , z j ) will play the role of starting data for our learning algorithms. This kind of algorithm could clearly be used on robo-advisor platforms, since it can "predict" (over a short period of time) the preferences of an agent and therefore can make decisions for him/her.

Machine Learning methods

Machine Learning algorithms automatically learn to perform a task or make predictions from data and improve their performance over time. Once trained, the algorithm will be able to find patterns in new data. Depending on the information available during the learning phase, learning is qualified in different ways. If the data are labeled (i.e., the response to the task is known for these data), it is supervised learning. It is called classification if the labels are discrete, or regression if they are continuous. If the model is learned in an iterative way according to a reward received by the program for each action taken, it is called reinforcement learning. In the most general case, without labels, we try to determine the underlying structure of the data (which can be a probability density) and it is then called unsupervised learning. Machine learning can be applied to different types of data, such as graphs, trees, curves or more simply feature vectors, which can be continuous or discrete qualitative or quantitative variables.

In this work, we are interested in supervised learning. Our task is to predict a target value which is the value of utility at fixed time τ i , denoted by (U i j ) j∈ 1,n a vector of the matrix (U(τ i , z j )) (i,j)∈ 1,N × 1,n from different initial wealth values (z j ) 1,n and fixed value τ i :

such that (τ i ) i∈ 1,N are the dates we built in the first simulation step.

From this dataset, we create a model, which is nothing else than a mathematical function. Once we have created it, we develop and train our model with new data (Z i j ) = (z j ) j∈ 1,n in order to predict the target or the answer (U i j ) j .

margin (the red points in figure 4.4), so the difference with the classical regression is that Support Vector Regression problem also takes into consideration the points which are outside the margin or the misplaced points (the red points in figure 4.4). Therefore, the constraint can be relaxed by allowing that a point may not be of the same label as the majority of the points as long as it is not too far from the boundary. Any value that lies outside the margin ξ, we can indicate its deviation from the margin as ϵ. In practice, we often cannot have U i j -(a j z j + b) > ϵ, for all j ∈ 1, n , we relax by introducing a positive variables ξ such that

The variable (ξ j , ξ * j ) j represent the error of the observations outside the hyperplane,

where l is the number of the points outside the margin and we remain that In this work, we implement the algorithm with the two models SVR and NuSVR.

Numerical results of SVR prediction

First, we consider a SVR model, and we take a vector of approximated Utility (U(t, z)) at fixed time t (simulated in previous Section 4.2.2 ) as a target value U i . We wish to predict this target U i from the input parameters z.

More precisely, we start by defining our data (Z in , U i ). We take as input Z in := (Z in j ) j = (z j ) j

Then, with a R 2 = 0.52, we can deduce that SVR model is not the best model to predict the dynamic of utility. It is clear form Figure 4.8 that the prediction misses a lot of points which justifies that the value of R 2 is not high. This ratio must be as close as possible to zero, in other words, the lower the MSE value is compared to the average value, the better it is. It is better that this ratio is lower than 0.4 ( 0.5 is much higher ). Unfortunately this is not the case here. This leads us to conclude that SVR model is not the best fit for our data.

Numerical results of NuSVR prediction

In this paragraph, we we will test the NuSVR model of SVM package with a regularization parameter C = 100.

As before, Z in := (Z in j ) j = (z j ) j is an input data and U i := (U(τ i , z j )) j∈ 1,n is the target vector of values. U i is the i-th column vector of the utility matrix (U(τ i , z j )) (i,j)∈ 1,J × 1,n .

In this model, the parameter ν allows us to control controls the number of support vectors and allows to compute automatically ϵ.

As on SVR model, we take Gaussian RBF as kernel k and for ν ∈ (0, 1], we want to solve the following optimization problem for fixed time t:

where l is the number of the points outside the margin where ξ j = U i j -(a j ϕ(z j ) + b -ϵ) and ξ * j = (a j ϕ(z j ) + b + ϵ) -U i j , under the following constraint

where we remain that ξ represents the errors of the points outside the hyperplane and ϕ is a transformation maps such that k(x, y) = ϕ(x) T ϕ(y).

Remarks 5.

• If ϵ increases, the quantity 1 l l i=1 (ξ i + ξ * i ) decreases (as less samples outside the margin), the function smoothness 1 2 ||a|| increases and the accuracy decreases.

• If ϵ decreases, the term νϵ decreases, but the quantity

samples outside the margin), the function 1 2 ||a|| is less smoothed and the the accuracy increases.

As for the SVR test model, we visualize the evolution of the error rate, we execute our algorithm in a for loop several times taking the Mean Squared Loss defined by (4.3.14).

After implementing many times, we calculated the error rate each iteration (see figure 4.9), comparing with the error rates for SVR model ( see figure 4.6), we can conclude that we succeeded in reducing the error by adding the ν parameter.

From Figure 4.11, we can see that predict values of utilities for different points of t are concave with respect to z which satisfies the dynamic utility properties. The NuSVR learned better from our data, we find the concavity and Monotonicity of the utility even by making few iterations. It illustrates that by adding the parameter ν, we reduce the learning error (see table 4.2).

Concerning the learning of the utility dynamics with the Neural Network model, we take a data (T j , U j ) in the following way: as input T j := (t i ) i∈ 1,N for different values of times between two update τ i and τ i+1 and as target

We remain that (U(τ i , z j )) i is a line vector with utility value for a fixed value of wealth z and different values of (τ i ) i∈ 1,N . We can see from Figure 4.14, that in the prediction there are a lot of missing values, especially for the part [0, 4, 0, 8], the prediction of the dynamics of utility with NuSVR model is not the best model to make the predictions but still it behaves better than SVR model in terms of R 2 coefficient. We get also more important MSE and MAE errors than the SVR model, see In the next section, we will try these learning tests with the Neural Network model, we will see

Learning utility with missing data

In this part, we want to show the impact of data lack where the agent is not telling all the data to the robo-advisor. We propose a simpler example of the model (4.2.1)-(4.2.2). We consider a pure jump process X whose size depends on its value at the previous time, defined by

where N is a random variable representing the jump's number assumed to follow a uniform law on [0, T ] and V = (V i ) 1≤i≤N , a sequence of random variables, represents the jump sizes and (τ n ) n a sequence of jump times.

Learning with SVR model

In this section, we take the SVR model. We consider the vector of targeted values U i as a fixed value of the dynamic utility U(τ, z) at fixed time τ i Then, for the following test, we hid the part of the data for z ∈ [START_REF] Arthur | Complexity and the economy[END_REF][START_REF] Buckdahn | A general stochastic maximum principle for sdes of mean-field type[END_REF] and z ∈ [START_REF] Carlini | A semi-lagrangian scheme for a degenerate second order mean field game system[END_REF][START_REF] Gèron | Hands-On Machine Learning with Scikit-Learn and TensorFlow: Concepts, Tools, and Techniques to Build Intelligent Systems[END_REF] in our input data and the target value. We obtain the following prediction with R 2 = 0.89. In the previous part, we try to learn the dynamic of utility using the SVR model. We define our data (T j , U j ) in following way: as input T j := (t i ) i in different values of times between two update τ i and τ i+1 and as target U j = (U j i ) i the utility value for a fixed value of wealth z j and different values of t : (U j i ) i∈ 1,N := (U(τ i , z j )) i∈ 1,N .

Learning with NuSVR model

We use the same data (Z in , U i ) to solve the optimization problem (4. We also test the prediction for dynamic of utility with missing data. According to figure 4.25, we can see that the machine doesn't predict the hidden data. It is clear that the machine misses several values and it is not precise enough.

Basically, to know which is the best model to use, we have to test all the models and then to choose the one with the best performance, the one, with, a higher determination score R 2 . The data are always very different, we can never define a precise rule to fix the model that works at a hundred percent, that's why we are based on the performance of our models.

In our future work, we will adapt the Neural Network model, the best performing among the tested models, as a learning model in a generalized market model framework. We also wish to generalize our learning algorithms to more advanced models, such as the Recurrent Neural

Network and/or Reinforcement Learning.