René Carmona

François Delarue

The Master Equation for Large Population Equilibriums

We use a simple N -player stochastic game with idiosyncratic and common noises to introduce the concept of Master Equation originally proposed by Lions in his lectures at the Collège de France. Controlling the limit N Ñ 8 of the explicit solution of the N -player game, we highlight the stochastic nature of the limit distributions of the states of the players due to the fact that the random environment does not average out in the limit, and we recast the Mean Field Game (MFG) paradigm in a set of coupled Stochastic Partial Differential Equations (SPDEs). The first one is a forward stochastic Kolmogorov equation giving the evolution of the conditional distributions of the states of the players given the common noise. The second is a form of stochastic Hamilton Jacobi Bellman (HJB) equation providing the solution of the optimization problem when the flow of conditional distributions is given. Being highly coupled, the system reads as an infinite dimensional Forward Backward Stochastic Differential Equation (FBSDE). Uniqueness of a solution and its Markov property lead to the representation of the solution of the backward equation (i.e. the value function of the stochastic HJB equation) as a deterministic function of the solution of the forward Kolmogorov equation, function which is usually called the decoupling field of the FBSDE. The (infinite dimensional) PDE satisfied by this decoupling field is identified with the master equation. We also show that this equation can be derived for other large populations equilibriums like those given by the optimal control of McKean-Vlasov stochastic differential equations.

The paper is written more in the style of a review than a technical paper, and we spend more time and energy motivating and explaining the probabilistic interpretation of the Master Equation, than identifying the most general set of assumptions under which our claims are true.

Introduction

In several lectures given at the Collège de France, P.L. Lions describes mean-field games by a single equation referred to as the master equation. Roughly speaking, this equation encapsulates all the ˚Paper presented at the conference "Stochastic Analysis", University of Oxford, September 23, 2013 : rcarmona@princeton.edu, Partially supported by NSF: DMS-0806591 ; delarue@unice.fr information about the Mean Field Game (MFG) into a single equation. The purpose of this paper is to review its theoretical underpinnings and to derive it for general MFGs with common noise.

The master equation is a Partial Differential Equation (PDE) in time, the state controlled by the players (typically an element of a Euclidean space, say R d ), and the probability distribution of this state. While the usual differential calculus is used in the time domain r0, T s and in the state space R d , the space PpR d q of probability measures needs to be endowed with a special differential calculus described in Lions' lectures, and explained in the notes Cardaliaguet wrote from these lectures, [START_REF] Cardaliaguet | Notes on mean field games[END_REF]. See also [START_REF] Carmona | Forward-Backward Stochastic Differential Equations and Controlled McKean Vlasov Dynamics[END_REF] and the appendix at the end of the paper.

Our goal is to emphasize the probabilistic nature of the master equation, as the associated characteristics are (possibly random) paths with values in the space R d ˆPpR d q. Our approach is especially enlightening for mean field games in a random environment (see Section 2 for definitions and examples), the simplest instances occurring in the presence of random shocks common to all the players. In that framework, the characteristics are given by the dynamics of ppX t , LpX t |W 0 qqq 0ďtďT , where pX t q 0ďtďT is the equilibrium trajectory of the game, as identified by the solution of the mean field game problem, and pLpX t |W 0 qq 0ďtďT which denotes its conditional marginal distributions given the value of the common noise, describes the conditional distribution of the population at equilibrium. Examples of mean field games with a common noise were considered in [START_REF] Guéant | Mean field games and applications[END_REF], [START_REF] Gomes | Saude Mean field games models -a brief survey[END_REF] and [START_REF] Carmona | Mean Field Games and Systemic Risk[END_REF]. Their theory is developed in the forthcoming paper [START_REF] Carmona | Mean field games with a common noise[END_REF] in a rather general setting.

As in the analysis of standard MFG models, the main challenge is the solution of a coupled system of a forward and a backward PDEs. However, in the random environment case, both equations are in fact stochastic PDEs (SPDEs). The forward SPDE is a Kolmogorov equation describing the dynamics of the conditional laws of the state given the common noise, and the backward SPDE is a stochastic Hamilton-Jacobi-Bellman equation describing the dynamics of the value function. Our contention is that this couple of SPDEs should be viewed as a Forward Backward Stochastic Differential Equation (FBSDE) in infinite dimension. For with this point of view, if some form of Markov property holds, it is natural to expect that the backward component can be written as a function of the forward component, this function being called the decoupling field. In finite dimension, a simple application of Itô's formula shows that when the decoupling field is smooth, it must satisfy a PDE. We use an infinite dimensional version of this argument to derive the master equation. The infinite dimension version of Itô's formula needed for the differential calculus chosen for the space of measures is taken from another forthcoming paper [START_REF] Chassagneux | McKean-Vlasov FBSDEs and related Master Equation[END_REF] and is adapted to the case of a random environment in the appendix.

While the MFG approach does not ask for the solution of stochastic equations of the McKean-Vlasov type at first, the required fixed point argument identifies the equilibrium trajectory of the game as the de facto solution of such an equation. This suggests that the tools developed for solving MFG problems could be reused for solving optimal control problems of McKean-Vlasov dynamics. In the previous paper [START_REF] Carmona | Forward-Backward Stochastic Differential Equations and Controlled McKean Vlasov Dynamics[END_REF], we established a suitable version of the stochastic Pontryagin principle for the control of McKean-Vlasov SDEs and highlighted the differences with the version of the stochastic Pontryagin principle used to tackle MFG models. Here we show in a similar way that our derivation of the master equation can be used as well for this other type of large population equilibrium problem.

The paper is organized as follows. Mean field games in a random environment are presented in Section 2. The problem is formulated in terms of a stochastic forward-backward system in infinite dimension. A specific example, taken from [START_REF] Carmona | Mean Field Games and Systemic Risk[END_REF], is exposed in Section 3. The master equation is derived explicitly. In Section 4, we propose a more systematic approach approach of the master equation for large population control problems in a random environment. We consider both the MFG problem and the control of McKean-Vlasov dynamics. Another example, taken from [START_REF] Guéant | Mean field games and applications[END_REF], is revisited in Section 5. We end up with the proof of the chain rule along flow of random measures in the Appendix.

When analyzed within the probabilistic framework of the stochastic maximum principle, MFGs with a common noise lead to the analysis of stochastic differential equations conditioned on the knowledge of some of the driving Brownian motions. These forms of conditioned forward stochastic dynamics are best understood in the framework of Terry Lyons' theory of rough paths. Indeed integrals and differentials with respect to the conditioned paths can be interpreted in the sense of rough paths while the meaning of the others can remain in the classical Itô calculus framework. We thought this final remark was appropriate given the raison d'être of the present volume, and our strong desire to convey our deepest appreciation to the man, and pay homage to the mathematician as a remarkably creative scientist.

Mean Field Games in a Random Environment

The basic purpose of mean-field game theory is to analyze asymptotic Nash equilibriums for large populations of individuals with mean-field interactions. This goes back to the earlier and simultaneous and independent works of Lasry and Lions in [START_REF] Lasry | Jeux à champ moyen I. Le cas stationnaire[END_REF][START_REF] Lasry | Jeux à champ moyen II. Horizon fini et contrôle optimal[END_REF][START_REF] Lasry | Mean field games[END_REF] and Caines, Huang and Malhamé in [START_REF] Huang | Large population stochastic dynamic games: closed-loop McKean-Vlasov systems and the Nash certainty equivalence principle[END_REF].

Throughout the paper, we shall consider the problem when the individuals (also referred to as particles or players) are subject to two sources of noise: an idiosyncratic noise, independent from one particle to another, and a common one, accounting for the common environment in which the individuals evolve. We decide to model the environment by means of a zero-mean Gaussian white noise field W 0 " pW 0 pΛ, Bqq Λ,B , parameterized by the Borel subsets Λ of a Polish space Ξ and the Borel subsets B of r0, 8q, such that

E " W 0 pΛ, BqW 0 pΛ 1 , B 1 q ‰ " ν `Λ X Λ 1 ˘|B X B 1 |,
where we used the notation |B| for the Lebesgue measure of a Borel subset of r0, 8q. Here ν is a non-negative measure on Ξ, called the spatial intensity of W 0 . Often we shall use the notation W 0 t for W 0 p ¨, r0, tsq, and most often, we shall simply take Ξ " R ℓ . We now assume that the dynamics in R d , with d ě 1, of the private state of player i P t1, ¨¨¨, N u are given by stochastic differential equations (SDEs) of the form:

dX i t " b `t, X i t , µ N t , α i t ˘dt `σ`t , X i t , µ N t , α i t ˘dW i t `żΞ σ 0 `t, X i t , µ N t , α i t , ξ ˘W 0 pdξ, dtq, (1)
where W 1 , . . . , W N are N independent Brownian motions, independent of W 0 , all of them being defined on some filtered probability space pΩ, F " pF t q tě0 , Pq. For simplicity, we assume that W 0 , W 1 , . . . , W N are 1-dimensional (multidimensional analogs can be handled along the same lines). The term µ N t denotes the empirical distribution of the particles are time t:

µ N t " 1 N N ÿ i"1 δ X i t .
The processes ppα i t q tě0 q 1ďiďN are progressively-measurable processes, with values in an open subset A of some Euclidean space. They stand for control processes. The coefficients b, σ and σ 0 are defined accordingly on r0, T s ˆRd ˆPpR d q ˆApˆΞq with values in R d , in a measurable way, the set PpR d q denoting the space of probability measures on R d endowed with the topology of weak convergence.

The simplest example of random environment corresponds to a coefficient σ 0 independent of ξ. In this case, the random measure W 0 may as well be independent of the spatial component. In other words, we can assume that W 0 pdξ, dtq " W 0 pdtq " dW 0 t , for an extra Wiener process W 0 independent of the space location ξ and of the idiosyncratic noise terms pW i q 1ďiďN , representing an extra source of noise which is common to all the players.

If we think of W 0 pdξ, dtq as a random noise which is white in time (to provide the time derivative of a Brownian motion) and colored in space (the spectrum of the color being given by the Fourier transform of ν), then the motivating example we should keep in mind is a function σ 0 of the form σ 0 pt, x, µ, α, ξq " σ 0 pt, x, µ, αqδpx ´ξq (with Ξ " R d and where δ is a mollified version of the delta function which we treat as the actual point mass at 0 for the purpose of this informal discussion). In which case the integration with respect to the spatial part of the random measure

W 0 gives ż R d σ 0 pt, X i t , µ N t , α i t , ξqW 0 pdξ, dtq " σ 0 pt, X i t , µ N t qW 0 pX i t , dtq,
which says that, at time t, the private state of player i is subject to several sources of random shocks: its own idiosyncratic noise W i t , but also, an independent white noise shock picked up at the very location/value of his own private state.

Asymptotics of the Empirical Distribution µ N t

The rationale for the MFG approach to the search for approximate Nash equilibriums for large games is based on several limiting arguments, including the analysis of the asymptotic behavior as N Ñ 8 of the empirical distribution µ N t coupling the states dynamics of the individual players. By the symmetry of our model and de Finetti's law of large numbers, this limit should exist if we allow only exchangeable strategy profiles pα 1 t , ¨¨¨, α N t q. This will be the case if we restrict ourselves to distributed strategy profiles of the form α j t " αpt, X j t , µ N t q for some deterministic (smooth) function pt, x, µq Þ Ñ αpt, x, µq P A.

In order to understand this limit, we can use an argument from propagation of chaos theory, as exposed in the lecture notes by Sznitman [START_REF] Sznitman | Topics in propagation of chaos[END_REF]. Another (though equivalent) way consists in discussing the action of μN t on test functions for t P r0, T s, T denoting some time horizon. Fixing a smooth test function φ with compact support in r0, T s ˆRd and using Itô's formula, we compute:

dxφpt, ¨q, 1 N N ÿ j"1 δ X j t y " 1 N N ÿ j"1 dφpt, X j t q " 1 N N ÿ j"1
ˆBt φpt, X j t qdt `∇φpt, X j t q ¨dX j t `1 2 tracet∇ 2 φpt, X j t qdrX j , X j s t u "

1 N N ÿ j"1 B t φpt, X j t qdt `1 N N ÿ j"1 ∇φpt, X j t q ¨σ`t , X j t , µ N t , αpt, X j t , µ N t q ˘dW j t `1 N N ÿ j"1 ∇φpt, X j t q ¨b`t , X j t , µ N t , αpt, X j t , µ N t q ˘dt `1 N N ÿ j"1 ∇φpt, X j t q ¨żΞ σ 0 `t, X j t , µ N t , αpt, X j t , µ N t q, ξ ˘W 0 pdξ, dtq `1 2N N ÿ j"1 trace "ˆr σσ : s `t, X j t , µ N t , αpt, X j t , µ N t q żΞ rσ 0 σ 0: s `t, X j t , µ N t , αpt, X j t , µ N t q, ξ ˘νpdξq ˙∇2 φpt, X j t q * dt
Our goal is to take the limit as N Ñ 8 in this expression. Using the definition of the measures µ N t we can rewrite the above equality as:

xφpt, ¨q, µ N t y ´xφp0, ¨q, µ N 0 y " OpN ´1{2 q `ż t 0 @ B t φps, ¨q, µ N s yds `ż t 0 @ ∇φps, ¨q ¨b`s , ¨, µ N s , αps, ¨, µ N s q ˘, µ N s yds `1 2 ż t 0 B trace "ˆr σσ : s `s, ¨, µ N s , αps, ¨, µ N s q żΞ rσ 0 σ 0: s `s, ¨, µ N s , αps, ¨, µ N s q, ξ ˘νpdξq ˙∇2 φpt, ¨q* , µ N s F ds `ż t 0 @ ∇φps, ¨q ¨żΞ σ 0 `s, ¨, µ N s , αps, ¨, µ N s q, ξ ˘W 0 pdξ, dsq, µ N s D ,
which shows (formally) after integration by parts that, in the limit N Ñ 8,

µ t " lim N Ñ8
µ N t appears as a solution of the Stochastic Partial Differential Equation (SPDE)

dµ t " ´∇ ¨"b `t, ¨, µ t , αpt, ¨, µ t q ˘µt ‰ dt ´∇ ¨ˆż Ξ σ 0 `t, ¨, µ t , αpt, ¨, µ t q, ξ ˘W 0 pdξ, dtqµ t 1 2 trace " ∇ 2 ˆ"σσ : ‰`t , ¨, µ t , αpt, ¨, µ t q ˘`ż Ξ " σ 0 σ 0: ‰`t , ¨, µ t , αpt, ¨, µ t q, ξ ˘νpdξq ˙µt  dt. (2)
This SPDE reads as a stochastic Kolmogorov equation. It describes the flow of marginal distributions of the solution of a conditional McKean-Vlasov equation, namely:

dX t " b `t, X t , µ t , αpt, X t , µ t q ˘dt `σ`t , X t , µ t , αpt, X t , µ t q ˘dW t `żΞ σ 0 pt, X t , µ t , αpt, X t , µ t q, ξ ˘W 0 pdξ, dtq, (3)
subject to the constraint µ t " LpX t |F 0 t q, where F 0 " pF 0 t q tě0 is the filtration generated by the spatial white noise measure W 0 . Throughout the whole paper, the letter L refers to the law, so that LpX t |F 0 t q denotes the conditional law of X t given F 0 t . The connection between ( 2) and (3) can be checked by expanding pxφpt, ¨q, µ t y " EpφpX t q|F 0 t qq 0ďtďT by means of Itô's formula. For the sake of illustration we rewrite this SPDE in a few particular cases which we will revisit later on:

1. If we assume that σpt, x, µ, αq " σ is a constant, that σ 0 pt, x, µ, αq " σ 0 pt, xq is also uncontrolled and that the spatial white noise is actually scalar, namely W pdξ, dtq " dW 0 t for a scalar Wiener process W 0 independent of the Wiener processes pW i q iě1 , then the stochastic differential equations giving the dynamics of the state of the system read

dX i t " bpt, X i t , µ N t , α i t qdt `σdW i t `σ0 pt, X i t qdW 0 t , i " 1, ¨¨¨, N (4)
and the limit µ t of the empirical distributions satisfies the equation

dµ t " ´∇ ¨"b `t, ¨, µ t , αpt, ¨, µ t q ˘µt ‰ dt ´∇ ¨`σ 0 pt, ¨qdW 0 t µ t 1 2 trace " ∇ 2 ´"σσ : `σ0 σ 0: ‰ pt, ¨q¯µ t ı dt. (5)
Writing the corresponding version (3), rough paths theory would permit to express the dynamics of the path pX t q tě0 conditional on the values of W 0 . This would be another way to express the dynamics of the conditional marginal laws of pX t q tě0 given W 0 . 2. Note that, when the ambient noise is not present (i.e. either σ 0 " 0 or W 0 " 0), this SPDE reduces to a deterministic PDE. It is the Kolmogorov equation giving the forward dynamics of the distribution at time t of the nonlinear diffusion process pX t q tě0 (nonlinear in McKean's sense).

Solution Strategy for Mean Field Games

When players are assigned a cost functional, a natural (and challenging) question is to determine equilibriums within the population. A typical framework is to assume that the cost to player i, for any i P t1, . . . , N u, writes

J i pα 1 , . . . , α N q " E "ż T 0 f `t, X i t , µ N t , α i t ˘dt `g`X i T , µ N T ˘,
for some functions f : r0, T s ˆRd ˆPpR d q ˆA Ñ R and g : R d ˆPpR d q Ñ R. Keep in mind the fact that the cost J i depends on all the controls ppα j t q 0ďtďT q jPt1,...,N u through the flow of empirical measures pµ N t q 0ďtďT . In the search for a Nash equilibrium α, one assumes that all the players j but one keep the same strategy profile α, and the remaining player deviates from this strategy in the hope of being better off. If the number of players is large (think N Ñ 8), one expects that the empirical measure µ N t will not be affected much by the deviation of one single player, and for all practical purposes, one can assume that the empirical measure µ N t is approximately equal to its limit µ t . So in the case of large symmetric games, the search for approximate Nash equilibriums could be done through the solution of the optimization problem of one single player (typically the solution of a stochastic control problem instead of a large game) when the empirical measure µ N t is replaced by the solution µ t of the SPDE (2) appearing in this limiting regime, the 'α' plugged in [START_REF] Carmona | Forward-Backward Stochastic Differential Equations and Controlled McKean Vlasov Dynamics[END_REF] denoting the strategy used by the players at equilibrium.

The implementation of this method can be broken down into three steps for pedagogical reasons:

(i) Given an initial distribution µ 0 on R d , fix an arbitrary measure valued adapted stochastic process pµ t q 0ďtďT over the probability space of the random measure W 0 . It stands for a possible candidate for being a Nash equilibrium. (ii) Solve the (standard) stochastic control problem (with random coefficients) inf

pαtq 0ďtďT E "ż T 0 f pt, X t , µ t , α t qdt `gpX T , µ T q  (6)
subject to

dX t " b `t, X t , µ t , α t ˘dt `σ`t , X t , µ t , α t ˘dW t `żΞ σ 0 `t, X t , µ t , α t , ξ ˘W 0 pdξ, dtq,
with X 0 " µ 0 , over controls in feedback form, Markovian in X conditional on the past of the flow of random measures pµ t q 0ďtďT . (iii) Plug the optimal feedback function αpt, x, µ t q in the SPDE [START_REF] Carmona | Forward-Backward Stochastic Differential Equations and Controlled McKean Vlasov Dynamics[END_REF]. Then, determine the measure valued stochastic process pµ t q 0ďtďT so that the solution of the SPDE (2) be precisely pµ t q 0ďtďT itself.

Clearly, this last item requires the solution of a fixed point problem in an infinite dimensional space, while the second item involves the solution of an optimization problem in a space of stochastic processes. Thanks to the connection between the SPDE (2) and the McKean-Vlasov equation (3), the fixed point item (iii) reduces to the search for a flow of random measures pµ t q 0ďtďT such that the law of the optimally controlled process (resulting from the solution of the second item) is in fact µ t , i.e. @t P r0, T s, µ t " LpX t |F 0 t q. In the absence of the ambient random field noise term W 0 , the measure valued adapted stochastic process pµ t q 0ďtďT can be taken as a deterministic function r0, T s Q t Þ Ñ µ t P PpR d q and the control problem in item (ii) is a standard Markovian control problem. Moreover, the fixed point item (iii) reduces to the search for a deterministic flow of measures r0, T s Q t Þ Ñ µ t P PpR d q such that the optimally controlled process (resulting from the solution of the second item) satisfies LpX t q " µ t .

Stochastic HJB Equation

In this subsection, we study the stochastic control (ii) when the flow of random measures µ " pµ t q 0ďtďT is fixed. Optimization is performed over the set A of F-progressively measurable A-

valued processes pα t q 0ďtďT satisfying E ż T 0 |α t | 2 dt ă 8.
For each pt, xq P r0, T sˆR d , we let pX t,x s q tďsďT be the solution of the stochastic differential equation (being granted that it is well-posed) dX s " bps, X s , µ s , α s qds `σps, X s , µ s , α s qdW s `żΞ σ 0 ps, X s , µ a , α s , ξqW 0 pdξ, dsq,

with X t " x. With this notation, we define the (conditional) cost

J µ t,x `pα s q tďsďT ˘" E " ż T t f ps, X t,x s , µ s , α s qds `gpX t,x T , µ T q ˇˇF 0 t  (8)
and the (conditional) value function

V µ pt, xq " ess inf pαsq tďsďT PA J µ t,x `pα s q tďsďT ˘. (9)
We shall drop the superscript and write X s for X t,x s when no confusion is possible. Under some regularity assumptions, we can show that, for each x P R d , pV pt, xqq 0ďtďT is an F 0 -semi-martingale and deduce, by identification of its Itô decomposition, that it solves a form of stochastic Hamilton-Jacobi Bellman (HJB) equation. Because of the special form of the state dynamics [START_REF] Chassagneux | McKean-Vlasov FBSDEs and related Master Equation[END_REF]

Assuming that the value function is smooth enough, we can use a generalization of the dynamic programming principle to the present set-up of conditional value functions to show that V µ pt, xq satisfies a form of stochastic HJB equation as given by a parametric family of BSDEs in the sense that:

V µ pt, xq " gpxq `ż T t L ˚`s, x, B x V ps, xq, B 2 x V µ ps, xq, pZ µ ps, x, ξqq ξPΞ ˘ds `ż T t Z µ ps, x, ξqW 0 pdξ, dsq. (11)
Noticing that W 0 enjoys the martingale representation theorem (see Chapter 1 in [START_REF] Nualart | The Malliavin Calculus and Related Topics[END_REF]), this result can be seen as part of the folklore of the theory of backward SPDEs (see for example [START_REF] Peng | Stochastic Hamilton Jacobi Bellman equations[END_REF] or [START_REF] Ma | On non-Markovian forward-backward SDEs and backward stochastic PDEs[END_REF]).

Towards the Master Equation

The definition of L ˚in [START_REF] Guéant | Mean field games and applications[END_REF] suggests that the optimal feedback in (8) could be identified as a function α of t, x, µ t , V µ pt, ¨q and Z µ pt, ¨, ¨q realizing the infimum appearing in the definition of L ˚. Plugging such a choice for α in the SPDE (2), we deduce that the fixed point condition in the item (iii) of a definition of an MFG equilibrium could be reformulated in terms of an infinite dimensional FBSDE, the forward component of which being the Kolmogorov SPDE (2) (with the specific choice of α) and the backward component the stochastic HJB equation [START_REF] Huang | Large population stochastic dynamic games: closed-loop McKean-Vlasov systems and the Nash certainty equivalence principle[END_REF]. The forward variable would be pµ t q 0ďtďT and the backward one would be pV µ pt, ¨qq 0ďtďT . Standard FBSDE theory suggests the existence of a decoupling field expressing the backward variable in terms of the forward one, in other words that V µ pt, xq could be written as V pt, x, µ t q for some function V , or equivalently, that V µ pt, ¨q could be written as V pt, ¨, µ t q. Using a special form of Itô's change of variable formula proven in the appendix at the end of the paper, these decoupling fields are easily shown, at least when they are smooth, to satisfy PDEs or SPDEs in the case of FBSDEs with random coefficients. The definition of the special notion of smoothness required for this form of Itô formula is recalled in the appendix. This is our hook to Lions's master equation. In order to make this point transparent, we strive in the sequel, to provide a better understanding of the mapping V : r0, T s ˆRd ˆPpR d q Ñ R and of its dynamics.

An Explicitly Solvable Model

This section is devoted to the analysis of an explicitly solvable model. It was introduced and solved in [START_REF] Carmona | Mean Field Games and Systemic Risk[END_REF]. We reproduce the part of the solution which is relevant to the present discussion. Our interest in this model is the fact that the finite player game can be solved explicitly and the limit N Ñ 8 of the solution can be controlled. We shall use it as a motivation and testbed for the introduction of the master equation of mean field games with a common noise.

Constructions of Exact Nash Equilibria for the N-Player Game

We denote by X i t the log-capitalization of a bank i P t1, ¨¨¨, N u at time t. We assume that each bank controls its rate of borrowing and lending through the drift of X i t in such a way that:

dX i t " " apm N t ´Xi t q `αi t ‰ dt `σˆa 1 ´ρ2 dW i t `ρdW 0 t ˙, (12)
where W i t , i " 0, 1, . . . , N are independent scalar Wiener processes, σ ą 0, a ě 0, and m N t denotes the sample mean of the X i t as defined by m N t " pX 1 t `¨¨¨`X N t q{N . So, in the notation introduced in (1), we have bpt, x, µ, αq " apm ´xq `α, with m " ż R xµpdxq, since the drift of pX i t q tě0 at time t depends only upon X i t itself and the mean m N t of the empirical distribution µ N t of X t " pX 1 t , ¨¨¨, X N t q, and σpt, x, µ, αq " σ a 1 ´ρ2 , and σ 0 pt, xq " σρ.

Bank i P t1, ¨¨¨, N u controls its rate of lending and borrowing (to a central bank) at time t by choosing the control α i t in order to minimize

J i pα 1 , ¨¨¨, α N q " E " ż T 0 f pt, X i t , µ N t , α i t qdt `gpX i T , µ N T q  , (13)
where the running and terminal cost functions f and g are given by:

f pt, x, µ, αq " 1 2 α 2 ´qαpm ´xq `ǫ 2 pm ´xq 2 , gpx, µq " c 2 pm ´xq 2 , (14)
where, as before, m denotes the mean of the measure µ. Clearly, this is a Linear-Quadratic (LQ) model and, thus, its solvability should be equivalent to the well-posedness of a matricial Riccati equation. However, given the special structure of the interaction, the Ricatti equation is in fact scalar and can be solved explicitly as we are about to demonstrate. Given an N -tuple pα i q 1ďiďN of functions from r0, T s ˆR into R, we define, for each i P t1, ¨¨¨, N u, the related value function V i by:

V i pt, x 1 , . . . , x N q " inf pα i s q tďsďT E " ż T t f `s, X i s , µ N s , α i s ˘ds `gi pX i T , µ N T q ˇˇX t " x  ,
with the cost functions f and g given in [START_REF] Lasry | Mean field games[END_REF], and where the dynamics of pX 1 s , . . . , X N s q tďsďT are given in [START_REF] Lasry | Jeux à champ moyen I. Le cas stationnaire[END_REF] with X j t " x j for j P t1, . . . , N u and α j s " αj ps, X j s q for j " i. By dynamic programming, the N scalar functions V i must satisfy the system of HJB equations:

B t V i pt, xq `inf αPR `apx ´xi q `α˘B x i V i pt, xq `1 2 α 2 ´qα `x ´xi ˘( `ǫ 2 px ´xi q 2
`ÿ j "i `apx ´xj q `α j pt, x j q ˘Bx j V j pt, xq `σ2

2 N ÿ j"1 N ÿ k"1 `ρ2 `δj,k p1 ´ρ2 q ˘B2 x j x k V i pt, xq " 0,
for pt, xq P r0, T s ˆRN , where we use the notation x for the mean x " px 1 `¨¨¨`x N q{N and with the terminal condition V i pT, xq " pc{2qpx ´xi q 2 . The infima in these HJB equations can be computed explicitly:

inf αPR `apx ´xi q `α˘B x i V i pt, xq `1 2 α 2 ´qα `x ´xi ˘( " apx ´xi qB x i V i pt, xq ´1 2 " q `x ´xi ˘´B x i V i pt, xq ‰ 2 ,
the infima being attained for α " q `x ´xi ˘´B x i V i pt, xq.

Therefore, the Markovian strategies pα i q 1ďiďN forms a Nash equilibrium if αi pt, xq " q `x ´xi ˘B xi V i pt, xq, which suggests to solve the system of N coupled HJB equations:

B t V i `N ÿ j"1 " pa `qq `x ´xj ˘´B x j V j ‰ B x j V i `σ2 2 N ÿ j"1 N ÿ k"1 `ρ2 `δj,k p1 ´ρ2 q ˘B2 x j x k V i `1 2 pǫ ´q2 q `x ´xi ˘2 `1 2 pB x i V i q 2 " 0, i " 1, ¨¨¨, N, (15)
with the same boundary terminal condition as above. Then, the feedback functions αi pt, xq " qpx ´xi q ´Bx i V i pt, xq are expected to give the optimal Markovian strategies. Generally speaking, these systems of HJB equations are usually difficult to solve. Here, because the particular forms of the couplings and the terminal conditions, we can solve the system by inspection, checking that a solution can be found in the form

V i pt, xq " η t 2 px ´xi q 2 `χt , (16)
for some deterministic scalar functions t Þ Ñ η t and t Þ Ñ χ t satisfying η T " c and χ T " 0 in order to match the terminal conditions for the V i s. Indeed, the partial derivatives

B x j V i and B x j x k V i read B x j V i pt, xq " η t `1 N ´δi,j ˘`x ´xi ˘, B 2 x j x k V i pt, xq " η t `1 N ´δi,j ˘p 1 N ´δi,k q.
and plugging these expressions into [START_REF] Lions | Théorie des jeux à champs moyen et applications[END_REF], and identifying term by term, we see that the system of HJB equations is solved if an only if

$ ' & ' % 9 η t " 2pa `qqη t ``1 ´1 N 2 ˘η2 t ´pǫ ´q2 q, 9 χ t " ´1 2 σ 2 p1 ´ρ2 q `1 ´1 N ˘ηt , (17)
with the terminal conditions η T " c and χ T " 0. As emphasized earlier, the Riccati equation is scalar and can be solved explicitly. One gets:

η t " ´pǫ ´q2 q `epδ `´δ ´qpT ´tq ´1˘´c`δ `epδ `´δ ´qpT ´tq ´δ´δ ´epδ `´δ ´qpT ´tq ´δ`˘´c p1 ´1{N 2 q `epδ `´δ ´qpT ´tq ´1˘, (18)
provided we set:

δ ˘" ´pa `qq ˘?R, with R " pa `qq 2 `ˆ1 ´1 N 2 ˙pǫ ´q2 q ą 0. ( 19
)
Observe that the denominator in ( 18) is always negative since δ `ą δ ´, so that η t is well defined for any t ď T . The condition q 2 ď ǫ implies that η t is positive with η T " c. Once η t is computed, one solves for χ t (remember that χ T " 0) and finds:

χ t " 1 2 σ 2 p1 ´ρ2 q ´1 ´1 N ¯ż T t η s ds. (20)
For the record, we note that the optimal strategies read αi

t " q `Xt ´Xi t ˘´B x i V i " ´q `p1 ´1 N qη t ¯`X t ´Xi t ˘, (21)
and the optimally controlled dynamics:

dX i t " ´a `q `p1 ´1 N qη t ¯`X t ´Xi t ˘dt `σ´a 1 ´ρ2 dW i t `ρdW 0 t ¯. ( 22
)

The Mean Field Limit

We emphasize the dependence upon the number N of players and we now write η N t and χ N t for the solutions η t and χ t of the system [START_REF] Nualart | The Malliavin Calculus and Related Topics[END_REF], and V i,N pt, xq " pη N {2qpx ´xi q 2 `χN t for the value function of player i in the N player game. Clearly, lim

N Ñ8 η N t " η 8 t ,
and lim

N Ñ8 χ N t " χ 8 t ,
where the functions η 8 t and χ 8 t solve the system:

$ & % 9 η 8 t " 2pa `qqη 8 t `pη 8 t q 2 ´pǫ ´q2 q, 9 χ 8 t " ´1 2 σ 2 p1 ´ρ2 qη 8 t , (23)
which is solved as in the case N finite. We find η 8 t " ´pǫ ´q2 q `epδ `´δ ´qpT ´tq ´1˘´c`δ `epδ `´δ ´qpT ´tq ´δ´δ ´epδ `´δ ´qpT ´tq ´δ`˘´c `epδ `´δ ´qpT ´tq ´1˘,

and

χ 8 t " 1 2 σ 2 p1 ´ρ2 q ż T t η 8 s ds. ( 25
)
Next we consider the equilibrium behavior of the players' value functions V i,N . For the purpose of the present discussion we notice that the value functions V i,N of all the players in the N player game can be written as

V i,N `t, px 1 , ¨¨¨, x N q ˘" V N ˆt, x i , 1 N N ÿ j"1 δ x j
ẇhere the single function V N is defined as

V N pt, x, µq " η N t 2 ˆx ´żR xdµpxq ˙2 `χN t , pt, x, µq P r0, T s ˆR ˆP1 pRq,
where P 1 pR d q denotes the space of integrable probability measures on R d . Since the dependence upon the measure is only through the mean of the measure, we shall often use the function

v N pt, x, mq " η N t 2 px ´mq 2 `χN t , pt, x, mq P r0, T s ˆR ˆR,
Notice that, at least for pt, x, mq fixed, we have lim

N Ñ8 v N pt, x, mq " v 8 pt, x, mq where
v 8 pt, x, mq " η 8 t 2 px ´mq 2 `χ8 t , pt, x, mq P r0, T s ˆR ˆR.
Similarly, all the optimal strategies in (21) may be expressed through a single feedback function αN pt, x, mq " rq `p1 ´1{N qη N t spm ´xq as αi t " αN pt, X i t , m N t q. Clearly, lim N Ñ8

αN pt, x, mq " α8 pt, x, mq, where α8 pt, x, mq " rq `ηt spm ´xq.

Repeating the analysis in Subsection 2.1, we find that the limit of the empirical distributions satisfies the following version of (5):

dµ t " ´Bx ˆrapm t ´¨q ´α8 pt, ¨qsµ t ˙dt `σ2 2 B 2 x µ t dt ´σρB x µ t dW 0 t , t P r0, T s, (26)
where m t " ş R d xdµ t pxq, which is the Kolmogorov equation for the conditional marginal law given W 0 of the solution of the McKean-Vlasov equation:

dX t " " a `mt ´Xt q `α8 pt, X t q ‰ dt `σ ´ρdW 0 t `a1 ´ρ2 dW t ¯, (27)
subject to the condition m t " ErX t |F 0 t s. Applying the Kolmogorov equation to the test function φpxq " x, we get

dm t " ˆż α 8 pt, xqdµ t pxq ˙dt `σρdW 0 t . (28)
We now write the stochastic HJB equation [START_REF] Huang | Large population stochastic dynamic games: closed-loop McKean-Vlasov systems and the Nash certainty equivalence principle[END_REF] in the present context. Remember that we assume that the stochastic flow pµ t q 0ďtďT is given (as the solution of (26) with some prescribed initial condition µ 0 " µ), and hence so is pm t q 0ďtďT . Here

L ˚pt, x, y, z, z 0 q " inf αPA " rapm t ´xq `αsy `σ2 2 z `σρz 0 `α2 2 ´qαpm t ´xq `ǫ 2 pm t ´xq 2  .
Since the quantity to minimize is quadratic in α, we need to compute it for ᾱ " ᾱpt, x, m t , yq with ᾱpt, x, m, yq " qpm ´xq ´y. We get:

L ˚pt, x, y, z, z 0 q " pa `qqpm t ´xqy ´1 2 y 2 `σ2 2 z `σρz 0 `1 2 pǫ ´q2 qpm t ´xq 2 .
Accordingly, the stochastic HJB equation takes the form

d t V µ pt, xq " " ´pa `qqpm t ´xqB x V µ pt, xq `1 2 rB x V µ pt, xqs 2 ´σ2 2 B 2 x V µ pt, xq ´σρB x Z µ pt, xq ´1 2 pǫ ´q2 qpm t ´xq 2  dt ´Zµ pt, xqdW 0 t , (29)
with the boundary condition V µ t pxq " pc{2qpm T ´xq 2 .

Search for a Master Equation

A natural candidate for solving (29) is the random field pt, xq Þ Ñ v 8 pt, x, m t q, where as above pm t q 0ďtďT denotes the means of the solution pµ t q 0ďtďt of the Kolmogorov SPDE (26). This can be checked rigorously by using the expression of v 8 and by expanding pv 8 pt, x, m t qq 0ďtďT by Itô's formula (taking advantage of (28)). As suggested at the end of the previous section, this shows that the stochastic HJB equation admits a solution V µ pt, xq that can be expressed as a function of the current value µ t of the solution of the Kolmogorov SPDE, namely

V µ pt, xq " v 8 ˆt, x, ż R d x 1 dµ t px 1 q ˙.
The same argument shows that pX t q 0ďtďT defined in (27) as a solution of a McKean-Vlasov SDE is in fact the optimal trajectory of the control problem considered in the item (ii) of the definition of a MFG, see [START_REF] Carmona | Mean Field Games and Systemic Risk[END_REF], under the solution pµ t q 0ďtďT of the stochastic PDE (26). Put it differently, pµ t q 0ďtďT is a solution of the MFG and the function α 8 is the associated feedback control, as suggested by the asymptotic analysis performed in the previous paragraph.

A natural question is to characterize the dynamics of the function v 8 in an intrinsic way. By definition of the value function (see ( 9)), we have

V µ pt, X t q " E " ż T t f `s, X s , µ s , α8 ps, X s q ˘ds `g`X T , µ T ˘ˇF t  so that dV µ pt, X t q " ´f `t, X t , µ t , α8 pt, X t q ˘dt `dM t , t P r0, T s,
for some pF t q 0ďtďT -martingale pM t q 0ďtďT . Recalling that ᾱpt, x, m, yq " qpm´xq´y, B x v 8 pt, x, mq " η 8 t px ´mq, and α8 pt, x, mq " rq `ηt spm ´xq, we deduce that

α8 pt, x, mq " ᾱ`t , x, m, B x v 8 pt, x, mq ˘,
which is the standard relationship in stochastic optimal control for expressing the optimal feedback in terms of the minimizer ᾱ of the underlying extended Hamiltonian and of the gradient of the value function v 8 . We deduce that f `t, X t , µ t , α8 pt, X t q ˘" ´1 2 `qpm t ´Xt q ´Bx v 8 pt, X t , m t q ˘`qpm t ´Xt q `Bx v 8 pt, X t , m t q ǫ 2 `mt ´Xt ˘2,

so that dV µ pt, X t q " ´´1 2 pǫ ´q2 qpm t ´Xt q 2 ´1 2 " B x v 8 pt, X t , m t q ‰ 2 ¯dt `dM t . (30)
We are to compare this Itô expansion with the Itô expansion of pv 8 pt, X t , m t qq 0ďtďT . Using the short-hand notation v 8 t for v 8 pt, X t , m t q and standard Itô's formula, we get:

dv 8 t " B t v 8 t dt `Bx v 8 t dX t `Bm v 8 t dm t `σ2 2 B 2 xx v 8 t `σ2 2 ρ 2 B 2 mm v 8 t `σ2 ρ 2 B 2 xm v 8 t " " B t v 8 t `Bx v 8 t apm t ´Xt q `Bx v 8 t α8 pt, X t q `Bm v 8 t xµ t , α 8 pt, ¨qy `σ2 2 B 2 x v 8 t `σ2 2 ρ 2 B 2 m v 8 t `σ2 ρ 2 B 2 xm v 8 t  dt `σρrB x v 8 t `Bm v 8 t sdW 0 t `σa 1 ´ρ2 B x v 8 t dW t . (31)
Identifying the bounded variation terms in (30) and (31), we get:

B t v 8 t `Bx v 8 t apm t ´Xt q `Bx v 8 t α8 pt, X t q `Bm v 8 t xµ t , α 8 pt, ¨qy `σ2 2 B 2 x v 8 t `σ2 2 ρ 2 B 2 m v 8 t `σ2 ρ 2 B 2 xm v 8 t " ´1 2 pǫ ´q2 qpm t ´Xt q 2 ´1 2 " B x v 8 t ‰ 2 ,
where α8 pt, x, mq " qpm ´xq ´Bx v 8 pt, x, mq. Therefore, for a general smooth function V : pt, x, mq Þ Ñ V pt, x, mq, the above relationship with v 8 replaced by V holds if

B t V pt, x, mq `pa `qqpm ´xqB x V pt, x, mq `1 2 pǫ ´q2 qpm ´xq 2 ´1 2 rB x V pt, x, mqs 2 `σ2 2 B 2 x V pt, x, mq `σ2 2 ρ 2 B 2 m V pt, x, mq `σ2 ρ 2 B 2 xm V pt, x, mq " 0, (32)
for all pt, x, mq P r0, T s ˆRd ˆRd provided we have

ż B x V pt, x, mqdµpxq " 0, 0 ď t ď T, (33)
(33) being used to get rid of the interaction between µ t and α 8 . Obviously, v 8 satisfies (33).

(Notice that this implies that the stochastic Kolmogorov equation becomes: dm t " ρσdW 0 t .) Equation (32) reads as the dynamics for the decoupling field permitting to express the value function V µ as a function of the current statistical state µ t of the population. We call it the master equation of the problem.

The Master Equation

While we only discussed mean field games so far, it turns out that the concept of master equation applies as well to the control of dynamics of McKean-Vlasov type whose solution also provides approximate equilibriums for large populations of individuals interacting through mean field terms. See [START_REF] Carmona | Forward-Backward Stochastic Differential Equations and Controlled McKean Vlasov Dynamics[END_REF] for a detailed analysis. We first outline a procedure common to the two problems. Next we specialize this procedure to the two cases of interest, deriving a master equation in each case. Finally, we highlight the differences to better understand what differentiates these two related problems.

General Set-Up

Stated in loose terms, the problem is to minimize the quantity

E " ż T 0 f ps, X α s , µ s , α s qds `gpX α T , µ T q  (34)
over the space of square integrable F-adapted controls pα s q 0ďsďT under the constraint that

dX α s " b `s, X α s , µ s , α s ˘ds `σps, X α s , µ s , α s qdW s `żΞ σ 0 ps, X α s , µ s , α s , ξqW 0 pdξ, dsq. (35)
Yet the notion of what we call a minimizer must be specified. Obvious candidates for a precise definition of the minimization problem lead to different solutions. We consider two specifications: on the one hand, mean field games and control of McKean-Vlasov dynamics on the other.

1. When handling mean-field games, minimization is performed along a frozen flow of measures pµ s " μs q 0ďsďT describing a statistical equilibrium of the population, and the stochastic process p Xs q 0ďsďT formed by the optimal paths of the optimal control problem (34) is required to satisfy the matching constraints μs " Lp Xs |F 0 s q for 0 ď s ď T . This is exactly the procedure described in Subsection 2.2.

2. Alternatively, minimization can be performed over the set of all the solutions of (35) subject to the McKean-Vlasov constraint pµ s " µ α s q 0ďsďT , with µ α s " LpX α s |F 0 s q for 0 ď s ď T , in which case the problem consists in minimizing the cost functional (34) over McKean-Vlasov diffusion processes.

As discussed painstakingly in [START_REF] Carmona | Control of McKean-Vlasov versus Mean Field Games[END_REF], the two problems have different solutions since, in mean field games, the minimization is performed first and the fitting of the distribution of the optimal paths is performed next, whereas in the control of McKean-Vlasov dynamics, the McKean-Vlasov constraint is imposed first and the minimization is handled next. Still, we show here that both problems can be reformulated in terms of master equations, and we highlight the differences between the two in these reformulations.

The reason for handling both problems within a single approach is that in both cases, we rely on manipulations of a value function defined over the enlarged state space R d ˆP2 pR d q. For technical reasons, we restrict ourselves to measures in P 2 pR d q which denotes the space of square integrable probability measures (i.e. probability measures with a finite second moment). Indeed, for each pt, x, µq P r0, T s ˆRd ˆP2 pR d q, we would like to define V pt, x, µq as the expected future costs:

V pt, x, µq " E " ż T t f ps, X α s , μs , αs qds `gpX α T , μT q ˇˇX α t " x  , (36)
where α minimizes the quantity (34) when we add the constraint µ t " µ and compute the time integral between t and T . In other words:

pα s q tďsďT " argmin α E " ż T t f ps, X α s , µ s , α s qds `gpX α T , µ T q  , (37)
the rule for computing the infimum being as explained above, either from the mean field game procedure as in 1, or from the optimization over McKean-Vlasov dynamics as explained in 2.

In both cases, the flow pμ s q tďsďT appearing in (36) satisfies the fixed point condition pμ s " LpX α s |F 0,t s qq tďsďT , which is true in both cases as pX α s q tďsďT is an optimal path. Here and in the following pF 0,t s q tďsďT is the filtration generated by the future increments of the common noise W 0 , in the sense that F 0,t s " σtW 0 r ´W 0 t : t ď r ď su. Recall that we use the notation W 0 r for tW 0 pΛ, r0, rqu Λ when Λ varies through the Borel subsets of Ξ. Below, the symbol 'hat' always refers to optimal quantities, and pX α s q tďsďT is sometimes denoted by p Xs q tďsďT . Generally speaking, the definition of the (deterministic) function V pt, x, µq makes sense whenever the minimizer pα s q tďsďT exists and is unique. When handling mean-field games, some additional precaution is needed to guarantee the consistency of the definition. Basically, we also need that, given the initial distribution µ at time t, there exists a unique equilibrium flow of conditional probability measures pμ s q tďsďT satisfying μt " µ and μs " Lp Xs |F 0,t s q for all s P rt, T s, where p Xs q tďsďT is the optimal path of the underlying minimization problem (performed under the fixed flow of measures pμ s q tďsďT ). In that case, the minimizer pα s q tďsďT reads as the optimal control of p Xs q tďsďT . In the case of the optimal control of McKean-Vlasov stochastic dynamics, minimization is performed over the set of conditional McKean-Vlasov diffusion processes with the prescribed initial distribution µ at time t, in other words, satisfying (35) with LpX t q " µ and µ s " µ α s " LpX α s |F 0,t s q for all s P rt, T s. In that case, the mapping pt, µq Þ Ñ ş R d V pt, x, µqdµpxq appears as the value function of the optimal control problem:

E " V pt, χ, µq ‰ " inf α E " ż T t f `s, X α s , LpX α s |F 0,t s q, α s ˘ds `g`X α T , LpX α T |F 0,t T q ˘, (38)
subject to X α t " χ where χ is a random variable with distribution µ, i.e. χ " µ. Our goal is to characterize the function V as the solution of a partial differential equation (PDE) on the space r0, T s ˆRd ˆP2 pR d q. In the framework of mean-field games, such an equation was touted in several presentations, and called the master equation. See for example [START_REF] Lions | Théorie des jeux à champs moyen et applications[END_REF], [START_REF] Cardaliaguet | Notes on mean field games[END_REF] or [START_REF] Gomes | Saude Mean field games models -a brief survey[END_REF]. We discuss the derivation of this equation below in Subsection 4.4. Using a similar strategy, we also derive a master equation in the case of the optimal control of McKean-Vlasov stochastic dynamics in Subsection 4.5 below.

Dynamic Programming Principle

In order to understand better the definition (36), we consider the case in which the minimizer pα s q tďsďT has a feedback form, namely αs reads as αps, X α s , μs q for some function α : r0, T s ˆRd P2 pR d q Ñ R. In this case, (36) becomes

V pt, x, µq " E " ż T t f `s, X α s , μs , αps, X α s , μs q ˘ds `gpX α T , μT q ˇˇX α t " x  , (39)
where pX α s q tďsďT is the solution (if well-defined) of (35) with α s replaced by αps, X α s , μs q. It is worth recalling that, in that writing, μs matches the conditional law LpX α s |F 0,t s q and is forced to start from μt " µ at time t.

Following the approach used in finite dimension, a natural strategy is then to use (39) as a basis for deriving a dynamic programming principle for V . Quite obviously, a very convenient way to do so consists in requiring the optimal pair p Xs " X α s , μs q tďsďT to be Markov in R d ˆP2 pR d q, in which case we get

V pt `h, X α t`h , μt`h q " E " ż T t`h
f ps, X α s , μs , αs qds `gpX α T , μT q ˇˇF 0,t t`h _ σ X α t , pW s ´Wt q sPrt,t`hs (  .

Here, the σ-field F 0,t t`h _ σtX α t , pW s ´Wt q sPrt,t`hs u comprises all the events observed up until time t `h.

The rigorous proof of the Markov property for the path p Xs " X α s , μs q tďsďT is left open. Intuitively, it sounds reasonable to expect that the Markov property holds if, for any initial distribution µ, there exists a unique equilibrium pμ s q tďsďT starting from μt " µ at time t P r0, T s. The reason is that, when uniqueness holds, there is no need to investigate the past of the optimal path in order to decide of the future of the dynamics. Such an argument is somehow quite generic in probability theory. In particular, the claim is expected to be true in both cases, whatever the meaning of what an equilibrium is. Of course, this suggests that the following dynamic version of (36)

V pt, x, µq " E " ż t`h t f ps, X α s , μs , αs qds `V `t `h, X α t`h , μt`h ˘ˇX α t " x  (40)
must be valid. The fact that (40) should be true in both cases is the starting point for our common analysis of the master equation. For instance, as a by-product of (40), we can derive a variational form of the dynamic programming principle:

E " V pt, χ, µq ‰ " inf E " ż t`h t f ps, X α s , µ s , α s qds `V pt `h, X α t`h , µ t`h q  , (41)
which must be true in both cases as well, provided the random variable χ has distribution µ, i.e. χ " µ, the minimization being defined as above according to the situation. The proof of (41) is as follows. First, we observe from (39) that (41) must be valid when t `h " T . Then, (40) implies that the left-hand side is greater than the ride-hand side by choosing pα s q tďsďT as a control. To prove the converse inequality, we choose an arbitrary control pα s q tďsďt`h between times t and t `h. In the control of McKean-Vlasov dynamics, this means that the random measures pµ s q tďsďt`h are chosen accordingly, as they depend on pα s q tďsďt`h , so that µ t`h is equal to the conditional law of X α t`h at time t `h. At time t `h, this permits to switch to the optimal strategy starting from pX α t`h , µ t`h q. The resulting strategy is of a greater cost than the optimal one. By (39), this cost is exactly given by the right-hand side in (41).

In the framework of mean field games, the argument for proving that the left-hand side is less than the right-hand side in (41) is a bit different. The point is that the flow pµ s q tďsďT is fixed and matches pμ s q tďsďT , so that αps, X α s , μs q reads as an optimal control for optimizing (34) in the environment pµ s " μs q tďsďT . So in that case, V pt, x, µq is expected to match the optimal conditional cost

V pt, x, µq " inf E " ż T t f ps, X α s , μs , α s qds `gpX α T , μT q ˇˇX α t " x  , (42)
where pX α s q tďsďT solves the SDE (35) with pµ s " μs q tďsďT therein. Going back to (41), the choice of an arbitrary control pα s q tďsďt`h between times t and t `h doesn't affect the value of pµ s q tďsďt`h , which remains equal to pμ s q tďsďt`h . At time t `h, this permits to switch to the optimal strategy starting from X α t`h in the environment pμ s q tďsďT . Again, the resulting strategy is of a greater cost than the optimal one and, by (39), this cost is exactly given by the right-hand side in (41).

We emphasize that, when controlling McKean-Vlasov dynamics, (42) fails as in that case, the flow of measures is not frozen during the minimization procedure. In particular, the fact that (42) holds true in mean-field games only suggests that V satisfies a stronger dynamic programming principle in that case:

V pt, x, µq " inf E " ż t`h t f ps, X α s , μs , α s qds `V `t `h, X α t`h , μt`h ˘ˇX α t " x  . ( 43
)
The reason is the same as above. On the one hand, (40) implies that the left-hand side is greater than the ride-hand side by choosing pα s q tďsďT as a control. On the other hand, choosing an arbitrary control pα s q tďsďt`h between t and t `h and switching to the optimal control starting from X α t`h in the environment pμ s q tďsďT , the left-hand side must be less than the right-hand side.

Derivation of the Master Equation

As illustrated earlier (see also the discussion of the second example below), the derivation of the master equation can be based on a suitable chain rule for computing the dynamics of V along paths of the form (35). This requires V to be smooth enough in order to apply an Itô like formula.

In the example tackled in the previous section, the dependence of V upon the measure reduces to a dependence upon the mean of the measure, and a standard version of Itô's formula could be used. In general, the measure argument lives in infinite dimension and different tools are needed. The approach advocated by P.L. Lions in his lectures at the Collège de France suggests to lift-up the mapping V into Ṽ : r0, T s ˆRd ˆL2 p Ω, F , P; R d q Q pt, x, χq Þ Ñ Ṽ pt, x, χq " V pt, x, Lp χqq, where p Ω, F, Pq can be viewed as a copy of the space pΩ, F, Pq. The resulting Ṽ is defined on the product of r0, T s ˆRd and a Hilbert space, for which the standard notion of Fréchet differentiability can be used. Demanding V to be smooth in the measure argument is then understood as demanding Ṽ to be smooth in the Fréchet sense. In that perspective, expanding pV ps, X α s , µ s qq tďsďT is then the same as expanding p Ṽ ps, X α s , χs qq tďsďT , where the process p χs q tďsďT is an Itô process with pµ s q tďsďT as flow of marginal conditional distributions (conditional on F 0,t ).

The fact that we require p χs q tďsďT to have pµ s q tďsďT as flow of marginal conditional distributions calls for some precaution in the construction of the lifting. A way to do just this consists in writing pΩ, F, Pq in the form pΩ 0 ˆΩ1 , F 0 b F 1 , P 0 b P 1 q, pΩ 0 , F 0 , P 0 q supporting the common noise W 0 , and pΩ 1 , F 1 , P 1 q the idiosyncratic noise W . So an element ω P Ω can be written as ω " pω 0 , ω 1 q P Ω 0 ˆΩ1 . Considering a copy p Ω1 , F1 , P1 q of the space pΩ 1 , F 1 , P 1 q, it then makes sense to consider the process p χs q tďsďT as the solution of an equation of the same form of (35), but on the space pΩ 0 ˆΩ 1 , F 0 b F1 , P 0 b P1 q, p Ω1 , F1 , P1 q being endowed with a copy W of W . The realization at some ω 0 P Ω 0 of the conditional law of χs given F 0 then reads as the law of the random variable χs pω 0 , ¨q P L 2 p Ω1 , F1 , P1 ; R d q. Put in our framework, this makes rigorous the identification of Lp χs pω 0 , ¨qq with µ s pω 0 q.

Generally speaking, we expect that p Ṽ ps, X α s , χs q " Ṽ ps, X α s pω 0 , ω 1 q, χs pω 0 , ¨qqq tďsďT can be expanded as d Ṽ `s, X α s , χs ˘" " B t Ṽ ps, X α s , χs q `Aα x Ṽ ps, X α s , χs q `Aα µ Ṽ ps, X α s , χs q `Aα xµ Ṽ ps, X α s , χs q ‰ ds `dM s , t ď s ď T,

with Ṽ pT, x, χq " gpx, Lp χqq as terminal condition, where piq A α x denotes the second-order differential operator associated to the process pX α s q tďsďT . It acts on functions of the state variable x P R d and thus on the variable x in Ṽ pt, x, χq in (44).

piiq A α µ denotes some second-order differential operator associated to the process p χs q tďsďT . It acts on functions from L 2 p Ω1 , F1 , P1 ; R d q into R and thus on the variable χ in Ṽ pt, x, χq.

piiiq A α xµ denotes some second-order differential operator associated to the cross effect of pX α s q tďsďT and p χs q tďsďT , as both feel the same noise W 0 . It acts on functions from R d ˆL2 p Ω1 , F1 , P1 ; R d q and thus on the variables px, χq in Ṽ pt, x, χq. pivq pM s q tďsďT is a martingale.

A proof of (44) is given in the appendix at the end of the paper. Observe that A xµ " 0 if there is no common noise W 0 . Plugging (44) into (41) and letting h tend to 0, we then expect:

B t E " Ṽ pt, χ, χq ‰ `inf α E " A α x Ṽ pt, χ, χq `Aα µ Ṽ pt, χ, χq `Aα xµ Ṽ pt, χ, χq `f pt, χ, µ, αq ‰ " 0, (45)
where χ are χ random variables defined on pΩ 1 , F 1 , P 1 q and p Ω1 , F1 , P1 q respectively, both being distributed according to µ. If the minimizer has a feedback form, namely if the optimization over random variables α reduces to optimization over random variables of the form αpt, χ, µq, α being a function defined on r0, T s ˆRd ˆP2 pR d q, then the same strategy applied to (40), shows that Ṽ satisfies the master equation

B t Ṽ pt, x, χq `Aαpt,x,µq x Ṽ pt, χ, χq `Aαpt,x,μq µ Ṽ pt, χ, χq `Aαpt,x,µq xµ Ṽ pt, χ, χq `f `t, χ, µ, αpt, x, µq ˘" 0. ( 46
)
Of course, the rule for computing the infimum in (45) depends on the framework. In the case of the optimal control of McKean-Vlasov diffusion processes, p χs pω 0 , ω1 qq tďsďT in (44) is chosen as a copy, denoted by p Xα s pω 0 , ω1 qq tďsďT , of pX α s pω 0 , ω 1 qq tďsďT on the space pΩ 0 ˆΩ 1 , F 0 b F1 , P 0 b P1 q. In that case, A α µ depends on α explicitly. In the framework of mean field games, p χs pω 0 , ω1 qq tďsďT is chosen as a copy of the optimal path p Xs q tďsďT of the optimization problem (36) under the statistical equilibrium flow initialized at µ at time t. It does not depend on α so that A α µ does not depend on α. Therefore, A µ " A α µ has no role in the computation of the infimum. For the sake of illustration, we specialize the form of (46) to a simpler case when (35) reduces to dX s " bps, X s , µ s , α s qds `σpX s qdW s `σ0 pX s qdW 0 s . In that case, we know from the results presented in the appendix that

A α x φpt, x, χq " xb `t, x, Lp χq, α ˘, B x φpt, x, χqy `1 2 Trace " σpxq `σpxq ˘:B 2 x φpt, x, χq ‰ `1 2 Trace " σ 0 pxq `σ0 pxq ˘:B 2 x φpt, x, χq ‰ , A α µ φpt, x, χq " b `t, χ, Lp χq, β˘¨D µ φpt, x, χq `1 2 D 2 µ φ`t , x, χ˘" σ 0 p χq, σ 0 p χq ‰ `1 2 D 2 µ φ`t , x, χ˘" σp χq G, σp χq G‰ , A α xµ φpt, x, χq " x B x D µ φ`t , x, χ˘¨σ 0 p χq ( , σ 0 pxq D , (47)
where G is an N p0, 1q random variable on the space p Ω1 , F1 , P1 q, independent of W . The notations D µ and D 2 µ refer to Fréchet derivatives of smooth functions on the space L 2 p Ω1 , F1 , P1 ; R d q. For a random variable ζ P L 2 p Ω1 , F1 , P1 ; R d q, the notation D µ φpt, x, χq ¨ζ denotes the action of the differential of φpt, x, ¨q at point χ along the direction ζ. Similarly, the notation D 2 µ φpt, x, χqr ζ, ζs denotes the action of the second-order differential of φpt, x, ¨q at point χ along the directions p ζ, ζq. We refer to the appendix for a more detailed account.

Notice that χ in A α µ φpt, x, χq denotes the copy of χ, χ standing for the value at time t of the controlled diffusion process pχ s q tďsďT . Specifying the value of χ according to the framework used for performing the optimization, we derive below the precise shape of the resulting master equation. Notice also that A α xµ φpt, x, χq does not depend upon α as the coefficients σ 0 and σ do not depend on it.

The Case of Mean Field Games

In the framework of Mean-Field Games, p χs q tďsďT is chosen as a copy of the optimal path p Xs q tďsďT . This says that, in (47), χ stands for the value at time t of the optimally controlled state from the optimization problem (36) under the statistical equilibrium flow initialized at µ at time t. Therefore, the minimization in (45) reduces to

inf α E " xbpt, χ, µ, αq, B x Ṽ pt, χ, χqy `f pt, χ, µ, αq ‰ " inf α E " xbpt, χ, µ, αq, B x V pt, χ, µqy `f pt, χ, µ, αq ‰ , (48)
the equality following from the fact that B x Ṽ pt, x, χq is the same as B x V pt, x, µq (as the differentiation is performed in the component x).

Assume now that there exists a measurable mapping ᾱ : r0, T s ˆRd ˆP2 pR d q ˆRd Q pt, x, µq Þ Ñ ᾱpt, x, µ, yq, providing the argument of the minimization: ᾱpt, x, µ, yq " arg inf

αPR d Hpt, x, µ, y, αq, (49)
where the reduced Hamiltonian H is defined as:

Hpt, x, µ, y, αq " xbpt, x, µ, αq, yy `f pt, x, µ, αq,

Then, the minimizer in (48) must be α " ᾱpt, χ, µ, B x V pt, χ, µqq, showing that αpt, x, µq " ᾱpt, x, µ, B x V pt, x, µqq is an optimal feedback. By (46), the master equation reads

B t Ṽ pt, x, χq `inf α H `t, x, µ, B x Ṽ pt, x, χq, α ˘``A µ `Axµ ˘Ṽ pt, x, χq `1 2 Trace " σpxq `σpxq ˘:B 2 x Ṽ pt, x, χq ‰ `1 2 Trace " σ 0 pxq `σ0 pxq ˘:B 2 x Ṽ pt, x, χq ‰ " 0. ( 51
)
By identification of the transport term, this says that the statistical equilibrium of the MFG with µ as initial distribution must be given by the solution of the conditional McKean-Vlasov equation:

d Xs " b `s, Xs , μs , ᾱ`s , Xs , μs , B x V ps, Xs , μs q ˘`σ `X s ˘dW s `σ0 `X s ˘dW 0 s , (52)
subject to the constraint μs " Lp Xs |F 0 s q for s P rt, T s, with Xt " µ. We indeed claim Proposition 4.1. On the top of the assumption and notation introduced right above, assume that, for all t P r0, T s, x P R d and µ P P 2 pR d q |ᾱpt, x, µ, yq| ď C

" 1 `|x| `|y| `ˆż R d |x 1 | 2 dµpx 1 q ˙1{2  , (53)
and that the growths of the coefficients b, σ and σ 0 satisfy a similar bound. Assume also that Ṽ is a (classical) solution of (51) satisfying, for all t P r0, T s, x P R d and χ P L 2 p Ω1 , F1 , P1 ;

R d q, |B x Ṽ pt, x, χq| `}D µ Ṽ pt, x, χq} L 2 p Ω1 q ď C ´1 `|x| `Ẽ 1 " | χ| 2 ‰ 1{2 ¯, (54)
and that, for any initial condition pt, µq P r0, T s ˆP2 pR d q, Equation (52) has a unique solution.

Then, the flow pLp Xs |F 0 s qq tďsďT solves the mean field game with pt, µq as initial condition.

Proof. The proof consists of a verification argument. First, we notice from (53) and (54) that the solution of (52) is square integrable in the sense that its supremum norm over r0, T s is square integrable. Similarly, for any square integrable control α, the supremum of X α (with X α t " µ) is square integrable. The point is then to go back to (42) and to plug μs " Lp Xs |F 0 s q in the right-hand side. Replacing g by V pT, ¨, ¨q and applying Itô's formula in the appendix (see Proposition 6.5), using the growth and integrability assumptions to guarantee that the expectation of the martingale part is zero, we deduce that the right-hand side is indeed greater than V pt, x, µq. Choosing pα s " ᾱps, Xs , μs , B x V ps, Xs , μs qq tďsďT , equality must hold. This proves that p Xs q tďsďT is a minimization path of the optimization problem driven by its own flow of conditional distributions, which is precisely the definition of an MFG equilibrium.

Remark 4.2. Proposition 4.1 says that the solution of the master equation (51) contains all the information needed to solve the mean field game problem. In that framework, it is worth mentioning that the flow of conditional distributions pμ s " Lp Xs |F 0 s qq tďsďT solves the SPDE (2), with αps, ¨, μs q " ᾱps, x, μs , B x V ps, x, μs qq. Notice finally that pY s " B x V ps, Xs , μs qq tďsďT may be reinterpreted as the adjoint process in the stochastic Pontryagin principle derived for mean field games in [START_REF] Carmona | Probabilistic analysis of mean field games[END_REF] (at least when there is no common noise W 0 ). In that framework, it is worth mentioning that the function pt, x, µq Þ Ñ B x V pt, x, µq reads as the decoupling field of the McKean-Vlasov FBSDE deriving from the stochastic Pontryagin principle. It plays the same role as the gradient of the value function in standard optimal control theory. See Subsection 4.6.

The Case of the Control of McKean-Vlasov Dynamics

When handling the control of McKean-Vlasov dynamics, p χs q tďsďT is chosen as a copy of pX α s q tďsďT . This says that, in (47), α reads as a copy of α so that the minimization in (45) takes the form

inf α E " xbpt, χ, µ, αq, B x Ṽ pt, χ, χqy `bpt, χ, µ, αq ¨Dµ Ṽ pt, χ, χq `f pt, χ, µ, αq ‰ " inf α E 1 " xbpt, χ, µ, αq, B x V pt, χ, µqy `Ẽ 1 " xbpt, χ, µ, αq, B µ V pt, χ, µqp χqy ‰ `f pt, χ, µ, αq ı ,
where the function B µ V pt, x, µqp¨q represents the Fréchet derivative D µ Ṽ pt, x, χq, that is D µ Ṽ pt, x, χq " B µ V pt, x, µqp χq. See the appendix at the end of the paper for explanations. By Fubini's theorem, the minimization can be reformulated as

inf α E 1 " @ bpt, χ, µ, αq, B x V pt, χ, µq `Ẽ 1 " B µ V pt, χ, µqpχq ‰D `f pt, χ, µ, αq ı . ( 55
)
The strategy is then the same as in the previous paragraph. Assume indeed that there exists a measurable mapping ᾱ : r0, T s ˆRd ˆP2 pR d q ˆRd Q pt, x, µq Þ Ñ ᾱpt, x, µ, yq minimizing the reduced Hamiltonian as in (49), then the minimizer in (55) must be

α " ᾱ`t , χ, µ, B x V pt, χ, µq `Ẽ 1 rB µ V pt, χ, µqpχqs " ᾱˆt , χ, µ, B x V pt, χ, µq `żR d B µ V pt, x 1 , µqpχqdµpx 1 q ˙,
showing that αpt, x, µq " ᾱpt, x, µ, B x V pt, x, µq `şR d B µ V pt, x 1 , µqpxqdµpx 1 qq is an optimal feedback. By (46), this permits to make explicit the form of the master equation. Notice that the term in α in (46) does not read as an infimum, namely:

xb `t, x, µ, αpt, x, µq ˘, B x V pt, x, µqy `b`t , χ, µ, αpt, χ, µq ˘¨D µ Ṽ pt, x, χq `f `t, x, µ, αpt, x, µq " inf α " bpt, x, µ, αq ¨Bx Ṽ pt, x, χq `bpt, x, µ, α˘¨D µ Ṽ pt, x, χq `f pt, x, µ, αq ‰ .
This says that the optimal path solving the optimal control of McKean-Vlasov dynamics must be given by:

d Xs " b " s, Xs , μs , ᾱˆs , Xs , μs , B x V ps, Xs , μs q `żR d B µ V ps, x 1 , μs qp Xs qdμ s px 1 q ˙dt `σ`X s ˘dW s `σ0 `X s ˘dW 0 s , (56)
subject to the constraint μs " Lp Xs |F 0 s q for s P rt, T s, with Xt " µ. We indeed claim Proposition 4.3. On the top of the assumptions and notations introduced above, assume that ᾱ, b, σ and σ 0 satisfy (53). Assume also Ṽ is a classical solution of (46) satisfying, for all t P r0, T s,

x P R d and χ P L 2 p Ω1 , F1 , P1 ; R d q, |B x Ṽ pt, x, χq| `}D µ Ṽ pt, x, χq} 2, Ω1 ď C ´1 `|x| `Ẽ 1 " | χ| 2 ‰ 1{2 ¯, (57)
and that, for any initial condition pt, µq P r0, T s ˆP2 pR d q, Equation (56) has a unique solution.

Then, the flow pLp Xs |F 0 s qq tďsďT solves the minimization problem (34), set over controlled McKean-Vlasov dynamics.

Proof. The proof consists again of a verification argument. As for mean field games, we notice from ( 53) and (57) that the supremum (over r0, T s) of the solution of ( 56) is square integrable and that, for any (square integrable) control α, the supremum of X α (with X α t " µ) is also square integrable. The point is then to go back to (38). Replacing g by V pT, ¨, ¨q and applying Itô's formula in the appendix (see Proposition 6.5) (taking benefit of the integrability condition (54) for canceling the expectation of the martingale part) and using the same Fubini argument as in (55), we deduce that the right-hand side is indeed greater than V pt, x, µq. Choosing α s " αps, Xs , μs q, with αpt, x, µq " ᾱpt, x, µ, B x V pt, x, µq `şR d B µ V pt, x 1 , µqpxqdµpx 1 qq, equality must hold.

Remark 4.4. The flow of conditional distributions pμ s " Lp Xs |F 0 s qq tďsďT solves an SPDE, on the same model as [START_REF] Carmona | Forward-Backward Stochastic Differential Equations and Controlled McKean Vlasov Dynamics[END_REF]. The formulation of that SPDE is left to the reader.

Notice finally that pB x V ps, Xs , μs q `şR d B µ V ps, x, μs qp Xs qdμ s pxqq tďsďT may be reinterpreted as the adjoint process in the stochastic Pontryagin principle derived for the control of McKean-Vlasov dynamics in [START_REF] Carmona | Forward-Backward Stochastic Differential Equations and Controlled McKean Vlasov Dynamics[END_REF] (at least when there is no common noise W 0 ). In particular, the function pt, x, µq Þ Ñ B x V ps, x, µq `şR d B µ V ps, x, µqpxqdµpxq reads as the decoupling field of the McKean-Vlasov FBSDE deriving from the stochastic Pontryagin principle for the control of McKean-Vlasov dynamics. It is interesting to notice that the fact that the formula contains two different terms is a perfect reflection of the backward propagation of the terminal condition of the FBSDE. Indeed, as seen in [START_REF] Carmona | Forward-Backward Stochastic Differential Equations and Controlled McKean Vlasov Dynamics[END_REF], this terminal condition has two terms corresponding to the partial derivatives of the terminal cost function g with respect to the state variable x and the distribution µ. See Subsection 4.6.

Viscosity Solutions

In the previous paragraph, we used the master equation within the context of a verification argument to identify optimal paths of the underlying optimal control problem, and we alluded to the connection with purely probabilistic methods derived from the stochastic Pontryagin principle. The stochastic Pontryagin principle works as follows: under suitable conditions, optimal paths are identified with the forward component of a McKean-Vlasov FBSDE. In that framework, our discussion permits to identify the gradient of the function V with the decoupling field of the FBSDE. This FBSDE has the form: # dX s " bps, X s , µ s , αps, X s , µ s , Y s qqds `σ0 pX s qdW 0 s `σpX s qdW s , dY s " ´Ψ`s , X s , µ s , αps, X s , µ s , Y s q ˘ds `Z0 s dW 0 s `Zs dW s , Y T " φpX T , µ T q (58)

for some functions pt, x, ν, αq Þ Ñ Ψpt, x, ν, αq and px, µq Þ Ñ φpx, µq, the McKean-Vlasov nature of the FBSDE being due to the constraints µ s " LpX s |F 0 s q and ν s " LppX s , Y s q|F 0 s q. In the mean field game case, the stochastic Pontryagin principle takes the form Ψpt, x, ν, αq " B x H `t, x, µ, y, α ˘, φpx, µq " B x gpx, µq,

where µ denotes the first marginal of ν, and Ψpt, x, ν, αq " B x H `t, x, µ, y, α ˘`ż

R d ˆRd `Bµ H `t, x 1 , µ, y 1 , α 1 ˘pxq ˘|α 1 " αpt,x 1 ,µ,y 1 q νpdx 1 , dy 1 q, φpx, µq " B x gpx, µq `żR d B µ gpx 1 , µqpxqµpdx 1 q (60)
in the case of the control of McKean-Vlasov dynamics.

One may wonder if a converse to the strategy discussed previously is possible: how could we reconstruct a solution of the master equation from a purely probabilistic approach? Put it differently, given the solution of the McKean-Vlasov FBSDE characterizing the optimal path via the stochastic Pontryagin principle, is it possible to reconstruct V and to prove that it satisfies a PDE or SPDE which we could identify to the master equation?

In the forthcoming paper [START_REF] Chassagneux | McKean-Vlasov FBSDEs and related Master Equation[END_REF], the authors investigate the differentiability of the flow of a McKean-Vlasov FBSDE and manage, in some cases, to reconstruct V as a classical solution of the master equation.

A more direct approach consists in constructing V as a viscosity solution of the master equation. This direct approach was used in [START_REF] Cardaliaguet | Notes on mean field games[END_REF] for non-stochastic games. In all cases the fundamental argument relies on a suitable form of the dynamic programming principle. This was our motivation for the discussion in Subsection 4.2. Still we must remember that Subsection 4.2 remains mostly at the heuristic level, and that a complete proof of the dynamic programming principle in this context would require more work. This is where the stochastic Pontryagin principle may help. If uniqueness of the optimal paths and of the equilibrium are known (see for instance [START_REF] Carmona | Probabilistic analysis of mean field games[END_REF] and [START_REF] Carmona | Forward-Backward Stochastic Differential Equations and Controlled McKean Vlasov Dynamics[END_REF]), then the definition of V in (36) makes sense. In this case, not only do we have the explicit form the optimal paths, but the dynamic programming principle is expected to hold.

We refrain from going into the gory details in this review paper. Instead, we take the dynamic programming principle for granted. The question is then to derive the master equation for V in the viscosity sense, from the three possible versions (43), ( 40) and (41). In the present context, since differentiability with respect to one of the variables is done through a lifting of the functions, we will be using the following definition of viscosity solutions. Definition 4.5. We say that V is a super-solution (resp. sub-solution) in the sense of viscosity of the master equation if whenever pt, x, µq P r0, T sˆR d ˆP2 pR d q and the function r0, T sˆR d ˆP2 pR d q Q ps, y, νq Þ Ñ ϕps, y, νq is continuously differentiable, once in the time variable s, and twice in the variables y and ν, satisfies V pt, x, µq " ϕpt, x, µq and V ps, y, νq ě ϕps, y, νq for all ps, y, νq then we have (45) and/or (46), with Ṽ replaced by φ and " 0 replaced by ď 0 (respectively by ě 0).

The reason why we say and/or might look rather strange. This will be explained below, the problem being actually more subtle than it seems at first. Following the approach used in standard stochastic optimal control problems, the proof could consist in applying Itô's formula to φps, X α s , μs q tďsďt`h . In fact, there is no difficulty in proving the viscosity inequality (46) by means of (40). Still, this result is rather useless as the optimizer α is expected to depend upon the gradient of Ṽ and much more, as α reads as ᾱ applied to the gradient of Ṽ . The question is thus to decide whether it makes sense to replace the gradient of Ṽ in ᾱ by the gradient of φ. To answer the question, we must distinguish the two problems:

1. In the framework of mean field games, the answer is yes. The reason is that, when V is smooth, the inequality V ě ϕ in the neighborhood of pt, x, µq implies B x V pt, x, µq " B x ϕpt, x, µq. This says that we expect φ to satisfy (51) with " 0 replaced by ď 0. Actually, this can be checked rigorously by means of the stronger version (43) of the dynamic programming principle, following the proof in [START_REF] Fleming | Controlled Markov Processes and Viscosity Solutions[END_REF].

2. Unfortunately, this is false when handling the control of McKean-Vlasov dynamics. Indeed, the gradient of V is then understood as B x V pt, x, µq `şR d B µ V pt, x 1 , µqpxqdµpx 1 q, which is 'non-local' in the sense that it involves values of V pt, x 1 , µq for x 1 far away from x. In particular, there is no way one can replace 1 q on the single basis of the comparison of ϕ and V . This implies that, in the optimal control of McKean-Vlasov dynamics, viscosity solutions must be discussed in the framework of (45). Obviously, this requires adapting the notion of viscosity solution as only the function pt, µq Þ Ñ ş R d V pt, x, µqdµpxq matters in the dynamic programming principle (41). Comparison is then done with test functions of the form pt, µq Þ Ñ ş R d φpt, x, µqdµpxq (or simply φpt, µq). The derivation of an inequality in (45) is then achieved by a new application of Itô's formula.

B x V pt, x, µq`ş R d B µ V pt, x 1 , µqpxqdµpx 1 q by B x ϕpt, x, µq`ş R d B µ ϕpt, x 1 , µqpxqdµpx

Comparison of the Two Master Equations

We repeatedly reminded the reader that the function V obtained in the case of mean field games (whether or not there is a common noise) is not a value function in the usual sense of optimal control. Indeed, solving a mean field game problem is finding a fixed point problem more than solving an optimization problem. For this reason, the master equation should not read (and should not be interpreted) as a Hamilton-Jacobi-Bellman equation. Indeed, even though the first terms in Equation (51) are of Hamiltonian type, the extra term A µ (specifically the first order term in A µ ) shows that this equation is not an HJB equation. On the other hand, the previous subsection shows that the master equation for the control of McKean-Vlasov dynamics, which comes from an optimization problem, can be viewed as an HJB equation when put in the form (45). In that case, the solution reads as the value function pt, µq Þ Ñ ş R d V pt, x, µqdµpxq of the corresponding optimization problem.

A Second Example: A Simple Growth Model

The following growth model was introduced and studied in [START_REF] Guéant | Mean field games and applications[END_REF]. We review its main features by recasting it in the framework of the present discussion of the master equation of mean field games with common noise. In fact the common noise W 0 is the only noise of the model since σ " 0 and the idiosyncratic noises do not appear.

Background

As it is the case in many economic models, the problem in [START_REF] Guéant | Mean field games and applications[END_REF] is set for an infinite time horizon (T " 8) with a positive discount rate r ą 0. As we just said, σ " 0. Moreover, the common noise is a one dimensional Wiener process pW 0 t q tě0 . As before, we denote by F 0 " pF 0 t q tě0 its filtration. We also assume that its volatility is linear, that is σ 0 pxq " σx for some positive constant σ, and that each player controls the drift of its state so that bpt, x, µ, αq " α. In other words, the dynamics of the state of player i read:

dX i t " α i t dt `σX i t dW 0 t . (61)
We shall restrict ourselves to Markovian controls of the form α i t " αpt, X i t q for a deterministic function pt, xq Þ Ñ αpt, xq, which will be assumed non-negative and Lipschitz in the variable x. Under these conditions, for any player, say player 1, X 1 t ě 0 at all times t ą 0 if X 1 0 ě 0 and for any two players, say players 1 and 2, the homeomorphism property of Lipschitz SDEs implies that

X 1 t ď X 2 t at all times t ą 0 if X 1 0 ď X 2 0 . Note that in the particular case αpt, xq " γx (62)
for some γ ą 0, then

X 2 t " X 1 t `pX 2 0 ´X1 0 qe pγ´σ 2 {2qt`σW 0 t . (63)
We assume that k ą 0 is a fixed parameter and we introduce a special notation for the family of scaled Pareto distributions with decay parameter k. For any real number q ě 1, we denote by µ pqq the Pareto distribution:

µ pqq pdxq " k q k x k`1 1 rq,8q pxqdx. (64)
Notice that X " µ p1q is equivalent to qX " µ pqq . We shall use the notation µ t for the conditional distribution of the state X t of a generic player at time t ě 0 conditioned by the knowledge of the past up to time t as given by F 0 t . Under the prescription (62), we claim that, if µ 0 " µ p1q , then µ t " µ pqtq where q t " e pγ´σ 2 {2qt`σW 0 t . In other words, conditioned on the history of the common noise, the distribution of the states of the players remains Pareto with parameter k if it started that way, and the left-hand point of the distribution q t can be understood as a sufficient statistic characterizing the distribution µ t . This remark is an immediate consequence of formula (63) applied to X 1 t " q t , in which case q 0 " 1, and X 2 t " X t , implying that X t " X 0 q t . So if X 0 " µ p1q , then µ t " µ pqtq . In particular, we have an explicit solution of the conditional Kolmogorov equation in the case of the particular linear feedback controls.

Optimization Problem

We now introduce the cost functions and define the optimization problem. We first assume that the problem is set for a finite horizon T . For the sake of convenience, we skip the stage of the N player game for N finite, and discuss directly the limiting MFG problem in order to avoid dealing with the fact that empirical measures do not have densities. The shape of the terminal cost g will be specified later on. Using the same notation as in [START_REF] Guéant | Mean field games and applications[END_REF], we define the running cost function f by f px, µ, αq " c

x a rpdµ{dxqpxqs b ´E p α p rµprx, 8qqs b , for some positive constants a, b, c, E and p ą 1 whose economic meanings are discussed in [START_REF] Guéant | Mean field games and applications[END_REF]. We use the convention that the density is the density of the absolutely continuous part of the Lebesgue's decomposition of the measure µ, and that in the above sum, the first term is set to 0 when this density is not defined or is itself 0. The extended Hamiltonian of the system (see (49)) reads Hpx, y, µ, αq " αy `c

x a rpdµ{dxqpxqs b ´E p α p rµprx, 8qqs b and the value ᾱ of α minimizing H is given by (for y ě 0):

ᾱ " ᾱpx, µ, yq " ˆy E " µprx, 8qq ‰ b ˙1{pp´1q (65)
so that:

Hpx, y, µ, ᾱq " ˆy E " µprx, 8qq ‰ b ˙1{pp´1q y `c x a rpdµ{dxqpxqs b ´E p ´py{Eqrµprx, 8qqs b ¯p{pp´1q rµprx, 8qqs b " p ´1 p E ´1{pp´1q y p{pp´1q " µprx, 8qq ‰ b{pp´1q `c x a rpdµ{dxqpxqs b .
In the particular case of linear controls (62), using the explicit formula (64) for the density of µ pqq and the fact that

µ pqq prx, 8qq " 1 ^qk x k , we get f `x, µ pqq , α ˘" c
x a pkq k {x k`1 q b 1 txěqu ´E p α p 1 ^pq kb {x kb q " c k b q kb x a`bpk`1q 1 txěqu ´E pq kb α p `xkb _ q kb ˘, and ᾱpx, µ, yq "

" y E ´qkb x kb ^1¯1{pp´1q , (66) so that
Hpx, y, µ pqq , ᾱq " p ´1 p E ´1{pp´1q y p{pp´1q ´qkb{pp´1q x kb{pp´1q ^1¯`c x a`pk`1qb k b q kb 1 txěqu .

Search for a Pareto Equilibrium

Assuming that the initial distribution of the values of the state is given by the Pareto distribution µ p1q , we now restrict ourselves in searching for equilibriums with Pareto distributions, which means that the description of the equilibrium flow of measures pμ t q 0ďtďT can be reduced to the description of the flow of corresponding Pareto parameters pq t q 0ďtďT . Introducing the letter V for denoting the solution of the master equation, we know from (51) and Proposition 4.1 that the optimal feedback control must read αpt, xq " ᾱ`x , μt , B x V pt, x, μt q ˘" " B x V pt, x, μt q E ´q kb t x kb ^1¯1{pp´1q .

In order to guarantee that the equilibrium flow of measures is of Pareto type, it must satisfy the condition:

γx " ˆBx V pt, x, μt q E qkb t x kb ˙1{pp´1q , x ě qt . ( 67
)
for some γ ą 0. There is no need for checking the condition for x ă qt as the path driven by the Pareto distribution is then always greater than or equal to pq t q tě0 . Since we focus on equilibriums of Pareto type, we compute the function V at distributions of Pareto type only. It then makes sense to parameterize the problem and to seek for V in the factorized form:

Vpt, x, qq " V pt, x, µ pqq q, for some function V : pt, x, qq P r0, T s ˆR ˆR Ñ R. Then, the relationship (67) takes the form:

γx " ˆBx Vpt, x, qq E q kb x kb ˙1{pp´1q , x ě q.
The point is then to write the equation satisfied by V, namely the equivalent of (51) but satisfied by V instead of V . First, we observe that, in (51), σpxq " 0. Obviously, the difficult point is to rewrite A µ and A xµ as differential operators acting on the variables q and px, qq respectively.

A natural solution is to redo the computations used for deriving (51) by replacing Itô's formula for the measures pμ t q 0ďtďT by Itô's formula for pq t q 0ďtďT , taking benefit that pq t q 0ďtďT solves the SDE dq t " γ qt dt `σ qt dW t ,

which is a consequence of (62) and (63). Then the term A µ Ṽ in (51), which reads as the Itô expansion of V along pμ t q 0ďtďT , turns into the second-order differential operator associated to the SDE satisfied by qt , namely A q Vpt, x, qq " γqB q Vpt, x, qq `1 2 σ 2 q 2 B 2 q Vpt, x, qq.

Similarly, the term A xµ Ṽ in (51), which reads as the bracket of the components in R d and in P 2 pR d q in the Itô expansion, turns into the second-order differential operator associated to bracket of the SDEs satisfied by pX t q 0ďtďT in (61) and by pq t q 0ďtďT , namely which always admits a solution if ppp ´1q ă bk. The fact that (69) is satisfied for x ě q is enough to prove that ˆVp Xt , qt q `ż t 0 f `X s , μs , γ Xs ˘ds ˙0ďtďT , with μs " µ pqsq for s P r0, T s, is a martingale, whenever d Xt " γ Xt dt `σ Xt dW 0 t , t P r0, T s, with X0 " µ q0 , and pq t q 0ďtďT also solves (68). The reason is that Xt ą qt for any t P r0, T s (equality Xt " qt holds along scenarios for which X0 " q0 , which are of zero probability).

A xq Vpt, x,
The martingale property is a part of the verification Proposition 4.1 for proving the optimality of p Xt q 0ďtďT when pμ t q 0ďtďT is the flow of conditional measures, but this is not sufficient. We must evaluate V along a pair pX t , qt q 0ďtďT , pX t q 0ďtďT denoting a general controlled process satisfying (61). Unfortunately, things then become more difficult as X t might not be larger than qt . In other words, we are facing the fact that V satisfies the PDE (69) on the set tx ě qu only. In order to circumvent this problem, a strategy consists in replacing V by Vpx, qq " Bx p ´xbk q bk ^1¯,

for the same constant B as above. Obviously, the PDE (69) is not satisfied when x ă q, but V defines a subsolution on the set t0 ď x ă qu, as (69) holds but with " 0 replaced by ě 0. Heuristically, this should show that

ˆVpX t , qt q `ż t 0 f `Xs , μs , α s ˘ds ˙0ďtďT (71)
is a submartingale when pX t q 0ďtďT is an arbitrary controlled process driven by the control pα t q 0ďtďT . Still, the justification requires some precaution as the function V is not C 2 (which is the standard framework to apply Itô's expansion), its first-order derivatives being discontinuous on the diagonal tx " qu. The argument for justifying the Itô expansion is a bit technical so that we just give a sketchy proof of it. Basically, we can write VpX t , qt q " BpX t q p rϕpX t {q t qs bk , with ϕprq " minp1, rq.

The key point is that pX t {q t q 0ďtďT is always a bounded variation process, so that the expansion of pφpX t {q t qq 0ďtďT , for some function φ, only requires to control φ 1 and not φ 2 . Then, we can regularize ϕ by a sequence pϕ n q ně1 such that pϕ n q 1 prq " 0, for r ď 1 ´1{n, pϕ n q 1 prq " 1, for r ě 1 and pϕ n q 1 prq P r0, 1s for r P r1 ´1{n, 1s. The fact that pϕ n q 1 prq is uniformly bounded in n permits to expand pBpX t q p rϕ n pX t {q t qs bk q 0ďtďT and then to pass to the limit. The submartingale property shows that ż R d

Vpx, q0 qdµ q0 pxq ď inf

pαtq 0ďtďT "ż T 0 f pX t , qt , α t qdt `VpX T , qT q  , (72)
which, together with the martingale property along p Xt q 0ďtďT , shows that equality holds and that the Pareto distributions pμ t q 0ďtďT form a MFG equilibrium, provided g is chosen as V. This constraint on the choice of g can be circumvented by choosing T " 8, as done in [START_REF] Guéant | Mean field games and applications[END_REF], in which case f must be replaced by e ´rt f for some discount rate r ą 0. The analysis in the case T " 8 can be done in the following way. In the proof of the martingale and submartingale properties, V must replaced by e ´rt V. Plugging e ´rt V and e ´rt f in (69) instead of V and f , we understand that V must now satisfy (69) but with an additional ´rV in the lefthand side. Then, we can repeat the previous argument in order to identify the value of B in (70). Finally, if r is large enough, Ere ´rT Vp XT , qT qs tends to 0 as T tends to the infinity in the martingale property (71). Similarly, if we restrict ourselves to a class of feedback controls with a suitable growth, Ere ´rT VpX T , qT qs tends to 0 in (72), which permits to conclude.

Control of McKean-Vlasov Equations

A similar framework could be used for considering the control of McKean-Vlasov equations. The analog of the strategy exposed in the previous paragraph would consist in limiting the optimization procedure to controlled processes in (61) driven by controls pα t q 0ďtďT of the form pα t " γ t X t q 0ďtďT for some deterministic pγ t q 0ďtďT . Using an obvious extension of (63), this would force the conditional marginal distributions of pX t q 0ďtďT to be Pareto distributed. Exactly as above, this would transform the problem into a finite dimensional problem. Precisely, this would transform the problem into a finite dimensional optimal control problem. In that perspective, the corresponding master equation could be reformulated as an HJB equation in finite dimension. In comparison with, we emphasize, once again, that the master equation (69) for the mean field game is not a HJB equation.

Appendix: A Generalized Form of Itô's Formula

Our derivation of the master equation requires the use of a form of Itô formula in a space of probability measures. This subsection is devoted to the proof of such a formula.

Notion of Differentiability

In Section 4, we alluded to a specific notion of differentiability for functions of probability measures. The choice of this notion is dictated by the fact that 1) the probability measures we are dealing with appear as laws of random variables; 2) in trying to differentiate functions of measures, the infinitesimal variations which we consider are naturally expressed as infinitesimal variations in the linear space of those random variables. The relevance of this notion of differentiability was argued by P.L. Lions in his lectures at the Collège de France [START_REF] Lions | Théorie des jeux à champs moyen et applications[END_REF]. The notes [START_REF] Cardaliaguet | Notes on mean field games[END_REF] offer a readable account, and [START_REF] Carmona | Forward-Backward Stochastic Differential Equations and Controlled McKean Vlasov Dynamics[END_REF] provides several properties involving empirical measures. It is based on the lifting of functions P 2 pR d q Q µ Þ Ñ Hpµq into functions H defined on the Hilbert space L 2 p Ω; R d q over some probability space p Ω, F , Pq by setting Hp Xq " HpLp Xqq, for X P L 2 p Ω; R d q, Ω being a Polish space and P an atomless measure.

Then, a function H is said to be differentiable at µ 0 P P 2 pR d q if there exists a random variable X0 with law µ 0 , in other words satisfying Lp X0 q " µ 0 , such that the lifted function H is Fréchet differentiable at X0 . Whenever this is the case, the Fréchet derivative of H at X0 can be viewed as an element of L 2 p Ω; R d q by identifying L 2 p Ω; R d q and its dual. It turns out that its distribution depends only upon the law µ 0 and not upon the particular random variable X0 having distribution µ 0 . See Section 6 in [START_REF] Cardaliaguet | Notes on mean field games[END_REF] for details. This Fréchet derivative rD Hsp X0 q is called the representation of the derivative of H at µ 0 along the variable X0 . It is shown in [START_REF] Cardaliaguet | Notes on mean field games[END_REF] that, as a random variable, it is of the form hp X0 q for some deterministic measurable function h : R d Ñ R d , which is uniquely defined µ 0 -almost everywhere on R d . The equivalence class of h in L 2 pR d , µ 0 q being uniquely defined, it can be denoted by B µ Hpµ 0 q (or BHpµ 0 q when no confusion is possible). It is then natural to call B µ Hpµ 0 q the derivative of H at µ 0 and to identify it with a function B µ Hpµ 0 qp ¨q :

R d Q x Þ Ñ B µ Hpµ 0 qpxq P R d .
This procedure permits to express rD Hsp X0 q as a function of any random variable X0 with distribution µ 0 , irrespective of where this random variable is defined. Remark 6.1. Since it is customary to identify a Hilbert space to its dual, we will identify L 2 p Ωq with its dual, and in so doing, any derivative D Hp Xq will be viewed as an element of L 2 p Ωq. In this way, the derivative in the direction Ỹ will be given by the inner product rD Hp Xqs ¨Ỹ . Accordingly, the second Frechet derivative D 2 Hp Xq which should be a linear operator from L 2 p Ωq into itself because of the identification with its dual, will be viewed as a bilinear form on L 2 p Ωq. In particular, we shall use the notation D 2 Hp Xqr Ỹ ¨Zs for `rD 2 Hp Xqsp Ỹ q ˘¨Z . Remark 6.2. The following result (see [START_REF] Carmona | Forward-Backward Stochastic Differential Equations and Controlled McKean Vlasov Dynamics[END_REF] for a proof ) gives, though under stronger regularity assumptions on the Fréchet derivatives, a convenient way to handle this notion of differentiation with respect to probability distributions. If the function H is Fréchet differentiable and if its Fréchet derivative is uniformly Lipschitz (i.e. there exists a constant c ą 0 such that }D Hp Xq´D Hp X1 q} ď c| X ´X 1 | for all X, X1 in L 2 p Ωq), then there exists a function B µ H P 2 pR d q ˆRd Q pµ, xq Þ Ñ B µ Hpµqpxq such as |B µ Hpµqpxq ´Bµ Hpµqpx 1 q| ď c|x ´x1 | for all x, x 1 P R d and µ P P 2 pR d q, and for every µ P P 2 pR d q, B µ Hpµqp Xq " D Hp Xq almost su if µ " Lp Xq.

Itô's Formula along a Flow of Conditional Measures

In the derivation of the master equation, the value function is expanded along a flow of conditional measures. As already explained in Subsection 4.3, this requires a suitable construction of the lifting.

Throughout this section, we assume that pΩ, F, Pq is of the form pΩ 0 ˆΩ1 , F 0 b F 1 , P 0 b P 1 q, pΩ 0 , F 0 , P 0 q supporting the common noise W 0 , and pΩ 1 , F 1 , P 1 q the idiosyncratic noise W . So an element ω P Ω can be written as ω " pω 0 , ω 1 q P Ω 0 ˆΩ1 , and functionals Hpµpω 0 qq of a random probability measure µpω 0 q P P 2 pR d q with ω 0 P Ω 0 , can be lifted into Hp Xpω 0 , ¨qq " HpLp Xpω 0 , ¨qqq, where Xpω 0 , ¨q is an element of L 2 p Ω1 , F1 , P 1 ; R d q with µpω 0 q as distribution, p Ω1 , F1 , P1 q being Polish and atomless. Put it differently, the random variable X is defined on p Ω " Ω 0 ˆΩ 1 , F " F 0 b F1 , P " P 0 b P1 q.

The objective is then to expand p Hp χt pω 0 , ¨qqq 0ďtďT , where p χt q 0ďtďT is the copy so constructed, of an Itô process on pΩ, F, Pq of the form:

χ t " χ 0 `ż t 0 β s ds `ż t 0 ż Ξ ς 0 s,ξ W 0 pdξ, dsq `ż t 0 ς s dW s ,
for t P r0, T s, assuming that the processes pβ t q 0ďtďT , pς t q 0ďtďT and pς 0 t,ξ q 0ďtďT,ξPΞ are progressively measurable with respect to the filtration generated by W and W 0 and square integrable, in the sense that

E ż T 0 ˆ|β t | 2 `|ς t | 2 `żΞ |ς 0 t,ξ | 2 dνpξq ˙dt ă `8. (73)
Denoting by p Wt q 0ďtďT , p βt q 0ďtďT , pς t q 0ďtďT and pς 0 t,ξ q 0ďtďT,ξPΞ the copies of pW t q 0ďtďT , pβ t q 0ďtďT , pς t q 0ďtďT and pς 0 t,ξ q 0ďtďT,ξPΞ , we then have

χt " χ0 `ż t 0 βs ds `ż t 0 ż Ξ ς0 s,ξ W 0 pdξ, dsq `ż t 0 ςs d Ws ,
for t P r0, T s. In this framework, we emphasize that it makes sense to look at Hp χt pω 0 , ¨qq, for t P r0, T s, since

E 0 Ẽ1 " sup 0ďtďT | χt | 2 ‰ " E 0 E 1 " sup 0ďtďT |χ t | 2 ‰ ă `8,
where E 0 , E 1 and Ẽ1 are the expectations associated to P 0 , P 1 and P1 respectively. In order to simplify notations, we let χt pω 0 q " χt pω 0 , ¨q for t P r0, T s, so that p χt q 0ďtďT is L 2 p Ω1 , F1 , P1 ; R d q-valued, P 0 almost surely. Similarly, we let βt pω 0 q " βt pω 0 , ¨q, ςt pω 0 q " ςt pω 0 , ¨q ςt,ξ pω 0 q " ςt,ξ pω 0 , ¨q, for t P r0, T s and ξ P Ξ. We then claim Proposition 6.3. On the top of the assumption and notation introduced right above, assume that H is twice continuously Fréchet differentiable. Then, we have P 0 almost surely, for all t P r0, T s,

H`χ t ˘" H`χ 0 ˘`ż t 0 D H`χ s ˘¨β s ds `ż t 0 ż Ξ D H`χ s ˘¨ς 0 s,ξ W 0 pdξ, dsq `1 2 ż t 0 ˆD2 Hp χs q " ςs G, ςs G‰ `żΞ D 2 H`χ s ˘"ς 0 s,ξ , ς0 s,ξ ‰ dνpξq ˙ds. ( 74
)
where G is an N p0, 1q-distributed random variable on p Ω1 , F1 , P1 q, independent of p Wt q tě0 . Remark 6.4. Following Remark 6.2 above, one can specialize Itô's formula to a situation with smoother derivatives. See [START_REF] Chassagneux | McKean-Vlasov FBSDEs and related Master Equation[END_REF] for a more detailed account. Indeed, if one assumes that 1. the function H is C 1 in the sense given above and its first derivative is Lipschitz; 2. for each fixed x P R d , the function µ Þ Ñ B µ Hpx, µq is differentiable with Lipschitz derivative, and consequently, there exists a function pµ, x 1 , xq Þ Ñ B 2 µ Hpx, µqpx 1 q P R dˆd which is Lipschitz in x 1 uniformly with respect to x and µ and such that B 2 µ Hpx, µqp Xq gives the Fréchet derivative of µ Þ Ñ B µ Hpx, µq for every x P R d as long as Lp Xq " µ; 3. for each fixed µ P P 2 pR d q, the function x Þ Ñ B µ Hpx, µq is differentiable with Lipschitz derivative, and consequently, there exists a bounded function px, µq Þ Ñ B x B µ Hpx, µq P R dˆd giving the value of its derivative.

Then, the second order term appearing in Itô's formula can be expressed as the sum of two explicit operators whose interpretations are more natural. Indeed, the second Fréchet derivative D 2 Hp Xq can be written as the linear operator Ỹ Þ Ñ A Ỹ on L 2 p Ω1 , F1 , P 1 ; R d q defined by rA Ỹ spω 1 q " ż Ω1 B 2 µ H `Xpω 1 q, Lp Xq ˘`X 1 pω 1 q ˘Ỹ pω 1 q d P1 pω 1 q `Bx B µ H `Lp Xq, Xpω 1 q ˘Ỹ pω 1 q.

The derivation of the master equation actually requires a more general result than Proposition 6.3. Indeed one needs to expand p HpX t , χt qq 0ďtďT for a function H of px, Xq P R d L2 p Ω1 , F1 , P1 ; R d q. As before, p χt q 0ďtďT is understood as p χt pω 0 , ¨qq 0ďtďT . The process pX t q 0ďtďT is assumed to be another Itô process, defined on the original space pΩ, F, Pq " pΩ 0 ˆΩ1 , F 0 b F 1 , P 0 b P 1 q, with dynamics of the form

X t " X 0 `ż t 0 b s ds `ż t 0 ż Ξ σ 0 s,ξ W 0 pdξ, dsq `ż t 0 σ s dW s ,
for t P r0, T s, the processes pb t q 0ďtďT , pσ t q 0ďtďT and pσ 0 t,ξ q 0ďtďT,ξPΞ being progressively-measurable with respect to the filtration generated by W and W 0 , and square integrable as in (73). Under these conditions, the result of Proposition 6.3 can be extended to: Proposition 6.5. On the top of the above assumptions and notations, assume that H is twice continuously Fréchet differentiable on R d ˆL2 p Ω1 , F1 , P1 ; R d q. Then, we have P almost surely, for all t P r0, T s, where G is an N p0, 1q-distributed random variable on p Ω1 , F1 , P1 q, independent of p Wt q tě0 . The partial derivatives in the infinite dimensional component are denoted with the index 'µ'. In that framework, the term xB x D µ HpX s , χs q ¨ς 0 s,ξ , σ 0 s,ξ y reads

H
d ÿ i"1
tB x i D µ HpX s , χs q ¨ς 0 s,ξ u `σ0 s,ξ ˘i.

Proof of Itô's Formula

We only provide the proof of Proposition 6.3 as the proof of Proposition 6.5 is similar.

By a standard continuity argument, it is sufficient to prove that Equation (74) holds for any t P r0, T s P 0 -almost surely. In particular, we can choose t " T . Moreover, by a standard approximation argument, it is sufficient to consider the case of simple processes pβ t q 0ďtďT , pς t q 0ďtďT and pς 0 t,ξ q 0ďtďT,ξ of the form

β t " M ´1 ÿ i"0
β i 1 rτ i ,τ i`1 q ptq, ς t " M ´1 ÿ i"0 ς i 1 rτ i ,τ i`1 q ptq, ς 0 t,ξ " M ´1 ÿ i"0 N ÿ j"1 ς 0 i,j 1 rτ i ,τ i`1 q ptq1 A j pξq, where M, N ě 1, 0 " τ 0 ă τ 1 ă ¨¨¨ă τ M " T , pA j q 1ďjďN are piecewise disjoint Borel subsets of Ξ and pβ i , ς i , ς 0 i,j q 1ďjďN are bounded F τ i -measurable random variables. The strategy is taken from [START_REF] Chassagneux | McKean-Vlasov FBSDEs and related Master Equation[END_REF] and consists in splitting Hp χT q ´Hp χ0 q into Hp χT q ´Hp χ0 q " K´1 ÿ k"0 `Hp χt k`1 q ´Hp χt k q ˘, where 0 " t 0 ă ¨¨¨ă t K " T is a subdivision of r0, T s of step h such that, for any k P t0, . . . , K 1u, there exists some i P t0, . . . , M ´1u such that rt k , t k`1 q Ă rτ i , τ i`1 q. We then start with approximating a general increment Hp χt k`1 q ´Hp χt k q, omitting to specify the dependence upon ω 0 . By Taylor's formula, we know that we can find some δ P r0, 1s such that Hp χt k`1 q ´Hp χt k q " D Hp χt k q ¨p χt k`1 ´χ t k q `1 2 D 2 H`χ t k `δp χt k`1 ´χ t k q ˘`χ (75)

By Kolmogorov continuity theorem, we know that, P 0 almost surely, the mapping r0, T s Q t Þ Ñ χt P L 2 p Ω1 , F1 , P1 ; R d q is continuous. Therefore, P 0 almost surely, the mapping ps, t, δq Þ Ñ D 2 Hp χt δp χs ´χ t qq is continuous from r0, T s 2 ˆr0, 1s to the space of bounded operators from L 2 p Ω1 , F1 , P1 ; R d q into itself, which proves that, P Assume that, for some 0 ď i ď M ´1, τ i ď t k ă t k`1 ď τ i`1 . Then, D Hp χt k q ¨ż t k`1 t k βs pω 0 , ¨qds " `tk`1 ´tk ˘D Hp χt k q ¨β t k pω 0 , ¨q.

Note that the right-hand side is well-defined as β t k is bounded. Similarly, we notice that D Hp χt k q ¨"ˆż t k`1 t k ςs d Ws ˙pω 0 , ¨q " `tk`1 ´tk ˘D Hp χt k q ¨"ς t k pω 0 , ¨q`W t k`1 ´W t k ˘‰. Now, using the specific form of D H, D Hp χt k pω 0 qq " pω 1 Þ Ñ B µ Hp χt k pω 0 , ω1 qqq appears to be a Ft k -measurable random variable, and as such, it is orthogonal to ςt k pω 0 , ¨qp Wt k`1 ´W t k q, which shows that D Hp χt k q ¨"ˆż s,ξ W 0 pdξ, dsq ˙pω 0 , ¨q " D Hp χt k q ¨" N ÿ j"1 ς0 i,j pω 0 , ¨qW 0 `Aj ˆrt k , t k`1 q ˘pω 0 q  . Now, W 0 `Aj ˆrt k , t k`1 q ˘pω 0 q behaves as a constant in the linear form above. Therefore,

D Hp χt k q ¨"ˆż t k`1 t k ż Ξ ς0
s,ξ W 0 pdξ, dsq ˙pω 0 , ¨q " N ÿ j"1 D Hp χt k q ¨ς 0 i,j pω 0 , ¨qW 0 `Aj ˆrt k , t k`1 q ˘pω 0 q " "ż t k`1

  qq " σ 2 xqB 2 xq Vpt, x, qq.

	Now we look for a constant B ą 0 such that
							Vpt, x, qq " Vpx, qq " B	x p`bk q bk ,	(70)
	solves the parameterized master equation (69) on the set tx ě qu. Under the additional condition that a `b " p, B must be the solution of the equation
	p	´1 p	E ´1{pp´1q `Bpp `bkq ˘p{pp´1q `c k b ´γBbk	`σ2 2	Bppp ´1q " 0.
	The condition (67) reads
							γ "	´Bpp `bkq E	¯1{pp´1q	,
	so that the above equation for B becomes
	pp `bkq 1{pp´1q E ´1{pp´1q `p ´1	´bk p	˘Bp{pp´1q `σ2 2	ppp ´1qB	`c k b " 0.
	Rewriting (51), we get
	B t Vpt, x, qq	`p	´1 p	E ´1{pp´1q `Bx Vpt, x, qq ˘p{pp´1q ´qkb{pp´1q x kb{pp´1q	^1¯`c x a`pk`1qb k b q kb 1 txěqu	(69)
	`γqB q Vpt, x, qq	`1 2	σ 2 " x 2 B 2 x Vpt, x, qq `q2 B 2 q Vpt, x, qq `2xqB 2 xv Vpt, x, qq ‰	" 0.

  t k`1 ´χ t k , χt k`1 ´χ t k " D Hp χt k q ¨p χt k`1 ´χ t k q `1 2 D 2 Hp χt k q `χ t k`1 ´χ t k , χt k`1 ´χ t k "D 2 H`χ t k `δp χt k`1 ´χ t k q ˘´D 2 H`χ

	˘‰`χ
	t k

t k`1 ´χ t k , χt k`1 ´χ t k ˘.

  0 almost surely, ~¨~2 , Ω1 denoting the operator norm on the space of bounded operators onL 2 p Ω1 , F1 , P1 ; R d q. Now, H`χ t k `δp χt k`1 ´χ t k q ˘´D 2 H`χ t k ˘‰`χ t k`1 ´χ t k , χt k`1 ´χ t k } χt k`1 ´χ t k } 2 L 2 p Ωq . } χt k`1 ´χ t k } 2 H`χ t k `δp χt k`1 ´χ t k q ˘´D 2 H`χ t k ˘‰ ¨`χ t k`1 ´χ t k , χt k`1´χ t k probability as h tends to 0. We now compute the various terms appearing in (75). We write D Hp χt k q ¨p χt k`1 ´χ t k q " D Hp χt k q ¨ż t k`1 W 0 pdξ, dsq ˙pω 0 , ¨q `D Hp χt k q ¨"ˆż t k`1 t k ςs d Ws ˙pω 0 , ¨q.

	Since						" K´1		K´1
							E 0	ÿ k"0	L 2 p Ωq	ď C	ÿ k"0	`tk`1 ´tk ˘ď CT,
	we deduce that
		ˇˇˇK	´1
			ÿ	"	D 2 ˘ˇˇˇÑ	0	(76)
			k"0
	in P 0 t k	βs pω 0 , ¨qds
	`D Hp χt k q	¨"ˆż t k`1 t k	ż Ξ	ς0 s,ξ
				lim hOE0	sup s,tPr0,T s,|t´s|ďh	sup δPr0,1s	~D2 H`χ
	ˇˇˇK	´1
	ÿ	" D 2 ˘ˇˇď
	k"0
							K´1
		sup s,tPr0,T s,|t´s|ďh	sup δPr0,1s	~D2 H`χ	t `δp χs ´χ t q ˘´D 2 H`χ	t ˘~2, Ω1	ÿ k"0

t `δp χt`h ´χ t q ˘´D 2 H`χ t ˘~2, Ω1 " 0,

  t k`1

			t k	ςs d Ws ˙pω 0 ,	¨q	" 0.	(78)
	Finally,
	D Hp χt k q	t k ¨"ˆż t k`1	ż

Ξ ς0

  `rt k , t k`1 q ˆAj ˘W 0 `rt k , t k`1 q ˆAj 1 ż probability as h tends to 0. It remains to computeD 2 Hp χt k q " ςt k pω 0 , ¨q`W t k`1 ´W t k ˘, ςt k pω 0 , ¨q`W t k`1 ´W t k ˘‰. k pω 0 , ¨q `ες t k pω 0 , ¨qp Wt k`1 ´W t k q H`χ t k pω 0 , ¨q ´ες t k pω 0 , ¨qp Wt k`1 ´W t k q ˘´2 H`χ t k pω 0 , ¨q˘‰ , k pω 0 , ¨q `ες t k pω 0 , ¨qa t k`1 ´tk G˘´H`χ t k pω 0 , ¨q˘‰ ,where G is independent of p Wt q 0ďtďT , and N p0, 1q distributed. Therefore,D 2 Hp χt k q " ςt k `W t k`1 ´W t k ˘, ςt k `W t k`1 ´W t k ˘‰ " `tk`1 ´tk ˘D2 Hp χt k q " ςt k G, σt k G‰ , Hp χt k q " ςt k `W t k`1 ´W t k ˘, ςt k `W t k`1 ´W t k ˘‰ Ñ

	in P 0 probability as h tends to 0. Noticing that
	K´1 ÿ k"0	N ÿ j"1	D 2 Hp χt k q "	ς0 i,j , ς0 i,j	‰`t	k`1 ´tk ˘νpA j q "	K´1 ÿ k"0	ż t k`1 t k	ż	Ξ	D 2 Hp χt k q "	ς0 s,ξ , ς0 s,ξ	‰	dνpξqds,
	we deduce that
			K´1			N
			ÿ k"0	ÿ j,j 1 "1 ż D 2 Hp χt k q " W 0 T ς0 i,j , ς0 i,j 1 ‰ 0 Ξ D 2 Hp χs q " ς0 s,ξ , ς0 s,ξ ‰ dνpξqds Ñ 0
	in P 0 Recall that this is the limit
			lim εÑ0	1 ε 2	" H`χ
	which is the same as
				lim εÑ0	1 ε 2	" H`χ
	which is enough to prove that
	K´1 ÿ k"0	D 2 ż T 0	D 2 Hp χs q "	ςs G, ςs	G‰ ds
	in P
															(79)
											ż				
									t k	Ξ	D Hp χt k q ¨ς 0 s,ξ pω 0 ,	¨q(	W pdξ, dsq	pω 0 q.

t t 0 probability as h tends to 0.

Therefore, in analogy with (76), we deduce from (77), ( 78) and (79) that K´1 ÿ k"0 D Hp χt k q ¨p χt k`1 ´χ t k q Ñ ż T 0 D Hp Xs q ¨β s ds `ż T 0 ż Ξ D Hp χs q ¨ς 0 s,ξ ( W pdξ, dsq, in P 0 probability as h tends to 0.

We now reproduce this analysis for the second order derivatives. We need to compute:

Clearly, the drift has very low influence on the value of Γ k . Precisely, for investigating the limit (in

We first notice that

the reason being that

D H`χ t k `ǫς 0 i,j pω 0 , ¨qW 0 `rt k , t k`1 q ˆAj ˘pω 0 q D Hp χt k q ‰" ςt k pω 0 , ¨q`W t k`1 ´W t k ˘‰, which is zero by the independence argument used in (78). Following the proof of (79),

i,j pω 0 , ¨qW 0 `rt k , t k`1 q ˆAj ˘pω 0 q, N ÿ j"1 ς0 i,j pω 0 , ¨qW 0 `rt k , t k`1 q ˆAj ˘pω 0 q ı " N ÿ j,j 1 "1 D 2 Hp χt k q " ς0 i,j pω 0 , ¨q, ς0 i,j 1 pω 0 , ¨q‰ W 0 `rt k , t k`1 q ˆAj ˘pω 0 qW 0 `rt k , t k`1 q ˆAj 1 ˘pω 0 q.

The second line reads as a the bracket of a discrete stochastic integral. Letting ς0 i,j pω 0 q " ς0 i,j pω 0 , ¨q, it is quite standard to check

D 2 Hp χt k q " ς0 i,j , ς0 i,j ‰`t k`1 ´tk ˘νpA j q Ñ 0