Marco Masoero

On the long time convergence of potential MFG

We look at the long time behavior of potential Mean Field Games (briefly MFG) using some standard tools from weak KAM theory. We first show that the time-dependent minimization problem converges to an ergodic constant -λ, then we provide a class of examples where the value of the stationary MFG minimization problem is strictly greater than -λ. This will imply that the trajectories of the time-dependent MFG system do not converge to static equilibria.

Introduction

Mean Field Games were first introduced by Lasry and Lions [START_REF] Lasry | Jeux à champ moyen. i-le cas stationnaire[END_REF][START_REF]Jeux à champ moyen. ii-horizon fini et contrôle optimal[END_REF] and, simultaneously, by Huang, Caines and Malamhé [START_REF] Huang | Large population stochastic dynamic games: closed-loop mckean-vlasov systems and the nash certainty equivalence principle[END_REF]. This theory is a branch of the broader theory of Dynamic Games and it is devoted to those models where infinitely many players interact strategically with each other.

In many cases the Nash equilibria of those games can be analyzed through the solutions of the, so called, MFG system

     -∂ t u -u + H(x, Du) = F (x, m) in R d × [0, T ] -∂ t m + m + div(mD p H(x, Du)) = 0 in R d × [0, T ] m(t) = m 0 , u(T, x) = u T (x) in R d .
with unknown the couple (u, m). We can think at m(t) as the distribution of players at time t and u(t, x) as the value function of any infinitesimal player starting from x at time t.

The aim of this paper is to shed some light on the long time behavior of potential MFG when monotonicity is not in place. The long time behavior and the existence of solutions which are periodic in time have been subject of several papers starting from [START_REF] Lions | Mean-field games[END_REF] and the Mexican wave model in [START_REF] Guéant | Mean field games and applications[END_REF] to more recent results in [START_REF]Long time average of first order mean field games and weak kam theory[END_REF][START_REF] Cardaliaguet | Long time average of mean field games with a nonlocal coupling[END_REF][START_REF] Cardaliaguet | Long time average of mean field games[END_REF][START_REF] Gomes | Discrete time, finite state space mean field games[END_REF], but in these papers either monotonicity was assumed or the MFG was not of potential type. Cirant and Nurbekyan were the first to recently provide some results in the direction of periodic solutions for non monotone MFG. Cirant in [START_REF] Cirant | On the existence of oscillating solutions in non-monotone mean-field games[END_REF] suggested the existence of non monotone configurations under which oscillatory behaviors were to be expected. Afterwards in [START_REF] Cirant | The variational structure and time-periodic solutions for mean-field games systems[END_REF], with Nurbekyan, they proved, through bifurcation methods, the existence of a path of branches which corresponds to a periodic trajectory. The main difference with our work is the choice of the class of solutions. In our case we look at paths which are energy minimizers whereas, in their, it might not be the case.

Potential MFG are those games whose MFG system can be derived as optimality condition of the following minimization problem

U(T, m 0 ) = inf (m,w) T 0 T d H * x, - dw(t) dm(t) (x) dm(t) + F(m(t))dt,
where (m, w) verifies the Fokker Plank equation -∂ t m + m + divw = 0 with m(0) = m 0 and the coupling function F in the MFG system is the derivative with respect to the measure of F. These games have been largely studied (see Lasry and Lions [START_REF]Jeux à champ moyen. ii-horizon fini et contrôle optimal[END_REF] for existence results and, among others [START_REF] Cardaliaguet | Second order mean field games with degenerate diffusion and local coupling[END_REF][START_REF] Briani | Stable solutions in potential mean field game systems[END_REF][START_REF] Mészáros | On the variational formulation of some stationary secondorder mean field games systems[END_REF] for further properties) but, so far, not much is known regarding their long time behavior outside the assumption of monotonicity, where Cardaliaguet, Lasry, Lions and Porretta [START_REF] Cardaliaguet | Long time average of mean field games with a nonlocal coupling[END_REF] proved the convergence to the ergodic system

-λ -u + H(x, Du) = F (x, m) in T d m + div(mD p H(x, Du)) = 0 in T d .
We show that in general this is not the case, even in the very regular setting of non local coupling. We look at the problem from the point of view of weak KAM theory. The link between the two theories is not new and it was already proposed by Cardaliaguet [START_REF]Long time average of first order mean field games and weak kam theory[END_REF] in the first order monotone case, even though in a different manner.

The paper is divided in three sections. In the first one we prove the convergence of T -1 U(T, •) when T goes to infinity. The method we use is directly inspired by Lions, Papanicolaou, and Varadhan [START_REF] Lions | Homogenization of hamilton-jacobi equations[END_REF]: instead of looking directly at lim T T -1 U we define the infinite horizon, discounted problem

V δ (m 0 ) = inf (m,w) ∞ 0 e -δt T d H * x, - w(t) m(t) dm(t) + F(m(t))dt
and we prove that lim δ δV δ (•) = -λ when δ → 0 + and that this limit is uniform with respect to the initial distribution. A key assumption is the boundedness of the second derivative of F (x, m) with respect to the state variable. This gives uniform semiconcavity estimates of the solutions of the MFG system associated to the discounted minimization problem. The existence of the limit lim δ δV δ (•) implies the existence of the limit lim T T -1 U and the two must coincide.

As byproduct, we have the existence of a corrector function χ on the space of measures which enjoys the following dynamic programming principle

χ(m 0 ) = inf (m,w) t 2 t 1 T d H * x, - w(t) m(t) dm(s) + F(m(s))ds + χ(m(t 2 )) + λ(t 2 -t 1 ).
The second section is devoted to the study of the set of corrector functions. A corrector is any continuous function on the space of measures which verifies the dynamic programming principle above. Both the terminology and the techniques are borrowed from weak KAM theory, in particular we rely on Fathi's book [START_REF]Weak kam theorem in lagrangian dynamics preliminary version number[END_REF], along with his seminal papers [START_REF] Fathi | Solutions kam faibles conjuguées et barrieres de peierls[END_REF][START_REF]Théoreme kam faible et théorie de mather sur les systemes lagrangiens[END_REF][START_REF]Sur la convergence du semi-groupe de lax-oleinik[END_REF]. In principle, as in the standard weak KAM theory, the corrector functions verify an HJB equation in the space of probability measure. In this work nothing is said about this property which is the subject of a paper that is still in progress.

Particular interest is given to the projected Mather set which is the set of probability measures contained in a calibrated curve. We say that ( m, w) is a calibrated curve associated to a corrector function χ if, for any t 1 , t 2 ∈ R, ( m, w) is optimal for the dynamic programming principle, that is

χ( m(t 1 )) = t 2 t 1 T d
H * x, -w(t) m(t) d m(s) + F( m(s))ds + χ( m(t 2 )) + λ(t 2 -t 1 ).

These curves play a fundamental role to understand the long time behavior of these MFG. They are indeed the attractors of the dynamics which minimize the discounted, infinite horizon MFG.

In the third section we focus on the relation between the limit value λ and the ergodic value λ, associated to the stationary MFG, defined by

-λ = inf (m,w) T d H * x, - w m dm + F(m).
We propose two examples which highlight how much important it is the structure of the coupling function F (x, •) in the dynamic of potential MFG. In the first example we impose monotonicity and we recover part of the results in [START_REF] Cardaliaguet | Long time average of mean field games with a nonlocal coupling[END_REF]. In this case the limit value and the ergodic one coincide.

On the other hand, in the second example, the minimization problems are no longer convex and we can prove that λ > λ. This means that it is not possible that the MFG system converges to a stationary equilibrium. The fact that λ > λ implies that the energy of the finite horizon game goes below the energy of the stationary one. Looking at the projected Mather set we can say even more. As this set is compact and it can not contain any stationary curve, calibrated curves can not even approach any static configuration.

Acknowledgement

I would like to thank Pierre Cardaliaguet (Paris Dauphine) for the fruitful discussions all along this work and Marco Cirant (Università di Padova) for a crucial hint which resulted in Lemma 4.

I was partially supported by the ANR (Agence Nationale de la Recherche) project ANR-16-CE40-0015-01.

Assumptions and definitions

We work on the d-dimensional flat torus T d = R d /Z d to avoid to deal with boundary conditions and to set the problem on a compact domain.

Notation: We denote by P(T d ) the set of Borel probability measures on T d . This is a compact, complete and separable set when endowed with the Monge-Kantorovich distance d(•, •). We define M(T d ; R d ) the set of Borel vector valued measures w with finite mass |w|. If m t is a time dependent probability measure on T d , then

L 2 m ([0, T ] × T d ) is the set of m-measurable functions f such that the integral of |f | 2 dm t over [0, T ] × T d is finite. Analogously for L 2 m (T d ) and L 2 m (T d ; R d ),
where in the latter case we consider vector valued functions. We use throughout the paper the notion of derivative for functions defined on P(T d ) introduced in [START_REF] Cardaliaguet | The master equation and the convergence problem in mean field games[END_REF]. We say that Φ :

P(T d ) → R is C 1 if there exists a continuous function δΦ δm : P(T d )×T d → R such that Φ(m 1 ) -Φ(m 2 ) = 1 0 T d δΦ δm ((1 -t)m 1 + tm 2 , x)(m 2 -m 1 )(dx)dt, ∀m 1 , m 2 ∈ P(T d ).
As this derivative is defined up to an additive constant, we use the standard normalization

T d δΦ δm (m, x)m(dx) = 0.
Assumptions: We impose the following assumptions on the hamiltonian H and the coupling function F so that we can derive uniform estimates on the solutions of the MFG system.

1.

H : T d × R d → R is of class C 2 , p → D pp H(x, p
) is Lipschitz continuous, uniformly with respect to x. Moreover there exists C > 0 that verifies

C-1 I d ≤ D pp H(x, p) ≤ CI d , ∀(x, p) ∈ T d × R d (1)
and θ ∈ (0, 1), C > 0 such that the following conditions hold true

|D xx H(x, p)| ≤ C(1 + |p|) 1+θ , |D x,p H(x, p)| ≤ C(1 + |p|) θ , ∀(x, p) ∈ T d × R d . (2) 2. F : P(T d ) → R is of class C 2 . Its derivative F : T d × P(T d ) → R is twice differentiable in x and D 2 xx F is bounded.
Examples of non monotone coupling functions which verify such conditions can be found in Lemma 4.2 and Lemma 4.3.

We recall that, if µ, ν ∈ P(T d ), the 1-Wasserstein distance is defined by

d(µ, ν) = sup T d φ(x) d(µ -ν)(x) continuous φ : T d → R, Lip(φ) ≤ 1 .
Minimization Problems: Under the above assumptions, we can introduce two minimization problems. Each one of those will be proposed in two different but equivalent forms. The first one is

U(T, m 0 ) = inf (m,α) T 0 T d H * (x, α)dm(t) + F(m(t))dt, m 0 ∈ P(T d ) where m ∈ C 0 ([0, T ], P(T d )), α ∈ L 2 m ([0, T ] × T d , R d ) and the following equation is verified in sense of distribution -∂ t m + m + div(mα) = 0 in [0, T ] × T d m(0) = m 0 in T d . (3)
Equivalently (see [START_REF] Briani | Stable solutions in potential mean field game systems[END_REF] for more details),

U(T, m 0 ) = inf (m,w)∈E T 2 (m 0 ) T 0 T d H * x, - dw(t) dm(t) (x) dm(t) + F(m(t))dt, m 0 ∈ P(T d )
where

E T 2 (m 0 ) is the set of time dependent Borel measures (m(t), w(t)) ∈ P(T d ) × M(T d ; R d ) such that m ∈ C 0 ([0, T ], P(T d )), w is absolutely continuous with respect to m, its density dw/dm belongs to L 2 m ([0, T ] × T d ) and -∂ t m + m -divw = 0 is verified in sense of distributions with initial condition m(0) = m 0 .
The second minimization problem reads

V δ (m 0 ) = inf (m,α) ∞ 0 e -δt T d H * (x, α)dm(t) + F(m(t))dt, m 0 ∈ P(T d ) where δ > 0, m ∈ C 0 ([0, +∞), P(T d )), α ∈ L 2 m,δ ([0, +∞) × T d , R d ), that is L 2 m
with weight e -δt , and (m, α) verifies (3) in [0, +∞) × T d . Equivalently, V δ can be defined as

V δ (m 0 ) = inf (m,w)∈E δ 2 (m 0 ) +∞ 0 e -δt T d H * x, - dw(t) dm(t) (x) dm(t) + F(m(t))dt, m 0 ∈ P(T d ) (4) where E δ 2 (m 0 ) is defined as E 2 (m 0 )
with the only difference that we ask dw/dm to be L 2 in [0, +∞) × T d with respect to e -δt m(t). For convenience we introduce the functional on E 2 (m 0 )

J δ (m 0 , m, w) = +∞ 0 e -δt T d H * x, - dw(t) dm(t) (x) dm(t) + F(m(t))dt, so that V δ (m 0 ) = inf (m,w) J(m 0 , m, w).
We also define the ergodic value λ as follows

-λ = inf m,α T d H * (x, α)dm + F(m) (5)
where (m, α)

∈ P(T d ) × L 2 m (T d ; R d ) verifies in sense of distribution m + div(mα) = 0 in T d . Or, equivalently, -λ = inf (m,w)∈E T d H * x, - dw dm (x) dm(x) + F(m) (6)
where E is the set of (m, w) ∈ P(T d ) × M(T d ; R d ) such that w is absolutely continuous with respect to m, its density dw/dm belongs to L 2 m (T d ) and m -divw = 0 is verified in sense of distributions.

Throughout the paper we will use the constant C > 0 which may change from line to line. We start proving that the minimization problem (4) admits a minimizer and we also give a characterization of such a minimizer in terms of solutions of the associated MFG system. Proposition 1.1. For any δ > 0 and any m 0 ∈ P(T d ), V δ (m 0 ) admits a minimizer (m, w).

Moreover there exists u ∈ C 1,2 ([0, +∞) × T d ) and m ∈ C 0 ([0, +∞) × P(T d )) solutions of      -∂ t u -u + δu + H(x, Du) = δF δm (x, m) := F (x, m) in T d × [0, +∞) -∂ t m + m + div(mD p H(x, Du)) = 0 in T d × [0, +∞) m(0) = m 0 , u ∈ L ∞ ([0, +∞) × T d ) (7)
such that w = -mD p H(x, Du).

Proof. First of all we show that V δ (m 0 ) is finite and that it is bounded by a constant K δ independent of m 0 . We can always use as competitor for V δ (m 0 ) the couple (µ, 0) where µ is the solution of

-∂ t µ + µ = 0 in T d × [0, +∞) µ(0) = m 0 in T d .
Given that F is bounded, if we use (µ, 0) as a competitor, we get

V δ (m 0 ) ≤ +∞ 0 e -δt H * (x, 0) + sup m F(m)dt := K δ .
We fix a minimizing sequence (m n , w n ). If we use (1), we have that

+∞ 0 T d e -δt |w n (t, x)|dxdt ≤ M δ
where M δ is a fixed constant that does not depend on m 0 and n. Hence, for any fixed k > 0

k 0 T d |w n (t, x)|dxdt ≤ M δ e δk . (8)
Following Lemma 3.1 in [START_REF] Cardaliaguet | Second order mean field games with degenerate diffusion and local coupling[END_REF], we get that for any t, s ∈ [0, k]

d(m n (t), m n (s)) ≤ C k δ |t -s| + C|t -s| 1/2 , (9)
where C k δ depends only on δ and k. Inequality (9) tells us that {m n } n is uniformly bounded in C 1/2 ([0, k), P(T d )). We have then that m n converges uniformly on any compact set to a limit m ∈ C 0 ([0, +∞), P(T d )). Thanks to the bounds [START_REF] Cardaliaguet | The master equation and the convergence problem in mean field games[END_REF] we also know that w n converges in M(I × T d ; R d ) to a certain w on any bounded interval I ⊂ [0, +∞).

As the couple ( m, w) belongs to E δ 2 (m 0 ) we have that J δ (m 0 , m, w) < +∞. This means that

J δ (m(0), m, w) = lim k→+∞ k 0 e -δt T d H * x, - d w(t) d m(t) (x) d m(t) + F( m(t))dt
Note also that the functional is bounded from below, so there exists a constant C δ such that, for any (m, w)

∈ E δ 2 , +∞ k e -δt T d H * x, - dw(t) dm(t) (x) dm(t) + F(m(t))dt ≥ e -δk +∞ 0 e -δt inf (x,p)∈T d ×R d H * (x, p) + inf m F(m)dt = e -δk C δ .
Therefore,

J δ (m 0 , m n , w n ) ≥ k 0 e -δt T d H * x, - dw n (t) dm n (t) (x) dm n (t) + F(m n (t))dt + e -δk C δ .
Thanks to the convergence of (m n , w n ) on compact sets, we can pass to the limit in n and we get

V δ (m 0 ) ≥ k 0 e -δt T d H * x, - d w(t) d m(t) (x) d m(t) + F( m(t))dt + C δ e -δk .
Taking the limit on k we finally get that V δ (m 0 ) ≥ J(m 0 , m, w).

The proof of the second statement relies again on classic tools for potential MFG (see for instance [START_REF] Cardaliaguet | Second order mean field games with degenerate diffusion and local coupling[END_REF] or [START_REF] Briani | Stable solutions in potential mean field game systems[END_REF]). Using the convexity of H * and the regularity of F, we can easily show that if ( m, w) is a minimizer for V δ (m 0 ) then it must be a minimizer for Jδ :

E δ 2 (m 0 ) → R, which reads Jδ (m, w) = +∞ 0 e -δt T d H * x, - dw(t) dm(t) (x) dm(t) + F (x, m(t))dm(t)dt.
As Jδ is convex we can define its dual problem (in the sense of Fenchel-Rockafellar) inf

u∈C 2 b ([0,+∞)×T d ) - T d u(0, x)dm 0 (x) where -∂ t u -u + δu + H(x, Du) ≤ F (x, m) .
Thanks to the comparison principle, the minimizer ū is the unique solution of

-∂ t ū -ū + δ ū + H(x, Dū) = F (x, m) in C 2 b ([0, +∞) × T d
). If we sum the two problems we get

+∞ 0 e -δt T d H * x, - d w(t) d m(t) (x) d m(t) + F (x, m(t))d m(t)dt - T d φ(0, x)dm 0 (x) = 0
Using the differential constraints of ( m, w) and ū we get that w = -mD p H(x, Dū) m-a.e.. This means that w is bounded that in turn implies that m > 0 so that w = -mD p H(x, Dū) is verified everywhere.

We now state without proof the dynamic programming principle for V δ . The proof relies on standard arguments in optimal control theory (see for instance [START_REF] Cannarsa | Semiconcave functions, hamilton-jacobi equations, and optimal control[END_REF]).

Lemma 1.2. The function V δ verifies the dynamic programming principle, which reads

V δ (m 0 ) = inf (m,α)∈E δ 2 (m 0 ) t 0 e -δs T d H * (x, α)dm(s) + F(m(s))ds + e -δt V δ (m(t)) .

Existence of a corrector

The main result of this section is Theorem 1.5 where show that the function V δ (•) is uniformly Lipschitz with respect to δ. As a consequence, we have Proposition 1.6 which claims on one side that the limit lim δ→0 δV δ (m 0 ) is well defined and it is uniform in m 0 and, on the other, that, up to subsequence, also V δ (•) -V δ (m 0 ) converges to a continuous function χ. In Lemma 1.7 we prove that χ enjoys the dynamic programming principle and, therefore, we have the existence of a corrector function.

The idea behind the proof of Theorem 1.5 is the following: we want to prove that there exists a constant K > 0 independent of δ, such that

|V δ (m 0 2 ) -V δ (m 0 1 )| ≤ Kd(m 0 1 , m 0 2 ).
We fix an horizon T > 0, to be chosen later, and we take (m 1 (•), α 1 (•)) a minimizer for V δ (m 0 1 ). We consider any couple (m 2 , α 2 ) such that [START_REF] Bogachev | Fokker-planckkolmogorov equations[END_REF]

is verified in [0, T ], m 2 (0) = m 0 2 , m 2 (T ) = m 1 (T ) and m 2 ≡ m 1 , α 2 ≡ α 1 in [T, ∞). The couple (m 2 , α 2 ) is admissible and V δ (m 0 2 ) ≤ T 0 e -δs T d H * (x, α 2 )dm 2 (s) + F(m 2 (s))ds + e -δT V δ (m 1 (T )). (10)
Therefore,

V δ (m 0 2 ) -V δ (m 0 1 ) ≤ T 0 e -δt T d H * (x, α 2 )dm 2 (t) -H * (x, α 1 )dm 1 (t) + F(m 2 (t)) -F(m 1 (t))dt.
In order to prove the continuity of V δ with respect to the initial data we need to introduce some standard estimates on the solutions of the MFG system (7).

Lemma 1.3. There exists C > 0 independent of m 0 , T, δ such that, if (u, m) is a classical solution of (7), then

• Du L ∞ ([0,+∞)×T d ) ≤ C • |D 2 u(s, •)| ≤ C for any s ∈ [0, +∞) • d(m(s), m(l)) ≤ C|l -s| 1/2 for any l, s ∈ [0, +∞)
Consequently, we also have that

|∂ t u(s.•)| ≤ C for any s ∈ [0, +∞).
Proof. The proof follows closely the one proposed in [START_REF] Cardaliaguet | Long time behavior of the master equation in mean-field game theory[END_REF] and it relies on semiconcavity estimates for the value function u. We recall that if φ ∈ C ∞ (T d ) then

Dφ L ∞ (T d ) ≤ d 1 2 sup x∈T d , |ξ|≤1 D 2 φ(x)ξ • ξ. (11)
We first prove the result for u

T : [0, T ] × T d → R, solution of      -∂ t u -u + δu + H(x, Du) = F (x, m) in T d × [0, T ] -∂ t m + m + div(mD p H(x, Du)) = 0 in T d × [0, T ] m(t) = m 0 , u(T, x) = 0 in T d We consider ξ ∈ R d , |ξ| ≤ 1, that maximizes sup t,x D 2 u T (t, x)ξ • ξ = M
and we look at the equation solved by w(t, x) = D 2 u T (t, x)ξ • ξ deriving twice in space the HJB equation in [START_REF]Long time average of first order mean field games and weak kam theory[END_REF]:

-∂ t w -w + δw + D ξξ H(x, Du) + 2D ξp H(x, Du) • D 2 uξ +D pp H(x, Du)D 2 uξ • D 2 uξ + D p H(x, Du) • Dw = D 2 ξξ F (x, m).
The maximum of w can be achieved either at t = T , but using the terminal condition of u T we get M = 0, or at a point (s, x) in the interior. In this case, if we use hypothesis ( 1) and ( 2) on H and the boundedness of D 2 xx F , then, at the maximum (s, x), we get the following inequality

δM -C(1 + |Du|) 1+θ -2C(1 + |Du|) θ |D 2 uξ| + C-1 |D 2 uξ| 2 ≤ C.
By Cauchy-Schwarz inequality we have that M = w(s, x) ≤ |D 2 u(s, x)ξ|. If we also plug [START_REF] Cardaliaguet | Long time average of mean field games[END_REF] we get, for a possible different constant C

-C(1 + M ) 1+θ -2C(1 + M ) 2θ + C-1 M 2 ≤ C.
Given that θ < 1 the above inequality ensures that M is bounded by a constant that does not depend on m 0 , δ and T . The bound on Du T L ∞ ([0,T ]×T d ) follows from [START_REF] Cardaliaguet | Long time average of mean field games[END_REF]. Now that we proved that Du T is bounded so that Theorem V 5.4 in [START_REF] Ladyzhenskaia | Linear and quasi-linear equations of parabolic type[END_REF] gives us the boundedness of D 2 u T . Note that the estimates on Du T and D 2 u T imply directly from the HJB equation that ∂ t u T is bounded as well. As all the estimates are independent of T , if we look at the sequence of u T we have that, on any compact set, u T is uniformly bounded and continuous. This means that u T converges to u solution of the HJB equation on [0, +∞) and the same estimates hold true for u.

Furthermore, it implies that also D p H(x, Du), the drift of the Fokker Planck equation, is uniformly bounded. Standard results on SDEs (for instance Lemma 3.4 in [START_REF] Cardaliaguet | [END_REF]) ensure the Holder continuity of s → m(s) uniformly with respect to m 0 and δ.

We now fix m 0 1 , m 0 2 ∈ P(T d ). According to [START_REF] Cardaliaguet | Long time average of mean field games with a nonlocal coupling[END_REF] we have

V δ (m 0 2 ) -V δ (m 0 1 ) ≤ T 0 e -δs T d H * (x, α 2 )dm 2 (s) -H * (x, DH(x, Du 1 ))dm 1 (s) + F(m 2 (s)) -F(m 1 (s))ds, (12)
where (u 1 , m 1 ) is a solution of ( 7) related to a minimizer (m 1 , w 1 ) of V δ (m 0 1 ) that we found in Proposition

1.1. The couple (m 2 , α 2 ) is such that (3) is verified in [0, h + τ ] with m 2 (0) = m 0 and m 2 (T ) = m 1 (T ).
The key point to prove the continuity of V δ is to construct a suitable (m 2 , α 2 ). We first consider m2 solution of

-∂ t m2 + m2 + div( m2 D p H(x, Du 1 )) = 0 in [0, T ] × T d m2 (0) = m 0 2 (13)
and then we set

m 2 (s, x) =      m2 (s, x), if s ∈ (0, h] τ +h-s τ m2 (s, x) + s-h τ m 1 (s, x), if s ∈ [h, h + τ ] m 1 (s, x) if s ∈ [τ + h, T ], (14)
where h and τ will be chosen later. Note that, thanks to the boundedness of their drifts, Corollary 6.3.2 in [START_REF] Bogachev | Fokker-planckkolmogorov equations[END_REF] ensures that m2 and m 1 have a density for any s > 0 . What we still need is to define

α 2 in [h, h + τ ] .
We compute the equation verified by m 2 in [h, h + τ ] and, using ( 7) and ( 13), we get

∂ t m 2 -m 2 = τ + h -s τ div ( m2 D p H(x, Du 1 )) + s -h τ div (m 1 D p H(x, Du 1 )) + m 1 -m2 τ that is, by linearity, ∂ t m 2 -m 2 = div(m 2 D p H(x, Du 1 )) + m 1 -m2 τ . Let ζ : [0, h + τ ] × T d → R be the solution to ζ = m 1 -m2 , in [0, h + τ ] × T d T d ζ(s, x) = 0. ( 15
)
We can now define the drift α 2 as follows: α 2 = D p H(x, Du 1 ) + Dζ m 2 τ in [h, h + τ ] and α 2 = D p H(x, Du 1 ) elsewhere. As (m 2 , α 2 ) verifies the Fokker Plank equation by construction with m 2 (0) = m 0 2 , it is admissible. For the continuity of V δ we still need estimates on the drift α 2 . We prove those estimates in the next lemma using the regularity of the solutions of the adjoint of the Fokker-Plank equation.

Lemma 1.4. For any time s < h + τ , there exists a constant

K s > 0, bounded for s > ε > 0, such that Dζ(s, •) L 2 (T d ) ≤ K s d(m 0 1 , m 0 2 ). The constant K s is independent of m 0 1 , m 0 2 .
Proof. We first note that, if we multiply ( 15) by ζ and we use Cauchy-Schwarz inequality on the right hand side, we get

Dζ(s) 2 L 2 (T d ) ≤ m 1 (s) -m2 (s) L 2 (T d ) ζ(s) L 2 (T d ) .

Now Pointcaré-Wirtinger inequality gives us

Dζ(s) L 2 (T d ) ≤ C m 1 (s) -m2 (s) L 2 (T d ) . (16)
If we define µ(s) = m 1 (s) -m2 (s) then µ verifies the following equation

-∂ t µ + µ + div(µD p H(x, Du 1 )) = 0 in [0, s] × T d µ(0) = m 0 1 -m 0 2 in T d . (17)
We now fix a φ ∈ L 2 (T d ) and we consider the adjoint backward equation

-∂ t φ -φ + D p H(x, Du 1 )Dφ = 0 in [0, s] × T d φ(s, x) = φ(x) in T d . (18)
Given that D p H(x, Du) is bounded, if φ is the solution of ( 18), then there exists a constant K s (Theorem 11.1 in [START_REF] Ladyzhenskaia | Linear and quasi-linear equations of parabolic type[END_REF]), such that

φ(0) C 1+α (T d ) ≤ K s φ L 2 (T d ) . (19)
As the equation ( 18) is the adjoint of ( 17),

T d φ(s)µ(s)dx = T d φ(0)µ(0)dx.
We now plug in the initial and terminal conditions and we estimate the righthand side as follows

T d φ(x)(m 1 (s)(dx) -m2 (s)(dx)) = T d φ(0, x)(m 0 1 (dx) -m 0 2 (dx)) ≤ Dφ(0) L ∞ d(m 0 1 , m 0 2 ).
If we use the interior estimate [START_REF]Weak kam theorem in lagrangian dynamics preliminary version number[END_REF] on the righthand side and we take the supremum over φ L 2 ≤ 1, we finally end up with

m 1 (s) -m2 (s) L 2 ≤ K s d(m 0 1 , m 0 2 ).
If we plug the last inequality into ( 16), we get the result.

Theorem 1.5. The family of functions {V δ (•)} δ is uniformly Lipschitz continuous.

Proof. Let α 1 (t, x) = D p H(x, Du 1 ) for any t ∈ [0, h + τ ]. We consider the same (m 2 , α 2 ) that we defined earlier: m 2 is defined in [START_REF] Cirant | The variational structure and time-periodic solutions for mean-field games systems[END_REF], α 2 = α 1 + Dζ m 2 τ in [h, h + τ ] and α 2 = α 1 elsewhere, where ζ solves [START_REF] Vries | Elements of topological dynamics[END_REF]. According to [START_REF] Cardaliaguet | Long time behavior of the master equation in mean-field game theory[END_REF] we have

V δ (m 0 2 ) -V δ (m 0 1 ) ≤ h 0 e -δs T d H * (x, α 1 )d(m 2 -m 1 )+ (20)
+ h+τ h e -δs T d H * (x, α 2 )dm 2 -H * (x, α 1 )dm 1 + h+τ 0 F(m 2 (s)) -F(m 1 (s))ds.
Using the convexity of H, we can estimate the term H * (x, α 2 (s)) for any time s ∈ [h, h + τ ] as follows

T d H * (x, α 2 (s))dm 2 (s) ≤ T d H * (x, α 1 (s))dm 2 (s) + D p H * (x, α 2 (s)) • Dζ(s) τ dx (21)
≤ T d H * (x, α 1 (s))dm 2 (s) + 1 τ |α 1 (s)||Dζ(s)| + C |Dζ(s)| 2 τ 2 m 2 (s) dx,
where in the last inequality we add and subtract α 1 (s) • Dζ/τ and we used the growth condition (1) on D pp H.

We recall that, as the drift D p H(x, Du 1 ) is continuous and bounded, according to Theorem 2.2.1 [START_REF] Bogachev | Elliptic and parabolic equations for measures[END_REF], the measure m 1 has a density m 1 (s, x) for any s > 0, then, using Theorem 2.5.1 in [START_REF] Bogachev | Elliptic and parabolic equations for measures[END_REF], for any l ∈ (0, s), we have

m 1 (s, x) > m 1 (l, x 0 )e -Q(1+ 1 s-l + 1 l ) ,
where Q does not depend on m 0 1 , l and s. As T d is bounded, for any l > 0, there exists a x 0 such that m 1 (l, x 0 ) > 1/2. Given that the same holds true for m2 then, for any s ∈ [h, h + τ ], the definition of m 2 in [START_REF] Cirant | The variational structure and time-periodic solutions for mean-field games systems[END_REF] implies that

m 2 (s, x) > 1 2 e -Q(1+ 1 s-l + 1 l )
∀l ∈ (0, h).

For l = h/2 we obtain that the infimum, with respect to s, in the righthand side is achieved when s = h. Thus,

m 2 (s, x) > 1 2 e -Q(1+ 4 h ) . (22)
We can now plug ( 21) and ( 22) into [START_REF] Gomes | Discrete time, finite state space mean field games[END_REF], which becomes

V δ (m 0 2 ) -V δ (m 0 1 ) ≤ h+τ 0 e -δs T d H * (x, α 1 )d(m 2 -m 1 ) + F(m 2 (s)) -F(m 1 (s))ds + h+τ h e -δs T d C 2 τ |Dζ| + 2 C τ 2 |Dζ| 2 e Q(1+ 4 h ) dxds.
Using the bounds on Du 1 found in Lemma 1.3, Lemma 1.4 and the regularizing property of F, we get

V δ (m 0 2 ) -V δ (m 0 1 ) ≤ C h+τ 0 d(m 2 (s), m 1 (s))ds + h+τ h Cd(m 0 1 , m 0 2 )K s + 2 C τ 2 d 2 (m 0 1 , m 0 2 )e Q(1+ 4 h ) K 2 s ds ≤ C h+τ 0 d(m 2 (s), m 1 (s))ds + Cd(m 0 1 , m 0 2 )e Q(1+ 4 h ) h+τ h 1 + K s τ 2 d(m 0 1 , m 0 2 ) K s ds ≤ C h+τ 0 d(m 2 (s), m 1 (s))ds + C τ 2 d(m 0 1 , m 0 2 )e Q(1+ 4 h ) h+τ h K 2 s ds. (23)
In the last inequality we neglected the terms which go to infinity slower than K 2 s and which vanish faster than d(m 0 1 , m 0 2 ). Note that the constant K s might explode when s goes to 0 but, otherwise, it is bounded. Therefore, as h > 0, there is no problem of integrability for the term

h+τ h K 2
s . We now focus on the first term in the above inequality. In order to estimate d(m 1 (s), m 2 (s)), we have to look at the SDEs verified by the stochastic processes whose laws are m 1 and m 2 . We first recall that an equivalent formulation of the 1-Wasserstein distance between two probability measures µ and ν is

d(µ, ν) = inf γ T d ×T d |x -y|dγ(x, y) s.t. π 1 γ = µ, π 2 γ = ν . ( 24
)
We consider a standard probability space (Ω, G, P) and two random variables

Z 1 , Z 2 such that L(Z i ) = m 0 i and E Z 2 -Z 1 = d(m 0 1 , m 0
2 ). Therefore, m 1 and m 2 are the laws of the processes defined by of the following SDEs

dX i s = α i (t, X s )ds + √ 2dB s X i 0 = Z i .
Using the definition of distance in [START_REF] Lasry | Jeux à champ moyen. i-le cas stationnaire[END_REF], we have

d(m 1 (s), m 2 (s)) ≤ E X 2 s -X 1 s ≤ E Z 2 -Z 1 + s 0 α 2 (l, X 2 l ) -α 1 (l, X 1 l ) dl .
We first split h+τ 0 d(m 2 (s), m 1 (s))ds in the sum of the integrals on the intervals [0, h] and

[h, h + τ ]. For any s ∈ [0, h], α 1 (l, x) = D p H(x, Du 1 (t, x)) = α 2 (l, x), then d(m 1 (s), m 2 (s)) ≤ d(m 0 1 , m 0 2 ) + E s 0 D p H(X 2 l , Du 1 (l, X 2 l )) -D p H(X 1 l , Du 1 (l, X 1 l )) .
Hypothesis (1), ( 2) and Lemma 1.3 ensure that both p → D p H(x, p) and x → D p H(x, Du 1 (l, x)) are Lipchitz continuous, hence

d(m 1 (s), m 2 (s)) ≤ d(m 0 1 , m 0 2 ) + C s 0 d(m 1 (l), m 2 (l))dl.
If we apply Gronwall's inequality, then for any s ∈ [0, h]

d(m 1 (s), m 2 (s)) ≤ d(m 0 1 , m 0 2 )e Cs . ( 25
)
We now look at

h+τ h d(m 1 (s), m 2 (s))ds. According to the definition of α 2 , for s ∈ [h, h + τ ], we have d(m 1 (s), m 2 (s)) ≤ d(m 1 (h), m 2 (h))+ +E s h D p H X 2 l , Du 1 (l, X 2 l ) + Dζ(l, X 2 l ) τ m 2 (l, X 2 l ) -D p H(X 1 l , Du 1 (l, X 1 l )) . (26)
Using ( 25) on d(m 1 (h), m 2 (h)) and splitting the last term, we get that ( 26) is smaller than

d(m 0 1 , m 0 2 )e Ch +E s h D p H X 2 l , Du 1 (l, X 2 l ) -D p H(X 2 l , Du 1 (l, X 1 l )) +E s h Dζ(l, X 2 l ) τ m 2 (l, X 2 l )
.

If we use again that x → D p H(x, Du 1 (l, x)) is Lipschitz continuous, we get d(m 1 (s), m 2 (s)) ≤ d(m 0 1 , m 0 2 )e Ch + C τ E s h Dζ(l, X 2 l ) m 2 (l, X 2 l ) + s h d(m 1 (l), m 2 (l)). (27)
Thanks to estimates [START_REF] Huang | Large population stochastic dynamic games: closed-loop mckean-vlasov systems and the nash certainty equivalence principle[END_REF] on m 2 we can use Tonelli's theorem and switch the integral with the expectation. Using Lemma 1.4, we eventually have

E s h Dζ(l, X 2 l ) m 2 (l, X 2 l ) = s h T d |Dζ(l, x)| dx ≤ d(m 0 1 , m 0 2 ) s h K l dl.
If we plug the last inequality into [START_REF] Lions | Homogenization of hamilton-jacobi equations[END_REF], we can use again Gronwall's inequality so that for

s ∈ [h, h + τ ] d(m 1 (s), m 2 (s)) ≤ e Cs + e s-h τ s h K l dl d(m 0 1 , m 0 2 ). (28)
We can now suppose h = τ = 1 and plugging ( 25) and ( 28) into ( 23), we finally get that, for a given constant C depending on all the other ones

V δ (m 0 2 ) -V δ (m 0 1 ) ≤ Cd(m 0 1 , m 0 2 ) 2 0 e Cs ds + 2 1 e s-1 s 1 K l dlds + e 5Q 2 1
K 2 s ds .

We recall that the constant K s of Lemma 1.4 is bounded when h is not close to 0 (Theorem 11.1 in [START_REF] Ladyzhenskaia | Linear and quasi-linear equations of parabolic type[END_REF]). The infimum in the expression above is finite and none of the constants therein depends on δ. Therefore, {V δ } δ is uniformly K-Lipschitz with

K = C 2 0 e Cs ds + 2 1 e s-1 s 1 K l dlds + e 5Q 2 1
K 2 s ds .

Proposition 1.6. For any fixed η ∈ P(T d ) there exists a subsequence δ n → 0, such that V δn (•) -V δn (η) uniformly converges to a function χ : P(T d ) → R when n → +∞. Moreover, δ n V δn uniformly converges to a constant -λ

Proof. The continuity proved in Theorem 1.5 ensures the boundedness of

V δ (•) -V δ (η). Indeed we have |V δ (•) -V δ (η)| ≤ Kd(•, η).
As P(T d ) is compact, the right hand side is bounded by a constant K. Arzelà-Ascoli theorem ensures that there exists a subsequence δ n → 0 such that V δn (•) -V δn (η) converges to a continuous function χ.

We now want to prove that δV δ is a bounded function. We fix a measure µ ≡ 1, then we define the control (m, α) as follows: m(t) = µ and α(t) = 0 for all t ∈ [0, +∞). The control is admissible, therefore we have

δV δ (µ) ≤ δ(H * (x, 0) + F(µ)) ∞ 0 e -δs ds = H * (x, 0) + F(µ).
Given that H * and F are bounded from below, then

δV δ (µ) ≥ δ inf (x,a)∈T d ×R d H * (x, a) + inf m∈P(T d ) F(m) ∞ 0 e -δs ds = inf (x,a) H * (x, a) + inf m F(m).
Therefore, δV δ (µ) is uniformly bounded in δ. If we fix any other measure m 0 we can use again the uniform continuity of V δ to get that |δV δ (m 0 )-δV δ (µ)| ≤ δK that in turn tells us that δV δ (•) is a sequence of uniformly continuous functions. Using again Arzelà-Ascoli theorem we get that δ n V δn uniformly converges to a function Ψ (we can suppose δ n to be the same subsequence that we identified earlier). Moreover, we have

|δ n V δn (•) -δ n V δn (µ)| ≤ δ n K.
Taking the limit we get |Ψ(•) -Ψ(µ)| ≤ 0 so that δ n V δn converges to the constant function Ψ(µ) := -λ.

Lemma 1.7. Dynamic programming principle for χ: for any m 0 ∈ P(T d ) and t > 0,

χ(m 0 ) = inf (m,α) t 0 H * (x, α)dm(s) + F(m(s))ds + χ(m(t)) + λt ( 29
)
where m ∈ C 0 ([0, t], P(T d )), α ∈ L 2 m ([0, t] × T d , R d
) and the pair (m, α) solves in sense of distribution -∂ t m + m + div(mα) = 0 with initial condition m 0 .

Proof. In Proposition 1.6 we proved the convergence of V δ (•) -V δ (η) to χ(•) along the subsequence {δ n } n , for a fixed measure η. Hereafter, {δ n } n and η will be the ones identified in that proposition.

We know from Proposition 1.1 that, for any δ > 0, there exists a solution (u δ , m δ ) to [START_REF]Long time average of first order mean field games and weak kam theory[END_REF] such that

V δ (m 0 ) = t 0 e -δs T d H * (x, α δ )dm δ (s) + F(m δ (s))ds + e -δt V δ (m δ (t)),
where α δ = D p H(x, Du δ ). If we take the expansion of e -δt and we subtract on both sides V δ (η) we get

V δ (m 0 ) -V δ (η) = t 0 e -δs T d H * (x, α δ )dm δ (s) + F(m δ (s))ds + (1 -tδ + o(tδ))V δ (m δ (t)) -V δ (η). ( 30
)
We recall that the estimates in Lemma 1.3 are uniform in δ hence D p H(x, Du δn ) converges uniformly to a function α. We can now take the limit n → +∞ and using that V δn (•) -V δn (η) → χ(•) and δ n V δn (•) → -λ, we get

χ(m 0 ) = t 0 T d H * (x, α)dm(s) + F(m(s))ds + λt + χ(m t )
In order to show that (α, m) is optimal, we fix a competitor (β, µ). According to the dynamic programming principle of V δ , if we plug (β, µ) into (30), we get

V δ (m 0 ) -V δ (η) ≤ t 0 e -δs T d H * (x, β)dµ(s) + F(µ(s))ds + (1 -tδ + o(tδ))V δ (µ(t)) -V δ (η).
Taking again the limit on the subsequence {δ n } n we eventually have that

χ(m 0 ) ≤ t 0 T d H * (x, β)dµ(s) + F(µ(s))ds + λt + χ(µ t ),
which proves the result.

Convergence of U(t, •)/t and δV δ (•)

In this section we propose a Tauberian-type result where we prove that the limit of δV δ (•) coincides with the one of U(t, •)/t when t → +∞.

Theorem 1.8. The limit value -λ is uniquely defined and δV δ (•) → -λ does not depend on a subsequence. Moreover, 1 T U(T, •) uniformly converges to -λ when T goes to +∞.

Proof. Let {δ n } n such that δ n V δn → -λ and V δn (•) -V δn (η) → χ(•). As χ is a continuous function on the compact set P(T d ), there exists a constant C > 0 such that 0 ≤ χ(m) + C for any m ∈ P(T d ). If (m(t), w(t)) ∈ E T 2 (m 0 ), then

T 0 T d H * x, - dw(t) dm(t) (x) dm(t) + F(m(t))dt ≤ T 0 T d H * x, - dw(t) dm(t) (x) dm(t) + F(m(t))dt + χ(m(T )) + λT -λT + C.
Taking the infimum over E T 2 , the definition of U(T, m 0 ) and the dynamic programming principle of χ lead to

U(T, m 0 ) ≤ χ(m 0 ) -λT + C.
As the constant C does not depend on m 0 and T , if we divide on both sides by T and we take the limit T → +∞, we get lim

T →+∞ 1 T U(T, m 0 ) ≤ -λ.
The other inequality is analogous. We just need to take a C 2 > 0 such that 0 ≥ χ(m) -C 2 and repeat the same computation.

Note that the limit U(T, •)/T → -λ is uniform and does not depend on the subsequence δ n or the function χ. Therefore, the limit δV δ is uniquely defined.

We conclude the section showing that our limit value λ is never lower than the ergodic one λ defined in [START_REF] Cannarsa | Semiconcave functions, hamilton-jacobi equations, and optimal control[END_REF].

Proposition 1.9. Under the above assumptions, λ ≥ λ.

Proof. We know that the convergence of U(•, T )/T is uniform, therefore, if (m, α) is an admissible couple for the static problem, we can use it as competitor for U(m, T ). So,

1 T U(T, m) ≤ 1 T T 0 T d H * (x, α)dm + F(m)dt = T d H * (x, α)dm + F(m).
If we take the infimum over all admissible static (m, α) we get

1 T U(T, m) ≤ -λ.
Letting T go to +∞, we get the result.

An other representation for λ

We can now introduce a third representation for λ, inspired again by classic results on weak KAM theory (see for instance [START_REF] Fathi | Solutions kam faibles conjuguées et barrieres de peierls[END_REF]), which consists in minimizing over paths with fixed endpoints.

Let Π T (m 0 , m 1 ) be the set of (m, α)

∈ C 0 ([0, +∞), P(T d )) × L 1 m,loc ([0, +∞) × T d , R d
) such that (m, α) solves the usual Fokker-Plank equation -∂ t m + m -div(mα) = 0 with the extra constraint m(0) = m 0 and m(T ) = m 1 . Note that, due to the smoothing property of the parabolic constraint, not for every m 1 we can find such a path, so Π T (m 0 , m 1 ) might be the empty set.

Proposition 1.10. Let m 0 , m 1 ∈ P(T d ). If m 1 has a density in H 1 (T d ) and there exists an ε > 0 such that m 1 > ε almost everywhere then

-λ = lim T →∞ 1 T inf Π T (m 0 ,m 1 ) T 0 T d H * (x, α)dm(s) + F(m(s))ds.
Proof. Let m 0 and m 1 be as above and ( m, ᾱ) be optimal for U(T, m 0 ). We extend ( m, w) in [0, T + 1] as follows: for any t ∈ [T, T + 1] we define ᾱ(t, x) = ᾱ(T, x) and m(t, x) as the solution of -∂ t m + m + div( ᾱm) = 0 with m(T, x) = m(T, x). Note that ᾱ is continuous and bounded in [0, T + 1], therefore, the estimates (22) still apply.

We now define a path from m 0 to m 1 as follows:

m 2 (s, x) = m(s, x) s ∈ [0, T ] (T + 1 -s) m(s, x) + (s -T )m 1 (x) s ∈ [T, T + 1]
.

Let also ζ(s, x) be solution of -ζ(s, x) = m 1 (x) -m(s, x) with T d ζ = 0. We can define the control

α 2 (s, x) = ᾱ(s, x) s ∈ [0, T ] ᾱ(s, x) -(s-T ) ᾱ(T,x)m 1 (x)+Dζ(s,x)+(s-T )Dm 1 (x) (T +1-s) m(s,x)+(s-T )m 1 (x) s ∈ [T, T + 1].
The couple (m 2 , α 2 ) belongs to Π T (m 0 , m 1 ). From the definition of U we deduce that

1 T + 1 inf m∈P(T d ) U(T + 1, m) ≤ 1 T + 1 inf Π T +1 (m 0 ,m 1 ) T +1 0 T d H * (x, α)dm(s) + F(m(s))ds ≤ 1 T + 1 T +1 0 T d H * (x, α 2 )dm 2 (s) + F(m 2 (s))ds = T T + 1 1 T U(T, m 0 ) + 1 T + 1 T +1 T T d H * (x, α 2 )dm 2 (s) + F(m 2 (s))ds. ( 31
)
If we prove that 1

T +1 T +1 T T d H * (x, α 2 )dm 2 (s) + F(m 2 (s)
)ds converges to zero we have the result. Indeed, if we let T go to +∞, according to Theorem 1.8, we have

-λ ≤ lim T →∞ 1 T + 1 inf Π T +1 (m 0 ,m 1 ) T +1 0 T d H * (x, α)dm(s) + F(m(s))ds ≤ -λ.
We now focus on the last part in (31). Given that F(m 2 ) is uniformly bounded, we look at the first term.

T +1 T T d H * (x, α 2 )dm 2 (s) ≤ C T +1 T T d |ᾱ(T, x)m 2 (s) + (s -T )ᾱ(T, x)m 1 (s) + Dζ(s, x) + (s -T )Dm 1 (s)| 2 m 2 2 (s) +1 dm 2 (s)ds ≤ C T +1 T T d (|ᾱ(T, x)m 2 | + |ᾱ(T, x)m 1 | + |Dζ(s, x)| + |Dm 1 |) 2 m 2 dxds + C (32)
If we use the hypothesis on m 1 and the estimates ( 22) on m with h = T + 1 and t = T -1, we get that m 2 ≥ τ for a certain τ > 0 independent of T . Lemma 1.3 ensures that ᾱ is uniformly bounded by a constant K independent of T . Therefore, (32) is lower than

1 τ K m 2 L 2 (T d ×[T,T +1]) + K m 1 L 2 (T d ) + Dζ L 2 (T d ×[T,T +1]) + Dm 1 L 2 (T d ) 2 + C.
Thanks again to the boundedness of ᾱ, standard result on parabolic equations tell us that m(s) (which is defined at the beginning of the proof) is uniformly bounded from above in [T,

T + 1]. Hence, Dζ L 2 ≤ C m 1 L 2 and m 2 L 2 (T d ×[T,T +1]) ≤ C m 1 L 2 (T d ) + C 2 . Thus T +1 T T d H * (x, α 2 )dm 2 (s) ≤ M m 1 2 H 1 (T d ) + M 2
where neither M nor M 2 depends on T . Dividing by T + 1 and taking the limit completes the proof.

2 Projected Mather set and Calibrated curves

Calibrated Curves

We borrow again some tools and some notations from the weak KAM theory (see Chapter 4 of [START_REF]Weak kam theorem in lagrangian dynamics preliminary version number[END_REF]) and in particular we will focus on the notion of calibrated curve. Before introducing this notion, we look back to the dynamic programming principle verified by corrector functions, which reads

χ(m 0 ) = inf (m,w)∈E t 2 (m 0 ) t 0 H * x, - w m dm(s) + F(m(s))ds + χ(m(t)) + λt.
As the function χ is continuous, standard arguments show that, for any fixed m 0 ∈ P(T d ) and t > 0, there exists a solution ( m, w) ∈ E t 2 (m 0 ) to the minimization problem described above.

Therefore, extending ( m, w) from m(t), it is easy to construct a new trajectory ( m, w), defined on [0, +∞) such that, for any τ > 0, it verifies

χ(m 0 ) = λτ + τ 0 T d H * x, - w(s) m(s) d m(s) + F( m(s))ds + χ( m(τ )).
We now prove that any of these trajectories is associated to a MFG system.

Proposition 2.1. Let m 0 ∈ P(T d ), χ be a corrector function and ( m, w) be a minimizing trajectory on [0, +∞) defined as above. Then, m ∈ C 1,2 ((0 + ∞) × T d ) and there exists a function ū ∈ C 1,2 ([0, +∞) × T d ) such that w = -mD p H(x, Dū) where ( m, ū) solves

     -∂ t u -u + H(x, Du) = F (x, m) in [0, +∞) × T d -∂ t m + m + div(mD p H(x, Du)) = 0 in [0, +∞) × T d m(0) = m 0 .
Remark 2.2. Due to the lack of regularity of χ we can not derive the MFG system as optimal condition for the minimization problem (33). Indeed, if χ were C 1 we would derive typical MFG system with terminal condition u(t) = δχ(m(t))/δm but this latter term is not well defined.

Proof. The proof relies on the same arguments as in Proposition 1.1. Let ( m, w) be as in the hypothesis. Then it verifies the Fokker-Plank equation and it is a minimizer of the problem inf

(m,w)∈E t 2 (m 0 ) t 0 T d H * x, - w m dm(s) + F(m(s))ds + χ(m(t)). (33)
As ( m, w) is optimal for the minimization problem above, then it must be also optimal for the following MFG planning problem inf (m,w)∈Π(m 0 , m(t))

t 0 T d H * x, - w m dm(s) + F(m(s))ds,
where Π(m 0 , m(t)) is the set of (m, w) ∈ E t 2 (m 0 ) that solves the usual Fokker-Plank equation on [0, t] with the constraints m(0) = m 0 and m(t) = m(t). We want to prove that w = -mD p H(x, Dū) where ( m, ū) solves in classical sense

     -∂ t u -∆u + H(x, Du) = F (x, m) in [0, t] × T d ∂ t m -∆m -div(mD p H(x, Du)) = 0 in [0, t] × T d m(0) = m 0 , m(t) = m(t).
We argue again as in Proposition 1.1. According to Proposition 3.1 in [START_REF] Briani | Stable solutions in potential mean field game systems[END_REF], ( m, w) minimizes also the following convex problem inf

(m,w)∈Π(m 0 ), m(t)) t 0 T d H * x, - w m dm(s) + T d F (x, m(s))dm(s)ds. ( 34
)
This problem admits a dual formulation, in the sense of Fenchel Rokafellar Theorem, which reads

inf ψ∈ K T d ψ(x, t)d m(t) - T d ψ(x, 0)dm 0 ) where K is the set of ψ ∈ C 1,2 ([0, t] × T d ) such that -∂ t ψ -ψ + H(x, Dψ) ≤ F (x, m).
A full justification of the result above can be found again in [START_REF] Cardaliaguet | Second order mean field games with degenerate diffusion and local coupling[END_REF].

In the definition of the dual problem we can replace K with K, where K is the set of

u ∈ C 1,2 ([0, t] × T d ) such that -∂ t u -∆u + H(x, Du) = F (x, m). Indeed, if ψ verifies -∂ t ψ -ψ + H(x, Dψ) ≤ F (x, m), we can alway consider u ∈ C 1,2 ([0, t] × T d ) solution of -∂ t u -∆u + H(x, Du) = F (x, m) in [0, t] × T d u(x, t) = ψ(x, t) in T d
Thanks to the comparison principle we have that u(0, x) ≥ ψ(0, x), thus inf

ψ∈ K T d ψ(x, 0)d m(t) - T d ψ(x, 0)dm 0 ≥ inf u∈K T d u(x, t)d m(t) - T d u(x, 0)dm 0 ( 35
)
The opposite inequality holds by inclusion. Lemma 3.2 in [START_REF] Cardaliaguet | Second order mean field games with degenerate diffusion and local coupling[END_REF] and Proposition 3.1 in [START_REF] Briani | Stable solutions in potential mean field game systems[END_REF], which rely on the Fenchel-Rokafellar Theorem, ensure that, if ( m, w) is a minimizer of (34) and ū ∈ K is a minimizer of the dual problem, then

T d u(x, t)d m(t) - T d u(x, 0)dm 0 + t 0 T d H * x, - w m d m(s) + T d F (x, m(s))d m(s)ds = 0.
This implies that w = -mD p H(x, ū). As a consequence, we have that m is driven by a smooth drift and so, by Schauder theory, m ∈ C 1,2 ((0, t] × T d ). In particular, given that t is arbitrary, then m ∈ C 1,2 ((0, +∞) × T d ).

We assumed that the minimization problem (35) admits a solution. The proof of this result is developed in Lemma 4.1 in appendix. We can now introduce the notion of calibrated curve. Let E ∞ 2 be the set of (m(t), w(t)) ∈ P(T d ) × M(T d ; R d ) such that m ∈ C 0 (R, P(T d )), w is absolutely continuous with respect to m, its density dw/dm belongs to L 2 m,loc (R × T d ) and -∂ t m + m -divw = 0 is verified in sense of distribution.

Definition 2.4. We say that ( m, w) ∈ E ∞ 2 is a calibrated curve if there exists a continuous function χ : P(T d ) → R which verifies the dynamic programming principle (29) and ( m, w) is optimal for χ: for any

t 1 < t 2 ∈ R χ( m(t 1 )) = λ(t 2 -t 1 ) + t 2 t 1 T d H * x, - w(s) m(s) d m(s) + F( m(s))ds + χ( m(t 2 )).
A direct consequence of Proposition 2.1 is the following result which tells that calibrated curves are smooth and associated to MFG systems defined for any time t ∈ R.

Proposition 2.5. If (m, w) ∈ E ∞ is a calibrated curve, then m ∈ C 1,2 (R × T d
) and there exists a function u ∈ C 1,2 (R × T d ) such that w = -mD p H(x, Du) where (m, u) solves

-∂ t u -u + H(x, Du) = F (x, m) in R × T d -∂ t m + m + div(mD p H(x, Du)) = 0 in R × T d

The projected Mather set

Definition 2.6. We say that m 0 ∈ P(T d ) belongs to the projected Mather set M ⊂ P(T d ) if there exists a calibrated curve (m(t), w(t)) such that m(0) = m 0 .

Note that, if from m 0 starts a calibrated curve m(t), then, by translation, m(t) ∈ M for any t ∈ R. Proposition 2.7. There exists a calibrated curve and, consequently, the projected Mather set M is not empty.

Proof. We fix a smooth density m 0 and we look at the δ-discounted problem (4) which reads

V δ (m 0 ) = inf (m,w) ∞ 0 e -δt T d H * x, - w m dm(t) + F(m(t))dt.
We recall that V δ satisfies the dynamic programming principle

V δ (m 0 ) = inf m,w T 0 e -δs T d H * x, - w m dm(s) + F(m(s))ds + e -δT V δ (m(t)),
where the infimum is taken over (m, w) ∈ E δ 2 (m 0 ). We already know that the solution of the minimization problem corresponds to a couple ( mT δ , -mT δ D p H(x, Dū T δ )) where ( mT δ , ūT δ ) solves

     -∂ t u -u + δu + H(x, Du) = F (x, m) in T d × [0, +∞) -∂ t m + m + div(mD p H(x, Du)) = 0 in T d × [0, +∞) m(0) = m 0 in T d . (36)
Note that, as the initial condition is smooth, the solution ( mT δ , ūT δ ) is smooth as well. We define the new couple (m T δ , w T δ ) as m T δ (t, x) = mT δ (t + T, x) and w T δ (t, x) = wT δ (t + T, x) so that our problem is set on [-T, +∞). We now want to prove that, when we take the limit T → +∞, our sequence converges to a couple (m δ , w δ ) defined on R × T d such that the Fokker-Plank equation is still verified. We proved in Lemma 1.3 that the drift D p H(x, Du T δ ) is uniformly bounded in T , therefore, m T δ is the solution of a Fokker-Plank equation with bounded and smooth drift. This means that m T δ is uniformly bounded in C 1,2 ([-T + 1, +∞) × T d ). This implies that, at least on compact subsets of R × T d , when we take the limit T → +∞, we have, up to a subsequence, uniform convergence of m T δ to a limit m δ . The same convergence holds true also for w T δ . Indeed, in Lemma 1.3 we proved also the uniform boundedness of D 2 u T δ and ∂ t u T δ that implies the uniform continuity and the uniform boundedness of w T δ . The convergence (m T δ , w T δ ) to (m δ , w δ ) ensures that the couple (m δ , w δ ) verifies the Fokker-Plank equation on R × T d . We fix two different times t 1 < t 2 . For sufficiently large T , the interval [t 1 , t 2 ] is included in [-T, +∞). If we apply the dynamic programming principle for V δ , we get

V δ (m T δ (t 1 )) = t 2 t 1 e -δ(s-t 1 ) T d H * x, - w T δ m T δ dm T δ (s) + F(m T δ (s))ds + e -δ(t 2 -t 1 ) V δ (m T δ (t 2 )).
We can now take the limit of T → +∞ in the above expression and we find that (m δ , w δ ) verifies

V δ (m δ (t 1 )) = t 2 t 1 e -δ(s-t 1 ) T d H * x, - w δ m δ dm δ (s) + F(m δ (s))ds + e -δ(t 2 -t 1 ) V δ (m δ (t 2 )).
for any t 1 < t 2 ∈ R.

As the function u T δ is uniformly bounded in T we have also uniform convergence of u T δ to a function u δ . We can then pass to the limit in the MFG system (36) and the couple (u δ , m δ ) solves

-∂ t u δ -u δ + δu δ + H(x, Du δ ) = F (x, m δ ) in T d × R -∂ t m δ + m δ + div(m δ D p H(x, Du δ )) = 0 in T d × R.
As in [START_REF] Barles | Space-time periodic solutions and long-time behavior of solutions to quasi-linear parabolic equations[END_REF], in order to let δ → 0, we need to define ūδ (t, x) = u δ (t, x) -u δ (0, 0) and θ δ = u δ (0, 0). The couple (ū δ , m δ ) solves

     -∂ t ūδ -ūδ + δ ūδ + δθ δ + H(x, Dū δ ) = F (x, m δ ) in T d × R -∂ t m δ + m δ + div(m δ D p H(x, Dū δ )) = 0 in T d × R ūδ (0, x) = u δ (0, x) -u δ (0, 0) in T d .
We restrict ourselves to the subsequence {δ n } n identified in the proof of Lemma 1.7. Using again the uniform estimates on Dū δ , we have that ūδ (0, x) is uniformly bounded which implies the boundedness of δθ δ . Moreover, thanks to the bounds on D 2 u δ and ∂ t u δ , ūδ is also uniformly continuous and the same holds true for m δ . We can then pass to the limit on any compact set and ūδn → u, m δn → m and δ n θ δn → θ where (u, m, θ) solves

-∂ t u -u + θ + H(x, Du) = F (x, m) in T d × R -∂ t m + m + div(mD p H(x, Du)) = 0 in T d × R
As we can always replace u(t, x) with u(t, x) -θt we can suppose θ = 0. The convergences above give us also the uniform convergence on compact sets (up to subsequence) of the couple (m δn , w δn ) = (m δn , -m δn D p H(x, Dū δn )) to (m, w) = (m, -mD p H(x, Du)) which solves the usual Fokker-Plank equation.

Let now η ∈ P(T d ) be the measure identified in the proof of Lemma 1.7. Then

V δn (m δn (t 1 )) -V δn (η) = t 2 t 1 e -δn(s-t 1 ) T d H * x, - w δn m δn dm δn (s) + F(m δn (s))ds + e -δn(t 2 -t 1 ) V δn (m δn (t 2 )) -V δn (η).
Given the continuity of V δ , the uniform convergence of m δ and w δ , we can pass to the limit in n and we finally get that for any interval [t 1 , t 2 ] the couple (m, w) verifies the Fokker-Plank equation on R and

χ(m(t 1 )) = t 2 t 1 T d H * x, - w m dm(s) + F(m(s))ds + χ(m(t 2 )) + λ(t 2 -t 1 ).
In particular we found a calibrated curve and, for any t ∈ R, m(t) belongs to the projected Mather set M.

Compactness of the projected Mather set

In Proposition 2.1 we proved that, if χ is a corrector function and (m, w) is a trajectory starting from m 0 ∈ P(T d ) which is optimal for the dynamic programming principle of χ, then (m, w) is associated to a MFG system which enjoys the estimates we proved in Lemma 1.3. Therefore, a completely analogous proof to the one proposed in Theorem 1.5 gives the following result.

Proposition 2.8. The set of corrector functions is uniformly Lipschitz continuous.

We can now prove the compactness of the projected Mather set M Proposition 2.9. The projected Mather set M is compact

Proof. Let m n ∈ M such that m 0 n → m 0 . Let (m n (t), w n (t)
) be the calibrated curve starting from m 0 n . For any t 1 , t 2 we know that (m n (t), w n (t)) verifies

χ n (m n (t 1 )) ≥ t 2 t 1 T d H * x, - w n m n dm n (s) + F(m n (s)) + λ(t 2 -t 1 ) + χ n (m n (t 2 )). (37)
We know from Proposition 2.8 that the set {χ n } n is uniformly Lipschitz. If we replace χ n with χ n (•) -χ n (η), then {χ n } n is also bounded and thus compact. Therefore, we can pick a subsequence such that χ n converges to a function χ.

Given that χ n are uniformly bounded, then

t 2 t 1 T d H * x, - w n m n dm n (s) + F(m n (s))ds ≤ C.
The constant C does not depend on n and, therefore,

t 2
t 1 |w n | is uniformly bounded as well. As we argued in Proposition 1.1, this implies that {m n } n is uniformly bounded in C 1/2 ([t 1 , t 2 ], P(T d )) for any t 1 , t 2 . We have then that m n converges uniformly on any compact set to a limit m ∈ C 0 (R, P(T d )) and the same holds true for w n in M(R × T d ; R d ), therefore (m, w) solves in sense of distribution the usual FP equation on R. By weak lower semicontinuity of the integral part in (37) and the uniform convergence of χ n we get that

χ(m(t 1 )) ≥ t 2 t 1 T d H * x, - w m dm(s) + F(m(s))ds + χ(m(t 2 )) + λ(t 2 -t 1 )
with m(0) = m 0 because m 0 n → m 0 . The opposite inequality is true by dynamic programming principle and so this proves that m 0 ∈ M and, eventually, that M is closed.

Minimal invariant set and Ergodicity

We say that a closed subset C of M is invariant if, for any m 0 ∈ C, there exists a calibrated curve (m, w) such that m(0) = m 0 and m(t) ∈ C for any t ∈ R. We say that an invariant set C is minimal if C does not contains any proper closed invariant subset.

Lemma 2.10. There exists a minimal set N .

We do not present the proof which is a standard application of Zorn's Lemma (see for instance [START_REF] Vries | Elements of topological dynamics[END_REF]). Proposition 2.11. Let N be a minimal invariant set. If m 0 ∈ N and {m(t), t ∈ R} is a calibrated curve such that m(0) = m 0 , then {m(t), t ∈ R} is dense in N .

Proof. The proof is analogous to the one we used to prove that M is closed. We define C the closure of the trajectory {m(t) t ∈ R}. C is a closed subset of N , in order to prove that it coincides with N we just need to prove that it is invariant or, in other words, that, if m ∈ C, then also a calibrated curve passing through m belongs to C.

Let m be the limit of m n = m(t n ) ∈ {m(t), t ∈ R} and {m n (t)} their corresponding calibrated curves. If w n (t) are the control associated to the calibrated curve m n (t) then, as in Lemma 2.10, we get that w n L 1 is uniformly bounded on any compact set. As we already pointed out, it implies the uniform convergence of m n (t) on compact sets. If m(t) is the trajectory to which m n (t) converges, then it must be a calibrated curve starting from m because we imposed that m n → m. This means that for any s ∈ R, m(s) is the limit of m n (s). As C is closed and m n (s) ∈ C, then m(s) ∈ C.

We proved that C is a not empty, invariant, closed subset of N . By the minimality of N the two sets must conicide.

The role of Monotonicity

So far, the hypothesis on F were mostly about its regularity and no structural assumptions were imposed. On the other hand, when we are interested in understanding whether the limit value λ coincides with λ, the ergodic one, the structure of F does actually play a fundamental role. In the next section we impose convexity and, as it was already proved in [START_REF] Cardaliaguet | Long time average of mean field games with a nonlocal coupling[END_REF], we get that λ = λ. More interestingly, in Section 3.2 we provide a class of explicit examples where λ > λ and, therefore, there is not convergence of the time dependent MFG system to the ergodic one.

The convex case: λ = λ

In this section we will show that under convexity those two values are the same.

We assume the following further assumptions on F :

T d (F (x, m 1 ) -F (x, m 2 ))d(m 1 -m 2 ) ≥ 0 (38)
We introduce the functional J T (m 0 , •, •) : E T (m 0 ) → R defined by

J T (m 0 , m, w) = T 0 T d H * x, - dw(t) dm(t) (x) dm(t) + F(m(t))dt, so that U(T, m 0 ) = inf (m,w)∈E T 2 (m 0 )
J T (m 0 , m, w).

Under the monotonicity assumption (38), the functional J T is convex, therefore we can easily prove that λ = λ. We recall that

-λ = inf (m,w)∈E T d H * x, - dw dm (x) dm(x) + F(m).
Proposition 3.1. Under the above assumptions λ = λ.

Proof. In order to prove the proposition we use the representation of λ that we discussed in Proposition 1.10:

-λ = lim T →∞ 1 T inf Π T (m 0 ,m 1 ) T 0 T d H * (x, α)dm(s) + F(m(s))ds.
Given that λ does not depend on the initial value m 0 , we take m 0 = m 1 . We can also suppose that m 0 is smooth and bounded from below by a positive constant, so that we can apply Proposition 1.10. We now consider any admissible (m T , w T ) for Π(m 0 , m 0 ) and we define mT = 1 T T 0 m T dt and wT = 1 T T 0 w T dt. Given that the m T (0) = m T (T ) = m 0 , the couple ( mT , wT ) verifies, in sense of distributions, -m + div(w) = 0 and it is an admissible competitor for the stationary problem. Now we just need to apply Jensen's inequality to get

J T (m 0 , m T , w T ) = 1 T T 0 T d H * (x, w T /m T )dm T +F(m T )dt ≥ T d
H * (x, wT / mT )d mT +F( mT ).

If we take the infimum over (m T , w T ) ∈ Π T (m 0 , m 0 ) and we take the limit in T , we end up with

-λ ≥ lim T →∞ inf Π T (m 0 ,m 0 ) T d H * (x, wT / mT )d mT + F( mT ) ≥ -λ.
We already proved the opposite inequality in Proposition 1.9, which we recall that it holds true also outside the monotonicity assumption.

As λ = λ, the dynamic programming principle for χ reads χ(m(t 1 )) = If ( m, w) is a minimizer for the static MFG problem then, if we define (m(t), w(t)) = ( m, w) for any t ∈ R, the relation above holds true. This means that the constant trajectory ( m, w) is a calibrated curve so that m ∈ M. Moreover, as the calibrated curve is stationary, the singleton N = { m} is a minimal invariant set because it is closed, invariant and it cannot contain any proper subset.

A non convex example where -λ < -λ

We now present an example where the non convexity of F leads to an ergodic configuration where the limit value -λ is strictly lower then the ergodic one -λ. A straightforward consequence will be that there cannot be any stationary calibrated curve, which in turn implies that any calibrated curve in a minimal invariant set has to be either periodic or chaotic. We say that a calibrated curve m(t) in a minimal invariant set N has a chaotic behavior if its trajectory is strictly included in N .

We fix e d ∈ R d \ {0} a unit vector parallel to one of the axes and we identify T d with T d-1 × T where T is the torus identified by the direction e d . We fix also H such that H * verifies the following conditions: H * (x, p) > 0 for any x ∈ T d , p = -e d and H * (x, -e d ) = 0 for any x ∈ T d .

The assumption (1) on the hamiltonian H implies that there exist two constants C 1 > 0, C 2 > 0 such that

C 1 I d ≤ D pp H * (x, p) ≤ C 2 I d ∀x ∈ T d , ∀p ∈ R d . (39)
Let us define the set A ⊂ P(T d ) as the set of µ ∈ P(T d ) for which there exits µ ∈ P(T d-1 ) such that µ = µ ⊗ dx d , where dx d si the Lebesgue measure on T. Note that µ ∈ A if and only if div(e d µ) = 0.

We fix m 0 : T d → R a smooth, strictly positive density such that m 0 / ∈ A. A measure m belongs to the set B if there exists z ∈ T d such that m(•) = m 0 (• + z). As A and B are closed and disjoint, they are separated by a positive distance ε > 0.

We choose F : P(T d ) → R such that F ≥ 0, F ≡ 2 in A and F ≡ 0 in B. The existence of such a function is ensured by Lemma 4.2 and Lemma 4.3 in Appendix. They also guarantee that we can choose F such that it verifies the regularity assumptions that were in place in the previous sections.

We recall that the functional J T (µ, •, •) is defined on E T 2 (µ) by

J T (µ, m, w) = T 0 T d H * x, - dw(t) dm(t) (x) dm(t) + F(m(t))dt.
In this section we add in the definition of E T 2 (µ) a viscosity constant σ > 0 so that (m, w) verifies -∂ t m + σ m -div(w) = 0.

We also define the ergodic functional J : E → R as follows

J(m, w) = T H * x, - dw(t) dm(t) (x) dm(t) + F(m(t))dt, (40)
where in this framework, (m, w) verifies σ m -divw = 0. According to the definition of λ in (6), we have

-λ = inf (m,w)∈E J(m, w) (41)
Proposition 3.2. There exists a σ 0 > 0 such that for any σ ∈ (0, σ 0 ] we have -λ < -λ.

Proof. We define m(t, x) = m 0 (x -e d t), and w(t, x) = e d m(t, x) + σDm 0 (x -e d t).

The couple (m, w) belongs to E T 2 (m 0 ), so -λ ≤ J T (m, w). By definition of F, we know that F(m(t)) = 0 for any time t. Moreover, since

D pp L ≤ C 2 I d with 0 = H * (x, -e d ) ≤ L(x, α), we have H * (x, α) ≤ 1 2 C 2 |α + e d | 2 . Thus -λ ≤ 1 T T 0 T d C 2 2 w(t, x) m(t, x) -e d 2 m(t, x)dxdt = C 2 2 σ 2 T d |Dm 0 (x)| 2 m 0 (x) dx = σ 2 I, where I = C 2 2 T d |Dm 0 (x)| 2
m 0 (x) dx. We now focus on the static case. We recall that the differential constraint on J is -σ m + divw = 0. By standard arguments we have that there exists a minimizer ( m, w) of (40).

As in the proof of Proposition 1.1 we can define a dual problem which reads inf

(u,λ)∈C 2 (T d )×R {λ s.t. -λ -u + H(x, Du) ≤ F (x, m)} .
Thanks to the regularity of F (•, m) we have a smooth solution (ū, λ) which solves -λ + ū + H(x, Dū) = F (x, m). By duality, if we argue as in Proposition 1.1 (see [START_REF] Briani | Stable solutions in potential mean field game systems[END_REF]) we get that w = -mD p H(x, Dū), so that, by Schauder theory, m is smooth and bounded from below.

We can now estimate -λ. Thanks to the regularity of ( m, w), the parabolic constraint ensures that w = σD m + ζ where ζ is a smooth, divergence free vector field. If ( m, w) = ( m, σD m + ζ) is a minimizer of (40) and we use the growth assumption (39), we have

T d H * x, σD m + ζ m m(dx) + F( m) ≥ C 1 2 T d σ Dm + ζ m + e d 2 m(dx) + F( m). ( 42
)
Note that, as divζ = 0 and m is smooth and bounded from below,

T d ζ • D m/ m = 0. Indeed, T d D m m ζdx = T d D(ln( m))ζdx = - T d ln( m)div(ζ)dx = 0.
Therefore, if we expand the square in (42), we get

T H * x, σD m + ζ m m(dx)+F( m) ≥ C 1 2 T d σ 2 |D m| 2 m + ζ m + e d 2 mdx ≥ C 1 2 T d ζ m + e d 2 mdx.
(43) Plugging ( 43) into (41) we eventually find that

-λ ≥ C 1 2 T d ζ m + e d 2 mdx + F( m).
The righthand side of the above inequality is bounded from below by a positive constant independent of σ. Indeed, we know that, for any σ, m > 0, so

-λ ≥ inf (m,ξ) T d ξ m + e d 2 mdx + F(m)
where the infimum is taken over all the probability densities m > 0 and the free divergence vectors ξ. Here, m and ξ do not verify the elliptic constraint, therefore we lose the dependence on σ.

Let (m n , ξ n ) be a minimizing sequence and m ∈ P(T d ) a the limit of m n (the existence of m is guaranteed by the compactness of P(T d )). If the infimum were achieved at zero then div(m n e d ) → div(me d ) = 0. Indeed, as both the addends should converge to zero, for any test function ϕ, we have

T d div(m n e d )ϕdx = T d m n e d • Dϕdx = T d (m n e d -ξ n ) • Dϕdx ≤ T d |m n e d -ξ| 2 m n dx 1/2 T d |Dϕ| 2 m n dx 1/2 → 0
On the other hand, if div(me d ) = 0, then, by construction of F, we have

F(m n ) → F(m) = 2.
Therefore, there exists a constant K > 0 independent of σ such that -λ > K.

We can conclude the proof choosing σ small enough such that 0 ≤ -λ ≤ σ 2 I < K ≤ -λ.

L (x, -w/ m) dm + F( m) = -λ. As -λ < -λ, it implies χ( m) -χ( m) = t 2 t 1 T d H * x, - w m d m(s) + F( m)ds + λ(t 2 -t 1 ) = (t 2 -t 1 )(-λ + λ) > 0,

Appendix

We present here the result used in Proposition 2.5, which is part of an on-going work with Marco Cirant.

Lemma 4.1. Let ( m, w) be as in Proposition 2.5. For any 0 ≤ t 1 < t where K is the set of u ∈ C 1,2 ([t 1 , t 2 ] × T d ) such that -∂ t u -∆u + H(x, Du) = F (x, m) and T d u(t 1 , x)dx = 0, admits a solution.

Proof. The difficulties of this minimization problem comes from the fact that, a priori, we have no regularity on the measure m, which does not allow us to directly get the compactness of the minimizing sequence that we need. On the other hand the dynamic programming principle of χ and some local in time semiconcavity estimates help to overcome this obstacle.

Let us recall that the dynamic programming principle for χ reads χ(m 0 ) = inf Fist of all we prove that for any t 1 < t 2 < t 3 we have that Āt 3 t 1 = Āt 2 t 1 + Āt 3 t 2 . Indeed, using the duality between the two minimization problems, we have We claim now and we prove later that, if {u n } n is a minimizing sequence for Āt 3 t 1 , then u n uniformly convergences to a function u ∈ C 1,2 ([t 1 , t 3 ) × T d ) on any [t 1 , t] with t < t 3 and u is admissible for Āt t 1 . This implies that the function u is a minimizer for Āt t 1 and in particular for t = t 2 . If we suppose that A t 2 t 1 (u) = Āt 2 t 1 + ε, then we have

A t 3 t 1 (u n ) = A t 2 t 1 (u n ) + A t 3 t 2 (u n ) ≥ A t 2 t 1 (u n ) + Āt 3 t 2 .
If we take the limit n → ∞ on both side, the uniform convergence of u n on [t 1 , t 2 ] and the fact that u n is a minimizing sequence for Āt 3 t 1 give us Āt 3 t 1 ≥ A t 2 t 1 (u n ) + Āt 3 t 2 = A t 2 t 1 (u) = Āt 2 t 1 + ε + Āt 3 t 2 = Āt 3 t 1 + ε which is impossible and so u has to be a minimizer for Āt 2 t 1 . We now prove our claim and we show that the set of functions u ∈ C We consider ξ ∈ R d , |ξ| ≤ 1 and we look at the equation solved by w(t, x) = D 2 u T (t, x)ξ • ξ. We now define w(t, x) = w(t, x)η(t), where η is the cutoff function η(t) = (t -T ) 2 . We choose ξ such that it maximizes sup t,x w(t, x). Rearranging the terms and using the boundedness of D 2 ξξ F we get |D 2 uξ| 2 η ≤ wη + C + 2C|D 2 uξ|η.

As η = 2η 1/2 we can apply the Young's inequality so that η w ≤ η/2|D 2 uξ| 2 + 4 and Here we propose the proof of the existence of the smooth functions that we used in Subsection 3.2. Then (Ψ N ) is a family which is uniformly Lipschitz continuous in P(T d ) and converges to d 1 (•, m 0 ). So, for η > 0 small there exists N large enough such that

Ψ N -d 1 (•, m 0 ) ∞ ≤ η.

Remark 2 . 3 .F

 23 Note that the convex duality between the minimization problems inf (m,w)∈Π(m 0 , m(t)) (x, m(s))dm(s)ds and inf u∈K T d u(x, t)d m(t) -T d u(x, 0)dm 0 holds true independently from the existence of minimizers for the latter one and, therefore, independently from Lemma 4.1.

  ) + F(m(s))ds + χ(m(t 2 )) + λ(t 2 -t 1 ).

F

  ) + F(m(s))ds + χ(m(t)) + λt and that the following convex duality holds true (see Remark 2.3) inf (m,w)∈Π( m(t 1 ), m(t 2 )) (x, m(s))dm(s)ds = -inf u∈K T d u(x, t 2 )d m(t 2 ) -T d u(x, t 1 )d m(t 1 ) .

  1,2 ([t 1 , t 3 ] × T d ) which solves -∂ t u -u + H(x, Du) = F (x, m) in [t 1 , t 3 ] × T d T d u(t 1 , x)dx = 0 (44) is uniformly bounded in C 1 ([t 1 , τ ] × T d )for any τ < t 3 . This gives the local convergence that we used earlier. Without loss of generality we can suppose t 1 = 0 and t 3 = T . As in Lemma 1.3 we argue by semiconcavity.

  If we derive twice in space the HJB equation in (44), then w solves-∂ t w -wη -w + D ξξ H(x, Du)η + 2D ξp H(x, Du) • D 2 uξη +D pp H(x, Du)D 2 uξ • D 2 uξη + D p H(x, Du) • D w = D 2 ξξ F (x, m)η.The cutoff function ensures the existence of a positive interior maximum of w. At the maximum, using also the boundedness of D 2 pp H, the above equation implies-wη -K + 2D ξp H(x, Du) • D 2 uξη + C-1 |D 2 uξ| 2 η ≤ D 2 ξξ F (x, m)η.

1 2 |D 2

 122 uξ| 2 η ≤ C + 2C|D 2 uξ|η, which in turn gives |D 2 uξ| 2 η ≤ C.If w + and w+ are the positive parts of w and w, then we have our semiconcavity estimates because in[0, τ ] × T d (w + η) 2 ≤ ( w+ ) 2 ≤ |D 2 uξ| 2 η 2 ≤ M.Note that M = M (τ ) and it diverges when τ → T . On the other hand the estimates above, along with[START_REF] Cardaliaguet | Long time average of mean field games[END_REF], gives uniform bounds on Du ∞ on [0, τ ] with τ < T .Integrating in space the HJB equation we get|∂ t T d udx| ≤ T d |H(x, Du)| + |F (x, m)|dx ≤ C(τ ).

As T d u

  (0, x)dx = 0, the above inequality ensures that | T d u(t, x)dx| ≤ C(τ ) for any t ≤ τ . This gives us osc(u(t, •)) ≤ | T d u(t, x)dx| + C sup x |Du(t, x)| ≤ C(τ ). As in Lemma 1.3, the boundedness of space derivatives implies also that |∂ t u(t)| ≤ C(τ ) for any t ≤ τ and so the claim.

Lemma 4 . 2 .

 42 For any m 0 ∈ P(T d ) and any > 0, there exists Φ :P(T d ) → R of class C 1 such that Φ(m 0 ) = 1 and Φ = 0 on B c ε (m 0 ). Moreover, we can choose Φ such that D m Φ ∞ ≤ 10/ .and with D x D m Φ bounded. Proof. Let E be the compact set of 1-Lipschitz continuous maps on T d vanishing at 0. Let (φ n ) be a dense family in E consisting of smooth maps. For N large, we consider Ψ N (m) = sup n=1,...,N T d φ n (m -m 0 ).

  Proposition 3.3. Under the hypothesis of Subsection 3.2 the projected Mather set M does not contain any stationary calibrated curve. Moreover, if m 0 belongs to a minimal invariant set N and m(t) is a calibrated curve starting from m 0 then m(t) is either periodic or it has a chaotic behavior.

	Proof. We recall that if m(t) is a calibrated curve then
	χ(m(t 1 )) -χ(m(t 2 )) =	t 2 t 1	T d	H * x, -	w m	dm(s) + F(m(s))ds + λ(t 2 -t 1 ).
	If m(t) is constantly equal to m then we have that	T

  If we use the dynamic programming principle of χ and the fact that ( m, w) is optimal, the expression above is equal to

	Āt 2 t 1 + Āt 3 t 2 =	inf (m,w)∈Π( m(t 1 ), m(t 2 ))	t 2 t 1	T d	H * x, -	w m	dm(s) +	T d	F (x, m(s))dm(s)ds
	+	inf (m,w)∈Π( m(t 2 ), m(t 3 ))	t 3 t 2	T d	H * x, -	w m	dm(s) +	T d	F (x, m(s))dm(s)ds.
			t 3 t 1	T d	H * x, -	w m		d m(s) +	T d	F (x, m(s))d m(s)ds =
	inf (m,w)∈Π( m(t 1 ), m(t 3 ))	t 3 t 1	T d	H * x, -	w m	dm(s) +	T d	F (x, m(s))dm(s)ds = Āt 3 t 1

Next we approximate the sup in the definition of Ψ N . We consider

For δ > 0 small (depending on N ), we have

In particular, for > 0, choose η = /5:

Moreover, if we derive (45) in space we get

To Proof. Let > 0 be the minimal distance between A and B:

Let (m n ) be dense in A and Φ n : P(T d ) → [0, 1] be associated with m n as in Lemma 4.2:

For δ > 0 small and N large, let us set

In particular,

For m ∈ B we have Φ n (m) = 0, so that Ψ δ N (m) = δ ln(N ). As (m n ) is dense, we can choose, for η > 0, N large enough so that max m∈A min n=1,...,N

Then, for m ∈ A, there exists n ∈ {1, . . . , N } with

We now choose η > 0 such that 1 -10 -1 η = 2/3 (which in turns fixes N ), and then δ > 0 small such that δ ln(N ) ≤ 1/3. Then we have inf Then conclusion follows easily.