Dan Goreac
email: dan.goreac@u-pem.fr

Hadjer Hechaichi

Oana-Silvia Serea

Reflected Dynamics: Viscosity Analysis for L ∞ Cost, Relaxation and Abstract Dynamic Programming

Keywords:

HAL is

Introduction

This paper focuses on the min-max control problem [START_REF] Aubin | Differential inclusions. Set-valued maps and viability theory[END_REF] Minimize

h(•, x(•), u(•) L ∞ [t,T ]
over the absolutely continuous solutions of differential variational inequalities of the type

   i) x (s) ∈ f (s, x(s), u(s)) -N K (x(s)) for almost all s ≥ t ii) x(s) ∈ K for all s ≥ t, x(t) = x.

Here, the set K is a nonempty closed subset of R N and N K (x) is the normal cone to K at x ∈ K (see Definition 2 below). The set U is a compact subset of a finite-dimensional space. The dynamics f is a regular function from [0, T ] × R N × U into R N and the control u(•) : [0, T ] → U is a measurable function. The precise assumptions will be made available at a latter time. The reader is invited to note that (2) includes the classical case without reflection because for K = R N , the normal cone N K (x) reduces to the set {0}. Moreover the function h is defined on [0, T ] × K × U and takes values into R.

Existence results for [START_REF] Bardi | Optimal Control and Viscosity Solutions of Hamiltonian Jacobi Bellman Equations[END_REF] are established in [START_REF] Henry | An existence theorem for a class of differential equations with multivalued right-hand side[END_REF] for a convex set. The convexity assumption on the set K has been relaxed in [START_REF] Cornet | Existence of slow solutions for a class of differential inclusions[END_REF] whose author merely requires the tangential regularity. We also refer to [START_REF] Thibault | Sweeping process with regular and nonregular sets[END_REF] for the case of a closed set K for an existence result of viable solution, when the reflection is obtained via the Clarke normal cone. Note that in [START_REF] Thibault | Sweeping process with regular and nonregular sets[END_REF] the set K may depend on the time parameter t.

This kind of dynamics is of particular relevance to mechanics as it is used to study he concept of inelastic shocks (as introduced in [START_REF] Moreau | Liaisons unilatérales sans frottement et chocs inélastiques[END_REF]). For this concept see also [START_REF] Monteiro Marques | Differential Inclusions in Nonsmooth Mechanical Problems -Shocks and Dry Friction[END_REF]. Moreover, numerical methods are developed in [START_REF] Brogliato | Numerical simulation of finite dimensional multibody nonsmooth mechanical systems[END_REF] for the study of systems composed of interacting rigid bodies. One of the methods relies on Moreau's sweeping process which is similar to a reflected problem. Furthermore, the controlled system (2) features properties ensuring the existence of solutions belonging to the fixed set K, while the controlled dynamics remains as close as possible to the original one. For the case of normal directions, the reader is referred to [START_REF] Serea | On reflecting boundary problem for optimal control[END_REF][START_REF] Serea | Reflected differential games[END_REF] and references therein. For control problems with usual costs (integral and/or final) and reflected dynamics, the associated Hamilton-Jacobi equation becomes a Hamilton-Jacobi Partial Differential Inclusion relying on the normal cone to the reflection set. Necessary and sufficient conditions of optimality for this kind of problems with standard costs, their asymptotic behavior, linearisation techniques and the regularity of their value functions are studied in [START_REF] Serea | Semiconcave solutions of partial differential inclusions, Nonlinear Anal[END_REF][START_REF] Serea | Viscosity solutions for partial differential equations with Neumann type boundary conditions and some aspects of Aubry-Mather theory[END_REF][START_REF] Serea | On the regularity of the value function for optimal control problems[END_REF].

In the present paper, we focus on value functions given by a supremum cost of type

V ∞ (t, x) ≡ inf u(•)∈U (t,T ) h (•, x (•, t, x, u(•), u(•))) L ∞ [t,T ] , for all (t, x) ∈ [0, T ] × K. (3)
The set of admissible controls is denoted by U(t, T ) = {u(•) : [t, T ] → U, u measurable}. Here and throughout the paper, unless explicitly needed, we will drop the dependency U(t, T ) and just write U. We consider that x (•, t, x, u(•)) is a solution of (2) associated with the control u(•) ∈ U and starting from (t, x) ∈ [0, T ] × K.

The value function V ∞ is to be characterized as a unique, continuous viscosity solution of the following quasi-variational inequality

(4)        max V ∞ t (t, x) + H(t, x, V ∞ (t, x), V ∞ x (t, x)) -N K (x), V ∞ x (t, x) ; min u∈U |h(t, x, u)| -V ∞ (t, x) 0, (t, x) ∈ [0, T ) × K; V ∞ (T, x) = inf u∈U |h(T, x, u)|, x ∈ K.
Here the Hamiltonian is a function from [0, T ] × R N × (0, ∞) R N to R, defined by H(t, x, r, q) := min{ q, f (t, x, u) : u ∈ U (t, x, r)}, where U (t, x, r) is given by

U (t, x, r) = {u ∈ U : |h(t, x, u)| r}, for every (t, x, r) ∈ [0, T ] × R N × R.
Notice that in the previous inequality some difficulties are introduced by the lower obstacle given by min u∈U |h(t, x, u)| and by the constraints on the control u i.e. the set U (t, x, V ∞ ). Moreover, the value of the Hamiltonian is minimized over a set which depends on the solution and that is why we refer to the inequality (4) as a quasi-variational inequality.

We adopt the L p -approach which already known in the literature for usual non-reflected dynamics, see e.g. [START_REF] Barron | The Bellman equation for minimizing the maximum cost, Nonlinear Anal[END_REF]). Therefore, we rely on the approximating value functions defined by V p (t, x) := inf u(•)∈U ( T t |h(τ, x(τ ), u(τ )| p dτ ) 1/p , for all (t, x) ∈ [0, T ] × K and p ≥ 1. Their associated Hamilton Jacobi Bellman inequalities are given by ( 5)

     V p t (t, x) + H(t, x, V p (t, x), V p x (t, x)) -N K (x), V p x (t, x) 0 if (t, x) ∈ [0, T ) × K;
with the final condition V p (T, x) = 0 if x ∈ K.

Note that the proofs are mostly relegated to the last Section 6.

Preliminaries

Some Notations

For our reader's sake, we gather here some of the classical notations employed throughout the paper • The fixed time horizon is T > 0.

• The state space will be some standard Euclidean space R N (with N ∈ N * = {n ∈ Z : n ≥ 1}) endowed with the usual norm denoted by • .

• For a subset K ⊂ R N , we denote by intK its interior and by ∂K the frontier.

• The set of controls, U , will stand for a compact subset of some Euclidean space R q .

• For notation purposes, we let -B(0, r) stands for the 0-centered, r-radius closed ball of R N (for some r > 0); -For δ > 0 and an initial time/space datum (t, x) ∈ R + × R N , we let B δ (t, x) be the open δ-radius ball of R N +1 with center (t, x). The associated closed ball will be denoted by Bδ (t, x).

• We will consider the following Banach-type spaces:

-C 1 the class of real-valued continuously differentiable functions. Depending on the context, they can be set on [0, T ] × R N or restricted on [0, T ] × K (where K ⊂ R N ). In the last part of the paper, we consider C 1 test functions according to the weak convergence of measures. In this setting, such functions will be defined on

R + × R N × U or R + × R 2N × U ;
-BU C ([t, s] × K) will stand for the family of real-valued bounded uniformly continuous functions defined on [t, s] × K ⊂ R 1+N .

-L p will stand for 1 ≤ p ≤ ∞ -power integrable real-valued functions. If the measure is not specified, it will implicitly be the Lebesgue measure (Leb). Otherwise, we will specify it (e.g., in the last part L p (γ (ds, dy, dz, du)) stands for L p R + × R 2N × U, γ; R where the support R + × R 2N × U is endowed with the Borel sets. -Since the costs will be taken with respect to E := L p (1 ≤ p ≤ ∞), we denote by • E the usual norms on these Banach spaces.

• For a function φ ∈ C 1 R N +1 , φ t and φ x denote the time partial derivative, respectively the space gradient.

• For a Polish space X we denote by P(X) the set of probability measures on X.

Definitions, Assumptions and Further Notations

We assume that f : [0, T ] × K × U → R N is continuous and satisfies the following conditions.

(H f ) f (s, x, u) -f (t, y, u) ≤ M (|s -t| + x -y ) for all t, s ∈ [0, T ], x, y ∈ K, u ∈ U. The set f (t, x, U ) is convex for all t ∈ [0, T ], x ∈ K.
Here, M > 0 is a given real constant. Throughout the paper we assume the following.

The function h is bounded, uniformly continuous with space continuity modulus µ h :

sup t≥0;u∈U |h(t, x, u) -h(t, y, u)| ≤ µ h ( x -y ) , ∀x, y ∈ R N . (H h )
Remark 1 Due to the qualitative aspects of our problem (set in a compact time/space framework), we can assume, without loss of generality that h is strictly positive and bounded away from 0 i.e.

inf t≥0; x∈R N ; u∈U h(t, x, u) ≥ δ > 0.
We continue with recalling the definitions of the tangent and normal cones [START_REF] Aubin | Differential inclusions. Set-valued maps and viability theory[END_REF].

Definition 2 For x ∈ K, we denote by 1 T K (x) := z ∈ R N : lim inf ε→0 + d K (x + εz) ε = 0 ,
the tangent cone to K at x and by

N K (x) = T K (x) -= p ∈ R N : p, z ≤ 0, for all z ∈ T K (x) ,
the normal cone to K at x.

Recall that T K (x) is a closed cone and N K (x) is a closed convex cone. Throughout the paper, we assume a particular feature of the set K, namely that it is a proximal retract. More precisely,

Definition 3 A closed set K ⊂ R n is called a proximal retract or prox-regular if there exists a neighborhood N of K such that the projection Π K : R N → K is single-valued on N .
We recall that, for all x ∈ R N , the projection is defined by

Π K (x) := z ∈ K : x -z = inf y∈K x -y .
The properties of such sets are the key for the proof of the existence and uniqueness results concerning (2) and (4). The class of proximal retracts includes closed, convex sets and C1,1 sets. A complete characterization of proximal retract sets is made in [START_REF] Poliquin | Local differentiability of distance functions[END_REF] (Theorem 4.1, p. 5245). In particular, such sets have the property that there exists ρ > 0 such that every nonzero normal "can be realized" by a ball with a radius equal to ρ. This characterization says specifically that only "exterior" corners are allowed. Moreover, if K is a proximal retract, then the normal cone has a hypo-monotonicity property cf. [START_REF] Poliquin | Local differentiability of distance functions[END_REF]Theorem 4.1] as well as cf. Lemma 4.2 and Theorem 2.2 in [START_REF] Cornet | Existence of slow solutions for a class of differential inclusions[END_REF]. That is, there exist r, c > 0 such that the application x → N K (x) ∩ B(0, r) + cx is monotone2 on K.

We now proceed with recalling some results from [START_REF] Serea | On reflecting boundary problem for optimal control[END_REF][START_REF] Serea | Reflected differential games[END_REF]. Consider (t, x) ∈ [0, T ] × K and denote by S f (t, x) the set of absolutely continuous solutions of

(6) i) x (s) ∈ f (s, x(s), u(s)) -N K (x(s)), for almost all s ≥ t. ii) x(s) ∈ K for all s ≥ t, x(t) = x.
and by S F (t, x) the set of absolutly continuous solutions of

(7) i) x (s) ∈ F (s, x(s)) -N K (x(s)), for almost all s ≥ t. ii) x(s) ∈ K for all s ≥ t, x(t) = x.
As usual, the set-valued function F is defined by F (s, x) := {f (s, x, u) : u ∈ U } .We will also make use of the following standard result (cf. [START_REF] Aubin | Differential inclusions. Set-valued maps and viability theory[END_REF])

Proposition 4 Let us suppose that K is compact and sleek 3 and

f (•, •, U ) is a Marchaud 4 map. Then the map (t, x) → S F (t, x) defined on [0, T ] × K with values in W 1,1 ([0, T ] × K) is u.s.c.
(upper semicontinuous) with non-empty compact images.

Moreover we have the following identity.

Proposition 5 Suppose that K is a compact sleek set and (H f ) holds true.

(i) If x(•) is a solution to (7) starting from (t, x) ∈ [0, T ] × K, then there exists u(•) ∈ U[t, T ] such that x(•) is equal to x(•; t, x, u(•)
), the solution of (6).

(ii) As a direct consequence of (i) we have that S F = S f . Finally, we recall the following continuity result with respect to the initial data.

Lemma 6 Assume that (H f ) holds true and K is a bounded proximal retract. Then for x(•) ∈ S f (t, x) and y(•) ∈ S f (s, y) controlled by u(•) ∈ U(t) and for T ≥ s ≥ t ≥ 0, there exist C > 0, a constant depending on T such that [START_REF] Borkar | Averaging of singularly perturbed controlled stochastic differential equations[END_REF] x(τ

; t, x, u(•)) -y(τ ; s, y, u(•)) ≤ C( x -y + |t -s|) for all τ ∈ [t, T ].

Viscosity solutions

We start by introducing the following definition in order to describe the value function as a unique solution of (4). For different notions of viscosity solutions see for instance [START_REF] Bardi | Optimal Control and Viscosity Solutions of Hamiltonian Jacobi Bellman Equations[END_REF] [4], [START_REF] Crandall | User's guide to viscosity solutions of second order partial differential equations[END_REF]. For our readers' sake, we recall that the Hamiltonian H has been defined in the introduction and H * respectively H * are the relaxed Hamiltonians defined, for (t, x, q, r) In what follows, we focus our attention on the study of L p -control problems and we formulate intermediate results wich will allow us to describe their behavior when p → ∞.

∈ [0, T ] × R N × (0, ∞) × R N as H * (t, x,

L p Control Problems and Asymptotic Behavior

Viscosity Arguments for the Penalization

Recall that the value functions associated to L p control problems are defined by

V p (t, x) := inf u(•)∈U T t |h(τ, x(τ ), u(τ )| p dτ 1 p ,
for all (t, x) ∈ [0, T ]×K and p ≥ 1. Moreover, their associated Hamilton-Jacobi-Bellman inequalities are given by ( 9)

     V p t (t, x) + H p (t, x, V p (t, x), ∇V p x (t, x)) -N K (x), ∇V p x (t, x) 0, if (t, x) ∈ [0, T ) × K; with the final condition V p (T, x) = 0 if x ∈ K. Here, N K (x), ∇V p x (t, x) is the set { q, ∇V p x (t, x) : q ∈ N K (x)} and the Hamiltonian H p : [0, T ] × [1, ∞] × R N × R N → R is defined by H p (t, x, r, q) := min u∈U q, f (t, x, u) + 1 p |h(t, x, u)| p r p-1 .
We begin the study of the L p control problem with the following auxiliary result.

Proposition 8 Suppose that K is a compact prox regular set and the assumptions (H f ) and (H h ) hold true. Then, the value function W defined by

(10) W (t, x) := inf u∈U [t,T ] T t |h(s, x(s; t, x, u(•)), u(s))| p ds, for all (t, x) ∈ [0, T ] × K is BU C([0, T ] × K) and satisfies the final condition W (T, x) = 0, for all x ∈ K.
The proof is rather classical but we provide it in the first subsection of Section 6 for our readers' sake.

Using the previous result we characterize the value function V p as follows.

Proposition 9 Suppose that K is a compact prox regular set and the hypotheses (H f ) and (H h ) hold. Then, for each 1 ≤ p < ∞, V p ∈ BU C([0, T ] × K) and it is the unique continuous viscosity solution of the following Hamilton Jacobi Bellman inequality

0 ∈L p (V p ) := V p t (t, x) + min u∈U { V p x (t, x), f (t, x, u) + 1 p |h(t, x, u)| V p (t, x) p V p (t, x)} -N K (x), V p x (t, x) (11)
if t ∈ [0, T ), x ∈ K satisfying the final condition V p (T, x) = 0 on K.

The proof is postponed to Section 6. Moreover, for determining the equation satisfied by W we employ a version of Proposition 11 from [START_REF] Serea | On reflecting boundary problem for optimal control[END_REF]. Then we use the transformation y → y p and the definition of viscosity solutions in order to find the inequality (11) characterizing V p .

Asymptotic behavior

The aim of this subsection is to prove the existence of the limit lim p→∞ V p and its link with V ∞ . To this purpose, we begin with the following result.

Proposition 10 Let us assume that K is prox regular set and that both (H f ) and (H h ) hold true. Then, the following assertions hold true simultaneously and for all 0 ≤ t ≤ T and all x ∈ K.

(i) V p (t, x) ≤ M (T -t) 1/p . (ii) If p ≤ p then V p (t, x) ≤ V p (t, x)(T -t) (p -p)/pp .
(iii) The limit lim p→∞ V p (t, x) exists.

(iv) The function γ : [0, T ] × K → R given by

(12)      γ(t, x) := lim p→∞ V p (t, x) = lim p→∞ V p (t, x)(T -t) -1/p if t < T, and
γ(T, x) := min u∈U |h(T, x, u)|
is globally lower semicontinuous and continuous at (T, x) for all x ∈ K.

The proof is similar with the original one which is given in [START_REF] Barron | The Bellman equation for minimizing the maximum cost, Nonlinear Anal[END_REF] for the case without reflected dynamics. However, we provide some elements for our readers' sake in Subsection 6.1. We proceed with a result stating that the value function V ∞ coincides with the candidate function γ(•) given by [START_REF] Evans | Partial Differential Equation[END_REF].

Proposition 11 We assume that K is a compact prox regular set and that (H f ) and (H h ) hold true. Then the identity γ(t, x) ≡ V ∞ (t, x) holds for all (t, x) ∈ [0, T ] × K.

The proof which is postponed to Subsection 6.1 is based on convenient estimates given in Lemma 29.

4 The L ∞ optimal control problem 4.1 Regularity of V ∞

We begin with some regularity aspects concerning the value function V ∞ .

Proposition 12 Let us assume that K is a compact prox regular set and that (H f ) and (H h ) hold true. Then the value function

V ∞ ∈ BU C([0, T ] × K) and V ∞ (T, x) = min u∈U |h(T, x, u)|.
The proof is presented in Subsection 6.2. It consists in showing the space (uniform) continuity, followed by the time continuity on [t, T ) and, finally, analyzing the behavior at T . Note that we use the help of time-shifted control policies.

Remark 13

We emphasize that V p converges uniformly to V ∞ on compact subsets of [0, T ) × K. Indeed, in this case the sequence of functions (t, x) → V p (t, x) (T -t)

1
p which is indexed by p satisfies the monotonicity assumption in Dini's (first) theorem. Consequently, the convergence of V p to the continuous function V ∞ is uniform.

Some Tools

This section gathers some results from [5, Section 2], as for example 1. the penalty result in mathematical programming (cf. [5, Proposition 2.1]), 2. basic properties of the Hamiltonian which is discontinuous in our case, 3. the proprieties of the set U (t, x, r), (t, x, r) ∈ [0, T ] × R N × R which is used in the definition of the Hamiltonian (cf. [5, Propositions 2.4, 2.5 and 2.3]).

The previous facts allow to formally infer the viscosity behavior of V ∞ . Moreover, they are important for several proofs contained in the next subsection.

Proposition 14 [5, Proposition 2.1] Let α, β be two real-valued continuous functions on U , where U ⊆ R q is a compact set and let µ and ν ∈ (0, +∞). Then

lim p→∞ min u∈U α(u) + 1 p |β(u)| ν p µ = min {α(u); u ∈ U (ν)} , where U (r) ≡ {u ∈ U : |β(u)| ≤ r} for r > 0.
We continue by recalling some properties of the discontinuous Hamiltonian and of the set U (t, x, r),

(t, x, r) ∈ [0, T ] × R N × R.
Note that, the precise definition appears in the introduction, but we recall it here, for reading purposes:

U (t, x, r) = {u ∈ U : |h(t, x, u)| ≤ r}. Lemma 15 (i) ([5, Proposition 2.3 (i)]). If r ≤ r , then U (t, x, r) ≤ U (t, x, r ).
(ii) ([5, Proposition 2.3 (ii)]). If r < r , x ∈ K and 0 ≤ t < T , then there exists δ = δ(r, r ) > 0 such that U (s, y, r) ⊆ U (t, x, r ) for all (s, y) ∈ B δ (t, x).

(iii) ([5, Lemma 2.4 (i)]). If r ≤ r then, H(t, x, r, q) ≥ H(t, x, r , q).

(iv) ([5, Lemma 2.4 (ii)]). If r < r , x ∈ K, and 0 ≤ t < T , then there exists some δ = δ(r, r ) > 0 such that H(s, y, r, q) ≥ H(t, x, r , q) for all (s, y) ∈ B δ (t, x).

Moreover, let us consider the relaxed Hamiltonians

(13)    H * (t, x, r, q) := lim sup →0 {H(s, y, ρ, m) : |t -s| + ||x -y|| + |ρ -r| + ||q -m|| ≤ }, H * (t, x, r, q) := lim inf →0 {H(s, y, ρ, m) : |t -s| + ||x -y|| + |ρ -r| + ||q -m|| ≤ }. (v) ([5, Proposition 2.5]).
We have that H * (t, x, r, q) = H(t, x, r-, q) and H * (t, x, r, q) ≡ H(t, x, r+, q)5 .

Viscosity Results for V ∞

In this subsection we gather the main results stating the connection between the value function V ∞ and the naturally associated Hamilton-Jacobi-Bellman differential (or quasi-variational) inclusion.

Theorem 16 Suppose that K is a compact prox regular set and the hypotheses (H f ) and (H h ) hold true. Then V ∞ is a viscosity solution of

(14) max      {V ∞ t (t, x) + H(t, x, V ∞ (t, x), ∇ x V ∞ (t, x)) -N K (x), ∇ x V ∞ (t, x) } ; min u∈U |h(t, x, u)| -V ∞ (t, x)      0, if t ∈ (0, T ) and x ∈ K, with the final condition V ∞ (T, x) = min u∈U |h(T, x, u)| on K.
Again, the proof is postponed to Subsection 6.2. One considers admissible test functions for V ∞ and employs the viscosity properties of the approximating functions V p in order to conclude.

Equivalent Formulation and Uniqueness

A somewhat different formulation is given by the following.

Theorem 17 Let us assume that K is a compact prox regular set and that (H f ) and (H h ) hold true. Then, the value function V ∞ (i) is a viscosity solution of the following differential inequality

(15) V ∞ (t, x) + H (t, x, V ∞ (t, x), V ∞ x (t, x)) -N K (x), V ∞ x (t, x) 0, (t, x) ∈ (0, T ) × K; (ii) and satisfies V ∞ (T, x) = min u∈U |h(T, x, u)| on K.
Recall that U (t, x, r) = {u ∈ U : h(t, x, u) ≤ r}, r > 0, and the Hamiltonian is given by

(16) H(t, x, r, q) = min{ q, f (t, x, u) : u ∈ U (t, x, r)}, q ∈ R N .
Conversely, every bounded, uniformly continuous viscosity solution of (15) satisfies (14) in the viscosity sense.

For the proof, please reefer to Subsection 6.2.

Remark 18 A direct proof of Theorem 17 in the spirit of [5, Proposition 2.2] and based on Lemma 15 can also be given. There are no particular difficulties except the need to reprove stability schemes from [START_REF] Ishii | Hamiltonian Jacobi Equation with Discontinuous Hamiltonians on[END_REF] in the framework of differential inclusions.

We end the section with a comparison result implying, in particular, the uniqueness of our viscosity solution.

Proposition 19 Let U and V be two real-valued functions on [0, T ] × K satisfying the following conditions 1. U and V are bounded uniformly continuous (the common bound is denoted by k); 2. U is a viscosity subsolution of (14); 3. V is a viscosity supersolution of (14);

4. U (T, x) = V (T, x) = min u∈U |h (T, x, u)| , x ∈ K. Then U (t, x) ≤ V (t, x), ∀(t, x) ∈ [0, T ] × K.
The proof (given in detail in Subsection 6.2) relies on usual contradiction arguments and makes use of the monotonicity of x → N K (x) ∩ B(0, 1) where Lispchitz-property is usually employed.

Linearization and Abstract Dynamic Programming

The aim of this section is to provide arguments allowing to extend the Bellman dynamic programming principle from the regular framework with uniformly continuous costs h to merely Borelregular costs. The dynamics is seen as occupation measures satisfying convenient constraints given by the state restrictions K and the differential formula. First, the main difficulty and novelty with respect to the abundant existing literature (see for instance [START_REF] Borkar | Averaging of singularly perturbed controlled stochastic differential equations[END_REF][START_REF] Borkar | On existence of limit occupational measures set of a controlled stochastic differential equation[END_REF][START_REF] Serea | Characterization of the optimal trajectories for the averaged dynamics associated to singularly perturbed control systems[END_REF][START_REF] Goreac | Linearization Techniques for Controlled Piecewise Deterministic Markov Processes; Application to Zubov's Method[END_REF]) is the presence of of the reflection elements z ∈ N K (x). Therefore, a further component is added to the occupation measures. This new component is given by the gradient of the Yosida-Moreau penalization or equivalently inf-convolution at the initial level. Moreover, it satisfies convenient restrictions, such as : support conditions compatible with the reflection and the state constraints, respectively a differential-type formula, cf. Lemma 22. Note that Lemma 22 provides the first main result. Second, the limit set of measures Θ ∞ defined by ( 25) needs to comply with convexity and closedness. In the second main result of this part Theorem 26, we prove an abstract semigroup behavior by extending the set of constraints to mimic dynamics issued from measures (cf. ( 26), ( 27)). As a by-product, we obtain dynamic programming principles for L ∞ problems with mere Borel measurable cost functions h.

The results presented here are constructing on some of the authors' previous results and the Moreau-Yosida approximations of the reflected dynamics (e.g. [START_REF] Goreac | Serea Linearization techniques for L ∞ -control problems and dynamic programming principles in classical and L ∞ -control problemsESAIM[END_REF]) and are considerably more challenging than our previous ones. The need for such extensions is equally justified by the optimality conditions through methods developed in [START_REF] Goreac | Optimality Issues for a Class of Controlled Singularly Perturbed Stochastic Systems[END_REF].

Moreau-Yosida Approximations and Their Linearization.

When K is a proximal retract, the differential inclusion (6) can be approximated using a Moreau-Yosida or equivalently inf-convolution argument for the distance function d K . Let n ≥ 1 and consider g n (x) := inf

y∈R N d K (y) + n 2 y -x 2 , ∀x ∈ R N ,
as well as the associated dynamics

(17) x n (t) = f (s, x n (s), u(s)) -∇g n (x n (s)) , for a.e. s ∈ [0, T ].
As before, we let x n (•; t, x, u(•)) designate the unique solution to the equation ( 17) starting from x ∈ R N at time t ∈ [0, T ]. Moreover, we consider the approximating value function

(18) W n (t, x) := inf u∈U (t,T ) T t |h (s, x n (s; t, x, u(•)) , u(s))| p ds, (t, x) ∈ [0, T ] × R N .
Proposition 20 Let us assume that K is a proximal retract and that (H f ), (H h ) hold true. Then, the following assertions hold true.

(i) ([26, Proposition 4.5])There exists some constant C, only depending on the coefficient f , such that6

(19) sup t≤s≤T x n (s; t, x n , u(•)) -x (s; t, x, u(•)) ≤ C √ n + C x n -x ,
for t ∈ [0, T ] and n ≥ 1.

(ii) The function W n is bounded and uniformly continuous for n ≥ 1.

(iii) ([26, Proposition 4.7]) W n converges to W uniformly on [0, T ] × K as n → ∞.
Recall that W is defined by [START_REF] Cornet | Existence of slow solutions for a class of differential inclusions[END_REF].

Assertion (i) can actually be found on the last-but-one line in the proof of [START_REF] Serea | Optimality conditions for reflecting boundary control problems[END_REF]Proposition 4.5]. Assertion (ii) is quite classical i.e. Lipschitz dynamics and bounded, uniformly continuous costs. Assertion (iii) appears under the pointwise form in [START_REF] Serea | Optimality conditions for reflecting boundary control problems[END_REF]Proposition 4.7]. The uniformity follows from (i).

We continue by recalling the following linearization procedure for the value function W n with fixed n ≥ 1. For doing this we have to introduce several notions.

Occupation measures

To every initial couple (t, x), every terminal datum T ∈ (t, T ] and every admissible control u(•) ∈ U (t, T ), we associate a measure

γ t, T ,x,u(•) := γ t, T ,x,u(•) 1 , γ t, T ,x,u(•) 2 ∈ P t, T × R N × U × P R N , by setting 7 (20)
       γ t, T ,x,u(•) 1 (da, dy, dv) := 1 T -t T t 1 (s,(xn(s;t,x,u(•)),u(s))∈(da,dy,dv)) ds, γ t, T ,x,u(•) 2 (dy) := 1 (xn( T ;t,x,u(•))∈dy) ,
where da, dy, dv stand for Borel subsets of t, T , R N , respectively U . Note that, this definition can be extended to T = t by setting γ t,t,x,u(•) 1 = δ t,x,u(t) , where δ stands for the Dirac mass. The family of all occupation measures is denoted by Γ n (t, T , x). Let K n := K + B 0, C n for n ≥ 1. In view of [START_REF] Serea | Optimality conditions for reflecting boundary control problems[END_REF]Proposition 4.4] and of Proposition 20 (i), whenever γ = (γ 1 , γ 2 ) ∈ Γ n (t, T , x), we have that γ ∈ P t, T × K n × U × P (K n ) .

Convex hull

We introduce the set

Θ 0 n t, T , x :=        γ = (γ 1 , γ 2 ) ∈ P t, T × K n × U × P (K n ) : ∀φ ∈ C 1 , R N φ T , y γ 2 (dy) -φ(t, x) = T -t [t, T ]×R N ×U L v n φ(s, y)γ 1 (ds, dy, dv)        . (21)
The generator of this set is given by

L v n φ(s, y) := L v φ(s, y) -∇g n (y), φ x (s, y) = φ t (s, y) + f (s, y, v) -∇g n (y), φ x (s, y) .
where,

L v φ(s, y) := φ t (s, y) + f (s, y, v), φ x (s, y) ,
By known results (see, e.g. [14, Corollary 2.5]) we have that

(22) Θ 0 n t, T , x = co Γ n t, T , x .
Here co stands for the closed convex hull. Note that, by definition, the family Θ 0 n t, T , x is non-empty, convex and compact.

Observe that, by adding a further component z = ∇g n (y), the set Θ 0 n t, T , x can be seen as

Θ n t, T , x :=            γ = (γ 1 , γ 2 ) ∈ P t, T × K n × B(0, r) × U × P (K n ) : z = ∇g n (y), γ 1 (ds, dy, dz, du) -a.s., ∀φ ∈ C 1 , R N φ T , y γ 2 (dy) -φ(t, x) = T -t [t, T ]×R N × B(0,r)×U L v (φ(s, y); z) γ 1 (ds, dy, dz, dv)            . (23)
Here, the infinitesimal generator is defined by

L v (φ (s, y) ; z) := φ t (s, y) + f (s, y, v) -z, φ x (s, y) , ∀ (s, y, z, v) ∈ [0, T ] × R N × R N × U. ( 24
)
Remark 21 In particular,

V p n (t, x) = inf γ∈Θn(t,T,x) (T -t) [t,T ]×R N ×U |h(s, y, v)| p γ 1 (ds, dy, dv) 1 p , ∀(t, x) ∈ [0, T ] × K.
Whenever n ≥ 1 changes, we will explicitly specify it in the occupation measure γ t, T ,x,u(•);n .

The Limit Set of Measures

We define the lower limit set by

(25) Θ ∞ (t, x) :=              γ ∈ P [t, T ] × R 2N × U × P R N : ∃n k ↑ ∞, m k ∈ N, ∃ α k i , x k i 1≤i≤m k ⊂ [0, 1] × K n k , γ k i ∈ Θ n k t, T, x k i s.t. ∀k ≥ 1, m k i=1 α k i = 1, m k i=1 α k i δ x k i k→∞ δ x , m k i=1 α k i γ k i k→∞ γ             
, for every (t, x) ∈ [0, T ]×K. The convergence of the probability measures γ n is in a weak sense. However, by noting that the support of such measures belongs to the compact set [0, T ]× K + B(0, 1) × U × K + B(0, 1) , the convergence can be taken with respect to the Wasserstein metrics. We get the following properties Lemma 22 Let (t, x) ∈ [0, T ] × K be fixed. We have the following assertions

(i) The set Θ ∞ (t, x) is a non-empty, convex, compact subset of P [t, T ] × K × B(0, r) × U × P (K). (ii) If γ ∈ Θ ∞ (t, x), then Supp (γ 1 ) ⊂ (s, y, z, u) ∈ [t, T ] × K × B(0, r) × U : z ∈ N K (y) ∩ B(0, r) . (iii) Let γ ∈ Θ ∞ (t, x) and φ be of class C 1 on [0, T ] × R N . Then, K φ (T, y) γ 2 (dy) = φ(t, x) + [t, T ]×K×B(0,r)×U (T -t) × L v (φ (s, y) ; z) γ 1 (ds, dy, dz, dv).
(iv) We have the linearized formulations

     (T -t) -1 p V p (t, x) = inf (γ 1 ,γ 2 )∈Θ∞(t,x) h L p (γ 1 (ds,dy,R N ,du)) , ∀1 ≤ p < ∞; V ∞ (t, x) = inf (γ 1 ,γ 2 )∈Θ∞(t,x)
h L ∞ (γ 1 (ds,dy,R N ,du)) , whenever 0 < t < T .

Remark 23 1. Condition (i) presents the state constraints i.e. the solution should remain in K, while (ii) gives a compatibility condition with the reflection via N K . The condition (iii) simulates the differential formula. The last condition (iv) links the linearization with the standard value functions V p (1 ≤ p ≤ ∞).

2. We emphasize that a similar method (and identical results!) can be employed to define Θ ∞ t, T , x for arbitrary t < T < T . In this case, Θ ∞ (t, T, x) = Θ ∞ (t, x).

Abstract Dynamic Programming Principles

We recall that for ε > 0 small enough, the projection of any y ∈ K + B(0, ε) on K is unique. By identifying x with δ x , we can extend the above definitions to µ ∈ P K + B(0, ε) by formally setting

(26) Θ n t, T , µ :=                      (γ 1 , γ 2 ) ∈ P t, T × K n × B(0, r) × U × P (K n ) : Supp (γ 1 ) ⊂ (s, y, ∇g n (y), u) ∈ R 2N +1 × U , ∀φ ∈ C 1 , R N φ T , y γ 2 (dy) - N R φ(t, x)µ(dx) = T -t [t, T ]×R N × B(0,r)×U L v (φ(s, y); z) γ 1 (ds, dy, dz, dv)                      and, respectively, ( 27
) Θ ∞ t, T , µ :=                γ ∈ P t, T × R 2N × U × P R N : ∃n k ↑ ∞, m k ∈ N, ∃ α k i , x k i 1≤i≤m k ∈ [0, 1] × K n k , γ k i ∈ Θ n k t, T, x k i s.t. ∀k ≥ 1, m k i=1 α k i = 1, m k i=1 α k i δ x k i k→∞ µ, m k i=1 α k i γ k i k→∞ γ                .
We provide the following consistency properties for the sets we just defined.

Proposition 24

The set Θ n s, T , γ 2 is non-empty, convex and compact for every

t ≤ s ≤ T ≤ T , 1 ≤ n ≤ ∞, x ∈ K n and γ ∈ Θ n (t, s, x). Let t ≤ s ≤ T ≤ T , γ ∈ Θ n (t, s, x) and γ ∈ Θ n s, T , γ 2 . Following [14, Definition 2.8], we define the concatenated measure γ ⊕ γ ∈ P t, T × K + B(0, ε) × B(0, r) × U × P K + B(0, ε) by setting (28)      γ ⊕ γ 1 (dl, dy, dz, du) := s -t T -t γ 1 (dl ∩ [t, s] , dy, dz, du) + T -s T -t γ 1 dl ∩ s, T , dy, dz, du ; γ ⊕ γ 2 := γ 2 .
Remark 25 When γ = γ t, T ,x,u(•);n is an occupation measure, one merely writes

1 T -t T t 1 (r,(xn(r;t,x,u(•)),u(r))∈(da,dy,dv)) dr = s -t T -t 1 s -t s t 1 (r,(xn(r;t,x,u(•)),u(r))∈(da,dy,dv)) dr + T -s T -t 1 T -s T s 1 (
r,(xn(r;t,x,u(•)),u(r))∈(da,dy,dv)) dr, to see that γ t, T ,x,u(•);n = γ s, T ,xn(s;t,x,u(•)),u(•);n ⊕ γ t,s,x,u(•);n .

Moreover, we set

(29) Θ n s, T , • ⊕ Θ n (t, s, x) := γ ⊕ γ : γ ∈ Θ n (t, s, x) , γ ∈ Θ n s, T , γ 2 ,
for every t ≤ s ≤ T ≤ T , 1 ≤ n ≤ ∞ and x ∈ K n . We get the following semigroup behavior of the sets of constraints which constitutes the main result of the section.

Theorem 26 For all t ≤ s ≤ T ≤ T , 1 ≤ n ≤ ∞ and x ∈ K n we have 1. Θ n s, T , • ⊕ Θ n (t, s, x) ⊃ Θ n t, T , x .
2. Moreover, we have the following partial converse:

if γ ⊕ γ ∈ Θ n s, T , • ⊕ Θ n (t, s, x), then γ ⊕ γ •, •, R N , •, • is the marginal of some measure in Θ n t, T , x .
As a direct consequence of the semigroup property, we get the following dynamic programming principle

Corollary 27 (Dynamic Programming Principle) Let h be a Borel-measurable bounded cost function, 0 ≤ t < s < T ≤ T , x ∈ K and µ ∈ P (K). We denote by

V ∞ s, T , µ := inf γ∈Θ∞(s, T ,µ) h L ∞ (γ 1 (dr,dy,R N ,du)) .
Then, we have

(30) V ∞ t, T , x = inf γ∈Θ∞(t,s,x) max h L ∞ (γ 1 (dr,dy,R N ,du)) , V ∞ s, T , γ 2 .
Remark 28 We emphasize, that Lemma (22) (iv) states that V ∞ (t, T, x) = V ∞ (t, x) with the previous definition and with V ∞ (t, x) defined in the previous sections, for 0 ≤ t ≤ T , x ∈ K.

6 Proofs of the Results

Proofs for Section 3

We begin with the proof of the regularity for the function W = (V p ) p .

Proof of Proposition 8. Boundedness follows from the assumptions on h.

To prove the continuity, we let (t n , x n ) n∈N be a sequence in [0, T ] × K which converges to (t, x).

Let ε > 0 be fixed. We consider xn (•) := xn (•; t n , x n , ūn (•)) an ε-optimal solution starting from (t n , x n ) for all n ∈ N. By the estimates [START_REF] Borkar | Averaging of singularly perturbed controlled stochastic differential equations[END_REF],

xn (τ ) -x (τ ; t, x, u n (•)) ≤ C ( x n -x + |t n -t|) , ∀τ ∈ [max {t n ,
(•))), u n (s))| p ds ≥ W (t, x).
It remains to show the inverse inequality i.e W (t, x) + ε ≥ lim n→∞ W (t n , x n ). To this end, we consider ū(•) an ε-optimal trajectory for W at (t, x). We know that x(•; t, x, ū(•)) is the limit along some sub-sequence of (x n (•; t n , x n , ū(•))) n∈N when n → ∞. Indeed, this follows from Proposition 4 and from the uniqueness properties of (2). Hence,

W (t, x) + ε ≥ T t |h(s, x(s; t, x, ū(•)), ū(s))| p ds = lim n→∞ T tn |h(s, x n (s; t n , x n , ū(•)), ū(s))| p ds ≥ lim n→∞ W (t n , x n )
Recall that ε > 0 is arbitrary, therefore we conclude that W is continuous. We now turn our attention to the viscosity equation satisfied by V p .

Proof of Proposition 9. Note that ( 11) is a detailed version of (9).

• We have seen that the function W given by ( 10) is uniformly continuous and bounded. Moreover, it has null final condition at t = T .

• We adapt the proof of Proposition 11 from [START_REF] Serea | On reflecting boundary problem for optimal control[END_REF]. The main difference is that now we are dealing with a running cost. Consequently, one easily checks that W is the unique bounded uniformly continuous viscosity solution of the following inequality (32)

W t (t, x) + min u∈U ( W x (t, x), f (t, x, u) + |h(t, x, u)| p ) -N K (x), W x (t, x) 0, 0 < t < T, x ∈ K, W (T, x) = 0, x ∈ K. • Let V p := (W ) 1
p . We claim that V p satisfies [START_REF] Crandall | User's guide to viscosity solutions of second order partial differential equations[END_REF] in the viscosity sense. We only prove the subsolution condition. Note that the remaining supersolution condition is quite similar and we omit it. To this end, we consider a C 1 -test function φ and (t, x) ∈ (0, T ) × K to be a maximizing argument for V p -φ i.e.

0 = V p (t, x) -φ(t, x) ≥ V p (s, y) -φ(s, y), ∀(s, y) ∈ (0, T ) × R N .
Recalling that W is lower-bounded by δ > 0 (see Remark 1), it follows that the function ψ := φ p is a convenient (bounded, C 1 -regular) test function for W and (t, x) realizes a maximum for W -ψ. We argue following two cases:

1. x ∈ int K. In this case, the subsolution condition for W at (t, x) using the test function ψ gives (see [START_REF] Serea | On reflecting boundary problem for optimal control[END_REF]Definition 3])

ψ t (t, x) + min u∈U ( ψ x (t, x), f (t, x, u) + |h(t, x, u)| p ) ≥ 0.
Recall that ψ = φ p and the fact that φ(t, x) > δ. Therefore, we can divide the previous inequality by pφ p-1 and get that

φ t (t, x) + min u∈U φ x (t, x), f (t, x, u) + 1 p |h(t, x, u)| φ(t, x) p φ(t, x) ≥ 0.
2. x ∈ ∂K. In this case, the subsolution condition for W at (t, x) using the test function ψ (see [START_REF] Serea | On reflecting boundary problem for optimal control[END_REF]Definition 3]) guarantees the existence of some z ∈ N K (x) such that

ψ t (t, x) + min u∈U ( ψ x (t, x), f (t, x, u) + |h(t, x, u)| p ) -z, ψ x (t, x) ≥ 0.
Recall that ψ = φ p and the fact that φ(t, x) > δ. Therefore we can divide the previous inequality by pφ p-1 and get that

φ t (t, x) + min u∈U φ x (t, x), f (t, x, u) + 1 p |h(t, x, u)| φ(t, x) p φ(t, x) -z, φ x (t, x) ≥ 0.
It follows that V p is a viscosity subsolution of [START_REF] Crandall | User's guide to viscosity solutions of second order partial differential equations[END_REF]. As indicated, the supersolution part is quite similar.

Let us now sketch the proof of the uniqueness part. Let V be another uniformly continuous, bounded away from 0 viscosity solution to [START_REF] Crandall | User's guide to viscosity solutions of second order partial differential equations[END_REF]. Then, the function W := (V ) p is bounded, uniformly continuous. By using the same argument as the previous step, one easily shows that W satisfies (32) in the viscosity sense. Note that we can easily adapt [START_REF] Serea | On reflecting boundary problem for optimal control[END_REF]Theorem 12] to running instead of final costs. Consequently, by uniqueness of the solution of (32), W = W which implies the uniqueness of V p .

We continue with the result gathering the monotonicity for V p and the existence of the limit as p → ∞.

Proof of Proposition10. The proof is similar with the original one which is given in [START_REF] Barron | The Bellman equation for minimizing the maximum cost, Nonlinear Anal[END_REF] for the case without reflected dynamics. However, we provide some elements for our readers' sake.

(i) The first inequality follows easily from (H f ) by recalling that M has been chosen no lower than the maximum of h.

(ii) The second inequality follows from Hölder's inequality for the integral cost.

(iii) We focus on the case when t < T and x ∈ K is fixed. For t = T , the boundary conditions for V p imply that the expected limit is 0. Due to (ii) we have that

V p (t, x)(T -t) -1/p ≤ V p (t, x)(T -t) -1/p if p ≤ p . Moreover, (i) implies that V p (t, x)(T -t) -1
p ≤ M for p ≥ 1. As a consequence lim p→∞ V p (T -t) -1/p exists. We infer that the limit

lim p→∞ V p (t, x) = lim p→∞ V p (t, x)(T -t) -1/p (T -t) 1/p
exists for all 0 ≤ t < T . To this purpose, we make use of the trajectory estimates in [START_REF] Borkar | Averaging of singularly perturbed controlled stochastic differential equations[END_REF]. For u ∈ U , we consider the constant control (still designated by u) and note that

|h (s, x (s; t, x, u) , u)| ≤ |h (T, x, u)| + µ h (C(T -t)), ∀t ≤ s ≤ T.
Recall that the generic constant C > 0 is time and space-independent. It follows that

1 T -t T t |h (s, x (s; t, x, u) , u)| p ds 1 p ≤ |h (T, x, u)| + µ h (C(T -t)).
By taking the infimum over u ∈ U , one gets lim

t→T V p (t, x) (T -t) -1 p ≤ min u∈U |h(T, x, u)| . For the converse, one writes, |h (s, x (s; t, x, u (•)) , u(s))| ≥ min v∈U |h (s, x (s; t, x, u (•)) , v)| ,
for u (•) ∈ U(t, T ). The conclusion follows from estimates on x (s; t, x, u (•)) -x . Our proof is now complete.

Before giving the proof of Proposition 11, we will need to make some additional notation and prove an intermediate result.

For u(•) ∈ U[t, T ] and a > 0, we define the set θ u(•) a by setting

θ u(•) a := {τ ∈ [t, T ] : |h(τ, x(τ ; t, x, u(•)), u(τ ))| > a} .
We have the following result.

Lemma 29

We assume K to be a compact prox regular set (H f ) and (H 2 ) to hold true. If, for some admissible control u ∈ U ([t, T ]) and some η ∈ (0, T -t], the (Lebesgue) measure of θ

u(•) a , satisfies Leb θ u(•) a < η,
then there exists a control ũ ∈ U[t, T ] such that

|h (τ, x (τ ; t, x, ũ(•)) , ũ(τ ))| ≤ a + µ h (C √ η) for t ≤ τ ≤ T.
Furthermore, C can be chosen independently of τ , x, ũ and η.

Proof of Lemma 29. For simplicity we consider that t = 0. We consider a partition of the interval [0, T ] as follows:

0 = t 0 < t 1 < . . . < t k = T, η ≤ t j -t j-1 ≤ 2η, ∀1 ≤ j ≤ k. Since Leb θ u(•) a < η, it follows that [t j-1 , t j ) \ θ u(•) a = ∅ for 1 ≤ j ≤ k. We pick s j ∈ [t j-1 , t j ) \ θ u(•) a
for each j and we define ũ(•) by setting

ũ(τ ) = u(τ )1 τ ∈[0,T ]\θ u(•) a + 1≤j≤k u(s j )1 τ ∈[t j-1 ,t j )∩θ u(•) a .
To simplify notations, we let x(•) := x (•; 0, x, u(•)) and x(•

) := x (•; 0, x, ũ(•)). Recall that x → N K (x) ∩ B(0, r) + cx is monotone. Consequently, x(τ ) -x(τ ) 2 ≤2 τ 0 f (s, x(s), u(s)) -f (s, x(s), ũ(s)) , x(s) -x(s) ds + c τ 0 x(s) -x(s) 2 ds ≤2 τ 0 x(s) -x(s) M x(s) -x(s) + 2C1 u(s) =ũ(s) ds + c τ 0 x(s) -x(s) 2 ds ≤ C τ 0 x(s) -x(s) 2 ds + η ,
for every 0 ≤ τ ≤ T . In the previous inequality we have a term 2C1 u(s) =ũ(s) and the constant C comes from an upper-bound for the coefficient f on the compact set [0, T ] × K × U . Note that the generic constant C is independent of x, u, ũ, τ ∈ [0, T ] and η and it will be allowed to change from one line to another. Gronwall's inequality yields

x(τ ) -x(τ ) 2 ≤ Cη.
Consequently, we have

|h(τ, x(τ ), ũ(τ )| ≤ |h(τ, x(τ ), u(τ )| + µ h ( x(τ ) -x(τ ) ) ≤ a + µ h (C (1 + x ) √ η) , for τ / ∈ θ u(•)
a . Moreover, due to estimates (8), we have

|h(τ, x(τ ), ũ(τ )| ≤|h(s j , x(s j ), u(s j )| + µ h ( x(s j ) -x (s j ) + x (s j ) -x(τ ) + |τ -s j |) ≤ a + µ h (C √ η), for τ ∈ [t j-1 , t j ) ∩ θ u(•)
a . The proof is now complete. We can now provide the proof for Proposition 11.

Proof of Proposition 11.

Let us prove the following inequality

(34) γ(t, x) ≤ V ∞ (t, x) for all (t, x) ∈ [0, T ) × K.
For every fixed u(•) ∈ U[t, T ] and for every p ≥ 1 we have that

(T -t) -1 p h(•, x(•; t, x, u), u(•)) L p ([t,T ]) ≤ h(•, x(•; t, x, u), u(•)) L ∞ ([t,T ])
because of Hölder's inequality. The inequality (34) follows by taking the infimum over U[t, T ] on both sides and taking the limit when p → ∞ on the left side.

2. We aim to proving the converse of (34) and reason by contradiction. We assume that, for some (t, x) ∈ [0, T ) × K and some ε > 0, γ(t, x) < V ∞ (t, x) -ε. As a consequence, for some subsequence p → ∞ (still denoted by (p) p≥1 ), we have that V p (t, x) < V ∞ (t, x) -ε. Therefore, there exists a control u p (•) ∈ U (t, T ) such that

T t |h (s, x (s; t, x, u p (•)) , u p (s))| p ds ≤ V ∞ (t, x) - ε 2 p .
Using the notation from Lemma 29, it follows that

Leb θ up(•) V ∞ (t,x)-ε 3 = Leb |h (s, x (s; t, x, u p (•)) , u p (s))| > V ∞ (t, x) - ε 3 ≤ V ∞ (t, x) -ε 2 V ∞ (t, x) -ε 3 p .
Applying Lemma 29, one constructs a family of controls (ũ p (•)) p≥1 ⊂ U (t, T ) such that

V ∞ (t, x) ≤ sup t≤s≤T |h (s, x (s; t, x; ũp (•)) , ũp (s))| ≤ V ∞ (t, x) - ε 3 + µ h C V ∞ (t, x) -ε 2 V ∞ (t, x) -ε 3 p 2 .
Letting ε → 0 leads to a contradiction and, thus, completes our proof.

Proofs for Section 4

The first proof concerns the regularity of V ∞ . Proof of Proposition 12. Observe that the assumption (H h ) implies that V ∞ is bounded.

i) We will first establish that V ∞ is (uniformly) continuous with respect to the state variable x.

To this end, we fix 0 ≤ t ≤ T , x 1 , x 2 ∈ K and u(•) ∈ U(t, T ). To simplify the notation, we let x i (•) be the trajectories corresponding to u(•) with the initial position x i ; i = 1, 2. [22, Lemma 3] yields the existence of some generic constant C (independent of x i , u(•), s) such that

x 1 (s) -x 2 (s) ≤ C x 1 -x 2 , for all s ∈ [t, T ].
Consequently, we have that

h(•, x 1 (•), u(•)) L ∞ ([t,T ]) -h(•, x 2 (•), u(•)) L ∞ ([t,T ]) ≤ sup t≤s≤T |h(s, x 1 (s), u(s)) -h(s, x 2 (s), u(s))| ≤ µ h (C x 1 -x 2 ) .
For ε > 0, one picks an ε-optimal control u(•) ∈ U(t, T ) at (t, x 2 ) and, due to the previous inequality,

V ∞ (t, x 1 ) -V ∞ (t, x 2 ) ≤ ε + µ h (C x 1 -x 2 ) . Since ε > 0 is arbitrary, one deduces V ∞ (t, x 1 ) -V ∞ (t, x 2 ) ≤ µ h (C x 1 -x 2 )
. By changing the roles of x i , one establishes uniform continuity.

ii) Secondly, to establish that

V ∞ is continuous in t, we fix x ∈ K, 0 ≤ t 1 ≤ t 2 ≤ T and u(•) ∈ U(t, T ). Let x i (•) := x(•; t i , x, u(•)
) be the trajectories corresponding to u(•) with the initial time t i , i = 1, 2. Standard estimates yield

x 1 (t 2 ) -x = x (t 2 ; t 1 , x, u(•)) -x ≤ C (1 + x ) (t 2 -t 1 ) .
Due to [START_REF] Borkar | Averaging of singularly perturbed controlled stochastic differential equations[END_REF], it follows that (35) sup

t 2 ≤s≤T x 1 (s) -x 2 (s) ≤ C (t 2 -t 1 )
.

Consequently, we have

|h(s, x 1 (s), u(s)) -h(s, x 2 (s), u(s))| ≤ µ h (Cδ) , with δ := |t 2 -t 1 | and s ∈ [t 2 , T ]. Therefore, (36) h(
•; x 1 (•, t 1 , x, u(•)) L ∞ [t 1 ,T ] ≥ h(•; x 1 (•, t 1 , x, u(•)) L ∞ [t 2 ,T ] ≥ h(•; x 2 (•, t 2 , x, u(•)) L ∞ [t 2 ,T ] -µ h (Cδ) .
We let η(•) := x (•; t 2 , x, u (• -δ)) and write the differential formula for 1 2 x 1 (s -δ) -η(s) 2 . Using the fact that the map y → N K (y) ∩ B(0, r) + cy is monotone, and the assumption (H f ), we get

x 1 (τ -δ) -η(τ ) 2 ≤ τ t 2 f (s -δ, x 1 (s -δ), u(s -δ)) -f (s, η(s), u(s -δ)) x 1 (s -δ) -η(s) ds + c τ t 2 x 1 (s -δ) -η(s) 2 ds ≤C δ 2 + τ t 2 x 1 (s -δ) -η(s) 2 ds , ∀t 2 ≤ τ ≤ T.
Grönwall's inequality yields sup

t 2 ≤τ ≤T x 1 (τ -δ) -η(τ ) ≤ Cδ. As a consequence, |h (s, x 1 (s), u(s)) -h (s + δ, η(s + δ), u(s))| ≤ µ h (Cδ) , ∀t 1 ≤ s ≤ T -δ, which implies (37) h (•, x (•; t 1 , x, u(•)) , u(•)) L ∞ [t 1 ,T -δ] ≤ V ∞ (t 2 , x) + µ h (Cδ).
On the other hand, using (35) and the uniform continuity of V ∞ (t 2 , •) whose continuity modulus is denoted by ω, one gets

h (•, x (•; t 1 , x, u(•)) , u(•)) L ∞ [t 2 ,T ] = h (•, x (•; t 2 , x 1 (t 2 ) , u(•)) , u(•)) L ∞ [t 2 ,T ] ≤ V ∞ (t 2 , x 1 (t 2 )) ≤ V ∞ (t 2 , x) + ω (C(1 + x )δ) . (38)
Putting together (37) and (38) yields

h (•, x (•; t 1 , x, u(•)) , u(•)) L ∞ [t 1 ,T ] ≤ V ∞ (t 2 , x) + ω (Cδ) + µ h (Cδ),
provided that T ≥ t 2 + δ. Together with (36) this inequality guarantees the desired continuity of V ∞ on [0, T ).

iii) It remains to show that

(39) lim t→T V ∞ (t, x) = min u∈U |h(T, x, u)|.
We consider u ∈ U and the constant control u(•) = u. For notation purposes, we set x(•) := x (•; t, x, u). We have

V ∞ (t, x) ≤ max t≤s≤T |h(s, x(s), u| ≤ µ h (C(T -t)) + |h(T, x, u)|.
We conclude that lim Now we turn our attention to the main theorem of Section 4 connecting V ∞ and the associated quasi-variational inequality.

Proof of Theorem 16. Let us consider ϕ ∈ C 1 ([0, T ] × K) be such that V ∞ -ϕ has a strict minimum at the point (t, x) ∈ (0, T ) × K. We assume, without loss of generality, that the value of this minimum is 0. Recall that lim p→∞

V p = V ∞ uniform on Bη (t, x) ⊂ [0, T ] × K, for some η > 0. Consequently, one has inf (s,y)∈∂ Bη(t,x) (V ∞ (s, y) -ϕ(s, y)) > 0.
Then, for every large enough p, there exist a point (t p , x p ) ∈ B η (t, x) such that V p -ϕ has a minimum at (t p , x p ), ϕ(t p , x p ) > 0 8 and (t p , x p ) → (t, x) as p → ∞. The viscosity supersolution condition written for V p at (t p , x p ) with the test function ϕ yields the existence of some z p ∈ N K (x p ) ∩ B(0, r) such that (40)

ϕ t (t p , x p ) + min u∈U ϕ x (t p , x p ) , f (t p , x p , u) + 1 p |h (t p , x p , u)| ϕ (t p , x p ) p ϕ (t p , x p ) -z p , ϕ x (t p , x p ) ≤ 0.
8 consequence of the fact that h can be seen as bounded away from 0 and

V p (t, x) → p→∞ V ∞ (t, x) = φ(t, x).
1. We point out that

ϕ t (t p , x p ) + min u∈U { ϕ x (t p , x p ) , f ((t p , x p , u) } -z p , ϕ x (t p , x p ) ≤ϕ t (t p , x p ) + min u∈U ϕ x (t p , x p ) , f (t p , x p , u) + 1 p |h (t p , x p , u)| ϕ (t p , x p ) p ϕ (t p , x p ) -z p , ϕ x (t p , x p ) ≤ 0.
As a consequence, we get the existence of some constant K > 0 (independent of p and determined by the bounds of f and ϕ on Bη (t, x)) such that

min u∈U 1 p |h(t p , x p , u| ϕ(t p , x p ) p ϕ(t p , x p ) ≤ K.
Moreover we have that

1 p   min u∈U |h(t p , x p , u| ϕ(t p , x p )   p ≤ K ϕ(t p , x p ) .
We let p → ∞ to conclude that ϕ(t, x) ≥ min u∈U |h(t, x, u)|.

2. We fix, for the time being, ε > 0. The reader is invited to recall that the map

x → N K (•)∩B(0, r) has closed graph i.e. if (x p , z p ) p∈N converges to (x, z) with z p ∈ N K (x p ) ∩ B(0, r), then z ∈ N K (y) ∩ B(0, r).
Using the continuity of h, ϕ, ϕ x , we get the existence of some great enough p ε such that, for some subsequence p ≥ p ε ,

min u∈U ϕ x (t p , x p ) , f (t p , x p , u) + 1 p |h (t p , x p , u) | ϕ (t p , x p ) p ϕ (t p , x p ) -z p , ϕ x (t p , x p ) ≥ min u∈U ϕ x (t, x) , f (t, x, u) + 1 p |h (t, x, u) | -ε ϕ (t, x) + ε p ϕ (t, x) -z, ϕ x (t, x) -ε (41)
Then, using (40), Proposition 14 and letting p → ∞ in the previous estimation we obtain that ϕ t (t, x) + min u∈U (t,x,ϕ(s,y)+2ε)

{ ϕ x (t, x), f (t, x, u) -z, ϕ x (t, x) } ≤ ε, ∀ε > 0 for some z ∈ N K (y). Consequently, ϕ t (t, x) + H (t, x, ϕ(t, x)+, ϕ x (t, x)) -z, ϕ t (t, x) ≤ 0.

Recalling that ε > 0 is arbitrary, we conclude that V ∞ is a viscosity supersolution of ( 14).

3. The subsolution argument is quasi-identical and therefore, it will be omitted.

Let us provide the proof for the equivalent formulation of the differential system whose solution is V ∞ .

Proof of Theorem 17.

1. Observe that the result is, of course, valid for every bounded, uniformly continuous solution of ( 14) denoted by V . Indeed, let us consider (t, x) ∈ (0, T ) × K and a C 1 -test function such that (t, x) attains a local maximum of (V -ϕ). One gets the existence of some z ∈ N K (x) ∩ B(0, r) such that

max ϕ t (t, x) + H (t, x, ϕ(t, x)+, ϕ x (t, x)) -z, ϕ t (t, x) , min u∈U |h(t, x, u)| -ϕ(t, x) ≥ 0.
If ϕ t (t, x) + H (t, x, ϕ(t, x)-, ϕ x (t, x)) -z, ϕ t (t, x) < 0, then, for some ε > 0, one has ϕ t (t, x) + H (t, x, ϕ(t, x) -ε, ϕ x (t, x)) -z, ϕ t (t, x) < -ε.

By picking u ∈ U (t, x, ϕ(t, x) -ε) one has |h(t, x, u)| ≤ ϕ(t, x) -ε. As a consequence,

min v∈U |h(t, x, v)| ≤ ϕ(t, x) -ε,
which leads to a contradiction. This shows that V is indeed a subsolution of [START_REF] Goreac | Linearization Techniques for Controlled Piecewise Deterministic Markov Processes; Application to Zubov's Method[END_REF]. The supersolution part is immediate.

2. To prove the converse, we focus on the supersolution assertion. To this purpose, we let V be a bounded, uniformly continuous viscosity supersolution to [START_REF] Goreac | Linearization Techniques for Controlled Piecewise Deterministic Markov Processes; Application to Zubov's Method[END_REF]. We consider (t, x) ∈ (0, T ) × K and a C 1 -test function such that (t, x) attains a local minimum of (V -ϕ). Moreover, we assume that V (t, x) = ϕ(t, x). Then, the supersolution condition for [START_REF] Goreac | Linearization Techniques for Controlled Piecewise Deterministic Markov Processes; Application to Zubov's Method[END_REF] written at (t, x) with the test function ϕ yields the existence of some z ∈ N K (x) ∩ B(0, r) such that ϕ t (t, x) + H (t, x, ϕ(t, x)+, ϕ x (t, x)) -z, ϕ x (t, x) ≤ 0.

In particular, it follows that U (t, x, ϕ(t, x)) is nonempty. As a consequence,

min u∈U |h (t, x, u)| ≤ ϕ(t, x),
which shows that V is equally a supersolution for [START_REF] Goreac | Serea Linearization techniques for L ∞ -control problems and dynamic programming principles in classical and L ∞ -control problemsESAIM[END_REF]. The subsolution condition is immediate.

To end the subsection, we provide the proof of the comparison result for sub and supersolutions.

Proof of Proposition 19. We reason by contradiction and assume that, for some (t 0 , x 0 ) ∈ (0, T ) × K, U (t 0 , x 0 ) -V (t 0 , x 0 ) > η > 0.

1. One easily checks that, for every α, β > 0, the modified function

(0, T ) × K (s, y) → W β (s, y) := V (s, y) + β s ≥ V (s, y), 0 < s < T, is a viscosity supersolution of W t (t, x) + H (t, x, W (t, x), W x (t, x)) -N K (x), W x (t, x) + β t 2 0.
2. We fix β > 0 small enough such that

U (t 0 , x 0 ) -W β (t 0 , x 0 ) > η.
For notation purposes, we drop the superscript in W β .

3. We consider ε > 0. Let Φ ε (s, x, y) := U (s, x) -W (s, y) -1 2ε x -y 2 for all s ∈ (0, T ], x, y ∈ K. We note that |U (s, x) -W (s, y)| ≤ 2k -β t ≤ 0, for all t ≤ β 2k . Let (t ε , x ε , y ε ) be a maximum of Φ ε on (0, T ]×K 2 (or, equivalently on the compact β 2k , T ×K 2 ). Then, on some subsequence, we have the following facts:

• 2k ≥ Φ ε (t ε , x ε , y ε ) ≥ Φ ε (t 0 , x 0 , x 0 ) > η; • lim ε→0 t ε = t * ∈ β 2k , T × K 2 ; lim ε→0 x ε = x * ; lim ε→0 y ε = y * ; • x * = y * ;
• η ≤ U (t * , x * ) -V (t * , x * ) and, consequently, t * = T . As a by-product, one assumes, without loss of generality, that t ε = T .

• As usual, lim ε→0

1 2ε x ε -y ε 2 = 0.
The supersolution condition for W at (t ε , y ε ) yields the existence of some z ε ∈ N K (y ε ) ∩ B(0, r) such that

H t ε , y ε , W (t ε , x ε ) -, x ε -y ε ε -z ε , x ε -y ε ε + β t 2 ε ≤ 0. (42)
Here, we used the usual test function constructed from Φ ε . Similarly, the subsolution condition for U at (t ε , x ε ) yields the existence of some

z ε ∈ N K (x ε ) ∩ B(0, r) such that H t ε , x ε , U (t ε , y ε ) +, x ε -y ε ε -z ε , x ε -y ε ε ≥ 0. ( 43
)
We have proven that U (t ε , x ε ) -V (t ε , y ε ) > η. Consequently, we can separate them i.e. there exist ρ 1 and ρ 2 such that

U (t ε , x ε ) > ρ 1 > ρ 2 > V (t ε , y ε ) .
Let ε be small enough such that y ε ∈ B δ (x ε ). Using Lemma 15 (iv), ( 42), ( 43) and the monotonicity of x → N K (x) ∩ B(0, r)) we obtain that

β t 2 ε ≤H t ε , x ε , U (t ε , y ε ) +, x ε -y ε ε -H t ε , y ε , W (t ε , x ε ) -, x ε -y ε ε + z ε -z ε , x ε -y ε ε ≤H t ε , x ε , ρ 1 , x ε -y ε ε -H t ε , y ε , ρ 2 , x ε -y ε ε + z ε -z ε , x ε -y ε ε ≤0 + z ε -z ε , x ε -y ε ε ≤ c x ε -y ε 2 ε .
Letting ε → 0 leads to a contradiction and completes our proof.

Proofs for Section 5

We begin with the proof of the support characterizations of Θ ∞ (state and differential formula), the associated linearized formulation and their compatibility with V p (1 ≤ p ≤ ∞).

Proof of Lemma 22.

(i) Observe that the measures γ n ∈ Θ n (t, T, x) ⊂ P t, T × K n × U × P (K n ) are defined on a compact space. Therefore, there exists a subsequence converging weakly to some γ. Consequently, the definition of Θ ∞ (t, x) is consistent.

In order to verify the convexity, let us consider β ∈ (0, 1) and γ 1 , γ 2 ∈ Θ ∞ (t, x). There exist

n j k ↑ ∞, m j k ∈ N, α j,k i , x j,k i ∈ [0, 1] × K n j k , γ j,k i ∈ Θ n j k t, T, x j,k i s.t. ∀k ≥ 1, m j k i=1 α j,k i = 1, m j k i=1 α j,k i δ x j,k i k→∞ δ x , m j k i=1 α j,k i γ j,k i k→∞ γ j for j ∈ {1, 2}. Since Θ n j k t, T, x j,k i
is the closed convex hull of occupation measures, there exist families of convex parameters, respectively controls in U(t, T ) denoted by (ρ, u) such that γ j,k i can be replaced with l ρ j,k i,l γ t,T,x j,k i ,u j,k i,l ;n j k for j ∈ {1, 2}. 9 In other words, there exists

n j k ↑ ∞, mj k ∈ N, ᾱj,k i , xj,k i ∈ [0, 1] × K n j k , u j,k i ∈ U(t, T ) s.t. ∀k ≥ 1, mj k i=1 ᾱj,k i = 1, mj k i=1 ᾱj,k i δ xj,k i k→∞ δ x , mj k i=1 ᾱj,k i γ t,T,x j,k i ,u j,k i ;n j k k→∞ γ j .
Observe that we can take the elements n j k , x j,k i independent of j. Indeed, if n 1 1 < n 2 1 , we add to the representation of γ 2 , the element

m2 1 i=1 ᾱ2,k i γ t,T,x 2,1 i ,u 2,1 i ;n 1
1 . The errors of these measures are of type 1 n 1 k (for k = 1) and go to 0 as k → ∞. Finally, by taking 0 coefficients, we can put together xj,k i (hence, also m j k , for j ∈ {1, 2}). To summarize: there

exist n k ↑ ∞, mk ∈ N, ᾱj,k i , xk i ∈ [0, 1] × K n k , u j,k i ∈ U(t, T ) s.t. ∀k ≥ 1, mk i=1 ᾱj,k i = 1, lim k→∞ mk i=1 ᾱj,k i δ xk i = δ x , mk i=1 ᾱj,k i γ t,T,x k i ,u j,k i ;n k k→∞ γ j , j ∈ {1, 2} . Then, observe that α k i := β ᾱ1,k i + (1 -β)ᾱ 2,k i γ k i := 1 α k i β ᾱ1,k i γ t,T,x k i ,u 1,k i ;n k + (1 -β)ᾱ 2,k i γ t,T,x k i ,u 2,k i ;n k ∈ Θ n k t, T, xk i ,
whenever α k i = 0 because of the convexity of Θ n k t, T, xk i . Finally, note that the triplet α k i , x i k , γ i k provides the desired representation for βγ 1 + (1 -β)γ 2 which implies that Θ ∞ (t, x) is convex. Verifying that Θ ∞ (t, x) is closed follows from a similar argument. We continue with verifying the support claim. Recall that, by definition, the measures

γ k i are supported by [t, T ] × K n k × B(0, r) × U × K n k , it follows that R 2N +1 ×U d K (y) m k i=1 α k i γ k i 1 (ds, dy, dz, du) + R N d K (y) m k i=1 α k i γ k i 2 (dy) ≤ C n k , ∀k ≥ 1.
Passing to the limit yields the K-support claim.

(ii) We recall that, for ε > 0 small enough, the projection of any y ∈ K + B(0, ε) on K is unique and we have that ∇q n (y) ∈ N K (Π(y)). In other words, we can apply the previous argument by considering the distance to the closed set K ⊗ N

K := {(y, z) : y ∈ K, z ∈ N K } to see that d K⊗N K (y, z)γ n k 1 (ds, dy, dz, du) ≤ C n k ,
where γn k := m k i=1 α k i γ k i . The conclusion follows, as before, by passing to the limit k → ∞.

(iii) This assertion follows from the continuity of the generator and from passing to the limit k → ∞ in the similar formula written for γ k i , and the starting points x k i .

(iv) Let us consider x n ∈ K n a sequence (x n ) converging to x ∈ K.

≤µ h C √ n + C x n -x . (44)
As we have already hinted before,

V p n (t, x n ) = inf γ∈Θn(t,T,xn) (T -t) [t,T ]×R 2N ×U |h(s, y, v)| p γ 1 (ds, dy, dz, dv) 1 p
.

Due to the compactness of Θ n (t, T, x n ), there exists an optimal γ n ∈ Θ n (t, T, x n ) such that

V p n (t, x n ) = (T -t) [t,T ]×R 2N ×U |h(s, y, v)| p γ n 1 (ds, dy, dz , dv) 1 p
.

Then, along to some subsequence, γ n converges to a γ * ∈ Θ ∞ (t, x). Using Proposition 20 (iii) and ( 44) we obtain that

V p (t, x) = lim inf n→∞ V p n (t, x n ) = (T -t) [t,T ]×R N ×U |h(s, y, v)| p γ * 1 (ds, dy, sz, dv) 1 p
.

This proves that the right-hand member cannot exceed the left-hand member. To prove the converse, we fix γ ∈ Θ ∞ (t, x). By definition, we find

n k ↑ ∞, m k ∈ N, α k i , x k i ∈ [0, 1]×K n , γ k i ∈ Θ n k t, T, x k i s.t. ∀k ≥ 1, m k i=1 α k i = 1, m k i=1 α k i δ x k i k→∞ δ x , m k i=1 α k i γ k i k→∞ γ. For each k ≥ 1 and 1 ≤ i ≤ m k , (T -t) [t,T ]×R 2N ×U |h(s, y, u| p γ k i 1 (ds, dy, dz, du) ≥ V p n k p t, x k i .
Multiplying by α k i and summing up implies that

(T -t) [t,T ]×R 2N ×U |h(s, y, u| p m k i=1 α k i γ k i 1 (ds, dy, dz, du) ≥ R N V p n k p (t, y) d m k i=1 α k i δ x k i (dy).
Using the uniform convergence of V p k to V p when k → ∞ one obtains that (T -t)

[t,T ]×R 2N ×U |h(s, y, u| p γ 1 (ds, dy, dz, du) ≥ (V p (t, x)) p .

This completes our proof. The result for L ∞ follows from the monotone convergence of L p norms.

We now turn to the proof of the properties of Θ n issued from measures.

Proof of Proposition 24.

1. Let n < ∞. We can conclude by using [14, Proposition 2.7].

2. Let n = ∞. We only need to check that Θ ∞ s, T , γ 2 is non-empty. The convexity and the closeness follow from the same arguments as Lemma 22 (i) for the case γ

2 = δ x . If γ ∈ Θ ∞ (t, s, x), then there exist n k ↑ ∞, m k ∈ N, α k i , x k i ∈ [0, 1] × K n k , u k i ∈ U(t, s) s.t. ∀k ≥ 1, m k i=1 α k i = 1, m k i=1 α k i δ x k i k→∞ δ x , m k i=1 α k i γ t,s,x k i ,u k i ;n k k→∞ γ.
For every k ≥ 1 and every 1 ≤ i ≤ m k , we set xk i := x n k s; t, x k i , u k i (•) and note that

m k i=1 α k i γ t,s,x k i ,u k i ;n k 2 = m k i=1 α k i δ xk i .
We consider u ∈ U s, T . Then, any limit point of m k i=1 α k i γ s, T ,x k i ,u;n k is an element of Θ ∞ s, T , γ 2 . Moreover, observe that such limit points exist due to the compactness of the underlying space of probability. This completes our proof.

We end this section with the proof of the abstract semigroup principle.

Proof of Theorem 26. Let n < ∞. We can conclude by using [14, Proposition 2.9 (i)].

We begin with showing that Θ ∞ t, T , x is contained Θ ∞ s, T , • ⊕ Θ ∞ (t, s, x). Indeed, if η ∈ Θ ∞ t, T , x , there exist n k ↑ ∞, m k ∈ N, α k i , x k i ∈ [0, 1] × K n k , u k i ∈ U t, T s.t.

∀k ≥ 1,

m k i=1 α k i = 1, m k i=1 α k i δ x k i k→∞ δ x , m k i=1 α k i γ t,T,x k i ,u k i ;n k k→∞ η.
The reader is invited to note (see Remark 25) that γ t,T,x k i ,u k i ;n k = γ s, T ,x k i ,u k i ;n k ⊕ γ t,s,x k i ,u k i ;n k , where xk i := x n k s; t, x k i , u k i (•) , ∀k ≥ 1.

By taking the convex combination, we obtain that

m k i=1 α k i γ t,T,x k i ,u k i ;n k = m k i=1 α k i γ s, T ,x k i ,u k i ;n k ⊕ m k i=1 α k i γ t,s,x k i ,u k i ;n k =: γ k ⊕ γ k .
The first marginals of the two measures γ k and (γ ) k have disjoint time sets for all k ≥ 1. Indeed, we observe that they have the corresponding uniform time-marginal which makes the common point s irrelevant. Moreover, their support is included in a compact set and therefore along some subsequence γ k k→∞ γ, (γ ) k k→∞ γ . Due to our initial construction, γ ∈ Θ ∞ (t, s, x) , η = γ ⊕ γ.

Moreover, the second marginal satisfies

m k i=1 α k i δ xk i = m k i=1 α k i γ t,s,x k i ,u k i ;n k 2 k→∞ γ 2 ,
which shows that γ ∈ Θ ∞ s, T , γ 2 .

We now proceed with showing the converse inclusion.

1. To this purpose, we let γ ∈ Θ ∞ (t, s, x) and γ ∈ Θ ∞ s, T , γ 2 . We are going to use the definition of both sets Θ ∞ . With the same procedure as in Lemma 22 (i), the subsequence n k , the length of the convex combinations m k and the actual points x i k can be taken common to γ and γ . This is also valid for the control processes u i k because they are given on [t, s] and s, T respectively 10 . With this in mind, there exist

n k ↑ ∞, m k ∈ N, α k i , β k i , x k i ∈ [0, 1] 2 × K n k , u k i ∈ U t, T s.t. m k i=1 α k i = m k i=1 β k i = 1, ∀k ≥ 1, m k i=1 α k i δ x k i k→∞ δ x , m k i=1 α k i γ t,s,x k i ,u k i ;n k k→∞ γ, m k i=1 β k i δ x k i k→∞ γ 2 , m k i=1 β k i γ s, T ,x k i ,u k i ;n k k→∞ γ . (45)
2. For every 1 ≤ i ≤ m k = m k , we denote by xk i := x n k s; t, x k i , u k i (•) , ∀k ≥ 1. We have that

W 1 m k i=1 α k i δ xk i , m k i=1 β k i δ x k i = inf          i≤i,j≤m k λ k i,j xk i -x k j : λ k i,j ∈ [0, 1] , 1≤i≤m k λ k i,j = β k j , 1≤j≤m k λ k i,j = α k i          .
We choose λ k i,j such that 1≤i≤m k λ k i,j = β k j , 1≤j≤m k λ k i,j = α k i and

(46) As we have already seen, this distance is related to the estimates of

x n k r; s, xk i , u k j -x n k r; s, x k j , u k i ≤ C xk i -x k j , ∀k ≥ 1.
Recall that, the constant C is generic and independent of r ∈ s, T . From (46) it follows that

W 1   1≤i,j≤m k λ k i,j γ s, T ,x k i ,u k j ;n k •, •, R N , •, • , 1≤j≤m k β k j γ s, T ,x k j ,u k j ;n k •, •, R N , •, •   = W 1   1≤i,j≤m k λ k i,j γ s, T ,x k i ,u k j ;n k •, •, R N , •, • , 1≤i,j≤m k λ k i,j γ s, T ,x k j ,u k j ;n k •, •, R N , •, •   ≤ C 1≤i,j≤m k λ k i,j xk i -x k j ≤ C k -1 + W 1 m k i=1 α k i δ xk i , m k i=1 β k i δ x k i . ( 47
)
•, •, R N , •, • = γ ⊕ γ •, •, R N , •, • .
Our proof is now complete.

( 5 ,

 5 iv) The function γ is consistent and at least lower semi continuous on [0, T ) × K as the supremum of a family of continuous functions i.e. (t, x) → γ(t, x) = sup p≥1 V p (t, x)(T -t) -1/p . Similar to [Page 1071], we only have to prove the continuity at T i.e. (33) lim t→T V p (t, x) (T -t) -1 p = min u∈U |h(T, x, u)| , ∀x ∈ K.

t→TV∞

  (t, x) ≤ min u∈U |h(T, x, u)|. For the reverse inequality, we have that, for every u(•) ∈ U[t, T ] and every s ∈ [t, T ], |h(s, x(s; t, x, u(•)), u(s))| ≥ min u∈U |h(s, x(s, x, u(•)), u)| ≥ min u∈U |h(T, x, u)| -µ h (C(T -t)) . Consequently, lim t→T V ∞ (t, x) ≥ min u∈U |h(T, x, u)|. This completes the proof.

  γ s, T ,x k i ,u k j ;n k ds, dy, R N , du, dy , 1≤i,j≤m k λ k i,j γ s, T ,x k j ,u k j ;n k ds, dy, R N , du, dy   .

  : |t -s| + ||x -y|| + |ρ -r| + ||q -m|| ≤ }.For further properties of H * and H * see Section 4. A viscosity solution of (4) is a function which is both subsolution and supersolution.

	and
	H * (t, x, r, q) := lim inf →0 {H(s, y, ρ, m) Note that a viscosity solution is a continuous function because it is simultaneously u.s.c and l.s.c.

r, q) := lim sup →0 {H(s, y, ρ, m) : |t -s| + ||x -y|| + |ρ -r| + ||q -m|| ≤ },

  (where we have extended u n from ūn if t ≤ τ ≤ t n by setting u n (τ ) = ūn (t n )) such that

				T
	(31)	lim inf = lim inf n→∞	n→∞ t |h(s, (x(s; t, x, u n T	tn	|h(s, xn (s; t n , x n , ūn (•)), ūn (s))| p ds

t} , T ] , n→∞ W (t n , x n ) + ε ≥ lim inf

  We set u k i,j :=u k i 1 [t,s) + u k j 1 [s, T ] , ∀k ≥ 1.To summarize, we have exhibitedλ k i,j ∈ [0, 1], ∀k ≥ 1 such that 1≤i,j≤m k λ k i,j = 1 and 1≤i,j≤m k λ k i,j δ x k It follows that γ ∈ Θ ∞ t,T , x by definition. Using (45), (47), (48) and recalling that W 1

	Finally, we note that
	(48)
											
		α k i γ t,s,x k i ,u k i ;n k		=		λ k i,j γ t,s,x k i ,u k i ;n k		=	λ k i,j δ xk i	=	α k i δ xk
	1≤i≤m k	2				1≤i,j≤m k				2	1≤i,j≤m k	1≤i≤m k
									i	= m k i=1 α k i δ x k i k→∞	δ x . Moreover,
												
	1≤i,j≤m k	λ k i,j γ t, T ,x k i ,u k i,j ;n k =		1≤i,j≤m k	λ k i,j γ s, T ,x k i ,u k j ;n k	 ⊕		1≤i,j≤m k	λ k i,j γ t,s,x k i ,u k i ;n k		k→∞	γ
	on some subsequence. m k i=1 α k i γ t,s,x k i ,u k i ;n k	2	, m k j=1 β k j δ x k j	→ k→∞	0 we conclude that
				γ

10

Indeed, if u i k and v i k are given, we construct

w i,i k = u i k 1 [t,s] + v i k 1 (s, T

] and rename the index.

i .

Here dK (•) is the distance function to the set K.

Recall that a set valued map G : K → R N is monotone if y1 -y2, x1 -x2 ≥ 0, ∀yi ∈ G(xi), i ∈ {1, 2}

The set K is sleek if the map x → TK (x) is lower semicontinuous (l.s.c.).

A set valued map F from R N onto R N is called Marchaud map if F is upper semicontinuous (u.s.c.) with non-empty compact convex values and has a linear growth.

Here the notations r-and r+ stand for left-hand or right-hand limits.

Note that C does not depend on t ∈ [0, T ], (xn, x, u(•)) ∈ K + C n × K × U (t, T ) and n ≥ 1.

For a set S, 1S denotes is characteristic function defined by 1S(s) = 1 if s ∈ S and 1S(s) = 1 if s ∈ S.

The last index specifies that the occupation measure is computed from the solution associated with the Moreau-Yosida penalization ∇g n j k .

* The work of the first author has been supported by the National Key R and D Program of P.R. China No. 2018YFA0703900 and the NSF of P.R. China No. 12031009.