Olivier Bokanowski

Athena Picarelli
email: picarelli@maths.ox.ac.uk

AND Hasnaa Zidani
email: hasnaa.zidani@ensta.fr

Sadco Project

G A Number

STATE-CONSTRAINED STOCHASTIC OPTIMAL CONTROL PROBLEMS VIA REACHABILITY APPROACH *

Keywords: Hamilton-Jacobi equations, state-constraints, stochastic optimal control, viscosity notion, stochastic target problems AMS subject classifications: 49L20, 49L25, 93E20, 35K55

This paper deals with a class of stochastic optimal control problems (SOCP) in presence of state-constraints. It is well-known that for such problems the value function is, in general, discontinuous and its characterization by a Hamilton-Jacobi equation requires additional assumptions involving an interplay between the boundary of the set of constraints and the dynamics of the controlled system. Here, we give a characterization of the epigraph of the value function without assuming the usual controllability assumptions. For this end, the SOCP is first translated into a state-constrained stochastic target problem. Then a level-set approach is used to describe the backward reachable sets of the new target problem. It turns out that these backward-reachable sets describe the value function. The main advantage of our approach is that it allows to handle easily the state constraints by an exact penalization. However, the target problem involves a new state variable and a new control variable that is unbounded.

1. Introduction. Consider a filtered probability space (Ω, F, F t , P). Let B(•) be a R p -Brownian motion, and U be a set of control processes. For any u ∈ U the following system of controlled stochastic differential equations (SDE) in R d is considered dX(s) = b(s, X(s), u(s))ds + σ(s, X(s), u(s))dB(s) s ∈ [t, T ] X(t) = x.

(1.1)

Under standard assumptions on b and σ, the SDE (1.1) admits a strong solution X u t,x (•) associated to the control u. Let K be a given non empty closed set of R d , and consider the value function ϑ associated to the following optimal control problem ϑ(t, x) := inf u∈U E ψ(X u t,x (T )) + T t (s, X u t,x (s), u(s))ds , such that X u t,x (s) ∈ K, ∀s ∈ [t, T ] a.s. (1.2) where the functions ψ and represent respectively the final and distributed costs (the precise assumptions are given in Section 2). In the unconstrained case (i.e. K = R d ), the function ϑ can be characterized as the unique viscosity solution of a second order Hamilton-Jacobi-Bellman (HJB) equation, see [START_REF] Touzi | Optimal Stochastic Control, Stochastic Target Problems, and Backward SDE[END_REF][START_REF] Yong | of Applications of Mathematics[END_REF] and the references therein. However in many practical applications, the state variable is constrained to remain in a given closed set K R d , where K takes into account the presence of an obstacle or economical/physical constraints ... etc. In this context, the characterization of the value function ϑ as a viscosity solution of a HJB equation becomes more complicated and involves a delicate interplay between the dynamics of state variable and the boundary of K. Furthermore, the value function ϑ can take infinite values in the regions where there is no viable trajectory that stays in K on the entire time interval [t, T ]. A rich literature has been developed for dealing with state constrained optimal control problems and the associated HJB equations. In the deterministic setting, we refer to [START_REF] Soner | Optimal control with state-space constraint[END_REF][START_REF] Soner | Optimal control with state-space constraint[END_REF][START_REF] Frankowska | Existence of Neighboring Feasible Trajectories: Applications to Dynamic Programming for state-constrained optimal control problems[END_REF][START_REF] Ishii | A new formulation of state constraint problems for first-order PDE's[END_REF][START_REF] Dolcetta | Hamilton-jacobi equations with state constraints[END_REF][START_REF] Hermosilla | Infinite horizon problems on stratifiable state-constraints sets[END_REF] where the HJB equation for the value function is discussed and several conditions are investigated in order to guarantee the characterization of ϑ. In the stochastic case, the problem has also attracted a great attention, see for example [START_REF] Katsoulakis | Viscosity solutions of second fully nonlinear elliptic equations with state constraints[END_REF][START_REF] Lasry | Nonlinear Elliptic Equations with Singular Boundary Conditions and Stochastic Control with State Constraints. I. The Model Problem[END_REF][START_REF] Ishii | A class of stochastic optimal control problems with state constraint[END_REF][START_REF] Bouchard | Optimal control under stochastic target constraints[END_REF]. It is by now clear, that the continuity of the value function and its characterization require additional properties on the drift b and the diffusion σ at the boundary of the set K.

The aim of the present work is to provide an alternative way for characterizing, in a very general setting, the value function associated with a state constrained optimal control problem, without assuming the classical controllability assumptions. This goal is achieved at the price of increasing the state variable by one additional component and introducing a new control variable that lies in an unbounded set. The main feature of the approach presented in this paper is to bypass the regularity issues often presented by the value function of a constrained control problem. By using the HJB characterization in a continuous setting, it opens a way to compute the value function by various existing numerical methods.

Our approach relies essentially on two ideas. First, we reformulate (1.2) as a state constrained stochastic target problem. Then, this auxiliary target problem is described by a level set approach where the state constraints are handled by using an exact penalization technique. More precisely, a straightforward adaption of some arguments in [START_REF] Bouchard | Optimal Control versus Stochastic Target problems: an equivalence result[END_REF] lead to the following statement: ϑ(t, x) = inf z ∈ R : ∃(u, α) ∈ U × A s.t.

(1.3) Z α,u t,x,z (T ) ≥ ψ(X u t,x (T )), X u t,x (s) ∈ K, ∀s ∈ [t, T ] a.s.

where A is the set of square-integrable predictable processes with values in R p and Z α,u t,x,z is a new state process whose existence is guaranteed by the Martingale representation theorem and that satisfies:

Z α,u t,x,z (θ) = z - θ t (s, X u t,x (s), u(s)) ds + θ t α T s B(s), θ ≥ t.
From the above formulation of ϑ, it appears that the epigraph of ϑ(t, •) is related to a state constrained reachability set (here, the target set is the epigraph of the final cost ψ). This result is very well known in the deterministic setting, see [START_REF] Cardaliaguet | Optimal times for constrained nonlinear control problems without local contrllability[END_REF][START_REF] Cardaliaguet | Numerical schemes for discontinuous value function of optimal control[END_REF][START_REF] Aubin | The viability kernel algorithm for computing value functions of infinite horizon optimal control problems[END_REF]. Its generalization to the stochastic setting is not very surprising, however it should be noticed that in contrary to the deterministic case, the motion of the new state component Z α,u t,x,z involves a new control variable that is unbounded (see [START_REF] Altarovici | A general Hamilton-Jacobi framework for nonlinear state-constrained control problems[END_REF] for the deterministic case).

Stochastic target problems have been extensively studied, in the case when K = R d , in [START_REF] Soner | Stochastic target problems, dynamic programming and viscosity solutions[END_REF] and [START_REF] Soner | Dynamic programming for stochastic target problems and geometric flows[END_REF]. In the present work, we propose to use a level-set approach to describe the backward reachable sets. This approach has the great advantage to characterize the backward-reachable sets by mean of an auxiliary control problem where the state constraints are taken into account in a simple way. Furthermore, the value function of the auxiliary control problem can be shown to be continuous (even when ϑ is discontinuous), which is a good property in view of numerical approximations. Let us recall that the level-set approach has been introduced by Osher and Sethian in [START_REF] Osher | Fronts propagating with curvature dependent speed: algorithms on Hamilton-Jacobi formulations[END_REF] to model some front propagation phenomena. Later, the idea has been used in [START_REF] Falcone | Level sets of viscosity solutions: Some applications to fronts and rendez-vous problems[END_REF] to describe the reachable sets for a deterministic problem. The level-set appraoch has been used in many application related to nonlinear controlled systems. Quite recently, the level set approach has been extended to the case of state-constrained controlled systems, see [START_REF] Margellos | Hamilton-Jacobi formulation for Reach-avoid differential games[END_REF][START_REF] Bokanowski | Reachability and minimal times for state constrained nonlinear problems without any controllability assumption[END_REF][START_REF] Bokanowski | Dynamic Programming and Error Estimates for Stochastic Control Problems with Maximum Cost[END_REF][START_REF] Assellaou | Error estimates for second order hamilton-jacobibellman equations. approximation of probabilistic reachable sets[END_REF]. In all the above mentioned literature on level-set approach, the control variables lie in a bounded set. The extension to the unbounded control setting is not a trivial task and is also a contribution of the present paper.

The paper is organized as follows. The setting of the problem and the assumptions are introduced in Section 2. In Section 3 we establish the link between the state constrained SOCP (1.2) and a suitable stochastic target problem. Section 3.2 focuses on the reachability problem and the level set method. This will lead to a characterization of the reachable set by a generalized HJB equation in Section 4. A comparison principle for the associated boundary value problem is proved in Section 5. Finally, Appendix A presents an existence result of optimal controls (when the set of controls is not compact).

Notation. In all the sequel, (Ω, F, P) denotes a probability space with a filtration {F t } t≥0 , P-augmentation of the filtration generated by a p-dimensional Brownian motion

B(•) (p ≥ 1). The notation L 2 (Ω, F T ; R) denotes the set of the R-valued F T -measurable random variables η such that E[|η| 2 ] < ∞.
For any n ≥ 1, S n denotes the space of n × n symmetric matrices. As usual, the abbreviations "s.t.", "a.e", "a.s.", "w.r.t." stand respectively for "such that", "almost everywhere", "almost surely", "with respect to".

2.

Setting and main assumptions. Given T > 0 and 0 ≤ t ≤ T , consider the

following system of controlled SDE in R d dX(s) = b(s, X(s), u(s))ds + σ(s, X(s), u(s))dB(s) s ∈ [t, T ] X(t) = x, (2.1)
where the control input u belongs to a set U of progressively measurable processes with values in a compact set U ⊂ R m (m ≥ 1). The following classical assumption will be considered on the coefficients b and σ:

(H1) b : [0, T ] × R d × U → R d and σ : [0, T ] × R d × U → R d×p are continuous functions. Moreover, there exist L b , L σ ≥ 0 such that for every x, y ∈ R d , t ∈ [0, T ], u ∈ U : |b(t, x, u) -b(t, y, u)| ≤ L b |x -y|, |σ(t, x, u) -σ(t, y, u)| ≤ L σ |x -y|.
We recall some classical results on (2.1) whose proof can be found in [START_REF] Touzi | Optimal Stochastic Control, Stochastic Target Problems, and Backward SDE[END_REF]Theorem 2.4].

Proposition 2.1. Under assumption (H1) there exists a unique F t -adapted process, denoted by X u t,x (•), strong solution of (2.1). Moreover there exists a constant

C 0 > 0, such that for any u ∈ U, 0 ≤ t ≤ t ≤ T and x, x ∈ R d E sup θ∈[t,t ] X u t,x (θ) -X u t,x (θ) 2 ≤ C 2 0 |x -x | 2 , (2.2a) E sup θ∈[t,t ] X u t,x (θ) -X u t ,x (θ) 2 ≤ C 2 0 (1 + |x| 2 ) |t -t |, (2.2b)
E sup θ∈[t,t ] X u t,x (θ) -x 2 ≤ C 2 0 (1 + |x| 2 ) |t -t |. (2.2c)
Consider two functions ψ and , namely the terminal and running cost, such that:

(H2) (i) ψ : R d → R, : [0, T ] × R d × U → R are continuous functions, (ii
) ψ, are non-negative functions:

ψ, ≥ 0, (iii) there exist L ψ , L ≥ 0 such that for every x, y ∈ R d , u ∈ U, t ∈ [0, T ]: |ψ(x) -ψ(y)| ≤ L ψ |x -y|, | (t, x, u) -(t, y, u)| ≤ L |x -y|. Remark 2.2.
The assumption (H2)(ii) can be replaced by: ψ, ≥ -M for some M ≥ 0.

Let K ⊆ R d be a given nonempty and closed set of state constraints. In this paper we deal with optimal control problems where the cost is given by:

J(t, x, u) := E ψ X u t,x (T ) + T t (s, X u t,x (s), u(s))ds , (2.3)
while the state X u t,x is required to satisfy the condition:

X u t,x (s) ∈ K, ∀s ∈ [t, T ] a.s.
More precisely, the optimal control problem and its value function are defined as:

ϑ(t, x) := inf u∈U J(t, x, u) : X u t,x (s) ∈ K, ∀s ∈ [t, T ] a.s. ( 2.4)
(observe that by assumption (H2)(ii), J(t, x, u) ≥ 0 for all (t, x, u)

∈ [0, T ] × R d × U).
It is well known that in general ϑ is merely discontinuous and satifsfies the following constrained HJB equation:

   F (t, x, ∂ t v, Dv, D 2 v) = 0 t ∈ [0, T ), x ∈ int(K) F (t, x, ∂ t v, Dv, D 2 v) ≥ 0 t ∈ [0, T ), x ∈ ∂K ϑ(T, x) = ψ(x)
x ∈ K.

with

F (t, x, r, q, Q) := -r + sup u∈U -q, b(t, x, u) - 1 2 T r[σσ T (t, x, u)Q] -(t, x, u) .
Moreover, a uniqueness result for the above HJB equation can be guaranteed under some additional compatibility conditions. The first condition that has been introduced in the literature, in the context of deterministic control systems, is the so-called inward pointing condition, originally stated by Soner in [START_REF] Soner | Optimal control with state-space constraint[END_REF] and [START_REF] Soner | Optimal control with state-space constraint[END_REF]. In the stochastic setting (i.e., when σ ≡ 0), state constrained problems were for instance introduced in [START_REF] Lasry | Nonlinear Elliptic Equations with Singular Boundary Conditions and Stochastic Control with State Constraints. I. The Model Problem[END_REF], where the diffusion is the identity matrix. In that case, the presence of state constraints requires to consider unbounded controls so that the trajectories with the action of the drift can still be maintained inside the desired domain. This results in an HJB equation with singular boundary conditions. On the contrary, in our framework, the set of possible controls is given. In this context, the value function may take infinite values in the regions where there is no feasible trajectories remaining in the set K. Actually, the question whether the set K is viable or not is by itself an important issue. This question has been addressed in many papers. In [START_REF] Bardi | Invariant sets for controlled degenerate diffusions: A viscosity solutions approach[END_REF], a condition for the viability of K is the following

∀x ∈ ∂K, ∃u ∈ U such that: 1 2 T r[σσ T (x, u)D 2 d K (x)] + b(x, u), Dd K (x) ≤ 0,
and (Dd K (x)) T σσ T (x, u)Dd K (x) = 0, where d K denotes the signed distance function to ∂K. However, the viability of K is not sufficient for the characterization of the value function as the unique viscosity solution of the constrained HJB equation, and stronger degeneracy assumptions may be required, see [START_REF] Katsoulakis | Viscosity solutions of second fully nonlinear elliptic equations with state constraints[END_REF][START_REF] Barles | A strong comparison result for the bellman equation arising in stochastic exit time control problems and its applications[END_REF][START_REF] Ishii | A class of stochastic optimal control problems with state constraint[END_REF] and [START_REF] Bouchard | Weak dynamic programming for generalized state constraints[END_REF]. Our purpose in this paper is to provide an alternative way for dealing with state constrained optimal control problems (2.4). Ongoing works concern the use of this new formulation for the computation of the value function ϑ, even though ϑ can be discontinuous, and no degeneracy condition is assumed.

3. Formulation as a target control problem and level set approach.

3.1. Target problem. The first result of this section concerns the link between (2.4) and a stochastic target control problem under state constraints (see the definition given in [START_REF] Soner | Stochastic target problems, dynamic programming and viscosity solutions[END_REF]).

Let A = L 2 F (0, T ; R p ) denotes the set of square-integrable R p -valued predictable processes, and for a given control pair (u, α) ∈ U × A, let Z u,α t,x,z (•) be the onedimensional process defined by

Z u,α t,x,z (•) = z - • t (s, X u t,x (s), u(s))ds + • t α T s dB(s). (3.1)
Proposition 3.1. Let assumptions (H1) and (H2) be satisfied. Then

ϑ(t, x) = inf z ≥ 0 : ∃(u, α) ∈ U × A such that Z u,α t,x,z (T ) ≥ ψ(X u t,x (T )) and X u t,x (s) ∈ K, ∀s ∈ [t, T ] a.s.
Proof. The argument, that will be reported here for completeness, is an adaptation of the one presented in [START_REF] Bouchard | Stochastic target problems with controlled loss[END_REF] and [START_REF] Bouchard | Optimal Control versus Stochastic Target problems: an equivalence result[END_REF]. It is straightforward to check that:

ϑ(t, x) = inf z ≥ 0 : ∃u ∈ U s.t. E ψ(X u t,x (T ))+ T t (s, X u t,x (s), u(s)) ds ≤ z and X u t,x (s) ∈ K, ∀s ∈ [t, T ] a.s. .
Therefore, it suffices to prove that, for any z ≥ 0, the two following statements are equivalent:

(i) ∃u ∈ U s.t. E ψ(X u t,x (T ))+ T t (s, X u t,x (s), u(s)) ds ≤ z and X u t,x (s) ∈ K, ∀s ∈ [t, T ] a.s. (ii) ∃(u, α) ∈ U × A s.t. Z u,α t,x,z (T ) ≥ ψ(X u t,x (T )) and X u t,x (s) ∈ K, ∀s ∈ [t, T ] a.s.
where Z u,α t,x,z is the process defined by (3.1). The implication (ii) =⇒ (i) follows by taking the expectation in (ii) and recalling that if α ∈ A then • t α T (s)dB(s) is a martingale. Furthermore, under (H1)-(H2), we have:

ψ(X u t,x (T )) + T t (s, X u t,x (s), u(s))ds ∈ L 2 (Ω, F T ; R)
for any u ∈ U. By consequence, the Martingale representation theorem applies (see [39, Theorem 5.7, Chapter I] for instance). Then there exists a process α ∈ A such that

z ≥ J(t, x, u) = ψ(X u t,x (T )) + T t (s, X u t,x (s), u(s))ds - T t αT s dB(s).
Hence, by defining

Z u, α t,x,z (θ) := z - θ t (s, X u t,x (s), u(s))ds + θ t αT s dB(s),
the implication (i) =⇒ (ii) is proved and the statement of the proposition follows.

Next, we give some useful properties of the process Z u,a t,x,z defined in (3.1). Lemma 3.2. Assume that (H1)-(H2) are satisfied. Let (u, α) be in A × U. (i) There exists C 1 > 0 (depending only on C 0 , T, L ψ , L ) s.t. for any t ∈ [0, T ], for any x, x ∈ R d and every z, z ∈ R,

E Z u,α t,x,z (T ) -Z u,α t,x ,z (T ) ≤ C 1 (|x -x | + |z -z |) (3.2) (ii) For any t, x, z ∈ [0, T ] × R d × R, we have lim t →t, t ∈[0,T ] E Z u,α t ,x,z (T ) -Z u,α t,x,z (T ) = 0, (3.3)
the limit being furthermore locally uniform w.r.t. the variables (x, z).

Proof. (i) is a consequence of the Lipschitz continuity of and the linear growth estimate of X u t,x given in Proposition 2.1. On other hand, for every h ≥ 0, classical estimates lead to

E Z u,α t+h,x,z (T ) -Z u,α t,x,z (T ) ≤ C √ h + E t+h t α T s dB(s) .
Then, by using the fact that α ∈ A, we get:

E t+h t α T s dB(s) 2 ≤ E t+h t α T s dB(s) 2 = t+h t E|α T s | 2 ds h→0 → 0.
Similar estimates can be obtained for h ≤ 0, and hence (3.3) is proved.

Remark 3.3. It could be noticed that the new set of controls A that appears in Proposition 3.1 can be restricted to the set of squared integrable R p -valued process satisfying a specific bound in the

L 2 F -norm. Indeed, for x ∈ R d , consider the control α ∈ L 2 F such that T t α T (s)dB s = ψ(X u t,x (T )) + T t (s, X u t,x (s), u(s))ds -E ψ(X u t,x (T )) + T t (s, X u t,x (s), u(s))ds .
By using the estimates in Proposition 2.1 along with assumptions (H2), we get the existence of a constant C 2 ≥ 0 (depending only on C 0 , T, L ψ , L ) s.t. for any t ∈ [0, T ] and x ∈ R d we have:

α L 2 F ≤ C 2 (1 + |x|). (3.4)
Now, let the backward reachable set be defined as:

R K t := (x, z) ∈ R d × R : ∃(u, α) ∈ U × A s.t. (3.5) (X u t,x (T ), Z u,α t,x,y (T )) ∈ Epi(ψ) and X u t,x (s) ∈ K ∀s ∈ [t, T ] a.s. ,
where Epi(ψ) := {(x, z) ∈ R d+1 : z ≥ ψ(x)} denotes the epigraph of the function ψ. By Proposition 3.1, we get the following result. Lemma 3.4. Assume (H1)-(H2). Then, for every t ∈ [0, T ], x ∈ R d :

ϑ(t, x) = inf z ≥ 0 : (x, z) ∈ R K t . (3.6)
The above Lemma establishes a clear connection between the value function and the backward reachable sets of the augmented differential system. From now on, we will focus on the characterization of the sets R K t . 3.2. Auxiliary control problem and level set approach. By the "levelset approach" we aim at describing the backward reachable set as the zero-level set of a suitable function. The idea was first introduced by Osher and Sethian [START_REF] Osher | Fronts propagating with curvature dependent speed: algorithms on Hamilton-Jacobi formulations[END_REF] in the context of front propagation, and then extended to reachability problems, see [START_REF] Kurzhanski | Ellipsoidal techniques for reachability under state constraints[END_REF][START_REF] Falcone | Level sets of viscosity solutions: Some applications to fronts and rendez-vous problems[END_REF][START_REF] Bokanowski | Reachability and minimal times for state constrained nonlinear problems without any controllability assumption[END_REF][START_REF] Assellaou | Error estimates for second order hamilton-jacobibellman equations. approximation of probabilistic reachable sets[END_REF][START_REF] Bokanowski | Dynamic Programming and Error Estimates for Stochastic Control Problems with Maximum Cost[END_REF] and the references therein.

Let us consider a function g K : R d → R + such that:

(H3) g K is a Lipschitz continuous function (with Lipschitz constant L K ) such that g K (x) = 0 ⇔ x ∈ K.
(3.7)

Remark 3.5. Notice that if K is a nonempty and closed set, a function g K satisfying (H3) always exists. It is for instance sufficient to consider g K (x) := d + K (x) where d + K denotes the Euclidean distance function to K. We consider the following unconstrained optimal control problem w(t, x, z) := inf

u∈U α∈A E max(ψ(X u t,x (T )) -Z u,α t,x,z (T ), 0) + T t g K (X u t,x (s))ds . (3.8)
The following assumption will be also considered:

(H4) For any (t, x, z) ∈ [0, T ] × R d × R, the problem (3.8
) admits an optimal pair of controls (ū, ᾱ) ∈ U × A.

Remark 3.6. When the set of control values U is compact, and if the set (b, σσ T , )(t, x, U ) is convex for every (t, x) ∈ [0, T ] × R d , then a problem of the form (2.4) admits an optimal control (see [39, Theorem 5.3, Chapter II] for instance). In the context of unbounded control problems, the existence of optimal control laws holds whenever the cost function satisfies some coercivity conditions w.r.t. the control variable, see [START_REF] Loewen | Existence Theory for a Stochastic Bolza Problem[END_REF] and [START_REF] Haussmann | On the existence of optimal controls[END_REF].

In this paper, the question of existence of optimal controls will not be fully investigated. However, we give in Appendix A an existence result in the case when b, and σ are linear functions w.r.t. the space and the control variables and under some convexity assumptions on g K and ψ.

Theorem 3.1. Assume that (H1)-(H4) are satisfied. Then (i) For every t ∈ [0, T ], we have R K t = (x, z) ∈ R d+1 : w(t, x, z) = 0

(ii) For every t ∈ [0, T ] and every x ∈ R d , ϑ(t, x) = inf z ≥ 0 : w(t, x, z) = 0 .

Proof. Statement (ii) clearly follows by (i) and by (3.6). Therefore, we need just to check the claim (i). If (x, z) ∈ R K t , by definition of the backward reachable set, there exists a pair (ū, ᾱ) ∈ U × A such that max(ψ(X ū t,x (T )) -Z ū, ᾱ t,x,z (T ), 0) = 0 and g K (X ū t,x (s)) = 0, ∀s ∈ [t, T ] a.s., which means that w(t, x, z) = 0. Therefore the "⊆" inclusion is proved.

Let us now assume that w(t, x, z) = 0. Thanks to assumption (H4), there exists

(ū, ᾱ) ∈ U × A such that E max(ψ(X ū t,x (T )) -Z ū, ᾱ t,x,z (T ), 0) + T t g K (X ū t,x (s))ds = 0.
Since g K is a positive function, it follows that:

max(ψ(X ū t,x (T )) -Z ū, ᾱ t,x,z (T ), 0) + T t g K (X ū t,x (s))ds = 0 a.s.
Hence, (X ū t,x (T ), Z ᾱ,ū t,x,z (T )) ∈ Epi(ψ) and X ū t,x (s) ∈ K, ∀s ∈ [t, T ] a.s., which concludes that (x, z) belongs to R K t , and the proof is completed.

In Proposition 3.7, it will be shown that, under the hypotheses (H1)-(H3), the function w is continuous. If one further assumes (H4), so the equality (i) of Theorem 3.1 implies that set R K t is closed. We believe that assumption (H4) may be replaced by an assumption on the closeness of R K t . This claim will be investigated in ongoing work while in the next sections we will focus on the characterization of the function w.

0 ≤ w(t, x, z) ≤ C(1 + |x|),
for some constant C > 0.

Proof. (i) First, we prove the Lipschitz continuity w.r.t. the space variables x and z. By straightforward calculations, we get:

|w(t, x, z) -w(t, x , z )| ≤ sup (u,α)∈U ×A E ψ(X u t,x (T )) -ψ(X u t,x (T )) + Z u,α t,x,z (T ) -Z u,α t,x ,z (T ) +E T t g K (X u t,x (s)) -g K (X u t,x (s)) ds ≤ C sup (u,α)∈U ×A E X u t,x (T ) -X u t,x (T ) + Z u,α t,x,z (T ) -Z u,α t,x ,z (T ) + +E T t X u t,x (s) -X u t,x (s) ds
where the constant C ≥ 0 depends on L ψ and L K . By using the estimates of Proposition 2.1 and those of Lemma 3.2, we finaly obtain

|w(t, x, z) -w(t, x , z )| ≤ C(|x -x | + |z -z |),
for some constant C ≥ 0 that depends on C 0 , C 1 , T, L ψ , L K .

Let us now prove the continuity in time. Let 0

≤ t ≤ t + h ≤ T . By the standard argument | inf u A u -inf u B u | ≤ sup u |A u -B u |, it follows that: |w(t + h, x, z) -w(t, x, z)| ≤ sup (u,α)∈U ×A E ψ(X u t+h,x (T )) -ψ(X u t,x (T ))| + |Z u,α t+h,x,z (T ) -Z u,α t,x,z (T ) + T t+h g K (X u t+h,x (s))ds - T t g K (X u t,x (s))ds ≤ sup (u,α)∈U ×A CE X u t+h,x (T ) -X u t,x (T )| + |Z u,α t+h,x,z (T ) -Z u,α t,x,z (T ) + t+h t (1 + |X u t,x (s)|)ds + T t+h |X u t+h,x (s) -X u t,x (s)|ds ≤ C( √ h + sup (u,α)∈U ×A E Z u,α t+h,x,z (T ) -Z u,α t,x,z (T ) ), (3.9)
where the constant C depends on C 0 , C 1 , T, Lψ, L K . Taking into account Remark 3.3, the supremum term in (3.9) can be restricted to controls α satisfying the bound (3.4).

Then for any > 0, we can find (u , α ) ∈ U × A such that

|w(t + h, x, z) -w(t, x, z)| ≤ C √ h + CE Z u ,α t+h,x,z (T ) -Z u ,α t,x,z (T ) + .
To conclude, using the result of Lemma 3.2(ii), we get lim sup

h→0 |w(t + h, x, z) -w(t, x, z)| ≤ .
Since this > 0 is arbitrary, the desired result is obtained. (Furthermore, notice that by the same arguments, the continuity in t is locally uniform in (x, z).) (ii) By definition of w, we have

w(t, x, z) ≤ inf (u,α)∈U ×{0} E max ψ(X u t,x (T )) -Z 0,u t,x,z (T ), 0 (3.10)
+ T t g K (X u t,x (s))ds .
If z ≥ 0, the positivity of and ψ leads to

w(t, x, z) ≤ inf u∈U E ψ(X u t,x (T )) + T t (s, X u t,x (s), u(s)) + g K (X u t,x (s))ds
and the desired result is obtained by using also the linear growth of ψ, and g K and classical estimates for the process X u t,x (•) (see Proposition 2.1). Another information that will be useful later on concerns the value of w for z ≤ 0. Let us consider the following optimal control problem free of state constraints:

w 0 (t, x) := inf u∈U E ψ(X u t,x (T )) + T t
(s, X u t,x (s), u(s)) + g K (X u t,x (s))ds .

Lemma 3.8. Under assumptions (H1)-(H3), for any t ∈ [0, T ], x ∈ R d and z ∈ R,

z ≤ 0 ⇒ w(t, x, z) = w 0 (t, x) -z.
Proof. By definition of the function w, for any (t, x, z)

∈ [0, T ] × R d × R w(t, x, z) = inf (u,α)∈U ×A E max ψ(X u t,x (T )) -Z u,α t,x,z (T ), 0 + T t g K (X u t,x (s))ds ≥ inf (u,α)∈U ×A E ψ(X u t,x (T )) -z + T t α T (s)dB s + T t (s, X u t,x (s), u(s)) + g K (X u t,x (s))ds = w 0 (t, x) -z,
using that T t α T (s)dB s is a martingale. Hence the "≥" inequality is satisfied for any z ∈ R. On the other hand, thanks to the positivity of and ψ, for z ≤ 0, max ψ(X u t,x (T )) -Z 0,u t,x,z (T ), 0 = ψ(X u t,x (T )) -z + T t (s, X u t,x (s), u(s)).

Therefore, by inequality (3.10) we have w(t, x, z) ≤ w 0 (t, x) -z, whenever z ≤ 0.

We conclude this section by giving a dynamic programming principle associated to the value function w (see [START_REF] Touzi | Optimal Stochastic Control, Stochastic Target Problems, and Backward SDE[END_REF]).

Lemma 3.9. Assume (H1), (H2), (H3). For every 0 ≤ t ≤ T , (x, z) ∈ R d+1 and any [t, T ]-valued stopping time θ, we have: In this section we aim to characterize the auxiliary value function w as the unique solution, in the viscosity sense, of a suitable partial differential equation. As we will see, the main difficulties come from the unboundedness of the control α. Formally, the HJB equation associated to the extended optimal control problem (3.8) is:

w(t, x, z) = inf (u,α)∈U ×A E w(θ, X u t,x (θ), Z α,u t,x,z (θ) .
-∂ t w(t, x, z) + H(t, x, Dw(t, x, z), D 2 w(t, x, z)) = 0,
where the notations Dw(t, x, z), D 2 w(t, x, z) stand respectively for the gradient and the Hessian of w w.r.t. the variables (x, z), and where the Hamiltonian H is defined from

[0, T ] × R d × R d+1 × S d+1 into R ∪ {+∞} by: H(t, x, q, Q) := sup (u,α)∈U ×R p -b(t, x, u), q - 1 2 T r[σσ T (t, x, u, α)Q] -g K (x).
Recalling the definition of σ, we have σσ T (t, x, u, α) = σσ T (t, x, u) σ(t, x, u)α α T σ T (t, x, u) α T α .

Denoting Q = Q 11 Q 12 Q 21 Q 22 the blocks of the symmetric matrix Q (with Q 11 ∈ R d×d , Q 12 = Q T 21 ∈ R d×1 and Q 22 ∈ R),
the Hamiltonian H can be re-written as:

H(t, x, q, Q) = sup (u,α) ∈U ×R p -b, q - 1 2 T r[σσ T Q 11 ] -α T σ T Q 12 - 1 2 α 2 Q 22 -g K (x).
Hence the following algebraic reformulation, for any r ∈ R:

-r + H(t, x, q, Q) = (4.1) sup (u,α) ∈U ×R p -r -b, q - 1 2 T r[σσ T Q 11 ] -g K (x) -α T σ T Q 12 - 1 2 α 2 Q 22 .
Because of the unbounded control α, the Hamiltonian function H can take +∞ values. Therefore, to give a precise meaning to the HJB equation satisfied by w, we will use an approach based on some ideas presented in [START_REF] Brüder | Super-replication of European options with a derivative asset under constrained finite variation strategies[END_REF] and exploited also in [START_REF] Bokanowski | Numerical approximation for a superreplication problem under gamma contraints[END_REF]. We point out that similar ideas have been introduced in [START_REF] Rampazzo | The Minimum Time Function with Unbounded Controls[END_REF][START_REF] Rampazzo | Hamilton-Jacobi-Bellman equations with fast gradientdependence[END_REF] for deterministic control systems and in [START_REF] Motta | Uniqueness results for boundary value problems arising from finite fuel and other singular and unbounded stochastic control problems[END_REF] for stochastic problems where the unbounded control acts only on the drift.

Let us first introduce some useful notations. For any t ∈ (0, T ), x ∈ R d , q ∈ R d+1 , r ∈ R and Q ∈ S d+1 , define a, c ∈ R and B ∈ R p as follows:

a := -r -b(t, x, u), q - 1 2 T r[σσ T (t, x, u)Q 11 ] -g K (x), B := - 1 2 σ T (t, x, u)Q 12 , and c := - 1 2 Q 22 .
Let us furthermore denote by B = (B 1 , . . . , B p ) T the components of the vector B.

Let L u ∈ S p+1 be the matrix defined as follows:

L u (t, x, r, q, Q) := a B T B cI p ≡         a B 1 B 2 . . . B p B 1 c 0 . . . 0 B 2 0 . . . . . . . . . . . . . . . . . . . . . 0 B p 0 . . . 0 c         . (4.2)
Finally, let Λ + (A) be the largest eigenvalue of a given matrix A. Lemma 4.1. For any t ∈ (0, T ), (x, z) ∈ R d+1 , q ∈ R d+1 , r ∈ R and Q ∈ S d+1 , The following assertion holds:

-r + H(t, x, q, Q) ≤ 0 ⇔ sup u∈U Λ + (L u (t, x, r, q, Q)) ≤ 0.
Proof of Lemma 4.1. Straightforward calculations yield to:

-r + H(t, x, q, Q) ≤ 0 ⇔ sup (u,α)∈U ×R p a + 2 α, B + c α 2 ≤ 0 ⇔ sup (u,β)∈U ×R p+1 , β1 =0 a + 2 p i=1 B i β i+1 β 1 + c p i=1 β 2 i+1 β 2 1 ≤ 0 ⇔ sup (u,β)∈U ×R p+1 , β =1 aβ 2 1 + 2 p i=1 B i β i+1 β 1 + c p i=1 β 2 i+1 ≤ 0 ⇔ sup u∈U sup β∈R p+1 β =1 β T L u (t, x, r, q, Q)β ≤ 0 ⇔ sup u∈U Λ + (L u (t, x, r, q, Q)) ≤ 0
which proves the desired result. Remark 4.2. Notice also that, with the notations of equation (4.2),

Λ + (L u (t, x, r, q, Q)) = 0 ⇔ max(a, c, B 2 -ac) = 0. (4.3)
Indeed, straightforward calculations give that max α∈R p (a + 2 α, B + c α 2 ) = max α ,α∈R p (a + 2 B α + c α 2 ), from which we deduce (4.3).

For convenience of the reader we report below the explicit definition of L u , for t ∈ [0, T ], x ∈ R d , r ∈ R, q = (q x , q z ) ∈ R d+1 , and Q ∈ S d+1 :

L u (t, x, r, q, Q) = -r -b(t, x, u), q x + (t, x, u)q z -1 2 T r[σσ T (t, x, u)Q 11 ] -g K (x) -1 2 (σ T (t, x, u)Q 12 ) T -1 2 σ T (t, x, u)Q 12 -1 2 Q 22 I p .
Theorem 4.1. Assume that (H1),(H2) and (H3) hold. Then, w is a continuous viscosity solution of the following equation:

sup u∈U Λ + (L u (t, x, ∂ t w, Dw, D 2 w)) = 0 in [0, T ) × R d × (0, +∞) (4.4a) w(t, x, 0) = w 0 (t, x) in [0, T ) × R d , (4.4b) w(T, x, z) = max(ψ(x) -z, 0) in R d × [0, +∞). (4.4c)
Proof. The boundary condition (4.4b) and (4.4c) are satisfied thanks to the definition of w and Lemma 3.

8. Let ϕ ∈ C 1,2 ([0, T ] × R d+1 ) and let ( t, x, z) ∈ [0, T ) × R d × (0, +∞) be such that (w -ϕ)( t, x, z) = max [0,T )×R d ×(0,+∞) (w -ϕ)(t, x, z).
By using the dynamic programming principle, we obtain that w(t, x, z) ≤ w(t + h, X u t,x (t + h), Z u,α t,x (t + h)) for any constant controls u(•) ≡ u ∈ U and α(•) ≡ α ∈ R p . This leads easily to ϕ( t, x, z) -ϕ(t + h, X u t,x (t + h), Z u,α t,x (t + h)) ≤ 0, and therefore, for any (u, α) ∈ U × R p , by using Itô's calculus,

-∂ t ϕ -b, q - 1 2 T r[σσ T D 2 11 ϕ] -g K (x) -α T σ T D 2 12 ϕ - 1 2 α 2 D 2 22 ϕ ≤ 0
(where (D 2 ij ϕ) denotes the block of the matrix D 2 ϕ). By taking the supremum on (u, α) this implies in particular the following

-∂ t ϕ( t, x, z) + H( t, x, Dϕ, D 2 ϕ) ≤ 0. Also, by Lemma 4.1, it holds sup u∈U Λ + (L u ( t, x, ∂ t ϕ, Dϕ, D 2 ϕ)) ≤ 0
and the sub-solution property is proved.

To prove the super-solution property, consider the test function

ϕ ∈ C 1,2 ([0, T ] × R d+1 ) and ( t, x, z) ∈ [0, T ) × R d × (0, +∞) s.t.: (w -ϕ)( t, x, z) = min [0,T )×R d ×[0,+∞) (w -ϕ)(t, x, z).
Without loss of generality, we can consider that ( t, x, z) is a strict minimizer. Define the set

M(ϕ) := (t, x, z) ∈ [0, T ) × R d × (0, +∞), sup u∈U Λ + (L u (t, x, ∂ t ϕ, Dϕ, D 2 ϕ)) < 0 ,
and assume that ( t, x, z

) ∈ M(ϕ). Since M(ϕ) is an open set in [0, T ) × R d × (0, ∞), there exists η > 0 such that S η ( t, x, z) := [0 ∨ ( t -η), t + η] × B((x, z), η) ⊂ M(ϕ).
Applying the same techniques exploited in [START_REF] Pham | On some recent aspects of stochastic control and their applications[END_REF]Lemma 3.1], it is possible to prove that min

∂ P (Sη( t,x,z)) (w -ϕ) = min Sη( t,x,z) (w -ϕ)
where

∂ P (S η ( t, x, z)) := [0 ∨ t -η, t + η] × ∂B((x, z), η) ∪ { t + η} × B((x, z), η)
is the forward parabolic boundary of S η . But since ( t, x, z) is a strict minimizer, for η small enough the contradiction is obtained since ( t, x, z) / ∈ ∂ p (S η ( t, x, z)). We can conclude that ( t, x, z) / ∈ M(ϕ) and w is a viscosity super-solution of (4.4).

Remark 4.3. We point out that problem (4.4) is equivalent to

           sup u∈U,ξ∈R p+1 ξ =1 ξ T L u (t, x, ∂ t w, Dw, D 2 w)ξ = 0 [0, T ) × R d × (0, +∞), w(t, x, 0) = w 0 (t, x) [0, T ) × R d , w(T, x, z) = max(ψ(x) -z, 0) R d × [0, +∞). (4.5)
Remark 4.4. We notice that, for any η > 0,

Λ + a B T B cI p ≤ 0 ⇐⇒ Λ + a ηB T ηB η 2 cI p ≤ 0. (4.6)
Hence, for any given continuous function λ : (0, +∞) → (0, +∞), defining the operator L u λ by:

L u λ (t, x, r, q, Q) := -r -b, q x + q z -1 2 T r[σσ T Q 11 ] -g K (x) -1 2 λ(z)(σ T Q 12 ) T -1 2 λ(z)σ T Q 12 -1 2 λ 2 (z)Q 22 I p (
where the dependence of the function , b, σ on (t, x, u) is omitted for clarity of presentation), the equation (4.4) is equivalent to:

sup u∈U Λ + (L u λ (t, x, ∂ t w, Dw, D 2 w)) = 0 in [0, T ) × R d × (0, +∞), (4.7a) w(t, x, 0) = w 0 (t, x) in [0, T ) × R d , (4.7b)
w(T, x, z) = max(ψ(x) -z, 0) in R d × [0, +∞), (4.7c)
Theorem 4.2. Assume that (H1), (H2) and (H3) are satisfied. Let

w ∈ U SC([0, T ]× R d × [0, +∞)) and w ∈ LSC([0, T ] × R d × [0, +∞))
be respectively a viscosity sub-and super-solution of (4.4a). Assume that w and w satisfy the following growth condition

w(t, x, z) ≤ C(1 + |x|) w(t, x, z) ≥ C(1 + |x|)
and the inequalities

w(T, x, z) ≤ max(ψ(x) -z, 0) ≤ w(T, x, z) (4.8)
w(t, x, 0) ≤ w 0 (t, x) ≤ w(t, x, 0). (4.9)

Then

w(t, x, z) ≤ w(t, x, z), for all (t, x, z) ∈ [0, T ] × R d × [0, +∞).
The proof of this theorem is given in the next section.

Comments and conclusion. Theorems 4.1 and 4.2 lead to the characterization of the auxiliary function w as the unique continuous viscosity solution of the HJB equation (4.4) (or equivalently (4.5)). This result is obtained under the assumptions (H1), (H2) and (H3).

On the other hand, under the additional assumption (H4), Theorem 3.1 characterizes the original value function ϑ by the region where the continuous function w vanishes. Here, no particular degeneracy assumption is needed and the function ϑ may take infinite values.

Proof of the comparison

Principle. This section is devoted to the proof of the comparison principle between upper semi-continuous (USC) viscosity sub-solution and lower semi-continuous (LSC) super-solution of equation (4.4).

The proof will follow the main lines of the proof presented in [START_REF] Brüder | Super-replication of European options with a derivative asset under constrained finite variation strategies[END_REF] where an optimal control problem with a one-dimensional unbounded control α appeared. We will have furthermore to deal with some technical difficulties coming here from the extention of the result to higher dimensions for the unbounded control α.

Proof. In the proof we shall in particular consider the following non-negative, Lipschitz continuous function

λ(z) := max(1, z), z ≥ 0, (5.1)
and use the fact that the HJB equation (4.4) is equivalent to (4.7) by Remark 4.4.

(Other non negative and Lipschitz continuous functions could be used). We start by proving that there exists a strict viscosity sub-solution of (4.7a). Let us introduce the function ζ defined by

ζ(t, z) := -(T -t) -ln(1 + z). One has that ζ ∈ C ∞ ([0, T ] × [0, +∞)) for all (t, z) ∈ [0, T ] × [0, +∞), ζ(t, z) ≤ 0
and, by elementary computations using block matrix notations,

L u λ (t, x, ∂ t ζ, Dζ, D 2 ζ) = -1 -(t, x, u) 1 1+z 0 0 -1 2 λ 2 (z) 1 (1+z) 2 I p .
Using the positivity assumption of the function , it follows that -1 -(t, x, u)/(1 + z) ≤ -1. Moreover using the fact that

1 4 ≤ max(1, z 2 ) (1 + z) 2 ≤ 1 2
we can conclude that

sup u∈U Λ + (L u λ (t, x, z, ∂ t ζ, Dζ, D 2 ζ)) ≤ - 1 4 . (5.2)
Next, let w η be defined by w η (t, x, z) := w(t, x, z) + ηζ(t, z).

We are going to prove that for any η > 0, w η is a strict viscosity sub-solution of (4.7a) in [0, T ] × R × [0, +∞) with a controlled gap, in the following sense:

sup u∈U Λ + (L u λ (t, x, ∂ t w η , Dw η , D 2 w η )) ≤ - η 4 , (5.3)
in the viscosity sense. Indeed, the boundary conditions are satisfied for w η , thanks to the non positivity of ζ, and we have w η (T, x, z) ≤ ψ(x) -z, w η (t, x, 0) ≤ w 0 (t, x).

Furthermore, let us consider a test function

ϕ ∈ C 1,2 ([0, T ] × R d × [0, +∞)) such that (w η -ϕ)(t, x, z) = max(w η -ϕ)(•)
for a given (t, x, z) ∈ [0, T ) × R d × (0, +∞). From the definition of w η , we have

w -(-ηζ + ϕ) (t, x, z) = max w -(-ηζ + ϕ) .
Therefore we can consider that the following function

ϕ := -ηζ + ϕ
is a test function for w at the point (t, x, z). By using the sub-linearity property of the operator Λ + , the sub-solution property for ϕ, and (5.2), the following inequalities hold:

sup u∈U Λ + (L u λ (t, x, ∂ t ϕ, Dϕ, D 2 ϕ)) ≤ sup u∈U Λ + (L u λ (t, x, ϕ t , Dϕ, D 2 ϕ)) + η sup u∈U Λ + (L u λ (t, x, ∂ t ζ, Dζ, D 2 ζ)) ≤ - η 4 ,
hence the desired result. Now, let us define for δ > 0, η > 0 and ρ > 0:

Φ δ,η,ρ (t, x, z) := w η (t, x, z) -w(t, x, z) -2δe -ρt (1 + |x| 2 + z).

For a given ρ > 0 (that will be made precise later on), for δ, η small enough, we aim to prove that Φ δ,η,ρ (t, x, z) ≤ 0, ∀(t, x, z).

(5.4)

Letting δ → 0 and η → 0, this will give the desired inequality w ≤ w.

Let us assume that (5.4) is false. Let ( t, x, ẑ) ≡ ( tδ,η,ρ , xδ,η,ρ , ẑδ,η,ρ ) be a maximum point for Φ δ,η,ρ (this maximum point exists because of the linear growth condition satisfied by w η -w): Φ δ,η,ρ ( t, x, ẑ) > 0.

(5.5)

We aim to show a contradiction. Notice that we have t < T , as well as ẑ > 0, because of the boundary conditions (4.9) and (4.9) which would otherwise contradicts (5.5).

The next passage is a doubling of variable argument in order to prove the comparison principle. Let Φ be defined by Φ (t, x, x , z, z ) :=

w η (t, x, z) -w(t, x , z ) -δe -ρt (1 + |x| 2 + z) -δe -ρt (1 + |x | 2 + z ) - 1 2 (|x -x | 2 + |z -z | 2 ) - 1 2 |t -t| 2 - 1 4 (|x -x| 4 + |z -ẑ| 4 ).
Let ( t, x, x , z, z ) := ( t , x , x , z , z ) be a maximum point for Φ . By standard arguments (see [START_REF] Crandall | User's guide to viscosity solutions of second order partial differential equations[END_REF]Proposition 3.7] for instance) we can prove that, in the limit → 0, one has

|x -x | → 0, |z -z | → 0 and t → t, x, x → x, z, z → ẑ, |x -x | 2 → 0, |z -z | 2 → 0. (5.6)
Let us define f δ,ρ , fδ,ρ , and ϕ as follows:

f δ,ρ (t, x, z) := δe -ρt (1 + |x| 2 + z) + 1 2 |t -t| 2 + 1 4 (|x -x| 4 + |z -ẑ| 4 ), fδ,ρ (t, x, z) := δe -ρt (1 + |x| 2 + z) ϕ(x, x , z, z ) := 1 2 (|x -x | 2 + |z -z | 2 ),
so that we can write Φ in the following way:

Φ (t, x, x , z, z ) := w η (t, x, z) -f δ,ρ (t, x, z) -w(t, x , z ) + fδ,ρ (t, x , z ) -ϕ(x, x , z, z ).
Applying the Crandall-Ishii Lemma (see [START_REF] Crandall | User's guide to viscosity solutions of second order partial differential equations[END_REF]Theorem 8.3] as well as section 3 of the same reference), we can find r, r ∈ R and two symmetric matrices X and X (depending of ), such that

r + r = ∂ t ϕ(x, x , z, z ) ≡ 0 (5.7a) (r + ∂ t f δ,ρ , D (x,z) (ϕ + f δ,ρ ), X + D 2 (x,z) f δ,ρ ) ∈ P 1,2,+ w η ( t, x, z) (5.7b) (-r -∂ t fδ,ρ , -D (x ,z ) (ϕ + fδ,ρ ), -X -D 2 (x ,z ) fδ,ρ ) ∈ P 1,2,-w( t, x , z ) (5.7c)
and

- 3 I 0 0 I ≤ X 0 0 X ≤ 3 I -I -I I (5.8)
where P 1,2,± denote the closures of the parabolic semijets (see [START_REF] Crandall | User's guide to viscosity solutions of second order partial differential equations[END_REF]). These definitions are recalled in Appendix B.

From the definition of viscosity sub-and super-solution, using (5.7b) and (5.7c) we have:

sup u∈U Λ + (L u λ ( t, x, r + ∂ t f δ,ρ , D (x,z) (ϕ + f δ,ρ ), X + D 2 (x,z) f δ,ρ )) ≤ - η 4 (5.9) and sup u∈U Λ + (L u λ ( t, x , -r -∂ t fδ,ρ , -D (x ,z ) (ϕ + fδ ), -X -D 2 (x ,z ) fδ )) ≥ 0. (5.10)
Let q, q and Q, Q be vectors and matrices defined by

q ≡ q 1 q 2 := D (x,z) (ϕ + f δ,ρ ), Q := D 2 (x,z) f δ,ρ q ≡ q 1 q 2 := D (x ,z ) (ϕ + fδ,ρ ), Q := D 2 (x ,z ) fδ,ρ .
Then from (5.9) and (5.10) we deduce that

η 4 ≤ sup u∈U Λ + (L u λ ( t, x , -r -∂ t fδ,ρ , -q , -X -Q )) (5.11) -sup u∈U Λ + (L u λ ( t, x, r + ∂ t f δ,ρ , q, X + Q)) ≤ sup u∈U Λ + (L u λ ( t, x , -r -∂ t fδ,ρ , -q , -X -Q )) -Λ + (L u λ ( t, x, r + ∂ t f δ,ρ , q, X + Q)) .
Now we aim to estimate from above the right hand side. Let A, A , X, X and Q, Q be defined in S p+1 as follows (we use block matrix notations)

A := -r -∂ t f δ,ρ -b( t, x, u)q 1 + ( t, x, u)q 2 -g K (x) 0 0 0 A := r + ∂ t fδ,ρ + b( t, x , u)q 1 -( t, x , u)q 2 -g K (x ) 0 0 0 , and
X := 1 2 T r[σσ T ( t, x, u)X 11 ] 1 2 λ(z)X T 12 σ( t, x, u) 1 2 λ(z)σ T ( t, x, u)X 12 1 2 λ 2 (z)X 22 I p X := 1 2 T r[σσ T ( t, x , u)X 11 ] 1 2 λ(z )(X 12 ) T σ( t, x , u) 1 2 λ(z )σ T ( t, x , u)X 12 1 2 λ 2 (z )X 22 I p , and
Q := -∂ t f δ,ρ + 1 2 T r[σσ T ( t, x, u)Q 11 ] 1 2 λ(z)Q T 12 σ( t, x, u) 1 2 λ(z)σ T ( t, x, u)Q 12 1 2 λ 2 (z)Q 22 I p Q := ∂ t fδ,ρ + 1 2 T r[σσ T ( t, x, u)Q 11 ] 1 2 λ(z)(Q 12 ) T σ( t, x, u) 1 2 λ(z)σ T ( t, x, u)Q 12 1 2 λ 2 (z)Q 22 I p .
We have

L u λ ( t, x , -r -∂ t fδ,ρ , -q , -X -Q ) = A + X + Q and L u λ ( t, x, r + ∂ t f δ,ρ , q, X + Q) = A -X -Q.
Therefore, using the sub-linearity of the operator Λ + , we obtain

η 4 ≤ sup u∈U Λ + (A + X + Q ) -Λ + (A -X -Q) ≤ sup u∈U Λ + (A -A) + Λ + ( X + X ) + Λ + ( Q + Q ) ≤ sup u∈U (Λ + (A -A)) (I) + sup u∈U (Λ + ( X + X )) (II) + sup u∈U (Λ + ( Q + Q )) (III)
.

(5.12)

The next step is to estimate separately the three terms (I), (II) and (III) of the right hand side of (5.12) and to show that they become negative in the limit → 0.

Estimate for (I): Direct computations give

∂ t f δ,ρ = ( t -t) -δρe -ρ t(1 + |x| 2 + z) (5.13) ∂ t fδ,ρ = -δρe -ρ t(1 + |x | 2 + z ) (5.14) and q ≡ q 1 q 2 = 2δe -ρ t x + (x -x)|x -x| 2 + 1 (x -x ) δe -ρ t + (z -ẑ) 3 + 1 (z -z ) (5.15) q ≡ q 1 q 2 = 2δe -ρ t x -1 (x -x ) δe -ρ t -1 (z -z ) . (5.16)
Using (5.7a) one gets

Λ + (A -A) = max ∂ t f δ,ρ + ∂ t fδ,ρ + b( t, x , u), q 1 + b( t, x, u), q 1 -( t, x , u)q 2 -( t, x, u)q 2 + g K (x) -g K (x ), 0 .
In the limit when → 0, using properties (5.6), we obtain a first bound of the form lim sup

→0 sup u∈U ∂ t f δ,ρ + ∂ t fδ,ρ (5.17)
≤ lim sup

→0 ( t -t) -δρe -ρ t(1 + |x| 2 + z) -δρe -ρ t(1 + |x | 2 + z ) ≤ -2δρe -ρ t(1 + |x| 2 + ẑ) ( 5
( t, x , u), q 1 + b( t, x, u), q 1 ≤ C(1 + |x|)(2δe -ρ t|x| + |x -x| 3 ) + C(1 + |x |)2δe -ρ t|x | + b( t, x , u) -b( t, x, u), 1 (x -x ) ≤ C(1 + |x|)(2δe -ρ t|x| + |x -x| 3 ) + C(1 + |x |)2δe -ρ t|x | + 1 |x -x | 2
Therefore, in the limit when → 0, using properties (5.6), we obtain a bound of the form lim sup for some constant C 1 ≥ 0. In the same way, using the Lipschitz properpty of (t, x, u) w.r.t the variable x, In particular, The closures P 1,2,+ v(t, x), P 1,2,-v(t, x) of these sets are defined as follows: P 1,2,± v(t, x) := (a, p, X) ∈ R × R d × S d , ∃(t n , x n ) → (t, x), ∃(a n , p n , X n ) ∈ P 1,2,± v(t n , x n ) s.t. (a n , p n , X n ) → (a, p, X) .

Λ + ( Q + Q) = max ∂ t f δ,
It is easy to observe that P 1,2,+ v(t, x) = -P 1,2,-(-v)(t, x).

3. 3 .

 3 Properties of the auxiliary value function w. Proposition 3.7. Assume that (H1), (H2) and (H3) are satisfied. Then (i) w is continuous w.r.t. the variable t and Lipschitz continuous w.r.t. the variables (x, z). (ii) Moreover, w satisfies the following growth condition on [0, T ] × R d × [0, +∞):

4 .

 4 The HJB equation. In order to simplify the notations, we shall use in the sequel the functions b : [0, T ]×R d ×U → R d+1 and σ : [0, T ]×R d ×U ×R p → R (d+1)×p defined by: b(s, x, u) := b(s, x, u) -(s, x, u) and σ(s, x, u, α) := σ(s, x, u) α T .

. 18 )

 18 Using the linear growth |b(t, x, u)| ≤ C(1 + |x|), we first have b

1 ≤ C 1 ( 1 +

 111 , x , u), q 1 + b( t, x, u), q |x| 2 )δe -ρ t(5.19)

- 2 ≤≤ 2C 2 ( 1 + 2 (≤ 3 Σ -Σ 2 F≤ 3 Σ -Σ 2 FΣ 2 F β 2 2 F

 22123232222 ( t, x , u)q 2 -( t, x, u)q C(1 + |x |)(δe -ρ t + |z -ẑ| 3 ) + C(1 + |x|)δe -ρ t + |x -x| 1 (z -z) and, in the limit when → 0, using2 |x -x ||z -z| ≤ 1 |x -x | 2 + 1 |z -z| 2 , we obtain a bound of the form lim sup →0 sup u∈U -( t, x , u)q 2 -( t, x, u)q 2 ≤ C 2 (1 + |x|)δe -ρ t |x| 2 )δe -ρ t (5.20)for some constant C 2 ≥ 0. Summing up the bounds (5.18),(5.19) and (5.20), and using also g K (x)-g K (x ) ≤ L g K |x -x | → 0 as → 0, we obtain a bound of the form lim sup→0 sup u∈U Λ + (A -A) ≤ max -(ρ -C 3 )δe -ρ t(1 + |x| 2 ), 0 , with the constant C 3 := C 1 + 2C 2 . Choosing ρ large enough (ρ ≥ C 3 ) we obtain lim sup →0 sup u∈U Λ + (A -A) ≤ 0.Estimate for (II): Let the block matrices Σ, Σ ∈ R (p+1)×(d+1) be as follows:Σ := σ T ( t, x, u)Let ξ be an arbitrary vector in R p+1 . Multiplying inequality (5.8) by ξ T (Σ Σ ) on the left side and by (Σ Σ ) T ξ on the right side we obtainξ T (Σ Σ ) ξ T (Σ X Σ T + ΣXΣ T )ξ ≤ 3 ξ T (Σ -Σ )(Σ T -Σ T )ξ ≤ 3 Σ -Σ T 2F ξ using Cauchy-Schwartz inequalities) where • F denotes the Frobenius matrix norm. In particular, for k ∈ {1, . . . , p}, choosing ξ of the formξ ≡ (0 . . . 0 β 0 k-th 0 . . . 0 β k ) a straightforward calculation gives β 2 0 σ T X 11 σ( t, x, u) + σ T X 11 σ( t, x , u) kk + β 2 k λ 2 (z)X 22 + λ 2 (z )X 22 +2β 0 β k λ(z)σ T ( t, x, u)X 12 + λ(z )σ T ( t, x , u)X 12 k (β 2 0 + β 2 k ), ∀k = 1, . . . , p.It follows that for anyβ ∈ R p+1 β T ( X + X )X 11 σ( t, x, u) + σ T X 11 σ( t, x , u) kk + β 2 k λ 2 (z)X 22 + λ 2 (z )X 22 +2β 0 β k λ(z)σ T ( t, x, u)X 12 + λ(z )σ T ( t, x , u)X 12 k This implies in particular that sup u∈U Λ + ( X + X ) ≤ 3p Σ -Σ 2 F .Thanks to the definition of Σ and Σ and the Lipschitz continuity of σ and λ: lim sup →0 sup u∈U Λ + ( X + X ) ≤ lim sup →0 3p Σ -Σ desired result. Estimate for (III): Direct calculations show that Q = (2δe -ρ t + |x -x| 2 )I d + 2(x -x)(x -x)

  By the form of Q 11 and Q 11 , and the linear growth|σ(t, x, u)| ≤ C(1 + |x|), we obtain ( t, x , z )Q 11 ] ≤ C 4 (1 + |x| 2 )δe -ρ t.

	ρ + ∂ t fδ,ρ +	1 2	T r[σσ T ( t, x, u)Q 11 ] +	1 2	T r[σσ T ( t, x , z )Q 11 ],
	1 2	λ 2 (z)Q 22 +	1 2	λ 2 (z )Q 22
	lim sup →0 T r[σσ T for some constant C 4 ≥ 0. Hence sup u∈U 1 2 T r[σσ T ( t, x, u)Q 11 + 1 2
	lim sup

→0 sup u∈U Λ + ( Q + Q) ≤ max -2δρe -ρ t(1 + |x| 2 + ẑ) + C 4 (1 + |x| 2 )δe -ρ t, 0

Therefore choosing ρ ≥ 1 2 C 4 , we obtain lim sup

To conclude, we fix ρ = max(C 3 , 1 2 C 4 ). Using the previous estimates for the terms (I), (II) and (III), and from inequality (5.11) we finally get η 4 ≤ 0, leading to the desired contradiction. This concludes the proof of Theorem 4.2.

Appendix A. A result of existence of optimal controls. We aim to show that assumption (H4), required in Section 3.2, is always valid for a particular class of control problems. Recall that we are interested in the SOCP

associated with the cost

for given functions g T and g K and for the dynamics given by (2.1) and (3.1). As pointed out in Remark 3.6, when dealing with existence results of an optimal control law for (A.1), one difficulty encountered arises from the unboundedness of the control α.

As in [39, Theorem 5.2, Chapter II]: we make the following linear/convexity assumptions:

where A, B, C, D, E and F are L ∞ functions with values in matrix spaces of suitable sizes; (B2) U ⊂ R m is a convex and closed set; (B3) g T and g K are Lipschitz continuous and convex functions, bounded from below. Theorem A.1. Assume (B1)-(B3) are satisfied. Then for any t ∈ [0, T ], (x, z) ∈ R d+1 , there exists an optimal control (ū, ᾱ) ∈ U × A for the problem (A.1)-(A.2).

Proof. The proof reported below is strongly based on the arguments in [39, Theorem 5.2, Chapter II]. First, the boundednesss assumption (B3) easily gives the existence of the value w(t, x, z). Then let (u j , α j ) j≥1 ∈ U × A be a sequence of minimizing controls, that is lim j→+∞ J(t, x, z, u j , α j ) = w(t, x, z).

Thanks to the compactness of U and the uniform bound on the L 2 F -norm that can be obtained on the control α (see Remark 3.3) we can extract from (u j , α j ) a subsequence (still indexed with j) such that (u j , α j ) → (ū, ᾱ) weakly in L 2 F .

Let > 0. There exists j 1 such that ∀j ≥ j 1 , J(t, x, z, u j , α j ) ≤ w(t, x, z) + .

By the convex structure of U × R p and as a consequence of Mazur's Lemma, there exists a convex combination:

which is strongly convergent to the same limit, that is (ũ j , αj ) → (ū, ᾱ) strongly in L 2 F -norm. One can observe that (ũ j , αj ) and (ū, ᾱ) are still elements of U × A thanks to the convexity and closure of U .

Then, by assumption (A1), it is easy to verify that

and

strongly in L ∞ F -norm. Hence there exists j 2 such that for any j ≥ j 2 , J(t, x, z, ū, ᾱ) ≤ J(t, x, z, ũj , αj ) + . Now, for j ≥ max(j 1 , j 2 ), using the convexity assumption (B3) and (A.3), it holds: Letting ↓ 0, we obtain J(t, x, z, ū, ᾱ) ≤ w(t, x, z) and therefore (ū, ᾱ) is an optimal control for (A.1).

Appendix