NECESSARY CONDITIONS FOR INFINITE HORIZON OPTIMAL CONTROL PROBLEMS WITH STATE CONSTRAINTS

Keywords: Contrôle optimal à l'horizon infini, Contraintes d'état, Conditions nécessaires, Fonction valeur, Régularité Lipschitz, Équations d'Hamilton-Jacobi-Bellman, Semiconcavité Controllo ottimo orizzonte infinito, Vincoli di stato, Condizioni necessarie, Funzione valore, Continuità Lipschitz, Equazioni di Hamilton-Jacobi-Bellman, Semiconcavità Infinite horizon optimal control, Pure state constraints, Necessary conditions, Value function, Lipschitz continuity, Hamilton-Jacobi-Bellman equations, Semiconcavity. XI

RESUMÉ

Dans cette thèse, nous abordons des problèmes de contrôle optimal non autonomes à l'horizon infini soumis à des contraintes d'état. Des relations de sensibilité, partielle et totale, sont obtenues, en supposant que la fonction valeur associée soit localement Lipschitzienne par rapport à la variable d'état. Nous discutons également des conditions suffisantes pour la régularité Lipschitz de la fonction valeur. Nous nous concentrons sur les problèmes liés aux fonctions de coût admettant un facteur d'actualisation, avec la dynamique et le Lagrangien dépendant du temps. De plus, les contraintes d'état peuvent être non-bornés et peuvent avoir une frontière non lisse. La régularité Lipschitz est obtenue à partir d'estimations sur la distance d'une trajectoire donnée de l'ensemble de toutes les trajectoires viables, à condition que le taux d'actualisation soit suffisamment élevé. Nous étudions également l'existence et l'unicité des solutions faibles des équations non autonomes d'Hamilton-Jacobi-Bellman sur un domaine de la forme (0, ∞)×A. L'Hamiltonien est supposé être uniquement mesurable par rapport au temps et l'ensemble A est fermé. En présence de contraintes d'état, (en général) l'équation d'Hamilton-Jacobi-Bellman n'admet pas de solutions continues. Dans ce travail, nous proposons une notion de solution faible pour laquelle, sous une hypothèse de contrôlabilité appropriée, les théorèmes d'existence et d'unicité sont valides dans la classe des fonctions semi-continues inférieurement s'annulant à l'infini. Enfin, nous étudions une équation autonome d'Hamilton-Jacobi-Bellman sur un sous-ensemble compact, avec des conditions de Dirichlet sur la frontière. Dans ce contexte, nous obtenons des résultats de semi-concavité de l'unique solution de l'équation et les relations de sensibilité sous la forme d'inclusions différentielles. Nous étendons ainsi un résultat connu pour la distance sous-Riemannienne sous la condition d'Hörmander.

where f : [0, ∞) × R n × R m → R n and L : [0, ∞) × R n × R m → R are given functions, and (t 0 , x 0 ) ∈ [0, ∞) × R n is the initial datum. The variable x(•) represents the state variable (also called trajectory) of the system, u(•) is the control, and f , L are the dynamics and the Lagrangian, respectively. Conditions are assumed on the dynamics to ensure the uniqueness of solutions x(•) of (2) for each initial datum and each control u(•). Moreover, integrability conditions on the Lagrangian are imposed to ensure the existence of the integral in (1).

Usually, in most control problems and science models, constraints on state variables and controls are to be imposed. A way to express restrictions on the control u(•) is by using a Lebesgue measurable set-valued map U : [0, ∞) ⇒ R m with closed nonempty images, i.e., u : [t 0 , ∞) → R m is assumed to be Lebesgue measurable and such that u(t) ∈ U (t) for a.e. t t 0 .

(3)

On the other hand, state constraints are, at least, of two different types: functional state constraints and set state constraints. The former requires the trajectory x(•), with INTRODUCTION initial datum (t 0 , x 0 ), to satisfy h(t, x(t)) 0 for all t t 0 , where h : [0, ∞) × R n → R is a given function. The latter requests that

x(t) ∈ A ∀t t 0 , ( 4
)
where A is a closed subset of R n . We focus on infinite horizon control problems subject to set state constraints (4) and we refer to problem (1)-(4) as B ∞ .

We point out that problem B ∞ is not well posed without suitable assumptions on the dynamics and Lagrangian. Indeed, control system (2)-(3) may not admit trajectories laying in A. Hence, a trajectory-control pair (x(•), u(•)) that satisfies (2)-( 4) is called feasible. We refer to such x(•) as a feasible trajectory. The infimum of the cost functional in (1) over all feasible trajectory-control pairs, with the initial datum (t 0 , x 0 ), is denoted by V (t 0 , x 0 ). If no feasible trajectory-control pair exists at (t 0 , x 0 ), or if the integral in (1) is not defined for every feasible pair, then we set V (t 0 , x 0 ) = +∞. The function

V : [0, ∞) × A → R ∪ {±∞} is called the value function of problem B ∞ .

State of the art

Infinite time horizon models arising in mathematical economics and engineering typically involve control systems with restrictions on both controls and states. Models of optimal allocation of economic resources were, in the late 50s, among the key incentives for the creation of the mathematical theory of optimal control. Moreover, economic systems are often assumed to operate for an infinitely long (or at least indefinitely long) time. Indeed, if one fixes a certain finite planning horizon for the growth process, one leaves it uncertain how the economic system will develop after a fixed time (see, for instance, the Ramsey macroeconomics model in [START_REF] Ramsey | A mathematical theory of saving[END_REF]).

The goal of optimal control theory is to find necessary and sufficient conditions for optimality in order to construct optimal controls, that is, controls for which a given functional reaches the minimum.

A possible approach to the problem is to give necessary conditions for a control to be optimal. This restricts the set of all controls to a smaller set, that should be further investigated to check if necessary conditions are also sufficient. These conditions are often available in the form of Pontryagin's maximum (or minimum) principle. Such result was formulated in the 50s by the Russian mathematician Lev Pontryagin and it has, as a special case, the Euler-Lagrange equation and Weierstrass condition of the calculus of variations. For constraint-free infinite horizon problems, the maximum principle takes the following form: if (x(•), ū(•)) is an optimal trajectory-control pair for the unconstrained infinite horizon control problem (1)-(3) at (t 0 , x 0 ) ∈ [0, ∞) × R n , then there exists an absolutely continuous co-state p : [t 0 , ∞) → R n and λ ∈ {0, 1} satisfying:

• (p, λ) = 0; The above necessary conditions for constraint-free problems have been extensively studied in the literature. We should underline that, in economic publications, methods of the mathematical optimal control theory, including necessary optimality conditions, are often applied without due mathematical validation. The infinite planning horizon may cause various phenomena in the relations of the maximum principle, and this possibility must be taken into account if one wants to avoid misleading conclusions. A possible cause of fallacy when one considers control problems with infinite horizon, subject or not to state constraints, is to assume that the necessary conditions, which are valid for the finite horizon case, can be carried to the infinite horizon framework by just replacing evaluations of quantities at the terminal time with evaluations of the limit of the same quantities as time tends to infinity (this was already observed by Halkin [START_REF] Halkin | Necessary conditions for optimal control problems with infinite horizons[END_REF] who provided several counterexamples to this way of doing for problems without state constraints). When state constraints are present, it is neither correct to think that if a trajectory-control pair satisfies both the constraints and the maximum principle, then it is optimal. We show in the next example that to apply necessary conditions stated for unconstrained problems to constrained ones may lead to contradictions.

Example. Consider the following problem: maximize J(u) = ∞ 0 e -λt (x(t) + u(t)) dt over all trajectory-control pairs (x(•), u(•)) satisfying x (t) = -au(t) and u(t) ∈ [-1, 1] for a.e. t 0, x(0) = 1, where a > λ > 0, and subject to the following state constraints

x(t) ∈ (-∞, 1] ∀ t 0.
Applying the maximum principle for unconstrained problems, it follows that any optimal trajectory-control pair satisfies one of the following three relations:

• x -(t) = 1 + at associated with u -(t) ≡ -1;

INTRODUCTION

• x + (t) = 1 -at associated with u + (t) ≡ +1;

• x ± (t) = (1-at)χ [0, t] (t)+(1-a t+a(t-t))χ ( t,∞) (t) associated with u ± (t) = χ [0, t] (t)χ ( t,∞) (t), for some t > 0.

Excluding now the trajectories x -and x ± , since they are not feasible, this analysis leads to the conclusion that x + is the only candidate for optimality. But one can easily see that the feasible trajectory x(t) ≡ 1, associated with the control u(t) ≡ 0, verifies J(ū) > J(u + ).

Controllability assumptions. In presence of state constraints, V may not be continuous, unless the dynamics satisfy a controllability assumption on the boundary of A. An example of such condition, called inward pointing condition, was introduced by Soner (see [START_REF] Soner | Optimal control problems with state-space constraints I[END_REF]) and was later extended to less restrictive frameworks (cfr. [START_REF] Clarke | Hamilton-jacobi characterization of the state constrained value[END_REF][START_REF] Frankowska | Discontinuous solutions of Hamilton-Jacobi-Bellman equation under state constraints[END_REF]). Such an assumption requires that at each point of A there exists an admissible velocity pointing inward the constraint set. More precisely, under the assumption that A is a bounded open domain with smooth boundary, f is continuous and time independent, and U (•) ≡ U , the inward pointing condition is as follows: for all x ∈ ∂A there exists u ∈ U satisfying f (x, u), n(x) < 0, (5)

and set F (t, x) := f (t, x, U (t)) ∀t 0, ∀x ∈ R n .

Any locally absolutely continuous function x(•) satisfying (6) on a closed subinterval of R is called F -trajectory. The neighboring feasible trajectories theorem ensures that for any 0 < t 0 < T , any ε > 0, and every F -trajectory x(•) on [t 0 , T ], with x(t 0 ) ∈ A, there exists β > 0 and an F -trajectory x(•) on [t 0 , T ], starting from x(t 0 ) = x(t 0 ), satisfying

x -x ∞ βρ, x((t 0 , T ]) ⊂ int A, (

where ρ = ε + max s∈[t 0 ,T ] d A (x(s)). We point out that the constant β depends on ε and also on the time interval [t 0 , T ]. Under suitable assumptions on F , neighboring feasible trajectories theorems ensure at least the continuity of the value function.

In [START_REF] Bettiol | L ∞ estimates on trajectories confined to a closed subset[END_REF] the authors provided an analogous result extending Soner's condition to sets A which are merely closed and assuming that the dynamics F is absolutely continuous in time and locally Lipschitz continuous with respect to space, under the following inward controllability assumption:

co F (t, x) int T C A (x) = ∅ ∀ (t, x) ∈ [0, T ] × ∂A, ( 8
)
where T C A (x) denotes the Clarke tangent cone to A at x. Furthermore, such theorems on bounded intervals are available when the dynamics is less regular with respect to time, under stronger estimates than the uniform one, as those in W 1,1 (cfr. [START_REF] Bettiol | Estimates for trajectories confined to a cone in R n[END_REF][START_REF] Bettiol | L ∞ estimates on trajectories confined to a closed subset[END_REF][START_REF] Bettiol | On trajectories satisfying a state constraint: W 1,1 estimates and counterexamples[END_REF]).

When the velocity set F is just measurable with respect to time, recovering neighboring feasible trajectories theorems becomes more challenging and additional controllability requirements are needed. In [START_REF] Frankowska | On relations of the adjoint state to the value function for optimal control problems with state constraints[END_REF] the authors extend these results, with F measurable in time and locally Lipschitz continuous with respect to state, providing W 1,1 -estimates under a stronger inward pointing condition than (8). Moreover, the authors in [START_REF] Frankowska | Filippov's and Filippov-Ważewski's theorems on closed domains[END_REF] showed a neighboring feasible trajectories theorem on unbounded intervals and for bounded dynamics, under the following controllability assumption: there exists r > 0 such that for all x ∈ ∂A and for any t 0 we can find v ∈ F (t, x) satisfying sup

n∈N A (x)∩S n-1 v, n < -r,
where N A (x) is the limiting normal cone to A at x.

INTRODUCTION

Necessary conditions for constrained problems

As a matter of fact, necessary conditions in the form of the maximum principle and partial sensitivity relations have been obtained for infinite horizon convex problems under smooth functional constraints such as (see [START_REF] Seierstad | Necessary conditions for nonsmooth, infinite-horizon, optimal control problems[END_REF][START_REF] Aseev | Infinite-horizon optimal control problems in economics[END_REF])

h(t, x(t)) 0. ( 9
)
For instance, suppose (x(•), ū(•)) is optimal at (t 0 , x 0 ) for the problem (1)-(3), subject to state constraints of the form (9), with U (t) ≡ U a closed convex subset of R m , h ∈ C 2 , f and L continuous together with their partial derivatives with respect to x and u.

Assuming further the controllability assumption inf u∈U ∇ x h(t, x(t)), f (t, x(t), u) -f (t, x(t), ū(t)) < 0 ∀t t 0 , one shows that there exist λ ∈ {0, 1}, a co-state q(•), and a nondecreasing function µ(•), constant on any interval where h(t, x(t)) < 0, such that (λ, q(t 0 )) = 0, µ(t 0 ) = 0, and q(•) satisfies the adjoint equation

q(t) = q(t 0 ) - t t 0
∇ x H λ (s, x(s), q(s), ū(s)) ds -

[t 0 ,t]
∇ x h(s, x(s)) dµ(s), and the maximum condition q(t), f (t, x(t), ū(t)) -λL(t, x(t), ū(t)) = H λ (t, x(t), q(t)) for a.e. t t 0 .

Furthermore, using the language of the calculus of variations, in [START_REF] Benveniste | Duality theory for dynamic optimization models of economics: the continuous time case[END_REF] the authors show that, under some very restrictive assumptions on f , if A is convex and int A = ∅ then, for any optimal trajectory x(•) of problem B ∞ , there exists an absolutely continuous arc q(•) which satisfies the adjoint equation and the partial sensitivity relation q(t) ∈ ∂ x V (t, x(t)) for all t t 0 .

Main result 1. We describe next our achievements on this topic. Let us assume that for all (t 0 , x 0 ) ∈ [0, ∞)×A the limit lim T →∞ T t 0 L(t, x(t), u(t)) dt exists for every trajectorycontrol pair (x(•), u(•)) satisfying (2)-(3) with initial datum (t 0 , x 0 ) and V (t 0 , x 0 ) = -∞ for all (t 0 , x 0 ) ∈ [0, ∞) × A. We impose the following regularity assumptions on f and L:

INTRODUCTION 11 (c) for all x ∈ R n the mappings f (•, x, •), L(•, x, •) are Lebesgue-Borel measurable; (d) for a.e. t > 0 and for all x ∈ R n the set {(f (t, x, u), L(t, x, u)) : u ∈ U (t)} is closed.

Moreover, we denote by (ipc) the following inward pointing condition:

(ipc) for any (t, x) ∈ [0, ∞) × ∂A there exists a set A t,x ⊂ [0, ∞), with µ L (A t,x ) = 0, such that for any v ∈ Lim sup (s,y)→(t,x), s / ∈At,x F (s, y), with max{ n, v : n ∈ N A (x) ∩ S n-1 } 0, we can find w ∈ Lim inf (s,y)→(t,x), s / ∈At,x co F (s, y) satisfying max n∈N A (x)∩S n-1 n, w -v < 0;

The Hamiltonian is defined by

H (t, x, p) := sup u∈U (t)
( p, f (t, x, u) -L(t, x, u)).

We give next the main result of this section.

Theorem (Necessary conditions for infinite horizon problems with state constraints, [START_REF] Basco | Necessary conditions for infinite horizon optimal control problems with state constraints[END_REF]). Assume (h1) and (ipc). Suppose that V (j, •) is locally Lipschitz continuous on A for all large j ∈ N. Then V is locally Lipschitz continuous on [0, ∞) × A. Moreover, if (x, ū) is optimal for B ∞ at (t 0 , x 0 ) ∈ [0, ∞) × int A, then there exist p ∈ W 1,1 loc (t 0 , ∞; R n ), a nonnegative Borel measure µ on [t 0 , ∞), and a Borel measurable function ν : [t 0 , ∞) → R n such that, setting q(t) = p(t) + η(t) with η(t 0 ) = 0, η(t) = [t 0 ,t] ν(s) dµ(s), ∀ t > t 0 , the following holds true: (i) ν(t) ∈ co N A (x(t)) ∩ B µ -a.e. t t 0 ;

(ii) p (t) ∈ co r : (r, q(t), -1) ∈ N graph G(t,•) (x(t), x (t), L(t, x(t), ū(t))) for a.e. t t 0 , where G(t, x) = {(f (t, x, u), L(t, x, u)) : u ∈ U (t)};

(iii) -p(t 0 ) ∈ ∂ + x V (t 0 , x(t 0 )); (iv) -q(t) ∈ ∂ 0

x V (t, x(t)) for a.e. t > t 0 ; (v) q(t), f (t, x(t), ū(t)) -L(t, x(t), ū(t)) = max u∈U (t) q(t), f (t, x(t), u) -L(t, x(t), u) for a.e. t t 0 ;

(vi) (H (t, x(t), q(t)), -q(t)) ∈ ∂ 0 V (t, x(t)) for a.e. t > t 0 , where

∂ 0 x V (t, x) := Lim sup

INTRODUCTION

The conditions (iv) and (vi) of the above Theorem are more precise sensitivity relations than those expressed in terms of the Clarke-Rockafellar subdifferential co ∂V (t, x(t)) and partial Clarke-Rockafellar subdifferential co ∂ x V (t, x(t)), since the definitions of ∂ 0 V (t, x) and ∂ 0 x V (t, x) involve limits which are taken within the interior of the set A. Indeed, one can see that co ∂V (t, x(t)) coincides with ∂ 0 V (t, x) for any (t, x) ∈ [0, ∞) × int A, while it may occur that ∂ 0 V (t, x) co ∂V (t, x(t)) for points (t, x) ∈ [0, ∞) × ∂A. We underline that the extended Euler-Lagrange inclusion in (ii) improves the one given in terms of Clarke's Hamiltonian inclusion. Furthermore, the transversality condition (iii) leads to a significant economic interpretation (see [START_REF] Aseev | On some properties of the adjoint variable in the relations of the Pontryagin maximum principle for optimal economic growth problems[END_REF], [START_REF] Seierstad | Optimal control theory with economic applications[END_REF]): the co-state p + η can be regarded as the 'shadow price' or 'marginal price', i.e., (iv) describes the contribution to the value function (the optimal total utility) of a unit increase of capital x (cfr. [PBGM64, Neu69, DM65, Gam60, VP82, VZ98]).

From the technical point of view, the results exposed in this thesis, concerning necessary conditions, rely on the idea of reformulating the infinite horizon problem B ∞ as a Bolza problem on each finite time interval, which can be analyzed in detail by appealing to the existing theory for finite horizon problems. Hence, problem B ∞ becomes a Bolza problem on [0, T ] with the additional final cost φ T (•) = V (T, •). Assuming the local Lipschitz regularity of V (T, •) and applying the necessary conditions for finite horizon control problems (cfr. [START_REF] Vinter | Optimal Control[END_REF][START_REF] Bettiol | Improved sensitivity relations in state constrained optimal control[END_REF]), we derive uniform bounds for the truncated co-states. Indeed, fixing any T > 0, we use that V (s, y) = inf {V (T, x(T )) + T s L(t, x(t), u(t)) dt} ∀(s, y) ∈ [0, T ] × A, (10) where the infimum is taken over all the feasible trajectory-control pairs (x(•), u(•)) satisfying (2)-(4) on [s, T ] with initial datum (s, y). Furthermore, if (x(•), ū(•)) is optimal at (t 0 , x 0 ) ∈ [0, T ] × int A for B ∞ , then the restriction of (x(•), ū(•)) to the time interval [t 0 , T ] is optimal for the Bolza problem on the right-hand side of (10) too.

to investigate the regularity of the value function, useful tools are neighboring feasible trajectory theorems. We observe that, in estimate (7), the constant β depends on time interval (as well as the choice of ε and reference trajectory x(•)). Recovering uniform estimates on such constant turns out to be helpful to construct feasible trajectories for the infinite horizon problem B ∞ .

To be more specific, denote by L loc the set of all ψ ∈ L 1 loc ([0, ∞); R + ) such that lim σ→0 θ ψ (σ) = 0, where θ ψ (σ) := sup{ J |ψ(τ )| dτ : J ⊂ I, µ L (J) σ} and consider the following assumptions on f and l: (h2) (a) for all x ∈ R n the mappings f (•, x, •), l(•, x, •) are Lebesgue-Borel measurable;

(b) there exist c

∈ L 1 loc ([0, ∞); R + ) and k ∈ L loc such that f (t,
∈ R n , u ∈ U (t); (c) for all (t, x) ∈ [0, ∞) × R n the set {(f (t, x, u), l(t, x, u)) : u ∈ U (t)} is closed; (d) there exists q ∈ L loc such that sup u∈U (t) (|f (t, x, u)| + |l(t, x, u)|) q(t) for all x ∈ ∂A and for a.e. t 0; (e) lim sup t→∞ t -1 t 0 (c(s) + k(s)) ds < ∞,
and a uniform inward pointing condition, stronger than (ipc):

(ipc u ) there exist η > 0, r > 0, M 0 such that for a.e. t 0, any y ∈ ∂A + ηB, and any v ∈ F (t, y), with sup n∈N 1 y,η n, v 0, there exists w ∈ F (t, y)

∩ B(v, M ) such that sup n∈N 1 y,η { n, w , n, w -v } -r, where N 1 y,η := {n ∈ S n-1 : n ∈ co N A (x), x ∈ ∂A ∩ B(y, η)}.
Next we state our uniform neighboring feasible trajectories result.

Lemma. Assume (h2) and (ipc u ). Then for every δ > 0 there exists a constant β > 0 such that for any

[t 0 , t 1 ] ⊂ [0, ∞) with t 1 -t 0 = δ, any F -trajectory x(•) defined on [t 0 , t 1 ] with x(t 0 ) ∈ A, and any ρ > 0 satisfying ρ sup t∈[t 0 ,t 1 ] d A (x(t)), we can find an F -trajectory x(•) on [t 0 , t 1 ] such that x(t 0 ) = x(t 0 ), x -x ∞,[t 0 ,t 1 ] βρ, x((t 0 , t 1 ]) ⊂ int A.
If F is more regular with respect to time, i.e., it is at least absolutely continuous from the left, then one may assume a weaker (uniform) inward pointing condition than (ipc u ) (see Chapter 2 for details). When further integrability assumptions on functions k(•) and q(•) are imposed, then we show that the constant β depends only on the length of the time interval and on the starting point (cfr. [START_REF] Frankowska | On relations of the adjoint state to the value function for optimal control problems with state constraints[END_REF][START_REF] Bettiol | L ∞ estimates on trajectories confined to a closed subset[END_REF]). The above result ensures, in particular, the existence of feasible trajectories. Moreover, it allows to deduce, assuming further that lim sup t→∞ t -1 t 0 (c(s) + k(s)) ds < ∞, that feasible INTRODUCTION trajectories depend on initial states in a Lipschitz way with an exponentially increasing in time Lipschitz constant (cfr. Section 3, Chapter 2).

Main result 2. We propose, in Chapter 2, sufficient conditions for the Lipschitz regularity of V , assuming both f and l to be time dependent, without requiring A to be compact and ∂A to be smooth. Our proof differs substantially from the one of the analogous Lipschitz continuity result contained in Chapter 1.

Theorem (Lipschitz continuity, [BF]). Assume (h2) and (ipc u ) hold true. Then there exist b > 1, K > 0 such that for all λ > K and every t 0 the function

V (t, •) is L(t)-Lipschitz continuous on A with L(t) = be -(λ-K)t .
Under assumptions (h2) and (ipc u ), if the dynamics and the Lagrangian are bounded, then the above Theorem implies that the value function V is locally Lipschitz continuous on [0, ∞)×A. However, this result holds again under a weaker inward pointing condition than (ipc u ), requiring more regularity with respect to time of the set-valued map t {(f (t, x, u), l(t, x, u)) : u ∈ U (t)} (see Chapter 2 for details).

Weak solutions of H-J-B equations with time-measurable data

The notion of weak (or viscosity) solution to a first-order partial differential equation was introduced in the pioneering works [CEL84, CL83, Lio82] by Crandall, Evans, and Lions to investigate stationary and evolutionary H-J-B equations, using sub/super solutions involving superdifferentials and subdifferentials of continuous functions. In particular, they obtained existence and uniqueness results in the class of continuous functions for Cauchy problems associated to H-J-B equations, when the Hamiltonian is continuous. In [START_REF] Barles | Existence results for first order Hamilton Jacobi equations[END_REF][START_REF] Souganidis | Existence of viscosity solutions of Hamilton-Jacobi equations[END_REF] the authors extended the existence results to a large class of continuous Hamiltonians. However, it is known that such notion of solution turns out to be quite unsatisfactory for H-J-B equations arising in control theory and the calculus of variations (we refer to [START_REF] Bardi | Optimal control and viscosity solutions of Hamilton-Jacobi-Bellman equations[END_REF][START_REF] Lions | Generalized solutions of Hamilton-Jacobi equations[END_REF] for further discussions). Indeed, the value function, that is a weak solution of H-J-B equation, loses the differentiability property (even in the absence of state constraints) whenever there are multiple optimal solutions at the same initial condition. When additional state constraints are present it may also lose its continuity. At most, we expect lower semicontinuity of the value function. Nevertheless, the study of uniqueness of weak solutions can be carried out by using the definition of solution from [START_REF] Frankowska | Measurable viability theorems and the Hamilton-Jacobi-Bellman equation[END_REF]. Previously, in order to deal with Hamiltonians which are measurable in time, Ishii ([Ish85]) proposed a new notion of weak solution (cfr. [START_REF] Lions | Remarks on Hamilton-Jacobi equations with measurable time-dependent Hamiltonians[END_REF] for equivalent formulations of such a kind of solutions) in the class of continuous functions, proving, by a blow-up method, the existence and uniqueness in the stationary case on a general open subset of R n and, for the evolutionary case, on (0, ∞) × R n .

Unfortunately, when addressing state constrained problems, the usual assumptions on data may be insufficient to derive existence and uniqueness results for the H-J-B equations. In [START_REF] Soner | Optimal control problems with state-space constraints I[END_REF], Soner proposed a controllability assumption to investigate an autonomous control problem, recovering the continuity of the value function through an inward pointing condition like in (5). Such condition implies uniqueness of viscosity solutions. However, such a property cannot be used for sets with nonsmooth boundaries and boundedness assumptions on A may be quite restrictive for many applied models: for instance, macroeconomics models often consider cones as state constraints. To allow for nonsmooth boundaries, Ishii and Koike generalized Soner's condition in the framework of infinite horizon problems and continuous solutions (cfr. [START_REF] Ishii | A new formulation of state constraint problems for first-order PDEs[END_REF] and the references therein). More generally, various versions of the inward pointing condition are useful to get the continuity or Lipschitz continuity of the value function, see for instance Chapter 2. Furthermore, in [START_REF] Frankowska | Hamilton-Jacobi equations for infinite horizon control problems with state constraints[END_REF][START_REF] Frankowska | Semicontinuous solutions of Hamilton-Jacobi-Bellman equations with degenerate state constraints[END_REF] the authors, dealing with paratingent cones and closed set of constraints with possibly empty interior, carry out the analysis under an outward pointing condition. Such condition ensures, roughly speaking, that any boundary point of A can be reached by trajectories laying in the relative interior of A. This property was used, in particular, in [START_REF] Frankowska | Discontinuous solutions of Hamilton-Jacobi-Bellman equation under state constraints[END_REF], to study an H-J-B equation on finite time interval, when the Hamiltonian is convex and positively homogeneous in the third variable. We would like to underline here that, in contrast, the inward pointing condition is neither needed, nor well adapted in the context of lower semicontinuous functions because it does not imply the uniqueness of solutions to the H-J-B equation, unless further regularity assumptions are imposed on the solutions.

To deal with discontinuous solutions, Ishii [START_REF] Ishii | Perron's method for monotone systems of second-order elliptic partial differential equations[END_REF] introduced the concept of lower and upper semicontinuous envelopes of a function, proving that the upper semicontinuous envelope of the value function of an optimal control problem is the largest upper semicontinuous subsolution and its lower semicontinuous envelope is the smallest lower semi-continuous supersolution. This approach, however, does not ensure the uniqueness of (weak) solutions of the H-J-B equation. On the other hand, the upper semicontinuous envelope does not have any meaning in optimal control theory while dealing with minimization problems (the lower semicontinuous envelope determines the value function of the relaxed problem). In [START_REF] Barron | Semicontinuous viscosity solutions for Hamilton-Jacobi equations with convex Hamiltonians[END_REF][START_REF] Barron | Optimal control and semicontinuous viscosity solutions[END_REF][START_REF] Frankowska | Lower semicontinuous solutions of Hamilton-Jacobi-Bellman equations[END_REF] a different concept of solutions was developed for the H-J-B equation associated to constraint-free Mayer optimal control problems, with a discontinuous cost. In this approach only subdifferentials are involved. In particular, in [START_REF] Frankowska | Lower semicontinuous solutions of Hamilton-Jacobi-Bellman equations[END_REF], results are expressed using the Fréchet subdifferentials instead of C 1 test functions. By [CEL84, Proposition 1.1], Fréchet subdifferentials of continuous INTRODUCTION functions coincide with those defined in [START_REF] Crandall | Viscosity solutions of Hamilton-Jacobi equations[END_REF] via C 1 test functions. While investigating, in [START_REF] Frankowska | Measurable viability theorems and the Hamilton-Jacobi-Bellman equation[END_REF], the merely measurable case, it became clear that, in order to get uniqueness, it is convenient to replace subdifferentials by normals to the epigraph of solutions. Such 'geometric' definition of solution avoids using test functions and allows to have a unified approach to both the continuous and measurable case.

Definition. A function W : [0, ∞) × A → R ∪ {+∞} is called a weak (or viscosity) solution of H-J-B equation on (0, ∞)×A if there exists a set C ⊂ (0, ∞), with µ L (C) = 0, such that for all (t, x) ∈ dom W ∩ (((0, ∞)\C) × ∂A) -p t + H -q (t, x, -p x ) 0 ∀ (p t , p x , q) ∈ T epi W (t, x, W (t, x)) -, ( 11
)
and for all (t, x)

∈ dom W ∩ (((0, ∞)\C) × int A) -p t + H -q (t, x, -p x ) = 0 ∀ (p t , p x , q) ∈ T epi W (t, x, W (t, x)) -. ( 12
)
We recall the following definition of absolutely continuous set-valued maps.

Definition. A set-valued map P : I R d is locally absolutely continuous if it takes nonempty closed images and for any [S, T ] ⊂ I, every ε > 0, and any compact subset K ⊂ R d , there exists δ > 0 such that for any finite partition

S t 1 < τ 1 t 2 < τ 2 ... t m < τ m T of [S, T ], m i=1 (τ i -t i ) < δ =⇒ m i=1 max dP(t i ) (P (τ i ) ∩ K), dP(τ i ) (P (t i ) ∩ K) < ε,
where dE (E ) := inf {β > 0 : E ⊂ E + βB} for any E, E ⊂ R d (the infimum over an empty set is +∞, by convention).

Main result 3. We provide next an existence and uniqueness theorem for weak solutions (in the sense of above definition) of nonautonomous H-B-J equations with timemeasurable data. The novelty of our work consists of recovering such a result under a backward controllability assumption, in a class of lower semicontinuous functions vanishing at infinity. More precisely, we prove the existence and uniqueness of weak solutions of the following problem

     - ∂W ∂t + H (t, x, -∇ x W ) = 0 on (0, ∞) × A lim t→∞ sup y∈dom W (t,•) |W (t, y)| = 0,
under the following assumptions on f and L:

(h3) (a) for all x ∈ R n the mappings f (•, x, •) and L(

{(f (t, x, u), L(t, x, u) + r) : u ∈ U (t), r 0} is convex; (d) there exists q ∈ L loc such that sup u∈U (t) (|f (t, x, u)| + |L(t, x, u)|) q(t) for all x ∈ ∂A and for a.e. t 0; (e) lim sup t→∞ t -1 t 0 (c(s) + k(s)) ds < ∞.
We denote by (opc u ) the conditions (ipc u ) in which F (t, y) is replaced by -F (t, y), and by (B) the following requirements:

(B) dom V = ∅ and there exist T > 0 and

ψ ∈ L 1 ([T, ∞); R + ) such that for all (t 0 , x 0 ) ∈ dom V ∩([T, ∞)×R n ) and any feasible trajectory-control pair (x(•), u(•)) on I = [t 0 , ∞), with x(t 0 ) = x 0 , |L(t, x(t), u(t))| ψ(t) for a.e. t t 0 .
Theorem (Existence and uniqueness of weak solutions, [START_REF] Basco | Hamilton-Jacobi-Bellman equations with time-measurable data and infinite horizon[END_REF]). Assume (h3) and

(opc u ). Let W : [0, ∞) × A → R ∪ {+∞}
|W (t, y)| = 0. ( 13
)
Then the following statements are equivalent:

(i) W=V; (ii) W is a weak solution of H-J-B equation on (0, ∞) × A and t epi W (t, •) is locally absolutely continuous.
Moreover, if in addition (B) holds true, then V is the unique weak solution satisfying (13) with locally absolutely continuous t epi V (t, •).

Semiconcavity and sensitivity relations for an Eikonal equation

In Chapter 4 we address the affine dynamics case, that is,

f (x, u) = m i=1 u i f i (x),
where f 1 , ..., f m are smooth vector fields on R n and A is compact. We study the semiconcavity of the weak solution to the following eikonal equation with Dirichlet boundary

INTRODUCTION conditions      |F (x) * ∇W (x)| -1 = 0 on A c W = 0 on A,
where F (x) is the matrix which has f 1 (x), ..., f m (x) as column vectors. We assume that the vectors fields f 1 , ..., f m satisfy Hörmander's condition (cfr. assumptions (h4) below). It is known that the (unique) solution of the above problem is the minimum time function W (•) = τ A (•) to reach the target A, associated with the above affine dynamics and controls taking values in the closed unit ball (cfr. [START_REF] Bardi | Optimal control and viscosity solutions of Hamilton-Jacobi-Bellman equations[END_REF]). More precisely, consider the following time optimal control problem

minimize θ A (x(•), u(•))
over all trajectory-control pairs (x(•), u(•)) satisfying the following control system

           x (s) = m i=1 u i (s) f i (x(s)) for a.e. s 0 x(0) = x 0 u ∈ B m , ( 14
)
where x 0 ∈ R m , B m denotes the set of all Lebesgue measurable controls u : [0, ∞) → R m such that u(s) ∈ B(0, 1) for a.e. s 0, and

θ A (x(•), u(•)) := inf {s 0 : x x 0 ,u (s) ∈ A}
is the so-called transfer time (to A) along the trajectory x(•) starting from x 0 and associated with the control u(•). For any x 0 ∈ R n and any control u(•) we denote by x x 0 ,u (•) the solution of the Cauchy problem x (s) = f (x(s), u(s)) for a.e. s 0,

x(0) = x 0 . By convention θ A (x x 0 ,u (•), u(•)) = +∞ if x x 0 ,u (s) /
∈ A for all s 0. The set A is called the target set and the function τ

A (x 0 ) = inf {θ A (x x 0 ,u (•), u(•)) : u ∈ B m } is called minimum time function. It is known (cfr. [CS04, Chapter 8]) that τ A is locally Lipschitz continuous on the set R\A, where R = {x 0 ∈ R n : ∃ u ∈ B m , θ A (x x 0 ,u (•), u(•)) < ∞}, if
and only if A satisfies the inward pointing condition: there exists r > 0 such that for any x ∈ ∂A and any ν proximal unit vector to A at x we can find u ∈ R m satisfying f (x, u), ν < -r (cfr. [START_REF] Cannarsa | Semiconcave functions, Hamilton-Jacobi equations, and optimal control[END_REF][START_REF] Bardi | Optimal control and viscosity solutions of Hamilton-Jacobi-Bellman equations[END_REF]). In addition, if the target set fulfils the uniform inner ball property, i.e., there exists r > 0 such that for every x ∈ ∂A we can find y ∈ A satisfying x ∈ B(y, r) ⊂ A, then τ A (•) is locally semiconcave on R\A. Recovering the local semiconcavity property for the minimum time function, associated with the above problem, when the target set does not satisfy the uniform inner ball property, becomes quite challenging.

Main result 4.

It is known that the minimum time to reach a point is equal to the sub-Riemannian distance d SR from such a point associated with the distribution ∆ = span {f 1 , ..., f m } on the manifold M = R n (cfr. [START_REF] Bellaıche | Sub-Riemannian geometry[END_REF][START_REF] Jerison | Subelliptic, second order differential operators[END_REF][START_REF] Montgomery | A tour of subriemannian geometries, their geodesics and applications[END_REF]). Regularity properties of d SR were obtained for subanalytic structures (cfr. [START_REF] Agrachev | Compactness for sub-riemannian length-minimizers and subanalyticity, in: Control theory and its applications (Grado[END_REF][START_REF] Trélat | Some properties of the value function and its level sets for affine control systems with quadratic cost[END_REF]). In particular, if the Lie algebra generated by ∆ is regular everywhere, i.e., it satisfies Hörmander's condition (cfr. [START_REF] Hörmander | Hypoelliptic second order differential equations[END_REF] and Chapter 4), then for any x 0 there exists a dense subset S x 0 of R n such that for all y ∈ S x 0 the function d SR (x 0 , •) is Lipschitz continuous on a suitable open neighborhood of y (cfr. [Rif14, Chapter 2]). Furthermore, P. Cannarsa and L. Rifford showed in [START_REF] Cannarsa | Semiconcavity results for optimal control problems admitting no singular minimizing controls[END_REF] that the function d SR (x 0 , •) is locally semiconcave on R n \ {x 0 }, assuming that any geodesics associated with ∆ connecting x to x 0 = x is not singular in the sense of the definition below.

Definition. We say that a control u ∈ B m is singular at (t, x 0 ) ∈ [0, ∞) × R n if there exists an absolutely continuous arc p : [0, t] → R n \ {0} such that for a.e. s ∈ [0, t]      x x 0 ,u (s) = ∇ p h(x x 0 ,u (s), p(s), u(s)), -p (s) = ∇ x h(x x 0 ,u (s), p(s), u(s)) p(s), f i (x x 0 ,u (s)) = 0 ∀i = 1, ..., m, where h(x, p, u) = m i=1 u i p, f i (x) . A time-minimizing control u ∈ B m at x 0 ∈ R is said to be singular if it is singular at (τ A (x 0 ), x 0 ).
= C(K, A) such that d SR (y, •) is C-semiconcave on K for every y ∈ A.
After establishing semiconcavity, we address sensitivity relations and transversality conditions for the minimum time function associated with the affine control system above. Sensitivity relations for the minimum time function to reach a set with the inner ball property were already investigated in [CF06, CMN15, CN10]. We recover, for time optimal control problems, sensitivity relations for the co-state in terms of proximal supergradients (cfr. [START_REF] Vinter | Optimal Control[END_REF][START_REF] Cannarsa | Optimality conditions and regularity results for time optimal control problems with differential inclusions[END_REF]). This is done under the assumption that there are no singular geodesics associated with ∆ and the target set is merely compact.

We impose the following assumptions on f 1 , ..., f m :

(h4) (a) f 1 , ..., f m are smooth vector fields (C ∞ or C ω ) and they satisfy Hörmander's condition, i.e., span

{X i (x)} i 1 = R n for all x ∈ R n , where X 1 (x) = INTRODUCTION {f 1 (x), ..., f m (x)}, X i+1 (x) = X i (x) ∪ {[f, g](x) : f ∈ X 1 (x), g ∈ X i (x)} for all i ∈ N ([•,
•] denotes the Lie bracket); (b) f 1 , ..., f m have sub-linear growth, Lipschitz continuous differential, and f 1 (x), ..., f m (x) are linearly independent for all x ∈ R n .

Theorem (Semiconcavity and sensitivity relations, [BCF]). Assume (h4) and that there are no singular time-minimizing controls for τ A (•). Then the following holds true:

(i) τ A (•) is locally semiconcave on A c ; (ii) if x 0 ∈ A c
and ū is an optimal control for the minimum time function at x 0 , then the solution of the adjoint equation

-p (t) = d x f (x x 0 ,ū (t), ū(t)) * p(t) for a.e. t ∈ [0, τ A (x 0 )]
satisfies the sensitivity relation

-p(t) ∈ ∂ P τ A (x x 0 ,ū (t)) ∀t ∈ [0, τ A (x 0 )), ( 15
)
and the transversality condition

p(τ A (x 0 )) ∈ Lim sup t → τ A (x 0 )- N P A c t (x x 0 ,ū (t)), (16)
where A t := {y ∈ R n : τ A (y) t}.

Conclusions

In this thesis we undertake an analytical approach to infinite horizon optimal control problems subject to state constraints. In particular we derive necessary conditions and sensitivity relations for such control problems. The novelty of our work relies on allowing for unbounded constraint sets with nonsmooth boundary, assuming the dynamics and the Lagrangian merely measurable in time. Lipschitz continuity of the value function is recovered for cost functionals admitting a discount factor and allowing time dependent dynamics and Lagrangians. We show such property as a consequence of a uniform neighboring feasible trajectory result, provided the discount rate is sufficiently large. The existence and uniqueness of weak solutions of the nonautonomous Hamilton-Jacobi-Bellman equation on the domain (0, ∞) × A are investigated assuming the Hamiltonian to be measurable in time. Using tools of viability theory, we develop the analysis providing a notion of weak solution for which, under a suitable controllability assumption, existence and uniqueness theorems are valid in the class of lower semicontinuous functions vanishing at infinity. Finally, the study of an autonomous H-J-B equation on a bounded domain, with Dirichlet boundary conditions, is addressed. We recover the semiconcavity of its (unique) solution on compact subsets, extending INTRODUCTION 21 a known result for the point-to-point sub-Riemannian distance when the Hörmander condition holds true.

Thesis outline

The dissertation is composed of 4 chapters. In Chapter 1, we focus on infinite horizon control problems under state constraints stating the maximum principle, sensitivity relations, and transversality conditions for the co-state. We also prove the uniform Lipschitz continuity of a large class of value functions when the constraint set is compact.

Introduction

Consider the infinite horizon optimal control problem B ∞ minimize

∞ t 0 L(t, x(t), u(t)) dt (1.1)
over all the trajectory-control pairs subject to the state constrained control system

                   x (t) = f (t, x(t), u(t)) a.e. t ∈ [t 0 , ∞) x(t 0 ) = x 0 u(t) ∈ U (t) a.e. t ∈ [t 0 , ∞) x(t) ∈ A t ∈ [t 0 , ∞) (1.2) where f : [0, ∞) × R n × R m → R n and L : [0, ∞) × R n × R m → R are given, A is a nonempty closed subset of R n , U : [0, ∞) ⇒ R m is
V (t 0 , x 0 ) = +∞). The function V : [0, ∞) × A → R ∪ {±∞} is called the value function of problem B ∞ .
Infinite horizon problems have a very natural application in mathematical economics (see, for instance, the Ramsey model in [START_REF] Ramsey | A mathematical theory of saving[END_REF]). In this case the planner seeks to find a solution to B ∞ (dealing with a maximization problem instead of a minimization one) with

L(t, x, u) = e -λt l(ug(x)) & f (t, x, u) = f (x) -ug(x)
where l(•) is called the "utility" function, f (•) the "production" function, and g(•)

the "consumption" function, while the variable x stands for the "capital" (in many applications one takes as constraint set

A = [0, ∞) with U (•) ≡ [-1, 1]
). The approach used by many authors to address this problem is to find necessary conditions of the first or second order (cfr. [START_REF] Arrow | Optimal growth with irreversible investment in a Ramsey model[END_REF], [START_REF] Bénassy | Macroeconomic theory[END_REF], [START_REF] Blanchard | Lectures on macroeconomics[END_REF], [START_REF] Sorger | On the long-run distribution of capital in the Ramsey model[END_REF]).

It happens quite often, in mathematical economics papers, that one considers as candidates for optimal solutions only trajectories satisfying simultaneously the unconstrained Pontryagin maximum principle and the state constraints. Such an approach, however, is incorrect as there are cases (see, e.g., Example 1.5.6) where no optimal trajectory exists in this class. There is, therefore, the need of a constrained maximum principle for infinite horizon problems with sufficiently general structure.

The literature dealing with necessary optimality conditions for unconstrained infinite horizon optimal control problems is quite rich (see, e.g., [START_REF] Aseev | Maximum principle for infinite-horizon optimal control problems under weak regularity assumptions[END_REF] and the reference therein), mostly under assumptions on f and L that guarantee the Lipschitz regularity of V (•, •). On the contrary, recovering optimality conditions in the presence of state constraints appears quite a challenging issue for infinite horizon problems, despite all the available results for constrained Bolza problems with finite horizon (cfr. [START_REF] Vinter | Optimal Control[END_REF]).

As a matter of fact, necessary conditions in the form of the maximum principle and partial sensitivity relations have been obtained for infinite horizon convex problems under smooth functional constraints such as h(t, x(t)) ≥ 0 (see, e.g., [START_REF] Seierstad | Necessary conditions for nonsmooth, infinite-horizon, optimal control problems[END_REF]). In this theses we prefer to deal with the constraint h(t, x(t)) ≤ 0 (without loosing the 1.1. INTRODUCTION 25 generality). For instance, suppose (x, ū) is optimal at (t 0 , x 0 ) for the problem

                         chapter 1 maximize ∞ t 0 L(t, x(t), u(t)) dt x (t) = f (t, x(t), u(t)) a.e. t ∈ [t 0 , ∞) x(t 0 ) = x 0 u(t) ∈ U a.e. t ∈ [t 0 , ∞) h(t, x(t)) ≤ 0 t ∈ [t 0 , ∞),
with U a closed convex subset of R m , h ∈ C 2 , f and L continuous together with their partial derivatives with respect to x and u, and assume the inward pointing condition

inf u∈U ∇ x h(t, x(t)), f (t, x(t), u) -f (t, x(t), ū(t)) < 0 ∀t t 0 .
If h(t 0 , x 0 ) < 0, then one proves that there exist q 0 ∈ {0, 1}, a co-state q(•), and a nondecreasing function µ(•), constant on any interval where h(t, x(t)) < 0, such that (q 0 , q(t 0 )) = (0, 0), µ(t 0 ) = 0, and q(•) satisfies the adjoint equation

q(t) = q(t 0 ) - t t 0 ∇ x H(s, x(s), q(s), ū(s)) ds - [t 0 ,t] ∇ x h(s, x(s)) dµ(s)
and the maximum principle

H(t, x(t), q(t), ū(t)) = max u∈U H(t, x(t), q(t), u) a.e. t ∈ [t 0 , ∞),
where H(t, x, p, u) := p, f (t, x, u) + q 0 L(t, x, u). Furthermore in [START_REF] Benveniste | Duality theory for dynamic optimization models of economics: the continuous time case[END_REF], using the language of the calculus of variations, the authors show that, under some very restrictive assumptions on f , if A is convex and int A = ∅ then, for any optimal trajectory x(•) of problem B ∞ , there exists an absolutely continuous arc q(•) which satisfies the adjoint equation and the partial sensitivity relation q(t) ∈ ∂ x V (t, x(t)) for all t ∈ [t 0 , ∞).

In the present work, for the first time we provide the normal maximum principle (i.e. q 0 = 1) together with partial and full sensitivity relations and a transversality condition at the initial time, under mild assumption on dynamics and constraints. To describe our results, assume for the sake of simplicity that

L(t, x, u) = e -λt l(x, u) is smooth, U (•) ≡ U is a closed subset of R m , V (t,
•) is continuously differentiable, and denote by N A (y) the limiting normal cone to A at y. Furthermore, the transversality condition and sensitivity relation in (1.3) lead to a significant economic interpretation (see [START_REF] Aseev | On some properties of the adjoint variable in the relations of the Pontryagin maximum principle for optimal economic growth problems[END_REF], [START_REF] Seierstad | Optimal control theory with economic applications[END_REF]): the co-state p + η can be regarded as the "shadow price" or "marginal price" , i.e., (1.3) describes the contribution to the value function (the optimal total utility) of a unit increase of capital x.

If (x, ū) is optimal for B ∞ at (t 0 , x 0 ) ∈ [0, ∞) × int A,
From the technical point of view, this theses relies on two main ideas. The first one consists in reformulating the infinite horizon problem as a Bolza problem on each finite time interval, which can be analyzed in detail by appealing to the existing theory for finite horizon problems. More precisely, fixing any T > 0, we have that

V (s, y) = inf V (T, x(T )) + T s L(t, x(t), u(t)) dt ∀(s, y) ∈ [0, T ] × A,
where the infimum is taken over all the feasible trajectory-control pairs (x, u) satisfying (1.2) with initial datum (s, y) (Lemma 1.4.2). Hence, problem B ∞ becomes a Bolza problem on [0, T ] with the additional final cost φ T (•) = V (T, •). Then, assuming the local Lipschitz regularity of V (T, •), we derive uniform bounds for the truncated costates (Lemma 1.3.6) which in turn allow to pass to the limit as t → ∞ in the necessary conditions (Theorem 1.4.3). The second key point is Therem 1.5.1 which provides structural assumptions on the data for V to be Lipschitz. A typical dynamic programming argument is used to obtain such a property for certain classes of Lagrangians, which include problems with a sufficiently large discount factor or a periodic dependence on time.

The outline of this chapter is as follows. In Section 2, we provide basic definitions, terminology, and facts from nonsmooth analysis. In Section 3, we give a bound on the total variation of measures associated to Mayer problems under state constraints. In Section 4, we focus on the main result, investigating problem B ∞ and stating sensitivity relations and transversality condition for the co-state. Finally, in the last Section, we prove the uniform Lipschitz continuity of a large class of value functions when A is 1.2. PRELIMINARIES ON NONSMOOTH ANALYSIS 27 compact.

Preliminaries on nonsmooth analysis

S(y)

:= {ξ ∈ R n : ∀x i -→ D x, ∃ξ i → ξ s.t. ξ i ∈ S(x i ) for all i}, Lim sup y→x D S(y) := {ξ ∈ R n : ∃x i -→ D x, ∃ξ i → ξ s.t. ξ i ∈ S(x i ) for all i}
are called, respectively, the lower and upper limits in the Kuratowski sense. Observe that these upper and lower limits are closed, possibly empty, and verify Lim inf

S(y).

We denote by W 1,1 (a, b; R n ) the space of all absolutely continuous R n -valued func-

tions u : [a, b] → R n endowed with the norm u W 1,1 (a,b) = |u(a)| + b a |u (t)| dt. Let u : [a, ∞) → R n , we write u ∈ W 1,1 loc (a, ∞; R n ) if u| [a,b] ∈ W 1,1 (a, b; R n ) for all b > a.
Let I be a compact interval in R. We denote by C(I; R n ) the set of all continuous R n -valued functions endowed with the uniform norm u ∞,I = sup {|u(t)| : t ∈ I}.

Let G : [a, b] × R n ⇒ R n be a multifunction taking nonempty values. We say that G(•, x) is absolutely continuous from the left, uniformly on R ⊂ R n , if for any ε > 0 there exists δ > 0 such that for any finite partition a t

1 < τ 1 t 2 < τ 2 ... t m < τ m b of [a, b] satisfying m 1 (τ i -t i ) < δ and for any x ∈ R we have m 1 d G(τ i ,x) (G(t i , x)) < ε, where for any E, E ⊂ R n d E (E ) := inf {β > 0 : E ⊂ E + βB} .
Take a closed set E ⊂ R n and x ∈ E. The regular normal cone NE (x) to E at x and the limiting normal cone N E (x) to E at x are defined, respectively, by

NE (x) :=    p ∈ R n : lim sup y→x E p, y -x |y -x| 0    N E (x) := Lim sup y→x E

NE (y).

1 we write y i -→

E

x for y i → x and y i ∈ E for any i.

THE VALUE FUNCTION

We denote by T C E (x) := (N E (x)) -the Clarke tangent cone to E at x, where " -" stands for the negative polar of a set. It is well known that co N E (x) = N C E (x) where N C E (x) := (T C E (x)) -denotes the Clarke normal cone to E at x (cfr. [RW98, Chapter 6]). Take an extended-valued function f : R n → R ∪ {+∞} and define the effective domain of f by dom f := {x ∈ R n : f (x) < +∞}. We denote by epi f and hypo f the epigraph and hypograph of f respectively. The subdifferential, the limiting subdifferential and the limiting superdifferential of an extended real function f at x ∈ dom f are defined respectively by ∂f (x

) := ξ ∈ R n : (ξ, -1) ∈ Nepi f (x, f (x)) ∂f (x) := {ξ ∈ R n : (ξ, -1) ∈ N epi f (x, f (x))} ∂ + f (x) := {ξ ∈ R n : (-ξ, 1) ∈ N hypo f (x, f (x))} .
If f is Lipschitz continuous on a neighborhood of x ∈ dom f , then ∂f (x) and ∂ + f (x) are nonempty. It is well known that ∂f (x) = ∅ on a dense subset of dom f , whenever f is lower semicontinuous.

The value function

Let τ > 0 and g τ : R n → R be a locally Lipschitz continuous function. Consider the problem M (g τ , τ ) on [0, τ ] minimize g τ (x(τ )) (1.4) over all the trajectories of the following differential inclusion under state constraints

                   x (t) ∈ F (t, x(t)) a.e. t ∈ [t 0 , τ ] x ∈ W 1,1 (t 0 , τ ; R n ) x(t 0 ) = x 0 x(t) ∈ Ω t ∈ [t 0 , τ ] (1.5)
with the initial datum (t 0 , x 0 ) ∈ [0, τ ]×Ω, where F : [0, ∞)×R n ⇒ R n is a multifunction and Ω ⊂ R n a nonempty closed set. Every trajectory x(•) that satisfies the state constrained differential inclusion (1.5) is called feasible. The infimum of the cost in (1.4) over all feasible trajectories, with the initial datum (t 0 , x 0 ), is denoted by V τ (t 0 , x 0 ) (if no feasible trajectory does exist, we define V τ (t 0 , x 0 ) = +∞). The function

V τ : [0, τ ] × Ω → R ∪ {±∞} 1.3. THE VALUE FUNCTION 29
is called the value function of problem M (g τ , τ ). We say that x(•) is a minimizer for problem M (g τ , τ ) at (t 0 , x 0 ) if x is feasible, x(t 0 ) = x 0 and V (t 0 , x 0 ) = g τ (x(τ )).

We start with the main assumptions on F (•, •) and Ω. Hypothesis (H1):

• F (•, •) takes closed nonempty values and F (•, x) is Lebesgue measurable for any x ∈ R n ;

• there exists k ∈ L ∞ ([0, ∞); R + ) such that F (t, x) ⊂ k(t)(1 + |x|)B for any x ∈ R n , a.e. t ∈ [0, ∞); • for all R 0 there exists γ R ∈ L 1 loc ([0, ∞); R + ) such that F (t, x) ⊂ F (t, x ) + γ R (t) |x -x | B for any x, x ∈ B(0, R), a.e. t ∈ [0, ∞); • (Relaxed Inward Pointing Condition-IPC ) For any (t, x) ∈ [0, ∞) × ∂Ω there exists a set Ω t,x ⊂ [0, ∞) with null measure such that for any v ∈ R n satisfying v ∈ Lim sup (s,y)→(t,x) s / ∈Ωt,x F (s, y) and max n∈N Ω (x)∩S n-1 n, v 0 we can find w ∈ R n such that w ∈ Lim inf (s,y)→(t,x) s / ∈Ωt,x co F (s, y) and max n∈N Ω (x)∩S n-1 n, w -v < 0.
Let us denote by (H2) the hypothesis as in (H1) under an additional assumption • For any R > 0 there exists r > 0 such that F (•, x) is absolutely continuous from the left, uniformly over x ∈ (∂Ω + rB) ∩ B(0, R), and with the Relaxed Inward Pointing Condition (IPC ) replaced by

• (Relaxed Inward Pointing Condition-IPC) For any (t, x) ∈ [0, ∞) × ∂Ω Lim inf (t ,x )→(t,x) (t ,x )∈[0,∞)×Ω co F (t , x ) int T C Ω (x) = ∅. Remark 1.3.1. We note that, if F is continuous, then the IPC condition reduces to co F (t, x) int T C Ω (x) = ∅ ∀(t, x) ∈ [0, ∞) × ∂Ω. (1.6)
Define the Hamiltonian

H(t, x, p) = max v∈F (t,x) p, v ∀ (t, x, p) ∈ R + × R n × R n .
Then, by the separation theorem, (1.6) is equivalent to

H(t, x, -p) > 0 ∀ 0 = p ∈ N C Ω (x
q(t) = p(t) + η(t) with η(t) =      [t 0 ,t] ν(s) dµ(s) t ∈ (t 0 , τ ] 0 t = t 0 ,
the following holds true:

(i) ν(t) ∈ coN Ω (x(t)) ∩ B µ -a.e. t ∈ [t 0 , τ ]; (ii) p (t) ∈ co r : (r, q(t)) ∈ N GrF (t,•) (x(t), x (t)) for a.e. t ∈ [t 0 , τ ]; (iii) -q(τ ) ∈ ∂g(x(τ )), -q(t 0 ) ∈ ∂ + x V τ (t 0 , x(t 0 )); (iv) q(t), x (t) = max { q(t), v : v ∈ F (t, x(t))} for a.e. t ∈ [t 0 , τ ]; (v) -q(t) ∈ ∂ 0 x V τ (t, x(t)) for a.e. t ∈ (t 0 , τ ]; (vi) (H(t, x(t), q(t)), -q(t)) ∈ ∂ 0 V τ (t, x(t)) for a.e. t ∈ (t 0 , τ ],
where

∂ 0 x V τ (t, x) := Lim sup x →x int Ω co ∂ x V τ (t, x ), ∂ 0 V τ (t, x) := Lim sup (t ,x )→(t,x) [t 0 ,τ ]×int Ω co ∂V τ (t, x ).
Remark 1.3.3. We would like to acknowledge here that the proof of the above result in [START_REF] Bettiol | Improved sensitivity relations in state constrained optimal control[END_REF] contains an erroneous claim which however does not have any impact neither on the rest of the proof nor on the final result. Namely on p. 373 the correct expression is ∂h > (x(t), f (t)) = (0, 0, 0, 0, 0, 0, -1) whenever x(t) ∈ int A and so the claim (27) is not correct. This does not influence however the rest of the arguments of the proof.

Theorem 1.3.4 ([BFV15]). The conclusion of Theorem 1.3.2 is also valid if we assume (H2) instead of (H1).

Definition 1.3.5. A family G of R-valued functions defined on E ⊂ R k is uniformly locally Lipschitz continuous on E if for all R 0 there exists L R 0 such that

|ϕ(z) -ϕ(z)| L R |z -z| for all z, z ∈ E ∩ B(0, R) and ϕ ∈ G .
Lemma 1.3.6. Assume (H1) or (H2). For all j ∈ N let g j : R n → R be a locally Lipschitz continuous function. Fix (t 0 , x 0 ) ∈ [0, ∞) × int Ω, T > t 0 and consider the problems M (g j , j). Assume also that {V j (•, •)} j T are uniformly locally Lipschitz continuous on [0, T ] × Ω. Let x ∈ W 1,1 loc (t 0 , ∞; Ω) be such that for any j T the restriction x| [t 0 ,j] (•) is a minimizer for problem M (g j , j) with initial datum (t 0 , x 0 ). Let, for every j ∈ N + , p j , q j , ν j , and µ j be as in the conclusion of Theorem 1.3.2 for the problem M (g j , j). Then (i) {p j } j T and {q j } j T are uniformly bounded on [t 0 , T ];

(ii) the total variation of the measures

{μ j } j T on [t 0 , T ] is uniformly bounded, where μj is defined by μ(dt) = |ν j (t)| µ j (dt).
The proof of the above lemma relies on the following proposition, which can be in turn justified following the same reasoning as in the proof of [CF05, Lemma 4.1].

Proposition 1.3.7. Let I ⊂ R be an interval and G : I ⇒ R n be a lower semicontinuous set valued map such that G(t) is a closed convex cone and int G(t) = ∅ for all t ∈ I. Then for every ε > 0 there exists a continuous function f :

I → R n such that for all t ∈ {s ∈ I : G(s) = R n } sup n∈G(t) -∩S n-1 n, f (t) -ε.
Proof of the Lemma 1.3.6. Since x(•) is continuous, hence locally bounded, by the uniform local Lipschitz continuity of {V j } j we deduce that

sup |ξ| : ξ ∈ t∈[t 0 ,T ] ∂ 0 x V j (t, x(t)) ∪ ∂ + x V j (t 0 , x(t 0 )), j T < ∞. (1.7) By Theorem 1.3.2-(iii), (v) we know that -q j (T ) ∈ ∂g j (x(T )), -q j (t 0 ) = -p j (t 0 ) ∈ ∂ + x V j (t 0 , x(t 0 )) and -q j (t) ∈ ∂ 0 x V j (t, x(t)) a.e. t ∈ (t 0 , T ]
for all j T . Since q j are right continuous on (t 0 , T ), from (1.7), it follows that , T ] and all j T . Hence, in view of (1.8),

q j ∞,[t 0 ,T ] j T is bounded. (1.
p j ∞,[t 0 ,T ] j T is bounded.
(1.9) So, the conclusion (i) follows. Also, since q j (t) = p j (t) + η j (t), from (1.8) and (1.9) we deduce that

η j ∞,[t 0 ,T ] j T is bounded. (1.10) Now let Γ := {s ∈ [t 0 , T ] : x(s) ∈ ∂Ω}. From the Relaxed Inward Pointing Condition, it follows that int T C Ω (x(t)) is nonempty for all t ∈ Γ and so int T C Ω (x(t)) is nonempty for all t ∈ [t 0 , T ]. Furthermore, this implies that the set valued map t T C Ω (x(t)) is lower semicontinuous on [t 0 , T ]. Since Γ = s ∈ [t 0 , T ] : T C Ω (x(s)) = R n , we can apply Proposition 1.3.7 with ε = 2 to conclude that there exists a continuous function f : [t 0 , T ] → R n such that sup n∈N C Ω (x(t))∩S n-1 f (t), n -2 ∀t ∈ Γ. (1.11)
We remark that the function f does not depend on j but only on x(•) and

T . Now, consider f ∈ C ∞ ([t 0 , T ]; R n ) such that f -f ∞,[t 0 ,T ]
1. We obtain from (1.11)

sup n∈N C Ω (x(t))∩S n-1 f (t), n -1.
(1.12)

Then, from (1.12) we deduce that for all j T ,

[t 0 ,T ] f (s), ν j (s) dµ j (s) = [t 0 ,T ]∩{s : ν j (s) =0} f (s), ν j (s) dµ j (s) = [t 0 ,T ]∩{s : ν j (s) =0} f (s), ν j (s) |ν j (s)| |ν j (s)| dµ j (s) - [t 0 ,T ]∩{s : ν j (s) =0} |ν j (s)| dµ j (s) = - [t 0 ,T ] |ν j (s)| dµ j (s).
So,

[t 0 ,T ] |ν j (s)| dµ j (s) [t 0 ,T ] -f (s), ν j (s) dµ j (s).
(1.13) Furthermore, from (1.10), integrating by parts, we obtain that, for some constant C 0 1.4. THE INFINITE HORIZON OPTIMAL CONTROL PROBLEM 33 and all j T ,

[t 0 ,T ] -f (s), ν j (s) dµ j (s) = [t 0 ,T ] -f (s) dη j (s) = -η j (T ) f (T ) + [t 0 ,T ] η j (s) f (s) ds C f ∞ + (T -t 0 ) f ∞ .
(1.14)

Now, since f does not depend on j, from (1.13) and (1.14) we deduce (ii).

The infinite horizon optimal control problem

Consider the infinite horizon optimal control problem with state constraints

B ∞ as in (1.1)-(1.2). We define the Hamiltonian function on [0, ∞) × R n × R n by H(t, x, p) = sup { p, f (t, x, u) -L(t, x, u) : u ∈ U (t)} .
Let us denote by (h) the following assumptions:

• there exist two locally essentially bounded functions b, θ : R + → R + and a nondecreasing function Ψ : R + → R + such that for a.e. t ∈ R + and for all

x ∈ R n , u ∈ U (t) |f (t, x, u)| b(t) (1 + |x|) ,

|L(t, x, u)| θ(t)Ψ(|x|);

• for any R > 0 there exist two locally integrable functions c R , α R , : R + → R + such that for a.e. t ∈ R + and for all x, y ∈ B(0, R), u ∈ U (t),

|f (t, x, u) -f (t, y, u)| c R (t)|x -y|, |L(t, x, u) -L(t, y, u)| α R (t) |x -y| ; • for all x ∈ R n the mappings f (•, x, •), L(•, x, •) are Lebesgue-Borel measurable;
• For a.e. t ∈ R + , and for all x ∈ R n the set

{(f (t, x, u), L(t, x, u)) : u ∈ U (t)}
is closed;

• the Relaxed Inward Pointing Condition-IPC is satisfied;

• for all (t 0 , x 0 ) ∈ [0, ∞) × A the limit lim T →∞ T t 0 L(t, x(t), u(t)) dt exists for all trajectory-control pairs (x, u) satisfying (1.2) with initial datum (t 0 , x 0 ); 34

1.4. THE INFINITE HORIZON OPTIMAL CONTROL PROBLEM • V (t 0 , x 0 ) = -∞ for all (t 0 , x 0 ) ∈ [0, ∞) × A.
Remark 1.4.1. A sufficient condition to guarantee that the last two hypothesis in (h) are satisfied is to assume that L takes nonnegative values. Alternatively, we may assume that for any initial datum (t 0 , x 0 ) there exists a function

φ t 0 ,x 0 ∈ L 1 (0, ∞) such that L(t, x(t), u(t)) φ t 0 ,x 0 (t) a.e. t ∈ [t 0 , ∞) for all trajectory-control pairs (x, u) satisfying (1.2).
The above hypotheses guarantee the existence and uniqueness of the solution to the differential equation in (1.2) for every initial datum x 0 and every control. So, denoting by x x 0 ,u 0 (•) such solution starting from x 0 at time t 0 , associated with a control u 0 (•), by Gronwall's lemma and our growth assumptions

|x x 0 ,u 0 (t)| |x 0 | + (t -t 0 ) b ∞,[t 0 ,t] e (t-t 0 ) b ∞,[t 0 ,t] ∀t t 0 . (1.15)
In particular, feasible trajectories are uniformly bounded on every compact time interval. Moreover, setting

M t 0 ,R (t) = R + (t -t 0 ) b ∞,[t 0 ,t] e (t-t 0 ) b ∞,[t 0 ,t] ,
by (1.15), Gronwall's lemma, and our assumptions we have that for all R, t > 0, all t 0 ∈ [0, t], and all x 0 , x 1 ∈ B(0, R)

|x x 1 ,u (s) -x x 0 ,u (s)| |x 1 -x 0 | e s t 0 c M t 0 ,R (t) (ξ) dξ ∀s ∈ [t 0 , t]. (1.16) Define the extended value function V : [0, ∞) × A → R ∪ {+∞} of problem B ∞ by V (t 0 , x 0 ) := inf ∞ t 0 L(t, x(t), u(t)) dt,
where the infimum is taken over all trajectory-control pairs (x, u) that satisfy (1.2) with the initial datum (t

0 , x 0 ) ∈ [0, ∞) × A.
We denote by dom V the set

{(t 0 , x 0 ) ∈ [0, ∞) × A : V (t 0 , x 0 ) < +∞}, and we say that a pair (x, ū) is optimal for B ∞ at (t 0 , x 0 ) ∈ dom V if ∞ t 0 L(t, x(t), ū(t)) dt ∞ t 0 L(t, x(t), u(t)) dt
for any feasible trajectory-control pair (x, u) starting from x 0 at time t 0 .

Lemma 1.4.2. Let T 0 and assume (h). Consider the Bolza problem B

T minimize V (T, x(T )) + T t 0 L(t, x(t), u(t)) dt
                   chapter 1 x (t) = f (t, x(t), u(t)) a.e. t ∈ [t 0 , T ] x(t 0 ) = x 0 u(t) ∈ U (t) a.e. t ∈ [t 0 , T ] x(t) ∈ A t ∈ [t 0 , T ].
Denote by

V B T : [0, T ] × A → R ∪ {+∞} the value function of the above problem. Then V B T (•, •) = V (•, •) on [0, T ] × A.
(1.17)

Furthermore, if (x, ū) is optimal at (t 0 , x 0 ) ∈ [0, T ] × A for B ∞ , then the restriction of (x, ū) to the time interval [t 0 , T ] is optimal for the Bolza problem B T too. Proof. Let (t 0 , x 0 ) ∈ [0, T ] × A and ε > 0. If V (t 0 , x 0 ) = +∞, then V (t 0 , x 0 ) V B T (t 0 , x 0 ). Otherwise, there exists a feasible trajectory-control pair (x ε , u ε ) for problem B ∞ at (t 0 , x 0 ) such that V (t 0 , x 0 ) T t 0 L(s, x ε (s), u ε (s)) ds + ∞ T L(s, x ε (s), u ε (s)) ds -ε T t 0 L(s, x ε (s), u ε (s)) ds + V (T, x ε (T )) -ε V B T (t 0 , x 0 ) -ε.
(1.18)

Since ε is arbitrary, we obtain V (t 0 , x 0 ) V B T (t 0 , x 0 ). On the other hand, if V B T (t 0 , x 0 ) = +∞, then V B T (t 0 , x 0 ) V (t 0 , x 0 ). Otherwise, there exists a feasible trajectory-control pair (x ε , ũε ) for problem B T at (t 0 , x 0 ) such that

V B T (t 0 , x 0 ) T t 0 L(s, xε (s), ũε (s)) ds + V (T, xε (T )) -ε.

By (1.15) and our assumptions on

L, T t 0 L(s, xε (s), ũε (s)) ds < ∞. Hence (T, xε (T )) ∈ dom V . So, there exists a feasible trajectory-control pair (x ε , ûε ) for problem B ∞ at (T, xε (T )) such that V B T (t 0 , x 0 ) T t 0 L(s, xε (s), ũε (s)) ds + ∞ T L(s, xε (s), ûε (s)) ds -2ε = ∞ t 0 L(s, x(s), u(s)) ds -2ε, (1.19)
where x(•) is the trajectory starting from x 0 at time t 0 satisfying the ordinary differential equation in (1.2) with the control u given by

u(s) :=      ũε (s) s ∈ [t 0 , T ] ûε (s) s ∈ (T, ∞). 1.4. THE INFINITE HORIZON OPTIMAL CONTROL PROBLEM Since u(•) ∈ U (•) and x([t 0 , ∞)) ⊂ A, (x, u) is feasible for problem B ∞ at (t 0 , x 0 ). Then, by (1.19), V B T (t 0 , x 0 ) V (t 0 , x 0 ) -2ε and, since ε is arbitrary, V B T (t 0 , x 0 ) V (t 0 , x 0 ).
The last part of the conclusion follows from (1.18), by setting ε = 0, (x ε , u ε ) = (x, ū), and using that

V B T (t 0 , x 0 ) = V (t 0 , x 0 ). Theorem 1.4.3. Assume (h) and suppose that V (i, •) is locally Lipschitz continuous on A for all large i ∈ N. Then V is locally Lipschitz continuous on [0, ∞) × A. Moreover, if (x, ū) is optimal for B ∞ at (t 0 , x 0 ) ∈ [0, ∞) × int Ω, then there exist p ∈ W 1,1 loc (t 0 , ∞; R n ), a nonnegative Borel measure µ on [t 0 , ∞), and a Borel measurable function ν : [t 0 , ∞) → R n such that, setting q(t) = p(t) + η(t) with η(t) =      [t 0 ,t] ν(s) dµ(s) t ∈ (t 0 , ∞) 0 t = t 0 ,
the following holds true:

(i) ν(t) ∈ co N A (x(t)) ∩ B µ -a.e. t ∈ [t 0 , ∞); (ii) p (t) ∈ co r : (r, q(t), -1) ∈ N Gr F (t,•) (x(t), x (t), L(t, x(t), ū(t))) for a.e. t ∈ [t 0 , ∞) where F (t, x) = {(f (t, x, u), L(t, x, u)) : u ∈ U (t)}; (iii) -p(t 0 ) ∈ ∂ + x V (t 0 , x(t 0 )), -q(t) ∈ ∂ 0 x V (t, x(t)) for a.e. t ∈ (t 0 , ∞); (iv) q(t), f (t, x(t), ū(t)) -L(t, x(t), ū(t)) = max u∈U (t) q(t), f (t, x(t), u) -L(t, x(t), u) for a.e. t ∈ [t 0 , ∞); (v) (H(t, x(t), q(t)), -q(t)) ∈ ∂ 0 V (t, x(t)) for a.e. t ∈ (t 0 , ∞). Remark 1.4.4. (a) Define G(t, x) = {(f (t, x, u), l(t, x, u)) : u ∈ U (t)
} and assume that for all R > 0 there exists r > 0 such that G(•, x) is absolutely continuous from the left uniformly over (∂A+rB)∩B(0, R). Then the conclusion of Theorem 1.4.3 holds if IPC is replaced by IPC;

(b) Theorem 1.4.3 implies a weaker hamiltonian inclusion

(-p (t), x (t)) ∈ co ∂ (x,p) H(t, x(t), q(t)) a.e. t ∈ [t 0 , ∞) (cfr. comment (e)-[BFV15, p. 362]); (c) If V (i, •
) is locally Lipschitz continuous on A for all large i, then, under assumptions of Theorem 1.4.3, V (t, •) is locally Lipschitz on A for every t ≥ 0;

(d) See Section 5 for the Lipschitz continuity of V (t, •) for the autonomous case and A compact. Also, sufficient conditions for the Lipschitz continuity of V (t, •) in the nonautonomous case for unbounded A are recently investigated in [BF].

Proof of Theorem 1.4.3. For any j ∈ N such that j t 0 consider the Bolza problem B j . We can rewrite the problem as a Mayer one on R n+1 : keeping the same notation as in Section 3, consider the Mayer problems M (g j , j) on R n+1 with

g j (ξ, z) := V (j, ξ) + z, F (t, x, z) := {(f (t, x, u), L(t, x, u)) : u ∈ U (t)} and Ω = A × R.
Denoting by V j the extended value function on [0, j] × Ω for problem M (g j , j) it follows, by standard arguments (cfr. [CS04, Chapter 7]), that

V j (t, x, z) = V B j (t, x) + z (1.20)
for all (t, x, z) ∈ [0, j]×A×R. Since, for all large j, V (j, •) is locally Lipschitz continuous on A, also g j is locally Lipschitz on A × R. For every j consider a locally Lipschitz function ḡj : R n+1 → R that coincides with g j on A × R. Note that replacing g j by ḡj does not change the value function of the Bolza problem B j . So, applying Theorem 1.3.2, it follows that V j is locally Lipschitz on [0, j] × A × R for all large j. Then V B j is locally Lipschitz on [0, j] × A and so, by Lemma 1.4.2, the value function V is locally Lipschitz on [0, j] × A. By the arbitrariness of j, V is locally Lipschitz continuous on [0, ∞) × A. Hence, if T > 0, from (1.20) and (1.17) it follows that V j 's are uniformly locally Lipschitz continuous on [0, T ] × A × R for all j T . Since the restriction of (x, ū) to [t 0 , j] is optimal for V B j at (t 0 , x 0 ), setting

z(t) = t t 0 L(s, x(s), ū(s)) ds,
we have that the restriction of ( X := (x, z), ū) to [t 0 , j] is optimal for V j at (t 0 , (x 0 , 0)) too. So, we may apply Theorem 1.3.2 with ḡj instead of g j on each time interval [t 0 , j]

with j ∈ N ∩ [t 0 , ∞).
Denoting by X the pair (x, z) in R n+1 , we obtain that there exist absolutely continuous arcs {P j } j and functions {Φ j } j of bounded variation defined on [t 0 , j], and nonnegative measures {µ j } j on [t 0 , j] such that {Φ j } j are continuous from the right on (t 0 , j) and

(a) Q j (t) = P j (t)+Φ j (t), where Φ j (t 0 ) = 0, Φ j (t) = [t 0 ,t] Π j (s) dµ j (s) for all t ∈ (t 0 , j] for some Borel measurable selections Π j (s) ∈ co N Ω ( X(s)) ∩ B µ j -a.e. s ∈ [t 0 , j]; (b) P j (t) ∈ co R : (R, Q j (t)) ∈ N Gr F (t,•) ( X(t), X (t)) for a.e. t ∈ [t 0 , j]; (c) -Q j (t 0 ) ∈ ∂ + X V j (t 0 , X(t 0 )); 1.4. THE INFINITE HORIZON OPTIMAL CONTROL PROBLEM (d) Q j (t), X (t) = max Q j (t), v : v ∈ F (t, X(t)) for a.e. t ∈ [t 0 , j]; (e) -Q j (t) ∈ ∂ 0 X V j (t, X(t)) for a.e. t ∈ (t 0 , j]; (f) ( H(t, X(t), Q j (t)), -Q j (t)) ∈ ∂ 0 V j (t, X(t)) for a.e. t ∈ (t 0 , j],
where H(t, X, P ) = max v∈ F (t,X) P, v .

Let P j (t) = (p j (t), p 0 j (t)), Q j (t) = (q j (t), q 0 j (t)), Φ j (t) = (η j (t), η 0 j (t)), and Π j (t) = (ν j (t), ν 0 j (t)).
Using the definition of limiting normal vectors as limits of strict normal vectors, relations (a)-(c), and the fact that

N Ω ( X(•)) = N A (x(•)) × {0} we obtain p j (t) ∈ co r : (r, q j (t), q 0 j (t)) ∈ N Gr F (t,•) (x(t), x (t), L(t, x(t), ū(t))) a.e. t ∈ [t 0 , j], (p 0 j ) ≡ 0, p 0 j (j) + η 0 j (j) = -1, ν 0 j ≡ 0.
Thus, on account of (d)-(f ), for a.e. t ∈ [t 0 , j] we derive the extended Euler-Lagrange condition

p j (t) ∈ co r : (r, q j (t), -1) ∈ N Gr F (t,•) (x(t), x (t), L(t, x(t), ū(t))) (1.21)
where q j (t) = p j (t) + η j (t), with

η j (t) =      [t 0 ,t] ν j (s) dµ j (s) t ∈ (t 0 , j] 0 t = t 0 , (1.22)
and ν j (t) ∈ co N A (x(t)) ∩ B µ j -a.e. on [t 0 , j], satisfy the maximum principle

q j (t), f (t, x(t), ū(t)) -L(t, x(t), ū(t)) = max u∈U (t) q j (t), f (t, x(t), u) -L(t, x(t), u) a.e. t ∈ [t 0 , j],
(1.23) the transversality condition in terms of limiting superdifferential

-p j (t 0 ) ∈ ∂ + x V (t 0 , x 0 ), (1.24)
and the sensitivity relations

-q j (t) ∈ ∂ 0 x V (t, x(t)) a.e. t ∈ (t 0 , j], (1.25) (H(t, x(t), q j (t)), -q j (t)) ∈ ∂ 0 V (t, x(t)) a.e. t ∈ (t 0 , j]. (1.26)
We extend the functions p j and η j to whole interval (j, ∞) as the constants p j (j) and η j (j), respectively. We denote again by p j and η j such extensions.

We divide the proof into three steps. Let k be an integer such that k > t 0 .

Step 1. Applying Lemma 1.3.6 to problems M (g j , j), we known that {p j } j k and {q j } j k are uniformly bounded on [t 0 , k]. Furthermore, for some ξ ∈ L 1 loc ([0, ∞); R + ) and a.e. t t 0 , we have p j (t) ξ(t) |q j (t)| for all j. So, by the Ascoli-Arzelà and Dunford-Pettis theorems we have, taking a subsequence and keeping the same notation, that there exists an absolutely continuous function

p k : [t 0 , k] → R n such that p j → p k uniformly on [t 0 , k] p j (p k ) in L 1 (t 0 , k).
Furthermore, from Lemma 1.3.6 again, we known that {η j } j k is uniformly bounded on [t 0 , k] and the total variation of such functions is uniformly bounded on [t 0 , k]. So, applying Helly's selection theorem, taking a subsequence and keeping the same notation, we deduce that there exists a function of bounded variation η k on [t 0 , k] such that η j → η k pointwise on [t 0 , k] (notice that since η j (t 0 ) = 0 for all j then η k (t 0 ) = 0). Furthermore, from Lemma 1.3.6-(ii) we deduce that there exists a nonnegative measure

µ k on [t 0 , k] such that, by further extraction of a subsequence, μj * µ k in C([t 0 , k]; R) * , where μj (dt) = |ν j (t)| µ j (dt). Let γ j (t) :=        ν j (t) |ν j (t)| ν j (t) = 0 0 otherwise.
Since γ j (t) ∈ coN A (x(t))∩B μj -a.e. t ∈ [t 0 , k] is a Borel measurable selection, applying [Vin00, Proposition 9.2.1], we deduce that, for a subsequence j i , there exists a Borel measurable function ν k such that

ν k (•) ∈ coN A (x(•)) ∩ B µ k -a.e. on [t 0 , k] and for all φ ∈ C([t 0 , k]; R n ) [t 0 ,k] φ(s), γ j i (s) dμ j i (s) → [t 0 ,k] φ(s), ν k (s) dµ k (s) as i → ∞. (1.27) Now since for t ∈ (t 0 , k] η j i (t) = [t 0 ,t] ν j i (s) dµ j i (s) = [t 0 ,t]∩{s : ν j i (s) =0} ν j i (s) dµ j i (s) = [t 0 ,t] γ j i (s) dμ j i (s), from (1.27) it follows that for all t ∈ (t 0 , k] η k (t) = [t 0 ,t] ν k (s) dµ k (s).
By Mazur's theorem, as in [AF09, Theorem 7.2.2], using the closedness of ∂ + x V (t 0 , x 0 ),

UNIFORM LIPSCHITZ CONTINUITY OF A CLASS OF VALUE FUNCTIONS

∂ 0 x V (t, x(t)) and convexity in (1.21), passing to the limit in (1.24), (1.21), and (1.23) on [t 0 , k], and in (1.25) and (1.26) on (t 0 , k], we obtain condition (iv) on [t 0 , k], inclusions (ii) on [t 0 , k], (iii) and (v) at t 0 and on (t 0 , k].

Step 2. Consider now the interval [t 0 , k + 1]. By the same argument as in the first step, taking suitable subsequences {p j i l } l ⊂ {p j i } i and {η j i l } l ⊂ {η j i } i , we deduce that there exist an absolutely continuous function p k+1 , a function of bounded variation η k+1 , and a nonnegative measure µ k+1 which satisfy condition (iv) on [t 0 , k + 1], inclusions (ii) on [t 0 , k + 1], (iii) and (v) at t 0 and on (t 0 , k + 1]. Moreover

p j i l → p k+1 uniformly on [t 0 , k + 1] p j i l (p k+1 ) in L 1 (t 0 , k + 1) p k+1 | [t 0 ,k] = p k ,
and for all t ∈ [t 0 , k + 1]

η j i l (t) → η k+1 (t) =      [t 0 ,t] ν k+1 (s) dµ k+1 (s) t ∈ (t 0 , k + 1] 0 t = t 0 , where ν k+1 (•) ∈ coN A (x(•)) ∩ B µ k+1 -a.e. on [t 0 , k + 1] is a Borel measurable selec- tion. Furthermore, since η k+1 | [t 0 ,k] = η k and µ k+1 | [t 0 ,k] = µ k , we have that ν k+1 | [t 0 ,k] = ν k µ k -a.e. on [t 0 , k].
We see that the functions p k+1 , η k+1 , and ν k+1 extend the functions p k , η k , and ν k respectively, and measure µ k+1 extends measure µ k .

Step 3. Repeating the argument of the second step for any interval [t 0 , k + s] with s ∈ N, we can extend p k , η k , ν k and µ k to the whole interval [t 0 , ∞), extracting every time a subsequence of the previously constructed subsequence. Finally, we conclude that there exists a locally absolutely continuous function p

: [t 0 , ∞) → R n , a function of locally bounded variation η : [t 0 , ∞) → R, a nonnegative measure µ on [t 0 , ∞), and a Borel measurable selection ν(t) ∈ co N A (x(t)) ∩ B µ -a.e. t ∈ [t 0 , ∞)
satisfying the conclusion of the therorem.

Uniform Lipschitz continuity of a class of value functions

We now investigate the uniform Lipschitz continuity of a class of value functions. In this section, we assume that f is time independent, i.e., f

(t, x, u) = f (x, u), U (•) ≡ U is closed, A is compact,
-p, f (x, u) > 0 ∀ 0 = p ∈ N C A (x) ∀ x ∈ ∂A.
Theorem 1.5.1. Assume that L(t, x, u) = e -λt l(x, u).

Then the function v(•) := V (0, •) is Lipschitz continuous on A for all large λ > 0.
Consequently, the value function V (t, x) of problem B ∞ , which is equal to e -λt v(x), is Lipschitz continuous on A uniformly in t 0 for all large λ > 0.

Proof. By our assumptions, dom v = A and v is bounded. For any x ∈ A let us denote by U x the set of all Lebesgue measurable functions u : [0, 1] → R m such that u(t) ∈ U a.e. t ≥ 0 and x x,u (s) ∈ A for all s ∈ [0, 1]. By the dynamic programming principle it follows that for any distinct x 1 , x 0 ∈ A there exists a control u 0 feasible at

x 0 for problem B ∞ , such that v(x 0 ) + |x 1 -x 0 | > 1 0 e -λs l(x x 0 ,u 0 (s), u 0 (s)) ds + e -λ v(x x 0 ,u 0 (1)).
Thus, applying again the dynamic programming principle, it follows that for any

u 1 ∈ U x 1 v(x 1 ) -v(x 0 ) |x 1 -x 0 | + 1 0 e -λs [l(x x 1 ,u 1 (s), u 1 (s)) -l(x x 0 ,u 0 (s), u 0 (s))] ds + e -λ |v(x x 1 ,u 1 (1)) -v(x x 0 ,u 0 (1))| |x 1 -x 0 | + 1 0 e -λs [l(x x 1 ,u 1 (s), u 1 (s)) -l(x x 1 ,u 0 (s), u 0 (s))] ds + 1 0 e -λs [l(x x 1 ,u 0 (s), u 0 (s)) -l(x x 0 ,u 0 (s), u 0 (s))] ds + e -λ |v(x x 1 ,u 1 (1)) -v(x x 0 ,u 0 (1))| .
(1.28) By (1.15), there exists a constant M 0 such that for all x ∈ A and all Lebesgue measurable u : [0, 1] → R m with u(t) ∈ U a.e., the trajectories x x,u (•) take values in B(0, M ) on the time interval [0, 1]. Let C > 0 be a Lipschitz constant for l on B(0, M ), with respect to the space variable. Then, by (1.16), there exists c > 1 such that for all

x 1 , x 0 ∈ A 1 0 e -λs |l(x x 1 ,u 0 (s), u 0 (s)) -l(x x 0 ,u 0 (s), u 0 (s))| ds C 1 0 e -λs |x x 1 ,u 0 (s) -x x 0 ,u 0 (s)| ds C • c |x 1 -x 0 | . (1.29) 1.5. UNIFORM LIPSCHITZ CONTINUITY OF A CLASS OF VALUE FUNCTIONS So, putting C = C • c + 1, from (1.28) it follows that v(x 1 ) -v(x 0 ) C |x 1 -x 0 | + 1 0 e -λs [l(x x 1 ,u 1 (s), u 1 (s)) -l(x x 1 ,u 0 (s), u 0 (s))] ds + e -λ |v(x x 1 ,u 1 (1)) -v(x x 0 ,u 0 (1))| .
(1.30)

Now we claim that there exist a constant β = β(f, l) 1 and a control u

1 ∈ U x 1 such that 1 0 e -λs [l(x x 1 ,u 1 (s), u 1 (s)) -l(x x 1 ,u 0 (s), u 0 (s))] ds β |x 1 -x 0 | , |x x 1 ,u 1 (1) -x x 0 ,u 0 (1)| β |x 1 -x 0 | . (1.31) Indeed, if max s∈[0,1] d A (x x 1 ,u 0 (s)) = 0 then u 0 ∈ U x 1 . So, (1.31) follows taking u 1 = u 0 . Otherwise, suppose max s∈[0,1] d A (x x 1 ,u 0 (s)) > 0 and consider the following control system in R n+1                          x (s) = f (x(s), u(s)) a.e. s ∈ [0, 1] z (s) = e -λs l(x(s), u(s)) a.e. s ∈ [0, 1] x(0) = x, z(0) = 0 u(•) is Lebesgue measurable u(s) ∈ U a.e. s ∈ [0, 1].
(1.32)

Let us denote by (X x,u , u) the trajectory-control pair that satisfies (1.32) where X x,u (•) := (x x,u (•), z 0,u (•)). Set Ω := A × R. By the neighbouring feasible trajectory theorem [FM13b, Theorem 3.3], there exists a constant β 1 (depending only on f and l) and a control u 1 ∈ U x 1 such that

X x 1 ,u 1 -X x 1 ,u 0 ∞,[0,1] β max s∈[0,1] d Ω (X x 1 ,u 0 (s)) . (1.33) Since d Ω (X x 1 ,u 0 (•)) = d A (x x 1 ,u 0 (•)) and x x 0 ,u 0 (•) ∈ A we have X x 1 ,u 1 -X x 1 ,u 0 ∞,[0,1] β max s∈[0,1] inf γ∈Ω |X x 1 ,u 0 (s) -γ| β max s∈[0,1] {|x x 1 ,u 0 (s) -x x 0 ,u 0 (s)|} β • c |x 1 -x 0 | . Furthermore 1 0 e -λs [l(x x 1 ,u 1 (s), u 1 (s)) -l(x x 1 ,u 0 (s), u 0 (s))] ds X x 1 ,u 1 -X x 1 ,u 0 ∞,[0,1]
1.5. UNIFORM LIPSCHITZ CONTINUITY OF A CLASS OF VALUE FUNCTIONS 43

|x x 1 ,u 1 (1) -x x 0 ,u 0 (1)| X x 1 ,u 1 -X x 1 ,u 0 ∞,[0,1] + c |x 1 -x 0 | .
So, replacing β with 2β • c, (1.31) follows. Now, let 0 r 1. Combining the inequalities in (1.30) and (1.31) we obtain that for all

x 1 , x 0 ∈ A with |x 1 -x 0 | r v(x 1 ) -v(x 0 ) (C + β) r + e -λ ω(βr)
where ω(r) := sup

|h-h | r h,h ∈A |v(h) -v(h )| .
By the symmetry of the previous inequality with respect to x 1 and x 0 we have that

|v(x 1 ) -v(x 0 )| (C + β) r + e -λ ω(βr).
(1.34)

Letting θ := e -λ and α := C + β, we deduce from (1.34) that for all 0 r 1 ω(r) αr + θ ω(βr).

(1.35) So, Lemma 1.5.2 below yields the Lipschitz continuity of v for λ > log β.

The last part of the conclusion follows observing that

V (t, •) = e -λt v(•).
The next lemma (proved in the Appendix) extends [LT94, Lemma 2.1].

Lemma 1.5.2. Let R > 0 and ω : [0, R] → [0, ∞) be a nondecreasing function. Suppose that there exists 0 < θ < 1, α > 0, β 1 such that

ω(r) αr + θω(βr) ∀ 0 r R/β. (1.36)
Let m 1 be a real number such that θ m β < 1. Then there exists a constant C 0 such that ω(r) Cr 1/m ∀ 0 r R.

Remark 1.5.3. (a) From Theorem 1.5.1 and Theorem (1

.4.3)-(iii), since V (t, •) = e -λt v(•), it follows that lim t→∞ q(t) = 0.
(b) From (1.35) and Lemma 1.5.2 it follows that, given any λ > 0, {V (t, •)} t 0 are uniformly Hölder continuous on A of exponent 1/m for all m ≥ 1 such that m > (log β)/λ, where β is as in the above proof.

Corollary 1.5.4. Assume that L(t, x, u) = e -λt l(t, x, u) and there exists T > 0 such that l is time independent for all t T . Then {V (t, •)} t 0 are uniformly Lipschitz continuous on A for all large λ > 0.

UNIFORM LIPSCHITZ CONTINUITY OF A CLASS OF VALUE FUNCTIONS

Corollary 1.5.5. Assume that L(t, x, u) = e -λt l(t, x, u) with the further assumption: l(•, x, u) is T -periodic, i.e. there exists T > 0 such that l(t + T, x, u) = l(t, x, u) for all t 0, x ∈ R n and u ∈ R m . Then {V (t, •)} t 0 are uniformly Lipschitz continuous on A for all large λ > 0.

Proof. Fix t ∈ [0, ∞). Then, by the dynamic programming principle, for any x, x 0 ∈ A there exists u 0 feasible for B ∞ at x 0 such that

V (t, x 1 ) -V (t, x 0 ) |x 1 -x 0 | + t+T t e -λs [l(s, x x 1 ,u 1 (s), u 1 (s)) -l(s, x x 0 ,u 0 (s), u 0 (s))] ds + |V (t + T, x x 1 ,u 1 (t + T )) -V (t + T, x x 0 ,u 0 (t + T ))|
for any u 1 feasible for B ∞ at x 1 . Now, the periodicity of l in the time variable implies that V (s + T, x) = e -λT V (s, x). From the previous inequality it follows that

V (t, x 1 ) -V (t, x 0 ) |x 1 -x 0 | + t+T t e -λs [l(s, x x 1 ,u 1 (s), u 1 (s)) -l(s, x x 0 ,u 0 (s), u 0 (s))] ds + e -λT |V (t, x x 1 ,u 1 (t + T )) -V (t, x x 0 ,u 0 (t + T ))| .
Proceeding as in the proof of Theorem 1.5.1, by the neighbouring feasible trajectory theorem [FM13b, Theorem 3.3] there exist two constants β 1 and C 0 (depending only on f, l, and T ) such that, for all |x 1 -x 0 | r 1, we have that

|V (t, x 1 ) -V (t, x 0 )| (C + β) r + e -λT sup |h-h | βr h,h ∈A |V (t, h) -V (t, h )|.
The conclusion follows applying Lemma 1.5.2 for λ > (log β)/T .

Example 1.5.6. In this example we will show the fallacy of applying the unconstrained Pontryagin maximum principle to B ∞ in order to obtain candidates for optimality that satisfy some given state constraints.

Consider the following infinite horizon optimal control problem:

maximize J(u) = ∞ 0 e -λt (x(t) + u(t)) dt
over all trajectory-control pairs (x, u) satisfying

                   x (t) = -au(t) a.e. t 0 x(0) = 1 u(t) ∈ [-1, 1] a.e. t 0 x(t) ∈ (-∞, 1] t 0,
(1.37)

UNIFORM LIPSCHITZ CONTINUITY OF A CLASS OF VALUE FUNCTIONS 45

with a > λ > 0.

Applying the Pontryagin maximum principle for unconstrained problems, it follows that any optimal trajectory-control pair satisfies one of the following three relations:

(i) x -(t) = 1 + at associated with u -(t) ≡ -1; (ii) x + (t) = 1 -at associated with u + (t) ≡ +1; (iii) x ± (t) = (1 -at)χ [0, t] (t) + (1 -a t + a(t -t))χ ( t,∞) (t) associated with u ± (t) = χ [0, t] (t) -χ ( t,∞) (t)
, for some t > 0.

Excluding now the trajectories x -and x ± , since they are not feasible, this analysis leads to the conclusion that x + is the only candidate for optimality. But one can easily see that the feasible trajectory x(t) ≡ 1, associated with the control u(t) ≡ 0, verifies

J(ū) > J(u + ).

Appendix

Proof of Lemma 1.5.2. Suppose first that m = 1. Let θ < τ < 1 be such that τ β 1.

Then τ R R β and by the growth assumption in (1.36) and the monotonicity of ω, we have that

ω(τ R) ατ R + θω(βτ R) ατ R + θω(R).
(1.38)

Applying again (1.36), the monotonicity of ω, and (1.38) we obtain

ω(τ 2 R) ατ 2 R + θω(τ R) ατ 2 R + θ [ατ R + θω(R)] = ατ R(τ + θ) + θ 2 ω(R).
So, by induction on k ∈ N it is straightforward to show that

ω(τ k R) ατ R(τ k-1 + θτ k-2 + ... + θ k-1 ) + θ k ω(R) = αRτ k   1 + θ τ + ... + θ τ k   + θ k ω(R) < αRτ k 1 1 -θ/τ + θ k ω(R) = αR τ -θ τ k+1 + θ k ω(R). 1.5. UNIFORM LIPSCHITZ CONTINUITY OF A CLASS OF VALUE FUNCTIONS Now let r ∈ [0, R]. Then there exists k ∈ N such that τ k+1 R < r τ k R. Finally ω(r) αR τ -θ τ k+1 + θ k ω(R) α τ -θ τ k+1 R + τ k+1 R ω(R) τ R α τ -θ + ω(R) τ R r.
The conclusion holds true with

C = α τ -θ + ω(R) τ R .
If m > 1, by the growth assumption in (1.36) and the monotonicity of ω we have that

ω(θ m R) αθ m R + θω(βθ m R) αθ m R + θω(R).
(1.39) Applying again (1.36), monotonicity, and (1.39) we obtain

ω(θ 2m R) αθ 2m R + θω(θ m R) αθ 2m R + θ [αθ m R + θω(R)] = αθ m+1 R(1 + θ m-1 ) + θ 2 ω(R).

So, by induction on

k ∈ N it is straightforward to show that ω(θ km R) αθ m+k-1 R(1 + θ m-1 + ... + θ (k-1)(m-1) ) + θ k ω(R) < αRθ m+k-1 1 1 -θ m-1 + θ k ω(R) = αRθ m-1 1 -θ m-1 + ω(R) θ k . Now let r ∈ [0, R]. Then there exists k ∈ N such that θ (k+1)m R < r θ km R. Thus, ω(r) ω(θ km R) Cθ k < C θ r R 1/m = C θR 1/m r 1/m where C = αRθ m-1 1 -θ m-1 + ω(R). The conclusion follows with C = C/θR 1/m .

CHAPTER 2 LIPSCHITZ CONTINUITY OF THE VALUE FUNCTION FOR THE INFINITE HORIZON OPTIMAL CONTROL PROBLEM UNDER STATE CONSTRAINTS

Abstract. In this chapter sufficient conditions for Lipschitz regularity of the value function for an infinite horizon optimal control problem subject to state constraints are investigated. We focus on problems with cost functional admitting a discount rate factor and allow time dependent dynamics and lagrangian. Furthermore, state constraints may be unbounded and may have a nonsmooth boundary. Lipschitz regularity is recovered as a consequence of estimates on the distance of a given trajectory of control system from the set of all its viable (feasible) trajectories, provided the discount rate is sufficiently large. These distance estimates are derived here under a uniform inward pointing condition on the state constraint and imply, in particular, that feasible trajectories depend on initial states in a Lipschitz way with an exponentially increasing in time Lipschitz constant. As an application we show that the value function of the original problem coincides with the value function of the relaxed infinite horizon problem.

Introduction

Infinite time horizon models arising in mathematical economics and engineering typically involve control systems with restrictions on both controls and states. For instance it is natural to request all the variables involved in an economic model to be non-2.1. INTRODUCTION negative. While dealing with control constraints is rather well understood, the major difficulties with state constraints arise whenever for small perturbations of the initial state (or of a feasible control) the corresponding trajectory violates the constraints as time goes. More generally, it may happen that the celebrated value function associated to an infinite horizon optimal control problem takes infinite values and is discontinuous. In particular, this prevents using such a classical tool of optimal control theory as Hamilton-Jacobi-Bellman equation and its viscosity solution. In the literature one finds some results concerning continuity of the value function for state constrained infinite horizon problems, see for instance [START_REF] Soner | Optimal control problems with state-space constraints I[END_REF]. However in this last reference the state constraints are given by a compact set with a smooth boundary. This clearly does not fit the state constraint described by the cone of positive vectors. In addition, results of [START_REF] Soner | Optimal control problems with state-space constraints I[END_REF] address only the autonomous case, which is also a serious restriction, because, as it was shown later on, arguments of its proof can not be extended to the non-autonomous case whenever the time dependence is merely continuous.

Because of their presence in various applied models, addressing non-autonomous control systems subject to unbounded and non smooth state constraints remains crucial. Let us note that (the finite horizon) state-constrained Mayer's and Bolza's problems have been successfully investigated by many authors, see for instance [START_REF] Bettiol | Improved sensitivity relations in state constrained optimal control[END_REF][START_REF] Frankowska | Discontinuous solutions of Hamilton-Jacobi-Bellman equation under state constraints[END_REF][START_REF] Vinter | Optimal Control[END_REF] and the references therein. However in the infinite horizon framework these results can not be used, because restricting optimal trajectories of the infinite horizon problem to a finite time interval, in general, does not lead to optimal trajectories of the corresponding finite horizon problem. See [START_REF] Cannarsa | Value function, relaxation, and transversality conditions in infinite horizon optimal control[END_REF] for a further discussion of this issue.

Infinite horizon problems exhibit many phenomena not arising in the finite horizon context and for this reason their study is still going on, even in the absence of state constraints, cfr. [AV12, AV14, CF18, CH87, Pic10]. This chapter deals with the infinite horizon optimal control problem B ∞ minimize

∞ t 0 e -λt l(t, x(t), u(t)) dt (2.1)
over all trajectory-control pairs (x(•), u(•)) of the state constrained control system

             x (t) = f (t, x(t), u(t)) a.e. t ∈ [t 0 , ∞) x(t 0 ) = x 0 u(t) ∈ U (t) a.e. t ∈ [t 0 , ∞) x(t) ∈ A ∀ t ∈ [t 0 , ∞) (2.2) where λ > 0, f : [0, ∞) × R n × R m → R n and l : [0, ∞) × R n × R m → R are given functions, U : [0, ∞) ⇒ R m
is a Lebesgue measurable set-valued map with closed nonempty images, A is a closed subset of R n , and (t 0 , x 0 ) ∈ [0, ∞) × A is the initial datum. Every trajectory-control pair (x(•), u(•)) that satisfies the state constrained control system (2.2) is called feasible. The infimum of the cost functional in (2.1) over all feasible trajectory-control pairs, with the initial datum (t 0 , x 0 ), is denoted by V (t 0 , x 0 ) (if no feasible trajectory-control pair exists at (t 0 , x 0 ) or if the integral in (2.1) is not defined for every feasible pair, we set V (t 0 , x 0 ) = +∞). The function

V : [0, ∞) × A → R ∪ {±∞} is called the value function of problem B ∞ .
Lipschitz continuity of V for a compact set of constraints A was recently investigated in [START_REF] Basco | Necessary conditions for infinite horizon optimal control problems with state constraints[END_REF] for autonomous control systems and lagrangian functions. It was used to get a maximum principle under state constraints and also to obtain sensitivity relations. However, in [START_REF] Basco | Necessary conditions for infinite horizon optimal control problems with state constraints[END_REF] the maximum principle was proved for the non-autonomous case and for possibly unbounded A under the assumption that V (t, •) is locally Lipschitz on A for every t ≥ 0. So the open question remained: how to guarantee the Lipschitz continuity of V (t, •) when the data are time dependent and without imposing the compactness of A. Then recovering Lipschitz continuity of the value function is not straightforward and calls for distinct arguments. Here we propose sufficient conditions (cfr. Section 3) for it, allowing both f and l to be time dependent and not requiring boundedness of A and smoothness of ∂A. Our proof differs substantially from the one in [START_REF] Basco | Necessary conditions for infinite horizon optimal control problems with state constraints[END_REF].

The outline of present chapter is as follows. In Section 2, we provide basic definitions, terminology, and facts from nonsmooth analysis. In Sections 3, we state a new neighboring feasible trajectory theorem under a uniform inward pointing condition. In Section 4, we give an example of uniform inward pointing condition for functional state constraints and in Section 5 we prove our main result on Lipschitz continuity of the value function. Section 6 is devoted to an application to the relaxation of our control problem. We denote by L loc the set of all functions f ∈ L 1 loc ( [0, ∞); R + ) such that lim σ→0 θ f (σ) = 0. Notice that L ∞ ( [0, ∞); R + ) ⊂ L loc and, for any f ∈ L loc , θ f (σ) < ∞ for every σ > 0.

Preliminaries

A set-valued map F : R n ⇒ R n taking nonempty images is said to be L-Lipschitz continuous, for some L 0, if

F (x) ⊂ F (x) + L|x -x|B for all x, x ∈ R n . Let I ⊂ R be an open interval and G : I × R n ⇒ R n be
If I = [S, T ],
then we have the following characterization of uniform absolute continuity from the left: G(•, x) is left absolutely continuous uniformly for x ∈ Λ, for some γ ∈ L 1 loc (I; R + ), if and only if for every ε > 0 there exists δ > 0 such that for any finite partition

S t 1 < τ 1 t 2 < τ 2 ... t m < τ m T of [S, T ], m i=1 (τ i -t i ) < δ =⇒ m i=1 d G(τ i ,x) (G(t i , x)) < ε ∀ x ∈ Λ, where d E ( Ẽ) := inf{β > 0 : Ẽ ⊂ E + βB} for any E, Ẽ ⊂ R n . Consider a closed set E ⊂ R n and x ∈ E. The Clarke tangent cone T C E (x) to E at x is defined by T C E (x) := {ξ ∈ R n : ∀x i → E x, ∀t i ↓ 0, ∃v i → ξ such that x i + t i v i ∈ E ∀i },
where x i → E x means x i ∈ E for all i. We denote by N C E (x) := (T C E (x)) -the Clarke normal cone to E at x, where " -" stands for the negative polar of a set.

Uniform distance estimates

We provide here sufficient conditions for uniform linear L ∞ estimates on intervals of the form I = [t 0 , t 1 ], with 0 t 0 < t 1 , for the state constrained differential inclusion

   x (t) ∈ F (t, x(t)) a.e. t ∈ I x(t) ∈ A ∀ t ∈ I,
where We denote by (H) the following hypothesis on F (•, •):

F : [0, ∞) × R n ⇒ R n is a given set-valued map and A ⊂ R n is a closed set. A function x : [t 0 , t 1 ] → R n is
(i) F has closed, nonempty values, a sub-linear growth, and F (•, x) is Lebesgue measurable for all x ∈ R n ;

(ii) there exist M 0 and α > 0 such that

sup{|v| : v ∈ F (t, x), (t, x) ∈ [0, ∞) × (∂A + αB)} ≤ M ; (2.4) (iii) there exists ϕ ∈ L loc such that F (t, •) is ϕ(t)-Lipschitz continuous for a.e. t ∈ R + .
We shall also need the following two assumptions:

(AC) there exist η > 0 and γ ∈ L loc such that F (•, x) is γ-left absolutely continuous, uniformly for x ∈ ∂A + ηB;

(IPC) for some ε > 0, η > 0 and every (t,

x) ∈ [0, ∞) × (∂A + ηB) ∩ A there exists v ∈ co F (t, x) satisfying {y + [0, ε](v + εB) : y ∈ (x + εB) ∩ A} ⊂ A.
(2.5)

We state next a uniform neighboring feasible trajectory theorem for left absolutely continuous with respect to time set-valued maps.

Theorem 2.3.1. Assume (H), (AC), and (IPC). Then for every δ > 0 there exists a constant β > 0 such that for any [t 0 , t 1 ] ⊂ [0, ∞) with t 1 -t 0 = δ, any F -trajectory x(•) defined on [t 0 , t 1 ] with x(t 0 ) ∈ A, and any ρ > 0 satisfying

ρ sup t∈[t 0 ,t 1 ] d A (x(t)),
we can find an

F -trajectory x(•) on [t 0 , t 1 ] such that x(t 0 ) = x(t 0 ), x -x ∞,[t 0 ,t 1 ] βρ & x(t) ∈ int A ∀ t ∈ (t 0 , t 1 ].
The following Proposition can be proved using the same arguments as in [BFV12, pp. 1922[BFV12, pp. -1923]].

Proposition 2.3.2. Assume (H), (AC), (IPC), and that the assertion of Theorem 2.3.1 is valid under the additional hypothesis:

F (t, x) is convex for all (t, x) ∈ [0, ∞) × R n .
Then the assertion of Theorem 2.3.1 is valid under (H), (AC), and (IPC) alone. of Theorem 2.3.1. Fix δ > 0 and let us relabel by η the constant given by min{η, η, α}. Let k > 0, ∆ > 0, ρ > 0, and

m ∈ N + (2.6) be such that k > 1/ε, (i) ∆ ε; (ii) ρ + M ∆ < ε, k ρ < ε; (iii) 4∆M η, (2.7) (i) e θϕ(∆) (θ γ (∆) + θ ϕ (∆)M ) < ε; (ii) 2e θϕ(∆) (θ γ (∆) + θ ϕ (∆)M )k < (kε -1), (2.8)
and δ m ∆.

(2.9)

We remark that all the constants appearing in (2.6) do not depend on the time interval [t 0 , t 1 ], the trajectory x(•), and ρ.

By Proposition 2.3.2, we may assume that F (•, •) = co F (•, •). We consider three cases.

Case 1: ρ ρ and δ ∆.

By (2.7)-(iii), if x(t 0 ) ∈ A\(∂A + η 2 B), then x(•) = x(•) is as desired. Suppose next that x(t 0 ) ∈ (∂A + η 2 B) ∩ A. Let v ∈ F (t 0 , x(t 0
)) be as in (IPC) and define y : [t 0 , t 1 ] → R n by y(t 0 ) = x(t 0 ) and

y (t) =    v t ∈ [t 0 , (t 0 + kρ) ∧ t 1 ] x (t -kρ) t ∈ (t 0 + kρ, t 1 ] ∩ J, ( 2
d F (s,y(s)) (y (s)) ds (θ ϕ (∆)M + θ γ (∆))kρ.
Thus, by (2.12), for all t ∈ [t 0 , (t 0 + kρ)

∧ t 1 ] y -x ∞,[t 0 ,t] e θϕ(∆) (θ γ (∆) + θ ϕ (∆)M )(t -t 0 ) (2.14) and y -x ∞,[t 0 ,t 1 ] 2e θϕ(∆) (θ γ (∆) + θ ϕ (∆)M )kρ. (2.15)
Finally, taking note of (2.11), it follows that x -x ∞,[t 0 ,t 1 ] β 1 ρ, where

β 1 = 2(M + e θϕ(∆) (θ γ (∆) + θ ϕ (∆)M ))k.
We claim next that x(t) ∈ int A for all t ∈ (t 0 , t 1 ]. Indeed, if t ∈ (t 0 , (t 0 + kρ) ∧ t 1 ], then from (IPC), (2.7)-(i) and (2.10) it follows that

y(t) + (t -t 0 )εB = x(t 0 ) + (t -t 0 )(v + εB) ⊂ A,
and it is enough to use (2.14) and (2.8)-(i).

On the other hand, if t ∈ (t 0 + kρ, t 1 ], then for π(t) Case 2: ρ > ρ and δ ∆.

∈ Π A (x(t -kρ)) we have |x(t -kρ) -π(t)| = d A (x(t -kρ)) ρ,
By the viability theorem from [START_REF] Frankowska | A measurable upper semicontinuous viability theorem for tubes[END_REF], we know that there exists a feasible Ftrajectory x(•) on [t 0 , t 1 ] starting from x(t 0 ). Note that d A (x(t)) = 0 for all t ∈ [t 0 , t 1 ]. By the Case 1, replacing x(•) with x(•), it follows that there exists a feasible F -trajectory x(•) on [t 0 , t 1 ] such that x(t 0 ) = x(t 0 ) and x((t 0 , t 1 ]) ⊂ int A. Hence, by (2.4), we have

x

-x ∞,[t 0 ,t 1 ] 2M ∆ β 2 ρ, with β 2 = 2M ∆ ρ . Case 3: δ > ∆.
The above proof implies that in Cases 1 and 2, β 1 , β 2 can be taken the same if δ is replaced by any 0 < δ 1 < δ. Define β = β 1 ∨ β 2 and let {[τ i -, τ i + ]} m i=1 be a partition of [t 0 , t 1 ] by the intervals with the length at most δ/m. -, τ 1 + ] and setting

ρ 0 = max{ρ, sup t∈[t 0 ,t 1 ] d A (x 0 (t))},
we conclude that there exists an F -trajectory

x 1 (•) on [τ 1 -, τ 1 + ] = [t 0 , τ 1 + ] such that x 1 (t 0 ) = x(t 0 ), x 1 ((t 0 , τ 1 + ]) ⊂ int A,
and

x 1 -x 0 ∞,[τ 1 -,τ 1 + ] βρ 0 .
Using Filippov's theorem, we can extend the trajectory x 1 (•) on whole interval [t 0 , t 1 ] so that

x 1 -x 0 ∞,[t 0 ,t 1 ] e t 1 t 0 ϕ(τ ) dτ βρ 0 K βρ 0 ,
where K := e θϕ(δ) .

Repeating recursively the above argument on each time interval [τ i -, τ i + ], we conclude that there exists a sequence of F -trajectories

{x i (•)} m i=1 on [t 0 , t 1 ], such that x i (t 0 ) = x(t 0 ), x i ((t 0 , τ i + ]) ⊂ int A for all i = 1, ..., m, x j (•)| [t 0 ,τ j-1 + ] = x j-1 (•)
for all j = 2, ..., m, and

x i -x i-1 ∞,[t 0 ,t 1 ] K βρ i-1 ∀ i = 1, ..., m, (2.18)
where

ρ i-1 = max{ρ, sup t∈[t 0 ,t 1 ] d A (x i-1 (t))}. Notice that ρ i ρ i-1 + x i -x i-1 ∞,[t 0 ,t 1 ] ∀ i = 1, ..., m. (2.19)
Taking note of (2.18) and (2.19) we get for all i = 1, ..., m

x i -x i-1 ∞,[t 0 ,t 1 ] K β(ρ i-2 + x i-1 -x i-2 ∞,[t 0 ,t 1 ] ) K β(1 + K β)ρ i-2 ≤ ... K β(1 + K β) i-1 ρ 0 .
Then, letting x(•) := x m (•) and observing that ρ 0 ρ, we obtain

x -x ∞,[t 0 ,t 1 ] m i=1 x i -x i-1 ∞,[t 0 ,t 1 ] K βρ 0 m i=1 (1 + K β) i-1 β 3 ρ,
where

β 3 = (1 + K β) m -1.
Then all conclusions of the theorem follow with β = β ∨ β 3 . Observe that β depends only on ε, η, M , δ, and on functions γ(•) and ϕ(•).

When F is merely measurable with respect to time, then a stronger inward pointing condition has to be imposed:

(IPC) there exist η > 0, r > 0, M 0 such that for a.e. t ∈ [0, ∞), any y ∈ ∂A + ηB, and any v ∈ F (t, y), with sup n∈N 1 y,η n, v ≥ 0, there exists w ∈ F (t, y) ∩ B(v, M ) such that sup

n∈N 1 y,η { n, w , n, w -v } -r,
where

N 1 y,η := {n ∈ S n-1 : n ∈ N C A (x), x ∈ ∂A ∩ B(y, η)}.
Let us denote by (H) the assumption (H) with (H)-(ii) replaced by a weaker requirement:

(H) (ii) ∃ q ∈ L loc such that F (t, x) ⊂ q(t) B, ∀ x ∈ ∂A, for a.e. t ∈ [0, ∞).
Remark 2.3.3. We notice that from (H) -(ii) and (iii) it follows that for any α > 0 there exists q α ∈ L loc such that F (t, x) ⊂ q α (t) B for a.e. t ∈ [0, ∞) and all x ∈ ∂A+αB. We provide next a condition that simplifies (IPC) .

Proposition 2.3.5. Assume that for some η > 0, r > 0, M 0, and Γ ⊂ [0, ∞), with µ(Γ) = 0, and for any t ∈ [0, ∞)\Γ, y ∈ ∂A + ηB, and v ∈ F (t, y), with sup n∈N 1 y,η n, v > -r, there exists w ∈ F (t, y)∩B(v, M ) satisfying sup n∈N 1 y,η n, w -v -r. Then, (IPC) holds true for all t ∈ [0, ∞)\Γ.

Proof. Indeed, otherwise there exist t ∈ [0, ∞)\Γ, y ∈ ∂A + ηB, and v ∈ F (t, y), with sup n∈N 1 y,η n, v > -r, such that for any w ∈ F (t, y)∩B(v, M ) satisfying sup n∈N 1 y,η n, wv -r we have sup n∈N 1 y,η n, w > -r. Now, by our assumptions, there exists

w 1 ∈ F (t, y) ∩ B(v, M ) such that sup n∈N 1 y,η n, w 1 -v -r.
Since ad absurdum we supposed that sup n∈N 1 y,η n, w 1 > -r, it follows that there exists

w 2 ∈ F (t, y)∩B(v, M ) satisfying sup n∈N 1 y,η n, w 2 -w 1 -r. Then for any n ∈ N 1 y,η , n, w 2 -v = n, w 2 -w 1 + n, w 1 -v -2r.
Iterating the same argument, we conclude that there exists a sequence

{w i } i∈N + in F (t, y) ∩ B(v, M ) such that sup n∈N 1 y,η n, w i -v
-ir for all i ∈ N + . This contradicts the boundedness of F (t, y) ∩ B(v, M ) and ends the proof.

UNIFORM DISTANCE ESTIMATES

Now, consider the following state constrained differential inclusion

   x (t) ∈ F (t, x(t)) a.e. t ∈ [t 0 , ∞) x(t) ∈ A ∀ t ∈ [t 0 , ∞),
where

t 0 0. A function x : [t 0 , ∞) → B n is said to be an F ∞ -trajectory or a feasible F ∞ -trajectory if x| [t 0 ,t 1 ] (•)
1 t t 0 ϕ(τ ) dτ < ∞.
Then there exist C > 1, K > 0 such that for any t 0 0, any x 0 , x 1 ∈ A, and any feasible

F ∞ -trajectory x : [t 0 , ∞) → R n , with x(t 0 ) = x 0 , we can find a feasible F ∞ -trajectory x : [t 0 , ∞) → R n , with x(t 0 ) = x 1 , such that |x(t) -x(t)| Ce Kt |x 1 -x 0 | ∀ t t 0 .
Proof. Let δ = 1 and β > 0 be as in Theorem 2.3.1 (or Theorem 2.3.4). Consider

K 1 > 0, K 2 > 0, k > 0 such that 2β + 1 < e K 1 and t+1 0 ϕ(s) ds K 2 t + k ∀ t 0.
(2.20) Fix x 0 , x 1 ∈ A, with x 1 = x 0 , and a feasible F ∞ -trajectory x : [t 0 , ∞) → R n with x(t 0 ) = x 0 . By Filippov's theorem, there exists an F -trajectory y 0 : [t 0 , t 0 + 1] → R n such that y 0 (t 0 ) = x 1 and

y 0 -x ∞,[t 0 ,t 0 +1] e t 0 +1 t 0 ϕ(s) ds |x 1 -x 0 |.
Denote by x 0 : [t 0 , t 0 + 1] → R n the feasible F -trajectory, with x 0 (t 0 ) = x 1 , satisfying the conclusions of Theorem 2.3.1 with x(•) = y 0 (•). Thus

x 0 -y 0 ∞,[t 0 ,t 0 +1] β(max t∈[t 0 ,t 0 +1] d A (y 0 (t)) + |x 1 -x 0 |) β( y 0 -x ∞,[t 0 ,t 0 +1] + |x 1 -x 0 |) 2βe t 0 +1 t 0 ϕ(s) ds |x 1 -x 0 |
and therefore

x 0 -x ∞,[t 0 ,t 0 +1] x 0 -y 0 ∞,[t 0 ,t 0 +1] + y 0 -x ∞,[t 0 ,t 0 +1] (2β + 1)e t 0 +1 t 0 ϕ(s) ds |x 1 -x 0 |. (2.21)
Now, applying again Filippov's theorem on [t 0 + 1, t 0 + 2], there exists an F -trajectory y 1 : [t 0 + 1, t 0 + 2] → R n , with y 1 (t 0 + 1) = x 0 (t 0 + 1), such that, thanks to (2.21),

y 1 -x ∞,[t 0 +1,t 0 +2] (2β + 1)e t 0 +2 t 0 ϕ(s) ds |x 1 -x 0 |. (2.22)

UNIFORM IPC FOR FUNCTIONAL SET CONSTRAINTS 57

Denoting by x 1 : [t 0 +1, t 0 +2] → R n the feasible F -trajectory, with x 1 (t 0 +1) = x 0 (t 0 +1), satisfying the conclusions of Theorem 2.3.1, for x(•) = y 1 (•), we deduce from (2.22), that

x 1 -y 1 ∞,[t 0 +1,t 0 +2] β(2β + 1)e t 0 +2 t 0 ϕ(s) ds |x 1 -x 0 |. (2.23)
Hence, taking note of (2.22) and (2.23),

x 1 -x ∞,[t 0 +1,t 0 +2] (2β + 1) 2 e t 0 +2 t 0 ϕ(s) ds |x 1 -x 0 |.
Continuing this construction, we obtain a sequence of feasible F -trajectories x i : [t 0 + i, t 0 + i + 1] → R n such that x j (t 0 + j) = x j-1 (t 0 + j) for all j 1, and

x i -x ∞,[t 0 +i,t 0 +i+1] (2β + 1) i+1 e t 0 +i+1 t 0 ϕ(s) ds |x 1 -x 0 | ∀ i ∈ N.
(2.24)

Define the feasible

F ∞ -trajectory x : [t 0 , ∞) → R n by x(t) := x i (t) if t ∈ [t 0 +i, t 0 +i+1] and observe that x(t 0 ) = x 1 .
Let t t 0 . Then there exists i ∈ N such that t ∈ [t 0 + i, t 0 + i + 1]. So, from (2.24) and (2.20), it follows that

|x(t) -x(t)| (2β + 1) i+1 e t 0 +i+1 t 0 ϕ(s) ds |x 1 -x 0 | e k(2β + 1)e (K 1 +K 2 )(t 0 +i) |x 1 -x 0 | Ce Kt |x 1 -x 0 |,
where K = K 1 + K 2 and C = e k(2β + 1).

Uniform IPC for functional set constraints

Consider the state constraints of the form

A = m i=1 A i , A i = {x ∈ R n : g i (x) 0} i = 1, ..., m,
where g i : R n → R is a C 1,1 function with bounded ∇g i (•) for all i ∈ I := {1, ..., m}. Furthermore, we assume in this section that there exist M 0 and ϕ > 0 such that sup{|v| :

v ∈ F (t, x), (t, x) ∈ [0, ∞) × ∂A} M and F (t, •) is ϕ-Lipschitz continuous for any t 0.
Proposition 2.4.1. Assume that for some δ > 0, r > 0 and for all (t, x) ∈ [0, ∞)×∂A there exists v ∈ co F (t, x) satisfying

∇g i (x), v -r ∀ i ∈ z∈B(x,δ) I(z),
where

I(z) = {i ∈ I : z ∈ ∂A i }. Then (IPC) holds true. Proof. Let us set J(x) := z∈B(x,δ) I(z) for all x ∈ ∂A. Fix (t, x) ∈ [0, ∞) × ∂A and v ∈ co F (t, x) satisfying ∇g i (x), v -r for all i ∈ J(x). Pick k > max i∈I sup x =y |∇g i (x) -∇g i (y)| |x -y| & L > max i∈I sup x∈R n |∇g i (x)|.
We divide the proof into three steps.

Step 1: We claim that there exists η > 0, not depending on (t, x), such that for all y ∈ B(x, η ) we can find w ∈ co F (t, y), with |w -v| r/4L, satisfying for all i ∈ J(x),

∇g i (y), w -r/2.
Indeed, for all i ∈ J(x) and y ∈ B(x, r/4kM ) we have

∇g i (y), v = ∇g i (y) -∇g i (x), v + ∇g i (x), v kM |y -x| -r - 3r 4 and for all w ∈ R n such that |w -v| r/4L ∇g i (y), w = ∇g i (y), w -v + ∇g i (y), v L|w -v| -3r/4 - r 2 .
Since F (t, •) is ϕ-Lipschitz continuous, there exists w ∈ co F (t, y) such that |w -v| r/4L whenever |y -x| r/4ϕL. So the claim follows with η = min{r/4ϕL, r/4kM }.

Step 2: We claim that there exists ε > 0, not depending on (t, x), such that for all y ∈ B(x, η ) we can find w ∈ co F (t, y) such that

∇g i (z), w -r/4 ∀ z ∈ B(y, ε ), ∀ w ∈ B(w, ε ), ∀ i ∈ J(x).
Indeed, let y ∈ B(x, η ) and w ∈ co F (t, y) be as in Step 1. Then for any w ∈ R n such that | w -w| r/8L and for all i ∈ J(x) and z ∈ R n , ∇g i (z), w = ∇g i (z) -∇g i (y), w + ∇g i (y), w -w + ∇g i (y), w k(M + r/4L + r/8L)|z -y| + r/8 -r/2.

So the claim follows with ε = min{k -1 (M + r/2L) -1 r/8, r/8L}.

Step 3: We prove that there exist η > 0, ε > 0, not depending on (t, x), such that for all y ∈ B(x, η) ∩ A we can find w ∈ co F (t, y) satisfying

z + τ w ∈ A ∀ z ∈ B(y, ε) ∩ A, ∀ w ∈ B(w, ε), ∀ 0 τ ε. (2.25)
Let y ∈ B(x, η ) ∩ A and w ∈ co F (t, y) be as in Step 2. Then, by the mean value theorem, for any τ 0, any z ∈ B(y, ε ) ∩ A, any w ∈ B(w, ε ), and any i ∈ J(x) there exists σ τ ∈ [0, 1] such that

g i (z + τ w) = g i (z) + τ ∇g i (z + σ τ τ w), w τ ∇g i (z), w + k(M + r/4L + ε ) 2 τ 2 -rτ 4 + k(M + r/4L + ε ) 2 τ 2 .

LIPSCHITZ CONTINUITY FOR A CLASS OF VALUE FUNCTIONS 59

Choosing η ∈ (0, η ] and ε ∈ (0, ε ] such that η + ε(M + r/4L + ε) δ and ε k -1 (M + r/4L + ε ) -2 r/4, it follows that for all z ∈ B(y, ε) ∩ A, w ∈ B(w, ε), and all 0 τ ε z + τ w ∈ B(x, δ) (2.26)

and g i (z + τ w) 0 ∀ i ∈ J(x).
(2.27) Furthermore, by (2.26) and since B(x, δ) ⊂ A j for all j ∈ I\J(x), we have for all z ∈ B(y, ε) ∩ A, w ∈ B(w, ε), and all 0 τ ε

g i (z + τ w) 0 ∀ i ∈ I\J(x).
(2.28)

The conclusion follows from (2.27) and (2.28).

Lipschitz continuity for a class of value functions

Now we give an application of the results of Section 3 to the Lipschitz regularity of the value function for a class of infinite horizon optimal control problems subject to state constraints.

Let us consider the problem B ∞ stated in the Introduction. Recall that for a function q ∈ L 1 loc ([t 0 , ∞); R) the integral ∞ t 0 q(t) dt := lim T →∞ T t 0 q(t) dt, provided this limit exists. We denote by (h) the following assumptions on f and l:

(i) there exists α > 0 such that f and l are bounded functions on

{(t, x, u) : t 0, x ∈ (∂A + αB), u ∈ U (t)}; (ii) for all (t, x) ∈ [0, ∞) × R n the set {(f (t, x, u), l(t, x, u)) : u ∈ U (t)} is closed; (iii) there exist c ∈ L 1 loc ( [0, ∞); R + ) and k ∈ L loc such that for a.e. t ∈ R + and for all x, y ∈ R n , u ∈ U (t), |f (t, x, u) -f (t, y, u)| + |l(t, x, u) -l(t, y, u)| k(t)|x -y|, |f (t, x, u)| + |l(t, x, u)| c(t)(1 + |x|);

LIPSCHITZ CONTINUITY FOR A CLASS OF VALUE FUNCTIONS (iv) lim sup t→∞

1 t t 0 (c(s) + k(s)) ds < ∞; (v) for all x ∈ R n the mappings f (•, x, •), l(•, x, •) are Lebesgue-Borel measurable.
Furthermore, we denote by (h) the assumptions (h) with (h)-(i) replaced by:

(h) (i) ∃ q ∈ L loc such that for a.e. t ∈ [0, ∞) sup u∈U (t) (|f (t, x, u)| + |l(t, x, u)|) q(t), ∀ x ∈ ∂A.
In what follows G : [0, ∞) × R n → R n+1 is the measurable with respect to t set-valued map defined by

G(t, x) = {(f (t, x, u), l(t, x, u)) : u ∈ U (t)}.
For control systems, the conditions (IPC), (AC), and (IPC) take the following form:

(ipc) for some ε > 0, η > 0 and every (t, x) ∈ [0, ∞) × (∂A + ηB) ∩ A there exist

{α i } n i=0 ⊂ [0, 1], with n i=0 α i = 1, and {u i } n i=0 ⊂ U (t) satisfying y + [0, ε] n i=0 α i f (t, x, u i ) + εB : y ∈ (x + εB) ∩ A ⊂ A;
(ac) there exist η > 0 and γ ∈ L loc such that G(•, x) is γ-left absolutely continuous, uniformly for x ∈ ∂A + ηB;

(ipc) there exist η > 0, r > 0, M 0 such that for a.e. t ∈ [0, ∞), any y ∈ ∂A + ηB, and any u ∈ U (t), with sup n∈N 1 y,η n, f (t, y, u) 0, there exists w ∈ {w ∈ U (t) :

|f (t, y, w ) -f (t, y, u)| M } such that sup n∈N 1 y,η n, f (t, y, w) , n, f (t, y, w) -f (t, y, u) } -r.
Remark 2.5.1. If there exist η > 0, γ, γ ∈ L loc , and k 0 such that (f (•, x, u), l(•, x, u)) is γ-left absolutely continuous, uniformly for (x, u) ∈ (∂A + ηB) × R m , U (•) is γ-left absolutely continuous, and f (t, x, •) is k-Lipschitz continuous for all (t, x) ∈ [0, ∞) × (∂A + ηB), then (ac) holds true.

Theorem 2.5.2. Assume that either (h), (ac), and (ipc) or (h) and (ipc) hold true. Then there exist b > 1, K > 0 such that for all λ > K and every t 0 the function

V (t, •) is L(t)-Lipschitz continuous on A with L(t) = be -(λ-K)t .
Furthermore, for all λ > K and for every feasible trajectory x(•), we have

lim t→∞ V (t, x(t)) = 0.
Proof. We notice that, by the inward pointing conditions (ipc) or (ipc) and the viability theorem from [START_REF] Frankowska | A measurable upper semicontinuous viability theorem for tubes[END_REF], the problem B ∞ admits feasible trajectory-control pairs for any initial condition. Pick (t 0 , x 0 ) ∈ [0, ∞) × A.

T t 0 + (λ -a 1 ) T t 0 e -(λ-a 1 )t ψ(t) dt (1 + |x 0 )e a 2 e -(λ-a 1 )T (a 1 T + a 2 ) + a 1 t 0 + a 1 λ-a 1 + a 2 e -(λ-a 1 )t 0
(2.31)

Passing to the limit when T → ∞, we deduce that for every feasible trajectory-control pair

(x(•), u(•)) at (t 0 , x 0 ) ∞ t 0 e -λt |l(t, x(t), u(t))| dt < +∞ ∀λ > a 1 .
From now on, assume that λ > a 1 . Fix t 0 and x 1 , x 0 ∈ A with x 1 = x 0 . Then, for any δ > 0 there exists a feasible trajectory-control pair (x δ (•),

u δ (•)) at (t, x 0 ) such that V (t, x 0 ) + e -δt |x 1 -x 0 | > ∞ t
e -λs l(s, x δ (s), u δ (s)) ds.

Hence

V (t, x 1 ) -V (t, x 0 ) e -δt |x 1 -x 0 |+ lim τ →∞ τ t e -λs l(s, x(s), u(s)) ds -τ t e -λs l(s, x δ (s), u δ (s)) ds (2.32)
for any feasible trajectory-control pair (x(•),

u(•)) satisfying x(t) = x 1 . Define G(t, x, z) = G(t, x) for all (t, x, z) ∈ [0, ∞) × R n × R and consider the following state constrained differential inclusion in R n+1    (x, z) (s) ∈ G(s, x(s), z(s)) a.e. s ∈ [t, ∞) x(s) ∈ A ∀ s ∈ [t, ∞).
Putting z δ (s) = s t l(ξ, x δ (ξ), u δ (ξ)) dξ, by Theorem 2.3.6 applied on A × R and the measurable selection theorem, there exist C > 1, K > 0 such that for all δ > 0 we can find a G∞ -trajectory (x δ (•), zδ (•)) on [t, ∞), and a measurable selection ũδ (s) ∈ U (s) 62

LIPSCHITZ CONTINUITY FOR A CLASS OF VALUE FUNCTIONS

a.e. s t, satisfying (x δ , zδ ) (s) = (f (s, xδ (s), ũδ (s)), l(s, xδ (s), ũδ (s)))

a.e. s t,

(x δ (t), zδ (t)) = (x 1 , 0), xδ ([t, ∞)) ⊂ A,
(ξ), u δ (ξ)) dξ) ds e -λτ |z δ (τ ) -z δ (τ )| + λ τ t e -λs |z δ (s) -z δ (s)|ds Ce -λτ e Kτ |x 1 -x 0 | + λC τ t e -(λ-K)s |x 1 -x 0 | ds = Ce -(λ-K)τ + λC -e -(λ-K)s λ-K τ t |x 1 -x 0 | = -CK λ-K e -(λ-K)τ + λC λ-K e -(λ-K)t |x 1 -x 0 | λC λ-K e -(λ-K)t |x 1 -x 0 |.
(2.34)

Taking note of (2.32), (2.34), and putting δ = λ -K, for all λ > K we get

V (t, x 1 ) -V (t, x 0 ) λC λ -K + 1 e -(λ-K)t |x 1 -x 0 |.
By the symmetry of the previous inequality with respect to x 1 and x 0 , and since λ, C, and K do not depend on t, x 1 , and x 0 , the first conclusion follows. Now, let (t 0 , x 0 ) ∈ [0, ∞) × A and consider a feasible trajectory X(•) at (t 0 , x 0 ). Let t > t 0 and (x(•), u(•)) be a feasible trajectory-control pair at (t, X(t)) such that T t e -(λ-a 1 )s c(s) ds. Then, arguing as in (2.31) with t 0 replaced by t and taking the limit when T → ∞, we deduce that

V (t, X(t)) > ∞ t e -λs l(s, x(s), u(s)) ds -1 t . Then |V (t, X(t))| ∞ t e -
|V (t, X(t))| (1 + |x 0 |)e a 2 a 1 t + a 1 λ -a 1 + a 2 e -(λ-a 1 )t + 1 t .
Since K a 1 , the last conclusion follows passing to the limit when t → ∞.

Corollary 2.5.3. Assume that either (h), (ac), and (ipc) or (h) and (ipc) hold true and that f, l are bounded. Consider any N > 0 with

N ≥ sup{|f (t, x, u)| + |l(t, x, u)| : t 0, x ∈ R n , u ∈ U (t)} < ∞.
Then, for any λ > 0 sufficiently large, for any x ∈ A, and any t 0 the function

V (•, x) is Lipschitz continuous on [t, ∞) with constant L(t) + 2e -λt N .
Proof. By Theorem 2.5.2, when λ > 0 is large enough, V (t, •) is L(t)-Lipschitz continuous on A. Fix x ∈ A and t 0. Let s, s ∈ [t, ∞). Suppose that s s. Then, by the dynamic programming principle, there exists a feasible trajectory-control pair (x(•), ū(•)) at (s, x) such that

V (s, x) -V (s, x) |V (s, x) -V (s, x(s))| + s s e -λξ |l(ξ, x(ξ), ū(ξ))| dξ +N |s -s|e -λt L(s)N |s -s| + N |s -s|e -λs + N |s -s|e -λt L(t) + 2e -λt N |s -s|.
(2.35)

Arguing in a similar way, we get (2.35) when s < s. Hence, by the symmetry with respect to s and s in (2.35), the conclusion follows.

Applications to the relaxation problem

Let f (•), l(•), and U (•) be as in B ∞ . Consider the relaxed infinite horizon state constrained problem

B rel ∞ Ṽ (t 0 , x 0 ) = inf ∞ t 0 e -λt l(t, x(t), w(t)) dt,
where the infimum is taken over all trajectory-control pairs (x(•), w(•)) subject to the state constrained control system

             x (t) = f (t, x(t), w(t)) a.e. t ∈ [t 0 , ∞) x(t 0 ) = x 0 w(t) ∈ W (t) a.e. t ∈ [t 0 , ∞) x(t) ∈ A ∀ t ∈ [t 0 , ∞),
where λ > 0, W : [0, ∞) ⇒ R (n+1)m × R n+1 is the measurable set-valued map defined by and the functions f : [0, ∞)×R n ×R (n+1)m ×R n+1 → R n and l : [0, ∞)×R n ×R (n+1)m × R n+1 → R are defined by: for all t 0, x ∈ R n , and w = (u 0 , ..., u n , α 0 , ..., α n ) ∈

W (t) := (× n i=0 U (t)) × {(α 0 , ..., α n ) ∈ R n+1 : n i=0 α i = 1, α i 0 ∀ i},
R (n+1)m × R n+1 f (t, x, w) = n i=0 α i f (t, x, u i ) & l(t, x, w) = n i=0 α i l(t, x, u i ).
Theorem 2.6.1. Assume that either (h), (ac), and (ipc) or (h) and (ipc) hold true.

Then, for all large λ > 0, Ṽ (•,

•) = V (•, •) on [0, ∞) × A.
Proof. Notice that Ṽ (t, x) V (t, x) for any (t, x) ∈ [0, ∞) × A, and that Theorem 2.5.2 implies that Ṽ (t, •) and V (t, •) are Lipschitz continuous on A for all t 0 whenever λ > 0 is sufficiently large. That is, in particular, they are continuous and finite.

Fix (t 0 , x 0 ) ∈ [0, ∞) × A. We claim that: for all j ∈ N + there exists a finite set of trajectory-control pairs {(x k (•), u k (•))} k=1,...,j satisfying the following:

x k (s) = f (s, x k (s), u k (s)) a.e. s ∈ [t 0 , t 0 + k] and x k ([t 0 , t 0 + k]) ⊂ A for all k = 1, ..., j; if j 2, x k | [t 0 ,t 0 +k-1] (•) = x k-1 (•)
for all k = 2, ..., j; and for all k = 1, ..., j

Ṽ (t 0 , x 0 ) Ṽ (t 0 + k, x k (t 0 + k)) + t 0 +k t 0 e -λt l(t, x k (t), u k (t)) dt -ε k i=1 1 2 i .
(2.36)

We prove the claim by the induction argument with respect to j ∈ N + . By the dynamic programming principle, there exists a trajectory-control pair (x(•), w(•)) on [t 0 , t 0 + 1], feasible for the problem B rel ∞ at (t 0 , x 0 ), such that

Ṽ (t 0 , x 0 ) + ε 4 > Ṽ (t 0 + 1, x(t 0 + 1)) + t 0 +1 t 0 e -
+ t 0 +1 t 0 e -λt l(t, x h (t), u h (t)) dt -t 0 +1 t 0 e -λt l(t, xh (t), ûh (t)) dt < h(2β + 1) and x h -x ∞,[t 0 ,t 0 +1] x -xh ∞,[t 0 ,t 0 +1] + x h -xh ∞,[t 0 ,t 0 +1] < h(2β + 1).
Hence, choosing 0 < h < ε/4(2β + 1) sufficiently small, we can find a trajectory-control pair (x h (•), u h (•)) on [t 0 , t 0 + 1], with u h (s) ∈ U (s) and (x h ) (s) = f (s, x h (s), u h (s)) a.e. s ∈ [t 0 , t 0 + 1], x h (t 0 ) = x 0 , and x h ([t 0 , t 0 + 1]) ⊂ A, such that, by (2.37) and continuity of Ṽ (t 0 + 1, •)

Ṽ (t 0 , x 0 ) > Ṽ (t 0 + 1, x h (t 0 + 1)) + t 0 +1 t 0 e -λt l(t, x h (t), u h (t)) dt - ε 2 . Letting (x 1 (•), u 1 (•)) := (x h (•), u h (•)
), the conclusion follows for j = 1. Now, suppose we have shown that there exist {(x k (•), u k (•))} k=1,...,j satisfying the claim. Let us to prove it for j + 1. By the dynamic programming principle there exists a trajectory-control pair (x(•), w(•)) on [t 0 + j, t 0 + j + 1], feasible for the problem B rel ∞ at (t 0 + j, x j (t 0 + j)), such that Ṽ (t 0 + j, x j (t 0 + j)) + ε 2 j+2 > Ṽ (t 0 + j + 1, x(t 0 + j + 1)) + t 0 +j+1 t 0 +j e -λt l(t, x(t), w(t)) dt.

(2.38)

As before, for every h > 0 there exist a feasible G-trajectory Putting

X h (•) = (x h (•), z h (•)) on [t 0 + j, t 0 + j + 1], with X h (t 0 ) = (x j (t 0 + j), 0),
(x j+1 (•), u j+1 (•)) :=    (x j (•), u j (•)) on [t 0 , t 0 + j] (x h (•), u h (•)) on [t 0 + j, t 0 + j + 1],
(2.39) and choosing 0 < h < ε/2 j+2 (2β + 1) sufficiently small, it follows from (2.38) that Ṽ (t 0 + j, x j (t 0 + j)) Ṽ (t 0 + j + 1, x j+1 (t 0 + j + 1))

+ t 0 +j+1 t 0 +j e -λt l(t, x j+1 (t), u j+1 (t)) dt -2ε 2 j+2 .
(2.40)

So, taking note of (2.39) and (2.40), we obtain

Ṽ (t 0 , x 0 ) Ṽ (t 0 + j, x j (t 0 + j)) + t 0 +j t 0 e -λt l(t, x j (t), u j (t)) dt -ε j i=1 1 2 i Ṽ (t 0 + j + 1, x j+1 (t 0 + j + 1)) -ε j i=1 1 2 i -ε 2 j+1 + t 0 +j+1 t 0 +j e -λt l(t, x j+1 (t), u j+1 (t)) dt + t 0 +j t 0 e -λt l(t, x j (t), u j (t)) dt = Ṽ (t 0 + j + 1, x j+1 (t 0 + j + 1)) + t 0 +j+1 t 0 e -λt l(t, x j+1 (t), u j+1 (t)) dt -ε j+1 i=1 1 2 i . Hence {(x k (•), u k (•))} k=1,.
..,j+1 also satisfy our claim. Now, let us define the trajectorycontrol pair (x(•), u(•)) by (x(t), u(t))

:= (x k (t), u k (t)) if t ∈ [t 0 + k -1, t 0 + k]. Then (x(•), u(•)
) is a feasible trajectory-control pair for the problem B ∞ at (t 0 , x 0 ). Since Ṽ (t, x(t)) → 0 when t → +∞, by (2.36), we have

Ṽ (t 0 , x 0 ) ∞ t 0 e -λt l(t, x(t), u(t)) dt -ε.
Hence, we deduce that Ṽ (t 0 , x 0 ) V (t 0 , x 0 ) -ε. Since ε is arbitrary, the conclusion follows.

INTRODUCTION

when the Hamiltonian H is continuous, while in [START_REF] Barles | Existence results for first order Hamilton Jacobi equations[END_REF][START_REF] Souganidis | Existence of viscosity solutions of Hamilton-Jacobi equations[END_REF] the authors extended the existence results to a large class of continuous Hamiltonians. When the solution is differentiable, then it solves the H-J-B equation also in the classical sense. However, it is well known that such a kind of notion turns out to be quite unsatisfactory for H-J-B equations arising in control theory and the calculus of variations (we refer to [START_REF] Bardi | Optimal control and viscosity solutions of Hamilton-Jacobi-Bellman equations[END_REF][START_REF] Lions | Generalized solutions of Hamilton-Jacobi equations[END_REF] for further discussions). Indeed, the value function, that is a weak solution of H-J-B equation, loses the differentiability property (even in the absence of state constraints) whenever there are multiple optimal solutions at the same initial condition. When additional state constraints are present it also loses its continuity. At most we expect lower semicontinuity of the value function. So, subsequently, the definition of solution was extended to lower semicontinuous functions.

For the Mayer problem (of optimal control theory) free of state constraints involving a continuous cost function and Lipschitz continuous dynamics, the uniqueness of continuous solutions of the associated H-J-B equation can be addressed using the notion of viscosity solution. Further, the definition of solution can be stated equivalently in terms of "normals" to the epigraph and the hypograph of the solution. But, when the dynamics is only measurable in time such equivalence may fail to be true. Nevertheless, the study of uniqueness of weak solutions can be carried out by using the solutions concept from [START_REF] Frankowska | Measurable viability theorems and the Hamilton-Jacobi-Bellman equation[END_REF], see also Sections 3 and 4 below, based on "normals" to the epigraph. Previously, to deal with Hamiltonian measurable in time, in [START_REF] Ishii | Hamilton-Jacobi equations with discontinuous Hamiltonians on arbitrary open sets[END_REF] the author proposed a new notion of weak solution (cfr. [START_REF] Lions | Remarks on Hamilton-Jacobi equations with measurable time-dependent Hamiltonians[END_REF] for equivalent formulations of such a kind of solutions) in the class of continuous functions, proving, by a blow-up method, the uniqueness and existence in the stationary case on a general open subset of R n and for the evolutionary case on (0, ∞) × R n . The C 1 test functions needed to define such solutions are more complex, involving in addition some integrable mappings. We point out that, under the assumptions that H is measurable in time, Lipschitz continuous in the space variable, and convex in the last variable, the so called representation theorems (cfr. [START_REF] Frankowska | Stable representation of convex Hamiltonians[END_REF][START_REF] Rampazzo | Faithful representations for convex Hamilton-Jacobi equations[END_REF] and the reference therein) associate to the H-J-B equation a control problem in such a way that the value function is a weak solution. This yields an existence result for weak solutions.

To deal with discontinuous solutions, in [START_REF] Ishii | Perron's method for monotone systems of second-order elliptic partial differential equations[END_REF], Ishii introduced the concept of lower and upper semicontinuous envelopes of a function, proving that the upper semicontinuous envelope of the value function of an optimal control problem is the largest upper semicontinuous subsolution and its lower semicontinuous envelope is the smallest lower semicontinuous supersolution. This approach, however, does not ensure the uniqueness of (weak) solutions of the H-J-B equation. On the other hand the upper semicontinuous envelope does not have any meaning in optimal control theory while dealing with 3.1. INTRODUCTION 69 minimization problems (the lower semicontinuous envelope determines the value function of the relaxed problem). In [START_REF] Barron | Semicontinuous viscosity solutions for Hamilton-Jacobi equations with convex Hamiltonians[END_REF][START_REF] Barron | Optimal control and semicontinuous viscosity solutions[END_REF][START_REF] Frankowska | Lower semicontinuous solutions of Hamilton-Jacobi-Bellman equations[END_REF] a different concept of solutions was developed for the H-J-B equation associated to the Mayer optimal control problem not involving state constraints, but having a discontinuous cost. In this approach only subdifferentials are involved. In particular, in [START_REF] Frankowska | Lower semicontinuous solutions of Hamilton-Jacobi-Bellman equations[END_REF], results are expressed using the Fréchet subdifferentials instead of C 1 test functions. By [CEL84, Proposition 1.1], Fréchet subdifferentials of continuous functions coincide with those defined in [START_REF] Crandall | Viscosity solutions of Hamilton-Jacobi equations[END_REF] via C 1 test functions. While investigating in [START_REF] Frankowska | Measurable viability theorems and the Hamilton-Jacobi-Bellman equation[END_REF] the merely measurable case, it became clear that in order to get uniqueness, it is convenient to replace subdifferentials by normals to the epigraph of solutions. Such "geometric" definition of solution avoids using test functions and allows to have a unified approach to both the continuous and the measurable case.

To deal with state constrained problems, the usual assumptions on data may be not sufficient to derive existence and uniqueness results for the H-J-B equations. In [START_REF] Soner | Optimal control problems with state-space constraints I[END_REF] Soner proposed a controllability assumption (the Slatter like assumption) to investigate an autonomous control problem, recovering the continuity of the value function through an inward pointing condition (under the assumption that the set Ω is bounded with ∂Ω ∈ C 2 ): that is, he assumed that for any x ∈ ∂Ω we can find a control u satisfying f (x, u), ν x < 0, where ν x is the outward unit normal to Ω at x and f is the dynamics of control system. Such condition implies uniqueness of viscosity solutions. However, it cannot be used for sets with nonsmooth boundary and the boundedness assumption on Ω may be quite restrictive for many applied models: for instance, macroeconomics models often consider cones as state constraints. To allow nonsmooth boundaries, Ishii and Koike generalized the concept of Soner's condition in the framework of infinite horizon problems and continuous solutions (cfr. [START_REF] Ishii | A new formulation of state constraint problems for first-order PDEs[END_REF] and the references therein). More generally, various versions of inward pointing condition are useful to get continuity or Lipschitz continuity of the value function, see for instance [BF]. Furthermore, in [START_REF] Frankowska | Hamilton-Jacobi equations for infinite horizon control problems with state constraints[END_REF][START_REF] Frankowska | Semicontinuous solutions of Hamilton-Jacobi-Bellman equations with degenerate state constraints[END_REF] the authors, dealing with paratingent cones and closed set of constraints with possibly empty interior, carry out the analysis under another controllability requirement named outward pointing condition. Such condition ensures, roughly speaking, that any boundary point of Ω can be reached by trajectories laying in the relative interior of Ω. The outward pointing conditions allow furthermore to use the so called backward neighboring feasible trajectory theorems, fundamental to address the control systems under state constraints. It was used, in particular, in [START_REF] Frankowska | Discontinuous solutions of Hamilton-Jacobi-Bellman equation under state constraints[END_REF], to study an H-J-B equation on finite time interval, when the Hamiltonian is convex and positively homogeneous in the third variable.

We would like to underline here that, in contrast, the inward pointing condition 3.2. PRELIMINARIES is neither needed, nor well adapted in the context of lower semicontinuous functions because it does not imply uniqueness of solutions to the H-J-B equation unless further regularity assumptions are imposed on the solutions.

The novelty of our work consists in examining the weak solutions (in the sense of Definition 3.3.2 below) of the H-J-B equation on (0, ∞) × Ω (where Ω is an open subset of R n with possibly nonsmooth boundary) and with time-measurable Hamiltonian (associated with an infinite horizon optimal control problem). Proofs of uniqueness make use of the geometric properties of epigraphs of such solutions. We recover the uniqueness, from a neighboring feasible trajectory theorem (cfr. [BF]) under a backward controllability assumption, in a class of lower semicontinuous functions vanishing at infinity. More precisely, we prove the existence and uniqueness of weak solutions of the following problem

     chapter 3 -∂ t W + H (t, x, -∇ x W ) = 0 on (0, ∞) × Ω lim t→∞ sup y∈dom W (t,•) |W (t, y)| = 0.
The outline of this chapter is as follows. In Section 2 we introduce notations and recall some results from nonsmooth analysis. The main result is stated in Section 3 whose proof is left to Section 4. In the last section we discuss the particular case of the Lipschitz continuous solutions.

Preliminaries

We denote by | • | and •, • the Euclidean norm and scalar product in R k , respectively, and by µ the Lebesgue measure. Let (X, | • | X ) be a normed space, B(x, δ) stand for the closed ball in X with radius δ > 0 centered at x ∈ X and B = B(0, 1). For a nonempty subset C ⊂ X we denote the interior of C by int C, the boundary of C by ∂C, the convex hull of C by co C, its closure by co C, and the distance from Let I and J be two closed intervals in R. We denote by L 1 (I; J) the set of all J-valued Lebesgue integrable functions on I. We say that f ∈ L 1 loc (I; J) if f ∈ L 1 (K; J) for any compact subset K ⊂ I. We denote by L loc the set of all functions f ∈ L 1 loc ([0, ∞); R + ) such that lim σ→0 θ f (σ) = 0 where θ f (σ) = sup { J f (τ ) dτ : J ⊂ [0, ∞), µ(J) σ}. We recall that for a function q ∈ L 1 loc ([0, ∞); R) the integral ∞ t 0 q(s) ds := lim T →∞ T t 0 q(s) ds, whenever this limit exists.

x ∈ X to C by d C (x) := inf {|x -y| X : y ∈ C}. If X = R k , in
Let D ⊂ R n be nonempty and {A h } h∈D be a family of nonempty subsets of R k .

PRELIMINARIES 71

The upper and lower limits, in the Kuratowski-Painlevé sense, of A h at h 0 ∈ D are the closed sets defined respectively by Lim sup

h→ D h 0 A h = v ∈ R k : lim inf h→ D h 0 d A h (v) = 0 , Lim inf h→ D h 0 A h = v ∈ R k : lim sup h→ D h 0 d A h (v) = 0 . Consider a nonempty subset E ⊂ R k and x ∈ E. The contingent cone T E (x) to E at x is defined as the set of all vectors v ∈ R k such that lim inf h→0+ d E (x + hv) h = 0. The limiting normal cone to E at x, written N E (x), is defined by N E (x) := Lim sup y→ E x T E (y) -. It is known that N E (x) -⊂ T E (x) whenever E is closed. The Clarke tangent cone is defined by N E (x) -.
Let ϕ : R k → R ∪ {±∞} be an extended real function. We write dom ϕ for the domain of ϕ, epi ϕ for the epigraph of ϕ, and hypo ϕ for the hypograph of ϕ. The (Fréchet) subdifferential, respectively the (Fréchet) superdifferential, of ϕ at x 0 ∈ dom ϕ are the possibly empty sets defined by

∂ -ϕ(x 0 ) = p ∈ R k : lim inf x→x 0 ϕ(x) -ϕ(x 0 ) -p, x -x 0 |x -x 0 | 0 , ∂ + ϕ(x 0 ) = -∂ -(-ϕ)(x 0 ).
The contingent epiderivative and the contingent hypoderivative of ϕ at x 0 ∈ dom ϕ, in the direction u ∈ R k , written D ↑ ϕ(x 0 )(u) and D ↓ ϕ(x 0 )(u), respectively, are defined by

D ↑ ϕ(x 0 )(u) = lim inf h→0+, u →u ϕ(x 0 + hu ) -ϕ(x 0 ) h , D ↓ ϕ(x 0 )(u) = -D ↑ (-ϕ)(x 0 )(u).
It is well known that (cfr. [AF09, Proposition 6.

1.4]) epi D ↑ ϕ(x 0 ) = T epi ϕ (x 0 , ϕ(x 0 )) & hypo D ↓ ϕ(x 0 ) = T hypo ϕ (x 0 , ϕ(x 0 )). (3.1)
From [START_REF] Cannarsa | Value function, relaxation, and transversality conditions in infinite horizon optimal control[END_REF] we know that, for a measurable mapping ϕ, p ∈ ∂ -ϕ(x 0 ) if and only if there exists a continuous function ψ : R k → R, differentiable at x 0 , such that ψ(x) < ϕ(x) for all x = x 0 , ϕ(x 0 ) = ψ(x 0 ), and ∇ψ(x 0 ) = p. If in addition ϕ is continuous, then ψ can be chosen to be of class C 1 . In this respect for a lower semicontinuous function ϕ the notion of the (Fréchet) subdifferential we consider differs from the one in [START_REF] Crandall | Viscosity solutions of Hamilton-Jacobi equations[END_REF], where only continuous viscosity solutions were investigated and C 1 support functions were used. Similar remark can be made about superdifferentials.

A set-valued map F : R k R n taking nonempty values is said to be upper semicontinuous at x ∈ R k if for any ε > 0 there exists δ > 0 such that F (x ) ⊂ F (x) + ε B for all x ∈ B(x, δ). If F is upper semicontinuous at every x then it is said to be upper semicontinuous. F is said to be lower semicontinuous at

x ∈ R k if Lim inf y→x F (y) ⊂ F (x).
F is said to be lower semicontinuous if F is lower semicontinuous at every x ∈ R k . F is called continuous at x ∈ R k if it is lower and upper semicontinuous at x and it is continuous if it is continuous at each point x. Definition 3.2.1. A set-valued map P : I R k is locally absolutely continuous if it takes nonempty closed images and for any [S, T ] ⊂ I, every ε > 0, and any compact subset K ⊂ R k , there exists δ > 0 such that for any finite partition

S t 1 < τ 1 t 2 < τ 2 ... t m < τ m T of [S, T ], m i=1 (τ i -t i ) < δ =⇒ m i=1 max dP(t i ) (P (τ i ) ∩ K), dP(τ i ) (P (t i ) ∩ K) < ε,
where dE (E ) := inf {β > 0 : E ⊂ E + βB} for any E, E ⊂ R k (the infimum over an empty set is +∞, by convention).

Main result

Consider the infinite horizon optimal control problem minimize ∞ t 0

L(t, x(t), u(t)) dt

(3.2) over all the trajectory-control pairs of the state constrained control system on I = [t 0 , ∞)

     x (t) = f (t, x(t), u(t)), u(t) ∈ U (t), for a.e. t ∈ I x(t 0 ) = x 0 , x(I) ⊂ A, (3.3) where f : [0, ∞) × R n × R m → R n and L : [0, ∞) × R n × R m → R are given, A is a nonempty closed subset of R n , U : [0, ∞)
R m is a Lebesgue measurable set-valued map with closed nonempty images and (t 0 , x 0 ) ∈ [0, ∞) × A is the initial datum. Every trajectory-control pair (x(•), u(•)) that satisfies the state constrained control system (3.3) on an interval of the form I = [t 0 , T ] or I = [t 0 , ∞) is called feasible on I. We refer to such x(•) as a feasible trajectory. The infimum of the cost functional in (3.2) over all feasible trajectory-control pairs on I = [t 0 , ∞), with the initial datum (t 0 , x 0 ), is denoted by V (t 0 , x 0 ) (if no feasible trajectory-control pair exists at (t 0 , x 0 ), or if the integral in (3.2) is not defined for every feasible pair, we set V (t 0 , x 0 ) = +∞). The function

V : [0, ∞) × A → R ∪ {±∞} is called the value function of problem (3.2)-(3.3). We say that (x(•), ū(•)) is an optimal trajectory-control pair at (t 0 , x 0 ) ∈ ([0, ∞) × A) ∩ dom V if V (t 0 , x 0 ) = ∞ t 0 L(s, x(s), ū(s)) ds. Finally, H (t, x, p) := sup u∈U (t) ( f (t, x, u), p -L(t, x, u))
is the Hamiltonian function associated to the above problem. We denote by (h) the following assumptions on f and L:

(h) (i) ∀ x ∈ R n the mappings f (•, x,
•) and L(•, x, •) are Lebesgue-Borel measurable and there exists φ ∈ L 1 ([0, ∞); R) such that L(t, x, u) φ(t) for a.e. t 0 and all (x, u

) ∈ R n × R m ; (ii) ∃ c ∈ L 1 loc ([0, ∞); R +
) such that for a.e. t 0 and for all x ∈ R n , u ∈ U (t)

|f (t, x, u)| + |L(t, x, u)| c(t)(1 + |x|);
(iii) for a.e. t 0 and all x ∈ R n , the set-valued map

R n y {(f (t, y, u), L(t, y, u)) : u ∈ U (t)} (3.4)
is continuous with closed images, and the set

{(f (t, x, u), L(t, x, u) + r) : u ∈ U (t), r 0} (3.5) is convex.
We denote by (h) the assumptions (h) with the further requirement:

(h) (iv) ∃ k ∈ L 1 loc ([0, ∞); R + ) such that for a.e. t 0 and for all x, y ∈ R n , u ∈ U (t) |f (t, x, u) -f (t, y, u)| + |L(t, x, u) -L(t, y, u)| k(t)|x -y|,
and by (h) the assumptions (h) with the further:

(h) (v) k ∈ L loc and lim sup t→∞ 1 t t 0 (c(s) + k(s)) ds < ∞; (vi) ∃ q ∈ L loc such that for a.e. t 0 sup u∈U (t) (|f (t, x, u)| + |L(t, x, u)|) q(t), ∀ x ∈ ∂A.
Moreover, we denote by (B) and (OPC) the following assumptions: (B) dom V = ∅ and there exist T > 0 and

ψ ∈ L 1 ([T, ∞); R + ) such that for all (t 0 , x 0 ) ∈ dom V ∩([T, ∞)×R n ) and any feasible trajectory-control pair (x(•), u(•)) on I = [t 0 , ∞), with x(t 0 ) = x 0 , |L(t, x(t), u(t))| ψ(t) for a.e. t t 0 ;
(OPC) there exist η > 0, r > 0, M 0 such that for a.e. t > 0 and any y ∈ ∂A + ηB, and any v ∈ f (t, y, U (t)), with inf n∈N 1 y,η n, v 0, we can find

w ∈ f (t, y, U (t)) ∩ B(v, M ) satisfying inf n∈N 1 y,η { n, w , n, w -v } r, where N 1 y,η := {n ∈ ∂B : n ∈ coN A (x), x ∈ ∂A ∩ B(y, η)}.
We denote by (IPC) the conditions (OPC) in which f (t, y, U (t)) is replaced by -f (t, y, U (t)).

Remarks 3.3.1.

(i) If L(t, x, u) = e -λt l(t, x, u), with l bounded and λ > 0, then (B) is satisfied.

(ii) If f (t, •, u) and L(t, •, u) are continuous, uniformly in u ∈ U (t), then the set-valued map in (3.4) is continuous for a.e. t 0.

Define the augmented Hamiltonian

H : [0, ∞) × R n × R n × R → R by H(t, x, p, q) = sup u∈U (t) ( f (t, x, u), p -qL(t, x, u)) . Definition 3.3.2. A function W : [0, ∞) × A → R ∪ {+∞} is called a weak (or viscosity) solution of H-J-B equation on (0, ∞) × A if there exists a set C ⊂ (0, ∞), with µ(C ) = 0, such that for all (t, x) ∈ dom W ∩ (((0, ∞)\C ) × ∂A) -p t + H(t, x, -p x , -q) 0 ∀ (p t , p x , q) ∈ T epi W (t, x, W (t, x)) -, (3.6)
and for all (t, x)

∈ dom W ∩ (((0, ∞)\C ) × int A) -p t + H(t, x, -p x , -q) = 0 ∀ (p t , p x , q) ∈ T epi W (t, x, W (t, x)) -. (3.7)
The next theorem ensures the existence and uniqueness of (weak) solutions of the Hamilton-Jacobi-Bellman equation in the class of the lower semicontinuous functions vanishing at infinity.

Theorem 3.3.3. Assume (h) and (OPC). Let

W : [0, ∞) × A → R ∪{+∞} be a lower semicontinuous function such that dom V (t, •) ⊂ dom W (t, •) = ∅ for all large t > 0 and lim t→∞ sup y∈dom W (t,•) |W (t, y)| = 0. (3.8)
Then the following statements are equivalent:

(i) W=V; (ii) W is a weak solution of H-J-B equation on (0, ∞) × A
|W (t, y)| = 0,
and assuming further regularity:

∃ τ > 0 : lim inf s→t-, y→ int A x W (s, y) = W (t, x) ∀ (t, x) ∈ (τ, ∞) × A.
(3.9) By Proposition 3.4.4-(iii) given below and [BF, Theorem 2], the value function V satisfies (3.9) whenever (h) holds true.

(iv) Under the assumption (OPC), if for all large t 0 and all

x ∈ A D ↑ W (t, x)(-1, -v) : v ∈ F (t, x) ∩ int (N A (x) -) ∩ R = ∅,
then condition (3.9) is satisfied. Indeed, let τ > 0 be such that for all t ∈ (τ, +∞) and

x ∈ A there exists v ∈ F (t, x) ∩ int (N A (x) -) with finite D ↑ W (t, x)(-1, -v).
Then, by [RW98, Theorem 2], there exists η > 0 such that x + sw ∈ A for all w ∈ B(v, η) and s ∈ [0, η]. Now, by the definition of contingent epiderivative there exists α ∈ R and

h i → 0+, w i → v satisfying W (t -h i , x -h i w i ) -W (t, x) αh i for all i. Since x -h i w i ∈ int
A for all large i, passing to the lower limit as i → ∞ and using the lower semicontinuity of W , we get (3.9).

(v) Under the assumptions of Theorem 3.3.3 and that f and L are continuous, by [Roc81, Theorem 1], the statement (i) of Theorem 3.3.3 is equivalent to the following: for all (t, x) ∈ dom W ∩ ((0, ∞) × ∂A)

-p t + H (t, x, -p x ) 0 ∀ (p t , p x ) ∈ ∂ -W (t, x),
and for all (t, x) ∈ dom W ∩ ((0, ∞) × int A)

-p t + H (t, x, -p x ) = 0 ∀ (p t , p x ) ∈ ∂ -W (t, x).

PROOFS

Proofs

We recall first two more definitions. Let I ⊂ R + be a given interval. Consider a setvalued map Q : I R k and let y ∈ Q(s) for some s ∈ I, y ∈ R k . The contingent derivative DQ(s, y) of Q at (s, y) is the set-valued map DQ(s, y) : R R k whose graph is given by graph DQ(s, y) = T graph Q (s, y). By [AF09, Proposition 5.1.4],

DQ(s, y)(1) = v ∈ R k : lim inf h→0+ d Q(s+h) (y + hv) h = 0 . (3.10)
For a set-valued map G :

I × R k R k taking nonempty values, a locally absolutely continuous function x : I → R k is called a G-trajectory if x (t) ∈ G(t, x(t)) for a.e. t ∈ I.
Let us define the set-valued maps

G : [0, ∞) × R n R n × R, F : [0, ∞) × R n R n , and G : [0, ∞) × R n × R R n × R by G(t, x) := {(f (t, x, u), -L(t, x, u) -r) : u ∈ U (t), r ∈ [0, c(t)(1 + |x|) -L(t, x, u)]} , F (t, x) := f (t, x, U (t)) & G(t, x, v) := G(t, x).
Remarks below follow directly from the assumptions.

Remarks 3.4.1.

(i) Notice that, if (OPC) holds true, then

-F (t, x) ∩ co T A (x) = ∅ for a.e. t 0, ∀x ∈ A. (3.11) (ii) Let (t 0 , x 0 ) ∈ [0, ∞) × R n .
Then, by Gronwall's lemma and our growth assumptions, any absolutely continuous trajectory x : [t 0 , ∞) → R n solving the differential equation in (3.3) and starting from x 0 at time

t 0 satisfies 1 + |x(t)| (1 + |x 0 |) e t t 0
c(s) ds for all t t 0 . In particular, feasible trajectories starting at the same initial condition are uniformly bounded on every finite time interval. Moreover, setting for all

R > 0 γ R (t) := (1 + R) c(t)e t 0 c(s) ds ∀t 0, it follows that γ R ∈ L 1 loc ([0, ∞); R + ) and for any R > 0, any (t 0 , x 0 ) ∈ [0, ∞) × (A ∩ B(0, R)), and any feasible trajectory-control pair (x(•), u(•)) on I = [t 0 , ∞), with x(t 0 ) = x 0 , we have |f (t, x(t), u(t))| + |L(t, x(t), u(t))| γ R (t) for a.e. t t 0 .
(iii) To apply the results from [FPR95, Sections 2 and 4] we extend them to maps 3.4. PROOFS 77 with sublinear growth in the following way: letting R > 0 and T > 0, the setvalued map

G * : [0, T ] × R n+1 R n+1 defined by G * (t, X) = G(t, X) for any (t, X) ∈ [0, T ] × B(0, M ) and G * (t, X) = G(t, π(X)) for any (t, X) ∈ [0, T ] × (R n+1 \B(0, M ))
, where π(•) stands for the projection operator onto B(0, M ) and

M = R + 2 T 0 γ R (s) ds, satisfies sup v∈G * (t,X), X∈R n+1 |v| 2γ R (t) for a.e. t ∈ [0, T ]. Thus, X : [t 0 , T ] → R n+1 , with X(t 0 ) ∈ B(0, R), is a G * -trajectory if and only if it is G-trajectory on [t 0 , T ].
(iv) Since we assume that the set-valued map U (•) takes nonempty images, so are G(•) and F (•). Moreover, (OPC) implies that A is the closure of its interior. Similarly, for (IPC).

Proposition 3.4.2. Under assumption (h), for all x ∈ R n the set-valued maps F (•, x) and G(•, x) are Lebesgue measurable. Furthermore, for a.e. t 0 the set-valued maps G(t, •) and F (t, •) are continuous with closed convex images.

Proof. The first statement follows from assumption (h)-(i). Notice that, by (h)-(iii), for a.e. t 0, F (t, •) is continuous and F (t, x) is closed convex, since it is the projection of the closed set in (3.4) and the convex set in (3.5). Now, consider t 0 and x ∈

R n such that {(f (t, x, u), L(t, x, u)) : u ∈ U (t)} is closed and (h)-(ii) holds true. Let (f (t, x, u k ), -L(t, x, u k )-r k ) → (a, b) ∈ R n ×R with u k ∈ U (t) and r k ∈ [0, c(t)(1+|x|)- L(t, x, u k )] for all k. Since {L(t, x, u k )} k is bounded we deduce that {r k } k is bounded.
So, we may assume that r k → r 0. Then (f (t, x, u k ), L(t, x, u k )) → (a, -b -r), and, by closedness, there exists u ∈ U (t) such that a = f (t, x, u) and -b -r = L(t, x, u). This proves that G(t, x) is closed.

Now, let t ∈ [0, ∞) be such that x {(f (t, x, u), L(t, x, u)) : u ∈ U (t)} is contin- uous. Then x G 1 (t, x) := {(f (t, x, u), -L(t, x, u)) : u ∈ U (t)} and x G 2 (t, x) := {(f (t, x, u), -c(t)(1 + |x|) : u ∈ U (t)} are continuous. Thus x G 1 (t, x) ∪ G 2 (t, x) is continuous, and it follows that Γ : x co (G 1 (t, x) ∪ G 2 (t, x)) is continuous too (cfr. [AF09]). Since G(t, x) = Γ(x), we deduce that G(t, x) is convex and G(t, •) is continuous.
In the same way as the proof of continuity of G(t, •) in the above Proposition, we show the next result. Proposition 3.4.3. If (h) holds true, then for a.e. t 0 the set-valued map G(t, •) is Lipschitz continuous with constant k(t) + c(t).

PROOFS

The following Proposition summarizes some properties satisfied by the value function V . Proposition 3.4.4. Assume (h). Then (i) V is lower semicontinuous and for any (t, x) ∈ dom V there exists an optimal trajectory-control pair (x(•), ū(•)) at (t, x). Moreover, for any

x ∈ A lim inf s→0+, y→ A x V (s, y) = V (0, x); (3.12) (ii) there exists a set C ⊂ [0, ∞), with µ(C) = 0, such that for any (t, x) ∈ dom V ∩ (([0, ∞)\C) × A) ∃ ū ∈ U (t), D ↑ V (t, x)(1, f (t, x, ū)) -L(t, x, ū); (3.13) (iii) there exists a set C ⊂ (0, ∞), with µ(C ) = 0, such that for any (t, x) ∈ dom V ∩ (((0, ∞)\C ) × int A) ∀u ∈ U (t), D ↑ V (t, x)(-1, -f (t, x, u)) L(t, x, u); (3.14) (iv) there exists a set C ⊂ (0, ∞), with µ(C ) = 0, such that for any (t, x) ∈ dom V ∩ (((0, ∞)\C ) × int A) ∀u ∈ U (t), -L(t, x, u) D ↓ V (t, x)(1, f (t, x, u)); (3.15) (v) if (3.11
) holds true and dom V = ∅ then t epi V (t, •) is locally absolutely continuous.

Remark 3.4.5. We would like to underline that the local absolute continuity of t epi V (t, •) does not yield local absolute continuity or even continuity of V (•, x). It implies however that lim inf s→t 0 -,

x→ A x 0 V (s, x) = V (t 0 , x 0 ) for all (t 0 , x 0 ) ∈ dom V ∩((0, ∞)×A) and that lim inf s→t 0 +, x→ A x 0 V (s, x) = V (t 0 , x 0 ) for all (t 0 , x 0 ) ∈ dom V ∩ ([0, ∞) × A).
Proof of Proposition 3.4.4. The first two statements in (i) are well known. Let x ∈ A. If V (0, x) = +∞ then, since V is lower semicontinuous, (3.12) holds true. Suppose next that (0, x) ∈ dom V . Consider an optimal trajectory-control pair (x(•), ū(•)) at (0, x). Then, by the dynamic programming principle, for all s 0

V (s, x(s)) = V (0, x) - s 0 L(ξ, x(ξ), ū(ξ)) dξ.
So, lim s→0+ V (s, x(s)) = V (0, x). The lower semicontinuity of V ends the proof of (i).

To prove (ii), let j ∈ N + . From [FPR95, Corollary 2.7] applied to the set-valued map G, there exists a set C j ⊂ [0, j], with µ(C j ) = 0, such that for any (t 0 , x 0 ) ∈ 3.4. PROOFS 79 (([0, j]\C j ) × A) ∩ dom V and any optimal trajectory-control pair (x(•), ū(•)) at (t 0 , x 0 ),

∅ = Lim sup ξ→t 0 + 1 ξ -t 0 x(ξ) -x 0 , - ξ t 0 L(s, x(s), ū(s)) ds ⊂ G(t 0 , x 0 ). (3.16)
Furthermore, by the dynamic programming principle, for all t t 0

V (t, x(t)) -V (t 0 , x 0 ) = - t t 0 L(s, x(s), ū(s)) ds.
So, dividing by t -t 0 this equality, passing to the lower limit as t → t 0 +, and using (3.16), we get (3.13). Then (ii) follows setting

C = ∪ j∈N + C j .
We prove next (iii). Let j ∈ N + . From Remark 3.4.1-(iii), [FPR95, Theorem 2.9] applied to the set-valued map -G(j -•, •, •), and from the measurable selection theorem, we can find a subset C j ⊂ [1/j, j], with µ(C j ) = 0, such that for any (t 0 , x 0 ) ∈ ((1/j, j]\C j )×int A and any u 0 ∈ U (t 0 ) there exist t 1 ∈ [1/j, t 0 ) and a trajectory-control pair ((x, v), (u, r))(•) satisfying

                   (x (t), v (t)) = (f (t, x(t), u(t)), -L(t, x(t), u(t)) -r(t)) for a.e. t ∈ [t 1 , t 0 ] (x(t 0 ), v(t 0 )) = (x 0 , 0) u(t) ∈ U (t), r(t) ∈ [0, c(t)(1 + |x(t)|) -L(t, x(t), u(t))] for a.e. t ∈ [t 1 , t 0 ] (x (t 0 ), v (t 0 )) = (f (t 0 , x 0 , u 0 ), -L(t 0 , x 0 , u 0 )),
(3.17) and x([t 1 , t 0 ]) ⊂ A. Hence, if (t 0 , x 0 ) ∈ dom V , by the dynamic programming principle it follows that for all s ∈ [t 1 , t 0 ]

V (s, x(s)) -V (t 0 , x 0 ) t 0 -s 1 t 0 -s (v(s) -v(t 0 )).
Passing to the lower limit when s → t 0 -, we have that

D ↑ V (t 0 , x 0 )(-1, -f (t 0 , x 0 , u 0 )) L(t 0 , x 0 , u 0 ).
Since u 0 ∈ U (t 0 ) is arbitrary and setting C = ∪ j∈N + C j , we get (iii). Moreover, arguing in a similar way, we deduce that (iv) holds true as well. Now, assume (3.11) and that dom V = ∅. Notice that the value function V is bounded from the below and since it is lower semicontinuous, t epi V (t, •) takes closed images. Let ( t, x) ∈ dom V . Then, by the dynamic programming principle, it follows that the set-valued map t epi V (t, •) takes nonempty values on [ t, ∞). If t > 0, consider τ ∈ [0, t). From (3.11) and (3.10), it follows that -F (t, x) ∩ DP (t, x)(1) = ∅ for a.e. t ∈ (τ, t] and all x ∈ A, where P (•) ≡ A. Hence, Remark 3.4.1-(iii), the viability theorem [FPR95, Theorem 4.2] applied to the set-valued map -F ( t -•, •), and the measurable selection theorem, imply that there exists a feasible trajectory-control pair (x(•), ũ(•)) on I = [τ, t] satisfying x( t) = x. So, applying again the dynamic 3.4. PROOFS programming principle and since τ ∈ [0, t) is arbitrary, it follows that t epi V (t, •) takes nonempty values on [0, t]. Now, fix 0 t 1 t 0 . Let K ⊂ R n+1 be a nonempty compact subset, (x 1 , v 1 ) ∈ epi V (t 1 , •) ∩ K, and put R = max y∈K |y|. Consider an optimal trajectory-control pair (x(•), ū(•)) at (t 1 , x 1 ). Then

V (t 1 , x 1 ) - t 0 t 1 φ(s) ds = ∞ t 1 L(s, x(s), ū(s)) ds - t 0 t 1 φ(s) ds ∞ t 0 L(s, x(s), ū(s)) ds = V (t 0 , x(t 0 )). Since v 1 V (t 1 , x 1 ) we get (x(t 0 ), v 1 -t 0 t 1 φ(s) ds) ∈ epi V (t 0 , •). Hence we deduce that (x 1 , v 1 ) ∈ epi V (t 0 , •) + t 0 t 1 (γ R (s) + |φ(s)|) ds B.
On the other hand, let (x 0 , v 0 ) ∈ epi V (t 0 , •) ∩ K. Applying again Remark 3.4.1-(iii), the viability theorem [FPR95, Theorem 4.2], and the measurable selection theorem, we deduce that there exists a feasible trajectory-control pair (x(•), ũ(•)) on I = [t 1 , t 0 ] satisfying x(t 0 ) = x 0 . So, by the dynamic programming principle, we get

V (t 1 , x(t 1 )) V (t 0 , x 0 ) + t 0 t 1 L(s, x(s), ũ(s)) ds v 0 + t 0 t 1 γ R (s) ds, i.e., (x(t 1 ), v 0 + t 0 t 1 γ R (s) ds) ∈ epi V (t 1 , •). Finally, since (x 0 , v 0 ) = (x(t 1 ), v 0 + t 0 t 1 γ R (s) ds)+(x 0 -x(t 1 ), -t 0 t 1 γ R (s) ds), we conclude (x 0 , v 0 ) ∈ epi V (t 1 , •) + 2 t 0 t 1 γ R (s) ds B,
and so (v) follows.

The proof of the following lemma can be found in the Appendix.

Lemma 3.4.6. Assume (h) . Let W : [0, ∞) × A → R ∪ {+∞} be such that t epi W (t, •) is locally absolutely continuous. If there exists a set C ⊂ (0, ∞), with µ(C ) = 0, such that for all (t, x) ∈ dom W ∩ (((0, ∞)\C ) × int A) -p t + H(t, x, -p x , -q) 0 ∀ (p t , p x , q) ∈ T epi W (t, x, W (t, x)) -, (3.18)
then for all 0 < τ 0 < τ 1 and any feasible trajectory-control pair

(x(•), u(•)) on I = [τ 0 , τ 1 ], with x([τ 0 , τ 1 ]) ⊂ int A and (τ 1 , x(τ 1 )) ∈ dom W , the solution w(•) of      w (t) = -L(t, x(t), u(t)) for a.e. t ∈ [τ 0 , τ 1 ] w(τ 1 ) = W (τ 1 , x(τ 1 )) (3.19) satisfies (x(t), w(t)) ∈ epi W (t, •) ∀t ∈ [τ 0 , τ 1 ].
Remark 3.4.7. By the definition of local absolutely continuity, our assumption implies that epi W (t, •) is a nonempty closed set for all t 0. In particular, dom W (t, •) = ∅ 3.4. PROOFS and W (t, •) is lower semicontinuous for all t 0.

Arguing in analogous way as in the proof of Lemma 3.4.6, we have the following result involving the hypograph:

Lemma 3.4.8. Assume (h) . Let W : [0, ∞) × A → R ∪ {+∞} be such that t {(x, v) : v W (t, x) = +∞} is locally absolutely continuous. If there exists a set C ⊂ (0, ∞), with µ(C ) = 0, such that for all (t, x) ∈ dom W ∩ (((0, ∞)\C ) × int A) -p t + H(t, x, -p x , -q) 0 ∀ (p t , p x , q) ∈ T hypo W (t, x, W (t, x)) + ,
then for all 0 < τ 0 < τ 1 and any feasible trajectory-control pair

(x(•), u(•)) on I = [τ 0 , τ 1 ], with x([τ 0 , τ 1 ]) ⊂ int A and (τ 0 , x(τ 0 )) ∈ dom W , the solution w(•) of      w (t) = -L(t, x(t), u(t)) for a.e. t ∈ [τ 0 , τ 1 ] w(τ 0 ) = W (τ 0 , x(τ 0 )) (3.20) satisfies (x(t), w(t)) ∈ hypo W (t, •) ∀t ∈ [τ 0 , τ 1 ].
Proposition 3.4.9. Let W : [0, ∞) × A → R ∪ {+∞} be such that t epi W (t, •) is locally absolutely continuous.

(i) If (h)(i)-(ii) hold true and G(t, •) is upper semicontinuous, with closed convex

images, for a.e. t 0, then the following two statements are equivalent:

(a) there exists a set C ⊂ (0, ∞), with µ(C) = 0, such that for all

(t, x) ∈ dom W ∩ (((0, ∞)\C) × A) ∃ ū ∈ U (t), D ↑ W (t, x)(1, f (t, x, ū)) -L(t, x, ū); (3.21) (b) there exists a set C ⊂ (0, ∞), with µ(C ) = 0, such that for all (t, x) ∈ dom W ∩ (((0, ∞)\C ) × A) -p t + H(t, x, -p x , -q) 0 ∀ (p t , p x , q) ∈ T epi W (t, x, W (t, x)) -.
(ii) If (h) holds true, then the following two statements are equivalent:

(a) there exists a set C ⊂ (0, ∞), with µ(C) = 0, such that for all (t, x)

∈ dom W ∩ (((0, ∞)\C) × int A) ∀u ∈ U (t), D ↑ W (t, x)(-1, -f (t, x, u)) L(t, x, u); (3.22) 3.4. PROOFS time interval [t 0 + j, t 0 + j + 1], we can find a G-trajectory X(•) = (x(•), v(•)) satisfying                    chapter 3 X (t) ∈ G(t, X(t)) for a.e. t ∈ [t 0 + j, t 0 + j + 1] x([t 0 + j, t 0 + j + 1]) ⊂ A x(t 0 + j) = x j (t 0 + j), v(t 0 + j) = v j (t 0 + j) v(t) W (t, x(t)) ∀t ∈ [t 0 + j, t 0 + j + 1]. Putting X j+1 (t) = (x j (t), v j (t)) if t ∈ [t 0 , t 0 + j] and X j+1 (t) = (x(t), v(t)) if t ∈ (t 0 + j, t 0 + j + 1],
we deduce that X j+1 (•) satisfies our claim. Now, consider the Gtrajectory X(t) = (x(t), v(t)) given by

X(t) = X j (t) if t ∈ [t 0 + j, t 0 + j + 1].
By the measurable selection theorem, there exist two measurable functions u(•) and r(•), with u(t) ∈ U (t) and r(t)

∈ [0, c(t)(1 + |x(t)|) -L(t, x(t), u(t))] for a.e. t t 0 , such that v(t) = W (t 0 , x 0 ) -t t 0 L(s, x(s), u(s)) ds -t t 0 r(s) ds for all t t 0 . Then W (t 0 , x 0 ) W (t, x(t)) + t t 0 L(s, x(s), u(s)) ds ∀ t t 0 . (3.27) Thus (t, x(t)) ∈ dom W for all t t 0 . Since L(t, •, •) φ(t)
for a.e. t 0, where φ ∈ L 1 ([0, ∞); R), it follows that the limit lim t→∞ t t 0 L(s, x(s), u(s)) ds exists. So, using (3.8) and passing to the limit in (3.27) as t → ∞ yields W (t 0 , x 0 ) ∞ t 0 L(s, x(s), u(s)) ds. Therefore W (t 0 , x 0 ) V (t 0 , x 0 ). Consequently W ≥ V .

We show next that W (t 0 , x 0 ) V (t 0 , x 0 ) for all (t 0 , x 0 ) ∈ [0, ∞) × A. If V (t 0 , x 0 ) = +∞, then V (t 0 , x 0 ) W (t 0 , x 0 ). So, let us assume that (t 0 , x 0 ) ∈ dom V . Fix ε > 0. By our assumptions, there exists T > t 0 such that domV (t, •) ⊂ domW (t, •) for all t T and sup y∈dom W (t,•)

|W (t, y)| ε ∀ t T .
(3.28)

Let (x(•), ū(•)) be an optimal trajectory-control pair at (t 0 , x 0 ) and consider

s i ↑ +∞ with {s i } i ⊂ (T , ∞). Put X(•) = (x(•), z(•)) where z(t) = -t t 0 L(s, x(s), ū(s)) ds. For all (t, x, w) ∈ [0, ∞) × R n × R define Q(t, x, w) := {(f (t, x, u), L(t, x, u)) : u ∈ U (t)} .
Applying [BF, Theorem 2] we deduce that for any i there exists a Q-trajectory X

i (•) = (x i (•), z i (•)) solving            chapter 3 X i (t) ∈ Q(t, X i (t)) for a.e. t ∈ [t 0 , s i ] X i (s i ) = (x(s i ), z(s i )) x i (t) ∈ int A ∀ t ∈ [t 0 , s i ) 3.4. PROOFS 85 and lim i→∞ X i -X ∞,[t 0 ,s i ] = 0.
Hence, by the measurable selection theorem, for any i there exists a measurable selection

u i (t) ∈ U (t) such that (x i (•), u i (•)) satisfies            chapter 3 x i (t) = f (t, x i (t), u i (t)) for a.e. t ∈ [t 0 , s i ] x i (s i ) = x(s i ) x i (t) ∈ int A ∀ t ∈ [t 0 , s i ), lim i→∞ x i (t 0 ) = x(t 0 ), (3.29)
and

lim i→∞ s i t 0 L(s, x i (s), u i (s)) ds = ∞ t 0 L(s, x(s), ū(s)) ds. (3.30)
Now, fix i ∈ N + and consider {τ j } j ⊂ (T , s i ) with τ j → s i . Note that, by the dynamic programming principle, x i (τ j ) ∈ dom V (τ j , •) for all j. Consider the solution w j (•) of the Cauchy problem

     chapter 3 w (t) = -L(t, x i (t), u i (t)) for a.e. t ∈ [t 0 , τ j ] w(τ j ) = W (τ j , x i (τ j )).
From Lemma 3.4.6, we conclude that

τ j t 0 L(s, x i (s), u i (s)) ds + W (τ j , x i (τ j )) W (t 0 , x i (t 0 )) ∀j.
Hence, by (3.28),

τ j t 0 L(s, x i (s), u i (s)) ds + ε W (t 0 , x i (t 0 )) ∀j,
and taking the limit as j → ∞ we get s i t 0 L(s, x i (s), u i (s)) ds+ε W (t 0 , x i (t 0 )). Passing now to the lower limit as i → ∞, using (3.29), (3.30), and the lower semicontinuity of W , we have

∞ t 0 L(s, x(s), ū(s)) ds + ε W (t 0 , x 0 ), i.e., V (t 0 , x 0 ) + ε W (t 0 , x 0 ). Since ε is arbitrary, we conclude that V (t 0 , x 0 ) W (t 0 , x 0 ). Hence V = W on (0, ∞) × A. Since t epi W (t,
•) is locally absolutely continuous and W is lower semicontinuous, lim inf s→0+, y→ A x W (s, y) = W (0, x) for all x ∈ A. So, fix x 0 ∈ A. From (3.12) and what precede, we have

W (0, x 0 ) = lim inf s→0+, y→ A x 0 W (s, y) = lim inf s→0+, y→ A x 0 V (s, y) = V (0, x 0 ).
Now, assume in addition (B). Let t ∈ [0, ∞) be such that dom V ( t, •) = ∅. By (OPC) this implies that dom V (t, •) = ∅ for all t ∈ [0, t]. Moreover, by the dynamic 3.5. LIPSCHITZ CONTINUOUS SOLUTIONS programming principle, it follows that dom V (s, •) = ∅ for all s t. Hence,

|V (s, y)| ∞ s ψ(ξ) dξ ∀y ∈ dom V (s, •), ∀ s T.
So, we deduce that V satisfies (3.8).

Lipschitz continuous solutions

In [BF] we provided sufficient conditions for the local Lipschitz continuity of the value function under state constraints. Before stating an existence and uniqueness result for Lipschitz continuous solutions (in the Crandall-Lions sense) of H-J-B equation, we show a geometric result (in the spirit of Section 3) involving the hypographs of functions. Proposition 3.5.1. Under all the assumptions of Theorem 3.3.3 suppose that the setvalued map

t {(x, v) ∈ A × R : v W (t, x) = +∞} (3.31)
is locally absolutely continuous.

Then the following statements are equivalent:

(i) W = V ;
(ii) there exists a set C ⊂ (0, ∞), with µ(C ) = 0, such that for all (t, x) ∈ dom W ∩ (((0, ∞)\C ) × A)

-p t + H(t, x, -p x , -q) 0 ∀ (p t , p x , q) ∈ T epi W (t, x, W (t, x)) -, for all (t, x) ∈ dom W ∩ (((0, ∞)\C ) × int A) -p t + H(t, x, -p x , -q) 0 ∀ (p t , p x , q) ∈ T hypo W (t, x, W (t, x)) + ,
and t epi W (t, •) is locally absolutely continuous.

Proof. Notice first of all that by the definition of locally absolutely continuous set-valued map, the hypograph of W (t, •) restricted to dom W (t, •) is closed. Assume (i). From Proposition 3.4.4-(iv), we can find a subset C ⊂ (0, ∞), with µ(C) = 0, such that for any (t 0 , x 0 ) ∈ ((0, ∞)\C) × int A we have -L(t 0 , x 0 , u 0 ) D ↓ V (t 0 , x 0 )(1, f (t 0 , x 0 , u 0 )) for all u 0 ∈ U (t 0 ), i.e., recalling (3.1),

(1, f (t 0 , x 0 , u 0 ), -L(t 0 , x 0 , u 0 )) ∈ T hypo V (t 0 , x 0 , V (t 0 , x 0 )) ∀ u 0 ∈ U (t 0 ). So, -p t + H(t, x, -p x , -q) 0 ∀ (p t , p x , q) ∈ T hypo V (t, x, V (t, x)) + .

The first inequality in (ii) follows from Theorem 3.3.3. Now assume (ii). By Theorem 3.3.3 and the proof of (ii) =⇒ (i) of Theorem 3.3.3, it is just sufficient to show (3.25). Arguing as in the proof of Proposition 3.4.4-(iii), there exists C ⊂ (0, ∞), with µ(C ) = 0, such that for any (t 0 , x 0 ) ∈ ((0, ∞)\C )×int A and u 0 ∈ U (t 0 ), we can find t 1 ∈ (0, t 0 ) and a trajectory-control pair ((x, v), (u, r))(•) satisfying (3.17) and x([t 1 , t 0 ]) ⊂ int A. By Lemma 3.4.8, taking {s i } i ⊂ (t 1 , t 0 ) with s i → t 0 -, we get that for all i the solution w i (•) of

     chapter 3 w (t) = -L(t, x(t), u(t)) for a.e. t ∈ [s i , t 0 ] w(s i ) = W (s i , x(s i )), satisfies w i (t 0 ) = W (s i , x(s i ))-t 0 s i L(s, x(s), u(s)) ds W (t 0 , x(t 0 )). Hence W (s i , x(s i ))- W (t 0 , x 0 ) t 0
s i L(s, x(s), u(s)) ds v(s i ) for all i. Dividing by t 0 -s i and passing to the lower limit as i → ∞, we have the conclusion.

Remark 3.5.2. Assuming further that f , L, and W : [0, ∞) × A → R are continuous functions, then, using the same arguments as in the proofs of [Fra93, Theorem 4.3 and Lemma 4.3], the assumption (3.31) in Proposition 3.5.1 can be skipped and (i) is equivalent to the following:

     chapter 3 -p t + H (t, x, -p x ) 0 ∀(t, x) ∈ (0, ∞) × A, ∀ (p t , p x ) ∈ ∂ -W (t, x) -p t + H (t, x, -p x ) 0 ∀ (t, x) ∈ (0, ∞) × int A, ∀ (p t , p x ) ∈ ∂ + W (t, x).
From Theorem 3.3.3 and Proposition 3.5.1 we get immediately the following three corollaries.

Corollary 3.5.3. Assume (h) and (OPC). Let W : [0, ∞) × A → R ∪ {+∞} be a lower semicontinuous function such that dom V (t, •) ⊂ dom W (t, •) = ∅ for all large t > 0 and (3.8) holds true. Suppose that

µ{ t ∈ [0, ∞) : ∃ x ∈ A, (t, x) ∈ dom W, {0} = T epi W (t, x, W (t, x)) -⊂ R × R n × {0} } = 0.
Then the following statements are equivalent:

(i) W = V ;
(ii) there exists a set C ⊂ (0, ∞), with µ(C ) = 0, satisfying for all (t, x) ∈ dom W ∩ (((0, ∞)\C ) × ∂A)

-p t + H (t, x, -p x ) 0 ∀ (p t , p x ) ∈ ∂ -W (t, x), for all (t, x) ∈ dom W ∩ (((0, ∞)\C ) × int A) -p t + H (t, x, -p x ) = 0 ∀ (p t , p x ) ∈ ∂ -W (t,
µ{ t ∈ [0, ∞) : ∃x ∈ A, (t, x) ∈ dom W, {0} = T hypo W (t, x, W (t, x)) + ⊂ R × R n × {0} } = 0.
Then the following statements are equivalent:

(i) W = V ;
(ii) there exists a set C ⊂ (0, ∞), with µ(C ) = 0, satisfying for all (t, x) ∈ dom W ∩ (((0, ∞)\C ) × A) Then it is well known that

-p t + H (t, x, -p x ) 0 ∀ (p t , p x ) ∈ ∂ -W (t, x), for all (t, x) ∈ dom W ∩ (((0, ∞)\C ) × int A) -p t + H (t, x, -p x ) 0 ∀ (p t , p x ) ∈ ∂ + W (t, x),
     chapter 3 0 = (p t , p x , q) ∈ T epi W (t, x, W (t, x)) -=⇒ q = 0 0 = (p t , p x , q) ∈ T hypo W (t, x, W (t, x)) + =⇒ q = 0. and if ∂ -W (t, x) = ∅, then T epi W (t, x, W (t, x)) -= ∪ λ 0 λ(∂ -W (t, x), -1). Similarly, if ∂ + W (t, x) = ∅, then T hypo W (t, x, W (t, x)) + = ∪ λ 0 λ(∂ + W (t, x), -1).
From Corollary 3.5.4 and Remark 3.5.5, we deduce the following:

Corollary 3.5.6. Assume (h) and (OPC). Let W : [0, ∞) × A → R be a locally Lipschitz continuous function satisfying (3.8). Then the following statements are equivalent:

(i) W = V ;
(ii) there exists a set C ⊂ (0, ∞), with µ(C ) = 0, satisfying for all (t, x) ∈ dom W ∩ (((0, ∞)\C ) × A)

-p t + H (t, x, -p x ) 0 ∀ (p t , p x ) ∈ ∂ -W (t, x), for all (t, x) ∈ dom W ∩ (((0, ∞)\C ) × int A) -p t + H (t, x, -p x ) 0 ∀ (p t , p x ) ∈ ∂ + W (t, x).

LIPSCHITZ CONTINUOUS SOLUTIONS 89

Now, let l : [0, ∞) × R n × R m → [0, ∞) be a bounded measurable function, λ > 0, and L(t, x, u) = e -λt l(t, x, u).

(3.32) Proposition 3.5.7. Assume (3.32), (h) , and (IPC).

Then, there exists λ > 0 such that for all λ λ the value function V is the unique locally Lipschitz continuous function on [0, ∞) × A satisfying Proof. From [BF, Theorem 4] and the proof of [BF, Corollary 1] it follows that there exists λ > 0 such that for all λ λ the value function V is locally Lipschitz continuous on [0, ∞) × A. Moreover, arguing as in the proofs (i) =⇒ (ii) of Theorem 3.3.3 and Proposition 3.5.1, and from Remarks 3.3.1-(i) and 3.5.5, we deduce that V satisfies (3.33). Now, let W : [0, ∞) × A → R be a locally Lipschitz continuous function satisfying (3.33). From the proof (ii) =⇒ (i) of Theorem 3.3.3 it follows that W V on (0, ∞)×A. Let (t 0 , x 0 ) ∈ (0, ∞) × A, (x(•), ū(•)) be optimal at (t 0 , x 0 ), and ε > 0, T > t 0 such that (3.28) holds true. Consider s i ↑ +∞ with {s i } i ⊂ (T , ∞). Fix i ∈ N + and let {τ j } j ⊂ (t 0 , s 0 ) and {y j } j ⊂ int A be such that τ j → t 0 and y j → x 0 . Repeating the same arguments as in the proof of the implication (ii) =⇒ (i) of Theorem 3.3.3 and using [BF, Theorem 2], we show that for all j there exists a measurable selection w(τ j ) = W (τ j , y j ).

           -p t + H (t, x, -p x ) 0 ∀ (p t , p x ) ∈ ∂ -V (t, x), for a.e. t > 0, ∀x ∈ A -p t + H (t, x, -p x ) 0 ∀ (p t , p x ) ∈ ∂ + V (t, x),
u j (•) ∈ U (•) on [τ j , s i ] such that (x j (•), u j (•)) satisfies            chapter 3 x j (t) = f (t, x j (t), u j (t)) for a.e. t ∈ [τ j , s i ] x j (τ j ) = y j x j (t) ∈ int A ∀ t ∈ [τ j , s i ], lim j→∞ x j -x ∞,[τ j ,s i ] = 0, ( 3

Introduction

Regularity properties of the value function of optimal control problems with finite horizon, in the absence of state constraints, have been widely investigated. For the Mayer and Bolza problems it can be shown that the value function is continuous, Lipschitz continuous, or semiconcave in line with the problem data (see [CF91, CF06, CF13, CF14, CFS15, CS04]). Even for optimal exit time problems, regularity results are available under suitable controllability assumptions (see [START_REF] Cannarsa | Semiconcavity for optimal control problems with exit time[END_REF][START_REF] Cannarsa | Convexity properties of the minimum time function[END_REF][START_REF] Cannarsa | On a class of nonlinear time optimal control problems[END_REF][START_REF] Cannarsa | Semiconcave functions, Hamilton-Jacobi equations, and optimal control[END_REF]). More precisely, let Γ be a compact subset of R n and consider the following time minimization

4.1. INTRODUCTION problem                    chapter 4 minimize θ Γ (x(•), u(•))
over all trajectory-control pairs (x, u)(•) satisfying

x (s) = f (x(s), u(s)) for a.e. s 0, x(0

) = x 0 u ∈ L 2 (R + ; R m ),
where f : R n × R m → R n is a given function, x 0 ∈ R m , and θ Γ (x(•), u(•)) := inf{s 0 | x x 0 ,u (s) ∈ Γ} is the so-called transfer time (to Γ) along the trajectory x(•) starting from x 0 and associated with the control u(•). For any x 0 ∈ R n and any control u(•) we denote by x x 0 ,u (•) the solution of the Cauchy problem x (s) = f (x(s), u(s)) for a.e. s 0, x(0) = x 0 . By convention θ Γ (x x 0 ,u (•), u(•)) = +∞ if x x 0 ,u (s) / ∈ Γ for all s 0. The set Γ is called the target set and the value function

τ Γ (x 0 ) = inf {θ Γ (x x 0 ,u (•), u(•)) | u ∈ L 2 (R + ; R m )} is the minimum time function. It is well known (see [CS04, Chapter 8]) that τ Γ is locally Lipschitz continuous on the set A = {x 0 ∈ R n | ∃ u ∈ L 2 (R + ; R m ), θ Γ (x x 0 ,u (•), u(•)) < ∞}
provided that Γ satisfies Petrov's condition: there exists r > 0 such that for any y ∈ ∂Γ and any proximal unit vector ν to Γ at y we can find u ∈ R m satisfying f (y, u), ν < -r. In addition, if the target set fulfils the uniform inner ball property, then τ Γ (•) is locally semiconcave on A \Γ and it is locally Lipschitz continuous on A \Γ if and only if Γ satisfies Petrov's condition.

Recovering the local semiconcavity property for the minimum time function, associated with the above problem, when the target set does not satisfy the uniform inner ball property, becomes quite challenging. Indeed, let us suppose that f (x, u) = m i=1 u i f i (x) and u(•) takes values in the m-dimensional closed unit ball, with f 1 , ..., f m smooth (C ∞ or C ω ) vector fields on R n and 1 m n. Then Petrov's condition may be not satisfied and, if Γ is a singleton, the uniform ball property fails. Nevertheless, the minimum time to reach a point is equal to the sub-Riemannian distance d SR from such a point associated with the distribution ∆ = span {f 1 , ..., f m } on the manifold M = R n (see [START_REF] Bellaıche | Sub-Riemannian geometry[END_REF][START_REF] Jerison | Subelliptic, second order differential operators[END_REF][START_REF] Montgomery | A tour of subriemannian geometries, their geodesics and applications[END_REF]). Regularity properties of d SR were obtained for subanalytic structures (see [START_REF] Agrachev | Compactness for sub-riemannian length-minimizers and subanalyticity, in: Control theory and its applications (Grado[END_REF][START_REF] Trélat | Some properties of the value function and its level sets for affine control systems with quadratic cost[END_REF], and Section 4). In particular, if the Lie algebra generated by ∆ is regular everywhere, i.e., it satisfies Hörmander's condition (see [START_REF] Hörmander | Hypoelliptic second order differential equations[END_REF] and Section 4), then for any x 0 there exists a dense subset S x 0 of R n such that for all y ∈ S x 0 the function d SR (x 0 , •) is Lipschitz continuous on a suitable open neighborhood of y (see [START_REF] Rifford | Sub-Riemannian geometry and optimal transport[END_REF]Chapter 2]). One can show (see [START_REF] Cannarsa | Semiconcavity results for optimal control problems admitting no singular minimizing controls[END_REF]), assuming furthermore that any geodesics associated with ∆ connecting x to x 0 = x is not singular (see Section 2 for the definition), that the function d SR (x 0 , •) is locally semiconcave on R n \ {x 0 }. So, under such assumptions, it follows that for any compact set Λ ⊂ R n and any y ∈ R n \Λ the function d SR (y, •) is C(y)-semiconcave on Λ. Such a property does not suffice to guarantee the local semiconcavity of d SR (Γ, •) = inf y∈Γ d SR (y, •) on R n \Γ, because the semiconcavity constant C(y) might blow up with y ∈ Γ. Nevertheless, in this chapter we analyze the local semiconcavity property of the function inf y∈Γ d SR (y, •) obtaining uniform bounds on the semiconcavity constant C(y) as y lies in a compact set (see Section 4). More precisely, we will show that for any compact set Γ ⊂ R n \Λ there exists a nonnegative constant C = C(Λ, Γ) such that d SR (x 0 , •) is C-semiconcave on Λ for every x 0 ∈ Γ.

In order to obtain the semiconcavity results we assume that there are no singular geodesics and we study the dependence of the semiconcavity constant with respect to the initial point, showing that it is bounded from the above when x 0 lies in a compact set. As it was the case in [START_REF] Cannarsa | Semiconcavity results for optimal control problems admitting no singular minimizing controls[END_REF], a key point of the reasoning is to show the local invertibility of the end-point map (x 0 , u) → x x 0 ,u (T ), where T > 0, and to prove the C 1,1 regularity of its inverse function (Proposition 4.4.5). Then, we use a compactness result ensuring that all optimal controls are uniformly Lipschitz continuous and uniformly bounded. The final step consists in combining the local semiconcavity property of the cost functional with the C 1,1 regularity of the inverse of end-point map.

After establishing semiconcavity, we address sensitivity relations and transversality conditions for the minimum time function associated with an affine control system as above. Such relations are given in the form of the following inclusions

     chapter 4 -p(t) ∈ ∂ P τ Γ (x x 0 ,ū (t)) ∀ t ∈ [0, τ Γ (x 0 )) p(τ Γ (x 0 )) ∈ Lim sup t → τ Γ (x 0 )-N P R n \Γt (x x 0 ,ū (t))
, where x 0 ∈ R n \Γ, ū(•) is an optimal control for τ Γ at x 0 , Γ t = {y ∈ R n | τ Γ (y) t}, and p(•) solves the adjoint equation -p (t) = d x f (x x 0 ,ū (t), ū(t)) * p(t) for a.e. t ∈ [0, τ Γ (x 0 )]. Sensitivity relations for the minimum time function to reach a set with the inner ball property were already investigated (see [START_REF] Cannarsa | Interior sphere property of attainable sets and time optimal control problems[END_REF][START_REF] Cannarsa | Optimality conditions and regularity results for time optimal control problems with differential inclusions[END_REF][START_REF] Colombo | On the structure of the minimum time function[END_REF]). We recover, for time optimal control problems, sensitivity relations for the co-state in terms of proximal normal cones (see [START_REF] Vinter | Optimal control[END_REF][START_REF] Cannarsa | Optimality conditions and regularity results for time optimal control problems with differential inclusions[END_REF] and Section 2). This is done under the assumption that there are no singular geodesics associated with ∆ and the target set is merely compact. The analysis, that applies to any compact target, is based on the dynamic programming principle and further properties of viscosity solutions of the eikonal equation |F (x) * ∇τ Γ (x)| -1 = 0 for x ∈ Γ c , where F (x) is the matrix which has f 1 (x), ..., f m (x) as column vectors.

The outline of this chapter is as follows. Section 2 recalls some basic notations and results from nonsmooth analysis and control theory. In Section 3, we state our main results. We give their proof in Section 4. Finally, in Section 5, we derive sensitivity relations for the minimum time function.

Preliminaries

Let (X, | • | X ) be a normed space. We denote by B X (z, r) the open ball centered at z with radius r > 0 in X (we write B r (z) in place of B R n (z, r) when no confusion arises) and we set S 1 = ∂B X (0, 1). For a subset C ⊂ X we write int C, C, and C c for the interior, the closure, and the complement of C, respectively. We denote by | • | and •, • the Euclidean norm and the scalar product in R n , respectively. Let A ⊂ X be a nonempty subset. The distance from x to A is defined by d(x, A) = inf {|x -y| X | y ∈ A}. A function ϕ : A ⊂ X → R is said to be C-semiconcave (with linear modulus) on A, with C 0, if it is continuous and

ϕ(x + h) + ϕ(x -h) -2ϕ(x) C |h| 2 ∀ x, h ∈ X, [x -h, x + h] ⊂ A.
We say that ϕ is locally semiconcave on A if for any compact subset K ⊂ A there exists

C K 0 such that ϕ is C K -semiconcave on K. If A is open, we say that ϕ ∈ C 1,1 or ϕ ∈ C 1,1
loc if ϕ is continuously differentiable with Lipschitz continuous or locally Lipschitz continuous differential on A, respectively. It is well known that any ϕ ∈ C 1,1 loc is locally semiconcave. We say that φ : X → X has a sub-linear growth if there exists M 0 such that |φ(x)| X M (1 + |x| X ) for all x ∈ X.

For p ∈ N + we denote by L p (0, T ; R n ) the set of all Lebesgue measurable functions g : [0, T ] → R n such that g p L p := T 0 |g(s)| p ds < ∞, by C(0, T ; R n ) the space of all R n -valued continuous functions on [0, T ], and by C p (0, T ; R n ) the space of R n -valued functions on [0, T ], p-times continuously differentiable.

Let D ⊂ R n be nonempty and {A h } h∈D be a family of nonempty subsets of R n . The upper limit (in the Kuratowski-Painlevé sense) of

A h at h 0 ∈ D, written Lim sup h→ D h 0 A h , is the set of all vectors v ∈ R n such that lim inf h→ D h 0 d A h (v) = 0. If D = N + , then Lim sup i→∞ S(i) := Lim sup y→0 A G(y) where A = {1/i} i∈N + and G(1/i) := S(i).
Let E be a closed subset of R n and x ∈ E. We denote by E -the negative polar of the set E, i.e. the set {y ∈ R n | y, x 0 ∀x ∈ E}. The proximal normal cone to E at x is the set defined by

N P E (x) = p ∈ R n | ∃σ = σ(x, p) 0 : p, y -x σ |y -x| 2 ∀y ∈ E .
Furthermore, p ∈ N P E (x) if and only if there exists λ > 0 such that B r|p| (x + rp) ⊂ E c for all 0 r λ (see [START_REF] Vinter | Optimal control[END_REF]).

The contingent cone to E at x is the set defined by

T E (x) = {v ∈ R n | ∃ t i → 0+, ∃ v i → v, x + t i v i ∈ E ∀i}.
It is known that N P E (ξ) ⊂ T E (ξ) -for all ξ ∈ ∂E. Let ϕ be a real valued function on E. The superdifferential D + ϕ(x) of ϕ at x ∈ E where D * ϕ(x) := {ξ ∈ R n | ∃x i → x, ∇ϕ(x i ) → ξ} and "co" stands for the convex hull. The proximal and horizontal proximal supergradient of ϕ at x are the sets defined, respectively, by

∂ P ϕ(x) = ξ ∈ R n | (-ξ, 1) ∈ N P hypo ϕ (x, ϕ(x)) , ∂ ∞,P ϕ(x) = ξ ∈ R n | (-ξ, 0) ∈ N P hypo ϕ (x, ϕ(x)) ,
where hypo ϕ denotes the hypograph of the function ϕ. For further properties of superdifferentials and proximal cones we refer to [START_REF] Clarke | Optimization and nonsmooth analysis[END_REF][START_REF] Vinter | Optimal control[END_REF].

Main Result

Let 1 m n be two natural numbers. Consider the optimal control problem minimize t 0 L(x x 0 ,u (s), u(s)) ds (4.2) over all controls u ∈ L 2 (0, t; R m ) such that the solution x x 0 ,u (•) of the affine control system

     x (s) = m i=1 u i (s) f i (x(s)) for a.e. s ∈ [0, t] x(0) = x 0 (4.3)
satisfies the end-point constraint

x x 0 ,u (t) = y, (4.4)
where (t, y) ∈ [0, ∞) × R n and f i : R n → R n , L : R n × R m → R are given functions. We say that a control v ∈ L 2 (0, t; R m ) steers x 0 to y in time t if x x 0 ,v (t) = y. The infimum of the cost functional in (4.2) over all controls steering x 0 to y in time t is denoted by V x 0 (t, y) (if there are no controls steering x 0 to y in time t, we set V x 0 (t, y) = +∞). The function V x 0 : [0, ∞) × R n → R ∪ {±∞} is called the value function of the problem (4.2)-(4.4) with starting point x 0 . A control v ∈ L 2 (0, t; R m ) is said to be an optimal control or a minimizer (for the problem (4.2)-(4.4)) at (x 0 , t, y) if x x 0 ,v (t) = y and V x 0 (t, y) = t 0 L(x x 0 ,v (s), v(s)) ds. We denote by U x 0 (s, y) the (possibly empty) set of all optimal controls steering x 0 to y in time s.

MAIN RESULT

Let us denote by (H) the following assumptions: (H) (i) f 1 , ..., f m are C 2 vector fields on R n with sub-linear growth and Lipschitz continuous differential; (ii) L ∈ C 2 and ∇ 2 u L(x, u) > 0 for all (x, u) ∈ R n × R m ; (iii) G is a given nonempty compact subset of R n and the following set is nonemepty

D G := {(t, x) ∈ [0, ∞) × R n | V x 0 (t, x) < +∞ ∀x 0 ∈ G} ; (iv) there exists a nonempty open subset Ω G ⊂ [0, ∞) × R n such that Ω G ⊂ D G ; (v) there exist c 0 and a function φ : [0, ∞) → R + such that lim inf r→∞ φ(r)/r 2 > 0 & L(x, u) φ(|u|) -c ∀(x, u) ∈ R n × R m ,
and for any r > 0 Let T > 0 and W ⊂ L 2 (0, T ; R m ) be such that all solutions of (4.3), with u ∈ W , are well defined on [0, T ]. The end-point map associated to the system (4.3) at time T , written E T , is the function given by

sup |∇ x L(x, u)| φ(|u|) | x ∈ B r (0), u ∈ R m < ∞.
E T (x 0 , u) = x x 0 ,u (T ) ∀ (x 0 , u) ∈ R n × W .
It can be proved that, if the vector fields f 1 , ..., f m are smooth, then W can be chosen to be open (see [START_REF] Bellaıche | Sub-Riemannian geometry[END_REF][START_REF] Montgomery | A tour of subriemannian geometries, their geodesics and applications[END_REF]).

Definition 4.3.2. A control u ∈ L 2 (0, t; R m ) is said to be singular at x 0 if dE t (x 0 , u)(0, •) is not a surjective map on L 2 (0, t; R m ). Definition
= C (G, Γ) 0 such that the value function V x 0 (•, •) is C-semiconcave on Γ for all x 0 ∈ G.
Now, let us denote by (H) the following assumptions on f 1 , ..., f m : (H) (i) f 1 , ..., f m are smooth vector fields (C ∞ or C ω ) satisfying Hörmander's condition, i.e., span X i (x)

i 1 = R n ∀ x ∈ R n , where X 1 (x) = {f 1 (x), ..., f m (x)}, X i+1 (x) = X i (x)∪{[f, g](x) | f ∈ X 1 (x), g ∈ X i (x)} for all i ∈ N + ([•,
•] denotes the Lie bracket); (ii) f 1 , ..., f m have sub-linear growth, Lipschitz continuous differential, and f 1 (x), ..., f m (x) are linearly independent for all x ∈ R n .

If (H) -(i) holds true, by the Chow-Rashevsky theorem (see [START_REF] Chow | Über systeme von liearren partiellen differentialgleichungen erster ordnung[END_REF][START_REF] Rashevsky | About connecting two points of a completely nonholonomic space by admissible curve[END_REF]), for any x 0 , y ∈ R n there exists an absolutely continuous arc x : [0, 1] → R n , with square integrable derivative, such that x(0) = x 0 , x(1) = y, and

x (t) ∈ span {f 1 (x(t)), ..., f m (x(t))} for a.e. t ∈ [0, 1]. (4.5)
An absolutely continuous arc on [0, 1] satisfying (4.5), with square integrable derivative, is said to be an horizontal arc.

Let us denote by S (x 0 , x) the set of all horizontal arcs β such that β(0) = x 0 and β(1) = x. Then, if (H) holds true, there exists a bijection between S (x 0 , x) and L 2 (0, 1; R m ) such that for any β ∈ S (x 0 , x) there exists a unique u β ∈ L 2 (0, 1; R m ) satisfying β (s) = m i=1 (u β (s)) i f i (β(s)) for a.e. s ∈ [0, 1]. We can associate to any horizontal arc [0, 1] t → β(t) its length given by l(β) = 1 0 |u β (t)| dt, and the sub-Riemannian distance between x 0 and x, written d SR (x 0 , x), is inf {l(β) | β ∈ S (x 0 , x)}. The following result is very useful (see [START_REF] Bellaıche | Sub-Riemannian geometry[END_REF]):

Proposition 4.3.5. Assume (H) -(i). For any x 0 , x ∈ R n d SR (x 0 , x) 2 = e(x 0 , x) := inf 1 0 |u β (t)| 2 dt | β ∈ S (x 0 , x) .
The function e(•, •) is said to be the sub-Riemannian energy, and an horizontal arc minimizing e(x 0 , x) is said to be a geodesic steering x 0 to x. A geodesic β is called a singular geodesic (or singular) if the associated control u β is singular.

Consider the following minimization problem

E x 0 (t, x) := inf t 0 |u(s)| 2 ds | u ∈ L 2 (0, t; R m ), x x 0 ,u (t) = x , where (t, x) ∈ [0, ∞) × R n and x 0 ∈ R n .
(x) = inf u∈Z u(x). If w(x) = -∞ for all x ∈ Γ then w(•) is C-semiconcave on Γ.
For any compact set Γ ⊂ R n , the sub-Riemannian distance between Γ and x is

d SR (Γ, x) = inf x 0 ∈Γ d SR (x 0 , x

Proof of the Main Result

We provide here the proof of our main result deferring technical details to the appendix. For any k > 0, T > 0, x 0 ∈ R n , and Γ ⊂ [0, ∞)×R n , we introduce the following notation

U x 0 (Γ) = (s,y)∈Γ U x 0 (s, y), L T k = {u : [0, T ] → R m | u ∞ k and u is k-Lipschitz continuous} .
We equip the set L T k with the uniform norm. Lemma 4.4.1. Assume (H) and let T > 0. Then, for any k 0 there exists K 0 such that for any z ∈ G the map

[0, T ] × L T k (t, u) → t 0 L(x z,u (s), u(s)) ds (4.6)
is K-Lipschitz continuous and K-semiconcave.

Proof. By Remark 4.6.1 from the Appendix, there exists r = r(k , G) > 0 such that x z,u ∞ r for all z ∈ G and all u ∈ L T k . Consider α > 0, M > 0, depending on k and G, such that for all x, y ∈ B r (0) and all u, w ∈ B k (0)

|L(x, u) -L(y, w)| α (|x -y| + |u -w|) & |L(x, u)| M , ( 4.7)
and for all x, η with [x + η, x -η] ⊂ B 3r (0) and every u ∈ B k (0)

L(x + η, u) + L(x -η, u) -2L(x, u) α |η| 2 . (4.8)
Fix z ∈ G. Denote for simplicity the map in (4.6) by C t (u). Then, from Lemma 4.6.2 and (4.7), there exists σ = σ(k , G) > 1 such that for any 0 s t T , and any

u, w ∈ L T k |C t (u) -C s (w)| t s |L(x z,w (ξ), w(ξ))| dξ + s 0 |L(x z,u (ξ), u(ξ)) -L(x z,w (ξ), w(ξ))| dξ M |t -s| + α s 0 (|x z,u (ξ) -x z,w (ξ)| + |u(ξ) -w(ξ)|) dξ M + ασT (|t -s| + u -w ∞ ) . Now, let t, h, u, and v be such that [t -h, t + h] ⊂ [0, T ] and u -v, u + v ∈ L T k . We have C t+h (u + v) + C t-h (u -v) -2C t (u) = C t (u + v) + C t (u -v) -2C t (u) + C t+h (u + v) + C t-h (u -v) -C t (u + v) -C t (u -v).
(4.9) Then,

C t (u + v) + C t (u -v) -2C t (u) = t 0 (L(x z,u+v (s), u(s) + v(s)) + L(x z,u-v (s), u(s) -v(s)) -2L(x z,u (s), u(s))) ds = t 0 (L(x z,u+v (s), u(s)) + L(2x z,u (s) -x z,u+v (s), u(s)) -2L(x z,u (s), u(s))) ds + t 0 (L(x z,u+v (s), u(s) + v(s)) -L(x z,u+v (s), u(s))) ds + t 0 (L(x z,u-v (s), u(s) -v(s)) -L(x z,u-v (s), u(s))) ds + t 0 (L(x z,u-v (s), u(s)) -L(2x z,u (s) -x z,u+v (s), u(s))) ds.
(4.10) From (4.8) and Lemma 4.6.2, there exists a constant σ

0 = σ 0 (k , G) > 0 such that t 0 (L(x z,u+v (s), u(s)) + L(2x z,u (s) -x z,u+v (s), u(s)) -2L(x z,u (s), u(s))) ds σ 0 v 2 ∞ .
According to Remark 4.6.4 below, there exists

σ 1 = σ 1 (k , G) > 0 such that |x z,u+v (s) + 4.4. PROOF OF THE MAIN RESULT x z,u-v (s)-2x z,u (s)| σ 1 v 2 L 2 for all s ∈ [0, T ]. Hence, by (4.7), t 0 (L(x z,u-v (s), u(s)) -L(2x z,u (s) -x z,u+v (s), u(s))) ds α t 0 |x z,u+v (s) + x z,u-v (s) -2x z,u (s)| ds σ 2 v 2 ∞ , where σ 2 = σ 2 (k , G) > 0.
For the second and third term in (4.10) we have, using the regularity of the Lagrangian in the second variable, Lemma 4.6.2, and the Cauchy-Schwarz inequality, that t 0 (L(x z,u+v (s), u(s) + v(s)) -L(x z,u+v (s), u(s))) ds

+ t 0 (L(x z,u-v (s), u(s) -v(s)) -L(x z,u-v (s), u(s))) ds = t 0 1 0 ∇ u L(x z,u+v (s), ξv(s) + u(s)), v(s) dξ ds - t 0 1 0 ∇ u L(x z,u-v (s), -ξv(s) + u(s)), v(s) dξ ds σ 3 t 0 (|x z,u+v (s) -x z,u-v (s)| + |v(s)|) |v(s)| ds σ 4 v 2
∞ , where σ i = σ i (k , G) > 0 for i = 3, 4. The above relations and (4.10) imply that

C t (u + v) + C t (u -v) -2C t (u) σ 5 v 2 ∞ (4.11) for a suitable σ 5 = σ 5 (k , G) > 0.
On the other hand,

C t+h (u + v) + C t-h (u -v) -C t (u + v) -C t (u -v) = t+h t (L(x z,u+v (s), u(s) + v(s)) -L(x z,u+v (t), u(s) + v(s))) ds + t+h t (L(x z,u+v (t), u(s) + v(s)) -L(x z,u+v (t), u(s) -v(s))) ds + t+h t (L(x z,u+v (t), u(s) -v(s)) -L(x z,u-v (t), u(s) -v(s))) ds + t+h t L(x z,u-v (t), u(s) -v(s)) ds - t t-h L(x z,u-v (s), u(s) -v(s)) ds,
and, on account of (4.7) and the Lipschitz regularity of trajectories, the first three terms are bounded by 3C

(h v ∞ + h 2 ), with C = C (k , G) > 0 while, since u -v ∈ L T 2k ,
then there exist k k, r > 0, and 0 such that for any (t, z, u) ∈ Λ we can find a map

F t,z,u : B r (t) × B r (z) × B r (x z,u (t)) → L T k ,
satisfying for all (t, z, u) ∈ Λ:

(i) F t,z,u ∈ C 1,1 ; (ii) E s (z , F t,z,u (s, z , β)) = β for all (s, z , β) ∈ B r (t) × B r (z) × B r (x z,u (t)); (iii) dF t,z,u is -Lipschitz. Proof. Let (t 0 , z 0 , u 0 ) ∈ Λ. We know that dE t 0 (z 0 , u 0 )(0, •) is surjective on L 2 (0, T ; R m ). Let V ⊂ C 1 (0, T ; R m
) be a countable subset such that spanV = L 2 (0, T ; R m ). By Lemma 4.6.6, there exist n linearly independent vectors {v 0

1 , ..., v 0 n } ⊂ V such that the map A 0 : R n → R n , defined by A 0 (α) = n i=1 α i dE t 0 (z 0 , u 0 )(0, v 0 i ), is an isomorphism. Define for any (t, z, u) ∈ (0, ∞) × R n × L 2 (0, T ; R m ) the map ϕ 0 t,z,u : R n → R n by ϕ 0 t,z,u (α) = n i=1 α i dE t (z, u)(0, v 0 i )
. By Lemma 4.6.2 and Corollary 4.6.5, there exist 0 > 0, µ 0 > 0 such that for any (t, z, u)

∈ J 0 := B 0 (t 0 ) × B 0 (z 0 ) × B L 2 (u 0 , 0 ) the map E 0 t,z,u : (0, ∞) × R n × R n → (0, ∞) × R n × R n , defined by E 0 t,z,u (s, y, α) = (s, y, E s (y, u + m i=1 α i v 0 i )), satisfies for all (t, z, u) ∈ J 0 | det dE 0 t,z,u (t, z, 0) | = det ϕ 0 t,z,u µ 0 .
Now, from (4.12) and the Ascoli-Arzelà theorem, the set x 0 ∈G U x 0 (Γ) is compact. Then there exists N ∈ N + such that, for all j = 1, ..., N , we can find ρ j > 0, µ j > 0, (t j , z j , u j ) ∈ Λ, and linearly independent v

j 1 , ..., v j n ⊂ V , such that Λ ⊂ j=1,...,N B j (t j ) × B j (z j ) × B L 2 (u j , j ) =: j=1,...,N J j .
Defining for any (t, z, u) ∈ J j the maps

E j t,z,u : (0, ∞) × R n × R n → (0, ∞) × R n × R n and ϕ j t,z,u : R n → R n by E j t,z,u (s, y, α) = (s, y, E s (y, u + m i=1 α i v j i )) and ϕ j t,z,u (α) = n i=1 α i dE t (z, u)(0, v j i )
, we deduce that for all (t, z, u) ∈ J j and j = 1, ..., N

| det dE j t,z,u (t, z, 0) | = det ϕ j t,z,u µ j min {µ 1 , ..., µ N } > 0. (4.13)
Applying the Inverse Mapping Theorem to the map E j t,z,u and using a compactness argument, we conclude that for each j there exists r j > 0 such that the set V j (t, z, u) :=

(t -r j , t + r j ) × B r j (z) × B r j (E t (z, u)) is isomorph to E j t,z,u -1
(V j (t, z, u)) for any (t, z, u) ∈ J j . Put r = min {r 1 , ..., r N } and define for any (t, z, u) ∈ J j , bounded by a suitable constant M 0 and v j i ∈ C 1 (0, T ; R m ), there exists a constant k k such that F t,z,u take values in L T k . Hence, (i) and (ii) follow. Moreover, from (4.13) and the C 1,1 loc regularity of the end-point map, there exists a constant 0, depending only on k and G, such that dF t,z,u is -Lipschitz for all (t, z, u) ∈ Λ. So, we get (iii).

F t,z,u (t , z , β) = u + m i=1 α i (t , z , β)v j i ∀(t , z , β) ∈ V j (t, z, u), where E j t,z,u -1 (t , z , β) = (t ,
Proof of Theorem 4.3.4. Define δ = dist(∂Ω G , Γ) and let r > 0 be as in Proposition 4.4.5 (we can pick r such that r δ). It is sufficient to prove the semiconcavity of

V x 0 , uniformly in x 0 , on the set [t -r, t + r] × B r (x) ∩ Γ whenever (t, x) ∈ Γ. So, fix x 0 ∈ G and let u ∈ U x 0 (t, x) and h, η ∈ R n be such that [t -h, t + h] × [x -η, x + η] ⊂ [t -r, t + r] × B r (x) ∩ Γ.
Hence, denoting for simplicity F t,x 0 ,u by F and using the same notation as in the proof of Lemma 4.4.1, from Lemma 4.4.4 and Proposition 4.4.5 we conclude that

V x 0 (t, x) = C t (F (t, x 0 , x)),
and for all (t , x )

∈ [t -r, t + r] × B r (x) ∩ Γ V x 0 (t , x ) C t (F (t , x 0 , x )).
So, by Lemma 4.4.1 and Proposition 4.4.5, there exist C = C (G, Γ) > 0 and

C = C (G, Γ) > 0 such that for all x 0 ∈ G V x 0 (t + h, x + η) + V x 0 (t -h, x -η) -2V x 0 (t, x) C t+h (F (t + h, x 0 , x + η)) + C t-h (F (t -h, x 0 , x -η)) -2C t (F (t, x 0 , x)) = C t+h (F (t + h, x 0 , x + η)) + C t-h (F (t -h, x 0 , x -η)) -2C t F (t + h, x 0 , x + η) + F (t -h, x 0 , x -η) 2 + 2 C t F (t + h, x 0 , x + η) + F (t -h, x 0 , x -η) 2 -C t (F (t, x 0 , x)) C|F (t + h, x 0 , x + η) -F (t -h, x 0 , x -η)| 2 + C |F (t + h, x 0 , x + η) + F (t -h, x 0 , x -η) -2F (t, x 0 , x)| C C2 (h + η) 2 + C C h 2 + η 2 .

SENSITIVITY RELATIONS 107

Since all constants involved in the previous inequality depend only on G and Γ, the conclusion follows.

Sensitivity Relations

We investigate next sensitivity relations for the minimum time function. Let Γ ⊂ R n be a compact subset.

Remark 4.5.1. It is known (see [JSC87, Proposition 3.1]) that the sub-Riemannian distance between two points y and x is equal to the minimum time τ {y} (x) to reach y from x, associated to the control system

           y (s) = m i=1 u i (s) f i (y(s)) for a.e. s 0 y(0) = x u ∈ B m , (4.14)
where B m denotes the set of all Lebesgue measurable controls u : [0, ∞) → R m such that u(s) ∈ B 1 (0) for a.e. s 0. So, the minimum time function τ Γ (•) to reach Γ for the control system (4.14) satisfies τ

Γ (x) = inf y∈Γ τ {y} (x) = inf y∈Γ d SR (y, x) = d SR (Γ, x) for all x ∈ R n . A control u ∈ B m is said to be optimal (for the minimum time function τ Γ ) at z if τ Γ (z) = θ Γ (x z,u (•), u(•)).
Subsequently, to shorten notation, we write f (x, u) in place of m i=1 u i f i (x). Next we recall a result from [CFS00, Theorem 3.1], stated under more general assumptions for the vector fields f 1 , ..., f m : Lemma 4.5.2 ([CFS00]). Assume (H)-(i). Let A ⊂ R n be a closed set, ū be an optimal control at x 0 ∈ A c for the minimum time function τ A (•), and put τ 0 = τ A (x 0 ).

Then for any ξ

∈ T A c (x x 0 ,ū (τ 0 )) -the solution q : [0, τ 0 ] → R n of the adjoint system      -q (t) = d x f (x x 0 ,ū (t), ū(t)) * q(t) for a.e. t ∈ [0, τ 0 ] q(τ A (x 0 )) = -ξ (4.15)
satisfies the minimum principle

q(t), f (x x 0 ,ū (t), ū(t)) = min u∈B 1 (0) q(t), f (x x 0 ,ū (t), u) ∀ t ∈ [0, τ 0 ].
We denote by H the Hamiltonian function on R n × R n , defined by

H(x, p) = max u∈B 1 (0) p, f (x, u) .

. Assume (H)-(i).

Let A ⊂ R n be a closed set and fix x 0 ∈ A c . Let ū be an optimal control at x 0 for the minimum time function and let ξ ∈ N P A c (x), where x := x x 0 ,ū (τ A (x 0 )).

The following statements hold true:

(i) if H(x, ξ) = 0, then -p(t) ∈ ∂ P τ A (x x 0 ,ū (t)) ∀ t ∈ [0, τ A (x 0 )),
where p(•) solves (4.15) with final condition p(τ A (x 0 )) = ξ/H(x, ξ);

(ii) if H(x, ξ) = 0, then -p(t) ∈ ∂ ∞,P τ A (x x 0 ,ū (t)) ∀ t ∈ [0, τ A (x 0 )),
where p(•) solves (4.15) with final condition p(τ A (x 0 )) = ξ.

Proof. Denote for simplicity by τ (•) the minimum time function τ A (•). Let q(•) be the solution of (4.15) with final condition -q(τ (x 0 )) = ξ ∈ N P A c (x), and put α := H(x, ξ) and p(•) := -q(•). We only show the conclusions (i) and (ii) at t = 0, i.e., (p(0), α) ∈ N P hypo τ (x 0 , τ (x 0 )). (4.16)

First of all we claim that α 0. Indeed, since p(τ (x 0 )) ∈ N P A c (x), there exists σ 0 such that ξ, ȳx σ |ȳ -x| 2 for all ȳ ∈ A c . So, for every 0 < t < τ (x 0 )

ξ, x x 0 ,ū (t) -x x 0 ,ū (τ (x 0 )) σ |x x 0 ,ū (t) -x x 0 ,ū (τ (x 0 ))| 2 ,
and, dividing the previous inequality by t -τ (x 0 ), it follows

ξ, 1 t -τ (x 0 ) t τ (x 0 ) f (x x 0 ,ū (s), ū(s)) ds σ 1 t -τ (x 0 ) |x x 0 ,ū (t) -x x 0 ,ū (τ (x 0 ))| t τ (x 0 ) f (x x 0 ,ū (s), ū(s)) ds . (4.17) Now, 1 t -τ (x 0 ) t τ (x 0 ) f (x x 0 ,ū (s), ū(s)) ds = 1 t -τ (x 0 ) t τ (x 0 ) (f (x x 0 ,ū (s), ū(s)) -f (x x 0 ,ū (τ (x 0 )), ū(s))) ds + 1 t -τ (x 0 ) t τ (x 0 ) f (x x 0 ,ū (τ (x 0 )), ū(s)) ds.
(4.18) By Lemma 4.6.2, there exists a constant C 0 such that

1 t -τ (x 0 ) t τ (x 0 ) (f (x x 0 ,ū (s), ū(s)) -f (x x 0 ,ū (τ (x 0 )), ū(s))) ds C |t -τ (x 0 )| . (4.19) Furthermore, since f (x, B 1 (0)) is compact and convex for all x ∈ R n , 1 t -τ (x 0 ) t τ (x 0 ) f (x x 0 ,ū (τ (x 0 )), ū(s)) ds ∈ f (x x 0 ,ū (τ (x 0 )), B 1 (0)),
and there exist t i → τ (x 0 )-and u * ∈ B 1 (0) satisfying

1 t i -τ (x 0 ) t i τ (x 0 ) f (x x 0 ,ū (τ (x 0 )), ū(s)) ds → f (x x 0 ,ū (τ (x 0 )), u * ).
(4.20) So, using (4.18), (4.19), and (4.20), passing to the limit in (4.17) when t = t i and

t i → τ (x 0 ) -we get that ξ, f (x x 0 ,ū (τ (x 0 )), u * ) 0.
Hence, the claim holds true.

To prove (4.16), we have to show that there exists σ 0 such that for all y ∈ A c and β τ (y) p( 0 ( p(τ (x 0 )), f (x x 0 ,ū (s), ū(s)) -f (x x 0 ,ū (τ (x 0 )), ū(τ (x 0 ))) ) ds + α (τ (x 0 ) -τ (y)) + p(τ (x 0 )), y 1 -x . and there exists σ 3 0 such that for all s ∈ [0, τ (x 0 )] p(τ (x 0 )), f (x x 0 ,ū (s), ū(s)) -f (x x 0 ,ū (τ (x 0 )), ū(τ (x 0 )))

= p(τ (x 0 )), f (x x 0 ,ū (s), ū(s)) -f (x x 0 ,ū (τ (x 0 )), ū(s))

+ p(τ (x 0 )), f (x x 0 ,ū (τ (x 0 )), ū(s)) -f (x x 0 ,ū (τ (x 0 )), ū(τ (x 0 ))) Finally, from the dynamic programming principle and with the same technique as those used for the case t = 0, we show that the conclusion holds on the whole time interval [0, τ (x 0 )). We state next the main result of this section. Theorem 4.5.5. Assume (H) . Let Γ ⊂ R n be a compact set and suppose that there are no singular geodesics for Γ. Let x 0 ∈ Γ c and ū be an optimal control for the minimum time function at x 0 . Denote Γ t = {y ∈ R n | τ Γ (y) t}.

σ 3 |τ (x 0 ) -s| .
Then the solution of the adjoint equation all i 1 there exists r i > 0 such that for any y ∈ ∂Γ i we can find a unit vector vi (y) satisfying B r i (y + r i vi (y)) ⊂ Γ i .

-p (t) = d x f (x x 0 ,ū (t
(4.46)

We note that (4.46) implies that the set N P Γ c i (y) contains a nonnull vector for all y ∈ ∂Γ i . Furthermore, since τ Γ is locally Lipschitz continuous on Γ c , applying [CS04, Theorem 8.2.3], we have for all i 1 and all y ∈ ∂Γ i that H(y, ξ) = 0 for any ξ ∈ N P Γ c i (y)∩S 1 . We next construct a solution p(•), solving the adjoint equation, associated to the sequence {λ i } i as follows. For all i 1 pick ξ i ∈ N P Γ c i (y i ) ∩ S 1 where y i = x x 0 ,ū (τ ∂Γ i (x 0 )). We denote by p i (•) the solution of (4.15) on [0, τ ∂Γ i (x 0 )], with final condition ξ i H(y i , ξ i ) -1 , and we extend such functions as solutions of the adjoint equation in (4.15) to the whole interval [0, τ Γ (x 0 )] (we continue to denote by p i (•) such extended functions). Notice that, since τ ∂Γ i (•) = τ Γ (•) -λ i on Γ c i for all i 1, ∂ P τ ∂Γ i (y) = ∂ P τ Γ (y) for all y ∈ Γ c i . From Proposition 4.5.3 it follows that -p i (t) ∈ ∂ P τ Γ (x x 0 ,ū (t))

∀t ) is bounded, the sequence {|p i (0)|} i is bounded. So, applying the Ascoli-Arzelà and the Dunford-Pettis theorems, taking a subsequence and keeping the same notations, there exists an absolutely continuous function p(•) on [0, τ Γ (x 0 )] such that p i → p uniformly on [0, τ Γ (x 0 )] and p i p in L 1 (0, τ Γ (x 0 )). Such p(•) satisfies the adjoint equation on [0, τ Γ (x 0 )]. Now, using the closedness of ∂ P τ Γ (y) for any y ∈ Γ c , we get (4.43) by passing to the limit in (4.47), and from the definition of upper limit we get (4.44).

Appendix

Below we assume that T > 0. Proof. Let B ⊂ R n and W ⊂ L 2 (0, T ; R m ) be bounded subsets. We have Hence, applying Gronwall's lemma, for some C = C (W , B) 0 we get (4.49). Finally, if W ⊂ B L ∞ (0, R), the last conclusion follows from (4.49) and Remark 4.6.1. Proposition 4.6.3. Assume (H)-(i). Then E T ∈ C 1,1 loc (R n × L 2 (0, T ; R m )).

Proof. Let (y, u) ∈ R n × L 2 (0, T ; R m ) and consider a bounded neighbourhood of (y, u) in R n × L 2 (0, T ; R m ) of the form B δ (y) × B L 2 (u, δ), with δ > 0. Set for simplicity ∆x(•) = x y+h,u+v (•) and x(•) = x y,u (•). So

x

  So, if Hörmander's condition holds true and there are no singular time-minimizing controls, then for any compact set K ⊂ R n and any y ∈ R n \K the function d SR (y, •) is C(y)-semiconcave on K. This property does not suffice to guarantee the local semiconcavity of d SR (A, •) = inf y∈A d SR (y, •) on R n \A, because the semiconcavity constant C(y) might blow up with y ∈ A. Nevertheless, we analyze the local semiconcavity property of the function inf y∈A d SR (y, •) obtaining uniform bounds on the constant C(y) as y lies in a compact set. More precisely, we show that for any compact set A ⊂ R n \K there exists a nonnegative constant C

  We denote by B the closed unit ball in R n and by |•| the Euclidean norm. The interior of C ⊂ R n is written as int C. Given a nonempty subset C and a point x we denote the distance from x to C by d C (x) := inf {|x -y| : y ∈ C}, the convex hull of C by co C, and its closure by co C. Take a family of sets {S(y) ⊂ R n : y ∈ D} where D ⊂ R n and x ∈ R n . The sets 1 Lim inf y→x D

1. 4 .

 4 THE INFINITE HORIZON OPTIMAL CONTROL PROBLEM 35 over all the trajectory-control pairs satisfying the state constrained equation

  Let B(x, δ) stand for the closed ball in R n with radius δ > 0 centered at x ∈ R n and set B = B(0, 1), S n-1 = ∂B. Denote by | • | and •, • the Euclidean norm and scalar product, respectively. Let C ⊂ R n be a nonempty set. We denote the interior of C by int C, the convex hull of C by co C, and the distance from x ∈ R n to C by d C (x) := inf{|x -y| : y ∈ C}. If C is closed, we let Π C (x) be the set of all projections of x ∈ R n onto C. For p ∈ R + ∪ {∞} and a Lebesgue measurable set I ⊂ R we denote by L p (I; R n ) the space of R n -valued Lebesgue measurable functions on I endowed with the norm • p,I . We say that f ∈ L p loc (I; R n ) if f ∈ L p (J; R n ) for any compact subset J ⊂ I. In what follows µ stands for the Lebesgue measure on R. Let I be an open interval in R, f ∈ L 1 loc (I; R n ) and θ f : [0, µ(I)) → [0, ∞) be defined 50 2.3. UNIFORM DISTANCE ESTIMATES by θ f (σ) = sup J |f (τ )| dτ : J ⊂ I, µ(J) σ .

  UNIFORM DISTANCE ESTIMATES Put x 0 (•) := x(•). From Cases 1 and 2, replacing [t 0 , t 1 ] by [τ 1

  Theorem 2.3.4. Let us assume (H) and (IPC) . Then the assertion of Theorem 2.3.1 is valid.Proof. (IPC) corresponds to the conclusion of [FM13a, Proposition 7] with r, η, and M defined uniformly over A. Thanks to this observation and Remark 2.3.3 exactly the same arguments as those in [FM13a, proof of Theorem 5] can be used to prove the theorem.

  what follows " -" stands for the negative polar cone of a set, i.e., C -= p ∈ R k : p, c 0 ∀c ∈ C . Moreover, we denote the positive polar cone of C by C + := -C -.

  and t epi W (t, •) is locally absolutely continuous. Remark 3.5.5. Let W : [0, ∞) × A → R be a locally Lipschitz continuous function.

  for a.e. t > 0, ∀x ∈ int A, lim t→∞ sup y∈A |V (t, y)| = 0. (3.33)

  the set of all p ∈ R n satisfying lim supy→x E ϕ(y) -ϕ(x) -p, y -x |y -x| 0. Moreover, if ϕ is locally semiconcave, then for all x ∈ int E holds the following property (see [CS04, Theorem 3.3.6]) co D * ϕ(x) = D + ϕ(x), (4.1)

Remark 4.3. 1 .

 1 Assume (H) and that |L(x, u)| ϕ(x)(1 + |u| 2 ), where ϕ(•) is a locally bounded function on R n . Then (t, x) ∈ D G if for any x 0 ∈ G there exists a square integrable control u : [0, t] → R m steering x 0 to x in time t.

ξ

  ), y -x 0 + α(β -τ (x 0 )) σ |y -x 0 | 2 + |β -τ (x 0 )| 2 . (4.21) On account of [CLSW08, Proposition 1.5], we prove (4.21) for all y ∈ A c and β τ (y) with |τ (y) -τ (x 0 )| 1. Fix such y and β, and let ξ(•) be the solution of the Cauchy problem (t) = f (ξ(t), ū(t)) for a.e. t 0 ξ(0) = y. (4.22) Case 1: τ (y) τ (x 0 ).Put y 1 = ξ(τ (y)) ∈ A c and x 1 = x x 0 ,ū (τ (y)) ∈ A c . By Gronwall's lemma there exists K 0 such that for any s ∈ [0, τ (x 0 )]|ξ(s) -x x 0 ,ū (s)| e Ks |y -x 0 | e Kτ (x 0 ) |y -x 0 | . (4.23) Furthermore, D s p(s), ξ(s) -x x 0 ,ū (s) = p (s), ξ(s) -x x 0 ,ū (s) + p(s), f (ξ(s), ū(s)) -f (x x 0 ,ū (s), ū(s)) = -d x f (x x 0 ,ū (s), ū(s)) * p(s), ξ(s) -x x 0 ,ū (s) + p(s), f (ξ(s), ū(s)) -f (x x 0 ,ū (s), ū(s))= p(s), f (ξ(s), ū(s)) -f (x x 0 ,ū (s), ū(s)) -d x f (x x 0 ,ū (s), ū(s))(ξ(s) -x x 0 ,ū (s)) = p(s),

1 0( 1

 11 -t)d 2 x f (tξ(s) + (1 -t)x x 0 ,ū (s), ū(s))(ξ(s) -x x 0 ,ū (s)) 2 dt ,and p(τ (y)), y 1 -x 1 = p(0), y -x 0 + τ (y) 0 D s p(s), ξ(s) -x x 0 ,ū (s) ds.

  23) we deduce that there exists σ 1 0 (not depending on y) satisfyingp(0), y -x 0 p(τ (y)), y 1 -x 1 + σ 1 |y -x 0 | 2 . (4.24)Since p(•) is Lipschitz continuous, there exists σ 2 0 such thatp(τ (y)), y 1 -x 1 = p(τ (x 0 )), y 1 -x 1 + p(τ (y)) -p(τ (x 0 )), y 1 -x 1 p(τ (x 0 )), y 1 -x 1 + σ 2 |τ (y) -τ (x 0 )| |y 1 -x 1 | p(τ (x 0 )), y 1 -x 1 + σ 2 2 |τ (y) -τ (x 0 )| 2 + |y 1 -x 1 | 2 , (4.25)andp(τ (x 0 )), y 1 -x 1 = p(τ (x 0 )), x x 0 ,ū (τ (x 0 )) -x x 0 ,ū (τ (y)) + p(τ (x 0 )), ξ(τ (y)) -x x 0 ,ū (τ (x 0 )) = τ (x 0 ) τ (y) p(τ (x 0 )), f (x x 0 ,ū (s), ū(s)) ds + p(τ (x 0 )), ξ(τ (y)) -x x 0 ,ū (τ (x 0 )) = τ (x 0 ) τ (y)

(4. 26 )

 26 Since p(τ (x 0 )) ∈ N P A c (x), we havep(τ (x 0 )), y 1 -x σ |y 1 -x| 2 σ |τ (x 0 ) -τ (y)| 2 + |y 1 -x| 2 , (4.27)

  (4.26) and inequalities (4.27) and (4.28), it follows that p(τ (x 0 )), y 1 -x 1 α (τ (x 0 ) -τ (y)) + max (σ, σ 3 ) |τ (x 0 ) -τ (y)| 2 + |y 1 -x| 2 .(4.29)On account of (4.23), by (4.29) there exists σ 4 0 satisfyingp(τ (x 0 )), y 1 -x 1 α (τ (x 0 ) -τ (y)) + σ 4 |y -x 0 | 2 + |τ (x 0 ) -τ (y)| 2 .Finally, from (4.23), (4.24), and (4.25), we deduce (4.21) (we assumed β τ (y)).Case 2: τ (y) > τ (x 0 ). τ (x 0 )), α) ∈ N P hypo τ (x, 0). (4.30)The inclusion (4.30) means that there exists σ 5 0 satisfying for all ỹ ∈ A c and β τ (ỹ)p(τ (x 0 )), ỹx + α β σ 5 |ỹ -x| 2 + β2 . (4.31)If β 0, then (4.31) follows from the condition p(τ (x 0 )) ∈ N P A c (x). On the other hand, suppose that 0 < β τ (ỹ) 1 and let z(•) be the solution of the problem     chapter 4 z (t) = f (z(t), ū(τ (x 0 ))) for a.e. t 0 z(0) = ỹ.Define ȳ1 = z( β) and observe that ȳ1 ∈ A c . So, letting K 0 to be the Lipschitz constant of f with respect to the space variable on the compact set {y ∈ R n | τ (y) 1}, we deduce that for all 0 < t β|z(t) -ỹ| t 0 |f (z(s), ū(τ (x 0 )))| ds t 0 |f (z(s), ū(τ (x 0 ))) -f (ỹ, ū(τ (x 0 )))| ds + t |f (ỹ, ū(τ (x 0 )))| K t 0 |z(s) -ỹ| ds + t |f (ỹ, ū(τ (x 0 )))| K t 0 |z(s) -ỹ| ds + β |f (ỹ, ū(τ (x 0 )))| .From Gronwall's lemma, it follows that for all 0 < t β 1|z(t) -ỹ| βe Kt |f (ỹ, ū(τ (x 0 )))| βe K (|f (ỹ, ū(τ (x 0 ))) -f (x, ū(τ (x 0 )))| + |f (x, ū(τ (x 0 )))|) βKe K |ỹ -x| + βe K |f (x, ū(τ (x 0 )))| Ke K |ỹ -x| + βe K |f (x, ū(τ (x 0 )))| Ke K |ỹ -x| + β |f (x, ū(τ (x 0 )))| ,(4.32)where K = K + 1. Now p(τ (x 0 )), ỹx = p(τ (x 0 )), ỹ -ȳ1 + p(τ (x 0 )), ȳ1 -x , the inclusion p(τ (x 0 )) ∈ N P A c (x) with (4.32), it follows thatp(τ (x 0 )), ȳ1 -x σ |ȳ 1 -x| 2 2σ |ȳ 1 -ỹ| 2 + |ỹ -x| 2 2σ K2 e 2 K 2 |ỹ -x| 2 + 2 β2 |f (x, ū(τ (x 0 )))| 2 + |ỹ -x| 2 σ5 β2 + |ỹ -x| 2 , (4.34) for a suitable constant σ5 0. On the other hand, p(τ (x 0 )), ỹ -ȳ1 = -β 0 p(τ (x 0 )), f (z(s), ū(τ (x 0 ))) ds = -β 0 p(τ (x 0 )), f (x, ū(τ (x 0 ))) ds β 0 p(τ (x 0 )), f (z(s), ū(τ (x 0 ))) -f (x, ū(τ (x 0 ))) ds = -α ββ 0 p(τ (x 0 )), f (z(s), ū(τ (x 0 ))) -f (x, ū(τ (x 0 ))) ds.

  σ = |p(τ (x 0 ))| K, (x 0 )), f (z(s), ū(τ (x 0 ))) -f (x, ū(τ (x 0 ))) ds |p(τ (x 0 ))| β 0 |f (z(s), ū(τ (x 0 ))) -f (x, ū(τ (x 0 )))| ds σ ) -x| ds + β |ỹ -x| σ β 0 (e K |ỹ -x| + s |f (x, ū(τ (x 0 )))|) ds + constant σ5 0. Now, from (4.35) and (4.36) it follows that p(τ (x 0 )), ỹ -ȳ1 + α β σ5 β2 + |ỹ -x| 2 . (4.37) Hence (4.31) follows from (4.33), (4.34), and (4.37). Now, consider the solution ξ(•) of (4.22) and put ȳ = ξ(τ (x 0 )) ∈ A c . In a similar fashion as in the previous step, there exists σ 6 0 such that p(0), y -x 0 p(τ (x 0 )), ȳx + σ 6 |y -x 0 | 2 . (4.38)From the dynamic programming principle it follows that if β τ (y) then β -τ (x 0 ) τ (y) -τ (x 0 ) τ (ȳ), so by (4.31) we havep(τ (x 0 )), ȳx + α (β -τ (x 0 )) σ 5 |ȳ -x| 2 + |β -τ (x 0 )| 2 ,and, using (4.23), we deduce that there exists σ 7 0 not depending on y such thatp(τ (x 0 )), ȳx + α (β -τ (x 0 )) σ 7 |y -x 0 | 2 + |β -τ (x 0 )| 2 . (4.39) So, combining (4.38) and (4.39), we deduce (4.21).

  Proposition 4.5.4. Let u be a C-semiconcave function on an open set O ⊂ R n , with C > 0. Suppose that, for some λ ∈ R,(i) U λ := {x ∈ O | u(x) λ} = ∅ (ii) ∂U λ ∩ O ⊂ O and ∂U λ ∩ O is compact (4.40) (iii) ∃ α > 0 such that D + u(x) ∩ B α (0) c = ∅ ∀x ∈ O,(4.41)then there exists r > 0 such that for all x ∈ ∂U λ ∩ O we can find vx ∈ S 1 satisfyingB r (x + rv x ) ⊂ U λ .Proof. We claim the following: if x ∈ ∂U λ ∩ O, p ∈ D + u(x), and R > 0 are such thatB R (x) ⊂ O & p = 0, (4.42) then for v := -p/ |p| we have B r (x + r v) ⊂ U λ , where r = min {R /2, |p| /2C}. Indeed, for such r , by (4.42), we have [x, x -r p |p| + r v] ⊂ O for all v ∈ S 1 , and, applying [CS04, Proposition 3.3.1], we getu(x -r p |p| + r v) u(x) + p, r v -r p |p| + Cr 2 v -p |p| 2 λ + r p, v -r |p| + 2Cr 2 1 -p, v |p| = λ + r ( p, v -|p|) 1 -2Cr |p| λ.So, the claim holds true for v = -p/ |p|.Now, denote R x = sup r > 0 | B r (x) ⊂ O for all x ∈ ∂U λ ∩ O. If R x = +∞ for some x ∈ ∂U λ ∩O, then O = R n .Otherwise, we claim that there exists R > 0 such that 4.5. SENSITIVITY RELATIONS R x R for all x ∈ ∂U λ ∩ O. Indeed, otherwise there exists a sequence {x i } i ⊂ ∂U λ ∩ O such that B Rx i +ε (x i ) ∩ O c = ∅ for all i 1 and R x i → 0 for any ε > 0. Using (4.40), by further subsequence extraction, we can suppose thatx i → x ∈ ∂U λ ∩ O (then x ∈ O), and since d(x i , O c ) R x i + ε,passing to the limit we obtain d(x, O c ) ε. By arbitrariness of ε it follows that d(x, O c ) = 0 and so x ∈ O c . Hence x ∈ O ∩ O c = ∂O, in contradiction with (4.40). So, the claim holds true. We can conclude that for some R > 0, B R (x) ⊂ O for all x ∈ ∂U λ ∩ O. From the first claim, we deduce that for any x ∈ ∂U λ ∩O and any p x ∈ D + u(x)∩B α (0) c , vx := -p x / |p x | satisfies B rx (x + r x vx ) ⊂ U λ , where r x = min {R x /2, |p x | /2C}. Finally, using (4.41), we have r x min {R/2, α/2C}, and the conclusion follows with r = min {R/2, α/2C}.

  (y)| -1 = 0 on Γ c , where F (y) is the matrix whose columns are the vectors f 1 (y), ..., f m (y). Since τ Γ is a viscosity solution of such an equation, we have that |F (y) * p| -1 = 0 for all y ∈ Γ c and all p ∈ D * τ (y) (see [CR08, Section 5.3]). So, putting M > max F (y) | y ∈ D , from (4.1) we deduce that for all i 1 and all y ∈ O i there exists p i ∈ D + τ Γ (y) such that |p i | M -1 . Hence, applying Proposition 4.5.4, for 4.6. APPENDIX 115

  Remark 4.6.1. If (H)-(i) holds true and B ⊂ R n , W ⊂ L 2 (0, T ; R m ) are bounded subsets then, by Gronwall's lemma, it follows that there exists r = r (W , B) > 0 such thatx x 0 ,u ∞ r ∀u ∈ W , ∀x 0 ∈ B. (4.48) If W ⊂ B L ∞ (0, R) for R 0,then all trajectories x x 0 ,u (•) with (x 0 , u) ∈ B × W are uniformly Lipschitz continuous on [0, T ].

  Assume (H)-(i). Let B ⊂ R n and W ⊂ L 2 (0, T ; R m ) be bounded subsets.Then there exists C = C(W , B) 0 such that for all t ∈ [0, T ], z, y ∈ B, andv, w ∈ W |x z,v (t) -x y,w (t)| C ( v -w L 2 + |z -y|) .(4.49)In particular, if W ⊂ B L ∞ (0, R), then there exists C = C(R, B) 0 such that for all t, s ∈ [0, T ], z, y ∈ B, and v, w ∈ W |x z,v (t) -x y,w (s)| C ( v -w L 2 + |z -y| + |t -s|) .

4

  |x z,v (t) -x y,w (ts) f i (x z,v (s)) ds -s) f i (x y,w (s)) ds + z -y (s) -w i (s)) f i (x z,v (s)) ds -s) (f i (x y,w (s)) -f i (x z,v (s))) ds + |z -y| i (s) -w i (s))| |f i (x z,v (s))| ds + t 0 |w i (s)| |f i (x y,w (s)) -f i (x z,v (s))| ds + |z -y| . (4.50) From (4.48) it follows that x z,v (•) takes values in a compact set of R n for all v ∈ W and z ∈ B. Then there exists M = M (W , B) 0 such that W ⊂ B L 2 (0, M ) and for all z, y ∈ B, v, w ∈ W , s ∈ [0, T ], and i = 1, ..., m holds |f i (x z,v (s))| M & |f i (x z,v (s)) -f i (x y,w (s))| M |x z,v (s) -x y,w (s)| . (4.51) Now, from the Cauchy-Schwarz inequality and since m i=1 |v i | √ m |v|, we havem i=1 t 0 |v i (s) -w i (s)| |f i (x z,v (s))| ds M i (s) -w i (s)| ds M √ m T 0 |v(s) -w(s)| ds M √ mT v -w L 2 s)| |f i (x y,w (s)) -f i (x z,v (s))| ds M s)| |x y,w (s) -x z,v (s)| ds s)| |x y,w (s) -x z,v (s)| ds.On account of the above inequalities, (4.50) becomes |x z,v (t) -x y,w (t)| M √ mT v -w L 2 + |z -y| + M t 0 m i=1 |w i (s)| |x z,v (s) -x y,w (s)| ds.

+ t 0 X

 0 (t)X(s) -1 R(s) ds (4.57) where X(•) is the fundamental solution, i.e.,     chapter 4 X (t) = A(t)X(t) for a.e. t ∈ [0, T ] X(0) = I. Furthermore, letting C 1 = max { X(T )X(s) -1 | s ∈ [0, T ]} and C 2 = max M C, M C , t)| dt + (|h| + v L 2 ) t)| dt L (|h| + v L 2 ) 2 ,where L 0 is a suitable constant depending only on δ. Finally, from (4.57) it follows that∆x(T ) -x(T ) -X(T )h + T 0 X(T )X(s) -1 B(s)v(s) ds L (|h| + v L 2 ) 2 and since (h, v) → X(T )h + T 0 X(T )X(s) -1 B(s)v(s) ds is linear and continuous on R n × L 2 (0, T ; R m ), we get dE T (y, u)(h, v) = X(T )h + T 0 X(T )X(s) -1 B(s)v(s) ds. (4.58)Now, from (4.58) and regularity of f i 's it follows that there exists C = C (δ) > 0 such that|dE T (y 1 , u 1 )(ŷ, û) -dE T (y 2 , u 2 )(ŷ, û)| C (|y 1 -y 2 | + u 1 -u 2 L 2 ) (|ŷ| + û L 2 ) ,and sodE T (y 1 , u 1 ) -dE T (y 2 , u 2 ) C (|y 1 -y 2 | + u 1 -u 2 L 2 ) . (4.59)Remark 4.6.4. From the last part of the proof of Proposition 4.6.3 we deduce that for any τ > 0, r > 0, and R > 0 we can findC = C(τ, r, R) > 0 satisfying dE s (y 1 , u 1 ) -dE s (y 2 , u 2 ) C (|y 1 -y 2 | + u 1 -u 2 L 2 ) for all s ∈ [0, τ ], u 1 , u 2 ∈ B L 2 (0, R), and y 1 , y 2 ∈ B(0, r). Therefore |x z,u+v (s) + x z,u-v (s)-2x z,u (s)| C v 2 L 2 for all s ∈ [0, τ ], u, v ∈ B L 2 (0, R),and z ∈ B(0, r). Corollary 4.6.5. Assume (H)-(i). Then the map [0, τ ] × R n × L 2 (0, τ ; R m ) (s, y, u) → dE s (y, u)

  NOTATIONSN Positive natural numbers R, R + Real numbers, Positive real numbers |x| Euclidean norm of x ∈ R n •, • Scalar product on R n d E (x) Euclidean distance from x ∈ R n to the set E µ L Lebesgue measure L 1 (I; R + ) Space of Lebesgue integrable functions from I to R + L 1 Interior of the set E E Closure of the set E co E Convex hull of the set E co E Closed convex hull of the set E E -Negative polar of the set E N E (x) Limiting normal cone to E at x N P Limiting subdifferential of g at x ∂ + g(x)Limiting superdifferential of g at x ∂ P g(x) Proximal supergradient of g at x ∇g(x) Gradient of g at x Adjoint of Φ Lim inf y→ D x F (y) Lower limit in the Kuratowski-Painlevé sense of F at x from points laying in D Lim sup y→ D x F (y) Upper limit in the Kuratowski-Painlevé sense of F at x from points laying in D

	dom Domain
	epi Epigraph
	hypo Hypograph
	graph Graph
	df (x) Differential of f at x
	Φ

loc (I; R + ) Space of locally Lebesgue integrable functions from I to R + W 1,1 loc (I; R n ) Space of locally absolutely continuous functions from

I to R n B(x, r) Ball centered at x of radius r in R n B Unit ball in R n S n-1 Unit sphere in R n E c Complement of the set E ∂E Boundary of the set E int E E (x) Proximal normal cone to E at x T E (x) Contingent cone to E at x T C E (x)

Clarke tangent cone to E at x g ∞ Uniform norm of g ∂g(x) * INTRODUCTION T his thesis addresses deterministic optimal control problems with infinite horizon subject to state constraints. The setting we take into account is the following optimal control problem minimize ∞ t 0 L(t, x(t), u(t)) dt (1) over all Lebesgue measurable trajectory-control pairs x : [t 0 , ∞) → R n and u : [t 0 , ∞) → R m satisfying x (t) = f (t, x(t), u(t)) for a.e. t t 0 , x(t 0 ) = x 0 ,

  •, u) and l(t, •, u) are k(t)-Lipschitz continuous for a.e. t > 0, uniformly with respect u ∈ U (t), and |f (t, x, u)| + |l(t, x, u)| c(t)(1 + |x|) for a.e. t > 0 and all x, y

  U (t)} is continuous on R n with closed images, and the following set

•, x, •) are Lebesgue-Borel measurable and there exists φ ∈ L 1 ([0, ∞); R) such that L(t, x, u) φ(t) for a.e. t 0 and all (x, u) ∈ R n × R m ; (b) there exist c, k ∈ L loc such that f (t, •, u) and L(t, •, u) are k(t)-Lipschitz continuous for a.e. t > 0, uniformly with respect u ∈ U (t), and |f (t, x, u)| + |L(t, x, u)| c(t)(1 + |x|) for a.e. t > 0 and all x, y ∈ R n , u ∈ U (t); (c) for a.e. t 0 and all x ∈ R n , the set-valued map y {(f (t, y, u), L(t, y, u)) : u ∈

  a multifunction taking nonempty values. We say that G has a sub-linear growth (in x) if, for some c ∈ L 1 loc (I; R + ), sup v∈G(t,x) |v| c(t)(1 + |x|) for a.e. t ∈ I and all x ∈ R n .

Let Λ ⊂ R n . We say that G(•, x) is γ-left absolutely continuous, uniformly for x ∈ Λ, where γ ∈ L 1 loc (I; R + ), if G(s, x) ⊂ G(t, x) + t s γ(τ ) dτ B ∀s, t ∈ I : s < t, ∀x ∈ Λ.

(2.3)

  the Fubini theorem, for any t ∈ (t 0 + kρ, t 1 ],

	t t 0 for all t ∈ [t 0 , t 1 ]. Then, using (H)-(iii), (2.3), and (2.10), it follows that ϕ(τ ) dτ t t 0 d F (s,y(s)) (y (s)) ds d F (s,y(s)) (y (s))   and, using t θ 2.3. UNIFORM DISTANCE ESTIMATES t 0 +kρ	(2.12) 53
	

.10) where J = {s ∈ (t 0 + kρ, t 1 ] : x (s -kρ) exists}. Hence

x -y ∞,[t 0 ,t 1 ] 2M kρ.

(2.11) By Filippov's theorem (cfr.

[START_REF] Aubin | Set-valued analysis[END_REF]

) there exists an

F -trajectory x(•) on [t 0 , t 1 ] such that x(t 0 ) = y(t 0 ) and y -x ∞,[t 0 ,t] e γ (∆) + ϕ(s)M (s -t 0 ) a.e. s ∈ [t 0 , (t 0 + kρ) ∧ t 1 ] ϕ(s)M kρ + s s-kρ γ(τ )

dτ a.e. s ∈ (t 0 + kρ, t 1 ]. (2.13) Hence, we obtain for any t ∈ [t 0 , (t 0 + kρ) ∧ t 1 ] t t 0 d F (s,y(s)) (y (s)) ds (θ γ (∆) + θ ϕ (∆)M )(t -t 0 ),

  is an F-trajectory or a feasible F-trajectory, respectively, for all t 1 > t 0 .

	Theorem 2.3.6. Assume that either (H), (AC), and (IPC) or (H) and (IPC) hold
	true. Furthermore, suppose that
	lim sup
	t→∞

  Using the sub-linear growth of f , l, and the Gronwall lemma, we have 1 + |x(t)| (1 + |x 0 )|e

				t t 0	c(s) ds for all t t 0 and for any
	trajectory-control pair (x(•), u(•)) at (t 0 , x 0 ).
	Let a 1 > 0, a 2 > 0 be such that
	t
	0	c(s) ds a 1 t + a 2	∀t 0.	(2.29)
	For all T > t 0 , we have
	T t 0 e -λt |l(t, x(t), u(t))| dt	T t 0 e -λt c(t)(1 + |x 0 )e (1 + |x 0 )e a 2 T t 0 e -(λ-a 1 )t c(t) dt. t t 0 c(s) ds dt	(2.30)
	Then, by (2.29) and denoting ψ(t) = t t

0 c(s) ds, for any λ > a 1

T t 0 e -λt |l(t, x(t), u(t))| dt (1 + |x 0 )|e a 2 e -(λ-a 1 )t ψ(t)

  We would like to underline that the infimum of a family of 4.4. PROOF OF THE MAIN RESULT semiconcave functions is not in general a semiconcave function. When each member of the family is semiconcave with same constant then the infimum is semiconcave too. Let {u | u ∈ Z } be a family of C-semiconcave functions on Γ ⊂ R n and put w

	Lemma 4.3.6 ([CS04]).

Then, by Proposition 4.3.5, it follows that d SR (x 0 , x) = E x 0 (1, x) for all x ∈ R n , and, assuming that any geodesic connecting x to x 0 = x is not singular, by [CR08, Theorem 5] the function d SR (x 0 , •) is locally semiconcave on R n \ {x 0 }.

  ). We say that there are no singular geodesics for Γ (associated to the distribution spanned by f 1 , ..., f m ) if any geodesics connecting x to y is not singular whenever x ∈ Γ and y ∈ Γ c . Assume (H) . Let Γ ⊂ R n be a compact set and suppose that there are no singular geodesics for Γ.

	Definition 4.3.7. Finally, in light of Proposition 4.3.5, Lemma 4.3.6, and Theorem 4.3.4, we get the
	following result:
	Corollary 4.3.8.

Then d SR (Γ, •) is locally semiconcave on Γ c .

  z , α(t , z , β)). Notice that, since the coefficients α i are

	106	4.4. PROOF OF THE MAIN RESULT
		β
		x u (t)
	z
	z

  Proof. First of all we notice that by Corollary 4.3.8 and Remark 4.5.1 the minimum time function τ Γ is locally semiconcave on Γ c . Let {λ i } i ⊂ (0, ∞) with λ i → 0+ and write for simplicity Γ i in place of Γ λ i . It is easy to see that the level sets Γ i are compact. Hence, for all i 1 there exist open sets A i , B i , D ⊂ R n such that D is bounded andΓ ⊂ B i ⊂ int Γ i & Γ i ⊂ A i ⊂ D.(4.45)Putting O i = A i \B i , it follows from (4.45) that ∂Γ i ∩ O i is bounded and ∂Γ i = {y ∈ R n | τ Γ (y) = λ i } ⊂ O i . So, ∂Γ i ∩ O i ⊂ O i .Consider now the eikonal equation |F (y) * ∇τ Γ

	N P Γ c t	(x x 0 ,ū (t)).	(4.44)

), ū(t)) * p(t) for a.e. t ∈ [0, τ Γ (x 0 )] satisfy the sensitivity relation

-p(t) ∈ ∂ P τ Γ (x x 0 ,ū (t)) ∀t ∈ [0, τ Γ (x 0 )), (4.43)

and the transversality condition

p(τ Γ (x 0 )) ∈ Lim sup t → τ Γ (x 0 )-

  (t)f i (∆x(t)) = m i=1 v i (t)f i (x(t))+ m i=1 v i (t) (f i (∆x(t)) -f i (x(t))), letting

	118			4.6. APPENDIX
	Since m i=1 v i    chapter 4 ξ(t) = ∆x(t) -x(t)    A(t) = m i=1 u i (t)df i (x(t))      B(t) = (f 1 (x(t))|...|f m (x(t))),
	the equation (4.53) becomes
				ξ = Aξ + Bv + R,	(4.54)
	where
	R(t) =
			m
			+	u i (t)
			i=1
				(4.55)
	Observe that there exists M 0 such that, by (4.55) and (4.51),	(4.52)
				m
	|R(t)|			|u i (t)|
	i=1 |u i (t)|. Solving the system (4.54) with initial condition ξ(0) = h we have that m m Observe that M C (|h| + v L 2 ) i=1 |v i (t)| + M C (|h| + v L 2 ) 2 i=1 0 f Hence we can rewrite (4.52) as ξ(t) = X(t)h + t X(t)X(s) -1 B(s)v(s) ds	(4.56)
	x y+h,u+v (t) -x y,u (t)
	m
	=	v i (t)f i (∆x(t))
	i=1
	+	m	u (4.53)
	i=1

y+h,u+v (t) -x y,u (t)

= m i=1 (u i (t) + v i (t))f i (∆x(t)) -m i=1 u i (t)f i (x(t)) = m i=1 v i (t)f i (∆x(t)) + m i=1 u i (t) (f i (∆x(t)) -f i (x(t))) . i (∆x(t)) -f i (x(t)) = df i (x(t))(∆x(t) -x(t)) + 1 0 (1 -s)d 2 f i ((1 -s)x(t) + s∆x(t))(∆x(t) -x(t)) 2 ds. i (t)df i (x(t))(∆x(t) -x(t)) + m i=1 u i (t) 1 0 (1 -s)d 2 f i ((1 -s)x(t) + s∆x(t))(∆x(t) -x(t)) 2 ds. m i=1 v i (t) (f i (∆x(t)) -f i (x(t))) 1 0 (1 -s)d 2 f i ((1 -s)x(t) + s∆x(t))(∆x(t) -x(t)) 2 ds.

We remark that ∆x and x depend on starting points y + h, y and on controls u + v, u respectively, while the matrices A and B depend only on y and u. By Lemma 4.6.2, there exists C = C (δ) > 0 such that for all t ∈ [0, T ]

|∆x(t) -x(t)| C(|h| + v L 2 ) ∀(h, v) ∈ B δ (y) × B L 2 (u, δ).

m i=1 |v i (t)| |f i (∆x(t)) -f i (x(t))| + M C (|h| + v L 2 ) 2

CHAPTER 3 HAMILTON-JACOBI-BELLMAN EQUATIONS WITH TIME-MEASURABLE DATA AND INFINITE HORIZON

Abstract. In this chapter we investigate the existence and uniqueness of weak solutions of the nonautonomous Hamilton-Jacobi-Bellman equation on the domain (0, ∞) × Ω. The Hamiltonian is assumed to be merely measurable in time variable and the open set Ω may be unbounded with nonsmooth boundary. The set Ω is called here a state constraint. When state constraints arise, then classical analysis of Hamilton-Jacobi-Bellman equation lacks appropriate notion of solution because continuous solutions could not exist. In this work we propose a notion of weak solution for which, under a suitable controllability assumption, existence and uniqueness theorems are valid in the class of lower semicontinuous functions vanishing at infinity.

Introduction

The notion of weak (or viscosity) solution to a first-order partial differential equation was introduced in the pioneering works [START_REF] Crandall | Some properties of viscosity solutions of Hamilton-Jacobi equations[END_REF][START_REF] Crandall | Viscosity solutions of Hamilton-Jacobi equations[END_REF][START_REF] Lions | Generalized solutions of Hamilton-Jacobi equations[END_REF] by Crandall, Evans, and Lions to investigate stationary and evolutionary Hamilton-Jacobi-Bellman (H-J-B) equations, using sub/super solutions involving superdifferentials and subdifferentials of continuous function associated to C 1 test functions. In particular, they obtained existence and uniqueness results in the class of continuous functions for the Cauchy problem associated to the following H-J-B equation

Proof. We prove (i). Suppose (a). Fix (t, x) ∈ dom W ∩ (((0, ∞)\C) × A) and let (p t , p x , q) ∈ T epi W (t, x, W (t, x)) -. From (3.1) and (3.21), we have (1,

x, ū) 0, and so

Suppose next that (b) is satisfied and let j ∈ N + . By the separation theorem, (b) implies that

with µ(C j ) = 0, and for all t 0 ∈ [0, j]\C j and all (x 0 , v 0 ) ∈ P (t 0 ) := epi W (t 0 , •) there exists a G-trajectory (x, v)(•) on [t 0 , j], with (x(t 0 ), v(t 0 )) = (x 0 , v 0 ), satisfying (x, v)(t) ∈ P (t) for all t ∈ [t 0 , j] and

Taking v 0 = W (t 0 , x 0 ), by the measurable selection theorem we conclude that there exist two measurable functions u(•) and r(•), with u(t) ∈ U (t) and r(t)

So, dividing by t -t 0 the last inequality and passing to the lower limit as t → t 0 +, (3.21) follows for C = ∪ j∈N + C j .

To prove (ii), suppose that (h) holds true. Assuming (a) and arguing similarly to (i), we can conclude that there exists C ⊂ (0, ∞), with µ(C ) = 0, such that -p t + H(t, x, -p x , -q) 0 for all (p t , p x , q) ∈ T epi W (t, x, W (t, x)) -and all (t, x) ∈ dom W ∩ (((0, ∞)\C ) × int A). Now, assume (b) and let j ∈ N + . From Remark 3.4.1-(iii), Proposition 3.4.2, and [FPR95, Theorem 2.9] applied to the set-valued map G(j -•, •), and the measurable selection theorem, we can find a subset

with µ(C j ) = 0, such that for any (t 0 , x 0 ) ∈ ((1/j, j]\C j ) × int A and any u 0 ∈ U (t 0 ) there exist t 1 ∈ [1/j, t 0 ) and a trajectory-control pair ((x, v), (u, r))(•) satisfying (3.17)

Hence, dividing by t 0 -s, passing to the lower limit as s → t 0 -, and since u 0 ∈ U (t 0 ) is arbitrary, we have (3.22) after taking C = ∪ j∈N + C j .

Proof of Theorem 3.3.3. By Proposition 3.4.9, (ii) is equivalent to the following:

and t epi W (t, •) is locally absolutely continuous.

Furthermore, the implication (i) =⇒ (iii) follows from Proposition 3.4.4. We have to prove (ii) =⇒ (i). Fix (t 0 , x 0 ) ∈ (0, ∞) × A.

We first show that W (t 0 , x 0 ) V (t 0 , x 0 ). If W (t 0 , x 0 ) = +∞, then W (t 0 , x 0 ) V (t 0 , x 0 ). Suppose next that (t 0 , x 0 ) ∈ dom W . From the separation theorem and (3.6) we deduce (3.23) for all (t, x) ∈ dom W ∩ (([0, ∞)\C ) × A). By [FP96, Corollary 3.2] applied with P (t) = epi W (t, •) there exists an absolutely continuous trajectory

We claim that for any j ∈ N + the trajectory X 0 (•) admits an extension on the interval [t 0 , t 0 + j] to a G-trajectory X j (•) satisfying (3.26) on [t 0 , t 0 + j]. We proceed by the induction argument on j ∈ N + . Let j ∈ N + and suppose that X j (•) = (x j (•), v j (•)) satisfies the claim. Then, using (3.23) and applying again [FP96, Corollary 3.2] on the 3.5. LIPSCHITZ CONTINUOUS SOLUTIONS From Lemma 3.4.8 we get W (τ j , y j ) -

So, by (3.28), passing to the limit as j → ∞, using (3.34), (3.35), and the continuity of W , we have W (t 0 , x 0 ) s i t 0 L(s, x(s), ū(s)) ds + ε. Then, passing to the limit as i → ∞ and since ε is arbitrarily small, we get W (t 0 , x 0 ) V (t 0 , x 0 ). Finally, since V = W on (0, ∞)×A, from the continuity of V and W , the conclusion follows.

Appendix

Proof of Lemma 3.4.6. Notice that, by the separation theorem, (3.18) is equivalent to

We claim that g(•) ≡ 0 on (0, τ 1 -τ 0 ]. Indeed, otherwise, we can find T ∈ (0, τ 1 -τ 0 ] with g(T ) > 0. Denoting t * = sup {t ∈ [0, T ] : g(t) = 0}, let ε > 0 be such that ξ(s) ∈ int A and g(s) > 0 for any s ∈ (t * , t * + ε]. Consider s ∈ (t * , t * + ε) where g(•), y(•), and w(•) are differentiable, with y (s) = f (s, y(s), ũ(s)) and w (s) = L(s, y(s), ũ(s)). Let (θ, v) ∈ T graph Q (s, ξ(s), r(s)) and

Then, setting Z = (y(s), w(s)) and Y = (ξ(s), r(s)), we get

From the Gronwall lemma we conclude that g(•) ≡ 0 on [t * , t * + ε], leading to a contradiction. Thus g = 0 and the proof is complete.

there exists a constant

Then

This and (4.9) and (4.11) complete the proof.

Remark 4.4.2. Arguing in a similar way as in the first part of the proof of Lemma 4.4.1 and using Lemma 4.6.2, we have that, for any T > 0, the map

Lemma 4.4.3. Assume (H) and suppose that the problem (??)-(??) does not admit singular minimizers. Then the function

By [LM67, Theorem 8, Chap. 4]), for every i 1 there exists a square integrable control

Without loss of generality we can suppose that for all large i the controls {u i } i are defined on [0, t + 1] putting

We first show that V x0 ( t, x) l. Notice that V x0 ( t, x) = +∞. Let v(•) be an optimal control steering x0 to x in time t. From [CR08, Lemma 3], v(•) is continuous. Moreover, according to Lemma 4.6.6 below, there exists {v j } n j=1 ⊂ C(0, t; R m ) such that the map ϕ : R n → R n , defined by ϕ(β) = n j=1 β j dEt(x 0 , v)(0, v j ), is an isomorphism. Then, from Lemma 4.6.2 and Corollary 4.6.5, we conclude that the map E : R n × R + × R n defined by

is C 1 in a neighborhood of (x 0 , t, 0) and det dE (x 0 , t, 0) = 0. So, applying the Inverse Function Theorem, the map E is open in a neighborhood of ( t, x0 , 0). It means that any point (z i , t i , x i ), sufficiently close to (x 0 , t, x), admits a control w i = v + 104 4.4. PROOF OF THE MAIN RESULT n j=1 β j v j close to v in C(0, t + 1; R m ) such that x z i ,w i (t i ) = x i . By Remark 4.4.2, we have that lim i

) ds for all i, passing to the limit we deduce that l V x0 ( t, x). Consequently l < +∞.

We next prove that V x0 ( t, x) l. By assumptions on φ(•), there exists α, C > 0 such that αr 2 φ(r) for all r C. So

By further extraction of a subsequence and from Gronwall's lemma and the Ascoli-Arzelà theorem, keeping the same notation, we have that u i ū in L 2 (0, t + 1; R m ) and x i (•) converges uniformly on [0, t + 1] to an absolutely continuous trajectory y(

Then, from the convexity of L with respect to the second variable (see [LM67, proof of Theorem 8 Chap. 3]), we deduce that lim i t i 0 L(x i (s), u i (s)) ds t 0 L(y(s), ū(s)) ds. Hence V x0 ( t, x) l.

From the proof of [CR08, Lemma 3], Remark 4.4.2, and Lemma 4.4.3, we get the following compactness result. Lemma 4.4.4. Assume (H) and suppose that the problem (??)-(??) does not admit singular minimizers.

Then, for any nonempty compact subset Γ ⊂ Ω G , we have U x 0 (s, y) = ∅ for all (s, y) ∈ Γ and x 0 ∈ G, and there exists k = k(G, Γ) > 0 such that

We give next an inverse mapping result for the end-point map.

Proposition 4.4.5. Assume (H) and suppose that the problem (??)-(??) does not admit singular minimizers. Let Γ ⊂ Ω G be a nonempty compact subset and define

If for some k > 0

x 0 ∈G U x 0 (Γ) ⊂ L T k , (4.12) 4.6. APPENDIX is continuous.

Proof. Notice that, from continuous dependence on parameters of solutions to ODE's and from the expression of the differential in (4.58), for any τ > 0 and every bounded subset A ⊂ R n × L 2 (0, τ ; R m ), the maps {s → dE s (y, u) | s ∈ [0, τ ]} are equicontinuous for (y, u) ∈ A and the constant C that appears in (4.59) may be taken the same for 0 T τ , (y 1 , u 1 ), (y 2 , u 2 ) ∈ A.

The following result is well known.

Lemma 4.6.6. Let X be a separable normed space and Φ : X → R n be a linear, continuous, and surjective operator. Consider {x i } i dense in X.

Then there exist linearly independent vectors x 1 , ..., x n such that Φ : W → R n is an isomorphism, where W = span {x 1 , ..., x n }.

Proof. Let {x i } i be dense in X. Then there exists a countable increasing family of finite dimensional subspaces W k = span {x i } k i=1 such that ∪ k W k = X. So, Φ(W k ) is a finite dimensional subspace and Φ(W k ) is increasing. Hence there exists k 0 such that Φ(W k 0 ) = R n . So we can choose n linearly independent vectors x 1 , ..., x n in W k 0 such that Φ is onto and injective on W = span {x 1 , ..., x n }.