Fernando A C C Fontes

Helene Frankowska
email: helene.frankowska@imj-prg.fr

NORMALITY AND NONDEGENERACY FOR OPTIMAL CONTROL PROBLEMS WITH STATE CONSTRAINTS

In this paper, we investigate normal and nondegenerate forms of the maximum principle for optimal control problems with state constraints. We propose new constraint qualifications guaranteeing nondegeneracy and normality, that have to be checked on smaller sets of points of an optimal trajectory than those in known sufficient conditions. In fact, the constraint qualifications proposed impose the existence of an inward pointing velocity just on the instants of time for which the optimal trajectory has an outward pointing velocity. optimal control and maximum principle and state constraints and constraint qualifications and normality and degeneracy and nonsmooth analysis and oriented distance.

Introduction

We consider optimal control problems with pathwise state constraints. For these problems, we study necessary conditions of optimality in the form of a maximum principle that, in some situations, are able to provide more information, by avoiding the abnormality or the degeneracy phenomena.

We say that the maximum principle is abnormal if the scalar multiplier associated with the objective function (ahead denoted by λ) is equal to zero. In this case, the necessary conditions cannot use the information of the objective function to select minimizers.

The degeneracy phenomenon arises when the state constraint is active at the initial time. Then, a possible choice of multipliers is offered by the socalled degenerate multipliers, for which the adjoint multiplier (denoted by q) and therefore the Hamiltonian are equal to zero for almost every time. This implies that all conditions in the maximum principle are satisfied for any candidate to solution we might test. Thus, with the degenerate multipliers, the maximum principle is useless to select minimizers.

Nondegenerate and normal forms of the maximum principle are established for problems that satisfy a suitable constraint qualification. In addition to helping, in some situations, to find minimizers or to eliminate some candidates for optimality, the normal maximum principle is also useful to establish regularity properties of the minimizes (as in e.g. [START_REF] Ferreira | On the regularity of optimal controls for a class of problems with state constraints[END_REF][START_REF] Frankowska | Lipschitzianity of optimal trajectories for the Bolza optimal control problem[END_REF][START_REF] Galbraith | Lipschitz continuity of optimal controls for state constrained problems[END_REF]) or to deduce second order optimality conditions (see e.g. [START_REF] Frankowska | A second-order maximum principle in optimal control under state constraints[END_REF][START_REF] Hoehener | Variational approach to second-order optimality conditions for control problems with pure state constraints[END_REF][START_REF] Hoehener | Feasible perturbations of control systems with pure state constraints and applications to second-order optimality conditions[END_REF]).

The degeneracy of the maximum principle for state constrained problems has already been well identified and studied in the literature even for nonsmooth data, see e.g. [START_REF] Arutyunov | Optimality Conditions: Abnormal and Degenerate Problems[END_REF][START_REF] Arutyunov | Investigation of the degeneracy phenomenon of the maximum principle for optimal control problems with state constraints[END_REF][START_REF] Arutyunov | The maximum principle in a problem with phase constraints[END_REF][START_REF] Dubovitskii | Necessary conditions for strong minimum in optimal control problems with degeneration of endpoint and phase constraints[END_REF][START_REF] Ferreira | When is the maximum principle for state constrained problems nondegenerate?[END_REF][START_REF] Ferreira | Nondegenerate necessary conditions for nonconvex optimal control problems with state constraints[END_REF][START_REF] Rampazzo | Degenerate optimal control problems with state constraints[END_REF][START_REF] Lopes | On constraint qualifications for nondegenerate necessary conditions of optimality applied to optimal control problems[END_REF][START_REF] Lopes | An integral-type constraint qualification to guarantee nondegeneracy of the maximum principle for optimal control problems with state constraints[END_REF]. Previous investigations on nondegenerate and normal forms of the maximum principle involved various types of constraint qualifications. The relations between some of them is discussed for instance in [START_REF] Frankowska | Normality of the maximum principle for absolutely continuous solutions to Bolza problems under state constraints[END_REF][START_REF] Frankowska | On relations of the adjoint state to the value function for optimal control problems with state constraints[END_REF][START_REF] Lopes | On constraint qualifications for nondegenerate necessary conditions of optimality applied to optimal control problems[END_REF]. In [START_REF] Lopes | An integral-type constraint qualification to guarantee nondegeneracy of the maximum principle for optimal control problems with state constraints[END_REF], an integral-type constraint qualification was introduced. There, it was shown that, to avoid degeneracy, an inward pointing condition has to be satisfied for some, not all, instants of a neighborhood of the initial time.

In the present paper, we introduce new constraint qualifications that just have to be satisfied on a subset of times at which the optimal trajectory has an outward pointing velocity. Furthermore, we remove the convexity assumption on the problem data imposed in [START_REF] Lopes | An integral-type constraint qualification to guarantee nondegeneracy of the maximum principle for optimal control problems with state constraints[END_REF]. We also provide, in a corollary, additional sufficient conditions to avoid having the adjoint multiplier q equal to zero, or to avoid having q and λ both equal to zero. In this way, we show that a nondegeneracy condition derived previously in a different context in [START_REF] Cernea | A connection between the maximum principle and dynamic programming for constrained control problems[END_REF] holds true also in our case.

Normality for optimal control problems with state constraints has also been extensively studied, see for instance [START_REF] Frankowska | Normality of the maximum principle for absolutely continuous solutions to Bolza problems under state constraints[END_REF][START_REF] Frankowska | Optimal synthesis and normality of the maximum principle for optimal control problems with pure state constraints[END_REF][START_REF] Frankowska | Inward pointing trajectories, normality of the maximum principle and the non occurrence of the Lavrentieff phenomenon in optimal control under state constraints[END_REF][START_REF] Lopes | Normal forms of necessary conditions for dynamic optimization problems with pathwise inequality constraints[END_REF][START_REF] Rampazzo | Degenerate optimal control problems with state constraints[END_REF] and references therein. In the literature, it was shown that normality might be guaranteed by assuming constraint qualifications with the inward pointing inequalities imposed on neighborhoods of times τ for which the optimal trajectory belongs to the boundary of the state constraint. The new constraint qualification CQn requires the inward pointing inequality to be satisfied only for times t < τ from a neighborhood of τ at which the trajectory has an outward pointing velocity. We discuss here few consequences of this constraint qualification. The inward pointing condition from [START_REF] Frankowska | Inward pointing trajectories, normality of the maximum principle and the non occurrence of the Lavrentieff phenomenon in optimal control under state constraints[END_REF] is in the same spirit, however it uses also t > τ .

The constraints qualifications here proposed help also to understand the relations between previous constraints qualifications reported in the literature. Namely, we discuss conditions involving different generalized gradients and, in the smooth case, we compare different types of constraint qualifications.

The outline of the paper is as follows. In Section 2, we recall some notions of nonsmooth analysis, the nonsmooth maximum principle and compare various gradients of distance functions. In Section 3, we state our main results.

Preliminaries

2.1. Notations and Definitions. Throughout B denotes the closed unit ball in R n , S n-1 the unit sphere in R n , B(x, r) the closed ball in R n of centre x and radius r, p • v the usual scalar product of p, v ∈ R n and | • | the Euclidean norm. The sets conv K, conv K, bdy K, K c and int K stand for the convex hull, closed convex hull, boundary, complement and interior of a set K ⊆ R n , respectively.

For a Borel measure µ on [0, 1], we denote by supp{µ} its support and we use (I) for the Lebesgue measure of a Lebesgue measurable set I ⊆ R. The product σ-algebra generated by the Lebesgue subsets L of [0, 1] and the Borel subsets of R m is denoted by L × B m and the norm in the space of essentially bounded functions from

[0, 1] into R n by • L ∞ . Finally, W 1,1 ([0, 1]; R n ) denotes the space of absolutely continuous functions from [0, 1] into R n .
Let Θ be a metric space. For a family of subsets A τ ⊆ R n with τ ∈ Θ and any τ ∈ Θ, the upper set limit of A τ at τ is defined by

Limsup τ →τ A τ := {v ∈ R n : ∃ τ i → τ , v i ∈ A τ i such that lim i→∞ v i = v}.
We recall next the notion of limiting normal cone to a closed set K ⊆ R n at x ∈ K. Define first the contingent cone to K at x by

T K (x) = Limsup r→0+ 1 r (K -x)
and consider its negative polar

N K (x) = {p | p • v ≤ 0, ∀ v ∈ T K (x)} .
The limiting normal cone to K at x is defined by

N K (x) := Limsup x→x N K (x).
The negative polar of N K (x) is the Clarke tangent cone to K at x:

T K (x) := {y ∈ R n : y • p ≤ 0 for all p ∈ N K (x)}.
Given a lower semicontinuous function f : R n -→ R ∪ {∞}, the limiting subdifferential of f at a point x ∈ R n with f (x) < +∞ is the set

∂f (x) := {η ∈ R n : (η, -1) ∈ N epi f (x, f (x))},
where epi f := {(x, α) : α ≥ f (x)}. If f is differentiable at x, then we denote by ∇f (x) its gradient.

For a locally Lipschitz h : R n → R the reachable gradient of h at x is defined by

∂ * h(x) := Limsup y→x {∇h(y)}.
Recall that conv ∂ * h(x) is equal to the generalized gradient ∂ C h(x) of h at x and that conv ∂h(x) = ∂ C h(x), see for instance [START_REF] Vinter | Optimal control[END_REF]Proposition 4.7.6].

The reachable hybrid subdifferential of h at x is defined as

∂ * > h(x) := {ζ | ∃x i → x such that h(x i ) > 0 ∀ i and lim i→∞ ∇h(x i ) = ζ} ⊆ ∂h(x)
and the hybrid subdifferential ∂ > h(x) is defined as

∂ > h(x) := conv ∂ * > h(x).
Observe that

∂ * > h(x) ⊆ ∂ * h(x).
Similarly, for a locally Lipschitz f : R n → R m and x ∈ R n the reachable Jacobian is defined by

∂ * f (x) = Limsup y→x {f (y)},
where f (y) denotes the Jacobian of f at y (which exists a.e. in R n ).

We refer to [START_REF] Aubin | Set-Valued Analysis[END_REF][START_REF] Vinter | Optimal control[END_REF] for further concepts of nonsmooth and set-valued analysis, the last one using notations similar to those of the present paper.

2.2. Generalized Gradients of Distance Functions. For a closed nonempty set K ⊆ R n and x ∈ R n , the distance d K (x) from x to K is given by d K (x) := min y∈K |x -y|.

For K different from R n , the oriented distance function is defined by

d(x) := d K (x) -d K c (x).
When K = R n we set d(x) = 0 for every x ∈ R n . These two functions are Lipschitz continuous with Lipschitz constant equal to one.

Therefore, we get ∂d(x) ⊆ B. Also,

∂ * > d K (•) = ∂ * > d(•). By [27, Example 8.53] we know that (1) ∂d K (x) = N K (x) ∩ B, ∀ x ∈ K. Proposition 2.1. Let x ∈ bdy K. If 0 / ∈ ∂ > d K (x), then T K (x)
has a nonempty interior and is equal to the negative polar cone of

∂ * > d K (x). Proof. If 0 / ∈ ∂ > d K (x), then the set R + (∂ > d K (x)
) is closed, see for instance [27, 3.48 (a)]. Consequently R + (∂ > d K (x)) is a closed convex cone spanned by a convex and compact set not containing zero. For this reason its negative polar cone has a nonempty interior.

By [6, Proposition 2.4 and Corollary 2.5],

∂ C d K (x) = conv{0, ∂ * > d K (x)}. Thus from (1) we deduce that ∂ > d K (x) ⊆ conv ∂d K (x) = conv(N K (x)∩B) = ∂ C d K (x) = {λ∂ > d K (x) | λ ∈ [0, 1]}. This implies that (2) R + ∂ > d K (x) = conv N K (x). Since T K (x) is equal to the negative polar of conv N K (x) the proof follows. Lemma 2.2. Let x ∈ bdy K be such that int T K (x) = ∅. Then ∂ * d K (x) = ∂ * > d K (x) ∪ {0}
and

∂ C d K (x) = {λ∂ > d K (x) | λ ∈ [0, 1]} ⊆ B. Furthermore, N K (x) ∩ S n-1 = ∂ * > d K (x) = ∂d K (x) ∩ S n-1 and ∂ * d(x) = ∂ * > d K (x) = ∂ * > d(x).
In particular,

∂ C d(x) = ∂ > d K (x).
Proof. Fix any v ∈ int T K (x). By [START_REF] Aubin | Set-Valued Analysis[END_REF], there exists ε > 0 such that

y + [0, ε](v + εB) ⊆ K, ∀ y ∈ K ∩ B(x, ε).
Thus for all small s > 0 and any b ∈ B,

d K (y + s(v + εb)) = 0. This implies that ∇d K (y), v + εb = 0 for any b ∈ B whenever d K is differentiable at y ∈ K ∩ B(x, ε).
Hence for any such y we have ∇d K (y) = 0. Consequently, if x i → x are such that d K (x i ) = 0 and ∇d K (x i ) exist and converge to some

ζ ∈ R n , then ζ = 0. Clearly, we have ∂ * > d K (x) ⊆ ∂ * d K (x). Let x i ∈ int K converge to x. Then ∇d K (x i ) = 0 and therefore 0 ∈ ∂ * d K (x). Pick any ζ ∈ ∂ * d K (x) and let x i → x be such that ∇d K (x i ) exist and converge to ζ when i → ∞. If d K (x i k ) = 0 for a subsequence {x i k }, then ζ = 0 by the first part of the proof. Otherwise ζ ∈ ∂ * > d K (x)
. This implies the first equality of our Lemma.

The second relation can be easily deduced from the first one.

Let ζ ∈ ∂ * > d K (x) and x i → x be such that x i / ∈ K and ∇d K (x i ) → ζ. Then for some y i ∈ bdy K, ∇d K (x i ) ∈ N K (y i ) ∩ S n-1 and we deduce that ζ ∈ N K (x) ∩ S n-1 . On the other hand, for any ζ ∈ ∂ > d K (x) \ ∂ * > d K (x) we have |ζ| < 1, because ∂ * > d K (x) ⊆ S n-1 and B is a strictly convex set. Hence ∂ C d K (x) ∩ S n-1 = ∂ * > d K (x). Consequently, by (1), ∂ * > d K (x) ⊆ N K (x) ∩ S n-1 ⊆ ∂d K (x) ∩ S n-1 ⊆ ∂ C d(x) ∩ S n-1 = ∂ * > d K (x).
To prove the last statement, observe that

∂ * > d K (x) = ∂ * > d(x) ⊆ ∂ * d(x). Let x i ∈ K be such that d(•) is differentiable at x i
and x i converge to x while ∇d(x i ) converge to some ζ. Then there exists a unique projection y i of x i on bdyK. Using that for all large i, int T K (y i ) = ∅, applying the same proof as the one of Proposition 5 in [START_REF] Frankowska | Mazzola Discontinuous solutions of Hamilton-Jacobi-Bellman equation under state constraints[END_REF] on a neighborhood of x, we deduce that ∇d(y

i ) ∈ N K (y i )∩S n-1 for all large i. Hence ζ ∈ N K (x)∩S n-1 = ∂ * > d K (x).
2.3. Maximum Principle. Consider the following optimal control problem with state constraints:

(P )            Minimize g(x(1)) subject to ẋ(t) = f (t, x(t), u(t)) , u(t) ∈ U (t) a.e. t ∈ [0, 1] x(0) = x 0 , x(1) ∈ K 1 x(t) ∈ K for all t ∈ [0, 1].
The data for this problem comprise functions g :

R n → R, f : [0, 1] × R n × R m → R n , an initial state x 0 ∈ R n , sets K, K 1 ⊆ R n of
the state and the end-point constraints, and a set-valued map U : [0, 1] ⇒ R m . The set of control functions for (P ) is

U := {u : [0, 1] → R m : u is Lebesgue measurable, u(t) ∈ U (t) a.e. t ∈ [0, 1]}.
We say that a trajectory-control pair (x, ū) of the above system is a local minimizer of (P) if for some δ > 0 and every trajectory-control pair (x, u) satisfying x -x L ∞ < δ we have g(x(1)) ≤ g(x(1)).

Using the distance or the oriented distance functions, the state constraint x(t) ∈ K can be written equivalently as the inequality constraint

d(x(t)) ≤ 0 for all t ∈ [0, 1] or as d K (x(t)) ≤ 0 for all t ∈ [0, 1].
We impose the following hypotheses which refer to a fixed local minimizer (x, ū) and a δ neighborhood of x.

H1: The function (t, u) → f (t, x, u) is L × B m measurable for each x and U (•) is measurable with closed nonempty images. H2: There exists The maximum principle for problems with state constraints has been reported in different versions with different degrees of generality (see e.g. [START_REF] Pontryagin | The mathematical theory of optimal processes[END_REF][START_REF] Ya | Extremum problems under constraints[END_REF][START_REF] Vinter | A maximum principle for nonsmooth optimal-control problems with state constraints[END_REF]). The version in the next theorem can be deduced from [28, pp. 329 and 204] applied to the state constraints described by an inequality involving the oriented distance function d(•).

C f ≥ 0 such that |f (t, x, u) -f (t, x , u)| ≤ C f |x -x | for x, x ∈ x(t) + δ B, u ∈ U (t)
Theorem 2.3. Let (x, ū) be a local minimizer and assume H1-H5. Then, there exist p ∈ W 1,1 ([0, 1]; R n ), a non-negative (finite) Borel measure µ on [0, 1] and a scalar λ ∈ {0, 1} such that

(3) µ ([0, 1]) + p L ∞ + λ = 0, (4) -ṗ(t) ∈ q(t)conv ∂ * x f (t, x(t), ū(t)) a.e. t ∈ [0, 1], -q(1) ∈ λ∂g(x(1)) + N K 1 (x(1)), γ(t) ∈ ∂ > d(x(t)) µ -a.e., supp{µ} ⊆ {t ∈ [0, 1] | x(t) ∈ bdy K},
and, for almost every t ∈ [0, 1], and all u ∈ U (t) [START_REF] Cernea | A connection between the maximum principle and dynamic programming for constrained control problems[END_REF] q(t)

• (f (t, x(t), ū(t)) -f (t, x(t), u)) ≥ 0,
where

q(t) :=        p(t) + [0,t[ γ(s)µ(ds), t ∈ [0, 1[ p(1) + [0,1] γ(s)µ(ds), t = 1. The relation γ(t) ∈ ∂ > d(x(t)) is equivalent to γ(t) ∈ ∂ > d K (x(t)). It follows from Lemma 2.2 that at every t ∈ [0, 1] such that int T K (x(t)) is nonempty, it can be equivalently written as γ(t) ∈ ∂ C d(x(t)).
Since µ is a finite Borel measure on [0, 1], it is regular and therefore q(•) is left continuous on ]0, 1[.

2.4.

Abnormality and Degeneracy Phenomena. The above maximum principle is called normal if λ = 1.

In this paper we say that a maximum principle is nondegenerate if

µ(]0, 1]) + q L ∞ + λ = 0.
The degeneracy phenomenon arises when the state constraint is active at the initial time, i.e. when x 0 ∈ bdy K. Then a possible choice of multipliers, here called degenerate multipliers, is

λ = 0, µ = δ {0} , p ≡ -ξ, with ξ ∈ ∂ > d(x(0)),
where δ {0} is the Dirac unit measure concentrated at 0. We note that with such multipliers, the expression p(t) + [0,t[ ξµ(ds), which features in the conditions of the maximum principle, vanishes for almost all times. So, the maximum principle holds true for any pair (x,ū) we might test. Such multipliers are useless to identify minimizers. Previous works on the nondegeneracy of the maximum principle mainly differ by the constraint qualification used, in addition to assumptions on the data of the problem. We can identify, in the literature, four types of constraint qualifications (CQ), which we adapt here to the context of state constraints x(t) ∈ K for all t ∈ [0, 1]:

1.: Inward pointing velocity CQ.

If x 0 ∈ bdyK, then there exist δ > 0, > 0 and a control function û ∈ U such that max

γ∈∂ > d K (x 0 ) γ • f (t, x 0 , û(t)) < -δ a.e. t ∈ [0, [. 2
.: CQ involving the optimal control. If x 0 ∈ bdyK, then there exist δ > 0, > 0 and a control function û ∈ U such that max

γ∈∂ > d K (x 0 ) γ • (f (t, x 0 , û(t)) -f (t, x 0 , ū(t)) < -δ a.e. t ∈ [0, [.

3.:

Integral-type CQ. If x 0 ∈ bdyK, then there exist δ > 0, > 0 and a control function û ∈ U such that

t 0 max γ∈∂ > d K (x 0 ) γ • (f (s, x 0 , û(s)) -f (s, x 0 , ū(s)) ds < -δt ∀ t ∈ [0, [. 4
.: CQ involving a strictly feasible initial trajectory. If x 0 ∈ bdyK, then there exists > 0 such that

x(t) ∈ int K ∀ t ∈]0, [.
The relation between the first two types of constraint qualification is discussed for instance in [START_REF] Frankowska | Normality of the maximum principle for absolutely continuous solutions to Bolza problems under state constraints[END_REF][START_REF] Frankowska | Optimal synthesis and normality of the maximum principle for optimal control problems with pure state constraints[END_REF][START_REF] Lopes | On constraint qualifications for nondegenerate necessary conditions of optimality applied to optimal control problems[END_REF], where some bibliographical references for each type of constraint qualifications are provided. The third type is described in [START_REF] Lopes | An integral-type constraint qualification to guarantee nondegeneracy of the maximum principle for optimal control problems with state constraints[END_REF]. The fourth type appears, for instance, in [START_REF] Ferreira | When is the maximum principle for state constrained problems nondegenerate?[END_REF].

Let us underline that, by Proposition 2.1, the constraint qualifications 1, 2, 3 imply that int T K (x 0 ) is nonempty. Thus, by Lemma 2.2, the first three constraints qualifications can be written in an equivalent form by replacing

∂ > d K (x 0 ) by ∂ * d(x 0 ) or by ∂ C d(x 0 ).

Nondegeneracy and Normality

We start by showing how the previous maximum principle can be strengthened to avoid the degeneracy by introducing a new type of constraint qualification.

For any α ∈]0, 1] define

D(α) := {r ∈ [0, α] : max ξ∈∂ * d(x(r)) ξ • f (r, x(r), ū(r)) ≥ 0}.
CQd: (Constraint Qualification to avoid degeneracy) If x 0 ∈ bdyK and for any α ∈]0, 1] we have (D(α)) > 0, then assume ∃ δ > 0 such that ∀ ∈]0, 1], there exists a (Lebesgue measurable) set F ( ) ⊂ D( ) with (F ( )) > 0 satisfying [START_REF] Clarke | Generalized gradients and applications[END_REF] inf

u∈U (t) max γ∈∂ * d(x 0 ) γ • (f (t, x 0 , u) -f (t, x 0 , ū(t))) < -δ a.e. t ∈ F ( ).
Remark 3.1. If (F ( )) > 0, then (6) implies that the interior of T K (x 0 ) is nonempty. Indeed, by [START_REF] Clarke | Generalized gradients and applications[END_REF], there exists t ∈ [0, 1] such that

inf u∈U (t) max γ∈∂ * d(x 0 ) γ • (f (t, x 0 , u) -f (t, x 0 , ū(t))) < -δ.
Hence, by Proposition 2.1, int T K (x 0 ) = ∅. Consequently, by Lemma 2.2, we can replace ∂ * d(x 0 ) in ( 6) by ∂ * > d K (x 0 ). Since the scalar product is bilinear, CQd takes then the form of a more familiar constraint qualification :

CQd : If x 0 ∈ bdyK and (D(α)) > 0 for any α ∈]0, 1], then suppose that ∃ δ > 0 such that ∀ ∈]0, 1] there exists F ( ) ⊂ D( ) with (F ( )) > 0 and

inf u∈U (t) max γ∈∂ > d K (x 0 ) γ • (f (t, x 0 , u) -f (t, x 0 , ū(t))) < -δ a.e. t ∈ F ( ).
We note that CQd requires less than the constraint qualifications of type 2 from Section 2.3 because the relevant inequality does not have to be satisfied a.e. in the whole time interval [0, ], but just on a subset of positive measure of times for which the optimal trajectory has an outward pointing velocity, i.e. when max ξ∈∂ * d K (x(r)) ξ • f (r, x(r), ū(r)) ≥ 0.

To illustrate the difference between constraint qualifications we provide next an elementary one dimensional example of a problem where even the end point constraints are absent and the inward pointing velocity conditions (CQ) are not verified while CQd holds true.

Example 3.2. For all t ∈ [0, 1] define

U (t) =              {2, 3} if t ∈ [2 -k , 2 -k + 2 -(k+1) [ k = 10 m {-1, -3} if t ∈ [2 -k + 2 -(k+1) , 2 -(k-1) [ k = 10 m {1, 2} if t ∈ [1/2, 1] {0} otherwise .
In the above m = 1, 2, ... are natural numbers. Let K = [0, ∞[= K 1 , g(z) = z and f (t, x, u) = u. Consider the optimal control ū of the Mayer problem (P) for these data given by

ū(t) =              3 if t ∈ [2 -k , 2 -k + 2 -(k+1) [ k = 10 m -3 if t ∈ [2 -k + 2 -(k+1) , 2 -(k-1) [ k = 10 m 1 if t ∈ [1/2, 1] 0 otherwise , where m = 1, 2, .... Then ∂ * d K (0) = {-1}.
It is clear that CQd is satisfied, while none of (CQ) mentioned above holds true.

We state our first main result.

Theorem 3.3. Let (x, ū) be a local minimizer and assume that H1-H5, CQd are satisfied. Then, the maximum principle of Theorem 2.3 holds true with the nontriviality condition (3) strengthened to

(7) µ(]0, 1]) + q L ∞ + λ > 0.
Remark 3.4. Since q(•) is left continuous on ]0, 1[, if q(•) = 0 a.e., then q(t) = 0 for all t ∈]0, 1[. In this case, the maximum principle (5) holds true for all t ∈]0, 1[ and u ∈ U (t). Furthermore, from the adjoint equation it follows that p(•) ≡ p(0). Since p(0) + [0,t[ γ(s)µ(ds) = 0 for all t ∈]0, 1[, we deduce that for all 0 < t 1 < t 2 < 1, [t 1 ,t 2 [ γ(s)µ(ds) = 0 and, because µ is regular, that for all 0 < t 1 < t 2 < 1 (8)

[t 1 ,t 2 ]
γ(s)µ(ds) = 0.

Assume that int T K (x(t)) = ∅ for every t ∈ [0, 1] satisfying x(t) ∈ bdy K. If µ(]0, 1[) > 0, then the above implies that q L ∞ = 0. Indeed, if we have q L ∞ = 0, then, using Lemma 2.2, that µ is nonnegative and that ∂ > d(x(•)) is upper semicontinuous on the compact interval [0, 1], we derive a contradiction with (8) for a choice of 0 < t 1 < t 2 < 1.

We next observe that if µ(]0, 1]) > 0 and

(9) convN K (x(1)) ∩ (-N K 1 (x(1))) = {0},
then q L ∞ +λ = 0. Indeed, assume for a moment that q L ∞ +λ = 0. Then µ(]0, 1[) = 0 and, therefore, µ({1}) > 0. We deduce that p(•) ≡ -γ(0)µ({0}) and therefore 1)), Lemma 2.2 and (2) yield q(1) ∈ convN K (x( 1)). On the other hand, -q(1) ∈ N K 1 (x( 1)) in contradiction with [START_REF] Ya | Extremum problems under constraints[END_REF]. This proves our claim. Furthermore, q L ∞ = 0 if in addition

q(1) = p(1) + γ(0)µ({0}) + γ(1)µ({1}) = γ(1)µ({1}). Since γ(1) ∈ ∂ > d(x(
(10) convN K (x(1)) ∩ (-∂g(x(1)) -N K 1 (x(1))) = ∅.
Indeed, otherwise λ = 1, µ(]0, 1[) = 0, -q(1) ∈ ∂g(x(1)) + N K 1 (x(1)). In the same way as before we show that q(1) = γ(1)µ({1}) ∈ convN K (x(1)), which contradicts [START_REF] Ferreira | When is the maximum principle for state constrained problems nondegenerate?[END_REF].

Hence we have proved the following corollary.

Corollary 3.5. Let (x, ū) be a local minimizer. Assume H1-H5, CQd, [START_REF] Ya | Extremum problems under constraints[END_REF] and that int T K (x(t)) = ∅ for every t ∈ [0, 1] satisfying x(t) ∈ bdy K. Then, the maximum principle of Theorem 2.3 holds true with the nontriviality condition (3) strengthened to

(11) q L ∞ + λ = 0.
Moreover, if also (10) is satisfied, then q L ∞ = 0.

Two more results in this section provide sufficient conditions for the maximum principle to be normal.

CQn: (Constraint Qualification to guarantee normality) For every τ ∈ ]0, 1] such that x(τ ) ∈ bdy K there exist > 0, δ > 0 satisfying [START_REF] Frankowska | Normality of the maximum principle for absolutely continuous solutions to Bolza problems under state constraints[END_REF] inf

u∈U (t) max γ∈∂ * d(x(τ )) γ • (f (t, x(τ ), u) -f (t, x(τ ), ū(t))) < -δ, for a.e. t ∈ {r ∈ [τ -, τ ] ∩ [0, 1] : max ξ∈∂ * d(x(r)) ξ • f (r, x(r), ū(r)) ≥ 0}. Remark 3.6. As in Remark 3.1, CQn implies that int T K (x(τ )) is nonempty whenever the set {r ∈ [τ -, τ ] ∩ [0, 1] : max ξ∈∂ * d(x(r)) ξ • f (r, x(r), ū(r)) ≥ 0}
has a positive Lebesgue measure. Then, by Lemma 2.2, we can replace ∂ * d(x(τ )) in ( 12) by ∂ > d K (x(τ )) and obtain a more familiar constraint qualification :

CQn : For every τ ∈]0, 1] such that x(τ ) ∈ bdy K there exist > 0, δ > 0 satisfying

inf u∈U (t) max γ∈∂ > d K (x(τ )) γ • (f (t, x(τ ), u) -f (t, x(τ ), ū(t))) < -δ for a.e. t ∈ {r ∈ [τ -, τ ] ∩ [0, 1] : max ξ∈∂ * d(x(r)) ξ • f (r, x(r), ū(r)) ≥ 0}.
We note that CQn is less restrictive than other constraint qualifications proposed in the literature because the relevant inequality does not have to be satisfied a.e. in the whole time interval [τ -, τ ], but just for a.e. instant for which the optimal trajectory has an outward pointing velocity, i.e. when max ξ∈∂ > d(x(r)) ξ • f (r, x(r), ū(r)) ≥ 0. Moreover, as in [22, (8)], in the case we have a candidate for the adjoint state to test, we do not need to test all points τ ∈]0, 1] such that x(τ ) ∈ bdy K, but just the point τ which is the last instant for which the measure µ is active, i.e.

τ := inf {t ∈ [0, 1] : µ{[t, 1]} = 0} .
In the proof of the normality result, in fact, only this point τ is used. So, this condition compares favourably also with [22, (8)].

Theorem 3.7. Let (x, ū) be a local minimizer. Assume H1 -H5 and that the maximum principle of Theorem 2.3 holds true with the strengthened nontriviality condition [START_REF] Dubovitskii | Necessary conditions for strong minimum in optimal control problems with degeneration of endpoint and phase constraints[END_REF].

If CQn is satisfied, then λ + |q(1)| = 0. In particular, if x(1) ∈ int K 1 , then λ = 1.

The above Theorem guarantees normality only when the end point constraint is inactive. It is well known that, even in the absence of state constraints, when the end point constraint is active at x(1), then it may happen that only abnormal maximum principles hold true. We provide next a sufficient condition guaranteeing normality when x(1) ∈ int K. Further investigation of normality conditions can be found in [START_REF] Frankowska | Inward pointing trajectories, normality of the maximum principle and the non occurrence of the Lavrentieff phenomenon in optimal control under state constraints[END_REF]. Theorem 3.8. Let (x, ū) be a local minimizer such that x(1) ∈ int K, H1 -H5 hold true and define t 0 := max {t ∈ [0, 1] : x(t) ∈ bdy K} . Assume that for every measurable selection A(t) ∈ conv ∂ *

x f (t, x(t), ū(t)) the reachable set R L (t 0 ; 1) at time 1 of the linear system ẏ(t) = A(t)y(t) + f (t, x(t), u(t)) -ẋ(t), u(t) ∈ U (t), y(t 0 ) = 0 satisfies R L (t 0 ; 1) ∩ int T K 1 (x(1)) = ∅. If CQn holds true, then for any λ, q, µ satisfying the maximum principle of Theorem 2.3 with the strengthened nontriviality condition [START_REF] Dubovitskii | Necessary conditions for strong minimum in optimal control problems with degeneration of endpoint and phase constraints[END_REF] we have λ = 1.

Example 3.9. Consider the problem

(P E )                Minimize 1 0 x(s) ds subject to ẋ(t) = u(t) a.e. t ∈ [0, 1] x(0) = 0 u(t) ∈ U (t) a.e. t ∈ [0, 1] x(t) ≥ 0 for all t ∈ [0, 1],
where

U (t) =        {3} if t ∈ [2 -1 -2 -k , 2 -1 -2 -k + 2 -(k+1) [ {-1, -3} if t ∈ [2 -1 -2 -k + 2 -(k+1) , 2 -1 -2 -(k+1) [ {1, 2} if t ∈ [1/2, 1].
in which k = 1, 2, . . . are natural numbers.

An optimal solution to this problem is

ū(t) =        3 if t ∈ [2 -1 -2 -k , 2 -1 -2 -k + 2 -(k+1) [ -3 if t ∈ [2 -1 -2 -k + 2 -(k+1) , 2 -1 -2 -(k+1) [ 1 if t ∈ [1/2, 1].
to which corresponds the trajectory x depicted in Fig. 3.9. We rewrite this problem in the form of problem (P), in Mayer form, by defining an additional state variable y satisfying ẏ(t) = x(t), y(0) = 0, and redefining the objective function to be g(x, y) = y. The state constraint set is K = R + × R.

For this problem, CQn is satisfied while previously cited conditions to guarantee normality do not hold.

To see this, observe that at instant τ = 1/2 the state (x(τ ), ȳ(τ )) is on the boundary of K and that ∂ * d K ((x(τ ), ȳ(τ ))) = {(-1, 0)}. Also, for < 1/2, the set

H( ) := {r ∈ [τ -, τ ] ∩ [0, 1] : max ξ∈∂ * d((x(r),ȳ(r))) ξ • f (r, (x(r), ȳ(r)), ū(r)) ≥ 0} coincides with k∈N [2 -1 -2 -k + 2 -(k+1) , 2 -1 -2 -(k+1) [ ∩ [τ -, τ ]. For a.e. t ∈ H( ) we have ū(t) = -3 and inf u∈U (t) γ • (f (t, (x(τ ), ȳ(τ )), u) -f (t, (x(τ ), ȳ(τ )), ū(t))) = -2 < -δ.
This inequality can also be obtained, by the same procedure, for all points τ ∈]0, 1] such that (x(τ ), ȳ(τ )) ∈ bdy K, i.e. points τ = 1/2-2 -(k+1) , k ∈ N. Therefore, CQn is satisfied and by Thm 3.2 the maximum principle holds with λ = 1.

Consider now, in addition, the terminal constraint (x(1), y(1)) ∈ K 1 := [1/2, 1]×R. Since (x(1), ȳ(1)) ∈ int K, we are in the conditions of Thm. 3.3. Observe that t 0 = 1/2, and choosing the control u(t) = 2 for t ∈ [1/2, 1] we reach at time t = 1 the state (1/2, 1/8) ∈ R L (t 0 ; 1). Noting that (1/2, 1/8) ∈ int T K 1 (x(1), ȳ(1)), we deduce that the maximum principle holds with λ = 1. Remark 3.10. When the oriented distance d(•) is continuously differentiable on a neighbourhood of x 0 , then ∂ > d(x 0 ) = {∇d(x 0 )} is the outer unit normal to K at x 0 and we can establish interesting connections between the various types of constraint qualifications discussed previously.

Namely, consider the following constraint qualification: CQd1: If x 0 ∈ bdyK and for every α ∈]0, 1], (D(α)) > 0, then assume ∃ δ > 0 such that ∀ ∈]0, 1], there exists F ( ) ⊂ D( ) with (F ( )) > 0 and a control û ∈ U satisfying

(13) ∇d(x 0 ) • f (t, x 0 , û(t)) < -δ a.e. t ∈ F ( ).
This constraint qualification involving an inward pointing inequality (13) on F ( ) and not depending on the optimal control is akin to the constraint qualifications of type 1 mentioned above. Using continuity of ∇d(•) and H2, H3, we deduce that for a sufficiently small > 0 and for a.e. t ∈ F ( ) we have

∇d(x(t)) • f (t, x(t), û(t)) < -δ/2. Thus ∇d(x(t)) • (f (t, x(t), û(t)) -f (t, x(t), ū(t))) < -δ/2 for a.e. t ∈ F ( ).
By the measurable selection theorem, CQd is equivalent to the following constraint qualification

CQd2: If x 0 ∈ bdyK and for every α ∈]0, 1], (D(α)) > 0, then assume ∃ δ > 0 such that ∀ ∈]0, 1] there exists F ( ) ⊂ D( ) with (F ( )) > 0 and a control û ∈ U satisfying

(14) ∇d(x 0 ) • (f (t, x 0 , û(t)) -f (t, x 0 , ū(t))) < -δ a.e. t ∈ F ( ).
We deduce that in the case when the oriented distance function d(•) is continuously differentiable on a neighborhood of x 0 and either CQd1 or CQd2 holds true, then we can write the maximum principle with the stronger nontriviality condition [START_REF] Dubovitskii | Necessary conditions for strong minimum in optimal control problems with degeneration of endpoint and phase constraints[END_REF].

When K is sufficiently smooth, for the normality results, we may consider the following two constraint qualifications CQn1: Assume that for every τ > 0 such that x(τ ) ∈ bdyK, d(•) is continuously differentiable on a neighborhood of x(τ ) and that there exist > 0, δ > 0 satisfying Using Lipschitz continuity of the functions involved and the same arguments as before, we can show that the constraint qualification CQn1 implies CQn2.

We can, therefore, write a constraint qualification that is less explicitly dependent on the optimal control and might be simpler to check.

CQn1 * : For every τ ∈]0, 1] such that x(τ ) ∈ ∂K there exist > 0, δ > 0, and a control û ∈ U such that ∇d(x(τ )) • f (t, x(τ ), û(t)) < -δ for a.e. t ∈ {r ∈ [τ -, τ ] ∩ [0, 1] : max ξ∈∂ * d(x(r)) ξ • f (r, x(r), ū(r)) ≥ 0}.

Conclusions

This paper is devoted nondegenerate and normal versions of the maximum principle. New constraint qualifications, under which these versions hold true, are introduced. They differ from the existing in the literature constraint qualifications because the inward pointing condition has no longer to be satisfied for almost all times in an interval, but just on subsets for which the optimal trajectory has an outward pointing velocity.

Relations between the new and previous constraint qualifications are discussed. Also, relations between some known constraint qualifications reported in the literature are clarified.

The new constraint qualifications allow to avoid the degeneracy occurring when the state constraint is active at the initial state. Under further conditions, the adjoint multiplier is shown not to be equal to zero. Normality is guaranteed by imposing an inward pointing condition on neighbourhoods of times when the optimal trajectory belongs to the boundary of state constraints. Again it has to be verified just for almost all times for which the optimal trajectory has an outward pointing velocity.

  and a.e. t ∈ [0, 1]. H3: There exists C u ≥ 0 such that |f (t, x, u))| ≤ C u for x ∈ x(t) + δ B, u ∈ U (t), and t ∈ [0, 1]. H4: The function g is Lipschitz on x(1) + δ B. H5: The sets K and K 1 are closed.

Figure 1 .

 1 Figure 1. Optimal trajectory x corresponding to the control ū in the example.

  inf u∈U (t) ∇d(x(τ )) • f (t, x(τ ), u) < -δ for a.e. t ∈ {r ∈ [τ -, τ ] ∩ [0, 1] : max ξ∈∂ * d(x(r)) ξ • f (r, x(r), ū(r)) ≥ 0}and CQn2: Assume that for every τ > 0 such that x(τ ) ∈ bdyK, d(•) is continuously differentiable on a neighborhood of x(τ ) and that there exist > 0, δ > 0 satisfyinginf u∈U (t) ∇d(x(τ )) • (f (t, x(τ ), u) -f (t, x(τ ), ū(t))) < -δfor a.e. t ∈ {r ∈ [τ -, τ ] ∩ [0, 1] : max ξ∈∂ * d(x(r)) ξ • f (r, x(r), ū(r)) ≥ 0}.

The partial supports of projects FP7-ITN-264735-SADCO "Sensitivity Analysis for Deterministic Controller Design" and FCT/FEDER Project PTDC/EEA-CRO/116014/2009 "Optimal Control in Constrained and Hybrid Nonlinear Systems", PTDC/EEI-AUT/1450/2012 "Optimal Control: Health, Energy and Robotics Applications" are gratefully acknowledged.Date:

January 7, 2015.