Hélène Frankowska
email: helene.frankowska@imj-prg.fr

Nikolai P Osmolovskii
email: osmolovski@uph.edu.pl

Second-Order Necessary Optimality Conditions for the Mayer Problem Subject to a General Control Constraint *

Keywords: Pontryagin maximum principle, second order necessary optimality conditions, weak minimizer, control constraints

This paper is devoted to second-order necessary optimality conditions for the Mayer optimal control problem with an arbitrary closed control set U ⊂ R m . Admissible controls are supposed to be measurable and essentially bounded. Using second order tangents to U , we first show that if ū(•) is an optimal control, then an associated quadratic functional should be nonnegative for all elements in the second order jets to U along ū(•). Then we specify the obtained results in the case when U is given by a finite number of C 2 -smooth inequalities with positively independent gradients of active constraints. The novelty of our approach is due, on one hand, to the arbitrariness of U . On the other hand, the proofs we propose are quite straightforward and do not use embedding of the problem into a class of infinite dimensional mathematical programming type problems. As an application we derive new second-order necessary conditions for a free end-time optimal control problem in the case when an optimal control is piecewise Lipschitz.

Introduction

This paper is devoted to second-order necessary optimality conditions for the Mayer optimal control problem with a control constraint u(t) ∈ U a.e. There is an extensive literature on the subject of second order conditions in optimal control. Earlier results are due to Hestenes [START_REF] Hestenes | Calculus of Variations and Optimal Control[END_REF]. They were followed by the rich Russian literature on this subject, notably the Milyutin school in Moscow and also by contributions of J. Warga [START_REF] Warga | A second-order Lagrangian condition for restricted control problems[END_REF][START_REF] Warga | A second-order condition that strengthens Pontryagin"s maximum principle[END_REF] and many others. We refer the interested reader to, e.g., [START_REF] Aronna | Quadratic conditions for bang-singular extremals[END_REF], [START_REF] Levitin | Conditions of high order for a local minimum in problems with constraints[END_REF], [START_REF] Hoehener | Variational approach to second-order optimality conditions for control problems with pure state constraints[END_REF], [START_REF] Osmolovskii | On second-order necessary conditions for broken extremals[END_REF] for historical comments and bibliographical remarks.

The classical no-gap necessary and sufficient second-order conditions have the following formulations: a necessary condition asserts that a certain quadratic form should be nonnegative on the so-called critical cone; the sufficient condition requires that the same quadratic form should be positive definite on the critical cone. (In simple cases, the critical cone consists of all directions tangent to the constraints.) In particular, if the control set in the Mayer problem (without endpoints constraints) is described by inequalities involving C 2 -smooth functions and having linearly independent gradients of active constraints, this classical no-gap condition holds true. But if in the same problem the gradients of active constraints are only positively independent, instead of being linearly independent, results become somewhat more complicated. The necessary conditions guarantee that with every element of the critical cone one can associate a quadratic form that is nonnegative on this element. The sufficient conditions then have to be formulated using a family of such quadratic forms depending on elements in the critical cone.

In the existing literature the control set U is traditionally defined by a family of equality and inequality constraints and the usual approach consists in looking at control problems as infinite dimensional mathematical programming optimization problems in an appropriately chosen Banach space. Once abstract necessary optimality conditions are derived, the major challenge is then to translate these conditions in terms of the original optimal control problem. Larger is the number and diversity of constraints, more difficult such translation becomes and more assumptions are needed to get back to the original setting from the abstract one.

In particular, some authors imposed assumptions that are verified only by continuous optimal controls or simply required optimal controls to be piecewise continuous to prove their results. Such assumptions weaken then the achievements on second order conditions, since, as it is well known, the existence theory for optimal solutions guarantees only measurability of optimal controls, cf. [START_REF] Cesari | Optimization, Theory and Applications[END_REF], and the first order necessary optimality conditions hold true for such controls, see for instance [START_REF] Milyutin | Maximum Principle in the General Optimal Control Problem[END_REF][START_REF] Vinter | Optimal control[END_REF].

Furthermore, this approach does not allow to treat the case when U is a union of sets described by inequality or equality constraints, that arise naturally in some models.

In his PhD thesis, D. Hoehener considered a Bolza optimal control problem under state constraints with a control set described by a set-valued map t ; U (t), t ∈ [0, 1] and, given an optimal control ū(•), associated to it a quadratic functional that should be nonnegative for all selections from the second order jets to U (t) at ū(t) when t ∈ [0, 1]. The results obtained in [START_REF] Hoehener | Variational approach to second-order optimality conditions for control problems with pure state constraints[END_REF] have rather nontraditional character, and in this sense they are different from most of the works of other authors. It suffices to say that, instead of a quadratic form, the new conditions employ a quadratic functional (with a first order term), and the critical cone is replaced by a set of the so-called second variations. It was shown that the quadratic functional has to be nonnegative on the set of second order variations. The advantage of this approach is due to the fact that the second order necessary optimality condition is the same for all elements in the second order variations. It was shown in [START_REF] Hoehener | Variational Approach to second-order sufficient optimality conditions in optimal control[END_REF] that for a convex state constraint these conditions are no-gap : sufficiency was investigated in [START_REF] Hoehener | Variational Approach to second-order sufficient optimality conditions in optimal control[END_REF] by requiring this quadratic functional to be positive on critical second order variations. Let us underline that in both [START_REF] Hoehener | Variational approach to second-order optimality conditions for control problems with pure state constraints[END_REF] and [START_REF] Hoehener | Variational Approach to second-order sufficient optimality conditions in optimal control[END_REF] the end-point constraints are absent. Another important future of this approach is that it avoids representing U analytically and involving such representation into the expressions of necessary optimality conditions. This opens the possibility to work directly with a particular class of control constraints (polyhedra, sets of class C 2 , etc.) and to get other second order necessary optimality conditions, as for instance those of Goh, cf. [START_REF] Frankowska | Pointwise second-order necessary optimality conditions for the Mayer problem with control constraints[END_REF] or a new second order maximum principle, cf. [START_REF] Frankowska | A second-order maximum principle in optimal control under state constraints[END_REF], for the whole class.

In the present paper we consider the Mayer optimal control problem involving only the control constraint U and derive second order necessary optimality conditions. Similar results can be obtained when U (•) is a measurable set-valued map with closed nonempty images, but to keep the presentation more tutorial and less technical, we purposely do not do it here. By adding some assumptions on the mapping describing the control system, our results can be also easily extended to the case when the reference optimal control is not essentially bounded and also when the control system depends on time in a measurable way.

Our aim is twofold. On one hand, we wish to extend some results from [START_REF] Hoehener | Variational approach to second-order optimality conditions for control problems with pure state constraints[END_REF] to a larger set of second variations being simpler to describe and making the whole approach more easy to understand. On the other hand, we want to show that the method of second order variations is able to produce certain well-known results and also some new ones in a rather short way, not using a heavy artillery of difficult abstract theories. In particular, when the set U is given by inequality constraints having positively independent gradients of active constraints, we obtain the classical result mentioned above. Let us underline again that in the present paper we imposed only control constraints. Our work in preparation do handle a more complex situation where also end-point constraints are present.

The outline of the paper is as follows. In Section 2 we discuss a finite dimensional minimization problem and introduce first and second order tangents to sets. In Section 3 we recall the notions of weak and strong minima and the first order necessary optimality conditions. Section 4 contains our main result dealing with a general control set U . It provides a second order necessary optimality condition in the form of a quadratic functional involving both the first and second order tangents to U . In section 5 we specify this result when the set U is given by inequality constraints having positively independent gradients of active constraints. Finally, in Section 6 we give an application of the results to a free end-time optimal control problem and derive second order necessary optimality conditions which take into account jumps of optimal controls.

Finite Dimensional Case

Let us first illustrate the idea of the method on a simple example in a finite dimensional space. Let U ⊂ R m be a closed nonempty set, and d U (u) denotes the distance from an arbitrary point u ∈ R m to the set U , i.e. d U (u) = min u ∈U |u -u |. Definition 2.1 Given u 0 ∈ U . The adjacent tangent cone to U at u 0 is defined by

T U (u 0 ) := u ∈ R m : lim h→0+ d U (u 0 + hu) h = 0 .
In other words, u ∈ T U (u 0 ) if and only if for every h > 0 there exists an element

r 1 (h) ∈ R m such that u 0 + hu + r 1 (h) ∈ U and |r 1 (h)| = o(h) (the latter means that |r 1 (h)|/|h| → 0 as h → 0+).
This cone was introduced by Peano in the XIX-th century. Let us note that it is closed, but, in general, is not convex. If u 0 ∈ int U , then T U (u 0 ) = R m , where int U is the interior of the set U . By ∂U we denote the boundary of U .

Let ψ : R m → R be a differentiable function, and let u 0 ∈ ∂U be a point of local minimum of ψ on the set U . Then, obviously, ∇ψ(u 0 ), u ≥ 0 for all u ∈ T U (u 0 ).

(

) 1
This is the first order necessary optimality condition (a generalized Fermat rule) for a local minimum of the function ψ on the set U at a point u 0 ∈ ∂U . Now, let us turn to the second order conditions.

Definition 2.2 Let u 0 ∈ U and u ∈ R m . For the pair (u 0 , u), define the second-order adjacent set to U at (u 0 , u) as follows T

(2)

U (u 0 , u) := v ∈ R m : lim h→0+ d U (u 0 + hu + h 2 v) h 2 = 0 . Clearly, v ∈ T (2)
U (u 0 , u) if and only if for every h > 0 there exists an element r 2 (h) ∈ R m such that u 0 + hu + h 2 v + r 2 (h) ∈ U and |r 2 (h)| = o(h 2 ) (the latter automatically implies that u ∈ T U (u 0 )). Any such couple (u, v) can be seen as a second order jet to U at u 0 (in the sense that

u 0 + hu + h 2 v ∈ U + o(h 2 )B
, where B stands for the closed unit ball.)

From the Lipschitz continuity of d U it follows that the set T

U (u 0 , u) is always closed, but, again, may be not convex. Moreover it may be empty, e.g. if u / ∈ T U (u 0 ). It may be empty also for u ∈ T U (u 0 ). For instance, let

U := u = (u 1 , u 2 ) ∈ R 2 : u 1 ≥ 0, u 2 = u 3 2 1 , u 0 = (0, 0), u = (1, 0). Then u 0 ∈ U , u ∈ T U (u 0 ), but obviously the set T (2)
U (u 0 , u) is empty. Recall that a direction u ∈ R m is inward pointing to U at u 0 (in the sense of Dubovitskii-Milyutin [START_REF] Dubovitskii | Problems for extremum under constraints[END_REF]) if there exists ε > 0 such that u 0 + [0, ε]B(u, ε) ⊂ U. Clearly, if u 0 ∈ ∂U and u is as above, then T

(2)

U (u 0 , u) = R m . Hence T (2)
U (u 0 , u) = R m only for the directions that are not inward pointing to U at u 0 . Later we will need the following simple estimate concerning T

(2)

U (u 0 , u). Let u 0 ∈ U , u ∈ T U (u 0 ) and v ∈ T (2)
U (u 0 , u). Then for every ε > 0 and all sufficiently small h > 0,

d U (u 0 + hu) ≤ |u 0 + hu + h 2 v + r 2 (h) -(u 0 + hu)| = |h 2 v + r 2 (h)| ≤ (|v| + ε)h 2 .
(

) 2
(Here, as before,

u 0 + hu + h 2 v + r 2 (h) ∈ U and |r 2 (h)| = o(h 2 )).
Let us stress that inequality (2) holds for all h ∈ (0, h 0 ) where h 0 > 0 depends not only on ε > 0, but also on u 0 and u. In Section 3 we will be interested in a situation when, for a particular set of pairs (u 0 , u), this estimate holds true with ε = 1 and h 0 > 0 independent from (u 0 , u) in this set.

Consider a C 2 -function ψ : R m → R. Assume that u 0 ∈ ∂U is a point of local minimum of ψ on U and v ∈ T (2) U (u 0 , u). Then u ∈ T U (u 0 ) and u 0 + hu + h 2 v + r 2 (h) ∈ U for all h > 0 (where |r 2 (h)| = o(h 2 ))
, and hence for all h > 0 small enough

ψ(u 0 + hu + h 2 v + r 2 (h)) -ψ(u 0 ) ≥ 0. ( 3
)
If ∇ψ(u 0 ), u > 0, then condition (3) is trivially satisfied for all small h > 0, regardless of whether u 0 is a local minimizer or not. Such tangents are not informative from the standpoint of the local minimum. Further, the tangents u such that ∇ψ(u 0 ), u < 0 are forbidden by the first order condition. Therefore, let us consider all u ∈ T U (u 0 ) for which ∇ψ(u 0 ), u = 0. Such directions will be called critical. It follows easily from (3) that for any critical direction u and for every v ∈ T

(2)

U (u 0 , u) we have ∇ψ(u 0 ), v + 1 2 ψ (u 0 )u, u ≥ 0. ( 4
)
We introduce the critical cone

C(u 0 ) := {u ∈ T U (u 0 ) : ∇ψ(u 0 ), u = 0}, (5)
and the set

V (2) (u 0 ) := {(u, v) : u ∈ C(u 0 ), v ∈ T (2)
U (u 0 , u)}. We have proved the following second-order necessary condition for a local minimum at u 0 :

∇ψ(u 0 ), v + 1 2 ψ (u 0 )u, u ≥ 0 for all (u, v) ∈ V (2) (u 0 ). (6)
Setting inf 6) can be equivalently presented as

∅ (•) = +∞, (
inf v∈T (2) U (u 0 ,u) ∇ψ(u 0 ), v + 1 2 ψ (u 0 )u, u ≥ 0 for all u ∈ C(u 0 ). (7)
To go further, if U is a union or an intersection of sets described by equality and inequality constraints, one can use the calculus of second order tangents to refine the inequality [START_REF] Frankowska | Regularity of minimizers and of adjoint states for optimal control problems under state constraints[END_REF], see [START_REF] Aubin | Set-valued analysis[END_REF]Section 4.7]. We shall pause first on the simplest situation when U is given by one (scalar) inequality g(u) ≤ 0, where g : R m → R is a C 2 -function with a nonzero gradient ∇g(u) = 0 for all u satisfying g(u) = 0. In this case, see for instance [2, pp. 150 -151], the tangent cone to U at a point u 0 ∈ U , satisfying g(u 0 ) = 0, is a half-space

T U (u 0 ) = {u : ∇g(u 0 ), u ≤ 0}.
The first order necessary condition (1) implies then the Lagrange multipliers rule: there exists λ ≥ 0 such that ∇ψ(u 0 ) + λ∇g(x 0 ) = 0.

Assume that ∇ψ(u 0 ) = 0. Then C(u 0 ) = {u : ∇g(u 0 ), u = 0}, and for any u ∈ C(u 0 ) the second-order adjacent set to U at u 0 has the form

T (2) U (u 0 , u) = {v : ∇g(u 0 ), v + 1 2 g (u 0 )u, u ≤ 0}. ( 9
)
It follows from ( 9) that sup

v∈T (2) U (u 0 ,u) ∇g(u 0 ), v = - 1 2 g (u 0 )u, u . (10)
Relations ( 7), ( 8) and (10) yield

ψ (u 0 )u, u + λ g (u 0 )u, u ≥ 0 ∀ u ∈ C(u 0 ).
This inequality constitutes the well-known second order necessary optimality condition for a C 2smooth problem with one inequality constraint having a nonzero gradient on the boundary of the constraint. (If ∇ψ(u 0 ) = 0, then λ = 0 and ψ (u 0 )u, u ≥ 0 for all u ∈ T U (u 0 ).) Similarly one can study the case where the set U is given by a finite number of C 2 -smooth inequality constraints under the assumption that the gradients of active constraints are positively independent.

Statement of Optimal Control Problem and Maximum Principle

Let f : R n × R m → R, ϕ : R n → R be given twice continuously differentiable mappings, U ⊂ R m be closed and nonempty and t f > 0, x 0 ∈ R n . Consider the Mayer optimal control problem min ϕ(x(t f )), ( 11)

ẋ(t) = f (x(t), u(t)), u(t) ∈ U for a.e. t ∈ [0, t f ], x(0) = x 0 . (12)
In what follows, we set for brevity

X = W 1,1 ([0, t f ], R n ), U = L ∞ ([0, t f ], R n ).
A pair (x(•), u(•)) ∈ X × U is said to be admissible if it satisfies constraints [START_REF] Milyutin | Maximum Principle in the General Optimal Control Problem[END_REF]. The minimum is sought over all admissible pairs (x(•), u(•)).

Definition 3.1 Let (x(•), ū(•)) be an admissible pair. It is a strong local minimizer for problem (11) -(12) if there exists ε > 0 such that ϕ(x(t f )) ≥ ϕ(x(t f )) for all admissible pairs (x(•), u(•)) satisfying max [0,t f ] |x(t) -x(t)| < ε.
We call (x(•), ū(•)) a weak local minimizer for the same problem if there exists ε > 0 such that ϕ(x(t f )) ≥ ϕ(x(t f )) for every admissible (x(•), u(•)) with u -ū ∞ < ε.

Clearly, each strong local minimizer is a weak local minimizer, but not vice versa. On the other hand, if (x(•), ū(•)) is a weak local minimizer of the Mayer problem, then there exists ε > 0 such that (x(•), ū(•)) is a strong local minimizer for problem (11) -( 12) with an additional restriction on controls: u(t) ∈ B(ū(t), ε) a.e., where B(z, ε) denotes the closed ball in R m with the center z ∈ R m and the radius ε > 0.

The Pontryagin Hamiltonian, is a mapping

H : R n × R n × R m → R defined by H(x, p, u) = p, f (x, u) . (13)
The maximized Hamiltonian H : R n × R n → R is defined by

H(x, p) := sup u∈U H(x, p, u). ( 14
)
Our aim is to find second-order necessary conditions for a weak minimum.

First Order Necessary Conditions for a Strong Minimum

We recall here the well known first order necessary condition for a strong minimum: the Pontryagin maximum principle.

As is known, cf. [START_REF] Pontryagin | The Mathematical Theory of Optimal Processes, Fitzmatgiz[END_REF], if (x(•), ū(•)) is a strong minimizer, then the maximum principle holds true: the solution p(•) of the linear system

-ṗ(t) = H x (x(t), p(t), ū(t)), -p(t f ) = ∇ϕ(x(t f )), ( 15
)
satisfies the equality

H(x(t), p(t)) = H(x(t), p(t), ū(t)) for a.e. t ∈ [0, t f ]. ( 16
)
The first ingredient in ( 15) is called the adjoint system, while the second one is the transversality condition, and ( 16) is the maximum condition. Note that the adjoint equation can be written in the form

-ṗ(t) = f x (x(t), ū(t)) * p(t), (17)
where for a matrix E, E * denotes its transpose and that (15) uniquely defines the adjoint variable p(•) on the time interval [0, t f ].

Theorem 3.1 (Maximum principle) If (x(•), ū(•)
) is a strong local minimizer for problem ( 11) -( 12), then the unique (Lipschitz continuous) solution p : [0, t f ] → R n of ( 15) satisfies [START_REF] Osmolovskii | On second-order necessary conditions for broken extremals[END_REF].

Let us underline that Theorem 3.1 holds true even when set U ⊂ R m is not closed. This fact will be used in Sec. 6. We will also need the formulation of the maximum principle for the Mayer problem with the control set depending on t. Namely, let U (•) : [0, t f ] ; R m be a measurable set-valued map with closed nonempty images. In problem [START_REF] Hoehener | Variational Approach to second-order sufficient optimality conditions in optimal control[END_REF], [START_REF] Milyutin | Maximum Principle in the General Optimal Control Problem[END_REF], let us replace the control constraint u(t) ∈ U by the constraint

u(t) ∈ U (t) for a.e. t ∈ [0, t f ]. ( 18
)
For the new problem, define the maximized Hamiltonian H : [0,

t f ] × R n × R n → R similarly to (14): H(t, x, p) := sup u∈U (t) H(x, p, u),
where H is as in [START_REF] Levitin | Conditions of high order for a local minimum in problems with constraints[END_REF]. Then the assertion, similar to Theorem 3.1, holds true: if an admissible pair (x(•), ū(•)) is a strong local minimizer for problem ( 11) -( 12) with the constraint u(t) ∈ U replaced by [START_REF] Pontryagin | The Mathematical Theory of Optimal Processes, Fitzmatgiz[END_REF], then there exists an absolutely continuous function p : [0, t f ] → R n , (uniquely) defined by [START_REF] Osmolovskii | Necessary quadratic conditions of extremum for discontinuous controls in optimal control problem with mixed constraints[END_REF], and such that the maximum condition is fulfilled:

H(t, x(t), p(t)) = H(x(t), p(t), ū(t)) for a.e. t ∈ [0, t f ], (19)
see for instance [20, Theorem 6.2.1].

First Order Necessary Conditions for a Weak Minimum

Let an admissible pair (x(•), ū(•)) be a weak local minimizer for the Mayer problem ( 11) -( 12) and let p(•) be the (unique) adjoint function satisfying [START_REF] Osmolovskii | Necessary quadratic conditions of extremum for discontinuous controls in optimal control problem with mixed constraints[END_REF]. Then there exists ε > 0 such that (x(•), ū(•)) is a strong local minimizer for the problem ( 11) -( 12) with an additional restriction on controls : u(t) ∈ B(ū(t), ε) a.e. Thus, by the maximum principle for the Mayer problem involving the time dependent control sets U (t), max u∈U ∩B(ū(t),ε)

H(x(t), p(t), u) = H(x(t), p(t), ū(t)) for a.e. t ∈ [0, t f ]. (20)
Denote for brevity

H u [t] := H u (x(t), p(t), ū(t)).
(Similar abbreviations will be used in the sequel for other compositions of mappings with time dependent mappings.) Then it follows from (20) that for a.e. t ∈ [0, t f ] we have

H u [t], u ≤ 0 ∀ u ∈ T U (ū(t)). (21)
In particular, H u [t] = 0 a.e. on the set {t ∈ [0, t f ] : ū(t) ∈ int U }.

Thus conditions ( 15) and ( 21) are necessary for a weak local minimum. They may be seen as a local maximum principle.

Similarly, it follows from ( 4) and ( 20) that for a.e. t ∈ [0, t f ] it holds

H u [t], v + 1 2 H uu [t]u, u ≤ 0 ∀ u, v ∈ R m with H u [t], u = 0, v ∈ T (2)
U (ū(t), u). (22)
This is an analog of the Legendre condition. Recall that the condition v ∈ T

U (ū(t), u) implies u ∈ T U (ū(t)).

4 Second-Order Necessary Conditions for a General Constraint u ∈ U

Here, we formulate for problem (11) -( 12), the second-order necessary optimality condition which generalize similar conditions obtained in [START_REF] Hoehener | Variational approach to second-order optimality conditions for control problems with pure state constraints[END_REF] and later also stated in [START_REF] Frankowska | Pointwise second-order necessary optimality conditions for the Mayer problem with control constraints[END_REF] (note that in [START_REF] Hoehener | Variational approach to second-order optimality conditions for control problems with pure state constraints[END_REF], the problem was a bit different: the author considered the integral cost instead of the terminal one).

Everywhere in this section (x(•), ū(•)) is a fixed weak local minimizer and p(•) is the unique absolutely continuous function satisfying conditions [START_REF] Osmolovskii | Necessary quadratic conditions of extremum for discontinuous controls in optimal control problem with mixed constraints[END_REF], ( 21) and [START_REF] Warga | A second-order condition that strengthens Pontryagin"s maximum principle[END_REF].

Critical Cone

Consider the linearized system

ẏ(t) = f x [t]y(t) + f u [t]u(t) a.e. in [0, t f ], y(0) = 0, ( 23
)
where f x [t] := f x (x(t), ū(t)) and f u [t] is defined in a similar way. We shall denote by S the set of all (y(•), u(•)) ∈ X × U satisfying (23).

Proposition 4.1 For every (y(•), u(•)) ∈ S we have

∇ϕ(x(t f )), y(t f ) = - t f 0 H u [t], u(t) dt.
Proof. Using ( 15), ( 17) and ( 23) we obtain

∇ϕ(x(t f )), y(t f ) = -p(t f ), y(t f ) + p(0), y(0) = - t f 0 d dt p(t), y(t) dt = - t f 0 p(t), f u [t]u(t) dt = - t f 0 H u [t], u(t) dt. 2
Given a weak local minimizer (x(•), ū(•)) of ( 11) -( 12), define the cone

C(x, ū) = (y(•), u(•)) ∈ S : ∇ϕ(x(t f )), y(t f ) = 0, u(t) ∈ T U (ū(t)) a.e. in [0, t f ] . (24)
In this paper, C(x, ū) is called the critical cone of problem ( 11) -( 12) at (x(•), ū(•)). We would like to underline that usual definitions of the critical cones, which the most of the known second order conditions use, involve the inequality ∇ϕ(x(t f )), y(t f ) ≤ 0 in (24) instead of the equality. But the strict inequality is impossible. Indeed, if (y( Consider the set

A := {t ∈ [0, t f ] : H u [t] = 0} (27)
and observe that, by [START_REF] Warga | A second-order Lagrangian condition for restricted control problems[END_REF], for a.e. t ∈ A, ū(t) belongs to ∂U .

Quadratic functional, main theorem

Set M (2) (ū) = (u, v) ∈ U × U : u(•) ∈ C loc (ū), v(t) ∈ T (2)
U (ū(t), u(t)) a.e. in A and consider the following quadratic functional : for any (u( 11) - [START_REF] Milyutin | Maximum Principle in the General Optimal Control Problem[END_REF]. Then

•), v(•)) ∈ U × U Φ(u, v) = 1 2 y(t f ) * ϕ (x(t f ))y(t f ) - t f 0 H u [t], v(t) dt - t f 0 1 2 y(t
Φ(u, v) ≥ 0 for all (u, v) ∈ M (2) (ū). Proof. Fix (u(•), v(•)) ∈ M (2) (ū).
Let y be the corresponding solution of (23) and c := v ∞ + 1. For every non negative integer i define

A i := {t ∈ A : d U (ū(t) + hu(t)) ≤ ch 2 ∀ h ∈ [0, 1/i]}.
It is not difficult to realize that the sets A i are Lebesgue measurable. Furthermore, the family {A i } i is nondecreasing and the set i A i is of full measure in A, because of the inequality (2) for u 0 = ū(t), ε = 1 and v = v(t). Define

u i (t) = u(t) if t ∈ A i ∪ ([0, t f ]\A) 0 otherwise and v i (t) = v(t) if t ∈ A i 0 otherwise
Observe that for a.e. t ∈ [0, t f ], lim i→∞ u i (t) = u(t) and

u i converge to u in L 1 ([0, t f ]; R m ).
Furthermore, for a.e. t ∈ A, lim i→∞ v i (t) = v(t). Let y i be the solution of (23) corresponding to the control u i . Then y i converge uniformly to y on [0, t f ]. Observe that if Φ(u i , v i ) ≥ 0 for all i, then, using the boundedness of {u i } in U and the Lebesgue dominated convergence theorem, passing to the limit we get Φ(u, v) ≥ 0. For this reason it is enough to show that Φ(u i , v i ) ≥ 0 for all i. Consequently, we have to verify that Φ(u, v) ≥ 0 for every (u(•), v(•)) ∈ M (2) (ū) such that there exist c > 0, h 0 > 0 satisfying the following inequality for a.e. t ∈ A,

d U (ū(t) + hu(t)) ≤ ch 2 , ∀ h ∈ [0, h 0 ].
From now on the proof proceeds in a way similar to [10, Proof of Theorem 3.2], but for the Mayer problem instead of the Bolza one. Fix any (u(•), v(•)) as above and let y be the corresponding solution of (23). By [10, Proposition 4.1] there exist ûh ∈ U such that the family {û h } h>0 is bounded in U and for a.e. t ∈ [0, t f ] and all h > 0 we have ū(t) + hû h (t) ∈ U , and lim h→0+ ûh (t) = u(t). Furthermore, consider ũ(•), ṽ(•) ∈ U defined by (ũ(t), ṽ(t)) = (u(t), v(t)) when t ∈ A and (ũ(t), ṽ(t)) = 0 otherwise. Applying [START_REF] Hoehener | Variational approach to second-order optimality conditions for control problems with pure state constraints[END_REF]Proposition 4.2] we prove the existence of vh ∈ U such that the family {v h } h>0 is bounded in U and for a.e. t ∈ A and all sufficiently small h > 0 we have ū(t) + hu(t) + h 2 vh (t) ∈ U and lim h→0+ vh (t) = v(t).

Define the new controls

u h (t) = u(t) if t ∈ A ûh (t) otherwise and v h (t) = vh (t) if t ∈ A 0 otherwise
For every h > 0 small enough consider the solution x h : [0, t f ] → R n of the nonlinear system

x h (t) = f (x h (t), ū(t) + hu h (t) + h 2 v h (t)), x h (0) = x 0 .
From the variational equation it follows that (x h -x)/h converge uniformly to y when h → 0+. Since ū is a weak local minimizer, for all small h > 0 we have ϕ(x h (t f )) ≥ ϕ(x(t f )). By the Taylor expansion, setting y h = (x h -x)/h and using the Newton-Leibniz formula we obtain

0 ≤ ϕ(x h (t f )) -ϕ(x(t f )) = ∇ϕ(x(t f )), x h (t f ) -x(t f ) + 1 2 ϕ (x(t f ))(x h (t f ) -x(t f )), x h (t f ) -x(t f ) + o(h 2 ) = -p(t f )), x h (t f ) -x(t f ) + h 2 2 ϕ (x(t f ))y(t f ), y(t f ) + o(h 2 ) = - t f 0 ṗ(t), x h (t) -x(t) + p(t), ẋh (t) -ẋ(t) dt + h 2 2 ϕ (x(t f ))y(t f ), y(t f ) + o(h 2 ) = t f 0 H x [t], hy h (t) dt - t f 0 p(t), ẋh (t) -ẋ(t) dt + h 2 2 ϕ (x(t f ))y(t f ), y(t f ) + o(h 2 ). ( 28
)
By the Taylor formula, the adjoint equation and the very definition of u h ,

t f 0 p(t), ẋh (t) -ẋ(t) dt = t f 0 p(t), f (x(t) + hy h (t), ū(t) + hu h (t) + h 2 v h (t)) -f [t] dt = t f 0 H x [t], hy h (t) + H u [t](hu h (t) + h 2 v h (t)) dt + o(h 2 ) +h 2 t f 0 1 2 H xx [t]y h (t), y h (t) + H ux [t]y h (t), u h (t) + 1 2 H uu [t]u h (t), u h (t) dt = t f 0 H x [t], hy h (t) + H u [t], h 2 v(t) + h 2 2 H xx [t]y(t), y(t) +h 2 H ux [t]y(t), u(t) + h 2 2 H uu [t]u(t), u(t) dt + o(h 2 ). ( 29
)
(to get the last equality we have used that H u [t], u(t) = 0 a.e., that H u [t] = 0 on [0, t f ]\A, the uniform convergence of y h to y when h → 0+, the boundedness of {u h } in U and the a.e. pointwise convergence of u h to u). From ( 28) and (29) we deduce that

0 ≤ h 2 2 ϕ (x(t f ))y(t f ), y(t f ) -h 2 t f 0 H u [t], v(t) + 1 2 H xx [t]y(t), y(t) + H ux [t]y(t), u(t) + 1 2 H uu [t]u(t), u(t) dt + o(h 2 ). ( 30
)
Dividing by h 2 and passing to the limit we end the proof. It is interesting to note that the proof of this theorem was not based on the usage of any abstract scheme, as the most of second order necessary conditions. It involves only direct variations of the control.

Let us also underline that, in contrast to usual second-order conditions (for problems with U described by a system of smooth inequalities and equalities), in which to each element of the critical cone corresponds an optimality condition, we allow that for some u ∈ C loc (ū) there is no pair (u, v) ∈ M (2) (ū). If this is the case, then we do not claim anything about such u ∈ C loc (ū). In the next example we do not have this situation, although the set U is not given by a smooth system of inequalities and equalities.

Example 4.1 Let us consider a particular situation when U ⊂ R 2 is the union of two intervals

U = ([-1, 1] × {0}) ∪ ({0} × [-1, 1]).
It is easy to realize that for every u 0 ∈ U and 0 = u ∈ T U (u 0 ) we have T

U (u 0 , u) = Ru. Furthermore, T (2)
U (u 0 , 0) = T U (u 0 ). Let (u, v) ∈ M (2) (ū). Then H u [t]
, u(t) = 0 a.e. and therefore, by what precedes, for a.e. t ∈ A such that u(t) = 0, we have H u [t], v(t) = 0. On the other hand, by [START_REF] Warga | A second-order Lagrangian condition for restricted control problems[END_REF], for a.e. t ∈ A such that u(t) = 0, we have -H

u [t], v(t) ≥ 0. Hence the infimum of Φ(u, v) over v(•) ∈ U satisfying (u, v) ∈ M (2) (ū) is attained at v(•) = 0.
Thus for such U the statement of Theorem 4.1 is equivalent to : for all u(•) ∈ C loc (ū) and the corresponding solution y(•) of (23) we have

Ω(u) := y(t f ) * ϕ (x(t f ))y(t f ) - t f 0 y(t) * H xx [t]y(t) + 2u(t) * H ux [t]y(t) + u(t) * H uu [t]u(t) dt ≥ 0.
That is the second order necessary condition takes the form: the quadratic form Ω is nonnegative on the critical cone C loc (ū).

Second-Order Necessary Conditions when the Control Set U is Given by Inequalities

The aim of this section is to compare the second order necessary conditions derived in Section 4 with the known results when the set U is described by a finite number of inequality constraints (see for instance [START_REF] Bonnans | Second-order analysis of optimal control problems with control and initial-final state constraints[END_REF], [START_REF] Osmolovskii | Necessary quadratic conditions of extremum for discontinuous controls in optimal control problem with mixed constraints[END_REF]).

Consider the problem min ϕ(x(t f )),

= f (x(t), u(t)), x(0) = x 0 , (31) ẋ(t)
g i (u(t)) ≤ 0, i = 1, . . . , q, t ∈ [0, t f ], (32)
where t f > 0 is fixed and ϕ, f, g i are C 2 -functions. Hence in this problem U := {u ∈ R m : g i (u) ≤ 0 for all i = 1, . . . , q}.

We shall assume that the control constraints satisfy the assumption of positive independence of gradients of active constraints at each point from the boundary of U . It means that at any point u ∈ ∂U the following condition holds true: 0 / ∈ co{∇g i (u) : i ∈ I g (u)}, where I g (u) = {i : u ∈ ∂U i }, and ∂U i is the boundary of the set U i := {u ∈ R m : g i (u) ≤ 0}. We say that I g (u) is the set of active indices at u. Note that if i ∈ I g (u), then certainly g i (u) = 0, but not vice versa.

Remark 5.1 Without loss of generality, we may assume that the functions f , g and ϕ satisfy the above assumptions on some open sets

Q f ⊂ R n × R m , Q g ⊂ R m and Q ϕ ⊂ R n , respectively. Then the condition u(t) ∈ Q g a.e
. should be understood with some "margin": there exists a compact set K ⊂ Q g (depending on u(•)) such that u(t) ∈ K a.e. Similar remark concerns the condition

(x(t), u(t)) ∈ Q f a.e.
Let (x(•), ū(•)) be a weak local minimizer of our problem and p(•) be the solution of [START_REF] Osmolovskii | Necessary quadratic conditions of extremum for discontinuous controls in optimal control problem with mixed constraints[END_REF]. Then, according to Sec. 3.2, conditions (21) and ( 22) are fulfilled for a.e. t ∈ [0, t f ].

Remark 5.2 In this section, in the results given below, the assumption of positive independence of gradients of active constraints at each point from the boundary of U can be weakened as follows. Let F ⊂ ∂U be a (minimal) compact set such that the set {t : u(t) ∈ F} has a full measure in the set {t : u(t) ∈ ∂U }. Then we may assume that the gradients of active constraints are positively independent only on the set F. All subsequent consideration remain valid under this weakened assumption.

For every t ∈ [0, t f ], set I(t) = I g (ū(t)). Observe that I(•) is a measurable set-valued map. If ū(t) ∈ ∂U and 0 / ∈ co{∇g i (ū(t)) : i ∈ I(t)}, then in view of (??),

T U (ū(t)) = {u ∈ R m : ∇g i (ū(t)), u ≤ 0, ∀ i ∈ I(t)}.
According to (25), it follows that u ∈ C loc (ū) if and only if u ∈ U and

∇g i (ū(t)), u(t) ≤ 0 a.e. in M 0i (ū(•)), ∀i = 1, . . . , q, (34)
H u [t], u(t) = 0 a.e. in [0, t f ], (35)
where

M 0i (ū(•)) := {t ∈ [0, t f ] : i ∈ I(t)}.
Furthermore, in view of (??), T U (ū(t)) -= i∈I(t) R + ∇g i (ū(t)) for a.e. t ∈ [0, t f ]. On the other hand, from [START_REF] Warga | A second-order Lagrangian condition for restricted control problems[END_REF], we know that H u [t] ∈ T U (ū(t)) -a.e. Hence, by [START_REF] Aubin | Set-valued analysis[END_REF]Theorem 8.2.15], there exists a measurable map λ(•) = (λ 1 (•), ..., λ q (•)) : [0, t f ] → R q satisfying for a.e. t ∈ [0, t f ] the following conditions:

λ i (t) ≥ 0, λ i (t) = 0 for i / ∈ I(t); (36)
H u [t] = q i=1 λ i (t)∇g i (ū(t)). (37)
From the positive independence assumption, essential boundedness of ū(•) and continuity of functions ∇g i , using [6, Corollary 2.2], we deduce that λ(•) is essentially bounded. If in addition {∇g i (ū(t)) : i ∈ I(t)} are linearly independent, then λ(•) is uniquely defined up to a set of measure zero. However, in general, this is not the case. Let u ∈ C loc (ū). It follows from conditions (34) -(37) that for a.e. t ∈ [0, t f ], the following holds true for any i ∈ I(t) : if ∇g i (ū(t)), u(t) < 0, then λ i (t) = 0. It can be written also in the form: λ i (t) ∇g i (ū(t)), u(t) = 0 a.e., for all i ∈ I(t).

Given λ(•) as above, consider the quadratic form

Ω λ (u(•)) := 1 2 y(t f ) * ϕ (ȳ(t f ))y(t f ) - t f 0 1 2 y(t) * H xx [t]y(t) + u(t) * H ux [t]y(t) + 1 2 u(t) * H uu [t]u(t) -1 2 q i=1 λ i (t)u(t) * g i (ū(t))u(t) dt,
where y(•) solves the variational system (23). It is convenient to introduce the so-called augmented Hamiltonian:

H a (x, u, p, λ) = p, f (x, u) -λ, g(u) , ∀ (x, u, p, λ) ∈ R n × R m × R n × R q .
Then (37) implies that H aλ u [t] = 0 a.e., where H aλ u [t] := H a u (x(t), ū(t), p(t), λ(t)), and the quadratic form can be represented as [START_REF] Osmolovskii | Necessary quadratic conditions of extremum for discontinuous controls in optimal control problem with mixed constraints[END_REF]. Then for every u ∈ C loc (ū) there exists λ ∈ L ∞ ([0, t f ]; R q ) such that for a.e. t ∈ [0, t f ], the relations (36), (37) are verified and

Ω λ (u(•)) = 1 2 y(t f ) * ϕ (ȳ(t f ))y(t f ) - t f 0 1 2 y(t
Ω λ (u) ≥ 0,
where Ω λ (u) is as in (38). Furthermore, if in addition the gradients {∇g i (ū(t)) : i ∈ I(t)} are linearly independent for almost every t ∈ [0, t f ], then such λ is unique up to a set of measure zero. Consequently, in this case the quadratic form (38) is nonnegative on C loc (ū).

Remark 5.3 When the gradients of active constraints g i (u) ≤ 0 are linearly independent, then the above result follows from [START_REF] Osmolovskii | Necessary quadratic conditions of extremum for discontinuous controls in optimal control problem with mixed constraints[END_REF]Theorem 1.3] proved for a much more general optimal control problem. A similar result was obtained in [START_REF] Bonnans | Second-order analysis of optimal control problems with control and initial-final state constraints[END_REF] under assumption of positive independence of the gradients of active constraints.

Recall that the local critical cone C loc (ū) is defined in the space U = L ∞ ([0, t f ]; R m ). Similarly, it can be defined in the same way in a larger space U 2 := L 2 ([0, t f ]; R m ) of square integrable mappings. Denote this new cone by C 2 loc (ū). Then,

C 2 loc (ū) = u ∈ U 2 : H u [t], u(t) = 0 and u(t) ∈ T U (ū(t)) a.e. in [0, t f ] . (39)
Then, obviously, C loc (ū) is dense in C 2 loc (ū) (for the norm • 2 ). As before, when U is given by a finite number of inequalities g i (u) ≤ 0 having the gradients of active constraints positively independent, then u ∈ C 2 loc (ū) if and only if u ∈ L 2 ([0, t f ]; R m ) and (34), (35) hold true.

Corollary 5.1 Under all the assumptions of Theorem 5.1, for every u ∈ C 2 loc (ū) and the solution y(•) of ( 23), there exists λ ∈ L ∞ ([0, t f ]; R q ) such that Ω λ (u) ≥ 0 and the relations (36), (37) are verified for a.e. t ∈ [0, t f ].

Furthermore, if {∇g i (ū(t)) : i ∈ I(t)} are linearly independent for almost every t ∈ [0, t f ], then such λ is unique up to a set of measure zero. Consequently, in this case the quadratic form (38) is nonnegative on C loc (ū).

Since each strong minimizer is a weak minimizer, we obtain the following necessary condition for a strong minimum which directly follows from the maximum principle and Corollary 5.1.

Corollary 5.2 Let (x(•), ū(•)) be a strong local minimizer for problem (31) -(33) and let the assumption of positive independence of gradients of active control constraints be fulfilled. Then for the solution p(•) of ( 15) the following necessary Condition A holds :

(a) condition ( 16) of the maximum principle is satisfied by p(•); (b) for every u ∈ C 2 loc (ū) there exists a measurable, essentially bounded function λ : [0, t f ] → R q such that for a.e. t ∈ [0, t f ], the relations (36), (37) are verified and Ω λ (u) ≥ 0. This corollary is used in the next section. We will also need the following definition of the critical cone in the space of square integrable functions. Set X

2 := W 1,2 ([0, tf ], R n ). Then C 2 (x, ū) := (y, u) ∈ X 2 × U 2 : u(•) ∈ C
ϕ(t f , x(t f )), t f ≥ 0, (41) ẋ
(t) = f (t, x(t), u(t)) for a.e. t ∈ [0, t f ], x(0) = x 0 , (42)
g i (u(t)) ≤ 0, i = 1, . . . , q, for a.e. t ∈ [0, t f ], (43)
where the functions ϕ : R + × R n → R, f : R + × R n × R m → R n and g i : R m → R are twice continuously differentiable, and satisfying the assumption of linear independence of the gradients of active constraints g i ≤ 0. The latter means that at every point u ∈ ∂U the gradients {∇g i (u) : i ∈ I g (u)} are linearly independent, where I g (u) = {i : u ∈ ∂U i }, and ∂U i is the boundary of the set U i := {u ∈ R m : g i (u) ≤ 0}. For brevity, we refer to problem (41) -(43) with free end-time t f as Problem P .

Let T denote a process (x(•), u(•), t f ), where x(•) is Lipschitz continuous, u(•) is measurable and essentially bounded on [0, t f ], and t f ≥ 0. Define

J (T ) := ϕ(t f , x(t f )).
A process T is called admissible if it satisfies (42) -(43).

Let T = (x(•), ū(•), tf ) be a fixed admissible process. Remarks similar to Remark 5.1 and Remark 5.2 can be done for the Problem P and the process T .

We say that T is a strong local minimizer if there exists ε > 0 such that J (T ) ≥ J (T ) for each admissible process T = (x(•), u(•), t f ) satisfying the conditions

|t f -tf | < ε, |x(t) -x(t)| < ε for all t ∈ [0, t f ] ∩ [0, tf ].
There is an immediate necessary condition for a strong local minimum: the function ψ(•) := ϕ(•, x(•)) has a local minimum at the point tf . In particular, if tf > 0 and x(•) is differentiable at tf , then ϕ t ( tf , x( tf )) + ϕ x ( tf , x( tf )), ẋ( tf ) = 0, (44) where ϕ t and ϕ x denote the partial derivatives of ϕ(•, •).

Critical Cone and Quadratic Form

Let us formulate a quadratic necessary condition for a strong local minimum at the process T . First, for this process, we introduce a Hilbert space Z 2 (Θ) and a "critical cone" C 2 Θ ⊂ Z 2 (Θ). We denote by P Θ W 1,2 ([0, tf ]; R n ) the Hilbert space of piecewise continuous functions y(•) : [0, tf ] → R n , absolutely continuous on each interval of the set [0, tf ] \ Θ and such that their first derivatives are square integrable. For each y(•) ∈ P Θ W 1,2 ([0, tf ]; R n ) and t k ∈ Θ define ∆y(t k ) = y(t k +) -y(t k -).

Thus ∆y(t k ) is the jump of the function y(•) at the point t k ∈ Θ. Such notation will be used to denote jumps also for other functions at a point t k ∈ Θ. For instance, for the scalar product of functions a( Recall that the local (pointwise) critical cone C 2 loc (ū) was defined by (39), which in our case is equivalent to (34), (35) with U replaced by U 2 and t f = tf . Here, as before, U := L ∞ ([0, tf ]; R m ) and U 2 := L 2 ([0, tf ]; R m ). Set (52) where f x [t] := f x (t, x(t), ū(t)), etc. It is obvious that C 2 Θ (x, ū) is a convex cone in the Hilbert space Z 2 (Θ). It will be convenient to call it again the critical cone, although now it is not a cone of "critical directions" in a usual sense (or its L 2 -closure). Let us stress that the component y(•) of any element z ∈ C 2 Θ (x, ū) may have a jump at any point t k ∈ Θ of discontinuity of the control ū(•) (while the state component x(•) of any process T of Problem P is a Lipschitz continuous function).

Let us introduce a quadratic form on Z 2 (Θ) × R. For z ∈ Z 2 (Θ) and ξ f ∈ R we set

η f := (ξ f , y( tf ) + ξ f ẋ( tf )), (53)
where C 2 Θ (x, ū) is as in (52), and Ω Θ is as in (54).

  Let us define the pointwise or local critical coneC loc (ū) := {u(•) ∈ U : u(t) ∈ T U (ū(t)) and H u [t], u(t) = 0 a.e. in [0, t f ]}. (25) Then C(x, ū) := {(y(•), u(•)) ∈ S : u(•) ∈ C loc (ū)}. (26)

  •) and b(•), a : [0, tf ] → R n , b : [0, tf ] → R n , with discontinuity of the first kind at a point t k ∈ Θ, we write ∆ a, b (t k ) := a(t k +), b(t k +) -a(t k -), b(t k -) . Set z = (ξ, y(•), u(•)) , where ξ = (ξ 1 , . . . , ξ s ) ∈ R s , y(•) ∈ P Θ W 1,2 ([0, tf ]; R n ), and u(•) ∈ L 2 ([0, tf ]; R m ). Thus, z ∈ Z 2 (Θ) := R s × P Θ W 1,2 ([0, tf ]; R n ) × L 2 ([0, tf ]; R m ).

C 2 Θ

 2 (x, ū) := {z = (ξ, y(•), u(•)) ∈ Z 2 (Θ) : u(•) ∈ C 2 loc (ū), ẏ(t) = f x [t]y(t) + f u [t]u(t) a.e., y(0) = 0, ∆y(t k ) + ∆ ẋ(t k )ξ k = 0, k = 1, . . . , s}.

Theorem 6 . 2

 62 Let (x(•), ū(•), tf ) be a strong local minimizer of Problem P and g i satisfy the assumption of linear independence of gradients of active constraints. Then the following necessary Condition A Θ holds true: there exists a unique pair of Lipschitz functions p(•), p 0 (•) satisfying (46) -(49), and a unique piecewise Lipschitz function λ : [0, tf ] → R q such that relations (50) are verified a.e. in [0, tf ], and Ω Θ (z, ξ f ) ≥ 0 for all (z, ξ f ) ∈ C 2 Θ (x, ū) × R,

  +2 H xu [t]u(t), y(t) + H a uu [t]u(t), u(t) dt s k=1 ∆ ṗ0 + ṗ, ẋ (t k )ξ 2 k -2∆ ṗ, y (t k )ξ k + ṗ0 ( tf ) + ṗ( tf ), ẋ( tf ) ξ 2 f + 2 ṗ( tf ), y( tf ) ξ f ,(54)where H xx [t] := H xx (t, x(t), ū(t), p(t)), etc., and H a is as in (51).

Ω Θ (z, ξ f ) = ϕ ( tf , x( tf ))η f , η f -tf 0 H xx [t]y(t), y(t)

The partial supports of projects FP7-ITN-264735-SADCO "Sensitivity Analysis for Deterministic Controller Design" is gratefully acknowledged.

† N. Osmolovskii gratefully acknowledges the partial support from the Foundation "Sciences Mathématiques de Paris" and the hospitality of the Institut de Mathématiques de Jussieu -Paris Rive Gauche during his stay in Paris.

Maximum Principle

The Pontryagin Hamiltonian and the maximized Hamiltonian are given by H(t, x, p, u) = p, f (t, x, u) , H(t, x, p) = sup u∈U H(t, x, p, u),

respectively, where p ∈ R n . As it is known, the conditions of the maximum principle for an admissible process T = (x(•), ū(•), tf ) are as follows (see, e.g. [START_REF] Milyutin | Maximum Principle in the General Optimal Control Problem[END_REF]):

- The proof of this theorem is provided below.

For every t ∈ [0, tf ] set I(t) := I g (ū(t)). According to the maximum principle, for a.e. t ∈ [0, tf ], the Hamiltonian H(t, x(t), p(t), u), considered as a function of u, achieves its maximum, subject to the constraint g(u) ≤ 0, at the point ū(t). Hence, there exist Lagrange multipliers λ i : [0, tf ] → R, i = 1, . . . , q satisfying the following conditions a.e. in [0, t f ] : λ i (t) ≥ 0, i = 1, . . . , q, λ i (t) = 0 for all i / ∈ I(t),

where H u [t] := H u (t, x(t), p(t), ū(t)). The functions λ i (•), i = 1, . . . , q are measurable, essentially bounded and uniquely defined by these conditions up to the set of zero measure. Again, it is convenient to introduce the augmented Hamiltonian:

Then the last condition in (50) can be written as H a u (t, x(t), ū(t), p(t), λ(t)) = 0 a.e., or briefly, H a u [t] = 0 a.e.

In the sequel, we consider an admissible process T = (x(•), ū(•), tf ) such that the control ū(•) is a piecewise Lipschitz-continuous function on the interval [0, tf ] with the set of discontinuity points given by Θ

Then condition (48) of the maximum principle is fulfilled not only almost everywhere on [0, tf ], but everywhere on each of intervals (t k-1 , t k ), k = 1, . . . , s + 1. Moreover, since the gradients of active constraints g i ≤ 0 are linearly independent, the Lagrange multipliers λ i (•), i = 1, . . . , q are also piecewise Lipschitz continuous functions such that all their discontinuity points belong to Θ.