Nikolai P Osmolovskii
email: osmolovski@uph.edu.pl

Helmut Maurer
email: maurer@math.uni-muenster.de

Second-Order Optimality Conditions for Broken Extremals and Bang-Bang Controls

Keywords: broken extremal, piecewise continuous control, bang-bang control, second order necessary and sufficient conditions, critical cone, induced optimization problem

We survey the results on no-gap second order optimality conditions (both necessary and sufficient) in the Calculus of Variations and Optimal Control, that were obtained in the monographs [31] and [40], and discuss their further development. First, we formulate such conditions for broken extremals in the simplest problem of the Calculus of Variations and then, we consider them for discontinuous controls in optimal control problems with endpoint and mixed state-control constraints, considered on a variable time interval. Further, we discuss such conditions for bangbang controls in optimal control problems, where the control appears linearly in the Pontryagin-Hamilton function with control constraints given in the form of a convex polyhedron. Bang-bang controls induce an optimization problem with respect to the switching times of the control, the so-called Induced Optimization Problem. We show that second-order sufficient condition for the Induced Optimization Problem together with the so-called strict bang-bang property ensure second-order sufficient conditions for the bang-bang control problem. Finally, we discuss optimal control problems with mixed control-state constraints and control appearing linearly. Taking the mixed constraint as a new control variable we convert such problems to bang-bang control problems. The numerical verification of second-order conditions is illustrated on three examples.

Introduction

We survey some main results presented in the recent monograph of the authors [START_REF] Osmolovskii | Applications to Regular and Bang-Bang Control: Second-Order Necessary and Sufficient Optimality Conditions in Calculus of Variations and Optimal Control[END_REF] (SIAM, 2012) and also some results obtained in the earlier monograph of Milyutin and Osmolovskii [START_REF] Milyutin | Calculus of Variations and Optimal Control[END_REF] (AMS, 1998). We discuss further developments of these results and give various applications.

Our main goal is present and discuss the no-gap second-order necessary and sufficient conditions in control problems with bang-bang controls. In [START_REF] Milyutin | Calculus of Variations and Optimal Control[END_REF], it was shown how, by using quadratic conditions for the general problem of the Calculus of Variations with regular mixed equality constraint g(t, x, u) = 0, one can obtain quadratic (necessary and sufficient) conditions in optimal control problems in which the control variable enters linearly and the control constraint is given in the form of a convex polyhedron. These features were proved in Milyutin, Osmolovskii [START_REF] Milyutin | Calculus of Variations and Optimal Control[END_REF], who first used the property that the set ex U of vertices of a polyhedron U can be described by a nondegenerate relation g(u) = 0 on an open set Q consisting of disjoint open neighborhoods of vertices. This allowed us to develop quadratic necessary conditions for bang-bang controls. Further, in [START_REF] Milyutin | Calculus of Variations and Optimal Control[END_REF] it was shown that a sufficient condition for a minimum on ex U guarantees (in the problem in which the control enters linearly) the minimum on its convexification U. In this way, quadratic sufficient conditions for bang-bang controls were obtained in Osmolovskii, Maurer [START_REF] Osmolovskii | Applications to Regular and Bang-Bang Control: Second-Order Necessary and Sufficient Optimality Conditions in Calculus of Variations and Optimal Control[END_REF]. This property, which is not discussed in the present paper, constitutes the main link between the second order optimality conditions for a broken extremals in the Calculus of Variations and the second order optimality conditions for bang-bang controls in optimal control.

The paper is organized as follows. In Section 2, we formulate no-gap second order conditions for broken extremals in the simplest problem of the Calculus of Variations. In Section 3, we consider such conditions for discontinuous controls in optimal control problems on a fixed time interval with endpoint constraints of equality and inequality type and mixed state-control constraints of equality type. In Section 4, we present an extension of the results of Section 3 to problems on a variable time interval. In Section 5, we discuss no-gap conditions for bang-bang controls. Bang-bang controls induce an optimization problem with respect to the switching times of the control that we call the Induced Optimization Problem (IOP). We have shown in our monograph [START_REF] Osmolovskii | Applications to Regular and Bang-Bang Control: Second-Order Necessary and Sufficient Optimality Conditions in Calculus of Variations and Optimal Control[END_REF] that the classical second-order sufficient condition for the IOP, together with the so-called strict bang-bang property, ensure second-order sufficient conditions for the bang-bang control problem. We discuss such conditions in Section 6.

In the next two sections, the theoretical results are illustrated by numerical examples. Namely, in Section 7, we study the optimal control of the chemotherapy of HIV, when the control-quadratic objective in [START_REF] Kirschner | Optimal control of the chemotherapy of HIV[END_REF] of L 2 -type is replaced by a more realistic L 1 -objective. In Section 8, we consider time-optimal controls in two models of two-link robots; cf. [START_REF] Osmolovskii | Applications to Regular and Bang-Bang Control: Second-Order Necessary and Sufficient Optimality Conditions in Calculus of Variations and Optimal Control[END_REF]. Finally, in Section 9, we discuss optimal control problems with running mixed control-state constraints and control appearing linearly. Taking the mixed constraint as a new control variable we convert such prob-lem to a bang-bang control problem. We use this transformation to study extremals in the optimal control problem for the Rayleigh equation.

x(t 0 ) = a, x(t f ) = b, (t, x(t), ẋ(t)) ∈ Q.

(

) 2
We consider this problem in the space W 1,∞ of Lipschitz continuous functions. The last condition in ( 2) is assumed to hold almost everywhere. A weak minimum is defined as a local minimum in the space W 1,∞ . We say that a function

x ∈ W 1,∞ ([t 0 ,t f ], IR d(x)
) is admissible if x satisfies (2) and, moreover, there exists a compact set C ⊂ Q such that (t, x(t), ẋ(t)) ∈ C a.e. in [t 0 ,t f ]. Set u := ẋ and w = (x, u). We call u the control. Let an admissible function x 0 (t) be an extremal in the sense that it satisfies the Euler equation

d dt L ẋ = L x . (3)
Here and in the sequel, partial derivatives are denoted by subscripts. Set u 0 (t) := ẋ0 (t), w 0 (t) = (x 0 (t), u 0 (t)).

Let w(•) = ( x(•), ū(•)) ∈ W 2 := W 1,2 × L 2 ,
where W 1,2 is the space of absolutely continuous functions with square integrable derivative and L 2 is the space of square integrable functions. In the space W 2 , let us define the subspace

K := { w ∈ W 2 | ẋ(t) = ū(t) a.e., x(t 0 ) = x(t f ) = 0}
and the quadratic form

Ω ( w) = t f t 0 L ww (t, w 0 (t)) w(t), w(t) dt = t f t 0 L xx x(t), x(t) + 2 L xu ū(t), x(t) + L uu ū(t), ū(t) dt.
The following theorem is well-known.

Theorem 1. (a) If the extremal x 0 is a weak minimum, then Ω ( w) ≥ 0 on K . (b) If Ω ( w) is positive definite on K , then the extremal x 0 is a (strict) weak minimum.

As is known, the quadratic conditions in Theorem 1 can be tested via the Jacobi conditions or via bounded solutions to an associated Riccati equation.

For a broken extremal, the quadratic form has to be stated in a different way that allows for the formulation of no-gap necessary and sufficient second-order conditions. We will formulate these conditions and discuss their extensions to different classes of optimal control problems, including bang-bang control problems and problems with mixed constraints and control appearing linearly.

Second-Order Optimality Conditions for Broken Extremals

Let again x 0 (t) be an extremal in the simplest problem (1), [START_REF] Aronna | Quadratic order conditions for bang-singular extremals[END_REF], and let u 0 (t) = ẋ0 (t) be the corresponding control. Assume now that the control u 0 (t) is piecewise continuous with one discontinuity point t * ∈ (t 0 ,t f ). Hence, x 0 (t) is a broken extremal with a corner at t * . We say that t * is an L-point of the function u 0 (t) if there exist ε > 0 and C > 0 such that |u 0 (t)u 0 (t * -)| ≤ C|tt * | for all t ∈ (t * -ε,t * ) and |u 0 (t) -u 0 (t * +)| ≤ C|t -t * | for all t ∈ (t * ,t * + ε). Henceforth, we assume that t * is an L-point of the function u 0 (t). The following question naturally arises: which quadratic form corresponds to a broken extremal?

Let us change the definition of a weak local minimum as follows. Set Θ := {t * } and define a notion of a Θ -weak minimum. Assuming additionally that the control u 0 (t) is left-continuous at t * , denote by cl u 0 (•) the closure of the graph of u 0 (t). Denote by V a neighborhood of the compact set cl u 0 (•). Definition 1. We say that x 0 is a point of a Θ -weak minimum (or an extended weak minimum) if there exits a neighborhood V of the compact set cl u 0 (•) such that J (x) ≥ J (x 0 ) for all admissible x(t) such that u(t) ∈ V a.e., where u(t) = ẋ(t).

Clearly, we have the following chain of implications among minima: strong minimum =⇒ Θ -weak minimum =⇒ weak minimum.

Let us formulate optimality conditions for a Θ -weak minimum. To this end, we introduce the Pontryagin function (Hamiltonian)

H(t, x, u, λ ) = λ u + L(t, x, u),
where λ is a row vector of the dimension n. Defining λ (t) := -L u (t, x 0 (t), u 0 (t)), we have in view of the Euler equation ( 3): H u (t, x 0 (t), u 0 (t), λ (t)) = 0, -λ (t) = H x (t, x 0 (t), u 0 (t), λ (t)).

Denote by [λ ] the jump of the function λ (t) at the point t * , i.e., [λ ] = λ + -λ -, where λ -= λ (t * -) and λ + = λ (t * +). Let [H] stand for the jump of the function H(t) := H(t, x 0 (t), u 0 (t), λ (t)) at the same point. The equalities [λ ] = 0, [H] = 0 constitute the Weierstrass-Erdmann conditions. They are known as necessary conditions for a strong minimum. However, they are also necessary for the Θ -weak minimum. We add one more necessary condition for the Θ -weak minimum:

D(H) := -L + x ẋ0-+ L - x ẋ0+ -[L t ] ≥ 0, where ẋ0-= ẋ0 (t * -), L - x = L x (t * , x 0 (t * -), u 0 (t * -)), [L t ] = L + t -L - t , etc. Clearly, D(H) := λ + ẋ0--λ -ẋ0+ + [ λ0 ],
where λ 0 (t) = -H(t) (recall, that d dt H(t) = H t (t) a.e.). Moreover, it can be shown that D(H) is equal to the negative derivative of the function

∆ H(t) := λ (t)[u 0 ] + L(t, x 0 (t), u 0 (t * +)) -L(t, x 0 (t), u 0 (t * -))
at t * . The existence of the derivative has been proved and, hence, this derivative can also be calculated as d dt (∆ H)(t * -) or as d dt (∆ H)(t * +). Now, let us formulate second order optimality conditions for a Θ -weak minimum. Denote by P Θ W 1,2 the Hilbert space of piecewise continuous functions x(t), absolutely continuous on each of the two intervals [t 0 ,t * ) and (t * ,t f ], and such that their first derivative is square integrable. Any x ∈ P Θ W 1,2 can have a nonzero jump

[ x] = x(t * + 0) -x(t * -0) at the point t * . Let t be a numerical parameter. Denote by Z 2 (Θ ) the space of triples z = (t, x, ū) such that t ∈ IR, x(•) ∈ P Θ W 1,2 , ū(•) ∈ L 2 , i.e., Z 2 (Θ ) = IR × P Θ W 1,2 × L 2 .
In this space, define the quadratic form

Ω Θ (z) = D(H)t 2 -2[L x ] xav t + t f t 0 L ww (t, w 0 (t)) w(t), w(t) dt,
where [L x ] is the jump of the function L x (t, w 0 (t)) at the point t * , and

xav = 1 2 x(t * -) + x(t * +) . Set K Θ = {z ∈ Z 2 (Θ ) : ẋ(t) = ū(t) a.e., [ x] + [ ẋ0 ]t = 0, x(t 0 ) = x(t f ) = 0}. Theorem 2. (a) If x 0 is a Θ -weak minimum, then Ω Θ (z) ≥ 0 on K Θ . (b) If Ω Θ (z) is positive definite on K Θ , then x 0 is a (strict) Θ -weak minimum.
The proof of this theorem is given in [START_REF] Milyutin | Calculus of Variations and Optimal Control[END_REF]. Let us note that in [START_REF] Milyutin | Calculus of Variations and Optimal Control[END_REF], instead of t, we used a numerical parameter ξ such that t = -ξ . This remark also applies to the subsequent presentation.

Second-Order Optimality Conditions for Discontinuous Controls in the General Problem of the Calculus of Variations on a Fixed Time Interval

The General Problem in the Calculus of Variations on a Fixed Time Interval

Now consider the following optimal control problem in Mayer form on a fixed time interval

[t 0 ,t f ]. It is required to find a pair of functions w(t) = (x(t), u(t)), t ∈ [t 0 ,t f ], minimizing the functional min J (w) := J(x(t 0 ), x(t f )) (4)
subject to the constraints

F(x(t 0 ), x(t f )) ≤ 0, K(x(t 0 ), x(t f )) = 0, (x(t 0 ), x(t f ))) ∈ P, ẋ(t) = f (t, x(t), u(t)), h(t, x(t), u(t)) = 0, (t, x(t), u(t)) ∈ Q, (5)
where P and Q are open sets, x, u, F, K, f , and h are vector-functions. We assume that J, F, and K are defined and twice continuously differentiable on P, and f and h are defined and twice continuously differentiable on Q. It is also assumed that the gradients with respect to the control h iu (t, x, u), i = 1, . . . , d(h) are linearly independent at each point (t, x, u) ∈ Q such that h(t, x, u) = 0 (the regularity assumption for the equality constraint h(t, x, u) = 0). Here h i are the components of the vector function h and d(h) is the dimension of this function.

Problem (1), ( 2) is considered in the space

W := W 1,1 ([t 0 ,t f ], IR n ) × L ∞ ([t 0 ,t f ], IR m ),
where n = d(x), m = d(u). Define a norm in this space as a sum of the norms:

w := x 1,1 + u ∞ = |x(t 0 )| + t f t 0 | ẋ(t)| dt + esssup [t 0 ,t f ] |u(t)|.
A weak minimum is defined as a local minimum in the space W . We say that w = (x, u) is an admissible pair if it belongs to W , satisfies the constraints of the problem, and, moreover, there exists a compact set

C ⊂ Q such that (t, x(t), u(t)) ∈ C for a.a. t ∈ [t 0 ,t f ].
It is well-known that an optimal control problem with a functional in Bolza form, min J (w) := J(x(t 0 ), x(t f )) +

t f t 0 f 0 (t, x(t), u(t)) dt, (6)
can be converted to Mayer form by introducing the ODE ẏ = f 0 (t, x, u), y(t 0 ) = 0.

First-Order Necessary Conditions

Let w 0 = (x 0 , u 0 ) be an admissible pair. We introduce the Pontryagin function (or the Hamiltonian)

H(t, x, u, λ ) = λ f (t, x, u)
and the augmented Pontryagin function (or the augmented Hamiltonian)

H a (t, x, u, λ , ν) = H(t, x, u, λ ) + νh(t, x, u),
where λ and ν are row-vectors of the dimensions d(x) = n and d(h), respectively. For brevity we set

x 0 = x(t 0 ), x f = x(t f ), η = (x 0 , x f ).
Denote by IR n * the space of n-dimensional row-vectors. Define the endpoint Lagrange function

l(η, α 0 , α, β ) = α 0 J(η) + αF(η) + β K(η), where α 0 ∈ IR, α ∈ (IR d(F) ) * , β ∈ (IR d(K) ) * . Introduce a tuple of Lagrange multipliers µ = (α 0 , α, β , λ (•), ν(•)) such that λ (•) : [t 0 ,t f ] → IR n * is absolutely continuous and ν(•) : [t 0 ,t f ] → (IR d(h) ) *
is measurable and bounded. Denote by Λ 0 the set of the tuples µ satisfying the following conditions at the point w 0 :

α 0 ≥ 0, α ≥ 0, αF(η 0 ) = 0, α 0 + ∑ d(F) i=1 α i + ∑ d(K) j=1 |β j | = 1, λ = -H a x , λ (t 0 ) = -l x 0 , λ (t f ) = l x f , H a u = 0,
where η 0 = (x 0 (t 0 ), x 0 (t f )), the derivatives l x 0 and l x f are at (η 0 , α 0 , α, β ) and the derivatives H a x , H a u are at (t, x 0 (t), u 0 (t), λ (t), ν(t)), t ∈ [t 0 ,t f ]. By α i and β j we denote the components of the row vectors α and β , respectively. Theorem 3. If w 0 is a weak local minimum, then Λ 0 is nonempty. Moreover, Λ 0 is a finite dimensional compact set, and the projector

(α 0 , α, β , λ (•), ν(•)) → (α 0 , α, β ) is injective on Λ 0 .
The condition Λ 0 = / 0 is called the local Pontryagin minimum principle, or the Euler-Lagrange equation. Let M 0 be the set of all µ = (α 0 , α, β , λ (•), ν(•)) ∈ Λ 0 satisfying the minimum condition for a.a. t ∈ [t 0 ,t f ]:

H(t, x 0 (t), u, λ (t)) ≥ H(t, x 0 (t), u 0 (t), λ (t)) ∀ u ∈ U(t, x 0 (t)), where U(t, x) := {u ∈ IR m | (t, x, u) ∈ Q, h(t, x, u) = 0}.
The condition M 0 = / 0 is called the (integral) Pontryagin minimum principle, which is a necessary condition for the so-called Pontryagin minimum. Definition 2. (A.A. Milyutin). The pair w 0 affords a Pontryagin minimum if for any compact set C ⊂ Q there exists ε > 0 such that J (w) ≥ J (w 0 ) for all admissible pairs w(t) = x(t), u(t) , satisfying the conditions max

[t 0 ,t f ] |x(t) -x 0 (t)| < ε, t f t 0 |u(t) -u 0 (t)| < ε, (t, x(t), u(t)) ∈ C a.e.

Second-Order Necessary Conditions

Set W 2 := W 1,2 ([t 0 ,t f ], IR n ) × L 2 ([t 0 ,t f ], IR m ),
Let K be the set of all w = ( x, ū) ∈ W 2 satisfying the following conditions:

J ′ (η 0 ) η ≤ 0, F ′ i (η 0 ) η ≤ 0 ∀ i ∈ I F (η 0 ), K ′ (η 0 ) η = 0, ẋ(t) = f w (t, w 0 (t)) w(t), for a.a. t ∈ [t 0 ,t f ], h w (t, w 0 (t)) w(t) = 0, for a.a. t ∈ [t 0 ,t f ],
where η = ( x(t 0 ), x(t f )), I F (η 0 ) := {i : F i (η 0 ) = 0} is the set of active indices. Obviously, K is a convex cone in the Hilbert space W 2 . We call it the critical cone.

Let us introduce a quadratic form in W 2 . For µ ∈ Λ 0 and w = ( x, ū) ∈ W 2 , we set

Ω (µ, w) = l µ ηη (η 0 ) η, η + t f t 0 H aµ ww (t) w(t), w(t) dt,
where l µ ηη (η 0 ) = l ηη (η 0 , α 0 , α, β ), H aµ ww (t) = H a ww (t, x 0 (t), u 0 (t), λ (t), ν(t)), and η = ( x(t 0 ), x(t f )). Theorem 4. If w 0 is a weak minimum, then the set Λ 0 is nonempty and

max µ∈Λ 0 Ω (µ, w) ≥ 0 for all w ∈ K .
The necessary condition for a Pontryagin minimum differs from this condition only by replacing the set Λ 0 by the set M 0 .

Theorem 5. If w 0 is a Pontryagin minimum, then the set M 0 is nonempty and max µ∈M 0 Ω (µ, w) ≥ 0 for all w ∈ K .

We now assume that the control u 0 is a piecewise continuous function on [t 0 ,t f ] with the set of discontinuity points Θ = {t 1 , . . . ,t s }, t 0 < t 1 < . . . < t s < t f . We also assume that each t k ∈ Θ is an L-point of the function u 0 (see the definition in Section 2.2). In this case, the regularity assumption for h implies that, for any µ = (α 0 , α, β , λ (•), ν(•)) ∈ Λ 0 , ν(t) has the same properties as u 0 (t): the function ν(t) is piecewise continuous and each of its point of discontinuity is an L-point which belongs to Θ . By virtue of the adjoint equation λ = -H a x , the same is true for the derivative λ (t) of the adjoint variable λ . Now, the second-order necessary conditions can be refined as follows.

For µ ∈ M 0 , set

D k (H aµ ) = λ k+ ẋ0k--λ k-ẋ0k+ -[H aµ t ] k , (7)
where We can calculate D k (H a ) using another method. Namely, D k (H a ) can be calculated as the derivative of the "jump of H a " at the point t k . Introduce the function

(∆ k H a )(t) = (∆ k H)(t) + (∆ k (νh))(t) = λ (t) f (t, x 0 (t), u 0k+ ) -f (t, x 0 (t), u 0k-) + ν k+ h(t, x 0 (t), u 0k+ ) -ν k-h(t, x 0 (t), u 0k-) .
It can be shown that the function (∆ k H a )(t) is continuously differentiable at the point t k ∈ Θ , and its derivative at this point coincides with -D k (H a ). Therefore, we can obtain the value of D k (H a ) by calculating the left or right limit of the derivatives of the function (∆ k H a )(t) at the point t k :

D k (H a ) = - d dt (∆ k H a )(t k ±).
For any µ ∈ M 0 , it can be shown that

D k (H aµ ) ≥ 0, k = 1, . . . , s. Set Z 2 (Θ ) := IR s × P Θ W 1,2 [t 0 ,t f ], IR n × L 2 [t 0 ,t f ], IR m ,
where P Θ W 1,2 ([t 0 ,t f ]; IR n ) is the Hilbert space of piecewise continuous functions x(t), absolutely continuous on each interval of the set [t 0 ,t f ] \ Θ and such that their first derivative is square integrable. Define a quadratic form in Z 2 (Θ ) as follows

Ω Θ (µ, z) = Σ s k=1 D k (H aµ )t 2 k + 2[ λ ] k xk av tk + l µ ηη (η 0 ) η, η + t f t 0 H aµ ww (t) w(t), w(t) dt,
where z = ( θ , x, ū), θ = (t 1 , . . . , ts ), η = ( x(t 0 ), x(t f )), xk av = 1 2 ( x(t k -) + x(t k +)), w = ( x, ū). Define the critical cone K Θ in the same space by the relations

J ′ (η 0 ) η ≤ 0, F ′ i (η 0 ) η ≤ 0 ∀ i ∈ I F (η 0 ), K ′ (η 0 ) η = 0, ẋ(t) = f w (t, w 0 (t)) w(t), for a.a. t ∈ [t 0 ,t f ], [ x] k + [ ẋ0 ] k tk = 0, k = 1, . . . , s h w (t, w 0 (t)) w(t) = 0 for a.a. t ∈ [t 0 ,t f ].
Theorem 6. If w 0 is a Pontryagin minimum, then the following Condition A Θ holds: the set M 0 is nonempty and

max µ∈M 0 Ω Θ (µ, z) ≥ 0 for all z ∈ K Θ .
Let us give another possible representation for the terms D k (H aµ )t 2 k + 2[ λ ] k xk av tk of the quadratic form Ω Θ (µ, z) on the critical cone K . Lemma 1. Let µ ∈ M 0 and z = ( θ , x, ū) ∈ K Θ . Then, for any k = 1, . . . , s, the following formula holds

D k ( Hµ )t 2 k + 2[ λ ] k xk av tk = [ λ0 + λ ẋ0 ] k t2 k + 2[ λ x] k tk . (8)
Proof. Everywhere in this proof we will omit the subscript and superscript k. Taking into account that

D(H a ) = λ + ẋ0--λ -ẋ0+ + [ λ0 ], [ x] + [ ẋ0 ]t = 0, we obtain D(H a )t 2 + 2[ λ ] xav t = t2 [ λ0 ] + t2 λ + ẋ0--λ -ẋ0+ + 2t[ λ ] xav = t2 [ λ0 ] + t2 [ λ ẋ0 ] -λ + [ ẋ0 ] -λ -[ ẋ0 ] + 2t[ λ ] xav = t2 [ λ0 ] + [ λ ẋ0 ] + λ + [ x]t + λ -[ x]t + 2t[ λ ] xav = [ λ0 + λ ẋ0 ]t 2 + λ + ( x+ -x-) + λ -( x+ -x-) + ( λ + -λ -)( x-+ x+ ) t = [ λ0 + λ ẋ0 ]t 2 + 2[ λ x]t.

Second-Order Sufficient Conditions

Here, we will formulate sufficient optimality conditions, but only in the case of discontinuous control u 0 . Let again u 0 be a piecewise continuous function with the set of discontinuity points Θ and let each t k ∈ Θ be an L-point. A natural strengthening of the necessary condition A in Theorem 6 turned out to be sufficient not only for the Pontryagin minimum, but also for the so-called bounded strong minimum. This type of minimum will be defined below.

Definition 3. The component x i of the state vector x is called unessential if the functions f and h do not depend on x i and the functions J, F, and K are affine in x i (t 0 ) and x i (t f ). Let x denote the vector composed by essential components of vector x.

For instance, the integral functional J = t f t 0 f 0 (t, x, u) dt can be brought to the endpoint form: J = y(t f )y(t 0 ), where ẏ = f 0 (t, x, u). Clearly, y is unessential component. Definition 4. An admissible pair w 0 affords a bounded strong minimum if for any compact set C ⊂ Q there exists ε > 0 such that J (w) ≥ J (w 0 ) for all admissible pairs w(t) = x(t), u(t) satisfying the conditions |x(t 0 )x 0 (t 0 )| < ε, max [t 0 ,t f ] |x(t) -x 0 (t))| < ε and (t, x(t), u(t)) ∈ C a.e. on [t 0 ,t f ]. Definition 5. An admissible pair w 0 affords a strong minimum if there exists ε > 0 such that J (w) ≥ J (w 0 ) for all admissible pairs w(t) = x(t), u(t) satisfying the conditions |x(t 0 )x 0 (t 0 )| < ε and max

[t 0 ,t f ] |x(t) -x 0 (t))| < ε.
The following assertion follows from the definitions.

Lemma 2. If there exists a compact set C ⊂ Q such that {(t, x, u) ∈ Q : h(t, x, u) = 0} ⊂ C ,
then the bounded strong minimum is equivalent to the strong minimum.

Let us formulate sufficient conditions for a bounded strong minimum. For µ ∈ M 0 , we introduce the following conditions of the strict minimum principle:

(a) H(t, x 0 (t), u, λ (t)) > H(t, x 0 (t), u 0 (t), λ (t)) for all t ∈ [t 0 ,t f ] \ Θ , u = u 0 (t), u ∈ U(t, x 0 (t)), (b) H(t k , x 0 (t k ), u, λ (t k )) > H k for all t k ∈ Θ , u ∈ U(t k , x 0 (t k )), u = u 0 (t k -), u = u 0 (t k +)
, where

H k := H(t k , x 0 (t k ), u 0 (t k -), λ (t k )) = H(t k , x 0 (t k ), u 0 (t k +), λ (t k )).
We denote by M + 0 the set of all µ ∈ M 0 satisfying conditions (a) and (b). For µ ∈ M 0 we also introduce the strengthened Legendre-Clebsch conditions:

(i) for each t ∈ [t 0 ,t f ] \ Θ the quadratic form H a uu (t, x 0 (t), u 0 (t), λ (t), ν(t))u, u is positive definite on the subspace of vectors u ∈ IR m such that h u (t, x 0 (t), u 0 (t))u = 0.

(ii)for each t k ∈ Θ , the quadratic form

Huu (t k , x 0 (t k ), u 0 (t k -), λ (t k ), ν(t k -))u, u
is positive definite on the subspace of vectors u ∈ IR m such that

h u (t k , x 0 (t k ), u 0 (t k -))u = 0.
(iii) this condition is symmetric to condition (ii) by replacing (t k -) everywhere by (t k +).

Note that for each µ ∈ M 0 the non-strengthened Legendre-Clebsch conditions hold, i.e., the same quadratic forms are nonnegative on the corresponding subspaces.

We denote by Leg + (M + 0 ) the set of all µ ∈ M + 0 satisfying the strengthened Legendre-Clebsch conditions (i), (ii), (iii) and also the conditions (iv) D k (H aµ ) > 0 for all k = 1, . . . , s.

Let us introduce the functional γ(z) = θ , θ + x(t 0 ), x(t 0 ) +

t f t 0 ū(t), ū(t) dt,
where z = ( θ , x, ū) and θ = (t 1 , . . . , ts ). Theorem 7. For the pair w 0 , assume that the following Condition B Θ holds: the set Leg + (M + 0 ) is nonempty and there exist a nonempty compact set M ⊂ Leg + (M + 0 ) and a number C > 0 such that

max µ∈M Ω Θ (µ, z) ≥ C γ(z)
for all z ∈ K . Then the pair w 0 affords a (strict) bounded strong minimum.

The sufficient condition B Θ guarantees a certain growth condition for the cost which will be presented below. We define now the concept of the order function Γ (t, u).

Assuming that the function u

0 (t) is left-continuous, denote by cl u 0 (•) the closure (in IR m+1 ) of its graph. Denote by cl u 0 (t k-1 , t k ) the closure in IR m+1 of the graph of the restriction of u 0 (t) to the interval (t k-1 , t k ), k = 1, . . . , s + 1, where t s+1 = t f . Then cl u 0 (•) = s+1 k=1 cl u 0 (t k-1 , t k ).
Denote by V k , k = 1, . . . , s + 1, a system of non-overlapping neighborhoods of the

compact sets cl u 0 (t k-1 , t k ). Let V = s+1 k=1 V k .
Definition 6. The function Γ (t, u) : IR 1+m → IR is said to be an order function if there exist disjoint neighborhoods V k of the compact sets cl u 0 (t k-1 ,t k ) such that the following five conditions hold:

(1)Γ (t, u) = |u -u 0 (t)| 2 if (t, u) ∈ V k , t ∈ (t k-1 ,t k ), k = 1, . . . , s + 1; (2)Γ (t, u) = 2|t -t k | + |u -u 0k-| 2 if (t, u) ∈ V k , t > t k , k = 1, . . . , s; (3)Γ (t, u) = 2|t -t k | + |u -u 0k+ | 2 if (t, u) ∈ V k+1 , t < t k , k = 1, . . . , s; (4)Γ (t, u) > 0 if (t, u) / ∈ V ; (5)Γ (t, u) is Lipschitz continuous on each compact set in IR 1+m . E T t u E ' t 0 t f t 1 u 1- u 0 (t) u 01+ u 0 (t) Γ (t, u) > 0 V 1 V 2 Γ (t, u) > 0 ' & $ % Γ (t, u) = |u -u 0 (t)| 2 T Γ (t, u) = |u -u 01-| 2 + 2|t -t 1 | T Γ (t, u) = |u -u 0 (t)| 2 c Γ (t, u) = |u -u 01+ | 2 + 2|t -t 1 | E Fig. 1 Illustration of the order function Γ (t, u). For δ w(t) = (δ x(t), δ u(t)) in W we set γ(δ w) = δ x 2 ∞ + t f t 0 Γ (t, u 0 (t) + δ u(t)) dt.
We call γ the higher order. This higher order corresponds to a typical minimum in the case of discontinuous control, and the order function Γ (t, v) corresponds to a typical Hamiltonian in this case.

Note that the order

t f t 0 Γ (t, u 0 (t)+ δ u(t)
) dt is much finer (smaller) than the functional

t f t 0 |δ u(t))| 2 dt.
On the other hand, it can be proved that, on each compact set (in IR m ), the following lower bound holds for

t f t 0 Γ (t, u 0 (t) + δ u(t)) dt : t f t 0 |δ u(t)| dt 2 ≤ C t f t 0 Γ (t, u 0 (t) + δ u(t)) dt,
where C > 0 depends only on the compact set.

Define the violation function at the point w 0 :

V (δ w) = (J(η 0 +δ η)-J(η 0 )) + + d(F) ∑ i=1 F i (η 0 +δ η) + +|K(η 0 +δ η)|+ δ ẋ-δ f 1 , where η 0 = (x 0 (t 0 ), x 0 (t f )), δ η = (δ x(t 0 ), δ x(t f )), δ f = f (t, w 0 + δ w) -f (t, w 0 ), δ w = (δ x, δ u), .
1 is the norm in the space L 1 of integrable functions, and a + := max{a, 0} for a ∈ IR.

Definition 7. We say that a bounded strong γ-sufficiency holds at the point w 0 if there exists C > 0 such that for any compact set C ⊂ Q there exists ε > 0 such that the inequality V (δ w) ≥ Cγ(δ w) holds for all δ w = (δ x, δ u) ∈ W satisfying the conditions

|δ x(t 0 )| < ε, δ x ∞ < ε, (t, w 0 (t) + δ w(t)) ∈ C , h(t, w 0 (t) + δ w(t)) = 0 a.e. (9)
Obviously, a bounded strong γ-sufficiency implies a (strict) bounded strong minimum. Moreover, if the point w 0 + δ w is admissible, then, obviously, V (δ w) = (J(w 0 + δ w) -J(w 0 )) + . Therefore, a bounded strong γ-sufficiency implies the following γ-growth condition for the cost: there exists C > 0 such that for any compact set C ⊂ Q there exists ε > 0 such that

J(w 0 + δ w) -J(w 0 ) ≥ Cγ(δ w)
for all δ w = (δ x, δ u) ∈ W satisfying ( 9) and such that (w 0 + δ w) is an admissible pair.

Theorem 8. The sufficient condition B Θ in Theorem 7 is equivalent to the bounded strong γ-sufficiency.

Theorems 6-8 were proved in [START_REF] Osmolovskii | Quadratic optimality conditions for broken extremals in the general problem of calculus of variations[END_REF]. Generalizations of these theorems for optimal control problem with regular mixed inequality state-control constraints were recently published in [START_REF] Osmolovskii | Sufficient quadratic conditions of extremum for discontinuous controls in optimal control problem with mixed constraints[END_REF] and [START_REF] Osmolovskii | Necessary quadratic conditions of extremum for discontinuous controls in optimal control problem with mixed constraints[END_REF]. An extension of the results of this section to problems on a variable time interval were obtained in [START_REF] Osmolovskii | Second order conditions in optimal control problem with mixed equality-type constraints on a variable time interval[END_REF].

The General Problem in the Calculus of Variations on a Variable Time Interval

Statement of the Problem

Here, quadratic optimality conditions, both necessary and sufficient, are presented in the following canonical problem on a variable time interval. Let T denote a process (x(t), u(t

) | t ∈ [t 0 ,t f ]),
η = (t 0 , x(t 0 ),t f , x(t f ))
the vector of the endpoints of time-state variable (t, x). It is required to find T minimizing the functional min J (T ) := J(η) [START_REF] Dubovitskii | Extremum problems in the presence of restrictions[END_REF] subject to the constraints

F(η) ≤ 0, K(η) = 0, η ∈ P, (11) ẋ
(t) = f (t, x(t), u(t)), h(t, x(t), u(t)) = 0, (t, x(t), u(t)) ∈ Q, (12)
where P and Q are open sets, x, u, F, K, f , and h are vector-functions. We assume that the functions J, F, and K are defined and twice continuously differentiable on P, and the functions f and h are defined and twice continuously differentiable on Q. It is also assumed that the gradients with respect to the control h iu (t, x, u), i = 1, . . . , d(h), are linearly independent at each point (t, x, u) ∈ Q such that h(t, x, u) = 0. Here d(h) is a dimension of the vector h.

First-Order Necessary Conditions

We say that the function u(t) is Lipschitz-continuous if it is piecewise continuous and satisfies the Lipschitz condition on each interval of the continuity. Let

T = (x(t), u(t) | t ∈ [t 0 ,t f ]) (13)
be a fixed admissible process such that the control u(•) is a piecewise Lipschitzcontinuous function on the interval ∆ with the set of discontinuity points

Θ = {t 1 , . . . ,t s }, where t 0 < t 1 < . . . < t s < t f .
In order to make the notations simpler we do not use here such symbols and indices as zero, hat or asterisk to distinguish the process T from others.

Let us formulate the first-order necessary condition for optimality of the process T . We introduce the Pontryagin function H (Hamiltonian), the augmented Pontryagin function H a , and the endpoint Lagrange function l as in Section 3.2, but remember that now η = (t 0 , x 0 ,t f , x f ), Also we introduce a tuple of Lagrange

multipliers µ = (α 0 , α, β , λ (•), λ 0 (•), ν(•)) (14)
such that λ (•) : ∆ → (IR d(x) ) * and λ 0 (•) : ∆ → IR 1 are piecewise smooth functions, continuously differentiable on each interval of the set ∆ \Θ , and ν(•) : ∆ → (IR d(h) ) * is a piecewise continuous function and Lipschitz continuous on each interval of the set ∆ \ Θ . Denote by M 0 the set of the normed tuples µ satisfying the conditions of the minimum principle for the process T :

α 0 ≥ 0, α ≥ 0, αF(η) = 0, α 0 + ∑αi + ∑ |β j | = 1, λ = -H a x , λ0 = -H a t , H a u = 0, t ∈ ∆ \ Θ , λ (t 0 ) = -l x 0 , λ (t f ) = l x f , λ 0 (t 0 ) = -l t 0 , λ 0 (t f ) = l t f , min u∈U(t,x(t)) H(t, x(t), u, λ (t)) = H(t, x(t), u(t), λ (t)), t ∈ ∆ \ Θ , H(t, x(t), u(t), λ (t)) + λ 0 (t) = 0, t ∈ ∆ \ Θ , (15)
where U(t, x) = {u ∈ IR d(u) | h(t, x, u) = 0, (t, x, u) ∈ Q}. The derivatives l x 0 and l x f are at (η, α 0 , α, β ), where η = (t 0 , x(t 0 ),t f , x(t f )), and the derivatives H a

x , H a u , and H a t are at (t, x(t), u(t), λ (t), ν(t)), where t ∈ ∆ \Θ . (Condition H a u = 0 follows from other conditions in this definition, and therefore, could be excluded; yet we need to use it later.)

Let us give the definition of Pontryagin minimum in problem ( 10)-( 12) on a variable interval [t 0 ,t f ]. Definition 8. The process T affords a Pontryagin minimum if for each compact set

C ⊂ Q there exists ε > 0 such that J ( T ) ≥ J (T ) for all admissible processes T = ( x(t), ũ(t) | t ∈ [t 0 , t f ]) satisfying the conditions (a) |t 0 -t 0 | < ε, |t f -t f | < ε, (b) max ∆ ∩∆ | x(t) -x(t)| < ε, where ∆ = [t 0 , t f ], (c)
∆ ∩∆ | ũ(t) -u(t)| dt < ε, (d) (t, x(t), ũ(t)) ∈ C a.e. on ∆ .
The condition M 0 = / 0 is equivalent to Pontryagin's minimum principle. It is the first-order necessary condition for Pontryagin minimum for the process T . Thus, the following theorem holds. Theorem 9. If the process T affords a Pontryagin minimum, then the set M 0 is nonempty.

Assume that the set M 0 is nonempty. Using its definition and the full rank condition for the matrix h u on the surface h = 0 one can easily prove the following statement: Proposition 1. The set M 0 is a finite-dimensional compact set, and the mapping µ → (α 0 , α, β ) is injective on M 0 .

As in Section 3, for each µ ∈ M 0 ,t k ∈ Θ , we define D k (H aµ ) by relation [START_REF] Üskens | Real-time solutions of bang-bang and singular optimal control problems[END_REF]. Then, for each µ ∈ M 0 the following inequalities hold: D k (H aµ ) ≥ 0, k = 1, . . . , s.

Second-Order Necessary Conditions

Let us formulate a quadratic necessary condition for a Pontryagin minimum for the process T as in [START_REF] Fourer | AMPL: A Modeling Language for Mathematical Programming[END_REF]. First, for this process, we introduce a Hilbert space Z 2 (Θ ) and the critical cone K ⊂ Z 2 (Θ ). Again, we denote by P Θ W 1,2 (∆ , IR d(x) ) the Hilbert space of piecewise continuous functions x(•) : ∆ → IR d(x) , absolutely continuous on each interval of the set ∆ \Θ and such that their first derivative is square integrable. We set z = (t 0 , t f , θ , x, ū) ,

where t0 ∈ IR 1 , t f ∈ IR 1 , θ = (t 1 , . . . , ts ) ∈ IR s , x ∈ P Θ W 1,2 (∆ , IR d(x) ), ū ∈ L 2 (∆ , IR d(u) ). Thus, z ∈ Z 2 (Θ ) := IR 2 × IR s × P Θ W 1,2 (∆ , IR d(x) ) × L 2 (∆ , IR d(u) ).
Moreover, for given z we set

w = ( x, ū), x0 = x(t 0 ), x f = x(t f ), ( 16
) x0 = x(t 0 ) + t0 ẋ(t 0 ), x f = x(t f ) + t f ẋ(t f ), η = (t 0 , x0 , t f , x f ). ( 17
)
By

I F (η) = {i ∈ {1, . . . , d(F)} | F i (η) = 0}
we denote the set of active indices of the constraints F i ≤ 0. Let K Θ be the set of all z ∈ Z 2 (Θ ) satisfying the following conditions:

J ′ (η) η ≤ 0, F ′ i (η) η ≤ 0 ∀i ∈ I F (η), K ′ (η) η = 0, ẋ(t) = f w (t, w(t)) w(t), for a.a. t ∈ [t 0 ,t f ], [ x] k + [ ẋ] k tk = 0, k = 1, . . . , s h w (t, w(t)) w(t) = 0, for a.a. t ∈ [t 0 ,t f ]. (18)
Clearly, K Θ is a convex cone in the Hilbert space Z 2 (Θ ). We call it the critical cone. If the interval ∆ is fixed, then we set η := (x 0 , x f ) = (x(t 0 ), x(t f )), and in the definition of K we have t0

= t f = 0, x0 = x0 , x f = x f , and η = η := ( x0 , x f ).
Let us introduce a quadratic form on Z 2 (Θ ). For µ ∈ M 0 and z ∈ K Θ , we set

2Ω Θ (µ, z) = l µ ηη η, η + t f t 0 H aµ ww w(t), w(t) dt + s ∑ k=1 D k (H aµ )t 2 k + 2[ λ ] k xk av tk + λ (t 0 ) ẋ(t 0 ) + λ0 (t 0 ) t2 0 + 2 λ (t 0 ) x(t 0 )t 0 -λ (t f ) ẋ(t f ) + λ0 (t f ) t2 f -2 λ (t f ) x(t f )t f , (19)
where l µ ηη = l ηη (η, α 0 , α, β ), H aµ ww = H a ww (t, x(t), u(t), λ (t), ν(t)). We now formulate the main necessary quadratic condition of Pontryagin minimum in the problem on a variable time interval.

Theorem 10. If the process T yields a Pontryagin minimum, then the following

Condition A Θ holds: the set M 0 is nonempty and

max µ∈M 0 Ω Θ (µ, z) ≥ 0 for all z ∈ K Θ .
Using [START_REF] Chen | Smoothing techniques of nonsmooth Newton methods for controlstate constrained optimal control problems[END_REF], we can represent the quadratic form Ω Θ on K Θ as follows

2Ω Θ (µ, z) = l µ ηη η, η + t f t 0 H aµ ww w(t), w(t) dt + s ∑ k=1 [ λ ẋ + λ0 ] k t2 k + 2[ λ x] k tk + λ (t 0 ) ẋ(t 0 ) + λ0 (t 0 ) t2 0 + 2 λ (t 0 ) x(t 0 )t 0 -λ (t f ) ẋ(t f ) + λ0 (t f ) t2 f -2 λ (t f ) x(t f )t f . ( 20
)

Second-Order Sufficient Conditions

Let us give the definition of a bounded strong minimum in problem ( 10)-( 12) on a variable interval [t 0 ,t f ]. Let again x denotes a vector composed of all essential components of vector x (cf. Definition 3).

Definition 9. The process T affords a bounded strong minimum if for each compact set C ⊂ Q there exists ε > 0 such that J ( T ) ≥ J (T ) for all admissible processes T = (

x(t), ũ(t) | t ∈ [t 0 , t f ]) satisfying the conditions (a) |t 0 -t 0 | < ε, |t f -t f | < ε, | x(t 0 ) -x(t 0 )| < ε, (b) max ∆ ∩∆ | x(t) -x(t)| < ε, where ∆ = [t 0 , t f ], (c) (t, x(t), ũ(t)) ∈ C a.e. on ∆ .
The strict bounded strong minimum is defined in a similar way, with the nonstrict inequality J ( T ) ≥ J (T ) replaced by the strict one and the process T required to be different from T . Let us formulate a sufficient optimality Condition B Θ , which is a natural strengthening of the necessary Condition A Θ . The condition B Θ is sufficient not only for a Pontryagin minimum, but also for a strict bounded strong minimum. Theorem 11. For the process T , assume that the following Condition B Θ holds: the set Leg + (M + 0 ) is nonempty and there exist a nonempty compact set M ⊂ Leg + (M + 0 ) and a number C > 0 such that

max µ∈M Ω Θ (µ, z) ≥ C γ(z) ( 21
)
for all z ∈ K Θ . Then the process T affords a strict bounded strong minimum.

Here the set Leg + (M + 0 ) has the same definition as in Section 3.4.

5 Second-Order Optimality Conditions for Bang-Bang Controls

Optimal Control Problems with Control Appearing Linearly

Let again T denote a process (x(t), u(t) | t ∈ [t 0 ,t f ]), where the time interval ∆ = [t 0 ,t f ] is not fixed. As above, we set η = (t 0 , x(t 0 ),t f , x(t f )).

We will refer to the following control problem ( 22)-( 24) as the basic control problem:

Minimize J (T ) := J(η) [START_REF] Malanowski | Sensitivity analysis for parametric control problems with control-state constraints[END_REF] subject to the constraints

F(η) ≤ 0, K(η) = 0, η ∈ P, (23)
ẋ(t) = f (t, x(t)) + g(t, x(t))u(t), u(t) ∈ U, (t, x(t)) ∈ Q, t 0 ≤ t ≤ t f . ( 24
)
Here x ∈ IR n , u ∈ IR m , F, K, and f are vector functions, g is n × m matrix function with column vector functions g 1 (t, x, u), . . . , g m (t, x, u), P ⊂ IR 2n+2 and Q ⊂ IR n+1 are open sets, U ⊂ IR m is a convex polyhedron. The functions J, F, and K are assumed to be twice continuously differential on P, and the functions f and g are twice continuously differential on Q.

A process T = (x(t), u(t) | t ∈ [t 0 ,t 1 ]
) is said to be admissible if x(•) is absolutely continuous, u(•) is measurable bounded and the pair of functions (x(t), u(t)) on the interval ∆ = [t 0 ,t 1 ] with the end-points η = (t 0 , x(t 0 ),t 1 , x(t 1 )) satisfies the constraints ( 23), [START_REF] Maurer | Optimization methods for the verification of second-order sufficient conditions for bang-bang controls[END_REF].

Let us give the definition of Pontryagin minimum for the basic problem.

Definition 10. The process

T = ( x(t), u(t) | t ∈ [ t 0 , t f ]
) affords a Pontryagin minimum in the basic problem if there exists ε > 0 such that J (T ) ≥ J ( T ) for all admissible processes T = (x(t), u(t

) | t ∈ [t 0 ,t f ]) satisfying |t 0 -t 0 | < ε, |t 1 -t 1 | < ε, max ∆ ∩ ∆ |x(t) -x(t)| < ε, ∆ ∩ ∆ |u(t) -u(t)| dt < ε, where ∆ = [t 0 ,t f ], ∆ = [ t 0 , t f ].
Note that, for a fixed time interval ∆ , a Pontryagin minimum corresponds to an L 1 -local minimum with respect to the control variable.

Necessary Optimality Conditions: The Minimum Principle of Pontryagin et al

Let T = (x(t), u(t) | t ∈ [t 0 ,t f ])
H(t, x, u, λ ) = λ f (t, x) + λ g(t, x)u = λ f (t, x) + m ∑ i=1 λ g i (t, x)u i , ( 25
)
where λ is a row-vector of the dimension d(λ ) = d(x) = n while x, u, f , F and K are column-vectors. The factor of the control u in the Pontryagin function is the switching vector function, a row vector of dimension d(u) = m.

σ (t, x, λ ) : = H u (t, x, u, λ ) = λ g(t, x), σ i (t, x, λ ) : = H u i (t, x, u, λ ) = λ g i (t, x), i = 1, . . . , m, σ i (t) : = σ i (t, x(t), u(t)). ( 26
)
The endpoint Lagrange function is

l(α 0 , α, β , η) = α 0 J(η) + αF(η) + β K(η),
where α and β are row-vectors with

d(α) = d(F), d(β ) = d(K), and α 0 is a number. By µ = (α 0 , α, β , λ (•), λ 0 (•))
we denote a tuple of Lagrange multipliers such that λ (•) : ∆ → IR n * and λ 0 (•) : ∆ → IR are continuous on ∆ and continuously differentiable on each interval of the set ∆ \ Θ .

Let M 0 be the set of the normed collections µ satisfying the conditions of Minimum Principle for the process T :

α 0 ≥ 0, α ≥ 0, αF(η) = 0, α 0 + d(F) ∑ i=1 α i + d(K) ∑ j=1 |β j | = 1, (27) λ
= -H x , λ0 = -H t ∀t ∈ ∆ \ Θ , (28)
λ (t 0 ) = -l x 0 , λ (t f ) = l x f , λ 0 (t 0 ) = -l t 0 , λ 0 (t f ) = l t f , (29)
min u∈U H(t, x(t), u, λ (t)) = H(t, x(t), u(t), λ (t)) ∀t ∈ ∆ \ Θ , (30)
H(t, x(t), u(t), λ (t)) + λ 0 (t) = 0 ∀t ∈ ∆ \ Θ . (31)
The derivatives l x 0 and l x f are taken at the point (α 0 , α, β , η), and the derivatives H x , H t are evaluated at the point (t, x(t), u(t), λ (t)). We use the simple abbreviation (t) for indicating all arguments (t, x(t), u(t), λ (t)), t ∈ ∆ \ Θ .

Theorem 12.

If the process T affords a Pontryagin minimum, then the set M 0 is nonempty. The set M 0 is a finite-dimensional compact set and the projector µ → (α 0 , α, β ) is injective on M 0 .

In view of this theorem, we can identify each tuple µ ∈ M 0 with its projection (α 0 , α, β ). In what follows we set µ = (α 0 , α, β ). For each µ ∈ M 0 and t k ∈ Θ , we define again the quantity D k (H µ ). Set

(∆ k H)(t) = H(t, x(t), u k+ , λ (t)) -H(t, x(t), u k-, λ (t)) = σ (t) [u] k . ( 32
)
For each µ ∈ M 0 the following equalities hold

d dt (∆ k H) t=t k -= d dt (∆ k H) t=t k + , k = 1, . . . , s.
Consequently, for each µ ∈ M 0 the function (∆ k H)(t) has a derivative at the point

t k ∈ Θ . Set D k (H µ ) = - d dt (∆ k H)(t k ).
Then, for each µ ∈ M 0 , the minimum condition (30) implies the inequalities:

D k (H µ ) ≥ 0, k = 1, . . . , s. ( 33
)
As we know, the value D k (H µ ) can be written in the form

D k (H µ ) = -H k+ x H k- λ + H k- x H k+ λ -[H t ] k = λ k+ ẋk--λ k-ẋk+ + [λ 0 ] k ,
where H k- x and H k+ x are the left hand and the right hand values of the function H x (t) := H x (t, x(t), u(t), λ (t)) at t k , respectively, [H t ] k is a jump of the function H t (t) at t k , etc. It also follows from the above representation that we have

D k (H µ ) = -σ(t k ±)[u] k , ( 34
)
where the values on the right hand side agree for the derivative σ (t k +) from the right and the derivative σ (t k -) from the left. In the case of a scalar control u, the total derivative σ t + σ x ẋ + σ λ λ does not contain the control variable explicitly and hence the derivative σ (t) is continuous at t k .

Definition 11. For a given extremal process T = { (x(t), u(t)) | t ∈ ∆ } with a piecewise constant control u(t) we say that u(t) is a strict bang-bang control if there exists

µ = (α 0 , α, β , λ , λ 0 ) ∈ M 0 such that Arg min u ′ ∈U σ (t)u ′ = [u(t-), u(t+)] , t ∈ [t 0 ,t f ] (35)
where [u(t-), u(t+)] denotes the line segment spanned by the vectors u(t-) and u(t+) in IR d(u) and σ (t) := σ (t, x(t), λ (t)) = λ (t)g(t, x(t)).

Note that [u(t-), u(t+)] is a singleton {u(t)} at each continuity point of the control u(t) with u(t) being a vertex of the polyhedron U. Only at the points t k ∈ Θ does the line segment [u k-, u k+ ] coincide with an edge of the polyhedron.

It is instructive to evaluate the condition [START_REF] Osmolovskii | Quadratic optimality conditions for broken extremals in the general problem of calculus of variations[END_REF] in greater detail when the control set is the hypercube

U = d(u) ∏ i=1 [u i,min , u i,max ], u i,min < u i,max (i = 1, ..., d(u)). ( 36
)
Let S i = {t k,i , k = 1, ..., k i }, k i ≥ 0, be the set of switching times of the ith control component u i (t) and let σ i (t) = λ (t)g i (t, x(t)) be switching function for

u i , i = 1, ..., d(u).
Then the set of all switching times is given by

Θ = {t 1 , . . . ,t s } = d(u) i=1 S i ,
and the condition [START_REF] Osmolovskii | Quadratic optimality conditions for broken extremals in the general problem of calculus of variations[END_REF] for a strict bang-bang control requires that

(a) σ i (t) = 0 ∀ t / ∈ S i (i = 1, . . . , d(u)), (b) there is no simultaneous switching of control components u i (t), i.e., S i ∩ S j = / 0 ∀ i = j. ( 37
)
Hence, the i-th control component is determined by the control law

u i (t) = u i,min , if σ i (t) > 0 u i,max , if σ i (t) < 0 ∀t ∈ ∆ \ S i . (38)
Remark 1. There exist examples, where condition (a) in ( 37) is slightly violated as σ i (t f ) = 0 holds for certain control components u i ; cf. the Rayleigh problem in Section 9.2 and the collision avoidance problem in Maurer et al. [START_REF] Maurer | Computation of optimal bang-bang and singular controls in planar collision avoidance[END_REF]. In this case, we require in addition that σi (t f ) = 0 holds to compensate for the condition σ i (t f ) = 0. This property is fulfilled for the Rayleigh problem in Section 9.2 and the control problem in [START_REF] Maurer | Computation of optimal bang-bang and singular controls in planar collision avoidance[END_REF].

Second-Order Necessary Optimality Conditions

Here, we formulate quadratic necessary optimality conditions for a Pontryagin minimum for a given bang-bang control. (Their strengthening yields quadratic sufficient conditions for a strong minimum.) These quadratic conditions are based on the properties of a quadratic form on the critical cone.

Let again T = (x(t), u(t) | t ∈ [t 0 ,t f ]) be a fixed admissible process such that the control u(•) is a piecewise constant function on the interval ∆ = [t 0 ,t f ], and let Θ = {t 1 , . . . ,t s }, t 0 < t 1 < . . . < t s < t f , be the set of discontinuity points of the control u(t). For the process T , we introduce the space Z (Θ ) and the critical cone K Θ ⊂ Z (Θ ) as follows. Denote by P Θ C 1 (∆ , IR d(x) ) the space of piecewise continuous functions x(•) : ∆ → IR d(x) that are continuously differentiable on each interval of the set ∆ \ Θ . For each x ∈ P Θ C 1 (∆ , IR d(x) ) and for

t k ∈ Θ we set xk-= x(t k -), xk+ = x(t k +) and [ x] k = xk+ -xk-. Now set z = (t 0 , t f , θ , x), where t0 , t f ∈ IR 1 , θ = (t 1 , . . . , ts ) ∈ IR s , x ∈ P Θ C 1 (∆ , IR d(x) ). Thus, z ∈ Z (Θ ) := IR 2 × IR s × P Θ C 1 (∆ , IR d(x) ).
For each z we set

x0 = x(t 0 ) + t0 ẋ(t 0 ), x f = x(t f ) + t f ẋ(t f ), η = t0 , x0 , t f , x f . ( 39
)
The vector η is considered as a column vector. Note that t0 = 0, respectively, t f = 0 for fixed initial time t 0 , respectively, final time t f . Let

I F (η) = {i ∈ {1, . . . , d(F)} | F i (η) = 0}
be the set of indices of all active endpoint inequalities F i ≤ 0 at the point η = (t 0 , x(t 0 ),t f , x(t f )). Denote by K Θ the set of all z ∈ Z (Θ ) satisfying the following conditions:

J ′ (η) η ≤ 0, F ′ i (η) η ≤ 0 ∀ i ∈ I F (η), K ′ (η) η = 0, (40) ẋ
(t) = ( f x (t, x(t)) + g x (t, x(t))u(t)) x(t), (41)
[ x] k + [ ẋ] k tk = 0, k = 1, . . . , s. ( 42
)
It is obvious that K Θ is a convex, finite-dimensional and finite-faced cone in the space Z (Θ ). We call it the critical cone. Each element z ∈ K Θ is uniquely defined by the numbers t0 , t f , the vector θ and the initial value x(t 0 ) of the function x(t). Two important properties of the critical cone are formulated in the next two propositions.

Proposition 2. For any µ ∈ M 0 and z ∈ K Θ , we have

α 0 J ′ (η) η = 0, α i F ′ i (η) η = 0 ∀ i ∈ I F (η).
Proposition 3. Suppose that there exist µ ∈ M 0 with α 0 > 0. Then adding the equal- ities α i F ′ i (η) η = 0 ∀i ∈ I F (η) to the system ( 40)- [START_REF] Pontryagin | The Mathematical Theory of Optimal Processes[END_REF] defining K Θ , one can omit the inequality J ′ (η) η ≤ 0 in that system without affecting K Θ . Thus, K Θ is defined by conditions ( 41), [START_REF] Pontryagin | The Mathematical Theory of Optimal Processes[END_REF] and by the condition η ∈ K e Θ , where K e Θ is the cone in IR 2d(x)+2 given by [START_REF] Osmolovskii | Applications to Regular and Bang-Bang Control: Second-Order Necessary and Sufficient Optimality Conditions in Calculus of Variations and Optimal Control[END_REF]. But if there exists µ ∈ M 0 with α 0 > 0, then we can put

K e Θ = { η ∈ IR d(x)+2 | F ′ i (η) η ≤ 0, α i F ′ i (η) η = 0 ∀ i ∈ I F (η), K ′ (η) η = 0}. (43)
If, in addition, α i > 0 holds for all i ∈ I F (η), then K e Θ is a subspace in IR d(x)+2 .

Let us introduce a quadratic form on the critical cone K Θ defined by the conditions ( 40)- [START_REF] Pontryagin | The Mathematical Theory of Optimal Processes[END_REF]. For each µ ∈ M 0 and z ∈ K Θ we set

2Ω Θ (µ, z) = l µ ηη η, η + t f t 0 H µ xx x(t), x(t) dt + s ∑ k=1 D k (H µ )t 2 k + 2[ λ ] k xk av tk + λ (t 0 ) ẋ(t 0 ) + λ0 (t 0 ) t2 0 + 2 λ (t 0 ) x(t 0 )t 0 -λ (t f ) ẋ(t f ) + λ0 (t f ) t2 f -2 λ (t f ) x(t f )t f , (44)
where

l µ ηη = l ηη (η, α 0 , α, β ), H µ xx = H xx (t, x(t), u(t), λ (t)
) and η was defined in [START_REF] Osmolovskii | Equivalence of second order optimality conditions for bang-bang control problems. Part 2: Proof, variational derivatives and representations[END_REF]. Note that for a problem on a fixed time interval [t 0 ,t f ] we have t0 = t f = 0. The following theorem gives the main second order necessary condition of optimality.

Theorem 13. If the process T affords a Pontryagin minimum, then the following

Condition A Θ holds: the set M 0 is nonempty and max µ∈M 0 Ω Θ (µ, z) ≥ 0 for all z ∈ K Θ .

Using [START_REF] Chen | Smoothing techniques of nonsmooth Newton methods for controlstate constrained optimal control problems[END_REF], we can also represent the quadratic form Ω Θ as follows

2Ω Θ (µ, z) = l µ ηη η, η + t f t 0 H µ xx x(t), x(t) dt + s ∑ k=1 [ λ ẋ + λ0 ] k t2 k + 2[ λ x] k tk + λ (t 0 ) ẋ(t 0 ) + λ0 (t 0 ) t2 0 + 2 λ (t 0 ) x(t 0 )t 0 -λ (t f ) ẋ(t f ) + λ0 (t f ) t2 f -2 λ (t f ) x(t f )t f . (45)

Second-Order Sufficient Optimality Conditions (SSC)

The state variable x i is called unessential if the function f does not depend on x i and the functions F, J, K are affine in x i0 := x i (t 0 ) and x i f := x i (t f ). Let x denote the vector of all essential components of state vector x. Let us define a strong minimum in the basic problem.

Definition 12. The process T affords a strong minimum if there exists ε > 0 such that J ( T ) ≥ J (T ) for all admissible processes T = (

x(t), ũ(t) | t ∈ [t 0 , t f ]) sat- isfying the conditions (a) |t 0 -t 0 | < ε, |t f -t f | < ε, | x(t 0 ) -x(t 0 )| < ε, (b) max ∆ ∩∆ | x(t) -x(t)| < ε, where ∆ = [t 0 , t f ],
The strict strong minimum is defined in a similar way, with the non-strict inequality J ( T ) ≥ J (T ) replaced by the strict one and the process T required to be different from T . A natural strengthening of the necessary Condition A Θ of Theorem 13 turns out to be a sufficient optimality condition not only for a Pontryagin minimum, but also for a strong minimum. Theorem 14. Let the following Condition B Θ be fulfilled for the process T : (a) u(t) is a strict bang-bang control (i.e., there exists µ ∈ M 0 such that condition [START_REF] Osmolovskii | Quadratic optimality conditions for broken extremals in the general problem of calculus of variations[END_REF] holds), (b) there exists µ ∈ M 0 such that D k (H µ ) > 0, k = 1, . . . , s, (c) max

µ∈M 0 Ω Θ (µ, z) > 0 for all z ∈ K Θ \ {0}.
Then T is a strict strong minimum.

Note that the condition (c) is automatically fulfilled if K Θ = {0}, which gives a first-order sufficient condition for a strong minimum in the problem. Also note that the condition (c) is automatically fulfilled if there exists µ ∈ M 0 such that

Ω Θ (µ, z) > 0 for all z ∈ K Θ \ {0}. (46)
Clearly, there is no gap between the necessary condition A Θ of Theorem 13 and the sufficient condition B Θ of Theorem 14.

Induced Optimization Problem for Bang-Bang Controls and the Verification of SSC

We continue our discussion of bang-bang controls. Second-order sufficient optimality conditions for bang-bang controls had been derived in the literature in two different forms. The first form was discussed in the last section. The second one is due to Agrachev, Stefani, and Zezza [START_REF] Agrachev | Strong optimality for a bang-bang trajectory[END_REF], who first reduce the bang-bang control problem to a finite-dimensional optimization problem and then show that the well-known sufficient optimality conditions for this optimization problem supplemented by the strict bang-bang property furnish sufficient conditions for the bang-bang control problem. The bang-bang control problem, considered in this section, is more general than that in [START_REF] Agrachev | Strong optimality for a bang-bang trajectory[END_REF]. Following [START_REF] Osmolovskii | Equivalence of second order optimality conditions for bang-bang control problems. Part 2: Proof, variational derivatives and representations[END_REF], we claim the equivalence of both forms of second order conditions for this problem.

Formulation of the Induced Optimization Problem and Necessary Optimality Conditions

Let T = ( x(t), u(t) | t ∈ [ t 0 , t f ]) be an admissible process for the basic control problem ( 22)- [START_REF] Maurer | Optimization methods for the verification of second-order sufficient conditions for bang-bang controls[END_REF]. We denote by ex U the set of vertices of the polyhedron U. Assume that u(t) is a bang-bang control in ∆ = [ t 0 , t f ] taking values in the set ex U,

u(t) = u k ∈ ex U for t ∈ ( t k-1 , t k ), k = 1, . . . , s + 1,
where t s+1 = t f . Thus, Θ = { t 1 , . . . , t s } is the set of switching points of the control u(•) with t k < t k+1 for k = 0, 1, ..., s. Assume now that the set M 0 of multipliers is nonempty for the process T . Put

x( t 0 ) = x 0 , θ = ( t 1 , . . . , t s ), ζ = ( t 0 , t f , x 0 , θ ). (47)
Then

θ ∈ IR s , ζ ∈ IR 2 × IR n × IR s , where n = d(x).
Take a small neighborhood V of the point ζ in IR 2 × IR n × IR s , and let

ζ = (t 0 ,t f , x 0 , θ ) ∈ V ,
where θ = (t 1 , . . . ,t s ) satisfies t 0 < t 1 < t 2 < . . . < t s < t f . Define the function u(t;t 0 ,t f , θ ) by the condition

u(t;t 0 ,t f , θ ) = u k for t ∈ (t k-1 ,t k ), k = 1, . . . , s + 1, (48)
where t s+1 = t f . The values u(t k ;t 0 ,t f , θ ), k = 1, . . . , s, may be chosen in U arbitrarily. For definiteness, define them by the condition of continuity of the control from the left:

u(t k ;t 0 ,t f , θ ) = u(t k -;t 0 ,t f , θ ), k = 1, . . . , s.
Let x(t;t 0 ,t f , x 0 , θ ) be the solution of the initial value problem (IVP)

ẋ = f (t, x) + g(t, x)u(t;t 0 ,t f , θ ), t ∈ [t 0 ,t f ], x(t 0 ) = x 0 . (49)
For each ζ ∈ V this solution exists if the neighborhood V of the point ζ is sufficiently small. We obviously have

x(t; t 0 , t f , x 0 , θ ) = x(t), t ∈ ∆ , u(t; t 0 , t f , θ ) = u(t), t ∈ ∆ \ Θ .
Consider now the following finite dimensional optimization problem in the space IR 2 × IR n × IR s of the variables ζ = (t 0 ,t f , x 0 , θ ):

J (ζ ) := J(t 0 , x 0 ,t f , x(t f ;t 0 ,t f , x 0 , θ )) → min, F (ζ ) := F(t 0 , x 0 ,t f , x(t f ;t 0 ,t f , x 0 , θ )) ≤ 0, G (ζ ) := K(t 0 , x 0 ,t f , x(t f ;t 0 ,t f , x 0 , θ )) = 0. ( 50
) E T E ' t d d t ' E ' t d t 1 d t t 0 t 0 t 1 t 2 t 2 E t f t f t u(t;t 0 ,t f ,t 1 ,t 2 ) u 1 u 3 u 2
Fig. 2 Bang-bang control with two switches.

We call (50) the Induced Optimization Problem (IOP) or simply Induced Problem which represents an extension of the IOP introduced in [START_REF] Agrachev | Strong optimality for a bang-bang trajectory[END_REF]. The following assertion is almost obvious.

Theorem 15.

Let the process T be a Pontryagin local minimum for the basic control problem ( 22)- [START_REF] Maurer | Optimization methods for the verification of second-order sufficient conditions for bang-bang controls[END_REF]. Then the point ζ is a local minimum of the IOP (50), and hence it satisfies first and second-order necessary conditions for this problem.

Second-Order Optimality Conditions for Bang-Bang Controls in Terms of the Induced Optimization Problem

We shall clarify a relationship between the second order conditions for the Induced Optimization Problem (50) at the point ζ and those in the basic bang-bang control problem ( 22)-( 24) for the process T . It turns out that there is an one-to-one correspondence between Lagrange multipliers in these problems and an one-to-one correspondence between elements of the critical cones. Moreover, for corresponding Lagrange multipliers, the quadratic forms in these problems take equal values on the corresponding elements of the critical cones. This allows to express the necessary and sufficient quadratic optimality conditions for a bang-bang control, formulated in Theorems 13 and 14, in terms of the IOP (50). Thus we are able to establish the equivalence between our quadratic sufficient conditions and those due to Agrachev, Stefani, Zezza [START_REF] Agrachev | Strong optimality for a bang-bang trajectory[END_REF].

Let T = ( x(t), u(t) | t ∈ [ t 0 , t f ]
) be an admissible process in the basic problem with the properties assumed in Section 5.2 and let ζ = ( t 0 , t f , x 0 , θ ) be the corresponding admissible point in the IOP. The Lagrange function for the Induced Optimization Problem (50) is

L (µ, ζ ) = L (µ,t 0 ,t f , x 0 , θ ) = α 0 J (ζ ) + αF (ζ ) + β G (ζ ), (51)
where µ = (α 0 , α, β ), ζ = (t 0 ,t f , x 0 , θ ), θ = (t 1 , ...,t s ). We denote by K 0 the critical cone at the point ζ in the IOP. Thus, K 0 is the set of collections ζ = (t 0 , t f , x0 , θ ) such that

J ′ ( ζ ) ζ ≤ 0, F ′ i ( ζ ) ζ ≤ 0, i ∈ I, G ′ ( ζ ) ζ = 0, ( 52
)
where

I = {i | F i ( ζ ) = 0}
is the set of indices of the inequality constraints active at the point ζ . For µ ∈ M 0 the quadratic form, of the induced optimization problem, is equal to

L ζ ζ (µ, ζ ) ζ , ζ .
Let us formulate now second-order optimality conditions for the basic control problem in terms of the IOP.

Theorem 16 (Second-Order Necessary Conditions). If the process T affords a

Pontryagin minimum in the basic problem, then the following Condition A 0 holds: the set M 0 is nonempty and

max µ∈M 0 L ζ ζ (µ, ζ ) ζ , ζ ≥ 0 for all ζ ∈ K 0 .

Theorem 17 (Second-Order Sufficient Conditions). Let the following Condition B 0 be fulfilled for an admissible process T in the basic problem:

(a) u(t) is a strict bang-bang control with finitely many switching times t k , k = 1, . . . , s (hence, the set M 0 is nonempty and condition [START_REF] Osmolovskii | Quadratic optimality conditions for broken extremals in the general problem of calculus of variations[END_REF] holds for some µ ∈ M 0 ), (b)there exists µ ∈ M 0 such that D k (H µ ) > 0, k = 1, . . . , s, (c) max

µ∈M 0 L ζ ζ (µ, ζ ) ζ , ζ > 0 for all ζ ∈ K 0 \ {0}.
Then T is a strict strong minimum in the basic problem.

Theorem 17 is a generalization of sufficient optimality conditions for bang-bang controls obtained in Agrachev et al. [START_REF] Agrachev | Strong optimality for a bang-bang trajectory[END_REF]. Detailed proofs of Theorems 16 and 17 are given in [START_REF] Osmolovskii | Equivalence of second order optimality conditions for bang-bang control problems. Part 2: Proof, variational derivatives and representations[END_REF] and in our book [START_REF] Osmolovskii | Applications to Regular and Bang-Bang Control: Second-Order Necessary and Sufficient Optimality Conditions in Calculus of Variations and Optimal Control[END_REF].

Remark 2. Noble, Schättler [START_REF] Noble | Sufficient conditions for relative minima of broken extremals[END_REF] and Schättler, Ledzewicz [START_REF] Sch | Geometric Optimal Control: Theory, Methods and Examples[END_REF] develop sufficient optimality conditions using methods of geometric optimal control. There is some evidence that their sufficient conditions are closely related to the SSC in our work [START_REF] Osmolovskii | Equivalence of second order optimality conditions for bang-bang control problems. Part 2: Proof, variational derivatives and representations[END_REF][START_REF] Osmolovskii | Applications to Regular and Bang-Bang Control: Second-Order Necessary and Sufficient Optimality Conditions in Calculus of Variations and Optimal Control[END_REF]. However, a formal proof of the equivalence of both types of sufficient conditions has not yet been worked out.

Numerical Methods for Solving the Induced Optimization Problem

The arc-parametrization method developed in [START_REF] Kaya | Computational method for time-optimal switching control[END_REF], [START_REF] Maurer | Optimization methods for the verification of second-order sufficient conditions for bang-bang controls[END_REF] provides an efficient method for solving the IOP. To better explain this method, for simplicity let us consider the basic control problem with fixed initial time t 0 = 0 and fixed initial condition x 0 (0) = x 0 , and without inequality constraints F (ζ ) ≤ 0. For this problem, we slightly change the notation and replace the resulting optimization vector ζ = (t f ,t 1 , ...,t s ) by the vector z = (t 1 , ...,t s ,t s+1 ), t s+1 = t f . Instead of directly optimizing the switching times t k , k = 1, . . . , s, we determine the arc lengths (arc durations)

ξ k := t k -t k-1 , k = 1, . . . , s, s + 1, (53)
of bang-bang arcs. Hence, the optimization variable z = (t 1 , . . . ,t s ,t s+1 ) * is replaced by the optimization variable

ξ := (ξ 1 , . . . , ξ s , ξ s+1 ) * ∈ IR s+1 , ξ k := t k -t k-1 . (54)
The variables z and ξ are related by a linear transformation involving the regular (s + 1) × (s + 1)-matrix R :

ξ = Rz, z = R -1 ξ , R =       1 0 0 . . . 0 0 -1 1 0 . . . 0 0 0 -1 1 . . . 0 0 . . . . . . . . . . . . . . . . . . 0 0 0 . . . -1 1       . ( 55
)
In the arc-parametrization method, the time interval [t k-1 ,t k ] is mapped to the fixed interval

I k := k -1 s + 1 , k s + 1 , k = 1, . . . , s + 1, (56)
by the linear transformation

t = a k + b k τ, τ ∈ I k , (57)
where

a k = t k-1 -(k -1)ξ k , b k = (s + 1)ξ k . ( 58
) Identifying x(τ) ∼ = x(a k + b k τ) = x(t) (59)
in the relevant intervals, we obtain the ODE system

ẋ(τ) = (s + 1)ξ k ( f (a k + b k τ, x(τ)) + g(a k + b k τ, x(τ))u k ) for τ ∈ I k . ( 60
)
By concatenating the solutions in the intervals I k we get the continuous solution x(t) = x(t; ξ ) in the normalized interval [0, 1]. When expressed via the new opti-mization variable ξ , the Induced Optimization Problem (IOP) in ( 50) is equivalent to the following optimization problem ( IOP) with

t f = s+1 ∑ k=1 ξ k : Minimize J (ξ ) := J(t f ; x(1, ξ )) subject to G (ξ ) := K(t f ; x(1, ξ )) = 0. ( 61
)
The Lagrangian function is given by

L (µ, ξ ) = α 0 J (ξ ) + β G (ξ ), µ = (α 0 , β ). ( 62
)
Using the linear transformation (55), it can easily be seen that the SSCs for the Induced Optimization Problems (IOP) and ( IOP) are equivalent; cf. similar arguments in [START_REF] Maurer | Optimization methods for the verification of second-order sufficient conditions for bang-bang controls[END_REF]. To solve the ( IOP), we use a suitable adaptation of the control package NUDOCCCS in Büskens [START_REF] Üskens | Optimierungsmethoden und Sensitivitätsanalyse für optimale Steuerprozesse mit Steuer-und Zustands-Beschrankungen[END_REF], [START_REF] Maurer | SQP-methods for solving optimal control problems with control and state constraints: adjoint variables, sensitivity analysis and real-time control[END_REF]. Then we can take advantage of the fact that NUDOCCCS also provides the Jacobian of the terminal constraints and the Hessian of the Lagrangian which are needed in the check of the second order condition in Theorem 17.

In practice, we shall verify the positive definiteness condition (c) in Theorem 17 in a stronger form. We assume that the multiplier can be chosen as µ = (1, β ) and that the following regularity condition holds:

rank G ξ ( ξ ) = d(K).
Let N be the n ξ × (n ξd(K)) matrix, n ξ = n + s + 1 (where n = d(x)), with full column rank n ξd(K), whose columns span the kernel of G ξ ( ξ ). Then condition (c) in Theorem 17,

L ξ ξ ( ξ , β ) ξ , ξ > 0 ∀ ξ = 0, G ξ ( ξ ) ξ = 0, ( 63
)
is equivalent to the condition that the projected Hessian is positive definite [START_REF] Maurer | Sensitivity analysis and real-time optimization of parametric nonlinear programming problems[END_REF],

N * L ξ ξ ( ξ , β )N > 0 . ( 64
)

Numerical Example with Fixed Final Time: Optimal Control of the Chemotherapy of HIV

The treatment of patients infected with the human immunodeficiency virus (HIV) is still of great concern today (Kirschner et al. [START_REF] Kirschner | Optimal control of the chemotherapy of HIV[END_REF]. The problem of determining optimal chemotherapies has been treated in Kirschner, Lenhart, Serbin [START_REF] Kirschner | Optimal control of the chemotherapy of HIV[END_REF] in the framework of optimal control theory. The optimal control model is based on a simple dynamic model in Perelson et al. [START_REF] Perelson | The dynamics of HIV infection of CD4 + T cells[END_REF] which simulates the interaction of the immune system with HIV. The treatment starts at t 0 = 0 and terminates at the fixed final time t f = 500 (days). Thus, the control process is considered in the interval [0,t f ]. The dynamics of the populations are (omitting the time argument):

dT /dt = s 1+V -µ T T + rT 1 -T +T * +T * * T max -k 1 V T , T (0) = T 0 , dT * /dt = k 1 V T -µ T T * -k 2 T * , T * (0) = T * 0 , dT * * /dt = k 2 T * -µ b T * * , T * * (0) = T * * 0 dV /dt = (1 -u) Nµ b T * * -k 1 V T -µ V V , V (0) = V 0 . (65)
The control constraint is given by

0 ≤ u(t) ≤ 1 ∀ t ∈ [0,t f ], (66)
where u(t) = 1 represents maximal chemotherapy, while u(t) = 0 means that no chemotherapy is administered. Note that Kirschner et al. [START_REF] Kirschner | Optimal control of the chemotherapy of HIV[END_REF] consider the control variable v = 1u. It is convenient to write the ODE (65) as the control affine system (24

), ẋ = f (x) + g(x)u, x(0) = x 0 , x = (T, T * , T * * ,V ) ∈ IR 4 , (67)
with obvious definitions of the vector functions f (x) and g(x). As in [START_REF] Kirschner | Optimal control of the chemotherapy of HIV[END_REF] we consider two sets of initial conditions which depend on the time at which the treatment starts after the infection. The following initial conditions are interpolated from [START_REF] Kirschner | Optimal control of the chemotherapy of HIV[END_REF] and have already been used in the [START_REF] Hannemann | [END_REF]. Initial conditions after 800 days:

T 0 = 982.8, T * 0 = 0.05155, T * * 0 = 0.0006175 , V 0 = 0.07306. ( 68
)
Initial conditions after 1000 days:

T 0 = 904.1, T * 0 = 0.3447, T * * 0 = 0.004167, V 0 = 0.4939. (69)
The parameter and constants are taken from [START_REF] Kirschner | Optimal control of the chemotherapy of HIV[END_REF] and are listed in Table 1. Kirschner et al. [START_REF] Kirschner | Optimal control of the chemotherapy of HIV[END_REF] consider the following objective of L 2 -type which is quadratic in the control variable:

Parameters and constants Values µ T : death rate of uninfected CD4 + T cell population 0.02 d -1 µ T * : death rate of latently infected CD4 + T cell population 0.02 d -1 µ b : death rate of actively infected CD4 + T cell population 0.24 d -1 µ V : death rate of free virus 2.4 d -1 k 1 : rate CD4 + T cells becomes infected by free virus 2.4 × 10 -5 mm 3 d -1 k 2 : rate T * cells convert to actively infected 3 × 10 -3 mm 3 d -1 r : rate of growth for the CD4 + T cell population 0.03 d -1 N : number of free virus produced by T * * cells 1200 T max : maximum CD4 + T cell population level 1.5 × 10 3 mm -3 s : source term for uninfected CD4 + T cells, 10 d -1 mm -3 where s is the parameter in the source term s/(1 +V )

Table 1 Parameters and constants

Minimize J(x, u) =

t f 0 (-T (t) + B u(t) 2 ) dt (B = 50). ( 70
)
Recall that the state variable is defined as x := (T, T * , T * * ,V ) ∈ IR 4 . The optimal control that minimizes (70) subject to the constraints (65)-( 69) is a continuous function, since the associated Hamiltonian H(x, λ , u) has a unique minimum with respect to u and the strict Legendre-Clebsch condition H uu = 2B > 0 holds. For the two sets of initial conditions (68) and (69), the numerical discretization and NLP approach using AMPL [START_REF] Fourer | AMPL: A Modeling Language for Mathematical Programming[END_REF] and IPOPT [START_REF] Biegler | On the implementation of an interior-point filter linesearch algorithm for large-scale nonlinear programming[END_REF] yields the optimal controls shown in Figure 7 which were also obtained in Hannemann [START_REF] Hannemann | [END_REF].

Fig. 3 Optimal control for the L 2 functional (70). (left) begin of treatment after 800 days: initial conditions (68). (right) begin of treatment after 1000 days: initial conditions (69).

Hannemann [START_REF] Hannemann | [END_REF] showed that second-order sufficient conditions (SSC) are satisfied for the controls displayed in Figure 3, since the associated matrix Riccati equation has a bounded solution. Note that Riccati equations are discussed in [START_REF] Malanowski | Sensitivity analysis for parametric control problems with control-state constraints[END_REF][START_REF] Maurer | Second order sufficient conditions for optimal control problems with mixed control-state constraints[END_REF] and in our book [START_REF] Osmolovskii | Applications to Regular and Bang-Bang Control: Second-Order Necessary and Sufficient Optimality Conditions in Calculus of Variations and Optimal Control[END_REF], Chapter 4.

Instead of the L 2 functional (70) we consider now the functional of L 1 -type:

Minimize J 1 (x, u) = t f 0 (-T (t) + B u(t)) dt (B = 50). (71)
The Hamiltonian or Pontryagin function for this control problem is given by

H(x, λ , u) = -T + Bu + λ ( f (x) + g(x)u) , (72)
where λ = (λ T , λ T * , λ T * * , λ V ) ∈ IR 4 denotes the adjoint variable. The adjoint equa- tion and transversality condition are given by λ = -H x (x, λ , u), λ (t f ) = (0, 0, 0, 0), since the terminal state x(t f ) is free and the objective (71) does not contain a Mayer term. We do not write out the adjoint equation λ = -H x (x, λ , u) explicitly, since this equation is not needed in the sequel. The adjoint variables can be computed from the Lagrange multipliers of the associated Induced Optimization Problem (IOP). The switching function is given by

σ (x, λ ) = H u (x, λ , u) = B -λ V Nµ b T * * , σ (t) = σ (x(t), λ (t)). (73)
The minimization of the Hamiltonian with respect to u yields the switching condition

u(t) = 1, if σ (t) < 0 0, if σ (t) > 0 . ( 74
)
The control has a singular arc in an interval

[t 1 ,t 2 ] ⊂ [0, T ], if σ (t) = 0 holds on [t 1 ,t 2 ]
. However, we do not discuss singular controls further because for the data in Table 1 we never found singular arcs. Indeed, the optimal control for the L 1functional (71) is the following bang-bang control with only one switch at t 1 :

u(t) = 1 for 0 ≤ t < t 1 0 for t 1 ≤ t ≤ t f ( 75
)
The terminal arc u(t) = 0 results from the terminal value σ (t f ) = B > 0 of the switching function and the minimum condition (74). Hence, the IOP has only the scalar optimization variable t 1 and thus the objective (71) reduces to a function J 1 (t 1 ) = J 1 (x, u). The arc-parametrization method [START_REF] Maurer | Optimization methods for the verification of second-order sufficient conditions for bang-bang controls[END_REF][START_REF] Osmolovskii | Applications to Regular and Bang-Bang Control: Second-Order Necessary and Sufficient Optimality Conditions in Calculus of Variations and Optimal Control[END_REF] and the code NUDOC-CCS [START_REF] Üskens | Optimierungsmethoden und Sensitivitätsanalyse für optimale Steuerprozesse mit Steuer-und Zustands-Beschrankungen[END_REF] yield the following numerical results, where state variables are listed with 8 digits and adjoint variables with 6 digits. The state and control variables and the switching function are displayed in Figure 4. To verify that the second-order sufficient conditions (SSC) are satisfied for the computed extremal solution, we have to check the conditions of Theorem 17. The strict bang-bang property is satisfied, since we infer from Figure 4 (bottom, right) that the switching function satisfies

J 1 = -489810.49, t 1 = 161.69571, T (t f ) = 983.49256, T * (t f ) = 0.04934668, T * * (t f ) = 0.00059104969, V (t f ) = 0.069932999, λ T (0) = -0.125173, λ T * (0) = -1.51988, λ T * * (0) = -2.94704, λ V (0) = -0.449700. (76)
σ (t) < 0 for 0 ≤ t < t 1 , σ (t 1 ) > 0, σ (t) > 0 for t 1 < t ≤ t f = 500. ( 77
)
To verify the positive definiteness in condition (63), we note that the Hessian is simply the second derivative of the objective J 1 (t 1 ) evaluated at the optimal switching time t 1 = 161.695711 for which we find

d 2 J 1 dt 2 1 = 1.5469 > 0.
Hence, the extremal solution (76 ) displayed in Figure 4 provides a strict strong minimum. Now we try to improve the optimal terminal value T (t f ) = 983.493 of the uninfected CDC4 + T cells. For that purpose we prescribe a higher terminal value and minimize the functional J 1 (x, u) subject to the boundary condition The arc-parametrization method [START_REF] Maurer | Optimization methods for the verification of second-order sufficient conditions for bang-bang controls[END_REF][START_REF] Osmolovskii | Applications to Regular and Bang-Bang Control: Second-Order Necessary and Sufficient Optimality Conditions in Calculus of Variations and Optimal Control[END_REF] and the control package NUDOCCCS [START_REF] Üskens | Optimierungsmethoden und Sensitivitätsanalyse für optimale Steuerprozesse mit Steuer-und Zustands-Beschrankungen[END_REF] furnish the results

T (t f ) = 995 . (78)
J 1 = -489044.529, t 1 = 198.566451, T (t f ) = 995.0, T * (t f ) = 0.014576433, T * * (t f ) = 0.00017436211, V (t f ) = 0.020625442, λ T (0) = -33.7027, λ T (t f ) = -211.377, λ T * (0) = 28078.4 λ T * * (0) = 2.76312,
λ V (0) = 405.843. (79) Figure 5 displays the state and control variables and the switching function. Figure 5 (bottom, right) shows that the strict bang-bang property (77) is satisfies. Condition (c) in Theorem 17 holds because the critical cone K 0 = {0} contains of the zero element. Therefore, the extremal displayed in Figure 5 provides a strict strong minimum.

Finally, we study the optimal solution for the initial values (69), when the treatment starts after 1000 days and, again, the boundary condition T (t f ) = 995 is prescribed. The arc-parametrization method in [START_REF] Maurer | Optimization methods for the verification of second-order sufficient conditions for bang-bang controls[END_REF][START_REF] Osmolovskii | Applications to Regular and Bang-Bang Control: Second-Order Necessary and Sufficient Optimality Conditions in Calculus of Variations and Optimal Control[END_REF] and the control package NUDOCCCS yield the results

J 1 = -483480.87, t 1 = 254.5443, T (t f ) = 995.0, T * (t f ) = 0.014576939, T * * (t f ) = 0.00017436817, V (t f ) = 0.020626158, λ T (0) = -35.

Numerical Example with Free Final Time: Time-Optimal Control of Two-Link Robots

In this section, we review the results in our book [START_REF] Osmolovskii | Applications to Regular and Bang-Bang Control: Second-Order Necessary and Sufficient Optimality Conditions in Calculus of Variations and Optimal Control[END_REF] on the optimal control of twolink robots which has been addressed in various articles; cf., e.g. [START_REF] Chernousko | Time-optimal control for robotic manipulators[END_REF][START_REF] Geering | Time-optimal motions of robots in assembly tasks[END_REF][START_REF] Öllmann | Numerische Berechnung zeitoptimaler Trajektorien für zweigliedrige Roboterarme[END_REF][START_REF] Oberle | Numerical computation of singular control functions for a two-link robot arm[END_REF]. In these papers, optimal control policies are determined solely on the basis of first order necessary conditions, since sufficient conditions were not available. In this section we show that SSC hold for both types of robots considered in [START_REF] Chernousko | Time-optimal control for robotic manipulators[END_REF][START_REF] Öllmann | Numerische Berechnung zeitoptimaler Trajektorien für zweigliedrige Roboterarme[END_REF][START_REF] Oberle | Numerical computation of singular control functions for a two-link robot arm[END_REF].

First, we study the robot model considered in Chernousko et al. [START_REF] Chernousko | Time-optimal control for robotic manipulators[END_REF]. Göllmann [START_REF] Öllmann | Numerische Berechnung zeitoptimaler Trajektorien für zweigliedrige Roboterarme[END_REF] has shown that the optimal control candidate presented in [START_REF] Chernousko | Time-optimal control for robotic manipulators[END_REF] is not optimal, since the sign conditions of the switching functions do not comply with the Minimum Principle. Figure 7 displays the two-link robot schematically. The state variables 7 Schematical representation of a two-link robot.

E T O x 1 x 2 ¨¨¨¨¨¨¨¨¨¨¨¨¨¨¨¨¨¡¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ r r r r h h h h h h h h h h h q 1 q 2 Q P C Fig.
are the angles q 1 and q 2 . The parameters I 1 and I 2 are the moments of inertia of the upper arm OQ and the lower arm QP with respect to the points O and Q, resp.. Further, let m 2 be the mass of the lower arm, L 1 = |OQ| the length of the upper arm, and L 1 = |QC| the distance between the second link Q and the center of gravity C of the lower arm. With the abbreviations

A = I 1 + m 2 L 2 1 + I 2 + 2m 2 L 1 L cos q 2 , B = I 2 + m 2 L 1 L cos q 2 , R 1 = u 1 + m 2 L 1 L(2 q1 + q2 ) q2 sin q 2 , R 2 = u 2 -m 2 L 1 L q2 1 sin q 2 , D = I 2 , ∆ = AD -B 2 , (81)
the dynamics of the two-link robot can be described by the ODE system

q1 = ω 1 , ω1 = 1 ∆ (DR 1 -BR 2 ) , q2 = ω 2 , ω2 = 1 ∆ (AR 2 -BR 1 ) , (82)
where ω 1 and ω 2 are the angular velocities. The torques u 1 and u 2 in the two links represent the two control variables. Therefore, the state variable and control variable are given by x = (q 1 , q 2 , ω 1 , ω 2 ) ∈ IR 4 , u = (u 1 , u 2 ) ∈ IR 2 .

The control problem consists in steering the robot from a given initial position to a terminal position in minimal final time t f ,

q 1 (0) = 0 , q 2 (0) = 0 , ω 1 (0) = 0 , ω 2 (0) = 0 , q 1 (t f ) = -0.44 , q 2 (t f ) = 1.83 , ω 1 (t f ) = 0 , ω 2 (t f ) = 0 . ( 83
)
The control components are bounded by

|u 1 (t)| ≤ 2 , |u 2 (t)| ≤ 1 , t ∈ [0,t f ] . (84)
The Pontryagin function (Hamiltonian) is

H = λ 1 ω 1 + λ 2 ω 2 + λ 3 ∆ (DR 1 (u 1 ) -BR 2 (u 2 )) + λ 4 ∆ (AR 2 (u 2 ) -BR 1 (u 1 )) .
(85) The adjoint equations are rather complicated and are not given here explicitly. The switching functions are

σ 1 (x, λ ) = H u 1 = λ 3 ∆ D - λ 4 ∆ B , σ 2 (x, λ ) = H u 2 = λ 4 ∆ A - λ 3 ∆ B . (86)
For the parameter values

L 1 = 1 , L = 0.5 , m 2 = 10 , I 1 = I 2 = 10 3 ,
Göllmann [START_REF] Öllmann | Numerische Berechnung zeitoptimaler Trajektorien für zweigliedrige Roboterarme[END_REF] has found the following control structure with four bang-bang arcs,

u(t) = (u 1 (t), u 2 (t)) =        (-2, 1) , 0 ≤ t < t 1 (2, 1) , t 1 ≤ t < t 2 (2, -1) , t 2 ≤ t < t 3 (-2, -1) , t 3 ≤ t ≤ t f        , 0 < t 1 < t 2 < t 3 < t f . ( 87
)
This control structure differs substantially from the one in Chernousko et al. [START_REF] Chernousko | Time-optimal control for robotic manipulators[END_REF] which violates the switching conditions. Obviously, the bang-bang control (87) satisfies the assumption that only one control components switches at a time. Since the initial point (q 1 (0), q 2 (0), ω 1 (0), ω 2 (0)) is specified, the optimization variable in the IOP (61) is

ξ = (ξ 1 , ξ 2 , ξ 3 , ξ 4 ), ξ 1 = t 1 , ξ 2 = t 2 -t 1 , ξ 3 = t 3 -t 2 , ξ 4 = t f -t 3 .
Using the code NUDOCCCS we compute the following arc durations and switching times t 1 = 0.7677893 , ξ 2 = 0.3358820 , t 2 = 1.1036713 , ξ 3 = 1.2626739 , t 3 = 2.3663452 , ξ 4 = 0.8307667 ,

t f = 3.1971119 . (88)
Numerical values for the adjoint functions are also provided by the code NUDOC-CCS, e.g., the initial values λ 1 (0) = -1.56972 , λ 2 (0) = -0.917955 , λ 3 (0) = -2.90537 , λ 4 (0) = -1.45440 . Bottom row: (left) angle q 1 and velocity ω 1 , (right) angle q 2 and velocity ω 2 .

Figure 8 shows that the switching functions σ 1 , , σ 2 comply with the minimum con- dition and that the strict bang-bang property [START_REF] Osmolovskii | Quadratic optimality conditions for broken extremals in the general problem of calculus of variations[END_REF] and the inequalities D k (H) > 0, k = 1, 2, 3 are satisfied:

σ 1 (t) = 0 for t = t 1 ,t 3 , σ 2 (t) = 0 for t = t 2 , σ1 (t 1 ) < 0 , σ1 (t 3 ) > 0 , σ2 (t 2 ) > 0 . This square matrix has full rank in view of det Gξ ( ξ ) = 0.0766524 = 0 , which means that the positive definiteness condition (63) trivially holds. Thus we have verified first-order sufficient conditions showing that the extremal solution given by ( 87)-(89) provides a strict strong minimum.

In the model treated above, some parameters like the mass of the upper arm and the mass of a load at the end of the lower arm appear implicitly in the system equations. The mass m 1 of the upper arm is included in the moment of inertia I 2 and the mass M of a load in the point P can be added to the mass m 2 , where the point C and therefore the length L have to be adjusted. The length L 2 of the lower arm is incorporated in the parameter L.

The second robot model that we are going to discuss is taken from Geering et al. [START_REF] Geering | Time-optimal motions of robots in assembly tasks[END_REF] and Oberle [START_REF] Oberle | Numerical computation of singular control functions for a two-link robot arm[END_REF]. Here, every physical parameter enters the system equation explicitly. The dynamic system is as follows:

q1 = ω 1 , ω1 = 1 ∆ (AI 22 -BI 12 cos q 2 ) , q2 = ω 2 -ω 1 , ω2 = 1 ∆ (BI 11 -AI 12 cos q 2 ) , (90)
where we have used the abbreviations

A = I 12 ω 2 2 sin q 2 + u 1 -u 2 , B = -I 12 ω 2 1 sin q 2 + u 2 , ∆ = I 11 I 22 -I 2 12 cos 2 q 2 , I 11 = I 1 + (m 2 + M)L 2 1 , I 12 = m 2 LL 1 + ML 1 L 2 , I 22 = I 2 + I 3 + ML 2 2 .
(91) I 3 denotes the moment of inertia of the load with respect to the point P and ω 2 is now the angular velocity of the angle q 1 + q 2 . For simplicity, we set I 3 = 0. Again, the torques u 1 and u 2 in the two links are used as control variables by which the robot is steered from a given initial position to a non-fixed end position in minimal final time t f ,

q 1 (0) = 0 , (x 1 (t f ) -x 1 (0)) 2 + (x 2 (t f ) -x 2 (0)) 2 = r , q 2 (0) = 0 , q 2 (t f ) = 0 , ω 1 (0) = 0 , ω 1 (t f ) = 0 , ω 2 (0) = 0 , ω 2 (t f ) = 0 , (92)
where (x 1 (t), x 2 (t)) are the Cartesian coordinates of the point P,

x 1 (t) = L 1 cos q 1 (t) + L 2 cos(q 1 (t) + q 2 (t)) , x 2 (t) = L 1 sin q 1 (t) + L 2 sin(q 1 (t) + q 2 (t)) .

The initial point

(x 1 (0), x 2 (0)) = (2, 0) is fixed. Both control components are bounded, |u 1 (t)| ≤ 1 , |u 2 (t)| ≤ 1 , t ∈ [0,t f ] . (94)
The Hamilton-Pontryagin function is given by

H = λ 1 ω 1 + λ 2 (ω 2 -ω 1 ) + λ 3 ∆ (A(u 1 , u 2 )I 22 -B(u 2 )I 12 cos q 2 ) + λ 4 ∆ (B(u 2 )I 11 -A(u 1 , u 2 )I 12 cos q 2 ) . (95)
The switching functions are computed as

σ 1 (x, λ ) = H u 1 = 1 ∆ (λ 3 I 22 -λ 4 I 12 cos q 2 ) , σ 2 (x, λ ) = H u 2 = 1
∆ (λ 3 (-I 22 -I 12 cosq 2 ) + λ 4 (I 11 + I 12 cos q 2 )) .

(96)

For the parameters values

L 1 = L 2 = 1 , L = 0.5 , m 1 = m 2 = M = 1 , I 1 = I 2 = 1 3 I 3 = 0 , r = 3,
we will show that the optimal control has the following structure with five bangbang arcs with 0 = t 0 < t 1 < t 2 < t 3 < t 4 < t 5 = t f :

u(t) = (u 1 (t), u 2 (t)) =            (-1, 1) for 0 ≤ t < t 1 (-1, -1) for t 1 ≤ t < t 2 (1, -1) for t 2 ≤ t < t 3 (1, 1) for t 3 ≤ t < t 4 (-1, 1) for t 4 ≤ t ≤ t f            . ( 97
)
Since the initial point (q 1 (0), q 2 (0), ω 1 (0), ω 2 (0)) is specified, the optimization vari- able in the optimization problem (50), resp., (61) is

z = (ξ 1 , ξ 2 , ξ 3 , ξ 4 , ξ 5 ), ξ k = t k -t k-1 , k = 1, . . . , 5.
The code NUDOCCCS yields the arc durations and switching times

t 1 = 0.546174 , ξ 2 = 1.21351 , t 2 = 1.75968 , ξ 3 = 1.03867 , t 3 = 2.79835 , ξ 4 = 0.906039 , t 4 = 3.70439 , ξ 5 = 0.185023 , t f = 3.889409 , (98)
as well as the initial values of the adjoint variables, λ 1 (0) = 0.184172 , λ 2 (0) = -0.011125 , λ 3 (0) = 1.482636 , λ 4 (0) = 0.997367 .

The strict bang-bang property [START_REF] Osmolovskii | Quadratic optimality conditions for broken extremals in the general problem of calculus of variations[END_REF] and the inequalities

D k (H) > 0, k = 1, 2, 3, hold in view of σ 1 (t) = 0 for t = t 2 ,t 4 , σ1 (t 2 ) < 0 , σ1 (t 4 ) > 0 , σ 2 (t) = 0 for t = t 1 ,t 3 , σ2 (t 1 ) > 0 , σ2 (t 3 ) < 0.
For the terminal conditions in (92), the Jacobian in the optimization problem is computed as the (4 × 5)-matrix Fig. This yields the projected Hessian (64) as the positive number

N * L ξ ξ ( ξ , β )N = 0.326929 .
Hence, all conditions in Theorem 17 are satisfied and thus the extremal (97)-(99) yields a strict strong minimum.

It is interesting to note that there exists a second local minimum with the same terminal time t f = 3.88941. Though the control has also five bang-bang arcs, the control structure is substantially different from that in (97),

u(t) = (u 1 (t), u 2 (t)) =            (1, -1) , 0 ≤ t < t 1 (-1, -1) , t 1 ≤ t < t 2 (-1, 1) , t 2 ≤ t < t 3 (1, 1) , t 3 ≤ t < t 4 (1, -1) , t 4 ≤ t ≤ t f            , (100)
where 0 < t 1 < t 2 < t 3 < t 4 < t 5 = t f . NUDOCCCS determines the switching times

t 1 = 0.1850163 , t 2 = 1.091075 , t 3 = 2.129721 , t 4 = 3.343237 , t f = 3.889409 , (101)
for which the strict bang-bang property [START_REF] Osmolovskii | Quadratic optimality conditions for broken extremals in the general problem of calculus of variations[END_REF] holds and D k (H) > 0 for k = 1, 2, 3, 4. Moreover, computations show that rank ( G ξ ( ξ ) ) = 4 and that the projected Hessian of the Lagrangian (64) is the positive number

N * L ξ ξ ( ξ , β )N = 0.326929 .
It is remarkable that this value is identical with the value of the projected Hessian for the first local minimum. Therefore, also for the second solution we have verified that all conditions in Theorem 17 hold, and thus the extremal (100), ( 101) is a strict strong minimum. The phenomenon of multiple local solutions all with the same minimal time t f has also been observed by Betts [START_REF] Betts | Practical Methods for Optimal Control and Estimation Using Nonlinear Programming, Second Editon[END_REF], Example 6.8 (Reorientation of a rigid body).

Optimal control problems with mixed control-state constraints and control appearing linearly

To the best of our knowledge, second order sufficient optimality conditions (SSC) for optimal control problems with mixed control-state constraints have only been studied for the class of regular controls, where the strict Legendre-Clebsch condition holds. Such control problems have not yet been considered, when the control variable appears linearly in the system dynamics and in the mixed control-state constraint. For a two-sided control-state constraint we will show that the constraining function itself can be taken as a new control variable, whereby the original control problem is transformed into a classical control problem with an affine control variable subject to simple control bounds. Hence, optimal controls for the transformed control problem are concatenations of bang-bang and singular arcs. The material in this section is based on our paper [START_REF] Maurer | Second-order conditions for optimal control problems with mixed control-state constraints and control appearing linearly[END_REF].

Statement of the Problem and Transformed Control Problem

For simplicity, we consider an optimal control problem with fixed initial time t 0 = 0, fixed initial conditions and terminal equality constraints, and with a scalar control. Let x ∈ IR n denote the state variable and u ∈ IR be the control variable. The terminal time t f > 0 is either fixed or free. The dynamic equation and boundary conditions are

ẋ = f (t, x) + g(t, x)u, x(0) = x 0 , K(x(t f )) = 0. ( 102
)
We consider a two-sided mixed control-state constraint which is affine in the control variable:

α ≤ a(x(t)) + b(x(t)) u(t) ≤ β for a.e. t ∈ [0,t f ] . (103)
The optimal control problem consists in finding a control u ∈ L ∞ ([0,t f ], IR) that minimizes the objective functional in Mayer form

J (x, u) = J(x(t f )). (104)
The functions f , g :

IR n → IR n , a, b : IR n → IR, J : IR n → IR and K : IR n → IR d(K) (0 ≤ d(K) ≤ n
) are assumed to be twice continuously differentiable. We remind the reader that a Bolza functional of the form

J (x, u) = J(x(t f )) + t f 0 ( f 0 (t, x) + g 0 (t, x)u)dt. ( 105
)
can be reduced to Mayer form by introducing the additional state variable y that solves the initial value problem ẏ = f 0 (t, x) + g 0 (t, x)u, y(0) = 0 and minimizing thr functional J(x(t f )) + y(t f ).

The following regularity assumption will be assumed to hold for feasible trajectories:

b(t, x(t)) = 0 for t ∈ [0,t f ] . (106)
This assumption allows us to introduce a new control variable v that is related to the control variable u as follows:

v := a(x) + b(x) u, i.e., u = (v -a(x))/b(x). ( 107
)
The transformed optimal control problem consists in minimizing the objective (104) subject to the transformed dynamics

ẋ = f (t, x) + ḡ(t, x)v, x(0) = x 0 , K(x(t f )) = 0, (108)
where the transformed functions f , ḡ are defined by f (t, x) = f (t, x)g(t, x)a(x)/b(x), ḡ(t, x) = g(t, x)/b(x).

(109)

The mixed control-state constraint (103) then is equivalent to the simple control constraint α ≤ v(t) ≤ β .

Thus, we can apply the second-order conditions developed in sections 5 and 6 to the transformed problem

Numerical Example: Optimal Control of the Rayleigh Equation

The Rayleigh equation describes oscillations of the electric current, resp., voltage in an electric circuit. The optimal control of the Rayleigh equation for a controlquadratic objective has been studied in Maurer, Augustin [START_REF] Maurer | Sensitivity analysis and real-time control of parametric optimal control problems using boundary value methods[END_REF], Osmolovskii, Maurer [START_REF] Osmolovskii | Applications to Regular and Bang-Bang Control: Second-Order Necessary and Sufficient Optimality Conditions in Calculus of Variations and Optimal Control[END_REF] and Chen, Gerdts [START_REF] Chen | Smoothing techniques of nonsmooth Newton methods for controlstate constrained optimal control problems[END_REF], where both simple control bounds and a mixed controlstate constraint were investigated. Let x 1 denote the electric current and x 2 the voltage. The control u represents the voltage at the generator which steers the following dynamic equations:

ẋ1 = x 2 ,
x 1 (0) = -5, ẋ2 = -x 1 + x 2 (1.4 -0.14x 2 2 ) + u, x 2 (0) = -5.

(111)

We consider the mixed control-state constraint

α ≤ u + x 1 ≤ β (α = -5, β = 0). (112)
Various other bounds α and β have been studied in Maurer, Omolovskii [START_REF] Maurer | Second-order conditions for optimal control problems with mixed control-state constraints and control appearing linearly[END_REF]. The mixed constraint is a slight modification of the one considered in [START_REF] Maurer | Sensitivity analysis and real-time control of parametric optimal control problems using boundary value methods[END_REF][START_REF] Chen | Smoothing techniques of nonsmooth Newton methods for controlstate constrained optimal control problems[END_REF]. The objective is to minimize the quadratic functional

J (x, u) = t f 0 (x 1 (t) 2 + x 2 (t) 2 )dt . (113)
First, we consider this control problem with fixed terminal time t f = 4.5. Later, we shall prescribe the terminal condition

x 1 (t f ) = 0 (114)
and solve the control problem with free terminal time t f . According to (107), the new control variable v is given by

v = u + x 1 , u = v -x 1 . (115)
The Pontryagin function (Hamiltonian) with respect to the control v becomes

H(x, λ , v) = x 2 1 + x 2 2 + λ 1 x 2 + λ 2 (-x 1 + x 2 (1.4 -0.14x 2 2 ) + v -x 1 ). (116)
The adjoint equations are λ1 = -H x 1 = -2x 1 + 2λ 2 , λ2 = -H x 2 = -2x 2 -λ 1 + λ 2 (0.42x 2 2 -1.4).

(117)

For the first control problem with free endpoint x(t f ) and fixed final time t f = 4.5, we get the transversality condition λ 1 (t f ) = 0, λ 2 (t f ) = 0, (118) while the second control problem with terminal constraint x 1 (t f ) = 0 and free terminal time t f gives the transversality conditions

H(t f ) = 0, λ 2 (t f ) = 0. ( 119
)
The switching function σ = H v = λ 2 determines the optimal control according to

v(t) =    β if λ 2 (t) < 0, α if λ 2 (t) > 0, singular if λ 2 (t) = 0 ∀ t ∈ I s ⊂ [0,t f ].
(120)

We do not discuss singular controls further, because for the chosen bounds α = -5 and β = 0 in the mixed constraint (112) we obtain bang-bang controls. Singular controls for smaller values of α are discussed in [START_REF] Maurer | Second-order conditions for optimal control problems with mixed control-state constraints and control appearing linearly[END_REF].

In the first control problem with free terminal state x(t f ) and fixed terminal time t f , we obtain a bang-bang control with three bang-bang arcs:

v(t) =    0 if 0 ≤ t ≤ t 1 -5 if t 1 < t ≤ t 2 0 if t 2 < t ≤ t f    . (121)
The code NUDOCCCS [START_REF] Üskens | Optimierungsmethoden und Sensitivitätsanalyse für optimale Steuerprozesse mit Steuer-und Zustands-Beschrankungen[END_REF] yields the following results:

J (x, u) = 62.165171, t 1 = 0.77996717, t 2 = 2.6835574, x 1 (t f ) = -0.40342897, x 2 (t f ) = -1.4332277, λ 1 (0) = -13.364385, λ 2 (0) = -5.591549.

The corresponding extremal solution is shown in Figure 10. Next, we compute the Hessian of the Lagrangian for the IOP: Lzz = 573.2366 458.8536 458.8536 377.3994 .

Obviously, this matrix is positive-definite. Moreover, Figure 10 (top row, right) shows that the switching function σ (t) = λ 2 (t) satisfies the strict bang-bang prop- erty (37); cf. also Remark 5.1:

λ 2 (t) < 0 ∀ 0 ≤ t < t 1 , λ 2 (t 1 ) = 0, λ2 (t 1 ) > 0, λ 2 (t) > 0 ∀ t 1 < t < t 2 , λ 2 (t 2 ) = 0, λ2 (t 2 ) < 0, λ 2 (t) < 0 ∀ t 2 < t < t f , λ 2 (t f ) = 0, λ2 (t f ) > 0.

Hence, the extremal shown in Figure 10 satisfies the SSC in Theorem 17 and thus is a strict strong minimum. Now we study the solution, when the terminal condition x 1 (t f ) = 0 is imposed and the terminal time t f is free. In this case we obtain a bang-bang control with only one switch:

v(t) = 0 if 0 ≤ t ≤ t 1 -5 if t 1 < t ≤ t f . (122)
The code NUDOCCCS yields the following results:

J(x, u) = 60.7269676, t 1 = 0.834309981, t f = 2.3646877, x 1 (t f ) = 0.0, x 2 (t f ) = 1.60020237, λ 1 (0) = -13.4868419, λ 1 (t f ) = -1.60020068, λ 2 (0) = -5.72868282, λ 2 (t f ) = 0.0.

The corresponding extremal solution is shown in Figure 11. To demonstrate that this extremal is a local optimum, we compute the Hessian of the Lagrangian for the IOP: Lzz = 573.2366 458.8536 458.8536 377.3994 .

Obviously, this matrix is positive-definite. Moreover, Figure 9.2 (top row, right) shows that the strict bang-bang property is fulfilled; cf. also Remark 5.1:

λ 2 (t) < 0 ∀ 0 ≤ t < t 1 , λ 2 (t 1 ) = 0, λ2 (t 1 ) > 0, λ 2 (t) > 0 ∀ t 1 < t < t f , λ 2 (t f ) = 0, λ2 (t f ) < 0.

Hence, the extremal shown in Figure 11 satisfies the SSC in Theorem 17 and thus provides a strict strong minimum.

Conclusion

We presented no-gap necessary and sufficient second-order optimality conditions for extremals with discontinuous controls in the simplest problem of the Calculus of Variations and the general optimal control problem with regular mixed constraint g(t, x, u) = 0 on a variable time interval [t 0 ,t f ]. We formulated similar conditions for bang-bang controls in an optimal control problem with a Mayer functional, where the dynamical system is affine in control variable and the control constraint is given by a convex polyhedron. Bang-bang controls induce an optimization problem with respect to the switching times of the control, the so-called Induced Optimization Problem IOP. We showed that the classical second-order sufficient condition for the IOP together with the strict bang-bang property of the switching function ensure second-order sufficient conditions (IOP) for the bang-bang control problem. The verification of SSC for bang-bang controls were illustrated on two numerical examples. First, we studied extremals in the optimal control of the chemotherapy of HIV. Then, following [START_REF] Osmolovskii | Applications to Regular and Bang-Bang Control: Second-Order Necessary and Sufficient Optimality Conditions in Calculus of Variations and Optimal Control[END_REF], we investigated extremals in time-optimal control problems of two-link robots. We also discussed optimal control problems with running mixed control-state constraints and control appearing linearly. Taking the mixed constraint as a new control variable we converted such problems to bang-bang control problems. As an example, we studied extremals in the optimal control problem for the Rayleigh equation.

The results on SSC naturally lend themselves to sensitivity results for the IOP and the underlying bang-bang control problem using the well-known sensitivity results for finite-dimensional optimization problems developed by Fiacco [START_REF] Fiacco | Introduction to Sensitivity and Stability Analysis in Nonlinear Programming[END_REF]. Sensitivity results for bang-bang controls may be found in Kim, Maurer [START_REF] Kim | Sensitivity analysis of optimal control problems with bangbang controls[END_REF] and Maurer, Vossen [START_REF] Maurer | Sufficient conditions and sensitivity analysis for bang-bang control problems with state constraints[END_REF]. Sensitivity results also allow to develop real-time control techniques as indicated already in Büskens et al. [START_REF] Üskens | Real-time solutions of bang-bang and singular optimal control problems[END_REF]. These issues will be the topic of a future paper. Related sensitivity results may be obtained for bang-singular controls as suggested in Vossen [START_REF] Vossen | Numerische Lösungsmethoden, hinreichende Optimalitätsbedingungen und Sensitivitätsanalyse für optimale bang-bang und singuläre Steuerungen[END_REF][START_REF] Vossen | Switching time optimization for bang-bang and singular controls[END_REF]. This approach still needs a practical method of verifying the more abstract SSC for bang-singular controls given in Aronna, Bonnans, Dmitruk, Lotito [START_REF] Aronna | Quadratic order conditions for bang-singular extremals[END_REF].

Fig. 4

 4 Fig. 4 Optimal solution for initial conditions (68): treatment starts after 800 days. Top row: (left) uninfected CD4 + T cells, (right) latently infected CD4 + T * cells. Middle row: (left) actively infected CD4 + T * * cells, (right) infectious virus particles V . Bottom row: (left) bang-bang control u, (right) bang-bang control u and (scaled) switching function σ in (73) satisfying the switching condition (74).

Fig. 5

 5 Fig. 5 Optimal solution for initial conditions (68): treatment starts after 800 days and and terminal condition T (t f ) = 995. Top row: (left) uninfected CD4 + T cells, (right) latently infected CD4 + T * . Middle row: (left) actively infected CD4 + T * * cells, (right) infectious virus particles V . Bottom row: (left) bang-bang control u, (right) bang-bang control u and scaled switching function σ in (73) satisfying the switching condition (74).

Fig. 6

 6 Fig. 6 Optimal solution for initial conditions (69): treatment starts after 1000 days and terminal condition T (t f ) = 995. Top row: (left) uninfected CD4 + T cells, (right) latently infected CD4 + T * . Middle row: (left) actively infected CD4 + T * * cells, (right) infectious virus particles V . Bottom row: (left) bang-bang control u, (right) bang-bang control u and scaled switching function σ in (73) satisfying the switching condition (74).

  629, λ T (t f ) = -214.021, λ T * (0) = 4209.98, λ T * * (0) = 2.69187, λ V (0) = 136.835. (80) Figure 6 depicts the state and control variables and the switching function. The SSC in Theorem 17 are satisfied, since the strict bang-bang property (77) holds and condition (63) holds in view of K 0 = {0}. Therefore, the extremal (80) provides a strict strong minimum.

Fig. 8

 8 Fig.8Control of the two-link robot (81)-(84). Top row: (left) control u 1 and scaled switching function σ 1 , (right) control u 2 and scaled switching function σ 2 . Bottom row: (left) angle q 1 and velocity ω 1 , (right) angle q 2 and velocity ω 2 .

For

Fig. 10

 10 Fig. 10 Objective (113) and constraint -5 ≤ v = u + x 1 ≤ 0 . Top row: (left) state variables x 1 , x 2 , (right) transformed control v and switching function σ = λ 2 . Bottom row: (left) adjoint variables λ 1 , λ 2 , (right) control u.

Fig. 11

 11 Fig. 11 Objective (113) and constraint -5 ≤ v = u + x 1 ≤ 0 . Top row: (left) state variables x 1 , x 2 , (right) transformed control v and switching function σ = λ 2 . Bottom row: (left) adjoint variables λ 1 , λ 2 , (right) control u = v -x 1 .

2 Second-Order Optimality Conditions for Broken Extremals in the Simplest Problem of the Calculus of Variations 2.1 The Simplest Problem in the Calculus of Variations Let

  a closed interval [t 0 ,t f ], two points a, b ∈ IR n , an open set Q ⊂ IR 2n+1 , and a function L : Q → IR of class C 2 be given. The simplest problem of the Calculus of Variations has the form

	(SP)	Minimize J (x(•)) :=	t f t 0	L(t, x(t), ẋ(t)) dt,

  = [ λ0 ] k . Sometimes we omit the superscript µ in the notation D k (H aµ ).

	[H t ] k is the jump of the derivative H a aµ t (t, x 0 (t), u 0 (t), λ (t), ν(t)) at the point t k , and λ k-:= λ (t k -), λ k+ := λ (t k +), etc. Note that H a t = -λ0 , where λ 0 (t) = -H(t), and hence -[H a t ] k

  be a fixed admissible process such that the control u(•) is a piecewise constant function on the interval ∆ = [t 0 ,t f ]. Denote byΘ = {t 1 , . . . ,t s }, t 0 < t 1 < . . . < t s < t f ,the finite set of all discontinuity points (jump points) of the control u(t). Then ẋ(t) is a piecewise continuous function whose discontinuity points belong to Θ , and hence x(t) is a piecewise smooth function on ∆ .Let us formulate the Pontryagin minimum principle, which is the first-order necessary condition for optimality of the process T . The Pontryagin function has the form

  Kirschner et al.[START_REF] Kirschner | Optimal control of the chemotherapy of HIV[END_REF] use a control quadratic cost functional of L 2type. It has been argued in Schättler, Ledzewicz, Maurer[START_REF] Ättler | Sufficient conditions fo strong local optimality in optimal control problems with L 2 -type objectives and control constraints[END_REF] that in a biological context it is more appropriate to consider cost functionals of L 1 -type which are linear in the control variable. Therefore, in this section, we are studying an objective of L 1 -type, where the quadratic control is replaced by a linear control. The state and control variables have the following meaning: T (t) : concentration of uninfected CD4 + T cells, T * (t) : concentration of latently infected CD4 + T cells, T * * (t) : concentration of actively infected CD4 + T cells, V (t) : concentration of free infectious virus particles, u(t) : control, rate of chemotherapy.

9

  Control of the two-link robot (90)-(94). Top row: (left) control u 1 , (right) control u 2 . Bottom row: (left) angle q 1 and velocity ω 1 , (right) angle q 2 and velocity ω 2 .

	G ξ ( ξ ) =	   	-10.8575 -12.7462 -5.88332 -1.14995 0.199280 -2.71051 -1.45055 -1.91476 -4.83871 0 -0.622556 3.31422 2.31545 2.94349 6.19355	   
					9.36085 3.03934 0.484459 0.0405811	0
	which has full rank (64). The Hessian of the Lagrangian is given by
	L ξ ξ ( ξ , β ) =	     	71.1424 90.7613 42.1301 8.49889 -0.0518216 90.7613 112.544 51.3129 10.7691 0.149854 42.1301 51.3129 23.9633 5.12403 0.138604 8.49889 10.7691 5.12403 1.49988 0.170781	     	.
					-0.0518216 0.149854 0.138604 0.170781	0.297359