Bernard Brogliato
email: bernard.brogliato@inrialpes.fr.

Some results on optimal control with unilateral state constraints

Keywords:

s sweeping process Complementarity problems Distributional differential inclusion

In this paper, we study the problem of quadratic optimal control with state variables unilateral constraints, for linear time-invariant systems. The necessary conditions are formulated as a linear invariant system with complementary slackness conditions. Some structural properties of this system are examined. Then it is shown that the problem can benefit from the higher order Moreau's sweeping process, that is, a specific distributional differential inclusion, and from ten Dam's geometric theory [A.A. ten Dam, K.F. Dwarshuis, J.C. Willems, The contact problem for linear continuous-time dynamical systems: A geometric approach, IEEE Trans. Automat. Control 42 (4) (1997) 458-472; A.A. ten Dam, Unilaterally Constrained Dynamical Systems, Ph.D. Thesis, Rijsuniversiteit Groningen, NL, available at http://irs.ub.rug.nl/ppn/159407869, 1997] for partitioning of the admissible domain boundary (in particular for the case of multivariable systems). In fact, the first step may be also seen as follows: does the higher order Moreau's sweeping process (developed in Acary et al. [V. Acary, B. Brogliato, D. Goeleven, Higher order Moreau's sweeping process: Mathematical formulation and numerical simulation, Math. Programm. A 113 (2008) 133-217]) correspond to the necessary conditions of some optimal control problem with an extended integral action? The knowledge of the qualitative behaviour of optimal trajectories at junction times is improved with the approach, which also paves the way towards efficient time-stepping numerical algorithms to solve the optimal control boundary value problem.

Introduction

Optimal control with state constraints is a topic of major importance, and which has attracted the attention of researchers for a long time, see [START_REF] Bryson | Applied Optimal Control; Optimization, Estimation, and Control[END_REF][START_REF] Hartl | A survey of the maximum principles for optimal control problems with state constraints[END_REF][START_REF] Jacobson | New necessary conditions of optimality for control problems with state-variable inequality constraints[END_REF][START_REF] Mcdanell | Necessary conditions for joning singular and nonsingular subarcs[END_REF][START_REF] Valentine | The problem of Lagrange with differential inequalities as added side conditions[END_REF] to cite a few. Most of this work studies the first order necessary conditions stemming from Boltyanskii-Pontryagin's maximum principle. Dynamic programming under inequality state constraints is examined in [START_REF] Augustin | Second order sufficient conditions and sensitivity analysis for the optimal control of a container crane under state constraints[END_REF][START_REF] Vinter | Optimal Control, in: Systems and Control: Foundations and Applications[END_REF], and second order optimality conditions are studied in [START_REF] Malanovski | Sensitivity analysis fo optimal control problems subject to higher order state constraints[END_REF]. The qualitative properties of optimal trajectories are studied in [START_REF] Hamilton | On nonexistence of boundary arcs in control problems with bounded state variables[END_REF][START_REF] Jacobson | New necessary conditions of optimality for control problems with state-variable inequality constraints[END_REF][START_REF] Mcdanell | Necessary conditions for joning singular and nonsingular subarcs[END_REF], while their regularity properties are examined in [START_REF] Vinter | Optimal Control, in: Systems and Control: Foundations and Applications[END_REF]. Numerical studies have received attention in [START_REF] Bonnans | Well-posedness of the shooting algorithm for state constrained optimal control problems with a single constraint and control[END_REF][START_REF] Oberle | A program for the numerical solution of optimal control problems, Report no 515 der DFVLR[END_REF][START_REF] Stryk | Direct and indirect methods for trajectory optimization[END_REF] and applications are presented in [START_REF] Augustin | Second order sufficient conditions and sensitivity analysis for the optimal control of a container crane under state constraints[END_REF][START_REF] Berkmann | Abort landing in windshear: Optimal control problem with third-order state constraint and varied switching structure[END_REF][START_REF] Bonnard | Optimal control with state constraints and the space shuttle re-entry problem[END_REF][START_REF] Bonnard | Une approche géométrique de contrôle optimal de l'arc atmosphérique de la navette spatiale[END_REF][START_REF] Rinaldi | An optimal control problem with state space constraints arising in the design of bipolar transistors[END_REF][START_REF] Steindl | Optimal control of a retrieval of a tethered subsatellite[END_REF][START_REF] Wen | Optimal feedback control of the deployment of a tethered subsatellite subject to perturbations[END_REF]. The optimal control of systems with inputs which are measures has also received attention [START_REF] Miller | Impulsive Control in Continuous and Discrete-Continuous Systems[END_REF][START_REF] Pereira | Necessary conditions of optimality for vector-valued impulsive control problems[END_REF][START_REF] Sattayatham | Strongly nonlinear impulsive evolution equations and optimal control[END_REF][START_REF] Silva | Necessary conditions for optimal impulsive control problems[END_REF]. The theory of systems with distributional inputs is progressing significantly [START_REF] Cobb | A characterization of bounded-input bounded-output stability for linear time-varying systems with distributional inputs[END_REF][START_REF] Dykhta | Optimal pulse control in models of economics and quatum electronics[END_REF][START_REF] Kinzebulatov | Systems with distributions and viability theorem[END_REF][START_REF] Orlov | Schwartz' distributions in nonlinear setting: Applications to differential equations, filtering and optimal control[END_REF], so that it becomes possible to consider control problems involving inputs which are higher order distributions based on solid ground. A further motivation for considering higher degree distributions, will be explained next. However as we shall see later, even problems with a continuous optimal controller, may involve higher degree distributions in the costate differential equation.

The following Bolza problem is of interest minimize

u(•)∈U I(u) = 1 2 T 1 0 [x(t) T Qx(t) + u(t) T Ru(t)]dt + 1 2
x(T 1 ) T Fx(T 1 )

(1) * Tel.: +33 (0) 4 76 61 53 93; fax: +33 (0) 4 76 61 54 52 52. subject to ẋ(t) = Ax(t) + Bu(t), x(0) = x0 , x(T 1 ) = x1 w(t) = Cx(t) + D ≥ 0 [START_REF] Acary | Numerical Methods for Nonsmooth Dynamical Systems[END_REF] where A, B, C and D are constant matrices, 1 (A, B, C ) is a minimal state space representation (i.e. (A, B) is controllable and (C, A) is observable [START_REF] Sontag | Deterministic Finite Dimensional Systems[END_REF]), x0 , x1 ∈ Φ = {x | Cx + D ≥ 0}, U is the set of admissible inputs, w(t) ∈ R m , u(t) ∈ R n u , x(t) ∈ R n and Q = Q T ≥ 0, R = R T > 0 (the Legendre-Clebsch condition). In the sequel, the case m = n u = 1 will be called the monovariable case, whereas m, n u ≥ 2 is the multivariable case. This is a particular example of optimal control of a dynamical system with state constraints. The least requirement for problem [START_REF] Acary | Higher order Moreau's sweeping process: Mathematical formulation and numerical simulation[END_REF] and [START_REF] Acary | Numerical Methods for Nonsmooth Dynamical Systems[END_REF] to possess a solution in U is that x1 belongs to the reachable set from x0 , 0 ≤ T 1 < +∞. A fundamental ingredient is therefore U, as illustrated by Example 7 in Appendix C. This example shows that distributions can easily appear in unilaterally constrained controlled systems. Clearly then the formulation of the optimal control problem in (1) and (2) will have to be modified so that the integral action makes sense. This will be the object of the first part of this paper, and detailed in the next sections.

It is worth noting the strong similarities between this optimal control problem, and the so-called Continuous Linear Programming Problem (CTLP) [START_REF] Anstreicher | Generation of feasible descent directions in continuous time linear programming[END_REF][START_REF] Perold | Fundamentals of a continuous time simplex method[END_REF], introduced by Bellman [START_REF] Bellman | Bottleneck problems and dynamical programming[END_REF] to model some economic processes

         maximize [0,T 1 ] c T (t)x(t)dt subject to: H(t)x(t) + t 0 G(s, t)x(s)ds ≤ a(t) x(t) ≥ 0, t ∈ [0, T 1 ]. (3)
When the integrand of I(u) in ( 1) is linear in x(•) and u(•), then the linear optimal control problem with both state and input constraints, is a particular case of the CTLP in (3) [START_REF] Perold | Fundamentals of a continuous time simplex method[END_REF]. The optimal control problem (1) and ( 2) and the dual problem of the CTLP in (3) share important features concerning the presence of higher degree distributions in their solution [3, Section 2.2], [START_REF] Perold | Fundamentals of a continuous time simplex method[END_REF], and the need for time-stepping numerical algorithms to numerically solve them. Some macroeconomics models directly yield optimal control problems with unilateral state constraints and optimal controllers with singular measures [START_REF] Dykhta | Optimal pulse control in models of economics and quatum electronics[END_REF]. This is why we will embed the optimisation problem (1) and (2) into a suitable framework which allows us to rigorously take into account the possible presence of higher degree distributions.

The first order necessary conditions for the optimal control problem in ( 1) and ( 2) can be formulated as [START_REF] Hartl | A survey of the maximum principles for optimal control problems with state constraints[END_REF][START_REF] De Pinho | An Euler-Lagrange inclusion for optimal control problems with state constraints[END_REF][START_REF] Van Der Schaft | An Introduction to Hybrid Dynamical Systems[END_REF][START_REF] Vinter | Optimal Control, in: Systems and Control: Foundations and Applications[END_REF]]

       ẋ(t) η(t) = A BR -1 B T Q -A T x(t) η(t) + 0 -C T λ(t) (a)
x(0) = x0 , η(T 1 ) = Fx(T 1 )

+ C T γ + β = F x1 + C T γ + β = η 1 (b) 0 ≤ w(t) = Cx(t) + D ⊥ λ(t) ≥ 0 (c) (4)
where η(t) ∈ R n , 0 ≤ γ ∈ R m , γ T (Cx(T 1 ) + D) = 0, β ∈ R n , and the optimal control is given on intervals where η(•) is a function by

u(t) = argmax u∈U - 1 2 u T Ru + η(t) T Bu = R -1 B T η(t). (5)
An implicit assumption which allows one to write (1), ( 4) and ( 5) is that the multiplier λ is a measure. If not, these expressions have to be generalized to be meaningful. This is the first objective of this work. In this paper, the optimal trajectory and optimal controller will not be given a special notation in order to lighten the presentation. The values of the multipliers γ and β can be calculated from some boundary conditions [START_REF] Bryson | Applied Optimal Control; Optimization, Estimation, and Control[END_REF]Section 2.8]. If x(T 1 ) is not specified and F = 0 then η(T 1 ) = 0. We assume that the Slater constraint qualification (∃x such that Cx + D > 0) is satisfied, and that the problem (1) and ( 2) is normal, a property that is related to the controllability of the constrained system (see e.g. [START_REF] De Pinho | An Euler-Lagrange inclusion for optimal control problems with state constraints[END_REF]Propositon 4.1] in case λ is a measure). We will assume that reachability and normality hold by taking u(•) in an enlarged set of distributions that will be described later (see Section 3.5). From a general point of view however, the controllability problem for [START_REF] Acary | Numerical Methods for Nonsmooth Dynamical Systems[END_REF] remains open, and we will not tackle it here (see Appendix C for some preliminary hints). In the sequel we shall denote xT = (x T , η T ), ( Ã, B, C) the triple associated to the system in (4)(a) (c), which we shall often refer to in the sequel as the necessary conditions system. It is clear that if λ is a measure, then from (4)(c) -C T λ(t) ∈ ∂ψ Φ (x(t)) [START_REF] Bellman | Bottleneck problems and dynamical programming[END_REF] (using [START_REF] Goeleven | Variational and Hemivariational Inequalities: Theory, Methods and Applications; Volume I: Unilateral Analysis and Unilateral Mechanics[END_REF]Proposition 1.3.11] on differentiation of convex functions) and the complementarity implies supp(λ) ⊂ {t | Cx(t) + D = 0} [START_REF] Berkmann | Abort landing in windshear: Optimal control problem with third-order state constraint and varied switching structure[END_REF] 1 In the paper D will also be used to denote the distributional derivative.

It is noteworthy that when λ is not a function (e.g. a Dirac measure), then the meaning of the inclusion in [START_REF] Bellman | Bottleneck problems and dynamical programming[END_REF] has to be stated rigorously, see [START_REF] Acary | Higher order Moreau's sweeping process: Mathematical formulation and numerical simulation[END_REF]Section 2] or [31, p. 76]. Then the dynamical system in (4)(a) (c) may be viewed as a measure differential inclusion ẋ(t) -Ãx(t) ∈ ∂ψ Φ (x(t)) [START_REF] Bonnans | Well-posedness of the shooting algorithm for state constrained optimal control problems with a single constraint and control[END_REF] with Φ = {x| C x + D ≥ 0}, and where a state reinitialisation mapping is missing in [START_REF] Bonnans | Well-posedness of the shooting algorithm for state constrained optimal control problems with a single constraint and control[END_REF] (let us notice from [START_REF] Berkmann | Abort landing in windshear: Optimal control problem with third-order state constraint and varied switching structure[END_REF] that the control input u(•) must be a continuous function of time (even analytic) everywhere outside supp(λ)). However as we shall see in Section 3, in general one has to resort to a more complex formalism to correctly handle [START_REF] Arutyunov | State constraints in optimal control. The degeneracy phenomenon[END_REF]. Except if additional constraints are added to [START_REF] Arutyunov | State constraints in optimal control. The degeneracy phenomenon[END_REF] to assure that λ is a measure, then the formalism in [START_REF] Arutyunov | State constraints in optimal control. The degeneracy phenomenon[END_REF] and especially the complementary slackness condition (4)(c) are meaningless. When m = 1, the relative degree r of the triple ( Ã, B, C) in ( 4) is twice the relative degree r wu of the triple (A, B, C ) in (2) [START_REF] Schumacher | Complementarity systems in optimization[END_REF] (what we call the relative degree in this paper is often referred to as the constraints order in the optimal control literature [START_REF] Jacobson | New necessary conditions of optimality for control problems with state-variable inequality constraints[END_REF]). In case n u = m ≥ 2 we will say that the triple (A, B, C ) has a uniform vector relative degree (r wu , . . . , r wu ) T ∈ R m denoted simply as r wu , if the matrix CA r wu -1 B ∈ R m×m whose entries are the scalars C i A r wu -1 B j , is full rank, and C i A k-1 B j = 0 for all 1 ≤ i ≤ m, 1 ≤ j ≤ m, and k ≤ r wu -2. This definition is useful to derive a special state space representation [START_REF] Acary | Higher order Moreau's sweeping process: Mathematical formulation and numerical simulation[END_REF][START_REF] Sannuti | Direct singular perturbation analysis of high-gain and cheap control problems[END_REF].

Remark 1. It is known [START_REF] Van Der Schaft | An Introduction to Hybrid Dynamical Systems[END_REF][START_REF] Vinter | Optimal Control, in: Systems and Control: Foundations and Applications[END_REF]] that (4)(a) (c) is a Hamiltonian system of the form ẋ(t) = J   ∂H(x, η) ∂x ∂H(x, η) ∂η

  + 0 n×n -I n C T λ, J = 0 n×n I n -I n 0 n×n
, whose unconstrained part has the Hamiltonian function

H(x, η) = 1 2 xT -Q A T A BR -1 B T x. (9)
Notice that in ( 5) u(t) = argmax u∈U H(x(t), u, η(t)) with

H(x, u, η) = - 1 2 x T Qx - 1 2 u T Ru + η T (Ax + Bu). ( 10
)
Let us assume that λ is a measure. Defining an augmented Lagrangian function as L(x, u, λ) = L(x, u) -λ T (Cx + D), the augmented Hamiltonian function is H(x, η) = H(x, η) + λ T (Cx + D). One sees that [START_REF] Bonnans | Well-posedness of the shooting algorithm for state constrained optimal control problems with a single constraint and control[END_REF] can also be rewritten as (η(t), ẋ(t)) ∈ ∂ H(x(t), η(t)), where ∂ is the subdifferential of convex analysis.

Remark 2. The support condition [START_REF] Berkmann | Abort landing in windshear: Optimal control problem with third-order state constraint and varied switching structure[END_REF] is fundamental. It will also hold when λ is a distribution of higher degree. It implies that in the problem we are examining, the control u and costate η are allowed to have a degree larger than 1, only at junction times.

In this paper, we provide a framework that allows us to give a meaning to the dynamical system in (4)(a) (c) when λ is not a measure; in particular new multipliers are introduced and it is shown that this framework may be useful even if the optimal controller u(•) is a function (but the costate η may be a distribution). This allows us to introduce a generalized action I(u) that handles distributional cases. Then we provide a detailed study of what happens at the entry times, using the geometrical approach of ten Dam [START_REF] Dam | The contact problem for linear continuous-time dynamical systems: A geometric approach[END_REF][START_REF] Dam | Unilaterally constrained dynamical systems[END_REF]. This allows us to generalize some results on the existence (or non-existence) of entry states for odd relative degree systems, and to tackle the multivariable case. Finally a study of boundary arcs is proposed using complementarity theory. All these tools are introduced for the first time in the context of optimal control with inequality state constraints and are proved to significantly improve the qualitative knowledge of the optimal trajectories. The approach also paves the way towards the design of time-stepping numerical algorithms to solve the BVP in [START_REF] Arutyunov | State constraints in optimal control. The degeneracy phenomenon[END_REF].

The paper is organised as follows: in Section 2 some properties of the system in (4)(a) are proved. In Section 3 a distributional differential inclusion is presented in which the system in (4)(a) (c) is embedded; the properties of its solutions are recalled. In Section 4 the geometric approach of ten Dam [START_REF] Dam | Unilaterally constrained dynamical systems[END_REF][START_REF] Dam | The contact problem for linear continuous-time dynamical systems: A geometric approach[END_REF] is used to study the behaviour of the trajectories of (2) at junction times. In Section 5 the behaviour of the system when trajectories of (4)(a) evolve on the boundary of Φ, is analysed.

A numerical scheme is outlined in Section 6. Conclusions are given in Section 7, followed by the notation used in the paper. Some mathematical definitions are recalled in Appendix A. An example of a function that belongs to the considered space of solutions is given in Appendix B. An example concerning the controllability of (2) and the set of admissible inputs is in Appendix C.

Some properties of the necessary conditions system

The next lemma is important for the well-posedness of (4)(a) seen as an IVP (x(0 -) = x 0 , η(0 -) = η 0 ), and will also be useful for the qualitative analysis of the BVP solutions, see Sections 4 and 5. We consider R = I n and m = 1 in Lemma 1.

Lemma 1. If r wu

= 1, then the leading Markov parameter of the triple ( Ã, B, C) is M (2) = -CBB T C T < 0. More generally if the transfer function of the triple (A, B, C ) in (2) has a relative degree r wu ≥ 2, the leading Markov parameter of the triple ( Ã, B, C)

is M (2r wu ) = (-1) r wu CA r wu -1 B(CA r wu -1 B) T (= C Ãr-1 B). Proof. Let us denote à = A BB T Q -A T = Ã1 Ã2 Ã3 Ã4
and Ãr = Ã(r)

1 Ã(r) 2 Ã(r) 3 Ã(r) 4
for some r ≥ 1. The first assertion of the Lemma is a simple calculation with Ã(1)

2

= BB T and M (1) = -C Ã(1)

2 C T . The leading Markov parameter we want to compute is

M (2r+2) = (C 0) Ã2r+1 (0 -C ) T = -C Ã(2r+1) 2 C T . If r wu = α + 1, then CA i B = 0 for all 0 ≤ i ≤ α -1. Our objective is to show that M (2α+2) = (-1) α+1 CA α BB T (A T ) α C T , α ≥ 1.
By direct calculation we have for any integer i ≥ 1

Ã(i) 1 = Ã(i-1) 1 A + Ã(i-1) 2 Q Ã(i+1) 2 = Ã(i) 1 BB T -Ã(i) 2 A T . ( 11
)
It follows that

         Ã(2α) 2 = 2α-1 i=1 (-1) i+1 Ã(i) 1 BB T (A T ) 2α-i-1 -Ã(1) 2 (A T ) 2α-1 (a) Ã(2α+1) 2 = 2α i=1 (-1) i Ã(i) 1 BB T (A T ) 2α-i + Ã(1) 2 (A T ) 2α . (b) (12)
This can be shown using [START_REF] Brogliato | Nonsmooth Mechanics[END_REF] and then by induction. From [START_REF] Brogliato | Nonsmooth Mechanics[END_REF], it follows by induction that Ã(α)

1 = A α + L α (Q )
, where L α (Q ) is some matrix depending on the matrix Q . Indeed, let us assume that Ã(i-1) [START_REF] Bryson | Applied Optimal Control; Optimization, Estimation, and Control[END_REF] and since Ã1 = A and Ã(2)

1 = A i-1 + L i-1 (Q ). Then Ã(i) 1 = (A i-1 +L i-1 (Q ))A+ Ã(i-1) 2 Q = A i +L i (Q ) with L i (Q ) = L i-1 (Q )A+ Ã(i-1) 2 Q . Using
1 = A 2 +BB T Q , the proof is complete. From (12)(b) one deduces that M (2α+2) = 2α i=1 (-1) i+1 C Ã(i) 1 BB T (A T ) 2α-i C T + C Ã(1) 2 (A T ) 2α C T . Assume for the time being that CL i (Q )BB T (A T ) 2α-i C T
= 0 for all 1 ≤ i ≤ α (the terms with α + 1 ≤ i ≤ 2α need not be considered as CA i B = 0 for all 0 ≤ i ≤ α -1). Then we obtain M (2α+2) = 2α i=1 (-1) i+1 CA i BB T (A T ) 2α-i C T , and the only nonzero term in this sum is for i = α. Therefore M (2α+2) = (-1) α+1 CA α BB T (A T ) α C T . To end the proof, it can be shown again by induction using [START_REF] Bryson | Applied Optimal Control; Optimization, Estimation, and Control[END_REF] that L i (Q ) is composed of terms of the general form A j B or B T (A T ) j , with 0 ≤ j ≤ i -2 and is some matrix. Indeed one has

L i (Q ) = L i-1 (Q )A + Ã(i-1) 2 Q , as shown above. Assume that L i-1 (Q ) = i-3 j=0 A j B + B T (A T ) j . Then L i (Q ) = i-3 j=0 A j B A + B T (A T ) j+1 + Ã(i-1) 2
Q . One concludes from [START_REF] Bryson | Applied Optimal Control; Optimization, Estimation, and Control[END_REF]. Thus

CL i (Q )BB T (A T ) 2α-i C T = 0 for all 1 ≤ i ≤ α as required.
Lemma 1 continues to hold for systems with a uniform relative degree and m ≥ 1 (recall that C ∈ R m×n ) since CA i B = 0 m×m for all 0 ≤ i ≤ r wu -2 in this multivariable case also. In such a case, having M (2r wu ) ∈ R m×m full-rank implies that the vectors C T i are independent, where C i is the i-th row of C , and B has rank m.

Example 1. Consider the system

               ẋ1 (t) = x 1 (t) + x 2 (t) ẋ2 (t) = x 3 (t) ẋ3 (t) = x 4 (t) + x 3 (t) ẋ4 (t) = x 4 (t) + x 6 (t) ẋ5 (t) = x 5 (t) + x 1 (t) ẋ6 (t) = u(t) w(t) = x 1 (t) ≥ 0. ( 13
)
Then r wu = 5, r = 10, CA 4 B = 1 and M (10) = -1.

Let us introduce a canonical state space representation for the system ( Ã, B, C, D). This is a canonical representation which makes the so-called zero-dynamics explicitly appear. It will be useful for some subsequent developments. For a triple ( Ã, B, C), with input λ ∈ R, output w ∈ R, and relative degree r ≥ 1, it reads [START_REF] Sannuti | Direct singular perturbation analysis of high-gain and cheap control problems[END_REF]:

                         ż1 (t) = z 2 (t) ż2 (t) = z 3 (t) ż3 (t) = z 4 (t) . . . żr-1 (t) = z r (t) żr (t) = C Ãr W -1 z(t) + C Ãr-1 Bλ(t) ξ (t) = Ãξ ξ (t) + Bξ z 1 (t) w(t) = z 1 (t) + D, z 1 (t) = Cx(t) z(0 -) = z 0 . (14)
We call it the z-dynamics, or z-canonical form, and the full-rank state transformation matrix from (4) to ( 14) is W : z = W x. A similar transformation can be applied to (A, B, C ), and we then call the obtained representation, the z-dynamics. Then z = Wx for a full-rank matrix W . We denote zT = (z 1 , . . . , z r ) and zT = (z 1 , . . . , z r wu ). Thus the z-dynamics is characterized by (r, C Ãr , C Ãr-1 B, Ãξ , Bξ ), and the z-dynamics by (r wu , CA r wu , CA r wu -1 B, A ξ , B ξ ). The z-dynamics may be seen as a dynamical extension of z-dynamics, taking into account u = B T η and the costate equation. In the sequel, the components of z and of z will both be denoted as z i . Since the zero dynamics play an important role in the systems we are dealing with, the following is of interest. Lemma 2. Let m = 1. Consider the z-and z-canonical forms that correspond to the systems (A, B, C ) in (2) and ( Ã, B, C) in (4), and let us denote the transitions matrices of their zero dynamics as A ξ and Ãξ respectively. Then if σ is an eigenvalue of A ξ with multiplicity m σ , -σ and σ are both eigenvalues of Ãξ , both with multiplicities m σ .

Proof. Let Q ≥ 0 and let L ∈ R n×n be such that L T L = Q . The transfer function that corresponds to the operator λ → w in ( 4) is equal to [START_REF] Schumacher | Complementarity systems in optimization[END_REF], where G(s) is the transfer of the triple (A, B, C ) in ( 2). H(s) is the transfer matrix of the triple (A, B, L), H * (s) is the transfer matrix of the triple (-A T , L T , B T ), i.e. the adjoint system to (A, B, L) [55, p. 280]. Consider now the two canonical forms associated to the systems in ( 2) and (4), respectively. From Lemma 1 one has ξ ∈ R n-r wu and ξ ∈ R 2(n-r wu ) . From the fact that H wλ (s) = G(s)G(-s)(1 -H * (s)H(s)) -1 and since the ξ -dynamics corresponds to the numerator B(s) of the transfer function G(s), we deduce that the roots of the polynomial B(α)B(-α) = 0 are modes of the matrix Ãξ . Since the order of B(α)B(-α) is precisely 2(nr wu ), the result follows.

H wλ (s) = C(sI 2n -Ã) -1 B = G(s)G(-s)(1 -H * (s)H(s)) -1
We therefore have a complete description of the triple ( Ã, B, C) in (4) in terms of its relative degree and zero dynamics.

The role played by the zero dynamics will prove to be important, see Section 5.

The higher order Moreau's sweeping process

Presentation of the differential inclusion

Let us consider the system in (4)(a) (c), and let us forget for the time being, that it may represent the necessary conditions of the maximum principle. An important point is first to understand the dynamics of such a dynamical system involving complementary-slackness conditions. For instance, what is the meaning of λ ≥ 0 if λ is not a measure (distributional multipliers may easily be needed to integrate (4) (a) (c))? The answer to this question is given later in Proposition 2. Before this, let us recall some facts about the higher order sweeping process (HOSP) as it is introduced in [START_REF] Acary | Higher order Moreau's sweeping process: Mathematical formulation and numerical simulation[END_REF], which provides a rigorous framework to study the dynamics in (4)(a) (c), and extends the well-known first and second order sweeping process (see references in [START_REF] Acary | Higher order Moreau's sweeping process: Mathematical formulation and numerical simulation[END_REF] and [START_REF] Brogliato | Nonsmooth Mechanics[END_REF]Section 5.3] for a non-mathematical introduction). Roughly speaking, the HOSP is a specific differential inclusion, whose solutions are distributions, and which permits to give a meaning to the system (4) (especially the positiveness of λ when it is not a measure). Let I be a real nondegenerate interval. We denote as T n (I) the set of distributions of degree n + 1 [START_REF] Ferreira | Introduction to the Theory of Distributions[END_REF] which are generated by RCSLBV functions on I (see ( 62) and (63) in Appendix A, whose successive derivatives possess an absolutely continuous part (denoted as [•]) that is also RCSLBV on I. The right derivative of

[h] is denoted as ĥ(1) = d + [h] dt (t) = lim σ →0 + [h](t+σ )-[h](t) σ
. The set of such functions is denoted as

F ∞ (I; R) = ∩ k∈N F k (I; R), with F k (I; R) = {h ∈ F k-1 (I; R) : ĥ(k) := d + dt [ ĥ(k-1) ] ∈ RCSLBV(I; R)}. In particular F 0 (I; R) = RCSLBV(I; R). If T ∈ T n (I) and is generated by a function F ∈ F ∞ (I; R), it has a ''function'' part denoted as {T }(•) = [ F (n)
](•), and a ''measure'' part denoted as

T such that T , ϕ = +∞ -∞ ϕd[ F (n-1)
], ∀ ϕ ∈ C ∞ 0 (I). D denotes the distributional derivative, and dz denotes the Stieltjes or differential measure generated by a function z of local bounded variation (see Appendix A). Thus

T n (I) denotes the set of all Schwartz' distributions, such that there exists a function F ∈ F ∞ (I; R) such that T = D n F . Let n be the smallest integer such that T ∈ T n (I), we set

deg(T ) = n + 1 if n ≥ 1 1 if n = 0 and E 0 ({T }) = ∅ 0 if n = 0 and E 0 ({T }) = ∅. (15)
The Dirac measure δ 0 has degree 2, its derivative δ0 has degree 3, etc. Continuous functions have degree 0, and discontinuous functions have degree 1. The distributions we work with originate from LBV functions. This is crucial for the characterization of their support. Since the notion of a solution for the formalism in which we shall embed (4) is crucial, an example is provided in detail in Appendix B. We insist here on the fact that that the solutions that will be considered next, are of special bounded variation. In another words, their derivatives do not contain any singular Lebesgue integrable part. See Definition 1 and Eq. [START_REF] Maurer | On optimal control problems with bounded state variables and control appearing linearly[END_REF]. The core of the HOSP lies in the definition of the following set of tangent cones. Here, and in the following, we suppose that m = 1 and D = 0 in [START_REF] Acary | Numerical Methods for Nonsmooth Dynamical Systems[END_REF], in order to fit within the framework of [START_REF] Acary | Higher order Moreau's sweeping process: Mathematical formulation and numerical simulation[END_REF]. A non zero D can be considered with slight modifications of the tangent cones below, where Φ = R + should be replaced by Φ = [-D, +∞). Let Φ be a nonempty closed convex subset of R. We denote by T Φ (x) the tangent cone of Φ at x ∈ R defined by

T Φ (x) = cone(Φ -{x}) (16)
where cone(Φ -{x}) denotes the cone generated by Φ -{x}, defined as in [START_REF] Moreau | Unilateral contact and dry friction in finite freedom dynamic[END_REF] to take into account constraint violations, and cone(Φ -{x}) is its closure. Given a closed nonempty convex set Φ ⊂ R, we set

T 0 Φ (0) = Φ, T 1 Φ (z 1 ) = T Φ (z 1 ), T 2 Φ (z 1 , z 2 ) = T T 1 Φ (z 1 ) (z 2 ), T i Φ (z 1 , . . . , z i ) = T T i-1 Φ (z 1 ,...,z i-1 ) (z i ), i ≥ 0.
In the sequel we will fix Φ = R + when m = 1.

Remark 3. If m ≥ 2 and Φ = (R + ) m then T i Φ (z 1 , . . . , z i ) = × m l=1 T i Φ (z l 1 , . . . , z l i ).
Note that

T R + (x) = R if x > 0 R + if x ≤ 0 and T R (x) = R.
∂ψ T i-1 Φ (z 1 ,...,z i-1 ) (z i ) = {w ∈ R : w, v -z i ≤ 0, ∀v ∈ T i-1 Φ (z 1 , . . . , z i-1 )}
and is the outward normal cone to T i-1 Φ (z 1 , . . . , z i-1 ) at z i .

(ii) It can be shown that

T i-1 Φ (z 1 , . . . , z i-1 ) = R ⇒ ∂ψ T i-1 Φ (z 1 ,...,z i-1 ) (z i ) = {0}, T i-1 Φ (z 1 , . . . , z i-1 ) = R + and z i > 0 ⇒ ∂ψ T i-1 Φ (z 1 ,...,z i-1 ) (z i ) = {0}, T i-1 Φ (z 1 , . . . , z i-1 ) = R + and z i ≤ 0 ⇒ ∂ψ T i-1 Φ (z 1 ,...,z i-1 ) (z i ) = R -.
Let us introduce now two mathematical formalisms of the HOSP. For this we rely upon the special state space representation in [START_REF] Çamlibel | Complementarity methods in the analysis of piecewise linear dynamical systems[END_REF], the reason for this particular choice being explained in [START_REF] Acary | Higher order Moreau's sweeping process: Mathematical formulation and numerical simulation[END_REF].

Distributional formalism. Find z 1 , . . . , z r ∈ T r-1 (R + ) and ξ i ∈ T r-1 (R + ) (1 ≤ i ≤ n -r) satisfying the distributional equations                    Dz 1 -z 2 = 0 Dz 2 -z 3 = 0 Dz 3 -z 4 = 0 . . . Dz r-1 -z r = 0 Dz r -C Ãr W -1 z = C Ãr-1 Bλ Dξ = Ãξ ξ + Bξ z 1 (17) λ = ( C Ãr-1 B) -1 [D (r-1) Dz 1 -{z 2 } + • • • + +D Dz r-1 -{z r } ] + Dz r -C Ãr W -1 {z} (18)
and

                     d{z 1 } -{z 2 }(t)dt ∈ -∂ψ T 0 Φ ({z 1 }(t + )) . . . d{z i } -{z i+1 }(t)dt ∈ -∂ψ T i-1 Φ ({z 1 }(t -),...,{z i-1 }(t -)) ({z i }(t + )) . . . d{z r-1 } -{z r }(t)dt ∈ -∂ψ T r-2 Φ ({z 1 }(t -),...,{z r-2 }(t -)) ({z r-1 }(t + )) ( C Ãr-1 B) -1 [d{z r } -C Ãr W -1 {z}(t)dt] ∈ -∂ψ T r-1 Φ ({z 1 }(t -),...,{z r-1 }(t -)) ({z r }(t + )). (19)
More compactly [START_REF] Cottle | The Linear Complementarity Problem[END_REF] is rewritten as Dz -W Ã W -1 z -W Bλ = 0 or Dx -Ãx -Bλ = 0. The relations given in [START_REF] Dam | Unilaterally constrained dynamical systems[END_REF]

d{z i } dµ i (t) -{z i+1 }(t) dt dµ i (t) ∈ -∂ψ T i-1 Φ ({z 1 }(t -),...,{z i-1 }(t -)) ({z i }(t + )), dµ i -a.e. t ∈ R (1 ≤ i ≤ r -1) (20)
and

( C Ãr-1 B) -1 d{z r } dµ r (t) -C Ãr W -1 {z}(t) dt dµ r (t) ∈ -∂ψ T r-1 Φ ({z 1 }(t -),...,{z r-1 }(t -)) ({z r }(t + )), dµ r -a.e. t ∈ R. (21)
The solutions of the distributional formalism will be shown to be distributions of degree possibly larger than 1. Let us now introduce a second formalism whose solutions are functions, independently of the relative degree r wu . This will be quite useful when the optimal control problem is embedded into the HOSP, see Sections 3.5 and 3.6.

Measure differential formalism. Find

z i ∈ F ∞ (R + ; R) (1 ≤ i ≤ r) and ξ i ∈ F ∞ (R + ; R)(1 ≤ i ≤ 2n -r) such that 2 dz i -z i+1 (t)dt ∈ -∂ψ T i-1 Φ (z 1 (t -),...,z i-1 (t -)) (z i (t + )) (1 ≤ i ≤ r -1) (22)
( C Ãr-1 B) -1 [dz r -C Ãr W -1 z(t)dt] ∈ -∂ψ T r-1 Φ (z 1 (t -),...,z r-1 (t -)) (z r (t + )) (23)
and

ξ (t) -Ãξ ξ (t) -Bξ z 1 (t) = 0, dt -a.e.t ∈ R. (24)
The system in [START_REF] Goeleven | Variational and Hemivariational Inequalities: Theory, Methods and Applications; Volume I: Unilateral Analysis and Unilateral Mechanics[END_REF] and [START_REF] Hamilton | On nonexistence of boundary arcs in control problems with bounded state variables[END_REF] has to be interpreted in the following sense: find nonnegative real-valued Radon measure dµ i relative to which the Lebesgue measure dt and the Stieltjes measure dz i possess densities dt dµ i and dz i dµ i respectively such that

dz i dµ i (t) -z i+1 (t) dt dµ i (t) ∈ -∂ψ T i-1 Φ (z 1 (t -),...,z i-1 (t -)) (z i (t + )), dµ i -a.e. t ∈ R (1 ≤ i ≤ r -1) (25)
and

( C Ãr-1 B) -1 dz r dµ r (t) -C Ãr W -1 z(t) dt dµ r (t) ∈ -∂ψ T r-1 Φ (z 1 (t -),...,z r-1 (t -)) (z r (t + )), dµ r -a.e. t ∈ R. (26)
Another, more intuitive way to see the measure differential formalism, is to consider the following:

Find z 1 , . . . , z r , ξ 1 , . . . , ξ n-r ∈ F ∞ (R + ; R) and measures dν 1 , . . . , dν r such that                            dz 1 = z 2 (t)dt + dν 1 dz 2 = z 3 (t)dt + dν 2 dz 3 = z 4 (t)dt + dν 3 . . . dz i = z i+1 (t)dt + dν i . . . dz r-1 = z r (t)dt + dν r-1 dz r = C Ãr W -1 z(t)dt + C Ãr-1 Bdν r ξ (t) = Ãξ ξ (t) + Bξ z 1 (t) (27)
with the inclusions

dν i ∈ -∂ψ T i-1 Φ ({z 1 }(t -),...,{z i-1 }(t -)) ({z i }(t + )) for all 1 ≤ i ≤ r ( 28
)
and λ is in [START_REF] Dam | The contact problem for linear continuous-time dynamical systems: A geometric approach[END_REF]:

λ = ( C Ãr-1 B) -1 [D (r-1) ν 1 + • • • + Dν r-1 ] + ν r . (29)
In other words λ is a distribution whose measure part is ν r and its positivity is understood as the positivity of ν r . If one imposes that λ is a measure, then ν 1 = ν 2 = • • • = ν r-1 = 0. The measures ν i = dν i are multipliers whose meaning in the context of optimality will be made clear later (see Corollaries 8 and 9, and Remark 9). From [START_REF] Kinzebulatov | Systems with distributions and viability theorem[END_REF] and [START_REF] Lancaster | The Theory of Matrices[END_REF] it follows that

dν i ({t}) = {z i }(t + ) -{z i }(t -) ∈ -∂ψ T i-1 Φ ({z 1 }(t -),...,{z i-1 }(t -)) ({z i }(t + )), ( 1
≤ i ≤ r -1) (30)
and

{z r }(t + ) -{z r }(t -) = M (r) dν r {t} ∈ -C Ãr-1 B ∂ψ T r-1 Φ ({z 1 }(t -),...,{z r-1 }(t -)) ({z r }(t + )). (31)
If z(τ -) ≺ 0, then from ( 30) and (31) a state jump occurs and deg(λ) ≥ 2. We recall that given a closed convex nonempty set K and M = M T > 0, prox M [K ; z] is the closest point to z in K , in the metric defined by M. To better understand [START_REF] Malanovski | Sensitivity analysis fo optimal control problems subject to higher order state constraints[END_REF] and [START_REF] Monteiro Marques | Differential Inclusions in Nonsmooth Mechanical Problems, Shocks and Dry Friction[END_REF] it is useful to recall here the equivalences for two vectors of appropriate dimension, M = M T > 0 and K a closed convex set:

x -y ∈ -M -1 ∂ψ K (x) ⇐⇒ x = prox M [K , y] ⇐⇒ x = argmin z∈K 1 2 (z -y) T M(z -y) ⇐⇒ x -y, v -x ≥ 0 for all v ∈ K .
The measure differential formalism is useful in the context of optimal control, see Section 3.5. Indeed it is a formalism which retains only the ''measure part'' of the distributional formalism, i.e. the states discontinuities. Most importantly its solutions are functions of time independently of the degree of λ. Both formalisms are related through the following.

Proposition 1 ([1]). (i) Let (z 1 , . . . , z r , ξ ) ∈ (T r-1 (R + )) n be a solution of Problem (17)-(19), with {z}(0 -) = z 0 ∈ R n . Then z 1 = {z 1 } ∈ F ∞ (R + ; R), z i ∈ T i-1 (R + ) (2 ≤ i ≤ r), ξ = {ξ } ∈ (F ∞ (R + ; R)) n-r and ({z 1 }, . . . , {z r }, ξ ) is a solution of Problem (22)-(24), with {z}(0 -) = z 0 ∈ R n . (ii) Let (w 1 , . . . , w r , ξ ) ∈ (F ∞ (R + ; R)) n be a solution of Problem (22)-(24), with {z}(0 -) = z 0 ∈ R n . Then (z 1 , . . . , z r , ξ ) ∈ (T r-1 (R + )) n ,
where z 1 := w 1 and

z i = w i + i-1 j=1   t k ∈E 0 (w j ) (w j (t + k ) -w i (t - k ))δ (i-j-1) t k   (2 ≤ i ≤ r),
is a solution of Problem (17)- [START_REF] Dam | Unilaterally constrained dynamical systems[END_REF], with {z}(0 -) = z 0 ∈ R n .

An example is treated in detail in Section 4.4.3 which helps understanding how ( 17)-( 19) is integrated in time and how the state jump mappings [START_REF] Malanovski | Sensitivity analysis fo optimal control problems subject to higher order state constraints[END_REF] and [START_REF] Monteiro Marques | Differential Inclusions in Nonsmooth Mechanical Problems, Shocks and Dry Friction[END_REF] work. Notice that Proposition 1 applies to the solutions of the BVP as well. It is noteworthy that if the distribution z is a solution of the distributional problem in ( 17)- [START_REF] Dam | Unilaterally constrained dynamical systems[END_REF] and if the function ζ (•) is a solution of the measure problem ( 22)- [START_REF] Hartl | A survey of the maximum principles for optimal control problems with state constraints[END_REF], then {z}(•) = ζ (•) almost everywhere on [0, T 1 ]. It is also clear that if λ is a measure, then both formalisms are equivalent one to each other. From a Control engineer point of view, the ''real'' solution is the solution of the distributional formalism. However the measure differential formalism will prove to be quite useful to formulate an extended optimal control problem.

Definition 1. Let 0 ≤ a < b ∈ R ∪ {+∞} be given. We say that a solution z ∈ (T r-1 (R + )) n of (17)-(19), with {z}(0 -) = z 0 ∈ R n , is regular on [a, b) if for each t ∈ [a, b), there exists a right neighborhood [t, σ ) (σ > 0) such that the restriction of {z} to [t, σ ) is analytic.
Regular solutions may possess accumulations of jumps, but only on the left of any t ∈ [a, b), as right accumulations cannot exist by definition. They encompass right-analytic solutions [START_REF] Perold | Fundamentals of a continuous time simplex method[END_REF]. The following is proved in [START_REF] Acary | Higher order Moreau's sweeping process: Mathematical formulation and numerical simulation[END_REF] and concerns the IVP in ( 17)- [START_REF] Dam | Unilaterally constrained dynamical systems[END_REF].

Theorem 1. Suppose that C Ãr-1 B > 0 and m = 1. For each z 0 ∈ R n , the system in (17)- [START_REF] Dam | Unilaterally constrained dynamical systems[END_REF], with {z}(0 -) = z 0 ∈ R n has at least one regular solution.

Moreover:

(i) z 1 (•) ≡ {z 1 }(•) ≥ 0 on R + (ii) {z}(t) ≤ √ e ct z0 , ∀ t ∈ R + ,
for some c > 0, (iii) If z1 and z2 are two regular solutions of (17)- [START_REF] Dam | Unilaterally constrained dynamical systems[END_REF], with {z}(0

-) = z0 ∈ R n then z1 , ϕ = z2 , ϕ , ∀ϕ ∈ C ∞ 0 (R + ; R n );
where z 0 is uniquely defined by

z 0,i = prox T i-1 Φ (z 0,1 , . . . , z 0,i-1 ); z 0,i , ∀ 1 ≤ i ≤ r -1, z 0,r = prox ( C Ãr-1 B) -1 T r-1
Φ (z 0,1 , . . . , z 0,r-1 ); z 0,r and z 0,l = z 0,l , (r

+ 1 ≤ l ≤ 2n).
It is proved that uniqueness holds in the class of regular solutions only. A crucial step is to understand the relationship between U and the class of solutions of the HOSP. Indeed it is possible that reachability of x1 from x0 in (2), holds for a u which belongs to a set U of distributions. However is there a correspondence between this U and T r-1 ([0, T 1 ])? An element of answer will be given in Section 3.5. An important property which helps understanding how the HOSP works is the following.

Notice first, that we may write dν i in ( 27) and (28) as

dν i = g i (t)dt + dJ i , (32)
where g i ∈ F ∞ (R + ; R) and dJ i is an atomic measure with countable and orderable set of atoms generated by the right continuous jump function J i . The following holds.

Proposition 2 ([1]

). Let z(•) be a solution of [START_REF] Dykhta | Optimal pulse control in models of economics and quatum electronics[END_REF] and [START_REF] Ferreira | Introduction to the Theory of Distributions[END_REF]. We have

g i (t) = 0, dt-a.e. t ∈ R + , ( 1
≤ i ≤ r -1), ( 33
) g r (t) ∈ -∂ψ T r-1 Φ ({z 1 }(t -),...,{z r-1 }(t -)) ({z r }(t + )), dt-a.e. t ∈ R + , ( 34
)
and

0 ≤ z 1 (t + ) ⊥ dν r (t + ) ≥ 0, for each t ∈ R + (35) 0 ≤ z 1 (t + ) ⊥ g r (t + ) ≥ 0, dt-a.e. t ∈ R + . ( 36
)
T i-1 Φ ({z 1 }(t -), . . . , {z i-1 }(t -)) {z i }(t + ) ⊥ -dν i ({t}) ∈ ∂ψ T i-1 Φ ({z 1 }(t -),...,{z i-1 }(t -)) ({z i }(t + )) (37)
for all 1 ≤ i ≤ r and all t ∈ R + .

Thus, the complementarity in (4)(c) is given a meaning with [START_REF] Moreau | Unilateral contact and dry friction in finite freedom dynamic[END_REF] and [START_REF] Moreau | Bounded variation in time[END_REF]. Notice that (36) may equivalently be rewritten as z 1 (t)g r (t)dt = 0, 0 ≤ z 1 (t + ), g r (t + ) ≥ 0. The cone CP in [START_REF] Moreau | Bounded variation in time[END_REF] shows what CP the measures dν i satisfy (the rigorous meaning of the inclusion in the right-hand-side of ( 37) is explained in [1, Proposition 1] or [31, p. 76]). It is important to notice that given {z 1 }(t -), . . . , {z i-1 }(t -), the measure value depends on {z i }(t + ). We shall come back on (37) in Section 4.2.

From [START_REF] Mcdanell | Necessary conditions for joning singular and nonsingular subarcs[END_REF] it follows that the non-atomic parts of the Lagrange multipliers are zero almost everywhere, except for g r (•).

Consequently the Lagrange multipliers associated to additional constraints to be imposed on the problem, are purely atomic.

This may explain why they have been ignored in the literature, since the condition that λ be a measure has always been imposed. It is crucial to keep in mind that the unilateral constraint Cx ≥ 0 of the original formulation, is replaced now by

{z i }(t + ) ∈ T i-1 Φ ({z 1 }(t -), . . . , {z i-1 }(t -)).
The definition of the cones and the inclusions (28) guarantee in particular that z 1 (t) = w(t) ≥ 0, see [START_REF] Moreau | Unilateral contact and dry friction in finite freedom dynamic[END_REF].

Remark 5. (i) Theorem 1 remains true if m ≥ 2 and C Ãr-1 B is a nonsingular symmetric M-matrix (see [START_REF] Cottle | The Linear Complementarity Problem[END_REF] for a definition) [START_REF] Acary | Higher order Moreau's sweeping process: Mathematical formulation and numerical simulation[END_REF].

(ii) The sets supp(dJ i ) are countable and orderable sets. In other words the set of times at which λ possesses atoms is countable and orderable.

(iii) If C x0 ≥ 0, then along the solutions of the HOSP IVP one has deg(w(

•)) = 0 on [0, T 1 ].
Another important tool is the following LCP which holds on time intervals [τ -, τ ), > 0, on which z 1 (t) is identically zero:

0 ≤ λ(t) ⊥ C Ãr W -1 z(t + ) + C Ãr-1 Bλ(t) ≥ 0. (38)
In [START_REF] Murray | Existence theorems for optimal control and calculus of variations problems where the state can jump[END_REF] one could write equivalently g r (t) instead of λ(t). This LCP monitors the evolution of the multiplier λ(•) for t ∈ [τ -, τ ). The matrix of this LCP is the leading Markov parameter M (2r wu ) . We therefore have at our disposal a complete dynamical system, which allows us to give a meaning to the system in (4), for any initial data and for all t ≥ 0. This is a first crucial step for the understanding of the necessary conditions of optimality.

Corollary 1. Along trajectories of (17)-( 19) one has d dt H({x}(t), {η}(t)) = 0 almost everywhere.

Proof. Follows from easy calculations and the fact that the support of the measures dJ i is of zero Lebesgue measure on

[0, T 1 ].
However, along the optimal trajectory the Hamiltonian may undergo jumps at atoms of λ (see Corollary 3).

Solutions of the necessary conditions IVP

From now on we will choose to embed the system in (4)(a) (c) in the HOSP. In particular, the degree of distributions refers to the degree of distributions in spaces T n ([0, T 1 ]) for some n. The following is going to be useful subsequently and follows from the state variable change z = W x and ( 17)-( 19) and ( 22)- [START_REF] Hartl | A survey of the maximum principles for optimal control problems with state constraints[END_REF].

Costate equation distributional formalism

Dη -Qx + A T η = [0 n×n I n ] W -1   0 r-1 M (r) λ 0 2n-r   . ( 39
)
Costate equation measure formalism [START_REF] Oberle | A program for the numerical solution of optimal control problems, Report no 515 der DFVLR[END_REF], while in [START_REF] Murty | Linear complementarity, linear and nonlinear programming[END_REF] x and η are distributions in T r-1 ([0, T 1 ]). We recall that Proposition 1 guarantees that there is a bijective relation between the solutions of the formalisms in ( 39) and [START_REF] Oberle | A program for the numerical solution of optimal control problems, Report no 515 der DFVLR[END_REF]. We

dη -Qx(t)dt + A T η(t)dt = [0 n×n I n ] W -1       dν 1 . . . dν r-1 M (r) dν r 0 2n-r       . ( 40
) Both x(•) and η(•) are F ∞ ([0, T 1 ], R)-functions in
also denote dν η = [0 n×n I n ] W -1 (dν 1 • • • dν r-1 ; M (r) dν r O 2n-r ) T .
In a first step, it is necessary to give a meaning to (4)(a) (c), as a well-posed dynamical system.

Lemma 3.

Let us embed the system in (4) (a) (c) in the framework of the HOSP. Assume that r wu = 2k, k ≥ 1 and m = 1. Then there exists a global solution starting from any initial data (x(0 -), η(0 -)) which is regular in the sense of Definition 1, and uniqueness holds in the set of regular solutions.

Proof. From Lemma 1, the leading Markov parameter M (2r wu ) = C Ã2r wu -1 B > 0. Apply Theorem 1 and the fact that the solution x of (4)(a) (c) is regular on R + for any initial data.

It is crucial to realise that Lemma 3 neither states uniqueness of solutions of the BVP (4)(a) (b) (c) when r wu is even, nor says that when r wu is odd, the BVP (4)(a) (b) (c) has no solution. The well-posedness of the IVP (4)(a) (c) and of the BVP (4)(a) (b) (c), are two very distinct notions. However studying a good formalism for the IVP, will allow us to better understand the BVP. In particular, the study of local properties of the optimal solution can take advantage of the IVP study, as well as for numerical integration. As pointed out in the introduction, the existence of solutions to the minimisation problem (1) and (2), is a reachability problem. Concerning the BVP in (4)(a)-(c), we make the choice to embed it into the HOSP, i.e. we allow for its solutions to be regular in the sense of Definition 1. The consequences of such a choice are important, as it may imply to extend the integral action I(u) in (1) since x, η and u may no longer be functions at contact times. Actually the central question is: let us allow the solutions of the BVP in (4) to be regular. Then how should one extend the minimisation problem in (1) and (2) so that the set of equations/conditions in ( 17)-( 19) represents the necessary optimality conditions?

Motivation example

Let us examine the simple case where the system (2) is given in the z-dynamics representation, Q = I n and R = I n u . Let us rewrite I(u) as the sum of a regular part 1 2

T 1 0 [{z}(t) T {z}(t) + 1 2 {u}(t) T R{u}(t)]dt and an algebraic part 1 2 k σ T {z} (t k )σ {z} (t k )
which takes into account the norm of the state vector jump. We notice that in addition, the minimisation of the algebraic part has to take into account some constraints on {z}(t + ), due to the unilateral constraints. In other words, we disregard what happens at instants t k , and retain only the right and left-limits of the function part of the state: we choose to work with the measure differential formalism ( 22)-( 24) or ( 25) and [START_REF] Jacobson | New necessary conditions of optimality for control problems with state-variable inequality constraints[END_REF]. Assume now that {z 1 }(t) = 0 and {z i }(t) < 0, for some t ∈ [0, T 1 ] and all 2 ≤ i ≤ r. Thus from ( 30) and (31) a jump in {z i }(•), 2 ≤ i ≤ r, occurs at t while the other variables are continuous.

Then we have {z

i }(t + ) -{z i }(t -) = argmin σ i +{z i }(t -)≥0 1 2 σ 2 i .
The HOSP jump rule therefore minimizes 1

2 k i (σ {z i } (t k )) 2 ,
over the admissible values of the post-impact state.

Meaning of the costate η(•) jump condition

In this subsection, we rely on the embedding of the system (4)(a)-(c) in the HOSP formalism to characterise both x and η as distributions. The condition η(τ + ) = η(τ -) -C T λ 1 with λ 1 ≥ 0 is usually given in the set of the necessary conditions. It is also sometimes indicated that if the optimal controller u is discontinuous, then a jump in η(•) occurs. In the light of the HOSP distributional and measure differential formalisms in ( 17)-( 21) and ( 22)-( 26) respectively, one may wonder what this jump condition really means3 : is λ a measure at τ (i.e., λ 1 is the magnitude of the atom of dJ r at τ , see [START_REF] Maurer | On optimal control problems with bounded state variables and control appearing linearly[END_REF]), or is it {η}(τ + ) = {η}(τ -) -C T λ 1 ? This is not at all equivalent. ≤ 0 and if we admit that the system in ( 4) is embedded in the HOSP, then the post-jump state is not well defined. Finally the jump in η(•) = {η}(•) has to satisfy stringent conditions to assure that only z r (•) jumps while the lower order variables z i (•) remain continuous.

Proposition 4. Let m = n u = 1. All the z i (•), 1 ≤ i ≤ r -1, are continuous at t, while z r (•) is a discontinuous function at t, if and only if (A T ) j σ η (t) ∈ Ker(B T ) for 0 ≤ j ≤ r wu -1.
Proof. Let us consider (4) and the state transformation in [START_REF] Çamlibel | Complementarity methods in the analysis of piecewise linear dynamical systems[END_REF]. It is easy to see that żi

(t) = z i+1 (t) = CA i x(t) for all 1 ≤ i ≤ r wu -1 when r wu ≥ 2. Let us assume that żi (t) = M i x(t) + i-r wu j=0 B T (A T ) j η(t), for some r wu ≤ i ≤ r -1,
where generically denotes some constant scalar and M i ∈ R 1×n is a row vector. Then żi+1 (t)

= M i (Ax(t) + BB T η(t)) + i-r wu j=0 B T (A T ) j (Qx(t) -A T η(t)) = M i+1 x(t) + i-r wu +1 j=0 B T (A T ) j η(t), where M i+1 = M i A + i-r wu j=0 B T (A T ) j Q (notice that M i B ∈ R). Since żr wu (t) = CA r wu x(t) + CA r wu -1 BB T η(t)
the proof is complete by induction with M r wu = CA r wu and = CA r wu -1 B. From Proposition 3 x(•) is continuous. The ''if '' part of the proof thus follows by letting j vary from 0 to ir wu and r wu ≤ i ≤ r -1. The ''only if'' part follows from the fact that the pair (A, B) is controllable. Therefore the rows B T (A T ) j are independent for all 0 ≤ j ≤ n -1. Since M i is an n-row and is a scalar, and since r wu ≤ n, the proof follows.

Corollary 2. Let η(•) have a jump σ η (t) at t and (A T ) j σ η (t) ∈ Ker(B T ) for 0 ≤ j ≤ r wu -1. Then u (r wu -1) (•) is the lowest order derivative of u(•) which is discontinuous at t.
Proof. From Proposition 4 it follows that deg(λ) ≤ 2 so that (4) does represent the optimality conditions for (1) (2) and ( 5) holds [START_REF] De Pinho | An Euler-Lagrange inclusion for optimal control problems with state constraints[END_REF]. Since 4 we deduce that σ u(t ) = -B T A T σ η (t) = 0. The reasoning can be continued until one attains u (r wu -1) , recalling that z (r-1)

u(•) = B T η(•), we deduce that σ u (t) = B T σ η (t) = 0. Since u(•) = B T η(•) = B T (Qx(t) -A T η(t)),
1 (•) = z r (•)
depends on u (r wu -1) (•).

One notices from Corollary 2 that the sum of the lowest derivative of u(•) that is discontinuous and of the relative degree are always equal to 2r wu -1 whenever there is a jump in η(•) at t (i.e. λ 1 > 0) and λ is a measure at t. This is in agreement with [START_REF] Jacobson | New necessary conditions of optimality for control problems with state-variable inequality constraints[END_REF]Theorem 6] which states that λ 1 = 0 if and only if u (r wu -1) (•) is discontinuous at t, at an entry time. However our proof completely differs from that in [START_REF] Jacobson | New necessary conditions of optimality for control problems with state-variable inequality constraints[END_REF].

Corollary 3. Let λ be a measure, and consider the Hamiltonian function in (9) evaluated along the solutions of the HOSP in (17)-( 19) (equivalently along the solutions of ( 22)-( 24)). Then H(t

+ ) = H(t -) at t ∈ [0, T 1 ] if and only if (η(t + ) - η(t -)) T Ax(t) = 0.
Proof. One finds from Proposition 3 that

H(t + )-H(t -) = -1 2 (η(t + )-η(t -)) T BB T (η(t + )-η(t -))+(η(t + )-η(t -)) T Ax(t). Thus from Proposition 4 one has H(t + ) -H(t -) = (η(t + ) -η(t -)) T Ax(t).
Therefore requiring the continuity of the Hamiltonian at contact states usually implies that deg(λ) ≤ 1. It is of interest to investigate whether or not conditions exist such that the Hamiltonian function, the optimal trajectory and the optimal control are functions of time (so that I(u) in (1) and ( 5) have a meaning), while λ is a distribution of degree ≥ 3. In a sense, the costate η should ''incorporate'' all the higher degree distributions. Let us denote the transformation matrix z = W x as

W = W1 0 n×n W2 W3
, where Wj ∈ R n×n , 1

≤ j ≤ 3, xT = (x T , η T ). Proposition 5. Let m = n u = 1 and r wu = n. Then (a) deg(λ) ≤ 2 ⇔ (b) z and x are functions ⇔ (c) B T η and A T η are functions. Also deg(λ) ≥ 3 ⇔ deg(B T η) ≥ 2 or deg(A T η) ≥ 2.
Proof. We first notice that under the stated conditions and from (4)(a), W possesses the above structure, and since it is full rank then necessarily W3 and W1 are full rank [28, p. 38]. Let us denote ζ T i = (z i , . . . , z r ), i ≥ r wu + 1. We first prove that (c) ⇒ (a). If z j (•), n + 1 ≤ j ≤ i -1 are functions and z i-1 (•) is discontinuous, then deg(λ) = r + 2 -i, and reciprocally. We notice that deg(λ) = r + 2 -i, i ≤ r -1, implies that deg(λ) ≥ 3. The vector ζ i may be considered as a (n -1)-vector of

distributions of degrees ≥ 2. Since η = -( W -1 3 W2 W -1 1 )z + W -1 3    z n+1 . . . z i-1 ζ i   , then B T η and A T η are time functions if and only if W3 -1 0 i-n+1
ζ i belongs to Ker(B T ) and to Ker(A T ). However since the pair (A, B) is controllable, the Kalman controllability matrix K and its transpose are square full rank

(n × n) matrices. Since K T =      B T B T A T B T (A T ) 2 . . . B T (A T ) n-1      , one sees that R n v ∈ Ker(B T )∩ Ker(A T ) implies that K T v = 0. Hence v = 0 and we deduce that ζ i = 0 which is a contradiction. Therefore deg(B T η) ≤ 1 and deg(A T η) ≤ 1 implies that deg(λ) ≤ 2.
The other implications/equivalences follow from the z-dynamics. The last equivalence is just a rewriting of (a) ⇔ (c).

Consequently one sees that the only case where B T η and A T η are functions, is when all z i (•), r wu +1 ≤ i ≤ r, are functions. In this latter case z r (•) may be time discontinuous, so that λ is a measure. However there may exist cases when deg(λ

) ≥ 3, deg(B T η) ≤ 1 and deg(A T η) ≥ 2. Proposition 6. Let m = n u = 1 and r wu = n (⇒ r = 2n).
Let us embed the necessary condition system in (4) in the HOSP in (17)- [START_REF] Ferreira | Introduction to the Theory of Distributions[END_REF]. Let a trajectory of (4) make contact with bdΦ at t = τ . Assume that z(τ -) = 0. Then deg(λ)

≤ n and x(•) is a continuous function at τ while deg(B T η) ≤ 1. Finally deg(A T η) ≤ n -1 at t.
Proof. Recall that z = (z 1 , . . . , z r wu ) T . We notice first that z(τ -) = 0 (=z(τ + ) from ( 28)-( 31)) is equivalent to having x(•) continuous at τ , because of the minimality of (A, B, C ) and consequently the structure of the transformation matrix W (since

r wu = n, the matrix W1 = [C T (CA) T • • • (CA n-1
) T ] T is the Kalman observability matrix). Using the z-dynamics ( 14) and ( 30) it is deduced from z(τ -) = 0 that deg(λ) ≤ r wu at τ and that deg(B T η) ≤ 1. The last statement is a consequence of the structure of à in (4).

It is worth recalling that all the distributions which appear in the problem, have an atomic part whose support is of zero Lebesgue measure, see Remark 5 (ii). As a consequence of Proposition 6, when z(τ -) = 0 at all entry times τ , the state jumps are jumps of {η}(•), i.e. {η}(t + k ) = {η}(tk ). But imposing the constraint z(τ -) = 0 at all entry times τ does not imply that λ is a measure.

Example 2.

Let us consider a triple integrator x (3)

(t) = u(t) with x 1 (t) ≥ 0. We have à =     0 1 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 1 1 0 0 0 0 0 0 1 0 -1 0 0 0 0 1 0 -1 0     , W =     1 0 0 0 0 0 0 1 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 1 0 0 1 0 -1 0 0 -1 0 1 0 1     , BT = 0 0 0 -1 0 0 , B T η = η 3 , A T η = 0 η 1 η 2 T . Assume that λ = δt , which happens if z 5 (•) jumps at t while z 6 (•) is continuous (for instance if z 1 (t -) = z 2 (t -) = z 3 (t -) = z 4 (t -) = 0, z 5 (t -) < 0, z 6 (t -) > 0). From the HOSP dynamics, this implies that z 1 (•), z 2 (•), z 3 (•), z 4 (•) and z 6 (•) are continuous at t, while z 5 (t + ) = 0. Thus deg(η 1 ) = 0, deg(η 2 ) =
(λ) = 3, necessarily either deg(B T η) ≥ 2 or deg(A T η) ≥ 2. Since deg(B T η) ≤ 1 necessarily deg(A T η) ≥ 2.
The preceding analysis suggests that there are two basic situations concerning the system in (4)(a) (c):

(1) λ is a measure, i.e. λ = g r (t)dt + dJ r , (2) deg(λ) ≥ 3 and

(2.1) B T η(•) and x(•) are time functions on [0, T 1 ], deg(A T η) ≥ 2, (2.2) deg(λ) ≥ 3, deg(B T η) ≥ 2.
Quantum electronics and laser control, portfolio optimisation, optimisation of the loan policy of a company, are problems which involve optimal controls with deg(u(•)) = 2 [START_REF] Dykhta | Optimal pulse control in models of economics and quatum electronics[END_REF]. Considering optimal inputs which are not functions, therefore, makes sense from the application point of view. There are several basic ingredients in the optimality conditions, among which: (i) the value of the optimal control in (5) which comes from ∂H ∂u (x, u, η) = 0, (ii) the costate equation η(t) = Qx(t) -A T η(t) -C T λ which comes from calculus of variations and Kuhn-Tucker conditions stating that ẋ(t) = J ∂H(x) ∂ x (t)

(see [START_REF] Bonnard | Optimal control with state constraints and the space shuttle re-entry problem[END_REF] and paragraph after), (iii) the expression of the integral action (or cost) in [START_REF] Acary | Higher order Moreau's sweeping process: Mathematical formulation and numerical simulation[END_REF].

In case (1) the three ingredients have a meaning and the BVP in (4) represents the optimality necessary conditions with I(u) in ( 1 19) and the function H(x, u, η) in [START_REF] Bonnard | Une approche géométrique de contrôle optimal de l'arc atmosphérique de la navette spatiale[END_REF].

Then the control which satisfies ∂H ∂u (x, u, η) = 0 at τ is given in [START_REF] Augustin | Second order sufficient conditions and sensitivity analysis for the optimal control of a container crane under state constraints[END_REF] if and only if (σ z n+1 (τ ), . . . , σ z r (τ ))

W -T 3 ∈ Ker(A). In this case deg(λ) ≤ 2.
Proof. From Proposition 6, one sees that deg(A T η) ≤ r wu -1 at τ . One may rewrite the Hamiltonian as

H(x, u, η) = -1 2 x T Qx -1 2 u T Ru + η T Bu + A T η, x since A T η ∈ T n-2 is a distribution. At τ we may write η as η = n-3 i=0 η i δ (i) τ for some η i .
We know that η = -W -1

3 W2 W -1 1 z + W -1 3   z n+1 . . . z r   . Since z(•) is a continuous function at τ , we deduce that η i = W -1 3 β i , with β i = 0 r-i-1 σ z r-i (τ ) 0 2n-r+i
(from ( 30) and ( 31) either

β i = 0 or β i = -z r-i (τ -) > 0). The result follows because deg(A T η) ≤ 1 if
and only if (β n-1 , . . . , β 0 ) T ∈ Ker(A T ) (otherwise the product A T η, x necessarily involves u(•) and its derivatives). The last result follows from Propositions 5 and 6.

In other words, the optimal input computed along the solution of the distributional problem ( 17)-( 19) is given by ( 5) at a tangential contact if and only if deg(λ) ≤ 2. When deg(λ) ≥ 3 one will have to resort to the measure differential formalism ( 22)-( 24) whose solutions are time functions in F ∞ to characterize the optimality, see Sections 3.5 and 3.6.

An extended integral action I(u)

Let us deal now with case (2), i.e. deg(λ) ≥ 3. The case (2.1) is of particular interest, because it shows that the optimal controller may be a function of time, while the costate η is a distribution of degree ≥ 3. Thus decreasing the ''regularity'' of u(•) while keeping deg(u) ≤ 1 may create serious difficulties in the analysis of this optimisation problem, the costate equation being a distributional differential equation as [START_REF] Murty | Linear complementarity, linear and nonlinear programming[END_REF]. On the sets supp(dJ i ) with dJ i in [START_REF] Maurer | On optimal control problems with bounded state variables and control appearing linearly[END_REF], the dynamics of the system in (4)(a) (c) becomes algebraic, see [START_REF] Malanovski | Sensitivity analysis fo optimal control problems subject to higher order state constraints[END_REF] and [START_REF] Monteiro Marques | Differential Inclusions in Nonsmooth Mechanical Problems, Shocks and Dry Friction[END_REF]. Consequently when deg(λ) ≥ 3 it is necessary that the optimal control problem in (1) and (2), involves an algebraic action in addition to an integral action. One path is the following: u 2 has no meaning if u = δ 0 . However δ 0 , ϕ(t) 2 = ϕ(0) 2 has a meaning for any test function ϕ(•) ∈ C ∞ 0 . Also

a 1 δ 0 + a 2 δ0 , ϕ 2 = a 2 1 δ 0 , ϕ 2 -2a 1 a 2 δ 0 , ϕ + φ + a 2 2 δ 0 , φ 2 = a 2 1 ϕ 2 (0) -2a 1 a 2 (ϕ(0) + φ(0)) + a 2 2 φ2 (
0). This way of introducing the singular distributions contributions in the quadratic cost, is different from that considered in [START_REF] Cobb | Descriptor variable systems and optimal state regulation[END_REF]. Let {δ n 0 (•)} be a fundamental sequence for the Dirac measure δ 0 [START_REF] Brogliato | Nonsmooth Mechanics[END_REF]Appendix A]. In [START_REF] Cobb | Descriptor variable systems and optimal state regulation[END_REF] it is argued that since +∞ 0 (δ n 0 ) 2 (t)dt → +∞ as n → +∞, then the cost to be associated to the distributional parts is +∞. It is concluded in that only those optimal control problems involving no distributions of degree ≥ 2 make sense. Our approach of considering the cost function (or action) when higher degree distributions are present in the optimal control problem, rather follows Moreau's results in [START_REF] Moreau | An expression of classical mechanics[END_REF]. An important tool in the developments which follow is the measure differential formalism of the HOSP in ( 22)- [START_REF] Jacobson | New necessary conditions of optimality for control problems with state-variable inequality constraints[END_REF].

Let us denote dν¯z = (dν 1 • • • dν r ) T ∈ R r .
Similarly for dJ z which denotes the atomic part of the vector measure dν z = g z (t)dt + dJ z (see [START_REF] Maurer | On optimal control problems with bounded state variables and control appearing linearly[END_REF]). Let us define

M = I r-1 0 r-1 0 r-1 M (r) ∈ R r×r , and M = M 0 r×(2n-r) 0 (2n-r)×r 0 (2n-r)×(2n-r) ∈ R 2n×2n .
From [START_REF] Kinzebulatov | Systems with distributions and viability theorem[END_REF] we have dz = Ãz(t 2 . In case {z i }(t -) < 0 and T i-1 Φ (z 1 (t -), . . . , {z i-1 }(t -)) = R + (which are the conditions required so that dJ i possesses an atom at t) the result follows. We have dJ¯z , ϕ T M dJ¯z , ϕ = r-1 i=1 dJ i , ϕ 2 + M (r) dJ r , ϕ 2 , and from Lemma 1 it follows that M (r) > 0. The result is proved.

)dt + M dν z 0 2n-r . Thus dν x = W -1 M dν z 0 2n-r = g x(t )dt + dJ x.

Proof. One has dν

i ({t}) = {z i }(t + ) -{z i }(t -) for all t ∈ [0, T 1 ]. If t is not an atom of dν i then dν i ({t}) = dJ i ({t}) = 0. If t is an atom of dν i this implies that all the tangent cones T k Φ ({z 1 }(t -), . . . , {z k }(t -)) satisfy T k Φ ({z 1 }(t -), . . . , {z k }(t -)) = R + for 0 ≤ k ≤ i -1. Then dν i ({t}) = dJ i ({t}) = argmin σ i +{z i }(t -)≥0 1 2 σ 2 i . Indeed from (30) it follows that {z i }(t + ) = prox[T i-1 Φ (z 1 (t -), . . . , {z i-1 }(t -)); {z i }(t -)]. We deduce that {z i }(t + ) = argmin σ ∈T i-1 Φ (z 1 (t -),...,{z i-1 }(t -)) 1 2 (σ -{z i }(t -))
Let us illustrate the second argument of the proof. If {z i }(t -) < 0 then argmin

σ i +{z i }(t -)≥0 1 2 σ 2 i = -{z i }(t -).
Since dν i ({t}) = {z i }(t + ) -{z i }(t -) we deduce that {z i }(t + ) = 0 (a plastic jump). The parallel with unilateral Mechanics and Moreau's second order sweeping process can be done, where the post-impact velocity satisfies q(t + ) = argmin w∈T Φ (q(t)) 1 2 (wq(t -) T )M(q(t))(wq(t -)) when the impacts are plastic [START_REF] Brogliato | Nonsmooth Mechanics[END_REF]Remark 5.11]. Lemma 4 extends to odd-r wu systems, provided dJ r = 0. We shall find again the specificity of odd-r wu systems in Sections 4.2 and 5. Let us propose the following extended action to be minimised:

minimize u(•)∈U 1 2 dJ x, ϕ T W T M W dJ x, ϕ + 1 2 T 1 0 [{x}(t) T Q {x}(t) + {u}(t) T {u}(t)]dt + 1 2 {x} T (T 1 )F {x}(T 1 ) (41)
for any test function ϕ(•)

∈ C 0 [R, R], with support containing [0, T 1 ], Definition
≤ 1 ⇒ deg(u) ≤ r wu + 1.
It is consequently useless to look for a larger set (in the set of distributions as defined in Section 3.1) of inputs for this optimal control problem. However as Example 7 shows it is easy to construct cases where x1 is not reachable from x0 in the set U r wu J . This is therefore an important issue that is not tackled in this paper. By taking u in U r wu J we assume that there is no need for the zero dynamics state ξ to jump in order for x1 to be reachable from x0 . Finally notice that {u}(•) ∈ L 2 ([0, T 1 ]; R n u ) since quadratic forms of LBV functions are LBV [START_REF] Moreau | Bounded variation in time[END_REF].

Let us notice that dJ x, ϕ T W T M W dJ x, ϕ contains terms of the form ϕ 2 (t k ) for all t k ∈ supp(dJ x), since dJ x is an atomic measure generated by a right continuous jump function J x(•). Let us recall that in [START_REF] Orlov | Schwartz' distributions in nonlinear setting: Applications to differential equations, filtering and optimal control[END_REF] the state jump times are totally free and are not a priori fixed. The only thing that we impose through the HOSP formalism is that the supports of the atomic measures dJ i are orderable and countable sets, and are in {t ∈ [0, T 1 ] : Cx(t) = 0} (even more: the atoms are in the set of junction times). It is noteworthy that even in case (2.1), the addition of an algebraic action is necessary. At a time t ∈ supp(dν i ) where (2.1) holds, then the value of {u}(•) = B T {η}(•) = B T η is irrelevant (and this is the case on the set of all atoms of λ as this set is of zero Lebesgue measure). The optimisation is done at such a t on the costate η. Remark 6. The idea of separating functions and distributions in the action is not new, see e.g. [START_REF] Murray | Existence theorems for optimal control and calculus of variations problems where the state can jump[END_REF] and [3, equ. (2.5)]. Also in [12, Section 3.7] I(u) is split into an integral part and an algebraic part taking into account state jumps. However the works [START_REF] Bryson | Applied Optimal Control; Optimization, Estimation, and Control[END_REF][START_REF] Murray | Existence theorems for optimal control and calculus of variations problems where the state can jump[END_REF] do not apply to the problem considered in this paper as they do not involve state unilateral constraints. Proposition 7. Suppose that x1 is reachable from x0 in a finite time T 1 ≥ 0, with u ∈ U r wu J . Let r wu be even. Then the HOSP measure differential inclusion in (22)-( 24) represents the optimality necessary conditions for the extended integral action I(u) in [START_REF] Orlov | Schwartz' distributions in nonlinear setting: Applications to differential equations, filtering and optimal control[END_REF], with u = B T η for all t ∈ [0, T 1 ] and subject to the constraints in [START_REF] Moreau | Unilateral contact and dry friction in finite freedom dynamic[END_REF] and [START_REF] Moreau | Bounded variation in time[END_REF]. If r wu is odd, then the result holds provided the additional constraint dJ r = 0 holds along the optimal trajectory. Proof. In view of the material of Sections 3.1, 3.2 and 3.4, one sees that if deg(λ) ≥ 2 an algebraic action has to be added to I({u}) in (1), taking into account the state {x}(•) and/or costate {η}(•) jumps (where {x} and {η} mean here the function part of the solution of the distributional formalism ( 17)-( 19)). From the measure differential formalism [START_REF] Oberle | A program for the numerical solution of optimal control problems, Report no 515 der DFVLR[END_REF]

, u = B T η for all t ∈ [0, T 1 ] means that u(t) = B T η(t) + B T [0,t) dν η , (42)
where dν η is defined after (40) and η(•) in ( 42) is to be understood as the solution of the ODE: η(t) -Qx(t) + A T η(t) = 0 with ẋ(t) -Ax(t) -BR -1 B T η(t) = 0. In other words (42) stems from

u(t) = argmax u∈U r wu J H x(t) + [0,t) dν x , η(t) + [0,t) dν η , u(t) (43)
and it is worthwhile noting that the arguments in the Hamiltonian function in [START_REF] Pereira | Necessary conditions of optimality for vector-valued impulsive control problems[END_REF], are functions of time (the fundamental role played by the measure differential formalism ( 22)-( 24) and Proposition 1 is clear in this context). Let us recall that solutions of the measure differential formalism are F ∞ functions, and so is u(•) in [START_REF] Paoli | A numerical scheme for impact problems with inelastic shocks: A convergence result in the multi-constraint case[END_REF]. From the distributional formalism [START_REF] Murty | Linear complementarity, linear and nonlinear programming[END_REF]

u = B T η means that u, ϕ = B T η, ϕ for all ϕ ∈ C ∞ 0 ([0, T 1 ]
). Therefore, if the necessary condition system (4) embedded in the distributional HOSP ( 17)- [START_REF] Dam | Unilaterally constrained dynamical systems[END_REF], possesses a distributional solution in T r-1 ([0, T 1 ]), the corresponding solution of the measure HOSP in ( 22)-( 24) decomposed as in [START_REF] Paoli | A numerical scheme for impact problems with inelastic shocks: A convergence result in the multi-constraint case[END_REF] into an unconstrained and a constrained parts, satisfies [START_REF] Paoli | A numerical scheme for impact problems with inelastic shocks: A convergence result in the multi-constraint case[END_REF] and ( 43) (by Proposition 1 there is a bijective correspondence between solutions of the distributional and the measure formalisms of the HOSP). The Hamiltonian function H({x}, {u}, {η}) evaluated along the optimal trajectory (and optimal control) satisfies

H({x}, {u}, {η}) = sup {v}∈U r J - 1 2 {x} T Q {x} - 1 2 {v} T R{v} + {η} T (A{x} + B{v}). ( 44
)
Lebesgue almost everywhere (equivalently {u}(•) = B T {η}(•) almost everywhere on [0, T 1 ]). Also the costate equation is

d dt ({η})(t + ) = Q {x}(t + ) -A T {η}(t + ) -C T (0 2n-1 g r (t + )) T (45)
on intervals (t k , t k+1 ] since the solutions are right-continuous. We recall that the values of the functions {x}(•), {u}(•), {η}(•) are irrelevant at the atoms t k of the distribution λ. Let us now deal with the algebraic action in [START_REF] Orlov | Schwartz' distributions in nonlinear setting: Applications to differential equations, filtering and optimal control[END_REF]. One has z = W x, and consequently dJ¯z , ϕ T M dJ¯z , ϕ

= dJ T z , ϕ M dJ z , ϕ = dJ x, ϕ T W T M W dJ x, ϕ . The solutions are distributions in T r-1 ([0, T 1 ]
), solutions of the HOSP in ( 17)- [START_REF] Dam | Unilaterally constrained dynamical systems[END_REF]. From Proposition 1 we can consider equivalently the solutions of the measure formalism in ( 22)- [START_REF] Hartl | A survey of the maximum principles for optimal control problems with state constraints[END_REF]. From Lemma 4, the solution of ( 22)-( 24) minimizes the quadratic term 1 2 dJ x, ϕ T W T M W dJ x, ϕ . All the above applies to even r wu , and to odd r wu , provided the constraint dJ r = 0 is added.

In case u(•) is a function, its derivatives may be distributions. This is taken into account by the algebraic action which is a function of η, hence implicitly of u(•) and its derivatives. Consider Example 2. One has at the considered time u(t) = η 3 (t). From ( 17) the distributional equality

Du = Dη 3 = x 3 -η 2 = z 3 -η 2 shows that deg(Du) = 2 since deg(η 2 ) = 2.
This is the consequence of a jump in z 5 (•) at t which is the optimal jump according to the action in [START_REF] Orlov | Schwartz' distributions in nonlinear setting: Applications to differential equations, filtering and optimal control[END_REF].

Most importantly, when λ is a measure, ( 42) and ( 43) coincide with [59, equ. (11.42)] and [59, equ. (11.41)], respectively.

We note however that the solution and optimal controller regularity conditions as studied in [START_REF] Vinter | Optimal Control, in: Systems and Control: Foundations and Applications[END_REF] are not relevant in our framework since the data in ( 1) and ( 2) are analytic and the solution we are looking for is regular (in the sense of Definition 1). Similarly [START_REF] Murty | Linear complementarity, linear and nonlinear programming[END_REF] and [START_REF] Oberle | A program for the numerical solution of optimal control problems, Report no 515 der DFVLR[END_REF] reduce to an equality of measures as in [8, equ. (2.21)] when λ is a measure.

It is worthwhile pointing out that in case (2.1), the integral term of the action I(u) is equal to

T 1 0 [x(t) T Qx(t) + u(t) T u(t)]dt + 1 2
x T (T 1 )Fx(T 1 ). However from Corollary 4 one does not have ∂H ∂u = 0 at atoms of dν i , 1 ≤ i ≤ r -1, but only almost everywhere on [0, T 1 ] (see also [START_REF] Paoli | A numerical scheme for impact problems with inelastic shocks: A convergence result in the multi-constraint case[END_REF]). The algebraic action is then equal to 1 2 (0 n dJ T η ), ϕ W T M W (0 n dJ T η ) T , ϕ , and takes into account possible jumps in the costate η(•).

ẋ1 (t) = x 2 (t) ẋ2 (t) = u(t) w(t) = l -x 1 (t) ≥ 0 (46) with Q = 0 and R = 1. Then (4) becomes in the z-canonical representation          ż1 (t) = z 2 (t) (= -x 2 (t)) ż2 (t) = z 3 (t) (= -η 2 (t)) ż3 (t) = z 4 (t) (= η 1 (t)) ż4 (t) = λ(t) 0 ≤ w(t) = l + z 1 (t), z 1 (t) = -x 1 (t). (47)
Let us consider the end-point conditions z 1 (0

-) = z 1 (T + 1 ) = -l and z 2 (0 -) = z 2 (T + 1 ) < 0.
Then the optimal solution is z(t) = 0 for all t ∈ (0, T 1 ) (so that u(t) = 0 for all t ∈ (0, T 1 ) and the integral action is zero along the optimal path), and z 2 (•) jumps at t = 0 and t = T 1 (here z 2 (•) is to be considered as a solution of the measure differential formalism ( 22)-( 24)). Thus dν 2 > 0 and dν 1 = dν 3 = dν 4 = 0. The algebraic action is equal to dν 2 , ϕ 2 . Along the solutions of the distributional formalism ( 17)-( 19) one has u = η 2 = -z 3 as an equality of distributions, and deg(u) = 2. The optimal solution that is presented in [START_REF] Bryson | Applied Optimal Control; Optimization, Estimation, and Control[END_REF]Section 3.11] is such that z 4 (t - 1 ) < 0 at an entry time t 1 , so that deg(λ) = 2. An open issue is to find whether or not admitting jumps in z 3 (•) = -η 2 (•) (solutions of ( 22)-( 24)), hence a discontinuous optimal controller, would allow one to decrease the action value compared to the case when λ is constrained to be a measure. In this case, the distributional formalism solution yields deg(λ) = 3. In this simple example, the test merely amounts to varying the entry and exit time, allowing for deg(λ) ≥ 3.

Hamilton's principle of Mechanics

Let us briefly expose here some facts about Hamilton's principle of Mechanics for Lagrangian systems subjected to complementarity relations and impact laws. As recalled in [11, Section 3.5], the addition of a unilateral constraint and of an impact law, implies that Hamilton's principle has to be modified in order to keep its meaning (otherwise it cannot represent the nonsmooth feature of the dynamics). The position is supposed to be constrained in Φ = {q | h(q(t)) ≥ 0} with h : R n → R, and we take T 1 = 1. The Lagrangian dynamics is embedded into Moreau's second order sweeping process [START_REF] Acary | Numerical Methods for Nonsmooth Dynamical Systems[END_REF]. Let us first consider the following minimization problem with e ∈ [0, 1]. minimize q(0)=q 0 ∈Φ q(1)=q 1 ∈Φ q(t + )+eq(t -)∈T Φ (q(t)) 1 2 σ q(t ) T M(q(t))σ q(t ) + 1 0 L(q(t), q(t))dt. [START_REF] Sannuti | Direct singular perturbation analysis of high-gain and cheap control problems[END_REF] The solution of the sweeping process on [0, 1] which satisfies q(0) = q 0 and q(1) = q 1 , minimises the action [START_REF] Sannuti | Direct singular perturbation analysis of high-gain and cheap control problems[END_REF] as it satisfies the smooth Euler-Lagrange dynamics outside impacts, and minimises the quadratic term at impacts. However the writing in [START_REF] Sannuti | Direct singular perturbation analysis of high-gain and cheap control problems[END_REF] is not satisfactory because it explicitly resorts to impact times in the formulation of the algebraic term of the action (notice that if t is not an impact time then the corresponding algebraic action becomes trivial). Let us define the Stieltjes measure dv associated to the acceleration, whose atoms are the impact times so that dJ v ({t}) = q(t + ) -q(t -).

Then one may rewrite (48) as minimize q(0)=q 0 ∈Φ q(1)=q 1 ∈Φ q(t + )+eq(t -)∈T Φ (q(t))

1 2 dJ v , ϕ T M(q(t)) dJ v , ϕ + 1 0
L(q(t), q(t))dt [START_REF] Sannuti | Multipe time-scale decomposition in cheap control problems -Singular control[END_REF] with ϕ(•) ∈ C 0 a test function whose support contains [0, 1]. As shown in [START_REF] Buttazzo | On the approximation of the elastic bounce problem on Riemannian manifolds[END_REF] for the case e = 1, the extremal of the functional 1 0 (L(q(t), q(t))dt + h(q(t))λ) [START_REF] Sannuti | Special coordinate basis for multivariable linear systems -Finite and infinite zero structure, suqaring down and decoupling[END_REF] under the constraints h(q(t)) ≥ 0 for all t ∈ [0, 1], some initial data and no kinetic energy loss, is a solution to the bounce problem, i.e. d dt ∂L(q,q) ∂ q (t) -∂L(q,q) ∂q (t) = ∇h(q(t))λ on [0, 1], and vice-versa. The multiplier λ in ( 50) is a measure whose support is in the set {t ∈ [0, 1] | h(q(t)) = 0}.

One sees that in [START_REF] Sannuti | Special coordinate basis for multivariable linear systems -Finite and infinite zero structure, suqaring down and decoupling[END_REF] the Lagrangian function is augmented to cope with the unilaterality, and the multiplier is a measure.

The basic idea is to augment the Lagrangian function L(x, u) = 1 2 x T Qx + 1 2 u T Ru, taking into account the specific features of the problem. Let us recall that dν¯z = (dν

1 • • • dν r ) T , dν x = W -1 M dν z 0 2n-r
, and dx -Ãx(t)dtdν x = 0 from ( 22)- [START_REF] Hartl | A survey of the maximum principles for optimal control problems with state constraints[END_REF]. We define the augmented Lagrangian as L({x}, {u}, dν x, dt) = L({x}, {u})dt + {x}(t + ) T Jdν x where J ∈ R 2n×2n is defined in Remark 1. The product {x}(t + ) T Jdν x is written here with a strong abuse of notation and requires some care. The function {x}(.) is right-continuous and dν x is a measure with possible atoms at times t k , k ≥ 0. The space of functions which are dν x-integrable contains functions continuous at t k , and also the functions which are dν x-almost everywhere equal to an integrable and continuous function g(.). Since the supports of the atoms are the singletons {t k }, it is sufficient that {x}(t k ) = g(t k ). Then, denoting the atoms of dν x as δ t k one has

{x}dδ t k = gdδ t k = g(t k ) = {x}(t k ) = {x}(t + k ). (51)
Equality [START_REF] Sattayatham | Strongly nonlinear impulsive evolution equations and optimal control[END_REF] shows that L({x}, {u}, dν x, dt) is meaningful. We therefore propose an alternative to Proposition 7.

Proposition 8. Suppose that x1 is reachable from x0 in a finite time T 1 ≥ 0, with u ∈ U r wu J . Let r wu be even. The HOSP measure differential formalism in (22)-( 24) represents necessary conditions satisfied by the extremals of the functional

I(u) = T 1 0 L({x}, {u}, dν x, dt) + 1 2 {x}(T 1 ) T F {x}(T 1 ) (52)
where the measures dν i satisfy the inclusions (28) and u = B T η.

Proof. Let us consider the augmented Hamiltonian

H({x}, {u}, ν x) = -L({x}, {u}) + {x}(t + ) T Jν x + {η} T (A{x} + B{u}). [START_REF] Schumacher | Complementarity systems in optimization[END_REF] It is noteworthy that in the framework of ( 22)-( 24) one can drop all brackets as {x}(•) = x(•), because the solution is considered as a function in F ∞ ([0, T 1 ]; R). Since the set of controllers is U r wu J and u = B T η (whose meaning is as in ( 42)), the support of the measure dν x is orderable and countable. Mimicking [13, Remark 1.1] and using the augmented Hamiltonian in [START_REF] Schumacher | Complementarity systems in optimization[END_REF], we deduce that the extremals of the functional I(u) satisfy dx = Jd H, i.e.:

         dx = ∂ ∂η -L(x, u) (t)dt + Ax(t)dt + Bu(t)dt = Ax(t)dt + Bu(t)dt + dν x dη = ∂ ∂x L(x, u) (t)dt -A T η(t)dt = Qx(t)dt -A T η(t)dt + dν η u = R -1 B T η (54)
as an equality of differential measures for the first two lines of [START_REF] Silva | Necessary conditions for optimal impulsive control problems[END_REF]. The third equality has the meaning explained in the proof of Proposition 7 (see [START_REF] Paoli | A numerical scheme for impact problems with inelastic shocks: A convergence result in the multi-constraint case[END_REF]) and allows one to calculate the optimal input using Proposition 1. One sees that ( 54) is the measure differential formalism ( 22)- [START_REF] Hartl | A survey of the maximum principles for optimal control problems with state constraints[END_REF]. One therefore has along solutions of the measure differential inclusion ( 54)

u(t) = argmax u∈F ∞ H(x, u, dν x, dt) Lebesgue almost everywhere (notice that if u ∈ U r wu J then {u}(•) ∈ F ∞ ).
It is noteworthy that if a solution of the measure differential formalism is a solution of the minimisation problem in [START_REF] Van Der Schaft | An Introduction to Hybrid Dynamical Systems[END_REF], then it satisfies [START_REF] Silva | Necessary conditions for optimal impulsive control problems[END_REF], and equivalently from Proposition 1 the corresponding solution of the distributional formalism will satisfy ( 44) and ( 45) Lebesgue a.e. on [0, T 1 ]. We have therefore extended the measure differential inclusion necessary conditions in ( 4) and ( 6) to the following BVP (named a Measure BVP)

MBVP :

-dx + Ãx(t)dt + dν x = 0

dν x = W -1 M dν¯z 0 2n-r dν¯z = (dν 1 • • • dν r ) T dν i ∈ -∂ψ T i-1 Φ (z 1 (t -),...,z i-1 (t -)) (z i (t + )) for all 1 ≤ i ≤ r z = W x x(0) = x0 , η(T 1 ) = Fx(T 1 ) + C T γ + β = F x1 + C T γ + β = η 1
which reduces to (4) and ( 6) when λ is a measure (see [START_REF] Moreau | Unilateral contact and dry friction in finite freedom dynamic[END_REF]). Let us recall that if x(T 1 ) is not specified and F = 0 then η(T 1 ) = 0, and that both x and z are time functions since they are the solution of the measure differential formalism of the HOSP in ( 22)-( 24). This will permit to settle a time discretisation of MBVP (see Section 6). It is also noteworthy to recall that the atoms of all the distributions of degree ≥ 2 belong to the set of junction times in the setting developed in this paper.

Remark 7.

Comparing the ''optimality'' of the optimal trajectories obtained with deg(λ) ≤ 2, and of those obtained with deg(λ) ≥ 3, is not easy because the two problems involve different cost functions. Depending on the application, one may anyway want to compare the cost of the problem with deg(λ) ≤ 2, and the continuous part of the cost with deg(λ) ≥ 3.

Behaviour of the optimal trajectories at junctions with bdΦ

There are two major points: what happens at an entry time, and what happens after an entry time. In this section we use the partitioning of bdΦ as proposed in [START_REF] Dam | The contact problem for linear continuous-time dynamical systems: A geometric approach[END_REF][START_REF] Dam | Unilaterally constrained dynamical systems[END_REF]. More precisely, we study the qualitative behaviour of optimal solutions which attain bdΦ, and possibly leave it or remain on it. The control input u(•) in ( 2) is assumed to be piecewise smooth (a piecewise C ∞ ([0, T 1 ]; R) time function). Roughly, the process consists of subdivising bdΦ into subsets and especially trying to select that portion of bdΦ such that no input u(•) exists that can keep an entry state x(τ ) inside Φ. Then surely an optimal trajectory which possesses a boundary arc, cannot pass through this x(τ ).

The admissible domain boundary partitioning

The following assumptions are in order in Sections 4.1-4.3:

(i) The pair (A, B) is controllable, (ii) Im(B) ⊆ Ker(C ), (iii) D = 0, C = 0 and m = 1, (iv) Φ = ∅.
One sees that (ii) implies that CB = 0, i.e. r wu ≥ 2. The case r wu = 1 will be treated, but deserves special attention. We restrict ourselves to the case D = 0, because as shown in [18, Section VII] it is always possible to transform the dynamics to recover this case. The subsets of bdΦ are invariant under linear state feedback. Three sets are of interest here: χ con the subset of bdΦ at which trajectories coming from Int(Φ) can make contact with bdΦ, χ rel the subset of bdΦ at which trajectories starting on bdΦ can leave bdΦ and stay on a positive time interval in Φ, and ν * the subset of bdΦ such that there exists one u(•) = u * (•) that can keep (locally) the trajectory on bdΦ (ν * is the largest controlled invariant subspace in bdΦ). Trajectories that make contact with bdΦ in χ con \ ν * , leave Φ. Let

r(x, u) : bdΦ × U → N ∪ {+∞} and r(x, u) = min{i ∈ N | w (i) (x, u) = 0} with r(x, u) = +∞ if w (i) (x, u) = 0 for all i ∈ N,
where U is the set of piecewise smooth inputs. According to the framework developed in Section 3, we are looking for a solution that is regular in the sense of Definition 1. Thus x(•) is analytic on [0, T 1 ] \ {t k }, and so is the optimal control u(•). Moreover the times at which the solution is not analytic are the jump times t k ∈ supp(λ) and are junction times. But left accumulations of state z jumps may occur. As a consequence, to derive most of the results on the qualitative behaviour of optimal trajectories at an entry time τ , one has to make the additional assumption that there is no left accumulation at τ (such accumulations of touch times on the left of an entry time may however exist ni some optimal trajectories, see [START_REF] Robbins | Junction phenomena for optimal control with state variable inequality of third order[END_REF]). In view of Proposition 7, this is equivalent to assuming that the set of admissible controls is restricted to piecewise analytic functions {u}(•) (i.e. the support of the atomic part of λ is finite) and that reachability holds within this set. Such an assumption is made for instance in [START_REF] Mcdanell | Necessary conditions for joning singular and nonsingular subarcs[END_REF]Theorem 1] (in a different context, though), where junction states are then called analytic junctions. Finally most of the results stated below hold for m ≥ 2, but assuming that there is only one constraint that is active at the considered time.

Lemma 5 ([18]). The following holds.

• χ con = {x ∈ bdΦ | ∃u ∈ U such that {r(x, u) < +∞ and even, and w (r(x,u)) > 0}, or {r(x, u) < +∞ and odd, and w (r(x,u)) < 0}},

• χ rel = {x ∈ bdΦ | ∃u ∈ U such that r(x, u) < +∞ and w (r(x,u)) > 0} • ν * = {x ∈ bdΦ | ∃u ∈ U such that r(x, u) = +∞}.
With some abuse of notation u in r(x, u), means that the involved functions depend also on the derivatives of u. The following result is obvious from the definition of the three subsets of bdΦ. Proposition 9. Boundary arcs of optimal trajectories exist only in the subset ν * of bdΦ. Contact states exist only in χ con , and exit states exist only in χ rel . Entry states satisfy z(t -) ∈ χ con and z(t + ) ∈ ν * . Touch states satisfy z(t -) ∈ χ con and z(t + ) ∈ χ rel . This shows that an entry time may exist in χ con \ ν * only if u(•) is not smooth (i.e. not infinitely differentiable), since there must exist a higher derivative that jumps to zero at the entry time. These results say nothing on the relationships between the regularity of the control and the nature of the state (contact, touch, entry). More details are given below. As the following example shows it may even be necessary to apply a distributional input.

Example 4. Let us choose

A = 0 1 0 0 , B = 0 1 , C = (1 0). Then χ con = {x|x 1 = 0, x 2 ≤ 0}, χ rel = {x|x 1 = 0, x 2 ≥ 0}, and ν * = {x|x 1 = x 2 = 0}. If u(•)
is restricted to be a function everywhere, all trajectories entering bdΦ in χ con \ ν * leave Φ. All trajectories initialised in χ rel \ ν * enter Int(Φ). Thus boundary arcs are restricted to ν * and u * = 0. If an entry time τ exists in χ con \ ν * , then it is necessary to apply a Dirac measure u τ to keep the state in Φ and make it jump in χ rel .

Let us notice that the case x 2 (•) ≥ 0 (i.e. C = (0

= Ker(C ) ∩ A -1 (ν k + Im(B)) = {x ∈ ν k | CA k x = 0}
converges towards ν * in at most r wu steps. 5 This is extremely important in view of the development of a numerical analysis of the problem of interest here.

Entry and contact times and states

In this subsection, we essentially focus on the qualitative analysis of optimal trajectories and control at entry and contact times, when m = n u = 1. We denote u * the control inside ν * .

(τ , τ + ), ≥ 0, then u * (t) = 0 on (τ , τ + ). (ii) If r wu < n then ν * = {x * (t) = W -1 0 r wu ξ (t) } = {z * (t) | z * = 0}, ξ (t) = A ξ ξ (t), and u * (t) = -(CA r wu -1 B) -1 CA r wu W -1 0 r wu ξ (t)
.

Proof. (i)

From the definition of ν * , given a state trajectory x(t) such that Cx(t) = 0 on a positive time interval, one has to look for a control u * (•) that makes w (i) (•) equal to 0 for all integers i ≥ 0. On (τ , τ + ) one has z (i)

1 = w (i) = 0. Since r wu
= n, z = z and thus ν *

x * = 0 since z = Wx with W full rank. Since w (i) (t) = CA i x(t) + i j=1 CA j-1 Bu * (i-j) (t), for all i ≥ 1, we deduce that on (τ , τ + ) one has 0 = i j=1 CA j-1 Bu * (i-j) (t) = i j=r wu CA j-1 Bu (i-j) for all i ≥ r wu . Since m = n u = 1 it follows that all the terms CA i B are scalars and u * = 0. (ii) is proved from the z-dynamics and the fact that ν * = lim k→r wu ν k , with ν k+1 = {x ∈ ν k | CA k x = 0} [START_REF] Dam | The contact problem for linear continuous-time dynamical systems: A geometric approach[END_REF].

When assumption (ii) is not satisfied, r wu

= 0) satisfies x un (t) ∈ Int(Φ) for all t ∈ [0, T 1 ] \ {t k } 1≤k≤l , l < +∞, and x un (t k ) ∈ ν * , then the solution x(•) of (4) is equal to x un (•) if u(•) is restricted to be a smooth time function on [0, T 1 ].
Proof. We know from Proposition 9 that boundary arcs and contact states belong to ν * , and from Proposition 10 that on boundary arcs and contact states x = 0 and u = 0. Any other state trajectory linking x0 to x1 in Φ with a smooth input u , will yield an integral action I(u ) > I(u).

If D = 0, one may recover the previous case by a simple transformation, however the new pair ( Ā, B) is never controllable. The subset ν * can nevertheless be computed [19, Section 6.7] [18, Section VII], as the following example shows. Proof. Since ν * = bdΦ = {z 1 = 0}, it is obvious from the z-dynamics of (A, B, C ) that a discontinuous u(•) is sufficient to keep x(•) in Φ. From [START_REF] Çamlibel | Complementarity methods in the analysis of piecewise linear dynamical systems[END_REF], which is the z-dynamics of ( Ã, B, C), we deduce that λ is a Dirac measure. Indeed r = 2 and z 1 (•) = {z 1 }(•) is a function, and its derivative may be discontinuous at an entry time τ . Thus Dz 2 is a Dirac measure at τ , and so is λ. We note also that x(•) is continuous at τ , so that η(•) (which is a function) ''concentrates'' all the state x jumps. Proposition 11 will be refined when we examine what happens along optimal trajectories for odd-r wu systems, see Corollary 10. Proposition 11 suggests that if assumption ii) is true (i.e. r wu ≥ 2), then at entry times in χ con \ν * a distributional input with degree ≥ 2 will be needed to keep x(•) inside Φ. This is a little more subtle. The following results show that trajectories attain bdΦ in a specific way.

The sets inside χ con \ ν * such that the first nonzero derivative of w(•) = z 1 (•) is of order 2k + 1 are denoted as

χ 2k+1 con = {z ∈ R n , z ≺ 0 | z (i) 1 = 0 for all 0 ≤ i ≤ 2k, w (2k+1) = z (2k+1) 1 < 0}.
In the same way

χ 2k con = {z ∈ R n , z 0 | z (i) 1 = 0 for all 0 ≤ i ≤ 2k -1, w (2k) = z (2k) 1 > 0} and χ 2k+1 rel = {z ∈ R n , z 0 | z (i) 1 = 0 for all 0 ≤ i ≤ 2k, z (2k+1) 1 > 0}.
From z (i)

1 (t) = CA i x(t) + i-1 k=r wu -1 CA k Bu (i-1-k) (t)
one sees that these sets may depend on u(•) and its derivatives. A trajectory which attains bdΦ in χ 2k con detaches immediately from bdΦ if the control u(•) is smooth, since the first nonzero derivative will keep its positive sign in a right-neighborhood of the contact time (hence it is a touch state). It is noteworthy that a trajectory which comes from Int(Φ) and enters χ 2k+1 con at τ satisfies z (2j) 1 (τ -) ≥ 0 and z (2j-1) 1 (τ -) ≤ 0 for all 0 ≤ j ≤ k. There is a sign inversion at each differentiation order due to the left analyticity in the neighborhood of τ and the fact that all derivatives are zero at τ up to the order 2k. An illustrative example with r wu = 3 can be found in [START_REF] Brogliato | Nonsmooth Mechanics[END_REF]Example 1.8]. A trajectory that leaves bdΦ at τ after a boundary arc has to do it with a non-analytic z 1 (•) (otherwise by a simple Taylor expansion it follows that the trajectory remains in ν * ) and such that the first non-zero derivative of z 1 (•) is positive, i.e. in a set χ 2k+1 rel . Corollary 6. Let a contact time τ ∈ [0, T 1 ] be such that x(τ ) ∈ χ 2k+1 con and x(•) ∈ Φ in a right neighborhood of τ . Then if 2k + 2 ≤ r wu , k ≥ 0, u has to be a distribution of degree 2 + r wu -2k with an atom at τ . If 2k + 2 > r wu then u(•) is a function of time in a neighborhood of τ , with its first nonzero derivative that is of order 2k + 1 -r wu and is discontinuous at τ . Proof. From the z-dynamics, one has that z (r wu -1)

1 (•) = z r wu (•) depends explicitly on u(•). In particular z (2k+1) 1 (•) = ż2k+1 (•) = z 2k+2 (•)
and is a function of u (2k+1-r wu ) (•), which has then to be discontinuous to keep the trajectory inside Φ.

The result of Corollary 6 will be refined next depending on r wu . In most of the studies (see e.g. [START_REF] Hamilton | On nonexistence of boundary arcs in control problems with bounded state variables[END_REF][START_REF] Jacobson | New necessary conditions of optimality for control problems with state-variable inequality constraints[END_REF][START_REF] Bonnard | Optimal control with state constraints and the space shuttle re-entry problem[END_REF][START_REF] Bryson | Applied Optimal Control; Optimization, Estimation, and Control[END_REF]) the constraint that the contact with bdΦ occurs ''tangentially'' is added. This means simply that at a time τ at which the optimal trajectory attains bdΦ, then z(τ -) = 0, where we take the left limit as it could be that jumps occur (see Proposition 6).

Definition 3. A contact state at time τ ∈ [0, T 1 ] is said tangential if z(τ -) = 0, and hypertangential if z(τ -) = 0.
The meaning of the overbar is given in the paragraph after [START_REF] Çamlibel | Complementarity methods in the analysis of piecewise linear dynamical systems[END_REF]. If the contact is hypertangential, it follows from ( 30) and

(31) that z(τ + ) = 0 (which does not mean that z(τ + ) ∈ ν * ). An hypertangential contact necessarily occurs in a set χ i con with i ≥ r. If the contact states are forced to be hypertangential, then dJ x = 0 and the action in ( 41) is purely integral.

Proposition 12. Let r wu = 2k, k ≥ 1, deg(λ) ≤ 2, and i ≥ 0. Then all optimal trajectories which make contact with bdΦ at t = τ in the set χ r wu

+2i con are touch states if u (2i) (•) is a continuous function at t = τ . If contact occurs in χ r wu +2i+1 con , then necessarily i ≥ k -1 and necessarily u (2i+1) (•) is discontinuous at τ . Proof. If contact occurs in χ r wu +2i con , then z (r wu +2i) 1 > 0. If u (2i) (•) is a continuous function at t = τ , so are all u (j) (•) with 0 ≤ j ≤ 2i -1. Therefore from z (r wu +2i) 1 = CA r wu +2i BW -1 z + 2i k=0 CA r wu -1-k+2i Bu (k) , we deduce that z (r wu +2i) 1 (•) is a continuous function at τ . From the definition of χ r wu +2i con , it results that in an open right neighbourhood of τ , z 1 (t) > 0. If contact occurs in χ r wu +2i+1 con , then z (r wu +2i+1) 1 < 0. If u (j) (•), 0 ≤ j ≤ 2i + 1, are continuous at τ , then in an open right neighbourhood of τ , z 1 (t) < 0. Also deg(λ) ≤ 2 implies that deg(u (r wu ) ) ≤ 2. If i ≤ k -2,
then the trajectory escapes from Φ in an open right neighbourhood of τ , because the lower order derivatives of u(•) must be continuous at τ , and so is

z (r wu +2i+1) 1 (•). If i ≥ k -1,
the degree condition on λ can be respected even if u (2i+1) (•) is discontinuous at τ . Moreover if u (2i+1) (•) is continuous at τ then the trajectory escapes from Φ in an open right neighbourhood of τ .

It is noteworthy that u (2i+1) (•) be discontinuous at τ when i ≥ k -1 is not only a necessary condition, but this is also sufficient. Indeed the conditions secure that (4) does represent the optimal conditions for (1) and (2), and the positivity of M (2r wu ) (see Lemma 1) assures that Lemma 3 holds. The case r wu = 2k + 1 can be analysed similarly. However it requires more care as Proposition 13 and Corollary 7 show. The case r wu = 1 is treated in Proposition 11 (since then ν * = bdΦ we get that χ con \ ν * = ∅ so that the condition of Proposition 12 degenerates). The next step is to study optimal trajectories which make contact with bdΦ in χ con \ ν * , and then are sent into ν * (where possibly a boundary arc exists). Proposition 13. Let m = n u = 1. Let an optimal trajectory make contact with bdΦ at t = τ , in the subset χ 2r wu -1 con , and be such that z(τ + ) ∈ ν * (thus τ is an entry time). Then necessarily r wu is even. The same holds if z(τ + ) ∈ χ 2r wu -1 rel .

Proof. From the z-dynamics and Lemma 1, it follows that if λ is a measure at a time t k , then dJ r has an atom λ τ = (-1) r wu (CA

r wu -1 B) -2 (z r (τ + ) -z r (τ -))δ τ . If z(τ + ) ∈ ν * , then z r (τ + ) = 0 from the definition of ν * . Now since contact is made in χ 2r wu -1 con we have that z (r-1) 1 (τ -) (= z r (τ -)) < 0, whereas z(τ + ) ∈ ν * implies that z (r-1) 1 (τ + ) = 0. Therefore if r wu
= 2k + 1 for some k ≥ 0, λ cannot be a positive measure. The last point can be proved similarly since then z r (τ + ) > 0.

The proof is complete.

Let us notice that under the conditions of Proposition 13, a jump in z r (•) is equivalent to a jump in u (r wu -1) (•), since the derivatives of smaller order of u(•) are time-continuous functions. Proposition 13 says that under the stated conditions (which in particular imply that deg(λ) = 2 since z r (•) jumps at τ ), only systems with an even relative degree possess entry states, which is in accordance with the result of [START_REF] Jacobson | New necessary conditions of optimality for control problems with state-variable inequality constraints[END_REF]. The second point of Proposition 13 implies in particular that odd-r wu systems do not even possess touch points with z(τ -) ∈ χ 2r wu -1 con and z(τ + ) ∈ χ 2r wu -1 rel . Consequently when deg(λ) = 2, it is excluded to get an entry time that is the accumulation of touch points. Proposition 13 concerns conditions under which λ is a singular measure. If the contact occurs in a set χ i con with i ≥ r, then z(τ -) = 0 and the problem of existence of a boundary arc is equivalent to the well-posedness of an LCP (see Section 5). The next Corollary is a consequence of Proposition 13.

Corollary 7. Let m = 1, λ be a measure and {η}(τ + ) -{η}(τ -) = -C T λ 1 . If λ 1 > 0, then r wu is an even integer.

Compiling Corollary 7 and Proposition 11, one deduces that if r wu = 1 and if u(•) has a discontinuity at an entry time τ , then necessarily λ 1 = 0, i.e. the costate η(•) is continuous at τ . Hence we retrieve the result of [9, Lemma 2.4 (2)]. Also from Proposition 4 and Corollaries 2 and 7, it follows that if (A T ) j σ η (τ ) ∈ Ker(B T ) for 0 ≤ j ≤ r wu -2 and u r wu -1 (•) jumps at τ , then r wu is even. Notice that if deg(λ) = 2, then contact cannot occur in a set χ 2k con with k < r wu , since r is even. If z(τ -) ∈ χ 2k con , k < r wu , and z(τ + ) ∈ ν * , then deg(λ) ≥ 3.

Proposition 14. Let m = n u = 1, and r wu

= 2k + 1, k ≥ 0. Then z(τ -) is an entry state, only if deg(λ) ≤ 1 or deg(λ) ≥ 3.
Proof. One has z(τ + ) ∈ ν * , hence z (i) 1 (τ + ) = 0 for all i ≥ 0. The fact that z(τ -) is an entry state means that z(τ -) ∈ χ con , so that z 1 (τ ) = 0, and the first nonzero z (i) 1 (τ -) is either > 0 or < 0. The higher order derivatives satisfy z (2k)

1 (τ -) ≥ 0 and z (2k+1) 1 (τ -) ≤ 0.
It follows that at t = τ the measures dJ i in (32) satisfy dJ 2k ≥ 0 and dJ 2k+1 = 0. Proposition 13 says that if deg(λ) = 2 at τ (i.e. dJ r > 0), then r wu is even. A first situation is when z (i) 1 (τ -) = 0 for some i < r -1. Depending on the values of the derivatives it is possible that dJ 2k > 0 for k < r wu and that dJ r = 0 and deg(λ) ≥ 3. A second situation is when the contact is hypertangential (see Definition 3), so that dJ i = 0 for all 1 ≤ i ≤ r (in this last case the trajectory may either detach from the constraint, or stay on it with g r (t) > 0 on some nonzero time interval, where g r (•) is in [START_REF] Maurer | On optimal control problems with bounded state variables and control appearing linearly[END_REF]). Then deg(λ) ≤ 1, i.e. λ is a function.

As an example, let us consider r wu

= 3. If z 1 (τ -) = 0, z 2 (τ -) < 0, and z 3 (τ -) = z 3 (τ -) = z 3 (τ -) = z 3 (τ -) > 0, i.e. z(τ -) ∈ χ 0 con , then dJ 2 = z 2 (τ + ) -z 2 (τ -) = -z 2 (τ -
) > 0 while all other dJ i have no atom at τ . Thus deg(λ) = 6. Another type of approach is detailed in Example 2, where this time deg(λ) = 3. It immediately follows from Proposition 14 that if λ is restricted to be a measure, then systems with r wu = 2k + 1 possess optimal trajectories such that z and consequently x are time continuous, because deg(λ) ≤ 1. It is important to recall that there may exist trajectories of (4) with x(•) and B T η(•) continuous functions, and with deg(λ) ≥ 3, see Proposition 6 and Example 2. Once z(τ + ) ∈ ν * , then the existence of a boundary arc relies upon the well-posedness of a LCP whose solution is λ(t), t > τ , see Section 5. But if r wu = 2k, k ≥ 1, nothing hampers an optimal trajectory to possess left accumulations of state jumps, when λ is a measure. In any case, left accumulations at an entry state can exist in the state variables {z i }(•) with i ≤ r -1. Proof. The reachability assumption is fundamental, as Example 7 in the appendix shows. Since it holds one can search for optimal inputs which are functions, see [START_REF] Cobb | Descriptor variable systems and optimal state regulation[END_REF]. From Proposition 14 it follows that deg(λ) is either ≤ 1 or ≥ 3. From ( 14) it follows that deg(λ) ≥ 3 ⇒ deg(z 2 ) ≥ 2. This is in contradiction with the choice of admissible inputs U. Thus deg(λ) ≤ 1 so that both x(•) and η(•) are time-continuous on [0, T 1 ].

This result is consistent with the fact that the HOSP solutions are in T 1 ([0, T 1 ]) for r = 2, which implies that deg(z) ≤ 2, hence deg(λ) ≤ 3. As Example 7 demonstrates, a necessary condition for the framework that is developed in this paper around the HOSP, is that the zero dynamics ξ (t) = A ξ ξ (t) + B ξ z 1 (t) be reachable from ξ0 to ξ1 with z 1 a function of time.

In [START_REF] Jacobson | New necessary conditions of optimality for control problems with state-variable inequality constraints[END_REF][START_REF] Hamilton | On nonexistence of boundary arcs in control problems with bounded state variables[END_REF], it is stated that if at a contact time u r wu -1 (τ + ) = u r wu -1 (τ -) with all the smaller order derivatives continuous, then odd-relative degree systems do not have a boundary arc. This optimal input jump condition implies that deg(λ) = 2, as can be easily seen from the z-dynamics. From Proposition 14 it follows that deg(λ) ≤ 1. However the conclusion that there is no boundary arc is premature, see Section 5. Remark 8. These results are of the same nature as results in [START_REF] Bonnans | Well-posedness of the shooting algorithm for state constrained optimal control problems with a single constraint and control[END_REF][START_REF] Hamilton | On nonexistence of boundary arcs in control problems with bounded state variables[END_REF][START_REF] Jacobson | New necessary conditions of optimality for control problems with state-variable inequality constraints[END_REF][START_REF] Mcdanell | Necessary conditions for joning singular and nonsingular subarcs[END_REF]. For instance, compiling Corollary 6 and Proposition 13, one finds that the sum of the order of the first discontinuous derivative of u(•) and of the relative degree r wu , is always odd at an entry time when deg(λ) ≤ 2 (see [32, Theorem 5.1, Corollary 5.2] for similar results). However our study is more general as it relies on a systematic and intrinsic to (A, B, C ) partitioning of bdΦ as well as structural properties of the system (4). In contrast, the condition which allows one to prove a similar result as Proposition 13, namely [26, equ. (81)], does not involve the Markov parameter M (r) , but is based on suitable Taylor expansions of ∂H ∂u and w(•). The partitioning of bdΦ also contains constraint qualifications of the form: there exist x and u such that (Ax+Bu) T C T < 0 for all x ∈ bdΦ (which is nothing else but z 1 ∈ χ 1 con ). Interestingly enough, the work in [START_REF] Hamilton | On nonexistence of boundary arcs in control problems with bounded state variables[END_REF] already defined sets similar to those in Lemma 5, but the developments remained at an embryonic stage. It is sometimes argued that boundary (or constrained) arcs are impossible for odd r wu [START_REF] Oberle | A program for the numerical solution of optimal control problems, Report no 515 der DFVLR[END_REF]. Proposition 13, Corollary 7 and Proposition 14, and the material in Section 5 show that this is more subtle. See also [START_REF] Bonnans | Well-posedness of the shooting algorithm for state constrained optimal control problems with a single constraint and control[END_REF]Remark 4.11] which presents an example which proves that boundary arcs may exist for any r wu . Notice that [START_REF] Mcdanell | Necessary conditions for joning singular and nonsingular subarcs[END_REF] study the problem with input constraints, so that the necessary conditions are not as in (4) since u(t) = -K sgn(B T η(t))

for some K . The necessary conditions can still be written under a complementarity framework since the sign function lends itself to a complementarity formulation:

ẋ(t) η(t) = A 0 Q -A T x(t) η(t) + 0 -C T λ(t) + 1 -1 0 0 λ 1 (t) λ 2 (t) 0 ≤ w(t) = Cx(t) + D w 1 (t) w 2 (t) ⊥ λ(t) λ 1 (t) λ 2 (t) ≥ 0 λ 1 (t) + λ 2 (t) = 1 (55)
where w 1 (•) and -w 2 (•) are the positive and negative parts of B T η(•), respectively. System (55) possesses a non uniform vector relative degree, since relay systems lend themselves to a description as relative degree 0 system [START_REF] Çamlibel | Complementarity methods in the analysis of piecewise linear dynamical systems[END_REF]. The HOSP is presented in [START_REF] Acary | Higher order Moreau's sweeping process: Mathematical formulation and numerical simulation[END_REF] for r ≥ 1, and should be extended to r = 0 in order to encompass systems as [START_REF] Sontag | Deterministic Finite Dimensional Systems[END_REF]. It is finally noteworthy that when input constraints are considered, then the material in Section 3.5 is meaningless, unless these constraints are considered almost everywhere except at contact or entry times.

The following concerns the complementarity conditions of Proposition 2 for an entry state.

Corollary 8. Let z(τ -) ∈ χ 2k+1 con and z(τ

+ ) ∈ ν * , k ≤ r+1 2 . Then 0 ≤ {z i }(τ + ) ⊥ dν i ({τ }) ≥ 0 for all 1 ≤ i ≤ 2k + 2. Moreover dJ 2k+2 > 0 if k ≤ r wu
-1 and r wu must be even if k = r wu -1.

Proof. Using the tangent cones definition in Section 3.1, the definition of the sets χ 2k+1 con and ν * , and the material of Remark 4, it follows that T i Φ ({z 1 }(t -), . . . ,

{z i }(t -)) = R + and ∂ψ T i-1 Φ ({z 1 }(t -),...,{z i-1 }(t -)) ({z i }(t + )) = R -for all 1 ≤ i ≤ 2k + 2 (recall that z (2k+1) 1 (•) = z 2k+2 (•)).
The first result follows from [START_REF] Moreau | Bounded variation in time[END_REF]. The second statement follows from ( 30) and ( 31), and using Lemma 1 and Proposition 13.

Exit times and states

Since the results which can be obtained for exit times are similar to those obtained for entry times, only one proposition is given in this section. We omit the proof which relies on the same arguments as above proofs. Similar results concerning costate and input regularity conditions can be derived for the exit times, and are not given here for the sake of brevity. The following result applies to a contact state that is not an entry state, but a grazing state, and is a direct consequence of Proposition 2.

0 ≤ {z 2k+1 }(τ + ) ⊥ -dν 2k+1 ({τ }) ∈ ∂ψ T 2k Φ ({z 1 }(τ -),...,{z 2k }(τ -)) {z 2k+1 }(τ + ) .
Corollaries 8 and 9 and Proposition 2 show that the additional constraints which may be imposed at contact times, are taken care of with the multipliers dν i , which in turn satisfy a special set of complementarity conditions, not present in (4). It is worth noting that the complementarity conditions satisfied by the multipliers dν i depend both on the pre and postcontact states. This generalizes Nonsmooth Mechanics in the framework of which the percussion value (i.e. the magnitude of the Dirac measure at an impact, see Definition 1.2 in [START_REF] Brogliato | Nonsmooth Mechanics[END_REF]) depends on both the pre and post-impact velocities. To the best of our knowledge this is introduced for the first time in the context of optimal control with inequality state constraints. Remark 9. The introduction of the measure multipliers dν i , 1 ≤ i ≤ r, generalises the case r wu = 1 treated in [START_REF] Augustin | Second order sufficient conditions and sensitivity analysis for the optimal control of a container crane under state constraints[END_REF][START_REF] Malanovski | Second-order sufficient conditions for state-constrained optimal control problems[END_REF] where a multiplier associated to z 2 (t) is introduced (denoted as μ(t) in these papers, and which corresponds in our framework to the function g 2 (t)). The conditions given for instance in [29, equ. (11b) (12b)] are a particular case of the conditions in Proposition 2 and especially [START_REF] Miller | Impulsive Control in Continuous and Discrete-Continuous Systems[END_REF] with r = 2. The inclusion (34) also includes [30, equ. (25)], which is stated directly along boundary intervals (t k , t k+1 ) and {t k } k≥0 is assumed to be a finite set in [START_REF] Malanovski | Sensitivity analysis fo optimal control problems subject to higher order state constraints[END_REF]. Let us consider the minimisation problem of Proposition 8, with I(u) in [START_REF] Van Der Schaft | An Introduction to Hybrid Dynamical Systems[END_REF]. The measures dν i , 1 ≤ i ≤ r, represent the Lagrange multipliers associated with the (unknown) contact times t k ∈ [0, T 1 ]. As noted above, Proposition 2 shows that the function part g i (•) of the measures dν i , 1 ≤ i ≤ r -1, is zero a.e. Only the atomic part dJ i may take positive values.

Multivariable systems

Let us assume that m = n u ≥ 2. The multivariable case differs in many points from the monovariable case. This section only aims at pointing out some of these discrepancies. We consider C ∈ R m×n , D = 0 and u ∈ R m , with m ≥ 2. We suppose that the system (A, B, C ) has a uniform relative degree r wu so that the z-dynamics can be written [START_REF] Sannuti | Direct singular perturbation analysis of high-gain and cheap control problems[END_REF]. Then from Lemma 1 the same holds for ( Ã, B, C) with relative degree r = 2r wu . One has mr wu ≤ n (equivalently mr ≤ 2n), z i = (z 1 i , . . . , z m i ) ∈ R m , 1 ≤ i ≤ r wu (notice that z i = CA j x for all 1 ≤ j ≤ r wu ), ξ ∈ R n-mr wu , ξ ∈ R 2n-mr , CA r wu -1 B ∈ R m×m , C Ãr-1 B ∈ R m×m are full-rank matrices. Similar, more general, canonical forms can be derived (see e.g. [START_REF] Sannuti | Multipe time-scale decomposition in cheap control problems -Singular control[END_REF][START_REF] Sannuti | Special coordinate basis for multivariable linear systems -Finite and infinite zero structure, suqaring down and decoupling[END_REF]) for multivariable systems, but we will content ourselves with the one presented in [START_REF] Sannuti | Direct singular perturbation analysis of high-gain and cheap control problems[END_REF]. One notices that if mr wu = n then there is no zero dynamics in the z-canonical form as defined above.

Let us introduce the following sets and functions, defined for each i ∈ {1, . . . , m}:

r C i : bdΦ → N with r C i (x) = min{j ∈ N | w (j) i (x, u) = 0, u ∈ U} = min{r wu ; inf{i ∈ N | CA i x = 0}},
and

ν C i = {x ∈ bdΦ i | r C i (x) = r wu } = {x ∈ Ker(C i ) | C i A j x = 0 for all 1 ≤ j ≤ r wu }.
where bd

Φ i = {x ∈ R n | C i x = 0}. Notice that one has bdΦ = Φ ∩ m i=1 {x ∈ R n | C i x = 0} . When m = 1
(4) = 1 1 1 2 > 0. The z-dynamics is (in the z coordinates, one has z 1 = (w 1 w 2 ) T = (x 1 x 1 + x 3 ) T )              ż1 1 (t) = z 1 2 (t) ż1 2 (t) = z 1 3 (t) ż1 3 (t) = z 1 4 (t) ż1 4 (t) = η 2 (t) + x 1 (t) + λ 1 + λ 2 w 1 (t) = z 1 1 (t)              ż2 1 (t) = z 2 2 (t) ż2 2 (t) = z 2 3 (t) ż2 3 (t) = z 2 4 (t) ż2 4 (t) = η 2 (t) -x 3 (t) + η 4 (t) -x 1 (t) + λ 1 + 2λ 2 w 2 (t) = z 2 1 (t) (56)
with

z 1 = (z 1 1 , z 1 2 , z 1 3 , z 1 4 ) T , z 2 = (z 2 1 , z 2 2 , z 2 3 , z 2 4
) T (these two vectors are needed to define the tangent cone as a product, see Remark 3),

z i = (z 1 i z 2 i ) T , 1 ≤ i ≤ 4. For the system (A, B, C ) one finds: r C 1 (x) = 1 for all x with x 2 = 0, r C 1 (x) = 2 for all x with x 2 = 0, r C 2 (x) = 1 for all x with x 2 + x 4 = 0, r C 1 (x) = 2 for all x with x 2 + x 4 = 0, ν C 1 = {(0, 0, x 3 , x 4 ) T ∈ R 4 }, ν C 2 = {(x 1 , x 2 , -x 1 , -x 2 ) T ∈ R 4 }. Consequently ν * = ν C 1 ∩ ν C 2 = {0}.
Theorem 1 on existence and uniqueness of solutions on R + of ( 17)-( 19) continues to hold in the multivariable case when M (r) is a nonsingular symmetric M-matrix [39, p. 227]. This secures that M (r) is positive definite and that all its entries M (r) ij ≥ 0 for all 1 ≤ i ≤ m.

Extension of Proposition 10

One has ν C i = {x ∈ Ker(C i ) and x ∈ Ker(C i A j ) for all 1 ≤ j ≤ r wu }. Thus for all z ∈ ν C i one has zi = (z i 1 , . . . , z i r ) T = 0 r , 1 ≤ i ≤ m, and for all z ∈ ν * one has z = (z T 1 , . . . , z T r ) T = 0 rm . Thus if mr = n one has that ν * = {0} as in the monovariable case. We can conclude that in this case the extension of Proposition 10 to the multivariable case is straightforward. It is however noteworthy that assuming m = n u and a uniform relative degree is quite stringent. Canonical forms that are extensions of the z-dynamics for rectangular multivariable systems (m = n u ) or for square systems with non-uniform relative degree exist [START_REF] Sannuti | Multipe time-scale decomposition in cheap control problems -Singular control[END_REF][START_REF] Sannuti | Special coordinate basis for multivariable linear systems -Finite and infinite zero structure, suqaring down and decoupling[END_REF]. The study in [START_REF] Dam | The contact problem for linear continuous-time dynamical systems: A geometric approach[END_REF]extends to the rectangular case. It is noteworthy that even if the system (A, B, C ) is rectangular, the necessary condition system ( Ã, B, C) is always square, but it may not possess a uniform relative degree. Keeping in mind that the HOSP is extended in [START_REF] Acary | Higher order Moreau's sweeping process: Mathematical formulation and numerical simulation[END_REF] only to the square uniform relative degree case, the non-uniform relative degree and rectangular cases are rich sources of problems that are left for future work.

Extension of Proposition 13

As noted in [START_REF] Dam | The contact problem for linear continuous-time dynamical systems: A geometric approach[END_REF], contact can occur on a portion of bdΦ, such that for instance x(τ -) ∈ χ 2k con,1 ∩ ν * 2 . Then the optimal trajectory has a contact time with bdΦ 1 but may have an entry time with bdΦ 2 . Let n u = m. If x(τ -) ∈ χ 2k+1 con,1 ∩ χ 2k+5 con,2 and 2k + 1 < r wu < 2k + 5, one optimal input will be distributional while the other one will be a function of time at τ . It may also happen that χ i rel,j ∈ bdΦ for some constraint j and some i ≥ 1 (see e.g. [START_REF] Dam | Unilaterally constrained dynamical systems[END_REF]Example 6.6.1]). An optimal trajectory with a boundary arc on the constraint j will possess an exit state with an optimal controller with reduced regularity, depending on i. Thus the qualitative behaviour of the optimal trajectory that is made above is still possible, but is more involved. It is however noteworthy that the HOSP state reinitialisation mapping in ( 30) and ( 31) still works, and brings an answer to the questioning about the collision map definition in [18, Section VI]. In the multivariable case (m ≥ 2, uniform relative degree r, definite Markov parameter M (r) ), the conclusions of Proposition 13 need to be refined. For instance, if contact occurs in χ 2r wu -1

con = × m i=1 χ 2r wu i -1 con,i
∩ bdΦ, which means that all z r,i (τ -) < 0, and if z(τ + ) ∈ ν * = ∩ m i=1 ν C i i , then r wu does not necessarily have to be even. It has to be even if the leading Markov parameter M (r) is diagonal. But in general if M (r) is negative definite (without being diagonal), then the reasoning in the proof of Proposition 13 may fail as (M (r) ) -1 (-z r (τ -)) may have positive components despite M (r) < 0 and all components of -z r (τ -) are positive.

Let us recall that a m × m strictly semimonotone matrix [17, Definition 3.3.9] (or

E-matrix) satisfies [v ∈ (R + ) m , v = 0] ⇒ [v i > 0 and (M (r) v) i > 0 for some i].
(f)], we get that if M (r) is an E-matrix, then [v ≥ 0] ⇒ [M (r) v ≥ 0]
. The algebraic dynamics are -z r (τ -) = M (r) λ τ , where λ τ ≥ 0 is the magnitude of dJ r at τ . Since contact occurs in the set χ 2r wu -1 con one has that -z r (τ -) > 0. It is therefore possible that the equality be satisfied for some 1 ≤ i ≤ m. -M (r) being an E-matrix means that whatever v ∈ (R + ) m , v = 0, there exists an index i such that v i > 0 and M (r) v i < 0. This is a contradiction to 0 < -z r (τ -) = M (r) λ τ > 0.

It is not possible in general to relate the E-property to positive or semi positive definiteness. Consequently, we cannot relate it neither to the fact that r wu is even or odd, except if m = 1. Many other generalisations would be possible, like the one indicated above when the leading Markov parameter is diagonal. The considered classes of matrices should all possess some kind of ''no sign reversing'' property. We do not tackle them here for the sake of briefness. From a general point of view, the contact, touch and exit problems (with λ being a measure) may be formulated as follows:

Contact [Touch] (Exit) Problem: Find dν r such that dz r = C Ãr W -1 z(τ )dt + M (r) dν r dν r ∈ -M (r) ∂ψ T r-1 Φ ({z 1 }(t -),...,{z r-1 }(t -)) ({z r }(t + )) z r (τ -) ∈ χ con [χ con ] ( ν * ) z r (τ + ) ∈ ν * [χ rel ] (χ rel )
where τ generically denotes a contact [touch] (exit) time. This can be easily generalised to the case when λ is not a measure, using the measure differential formalism ( 22)- [START_REF] Hartl | A survey of the maximum principles for optimal control problems with state constraints[END_REF]. Proposition 17 gives some conditions under which this problem possesses or not a solution.

that is not a small perturbation of η(0). Therefore the discontinuous dependence behaviour of the IVP solution, may create a jump in the initial condition of the BVP solution. Consequently, the optimal controller will also suffer from a ''discontinuity''.

Boundary arcs, relative degree r

wu = 2k + 1, k ≥ 0
In the previous section, nothing was said about the influence of the sign of the leading Markov parameter on the behaviour of optimal trajectories on bdΦ (i.e. on boundary arcs in ν * ). We now study this aspect, i.e. we study what happens when a solution of the system in (4) has a boundary arc. From Lemma 1 one sees that systems in (2) with r wu = 2k + 1, k ≥ 0 yield ( Ã, B, C) with a leading Markov parameter that is always (semi) negative definite. Let us investigate the consequences on the well-posedness of the dynamics on boundary arcs, i.e. those portions of optimal trajectories that satisfy Cx(t) + D = 0 for t ∈ (τ , τ + ) ⊆ [0, T 1 ], > 0. On (τ , τ + ) one has u(t) = R -1 B T η(t) and η(t) = Qx(t) -A T η(t) -C T λ(t), with λ(t) the solution of the LCP in (38) (equal to g r (t) in this case, where g r (•) is in [START_REF] Maurer | On optimal control problems with bounded state variables and control appearing linearly[END_REF] and Proposition 2). One notices that the complementarity relations 0 ≤ {z i }(t + ) ⊥ g i (t) ≥ 0 are trivially satisfied for all 1 ≤ i ≤ r -1 (see [START_REF] Mcdanell | Necessary conditions for joning singular and nonsingular subarcs[END_REF]).

Remark 10. Following [START_REF] Bryson | Applied Optimal Control; Optimization, Estimation, and Control[END_REF]Sections 3.10,3.11] let us define the Hamiltonian function H(x, u, η, α) = H(x, u, η)+α T z r wu +1 . On a boundary arc one has z r wu +1 (t) = żr wu (t) = CA r wu x(t) + CA r wu -1 Bu(t) = 0. Thus we deduce that u(t) = -(CA r wu -1 -1 CA r wu x(t) (and consequently B T η(t) = -B T (CA r wu -1 B) -1 CA r wu x(t)). Let us now compute the multiplier α(•)

from ∂ H ∂u = 0. One finds α(t) = (-1) r wu +1 [M (r)
] -1 (CA r wu x(t) + CA r wu -1 BB T η(t)). Inserting the value for B T η(t) one finds that α(t) = 0 which is consistent with (33) since α(•) = g r wu +1 (•), the multiplier associated with the coordinate z r wu +1 .

Let us examine the case r wu = 1. On boundary arcs one has w(t) = C x(t) + D = 0 and ẇ(t) = C Ãx(t) = 0, since r = 2, and ẅ(t) = C Ã2 x(t) + M (2) λ(t) = 0. The fact that the trajectory keeps on evolving on bdΦ or detaches from bdΦ is monitored by the LCP: 0 ≤ λ(t) ⊥ ẅ(t) = C Ã2 x(t) + M (2) λ(t) ≥ 0. The fact that the dynamics on boundary arcs are well-posed or not is closely linked to the well-posedness of this LCP, which holds on the interval (τ , τ + ). The following is true since M (2) < 0:

Lemma 6. Let m = 1. Consider the LCP: 0 ≤ λ(t) ⊥ C Ã2 x(t) + M (2) λ(t) ≥ 0. Then: • If C Ã2 x(t) < 0, the LCP has no solution, • If C Ã2 x(t) = 0, the LCP has one solution λ(t) = 0, • If C Ã2 x(t) > 0, the LCP has two solutions λ(t) = 0 and λ(t) = -(M (2) ) -1 C Ã2 x(t).
Let the state satisfy C Ã2 x(τ + ) > 0 at an entry time τ ∈ [0, T 1 ). The last item shows that there may not be uniqueness of the solution to the optimal control problem, if the optimal trajectory grazes the constraint boundary bdΦ. Either the trajectory ''detaches'' from bdΦ (λ(t) = 0 ⇒ ẅ(t) > 0), or remains on bdΦ (λ(t) > 0 ⇒ ẅ(t) = 0). This is not surprising, since the leading Markov parameter is negative. The problem loses its convexity and the system in ( 4) is no longer wellposed as an IVP. It seems that little attention has been paid in the literature to the fact that the multiplier λ is the solution of an LCP on boundary arcs. The boundary arc input satisfies ẇ(t) = CAx(t) + CBu(t), however this may not be the optimal controller. We shall see later what may happen when C Ã2 x(τ ) < 0 at an entry time τ . Lemma 6 can be extended to the general case r wu = 2k + 1 and m ≥ 1, where the LCP in [START_REF] Murray | Existence theorems for optimal control and calculus of variations problems where the state can jump[END_REF] has to be studied. In view of Propositions 13 and 14 and Corollary 7, there are severe restrictions for the existence of entry times for systems with odd r wu . The problem of interest here is to investigate what may happen after a contact time.

Proposition 18. Let r wu = 2k + 1, k ≥ 0, and let m = 1. Let x(τ + ) ∈ bdΦ. Then:

• If C Ãr x(τ + ) < 0, the trajectory leaves Φ in a right neighbourhood of τ .

• If C Ãr x(τ + ) = 0, τ may be an entry time followed by a grazing trajectory.

• If C Ãr x(τ + ) > 0, τ may be either a touch time (λ(τ + ) = 0) or an entry time (λ(τ + ) > 0).

Consequently, optimal trajectories satisfy C Ãr x(τ + ) ≥ 0.

Proof. The proof follows from [START_REF] Murray | Existence theorems for optimal control and calculus of variations problems where the state can jump[END_REF], Lemmas 6 and 1.

Let us notice that it is possible that z(τ -) = 0, which implies from [START_REF] Malanovski | Sensitivity analysis fo optimal control problems subject to higher order state constraints[END_REF] and (31) that z(τ + ) = 0, but that C Ãr x(τ + ) < 0, because of the value of the zero-dynamics state ξ (τ ). In other words, odd-r wu systems can possess entry times with deg(λ) ≤ 1 (see Proposition 14). However ξ (τ ) has to be such that C Ãr x(τ + ) ≥ 0 for a boundary arc to exist on the right of τ . Clearly for an even-r wu system, the trajectory can be kept in Φ as there always exists a multiplier g r (τ + ) solution of the LCP [START_REF] Murray | Existence theorems for optimal control and calculus of variations problems where the state can jump[END_REF]. The zero dynamics plays a major role in the well-posedness of the LCP and whether or not odd-r wu systems possess boundary arcs. For instance if r wu = n, the first item of Proposition 18 becomes irrelevant at an hypertangential contact state, because necessarily C Ãr x(τ + ) = 0.

Corollary 10. Let r wu

= m = n u = 1. If an entry time τ exists on [0, T 1 ], then uniqueness of the solution of the necessary conditions system holds locally if and only if the trajectory along the boundary arc is grazing, that is (CA

2 + CBB T Q )x(t) + (CABB T -CBB T A T )η(t) = 0 for all t ∈ [τ , τ + ]
where τ + is the exit time. Proof. Follows from Proposition 18 and the calculation of C Ã2 . Definition 4. A boundary arc with entry time τ and exit time τ + is well-posed if the LCP: 0 ≤ w (r) (t) ⊥ λ(t) ≥ 0 possesses at least one solution for all t ∈ (τ , τ + ). ξ (t) ≥ 0, ∀ t ∈ (τ , τ + ).

Proof. Since (ii) is an easy consequence of Lemma 6, we just prove (i). Integrating the dynamics without constraint, one finds the optimal trajectory on [0, T 1 ]:

x(t) = x 0 exp(- √ a 2 + b 2 T 1 ) -x 1 exp(- √ a 2 + b 2 T 1 ) -exp( √ a 2 + b 2 T 1 ) exp( a 2 + b 2 t) + x 1 -x 0 exp( √ a 2 + b 2 T 1 ) exp(- √ a 2 + b 2 T 1 ) -exp( √ a 2 + b 2 T 1 ) exp(-a 2 + b 2 t). ( 57
)
The optimal control can be calculated with η(t) = 1 b 2 (ẋ(t) -ax(t)) and u(t) = bη(t). It can be checked that x(t) > -d c for all t ∈ [0, T 1 ). Consequently the boundary ∂Φ is attained only at t = T 1 , and T 1 is a contact time. The LCP for λ on bdΦ is given by: 0 ≤ λ ⊥ -d(a 2 + b 2 ) -c 2 b 2 λ ≥ 0. We deduce that if -d < 0 the LCP has no solution, if d = 0 then λ = 0 and if -d > 0 then two solutions λ = 0 and λ = -d(a 2 = -d 2 < 0. Let us take Q =diag(q 1 , q 2 ). With this system we associate its optimality conditions as in (4) and we check that C ÃB = -d 2 . We get H wλ (s) = -cd 2 (q 2 +a+b 2 )+c 3 ad 2 -c 2 -bcd , β = -q 2 dc+abd 2 -bc 2 -b 2 cd+abd 3 -cd 2 (q 2 +a+b 2 )+c 3 -ad 2 +c 2 +bcd , γ = abd 3 -cd 2 (q 2 +a+b 2 )+c

) 59
Along the boundary arc we have from Lemma 6: λ(t) = 1 d 2 C Ã2 x(t) provided C Ã2 x(t) ≥ 0. Now C Ã2 x(t) = a(c+bd)x 1 (t)+ (cb+d(q 2 +a+b 2 ))x 2 (t)-dη 1 (t)+cη 2 (t). Combining the above values we get that λ(t) = abd 3 -cd 2 (q 2 +a+b 2 )+c 3 -d 2 (ad 2 -c 2 -bcd) η 2 (t)-1 d η 1 (t). Injecting all the calculated values for x 1 (t), x 2 (t) and λ(t) into (59), the result follows. The conditions are sufficient but also necessary for if γ η 2 (t) -dη 1 (t) + < 0 the boundary arc does not exist at time t.

Since the system in ( 58) is linear, of order two, it is quite possible in practice to test whether or not [START_REF] Valentine | The problem of Lagrange with differential inequalities as added side conditions[END_REF] holds. This provides conditions on the entry values η 1 (τ ) and η 2 (τ ), such that a boundary arc is well-posed or not. Clearly Lemma 7 does not answer the question whether or not the optimal trajectory possesses boundary arcs. Notice that x(0) is not known as η(0) is an unknown of the BVP.

Numerical solution of the BVP

The objective is to numerically solve the extremisation problem in Proposition 8, using the time-stepping scheme developed in [START_REF] Acary | Higher order Moreau's sweeping process: Mathematical formulation and numerical simulation[END_REF] which approximates the solutions of the measure differential formalism in ( 22)-( 24), whose solutions are functions of time. Though no complete convergence result has been presented yet for this time-stepping algorithm, it is shown in [1, Section 5] that it possesses strong properties. Convergence is proved in specific examples. Numerically solving the minimisation problem in ( 52) is equivalent to solving the MBVP in Section 3.6. Let us outline the way the MBVP may be discretised. For details and subtleties on the time-stepping scheme that is presented next, the reader is referred to [1, Section 5]. Since the time-stepping scheme is based on the z-dynamics, let us first rewrite the MBVP as:

       -dz + W Ã W -1
z(t)dt + Mdν z = 0 dν z = (dν 1 , . . . , dν r , 0 2n-r ) T dν i ∈ -∂ψ T i-1 Φ (z 1 (t -),...,z i-1 (t -)) (z i (t + )) for all 1 ≤ i ≤ r G(z(0), z(T 1 )) = 0 for some function G(•, •) representing the boundary constraints, and where z = W x and M is defined in Section 3.5. Let us define a finite partition 0 = τ 0 < τ

1 < • • • < τ k < • • • < τ N = T 1 of [0, T 1 ].
For the sake of simplicity, we choose h = τ k+1 -τ k > 0 so that T 1 = Nh. The approximation of z(τ k ) is denoted as zk . Then the MBVP is discretised as follows:

Discrete MBVP : zk+1 -zk = h W Ã W -1 zk+1 + Mµ k+1 , 0 ≤ k ≤ N µ k+1 = (µ 1,k+1 , ..., µ r,k+1 , 0 2n-r ) T µ i,k+1 ∈ -∂ψ T i-1 Φ (z 1,k ,...,z i-1,k ) (z i,k+1 ), 1 ≤ i ≤ r G(z 0 , zN ) = 0 It is crucial to note that the primary variables in this algorithm are the quantities dz((τ k , τ k+1 ]) and the discretised multipliers µ i,k , which do not intend to approximate the Dirac measures at atoms of the measures dν i , but their magnitude, i.e. µ i,k+1 = dν i ((τ k , τ k+1 ]). They are therefore always bounded quantities. Moreover this time-stepping scheme is not an implicit Euler scheme, though it looks like [START_REF] Acary | Numerical Methods for Nonsmooth Dynamical Systems[END_REF]. This time-stepping scheme approximates the solutions of the measure differential formalism, which are functions in the set F ∞ (R + ; R). The practical implementation of the algorithm is detailed in [START_REF] Acary | Higher order Moreau's sweeping process: Mathematical formulation and numerical simulation[END_REF]Section 5.4]. The next step is to construct a multiple shooting algorithm starting from the Discrete MBVP. This will be the object of a future work dedicated to the numerical aspects of this optimal control problem.

Conclusions

This paper concerns the optimal control of linear invariant systems, with state inequality constraints. This is a major topic, which has continuously attracted the interest of researchers for more than 70 years. The main contributions are the following. The Bolyantskii-Pontryagin necessary conditions are embedded into a distributional differential inclusion framework (the higher order Moreau's sweeping process [START_REF] Acary | Higher order Moreau's sweeping process: Mathematical formulation and numerical simulation[END_REF]) which allows one to clearly understand their dynamics and provides a clear formalism for the derivation of an extended action to be minimised (allowing for costate and/or inputs and/or system's state, to belong to a specific set of Schwarz' distributions). The powerful geometric tools introduced by ten Dam in the context of unilaterally constrained controlled dynamical systems [START_REF] Dam | The contact problem for linear continuous-time dynamical systems: A geometric approach[END_REF][START_REF] Dam | Unilaterally constrained dynamical systems[END_REF], are used to improve the qualitative analysis of optimal trajectories and controllers, both in mono-and multivariable cases. They allow us to generalise results concerning the specificity of odd relative degree systems (one of the results being that odd relative degree systems can possess boundary arcs with entry times such that the multiplier associated to the unilateral constraint is a distribution of degree ≥ 3; then the action to be minimised has to be extended to incorporate the state and/or costate jumps). The theory of complementarity problems is shown to be quite useful in order to better understand the behaviour of optimal path on the boundary of the admissible domain. Most of the tools which are used in this paper (distributional inclusion, ten Dam's geometrical study, complementarity problems, the role played by the zero dynamics) have never been introduced before in the context of optimal control with unilateral state constraints. Let us point out some open issues which might be the object of future works:

• Extend the qualitative study of trajectories at junction times to nonlinear systems (ten Dam's geometric approach extends to a class of nonlinear systems).

• Extend the study to rectangular or non-uniform relative degree systems.

• Design a multiple shooting algorithm based on the higher order sweeping process time-stepping discretization. Notation and nomenclature: contact time and state (state on the boundary coming from an interior arc), entry time and state (contact state followed by a boundary arc), touch time and state (contact state coming from an interior arc and followed by an interior arc), junction time and state (entry, contact, exit, touch time and state), CP (complementarity problem), LCP (higher order sweeping process), δ t the Dirac measure at t, 0 m = (0, . . . , 0) T ∈ R m , 0 m = (0 m ) T , I n is the n-identity matrix, 0 n×n is the zero n × n matrix. Contact times are denoted as τ (when only one contact is analysed) or t k when a sequence of contact times is examined, E 0 (h) denotes the set of jumps of the function h(•). e i is the i-th unit vector of R n ; a function in C ∞

Remark 4 .

 4 (i) The subdifferential of the indicator function of the cone T i-1 Φ (z 1 , . . . , z i-1 ) is given by[START_REF] Goeleven | Variational and Hemivariational Inequalities: Theory, Methods and Applications; Volume I: Unilateral Analysis and Unilateral Mechanics[END_REF] Section 1.3.1]

  2 and deg(η 3 ) = 0. This is a case where deg(B T η) = 0 while deg(A T η) = 2, and deg(λ) = 3. This is in agreement with Proposition 5: if deg

Corollary 4 .

 4 ) [4, Theorem 1] [9] [45, (4.1)-(4.5)]. From [45, Proposition 4.1] normal extremals are minimisers of (1) and (2) in such a case. Also we note that x = {x}, u = {u}, and η = {η} on [0, T 1 ]. Assume the conditions of Proposition 6 are satisfied and that U is a set of functions of time. Let τ ∈ [0, T 1 ] be an entry time. Let us consider the solution x = W -1 z of the distributional formalism in (17)-(

Lemma 4 .

 4 Let r wu = 2k, k ≥ 0, and m = 1. The HOSP jump rule in (30) and (31) minimises the quadratic term 1 2 dJ¯z , ϕ T M dJ¯z , ϕ for any test function ϕ(•) ∈ C 0 [R; R] with support containing [0, T 1 ], under the constraints in (36) and (37).

2 .

 2 Let U be a set of distributions in T r wu ([0, T 1 ]) such that the atoms and points of non-analyticity of any u ∈ U are in the set r i=1 supp(dJ i ). We denote it as U r wu J .The definition of U r wu J is motivated by the fact that the HOSP solutions are constrained to exist in the set T r-1 ([0, T 1 ]), see Proposition 1. In particular deg(z 1 ) ≤ 1. Having u ∈ U r wu J means that deg(u) ≤ r wu + 1, while along solutions of the HOSP deg(λ) ≤ r + 1 = 2r wu + 1. The inspection of the z-dynamics shows that deg(z 1 )

Example 3 .

 3 Following[START_REF] Bryson | Applied Optimal Control; Optimization, Estimation, and Control[END_REF] Section 3.11] let us consider the system with relative degree r wu = 2, l > 0,

Proposition 10 .

 10 Let assumptions (i)-(iv) stand. (i) Let r wu = n. Then ν * = {0}. If the optimal trajectory has a boundary arc on

= 1 Corollary 5 .

 15 and ν * = {z | z 1 = 0} = bdΦ. Point (ii) of Proposition 10 holds for any 1 ≤ r wu ≤ n. Let r wu = n and the above assumptions hold. If the unconstrained optimal trajectory x un (•) (the solution of (4)(a) (b) with C

Example 5 .Proposition 11 .

 511 Let A = w(t) = -x 1 (t) + 1. Then ν * = {(1, 0)},and u * = 0. If assumption (ii) does not hold, i.e. r wu = 1, then a (possibly discontinuous) time function u(•) is sufficient to keep x(•) inside Φ at entry times. At such instants 2 ≥ deg(λ).

Proposition 15 .

 15 Let r wu = 1, n u = m = 1. Assume that x1 is reachable from x0 with U = U 0 J . Then the optimal trajectory and controller are time-continuous functions on [0, T 1 ].

Proposition 16 .

 16 Let deg(λ) ≤ 2. Then the connection between a boundary arc and an interior arc has to occur at an exit state in χ 2k+1 rel with k ≥ r wu -1.

Corollary 9 .

 9 Let z(τ -) ∈ χ 2k+1 con and z(τ + ) ∈ χ 2k+1 rel , k ≤ r+1 2 .Then the following CP is satisfied:

Example 6 .

 6 and assumptions (i)-(iv) hold, one has ν C = ν * [18, Lemma A.1]. However in the multivariable case one has Let us consider A = = I 2 , Q = I 4 . Then r wu = 2 (i.e. the vector relative degree is (2 2) T ) and the leading Markov parameter of ( Ã, B, C) is M

1 con,

 1 This allows us to propose an extension of Proposition 13. Proposition 17. • (a) Let M (r) be a nonsingular symmetric M-matrix. Then it is possible that contact occurs in the set χ 2r wu -1 con , and with z(τ + ) ∈ ν * . This implies that r wu is even. • (b) Let M (r) be an E-matrix. Then it is possible that contact occurs in the set χ 2r wu and with z(τ + ) ∈ ν * . If -M (r) is an E-matrix, this is impossible. Proof. One first notices that contact occurring in χ 2r wu -1 con implies that deg(λ) ≤ 2, i.e. λ is a measure. Moreover, we recall that ν * = {0}. (a) From the property of the inverse of a nonsingular symmetric M-matrix the result follows. The second result is true since a nonsingular symmetric M-matrix is positive definite, and using Lemma 1. (b) From [17, Theorem 3.10.7 (b)

On a boundary arc zT = (z 1 ,

 1 z 2 , . . . , z r ) = (0, . . . , 0). Definition 4 applies to systems with m ≥ 1. The next corollary concerns scalar systems as ẋ(t) = ax(t) + bu(t), w(t) = cx(t) + d, Q = R = 1. Then the Hamiltonian dynamics are ) + 0 -1 λ. The LCP to be solved on boundary arcs is:0 ≤ λ ⊥ -b 2 c 2 λd(a 2 + b 2 ) ≥ 0.Corollary 11. (i) If r wu = n = 1 and cx 0 + d > 0, cx 1 + d = 0, then the dynamics on boundary arcs are well-posed if and only if d ≤ 0. Moreover bdΦ = {x = -d c } is attained only at t = T 1 on the optimal trajectory. (ii) Let us now consider n ≥ 2, D = 0 and m = 1 in (1). If r wu < n, then the well-posedness of the dynamics along boundary arcs implies that C Ã2 W -1 0

+b 2 )

 2 c 2 b 2 are possible. However the behaviour at T 1 has no consequence on the value of the action integral I(u) since the optimal control and state trajectory are Lebesgue measurable functions. The last condition in Corollary 11 can be written also as C Ã2 W -1 0 ξ (τ ) exp( Ãξ (tτ )) ≥ 0. The matrix Ãξ is characterised from the zero dynamics of (A, B, C ), see Lemma 2. In case (i) one sees that ν * = {-d c } and u * = ad cb . From Proposition 9 boundary arcs and entry states belong to ν * , which is consistent with Corollary 11(i). Let us now consider the planar system A = 0 1 a b , B = 0 1 , C = (c d), D = e. This system has transfer function G(s) = c+ds s(s-b)-a , relative degree r wu = 1 and CB = d = 0. Therefore M(2)

(Lemma 7 . 2

 72 c+ds)(c-ds) (s 2 +bs-a)(s 2 -bs-a)-s 2 +1 . Assume that -ad 2 + c 2 + bcd = 0 and that c = 0. The above planar system has a well-posed boundary arc on the time interval[τ , τ + ) ⊆ [0, T 1 ] if and only if ) = αη 2 (t) -cη 1 (t) + δ η2 (t) = -(1 + d)η 1 (t) + βη 2 (t) (t) -dη 1 (t) + ≥ 0 for all t ∈ [τ , τ + )

(

  linear CP), ψ K (the indicator function of a set K [25, p. 82]), ∂ψ K (the subdifferential of the indicator of a convex set K [22, p. 67]), supp(T ) (the support of a distribution T [21, p. 142]), x 0 (x is not zero and the first nonzero element of the vector x is positive), x 0 (the first nonzero element of the vector x is non negative), similarly for ≺ and , σ f (t) = f (t + ) -f (t -) the jump of the function f (•) at time t, BV (bounded variation), RCLBV (right continuous of local bounded variation), RCSLBV (right continuous of special local bounded variation), BVP (boundary value problem), IVP (initial value problem), HOSP

  have to be interpreted in the following sense: find nonnegative real-valued Radon measures dµ i (1 ≤ i ≤ r) relative to which the Lebesgue measure dt and the Stieltjes measure d{z i } possess densities dt

	dµ i	and	d{z i } dµ i	respectively such that:

Proposition 3 .

 3 The degree of λ at a time t is ≤ 2 if and only if all the z i (•), 1 ≤ i ≤ r -1, are continuous at t, while z

r (•) is a possibly discontinuous function at t. Moreover in such a case x(•) = {x}(•) is a continuous function at t while η(•) = {η}(•)

jumps at t.

Proof. The first assertion comes from the z-dynamics. Now if x(•) is a function and has a jump, necessarily from (4) deg(η) ≥ 2. Thus necessarily deg(λ) ≥ 3 which is a contradiction. Finally z = W x and the transformation matrix W is square full rank. Thus if η(•) is continuous at t, so is z(•). So necessarily η(•) has to be discontinuous if z r (•) is. The second part of the Proposition is proved. Proposition 3 says nothing on the definition of the jump: clearly if M

(r)

  3 -ad 2 +c 2 +bcd On the boundary arc one has C x(t) + D = 0 and C Ãx(t) = 0, which respectively yield cx 1 (t) + dx 2 (t) + e = 0 and adx 1 (t) + (c + bd)x 2 (t) + dη 2 (t) = 0. We deduce that x 2 (t) = -cd -ad 2 +c 2 +bcd η 2 (t) + ade(-ad2 + c 2 + bcd), andx 1 (t) = -ad 2 +c 2 +bcd η 2 (t) -ad 2 e c (-ad 2 + c 2 + bcd) -e c .Moreover from the values of à and C we get that

							,
	= -e a c (c + bd)[ad 2 (-ad 2	+ c 2	+ bcd) + 1], δ = -q 1	ad 2 e c (-ad 2	+ c 2	+ bcd) -q 1	e c .
	Proof. d 2

η1 (t) = q 1 x 1 (t) -aη 2 (t) -cλ(t) η2 (t) = q 2 x 2 (t) -η 1 (t) -bη 2 (t) -dλ(t) 0 ≤ cx 1 (t) + dx 2 (t) + e ⊥ λ(t) ≥ 0.

(

In this formalism, {z i } = z i because z i is a function and dz i is a Stieltjes measure.

We say that a vector jumps if at least one of its components jumps.

We can safely differentiate B T η(t) as B T η(t), as we know that the function B T η(•) is continuous at t.

A -1 ν = {x ∈ R n |; Ax ∈ ν}. It is not required that A be full rank.

Continuous dependence of BVP solutions

It is well known from Mechanics that solutions of differential inclusions like the sweeping process, may be discontinuous with respect to initial conditions when m ≥ 2 [START_REF] Monteiro Marques | Differential Inclusions in Nonsmooth Mechanical Problems, Shocks and Dry Friction[END_REF][START_REF] Paoli | A numerical scheme for impact problems with inelastic shocks: A convergence result in the multi-constraint case[END_REF]. If ones embeds (4) into the HOSP, and proves that the solution is optimal with respect to an extended integral action, then the optimal pair (x(•), u(•)) may be very sensitive to the BVP data. It is noteworthy that this phenomenon may already exist when λ is a measure, i.e. when the HOSP is a measure differential inclusion, and should therefore be considered as an important feature. We shall say that the BVP in ( 4) is continuous with respect to data if small perturbations on the data (x 0 , x1 , T 1 ), result in a small perturbation of the solution z. More formally, let the sequences {x 0,n }, {x 1,n }, and {T 1,n } converge in R towards x0 , x1 , and T 1 respectively. Let us consider the solutions of the BVP (4) as being elements in F ∞ ([0, T 1 ], R), and solutions of the measure differential formalism in ( 22)-(24) (or [START_REF] Hiriart-Urruty | Fundamentals of Convex Analysis[END_REF] and ( 26)). Let us denote them as zn (•) for the corresponding data. Then weak (resp. uniform) continuity in the data holds if and only if {z n (•)} n≥0 converges weakly (resp. uniformly) towards z(•). The analysis which follows is essentially qualitative and illustrates why continuity w.r.t. the data may fail. Let us consider the same system as in Example 6.

Let us now consider a solution x(•) of the BVP (4), with initial condition (x 0 , η(0 -)) (point A on Fig. 1). Let us assume that x(•) hits the boundary w 2 = 0 at a time t 0 < T 1 in a neighbourhood of the origin (point B on Fig. 1), with the following data:

This way we assume that the contact is made in the set χ 1 con,2 . The tangent cones can be computed as indicated in Section 3.1 (see in particular Remarks 3 and 4), and we find T 0

It follows from [START_REF] Malanovski | Sensitivity analysis fo optimal control problems subject to higher order state constraints[END_REF] and

does not satisfy the condition in Remark 5 (i), there are two possibilities:

Since it is assumed that B is in a neighbourhood of the origin, and since the trajectory is analytic in a right neighborhood of t 0 , there exists a junction time t 1 > t 0 with contact either at the origin (the trajectory ''slides'' on the boundary w 2 = 0) or with the boundary w 1 = 0 at some point C that is also in a neighbourhood of the origin. In any case the trajectory will either continue to evolve along the boundary w 1 = 0, or hit it and leave it with a large tangential velocity. If the optimal trajectory detaches at B and hits the boundary w 1 = 0 at t 1 at some point C , then the ''velocity'' is reinitialised to ẋ1 (t + 1 ) = 0 and ẋ3 (t

. Since we can assume that the trajectory evolves arbitrarily close to the corner at the origin, it follows that ẋ3 (t + 1 ) has a magnitude that is of the same order as that of ẋ1 (t

). Therefore given any > 0, it is possible to initialise the trajectory in A with a large enough |ẋ 1 (t - 0 )|, so that at a time t 1 + δt 1 > t 1 > t 0 , it enters the domain D = {0 ≤ x 1 ≤ , x 3 ≥ }. The same reasoning for the trajectory starting at A + δA can be done to conclude that it enters D = {x 1 ≥ , 0 ≤ x 1 + x 3 ≤ } after a finite time. It is noteworthy that δx 0 may be arbitrarily small while in D and D keeps its value. Whatever the behaviour at the corner may be, the continuous dependence on data as defined above cannot hold.

We conclude that provided the initial data satisfies some magnitude constraint, the unperturbed optimal trajectory starting at (x 0 , η(0)) reaches an end-point (x 1 , η 1 ) in a time T 1 , but the perturbed trajectory will not reach a neighbourhood of (x 1 , η 1 ) in a time T 1 + δT 1 . In order that it does, the BVP solution starting at x0 + δx 0 has to be initialized with a costate

Appendix A. Some mathematical definitions

Given A ∈ R n×n and B ∈ R n , the problem of finding x ∈ R n and y ∈ R n satisfying

is called a Linear Complementarity Problem (LCP). It can be equivalently written as

Roughly put, the LCP has a unique solution x * whatever B if and only if A satisfies some positivity conditions, see [START_REF] Cottle | The Linear Complementarity Problem[END_REF]. Positive definiteness of A is sufficient. The so-called P-property of matrices is necessary and sufficient for the LCP to have a unique solution for any B. The variables x and y in (61) satisfy a CP. The next notions may be found in [START_REF] Acary | Higher order Moreau's sweeping process: Mathematical formulation and numerical simulation[END_REF][START_REF] Moreau | Bounded variation in time[END_REF][START_REF] Monteiro Marques | Differential Inclusions in Nonsmooth Mechanical Problems, Shocks and Dry Friction[END_REF] Let I denote a non-degenerate real interval (not empty nor reduced to a singleton).

• By z ∈ BV (I; R n ) it is meant that z is a R n -valued function of bounded variation if there exists a constant C > 0 such that for all finite sequences t 0 < t 1 < • • • < t N (N arbitrary) of points of I, we have

Let J be a subinterval of I. The real number var(z, J) := sup

where the supremum is taken with respect to all the finite sequences t 0 < t 1 < • • • < t N (N arbitrary) of points of J, is called the variation of z in J.

Any • We denote by LBV(I; R n ) the space of functions of locally bounded variation, i.e. of bounded variation on every compact subinterval of I.

• We denote by RC LBV(I; R n ) the space of right-continuous functions of locally bounded variation. It is known that if z ∈ RC LBV(I; R n ) and [a, b] denotes a compact subinterval of I, then z can be represented in the form:

where J z is a jump function, [z] is an absolutely continuous function and ζ z is a singular function. Here J z is a jump function in the sense that J z is right-continuous and given any ε > 0, there exist finitely many points of discontinuity t 1 , . . . , t N of J z such that [z] is an absolutely continuous function in the sense that for every ε > 0, there exists δ > 0 such that • By z ∈ RCSLBV(I; R n ) it is meant that z is a right-continuous function of special locally bounded variation, i.e. z is of bounded variation, and can be written as the sum of a jump function and an absolutely continuous function on every compact subinterval of

where [z] is a locally absolutely continuous function called the absolutely continuous component of z and J z is uniquely defined up to a constant by

where t 1 , t 2 , . . . , t n , . . . denote the countably many points of discontinuity of z in I.

Stieltjes (or differential) measure. Let z ∈ LBV(I; R n ) be given. We denote by dz the Stieltjes measure generated by z.

In particular, we have dz({a}) = z(a + ) -z(a -).

Appendix B. Example of a function in F ∞ and distributions in T n

This is taken from [START_REF] Acary | Higher order Moreau's sweeping process: Mathematical formulation and numerical simulation[END_REF]. Set I = (0, +∞) and let z : I → R be the function given by

It is clear that

We see that E 0 (ẑ (1) ) = {kπ; k ∈ N \ {0}} and ẑ(1) (•) = [ẑ (1) ](•) + J(•), where [ẑ (1) ](t) = -2k + cos(tkπ

and

And so on, we see that

so that ẑ(k) ∈ RCSLBV(I; R), ∀k ∈ N, and thus z(•) ∈ F ∞ (I; R).

Let us now consider the distribution T defined by

Then for a given function ϕ ∈ C ∞ 0 (I), we see that:

and so on. We have: (1) .

Appendix C. Hints on the controllability of (2)

Example 7. Let us consider the system

Clearly if U is restricted to the set of bounded functions of time, then (A, B) is not controllable in the set Φ = {x | x 2 ≥ 0}.

However if distributions of degree 3 (derivatives of Dirac measures in the sense of Schwartz' distributions) are allowed in U, then controllability holds (but, the quadratic term in u in (1) is meaningless for such inputs). Indeed negative jumps in x 1 (•) are then possible to reach a state x(T 1 ) from x(0) with x 1 (T 1 ) < x 1 (0), T 1 > 0. In particular x(T 1 ) = 0 belongs to the reachable space from x(0 -) > 0. If the constraint is replaced by x 1 (t)+d ≥ 0, d ∈ R, then controllability holds in Φ with U a set of functions. One sees that reachability does not hold in the set of inputs U 1 J (see Definition 2) because the zero dynamics state ξ = x 1 needs to jump and the degree of the input (as a distribution in U 1 J ) is ≤ 2. This is not permitted by the HOSP framework.