Oana-Silvia Serea

ON REFLECTING BOUNDARY PROBLEM FOR OPTIMAL CONTROL *

Keywords: control of variational inequality, boundary reflection, viscosity solutions

This paper deals with Mayer's problem for controlled systems with reflection on the boundary of a closed subset K. The main result is the characterization of the possibly discontinuous value function in terms of a unique solution in a suitable sense to a partial differential equation of Hamilton-Jacobi-Bellman type.

Introduction. We investigate the Mayer control problem:

Minimize g(x(T )) [START_REF] Aubin | Differential Inclusions[END_REF] for a given T > 0 over all absolutely continuous solutions of the following differential variational inequality:

⎧ ⎨ ⎩ (i) x (t) ∈ f (x(t), u(t)) -N K (x(t))
for almost all t ≥ t 0 , (ii) x(t) ∈ K for all t ≥ t 0 , x(t 0 ) = x 0 , and

u(•) : [0, ∞) → U is a measurable function, (2)
where N K (x) is the normal cone to K at x ∈ K (see Definition 1).

Here K is a nonempty closed subset of R N , g : K → R and f is a function from

K × U into R N .
If U(t 0 ) is the set of measurable controls on [t 0 , ∞) with values in U , the value function corresponding to the optimal control problem (1), ( 2) is given by

V (t 0 , x 0 ) = inf u(•)∈U (t 0 )
g(x(T ; t 0 , x 0 , u(•)) for all (t 0 , x 0 ) ∈ [0, T ] × K, [START_REF] Barles | Solutions de viscosité des équations de Hamilton-Jacobi[END_REF] where x(•; t 0 , x 0 , u(•)) denotes the solution of (2) starting from (t 0 , x 0 ).

By the very definition it is easy to see that the value function is finite on its domain [0, T ] × K, if and only if [START_REF] Aubin | Viability Theory[END_REF] has solutions. This explains the choice of the form of the right-hand side of the differential inclusion [START_REF] Aubin | Viability Theory[END_REF]. We notice that N K (x) = {0} whenever x ∈ intK; f is modified only on the boundary of K, so (2) is a problem with reflection at the boundary. We shall show that this reflection allows us to obtain the existence of solutions to (2) (see section 1).

Our main purpose in this paper is to characterize the value function (3) by an equation of Hamilton-Jacobi type.

Of course, the characterization is based on a suitable definition for the notion of viscosity solutions of a Hamilton-Jacobi-Bellman inequality (HJBI) that we will introduce below.

More precisely, we will prove that the value function V is the unique solution to ⎧ ⎨ ⎩ ∂V ∂t (t, x) + H(x, ∂V ∂x (t, x)) -∂V ∂x (t, x), N K (x) 0 if (t, x) ∈ [0, T ) × K, with the condition V (T, x) = g(x) if x ∈ K, (HJBI) where H(x, p) := min u∈U f (x, u), p .

If the boundary of K, ∂K ∈ C 1 , and K is the closure of an open set, we will show that V is a viscosity solution of the following Hamilton-Jacobi equation with Neumann-type boundary condition in the sense of [START_REF] Lions | Neumann type boundary conditions for Hamilton-Jacobi equations[END_REF]:

⎧ ⎨ ⎩ ∂V ∂t (t, x) + H(x, ∂V ∂x (t, x)) = 0 if (t, x) ∈ [0, T ) × K,
∂V ∂n (t, x) = 0 if (t, x) ∈ [0, T ) × ∂K, with the condition V (T, x) = g(x), x ∈ K, [START_REF] Barles | Fully Neumann boundary conditions for quasilinear degenerate elliptic equations and applications[END_REF] where n(x) is the unit outward normal to K at x ∈ ∂K.

It is well known that the value function for the Skorokhod control problem (see [START_REF] Barles | Solutions de viscosité des équations de Hamilton-Jacobi[END_REF], [START_REF] Lions | Neumann type boundary conditions for Hamilton-Jacobi equations[END_REF]) with a smooth K is a viscosity solution of [START_REF] Barles | Fully Neumann boundary conditions for quasilinear degenerate elliptic equations and applications[END_REF]. The Skorokhod problem for a smooth K has been considered and solved by Lions [START_REF] Lions | Neumann type boundary conditions for Hamilton-Jacobi equations[END_REF] and Lions and Snitzman [START_REF] Lions | Stochastic differential equations with reflecting boundary conditions[END_REF]. Another study was made by Tanaka in [START_REF] Tanaka | Stochastic differential equations with reflecting boundary condition in convex regions[END_REF] when K is convex with normal reflection. By a different approach, this problem was considered like a viability problem for a differential inclusion by Frankowska in [START_REF] Frankowska | A viability approach to the Skorohod problem[END_REF]. Note that the notion of solutions of the Skorokhod problem is not the same as the notion of solutions to (2) that we use in this paper, but for the smooth case the two control problems lead to the same Hamilton-Jacobi equation [START_REF] Barles | Fully Neumann boundary conditions for quasilinear degenerate elliptic equations and applications[END_REF].

Our second interest is to establish that the two following systems,

(i) x (t) ∈ F (x(t)) -N K (x(t)) for almost all t ≥ t 0 , (ii) x(t) ∈ K for all t ∈ [t 0 , ∞), t 0 ≥ 0, x(t 0 ) = x 0 ∈ K (5) and (i) x (t) ∈ Π coT K (x(t)) F (x(t)) for almost all t ≥ t 0 , (ii) x(t) ∈ K for all t ∈ [t 0 , ∞), t 0 ≥ 0, x(t 0 ) = x 0 ∈ K, (6)
have the same set of solutions.

Here K is compact, F : K → R N is a set valued map and coA is the closed convex hull of a set A.

In general, the map x → N K (x) has no easy continuity properties and so the right side of the differential inclusion is [START_REF] Aubin | Viability Theory[END_REF]. For this reason the set of solutions to [START_REF] Barron | Semicontinuous viscosity solutions of Hamilton-Jacobi equations with convex Hamiltonian[END_REF] or ( 6) may be empty. So it is necessary to find regularity hypotheses for K in order to provide existence and eventually uniqueness results for [START_REF] Barron | Semicontinuous viscosity solutions of Hamilton-Jacobi equations with convex Hamiltonian[END_REF] or [START_REF] Cornet | Existence of slow solutions for a class of differential inclusions[END_REF].

These kind of results for a general map F can be applied, in particular, when F (•) = f (•, U), allowing us to obtain properties of the set of solutions to [START_REF] Aubin | Viability Theory[END_REF].

Our main contribution here is the fact that by introducing the projection on the closed convex hull of T K (x) in [START_REF] Cornet | Existence of slow solutions for a class of differential inclusions[END_REF] we succeed in treating the case where the set K is only compact, improving the already known equivalence and existence results of [START_REF] Aubin | Viability Theory[END_REF] where K is supposed to be sleek.

Existence and equivalence results for ( 5) and ( 6) are established by Henry [START_REF] Henry | An existence theorem for a class of differential equations with multivalued right-hand side[END_REF] for a convex set. The convexity assumption on the set K, has been relaxed by Cornet in [START_REF] Cornet | Existence of slow solutions for a class of differential inclusions[END_REF], who merely required the tangential regularity. We also refer to Thibault [START_REF] Thibault | Sweeping process with regular and nonregular sets[END_REF] for the case of a closed set K for an existence result of viable solution, but the reflection is made using the Clarke normal cone. Note that in [START_REF] Thibault | Sweeping process with regular and nonregular sets[END_REF] the set K may depend on t.

We note that the boundary reflection control problem was not yet well studied for nonsmooth K. We also succeed in generalizing some existence and equivalence results of [START_REF] Aubin | Viability Theory[END_REF] for the systems ( 5) and [START_REF] Cornet | Existence of slow solutions for a class of differential inclusions[END_REF].

Let us explain how this paper is organized.

In the first section we introduce some preliminaries and we study the systems ( 5), [START_REF] Cornet | Existence of slow solutions for a class of differential inclusions[END_REF].

In the second section we prove that the value function is a viscosity solution of (HJBI) in the sense of Definition 3, and a uniqueness result for the solutions of this partial differential inequality is also established.

In the third section we study the case of discontinuous and only bounded value functions for our control problem. Our main result says that V is the unique generalized solution to the corresponding (HJBI) for arbitrary discontinuous terminal cost g.

The fourth section concerns existence and uniqueness results of l.s.c. solutions to (HJBI) in the sense of Definition 16.

The last section is an appendix with technical proofs of our claims.

Preliminaries.

2.1. Definitions, assumptions, and notations. We assume that f :

K ×U → R is continuous and satisfies ⎧ ⎨ ⎩ ||f (x, u)|| ≤ a(1 + x ), ||f (x, u) -f (y, u)|| ≤ c 1 x -y the set f (x, U ) is convex, for all x, y ∈ K, u ∈ U, (H f )
where c 1 , a > 0 are constants; U is a compact metric space.

We recall the notions of tangent and normal cones. Definition 1. For x ∈ K, we define by

T K (x) = v ∈ R N | lim h→0 + inf d K (x + hv)/h = 0
the tangent cone to K at x and by

N K (x) = T K (x) -= {p ∈ R N | p, v ≤ 0 for all v ∈ T K (x)} the normal cone to K at x. Recall that T K (x) is a closed cone and N K (x) is a closed convex cone.
Let us describe some classes of sets which will be used in the following sections.

Definition 2. A closed set K ⊂ R N is called proximal retract if there ex- ists a neighborhood I of K such that the projection Π K (•) is single-valued in I, with Π K (x) := {z ∈ K | ||x -z|| = inf y∈K ||x -y||} for all x ∈ R.
We will describe some of the properties of such sets. This will be the key for the proof of the existence and uniqueness results concerning [START_REF] Barron | Semicontinuous viscosity solutions of Hamilton-Jacobi equations with convex Hamiltonian[END_REF] and (HJBI). The class of proximal retracts includes closed, convex subsets of R N and submanifolds of R N of class C 1,1 . Another class of proximal retracts is the class of weakly convex sets (see [START_REF] Frankowska | A viability approach to the Skorohod problem[END_REF] for the definition and the geometrical interpretation). A complete characterization of proximal retract sets is made in [START_REF] Poliquin | Local differentiability of distance functions[END_REF] (see Theorem 4.1, p. 5245). In particular, such sets have the property that there exists ρ > 0 such that every nonzero normal "can be realized" by a ball with a radius equal to ρ. This characterization says, in particular, that only "exterior" corners are allowed.

So, if K is proximal retract, then from Theorem 4.1 in [START_REF] Poliquin | Local differentiability of distance functions[END_REF], Lemma 4.2 and Theorem 2.2 in [START_REF] Cornet | Existence of slow solutions for a class of differential inclusions[END_REF] we have the following:

-There exist r, c > 0 such that the application x → N K (x) ∩ B(0, r) + cx is monotone1 on K. This monotonicity property, which is equivalent to Definition 2, is very important because it allows us to establish the uniqueness of solutions to [START_REF] Aubin | Viability Theory[END_REF].

-The set K is sleek, i.e., the map x → T K (x) is l.s.c.

-For all x ∈ K, T K (x) = C K (x), where C K (x) denotes Clarke's tangent cone.2 Note that the class of sleek sets is larger then the class of proximal retracts.

Viscosity solutions.

To describe the value function as a unique solution to the corresponding HJBI, we introduce the following definition of solutions to (HJBI).

Definition 3. A viscosity supersolution of (HJBI) is an l.s.c. function ψ : (0, T ) × K → R such that for any φ ∈ C 1 and (t 0 , x 0 ) ∈ arg min (ψφ) ,

if x 0 ∈ intK, ∂φ ∂t (t 0 , x 0 )+ H x 0 , ∂φ ∂x (t 0 , x 0 ) ≤ 0 and if x 0 ∈ ∂K, there exists y 0 ∈ N K (x 0 ) such that ∂φ ∂t (t 0 , x 0 )+ H x 0 , ∂φ ∂x (t 0 , x 0 ) -y 0 , ∂φ ∂x (t 0 , x 0 ) ≤ 0
and a viscosity subsolution of (HJBI) is a u.s.c. function ϕ : (0, T ) × K → R such that for any φ ∈ C 1 and (t 0 , x 0 ) ∈ arg max (ϕφ) ,

if x 0 ∈ intK, ∂φ ∂t (t 0 , x 0 )+ H x 0 , ∂φ ∂x (t 0 , x 0 ) ≥ 0 and if x 0 ∈ ∂K, there exists z 0 ∈ N K (x 0 ) such that ∂φ ∂t (t 0 , x 0 )+ H x 0 , ∂φ ∂x (t 0 , x 0 ) -z 0 , ∂φ ∂x (t 0 , x 0 ) ≥ 0.
A viscosity solution of (HJBI) is a function which is both subsolution and supersolution.

It is clear that a viscosity solution is a continuous function because it is simultaneously u.s.c. and l.s.c. Remark 4. A motivation for our definition of (HJBI) is the fact that, when (t 0 , x 0 ) is a differentiability point of V , we have in the usual sense

∂V ∂t (t 0 , x 0 )+ H x 0 , ∂V ∂x (t 0 , x 0 ) -N K (x 0 ), ∂V ∂x (t 0 , x 0 ) 0. (7) Indeed, there exists λ ∈ [0, 1] such that 0 = λ ∂V ∂t (t 0 , x 0 )+ H x 0 , ∂V ∂x (t 0 , x 0 ) -y 0 , ∂V ∂x (t 0 , x 0 ) + (1 -λ) ∂V ∂t (t 0 , x 0 )+ H x 0 , ∂V ∂x (t 0 , x 0 ) -z 0 , ∂V ∂x (t 0 , x 0 ) , and because N K (x 0 ) is convex, (7) is verified.
It is quite natural to obtain an equation of the form [START_REF] Evans | Partial Differential Equations[END_REF], namely a partial differential inequality. The motivation lies in the fact that for a smooth set, the reflection is channeled in a fixed direction, given by the outward normal. For nonsmooth sets the outward normal will be replaced with the normal cone which, in general, contains many directions.

Note that this definition contains those given by Lions in [START_REF] Lions | Neumann type boundary conditions for Hamilton-Jacobi equations[END_REF], when the boundary of K, ∂K ∈ C 1 .

Control systems with reflection on the boundary of a constraint set.

In this section we study the differential inequalities ( 5) and ( 6) by explaining the method which we use in order to get a boundary reflection for closed sets K. This allows us to give some applications to the properties of solutions to the controlled system [START_REF] Aubin | Viability Theory[END_REF].

We consider a closed set K, a set valued map F : K → R N , and the following differential inclusion:

(i) x (t) ∈ F (x(t)) for almost all t ≥ t 0 , (ii) x(t 0 ) = x 0 ∈ K, t 0 ≥ 0. ( 8
)
The equation ( 6) appears naturally if we want a given closed set to become a viability3 domain of a new system which is "as close as possible" to the original dynamic system [START_REF] Frankowska | A viability approach to the Skorohod problem[END_REF].

Indeed, when the necessary and sufficient condition for the existence of viable solutions

F (x) ∩ T K (x) = ∅ for all x ∈ K
is not satisfied, the natural way to solve the above problem is to introduce the projected problem [START_REF] Cornet | Existence of slow solutions for a class of differential inclusions[END_REF].

We note that Π coT K (x) F (x) = F (x) whenever x ∈ intK; F is modified only on the boundary of K, so ( 6) is a problem of reflection at the boundary. Moreover, the application x → Π coT K (x) F (x) has no easy continuity properties, but, thanks to the properties of the projection on a convex cone, it is possible to prove that the solutions to ( 5) and ( 6) coincide. We do not make any assumption on the regularity of the set K, improving already known results of [START_REF] Aubin | Viability Theory[END_REF] where the set K is sleek.

It is easier to find sufficient conditions for the set K in order to obtain continuity properties of the right-hand side of [START_REF] Barron | Semicontinuous viscosity solutions of Hamilton-Jacobi equations with convex Hamiltonian[END_REF]. So, for the study of existence and uniqueness of solutions we consider [START_REF] Barron | Semicontinuous viscosity solutions of Hamilton-Jacobi equations with convex Hamiltonian[END_REF]. We have the following proposition.

Proposition 5. (i) Suppose that K is closed and F is a set valued map. Then the sets of absolutely continuous solutions to (5) and (6) are equal.

Moreover if F is a Marchaud map 4 and K is bounded and sleek, then (ii) for every (t 0 , x 0 ) ∈ [0, ∞) × K there exists a solution of [START_REF] Barron | Semicontinuous viscosity solutions of Hamilton-Jacobi equations with convex Hamiltonian[END_REF] or equivalently of [START_REF] Cornet | Existence of slow solutions for a class of differential inclusions[END_REF].

(iii) the restriction of the map

(t 0 , x 0 ) ∈ [0, T ] × K → S F (t 0 , x 0 ) to a compact set C is compact into [0, ∞) × K × W 1,1 (0, ∞ ; K)e -bt
for all b with b > a. Here S F (t 0 , x 0 ) denotes the set of solutions to (5) starting from (t 0 , x 0 ).

Before giving the proof, we note that the first part of the above proposition is a generalization of Theorem 10.1.1 in [START_REF] Aubin | Viability Theory[END_REF] where the set K is supposed to be sleek; here K is only bounded. The second part recalls well-known existence and compactness results (see [START_REF] Aubin | Differential Inclusions[END_REF] and [START_REF] Aubin | Viability Theory[END_REF]).

Proof. (i) Using Proposition 0.6.4 from [START_REF] Aubin | Differential Inclusions[END_REF] we deduce that

Π coT K (x) F (x) ⊂ F (x) -N K (x) for all x ∈ K,
and, consequently, a solution to ( 6) is also a solution to [START_REF] Barron | Semicontinuous viscosity solutions of Hamilton-Jacobi equations with convex Hamiltonian[END_REF].

Conversely, if x(t) ∈ K for all t ≥ t 0 , we have

lim h→0 x(t + h) -x(t) h ∈ T K (x(t)) and lim h→0 x(t) -x(t -h) h ∈ -T K (x(t)) for a.e. t ≥ t 0 , so x (t) ∈ T K (x(t)) ∩ -T K (x(t)) ⊂ N K (x(t)) ⊥ a.e. t ≥ t 0 .
Let t ≥ t 0 be a derivability point of x(•), and let

x (t) = f (t) -p(t) with f (t) ∈ F (x(t)) and p(t) ∈ N K (x(t)).
The above arguments say that

x (t) -f (t), x (t) = 0. Thus, x (t) ∈ Π coT K (x(t)) f (t) ⊂ Π coT K (x(t)) F (x(t)) for a.e. t ≥ t 0 .
(ii) We prove now the existence of a solution to the differential inequality [START_REF] Barron | Semicontinuous viscosity solutions of Hamilton-Jacobi equations with convex Hamiltonian[END_REF]. If K is sleek, then the map x → N K (x) has a closed graph.

For all x ∈ K, we set

H(x) = F (x) -a(1 + ||x||)B ∩ N K (x)
, where B denotes the unit ball of R N .

Because the map x → a(1

+ ||x||)B ∩ N K (x)
is Marchaud, H is also Marchaud. Hence by Theorem 2.1.3 in [START_REF] Aubin | Differential Inclusions[END_REF] the existence of solutions of (5) follows.

Let us prove that the closed subset K is a viability domain for the differential inclusion [START_REF] Barron | Semicontinuous viscosity solutions of Hamilton-Jacobi equations with convex Hamiltonian[END_REF].

Indeed, using the equality

I -Π N K (x) = Π T K (x) , we have that for any x ∈ K and f ∈ F (x), f -Π N K (x) f ∈ (F (x) -N K (x)) ∩ T K (x).
Using the estimation

||Π N K (x) f || ≤ ||f || ≤ a(1 + ||x||), we get that Π N K (x) f ∈ a(1 + ||x||)B ∩ N K (x) and consequently f -Π N K (x) f ∈ H(x) ∩ T K (x).
The above arguments say that H satisfies the hypotheses of viability theorem 4.2.1 of [START_REF] Aubin | Differential Inclusions[END_REF], and since H(x) ⊂ F (x) -N K (x), the second part ensues.

(iii) See Theorem 2.2.1 in [START_REF] Aubin | Differential Inclusions[END_REF]. Now, let us begin a short study of the optimal control problem with reflected trajectories. From now on, we consider that the set valued map F is given by the equality

F (x) = f (x, U ) = {f (x, u), u ∈ U } for all x ∈ K.
We denote by S f (t 0 , x 0 ) the set of absolutely continuous solutions to

(i) x (t) ∈ f (x(t), u(t)) -N K (x(t)) for almost all t ≥ t 0 , (ii) x(t) ∈ K for all t ≥ t 0 , x(t 0 ) = x 0 ∈ K for all u(•) ∈ U(t 0 ), (9)
and by S F (t 0 , x 0 ) the set of absolutely continuous solutions to

(i) x (t) ∈ F (x(t)) -N K (x(t)) for almost all t ≥ t 0 , (ii) x(t) ∈ K for all t ≥ t 0 , x(t 0 ) = x 0 ∈ K. ( 10
)
We will prove now that ( 9) and [START_REF] Frankowska | Measurable viability theorems and the Hamilton-Jacobi-Bellman equation[END_REF] are equivalent. In this paper, we will use one of these systems to simplify our proofs. Proposition 6. Suppose that K is a compact sleek set and (H f ) holds.

(i) If x(•) is a solution to [START_REF] Frankowska | Measurable viability theorems and the Hamilton-Jacobi-Bellman equation[END_REF] starting from (t 0 , x 0 ) ∈ [0, T ] × K, then there exists u(•) ∈ U(t 0 ) such that x(•) is equal to x(•; t 0 , x 0 , u(•)), the solution of [START_REF] Frankowska | Lower semicontinuous solutions of the Hamilton-Jacobi-Bellman equations[END_REF].

(ii) As a direct consequence of (i),

S F (t 0 , x 0 ) = S f (t 0 , x 0 ) for all (t 0 , x 0 ) ∈ [0, ∞) × K.
Proof. We essentially use the fact that K is sleek (which implies that the application x → N K (x) has a closed graph) and Theorem 1.14.1 from [START_REF] Aubin | Differential Inclusions[END_REF]. Consider Φ(t

) := {v ∈ U | x (t) ∈ f (x(t), v) -N K (x(t)
)} for a.e. t ≥ t 0 . We can prove that the multivalued function Φ has a measurable selection which gives our measurable control u(•) ∈ U(t 0 ).

Moreover, with an easy computation, using the fact that K is proximal retract and Gronwall's inequality, we obtain the following estimation.

Lemma 7. Assume that (H f ) holds true and K is a bounded proximal retract. Then for

x 0 (•) ∈ S f (t 0 , x 0 ), x 1 (•) ∈ S f (t 1 , x 1 ) with fixed u(•) ∈ U(t 0 ) and for t ≥ t 1 ≥ t 0 , there exists C > 0, a constant depending on t, such that x 0 (t; t 0 , x 0 , u(•)) -x 1 (t; t 1 , x 1 , u(•)) ≤ C( x 0 -x 1 + |t 0 -t 1 |).
We omit the proof of Lemma 7 because it is an easy adaptation of Lemma 4.4, p. 143, proved in [START_REF] Cornet | Existence of slow solutions for a class of differential inclusions[END_REF]. As a direct consequence of the above estimation we obtain the following.

Corollary 8. Assume that (H f ) holds true and K is a bounded proximal retract. Then for fixed u(•) ∈ U(t 0 ) there exists an unique solution of (2).

2.4.

The optimal control problem. First, we give some standard results concerning the regularity of V without proof. Later we shall prove the existence and the uniqueness of viscosity solutions of (HJBI).

Lemma 9. Suppose that (H f ) holds true and K is a compact proximal retract. Then we have the following:

(i) (Existence of an optimal control.) If g is l.s.c., then V is l.s.c. and there exists an optimal trajectory starting from each point (t 0 , x 0 ) ∈ [0, T ] × K, i.e., there exists x(•) ∈ S F (t 0 , x 0 ) such that

V (t 0 , x 0 ) = g(x(T ; t 0 , x 0 , ū(•))) for all (t 0 , x 0 ) ∈ [0, T ] × K.
(ii) If g is a Lipschitz function, then V is locally Lipschitz and bounded. Next we give the Bellman dynamic programming. Proposition 10 (dynamic programming principle). Let g : K → R be a bounded function, K a compact proximal retract, and suppose that (H f ) holds. Then, for all (t 0 , x 0 ) ∈ [0, T ] × K we have

V (t 0 , x 0 ) = inf x∈S F (t 0 ,x 0 ) V (t 0 + h, x(t 0 + h)) with h > 0 small enough. ( 11
)
3. The Hamilton-Jacobi partial differential variational inequality.

3.1. Existence result. The aim of this section is to provide an existence and a comparison result for viscosity solutions to a partial differential inequality with a kind of boundary conditions for nonsmooth sets, which generalizes first order Hamilton-Jacobi equations with Neumann conditions for smooth sets.

Using the dynamic programming principle we prove that the value function for the control problem (1), ( 2) is the viscosity solution of (HJBI) in the sense of Definition 3.

Proposition 11. If K is a compact proximal retract, g a Lipschitz function, and (H f ) holds true, then V is a locally Lipschitz viscosity solution of (HJBI) with the final condition V (T, x) = g(x) for all x ∈ K.

This theorem can be considered as an existence result of solutions to (HJBI).

Proof. First we prove that V is a supersolution. We consider (t 0 , x 0 ) ∈ arg min(Vψ), ψ ∈ C 1 , with

V (t 0 , x 0 ) = ψ(t 0 , x 0 ) and V (t, x) ≥ ψ(t, x)
in a neighborhood of (t 0 , x 0 ). For all h > 0 small enough, there exists x h (•) ∈ S F (t 0 , x 0 ) such that

ψ(t 0 , x 0 ) + h 2 = V (t 0 , x 0 ) + h 2 ≥ V (t 0 + h, x h (t 0 + h)) ≥ ψ(t 0 + h, x h (t 0 + h)).
For a subset A of R N we denote by A,ε) denotes the neighborhood of the set A with a radius equal to ε > 0.

B(A, ε) = {x ∈ R N | inf y∈A ||y -x|| ≤ ε}. B(
Let M be a bound of F on K. Using the Lipschitz property of F (•) and the upper semicontinuity of N K (•) ∩ B(0, M), we have that, for all ε > 0, there exists an h > 0 small enough such that the following inclusions hold:

1 (x h (t 0 + h) -x 0 ) ∈ 1 h t 0 +h t 0 (F (x h (s)) -N K (x h (s)) ∩ B(0, M))ds ⊂ 1 h t 0 +h t 0 (F (x 0 ))ds + B(0, 1) 1 h t 0 +h t 0 L||x h (s) -x 0 ||ds - 1 h t 0 +h t 0 B(N K (x 0 ) ∩ B(0, M), ε)ds = F (x 0 ) + B(0, 1) 1 h t 0 +h t 0 L||x h (s) -x 0 ||ds -B(N K (x 0 ) ∩ B(0, M), ε).
Hence for all ε > 0, there exists a sequence h n such that lim n→∞ h n = 0 and

lim n 1 h n (x h n (t 0 + h n ) -x 0 ) ∈ F (x 0 ) -B(N K (x 0 ) ∩ B(0, M), ε).
Letting ε → 0 we obtain that

lim n 1 h n (x h n (t 0 + h n ) -x 0 ) ∈ F (x 0 ) -N K (x 0 ) ∩ B(0, M). (12) Moreover, lim n 1 h n [ψ(t 0 + h n , x h n (t 0 + h n ; t 0 , x 0 , u(•))) -ψ(t 0 , x 0 )] -h n (13) = lim n 1 h n ψ t 0 + h n , x 0 + h n 1 h n (x h n (t 0 + h n ) -x 0 ) -ψ(t 0 , x 0 ) -h n .
Using ( 12) and ( 13) we have the following.

First case (x 0 ∈ intK). Then N K (x 0 ) = {0} and there exists u ∈ U such that ∂ψ ∂t (t 0 , x 0 ) + ∂ψ ∂x (t 0 , x 0 ), f(x 0 , u) ≤ 0, and consequently ∂ψ ∂t (t 0 , x 0 ) + inf u∈U ∂ψ ∂x (t 0 , x 0 ), f(x 0 , u) ≤ 0.

Second case (x 0 ∈ ∂K). Then {0} ⊂ N K (x 0 ) and there exist u ∈ U , y u ∈ N K (x 0 ) such that:

∂ψ ∂t (t 0 , x 0 ) + ∂ψ ∂x (t 0 , x 0 ), f(x 0 , u) -y u ≤ 0.
So, there exists

w 0 = y u ∈ N K (x 0 ) ∩ B(0, M) such that ∂ψ ∂t (t 0 , x 0 ) + inf u∈U ∂ψ ∂x (t 0 , x 0 ), f(x 0 , u) -w 0 , ∂ψ ∂x (t 0 , x 0 ) ≤ 0
and V is a supersolution. The proof of the fact that V is subsolution is similar and we omit it.

Uniqueness result.

This section concerns the uniqueness of the viscosity solutions of (HJBI). The importance of this result leads us to treat it separately. Moreover, the characterization of the value function as the unique solution of (HJBI) ensues.

Theorem 12 (uniqueness result in the Lipschitz case). Assume that (H f ) holds true. Let K be a compact proximal retract and g be a Lipschitz function. Then there exists at most one uniformly continuous viscosity solution of (HJBI) which satisfies the final condition V (T, x) = g(x) for all x ∈ K.

The proof can be adapted from Evans [START_REF] Evans | Partial Differential Equations[END_REF]. We only underline that the difference to Evans' proof is due to the monotonicity of the multivalued function x → N K (x) ∩ B(0, M) + cx.

The discontinuous case.

In this section we investigate the value function V when g : K → R is supposed to be bounded. In this case the value is only a bounded function. A natural question is how to use the viscosity theory to describe V. Here we establish a relation between the value and the viscosity sub or supersolutions of (HJBI). This kind of problem has been studied for the Bolza problem in [START_REF] Plaskacz | Discontinuous Mayer problem under stateconstraints[END_REF], [START_REF] Plaskacz | Value-functions for differential games and control systems with discontinuous terminal cost[END_REF].

The main point of this section is to prove the following. Theorem 13. Suppose that K is a proximal retract and (H f ) holds.

(i) If g is bounded, then for every (t, x) ∈ [0, T ] × K V (t, x) = inf{ψ(t, x)| ψ l.s.c. supersolution of (HJBI); ψ(T, •) ≥ g(•)} and V (t, x) = sup{ϕ(t, x)| ϕ u.s.c. subsolution of (HJBI); ϕ(T, •) ≤ g(•)}. (ii) If g is l.s.c., then V = min{ψ | ψ l.s.c. supersolution of (HJBI); ψ(T, •) ≥ g(•)}.
5. On l.s.c. solutions of HJBI with reflection on smooth sets. If g : R N → R is an l.s.c. function, then V is also l.s.c. In [START_REF] Barron | Semicontinuous viscosity solutions of Hamilton-Jacobi equations with convex Hamiltonian[END_REF], [START_REF] Frankowska | A viability approach to the Skorohod problem[END_REF] a modification of the concept of viscosity solutions for semicontinuous functions was proposed. This approach is based on a construction of "touching from one side" functions, which is usual for viscosity solutions theory.

We suppose that K is a C 1,1 submanifold with boundary. If we denote by n(x) the unit outward normal to K at x ∈ ∂K, the normal cone N K (x) is generated by n(x), i.e., N K

(x) = [0, ∞)n(x) for x ∈ ∂K and N K (x) = {0} for x ∈ intK.
We propose a definition for l.s.c. solutions to the HJBI of Barron-Jensen-Frankowska type.

Definition 16. A viscosity l.s.c. solution of (HJBI) is a function ψ : [0, T ]×K → R such that for any φ ∈ C 1 and (t 0 , x 0 ) ∈ arg min (ψφ) ,

if (t 0 , x 0 ) ∈ [0, T ) × intK, we have ∂φ ∂t (t 0 , x 0 )+ H x 0 , ∂φ ∂x (t 0 , x 0 ) ≤ 0; if (t 0 , x 0 ) ∈ (0, T ] × intK, we have ∂φ ∂t (t 0 , x 0 )+ H x 0 , ∂φ ∂x (t 0 , x 0 ) ≥ 0; if (t 0 , x 0 ) ∈ [0, T ) × ∂K, then there exists u ∈ U such that ∂φ ∂t (t 0 , x 0 ) + (f (x 0 , u) -Π N K (x) f (x 0 , u)), ∂φ ∂x (t 0 , x 0 ) ≤ 0; if (t 0 , x 0 ) ∈ (0, T ] × ∂K and min u∈U f (x 0 , u), n(x 0 ) > 0, then for all u ∈ U, ∂φ ∂t (t 0 , x 0 ) + (f (x 0 , u) -Π N K (x) f (x 0 , u)), ∂φ ∂x (t 0 , x 0 ) ≥ 0.
Note that in intK the equation is satisfied in the Barron-Jensen-Frankowska sense (see [START_REF] Barron | Semicontinuous viscosity solutions of Hamilton-Jacobi equations with convex Hamiltonian[END_REF], [START_REF] Frankowska | A viability approach to the Skorohod problem[END_REF]).

We obtain the following uniqueness results. Proposition 17. Suppose that K is a C 1,1 submanifold with boundary and for any u ∈ U and for all x 0 ∈ K we have f (x 0 , u), n(x 0 ) < 0. If g is l.s.c. and (H f ) holds true, then the value function V is the unique l.s.c. viscosity solution of (HJBI) which verifies the final condition V (T, x) = g(x) for all x ∈ K, and for all (t, x) ∈ (0, T ] × ∂K we have lim inf

(t ,x )→(t,x) x∈intK V (t , x ) = V (t, x).
The proof is similar to the proof of Theorem 2.3 in [START_REF] Frankowska | A viability approach to the Skorohod problem[END_REF]. Proposition 18. Suppose that K is a C 1,1 submanifold with boundary and for any u ∈ U and for all x 0 ∈ K we have f (x 0 , u), n(x 0 ) > 0. If g is l.s.c. and (H f ) holds true, then the value function V is the unique l.s.c. viscosity solution of (HJBI) which verifies the final condition V (T, x) = g(x) for all x ∈ K, and for all (t, x) ∈ (0, T ] × ∂K we have lim inf

(t ,x )→(t,x) x∈intK V (t , x ) = V (t, x).
Proof.

Step 1. V satisfies Definition 16. The proof of the first inequality is similar to the proof of the fact that V is an l.s.c. supersolution of (HJBI).

For proving the second inequality we observe that for all x 0 ∈ ∂K and u ∈ U ,

Φ u (x 0 ) := f (x 0 , u) -Π N K (x) f (x 0 , u) ∈ Π T K (x) f (x 0 , u) = Π ∂T K (x) f (x 0 , u). Consequently, -f (x 0 , u) + Π N K (x) f (x 0 , u) ∈ -Π ∂T K (x) f (x 0 , u). As K is a C 1,1 submanifold and for all x 0 ∈ K, min u∈U f (x 0 , u), n(x 0 ) > 0, Φ u (•) is a Lipschitz application on ∂K. Moreover, ∂K is locally invariant (see [2, viability theorem 3.2.4] by Φ u (•) and by -Φ u (•) (because ∂T K (x) = -∂T K (x)). Now let (t 0 , x 0 ) ∈ arg min (V -φ) , φ ∈ C 1 .
We have two cases.

First case (x 0 ∈ ∂K). For a fixed constant control u ∈ U , we consider the solution of

x (t) = -f (x(t), u) + Π N K (x(t)) f (x(t), u), x(t 0 ) = x 0 ,
which stays in ∂K because of the invariance properties of Φ u (•). Using the dynamic programming principle we get

V (t 0 , x 0 ) ≥ V (t 0 -h, x(t 0 -h)) with h > 0 small enough. So, φ(t 0 , x 0 ) ≥ φ(t 0 -h, x(t 0 -h)) with h > 0 small enough. Recall that φ ∈ C 1 and consequently ∂φ ∂t (t 0 , x 0 ) + (f (x 0 , u) -Π N K (x) f (x 0 , u)), ∂φ ∂x (t 0 , x 0 ) ≥ 0.
Second case (x 0 ∈ intK). N K (x 0 ) = {0} and for all u ∈ U, because f is a Lipschitz application, there exists B(x 0 ; r u ), r u > 0, such that the solution to

x (t) = -f (x(t), u),
x(t 0 ) = x 0 stays in B(x 0 ; r u ). Using the dynamic programming principle, for h > 0 small enough

V (t 0 , x 0 ) ≥ V (t 0 -h, x(t 0 -h)) so φ(t 0 , x 0 ) ≥ φ(t 0 -h, x(t 0 -h)). Because φ ∈ C 1
we obtain ∂φ ∂t (t 0 , x 0 ) + (f (x 0 , u), ∂φ ∂x (t 0 , x 0 ) ≥ 0. This allows us to say that V is an l.s.c. solution of (HJBI).

Step 2 (uniqueness). Now let us prove that V is the unique l.s.c. solution of (HJBI). Let W be an l.s.c. solution of (HJBI) with W (T, x) = g(x) for all x ∈ K. We have already proved (see Theorem 13) that W ≥ V.

For the reverse inequality we consider (t 0 , x 0 ) ∈ (0, T ) × K and x(•) ∈ S F (t 0 , x 0 ). There exists u(•) ∈ U(t 0 ) such that x(•) = x(•; t 0 , x 0 , u(•)). We have two cases.

First case (x(T ) ∈ intK). For a fixed u(•) ∈ U(t 0 ), ∂K is invariant by Φ u(.) (•) and Φ u(.) (•) is Lipschitz in the second variable, so we have that x([t 0 , T ]) ⊂ intK.

By the measurable viability theorem (see Theorem 4.7 in [START_REF] Frankowska | Measurable viability theorems and the Hamilton-Jacobi-Bellman equation[END_REF], [START_REF] Aubin | Viability Theory[END_REF]) Epi(W ) is viable for the dynamics given by (t, x, y) → (-1, -f (x(t), u(t)), 0). For the solution starting from (T, x(T ), W (T, x(T ))), we have for all t ∈ [t 0 , T ], W (Tt, x(Tt)) ≤ W (T, x(T )), so W (t 0 , x 0 ) ≤ W (T, x(T )) = g(x(T )).

Second case (x(T ) ∈ ∂K). Denote by τ the first time with the property x(τ ) ∈ ∂K. Using invariance properties of Φ u(.) (•) and because Φ u(.) (•) is Lipschitz in the second variable, we obtain that x([t 0 , τ )) ⊂ intK and x([τ, T ]) ⊂ ∂K.

As in the above case, we apply the measurable viability theorem (see Theorem 4.7 in [START_REF] Frankowska | Measurable viability theorems and the Hamilton-Jacobi-Bellman equation[END_REF], [START_REF] Aubin | Viability Theory[END_REF]) to Epi(W ), on the one hand, to [τ, T ] for the dynamics given by (t, x, y) → (-1, -f (x(t), u(t))+Π N K (x(t)) f (x(t), u(t)), 0) for the solution starting from (T, x(T ), W (T, x(T )) and, on the other hand, to [t 0 , τ n ) for the dynamics given by (t, x, y) → (-1, -f (x(t), u(t)), 0) for the solution starting from (τ, x(τ ), W (τ, x(τ )) = lim n (τ n , x n , W (τ, x(τ )), x n ∈ intK.

We have W (t 0 , x 0 ) ≤ W (τ, x(τ )) and W (τ, x(τ )) ≤ W (T, x(T )) = g(x(T )). Consequently, by definition of the value function W (t 0 , x 0 ) ≤ V (t 0 , x 0 ).

We note that here we can obtain uniqueness for l.s.c. solutions only in two (extremal) cases, where the vector field f (x, u) is pointing only outside of the domain or only inside. For the intermediate situation it seems that we cannot obtain uniqueness (see the counterexample given below). The lack of uniqueness can be a consequence of the fact that in the intermediate situation we lose the Lipschitz regularity of Φ u (•) in ∂K and the idea of the above proof will fail.

Counterexample. Now we will show that a uniqueness result is not possible using our definition without imposing boundary properties on our dynamics as we did in the above propositions. We do this by giving a counterexample.

Let K = [0, 1] ⊂ R. For a dynamics given by f = 0 and g = 1 the value function is V (t, x) = 1 for all (t, x) ∈ [0, 1] × [0, 1]. Moreover V is an l.s.c. solution of HJBI in the sense of our definition. Define

u(t, x) = 1 if (t, x) ∈ [0, 1] × (0, 1], 0 if (t,x) ∈ [0, 1] × {0}.
It is easy to verify that u is also an l.s.c. solution for the HJBI and we do not have uniqueness because f (x 0 ), n(x 0 ) = 0 for all x 0 ∈ ∂K.

For another definition of the discontinuous solution Ley [START_REF] Ley | A counter-example to the characterization of the discontinuous value function of control problems with reflection[END_REF] obtained a counterexample proving that there is no uniqueness to HJB with a notion of the solution in the Ishii-Barles-Perthame sense.

Appendix.

Let us give the proof of Lemma 14 and Lemma 15. We shall use the following classical viability theorem and the fact that the definition of super and subsolutions to (HJBI) can be written equivalently in terms of subdifferentials. (See [START_REF] Plaskacz | Discontinuous Mayer problem under stateconstraints[END_REF] to get formulations of viscosity solutions in terms of subdifferentials of the PDE associated to the Mayer control problem with K = R N .)

Theorem 19 (see [2, viability theorem 3.2.4]). Assume that G is a Marchaud map and let D ⊂ R N be closed. If for every z ∈ D we have for all p ∈ N D (z), min y∈G(z) y, p ≤ 0, [START_REF] Plaskacz | Discontinuous Mayer problem under stateconstraints[END_REF] then for every x 0 ∈ D, t 0 < T, there exists a solution x(•) to the Cauchy problem

x (s) ∈ G(x(s)), x(t 0 ) = x 0 such that x(t) ∈ D for all t ∈ [t 0 , T ].
Now we give the proof of Lemma 14 and Lemma 15.

Proof of Lemma 14. Fix t 0 ∈ (0, T ). We set

D ψ = cl({(t, x, r) : t ∈ (0, T ], x ∈ K, r ≥ ψ(t, x)}) ∪ [T, ∞) × K × R, F (t, x, r) = ⎧ ⎪ ⎪ ⎨ ⎪ ⎪ ⎩ 0 if t < 0, t t 0 (1, F (x) -N K (x) ∩ B(0, M), 0) if t ∈ [0, t 0 ], (1, F (x) -N K (x) ∩ B(0, M), 0) if t ∈ [t 0 , T ], (1, F (x) -N K (x) ∩ B(0, M), 0) if t > T,
where cl denote the closure and M is a bound of F on K. We show that (15) holds true for F and D ψ . First case (x 0 ∈ intK). Let z 0 = (s 0 , x 0 , r 0 := ψ(t 0 , x 0 )) ∈ D ψ . If s 0 = 0, then F = 0. Obviously (15) holds.

If s 0 ≥ T and (p s , p x , p r ) ∈ N D ψ (s 0 , x 0 , r 0 ), then p s ≤ 0, p x = 0, p r = 0. Hence [START_REF] Plaskacz | Discontinuous Mayer problem under stateconstraints[END_REF] holds.

It remains to consider the case s 0 ∈ (0, T ). We have N D ψ (s 0 , x 0 , r 0 ) ⊂ N D ψ (s 0 , x 0 , ψ(s 0 , x 0 )). Let (p s , p x , p r ) ∈ N D ψ (s 0 , x 0 , ψ(s 0 , x 0 )).

If p r < 0, then (p s /p r , p x /p r ) ∈ ∂ -ψ(s 0 , x 0 ) (see Proposition 4.1 in [START_REF] Frankowska | Lower semicontinuous solutions of the Hamilton-Jacobi-Bellman equations[END_REF]). Since ψ is a supersolution of (HJBI) there exists y 0 ∈ N K (x 0 ) such that

p s -p r + min z∈F (x 0 ) z, p x -p r -y 0 , p x -p r ≤ 0 and p s -p r + min z∈{F (x 0 )-N K (x 0 )∩B(0,M )} z, p x -p r ≤ 0.
Hence min ỹ∈ F (s 0 ,x 0 ,r 0 ) ỹ, (p s , p x , p r ) ≤ 0. Now we consider the case p r = 0. By a Rockafellar's lemma (see, for instance, Lemma 4.2 in [START_REF] Frankowska | Lower semicontinuous solutions of the Hamilton-Jacobi-Bellman equations[END_REF]) there exists a sequence s n → s 0 , x n → x 0 , and

p s n → p s , p x n → p x , p r n → 0, p r n < 0 such that (p s n , p x n , p r n ) ∈ N D ψ (p s n , p x n , p r n ). Since p r n < 0 we obtain from the previous case that min ỹn ∈ F (s n ,x n ,r n ) ỹ, (p s n , p x n , p r n ) ≤ 0.
We get min ỹ∈ F (s 0 ,x 0 ,r 0 ) ỹ, (p s , p x , p r ) ≤ 0, because F is Marchaud. Second case (x 0 ∈ ∂K). Let z 0 = (s 0 , x 0 , r 0 := ψ(t 0 , x 0 )) ∈ D ψ . If s 0 = 0, then F = 0. Obviously (15) holds true.

If s 0 ≥ T and (p s , p x , p r ) ∈ N D ψ (s 0 , x 0 , r 0 ), then p s ≤ 0, p x ∈ N K (x 0 ), p r = 0. Hence [F (x 0 ) -N K (x 0 )] ∩ T K (x 0 ) = ∅ and (15) holds.

It remains to consider s 0 ∈ (0, T ), which is similar to the first case.

Finally we obtain min ỹ∈ F (s 0 ,x 0 ,r 0 ) ỹ, (p s , p x , p r ) ≤ 0.

In view of the above theorem we have a solution z(•) to the Cauchy problem z (s) ∈ F (z(s)), z(t 0 ) = z 0 . Let z(s) = (t(s), x(s), r(s)).

By the definition of F we have t(s) = s, r(s) = r 0 = ψ(t 0 , x 0 ). Hence, ( 14) holds true and the proof is completed.

Proof of Lemma 15. The proof is divided into several steps. First step. We fix t 0 ∈ (0, T ), u(•) ∈ U(t 0 ) such that u(•) is a continuous function. We set

D ϕ = cl({(t, x, r) : t ∈ (0, T ], x ∈ K, r ≤ ϕ(t, x)}) ∪ [T, ∞) × K × R, G(t, x, r) = ⎧ ⎪ ⎪ ⎨ ⎪ ⎪ ⎩ 0 if t < 0, t t 0 (1, F (x, u(t)) -N K (x) ∩ B(0, M), 0) if t ∈ [0, t 0 ], (1, F (x, u(t)) -N K (x) ∩ B(0, M), 0) if t ∈ [t 0 , T ],
(1, F (x, u(t)) -N K (x) ∩ B(0, M), 0) if t > T, and we want to prove that for every (t 0 , x 0 , ϕ(t 0 , x 0 )) ∈ (0, T ) × K × R the solution x(•; t 0 , x 0 , u(•)) to (2) satisfies ϕ(t, x(t)) ≥ ϕ(t 0 , x 0 ) for all t ∈ [t 0 , T ]. [START_REF] Plaskacz | Value-functions for differential games and control systems with discontinuous terminal cost[END_REF] Since ϕ is a u.s.c. viscosity subsolution of (HJBI) we have for any φ ∈ C 1 and (t 0 , x 0 ) ∈ arg max (ϕφ) , there exists z 0 ∈ N K (x 0 ) such that ∂φ ∂t (t 0 , x 0 )+ H x 0 , ∂φ ∂x (t 0 , x 0 )z 0 , ∂φ ∂x (t 0 , x 0 ) ≥ 0, so -∂φ ∂t (t 0 , x 0 )+ min y 0 ∈N K (x 0 )∩B(0,M ) (f (x 0 , u(t 0 ))y 0 ), -∂φ ∂x (t 0 , x 0 ) ≤ 0.

Using the same arguments as we used in the proof of the above lemma and because G is a Marchaud map, we obtain min y 0 ∈N K (x 0 )∩B(0,M ) (f (x 0 , u(t 0 ))y 0 ), (p s , p x , p r ) ≤ 0.

So, for every (p t , p x , p r ) ∈ N D ϕ (t 0 , x 0 , ϕ(t 0 , x 0 )), min ỹ∈G(t 0 ,x 0 ,ϕ(t 0 ,x 0 )) ỹ, (p s , p x , p r ) ≤ 0.

Using Theorem 19 we obtain that for every (t 0 , x 0 , ϕ(t 0 , x 0 )) ∈ (0, T ) × K × R the solution x(•; t 0 , x 0 , u(•)) to ( 2) satisfies [START_REF] Plaskacz | Value-functions for differential games and control systems with discontinuous terminal cost[END_REF].

Second step. Fix u(•) ∈ U(t 0 ). Then there exists a sequence u n (•) ⊂ U(t 0 ) of continuous functions such that u n (•) → u(•) in L ∞ (0, T ; U ).

For all n ∈ N, x n (•; t 0 , x 0 , u n (•)) satisfies

x n (t) ∈ f (x n (s), u n (s))ds - Recall that the restriction of the application (t 0 , x 0 ) ∈ [0, T ] × K → S F (t 0 , x 0 ) to a compact set C is compact into [0, ∞) × K × W 1,1 (0, ∞ ; K)e -bt for all b with b > a. Since x n (•; t 0 , x 0 , u n (•)) ∈ S F (t 0 , x 0 ) there exists x(•) ∈ S F (t 0 , x 0 ) such that lim n→∞ x n (•) = x(•) in W 1,1 (0, T ; K).

Passing to the limit in [START_REF] Poliquin | Local differentiability of distance functions[END_REF] we obtain that for almost all t ∈ [t 0 , T ] x(t) ∈ x 0 + t t 0 f (x(s), u(s))ds-t t 0 N K (x(s))∩B(0, M))ds and consequently, x(•) = x(•; t 0 , x 0 , u(•)). Third step. Assume that x(•; t 0 , x 0 , ū(•)) is an optimal trajectory for V, starting from (t 0 , x 0 ) ∈ [0, T ] × K, i.e., V (t 0 , x 0 ) = g(x(T ; t 0 , x 0 , ū(•)).

Then there exists a sequence of continuous functions u n (•), such that u n (•) → ū(•) in L ∞ (0, T ; U ) and consequently x n (•; t 0 , x 0 , u n (•)) → x(•; t 0 , x 0 , ū(•)) in W 1,1 (0, T ; K).

t t 0 N 0 f 0 NN

 000 K (x n (s)) ∩ B(0, M)ds, or equivalentlyx n (t) ∈ x 0 + t t (x n (s), u n (s))ds -t t K (x n (s)) ∩ B(0, M)ds. (17)As B(0, M) is a compact set and the application N K (•)∩B(0, M) is u.s.c., there exists a measurable selection y n (•) ∈ N K (x n (s))∩B(0, M). Moreover, for all s ∈ [t 0 , T ] there existslim n→∞ y n (s) = y(s) ∈ N K (x(s)) ∩ B(0, M).Hence, by the Lebesgue theorem we obtain that K (x(s)) ∩ B(0, M)ds a.e. t ∈ [t 0 , T ].

† Laboratoire de Mathématiques, Unité CNRS, FRE-2218, Université de Bretagne Occidentale, 6 avenue Victor Le Gorgeu, BP 809, 29285 Brest cedex, France (Oana-Silvia.Serea@univ-brest.fr).

Recall that a set valued mapG : K → R N is monotone if y 1y

, x 1x 2 ≥ 0 for all y i ∈ G(x i ), i ∈ {1, 2}. 2 C K (x) = {v| lim h→0 + ,K x →x d K (x + hv)/h = 0}. This tangent cone is always convex.

Recall that a solution x(•) to (8) is called viable in K if x(t) ∈ K for all t ≥ 0. The set K is a viability domain for[START_REF] Frankowska | A viability approach to the Skorohod problem[END_REF] if for all x 0 ∈ K there exists a solution to (8) which is viable in K.

A set valued map F from R N onto R N is called Marchaud map if F is u.s.c. with nonempty compact convex values and has a linear growth.

(iii) If g is u.s.c., then V = max{ϕ | ϕ u.s.c. subsolution of (HJBI) ; ϕ(T, •) ≤ g(•)}.

Before giving the proof, we note that the above theorem allows us to get, in particular, a stronger uniqueness result. More precisely, if ψ is an l.s.c. supersolution and ϕ is a u.s.c. subsolution of (HJBI) satisfying ψ(T, •) ≥ ϕ(T, •) on K, then ψ ≥ ϕ on [0, T ] × K.

Proof. (i) Let ψ be an l.s.c. supersolution of (HJBI) with ψ(T, •) ≥ g(•). We want to prove that V ≤ ψ on [0, T ] × K. To do this we use the following lemma proved in the appendix.

Lemma 14. Assume that (H f ) holds true, K is a compact proximal retract, and ψ : (0, T ) × K → R is an l.s.c. viscosity supersolution of (HJBI). Then for every (t 0 , x 0 ) ∈ (0, T ) × K there exists a solution x(•; t 0 , x 0 , u(•)) of (2) such that

So we obtain that there exists an x(•) ∈ S F (t 0 , x 0 ) satisfying ( 14). Hence we have

Using the very definition of the value function, for all ε > 0 there exists

For M 1 > sup x∈K g(x) we define l : R N → R by the following formula:

Obviously l ε is l.s.c. so V l ε , the value function of the control problem with g replaced by l ε , is a l.s.c. supersolution of (HJBI) and

We also have V l (t 0 , x 0 ) = g (x (T ; t 0 , x 0 , u ε (•))) ≤ V (t 0 , x 0 ) + ε. By the definition of the infimum we obtain

Moreover, V h is (see Lemma 21 in the appendix) a u.s.c. subsolution for (HJBI). Now, to complete the proof of (i) we use the definition of the supremum and the following lemma proved in the appendix.

Lemma 15. Assume that (H f ) holds true, K is a compact proximal retract, and ϕ : (0, T )×K → R is a u.s.c. viscosity subsolution of (HJBI) such that ϕ(T, x) ≤ g(x) for all x ∈ K. Then V (t, x) ≥ ϕ(t, x) for every (t, x) ∈ (0, T ) × K.

The proofs of (ii) and (iii) are direct consequences of Lemma 14, Lemma 15, and Lemma 21.

Using the above arguments for every n ∈ N, ϕ(t, x n (t; t 0 , x 0 , u n (•)) ≥ ϕ(t 0 , x 0 ) for all t ∈ [t 0 , T ] and consequently, ϕ(T, x n (T ; t 0 , x 0 , u n (•)) ≥ ϕ(t 0 , x 0 ).

As ϕ is u.s.c., we obtain by letting n → ∞

Then V ≥ ϕ and the proof is complete.

For the proof of Lemma 21 let us establish a stability result for (HJBI). Lemma 20. Assume that H : K × R N → R is a continuous Hamiltonian and let K be a compact proximal retract. If w n : (0, T ) × K → R is an increasing (decreasing) sequence of uniformly locally bounded l.s.c. (u.s.c.) supersolutions (subsolutions) of (HJBI) and w is a pointwise limit of w n , then w is an l.s.c. (u.s.c.) supersolution (subsolution) of (HJBI).

The proof of this lemma is adapted from [START_REF] Barles | Solutions de viscosité des équations de Hamilton-Jacobi[END_REF]. The main difficulties and changes with Barles's proof are given by the regularity of the application N K (•).

Lemma 21. Assume that (H f ) holds. Let K be a compact proximal retract and g be a u.s.c. (respectively, l.s.c.) function. Then V is a u.s.c. subsolution (respectively, l.s.c. supersolution) of (HJBI).

Proof. We define a sequence g n : K → R by

The supconvolutions g n are Lipschitz, g n (x) ≥ g n+1 (x), and lim g n (x) = g(x) for every x ∈ K. Using Proposition 11, V g n is a Lipschitz solution to (HJBI) with V g n (T, •) = g n (•) and V g n ≥ V.

Denote U (t, x) = lim V g n (t, x). Then, using the above result, U is a u.s.c. subsolution of (HJBI) and U (T, x) = g(x). Obviously U ≥ V and by Lemma 15 we have that U = V . So V is a u.s.c. subsolution for (HJBI).

The proof in the l.s.c. case is similar to the u.s.c. case.


Frédéric Mazenc
email: frederic.mazenc@l2s.centralesupelec.fr

Michael Malisoff
email: malisoff@lsu.edu

Extensions of Razumikhin's Theorem and Lyapunov-Krasovskii Functional Constructions for Time-Varying Systems with Delay *

Keywords: Delay, robustness, stability, time-varying

We prove extensions of Razumikhin's theorem for time-varying continuous and discrete time nonlinear systems. Our results include a novel 'strictification' technique for converting a nonstrict Lyapunov function into a strict one. We also provide new constructions of Lyapunov-Krasovskii functionals that can be used to prove robustness to perturbations. Our examples include a key model from identification theory, and they show how our method can sometimes allow broader classes of delays than the results in the literature.

Introduction

Input delays are ubiquitous in engineering, where they can model time consuming information gathering or latencies; see Bekiaris-Liberis and Krstic (2013a), Bekiaris-Liberis and Krstic (2013b), [START_REF] Dieulot | Tracking control of a nonlinear system with input-dependent delay[END_REF], [START_REF] Downey | Compensating for fatigue-induced time-varying delayed muscle response in neuromuscular electrical stimulation control[END_REF], Gu et al. (2003), [START_REF] Gu | Survey on recent results in the stability and control of time-delay systems[END_REF], [START_REF] Marquez | Analysis and Control of Oilwell Drilling Vibrations: A Time-Delay Systems Approach[END_REF], [START_REF] Mazenc | Generating positive and stable solutions through delayed state feedback[END_REF], [START_REF] Mondié | Finite spectrum assignment of unstable time-delay system with a safe imple-mentation[END_REF], and [START_REF] Petit | Motion planning for two classes of nonlinear systems with delays depending on the control[END_REF]. However, such systems are usually too complicated to be covered by standard methods for undelayed systems; see [START_REF] Richard | Time-delay systems: an overview of some recent advances and open problems[END_REF]. Hence, this note builds on our research (begun in Mazenc and Malisoff (2015b), [START_REF] Mazenc | Reduction model approach for linear time-varying systems with delays[END_REF], and Mazenc et al. (2015)) on novel methods to prove important stability properties for time delayed systems.

Since the flow map for a nonlinear system usually cannot be written in explicit closed form, it is natural to use indirect Lyapunov approaches to prove stability for undelayed systems. Lyapunov functions provide a generalized notion of energy in dynamical systems, so the decay of a Lyapunov function often implies asymptotic convergence of solutions towards an equilibrium. Classical Lya-punov function approaches require that the time derivative of the Lyapunov function be nonpositive along all solutions, which can sometimes be a demanding condition, especially for time-varying or time delay systems. While classical Lyapunov functions are suited for proving stability of systems without delays, one often replaces Lyapunov functions by Lyapunov-Krasovskii or Razumikhin functions to help solve stability problems for delayed systems; see [START_REF] Fridman | On complete Lyapunov-Krasovskii functional techniques for uncertain systems with fast-varying delays[END_REF]. In particular, if we consider the dynamics for the extended state q(t) = (q 1 (t), q 2 (t)) = (x(t), x(t -τ )) of a system ẋ(t) = f (t, x(t), x(t -τ )) with a constant delay τ , then we obtain q2 (t) = f (t -τ, q(t -τ )), so we cannot eliminate the delay from the system by simply considering the dynamics for the extended state q(t).

As noted in [START_REF] Mazenc | Reduction model approach for linear time-varying systems with delays[END_REF] and [START_REF] Zhou | Truncated Predictor Feedback for Time-Delay Systems[END_REF], timevarying systems with delay are important, e.g., for tracking problems. The works Bresch-Pietri and [START_REF] Bresch-Pietri | Robust compensation of a chattering time-varying input delay[END_REF], Mazenc and Malisoff (2015a), and Mazenc et al. (2015) are significant, in part because they use Lyapunov functionals to prove stability but allow the time derivatives of the functionals to take positive values along trajectories. For delayed systems, one often builds Lyapunov-Krasovskii functionals by adding together (a) a Lyapunov function for the corresponding undelayed system and (b) an integral involving the delay, whose integrand is a function of the state; see [START_REF] Mazenc | Further results on input-to-state stability for nonlinear systems with delayed feedbacks[END_REF]. Applying Razumikhin's theorem does not generally involve such integral terms; see Gu et al. (2003), [START_REF] Hale | Introduction to Functional Differential Equations[END_REF], and (Zhou, 2014, Theorem B.2). Razumikhin's approach is useful under time-varying delays. Our work Mazenc and Mal-isoff (2015a) pursues a different approach, involving neither Krasovskii nor Razumikhin functionals. This paper extends Razumikhin's theorem for time-varying systems. We extend the strictification technique, developed in [START_REF] Malisoff | Constructions of Strict Lyapunov Functions[END_REF]. Our first result does not use periodicity. However, due to the importance of periodic systems, we later present a simpler result in the periodic case. We obtain less conservative stability conditions than those in Mazenc et al. (2015). We also provide new constructions for input-to-state stable (or ISS) Lyapunov-Krasovskii functionals for continuous time delay systems; see, e.g., , [START_REF] Pepe | A Lyapunov-Krasovskii methodology for ISS and iISS of time-delay systems[END_REF] and [START_REF] Pepe | On the Lyapunov-Krasovskii methodology for the ISS of systems described by coupled delay differential and difference equations[END_REF] for ISS Lyapunov-Krasovskii functionals and analogs for discrete time systems with delay, which are important for discretization and sampling (as explained, e.g., in [START_REF] Astrom | Computer-Controlled Systems: Theory and Design, Third Edition[END_REF], [START_REF] Karafyllis | Nonlinear stabilization under sampled and delayed measurements, and with inputs subject to delay and zero-order hold[END_REF] and [START_REF] Montagner | Stability and stabilizability of discrete-time switched linear systems with state delay[END_REF]). See, e.g., [START_REF] Cloosterman | Stability of networked control systems with large delays[END_REF] for motivation for discrete time systems, using networked systems that are inspired by communication networks.

Our four examples below demonstrate the value of our theory. In Section 6.5, we illustrate our findings in a model from identification theory with a time-varying delay, building on our treatment of this dynamics in [START_REF] Mazenc | Further results on input-to-state stability for nonlinear systems with delayed feedbacks[END_REF] where the delays were constant. The preliminary version of this paper is [START_REF] Mazenc | Extension of Razumikhin's theorem for time-varying systems with delay[END_REF], which did not include our ISS Lyapunov-Krasovskii functional constructions, nor did it cover discrete time systems. Also, the present paper includes three illustrations that were not in [START_REF] Mazenc | Extension of Razumikhin's theorem for time-varying systems with delay[END_REF].

Definitions and Notation

Throughout this work, all dimensions are arbitrary, unless indicated otherwise. The usual Euclidean norm, and its induced matrix norm, are denoted by | • |, and | • | I denotes the (essential) supremum over any interval I ⊆ R. Let C 1 be the set of all continuously differentiable functions, whose domains and ranges will be clear from the context. For each constant delay bound τ , let C([-τ, 0], R n ) be the set of all continuous R n -valued functions that are defined on [-τ, 0], which we denote by C in and call the set of all initial functions. For each continuous function ϕ :

[-τ, ∞) → R n and t ≥ 0, set ϕ t (m) = ϕ(t + m) for all m ∈ [-τ, 0]. A
locally bounded function φ defined on an interval I ⊆ R is called piecewise continuous provided that for each bounded set S ⊆ I, the restriction of φ to S has only finitely many points where it is discontinuous (which includes continuous functions as a special case). When we say that a function φ defined on [0, ∞)×R n is differentiable on ([0, ∞)×R n )\{0}, we view its partial derivative φ t (0, x) with respect to its first argument as a right derivative at 0 for each x = 0. Let K denote the set of all strictly increasing continuous functions α : [0, ∞) → [0, ∞) such that α(0) = 0, and K ∞ denote the set of all unbounded class K functions. A function F : [0, ∞) × C in → R n is called uniformly bounded with respect to its first argument provided that there are a function α ∈ K ∞ and a constant c > 0 such that [-τ,0] ) holds for all t ≥ 0 and φ ∈ C in ; it is called Lipschitz continuous with respect to its second argument provided that there is a function α ∈ K ∞ such that for each constant K > 0, we have

|F (t, φ)| ≤ c + α(|φ|
|F (t, φ) -F (t, ψ)| ≤ α( K)|φ -ψ| [-τ,0]
(1) for all t ≥ 0 and for all φ and ψ in C in such that max{|φ| [-τ,0] , |ψ| [-τ,0] } ≤ K. Also, we use the standard definitions of input-to-stability (or ISS, which we also use to abbreviate input-to-state stable) and ISS Lyapunov-Krasovskii functionals; see [START_REF] Zhou | Truncated Predictor Feedback for Time-Delay Systems[END_REF] for their standard formulations for delay systems. Finally, for each s ∈ R, we let Floor(s) denote the largest integer J such that J ≤ s.

3 General Result for Continuous Time Unperturbed Systems

Statement of Result

We consider a nonlinear time-varying system

ẋ = F (t, x t ) (2)
whose (possibly time-varying) delay is bounded by some constant τ , having initial conditions in C in (but see below for analogs for systems with perturbations). While stated for systems with state delays, our results apply to systems with input delays as well, by viewing (2) as a closed loop system, with

F (t, x t ) = G(t, x t , u(x t )
) for an open loop dynamics ẋ(t) = G(t, x t , u) and a control u(x t ). We assume:

Assumption 1 The function F is uniformly bounded with respect to its first argument and Lipschitz continuous with respect to its second argument. Also, there exist a function

V : [0, ∞) × R n → [0, ∞) and functions α 1 and α 2 of class K ∞ such that α 1 (|x|) ≤ V (t, x) ≤ α 2 (|x|) (3)
hold for all (t, x) ∈ [0, ∞) × R n and such that there are bounded piecewise continuous functions a : R → R and b

: R → [0, ∞) such that d dt V (t, x(t)) ≤ a(t)V (t, x(t)) + b(t) sup ∈[t-τ,t] V ( , x( )) (4)
holds along all trajectories of (2).

Assumption 2 There are a positive constant β and a bounded piecewise continuous function : R → R such that

µ(t) = a(t) + b(t), (5)
with the choices of a and b from Assumption 1, satisfies

t 0 ( ( ) + µ( ))d ≤ β (6)
for all t ≥ 0. Moreover, the function

κ(t) = sup ∈[t-τ,t] t (-(s) -µ(s))ds (7) admits a constant > 0 such that sup t≥0 e κ(t) -1 b(t) -(t) ≤ - (8)
holds.

See below for interesting applications where we can easily check the preceding assumptions. We prove:

Theorem 1 If Assumptions 1-2 hold, then (2) is globally uniformly asymptotically stable at the origin.

Remark 1 To better understand the significance of the preceding theorem, consider the special case where there is a constant µ > 0 such that the function (5) satisfies

µ(t) ≤ -µ for all t ≥ 0 , (9)
and let b > 0 be any constant bound for b. Choose any constant q ∈ 1, 1 + µ/(2b) . Then, for all t ≥ 0,

a(t) + qb(t) = µ(t) + (q -1)b(t) ≤ -µ/2. ( 10
)
Hence, if qV (t, x(t)) ≥ sup ∈[t-τ,t] V ( , x (
)) for some t ≥ 0, then the nonnegativity of V and b and (4) imply that

d dt (V (t, x(t))) ≤ a(t)V (t, x(t)) + b(t)qV (t, x(t)) ≤ -(µ/2)V (t, x(t)) (11)
are satisfied. Then Razumikhin's theorem ensures the global uniform asymptotic stability of the origin of (2). However, our objective is precisely to establish stability results in cases where (9) may not be satisfied. Our assumptions allow the function a to take positive and negative values. See Section 6 for an analysis in the special case of periodic systems, which further explains the motivation for Assumption 2. While our decay estimate (4) is reminiscent of (A.1) in (Mazenc et al., 2015, Lemma A.1), a valuable feature of Theorem 1 is that it does not require the contractiveness assumption that was made in (Mazenc et al., 2015, Lemma A.1).

Proof of Theorem 1

Throughout the sequel, all inequalities and equalities are for all t ≥ 0 and along all solutions of (2), unless otherwise indicated. Assumption 1 ensures the standard existence and uniqueness properties for solutions of (2). Let ā > 0, b > 0, and ¯ > 0 be any constant bounds for |a(t)|, b(t), and | (t)|, respectively. We define

θ(t) = exp - t 0 ( (s) + µ(s))ds and U (t, x) = θ(t)V (t, x) . (12)
Then for all ∈ [t -τ, t], we have t) .

θ(t) θ( ) ≤ e κ(
(13) Also, (4)-( 5) give

d dt (U (t, x(t))) = θ(t)V (t, x(t)) + θ(t) d dt (V (t, x(t))) ≤ -( (t) + µ(t))U (t, x(t)) + a(t)U (t, x(t)) + θ(t)b(t) sup ∈[t-τ,t] V ( , x( )) = (-(t) -b(t))U (t, x(t)) + b(t) sup ∈[t-τ,t] θ(t)V ( , x( )).
It follows from (13) that

d dt (U (t, x(t))) ≤ -( (t) + b(t))U (t, x(t)) + b(t) sup ∈[t-τ,t] θ(t) θ( ) × sup ∈[t-τ,t] U ( , x( )) ≤ -( (t) + b(t))U (t, x(t)) + e κ(t) b(t) sup ∈[t-τ,t]
U ( , x( )). ( 14) Also, we can use the nonnegativity of b to get

e κ(t) b(t) ≤ exp t t-τ | (s) + µ(s)|ds b(t) ≤ e τ ( +a+b) b. (15)
On the other hand, for all r > 0 such that re τ ( +a+b) b ≤ 2 , we can use (15) to check that

re κ(t) b(t) ≤ 2 . ( 16
)
Also, (8) ensures that for all t ≥ 0, we have

e κ(t) -1 b(t) -(t) ≤ -. (17)
From ( 16) and ( 17), it follows that for all t ≥ 0, we have

e κ(t) -1 b(t) + re κ(t) b(t) -(t) ≤ - 2 . ( 18
)
Set q = 1+r. Then grouping terms gives qe

κ(t) -1 b(t)- (t) ≤ -2 .
Next note that when qU (t, x(t)) ≥ sup

e -β α 1 (|x|) ≤ U (t, x) ≤ e β α 2 (|x|). ( 20
)
The theorem now follows from the classical Razumikhin theorem, by combining ( 19)-( 20).

ISS Lyapunov-Krasovskii Functionals

Before turning to our discrete time results, we present new constructions for ISS Lyapunov-Krasovskii functionals, which are useful tools for the stability analysis of nonlinear systems with delays or for the design of control laws; see the examples section below. In particular, ISS Lyapunov-Krasovskii functionals make it possible to prove that a system possesses the ISS robustness property, and are useful when one aims to establish local asymptotic results or find estimates of basins of attraction. For simplicity, the strictification results in this section assume that we have time-varying systems with constant pointwise and distributed delays, but we can also cover time-varying delays as discussed in the preceding sections. It is important to study systems with distributed delays because frequently systems with pointwise delays are transformed into systems with distributed delays to ease the analysis. Throughout this section, we study nonlinear time-varying systems

ẋ = F (t, x t , ξ(t)) (21)
with delays and initial conditions in C in , where τ ≥ 0 is an upper bound on the delays, ξ : [0, ∞) → R p is a piecewise continuous disturbance, and F is a continuous function that is locally Lipschitz with respect to its x t argument.

In addition, we assume throughout this section that the following holds:

Assumption 3 There is a function K of class K such that |F (t, φ, ξ)| ≤ K (|φ| ∞ + |ξ|) (22)
holds for all t ≥ 0, φ ∈ C in , and ξ ∈ R p .

In Section 4.1, we provide a general Lyapunov-Krasovskii functional construction that uses a suitable nonstrict Lyapunov-like function. Then in Section 4.2, we provide an alternative construction that leads to a large class of systems for which the assumptions from Section 4.1 hold.

Construction of Lyapunov-Krasovskii Functional in General Case

We provide our general construction under the following assumption, where we use V (t) to mean (d/dt)V (t, x(t)) to make our notation concise:

Assumption 4 There exist a continuous functional V :

[-τ, ∞) × C in → [0, ∞); functions α 1 and α 2 of class K ∞ ; a continuous nondecreasing function β : [0, ∞) → [0, ∞);
a function γ ∈ K; positive constants µ B , δ, and T ; and a continuous function

µ : R → [-µ B , µ B ] such that α 1 (|φ(0)|) ≤ V (t, φ) ≤ α 2 (|φ| ∞ ) (23)
holds for all t ≥ 0 and φ ∈ C in , such that

V (t) ≤ µ(t)V (t, x t )+β(V (t, x t ))V 2 (t, x t )+γ(|ξ(t)|) (24)
holds along all trajectories of the system ( 21), and such that

1 T t t-T µ(m)dm ≤ -δ (25)
holds for all t ∈ R.

Assumption 4 is significantly weaker than the usual requirement that µ must have a negative constant upper bound. In fact, (25) says that -µ is a persistency of excitation parameter, but the existing literature on persistence of excitation did not solve the challenging problem of constructing Lyapunov-Krasovskii functionals that we solve in this section. Condition (25) says that µ is negative in a suitable averaged sense, and includes functions such as

µ(m) = -max{0, sin(2πt/T )}
and many other cases where µ can be zero throughout intervals of any positive length L. We prove the following result (but see Remark 2 for local ISS results for cases where β is not necessarily zero):

Theorem 2 Let the system (21) satisfy Assumptions 3-4. If β(m) = 0 for all m ≥ 0, then (21) admits the ISS Lyapunov-Krasovskii functional

W(t, φ) = e -1 T t t-T t µ(m)dmd V (t, φ) (26)
and therefore is ISS with respect to ξ.

Proof: In the sequel, all inequalities and equalities should be understood to hold along all trajectories of ( 21) and all t ≥ 0, and we use the notation

Ẇ(t) = d dt W(t, x t )
as we did for V . Since we are assuming that β is identically equal to zero, we can use ( 24)-( 25) to obtain

Ẇ(t) = e -1 T t t-T t µ(m)dmd V (t) + -µ(t) + 1 T t t-T µ(m)dm W(t, x t ) ≤ e -1 T t t-T t µ(m)dmd γ(|ξ(t)|) + 1 T t t-T µ(m)dmW(t, x t ) ≤ -δW(t, x t ) + e -1 T t t-T t µ(m)dmd γ(|ξ(t)|). (27) Since |µ| is bounded by µ B , it follows that Ẇ(t) ≤ -δW(t, x t ) + e T 2 µ B γ(|ξ(t)|) (28)
holds for all t ≥ 0, and ( 23) gives

e -T 2 µ B α 1 (|φ(0)|) ≤ e -T 2 µ B V (t, φ) ≤ W(t, φ) ≤ e T 2 µ B V (t, φ) ≤ e T 2 µ B α 2 (|φ| ∞ ) (29)
for all t ≥ 0 and φ ∈ C in . Conditions ( 28)-( 29) imply that W is an ISS Lyapunov-Krasovskii functional for (21). This proves the theorem.

Remark 2 When β is present, we can prove local ISS results, as follows. Using (24) and the equality in ( 27), we deduce that

Ẇ(t) ≤ -δW(t, x t ) + e T 2 µ B γ(|ξ(t)|) + e -1 T t t-T t µ(m)dmd β(V (t, x t ))V 2 (t, x t ) . ( 30
)
Since |µ| is bounded by our constant µ B and β is nondecreasing and

V (t, x t ) ≤ e T µ B /2 W(t, x t ) holds for all t, we have Ẇ(t) ≤ -δ + e 3T 2 µ B β e T 2 µ B W(t, x t ) W(t, x t ) ×W(t, x t ) + e T 2 µ B γ(|ξ(t)|). (31)
Since γ ∈ K, we can then choose a small enough constant W B > 0 and then a small enough constant ξ B > 0 such that

-δ + e 3T 2 µ B β e T 2 µ B W B W B W B +e T 2 µ B γ(ξ B ) < 0 (32)
is satisfied. We now show that the ISS property is satisfied for all perturbations ξ that are bounded in the supremum norm by ξ B and all initial functions φ ∈ C in such that

W(t, φ) ≤ W B for all t ≥ 0. ( 33
) (Notice that (33) holds if |φ| ∞ ≤ α -1 2 (exp(-T µ B /2)W B ), by (29), where | • | ∞ is the usual supremum norm.)
To this end, we first observe that if the initial function φ satisfies (33), then (31) implies that

Ẇ(0) ≤ -δ + e 3T 2 µ B β e T 2 µ B W B W B W B + e T 2 µ B γ(ξ B ) . (34)
From ( 32), it follows that Ẇ(0) < 0. By continuity of W(t, x t ), we deduce that there is t 1 > 0 such that W(t, x t ) < W B holds for all t ∈ (0, t 1 ]. Now we proceed by contradiction. Suppose that there were a

t 2 > t 1 such that W(t, x t ) < W B for all t ∈ [t 1 , t 2 ) and W(t 2 , x t2 ) = W B . Then Ẇ(t 2 ) ≤ -δ + e 3T 2 µ B β e T 2 µ B W B W B W B + e T 2 µ B γ(ξ B ) < 0 . (35) Therefore there is t 3 ∈ (t 1 , t 2 ) such that W(t 3 , x t3 ) > W(t 2 , x t2 ) = W B . This yields a contradiction.
It follows from ( 31) that for all t ≥ 0, we have

Ẇ(t) ≤ -λW(t, x t ) + e T 2 µ B γ(|ξ(t)|), where λ = δ -e 3T 2 µ B β e T 2 µ B W B W B . (36)
Notice for later use that (32) implies that λ > 0. By integrating (36) on any interval [t 0 , t], and using (29), we obtain the ISS inequality

|x(t)| ≤ α -1 1 2e -λ(t-t0) e T µ B α 2 (|x t0 | ∞ ) +α -1 1 2e T µ B λ sup ∈[t0,t] γ(|ξ( )|) , (37)
where we also used the fact that the function α -1

1 ∈ K ∞ satisfies α -1 1 (a + b) ≤ α -1 1 (2a) + α -1 1 (2b)
for all a ≥ 0 and b ≥ 0. This proves the local ISS result.

System with Pointwise and Distributed Delays

Theorem 2 requires a function V that satisfies Assumption 4. In this section, we study the system (21) under assumptions which are frequently met in practice and again show how Lyapunov-Krasovskii functionals can be constructed. Our first assumption is:

Assumption 5 There are a constant η > 0, a piecewise C 1 Lyapunov function V : [-max{τ, η}, ∞) × R n → [0, ∞), functions α 1 and α 2 of class K ∞ , a function γ ∈ K, a constant ω ∈ [0, η],
→ [0, ∞) such that α 1 (|x)|) ≤ V(t, x) ≤ α 2 (|x|) (38)
holds for all t ≥ 0 and x ∈ R n and such that along all trajectories of the system (21), we have

V(t) ≤ a(t)V(t, x(t)) + b(t)V(t -τ, x(t -τ )) + t-ω t-η c(m, t)V(m, x(m))dm + γ(|ξ(t)|) (39)
almost everywhere.

In terms of the functions from Assumption 5, we then set

h(t) = e - t+τ t a( )d b(t + τ ) + t+η t+ω e - s t a( )d c(t, s)ds (40)
and add this assumption:

Assumption 6 There are constants T > 0 and δ > 0 such that

1 T t t-T [a(m) + h(m)]dm ≤ -δ (41)
holds for all t ≥ 0.

Condition (39) implies (4) from Assumption 1 when the perturbation ξ is the zero function, so ( 39) is more restrictive than (4). However the functions b and c in (39) will be useful for building a Lyapunov-Krasovskii functional; see (47). Hence, this section adds considerable value, as compared with Section 3. To build our Lyapuov-Krasovskii functional for (21), we introduce the function

q(m) = m-ω m-η e m s+η λ( )d c(m -ω, s + η)ds (42)
and the operators

Γ 1 (t, x t ) = t t-τ e t m+τ λ( )d b(m+τ )V(m, x(m))dm, (43) Γ 2 (t, x t ) = t t-ω e t m+ω λ( )d q(m + ω)V(m, x(m))dm, and (44)
Γ 3 (t, x t ) = t-ω t-η e t m+η λ( )d G(t, ω, m)dm, (45)
where λ = a + h and

G(t, ω, m) = t-ω m c(s, m + η)V(s, x(s))ds. (46)
We prove the following result (but see Remark 3 for local versions under weaker conditions, and generalizations with several types of delays, and see below for illustrations):

Corollary 1 Let the system (21) satisfy Assumptions 3 and 5-6. Then the system is ISS with respect to ξ, and

W(t, x t ) = e -1 T t t-T t λ(m)dmd U(t, x t ) with the choice U(t, x t ) = V(t, x(t)) + Γ 1 (t, x t ) + Γ 2 (t, x t ) + Γ 3 (t, x t ) (47)
is an ISS Lyapunov-Krasovskii functional for (21).

Proof: Throughout the proof, we let the function ξ be equal to zero. The general case follows by adding γ(|ξ(t)|) to the right sides of the relevant inequalities in the rest of the proof. First note that

Γ1 (t) = λ(t)Γ 1 (t, x t ) + e t t+τ λ( )d b(t + τ )V(t, x(t)) -b(t)V(t -τ, x(t -τ )) , Γ2 (t) = λ(t)Γ 2 (t, x t ) + e t t+ω λ( )d q(t + ω)V(t, x(t)) -q(t)V(t -ω, x(t -ω)) and Γ3 (t) = λ(t)Γ 3 (t, x t ) - t-ω t-η c(s, t)V(s, x(s))ds + t-ω t-η e t m+η λ( )d c(t -ω, m + η)V(t -ω, x(t -ω))dm = λ(t)Γ 3 (t, x t ) - t-ω t-η c(s, t)V(s, x(s))ds +q(t)V(t -ω, x(t -ω))
hold along all trajectories of the system. Using Assumption 5 and summing the right sides of the preceding equalities for the Γi 's for i = 1, 2, 3 and then combining terms, it follows that the derivative of U along all trajectories of (21

) satisfies U(t) ≤ a(t)V(t, x(t)) + b(t)V(t -τ, x(t -τ )) + t-ω t-η c(m, t)V(m, x(m))dm + Γ1 (t) + Γ2 (t) + Γ3 (t) = a(t) + e t t+τ λ( )d b(t + τ ) +e t t+ω λ( )d q(t + ω) V(t, x(t)) +λ(t)[Γ 1 (t, x t ) + Γ 2 (t, x t ) + Γ 3 (t, x t )] .
U(t) ≤ a(t)U(t, x t ) + e t t+τ (a( )+h( ))d b(t + τ ) +e t t+ω (a( )+h( ))d q(t + ω) V(t, x(t)) +h(t)[Γ 1 (t, x t ) + Γ 2 (t, x t ) + Γ 3 (t, x t )]. (50)
Using the definitions of q, b * , and c * , we get

U(t) ≤ a(t)U(t, x t ) + e - t+τ t h( )d b * (t) + t t+ω-η e - s+η t h( )d c * (t, s)ds V(t, x(t)) +h(t)[Γ 1 (t, x t ) + Γ 2 (t, x t ) + Γ 3 (t, x t )]
and (52)

h(t) = b * (t) + t t+ω-η c * (t, s)ds . ( 51
)
As an immediate consequence, we get

U(t) ≤ [a(t) + h(t)]U(t, x t ) . (53)
Hence, using Assumption 6 and Theorem 2 with β = 0, we can conclude.

Remark 3 We can prove a local result when extra nonlinear terms are present in (39) as we did in Remark 2. We can also extend Theorem 1 to cases where several pointwise and distributed delays are present.

Comparison of Approaches

The purpose of this subsection is to compare the Lyapunov-Krasovskii functional construction method from the preceding proof with a possible alternative approach to building Lyapunov-Krasovskii functionals, and to show why the alternative approach cannot be used. This will further illustrate the value of our new approaches.

To this end, assume again that (21) satisfies Assumption 3, and that there are a

C 1 function V : [-τ, ∞)×R n → [0, ∞) and functions α 1 and α 2 of class K ∞ such that α 1 (|x|) ≤ V (t, x) ≤ α 2 (|x|)
hold for all t ≥ 0 and x ∈ R n and such that along the trajectories of ( 21) with ξ ≡ 0, we have

V (t) ≤ a(t)V (t, x(t)) + b(t)V (t -τ, x(t -τ )) ( 54
)
where a is a periodic function of some period T > 0, the function b is bounded and nonnegative valued, and both a and b are continuous. We also assume that there is a positive constant a c such that

1 T T 0 a(m)dm = -a c . (55)
In this context, a first possible approach to establish stability conditions for (21) consists of the following strictification of the function V with respect to a.

We define the function

ν 1 (t, x) = e R(t) V (t, x), where
R(t) = -1 T t t-T t a(m)dm d . ( 56
)
Since Ṙ(t) = -a(t) -a c , it follows that the time derivative of ν 1 along all trajectories of (21) satisfies

ν1 (t) = e R(t) V (t) + Ṙ(t)e R(t) V (t, x(t)) ≤ a(t) + Ṙ(t) ν 1 (t, x(t)) + e R(t) b(t)V (t -τ, x(t -τ )) ≤ -a c ν 1 (t, x(t)) + e R(t)-R(t-τ ) b(t)ν 1 (t -τ, x(t -τ )) . (57)
We deduce from Razumikhin's theorem that if there is a constant s c > 0 such that

-a c + e R(t)-R(t-τ ) b(t) ≤ -s c (58)
for all t, then the origin of ( 21) with ξ ≡ 0 is globally asymptotically stable.

If we now wish to construct a Lyapunov-Krasovskii functional, then we can proceed as follows. Let

ν 2 (t, x t ) = ν 1 (t, x(t))+ t t-τ e R(m+τ )-R(m) b(m + τ )ν 1 (m, x(m))dm . (59)
Then for all t ≥ 2τ , we have

ν2 (t) ≤ -a c + e R(t+τ )-R(t) b(t + τ ) ν 1 (t, x(t)). ( 60
)
Using the Lyapunov-Krasovskii theorem and (58), one can deduce from (60) that the system ( 21) is globally uniformly asymptotically stable. If τ = 0, then

-a c + e R(t+τ )-R(t) b(t + τ ) = -a c + b(t).
We deduce by continuity that when there are instants t such that b(t) > a c , global asymptotic stability for arbitrarily small delay cannot be deduced from (60). By contrast, our assumptions allow cases where b(t) > a c for some values t, so our results are less conservative; see the examples below.

Discrete Time Versions

In this section, we provide discrete time analogs of some of the continuous time ideas from the previous sections. The work in this section builds on the Razumikhin-like theorems in [START_REF] Elaydi | Stability and periodicity of difference equations with finite delay[END_REF] and [START_REF] Zhou | Truncated Predictor Feedback for Time-Delay Systems[END_REF], by developing nonstrict Lyapunov-like decay conditions that imply exponential convergence properties. Discrete time analogs of the decay conditions on a continuous time Lyapunov function involve a sequence of values {V (x(k))} taken by a nonnegative valued function V along sequences {x(k)} of state values of a delay system of the form

x(k + 1) = f (k, x(k -τ ), ..., x(k)). (61)
Setting V k = V (x(k)) for all k ∈ N leads to the study of sequences {V k } that are not necessarily decreasing in k, where N = {1, 2, . . .}. Such sequences are the subject of this section.

Discrete time systems naturally arise from discretizing continuous time systems in control applications. Moreover, as in the continuous time case, time-varying discrete time systems naturally arise from tracking problems and from linearizing around a reference trajectory, even if the original system is time invariant, which is an important motivation for our allowing time-varying systems. To see how time-varying systems arise from tracking problems in a special case, assume that we are given a discrete time system q(k + 1) = F(q(k)) + u(q(k -τ )) with a control u and a constant delay τ , and a reference trajectory q r such that q r (k + 1) = F(q r (k)) for all k that we wish to track. Then the dynamics of the tracking error x(k) = q(k) -q r (k) are x(k + 1) = f (k, x(k -τ ), x(k)) where f (a, b, c) = F(c + q r (a)) -F(q r (a)) + u(b + q r (a -τ )).

Preliminary Result for Scalar Sequences

To prove our discrete time results, we use:

Lemma 1 Let ξ 1 , ξ 2 , . . . be any sequence of positive real numbers such that there are constants s 1 > 0 and s 2 ≥ s 1 and an integer p ≥ 1 for which (i)

ξ i ∈ [s 1 , s 2 ] for all i ≥ 1 and (ii) the constant σ =   p+j i=1+j ξ i   1 p (62)
is independent of the integer j ≥ 0 and satisfies σ ∈ (0, 1).

Then the sequence ρ 1 , ρ 2 , . . . defined by

ρ k = k-1 i=1 σ ξ i (63)
for all k ≥ 2 and ρ 1 = 1 satisfies (σ/s 2 ) p+1 ≤ ρ k ≤ (σ/s 1 ) p+1 for all k ≥ 1.

Proof: If k is any integer such that k ≥ p + 2, and if we set = Floor((k -2)/p), then the fact that σ in ( 62) is independent of j, and the fact that k -1 ≤ ( +1)p+1, give

ρ k = p i=1 σ ξ i   k-1 i= p+1 σ ξ i   = k-1 i= p+1 σ ξ i . (64)
The last product in (64) lies in [(σ/s 2 ) p+1 , (σ/s 1 ) p+1 ] because our assumption that ξ i ∈ [s 1 , s 2 ] for all i implies that σ ∈ [s 1 , s 2 ], so σ/s 2 ≤ 1 ≤ σ/s 1 . Since we can check directly that ρ k ∈ [(σ/s 2 ) p+1 , (σ/s 1 ) p+1 ] for all k ∈ {1, 2, . . . , p + 1}, the lemma follows.

Before turning to our main result for discrete time cases, we present a discrete time strictification result, which is of interest for its own sake and eases the understanding of the main result.

Proposition 1 Let {ξ k } and the constants σ ∈ (0, 1), s 1 > 0, and s 2 ≥ s 1 satisfy the assumptions of Lemma 1, and {V k } be a sequence of nonnegative real numbers such that

V k+1 ≤ ξ k V k (65) holds for all k ≥ 1. Then V k ≤ (s 2 /s 1 ) p+1 σ k-1 V 1 holds for all k ≥ 1, so {V k } is exponentially stable. Proof: We use the sequence Q k = ρ k V k , where {ρ k } is from (63) in Lemma 1.
Then the fact that ρ k+1 /ρ k = σ/ξ k for all k ≥ 1 implies that for all k ≥ 1, we obtain

Q k+1 = ρ k+1 V k+1 ≤ ρ k+1 ξ k V k = ρ k+1 ρ k ξ k Q k = σQ k (66)
and so also

Q k ≤ σ k-1 Q 1 .
The result now follows from the positive bounds for {ρ k } from Lemma 1.

Remark 4 The assumptions of Proposition 1 do not imply that ξ k ≤ 1 for all k ∈ N. The proof of Proposition 1 relies on a strictification approach, and ensures exponential stability. An alternative proof of the exponential stability conclusion consists of observing that since σ ∈ (0, 1) in ( 62) is independent of j, it follows from ( 65) that for all k ≥ p, we can choose j = k -p in (62) to get

V k+1 ≤ ξ k V k ≤ k i=k-p+1 ξ i V k-p+1 = σ p V k-p+1 ≤ σV k-p+1 . (67)
Hence, for each constant g ∈ {1, ..., p -1}, the sequence {V kp+g } is exponentially stable. We deduce that {V k } is exponentially stable. In fact, for all k ≥ p+1, we can choose = Floor((k -1)/p) and use the facts that ξ i ≤ s 2 for all i and σ < 1 to get

V k ≤ k-1 i= p ξ i V p ≤ k-1 i= p ξ i σ -1 V p ≤ k-1 i= p ξ i p-1 i=1 ξ i σ -1 V 1 ≤ max s 2p
2 , 1 σ Floor((k-1)/p)-1 V 1 , by separately considering the cases where s 2 ≥ 1 and s 2 < 1.

Main Result for Discrete Time Case

The section provides a discrete-time version of results of the previous sections. However, it is not a strictified version of the Razumikhin theorem for discrete time systems but is an extension of Proposition 1 that allows delays.

Consider a sequence of nonnegative real numbers {V k } such that there exist an integer p > 0 and sequences {α k,j } of nonnegative values for all j ∈ {0, ..., p} such that for all k > p, the following inequality is satisfied:

V k+1 ≤ p j=0 α k,j V k-j (68)
For instance, the V k 's can be the values taken by a Lyapunov function at discrete times. We define the sequence

α k = p j=0 α k,j (69)
and make the following two assumptions:

Assumption 7 There are constants a 1 > 0 and a 2 ≥ a 1 such that α k ∈ [a 1 , a 2 ] holds for all k ∈ N. Also,

α * =   p+j i=1+j α i   1 p (70)
is independent of the integer j ≥ 0 and satisfies α * ∈ (0, 1).

Assumption 8 The inequality α p+1 * < a p 1 is satisfied. We can then prove this exponential stability result:

Theorem 3 If {V k } is such that Assumptions 7-8 hold, then we can find a constant λ ∈ (0, 1) such that

V k+1 ≤ a 2 a 1 p+1 λk-p-1 sup {i=1,...,p+1} V i (71)
holds for all k > p.

Proof: We define the sequence λ k by

λ 1 = 1 and λ k = k-1 i=1 α * α i when k > 1 (72)
and the sequence W k = λ k V k . Then for all k > p, we can use our inequality (68) to get

W k+1 ≤ p j=0 α k,j λ k+1 λ k-j W k-j = λ k+1 λ k p j=0 α k,j λ k λ k-j W k-j ≤ λ k+1 λ k p j=0 α k,j λ k λ k-j sup i∈{0,...,p} W k-i ≤ λ k+1 λ k α k sup i∈{k-p,...,k} λ k λ i sup i∈{k-p,...,k} W i . (73)
From ( 72), it follows that for all k > p, we have

W k+1 ≤ α * α k α k sup i∈{k-p,...,k} λ k λ i sup i∈{k-p,...,k} W i = α * max sup i∈{k-p,...,k-1} k-1 m=i α * α m , 1 sup i∈{k-p,...,k} W i .
Using the lower bound a 1 for the sequence α m , we obtain

W k+1 ≤ α * max 1, α * a 1 p sup i∈{k-p,...,k} W i , (74)
by separately considering the cases α * ≤ a 1 and α * > a 1 . Let λ be the quantity in squared brackets in (74). Then Assumptions 7-8 imply that λ ∈ (0, 1), and (74) gives

W k+1 ≤ λk-p-1 sup {i=1,...,p+1} W i . ( 75
)
If we now apply Lemma 1 with the choices ξ i = ᾱi , s 1 = a 1 , s 2 = a 2 , and σ = α * , then we get

α * a2 p+1 ≤ λ i ≤ α * a1 p+1
for all i, so the desired decay estimate (71) follows from our formula W k = λ k V k for the W k 's.

6 Periodic Case

Statement of Result for Periodic Case

We now consider key cases where the following holds:

Assumption 9 Assumption 1 holds, and there is a constant T > 0 such that a and b have period T . Also, there exists a bounded piecewise continuous function : R → R that is periodic of period T and that satisfies

T 0 [ (m) + a(m) + b(m)]dm = 0 (76)
and is such that the function κ defined in ( 7) is such that

sup t∈[0,T ] e κ(t) -1 b(t) -(t) < 0 ( 77
)
is satisfied.

In Section 6.3, we prove:

Corollary 2 Let (2) satisfy Assumption 9. Then the origin of ( 2) is globally uniformly asymptotically stable.

Discussion of Corollary 2

Assumption 9 is often satisfied. For instance, consider the case where a and b have some period T and choose the constant function (t) = * , where * = -1

T T 0 µ( )d (78)
and µ = a + b as before. Then the function (7) satisfies

κ(t) = sup ∈[t-τ,t] t (- * -µ(s))ds ≤ τ s µ (79)
for all t ≥ 0, where s µ = sup s∈[0,T ] (- * -µ(s)), if s µ ≥ 0, and κ(t) ≤ 0 if s µ ≤ 0. Assume that * > 0 and s µ ≥ 0. Then ( 76)-( 77) hold if the inequality

sup t∈[0,T ] [(e τ sµ -1) b(t)] < * (80)
is satisfied. If s µ > 0, then (80) holds if τ ∈ (0, τ * ), where

τ * = 1 sµ ln 1 + * b . (81)
Another sufficient condition can be obtained when µ is globally Lipschitz, as follows. Suppose that we have a global Lipschitz constant λ µ > 0 for µ. For all s ≥ 0, we get * = -1

T s s-T µ( )d , (82)
by the periodicity of a and b. Hence,

κ(t) = sup ∈[t-τ,t] t 1 T s s-T µ(m)dm -µ(s) ds , (83)
by our choice (t) = * . Since the function

ϕ(t) = 1 T t t-T t s µ(m)dmds (84) satisfies φ(t) = µ(t) -1 T t t-T µ(m)dm, (85)
we can integrate φ over [ , t] to get

κ(t) = sup ∈[t-τ,t] t (-φ(s))ds = sup ∈[t-τ,t] (-ϕ(t) + ϕ( )) . (86)
Since we can change variables to get

ϕ( ) = 1 T t t-T t s µ(m -t + )dm ds, (87)
our Lipschitz constant λ µ > 0 and ( 86)-(87) give (89)

κ(t) = sup ∈[t-τ,t] 1 T t t-T t s [-µ(m) + µ(m -t + )]dm ds ≤ sup ∈[t-τ,t] λ µ T t t-T t s (t -)d ds ≤ T λ µ τ 2 ,

Proof of Corollary 2

We show that Assumption 9 implies that Assumption 2 holds. Let ¯ > 0, ā > 0, and b > 0 be bounds on the functions | |, |a|, and b, respectively, t > 0 be given, and k be the integer such that t ∈ [kT , (k + 1)T ). Then

t 0 ( ( ) + µ( ))d ≤ kT 0 ( ( ) + µ( ))d + t kT ( ( ) + µ( ))d . (90)
Since ( 76) and our choice µ = a + b imply that

kT 0 ( ( ) + µ( ))d = 0, (91)
it follows from (90) that

t 0 ( ( ) + µ( ))d ≤ t kT ( + a + b)d ≤ T ( + a + b) . (92)
Next, we prove that κ is periodic of period T . We have

κ(t + T ) = sup ∈[t+T -τ,t+T ] t+T (-(m) -µ(m))dm = sup ∈[t-τ,t] t+T +T (-(m) -µ(m))dm = sup ∈[t-τ,t] t (-(m + T ) -µ(m + T ))dm.
Since both µ and are periodic of period T , it follows that

κ(t + T ) = sup ∈[t-τ,t] t (-(m) -µ(m))dm = κ(t). (93)
From the fact that κ, and µ are all periodic of period T , we easily deduce that for all t ≥ 0, we have

sup t≥0 e κ(t) -1 b(t) -(t) ≤ -, where = -sup t∈[0,T ] e κ(t) -1 b(t) -(t) . ( 94
)
The corollary now follows from Theorem 1.

First Illustration of Corollary 2

Consider the one dimensional system

ẋ(t) = -x(t) + b(t)x(t -τ (t)) (95)
where b : R → R is periodic of period T = 1 and such that there are constants c ∈ (0, 1) and d > 0 satisfying (i) b(t) = 0 for all t ∈ [0, c), (ii) b(t) = d for all t ∈ [c, 1), and (iii) d(1 -c) ∈ (0, 1). Assume that τ : [0, ∞) → [0, ∞) is continuous and bounded by a constant τ > 0. This is more general than the systems in Mazenc et al. (2015), since (95) has a time-varying delay. See Section 6.5 for a higher dimensional example.

Since b is periodic, we apply Corollary 2. With V (x) = |x| and a(t) = -1, Assumption 1 is satisfied. Next, we give conditions ensuring that Assumption 9 holds. We first apply the method from Section 6.2. Let * = -

1 0 [-1 + b(m)]dm = 1 -d(1 -c). (96)
Using the notation from Section 6.2, we set b = d and µ(s) = a(s) + b(s) = -1 + b(s), which gives

s µ = sup s∈[0,1] (- * -µ(s)) = sup s∈[0,1] d(1 -c) -b(s) = d(1 -c). (97)
Therefore, the delay bound τ * from (81) in Section 6.2 is

τ * = 1 sµ ln 1 + * b = 1 d(1-c) ln 1+cd d . (98)
To get less restrictive conditions than those obtained with (t) = * , pick any constant ν ∈ (0, min {1, * /c}), and let be the period 1 function such that (a

) (t) = ν for all t ∈ [0, c) and (b) (t) = * -cν 1-c for all t ∈ [c, 1
). Then has positive upper and lower bounds. The function κ in ( 7) is

κ(t) = sup ∈[t-τ ,t] t [-(m) + 1 -b(m)]dm. ( 99
κ(t) < 1 -cν d(1 -c) . ( 101
)
Now observe that

-(m) + 1 -b(m) = -1-cν 1-c -d + 1 -d = c ν-1 1-c < 0 (102)
for all m ∈ [c, 1), while

-(m) + 1 -b(m) = -ν + 1
for all m ∈ [0, c). We deduce from (99) that sup

t∈[c,1] κ(t) ≤ τ (1 -ν).
Then ( 101) is satisfied if

τ < 1 1-ν ln 1-cν d(1-c) . ( 103
)
Taking the limit from the right as ν → 0 + in (103), we get τ < τ * , where τ * = -ln(d(1 -c)).

(104) This is a less restrictive upper bound for τ than (98), since for instance, if we take c = 0.94 and d = 10, then d(1-c) = 0.6 ∈ (0, 1), and the difference between the delay bounds ( 98) and ( 104) is

-ln(d(1 -c)) - 1 d(1 -c) ln 1 + cd d > 0.44. ( 105
)
This justifies our use of the more complicated function (t), instead of the constant * .

6.5 Second Illustration of Corollary 2

Consider the system ẋ = -m(t)m (t)u(t -τ (t)) ( 106
)
where x is valued in R n for any n, the function m : R → R n is continuous and has some period T > 0, the input u is valued in R n , and the delay τ is a time-varying piecewise continuous function that is bounded by a constant τ > 0.

The system (106) arises in identification theory; see [START_REF] Mazenc | Further results on input-to-state stability for nonlinear systems with delayed feedbacks[END_REF]. In [START_REF] Mazenc | Further results on input-to-state stability for nonlinear systems with delayed feedbacks[END_REF], we studied ( 106), but only in the special case where τ is constant. The approach in [START_REF] Mazenc | Further results on input-to-state stability for nonlinear systems with delayed feedbacks[END_REF] is based on the construction of a Lyapunov-Krasovskii functional, which is written as the sum of a strict Lyapunov function for the corresponding undelayed system, plus a double integral term in which the integrand is a function of the norm of the state. Here, we use Corollary 2 to provide stabilizability conditions in cases where the delay is time-varying. Notice that [START_REF] Mazenc | Robustness of nonlinear systems with respect to delay and sampling of the controls[END_REF] does not apply to systems with a time-varying delay. We first introduce:

Assumption 10 The matrix M ∈ R n×n defined by

M = 1 T T 0 m(s)m (s)ds (107)
is positive definite.

We define the constants

k m = 1 T T 0 |m(s)| 2 ds and s m = sup t∈[0,T ] |m(t)| (108)
and let λ M be the smallest eigenvalue of M . Then k m > 0, s m > 0 and λ M > 0. We now add three assumptions:

Assumption 11 The inequality λ M 2 + T s 2 m > s 4 m τ 2 + 2T k m k m ( 109
)
is satisfied.

Assumption 12 The function |m(t)| is globally Lipschitz, with some Lipschitz constant l m > 0.

Assumption 13 The inequality

2 s 4 m τ 2 + 2T k m s m l m τ T < ln 1 + 2λ M 2+T s 2 m -2(s 4 m τ 2 +2T km)km b (110)
is satisfied, where b = 2(s 4 m τ 2 + 2T k m )s 2 m . One can easily determine values of τ such that (110) is satisfied.

Corollary 3 Let the system (106) satisfy Assumptions 10-13. Then (106), in closed loop with the feedback u(x(t)) = x(t), is globally uniformly exponentially stable to the origin.

Proof: The feedback produces the system ẋ = -m(t)m (t)x(t -τ (t)), ( 111) so we prove that the origin of ( 111) is globally uniformly exponentially stable. This system is a linear time-varying system and therefore we can restrict our stability analysis to the time interval [2τ , ∞).

For all t ≥ 2τ , we can rewrite the system as

ẋ = -m(t)m (t)x(t) -m(t)m (t) × t t-τ (t) m(s)m (s)x(s -τ (s))ds. (112)
Let us consider ν 1 (x) = 1 2 |x| 2 and

ν 2 (t, x) = 1 T x t t-T t m(s)m (s)ds d x (113)
By ( 112), we deduce that along all trajectories of (111), we have

ν1 (t) = -(m (t)x(t)) 2 -x(t) m(t)m (t) × t t-τ (t) m(s)m (s)x(s -τ (s))ds (114)
and

ν2 (t) = (m (t)x(t)) 2 -x(t) M x(t)- 2 T x(t) t t-T t m(s)m (s)dsd m(t)m (t)x(t -τ (t)),
where M is the matrix defined in (107).

Let us introduce the function

V (t, x) = 2ν 1 (x) + ν 2 (t, x).
Then for all t ≥ 0 and x ∈ R n , the inequalities

1 2 |x| 2 ≤ V (t, x) ≤ 1 2 2 + T s 2 m |x| 2 (115)
are satisfied. For all t ≥ 2τ , we get

V (t) = -x(t) M x(t) -(m (t)x(t)) 2 -2 x(t) m(t) × m (t) t t-τ (t) m(s)m (s)x(s -τ (s))ds -2 T x(t) t t-T t m(s)m (s)dsd ×m(t)m (t)x(t -τ (t)). (116)
Using Young's inequality to get 2ab ≤ a 2 +b 2 where a and b correspond to the terms in curly braces in (116), we obtain

V (t) ≤ -x(t) M x(t)+ m (t) t t-τ (t) m(s)m (s)x(s -τ (s))ds 2 -2 T x(t) t t-T t m(s)m (s)dsd ×m(t)m (t)x(t -τ (t)). (117)
Then Jensen's inequality gives

V (t) ≤ -x(t) M x(t) + τ (t)|m(t)| 2 × t t-τ (t) m(s)m (s) 2 |x(s -τ (s))| 2 ds + 2|m(t)| 2 T t t-T t |m(s)| 2 dsd |x(t)||x(t -τ (t))|. ( 118
)
From the definition of k m in (108) and our choice of λ M , it follows that

V (t) ≤ -x(t) M x(t) + τ (t)|m(t)| 2 × t t-τ (t) m(s)m (s) 2 |x(s -τ (s))| 2 ds + 2|m(t)| 2 T k m |x(t)||x(t -τ (t))| ≤ -λ M |x(t)| 2 + 2T k m |m(t)| 2 |x(t)||x(t -τ (t))| +τ (t)s 4 m |m(t)| 2 t t-τ (t) |x(s -τ (s))| 2 ds. (119)
Hence, our bounds (115) give

V (t) ≤ -2λ M 2+T s 2 m V (t, x(t)) + 2τ (t)s 4 m |m(t)| 2 × t t-τ (t) V s -τ (s), x(s -τ (s)) ds + 4T k m |m(t)| 2 V (t, x(t))V t -τ (t), x(t -τ (t)) . (120) It follows that V (t) ≤ -2λ M 2+T s 2 m V (t, x(t)) + 2τ 2 (t)s 4 m |m(t)| 2 sup s∈[t-2τ ,t] V (s, x(s)) +4T k m |m(t)| 2 sup s∈[t-2τ ,t] V (s, x(s)) ≤ -2λ M 2+T s 2 m V (t, x(t)) +2 τ 2 s 4 m + 2T k m |m(t)| 2 sup s∈[t-2τ ,t] V (s, x(s)).
We next apply Corollary 2 with

a(t) = -2λ M 2+T s 2 m and b(t) = 2 s 4 m τ 2 + 2T k m |m(t)| 2 .
Then our choices µ = a + b and (t) = * from (78) in Section 6.2 and k m from (108) give

(t) = * = -1 T T 0 µ( )d = 2λ M 2+T s 2 m -2 s 4 m τ 2 + 2T k m k m .
From (109), it follows that * > 0. Moreover, µ is globally Lipschitz with l µ = 4 s 4 m τ 2 + 2T k m s m l m as Lipschitz constant. We deduce from our discussion of Corollary 2 in Section 6.2 that

l µ τ T < 2 ln 1 + * sup t∈[0,T ] b(t) (121)
ensures global asymptotic stability; see (89). It holds if ( 110) is satisfied. This concludes the proof.

7 Other Examples

First Illustration of Corollary 1

First consider the one dimensional system

ẋ(t) = -[1 + 2 cos(t)]x(t -τ ) (122)
with τ ≥ 0 being constant. Determining conditions ensuring that this system is exponentially stable is challenging because it is a time-varying system whose stability is caused by a term with delay and because no obvious Lyapunov-Krasovskii functional can be proposed.

For all t ≥ 2τ , we have

ẋ(t) = -[1 + 2 cos(t)]x(t) + [1 + 2 cos(t)] t t-τ ẋ(m)dm = -[1 + 2 cos(t)]x(t) - t-τ t-2τ [1 + 2 cos(t)][1 + 2 cos(m + τ )]x(m)dm. (123)
We analyze the system (123) via Corollary 1. Choosing V(x) = |x| and η = 2τ , we get

V(t) ≤ -[1 + 2 cos(t)]V(x(t)) + t-1 2 η t-η |1+2 cos(t)||1+2 cos(m + τ )|V(x(m))dm. ( 124
)
We choose ω = 0, a(t) = -[1 + 2 cos(t)], c(m, t) = |1 + 2 cos(t)||1 + 2 cos(m + τ )|, and b = 0. By (40), we have

h(t) ≤ t t-η e m+η t [1+2 cos( )]d c(t, m + η)dm ≤ 1 + 2 cos t + η 2 × t t-η I(m, η, t)|1 + 2 cos(m + η)|dm , (125)
where I(m, η, t) = e m+η-t+2(sin(m+η)-sin(t)) . Since |sin(m + η) -sin(t)| ≤ |m + η -t|, it follows from Hölder's inequality that

h(t) ≤ 1 + 2 cos t + η 2 × t t-η e m+η-t+2(m+η-t) |1 + 2 cos(m + η)|dm = 1 + 2 cos t + η 2 × t+η t e 3(m-t) |1 + 2 cos(m)|dm ≤ 1 + 2 cos t + η 2 × e 6η -1 6 t+η t (1 + 2 cos(m)) 2 dm = √ e 6η -1 √ 6 1 + 2 cos t + η 2 × t+η t
(1 + 4 cos(m) + 4 cos 2 (m))dm .

Let us choose

T = τ = 2π. Then t+2π t [a( ) + h( )]d ≤ -2π + √ e 6η -1 √ 6 t+2π t 1 + 2 cos + η 2 G a (η, )d ≤ -2π + √ e 6η -1 √ 6 3 √ η t+2π t 1 + 2 cos + η 2 d ,
where G a (η, ) = (3η + 4 sin( + η)-

4 sin( ) + 2 +η cos(2m)dm 1/2 .
Since cos is periodic of period 2π, we deduce that

t+2π t [a( ) + h( )]d ≤ -2π + √ e 6η -1 √ 2 √ 3η 2π 0 |1 + 2 cos ( )| d = -2π + 6η (e 6η -1) π 0 |1 + 2 cos ( )| d , (126)
where we also used the fact that

2π π |1 + 2 cos( )|d = 2π π |1 + 2 cos(2π -)|d = π 0 |1 + 2 cos( )|d .
Hence, Assumption 6 will be satisfied if the right side of ( 126) is negative, i.e., if η (e 6η -1) <

√ 2π √ 3 π 0 |1 + 2 cos ( )| d = √ 2π √ 3(π/3 + 2 √ 3) (127)
holds, so this is a sufficient condition for the conclusions of Corollary 1 to hold.

Second Illustration of Corollary 1

We choose the system ẋ(t) = sin(t)x(t) + u(t -τ ) (128)

and the control u(t) = -[sin(t+τ )+1]x(t). Then the closed loop system can be rewritten as ẋ(t) = -x(t) + [sin(t) + 1][x(t) -x(t -τ )] and so also as ẋ(t) = -x(t) +

Hence, the upper bound will exist provided η < ln(5/4). Hence, we get the stability condition τ < 1 2 ln 5 4 .

Then the proof of Corollary 1 provides a Lyapunov-Krasovskii functional construction.

Discrete Time Example

We illustrate Theorem 3. Let p > 1 be an integer, and define {β 1 , β 2 , . . . , β p } by β k = 1 2(p+1) for all k ∈ {1, ..., p -1} and β p = 3. (133) Then we extend {β 1 , β 2 , . . . , β p } to form a sequence, by setting β k+p = β k for all k ∈ N. Then for each k ≥ 0, the set {β 1+k , β 2+k , . . . , β p+k } has exactly one 3. For each j ∈ {0, ..., p}, we define the sequence {α k,j } by α k,j = β k for all k ∈ N. Then ( 69

which holds if p > 3. This provides sufficient conditions for Assumption 7 to hold, with a 1 = 1/2 and a 2 = 3(p + 1). Also, Assumption 8 will be satisfied if 3 2 p-1 (p + 1) p-2

1/p < 1 2 p/(p+1) , (135)
which can be satisfied if we choose p large enough, and then Assumptions 7-8 will both be satisfied. Hence, we have given sufficient conditions for the assumptions of Theorem 3 to hold.

Conclusions

Time delay systems with time-varying delays play a central role in controls, but their stability analysis is beyond the scope of traditional Lyapunov-Krasovskii functional or other standard techniques. To help overcome these significant challenges, we applied a strictification approach to extend Razumikhin's theorem for time-varying systems.

The approach entails converting a nonstrict Lyapunov-like function into a strict one, but was beyond the scope of the strictification methods in [START_REF] Malisoff | Constructions of Strict Lyapunov Functions[END_REF].

A key feature of our analysis is that we do not require the Lyapunov functional to decay along trajectories when the delay is set to 0, and our strictification analysis is outside the scope of existing results on the Razumikhin approach. We illustrated how our method compares with results that were obtained in earlier literature. We also provided new constructions for Lyapunov-Krasovskii functionals for time-varying delay systems, as well as analogous results for discrete time systems. In future work, we plan to extend our analysis to adaptive cases where the objectives include both tracking and parameter identification.

  a bounded continuous function a : R → R, and bounded continuous functions b : R → [0, ∞) and c : R 2

  d b(t + τ ) and c * (t, s) = e t s+η a( )d c(t, s + η). (49) Since b and c are nonnegative valued, it follows that b * and c * are nonnegative valued. Also, since λ = a + h, the estimates (48) give

  by upper bounding the last integrand by τ . Hence, (76)-(77) from Assumption 9 are satisfied if * > 0 and e This leads to the delay dependent condition λ µ τ T < 2 ln 1 + * b .

  -1 + b(m)) dm = 0, so (76) from Assumption 9 is satisfied. Hence, all requirements from Assumption 9 will be satisfied ifsup t∈[0,1] e κ(t) -1 b(t) -(t) < 0 .(100)Since b = 0 on [0, c) and is positive valued, it follows that (

  [sin(t) + 1] sin(m)x(m)dm t-τ t-2τ [sin(t) + 1][sin(m + τ ) + 1]x(m)dm(129) for all t ≥ 2τ . Along all trajectories of (129), the functionV (x) = |x| satisfies V (t) ≤ -V (x(t)) + t t-τ [sin(t) + 1]V (x(m))dm + t-τ t-2τ [sin(t) + 1][sin(m + τ ) + 1]V (x(m))dm ≤ -V (x(t))+ t t-2τ [sin(t) + 1][| sin(m + τ )| + 1]V (x(m))dm .We apply Corollary 1 with b = 0, T = 2π, η = 2τ , and ω = 0. We set a(t) = -1 and c(m, t) = (sin(t) + 1)(| sin(m + τ )| + 1). To apply Corollary 1, we need the function n[| sin(m + η/2)| + 1] × m m-η e +η-m [|sin( + η)| + 1] d dm to have a negative upper bound. We find sufficient conditions for such an upper bound to exist. To this end, first note that since (| sin(m + η/2)| + 1) m m-η e +η-m [|sin( + η)| + 1] d is a periodic function of m having period 2π in m, we get n(t) = -1 + 1 2π 2π 0 (| sin(m + η/2)| + 1) × η 0 e [|sin( + m)| + 1] d dm . (130) Next observe that n(t) ≤ -1 + e η -1 π 2π 0 (| sin(m + η/2)| + 1)dm = -1 + e η -1 π 4π = -5 + 4e η .

  )-(70) become α k = (p + 1)β k and

Acknowledgement

We thank Profs. Zhou and Egorov for sharing their inspiring information about their work on delay systems.

Malisoff was supported by NSF-ECCS Grants 1102348 and 1408295.