Nicolas Forcadel

Cyril Imbert

Régis Monneau

The twin blow-up method for Hamilton-Jacobi equations in higher dimension

In this paper, we show how to extend the twin blow-up method recently developped by the authors (Comptes Rendus. Math., 2024), in order to obtain a new comparison principle for an evolution coercive Hamilton-Jacobi equation posed in a domain of an Euclidian space of any dimension and supplemented with a boundary condition. The method allows dealing with the case where tangential variables and the variable corresponding to the normal gradient of the solution are strongly coupled at the boundary. We elaborate on a method introduced by P.-L. Lions and P. Souganidis (Atti Accad. Naz. Lincei, 2017). Their argument relies on a single blow-up procedure after rescaling the semi-solutions to be compared while two simultaneous blow-ups are performed in this work, one for each variable of the classical doubling variable technique. A one-sided Lipschitz estimate satisfied by a combination of the two blow-up limits plays a key role.

Introduction

This work is concerned with strong uniqueness (comparison principle) of viscosity solutions to a Hamilton-Jacobi equation of evolution type of the form, u t + H(X, Du) = 0 on (0, T ) × Ω (1.1)

where X := (t, x), supplemented with the (desired) boundary condition u t + F (X, Du) = 0 on (0, T ) × ∂Ω and the initial condition u(0, •) = u 0 on {0} × Ω.

(1.

2)

The spatial domain Ω is a subset of the Euclidian space of dimension d ≥ 1. We will first see how to deal with a half-space and we will then consider the case of a C 1 bounded domain.

Since the desired boundary condition can be lost when characteristics reach ∂Ω, it has to be imposed in a weak sense. In the viscosity solution framework, the weak sense means that either the desired boundary condition is satisfied or the PDE is satisfied on the boundary. More precisely, subsolutions and supersolutions of (1.1) are assumed to satisfy at the boundary the following inequalities, u t + min(F, H)(X, Du) ≤ 0 on (0, T ) × ∂Ω (subsolutions), u t + max(F, H)(X, Du) ≥ 0 on (0, T ) × ∂Ω (supersolutions). (1.3) We present in the introduction the comparison principle for (1.1), (1.2), (1.3) with Ω = R d-1 × (0, +∞). In order to present the structure conditions imposed to the Hamiltonian H and the nonlinearity F associated with the boundary condition, we set x = (x ′ , x d ) ∈ R d-1 × [0, +∞) and p = (p ′ , p d ) ∈ R d-1 × R for the variable for the gradient. In particular, p ′ corresponds to the space tangential gradient of the solutions and p d to the normal gradient. In the following assumption, ω, ω L denote moduli of continuity.

                                                         i) (Continuity and bound) H : [0, T ] × Ω × R d → R is continuous the map X → H(X, 0) is bounded.
ii) (Uniform continuity in the gradient) For any L > 0, we have for all X ∈ [0, T ] × Ω and p, q ∈ [-L, L] d |H(X, p) -H(X, q)| ≤ ω L (|p -q|).

iii) (Continuity in the tangential variables) For X = (t, x ′ , x d ) and Y = (s, y ′ , x d ) with t, s ∈ [0, T ] and x ′ , y ′ ∈ R d-1 and x d ≥ 0

H(Y, p) -H(X, p) ≤ ω(|Y -X| (1 + |p ′ | + max {0, H(X, p)})).

iv) (Uniform normal coercivity)

For any L > 0, we have lim (1.4) and making artificially appear the dependence on x ∈ Ω for F (in order to unify the presentation of H and F ), we consider

                                                        
i) (Continuity, bound and monotonicity) F : [0, T ] × ∂Ω × R d → R is continuous, the map X → F (X, 0) is bounded, the map p d → F (X, p ′ , p d ) is nonincreasing.

ii) (Uniform continuity in the gradient) For any L > 0, we have for all X ∈ [0, T ] × ∂Ω and p, q ∈ [-L, L] d |F (X, p) -F (X, q)| ≤ ω L (|p -q|).

iii) (Continuity in the tangential variables) for all X, Y ∈ [0, T ] × ∂Ω and p ∈ R d , F (Y, p) -F (X, p) ≤ ω(|Y -X| (1 + |p ′ | + max {0, max(F, H)(X, p)})).

iv) (Uniform normal semi-coercivity)

For any L > 0, we have lim

p d →-∞ inf{F (X, p ′ , p d ) : X ∈ [0, T ] × ∂Ω, p ′ ∈ [-L, L] d-1 } = +∞.
(1.5)

Under the previous structural conditions, sub and super-solutions of the Hamilton-Jacobi equation under study can be compared.

Theorem 1.1 (A comparison principle with strong tangential coupling). Let T > 0 and assume that H, F satisfy (1.4)- (1.5). Assume that the initial data u 0 is uniformly continuous. Let u, v : [0, T ) × Ω → R be two functions with u upper semi-continuous and v lower semi-continuous. Assume that u (resp. v) is a viscosity subsolution (resp. supersolution) of (1.1)-(1.2). Assume moreover that there exists a constant

C T > 0 such that u ≤ u 0 + C T and v ≥ u 0 -C T on [0, T ) × Ω. (1.6) If we have u(0, •) ≤ u 0 ≤ v(0, •) on {0} × Ω then we have u ≤ v on [0, T ) × Ω.
Remark 1.2. A simplified version of Theorem 1.1 is presented in [START_REF] Forcadel | Coercive Hamilton-Jacobi equations in domains: the twin blow-ups method[END_REF]. It was assumed in this note that dimension d = 1 and that initial data are Lipschitz continuous. Some details were skept and they are presented in this new work.

Remark 1.3. In Section 5, we also extend this result to the case where Ω is a C 1 bounded open set.

Remark 1.4. Notice that, given (1.4), we can always define the state constraint boundary function

H -(X, p ′ , p d ) := inf q d ≤p d H(X, p ′ , q d ) for X ∈ [0, T ] × ∂Ω and p = (p ′ , p d ) ∈ R d-1 × R
and it satisfies (1.5). Up to our knowledge, the comparison principle was also an open problem for F = H - in this generality.

Remark 1.5. Notice that in Theorem 1.1, semi-coercivity of F in condition (1.5) iv) can be replaced by the weak continuity of the subsolution u on the boundary (0, T ) × ∂Ω, using [4, Proposition 3.12]) and replacing F by

F 1 := max(F, H -).
Main contribution. When comparing non-Lipschitz sub/supersolutions (for instance after constructing solutions by Perron's method), a strong coupling between tangential coordinates (t, x ′ ) ∈ [0, T ] × R d-1 and the normal gradient ∂ x d u is well identified in the literature as a technical difficulty, especially when this coupling arises in the boundary condition, see for instance [START_REF] Barles | Solutions de Viscosité des Équations de Hamilton-Jacobi[END_REF][START_REF] Barles | Nonlinear Neumann Boundary Conditions for Quasilinear Degenerate Elliptic Equations and Applications[END_REF][START_REF] Barles | An Illustrated Guide of the Modern Approaches of Hamilton-Jacobi Equations and Control Problems with Discontinuities[END_REF][START_REF] Lions | Viscosity solutions for junctions: well posedness and stability[END_REF][START_REF] Lions | Well-posedness for multi-dimensional junction problems with Kirchofftype conditions[END_REF].

It is standard to make the (strong) assumption of uniform continuity in time t, uniformly in the gradient Du. Such an assumption is not satisfied by the following simple example,

u t + a(X)|Du| = 0 in (0, T ) × Ω, u t + max {0, -b(X)∂ x d u} = 0 in (0, T ) × ∂Ω (1.7)
when a, b ≥ 1 are bounded Lipschitz continuous functions (here with b(t, x) = b(t, x ′ , 0)).

Comparison with known results. J. Guerand [START_REF] Guerand | Effective nonlinear Neumann boundary conditions for 1D nonconvex Hamilton-Jacobi equations[END_REF] proved a comparison principle in our geometric setting in dimension d = 1 in the case where H and F are independent of (t, x). She also proved a comparison principle for non-coercive Hamiltonians. P.-L. Lions and P. Souganidis [START_REF] Lions | Well-posedness for multi-dimensional junction problems with Kirchofftype conditions[END_REF] introduced a new method for proving comparison principles for bounded uniformly continuous sub/supersolutions for equations posed on junctions with several branches (or halfspaces). They use a blow-up argument that reduces the study to a 1D problem. They show the comparison principle in the case of Kirchoff-type boundary conditions and non-convex Hamiltonians. As far as (t, x) dependence is concerned, their method allows them to handle Hamiltonians that are Lipschitz continuous in t, see [START_REF] Lions | Well-posedness for multi-dimensional junction problems with Kirchofftype conditions[END_REF]Assumption (4)].

This result is generalized by G. Barles and E. Chasseigne [START_REF] Barles | An Illustrated Guide of the Modern Approaches of Hamilton-Jacobi Equations and Control Problems with Discontinuities[END_REF]Theorem 15.3.7,page 295] to the case of bounded semi-continuous sub/supersolutions under three different junction conditions. Even if they are presented for N = 2 branches, we present their results in our geometric setting: a junction reduced to a single branch N = 1 in dimension d ≥ 1. The three cases are the following: (1) F is independent on p d , (2) the Neumann problem and (3) general nonincreasing continuous p d → F (X, p ′ , p d ). In the third case, the normal derivative is not coupled with the tangential coordinates (t, x ′ ) in F (see also the very end of [3, Subsection 13.2.2 and condition (GA-G-FLT) p. 247]).

As explained above, we improve these results, using the twin blow-up method introduced in [START_REF] Forcadel | Coercive Hamilton-Jacobi equations in domains: the twin blow-ups method[END_REF]. A close look at the proof reveals that new ideas appear at the beginning of Step 4, when the reasoning focuses on the case where the point of maximum is on the boundary of the domain. Compared to the note [START_REF] Forcadel | Coercive Hamilton-Jacobi equations in domains: the twin blow-ups method[END_REF], we also extend the result by considering uniformly continuous initial data (and not only Lipschitz continuous ones) and working in dimension greater than one.

Organization of the paper. In Section 2, we present two key boundary results stated for stationary problems in space dimension d = 1. We also extend these results to the case of junctions (that will be used in future works). In Section 3, we recall two classical results which are suitable for our purpose. We first construct barriers. We next present some a priori estimates for the sup-convolution of subsolutions to coercive HJ equations. The proof of the comparison principle in the case of the half space (Theorem 1.1) is done in Section 4. Finally in Section 5, we show how to adapt our twin blow-up method to the case of a C 1 bounded open domain.

Acknowledgements. The authors thank G. Barles and E. Chasseigne for enlighting discussions during the preparation of this work. The last author also thanks J. Dolbeault, C. Imbert and T. Lelièvre for providing him good working conditions. This research was partially funded by l'Agence Nationale de la Recherche (ANR), project ANR-22-CE40-0010 COSS. For the purpose of open access, the authors have applied a CC-BY public copyright licence to any Author Accepted Manuscript (AAM) version arising from this submission.

Boundary lemmas

In this section, we work in dimension d = 1 and set Ω := (0, +∞). We present some fundamental boundary results that will allow us to prove our comparison principle. At the end of this section, we also extend them naturally to the case of junctions (that will be useful for future works).

Before to state our result, we need to introduce the following notion of (limiting) semi-differentials.

Definition 2.1 ((Limiting) semi-differentials). Let A ⊂ Ω and x 0 ∈ A. For (+/-), we define the (first order) super/subdifferential at x 0 of a function u on A as

D ± A u(x 0 ) = {p ∈ R, such that 0 ≤ ± {u(x 0 ) + p • (x -x 0 ) + o(x -x 0 ) -u(x)} on A} (2.1)
and the limit (first order) super/subdiffential at the boundary point x 0 ∈ ∂Ω of u as

D± Ω u(x 0 ) = p ∈ R, there exists a sequence p k ∈ D ± Ω u(x k ) with x k ∈ Ω and (x k , p k ) → (x 0 , p) . (2.2) Remark 2.2. Note that if p ∈ D+ Ω u(x 0 ) with x 0 ∈ ∂Ω, and if u is a subsolution of H(Du) ≤ 0 in Ω, then H(p) ≤ 0.
We then have the following result.

Lemma 2.3 (Critical slopes and semi-differentials).

Let Ω := (0, +∞). We consider two functions u, v : Ω → R ∪ {-∞, +∞} with u upper semi-continuous and v lower semicontinous satisfying u(0) = 0 = v(0) with u ≤ v on Ω. We define the critical slopes by

p := lim sup Ω∋x→0 u(x) x , p := lim inf Ω∋x→0 v(x) x . ( 2

.3)

Then we have the following (limiting) semi-differential inclusions

R ∩ p, p ⊂ D+ Ω u(0) ∩ D- Ω v(0) if p ≥ p (2.4) R ∩ p, p ⊂ D + Ω u(0) ∩ D - Ω v(0) if p ≤ p (2.5) p ∈ D+ Ω u(0) if p = -∞ p ∈ D- Ω u(0) if p = +∞. (2.6)
Proof. The proof of this lemma is already contained in [START_REF] Forcadel | Coercive Hamilton-Jacobi equations in domains: the twin blow-ups method[END_REF] but for sake of completeness, we give it here. We first notice that (2.5) is a straightforward consequence of the definition of sub and superdifferentials.

In order to prove (2.4), we first focus on the proof of

R ∩ p, p ⊂ D+ Ω u(0) in case p > p (2.7)
and we will even show the follower better result

R ∩ q, p ⊂ D+ Ω u(0) in case p > q := lim inf Ω∋x→0 u(x) x . (2.8)
Note that u ≤ v implies q ≤ p and so (2.7) is a consequence of (2.8). The claim is a variant of (18) in [START_REF] Lions | Well-posedness for multi-dimensional junction problems with Kirchofftype conditions[END_REF] and the proof is a variant of the one done in Barles, Chasseigne [START_REF] Barles | An Illustrated Guide of the Modern Approaches of Hamilton-Jacobi Equations and Control Problems with Discontinuities[END_REF]Lemma 15.3.1]. We give the details for sake of completeness. We first assume that p ∈ (q, p). This implies that lim sup

Ω∋x→0 u(x) x = p > p > q = lim inf Ω∋x→0 u(x) x
and so for any ε > 0, there exists y ε ∈ (0, ε) and z ε ∈ (0, y ε ) such that

u(z ε ) z ε > p > u(y ε ) y ε .
Hence the function ζ(x) := u(x)px satisfies

ζ(0) = 0 > ζ(y ε ) with M := sup [0,yε] ζ ≥ ζ(z ε ) > 0.
Let x ε ∈ (0, y ε ) be a point of maximum of ζ in [0, y ǫ ]. We see that the function x → px + M is a test function touching u from above at x ε , which implies that p ∈ D + Ω u(x ε ). In the limit ε → 0, we recover p ∈ D+ Ω u(0) which proves the claim. In the case where p ∈ [q, p], we get the result by the closedness of D+ Ω u(0). This proves (2.8). A similar inclusion for v implies (2.4) in the special case where p > p. On the other hand, notice that (2.6) implies (2.4) in the case p = p.

Hence it remains to show (2.6). We claim that

p ∈ D- Ω v(0) if p ∈ R. (2.9)
This result is a property of the critical slope for lower semi-continuous functions. Its proof follows exactly the lines of [7, Proof of Lemma 2.9] (where the proof does not use any Hamiltonian). A similar result holds for u and proves (2.6). This ends the proof of the lemma.

Before to state the fundamental lemma for the comparison principle, we recall the definition of (semi-) coercive functions.

Definition 2.4 (Coercive and semi-coercive functions). Consider a function

G : R → R. Then G is coercive if lim |p|→+∞ G(p) = +∞, and semi-coercive if lim p→-∞ G(p) = +∞.
As a consequence of Lemma 2.3, we have the following result which will be used to prove the comparison principle.

Corollary 2.5 (Boundary viscosity inequalities).

Let Ω and u, v be as in statement of Lemma 2.3. For γ = α, β, consider continuous functions H γ , F γ : R → R with H α coercive and F α semi-coercive. Assume that we have the following viscosity inequalities for some η > 0

           H α (u x ) ≤ 0 on Ω ∩ {|u| < +∞} min {F α , H α } (u x ) ≤ 0 on {0} ∩ {|u| < +∞} H β (v x ) ≥ η on Ω ∩ {|v| < +∞} max {F β , H β } (v x ) ≥ η on {0} ∩ {|v| < +∞} .
(2.10)

For p, p defined in (2.3), we set a := min p, p and b := max p, p . Then p ∈ [a, b] ∩ R and there exists a

real number p ∈ [a, b] such that either H α (p) ≤ 0 < η ≤ (H β -H α )(p) or max {F α , H α } (p) ≤ 0 < η ≤ (F β -F α )(p). (2.11)
Proof. The main steps of the proof is given in [START_REF] Forcadel | Coercive Hamilton-Jacobi equations in domains: the twin blow-ups method[END_REF], but for sake of completeness, we give all the details here. We begin to explain why p ∈ R. Because H α is coercive and F α is semi-coercive, we know from [4, Lemma 3.8] that u is weakly continuous at x = 0, i.e.

0 = u(0) = lim sup Ω∋x→0 + u(x).
(2.12) Then [7, Proof of Lemma 2.10] shows additionally that p > -∞. Now we claim that we also have p < +∞. Indeed, assume by contradiction that p = +∞. Then, there exists y n → 0 such that

p n := u(y n )/y n → +∞. For b ∈ R, let us define φ b (x) := p n x + b and b = inf{b, u ≤ φ b in [0, y n ]}.
In particular, there exists

x n ∈ [0, y n ] such that φ b touches u from above at x n . If x n = 0, then 0 = u(0) -φ b (0) = -b. In the same way, if x n = y n , then u(y n ) = φ b (y n ) = u(y n ) + b
and we recover again that b = 0. This implies that u(x) ≤ p n x and so

+∞ = lim sup x→0 u(x) x ≤ p n < +∞.
We then deduce that x n ∈ (0, y n ) and since u is a sub-solution, we get

H α (p n ) ≤ 0
which is absurd for n large enough by coercivity of H α . This implies that p < +∞. We conclude that

p ∈ R ∩ [a, b].
We now turn to the proof of (2.11). If p ≤ p, then (2.4) shows, for all p ∈ p, p ∩ R, that

H α (p) ≤ 0 < η ≤ H β (p)
which implies in particular the desired conclusion. We now assume that p > p. We have in particular [a, b] ⊂ (-∞, +∞] with a < b and

   H α (a) ≤ 0 because a ∈ R 0 < η ≤ H β (b) if b ∈ R min {H α , F α } ≤ 0 < η ≤ max {H β , F β } on [a, b] ∩ R (2.13)
where the last line follows from (2.5), and the first two lines follow from (2.6).

We now claim that for all ε > 0 small enough, there exists some

p ε ∈ [a, b] ∩ R such that we have at p ε i) H α ≤ ε < η -ε ≤ H β -H α or ii) max {F α , H α } ≤ ε < η ≤ F β -F α . (2.14)
By contradiction, we assume that there exists ε > 0 (small enough) such that

   i) H β -H α < η -ε or ε < H α and ii) F β -F α < η or ε < max {F α , H α } for all p ∈ [a, b] ∩ R. (2.15)
Recall that the coercivity of H α means H α (±∞) := lim inf p→±∞ H α (p) = +∞. We distinguish two cases.

Case 1: H α (b) > ε Here b can be finite or equal to +∞. We get

H α (b) > ε > 0 ≥ H α (a).
Therefore by continuity, there exists p ∈ (a, b) such that H α (p) = ε. Hence in the last line of (2.13), the first inequality implies that F α (p) ≤ 0. Because (2.15) i) and ii) hold true for p, we get

H β (p) < η and F β (p) < η
which leads to a contradiction with the second inequality in the last line of (2.13).

Case 2:

H α (b) ≤ ε Then b ∈ R and (2.15) i) implies for p = b that H β (b) < η,
which is in contradiction with the second line of (2.13).

In all the cases, we get a contradiction, which proves (2.14). Since H α is coercive, we see in both cases i) or ii) of (2.14), that we can always extract a subsequence as ε → 0 such that p ε → p ∈ [a, b] ∩ R. Passing to the limit in (2.14), we get the desired conclusion. This ends the proof of the corollary.

Notice that it is very easy to show the following extension of Corollary 2.5 to the case of junctions. Proposition 2.6 (Junction viscosity inequalities). For N ≥ 1, let J i := (0, +∞) for i = 1, . . . , N , and set

J := {0} ∪   i=1,...,N J i  
with the topology of glued branches. For a piecewise C 1 function u on J, and u i := u |J i ∪{0} , we set

u x (x) = (u 1 x (0), . . . , u N x (0)) if x = 0 u i x (x) if x ∈ J i .
We consider two sets of functions u, v : J → R ∪ {-∞, +∞} with u upper semi-continuous and v lower semicontinous satisfying u(0

) = 0 = v(0) with u ≤ v on J. (2.

16)

For i = 1, . . . , N , we define

p i := lim sup J i ∋x→0 u(x) x , p i := lim inf J i ∋x→0 v(x) x .
(2.17)

We also set

a i := min p i , p i , b i := max p i , p i and [a, b] ∩ R N := i=1,...,N [a i , b i ] ∩ R.
For γ = α, β, consider continuous functions H i γ : R → R and F γ : R N → R with H i α coercive and F α semi-coercive. For p = (p 1 , . . . , p N ) ∈ R N , we set

H γ;min (p) = min i=1,...,N H i γ (p i ), H γ;max (p) = max i=1,...,N H i γ (p i ).
Then assume that we have the following viscosity inequalities for some η > 0

           H i α (u x ) ≤ 0 on J i ∩ {|u| < +∞} for i = 1, . . . , N min {F α , H α;min } (u x ) ≤ 0 on {0} ∩ {|u| < +∞} H i β (v x ) ≥ η on J i ∩ {|v| < +∞} for i = 1, . . . , N max {F β , H β;max } (v x ) ≥ η on {0} ∩ {|v| < +∞} .
(2.18)

Then there exists p = (p 1 , . . . , p N ) ∈ [a, b] ∩ R N = ∅ such that either H i α (p i ) ≤ 0 < η ≤ (H i β -H i α )(p i ) for some i ∈ {1, . . . , N } or max(F α , H α;max )(p) ≤ 0 < η ≤ (F β -F α )(p). (2.19)
3 Barriers and regularization Lemma 3.1 (Barriers). Let T > 0 and assume that H, F satisfy (1.4)-(1.5), and that the initial data u 0 is uniformly continuous. Assume that u (resp. v) is an upper semi-continuous subsolution (resp. a lower semi-continuous supersolution) of (1.1), (1.3), satisfying the a priori bounds (1.6) for some constant C T .

Then there exists a continuous increasing function f

: [0, T ] → [0, +∞) with f (0) = 0 such that the functions u ± (t, x) := u 0 (x) ± f (t)
satisfy the following barrier properties:

• if u ≤ u 0 in {0} × Ω, then u ≤ u + in [0, T ) × Ω, • if v ≥ u 0 in {0} × Ω, then v ≥ u -in [0, T ) × Ω.
Proof. The idea of the proof is somehow very standard. We first extend by continuity the initial data defined on

R d-1 × [0, +∞) to a function defined on R d-1 × R, setting u 0 (x ′ , x) := u 0 (x ′ , 0) for all x ≤ 0. Hence u 0 still satisfies |u 0 (x) -u 0 (y)| ≤ ω 0 (|x -y|) for all x, y ∈ R d
where ω 0 is the modulus of continuity of (u 0 )| R d-1 ×[0,+∞) . We do the proof to compare u and u + , the one to compare v and u -being similar.

Case 1:

u 0 ∈ (C 1 ∩ Lip)(R d )
In this case, there exists some

L > 0 such that |Du 0 | L ∞ (R d ) ≤ L.
From assumptions i) and ii) of both (1.4) and (1.5), we see that there exists λ = λ(T, L) ≥ 0 minimal such that, for

B L := B L (0), -λ ≤ inf [0,T ]×Ω×B2L min {H, F } ≤ sup [0,T ]×Ω×B2L max {H, F } ≤ λ (3.1)
where we have extended the function F as follows: F (t, x ′ , x d , p) := F (t, x ′ , 0, p) for all x d ≥ 0. Setting f (t) := λt, we see that u + is a supersolution of (1.1), (1.3). Assume now by contradiction that

M := sup QT (u -u + ) > 0 with Q T := [0, T ) × Ω. Now for η, α > 0, let us consider M η,α := sup QT Φ with Φ(t, x) := u(t, x) -u + (t, x) - η T -t - α 2 x 2 .
For η, α > 0 small enough, we have M η,α ≥ M/2 > 0. Moreover from the bound (1.6) on u, we see that the supremum in M η,α is reached for some point X = ( t, x) ∈ Q T . We also have lim sup

(η,α)→(0,0) η T - t + α 2 x2 = 0
and then we can fix η, α > 0 small enough such that |αx| ≤ L. Assume that t = 0. Then 0 < M/2 ≤ M η,α = Φ(0, x) ≤ -η T which leads to a contradiction. Hence t > 0 and X = ( t, x) = ( t, x′ , xd ) ∈ (0, T ) × R d-1 × [0, +∞). Therefore we have the viscosity inequalities for

p := Du 0 (x) ∈ B L and αx ∈ B L η (T -t) 2 + λ + H( X, p + αx) ≤ 0 if xd > 0 η (T -t) 2 + λ + max(F, H)( X, p + αx) ≤ 0 if xd = 0
which leads to a contradiction from the choice of λ in (3.1). This implies that M ≤ 0 and then u ≤ u + .

Case 2: u 0 is only uniformly continuous Let ϕ be a smooth nonnegative function satisfying ϕ = 0 on R n \B 1 (0) and R n ϕ(x)dx = 1. For ε > 0, we set the convolution

u ε 0 := ϕ ε ⋆ u 0 with ϕ ε (x) = 1 ε n ϕ( x ε ). Hence we have |u ε 0 -u 0 | L ∞ (R n ) ≤ ω 0 (ε)
and

Du ε 0 (x) = 1 ε R n dy 1 ε n Dϕ( y ε ) {u 0 (x -y) -u 0 (x)}.
Therefore, we get

|Du ε 0 | L ∞ (R) ≤ |Dϕ| L 1 (R) • ω 0 (ε) ε ≤ L ε := |Dϕ| L 1 (R n ) • sup δ≥ε ω 0 (δ) δ .
We define λ ε = λ T,Lε ≥ 0 minimal such that

-λ ε ≤ inf [0,T ]×Ω×B2L ε min(H, F ) ≤ sup [0,T ]×Ω×B2L ε max(H, F ) ≤ λ ε ,
where by construction the map ε → λ ε is nonincreasing, and we set

f ε (t) := λ ε t. Using that u 0 (x) ≤ u ε 0 (x) + ω 0 (ε), we can show as in Case 1 that u(t, x) ≤ u ε 0 (x) + ω 0 (ε) + λ ε t ≤ u 0 (x) + 2ω 0 (ε) + λ ε t. If we set f (t) := inf ε>0 {2ω 0 (ε) + λ ε t} ,
where f is a (continuous) concave nondecreasing function satisfying f (0) = 0, we get u ≤ u + . This ends the proof of the lemma.

We now consider a (classical) regularization of a subsolution u by tangential sup-convolutions. Because we only assume a bound from above u ≤ u 0 + C T , we have additionally to truncate u from below by some function. We will use the function u 0 := u 0 -C T (which will also be later in the next section a bound from below for the supersolution v), where u 0 is the initial data, which is assumed to be Lipschitz continuous, in order to simplify the presentation. Then we have the following result. Lemma 3.2 (Tangential regularization after truncation by Lipschitz initial data). Let T > 0 and assume that H satisfies (1.4) and that the initial data u 0 is Lipschitz continuous of Lipschitz constant L 0 . Let u be an upper semi-continuous subsolution of (1.1)-(1.2), satisfying moreover the a priori bound (1.6) for some constant C T , namely

u(t, x) ≤ u 0 (x) + C T for all (t, x) ∈ [0, T ) × Ω. (3.2)
We define u(T, x) := lim sup ), and extend u to R × Ω, setting u(t, x) := u(T, x) if t ≥ T , and u(t, x) := u(0, x) if t ≤ 0. We set

(s,y)→(T,x), s<T u(s, y) for all x ∈ Ω = R d-1 × [0, +∞
ũ := max(u, u 0 ) with u 0 (t, x) := u 0 (x) -C T .
We denote the tangential variable by ξ = (ξ 0 , ξ ′ ) = (s, x ′ ) ∈ R d and the normal variable by x d ∈ [0, +∞) and we define for ν > 0 the tangential sup-convolution

ũν (ξ, x d ) := sup ζ∈R d ũ(ζ, x d ) - |ξ -ζ| 2 2ν = ũ( ζ, x d ) - |ξ -ζ| 2 2ν
where each ζ depends on (ξ,

x d ) ∈ R d × [0, +∞) with |ζ -ξ| ≤ θ ν := 5(4νC T + ν 2 L 2 0 ) < T /2, for ν small enough.
Then the function ũν is Lipschitz continuous in R × Ω with respect to the variable ξ. Moreover it is Lipschitz continuous in I ν × Ω with respect to the variable x d , with

I ν := (θ ν , T -θ ν ), |D ξ ũν | L ∞ (R×Ω) ≤ θ ν ν and |∂ x d ũν | L ∞ (I ν ×Ω) ≤ max {L ν , L 0 } where L ν := sup p d ∈ R, inf (X,p ′ )∈([0,T ]×Ω)×B θ ν ν H(X, p ′ , p d ) ≤ θ ν ν .
Assume furthermore that u is a subsolution at the boundary (0, T ) × ∂Ω, i.e. satisfies the first line of (1.3) for some F satisfying (1.5). Then ũν is Lipschitz continuous in space and time on I ν × Ω of Lipschitz constant L ν := max θ ν ν , L ν , L 0 . Proof. The proof is splited into three steps.

Step 1: first bounds using the 2-sided bound We begin to show that

| ζ -ξ| ≤ θ ν . (3.3)
From the 1-sided bound (3.2) and the definition of u 0 and ũ, we get the 2-sided bound

|ũ(t, x) -u 0 (x)| ≤ C T for all (t, x) ∈ R × Ω. ( 3.4)
For ξ = (t, ξ ′ ) := (t, x ′ ) and z := x d (to simplify the notation), we have

ũ(ξ, z) ≤ ũν (ξ, z) = ũ( ζ, z) - |ξ -ζ| 2 2ν
with ζ = ( t, ζ′ ). Using (3.4), we then get

|ξ -ζ| 2 2ν -2C T ≤ u 0 ( ζ′ , z) -u 0 (ξ ′ , z) ≤ L 0 | ζ′ -ξ ′ |.
This implies

|ξ ′ -ζ′ | -νL 0 2 + |t -t| 2 ≤ 4νC T + ν 2 L 2 0 = (θ ν ) 2 5 .
We then deduce that |t -t|

≤ θ ν √ 5 , |ξ ′ -ζ′ | ≤ 2 θ ν √ 5 which implies (3.3). We now prove that |D ξ ũν | L ∞ (R×Ω) ≤ θ ν ν . (3.5)
For ξ a ∈ R d , we set ũν (ξ a , z) := sup

ζ∈R d ũ(ζ, z) - |ξ a -ζ| 2 2ν = ũ( ζa , z) - |ξ a -ζa | 2 2ν .
Hence, by definition, we have

ũν (ξ a , z) ≥ ũ( ζ, z) - |(ξ a -ξ) + ξ -ζ| 2 2ν = ũν (ξ, z) -(ξ a -ξ) • (ξ -ζ) ν - |ξ a -ξ| 2 2ν
and also by symmetry

ũν (ξ, z) ≥ ũν (ξ a , z) -(ξ -ξ a ) • (ξ a -ζa ) ν - |ξ -ξ a | 2 2ν i.e. |ũ ν (ξ a , x) -ũν (ξ, x)| |ξ a -ξ| ≤ max |ξ -ζ| ν , |ξ a -ζa | ν + |ξ a -ξ| 2ν ≤ θ ν ν + |ξ a -ξ| 2ν .
This implies that ũν is Lipschitz continuous in the tangential coordinates and (3.5).

Step 2: bounds on the normal gradient It is easy to check that ũν is upper semi-continuous (because this is the case for u itself and the supremum in ξ is locally taken in a compact set). Let ϕ be a test function touching ũν from above at X 0 := (t 0 , x 0 ) ∈ (I ν × Ω) ∩ {ũ ν > u ν 0 } and set ξ 0 := (t 0 , x ′ 0 ) and z 0 := (x 0 ) d . We have

u ν 0 (X 0 ) < ũν (X 0 ) := sup h∈R d ũ(ξ 0 + h, z 0 ) - |h| 2 2ν = ũ(ξ 0 + h0 , z 0 ) - | h0 | 2 2ν with | h0 | ≤ θ ν and ũ(ξ + h0 , z) - | h0 | 2 2ν ≤ ũν (ξ, z) ≤ ϕ(ξ, z) with equality at (ξ, z) = (ξ 0 , z 0 ). Setting φ(ξ, z) := ϕ(ξ -h0 , z) + | h0 | 2 2ν
, X0 := (ξ 0 + h0 , z 0 ), we then get

     ũ ≤ φ with equality at X0 ũ( X0 ) = ũν (X 0 ) + | h0| 2 2ν > u ν 0 (X 0 ) + | h0| 2 2ν = sup h∈R d u 0 (ξ 0 + h, z 0 ) -|h| 2 2ν + | h0| 2 2ν ≥ u 0 ( X0 ). Hence ũ = u at X0 ∈ [-| h0 |, | h0 |] + I ν × Ω ⊂ (0, T ) × Ω.
Because u ≤ ũ, we have that φ touches u from above at X0 and since u satisfies the viscosity inequalities on (0, T ) × Ω, we get

φt ( X0 ) + H( X0 , D φ( X0 )) ≤ 0 i.e. ϕ t (X 0 ) + H( X0 , Dϕ(X 0 )) ≤ 0.
Setting for θ > 0 and X = (t, x ′ , x d )

H θ (X, p) := min h∈B θ H((t, x ′ ) + h, x d , p),
we see that we have ϕ t (X 0 ) + H θ ν (X 0 , Dϕ(X 0 )) ≤ 0 which shows that ũν satisfies this viscosity inequality on (I ν × Ω) ∩ {ũ ν > u ν 0 }. Recall that we have ũ ≥ u 0 and so ũν ≥ u ν 0 with |Du ν 0 | ≤ |Du 0 | ≤ L 0 . Hence we get in the viscosity sense min -

θ ν ν + H θ ν (X, D ũν ), |D ũν | -L 0 ≤ 0 on I ν × Ω which implies that |∂ x d ũν | ≤ max {L ν , L 0 } on I ν × Ω.
Step which is again the case for ũ = max {u, u 0 }. By sup-convolution, it is then easy to check that this is also true for ũν (ξ,

x d ) = sup |ζ-ξ|≤θ ν ũ(ζ, x d ) - |ξ -ζ| 2 2ν
at least for all t ∈ I ν = (θ ν , Tθ ν ). Because ũν is uniformly Lipschitz continuous on I ν × Ω, we deduce that ũν is also Lipschitz continuous on I ν × Ω. Finally, because the bound on ∂ t ũν is uniform in space and time, we deduce that ũν is Lipschitz continuous on [θ ν , Tθ ν ] × Ω with the same Lipschitz constants. This ends the proof of the lemma.

The comparison principle on a half space

This Section is devoted to the proof of the comparison principle Theorem 1.1.

Proof of Theorem 1.1. The strategy of the proof is similar to the one of the note [START_REF] Forcadel | Coercive Hamilton-Jacobi equations in domains: the twin blow-ups method[END_REF] but need technical adaptations. We first follow the proof of the comparison principle in [START_REF] Imbert | Flux-limited solutions for quasi-convex Hamilton-Jacobi equations on networks[END_REF], but then modify the proof on the boundary, introducing the twin blow-ups method. Let η, θ > 0 and consider

M (θ) := sup Ψ(ξ, ζ, x d ), x d ∈ [0, +∞), ξ, ζ ∈ [0, T ) × R n-1 , |ξ -ζ| ≤ θ (4.1) with ζ = (s, x ′ ), x = (x ′ , x d ) and Ψ(ξ, ζ, x d ) := ũ(ξ, x d ) -v(ζ, x d ) - η T -s , ũ = max {u, u 0 } , u 0 (t, x) := u 0 (X) -C T =: u 0 (x)
where we choose carefully η T -s instead of η T -t , because we want to do later a doubling of variables which looks like a sup-convolution (in particular in time) to the function ũ. We want to prove that M := lim θ→0 M (θ) ≤ 0. Assume by contradiction that

M > 0. (4.2)
Step 0. Reduction to u 0 Lipschitz continuous By assumption, the initial data u 0 is uniformly continuous. We follow the line of Case 2 of the proof of Lemma 3.1. We first extend u 0 by the value u 0 (x ′ , 0) for z = x d ≤ 0 and x ′ ∈ R n-1 . For the ball B 1 = B 1 (0), we then consider a smooth nonnegative function ϕ satisfying ϕ = 0 on R n \B 1 with R n ϕ(x)dx = 1, and for β > 0, we set the convolution u 0,β := ϕ β ⋆ u 0 with ϕ β (x) = 1 β n ϕ( x β ). Then we can insure that

|u 0,β -u 0 | ≤ ω 0 (β), |Du 0,β | ≤ L β
where ω 0 is the modulus of continuity of u 0 and L β is some constant. We have in particular

u -ω 0 (β) ≤ u 0,β ≤ v + ω 0 (β) on {0} t × Ω.
Hence the problem is reduced to replace the quantities (u, u 0 , v) by (uω 0 (β), u 0,β , v + ω 0 (β) and M by M -2ω 0 (β), and keep C T unchanged. Therefore fixing some β := β 1 > 0 small enough such that 2ω 0 (β) < M , we see that we can redefine u, u 0 , v and assume without loss of generality that u 0 is Lipschitz continuous, say (forgetting now β) for some Lipschitz constant L 0 , with M > 0.

Step 1. Doubling naively the space variables We first consider a space penalization and standard doubling of variables in space and time, but we distinguish the tangential variables from the normal variable. To this end, we introduce parameters α, ν, δ > 0 and set

M ν,α,δ := sup (t,x),(s,y)∈[0,T )×Ω Ψ ν,α,δ (t, x, s, y) with for ξ = (t, x ′ ), ζ = (s, y ′ ), x = (x ′ , x d ), y = (y ′ , y d ) Ψ ν,α,δ (t, x, s, y) := ũ(t, x) -v(s, y) -αg(y) - η T -s - |(t, x ′ ) -(s, y ′ )| 2 2ν - |x d -y d | 2 2δ , g(y) := y 2 2 (4.3) which satisfies lim inf α→0 lim inf δ→0 M ν,α,δ ≥ M > 0.
Hence we see that (independently on ν > 0) for α > 0 small enough, and for δ > 0 small enough (say δ ∈ (0, δ α ]), we get

M ν,α,δ ≥ M/2 > 0. (4.4)
In particular, the maximum is reached at some point that ( Xδ , Ȳδ ) = ( tδ , xδ , sδ , ȳδ ) and we claim that we have the following estimate.

Lemma 4.1 (Bounds on any optimizing sequence). Given T, C T > 0, there exists η > 0 small enough such that the following holds true. Let X, Y ∈ [0, T ) × Ω with X = (t, x), Y = (s, y) be such that

0 < Ψ ν,α,δ (X, Y ).
Then for ν, δ > 0 small enough (depending on η > 0), we have

                         t, s ∈ [τ η , T -τ η ], τ η := η 4C T αg(y) + η T -s + |t -s| 2 2ν ≤ 3C T |x ′ -y ′ | ≤ 2ν C T + δL 2 0 4 + 2νL 0 |x d -y d | ≤ 2δ C T + νL 2 0 4 + 2δL 0 .
This result is standard but since we need precise constants in the estimation, we postponed the proof.

Step 2. When the doubled normal variable converges to a single variable

Step 1 shows that up to extract a subsequence, we have for B ρα = B ρα (0) and for ν small enough,

( Xδ , Ȳδ ) → ( X, Ȳ ) ∈ [τ η , T -τ η ] × B ρα 2 as δ → 0 where ρ α := 2 6C T α , (4.5)
with X = ( t, x′ , xd ) = ( ξ, xd ), Ȳ = (s, ȳ′ , ȳd ) = ( ζ, ȳd ), ȳd = xd , and

αg(ȳ) + η T - s + | t -s| 2 2ν ≤ 3C T and |x ′ -ȳ′ | ≤ 2νC T + 2νL 0 = o ν (1) → 0 as ν → 0
where the last bound follows from estimate of Lemma 4.1. Moreover, we have

0 < M/2 ≤ M ν,α,δ → M ν,α := sup X,Y ∈[0,T )×Ω Ψ ν,α (X, Y ) = Ψ ν,α ( X, Ȳ ) as δ → 0 with Ψ ν,α (t, x, s, y) :=    ũ(t, x ′ , x d ) -v(s, y ′ , x d ) -αg(y) - η T -s - |(t, x ′ ) -(s, y ′ )| 2 2ν if x d = y d -∞ if x d = y d .
From the fact that M ν,α → M ν,0 as α → 0 (with obvious definitions), we deduce that all maximizer in the definition of M ν,α satisfies lim α→0 αg(ȳ) = 0. (

Moreover we have lim

ν→0 lim α→0 M ν,α = M = lim θ→0 M (θ)
where M (θ) is defined in (4.1). This also implies that

lim ν→0 lim α→0 |( t, x′ ) -(s, ȳ′ )| 2 ν = 0. (4.7)
We now prove that X ∈ {ũ > u 0 }. Assume by contradiction that ũ( X) = u 0 ( X). Then

Ψ ν,α ( X, Ȳ ) ≤ u 0 ( X) -v( Ȳ ) - |x ′ -ȳ′ | 2 2ν
which implies (using the a priori bound v ≥ u 0 )

0 < M/4 ≤ M ν,α = Ψ ν,α ( X, Ȳ ) ≤ u 0 (x) -u 0 (ȳ) - |x ′ -ȳ′ | 2 2ν ≤ L 0 |x ′ -ȳ′ | - |x ′ -ȳ′ | 2 2ν . (4.8)
This leads to a contradiction as ν → 0. Hence

X ∈ {ũ > u 0 } . (4.9)
This implies also that for δ small enough Xδ ∈ {ũ > u 0 } .

Step 3: proof that xd = 0 By contradiction, we assume that we are in the standard case xd > 0. Then we also have (x δ ) d , (ȳ δ ) d > 0 and the viscosity inequalities with pδ :=

x′ δ -ȳ ′ δ ν , (x δ ) d -(ȳ δ ) d δ    tδ -s δ ν + H( Xδ , pδ ) ≤ 0 because Xδ ∈ {ũ > u 0 } -η (T -s δ ) 2 + tδ -s δ ν + H( Ȳδ , -αDg(ȳ δ ) + pδ ) ≥ 0. (4.10)
We know from Lemma 4.1, that

x′ δ -ȳ′ δ ν ≤ ν -1 2ν C T + δL 2 0 4 + 2νL 0 and H( Xδ , pδ ) ≤ - tδ -sδ ν ≤ 6C T ν .
Moreover, the uniform coercivity of H (see (1.4) iv)) implies the existence of some Lν > 0 (independent on δ, for δ > 0 small enough, and independent on α) such that tδsδ ν , pδ ≤ Lν .

We can then subtract the two viscosity inequalities in (4.10), and get

η (T -sδ ) 2 ≤ H( Ȳδ , -αDg(ȳ δ ) + pδ ) -H( Xδ , pδ ).
Passing to the limit δ → 0, we get (up to extraction of a subsequence) that tδ -s δ ν , pδ → t-s ν , p with

p = x′ -ȳ′ ν , pd ∈ R d-1 × R and t - s ν , p ∈ D1,+ t,x u( X) with t - s ν , p ≤ Lν and η (T -s) 2 ≤ H( Ȳ , -αDg(ȳ) + p) -H( X, p) with ȳd = xd . (4.11)
Using assumptions (1.4) ii) and iii), this implies that (say with L := 2 Lν and α small enough)

η T 2 ≤ ω L (αDg(ȳ))+ ω(| X -Ȳ |(1 + |p ′ |+ max(0, H( X, p)) ≤ ω L (αDg(ȳ))+ ω(| X -Ȳ |(1 + |x ′ -ȳ′ | ν + | t -s| ν )).
Using the fact that αDg(ȳ) → 0 as α → 0 and estimate (4.7), we get a contradiction for α and ν small enough.

Step 4: The key one-sided Lipschitz estimate In the remaining of the proof we then have xd = 0. For ξ = (t, x ′ ), ζ = (s, y ′ ), and

x d ∈ [0, +∞), we set Ψ α ν (ξ, ζ, x d ) := Ψ ν,α (ξ, x d , ζ, x d ) = ũ(ξ, x d ) -v(ζ, x d ) - η T -s -αg(y ′ , x d ) - |ξ -ζ| 2 2ν and consider 0 < M/2 ≤ M ν,α = sup ξ,ζ∈([0,T )×R d-1 ) 2 , x d ∈[0,+∞) Ψ α ν (ξ, ζ, x d ) = Ψ α ν ( ξ, ζ, xd ). (4.12)
We define

V (s, y) := v(s, y) + η T -s + αg(y)
so that we have

                   ∂ t ũ + H(X, D ũ) ≤ 0 in ((0, T ) × Ω) ∩ {ũ > u 0 } ∂ t ũ + min {F, H} (X, D ũ) ≤ 0 on ((0, T ) × ∂Ω) ∩ {ũ > u 0 } -η (T -s) 2 + ∂ s V + H(Y, DV -αDg) ≥ 0 in (0, T ) × Ω -η (T -s) 2 + ∂ s V + max {F, H} (Y, DV -αDg) ≥ 0 on (0, T ) × ∂Ω.
We now claim the following one-sided "Lipschitz" estimate ũ(ξ,

x d ) -V (ζ, y d ) ≤ ũ( ξ, xd ) -V ( ζ, xd ) + |ξ -ζ| 2 2ν - | ξ -ζ| 2 2ν + L ν |x d -y d | (4.13)
where L ν is given in Lemma 3.2, and where equality holds for t = t, s = s, x ′ = x′ , y ′ = ȳ′ and x d = y d = xd , with X = ( ξ, xd ), Ȳ = ( ζ, xd ). For clarity, the proof of (4.13) is postponed at the end of the proof of the theorem.

Step 5: the twin blow-ups.

We then consider the following twin blow-ups with small parameter ε > 0: one blow-up for ũ at the point X = ( ξ, xd ) and one blow-up for V at the point Ȳ = ( ζ, xd ),

   U ε ( X) := ε -1 ũ( X + ε X) -ũ( X) , U ε (0) = 0, V ε ( Ŷ ) := ε -1 V ( Ȳ + ε Ŷ ) -V ( Ȳ ) , V ε (0) = 0. (4.14)
Before passing to the limit ε → 0, they satisfy for X = ( t, x) and Ŷ = (ŝ, ŷ) From (4.13), they also satisfy

           ∂ tU ε + H( X + ε X, D xU ε ) ≤ 0 in I ε t × Ω ∂ tU ε + min(F, H)( X + ε X, D xU ε ) ≤ 0 on I ε t × ∂Ω -η ε + ∂ ŝV ε + H( Ȳ + ε Ŷ , D ŷV ε -αD y g(ȳ + εŷ)) ≥ 0 in I ε s × Ω -η ε + ∂ ŝV ε + max(F, H)( Ȳ + ε Ŷ , D ŷV ε -αD y g(ȳ + εŷ)) ≥ 0 on I ε s × ∂Ω
U ε ( ξ, xd ) -V ε ( ζ, ŷd ) ≤ L ν |x d -ŷd | + b • ( ξ -ζ) + ε | ξ -ζ| 2 2ν with b := ξ - ζ ν . (4.16)
We then define the following half-relaxed limits

   U 0 := lim sup ε→0 * U ε , U 0 (0) ≥ 0, V 0 := lim inf ε→0 * V ε , V 0 (0) ≤ 0.
Passing to the limit in (4.16), we get

U 0 ( ξ, xd ) -V 0 ( ζ, ŷd ) ≤ L ν |x d -ŷd | + b • ( ξ -ζ), (4.17)
which implies in particular that U 0 (0) = 0 = V 0 (0). Passing to the limit in (4.15) and using the discontinuous stability of viscosity solutions, we also get

           ∂ tU 0 + H( X, D xU 0 ) ≤ 0 in (R × Ω) ∩ |U 0 | < +∞ ∂ tU 0 + min(F, H)( X, D xU 0 ) ≤ 0 on (R × ∂Ω) ∩ |U 0 | < +∞ -η + ∂ ŝV 0 + H( Ȳ , D ŷ V 0 -αD y g(ȳ)) ≥ 0 in (R × Ω) ∩ |V 0 | < +∞ -η + ∂ ŝV 0 + max(F, H)( Ȳ , D ŷ V 0 -αD y g(ȳ)) ≥ 0 on (R × ∂Ω) ∩ |V 0 | < +∞ (4.18) with η := η (T -s) 2 .
Step 6: the 1D problem. We now define the following functions on [0, +∞) as the supremum/infimum in the tangential variables of the functions defined in R × Ω, u(x d ) := sup

ξ∈R d U 0 ( ξ, xd ) -b • ξ , v(ŷ d ) := inf ζ∈R d V 0 ( ζ, ŷd ) -b • ζ .
From (4.17), these functions satisfy

-∞ ≤ -L ν |x d -ŷd | + u(x d ) ≤ v(ŷ d ) ≤ +∞, 0 ≤ u(0) ≤ v(0) ≤ 0.
In particular, this implies that u(0) = 0 = v(0). Because of this one-sided Lipschitz inequality, this is also the case for their semi-continuous envelopes, i.e. we have (and this is important)

-∞ ≤ -L ν |x d -ŷd | + u * (x d ) ≤ v * (ŷ d ) ≤ +∞, u * (0) = 0 = v * (0). (4.19)
We set H α (Y, p) = H(Y, p -αDg(ȳ)) and F α (Y, p) = F (Y, p -αDg(ȳ)). From (4.18), we get (again from stability) that these functions satisfy in particular for X := ( t, x), Ȳ := (s, x) and b = ( b0

, b′ ) ∈ R × R d-1            b0 + H( X, b′ , ∂ xd u * ) ≤ 0 in (0, +∞) ∩ {|u * | < +∞} b0 + min(F, H)( X, b′ , ∂ xd u * ) ≤ 0 in {0} ∩ {|u * | < +∞} -η + b0 + H α ( Ȳ , b′ , ∂ ŷd v * ) ≥ 0 in (0, +∞) ∩ {|v * | < +∞} -η + b0 + max(F α , H α )( Ȳ , b′ , ∂ ŷd v * ) ≥ 0 in {0} ∩ {|v * | < +∞} .
(4.20)

Step 7: getting a contradiction from structural assumptions. We now apply Corollary 2.5. In order to do so, we now set z = xd = ŷd and consider and we get that there exists

p d ∈ [a d , b d ] ∩ R = ∅ such that either b0 + H( X, b′ , p d ) ≤ 0 < η ≤ H α ( Ȳ , b′ , p d ) -H( X, b′ , p d ) or b0 + max(F, H)( X, b′ , p d ) ≤ 0 < η ≤ F α ( Ȳ , b′ , p d ) -F ( X, b′ , p d ).
One of these facts are true along a subsequence ν → 0. In the first case, we get from the assumption on the Hamiltonian H, see (1.4) ii), that (using p′ = b′ ) and again L := 2 Lν for α small enough, From the assumption on the function F , see (1.5) ii), we get a similar contradiction in the second case,

η ≤ H α ( Ȳ , b′ , p d ) -H( X, b′ , p d ) ≤ ω | X -Ȳ | • 1 + | b′ | + max 0, H( X, b′ , p d ) + ω L (αDg(ȳ)) ≤ ω | ξ -ζ| • 1 + | b′ | + max 0, -b0 + ω L (αDg(ȳ)) ≤ ω 2 | ξ -ζ| 2 ν + | ξ -ζ| + ω L (αDg (
η ≤ F α ( Ȳ , b′ , p d ) -F ( X, b′ , p d ) ≤ ω | ξ -ζ| • 1 + | b′ | + max 0, max {F, H} ( X, b′ , p d ) + ω L (αDg(ȳ)) ≤ ω | ξ -ζ| • 1 + | b′ | + max 0, -b0 + ω L (αDg(ȳ)) ≤ ω 2 | ξ -ζ| 2 ν + | ξ -ζ| + ω L (αDg (
ȳ)) → 0 as α → 0, and then ν → 0.

We conclude that M ≤ 0. Recalling that

M = sup t∈[0,T ),x∈Ω ũ(t, x) -v(t, x) - η T -t ≤ 0,
it is enough to let η → 0 to get u ≤ ũ ≤ v as desired.

Back to

Step 3: proof of the key one-sided Lipschitz estimate (4.13) We now justify (4.13). Following Lemma 3.2, we extend ũ and consider

Ũ ν (ζ, x d ) := sup ξ∈R×R d-1 ũ(ξ, x d ) - |ζ -ξ| 2 2ν
and there exists some (possibly non unique) ξζ

∈ [s-θ ν , s+θ ν ]×R d-1 such that Ũ ν (ζ, x d ) = ũ( ξζ , x d )- | ξζ -ζ| 2 2ν . If s ∈ [θ ν , T -θ ν ], then we see that ξζ ∈ (0, T ) × R d-1 and we also have Ũ ν (ζ, x d ) := sup ξ∈[0,T )×R d-1 ũ(ξ, x d ) - |ξ -ζ| 2 2ν .
In particular for (ζ, x d ) = ( ζ, xd ), we can choose ξζ = ξ where ξζ is given by Lemma 3.2 and X = ( ξ, xd ), Ȳ = ( ζ, xd ) appear in (4.5). Now we choose ν > 0 small enough such that θ ν < τ η , and we set

I ν := (θ ν , T -θ ν ). Moreover we have for all ζ ∈ I ν × R d-1 , y d ∈ [0, +∞), Ũ ν (ζ, y d ) -V (ζ, y d ) ≤ sup ξ∈[0,T )×R d-1 , x d ∈[0,+∞) Ψ ν,α (ξ, ζ, y d ) = Ψ ν,α ( ξ, ζ, xd ) = Ũ ν ( ζ, xd ) -V ( ζ, xd ).
Now from Lemma 3.2, we also know that Ũ ν is L ν -Lipschitz, and then Ũ

ν (ζ, x d ) -Ũ ν (ζ, y d ) ≤ L ν |x d -y d |, which implies Ũ ν (ζ, x d ) -V (ζ, y d ) ≤ Ũ ν ( ζ, xd ) -V ( ζ, xd ) + L ν |x d -y d |
which gives exactly (4.13). This ends the proof of the theorem.

We now turn to the proof of Lemma 4.1.

Proof of Lemma 4.1. Recall that we have

ũ(t, x) ≤ u 0 (x) + C T , v(t, x) ≥ u 0 (x) -C T . Hence Ψ ν,α,δ (t, x, s, y) ≤ 2C T + B δ (x, y) -αg(y) - η T -s - |t -s| 2 2ν with B δ (x, y) := u 0 (x) -u 0 (y) - |x d -y d | 2 2δ - |x ′ -y ′ | 2 2ν ≤ φ ν (|x ′ -y ′ |) + φ δ (|x d -y d |)
and

φ δ (r) := L 0 r - r 2 2δ ≤ δL 2 0 2 .
Here φ δ is concave with φ δ (r δ ) = 0 for r δ := 2δL 0 . Moreover

φ δ (r) ≤ (r -r δ )φ ′ δ (r δ ) = (r -r δ )(L 0 - r δ ) i.e. φ δ (r) ≤ -δ -1 (r -r δ ) 2 for r ≥ r δ = 2δL 0 . (4.21)
We get in particular 0

< Ψ ν,α,δ (X, Y ) ≤ 2C T + φ δ (|x d -y d |) + φ ν (|x ′ -y ′ |) and then 0 < Ψ ν,α,δ (X, Y ) ≤ 2C T + φ δ (|x d -y d |) + νL 2 0 2 which implies from (4.21) that |x d -y d | ≤ 2δ C T + νL 2 0 4 + 2δL 0 (4.22)
and symmetrically that

|x ′ -y ′ | ≤ 2ν C T + δL 2 0 4 + 2νL 0 . ( 4

.23)

We also deduce from 0 < Ψ ν,α,δ (X, Y ) that

αg(y) + η T -s + |t -s| 2 2ν ≤ 2C T + (ν + δ)L 2 0 2 ≤ 2C T + η T ≤ 3C T (4.24)
for η > 0 small enough (the size of η depending on C T and T , but not on ν, α, δ), and for δ, ν > 0 small enough (for a size depending on η). Therefore we have

|t k -s k | ≤ θν := 3 νC T (4.25) and T -s k > η 3C T , T -t k > η 3C T -θν ≥ η 4C T
for ν > 0 small enough (for a size depending on η).

Similarly, from Lemma 3.1 on the barriers (in particular using Case 1 of the proof, for Lipschitz initial data u 0 ), we know that there exists some λ > 0 such that u(t, x) ≤ u 0 (x) + λt, v(s, y) ≥ u 0 (y)λs.

Hence

Ψ ν,α,δ (P k ) ≤ λ(t + s) + L 0 |x -y| -η T ≤ λ(t k + s k ) -2η 3T where we have used bound (4.22)-(4.23) for δ, ν > 0 small enough (for a size depending on η > 0). Therefore max(t, s) > η 3λT

, min(t, s) > η 3λT

θν ≥ η 4λT for ν > 0 small enough (for a size depending on η). Up to increase λ or C T (and decrease ν > 0 if necessary), we can assume that λT ≡ C T .

Setting τ η := η 4λT = η 4C T and for ν > 0 small enough, we see that

X, Y ∈ [τ η , T -τ η ] × Ω.
This gives the result with (4.22), (4.23) and (4.24). This ends the proof of the lemma. (5.1)

The comparison principle on a bounded domain

Let T > 0. We consider the following equation for u(t, x) with X := (t, x) ∈ [0, T ) × Ω u t + H(X, Du) = 0 on (0, T ) × Ω (5.2)

and the boundary condition u t + F (X, Du) = 0 on (0, T ) × ∂Ω.

We also consider an initial boundary condition u(0, •) = u 0 on {0} × Ω.

(5.

3)

The rigorous meaning of desired boundary conditions is the following, u t + min(F, H)(X, Du) ≤ 0 on (0, T ) × ∂Ω (for subsolutions) u t + max(F, H)(X, Du) ≥ 0 on (0, T ) × ∂Ω (for supersolutions) (5.4) Lemma 5.2 (Action of a diffeomorphism on the structural conditions satisfied by H, F ). Assume that Ω has the regularity given in assumption (5.1). For T > 0, we set Q T := (0, T ) × Ω with Ω ⊂ R d and for P = (p 0 , p) ∈ R × R d , we set H 0 (Y, P ) := p 0 + H(Y, p), F 0 (Y, P ) := p 0 + F (Y, p).

We say by extension that H 0 , F 0 satisfy respectively (5.5) and (5.6), if H, F do it.

Assume that H 0 , F 0 satisfy respectively (5.5) and (5.6) and for x 0 ∈ ∂Ω, consider, locally around x 0 , a C 1 -diffeomorphism Φ from Ω to Ω with Φ(x 0 ) = y 0 ∈ ∂ Ω, that we extend by the identity on the time variable. Still denoting by Φ this diffeomorphism, we assume that Φ maps locally Q T to QT ⊂ R Then H0 , F0 satisfy respectively (5.5) and (5.6) locally around [0, T ] × {y 0 }, with some suitable moduli.

We now turn to the proof of Theorem 5.1.

Proof of Theorem 5.1. Up to proceed as in Step 0 of the proof of Theorem 1.1, we can assume that u 0 belongs to C 1 (Ω). We set ũ := max {u, u 0 } , u 0 (t, X) = u 0 (X) -C T = u 0 (X) and M := sup (t,x)∈[0,T )×Ω Ψ(t, x) with Ψ(t, X) = ũ(t, x)v(t, x) -η Tt .

Assume by contradiction that 0 < M = Ψ(X 0 ) with X 0 := (t 0 , x 0 ) ∈ [0, T ) × Ω.

By assumption, we have t 0 > 0. If x 0 ∈ Ω, then we can localize, and then get a contradiction by standard method of doubling of variables. Hence assume that x 0 ∈ ∂Ω. Up to modify slithly the functions, we can assume that the suppremum is strict at X 0 . Up to change the coordinates, we can also assume that x 0 = 0, Ω ∩ B r (x 0 ) = x = (x ′ , x d ) ∈ R d-1 × R, x d > h(x ′ ) ∩ B r (x 0 ), h(0) = 0 = Dh(0).

Setting

x = Φ(y) with y = (y ′ , y d ) = Φ -1 (x) := (x ′ , x d -h(x ′ )) and Ũ (t, y) := ũ(t, Φ(y)), V (t, y) := v(t, Φ(y))

we see that Φ is locally invertible and its inverse is a C 1 map, given, for some ρ > 0 small enough, by

Φ : K + ρ → Ω ∩ B r (x 0 ) with K + ρ := [-ρ, ρ] d-1 × [0, ρ].

  |p d |→+∞ inf{H(X, p ′ , p d ) : X ∈ [0, T ] × Ω, p ′ ∈ [-L, L] d-1 } = +∞.

  (s + εŝ)) 2 and I ε r := -

p

  d := lim sup [0,+∞)∋z→0 u * (z) z , p d := lim inf [0,+∞)∋z→0 v * (z) z , a d := min(p d , p d ), b d := max(p d , p d )

  ȳ)) → 0 as α → 0, and then ν → 0 where we have used the expression of b = ξζ ν in the third line, and (4.7) in the last line. Contradiction because η ≥ η/T 2 > 0.

  Let us consider an open set Ω satisfying for, d ≥ 1, Ω ⊂ R d is a bounded open set with C 1 boundary and outward unit normal n(x).

  1+d with locally Φ(∂Q T ) = ∂ QT . For Y ∈ QT and P ∈ R 1+d , we setH0 (Y, P ) := H 0 (Φ -1 (Y ), P • B(Y )) locally around [0, T ] × {y 0 }, on QT F0 (Y, P ) := F 0 (Φ -1 (Y ), P • B(Y ))locally around [0, T ] × {y 0 }, on ∂ QT with(P • B) j = d i=0 P i (D j Φ i ) • Φ -1for j = 0, . . . , d.

D

  Hence we havex = (x ′ , x d ) = (y ′ , y d + h(y ′ )) = Φ(y), ũ(t, x) = Ũ (t, Φ -1 (x)), v(t, x) = V (t, Φ -1 (x)x d ũ(t, x) = D y d Ũ (t, Φ -1 (x)), D x ′ ũ(t, x) = D y ′ Ũ (t, Φ -1 (x)) -D y d Ũ (t, Φ -1 (x)) • D x ′ h(x ′ ), ũt (t, x)= Ũt (t, Φ -1 (x)).

As far as Hamiltonians are concerned, we assume the following structure conditions, where ω, ω L are moduli of continuity.

and as previously, making artificially appear the dependence on x ∈ Ω for F (in order to unify the presentation of H and F ), we consider for X = (t, x)

iii) (Continuity in the tangential variables)

(5.6) We then have the following theorem.

Theorem 5.1 (Comparison principle on a bounded open set Ω). Let T > 0 and assume that H, F satisfy respectively (5.5) and (5.6). Assume that the initial data u 0 is continuous. Let u, v : [0, T ) × Ω → R be two functions with u upper semi-continuous and v lower semi-continuous. Assume that u (resp. v) is a viscosity subsolution (resp. supersolution) of (5.2)-(5.3). Assume moreover that there exists a constant

In order to give the proof of Theorem 5.1, we need the following lemma which proof is left to the reader.

This gives the new Hamiltonian H and boundary function F for Y = (t, y), X = (t, x) and x = Φ(y)

which are defined by (for y := (y ′ , y d ))

Hence Ũ and V are respectively sub/supersolutions of

We now apply Lemma 5.2 to insure that H and F satisfy (locally) the same structural conditions than H and F . Moreover, we have

Up to add some small and smooth tangential correction term |tt 0 | 2 + |y ′ | 2 to V (here we neglect this correction which can be treated in a very classical way), we can assume that We are then back to the begining of the proof of Theorem 1.1, which leads to a contradiction. Again, we conclude that M ≤ 0 for all η → 0 + , and then deduce that Ũ ≤ V , and then u ≤ v. This ends the proof of the theorem.


Madani Bezoui
email: mbezoui@cesi.fr

Saeed Mian Qaisar
email: smianqaisar@cesi.fr

Abdulaziz Turki Almaktoom
email: abalmaktoom@effatuniversity.edu.sa

Deep Reinforcement Learning for multiobjective Scheduling in Industry 5.0 Reconfigurable Manufacturing Systems ⋆

Keywords: Reconfigurable Manufacturing Systems, Sustainability, Deep Reinforcement Learning, Multiobjective Scheduling, Industry 5.0

In modern-day manufacturing, it is imperative to react promptly to altering market requirements. Reconfigurable Manufacturing Systems (RMS) are a significant leap forward in achieving this criteria as they offer a flexible and affordable structure to comply with evolving production necessities. The ever-changing nature of RMS demands a sturdy induction of learning algorithms to persistently improve system configurations and scheduling. This study suggests that using Reinforcement Learning (RL), specifically, the Double Deep Q-Network (DDQN) algorithm, is a feasible way to navigate the intricate, multi-objective optimization landscape of RMS. Key points to consider regarding this study include cutting down tardiness costs, ensuring sustainability by reducing wasted liquid and gas emissions during production, optimizing makespan, and improving ergonomics by reducing operator intervention during system reconfiguration. Our proposal consists of two layers. Initially, we suggest a hierarchical and modular architecture for RMS which includes a multi-agent environment at the reconfigurable machine tool level, which improves agent interaction for optimal global results. Secondly, we incorporate DDQN to navigate the multi-objective space in a clever manner, resulting in more efficient and ergonomic reconfiguration and scheduling. The findings indicate that employing RL can help solve intricate optimization issues that come with contemporary manufacturing paradigms, clearing the path for Industry 5.0.

Introduction

In an era of rapid technological evolution and fluctuating market dynamics, industries, particularly the manufacturing sector, are in a constant search for innovative solutions to meet the escalating demand for product customisation and to skilfully navigate the unpredictability of market trends [START_REF] Rajkumar | Recent advances in evolutionary and adaptable manufacturing systems[END_REF]. In the face of these challenges, Reconfigurable Manufacturing Systems (RMS), a concept introduced by Koren et al. [START_REF] Koren | Reconfigurable manufacturing systems[END_REF], has emerged as a seminal solution. RMS represents a pivotal evolution in manufacturing paradigms, seeking to merge harmoniously the high-throughput features of Dedicated Manufacturing Lines (DML) -characterised by the use of Dedicated Machines (DM) -with the adaptability and responsiveness of Flexible Manufacturing Systems (FMS), characterised by the use of Computer Numerical Control (CNC) machines. This transformation from DML to FMS and finally to RMS represents a profound shift in manufacturing philosophy, where adaptability, modularity and rapid reconfiguration are essential to meet the dynamic demands of today's industrial landscapes [START_REF] Koren | Reconfigurable manufacturing systems[END_REF].

The core challenges associated with RMS are orchestrated around three key stages: design, implementation and optimisation. The design stage provides the fundamental blueprint that defines the architecture of the system, which in turn significantly influences the subsequent implementation and optimisation stages. Crucial decisions regarding system components and their potential reconfigurations are made during this phase [START_REF] Bilberg | New model for development and manufacturing of tailored solutions in the industrial market[END_REF]. The design phase is followed by the implementation phase, which translates theoretical designs into tangible, operational systems. This transition is fraught with intricacies, encapsulating the integration of diverse system components, ensuring operational coherence, and accommodating practical considerations [START_REF] Li | Design and implementation of a reconfigurable manufacturing system[END_REF]. Ultimately, the optimisation stage continually refines system operations to increase efficiency and productivity while reducing costs and waste. This stage employs a range of optimisation tools and strategies such as scheduling, machine allocation and production planning, and also requires the consideration of real-time data and prospective reconfigurations [START_REF] Dashchenko | Reconfigurable manufacturing systems and transformable factories[END_REF].

In recent years, the infusion of Artificial Intelligence (AI) and learning methods, particularly Reinforcement Learning (RL), has gained considerable traction in solving complex puzzles in various domains. These technological advancements promise to improve decision making, system adaptation and optimisation, thus addressing some of the critical challenges associated with RMS implementation.

Literature Review

The advent of Reconfigurable Manufacturing Systems (RMS) marked a significant milestone towards addressing the dynamic demands of modern manufacturing landscapes. The application of various optimization techniques within the RMS domain has garnered substantial attention, owing to the potential to address complex scheduling and reconfiguration problems.

Several studies have underscored the application of mathematical programming approaches like mixed-integer linear programming (MILP) in optimizing scheduling and reconfiguration in RMS settings. For instance, Aljuneidi and Bulgak [START_REF] Aljuneidi | Designing a cellular manufacturing system featuring remanufacturing, recycling, and disposal options: A mathematical modeling approach[END_REF] proposed a MILP model for designing reconfigurable cellular manufacturing systems with hybrid manufacturing-remanufacturing capabilities. The model incorporated constraints related to machine capacities, inventory levels, and processing requirements. Khezri et al. [START_REF] Khezri | Towards a sustainable reconfigurable manufacturing system (srms): Multi-objective based approaches for process plan generation problem[END_REF] developed a multi-objective MILP model to generate sustainable process plans in a reconfigurable context. The model incorporated objectives related to minimizing production cost, time as well as environmental impacts measured through liquid waste generation and greenhouse gas emissions. The model was solved using an augmented epsilonconstraint method. In another study, Khezri et al. [START_REF] Khezri | Diagnosis on energy and sustainability of reconfigurable manufacturing system (rms) design: A bi-level decomposition approach[END_REF] proposed a bi-level decomposition approach involving two MINLP models to integrate diagnosability and sustainability considerations in RMS design. The upper-level model focused on preventive maintenance planning to minimize hazardous energy consumption, while the lower-level model concerned sustainable process plan generation by minimizing energy losses. Several studies have also adapted metaheuristic techniques like genetic algorithms, particle swarm optimization, ant colony optimization to effectively solve optimization problems related to scheduling and reconfiguration in RMS settings under complexity constraints.Musharavati and Hamouda [START_REF] Musharavati | Simulated annealing with auxiliary knowledge for process planning optimization in reconfigurable manufacturing[END_REF] developed a simulated annealing algorithm for optimizing process plans in RMS environments. The proposed approach outperformed a standard SA implementation in the comparative analysis. Mohapatra et al. [START_REF] Mohapatra | Realising process planning and scheduling integration through adaptive setup planning[END_REF] employed an adapted NSGA-II algorithm for integrating process planning and scheduling decisions in an RMS through adaptive setup planning. The objectives of minimizing completion time and cost were simultaneously optimized through the NSGA-II implementation.

The integration of artificial intelligence techniques like reinforcement learning, neural networks and fuzzy logic has also garnered attention for automating and enhancing RMS scheduling and reconfiguration capabilities. For instance, Yang and Xu [START_REF] Yang | Intelligent scheduling and reconfiguration via deep reinforcement learning in smart manufacturing[END_REF] proposed a multi-agent deep reinforcement learning architecture for optimized scheduling and reconfiguration in smart RMS environments. The model design demonstrated the versatility of deep reinforcement learning for automated decision-making in complex RMS settings. Tang and Salonitis [START_REF] Tang | Reconfigurable manufacturing system scheduling: a deep reinforcement learning approach[END_REF] explored a deep reinforcement learning-based scheduling policy tailored for reconfigurable manufacturing systems, elucidating the role of deep reinforcement learning in devising adaptive scheduling policies. Zhou et al. [START_REF] Zhou | Deep reinforcement learningbased dynamic scheduling in smart manufacturing[END_REF] presented a dynamic scheduling approach based on deep reinforcement learning to address scheduling challenges in smart manufacturing contexts enabled by RMS.

In summary, mathematical programming, metaheuristics and artificial intelligence have been widely leveraged to tackle scheduling and reconfiguration challenges in RMS environments, with the techniques offering complementary strengths. While mathematical models support optimization under precisely de-fined constraints, metaheuristics and AI facilitate solving highly complex problems through exploration of large search spaces. Further hybridization of these techniques can pave the way for more holistic and powerful RMS optimization capabilities.

Problem Description

The main focus of this work is to explore the complexity of scheduling and reconfiguration in the field of reconfigurable manufacturing systems, especially in the emerging framework of Industry 5.0. The problem is multi-objective in nature, involving the simultaneous optimisation of objectives such as minimising tardiness costs, reducing reconfiguration operator interventions, optimising makespan, and improving system ergonomics. This section unfolds the core aspects of the proposed modelling of the RMS architecture, the underlying multiobjective problem and the contextual relevance of Industry 5.0, providing a solid foundation for the proposed methodology and its subsequent implementation and experimentation.

Reconfigurable Manufacturing System (RMS)

The key components of RMS include reconfigurable machine tools (RMTs), modular equipment and intelligent control systems. These elements work in a complementary way to facilitate rapid reconfiguration, rescheduling and adaptation to new manufacturing tasks. The modular design of RMT enables a wide range of configurations to meet different product specifications and production volumes. Meanwhile, the intelligent control systems use advanced algorithms and real-time data to orchestrate the seamless transition between different configurations and schedules.The proposed hierarchical environment for RMS architecture consists of a top-level RMS environment that includes several lower-level RMT sub-environments. Each of these RMT sub-environments is as a multi-agent environment.

Fig. 1: RMS Modelling with RMT machines with differents configurations.

Reconfigurable Machine Tool (RMT)

Reconfigurable Machine Tool (RMT) accepts multiple configurations, each configuration is represented by a virtual buffer in this model. A configuration refers to a specific arrangement of the machine's modules to obtain certain capabilities. RMT is controlled by two cooperative but independent agents inspired from the work of [START_REF] Yang | Intelligent scheduling and reconfiguration via deep reinforcement learning in smart manufacturing[END_REF]:

✓ The Reconfiguration Agent (RCA) handles dynamic Reconfiguration of the machine's modules among a predefined set of configurations to adapt its capabilities. ✓ The Scheduling Agent (SA) is responsible for optimizing the scheduling and sequencing of jobs on the machine in its current configuration.

By jointly learning coordinated policies, the RCA and SA agents can intelligently RCAigure the machine's modules and adjust the production SAule to optimize objectives like minimizing job tardiness as conditions change.

Reconfiguration agent RCA

The objective is to select reconfiguration actions that minimize any rise in total tardiness costs of jobs during the Reconfiguration interval. Let t S and t ′ S be the start and end times of a reconfiguration.

The reward R t for a reconfiguration is:

R RCA = - 1 t S ′ -t S (T C W J + T C CJ ) (1)
T

C W J = X k=1 n k j=1 α j z j S ′ [t S ′ -max (t S , d j )] z j S ′ = 1, if d j < t S ′ 0, else T C CJ = n F N S j=1 α j z jC [C j -max (t S , d j )] z jC = 1, if d j < C j 0, else.
Where T C W J and T C CJ are the newly added tardiness costs of waiting and completed jobs. This rewards actions that keep the increase in total tardiness low.

Reconfiguration Trigger Reconfiguration is triggered when:

✓ The current buffer is empty ✓ Overdue jobs exist and the current buffer has relatively low average tardiness ✓ The number of finished jobs exceeds a threshold and average tardiness is low This balances reconfiguration and scheduling optimization. The state space includes statistical measures on these features. States are normalized using min -max scaling.

Scheduling agent SA

The goal of SA is selecting the next job from the current buffer (selected RMT configuration) to execute.

Reward Function It is inversely related to the change in total tardiness per second:

R SA = - 1 t s ′ -t s (T C W F ) (2)
Where T C W F is the newly added tardiness cost of waiting jobs in [t s , t s ′ ]. This is calculated as:

T C W F = n j=1 y js ′ β j [t s ′ -max(t s , q j )] (3)
Where y js ′ indicates if job j is overdue at t s ′ . Rewarding lower incremental tardiness aims to have SA learn optimized job sequencing.

Action Space: The SA has five actions for selecting the next job j from the current buffer:

✓ Action 1 (act1): Choose job with maximum tardiness cost ψ j . ✓ Action 2 (act2): Choose job with maximum unit tardiness cost β j . ✓ Action 3 (act3): Choose job with minimum safe time ST j . ✓ Action 4 (act4): Choose job with nearest due date q j . ✓ Action 5 (act5): Choose job with minimum processing time

i t ij .
The diverse actions enable SA to learn nuanced scheduling policies.

State Features

RMSA Agent

The RMSA supervises scheduling and reconfiguration decisions across multiple Reconfigurable Machine Tool (RMT) environments. This decentralised structure enables independent learning while reducing dimensionality. At each timestep, the RMSA assigns the next pending job to an RMT based on aggregated state data. Once initial jobs are assigned, the RMSA steps the RMTs to simulate system operation. This allows system oversight with machine autonomy.

Multiobjective scheduling and reconfiguration Optimization: The heart of the scheduling and reconfiguration problem in RMS is to determine the optimal sequence of operations on various machines, along with the optimal configuration path of machines and production lines to meet specified manufacturing objectives. The multiobjective nature of the problem requires a careful optimisation approach to find a Pareto-optimal solution that represents a balanced trade-off between the conflicting objectives. Each objective represents a critical dimension of operational performance and cost efficiency. For example, minimising the tardiness cost is critical to maintaining delivery scheduling and customer satisfaction, while reducing number of reconfiguration operator interventions is critical to improving system responsiveness to market changes. Optimising makespan has a direct impact on throughput and inventory levels, and improving system ergonomics is critical to ensuring operator safety and job satisfaction. Exploring these objectives, both individually and collectively, reveals a complex optimisation landscape with countless local optima and intricate constraints. Industry 5.0 Context: The advent of Industry 5.0 heralds a human-centred approach to manufacturing, with an emphasis on fostering collaboration between people and intelligent systems. Unlike previous industrial revolutions, Industry 5.0 seeks to harmonise the strengths of human creativity and machine precision. The problem described in this paper is directly aligned with the ethos of Industry 5.0, as it seeks to optimise the operational efficiency of RMS while promoting ergonomic considerations for human operators. The envisaged solution aims to harness the capabilities of intelligent algorithms to not only enhance manufacturing performance, but also to create a conducive and ergonomic working environment. The synergy between man and machine is seen as a linchpin for achieving sustainable and inclusive growth in the manufacturing sector.

Multiobjective Reward: The RMSA agent employs a weighted multiobjective reward function to optimize multiple performance goals:

R RMSA = w 1 F t -w 2 F s -w 3 F o (4)
Where: The tardiness cost objective F t aims to minimize job delays. The sustainability objective F s encourages energy-efficient operation with lower emissions. The operator intervention objective F o penalizes excessive reconfigurations. By combining these objectives with tunable weights, the multiobjective reward allows the RMSA agent to learn policies that balance productivity, efficiency, human effort and other critical goals. The weights enable prioritizing the different objectives as needed for the manufacturing application.

✓ F t =
Optimizing this composite reward function leads to Pareto optimal solutions that make trade-offs between the individual objective costs. This provides coordinated reconfiguration and scheduling control to maximize overall system performance across key dimensions.

Tardiness cost: The RMSA tardiness cost reward is the average of the RMT tardiness cost rewards at each timestep. This aligns the global objective of maximizing productivity with the RMT rewards to minimize tardiness.

F t = 1 N N i=1 R RM T i (5)
where:

R RM T i = R RCA i + R SA i 2 (6)
Where N is the number of RMTs.

Sustainability Objective

The sustainability objective F s in the RMSA reward function aims to minimize the environmental impact of the manufacturing system operations. It accounts for both liquid hazardous waste (LHW) and greenhouse gas (GHG) emissions associated with each job and machine configuration inspired from the work of [START_REF] Khezri | Towards a sustainable reconfigurable manufacturing system (srms): Multi-objective based approaches for process plan generation problem[END_REF].

Where the sustainability cost F S is:

F S = w LHW N i=1 ni j=1 (LHW ij -LHW min ) (LHW max -LHW min ) +w GHG N i=1 ni j=1 (GHG ij -GHG min ) (GHG max -GHG min ) (7) Where:
✓ N is the number of RMT environments ✓ n i is the number of finished jobs in RMT i ✓ LHW ij and GHG ij are the LHW and GHG emissions for job j in RMT i ✓ LHW min , LHW max , GHG min , GHG max are normalization constants ✓ w LHW , w GHG are objective weights Minimizing this sustainability cost encourages configurations and job assignments that reduce hazardous wastes and emissions. The weights allow tuning the relative importance of LHW versus GHG reduction. Normalization accounts for variability in emission levels across jobs and machines.

Operator Intervention Objective The operator intervention objective F o aims to minimize the total number of reconfigurations across all RMT environments. It penalizes frequent machine reconfiguration that requires human effort.

The operator intervention cost is:

F o = M i=1 N r,i (8)
Where M is the number of RMTs and N r,i is the number of reconfigurations performed in RMT i.

The total reconfigurations N r is calculated by summing the reconfiguration counts N r,i from each RMT environment i.

By penalizing the total reconfiguration count through this objective term, the RMSA agent is incentivized to learn policies that minimize operator effort required for machine reconfiguration across the manufacturing system. The weight w r controls the relative importance of this goal.

Action Space

The RMSA actions assign each pending job to an RMT selected based on aggregated states to optimize system performance. There are 5 actions corresponding to RMT with: ✓ minimum configuration buffer length ✓ maximum configuration buffer safe time ✓ minimum configuration buffer total tardiness ✓ minimum configuration buffer average tardiness RMT already running the configuration, or minimum length By considering factors like queue occupancy, safe time, and tardiness, the RMSA can improve system scheduling.

State Features

The RMSA state summarizes high-level features across RMTs, including normalized: ✓ Queue lengths per RMT ✓ Total tardiness per RMT ✓ Average tardiness per RMT ✓ Job safe times per RMT.

It tracks RMT metrics like queue occupancy and scheduling to inform system job assignments. The compact representation allows assessing global status for joint reconfiguration and scheduling.

Implementation and Experimentation

Implementation of DDQN

The DDQN algorithm, an extension of the conventional Deep Q-Networks (DQN) algorithm, is employed to tackle the multiobjective optimization problem at hand. DDQN mitigates the overestimation bias of Q-values, a known issue in standard Q-learning and DQN, by decoupling the selection and evaluation of actions.

The core components of the DDQN algorithm include:

✓ State Representation: The state of the system is represented using a set of features that encapsulate the current configuration and status of the RMS. ✓ Action Space: The action space comprises all possible reconfiguration and scheduling actions that can be executed at any given time. ✓ Reward Function: The reward function is designed to reflect the objectives of the optimization problem, such as minimizing tardiness costs, optimizing makespan, and enhancing system ergonomics. ✓ Q-Network and Target Q-Network: Two separate neural networks are employed to approximate the Q-values of state-action pairs, aiding in reducing overestimation bias.

The DDQN algorithm in this work is implemented in Python 3.10 version using GYMNASIUM API and Ray library tailored for deep reinforcement learning applications, in a machine with 1 GPU and 24 CPU and 32 GO of RAM. The implementation follows a systematic procedure:

1. Initialization: Initializing the Q-network and Target Q-network with random weights, and setting initial values for the learning rate = 4e -4 and exploration rate = 8e -4 , which are the defaults used values for Ray Library.

Training:

Training the DDQN algorithm using a replay buffer (with length of 5000) to store and sample experience tuples, and updating the Q-network weights using mini-batch gradient descent. 3. Evaluation: Evaluating the trained DDQN model on a set of test instances, generated like shown in Algorithm 2 to assess its performance in solving the scheduling and reconfiguration problem.

The algorithm begins by initialising the RMS agent with random weights and proceeds to execute a series of RMS episodes. Within each episode, the RMS environment is reset, and for each RMT environment in the set, machine configurations and job distributions are sampled. The algorithm maintains states for both the RMS and RMT environments, updating them as actions are taken. The hierarchical structure becomes apparent as the algorithm handles reconfigurations initiated by the RCA agents and scheduling decisions made by the SA agents within the RMT environments. Information is aggregated between the RMS and RMT environments to enable holistic decision making. Throughout the process, the RL agents learn from their experience and adapt their policies. The algorithm demonstrates a multi-agent, hierarchical approach to resource management, which can be particularly useful in complex systems where coordination and decision making span multiple levels.

Experimental instances generation

In this algorithm, we first generate a set of jobs with random parameters including unit tardiness cost in this interval, due date, completion time, configuration choices, load handling weights, and greenhouse gas emissions. All these values are mentioned in Algorithm 2. We then create a Job object for each job instance and store it in the list of jobs. Finally, we provide a comment on how to represent the job's attributes, including its name, due date, and completion time.

Algorithm 2: Input Jobs Generation

Training methodology

In our exploration of training efficiency, we adopted a segmented approach, particularly focusing on the Reconfigurable Manufacturing Systems (RMS) and the Reconfigurable Manufacturing Tools (RMTs) within a hierarchical environment. Initially, we directed our attention towards individually training the multi-agent system associated with RMTs. A meticulous process of hyperparameter tuning was undertaken to find the optimal settings for the Learning Rate (LR) and the exploration factor ε. This tuning process employed a grid search technique, which is a systematic way of traversing through a manually specified subset of the hyperparameter space. The grid search mechanism allowed us to scrutinize various combinations of LR and ε to pinpoint the configuration that yielded superior training outcomes.

The training curves for the Reconfiguration Agent (RCA) in Figure 2a demonstrate its ability to learn effective policies for initiating reconfiguration actions based on the state of the manufacturing system. The smoothing trend of the reconfiguration time metric indicates the agent's proficiency in generalizing its experience to make timely and impactful reconfiguration decisions.

For the scheduling Agent (SA), Figure 2b exhibits convergence in the tardiness cost. This highlights the agent's competence in developing robust job scheduling sequences that minimize delays and workflow duration even as manufacturing conditions evolve. The stability of the learning process enables the SA agent to consistently generate high-quality solutions, as noted by [START_REF] Zhang | Learning to dispatch for job shop scheduling via deep reinforcement learning[END_REF] in their work on deep reinforcement learning for job shop scheduling.

The tuning config was defined as: Upon obtaining the desired tuning, the training proceeded, during which the Q-Network and Target Network were evolved and refined. These networks embody the core learning mechanisms that guide the agents in making intelligent decisions. Once the training of the RMT multi-agent system reached a satisfactory level, the trained networks were preserved.

Transitioning to the broader hierarchical environment of RMS, we integrated the pre-trained networks from the RMT multi-agent system. This integration serves as a bedrock, providing a substantial head start for the agents operating within the RMS environment. By leveraging the insights and learned behaviors encapsulated in the pre-trained networks, the RMS hierarchical agent could navigate the environment with a higher degree of competency right from the outset. This strategy significantly expedited the training phase, fostering a more efficient learning trajectory and accelerating the attainment of desirable performance levels within the RMS hierarchical environment.

Results and Discussion

The results section encapsulates the insights obtained from the implementation and experimentation phase. Various performance metrics were analyzed to ascertain the effectiveness and efficiency of the proposed DDQN algorithm in managing multi-objective scheduling and reconfiguration challenges in reconfigurable manufacturing systems.

At the higher level, the manufacturing system agent (RMSA) in Figure 3 demonstrates its capability to learn meta-policies that coordinate the lower-level RCA and SA agents to enhance overall system performance. The simultaneous optimization across all key metrics shows the agent's proficiency in balancing trade-offs through hierarchical reinforcement learning.

In summary, the empirical results provide quantitative evidence for the efficacy of the proposed hierarchical DDQN technique in enabling intelligent reconfiguration planning, scheduling, and coordination for next-generation reconfigurable manufacturing systems. Additionally, the Pareto frontier in Figure 4 obtained through the multiobjective DDQN approach exhibits a rich set of non-dominated solutions that make strategic compromises between the conflicting metrics. This highlights the adaptability of the intelligent agents in tailoring decisions to dynamic manufacturing conditions. The findings from the experimentation phase underscore the potential of deep reinforcement learning, particularly the DDQN algorithm, in revolutionizing resource management in reconfigurable manufacturing systems. The ability of the DDQN algorithm to dynamically adapt to changing manufacturing conditions, optimize resource allocation, and significantly improve operational efficiency elucidates its promise as a formidable tool for modern manufacturing environments.

Moreover, the hierarchical structure of the algorithm, as delineated in the implementation section, exhibits a nuanced approach to managing multi-level decision-making processes in complex manufacturing settings. This hierarchical approach enables a harmonized coordination between scheduling and reconfiguration decisions, thereby contributing to enhanced system performance. The success of the DDQN algorithm in both simulated and real-world environments substantiates the viability of leveraging advanced machine learning techniques to tackle intricate scheduling and reconfiguration challenges inherent in reconfigurable manufacturing systems. The positive outcomes from this project pave the way for further exploration and development of intelligent solutions for modern manufacturing challenges, aligning with the broader objectives of Industry 5.0 and smart manufacturing.

Conclusion and Future Work

The findings from this study underscore the potential of leveraging Hierarchical Deep Q-Networks (HDQNs) in addressing the intricate challenges inherent in reconfigurable manufacturing systems (RMS). The proposed framework, through a two-tiered approach, adeptly handles the reconfiguration and scheduling tasks, demonstrating noteworthy improvements in key performance metrics such as reconfiguration time, tardiness cost, and makespan. The agents' capability to learn and adapt to dynamic manufacturing environments, as seen in the training curves and the Pareto frontier, presents a promising pathway towards the realization of more agile and efficient RMS in line with the paradigms of Industry 4.0 and 5.0.

Looking ahead, there are several avenues that can be explored to further enhance the proposed framework: ✓ Investigate multi-objective RL techniques like MO-PPO to explicitly optimize for conflicting objectives like cost, sustainability, and ergonomics in a pareto-optimal manner. ✓ Evaluate the framework on real and larger problem instances with more machines, jobs, and configurations to assess scalability. ✓ Study the integration of the DRL framework with other Industry 4.0 technologies like digital twins, IoT, edge computing to enable real-time intelligent optimization. ✓ Analyze the reliability, robustness and worst-case performance of the DRL agents using formal verification methods. ✓ Evaluate alternate DRL algorithm architectures like attention-based transformers for RMS optimization. Assess interpretability. ✓ Benchmark against traditional optimization techniques like MILP and metaheuristics on extensive problem sets to gain deeper insights.

Action Space:✓✓ F 1 :

 1 Four actions are defined for selecting a new configuration k from 1, 2, ..., K: ✓ Action 1 (act1): Choose k with maximum total tardiness cost j β j of its jobs ✓ Action 2 (act2): Choose k with maximum average tardiness cost 1 n k j β j Action 3 (act3): Choose k with maximum number of jobs n k ✓ Action 4 (act4): Choose k with minimum average safe time 1 n k j ST j If selected k matches current, no reconfiguration is done. State Features: Four state features are used: Number of jobs in each configuration. ✓ F 2 : Total tardiness cost in each configuration. ✓ F 3 : Average tardiness cost in each configuration. ✓ F 4 : Average safe time in each configuration.

  Key state features for SA include: ✓ F 1 : Number of waiting jobs. ✓ F 2 : Current tardiness cost of each job. ✓ F 3 : Unit tardiness cost of each job. ✓ F 4 : Safe time of each job. ✓ F 5 : Due date of each job ✓ F 6 : Processing time of each job Statistical measures on array features produce a 23-dim state space. States are normalized via min -max scaling.

  Average of RMTs agents' Tardiness cost reward ✓ F s = Sustainability objective ✓ F o = Operator intervention objective ✓ w 1 , w 2 , w 3 = Weight factors, which indicated the importance of each criterion. Usualy, these weights are given by the Decision Maker.

Fig. 2 :

 2 Fig. 2: Training curves for RCA and SA

Fig. 3 :

 3 Fig. 3: Manufacturing system agent (RMSA) training curves

Fig. 4 :

 4 Fig. 4: Pareto frontier of non-dominated solutions

Algorithm 1 :

 1 Hierarchical RL for RMS and RMT Input: RMS environment ERMS, Set of RMT environments {ERMT}, Initial jobs Learning rates αRMSA, , 1 Initialize RMSA with random weights θRMS; 2 for each RMS episode do Train all RMT agents {RCA, SA} on batches with learning rates αRCA, αSA;

	3	Reset ERMS;
	4	for each ERMT in {ERMT} do
	5	Sample machine configurations;
	6	Sample new job distribution;
	7	Initialize state s0;
	8	Reset buffers B, completed jobs J;
	9	Aggregate info from {ERMT} into s0,RMS;
	10	while ERMS not done do
	11	Observe st,RMS;
	28	Update st+1,RMT, B, J;
	29	Calculate rewards rt,RCA, rt,SA;
	30	Aggregate RMT info into st+1,RMS;
	31	Store (st,RMS, at,RMS, rt, st+1,RMS);
	32	Train RMSA on batch from replay buffer with learning rate αRMS;

12 Sample at,RMS ∼ ARMS(st,RMS; θRMS); 13 for each ERMT in {ERMT} do 14 if RCA triggered reconfiguration then 15 Calculate reconfiguration time ∆t; 16 Update state st+∆t,RMT; 17 Reset ERMT; 18 Sample new machine configuration; 19 Sample new job distribution; 20 Reset B, J; 21 Initialize state s0,RMT; 22 Update aggregated RMS state st+∆t,RMS; 23 while ERMT not done do 24 Observe state st,RMT; 25 Sample at,RCA ∼ RCA(st,RMT; ); 26 Sample at,SA ∼ SA(st,RMT; ); 27 Execute at,RCA, at,SA in ERMT; 33 Update θRMS; 34

  Number of Jobs num_of _jobs, Available Configurations available_conf igs Result: List of generated jobs jobs 35 for each j in range num_of _jobs do 36 Generate a unique job name name based on j;

	37	Generate unit tardiness cost unit_tard_cost randomly between 2 and 10;
	38	Generate due date due_date randomly between 10 and 50 after the arrival
		of the job;
	39	Generate completion time completion_time randomly between 3 and 20;
	40	Generate the number of configuration choices num_choices randomly
		between 1 and 2;
	41	Generate num_choices random configuration choices conf igs from
		available_conf igs;
	42	Generate quantity of liquid emitted by job of each configuration
		conf igs_lhw;
	43	Generate num_choices random configurations' greenhouse gas emissions
		of each configuration conf igs_ghg;
	44	Create a job object job with attributes name, unit_tard_cost, due_date,
		completion_time, conf igs, conf igs_lhw, conf igs_ghg;

Data:

45 Append job to the list of jobs jobs;

CESI LINEACT, Effat University,and King Fahd University of Petroleum and Minerals for the technical support. They are also thankful to the Effat University for financially supporting this project under the grant number (UC No. 9/12June2023/7.1-21(4)11).