Mark H Davis

Rama Cont

Xin Guo

Renyuan Xu

Pareto Optima for a Class of Singular Control Games In memory of

Keywords: LIBOR rate, interbank markets, stochastic differential game, singular stochastic control, Pareto optimum, Nash equilibrium, Skorokhod problem

We study a class of N -player stochastic differential games of singular control, motivated by the study of a dynamic model of interbank lending with benchmark rates. We describe Pareto optima for this game and show how they may be achieved through the intervention of a regulator, whose policy is a solution to a singular stochastic control problem. Pareto optima are characterized in terms of the solution to a new class of Skorokhod problems with piecewise-continuous free boundary.

Pareto optimal policies are shown to correspond to the enforcement of endogenous bounds on interbank lending rates. Analytical comparison between Pareto optima and Nash equilibria for the case of two players allows to quantify the impact of regulatory intervention on the stability of the interbank rate.

Introduction

The market for interbank lending offers an interesting example of strategic interaction among financial institutions in which players react to an average of the action of other players. One of the widely commented features of the interbank market is the fixing mechanism for interbank benchmark interest rates, the most well-known example of which is the London Interbank Offer Rate (LIBOR) which plays a central role in financial markets. Historically these benchmarks have not been negotiated rates but a 'trimmed' average of quotes collected daily from major banks. At every date t, participating banks contribute a quote X i t representing their offered rate; a calculation agent then 'trims' the tails of the distribution by removing the highest and lowest quotes and computes the value of the benchmark rate X t as a weighted average of the remaining non-discarded quotes [START_REF] Avellaneda | Transparency in over-the-counter interest rate derivatives markets[END_REF]. The resulting benchmark rate X t -the LIBOR rate-then serves as a reference for the valuation of interbank loans and debt contracts, as well as many other financial contracts indexed on the benchmark rate. A deviation (spread) of a bank's rate from the benchmark may lead to a perception of credit risk and loss of market share -if the spread is positive-or an opportunity cost if the spread is negative, thus incentivizing banks to align their offered rates with the benchmark.

This mechanism leads to strategic interactions among market participants in a dynamic setting, where interactions are mediated through an average action, or more generally through the distribution of actions of other participants and has been criticized for its vulnerability to manipulations [START_REF] Avellaneda | Transparency in over-the-counter interest rate derivatives markets[END_REF], which have been extensively documented (H.M. [START_REF] Treasury | The Wheatley Review of LIBOR: Final Report[END_REF][START_REF] Duffie | Reforming LIBOR and other financial market benchmarks[END_REF]. One of the lessons from the manipulation of LIBOR and other benchmarks is that insufficient attention had been paid to incentives, strategic interactions, mechanism design and the role of the regulator in such markets.

A model of interbank lending with benchmark rates

We shall now describe a stylized model of interbank rates which represents interactions among banks in terms of a stochastic dynamic game. Consider first an exogenous process r t representing a rate set by the central bank, with respect to which banks will position their lending rates. r t is typically modeled as a mean-reverting diffusion process driven by a Brownian motion B representing random macroeconomic shocks. Each bank i quotes a rate r i t at a 'spread' X i t with respect to the reference rate r t : r i t = r t + X i t . Bank i is also affected by the macroeconomic shocks represented by B but may control its rate r i t through positive or negative adjustments to its spread X i t , which we may represent by a pair (ξ i,+ , ξ i,-) of non-decreasing processes.

dX i t = σ i dB t + dξ i,+ t -dξ i,- t .
(1.1)

The benchmark ('LIBOR') rate L t is then defined as a weighted average of these offered rates:

L t = r t + X t , X t = N i=1 a i X i t and a i ≥ 0, N i=1 a i = 1.
Note that the 'drift' term in the dynamics (1.1) originates from the control. One may also consider an additional drift term µ i dt in the uncontrolled dynamics, a positive drift corresponding to a bank whose creditworthiness is gradually deteriorating, leading to a steady increase of its spread. We now turn to the incentives and costs faced by banks. Each bank i receives interest income from its lending activity, at rate r i t . The interest income of the bank over a short period [t, t + dt] is r i t Q i t dt where Q i t > 0 is the volume of lending activity (loan volume). Given that the bank can borrow at the interbank L t = r t + X t , this represents an opportunity cost of (X t -X i t )Q i t dt. In a competitive lending market, the market share Q i t of bank i will be a decreasing function q i (.) of its spread r i t -L t = X i t -X t relative to the benchmark rate: Q i t = q i (X i t -X t ). Assuming an inter-temporal discount rate of ρ > 0, this leads to a running cost term ∞ 0 e -ρt (X t -X i t )q i (X i t -X t ) dt.

For example, a linear dependence q i (x) = -κ i x, where κ i > 0 represents the sensitivity of loan volume to the interest rate, leads to a quadratic running cost ∞ 0 κ 1 exp -ρt (X t -X i t ) 2 dt. These considerations only pertain to the relative costs of bank simultaneously engaging in borrowing and lending. Other constraints prevent the banks from deviating from the reference rate beyond a certain level; these are often 'soft', rather than hard (i.e., inequality), constraints and may be modeled by a penalty on |r i t |, or equivalently a running cost f i (X i t ) where f i is centered at some reference value and increases fast enough (e.g., quadratically) at infinity. As an example we shall use f i (x) = ν i (x -s 0 ) 2 with ν i > 0.

The benchmark fixing mechanism described above may be incorporated in the model through a cost term associated with the control (ξ i,+ , ξ i,-). Recall that the LIBOR is computed as a trimmed average of quotes, discarding the highest and lowest 'outliers'. This means an offered rate X i will not be taken into account if it lies too far from the mean. In absence of collusion between banks, this mechanism discourages them from making large daily adjustments to their offered rates, as a large upward or downward adjustment may result in their quote being disregarded in the benchmark calculation. This may be modeled through a cost term which penalizes the size of the adjustment e.g.,

K + i dξ i,+ t +K - i dξ i,- t , with K + i , K - i > 0, where 1/K + i (resp. 1/K - i ) represents a typical distance (X i -X) + (resp. (X -X i ) +
) beyond which quotes are discarded. For instance one can take K + i = K - i = 1/γ where γ represents a measure of dispersion (interquartile range or multiple of standard deviation) of the quote distribution. The case of an asymmetric penalty

K + i > K - i (resp. K + i < K - i
) is useful to model the case of a bank i systematically quoting above (or below) the benchmark. This leads to an objective function

J i (x x x; ξ ξ ξ) = E ∞ 0 e -ρt (X t -X i t )q i (X i t -X t )dt + ν i |X i t -s 0 | 2 dt + K + i dξ i,+ t + K - i dξ i,- t X X X 0-= x x x (1.2)
for bank i, where the control variable is a pair of non-decreasing processes ξ i,+ t , ξ i,- t representing the rate adjustments of bank i and the expectation is taken with respect to the law of the controlled process (1.1). The controls ξ i,+ , ξ i,-are in general allowed to be right-continuous with left limits (càdlàg) which allows for jumps, representing impulses, as well as continuous adjustments to the rates. Such controls are called singular controls [START_REF] Karatzas | A class of singular stochastic control problems[END_REF] and have been used for analyzing optimal investment policy and option pricing and hedging problems with transaction costs [START_REF] Davis | Portfolio selection with transaction costs[END_REF][START_REF] Davis | European option pricing with transaction costs[END_REF][START_REF] Kallsen | Option pricing and hedging with small transaction costs[END_REF], 2017;[START_REF] Zariphopoulou | Investment-consumption models with transaction fees and Markov-chain parameters[END_REF].

In the case where a i = 1 N , q i = q j , ν i = ν j and K ± i = K ± j for i = j, the payoff structure is symmetric under permutation of indices and this can be formulated as mean field game, which was studied in [START_REF] Lasry | Mean field games[END_REF][START_REF] Guo | Stochastic games for fuel follower problem: N versus MFG[END_REF]). However we shall not need this assumption and will treat below the case of a more general, not necessarily symmetric, cost function h i (X X X t ).

A class of stochastic differential games of singular control

Motivated by the example above, we study a class of N -player stochastic differential games, where each player i = 1, • • • , N controls a diffusive process X i t through (positive or negative) ξ ξ ξ i := (ξ i,+ , ξ i,-) additive control terms

dX i t = µ i dt + σ σ σ i • dB B B t + dξ i,+ t -dξ i,- t , X i 0-= x i . (1.3)
and a seeks to minimize the sum of a discounted running cost and a proportional cost of intervention

J i (x x x; ξ ξ ξ) = E ∞ 0 e -ρt h i (X X X t )dt + K + i dξ i,+ t + K - i dξ i,- t X X X 0-= x x x .
The first two terms in (1.3) correspond to the 'baseline' (uncontrolled) diffusion dynamics, and the last two term correspond to the control ξ ξ ξ i = (ξ i,+ , ξ i,-), modeled as a pair of non-decreasing càdlàg processes, leading to a singular control problem [START_REF] Karatzas | A class of singular stochastic control problems[END_REF] for each player. Nash equilibria for such singular control games have been studied in [START_REF] Guo | Stochastic games for fuel follower problem: N versus MFG[END_REF]. In the present work, we focus on Pareto-optimal outcomes.

Contribution. The present work is a study of Pareto-optimal policies for the class of stochastic singular control games considered above, motivated by the interbank lending problem. We relate the Pareto optima of this game to the solution of a 'regulator's problem', characterized as a singular stochastic control problem which we study in detail. The regularity analysis of the value function, following the approach of [START_REF] Soner | Regularity of the value function for a two-dimensional singular stochastic control problem[END_REF], for the regulator's problem enables us to characterize the optimal controls for this problem and subsequently the Pareto-optimal policies for the N -player game.

We obtain a description of Pareto-optimal policies in terms of a multidimensional Skorokhod problem for a 'regulated diffusion' in a bounded region whose boundary is piece-wise smooth with possible corners. The state process follows a diffusion process in the interior, and the control intervenes only at the boundary to reflect it back into the interior.

Finally, we derive explicit descriptions of Pareto-optimal policies when N = 2. This complements the existing literature on Nash equilibrium for stochastic two player games [START_REF] De Angelis | Stochastic nonzero-sum games: a new connection between singular control and optimal stopping[END_REF][START_REF] Dianetti | Nonzero-sum submodular monotone-follower games: existence and approximation of Nash equilibria[END_REF][START_REF] Hernandez-Hernandez | A zero-sum game between a singular stochastic controller and a discretionary stopper[END_REF][START_REF] Kwon | Game of singular stochastic control and strategic exit[END_REF]. Analytical comparison between the Pareto-optimal and the Nash equilibrium solutions demonstrates the role of regulator in the interbank lending game.

Our analysis provides insights for regulatory intervention on the interbank market. In particular, it allows us to quantify the impact of a regulator on the stability of the benchmark rate.

Relation with previous literature. Stylized mean-field models of interbank borrowing and lending have been considered by [START_REF] Carmona | Mean field games and systemic risk[END_REF] and [START_REF] Sun | Systemic risk and interbank lending[END_REF], who focus on Nash equilibria and general stochastic controls.

A related strand of literature consists of studies on central bank interventions on interest rates and exchange rates using an impulse control approach [START_REF] Bensoussan | Impulse control with random reaction periods: a central bank intervention problem[END_REF][START_REF] Cadenillas | Classical and impulse stochastic control of the exchange rate using interest rates and reserves[END_REF][START_REF] Jeanblanc-Picqué | Impulse control method and exchange rate[END_REF]. In these approaches, interventions are associated with a fixed cost and thus finite in number. The singular control framework adopted here seems more natural for modeling situations such as interbank markets where participants intervene continuously and where the cost of intervention is proportional to the action rather than fixed. Singular controls allow for discontinuities, so include impulse control as a special case. However as we will observe in Section 4, Pareto-optimal solutions do not include impulse and instead involve only continuous controls.

Nash equilibria for stochastic games of singular control have been studied by [START_REF] Chiarolla | Generalized Kuhn-Tucker conditions for N-firm stochastic irreversible investment under limited resources[END_REF][START_REF] De Angelis | Stochastic nonzero-sum games: a new connection between singular control and optimal stopping[END_REF]; [START_REF] Dianetti | Nonzero-sum submodular monotone-follower games: existence and approximation of Nash equilibria[END_REF][START_REF] Hernandez-Hernandez | A zero-sum game between a singular stochastic controller and a discretionary stopper[END_REF]; on the other hand, there are few studies of Pareto-optimal strategies for such games. [START_REF] Aïd | The coordination of centralised and distributed generation[END_REF] considered a two-player game in an impulse control framework between a representative energy consumer and a representative electricity producer, and derived an asymptotic Pareto-optimal policy. [START_REF] Fischer | Continuous time mean-variance portfolio optimization through the mean field approach[END_REF] solved explicitly a mean-variance portfolio optimization problem with N stocks. [START_REF] Ferrari | Continuous-time public good contribution under uncertainty: a stochastic control approach[END_REF] and [START_REF] Wang | Dynamic voluntary provision of public goods with uncertainty: a stochastic differential game model[END_REF] considered the problem of public good contribution and analyzed the Pareto-optimal policy for the N -player stochastic game under the framework of regular control and singular control, respectively.

The analysis of Pareto optima in stochastic games is often done by studying an auxiliary Ndimensional stochastic control problem. This approach can be traced back to the economic literature on mechanism design and social welfare optimization in [START_REF] Bator | The simple analytics of welfare maximization[END_REF] and [START_REF] Coleman | Efficiency, utility, and wealth maximization[END_REF]. The mathematical challenge lies in analyzing the associated high-dimensional Hamilton-Jacobi-Bellman (HJB) equations and characterizing the optimal control policy.

Outline. The remainder of the paper is organized as follows. Section 2 presents the mathematical formulation of the N -player stochastic differential game, and describes its relation with the auxiliary control problem. Section 3 provides detailed analysis of the auxiliary control problem and the construction of the optimal strategies. Section 4 characterizes the Pareto optima in terms of a sequence of Skorokhod problems. Implications of our analysis for the interbank lending problem are discussed in Section 4.3. Section 5 provides explicit solutions in the case N = 2, and compares it with the Nash equilibrium.

Mathematical formulation of the game

In this section, we describe the mathematical framework of the N -player game.

Controlled dynamics. Let (X i t ) t≥0 ∈ R denote the state of player i at time t, 1 ≤ i ≤ N . In the absence of controls, X X X

t := (X 1 t , . . . , X N t ) ∈ R N follows X X X t = X X X 0 + µ µ µt + σ σ σB B B t , X X X 0 = (x 1 , . . . , x N ), (2.1)
where B B B := (B 1 , . . . , B D ) ∈ R D is a D-dimensional Brownian motion on a filtered probability space (Ω, F, {F t } t≥0 , P), and µ µ µ := (µ 1 , . . . , µ N ) ∈ R N and σ σ σ := (σ ij ) 1≤i≤N,1≤j≤D ∈ R N ×D are constants with σ σ σσ σ σ T λI for some λ > 0.

When player i chooses a control ξ ξ ξ i := (ξ i,+ , ξ i,-) from an admissible control set U i N , then X i t evolves as

dX i t = µ i dt + σ σ σ i • dB B B t + dξ i,+ t -dξ i,- t , X i 0-= x i . (2.2)
Here ξ ξ ξ i = (ξ i,+ , ξ i,-) is a pair of non-decreasing càdlàg processes and σ σ σ i is the i th row of the volatility matrix σ σ σ. We will denote by P x x x the law of the process (2.2) and E x x x the expectation with respect to this law.

Admissible controls U i N . The admissible control set for player i is defined as

U i N = (ξ i,+ t , ξ i,- t ) t≥0 | ξ i,+ t and ξ i,- t are F t -progressively measurable, càdlàg non-decreasing, with E ∞ 0 e -ρt dξ i,+ t < ∞, E ∞ 0 e -ρt dξ i,- t < ∞, ξ i,+ 0-= 0, ξ i,- 0-= 0 .
(2.3) Objective functions. Each player i chooses a control (ξ i,+ , ξ i,-) in U i N to minimize

J i (x x x; ξ ξ ξ) = E x x x ∞ 0 e -ρt h i (X X X t )dt + K + i dξ i,+ t + K - i dξ i,- t . (N-player)
Here ρ > 0 is a constant discount factor, K + i , K - i > 0 are the cost of controls, and h i (x x x) : R N → R + is the running cost function.

In this paper, we focus on characterizing Pareto optima of the game (N-player) subject to the dynamics (2.2).

Definition 1 (Pareto optimality). ξ ξ ξ * ∈ U

N := (U 1 N , • • • , U N N
) is a Pareto-optimal policy for the game (N-player) if and only if there does not exist ξ ξ ξ ∈ U N such that ∀i ∈ {1, . . . , N }, J i (x x x; ξ ξ ξ) ≤ J i (x x x; ξ ξ ξ * ) ; and ∃j ∈ {1, . . . , N }, J j (x x x; ξ ξ ξ) < J j (x x x; ξ ξ ξ * ) .

Pareto optima correspond to efficient outcomes of a game, which may or may not result from decentralized optimization by N players. The intervention of a regulator may be necessary to enforce a Pareto-optimal policy.

Regulator's problem

To study Pareto optima for game (N-player), we consider a 'welfare function' defined as an aggregated cost:

J(x x x; ξ ξ ξ) = N i=1 L i J i (x x x, ξ ξ ξ) (3.1) = E x x x ∞ 0 e -ρt H(X X X t )dt + N i=1 L i K + i dξ i,+ t + N i=1 L i K - i dξ i,- t ,
were the dynamics of X X X t is given by (2.2), and

H(x x x) := N i=1 L i h i (x x x), with L i > 0 and N i=1 L i = 1. (3.2)
We will show that Pareto optima of (N-player) correspond to solutions of the following auxiliary stochastic control problem v(x x x) = min

ξ ξ ξ∈U N J L (x x x; ξ ξ ξ), (Regulator)
which may be interpreted as the problem facing a market regulator seeking to optimize the aggregate cost (3.1).

To ensure that the game is well defined, the following assumptions will be made throughout the paper, unless otherwise specified:

Assumptions. There exist C > c > 0 such that A1. ∀x x x ∈ R N , 0 ≤ H(x x x) ≤ C(1 + x x x 2 ). A2. ∀x x x, x x x ∈ R N , |H(x x x) -H(x x x )| ≤ C(1 + x x x + x x x ) x x x -x x x . A3. H(x x x) ∈ C 2 (R N ), H is convex, with 0 < c ≤ ∂ 2 z z z H(x x x) ≤ C for all unit direction z z z ∈ R N .
For example, for the payoff described in the interbank lending problem in Section 1.1,

H(x x x) = N i=1 L i   κ i x i - j =i a j x j 2 + ν i (x i ) 2   with κ i , ν i > 0. (3.3)
Then H satisfies A1-A3 for any choice of weight L i > 0.

We shall first analyze the regularity property of the value function v, which is necessary for establishing subsequently the existence and uniqueness of the optimal control. We then see that the optimal control for (Regulator) yields a Pareto-optimal policy for game (N-player).

The regularity analysis of the value function involves several steps. The first step is to show that the value function for (Regulator) is a viscosity solution to the following HJB equation

max{ρu -Lu -H(x x x), β(∇u) -1} = 0, (3.4) with the operator L = 1 2 N i,j=1 σ σ σ i • σ σ σ j ∂ 2 x i x j + N i=1 µ i ∂ x i , and
β(q q q) = max 1≤i≤N q i L i K - i + ∨ q i L i K + i - , (3.5)
where q q q := (q 1 , • • • , q N ), (a) + = max{0, a} and (a) -= max{0, -a} for any a ∈ R.

The second step is to show that the value function for (Regulator) is W 2,∞ loc . The third step is to show that the HJB equation (3.4) has a unique W 2,∞ loc solution. Let us start with the following property of the value function v for the control problem (Regulator).

Proposition 2. Under Assumptions A1-A2, there exists K > 0 such that,

(i) 0 ≤ v(x x x) ≤ K(1 + x x x 2 ), ∀x x x ∈ R N ; (ii) |v(x x x) -v(x x x )| ≤ K(1 + x x x + x x x ) x x x -x x x , ∀x x x, x x x ∈ R N .
Proof. First, v(x x x) ≥ 0 is clear by the non-negativity of H(x x x). Moreover, by the property that σ σ σσ σ σ T λI with λ > 0, it follows from a known estimate and the martingale argument (Menaldi & Robin, 1983, (2.15)

) that the solution { X X X t } t≥0 := {x x x + µ µ µt + σ σ σB B B t } t≥0 with ξ ξ ξ = 0 0 0 satisfies E x x x ∞ 0 e -ρt X X X t 2 dt ≤ K(1 + x x x 2 ), ∀x x x ∈ R N ,
for some constant K > 0. By Assumption A1, there exists a constant K > 0 such that

v(x x x) ≤ J(x x x, 0 0 0) ≤ K(1 + x x x 2 ), ∀x x x ∈ R N . Thus (i) of Proposition 2 is established. For each fixed x x x ∈ R N , let U x x x = {ξ ξ ξ ∈ U : J(x x x, ξ ξ ξ) ≤ J(x x x; 0 0 0)}. (3.6) By Assumption A1, E x x x ∞ 0 e -ρt X X X t 2 dt ≤ K(1 + x x x 2 ), ∀x x x ∈ R N , ξ ξ ξ ∈ U x x x . (3.7) For ξ ξ ξ ∈ U x x x , it is easy to verify that E x x x ∞ 0 e -ρt ξ ξ ξ t 2 dt ≤ K(1 + x x x 2 ), (3.8) and |v(x x x) -v(x x x )| ≤ sup {|J(x x x; ξ ξ ξ) -J(x x x ; ξ ξ ξ)| : ξ ξ ξ ∈ U x x x ∪ U x x x } , ∀x x x, x x x ∈ R N . Meanwhile, |J(x x x; ξ ξ ξ) -J(x x x ; ξ ξ ξ)| ≤ E ∞ 0 e -ρt |H(X X X x x x t ) -H(X X X x x x t )|dt.
Statement (ii) for v follows by Assumption A2, along with the facts that X X X x x x t -X X X x x x t = x x x -x x x and that for any

ξ ξ ξ ∈ U x x x ∪ U x x x , E x x x ∞ 0 e -ρt X X X x x x t dt ≤ K(1 + x x x + x x x ), (3.9) E x x x ∞ 0 e -ρt X X X x x x t dt ≤ K(1 + x x x + x x x ).
In fact, if ξ ξ ξ ∈ U x x x , (3.9) follows immediately from (3.8) by the Hölder inequality. Meanwhile, if ξ ξ ξ ∈ U x x x , (3.9) holds because

X X X x x x t ≤ X X X x x x t + x x x -x x x ≤ X X X x x x t + x x x + x x x .
Next, we establish the viscosity property of the value function in the following sense.

Definition 3 (Continuous viscosity solution). The value function v for problem (Regulator) is a continuous viscosity solution to

(3.4) on R N if • ∀x x x 0 ∈ R N , ∀φ ∈ C 2 (R N ) such that x x x 0 is a local minimum of (v -φ)(x x x) with v(x x x 0 ) = φ(x x x 0 ), max{ρφ -Lφ -H(x x x), β(∇φ) -1} ≥ 0. • ∀x x x 0 ∈ R N , ∀φ ∈ C 2 (R N ) such that x x x 0 is a local maximum of (v -φ)(x x x) with v(x x x 0 ) = φ(x x x 0 ), max{ρφ -Lφ -H(x x x), β(∇φ) -1} ≤ 0.
Theorem 4 (Viscosity solution). Under Assumptions A1 and A3, the value function v to the control problem (Regulator) is a continuous viscosity solution of the HJB equation (3.4).

Proof. The convexity of v follows from the joint convexity of J(x x x; ξ ξ ξ) in the following sense:

J(θx x x + (1 -θ)x x x ; θξ ξ ξ + (1 -θ)ξ ξ ξ ) ≤ θJ(x x x; ξ ξ ξ) + (1 -θ)J(x x x ; ξ ξ ξ ), (3.10)
for any x x x, x x x ∈ R N and any ξ ξ ξ, ξ ξ ξ ∈ U N . The convexity of J in (x x x; ξ ξ ξ) is then obvious since X X X x x x t depends linearly on (x x x, ξ ξ ξ) and the set U N and the function H are both convex.

We now show that v is both a viscosity super-solution and sub-solution to the HJB equation (3.4).

Sub-solution. Consider the following controls: ξ i,- t = 0 and

ξ i,+ t = 0, t = 0, η i,+ , t ≥ 0,
where 0 ≤ η i,+ ≤ . Define the exit time

τ := inf{t ≥ 0, X X X t / ∈ B (x x x 0 )}.
Note that X X X has at most one jump at t = 0 and is continuous on [0, τ ). By the dynamic programming principle,

φ(x x x 0 ) = v(x x x 0 ) ≤ E x x x 0 τ ∧h 0 e -ρt H(X X X t )dt + N i=1 L i K + i dξ i,+ t + E x x x 0 e -ρ(τ ∧h) φ(X X X τ ∧h ) . (3.11)
Applying Itô's formula to the process e -ρt φ(X X X t ) between 0 and τ ∧ h, and taking expectation, we obtain

E x x x 0 e -ρ(τ ∧h) φ(X X X τ ∧h ) = φ(x x x 0 ) + E x x x 0 τ ∧h 0 e -ρt (-ρφ + Lφ)(X X X t )dt + E x x x 0 0≤t≤τ ∧h [φ(X X X t ) -φ(X X X t-)] .
(3.12)

Combining (3.11) and (3.12), we have

E x x x 0 τ ∧h 0 e -ρt (ρφ -Lφ -H)(X X X t )dt -E x x x 0 τ ∧h 0 e -ρt ( N i=1 L i K + i dξ i,+ t ) -E x x x 0 0≤t≤τ ∧h φ(X X X t ) -φ(X X X t-) ≤ 0. (3.13)
• Taking first η i,+ = 0 for all i = 1, 2, • • • , N , i.e., ξ i,+ = ξ i,-= 0, we see that X X X is continuous and that only the first term in the LHS of (3.13) is nonzero. Dividing the above inequality (3.13) by h and letting h → 0, then by the dominated convergence theorem,

ρφ(x x x 0 ) -Lφ(x x x 0 ) -H(x x x 0 ) ≤ 0.
• Now, by taking η i,+ > 0 and η j,+ = 0 for j = i in (3.13), and noting that ξ i,+ and X X X jump only at t = 0 with size η i,+ , we get

τ ∧h 0 e -ρt (ρφ -Lφ -H)(X X X t )dt -L i K + i η i,+ -φ(x x x 0 + η i,+
e e e i ) + φ(x x x 0 ) ≤ 0.

Taking h → 0, then dividing by η i,+ and letting η → 0, we have

-L i K + i ≤ ∂ x i φ(x x x).
• Meanwhile, taking an admissible control such that ξ i,+ = 0 and

ξ i,- t = 0, t = 0, η i,-, t ≥ 0,
where 0 ≤ η i,-≤ . By a similar argument, we have

∀i = 1, 2, • • • , N, ∂ x i φ(x x x) ≤ L i K - i .
This proves the sub-solution viscosity property

max{ρφ -Lφ -H(x), β(∇φ) -1} ≤ 0.
Super-solution. This part is proved by contradiction. Suppose the claim is not true. Then there exist

x x x 0 ∈ R N , > 0, φ(x x x) ∈ C 2 (R N ) with φ(x x x 0 ) = v(x x x 0 ), v ≥ φ in B (x x x 0 ) and ν > 0 such that for all x x x ∈ B (x x x 0 ), ρφ(x x x 0 ) -Lφ(x x x 0 ) -H(x x x 0 ) ≤ -ν, (3.14)
and

-L i K + i + ν ≤ ∂ x i φ ≤ L i K - i -ν. (3.15)
Given any admissible control ξ ξ ξ, consider the exit time τ = inf{t ≥ 0, X X X t / ∈ B (x x x 0 )}. Applying Itô's formula (Meyer, 1976, Theorem 21) to e -ρt φ(x x x) and any semi-martingale {X X X t } t≥0 under admissible control (ξ i,+ , ξ i,-) N i=1 leads to

E x x x 0 e -ρτ φ(X X X τ -) = φ(x x x 0 ) + E x x x 0 τ 0 e -ρt (-ρφ + Lφ)(X X X t )dt + E x x x 0 τ 0 e -ρt N i=1 ∂ x i φ(X X X t )[(dξ i,+ t ) c -(dξ i,- t ) c ] + E x x x 0 0≤t<τ e -ρt [φ(X X X t ) -φ(X X X t-)] .
Note that for all 0 ≤ t < τ , X X X t ∈ B (x x x 0 ). Then, by (3.14), and noting that

∆X i t = ∆ξ i,+ t -∆ξ i,- t , we have for all 0 ≤ t < τ , φ(X X X t ) -φ(X X X t-) = N i=1 ∆X i t 1 0 ∂ x i φ(X X X t + z∆X X X t )dz ≤ N i=1 (L i K - i -ν)∆ξ i,+ t + (L i K + i -ν)∆ξ i,- t . Similarly, φ(X X X t ) -φ(X X X t-) ≥ N i=1 -(L i K - i -ν)∆ξ i,- t -(L i K + i -ν)∆ξ i,+ t . (3.16)
In light of relations (3.14)-(3.16),

E x x x 0 e -ρτ φ(X X X τ -) ≥ φ(x x x 0 ) + E x x x 0 τ 0 e -ρt (-H + ν)(X X X t )dt + E x x x 0 τ - 0 e -ρt N i=1 -(L i K + i -ν)dξ i,+ t -(L i K - i -ν)dξ i,- t = φ(x x x 0 ) -E x x x 0 τ 0 e -ρt H(X X X t )dt + N i=1 L i K + i dξ i,+ t + N i=1 L i K - i dξ i,- t + N i=1 E x x x 0 e -ρτ L i K + i ∆ξ i,+ τ + E x x x 0 e -ρτ L i K - i ∆ξ i,- τ + ν E x x x 0 τ 0 e -ρt dt + E x x x 0 τ - 0 e -ρt (dξ i,+ t + dξ i,- t ) . (3.17)
Note that X X X τ -∈ B (x x x 0 ), X X X τ is either on the boundary ∂B (x x x 0 ) or out of B (x x x 0 ). However, there is some random variable δ valued in [0, 1] such that

x x x δ = X X X τ -+ δ∆X X X τ = X X X τ -+ δ(∆ξ ξ ξ + t -∆ξ ξ ξ - t ) ∈ ∂ B (x x x 0 ). Then similar to (3.16), we have φ(x x x δ ) -φ(X X X τ -) ≥ δ N i=1 -(L i K - i -ν)∆ξ i,- τ -(L i K + i -ν)∆ξ i,+ τ . (3.18) Note that X X X τ = x x x δ + (1 -δ)(∆ξ ξ ξ + t -∆ξ ξ ξ - t ), thus v(x x x δ ) ≤ (1 -δ) N i=1 L i K + i ∆ξ i,+ τ + L i K - i ∆ξ i,- τ + v(X X X τ ). (3.19) Recalling that v(x x x δ ) ≥ φ(x x x δ ), inequalities (3.18)-(3.19) imply (1-δ) N i=1 L i K + i ∆ξ i,+ τ + L i K - i ∆ξ i,- τ +v(X X X τ ) ≥ φ(X X X τ -)+δ N i=1 -(L i K - i -ν)∆ξ i,- τ -(L i K + i -ν)∆ξ i,+ τ . Therefore, N i=1 (L i K + i -δν)∆ξ i,+ τ + (L i K - i -δν)∆ξ i,- τ + v(X X X τ ) ≥ φ(X X X τ -).
Plugging the last inequality into (3.17), along with φ(x x x 0 ) = v(x x x 0 ), yields

E x x x 0 e -τ N i=1 (L i K + i -δν)∆ξ i,+ τ + (L i K - i -δν)∆ξ i,- τ + v(X X X τ ) ≥ v(x x x 0 ) -E x x x 0 τ 0 e -ρt H(X X X t )dt + N i=1 L i K + i dξ i,+ t + N i=1 L i K - i dξ i,- t + N i=1 E x x x 0 e -ρτ L i K + i ∆ξ i,+ τ + E x x x 0 e -ρτ L i K - i ∆ξ i,- τ + ν E x x x 0 τ 0 e -ρt dt + E x x x 0 τ - 0 e -ρt (dξ i,+ t + dξ i,- t ) . Hence E x x x 0 e -τ v(X X X τ ) + E x x x 0 τ 0 e -ρt H(X X X t )dt + N i=1 L i K + i dξ i,+ t + N i=1 L i K - i dξ i,- t ≥ v(x x x 0 ) + ν E x x x 0 τ 0 e -ρt dt + E x x x 0 τ - 0 e -ρt (dξ i,+ t + dξ i,- t ) + δE x x x 0 e -τ ∆ξ i,+ τ + e -τ ∆ξ i,- τ .
We now claim that there exists a constant g 0 > 0 such that for all admissible control ξ ξ ξ,

E x x x 0 τ 0 e -ρt dt + E x x x 0 τ - 0 e -ρt (dξ i,+ t + dξ i,- t ) + δE x x x 0 e -τ ∆ξ i,+ τ + e -τ ∆ξ i,- τ ≥ g 0 . (3.20)
Indeed, one can always find some constant G 0 such that the C 2 function

ψ(x x x) = G 0 ((x x x -x x x 0 ) 2 -2 ) satisfies min i {ρψ -Lψ + 1, 1 -|∂ x i ψ|} ≥ 0, on B (x x x 0 ), ψ = 0 on ∂B (x x x 0 ).
Applying Meyer's version of Itô's formula (Meyer, 1976, Theorem 21) to e -ρt ψ(x x x) and any semimartingale {X X X t } t≥0 under admissible control (ξ i,+ , ξ i,-) N i=1 leads to

E x x x 0 e -ρτ ψ(X X X τ -) ≤ ψ(x x x 0 ) + E x x x 0 τ 0 e -ρt dt + N i=1 E x x x 0 τ - 0 e -ρt (dξ i,+ t + dξ i,- t ) . (3.21) Since ψ x i (x x x 0 ) ≥ -1 for all i = 1, 2, • • • , N , ψ(X X X τ -) -ψ(x x x δ ) ≥ -∇ψ(X X X τ --x x x δ ) ≥ -δ N i=1 ∆ξ i,- τ ,
which, combined with (3.21), yields

E x x x 0 τ 0 e -ρt dt + N i=1 E x x x 0 τ - 0 (dξ i,+ t + dξ i,- t ) + E x x x 0 e -ρτ δ N i=1 ∆ξ i,- τ ≥ E x x x 0 e -ρτ ψ(x x x δ ) -ψ(x x x 0 ) = G 0 2 .
Hence (3.20) holds with g 0 = G 0 2 .

We can further show that the value function is the unique and W 2,∞ loc (R N ) solution to the HJB equation (3.4).

Theorem 5 (Regularity and uniqueness.). Under Assumptions A3, the value function v to the control problem (Regulator) is the unique W 2,∞ loc (R N ) solution to the HJB equation (3.4). In addition, there exists K > 0 such that

0 ≤ ∂ 2 z z z v(x x x) ≤ K, a.e. for x x x ∈ R N , (3.22)
for any second order directional derivative ∂ 2 z z z . Finally, v is C 4,α in the continuation region

C N := {x x x | β(∇v(x x x)) < 1} , (3.23)
and C N is bounded.

Proof. To prove v ∈ W 2,∞ loc , let B be any open ball and let ψ ∈ C ∞ 0 (R N ) be any test function with a support contained in B. Since (∆x i ) -2 δ 2 i v(x x x) is bounded on B for |∆x i | ≤ 1, there is a sequence η k → 0+ as k → ∞ such that, denoting by g k the result of replacing ∆x i by η k in (∆x i ) -2 δ 2 i v(x x x), we have g k → Q weakly in L p (B) for some p with 1 < p < ∞. It is then easy to see that

R N ψ(x x x)Q(x x x)dx x x = R N ∂ 2 x i ψv(x x x)dx x x, ∀ψ ∈ C ∞ 0 (B), (3.24)
where

Q = ∂ 2 x i v.
The existence and local boundedness of second order derivatives is now immediate: for k = 1, 2, . . . , N , let e e e k denote the unit vector in the direction of the positive x k axis; for any fixed i = j with 1 ≤ i, j ≤ N , let y y y be a new coordinate whose axis points to the e e e i +e e e j √ 2 direction, then

∂ 2 x i x j v = ∂ 2 y y y v -1 2 (∂ 2 x i v + ∂ 2 x j v).
To show that v is the unique solution to HJB, we proceed by a contradiction argument. Suppose v 1 and v 2 are two non-negative solutions. Let y y y 0 be the point where v 2 attains its minimum value. Given δ > 0, define

φ δ (x x x) := v 1 (x x x) -v 2 (x x x) -δ x x x -y y y 0 2 , ∀x x x ∈ R N .
The function φ δ attains its maximum at some x δ ∈ R N and

0 = ∇φ δ (x x x δ ) = ∇v 1 (x x x) -∇v 2 (x x x) -2δ(x x x δ -y y y 0 ). (3.25)
This leads to

∇v 1 (x x x δ ) = ∇v 2 (x x x δ ) + 2δ(x x x δ -y y y 0 ).
Consequently,

1 ≥ β(∇v 1 (x x x δ )) = β(∇v 2 (x x x δ ) + 2δ(x x x δ -y y y 0 )).
Since y 0 is the minimal point of v 2 , we have

∇v 2 (x x x δ ) • (x x x δ -y y y 0 ) ≥ 0.
This means that either β(∆v 2 (x x x δ )) < 1, or for any i ∈ arg max β(∇v 2 (x x x δ )), (x x x δ -y y y 0 ) i = 0. Suppose the latter, then by (3.25),

0 = ∂ x i v 1 (x x x δ ) -∂ x i v 2 (x x x δ ) -2δ(x x x δ -y y y 0 ) i .
Hence

∂ x i v 1 (x x x δ ) = ∂ x i v 2 (x x x δ ) = 0,
for i ∈ arg max β(∇v 2 (x x x δ )). This implies β(∇v 2 (x x x δ )) < 1. Meanwhile from (3.4), we know

∆v 2 (x x x δ ) = v 2 (x x x δ ) -H(x x x δ ).
By Bony's maximum principle [START_REF] Lions | A remark on Bony maximum principle[END_REF],

0 ≥ lim inf ess x x x→x x x δ ∆φ δ (x x x) = lim inf ess x x x→x x x δ ∆v 1 (x x x) -∆v 2 (x x x) -4δ ≥ v 1 (x x x δ ) -v 2 (x x x δ ) -4δ.
It follows that for any x x x ∈ R N , v 1 (x x x) -v 2 (x x x) = φ δ (x x x) + δ x x x -y y y 0 2 ≤ φ δ (x x x δ ) + δ x x x -y y y 0 2 ≤ δ(4 + x x x -y y y 0 2 ).

Letting δ → 0, we have v 1 (x x x) ≤ v 2 (x x x). Similarly, we have v

2 (x x x) ≤ v 1 (x x x). Since |∂ x i v(x x x)| ≤ L i max{K + i , K - i } (i = 1, 2, • • • , N ) on R N
but H grows at least quadratically by Assumption A3, C N must be bounded.

Finally, let B be any open ball such that B ∈ C N . By Theorem 6.13 in [START_REF] Gilbarg | Elliptic Partial Differential Equations of Second Order[END_REF], the Dirichlet problem in B, ρṽ -Lṽ = H(x x x), ∀x ∈ B, ṽ = v, ∀x ∈ ∂B, (3.26) has a solution ṽ ∈ C 0 ( B)∩C 2,α (B). In particular, ṽ -v ∈ W 2,∞ (B), therefore by (3.26), ṽ -v ∈ W 1,2 0 (B). By Theorem 8.9 of [START_REF] Gilbarg | Elliptic Partial Differential Equations of Second Order[END_REF], v = ṽ in B, thus v ∈ C 2,α (B). By Theorem 6.17 of [START_REF] Gilbarg | Elliptic Partial Differential Equations of Second Order[END_REF], v ∈ C 4,α (B) thus v ∈ C 4,α (C N ) for all α ∈ (0, 1).

To prove (3.22), let ∆ i x x x := (0, • • • , 0, ∆x i , 0, • • • , 0) be the N-dimensional row vector with the i-th entry being ∆x i for i = 1, 2, • • • , N . For any function F : R N → R, define the second difference of F in the x i direction by

δ 2 i F (x x x) = F (x x x + ∆ i x x x) + F (x x x -∆ i x x x) -2F (x x x). (3.27)
It is easy to check that

δ 2 i v(x x x) ≤ sup{δ 2 i J (x x x; ξ ξ ξ) : ξ ξ ξ ∈ U x x x }. (3.28) Since H ∈ C 2 (R N ), for x x x ∈ R N , δ 2 i H(x x x) = (∆x i ) 2 1 0 λ -λ ∂ 2 x i H(x 1 , . . . , x i + µ∆x i , . . . , x N )dµdλ. (3.29)
By Assumption A3,

δ 2 i H(x x x) ≤ K(∆x i ) 2 1 0 λ -λ dµdλ = (∆x i ) 2 K. (3.30) Hence 0 ≤ δ 2 i v(x x x) ≤ K(∆x i ) 2 , x x x ∈ R N , |∆x i | ≤ 1. (3.31)
The lower bound of (3.31) follows from the convexity of v by Theorem 4.

Remark 6. The proof of Theorem 5 is inspired by the approach in (Soner & Shreve, 1989, Theorem 4.5) and (Williams et al., 1994, Theorem 3.1). In [START_REF] Soner | Regularity of the value function for a two-dimensional singular stochastic control problem[END_REF], the following HJB equation (3.32) (See Eqn. 3.1 in [START_REF] Soner | Regularity of the value function for a two-dimensional singular stochastic control problem[END_REF]) was studied for an

N -dimensional control problem max    ρu -Lu -H(x x x), N i=1 (∂ x i u) 2 -1    = 0. (3.32)
Comparing the gradient constraints in (3.32) with (3.4), it is clear that the operator β in (3.4) is less regular than ∇u 2 in (3.32) as ∇u(•) 2 has smoother and gradual changes in the state space R N . In contrast, β in (3.4) involves a maximum operator as a result of game interactions.

The very same HJB equation (3.4) has appeared in [START_REF] Menaldi | Optimal correction problem of a multidimensional stochastic system[END_REF] for analyzing the convergence of finite variation controls from controls of bounded velocity. However, no characterization of the optimal control nor regularity analysis has been studied.

Pareto-optimal policies

The regularity analysis of the value function for problem (Regulator) enables us to establish the existence and the uniqueness of its optimal control, for any given weight

(L 1 , • • • , L N ) such that L i > 0 and N i=1 L i = 1 (Section 4.1
). The optimal control in (Regulator) is then shown to lead to a Paretooptimal policy for game (N-player) ( Theorem 12) for each choice of weights

(L 1 , • • • , L N ).

Optimal policy for the regulator

To ensure the uniqueness of the Pareto-optimal policy, we impose the following assumption on the value function v.

A4. The diagonal dominates the row/column in the Hessian matrix ∇ 2 v. That is,

∂ 2 x i v(x x x)> j =i ∂ 2 x i x j v(x x x) , ∀i, = 1, 2, • • • , N and x x x ∈ C N . (4.1)
Note that a similar assumption was used in (Gomes et al., 2010, Assumption 3) to analyze Nash equilibrium strategies. This assumption guarantees that the reflection direction of the Skorokhod problem is not parallel to the boundary, and that the controlled dynamics are continuous when x x x ∈ C N . Assumption A4 may be relaxed using techniques of [START_REF] Kruk | Optimal policies for N-dimensional singular stochastic control problems part I: the Skorokhod problem[END_REF] to deal with possible jumps at the reflection boundary.

Given this additional assumption and the regularity of the value function, we are now ready to characterize the Pareto-optimal policy to game (N-player).

We shall show that when x x x ∈ C N , the optimal policy can be constructed by formulating and solving a sequence of Skorokhod problems with piecewise C 1 boundaries, and then by passing to the limit of this sequence of -optimal policies. We shall also show that the reflection field of the Skorokhod problem can be extended to the entire state space under appropriate conditions, completing the construction of the Pareto-optimal policy when x x x is outside C N .

Optimal policy for x x x ∈ C N . First, recall the definition of the Skorokhod problem [START_REF] Ramanan | Reflected diffusions defined via the extended Skorokhod map[END_REF].

Definition 7 (Skorokhod problem). Let G be an open domain in R N with S = ∂G. Let Γ(a, b) = {x ∈ R N : |x -a| = b}.
To each point x x x ∈ S, we will associate a set r r r(x x x) ⊂ Γ(0, 1) called the directions of reflection. We say that a continuous process

ξ ξ ξ t = t 0 N N N s dη s , (4.2) with η t = [0,t] ξ ξ ξ, is a solution to a Skorokhod problem with data (x x x + µ µ µt + σ σ σB B B t , G, r r r, x x x) if (a) |N N N t | = 1, η t is continuous and nondecreasing; (b) the process X X X t = x x x + µ µ µt + σ σ σB B B t + t 0 N N N s dη s satisfies X X X t ∈ G, 0 ≤ t < ∞, a.s; (c) for every 0 ≤ t < ∞, η t = t 0 1 (X X Xs∈∂G,N N N s∈r r r(X X Xs)) dη s .
Now let us introduce some notations for the Skorokhod problem associated with the continuation region C N defined in (3.23). By definition,

C N = {x x x | β(∇v(x x x)) < 1} = ∩ 2N j=1 G j , (4.3) where for i = 1, 2, • • • , N , G i = {x x x | ∂ x i v(x x x) < L i K - i }, G i+N = {x x x | ∂ x i v(x x x) > -L i K + i }. (4.4) Denote S = ∂C N as the boundary of C N , denote I(x x x) = {j | x x x / ∈ G j , j = 1, 2, • • • , 2N
} as the boundary that x x x lines on, and define the vector field γ j on each face G j as γ i = -e e e i , γ i+N = e e e i , (4.5)

where e e e i = (0, • • • , 0, 1, 0, • • • , 0) with the i th component being 1. Then the directions of the reflection is defined as

r r r(x x x) =    j∈I(x x x) c j γ j (x x x) : c i ≥ 0 and j∈I(x x x) c j γ j (x x x) = 1    . (4.6)
Theorem 8 ( -policy). Assume Assumptions A1-A4 holds and x x x ∈ C N . For any > 0, there exists a unique solution to the Skorokhod problem with data (x x x + µ µ µt + σ σ σB B B t , C , r r r , x x x) is an -optimal (admissible) policy of the control problem (Regulator) with

ξ ξ ξ t = t 0 N N N s • dη s , (4.7)
and N N N s = r r r (X X X s ) on S , where X X X t = x x x + µ µ µt + σ σ σB B B t + ξ ξ ξ t . That is,

J(x x x, ξ ξ ξ ) < v(x x x) -C 0 ,
for some constant C 0 that is independent of . Here C ⊆ C has piecewise smooth boundaries.

Proof. The proof consists two steps. We first construct an approximation

C of C N with piecewise C 1 boundaries. Clearly, if ∂C N itself is piecewise C 1 , the C = C N .
We then show that the solution to the Skorokhod problem with piecewise smooth boundary provides an -policy to the regulator's control problem.

Step 1: Skorokhod problem with piecewise smooth boundary.

Let φ δ (x) ∈ C ∞ (R N , R + ) be such that φ δ (x x x) = 0 for |x x x| ≥ δ and R N φ δ (x x x)dx x x = 1. (4.8) Since v ∈ W 2,∞ loc (R N ), consider a regularization of v(x x x) via φ , such that v δ (x x x) = φ δ * v(x x x).
(4.9)

The boundedness of v, ∇v, D 2 v on B R (0), with C N ⊂ B R-1 (0), implies that H δ , v δ are bounded uniformly on C N for δ < 1, and

v δ → v, ∇v δ → ∇v, H δ → H uniformly in C N . Denote K max = max i=1,2,••• ,N {L i K + i , L i K - i }, K min = min i=1,2,••• ,N {L i K + i , L i K - i } and recall K in (3.22) such that 0 ≤ ∂ 2 z z z v(x x x) ≤ K for any second order directional derivative ∂ 2 z z z . Then, for any k ∈ (0, 1 4 ), there exists δ k := δ k ( k ) ∈ 0, k K min K such that for all δ ∈ [0, δ k ], ∇v δ -∇v 1 < K min k . Take a non-negative and non-increasing sequence { k } k such that lim k→∞ k = 0. Denote w δ k (x x x) = β(∇v δ k (x x x)) and C k := {x x x | w δ k (x x x) < 1 -2 k } = ∩ 2N j=1 G k j , where i = 1, 2, • • • , N , G k i = {x x x | ∂ x i v δ k (x x x) < (1 -2 k )L i K - i }, G k i+N = {x x x | ∂ x i v δ k (x x x) > (-1 + 2 k )L i K + i }. (4.10)
Since ∇v δ k -∇v 1 < K min k in C N and by the definition in (4.10), we have

C k ⊂ C N . Also notice that ∂G k j ∩ C k ∈ C 2 because v δ k
is smooth. Now, take any from the sequence { k } k , and denote S = ∂C as the boundary of C , and

I (x x x) = j | x x x / ∈ G j , j = 1, 2, • • • , 2N
. Define the vector field γ j on each face G j as (4.5) and the directions of reflection by

r r r (x x x) =    j∈I (x x x) c j γ j (x x x) : c i ≥ 0 and j∈I (x x x) c j γ j (x x x) = 1    .
(4.11) When = 0, denote I(x x x) := I 0 (x x x) and r r r(x x x) := r r r 0 (x x x) for the index set and reflection cone of region C N , respectively. Then define the normal direction on face G j as n j (j = 1, 2, • • • , 2N ) with

n i = - ∇(∂ x i v δ ) ∇(∂ x i v δ ) 2 , n i+N = ∇(∂ x i v δ ) ∇(∂ x i v δ ) 2 , i = 1, 2, • • • , N.
Note that the normal direction n j (j = 1, 2, • • • , 2N ) is well-defined by the construction of (4.10).

Next we want to show that

n i •γ i = ∂ 2 x i v δ ∇(∂ x i v δ ) 2 > 0 and n i+N •γ i+N = ∂ 2 x i v δ ∇(∂ x i v δ ) 2 > 0 for i = 1, 2, • • • , N . To do so, we want to show that B δ (x x x) ∈ C N for x x x ∈ S . Note that (-1 + 2 )L i K + i ≤ ∂ x i v(x x x) ≤ (1 -2 )L i K - i for x x x ∈ C . For any y y y ∈ B δ (x x x), |∂ x i v(x x x) -∂ x i v(y y y)| ≤ K x x x -y y y ≤ Kδ ≤ K min . Therefore, (-1+ )L i K + i ≤ (-1+2 )L i K + i -K min ≤ ∂ x i v(y y y) ≤ (1-2 )L i K - i + K min ≤ (1-)L i K - i .
Thus, y y y ∈ C N for all y y y ∈ B δ (x x x) and x x x ∈ S . Moreover, under Assumption A4, ∂ 2

x i v δ (x x x) = y y y∈B δ (x x x) ∂ 2 x i v(x x x)φ δ (x x x -y y y)dy y y > 0 for all x x x ∈ S .

Furthermore, at each point x x x ∈ S , there exists γ ∈ r r r (x x x) pointing into C . This is because there is no x x x ∈ ∂C such that i, i + N ∈ I (x x x) for all i = 1, 2, • • • , N . This implies |I (x x x)| ≤ N for all x x x ∈ ∂C . Now Assumption A4 implies the following condition (3.8) in [START_REF] Dupuis | SDEs with oblique reflection on nonsmooth domains[END_REF]): the existence of scalars b j ≥ 0 j ∈ I (x x x), such that

b j γ j (x x x), n j (x x x) > k∈I (x x x)\{i} b k | γ k (x x x), n k (x x x) | .
Here we can simply take b j = 1 for all j ∈ I (x x x). Therefore, by (Dupuis & Ishii, 1993, Theorem 4.8 and Corollary 5.2), there exists a unique strong solution to the Skorokhod problem with data ({x x x + µ µ µt + σ σ σB t } t≥0 , C , r r r , x x x).

Step 2. -optimal policy. Now we shall show that the solution to the Skorokhod problem with data(x x x + µ µ µt + σ σ σB B B t , C , r r r , x x x) is an -optimal policy of the control problem (Regulator) with

ξ ξ ξ t = t 0 N N N s • dη s , (4.12)
and N N N s ∈ r r r (X X X s ) on S , with X X X t = x x x + µ µ µt + σ σ σB B B t + ξ ξ ξ t . By (Dupuis & Ishii, 1993, Theorem 4.8), X X X is a continuous process. Since v ∈ C 4,α (C N ), applying the Itô formula to the continuous semi-martingale yields

v(x x x) = E x x x ∞ 0 e -ρt [H(X X X t )dt + ∇v(X X X t ) • N N N t dη t ] ≥ E x x x ∞ 0 e -ρt H(X X X t )dt + (1 -3 ) (N N N t ) + • K + L K + L K + L + (N N N t ) -• K + L K + L K + L dη t = E x x x ∞ 0 e -ρt H(X X X t )dt + (N N N t ) + • K + L K + L K + L + (N N N t ) -• K + L K + L K + L dη t (4.13) -3 E x x x ∞ 0 e -ρt (N N N t ) + • K + L K + L K + L + (N N N t ) -• K + L K + L K + L dη t ≥ E x x x ∞ 0 e -ρt H(X X X t )dt + (N N N t ) + • K + L K + L K + L + (N N N t ) -• K + L K + L K + L dη t -3 K max E x x x ∞ 0 e -ρt dη t ,
where N N N (x x x) ∈ r r r (x x x) on S ,

K + L K + L K + L := (L 1 K + 1 , • • • , L N K + N ), K - L K - L K - L := (L 1 K - 1 , • • • , L N K - N ), and K max = max 1≤i≤N {L i K + i , L i K - i }. (4.14)
The first inequality of (4.13) holds since ∇v δ -∇v L 1 < K min for δ ∈ [0, δ( )] and (4.10). Moreover, there exists constant C > 0 such that E x x x ∞ 0 e -ρt dη t ≤ C for all < 1 2 . Hence v(x x x) ≥ J(x x x; ξ ξ ξ t ) -3 CK max .

Proposition 9. Assume Assumptions A1-A4 hold, when x x x ∈ C N .The optimal policy acts only on ∂C N , and its reflection direction is in r r r(x x x).

Proof. Take the smooth function φ in (4.8) and the smooth version of value function v in (4.9). Let

H (x x x) = φ * H(x x x). From the HJB Equation (3.4), ρv -Lv ≤ H, β(∇v) ≤ 1 in R N , and
ρv -Lv ≤ H , β(∇v ) ≤ 1 in R N . (4.15)
Letting T > 0 and applying Meyer's version of Itô's formula (Meyer, 1976, Theorem 21) to e -ρt v (x x x) and any semi-martingale {X X X t } t≥0 under admissible control (ξ i,+ , ξ i,-) N i=1 yield

E x x x e -ρt v (X X X T ) = v (x x x) + E x x x T 0 e -ρt (Lv -ρv ) (X X X t )dt + E x x x T 0 e -ρt ∇v (X X X t ) • dξ ξ ξ t + E x x x T 0 0≤t<T e -ρt (v (X X X t ) -v (X X X t-) -∇v • (X X X t )(ξ ξ ξ t -ξ ξ ξ t-)),
with the last term coming from the jumps of X X X t . By (4.15),

E x x x e -ρT v (X X X T ) + E x x x T 0 e -ρt H (X X X t )dt -E x x x T 0 e -ρt ∇v (X X X t ) • dξ ξ ξ t +E x x x T 0 0≤t<T e -ρt (-v (X X X t ) + v (X X X t-) + ∇v (X X X t ) • (ξ ξ ξ t -ξ ξ ξ t-)) ≥ v (x x x). (4.16) Moreover, H , v are bounded uniformly on C N for < 1 because v, ∇v, D 2 v are bounded on B R (0), with C N ⊂ B R-1 (0), thus v → v, ∇v → ∇v, H → H uniformly in C N . Meanwhile, for ∀x x x ∈ C N , v(x x x) = E x x x ∞ 0 e -ρt H(X X X * t )dt + (N N N * t ) + • K + L K + L K + L + (N N N * t ) -• K - L K - L K - L dη * t , (4.17)
where X X X * t = x x x + µ µ µt + σ σ σB B B t + ξ ξ ξ * t with ξ ξ ξ * t := t 0 N N N * s dη * s the optimal control, and

K + L K + L K + L and K - L K - L K - L are defined in (4.14). In particular, E x x x ∞ 0 e -ρt dη * t < ∞, (4.18)
which leads to

E x x x T 0 e -ρt (N N N * t ) + • K + L K + L K + L + (N N N * t ) -• K - L K - L K - L dη * t < ∞.
By the bounded convergence theorem and (4.16),

E x x x e -ρT v(X X X * T ) + E x x x T 0 e -ρt H(X X X * t )dt -E x x x T 0 e -ρt ∇v(X X X * t ) • N N N * t dη t +E x x x T 0 0≤t<T e -ρt -v(X X X * t ) + v (X X X * t-) + ∇v(X X X * t ) • N N N * t (η * t -η * t-) ≥ v(x x x). (4.19)
The last term on the left-hand side is nonpositive because of convexity of v, hence

E x x x e -ρT v(X X X * T ) + E x x x T 0 e -ρt H(X X X * t )dt -E x x x T 0 e -ρt ∇v(X X X * t ) • N N N * t dη * t ≥ v(x x x). Letting T → ∞, by the boundedness of X X X * t , β(∇v) ≤ 1, |N N N * t | = 1
, and (4.18),

E x x x T 0 e -ρt H(X X X * t )dt -E x x x T 0 e -ρt ∇v(X X X * t ) • N N N * t dη * t ≥ v(x x x).
Along with (4.17), we have

0 ≥ E x x x ∞ 0 e -ρt ∇v(X X X * t ) + K + L K + L K + L • (N N N * t ) + dη * t + -∇v(X X X * t ) + K - L K - L K - L • (N N N * t ) -dη * t . Given β(∇v) ≤ 1, we have -K + i ≤ v x i (x x x) ≤ K - i , ∀x ∈ R N and i = 1, 2, • • • , N. Hence 0 ≥ E x x x ∞ 0 e -ρt ∇v(X X X * t ) + K + L K + L K + L • (N N N * t ) + dη * t + -∇v(X X X * t ) + K - L K - L K - L • (N N N * t ) -dη * t ≥ 0.
This implies dη * t = 0 when β(∇v(X X X * t )) < 1 a.e. in t. Also, when dη * t = 0, N N N * t (x x x) ∈ r r r(x x x) for x x x ∈ S a.e. for t ∈ [0, ∞), where the reflection cone r r r(x x x) is defined in (4.6). Now we are ready to establish the main theorem when x x x ∈ C N .

Theorem 10 (Existence and uniqueness of optimal controls). Take x x x ∈ C N and assume A1-A4. Then there exists a unique optimal control ξ ξ ξ * to problem (Regulator), which is a solution to the Skorokhod problem (7) with data (x x x + µ µ µt + σ σ σB B B t , C N , r r r, x x x) such that X X X * t ∈ C N .

Proof. The existence of the optimal control to the control problem (Regulator) follows from an appropriate modification of Theorem 4.5 and Corollary 4.11 in [START_REF] Menaldi | Optimal correction problem of a multidimensional stochastic system[END_REF], as below. First, J(x x x; ξ ξ ξ ) → v(x x x) as → 0, then ξ ξ ξ t (ω) converges under measure m T on ([0, T ] × Ω, B[0, T ] × F) for any T > 0, which equals to the product of Lebesgue measure and P. Furthermore, there exists an unique optimal policy ξ ξ ξ * which is the limit of a subsequence of {ξ ξ ξ } .

If (N N N k , η k ) is a sequence of k -optimal policies for x x x and lim k→∞ k → 0, then one can extract a subsequence k such that (4.20) under Leb × P for almost all (t, ω), where Leb is the Lebesgue measure on [0, ∞). By the analysis in Theorem 8, there exits a sequence of k -optimal policy and k → 0 when k → ∞. Therefore, the optimal control exists. Let

ξ ξ ξ k t = t 0 N N N k s dη k s → ξ ξ ξ * t ,
A = ω | X X X k t (ω) ∈ C
k for all 0 ≤ t < ∞ and all k ≥ 0 , then by definition (4.12), P (A) = 1. Also define

B = ω | X X X k t → X X X t a.e. Leb on [0, ∞) ,
then by (4.20),

P (B) = 1. For all ω ∈ A ∩ B, since C N is closed, X X X t (ω) ∈ C N Leb a.e. on [0, ∞).
Properties (b) and (c) of the Skorokhod problem (Definition 7) follow from Theorem 8 and Proposition 9, respectively. By Assumption A4, for any x x x ∈ ∂C N and γ(x x x) ∈ r r r(x x x), γ(x x x) is not parallel to ∂C N at x x x. Hence, property (a) holds, i.e., the optimal control is continuous.

It remains to show the uniqueness of the optimal control. This is done by a contradiction argument. Suppose that there are two optimal controls {ξ ξ ξ * } t≥0 and {ξ ξ ξ * * } t≥0 such that ξ ξ ξ * = ξ ξ ξ * * almost surely. Let {X X X * t } t≥0 and {X X X * * t } t≥0 be the corresponding trajectories. Let

ξ ξ ξ t = ξ ξ ξ * t +ξ ξ ξ * * t 2 and X X X t = X X X * t +X X X * * t 2
. Then by Assumption A3, v(x x x) -J(x x x; ξ ξ ξ t ) = (J(x x x; ξ ξ ξ * ) + J(x x x; ξ ξ ξ * * )) 2 -J(x; ξ ξ ξ)

≥ E x x x ∞ 0 e -ρt H(X X X * t ) + H(X X X * * t ) 2 -H X X X * t + X X X * * t 2 dt > 0.
Therefore v(x x x) > J(x x x; ξ ξ ξ), which contradicts the optimality of {ξ ξ ξ * t } t≥0 and {ξ ξ ξ * * t } t≥0 . Hence the uniqueness of the optimal control.

Optimal policy for x x x / ∈ C N . When x x x / ∈ C N , the optimal policy is to jump immediately to some point x x x ∈ C N and then follows the optimal policy in C N . We will need the following assumption so that the reflection field of the Skorokhod problem is extendable to the R N plane [START_REF] Dupuis | On Lipschitz continuity of the solution mapping to the Skorokhod problem, with applications[END_REF]. Note that when N = 2, A5 follows directly from Assumptions A1-A3.

A5. There is a map π : R N → C N satisfying π(x x x) = x x x for all x x x ∈ C N and π(x x x) -x x x ∈ r r r(π(x x x)).

This assumption was also adopted in (Dupuis & Ishii, 1991, Assumption 3.1).

Theorem 11. Given A1-A3, and A5. For any x x x / ∈ C N , there exists an optimal policy π such that π(x x x) ∈ ∂C N at time 0 and v(x x x) = v(π(x x x)) + l(x x x -π(x x x)), with l(y y y) = i l i (y i ), where

l i (y i ) = L i K - i y i , if y i ≥ 0, -L i K + i y i , if y i < 0. (4.21)
Combining Theorems 10, 11 and 12 yields the following result which summarizes the structure of the set of Pareto optima:

Theorem 13 (Pareto-optimal policies). Under Assumptions A1-A5, for any set of weights L L L = (L 1 , • • • , L N ) with L i > 0 and N i=1 L i = 1, the unique solution ξ L ξ L ξ L ∈ U N to the regulator's problem (Regulator) yields a Pareto-optimal policy for the game (N-player).

The analytical structure of the continuation region (4.3) and the Pareto-optimal policy suggest the following description: X X X t evolves according to the uncontrolled diffusion process inside the interior of C N and when it hits boundary at a point belonging to ∂G i or ∂G i+N , then bank i will adjust its rate to push it back continuously inside C N . In particular the optimal policies lead to continuous controls ξ i i.e. no impulses are used.

Pareto-optimal policies for interbank lending

Let us now translate these results in the setting of the interbank lending model described in Section 1.1.

Theorem 13 implies that Pareto optima for the interbank lending market may be described in terms of policies of a regulator facing the optimization problem (Regulator) with an aggregate payoff function (3.2) representing a weighted average of payoffs of individual banks. Under a Pareto-optimal policy, the interbank rates may be described as a 'regulated diffusion' in a bounded region C N defined by (4.3). The boundedness of C N implies that from the payoff structure (1.2) emerges an endogenous bound on the interbank rates: the Pareto-optimal policy guides X X X t to remain confined in the bounded region C N , which implies in particular that both the benchmark and the spreads X i remain bounded. In the context of the LIBOR mechanism, this can be seen as the impact of 'trimmed' averaging, which is the origin of the terms K + i , K - i , as explained in Section 1.1: as banks internalize the risk of being 'outliers' in the benchmark fixing, they confine their rates to a bounded region.

The process X X X t diffuses in the interior of C N , following the random shocks banks are subjected to, and is pushed into the interior when it reaches the boundary. More precisely, the boundary ∂C N is composed of 2N 'faces' corresponding to the saturation of the constraints in (4.4). Edges correspond to intersections of two or more faces. When X X X t reaches a point x x x ∈ ∂C N , action is taken by all banks i such that x x x / ∈ G i ∪ G i+N : if x x x / ∈ G i then X i is reduced i.e. dξ i,-> 0 and if x x x / ∈ G i+N then X i is increased i.e. dξ i,+ > 0. When X X X t reaches the interior of such a face, only bank i adjusts its rate in order to continuously push back X X X t to the interior. Similarly, if X X X t reaches an edge, two or more banks need to simultaneously adjust their rates. The rate at which such simultaneous adjustments occur is given by the intersection local time [START_REF] Rosen | Joint continuity of the intersection local times of Markov processes[END_REF] of (X 1 , ..., X N ) on the boundary. Therefore Pareto-optimal policy rarely leads to more than one bank's rate to be adjusted; a simultaneous rate adjustment by several banks is most likely not associated with a Pareto-optimal policy and is thus the signature of a non-optimal behavior by banks.

We also note that, although the admissible controls allow for 'impulses' i.e. discontinuous adjustments of rates, such impulses are not optimal and Pareto-optimal policies correspond to continuously pushing the process to the interior. The only possible discontinuity in Pareto-optimal policies (discussed in Theorem 11) is the case of an initial impulse at t = 0 to bring the initial condition into C N , which we may interpret as the entry of a new bank into the interbank market.

The set of all such Pareto optima is parameterized by the set of allocations L = (L 1 , ..., L N ) with L i > 0 and N i=1 L i = 1. These allocations lead to different outcomes across banks. A natural choice is to take L i proportional to the market share (or loan volume) of bank i; (3.2) then represents an aggregate wealth maximization problem and this policy leads to the same pro-rata cost across banks. As is clear from (3.5), choosing a higher weight L i leads to a tighter control on the rates of bank i.

Explicit solution for N = 2 players

We now study in more detail the structure of the optimal strategies for the case of N = 2. Our analytical results illustrate the difference between Nash equilibria and Pareto optima and demonstrate the impact of regulatory intervention in this game.

5.1 Pareto-optimal solution for N = 2

For the special case of N = 2, we can derive explicitly its Pareto-optimal solution. For ease of exposition, we shall assume the following conditions in the case of N = 2. B1. a 1 = a 2 and L 1 = L 2 . In other words, the regulator allocates equal weights to the banks.

B2. h 1 (x 1 , x 2 ) = h 2 = h 1 (x 1 , x 2 ) = h(x 1 -x 2 ), h ∈ C 3 (R)
is symmetric, and there exist 0 < c < C such that c < h < C, and h is non-decreasing and bounded away from 0.

B3. µ 1 = µ 2 = 0, K + 1 = K - 1 =: K 1 > 0 and K + 2 = K - 2 =: K 2 > 0.
Note that Assumption B2 is more general than Assumptions A1-A3. As a result, we will see in Proposition 16 that the non-action region may not necessarily be bounded and the Pareto-optimal policy for the game may not be unique with fixed weights L 1 = L 2 .

Under Assumption B3, the rates X 1 t and X 2 t are assumed to be

X i t = σ σ σ i • dB B B t + dξ i,+ t -dξ i,- t , with x i 0-= x i , i = 1, 2. (5.1) The regulator's value function v(x 1 , x 2 ) is v(x 1 , x 2 ) = inf (ξ ξ ξ 1 ,ξ ξ ξ 2 )∈U 2 J(x 1 , x 2 , ξ ξ ξ 1 , ξ ξ ξ 2 ) = inf (ξ ξ ξ 1 ,ξ ξ ξ 2 )∈U 2 1 2 J 1 (x 1 , x 2 , ξ ξ ξ 1 , ξ ξ ξ 2 ) + J 2 (x 1 , x 2 , ξ ξ ξ 1 , ξ ξ ξ 2 ) (5.2) = inf (ξ ξ ξ 1 ,ξ ξ ξ 2 )∈U 2 E (x 1 ,x 2 ) ∞ 0 e -ρt h X 1 t -X 2 t dt + K 1 2 dξ 1,+ t + K 1 2 dξ 1,- t + K 2 2 dξ 2,+ t + K 2 2 dξ 2,- t ,
subject to (5.1).

Lemma 14. Assume K 2 < K 1 and B1-B3. Then for any (ξ ξ ξ 1 * , ξ ξ ξ 2 * ) ∈ arg inf (ξ ξ ξ 1 ,ξ ξ ξ 2 )∈U 2 J(x 1 , x 2 , ξ ξ ξ 1 , ξ ξ ξ 2 ), (ξ 1,+ * t , ξ 1,- * t ) = (0, 0) for any t ≥ 0 a.s..

Proof. The statement is proved by contradiction. Assume there exists an optimal policy (ξ ξ ξ

1 * , ξ ξ ξ 2 * ) ∈ arg inf (ξ ξ ξ 1 , ξ ξ ξ 2 )∈U 2 J(x 1 , x 2 , (ξ ξ ξ 1 , ξ ξ ξ 2 )) and t 0 ≥ 0 such that ξ 1 * ,+ t 0 > 0.
Since ξ 1,+ * is a non-decreasing process, we have ξ 1,+ * t > 0 for all t ≥ t 0 . Now construct the following admissible policy (ξ ξ ξ 1 , ξ ξ ξ 2 ) such that, ∀t ≥ 0, (5.3) Then J(x 1 , x 2 , ξ ξ ξ 1 * , ξ ξ ξ 2 * ) -J(x 1 , x 2 , ξ ξ ξ 1 , ξ ξ ξ

2 ) = E (x 1 ,x 2 ) ∞ 0 e -ρt K 1 -K 2 2 dξ 1 * ,+ t > 0,
which contradicts the optimality of the control process (ξ ξ ξ 1 * , ξ ξ ξ 2 * ).

There is a C 2 solution [START_REF] Beneš | Some solvable stochastic control problems[END_REF][START_REF] Karatzas | A class of singular stochastic control problems[END_REF] given by

u(x) =        - σ 2 p 1 ( c 1 ) cosh x √ 2ρ σ 2ρ cosh c 1 √ 2ρ σ + p 1 (x), 0 ≤ x ≤ c 1 , u( c 1 ) + K 2 2 (x -c 1 ), x ≥ c 1 , u(-x),
x < 0,

( 5.19) where c 1 is the unique positive solution to (5.15) and p 1 (x) is defined as in (5.10). The corresponding control of the regulator is a bang-bang type such that (5.13)-(5.14) hold. Furthermore, it is easy to see that v(x 1 , x 2 ) := u(x 1 -x 2 ), with u(x) defined in (5.19), is indeed the value function of problem (5.2).

Next when K 1 = K 2 , ξ 1,+ and ξ 2,-controls Y t in the same direction with the same cost. The same holds for ξ 2,+ or ξ 1,-, hence the Pareto-optimal policy (5.8) and (5.17).

Pareto optimum vs Nash equilibrium

We now use the above analytical results to compare the Pareto-optimal strategies with the Nash equilibrium strategies, whose definition we recall:

Definition 19 (Nash equilibrium). η η η = η 1 , . . . , η N ∈ U N is a Nash equilibrium strategy of the stochastic game (N-Player), if for any i = 1, . . . , N , X X X 0-= x x x, and any (η η η -i , ξ i ) ∈ U N , the following inequality holds, J i (x x x; η η η) ≤ J i x x x; η η η -i , ξ i . v i (x x x) := J i (x x x; η η η) is called the Nash equilibrium value for player i associated with η η η.

Proposition 20 (Pareto-optimal vs Nash equilibrium solutions when N = 2). Assume B1-B3 and K 1 = K 2 = K, then (i) The following controls give a Nash equilibrium policy to game (5.1)-(5.2):

(η 1,+ t , η 1,- t ) = 0, max 0, max 0≤u≤t (x 1 -x 2 ) + (σ σ σ 1 -σ σ σ 2 ) • B B B u + η 2,- u -c 2 , (η 2,+ t , η 2,- t ) = 0, max 0, max with p 1 defined in (5.10). The value functions v 1 and v 2 corresponding to the Nash equilibrium (η η η 1 , η η η 2 ) defined in (5.20) are

v 1 (x 1 , x 2 ) =        v 1 (x 2 -c 2 , x 2 ), x 1 ≤ x 2 -c 2 , - σ 2 p 1 (c 2 ) cosh √ 2ρ σ (x 1 -x 2 ) 2ρ cosh c 2 √ 2ρ σ + p 1 (x 1 -x 2 ), x 2 -c 2 ≤ x 1 ≤ x 2 + c 2 , K(x 1 -x 2 -c 2 ) + v 1 (x 2 + c 2 , x 2 ), x 1 ≥ x 2 + c 2 ,
(5.22) and

v 2 (x 1 , x 2 ) =        v 2 (x 1 , x 1 -c 2 ), x 2 ≤ x 1 -c 2 , - σ 2 p 1 (c 2 ) cosh √ 2ρ σ (x 2 -x 1 ) 2ρ cosh c 2 √ 2ρ σ + p 1 (x 2 -x 1 ), x 1 -c 2 ≤ x 2 ≤ x 1 + c 2 , K(x 2 -x 1 -c 2 ) + v 2 (x 1 , x 1 + c 2 ),
x 2 ≥ x 1 + c 2 ;

(5.23) (ii) c 2 > c 1 , where c 1 is the unique positive solution to (5.9) and c 2 is the unique positive solution to (5.21).

That is, Pareto-optimal policy yields tighter threshold for spread, hence game is more stable than that under the Nash equilibrium policy. (See Figure 1.) Proof. Similar to the derivation in [START_REF] Guo | Stochastic games for fuel follower problem: N versus MFG[END_REF], we have the following Quasi-variational inequalities for the Nash equilibrium of game (5.1) with J 1 and J 2 and K 1 = K 2 = K,

             max ρv i (x 1 , x 2 ) -h(x 1 -x 2 ) -σ 2 2 ∂ 2 x 1 v i (x 1 , x 2 ) + ∂ 2
x 2 v i (x 1 , x 2 ) , ∂ x i v i (x 1 , x 2 ) -K, -∂ x i v i (x 1 , x 2 ) -K = 0, on {(x 1 , x 2 ) : -K < ∂ x j v j (x 1 , x 2 ) < K} , ∂ x i v i (x 1 , x 2 ) = 0, on {(x 1 , x 2 ) : ∂ x j v j (x 1 , x 2 ) = K or ∂ x j v j (x 1 , x 2 ) = -K} , for i = j and 1 ≤ i, j ≤ 2. Moreover, one can show that (5.22)-(5.23) are the solution to (5.24). By verification theorem (Guo & Xu, 2019, Theorem 3), some detailed calculations can verify that (5.22)-(5.23) are the game values associated with the Nash equilibrium policy (5.20). Now we provide the proof for Claim (ii). Define g(x) = σ √ 2ρ tanh

√ 2ρ σ x , g 1 (x) = p 1 (x)-K 2 p 1 (x)
and x) , where p 1 is defined in (5.10). Then g(0) = 0, g (x) > 0 for any x ∈ R + , and lim x→∞ g(x) = σ The function p 1 (x) is negative at x = 0 and increases monotonically to ∞ on R + . Hence there exists an unique positive zero c 0 . Moreover, for any x > c 0 , g 1 (x) = 1 -

g 2 (x) = p 1 (x)-K p 1 (
p 1 (x)
p 1 (x) g 1 (x) ≥ 1. This is because p 1 (x) ≤ 0 for x ≥ 0. We conclude that there exists a unique point c 0 < c 1 < ∞ such that g(c 1 ) = g 1 (c 1 ).

Now apply similar analysis to c 2 , which is the unique solution to g(x) = g 2 (x) such that 0 < c 2 < ∞. Notice that, g 1 (x) -g 2 (x) = K 2p 1 (x) > 0 because p 2 (x) > 0. Hence c 2 > c 1 .

0≤u≤t-

  (x 1 -x 2 ) + (σ σ σ 2 -σ σ σ 1 ) • B B B u + η 1,- u -c 2 ,(5.20)where c 2 > 0 is the unique positive solution to

Figure 1 :

 1 Figure 1: Comparison: Nash and Pareto (K 1 = K 2 ).

  h (x + σB t )dt ≤ C ρ .

Proof. Notice that l(y y y) is convex and

Here we define two linear approximations which correspond to the lower and the upper bounds of the value function v(x x x), respectively. For x x x ∈ C N , define u 1 (x x x) = v(π(x x x)) + ∇v(π(x x x)) • (x x x -π(x x x)), u 2 (x x x) = v(π(x x x)) + l(x x x -π(x x x)).

(4.22)

Then u 2 (x x x) ≥ v(x x x) by the sub-optimality of the policy, and u 1 (x x x) ≤ v(x x x) by convexity. Thus,

(4.23)

We now show u 1 (x x x) = u 2 (x x x). By Assumption A5, u 1 and u 2 in (4.22) can be rewritten as

where d(π(x x x)) ∈ r r r(π(x x x)) and P P P (x x x) = (P 1 , • • • , P N )(x x x), with

Pareto-optimal policy

Pareto-optimal policy for (N-player) can now be constructed from the optimal control for problem (Regulator) according to the following.

Theorem 12. The optimal control of problem (Regulator) yields a Pareto-optimal policy for the game (N-player).

Proof. The proof is straightforward. To see this, take the payoff function J i in (N-player), v(x x x) the value function in (Regulator), and the optimal control ξ ξ ξ * := (ξ ξ ξ 1 * , . . . , ξ ξ ξ N * ), if exists, to problem (Regulator), then for any ξ ξ ξ := (ξ ξ ξ 1 , . . . , ξ ξ ξ N ) ∈ U N and L i , with

where value v(x x x) is reached when player i takes the control ξ ξ ξ i * t (i = 1, 2, . . . , N ). If there is another ξ ξ ξ := (ξ ξ ξ 1 , . . . , ξ ξ ξ N ) ∈ U N and k ∈ {1, . . . , N } such that

then given L i > 0 for all i, there must exists j ∈ {1, . . . , N } such that

Hence the control ξ ξ ξ * is a Pareto-optimal policy by definition.

We now show that solving the control problem (5.1)-( 5.2) is equivalent to the following control problem (5.4)-(5.5) when

(5.4) subject to

(5.5)

Lemma 15 (Equivalence). Assume B1-B3 and

Proof. By Lemma 14, (ξ 1 * ,+ t , ξ 1 * ,- t ) = (0, 0) for any t ≥ 0 a.s.. Therefore, we can consider a smaller class of admissible control set where (ξ 1,+ t , ξ 1,- t ) = (0, 0) ∀t ≥ 0 and ξ ξ ξ 2 ∈ U 1 . Note that with (ξ 1,+ t , ξ 1,- t ) = (0, 0), we have

and

(5.7)

Clearly problem (5.6)-(5.7) is equivalent to the one-dimensional control problem (5.4)-(5.5) with y = x 1 -x 2 . Hence the claim.

Proposition 16 (Pareto-optimal solution when N = 2). Assume B1-B3, then

then the following control yields one Pareto-optimal policy to game (5.1)-(5.2):

where c 1 is the unique solution to

(5.9)

and

(5.10)

x 1 -x 2 < 0.

(5.11) (ii) If K 1 > K 2 then the following control yields a Pareto-optimal policy to game (5.1)-(5.2), ξ ξ ξ 1 * t = (0, 0), and ξ ξ ξ 2 * t = (ξ 2 * ,+ t , ξ 2 * ,- t ) with (5.12)

where c 1 is the unique solution to

and the associated Pareto-optimal value is

x 1 -x 2 < 0.

(5.16)

Remark 17. Note that under B1-B3, the Pareto-optimal policy is no longer unique with fixed L 1 = L 2 = 1 2 . For instance, when K 1 = K 2 = K, the following control yields another Pareto-optimal policy with the same value function defined in (5.11):

(5.17)

Remark 18. According to the Pareto-optimal policy, the optimally controlled dynamics X 1 * t and X 2 * t are such that P( X

This suggests that there should be mechanism, such as 'trimming', to maintain the dispersion of rates within a certain range. In addition, this solution form indicates that it is socially optimal for the more efficient bank (i.e., the one with the lower cost of control) to take the lead in lending rate adjustment. Consequently, the other bank is a free rider in the game.

Proof. First let us prove the case when K 1 > K 2 . By Lemma 15, it is sufficient to focus on the single-agent problem (5.4)-(5.5) with y = x 1 -x 2 . Following the standard analysis [START_REF] Beneš | Some solvable stochastic control problems[END_REF][START_REF] Karatzas | A class of singular stochastic control problems[END_REF], the HJB equation for the one-dimensional control problem follows (5.4)-(5.5) is

(5.18)


Daniele Battista
email: daniele.battista@uniroma1.it

Valeriy Luchnikov
email: valeriy.luchnikov@uha.fr

Paola Nardinocchi
email: paola.nardinocchi@uniroma1.it

Shape-shifting of polymer beams and shells due to oil extraction

Keywords: Shape-morphing, Bilayer beams, Cylindrical shells, Oil extraction

We investigate the morphing of bilayer naturally curved beams and cylinders due to oil extraction from the outer layers. We fabricate bilayer naturally curved beams and cylindrical shells made of PDMS/(PDMS + silicone oil), use the experimental results to validate an explicit formula delivering the change in curvature of the beams, based on the modeling of oil extraction as a bulk contraction. We show as the same model, set up within a 3D context, delivers the morphing of bilayer cylinders and identify potentially interesting results for designing future experiments. In particular, we show as stable states corresponding to the original cylinders can be got in the form of saddle-like shells and cylinders with the axes of principal curvatures switched with respect to the original axes.

Introduction

In the last decades, polydimethylsiloxane (PDMS) has become popular for the rapid prototyping of many devices widely used in manifacturing and biomedical industries [START_REF] Thangawng | An ultra-thin pdms membrane as a bio/micro-nano interface: fabrication and characterization[END_REF]. This is due to the many advantageous characteristics of the material: transparency, biocompatibility, flexibility, high cost-effectiveness and good moldability. [START_REF] Mukhopadhyay | When pdms isn't the best[END_REF] PDMS is a rubber-like material. However, owing to its porous nature and compatibility with organic solvents [START_REF] Lee | Solvent compatibility of poly(dimethylsiloxane)-based microfluidic devices[END_REF], it shows a gel-like behaviour swelling when a solvent diffuses into the polymer and, viceversa, shrinking when it comes out from it. This property, together with the PDMS hydrophobicity, allowed to realize several devices: sponges for water-oil separation [START_REF] Zhao | Durable superhydrophobic/superoleophilic pdms sponges and their applications in selective oil absorption and in plugging oil leakages[END_REF][START_REF] Zhu | Recent progress in fabrication and application of polydimethylsiloxane sponges[END_REF][START_REF] Turco | An innovative, fast and facile soft-template approach for the fabrication of porous pdms for oil-water separation[END_REF], solvent-tunable microlens [START_REF] Lu | Solvent-tunable pdms microlens fabricated by femtosecond laser direct writing[END_REF], wrinkled microelectrodes [START_REF] Ji | Wrinkled microelectrode interface based on oil-pretreated hyperelastic substrate[END_REF], and wireless passive radio frequency identification sensors (RFID) based on PDMS swelling caused by organic vapour solvents [START_REF] Rumens | Swelling of pdms networks in solvent vapours; applications for passive rfid wireless sensors[END_REF]. In functionalized or composite PDMS, deformation can also be induced in response to external stimuli as pH [START_REF] Liu | Constitutive modeling for polymer hydrogels: A new perspective and applications to anisotropic hydrogels in free swelling[END_REF], UV irradiation [START_REF] Kim | Uv-responsive nano-sponge for oil absorption and desorption[END_REF] and magnetic field [START_REF] Stanier | Fabrication and characterisation of short fibre reinforced elastomer composites for bending and twisting magnetic actuation[END_REF]. Some drawbacks may arise when the swelling(shrinking)-induced deformation of PDMS structures is undesired. For example, the modification of PDMS microchannel geometries due to swelling can affect the trajectories of droplets [START_REF] Dangla | Microchannel deformations due to solvent-induced pdms swelling[END_REF]. In some cases, such as after the extraction of silicone uncrosslinked chains [START_REF] Hourlier-Fargette | Extraction of silicone uncrosslinked chains at air-water-polydimethylsiloxane triple lines[END_REF] or after ion beam and electron beam irradiation [START_REF] Liu | Modulating surface stiffness of polydimethylsiloxane (pdms) with kiloelectronvolt ion patterning[END_REF] these undesired deformations are irreversible, too.

In the present paper, we investigate shape-morphing of PDMS-based thin bilayer structures, which is a quite common configuration employing PDMS as both passive [START_REF] Xie | Self-supported crack-free conducting polymer films with stabilized wrinkling patterns and their applications[END_REF] and active layer [START_REF] Jackson | Design and near-infrared actuation of a gold nanorod-polymer microelectromechanical device for on-demand drug delivery[END_REF] (see also [START_REF] Chia | Vapour processed selfrolled poly(dimethylsiloxane) microcapillaries form microfluidic devices with engineered inner surface[END_REF]). Mismatch strains due to expansion(shrinkage) of the active layer can induce a macroscopic bending or surface instabilities, depending on different material and geometrical parameters [START_REF] Sigaeva | Wrinkles and creases in the bending, unbending and eversion of soft sectors[END_REF]. Slender bilayer self-bending soft structures can be used as sensors and actuators in soft robotics, as drug delivery system, or can represent the single element of an architectural material. All these situations require an accurate control of the deformation process and we wish to make some progress towards addressing this question. We present the results of a campaign of experimental tests made on bilayer naturally curved PDMS beams, with the outer layer containing oil. The beams are made to flatten and evert through oil extraction from the outer layer. The final change in curvature of the beam can be explicitly determined starting from the spontaneous deformation shrinking deformation of the outer layer due to oil extraction. The explicit model is borrowed from [START_REF] Battista | Swelling-induced eversion and flattening in naturally curved gel beams[END_REF] and it is shown to be in good agreement with the experimental results. Shape-shifting due to oil extraction in bilayer PDMS cylindrical shells is also presented in terms of a model set within the context of finite elasticity with spontaneous deformations. The model is solved via finite element method and it allows to determine the amount of oil to put in the outer layer of a cylindrical shell of given axis and curvature to get after oil extraction a cylinder with axis orthogonal to the original one, saddle-like shapes and double-curved shapes.

Material and methods

The goal of our experiments is inducing bending in bilayer naturally curved PDMS beams through shrinking of just one of the two layers. The naturally curved beams are obtained by cutting them from a cylindrical bilayer thin structure in the hoop direction just after the end of the curing process described in the following. The preparation of the bilayer cylinders is resumed in the cartoon in figure 1. The procedure followed to realize the two layers had already been used by one of the Authors and exhaustively described in [START_REF] Egunov | Polydimethylsiloxane bilayer films with an embedded spontaneous curvature[END_REF]. Shortly, one layer is made up of polydimethylsiloxane (PDMS) (SYLGARD TM 184) at a 10 : 1 ratio of prepolymer/crosslinker (the passive layer). The second (active) layer is made up of PDMS mixed with silicone oil (Rhodorsil R Oil 47V20) at 10 : x : 1 ratio of prepolymer/oil/crosslinker, with x = 2, 4, 6, 8, 10; during the curing process, as the PDMS reticulates, the silicone oil remains trapped into the polymer matrix.

The external (active) layer of the cylindrical shell is formed by pouring the PDMSoil-curing agent liquid mixture into the space between the hollow cylinder (B) of internal radius 22 mm and the full cylinder (C) of radius 20 mm. After degassing and curing at 100 • C for 1 hour, the cylinder (C) is extracted and another one (D) of smaller radius 18 mm is inserted coaxially to (B). The inner (passive) layer is formed by pouring the PDMS-curing agent liquid mixture into the space between the outer layer just crosslinked and the cylinder (D). It follows a second degassing and heating, at 100 • C for a time sufficient to complete all chemical reactions (about 2 hours). After the mold opening the two parts result glued each other and form a cylindrical bilayer shell with middle radius 20 mm and thickness 4 mm. Already at this stage, it is expected that the oil starts partially diffusing through the inner layer. However, we do not observe relevant effects on the curving of the sample since the curing time is much shorter than the diffusion time. The samples are then immersed in a toluene bath. The complete extraction of the oil is produced in a few hours. The final configuration which the bilayer beam gets is bent with respect to the initial, stable and maintained.

Beam flattening and eversion induced by oil extraction

In the process described above, four main phases can be identified: i) diffusion of oil from the outer (active) to the inner (passive) layer, which takes place even when the layers are still in the mold; ii) swelling of the bilayer in toluene; iii) diffusion of oil from the swollen bilayer into the toluene bath; iv) evaporation of toluene. At the stationary state, corresponding to the complete oil extraction, the PDMS bilayer beam looks like bent, due to the loss of oil from one layer, and swollen, due to the toluene absorption by both layers. The final shape is attained when all the toluene is evaporated and the bilayer is shrinked. Figure 2 shows the final shapes of a set of beams sharing the same initial beam curvature κ c and thickness h but different one from another for the silicone oil content x: the ratio of prepolymer/oil/crosslinker is 10 : x : As observed, oil extraction from the outer layer induce relevant deformations in the naturally curved beam, which produce beam flattening and eversion. In this situation, the well-known linear Timoshenko formula [START_REF] Timoshenko | Analysis of bimetal thermostats[END_REF] fails in describing the change in curvature, as it fails when swelling-induced eversion and flattening in gel beams is studied [START_REF] Battista | Swelling-induced eversion and flattening in naturally curved gel beams[END_REF], because it is valid for small deformations and slightly naturally curved beams. We adopt the explicit nonlinear formula presented in Ref. [START_REF] Battista | Swelling-induced eversion and flattening in naturally curved gel beams[END_REF] to evaluate the curvature κ acquired by the beam at the end of the extraction process. It has been shown to depend on geometrical parameters such as the beam thickness h, the layer thicknesses h i and h e (i=inner layer, e=external layer), and the initial curvature κ c as well as on the shear moduli G i and G e of the two layers, and on the natural longitudinal deformations λ oi and λ oe . These latter are defined as the deformations which would take place in the two layers if they were free one from another. The curvature formula has the following form We assume that λ oi and λ oe are determined by the bulk contraction due to oil extraction from the bilayer PDMS beam. Specifically, as oil is only present in the external layer, we set λ oi = 1, that is, we consider the inner layer as a passive layer. On the other hand, we assume that oil is completely extracted from the external layer and set

2 κ = κ(α, β, Γ, κc , h, λ oe ) , (3.1)
with α = G i /G e , β = h i /h, Γ = λ oe /
λ oe = (1 -f ) 1/3 with f = V o /(V p + V c-l + V o ) , (3.2)
with the oil fraction f = x/(11 + x) defined as the ratio between the amount of oil V o and the total volume of the layer containing the oil, that is, the sum of the volume V p of the prepolymer, the volume V c-l of the cross-linker and V o . It holds: f < 1 and λ oe < 1.

In our experiments, α 1, β = 1/2, Γ = λ oe = (1f ) 1/3 . Moreover, for small initial curvatures κc → 0 we get a simpler formula from equation 3.1 which is valid for almost flat beams:

κ flat = 12λ oe (1 -λ oe )(λ oe + 1) 2 λ 2 oe + 6λ oe + 1 h (λ 4 oe + 3λ 3 oe + 8λ 2 oe + 3λ oe + 1) 2 . ( 3.3)
Equations (3.1) and (3.3) deliver the curvature of the beam axis for a naturally curved and flat beam, respectively. Figure 3 shows that they are in excellent agreement with experimental data.

Shape-shifting of cylindrical shells due to oil extraction

The good agreement between experimental and explicit results for beams, suggested to study the deformations induced by oil extraction in bilayer PDMS cylindrical shells within the context of three-dimensional finite elasticity with distortions (natural deformations). These latter, as for the beams, are induced by oil extraction from the outer layer. The difference between curved beams and cylindrical shell is in the aspect ratio: small aspect ratios identify beams, large aspect ratios identify cylindrical shells.

Similarly to what we did for the beam curvature formulas (3.1) and (3.3), we assume that a bulk contraction corresponds to the complete extraction of oil from the external layer; it determines the stationary configuration of the cylindrical shell we aim to characterize. The reference state B of the 3D continuum is a bilayer cylindrical shell with a principal curvature κ 2 = 0 equal to the curvature κ c of the beam cutted from it in the hoop direction (and κ 1 = 0). Figure 4 shows the reference states of cylindrical shells (left top and center cartoons) and beam (left bottom cartoon). Moreover, denoted with L the length of the beam, as well as the arc length of the cylindrical shells(beam), and with AR the ratio between the width w of the cylindrical shells(beam) and L, it holds AR = 0.1, AR = 0.5 and AR = 1 for the beam and the cylindrical shells in the left cartoons in figure 4. The dynamics of the process is beyond the scope of the communication and we focus on the equilibrium states of the polymer bodies. Introduced as state variable of the problem the displacement field u from B, we denote with x = X + u(X) the position of a point X ∈ B at the steady state attained after oil extraction. We multiplicatively decompose the deformation gradient F = I + ∇u in a pure inelastic deformation (bulk contraction) F o and an elastic deformation F e . We also assume that F o = λ o I with λ o = 1 in the passive layer and λ o < 1 in the active layer. On the other hand, the elastic deformation F e is assumed to be isochoric, that is, such that det F e = 1. Hence, volume changes are only due to the distortion λ o which models the bulk contraction in the active layer due to oil extraction. It can be shown [START_REF] Curatolo | Swelling and growth: a constitutive framework for active solids[END_REF] that the following constitutive equations hold for the (reference) stress S, corresponding to an elastically incompressible neo-Hookean material:

S = Gλ o F -p(det F)F -T , (4.4)
with p the reactive stress maintaining the volumetric constraint det F e = 1. 3We implement within the finite element code COMSOL Multiphysics the mechanical equilibrium equation div S = 0, in a weak form, together with the boundary conditions of zero tractions. Eventually, we assume some boundary conditions on u that eliminate any rigid motions without generating reaction forces. We look for the solutions of the stationary problem corresponding to different values of the deformation λ o , and for different AR (keeping fixed the arc length L). We set L = 35mm, h = 4mm, β = 1/2, G i = G e = 1MPa and fixed the initial curvature κ 2 = κ c = 50m -1 of the cylinder. Figure 4 shows the outcomes of the numerical simulations for AR = 0.1 (bottom row of cartoons corresponding to a curved beam), AR = 0. contrary, the morphing of the shells with AR = 0.5 and AR = 1 shown in figure 4 is noteworthy. Firstly, at λ o = λ o = 0.875, corresponding to f = f = 0.33, we get cylindrical shells with the principal curvature axes switched with respect to the reference state: κ 1 = 0 and signκ 1 = -signκ 2 (see the cylindrical shells within the dashed frame in figure 4, third column). The value λ o which identifies the switched cylinder is the same for any AR. Moreover, changing the initial curvature κ 2 , the value λ o changes even if it is still the same for any AR (see the inset in figure 5). Hence, given the initial curvature, we can evaluate the amount of oil to add to the polymer mixture to produce the shift of the two axes of the cylinder. For λ o < λ o < 1, the beam decreases its curvature and flattens, while the cylindrical shells morph into cylindrical shells with the axes of principal curvatures switched, passing through saddle-like shapes (figure 4, second and third columns from left). For λ o < λ o , that is, f > f , the beam everts, the shell with AR = 0.5 attains a double-curved shape with κ 1 < 0 and κ 2 < 0, and the shell with AR = 1 bends along the direction orthogonal to the initial one (figure 4, fourth and fifth column from left). The morphing described above via the cartoons in figure 4 is also shown in figure 5 through the values taken by the normalized Gaussian curvature K = Kh 2 , with K = κ 1 κ 2 , evaluated at the center of the middle surface in the stationary configuration at different values of λ o from 1 to 0.75, for several AR ranging between 0.1 and 1. For any AR, all the curves cross the zero line in two points: at λ o = 1, which corresponds to the reference state, and at λ o = λ o , corresponding to the cylindrical shells with the axes of principal curvature switched with respect to the reference axes. Interestingly, the value λ o = λ * o corresponds to the value which makes null the curvature of a naturally curved beam (see equation (3.1)). In figure 6, we show how the transition from a given cylindrical shell to the cylindrical shell with switched axes changes when the normalized initial curvature κc = κ c h of the shell decreases. The normalized principal curvatures κ 1 h (blue lines) and κ 2 h (orange lines) are evaluated at the center of the middle surface in the stationary configuration at different values of λ o from 1 to 0.85, for κc = 0.2, 0.1, 0.05 and keeping fixed AR = 1. It is observed that the range of λ o values which allows to get a saddle-like shape is smaller and smaller as the cylindrical shell is thinner and thinner, and the green band in figure 6 is narrow and narrow. These observations allow us to make a precise design of the recipe used to realize the PDMS layer with oil inside, once fixed the thickness and the curvature of the cylinder. 6: Normalized principal curvatures κ1h (blue lines) and κ2h (orange lines) versus the bulk contraction λo varying from 1 to 0.85, for κc = 0.2 (top plot), κc = 0.1 (center plot), κc = 0.05 (bottom plot), and keeping fixed AR = 1. The green area identifies, for each value of κc, the range of λo for which a saddle-like shape is attained.

Conclusions

We investigated the shape-shifting of bilayer polymer beams and cylindrical shells due to oil extraction. Polymer beams are made to evert through oil extraction from the outer layer. The morphing mechanism is also studied through an explicit model which accurately describes the observed morphing. The good agreement between experiments and modeling in beams inspired the computational study of cylindrical shells under the same working conditions, which will be supported in the next future by the appropriate experimental tests and also reviewed in view of the recent study presented in Ref. [START_REF] Battista | Enforcing shaping of thin gel sheets by anisotropic swelling[END_REF].

Figure 1 :

 1 Figure1: Top: bilayer cylinder fabrication. The outer layer is formed by pouring the PDMS-oil-curing agent liquid mixture into the space between the hollow cylinder (B) and the full cylinder (C). After degassing and curing, the cylinder (C) is removed and (D) is inserted coaxially to (B). The inner layer is formed by pouring the PDMS-curing agent liquid mixture into the space between the outer layer just crosslinked and the cylinder (D). Bottom: some shapes made by the same bilayer structure fabricated by using the same kind of procedure.

  1 and x = 2, 4, 6, 8, 10. It is shown that there are two values of x which determine, after oil extraction, beam flattening (x = 6) and beam eversion (x = 10).

Figure 2 :

 2 Figure 2: Bilayer naturally curved beams after oil extration for several values of the oil fraction. The initial configuration has a curvature κc = 50 m -1 , thickness h = 4 mm. The oil is in the outer layer.

  λ oi and κc = κ c h.

Figure 3 :

 3 Figure 3: Normalized curvature κ = κh versus oil fraction f for a naturally straight beam and a naturally curved beam with silicone oil in the outer layer. The blue solid lines represent the 1D reduced model; the red triangles represent the experimental results.

Figure 4 :

 4 Figure 4: Morphing of bilayer naturally curved beam (bottom) and cylindrical shells (center and top) due to a bulk contraction λo in the outer layer. The corresponding values of the oil fraction increase from left to right. The initial curvature κ2 = κc can be find by looking at the f = 0 column. At λo = λ o = 0.875(f = f = 0.33) (third column) the Gaussian curvature K vanishes.

Figure 5

 5 also shows as λ o identifies the transition from saddle-like shapes and shapes with positive Gaussian curvature. Indeed, it can be observed that for λ o > λ * o , we have K < 0 for all the aspect ratios (saddle-like shape); for λ o < λ * o , we have K > 0 for AR = 0.1, 0.4, 0.7, and K 0 for AR = 1.

Figure 5 :

 5 Figure5: Normalized Gaussian curvature K = Kh 2 versus the bulk contraction λo varying from 1 to 0.75, for aspect ratios AR = 0.1, 0.4, 0.7, 1 (green, bleu, orange, red lines). The inset shows the dependence of the λ o on the initial dimensionless curvature κ2 of the cylinder; the pattern corresponds to L = 35mm, h = 4mm, β = 1/2, Gi = Ge = 1MPa.

Figure

  Figure6: Normalized principal curvatures κ1h (blue lines) and κ2h (orange lines) versus the bulk contraction λo varying from 1 to 0.85, for κc = 0.2 (top plot), κc = 0.1 (center plot), κc = 0.05 (bottom plot), and keeping fixed AR = 1. The green area identifies, for each value of κc, the range of λo for which a saddle-like shape is attained.

See the appendix of Ref.[START_REF] Battista | Swelling-induced eversion and flattening in naturally curved gel beams[END_REF] for the explicit formula

It is worth noting that, due to the elastic incompressibility constraint, det F = λ 3 o .

Acknowledgments

DB and PN would like to thank MIUR (Italian Minister for Education, Research, and University) and the PRIN 2017, Mathematics of active materials: From mechanobiology to smart devices, project n. 2017KL4EF3, for financial support.

Conflicts of interest

There are no conflicts to declare.