Aneel Tanwani
email: aneel.tanwani@laas.fr.

Quanyan Zhu
email: quanyan.zhu@nyu.edu.

Feedback Nash Equilibrium for Randomly Switching Differential-Algebraic Games

Keywords: Noncooperative games, differential-algebraic dynamical systems, piecewise deterministic Markov processes, infinitesimal generators, generalized HJB equation, coupled Riccati equations, minimax robust control, Leontief input-output models

As a subclass of stochastic differential games with algebraic constraints, this article studies dynamic noncooperative games where the dynamics are described by Markov jump differential-algebraic equations (DAEs). Theoretical tools, which require computing the extended generator and deriving Hamilton-Jacobi-Bellman (HJB) equation for Markov jump DAEs, are developed. These fundamental results lead to pure feedback optimal strategies to compute the Nash equilibrium in noncooperative setting. In case of quadratic cost and linear dynamics, these strategies are obtained by solving coupled Riccatilike differential equations. Under an appropriate stabilizability assumption on system dynamics, these differential equations reduce to coupled algebraic Riccati equations when the cost functionals are considered over infinite-horizon. As a first casestudy, the application of our results is studied in the context of an economic system where different suppliers aim to maximize their profits subject to the market demands and fluctuations in operating conditions. The second case-study refers to the conventional problem of robust control for randomly switching linear DAEs, which can be formulated as a two-player zero sum game and is solved using the results developed in this paper.

I. INTRODUCTION

Noncooperative dynamic games provide a framework for interaction of several decision makers, or players, competing against each other for maximizing or minimizing their respective objective functions. The reader may consult the books [START_REF]Handbook of Dynamic Game Theory[END_REF], [START_REF] Haurie | Games and Dynamic Games[END_REF] to get an overview of the extensive work done in this field, and its relevance to control theoretic problems. Different from the conventional setup of dynamic games, many practical applications require us to incorporate algebraic constraints on the state trajectories and the action variables (control inputs). For example, in electric power systems, multiple generators are constrained by algebraic power flow constraints arising from Kirchhoff's circuit laws. Similarly, in economic systems, interdependencies between different industries or regions within an economy are captured using algebraic Leontief input-output models [START_REF] Leontief | Studies in the Structure of the American Economy: Theoretical and Empirical Explorations in Input-Output Analysis[END_REF], [START_REF] Luenberger | Singular dynamic Leontief systems[END_REF]. These examples provide the motivation to study dynamic noncooperative games where the evolution of the states is described by differential-algebraic equations (DAEs), which comprise ordinary differential equations (ODEs) and algebraic equations. In addition, we allow the differential and algebraic constraints to change randomly with time, and this switching in system dynamics is governed by a finite state Markov chain, giving rise to piecewisedeterministic Markov DAEs. The switching can represent the structural changes in the system dynamics and constraints for several applications of interest, such as power systems [START_REF] Groß | Solvability and stability of a power system dae model[END_REF].

This article investigates decentralized decision-making and control for the aforementioned class of dynamical systems. Each player designs an optimal controller to minimize its cost, which is defined by taking the expected value of a continuous functional over all sample paths resulting from randomness in switching. In particular, we establish conditions for the existence of Nash equilibrium using pure feedback strategies for individual players, and provide algorithms for computing the optimal strategies. It will be assumed that the players are aware of the system dynamics, and for each sample path, the knowledge of the switching signal is available for computing the feedback strategy. Applications of our theoretical results are proposed in the context of robust control of Markov jump DAEs where the objective is to design a control input which attenuates the effect of disturbance in the dynamics; and secondly, we consider economic systems described by Leontief input-output model where different suppliers aim to maximize their profits subject to a given demand of goods.

Noncooperative games with continuous kernels have been extensively studied when the formulation of the cost function involves deterministic ordinary differential equations, see [START_REF] Bas | Dynamic Noncooperative Game Theory[END_REF] for an overview. The games where the parameters of the differential equations may change according to some Markov transition matrix appear in [START_REF] Bas | Feedback equilibria in differential games with structural and modal uncertainties[END_REF], [START_REF] Rishel | Control of systems with jump Markov disturbances[END_REF], and an application of frameworks in modern control systems can be seen in [START_REF] Zhu | Game-theoretic methods for robustness, security, and resilience of cyber-physical systems: Games-in-games principle for optimal cross-layer resilient control systems[END_REF]. However, games comprising differential equations with equality constraints [START_REF] Engwarda | Feedback Nash equilibria for linear quadratic descriptor differential games[END_REF], [START_REF] Engwerda | The open-loop linear quadratic differential game for index one descriptor systems[END_REF], or with inequality constraints [START_REF] Reddy | Feedback properties of descriptor systems using matrix projectors and applications to descriptor differential games[END_REF], [START_REF] Reddy | Feedback Nash equilibria in linear-quadratic difference games with constraints[END_REF], have only attracted attention recently. The interest in studying games with constraints is also visible in some recent works on finite-stage games (without differential equations), where the dynamic nature arises from the timevarying information patterns available to the players [START_REF] Singh | Linear quadratic game of exploitation of common renewable resources with inherent constraints[END_REF], [START_REF] Singh | Discontinuous Nash equilibria in a two stage linear-quadratic dynamic game with linear constraints[END_REF]. This paper addresses an N -player nonzero sum game in the presence of differential-algebraic equations (DAEs), where the transitions in the system parameters may occur according to a Markov process. Thus, the main focus of this paper is to solve multi-player game-theoretic problems using feedback strategies, where the dynamical systems are described by randomly switching DAEs.

When studying stochastic games with jump linear systems [START_REF] Bas | Feedback equilibria in differential games with structural and modal uncertainties[END_REF], [START_REF] Pan | H ∞ -control of Markovian jump systems and solutions to associated piecewise-deterministic differential games[END_REF], one needs several tools such as infinitesimal generator for the corresponding cost functionals, which for the ODE case first appeared in [START_REF] Rishel | Control of systems with jump Markov disturbances[END_REF]. On the other hand, for switched DAEs, we have seen Lyapunov function based techniques in [START_REF] Liberzon | Switched nonlinear differential algebraic equations: Solution theory, Lyapunov functions, and stability[END_REF]. Since Lyapunov functions have not been studied for stochastic DAEs, we first provide a rigorous derivation of the infinitesimal generator associated with a real-valued functional. We then use this result in the context of dynamic programming to derive the Hamilton-Jacobi-Bellman equation associated with Markov jump DAEs under consideration. These statements lay the foundation for us to compute the optimal strategies of the individual players in the game setting. In the general case, with nonlinear dynamics, these strategies rely on solving coupled partial differential equations. For the case of linear dynamics and quadratic cost functionals, these strategies can be obtained by solving coupled generalized Riccati differential equations. When studying the infinitehorizon case, the solution of these differential equations are shown to converge to a stationary solution under a stabilizability condition on system dynamics; consequently, the optimal strategies for infinite-horizon case can be computed by solving generalized coupled algebraic Riccati equations.

As an application of our results, we design robust H ∞ controllers for Markov jump DAEs subject to some disturbances in the dynamics. This is done by studying the two-player zero-sum game where the performance index is the expected value of a quadratic function of the state, input, and the disturbance. One can also interpret DAEs as the limiting case of singularly perturbed systems where the fast dynamics are replaced by an algebraic equation; some results on H ∞ -control for nonlinear singularly perturbed systems can be found in [START_REF] Fridman | H ∞ -control of nonlinear singularly perturbed systems and invariant manifolds[END_REF]. An overview of robust control problems for DAEs in deterministic setting without switching can be found in [START_REF] Kunkel | Optimal control for unstructured nonlinear differential-algebraic equations of arbitrary index[END_REF], [START_REF] Kurina | Feedback solutions of optimal control problems with DAE constraints[END_REF], [START_REF] Mehrmann | The Autonomous Linear Quadratic Control Problem, Theory and Numerical Solution[END_REF], [START_REF] Voigt | On Linear Quadratic Optimal Control and Robustness of Differential-Algebraic Systems[END_REF], and this article generalizes those results when the dynamics undergo switching driven by a Markov process.

Another application of our results lies in the study of economic systems for a certain class of models which relate the production of different industries in the national economy. In the seminal work [22, Part I] carried out by W. Leontief,1 static and dynamic models are proposed which relate the production of different industries with one another and the given demand of goods determined by the market and consumers. Switched DAEs provide one possible framework to unify the static and dynamic models proposed by Leontief, and the random changes in the system matrices allow us to capture the structural changes in the economy which could result from unknown factors such as climate change, varying natural resources, or unpredictable changes in labor resources. In the game setting, the problem of interest that can be addressed in such models is the design of optimal strategies for the supply functions of individual industries.

A preliminary version of some of the results appearing in this paper have been presented in the conference paper [START_REF] Tanwani | Feedback Nash equilibrium for Markov jump games under differential-algebraic constraints with application to robust control[END_REF]. While the paper [START_REF] Tanwani | Feedback Nash equilibrium for Markov jump games under differential-algebraic constraints with application to robust control[END_REF] focused on linear dynamics, this paper carries out a more general treatment with nonlinear dynamics. We also provide detailed proofs of the theoretical tools required to solve the problem, and relax some structural assumptions on the system dynamics.

II. SYSTEM CLASS AND PROBLEM FORMULATION

The multi-player differential games considered in this paper have two central elements: first is the cost functional which is to be minimized by each player, and secondly, the controlled dynamics and algebraic constraints subject to which each functional is to be minimized. In this section, we provide a short description of the stochastic DAEs which will be used to describe the constraints in the differential games. Then, we formulate the game problem and describe the cost functionals and strategies for which we propose a solution.

A. System Class

For the problems studied in this paper, the dynamics of the state are described by the following differential-algebraic equation:

E σ (x) ẋ = f σ (t, x, u), x(0) = x 0 ∈ R n , (1)
where σ is a continuous-time finite-state Markov chain defined on the state space S := {1, 2, • • • , S}, characterized by the rate transition matrix Λ := {λ ij } ∈ R S×S , for (i, j) ∈ S × S.

The transition rates are such that

λ ij ≥ 0, j = i (2a) j∈S λ ij = 0, ∀ i ∈ S, (2b)
where the later condition is true only if λ ii ≤ 0, for each i ∈ S. Let (Ω, F, P) denote the underlying probability space2 , then by definition, the stochastic process σ satisfies

P σ(t + h) = j σ(t) = i = λ ij h + o(h) (3a)
P σ(t + h) = i σ(t) = i = 1 + λ ii h + o(h) (3b)
for h > 0 sufficiently small, and o(h) is independent of t satisfying lim h↓0 o(h) h = 0. The sample paths of σ are rightcontinuous and piecewise constant; Its value at a given time determines the following active subsystem modeled as a DAE:

E i (x) ẋ = f i (t, x, u), i ∈ S (4)
between two consecutive jumps of σ. The matrix E i (x) ∈ R n×n may be singular, and hence DAEs provide a generalization of ODEs with algebraic constraints involving states and inputs embedded in the formulation of the dynamical equation.

B. Game Formulation

We are interested in a class of Nperson continuous-time dynamic games, or differential games, wherein the evolution of the state is described by the differential equation

E σ (x) ẋ = f σ (t, x, u 1 , • • • , u N ), x(0) = x 0 , (5)
with σ, as earlier, describing a Markov process on S. The control action of player p is denoted by

u p ∈ U p ⊆ R mp . The input of each player p ∈ N := {1, 2, • • • , N } is generated by a strategy µ p so that 3 u p (t) := µ p (t, x(t), σ(t)) (6)
where µ p : R × R n × S → U p is measurable in t, locally Lipschitz continuous in x, continuous in σ, and satisfies the causality condition. Thus, the class of strategies we consider for each player are memoryless, and require full information of the state at current time, and also the information about currently active mode through σ(t). Let Σ p denote this admissible class of strategies of player p.

The objective of the player p is to choose a strategy µ p that minimizes the following finite-horizon performance index:

J p (µ 1 , . . . , µ N ) = E c p,T (x(T ); σ(T )) + E T t0 c p (s, x(s), u 1 (s), . . . , u N (s); σ(s)) ds (7)
where we fix u p (s) = µ p (s, x(s), σ(s)). We stipulate that the terminal cost c p,T , and the running cost c p , are jointly continuous in its arguments for each p ∈ {1, . . . , N }.

Definition 1 (Nash equilibrium [START_REF] Nash | Equilibrium points in N -person games[END_REF]). An N -tuple of strategies {µ * p | p ∈ N } constitutes a feedback Nash equilibrium if, and only if, the following inequalities hold for all µ p ∈ Σ p , p ∈ N ,

J * p := J p (µ * 1 , . . . , µ * p-1 , µ * p , µ * p+1 , . . . , µ * N ) ≤ J p (µ * 1 , . . . , µ * p-1 , µ p , µ * p+1 , . . . , µ * N ). (8)
The fundamental problem addressed in this paper relates to finding the Nash equilibrium for the N -player game, where a player p minimizes the functional J p in [START_REF] Chatterjee | Stabilizing randomly switched systems[END_REF] subject to the stochastic dynamic constraints [START_REF]Handbook of Dynamic Game Theory[END_REF]. To address this problem, we study solutions of system (1), and use them to derive the tools that allow us to compute the optimal strategies that constitute the Nash equilibrium.

III. BASIC HYPOTHESIS AND SOLUTIONS

Within the literature on stochastic systems, the system class (1) basically falls under the framework of piecewisedeterministic Markov processes, introduced in [START_REF] Davis | Piecewise-deterministic Markov processes: A general class of non-diffusion stochastic models[END_REF]. The holding time between two jumps of σ is determined by a random variable with exponential distribution, and in between these jumps, the system evolves according to a deterministic DAE. However, in the conventional setup of [START_REF] Davis | Piecewise-deterministic Markov processes: A general class of non-diffusion stochastic models[END_REF], only the evolution due to deterministic ODEs is considered. In the sequel, we first describe the hypotheses on each subsystem which allow us to construct the solution for a fixed mode, and then use it to describe solutions for the stochastic process with Markov jumps.

A. Nonlinear time-invariant DAEs

When focusing on the individual DAE (4) described by the pair (E i , f i ) with i ∈ S fixed, there are several technical issues to be considered when talking about their solution, see [START_REF] Kunkel | Differential-Algebraic Equations: Analysis and Numerical Solution[END_REF], [START_REF]Surveys in Differential-Algebraic Equations I[END_REF], and [START_REF] Trenn | Switched differential algebraic equations[END_REF]. Here, we focus our attention to the case where each subsystem (E i , f i ) describes an index-one 4 DAE. This assumption allows us to define the algebraic constraints explicitly as formalized below: (A1) For each i ∈ S, it holds that the pair (E i , f i ) describes a regular index-one DAE, that is, there exist diffeomorphisms S i , T i : R n → R n such that, with

z := (z 1 i , z 2 i ) := T -1 i (x), z 1 i ∈ R νi , z 2 i ∈ R n-νi and ν i := rank(E i (x)) for each x ∈ R n , it holds that for every (t, z i , u) ∈ R × R n × R m , S i E i (T i (z)) ∂T i (z) ∂z ω = I 0 0 0 ω, ∀ ω ∈ R n S i (f i (t, T i (z), u)) = f 1 i (t, z 1 i , z 2 i , u) f 2 i (t, z 1 i , z 2 i , u) . (9) Moreover, the function f 2 i (t, z 1 , •, u) has the property that rank ∂f 2 i (t, z 1 i , z 2 i , u) ∂z 2 i = n -ν i , (10)
and there exists a map

g i : R × R νi × R m → R n-νi , so that f 2 i (t, z 1 i , z 2 i , u) = 0 implies z 2 i = g i (t, z 1 i , u).
It is thus seen that (A1) allows us to rewrite each subsystem as a semi-explicit DAE with index-one, also called Hessenberg form with index-one [START_REF] Kunkel | Differential-Algebraic Equations: Analysis and Numerical Solution[END_REF]Chapter 4]. The rank condition in [START_REF] Davis | Piecewise-deterministic Markov processes: A general class of non-diffusion stochastic models[END_REF] allows us to invoke the implicit function theorem and solve the equation f 2 i (t, z 1 i , z 2 i , u) = 0 for z 2 i , and the resulting map (t, z 1 i , u) → z 2 i is denoted by g i . Because of this structure resulting from (A1), we can introduce the consistency manifold for each pair (E i , f i ), denoted by C i . To do so, we let

T i := T -1 i , so that z 1 i z 2 i = T 1 i (x) T 2 i (x)
and the system in z-coordinates satisfies the following equations

I 0 0 0 ż1 i ż2 i = f 1 i (t, z 1 i , z 2 i , u) f 2 i (t, z 1 i , z 2 i , u) . ( 11
)
As a result, we define the consistency set as

C i (t, u) := x ∈ R n f 2 i t, T 1 i (x), T 2 i (x), u = 0 .
We say that u ∈ dom(C(t, •)) whenever C(t, u) is not an empty set. As a consequence of index-one assumption imposed on the system structure, one can essentially solve the algebraic constraint and get an ODE in the variable z 1 i . More precisely, if a subsystem i ∈ S is active on an interval [t 0 , t 1 ), with x(t 0 ) ∈ C i (t 0 , u(t 0 )), then the state-trajectories evolve in a manner such that x(t) ∈ C i (t, u(t)), for each t ∈ [t 0 , t 1 ).

When the initial condition x(t - 0 ) is not consistent with C(t 0 , u(t 0 )), there is a jump in the value of the state such that x(t + 0 ) ∈ C(t + 0 , u(t + 0 )). Thus, we introduce a mapping Π i which maps possibly inconsistent initial conditions to the consistency manifold of the active subsystem:

x → Π i (t, x, u) := T i T 1 i (x) g i t, T 1 i (x), u . (12)
To guarantee existence of solutions for a fixed subsystem starting with a consistent initial condition, we introduce the following assumption:

(A2) For each i ∈ S, the mapping g i is continuous and

g i (t, •, u) is continuously differentiable for each (t, u) ∈ R × R m .
Also, for each admissible feedback control input of the form (6) such that, for each initial condition z 1 i ∈ R νi , the solution to the ODE

ż1 i = f 1 i (t, z 1 i , g i (t, z 1 i , u), u), z 1 i (t 0 ) = z 1 i
is a uniquely defined absolutely continuous function over [t 0 , t 1 ), for every t 1 > t 0 , and has no finite escape times.

With assumptions (A1) and (A2), the system (4) is solvable for each sample path of σ. The following proposition describes the solutions of the system when the switching signal is described by a random process.

Proposition 1. Consider system (1) in which σ is a Markov process, and for each i ∈ S, assumptions (A1) and (A2) hold.

For each x(0) = x 0 ∈ R n , the process (x(t), σ(t)) t≥0 is a Markov process. Along each sample path of this process, if t j , j ∈ N denotes the time instants at which σ changes its value, then x is a piecewise absolutely continuous function that satisfies, for each j ∈ N,

E σ(t) (x(t)) ẋ(t) = f σ(t) (t, x(t), u(t)), t ∈ (t j , t j+1 ), (13a) x(t j ) = Π σ(tj ) (t j , x(t - j ), u(t j )) ∈ C σ(tj ) (t j , u(t j )). (13b)
In other words, along a sample path of the stochastic process (1), σ only changes its value at time instants t j , j ∈ N which are determined randomly by the underlying probability distribution. The state trajectory x evolves as a piecewise absolutely continuous function so that x(t) ∈ C σ(tj ) (t, u(t)), for t ∈ [t j , t j+1 ), and is obtained by integrating the equation (13a). Then at t = t j+1 , when σ changes its value, so that σ(t j+1 ) = σ(t j ), there is a possible jump in x trajectory and x(t j+1 ) = Π σ(tj+1) (t j+1 , x(t - j+1 ), u(t j+1 )) ∈ C σ(tj+1) (t j+1 , u(t j+1 )). Again, the evolution of x over the interval [t j+1 , t j+2 ) is described by integrating (13a) and x(t) ∈ C σ(tj+1) (t, u(t)), for t ∈ [t j+1 , t j+2 ); the process continues like that.

B. The linear regular case

Let us apply the foregoing discussion to the linear systems. In this case, the system dynamics are described as:

E σ ẋ = A σ x + B σ u, ( 14
)
where σ is a continuous-time finite-state Markov process over the finite set S, and E i , A i ∈ R n×n , and B i ∈ R n×m for each i ∈ S. For such linear systems, the assumptions (A1) and (A2) hold if the matrix pair (E i , A i ) is regular and index-one. The pair (E i , A i ) is regular, if there exists λ ∈ C such that the complex matrix λE i -A i is nonsingular.

Proposition 2. Assume that the pair (E i , A i ) describes a regular DAE, for each i ∈ S. Then there exist invertible matrices S i , T i such that

S i E i T i = I νi×νi 0 0 N (n-νi)×(n-νi) , (15a)
S i A i T i = J νi×νi 0 0 I (n-νi)×(n-νi) , (15b)
where J νi×νi ∈ R νi×νi is a matrix in block Jordan form and

N (n-νi)×(n-νi) is a nilpotent matrix; the smallest κ ∈ N, for which N κ (n-νi)×(n-νi) = 0, is called the index of the pair (E i , A i ). In particular, if the pair (E i , A i ) has index-one, then N (n-νi)×(n-νi) is a zero matrix. The mapping Π i (t, x, u) is defined by x → Π i (t, x, u) = Π i,aut x + Π i,in u (16)
where Π i,aut :=

T i I νi×νi 0 0 0 T -1 i , Π i,in := T i 0 -B i , and
B i denotes the last n -ν i rows of the matrix S i B i .
The matrices S i , T i in Proposition 2 transform the system [START_REF] Groß | Solvability and stability of a power system dae model[END_REF] in the so-called quasi-Weierstraß form [START_REF] Berger | The quasi-Weierstraß form for regular matrix pencils[END_REF], [START_REF] Wong | The eigenvalue problem λT x+Sx[END_REF]. In the context of switched DAEs with deterministic transitions, the transformations introduced in Proposition 2 have been used for stability analysis [START_REF] Liberzon | Switched nonlinear differential algebraic equations: Solution theory, Lyapunov functions, and stability[END_REF], and solving observability related problems [START_REF] Tanwani | Determinability and state estimation for switched differential-algebraic equations[END_REF]. In the remainder of the paper, (A1) and (A2) will be considered as standing assumptions so that the solutions of system (1) are well-defined, either in open-loop or closed-loop.

IV. GENERATOR AND STABILITY CONDITIONS

Using the fundamental development on existence of solutions from previous section, we now develop two further results for stochastic DAEs (1). The first one concerns computation of infinitesimal generator for the class of DAEs (1) with stochastic Lyapunov function. This result is used in deriving an analogue of Hamilton-Jacobi-Bellman equation for an optimization problem with constraints described by a Markov jump DAE.

A. Infinitesimal Generator

In the context of this paper, the notion of infinitesimal generator generalizes the notion of differential operator when the system dynamics are described by a stochastic process.

Definition 2 (Infinitesimal Generator). The infinitesimal generator of the joint process x(t), σ(t) t≥0 is the linear operator V → LV , defined as

(t, z, i) → LV (t, z, i) := lim h↓0 1 h -V (t, z, i) + E V t + h, x(t + h), σ(t + h) x(t) = z, σ(t) = i (17)
for all maps V : R×R n ×S → R such that the limit is defined everywhere.

We are interested in computing the infinitesimal generator for the systems described by

E σ (x) ẋ = f σ (t, x, u) (18)
where σ is a Markov jump process with right-continuous sample paths taking values in

S := {1, 2, • • • , S}. The transition rate matrix is Λ = {λ ij }, with (i, j) ∈ S × S, λ ij ∈ R ≥0 , and λ ii = -j =i λ ij .
The main result of this section provides an expression for the infinitesimal generator of the process described by [START_REF] Kunkel | The linear quadratic optimal control problem for linear descriptor systems with variable coefficients[END_REF]. For piecewise deterministic Markov processes with ODEs and a renewal jump process, one can find the computation of generator in [START_REF] Davis | Markov Models and Optimization[END_REF]. These tools can be tailored to study the stability of randomly switching ODEs [START_REF] Chatterjee | Stabilizing randomly switched systems[END_REF]. Here, we generalize this operator based approach to compute the infinitesimal generator explicitly for randomly switching DAEs.

Proposition 3. The joint process σ, x described by (1) and

(3) is a Markov process under (A1) and (A2). Moreover, for a function V : R × R n × S → R ≥0 continuously differentiable in first two arguments, if (A3) For each i ∈ S, there exists a function

F i : R × R n × R n → R such that ∂ z V (t, z, i), ω = F i (t, z, E i (z)ω), ∀ ω ∈ R n ;
Then, for each z ∈ C i (t, u),

LV (t, z, i) = F i (t, z, f i (t, z, u)) + λ ii V (t, z, i)+ j =i λ ij V (t, Π j (t, z, u), j) + ∂ t V (t, z, i). ( 19
)
Before giving the proof of Proposition 3, we compute the expression in [START_REF] Kunkel | Differential-Algebraic Equations: Analysis and Numerical Solution[END_REF] for the case of autonomous linear systems and V of quadratic form. As an application of this result, we provide conditions for mean-square stability of linear systems.

B. Stability of Markov Switched Linear DAEs

Consider an autonomous linear switched DAE

E σ ẋ = A σ x, x(0) = x 0 , ( 20
)
where σ is again considered to be a Markov processes over a finite set. If each pair (E i , A i ) is regular and index-one, then using Proposition 2, it is seen that Π i (t, x) = Π i,aut x and the consistency space C i (independent of t and u) is given by

C i = im Π i,aut := {Π i,aut x | x ∈ R n }.
We say that ( 20) is asymptotically mean-square stable, if E |x(t)| 2 x(0), σ(0) converges to the origin asymptotically as t → ∞. To derive stability conditions, we introduce the following assumptions on the dynamics of individual modes: (S1) For each pair (E i , A i ), i ∈ S, there exists a symmetric positive definite matrix P i such that5

A i P i E i + E i P i A i ≤ αE i P i E i , on C i
for some α ∈ R. (S2) There exists β > 0 such that for each i, j ∈ S, j = i,

E j P j E j ≤ β E i P i E i , on C i .
Proposition 4. Consider the autonomous switched system [START_REF] Kunkel | Optimal control for unstructured nonlinear differential-algebraic equations of arbitrary index[END_REF] driven by Markov process σ over the finite set {1, • • • , S} with transition rates {λ ij } (i,j)∈S×S , and suppose that each matrix pair (E i , A i ) is regular and index-one. Assume, in addition, that statements (S1), (S2) hold, and

α + λ ii (1 -β) < 0, ∀ i ∈ S, (21)
then system (20) is mean-square stable.

The proof of this result relies on computing LV (t, z, i) for ( 19) with some appropriate V , and then employing Dynkin's formula 6 to get a bound on E V (x(t), σ(t)) x(0), σ(0) by integration.

Proof. For each i ∈ S, we consider the time-invariant V :

R n × S → R + to be V (z, i) = (E i z) P i (E i z),then this function satisfies (A3) by choosing F i (t, z, E i ω) = (E i z) P i (E i ω) + (E i ω) P i E i z.
When the individual modes are described by linear timeinvariant dynamics, f i (t, x) = A i x, i ∈ S, then Π i,in = 0, and

Π i (x) = Π i,aut x. Moreover, it is observed that E i Π i,aut = E i .
The expression [START_REF] Kunkel | Differential-Algebraic Equations: Analysis and Numerical Solution[END_REF] thus takes the form

LV (z, i) = z (A i P i E i + E i P i A i )z + λ ii (E i z) P i (E i z) + j =i λ ij (E j z) P j (E j z) (22)
for each z ∈ C i . Using (S1) and (S2), we thus get

LV (z, i) ≤   α + λ ii + β j =i λ ij   V (z, i), z ∈ C i .
Due to the property of the rate-transition matrix, λ ii = j =i λ ij < 0, so that

LV (z, i) ≤ (α + λ ii (1 -β))V (z, i), z ∈ C i .
Using condition [START_REF] Kurina | Feedback solutions of optimal control problems with DAE constraints[END_REF], choose ε > 0 such that (α + λ ii (1β)) < -ε. It then follows from Dynkin's formula that

E V (x(t), σ(t)) x(0), σ(0) ≤ exp (-εt)V (x(0), σ(0))
and hence the desired result follows.

C. Proof of Proposition 3

The basic idea of the proof of the result is to use the property of Markovian switching to write down the expression for the joint process σ, x up to first order terms. The resulting expressions are then substituted in the definition of infinitesimal generator [START_REF] Ji | Controllability, stabilizability, and continuoustime Markovian jump linear quadratic control[END_REF], along with Assumption (A3), to get [START_REF] Kunkel | Differential-Algebraic Equations: Analysis and Numerical Solution[END_REF]. Proof of Proposition 3. Based on the definition of LV (t, z, i) given in [START_REF] Ji | Controllability, stabilizability, and continuoustime Markovian jump linear quadratic control[END_REF], the primary step in the proof is to compute

E V t + h, x(t + h), σ(t + h) x(t) = z, σ(t) = i for h > 0
arbitrarily small. Toward this end, we compute the conditional probability distribution of x(t + h), σ(t + h) for small h > 0 given x(t), σ(t) using the definition of Markovian switching given in [START_REF] Bas | Feedback equilibria in differential games with structural and modal uncertainties[END_REF]. For h > 0 small, it is seen that

E V t + h, x(t + h), σ(t + h) σ(t), x(t) = E V t + h, x(t + h), σ(t + h) 1 {σ(t+h)=σ(t)} + 1 {σ(t+h) =σ(t)} x(t), σ(t) = E V t + h, x(t + h), σ(t + h) x(t), σ(t) = σ(t + h) = i (1 + λ ii h + o(h))+ i =j (λ ij h)E V t + h, x(t + h), σ(t + h) x(t), σ(t) = i, σ(t + h) = j + o(h), (23)
where 1 {σ(t+h)=σ(t)} is the indicator for the event when σ

does not change its value on the interval [t, t + h], while 1 {σ(t+h) =σ(t)} is the indicator for the event when σ changes its value over the interval [t, t + h]. For the case when σ(t + h) = σ(t) = i, there is no change in the dynamics on the interval [t, t + h[, and hence

E i (x(t))x(t + h) = E i (x(t))x(t) + hf i (t, x(t), u) + o(h).
We thus obtain

V t + h, x(t + h), σ(t + h) = V (t, x(t), σ(t))+ h ∂ z V (t, x(t), σ(t)), ẋ(t) + h∂ t V (t, x(t), σ(t)) + o(h).
Using Assumption (A3), we get

V t + h, x(t + h), σ(t + h) = V (t, x(t), σ(t))+ hF i (t, x(t), f i (t, z, u)) + h∂ t V (t, x(t), σ(t)) + o(h)
leading to the estimate

E V t + h, x(t + h), σ(t + h) x(t), σ(t) = σ(t + h) = i (1 + λ ii h + o(h)) = V (t, x(t), σ(t)) + λ ii hV (t, x(t), σ(t))+ hF i (t, x(t), f i (t, x(t), u)) + h∂ t V (t, x(t), σ(t)) + o(h). (24)
Concerning the second term on the right-hand side of ( 23), we let t ij ∈]t, t + h[ denote the time at which σ changes its value from i to j. We can then write

x(t - ij ) = x(t) + (t ij -t) ẋ(t) + o(h).
Because of the jump at switching time t ij

x(t + ij ) = Π j (t ij , x(t - ij ), u) = Π j (t ij , x(t) + (t ij -t) ẋ(t) + o(h), u) = Π j (t ij , z, u) + (t ij -t)∂ z Π j (t ij , z, u) ẋ(t) + o(h),
where we substituted x(t) = z ∈ C i (t, u) in the last equation, and we recall that Π i is differentiable in second argument due to (A2). Next, we observe that

x(t + h) = x(t + ij ) + (t + h -t ij ) ẋ(t ij ) + o(h) = Π j (t ij , z, u) + (t ij -t)∂ z Π j (t ij , z, u) ẋ(t) + (t + h -t ij ) ẋ(t ij ) + o(h).
We now recall that given σ(t) = i, the probability that there are no jumps of σ in the interval [t, t] is e λii(t-t) . Also, given σ(t) = i, the probability density that the first jump of σ after time t is from mode i to j occurs at t is λ ij e λii(t-t) . Taking the expectation with respect to the values of σ and the holding times in the modes before and after the switch results in

E V t + h, x(t + h), σ(t + h) x(t), σ(t) = i, σ(t + h) = j = V (t, Π j (t, x(t), u), j) + h∂ t V (t, x(t), j)+ h ∂ z V (t, z, j), ∞ t (t -t)∂ z Π(t, z, u) ẋ(t)e λii(t-t) dt + h ∂ z V (t, z, j), λ ij ∞ t (t -t) ẋ(t ij )e λii(t-t) dt + o(h).
Using the properties of the decaying exponential function, we thus get

E V t + h, x(t + h), σ(t + h) x(t), σ(t) = i, σ(t + h) = j = V (t, Π j (x(t)), j) + O(h). (25)
Substituting ( 24) and ( 25) in [START_REF] Li | Lyapunov iterations for solving coupled algebraic Riccati equations of Nash differential games and algebraic Riccati equations of zero-sum games[END_REF] results in

E V t + h, x(t + h), σ(t + h) σ(t), x(t) = V (t, x(t), σ(t)) + λ ii hV (t, x(t), σ(t)) + hF i (t, x(t), f i (t, z, u)) + h∂ t V (t, x(t), σ(t)) + h i =j λ ij V (t, Π j (t, x(t), u), j) + o(h).
We can now invoke the definition of LV given in [START_REF] Ji | Controllability, stabilizability, and continuoustime Markovian jump linear quadratic control[END_REF], from where the desired expression (19) follows.

D. Dynamic Programming

The next tool that we need is an analogue of the Hamilton-Jacobi-Bellman equation for Markov jump DAEs. This is done by using the result of Proposition 3 to address the single player optimal control problem in Markov jump DAEs. For a given initial condition, let us consider the cost functional defined by

J (u) = E T t0 c(s, x(s), u(s), σ(s)) ds + c T (x(T ), σ(T )) (26)
where the expectation is taken over all possible sample paths of σ conditioned upon x(0) and σ(0). We consider the optimal control problem inf u J (u)

subject to E σ (x) ẋ = f σ (t, x, u), x(0) = x 0 . ( 27
)
As is well-known, there are two common techniques for addressing such optimization problems. We can get candidates for open-loop optimal control which satisfy the necessary conditions provided by Pontryagin's Maximum principle. But since we are interested in optimal feedback policies in our game formulation, here we use the dynamic programming approach which provides sufficient conditions for optimal feedback control law. While in the general case, this approach leads to coupled partial differential equations, we will show that for linear dynamics with quadratic cost, the problem of finding optimal control boils down to solving coupled Riccati differential equations. Before going further, we remark that dynamic programming has been extensively used in the literature for solving optimal control problems for stochastic systems. The following result is in particular a generalization of the optimality conditions developed for Markov jump processes with ODEs [START_REF] Azevedo | Dynamic programming for a Makov-switching jump-diffusion[END_REF], [START_REF] Rishel | Control of systems with jump Markov disturbances[END_REF], [START_REF] Rishel | Dynamic programming and minimum principles for systems with jump Markov disturbances[END_REF], where unlike ODEs, the state jumps between consistency spaces of the constituent DAE subsystems.

To implement the dynamic programming approach, we introduce the value function V : R × R n × S → R defined via conditional expectation as

V (t, z, i) := inf u [t,T ] E T t c(s, x(s), u(s), σ(s)) ds + c T (x(T ), σ(T )) x(t) = z, σ(t) = i (28)
satisfying the boundary condition

V (T, z, σ) = c T (z, σ).
Intuitively speaking, V (t, z, i) is the cost-to-go from time t with x(t) = z, σ(t) = i, as it provides the minimum value of the cost functional in [START_REF] Luenberger | Singular dynamic Leontief systems[END_REF] over the interval [t, T ].

Proposition 5. Consider the optimal control problem (27) under the assumptions (A1) and (A2). If, for each i ∈ S, the optimal cost-to-go function V (•, •, i) is continuously differentiable in its arguments, and satisfies (A3), then it satisfies the following partial differential equation:

-∂ t V (t, z, i) = inf u∈dom Ci(t,•) c (t, z, u, i)+F i (t, z, f i (t, z, u)) + λ ii V (t, z, i) + j =i,j∈S λ ij V (t, Π j (t, z, u), j) , (29)
for every z ∈ C i (t, u), with the boundary condition V (T, z, i) = c T (z, i).

Remark 1. As a result of Proposition 5, the optimal control problem for randomly switching DAEs is solved by computing, for each i ∈ S, V (•, •, i) that satisfies [START_REF] Nash | Equilibrium points in N -person games[END_REF] with the terminal condition V (T, z, i) = c T (z, i); and the optimal control (at each time instant) is obtained by minimizing the socalled Hamiltonian on the right-hand side of [START_REF] Nash | Equilibrium points in N -person games[END_REF]. In contrast to the standard HJB equation that appears in the optimal control problems associated with nonlinear control systems [START_REF] Liberzon | Calculus of Variations and Optimal Control Theory: A Concise Introduction[END_REF], there is a coupling in [START_REF] Nash | Equilibrium points in N -person games[END_REF] between the value functions associated with individual subsystems. One also has to be careful about the fact that the equation ( 29) is only required to hold for z ∈ C i (t, u). Also, when solving for V (•, •, i), the minimization problem on the right-hand side of ( 29) contains the terms V (t, Π j (t, z, u), j), j = i which depend on u. In Example 1 below, it is shown that this dependence on u can be ruled out in certain cases.

Before giving the proof of this result, we present some examples to see how ( 29) can be solved.

Example 1. Consider the nonlinear system in R 2 with two subsystems described as,

σ = 1 : ẋ1 = f 1 (x 1 , x 2 ) + g 1 (x 1 , x 2 )u 0 = x 2 -x 1 h 1 (x 1 ) -u (30)
σ = 2 : 0 = x 1 -x 2 h 2 (x 2 ) -u ẋ2 = f 2 (x 1 , x 2 ) + g 2 (x 1 , x 2 )u (31)
The jump maps are thus given by

x := (x 1 , x 2 ) → Π 1 (x, u) = x 1 x 1 h 1 (x 1 ) + u
and for the second subsystem

x = (x 1 , x 2 ) → Π 2 (x, u) = x 2 h 2 (x 2 ) + u x 2 .
The cost function that we consider is

c(x, u, 1) = x 2 1 + u 2 , and c(x, u, 2) = x 2 2 + u 2
with terminal cost given by

c T (x, 1) = a 1 x 2 1 , and c T (x, 2) = a 2 x 2 2 for a 1 , a 2 ≥ 0. Let V (t, z, 1) := a 1 x 2 1 , and V (t, z, 2) := a 2 x 2 2 .
With this choice of value functions, for t with σ(t) = 1, we choose

µ * (t, x) = arg min u∈R x 2 1 + u 2 + 2a 1 x 1 f 1 (x 1 , x 2 ) + λ 11 a 1 x 2 1 + 2a 1 x 1 g 1 (x 1 , x 2 )u -λ 11 a 2 (x 1 h 1 (x 1 ) + u) 2 , so that µ * (t, x) = b 11 x 1 h 1 (x 1 ) -b 12 x 1 g 1 (x 1 , x 2 )
where b 11 := λ 11 a 2 /(1 -λ 11 a 2 ) and b 12 := a 1 /(1 -λ 11 a 2 ). Therefore, [START_REF] Nash | Equilibrium points in N -person games[END_REF] takes the form

0 = (1+λ 11 a 1 )x 2 1 +2a 1 x 1 f 1 +(b 2 12 -2a 1 b 2 12 -λ 11 a 2 b 2 12 )x 2 1 g 2 1 +(b 2 11 -λ 11 a 2 (1+b 11 ) 2 )x 2 1 h 2 1 +b 11 (2(a 1 -b 12 )+λ 11 a 2 b 12 )
x 2 1 h 1 g 1 where we have suppressed the arguments of the functions f 1 , g 1 , h 1 for simplifying the expression. The foregoing equation holds if

f 1 (x 1 , x 2 ) = c 11 x 1 + c 12 x 1 g 2 1 (x 1 , x 2 ) + c 13 x 1 h 2 1 (x 1 ) + c 14 x 1 h 1 (x 1 )g 1 (x 1 , x 2 )
with appropriately chosen constants c 1i , i = 1, 2, 3, 4.

Similarly, for t when σ(t) = 2, we compute

µ * (t, x) = arg min u∈R x 2 2 + u 2 + 2a 2 x 2 f 2 (x 1 , x 2 ) + λ 22 a 2 x 2 2 + 2a 2 x 2 g 2 (x 1 , x 2 )u -λ 22 a 1 (x 2 h 2 (x 2 ) + u) 2 ,
so that, with b 21 := λ 22 a 1 /(1 -λ 22 a 1 ) and b 22 := a 2 /(1 -

λ 22 a 1 ) µ * (t, x) = b 21 a 1 x 2 h 2 (x 2 ) -b 22 x 2 g 2 (x 1 , x 2 ).
Equation ( 29) then takes the form

0 = (1+λ 22 a 2 )x 2 2 +2a 2 x 2 f 2 +(b 2 22 -2a 2 b 2 22 -λ 22 a 1 b 2 22 )x 2 2 g 2 2 +(b 2 21 -λ 22 a 2 (1+b 21 ) 2 )x 2 2 h 2 2 +b 21 (2(a 2 -b 22 )+λ 22 a 1 b 22 )x 2 2 h 2 g 2 and is satisfied if f 2 (x 1 , x 2 ) = c 21 x 2 + c 22 x 2 g 2 2 (x 1 , x 2 ) + c 23 x 2 h 2 2 (x 2 ) + c 24 x 2 h 2 (x 2 )g 2 (x 1 , x 2 )
with appropriately chosen constants c 2i , i = 1, 2, 3, 4.

As the next example, we study linear dynamics and quadratic cost functionals. In such cases, it is seen that the partial differential equation in [START_REF] Nash | Equilibrium points in N -person games[END_REF] leads to a Riccati-like differential equation. In the context of ODEs, such equations can be found in [START_REF] Costa | Continuous-Time Markov Jump Linear Systems[END_REF], [START_REF] Safaei | Quadratic control of stochastic hybrid systems with renewal transitions[END_REF].

Example 2. Let us consider the cost functional

J (u) = E T t0 x(s) Q σ(s) x(s) + u(s) R σ(s) u(s) ds + x(T ) M σ(T ) x(T )
subject to the dynamics E σ ẋ = A σ x + B σ u, where σ is a Markov process over the finite set S as introduced in Section II. Let us take V (t, z, i) = (E i z) P i (t)(E i z), for some nonnegative definite matrix valued function P i (•). Following the same calculations as in the linear autonomous case, we let

F i (t, z, E i ω) = (E i z) P i (t)(E i ω) + (E i ω) P i (t)E i z.
Choosing the matrices S i , T i as in Proposition 2, it is seen that

E i Π i,in = S -1 i S i E i T i 0 -B i,2 = 0 (32)
and using the fact that E i Π i,aut = E i , we have V (t, Π j (t, z, u), j) = z Π j,aut E j P j E j Π j,aut z = z E j P j E j z.

For z ∈ C i (t, u), we have that z = Π i,aut z + Π i,in u, so that V (t, Π j (t, z, u), j) also depends on u. As a result, the minimization problem in ( 29) can be solved by computing

inf u c (t, z, u; i) + F i (t, z, A i z + B i u) + j∈S λ ij V (t, z, j) = inf u u R i u + z (E i P i A i + A i P i E i + λ ii E i P i E i )z + z E i P i B i u + u B i P i E i z + z (Q i + j =i λ ij E j P j E j )z
We will provide conditions to obtain P i (t) such that (29) holds for z ∈ C i,u for the optimal choice of u. We write z = z i,aut + z i,in , where z i,aut := Π i,aut z, and z i,in := Π i,in u, so that z i,aut does not depend on the input, and z i,in depends on the input directly. For each i ∈ S, let us introduce the matrix O i ,

O i := Q i + j =i λ ij E j P j E j .
The right-hand side of ( 29) is obtained by taking the infimum of the following Hamiltonian over all u ∈ R m :

z i,aut (E i P i A i + A i P i E i + λ ii E i P i E i + O i )z i,aut + 2z i,aut E i P i (A i Π i,in + B i ) + Π i,aut O i Π i,in u + u R i + Π i,in O i Π i,in u,
where we used the fact that Π i,aut being a projector has the property that Π i,aut z i,aut = z i,aut . It is thus seen that the optimal input is

µ * (t, z) = -R -1 σ B σ P σ (t)E σ + Π σ,in O σ Π σ,aut z
where R i := R i + Π i,in O i Π i,in is positive definite, and we let

B i := B i + A i Π i,in . Using the notation, G i (P i , B i ) := B i P i (t)E i + Π i,in O i Π i,aut
and plugging the expression for µ * in (29), we obtain

E i Ṗi E i + E i P i A i Π i,aut + Π i,aut A i P i E i + λ ii E i P i E i + Π i,aut O i Π i,aut -G i (P i , B i ) R -1 i G i (P i , B i ) = 0, (33)
which must be solved backward in time for P i with boundary condition

E i P i (T )E i = M i , for each i ∈ S.
Remark 2. The term G i (P i , B i ) captures the difference observed in the Riccati differential equation because of the dependence of the algebraic constraints on the input. In particular, if the algebraic constraints do not depend on the input, then Π i,in = 0, which results in R i = R i and G i (P i , B i ) = B i P i (t)E i . This case was studied in our conference paper [START_REF] Tanwani | Feedback Nash equilibrium for Markov jump games under differential-algebraic constraints with application to robust control[END_REF]. Remark 3. Unlike linear ODEs, it is not true for DAEs that the same quadratic function satisfying the Hamilton-Jacobi-Bellman equation satisfies also the Euler-Lagrange equations [START_REF] Kunkel | The linear quadratic optimal control problem for linear descriptor systems with variable coefficients[END_REF]. Therefore, it is also possible to choose another structure for the value function and work out different equations for computing the feedback control. As an alternate choice for the value function, one may choose

V (t, z, i) = (E i z) P i (t)z where P (t) is such that P i (t) E i = E i P i (t) ≥ 0.
This completes the examples, and we now prove Proposition 5 and derive [START_REF] Nash | Equilibrium points in N -person games[END_REF] in the process. Proof of Proposition 5. According to the principle of optimality, it can be shown that V in (28) satisfies the relation

V (t, z, i) = inf u [t,t+h] E t+h t c(s, x(s), u(s), σ(s)) ds + E V (t + h, x(t + h), σ(t + h)) x(t) = z, σ(t) = i (34)
for every z ∈ C i (t, u(t)) and arbitrarily small h > 0. Using the definition of infinitesimal generator, we have

E V (t + h, x(t + h), σ(t + h)) x(t) = z, σ(t) = i = V (t, z, i) + hLV (t, z, i) + o(h), (35)
and similarly, we can obtain

E t+h t c(s, x(s), u(s), σ(s)) ds x(t) = z, σ(t) = i = h c(t, z, u(t), i) + o(h). (36)
Plugging the values from ( 35) and ( 36) in (34) yields

V (t, z, i) = inf u [t,t+h]
V (t, z, i) + h c (t, z, u(t), i)

+ hLV (t, z, i) + o(h) .
Cancelling out V on both sides, we get

0 = inf u [t,t+h] h c (t, z, u(t), i) + h LV (t, z, i) + o(h) .
Divide the last expression by h, and let h go to zero, then the term o(h)/h vanishes. Plugging the expression for LV (t, z, i) from ( 19) results in

0 = inf u c (t, z, u, i) + F i (t, z, f i (t, z, u)) + λ ii V (t, z, i)+ j =i λ ij V (t, Π j (u, z), j) + ∂ t V (t, z, i) ,
or equivalently, for every z ∈ C i (t, u),

-∂ t V (t, z, i) = inf u c (t, z, u, i) + F i (t, z, f i (t, z, u))+ λ ii V (t, z, i) + j =i λ ij V (t, Π j (t, z, u), j)
which is the desired equality.

V. OPTIMAL STRATEGIES FOR NONCOOPERATIVE GAME

We now use the theoretical results from previous section to address the game problem formulated in Section II.

A. General Case

The general case concerns an N -player dynamic noncooperative game, where the objective of player p ∈ N = {1, . . . , N } is to find a strategy µ p to minimize the cost J p given in [START_REF] Chatterjee | Stabilizing randomly switched systems[END_REF], that is,

inf µp J p (µ 1 , . . . , µ N ), (37a)
subject to

E σ ẋ = f σ (t, x, µ 1 , . . . , µ N ) (37b)
where we recall that σ is a Markov process over the finite set S and µ p : R × R n × S → U p belongs to the class of memoryless pure feedback strategies. To state the main result on existence of Nash equilibrium and the optimal strategies, we introduce the notation

f * i,p (t, z, u p ) := f i (t,
+ λ ii V p (t, z, i) + j =i λ ij V p (t, Π * j,p (t, z, u p ), j) (38
) for every z ∈ C * i,p (t, u p ), with the boundary condition V p (T, z, i) = c p,T (z, σ(T )).

• For a player p ∈ N , given a triplet (t, z, i), let µ * p (t, z, i) := u * p , with u * p being the minimizer of the optimization problem

min up∈Up F i,p (t, z, f * i,p (t, z, u p )) + c * p (t, z, u p , i) + λ ii V p (t, z, i) + j =i λ ij V p (t, Π * j,p (t, z, u p ), j)
subject to z ∈ C * i,p (t, u p ), and assume that the functions µ * p : R × R n × S → U p are such that the pair

(E i , f i (µ * 1 , • • • , µ * N )
) satisfies (A1) and (A2) for each i ∈ S. Then, the memoryless pure-feedback strategies µ * p (t, x, σ(t)) | p ∈ {1, . . . , N } constitute a Nash equilibrium and the minimum cost for each player is V p (t 0 , x 0 , σ(t 0 )).

Proof. For each player p ∈ {1, . . . , N }, the minimum costto-go function from an initial state z at an initial time t is described by the corresponding value function V p : R × R n × S → R given by

V p (t, z, i) := inf up [t,T ] E T t c * p (s, x(s), u p (s), σ(s)) ds + c p,T (x(T ), σ(T )) x(t) = z, σ(t) = i ( 39
)
where x is the solution of the system E σ ẋ = f * σ,p (t, x, u p ) over the interval [t, T ]. The function V p satisfies the boundary condition

V p (T, x(T ), σ(T )) = c p,T (x(T ), σ(T )). ( 40
)
According to Proposition 5, this function satisfies the equation ( 38) and the strategy µ p which makes J p (µ * 1 , . . . , µ p , . . . , µ * N ) equal to its minimal value is given by µ * p . The existence of solutions is guaranteed due to Proposition 1. According to Definition 1, such strategies constitute the Nash equilibrium and the corresponding optimal cost for each player p is V p (t 0 , x 0 , σ(t 0 )).

From implementation viewpoint, the major obstacle in the application of Theorem 1 lies in solving the coupled partial differential equations in [START_REF] Tanwani | Stabilization of deterministic control systems under random sampling: Overview and recent developments[END_REF], as the computation of optimal strategy directly follows from knowing V p . In certain cases, we can formulate [START_REF] Tanwani | Stabilization of deterministic control systems under random sampling: Overview and recent developments[END_REF] as finite dimensional ordinary differential equations or even as static algebraic equations, as we discuss such cases in the sequel.

B. Linear Quadratic Dynamic Game

We consider the particular case of the games with quadratic cost functions for each player and linear dynamics with Markovian switching in the matrices describing the constraints. In this case, the coupled partial differential equations from Theorem 1 lead to coupled Riccati differential equations and the optimal feedback strategies are described by linear time-varying functions of the state.

The differential game ( 37) is now described by taking the cost functional of player p ∈ {1, . . . , N } as,

J p (µ 1 , . . . , µ n ) = E x(T ) M σ(T ),p x(T ) + E T t0 x(s) Q σ(s),p x(s) + u p (s) R σ(s),p u p (s) ds (41)
where Q i,p is symmetric positive semidefinite, R i,p is symmetric positive definite for each i ∈ S, and fix u p to be of form [START_REF] Berger | The quasi-Weierstraß form for regular matrix pencils[END_REF]. For the sake of simplicity, we have taken the cost functional for player p in (41) to be independent of other players. The dynamical system which describes the evolution of the state is given by

E σ ẋ = A σ x + N p=1 B σ,p u p , (42)
where E i , A i ∈ R n×n and B i,p ∈ R n×mp , i ∈ S. To work out a specific case of Theorem 1 for the aforementioned case, we introduce the following notation in the spirit of Example 2.

For each i ∈ S, p ∈ N , let O i,p := Q i,p + j =i λ ij E j P j,p E j ; let Π i,p := T i 0 -B i,p
, where B i,p denotes the last (n -ν i ) rows of the matrix B i,p . Next, we let B i,p := (A i Π i,p + B i,p ), R i,p := (R i,p + Π i,p O i,p Π i,p ), and finally, let

G i (P i,p , B i,q ) := B i,q P i,p (t)E i + Π i,q O i,p (t)Π i,aut . ( 43
)
Corollary 1. Consider the N -player quadratic game where the objective of each player is described by (37a), [START_REF] Trenn | Switched differential algebraic equations[END_REF], subject to [START_REF] Voigt | On Linear Quadratic Optimal Control and Robustness of Differential-Algebraic Systems[END_REF]. For each i ∈ S, assume that the pair (E i , A i ) is regular with index one and that there exist symmetric nonnegative definite matrix-valued P i,p (•), p ∈ {1, . . . , N }, which satisfy the condition7

Π i,p O i,p Π i,q = 0, ∀p, q ∈ N , p = q, ( 44
)
and solve the coupled Riccati differential equations

E i Ṗi,p E i + E i P i,p A i Π i,aut + Π i,aut A i P i,p E i + λ ii E i P i,p E i + Π i,aut O i,p Π i,aut -G i (P i,p , B i,p ) R -1 i,p G i (P i,p , B i,p ) - N q=1,q =p G i (P i,p , B i,q ) R -1 i,q G i (P i,q , B i,q ) - N q=1,q =p G i (P i,q , B i,q ) R -1 i,q G i (P i,p , B i,q ) + q =p G i (P i,q , B i,q )Π i,q R -1 i,q Π i,q O i,p G i (P i,q , B i,q ) = 0, (45
) with the boundary condition

E i P i,p (T )E i = M i,p . (46)
Then, the memoryless feedback optimal strategies

µ * p (t, x) = -R -1 σ(t),p G σ(t) (P σ(t),p (t), B σ(t),p )x ( 47
)
constitute the Nash equilibrium, and the optimal value for player p is given by x 0 E σ(0) P σ(0),p E σ(0) x 0 .

Proof. The proof relies on considering the value function for player p ∈ {1, . . . , N } to be V p (t, x, σ) := x E σ P σ,p (t)E σ x and showing that the conditions of Theorem 1 hold. It is seen from the calculations carried out in Example 2 that, 8 with P i,p satisfying ( 45) and ( 46), equation [START_REF] Tanwani | Stabilization of deterministic control systems under random sampling: Overview and recent developments[END_REF] indeed holds for each i ∈ S, p ∈ {1, . . . , N }. We just need to check that the closed-loop system satisfies (A1) and (A2) under the feedback policies (47). Indeed, this is seen from the fact that if the pair (E i , A i ) is regular and index-one, then the pair . Choose T i := T i I 0

(E i , A i - K i E i ), for any matrix K i ∈ R n×n ,
Ki,21 I , then it is seen that

(S i E i T i , S i (A i -K i E i ) T i ) = I 0 0 0 , J i -K i,11 0 0 I .
For each mode i ∈ S, the optimal policy µ * p indeed results in the closed-loop system of the form

E σ ẋ = A σ x -K σ E σ x, where K i = N p=1 B i,p R -1 i,p ( B i,p P i,p (t) + Π i,p O i,p T i S i )
, and hence (A1) and (A2) hold.

Compared to the Riccati differential equation derived in the single player case in Example 2, we now see more complicated expressions in (45), which involve products of the matrices P i,p and P i,q that describe the value function for players p and q, respectively. This adds the difficulty in solving such equations numerically and even establishing existence of solutions. Remark 4. The feedback strategies given in (47) have a particular structure in the sense that u(t) can be seen as a matrix multiplying E σ(t) x(t) and in the proof of Corollary 1, we saw that the resulting closed-loop system is of the form E σ ẋ = (A σ -K σ E σ )x and that this particular feedback preserves the index of the system. Moreover, if we let ( z 1 i , z 2 i ) := T -1 i x to be the new coordinates for the closedloop system in mode i ∈ S, then we see that z 1 i remains unchanged when the system switches to mode i but we have z 2 i = 0. In other words, z 1 i does not jump, but z 2 i may jump. Remark 5. Corollary 1 only provides one possible set of feedback strategies and it is subject to the solution of (45). But it is not necessarily the unique solution to the linear quadratic game under consideration. As stated in Remark 3, a different choice of the value function for each player would result in a different set of Riccati-like differential equations, and also different feedback strategies. It is instructive to recall that this nonuniqueness feature also arises in the simplest case of nonswitching deterministic ODEs [START_REF] Bas | Dynamic Noncooperative Game Theory[END_REF]Remark 6.16].

VI. INFINITE HORIZON LINEAR QUADRATIC GAME

An important class of problems within the framework of linear quadratic dynamic games deals with cost functionals over infinite horizon. In generalizing the finite-horizon approach to deal with such cases, one has to make sure that the solutions of the coupled Riccati equations converge to constant matrices asymptotically in time. This issue is addressed in the sequel by studying the feasibility of time-invariant stationary problems.

The differential game (37) is now described by taking, for each p ∈ {1, . . . , N },

J p (µ 1 , . . . , µ n ) = E ∞ 0 x(s) Q σ(s),p x(s) + u p (s) R σ(s),p u p (s) ds ( 48
)
where it is assumed that Q i,p is symmetric positive semidefinite, and R i,p is symmetric positive definite for every i ∈ S.

The dynamical system under consideration is given by [START_REF] Voigt | On Linear Quadratic Optimal Control and Robustness of Differential-Algebraic Systems[END_REF]. The terminal state is taken out of the cost function since x must converge to zero asymptotically for the integral to be finite.

A. Stabilizability of Markov Jump Linear DAEs

For the cost over infinite horizon to be finite, we need to make sure that there is at least one input (in the class of admissible feedbacks) which can steer the state to the origin. This question is equally formulated by asking whether the system under consideration (42) is stabilizable. We refer the reader to a survey [START_REF]Surveys in Differential-Algebraic Equations I[END_REF] for different notions of controllability for nonswitching DAEs. Here, we are interested in studying stabilizability for randomly switching DAE, and toward that end, we rewrite [START_REF] Voigt | On Linear Quadratic Optimal Control and Robustness of Differential-Algebraic Systems[END_REF] with single player

E σ ẋ = A σ x + B σ u. (49)
We say that a collection of triplets {(E i , A i , B i )} i∈S , is meansquare stabilizable over {Q i } i∈S if, for each i ∈ S, there exists L i such that the consistency space of

(E i , A i -B i L i E i ) is C i,aut = Q i ,
and there exist positive definite matrices {P i } i∈S solving

A i P i E i + E i P i A i + λ ii E i (P i -P i B i L i -L i B i P i )E i + j =i,j∈S λ ij E j P j E j ≤ -Q i , on C i,aut , (50)
for every matrix Q i which is positive definite on C i,aut .

Proposition 6. If the set of triplets {(E i , A i , B i )} i∈S is mean-square stabilizable over some finite family of subspaces {Q i } i∈S , then there exists a feedback law which renders system (49) mean-square asymptotically stable.

In the definition of stabilizability, it is required that L i should not depend on the choice of Q i , whereas P i may change as a function of Q i . In contrast to the result of Proposition 4, the condition in (50) allows us to compute stabilizing gains which render the closed-loop system stable for given values of transition rates associated with the Markovian switching.

Proof of Proposition 6. Let L i and P i , i ∈ S, be such that (50) holds, and choose u = -L σ E σ x in (49), so that the closed-loop system is

E σ ẋ = (A σ -B σ L σ E σ )x. (51)
We show that (51) is mean-square asymptotically stable, while using (50), by working with the stochastic Lyapunov function

V (x, i) = (E i x) P i (E i x).
The infinitesimal generator for this Lyapunov function was computed in [START_REF] Leontief | Studies in the Structure of the American Economy: Theoretical and Empirical Explorations in Input-Output Analysis[END_REF], that is,

LV (x, i) = x ((A i -B i L i E i ) P i E i +E i P i (A i -B i L i E i ))x + λ ii (E i x) P i (E i x) + j =i λ ij (E j x) P j (E j x). (52)
Let Π i,orth be a matrix with orthonormal columns which span C i,aut . From condition (50), it then follows that9

LV (x, i) ≤ -x Q i x ≤ -λ min (Π i,orth Q i Π i,orth ) λ max (E i P i E i ) V (x, i).
Let ε := min i∈S λmin(Π i,orth QiΠi,orth)

λmax(E i PiEi) ; then, by Dynkin's formula

E V (x(t), σ(t)) x(0) = x 0 , σ(0) = i ≤ e -εt V (x 0 , i).
From the definition of V , it follows that E |x(t)| 2 x 0 , σ(0) converges to the origin asymptotically.

B. Solution of the Infinite-Horizon Game

We can now use the aforementioned property of stabilizability to provide a solution to the N -player game where the player p minimizes the cost function (48) subject to the dynamics given in [START_REF] Voigt | On Linear Quadratic Optimal Control and Robustness of Differential-Algebraic Systems[END_REF]. Thus, compared to the finite-horizon case, we seek value functions V p with terminal condition lim T →∞ E[V p (x(T ), σ(T ))] = 0, for each p ∈ N . The following result generalizes the single-player ODE based result [START_REF] Ji | Controllability, stabilizability, and continuoustime Markovian jump linear quadratic control[END_REF]Theorem 5] to the multi-player case where dynamics are given by DAEs. To state the result, we use the same notation as in Corollary 1 with one slight modification: Instead of timevarying G i , we use its time-stationary counter part

G i (P i,p , B i,q ) = B i,q P i,p E i + Π i,q O i,p Π i,aut ,
where O i,p = Q i,p + j =i λ ij E j P j,p E j . The corresponding stationary version of the Riccati equation is written as

E i P i,p A i Π i,aut + Π i,aut A i P i,p E i + λ ii E i P i,p E i + Π i,aut O i,p Π i,aut -G i (P i,p , B i,p ) R -1 i,p G i (P i,p , B i,p ) - N q=1,q =p
G i (P i,p , B i,q ) R -1 i,q G i (P i,q , B i,q ) -N q=1,q =p G i (P i,q , B i,q ) R -1 i,q G i (P i,p , B i,q ) + q =p G i (P i,q , B i,q )Π i,q R -1 i,q Π i,q O i,p G i (P i,q , B i,q ) = 0.

(53) To state the result, we need the following notation

A i,p := A i - q =p B i,q R -1 i,q G i (P i,q , B i,q ),
where R i,p := (R i,p + Π i,p O i,p Π i,p ).

Theorem 2. Consider the N -player quadratic game where the objective of each player is described by (37a), (48), subject to [START_REF] Voigt | On Linear Quadratic Optimal Control and Robustness of Differential-Algebraic Systems[END_REF]. For each i ∈ S, assume that

• the pair (E i , A i ) is regular with index one,

• there exist symmetric positive definite matrix-valued P i,p , p ∈ {1, . . . , N } which solve (53), and (44) holds with O i,p , • for each player p, the triplets {(E i , A i,p , B i,p )} i∈S is stabilizable over {Q i,p } i∈S , and for each i ∈ S, the matrix Q i,p in (48) is positive definite on the subspace Q i,p . Then, the static feedback optimal strategies

µ * p (t, x) = -R -1 σ(t),p G σ(t) (P i,p , B i,p )x (54)
constitute the Nash equilibrium, and the optimal value for player p is given by x 0 E σ(0) P σ(0),p E σ(0) x 0 .

Proof. We recall that the matrix R i,p is positive definite, and Q i,p is positive definite on Q i,p for each i ∈ S, p ∈ N . Under the stabilizability assumption, there exist matrices L i,p such that, for the closed-loop system

A i P i,p E i + E i P i,p A i + λ ii E i ( P i,p -P i,p B i,p L i,p -L i,p B i,p P i,p )E i + j =i λ ij E j P j,p E j ≤ -Q i,p -L i,p R i,p L i,p
holds on Q i,p for some positive definite matrix P i,p , i ∈ S.

The foregoing equation was obtained from (50) by replacing Q i,p on the right-hand side by the matrix Q i,p + L i,p R i,p L i,p , which is positive definite on C i,aut . To show that the strategies (54) constitute a Nash equilibrium, let us now analyze the cost of player p with (a not necessarily optimal) input u p (t) = -L σ(t),p x(t), while the other players adopt the strategy (54). In this case, by letting V p (z, i) = (E i z) P i,p (E i z), we get

J p (µ * 1 , . . . , µ * p-1 , u p , µ * p+1 , . . . , µ * N ) = E ∞ 0 x(s) Q σ(s),p + L σ(s),p R σ(s),p L σ(s),p x(s) ds ≤ E ∞ 0 -L V p (x(s), σ(s)) ds ≤ V p (x(0), σ(0))
where, in the last inequality, we used Dynkin's formula and the fact that E |x(t)| 2 converges to zero with the chosen control input as t goes to infinity. Thus, the cost J p , for each player p ∈ N , is bounded by V p (x(0), σ(0)) obtained from (not necessarily optimal) input u p (t) = -L σ(t),p x(t). Also, in case of finite-horizon problems over the interval [t 0 , T ], it is seen from ( 41) that the optimal cost J * p is a monotonically nondecreasing function of the terminal time T . Thus, the solutions of the Riccati differential equations (45) given by P i,p (•) are bounded and monotonically nondecreasing functions of time. As T goes to infinity, each of the matrices P i,p (t) converges to a constant matrix P i,p that satisfies (53), which is obtained from (45) by setting Ṗi,p to 0.

VII. CASE STUDY I: INPUT-OUTPUT MODELS IN ECONOMICS

The dynamic input-output (IO) analysis plays an important role in understanding the interdependencies among industries of an economy. In particular, the Leontief model has been used for development of economic growth plans, social policies, and the life cycle assessment in environmental sciences [START_REF] Leontief | Studies in the Structure of the American Economy: Theoretical and Empirical Explorations in Input-Output Analysis[END_REF]. In the description of these models, the state x ∈ R n is the vector of the output levels with each component x p ∈ R denoting the total output of industry p, and y(t) is final demand vector. Another essential element in the model is the matrix H σ ∈ R n×n , called the Leontief input-output function: the (p, q) entry of the matrix H σ represents the amount of commodity p used by per unit of production of industry q. We denote by E σ the possibly singular capital matrix function. Both H σ and E σ are constant matrices for a given σ as the market and technology usually do not change quickly over a period of time. The process σ denotes the disruptive changes in the economy that can be caused by different types of events including natural disasters, technology upgrade, and government policies. The framework of DAEs without switching has been widely used for Leontief models. To accommodate sudden changes being modeled by σ, we assume that these events occur in a stochastic manner that can be modeled by a Markov chain, which results in

x(t) = H σ x(t) + y(t) + E σ ẋ(t). (55)
The demand vector y(t) is specified over a period of time from t 0 to T . Demand can be increased and reduced by each industry to make profit. Consider that y(t) can be represented by y(t) = n p=1 b p (y p (t) -u p (t)), where b p ∈ R n is an n-dimensional vector with its p-th element being 1, but 0 otherwise. The variable u p (t) ∈ R is the amount of resource spent by p-th industry to meet the corresponding demand, and it is of interest to minimize this effort. In addition, each industry aims to regulate its output x p to a desired level xp without adjusting the demand too significantly. This fact can be captured by an n-person game problem with each individual p solving the following optimal control problem:

J p = E T t0 (x p (s) -xp ) 2 + r σ(s),p u 2 p (s) ds , (56)
where r σ(t),p is the weighting factor that tradeoffs between the control and the deviation of the output from the desired trajectory. This problem falls into the framework addressed in Section 5. The Leontief model can be rewritten in the following form for computational convenience:

E σ ż = A σ z + h σ + n p=1 b p u p (57)
where z = x -x, A i := I -H i , and

h i (t) = A i x - n p=1 b p y p (t).
Finding control inputs as a solution to the n-person game problem described above is an extension to the linear-quadratic case in Section 5. We let the value function to take the following generalized form:

V p (t, z, i) = (E i z) P i,p (t)(E i z) + (E i z) v i,p (t) + w i,p (t)
where P i,p (t) ∈ R n×n is positive definite, v i,p (t) ∈ R n , and w i,p (t) ∈ R. Working with the notation B i,p := b p + A i Π i,p as before, and applying the result of Theorem 1, the optimal strategy is given by

u p (t) = -R -1 σ,p G σ (P σ,p (t), B σ,p )(x -x) + W σ,p (t)
where W i,p (t) := B i,p v i,p (t) + j =i λ ij Π i,p E j v j,p (t), and P i,p (t) is obtained by solving the backward Riccati differential equation (45) with boundary condition P i,p (T ) = 0, and Q i,p a diagonal matrix with 1 only in p-th row. The vector-valued function v i,p (•) is obtained by solving the differential equation

E i vi,p (t) = E i P i,p (t)h i (t) + Π i,aut A i v i,p (t) + j∈S λ ij E j v j,p (t) - n q=1 G i (P i,q , B i,q ) R -1 i,q B i,q v i,p (t) -2 n q=1 G i (P i,p , B i,q ) R -1 i,q W i,q (t) + 2 q =p G i (P i,q , B i,q ) R -1 i,q Π i,q O i,p Π i,q R -1 i,q W i,q (t) - n q=1 G i (P i,q , B i,q ) R -1 i,q Π i,q   j =i λ i,j E j v j,p (t)
  with terminal condition v i,j (T ) = 0. For computing the value function, we need to solve for w i,p (t), which is given by ẇi

,p (t) = n j=1 λ ij w i,j (t)+h i v i,p (t)- n q=1 W i,q R -1 i,q B i,q v i,p (t) + q =p W i,q R -1 i,q Π i,q O i,p Π i,q R -1 i,q W i,q (t) - n q=1 W i,q R -1 i,q Π i,q   j =i λ i,j E j v j,p (t)
  with terminal condition w i,j (T ) = 0.

VIII. CASE STUDY II: ROBUST CONTROL OF MARKOV JUMP DAES

We now address an application of our tools in controller synthesis for Markov jump DAEs subject to disturbances in the dynamics. For linear time-invariant plants, H ∞ -controllers address the issue of worst-case design subject to unknown additive disturbances and uncertainties in plants. 10 While the problem was originally studied in the context of frequency domain, several research works from the mid 80's have shown that a time-domain characterization of these controllers uses generalized Riccati differential equations which are closely related to the ones found in linear quadratic dynamic games. This naturally draws connections between H ∞ -control design problems and the game-theoretic approaches [START_REF] Bas | H ∞ Control and Related Minimax Design Problems: A Dynamic Game Approach[END_REF]. In particular, H ∞ -optimal control problem is reformulated as minimax opitmization problem in time-domain, and hence they form a particular class of a two-player zero-sum game where the controller can be viewed as the minimizing player and the disturbance as the maximizing player.

For Markovian jump systems with ODEs, this problem was studied in [START_REF] Pan | H ∞ -control of Markovian jump systems and solutions to associated piecewise-deterministic differential games[END_REF]. In this section, we adopt this viewpoint to design H ∞ controllers which cater for the worst-case effect of the disturbance. In particular, we consider Markov jump DAEs with the control input u and the disturbance w, described by

E σ ẋ = A σ x + B σ u + D σ w, x(0) = x 0 . (58)
We will impose the following assumption on the system dynamics for this section, which basically says the algebraic constraints do not depend on the input u, or the disturbance w. In other words, for each i ∈ S, we have

T i 0 B i = 0, and
T i 0 D i = 0. (59)
The zero-sum game between the control input u and the disturbance w is set up by taking the quadratic performance index, parameterized by γ > 0:

J γ (µ, ν) = E T t0 x(s) Q σ(s) x(s) + u(s) R σ(s) u(s) -γ 2 w(s) w(s) ds -γ 2 x 0 M σ(0) x 0 , (60)
where Q i , M i are symmetric positive semidefinite and R i is symmetric positive definite for each i ∈ S. The first player chooses the strategy µ so that the input u = µ(t, x) minimizes the cost function and the second player chooses the strategy ν to define the function w and the initial condition x 0 to maximize the cost function. It is assumed that there exist nonempty function spaces U ⊂ L 2 ([t 0 , T ]; R m ) and V ⊂ L 2 ([t 0 , T ]; R m ) × R n such that for each u ∈ U and (w, x 0 ) ∈ V, the solution of system (58) is uniquely defined. The upper value of this game is

V := inf µ∈U sup ν∈V J γ (µ, ν). (61)
It is noted that for each u, one can take x 0 = 0 and w = 0 and hence, sup ν∈V J γ (µ, ν) ≥ 0; consequently, V ≥ 0 for every γ > 0. The case V = 0 is the only case of interest as the value of the game will be infinite if V > 0. If γ * is the smallest value of γ for which the game admits a finite value (V = 0), then this value characterizes the L 2 gain from the disturbance to the state, that is,

γ * = inf µ∈U sup ν∈V J (µ, ν) 1/2 ( w 2 + x 0 M σ(0) x 0 ) 1/2 where J (µ, ν) = E T t0 x(s) Q σ(s) x(s) + u(s) R σ(s) u (
s)ds . We now have the following result which relates the value of γ for which V = 0 (with x 0 = 0) to the Riccati differential equations associated with the game.

Theorem 3. Consider the two player zero-sum game with quadratic performance index (60) subject to the dynamic constraints (58) under the condition (59). For each i ∈ S, assume that the pair (E i , A i ) is regular with index one and that there exist symmetric nonnegative definite matrix-valued P i (•), which solve the coupled Riccati differential equations

E i Ṗi E i + E i P i A i Π i,aut + Π i,aut A i P i E i + λ ii E i P i E i -E i P i B i R -1 i B i - 1 γ 2 D i D i P i E i + j =i
λ ij Π i,aut E j P j E j Π i,aut + Π i,aut Q i Π i,aut = 0 (62) on C i,aut , with boundary condition P i (T ) = 0 for some γ > 0, and furthermore

γ 2 M i -E i P i (t 0 )E i ≥ 0.
Then the game has a finite value, that is, V = 0, and the corresponding minimax strategy which attains this upper value is µ * (t, x) = -R -1 σ(t) B σ(t) P σ(t) E σ(t) x. This result can be proved using the arguments similar to Corollary 1. For the minimizing player, we choose the value function V (t, x, σ) := x E σ P σ (t)E σ x, and for the maximizing player, the value function is -V (t, x, σ), where P (•) satisfies the equation (62). The case of inifinite-horizon can be treated similarly as in Section VI by setting Ṗi to 0, for each i ∈ S, and solving the resulting algebraic Riccati equation.

A. Numerical Solvability

Let us now briefly discuss the issue of solving the coupled Riccati equation (62). For the sake of simplicity, consider the stationary case over infinite horizon, so that Ṗ is set equal to zero. We know that for each i ∈ S, there exist invertible matrices S i , T i such that

E i = S -1 i E i T -1 i , and
A i = S -1 i A i T -1
i where E i := I νi×νi 0 0 0 , and A i := J νi×νi 0 0 I (n-νi)×(n-νi) .

In the following, we also use the notation B i := S i B i , D i := S i D i , and

J i := J νi×νi 0 0 0 (n-νi)×(n-νi) .
Plugging these values of E i and A i in (62), multiplying from left by T i and from right by and T i , and using the notation P i = S - i P i S -1 i , we get (with Ṗ = 0)

E i P i J i + J i P i E i -E i P i B i R -1 i B i - 1 γ 2 D i D i P i E i +λ ii E i P i E i + E i T i   j =i λ ij E j P j E j + Q i   T i E i = 0
The advantage of this last matrix equation is that it separates out the coupled Riccati algebraic equations in two parts. Because of the special structure of E i , one can use the existing solvers for Riccati algebraic equation to compute P i , i ∈ S. Based on the algorithm proposed in [START_REF] Li | Lyapunov iterations for solving coupled algebraic Riccati equations of Nash differential games and algebraic Riccati equations of zero-sum games[END_REF], it turns out that if we use a line search method for γ, and an iterative procedure to handle the coupling, then this algorithm is seen to converge to a solution under the usual hypotheses. This technique was used to study the following academic example. It can easily be checked that for this system Π 1 (x) = Π 1,aut x, and C 1 = C 1,aut = im Π 1,aut . In particular, Π 1,aut = 1 0 0 0 1 0 0 0 0

, C 1 = {x ∈ R 3 | x 3 = 0}.
For the second subsystem, the quadruple (E 2 , A 2 , B 2 , D 2 ) is given by: Using the numerical methods described in this section, and choosing Q i to be identity on C i , we find the solutions of Riccati equations, corresponding to, γ = 7.53, which results in P 1 = Hence, the L 2 gain from the disturbance to state is γ * ≤ 7.53.

(E 2 , A 2 ) =

IX. CONCLUSION

In this article, we have considered multi-player stochastic differential games with algebraic constraints embedded in the dynamics. Theoretical tools related to computation of the infinitesimal generators, and derivation of Hamilton-Jacobi-Bellman equation have been developed for Markov jump DAEs. In the game setting, these tools have provided ways to compute Nash equilibrium by solving coupled partial differential equations. In the particular case of linear quadratic dynamic games, the problem of computing Nash equilibrium is reformulated as a solution to coupled Riccati differential equations, which under appropriate stabilizability assumption, reduce to generalized algebraic Riccati equations for infinitehorizon games. From application viewpoint, we have computed minimax robust control for Markov jump DAEs with disturbances in dynamics. As an immediate aftermath of the results developed in this paper, we are interested in addressing the cases where the cost-functionals include a risk sensitivity factor. One can also generalize the scope of the results of this paper to consider systems where the matrices E i are not necessarily square. Several physical systems, such as multibody dynamics in mechanics, are modeled as semi-explicit DAEs in Hessenberg form of index 2 or higher, and it is of interest to generalize the results to cover such cases. Further applications, especially the ones in the context of Leontief input-output systems, are being considered to understand interdependencies and resiliency of critical infrastructures. Another interesting application arises in the context of power systems, as the DAEs are particularly suitable for modeling such systems [START_REF] Groß | Solvability and stability of a power system dae model[END_REF]. It remains to be seen whether the particular structure of these systems bring computational tractability in implementing the solutions proposed in this paper.

Example 3 .

 3 Consider a Markov switched DAE (58) with two subsystems. For the first subsystem, the quadruple (E 1 , A 1 , B 1 , D 1 ) is given by:(E 1 , A 1 ) =

,(B 2 , D 2 ) = 1 0 1 ,Π 2 , C 2 =

 22122 0 1 0 , and once again, we have Π 2 (x) = Π 2,aut x, andC 2 = C 2,aut = im Π 2,aut , where {x ∈ R 3 | x 1 = 0}.We consider continuous-time Markov process over S = {1, 2}, and choose the parameters of the rate-transition matrix to beλ 11 = -1, λ 12 = 1, λ 22 = -2, λ 21 = 2.

  Theorem 1. Consider the N -player noncooperative stochastic differential game where the objective of each player is described by (37a) subject to (37b). Assume that, for each player p ∈ N ,

	z, µ * 1 , . . . , µ * p-1 , u p , µ * p+1 , . . . , µ * N )
	Π * i,p (t, z, u p ) := Π i (t, z, µ * 1 , . . . , µ * p-1 , u p , µ * p+1 , . . . , µ * N )
	C

* i,p (t, u p ) := C i (t, µ * 1 , . . . , µ * p-1 , u p , µ * p+1 , . . . , µ * N ) c * p (t, z, u p , i) := c p (t, z, µ * 1 , . . . , µ * p-1 , u p , µ * p+1 , . . . , µ * N , i). • There exist continuously differentiable functions V p (•, •, i), and F i,p , i ∈ S, which satisfy (A3) and the coupled partial differential equations -∂V p (t, z, i) ∂t = min up∈Up F i,p (t, z, f * i,p (t, z, u p )) + c * p (t, z, u p , i)

  is also regular and indexone. To see that, let S i , T i be the matrices as in Proposition 2, and let S i K i S -1

	i	:=	Ki,11 Ki,12 Ki,21 Ki,22

In 1973, W. Leontief got the Nobel prize in economics for his work on input-output tables which provide a quantitative relationship between different economic sectors in the national economy.

All random variables in this article will be considered with respect to this probability space.

It is assumed that the sample paths of σ are right-continuous, which results in sample-paths of the state being right-continuous. Unless explicitly mentioned, the value of input or state at a time instant denotes its right limit.

For nonlinear DAEs, there are different notions of index that appear in the literature. In this paper, we refer to the "differentiation-index" only and refer the reader to[START_REF] Kunkel | Differential-Algebraic Equations: Analysis and Numerical Solution[END_REF] for further details.

We say that a matrixQ ≥ 0 on C i , if for each x ∈ C i , x Qx ≥ 0. Similarly, Q > 0 on C i if x Qx > 0 for each x ∈ C i , x = 0. Finally, we say that A ≤ B on C i if B -A ≥ 0 on C i .

Dynkin's formula, see for example[START_REF] Meyn | Stability of Markovian processes III: Foster-Lyapunov criteria for continuous-time processes[END_REF], states that E V (x(t)) = E V (x(0)) + E t 0 LV (x(s)) ds .

In our conference paper[START_REF] Tanwani | Feedback Nash equilibrium for Markov jump games under differential-algebraic constraints with application to robust control[END_REF], instead of (44), the feedback strategies were computed under the stronger condition that Π i,p = 0 for each i ∈ S, and each player p ∈ N .

One difference compared to Example 2 is that we write z ∈ R n as z = Π i,aut z + N p=1 Π i,p up and condition[START_REF] Zhu | Game-theoretic methods for robustness, security, and resilience of cyber-physical systems: Games-in-games principle for optimal cross-layer resilient control systems[END_REF] eliminates the products between up and uq which arise due to algebraic constraints.

We use the fact that, for eachx ∈ C i,aut ⊆ R n , there is a unique z ∈ R ν i , ν i = dim(C i,aut ) such that x = Π i,orth z, z = Π i,orth x, and |x| = |z|. This leads to x Q i x ≥ λ min (Π i,orth Q i Π orth i )|x| 2, while we already have the bound V (x, i) ≤ λmax(E i P i E i )|x| 2 .

The notation H ∞ denotes Hardy space of complex-valued functions of a complex variable, which are analytic and bounded in the open right-half complex plane. For a continuous-time, linear time-invariant system, the H ∞norm of the transfer function matrix G is defined as sup ω∈R µmax(G(jω)), where µmax denotes the largest singular value.

ACKNOWLEDGEMENTS

The first author would like to thank Stephan Trenn for discussions related to solution concepts of switched DAEs, and Debasish Chaterjee as the result of Proposition 3 builds on the discussions carried out during their joint work [38]. The authors also thank anonymous reviewers and the associate editor for their helpful comments.

This work was partially supported by the ANR project CONVAN with grant number ANR-17-CE40-0019-01, in part by NSF grants ECCS-1847056, CNS-1544782, and SES-1541164, and ARO grant W911NF1910041.