Joon Kwon

Xiaoxi Li

Miquel Oliu Barton

Vianney Perchet

Thomas Rivera

Tristan Tomala

Xavier Venel

Guillaume Vigeral

Bruno Ziliotto

Florine Alexey

Jean-Bernard Gustaw

Pedro Pascal

Thi Phong

Enfin

Bo- Ris Philippe

Laurent

Hélène Aussi Ahmed

Vincent

Mes premiers remerciements s'adressent à mes directeurs de thèse Stéphane Gaubert et Marianne Akian, qui ont guidé mes premiers pas dans la recherche. L'enthousiasme et la grande culture mathématique de Stéphane ont été une source d'inspiration et de motivation constante durant ces trois années, quand la grande rigueur, le souci d'exactitude et de précision de Marianne m'incitaient à explorer toujours davantage les concepts et les idées. Je leur suis très reconnaissant de m'avoir fait confiance, et je m'estime très chanceux d'avoir travaillé à leur côté.

Je remercie chaleureusement Daniel Hernández-Hernández et Jérôme Renault d'avoir accepté d'être les rapporteurs de cette thèse. Jérôme avait déjà manifesté son intérêt pour mon travail lors de mon tout premier exposé en tant que doctorant, et c'est un honneur et une grande satisfaction de susciter encore son intérêt.

C'est également un grand honneur de compter parmi les membres du jury Rida Laraki, Sylvain Sorin et Wiesław Zielonka. Je remercie en particulier vivement Rida et Sylvain pour m'avoir fait découvrir les premiers, pendant mon master à Jussieu, la théorie des jeux. Cette thèse doit beaucoup à la passion qu'ils communiquent et la bienveillance dont ils font preuve. Je tiens ici à remercier aussi Hélène Frankowska qui, depuis mon M2, a toujours manifesté à mon égard la même bienveillance.

Résumé (en Français) Contexte et motivations

Les jeux stochastiques à somme nulle forment une classe de jeux répétés à deux joueurs, introduite par Shapley [START_REF] Shapley | Stochastic games[END_REF]. Ils décrivent des interactions qui s'étalent sur une période de temps donnée (éventuellement infinie), entre deux agents (appelés « joueurs ») dont les intérêts s'opposent. Ces joueurs prennent des décisions à intervalles de temps fixés, dans le but d'optimiser leur gain respectif sur la durée totale de leur échange. Ce type de jeux apparaît dans plusieurs domaines, dont les sciences informatiques (analyse statique, vérification de modèle), l'économie mathématique, la finance mathématique, ou la dynamique des populations. Le principal trait caractéristique de cette classe de jeux réside dans le fait que les décisions, prises à chaque étape, modifient l'état de la nature selon un processus stochastique contrôlé. Ces décisions génèrent également un paiement instantané d'un joueur vers l'autre, qui dépend en outre de l'état courant de la nature. Ainsi, à chaque étape, les joueurs doivent trouver un compromis entre leur gain instantané et l'évolution de l'état, qui influence les paiements futurs.

Étant donné une évaluation de la suite des paiements générés par les décisions prises par les joueurs à chaque étape, on dit que le jeu à une valeur v si le joueur qui reçoit les paiements (joueur « maximisant ») peut faire en sorte de recevoir au minimum cette quantité v, et si le joueur qui donne les paiements (joueur « minimisant ») peut s'assurer de ne pas perdre plus que cette quantité v, indépendamment de la stratégie suivie par le joueur adverse. Un problème standard est de savoir si la valeur calculée à partir de la moyenne arithmétique des k premiers paiements a une limite quand l'horizon k tend vers l'infini. On appelle cette limite la valeur limite (ou encore, la valeur asymptotique). Ce problème a été largement étudié pour différentes classes de jeux répétés, par un grand nombre d'auteurs dont Bewley, Kohlberg, Mertens, Neyman, Sorin, Zamir. Nous renvoyons le lecteur aux articles [MZ72, Koh74, BK76, KN81, MN81, RS01b, Ney03, Sor03]. En particulier, l'existence de la valeur limite des jeux stochastiques dont l'ensemble des états et l'ensemble des actions sont finis a été établie dans [START_REF] Bewley | The asymptotic theory of stochastic games[END_REF]. Cependant, quand les espaces d'action sont infinis, cette valeur limite peut ne pas exister, même si les données du problèmes sont standards (compacité des espaces d'action, continuité des fonctions de transition) [START_REF] Vigeral | A zero-zum stochastic game with compact action sets and no asymptotic value[END_REF].

D'autres problèmes classiques proviennent des jeux stochastiques à paiement moyen, pour les-v quels la valeur est déterminée à partir de la limite (lim sup ou lim inf) des moyennes de Cesàro des paiements d'étapes. Dans ce cas, au lieu d'optimiser sur une période de temps finie, puis de considérer le comportement de la valeur quand les joueurs deviennent de plus en plus patient (horizon k → ∞), on étudie un jeu qui comporte une infinité d'étapes. A l'origine, les jeux à paiement moyen ont été étudiés dans un cadre déterministe par Ehrenfeucht et Mycielski [START_REF] Ehrenfeucht | Positional strategies for mean payoff games[END_REF], et Gurvich, Karzanov er Khachiyan [START_REF] Gurvich | Cyclic games and finding minimax mean cycles in digraphs[END_REF]. Les premiers ont montré l'existence de stratégies optimales stationnaires, c'est-à-dire dont les décisions prises par les joueurs à chaque étape dépendent uniquement de l'état courant (et non de l'ensemble des actions passées). Les seconds ont donné un algorithme pour déterminer celles-ci. Cependant, l'existence d'un algorithme en temps polynomial pour résoudre les jeux à paiement moyen demeure, depuis lors, une question ouverte. Il est à noter que ces jeux [START_REF] Zwick | The complexity of mean payoff games on graphs[END_REF], ainsi que d'autres classes de jeux stochastiques avec états absorbants [START_REF] Condon | The complexity of stochastic games[END_REF] appartiennent à la classe de complexité NP ∩ coNP. Ainsi, ils font partie des rares problèmes qui sont dans NP ∩ coNP et pour lesquels aucun algorithme en temps polynomial n'est connu. Pour une comparaison des différentes classes de jeux concernées par ces résultats, nous renvoyons à [START_REF] Andersson | The complexity of solving stochastic games on graphs[END_REF].

Les jeux stochastiques à paiement moyen soulèvent aussi des problèmes théoriques, tels que l'existence de la valeur ou de stratégies optimales. Ces problèmes ont été étudiés non seulement en théorie des jeux [START_REF] Filar | Competitive Markov decision processes[END_REF], mais aussi en théorie du contrôle (c'est-à-dire pour des jeux à un joueur) sous la dénomination de problèmes de contrôle ergodique. Ceux-ci sont standards pour les processus de décisions Markoviens [START_REF] Puterman | Markov decision processes: discrete stochastic dynamic programming[END_REF][START_REF] Hernández-Lerma | Discrete-time Markov control processes[END_REF], ainsi que pour les problèmes de contrôle stochastique sensible au risque, étudiés en temps discret par plusieurs auteurs dont Hernández-Hernández et Marcus [START_REF] Hernández-Hernández | Risk sensitive control of Markov processes in countable state space[END_REF], Fleming et Hernández-Hernández [START_REF] Fleming | Risk-sensitive control of finite state machines on an infinite horizon[END_REF][START_REF] Fleming | Risk-sensitive control of finite state machines on an infinite horizon[END_REF], voir aussi [START_REF] Cavazos-Cadena | A characterization of the optimal risk-sensitive average cost in finite controlled Markov chains[END_REF] ou [START_REF] Anantharam | A variational formula for risk-sensitive reward[END_REF] pour des résultats plus récents.

Une manière d'analyser les problèmes évoqués ci-dessus est d'exploiter la structure récursive des jeux stochastiques. Celle-ci s'exprime dans leur opérateur de programmation dynamique, appelé opérateur de Shapley, dont il est possible de déduire des informations utiles en étudiant leur propriétés analytiques. Cette méthode, appelé « approche opérateur », a été développée par Kohlberg [START_REF] Kohlberg | Repeated games with absorbing states[END_REF][START_REF] Kohlberg | Invariant half-lines of nonexpansive piecewise-linear transformations[END_REF], Kohlberg et Neyman [START_REF] Kohlberg | Asymptotic behavior of nonexpansive mappings in normed linear spaces[END_REF], Rosenberg et Sorin [START_REF] Rosenberg | An operator approach to zero-sum repeated games[END_REF], Neyman [START_REF] Neyman | Stochastic games and nonexpansive maps[END_REF], Sorin [START_REF] Sorin | Asymptotic properties of monotonic nonexpansive mappings[END_REF], Renault [START_REF] Renault | The value of repeated games with an informed controller[END_REF], entre autre. En particulier, la valeur limite des jeux stochastiques ainsi que la valeur des jeux stochastiques à paiement moyen existent et sont indépendantes de l'état initial, si une certaine équation non linéaire aux valeurs propres admet une solution. Cette équation est appelée équation ergodique (en Anglais, « average reward optimality equation »). Quand l'espace d'état est fini, disons {1, . . . , n}, cette équation s'écrit

T (u) = λe + u , λ ∈ R , u ∈ R n , (0.1)
où T : R n → R n est l'opérateur de Shapley et e est le vecteur unité de R n . Le scalaire λ, appelé constante ergodique, fournit la valeur pour n'importe quel état initial. En outre, il est possible de déduire du vecteur u, appelé vecteur de biais, des stratégies optimales stationnaires. Pour cela, il suffit d'identifier des actions, dans la représentation minimax de T , qui réalisent le maximum et le minimum dans (0.1). Ainsi, comprendre la structure de l'espace des vecteurs de biais est un problème théorique d'une importance fondamentale, puisque cette espace permet d'avoir accès à l'ensemble des stratégies optimales stationnaires. Dans le cas déterministe à un joueur, la représentation des vecteurs de biais est bien comprise. L'analyse repose sur la théorie spectrale max-plus, dont l'origine remonte aux travaux de Romanovsky [START_REF] Romanovsky | Optimization of stationary control of a discrete deterministic process[END_REF], Gondran et Minoux [START_REF] Gondran | Valeurs propres et vecteurs propres dans les dioïdes et leur interprétation en théorie des graphes[END_REF] et Cunnighame-Green [START_REF] Cuninghame-Green | Minimax algebra[END_REF]. Nous renvoyons aussi le lecteur à [START_REF] Maslov | of Advances in Soviet Mathematics[END_REF][START_REF] Baccelli | Synchronization and linearity[END_REF] pour un aperçu plus ample de cette théorie. Quand l'espace d'état et le temps sont continus, une équation aux dérivées partielles de type Hamilton-Jacobi remplace l'équation de programmation dynamique (0.1). Les solutions de cette équation vi aux dérivées partielles ont été étudiées dans le cadre de la théorie KAM faible, en lien avec des problèmes de contrôle optimal en temps continu, c.f. la monographie de Fathi [START_REF] Fathi | Weak KAM theorem in Lagrangian dynamics[END_REF]. Dans le cas stochastique, la structure de l'espace des vecteurs de biais est également bien comprise dans le cadre des problèmes à un joueur en temps discret, c.f. Akian et Gaubert [START_REF] Akian | Spectral theorem for convex monotone homogeneous maps, and ergodic control[END_REF]. Mais dans le cadre des problèmes à deux joueurs, la structure de cet espace est bien moins comprise, et l'unicité du vecteur de biais, à l'addition d'un vecteur constant près, est déjà un problème important, notamment pour des raisons algorithmiques. En effet, des phénomène de cyclage peuvent se manifester dans l'algorithme d'itération sur les politiques de Hoffman et Karp [START_REF] Hoffman | On nonterminating stochastic games[END_REF], quand l'unicité n'a pas lieu (voir l'exemple dans [START_REF] Akian | Policy iteration algorithm for zero-sum multichain stochastic games with mean payoff and perfect information[END_REF]).

Pour résoudre l'équation (0.1), il est habituel de supposer valides des « conditions de communication », basées sur la structure des probabilités de transition. Par exemple, une condition classique impose aux matrices de transition d'être toutes irréductibles, voir [START_REF] Bather | Optimal decision procedures for finite Markov chains. II. Communicating systems[END_REF]. De manière alternative, il est possible de faire appel aux techniques de la théorie de Perron-Frobenius non linéaire. En effet, trouver une solution à l'équation (0.1) est équivalent à trouver un vecteur u ∈ R n dont toutes les coordonnées sont strictement positives, et un scalaire λ strictement positif, tels que l'égalité f (u) = λu soit satisfaite pour une certaine fonction f , définie de l'intérieur du cône standard positif de R n dans lui-même, et qui est homogène de degré un et préserve l'ordre partiel standard de R n .

La théorie de Perron-Frobenius non linéaire s'intéresse plus généralement aux fonctions f , définies d'un cône convexe fermé K d'un espace de Banach dans lui-même, qui sont non expansives par rapport à une métrique induite par K. Selon les propres mots de Nussbaum dans [START_REF] Nussbaum | Hilbert's projective metric and iterated nonlinear maps[END_REF], « la question centrale [. . . ] et la difficulté analytique irréductible » est de savoir si f possède un vecteur propre dans l'intérieur de K, c'est-à-dire, si il existe un vecteur u ∈ int(K) tel que f (u) = λu (0.2) pour un certain scalaire λ > 0. L'intérêt pour l'équation (0.2) remonte aux travaux de Kreȋn et Rutman [START_REF] Kreȋn | Linear operators leaving invariant a cone in a Banach space[END_REF]. Elle a été particulièrement étudiée par Nussbaum [START_REF] Nussbaum | Hilbert's projective metric and iterated nonlinear maps[END_REF][START_REF] Nussbaum | Iterated nonlinear maps and Hilbert's projective metric[END_REF], qui a donné des conditions analytiques garantissant l'existence d'une solution. Ces conditions reposent sur la théorie du point fixe. Elle a aussi été étudiée d'un point de vue combinatoire par Gaubert et Gunawardena [START_REF] Gaubert | The Perron-Frobenius theorem for homogeneous, monotone functions[END_REF], dans le cas où K est l'orthant positif de R n et f préserve l'ordre standard associé et est homogène de degré un. Remarquons que ces dernières conditions imposent en particulier que f est non expansive par rapport à la métrique projective de Hilbert. Il est à noter que l'unicité du vecteur propre u, au produit d'un scalaire strictement positif près, est aussi un problème important, qui a été étudié dans les références mentionnées ci-dessus.

Mentionnons que la théorie de Perron-Frobenius non linéaire a des applications dans des domaines aussi variés que la théorie des jeux, la théorie de l'information, la théorie des systèmes dynamiques, la biologie mathématique, l'économie mathématique. En outre, des liens avec les tenseurs positifs ont récemment été mis en lumière (voir le survey [START_REF] Chang | A survey on the spectral theory of nonnegative tensors[END_REF]). Nous renvoyons le lecteur à la monographie [START_REF] Lemmens | Nonlinear Perron-Frobenius theory[END_REF] pour un vaste panorama de la théorie de Perron-Frobenius non linéaire.

Dans la présente thèse, nous utilisons des outils de la théorie de Perron-Frobenius non linéaire, introduits dans [START_REF] Nussbaum | Hilbert's projective metric and iterated nonlinear maps[END_REF] et [START_REF] Gaubert | The Perron-Frobenius theorem for homogeneous, monotone functions[END_REF], afin de traiter les problèmes en lien avec l'équation ergodique (0.1), évoqués ci-dessus. Le Chapitre 2 est dédié à la présentation détaillée du cadre de ce travail. vii

Contributions

Dans cette thèse, nous présentons des résultats nouveaux concernat les jeux stochastiques avec paiement moyen et espace d'état fini. L'ensemble du travail exposé dans ce manuscrit repose sur la notion centrale d'« ergodicité » des jeux stochastiques, que nous relions à l'existence d'une solution pour toute une famille de problèmes ergodiques (0.1). Pour caractériser cette notion, nous employons des méthodes de théorie des graphes, impliquant notamment les hypergraphes. Nous étudions aussi l'unicité, à l'addition d'un vecteur constant près, du vecteur de biais. Nous mentionnons que certains des résultats obtenus peuvent se formuler dans le cadre de la théorie de Perron-Frobenius non linéaire, dont les outils sont largement utilisés. Dans les paragraphes suivant, nous passons en revue les principaux résultats.

Dans les Chapitres 4 à 6, nous introduisons et donnons une caractérisation de la notion d'ergodicité pour les jeux stochastiques avec espace d'état fini. Nous empruntons la terminologie au cas des chaînes de Markov.

Selon la définition de Kemeny et Snell [START_REF] Kemeny | Finite Markov chains[END_REF], l'ergodicité d'une chaîne de Markov finie peut être caractérisée par plusieurs propriétés de sa matrice de transition : unicité de la mesure de probabilité invariante ; unicité de la classe finale ; moyennes de Cesàro des paiements obtenus le long d'une trajectoire qui tendent vers une limite indépendante de l'état initial ; existence d'une solution pour toute une famille de problèmes spectraux.

Nous montrons que la plupart de ces caractérisations reste valides dans le cas des jeux stochastiques, avec toutefois une différence majeure concernant la propriété relative à la théorie des graphes : le graphe dirigé associé à la matrice de transition d'une chaîne de Markov finie doit être remplacé par une paire d'hypergraphes dirigés.

Ainsi, nous dirons qu'un jeu, ou son opérateur de Shapley T : R n → R n , est ergodique si l'équation ergodique (0.1) admet une solution pour tous les opérateurs qui s'écrivent sous la forme g + T avec g ∈ R n . Ces derniers correspondent à des versions modifiées du jeu original, consistant à ajouter la quantité g i à tous les paiements dans l'état i. Il s'ensuit que si le jeu est ergodique, la valeur limite est indépendante de l'état initial pour n'importe quelle perturbation additive des paiements.

Dans le Chapitre 4, nous considérons d'abord le cas des jeux dont la fonction de paiement est bornée. Nous montrons, dans le Théorème 4.6, que l'ergodicité est équivalente à l'unicité, à l'addition d'un vecteur constant près, du point fixe d'un opérateur de Shapley auxiliaire. Ce dernier est positivement homogène, c'est-à-dire qu'il commute avec le produit par un scalaire strictement positif. De tels opérateurs apparaissent dans les jeux dont les paiements instantanés sont nuls, et pour cette raison nous les appelons opérateurs de Shapley sans paiement.

Cette caractérisation de l'ergodicité nous amène à considérer, dans le Chapitre 5, le problème de l'existence de points fixes d'opérateurs de Shapley sans paiement dont l'arg min ou l'arg max est fixé. Nous donnons une condition combinatoire en termes de correspondance de Galois (Théorème 5.7) ainsi qu'en termes d'hypergraphes (Théorème 5.12), et nous proposons un algorithme en temps polynomial (Algorithme 1) qui résout ce problème. Mentionnons que ce problème est intéressant en lui-même puisque la structure de l'ensemble des vecteurs de biais d'un opérateur de Shapley polyédral (associé à un jeu à information parfaite avec espaces d'action finis) est donné localement par l'ensemble des points fixes d'un opérateur de Shapley sans paiement. Nous déduisons ensuite une caractérisation combinatoire de l'ergodicité, qui fait intervenir une paire d'hypergraphes (Corollaire 5.13), dont la construction ne dépend (sous certaines hypothèses) que de la structure du jeu, et non de la valeur que peuvent prendre les différentes données. Traduit dans le langage de la théorie des jeux, nous montrons que l'ergodicité se caractérise par l'absence de « territoires » (dominions, en Anglais) disjoints pour viii chaque joueur (Théorème 5.16), un territoire étant un ensemble d'état qu'un joueur peut rendre invariant. Nous concluons que vérifier l'ergodicité est coNP-dur, et dépend essentiellement du nombre d'état (Théorème 5.28) : si les états sont fixés, ce problème peut être résolu en temps polynomial par rapport au nombre d'actions.

Nous traitons ensuite le cas général, c'est-à-dire celui des jeux stochastiques dont la fonction de paiement est potentiellement non bornée. D'après un corollaire de Gaubert et Gunawardena [START_REF] Gaubert | The Perron-Frobenius theorem for homogeneous, monotone functions[END_REF], un jeu stochastique est ergodique, au sens que nous venons de définir, si tous les espaces de tranche sont bornés pour la semi-norme de Hilbert. Ces espaces sont des ensembles invariants par T , définis pour tout α, β ∈ R par {x ∈ R n | αe + x T (x) βe + x}. Dans le Chapitre 4, nous montrons que cette condition est non seulement suffisante, mais aussi nécessaire (Théorème 4.21). Pour établir ce résultat, nous employons la théorie des opérateurs accrétifs. Plus précisément, nous établissons une condition nécessaire de surjectivité pour de tels opérateurs définis sur un espace normé de dimension finie (Théorème 4.16). En guise de conséquence, nous obtenons un résultat de stabilité, concernant l'existence de points fixes pour toute perturbation additive d'une application non expansive (Corollaire 4.20).

Dans le Chapitre 6, nous caractérisons de manière combinatoire la propriété selon laquelle tous les espaces de tranche sont bornés pour la semi-norme de Hilbert, répondant ainsi à une question ouverte posée dans [START_REF] Gaubert | The Perron-Frobenius theorem for homogeneous, monotone functions[END_REF]. Cette caractérisation se manifeste par l'absence de territoires disjoints pour chaque joueur dans un jeu stochastique auxiliaire, dont les actions sont définies par le comportement asymptotique de T le long de certaines demi-droites (Théorème 6.2). Nous donnons également une caractérisation en termes d'hypergraphes (Théorème 6.9). Sous l'hypothèse que la fonction de paiement est bornée, cette dernière caractérisation est équivalente à celle fournie dans le Chapitre 5. Quand un seul joueur a des ensembles d'action non triviaux (c'est-à-dire quand T est convexe), nous montrons que ces conditions se simplifient.

Dans les Chapitres 7 et 8 nous étudions l'unicité, à l'addition d'un vecteur constant près, des vecteurs de biais d'un opérateur de Shapley T . Nous commençons par traiter, dans le Chapitre 7, le cas des jeux stochastiques ergodiques à information parfaite et espaces d'action finis, c'est-à-dire quand T est une application polyédrale telle que l'équation ergodique (0.1) admet une solution pour tout opérateur g + T avec g ∈ R n . Nous montrons que les vecteurs de perturbation g pour lesquels l'unicité, à l'addition d'un vecteur constant près, n'a pas lieu sont contenus dans les cellules d'un complexe polyédral, dont la codimension est au moins 1 (Théorème 7.8). L'application de ce résultat à l'algorithme d'itération sur les politiques nous permet d'obtenir un schéma de perturbation pour traiter les instances dégénérées de jeux stochastiques (Théorème 7.22).

Ensuite, dans le Chapitre 8, nous généralisons cette propriété « générique » à n'importe quel opérateur de Shapley. Pour cela, nous nous basons sur l'« approche opérateur accrétif » introduite dans le Chapitre 4. Nous montrons que si l'équation ergodique (0.1) admet une solution pour toute perturbation locale de T , alors le vecteur de biais est unique, à l'addition d'un vecteur constant près, pour une perturbation locale « générique » (Théorème 8.6).

Les opérateurs de Shapley définis sur R n sont caractérisés par deux propriétés fondamentales : ils préservent l'ordre standard partiel de R n et ils sont additivement homogènes, c'est-àdire qu'ils commutent avec l'addition par un vecteur constant. Cette caractérisation repose sur plusieurs représentations minimax des opérateurs vérifiant ces propriétés, voir [START_REF] Kolokoltsov | On linear, additive, and homogeneous operators in idempotent analysis[END_REF][START_REF] Rubinov | Topical and sub-topical functions, downward sets and abstract convexity[END_REF].

Les opérateurs de Shapley qui sont en outre positivement homogènes occupent une place centrale dans la caractérisation de l'ergodicité (Chapitre 4) et sont le principal objet d'étude du Chapitre 5. Dans le Chapitre 3, nous donnons une représentation minimax de tels opérateurs (Corollaire 3.12), justifiant ainsi leur dénomination d'« opérateurs de Shapley sans paiement ».

Pour parvenir à cette représentation, nous montrons qu'une fonction à valeur réelle sur un espace vectoriel topologique est positivement homogène de degré un et non expansive par rapport à une norme faible de Minkowski si, et seulement si, elle peut s'écrire comme le minimax des formes linéaires qui sont non expansives par rapport à cette même norme (Théorème 3.8).

Présentation des chapitres

Chapitre 2

Dans le Chapitre 2, nous introduisons le modèle des jeux stochastiques à somme nulle, nous présentons l'approche opérateur, et nous établissons le lien avec la théorie de Perron-Frobenius non linéaire. En particulier, nous appellerons opérateur de Shapley (abstrait) sur R n , tout opérateur de R n qui est monotone (M) et additivement homogène (AH) Chapitre 3

Il existe plusieurs théorèmes de représentation minimax des opérateurs de R n qui sont monotones (M) et additivement homogènes (AH) : caractérisation en termes de jeux stochastiques [START_REF] Kolokoltsov | On linear, additive, and homogeneous operators in idempotent analysis[END_REF] (Théorème 2.2), ou en termes de jeux répétés à information parfaite avec transitions déterministes [START_REF] Rubinov | Topical and sub-topical functions, downward sets and abstract convexity[END_REF][START_REF] Gunawardena | From max-plus algebra to nonexpansive mappings: a nonlinear theory for discrete event systems[END_REF] (Théorème 2.3). Dans le cadre de la convexité abstraite, mentionnons également [START_REF] Dutta | Monotonic analysis over cones[END_REF][START_REF] Dutta | Monotonic analysis over cones[END_REF], qui fournissent une caractérisation d'opérateurs vérifiant des propriétés similaires dans un cadre multiplicatif. En outre, on obtient le même type de théorèmes de représentation par la dualité de Fenchel-Legendre quand les opérateurs sont en plus convexes (dans ce cas, il n'y a plus d'infimum dans la formule de représentation). En dimension infinie, cela a été appliqué au cas des mesures de risque convexes [START_REF] Föllmer | Convex measures of risk and trading constraints[END_REF] par exemple.

Une sous-classe importante d'opérateurs monotones et additivement homogènes est constituée par ceux qui sont aussi positivement homogènes (PH). C'est notamment le cas des opérateurs de récession, utilisés par Gaubert et Gunawardena [START_REF] Gaubert | The Perron-Frobenius theorem for homogeneous, monotone functions[END_REF] en théorie de Perron-Frobenius non linéaire. C'est aussi le cas, en dimension infinie, des mesures de risque cohérentes [START_REF] Artzner | Coherent measures of risk[END_REF][START_REF] Delbaen | Coherent risk measures on general probability spaces[END_REF].

Lorsque T : R n → R n est l'opérateur de Shapley d'un jeu stochastique Γ(r) dont la fonction de paiement r est bornée, l'opérateur de récession T est bien défini, et est lui-même l'opérateur de Shapley du jeu stochastique Γ(0), dont la fonction de paiement est nulle. Cependant, la réciproque ne peut pas être déduite des théorèmes de représentation évoqués ci-dessus : Si T : R n → R n est un opérateur de Shapley (abstrait) qui est positivement homogène, ni le Théorème 2.2, ni le Théorème 2.3 ne fournissent une formule de représentation de T comme opérateur de programmation dynamique d'un jeu dont la fonction de paiement est nulle. Dans le Chapitre 3, nous établissons une telle caractérisation. Ce résultat justifie le fait que nous appellerons, dans la suite, opérateur de Shapley (abstrait) sans paiement sur R n , tout opérateur de R n qui est monotone (M), additivement homogène (AH) et positivement homogène (PH).

Les deux propriétés qui définissent les opérateurs de Shapley (M) et (AH), sont équivalentes à la propriété d'être non expansif par rapport à une certaine norme faible de Minkowski. Ces dernières sont des substituts de normes qui ne sont pas nécessairement symétriques, ou ne séparent pas nécessairement les points. Elles ont été étudiées en géométrie métrique, en particulier par Papadopoulos et Troyanov [START_REF] Papadopoulos | Weak Minkowski spaces[END_REF], et apparaissent naturellement dans l'étude des opérateurs de Shapley, voir [START_REF] Gaubert | A maximin characterisation of the escape rate of nonexpansive mappings in metrically convex spaces[END_REF].

Pour obtenir une caractérisation minimax des opérateurs monotones, additivement et positivement homogènes, nous commençons par établir un théorème général de représentation minimax qui s'applique à des fonctions réelles définies sur un espace vectoriel topologique, et qui sont non expansives par rapport à une norme faible de Minkowski. Nous caractérisons x ensuite les fonctions non expansives qui sont en outre positivement homogènes. Nous en déduisons comme corollaire notre principale application : un théorème de représentation des opérateurs de programmation dynamique des jeux stochastiques dont les paiements de transition sont nuls. Nous en déduisons également qu'il est possible d'approcher ces opérateurs par une application polyédrale positivement homogène. Ce type d'approximation peut être utilisée dans le cadre des méthodes d'éléments finis max-plus pour la résolution numérique d'équations aux dérivées partielles de Hamilton-Jacobi, voir [START_REF] Mceneaney | Max-plus methods for nonlinear control and estimation[END_REF][START_REF] Akian | The max-plus finite element method for solving deterministic optimal control problems: basic properties and convergence analysis[END_REF]. Nous déduisons aussi un théorème de représentation de mesures de risque non convexes.

Chapitres 4 à 6 L'existence de la valeur limite d'un jeu stochastique à espace d'état fini est un problème qui peut se résoudre en prouvant l'existence d'une solution à l'équation ergodique (0.1), faisant intervenir l'opérateur de Shapley T du jeu. Pour que cette équation admette une solution, il est nécessaire et suffisant qu'une orbite de T soit bornée pour la semi-norme de Hilbert (Théorème 2.6). Prouver que cette dernière propriété est satisfaite est un problème difficile en général, qui dépend non seulement de la structure du jeu (c'est-à-dire, le support des probabilités de transition), mais aussi de la valeur des différents paramètres (paiements et probabilités de transition). En suivant les idées de Gaubert et Gunawardena [START_REF] Gaubert | The Perron-Frobenius theorem for homogeneous, monotone functions[END_REF], un problème mieux posé consiste à trouver des conditions pour lesquelles tous les éléments d'une famille donnée d'espaces invariants par T sont bornés pour la semi-norme de Hilbert. De telles conditions ne dépendent habituellement que de la structure du jeu (voir, par exemple, Théorème 2.8). Dans ce cas, l'équation ergodique a une solution non seulement pour T , mais aussi pour toute une famille d'opérateurs perturbés.

Le cas des chaînes de Markov ergodiques est un exemple élémentaire illustrant cette propriété de stabilité. D'après Kemeny et Snell [START_REF] Kemeny | Finite Markov chains[END_REF], une chaîne de Markov finie est ergodique si sa matrice de transition satisfait l'une des assertions listées dans le théorème ci-dessous. Nous rappelons à cet effet que pour une matrice stochastique carrée P de dimension n, le graphe dirigé associé est constitué des noeuds 1, . . . , n et des arcs (i, j) tels que P ij > 0. Une classe de la matrice P est un ensemble maximal de noeuds tel que pour n'importe quelle paire de noeuds dans cet ensemble, il existe un chemin dans le graphe de P reliant l'un à l'autre. Une classe est dite finale si tout chemin issu d'un noeud de cette classe reste dans celle-ci. Le lecteur pourra trouver la preuve du théorème suivant dans [START_REF] Kemeny | Finite Markov chains[END_REF] et [START_REF] Berman | Nonnegative matrices in the mathematical sciences[END_REF] Theorem 0.1. Soit P ∈ R n×n une matrice stochastique. Les assertions suivantes sont équivalentes :

(i) tout vecteur v ∈ R n tel que P v = v est constant ; (ii) pour tout vecteur g ∈ R n , il existe une paire (λ, u) ∈ R × R n telle que g + P u = λe + u ; (iii) pour tout vecteur g ∈ R n , la limite des moyennes de Cesàro lim k→∞ (g + P g + • • • + P k-1 g)/k
est un vecteur constant ; (iv) Le graphe dirigé associé à P a une unique classe finale ; (v) la matrice stochastique P a une unique mesure de probabilité invariante.

Une chaîne de Markov finie peut être vue comme un jeu stochastique à zéro joueur. Si P ∈ R n×n est sa matrice de transition, et si g i est le paiement reçu quand l'état i ∈ [n] est visité, l'opérateur de Shapley s'écrit alors T (x) = g + P x pour tout x ∈ R n . On déduit immédiatement que son opérateur de récession est égal à T (x) = P x. Ainsi, en utilisant le vocabulaire de la théorie des jeux, l'ergodicité d'une chaîne de Markov finie peut se caractériser soit par le fait que tous les points fixes de l'opérateur de récession T sont constants (Point (i)), soit par le fait que l'équation ergodique a une solution pour tout opérateur g + T avec g ∈ R n (Point (ii)), soit encore par le fait que la valeur limite est constante pour tout vecteur de paiement g ∈ R n xi (Point (iii)). Une question naturelle est de savoir si ces caractérisations s'étendent aux jeux stochastiques à espace d'état fini.

Rappelons que la stabilité de l'existence d'une valeur limite constante, basée sur des propriétés structurelles, a déjà été étudiée dans plusieurs cadres. Dans le cas des chaînes de Markov finies, pour lesquelles on considère un critère sensible au risque basé sur le coût moyen en temps long, Cavazos-Cadena et Hernández-Hernández [START_REF] Cavazos-Cadena | Necessary and sufficient conditions for a solution to the risk-sensitive Poisson equation on a finite state space[END_REF] ont donné une condition nécessaire et suffisante sur la matrice de transition pour que l'équation de Poisson -une équation de type programmation dynamique -ait une solution quelle que soit la fonction de coût associée à la chaîne. Toujours motivés par des problèmes dits sensibles au risque, les mêmes auteurs [START_REF] Cavazos-Cadena | Poisson equations associated with a homogeneous and monotone function: necessary and sufficient conditions for a solution in a weakly convex case[END_REF] ont donné, sous une hypothèse de convexité faible, une condition nécessaire et suffisante pour que le problème spectral non linéaire (0.2) ait une solution pour toute perturbation des paiements. En théorie du contrôle optimal (espace d'état et temps continu), Arisawa [START_REF] Arisawa | Ergodic problem for the Hamilton-Jacobi-Bellman equation. I. Existence of the ergodic attractor[END_REF][START_REF] Arisawa | Ergodic problem for the Hamilton-Jacobi-Bellman equation[END_REF] a étudié le problème ergodique pour les équations de Hamilton-Jacobi-Bellman, et a établi un lien entre l'existence de la constante ergodique pour toute fonction de coût ne dépendant que de l'état et l'existence d'un « attracteur ergodique », qui ne dépend que de la dynamique.

Dans les Chapitres 4 à 6, nous étendons la notion d'ergodicité aux jeux stochastiques à somme nulle. Ainsi, nous dirons qu'un jeu (ou de manière équivalente, son opérateur de Shapley T : R n → R n ) est ergodique si l'équation ergodique (0.1) a une solution pour tout opérateur g + T avec g ∈ R n .

Dans le Chapitre 4, en nous inspirant des travaux de Gaubert et Gunawardena [START_REF] Gaubert | The Perron-Frobenius theorem for homogeneous, monotone functions[END_REF], nous commençons par caractériser l'ergodicité en termes d'opérateur de récession. Pour cela, nous supposons que l'opérateur de Shapley et son opérateur de récession ont le même comportement asymptotique. Ensuite, nous relâchons cette hypothèse et caractérisons l'ergodicité en toute généralité, en termes d'espaces de tranche. Ce résultat s'appuie sur le lien entre les applications non expansives et les opérateurs accrétifs.

Dans le Chapitre 5 et le Chapitre 6, nous traitons de l'aspect combinatoire de ces conditions. En particulier, nous les formulons en termes de relation d'accessibilité dans des hypergraphes, ainsi qu'en termes de « territoires » (ou dominions).

Chapitres 7 et 8

La description de l'ensemble des vecteurs de biais d'un jeu stochastique à somme nulle est un problème fondamental. Dans le cas des problèmes à un joueur, c'est-à-dire pour les problèmes de contrôle optimal en temps discret, la représentation des vecteurs de biais et leur relation avec les stratégies optimales sont bien comprises, que les transitions soient déterministes ou stochastiques (processus de décision Markoviens). Quand les transitions sont déterministes, l'analyse de ces problèmes repose sur la théorie spectrale max-plus, qui remonte aux travaux de Romanovsky [START_REF] Romanovsky | Optimization of stationary control of a discrete deterministic process[END_REF], Gondran et Minoux [START_REF] Gondran | Valeurs propres et vecteurs propres dans les dioïdes et leur interprétation en théorie des graphes[END_REF], et Cuninghame-Green [START_REF] Cuninghame-Green | Minimax algebra[END_REF]. Pour une présentation de la théorie spectrale max-plus, le lecteur pourra se référer à [MS92, BCOQ92, ABG13]. Kontorer et Yakovenko [START_REF] Kontorer | Nonlinear semigroups and infinite horizon optimization[END_REF] et Kolokoltsov et Maslov [START_REF] Kolokoltsov | Idempotent analysis and its applications[END_REF] se sont en particulier intéressés aux problèmes d'optimisation en horizon infini et aux problèmes à paiement moyen. Dans ce cadre, l'ensemble des vecteurs de biais possède la structure d'un cône max-plus, c'est-à-dire qu'il est invariant par combinaisons linéaires max-plus, et il est engendré par une unique famille minimale qui s'identifie au support des mesures maximisantes dans la formulation du problème de contrôle optimal sous forme de programme linéaire. Ces générateurs « extrêmes » correspondent également aux états récurrents des trajectoires optimales infinies. Une interprétation combinatoire de certains de ces résultats, en termes de complexes polyédrales, a récemment été proposée par Sturmfels et Tran [START_REF] Sturmfels | Combinatorial types of tropical eigenvectors[END_REF]. L'équation ergodique (0.1) et la structure des vecteurs de biais a aussi été étudié dans le cas d'espaces d'état continus. Un premier cadre xii d'étude est fourni par la théorie spectrale max-plus en dimension infinie (voir Akian, Gaubert et Walsh [START_REF] Akian | The max-plus Martin boundary[END_REF]), un second par la théorie KAM faible (voir Fathi [START_REF] Fathi | Weak KAM theorem in Lagrangian dynamics[END_REF]). En outre, quand les transitions sont stochastiques, la structure de l'ensemble des vecteurs de biais est également bien comprise dans le cas d'espaces d'état finis, voir Akian et Gaubert [START_REF] Akian | Spectral theorem for convex monotone homogeneous maps, and ergodic control[END_REF].

Dans le cas des jeux à deux joueurs, la structure de l'ensemble des vecteurs de biais est bien moins comprise. Le problème de comprendre quand le vecteur de biais est unique, à une constante additive près, est déjà intéressant, notamment pour des questions algorithmiques. C'est particulièrement le cas pour l'algorithme d'itération sur les politiques de Hoffman et Karp, puisque la non unicité est en général source d'instabilités numériques ou de situations dégénérées. Hoffman et Karp [START_REF] Hoffman | On nonterminating stochastic games[END_REF] ont introduit cet algorithme pour résoudre les jeux stochastiques à somme nulle avec paiement moyen, information parfaite, et espaces d'état et d'action finis. Ils ont montré que l'itération sur les politiques termine si pour chaque paire de stratégie stationnaires des joueurs, la chaine de Markov associée est irréductible. Cependant, des phénomènes de cyclage peuvent apparaître si cette hypothèse d'irréductibilité n'est pas satisfaite, ce qui est le cas pour beaucoup de classes de jeux -en particulier, elle n'est essentiellement jamais satisfaite pour les jeux avec transitions déterministes. Pour contourner cette obstruction, des raffinements ont été proposés par Cochet-Terrasson et Gaubert [START_REF] Cochet-Terrasson | A policy iteration algorithm for zero-sum stochastic games with mean payoff[END_REF], Akian, Cochet-Terrasson, Detournay et Gaubert [START_REF] Akian | Policy iteration algorithm for zero-sum multichain stochastic games with mean payoff and perfect information[END_REF], Bourque et Raghavan [START_REF] Bourque | Policy improvement for perfect information additive reward and additive transition stochastic games with discounted and average payoffs[END_REF]. Mais le prix à payer pour traiter les situations de non unicité, est une augmentation de la complexité de l'algorithme. Ainsi, il est très intéressant de comprendre quand de tels détails techniques peuvent être évités.

Dans le Chapitre 7, nous considérons le problème de l'unicité du vecteur de biais de jeux stochastiques avec information parfaite et espaces d'état et d'action finis, quand ceux-ci sont ergodiques, c'est-à-dire quand l'équation ergodique (0.1) a une solution pour toute perturbation des paiements de transitions. Le résultat principal (Théorème 7.8) montre que le vecteur de biais est génériquement unique, à l'addition d'un vecteur constant près. Plus précisément, il montre que les vecteurs de perturbation pour lesquels le vecteur de biais n'est pas unique sont contenus dans les cellules d'un complexe polyédrale dont la codimension est au moins 1. Nous déduisons ensuite que l'itération sur les politiques de Hoffman et Karp converge bien pour un paiement générique, ce qui nous amène à proposer un schéma de perturbation explicite qui permet de résoudre les instances non génériques (pour lesquelles des cycles peuvent se produire) sans faire appel à la condition d'irréductibilité.

Dans le Chapitre 8, nous nous occupons du problème de l'unicité du vecteur de biais pour n'importe quel jeu stochastique à somme nulle dont l'espace d'état est fini. Nous montrons que si l'équation ergodique (0.1) a une solution pour toute perturbation locale, alors l'unicité du vecteur de biais est générique localement.

Le Chapitre 3 est basé sur la prépublication [START_REF] Akian | Minimax representation of nonexpansive functions and application to zero-sum recursive games[END_REF]. La première partie du Chaptire 4 (concernant les jeux stochastiques avec paiements bornés) et le Chapitre 5 sont issues de l'article [START_REF] Akian | Ergodicity conditions for zero-sum games[END_REF], publié dans le journal Discrete and Continuous Dynamical Systems, Series A. Les résultats du Chapitre 6 ont été partiellement annoncés dans les actes de la conférence CDC 2015 [START_REF] Akian | Hypergraph conditions for the solvability of the ergodic equation for zero-sum games[END_REF]. Les résultats du Chapitre 7, exceptés ceux relatifs à l'itération sur les politiques, ont été annoncés dans les actes de la conférence CDC 2014 [START_REF] Akian | Generic uniqueness of the bias vector of mean payoff zero-sum games[END_REF]. Les résultats se rapportant à l'« approche opérateur accrétif », présentés dans le Chapitre 4 et le Chapitre 8, ont été annoncés dans les actes de la conférence MTNS 2016 [START_REF] Hochart | An accretive operator approach to ergodic problems for zero-sum games[END_REF].

CHAPTER 1 Introduction

Context and motivations

Zero-sum stochastic games are a class of two-player repeated games introduced by Shapley [START_REF] Shapley | Stochastic games[END_REF]. They describe long-term interactions between two agents (called "players") with opposite interests, which take decisions stage by stage with the objective to optimize their gains over time. They appear in several domains, including computer science, economics or population dynamics. The main feature of this class of games is that the decisions taken at each stage modify the state of nature, according to a controlled stochastic process, and incur an instantaneous payment from one player to the other, which also depends on the current state. Hence, at each stage, the players have to make a trade-off between the instantaneous payment and the evolution of the state which influences future payoffs.

Given an evaluation of the stream of payments, the game has a value v if the maximizing player can be sure to get at least this quantity v, whereas the minimizing player can be sure not to loose more than v, independently of the strategy followed by the other player. A standard problem is to know if the value arising from the arithmetic mean of the first k payments has a limit when the horizon k tends to infinity. We shall call this limit the mean payoff (also known as the asymptotic value). This problem has been widely studied for different classes of repeated games, by several authors, including Bewley, Kohlberg, Mertens, Neyman, Sorin, Zamir, see [MZ72, Koh74, BK76, KN81, MN81, RS01b, Ney03, Sor03]. In particular, the existence of the mean payoff of stochastic games with finite state and action spaces was established in [START_REF] Bewley | The asymptotic theory of stochastic games[END_REF]. However, when the action spaces are infinite, the mean payoff may not exist even with standard assumptions on the data (compactness of the action spaces, continuity of the transition functions) [START_REF] Vigeral | A zero-zum stochastic game with compact action sets and no asymptotic value[END_REF].

Other standard problems arise from mean-payoff stochastic games, for which the global payoff is the limit (lim sup or lim inf) of the Cesàro mean of the payments. Here, instead of optimizing over a finite horizon k and considering the behavior of the value as the players become more patient (k → ∞), we consider a game played in infinitely many stages. Mean-payoff games were originally considered in the deterministic framework by Ehrenfeucht and Mycielski [START_REF] Ehrenfeucht | Positional strategies for mean payoff games[END_REF] and Gurvich, Karzanov and Khachiyan [START_REF] Gurvich | Cyclic games and finding minimax mean cycles in digraphs[END_REF]. They respectively showed the existence of optimal stationary strategies, i.e., such that the decisions only depend on the current position, and gave an algorithm to find them. However, the existence of a polynomial-time algorithm to solve mean-payoff games has remained, since then, an open question. Let us note that these games [START_REF] Zwick | The complexity of mean payoff games on graphs[END_REF], as well as related classes of stopping stochastic games [START_REF] Condon | The complexity of stochastic games[END_REF], were shown to be in the complexity class NP ∩ coNP. Hence, they belong to the few problems that are in NP ∩ coNP for which no polynomial-time algorithm is known. We refer to [START_REF] Andersson | The complexity of solving stochastic games on graphs[END_REF] for a comparison of the various classes of games which have been considered. Mean-payoff stochastic games also rise theoretical problems, such as the existence of the value or optimal strategies. These problems have been considered not only in game theory [START_REF] Filar | Competitive Markov decision processes[END_REF], but also in control theory (i.e., one-player games) under the name of ergodic control problems or long-run average-reward problems. The latter problems are standard for Markov decision processes [START_REF] Puterman | Markov decision processes: discrete stochastic dynamic programming[END_REF][START_REF] Hernández-Lerma | Discrete-time Markov control processes[END_REF] as well as for risk-sensitive stochastic control, developed in the framework of discrete time by several authors including Hernández-Hernández and Marcus [START_REF] Hernández-Hernández | Risk sensitive control of Markov processes in countable state space[END_REF], Fleming and Hernández-Hernández [START_REF] Fleming | Risk-sensitive control of finite state machines on an infinite horizon[END_REF][START_REF] Fleming | Risk-sensitive control of finite state machines on an infinite horizon[END_REF], see also [START_REF] Cavazos-Cadena | A characterization of the optimal risk-sensitive average cost in finite controlled Markov chains[END_REF] or [START_REF] Anantharam | A variational formula for risk-sensitive reward[END_REF] for more recent developments.

One way to analyze the aforementioned problems is to exploit the recursive structure of stochastic games, encompassed in their dynamic programming operator, so-called Shapley operator. One infers useful information by studying the analytical properties of these operators. This method, known as "operator approach", was developed by Kohlberg [START_REF] Kohlberg | Repeated games with absorbing states[END_REF][START_REF] Kohlberg | Invariant half-lines of nonexpansive piecewise-linear transformations[END_REF], Kohlberg and Neyman [START_REF] Kohlberg | Asymptotic behavior of nonexpansive mappings in normed linear spaces[END_REF], Rosenberg and Sorin [START_REF] Rosenberg | An operator approach to zero-sum repeated games[END_REF], Neyman [START_REF] Neyman | Stochastic games and nonexpansive maps[END_REF], Sorin [START_REF] Sorin | Asymptotic properties of monotonic nonexpansive mappings[END_REF], Renault [START_REF] Renault | The value of repeated games with an informed controller[END_REF] and others. In particular, both the mean payoff and the value of mean-payoff games exist and are independent of the initial state if some nonlinear eigenvalue equation, known as ergodic equation or average reward optimality equation, has a solution. When the state space is finite, say {1, . . . , n}, this equation writes

T (u) = λe + u , λ ∈ R , u ∈ R n , (1.1)
where T : R n → R n is the Shapley operator and e is the unit vector of R n . The scalar λ, called ergodic constant, gives the value for any initial state, and one can obtain optimal stationary strategies from the vector u, called bias vector, by identifying the actions, in the minimax representation of T , that attain the maximum or the minimum in (1.1). Thus, from a theoretical point of view, it is a fundamental issue to understand the structure of the set of bias vectors, since it allows one to access all optimal stationary strategies. In the one-player deterministic case, the representation of bias vectors is well understood. The analysis relies on max-plus spectral theory, which goes back to the work of Romanovsky [START_REF] Romanovsky | Optimization of stationary control of a discrete deterministic process[END_REF], Gondran and Minoux [START_REF] Gondran | Valeurs propres et vecteurs propres dans les dioïdes et leur interprétation en théorie des graphes[END_REF] and Cunnighame-Green [START_REF] Cuninghame-Green | Minimax algebra[END_REF], see also [START_REF] Maslov | of Advances in Soviet Mathematics[END_REF][START_REF] Baccelli | Synchronization and linearity[END_REF] for more background on this theory. With continuous time and state space, a Hamilton-Jacobi PDE replaces the dynamicprogramming equation (1.1), the solutions of which have been studied in the framework of weak KAM theory, in relation with continuous-time deterministic optimal control problems, see the monograph by Fathi [START_REF] Fathi | Weak KAM theorem in Lagrangian dynamics[END_REF]. In the stochastic case, the structure of bias vectors is still well understood for discrete-time one-player problems with finite state space, see Akian and Gaubert [START_REF] Akian | Spectral theorem for convex monotone homogeneous maps, and ergodic control[END_REF]. But in the two-player case, the structure is less well known and the uniqueness of the bias vector, up to the addition by a constant vector, is already an important issue. The latter problem is also a matter of importance for algorithmic purposes, as nonuniqueness may cause cycling problems in Hoffman-Karp policy iteration algorithm [START_REF] Hoffman | On nonterminating stochastic games[END_REF] (see an example in [START_REF] Akian | Policy iteration algorithm for zero-sum multichain stochastic games with mean payoff and perfect information[END_REF]).

The solvability of (1.1) usually requires some "communication conditions" based on the structure of transition probabilities -for instance, all transition matrices being irreducible, see [START_REF] Bather | Optimal decision procedures for finite Markov chains. II. Communicating systems[END_REF]. Alternatively, one can use techniques from nonlinear Perron-Frobenius theory. Indeed, solving Equation (1.1) is the reflection through "log-glasses" of finding a vector u ∈ R n with positive coordinates and a positive scalar λ such that f (u) = λu for some self-map f acting on the standard positive cone of R n , that is homogeneous of degree one and preserves the standard partial order of R n .

Nonlinear Perron-Frobenius theory deals more generally with self-maps f acting on a closed convex cone K in a Banach space, that are nonexpansive with respect to some metric induced by K. According to the words of Nussbaum in [START_REF] Nussbaum | Hilbert's projective metric and iterated nonlinear maps[END_REF], "the central question [. . . ] and the irreducible analytic difficulty" is to know if f has an eigenvector in the interior of K, i.e., if there exists a vector u ∈ int(K) such that

f (u) = λu (1.2)
for some scalar λ > 0. The interest for Equation (1.2) goes back to the work of Kreȋn and Rutman [START_REF] Kreȋn | Linear operators leaving invariant a cone in a Banach space[END_REF]. It has been especially studied by Nussbaum [START_REF] Nussbaum | Hilbert's projective metric and iterated nonlinear maps[END_REF][START_REF] Nussbaum | Iterated nonlinear maps and Hilbert's projective metric[END_REF], who gave analytic conditions for its solvability, relying on fixed-point theory. It has also been studied from a combinatorial point of view by Gaubert and Gunawardena [START_REF] Gaubert | The Perron-Frobenius theorem for homogeneous, monotone functions[END_REF] when K is the standard nonnegative cone of R n and f is order-preserving and homogeneous of degree one (implying nonexpansiveness with respect to Hilbert's projective metric). Note that the uniqueness of the eigenvector u, up to a positive scalar multiple, is also an important issue, investigated in the latter references. Let us mention that nonlinear Perron-Frobenius theory has applications in various domains, including game theory, information theory, dynamical systems theory, mathematical biology, economics, and links with nonnegative tensors have been recently pointed out (see the survey paper [START_REF] Chang | A survey on the spectral theory of nonnegative tensors[END_REF]). We refer to [START_REF] Lemmens | Nonlinear Perron-Frobenius theory[END_REF] for an overview of this theory.

In the present work, we use some tools of nonlinear Perron-Frobenius theory introduced in [START_REF] Nussbaum | Hilbert's projective metric and iterated nonlinear maps[END_REF] and [START_REF] Gaubert | The Perron-Frobenius theorem for homogeneous, monotone functions[END_REF] to address the aforementioned problems related with the ergodic equation (1.1). Chapter 2 is dedicated to the detailed presentation of these topics.

Contributions

In this thesis, we present new results on mean-payoff stochastic games with finite state space. The whole work is built upon the key notion of ergodicity of stochastic games, which we relate with the solvability of a family of ergodic problems (1.1). We make an extensive use of graphtheoretic methods, involving particularly hypergraphs, to characterize the latter notion. We also study the uniqueness, up to the addition by a constant vector, of the bias vector. We mention that some of these results may be transposed to the framework of nonlinear Perron-Frobenius theory, the tools of which are being used to a great extent. We next review the main results.

In Chapters 4 to 6, we introduce and characterize the notion of ergodicity for stochastic games with finite state space. We borrow the term from the Markov chain case. Following the definition of Kemeny and Snell [START_REF] Kemeny | Finite Markov chains[END_REF], the ergodicity of a finite Markov chain may be characterized by several properties involving its transition matrix: uniqueness of the invariant probability measure, uniqueness of the final class, constant limit value of the Cesàro mean of the payments obtained along a trajectory, solvability of a family of spectral problems. We show that most of these characterizations are valid in the case of stochastic games, with however one major discrepancy regarding the graph-theoretic aspect: the directed graph associated with the transition matrix of a finite Markov chain must be replaced by a pair of directed hypergraphs. Thus, we shall say that a game, or its Shapley operator T : R n → R n , is ergodic if the ergodic equation (1.1) has a solution for all operators g + T with g ∈ R n . The latter operators correspond to modified versions of the original game where the quantity g i is added to all payments in state i. Then, if the game is ergodic, the mean payoff is constant for any such additive perturbation.

In Chapter 4, we first consider the case of games with a bounded payment function. We show in Theorem 4.6 that ergodicity is equivalent to the uniqueness, up to an additive constant, of the fixed point of an auxiliary Shapley operator, which is positively homogeneous, meaning that it commutes with the product by a positive scalar. Such operators arise in games with no instantaneous payments, and for that reason we shall call them payment-free Shapley operators. This leads us to consider in Chapter 5 the problem of existence of fixed points with prescribed arg min or arg max for such operators. We give combinatorial conditions in terms of Galois connections (Theorem 5.7) and in terms of hypergraphs (Theorem 5.12), and provide a polynomial-time algorithm (Algorithm 1) to solve the latter problem. Note that this problem is interesting in itself since the structure of the set of bias vectors of a polyhedral Shapley operator (game with perfect information and finite action spaces) is given locally by the set of fixed points of some payment-free Shapley operator. We then infer a combinatorial characterization of ergodicity involving a pair of hypergraphs (Corollary 5.13), the construction of which only depends on the structure of the game (under standing assumption). Put in game-theoretic terms, we show that ergodicity is characterized by the absence of disjoint dominions for each player (Theorem 5.16), a dominion being a subset of state controlled by one player. We conclude that checking ergodicity is coNP-hard, but fixed parameter tractable (Theorem 5.28): if the number of states is fixed, the latter problem can be solved in time polynomial with respect to the number of actions.

Then, we address the general case, i.e., games with possibly unbounded payment function. It follows from a corollary of Gaubert and Gunawardena [START_REF] Gaubert | The Perron-Frobenius theorem for homogeneous, monotone functions[END_REF] that a stochastic game is ergodic, in the sense of our definition, if all slice spaces are bounded in Hilbert's seminorm. The latter spaces are subsets invariant by T and defined for all α, β ∈ R by {x ∈ R n | αe + x T (x) βe + x}. In Chapter 4, we show that this condition is not only sufficient but also necessary (Theorem 4.21). To that purpose, we use the theory of accretive operators, establishing a necessary condition of surjectivity for any accretive map in a finite-dimensional normed space (Theorem 4.16). As a consequence, we get a stability result (under additive perturbations) for the existence of a fixed point for any nonexpansive map (Corollary 4.20).

In Chapter 6, we provide a combinatorial characterization of the boundedness of all slice spaces in Hilbert's seminorm, answering an open question raised in [START_REF] Gaubert | The Perron-Frobenius theorem for homogeneous, monotone functions[END_REF]. This characterization is formulated as the nonexistence of disjoint dominions for each player in an auxiliary stochastic game, the action of which are defined by the asymptotic behavior of T along some rays (Theorem 6.2). We also give a graph-theoretic characterization (Theorem 6.9) which recovers the bounded case. When one player is a dummy (i.e., T is convex), we show that these conditions simplify.

In Chapters 7 and 8 we study the uniqueness, up to an additive constant, of the bias vector of a Shapley operator T . In Chapter 7, we first deal with the case of ergodic stochastic games with perfect information and finite action spaces, that is, when T is a polyhedral map such that the ergodic equation (1.1) has a solution for all g + T with g ∈ R n . We show that the perturbation vectors g for which uniqueness, up to an additive constant, of the bias vector of g + T fails are contained in the cells of a polyhedral complex, the codimension of which are at least 1 (Theorem 7.8). As an application, we obtain a perturbation scheme allowing one to solve degenerate instances of stochastic games by policy iteration (Theorem 7.22).

Then in Chapter 8, we generalize this "generic property" to any Shapley operator, following the "accretive operator approach" introduced in Chapter 4. We show that if the ergodic equation (1.1) is solvable for local perturbations of T , then the bias vector is unique, up to an additive constant, for a "generic" local perturbation (Theorem 8.6). Shapley operators on R n are characterized by two fundamental properties: they are orderpreserving and additively homogeneous, meaning that they commute with the addition by a constant vector. This characterization follows from several minimax representations of operators satisfying the latter properties, see [START_REF] Kolokoltsov | On linear, additive, and homogeneous operators in idempotent analysis[END_REF][START_REF] Rubinov | Topical and sub-topical functions, downward sets and abstract convexity[END_REF].

Shapley operators which are also positively homogeneous play a key role in characterizing ergodicity (Chapter 4) and are the central subject of study in Chapter 5. In Chapter 3, we give a minimax representation for such operators (Corollary 3.12), justifying the term of "payment-free Shapley operators". To that purpose we show that a real-valued function on any topological vector space is positively homogeneous of degree one and nonexpansive with respect to a weak Minkowski norm if, and only if, it can be written as a minimax of linear forms that are nonexpansive with respect to the same norm (Theorem 3.8).

Chapter 3 is based on the preprint [START_REF] Akian | Minimax representation of nonexpansive functions and application to zero-sum recursive games[END_REF]. The first part of Chapter 4 (regarding stochastic games with bounded payment function) and Chapter 5 are based on the paper [START_REF] Akian | Ergodicity conditions for zero-sum games[END_REF], published in Discrete and Continuous Dynamical Systems, Series A. The results of Chapter 6 have been partly announced, with a different point of view, in the CDC conference proceedings [START_REF] Akian | Hypergraph conditions for the solvability of the ergodic equation for zero-sum games[END_REF]. The theoretic aspects of Chapter 7 (not dealing with the application to policy iteration) have been announced in the CDC conference proceedings [START_REF] Akian | Generic uniqueness of the bias vector of mean payoff zero-sum games[END_REF]. The results of the accretive operator approach, presented in Chapter 4 and Chapter 8, have been announced in the MTNS conference proceedings [START_REF] Hochart | An accretive operator approach to ergodic problems for zero-sum games[END_REF].

Part I

Operator approach of zero-sum stochastic games CHAPTER 2 Preliminaries 2.1 Zero-sum stochastic games

In this section, we present the model of zero-sum stochastic, first in its general form (imperfect information), then in the particular case of perfect information. We refer the reader to [FV97, NS03, MSZ14] for more background on these topics.

In the remainder, we shall use the following notation: -we denote by ∆(X) the set of probability measures on any measurable space (X, F ); -let [n] := {1, . . . , n} for any positive integer n.

General model

Structure of the game

A (zero-sum) stochastic game between two players, that we call MIN and MAX respectively, is a class of zero-sum repeated games where these two agents, with opposite interests, interact at given time steps. It is defined by a 7-tuple

Γ := (S, A, B, K A , K B , r, p) ,

where

-S is the state space, that we assume finite; -(A, A ) and (B, B) are measurable sets, corresponding to the action spaces of players MIN and MAX respectively; -K A ⊂ S ×A and K B ⊂ S ×B are measurable sets, representing the constraint sets of players MIN and MAX respectively. For each state s ∈ S, the section

A s := {a ∈ A | (s, a) ∈ K A }
is the set of admissible actions of player MIN in state s and similarly, the section

B s := {b ∈ B | (s, b) ∈ K B }
is the set of admissible actions of player MAX in state s. We assume that all the sets of admissible actions are nonempty, and we let

K := {(s, a, b) | s ∈ S, a ∈ A s , b ∈ B s } ,
which is a measurable subset of S × A × B; -r : K → R is a measurable function, representing the (stage) payment function (we shall also use the word payoff function) for player MAX (and therefore the cost function for MIN); -p : K → ∆(S) is a measurable function, corresponding to the transition function. The game is played in stages, the players having a perfect knowledge of all the past and current information, i.e., they know the previously chosen actions and all the states visited up to the current one. It starts with a given initial state s 1 , and then proceeds as follows: at step , if the current state is s , the players choose independently actions a and b in their respective set of admissible actions, namely A s and B s . Then, player MAX receives from MIN the (stage) payment r(s , a , b ) and the next state s +1 is chosen according to the transition probability p(• | s , a , b ).

In order to avoid technical problems, we shall always assume that for any states s ∈ S and any probability measures µ and ν over A s and B s , respectively, the payoff function (a, b) → r(s, a, b) is integrable with respect to the product measure of µ and ν. This is in particular the case when r is bounded.

Strategies

Given a finite horizon k 1, let H k := K k-1 × S be the set of admissible histories of length k, and let H ∞ := K ∞ be the set of all histories of infinite length, also called plays. Note that all the sets H k have a measurable structure since they are products of measurable sets. Therefore, we can endow H ∞ with the σ-algebra generated by the cylinder sets. We also denote by H := k 1 H k the set of all finite histories.

A (behavioral) strategy of player MIN is a map σ : H → ∆(A) such that, for every finite history h k ∈ H k with h k = (s 1 , a 1 , b 1 , . . . , s k ), we have σ(A s k | h k ) = 1. We denote by S the set of all strategies of MIN. Similarly, the set T of strategies of player MAX is defined as the set of all maps τ : H → ∆(B) such that τ (• | h k ) is an element of ∆(B s k ) for every finite history h k ending with s k . A strategy is said to be deterministic or pure if the image of any finite history is a Dirac measure. In other words, a deterministic strategy of MIN (resp., MAX) can be identified with a map σ : H → A (resp., τ : H → B) such that σ(h k ) ∈ A s k (resp., τ (h k ) ∈ B s k ) for every finite history h k ending with s k . We say that σ(h k ) (resp., τ (h k )) is a pure action, as opposed to a mixed action.

Let us also introduce the following classes of strategies. A strategy is Markovian if it only depends on the length of the history and on its last state. A strategy σ of MIN (resp., τ of MAX) is stationary if it only depends on the current state, regardless of all the past information, that is, if there exists a map µ :

S → ∆(A) (resp., ν : S → ∆(B)) such that σ(• | h k ) = µ(• | s k ) ∈ ∆(A s k ) (resp., τ (• | h k ) = ν(• | s k ) ∈ ∆(B s k ))
for every finite history h k ending with s k .

Finally, an initial state s ∈ S and a pair of strategies (σ, τ ) ∈ S × T define, by Kolmogorov extension theorem, a unique probability measure on the set of plays H ∞ . We denote by E s,σ,τ the expectation with respect to that probability measure.

Global payos

Given a sequence of stage payments, generated by an initial state s and a choice of strategies (σ, τ ) ∈ S × T , there are several ways of defining a global payoff. In this work, we focus our attention particularly on games played in finitely many stages. For a finite horizon k 1, the payoff is thus given by the expected value of the sum of the k first stage payments, that is,

γ k (s, σ, τ ) := E s,σ,τ k =1 r(s , a , b ) .
Let us mention that a second classical way to evaluate the payoff is to fix a discount factor δ ∈ (0, 1] and to consider the discounted sum of the stage payments:

γ δ (s, σ, τ ) := E s,σ,τ δ ∞ =1 (1 -δ) ( -1) r(s , a , b ) .
The methods used to deal with problems arising from these payoffs are similar. They usually boil down to consider strategy spaces as compact sets for some topology, under which the payoff function satisfies some continuity assumptions (the approach is usually referred to as the "compact case").

Another way to define the payoff is to consider the asymptotic behavior of the plays, on contrary to the finite-horizon or the discounted payoffs which depend essentially on a finite number of stages. Here, we shall especially consider the limiting average reward criterion, defined by

γ AR (s, σ, τ ) := lim inf k→∞ E s,σ,τ 1 k k =1 r(s , a , b ) .
The problems based on this evaluation are also known as ergodic control problems in optimal control theory. Also, a stochastic game with this payoff is usually referred to as mean-payoff game in the computer science literature.

Value

Given a global payoff function γ, an initial state s ∈ S, and strategies σ ∈ S and τ ∈ T , player MAX intends to maximize the quantity γ(s, σ, τ ), while player MIN wants to minimize it. The stochastic game Γ with payoff function γ has a value in s ∈ S if

sup τ ∈T inf σ∈S γ(s, σ, τ ) = inf σ∈S sup τ ∈T γ(s, σ, τ ) .
In that case, the value, denoted by v s , is equal to either of the terms in the above equality. Its existence means that for every ε > 0, player MAX can guarantee v s -ε, that is, he has an ε-optimal strategy τ ε ∈ T such that ∀σ ∈ S , γ(s, σ, τ ε ) v s -ε , and dually player MIN can guarantee v s + ε, that is, he has an ε-optimal strategy σ ε ∈ S such that ∀τ ∈ T , γ(s, σ ε , τ ) v s + ε .

If the k-stage game has a value in s, we denote it by v k s ; if the δ-discounted game has a value in s, we denote it by v δ s ; and if the limiting average reward game, also known as mean payoff game, has a value in s, we denote it by v AR s . Once a payoff is fixed, a first problem is to find conditions under which the value exists. A second problem is to characterize the optimal, or ε-optimal, strategies. Moreover, for finitehorizon or discounted stochastic games, another classical problem concerns the asymptotic behavior of the value as the horizon goes to infinity or the discount factor vanishes. All these issues may be considered from either a theoretical or an algorithmic point of view.

In the finite-horizon case or the discounted case, the values are known to exist under standard assumptions, see [START_REF] Sorin | A first course on zero-sum repeated games[END_REF]. Let us mention the following two cases (recalling that in our framework, the state space is finite):

-the action spaces are finite (we say that the stochastic game Γ is finite);

-the sets of admissible actions are compact Hausdorff in every state, and the payment and transition functions are continuous. Note that for particular classes of stochastic games, these values exist with mild assumptions. The class of perfect-information stochastic games, which will be presented in the next subsection, is one of them. Let us also mention the case of Markov decision processes (MDPs for short), which are one-player stochastic games, i.e., games in which one of the players has only a single admissible action in each state. Then, it suffices that the payoff function is bounded to ensure the existence of the value in finite horizon.

With the limiting average reward criterion, proving the existence of the value or finding optimal strategies are more complex tasks. However, the value is known to exist in the finite setup, that is, with finite state space and action spaces. In this framework, we have furthermore v AR s = lim δ→0 v δ s = lim k→∞ v k s /k. We refer the reader to [START_REF] Filar | Competitive Markov decision processes[END_REF] for a treatment of these issues in the case of stochastic games, and to [START_REF] Hernández-Lerma | Discrete-time Markov control processes[END_REF][START_REF] Hernández-Lerma | Further topics on discrete-time Markov control processes[END_REF] in the case of MDPs.

Games with perfect information

In this subsection, we introduce the subclass of stochastic games with perfect information. It differs from the general model in the structure of information: a stochastic game has perfect information when in each state one player has a unique admissible action, hence his decision has no effect on the payment nor the transition probabilities. If the player with trivial action space is always the same, the game is an MDP.

In this thesis, we consider a more general model where in each state, players take decisions in turn (MIN then MAX). Here, a perfect-information (zero-sum) stochastic game is defined by a 7-tuple Γ := (S, A, B, K A , K B , r, p), where S is the state space, which is assumed to be finite, and A and B are the action spaces of players MIN and MAX respectively, which are assumed to be measurable sets. The constraint set of MIN is the same as the general model, that is, K A ⊂ S × A is a measurable set such that each section A s := {a ∈ A | (s, a) ∈ K A } represents the admissible actions of MIN in state s. As for the constraint set of MAX, it is a measurable set K B ⊂ S × A × B, so that his admissible actions depend on the current state and the action chosen by the first player. If MIN has chosen action a ∈ A s in state s, then the set of admissible actions of MAX is

B s,a := {b ∈ B | (s, a, b) ∈ K B } .
We assume that all the sets of admissible actions are nonempty, and we let

K := {(s, a, b) | s ∈ S, a ∈ A s , b ∈ B s,a }
be the domain of definition of the payment function r and the transition function p, which are defined as in the general model. The game is played repeatedly in discrete time like the "imperfect information" case, except that in stage 1, if the current state is s , player MIN first select an action a ∈ A s , then MAX observes this move and chooses an action b ∈ B s ,a .

To introduce the notion of strategy in our model, we need to distinguish histories of player MIN from histories of player MAX. The set of admissible histories of length k 1 for player MIN is H k := K k-1 × S, and we denote by H := k 1 H k the set of all finite histories of MIN. As for player MAX, the set of admissible histories of length k 1 is H k := K k-1 × K A , and we denote by H := k 1 H k the set of all finite histories for MAX.

A strategy of player MIN is a map σ : H → ∆(A) such that, for every finite history

h k ∈ H k with h k = (s 1 , a 1 , b 1 , . . . , s k ), we have σ(• | h k ) ∈ ∆(A s k ) (
and S denotes the set of all his strategies). As for player MAX, the set of his strategies, denoted by T , is composed of all the maps τ : H → ∆(B) such that τ (• | h k ) is an element of ∆(B s k ,a k ) for every finite history h k ending with (s k , a k ). The notion of Markovian, stationary, or deterministic strategy are defined in the same way as in the general case.

Given a payoff evaluation γ, the perfect-information game Γ has a value in state s ∈ S if

sup τ ∈T inf σ∈S γ(s, σ, τ ) = inf σ∈S sup τ ∈T γ(s, σ, τ ) .
Contrary to the general case, the existence of the value requires less hypotheses, in particular regarding the topology of the action spaces and the regularity of the payment and transition functions. For instance, the existence is guaranteed in the finite-horizon case or the discounted case as soon as the payment function is bounded (with no continuity assumptions on r or p, nor topological assumptions on A or B). Furthermore, in the finite case, existence of deterministic stationary strategies was shown for discounted games [START_REF] Shapley | Stochastic games[END_REF] and mean-payoff games [START_REF] Liggett | Stochastic games with perfect information and time average payoff[END_REF].

In the sequel, the same notation Γ will refer either to a stochastic game (with imperfect information) or a perfect-information stochastic game. When no precision is made, a stochastic game will always refer to the general model.

Asymptotic approach: the mean payo

A major topic in the theory of repeated games concerns the asymptotic behavior of the value in finite horizon when the horizon goes to infinity. Particularly, given an initial state s ∈ S, a standard question is to understand when the sequence of mean values per time unit, (v k s /k) k 1 , has a finite limit. This limit, when it exists, is referred to as the mean payoff (in state s), and we denote it by χ s :

χ s := lim k→∞ v k s k , s ∈ S .
We shall denote by χ the mean payoff vector, i.e., the vector in R S whose entries are χ s with s ∈ S. The latter question was first addressed for particular classes of repeated games with finite state space. Everett [START_REF] Everett | Recursive games[END_REF] proved the existence of the mean payoff for recursive games, a class of stochastic games where a nonzero payment only occurs in absorbing states (states that cannot be escaped from and where the payoff remains the same). Kohlberg [START_REF] Kohlberg | Repeated games with absorbing states[END_REF] proved it for absorbing games, where the transition are deterministic and some state are absorbing. Then, Bewley and Kohlberg [START_REF] Bewley | The asymptotic theory of stochastic games[END_REF] showed that the limit exists for all finite stochastic games. For more recent results, see [START_REF] Rosenberg | An operator approach to zero-sum repeated games[END_REF][START_REF] Sorin | Asymptotic properties of monotonic nonexpansive mappings[END_REF] or [START_REF] Bolte | Definable zero-sum stochastic games[END_REF]. In the last reference, Bolte, Gaubert and Vigeral proved that the mean payoff exists for definable stochastic games, where all the data (action spaces, payment and transition functions) are definable in some o-minimal structure, a condition that often holds in practice. In the one-player case, the existence of the mean payoff is ensured with even milder conditions. Thus, Renault [START_REF] Renault | Uniform value in dynamic programming[END_REF] proved that for any MDP with bounded payment function (and finite state space) the mean payoff exists -actually, the result is stronger and concerns the existence of the uniform value. Note that existence results concerning the asymptotic approach were also obtained for games with incomplete information [START_REF] Aumann | Repeated games with incomplete information[END_REF][START_REF] Mertens | The value of two-person zero-sum repeated games with lack of information on both sides[END_REF].

On the other hand, the sequence of mean values may not converge, even with standard hypotheses. Vigeral [START_REF] Vigeral | A zero-zum stochastic game with compact action sets and no asymptotic value[END_REF] provided an example of a stochastic game with finite state space, compact action spaces, and continuous payment and transition functions, such that the mean value in finite horizon does not converge as the horizon goes to infinity. See also [START_REF] Ziliotto | Zero-sum repeated games: Counterexamples to the existence of the asymptotic value and the conjecture maxmin = lim v n[END_REF] for similar nonexistence results.

Operator approach of stochastic games 2.2.1 Shapley operator and recursive structure

General case

Let Γ be a stochastic game with state space S = [n]. To any vector x ∈ R n and any state i ∈ [n], we associate an auxiliary "one-shot" game, i.e., a repeated game played in one stage, defined by the same action spaces and constraint sets as Γ and by the payment function

(a, b) → r(i, a, b) + j∈[n] x j p(j | i, a, b) , ∀(a, b) ∈ A i × B i .
Assume that the latter one-stage game is well defined and has a value, meaning that

inf µ∈∆(A i ) sup ν∈∆(B i ) A i B i r(i, a, b) + j∈[n] x j p(j | i, a, b) dν(b) dµ(a) = sup ν∈∆(B i ) inf µ∈∆(A i ) B i A i r(i, a, b) + j∈[n] x j p(j | i, a, b) dµ(a) dν(b) .
We denote by T i (x) this value, which represents the value of the stochastic game Γ starting in state i, played in one stage, and with an additional payoff of x j if the terminal state is j. Let us mention that this value exists, according to Sion's minimax theorem [START_REF] Sion | On general minimax theorems[END_REF], as soon as the sets of actions are compact Hausdorff and the payment and transition functions are continuous. Then, we define the Shapley operator T : R n → R n of the stochastic game Γ as the operator whose ith coordinate map is given by

T i (x) = inf µ∈∆(A i ) sup ν∈∆(B i ) r(i, µ, ν) + j∈[n] x j p(j | i, µ, ν) = sup ν∈∆(B i ) inf µ∈∆(A i ) r(i, µ, ν) + j∈[n] x j p(j | i, µ, ν) , ∀x ∈ R n , (2.1)
where, given two probability measures µ ∈ ∆(A i ) and ν ∈ ∆(B i ),

f (i, µ, ν) := A i B i f (i, a, b) dν(b) dµ(a)
is the multilinear extension of any function f : K → R.

This operator encompasses the recursive structure of Γ, as shown by Shapley [Sha53] in the case of finite stochastic games. Indeed, if

v k := (v k i ) i∈[n]
denotes the value vector of the k-stage game Γ, i.e., the vector in R n the ith entry of which is the value of the game in finite horizon k with initial state i, then by applying a dynamic programming principle, we have

v 0 = 0 and v k+1 = T (v k ) , ∀k ∈ N . (2.2)
As a consequence, if the mean payoff vector exists, then it is given by the growth rate of the orbit of T with initial point 0:

χ = lim k→∞ T k (0) k , (2.3)
where

T k := T • • • • • T denotes the kth iterate of T .
The "operator approach", initiated by Rosenberg and Sorin [START_REF] Rosenberg | An operator approach to zero-sum repeated games[END_REF], consists in studying the properties of the Shapley operator and its iterates to infer results about the asymptotic behavior of the sequence of values (v k ) k∈N .

Perfect-information case

When Γ is a perfect-information stochastic game with state space S = [n], the Shapley operator simplifies. The fact that players choose actions one after the other with a perfect knowledge of all available information makes the use of mixed actions (i.e., choices of probability measures over the set of admissible actions) irrelevant in finitely repeated games. Thus, with perfect information, we only need to consider pure actions, and the auxiliary one-shot game with initial state i and terminal payoff x ∈ R n has a value as soon as

inf a∈A i sup b∈B i,a r(i, a, b) + j∈[n] x j p(j | i, a, b) ∈ R .
In particular, this is the case when the payment function is bounded (the action spaces being any sets and no continuity assumption on the payment and transition functions being made).

Then, the Shapley operator T : R n → R n has a simpler form, its ith coordinate writing

T i (x) = inf a∈A i sup b∈B i,a r(i, a, b) + j∈[n] x j p(j | i, a, b) , ∀x ∈ R n .
(2.4)

Note that the recursive property (2.2) of the values in finite horizon still holds.

Axiomatization of Shapley operators

Let T : R n → R n be the Shapley operator of a stochastic game with either perfect information (2.4) or imperfect information (2.1). We endow R n with the usual partial order and denote by e the unit vector, i.e., the vector whose entries are all equal to 1. Then, it is readily seen that T satisfies the following two properties:

monotonicity:

x y =⇒ T (x) T (y) , ∀x, y ∈ R n ; (M) -additive homogeneity:

T (x + λe) = T (x) + λe , ∀(λ, x) ∈ R × R n . (AH)
An immediate consequence is that T is sup-norm nonexpansive:

T (x) -T (y) ∞ x -y ∞ , ∀x, y ∈ R n , (N ∞ )
where

z ∞ := max{|z i | | 1 i n} is the standard supremum norm in R n .
Let us mention that Crandall and Tartar [START_REF] Crandall | Some relations between nonexpansive and order preserving mappings[END_REF] proved that under (AH), the properties (M) and (N ∞ ) are equivalent.

Remark 2.1. The nonexpansiveness of T implies that the asymptotic behavior of the sequence (T k (x)/k) k 1 is independent of the vector x ∈ R n : either it always converges (to a finite limit), or it never converges. Thus, the mean payoff vector (2.3) is given by the growth rate of any orbit of T .

The importance of monotonicity and additive homogeneity was recognized early on in dynamic programming [START_REF] Blackwell | Discounted dynamic programming[END_REF]. The study of operators satisfying these properties, so called operator approach, has proven useful not only in game theory [BK76, FV97, RS01a, Sor04, Vig10, BGV15], but also in optimal control [START_REF] Kohlberg | Invariant half-lines of nonexpansive piecewise-linear transformations[END_REF][START_REF] Whittle | Optimization over time[END_REF][START_REF] Kolokoltsov | On linear, additive, and homogeneous operators in idempotent analysis[END_REF], or in the modelling of discrete event systems (such as communication networks, digital circuits, manufacturing processes) [START_REF] Baccelli | Synchronization and linearity[END_REF][START_REF] Gunawardena | From max-plus algebra to nonexpansive mappings: a nonlinear theory for discrete event systems[END_REF].

Conversely, Kolokoltsov showed that every operator from R n to itself which is monotone and additively homogeneous can be written as the Shapley operator (2.1) of a stochastic game. Theorem 2.2 ( [START_REF] Kolokoltsov | On linear, additive, and homogeneous operators in idempotent analysis[END_REF], see also [KM97, Th. 2.15]). Let F : R n → R n be a monotone and additively homogeneous map. Then, there exist a payment function r : [n] × R n × R n → R and a transition function p from [n] × R n × R n to the standard simplex of R n , such that the ith coordinate map of F is given by

F i (x) = min a∈R n max b∈R n r(i, a, b) + j∈[n]
x j p j (i, a, b)

= max b∈R n min a∈R n r(i, a, b) + j∈[n] x j p j (i, a, b) , ∀x ∈ R n .
The proof of this result, which only holds in finite dimension, is based on a minimax representation formula due to Evans, which applies more generally to Lipschitz functions [START_REF] Evans | Some min-max methods for the Hamilton-Jacobi equation[END_REF]. Rubinov and Singer proposed a similar representation, but with perfect information. Moreover, in their formula, the transition probabilities are degenerate, i.e., deterministic.

Theorem 2.3 ([RS01b, Th. 5.3]). Let F : R n → R n be a monotone and additively homogeneous map. Then, its ith coordinate map is given by

F i (x) = min y∈R n max j∈[n]
x j -y j + F i (y) , ∀x ∈ R n .

Note that Gunawardena and Sparrow obtained independently an equivalent result [Gun03, Prop. 2.3]. Also note that in the context of abstract convexity, a representation of functions on cones with similar properties has been given in [START_REF] Dutta | Monotonic analysis over cones[END_REF].

The above considerations motivate us to identify, in this thesis, Shapley operators with maps that satisfy properties (M) and (AH).

Definition 2.4. We call (abstract) Shapley operator on R n any map from R n to itself that is both monotone (M) and additively homogeneous (AH).

Ergodic equation

As mentioned above, the operator approach for stochastic games consists in studying the properties of Shapley operators in order to infer information about the asymptotic behavior of the values in finite horizon. A basic tool in this respect is the following equation, called ergodic equation (also known as the average reward optimality equation in the optimal control and MDP literature):

T (u) = λe + u , (λ, u) ∈ R × R n . (2.5)
We say that the ergodic equation is solvable if their exist both a scalar λ ∈ R and a vector u ∈ R n for which (2.5) holds. If this is the case, we shall say that λ is the ergodic constant and u is a bias vector of T . Since the solvability of the ergodic equation can be seen as a nonlinear spectral problem in the additive framework (see Section 2.3), we shall also call λ the (additive) eigenvalue of T , and u an (additive) eigenvector. Consequently, saying that T has an (additive) eigenvalue is equivalent to saying that the ergodic equation is solvable for T .

Observe that if the ergodic equation is solvable, then T k (u) = kλe + u for all integers k ∈ N, and so the mean payoff vector exists (see Remark 2.1) and each of its components is equal to λ, that is, the mean payoff of the associated game is independent of the initial state. It follows that the ergodic constant, when it exists, is necessarily unique. Also, since T is additively homogeneous, any vector u + αe with α ∈ R is a bias vector. We shall say that u is defined "up to an additive constant". Here, by "constant (vector)", we mean a vector proportional to the unit vector e.

The ergodic equation is useful not only for the study of the mean-payoff vector, but also for the study of the limiting average reward game. In particular, knowing explicitly an eigenvector gives access to optimal or ε-optimal stationary strategies, see [START_REF] Kontorer | Nonlinear semigroups and infinite horizon optimization[END_REF] for the deterministic case, which readily extends to the stochastic case. See also [START_REF] Filar | Competitive Markov decision processes[END_REF] and [START_REF] Hernández-Lerma | Discrete-time Markov control processes[END_REF][START_REF] Hernández-Lerma | Further topics on discrete-time Markov control processes[END_REF] for the links between the ergodic equation and optimal strategies in the framework of stochastic games and MDPs, respectively.

The main purpose of this work is to find conditions guaranteeing the solvability of the ergodic equation, as well as to characterize the set of bias vectors.

Piecewise ane operators and invariant half-lines

We conclude this section by mentioning the particular case of perfect-information finite stochastic games. Recall that a stochastic game (with perfect information or not) is finite if the state space and the action spaces are finite. In that case, the Shapley operator T : R n → R n is piecewise affine , meaning that R n can be covered by finitely many polyhedra (a polyhedron being an intersection of finitely many half-spaces) such that the restriction of the coordinate functions of T to each polyhedron are affine.

Note that continuous piecewise affine functions are exactly the ones that can be represented as a minimax over finite sets of affine functions, i.e., as in (2.4) where the infimum and the supremum are taken over finite sets and the coefficients can be any real numbers, see [START_REF] Ovchinnikov | Max-min representation of piecewise linear functions[END_REF][START_REF] Aliprantis | Cones and duality[END_REF].

Then, the following result applies.

Theorem 2.5 ([Koh80, Th. 2.1]). Let F : R n → R n be a piecewise affine map that is nonexpansive in any norm. Then, it has an invariant half-line, that is, there exist two vectors u, ν ∈ R n such that

F (u + αν) = u + (α + 1)ν
for all α 0. Moreover, the vector ν is unique.

Hence, the Shapley operator T of a perfect-information finite stochastic game has an invariant half-line, and so there exist vectors u, ν as in the above theorem. This yields T k (u) = u + kν for all integers k ∈ N, which implies that the mean-payoff vector exists (see Remark 2.1) and is equal to ν. Moreover, it is easily seen that the solvability of the ergodic equation (2.5) is equivalent to the mean payoff being a constant vector.

Connection with nonlinear Perron-Frobenius theory 2.3.1 Additive versus multiplicative framework

Nonlinear Perron-Frobenius theory is primarily concerned with the study of nonexpansive maps f acting on a closed convex cones K in a Banach space. An important special case arises when f is homogeneous of degree one and preserves the partial order induced by K. The central question is to know if f has an eigenvector in the interior of K, i.e., if there exists a vector

u ∈ int(K) such that f (u) = λu (2.6)
for some scalar λ > 0. Other standard problems include the characterization of the eigenvalue, which may be seen as a spectral radius, or the uniqueness, up to the product by a positive scalar, of the eigenvector, see [START_REF] Lemmens | Nonlinear Perron-Frobenius theory[END_REF]. These problems are already interesting when K is the standard nonnegative cone of R n , namely R n + := {x ∈ R n | x 0}, having applications, e.g., in matrix scaling problems or nonnegative tensor problems. Note that if f is a n × n nonnegative matrix, we recover the classical Perron-Frobenius theory.

In this "multiplicative" framework, the aforementioned problems have been considered in particular by Nussbaum [START_REF] Nussbaum | Hilbert's projective metric and iterated nonlinear maps[END_REF][START_REF] Nussbaum | Iterated nonlinear maps and Hilbert's projective metric[END_REF]. But when f leaves the interior of R n + invariant, one can equivalently work in the "additive" framework, see [START_REF] Gaubert | The Perron-Frobenius theorem for homogeneous, monotone functions[END_REF]. Indeed, the space R n can be placed in bijection with the interior of R n + via the map exp : R n → int(R n + ) and its inverse log : int(R n + ) → R n , which applies respectively the exponential and the logarithm componentwise. Hence, any map f : int(R n + ) → int(R n + ) which is homogeneous of degree one and preserves the standard partial order of R n is conjugated with a unique map T : R n → R n , namely T := log •f • exp, which is monotone (M) and additively homogeneous (AH), i.e., a Shapley operator. Then, through "log-glasses", it is readily seen that the nonlinear eigenproblem (2.6) is equivalent to finding a solution to the ergodic equation (2.5).

Thus, it is equivalent to work in the multiplicative framework and the additive one, as far as the above properties are concerned, and all the results in the present work, established in the additive framework, can be translated in the multiplicative one.

Hilbert's seminorm and invariant sets

A useful tool in nonlinear Perron-Frobenius theory is Hilbert's seminorm. In the space R n , it is defined by

x H := max i∈[n] x i -min j∈[n] x j = max i,j∈[n] (x i -x j ) , ∀x ∈ R n .
The following result justifies the interest of Hilbert's seminorm for the nonlinear spectral problem we are considering. As mentioned in [START_REF] Gaubert | The Perron-Frobenius theorem for homogeneous, monotone functions[END_REF], it can be seen as a special case, in the additive framework, of a general result by Nussbaum [Nus88, Th. 4.1].

Theorem 2.6 ([GG04, Th. 9]). Let T : R n → R n be a Shapley operator. The ergodic equation (2.5) is solvable if, and only if, the orbit {T k (x) | k ∈ N} is bounded in Hilbert's seminorm for some (hence all)

x ∈ R n .

The above theorem provides a method to tackle the problem of the solvability of the ergodic equation, method which consists in finding invariant subsets that are bounded in Hilbert's seminorm. Following this direction, let us introduce three particular families of subsets that are structurally invariant under all Shapley operators T . Given scalars α, β ∈ R, we define:

-the sub-eigenspace

S α (T ) := {x ∈ R n | αe + x T (x)} ; -the super-eigenspace S β (T ) := {x ∈ R n | T (x) βe + x} ;
-the slice space

S β α (T ) := S α (T ) ∩ S β (T ) = {x ∈ R n | αe + x T (x) βe + x} .
Even without the boundedness property, the nonemptiness of those spaces may be useful. Indeed, a pair

(α, x) ∈ R × R n such that x ∈ S α (T ), or (β, x) ∈ R × R n such that x ∈ S β (T )
, may be seen as the solution of a generalized ergodic equation. It provides information about the asymptotic behavior of the iterates of T , as shown by the following result. Note that in the MDP literature, this tool is known as the average-cost optimality inequality, see [START_REF] Hernández-Lerma | Discrete-time Markov control processes[END_REF].

Proposition 2.7 (compare with [Sor04, Prop. 7]). Let T : R n → R n be a Shapley operator. Assume that the sub-eigenspace S α (T ) is nonempty for a given scalar α. Then

lim inf k→∞ T k (0) k αe .
Similarly, if the super-eigenspace S β (T ) is not empty for β ∈ R, then

lim sup k→∞ T k (0) k βe .
Finally, let us mention that the slice space

S β α (T ) is nonempty if α min i∈[n] T i (0) and β max i∈[n] T i (0)
. Hence, the difficulty in applying Theorem 2.6 with some slice space resides in checking its boundedness with respect to Hilbert's seminorm.

Conditions for the solvability of the ergodic equation

The question of characterizing the Shapley operators for which one of the aforementioned invariant spaces is bounded in Hilbert's seminorm (but not necessarily all of them) is delicate, and depends heavily not only on the structure but also on the parameters defining the operator. On the other hand, there exist conditions ensuring the boundedness of all invariant spaces of a given family. These conditions, that we present in the sequel, rely on the asymptotic behavior of the Shapley operators. Hence, they are insensitive to perturbations that preserve the monotonicity and the additive homogeneity and that do not change the behavior at infinity.

Boundedness of super-eigenspaces

The first condition involves the super-eigenspaces (note that a dual statement holds for subeigenspaces). We need the following construction. To any Shapley operator T : R n → R n , we associate the directed graph G(T ), with set of nodes [n] and an arc from i to j if T i (αe {j} ) tends to +∞ as α goes to +∞, where e {j} is the jth vector of the canonical basis of R n . The directed graph G(T ) is said to be strongly connected if there is a path between any two distinct nodes.

Theorem 2.8 ([GG04, Th. 10]). Let T : R n → R n be a Shapley operator. If G(T ) is strongly connected, then all super-eigenspaces are bounded in Hilbert's seminorm.

Note that the above condition is only sufficient, but in the same paper [START_REF] Gaubert | The Perron-Frobenius theorem for homogeneous, monotone functions[END_REF], a combinatorial characterization of the boundedness of all super-eigenspaces is given, involving aggregated graphs constructed in the same spirit as G(T ).

Boundedness of slice spaces

The second condition involves the slice spaces. Let us denote by T the recession operator of any Shapley operator T : R n → R n , defined by

T (x) := lim α→+∞ T (αx) α , ∀x ∈ R n .
(2.7)

Note that the recession operator does not always exist, but when it does it inherits from T the monotonicity and the additive homogeneity. In addition, it is positively homogeneous, meaning that

T (αx) = α T (x) , ∀α > 0 , ∀x ∈ R n . (PH)
As a consequence, any constant vector is a fixed point of T . We shall call such fixed points trivial fixed points.

Theorem 2.9 ([GG04, Th. 13]). If a Shapley operator T on R n has a recession operator whose only fixed points are trivial, then all slice spaces of T are bounded in Hilbert's seminorm.

The above result provides a sufficient condition for the boundedness of all slice spaces, but a counterexample in [START_REF] Gaubert | The Perron-Frobenius theorem for homogeneous, monotone functions[END_REF] showed that the condition is not necessary. In particular, the following problem remained open.

Problem ([GG04]

). Give a combinatorial condition or graph-theoretic characterization of the property that all slice spaces are bounded in Hilbert's seminorm.

The present work gives an answer to that problem.

Weakly convex operators

We mention that Cavazos-Cadena and Hernández-Hernández proved in [START_REF] Cavazos-Cadena | Poisson equations associated with a homogeneous and monotone function: necessary and sufficient conditions for a solution in a weakly convex case[END_REF], under a weak form of convexity, that T has only trivial fixed points if, and only if, the ergodic equation is solvable for all operators g + T with g ∈ R n . The weak convexity property was motivated by risk sensitive control problems. A typical example is the following Shapley operator T :

T i (x) = log j∈[n] M ij exp(x j ) , i ∈ [n] , x ∈ R n , (2.8)
where M = (M ij ) is a nonnegative matrix without zero row. Note that a supremum of weakly convex Shapley operators is weakly convex, hence, one can construct further examples of weakly convex operators by taking suprema of operators of the form (2.8). This result gives a characterization, under the weakly convex property, of the boundedness of all slice spaces in Hilbert's seminorm. Indeed, it follows from Theorem 2.6 that all Shapley operators g + T with g ∈ R n have an eigenvector if all the slice spaces of T are bounded in Hilbert's seminorm.

Let us also mention that equivalent combinatorial conditions were given in the same paper, involving the directed graph G(T ).

From nonlinear spectral problems to xed point problems

The existence results in nonlinear Perron-Frobenius theory usually rely on classical fixed-point theorems. In this subsection, we detail the link between the two topics in regard to our problems.

Let TP n be the space R n quotiented by the equivalence relation ∼ defined as follows: for two vectors x, y ∈ R n , we write x ∼ y if there is a scalar α ∈ R such that x -y = αe. By analogy with the real projective space, we call it the additive projective space. We denote by [x] the equivalence class of any vector x ∈ R n , which is the affine line directed by the unit vector e and passing through x. Observe that x H = y H if x ∼ y. Hence, Hilbert's seminorm can be quotiented into a norm of TP n , that we denote by q H :

q H ([x]) := x H , ∀x ∈ R n .
Furthermore, TP n has the structure of a vector space:

α[x] + [y] := [αx + y] , ∀x, y ∈ R n , ∀α ∈ R .
Thus, (TP n , q H ) is a normed vector space of dimension n -1.

Since any Shapley operator T : R n → R n is additively homogeneous, it is readily seen that it can be quotiented into a map [T ] : TP n → TP n , sending each equivalence class [x] to the class [T (x)]. Moreover, it is a standard result that any Shapley operator is nonexpansive with respect to Hilbert's seminorm (see [START_REF] Gaubert | The Perron-Frobenius theorem for homogeneous, monotone functions[END_REF]), that is,

T (x) -T (y) H x -y H , ∀x, y ∈ R n . (N H )
Hence, the quotiented map [T ] is nonexpansive with respect to the norm q H on TP n . Now observe that a vector u ∈ R n is an eigenvector of T if, and only if, its equivalence class [x] ∈ TP n is a fixed point of [T ]. Thus, the ergodic problem (2.5) reduces to a fixed point problem involving a nonexpansive map.

CHAPTER 3

Minimax representation of positively homogeneous nonexpansive functions

Introduction

Several minimax representation theorems are available for monotone additively homogeneous operators on R n : characterization in terms of stochastic games [Kol92] (Theorem 2.2), or in terms of perfect-information repeated games with deterministic transitions [START_REF] Rubinov | Topical and sub-topical functions, downward sets and abstract convexity[END_REF][START_REF] Gunawardena | From max-plus algebra to nonexpansive mappings: a nonlinear theory for discrete event systems[END_REF] (Theorem 2.3). In the context of abstract convexity, we also refer to [START_REF] Dutta | Monotonic analysis over cones[END_REF][START_REF] Dutta | Monotonic analysis over cones[END_REF] for a characterization of similar operators in the multiplicative framework. The Fenchel-Legendre duality provides the same kind of representation theorems (but with no infimum) for operators, or real functions, that are in addition convex. It has been applied in infinite dimension to the case of convex risk measures [START_REF] Föllmer | Convex measures of risk and trading constraints[END_REF] for instance.

An important subclass of monotone and additively homogeneous operators arises when considering positive homogeneity (PH). This is particularly the case for recession operators, used by Gaubert and Gunawardena [START_REF] Gaubert | The Perron-Frobenius theorem for homogeneous, monotone functions[END_REF] in nonlinear Perron-Frobenius theory, or for coherent risk measures [START_REF] Artzner | Coherent measures of risk[END_REF][START_REF] Delbaen | Coherent risk measures on general probability spaces[END_REF] in infinite dimension.

Observe that if T : R n → R n is the Shapley operator of a stochastic game (with perfect information or not) Γ(r) = ([n], A, B, K A , K B , r, p) the payment function of which is bounded, then its recession operator T exists and it is the Shapley operator of the stochastic game Γ(0) which has a zero payment function. However, the converse cannot be deduced from the representation theorems mentioned above: if T : R n → R n is an abstract Shapley operator that is also positively homogeneous, we cannot derive from Theorem 2.2 or 2.3 a representation formula of T as the dynamic programming operator of a game with zero payment function. In this chapter, we provide such a characterization. This leads us to the following definition. Definition 3.1. We call (abstract) payment-free Shapley operator on R n any self-map of R n that is monotone (M), additively homogeneous (AH) and positively homogeneous (PH).

It is known that the two properties characterizing Shapley operators, (M) and (AH), are equivalent to nonexpansiveness with respect to some weak Minkowski norm. The latter are substitutes of norms that are not necessarily symmetric or separating. They have been studied in metric geometry, in particular by Papadopoulos and Troyanov [START_REF] Papadopoulos | Weak Minkowski spaces[END_REF], and arise naturally in the study of Shapley operators, see [START_REF] Gaubert | A maximin characterisation of the escape rate of nonexpansive mappings in metrically convex spaces[END_REF].

To get a minimax characterization of monotone, additively and positively homogeneous operators, we first establish a general minimax representation theorem which applies to real functions on a topological vector space that are nonexpansive with respect to any weak Minkowski norm. We then characterize nonexpansive maps that are positively homogeneous. As a corollary, we arrive at our main application: a representation theorem for dynamic programming operators of stochastic games with zero payments. This representation also leads to an approximation of the latter operators by polyhedral maps. Such approximation can been used in the setting of "max-plus basis methods" for the numerical solutions of Hamilton-Jacobi type partial differential equations, see [START_REF] Mceneaney | Max-plus methods for nonlinear control and estimation[END_REF][START_REF] Akian | The max-plus finite element method for solving deterministic optimal control problems: basic properties and convergence analysis[END_REF] for background. We also arrive at a representation theorem for nonconvex risk measures.

The results persented in this chapter are based on the preprint [START_REF] Akian | Minimax representation of nonexpansive functions and application to zero-sum recursive games[END_REF].

Preliminaries: nonexpansiveness and weak Minkowski norms

In this chapter, we denote by V a real topological vector space (t.v.s. for short). We denote by V * the dual space of V (i.e., the space of linear forms on V), by V its topological dual space (i.e., the space of continuous linear forms on V), and by •, • the duality product. We shall specially consider the situation in which V is a vector space with an (Archimedean) order unit. This means that we assume that V is a real vector space with an order relation, denoted by , that is compatible with the algebraic structure of V, that is, satisfying the following two axioms:

(i) x y =⇒ x + z y + z , ∀z ∈ V;

(ii) x y =⇒ λx λy , ∀λ ∈ R + ; where R + is the set of nonnegative real numbers. We also assume that V is equipped with a special vector denoted by e V (or simply by e if the context is clear), called an order unit and such that for every x ∈ V, there exists a scalar λ > 0 satisfying x λe V . Finally, we assume that this order unit is Archimedean, i.e., for any x ∈ V, we have x 0 if λe V + x 0 for all scalars λ > 0. Such a space will be endowed with the topology defined by the following norm:

x e V = inf{λ ∈ R | -λe V x λe V } , x ∈ V , (3.1)
see [START_REF] Paulsen | Vector spaces with an order unit[END_REF]. Note that if V is the Euclidean space R n equipped with the standard partial order, then the standard unit vector of R n is an Archimedean order unit, and the corresponding norm (3.1) is the usual supremum norm. Hence, we will abusively refer to (3.1) as the "supnorm" even in general situations. An important particular case is obtained when V is an AM-space with unit, i.e., a Banach lattice equipped with an order unit and such that the norm satisfies (3.1), see [START_REF] Aliprantis | Infinite dimensional analysis[END_REF]. By the Kakutani-Krein theorem, any AM-space with unit is isomorphic (lattice isometric) to a space C(K) of continuous functions over some compact Hausdorff set K, equipped with the supnorm, see [START_REF] Aliprantis | Infinite dimensional analysis[END_REF]Th. 8.29] or [Sch74, Ch. II, Th. 7.4].

Given two vector spaces with an order unit, (V, e V ) and (W, e W ), we will be interested in maps F : V → W that satisfy, for all x, y ∈ V, some of the following properties:

-monotonicity:

x y =⇒ F (x) F (y) ; (M)
-additive homogeneity:

F (λe V + x) = λe W + F (x) , ∀λ ∈ R ; (AH)
-additive subhomogeneity:

F (λe V + x) λe W + F (x) , ∀λ > 0 ; (ASH)
-positive homogeneity:

F (λx) = λF (x) , ∀λ > 0 ; (PH)
-sup-norm nonexpansiveness:

F (x) -F (y) e W x -y e V . (N ∞ )
The importance of monotonicity and additive homogeneity in optimal control and game theory has been pointed in Subsection 2.2.2. Crandall and Tartar [START_REF] Crandall | Some relations between nonexpansive and order preserving mappings[END_REF] showed that (M) and (AH) ⇐⇒ (N ∞ ) and (AH) ,

when V = W is a L ∞ space. It is also known that (M) and (ASH) ⇐⇒ (M) and (N ∞ ) ,
see, e.g., [START_REF] Akian | Spectral theorem for convex monotone homogeneous maps, and ergodic control[END_REF]. These relations are readily generalized to any vector spaces with an order unit.

The monotonicity and additive homogeneity properties turn out to be related with nonexpansiveness with respect to weak Minkowski norms. By weak Minkowski norm on V, we mean a function q : V → R that is subadditive (i.e., q(x + y) q(x) + q(y) for all x, y ∈ V) and positively homogeneous (PH), hence convex but not necessarily symmetric (i.e., we do not require that q(x) = q(-x) for all x ∈ V). Our definition is a variant of the one in [START_REF] Papadopoulos | Weak Minkowski spaces[END_REF], where q is also required to be nonnegative and may take infinite values. We say that a real function on V, f : V → R, is nonexpansive with respect to q if f (x) -f (y) q(x -y) for all x, y ∈ V.

When V = R n , a useful example of weak Minkowski norm, arising in Hilbert geometry [START_REF] Papadopoulos | Weak Minkowski spaces[END_REF], is the "top" map, t : R n → R, defined by

t(x) := max i∈[n] x i , x ∈ V ,
or its variant, t + : R n → R + , defined by

t + (x) := max{t(x), 0} , x ∈ V . When V = W = R n ,
we shall consider for any map F : R n → R n the following properties:

-nonexpansiveness in t:

t(F (x) -F (y)) t(x -y) ; (N t ) -nonexpansiveness in t + : t + (F (x) -F (y)) t + (x -y) . (N + t ) Gunawardena and Keane showed in [GK95] that (M) and (AH) ⇐⇒ (N t ) , (M) and (ASH) ⇐⇒ (N + t ) .
Again, these relations are readily generalized to any vector space V with an order unit e, defining the function t : V → R by

t(x) := inf{λ ∈ R | x λe} , x ∈ V . (3.2)

Minimax representation theorems

In this section, we give a general minimax characterization of nonexpansive functions with respect to a weak Minkowski norm, which will lead to extension and refinements of the known minimax representations of Shapley operators.

Nonexpansive functions with respect to weak Minkowski norms

We shall need the following lemma, which is a variation of Legendre-Fenchel duality.

Lemma 3.2. Let V be a real t.v.s. and let q : V → R be a weak Minkowski norm, continuous with respect to the topology of V. Then, for every vector x ∈ V,

q(x) = max p∈P p, x , (3.3)
where P := {p ∈ V * | ∀x ∈ V, p, x q(x)} is a nonempty convex set of V , compact for the weak-* topology.

Proof. For any x ∈ V \ {0} we have, by definition of P, q(x) sup p∈P p, x . Let p be the linear form defined on the vector subspace of V generated by x, Rx, and such that p, x = q(x). If λ 0, then p, λx = λ p, x = λq(x) = q(λx) ,

since q is positively homogeneous. If λ < 0, then p, λx = λ p, x = -|λ|q(x) |λ|q(-x) = q(λx) ,
the inequality coming from the subadditivity of the weak norm: 0 = q(0) q(x)+q(-x). Hence p is dominated by q on the vector subspace Rx, and according to the Hahn-Banach extension theorem [AB06, Th. 5.53], there is a linear extension p of p to V which is dominated by q on V. Therefore, p ∈ P and q(x) = p, x , which shows that q(x) = max p∈P p, x . This also proves that P is nonempty. Note that, since q is continuous, every linear form p dominated by q is also continuous, hence P ⊂ V . The convexity of P is straightforward and since this set can also be written as

P = x∈V {p ∈ V * | p, x q(x)} ,
we deduce that P is a weak-* closed set. Furthermore, for every x ∈ V and every p ∈ P we have -q(-x) p, x q(x). Hence, P is pointwise bounded. Applying the Tychonoff Product theorem [AB06, Th. 2.61], we deduce that P is weak-* compact.

Remark 3.3. In Lemma 3.2, we did not assume that V is locally convex or Hausdorff. When these properties hold, Lemma 3.2 becomes a direct application of [AB06, Th. 7.52], which applies more generally to dual pairs. Alternatively, this result can be obtained using the Legendre-Fenchel duality for convex proper lower semicontinuous functions [ET99, Prop. 4.1], still assuming that V is locally convex and Hausdorff.

The following simple observation shows that the maximum in (3.3) is attained in the closure of the set of extreme points of P. Lemma 3.4. Let q : V → R be a continuous weak Minkowski norm, and define P as in Lemma 3.2. Denote by ext P the set of extreme points of P, and by ext P its closure with respect to the weak-* topology. Then, for every x ∈ V,

q(x) = sup p∈ext P p, x = max p∈ext P p, x .
(3.4)

In particular, if ext P is closed, then the maximum in (3.3) is attained at an extreme point of P .

Proof. We already showed in Lemma 3.2 that for every x ∈ V, the supremum q(x) = sup p∈P p, x is attained at some linear form r ∈ P. It follows from the Krein-Milman theorem that P is the closed convex hull of the set ext P, the closure being understood with respect to the weak-* topology. Hence, there exists a net (r α ) of elements of the convex hull of ext P that converges to r in this topology. In particular, every r α can be written as a finite sum

r α = 1 i m λ i p i with
We deduce from the previous lemmas a minimax representation theorem that directly extends the result of Rubinov and Singer [RS01b, Th. 5.3].

Theorem 3.5. Let V be a real t.v.s. and let q : V → R be a weak Minkowski norm, continuous with respect to the topology of V.

Let P := {p ∈ V * | ∀x ∈ V, p, x q(x)}. A function f : V → R is nonexpansive with respect to q if,
and only if,

f (x) = min y∈V max p∈P p, x -y + f (y) , ∀x ∈ V . (3.5)
Moreover, the value of the latter expression does not change if the maximum is restricted to the points p ∈ ext P.

Proof. If f is nonexpansive with respect to q, then by definition we have f (x) -f (y) q(x -y) for all x, y ∈ V. We readily deduce that f (x) = min y∈V q(x -y) + f (y), the minimum being attained in x. Replacing q by its expression given in (3.3), we get the minimax representation formula (3.5). The remaining part of the theorem follows directly from Lemma 3.4. Conversely, as a consequence of Lemma 3.2, any real function given by (3.5) is easily seen to be nonexpansive with respect to q.

Corollary 3.6. Let V be a real vector space with an order unit, denoted by e, and the topology defined by the sup-norm (3.1) associated with that order unit. Let

∆ := {p ∈ V * | p, e = 1, p, x 0, ∀x ∈ V + } , where V + := {x ∈ V | x 0}.
Then, a function f : V → R is monotone and additively homogeneous if, and only if,

f (x) = min y∈V max p∈∆ p, x -y + f (y) .
Moreover, the value of the latter expression is not changed if the minimum is restricted to the vectors y ∈ V such that f (y) = 0, or if the maximum is restricted to the linear forms p ∈ ext ∆.

Proof. Let f be a monotone and additively homogeneous real function on V. As exposed in Section 3.2, this is equivalent to the function f being nonexpansive with respect to the weak Minkowski norm t(•) defined in (3.2). Let P be defined as in Theorem 3.5. We next show that P = ∆.

If p ∈ P, then for all x ∈ V + we have t(-x) 0, so that p, -x 0, hence p, x 0. Moreover, t(e) = 1 and t(-e) = -1, so that p, e 1 and p, -e -1, which shows that p, e = 1. This shows that P ⊂ ∆. Conversely, if p ∈ ∆, then for all x ∈ V such that t(x) < λ, we have x λe, so that p, x = p, x -λe + λ p, e λ. This implies that p, x t(x) for all x ∈ V, hence p ∈ P. Applying Theorem 3.5, we get the first equality in the corollary. Now, remark that p, x -y + f (y) = p, x -z with z = y -f (y)e and f (z) = 0. Then, a change of variable leads to

f (x) = min y∈V f (y)=0 max p∈∆ p, x -y .
The remaining part of the corollary follows from Theorem 3.5 and the converse is straightforward.

Example 3.7. If V = R n and F : V → V is monotone and additively homogeneous, then, as recalled in Section 3.2, each coordinate function F i , i ∈ [n], is nonexpansive with respect to the weak Minkowski norm t(•). Then, the representation result of Rubinov and Singer [RS01b, Th. 5.3], which shows that

F i (x) = min y∈R n max j∈[n] x j -y j + F i (y) , ∀x ∈ R n ,
is a special case of Corollary 3.6.

Positively homogeneous nonexpansive functions

We now consider nonexpansive functions that are positively homogeneous. The following theorem characterizes these functions as minimax of nonexpansive linear forms.

Theorem 3.8. Let V be a real t.v.s. and let q : V → R be a weak Minkowski norm, continuous with respect to the topology of V. Denote by P := {p ∈ V * | ∀x ∈ V, p, x q(x)}. A function f : V → R is positively homogeneous and nonexpansive with respect to q if, and only if,

f (x) = min y∈V max p∈Py p, x , (3.6)
where P y := {p ∈ P | p, y f (y)}. Moreover, the value of the expression (3.6) does not change if the maximum is restricted to the linear forms p ∈ ext P y .

Proof. The "if" part of the condition is straightforward, so we only prove that it is necessary. Let f be a real function positively homogeneous and nonexpansive with respect to q. As a direct consequence of nonexpansiveness, we get that

f (x) = min y∈V f (y) + q(x -y) .
Since f is also positively homogeneous, by a change of variable, we have

f (x) = min y∈V inf λ>0 λf (y) + q(x -λy) .
There, the minimum in y is attained at all µx with µ > 0. Given a vector y ∈ V, let g y be the real function defined on V by g y (x) = inf λ>0 λf (y) + q(x -λy), so that f (x) = min y∈V g y (x). We next show that g y is a continuous weak Minkowski norm.

First, g y is bounded above by q and below by f . In particular, g y is bounded above on a neighborhood of each point of V. Second, using the positive homogeneity of f and q, we check that g y also satisfies this property. Third, g y is convex. Indeed, the function x → f (y) + q(x -y) is convex because so is q. Its perspective function, defined on R × V by (λ, x) → λf (y) + q(xλy) if λ > 0 and +∞ otherwise, is also convex, see [START_REF] Bauschke | Convex analysis and monotone operator theory in Hilbert spaces[END_REF]Prop. 8.23]. Hence, the convexity follows from the fact that g y is the marginal function with respect to the first variable of the latter function.

We have shown that g y is a positively homogeneous convex function, finite everywhere and bounded above on a neighborhood of each point of V. Hence, it is continuous on V, see [AB06, Th. 5.43], and we deduce from Lemma 3.2 that it is the support function of the weak-* compact convex set

Q y := {p ∈ V * | ∀x ∈ V, p, x g y (x)}.
To conclude, it remains to show that this set is P y .

Let p ∈ Q y . Then, for every x ∈ V and every λ > 0, we have p, x λf (y) + q(x -λy). Taking λ → 0 we deduce that p ∈ P, and taking x = y and λ = 1 we deduce that p, y f (y). Hence, p ∈ P y which shows that Q y ⊂ P y . Consider now p ∈ P y . For every x ∈ V and every λ > 0 we have p, x = p, x -p, λy + p, λy = p, x -λy + λ p, y q(x -λy) + λf (y) .

Taking the infimum over all λ > 0 in the right-hand side of the last inequality, we get that p, x g y (x). Hence p ∈ Q y which shows that Q y ⊂ P y and consequently that Q y = P y .

The following is an immediate corollary.

Corollary 3.9. Let V be a real vector space with an order unit, denoted by e, and the topology defined by the sup-norm (3.1) associated with that order unit. Then, a function f : V → R is monotone, additively homogeneous, and positively homogeneous if, and only if,

f (x) = min y∈V max p∈∆y p, x , (3.7)
where

∆ y := {p ∈ V * | p, y f (y), p, e = 1, p, x 0, ∀x ∈ V + } .
Moreover, the value of the expression in (3.7) does not change if the minimum is restricted to the vectors y ∈ V such that f (y) = 0, or if the maximum is restricted to the linear forms p ∈ ext ∆ y .

Applications

We now point out some applications of the present representation theorems to nonconvex risk measures and Shapley operators.

Representation of nonconvex risk measures

Let (Ω, F , P) be a probability space. Denote by L ∞ (P) the space of almost surely bounded realvalued random variables, equipped with the usual essential sup-norm. This is a special case of an AM-space with unit, the unit 1 being the random variable a.s. equal to 1. Its topological dual space is the space of finitely additive measures of bounded variation which are absolutely continuous with respect to P, see [DS88, Th. IV.8.16]. We denote it by ba(P). Following standard notation, we shall write E p [X] instead of p, X for a random variable X ∈ L ∞ (P) and a measure p ∈ ba(P). We denote by ba + (P) the set of positive bounded finitely additive measures and by ∆(P) := {p ∈ ba + (P) | E p [1] = 1} the set of finitely additive probability measures.

A risk measure is a real function µ : L ∞ (P) → R that satisfies the following conditions, for all X, Y ∈ L ∞ (P): [START_REF] Föllmer | Stochastic finance[END_REF]. Observe that this is equivalent to the function ρ : X → µ(-X) being monotone and additively homogeneous.

(i) X Y =⇒ µ(X) µ(Y ) ; (ii) µ(X + λ) = µ(X) -λ, ∀λ ∈ R ; see
A risk measure µ is coherent if it is in addition convex and positively homogeneous, see e.g. [START_REF] Artzner | Coherent measures of risk[END_REF][START_REF] Delbaen | Coherent risk measures on general probability spaces[END_REF]. It is known that a coherent risk measure can be represented in the form

µ(X) = sup p∈Q ba E p [-X]
where Q ba ⊂ ∆(P) is a convex subset of the space of finitely additive probability measures on Ω, closed with respect to the σ(ba(P), L ∞ (P))-topology. As a direct application of Corollary 3.6, we obtain a similar representation for general nonconvex risk measures.

Corollary 3.10. Let µ be a risk measure on L ∞ (P). Then,

µ(X) = min Y ∈L ∞ (P) µ(Y )=0 max p∈∆(P) E p [Y -X] .
The following corollary characterizes the nonconvex risk measures that are positively homogeneous. It is a direct application of Corollary 3.9.

Corollary 3.11. Let µ be a positively homogeneous risk measure. Then,

µ(X) = min Y ∈L ∞ (P) µ(Y )=0 max p∈∆(P) Ep[Y ] 0 E p [-X] .

Representation of payment-free Shapley operators

Here, we consider the vector space V = R n , where •, • denotes the standard scalar product. Recall that a payment-free Shapley operator on R n is a monotone, additively and positively homogeneous operator from R n to itself. As mentioned in the introduction, this terminology is justified by the following corollary, which shows that all operators of this kind precisely arise from zero-sum stochastic games the payment function of which is identically zero. Corollary 3.12. Let F : R n → R n be a monotone, additively and positively homogeneous operator. Then, each coordinate map F i with i ∈ [n] can be represented as

F i (x) = min a∈A i max b∈B i,a j∈[n] x j p(j | i, a, b) , ∀x ∈ R n , (3.8)
where

A i := {a ∈ R n | F i (a) = 0}; for every i ∈ [n] and every a ∈ A i , B i,a is a finite subset of R n ; for every a ∈ A i and every b ∈ B i,a , (p(j | i, a, b)) j∈[n]
is a stochastic vector with at most two positive coordinates.

Proof. Each coordinate function of F is monotone, additively and positively homogeneous. Then, it follows from Corollary 3.9 that, for all i ∈ [n] and all x ∈ R n ,

F i (x) = min y∈R n F i (y)=0 max p∈∆n p,y 0 p, x . (3.9)
where

∆ n is the standard simplex of R n . Let A i := {a ∈ R n | F i (a) = 0}. For a ∈ A i , let B i,a be the set of extreme points of the polytope {p ∈ ∆ n | p, a
0} (points at which the maximum in (3.9) is attained). Finally, for

a ∈ A i , b ∈ B i,a and j ∈ [n], let p(j | i, a, b) = b.
Rewriting equation (3.9) using these notations we get exactly the formula (3.8).

A standard result of convex geometry shows that every extreme point of the intersection of a polytope with a half-space is either an extreme point of the polytope, or a convex combination of two extreme points of this polytope (see for instance [START_REF] Fukuda | Double description method revisited[END_REF]Lem. 3]). It follows that every element in B i,a is either an extreme point of ∆ n or a convex combination of two extreme points of ∆ n .

Approximation of Shapley operators

We use the previous result to approximate payment-free Shapley operators on R n by minimax maps where the min and max operators are taken over finite sets (i.e., continuous piecewise affine operators). The latter maps play an important role algorithmically, in max-plus finite element method [START_REF] Akian | The max-plus finite element method for solving deterministic optimal control problems: basic properties and convergence analysis[END_REF], and more generally in idempotent methods [START_REF] Mceneaney | Distributed dynamic programming for discrete-time stochastic control, and idempotent algorithms[END_REF][START_REF] Mceneaney | Development of an idempotent algorithm for a network-delay game[END_REF].

Let q : R n → R be a weak Minkowski norm. We say that a subset A ⊂ R n is an ε-net of any set K ⊂ R n with respect to (the symmetrization of) q if inf a∈A max {q(x -a), q(a -x)} < ε , ∀x ∈ K .

Note that here, since the dimension is finite, q is continuous, and then it is always possible to find a finite ε-net of any compact set with respect to q.

The following result, when applied to any Shapley operators on R n , shows that the latter can be approximated on any compact set by the Shapley operators of a repeated game with deterministic transitions and finite action spaces. Proposition 3.13. Let q : R n → R be a weak Minkowski norm and let f : R n → R be a map nonexpansive with respect to q. Then, for every subset K ⊂ R n , and for every finite ε-net (y ) ∈[m] of K with respect to q, the function

g(x) = min ∈[m] f (y ) + q(x -y ) is such that f (x) g(x) f (x) + 2ε , ∀x ∈ K .
Proof. Let x ∈ K. Since f is nonexpansive with respect to q, we have f (x) f (y ) + q(x -y ) for all indices ∈ [m]. Thus, we have f (x) g(x). We also know that there exists an index 0 such that max{q(x -y 0 ), q(y 0 -x)} < ε. Hence,

f (y 0 ) + q(x -y 0 ) f (x) + q(y 0 -x) + q(x -y 0 ) f (x) + 2ε ,
from which the second inequality follows.

Note that in the previous proposition, the polyhedral function g is not positively homogeneous, even if f is. The next result shows that when a Shapley operator is payment-free, it can be approximated by another payment-free Shapley operator.

Corollary 3.14. Let F : R n → R n be a payment-free Shapley operator. Then, for all ε > 0, there exists a payment-free Shapley operator G : R n → R n such that

F (x) G(x) F (x) + ε x ∞ e , ∀x ∈ R n ,
which can be represented in the form

G i (x) = min a∈A i max b∈B i,a j∈[n] x j p(j | i, a, b) , ∀x ∈ R n , ∀i ∈ [n] ,
where A i and B i,a are finite sets, and every (p(j | i, a, b)) j∈[n] is a stochastic vector with at most two positive coordinates.

Proof. Let (z ) ∈[m] be an ε/2-net of the unit sphere of R n with respect to the sup-norm. Applying Corollary 3.12 to F , we know that for all i ∈ [n] and ∈ [m], there exists an action a i ∈ A i for which the minimum is attained in the minimax representation (3.8) of

F i (z ). Let A * i := {a i | ∈ [m]
} be the finite subset of A i containing all these optimal actions in state i ∈ [n]. Then, let G : R n → R n be the payment-free Shapley operator the ith coordinate map of which is given by

G i (x) := min a∈A * i max b∈B i,a j∈[n] x j p(j | i, a, b) , ∀x ∈ R n ,
where the action spaces B i,a are the same than in the minimax representation (3.8) of F i . In particular, we know that they are finite and that all the vectors (p(j | i, a, b)) j∈[n] are stochastic, with at most two positive coordinates.

By construction, we have F (x) G(x) for all vectors x ∈ R n , with equality for every z , ∈ [m]. Now, given a vector x in the unit sphere of R n , we can choose a vector z such that x -z ∞ ε/2. Since both F and G are nonexpansive with respect to the sup-norm, we deduce that for all i ∈ [n],

G i (x) -F i (x) G i (z ) + ε/2 -F i (x) = F i (z ) -F i (x) + ε/2 ε .
The conclusion follows from the positive homogeneity of F and G.

Part II

Ergodicity of zero-sum stochastic games

CHAPTER 4

Analytic characterizations of ergodicity of stochastic games

Introduction

The problem of existence of the mean-payoff vector for a stochastic game with finite state space may be addressed by finding a solution to the ergodic equation (2.5), involving its Shapley operator T . The latter equation has a solution if, and only if, some orbit of T is bounded in Hilbert's seminorm (Theorem 2.6). Finding conditions under which this property is true is difficult in general, and depends not only on the structure of the game (i.e., the support of the transition probabilities), but also on the values of the parameters (payment and transition functions). Following Gaubert and Gunawardena [START_REF] Gaubert | The Perron-Frobenius theorem for homogeneous, monotone functions[END_REF], a better posed problem consists in finding conditions under which all the subsets of a given family of invariant spaces are bounded in Hilbert's seminorm. Such conditions usually depend only on the structure of the game (see Theorem 2.8 for instance). Then the solvability of the ergodic equation is ensured not only for T , but also for some family of perturbations of its parameters.

An elementary case illustrating this stability property is given by ergodic finite Markov chains. Following Kemeny and Snell [START_REF] Kemeny | Finite Markov chains[END_REF], we define the ergodicity of a finite Markov chain by one of the equivalent properties listed in the theorem below, involving its transition matrix. Recall in particular that for a n × n stochastic matrix P , the directed graph associated with P is composed of the nodes 1, . . . , n and of the arcs (i, j) such that P ij > 0. A class of the matrix P is a maximal set of nodes such that every two nodes in this set are connected by a directed path. A class is said to be final if every path starting from a node of this class remains in it. We refer the reader to [START_REF] Kemeny | Finite Markov chains[END_REF] and [START_REF] Berman | Nonnegative matrices in the mathematical sciences[END_REF] for the proof of the theorem. Theorem 4.1. Let P ∈ R n×n be a stochastic matrix. The following assertions are equivalent:

(i) every vector v ∈ R n such that P v = v is constant;

(ii) for all vectors g ∈ R n , there exists an eigenpair (λ, u) ∈ R × R n such that g + P u = λe + u;

(iii) for all vectors g ∈ R n , the Cesàro limit lim k→∞ (g + P g + • • • + P k-1 g)/k is a constant vector;

(iv) the directed graph associated with the matrix P has a unique final class; (v) the stochastic matrix P has a unique invariant probability measure.

A finite Markov chain with transition matrix P ∈ R n×n may be seen as a zero-player stochastic game with state space [n]. If g i is the payment in state i ∈ [n], then its Shapley operator is given by T (x) = g + P x for all x ∈ R n , and it is readily seen that its recession operator is equal to T (x) = P x. Thus, the ergodicity of a finite Markov chain may be characterized, in game-theoretic terms, either by all the fixed points of the recession operator T being constant (Point (i)), or by the ergodic equation having a solution for all Shapley operators g + T with g ∈ R n (Point (ii)), or by the mean-payoff vector being constant for all payment vectors g ∈ R n (Point (iii)). A natural question is to know if this result may be extended to any finite stochastic game.

Let us recall that the stability of existence of a constant mean payoff, based on structural properties, was observed and studied in various framework. In the case of finite Markov chains with risk-sensitive long-run average cost criterion, Cavazos-Cadena and Hernández-Hernández [START_REF] Cavazos-Cadena | Necessary and sufficient conditions for a solution to the risk-sensitive Poisson equation on a finite state space[END_REF] gave a necessary and sufficient condition on the transition matrix for the Poisson equation -a dynamic programming type equation -to have a solution for any cost function. Still motivated by risk-sensitive applications, the same authors [START_REF] Cavazos-Cadena | Poisson equations associated with a homogeneous and monotone function: necessary and sufficient conditions for a solution in a weakly convex case[END_REF] gave, under a weakly convex property and in a multiplicative framework, necessary and sufficient conditions for the nonlinear spectral problem (2.6) to have a solution for arbitrary (multiplicative) perturbations of the payments. Also, in optimal control (with continuous time and infinite state space) Arisawa [START_REF] Arisawa | Ergodic problem for the Hamilton-Jacobi-Bellman equation. I. Existence of the ergodic attractor[END_REF][START_REF] Arisawa | Ergodic problem for the Hamilton-Jacobi-Bellman equation[END_REF] studied the ergodic problem of Hamilton-Jacobi-Bellman equations and particularly the link between the existence of the ergodic constant for arbitrary cost functions depending only on the state and the existence of an "ergodic attractor", depending on the dynamic.

In this chapter and the next one, we extend the notion of ergodicity to zero-sum stochastic games. This leads us to the following definition. Definition 4.2. A Shapley operator T : R n → R n is ergodic if the ergodic equation (2.5) is solvable for all operators g + T with g ∈ R n . A zero-sum stochastic game with finite state space is ergodic if its Shapley operator is ergodic.

In this chapter, along the lines of Gaubert and Gunawardena [START_REF] Gaubert | The Perron-Frobenius theorem for homogeneous, monotone functions[END_REF], we start by characterizing ergodicity in terms of recession operator, assuming that the asymptotic behavior of the Shapley operator and its recession operator are the same. Then we characterize ergodicity in full generality, in terms of slice spaces, using the connections between nonexpansive and accretive maps. In the following two chapters we deal with the combinatorial aspects of these conditions. In particular, we formulate them in terms of hypergraphs, underlying their structural aspects.

The notion of ergodicity for stochastic games and the results of Section 4.2 are based on the first part of the paper [START_REF] Akian | Fixed point sets of payment-free Shapley operators and structural properties of mean payoff games[END_REF], published in Discrete and Continuous Dynamical Systems, Series A. The results regarding the accretive operator approach (Section 4.3 and 4.4) have been announced in the MTNS conference proceedings [START_REF] Hochart | An accretive operator approach to ergodic problems for zero-sum games[END_REF].

Characterization of ergodicity in terms of recession operator

Let T be a Shapley operator on R n . Assume that its recession operator T exists. We know that if all the fixed points of T are trivial (i.e., constant vectors), then all the slice spaces of T are bounded in Hilbert's seminorm (Theorem 2.9). This implies that the ergodic equation (2.5) is solvable and that the mean-payoff vector is constant. Note that it remains true for all operators g + T with g ∈ R n , since they all have the same recession operator. Hence, T is ergodic.

The following proposition specifies the link between the recession operator and the (not necessarily constant) mean-payoff vector, when they do exist. We give the short proof for convenience of the reader.

Proposition 4.3 ([RS01a]

). Let T be a Shapley operator on R n . Assume that its recession operator T and the mean-payoff vector χ ∈ R n exist. Then, T (χ) = χ.

Proof. Since T is nonexpansive in the sup-norm, we have, for all vectors x, y ∈ R n and all integer k, T (kx) -T (ky) k x -y .

Then, letting x = χ and y = T k (0)/k, we get

T (kχ) k - T k+1 (0) k χ - T k (0) k .
Now, taking the limits in k, we obtain that T (χ) -χ = 0.

A converse statement holds.

Proposition 4.4. Let F be a payment-free Shapley operator on R n , and let v be a fixed point of F . Then, there is a Shapley operator T : R n → R n such that its recession operator exists and is given by T = F , and such that its mean-payoff vector is equal to v.

Proof. Let T := v + F . It is readily seen that T = F . Moreover, for every integer k, we have T (kv) = (k + 1)v, so that, by induction, T k (0) = kv. Hence the mean-payoff vector associated with T exists and is equal to v.

Remark 4.5. Let Γ(r) = ([n], A, B, K A , K B , r, p) be a parametric stochastic game whose parameter r is chosen among all bounded real functions on K. It is readily seen that all Shapley operators of the games Γ(r) have the same recession operator, denoted by F . The latter is the (payment-free) Shapley operator of the game Γ(0). Then, the fixed points of F give exactly all the realizable mean-payoff vectors of the family of games (Γ(r)).

We now state the main result of this section, which gives a characterization of ergodicity of a Shapley operator T : R n → R n in the sense of Definition 4.2. This characterization relies on the existence of its recession operator. More precisely, we shall need the following assumption.

Assumption 4.A. The recession operator T : R n → R n is well defined and there exists a real constant M > 0 such that

T i (x) -T i (x) M , ∀x ∈ R n , ∀i ∈ [n] .
Note that Assumption 4.A is satisfied as soon as T is the Shapley operator of a stochastic game with bounded payment function, the constant M being a bound on the payments. Indeed, if T : R n → R n is given by formula (2.1), then the ith component of its recession operator is readily seen to be

T i (x) = inf µ∈∆(A i ) sup ν∈∆(B i ) j∈[n] x j p(j | i, µ, ν) , x ∈ R n ,
where the infimum and the supremum commutes. Moreover, Assumption 4.A implies that the convergence in the limit (2.7) defining T is uniform in x.

Theorem 4.6. Let T : R n → R n be a Shapley operator satisfying Assumption 4.A. The following assertion are equivalent:

(i) T has only trivial fixed points;

(ii) T is ergodic, i.e., for all vectors g ∈ R n , the ergodic equation (2.5) is solvable for g + T ;

(iii) for all vectors g ∈ R n , the mean-payoff vector associated with g + T exists and is constant.

Proof. From Theorem 2.9, we know that (i) ⇒ (ii) and the implication (ii) ⇒ (iii) is straightforward. Hence, all the equivalences will follow from the implication (iii) ⇒ (i).

Assume that Point (iii) holds and let v be a fixed point of T . Let be any integer and let T := v + T . We next show by induction that, for all integers k, we have

k( v -M e) (T ) k (0) k( v + M e) .
(4.1)

For k = 0, the inequalities are trivial. Assume now that (4.1) holds for some integer k. Then, by monotonicity of T we have

v + T k( v -M e) (T ) k+1 (0) v + T k( v + M e) .
By additive homogeneity of T we deduce that

v -kM e + T (k v) (T ) k+1 (0) v + kM e + T (k v) .
Using Assumption 4.A, we get that

v -(k + 1)M e + T (k v) (T ) k+1 (0) v + (k + 1)M e + T (k v) ,
and since v is a fixed point of T , the positive homogeneity yields

(k + 1)( v -M e) (T ) k+1 (0) (k + 1)( v + M e) .
According to Point (iii), the mean-payoff vector of T exists and is constant. Let λ be this constant value. Then we have v -M e λ e v +M e, which may be rewritten as (λ -M )e v (λ + M )e. This entails v H = v H 2M . Since the latter inequality holds for any integer , we deduce that v H = 0, which means that v is proportional to the unit vector. Note that the above theorem readily extends the case of finite Markov chains, stated in Theorem 4.1 (Points (i) -(iii)). The graph-theoretic aspect of Point (iv) will be treated in the next chapter.

Accretive operator approach surjectivity conditions for accretive maps

The purpose of this section and the next one is to extend the characterization of ergodicity to any Shapley operators on R n . Indeed the conclusions of Theorem 4.6 are not valid in general. For instance, one cannot conclude when the recession operator of T does not exist. But there are also situations where T , although well defined, fails to have only trivial fixed points, whereas all slice spaces S β α (T ) are bounded in Hilbert's seminorm, implying ergodicity. Such a counterexample is provided in Subsection 6.3, others may be found in [START_REF] Gaubert | The Perron-Frobenius theorem for homogeneous, monotone functions[END_REF].

In the present section, we establish results about the surjectivity of accretive maps in finite dimension. Then, in the next section, we exploit the link between nonexpansiveness and accretivity to infer stability results about the existence of fixed points for nonexpansive maps, including Shapley operators.

Preliminaries

In the remainder of the chapter, X shall always be a finite-dimensional real vector space. We denote by X * its dual space, and by •, • the duality product. We assume that X is equipped with a given norm, denoted by • . The dual norm on X * is denoted by • * .

Set-valued maps

Given a space Y, we denote by A : X ⇒ Y a set-valued map A from X to Y, i.e., a map from X to the powerset of Y. We shall need the following definitions.

-The domain of A is the subset of X defined by dom(A) :

= {x ∈ X | A(x) = ∅}; -The range of A is the subset of Y defined by rg(A) := x∈X A(x); -The image of a subset V ⊂ X under A is the subset of Y given by A(V) := x∈V A(x)
. The inverse of A, denoted by A -1 , is the set-valued map from Y to X sending an element y ∈ Y to the subset {x ∈ X | y ∈ A(x)}, so that x ∈ A -1 (y) if, and only if, y ∈ A(x). In particular, note that (A -1 ) -1 = A, and that rg(A) = dom(A -1 ). Furthermore, the image of a subset W ⊂ Y under A -1 can be written as

A -1 (W) = {x ∈ X | A(x) ∩ W = ∅}.
We shall also need the notions of upper and lower semicontinuity. The set-valued map A : X ⇒ Y is upper semicontinuous at point x ∈ dom(A) if for every neighborhood V of A(x), there exists a neighborhood U of x such that A(y) ⊂ V for all y ∈ U; -lower semicontinous at point x ∈ dom(A) if for every open set V that meets A(x), meaning that V ∩ A(x) = ∅, there exists a neighborhood U of x such that A(y) ∩ V = ∅ for every y ∈ U; -continuous at x ∈ dom(A) if it is both upper and lower semicontinuous at point x. The map A is upper (resp., lower) semicontinuous, if it is upper (resp., lower) semicontinuous at every point x ∈ dom(A). Likewise, it is continuous if it is continuous at every point x in dom(A). Assume now that X and Y are metric spaces. The latter notions may be described in terms of sequences. Let x be a point in the domain of A. The map A is -upper semicontinous at x and A(x) is compact if, and only if, for every sequence of points x k ∈ dom(A) converging to x and every y k ∈ A(x k ), the sequence (y k ) has an accumulation point in A(x); -lower semicontinuous at x if, and only if, for every y ∈ A(x) and for every sequence of points x k ∈ dom(A) converging to x, there exists a sequence of points y k ∈ A(x k ) converging to y.

We refer the reader to the monograph of Aubin and Frankowska [START_REF] Aubin | Modern Birkhäuser Classics[END_REF] for background on set-valued maps.

Normalized duality mapping

The concept of duality mappings was introduced by Beurling and Livingston [START_REF] Beurling | A theorem on duality mappings in Banach spaces[END_REF], and developed later on by Asplund [START_REF] Asplund | Positivity of duality mappings[END_REF], Browder [Bro65, Bro66, Bro76], Browder and de Figueiredo [START_REF] Browder | J-monotone nonlinear operators in Banach spaces[END_REF]. In a Banach space, there is a strong link between the geometric properties of the space (strict convexity, smoothness, etc.) and the analytic properties of the duality mappings (singlevalued, continuity, etc.). Here, we only consider the normalized duality mapping, and refer the reader to the above-mentioned papers for a more general presentation.

Definition 4.7. The (normalized) duality mapping on (X , • ) is the set-valued map J : X ⇒ X * defined by

J(x) := {x * ∈ X * | x * * = x , x, x * = x * * x } , x ∈ X .
Note that, by the Hahn-Banach theorem, dom(J) = X , i.e., J(x) is nonempty for every vector x ∈ X . Furthermore, Asplund [Asp67, Th. 1] characterized duality mappings as subdifferentials of convex functions. This entails that J(x) is a compact convex subset of X * for every x ∈ X . Also, it readily stems from the definition that J is homogeneous of degree one, i.e., for every x ∈ X and every λ ∈ R, we have J(λx) = λJ(x).

Finally, we shall need the following result, obtained by Browder (see the proof of Th. 7.3 in the reference).

Proposition 4.8 [START_REF] Browder | Nonlinear operators and nonlinear equations of evolution in Banach spaces[END_REF]). The normalized duality mapping on a finite-dimensional vector space is upper semicontinuous.

Example 4.9. Let X = R n . If X is equipped with the standard Euclidean norm, observe that J is the identity map. More generally, if X is equipped with an L p -norm with p > 1, then J is single-valued and given for all x = 0 by

J(x) = x p x q x ,
where q is the positive real number such that 1 p + 1 q = 1.

Example 4.10. Assume that the norm • on X is polyhedral, meaning that there is a finite symmetric family W ⊂ X * of linear forms on X such that

x = max x * ∈W x, x * , ∀x ∈ X .
We may assume that W is minimal, in the sense that no element in W may be written as a convex combination of other elements in W. This case includes the standard L 1 -and L ∞norms. Then, one may check that we have, for every x ∈ X ,

J(x) = x co{x * ∈ W | x, x * = x } ,
where co(V) stands for the convex hull of any set V.

Accretive maps

Accretive operators initially appeared as differential operators, and more precisely as infinitesimal generators of one-parameter semigroup of nonexpansive self-maps on a Banach space. We refer the reader to the series of papers [START_REF] Browder | Nonlinear accretive operators in Banach spaces[END_REF][START_REF] Browder | Nonlinear mappings of nonexpansive and accretive type in Banach spaces[END_REF][START_REF] Browder | Nonlinear monotone and accretive operators in Banach spaces[END_REF] by Browder and to the references therein for background on the subject, as well as to the more detailed monograph [START_REF] Browder | Nonlinear operators and nonlinear equations of evolution in Banach spaces[END_REF]. See also Brezis [START_REF] Brézis | Opérateurs maximaux monotones et semi-groupes de contractions dans les espaces de Hilbert[END_REF], Crandall and Liggett [START_REF] Crandall | Generation of semi-groups of nonlinear transformations on general Banach spaces[END_REF], Crandall and Pazy [START_REF] Crandall | Nonlinear evolution equations in Banach spaces[END_REF], Kato [START_REF] Kato | Nonlinear semigroups and evolution equations[END_REF], Martin [START_REF] Martin | A global existence theorem for autonomous differential equations in a Banach space[END_REF], for standard results in the theory of accretive operators.

Definition 4.11. Let A : X ⇒ X be a set-valued map. Denote by Id the identity map on X .

-A is accretive if, for every x, y ∈ X , every u ∈ A(x) and every v ∈ A(y), there exists an element x * ∈ J(x -y) such that u -v, x * 0. -A is m-accretive if it is accretive and if rg(Id +λA) = X for every λ > 0.

-A is coaccretive if, for every x, y ∈ X , every u ∈ A(x) and every v ∈ A(y), there exists an element x * ∈ J(u -v) such that x -y, x * 0.

Let us mention that in a Hilbert space, the normalized duality mapping J is the identity map, and accretivity recovers the notion of monotony, while m-accretive maps are maximal monotone operators. In particular, note that the subdifferential of any lower semicontinuous proper convex function is m-accretive (see [START_REF] Rockafellar | Variational analysis, volume 317 of Grundlehren der Mathematischen Wissenschaften[END_REF]).

Remark 4.12. An equivalent definition of accretivity is the following: a set-valued map A : X ⇒ X is accretive if, and only if, for every x, y ∈ X , every u ∈ A(x) and v ∈ A(y), and every λ > 0, we have x -y

x -y + λ(u -v) . Furthermore, it is known that if A : X ⇒ X is accretive, then rg(Id +λA) = X for every λ > 0 if, and only if, rg(Id +λA) = X for some λ > 0. Finally, a set-valued map A : X ⇒ X is coaccretive if, and only if, A -1 is accretive.

Local boundedness of coaccretive maps

It is known that in a reflexive Banach space, any accretive map A is locally bounded at any point x in the interior of its domain, meaning that there exists a neighborhood V of x such that A(V) is bounded, see [START_REF] Fitzpatrick | Local boundedness of monotone-type operators[END_REF]. The following result shows that the same holds true for coaccretive maps, at least in finite dimension. Proposition 4.13. Let (X , • ) be a finite-dimensional real vector space, and let A : X ⇒ X be a coaccretive map. Then, for any vector x in the interior of dom(A), A is locally bounded at x.

Proof. Assume by contradiction that A is not locally bounded at x, that is, for any neighborhood V of x, A(V) is not bounded. Then, there exists a sequence (x k ) k∈N in X converging to x such that, for all integers k, there is a vector y k ∈ A(x k ), and such that the sequence ( y k ) tends to infinity. Since the dimension is finite, the bounded sequence (y k / y k ) is converging, up to an extraction, toward some point z ∈ X .

The point x being in the interior of dom(A), we may choose a scalar α > 0 such that x+αz ∈ dom(A). Let y ∈ A(x + αz). Since A is coaccretive, for all integers k there is a linear form

y * k ∈ J(y k -y ) such that x k -x -αz, y * k 0 .
By homogeneity of the duality mapping, we know that z * k :=

y * k y k ∈ J y k -y y k
for every integer k, and that

x k -x -αz, z * k 0 . (4.2)
Since J is upper semicontinuous and with compact values (see Subsection 4.3.1), and since the sequence ((y k -y )/ y k ) converges, up to an extraction, toward z, we deduce that the sequence (z * k ) has an accumulation point z * ∈ J(z). Then, Inequality (4.2) implies that z, z * 0, a contradiction since z, z * = z 2 = 1 by definition of the duality mapping.

We deduce an immediate corollary about the image of compact sets under coaccretive maps.

Corollary 4.14. Let (X , • ) be a finite-dimensional real vector space, and let A : X ⇒ X be a coaccretive map. Then, for any compact set K ⊂ X included in the interior of dom(A), A(K) is bounded.

Characterization of surjectivity of accretive maps

Let us denote by B(x, ρ) the closed ball centered at point x ∈ X with radius ρ 0, and by dist 0 (V) the distance of the origin to a subset V ⊂ X , i.e., dist 0 (V) := inf x∈V x .

To show the main theorem of this section, we shall need the following result. It is a special case of a result established by Kirk and Schöneberg [START_REF] Kirk | Zeros of m-accretive operators in Banach spaces[END_REF] (see the last corollary of Th. 3) which is itself a generalization of theorems obtained by Lange [START_REF] Lange | Abbildungssätze für monotone Operatoren in Hilbert-und Banach-Räumen[END_REF] and Kartsatos [START_REF] Kartsatos | On the equation T x = y in Banach spaces with weakly continuous duality maps[END_REF][START_REF] Kartsatos | Some mapping theorems for accretive operators in Banach spaces[END_REF].

Theorem 4.15 ([KS80]

). Let (X , • ) be a finite-dimensional real vector space, and let A : X ⇒ X be an m-accretive set-valued map. Assume that

lim x →∞ dist 0 (A(x)) = +∞ .
Then, rg(A) = X .

We now state the main result of this section. Theorem 4.16. Let (X , • ) be a finite-dimensional real vector space, and let A : X ⇒ X be an accretive map. If rg(A) = X then, for all scalars α 0, the subset D α := {x ∈ X | dist 0 (A(x)) α} is bounded. Moreover, if A is m-accretive, then the converse is also true.

Proof. Assume that rg(A) = X . Equivalently, we have dom(A -1 ) = X . Since A is accretive, its inverse A -1 is coaccretive (see Subsection 4.3.1). Hence, according to Corollary 4.14, the image of any compact set under A -1 is bounded. Now, observe that for every nonnegative real numbers α, α such that α > α, we have

D α ⊂ A -1 B(0, α ) . Indeed, A -1 B(0, α ) = {x ∈ X | A(x) ∩ B(0, α ) = ∅} by definition. Hence, D α is bounded for every α 0.
For the converse property, let us assume that A is m-accretive. It is readily seen that the coercivity condition in Theorem 4.15 is equivalent to the boundedness of all subsets D α , hence the result.

Remark 4.17. Here, since the dimension is finite, all the norms are equivalent. Hence, in the latter theorem, the choice of the norm only matters for the definition of accretivity, but any other norm may be considered in the definition of dist 0 .

Example 4.18 (discrete p-Laplacian). Let G = (V, E) be a connected (undirected) graph with a finite set of nodes an no multiple edges. For brevity, we write i ↔ j if there is an edge between i and j. Let us equip every edge {i, j} ∈ E with a weight C ij > 0. For p > 1, we denote by L p : R V → R V the discrete p-Laplacian, whose ith entry is defined by

[L p (v)] i := j∈V i↔j C ij (v i -v j ) |C ij (v i -v j )| p-2 , v ∈ R V .
Let B be a proper subset of V , i.e., different from ∅ and V . Assume that for every vertex i ∈ B, some real number w i is given. We are interested in the following boundary value problem:

given g ∈ R V \B , find v ∈ R V such that [L p (v)] i = -g i , ∀i ∈ V \ B , v i = w i , ∀i ∈ B . (P g )
Note that the discrete p-Laplacian L p appears as the gradient of the energy function J p defined on R V by

J p (v) := {i,j}∈E 1 pC ij |C ij (v i -v j )| p , v ∈ R V .
Then, Problem (P g ) is equivalent to the following optimization problem:

minimize J p (v) + i∈V \B g i v i s.t. v i = w i , ∀i ∈ B .
In particular, for p = 2 it recovers the classical problem of computing the electrical potential on the graph G, with a prescribed potential w i at each node i ∈ B and a prescribed current g i passing through each node i ∈ V \ B, C ij being the conductance of the edge {i, j}. Let X = R V \B . We reformulate Problem (P g ) by introducing the operator A : X → X whose ith coordinate map is defined, for all x ∈ X , by

A i (x) := j∈V \B i↔j C ij (x i -x j ) |C ij (x i -x j )| p-2 + j∈B i↔j C ij (x i -w j ) |C ij (x i -w j )| p-2 .
Hence, given g ∈ R V \B , Problem (P g ) is equivalent to finding a solution x ∈ X to the equation A(x) = -g. Since A is the gradient of a continuous convex function, then it is m-accretive, X being endowed with the standard Euclidean norm. Thus, according to Theorem 4.16, the latter equation has a solution for all g ∈ X if, and only if, the subsets D α = {x ∈ X | A(x) ∞ α} are bounded for all α 0. We next show that this is the case when the graph G is connected. Let α 0 be such that D α is not empty. Let x ∈ D α and choose i 0 ∈ V \ B such that |x i 0 | = x ∞ . We may assume, without loss of generality, that x i 0 0. If x i 0 max j∈B w j , then x is bounded by a constant which only depends on w ∈ R B . Assume now that x i 0 > max j∈B w j . Since G is connected, there exists an elementary path (i.e., without loop), i 0 , i 1 , . . . , i m , connecting i 0 to some node i m ∈ B and such that i ∈ V \ B for all ∈ {0, 1, . . . , m -1}. By induction, we easily show that |x i -1 -x i | is bounded for all ∈ [m] by a constant M that only depends on α, p and the parameters of G. Thus, we obtain that

x ∞ = x i 0 mM + w im |V |M + w ∞ ,
where |V | is the cardinality of V . This shows that D α is bounded.

Hence, Problem (P g ) has solution for all g ∈ R V \B .

4.4 Accretive operator approach application to nonexpansive maps

Existence stability of xed points under additive perturbations

Let (X , • ) be a finite-dimensional real vector space. Recall that any operator T :

X → X is nonexpansive (with respect to • ) if T (x) -T (y)
x -y for all x, y ∈ X . The following lemma relates nonexpansive operators to accretive maps. Lemma 4.19. Let T : X → X be a nonexpansive operator. Then, the map A := Id -T is m-accretive.

Proof. We first show that A is accretive. Let x, y ∈ X , and let x * ∈ J(x -y). We have

T (x) -T (y), x *
x * * T (x) -T (y)

x * * x -y = x -y, x * .

We deduce that A(x) -A(y), x * = x -y -(T (x) -T (y)), x * 0, which proves that A is accretive. Now, let λ > 0 and z ∈ X . The vector z is in the range of Id +λA if, and only if, the map x → T λ,z (x) := λ 1+λ T (x) + 1 1+λ z has a fixed point. Since T is nonexpansive, T λ,z is a contraction. More precisely, for all x, y ∈ X , we have

T λ,z (x) -T λ,z (y) λ 1 + λ x -y ,
with λ 1+λ < 1. Hence, by the Banach fixed-point theorem, T λ,z has a (unique) fixed point. This proves that rg(Id +λA) = X for every λ > 0, and so A is m-accretive.

Using the previous lemma, we now adapt Theorem 4.16 to the case of nonexpansive maps. Thus, we get a necessary and sufficient condition for the existence of a fixed point for any additive perturbation of any nonexpansive operator.

Corollary 4.20. Let (X , • ) be a finite-dimensional real vector space, and let T : X → X be a nonexpansive operator. Then, the following are equivalent:

(i) for every vector g ∈ X , the operator g + T has a fixed point;

(ii) every nonexpansive operator G : X → X such that sup x∈X T (x) -G(x) < ∞ has a fixed point; (iii) for every scalar α 0, the set

D α (T ) := {x ∈ X | x -T (x)
α} is bounded.

Proof. The equivalence between Point (i) and Point (iii) is a mere application of Theorem 4.16 to Id -T , which is m-accretive according to Lemma 4.19. Moreover, it is straightforward to check that (ii) ⇒ (i). We now prove that (i) ⇒ (ii). Assume that (i) holds and let G : X → X be a nonexpansive operator such that sup x∈X T (x) -G(x) < ∞. Since (i) ⇔ (iii), we know that all the subsets D α (T ) are bounded. We readily deduce that all the subsets D α (G) are also bounded. This implies in particular that G has a fixed point.

Characterization of ergodicity of stochastic games

Let T : R n → R n be a Shapley operator. Recall that T can be quotiented into a self-map [T ] of the additive projective space TP n . This quotiented map is nonexpansive with respect to the norm q H , the quotiented version of Hilbert's seminorm on R n . Moreover, the solvability of the ergodic equation is equivalent to the existence of a fixed point for [T ] (see Subsection 2.3.4). These considerations allow us to apply the result of the previous subsection to characterize the ergodicity of stochastic games.

Theorem 4.21. Let T : R n → R n be a Shapley operator. The following are equivalent:

(i) T is ergodic, i.e., for all vectors g ∈ R n , the ergodic equation (2.5) is solvable for g + T ;

(ii) for all scalars α 0, the space

D H α (T ) := {x ∈ R n | x -T (x) H α} is bounded in Hilbert's seminorm; (iii) for all scalars α, β ∈ R, the slice space S β α (T ) := {x ∈ R n | αe+x T (x) βe+x} is bounded in Hilbert's seminorm; (iv) the ergodic equation (2.5) is solvable for all Shapley operators G on R n such that sup x∈R n T (x)- G(x) H < ∞.
Proof. From the definition of Hilbert's seminorm, we see that S β α (T ) ⊂ D H δ (T ) as soon as β-α δ. It readily follows that (ii) ⇒ (iii).

Let g ∈ R n , and choose α, β ∈ R such that αe g + T (0) βe. Then, S β α (g + T ) is nonempty, and we know that it is invariant under g + T . Moreover, we have S β α (g + T ) ⊂ S β α (T ), with α := α -g ∞ and β := β + g ∞ . Thus, we deduce from Theorem 2.6 that (iii) ⇒ (i).

Assume now that Point (i) holds. Then, in the quotient space (TP n , q H ), the nonexpansive map [g] + [T ] = [g + T ] has a fixed point for all equivalence classes [g] ∈ TP n . Applying Corollary 4.20, we get that for all scalars α 0, the space

D α ([T ]) = {[x] ∈ TP n | q H ([x] -[T (x)]) α} is bounded (in the norm q H ). Since x ∈ D H α (T ) if, and only if, [x] ∈ D α ([T ]
), we deduce that (i) ⇒ (ii). Likewise, we readily get from Corollary 4.20 that (iv) ⇔ (i).

CHAPTER 5

Fixed-point problems for payment-free Shapley operators

In the previous chapter, a characterization of ergodicity for stochastic games with bounded payments (Assumption 4.A) has been given in terms of fixed points of some payment-free Shapley operator (Theorem 4.6). In this chapter, we first give a combinatorial characterization of the latter condition. This characterization involves a pair of directed hypergraphs. Then, we address some algorithmic and complexity issues related to the problem of checking ergodicty of a stochastic game and to the structure of the fixed-point set of a payment-free Shapley operator.

The results of this chapter are based on the paper [START_REF] Akian | Fixed point sets of payment-free Shapley operators and structural properties of mean payoff games[END_REF], published in Discrete and Continuous Dynamical Systems, Series A.

Galois connection associated with payment-free Shapley operators

Preliminaries on lattices and Galois connections

Let (L, ≺) be a partially ordered set (poset, for short), and let X be a subset of L. An element a ∈ L is a lower bound for X if a ≺ x for all x ∈ X. The subset X has a greatest lower bound, or infimum, if there exists a lower bound a for X such that a ≺ a for every other lower bound a.

Dually, an element b ∈ L is an upper bound for X if x ≺ b for all x ∈ X. The subset X has a least upper bound, or supremum, if there exists an upper bound b for X such that b ≺ b for every other upper bound b. Furthermore, if b ∈ X, then the latter element is said to be the maximum of X, and we denote it by max X.

A poset L is an inf-semilattice (resp., sup-semilattice) is every nonempty finite subset has a greatest lower bound (resp., least upper bound). It is a lattice if it is both an inf-semilattice and a sup-semilattice. In this work, we shall consider lattices the elements of which are subsets, equipped with the inclusion partial order, and particularly the powerset lattice P(X) of any set X.

We next recall the definition of a Galois connection. This notion was first introduced by Birkhoff for lattices of subsets, in the first edition of [START_REF] Birkhoff | Lattice theory[END_REF], and then generalized by Ore [START_REF] Ore | Galois connexions[END_REF]. Let (A, ≺ A ) and (B, ≺ B ) be two posets, and let φ : A → B and ψ : B → A be two maps. The map φ is antitone if a ≺ A a implies φ(a ) ≺ B φ(a) (and similarly for ψ). The pair (φ, ψ) is a Galois connection between A and B if it satisfies one of the equivalent properties listed in the following theorem, where id A (resp., id B ) denotes the identity map on A (resp., on B). (5.1)

This implies in particular that, for all b ∈ B,

(∃a ∈ A, b = ϕ(a)) ⇐⇒ φ • φ (b) = b .
Also, observe that the definition of a Galois connection is symmetric, in the sense that (φ, ψ) is a Galois connection between A and B if, and only if, (ψ, φ) is a Galois connection between B and A. Hence, for any statement involving the Galois connection (φ, ψ), a dual statement holds for the Galois connection (ψ, φ). In particular, we get that

(φ ) = φ .
We say that an element a ∈ A is closed (with respect to the Galois connection (φ, φ )) if a = φ • φ(a). We can show that the set of closed elements in A with respect to (φ, φ ) is

Ā = φ (B). Dually, the set of closed element in B is B := {b ∈ B | b = φ • φ(b)} = φ(A).
Then, φ is a bijection from Ā to B, and its inverse is φ .

Galois connection between invariant faces of the hypercube

Before introducing the Galois connection that will be useful to describe the fixed points of a payment-free Shapley operator, let us make some observation. We shall us in the remainder the following notation: the complement of any subset I ⊂ [n] is denoted either by I c or [n] \ I, and we denote by e I the vector in R n with entries 1 on I and 0 on I c :

[e I ] j = 1, ∀j ∈ I and [e I ] j = 0, ∀j / ∈ I .
Also, for x ∈ R n , we denote

arg min x := {i ∈ [n] | x i = min ∈[n]
x } and arg max x

:= {i ∈ [n] | x i = max ∈[n]
x } .

Lemma 5.2. Let F : R n → R n be a payment-free Shapley operator. If u ∈ R n is a fixed point of F then, denoting by I := arg min u and J := arg max u, we have

F (e I c ) e I c , (5.2)
e J F (e J ) .

(5.3)

Proof. Let us assume that min ∈[n] u = 0. Since F is additively homogeneous, there is no loss of generality. We also may assume that e I c u, up to the product by a positive constant. Hence, by monotonicity of F , we get that F (e I c ) u. In particular, we have F i (e I c ) 0 for every index i ∈ I. Moreover, since e I c e, and since the unit vector is a fixed point of F , we have F (e I c ) e. It follows that F (e I c ) e I c .

We show the second inequality using the same arguments, making the nonrestrictive assumption that max ∈[n] u = 1 and u e J .

Remark 5.3. Conditions (5.2) and (5.3) are dual. Indeed, let F be the conjugate operator of F defined by F (x) := -F (-x). Then, it is readily checked that F is a payment-free Shapley operator. Moreover, since F (x) = e -F (e -x), then F (e I ) = e -F (e I c ) for any subset I of [n]. Hence, condition (5.2) holds for F and I if, and only if, condition (5.3) holds for F and I. Furthermore, if u is a (nontrivial) fixed point of F , then the vector ũ := e -u is a (nontrivial) fixed point of F , verifying arg max ũ = arg min u.

Remark 5.4. The inequalities in Lemma 5.2 may be stated in geometric terms. First note that the monotonicity of F and the fact that every trivial vector (i.e., proportional to the unit vector) is a fixed point yields that the hypercube [0, 1] n is invariant by F . Thus, it is readily seen that

F (e I c ) e I c ⇐⇒ F i (e I c ) = 0 , ∀i ∈ I ,
(5.4) e J F (e J ) ⇐⇒ F j (e J ) = 1 , ∀j ∈ J .

(5.5)

Now, let us define

K - I := {x ∈ [0, 1] n | x i = 0, ∀i ∈ I} and K + J := {x ∈ [0, 1] n | x j = 1, ∀j ∈ J} ,
two faces of the hypercube. We shall call them lower and upper faces, respectively. Alternatively, we can define these faces by

K - I = {x ∈ [0, 1] n | x e I c } and K + J = {x ∈ [0, 1] n | x e J } .
From the monotonicity of F , we deduce that the inequalities in Lemma 5.2 are equivalent to the invariance of such faces of the hypercube, that is,

F (e I c ) e I c ⇐⇒ F (K - I ) ⊂ K - I , e J F (e J ) ⇐⇒ F (K + J ) ⊂ K + J .
Given a payment-free Shapley operator F on R n , we now introduce the two families of subsets of [n], denoted byF + and F -, satisfying the inequalities (5.2) and (5.3), respectively:

F + := {I ⊂ [n] | F (e I c ) e I c } , F -:= {J ⊂ [n] | e J F (e J )} .
According to the geometric interpretation (Remark 5.4), the two families F + and F -can be identified with the families of lower and upper invariant faces of the hypercube, respectively. However, the choice of the superscripts is motivated by the fact that I ∈ F + (resp., J ∈ F -) if the vector e I c (resp., e J ) is a "superharmonic" (resp., "subharmonic") vector of F .

The latter families are lattices of subset with respect to the inclusion partial order. Indeed, since F is monotone, for all I 1 , I 2 ∈ F + , we have

F (e (I 1 ∪I 2 ) c ) = F inf{e (I 1 ) c , e (I 2 ) c } inf F (e (I 1 ) c ), F (e (I 2 ) c ) inf{e (I 1 ) c , e (I 2 ) c ) = e (I 1 ∪I 2 ) c ,
so that I 1 ∪ I 2 ∈ F + . This implies that the supremum of two sets in F + coincides with their supremum in the powerset lattice P([n]), namely the union I 1 ∪ I 2 . Hence, F + is a supsubsemilattice of P([n]). Moreover, F + has a bottom element (namely the empty set) and it is a finite poset. Then, it is automatically an inf-semilattice. Check that the infimum of two sets I 1 , I 2 ∈ F + is given by

I 3 ∈F + I 3 ⊂I 1 , I 3 ⊂I 2 I 3 .
Note that the latter infimum may differ from the infimum in P([n]), namely the intersection I 1 ∩ I 2 . We prove that F -is a lattice by dual arguments. Also, note that F + and F -both contain ∅ and [n].

Following the conclusions of Lemma 5.2, given a subset I ∈ F + , we are interested in the subsets J ∈ F -satisfying I ∩ J = ∅. We shall consider in particular the greatest subset J with the latter property. Vice versa, given a subset J ∈ F -, we shall consider the greatest subset I ∈ F + such that I ∩ J = ∅. Put in geometric terms, to each lower invariant face K - I of the hypercube [0, 1] n , we consider the greatest upper invariant face K + J with nonempty intersection with K - I . Let us show that this defines a Galois connection between the lattices F + and F -. Let (Φ, Φ ) be the pair of functions from F + to F -and from F -to F + , respectively, that have just been introduced. Formally, they are defined for every I ∈ F + and J ∈ F + by:

Φ(I) := J∈F - I∩J=∅ J and Φ (J) := I∈F + I∩J=∅ I .
(5.6)

It follows readily from their definition that Φ and Φ are antitone, that I ⊂ Φ • Φ(I) and J ⊂ Φ • Φ (J). Hence condition (i) in Theorem 5.1 is satisfied for the pair (Φ, Φ ) which proves that it is a Galois connection between the lattices of subsets F + and F -.

Galois connection and nontrivial xed points

We now explore some properties of the Galois connection introduced in the previous subsection. Let us first rephrase and complete Lemma 5.2, by a direct application of its definition.

Lemma 5.5. Let F be a payment-free Shapley operator on R n . If u is a fixed point of F , then arg min u ∈ F + and arg max u ∈ F -.

Furthermore, if u is nontrivial, we have arg max u ⊂ Φ(arg min u) and arg min u ⊂ Φ (arg max u) .

In the next lemma, we use the notation

F ω (x) := lim k→∞ F k (x) , x ∈ R n ,
as soon as the limit exists. This is the case in particular when F (x)

x or x F (x). Indeed, since F is monotone, the former (resp., the latter) inequality implies that the sequence (F k (x)) k∈N is nonincreasing (resp., nondecreasing). Moreover, since F is nonexpansive in the sup-norm and has a fixed point (namely 0), the sequence (F k (x)) k∈N is bounded, so that it converges to a finite point when k tends to infinity. Also, observe that F ω (x), if it is well defined, is a fixed point of F . Lemma 5.6. Let F be a payment-free Shapley operator on R n . Let I ∈ F + and J ∈ F -be such that Φ(I) = ∅ and Φ (J) = ∅. Then, arg max F ω (e I c ) = Φ(I) and arg min F ω (e J ) = Φ (J) .

Furthermore, if I (resp., J) is closed with respect to the Galois connection (Φ, Φ ) (resp., (Φ , Φ)), then arg min F ω (e I c ) = I (resp., arg max F ω (e J ) = J) .

Proof. First, note that since F (e I c ) e I c , the sequence (F k (e I c )) k∈N is nonincreasing and so the limit

u := F ω (e I c ) does exist. Since F leaves [0, 1] n invariant, we have u ∈ [0, 1] n .
Second, by definition of the Galois connection, we have e Φ(I) e I c . Using the monotonicity of F and the characterization of F + and F -, we get that The vector u being a fixed point of F , we know from Lemma 5.5 that arg min u ∈ F + , arg max u ∈ F -, and arg max u ⊂ Φ(arg min u). Hence, from the previous inclusions we deduce that Φ(arg min u) = Φ(I) = arg max u.

Suppose now that I is closed with respect to the Galois connection. This means that Φ • Φ(I) = I. Then, from the previous equalities, we get that Φ • Φ(arg min u) = I. This implies that arg min u ⊂ I and since we already know that I ⊂ arg min u, we conclude that I = arg min u.

The statements involving J ∈ F -follow by duality.

We next characterize the existence of a nontrivial fixed point for any payment-free Shapley operator in terms of the Galois connection (Φ, Φ ). We shall say that I, J ⊂ [n] are conjugate subsets with respect to the Galois connection (Φ, Φ ) if

           I ∈ F + \ ∅ , J ∈ F -\ ∅ , J = Φ(I) , I = Φ (J) .
Theorem 5.7. A payment-free Shapley operator F : R n → R n has a nontrivial fixed point if, and only if, there exists a pair of conjugate subsets with respect to the Galois connection (Φ, Φ ).

Proof. Assume first that F has a nontrivial fixed point u. Let I := arg min u and J := arg max u. We know from Lemma 5.5 that I ∈ F + and J ∈ F -. Since Φ • Φ • Φ = Φ, the subsets Φ • Φ(I) and Φ(I) are conjugate if, and only if, they are nonempty. This is the case since I ⊂ Φ • Φ(I) (by definition of Galois connections), and J ⊂ Φ(I) (by Lemma 5.5).

Assume now that (I, J) is a pair of conjugate subsets. Then, Φ(I) = J is nonempty and I is closed with respect to the Galois connection (Φ, Φ ), since Φ • Φ(I) = Φ (J) = I. Thus, it readily follows from Lemma 5.6 that u := F ω (e I c ) is fixed point of F verifying arg min u = I and arg max u = J.

We say that a subset of any set X is proper if it differs from the empty set and from X. Using the properties of the Galois connection, we get alternative (milder) conditions characterizing the existence of a nontrivial fixed point.

Corollary 5.8. A payment-free Shapley operator F : R n → R n has a nontrivial fixed point if, and only if, one of the following equivalent assertions holds:

(i) there exist nonempty disjoint subsets I, J ⊂ [n] such that I ∈ F + and J ∈ F -; (ii) there exists a nonempty subset I ∈ F + such that Φ(I) = ∅;

(iii) there exists a nonempty subset J ∈ F -such that Φ (J) = ∅;

(iv) there is a proper subset that is closed with respect to the Galois connection (Φ, Φ ) or (Φ , Φ); (v) there exists a pair of conjugate subsets with respect to the Galois connection (Φ, Φ ).

Proof. Since the equivalence between the existence of a nontrivial fixed point and condition (v) has been stated in Theorem 5.7, we only need to show the equivalence of Points (i) to (v).

(i) ⇒ (ii), (iii): If (i) holds, then by definition of the Galois connection, we have J ⊂ Φ(I) and I ⊂ Φ (J). Thus Φ(I) = ∅ and Φ (J) = ∅, which shows both (ii) and (iii).

(ii) ⇒ (iv): Let I be a subset as in (ii), that is, I ∈ F + is nonempty and Φ(I) = ∅. We cannot have Φ(I) = [n], otherwise, this would implies that I ⊂ Φ • Φ(I) = ∅. Hence, Φ(I) is proper. Moreover, we know that Φ(I) is closed with respect to the Galois connection (Φ , Φ), which shows (one case of) (iv).

(iii) ⇒ (iv): Similarly if J ∈ F -is nonempty and if Φ (J) = ∅, then Φ (J) is proper and closed with respect to the Galois connection (Φ, Φ ), which shows (iv).

(iv) ⇔ (v): This follows readily from the definition of conjugate subsets. (v) ⇒ (i): This also readily follows from the definition of conjugate subsets and of the Galois connection (Φ, Φ ).

Hypergraph characterization of the Galois connection

In this section, we introduce directed hypergraphs, which shall be useful to characterize the Galois connection (Φ, Φ ). In particular, we shall see that finding Φ(I) (resp., Φ (J)) for any given I ∈ F + (resp., J ∈ F -), is equivalent to solving a reachability problem in a directed hypergraph. We refer the reader to [START_REF] Gallo | Directed hypergraphs and applications[END_REF][START_REF] Allamigeon | On the complexity of strongly connected components in directed hypergraphs[END_REF] for background on reachability problems in hypergraphs.

Preliminaries on hypergraphs

A directed hypergraph is a pair (N, E) where N is a set of nodes and E is a set of (directed) hyperarcs.

A hyperarc e is an ordered pair (t(e), h(e)) of disjoint nonempty subsets of nodes; t(e) is the tail of e and h(e) is its head. For brevity, we shall write t and h instead of t(e) and h(e), respectively, if no confusion can be made about the edge e. When t and h are both of cardinality one, the hyperarc is said to be an arc, and when every hyperarc is an arc, the directed hypergraph becomes a directed graph.

In the following, the term hypergraph will always refer to a directed hypergraph Let G = (N, E) be a hypergraph. A hyperpath of length m from a set of nodes I ⊂ N to a node j ∈ N is a sequence of m hyperarcs (t 1 , h 1 ), . . . , (t m , h m ), such that t i ⊂ ∪ i-1 k=0 h k for all i = 1, . . . , m + 1, with the convention h 0 = I and t m+1 = {j}. Then, we say that a node j ∈ N is reachable from a set I ⊂ N if there exists a hyperpath from I to j. Alternatively, the relation of reachability can be defined in a recursive way: a node j is reachable from the set I if either j ∈ I or there exists a hyperarc (t, h) such that j ∈ h and every node of t is reachable from the set I. A set J is said to be reachable from a set I if every node of J is reachable from I. We denote by reach(I, G) the set of reachable nodes from I in G. Note that, by definition, we always have I ⊂ reach(I, G).

A subset I ⊂ N is invariant in the hypergraph G if it contains every node that is reachable from itself, that is reach(I, G) = I. One readily checks that the set of nodes in N that are reachable from I is the smallest invariant set in the hypergraph G containing I. The following example illustrate the notion of reachability.

Example 5.9. Figure 5.1 shows a directed hypergraph G with 9 nodes, 4 hyperarcs and 2 arcs. Check that node 8 is reachable from subset {1, 2} through the hyperpath ({1, 2}, {3}), ({2}, {4}), ({3, 4}, {6}), ({1}, {5}), ({5, 6}, {8}) .

But node 9 is not reachable from {1, 2} since the latter subset does not have access to node 7. Also, we have here reach({1, 2}, G) = {1, 2, 3, 4, 5, 6, 8} .

Hypergraphs associated with payment-free Shapley operators

In order to characterize the Galois connection (Φ, Φ ) associated with a payment-free Shapley operator F on R n , we introduce a pair of hypergraphs (G + , G -) constructed as follows:

-the set of nodes for both G + and G -is [n]; -the hyperarcs of G + are the pairs (J, {i}) such that F i (e J ) > 0 and i / ∈ J; -the hyperarcs of G -are the pairs (J, {i}) such that F i (e J c ) < 1 and i / ∈ J. When F arises from a stochastic game, with a specific minimax representation, it is possible to give an interpretation of these hypergraphs in game-theoretic terms, providing a more intuitive understanding of the latter. These aspects shall be treated in Section 5.3. In the present section, we only exploit the properties (M), (AH) and (PH), characterizing payment-free Shapley operators.

The next proposition sheds light on the link between the hypergraphs G ± and the families of subsets F ± . It is a direct consequence of the alternative characterizations (5.4) and (5.5) of the families F ± , as well as the following observation. By monotonicity of F , if (J, {i}) is a hyperarc of G ± , then (J , {i}) is also a hyperarc of G ± for any subset J ⊃ J which does not contain i. Hence, a subset J ⊂ [n] is invariant in G ± if, and only if, there is no hyperarc in G ± from J to any node i / ∈ J.

Proposition 5.10. Let F : R n → R n be a payment-free Shapley operator, to which is associated the pair of hypergraphs (G + , G -). Then, a subset I ⊂ [n] belongs to F + (resp., F -) if, and only, if its complement in [n] is invariant in G + (resp., G -):

I ∈ F + ⇐⇒ reach(I c , G + ) = I c , I ∈ F -⇐⇒ reach(I c , G -) = I c .
We now make the link with the Galois connection (Φ, Φ ).

Corollary 5.11. Let F : R n → R n be a payment-free Shapley operator. Let (Φ, Φ ) and (G + , G -) be the Galois connection and the pair of hypergraphs, respectively, defined as above. Then, for all I ∈ F + and J ∈ F -, we have

Φ(I) = [n] \ reach(I, G -) , Φ (J) = [n] \ reach(J, G + ) .
Proof. It follows readily from the definition of Φ that [n] \ Φ(I) is the smallest subset I containing I such that I c is in F -. By Proposition 5.10, the latter condition holds if, and only if, I is invariant in G -. Hence, by definition of reachability, Φ(I) is the complement in [n] of the set of nodes that are reachable from I in G -. For Φ the argument is dual.

We shall say that I, J ⊂ [n] are conjugate subsets with respect to the hypergraphs G ± , if

     I, J = ∅ , I = [n] \ reach(J, G + ) , J = [n] \ reach(I, G -) .
It readily follows from Corollary 5.11 that any sets I, J ⊂ [n] are conjugate with respect to the Galois connection (Φ, Φ ) if, and only if, they are conjugate with respect to the hypergraphs (G + , G -). Hence, a reformulation of Corollary 5.8 in terms of the hypergraphs (G + , G -) leads to the following.

Theorem 5.12. Let F : R n → R n be a payment-free Shapley operator, to which is associated the pair of hypergraphs (G + , G -). The following assertions are equivalent: (i) F has a nontrivial fixed point;

(ii) there exist nonempty disjoint subsets I, J ⊂ [n] such that I c and J c are invariant in G + and G -, respectively. (iii) there exists a pair of conjugate subsets I, J ⊂ [n] with respect to the hypergraphs (G + , G -).

Ergodicity of stochastic games with bounded payments

As an immediate corollary of Theorem 5.12, we get a combinatorial characterization of the ergodicity of stochastic games with bounded payment function, relying on the analytical characterization of Theorem 4.6.

Corollary 5.13. Let T : R n → R n be a Shapley operator satisfying Assumption 4.A, and let (G + , G -) be the pair of hypergraphs associated with the recession operator of T . The following assertions are equivalent:

(i) T is ergodic, i.e., for all vectors g ∈ R n , the ergodic equation (2.5) is solvable for g + T ;

(ii) there do not exist nonempty disjoint subsets I, J ⊂ [n] such that I c and J c are invariant in G + and G -, respectively; (iii) there does not exist conjugate subsets I, J ⊂ [n] with respect to the hypergraphs (G + , G -).

Remark 5.14. It is instructive to specialize the latter result to the case in which T arises as the dynamic programming operator of a finite Markov chain with rewards. Then, we can write T (x) = g + P x, where g ∈ R n is the payment vector and P is a n × n stochastic matrix. We also have T (x) = P x.

In that case, the two hypergraphs G + and G -are identical. More precisely, there exists a hyperarc (J, {i}) in G ± if, and only if, there is some state j ∈ J such that P ij > 0. Consequently, the reachability relation in G ± is the same as in the directed graph G with same set of nodes [n] and an arc from j to i if P ij > 0. Note that here, on contrary to hypergraphs, loops are possible. The latter directed graph G is the transpose of the directed graph G associated with the transition matrix P , i.e., the arcs of G are in the opposite direction. In particular, I c is invariant in G (hence in G ± ) if, and only if, there are no arcs nor paths from I to I c in G. Similarly, (I, J) is a pair of conjugate subsets with respect to (G + , G -) if, and only if, I is the greatest set of nodes with no paths in G to a node of J, and vice versa.

So, Assertion (ii) of Corollary 5.13 implies the existence of a unique final class in G. Indeed, if this is not the case, that is, if there exist two distinct final classes I, J in G, then there are no arcs nor paths from I to I c in G, and the same is true for J, a contradiction. Conversely, if G has a unique final class, then Point (ii) of Corollary 5.13 is necessarily true. Also by contradiction, if there exist two nonempty disjoint subsets I, J such that there are no arcs from I to I c , nor from J to J c in G, then there exists a final class of G included in I and also a final class of G included in J, a contradiction since I and J are disjoint. Hence in the present case, Assertion (ii) of Corollary 5.13 is equivalent to the condition that the directed graph associated with P has a unique final class, that is, Assertion (iv) in Theorem 4.1.

Game-theoretic interpretation

In this section, we consider a stochastic game Γ = ([n], A, B, K A , K B , r, p), which satisfies the following standing assumption.

Assumption 5.B.

(i) The action spaces A and B are compact sets endowed with their respective Borel σalgebra, and for all states i ∈ [n], the sets of admissible actions A i and B i are nonempty closed sets. (ii) The payment function r and the transition function p are continuous.

Note that with the latter assumption, the value in finite horizon and the discounted value exist. In particular, the Shapley operator, T : R n → R n , of Γ is well defined and its ith component is given by

T i (x) = min µ∈∆(A i ) max ν∈∆(B i ) r(i, µ, ν) + j∈[n] x j p(j | i, µ, ν) = max ν∈∆(B i ) min µ∈∆(A i ) r(i, µ, ν) + j∈[n] x j p(j | i, µ, ν) , x ∈ R n .
Let us make some comments. First, observe that inf and sup operators in (2.1) have been replaced by min and max operators, respectively. This is justified by the fact that the set of probability measures on A i and B i , respectively, are compact with respect to the weak-* topology, and the multilinear extensions r(i, •, •) and p(j | i, •, •) are continuous with respect to this topology. Another implication of Assumption 5.B is that the payment function is bounded. As a consequence, the recession operator of T exists and its ith component is given by

T i (x) = min µ∈∆(A i ) max ν∈∆(B i ) j∈[n] x j p(j | i, µ, ν) , x ∈ R n ,
where the minimum and the maximum commutes. Moreover, Assumption 4.A is satisfied. Hence, according to Theorem 4.6, the ergodicity of Γ can be characterized by the absence of nontrivial fixed points for the payment-free Shapley operator T .

Dominions and ergodicity

We shall say that the families F + and F -of subsets of [n] and the hypergraphs G + and G -are associated with the game Γ when the latter objects are constructed from the recession operator T . We next give a game-theoretic interpretation of the results obtained in the previous sections. We shall call dominion of player MIN (resp., player MAX) in the game Γ any nonempty subset of states I ⊂ [n] subject to the control of MIN (resp., MAX), in the sense that he has a pure stationary strategy such that a sequence of states starting in any state i ∈ I stays in I almost surely, whatever the strategy of the other player is.

Proposition 5.15. Let Γ be a stochastic game with finite state space satisfying Assumption 5.B, and let (F + , F -) be the families of subsets of states associated with Γ. Then, I ∈ F + \ ∅ if, and only if, I is a dominion of MIN in Γ. Likewise, J ∈ F -\ ∅ if, and only if, J is a dominion of MAX in Γ.

Before giving the proof, let us recall the following definition: for any topological space X, endowed with its Borel σ-algebra, denoted by B, the support of a measure µ on (X, B) is the unique closed set, denoted by supp µ, satisfying:

(i) µ(X \ supp µ) = 0;

(ii) If U ⊂ X is open and U ∩ supp µ = ∅, then µ(U ∩ supp µ) > 0. When X = [n],
we shall identify a probability measure µ ∈ ∆(X) with the stochastic vector p ∈ R n whose ith entry is equal to p i = µ({i}). The support of p is then given by supp p := {i ∈

[n] | p i > 0}.
Proof. Let T : R n → R n be the Shapley operator of Γ. Let I ∈ F + \ ∅. Then, according to (5.4), we have T i (e I c ) = 0 for all i ∈ I. Let µ ∈ ∆(A i ) be a mixed action of player MIN in state i attaining the minimum in the minimax formula of T (e I c ). For all mixed actions ν ∈ ∆(B i ) of player MAX, we have Let σ be a pure stationary strategy of player MIN, identified with a map from [n] to A, such that σ(i) = a i for all state i ∈ I. Suppose MIN chooses this strategy. At each step k of Γ, if the current state i k is in I, and if player MAX chooses action b, then the probability that the state i k+1 at the following stage is in I c is equal to p(I c | i k , a i k , b) = 0, whatever MAX chooses. Hence, if the initial state is in I, then for any strategy of MAX, the probability that the sequence of states (i k ) k 1 leaves I is 0. This shows the "only if" part of the first statement.

Conversely, suppose that there exists a pure stationary strategy σ of player MIN, mapping i ∈ [n] to a i = σ(i) ∈ A i , such that for any initial state i 1 in I and any strategy of player MAX, the state of Γ stays in I almost surely. In particular, for any state i ∈ I and any action b ∈ B i , if the initial state is i 1 = i, if MIN chooses strategy σ and if MAX chooses action b at first stage, then the probability that i 2 is outside I is equal to 0. This probability coincides with p(I c | i, a i , b). Hence, for all states i ∈ [n], we have

T i (e I c ) max ν∈∆(B i ) j / ∈I p(j | i, a i , ν) = 0 .
This shows that T i (e I c ) = 0 for all i ∈ I, that is I ∈ F + . The second assertion follows by duality.

We deduce from the above proposition, combined with Corollary 5.8 and Theorem 4.6, a necessary and sufficient condition for ergodicity of stochastic games in terms of dominions Corollary 5.16. A stochastic game Γ with finite state space satisfying Assumption 5.B is ergodic if, and only if, there does not exist a pair of disjoint dominions of players MIN and MAX, respectively, in Γ.

Remark 5.17. The hypergraphs G + and G -can also be interpreted in terms of strategies. Indeed, in G -, there is a hyperarc from a subset J ∈ [n] to a state i if, and only if,

min µ∈∆(A i ) max ν∈∆(B i ) p(J c | i, µ, ν) < 1 ,
which amounts to saying that in state i, player MIN has a mixed action µ i such that, for any action b chosen by player MAX, the probability for the state to be in J at the next stage, i.e., p(J | i, µ i , b), is bounded below by a positive constant. Note that, contrary to the family F -, we need to consider mixed actions of MIN (and not only pure actions).

The same interpretation holds for G + and player MAX: there is a hyperarc from J ⊂ [n] to i ∈ [n] in G + if, and only if, in state i, MAX has a mixed action such that, for any action chosen by MIN, the state at the next stage is in J with positive probability.

Ergodicity is a structural property

We call upper and lower Boolean abstractions of T , the operators defined on the set of Boolean vectors {0, 1} n , the ith component of which are given respectively by

T + i (x) := min a∈A i max b∈B i max{x j | p(j | i, a, b) > 0} , T - i (x) := max b∈B i min a∈A i min{x j | p(j | i, a, b) > 0} .
Let us make some observations. First, these Boolean operators can be extended to R n . Then, we have T -T T + . Second, the operators T + and T -are only determined by the supports of the transition probabilities, that is, by the set of all (i, a, b, j) such that p(j | i, a, b) > 0.

The latter Boolean operators are helpful to determine the set of dominions, as well as to construct the hypergraphs G + and G -. These facts are a consequence of the following lemmas.

Lemma 5.18. Let Γ be a stochastic game with state space [n] satisfying Assumption 5.B, and with Shapley operator T : R n → R n . Then, a nonempty subset I ⊂ [n] is a dominion of player MIN in Γ (i.e., T (e I c ) e I c ) if, and only if, T + (e I c ) e I c . Likewise, a nonempty subset J ⊂ [n] is a dominion of player MAX in Γ (i.e., e J T (e J )) if, and only if, e J T -(e J ).

Proof. Recall that a nonempty subset I is a dominion of MIN if, and only if, T (e I c ) e I c holds (Proposition 5.15). Furthermore, since T T + , we only need to prove the implication

T (e I c ) e I c =⇒ T + (e I c ) e I c .
So, let us assume that T (e I c ) e I c , that is, I is a dominion of MIN. Then, for all i ∈ I, there exists an action a i ∈ A i such that the probability p(

I c | i, a i , b) is equal to 0 for all b ∈ B i . Let x = e I c . For all i ∈ I and b ∈ B i , we have p(I c | i, a i , b) = j x j p(j | i, a i , b) = 0.
Since all the terms in the latter sum are nonnegative, we must have x j = 0 as soon as p(j | i, a i , b) > 0, which also writes max{x j | p(j | i, a i , b) > 0} = 0. We deduce that T + i (x) = 0 for all i ∈ I, that is, T + (e I c ) e I c . This shows the first equivalence, and the second follows by duality.

Note that, according to the equivalences (5.4) and (5.5) and their Boolean counterparts, the equivalences in the previous lemma can be restated in the following way:

T i (e J ) > 0 ⇐⇒ T + i (e J ) = 1 , T i (e J c ) < 1 ⇐⇒ T - i (e J c ) = 0 , for all i ∈ [n] and J ⊂ [n].
Hence, the following characterization.

Lemma 5.19. Let Γ be a stochastic game with state space [n] satisfying Assumption 5.B and with Shapley operator T : R n → R n . Let G + and G -be the hypergraphs associated with Γ. Then (J, {i}) is a hyperarc of G + if, and only if, T + i (e J ) = 1 and i / ∈ J. Likewise, (J, {i}) is a hyperarc of G -if, and only if, T - i (e J c ) = 0 and i / ∈ J.

We deduce from the previous results that the set of dominions and the hypergraphs G + and G -are structural, in the sense that they only depend on the supports of the transition probabilities. Together with Corollary 5.16, we get in particular the following.

Corollary 5.20. The ergodicity of a stochastic game Γ with finite state space satisfying Assumption 5.B only depends on the support of the transition probabilities.

We conclude this section with the following theorem, providing a way to compute the images of subsets of states by the Galois connection (Φ, Φ ). Since e L is a fixed point of T -, L belongs to F + . Furthermore, it satisfies e L e I c , since I ∩ L = ∅. Then L ⊂ Φ(I).

Let K ∈ F + be a subset such that I ∩ K = ∅, that is, e K e I c . By induction, we get that

( T -) k (e K ) ( T -) k (e I c
) for every integer k. We also have e K T -(e K ), hence ( T -) ω (e K ) exists and ( T -) ω (e K ) e K . This leads to e K ( T -) ω (e K ) ( T -) ω (e I c ) = e L , which implies that K ⊂ L. This holds for all K ∈ F + such that I ∩ K = ∅, hence, by definition of Φ, we have Φ(I) ⊂ L.

Algorithmic and complexity issues

In this section, we consider a finite stochastic game Γ = ([n], A, B, K A , K B , r, p) with perfect information. Assumption 5.B (hence, Assumption 4.A) is trivially satisfied, since the action spaces are finite. Moreover, the Shapley operator T : R n → R n of Γ is a polyhedral map, the ith component of which writes

T i (x) = min a∈A i max b∈B i,a r(i, a, b) + j∈[n] x j p(j | i, a, b) , x ∈ R n .
As for its recession operator, which we denote here by F for simplicity, its ith component is given by

F i (x) := T i (x) = min a∈A i max b∈B i,a j∈[n] x j p(j | i, a, b) , x ∈ R n .
(5.8)

Note that all the results of Section 5.3 readily apply to the present setting, with minor adaptations coming from the fact that the min and max operators cannot be interchanged, see [START_REF] Akian | Ergodicity conditions for zero-sum games[END_REF] where the perfect-information case has been specifically treated. In particular, the ith components of the Boolean abstractions of F are given by

F + i (x) = T + i (x) := min a∈A i max b∈B i,a max{x j | p(j | i, a, b) > 0} , F - i (x) = T - i (x) := min a∈A i max b∈B i,a min{x j | p(j | i, a, b) > 0} .

Syntactic hypergraphs

In this subsection, we introduce a pair of hypergraphs (G + s , G - s ) representing the Boolean operators F + and F -. We shall see that the links between the Galois connection (Φ, Φ ) and the hypergraphs (G + , G -) established in Section 5.2 still hold with this new pair of hypergraphs. Hence, finding Φ(I) (resp., Φ (J)) for any subset I ∈ F + (resp., J ∈ F -) is equivalent to solving a reachability problem in (a modified version of) G - s (resp., G + s ). We first construct the hypergraph G + s , a representation of which is shown in Figure 5.2. Let us introduce to that purpose a copy of [n], which is a set disjoint from [n] and given by a bijection c from [n] to this copy. We also need to assume c([n]) disjoint from the two constraints sets K A and K B .

The set of nodes of

G + s is [n] ∪ K A ∪ c([n]
). The hyperarcs of G + s are of the form: -({i} × A i , {c(i)}) for each state i ∈ [n] ; -({j}, {(i, a)}) for each states i, j ∈ [n] and each action a ∈ A i such that there exists some action b ∈ B i,a with p(j | i, a, b) > 0. As shown on Figure 5.2, this hypergraph is structured in two layers; the first layer consists of the arcs ({j}, {(i, a)}) whereas the second layer consists of the hyperarcs ({i} × A i , {c(i)}). Furthermore, the size of

G + s satisfies size(G + s ) = O(n 2 |A|). j (i, a) c(i) [n] {i} × A i c([n]) Figure 5.2: Hypergraph G + s associated with F +
The hypergraph G + s encodes the Boolean operator F + in the following sense. Let x ∈ {0, 1} n be a Boolean vector. Denote

x i,a := max b∈B i,a max{x j | p(j | i, a, b) > 0} , so that have F + i (x) = min a∈A i x i,a .
If x = e J for any subset J ⊂ [n], then x i,a = 1 if, and only if, there exists some action b ∈ B i,a and some state j ∈ J such that p(j | i, a, b) > 0. This is also equivalent to the node (i, a) being reachable from J in G + s . Then, F + i (x) = 1 if, and only if, x i,a = 1 for every action a ∈ A i , which is equivalent to all the nodes in the tail of the hyperarc ({i} × A i , {c(i)}) being reachable from J in G + s . According to the recursive definition of reachability, this is equivalent to c(i) being reachable from J in G + s . Hence, the following result.

Proposition 5.22. Let F : R n → R n be the payment-free Shapley operator associated with a perfectinformation finite stochastic game. Then, the node c(i) ∈ c([n]) is reachable from the subset J ⊂ [n] in G + s if, and only if, F + i (e J ) = 1.

We now construct the second hypergraph, a representation of which is shown in Figure 5.3. The node set of

G - s is [n] ∪ K B ∪ c([n]
), and its hyperarcs are: -({(i, a)} × B i , {c(i)}) for each state i ∈ [n] and action a ∈ A i ; -({j}, {(i, a, b)}) for each states i, j ∈ [n] and each actions a ∈ A i and b ∈ B i,a such that p(j | i, a, b) > 0. Again, the hypergraph G - s consists of two layers (see Figure 5.3). Furthermore, the size of G -

s satisfies size(G - s ) = O(n 2 |A||B|). Like G +
s , the hypergraph G - s encodes the Boolean operator F -, as shown by the following result.

Proposition 5.23. Let F : R n → R n be the payment-free Shapley operator associated with a perfectinformation finite stochastic game. Then, the node c(i

) ∈ c([n]) is reachable from the subset J ⊂ [n] in G - s if, and only if, F - i (e J c ) = 0.
Note that the absence of symmetry between G + s and G - s reflects the lack of symmetry between F + and F -. Denote by G

(i, a , b) (i, a , b ) c(i) [n] {(i, a)} × B i,a {(i, a )} × B i,a c([n])
reach(I, G + ) = reach(I, G + s ) ∩ [n] , reach(I, G -) = reach(I, G - s ) ∩ [n] .
Thus, all the results of the previous sections, involving the hypergraphs (G + , G -) and the families (F + , F -) can be stated with the syntactic hypergraphs (G + s , G s ). In particular we have the following characterization of dominions, which uses a stronger notion of invariance: in a directed hypergraph G = (N, E), if N ⊂ N , we say that a subset I of N is invariant in the hypergraph G relatively to N if it contains all the nodes of N that are reachable from itself, that is, reach(I, G) ∩ N = I. Furthermore, the images of dominions by the Galois connection (Φ, Φ ) can be effectively computed.

Corollary 5.26. Let Γ be a perfect-information finite stochastic game with state space [n]. Let (F + , F -) and (G + s , G s ) be the families of subsets of states and the syntactic hypergraphs, respectively, associated with Γ. Then, for all I ∈ F + (resp., J ∈ F -), Φ(I) (resp., Φ (J)) is given by the complement in [n] of all the nodes of [n] that are reachable from I (resp., J) in G s (resp., G + s ):

Φ(I) = [n] \ reach(I, G - s ) resp., Φ (J) = [n] \ reach(J, G + s ) .

Checking ergodicity

According to Theorem 4.6, the negation of the following problem is equivalent to the next one.

Problem (NontrivialFP). Does a given payment-free Shapley operator F (5.8) associated with a perfect-information finite stochastic game have a nontrivial fixed point, that is, does there exist a vector u ∈ R n \ Re such that u = F (u)?

Problem (Ergodicity). Is a given perfect-information finite stochastic game Γ ergodic?

It is known that in a directed hypergraph G, the set of reachable nodes from a set I can be computed in O(size(G)) time, see [START_REF] Gallo | Directed hypergraphs and applications[END_REF]. Hence, from Corollary 5.24 and Corollary 5.26 we get the following.

Proposition 5.27. Let Γ be a perfect-information finite stochastic game with state space [n], and let (F + , F -) be the families of subsets of states associated with Γ. For any subsets I, J ⊂ [n], checking that I ∈ F -and J ∈ F + can be done in O(n 2 |A|) and O(n 2 |A||B|) time, respectively. Moreover, Φ(I) and Φ (J) can be evaluated in O(n 2 |A||B|) and O(n 2 |A|) time, respectively.

Using Proposition 5.27 and Corollary 5.8, we obtain the following result, which shows that checking the ergodicity of a game is fixed-parameter tractable: if the dimension is fixed, we can solve it in a time which is polynomial in the input size. Thus, for instances of moderate dimension but with large action spaces, ergodicity condition can be checked efficiently.

Theorem 5.28. The ergodicity of a perfect-information finite stochastic game with state space [n], that is the property " T has only trivial fixed points", can be checked in O(2 n n 2 |A||B|) time.

Problem NontrivialFP has already been addressed in the deterministic case with finite action spaces by Yang and Zhao [START_REF] Yang | The balance problem of min-max systems is co-NP hard[END_REF]. Suppose indeed that in the expression (5.8), the support of each transition probability is concentrated on just one state and consider the restriction of such an operator to the set of Boolean vectors {0, 1} n . Then, we obtain a monotone Boolean operator.

Recall that a Boolean operator, defined on the set of Boolean vectors {0, 1} n , is expressed using the logical operators AND, OR and NOT. Monotone Boolean operators are operators the expression of which involves only AND and OR operators. The latter can be interpreted as min and max operators, respectively. So, deterministic payment-free Shapley operators are equivalent to monotone Boolean operators and Problem NontrivialFP can be expressed in a simpler form. Problem (MonBoolFP). Does a given monotone Boolean operator have a nontrivial fixed point, that is, different from the zero vector and the unit vector? Proof. Suppose on contrary that there is a nontrivial fixed point u such that I ⊂ arg min u. Let I := arg min u and J := arg max u. We know from Lemma 5.5 that I ∈ F + , J ∈ F -and that J ⊂ Φ(I ). Since I ⊂ I , we have Φ(I ) ⊂ Φ(I). Hence J ⊂ Φ(I), and since J = ∅, we get a contradiction.

If I = ∅, the answer to Problem MinFP is trivially negative, and if I = [n] it is trivially positive. Assume now that I is a proper subset of [n]. The above results show that a necessary condition to have a positive answer to problem MinFP is that I ∈ F + and Φ(I) = ∅. Moreover, by Lemma 5.6, a sufficient condition to have a positive answer to problem MinFP is that I is closed with respect to the Galois connection (Φ, Φ ). It remains to examine the case in which I ∈ F + is a proper subset, with Φ(I) = ∅, and distinct from its closure I := Φ • Φ(I) with respect to the Galois connection (Φ, Φ ). Note that we must have, in particular, I = [n], otherwise we would have Φ(I) = Φ(I) = ∅.

In the remainder of the subsection, F is the payment-free Shapley operator (5.8) associated with a perfect-information finite stochastic game Γ (hence, satisfying Assumption 5.B). We define a reduced operator F * : R I → R I as follows. According to the game-theoretic interpretation (Proposition 5.15), we know that MIN can force the state of the game Γ to stay in I from any initial position in I. Hence, we consider the actions of MIN that achieve this goal: for every i ∈ I, let

A * i := a ∈ A i | ∀b ∈ B i,a , ∀j / ∈ I, p(j | i, a, b) = 0 .
These sets are nonempty since I ∈ F + . Alternatively, A * i can be defined as the set of actions a ∈ A i which attain the minimum in the minimax formula (5.8) of F i e [n]\I , that is,

A * i = a ∈ A i max b∈B i,a j / ∈I p(j | i, a, b) = F i e [n]\I = 0 . Given x ∈ R n and K ⊂ [n]
, let us denote by x K the restriction of x to R K . Then, we define the reduced map F * from R I to itself, whose ith component (i ∈ I) is given by

F * i (x) := min a∈A * i max b∈B i,a j∈I x j p(j | i, a, b) , x ∈ R I .
The latter map is a payment-free Shapley operator on R I . Indeed, from the definition of A * i , any transition probability p(• | i, a, b) is supported by I as soon as a ∈ A * i with i ∈ I. In particular, we have, for all i ∈ I,

F * i (x I ) = min a∈A * i max b∈B i,a j∈[n]
x j p(j | i, a, b) , ∀x ∈ R n .

(5.9)

Theorem 5.33. Let F : R n → R n be the payment-free Shapley operator associated with a perfectinformation finite stochastic game. Let I ∈ F + be proper, such that Φ(I) = ∅ and I = I. Then F has a fixed point the arg min of which is I if, and only if, the same holds for the reduced operator F * .

Proof. We first show the "only if" part of the theorem. Let u be a fixed point of F such that I = arg min u. Recall that I = [n] by hypothesis. So we may suppose that max i∈[n] u i = 1 and min i∈[u] u i = 0. It follows from (5.9) that u i = F i (u) F * i (u I ) for all i ∈ I, so that (F * ) ω (u I ) exists. Let us denote this limit point by v. It is a fixed point of F * and it satisfies u I v. As a consequence, v i > 0 for every i ∈ I \ I.

Furthermore, Lemma 5.5 implies that I ∈ F + , meaning that F (e Hence v i = 0 for every i ∈ I, which shows that arg min v = I.

We now prove the "if" part of the theorem. Assume that F * has a fixed point v such that arg min v = I. We may suppose that max i∈

[n] v i = 1 and min i∈[n] v i = 0. Let w := F ω (e [n]\I ).
We know from Lemma 5.6 that w is a fixed point of F verifying arg min w = I. Thus, it satisfies w I = 0 and w j > 0 for every j ∈ [n] \ I, hence there exists some constant α > 0 such that w αe [n]\I .

We next use the notions of semidifferentiability and semiderivative, referring the reader to [START_REF] Rockafellar | Variational analysis, volume 317 of Grundlehren der Mathematischen Wissenschaften[END_REF][START_REF] Akian | Uniqueness of the fixed point of nonexpansive semidifferentiable maps[END_REF] for the definition of these notions and for their basic properties. Since the action spaces are finite, F is piecewise affine and so, it is semidifferentiable at point w. Furthermore, denoting by F w its semiderivative at w, there is a neighborhood V of 0 such that

F (w + x) = F (w) + F w (x) , ∀x ∈ V .
(5.10)

We next give a formula for F w . For every i ∈ [n], let

A i (w) := a ∈ A i max b∈B i,a j∈[n] w j p(j | i, a, b) = F i (w)
and for all a ∈ A i (w), let

B i,a (w) := b ∈ B i,a j∈[n] w j p(j | i, a, b) = F i (w) .
Then, the ith component of F w is given by

F w (x) i = min a∈A i (w) max b∈B i,a (w) j∈[n] x j p(j | i, a, b) , x ∈ R n .
Observe that for each i ∈ I, we have A i (w) = A * i and B i,a (w) = B i for all a ∈ A i (w). This is because, for all i ∈ I we have F i (w) = w i = 0 and w j > 0 for all j / ∈ I, so that a ∈ A i (w) if, and only if, max b∈B i,a p([n] \ I | i, a, b) = 0 and b ∈ B i,a (w) if, and only if, p([n] \ I | i, a, b) = 0. Then, using (5.9), we obtain that

[F w (x)] I = F * (x I ) , ∀x ∈ R n .
(5.11)

We now introduce the vector z ∈ [0, 1] n given by z I = v and z [n]\I = 0. By the above property (5.11) of F w , we get that [F w (z)] I = F * (v) = v = z I . Moreover, since F w is a payment-free operator and z 0, we get that F w (z) 0, so F w (z) z. Hence, the limit point z := (F w ) ω (z) exists and is a fixed point of F w , belonging to [0, 1] n . Again, by the above property (5.11) of F w , we get that [(F w ) k (z)] I = F * ([(F w ) k-1 (z)] I ) for all integers k 1, so that by induction [(F w ) k (z)] I = v, and finally zi = v i for all i ∈ I.

Choose ε > 0 small enough so that εz is in V and let u = w + ε z. Then, from (5.10), we get that F (u) = F (w) + εF w (z) = w + εz = u, where we used the fact that F w is positively homogeneous. Then u is a fixed point of F . Moreover, by construction u = w + ε z w and u εz, and since arg min w = I and arg min z ∩ I = I, we deduce that u I = 0 and u j > 0 for every j ∈ [n] \ I, that is, arg min u = I.

The previous result, together with the observations made before it, lead to Algorithm 1 below, which solves Problem MinFP, as detailed in Theorem 5.34. There, if Γ denotes a finite stochastic game, we write Γ * the reduced game with, in particular, state space I and set of admissible actions A * i for player MIN when in state i ∈ I. Also, if F is a payment-free Shapley operator, we write (Φ F , Φ F ) the Galois connection associated with that operator. Proof. The fact that Algorithm 1 provides the right answer to Problem MinFP is a direct consequence of Lemma 5.31, Lemma 5.32, Lemma 5.6 and Theorem 5.33.

We next show that it stops after at most n iterations of the loop. Suppose that during the execution of a loop, the first two conditions (which are stopping criteria) are not satisfied. Then the closure of I with respect to the Galois connection (Φ F , Φ F ) associated with F is a proper subset of states. Hence, the cardinality of the state space for the reduced operator F * is strictly less than the one for F . Moreover, each operation in the loop requires at most O(n 2 |A||B|) time (see Proposition 5.27).

Fixed points with prescribed argmin and argmax

So far, we have only considered the problem with a single constraint on the fixed point, concerning the indices of the minimal entries. The dual problem, concerning the maximal entries of fixed points, is equivalent. We address now a mixed-condition problem.

Problem (MinMaxFP). Let I and J be nonempty disjoint subsets of [n]. Does a given paymentfree Shapley operator (5.8) associated with a perfect-information finite stochastic game have a fixed point u satisfying arg min u = I and arg max u = J?

The following theorem shows that Problem MinMaxFP can be split in one instance of Problem MinFP and one instance of its dual problem.

Theorem 5.35. Let F : R n → R n be the payment-free Shapley operator associated with a perfectinformation finite stochastic game. Let I ∈ F + and J ∈ F -be two nonempty disjoint subsets. Then F has a fixed point u satisfying arg min u = I and arg max u = J if, and only if, F has fixed points v and w satisfying arg min v = I and arg max w = J, respectively.

Proof. We only need to prove the "if" part of the theorem. Suppose that F has fixed points v and w such that arg min v = I and arg max w = J, respectively. Then, we may impose

min i∈[n] v i = 0 , max i∈[n] v i = min i∈[n] w i = 1/2 , max i∈[n] w i = 1 . Let L := {z ∈ R n | v ∨ e J z
w ∧ e I c }, where ∨ and ∧ stands for the supremum and the infimum operators, respectively. Put in words, L is the set of all elements in the hypercube [0, 1] n whose entries are 0 on I, 1 on J and between the ones of v and w elsewhere. In particular, the entries outside I or J of the vectors in L are in (0, 1).

The set L is a complete lattice. We next show that it is invariant by

F . Since J ∈ F -, we have v ∨ e J F (v) ∨ F (e J ) F (v ∨ e J ) ,
and since I ∈ F + , we have

w ∧ e I c F (w) ∧ F (e I c ) F (w ∧ e I c ) . Hence, v ∨ e J z w ∧ e I c implies v ∨ e J F (v ∨ e J ) F (z) F (w ∧ e I c ) w ∧ e I c ,
which shows that L is invariant by F . Thus, since F is order-preserving, Tarski's fixed-point theorem guarantees the existence of a fixed point of F in L.

Corollary 5.36. Problem MinMaxFP can be solved in O(n 3 |A||B|) time.

Proof. According to Theorem 5.35, Problem MinMaxFP can be solved by two instances of Problem MinFP, one with inputs F and I, one with inputs F (Remark 5.3) and J.

Summary of complexity results

The following table summarizes the results of this section.

Example

Checking ergodicity

Let us consider the zero-sum stochastic game with perfect information defined by the graph represented in Figure 5.4. There are four states represented by gray nodes. A token is initially placed in one of these nodes. At each stage, the token is moved along the edges of the graph until it reaches another state, according to the following rule: player MIN moves the token at circle nodes, player MAX at square ones and at diamond nodes, an edge is selected at random according to the probabilities indicated on the edges starting from the node. A payment only occurs for the edges starting from a MAX node (its value is given by the label attached to them).

1 1 / 2 1 -2 -1 2 2 2 1/2 -3 3 1/2 2 4 -2 1/2 1/2 -1 1/2 1/2 1/2
Figure 5.4: Graph on which is played a perfect-information stochastic game

The Shapley operator of this game is

T (x) =     (2 + x 1 ) ∧ (1 + 1 2 (x 1 + x 2 )) (-2 + 1 2 (x 1 + x 2 )) ∨ (-1 + x 1 ) ∧ (2 + 1 2 (x 1 + x 3 )) (-3 + 1 2 (x 1 + x 3 )) ∨ (-1 + 1 2 (x 2 + x 4 )) (-2 + x 4 ) ∨ (2 + 1 2 (x 3 + x 4 ))     , x ∈ R 4 .
It can be checked that the ergodic equation (2.5) is solvable, with ergodic constant λ = 1/3 and bias vector u = (4/3, 0, 2/3, 4) . Let us decide whether this game is ergodic. This can be easily done following the game-theoretic interpretation of Section 5.3. Indeed, observe on Figure 5.4 that the dominions of MIN are {1}, {1, 2} and {1, 2, 3, 4}, since, apart from the whole set of states, MIN can always make sure that the state remains in {1} or in {1, 2}. As for MAX, its dominions are {4} and {1, 2, 3, 4}. Thus, there exists at least one pair of disjoint dominions of MIN and MAX, e.g., {1} and {4}, and so we conclude from Corollary 5.16 that the game is not ergodic.

Alternatively, and in order to illustrate the Galois connection of Section 5.1, we can study the fixed points of the recession operator of T , denoted here by F and given by

F (x) =     x 1 ∧ 1 2 (x 1 + x 2 ) 1 2 (x 1 + x 2 ) ∨ x 1 ∧ 1 2 (x 1 + x 3 ) 1 2 (x 1 + x 3 ) ∨ 1 2 (x 2 + x 4 ) x 4 ∨ 1 2 (x 3 + x 4 )     , x ∈ R 4
(5.12)

From Proposition 5.15 we readily get that the families F + and F -associated with F are

F + = ∅, {1}, {1, 2}, {1, 2, 3, 4} and
F -= ∅, {4}, {1, 2, 3, 4} .
We can also obtain a description of the latter families by constructing the Boolean abstractions of F :

F + (x) =     x 1 x 1 ∨ (x 2 ∧ x 3 ) x 1 ∨ x 3 ∨ x 2 ∨ x 4 x 3 ∨ x 4     and F -(x) =     x 1 ∧ x 2 x 1 ∧ x 3 (x 1 ∧ x 3 ) ∨ (x 2 ∧ x 4 ) x 4     ,
and check that

F + (e {2,3,4} ) e {2,3,4} , F + (e {3,4} ) e {3,4} and
F -(e {4} ) e {4} .
Then, by definition of the Galois connection, we have

Φ({1}) = Φ({1, 2}) = {4} and Φ ({4}) = {1, 2} .
So, ({1, 2}, {4}) is a pair of conjugate subsets with respect to the Galois connection, which proves (Theorem 5.7 together with Theorem 4.6) that the game is not ergodic.

Finding a xed point with prescribed argmin

We now address the problem of finding fixed points of F with fixed arg min. Since e {2,3,4} and e {3,4} are the only nontrivial fixed points of F + , we know from Lemma 5.31 that {1} and {1, 2} are the only possible candidates for nontrivial arg min. The set {1, 2} is closed with respect to the Galois connection (Φ, Φ ). Thus, according to Lemma 5.6, F has a fixed point whose arg min is {1, 2}. Moreover, its arg max can only be {4}. Check that the vector (0, 0, 1/2, 1) is a fixed point with these properties. As for the set {1}, we cannot conclude directly, since the hypotheses of Lemma 5.32 or Lemma 5.6 are not satisfied. Hence, following Theorem 5.33, we need to construct the reduced operator F * defined on R {1,2} ({1, 2} being the closure of {1}) by

F * (x) = x 1 ∧ 1 2 (x 1 + x 2 ) x 1 ∨ 1 2 (x 1 + x 2 ) , x ∈ R 2
The directed graph associated with this operator is represented in Figure 5.5.

We check that for this reduced operator we have

F + = ∅, {1}, {1, 2} and
F -= ∅, {1, 2} .
Hence, Φ({1}) = ∅ and by Lemma 5.32, we know that F * has no fixed point the arg min of which is {1}. According to Theorem 5.33, the same holds for F . We conclude that any nontrivial fixed point u of F verifies u 1 = u 2 < u 3 < u 4 , and so, from (5.12), also verifies u 3 = 1 2 (u 2 + u 4 ). Finally, note that if the value of the payments are considered as parameters, then, according to Proposition 4.3 and Proposition 4.4, all the realizable mean payoff vectors χ are characterized by

χ 1 = χ 2 χ 4 , χ 3 = 1 2 (χ 1 + χ 4 ) .
CHAPTER 6

A game-abstraction condition for ergodicity of stochastic games

In this chapter, we provide a combinatorial condition for the boundedness of all slice spaces in Hilbert's seminorm for any Shapley operator T on R n . This condition is based on a simplified version of a stochastic game (a game abstraction), constructed from T . The results presented in this chapter have been partly announced in the CDC conference proceedings [START_REF] Akian | Hypergraph conditions for the solvability of the ergodic equation for zero-sum games[END_REF], with a different point of view entirely based on graph-theoretic aspects.

6.1 Game abstraction, dominions and slice spaces 6.1.1 Dominion condition and boundedness of slice spaces

Game abstraction

Let us fix a Shapley operator T : R n → R n . We associate to T an auxiliary two-player stochastic game, denoted by Γ T , defined in the following way. The state space is [n] and when in state i, player MIN can choose a subset of states

I ⊂ [n] such that lim α→+∞ T i (αe I c ) < +∞ . Likewise, in state i, player MAX can choose a subset of states J ⊂ [n] such that lim α→-∞ T i (αe J c ) > -∞ .
At each stage, MIN and MAX choose simultaneously a subset I and J, respectively, and the state at the next stage is drawn uniformly in I ∩ J. Note that here, we are only interested in the dynamic of the state, and therefore we do not need to define a payoff function.

Let us make some observations. First, since T is monotone, the condition for I to be in the action space of MIN in state i is equivalent to the nondecreasing sequence (T i (ke I c )) k∈N being bounded above. Likewise, J is in the action space of MAX in state i if, and only if, the nonincreasing sequence (T i (-ke J c )) k∈N is bounded below. The monotonicity of T also implies that if any player can choose a subset I in a given state, then he can also choose any other subset K ⊃ I.

Second, the action spaces of the two players are never empty in any state, since they contain the set of all states [n]. Also observe that no action space contains the empty set.

Finally, suppose that in state i the subsets I and J chosen by MIN and MAX, respectively, are disjoint. Then, we have e -e J c = e J e I c . By monotonicity and additive homogeneity of T , we deduce that α + T i (-αe J c ) T i (αe I c ) for all scalars α 0. The latter inequality yields that lim α→+∞ T i (αe I c ) and lim α→+∞ T i (-αe J c ) cannot be both finite, a contradiction. Hence, we necessarily have I ∩ J = ∅, which shows, along with the previous observation, that the game is well defined.

Dominions

In the game Γ T , we call dominion a nonempty subset of states D ⊂ [n] such that from any position in D, one player can force the state at the next stage to remain in D almost surely, whatever action the other player chooses. Equivalently, D is a dominion of one player in Γ T if for all states i ∈ D, that player can pick a subset I ⊂ D. This is also equivalent to saying that in each state i ∈ D, the action space of the player contains the subset D.

The main result of this chapter characterizes the boundedness of slice spaces in Hilbert's seminorm in terms of these dominions. To that purpose, we shall need the following condition. Assumption 6.C (Dominion condition). In Γ T , players MIN and MAX have each a dominion, I and J respectively, such that I ∩ J = ∅.

Remark 6.1. The notion of dominion defined here is the same as the one introduced in Section 5.3. The only difference lies in the game with respect to which it is considered. In this chapter, we do not assume that T has an explicit minimax representation, and we construct a game abstraction Γ T , whereas in Section 5.3, the Shapley operator T arises as the dynamic programming operator of a given zero-sum stochastic game Γ, the data of which are known. However, when T arises explicitly from a zero-sum stochastic game Γ satisfying Assumption 5.B, then the dominions are the same in Γ T and in Γ. Indeed, when in state i, player MIN can choose subset I in Γ T if in the one-shot game associated with Γ, MIN is not "penalized" by an arbitrary large final payment of α in states outside I. In particular, if Assumption 5.B holds, it is easy to see that MIN can choose subset I in state i if, and only if, T i (e I c ) = 0. Thus, the subset I is a dominion of MIN in Γ T if, and only if, T (e I c ) e I c , i.e., I is a dominion of MIN in Γ (see Proposition 5.15). A dual statement holds for player MAX.

We now state the main result. Theorem 6.2. Let T : R n → R n be a Shapley operator. All slice spaces S β α (T ) are bounded in Hilbert's seminorm if, and only if, the dominion condition 6.C does not hold in the game Γ T .

According to Theorem 4.21, the above result yields a combinatorial condition of ergodicity. Corollary 6.3. A Shapley operator T : R n → R n is ergodic if, and only if, the dominion condition 6.C does not hold in the game Γ T .

Proof of Theorem 6.2 if part

We prove the converse statement. Suppose that there is some slice space S β α (T ) unbounded in Hilbert's seminorm. So there exists a sequence (u k ) k∈N of vectors in S β α (T ) such that u k H → +∞ as k → +∞. We may assume, without loss of generality, that min i∈[n] u k i = 0 for all integers k, and we let

τ k := max i∈[n] u k i = u k H .
Among all pairs of indices, there is at least one, (i 0 , j 0 ), such that u k i 0 = 0 and u k j 0 = τ k for infinitely many integers k. Then, up to the extraction of a subsequence, we may assume that the latter equalities hold for all integers.

We next explain how to construct a partition of [n] in three subsets I, J, L such that there exists a strictly increasing function φ : N → N for which we have:

-for all i ∈ I, the sequence (u

φ(k) i ) k is bounded; -for all j ∈ J, the sequence (τ φ(k) -u φ(k) j ) k is bounded; -for all ∈ L, the sequences (u φ(k) ) k and (τ φ(k) -u φ(k) ) k tend to ∞.
We proceed by induction, starting with I 0 = {i 0 }, J 0 = {j 0 }, L 0 = ∅, and letting φ 0 be the identity function. Then, assume that we have constructed three disjoint subsets, I m , J m and L m for which the above statements are satisfied for some function φ m : N → N, and such that

I m ∪ J m ∪ L m = [n]. Let i ∈ [n] \ (I m ∪ J m ∪ L m ). If lim inf k→∞ u φm(k) i
is finite, then there exists a bounded subsequence of (u φm(k) i

). In this case, we let I m+1 = I m ∪ {i}, J m+1 = J m , and

L m+1 = L m . If (u φm(k) i ) tends to ∞ but lim inf k→∞ τ φm(k) -u φm(k) i
is finite, then there exists a bounded subsequence of (τ φm(k) -u φm(k) i

). In that case, we let J m+1 = J m ∪ {i}, I m+1 = I m and L m+1 = L m . In both cases, we denote by φ m+1 : N → N the strictly increasing function such that (u

φ m+1 (k) i
) is the aforementioned subsequence. If neither cases hold, then we let L m+1 = L m ∪ {i}, I m+1 = I m , J m+1 = J m and φ m+1 = φ m . The induction is finite and we obtain at the last step a partition of [n] with the required properties. Furthermore, we know that I and J are nonempty.

For the sake of simplicity, assume now that φ is the identity function. For all integers k, let us denote

ρ k := min j / ∈I u k j and σ k := max i / ∈J u k i .
By construction, we have lim k→∞ ρ k = lim k→∞ τ k -σ k = +∞. Let M > 0 be a joint upper bound of the sequences (u k i ) k for all i ∈ I, and (τ k -u k j ) k for all j ∈ J. Then we have, for all indices i ∈ I,

T i (ρ k e I c ) T i (u k ) β + u k i β + M .
This proves that in the game Γ T , MIN can choose subset I in each state i ∈ I. Hence, I is a dominion of MIN. Likewise, for all indices j ∈ J, we have

T j (σ k -τ k )e J c T j (u k -τ k e) α + u k j -τ k α -M ,
which proves that J is a dominion of MAX, and thus that the dominion condition holds in the game Γ T .

only if part

We also prove the converse statement, and so we assume that the dominion condition holds in Γ T . Let I, J ⊂ [n] be disjoint dominions of MIN and MAX, respectively. Then, by definition of the game Γ T , there exists a scalar M > 0 such that, for all k ∈ N,

T i (0) T i (ke I c ) M , ∀i ∈ I , -M T j (-ke J c ) T j (0) , ∀j ∈ J . (6.1)
We can assume that M T (0) ∞ . Let L = [n] \ (I ∪ J) and, for all integers k, let u k be a vector of R n such that

     u k j = k , ∀j ∈ J , -k u k k , ∀ ∈ L , u k i = -k , ∀i ∈ I . (6.2)
Observe that u k H = 2k. Moreover, by construction, we have 0 u k + ke 2ke I c and -2ke J c u k -ke 0. From the inequalities (6.1), we then deduce that

-M T i (0) T i (u k ) + k = T i (u k ) -u k i M , ∀i ∈ I , -M T j (u k ) -k = T j (u k ) -u k j T j (0) M , ∀j ∈ J .
If L = ∅, we readily have that u k ∈ S M -M (T ) for all k ∈ N, so that S M -M (T ) contains a sequence which is unbounded in Hilbert's seminorm, and the proof of the "only if" part is complete in this case.

Assume now that L = ∅. We next show that we can find scalars α, β for which it is possible to construct, for all k ∈ N, a vector u k satisfying (6.2) and such that α T (u k ) -u k β for every ∈ L. To that purpose, we shall use the Poincaré-Miranda theorem, a generalization of the intermediate value theorem, see [START_REF] Kulpa | The Poincaré-Miranda theorem[END_REF].

Theorem 6.4 (Poincaré-Miranda). Let f : [0, 1] n → R n be a continuous map such that for each i ∈ [n], f i (x)
0 whenever x i = 0 and f i (x) 0 whenever x i = 1. Then there exists a point

x * ∈ [0, 1] n such that f (x * ) = 0.
Let k ∈ N be a fixed integer and G : [0, 1] L → [-k, k] n be the map given by

     G j (x) = k , ∀j ∈ J , G (x) = -k + 2kx , ∀ ∈ L , G i (x) = -k , ∀i ∈ I .
Let F : [0, 1] L → R L be the map whose th entry is defined by

F (x) = G (x) -T • G(x) + T (0) . Since we have -k + T (0) T • G(x) k + T (0)
for every x ∈ [0, 1] L and every ∈ L, then we deduce that F (x) 0 for all x such that x = 0. Likewise, F (x) 0 for all x such that x = 1. Thus, according to the Poincaré-Miranda theorem, there exists a point

x k ∈ [0, 1] L such that F (x k ) = 0. Fixing u k = G(x k ), we get that T (u k ) -u k = T (0) for all ∈ L.
Hence, it is possible to choose a vector u k satisfying (6.2), and such that u k ∈ S M -M (T ) for all integers k ∈ N. So, S M -M (T ) contains a sequence which is unbounded in Hilbert's seminorm, which conclude the proof of the "only if" part and of Theorem 6.2.

Convex case

In this subsection, we consider a convex Shapley operator T : R n → R n , meaning that every coordinate function of T is convex. In this particular case, we show that Theorem 6.2 simplifies. More precisely, we next show that the two-player game Γ T can be reduced to a one-player game in which the actions of MIN are not taken into account while the actions of MAX are essentially the same. However, since many actions of MAX in Γ T are superfluous (recall that if he can choose a subset J, then he can also choose any subset K ⊃ J), we shall need to restrict his action spaces.

Before introducing this one-player game, let us give the following definition. For any index i ∈ [n], we call support of T i , and we denote it by supp(T i ), the subset of indices j ⊂ [n] such that T i depends effectively on x j , in the sense that there is no map h : R n-1 → R such that

T i (x) = h(x 1 , . . . , x j-1 , x j+1 , . . . , x n ) , ∀x ∈ R n .
The following lemma provides an alternative characterization of the support. Lemma 6.5 ([GG04, Prop. 2]). Let T : R n → R n be a convex Shapley operator. For all i ∈ [n], index j is in supp(T i ) if, and only if, lim α→+∞ T i (αe {j} ) = +∞.

We now define a one-player game, denoted by Γ T , as follows. The state space is [n] and when in state i the player, called MAX, can choose a nonempty subset of state

J ⊂ supp(T i ) such that lim α→-∞ T i (αe J c ) > -∞ .
Once this subset is selected, the next state is chosen in J with uniform probability. For the same reason as with Γ T , we do not need to define a payoff function.

The game Γ T is well defined since in any state i ∈ [n], MAX has the possibility to choose action J = supp(T i ). Indeed, T i does not depend on any x j with j / ∈ J, which yields that T i (αe J c ) is independent of α. Moreover, supp(T i ) is nonempty, otherwise T i would be constant.

By construction, any action of MAX in Γ T is also an action of MAX in Γ T . The following result gives a converse statement. Lemma 6.6. Let T be a convex Shapley operator on R n . For every state in [n], if player MAX can choose a subset of states J in Γ T , then there exists an action K ⊂ J for MAX in Γ T .

Proof. Let J ⊂ [n] be a possible action of MAX in state i in the game Γ T , that is, such that α → T i (αe J c ) is lower bounded. If there is some j ∈ J \ supp(T i ), then T i does not depend on x j and so α → T i (αe J c ∪{j} ) is still lower bounded. Thus, proceeding by induction, we find a subset K ⊂ J ∩ supp(T i ) such that the sequence lim α→-∞ T i (αe K c ) is finite.

We define the same notion of dominion in Γ T as in Γ T , which amounts to say that a subset D is a dominion if in any state i ∈ D, MAX can choose a subset J ⊂ D. In particular, Lemma 6.6 yields that a subset of states is a dominion of MAX in the two-player game Γ T if, and only if, it is a dominion in the one-player game Γ T . We shall also need the following notion. We say that a nonempty subset S ⊂ [n] is a sink if MAX cannot make the state to leave S with positive probability once it has reached it, that is, for any state i ∈ S, the action space of MAX contains only subsets J ⊂ S. Equivalently, S is a sink if, and only if, supp(T i ) ⊂ S for all i ∈ S. Observe that the set of sinks is not empty, since it contains the set of all states. Moreover, a sink is by definition also a dominion.

We can now adapt Theorem 6.2 to the convex case.

Theorem 6.7. Let T : R n → R n be a convex Shapley operator. All slice spaces S β α (T ) are bounded in Hilbert's seminorm if, and only if, MAX has no dominion which has an empty intersection with a sink in Γ T .

Proof. Suppose first that in Γ T there exist a sink S and a dominion D with empty intersection. We readily have that D is a dominion of MAX in Γ T . Moreover, for every i ∈ S, we have supp(T i ) ⊂ S. Hence, T i (αe S c ) is independent of α and so, S can be chosen by MIN in the game Γ T . This shows that S is a dominion of MIN in the latter two-player game. Thus, the dominion condition holds in Γ T which implies that there exists a slice space unbounded in Hilbert's seminorm, according to Theorem 6.2.

Suppose now there exist an unbounded slice space, that is, according to the aforementioned theorem, players MIN and MAX have each a dominion, I and J respectively, such that I ∩J = ∅. We already know that J is also a dominion of MAX in Γ T . We next show that I is a sink in Γ T , which will conclude the proof. Let i ∈ I and j / ∈ I. By monotonicity of T we have, for all positive scalars α, T i (αe {j} )

T i (αe I c ). Since the right-hand side of the latter inequality is bounded by a constant independent of α, we deduce from Lemma 6.5 that j / ∈ supp(T i ). This yields that supp(T i ) ⊂ I. Since this is true for any i ∈ I, we deduce that I is a sink in Γ T .

Hypergraph characterization and complexity aspects

In this section, we give a graph-theoretic construction that allows us to check the dominion condition.

Reachability conditions and dominions

Given any Shapley operator T : R n → R n , the actions in the stochastic game Γ T are determined by the "behavior at infinity" of T along some directions. To express that behavior, we introduce a pair of hypergraphs, denoted by (H + , H -), defined as follows:

-the set of nodes for both H + and H -is [n]; -the hyperarcs of H + are the pairs (J, {i}) such that i / ∈ J and

lim α→+∞ T i (αe J ) = +∞ ;
-the hyperarcs of H -are the pairs (J, {i}) such that i / ∈ J and

lim α→-∞ T i (αe J ) = -∞ .
From the definition of the hypergraphs H ± , we readily get the following characterization of invariant subsets in these hypergraphs. Lemma 6.8. A subset of nodes J ⊂ [n] is invariant in H + (resp., H -) if, and only if,

lim α→+∞ T i (αe J ) < +∞ , ∀i ∈ [n] \ J resp., lim α→-∞ T i (αe J ) > -∞ , ∀i ∈ [n] \ J .
The above lemma yields that a nonempty subset I is a dominion of MIN in Γ T if, and only if, its complement is invariant in H + , i.e., reach(I c , H + ) = I c . Likewise, a nonempty subset J is a dominion of MAX in Γ T if, and only if, its complement is invariant in H -, i.e., reach(J c , H -) = J c . Thus, Theorem 6.2 can be reformulated in terms of reachability Theorem 6.9. Let T : R n → R n be a Shapley operator operator. All slice spaces S β α (T ) are bounded in Hilbert's seminorm if, and only if, there are no nonempty disjoint subsets of [n], (I, J), such that reach(I c , H + ) = I c and reach(J c , H -) = J c . Remark 6.10. If T satisfies Assumption 4.A, then T and its recession operator T have the same asymptotic behaviors. Therefore, the hypergraph G + (resp., G -) defined in Section 5.2 is the same as H + (resp., H -). Thus, the condition in Theorem 6.9 recovers Point (ii) of Corollary 5.13.

Convex case

In this subsection, we consider a convex Shapley operator T : R n → R n . In the same way as the dominion condition 6.C in the two-player game Γ T simplifies to a condition in the one-player game Γ T , we next show that the hypergraph-reachability conditions of Theorem 6.9 have a simpler formulation.

We associate to T the directed graph, denoted by G, with set of vertices [n] and an edge from i to j if lim α→+∞ T i (αe {j} ) = +∞, or equivalently, if j ∈ supp(T i ) (see Lemma 6.5). Recall that a final class of any directed graph is a nonempty set of nodes C such that every two nodes of C are connected by a directed path and every path starting from a node in C remains in it. Also recall that a subset S is a sink in the one-player game Γ T if, and only if, supp(T i ) ⊂ S for every i ∈ S. Thus, a final class of G is also a sink in Γ T and it is readily seen that any sink of the latter game contains a final class of G.

The translation of Theorem 6.7 in terms of graph leads to the following.

Theorem 6.11. Let T be a convex Shapley operator on R n . All slice spaces S β α (T ) are bounded in Hilbert's seminorn if, and only if, G has a unique final class C and reach(C,

H -) = [n].
Proof. Following Theorem 6.7, we show that in the one-player game Γ T , player MAX has a dominion disjoint from a sink if, and only if, the directed graph G has more than one final class, or a unique final class which does not have access to the whole set of nodes in H -.

First suppose that G has two distinct final classes. Then, these sets are both sinks in Γ T , and since any sink is also a dominion of MAX, it follows that the dominion-sink condition holds.

Next, assume that G has a unique final class C and that reach(C,

H -) = [n]. Let D := [n] \ reach(C, H -).
The subsets C and D are nonempty and disjoint. Furthermore, C is a sink and D is by construction a dominion of MAX in Γ T (since its complement is invariant in H -), hence it is also a dominion in Γ T . Now, assume that S is a sink and D a dominion in Γ T such that S ∩ D = ∅, and that G has a unique final class, denoted C. Then, we necessarily have C ⊂ S, since any sink contains a final class. Hence, C and D are disjoint, that is,

C ⊂ D c , which yields reach(C, H -) ⊂ reach(D c , H -) = D c = [n].

Complexity aspects

Given any Shapley operator T on R n , the basic issue under consideration is to check whether the ergodic equation (2.5) is solvable. Theorem 6.9 (or Theorem 6.11 in the convex case) provides a combinatorial condition for this property to hold. This condition can be effectively checked as soon as the limits lim α→±∞ T i (αe J ) arising in the definition of the hyperarcs of H ± can be computed, which happens in general situations, see examples in Section 6.3. To set aside the latter problem, let us introduce the oracle Ω which answers "yes" to any instance (J, i, ±) if, and only if, T i (αe J ) → ±∞ as α → ±∞. We call Turing machine with oracle Ω a Turing machine which can send a query to Ω and read the output. A call to Ω is counted as one computational step of the Turing machine. We refer the reader to [START_REF] Arora | Computational complexity[END_REF] for a detailed presentation of oracle Turing machines.

We shall need the following lemma, which gives a bound for the time required to compute the set of reachable nodes in H ± from any subset. We use the notation poly(n) to indicate a polynomial function of n. Lemma 6.12. For any subset J ⊂ [n], reach(J, H ± ) can be computed in O(n 2 ) steps by a Turing machine with oracle Ω.

Proof. Let J 1 = J. If T i (αe J 1 ) remains bounded as α → ±∞ for all i / ∈ J 1 , then J 1 is invariant in H ± according to Lemma 6.8. This means that J 1 = reach(J, H ± ). Otherwise, define J 2 as the union of J 1 and all the nodes i / ∈ J 1 for which T i (αe J 1 ) tends to ±∞ as α → ±∞. Repeating the same steps, we arrive at a subset J k for some integer k n, which is invariant by H ± and contains J. Hence, we must have reach(J, H ± ) ⊂ J k since reach(J, H ± ) is the smallest subset satisfying the latter properties. The other inclusion being trivial, we get J k = reach(J, H ± ). Now observe that each step of the latter recursion requires |(J ) c | calls to the oracle Ω. So, the number of calls is bounded by n 2 . Furthermore, the number of elementary operations in each step is linear with respect to the number of calls. Hence the result.

It readily follows from the characterization of dominions that the problem of deciding whether a subset J is a dominion, i.e., reach(J c , H ± ) = J c , can be solved in O(|J|) steps by the oracle Turing machine. Furthermore, it is easily seen that the reachability conditions of Theorem 6.9 boil down to check that, for every

I ⊂ [n], either reach(I c , H + ) = I c or reach(I, H -) = [n]
. Then, we readily get the following. Theorem 6.13. Let T : R n → R n be any Shapley operator. The problem of deciding whether all slice spaces are bounded in Hilbert's seminorm can be solved in O(2 n n 2 ) steps by a Turing machine with oracle Ω.

Recall that the exponential bound cannot be reduced to a polynomial one unless P = coNP. Indeed, the restricted version of this problem concerning Shapley operators with finite action spaces reduces to check the existence of a nontrivial fixed point of a payment-free Shapley operator, a problem which has been shown to be coNP-complete, see Subsection 5.4.2 and in particular Corollary 5.30. However, when T is convex, the condition in Theorem 6.11 requires the computation of the final classes of the directed graph G. Since it has n nodes, it is known that finding its strongly connected components can be done in O(n 2 ) time. This leads us to the following bound. Theorem 6.14. Let T : R n → R n be a convex Shapley operator. The problem of deciding whether all slice spaces are bounded in Hilbert's seminorm can be solved in O(n 2 ) steps by a Turing machine with oracle Ω.

Let us point out that simpler sufficient conditions given in [START_REF] Gaubert | The Perron-Frobenius theorem for homogeneous, monotone functions[END_REF] (see Subsection 2.3.3), involving directed graphs instead of hypergraphs, can be checked using only a polynomial number of elementary operations, but they are less accurate.

Examples and applications

Shapley operator with unbounded payments

Here, we provide the example of an ergodic stochastic game for which the conditions given in [START_REF] Gaubert | The Perron-Frobenius theorem for homogeneous, monotone functions[END_REF][START_REF] Akian | Ergodicity conditions for zero-sum games[END_REF] do not apply. We also illustrate the hypergraph characterization of Theorem 6.9. The example is an adaptation of the classical Blackmailer's Dilemma [START_REF] Whittle | Optimization over time[END_REF], a one-player game with two states, one absorbing with a zero payoff and one characterized by a dilemma between obtaining a good stage payoff and increasing the probability to leave the state.

Let us consider the Shapley operator T : R 3 → R 3 given by

T (x) =      sup 0<p 1 log p + p(x 2 ∧ x 3 ) + (1 -p)x 1 inf 0<p 1 -log p + px 3 + (1 -p)x 1 x 3      .
where ∧ stands for min. The map T is the dynamic programming operator of a zero-sum stochastic game with three states: the first player, called here Player I, partially controls state 1, the second player, called Player II, controls state 2, and state 3 is an absorbing state. In state 1, Player I chooses an action p ∈ (0, 1] and receives log p from Player II. Then with probability 1 -p, the next state remains 1, and with probability p it is chosen by Player II between state 2 and state 3. Thus, in order to maximize the stage payoff, one would select p = 1, but this leads to leave state 1 with probability one. A dual interpretation applies to Player II in state 2.

Let us denote by A i and B i the action spaces of players MIN and MAX, respectively, in any state i of the game Γ T . In order to determine these spaces, it is convenient to notice that

T 1 (x) = h (x 2 ∧ x 3 ) -x 1 + x 1 and T 2 (x) = -h(x 1 -x 3 ) + x 1 ,
where h is the real function defined by

h(z) = sup 0<p 1 (log p + pz) , ∀z ∈ R .
Also note that h satisfies h(z) = -1 -log(-z) for z -1 and h(z) = z for z -1. Thus, we get

A 1 = {{1, 2}, {1, 3}, {1, 2, 3}} , B 1 = {{2, 3}, {1, 2, 3}} , A 2 = {{3}, {1, 3}, {2, 3}, {1, 2, 3}} , B 2 = {{1, 3}, {1, 2, 3}} , A 3 = {{3}, {1, 3}, {2, 3}, {1, 2, 3}} , B 3 = {{3}, {1, 3}, {2, 3}, {1, 2, 3}} .
Then, the dominions of MIN in Γ T are {3}, {1, 3}, {2, 3} and {1, 2, 3}, whereas the dominions of MAX are {3} and {1, 2, 3}. It follows that the dominion condition is not satisfied, and so all slice spaces of T are bounded in Hilbert's seminorm. As a consequence, T is ergodic, that is, the ergodic equation (2.5) is solvable for all operators g + T with g ∈ R 3 . It is instructive to observe that MAX (resp., MIN) can choose a subset J in state i in the game Γ T if, and only if, Player I (resp., Player II) can force the state at the next stage to be in J almost surely in the game with Shapley operator T .

In order to check the ergodicity of T , one may alternatively construct the hypergraphs H ± associated with T . Figure 6.1 shows a concise representation of these hypergraphs, in which only the hyperarcs with minimal tail (with respect to the inclusion partial order) have been represented. For instance, there is no arc from {2} to {1} in H + since T 1 (αe 2 ) = 0 for all α 0. However, there is a hyperarc from {2, 3} to {1}, since T 1 (αe {2,3} ) = α for all α 0, which yields lim α→+∞ T 1 (αe {2,3} ) = +∞. Then, one may check that there are no nonempty disjoint subsets of [n], (I, J), such that reach(I c , H + ) = I c and reach(J c , H -) = J c , as in Theorem 6.9.

The above conclusions cannot be obtained from the conditions stated in [START_REF] Gaubert | The Perron-Frobenius theorem for homogeneous, monotone functions[END_REF] or in Section 4.2 in terms of fixed points of the recession operator. Indeed, here the latter operator writes

T (x) =   x 1 ∨ (x 2 ∧ x 3 ) x 1 ∧ x 3 x 3   .
Hence, any vector (α, 0, 0) with α 0 is a fixed point of T .

f i (x) = (Ax (d-1) ) i := i 2 ...i d ∈[n] a i i 2 ...i d x i 2 . . . x i d .
An important problem, introduced independently by Lim [START_REF] Lim | Singular values and eigenvalues of tensors: a variational approach[END_REF] and Qi [START_REF] Qi | Eigenvalues of a real supersymmetric tensor[END_REF], concerns the existence of an eigenvalue λ ∈ R and an eigenvector u ∈ R n , solution of

f (u) = Au (d-1) = λu d-1 , (6.3)
where

u d-1 = (u d-1 1 , . . . , u d-1 n ).
If the tensor A is nonnegative, meaning that a i 1 ...i d 0 for all indices, a variant of this problem is the existence of a positive eigenvalue λ > 0 and an eigenvector u ∈ int(R n + ) in the standard positive cone of R n . The latter problem can be seen as a particular case of a nonlinear Perron-Frobenius eigenproblem. In particular, Friedland, Gaubert and Han [START_REF] Friedland | Perron-Frobenius theorem for nonnegative multilinear forms and extensions[END_REF] showed, as a consequence of [GG04, Th. 2], that a nonnegative tensor A has a positive eigenvalue λ > 0 and an eigenvector u ∈ int(R n + ) if A is weakly irreducible. This notion can be defined by means of the directed graph G(A), with set of nodes [n] and an edge from i to j if there exists a set of indices i 2 , . . . , i d containing j and such that a i i 2 ...i d > 0. Then, the nonnegative tensor A is weakly irreducible if G(A) is strongly connected. Theorem 6.11 allows us to extend the latter result. For that purpose, let us introduce the hypergraph H(A) with set of nodes [n] and an hyperarc from J ⊂ [n] to {i} if i / ∈ J and a i i 2 ...i d > 0 implies that J ∩ {i 2 , . . . , i d } = ∅. Proof. Consider T := (d -1) -1 log •f • exp. This is a monotone additively homogeneous selfmap of R n . Furthermore, any eigenpair (µ, v) ∈ R × R n of T yields an eigenpair (λ, u) of f with the required properties, namely λ = e µ(d-1) > 0 and u = exp(v) ∈ int(R n + ). This is a standard result that functions such as log •f i • exp are convex -it relies on the convexity of the "log-exp" function x → log(e x 1 + • • • + e xn ), see [RW98, Ex. 2.16, Ex. 2.52]. Hence T is convex. Now, observe that the directed graph G and the hypergraph H -associated with the convex Shapley operator T are the same as G(A) and H(A), respectively. The conclusion follows from an application of Theorem 6.11.

Part III

Generic uniqueness of the bias vector of zero-sum stochastic games

CHAPTER 7

Generic uniqueness of the bias of perfect-information nite stochastic games

Introduction

The description of the set of bias vectors of zero-sum stochastic games is a fundamental issue.

For one-player problems, i.e., for discrete optimal control, the representation of bias vectors and their relation with optimal strategies is well understood, in either the deterministic or the stochastic case (MDPs). In the deterministic case, the analysis relies on max-plus spectral theory, which goes back to the work of Romanovsky [START_REF] Romanovsky | Optimization of stationary control of a discrete deterministic process[END_REF], Gondran and Minoux [START_REF] Gondran | Valeurs propres et vecteurs propres dans les dioïdes et leur interprétation en théorie des graphes[END_REF] and Cuninghame-Green [START_REF] Cuninghame-Green | Minimax algebra[END_REF]. We also refer the reader to [MS92, BCOQ92, ABG13] for more background on max-plus spectral theory. Kontorer and Yakovenko [START_REF] Kontorer | Nonlinear semigroups and infinite horizon optimization[END_REF] and Kolokoltsov and Maslov [START_REF] Kolokoltsov | Idempotent analysis and its applications[END_REF] deal specially with infinite-horizon optimization and mean-payoff problems. In this framework, the set of bias vectors has the structure of a max-plus (tropical) cone, i.e., it is invariant by max-plus linear combinations, and it has a unique minimal generating family consisting of certain "extreme" generators, which can be identified by looking at the support of the maximizing measures in the linear programming formulation of the optimal control problem, or at the "recurrence points" of infinite optimal trajectories. A combinatorial interpretation of some of these results, in terms of polyhedral fans, has been recently given by Sturmfels and Tran [START_REF] Sturmfels | Combinatorial types of tropical eigenvectors[END_REF]. The ergodic equation (2.5) and the structure of bias vectors has also been studied for an infinite-dimensional state space in the context of infinite-dimensional maxplus spectral theory, see Akian, Gaubert and Walsh [START_REF] Akian | The max-plus Martin boundary[END_REF], and also in the setting of weak KAM theory, for which we refer the reader to Fathi [START_REF] Fathi | Weak KAM theorem in Lagrangian dynamics[END_REF]. In the stochastic case, the structure of the set of bias vectors is also known, at least when the state space is finite, see Akian and Gaubert [START_REF] Akian | Spectral theorem for convex monotone homogeneous maps, and ergodic control[END_REF].

In the two-player case, the structure of the set of bias vectors is less well known. In particular, the uniqueness of the bias vector up to an additive constant is an important matter for algorithmic purposes. This is specially the case when applying the standard Hoffman-Karp policy iteration algorithm, since nonuniqueness typically leads to numerical instabilities or degeneracies. Hoffman and Karp [START_REF] Hoffman | On nonterminating stochastic games[END_REF] introduced this algorithm to solve mean-payoff zero-sum stochastic games with perfect information and finite state and action spaces. They showed that policy iteration does terminate if every pair of strategies of the two players yields an irreducible Markov chain. However, policy iteration may cycle if this irreducibility assumption is not satisfied, which is the case for many classes of games -in particular, it is essentially never satisfied for deterministic games. Some refinements of the Hoffman-Karp scheme have been proposed by Cochet-Terrasson and Gaubert [START_REF] Cochet-Terrasson | A policy iteration algorithm for zero-sum stochastic games with mean payoff[END_REF], Akian, Cochet-Terrasson, Detournay and Gaubert [START_REF] Akian | Policy iteration algorithm for zero-sum multichain stochastic games with mean payoff and perfect information[END_REF], Bourque and Raghavan [START_REF] Bourque | Policy improvement for perfect information additive reward and additive transition stochastic games with discounted and average payoffs[END_REF], allowing one to circumvent such degeneracies at the price of an increased complexity of the algorithm (handling the nonuniqueness of the bias vector). Hence, it is of interest to understand when such technicalities can be avoided.

In this chapter, we address the question of the uniqueness of the bias vector of stochastic games with perfect information and finite state and action spaces, restricting our attention to ergodic games, i.e., for which the ergodic equation (2.5) is solvable for all (state-dependent) perturbations of the transition payments. The main result, Theorem 7.8, shows that the bias vector is generically unique up to an additive constant. More precisely, it shows that the set of perturbation vectors for which the bias vector is not unique belongs to a polyhedral complex the cells of which have codimension one at least. We then deduce that the Hoffman-Karp policy iteration algorithm does converge if the payment is generic. This leads to an explicit perturbation scheme, allowing one to solve nongeneric instances by policy iteration, avoiding the classical irreducibility condition.

In the following chapter, we address the generic uniqueness problem for any zero-sum stochastic game with finite state space. We show that generic uniqueness of the bias vector holds locally, that is, for a generic perturbation in a closed set with nonempty interior, as soon as the ergodic equation (2.5) is solvable for all perturbations in a neighborhood of this set.

The theoretic results (Section 7.3) persented in this chapter have been announced in the CDC conference proceedings [START_REF] Akian | Generic uniqueness of the bias vector of mean payoff zero-sum games[END_REF]. The application to policy iteration (Section 7.4) is new.

Further preliminaries in nonlinear Perron-Frobenius theory

In the remainder of this chapter, we consider a finite zero-sum stochastic game with perfect information Γ = ([n], A, B, K A , K B , r, p). Since the action spaces A and B are finite, its Shapley operator is a piecewise affine map, the ith coordinate of which writes

T i (x) = min a∈A i max b∈B i,a r(i, a, b) + j∈[n] x j p(j | i, a, b) , ∀x ∈ R n . (7.1)
Recall in particular that T has an invariant half-line (Theorem 2.5), that is, there exist two vectors u, ν ∈ R n such that T (u + αν) = u + (α + 1)ν for all α 0. Furthermore, the vector ν is unique and equal to the mean-payoff vector of Γ: χ(T ) = lim k→∞ T k (0)/k.

Characterization of the upper mean payo

Let us denote by S s the finite set of deterministic stationary strategies of player MIN, also called (deterministic) policies. It is the set of maps σ : [n] → A such that σ(i) ∈ A i for every state i ∈ [n]. Likewise, we denote by T σ s the set of deterministic stationary strategies of player MAX when the policy σ has been chosen by MIN, i.e., the set of maps τ :

[n] → B such that τ (i) ∈ B i,σ(i) for every state i ∈ [n].
We introduce the reduced Shapley operator T σ : R n → R n associated with the policy σ ∈ S s of player MIN. Its ith coordinate map is given by

T σ i (x) = max b∈B i,σ(i) r(i, σ(i), b) + j∈[n] x j p(j | i, σ(i), b) , ∀x ∈ R n .
We endow R n with the usual partial order. Then, since the action spaces are finite, we readily have, for all x ∈ R n , T (x) = min σ∈Ss T σ (x) . (7.2) Likewise, we have, for all x ∈ R n ,

T σ (x) = max τ ∈T σ s r στ + P στ x , (7.3)
where r στ is the vector in R n whose ith entry is defined by

r στ i = r(i, σ(i), τ (i))
and P στ is the n × n stochastic matrix whose (i, j) entry is given by

P στ ij = p(j | i, σ(i), τ (i)) .
Observe that T σ is monotone, additively homogeneous and convex, in the sense that each of its coordinates is a convex function.

Recall that if P is a n×n stochastic matrix, the directed graph associated with P is composed of the nodes 1, . . . , n and of the arcs (i, j) such that P ij > 0. A class of the matrix P is a maximal set of nodes such that every two nodes in this set are connected by a directed path. A class is said to be final if every path starting from a node of this class remains in it. Let us denote by M(P ) the set of invariant probability measures of P , i.e., the set of stochastic vector m ∈ R n such that m P = m . Given a final class C of P , there is a unique invariant probability measure m ∈ M(P ) the support of which is C, i.e., {i ∈ [n] | m i > 0} = C. Moreover, the set M(P ) is the convex hull of such measures, and since the number of final class of P is finite, M(P ) is a convex polytope.

Let us denote by χ(T ) the upper mean payoff of T , i.e., the greatest entry of the mean-payoff vector χ(T ). We next give a characterization of χ(T ). Obviously, if the ergodic equation (2.5) is solvable, then χ(T ) is a constant vector, the coordinates of which are equal to the eigenvalue λ(T ), so that the following result also provides a characterization of the eigenvalue. In the sequel, we denote by x, y the standard scalar product in R n of two vectors x, y. Lemma 7.1. Let T : R n → R n be the Shapley operator (7.1) of a finite stochastic game with perfect information Γ. Then the upper mean payoff of T is given by

χ(T ) = min σ∈Ss max{ m, r στ | τ ∈ T σ s , m ∈ M(P στ )} . (7.4)
Proof. First, observe that for all policies σ ∈ S s we have T T σ , which yields, by monotonicity of the operators, χ(T ) χ(T σ ) and in particular χ(T ) χ(T σ ).

Considering an invariant half-line of T , we know that there exist a vector u ∈ R n such that T (u) = u + χ(T ). Let σ ∈ S s be a policy of player MIN such that T (u) = T σ (u). Then, we have T σ (u) u + χ(T )e. Furthermore, we know by a Collatz-Wielandt formula (see [GG04, Prop. 1]) that, for any monotone and additively homogeneous map F : R n → R n , we have

χ(F ) = inf{µ ∈ R | ∃x ∈ R n , F (x) µe + x} .
So, we deduce that χ(T σ ) χ(T ), and finally that χ(T ) = min σ∈Ss χ(T σ ) . Now we fix a policy σ of MIN, and we let χ := χ(T σ ) and µ σ := max{ m, r στ | τ ∈ T σ s , m ∈ M(P στ )} . Since T σ has an invariant half-line with direction χ, there exists a vector v ∈ R n such that T σ (v + αχ) = v + (α + 1)χ for all α 0. In particular, for every policy τ ∈ T σ s we have

r στ + P στ v T σ (v) = v + χ v + χ(T σ )e .
Then, we easily deduce that µ σ χ(T σ ).

Furthermore, since the germs of affine functions from R to R at infinity are totally ordered, there exists a policy τ ∈ T σ s such that

T σ (v + αχ) = r στ + P στ (v + αχ)
for all α large enough. In particular, since the equality v + (α + 1)χ = r στ + P στ (v + αχ) (7.5) holds for all α large enough, we get that P στ χ = χ. Thus, χ is an harmonic vector for the stochastic matrix P στ . As such, it is constant on any final class of P στ and its maximum is attained on one of these final class (see [AG03, Lem. 2.9]). Let m ∈ M(P στ ) be the invariant probability measure associated with a final class C of P στ such that χ i = χ(T σ ) for all i ∈ C. Then, we deduce from (7.5) that m, r στ = χ(T σ ), which yields µ α χ(T σ ) and finally µ α = χ(T σ ).

Let us mention that in (7.4), the set of invariant probability measures of P στ , M(P στ ), may be replaced by the set of its extreme points, denoted by M * (P στ ), since it is a convex polytope. Recall that M * (P στ ) is the set of invariant probability measures the support of which are the final classes of P στ .

Structure of the eigenspace

An ingredient of our approach is a result of [START_REF] Akian | Spectral theorem for convex monotone homogeneous maps, and ergodic control[END_REF] which describes the eigenspace, i.e., the set of bias vectors, of one-player Shapley operators T . We next recall this result. Let us assume that T arises from a game in which only player, MAX, has nontrivial actions, whereas player MIN has only one possible policy σ. In this case, the representation of the eigenvalue λ(T ) = χ(T ) in Lemma 7.1 simplifies as the dependency in σ can be dropped, and we arrive, with a trivial simplification of the notation, to

λ(T ) = max{ m, r τ | τ ∈ T s , m ∈ M(P τ )} . (7.6)
It is shown in [START_REF] Akian | Spectral theorem for convex monotone homogeneous maps, and ergodic control[END_REF] that the dimension of the eigenspace of T is controlled by the number of critical classes. The latter can be defined through the notion of maximizing measures, which rely on behavioral policies. Recall that for every state i, such a policy τ assigns to every action b ∈ B i a probability τ (b | i) that this action is selected. This leads to the stochastic matrix P τ with entries P τ ij = b∈B i p(j | i, b) τ (b | i), and to the payment vector r τ with entries

r τ i = b∈B i r(i, b) τ (b | i).
Then, the maximum in (7.6) is unchanged if it is taken over the set of behavioral policies τ and of invariant measures m of the corresponding stochastic matrix P τ (see [START_REF] Akian | Spectral theorem for convex monotone homogeneous maps, and ergodic control[END_REF]Prop. 7.2]). Then a measure m is maximizing if λ(T ) = m, r τ for some behavioral policy τ and if m ∈ M * (P τ ), that is, the support of m is a final class of P τ . A subset I ⊂ [n] is a critical class if there exists a maximizing measure m whose support is I and if I is a maximal element with respect to inclusion among all the subsets of [n] which arise in this way. Note that critical classes are disjoint.

The following lemma gives a sufficient condition for the critical class to be unique. Note that the condition does not require behavioral stationary strategies. Lemma 7.2. Let T : R n → R n be a convex Shapley operator. Suppose that the ergodic equation (2.5) is solvable. If there is a unique probability measure which attains the maximum in (7.6), then T has a unique critical class.

Proof. Let C be a critical class of T . There exists a behavioral policy τ such that C is a final class of P τ and such that the invariant measure m with support C satisfies λ := λ(T ) = m, r τ .

Let u be an eigenvector of T . For every i ∈ [n], we have λ + u i -r τ i -P τ i u 0. Furthermore, since m P τ = m , we also have

i∈C m i (λ + u i -r τ i -P τ i u) = λ + m, u -m, r τ -m, (P τ u) = 0 .
This yields that λ + u i -r τ i -P τ i u = 0 for every i ∈ C. With a similar argument, we show that λ + u i -r b i -P b i u = 0 for every i ∈ C and every b such that τ (b | i) > 0. Let τ be a deterministic policy such that for all indices i, τ

(i) = b i ∈ supp(τ (• | i)), i.e., τ (b i | i) > 0. Since P τ
i is a convex combination of vectors one of which is P τ i , we deduce that C contains a final class C of P τ . Let m be the invariant measure of P τ whose support is C . Then, we have λ = m , r τ , which implies that m is the unique measure attaining the maximum in (7.6). It follows that that C is necessarily unique since it must contain the support of m .

We now describe the eigenspace of T .

Theorem 7.3 ([AG03, Th. 1.1]). Let T : R n → R n be a convex Shapley operator. Suppose that the ergodic equation (2.5) is solvable, and let C ⊂ [n] be the union of all critical classes. We denote by π C the restriction map R n → R C , x → (x i ) i∈C . Then, the set of eigenvectors of T , denoted by E(T ), satisfies the following properties:

(i) every element x of E(T ) is uniquely determined by its restriction π C (x);

(ii) the set π C (E(T )) is convex and its dimension is at most equal to the number of critical classes of T ; moreover, the latter bound is attained when T is piecewise affine.

In particular, combined with Lemma 7.2, the above result yields the following.

Corollary 7.4. Let T : R n → R n be a convex Shapley operator. Suppose that the ergodic equation (2.5) is solvable. If there is a unique probability measure which attains the maximum in (7.6), then T has a unique eigenvector up to an additive constant.

We refer the reader to [START_REF] Akian | Spectral theorem for convex monotone homogeneous maps, and ergodic control[END_REF] for more background on critical classes, which admit several characterizations and can be computed in polynomial time when the game is finite. We only provide here a simple illustration in order to understand the latter theorem.

Example 7.5. Let T : R 2 → R 2 be such that

T (x) = x 1 ∨ 1 2 (x 1 + x 2 ) -3 + x 2 ∨ 1 2 (x 1 + x 2 )
.

We have T (0) = 0, which shows in particular that the upper mean payoff is 0. If MAX chooses, when in state 1, the action corresponding to the first term in the expression of T 1 , and when in state 2, the action corresponding to the second term in the expression of T 2 , then we arrive at the transition matrix

P = 1 0 1/2 1/2
which has the invariant measure m = (1, 0). This measure attains the maximum in (7.6). However, its support I = {1} is not a critical class, for it is not maximal with respect to inclusion. Indeed, if MAX chooses instead, when in state 1, the action corresponding to the second term in the expression of T 1 , we arrive at the transition matrix

P = 1/2 1/2 1/2 1/2
which has the invariant measure m = (1/2, 1/2). This measure attains the maximum in (7.6) and its support I = {1, 2} is maximal with respect to inclusion. Hence, I = {1, 2} is the unique critical class. It follows from Theorem 7.3 that 0 is the unique eigenvector of T , up to an additive constant.

Another ingredient is a variant of a result of Bruck [START_REF] Bruck | Properties of fixed-point sets of nonexpansive mappings in Banach spaces[END_REF] concerning the topology of fixedpoint sets of nonexpansive maps. We now consider a two-player Shapley operator T such that the ergodic equation (2.5) is solvable, and denote by

E(T ) := {u ∈ R n | T (u) = λe + u}
the set of eigenvectors of T .

Theorem 7.6 (Compare with [Bru73, Th. 2]). Let T : R n → R n be a Shapley operator. Assume that the ergodic equation (2.5) is solvable. Then, the set of eigenvectors E(T ) is a retract of R n by a sup-norm nonexpansive map, meaning that E(T ) = π(R n ) where π is a sup-norm nonexpansive self-map of R n such that π = π 2 . In particular, E(T ) is arcwise connected.

Proof. The result of Bruck [Bru73,Th. 2] shows that, under some compactness conditions, the fixed-point set of a nonexpansive self-map of a Banach space is a retract of the whole space by a nonexpansive map.

Assume now that T is a Shapley operator and admits an eigenvector for the eigenvalue λ. Then, the eigenspace E(T ) coincides with the fixed-point set of the map x → -λe + T (x). The latter map is sup-norm nonexpansive and satisfies the conditions of [Bru73, Th. 2], and so E(T ) is a nonexpansive retract of R n .

Remark 7.7. The retraction π in Theorem 7.6 can be chosen to be monotone and additively homogeneous. This can actually be shown by elementary means, following a construction in the proof of [START_REF] Gaubert | The Perron-Frobenius theorem for homogeneous, monotone functions[END_REF]Lem. 3]. Indeed, we may assume without loss of generality that λ = 0, and consider q(x) := lim k→∞ inf k T (x), which is finite because every orbit of a nonexpansive map that admits a fixed point must be bounded. Since T is monotone and continuous, we get T (q(x)) q(x), and so π(x) := lim k→∞ T k (q(x)), which is the limit of a nonincreasing and bounded sequence, exists and is finite. The map π is easily shown to be monotone and additively homogeneous and to satisfy π = π 2 .

Generic uniqueness of the eigenvector

Let us first recall some definitions. A polyhedron in R n is an intersection of finitely many halfspaces, a face of a polyhedron is an intersection of this polyhedron with a supporting half-space, and a polyhedral complex is a finite set K of polyhedra satisfying the following two properties:

(i) P ∈ K and F is a face of P implies that F ∈ K;

(ii) for all P, Q ∈ K, P ∩ Q is a face of P and Q.

A polyhedron in K is called a cell of the polyhedral complex. We refer to the textbook [START_REF] De Loera | Triangulations[END_REF] for background on polyhedral complexes. Also recall that a real map on R n is piecewise affine if, and only if, R n can be covered by a finite union of polyhedra (with nonempty interior) on which its restriction is affine. If the latter map is continuous, then the set of such polyhedra can be refined in a polyhedral complex. Finally, recall that continuous piecewise affine functions are exactly the maps whose coordinate functions can be written as a minimax over finite sets of affine functions. We now state the main result of this chapter.

Theorem 7.8. Let T : R n → R n be the Shapley operator of a finite stochastic game with perfect information. Assume that T is ergodic. Then, the space R n can be covered by a polyhedral complex such that, for any additive perturbation vector g ∈ R n in the interior of a full-dimensional cell, g + T has a unique eigenvector, up to an additive constant. In particular, the set of perturbation vectors g for which g + T has more than one eigenvector, up to an additive constant, is included in the finite union of subspaces of codimension at least 1.

Remark 7.9. This perturbation theorem bears some conceptual similarity with results of weak KAM theory -we refer to the monograph by Fathi [START_REF] Fathi | Weak KAM theorem in Lagrangian dynamics[END_REF] for more information. The latter theory deals with a class of one-player deterministic games with continuous time and space. In this setting, the eigenvector u and the eigenvalue λ are solution of an ergodic Hamilton-Jacobi PDE H(x, D x u) = λ where the Hamiltonian (x, p) → H(x, p) is convex in the adjoint variable p. One may consider the perturbation of a Hamiltonian by a potential, which amounts to replacing H(x, p) by H(x, p)+V (x) for some function V . This is similar to the replacement of the Shapley operator T by the perturbed Shapley operator g + T in Theorem 7.8. As observed by Figalli and Rifford in [START_REF] Figalli | Aubry sets, Hamilton-Jacobi equations, and the Mañé conjecture[END_REF]Th. 4.2], it follows from weak KAM theory results that under some assumptions, the solution u of V (x) + H(u, D x u) = λ is unique up to an additive constant for a generic function V . Theorem 7.8 shows that an analogous property is valid for finite twoplayer zero-sum stochastic games. We note however that Theorem 7.8 does not extend easily to the case of PDE, since zero-sum games correspond to Hamilton-Jacobi PDE with a nonconvex Hamiltonian (to which current weak KAM methods do not apply).

Proof. Let T be the Shapley operator of a finite stochastic game with perfect information Γ which is assumed to be ergodic. Let σ ∈ S s be a policy of player MIN. We define the real map

λ σ (•) on R n by λ σ (g) := max{ m, (g + r στ ) | τ ∈ T σ s , m ∈ M * (P στ )} , (7.7)
where M * (P ) denotes the set of extreme points of the convex polytope M(P ), that is, the set of invariant probability measures the support of which are final classes of the stochastic matrix P . The fact that M * (P στ ) is a set of probability measures yields that λ σ is monotone and additively homogeneous, hence sup-norm nonexpansive (and continuous). Furthermore, since the set of policies T σ s of player MAX is finite, as well as all the sets M * (P στ ), then the map λ σ is piecewise affine. We now define the polyhedral complex C σ covering R n , the fulldimensional cells of which are precisely the maximal polyhedra on which the piecewise affine map λ σ coincides with a unique affine map. Therefore, if Q is a cell of C σ with full dimension, then there is a unique vector m ∈ R n and a unique scalar d ∈ R such that λ σ (g) = m, g + d in the interior of Q. Observe that m must be a stochastic vector since the map g → λ σ (g) is monotone and additively homogeneous.

We claim that if a vector g is in the interior of Q, then the set of eigenvectors of the reduced one-player Shapley operator F := g + T σ is either empty or reduced to a line. To see this, it suffices to observe that the measure m attaining the maximum in (7.7) is unique for all g in the interior of Q and independent of the choice of g in this interior, because g → λ σ (g) is affine on the interior of Q and m must coincide with the linear part of this affine map. We deduce from Corollary 7.4 that E(F ) is either empty or reduced to a line of direction e.

Consider now the polyhedral complex C obtained as the refinement of all the complexes C σ . This complex covers R n . Let g be a perturbation vector in the interior of a full-dimensional cell of C. Since the game Γ is ergodic, E(g + T ) is not empty. Let u be an eigenvector of g + T . According to (7.2), there is a policy σ ∈ S s of player MIN such that g + T (u) = g + T σ (u). Hence u is also an eigenvector of g + T σ . So, there is a finite family Σ * of S s such that E(g + T ) = σ∈Σ * E(g+T σ ). Moreover, we have proved that for any policy σ ∈ Σ * , the eigenspace E(g+T σ ) is reduced to a line. Thus, E(g+T ) is composed of a finite union of lines which all have the same direction, namely e. Consider the hyperplane orthogonal to the unit vector, H := {x ∈ R n | x, e = 0}, and let π denote the orthogonal projection on H. Then, π(E(g + T )) = E(g + T ) ∩ H is finite. However, by Theorem 7.6, E(g + T ) is connected. Then, the set π(E(g + T )) is also connected, and since it is finite, it must be reduced to a point. It follows that g + T has a unique eigenvector, up to an additive constant.

Example 7.10. Consider the following Shapley operator defined on R 3 -we use ∧ and ∨ instead of min and max, respectively, and we recall that the addition has precedence over the latter operators:

T (x) =   1 2 (x 1 + x 3 ) ∧ 1 + 1 2 (x 1 + x 2 ) 2 + 1 2 (x 1 + x 3 ) ∧ 1 + 1 2 (x 1 + x 2 ) ∨ -2 + x 3 3 + 1 2 (x 1 + x 3 ) ∨ 1 + x 3   .
It can be proved, using Theorem 4.6, that T is ergodic. Figure 7.1 shows the intersection of the hyperplane {g ∈ R 3 | g 3 = 0} with the polyhedral complex introduced in Theorem 7.8. Here, for each vector g in the interior of a full-dimensional polyhedron, g + T has a unique eigenvector up to an additive constant. Let us detail what happens in the neighborhood of g = 0, point in which g + T fails to have a unique eigenvector. Note that in the neighborhood of g = 0, the eigenvalue of g + T remains equal to 1.

-If g 1 + g 2 = 0, the eigenvectors of g + T are defined by

x 1 = x 2 + 2g 1 , -3 + g 2 x 2 -x 3 -2 -g 1 .
-If g 1 + g 2 > 0, the unique eigenvector, up to an additive constant, is

(-2 + 2g 1 , -2 + 2g 1 + 2g 2 , 0) .
-If g 1 + g 2 < 0, the unique eigenvector, up to an additive constant, is

(-3 + 2g 1 + g 2 , -3 + g 2 , 0) .

Application to policy iteration

In this section, we apply the previous result to show that policy iteration combined with a perturbation scheme can solve degenerate instances of stochastic games.

Homan-Karp policy iteration

Let us recall the notation of Section 7.3. We denote by Γ a finite stochastic game with perfect information. The (finite) set of policies of player MIN is denoted by S s , an element of S s being a map σ :

[n] → A such that σ(i) ∈ A i for every state i ∈ [n]
. The (finite) set of policies of player MAX when policy σ ∈ S s of MIN is fixed is denoted by T σ s , an element of T σ s being a map τ : [n] → B such that τ (i) ∈ B i,σ(i) for every state i ∈ [n]. Finally, recall that for σ ∈ S s and τ ∈ T σ s , P στ denotes the n × n stochastic matrix whose (i, j) entry is given by

P στ ij = p(j | i, σ(i), τ (i)).
When T : R n → R n is the Shapley operator (7.1) of a finite stochastic game with perfect information, Hoffman and Karp [HK66] introduced a policy iteration algorithm, which takes the description of the game as input and returns the eigenvalue λ and an eigenvector u of T , i.e., a solution (λ, u) ∈ R × R n of the ergodic equation T (u) = λe + u. Also, optimal stationary strategies for both players can be derived from the output of the algorithm. It is convenient here to state an abstract, slightly more general, version of the Hoffman-Karp algorithm, described in terms of the operators T and T σ (Algorithm 2).

Algorithm 2: Policy iteration -compare with [START_REF] Hoffman | On nonterminating stochastic games[END_REF] input : perfect-information finite stochastic game ([n], A, B, K A , K B , r, p) with Shapley operator T . output: eigenvalue λ and eigenvector u of T .

1 initialization: select an arbitrary policy σ 0 ∈ S s 2 repeat 3 compute an eigenpair (λ k , v k ) of T σ k 4 improve the policy σ k in a conservative way: select a policy σ k+1 ∈ S s such that T (v k ) = T σ k+1 (v k ) and satisfying, for every state i ∈

[n], σ k+1 (i) = σ k (i) if T i (v k ) = T σ k i (v k ) 5 until σ k+1 = σ k 6 return λ k and v k
We assume Algorithm 2 is interpreted in exact arithmetics (the vectors v k have rational coordinates and the λ k are rational numbers). To implement Step 3, we may call any oracle able to compute the eigenvalue and an eigenvector of a one-player stochastic game. In the original approach of Hoffman and Karp, the oracle consists in applying the same policy iteration algorithm for the one-player game with fixed policy σ k . The proof of Hoffman and Karp shows that Algorithm 2 is valid under a restrictive assumption.

Theorem 7.11 (Cor. of [START_REF] Hoffman | On nonterminating stochastic games[END_REF]). Algorithm 2 terminates and is correct if for all choices of policies σ and τ of the two players, the corresponding transition matrix P στ is irreducible.

Indeed, it is easy to see that the sequence (λ k ) k of Algorithm 2 is nonincreasing, that is, λ k+1 λ k for all iterations k. The irreducibility assumption was shown to imply that the latter inequalities are always strict, which entails the finite time convergence (each policy yields a unique well defined eigenvalue, these eigenvalues constitute a decreasing sequence, and there are finitely many policies).

However, the assumption that all stochastic matrices P στ are irreducible is way too strong to guarantee that Algorithm 2 is properly posed. Indeed, to execute the algorithm, it suffices that at every iteration k the operator T σ k admits an eigenvalue and an eigenvector, which is the case in particular if for all policies σ, the graph obtained by taking the union of the edge sets of all the graphs associated with P στ for the different choices of τ is strongly connected (see Subsection 5.2). In particular, the irreducibility assumption of Hoffman and Karp is essentially never satisfied for deterministic games, whereas the condition involving the union of the edge sets is satisfied by relevant classes of deterministic games.

It should be noted that Algorithm 2 may, in general, lead to degenerate iterations, in which λ k+1 = λ k . As shown by an example in [ACTDG12, Sec. 6], this may lead the algorithm to cycle when the bias vector is not unique. This difficulty was solved first in the deterministic framework in [START_REF] Cochet-Terrasson | A constructive fixed point theorem for min-max functions[END_REF], where it was shown that cycling can be avoided by enforcing a special choice of the bias vector, obtained by a nonlinear projection operation. This approach was then extended to the stochastic framework in [START_REF] Cochet-Terrasson | A policy iteration algorithm for zero-sum stochastic games with mean payoff[END_REF][START_REF] Akian | Policy iteration algorithm for zero-sum multichain stochastic games with mean payoff and perfect information[END_REF]. As a special case of these results, we get that policy iteration is correct and does terminate under much milder conditions than in Theorem 7.11. Theorem 7.12 (Cor. of [CTG06, Th. 7]). Algorithm 2 terminates and is correct if for each choice of policy σ of player MIN, the operator T σ has an eigenvalue and a unique eigenvector, up to an additive constant.

We next show that the conditions of Theorem 7.12 are satisfied for generic payments, and conclude that nongeneric instances can still be solved by the Hoffman-Karp algorithm after an effective perturbation of the input.

Generic termination of policy iteration

Let T : R n → R n be the Shapley operator of a finite stochastic game with perfect information. The following assumption guarantees that Algorithm 2 is well posed for any additive perturbation of T . Assumption 7.D. For any policy σ ∈ S s of player MIN, the one-player Shapley operator T σ is ergodic, i.e., g + T σ has an eigenvalue for all perturbation vectors g ∈ R n .

This assumption is much milder than the original assumption of Hoffman and Karp, requiring all the transition matrices P στ to be irreducible (see Theorem 7.11). Moreover, we shall see in the next subsection that one can always transform a game (in polynomial time) by a "big M " trick in such a way that Assumption 7.D becomes satisfied.

By using the arguments of Section 7.3, we now show that under Assumption 7.D, Algorithm 2 terminates for a generic perturbation of the payments. Theorem 7.13. Let T : R n → R n be the Shapley operator of a finite stochastic game with perfect information satisfying Assumption 7.D. Then, the space R n can be covered by a polyhedral complex such that for each perturbation vector g ∈ R n in the interior of a full-dimensional cell, Algorithm 2 terminates after a finite number of steps and gives an eigenpair of g + T .

Proof. Consider the same complex C as in Section 7.3 and let g be a perturbation vector in the interior of a full-dimensional cell of C. It follows from the proof of Theorem 7.8 that for any policy σ ∈ S s , the eigenvector of g + T σ , which exists according to Assumption 7.D, is unique up to an additive constant. Hence, at each step k of Algorithm 2, the bias vector v k of g + T σ k is unique up to an additive constant. The conclusion follows from Theorem 7.12.

We next provide an explicit perturbation g, depending on a parameter ε, for which the policy iteration algorithm applied to g +T is valid. We shall see in the next subsection that ε can be instantiated with a polynomial number of bits, in such a way that the original unperturbed problem is solved. Proposition 7.14. Let T : R n → R n be the Shapley operator of a finite stochastic game with perfect information. Then, there exists ε 0 > 0 such that all the perturbation vectors g ε := (ε, ε 2 , . . . , ε n ) with 0 < ε < ε 0 are in the interior of the same full-dimensional cell of the polyhedral complex of Theorem 7.13.

Proof. The cells of the polyhedral complex C introduced in Theorem 7.8 that do not have a full dimension are included in an arrangement of finitely many hyperplanes. The real curve ε → g ε = (ε, . . . , ε n ) cannot cross a given hyperplane in this arrangement more than n times (otherwise, a polynomial of degree n would have strictly more than n roots). We deduce that there is a value ε 0 > 0 such that the restriction of the curve ε → g ε to the open interval (0, ε 0 ) crosses no hyperplane of the arrangement. Therefore, it must stay in the interior of a fulldimensional cell of the complex C.

The following result is a refinement of the previous one.

Proposition 7.15. Let T : R n → R n be the Shapley operator of a finite stochastic game with perfect information. Then, there exist ε 1 > 0 and policies σ ∈ S s and τ ∈ T σ s such that for all ε ∈ [0, ε 1 ], the upper mean payoff of g ε + T is given by

χ(g ε + T ) = m στ , g ε + r στ ,
where m στ is an invariant probability measure of the stochastic matrix P στ . Proof. Recall that the upper mean payoff of g + T is given by

χ(g + T ) = min σ∈Ss χ σ (g) where χ σ (g) = max{ m, g + r στ | τ ∈ T σ s , m ∈ M * (P στ )} .
By construction of the polyhedral complex C of Theorem 7.8, in the interior of a full-dimensional cell, each piecewise affine map χ σ coincides with a unique affine map, but g → χ(g + T ) need not be affine. Hence, we can refine the complex C into a complex C such that the latter piecewise affine map also coincides with a unique affine map on each full-dimensional cell. The exact same proof as Proposition 7.14 leads to the conclusion.

It readily follows that solving the game with Shapley operator g ε + T for ε small enough entails a solution of the original game. Proposition 7.16. If T satisfies Assumption 7.D, then Algorithm 2 terminates for any input g ε +T with ε ∈ (0, ε 1 ). Furthermore, any policy σ which is optimal for g ε +T , meaning that λ(g ε +T ) = λ(g ε +T σ ), is also optimal for T , i.e., λ(T ) = λ(T σ ).

Complexity issues

In this subsection, we show that computing the upper mean payoff of a Shapley operator (a fortiori the eigenvalue if it exists) is polynomial-time reducible to the computation of the eigenvalue of a Shapley operator for which Algorithm 2 terminates. This fact is a direct consequence of Theorem 7.22 below. To do so, we shall need to give explicit bounds on the perturbation parameter ε. Lemma 7.21. (v, w) is an eigenvector of T M with eigenvalue λ if, and only if, v is an eigenvector of T • R M with eigenvalue 2λ and w = R M (v) -λe.

The operator T M is the dynamic programming operator of a game, denoted by Γ M , with state space [2n]. In each state i ∈ [n], the actions, the payment and transition functions are the same as in Γ, except that the next state is labeled by an element of {n + 1, . . . , 2n} instead of [n]. Moreover, in each state i ∈ {n + 1, . . . , 2n}, player MIN has only one possible action, while player MAX chooses the next state j among [n] with a cost M if i -j = n. In particular, the policies of MIN in the two games Γ and Γ M are in one-to-one correspondence, and to simplify the presentation, we shall use the same notation for these policies. Hence, we shall write (T M ) σ (x, y) = (T σ (y), R M (x)) for such a policy σ.

We saw in Lemma 7.17 that the operator T • R M has an eigenvalue. The same is true for the operator T M by Lemma 7.21, as well as for any operator (T M ) σ . Thus, T M satisfies Assumption 7.D. We know that for ε > 0 small enough, the perturbed operator g ε + T • R M has a unique bias vector, up to an additive constant, where g ε = (ε, . . . , ε n ). This leads to considering, for ε > 0, T M,ε := (g ε , 0) + T M .

Theorem 7.22. Let Γ be a perfect-information finite stochastic game whose transition payments and probabilities are rational numbers with numerators and denominators bounded by an integer D 2. Let T : R n → R n be the Shapley operator of Γ.

If M > 2(2n) n/2 D 2n 2 +1 and 0 < ε < 1 n n D 2n(n+1) ,
then the upper mean payoff of T can be recovered from T M,ε = (g ε , 0) + T M , in the sense that for any policy σ of player MIN such λ(T M,ε ) = λ((T M,ε ) σ ), we have χ(T ) = χ(T σ ). Furthermore, such an optimal policy can be obtained by applying Algorithm 2 with input T M,ε .

Proof. Let g ∈ [0, 1] n , and fix a policy σ of MIN. In the game Γ M , consider a policy of MAX such that, when in state i ∈ [n], he chooses some b i ∈ B i,σ(i) and when in state i ∈ {n + 1, . . . , 2n}, he chooses the next state to be some j(i) ∈ [n]. Then, the transition matrix associated with that choice of policy is the following 2n × 2n block matrix:

0 P στ Q 0 , (7.9)
where τ ∈ T σ s is the policy of MAX in the game Γ defined by τ (i) = b i for each state i, and where Q is the n × n stochastic matrix whose coefficients are Q ij = 1 for j = j(i + n) and 0 otherwise.

Let (m, m ) ∈ R n × R n be an invariant probability measure of the stochastic matrix (7.9). The vectors m and m satisfy

mP στ = m , m Q = m ,
m, e + m , e = 1 . (7.10)

We are interested in the eigenvalue of the perturbed one-player Shapley operator (g, 0)+(T M ) σ . Hence, following formula (7.6), we consider the quantity

γ := m, g + r στ -M 1 i n Q ii =0 m i .
If for every index i in the support of m , we have Q ii = 1, then we deduce from the second equality in (7.10) that m = m. This yields that 2m is an invariant probability measure of P στ , and that γ = m, g + r στ 1 2 χ(g + T σ ) .

Note that the equality is attained in the above inequality for some policy τ and some invariant measure m. If there is an index i in the support of m such that Q ii = 0, then we have

γ m, g + r στ -M m i 1 + max i,a,b r(i, a, b) -K 1 M ,
where K 1 is a positive constant such that K 1 < m i . Note that K 1 can be chosen independently of M and of the choice of policies. Then, taking M > M 0

:= (K 1 ) -1 (1 + (3/2) r ∞ ), we obtain that γ < 1 2 min i,a,b r(i, a, b) 1 2 χ(g + T σ ) .
Thus, we have proved that, for all policies σ of MIN and for all g ∈ [0, 1] n , we have

λ (g, 0) + (T M ) σ = 1 2 χ(g + T σ ) ,
as soon as M > M 0 . In particular, the choice of the parameter ε such that T M,ε is a generic instance only relies on T (Proposition 7.15). We now fix some M > M 0 . Consider, for policies σ, σ of MIN and τ, τ of MAX, two distinct pairs (m, d) = (m , d ), where m ∈ M * (P στ ), m ∈ M * (P σ τ ), d := m, r στ and d := m , r σ τ . We need to compare the affine maps g → m, g + d and g → m , g + d along the curve ε → g ε with ε ∈ (0, 1).

Assume first that d = d . Then m = m and we can select the smallest index i such that m i = m i . Note that since m and m are stochastic vectors, we necessarily have i < n and we also have the existence of another index j such that i < j n and m j = m j . Without loss of generality, we may assume that m i -m i > 0. Let K 2 ∈ R be such that 0 < K 2 < m i -m i . Then, for any positive parameter ε < K 2 n -1 , we have

( m, g ε + d) -( m , g ε + d ) = (m i -m i )ε i + i<j n (m j -m j )ε j > K 2 ε i -nε i+1 = ε i (K 2 -nε) > 0 .
Assume now that d = d , say d > d , and let K 3 ∈ R be such that 0 < K 3 < d -d . Then, for any positive parameter ε < K 3 , we have

( m, g ε + d) -( m , g ε + d ) > ε n -ε + K 3 > 0 .
Note that we can choose the positive constants K 2 and K 3 independently of σ, σ , τ, τ , m and m . Hence, the above arguments show that the set of polynomial functions ε → m, g ε + r στ with σ ∈ S s , τ ∈ T σ s and m ∈ M * (P στ ), is totally ordered if ε is restricted to the interval (0, min{K 2 n -1 , K 3 }). Thus, the parameter ε 1 of Proposition 7.15 may be taken equal to min{K 2 n -1 , K 3 }.

To complete the proof, we next explain how to instantiate the constants K 1 to K 3 . Let us start with K 2 . It is a lower bound on the absolute values of the differences between two distinct entries (with same index) of invariant probability measures associated with the transition matrices of Γ. These differences are of the form |p 1 /q 1 -p 2 /q 2 | 1/(q 1 q 2 ), where p 1 , p 2 , q 1 , q 2 are integers. By Lemma 7.20, we know that q 1 , q 2 n n/2 D n 2 , and so

K 2 1 n n D 2n 2 .
Likewise, K 3 is a lower bound on the absolute values of the differences between two distinct scalar products m, r στ . Let m ∈ M(P στ ). It follows from the proof of Lemma 7.20 that the ith entry of m can be written as m i = p i /q where p i is an integer and q n n/2 D n 2 is an integer independent of i. Since every entry of r στ has a denominator at most D, it follows that m, r στ is a rational number with denominator at most n n/2 D n 2 D n . Therefore, the difference between two distinct values of m, r στ is at least n n D 2n 2 D 2n , and so

min{K 2 n -1 , K 3 } 1 n n D 2n 2 min 1 n , 1 D 2n = 1 n n D 2n(n+1) .
Finally the constant K 1 is a lower bound for the positive entries of the invariant probability measures of the transition matrices (7.9) arising in the game Γ M . A direct application of Lemma 7.20 provides the following bound:

K 1 1 (2n) n D 4n 2 .
However, the matrices (7.9) have a particular structure, and an invariant measure m solves the linear system

     I -Q -P I m = 0 e m = 0
where P is a transition matrix of the game Γ and Q is a stochastic matrix with entries either 0 or 1. Repeating the same arguments used to prove Lemma 7.20, we obtain that the positive entries of m have a denominator dividing an integer bounded by

(n -1)(D n ) 2 + 2 n 2 (D n ) 2 + 2 n-1 2 2 1 2 (2n) n/2 D 2n 2 .
Hence,

K 1 1 (2n) n/2 D 2n 2 , and M 0 (1 + (3/2)D) (2n) n/2 D 2n 2 .
An important special case to which the method of Theorem 7.22 can be applied concerns deterministic mean-payoff games [START_REF] Gurvich | Cyclic games and finding minimax mean cycles in digraphs[END_REF][START_REF] Zwick | The complexity of mean payoff games on graphs[END_REF]. The input of such games can be described, as in [START_REF] Akian | Tropical polyhedra are equivalent to mean payoff games[END_REF], by means of two matrices A, B ∈ (Z ∪ {-∞}) m×n . The corresponding Shapley operator can be written as

T i (x) = min j∈[m] -A ji + max k∈[n] (B jk + x k ) , ∀x ∈ R n , ∀i ∈ [n] . ( 7
+ T • R M ) = λ(g ε + T σ • R M )
, yields an optimal policy for the upper mean payoff of T , that is, χ(T ) = χ(T σ ).

Proof. We adapt the proof of Theorem 7.22 to the case of deterministic transition matrices. In that special case, every invariant measure is uniform, with positive entries bounded below by 1/n if the state space has cardinality n. Hence, the constant K 1 , arising as a lower bound for the positive entries of the invariant probability measures of the transition matrices in Γ M , is such that K 1 1/(2n), and then

M 0 (2n)(1 + (3/2)D) 4nD .
The constant K 2 , which is a lower bound on the absolute values of the differences between two distinct entries of invariant measures of transition matrices in Γ, is such that K 2 1/n 2 . As for K 3 , it is a lower bound on the absolute values of the differences between two distinct values of m, r στ . Since the payments are integers, every scalar product m, r στ is a rational number whose denominator divides the denominator of the positive entries of m. Hence, K 3 1/n 2 , and the parameter ε must be lower than min{K 2 n -1 , K 3 } 1/n 3 . Remark 7.24. One step in Algorithm 2 consists in computing an eigenpair (λ k , v k ) of the reduced Shapley operator T σ k obtained by fixing the strategy σ k of MIN. This is a simpler problem which can be solved by several known methods. We may apply, for instance, a similar policy iteration algorithm to T σ k , iterating this time in the space of policies τ of MAX. In this way, for each choice of τ , we arrive at an operator of the form T σ k ,τ (x) = g + P x, where P is a stochastic matrix which cannot in general be assumed to be irreducible. However, for one-player problems, a classical version of policy iteration, the multichain policy iteration introduced by Howard [How60] and Denardo and Fox [START_REF] Denardo | Multichain Markov renewal programs[END_REF], does allow one to determine (λ k , v k ) without genericity conditions. Moreover, in the special case of deterministic games, the vector v k is known to be a tropical eigenvector and λ k a tropical eigenvalue. The tropical eigenpair can be computed by direct combinatorial algorithms, see e.g. the discussion in [START_REF] Cochet-Terrasson | A constructive fixed point theorem for min-max functions[END_REF].

Remark 7.25. Theorem 7.23 should be compared with the other known perturbation scheme, relying on vanishing discount. The latter method requires the computation of a fixed point of the operator x → T (αx) for 0 < α < 1 sufficiently close to one. It is known that, for deterministic mean-payoff games, if the discount factor α is chosen so that α > 1 -1 4(n + m) 3 D , where D denotes the maximal absolute value of a finite entry A ij or B ij , then, the solution of the mean-payoff problem can be derived from the solution of the discounted problem, see [START_REF] Zwick | The complexity of mean payoff games on graphs[END_REF]Sec. 5]. The latter can be obtained by policy iteration (which terminates without any nondegeneracy conditions in the discounted case). The present perturbation scheme requires shorter rational numbers. Note in particular that applying Algorithm 2 to the map g ε + T requires solving a linear systems in which the matrix is independent of ε and well conditioned, whereas vanishing discount requires the inversion of a matrix which becomes singular as α → 1. If the vanishing discount approach is interpreted in exact arithmetics, it leads to a blow up of the bitsize of the intermediate data as α → 1, whereas if this is interpreted in floating point arithmetics, it may lead to numerical instabilities or overflows.

Remark 7.26. The present approach allows one to compute the upper mean payoff, i.e., the maximum of the mean payoff over all initial states. This leads to no loss of expressivity since it follows from known reductions that this problem is polynomial-time equivalent to solving a mean-payoff game in which the initial state is fixed (combine [AGS16, Cor. C.3] with the reductions in [START_REF] Andersson | The complexity of solving stochastic games on graphs[END_REF]). An alternative route to compute the mean payoff of any given initial state, avoiding the use of such reductions, would be to extend the present perturbation scheme to the "multichain" version of policy iteration, discussed in [START_REF] Cochet-Terrasson | A policy iteration algorithm for zero-sum stochastic games with mean payoff[END_REF][START_REF] Akian | Policy iteration algorithm for zero-sum multichain stochastic games with mean payoff and perfect information[END_REF].

CHAPTER 8

Accretive operator approach to generic uniqueness of the bias vector

The results persented in this chapter have been announced, in a milder version, in the MTNS conference proceedings [START_REF] Hochart | An accretive operator approach to ergodic problems for zero-sum games[END_REF].

8.1 Generic uniqueness of the xed point of nonexpansive maps 8.1.1 Uniqueness and continuity of the xed-point map Let (X , • ) be a real vector space of finite dimension, endowed with a given norm. In this section, we fix an operator T : X → X , nonexpansive with respect to the norm of X , and we assume that there exists an open subset V ⊂ X such that g + T has at least one fixed point for all g ∈ V.

Let us introduce the set-valued map FP : V ⇒ X defined by FP(g) := {x ∈ X | x -T (x) = g} , g ∈ V .

Put in words, the mapping FP sends each vector g ∈ V to the set of fixed points of g + T .

Observe that the inverse map of FP is

FP -1 = Id -T ,
so that it is coaccretive by Lemma 4.19. Since we have assumed that dom(FP) = V, then we get from the local boundedness of coaccretive maps (Proposition 4.13) the following.

Lemma 8.1. Let (X , • ) be a finite-dimensional real vector space, and let T : X → X be a nonexpansive map such that g + T has a fixed point for all vectors g in an open subset V ⊂ X . Then, the set-valued map FP : V ⇒ X has compact values and is upper semicontinuous.

Proof. It readily follows from Proposition 4.13 that FP(g) is bounded for any g ∈ V. Furthermore, since g + T is continuous, FP(g) is closed, hence compact.

It is now possible to characterize sequentially the upper-semicontinuity of FP. Let (g k ) k∈N be a sequence in V converging to some vector g ∈ V and for all integers k, let x k ∈ FP(g k ).

We know from Proposition 4.13 that FP is locally bounded at g. Hence the sequence (x k ) k∈N is bounded, and therefore has an accumulation point x. Since x k -T (x k ) = g k for all k, the continuity of T yields that x -T (x) = g, that is, x ∈ FP(g). This proves that FP is upper semicontinuous at g. We shall need the following technical result, which is a variant of the Hahn-Banach theorem. Lemma 8.2. Let J be the duality mapping on (X , • ), and let x be any vector in X . Then, x = 0 ⇐⇒ ∃y ∈ X \ {0} , ∀x * ∈ J(x) , y, x * > 0 .

Proof. Let x ∈ X \ {0}. We know that J(x) is a compact convex subset of X * . Furthermore, 0 / ∈ J(x). Hence, according to the Hahn-Banach theorem, there exists an affine hyperplane of X * strongly separating the two compact convex subsets J(x) and {0}, i.e., there exists some vector y ∈ X \ {0} and a constant ε > 0 such that for all x * ∈ J(x) we have y, x * ε 0. Conversely, if x = 0, then J(x) = {0} and so, for all y ∈ X we have y, x * = 0 with x * = 0 ∈ J(x).

We now state the main result of this section.

Theorem 8.3. Let (X , • ) be a finite-dimensional real vector space, and let T : X → X be a nonexpansive map such that g + T has a fixed point for all vectors g in an open subset V ⊂ X . Then, the set-valued map FP : V ⇒ X is continuous at point g ∈ X if, and only if, FP(g) is a singleton.

Proof. Suppose first that the map FP is single-valued at g. We have shown in Lemma 8.1 that FP is upper semicontinuous at g. It readily follows from the definition (Subsection 4.3.1) that FP is also lower semicontinuous at g, since it is single-valued. Hence, FP is continuous at g.

Conversely, suppose that FP is continuous at point g ∈ V and let x and x be two vectors in FP(g), i.e., two fixed points of g + T . Choose y ∈ X \ {0}, and for every positive integer k, let g k := g -k -1 y. We may assume, without loss of generality, that g k is in V for all k. Since FP is continuous at g, hence lower semicontinuous, there exists a sequence of elements x k ∈ FP(g k ) converging to x. Furthermore, we know from Lemma 4.19 that the map FP = (Id -T ) -1 is coaccretive. Hence, for every integer k > 0, there is a point x * k ∈ J(x k -x ) such that g k -g, x * k 0 , which yields y, x * k 0 .

Since the duality mapping J is upper semicontinuous (Proposition 4.8) and with compact values, then the sequence of elements x * k ∈ J(x k -x ) has an accumulation point x * ∈ J(x -x ), which satisfies y, x * 0 .

Thus, we have proved that for any point y ∈ X \ {0}, there exists an element x * ∈ J(x -x ) such that y, x * 0. We deduce from Lemma 8.2 that x -x = 0, and consequently that FP(g) is a singleton.

Generic continuity

Under mild conditions, the continuity of any upper semicontinuous set-valued map A is a generic property. By generic, we mean that the set of points where the map A is continuous is a residual. Recall that a residual of any space Y is a countable intersection of dense open subsets of Y. In particular, if Y is a Baire space, then a residual is dense. Theorem 8.4 (see [START_REF] Aubin | Modern Birkhäuser Classics[END_REF]Th. 1.4.13]). Let A be a set-valued map from a complete metric space X to a complete separable metric space Y. If A is upper semicontinuous on X , then it is continuous on a residual of X .

Note that Theorem 8.4 states that the set of points where an upper semicontinous set-valued map is continuous (only) contains a residual. We can in fact be more specific about the characterization of elements where the fixed-point map FP is single-valued. Theorem 8.5. Let (X , • ) be a finite-dimensional real vector space, and let T : X → X be a nonexpansive map. Let U be an open subset of X such that g + T has a fixed point for all vectors g in a neighborhood of U. Then, the set of vectors g ∈ U for which g + T has a unique fixed point is a residual of U. In particular, it is dense in U.

Proof. According to Lemma 8.1, the set-valued map FP is upper semicontinuous on a neighborhood of U. Hence, its restriction to U (which is also upper semicontinuous) is continuous on a dense subset (Theorem 8.4). Consider the intersection of this dense subset with U, which is still dense in U. According to Theorem 8.3, at any point g in the latter subset, g + T has a unique fixed point. So there is a dense subset of U such that for each point g in this subset, g + T has a unique fixed point. It remains to show that the set of points in U for which FP is a singleton is a countable intersection of open subsets.

There is no loss of generality in assuming that 0 ∈ U. Let (y i ) i 1 be a countable dense subset of U. For all positive integers i and k, let F i,k := {g ∈ U | ∃x, x ∈ FP(g), ∃x * ∈ J(x -x ), y i , x * 1/k} .

We show that FP is single-valued at g if, and only if, g / ∈ i,k∈N F i,k . First, if FP(g) is a singleton, then g cannot be in any set F i,k . Conversely, if it is not a singleton, we can find two distinct fixed points of g + T , x, x , and according to Lemma 8.2, there exists some vector y ∈ X \ {0} such that y, x * > 0 for all x * ∈ J(x -x ). Since U is an open set containing 0, we may assume, up to a scaling, that y is in U. Therefore, y can be approximated by some point y i such that the inequality y i , x * > 0 still holds for all x * ∈ J(x -x ). Hence g ∈ F i,k for some integer k large enough.

We next show that all the sets F i,k are closed, which will prove that FP is single-valued on a residual of U. Fix the integers i, k 1 and let (g ) be a sequence in F i,k converging to some vector g. For every integer , there exist x , x ∈ FP(g ) and x * ∈ J(x -x ) such that y i , x * 1/k. Since the map FP is locally bounded at g (Proposition 4.13), we may suppose, up to an extraction, that the sequences (x ) and (x ) converge to some points x and x , respectively. By continuity of T , it is readily seen that x, x ∈ FP(g). Furthermore, since the duality mapping J has compact values and is upper semicontinuous, the sequence (x * ) has an accumulation point x * in J(x -x ), which satisfies y i , x * 1/k. Thus g ∈ F i,k .

Application to nonlinear Perron-Frobenius theory

We apply the main result of the previous section (Theorem 8.5) to the case of Shapley operators arising from ergodic stochastic games.

Theorem 8.6. Let T : R n → R n be an ergodic Shapley operator. Then, the set of vectors g for which g + T has a unique bias vector, up to an additive constant, is a residual of R n .

Proof. We apply Theorem 8.5 to the quotiented self-map [T ] of the additive projective space TP n , endowed with the norm q H . Thus, there exists a residual of TP n , denoted by R, characterizing the set of equivalence classes [g] for which the quotiented map [g] + [T ] = [g + T ] has a unique fixed point. Hence, the operator g + T has a unique bias vector up to an additive constant if, and only if, the equivalence class of g is in this residual.

Let R be the preimage in R n of R by the canonical map R n → TP n . The latter quotient map is continuous, since the preimage of any open ball, that is, any set {x ∈ R n | x -g H < ρ} with ρ > 0 and g ∈ R n , is open. Hence, it follows that R is also a countable intersection of open subsets. Moreover, for any vector g ∈ R n and any real number ε > 0, there exists a vector h ∈ R such that q H ([g] -[h]) = g -h H < ε, since the residual R is dense in TP n . The vector h is defined up to an additive constant, so we may assume that min i∈[n] (g i -h i ) = 0 , in which case g -h H is equal to the sup-norm of g -h. Therefore, for any vector in R n we can find a vector in R arbitrarily close. This shows that R is dense in R n . CHAPTER 9

Conclusion and perspectives

We now briefly summarize our main contributions and point out some open problems.

In Chapters 4 to 6, we have introduced a notion of ergodicity for zero-sum stochastic games with finite state space, which recover the finite Markov chain case. We have also given several characterizations for this property to hold, in particular in terms of hypergraph reachability. To that purpose, we have followed the operator approach by applying techniques from nonlinear Perron-Frobenius theory to the study of Shapley operators. The perspectives of this work follow naturally two paths, one in relation with game-theoretic aspects, and the other with nonlinear Perron-Frobenius theory.

Regarding nonlinear Perron-Frobenius theory, we have given a condition that guarantees the existence of a fixed point for any additive perturbation of a nonexpansive map T (Corollary 4.20) for any norm in a finite-dimensional vector space. This condition, established in the additive framework, requires that all the spaces {x | x -T (x) α} are bounded. A natural question is to know if a similar result holds in the multiplicative framework. More precisely, let us consider a symmetric cone K of a Euclidean space, i.e., a self-dual open convex cone, the automorphism group G of which acts transitively on it. Let f : K → K be an order-preserving homogeneous map, such that the self-map of K sending x to g(f (x)) has a fixed point for all automorphisms g ∈ G. Does this imply that all the spaces D α (f ) := {x ∈ K | d H (x, f (x)) α} are bounded in Hilbert's projective metric d H arising from K? In particular, is this true when K is the cone of positive semi-definite matrices? Another problem, following the results in Chapter 6, is to find a combinatorial characterization, similar to the one in Theorem 6.2, of the boundedness of all spaces D α (f ) for any cone K. When the cone K is simplicial, this problem should reduce to the case of the standard nonnegative cone of R n treated in this thesis. When K is polyhedral but not simplicial, we expect that the characterization involves the asymptotic behavior of f along the faces of K.

As for game theory problems, it would be interesting to extend the characterization of ergodicity to stochastic games with infinite state space. It would allow in particular to broaden the latter notion to other classes of zero-sum repeated games, including games with incomplete information. Indeed, a wide range of repeated games have a recursive structure, expressing itself in a Shapley operator, see [START_REF] Sorin | Symmetric incomplete information games as stochastic games[END_REF] and references therein. It is then possible to reduce these games to stochastic games, the state space of which is a probability space over the set of un-known parameters.

A first step to solve the above problem could be to generalize the surjectivity conditions for accretive maps (Theorem 4.16) to the case of infinite-dimensional Banach spaces. However, this generalization would most likely require strong topological assumptions (reflexive Banach space, fixed-point property) which might not be compatible with our framework (say space of continuous or bounded functions on a compact set, endowed with Hilbert's seminorm). Another idea would be to start directly from the formulation of ergodicity in terms of dominions and discretize the infinite state space in order to approximate the original game with a simpler one with finite state space.

With continuous time and space (i.e., differential games), the problem of the asymptotic behavior of the value in finite horizon, known as ergodic problem for Hamilton-Jacobi equations, has attracted much attention lately, see [START_REF] Bardi | On differential games with long-time-average cost[END_REF][START_REF] Alvarez | Ergodicity, stabilization, and singular perturbations for Bellman-Isaacs equations[END_REF]. In the deterministic framework, the value u(x, t) in horizon t with initial state x is the viscosity solution of an Hamilton-Jacobi PDE ∂ t u + H(x, ∂ x u) = 0 , for some Hamiltonian H. An important question is then to understand when u(x, t)/t converges as t → +∞ to some constant λ. Note that if this property holds, λ is the unique constant c for which the Hamilton-Jacobi PDE

H(x, ∂ x v) = -c
has a viscosity solution v. Hence, the latter equation replaces the ergodic equation (2.5) when time and space are continuous. In the one-player case, i.e., when (x, p) → H(x, p) is convex in the adjoint variable p, the study of the ergodic problem is treated in the framework of weak KAM theory. However, in the two-player case (nonconvex Hamiltonian), weak KAM methods no longer apply and the problem is less well understood [START_REF] Cardaliaguet | Ergodicity of Hamilton-Jacobi equations with a noncoercive nonconvex Hamiltonian in R 2 /Z 2[END_REF]. Although the results of this thesis do not extend easily to the PDE case, we hope they could provide a good inspiration to tackle the latter issues. In particular, an operator approach is possible by considering the evolution semigroup associated with the game.

In Chapters 7 and 8, we have used tools of nonlinear Perron-Frobenius theory to establish the uniqueness, up to an additive constant, of the bias vector of an ergodic Shapley operator for a generic perturbation. We have also shown more generally that this generic property is true locally, as soon as the ergodic equation is solvable for all additive perturbations in any open subset. These results make a contribution to the more general problem of understanding the structure of the set of bias vectors of Shapley operators.

In the case of payment-free Shapley operators, the results in Chapter 5 give partial answers to the latter problem, which seems hard to tackle in full generality. The case of Shapley operators with perfect information and deterministic transitions is already interesting. When the operator is Boolean monotone (corresponding to payment-free Shapley operator with perfect information and deterministic transitions), it is possible to show that the set of bias vectors (which coincides with the set of fixed points) is uniquely determined by the Boolean fixed points (i.e., the coordinates of which are either 0 or 1). More generally, in the deterministic perfect-information framework, the set of bias vectors is a polyhedral complex, intersection between two tropical convex sets, one max-plus and the other min-plus, but it remains an open question to find a suitable description of this space.

Index

The page number refers to the definition of the indexed topic. Abstract : Zero-sum stochastic games have a recursive structure encompassed in their dynamic programming operator, so-called Shapley operator. The latter is a useful tool to study the asymptotic behavior of the average payoff per time unit. Particularly, the mean payoff exists and is independent of the initial state as soon as the ergodic equationa nonlinear eigenvalue equation involving the Shapley operator -has a solution. The solvability of the latter equation is a central question in nonlinear Perron-Frobenius theory, and the main focus of the present thesis. Several known classes of Shapley operators can be characterized by properties based entirely on the order structure or the metric structure of the space. We first extend this characterization to "payment-free" Shapley operators, that is, operators arising from games without stage payments. This is derived from a general minimax formula for functions homogeneous of degree one and nonexpansive with respect to a given weak Minkowski norm. Next, we address the problem of the solvability of the ergodic equation for all additive perturbations of the payment function. This problem extends the notion of ergodicity for finite Markov chains. With bounded payment function, this "ergodicity" property is characterized by the uniqueness, up to the addition by a constant, of the fixed point of a payment-free Shapley operator. We give a combinatorial solution in terms of hypergraphs to this problem, as well as other related problems of fixed-point existence, and we infer complexity results. Then, we use the theory of accretive operators to generalize the hypergraph condition to all Shapley operators, including ones for which the payment function is not bounded. Finally, we consider the problem of uniqueness, up to the addition by a constant, of the nonlinear eigenvector. We first show that uniqueness holds for a generic additive perturbation of the payments. Then, in the framework of perfect information and finite action spaces, we provide an additional geometric description of the perturbations for which uniqueness occurs. As an application, we obtain a perturbation scheme allowing one to solve degenerate instances of stochastic games by policy iteration.

Theorem 5. 1 .

 1 Let (A, ≺ A ) and (B, ≺ B ) be two posets, and let φ : A → B and ψ : B → A. The following are equivalent:(i) id A ≺ A ψ • φ , id B ≺ B φ • ψ , and φ, ψ are antitone; (ii) b ≺ B φ(a) ⇐⇒ a ≺ A ψ(b) , ∀a ∈ A , ∀b ∈ B ; (iii) φ is antitone and ψ(b) = max{a ∈ A | b ≺ B φ(a)}, ∀b ∈ B ; (iv) ψ is antitone and φ(a) = max{b ∈ B | a ≺ A ψ(b)}, ∀a ∈ A .Point (i) is the definition introduced by Birkhoff. The equivalence with Point (ii) is straightforward, and the proof of the equivalences with Points (iii) and (iv) may be found in [BJ72, Ch. 2] or [GHK + 80, Ch. 0, Sec. 3]. It follows from Point (iii) of Theorem 5.1 that for any antitone map φ : A → B, there is at most one map ψ : B → A such that (φ, ψ) is a Galois connection between A and B. Let us denote this map by φ := ψ. The inequalities in Point (i) yields φ • φ • φ = φ and φ • φ • φ = φ .

e

  Φ(I) F (e Φ(I) ) F (e I c ) e I c , and again, by monotonicity of F , we obtain that, for all integers k, e Φ(I) F k (e Φ(I) ) F k (e I c ) e I c . It follows that e Φ(I) u e I c and we deduce that I ⊂ arg min u and Φ(I) ⊂ arg max u. Since Φ is antitone, we readily have Φ(arg min u) ⊂ Φ(I) ⊂ arg max u.

  . The size of a hypergraph G = (N, E) is defined as size(G) := |N | + e∈E |t(e)| + |h(e)|, where |X| denotes the cardinality of any set X.

  Figure 5.1: A directed hypergraph

  the support of a measure, we necessarily have supp µ = ∅, otherwise µ(A i ) = 0. So, we may choose an action a i ∈ A i contained in supp µ. Then, it satisfies p(I c | i, a i , b) = 0 for all actions b ∈ B i . Indeed, if this is not the case, then there exists some b ∈ B i such that p(I c | i, a i , b) is bounded below by a positive constant. By continuity of a → p(I c | i, a, b), this implies that the function a → p(I c | i, a, b) is also bounded below by a positive constant on a neighborhood U of a i , and since µ(U ∩ supp µ) > 0, this is a contradiction with (5.7).

Theorem 5. 21 .

 21 Let Γ be a stochastic game with state space [n] satisfying Assumption 5.B and with Shapley operator T : R n → R n . Let F + and F -be the families of subsets of states associated with Γ. Then, for I ∈ F -and J ∈ F + we havee Φ(I) = T -ω (e I c ) , e [n]\Φ (J) = T + ω (e J ) .Proof. We only show the first equality, the second follows by duality. Let I ∈ F -. We haveT + (e I c) e I c , and using T - T + , we obtain that T -(e I c ) e I c . It follows that ( T -) ω (e I c ) is well defined and satisfies ( T -) ω (e I c ) e I c . Since T -is a Boolean map, there exists some subset L ⊂ [n] such that e L = ( T -) ω (e I c ). We need to show that L = Φ(I).

+

  s and G s the hypergraphs obtained from G + s and G - s , respectively, by identifying every node i ∈ [n] with node c(i) ∈ c([n]). It follows readily from j (i, a, b ) (i, a, b)

Figure 5

 5 Figure 5.3: Hypergraph G - s associated with F -

Corollary 5 .

 5 24.Let Γ be a perfect-information finite stochastic game with state space[n]. Let (F + , F -) and (G + s , G s ) be the families of subsets of states and the syntactic hypergraphs, respectively, associated with Γ. Then, a nonempty subsetI ⊂ [n] is a dominion of player MIN (i.e., I ∈ F + ) if, and only if, its complement in [n] is an invariant set in the hypergraph G + s relatively to [n]: reach(I c , G + s ) ∩ [n] = I c .Likewise, a nonempty subset J ⊂ [n] is a dominion of player MAX (i.e., J ∈ F -) if, and only if, its complement in[n] is an invariant set in the hypergraph G s relatively to [n]: reach(J c , G s ) ∩ [n] = J c .A characterization of ergodicity of stochastic games with perfect information follows.Corollary 5.25. A perfect-information finite stochastic game with state space [n] is ergodic if, and only if, there do not exist nonempty disjoint subsets of states that are invariant in G + s and G s , respectively, relatively to [n].

Theorem 5 .

 5 29 ([YZ04, Cor. 1]). Problem MonBoolFP is NP-complete.We continue with another necessary condition.Lemma 5.32. Let F : R → R n be the payment-free Shapley operator associated with a stochastic game satisfying Assumption 5.B, and let I ∈ F + . If Φ(I) = ∅, then F does not have any nontrivial fixed point u satisfying I ⊂ arg min u.

  [n]\I ) e [n]\I . Then, for all i ∈ I, there exists some action a ∈ A i such that p([n] \ I | i, a, b) = 0 for all actions b ∈ B i,a . Since I ⊂ I, this implies that p(j | i, a, b) = 0 for all j ∈ [n] \ I, and since this holds for all b ∈ B i,a , we deduce that a ∈ A * i . Hence, min a∈A * i max b∈B i,a j / ∈I p(j | i, a, b) = 0 , which yields F * i (e I\I ) = 0 for all i ∈ I. Therefore F * (e I\I ) e I\I , and since u I e I\I , it follows that v = (F * ) ω (u I ) (F * ) ω (e I\I ) e I\I .

S

  perfect-information finite stochastic game Γ := (S, A, B, K A , K B , 0, p), corresponding payment-free Shapley operator F : R S → R S , subset I ⊂ S. output: answer to Problem MinFP. (e I c ) = e I c or Φ F (I) = ∅ then 8 return false 9 else if Φ F • Φ F (I) = I then ← Φ F • Φ F (I), Γ ← Γ * , F ← F * Algorithm 1 solves Problem MinFP in O(n 3 |A||B|) time.

Figure 5

 5 Figure 5.5: Reduced game associated with F *

Figure 6

 6 Figure 6.1: Hypergraphs H + (left) and H -(right) associated with T

Corollary 6 .

 6 15. Let A be a nonnegative tensor of order d and dimension n. If the directed graph G(A) has a unique final class C and if reach(C, H(A)) = [n], then A has a positive eigenvalue and an eigenvector in int(R n + ).

Figure 7 . 1 :

 71 Figure 7.1: Polyhedral complex C

Titre:

  Théorie de Perron-Frobenius non-linéaire et jeux stochastiques à somme nulle avec paiement moyen.Mots clés : jeux répétés à somme nulle, jeux stochastiques à paiement moyen, contrôle ergodique, opérateurs de Shapley, applications non-expansives, opérateurs accrétifs, hypergraphes.Résumé : Les jeux stochastiques à somme nulle possèdent une structure récursive qui s'exprime dans leur opérateur de programmation dynamique, appelé opérateur de Shapley. Ce dernier permet d'étudier le comportement asymptotique de la moyenne des paiements par unité de temps. En particulier, le paiement moyen existe et ne dépend pas de l'état initial si l'équation ergodique -une équation non-linéaire aux valeurs propres faisant intervenir l'opérateur de Shapley -admet une solution. Comprendre sous quelles conditions cette équation admet une solution est un problème central de la théorie de Perron-Frobenius non-linéaire, et constitue le principal thème d'étude de cette thèse. Diverses classes connues d'opérateur de Shapley peuvent être caractérisées par des propriétés basées entièrement sur la relation d'ordre ou la structure métrique de l'espace. Nous étendons tout d'abord cette caractérisation aux opérateurs de Shapley « sans paiements », qui proviennent de jeux sans paiements instantanés. Pour cela, nous établissons une expression sous forme minimax des fonctions homogènes de degré un et non-expansives par rapport à une norme faible de Minkowski. Nous nous intéressons ensuite au problème de savoir si l'équation ergodique a une solution pour toute perturbation additive des paiements, problème qui étend la notion d'ergodicité des chaînes de Markov. Quand les paiements sont bornés, cette propriété d'"ergodicité" est caractérisée par l'unicité, à une constante additive près, du point fixe d'un opérateur de Shapley sans paiement. Nous donnons une solution combinatoire s'exprimant au moyen d'hypergraphes à ce problème, ainsi qu'à des problèmes voisins d'existence de points fixes. Puis, nous en déduisons des résultats de complexité. En utilisant la théorie des opérateurs accrétifs, nous généralisons ensuite la condition d'hypergraphes à tous types d'opérateurs de Shapley, y compris ceux provenant de jeux dont les paiements ne sont pas bornés. Dans un troisième temps, nous considérons le problème de l'unicité, à une constant additive près, du vecteur propre. Nous montrons d'abord que l'unicité a lieu pour une perturbation générique des paiements. Puis, dans le cadre des jeux à information parfaite avec un nombre fini d'actions, nous précisons la nature géométrique de l'ensemble des perturbations où se produit l'unicité. Nous en déduisons un schéma de perturbations qui permet de résoudre les instances dégénérées pour l'itération sur les politiques. Title : Nonlinear Perron-Frobenius theory and mean-payoff zero-sum stochastic games.Keywords : zero-sum repeated games, mean-payoff stochastic games, ergodic control, Shapley operators, nonexpansive maps, accretive operators, hypergraphs.

  Université Paris-Saclay Espace Technologique / Immeuble Discovery Route de l'Orme aux Merisiers RD 128 / 91190 Saint-Aubin, France

  In state i ∈ {1, . . . , n}, MIN can move the token to a state j ∈ {1 , . . . , m } such that A ji = -∞, receiving A ji . In state j, MAX can move the token to a state k ∈ {1, . . . , n} such that B jk = -∞, receiving B jk . We assume that the matrix B has no identically infinite row, and that the matrix A has no identically infinite column, meaning that each player has at least one available action in each state. Then, the modified operator T • R M corresponds essentially to the matrix B M in which infinite entries of B are replaced by -M , and the operator g + T • R M arises by subtracting the constant g i to every entry in the ith column of A. Let T denote the Shapley operator (7.11) of a deterministic mean-payoff game, with integer payoffs bounded in absolute value by D 2. Then, for M > 4nD and 0 < ε < 1/n 3 , the policy iteration Algorithm 2 applied to the operator g ε + T • R M terminates, and any optimal policy σ of player MIN, i.e., such that λ(g ε

	Theorem 7.23.

.11)

The corresponding game is played by moving a token on a graph in which n nodes, denoted by 1, . . . , n, belong to player MIN, whereas m other nodes, denoted by 1 , . . . , m , belong to player MAX.

i m λ i = 1, λ i 0 and p i ∈ ext P for all i ∈ [m] and some integer m 1, and so, r α , x = 1 i m λ i p i , x sup p∈ext P p, x . We deduce that q(x) = lim α r α , x sup p∈ext P p, x . The opposite inequality q(x) sup p∈ext P p, x follows readily from (3.3). Using the weak-* continuity of the map p → p, x , we get sup p∈ext P p, x = sup p∈ext P p, x . Finally, since ext P is a weak-* closed subset of the weak-* compact set P, it is also weak-* compact. So, we deduce that the latter supremum is attained.

Remerciements

Using this result and the characterizations of the previous sections, we obtain the following.

Corollary 5.30. Problem NontrivialFP is NP-complete.

Proof. As a direct consequence of Theorem 5.29, we get that Problem NontrivialFP is NP-hard. We now show that it is in NP. Suppose that a payment-free Shapley operator F (5.8) has a nontrivial fixed point u ∈ R n . Then arg min u and arg max u are proper subsets of states, and by Lemma 5.5, we have arg min u ∈ F + and Φ(arg min u) ⊃ arg max u = ∅. Hence, arg min u is a proper subset of states in F + such that Φ(arg min u) is nonempty, and we know by Corollary 5.8 that these conditions are sufficient to guarantee the existence of a nontrivial fixed point. Furthermore, these conditions can be checked in polynomial time, as a consequence of Proposition 5.27. Hence, arg min u is a short certificate to Problem NontrivialFP.

Mean-payo vectors with prescribed extrema

A way to analyze Problem NontrivialFP would be to characterize the set of fixed points W := {w ∈ R n | F (w) = w} of a payment-free Shapley operator F . This problem also arises in several other situations. First, in Proposition 4.4, we have shown that W is exactly the set of possible mean-payoff vectors of the parametric game Γ(r) := ([n], A, B, K A , K B , r, p) when the payment function r varies. Next, W allows one to determine the set E of solutions u of the ergodic equation T (u) = λe + u. Indeed, it is shown in [START_REF] Akian | Uniqueness of the fixed point of nonexpansive semidifferentiable maps[END_REF] that if the Shapley operator T is piecewise affine, if u is any point in E and if V is a neighborhood of 0, then

where F is a payment-free Shapley operator (namely the semidifferential of T at point u). Hence, the local study of the ergodic equation reduces to the characterization of the fixed-point set W.

Fixed points with prescribed argmin

In an attempt to understand the structure of the set of fixed points of a payment-free Shapley operator, we shall consider the following simpler problem. Problem (MinFP). Let I be any subset of [n]. Does a given payment-free Shapley operator (5.8) associated with a perfect-information finite stochastic game have a fixed point u satisfying arg min u = I?

We know from Lemma 5.5 that a necessary condition is I ∈ F + . Under Assumption 5.B, this is equivalent to F + (e I c ) e I c (Lemma 5.18). We next show that there is a stronger necessary condition.

). The last vector is Boolean, so this inequality implies e I c F + (e I c ). Moreover, according to Lemma 5.5 and Lemma 5.18, we already know that F + (e I c ) e I c . Hence the result.

We first explain how the general case can be reduced to the situation in which Assumption 7.D holds. To that purpose, let use introduce for any real number M 0, the map R M : R n → R n the ith coordinate of which is given by

Lemma 7.17. Let T : R n → R n be the Shapley operator of a perfect-information finite stochastic game, and let M 0. Then, T • R M has an eigenvalue.

Proof. First, note that the recession operator of R M is given by

where max x := max 1 i n x i . Second, since Assumption 4.A holds, it is readily seen that the limit (2.7) defining T is uniform in x. Hence, we get that T • R M = T • R M . Thus, using the properties of recession operators, we have for any vector x ∈ R n ,

This proves that the only fixed points of T • R M are trivial. The conclusion follows from Theorem 4.6.

The operator T • R M can be interpreted as the Shapley operator of a perfect-information finite stochastic game with state space [n]. In this game, at each step, if the current state is i ∈ [n], player MIN start by choosing an action a ∈ A i . Then, player MAX chooses an action b ∈ B i,a which gives rise to a transition payment r(i, a, b) and a state j is chosen randomly with probability p(j | i, a, b) and announced to the players. Finally, MAX has the possibility to choose the next state: if he pick any state k different from j, then he has an additional payment of -M . In other words, MAX has the option of teleporting himself to any other state, by accepting a penalty M .

Note that, since MIN has the same action space in the latter game as in the game Γ, the sets of his stationary strategies in both games are identical. Then, for a fixed policy σ of MIN, the one-player Shapley operator (T • R M ) σ is equal to T σ • R M , and we get from Lemma 7.17 the following result.

Corollary 7.18. Let T : R n → R n be the Shapley operator of a perfect-information finite stochastic game. Then, T • R M satisfies Assumption 7.D.

In the modified game with Shapley operator T • R M , player MAX makes, at each step, the final decision about the next state, provided an additional cost of M . The following result shows that if this cost is large enough, then MAX cannot do better, in the long run, than in the game Γ.

Lemma 7.19. Let T : R n → R n be the Shapley operator of a perfect-information finite stochastic game. Then, there exists a positive constant M 0 such that for any M > M 0 , the eigenvalue of T • R M is equal to the upper mean payoff χ(T ).

Proof. First, note that R M (x) x for all x ∈ R n . Hence, by monotonicity of T , we deduce that

By application of a Collatz-Wielandt formula (see [START_REF] Gaubert | The Perron-Frobenius theorem for homogeneous, monotone functions[END_REF]), we know that the eigenvalue of

We also know that T has an invariant half-line with direction χ(T ). So there exists a vector u ∈ R n such that T (u) = u + χ(T ). Now let M 0 := u H . For every M > M 0 , we have

We shall need a technical bound on invariant measures of stochastic matrices arising from strategies. We state it here for an arbitrary irreducible stochastic matrix.

Lemma 7.20. Let P be a n × n irreducible stochastic matrix whose entries are rational numbers with numerators and denominators bounded by an integer D. Then, every entry of the invariant probability measure of P is a rational number whose denominator is bounded by

Proof. The invariant probability measure m of P is the unique solution of the linear system

where I is the identity matrix. Note that one row of the subsystem (I -P ) m = 0 is redundant since we are dealing with stochastic vectors and e m = 1. Then, by deleting this row and by multiplying every row of the latter subsystem by all the denominators of the coefficients appearing in this row, we arrive at a Cramer linear system with integer coefficients of absolute value less than D n , and with unit coefficients on the last row. Solving this system by Cramer's rule, we obtain that the entries of m are rational numbers whose denominators divide the determinant of the system. Using Hadamard's inequality for determinants, we deduce that these denominators are bounded by

We just showed that the upper mean payoff of T can be recovered from the upper mean payoff of the operator T • R M for a suitable large M . The latter operator satisfies Assumption 7.D, and so, we can in principle apply Algorithm 2 to it. However, to do so in a way which leads in a polynomial-time transformation of the input, it is convenient to introduce the following modified Shapley operator T M : R 2n → R 2n given, for all (x, y) ∈ R n × R n , by T M (x, y) := (T (y), R M (x)) .

Note that we have

The following immediate lemma shows that one can recover the eigenvectors and the eigenvalue of T • R M from those of T M .