Fabien Gensbittel
email: fabien.gensbittel@tse-fr.eu

Miquel Oliu-Barton
email: miquel.oliu.barton@normalesup.org

Xavier Venel
email: xavier.venel@sip.univ-tlse1.fr

Existence of the uniform value in repeated games with a more informed controller

. A key element in this work is to prove that player 1 can use strategies of the auxiliary game in the initial game in our general framework, which allows to deduce that the value of the auxiliary game is also the value of our initial repeated game by using classical arguments.

Introduction

Zero-sum repeated games with incomplete information were introduced by Aumann and Maschler in 1966 [START_REF] Aumann | Repeated games with incomplete information, with the collaboration of R. Stearns[END_REF] in order to study repeated interactions between two players having a different information. The authors also introduced a notion of value for these games usually called uniform value and proved its existence for games with incomplete information on one side. Mertens and Neyman [START_REF] Mertens | Stochastic games[END_REF] proved that the uniform value exists for finite stochastic games and several works were devoted since then to prove the existence of the uniform value for some subclasses of the general model of repeated games. Recently, Renault proved in [START_REF] Renault | The value of repeated games with an informed controller[END_REF] that the uniform value exists in repeated games with an informed controller using an approach based on an existence result for dynamic programming problems (Renault,[START_REF] Renault | Uniform value in Dynamic Programming[END_REF]). The existence theorem in [START_REF] Renault | The value of repeated games with an informed controller[END_REF] requires that the first player observes the state variable at each stage and controls and observes the evolution of the beliefs of the second player on the state variable.

In the present work, we prove that the uniform value exists in the class of repeated games with a more informed controller. Our existence result requires that the first player is more informed about the state variable than the second player and also that he controls the evolution of beliefs of the second player. A weaker version of our result was conjectured in the conclusion of [START_REF] Renault | The value of repeated games with an informed controller[END_REF], and it was suggested that the proof may be based on an auxiliary game whose state space would be the pair of beliefs of both players about the original state variable. We show that the analysis requires actually to introduce an auxiliary game whose state space is the set of second order beliefs of the less informed player and provide a set of weaker assumptions than those suggested in [START_REF] Renault | The value of repeated games with an informed controller[END_REF], allowing to deal with imperfect monitoring of actions.

The paper is organized as follows: In section 2, we describe the general model of repeated games and introduce three assumptions that formalize the notion of a more informed controller. In section 3, we check that several models previously studied in the literature satisfy these three assumptions. Section 4 is dedicated to a discussion of the assumptions and a precise study of their implications. In addition, we provide a second version of the theorem with stronger, but easier to check, assumptions. The last section 5 is dedicated to the proof of existence of the uniform value. We introduce there an auxiliary stochastic game with perfect monitoring on an auxiliary state variable which represents the beliefs of player 2 about the beliefs of player 1 about the state variable of the original game. We prove that this auxiliary game has a uniform value using the main theorem of Renault [START_REF] Renault | The value of repeated games with an informed controller[END_REF] and that player 1 can use optimal strategies in this auxiliary stochastic game in order to play optimally in the original repeated game. Finally, we prove that player 2 can also guarantee this value by playing by blocks, so that both games have a uniform value and these values are equal.

Model

General definitions and notation

For any metric space X, let ∆(X) denote the set of Borel probability distributions on X. If X is a finite set (endowed with the discrete metric) of cardinal |X|, then ∆(X) is precisely the |X|-dimensional simplex. ∆ f (X) ⊂ ∆(X) denotes the probability distributions supported on a finite subset of X and δ x denotes the Dirac measure on x ∈ X.

A zero-sum repeated game is described by a 8-tuple (K, I, J, g, C, D, π, q), where K is the state space, I and J are the action sets for player 1 and 2 respectively, g is a payoff function g : K × I × J → [0, 1], and C and D are the signal sets for player 1 and 2 respectively. π ∈ ∆ f (K × N × N) denotes the initial probability and q : K × I × J → ∆(K × C × D) denotes the transition function.

The game is played as follows: At the beginning of the game, the triple (k 1 , c 1 , d 1 ) is chosen according to the initial probability distribution π ∈ ∆ f (K × N × N). For each stage m ≥ 1, player 1 observes the signal c m and player 2 observes the signal d m . Then both players choose actions (i m , j m ) ∈ I × J based on their own past actions and on the sequence of signals they observed (i.e. we assume perfect recall). Given the state k m and the actions (i m , j m ), a new triple (k m+1 , c m+1 , d m+1 ) ∈ K × C × D is chosen according to the probability distribution q(k m , i m , j m ). The payoff for stage m of player 1 is g(k m , i m , j m ) and the game proceeds to stage m+1. The stage payoffs are not directly observed by the players and cannot be deduced, in general, from their observations. The sets of initial signals can be any finite subset of N. This generalization is for technical reasons only. Indeed, it will very convenient in the sequel to consider this possibly larger set of initial signals in order to have a simple way to deal with the recursive structure of the game.

The information held by player 1 before his play at stage m, called player 1's private history, is given by

h I m (c 1 , i 1 , . . . , c m-1 , i m-1 , c m ) ∈ N × (I × C) m-1 .
Similarly, the information held by player 2 is represented by

h II m (d 1 , j 1 , . . . , d m-1 , j m-1 , d m ) ∈ N × (J × D) m-1 .
Let H I (resp. H II ) denote the set of all finite private histories for player 1 (resp. of all private histories for player 2). We assume the sets K, I, J, C and D are all finite and that the description of the model is common knowledge.

Instead of N, the sets of initial signals will often be denoted by C ′ and D ′ where C ′ and D ′ are finite subsets of N. We will also write abusively that π ∈ ∆(K × C ′ × D ′ ). The initial signals will still be denoted by (c 1 , d 1 ). Reciprocally, given finite sets C ′ and D ′ , any π ∈ ∆(K × C ′ × D ′ ) can be seen as an element of ∆ f (K × N × N) using some enumerations of C ′ and D ′ . The main advantage is that any couple of finite private histories can be embedded in N × N via some enumerations. This advantage will become clear in the proof.

Strategies A behavior strategy for player 1 is a map from private histories H I to probabilities over I . The set of behavior strategies of player 1 is denoted by Σ. Every strategy σ ∈ Σ corresponds to a sequence {σ m } m≥1 , where σ m is defined on the set of histories up to stage m. That is,

σ m : N × (I × C) m-1 → ∆(I).
Similarly, a behavior strategy τ for player 2 is a map from private histories H II to probability distributions over J. The set of behavior strategies of player 2 is denoted by T . Any τ corresponds to a sequence {τ m } m≥1 , with

τ m : N × (J × D) m-1 → ∆(J).
The initial distribution π, the transition function q and a behavior strategy profile (σ, τ ) ∈ Σ × T induce a unique probability distribution over the set of plays K × N × N × (K × C × D × I × J) ∞ , denoted by P π στ . Let E π στ = E P π στ denote the expectation with respect to the probability P π στ .

Evaluations of the payoff A second component of the model is the way in which the total payoff of player 1 is evaluated, in terms of the sequence of stage payoffs {g(k m , i m , j m )} m≥1 .

The two classical evaluations correspond to the n-stage game and the λ-discounted game. In the former, the payoff function is the expected Cesàro mean of the stage payoffs of the n first stages, i.e.

γ n (π, σ, τ ) = E π στ 1 n n m=1 g(k m , i m , j m ) .
In the latter, the payoff is taken as the expected Abel sum, with respect to the discount factor 0

< λ ≤ 1, i.e. γ λ (π, σ, τ ) = E π στ λ m≥1 (1 -λ) m-1 g(k m , i m , j m ) .
More generally, one may consider any compact evaluation. That is, for any θ ∈ ∆(N * ), let

γ θ (π, σ, τ ) = E π στ m≥1 θ m g(k m , i m , j m ) . (2.1)
Denote by Γ θ (π) the 8-tuple defined above together with the θ-evaluation.

The value function For any π ∈ ∆ f (K × N × N) and any θ ∈ ∆(N * ), Γ θ (π) is known to have a value, denoted by v θ (π). It satisfies

v θ (π) = sup σ∈Σ inf τ ∈T γ θ (π, σ, τ ) = inf τ ∈T sup σ∈Σ γ θ (π, σ, τ ).
Remark 2.1. These general evaluations will be used in section 5, and we will only need to consider probabilities with finite support (i.e. θ ∈ ∆ f (N * )).

Let σ ∈ Σ be a behavior strategy and let h ∈ H I be some finite private history of player 1. We denote by σ(h) the behavior strategy of player 1 after the history h. Equivalently, σ(h) is the restriction of the map σ to the subset of histories beginning with h. In particular, given some strategy profile (σ, τ ) ∈ Σ × T and two signals (c, d) ∈ C ′ × D ′ , consider the profile (σ(c), τ (d)). It may be interpreted as a strategy profile in a game in which the players have no initial signals. More formally, for any p ∈ ∆(K), we will use the following notation:

γ θ (k, σ(c), τ (d)) γ θ (δ (k,c,d) , σ, τ ), and γ θ (p, σ(c), τ (d)) γ θ (p ⊗ δ (c,d) , σ, τ ).
With this notation, the payoff can be written as

γ θ (π, σ, τ ) = E π [γ θ (k, σ(c), τ (d))] = (k,c,d)∈K×C ′ ×D ′ γ θ (k, σ(c), τ (d))π(k, c, d).
Alternatively, one can consider the game as having per se infinitely many stages.

Uniform value

The infinitely repeated game is denoted by Γ ∞ (π). Let us present here some important definitions relative to the game Γ ∞ (π). Its value will be called the uniform value and denoted by v ∞ (π). Definition 2.2. Let v be a real number,

• Player 1 can guarantee v in Γ ∞ (π) if for any ǫ > 0 there exists a strategy σ ∈ Σ of player 1 and an integer N ∈ N, such that

∀n ≥ N, ∀τ ∈ τ , γ n (π, σ, τ ) ≥ v -ǫ.
We say that such a strategy σ guarantees v -ǫ in Γ ∞ (π) and define

v ∞ (π) sup{v ∈ R | player 1 can garantee v}.
• Player 2 can guarantee v in Γ ∞ (π) if for any ǫ > 0 there exists a strategy τ ∈ τ of player 2 and an integer N ∈ N, such that

∀n ≥ N, ∀σ ∈ Σ, γ n (π, σ, τ ) ≤ v + ǫ.
We say that such a strategy

τ guarantees v + ǫ in Γ ∞ (π) and define v ∞ (π) inf{v ∈ R | player 2 can garantee v}. • If v ∞ (π) = v ∞ (π)
the uniform value exists and we denote by v ∞ (π) the common value.

The existence of a uniform value v ∞ is stronger than the existence of a limit value (or asymptotic value), in the sense that it implies (see e.g. Neyman and Sorin [6, Theorem 1] for more general evaluations) lim

λ→0 v λ = lim n→∞ v n = v ∞ .

Model with a more informed controller

We will consider a particular class of the general model presented above, which generalizes both the class of repeated games considered by Renault [START_REF] Renault | The value of repeated games with an informed controller[END_REF] and the model of Partially Observable Markov Decision Processes (see section 3.1). As usual in games with incomplete information, we call belief of player 1 at stage m about some random variable ξ the conditional law of ξ given the information held by player 1 at stage m. In the sequel, the first order beliefs of a player denote beliefs about the state variable k, and second order beliefs of a player denote beliefs about the first order beliefs of his opponent.

We assume the following three hypotheses at every stage m of the game:

(a1) Player 1's first order belief is more accurate than player 2's first order belief.

(a2) Player 1 can compute the second order beliefs of player 2.

(a3) Player 1 controls the evolution of second order beliefs of player 2.

The main result of this paper is to establish the existence of the uniform value under these assumptions. Let the formal transcription of (a1) -(a3) defined below be denoted by (A1) -(A3).

Theorem 2.3. Let Γ be a repeated game with a more informed controller, i.e. such that assumptions (A1), (A2), (A3) hold. Then the uniform value exists.

Remark 2.4. It was already pointed out in the literature (see e.g. Mertens [START_REF] Mertens | Repeated Games[END_REF]) that in games with a more informed player, the analysis of beliefs can be restricted to second order beliefs of the less informed player. In this work, the definition of more informed is slightly more general than the inclusion of signals and a similar reduction is made formally in Lemma 4.4.

Formal assumptions

Let us present here a rigorous transcription of the informal assumptions (a1) -(a3). In the next section, we will present some of the models which satisfy our three assumptions and to which, consequently, Theorem 2.3 applies.

Let us start with some notations:

Given some probability distribution µ ∈ ∆ f (X × Y ) over a product, we denote by

µ(x) y∈Y µ(x, y).
For any random variable ξ defined on a probability space (Ω, A, P) and F a sub σ-algebra of A, let L P (ξ | F) denote the conditional distribution of ξ given F, which is seen as a F-measurable random variable1 and let L P (ξ) denote the distribution of ξ.

In the sequel, both functions g and q are linearly extended to ∆(K) × I × J.

Assumption (a1) can now be formalized as follows:

(A1) ∀n ≥ 1, ∀(σ, τ ) ∈ Σ × T , L P π στ (k n | h I n , h II n ) = L P π στ (k n | h I n ).
In words, at every stage and given any strategy profile, player 2's information does not contain any information about the state variable that is not already contained in player 1's information. Assumption (A1) is equivalent to the conditional independence of k n and h II n , given h I n , under the probability P π στ . For n = 1, this equation does not depend on σ and τ and it can be reformulated as

(A1a) π(c)π(k, c, d) = π(k, c)π(c, d), ∀(k, c, d) ∈ K × C ′ × D ′ .
In order to model the players' information about the state variable at stage n, we need to define three variables x n , y n and η n . Before choosing their first action, the players receive signals (c 1 , d 1 ) ∈ C ′ × D ′ . The (random) variable

x 1 L π (k 1 | c 1 ) ∈ ∆(K)
represents the first order beliefs of player 1 about the initial state. Let x 1 (c 1 ) ∈ ∆(K) denote its realization, i.e. the beliefs of player 1 once he has received the signal c 1 ∈ C ′ . Thus, x 1 (c 1 ) = L π (k 1 |c 1 ) and each signal c 1 ∈ C ′ occurs with probability π(c 1 ), so that

L π (x 1 ) = c 1 ∈C ′ π(c 1 )δ x 1 (c 1 )
. Similarly, define the second order beliefs of player 2, i.e. beliefs about player 1's beliefs about the initial state

y 1 L π (x 1 |d 1 ) ∈ ∆ f (∆(K).
With probability π(d 1 ), player 2's beliefs about player 1's beliefs (about the state variable) are distributed as follows:

y 1 (d 1 ) = c 1 ∈C ′ π(c 1 |d 1 )δ x 1 (c 1 ) ∈ ∆ f (∆(K)),
with a slight abuse of notations since we write π(c 1 |d 1 ) instead of π(c 1 = c|d 1 ) with a sum over c ∈ C ′ . Finally, let η 1 be the distribution of the second order beliefs of player 2

η 1 L π (y 1 ) = d 1 ∈D ′ π(d 1 )δ y(d 1 ) ∈ ∆ f (∆ f (∆(K))).
Notice that the Dirac measures involved in the definition of L π (x 1 ) or of L π (y 1 ) refer to different spaces: the former refers to ∆(K), the latter to ∆ f (∆(K)).

More generally, for some fixed strategy profile (σ, τ ) ∈ Σ × T , let us denote the first order beliefs of player 1 at stage n by x n ∈ ∆(K), the second order beliefs of player 2 at stage n by If player 2 receives u 2 , then we obtain

y n ∈ ∆ f (∆(K)),
y 1 (u 2 ) = 6 24 δ 1 2 + 6 24 δ 0 .
To conclude, player 2 receives each signal with probability 1 2 , so that η 1 is equal to

η 1 = 1 2 δ 8 24 δ 1 + 2 24 δ 1 2 + 2 24 δ 0 + 1 2 δ 6 24 δ 1 2 + 6 24 δ 0 .
Assumption (a2) will be split in two parts (A2a) and (A2b). At first, we assume that player 1 is able to compute the variable y 1 , which is a constraint on the initial probability π only.

(A2a) There exists a map

f 1 = f π 1 : C ′ → ∆(∆(K)) such that y 1 = f 1 (c 1 ), π-almost surely.
Assuming (A2a), we can introduce a special class of strategies for player 1 which will be needed for the second part of the formal assumption.

Definition 2.7. If π ∈ ∆ f (K × N × N) fulfills (A2a), a strategy σ ∈ Σ is called a reduced strategy if it depends on the initial signal c 1 in C ′ only through (x 1 , y 1 ). Let Σ ′ (π) ⊂ Σ denote the subset of reduced strategies.
The second part of (A2) requires that when player 1 is using a reduced strategy, the variable y 2 has to be h I 2 -measurable. Formally:

(A2b) ∀π ∈ ∆ f (K × N × N) satisfying (A2a), ∀σ ∈ Σ ′ (π), ∀τ ∈ T , ∃f 2 = f π,σ,τ 2 : N × I × C → ∆(∆(K)) such that y 2 = f 2 (c 1 , i 1 , c 2 ), P π σ,τ -almost surely.
The introduction of reduced strategies for player 1 is necessary in order to exclude non relevant correlations between players (see example 4.1 in section 4). It will be shown in Lemma 4.4 and in the proof of the main Theorem that there is no loss in restricting player 1 to reduced strategies.

In order to state the last assumption, we reduce the set of initial probabilities.

Definition 2.8. Let ∆ * f (K × N × N) be the set of probability distributions satisfying (A1a) and (A2a). Assumption (a3) can now be formalized as

(A3) ∀π ∈ ∆ * f (K × N × N), ∀σ ∈ Σ ′ (π)
, η 2 is independent of τ ∈ T . Remark 2.9. Assumptions (A1, A2) imply that the properties (A1a) and (A2a) of the initial probability π are preserved by the transition when player 1 plays reduced strategies. Precisely, for all (σ, τ ) with σ reduced, the law of (k 2 , h I 2 , h II 2 ) under P π στ , seen as an element of ∆ f (K × N × N), belongs to the set ∆ * f (K × N × N). We will prove in section 4 that even if the two last assumptions (A2b) and (A3) are stated in terms of y 2 and η 2 , it is possible to extend these properties by induction for y n and η n for appropriate strategies. Thus the formal assumptions are coherent with the informal assumptions. In particular, player 1 can compute the auxiliary variables (x 2 , y 2 , η 2 ) without knowing the strategy of player 2 and therefore play again a reduced strategy at the second stage (i.e. which depends only on (x 2 , y 2 )).

Applications

We present in this section several models which satisfy our assumptions.

Partially Observable Markov Decision Processes.

A POMDP is a one-player game, given by a tuple (K, I, C, g, q, π), where K is the state space, I is the action set, C is the signals set, g : K × I → [0, 1] is the payoff function, q : K × I → ∆(K × C) is the transition function and π is an initial distribution on K × N. In the finite framework, the existence of the uniform value has been proven by Rosenberg, Solan and Vieille [START_REF] Rosenberg | Blackwell optimality in Markov decision processes with partial observation[END_REF] and it was extended by Renault [START_REF] Renault | Uniform value in Dynamic Programming[END_REF] to arbitrary set of actions and signals with the additional assumption that all the probabilities appearing in the transition or in the definition of strategies have finite support. We will only consider here the finite case.

Formally, a POMDP can be seen as a repeated game in which player 2 is dummy (i.e. his action set J is a singleton). Since player 2 has only one action, his information plays no role here. The assumptions (A1) -(A3) hold obviously.

Repeated game with a perfectly informed controller.

The model of a repeated game with an informed controller introduced by Renault [10] fulfils our assumptions. In this model, player 1 is perfectly informed of the state and of the signal of player 2, in the sense that he can deduce the true state variable and the signal of player 2 from his signals. Moreover, the transition q is such that player 2 has no influence on the joint distribution of the pair made by the state variable and his signal.

In [START_REF] Renault | The value of repeated games with an informed controller[END_REF], the sets of initial signals are C ′ = C and D ′ = D. Formally, the first assumption (i.e. that player 1 is perfectly informed of the state and of the signal of player 2) is given by (HA ′ ) There exists two mappings k :

C → K and d : C → D such that, if E denotes {(k, c, d) ∈ K × C × D, k(c) = k, d(c) = d}, then π(E) = 1, and q(k, i, j)(E) = 1, ∀(k, i, j) ∈ K × I × J.
The second assumption is formalized by (HB ′ ) Player 1 controls the transition in the sense that the marginal of the transition q on K × D does not depend on player 2's action. For k ∈ K, i ∈ I, j ∈ J, we denote by q(k, i) the marginal of q(k, i, j) on K × D.

Let us check that this model satisfies our assumptions. Assuming that the initial distribution π ∈ ∆(K × C × D) fulfils assumption (HA ′ ), we have

y 1 (d 1 ) = c 1 ∈C π(c 1 |d 1 )δ Lπ(k 1 |c 1 ,d 1 ) = c 1 ∈C π(c 1 |d 1 )δ δ k(c 1 ) = k 1 ∈K π(k 1 |d 1 )δ δ k 1 .
We deduce that π can be seen as an element of ∆ * f (K × N × N). Formally, we have to verify our assumptions, starting from any initial distribution in ∆ * f (K ×N×N). From now on, initial signals (c 1 , d 1 ) belong to arbitrary finite subsets of N denoted by C ′ , D ′ as in the previous section 2 .

First, note that any stage m ≥ 2, player 1's first order belief at each stage is a Dirac mass on the current state (i.e. x n = δ kn ). Thus, adding the signal of player 2 to the signal of player 1 does not change the beliefs of the latter, which proves (A1). It also implies that the second order beliefs of player 2 can be identified with the first order beliefs of player 2. Let π ∈ ∆ * f (K × N × N), τ be a strategy for player 2 and σ a reduced strategy for player 1. Recall that σ is function only of (x 1 , y 1 ) which are by assumption c 1 measurable and that that y 1 is d 1 -measurable, so that there exist two functions h 1 and f 1 such that, with probability 1,

L π (x 1 |d 1 ) = y 1 = h 1 (d 1 ) = f 1 (c 1 ). It follows that P π στ (k 1 , x 1 , d 1 , i 1 , j 1 , k 2 , d 2 ) = π(d 1 )h 1 (d 1 )(x 1 )x 1 (k 1 )σ(x 1 , h 1 (d 1 ))(i 1 )τ (d 1 )(j 1 )q(k 1 , i 1 )(k 2 , d 2 ).
We deduce that

y 2 (d 1 , j 1 , d 2 ) = c 1 ,i 1 ,c 2 P π στ (c 1 , i 1 , c 2 |d 1 , j 1 , d 2 )δ L P π στ (k 2 |c 1 ,i 1 ,c 2 ) = c 1 ,i 1 ,c 2 P π στ (c 1 , i 1 , c 2 |d 1 , j 1 , d 2 )δ δ k(c 2 ) = k 2 P π στ (k 2 |d 1 , j 1 , d 2 )δ δ k 2 = k 1 ,x 1 ,k 2 ,i 1 π(d 1 )h 1 (d 1 )(x 1 )σ(x 1 , h 1 (d 1 ))(i 1 )q(k 1 , i 1 )(k 2 , d 2 )δ δ k 2 k ′ 1 ,x ′ 1 ,k ′ 2 ,i ′ 1 π(d 1 )y 1 (d 1 )(x ′ 1 )σ(x ′ 1 , y 1 (d 1 ))(i ′ 1 )q(k ′ 1 , i ′ 1 )(k ′ 2 , d 2 ) = k 1 ,x 1 ,k 2 ,i 1 f 1 (c 1 )(x 1 )σ(x 1 , f 1 (c 1 ))(i 1 )q(k 1 , i 1 )(k 2 , d 2 )δ δ k 2 k ′ 1 ,x ′ 1 ,k ′ 2 ,i ′ 1 f 1 (c 1 )(x ′ 1 )σ(x ′ 1 , f 1 (c 1 ))(i ′ 1 )q(k ′ 1 , i ′ 1 )(k ′ 2 , d 2 )
where the above equalities hold almost surely whenever the conditional probabilities are welldefined. We deduce that y 2 is only a function of c 1 and d 2 which does not depend on τ , so that the function

f 2 (c 1 , c 2 ) = y 2 (c 1 , d(c 2 )
) proves that assumption (A2) is satisfied. Finally the distribution of the random variable y 2 is equal to

η 2 = c 1 ,d 2 P π στ (c 1 , d 2 )δ y 2 (c 1 ,d 2 ) .
Using the preceding result and that the marginal of q on D does not depend on the action played by player 2, η 2 does not depend on τ and (A3) is satisfied.

Remark 3.1. In a previous work, Renault [START_REF] Renault | The value of Markov chain games with lack of information on one side[END_REF] studied the particular case where the state follows a Markov chain f : K → ∆(K), player 1 observes the state and both players observe the actions. This is easily seen as a particular case of the above model. In a more recent work, Neyman [START_REF] Neyman | Existence of optimal strategies in Markov games with incomplete information[END_REF] proved the existence of the uniform value when allowing for any signalling structure on the actions. This last result is not covered by our main theorem since in this case, player 1 cannot control player 2's information about the state variable.

3.3 Player 1 is more informed about the state.

In this last paragraph, we assume that actions are observed by both players after each stage. Moreover, both players receive a public signal in a set U , player 1 receives a signal in a set S and player 2 has influence on the joint distribution of the state and signals in

K × U × S.
Formally, it is a repeated game where C = I × J × S × U , D = I × J × U and the transition function satisfies the following two conditions. At first, the signal u is public and the actions are observed:

∀(k, i, j) ∈ K × I × J, k ′ ,s,u∈K×S×U q(k, i, j)(k ′ , (i, j, u), (i, j, s, u)) = 1.
Secondly, there exists a function q from K × I to ∆(K × S × U ) such that

∀(k, k ′ , i, j, s, u) ∈ K × K × I × J × S × U, q(k, i, j)(k ′ , (i, j, s, u), (i, j, u)) = q(k, i)(k ′ , s, u).
Let us stress out that the transition q in itself depends on player 2 since it has to reveal his actions but as we will see our assumptions are still satisfied. It was already noticed in Renault [START_REF] Renault | The value of repeated games with an informed controller[END_REF] that it is too restrictive to assume that the transition is fully controlled by player 1. This model is a natural generalization of Renault's model, dropping the (important) condition of Player 1 to know the state at every stage. However, it does not allow for imperfect monitoring of actions as in the previous examples.

Let us check that this model satisfies our assumptions. According to the description of the model, an initial distribution π of (k 1 , s 1 , u 1 ) can be seen as an element of ∆ * f (K ×N×N) since the signal of player 2 is contained in the signal of player 1. As for the previous example, we will start with a general initial probability π ∈ ∆ * f (K × N × N) and initial signals (c 1 , d 1 ). At first, note that apart from the initial signal d 1 , histories of player 2 are contained in histories of player 1, so that assumption (A1) reduces to

∀n ≥ 1, ∀(σ, τ ) ∈ Σ × T , L P π στ (k n | h I n , d 1 ) = L P π στ (k n | h I n ).
This property is true for n = 1 by assumption. Let us proceed by induction on n. Assume that n ≥ 2 and that the property is proved for n -1, i.e. that

x n-1 = L P π στ (k n-1 | h I n-1 , d 1 ) = L P π στ (k n-1 | h I n-1 ).
Again, let q be linearly extended to ∆(K) × I. It follows that,

L P π στ (k n , s n , u n |h I n-1 , d 1 , i n-1 , j n-1 ) = q(x n-1 , i n-1 ) = L P π στ (k n , s n , u n |h I n-1 , i n-1 , j n-1 ),
(A1) follows then directly by disintegration. Moreover, we deduce that

P π σ,τ (k n = k|h I n ) = q(x n-1 , i 1 )(k, s n , u n ) k ′ ∈K q(x n-1 , i 1 )(k ′ , s n , u n ) . (3.1)
The latter proves that x 2 can be expressed as a function of (x 1 , i 1 , s 2 , u 2 ) which does not depend on τ . Recall then that by assumption there exist functions h 1 and f 1 such that with probability 1, we have

y 1 = L π (x 1 |d 1 ) = h 1 (d 1 ) = f 1 (c 1 ).
If player 1 uses a reduced strategy σ and player 2 uses a strategy τ , we have

P π σ,τ (k 1 , x 1 , d 1 , i 1 , j 1 , k 2 , s 2 , u 2 ) = π(d 1 )h 1 (d 1 )(x 1 )x 1 (k 1 )σ(x 1 , h 1 (d 1 ))(i 1 )τ (d 1 )(j 1 )q(k 1 , i 1 )(k 2 , s 2 , u 2 ).
We deduce that

y 2 (d 1 , i 1 , j 1 , u 2 ) = x 1 ,i 1 ,s 2 P π σ,τ (x 1 , i 1 , s 2 |d 1 , j 1 , u 2 )δ L P π σ,τ (k 2 |x 1 ,d 1 ,i 1 ,j 1 ,s 2 ,u 2 ) = x 1 ,i 1 ,s 2 P π σ,τ (x 1 , i 1 , s 2 |d 1 , j 1 , u 2 )δ x 2 (x 1 ,i 1 ,s 2 ,u 2 ) .
From the previous formula, we deduce

P π σ,τ (x 1 , i 1 , s 2 |d 1 , j 1 , u 2 ) = k 1 π(d 1 )h 1 (d 1 )(x 1 )x 1 (k 1 )σ(x 1 , h 1 (d 1 ))(i 1 )τ (d 1 )(j 1 )q(k 1 , i 1 )(s 2 , u 2 ) k ′ 1 ,x ′ 1 ,i ′ 1 ,s ′ 2 π(d 1 )h 1 (d 1 )(x ′ 1 )x ′ 1 (k ′ 1 )σ(x ′ 1 , h 1 (d 1 ))(i ′ 1 )τ (d 1 )(j 1 )q(k ′ 1 , i ′ 1 )(s ′ 2 , u 2 ) = k 1 f 1 (c 1 )(x 1 )x 1 (k 1 )σ(x 1 , f 1 (c 1 ))(i 1 )q(k 1 , i 1 )(s 2 , u 2 ) k ′ 1 ,x ′ 1 ,i ′ 1 ,s ′ 2 f 1 (c 1 )(x ′ 1 )x ′ 1 (k ′ 1 )σ(x ′ 1 , f 1 (c 1 ))(i ′ 1 )q(k ′ 1 , i ′ 1 )(s ′ 2 , u 2 )
.

Thus, y 2 does not depend on τ nor on (j 1 , d 1 ). Player 1, knowing i 1 , c 1 and u 2 , can compute y 2 , which proves that assumption (A2) is satisfied. Finally, the distribution of the random variable y 2 is equal to

η 2 = c 1 ,i 1 ,u 2 P π σ,τ (c 1 , i 1 , u 2 )δ y 2 (c 1 ,i 1 ,u 2 ) .
Since the function q does not depend on j 1 , we deduce as above that η 2 does not depend on τ and therefore that (A3) is satisfied.

Discussion of the assumptions

In this section, we discuss several implications of the formal assumptions (A1) ,(A2), (A3). At first, we show that it is necessary to introduce the notion of reduced strategy in order to exclude non relevant correlations between the players. Then, in order to answer to a suggestion made in [START_REF] Renault | The value of repeated games with an informed controller[END_REF], we show that the analysis cannot be made in terms of first order beliefs only, and that it is necessary to introduce second order beliefs. Lemma 4.4 shows that the value can be expressed as a function of second order beliefs. Then, we prove that player 1 can compute his first order beliefs, the second order beliefs of player 2 and the distribution of these beliefs without knowing the strategy of player 2 as soon as he plays a Markovian strategy with respect to the beliefs at each stage. Finally, we give a weaker version of the theorem where the assumptions are formulated more directly in terms of the data of the game.

Necessity of reduced strategies.

The introduction of reduced strategies for player 1 is necessary in order to exclude non relevant correlations between players as shown in example 4.1 below. It will be shown in Lemma 4.4 and in the main Theorem that in our model, there is no loss in restricting player 1 to reduced strategies. The transition q depends only on the action of player 1 and is described by the matrices

T B δ k 1 1 2 δ k 1 + 1 2 δ k 2 1 2 δ k 1 + 1 2 δ k 2 δ k 2 . k 1 k 2 . At

Second-order beliefs

Renault [START_REF] Renault | The value of repeated games with an informed controller[END_REF] conjectured that the pair of distributions of first order beliefs of both players could be sufficient auxiliary variables. We present here an example showing the necessity to take into account second order beliefs in the sense that there exist a game and two initial probabilities π and π ′ such that the law of first-order beliefs are the same under π and π ′ while the values differ.

0 0   s 1 s 2 s 3   0 0 3 24 1 24 0 8 24   k 1 k 2 .
We will identify ∆(K) and [0, 1] as in example 2.6. The beliefs of Player 2 about the state are the same in both cases and are equal to 1 2 δ 3 4

+ 1 2 δ 1 4
. Similarly the beliefs of Player 1

are 1 3 δ 1 + 1 3 δ 1 2
+ 1 3 δ 0 in both games. Moreover player 1 observes the signal of player 2, so that assumption (A2) is satisfied. Thus, the laws of first-order beliefs are not sufficient to discriminate between π and π ′ . Let Γ be the repeated game where K = {k 1 , k 2 }, I = {T, B}, J = {L, R} and payoff g given by

0 1 1 2 1 1 2 1 0 k 1 k 2 .
The average payoff matrix with coefficients

( 1 2 , 1 2 ) is 1 2 3 4 1 1 .
Let us prove that v 1 (π) and v 1 (π ′ ) are different. If player 1 receives s 3 , then his beliefs on the state is 0 and thus Top is a weakly dominant action. If he receives s 1 or s 2 , his beliefs is 1 or 1 2 and Bottom is a strictly dominant action. From the point of view of player 2, playing Left if receiving u 1 and playing Right if receiving u 2 is a best reply to this strategy. Using these strategies, we find that v 1 (π) = 7 8 and v 1 (π ′ ) = 11 12 .

Let us prove that if assumptions (A1a) and (A2a) hold, then v θ (π) depends only on the law of second order beliefs of player 2.

Definition 4.3. For all π ∈ ∆ * f (K × N × N), define Φ(π) L π (L π (L π (k 1 |c 1 )|d 1 )) . Lemma 4.4. Let π, π ′ ∈ ∆ * f (K × N × N). If Φ(π) = Φ(π ′ ), then v θ (π) = v θ (π ′ ).
Proof. Let (σ, τ ) be a pair of behavior strategies in Γ θ (π). It is enough to show that v θ (π) depends on π only through η 1 = Φ(π). Recall that x 1 := L π (k 1 |c 1 ) and y 1 := L π (L π (k 1 |c 1 )|d 1 ). Note that y 1 is a function of d 1 , and that x 1 is a function of c 1 . Moreover, by assumption (A2a), there exists a map f π 1 :

C ′ → ∆(∆(K)) such that f π 1 (c 1 ) = y 1 π-almost surely.
Let us construct a reduced version of the game Γ θ (π) in which player 1 and player 2 are constrained to choose strategies that depend only on c 1 and d 1 through the variables (x 1 , y 1 ) and y 1 respectively, and keeping the same payoff function. This game has a value since the sets of possible values of (x 1 , y 1 ) is finite and this value is exactly the value of Γ θ (π) where π is the joint distribution of (k 1 , (x 1 , y 1 ), y 1 ) seen as an element of ∆ * f (K × N × N). The sets of strategies in Γ θ (π) (denoted by Σ ′ (π) and T ′ (π)) can be seen as subsets of Σ and T via the previous identification and we will prove that both games have the same value and that v θ (π) depends only on η 1 .

Assume at first that τ ∈ T ′ (π) and σ ∈ Σ and let µ denote the joint law of (k 1 , c 1 , d 1 , x 1 , y 1 ) induced by π. By disintegration, we have

γ θ (π, σ, τ ) = K×N×∆(K)×∆(∆(K)) γ θ (k 1 , σ(c 1 ), τ (y 1 ))dµ(k 1 , c 1 , x 1 , y 1 ), = N×∆(K)×∆(∆(K)) K γ θ (k 1 , σ(c 1 ), τ (y 1 ))dL µ (k 1 |c 1 , x 1 , y 1 ) dµ(c 1 , x 1 , y 1 ), = N×∆(K)×∆(∆(K)) K γ θ (k 1 , σ(c 1 ), τ (y 1 ))dL µ (k 1 |c 1 )dµ(c 1 , x 1 , y 1 ), = N×∆(K)×∆(∆(K)) γ θ (•, σ(c 1 ), τ (y 1 )), x 1 R K dµ(c 1 , x 1 , y 1 ),
where we used that L µ (k 1 |c 1 , x 1 , y 1 ) = L µ (k 1 |c 1 ) since (x 1 , y 1 ) are c 1 -measurable and the notations γ θ (., σ(c 1 ), τ (y 1 )) for (γ θ (k, σ(c 1 ), τ (y 1 ))) k∈K ∈ R K and •, • R K for the scalar product in R K . Taking the supremum over all strategies of player 1, we obtain

sup σ∈Σ γ θ (π, σ, τ ) = N×∆(K)×∆(∆(K)) sup σ(c 1 ) γ θ (•, σ(c 1 ), τ (y 1 )), x 1 R K dµ(c 1 , x 1 , y 1 ).
The supremum inside the integral is achieved by strategies depending only on (x 1 , y 1 ) since these variables are c 1 measurable. It means that there exists an optimal strategy in Σ ′ (π), which proves inf

τ ∈T ′ (π) sup σ∈Σ γ θ (π, σ, τ ) = inf τ ∈T ′ (π) sup σ∈Σ ′ (π) γ θ (π, σ, τ ).
Moreover the value of the reduced game depends only on η 1 since taking the infimum over

τ ∈ T ′ (π), inf τ ∈T ′ (π) sup σ∈Σ ′ (π) γ(π, σ, τ ) (4.1) = ∆(∆(K)) inf τ (y 1 ) ∆(K) sup σ(x 1 ,y 1 ) γ θ (•, σ(x 1 , y 1 ), τ (y 1 )), x 1 R K dL µ (x 1 | y 1 ) dµ(y 1 ), (4.2)
which depends only on the law of y 1 , since

y 1 = L µ (x 1 | d 1 ) = L µ (x 1 | y 1 ).
Let us prove a dual equality starting with σ ∈ Σ ′ (π) and τ ∈ T :

γ θ (π, σ, τ ) = K×N×N×∆(K)×∆(∆(K)) γ θ (k 1 , σ(x 1 , y 1 ), τ (d 1 ))dµ(k 1 , c 1 , d 1 , x 1 , y 1 ), = N×N×∆(K)×∆(∆(K)) K γ θ (k 1 , σ(x 1 , y 1 ), τ (d 1 ))dL µ (k 1 |c 1 , d 1 , x 1 , y 1 ) dµ(c 1 , d 1 , x 1 , y 1 ) = N×N×∆(K)×∆(∆(K)) γ θ (•, σ(x 1 , y 1 ), τ (d 1 )), x 1 R K dµ(c 1 , d 1 , x 1 , y 1 ) = N×∆(K)×∆(∆(K)) γ θ (•, σ(x 1 , y 1 ), τ (d 1 )), x 1 R K dµ(d 1 , x 1 , y 1 ) = N×∆(∆(K)) ∆(K) γ θ (•, σ(x 1 , y 1 ), τ (d 1 )), x 1 R K dL µ (x 1 | d 1 , y 1 ) dµ(d 1 , y 1 ).
For the second equality, we used that

L µ (k 1 | c 1 , d 1 , x 1 , y 1 ) = L µ (k 1 | c 1 , d 1 ) = L µ (k 1 | c 1 ) = x 1
which follows from the fact that (x 1 , y 1 ) is c 1 -measurable and assumption (A1). Taking the infimum over all τ ∈ T , it follows that inf

τ ∈τ γ(π, σ, τ ) = N×∆(∆(K)) inf τ (d 1 ) ∆(K) γ θ (•σ(x 1 , y 1 ), τ (d 1 )), x 1 R K dL µ (x 1 | d 1 ) dµ(d 1 , y 1 ). (4.3)
The infimum inside the integral is achieved for strategies depending only on

y 1 = L µ (x 1 | d 1 ) since y 1 is d 1 -measurable. We proved that sup σ∈Σ ′ (π) inf τ ∈T γ θ (π, σ, τ ) = sup σ∈Σ ′ (π) inf τ ∈T ′ (π)
γ θ (π, σ, τ ).

Finally, using that T ′ (π) ⊂ T and Σ ′ (π) ⊂ Σ, it follows that

v θ (π) = sup σ∈Σ inf τ ∈T γ θ (π, σ, τ ) ≥ sup σ∈Σ ′ (π) inf τ ∈T γ θ (π, σ, τ ) = v θ (π), v θ (π) = inf τ ∈T sup σ∈Σ γ θ (π, σ, τ ) ≤ inf τ ∈T ′ (π) sup σ∈Σ γ θ (π, σ, τ ) = v θ (π),
which proves the equality. Since v θ (π) depends only on η 1 , the proof is complete.

Player 1 can compute his beliefs without knowing player 2's strategy.

Assumption (A1) can be reformulated as a couple of assumptions (A1a) and (A1b) which are expressed in terms of π, i.e. the initial information, and q, i.e. the evolution of the information structure for stages m ≥ 2 respectively.

(A1a) The probability π ∈ ∆(K × C ′ × D ′ ) is such that ∀(k, c ′ , d ′ ) ∈ K × C ′ × D ′ , π(c ′ )π(k, c ′ , d ′ ) = π(k, c ′ )π(c ′ , d ′ ) (A1b) There exists a map F from ∆(K) × I × C to ∆(K) such that ∀(p, i, j, c, d, k) ∈ ∆(K)×I ×J ×C ×D×K, q(p, i, j)[k, c, d] = F (p, i, c)[k] k ′ ∈K q(p, i, j)[k ′ , c, d].
Note that (A1a) is equivalent to π(k|c ′ , d ′ ) = π(k|c ′ ) for any (k, c ′ , d ′ ) such that π(c ′ , d ′ ) > 0. Similarly (A1b) could be written in terms of conditional probabilities, though we shall distinguish events with probability 0. In addition, it highlights the first important consequence of assumption (A1): player 1 can compute his beliefs about the state variable (i.e. the conditional distribution in the right-hand-side of (A1)) without knowing the strategy, nor the signals, of his opponent. Proposition 4.5. Assuming (A1), player 1 can compute x n for each n ≥ 1 without knowing the strategy of player 2.

The proof of the Proposition follows directly from the following Lemma. Lemma 4.6. Assumptions (A1) and (A1a + A1b) are equivalent. Furthermore, the map F from ∆(K) × I × C to ∆(K) defined in (A1b) is such that for all n ≥ 2 and for all strategy profile (σ, τ )

x n = F (x n-1 , i n-1 , c n-1 ), P π στ -almost surely. Proof. Using the definition of conditional independence, assumption (A1) at stage 1 is equivalent to (A1a). It remains to prove that (A1) for n ≥ 2 implies (A1b) and the converse. Assume that π fulfils (A1a) and let (σ 1 , τ 1 ) ∈ ∆(I) C ′ × ∆(J) D ′ be strategies with full support. By construction, we have

L P π στ (k 2 , c 2 , d 2 | k 1 , c 1 , i 1 , d 1 , j 1 ) = q(k 1 , i 1 , j 1 ) ∈ ∆(K × C × D).
It follows, using the tower property of conditional expectation and (A1a) that

L P π στ (k 2 , c 2 , d 2 | c 1 , i 1 , d 1 , j 1 ) = q(x 1 , i 1 , j 1 )
, where, by definition, x 1 can be written as a function of c 1 . On one hand, one obtains by disintegration

P π στ (k 2 = k | c 2 , d 2 , c 1 , i 1 , d 1 , j 1 )( k∈K q(x 1 , i 1 , j 1 )[ k, c 2 , d 2 ]) = q(x 1 , i 1 , j 1 )[k, c 2 , d 2 ].
On the other hand, the conditional law

L P π στ (k 2 | c 1 , i 1 , c 2 )

is characterized by the following expression

P π στ (k 2 = k | c 1 , i 1 , c 2 )( k, d 1 , d 2 , j 1 π(c 1 , d 1 )τ 1 ( d 1 )[ j 1 ]q(x 1 (c 1 ), i 1 , j 1 )[ k, c 2 , d 2 ]) = d 1 , d 2 , j 1 π(c 1 , d 1 )τ 1 ( d 1 )[ j 1 ]q(x 1 (c 1 ), i 1 , j 1 )[k, c 2 , d 2 ].
Assumption (A1) for n = 2 implies that these two conditional probabilities are equal, which in turn implies

q(x 1 , i 1 , j 1 )[k, c 2 , d 2 ] k∈K q(x 1 , i 1 , j 1 )[ k, c 2 , d 2 ] = d 1 , d 2 , j 1 π(c 1 , d 1 )τ 1 ( d 1 )[ j 1 ]q(x 1 (c 1 ), i 1 , j 1 )[k, c 2 , d 2 ] k, d 1 , d 2 , j 1 π(c 1 , d 1 )τ 1 ( d 1 )[ j 1 ]q(x 1 (c 1 ), i 1 , j 1 )[ k, c 2 , d 2 ] (4.4)
whenever the left-hand side is well-defined. Since τ 1 has full support, this implies that the right-hand side is also well-defined in this case and does not depend on d 1 , j 1 , d 2 . Moreover, for all p ∈ ∆(K), we can choose an initial distribution π such that π(x 1 = p) > 0. It follows that there exists a function F such that

F (p, i, c)[k] = q(p, i, j)[k, c, d] k ′ ∈K q(p, i, j)[k ′ , c, d] ,
whenever the right hand side is well-defined for some (j, d) and extended by 1/|K| (say) otherwise.

For the converse assertion, we already mentioned that (A1a) implies (A1) for n = 1. We are therefore allowed to write the following formula for the conditional laws,

P(k 2 = k | c 2 , d 2 , c 1 , i 1 , d 1 , j 1 ) = q(x 1 , i 1 , j 1 )[k, c 2 , d 2 ] k∈K q(x 1 , i 1 , j 1 )[ k, c 2 , d 2 ] . (4.5)
It follows therefore that

P(k 2 = k | c 2 , d 2 , c 1 , i 1 , d 1 , j 1 ) = F (x 1 , i 1 , c 2 ), P π στ -almost surely,
and since the right-hand-side is measurable with respect to the history of player 1, we have the equality

P(k 2 = k | c 2 , c 1 , i 1 ) = E P(k 2 = k | c 2 , d 2 , c 1 , i 1 , d 1 , j 1 )|c 1 , i 1 , c 2 , = F (x 1 , i 1 , c 2 ), = P(k 2 = k | c 2 , d 2 , c 1 , i 1 , d 1 , j 1 ).
which proves (A1) and our last assertion for n = 2. Finally the distribution of (k 2 , (c 1 , i 1 , c 2 ), (d 1 , j 1 , d 2 )), seen as an element of ∆ f (K × N × N), fulfils (A1a). Applying exactly the same argument with these new initial signals allows us therefore to conclude by induction on n.

Player 1 can compute the beliefs of player 2.

The assumptions (A1) and (A2) are independent, as shown in example 4.7 below. However, (A2) really makes sense only when player 1 is better informed.

Example 4.7. Let Γ = (K, I, J, C, D, q, g) be such that player 1 is in the dark and player 2 is perfectly informed: K = {α, β}, I and J are finite, C is a singleton {c} and D = K. The payoff mapping is anything and the state is randomly chosen at each stage with probability (1/2, 1/2). Player 1 observes nothing and player 2 learns the state. It is clear that player 1's signal is less accurate than player 2's, so that assumption (A1) is not satisfied. On the other hand, (A2) is satisfied since player 1 knows the beliefs of player 2 about himself which is ( 1 2 , 1 2 ) whatever are the signals.

Under the assumptions (A1) and (A2), if player 1 plays a reduced strategy, he can compute y 2 , the belief of player 2 about his own belief on the state, without knowing the strategy of player 2. Lemma 4.8. Assume (A1b) and (A2b), and let π ∈ ∆ * f (K × N × N). Then, for all σ ∈ Σ ′ (π), there exists a map f 2 = f π,σ 2 such that for all τ ∈ τ

y 2 = f 2 (h I 2 ), P π στ -almost surely.
Proof. It is sufficient to prove that the map f 2 appearing in (A2b) does not depend on τ . Note that since we assumed (A1), we have x 2 = F (x 1 (c 1 ), i 1 , c 2 ) almost surely, where F is defined in (A1b). Moreover, the conditional probability

P π στ (c 1 = c1 , i 1 = ĩ1 ,c 2 = c2 |d 1 , j 1 , d 2 ) = π(c 1 , d 1 )σ(x 1 ( c1 ), y 1 (d 1 ))( ĩ1 )τ (d 1 )(j 1 )q(x 1 (c 1 , ĩ1 , j 1 )(c 2 , d 2 ) c ′ 1 ,i ′ 1 ,c ′ 2 π(x ′ , d 1 )σ(x 1 (c ′ 1 ), y 1 (d 1 ))(i ′ 1 )τ (d 1 )(j 1 )q(x 1 (c ′ 1 ), i ′ 1 , j 1 )(c ′ 2 , d 2 ) = π(c 1 , d 1 )σ(x 1 ( c1 ), y 1 (d 1 ))( ĩ1 )q(x 1 (c 1 , ĩ1 , j 1 )(c 2 , d 2 ) c ′ 1 ,i ′ 1 ,c ′ 2 π(x ′ , d 1 )σ(x 1 (c ′ 1 ), y 1 (d 1 ))(i ′ 1 )q(x 1 (c ′ 1 ), i ′ 1 , j 1 )(c ′ 2 , d 2 )
does not depend on τ . There exists therefore a map y 2 (d 1 , j 1 , d 2 ) which does not depend on τ , defined by the above expression everywhere it makes sense and arbitrarily elsewhere. Let τ * be a strategy with full support. Using (A2b), there exists a map f π,σ,τ * 2 such that

y 2 (d 1 , j 1 , d 2 ) = f π,σ,τ * 2 (c 1 , i 1 , c 2 ), P π στ * -almost surely.
The previous computation shows that the conditional law of (c 1 , i 1 , c 2 ) given (d 1 , j 1 , d 2 ) does not depend on τ . Therefore, if the event

{d 1 = d1 , j 1 = j1 , d 2 = d2 , c 1 = c1 , i 1 = ĩ1 , c 2 = c2 }
has positive probability under P π στ , it also has positive probability under P π στ * . We deduce that f π,σ,τ 2 = f π,σ,τ * 2 , P π στ -almost surely for all τ , which concludes the proof.

Let us now prove that player 1 is able to play a strategy which is Markovian with respect to the beliefs. The idea is to prove by induction that if player 1 plays a strategy which depends at stage n -1 only on (x n-1 , y n-1 ), then he can compute the variables (x n , y n ) at stage n and play at stage n a strategy which depends only on (x n , y n ), etc... Formally, we have the following. Lemma 4.9. For all π ∈ ∆ * f (K × N × N), and for any sequence of ∆(I)-valued measurable functions ψ 1 , ψ 2 , ... defined on ∆(K) × ∆ f (∆(K), there exists a strategy σ such that for all τ and for all n σ(h

I n ) = ψ n (x n , y n ), P π στ -almost surely.
Proof. We will prove the result by induction. It is obviously true for n = 1 due to the definition of ∆ * f (K × N × N). For n = 2, due to the Lemmas 4.6 and 4.8, player 1 can compute x 2 and y 2 as a function of h I 2 independently of the chosen strategy of player 2. However, to prove the property for n ≥ 3, we cannot rely on the same argument. It would be tempting to say that the distribution of (k 2 , h I 2 , h II 2 ) belongs to ∆ * f (K × N × N) and to apply the preceding argument when starting from this new initial distribution. But this would be wrong since this distribution may depend on τ . To overcome this problem, it is sufficient to prove that Proof. It follows from the definitions and from Lemma 4.6 that

L P π στ (x 2 , d 2 | c 1 , d 1 , i 1 , j 1 ) = L P π στ (F (x 1 , i 1 , c 2 ), d 2 | c 1 , d 1 , i 1 , j 1 ) = L P π στ (H x 1 ,i 1 (c 2 , d 2 ) | c 1 , d 1 , i 1 , j 1 ) = φ(x 1 , i 1 ), since (x 1 , i 1
) is measurable with respect to (c 1 , d 1 , i 1 , j 1 ) and φ(x 1 , i 1 ) is the image probability of q C×D (x 1 , i 1 , j 1 ) by the map H x 1 ,i 1 . Therefore, the conditional law of the pair (x 2 , d 2 ) does not depend on the strategy of player 2. Precisely, we have

L P π στ (x 2 , d 2 | d 1 , j 1 ) = E π στ [φ(x 1 , i 1 ) | d 1 , j 1 ].
Since j 1 and (x 1 , i 1 ) are conditionally independent given d 1 it follows that

L P π στ (x 2 , d 2 | d 1 , j 1 ) = E π στ [φ(x 1 , i 1 ) | d 1 ].
The right hand side does not depend on τ , so L P π στ (x 2 , d 2 | d 1 , j 1 ) does not depend on τ and j 1 , and the same is true for the (unconditional) law of (x 2 , y 2 ). As a consequence, the law of y 2 (denoted η 2 ) does not depend on τ which proves (A3). It remains to prove that player 1 can compute the auxiliary random variable y 2 . Using (A2a) and that σ is reduced, i 1 can be written as a measurable function of (x 1 , y 1 ) and of an independent random variable u uniformly distributed on [0, 1]. Recall that the conditional law of x 1 given d 1 is y 1 , so that

L P π στ (x 2 , d 2 | d 1 , j 1 ) = E π στ [φ(x 1 , i 1 (x 1 , y 1 , u)) | d 1 ] = ∆(K)×[0,1] φ(x, i 1 (x, y 1 , u)dy 1 [x]du.
Player 1 can compute the conditional law of (x 2 , d 2 ) given (d 1 , j 1 ) since it depends only on (y 1 , σ). Moreover by assumption (A ′ 2b), he can deduce d 1 from his initial signal c 1 , so he is able to compute y 2 which proves (A2b).

5 Proof of Theorem 2.3.

The proof is divided into three steps. First, using Lemma 4.4, we define a value function v on ∆ f (∆ f (∆(K))) and prove that it is concave and Lipschitz. Secondly, we introduce an auxiliary game G on ∆ f (∆(K)) and check it satisfies some (slightly) weakened assumptions needed to apply a Theorem of Renault [START_REF] Renault | The value of repeated games with an informed controller[END_REF]. This implies the existence of a uniform value in the auxiliary game. Finally we show that both players can guarantee this value in the original game: player 2 by playing by blocks and player 1 by using optimal Markovian strategies in the auxiliary game.

The canonical value function vθ

In view of Lemma 4.4, it is appropriate to work directly on the set ∆ f (∆ f (∆(K))), i.e. for any π, π ′ such that Φ(π) = Φ(π ′ ) the value of the game is the same. At first, given

η ∈ ∆ f (∆ f (∆(K)))
, there is a canonical way to build a distribution π such that φ(π) = η.

Definition 5.1. Let Γ = (K, I, J, C, D) be a repeated game. For any

η ∈ ∆ f (∆ f (∆(K))), we define D ′ = supp(η) ⊂ ∆ f (∆(K)) and C ′ = D ′ × ∪ z∈supp(η) supp(z)
. By definition of η, these sets are finite and we can define

π ∈ ∆ * f (K × C ′ × D ′ ) by ∀(k, p, z) ∈ K × ∆(K) × ∆ f (∆(K)), π(k, (p, z), z) = η(z)z(p)p(k).
To canonical game Γ(π) will be denoted Γ(η), and its value vθ (η). If η = δ z for some z ∈ ∆ f (∆(K)), we will use the shorter notations Γ(z) = Γ(δ z ) and vθ (z) for the value.

Informally, the game Γ(η) proceeds as follows: η is common knowledge, player 2 is informed about the realization z of a random variable of law η (player 2 learns his beliefs). Then player 1 is informed about z (his opponent's beliefs) and about the realization p of a random variable of law z (his own beliefs). The state variable is finally selected according to p, but none of the players observe it. If η = δ z , for some z ∈ ∆ f (∆(K)), then the set of initial signals for player 2 is reduced to a singleton. In this case, player 1 receives a partial information about the state, whereas player 2 only knows the joint distribution over the state and player 1's signal. Using these notations, Lemma 4.

4 implies that if π, π ′ ∈ ∆ * f (K × N × N) are such that Φ(π) = Φ(π ′ ), we have that v θ (π) = v θ (π ′ ) = vθ (Φ(π)).
In order to study the regularity of the canonical value function, let us recall some properties of the Wasserstein distance Let (Z, d) be a compact metric space and Lip 1 (Z) the set of 1-Lipschitz functions on Z. The function where P(µ, ν) is the set of probabilities on Z × Z having for marginals µ and ν (see e.g. [START_REF] Villani | Topics in Optimal Transportation[END_REF]).

d : ∆(Z) × ∆(Z) : (µ, ν) → sup f ∈Lip 1 (Z) Z f dµ - Z f dν
If f is a bounded measurable function on Z, define f :

∆(Z) → R by f (µ) = Z f dµ. Then f ∈ Lip 1 (∆(Z), d) ⇔ f ∈ Lip 1 (Z).
In the following, ∆(K) is endowed with the ℓ 1 -norm induced by R K and ∆(∆(K)) is endowed with the Wasserstein metric d induced by the metric space (∆(K), ℓ 1 ).

Lemma 5.2. Let η ∈ ∆ f (∆ f (∆(K))) and z ∈ ∆ f (∆(K)). Then vθ (η) is linear on ∆ f (∆ f (∆(K)))
and the mapping on ∆(∆(K)), vθ (z) is 1-Lipschitz for the Wasserstein metric d.

Proof. The first assertion is immediate since by definition both players learn the realization of η. Let z,z ′ ∈ ∆ f (∆(K)). By definition of the Wasserstein distance, there exists µ ∈ ∆(∆(K) × ∆(K)) such that the first marginal is z, the second is z ′ and

d(z, z ′ ) = ∆(K)×∆(K) p -p ′ 1 dµ(p, p ′ ).
We denote by L µ (p|p ′ ) the conditional law of p given p ′ .

Let σ ∈ Σ be a behavior strategy for player 1 in the game Γ(z). As in Section 2.2, σ(p) denotes the strategy of player 1 conditionally on the signal p. Let us construct a general strategy for P1 as follows. Let (Ω, P) = ([0, 1], dx) be the auxiliary probability space3 that will be used as a "tossing coin". The classical representation result of Blackwell-Dubins (see [START_REF] Blackwell | An extension of Skorohod's almost sure representation theorem[END_REF]) asserts that there exists a jointly Borel-measurable map φ : Ω × ∆(∆(K)) → ∆(K) such that for all ν ∈ ∆(∆(K)), φ(•, ν) is a ν-distributed random variable. Therefore, the map σ ′ (ω, p ′ ) = σ(φ(ω, µ(p|p ′ ))) defines a general strategy which is equivalent to a behavior strategy by Kuhn's theorem. It follows that

γ θ (z ′ , σ ′ , τ ) = ∆(K)×Ω γ θ (p ′ , σ ′ (ω, p ′ ), τ )dz ′ (p ′ ) ⊗ dP(ω), = ∆(K) Ω γ θ (p ′ , σ(φ(ω, µ(p|p ′ ))), τ )dP(ω) dz ′ (p ′ ), = ∆(K) ∆(K) γ θ (p ′ , σ(p), τ )dL µ (p|p ′ ) dµ(p ′ ), = ∆(K)×∆(K) γ θ (p ′ , σ(p), τ )dµ(p, p ′ ),
where the last equality follows from dz ′ (p ′ ) = dµ(p ′ ). Recall that by assumption g takes values in [0

Hence

|γ θ (z ′ , σ ′ , τ ) -γ θ (z, σ, τ )| ≤ ∆(K)×∆(K) |γ θ (p, σ(p), τ ) -γ θ (p ′ , σ(p), τ )|dµ(p, p ′ ), ≤ ∆(K)×∆(K) p -p ′ 1 dµ(p, p ′ ), = d(z, z ′ ). It follows that |v θ (z) -vθ (z ′ )| ≤ d(z, z ′ ), for any z, z ′ ∈ ∆ f (∆(K)).
Note that usually, the underlying space is ∆(K) with discrete metric on K and, in order to prove that the value is 1-Lipschitz, we can use the same strategy in Γ(z) in Γ(z ′ ). Here, we cannot use directly σ. The state space is ∆ f (∆(K)) with the norm 1 on ∆(K), and two states may be close while having disjoint supports. Therefore an optimal strategy σ in Γ(z) may have no sense in z ′ . The idea behind the above proof is to construct, given σ in Γ(z), a strategy σ ′ in Γ(z ′ ) which behaves in z ′ like σ in z.

Example 5.3. Assume that K = {k 1 , k 2 } and let z = δ 1 2 and z ′ = 1 2 δ 1 2 -ǫ + 1 2 δ 1
2 +ǫ be two initial distributions in ∆ f (∆(K) (where we identified ∆(K) and [0, 1]). A strategy in Γ(z) is defined only at 1 2 since it can be modified elsewhere without altering the payoff. Therefore an optimal strategy σ in in Γ(z) can play anything in 1 2 -ǫ and in 1 2 + ǫ since no regularity for σ is required. The good way to use the proximity between z and z ′ is to always play as if the initial distribution was 1 2 . Here we have to define σ ′ such that for all z ∈ ∆(K), σ ′ (z) = σ( 1 2 ).

Lemma 5.4 (Splitting procedure). The mapping vθ (z) is concave on ∆ f (∆(K)).

Proof. We follow the same scheme as for games with incomplete information on one side (see e.g. [START_REF] Mertens | Repeated Games, CORE Discussion Papers 9420[END_REF] Corollary 1.3 p.184). Let λ ∈ [0, 1] and let Y be a random variable with values in {0, 1}, such that P(Y = 0) = λ. Let z, z ′ ∈ ∆ f (∆(K)). The random variable P is selected according to the distribution z if Y = 0 and z ′ if Y = 1, the state variable k 1 is finally selected according to p if P = p. Compare now the two following situations: on one hand, the game with initial signals (Y, P ) for player 1 and nothing for player 2 and on the other hand the game with initial signals (Y, P ) for player 1 and Y for player 2. These two distributions of initial signals and states fulfill our assumptions and it's clear that the value of the second is less or equal than the value of the first for any evaluation θ ∈ ∆ f (N * ) since the set of behavior strategies of player 2 in the second game is larger than in the first game. Translating this inequality using v, we deduce directly

vθ (δ λz+(1-λ)z ′ ) ≥ vθ (λδ z + (1 -λ)δ z ′ ) = λv θ (z) + 1 -λv θ (z ′ ),
which proves the Lemma.

Auxiliary game G

Let X = ∆ f (∆(K)) be the state space, which corresponds to player 2's belief about player 1's belief about the current state. It is a convex relatively compact subset of a normed vector space and we are going to express the auxiliary game and the recursive formula on this state space.

Let G be the stochastic game defined by

• the state space X = ∆ f (∆(K)),

• the action space A = {f : ∆(K) → ∆(I), measurable} for player 1,

• the action space B = ∆(J) for player 2,

• the payoff function G :

X × A × B → [0, 1] defined, for any z ∈ X by G(z, a, b) = p∈supp(z) (i,j)∈I×J b(j)a(p, i)g(p, i, j))z(p),
where supp(z) stands for the support of z,

• the transition function ℓ :

X × A × B → ∆ f (X) is defined as ℓ(z, a, b) = Φ(Q(z, a, b)), where Q(z, a, b) ∈ ∆ f ((K) × (∆(K) × C) × (D)) is the induced joint distribution of (k 2 , (p, i 1 , c 2 ), (j 1 , d 2 )
) in the canonical game Γ(δ z ) where players play at the first stage σ 1 = a and τ 1 = b. The sets C, D, K and supp(z) being finite and using assumptions (A1) and (A2), we may consider Q as an element in ∆ * f (K × N × N). Let us recall the definition of Choquet order on ∆ f (X).

Definition 5.5. The order ≤ on ∆ f (X) called (reversed) Choquet order is defined by the relation µ ≤ ν ⇔ For all continuous concave function on X, f (µ) ≤ f (ν).

We aim to apply a weakened version of Renault [START_REF] Renault | The value of repeated games with an informed controller[END_REF] to the game G, thus let us first recall the hypotheses of the Theorem as they appear in the original article. The main consequence of assumption (A3) is that player 2 cannot influence the transition in the auxiliary game so the map ℓ does not depend on b, i.e.

∀(z, a) ∈ X × A, ∀b, b ′ ∈ B, ℓ(z, a, b) = ℓ(z, a, b ′ ).
Thus (H1) is satisfied and from now on, we will work under the shorter notation l(z, a) for l(z, a, b).

The hypotheses (H2, H3, H4, H6, H7) ensure the application of Sion's theorem in several steps of Renault's proof. Here they are not all satisfied since, for example, the set A is not compact. However, it is well known that adding some geometrical hypotheses allows to weaken the topological assumptions in Sion's theorem (see, for instance, Proposition A.8 in Sorin's monography [START_REF] Sorin | A first course on zero-sum repeated games[END_REF]). For instance, if A is a convex set, B is a compact convex subset of a topological vector space, (a

→ G(z, a, b)) is concave ∀(z, b) ∈ X × B and (b → G(z, a, b))
is convex and lower semi-continuous ∀(z, a) ∈ X × A, Sion's result applies to the one-stage game: the game G 1 (z) has a value. They can be replaced without altering the proof by the following hypotheses.

Hypotheses 5.2.

H2') X is a relatively compact convex subset of a normed vector space. H3') B is a convex compact subset of a topological vector space, A is a convex set. Assumption (H2 ′ ) is satisfied since the Wasserstein distance can be extended to a norm on the space of finite signed measures. Moreover assumptions (H3 ′ ) and (H4 ′ ) are clearly satisfied. Therefore, we need to prove (H6 ′ ) and (H7).

Lemma 5.6. The game G fulfills H6 ′ and H7.

Proof. Let z be a convex combination in X, z = S s=1 λ s z s and (a s ) s∈S be a family of actions in A S . Denote µ(z s , a s ) ∈ ∆ f (∆(K) × I) the joint law induced on ∆(K) × I by (z s , a s ). By disintegration, there exists a ∈ A such that µ(z, a) = s∈S λ s µ(z s , a s ). A first, note that

Q(z, a, b) = s∈S λ s Q(z s , a s , b).
Given (z, a, b), we consider the canonical game Γ(z). In this game, a pair (k, p) is chosen according to the probability π ∈ ∆ f (K × ∆(K)) defined by π(k, p) = p k z(p) for all (k, p) ∈ K × ∆(K). Then, player 1 receives the signal c 1 = p and player 2 receives no initial signal. We associate to (a, b) a pair strategies for the first stage (σ 1 , τ 1 ) by σ 1 (p) = a(p) and τ 1 = b. Then, Q(z, a, b) denotes the joint distribution of (k 2 , (p, i 1 , c 2 ), (j 1 , d 2 )). Since the conditional law of (k 2 , c 2 , d 2 ) given (p, i 1 , j 1 ) is q(p, i 1 , j 1 ), it follows that Q(z, a, b) is bilinear with respect to (µ(z, a), b) (with abusive notations).

Let

ρ = Q(z, a, b) (resp. ρ s = Q(z s , a s , b)) and c ′ = (p, i 1 , c 2 ) (resp. d ′ = (j 1 , d 2 )). Let C ′ = (∪ s∈S supp(z s )) × I × C and D ′ = J × D. By construction, ℓ(z, a) = Φ(ρ) = L ρ (L ρ (L ρ (k 2 | c ′ ) | d ′ )) = d ′ ∈D ′ ρ(d ′ )δ Lρ(Lρ(k 2 |c ′ )|d ′ ) .
Using Lemma 4.6, we have the following equality ρ-almost surely

L ρ (k 2 | p, i 1 , c 1 ) = F (p, i 1 , c 2 ). This implies that L ρ (L ρ (k 2 | c ′ ), d ′ ) = L ρ (F (c ′ ), d ′ ) ∈ ∆ f (C ′′ × D ′ )
where C ′′ = F (C ′ ). A similar equality holds with ρ s instead of ρ for all s ∈ S. By definition of l(z, a), we deduce that

ℓ(z, a) = L ρ (L ρ (F (c ′ ) | d ′ )) = Ψ(L ρ (F (c ′ ), d ′ )),
where Ψ is the disintegration map defined by

Ψ : ∆(C ′′ × D ′ ) → ∆ f (∆(C ′′ )) : m → d ′ ∈D ′ m(d ′ )δ Lm(c ′′ |d ′ ) ,
where L m (c ′′ |d ′ ) denoted the conditional law of c ′′ given d ′ . It was proved in Renault [START_REF] Renault | The value of repeated games with an informed controller[END_REF] (Lemma 4.16) that Ψ is concave for the Choquet order on ∆ f (∆(C ′′ )). However, C ′′ being a finite subset of ∆(K), ∆(C ′′ ) is identified as a compact convex subset of X. It follows easily that the convex order on ∆ f (∆(C ′′ )) coincides with the order induced by the convex order on ∆ f (X). We conclude that the first part of H7 holds since

l(z, a) = Ψ( s∈S λ s L ρs (F (c ′ ), d ′ )) ≥ s∈S λ s Ψ(L ρs (F (c ′ ), d ′ )) = s∈S λ s l(z s , a s ).
For the second part of H7, it is sufficient to note that (again with abusive notations) µ(z, a) →

G(z, a, b) is linear so that for all b ∈ B G(z, a, b) = s∈S λ s G(z s , a s , b),
which implies the result. Finally, in case z s = z for all s, the same arguments also imply (H6 ′ ) since in this case one can choose a = s∈S λ s a s in the above proof.

The proof of the following proposition follows from Proposition 3.21 in Renault [START_REF] Renault | The value of repeated games with an informed controller[END_REF].

Proposition 5.7. Assuming (H1, H2 ′ , H3 ′ , H4 ′ , H6 ′ , H7), then for any θ ∈ ∆ f (N * ) and any η

∈ ∆ f (X), the game G θ (η) has a value w θ (η) such that ∀z ∈ X, w θ (z) = sup a∈A min b∈B {θ 1 G(z, a, b) + (1 -θ 1 )w θ + (ℓ(z, a))} , (5.1)
= min b∈B sup a∈A {θ 1 G(z, a, b) + (1 -θ 1 )w θ + (ℓ(z, a))} , (5.2)
where θ + is defined by θ + t = θ t+1 m≥2 θm for t ≥ 1 whenever m≥2 θ m > 0 and is defined arbitrarily otherwise. Moreover, in G θ (η), player 1 has ε-optimal Markov strategies for all ε > 0 and player 2 has optimal Markov strategies.

In order to prove the last assumption (H5), we first prove that the value of the game G θ is equal to the canonical value function vθ . Since we proved that the canonical value is 1-Lipschitz, it will imply using the previous Proposition that the set of functions C = {v θ , θ ∈ ∆ f (N * )} satisfies (H5).

We now prove that the value functions of both games are the same. The proof is classic and consists to show that both families of functions are linked by the same recursive formula. Proposition 5.8. For all θ ∈ ∆ f (N * ) and for any z ∈ X, w θ (z) = vθ (z).

Corollary 5.9. The game G fulfills (H5).

Proof of Proposition 5.8. Notice first that v1 (z) = w 1 (z) for all z. This comes indeed almost from the definition v1 (z) = sup

σ 1 :∆(K)→∆(I) inf b∈∆(J) ∆(K) g(p, σ 1 (p), b)dz(p) = sup a∈A min b∈∆(J) G(z, a, b) = min b∈∆(J) sup a∈A G(z, a, b) = w 1 (z).
It is enough to prove that w and v satisfy the same recurrence formula. We will prove that v satisfies the recurrence formula in G, i.e. vθ (z) = sup

a∈A min b∈B θ 1 G(z, a, b) + (1 -θ 1 )v θ + (ℓ(z, a)) = min b∈B sup a∈A θ 1 G(z, a, b) + (1 -θ 1 )v θ + (ℓ(z, a)).
We prove the recursive formula by induction on the greatest element in the support of θ. If θ = δ 1 , it follows from the preceding equality. Fix now n ≥ 2, and assume that the proposition is true for every θ supported by {1, ..., n -1}. Let z ∈ ∆ f (∆(K)). We first prove that player 1 can defend in Γθ (z) the quantity

min b∈B sup a∈A (θ 1 G(z, a, b) + (1 -θ 1 )v θ + (ℓ(z, a))) . Using the canonical representation Γ, vθ (z) = v θ (π) where π ∈ ∆ f (K × ∆(K) × ∆ f (∆(K))) is defined by ∀(k, p, x) ∈ K × ∆(K) × ∆ f (∆(K)), π(k, p, x) = p(k)z(p)1 x=z .
Consider the game Γ θ (π). Let ε > 0 and τ be a strategy of player 2. Denoting by b the law induced by τ 1 , let a * ∈ A an action which realizes the supremum up to ε in the expression

θ 1 G(z, a, b) + (1 -θ 1 )v θ + (ℓ(z, a)).
Let σ * be an ε-optimal strategy in the game Γ θ + (Q(z, a * , b)). Define then σ by σ 1 = a * and for all n ∈ N * , h

I n = (p, i 1 , c 2 , ..., i n-1 , c n ), σ n (h I n ) = σ * n-1 (c ′ , h 1,+ n-1 ) where c ′ = (p, i 1 , c 2 ) and h 1,+ n-1 = (i 2 , c 3 , ..i n-1 , c n ). We have γ θ (µ, σ, τ ) = θ 1 G(z, a * , b) + (1 -θ 1 )γ θ + (Q(z, a * , b), σ * , τ + ),
where τ + is a continuation strategy. Precisely, for all n ∈ N * , τ

+ n-1 (d ′ , h 2,+ n-1 ) = τ n (h II n ) with h 2,+ n-1 = (j 2 , d 3 , .., j n-1 , d n ), h II n = (d ′ , h 2,+ n-1 ) and d ′ = (j 1 , d 2 )
is the "signal" for player 2 given by Q(z, a * , b).

Therefore, σ * and τ + can be seen as behavior strategies in a new game with initial signals corresponding to the past history in the original game and since σ * is ε-optimal in Γ(Q(z, a * , b)), we have

γ θ (µ, σ, τ ) ≥ θ 1 G(z, a * , b) + (1 -θ 1 )v θ + (Q(z, a * , b)) -ε ≥ sup a∈A θ 1 G(z, a, b) + (1 -θ 1 )v θ + (Q(z, a, b)) -2ε = sup a∈A θ 1 G(z, a, b) + (1 -θ 1 )v θ + (ℓ(z, a)) -2ε ≥ min b∈B sup a∈A θ 1 G(z, a, b) + (1 -θ 1 )v θ + (ℓ(z, a)) -2ε. It follows that vθ (z) ≥ min b∈B sup a∈A θ 1 G(z, a * , b) + (1 -θ 1 )v θ + (ℓ(z, a)) by sending ε to zero. Let us show that player 2 can defend sup a∈A min b∈B (θ 1 G(z, a, b) + (1 -θ 1 )v θ + (ℓ(z, a)) in Γ(µ).
Fix a strategy σ of player 1 and let a = σ 1 , there exists b * ∈ B achieving min b G(z, a, b). We also choose τ * an optimal strategy for player 2 in the game Γ θ + (Q(z, a, b * )). This defines a strategy τ such that

γ µ θ (σ, τ ) = θ 1 G(z, a, b * ) + (1 -θ 1 )γ Q(z,a,b * ) θ + (σ + , τ * ) ≤ θ 1 G(z, a, b * ) + (1 -θ 1 )v θ + (Q(z, a, b * )) = θ 1 G(z, a, b * ) + (1 -θ 1 )v θ + (ℓ(z, a)) = min b∈B θ 1 G(z, a, b) + (1 -θ 1 )v θ + (ℓ(z, a)).
Thus vθ (z) ≤ sup a∈A min b∈B (θ 1 G(z, a, b) + (1 -θ 1 )v θ + (ℓ(z, a)). Finally, since the maxmin is always smaller than the minmax, all the intermediate inequalities are equalities.

Existence of the uniform value

Let us at first recall the first main result proved in [START_REF] Renault | The value of repeated games with an informed controller[END_REF] which holds under our set of weakened assumptions.

Theorem 5.10 (Renault(2012)). Assume that H1, H ′ 2, H ′ 3, H ′ 4, H5, H ′ 6, H7 hold. Then for every initial distribution η ∈ ∆ f (X), the game has a uniform value w * (η). Moreover player 1 can guarantee w * (η) with a Markov strategy:

∀ǫ > 0, ∃σ ∈ Σ M , ∃N 0 ∈ N, ∀N ≥ N 0 ∀τ ′ ∈ τ , γ N (η, σ, τ ′ ) ≥ w * (η) -ǫ.

and we have w * (η) = inf n≥1 sup m≥0 w m,n (η).

In order to conclude the proof, we show that both players can guarantee Proof. Let σ be a behavior strategy in G ∞ (z). Let us describe the strategy σ. Player 1 plays at the first round in Γ ∞ (z) the mixed action σ 1 (z)(p) where p is his initial signal. Then, at round n, he plays the mixed action σ n (y n )(x n ). That this strategy is a well-defined strategy follows from Lemma 4.9.

It remains to prove that this strategy guarantees the same quantity as σ. Let us fix n ∈ N * , we will prove that there exists a best reply τ to σ in Γ n (z) which can be seen as a strategy τ in G θ (z) and such that γ θ (z, σ, τ ) = γ θ (z, σ, τ ).

We will proceed by backward induction. Let us fix a best reply τ to σ in Γ θ (z). We will construct a strategy τ which depends at stage m on h II m only through y m . Recall that σ is fixed so that y m (h II m ) can be computed by player 2. At first let us replace τ n by

τ n (y n ) = E P z στ [τ (h II n ) | y n ].
Note that this conditional expectation depends on the strategies σ, τ up to stage n -1. Let us prove that the payoff at the last stage n is not modified. = E P z στ [ γ 1 (x, σ n (y n , x), τ n (y n ))dy n [x]] = E P z σ,(τ 1 ,...,τ n-1 , τn) [g(k n , i n , j n )]. The above equations show that the expected payoff at stage n when player 2 is playing the best reply (τ 1 , ..., τ n-1 , τ n ) against σ is a function of σ and of the law of y n . Assume now that at step m, we have proved that there exists a best reply to σ of player 2 such that the sum of expected payoffs for the stages m + 1, ..., n is a function of σ and of the law of (y m+1 , .., y n ) only. We can replace τ m (h II m ) by τ m (y m ) = E P z στ [τ m (h II m ) | y m ] without modifying the expected payoff of stage m with the same argument as above. Using assumption (A3), Lemma 4.9 and the definition of σ, the law of (y m+1 , ...y n ) is not modified by this operation which proves that this modified strategy is still a best reply to σ.

Secondly, we prove that Player 2 can guarantee v * (π) by splitting the stage in blocks and playing on each block separately since he has no influence on the transition. The following results are quite similar to the corresponding ones proved in Renault [START_REF] Renault | The value of repeated games with an informed controller[END_REF] and are reproduced here since their proofs are very short. Lemma 5.12. For every π ∈ ∆ * f (K ×C ′ ×D ′ ), n ≥ 1 and m ≥ 1, ∀ τ 1 , ..., τ m , ∃ τ m+1 , ..., τ m+n such that the strategy τ 1 , ..., τ m , ..., τ m+n of player 2 is optimal in the game Γ m,n .

Proof. Let π ∈ ∆(K×C ′ ×D ′ ), n ≥ 1 and m ≥ 0, and τ 1 , ..., τ m such that τ i : D ′ ×(J ×D) i-1 → ∆(J). We define T * the subset strategies of player 2 which start with τ 1 , ..., τ m and we consider the game with the evaluation θ m,n and the set of strategies Σ and T * . It can be seen as the mixed extension of a finite game, thus the value exists and will be denoted v * m,n (π). Since the set of strategies of player 2 is smaller than T , we have v m,n (π) ≤ v * m,n (π). But using the same method as for proving the recursive formula of Proposition 5.8, for any σ, we can build a strategy which defends v m,n (π). Both values are therefore equal and any optimal strategy in the restricted game satisfies the conclusion of the lemma. Proof. We prove that for all n ∈ N, Player 2 can guarantee the payoff sup m≥0 v m,n (p). Let n ∈ N be a number of stages, then for each L ∈ N we split the game of length nL in L blocks of length n: B 1 , ..., B L . We define the strategy τ * by induction on the block.

Let τ be an optimal strategy in Γ 1,n (π) then we set τ * i = τ i for all i ∈ {1, .., n}. Once we have constructed τ * 1 , ..., τ * nl for some 1 ≤ l ≤ L -1, we define the game Γ # nL+1,n (π) where the player 2 has to play τ * i for all i ≤ nl. We have v # nL+1,n (π) = v nL+1,n (π) using the preceding Lemma. Let τ be an optimal strategy in Γ # nL+1,n (π) and set τ * i = τ i for all i ∈ {nL + 1, ..., (n + 1)L}. We have The payoff being bounded, we deduce that this strategy guarantees v m,n (π). Finally, Player 2 can guarantee the minimum on n ∈ N, inf n∈N sup m≥0 v m,n (π) = v * (π).

Since each player can guarantee v * (π), the game has a uniform value given by v * (π) which concludes the proof of Theorem 2.3.

Example 4 . 1 .

 41 Let K = {k 1 , k 2 }, I = {T, B}, C = {a, b}, D = {α, β} and J any finite set.

  is a distance on ∆(Z) which makes ∆(Z) compact. Moreover, for all µ, ν ∈ ∆(Z) d(µ, ν) = min π∈P(µ,ν) Z×Z |y -x|dπ(x, y),

Hypotheses 5. 1 .

 1 H1) The map ℓ does not depend on b. H2) X is a compact convex subset of a normed vector space, H3) A and B are convex compact subsets of some topological vector spaces,H4) (a → G(z, a, b)) is concave upper semi-continuous ∀(z, b) ∈ X × B and (b → G(z, a, b)) is convex and lower semi-continuous ∀(z, a) ∈ X × A.H5) There exists a subsetC of 1-Lipschitz functions containing φ(1, 0) such that for all f in C, α ∈ [0, 1], the function φ(α, f ) is in C, where φ(α, f ) is defined by ∀z ∈ ∆ f (X) φ(α, f )(z) = sup a∈A min b∈B αG(z, a, b) + (1 -α) f (ℓ(z, a)) .H6) The mapping a → ℓ(z, a) is concave for the Choquet order and continuous. H7) (Splitting assumption) Let z be a convex combination in ∆ f (∆(K)), z = S s=1 λ s z s and (a s ) s∈S be a family of actions in A S . Then there exists a ∈ A such that ℓ(z, a) ≥ s∈S λ s ℓ(z s , a s ) and min b∈B G(z, a, b) ≥ s∈S λ s min b∈B G(z s , a s , b).

  H4') (a → G(z, a, b)) is concave ∀(z, b) ∈ X × B and (b → G(z, a, b)) is convex and lower semicontinuous ∀(z, a) ∈ X × A.H6') The mapping a → ℓ(z, a) is concave for the Choquet order.

v

  * (π) = inf n≥1 sup m≥0 v m,n (π),where v m,n (π) = v θm,n (π) and θ m,n is the uniform law between stage m and m + n.The game G(z) satisfies assumptions H1, ..., H7 ′ so it has a uniform value given byw * (z) = inf n≥1 sup m≥0 w m,n (z).And by proposition 5.8, the value in G and in the reduced game are equal, so ifπ ∈ ∆ * f (K × N × N) we have v * (π) = inf n≥1 sup m≥0 v m,n (π) = infn≥1 sup m≥0 vm,n (Φ(π)) = inf n≥1 sup m≥0 w m,n (Φ(π)) = w * (Φ(π)), Thus player 1 can guarantee v * (π) in G(Φ(π)) with a Markov strategy. Let us check that he can guarantee v * (π) in the game Γ(Φ(π)) or equivalently in Γ(π). Proposition 5.11. Any Markovian strategy σ of player 1 in G ∞ (z) induces a strategy σ in Γ∞ (z) guaranteeing the same amount.

Proposition 5 . 13 .

 513 For every π ∈ ∆ * f (K × C ′ × D ′ ), player 2 can guarantee v * (π) in the game Γ ∞ (π).

γ

  Ln (σ, τ * )

  and the distribution of y n by η n .Example 2.6. Let K = {k 1 , k 2 } be set of states space, U = {u 1 , u 2 } a set of public signals and S = {s 1 , s 2 , s 3 } a set of private signals for player 1. Using the notations above, let C = U × S (resp. D = U ) be the set of signals for player 2 (resp. 2). We consider π ∈ ∆(K × S × U ) If player 1 receives signal (s 1 , u 1 ), then x 1 = 1. If he receives (s 2 , u 1 ) or (s 2 , u 2 ), then x 1 = 1 2 . Finally, if he receives (s 3 , u 1 ) or (s 3 , u 2 ), then x 1 = 0. The value of x 1 depends only on his private signal.

	defined by
	s 1 s 2 s 3	u 1 u 2   8 24 0 1 24 3 24 0 0	 	s 1 s 2 s 3	u 1 u 2  0 0 24 24  1 24 3 2 6 24	  .
		k 1							k 2
	It is more convenient here to use K × S × U but π can be understood as a probability on
	K × C × D. To simplify notations, let us identify ∆(K) with [0, 1] with the convention that
	p ∈ ∆(K) is identified with p(k 1 ). We now compute the second order beliefs of player 2. If player 2 receives u 1 then his beliefs about the private signal of player 1 are 8 24 δ s 1 + 2 24 δ s 2 + 2 24 δ s 3 , so that
	y 1 (u 1 ) =	8 24	δ 1 +	2 24	δ 1 2	+	2 24	δ 0 .
	Definition 2.5. Put x n L P π στ (k n | h I n ), y n L P π στ (x n |h II n ), and η n L P π στ (y n ).
	Let us illustrate these definitions through the following example.

  The second order beliefs y 2 of player 2 will differ if his initial private signal is equal to α or β. Since player 1 is not able to compute the initial signal of player 2, he is not able to compute the variable y 2 at the second stage. Nevertheless, when considering reduced strategies, signals can be omitted and player 1 is able to compute the beliefs of player 2 which implies that (A2) is satisfied.

		α β
	a b	1/6 2/6 2/6 1/6	.

each stage (including at the initial stage), the signals of the players are randomly chosen independently of the state variable with distribution It is clear that signals do not contain any information on the state variable. However, assume that the initial state is k 1 and that player 1 plays at the first stage action T if he receives the signal a and action B if he receives the signal b.

  Example 4.2. We consider again the situation of example 2.6. Recall that K = {k 1 , k 2 }, that there are two public signals U = {u 1 , u 2 } available to both players and three private signals S = {s 1 , s 2 , s 3 } for player 1. The set of signals of player 1 is C = U × S and the set of signals of player 2 is D = U . Let π, π ′ ∈ ∆(K × S × U ) be defined by

		s 1 s 2 s 3	u 1 u 2   8 24 0 1 24 3 24 0 0	 	s 1 s 2 s 3	u 1 u 2  0 0 24 24  1 24 3 2 6 24	  .
			k 1			k 2
	and	s 1 s 2 s 3	u 1 u 2  6 2 24 24 3 1  24 24			u 1 u 2

  E P z στ [g(k n , i n , j n )] = E P z στ [E P z στ [g(k n , i n , j n ) | h I n , h II n ]] = E P z στ [γ 1 (x n , σ n (y n , x n ), τ n (h II n ))] = E P z στ [E P z στ [γ 1 (x n , σ n (y n , x n ), τ n (h II n )) | h II n ]] = E P z στ [ γ 1 (x, σ n (y n , x), τ n (h II n ))dy n [x]] = E P z στ [E P z στ [ γ 1 (x, σ n (y n , x), τ n (h II n ))dy n [x] | y n ]] = E P z στ [ γ 1 (x, σ n (y n , x), E P z στ [τ n (h II n ) | y n ])dy n [x]]

All random variables appearing here take only finitely many values so that the definition of conditional laws does not require any additional care about measurability.

One may easily reduce the analysis to a smaller set of initial probabilities, but we chose to keep this general formulation since the reduction does not really simplify the proofs.

Using a continuum of alternatives is clearly unnecessary but allows to simplify the proof.

Acknowledgements : The authors gratefully acknowledge the support of the Agence Nationale de la Recherche, under grant ANR JEUDY, ANR-10-BLAN 0112. The third author acknowledges the support of the the Israel Science Foundation under Grant ♯1517/11. Part of this work was done when the third author was Ph.D. student at the Université Toulouse 1 Capitole.

the map f 2 appearing in (A2b) and the distribution η 2 appearing in (A3) depend on π only through Φ(π). Indeed, in this case, reasoning by induction, player 1 can compute η n as a function of η n-1 and his new signals, x n using Lemma 4.6, and y n using the map given by (A2b) which will depend only on η n-1 and his own strategy.

Let us prove this assertion. Let π ∈ ∆ * f (K × N × N), and σ ∈ Σ ′ (π) be a reduced strategy, which implies that there exists a map ψ : K × ∆(K) → ∆(I) such that π-almost surely σ 1 (c 1 ) = ψ(x 1 , y 1 ). Assumption (A3) implies that η 2 is a function of the initial distribution π and σ 1 only. We denote it by η 2 (π, σ). We now prove that η 2 (π, σ) and the map f π,σ 2 appearing in (A2b) depend only on the projection of π, η 1 = Φ(π), and on the map ψ.

At first, given

) almost surely and y 2 has law η 2 (π, ψ) for all τ . Recall that π is such that Φ(π) = η 1 and that σ 1 is such that σ 1 (c 1 ) = ψ(x 1 , y 1 ). Note also that d 1 and x 1 are conditionally independent given y 1 under π. Therefore, for any τ 1 , the joint law of (x 1 , y 1 , i 1 , j 1 , c 2 , d 2 ) is the same under P π σ 1 ,τ 1 and under the probability

where τ ′ 1 is defined as follows: choose d 1 using some exogenous lottery such that the conditional law of d 1 given y 1 is the same as under π and then play τ 1 (d 1 ). We deduce that η 2 (π, σ 1 ) = η 2 (π, ψ) and y 2 = f π,ψ 2 (x 1 , y 1 ) under the probability P π σ 1 ,τ 1 which concludes the proof.

A stronger version of the theorem

To conclude this section, let us state a couple of stronger assumptions, which are expressed in terms of the data of the game more directly: Player 1 can deduce exactly the signal received by player 2 and player 2 can not influence the joint law of (x 2 , d 2 ).

Definition 4.10. For all, x, i, j ∈ ∆(K) × I × J, let q C×D (x, i, j) denote the marginal distribution on C × D induced by q(x, i, j), i.e. q C×D (x, i, j)(c, d) = k, k x(k)q(k, i, j)( k, c, d).

Let also H x,i the map defined on C × D by

With these notations, we can define a set of assumptions on the marginal of q. The assumptions (A1), (A2a) are unchanged and we define (A ′ 2b) and (A ′ 3).

(A ′ 2b) Player 1 knows the signal of player 2 i.e. there exists a map h : C → D such that for all (k, i, j) ∈ K × I × J, c∈C q(k, i, j)[c, h(c)] = 1.

(A ′ 3) The image probability φ(x, i) of q C×D (x, i, j) by the map H x,i does not depend on j.

Corollary 4.11. Let Γ be such that assumptions (A1), (A2a), (A ′ 2b) and (A ′ 3) are true.

Then: For all π ∈ ∆ * f (K × N × N), Γ(π) has a uniform value.

The proof of this corollary follows directly from the next Lemma.

Lemma 4.12. If A1 and A2a hold, then A ′ 2b and A ′ 3 imply A2b and A3.