Mariem Mhiri
email: mariem.mhiri@gmail.com

Vineeth S Varma
email: vineeth.varma@lss.supelec.fr

Maël Le Treust
email: mael.le.treust@emt.inrs.ca

Samson Lasaulce
email: samson.lasaulce@lss.supelec.fr

Abdelaziz Samet
email: abdelaziz.samet@ept.rnu.tn

On the benefits of repeated game models for green cross-layer power control in small cells (Invited Paper)

Keywords: distributed power control, energy efficiency, repeated game, channel state information

In this paper, we consider the problem of distributed power control for multiple access channels when energy-efficiency has to be optimized. In contrast with related works, the presence of a queue at each transmitter is accounted for and globally efficient solutions are sought. To this end, a repeated game model is exploited and shown to lead to solutions which are distributed in the sense of the decision, perform well globally, and may rely on limited channel state information at the transmitter.

I. INTRODUCTION

Designing energy-efficient communication systems has become a critical issue in modern day wireless networks. The problem treated in this work deals with power control when energy efficiency (EE) has to be optimized. This metric (EE) has been defined in [START_REF] Goodman | Power control for wireless data[END_REF] as a ratio of the net data rate (goodput) to the transmit power level. The problem was formulated as a non-cooperative game where each transmitter aims at selfishly maximizing its individual energy-efficiency. The considered solution is the Nash equilibrium (NE) which is shown to be unique but generally Pareto inefficient. To deal with this inefficiency, an operating point (OP) was proposed in [START_REF] Treust | A repeated game formulation of energyefficient decentralized power control[END_REF] where repeated game was exploited. Authors in [START_REF] Treust | A repeated game formulation of energyefficient decentralized power control[END_REF] showed that when playing with the developed OP according to a cooperation plan, only channel state information (CSI) is needed and transmitters can improve the social welfare (sum of utilities). Recently, a generalized EE metric has been proposed in [START_REF] Varma | A cross-layer approach for energy-efficient distributed power control[END_REF] for two important transport layer protocols (Transmission Control Protocol (TCP) and User Datagram Protocol (UDP)). The new EE metric is based on a cross-layer approach and takes into account the effects of the presence of a queue with a finite size at the transmitter. An interference channel system was studied and it was shown that a unique NE exists for a noncooperative game. In this paper, we consider the problem of distributed power control with the new EE metric according to UDP protocol developed in [START_REF] Varma | A cross-layer approach for energy-efficient distributed power control[END_REF] and for multiple access channels (MAC) system. Our goal is to find another unique solution concept which is efficient and may rely on limited CSI at the transmitter. We refer to a repeated game model (RG) developed in [START_REF] Treust | A repeated game formulation of energyefficient decentralized power control[END_REF] and try to apply the results on the crosslayer power control game. One of the major mathematical distinction between the two metrics used is the presence of a constant power term in the denominator of the EE metric. Although it appears to be a small change, the structure of the equilibrium solution is quite different. The optimal SINR when using the [START_REF] Goodman | Power control for wireless data[END_REF] metric is independent of the channel state. This property is lost when accounting for the constant power consumption, and motivates us to propose a new OP for the cross-layer metric. The main contributions of this work are:

1) Study the RG when using the cross layer EE as the utility of the game; 2) Establish the threshold on the game length beyond which the equilibrium policy can be pareto-optimal; 3) Propose a new OP that is efficient and can be reached in a distributed manner.

This paper is structured as follows. In section II-A, we introduce the system model under study. Then, we define (in section II-B) the static power control game. This is followed (section II-C) by a review of the non-cooperative one-shot game. In section III, we give the formulation of the RG model.

In section IV, we introduce the new OP and an equilibrium for the finite RG is proposed. Numerical results are presented and discussed in section V. Finally, concluding remarks are proposed in section VI.

II. PROBLEM STATEMENT

A. System model

The communication network under study is that of a MAC system, where N small transmitters are communicating with a receiver and are operating in the same frequency band. Transmitter i ∈ {1, . . . , N } sends a signal √ p i x i with power

p i ∈ [0, P max i
] where P max i > 0 is the maximum transmit power. The channel gain of the link between transmitter i and the destination is denoted as g i . Thus, the baseband signal received is written:

y i = g i √ p i x i + N j=1 j =i g j √ p j x j + n i , (1)
with n i is additive white Gaussian noise (AWGN) with mean 0 and variance σ 2 i . We assume that σ 2 i is identical for all the transmitters such that: σ 2 i = σ 2 . Therefore, the resulting SINR γ i at the receiver is given by:

γ i (p) = p i |g i | 2 σ 2 + 1 L N j=1 j =i p j |g j | 2 , (2)
where p = (p 1 , p 2 , . . . , p N ) is the power vector which will describe later the power actions of the N transmitters and L refers to the spreading factor [START_REF] Varma | A cross-layer approach for energy-efficient distributed power control[END_REF]. We assume that the described system is based on the IP (Internet Protocol) stack where packets arrive from an upper layer into a finite memory buffer of size K (in packets). Here, the considered protocol is UDP for which the packet arrival process follows a Bernoulli process with a constant probability q, independent from the SINR. This results in an effective packet loss denoted by Φ(γ i ) and an energy efficiency η i given by:

η i (p i , p -i ) = Rq(1 -Φ(γ i (p))) b + qp i (1 -Φ(γ i (p))) f (γ i (p)) , (3)
where p -i = (p 1 , .., p i-1 , p i+1 , .., p N ), R is the used throughput (in bit/s) and b represents the fixed consumed power when the radiated power is zero [START_REF] Varma | A cross-layer approach for energy-efficient distributed power control[END_REF].

B. Static power control game

The major motivation behind this work is in order to establish an efficient equilibrium point to which a completely distributed system can converge to. A non-cooperative game has been introduced in [START_REF] Varma | A cross-layer approach for energy-efficient distributed power control[END_REF] where the existence of a unique Nash equilibrium was proved. Here, we are looking for more efficient solutions which are distributed in the sense of the decision making, but may rely on limited channel state information at the transmitter. As motivated in [START_REF] Varma | A cross-layer approach for energy-efficient distributed power control[END_REF], the power control can be modeled by a strategic form game (see e.g., [START_REF] Lasaulce | Game Theory and Learning for Wireless Networks: Fundamentals and Applications[END_REF]).

Definition 2.1: The game is defined by the ordered triplet G = N , (A i ) i∈N , (u i ) i∈N where

• N is the set of players. Here, the players of the game are the sources/transmitters, N = {1, . . . , N }; • A i is the set of actions. Here, the action of source/transmitter i consists in choosing p i in its action set A i = [0, P max i ]; • u i is the utility function of each user according to UDP given by:

u i (p i , p -i ) = η i (p i , p -i ) (4)
The function f : [0, +∞) → [0, 1] is a sigmoidal efficiency function which corresponds to the packet success rate verifying f (0) = 0 and lim x→+∞ f (x) = 1. The function Φ identifies the packet loss due to both bad channel conditions and the finiteness of the packet buffer. This can be calculated as:

Φ(γ i ) = (1 -f (γ i ))Π K (γ i ) (5)
where Π K (γ i ) is the stationary probability that the buffer is full and is given by:

Π K (γ i ) = ω K (γ i ) 1 + ω(γ i ) + . . . + ω K (γ i ) (6)
with:

ω(γ i ) = q(1 -f (γ i )) (1 -q)f (γ i ) (7)
In [START_REF] Varma | A cross-layer approach for energy-efficient distributed power control[END_REF], the authors prove that the non-cooperative game with rational players, G, allows for a unique pure Nash equilibrium (NE). This NE is the set of powers from which no player has anything to gain by changing only his own strategy unilaterally. This is explained in the following section.

C. Review of the non-cooperative game

The non-cooperative power control game has been investigated in [START_REF] Varma | A cross-layer approach for energy-efficient distributed power control[END_REF] where the quasi-concavity of the utility function given in (4) was proved. Accordingly, as the NE represents the fundamental solution for a non-cooperative game, existence and uniqueness of such a solution have been studied and demonstrated as well. Thus, the optimal power denoted as p * i is obtained by setting ∂u i /∂p i to zero, which leads to solve the following equation:

bγ i Φ (γ i ) + q 1 -Φ(γ i ) f (γ i ) 2 [f (γ i ) -p i γ i f (γ i )] = 0, ( 8
)
where

γ i = dγ i dp i = γ i p i , f = df dγ i and Φ = dΦ dγ i .
However, the NE solution is not always Pareto efficient for many scenarios. An example is presented in Fig. 1 where we stress that the NE is far from the Pareto frontier. Motivated by the need to design an efficient solution relying on limited CSI at the transmitter, we move to the repeated game framework.

III. REPEATED POWER CONTROL GAME

In repeated games (RG), as the name suggests, the same game is played several times. The long-term interactions between the players in such a situation is studied under the RG framework. The players react to past experience by taking into account what happened in all previous stages and make decisions about their future choices [START_REF] Hart | Robert aumann's game and economic theory[END_REF], [START_REF] Sorin | Repeated games with complete information[END_REF]. The resulting utility of each player is an average of the utility of each stage. A game stage t corresponds to the instant in which all players choose their actions simultaneously and independently and thus a profile of actions can be defined by p(t) = (p 1 (t), p 2 (t), . . . , p N (t)). When assuming full monitoring, this profile choice is observed by all the players and the game proceeds to the next stage [START_REF] Sorin | Repeated games with complete information[END_REF]. The sequence of actions p i (t) of a transmitter i at time t defines his history denoted as h(t) = p i (t) = (p i (1), p i (2), . . . , p i (t -1)) and which lies in the set H t = P t-1 i . Before playing stage t, all histories are known by all the players [START_REF] Treust | A repeated game formulation of energyefficient decentralized power control[END_REF]. According to the above descriptions, a pure strategy δ i,t of player i ∈ N is a mapping from H t to the action set A i = [0, P max i ] specifying the action to choose after each history [START_REF] Treust | A repeated game formulation of energyefficient decentralized power control[END_REF], [START_REF] Sorin | Repeated games with complete information[END_REF]:

δ i,t : H t → [0, P max i ] h(t) → p i (t) (9)
We define the joint strategy δ = (δ 1 , δ 2 , . . . , δ N ) as the vector of all the players strategies.

In this paper, we are interested in the finite repeated game, i.e the game is played for a finite number of steps (T steps). The utility function of each player results from averaging over the instantaneous utilities over all the game stages. At each stage t, the instantaneous utility of player i is a function of the profile of actions of all the players p(t).

Definition 3.1: The utility function of the i th player for the finite RG is the arithmetic average of the sum of the utilities for the initial T first stages [START_REF] Sorin | Repeated games with complete information[END_REF], [START_REF] Aumann | Long-term competition-a gametheoretic[END_REF]. We have [START_REF] Treust | A repeated game formulation of energyefficient decentralized power control[END_REF]:

v T i (δ) = 1 T T t=1 u i (p(t))
for the finite RG (10)

where T ≥ 1 defines the number of game stages in the finite RG.

An equilibrium solution of the RG is defined in the following manner: Definition 3.2: A joint strategy δ satisfies the equilibrium condition for the finite repeated game if for all players i ∈ N , for all other strategies δ i , we have

v T i (δ) ≥ v T i (δ i , δ -i ).
It means that no deviating strategy δ i can increase the utility v t i (δ i , δ -i ) of any one player. This equilibrium solution is exactly what we are interested in, as a strategy δ satisfying the above condition would be precisely what rational players in a RG would play. In an RG with complete information and full monitoring, the Folk theorem characterizes the set of possible equilibrium utilities [START_REF] Treust | A repeated game formulation of energyefficient decentralized power control[END_REF], [START_REF] Sorin | Repeated games with complete information[END_REF]. It states that the set of Nash equilibrium in a RG is precisely the set of feasible and individually rational outcomes of the one-shot game (non-cooperative game) [START_REF] Hart | Robert aumann's game and economic theory[END_REF], [START_REF] Sorin | Repeated games with complete information[END_REF]. In an RG, interesting patterns of behavior between players can be seen and studied. This includes: rewarding and punishing, cooperation and threats, transmitting information and concealing [START_REF] Hart | Robert aumann's game and economic theory[END_REF].

IV. AN OPERATING POINT AND REPEATED GAME CHARACTERIZATION

A. New OP for the game G Consider the operating point (OP) described in [START_REF] Treust | A repeated game formulation of energyefficient decentralized power control[END_REF]. It is identified by a subset of points of the achievable utility region such that p i |g i | 2 = p j |g j | 2 for all (i, j) ∈ N . The optimal subset of powers consists of the solutions of the following system of equations:

∀(i, j) ∈ N , ∂u i ∂p i (p) = 0 with p i |g i | 2 = p j |g j | 2 (11)
with u i is the utility function defined in (4). Due to the presence of the parameter b which we consider different from 0, it can be observed that there will be N different solutions corresponding to equation (11) in terms of p i and thus the operating point from [START_REF] Treust | A repeated game formulation of energyefficient decentralized power control[END_REF] is not well defined when using the utility defined in [START_REF] Varma | A cross-layer approach for energy-efficient distributed power control[END_REF]. To deal with this problem, a new OP is proposed. The new OP consists in setting p i |g i | 2 to a constant denoted as α that can be optimized. We propose to determine a unique optimal α by maximizing the expected sum utility over all the channel states as follows:

α = arg max E g N i=1 u i (α, g) (12)
When playing at the OP, the power played by the i th player, denoted as pi , is given by:

pi = α |g i | 2 (13)
In the following section, we focus on the characterization of the finite RG.

B. Repeated power control game characterization

As a first step, we determine the minimum number of stages (T min ) corresponding to the finite RG. When the number of stages in the game is less than T min , the equilibrium of the RG is to simply play at the NE. However, if T > T min , a more efficient equilibrium point can be reached. Assuming that channel gains

|g i | 2 lie in a compact set [η min i , η max i ] [2],
we have the following proposition: Proposition 4.1 (Equilibrium solution for the finite RG.): For a finite RG, if T > T min , then the corresponding equilibrium solution is given by [START_REF] Treust | A repeated game formulation of energyefficient decentralized power control[END_REF]:

δ i,t : pi for t ∈ {1, 2, . . . , T -T min } p * i for t ∈ {T -T min + 1, . . . , T } P max i for any deviation detection (14)
where T min is:

Tmin=        Aη max i bη min i +γ i σ 2 B - Gη max i bη min i + αH Eη min i bη max i +γ * i ( σ 2 + 1 L j =i p * j η max i ) F - Cη min i bη max i + γ i( σ 2 + 1 L j =i p max j η max i ) D        (15)
The proof for this proposition is given in Appendix A, as well as the quantities A, B, C, D, E, F , G and H. γ * i is the SINR at the NE while γi and γ i are the SINRs related to the utility max and the utility minmax respectively (see Appendix A).

V. NUMERICAL RESULTS

We consider a scenario with a MAC where N transmitters are communicating with their corresponding receiver (e.g. base station). The efficiency function is assumed to be f (x) = e -c/x where c = 2 R R 0 -1 with R and R 0 are the throughput and the used bandwidth and supposed to be 1 Mbps and 1 MHz respectively. The other parameters are set as follows:

•

σ 2 = 10 -3 W • b = 10 -2 W • K = 10 • q = 0.5 • P max i = P max = 10 -1 W
The channel gains are assumed to be |g i | 2 = 1 and |g j | 2 = 0.5. Our simulations consist in showing firstly the advantage of the OP regarding the NE of the one shot game. Thus, we plot the achievable utility region, the NE and the proposed OP when considering a system of two transmitters and a spreading factor L = 2. In Fig. 2, the region delimited by the Pareto frontier and the minmax level defines, according to the Folk theorem, the possible set of equilibrium utilities of the RG. In addition, we highlight that the new OP dominates in terms of Pareto the NE and it is Pareto efficient. Fig. 3 represents the ratio w F RG w N E for the finite RG as a function of the number of stages. We have:

w F RG w N E = N i=1 ( T -Tmin t=1 ũi (p(t)) + T t=T -Tmin+1 u * i (p(t))) N i=1 T t=1 u * i (p(t)) (16
) We consider a system with 25 transmitters and a spreading factor L = 100. We proceed to an averaging over channel gains lying in a compact set such that 10 log 10 ηmax ηmin = 20. According to equation (15), the minimum number of stages T min is equal to 1200. According to this figure, we deduce that the social welfare can be improved when playing an RG. 3. Improvement of the social welfare in finite repeated game vs the Nash equilibrium. While the efficiency of the RG while using the traditional metric defined in [START_REF] Goodman | Power control for wireless data[END_REF] seems to be higher, it requires a longer game than in the cross layer model.

Figure 3 plots the improvement of the social welfare as defined in 16. This improvement obtained is compared for case when using the metric defined in [START_REF] Goodman | Power control for wireless data[END_REF] to the cross-layer metric used. The required time for profiting from the RG scenario is much lower in the cross-layer case, but the improvement seems to be relatively smaller. However, note that the NE in the cross-layer game itself is more efficient than the NE in [START_REF] Goodman | Power control for wireless data[END_REF] and so in absolute terms, the proposed OP is still quite efficient and can be utilized for shorter games. This validates our approach and shows that the RG formulation is a useful technique for efficient distributed power control.

VI. CONCLUSION

In this paper, we study an efficient solution for a relevant game with a new EE metric considering a cross-layer approach and taking into account the effects of the presence of a queue with a finite size at the transmitter. As the NE is generally inefficient in terms of Pareto, we design a new OP and exploit a repeated game model to improve the performance of a MAC system. We contribute to express the analytic form for the minimum number of stages in a finite RG. Moreover, our approach provides an efficient solution relying on limited CSI at the transmitter when comparing to the NE and contributes to considerable gains in terms of social welfare for the finite RG. As a first step, we determine the power p i maximising u i and which we denote as ṗi . This amounts to reduce ∂u i /∂p i to 0. We recall that we consider the following notations: γ i = dγi dpi = γi pi , f = df dγi and Φ = dΦ dγi . The power ṗi maximising u i is then the solution of the following equation:

b γ i p i Φ (γi)+q 1-Φ(γ i ) f (γ i ) 2 [f(γi)-γif (γi)]=0 (17)
Therefore, the expression of the maximum utility function writes as:

ui( ṗi,p-i)= Rq(1-φ( γi )) b+ ṗi q(1-φ( γi )) f ( γi )
, with:

γi= ṗi |g i | 2 σ 2 + 1 L j =i p j |g j | 2
In a second step, we are interested in studying the behavior of ui ( ṗi , p -i ) as a function of p j for j = i; which amounts to calculating the sign of ∂ ui( ṗi,p-i) ∂pj ,which is shown to be negative in [START_REF] Varma | A cross-layer approach for energy-efficient distributed power control[END_REF]. Therefore ∂ ui( ṗi,p-i) ∂pj < 0. As ui is a decreasing function of p j , it reaches its maximum when p j = 0 and it is minimum when p j = p max j (for all j = i).

A. Expression of ūi

The utility ui reaches its maximum when p j = 0. When substituting p j = 0 in the SINR expression γi , this allows the determination of the optimal power ṗi :

b |g i | 2 σ 2 Φ (γi)+q 1-Φ(γ i (p i )) f (γ i (p i )) 2 [f(γi(pi))-γif (γi(pi))]=0 (18)
As the latter equation is a function of the SINR, the solution will be in terms of SINR and will be denoted as γi . The corresponding optimal power is pi = γiσ 2 |gi| 2 . Then, we have:

ūi= Rq(1-φ(γ i )) b+ γi σ 2 |g i | 2 q(1-φ(γ i )) f (γ i )

B. Expression of u i

We proceed as described previously and determine the optimal SINR denoted as γ i which is the solution of the following equation:

b|g i | 2 σ 2 + 1 L j =i p max j |g j | 2 Φ (γi)+q 1-Φ(γ i ) f (γ i ) 2 [f(γi)-γif (γi)]=0 (19)
Then, we have:

ui= Rq(1-φ( γ i )) b+ γ i |g i | 2 ( σ 2 + 1 L j =i p max j |g j | 2 ) q(1-φ( γ i )) f ( γ i )
C. Existence proof of γ i and γi

Both equations ( 18) and ( 19) are resulting from the same equation (17) for two different forms of the SINR (γ i for p j = 0 and γ i for p j = p max j ). Showing the existence of these two solutions amounts to prove the existence of the solution of equation (17). However, according to the study established in [START_REF] Varma | A cross-layer approach for energy-efficient distributed power control[END_REF], it was proved that u i is quasi-concave in (p i , p -i ) and then it exists γ + such that the first derivative of u i regarding to p i is strictly positive on [0, γ + ] and strictly negative on [γ + , +∞] for all p j ∈ [0, p max j ] : the first derivative is continous and is equal to zero in γ + . According to the utility which we are studying (max or minmax), γ + is either γi (eq. ( 18)) or γ i (eq. ( 19)).

D. Proof

The SINR γi refers to the SINR when playing the new OP. In order to simplify expressions, we use the following notations:

A = Rq(1 -φ(γ i )) B = q(1-φ(γi)) f (γi) C = Rq(1 -φ( γ i )) D = q(1-φ( γi)) f ( γi) E = Rq(1 -φ(γ * i )) F = q(1-φ(γ * i )) f (γ * i ) G = Rq(1 -φ(γ i )) H = q(1-φ(γi)) f (γi)
The inequality (20) becomes:

A|g i | 2 b|g i | 2 +γ i σ 2 B + T s=T -T min +1 Eg C|g i | 2 b|g i | 2 + γ i( σ 2 + 1 L j =i p max j |g j | 2 ) D ≤ G|g i |

Fig. 1 .

 1 Fig. 1. Pareto inefficiency of the NE.

Fig. 2 .

 2 Fig. 2. Pareto dominance and Pareto efficiency of the proposed OP regarding the NE.

Fig.

  Fig.3. Improvement of the social welfare in finite repeated game vs the Nash equilibrium. While the efficiency of the RG while using the traditional metric defined in[START_REF] Goodman | Power control for wireless data[END_REF] seems to be higher, it requires a longer game than in the cross layer model.

  utilities max and minmax are expressed respectively as follows: ūi = maxp -i maxp i ui(pi,p-i) ui = minp -i maxp i ui(pi,p-i)

From [ 2 ]

 2 , we have:ūi (p(t)) + T s=T -Tmin+1 E g { u i (p(s))} ≤ ũi (p(t)) + T s=T -Tmin+1 E g {u * i (p(s))}