Hélène Le Cadre
email: helene.le-cadre@ensta-paristech.fr

Jean-Sébastien Bedo
email: jeansebastien.bedo@orange.com

Dealing with Uncertainty in the Smart Grid: A Learning Game Approach *

Keywords: Learning, Regret, Optimization Under Uncertainty, Game Theory

We model the smart grid as a decentralized and hierarchical network, made up of three categories of agents: suppliers, generators and captive consumers organized in microgrids. To optimize their decisions concerning prices and traded power, agents need to forecast the demand of the microgrids and the fluctuating renewable productions. The biases resulting from the decentralized learning could create imbalances between demand and supply leading to penalties for suppliers and for generators. We analytically determine prices that provide generators with a guarantee to avoid such penalties, transferring risk to the suppliers. Additionally, we prove that collaborative learning, through coalitions of suppliers among which information is shared, minimizes the sum of their average risk. Simulations, run for a large sample of parameter combinations, using external and internal regret minimization, show that the convergence of collaborative learning strategies is clearly faster than that resulting from individual learning. Finally, we analyze the suppliers' incentives to organize in a grand coalition versus multiple coalitions, and the tightness of the learning algorithm's theoretical bounds.

Introduction

In Europe, and especially France, power networks rely heavily on nuclear-based technology. With this type of non-renewable technology, generations can be adapted by the plant operator who alternates openings and closings and optimizes the duration of the switches between modes. The objective is thus to adapt the generations so as to meet the uncertain demand level [START_REF] Cadre | Distributed Learning in Hierarchical Networks[END_REF]. For renewables, generations can only be partially controlled, for instance, by lowering the wind turbine speed [START_REF] Marden | Achieving Pareto Optimality Through Distributed Learning[END_REF]. Renewable integration in the power network requires deploying smart Information and Communication Technologies (ICTs) to supervise the grid operations [START_REF] Wong | Autonomous Demand-Side Management Based on Game-Theoretic Energy Consumption Scheduling for the Future Smart Grid[END_REF]. Indeed, renewable generation is highly unpredictable since it depends on uncontrollable exogenous factors like wind, sun, swell, etc. Furthermore, the new active role of end users, who can become power generators, dynamically adapt their consumption and fit into a multitude of microgrids [START_REF] Saad | Coalitional Game Theory for Communication Networks: A Tutorial[END_REF], [START_REF] Saad | Coalitional Game Theory for Cooperative Micro-Grid Distribution Networks[END_REF], dramatically increases the volume of exchanged data flows. ICTs appear to be a means to retrieve the most salient information from this large amount of data and to train forecasters to provide efficient predictions regarding fluctuating generations such as renewables. These predictions will then be used as inputs to optimize the smart grid operations [START_REF] Atzeni | Demande-Side Management via Distributed Energy Generation and Storage Optimization[END_REF].

In practice, it is increasingly apparent that current forecasting methods cannot properly handle extreme situations corresponding to either severe weather phenomena or critical periods for power system operations. For example, forecasting methods used to predict wind power were mostly designed to provide single value forecasts to estimate the generations. Only recently, probabilistic methods have been introduced to provide estimations of the entire distribution of future generations [START_REF] Bossavy | Forecasting Ramps of Wind Power Production with Numerical Weather Prediciton Ensembles[END_REF] or predictions based on intervals [START_REF] Cadre | Probabilistic Solar PV Power Prediction Using Extreme Learning Machine and Information Fusion[END_REF]. In such methods, forecasts may take the form of either quantile estimations or density estimations [START_REF] Costa | A Review on the Young History of the Wind Power Short-Term Prediction[END_REF]. The difficulty to process massive, heterogeneous and dynamic volumes of data emanating from decentralized and heterogeneous sources has favored the launching of automatic data processing methods grouped under the umbrella of Machine Learning [START_REF] Cadre | Probabilistic Solar PV Power Prediction Using Extreme Learning Machine and Information Fusion[END_REF]. Learning based on regret minimization [START_REF] Cesa-Bianchi | Prediction, Learning, And Games[END_REF] belongs to this latter category. This class of method provides the forecaster with a density function that associates a weight to each possible output. The density function is updated by merging (online) information from various experts' reports. As a result, these methods are more robust in the face of extreme events and appear particularly well suited to model erratic processes such as renewables.

In the framework of the smart grid, learning is performed in a decentralized manner since each agent primarily learns the hidden information using its own observations. Existing literature on distributed learning mostly focuses on distributed learning algorithms that are suitable for implementation in large-scale engineering systems [START_REF] Cadre | Wind Farm Portfolio Optimization under Network Capacity Constraints[END_REF][START_REF] Li | Decoupling Coupled Constraints through Utility Design[END_REF][START_REF] Pradelski | Learning efficient Nash Equilibria in distributed systems[END_REF]. Many articles concentrate on games of potential [START_REF] Monderer | Potential Games[END_REF]. This class of games is of particular interest since they have inherent properties that can provide guarantees on the convergence and stability of the system but remain very difficult to use in a full systemic framework. Indeed some systems may not be modeled as potential games [START_REF] Marden | Achieving Pareto Optimality Through Distributed Learning[END_REF]. A signaling game is also introduced in [START_REF] Cadre | Wind Farm Portfolio Optimization under Network Capacity Constraints[END_REF] to model competition among geographic demand markets constrained by limited transmission capacities.

The learning game studied in this paper belongs to the category of repeated uncoupled games since one agent cannot predict the forecasts and so actions of the other agents at a given time period. To take its decision i.e., optimal prices and power orders, each agent is aware of the forecast history of all the agents in its coalition and of its utility. For finite games with generic payoffs, recent work has shown the existence of completely uncoupled learning rules i.e., rules where the agents observe only their own prediction history and their utility, leading to Pareto-optimal Nash equilibria [START_REF] Pradelski | Learning efficient Nash Equilibria in distributed systems[END_REF]. Marden et al. exhibit a different class of learning procedures that lead to a Pareto-optimal vector of actions that do not necessarily coincide with Nash equilibria [START_REF] Marden | Achieving Pareto Optimality Through Distributed Learning[END_REF]. Close to the work exposed in our article, Zheng et al. propose an online algorithm that simultaneously updates the weight given to each forecaster using regularized sequential linear regression, while allowing each forecaster to be retrained based on the latest observations in an online manner [START_REF] Zheng | A Sequential Predictor Retraining Algorithm and its Application to Market Prediction[END_REF]. The updating of the individual forecasters to accommodate the online observations relies on a gradient-descent algorithm. In the same spirit, another approach based on reinforcement learning has been proposed by Khan et al. in [START_REF] Khan | Game Dynamics and Cost of Learning in Heterogeneous 4G Networks[END_REF]. Their heterogeneous learning algorithm allows the users to learn their own optimal payoff at a certain cost while simultaneously optimizing their strategy. Expert system coordination can also be used to aggregate the set of predictors into a better global predictor [START_REF] Holsapple | A Machine Learning Method for Multi-Expert Decision Support[END_REF].

Most collaborative mechanisms studied in literature lead to price or quality of service alignment. In addition, the group composition provides an additional state space in which information about the environment can be accumulated [START_REF] Morales | On the Role of the Group Composition for Achieving Optimality[END_REF]. To our knowledge, no study has so far been made of the impact of collaboration through information sharing, when prices are individually determined, on the underlying system performance. Of course, collaboration might not emerge due to the agents' natural incentives to cheat and deviate from the cooperative equilibrium and also, most frequently, due to the regulator's intervention. There are a number of well-understood reasons why regulators often do not allow horizontal collaboration: if suppliers are allowed to collaborate, they might cooperate to raise the price i.e., reduce quantity below the efficient baseline, and create market power [START_REF] Creti | Supply Security and Short-Run Capacity Markets for Electricity[END_REF]. Alternatively, suppliers might cooperate to reduce quality of service. Courts punish agreements that explicitly aim to decrease competition.

In this article, we answer the following questions:

1) How will the biases caused by the errors made by the agents in their predictions affect the agents' average risk?

2) Does collaborative learning improve the smart grid's overall performance?

3) Do the agents have greater incentives to organize in a grand coalition than in a multiplicity of smaller size coalitions?

The article is organized as follows. In Section 2, we introduce the foundation of our model i.e., the agents, their utility and their optimization program. Complete information Stackelberg game is then solved in Section 3, proceeding by backward induction. We analytically derive the optimal prices and power orders for the agents. Partial information is introduced in Section 4 where the interacting agents learn hidden individual sequences in a distributed fashion. To illustrate the theoretical results derived in the previous sections, in Section 5 we compare: firstly, the time of convergence of suppliers' learning strategies under external and internal regret minimization in cooperative and non-cooperative scenarios, secondly, which behaviors should emerge depending on the game parameters value and finally, analyze the tightness of the bounds derived theoretically in various scenarios.

Notation and modeling assumptions

Agents and utility functions:

M i Microgrid i S i Supplier i G k Generator k TSO Transmission System Operator Π i (t) S i utility Πk (t) G k utility B i (t) Net benefit for M i Π 0 i (t
S i (t) M i generation ν G k (t) G k production E G
Set of all the possible production levels for one generator E S Set of all the possible generation levels for one microgrid

f i (t)
Vector of the predictions made by

S i f i (ν S i , t) S i forecast of M i generation f i (ν G k , t) S i forecast of G k production f(t)
Vector of forecasts of all the suppliers f -i (y, t)

Vector of forecasts of all the suppliers except S i whose prediction is replaced by y ν(t)

Vector of productions of microgrids and generators

d k ij (t)
Disagreement between S i and S j on the prediction of G k production D SS (i)

Lower bound on the disagreements between S i and the other suppliers

D SS (i)
Upper bound on the disagreements between S i and the other suppliers d t (.)

Supplier's learning strategy Losses and regrets:

T Horizon of the repeated game

l i f(t), ν(t) S i loss l (1) i f i (t), ν(t)
Upper bound on S i loss depending exclusively on the supplier's predictions

l (2) i (d k ij (t)) j,k , ν(t) Upper bound on S i loss depending on d k ij (t) j =i R i (T ) S i external regret over horizon T RI i (T ) S i internal regret over horizon T l(f(t), ν(t)) TSO loss lg(f(t), ν(t))
sum of the suppliers' losses F S Set of predictors for each supplier Fm Set of predictors for the TSO Ṽg Partial value of the game Payoffs:

H f i (ν S i ) (x, t) S i payoff evaluated in its forecast of M i generation H f i (ν G k ) (x, t) S i payoff evaluated in its forecast of G k 's production H ν s i (x, t) TSO payoff evaluated in its forecast of M i power generation H ν G i (x, t) TSO payoff evaluated in its forecast of G k 's production H f C (ν S i ) (x, t) Grand coalition payoff evaluated in its forecast of M i generation H f C (ν G k ) (x, t)
Grand coalition payoff evaluated in its forecast of G k production d y→y t (.) Modified strategy obtained when the forecaster predicts y each time it would have predicted y w t (x)

Weights in the external regret learning algorithm w (y,y ) (x)

Weights in the internal regret learning algorithm

Notation:

|E| Cardinality of the set E,

β i = 1 -n-1 δγ i γi , δ = j=1,...,n 1 γ j γj x+ max{x; 0} positive part of real x ξ 1 (x) = 1 - x min{E G } 1 x<0 - x max{E G } 1 x≥0 , ξ 2 (x) = 1 - x min{E G } 1 x≥0 - x max{E G } 1 x<0 ψ i (x, y) = γ i β i 1 ξ 2 (y) -1 ξ 1 (x) + max{E G } min{E G } - min{E G } max{E G } lg f(t), ν(t) = i=1,...,n l (1) i f i (t), ν(t) 2
The model A large number of agents interact in the smart grid. In this article, we model the smart grid as a three layer hierarchical network whose evolution depends on the interactions between the agents composing each layer and also, on the ability of the agents to cope with fluctuating power generations and demand [START_REF] Atzeni | Demande-Side Management via Distributed Energy Generation and Storage Optimization[END_REF], [START_REF] Wong | Autonomous Demand-Side Management Based on Game-Theoretic Energy Consumption Scheduling for the Future Smart Grid[END_REF]. We detail the three categories of agents and the repeated game which captures the interactions between them in Subsection 2.1. Then, we define each agent's optimization program in Subsection 2.2.

Description of the agents

We model the smart grid through three categories of agents: microgrids, suppliers and generators. The first category of agents is composed of microgrids. Microgrids generate some power demands (mostly for heating/cooling buildings and for the individual usages of their inhabitants) and also some power through non renewable sources like solar panels and wind roofings. Demand can be flattened through pricing demand response mechanisms which give incentives to end users to change their normal power consumption patterns in response to price variations [START_REF] Albadi | Demand Response in the Electricity Markets: An Overview[END_REF], [START_REF] Li | Optimal Demand Response Based on Utility Maximization in Power Networks[END_REF], [START_REF] Papavasiliou | Market-based control mechanisms for electric power demand response[END_REF]. The second category of agents is composed of suppliers who buy power from several generators and resell it to the end users. The third category is composed of generators. Each generator produces and sells power to all suppliers. We assume that each end user is captive i.e., it has a contract with one supplier only and does not churn from one supplier to another for all the duration of our study. This assumption holds good if we consider local or regional utility companies. In this sense, the set made of end users supplied by a single supplier can be seen as an individual microgrid, as defined in [START_REF] Saad | Coalitional Game Theory for Communication Networks: A Tutorial[END_REF], [START_REF] Saad | Coalitional Game Theory for Cooperative Micro-Grid Distribution Networks[END_REF] and in Section 1. Largely speaking, the microgrid economic model is based on sharing economy, which refers to peer to peer based sharing of access to goods and services. The need to crowdsource technologies such as solar panels, rooftop wind turbines, demand response applications and local storage facilities, will encourage individuals to connect together through communities to use other members' rooftops, applications, facilities, etc. It is already possible to share apartments (Airbnb), cars (Uber), and bikes. For solar power, Yeloha1 relies on consumers with solar panels, to provide the electricity that is shared among the community. The community members use the energy that is generated by their solar panels to reduce their own electricity bills as well as share the supply with the community. In the same spirit, Vandebron2 arranges from consumers to buy electricity directly from independent producers, such as farmers with wind turbines in their fields. Note that the modeling of bilateral exchanges between the consumers in the microgrid is out of the scope of this article. Indeed, we take an aggregated view of the microgrids (i.e., we do not represent the individual consumers, that would require to take into account behavioral models). Instead the microgrid is seen as a global player which can use its generation and buy from the supplier in case of under supply.

We denote by S i , with i varying between 1 and n, the i-the supplier and by M i the corresponding group of end users. The generators are denoted G k with k varying between 1 and K. The generators can be associated with photovoltaic park managers, wind farm administrators, etc. In this article, we assume that the generator cannot directly influence its power generation at each time period. This assumption holds good if we look at renewable sources like solar photovoltaic panels without any investment in additional solar panels during the study period. The variation of sun intensity will impact the amount of power generated without any lever for the generator 3 .

We model the interplay between the agents through a repeated game. To fix ideas, a time period of the game might coincide with an hour of the day and the horizon of the game with a finite number of days so that there is no change in the game parameters. At each time period t, the following game is played:

Basic Game Description (i) Each generator G k communicates its unitary price pk (t) > 0 to the suppliers. The prices are fixed independently and simultaneously by each generator so as to maximize its utility.

(ii) Each supplier S i places power orders with the generators: the quantity ordered by supplier S i from generator G k is denoted by q ik (t). Each supplier S i communicates its unitary price p i (t) > 0 to its microgrid. The power orders and prices are fixed independently and simultaneously by each supplier so as to maximize its utility.

(iii) Microgrid M i generates ν S i (t) power units and buys a i (t) power units from supplier S i . The quantity a i (t) is chosen so as to maximize the benefits for M i .

At each time period, generator G k produces ν G k (t) power units. It then delivers α ki (t)ν G k (t) units to supplier S i where α ki (t) ≥ 0 denotes the proportion of its generation that generator G k allocates to supplier S i , with the normalization constraint: i=1,...,n α ki (t) = 1. This proportion is defined depending on power orders received by G k . The sum of orders received by G k may exceed ν G k (t) and so each supplier may receive less power than it initially ordered. Penalties are incurred by both suppliers and power generators if they cannot fully satisfy their customers' demand. Supplier S i incurs a cost γ i > 0 per power unit missing for the supply of its microgrid, measured a posteriori. The cost scale is defined through the Balance Operator Rules as designed by the Transmission System Operator (TSO) 4 . Note that we do not consider over provision and assume that the excess supply is lost. In our article, the penalty is specific to each supplier. This assumption is justified by the fact that depending on its geographic location, a negative energy balance can be easily corrected in densely interconnected areas whereas it is much more difficult in remote ones due to the high cost of electricity transmission. As a result, γ i is higher for suppliers serving remote locations than in densely interconnected areas. Similarly generator G k incurs a cost γi > 0 per missing power unit in the provision of supplier S i , measured a posteriori. In the short term, the level of production on suppliers' orders can be readjusted by imports from neighboring energy markets.

Optimization program for each agent

In this subsection, we describe the decision variables and utilities for each category of agents. The optimization program for each agent is presented using a mathematical formulation.

Microgrid programs

The only decision variable for microgrid M i is the power that it demands from supplier S i at time period t: a i (t). We assume that the microgrid has no lever to influence the random generation from its intermittent sources (solar panels): ν S i (t). We model the microgrid benefit of consuming x power units as a quadratic function. More precisely we use the model of Fahrioglu and Alvarado [START_REF] Fahrioglu | Designing Cost Effective Demand Management Contracts using Game Theory[END_REF]. The utility of consuming x power units then equals θ i b 0 x -1 2 b 1 x 2 where θ i is a positive parameter depending on the microgrid composition (households, firms, etc.) and b 0 and b 1 are generic positive parameters. We note that this model is valid only when x < b 0 b 1 . Beyond this threshold, the marginal benefit of consuming an additional power unit is zero and the total benefit of consuming power is constant and equals θ i b 2 0 2b 1 . As a result, the net benefit for microgrid M i , defined as the benefit of consuming ν S i (t) + a i (t) power units minus the cost of buying a i (t) power units from supplier S i , is:

B i (t) =        θ i b 0 (ν S i (t) + a i (t)) -1 2 b 1 (ν S i (t) + a i (t)) 2 -p i (t)a i (t) if ν S i (t) + a i (t) < b 0 b 1 θ i b 2 0 2b 1 -p i (t)a i (t) if ν S i (t) + a i (t) ≥ b 0 b 1 (1)
Microgrid M i chooses a i (t) ≥ 0 in order to maximize its net benefit 5 . Therefore, the microgrid optimization program takes the form: max a i (t)≥0 B i (t). Note that the microgrid decision depends on the price p i (t) fixed by supplier S i .

Supplier programs

Supplier S i decision variables are its unit price p i (t) and the power orders q ik (t) k placed with each generator G k . Following our description of the interplay between the agents, supplier S i 's utility at time period t is:

Π i (t) = p i (t)a i (t) - k=1,...,K q ik (t) pk (t) -γ i a i (t) - k=1,...,K α ki (t)ν G k (t) + shortfall in the supply of M i (2)
Supplier S i chooses its unit price and its power orders in order to maximize Π i (t) defined in Equation ( 2). Its optimization program takes the form: max

p i (t)>0,(q ik (t)) k ∈R K + Π i (t) (3)

Generator programs

Generator G k decision variable is the unit price, pk (t), at which it provides power to the suppliers. Then G k 's utility at time period t equals:

Πk (t) = pk (t) i=1,...,n q ik (t) - i=1,...,n γi q ik (t) -α ki (t)ν G k (t) + shortfall in the provision of S i (4)
To define the sharing coefficients, α ki (t) i , we consider a weighted proportional allocation of resources that allows generators to discriminate power allocation by supplier while allocating its power simultaneously among the suppliers. This framework is a generalization of the well-known proportional allocation mechanism in which the suppliers' bids coincide with their power orders weighted by their penalty coefficients ( γi ) i [START_REF] Nguyen | Weighted Proportional Allocation[END_REF], [START_REF] Saad | Coalitional Game Theory for Cooperative Micro-Grid Distribution Networks[END_REF]. This means that when two suppliers have identical power orders, the one with the highest penalty coefficient receives the largest share of the generator's available power. Indeed, the generator wants to minimize its overall penalty and, therefore, allocates larger shares of its power to suppliers serving isolated areas where failure of electricity supply may be critical. The choice of such a resource sharing mechanism can be justified by three factors: firstly, a small extension of Nguyen and Vojnović's work [START_REF] Nguyen | Weighted Proportional Allocation[END_REF] shows that weighted payment auction achieves competitive transfers to generators compared to standard price discrimination schemes at the equilibrium; secondly, the implementation of a sequential resource allocation mechanism based on priority and without storage facilities should be avoided since some suppliers might be left without allocation at all; thirdly, a sequential allocation of the power at the beginning of the time period is almost impossible because the generations are random individual sequences of which outputs are only (partially) observed at the end of the time period. This last point will be detailed in Section 4. We set:

α ki (t) γi q ik (t) C k (t) (5)
where C k (t) = j=1,...,n γj q jk (t). Using Equation ( 5), generator G k 's utility at time period t can be rewritten:

Πk (t) = pk (t) i=1,...,n q ik (t) - i=1,...,n γi q ik (t) 1 - γi C k (t) ν G k (t) + (6)
Generator G k chooses its unitary price so that Πk (t), as defined in Equation ( 6), is maximized. Its optimization program takes the form: max

pk (t)>0 Πk (t) (7)
3 Complete information game resolution

The game setting described in Subsection 2.1 implies that in generator-supplier relationship (Steps (i)-(ii) of Basic Game), generators act as leaders and suppliers as followers. Similarly, in supplier-microgrid relationships (Steps (ii)-(iii) of Basic Game), suppliers are leaders and microgrids followers. Under such a setting, the game is called a Stackelberg game and, as usual, it should be solved using backward induction [START_REF] Myerson | Game Theory: An Analysis of Conflict[END_REF].

Additionally, we assume that each generator receives at least one power order from a supplier, guaranteeing that the Stackelberg game admits non trivial solutions.

Optimization of the microgrids' decisions

Microgrid M i has to choose a i (t) in order to maximize its net benefit, defined by Equation [START_REF] Albadi | Demand Response in the Electricity Markets: An Overview[END_REF].

If

a i (t) + ν S i (t) ≥ b 0 b 1 , then the derivative of B i (t)
with respect to a i (t) equals -p i (t). Under this assumption, the optimal a i (t) is reached at the lower bound of the interval i.e., a i

(t) = b 0 b 1 -ν S i (t). If a i (t) + ν S i (t) < b 0 b 1 , the derivative of B i (t) with respect to a i (t) equals: ∂B i (t) ∂a i (t) = θ i b 0 -b 1 (a i (t) + ν S i (t)) -p i (t)
This derivative equals 0 when:

a i (t) = b 0 b 1 -ν S i (t) - p i (t) b 1 θ i (8)
We note that for this value of a i (t) we automatically obtain

a i (t) + ν S i (t) < b 0 b 1 .
In addition, we compute the second order derivative of B i (t) with respect to a i (t):

∂ 2 B i (t) ∂a i (t) 2 = -θ i b 1 < 0.
This implies that the maximum of B i (t) is reached when a i (t) is defined by Equation [START_REF] Foster | Learning, Hypothesis testing, and Nash equilibrium[END_REF].

To avoid a negative demand, we need to impose the following constraint:

ν S i (t) < b 0 b 1 -p i (t) b 1 θ i .
Otherwise, the optimal demand from the microgrid is 0.

Optimization of the suppliers' decisions

To find its optimal price and power orders, supplier S i has to replace a i (t) by its optimal value, defined in Equation ( 8), in Π i (t) which was defined in Equation ( 2). The resulting utility function Π i (t) is non differentiable because of the positive part in its right hand side. Therefore to derive supplier i's utility in p i (t) and in q ik (t), we need to distinguish between two cases.

Case 1: power generation satisfies demand from the microgrid

This is the case when

: b 0 b 1 -ν S i (t) - p i (t) b 1 θ i < k=1,...,K α ki (t)ν G k (t) (9)
Then deriving supplier i's utility in p i (t) leads to:

∂Π i (t) ∂p i (t) = b 0 b 1 -ν S i (t) -2p i (t)b 1 θ i . This derivative equals 0 when p i (t) = θ i b 0 -b 1 ν S i (t) 2
which is the optimal price for supplier S i . Then the positivity constraint for a i (t) becomes ν S i (t) < b 0 b 1 . In addition, the derivative of the supplier's utility in q ik (t) leads to: ∂Π i (t) ∂q ik (t) =pk (t) which means that supplier S i will try to minimize all of its power orders to maximize its utility. As a result, α ki (t) will tend to be small. This implies, in turn, that supplier S i will tend to break the inequality defining Case 1 in Inequality ( 9) and we will always fall on the frontier between Case 1 and Case 2. The frontier between these two cases is defined by the equation:

b 0 2b 1 - ν S i (t) 2 = k=1,...,K α ki (t)ν G k (t) (10)
3.2.2 Case 2: power generation does not satisfy power demand from the microgrid

This is the case when b 0 b 1 -ν S i (t) -p i (t) b 1 θ i ≥ k=1,...,K α ki (t)ν G k (t).
Then deriving supplier S i 's utility with respect to p i (t)

and q ik (t) gives us:

∂Π i (t) ∂p i (t) = b 0 b 1 -ν S i (t) + γ i b 1 θ i - 2p i (t) b 1 θ i ∂Π i (t) ∂q ik (t) = -pk (t) + γ i ν G k (t) ∂α ki (t) ∂q ik (t) (11)
By using the definition of α ki (t), given in Equation ( 5), we obtain:

∂α ki (t) ∂q ik (t) = γi C k (t) -γi q ik (t) C k (t) 2
Going back to the system of Equations ( 11), we conclude that the derivatives equal 0 when:

p i (t) = θ i b 0 -b 1 ν S i (t) + γ i 2 (12) pk (t)C k (t) 2 = γ i ν G k (t) γi C k (t) -γi q ik (t) (13)
On the one side, we directly obtain the price at which the derivative of Π i (t) equals 0 through Equation [START_REF] Cadre | Probabilistic Solar PV Power Prediction Using Extreme Learning Machine and Information Fusion[END_REF]. We derive from this equation the positivity constraint for a i (t) which is:

ν S i (t) < b 0 b 1 - γ i b 1 θ i
On the other side, Equation ( 13) can be rewritten as follows:

γi q ik (t) = C k (t) - pk (t)C k (t) 2 ν G k (t)γ i γi (14)
If supplier S i anticipates that the other suppliers will make the same optimization program, replicating Equation ( 14) for the n suppliers and adding them all together results in the following equality:

C k (t) = nC k (t) -pk (t)C k (t) 2 ν G k (t) j=1,...,n 1 γ j γj by definition of C k (t).
Then, given that C k (t) is not zero because each generator G k receives at least one power order otherwise it would be out of the game, by dividing the previous equation by C k (t) and reordering we obtain:

C k (t) = ν G k (t) pk (t) n-1 δ where δ = j=1,...,n 1 γ j γj
. By replacing C k (t) in Equation ( 14), we obtain the power orders for which the derivatives of Π i (t) equal 0:

q ik (t) = ν G k (t) pk (t) n -1 δ β i γi ( 15
)
in which we have introduced the notation β i = 1 -n-1 δγ i γi to simplify future calculations. We now have to check that the price and power orders for which the derivatives of Π i (t) equal 0 satisfy the conditions of Case 2.

Firstly, it is easy to check that the price is positive through Equation ( 12) and the positivity constraint for a i (t). However, the power orders defined in Equation ( 15) are non-negative if, and only if, β i ≥ 0 which is equivalent to 1 ≥ n-1 δγ i γi which, in turn, is equivalent to:

γ i γi ≥ n -1 δ ⇔ 1 γ i γi ≤ 1 n -1 j=1,...,n 1 γ j γj (16)
This inequality means that the penalties related to supplier S i i.e., γ i , γi , are close to the penalties related to the other suppliers i.e., (γ j , γj ) j . Indeed, if all penalties are equal to γ, then δ = n γ 2 and Inequality ( 16) is true for all suppliers. On the contrary, if all penalties are equal to γ except for supplier S 1 which has a penalty of γ n-1 , then δ = (n-1)n

γ 2
and Inequality ( 16) for supplier S 1 becomes n ≥ (n -1) 2 which is false as soon as n > 2. In this case, supplier S 1 would not buy any power from the generators, and so would be out of the game.

Secondly, by replacing the power orders, defined by Equation ( 15), in Equation ( 5), we obtain:

α ki (t) = ν G k (t) pk (t) n-1 δ β i j=1,...,n ν G k (t) pk (t) n -1 δ β j = β i j=1,...,n β j = β i
This proves that the sharing coefficient α ki (t) depends on neither generator G k nor time instant t. Furthermore, the above result means that the total power delivered to microgrid M i is:

k=1,...,K α ki (t)ν G k (t) = β i k=1,...,K ν G k (t).
As a result, the price and power orders for which the derivatives of Π i (t) equal 0 verify the inequality defining Case 2 if, and only if:

b 0 θ i -γ i 2b 1 θ i β i - ν S i (t) 2β i ≥ k=1,...,K ν G k (t) (17)
This inequality states that the total generation should not be too high. If this is not the case, then the over-supply situation would probably end up with the most expensive generator out of the game. Indeed, the suppliers have the choice to buy electricity coming from a set of generators. They base their choice on the price at which electricity is provided by each generator. As a result, the cheapest generators are chosen first. However, the productions of the generators are limited by their capacity and the supplier might not get their complete order. In case of over-supply, the total generation exceeds the demand from the suppliers. As a result, only the cheapest generators would be selected to satisfy the microgrid demand while the most expensive generator would have no client and would be forced to leave the market place. If Inequalities ( 16) and ( 17) are true, the optimum for supplier S i is reached for p i (t) defined by Equation ( 12) and q ik (t) defined by Equation [START_REF] Li | Decoupling Coupled Constraints through Utility Design[END_REF]. If one of these inequalities is not true, then the optimum for supplier S i is reached on the frontier defined by Equation (10).

Optimization of the generators' decisions

After substituting q ik (t) and C k (t) by the expressions found Subsection 3.2.2 in generator G k 's utility, as defined in Equation [START_REF] Creti | Supply Security and Short-Run Capacity Markets for Electricity[END_REF], we obtain:

Πk (t) = ν G k (t) n -1 δ i=1,...,n β i 1 γi -γi 1 pk (t) γi - δ n -1 + (18)
The only part of this equation depending on pk (t) always has a negative impact on the utility of the generator under the assumption of fair penalties. Indeed, in that case, as raised in the previous section, we obtain:

β i ≥ 0 for all suppliers (S i ) i .
As a result, to maximize its utility, the generator has to choose pk (t) such that the part depending on pk (t) in Equation [START_REF] Marden | Achieving Pareto Optimality Through Distributed Learning[END_REF] equals 0. This implies that the term

1 pk (t) γi -δ n-1 is below 0 for all i = 1, ..., n.
It is equivalent to: pk (t) ≥ n-1 δ γi . Consequently, the optimal price for the generator with fair penalties should satisfy:

pk (t) ≥ n -1 δ min i=1,...,n { γi } (19)

A distributed learning game

In this section, we assume that the generations from the microgrids ν S i (t)

i and the productions of the generators ν G k (t) k are random individual sequences. As explained in the Introduction, this means that the underlying random processes generating the sequences do not necessarily have a probabilistic structure. They can be quite erratic [START_REF] Cesa-Bianchi | Prediction, Learning, And Games[END_REF]. To guarantee optimal system-wide operation, it is fundamental that the suppliers elaborate efficient learning strategies regarding the microgrids' generation and the generators' production. The risk associated with this learning task will be measured by the supplier's loss. It will be defined in Subsection 4.1.

Suppliers should optimize their prices and quantities ordered at each time period, at which point they possess no information on either the microgrids' generation or about the generators' decentralized production. As a result, the game can be considered as having incomplete information [START_REF] Myerson | Game Theory: An Analysis of Conflict[END_REF]. Each supplier S i has to forecast ν S i (t) and ν G k (t) for all k = 1, ..., K, at each time period, in order to optimize its decisions. The game will be repeated over a finite time horizon 0 < T < +∞.

To simplify, we will consider a common closed space E G of possible values for the production of each generator and a common closed space E S of possible values for the generation from a microgrid. E G ,E S ⊆ R are considered to be of finite dimension i.e., their cardinalities |E G | and |E S | are such that |E G | < +∞ and |E S | < +∞. We will denote by f i (X, t) the forecast of supplier S i about the variable X at time period t. We will use boldface type to denote vectors. We will also use the simplifying notations:

• f i (t) f i (ν S i , t) forecast of M i 's generation , f i (ν G 1 , t), ..., f i (ν G K , t)
forecasts of the generators' productions to denote the predictions made by supplier S i about the generation from microgrid M i and about the production of each generator G k , k = 1, ..., K, at time period t

• f(t) f 1 (t), ..., f n (t) which contains the forecasts of the suppliers, at time period t

• f -i (y, t) = f 1 (t), ..., f i-1 (t), y, f i+1 (t), ..., f n (t) which contains the forecasts of the suppliers except S i whose prediction is set equal to y, at time period t

• ν(t) ν S 1 (t), ..., ν S n (t) microgrids' generations , ν G 1 (t), ..., ν G K (t)
generators' productions which contains the generation from each microgrid M i and the production of each generator G k , k = 1, ..., K, at time period t

Under conditions of power shortage as defined in Subsection 3.2.2, we obtain the optimal price for supplier S i by substitution of the forecasters in Equation ( 12) and the optimal power orders for supplier S i by doing the same in Equation [START_REF] Li | Decoupling Coupled Constraints through Utility Design[END_REF]. The optimal decisions for supplier S i at each time period t are then:

p i (f i (t), t) = γ i +θ i b 0 -b 1 f i (ν S i ,t) 2 and q ik (f i (t), t) = f i (ν G k ,t) pk (t) β i γi n-1 δ . Therefore, the demands from the microgrids are a i (f i (t), ν(t)) = b 0 2b 1 -γ i 2b 1 θ i -ν S i (t) + f i (ν S i ,t) 2
because the microgrids have exact knowledge of their generation, unlike the suppliers, which have to forecast theirs. In addition, since suppliers may differ in their forecasts, α ki can no longer be reduced to β i , and instead we obtain

α ki (f(t)) = f i (ν G k ,t)β i j=1,...,n f j (ν G k , t)β j
. As a result, the utility of supplier S i at each time period t is:

Π i (f(t), ν(t)) = γ i + θ i b 0 -b 1 f i (ν S i , t) 2 b 0 2b 1 - γ i 2b 1 θ i -ν S i (t) + f i (ν S i , t) 2 - β i γi n -1 δ k=1,...,K f i (ν G k , t) -γ i b 0 2b 1 - γ i 2b 1 θ i -ν s i (t) + f i (ν S i , t) 2 - k=1,...,K f i (ν G k , t)β i j=1,...,n f j (ν G k , t)β j ν G k (t) + (20)
The game parameters and random events (fluctuating generations and penalties) are chosen to obtain a constant power shortage, in the sense that:

• Inequality ( 17) transposed to the incomplete information setting will always be true i.e.:

b 0 2b 1 - γ i 2b 1 θ i - ν S i (t) 2 ≥ k=1,...,K α ki (f(t))ν G k (t) (21)
• The positivity constraint on a i (f i (t), ν(t)) holds i.e., b 0 2b 1 -

γ i 2b 1 θ i ≥ ν S i (t) - f i (ν S i ,t) 2
Lemma 1. To maximize its utility, supplier S i should be unbiased in its forecast of the generation from microgrid M i and its forecast of the generators' productions.

Proof of Lemma 1. Firstly, since the conditions are a constant power shortage, we have ∂Π i (f(t),ν(t))

∂f i (ν S i ,t) = -θ i b 1 2 a i (f i (t), ν(t))+ p i (f i (t),t) 2
-γ i 2 = θ i b 1 2 ν S i (t) -f i (ν S i , t
) . This derivative equals 0 if, and only if,

f i (ν S i , t) = ν S i (t).
In addition,

∂ 2 Π i (f(t),ν(t)) ∂f i (ν S i ,t) 2 = -θ i b 1 2 < 0 which means that Π i is concave in f i (ν S i , t) so its maximum is reached when f i (ν S i , t) = ν S i (t)
. Secondly, we have:

∂Π i (f(t), ν(t)) ∂f i (ν G k , t) = - β i γi n -1 δ + γ i ν G k (t) β i C k (f(t)) -β 2 i f i (ν G k , t) C k (f(t)) 2 where C k (f(t)) j=1,...,n f j (ν G k , t)β j .
This derivative equals 0 if, and only if:

β i f i (ν G k , t) = C k (f(t)) - 1 γ i γi n -1 δ C k (f(t)) 2 ν G k (t) (22)
By summing this condition for all suppliers, we obtain:

C k (f(t)) = nC k (f(t)) -n-1 δ C k (f(t)) 2 ν G k (t) j=1,...,n 1 γ j γj which is equiva- lent to: C k (f(t)) = C k (f(t)) 2 ν G k (t) by definition of δ, which means C k (f(t)) = ν G k (t)
. By replacement in Equation ( 22), we obtain that the derivative equals 0 if, and only if:

f i (ν G k , t) = ν G k (t). In addition, ∂ 2 Π i (f(t),ν(t)) ∂f i (ν G k ,t) 2 = -2γ i β i ν G k (t) j =i β j f j (ν G k , t) C k (f(t)) 3 < 0 which means that Π i is concave in f i (ν G k , t) so its maximum is reached when f i (ν G k , t) = ν G k (t).

Learning risk measure definition and first observations

As already mentioned, the supplier's risk, associated with the learning task, will be measured by its resulting loss. We have chosen a loss function representing the lack of profit compared to a case in which the supplier produces perfect forecasts of power demands and generations. More precisely, for any supplier S i , i = 1, ..., n, its loss is defined as:

l i f(t), ν(t) = Π 0 i (t) -Π i (f(t), ν(t)) (23)
where

Π 0 i (t) corresponds to supplier S i 's utility evaluated in f i (ν S i , t) = ν S i (t) and f i (ν G k , t) = ν G k (t)
for any k = 1, ..., K. We now decouple the upper bound in the supplier's loss to highlight the impact of information structures i.e., how information is shared and forecasts are made, on the overall performance of the smart grid. To that purpose, we upper bound supplier S i 's loss as the sum of a loss function depending exclusively on the supplier's predictions, l

i , and on another function, l

i , which relies on the disagreements between all the suppliers' predictions. The notion of disagreement needs to be carefully explained. To that purpose, we introduce:

d k ij (t) f i (ν G k , t) -f j (ν G k , t), ∀i, j = 1, ..., n, j = i, ∀k = 1, ..., K (24)
as a measure of the disagreement between supplier S i and supplier S j , i = j, in the prediction of generator G k 's power production, at time period t.

Proposition 2. Supplier S i 's loss can be upper bounded by the sum of two functions: the first, l

i , depends exclusively on its forecasts and the second, l

i , depends on its disagreement with the other suppliers' predictions:

l i f(t), ν(t) ≤ l (1) i f i (t), ν(t)
loss caused exclusively by supplier S i 's forecasts

+ l (2) i (d k ij (t)) j,k , ν(t)
loss caused by the disagreements of supplier S i with the others' forecasts ∀i = 1, ..., n

with

l (1) i f i (t), ν(t) = b 1 θ i f i (ν S i , t) -ν S i (t) 2 4 - β i γi n -1 δ k=1,...,K ν G k (t) -f i (ν G k , t) (26)
and

l (2) i (d k ij (t)) j,k , ν(t) = γ i β i k=1,...,K ν G k (t) 2 min{E g } - ν G k (t) j=1,...,n β j ξ 1 (d k ij (t)) (27)
where

ξ 1 (x) = 1 -x min{E G } 1 x<0 - x max{E G } 1 x≥0 .
Proof of Proposition 2. The proof can be found in Appendix A.

Optimal learning strategies for each supplier

In this context of incomplete information on the power generations from the microgrids and on the fluctuating renewable productions, we test two regret criteria to build the suppliers' learning strategy, d t (.): external and internal regret minimization [START_REF] Cesa-Bianchi | Prediction, Learning, And Games[END_REF]. Both regret minimization algorithms give rise to an optimized learning strategy [START_REF] Cesa-Bianchi | Prediction, Learning, And Games[END_REF] i.e., a density function defined over the space E S × E K G . As explained in the Introduction, regret minimization is more robust to extreme events as it provides a density function over the prediction set. Other learning rules based on different regret criteria exist such as regret-matching [START_REF] Hart | A Simple Adaptive Procedure Leading to Correlated Equilibrium[END_REF] and regret-testing [START_REF] Foster | Learning, Hypothesis testing, and Nash equilibrium[END_REF]. However, they offer no guarantee on the convergence of the algorithm and require longer times to reach an equilibrium when it exists.

The external regret over the sequence of time periods 1, ..., T , is the difference between the observed cumulative loss and the cumulative loss of the best constant prediction i.e., pure strategy. To be more precise, for supplier S i , it takes the form:

R i (T ) T t=1 l i f(t), ν(t) -min y∈E S ×E K G T t=1 l i f -i (y, t), ν(t) (28)
We will consider that the learning strategy of supplier S i is optimal if asymptotically its external regret remains in o(T ) where T is the number of time periods that have been played. This means that with probability 1:

lim sup T →+∞ 1 T T t=1 R i (t) = 0.
Forecasters satisfying these inequalities are said to be Hannan consistent [START_REF] Cesa-Bianchi | Prediction, Learning, And Games[END_REF]. By definition, a strategy d t (.) has a small internal regret if for every couple of predictions y, y ∈ E S × E K G , the forecaster does not regret not having chosen prediction y for time periods for which it chose prediction y:

RI i (T ) max y,y ∈E S ×E K G t=1,...,T d t (y) l i f -i (y, t), ν(t) -l i f -i (y , t), ν(t)) (29)
In a repeated game, this regret criterion ensures that the joint empirical frequencies of play converge with the set of correlated equilibria [START_REF] Cesa-Bianchi | Prediction, Learning, And Games[END_REF] whereas there is no guarantee that the product of the marginal empirical frequencies of play will converge with the Nash equilibria, under external regret minimization in a general game. Internal regret minimization will be used exclusively in the simulations in Subsection 5.3 as a benchmark to compare external regret minimization properties.

In the following lemma, we prove that it is possible to construct learning strategies for the suppliers that asymptotically minimize their external regret. Lemma 3. A Hannan consistent learning strategy exists for each supplier S i .

Proof of Lemma 3. In our case setting, at the end of each time period, supplier S i knows 6 the power demand from microgrid M i and it can infer its generation, ν S i (t), from Equation [START_REF] Foster | Learning, Hypothesis testing, and Nash equilibrium[END_REF]. Supplier S i also knows the power that has been delivered to it by each generator G k , from which it can infer the power that could have been delivered to it if it had ordered a different quantity q ik (t), all other suppliers ordering the same power quantities, using Equation ( 5). As a result, supplier S i can calculate its loss for all of its possible actions. In [START_REF] Cesa-Bianchi | Prediction, Learning, And Games[END_REF], Cesa-Bianchi and Lugosi proved that a Hannan consistent learning strategy always exists if the agent can compute its loss for each possible action at the end of each time period.

We now introduce lower and upper bounds on the disagreements between supplier S i and the other suppliers regarding the predictions of the generations: D SS (i) min t=1,...,T min j =i,k d k ij (t) and D SS (i) max t=1,...,T max j =i,k d k ij (t). They contain the extreme disagreement values between the suppliers, about the estimated generations. Lemma 4. If supplier S i plays according to a Hannan consistent strategy then an upper bound exists for the external regret associated with supplier S i 's loss caused by its own predictions, l

i , which depends only on the extreme disagreement values between the suppliers regarding the estimated generations, D SS (i) and D SS (i). More precisely:

lim sup T →+∞ 1 T T t=1 l (1) i f i (t), ν(t) -min y∈E S ×E K G T t=1 l (1) i y, ν(t) ≤ ψ i D SS (i), D SS (i) k=1,...,K ν G k (t) (30)
where the function ψ i from R 2 to R is defined by:

ψ i (x, y) = γ i β i 1 ξ 2 (y) - 1 ξ 1 (x) + max{E G } min{E G } - min{E G } max{E G } (31) with ξ 2 (x) = 1 -x min{E G } 1 x≥0 - x max{E G } 1 x<0 . Proof of Lemma 4.
The proof can be found in Appendix B. The aim of the next subsections will be to derive bounds for suppliers' losses under cooperative and non-cooperative scenarios.

Analysis of the upper bounds of the sum of suppliers' loss functions

We express the TSO's loss as the opposite of the sum of all of the suppliers' losses. This coincides with the price of imports from neighboring markets that the TSO should pay to guarantee that the production level meets the suppliers' demand:

l f(t), ν(t) i=1,...,n Π i (t) -Π 0 i (t) (32)
It is also possible to consider that the suppliers play against Nature [START_REF] Cesa-Bianchi | Prediction, Learning, And Games[END_REF] which exhibits its worst behavior towards suppliers when setting the random individual sequences. Similarly to the suppliers, the TSO will try to keep its external regret R(T ) in o(T ).

We define lg as the sum of the suppliers' losses exclusively caused by their own predictions:

lg f(t), ν(t) i=1,...,n l (1) i f i (t), ν(t) (33)
We let F S be the set of predictors (i.e., discrete density function set or alternatively, randomized prediction set) for each supplier and F m the set of predictors for the TSO. The value of the game, in which the suppliers exclusively consider the losses caused by their own predictions as utilities, is defined as follows:

Ṽg min

⊗ i=1,...,n d(f i )∈F n S suppliers' predictors max d(ν)∈Fm TSO's predictor E lg (X, Y)|X ∼ ⊗ i=1,...,n d(f i ), Y ∼ d(ν) ( 34
)
where lg is defined in Equation (33).

Theorem 5. Assume that all suppliers play according to Hannan consistent strategies for their loss upper bound then when T is large enough (T → +∞):

1 T T t=1 lg f(t), ν(t) ≤ Ṽg + 1 T T t=1 i=1,...,n ψ i D SS (i), D SS (i) k=1,...,K ν G k (t) (35)
Proof of Theorem 5. The proof can be found in Appendix C.

Corollary 6. Assume that the TSO plays according to a Hannan consistent strategy for its loss upper bound. Then when T is large enough (T → +∞):

1 T T t=1 lg f(t), ν(t) ≥ Ṽg - 1 T T t=1 i=1,...,n ψ i D SS (i), D SS (i) k=1,...,K ν G k (t) (36)
Proof of Corollary 6. Applying Theorem 5 to the TSO i.e, by symmetry, considering that the TSO's loss upper bound is the opposite of the sum over i of supplier S i 's loss upper bounds, and using von Neuman-Morgenstern's Minimax Theorem [START_REF] Myerson | Game Theory: An Analysis of Conflict[END_REF] for Ṽg , we derive the proposed inequality.

We let:

l g f(t), ν(t) i=1,...,n l i f(t), ν(t) (37)
be the sum of the suppliers' losses. Using the definitions established in Equations ( 33) and (37), we derive the following inequality:

l g f(t), ν(t) ≤ lg f(t), ν(t) + i=1,...,n l (2) i (d k ij (t)) j=1,...,n,k=1,...,K , ν(t) (38)
By substitution in Theorem 5, we obtain the following result:

Corollary 7. If all suppliers play according to a Hannan consistent strategy for their loss upper bounds then, when T is large enough (T → +∞), their average loss cannot be greater than:

Ṽg + 1 T T t=1 i=1,...,n ψ i D SS (i), D SS (i) k=1,...,K ν G k (t) + 1 T T t=1 i=1,...,n l (2) i (d k ij (t)) j=1,...,n,k=1,...,K , ν(t) ( 39
)
whatever strategy is chosen by the TSO.

Collaborative learning strategies

Cooperation takes place within coalitions. In cooperative Game Theory literature, a coalition is a group of agents who have incentives to collaborate by sharing resource access, information, etc., in the hope of increasing their revenue, knowledge, social welfare (in case of altruism), etc., compared to a case where they behave non-cooperatively [START_REF] Myerson | Game Theory: An Analysis of Conflict[END_REF], [START_REF] Saad | Coalitional Game Theory for Cooperative Micro-Grid Distribution Networks[END_REF]. Adapted to our learning context, we define coalitions of agents as follows:

Definition 8.

• A coalition of suppliers is a group of suppliers which collaborate to learn the hidden productions of the generators ν G k (t) k .

• Cooperation takes place within the coalition when its members share their information and align their predictions to a common value.

• The grand coalition contains all the suppliers involved in the learning task i.e., S i i=1,...,n

.

Shared information concerns only the power productions of the generators. Indeed, each supplier independently predicts the generation from its microgrid and has no impact on the other suppliers' forecasts of their microgrid generation.

In the following, we consider two information structures i.e., either suppliers cooperate through a grand coalition or coalitions of intermediate size might emerge. We derive bounds for the grand coalition average loss and compute analytically the probability for suppliers to deviate from the grand coalition.

The suppliers form a grand coalition

At this stage, the objective is to identify conditions on the disagreement levels between the suppliers regarding the forecasted power productions such that the term at the right of Ṽg defined in Corollary 7, remains as small as possible. Indeed, the smaller the term defined in Corollary 7, the smaller the upper bound of the sum of the agents' losses will be.

Such a strategy would minimize ψ i D SS (i), D SS (i) and l

(2) i (d k ij (t)) j,k , ν(t) at any time period. This implies that D SS (i) = D SS (i) = 0 and d k ij (t) = 0, ∀i, j, k, ∀t. This means that suppliers can decrease the upper bound of their average loss by coordinating their predictions about the power productions ν G k (t) k , at any time period t. Suppliers therefore have an incentive to form a grand coalition because it might enable them to decrease their total loss. Proposition 9. If the suppliers cooperate through a grand coalition and play Hannan consistent strategies, the suppliers' average loss over time interval [1; T ] when T is large enough (T → +∞) cannot be larger than: Ṽg + i=1,...,n

γ i β i 2 max{E G } min{E G } - min{E G } max{E G } -1 1 T T t=1 k=1,...,K ν G k (t)
Proof of Proposition 9. By substitution in l

i , as introduced in Proposition 2, since d k ij (t) = 0 for all i, j, k, we have:

l (2) i (d k ij ) j,k (t), ν(t) = γ i β i k=1,...,K ν G k (t) 2 min{E G } -ν G k (t) (40)
This depends on the supplier index (i) and on time period (t), and not on the suppliers' forecasts. In addition,

ψ i (0, 0) = γ i β i max{E G } min{E G } -min{E G } max{E G } .
As a result, applying Corollary 7, we obtain that the suppliers' average loss over time interval [1; T ] when T is large enough (T → +∞) cannot be larger than:

Ṽg + 1 T T t=1 i=1,...,n γ i β i max{E G } min{E G } - min{E G } max{E G } k=1,...,K ν G k (t) + 1 T T t=1 i=1,...,n γ i β i k=1,...,K ν G k (t) 2 min{E G } -ν G k (t) (41)
Then the proposition statement is straightforward.

Multiple coalitions of suppliers emerge

Let c and c be two coalitions of suppliers strictly included in the set {1, 2, ..., n}. The difference between supplier i's loss in case where it belongs to coalition c and coalition c can be obtained analytically:

l i f(t), ν(t) | i∈c -l i f(t), ν(t) | i∈c = γ i k=1,...,K ν G k (t) f i (ν G k , t)β i f i (ν G k , t) j∈c β j + j / ∈c f j (ν G k , t), β j - f i (ν G k , t)β i f i (ν G k , t) j∈c β j + j / ∈c f j (ν G k , t)β j (42)
Using the definition of the disagreement measure introduced in Equation ( 24) and the fact that i=1,...,n β i = 1, we obtain the following simplification:

f i (ν G k , t) j∈c β j + j / ∈c f j (ν G k , t)β j = f i (ν G k , t) -j / ∈c d k ij (t)β j and f i (ν G k , t) j∈c β j + j / ∈c f j (ν G k , t)β j = f i (ν G k , t) -j / ∈c d k ij (t)β j .
By substitution in Equation (42), we obtain:

l i f(t), ν(t) | i∈c -l i f(t), ν(t) | i∈c = β i γ i k=1,...,K ν G k (t) estimated by f i (ν G k , t) f i (ν G k , t) 1 f i (ν G k , t) -j / ∈c d k ij (t)β j - 1 f i (ν G k , t) -j / ∈c d k ij (t)β j (43)
Since at the beginning of period t, ν G k (t) is unknown, supplier i has to estimate it by f i (ν G k , t) in Equation (43). However, supplier i's choice of coalition entering depends on its forecast of each generator's production and also on its disagreements with the forecasts made by coalition c and c , which are unknown at the beginning of period t. Supplier i might estimate them by assuming that information is exchanged among coalitions but this is out of the scope of the article. Furthermore, due to the fair penalty constraint, we recall that β i ≥ 0, ∀i ∈ {1, ..., n}. Therefore an upper bound for Equation (43) is:

β i γ i k=1,...,K f i (ν G k , t) 2 1 f i (ν G k ,t)-Dss(i) j / ∈c β j - 1 f i (ν G k ,t)-Dss(i) j / ∈c β j
. If it is negative then supplier i will prefer entering coalition c than coalition c .

We want now to compute analytically the probability that supplier i leaves the grand coalition to enter a smaller size coalition c: Proposition 10. At period t, the probability for supplier i to prefer coalition c over the grand coalition depends on its forecasts of the generators' productions and of the game parameters as follows:

k=1,...,K f i (ν G k , t) f i (ν G k ,t)-max{E G } f i (ν G k ,t)(1-j / ∈c β j )+min{E G } j / ∈c β j k=1,...,K f i (ν G k , t) f i (ν G k ,t)-max{E G } f i (ν G k ,t)(1-j / ∈c β j )+min{E G } j / ∈c β j - f i (ν G k ,t)-min{E G } f i (ν G k ,t)(1-j / ∈c β j )+max{E G } j / ∈c β j
Proof of Proposition 10: We detail the computation of the difference between supplier i's loss in case where it belongs to coalition c and the grand coalition:

l i f(t), ν(t) | i∈c -l i f(t), ν(t) | i∈{1,..,n} = -β i γ i k=1,...,K ν G k (t)f i (ν G k , t) j / ∈c d k ij (t)β j f i (ν G k , t) f i (ν G k , t) -j / ∈c d k ij (t)β j (44)
The term between brackets can be lower bounded as follows:

- (f i (ν G k , t) -min{E G }) j / ∈c β j f i (ν G k , t) f i (ν G k , t)(1 -j / ∈c β j ) + max{E G } j / ∈c β j <0 ≤ j / ∈c d k ij (t)β j f i (ν G k , t) f i (ν G k , t) -j / ∈c d k ij (t)β j (45)
and upper bounded as follows:

j / ∈c d k ij (t)β j f i (ν G k , t) f i (ν G k , t) -j / ∈c d k ij (t)β j ≤ - (f i (ν G k , t) -max{E G }) j / ∈c β j f i (ν G k , t) f i (ν G k , t)(1 -j / ∈c β j ) + min{E G } j / ∈c β j >0 (46)
This implies that at period t the probability for supplier i to prefer coalition c over the grand coalition can be computed analytically as follows:

k=1,...,K f i (ν G k ,t) f i (ν G k ,t)-max{E G } f i (ν G k ,t)(1-j / ∈c β j )+min{E G } j / ∈c β j k=1,...,K f i (ν G k ,t) f i (ν G k ,t)-max{E G } f i (ν G k ,t)(1-j / ∈c β j )+min{E G } j / ∈c β j - f i (ν G k ,t)-min{E G } f i (ν G k ,t)(1-j / ∈c β j )+max{E G } j / ∈c β j .

Simulations

The aim of this section is to explain how the economic model of the hierarchical network, described in Section 2, can be applied in practice to take decisions in an uncertain context and then to check that the results derived analytically in Section 4 hold, for a given smart grid structure. The rest of the section is organized as follows: Subsection 5.1 deals with payoff function estimation for each forecast, Subsection 5.2 elaborates on the update of mixed strategies for each forecast and in the last part we discuss the numerical illustrations that we have obtained, for a large sample of parameters, considering non-cooperative and cooperative scenarios.

Payoff functions

At each time period, each supplier must make K + 1 forecasts: one for its microgrid power generation and one to evaluate the fluctuating production of each of the K generators. As a result, each supplier should define a randomized strategy on the space E S × E K G . We recall that a randomized strategy is the standard terminology used in Game Theory for a discrete density function defined over the considered set [START_REF] Myerson | Game Theory: An Analysis of Conflict[END_REF]. The size of the set grows very fast with K and, as a result, each probability in the randomized strategy of forecasts is very low, with the effect that errors are rounding off during computation. In order to overcome this issue, we decided to divide the suppliers into smaller entities, with each making only one forecast at each time period, and to consider that these entities are uncoupled. This trick results in K + 1 randomized strategies in the space of forecasts E S × E K G for each supplier. For a given forecast X, we derive the payoffs for each value x ∈ E (E = E S for power generation from the microgrid and E = E G for power productions from the generators) of the forecast at each time period t by using the utilities of the suppliers and retaining only those terms that depend on forecast X. This is summarized in the following definition: Definition 11. The payoff function associated with forecast X, evaluated in x ∈ E, coincides with the utility of supplier S i restricted to its terms depending solely on forecast X and evaluated in x.

For the forecasts of power generation from microgrid M i , supplier S i 's payoff takes the form:

H f i (ν S i ) (x, t) = γ i + θ i (b 0 -b 1 x) 2 b 0 2b 1 - γ i 2b 1 θ i -ν S i (t) + x 2 -γ i x 2 (47)
Concerning the forecasts of generator G k 's power production, supplier S i 's payoff takes the form:

H f i (ν G k ) (x, t) = - β i γi n -1 δ x + γ i β i x j=1,...,n,j =i β j f j (ν G k , t) + β i x ν G k (t) (48)
As already stated in Section 4, we will also consider that the TSO is non oblivious and tries to maximize the sum of the suppliers' losses. As for the suppliers, we uncouple ν S i (t) and ν G k (t) to improve the computation. More precisely the TSO's payoffs are:

H ν S i (x, t) = θ i (b 0 -b 1 f i (ν S i , t)) -γ i 2 x (49)
and

H ν G k (x, t) = -i=1,...,n γ i β i f i (ν G k , t) j=1,...,n β j f j (ν G k , t) x (50)
It is very straightforward to adapt the repeated learning game and payoffs when considering that the suppliers integrate a coalition c. The coalition c payoffs take the following forms:

H fc(ν S i ) (x, t) = H f i (ν S i ) (x, t)∀i ∈ c H fc(ν G k ) (x, t) = - i∈c β i γi n -1 δ x + i∈c γ i β i x i∈c β i x + i / ∈c β i f i (ν G k , t) ν G k (t) (51)
The TSO's payoffs H ν S i (x, t) and H ν G k (x, t) are unchanged.

Updates of forecasting strategies

We consider two types of update for the forecasting randomized strategies based on the exponential forecaster for signed games: one based on external regret and the other based on internal regret. We assume that the game considered in this article is a signed game because the range of values of payoff function H X might include a neighborhood of 0.

We let:

ϑ t t s=1 Var H X (X s , s)) = t s=1 E H X (X s , s))-E[H X (X s , s)]
2 be the sum of the variances associated with the random variable H X (X t , t) under the mixed strategy d t (X) which is defined over space E. Using the exponential forecaster for signed games with external regret [START_REF] Cesa-Bianchi | Prediction, Learning, And Games[END_REF] means that the mixed strategy is updated according to the algorithm described below.

External Regret Learning Algorithm: Updating of the Exponential Forecaster Initialization. For t = 0, we set: w 0 (x) = 1 |E| , ∀x ∈ E.

Step 1 to T . The updating rules are the following:

dt(x) = wt(x) x∈E wt(x) , ∀x ∈ E w t+1 (x) = exp η t+1 t s=1 H X (x, s) = wt(x) η t+1 η t exp η t+1 H X (x, t) , ∀x ∈ E η t+1 = min 1 2 max{|H X |} ; 2( √ 2 -1) e -2 ln|E| ϑt ϑ t+1 = ϑt + Var H X (X t+1 , t + 1)
For the internal regret, the definition of which was introduced in Subsection 4.2, the updating rules are similar but with d t (.) = y =y d y→y t (.)∆ (y,y ) (t) where d y→y t (.) is the modified forecasting strategy obtained when the forecaster predicts y each time it would have predicted y and

∆ (y,y ) (t) exp η t t-1 s=1 x∈E d y→y s (x)H X (x, s) z =z exp η t t-1 s=1 x∈E d z→z s (x)H X (x, s) (52)
We note that, if we take the notation w (y,y ) (t) exp η t t-1 s=1 x∈E d y→y s (x)H X (x, s) , then the weights in the internal regret learning algorithm are updated according to the following updated rule:

∆ (y,y ) (t) = w (y,y ) (t) z =z w (z,z ) (t) (53)
w (y,y ) (t) = w (y,y ) (t -1)

η t η t-1 exp η t x∈E d y→y t-1 (x)H X (x, t -1) (54)

Numerical illustrations

Convergence times and incentives to cooperate: In this first part of the simulation, we consider two suppliers (n = 2). We compare the cumulative payoff of each agent (supplier or grand coalition) to the cumulative payoff of the same agent in a case where it has forecasted the best value at each time period in terms of payoffs. More precisely, we compute for each agent a (the supplier S 1 , S 2 or the grand coalition C), the following performance metric:

R a (T ) 1 T T s=1 X∈Fa H X (X s , s) -max x∈E (H X (x, s))
where F a is the generic set of forecasts made by agent a. Then, we measure the convergence of the learning algorithm through the convergence of this performance metric. That is to say, we consider that convergence is reached when the variation of the performance metric, Ra(T )-Ra(T -1)

Ra(T -1)
, is less than 10 -2 . We let In our simulations, we calculated the convergence times of learning algorithms for a wide range of combinations of penalty coefficients (γ 1 ,γ 2 , γ1 , γ2 ). More precisely, in Figures 1 and2 (a), we make the assumption that:

γ 1 = 2γ 2 = 2γ ∈ [0; 1] while in (b): γ = γ 1 = γ 2 ∈ [0; 1].
For each figure, we use 1000 combinations corresponding to 10 values equally distributed between 0.1 and 1 for γ, γ1 and γ2 . We can easily check that all these penalty coefficient combinations satisfy Equation ( 16).

In addition, we chose E S = [START_REF] Costa | A Review on the Young History of the Wind Power Short-Term Prediction[END_REF][START_REF] Foster | Learning, Hypothesis testing, and Nash equilibrium[END_REF] and E G = [1, 2] so that Equation ( 21), i.e. power shortage, is always true.

In Figure 1, the learning strategies of the suppliers and the grand coalition are based on external regret minimization while in Figure 2, they rely on internal regret minimization. In the top of Figures 1 and2 ). From the top figures in both cases we observe that, for a far larger number of penalty coefficients, the learning algorithm convergence times are shorter under cooperative scenarios than under non-cooperative scenarios. Furthermore, by comparison of Figures 1 and2 top, the convergence times are shorter for learning algorithms based on internal regret minimization than for learning algorithms based on external regret minimization, though convergence occurs under both regret criteria.

Regarding the potential for a grand coalition to emerge, we infer from Figure 1 (a) and (b) (resp. 2 (a) and (b)) bottom, that for 97% of the penalty coefficient combinations at least one supplier has incentives to cooperate and that for 95% (resp. 94.5%) of the combinations of penalty coefficients both suppliers have incentives to cooperate, using external regret minimization (resp. internal regret minimization) as criterion.

In terms of scalability, the complexity of our learning algorithm is in O(nK). An interesting property is that it can be easily parallelized (one agent corresponding to one core) due to the fact that the weight updating rules are specific to each supplier (i.e., they do not depend on the other suppliers' forecasts).

Tightness of the bounds:

To measure performance, we chose to compute two upper bounds: firstly, the upper bound derived in Corollary 7, where the suppliers perform individual learning. This is called BOUND SELFISH. Secondly, the upper bound derived in Proposition 9, where the suppliers enter a grand coalition and align their forecasts of the generators' productions. This is called BOUND COALITION. Both upper bounds are matched with the sum of the suppliers' average loss:

1 T T t=1 l g (f(t), ν(t)). The latter is computed under external (AV. LOSS ext,. ) and internal regret minimization (AV. LOSS int,. ), assuming that either the suppliers' learning process is non-cooperative or cooperative. These performance measures are averaged over 10 3 combinations of parameters (γ, γ1 , γ2 ), each parameter takes one of the 10 values equally distributed between 0.1 and 1. In Table Incentives for two player coalitions to emerge: In this lart part of the simulation, we consider three suppliers (n = 3). In terms of organization, this means that five coalition structures might emerge: either the suppliers learn selfishly leading to the coalition structure {S 1 }, {S 2 }, {S 3 }; or they form a grand coalition C = {S 1 , S 2 , S 3 }; or two player coalitions emerge leading to three possible combinations {S 1 , S 2 }, {S 3 } or {S 1 , S 3 }, {S 2 } or {S 2 , S 3 }, {S 1 }. Staying in line with the notations used in the two supplier simulations, we denote CS the set containing all the coalition structures and T * c the number of time steps needed for the regret-based algorithm for coalition c ∈ CS to converge. The algorithms are run for T = 100 time periods and we use 10 4 combinations corresponding to 10 values equally distributed between 0.1 and 1 for γ, γi , ∀i = 1, 2, 3. Going back to the definition of convergence times, coalition structures different from the grand coalition will emerge if, and only if,

Conclusion

In this article, we study a model of decentralized renewable generations in which generators, suppliers and microgrids are organized into a hierarchical network. Renewable generations are modeled by random individual sequences which need not have a probabilistic structure. This general demand and supply structure is capable of taking into account exogenous events. As a result, it is more robust to extreme events and appears particularly suitable for modeling fairly erratic processes such as renewables. We analytically determine the optimal prices that enable generators to avoid the penalties that the balance operators threaten to apply when suppliers' orders are not entirely satisfied. All the risk is then transferred to the suppliers. Additionally, we prove that the latter can minimize their average risk by sharing information in a grand coalition containing all the suppliers. Finally, numerical simulations, run on a large sample of parameter combinations, lead us to observe that the convergence times in collaborative learning, performed either through a grand coalition or through coalitions having smaller cardinality but containing more than one supplier, are clearly lower than times resulting from individual learning and that they are lower for learning algorithms based on internal regret minimization than for external regret minimization, though convergence occurs under both criteria. The tightness of convergence bound under collaborative learning is shown to be clearly better than for individual learning. where ξ 2 (x) = 1 -x min{E G } 1 x≥0 -

x max{E G } 1 x<0 . Similarly, we have:

1 max{E G } ≤ 1 C k (f -i (y, t)) ≤ 1 min{E G } (64)
As a result, we obtain for all y ∈ E S × E K G :

γ i β i k=1,...,K ν G k (t) 2 max{E G } - ν G k (t
In Section 4, we introduced lower and upper bounds on the disagreements between supplier s i and the other suppliers regarding the forecasts of the generations: D SS (i) and D SS (i). Since ξ 1 and ξ 2 are decreasing in x (they are linear functions by parties with negative coefficients), we have ξ 1 (D SS (i)) ≤ ξ 1 (x) ≤ ξ 1 (D SS (i)) and ξ 2 (D SS (i)) ≤ ξ 2 (x) ≤ ξ 2 (D SS (i)) for any disagreement value x.

Then, since j=1,...,n β j = 1, the lower bound of ζ(y, f(t), ν(t)) is: b l (i, t) = γ i β i k=1,...,K

ν G k (t) 2 max{E G } - ν G k (t) ξ 2 (D SS (i))
.

Whereas, the upper bound takes the form: b u (i, t) = γ i β i k=1,...,K

ν G k (t) 2 min{E G } - ν G k (t) ξ 1 (D SS (i))
. If Inequality (62) is checked, then the following inequality holds: Appendix C: Proof of Theorem 5

lim sup T →+∞ 1 T T t=1 l (1)
With the proposed expression of ψ i , the upper bound of the external regret evaluated in supplier S i 's loss, l

i , becomes:

We assume that each supplier makes its forecasts independently of the other suppliers. Then lg X, ν(t) is linear in X. As a result, its minimum over the simplex of probability vectors is reached in one of the corners of the simplex. Consequently:

  (a) and (b), we plot the histograms of the maximum of T * S 1 , T * S 2 (resp. T * C ) left (resp. right) for all the combinations of penalty coefficients (γ, γ1 , γ2 ). The bin heights of each histogram are determined by the number of penalty coefficients that have the same convergence time. The algorithms are run for T = 100 time periods. At the bottom of Figures1 and 2

  (a) and (b), we plot the ratio of the maximum (resp. minimum) of T * S 1 and T * S 2 over T * C , left (resp. right), for all penalty coefficient combinations (γ, γ1 , γ2

Figure 1 :

 1 Figure 1: Convergence times and incentives to collaborate under external regret minimization for two suppliers. In (a), we have: γ 1 = 2γ 2 = 2γ ∈ [0; 1] and in (b), we have: γ = γ 1 = γ 2 ∈ [0; 1]. At the top of each subfigure, we plot the histograms of the maximum of T * S 1 , T * S 2 (resp. T * C ) left (resp. right) for all the combinations of penalty coefficients and for a maximum number of time periods T = 100. At the bottom left (resp. right), we plot the ratio of the maximum (resp. minimum) of T * S 1 , T * S 2 over T * C as a function of all the penalty coefficient combinations.

Figure 2 :

 2 Figure 2: Convergence times and incentives to collaborate under internal regret minimization for two suppliers. In (a), we have: γ 1 = 2γ 2 = 2γ ∈ [0; 1] and in (b), we have: γ = γ 1 = γ 2 ∈ [0; 1]. At the top of each subfigure, we plot the histograms of the maximum of T * S 1 , T * S 2 (resp. T * C ) left (resp. right) for all the penalty coefficient combinations and for a maximum number of time periods T = 100. At the bottom left (resp. right), we plot the ratio of the maximum (resp. minimum) of T * S 1 , T * S 2 over T * C as a function of all the penalty coefficient combinations.

≤ 1 ..

 1 Furthermore, coalition structure CS 1 is preferred other coalition structure CS 2 if, and only if, In Figures3 (a) and (c) we have represented the ratio max{T * c ,c∈CS} T * C for the five coalition structures listed above, under external and internal regret minimization respectively. We observe that: • Coalitions containing less than three suppliers might emerge for 50 combinations of the parameters under external regret minimization and under 150 combinations of the parameters under internal regret minimization • Median values of convergence times of coalitions containing two suppliers are smaller than full competition but larger than the grand coalition, as illustrated by the Box and Whiskers plots in Figure 3 (b) and (d)

Figure 3 :

 3 Figure 3: Incentives for suppliers to form intermediate size coalitions under external (a),(b) and internal (c),(d) regret minimization. For the simulations, we take: γ = γ i ∈ [0; 1], ∀i{1, 2, 3}. In (a), (c), we evaluate the agents' incentives to maintain their coalition structure rather than integrating the grand coalition by comparing their convergence times. In (b), (d), we represent Box and Whiskers plots associated with the convergence times for each coalition structure.

  i, t) ≤ 0. This last inequality provides an upper bound for the external regret associated with supplier S i 's partial loss.

  SS (i), D SS (i) T t=1 k=1,...,K ν G k (t)(66)Summing Inequality (66) over all i = 1, ..., n, the external regret evaluated in the sum of the suppliers' losses (l ..,n ψ i D SS (i), D SS (i)

E 1 E

 1 lg (X, Y)|X ∼ ⊗ i d(f i ), Y = ν(t) {ν(t)=z} be the marginal empirical frequency of play evaluated in prediction z ∈ E n S × E K G . We obtain: min⊗ i d(f i )∈F n S lg (X, Y)|X ∼ ⊗ i d(f i ), Y = ν(t) = min ⊗ i d(f i )∈F n S z∈E n S ×E K G d T (z)E lg (X, Y)|X ∼ ⊗ i d(f i ), Y = z = min ⊗ i d(f i )∈F n S E lg (X, Y)|X ∼ ⊗ i d(f i ), Y ∼ d T ≤ max d(ν)∈Fm min ⊗ i d(f i )∈F n S E lg (X, Y)|X ∼ ⊗ i d(f i ), Y ∼ d(ν) Ṽg (69)

  Proportion of power allocated by G k to S i Random variables and forecasters: ν

	) Parameters: S i utility evaluated in unbiased forecasts t Generic time period n Number of suppliers K Number of generators pk (t) G k unitary price q ik (t) Power order from S i to G k p i (t) S i unitary price a i (t) M i decision variable θ Decision variables: α ki (t)

i Parameter depending on the composition of M i b 0 , b 1 Generic positive parameters γ i S i unitary penalty cost γi G k unitary penalty cost in the provision of S i

  T * S i , i = 1, 2 (resp. T * C ) be the number of time steps needed for the regret-based algorithm for supplier S i (resp. the grand coalition) to converge. According to these notations, supplier S i has incentives to cooperate if, and only if, T * S i ≥ T * C . Depending on the position of T * C with respect to min{T * S 1 ; T * S 2 } and max{T * S 1 ; T * S 2 } we identify three emerging behaviors: • Both suppliers have incentives to cooperate if, and only if, min{T * S 1 ; T * S 2 } ≥ T * C . The smart grid is instable (one supplier having an incentive to cooperate and not the other) if, and only if, min{T *

	T * C ≤ max{T * S 1 ; T * S 2 }	S 1 ; T * S 2 } <

• The suppliers have no incentive to cooperate if, and only if, max{T * S 1 ; T * S 2 } < T * C •

Table 1 :

 1 1, we compute BOUND SELFISH, AV. LOSS ext,selfish and AV. LOSS int,selfish for a fixed number of values of γ 1 γ 2 . In Table 2, we compute BOUND COALITION, AV. LOSS ext,coal and AV. LOSS int,coal for a fixed number of values of γ 1 γ 2 . According to both tables, we observe that BOUND COALITION is striclty smaller than BOUND SELFISH and that AV. LOSS .,coal is striclty smaller than AV. LOSS .,selfish whichever learning scenario (i.e., either noncooperative or cooperative) is chosen by the suppliers. Comparison of the upper bound derived in Corollary 7 with the suppliers' average loss in cases where the latter learn the generators'

	γ 1 γ 2 0.2	BOUND SELFISH 1.539.10 -2	AV. LOSS ext,selfish -8.152.10 -3	AV. LOSS int,selfish -2.204.10 -2
	0.5	0.00	-6.611.10 -3	-1.738.10 -2
	0.7	1.634.10 -2	-7.150.10 -3	-1.604.10 -2
	1.0	3.668.10 -2	-7.630.10 -3	-1.624.10 -2
	1.5	6.627.10 -3	-4.395.10 -3	-1.719.10 -2
	2.0	0.0	-3.124.10 -3	-1.552.10 -2
	2.5	3.371.10 -2	-2.853.10 -3	-1.233.10 -2
	3.0	0.00	-1.755.10 -3	-9.433.10 -3
	3.5	0.00	-1.600.10 -3	-1.060.10 -2
	4.0	0.00	-7.738.10 -4	-6.675.10 -3

power productions non-cooperatively through external and internal regret minimization.

Table 2 :

 2 Comparison of the upper bound derived in Proposition 9 with the suppliers' average loss in cases where the latter enter a grand coalition and align their forecasts of the generators' power productions under external and internal regret minimization.

  )

					ij (y, t)) β j ξ 2 (d k	≤ ζ(y, f(t), ν(t))
					j=1,...,n
	≤ γ i β i	k=1,...,K	ν G k (t) 2 min{E G }	-	ν G k (t) β j ξ 1 (d k ij (y, t))
					j=1,...,n

Yeloha http://www.digitaltrends.com/cool-tech/yeloha-solar-panel-sharing/

Vanderbron http://www.fastcoexist.com/3036271/the-sharing-economy-takes-on-electricity-so-you-can-buy-your-power-from-neighbors

Non-renewable generators like nuclear plants could be integrated into the grid. This would require using distributed control rules, such as those described in[START_REF] Cadre | Distributed Learning in Hierarchical Networks[END_REF],[START_REF] Marden | Achieving Pareto Optimality Through Distributed Learning[END_REF]. Furthermore, control variables on both the generators and microgrid size on their production could be introduced. This would constrain the values taken by the generators' production processes and change the updating rules of the microgrid generation forecasters.

French Transmission System Operator Balance Rules http://clients.rte-france.com/lang/fr/clients_producteurs/ services_clients/dispositif_re.jsp [Online September 2015]

Note that at the individual household scale, the net demand might be negative since the household can produce power through non renewable sources and use it for its own consumption.

In the smart grid, the monitoring is performed through communicating meters deployed at the end user level[START_REF] Atzeni | Demande-Side Management via Distributed Energy Generation and Storage Optimization[END_REF],[START_REF] Wong | Autonomous Demand-Side Management Based on Game-Theoretic Energy Consumption Scheduling for the Future Smart Grid[END_REF].

Appendix A: Proof of Proposition 2

Following a number of calculations, we obtain: a i (ν(t), ν(t))p i (ν(t), t) -a i (f i (t), ν(t))p i (f i (t), t)

and

Given supplier S i 's loss due to power shortage conditions, we have:

where:

In addition, we observe that:

where

To obtain the inequality above, we followed the logic that since E G is a close subset of R, the forecasters are upper and lower bounded i.e., min{E

As a result, we have:

We then introduce the notation:

We obtain an upper bound of l i f(t), ν(t) as the sum of two parts: the first, l

i , depends exclusively on its predictions and the second, l

i , depends on its interactions with the others' predictions.

Appendix B: Proof of Lemma 4

Suppose that supplier S i plays according to a Hannan consistent strategy. Taking into account Equation (57) defining the loss of supplier S i , this means that:

.

Let d k ij (y, t) denote the disagreement between supplier S i and supplier S j when supplier S i makes the prediction y at time period t without any change in the predictions of the other suppliers. Following the same approach as in Appendix A, we obtain for all y ∈ E S × E K G : 1 j=1,...,n