Incentives to Implement Controllers for a Fleet of Batteries Using Cooperative Game Theory

In the previous chapters we investigated a model in which a set of players jointly invested into energy storage. In this chapter, we will briefly divert from the previous model, and we will show how the same techniques and tools, namely linear production games, can be used to model the distributed and cooperative control of a fleet of batteries. This problem is in a sense, complementary to our previous studies since we now deal with batteries that are already installed.

R ésum é : Le d éploiement massif de ressources d' énergie renouvelable distribu ée (RED) repr ésente une opportunit é majeur pour atteindre les objectifs de r éduction des émissions de carbone en Europe, mais aussi dans le monde entier. Visant à mobiliser des capitaux publics et priv és, plusieurs plans ont ét é d évelopp és pour placer les clients finaux au coeur de la transition énerg étique, dans l'espoir d'acc él érer l'adoption de l' énergie verte en augmentant son attractivit é et sa rentabilit é. Certains syst èmes propos és incluent la cr éation de march és locaux d' énergie, o ù les r ésidences peuvent vendre leur énergie dans leur quartier à un prix plus élev é que celui que les fournisseurs classiques seraient pr êts à payer (mais inf érieur à ce que les autres clients paieraient à ces fournisseurs); des investissement partag és, les consommateurs poss édant dans ce cadre un g én érateur d écarbon é et/ou du stockage pour lesquels ils obtiennent des dividendes dans un contexte d'autoconsommation collective o ù plusieurs familles sont " cach ées " derri ère le m ême compteur intelligent, leur permettant d'optimiser leur profil de consommation agr ég é et donc maximiser la valeur de leur investissement. L'un des principaux objectifs de la th èse est de fournir des m éthodes pour augmenter les gains potentiels et des mod èles pour évaluer l'impact que l'on peut attendre de ces diff érentes solutions, afin qu'elles deviennent une incitation plus forte à la g én ération et usage d' énergies renouvelables, car celles-ci joueront un r ôle crucial dans la lutte contre le changement climatique seulement si elles sont correctement mises en oeuvre. Pour ce faire, nous concevons un cadre permettant de concevoir et de comparer divers paradigmes d'investissements partag és et d' échanges mon étis és locaux de l' énergie , dont le potentiel de gains se traduit par une incitation forte à leur mise en oeuvre. Dans le cadre d' échanges mon étis és locaux d' énergie, nous étudions les interactions entre prosommateurs (consommateurs avec capacit é de production et éventuellement de stockage) situ és dans le m ême r éseau Basse Tension, éventuellement derri ère le m ême d épart. Dans nos syst èmes, ces prosommateurs seront toujours connect és au r éseau électrique principal et ils auront la possibilit é, comme ils le font aujourd'hui, d'acheter et de vendre à un op érateur de services de distribution d' électricit é, suivant une politique tarifaire connue à l'avance (un taux forfaitaire ou un temps d'utilisation, pour exemple). Pour que ces agents b én éficient pleinement des avantages des échanges locaux d' énergie, nous supposons qu'ils poss èdent des appareils (tels que des batteries) qui, sans modifier leur demande énerg étique interne (sans changer leur comportement d'usage), peuvent leur permettre de modifier leur demande énerg étique nette vue de l'ext érieur de leur domicile. En mod élisant les prosommateurs comme des maximisateurs rationnels de l'utilit é (que nous d éfinissons), ils planifieront les flux entrant et sortant de leur batterie pour diminuer le co ût associ é à leur demande nette d' énergie (avec comme signal é, une demande perc ¸ue qui reste inchang ée). Dans la premi ère partie de la th èse, nous étudions des mod èles concurrentiels dans lesquels les prosommateurs vendent leur surplus à leurs voisins via un march é local d' énergie. Nous analysons diff érents types de march és et donc diff érentes strat égies que les acteurs pourraient utiliser pour participer à ces march és, ainsi que leur impact sur le r éseau électrique et sur le gestionnaire du r éseau de distribution. Dans la deuxi ème partie de la th èse, nous explorons les incitations qui peuvent être mises en oeuvre par la coop ération. À cet égard, nous utilisons la th éorie des jeux coop ératifs pour mod éliser l'investissement partag é dans l'acquisition de dispositifs de stockage énergie et de panneaux photovoltaïques (PV) par un groupe de prosommateurs.

Title : Mechanisms and architectures to encourage the massive and efficient use of local renewable energy Keywords : Smart Grids, Renewable energy, Local Energy Markets, Energy cooperatives, Demand response, Game theory Abstract : To meet carbon reduction goals in Europe but worldwide too, a large number of renewable distributed energy resources (DER) still need to be deployed. Aiming at mobilizing private capital, several plans have been developed to put end-customers at the heart of the energy transition, hoping to accelerate the adoption of green energy by increasing its attractiveness and profitability. Some of the proposed models include the creation of local energy markets where households can sell their energy to their neighbors at a higher price than what the government would be willing to pay (but lower than what other customers normally pay), shared investment models in which consumers own a carbon-free power plant such as a wind turbine or a solar farm and they obtain dividends from its production to collective auto-consumption models in which several families are 'hidden' behind the same smart meter, allowing them to optimize their aggregated consumption profile and therefore maximizing the value of their DER. One of the main objectives of the thesis is to understand these different incentives as they will play a crucial role in tackling climate change if correctly implemented. To do so, we design a framework for 'local energy trading' that encompasses a large number of incentives. In the context of local energy trading, we study the interactions of prosumers (consumers with generation capabilities) located in the same Low Voltage network, possibly behind the same feeder. These prosumers will still be connected to the main power grid and they will have the option, as they do today, to buy and sell to/from their utility company at a fixed price (a flat rate or a Time-of-Use, for example). For these agents to fully benefit from the advantages of local energy trading, we shall assume that they own appliances (such as batteries) that, without changing their perceived energy demand, can enable them to change their net energy demand as seen from outside their homes. Modeling prosumers as rational uti-lity maximizers, they will schedule their battery to decrease the cost associated with their net energy demand (as their perceived demand remains unchanged). In the first part of the thesis, we investigate competitive models in which prosumers sell their surplus to their neighbors via a local energy market. We analyze different strategies that players could use to participate in these markets and their impact on the normal operation of the power grid and the Distribution System Operator. In this regard, it is shown that sequential markets can pose a problem to the system and a new market mechanism that exploits domain knowledge is proposed to increase the efficiency of the local trades. In the second part of the thesis, we delve into incentives that can be implemented through cooperation. In this regard, we use cooperative game theory to model the shared investment into energy storage and photovoltaic panels (PV) by a group of prosumers. For the studied model we show that a stable solution (in the core of the game) exists in which all participants cooperate and we provide an efficient algorithm to find it. Furthermore, we also show that cooperation is stable for participants that already own batteries and PVs but prefer to operate them in coordination to increase their value, effectively implementing collective auto-consumption. Finally, we demonstrate how to integrate both models: the shared investment and the cooperative control of existing resources into a single cooperative framework which also enjoys the existence of stable outcomes. For this later model, we propose to decouple the return over investments (ROI) obtained between the ROI produced by the investment in hardware and the ROI obtained by cooperation itself. By doing so, we can offer the former profit to external investors to raise the required capital (although nothing forbids the member of the coalition to contribute) and the latter to the actual consumers.

Institut Polytechnique de Paris 91120 Palaiseau, France

Résumé

Le déploiement massif de ressources d'énergie renouvelable distribuée (RED) représente une opportunité majeure pour atteindre les objectifs de réduction des émissions de carbone en Europe, mais aussi dans le monde entier. Visant à mobiliser des capitaux publics et privés, plusieurs plans ont été développés pour placer les clients finaux au coeur de la transition énergétique, dans l'espoir d'accélérer l'adoption de l'énergie verte en augmentant son attractivité et sa rentabilité.

Certains systèmes proposés incluent la création de marchés locaux d'énergie, où les résidences peuvent vendre leur énergie dans leur quartier à un prix plus élevé que celui que les fournisseurs classiques seraient prêts à payer (mais inférieur à ce que les autres clients paieraient à ces fournisseurs); des investissement partagés, les consommateurs possédant dans ce cadre un générateur décarboné et/ou du stockage pour lesquels ils obtiennent des dividendes dans un contexte d'autoconsommation collective où plusieurs familles sont " cachées " derrière le même compteur intelligent, leur permettant d'optimiser leur profil de consommation agrégé et donc maximiser la valeur de leur investissement.

L'un des principaux objectifs de la thèse est de fournir des méthodes pour augmenter les gains potentiels et des modèles pour évaluer l'impact que l'on peut attendre de ces différentes solutions, afin qu'elles deviennent une incitation plus forte à la génération et usage d'énergies renouvelables, car celles-ci joueront un rôle crucial dans la lutte contre le changement climatique seulement si elles sont correctement mises en oeuvre. Dans le même temps, nous restons vigilants quant aux éventuels effets secondaires que pourraient avoir les nouvelles incitations, qui pourraient conduire à une dégradation des performances du système.

Pour ce faire, nous concevons un cadre permettant de concevoir et de comparer divers paradigmes «d'investissements partagés et d'échanges monétisés locaux de l'énergie», dont le potentiel de «gains» se traduit par une incitation forte à leur mise en oeuvre.

Dans le cadre d'échanges monétisés locaux d'énergie, nous étudions les interactions entre prosommateurs (consommateurs avec capacité de production et éventuellement de stockage) situés dans le même réseau Basse Tension, éventuellement derrière le même départ. Dans nos systèmes, ces prosommateurs seront toujours connectés au réseau électrique principal et ils auront la possibilité, comme ils le font aujourd'hui, d'acheter et de vendre à un opérateur de services de distribution d'électricité, suivant une politique tarifaire connue à l'avance (un taux forfaitaire ou un temps d'utilisation, pour exemple). Pour que ces agents bénéficient pleinement des avantages des échanges locaux d'énergie, nous supposons qu'ils possèdent des appareils (tels que des batteries) qui, sans modifier leur demande énergétique interne (sans changer leur comportement d'usage), peuvent leur permettre de modifier leur demande énergétique nette vue de l'extérieur de leur domicile. En modélisant les prosommateurs comme des maximisateurs rationnels de l'utilité (que nous définissons), ils planifieront les flux entrant et sortant de leur batterie pour diminuer le coût associé à leur demande nette d'énergie (avec comme signalé, une demande perçue qui reste inchangée).

Dans la première partie de la thèse, nous étudions des modèles concurrentiels dans lesquels les prosommateurs vendent leur surplus à leurs voisins via un marché local d'énergie. Nous analysons différents types de marchés et donc différentes stratégies que les acteurs pourraient utiliser pour participer à ces marchés, ainsi que leur impact sur le réseau électrique et sur le gestionnaire du réseau de distribution.

Dans la deuxième partie de la thèse, nous explorons les incitations qui peuvent être mises en oeuvre par la coopération. À cet égard, nous utilisons la théorie des jeux coopératifs pour modéliser l'investissement partagé dans l'acquisition de dispositifs de stockage énergie et de panneaux photovoltaïques (PV) par un groupe de prosommateurs.

5

Acknowledgments

This thesis was financed by: The VALADOE Chair of IMT Atlantique in partnership with Télécom ParisTech and Mines Saint Etienne and supported by ENEDIS, Région Pays de la Loire, Nantes Metropole.

Writing my PhD thesis has been a roller coaster of emotions, from frustrating moments where the only option seemed to be quiting, to moments of pure joy and excitement. I am completely aware that this journey would not have been possible without the support and help that I received. I would like to thank everyone who was there with me in these years.

I would specially like to thank Daniel Kofman for providing me with the opportunity, the vision and the companionship during these years. Your help was invaluable to figure out how to continue when the path forward was not clear and your mentorship transcended the boundaries of the PhD. I would also like to thank Patrick Maillé, your insights and discussions about models and approaches to research helped me greatly to improve.

I would like to thank Ariel Orda, for his knowledge of game theory, good practices for writing but mostly, for being a companion in most of the projects and ideas that I had during the thesis. I would also like to thank Ana Busic, who opened her door to work with me since the beginning and from whom I have learned a lot.

I would like to thank José Horta, who was always there when I had doubts, ideas or even needed help in opening a bank account.

I would like to thank Pierre Achaichia for his time and explanations on the inner workings of a DSO, as well as for his hospitality while I was in Nantes. I would also like to thank the members of the VALADOE chair for their cooperation and support.

Finally, to my family, whose unconditional support is the reason I am here today (not here à la Tom Riddle, inside the paper, but in a figurative way of speaking).

CONTENTS

List of Figures

1. [START_REF] Kiedanski | Strategy-proof local energy market with sequential stochastic decision process for battery control[END_REF] The famous duck curve in California. It depicts a very low net energy demand at times when solar production is at its peak followed by a very steep increase in consumption afterwards. Image obtained from [START_REF]Duck curve[END_REF]. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.2 A graphical example of the three scenarios described for a household with a solar panel and a smart meter that consumes and injects into the main grid. Depicted in the image are the three possible total costs of the participant in each of the scenarios at the end of the billing period. The arrows on the left represent the net energy in the time-slot after using as much solar as needed for self-consumption. 1.3 Simple representation of a High -Medium and a Medium -Low Voltage transformers, the load of two households and the aggregated view of the load from the MV grid. The traditional electricity company sells energy from the HV/MV side to the LV side. The scope of the thesis is mostly limited to the Distribution Grid and in particular, to households behind the same Low voltage substation, such as the group in the right side of the image. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . topologies for an increasing number of players. The simulations had a 10000 cut-off. . . . . . . . . . 12.3 Running time of the different algorithms with a varying number of players. The y-axis is in log scale. 12.4 Cumulative distribution of the total cost . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.1 Comparison of two different ways to assign to distribute the cost to the players. In one way, all the profits of cooperation are evenly distributed among the participants whereas in the other one, an approximation of the payoffs in the core as described in the previous section was used. The top figure plots the results with the corresponding outliers, whereas the bottom image zooms-in near the x-axis. 13.2 Comparison of the distance between the optimal discrete solution and the optimal continuous solution with respect to the size of the investment. .

Introduction

General Context and Motivation

Energy consumption is a basic human need in today's western modern society: it powers the artificial respirators needed to face the COVID-19 crisis and the TV's in which we watch Netflix to numb us from the same global pandemic.

Unlike other commodities, massively storing energy has always been mostly unfeasible (partly due to the costs of energy storage solutions) and power systems had to be designed to balance consumption and production at all times [START_REF] Luo | Overview of current development in electrical energy storage technologies and the application potential in power system operation[END_REF].

The task of generating the exact amount of energy that is being consumed at all times is not an easy one. It is not surprising then that electrification has been considered the greatest achievement of human engineering in the past century [START_REF] Constable | A Century of Innovation: Twenty Engineering Achievements that Transformed our Lives[END_REF]. Historically, the output of most power plants (industrial facilities for the generation of electric power) was completely controllable (up to the physical constraints of the technologies used), which meant that, through the design of appropriate controllers, it was possible to adapt production to match consumption at all times.

Unfortunately, most traditional electricity generation sources, and particularly the more controllable ones, called peak plants, emit an enormous amount of CO2 into the atmosphere and represent one of the main causes of global warming (In 2018 alone, coal-fired generation emitted 10.1 gigatons of CO2 into the atmosphere, one third of total emissions [START_REF]Co2 status report[END_REF]). The solution to this problem seems to be the deployment of power plants that emit very little to no CO2, such as photovoltaic panels, wind turbines, etc 1 . The downside of technologies that rely on solar and wind is that the amount of energy produced is completely tied to variable weather conditions 2 . What once was a predictable and controllable asset, now depends on weather conditions that we are still learning how to predict with high accuracy. In the face of a massive deployment of renewable distributed energy resources, the problem of matching consumption and production at all times is becoming increasingly more difficult, requiring more advanced technologies and algorithms.

Renewable energy resources such as photovoltaic panels (PV) can be deployed in large "solar farms" by companies whose sole business is to produce energy and sell it in the wholesale market, but they can also be installed in private residential facilities (households) and small businesses. The latter is usually done to reduce the facilities' energy bill, as all the energy generated by the panel is "free" (the marginal cost of generation is 0).

As the climate crisis worsens, incentives were put in place to speed the deployment of clean 3 energy resources. These incentives can take many forms: installation of solar panels can be rewarded with a subsidy by the government, end users with renewable energy sources are paid for the surplus of energy production if they inject it into the grid, installing renewable energy sources off-premises is allowed if the conditions are not appropriate for the devices to be located on-site (such as a shadowy rooftop), etc. [START_REF] Daniel | The comparative effectiveness of residential solar incentives[END_REF].

Moreover, the purchase of a shared PV for the case in which owners could not afford the full installation on their own is becoming possible in some places.

All the scenarios described above can be seen as different incentives to favor investments on these technologies, which play a key role in the transition into a deeply decarbonized power grid. Nevertheless, can these incentives have side-effects that negatively impact the power grid? As we shall see, it is possible that if these incentives are 1 Nuclear energy emits very little CO2 in operation, but has other problems associated with it that make it an undesirable choice for a large part of the population 2 And we all know from personal experience that forecasting the weather is never as accurate as we would like it to be 3 Carbon free emissions, usually excluding nuclear 19 Figure 1.1: The famous duck curve in California. It depicts a very low net energy demand at times when solar production is at its peak followed by a very steep increase in consumption afterwards. Image obtained from [START_REF]Duck curve[END_REF]. . As an example, consider the duck curve effect that occurs in California, where the penetration of photovoltaic panels is quite large. The duck curve refers to the shape of the aggregated energy demand profile in a single day and is depicted in Figure 1.1. During the day, when most of the PV panels are active, the net load is considerably low. As the sun sinks in the horizon and the PV panels stop producing energy, the net demand skyrockets so fast that it is quite hard for generators to turn on or speed up to deal with the new values of demand. We will consider three types of solar incentives that are commonly offered to residential households without energy storage, and we will briefly discuss the effect they can have in the grid.

In the first scenario, the household has no way to measure how much energy is injected into the grid: all of its surplus energy is injected in the main grid but it is not paid for this. Under such conditions, the household will try, as much as possible (taking into account its preferences and its willingness to change their consumption habits), to consume all its solar energy when it is produced, reducing the energy that is injected into the grid. This incentive (or lack of it) contributes very little to the "duck-curve effect", because most of the energy will never enter the main grid 4 . In the second scenario, the household has a bi-directional meter and it is paid a Feed-in-Tariff (FiT) for every kWh injected. The meter records the amount of energy consumed and the total amount of energy injected at every moment: if the house uses more energy than what it generates, it draws from the grid and pays the grid price, otherwise, it injects and receives the FiT. At the end of the month, the household's energy bill will be total amount of energy consumed × price for consuming + total amount of energy injected × Feed-in-Tariff. If the price for consuming is higher than the price of injecting, then the household will also try to consume as much solar as possible, albeit with less enthusiasm than in the first scenario. We could say that from the duck curve effect perspective, the second scenario is a bit worse than the second one. For our third and final scenario the household is equipped with a net meter and is subject to a FiT. The net meter records the net amount of energy injected with the grid but unlike the other two scenarios, a kWH injected at noon can offset a kWh consumed at dusk, when consumption is peaking. In this last scenario, the household has no incentive to auto consume the energy that was generated, it can simply dump it into the grid and consume from the grid when it desires, at the exact same cost (further worsening the duck curve).

We can observe how simple differences in policy and implementation can lead to different incentives and changes in the behavior of end-customers. We cannot isolate these incentives from the change that they cause in the behavior of the participants involved, especially since their consumption directly impacts the power grid.

Regarding incentives and changes in consumer behavior, probably the most recent innovation are local energy markets (LEM). These enable prosumers (consumers playing an active role in the electricity system, by producing, storing, and/or managing electricity demand) to exchange energy with each other, for example selling their surplus to their neighbors at a price settled in a market (usually an auction) instead of selling it back to the grid at the fixed tariff (which in many cases is 0).

Local energy markets have attracted a lot of attention recently from academics and entrepreneurs alike and several test pilots are on their way across the world.

LEMs have been proposed as mechanisms that can increase the adoption of renewable resources while easing the congestion in the main grid, reducing the load on Medium-to-Low Voltage transformers (as most of the trades occur locally) and increasing the reliability of the system as a whole while decreasing the amount of capital required from the government in the form of subsidies. There are many possible implementations of LEMs, most of them add complexity to the system and therefore also have additional consequences to consider.

When dealing with residential households, the responsible entity for maintaining the quality of the power grid is the Distribution System Operator (DSO) 5 . Deploying incentives that integrate renewable resources in the distribution grid without taking into account how they will impact the DSO and its ability to operate the grid can lead to significant problems.

In the light of such complexity, we should ask ourselves: can local energy trading platforms really provide all the benefits that have been attributed to them?

There is a general agreement that local energy markets have the potential to do good, as has been shown repeatedly in numerical experiments [START_REF] Guerrero | Decentralized p2p energy trading under network constraints in a low-voltage network[END_REF][START_REF] Horta | Real-time enforcement of local energy market transactions respecting distribution grid constraints[END_REF][START_REF] Horta | Novel market approach for locally balancing renewable energy production and flexible demand[END_REF][START_REF] Moret | Loss allocation in joint transmission and distribution peer-to-peer markets[END_REF]. At the same time, policy is evolving to allow for local energy markets and communities to be deployed: the Winter Package in Europe being one of such examples. In spite of this, there are several unanswered questions regarding these trading platforms that need to be addressed before a massive roll out can take place.

The objective of this thesis is to tackle some of these unknowns surrounding what seems to be the panacea of future Smart Grids. Some of the typical research questions that we seek to answer include:

• How should we define a local trade, what should be considered as one and what should not? If there is a seller than can inject at some point in time and a buyer that is willing to buy that energy later that day, will that be considered a trade or does the transaction need to be simultaneous?

• In case that several platforms yield different properties (and they will), which objective shall take precedence? Is it maximizing the profit of all the participants or increasing the reliability of the power grid? One simple example of this is including grid constraints into the market clearing mechanisms. Doing so will increase the reliability of the system at the cost of reducing the social welfare (adding constraints cannot increase it). We can imagine that depending on who is the actor in charge of the implementation, the pursued objectives might be different and preferences might be set on some solutions over others.

• How should we transition into these markets? Are there any problems with co existing incentives such as Time-of-Use tariffs?

In this thesis we focus on the period before a full Smart Grid is implemented. During this transition, old and new mechanisms will have to co-exist and it is possible that the existence of conflicting incentives such as a LEM and a Feed-in-Tariff can cause unexpected behavior that could have negative impacts on the grid.

• Will these platforms allow for direct human participation or will they be completely automated, limiting the human interaction to setting simple preferences?

In this thesis we assume that an automated device will control and trade on behalf of a human. This guarantees that protocols can be conditioned to improve the predictability of the system. If humans are allowed to trade, how often can they change their preferences and up to what exchange?

• Shall we expect some sort of community oriented interaction in which participants will be willing to cooperate, or does this type of platforms yield only competitive environments? Most of the developments regarding local energy markets revolve around the idea of competitive markets. In spite of this, new regulation and several pilots across Europe seem to suggest that cooperation (instead of competition) will drive the energy transition at the residential level. Cooperation has many advantages, among which we can highlight consistently higher welfare outcomes. We dedicate a big part of the thesis to study models for cooperation that could replace their competitive equivalents.

Scope of the Thesis

Throughout this thesis, we study the interaction among households through local energy trading in a distribution grid, and how their behavior can change, creating impacts on the power system, controlled by the DSO, in which these trades take place. We study this problem in the context of the energy transition and because of this we assume that households have always the option to buy and sell their energy back to the main grid. In this regard, a household could perfectly be a small business or a parking garage for Electric Vehicles (EV), as long as, in addition to trading in the local market, they are subscribed to a normal electricity tariff. This scope is partly motivated by the new European regulation on local energy communities [START_REF] Hancher | The eu winter package: briefing paper[END_REF] that in several cases restricts the distance between local energy trades to participants connected to the same Low Voltage transformer [8]. The concept of local energy market studied here, is more akin to the concept of Peer-to-Peer trading [START_REF] Sousa | Peer-to-peer and community-based markets: A comprehensive review[END_REF] in the literature than to wholesale energy markets. Sometimes, the term local energy market is used to refer to a decentralized implementation of the wholesale electricity market incorporating grid constraints [START_REF] Cherrelle Eid | Market integration of local energy systems: Is local energy management compatible with european regulation for retail competition?[END_REF][START_REF] Saeed | A hierarchical electricity market structure for the smart grid paradigm[END_REF][START_REF] Ødegaard Ottesen | Prosumer bidding and scheduling in electricity markets[END_REF][START_REF] Martin Zepter | Prosumer integration in wholesale electricity markets: Synergies of peer-to-peer trade and residential storage[END_REF]. That is not the object of study of this thesis. Instead, a better categorization would be to say that it is about indirect demand response programs, i.e., a demand response program without active participation of a third party such as a DSO or a TSO [START_REF] Heussen | Indirect control for demand side management -a conceptual introduction[END_REF]. An example of such a static program would be a Time-of-Use tariff. In contrast to this, an example of a direct demand response program would be Critical Peak pricing [START_REF] Kii | The effects of critical peak pricing for electricity demand management on home-based trip generation[END_REF][START_REF] Wang | Critical peak pricing tariff design for mass consumers in great britain[END_REF], in which the utility has the right to call on a critical event a few times each year for which the electricity price will be much higher (observe that there is an active action of the utility in this case).

An important family of direct demand response programs is Local Flexibility Markets often reefed to also as local energy markets. In a Local Flexibility Market, a buyer can pay customers to change their consumption profile with respect to a baseline [START_REF] Correa-Florez | Optimal participation of residential aggregators in energy and local flexibility markets[END_REF][START_REF] Adrian | Optimization of prosumers flexibility in electricity markets[END_REF][START_REF] Ziras | Why baselines are not suited for local flexibility markets[END_REF]. For the case of a DSO, this could be motivated by the need to reduce consumption on a particular day on a particular region of the grid that is prone to congestion. An example of such flexibility market is the Cornwall project by Centrica6 .

The family of indirect demand response programs referred here as LET, rely on a variant of virtual net metering. Unlike conventional net metering (described in the example at the beginning of this chapter), virtual net metering allows to join several customers and locations behind a "virtual" meter [START_REF] Barnes | Aggregate net metering: Opportunities for local governments[END_REF][START_REF] Mckenna | Photovoltaic metering configurations, feed-in tariffs and the variable effective electricity prices that result[END_REF]. With this accounting method, trades among the participants are "invisible" to the external utility and the consumers are free to decide how to price such exchanges. We will consider a variation of Virtual Net Metering in which the net metering takes place dynamically if certain conditions are met. That is, for some prescribed situations, players will behave as behind a virtual meter (to allow for local trades), but for the rest they will be charged individually, as you and me normally do.

The idea of controlling demand flexibility through prices and in particular through Time-of-Use tariffs has been challenged on many occasions [START_REF] Lo | Electricity rates for the zero marginal cost grid[END_REF][START_REF] Mathias | State space collapse in resource allocation for demand dispatch[END_REF]. This is especially true when the commodity traded is energy, but the utility function of consumers is not a direct function of it. The main part of the thesis deals with customers reacting to energy prices to decrease their costs while improving the net demand profile. This should not be taken as a reflection that we believe real time prices for consumers work. If anything, it is our attempt to translate the results known for wholesale energy markets to local energy markets. Furthermore, the second part of the thesis, in which we use cooperative games already assumed that prices are a mechanisms for rewarding engagement but not a direct tool to control energy consumption7 .

Methodology

As we mentioned at the end of the previous section, the idea of LEMs implies competition, and I am not entirely convinced that this is indeed the best solution out there. To study local energy markets, we consider a broader framework that we refer to as Local Energy Trading (LET) and that encompasses competitive and cooperative models alike. Simply put, the objective of the thesis is to understand and improve the interaction of agents whose interests are tied together by the incentives that result from a Local Energy Trading platform and shared investments.

To do this, we define mathematical models that emulate the change in consumption of agents (households) when presented with the monetary incentives created by these trades. We mostly deal with prosumers with batteries as they provide a powerful but tractable formalism that is realistic. Then, we formalize the interaction of agents using tools from game theory, both as cooperative and non-cooperative games.

Since the non-cooperative games studied are quite complex, we do not try to find nor prove the existence of equilibrium points. Instead, we look for interesting and usually problematic behavior in small examples, which we later validate with large scale simulation of the system. Furthermore, one might argue that even if such equilibrium points were to exist and are well behaved, if the learning procedure used by agents to arrive to such point causes major problems in the power grid, the outcome should still be labeled as a negative one. On the other hand, we are able to prove the existence of stable solution for all the cooperative games studied and for most of them we provide efficient algorithms to find such solutions.

Contributions of the Thesis

The manuscript is organized in four parts and two special chapters as part of the appendix. The order of parts follows a logical path that traverses the most important ideas behind the thesis: traditional sequential local energy markets have many shortcomings, and from the many candidates to substitute them, energy communities seem to be the best for the role.

The organization of the rest of the thesis is as follows:

Part I: Local Energy Trading

In the first part of the thesis we will formalize the main object of study: a local energy trade. In chapter 2 and 3 we will preset an abstract model of consumers and derive how their behavior, once independent from each other, evolves and yields a game in a game-theoretical sense with the introduction of local trades.

In chapter 4 we will introduce a model for a prosumer with a battery that maximizes her utility by reducing her energy bill. This model will be used in almost all of the remaining chapters and serves as a backbone of the thesis.

Part II: Sequential local energy markets

The second part of the thesis focuses on sequential local energy markets. First, we will show how to adapt the general framework introduced in chapter 2 to this particular case. Different market mechanisms will be introduced and their different properties will be discussed. In particular, we will discuss the role of strategy-proofness in the market mechanism and how it changes the way in which players reason about the markets.

This part is also composed of two chapters. In chapter 5 we introduce the main notions of LEMs, while in the second one (chapter 6) we will present a case study in which some of the negative consequences of local energy markets become evident. This will serve as motivation for the next part.

One interesting question that arises is whether the gains obtained by trading in local energy markets are sufficient to motivate the investment into photovoltaic panels by customers that don't already have one. During our analysis of local energy markets we will ignore the question, but we will look at it again when dealing with cooperative games. There, we will study the investment in photovoltaic panels and batteries and assess when is it profitable for a set of players to jointly invest in new technology.

Part III: Combinatorial local energy markets

Part III describes a new model that exploits the flexibility available to players (in the form of storage) and describe how it can be used to design a new market mechanism. In the first chapter of this part, chapter 7, we introduce a first and simple model of a combinatorial auction that clears day-ahead to replace the sequential model studied before. In the last chapter of the part, chapter 8, we present a modified auction model that exploits domain knowledge and yields a more computationally efficient mechanism.

Part IV: Energy communities

Part IV deals mostly with the development of energy communities, the cooperative alternative to energy markets. This is probably the central part of the thesis since most of our new results deal with these cooperative games. The chapters within this part follow the chronological order of their development.

Chapter 9 serves as a general introduction to cooperative game theory. Our first model (chapter 10), studies the question of how consumers without energy storage can cooperate and invest together in the technology. This was inspired by our assumption that residential households have batteries which they can use to enable a flexible energy consumption. Chapter 11 presents an extension of the previous chapter, in which the mathematical model is developed in more detail, allowing us to incorporate into the investment model discreteness of the battery sizes and stochastic load profiles. Chapter 12 takes a different direction. There, we study how to implement a centralized (or distributed) control of a fleet of batteries (assuming that players already have them) and how to design the incentives such that players actively decide to cooperate. Finally, in the last chapter of part IV, we unify both models: the investing in storage and the cooperative control of a fleet of batteries to show that we can model both tasks together. We prove the existence of stable solutions also for this case.

Special chapters in the Appendix

Two projects were developed throughout the thesis that are slightly out of the context introduced in Part I, but have the potential to be of real importance in the future. We mention them separately here, possibly looking at future areas of research. The first project, presented in Appendix A, studies how to exploit the flexibility in natural process such as plant growth (possibly in vertical farms) while minimizing the electricity costs. The second project offers a quick view on how to design a benchmark for demand response applications and the uncovers some of the limitations of current approaches. This was identified throughout the thesis as a major gap in the literature to be addressed, an issue that will be further discussed in the Conclusion of the thesis.

Part I

Local Energy Trading

Chapter2

An Introduction to Local Energy Trading

General Trends in Local Energy Trading

In our use of local energy trading, we encompass all applications in which energy generated at the local level (low voltage distribution grid) is traded virtually to another customer located "close enough" to the customer that generated the energy. Since the energy is not physically exchanged but only virtually exchanged through proper accounting, many implementations of local energy trades can exist.

We can distinguish between active and passive trades. In a passive trade, a rule exists that allocates some production to the consumers, without them having to actively interact with the system. An example of this is solar energy sharing schemes. On the other hand, in an active trade, customers have to take action in order to buy or sell energy at any given time. An example of the later would be an auction.

Apart from the classification into active and passive trading schemes, we could further differentiate them between cooperative and competitive. In the former, participants are willing to sign deals that improve their position. In the later, customers are assumed not to communicate with other participants and only maximize their own benefit (utility).

We proceed to highlight the relevant work in the area of local energy trading and in particular, efforts to find a unifying framework to study it.

Related Work

Local energy trading has seen an increase in popularity in recent years and has become one of the most important ways for incentivizing customers to become prosumers and to establish mechanisms for managing their distributed energy resources. Most of these developments can be broadly characterized into three trends, P2P exchanges, local energy markets and energy communities.

One of our main goals is to provide a general framework to study all types of local energy trading mechanisms. A similar goal focusing on policy rather than incentives was recently pursued in Moura and Centeno [START_REF] Moura | Prosumer aggregation policies, country experience and business models[END_REF]. There, they describe a new perspective to analyse several emerging concepts such as: virtual net metering, energy trading, shared generation, etc. They show several examples of policies for such schemes and argue that regulation should treat them similarly.

Numerous developments have occurred in the area of local trading. Mengelkamp et al. [START_REF] Mengelkamp | Tracing local energy markets: A literature review[END_REF] created a comprehensive review of local energy markets in recent years. In it, they identify several research gaps among which we can identify: properly defining LEMs and developing methodological comparisons among different mechanisms. The methodology presented in this chapter can be seen as a major step towards filling those gaps.

When local trading is implemented through markets, there are a wide range of mechanisms that can be used. One sided auctions [START_REF] Ygge | Power load management as a computational market[END_REF], traditional double auctions, in both, periodic and continuous variants [START_REF] Ramachandran | An intelligent auction scheme for smart grid market using a hybrid immune algorithm[END_REF], or even custom market designs, have all been proposed to implement such markets [START_REF] J Koen Kok | Powermatcher: multiagent control in the electricity infrastructure[END_REF], [START_REF] Penya | Combinatorial markets for efficient energy management[END_REF], [START_REF] Kiedanski | Design of a combinatorial double auction for local energy markets[END_REF]. Tightly coupled with the research in markets for local trades, is the study of trading strategies, commonly involving reinforcement learning [START_REF] Mengelkamp | Trading on local energy markets: A comparison of market designs and bidding strategies[END_REF], [START_REF] Karaca | No-regret learning from partially observed data in repeated auctions[END_REF]. We will study some of these trading strategies in chapter 6 in which we show some of the problems that might arise.

The recent spike in popularity of blockchains was accompanied with a big development of P2P exchanges as implementations of local energy markets. A recent survey by Sousa et al. [START_REF] Sousa | Peer-to-peer and community-based markets: A comprehensive review[END_REF] described recent advances in the area.

The local trading mechanisms presented so far fall into the category of competitive mechanisms and are usually studied independently of their cooperative counterparts. Nevertheless, the comparison is not futile and has been 29 CHAPTER 2. AN INTRODUCTION TO LOCAL ENERGY TRADING considered recently in other application areas. In Bielik et al. [START_REF] Bielik | Cooperative versus non-cooperative game theoretical techniques for energy aware task scheduling[END_REF], the authors study the advantages of cooperation (in contrast to a competitive scenario) for a set of machines that need to schedule tasks in an energy efficient way. The work of Khan [START_REF] Khan | Non-cooperative, semi-cooperative, and cooperative games-based grid resource allocation[END_REF] also studies cooperative versus non cooperative games for a task scheduling problem and they show that the cooperative approach achieves higher social welfare than any non cooperative alternative.

We recognize several cooperative approaches as implementations of local trading. For example, in [START_REF] Rutovitz | Building a level playing field for local energy: Local network charges and local electricity trading explained[END_REF], the authors describe several local energy trading scenarios and configurations. Among them, a co owned generator that distributes its surplus to all of its shareholders. Our framework can also incorporate programs such as the later. More direct applications include the investment and co-ownership/co-operation of energy storage as described extensively in [START_REF] Kiedanski | The effect of ramp constraints on coalitional storage games[END_REF], [START_REF] Kiedanski | Discrete and stochastic coalitional storage games[END_REF], [START_REF] Kiedanski | Efficient distributed solutions for sharing energy resources at local level: a cooperative game approach[END_REF].

Finally, the policy regarding virtual net metering and its uses has been properly documented [START_REF] Rutovitz | Local network credits and local electricity trading: Results of virtual trials and the policy implications[END_REF], [START_REF] Mckenna | Photovoltaic metering configurations, feed-in tariffs and the variable effective electricity prices that result[END_REF], [START_REF] Barnes | Aggregate net metering: Opportunities for local governments[END_REF].

The Proposed System

We consider a system composed of end customers, mostly households, connected in the same Low Voltage grid and possibly behind the same feeder. The system is depicted in Figure 2.1 These consumers are individually subject to an electricity tariff with their Traditional Electricity Company (TEC). They have flexibility in that they can change their net energy consumption and they will do so to minimize their costs. This is shown in the red section of the figure (positive sign denotes consumption of energy). As we see in the figure, energy can be bought from the grid or produced locally (negative signs represent generation). If there is surplus (only resonable if there is local generation), then it can be sold back to the grid or to the customer's neighbours. We call this local energy trading. We allow actions to be taken in a continuous or time-slotted fashion.

Grid / TEC

Appliances

Chapter3

A General Mathematical Model for Local Energy Trading

Model for Customers

Following conventions used in game theory, we will use the term player to refer to customer or participant in the local trading scheme. Furthermore, the superscript i (or j) will denote a single player and -i the subset of all other players except i.

Let N = {1, . . . , N } be the set of players in the local community and let T = [t 0 , t f ] ⊂ R be the time interval in consideration. We usually deal with time intervals of 1 day. Furthermore, when dealing with discrete time-steps, we usually assume time-slots of 15 minutes up to 1 hour each. This mimics the duration of blocks in wholesale energy markets. Player i flexibility will be encoded in her action set X i (the set of actions of all players will be denoted

X = N i=1 X i ).
Each element x i ∈ X i encodes a whole sequence of actions that span the full time interval. For each action x i ∈ X i of player i, there is an associated net consumption profile Z i : X i → L that represents the amount of energy consumed as seen from the grid at every instant, where L denotes the family of mappings from T to R. Whenever Z i (x i , t) is positive, it represents that player i consumed more energy that what she generated at time-slot t ∈ T when taking action x i , while negative values represent surplus of generation. Furthermore, each player i has an utility associated with each different level of consumption and how much she needs to pay for it:

u i : L × R → R.
In turn, each consumption level Z i (x i ) has an associated cost (or profit if surplus is large enough). Traditionally, we would have a price function P i : L → R, that maps individual consumption profiles to prices. Assumption 3.1. Players are rational and utility maximizers.

It is well known that humans are not rational [START_REF] Upadhyay | Thinking fast and slow[END_REF], [START_REF] Scott | Econs vs. humans: An introduction to behavioral economics[END_REF]. Hence the distinction between Humans and Econs. In spite of this, when decision making is relegated to a computer (which might be the irrational action), we can assume rationality with regard to the computer program that is being implemented and the utility function that has been coded. In this regard, even if it would be irrational for a consumer to only value its monetary profit and transmit those values into its Home Energy Management System (HEMS), once that is done, the HEMS is, almost by definition, an Econ.

Coupling Assumption 3.1 with the model presented so far, we derive the optimal action of player i as the solution of the optimization problem (3.1).

x i * ∈ arg min

x i u i (Z(x i ), P i (Z(x i ))) subject to: x i ∈ X i (3.1)
In this model, the optimal action of every players does not depend on the action of the other players.

Model for Aggregated Utility Functions

Having introduced the setting for a single player, we proceed to lay the foundations for the framework presented in the chapter. As explained in the introduction, one of the main ideas of local energy trading is to provide participants the incentives to self-organize and increase local consumption of renewable energy surplus. We argue that variations of virtual net metering can provide the necessary and sufficient environment for such exchanges. Formally, we can do so by defining a Local Pricing Profile. Definition 3.1. A local pricing profile P L :

P L : N i=1 L i → R.
is a function that maps the consumption of all players to a price. Remark 3.1. A local pricing profile is a generalization of the traditional individual pricing. Indeed, taking

P L = N i=1 P i .
we recover the previous pricing rule. Definition 3.2. A distribution of the costs (DoC) is a vector y ∈ R N such that:

i∈N y i (x) ≥ P L (Z(x)), ∀x ∈ X .
Observe that the DoC is not unique, and different mechanisms can result in different costs for each player. The greater in the inequality occurs when there are external costs to pure energy consumption. An example of this would be an auction that is weakly-budget balanced, in which the market maker gets a cut of all the trades. An important realization under the local pricing profile and a DoC that depends on the actions of other players is that the optimal decision of a player becomes the solution of a new optimization problem (3.2).

x i * ∈ arg min

x i u i (Z(x i ), y i (Z(x i ), Z -i (x -i ))) subject to: x i ∈ X i (3.2)
The optimal decision of a player no longer depends only of herself, but also on the other players. This gives place to a game (in the game theoretical sense of the word) among the different participants.

Depending on how the local pricing policy and the distribution of the costs are implemented, the outcome of the game will change and so will the benefits (or consequences) associated with it.

If the implemented mechanism results in a cooperative game, a few modifications are added to the optimization problem (3.2) to guarantee cooperation. For that case, for each subgroup of players S, an optimization problem obtained by combining all the individual optimization problems of the participants in S is solved and each participant finds her value in joining S (as assigned by a DoC). Then, players join the coalition that maximizes their value (if possible).

In the next section, we will show how to model local energy trades from local pricing profiles. We will proceed to formalize one rule in particular that plays a vital role in the development of local energy trades. Definition 3.3. A consumption profile R i l is said to be a partial consumption with respect to a consumption profile l i ∈ L i if:

• R i l i × l i (t) ≥ 0, ∀t ∈ T • |R i l i | ≤ |l i (t)|, ∀t ∈ T
In other words, R i l i is always between l i and the x-axis. We will denote R(l i ) the set of all partial consumptions with respect to l i .

P L = N j=i P i l i -R i l i + Q i R i l i (3.3)
defines a local matching pricing rule, where Q is a price that applies only to the energy locally matched a .

a For example, this could be a payment for the use of the grid. This are sometimes referred as wheel charges.

The intuition behind the above rule is as follows. Some fraction of the players' consumption (R i L ) can be "hidden" behind a virtual net meter. Only "matched energy" can be aggregated. Namely, the same amount of consumption and surplus at the same time. By doing so, players can decide on the price of the "matched" exchanges.

Observe that the energy behind the virtual net meter might still be charged for a different service such as the usage of the distribution grid (Q i , i ∈ N ). Furthermore, it is not required that the partial consumption profiles are maximal, in the sense that they match as much energy as possible. As we shall show later, some mechanisms might decide to match less energy intentionally. Auctions are an example of this.

Example 3.1. Figure 3.2 depicts a solar farm and two players. For each demand profile, the blocks in green represent one possible partial consumption profile. It can be observed that for each timeslot, the sum of the partial consumption profiles is exactly 0, so they define a local matching pricing rule. The two blocks in red (one for the solar farm) and one for households number could have been considered for the partial matching, but were not. This is sometimes the case, specially when the matching is decided using an auction. The last curve of the picture, below the dashed line, shows the different blocks of energy that will have to be transacted with the Traditional Energy Company. The blue blocks are to be paid at the normal price for buying and the yellow blocks are going to be remunerated according to a Feed-in-Tariff. The money transfers associated with the energy that was locally traded are to be settled by the three participants by themselves. players will prefer to auto consume locally than trading with the TEC.

Remark 3.2. As long as

N i=1 P i (l i ) -P i (l i -R i l i ) ≥ N i=1 Q i (R i l i ),
One of the differences between a local matching pricing rule and a pure virtual net metering scheme is that while the later can provide consumers with virtual credits, the former does not provide incentives to use the main grid as a backup battery. The metering system proposed by us only works when the energy is matched at the same time. As a consequence, players have incentives to change their behavior and auto-consume (or collectively auto consume).

If the selling price (Feed-in-Tariff) is lower than the buying price (normal tariff), there is money flowing outside the community whenever some participants consume and others inject at the same time. The new pricing allows them to retain that excess capital and share it in any desirable way among the participants.

Local Matching as Mechanism Design

Local matching pricing rules as described so far do not provide information on how to choose the partial consumption profile of players nor how to distribute the total costs. As we shall show, these two missing pieces define the different mechanisms. As an example, in a P2P exchange, the partial auto consumption profiles will be determined by the trades among peers and the distribution of the costs will also be determined by the price used in each trade. Naturally, there are mechanisms that make "better use" of the local matching rule than others. This is a central question in this thesis: what mechanisms are better suited to exploit such rule?

We conclude this chapter with a few spoilers.

A Local Matching Pricing Rule Implementing: P2P Exchange

In a P2P exchange, players can agree to exchange energy if the price offered by the seller is lower than what the buyer is willing to pay. When that happens, an amount equal to the energy transacted gets hidden behind the dynamic virtual net meter. Only these trades are accounted for, even if more energy could have been matched. The distribution of the cost also comes with the trades, as each player will pay what she transacted in the market.

A Local Matching Pricing Rule Implementing: A Cooperative Game

In a cooperative scheme, players will try to maximize the amount of locally matched energy, as that results in an increased social welfare for all the participants. The central task in the cooperative framework is to find a way of dividing the savings among all participants such that they remain cooperative: it is not hard to imagine that even a player that wishes to cooperate, will soon stop doing so if she has to pay for the whole costs of the cooperative while other participants pay nothing! In chapter 9 we will introduce several concepts of "stability" in such distribution of costs among cooperating agents.

Chapter4

Model of Prosumer's Flexibility Using Energy Storage

Main Assumptions

In the previous chapter, we introduced the notion of a local matching pricing rule and we explained how the rule can give place to several mechanisms. To compare such mechanisms, we will make use of a specific model of a consumer with flexibility.

In this chapter we will describe such model, which will be used throughout the whole thesis. For this model, we consider Assumptions 4.1, 4.2, 4.3. Assumption 4.1. Fixed comfort We assume that players are not willing to change the comfort they obtain from using appliances that run on electricity. For most appliances, energy consumption and the comfort obtained by using them is loosely correlated. As a consequence, there is usually a feasible set of power set points that result in the same comfort for a player. We will refer to those set points as the feasible action set of the player. For simplicity, during these thesis, we will assume that the only deferrable appliance is a battery, whose feasible action set can be described using linear constraints.

Assumption 4.2. Quasilinear utilities

Players have quasilinear utilities of the form: u(l, y) = v(l) -y, where y is the amount of money paid (or received) for the consumption profile l. Given that players are not willing to shift their consumption, their utility function for a net consumption profile l is given by v(l) = 0 if l satisfies their base load and v(l) = -∞ if it does not.

The idea behind this is quite simple, if players do not want to change their consumption, then they only should care about how much money they are paying for it.

Assumption 4.3. Risk neutral

Players considered in this thesis are risk neutral utility maximizers.

There is a vast literature in modeling consumer's energy flexibility, most of it developed with the intent of designing and improving demand response programs.

One approach consists on considering demand elasticity with respect to prices as it has been done in [START_REF] Kirschen | Factoring the elasticity of demand in electricity prices[END_REF], [13]. This can be further classified into self-elasticity -how consumption in one time-slot changes with respect to prices, and cross-elasticity -how consumption shifts from one time-slot to another one. A different and widely used approach is using an utility function to measure the comfort or discomfort associated with a deviating from a default consumption profile [START_REF] Willett | Residential demand for energy commodities: A household production function approach[END_REF], [START_REF] Yang | A game-theoretic approach for optimal time-of-use electricity pricing[END_REF]. Similarly, a very fine grained control based on the utility obtained has been proposed [START_REF] Kaddah | Demand response solutions Based on connected appliances[END_REF].

All these approaches can work very well but require assumptions on the "satisfaction" of consumers with respect to their energy consumption. These assumptions are difficult to measure and often involve a socio-economical 37 dimension that is often neglected (the cross-elasticity of a kWh at the same price difference that will be used to heat a pool in Malibú might be different than the same cross-elasticity for the same kWh for the same price difference but used to power a small fan in the hottest day somewhere in South America). To avoid introducing those types of biases into our models, we opted to ignore all "voluntary" changes in consumption as a response to change in electricity prices. Instead, we assume the existence of appliances that enable their owners to see their comfort and their net power consumption as completely uncorrelated1 . This is the case of energy storage, for example, where its usage has an almost 0 impact on the comfort of its owner as long as the monthly electricity bill remains the same. Furthermore, since this thesis is situated during the energy transition, it is reasonable to assume that batteries will be available, given the efforts that are being allocated for their development. As an example, many electric car's batteries will be close to reaching their end of life but could still be re purposed as domestic "load shifters".

Mathematical Model

In this subsection we will introduce a practical model of a prosumer with a battery.

Without loss of generality, we shall assume a slotted time frame T = {1, . . . , T }, where each time-slot lasts usually 15 or 30 minutes. Each player i has a fixed demand profile l i that can be observed every time-slot l i = (l i 1 , . . . , l i T ), where l i t is the demand of player i at time-slot t. The later represents a consumption of energy when its positive and a surplus of renewable generation when it is negative.

The player can own a battery. If she does, its capacity will be denoted S i (S i = 0 if it owns no battery), and its initial state of charge by S i 0 . The maximum energy that the battery can provide in a given time-slot (ramp constraint) will be denoted δ i and the maximum energy that the battery can storage in a single time-slot δ i . The actions of the player are limited to the operation of the batter, i.e., in each time-slot, the player can only decide how much to charge or discharge the battery. We will denote by x i t the real amount of energy that enters (positive value of x i t ) or leaves the battery (negative values of x i t ) and x i = (x i 1 , . . . , x i T ). The battery has efficiency losses for charging (η c ∈ (0, 1]) and for discharging (η d ∈ (0, 1]).

The feasible set of actions of player i is defined by Equation (4.1).

X i = x i : 0 ≤ S i 0 + k t=1 x i t ≤ S, ∀k ∈ T , x i t ∈ [-δ i , δ i ] (4.1)
Remark 4.1. The set X i as defined in Equation (4.1) is non-empty, closed and convex.

Sometimes, we will have to refer to the specific action implemented during time-slot j and we will use the notation x i j for that. Furthermore, when we will need to denote the feasible set of battery set points at time-slot j we will use the notation X i (S i j ) where

S i j = S i 0 + j t=1 x i t .
In other words, the feasible values of x i j depend on the state of charge of the battery at that point. In this context, the operator Z that maps actions to net consumption profiles is

Z i (x i , t) = l i t + max{x i t , 0} η c -max{-x i t , 0}η d . with x i ∈ X i , i ∈ N .
Recalling that we defined the utility of player i as 0 if her required load profile was satisfied or -∞ if it was not minus payments, and because the set of net consumption profiles Z i always satisfies player's i load, we obtain that player's i utility is purely given by the amount of money she needs to pay. Namely, she maximizes her utility when she minimizes how much she is paying for the same energy consumption.

There is a contract between player i and a Traditional Electricity Company (TEC) that specifies a price for buying electricity β i t at time-slot t and a price for selling electricity γ i t at time-slot t. With this convention the utility function of player i is given by:

u i (Z i (x i ), β i , γ i ) = T t=1 β i t max{z i t , 0} -γ i t max{-z i t , 0} (4.2)
As in most contracts of this kind, we will assume that player i is allowed to buy or sell as much energy as she desires. With the above in place, the decision model of player i can be summarized by optimization problem (4.3).

p i = min x i T t=1 β i t max{z i t , 0} -γ i t max{-z i t , 0}
subject to:

x i ∈ X i Z i (x i , t) = z i t = l i t + max{x i t , 0} η c -max{-x i t , 0}η d , ∀t ∈ T (4.3)
Optimization problem (4.3) is very useful, as it allows us to define a simple baseline for each player (p i ). We know that player i can always guarantee p i for herself by acting alone. Assumption 4.4. In every time-slot t ∈ T and for every player i ∈ N it holds that

β i t ≥ γ i t .
Assumption 4.4 is quite natural since most residential energy tariffs satisfy it2 (feed-in-tariff is lower than the buying price). If the assumption holds, then the cost function in (4.3) is convex. Proof. For the proof, we will use a variation of the optimization problem (4.3) in which we will replace x i t with x i,+ t , x i,- t such that x i t = x i,+ t -x i,- t with x i,+ t , x i,- t ≥ 0. By doing so, we can write max{x i t , 0} = x i,+ t and max{-x i t , 0} = x i,- t . We proceed in the same fashion for z i t by introducing z i,+ t and z i,- t . The resulting problem is linear and it remains to show that the solution is exactly the same one as in the original problem. This is equivalent to show that in any optimal solution it holds that z i,+ t z i,- t = 0 and x i,+ t x i,- t = 0. We proceed by contradiction. Suppose that there is a time-slot in which for the optimal solution it holds that z i,+ t z i,- t = 0 and z i,+ t -z i,- t = K > 0. We can substitute in the objective function and obtain that the

cost at time-slot t is c t = β i t K + z i,- t ≥0 β i t -γ i t ,
which is minimized by taking z i,- t = 0, hence a contradiction. The same is analogous for K < 0.

We proceed to look at the charging and discharging variables. Suppose that there is a time-slot in which for the optimal solution it holds that x i,+ t x i,- t = 0 and

x i,+ t -x i,- t = K > 0. Then, z i t = l i t + Kη d + x i,+ ≥0 ( 1 η c -η d ).
Because the cost at time-slot t is increasing in z i t , the optimal value of x i t is 0. The proof is analogous for K > 0. This concludes the proof. Proposition 4.1. If for every pair of time-slots t 2 > t 1 , t 1 , t 2 , ∈ T , it holds that β i t1 ≥ η c η d γ i t2 , then buying form the TEC to re-sell later to the TEC is not profitable.

Proof. The proof can be found in [START_REF] Umar | Optimization and Control of Storage in Smart Grids[END_REF][START_REF] Md | Energy storage applications for low voltage consumers in uruguay[END_REF].

Numerical Experiments using the Proposed Model

The model described so far assumes that the system is fully observable: future energy demand and energy prices are known. In this section, we will present how we assumed that players dealt with their lack of information about the future.

In this regard two things are done throughout the thesis: we forecast the consumption and production profiles (TEC prices are known and fixed) and we use a model predictive control approach to take decisions as time goes by. We studied the effect of forecast errors for this particular problem using different forecasting techniques in [START_REF] Kiedanski | Sensitivity to forecast errors in energy storage arbitrage for residential consumers[END_REF]. Here, we summarize the main results, which will be used in the rest of the thesis.

Load Forecasting

Since prices are known beforehand, we only need to forecast the load. Forecasting the load of individual households is in general a difficult problem and there are many approaches and papers describing a wide range of techniques. Since the goal of the thesis is to understand local energy trading and not implementing a perfect battery control3 , we are satisfied with a simple forecast that performs fairly well.

As our default forecasting technique we will use what we call the AveragePast forecast, which is quite intuitive. Let D denote the amount of time-slots in a day, if D = 48 then every time-slot lasts 30 minutes and if D = 96, time-slots are 15 minutes long instead. Using the AveragePast forecast, we will forecast the load l i t of player i ∈ N at time-slot t ∈ T with:

li t = l i t-7D + l i t-2×7D + • • • + l i t-7BD B .
where B, the number of data points involved depends on the available data and we use the notation • to represent the forecast of a variable. The interpretation is quite simple, the demand profile of player is forecast using the average of the past time-slots the same day of the week at the same time of the day. We will provide empirical evidence of the performance of this forecast at the end of this chapter.

Because all the time-slots considered in this thesis are short enough we will assume that the load in the current time-slot is always known.

Model Predictive Control

In most of the models discussed in the rest of the thesis, agents can and take their actions in a sequential fashion. Therefore, we consider two models for such decision making. The first one is a receding horizon Model Predictive Control, described in Algorithm 1. There, the player uses her real end-of-horizon in each subproblem solved, reducing the size of the problem in each iteration. On the other model, a fictitious time-horizon is used and a problem of the same size is solved in each iteration. The later is described in Algorithm 2.

In both algorithms, we use the notation A c:d to indicate that we are indexing the variable A in the interval

[c, d] ∩ N Algorithm 1 Receding Model Predictive Control Input: l i , β i , γ i , δ i , δ i , S i , S i 0 , η i c , η i d Output: x i 1: SoC ← S i 0 2: for t = 1 to T do 3: li k ← AverageP ast(l i ), ∀k = t + 1, . . . , T 4:
x i t:T ← SolveOpt(l i t , li t+1:T , β i t:T , γ i t:T , SoC)

Results and Discussion

We conclude this chapter with a simple numerical evaluation of the two algorithms described in the section above, using the presented forecast for a large number of real consumption profiles obtained from the Ausgrid Dataset [START_REF] Ratnam | Residential load and rooftop PV generation: an australian distribution network dataset[END_REF].

The Ausgrid Dataset contains the consumption profile (including some solar generation) for 128 consumers during the years 2012 and 2013. Figure 4.1 shows the amount of energy consumed by these players. The demand profile of these players already contains the amount of energy generated by them: positive values denote energy consumption while negative values denote surplus of generation.

In the simulations, each consumer was subscribed to a Time-of-Use electricity tariff similar to the French Heures Pleines/Creuses. During 23:00 to 07:00 the price for buying was 12.3 cents per kWh, and during the rest of the day the price was 15.8 cents per kWh. Regarding the Feed-in-Tariff, we also used the French Tarif d'Achat at a price of 10 cents per kWh.

For each consumer in the dataset and for 5 different starting dates, we simulated six consecutive days of battery usage using the two proposed algorithms and the proposed forecast.

The batteries in this experiment are modeled after a Tesla's Powerwall 2: they have a capacity of 13 kWh, a round trip efficiency of 0.9 and maximum ramp rate of 5 kW for charging and discharging.

For the rolling horizon algorithm, we further compared two cases: selling all the energy remaining in the battery at the end of the simulation at the Feed-in-Tariff price or keeping it.

In Figure 4.2, the three box plots depict the percentage of change in the total cost incurred by players when using the forecast versus using the real load. Positive values represent an increase of the costs by using a forecast whereas negative values represent a decrease in costs.

Observe that when using a receding horizon, the forecast always performs worst than when using the real load, but this does not need to happen when using the rolling horizon. The later is mostly because the value of the energy stored in the battery can vary and a prosumer storing energy for later consumption might end up having higher costs than the forecast equivalent at a given point.

Overall, we observe that the mean error is less than 5% in all cases and the errors hardly surpass the 10%, which for our purposes (evaluating the behavior of agents in local energy trading scenarios) is reasonable enough.

Throughout the rest of the thesis, we shall use this battery model, with either one of the algorithms and the Average Past forecast.

Key Outcomes and Conclusions

In this chapter we introduced the model of a prosumer with a battery. This model will provide one of the most important building blocks throughout the whole thesis. We showed how to model the problem as a simple linear programming problem and how to simulate such model in the presence of uncertainty and forecast. Furthermore, we provided numerical evidence supporting the use of the two MPC-like algorithms and a simple forecast.

In the next chapters, we will delve into the actual energy trading models, always assuming that the agents participating in them follow the model described in this section (unless stated otherwise).

Receding Horizon

Rolling Horizon (Keeping) -10.0% -5.0% 0.0% 5.0% 10.0% 15.0% 20.0%

Rolling Horizon (Selling)

.2: Percentage of error between the optimal value using real data and the forecast for the algorithms described above.

Chapter5

Introduction to Local Energy Markets

In this chapter, the first of the two dedicated to sequential local energy markets, we will introduce the reader to the concept as it is considered in this thesis. Chronologically, local energy markets were the starting point of this thesis and are strongly inspired by the work of [START_REF] Luis | Innovative paradigms and architecture for future distribution electricity networks supporting the energy transition[END_REF]. They served as the entry point to our study of the more general local energy trading paradigm and therefore, are a cornerstone to the analysis and motivations behind this document.

In the literature of local energy markets, the term has been used to refer to different concepts: from one-shot auctions to hierarchical systems that compose the wholesale energy market ecosystem. The goal of this chapter is to develop the models to study one of such interpretations of local energy markets in which sequential auctions are used for intra day energy trading among residential households. One of the main features of our approach is that we model the intra temporal correlations of energy demand associated with the flexibility obtained by using a battery, as opposed to assuming independent loads in each time-slot.

Overview of Auction Theory and Double Sided Auctions

In the traditional auction model, there is a seller with an object for sell and a group of interested buyers [START_REF] Klemperer | Auction theory: A guide to the literature[END_REF]. For this simple case, several implementations satisfy different properties. In the first-price sealed-bid auction, players secretly bid the price at which they are willing to buy the object and the player whose bid was the highest gets it, paying what he bided. It is not difficult to see that a player might benefit from lying. Indeed, if the player with the highest value for the object bids the second highest value plus a small margin, she would get the object while paying less. On the other hand, by changing a small detail in the market mechanism, the result can be quite different. If instead of she (the winner) paying the exact amount of what she bided, she pays the second highest price, then players have no incentive to lie: the best strategy is to bid the maximum price a player is willing to pay. This simplifies the problem for an agent that no longer needs to understand at which price she should bid, but only needs to find out how much the object is worth for her. So far we have described a traditional auction in which the supply is fixed, "one of something (a hat, a house, a book)" and the demand varies. For the case of local energy markets, we would like to have several buyers and also several sellers. When an auction allows for buy and sell bids, it is often called double auction. Furthermore, unlike the previous example, we want to allow participants to buy and sell any fraction of energy, not only integer quantities. Auctions that allow to trade continuous quantities of a good are called auctions for divisible goods.

Having said the above, we are interested in studying double auctions for divisible goods as mechanisms to implement local energy markets. This family of auctions has been studied considerably less than the one-sided counterparts.

Example A: A Pseudo Strategy-proof Auction Mechanism

Huang et al. [START_REF] Huang | Design of a multi-unit double auction e-market[END_REF] describe the design of a double auction for divisible goods and Horta et al. [START_REF] Horta | Novel market approach for locally balancing renewable energy production and flexible demand[END_REF] used it as their market mechanism to implement LEMs. Figure 5.1 shows an instance of the Huang market.

To participate in the auction, agents can submit a bid for buying or for selling. In both cases, a bid consists of a finite list of quantity-price pairs B = ((q 1 , p 1 ), (q 2 , p 2 ), . . . , (q m , p m )) with q i < q i+1 and p i > p i+1 (p i < p i+1 for selling). this player is expressing the following preferences. If the price is higher than 3, she does not want to buy at all. If the price is between 3 and 2, she is willing to buy any quantity in the range [0, 1]. If the price is between 2 and 1, then she is willing to by any quantity between in the range [0, 3]. Finally, if the price is 1 or lower, she is willing to buy any quantity in the range [0, 5]. demand) with the addition that the two players that define the clearing price do not trade. There is a different price for buying and for selling and all the profit resulting from the difference between both prices goes to the market maker.

Huang's auction satisfies several useful properties:

• Individually rational (IR)

• (Weakly)-Budget balanced (WB)

• Asymptotically Efficient in the number of players.

• Strategy-proof (SP) in the bided price, but not in the quantity.

We proceed to detail the different properties. An auction is IR if players do not loose money by participating (bidding). This is an important property, because without it, participation in the LEM could be greatly reduced by neighbors unwilling to take the risk. For clarity, an auction that is NOT IR is the case of a "pay-as-bid" auction, in which players pay their bid regardless of whether they win or not (this can make sense if the cost of preparing a bid in a large project is considerable).

Weakly budget balance refers to whether all the profit is distributed among the players or whether the market maker obtains a cut. In the discussed mechanisms, the market maker keeps part of the profit and this is represented by the area with the sky blue background.

A market is efficient if the outcome maximizes the social welfare (the sum of the utilities of all players). In Figure 5.1, an efficient mechanism will collect all the profit within the dashed horizontal lines and the vertical segment of the supply and demand curves. It can be seen that the mechanisms "throws away" two of this trades and therefore cannot be efficient. Because the market is asymptotically efficient, as the number of players increases, this loss becomes negligible.

Finally, a mechanisms is incentive compatible or strategy-proof if "telling the truth" is a weakly dominant strategy for all players. In the case of an auction, this could mean biding the desired quantity at the maximum price the player is willing to pay if buying (or minimum price willing to sell). One important consequence of incentive compatibility is that players do not need to learn how to play the game (trying to guess what to bid depending of what others might be doing), they just tell the auctioneer what they want. Huang's auction is SP in the price (players have no incentive to lie about their desired prices), but it is not in the quantity: it is possible for players to lie about their desired quantity while increasing their profit. This happens because by changing the bided quantity, the supply or demand curve shift horizontally, changing the price at which the participants trade.

Example B: A Strategy-proof Auction Mechanism

Throughout this thesis, we usually use a different auction mechanism than Huang's auction. Instead we use the Multi Unit Double Auction (MUDA) mechanism proposed by Segal-Halevi [START_REF] Segal-Halevi | MUDA: A truthful multi-unit double-auction mechanism[END_REF].

The main difference between both mechanisms is their clearing algorithm (how to decide who trades and who does not) which results in MUDA being completely strategy proof: not only in the price but also in the bided quantity.

Strategy-proofness is achieved as follows. Fist, after all bids have been received (the bid format used in this auction is exactly the same used in Huang's auction), the market mechanism randomly splits all participants into two groups and determines the clearing price1 for each of the two. Secondly, participants trade with the clearing price of the group to which they do not belong. To do so, all the buyers in one side of the market that offered to buy at a price higher than the clearing price of the other side and all the sellers that offered to sell below that price are pre-selected to trade. Because there might be more supply than demand (or vice versa), a rule is used to select which of the pre-selected sellers (or buyers) gets to trade. By doing so, agents cannot influence their trading price. Furthermore, MUDA is individually rational (agents do not lose money by participating), weakly budget balanced (the market maker does not lose money by running the market and might have a profit) and is efficient only asymptotically in the number of players (as it is impossible to satisfy all described properties at the same time [START_REF] Roger | Efficient mechanisms for bilateral trading[END_REF]). More details on the mechanism can be found [START_REF] Segal-Halevi | MUDA: A truthful multi-unit double-auction mechanism[END_REF].

From this point onwards, we will start describing LEMs in detail, concluding with a multi-stage stochastic LEM game and a simple decision model for a prosumer with a battery participating in the aforementioned game.

Proposed Architecture of a Sequential Local Energy Market

Before we dive into the details of local energy markets, we describe how these are considered in the context of the thesis work. A graphical representation of this process can be found in Figure 5.3.

First, players forecast their future consumption and production profiles as well as the future market prices as they expect to experience them. Then, they find an associated optimal battery schedule (with respect to the forecasts) that minimizes their costs of energy, using the decision model described in Chapter 4.

Secondly, knowing how much energy they need, they decide how much energy to trade in the local market and they bid that quantity. After the market clears, they receive the results and decide if they need to consume (or inject) more than the traded amount of energy with the main grid, which they are always allowed to do. The reader might notice that in the optimization problem (4.3) introduced in the previous chapter, no markets are considered. It is expected that if players use that model to participate in LEMs, the resulting behavior would end up being exactly the same as if no local market existed2 . Because of this, we will have to introduce in the decision problem of players the dependence on future market prices (analogously as we did when introducing the local matching pricing rule in Chapter 2). We will explain how to appropriately modify the decision problem of players later in this chapter.

Interpreting a LEM as a LEC is quite simple. For every players and every time-slot, the partial consumption profile R i l i = qi t where qi is the energy traded in the market by player i at time-slot t. Furthermore, the distribution of the costs of player i will be equal to the price paid for the energy normally traded with the TEC plus the quantity qi t pi t which represents the costs associated with trading in the market.

Related Work on Sequential Local Energy Markets

There is a vast literature in energy trading. Most of it is about trading at the wholesale level. In that regard there are many description of market mechanisms, but also optimal bidding strategies for producers.

One of the first proposals to use computational markets to allocate power was presented in [START_REF] Ygge | Power load management as a computational market[END_REF]. More recently, [START_REF] J Koen Kok | Powermatcher: multiagent control in the electricity infrastructure[END_REF] [15], [START_REF] Mengelkamp | Designing microgrid energy markets[END_REF] describe the design of different LEMs. Authors in [START_REF] Ilic | An energy market for trading electricity in smart grid neighbourhoods[END_REF] propose to use continuous double auctions for trading instead of periodic ones. For a survey on different characteristics of proposed markets the readers are referred to [START_REF] Lopez-Rodriguez | Methods for the management of distributed electricity networks using software agents and market mechanisms: A survey[END_REF] and the references therein, but also to [START_REF] Mengelkamp | Tracing local energy markets: A literature review[END_REF] and [START_REF] Sousa | Peer-to-peer and community-based markets: A comprehensive review[END_REF]. Of particular interest is the recent work of [START_REF] Guerrero | Decentralized p2p energy trading under network constraints in a low-voltage network[END_REF], where constraints of the power grid are embedded on the market design. In this chapter we analyze how players whose decision model follows a modified version of the model presented in chapter 4 would participate in a sequential local energy market. This is related to the literature on optimal bidding in LEMs but also whole-sale energy markets. In terms of the comparison of different trading strategies, our approach is similar to Mengelkamp et al. [START_REF] Mengelkamp | Trading on local energy markets: A comparison of market designs and bidding strategies[END_REF] , where different trading strategies are compared for non-truthful auctions. Bidding with inter-temporal constraints has been studied in [START_REF] Carlsson | Algorithms for electronic power markets[END_REF], but the authors did not consider substitutes and their value functions are pre-specified for a given set of loads. Deriving an optimal bid by solving an optimization problem has been studied

Market TEC

b i 1 qi 1 , pi 1 g i 1 C(g i 1 ) Stage 1 Market TEC b i 2 qi 2 , pi 2 g i 2 C(g i 2 ) Stage 2 Market TEC b i 3 qi 3 , pi 3 g i 3 C(g i 3 )
Stage 3

Figure 5.4: Timeline of decision process and decision flow for a player in the LEM game mostly for generators and aggregators trading in wholesale electricity markets. The optimal bidding strategy of a generator is studied in Conjeo et al. [START_REF] Antonio J Conejo | Price-taker bidding strategy under price uncertainty[END_REF]. The authors derive an optimal bid using a forecast of the market prices embedded on their scheduler. Their work differentiates from ours in that their model does not include substitutes for selling energy. Here, by substitute we mean that players always have the option to buy from the Traditional Electricity Company and are not forced to trade in the LEM. That is, energy from the TEC and from the LEM are perfect substitutes in the eyes of prosumers which we assume have no preference for "local" energy asides from its possible cheaper price. In [START_REF] Ødegaard Ottesen | Prosumer bidding and scheduling in electricity markets[END_REF], the authors model the bidding and scheduling of an aggregator using stochastic programming. In their work they consider a penalty cost for all the energy that is not traded in the market, which resembles the price at which players in this thesis buy energy from their TEC. Nevertheless, they do not attempt to derive an optimal bid. Instead, they decide on the price points of which the bid is composed in an exogenous manner.

A Game Theoretical Model for Sequential Local Energy Markets

Having provided an overview of how a LEM works and examples of two market mechanisms that can implement a LEM, we proceed to introduce a formal model. Consider a setting with N players and T = {1, 2, . . . , T } stages (or time-slots). Each stage is composed of two steps: a market trading and a final settlement with the traditional electricity company (TEC).

Following the conventions used in game theory, the superscript -i will stand for all players expect player i. The state of player i (s i t ) at time-slot t will contain information about the state of charge of the battery and the load of the current and future time-slots of the player. It will also contain information about the beliefs regarding future market prices and the possible actions of all other players. The set B i t (s i t ) will denote the feasible bids of player i at time-slot t given her state of charge.

Assumption 5.1. Players restrict their bids in the market to quantities that they can physically buy or sell.

In practice, a player that often bids a quantity that she cannot latter provide (or consume) could see her access to the market restricted.

Players can be buyers or sellers, but not both in the same time-slot.

After the market clears, player i observes her traded quantity qi t and price pi t just before the first step finishes. In the second step, she makes sure that her energy demands are satisfied. She can decide to schedule some of her energy needs for later but the rest will have to be bought (or sold) from the TEC. The set of possible settlements with the TEC at time t that player i can offer is given by

G i t ∆ = G i t (s i t , qi t , pi t )
and depends on the state of the player (how much energy is needed and how flexible that consumption is) and the result of the market. The cost of trading a quantity g i t with the TEC at time-slot t is given by C i t (g i t ). In the model described in chapter 4 this would imply that C i t (g i t ) = β i t max{g i t , 0} -γ i max{-g i t , 0}. Figure 5.4 illustrates the decision and information flow for a given player as described above. We shall use the convention that positive quantities g i , qi imply buying energy while negative values are used for selling.

After implementing an action g i t ∈ G i t , player i transitions to the new state

s i t+1 = f i t (s i t , qi t , pi t , g i t )
(where f i t defines the dynamics of the state transition) and the stage finishes. Denoting by a t = (b i t , g i t ) the pair of actions taken by player i during stage t, where b i t is the bid submitted and g i t is the quantity traded with the TEC, the set of all such possible actions at the beginning of the stage will be denoted by A i t The set of all possible actions that players other than i can take at stage t is defined as

A -i t ∆ = j =i A j
t . This set depends on the past actions of all players and their future load profiles.

In a setting with perfect information, player i will have a belief about the likelihood of her opponents playing a given strategy and the equally likely splits of the market into two groups. Let ∆A -i t be the set of probability distributions over A -i t × W , where W is the set of all possible 2 N -1 splits into two groups. Then player's i belief about the possible outcomes of the game at time-slot t is given by d i t ∈ ∆A -i t . Finally, we will denote the expected Cost-to-go of player i from stage t onwards while being in state s i t as Q i t (s i t ) 3 . In this setting, it is defined as the solution of the two step stochastic optimization problem (5.1a).

Q i t (s i t ) = minimize b i t ∈ B i t (s i t ) E d i t ∼∆A -i t [A(b i t , a -i t , w, s i t )] (5.1a)
where

A(b i t , a -i t , w, s i t ) = minimize g i t pi t qi t + C i t (g i t ) + Q i t+1 (s i t+1 ) (5.2a) subject to qi t , pi t = M(b i t , a -i t , w), (5.2b)
g i t ∈ G i t (s i t , qi t , pi t ), (5.2c)
s i t+1 = f i t (s i t , qi t , pi t , g i t ) (5.2d)
with Q i T +1 = 0. In optimization problem (5.2a), M is a function that outputs the results of the market according to rules of the MUDA mechanism, taking as an input the bids of all the players. Optimization problem (5.1a) is solved every stage before bidding in the market. We shall assume that if no quantity is traded in the market, it is always possible to buy or sell the required amount of energy with the traditional utility company. As a consequence, the set G i t (s i t , 0, pi t ) is always non-empty. This is a realistic assumption as consumers can always buy all their energy from their TEC (at least in the context of the energy transition considered in this thesis). The model above defines a multi-stage stochastic game.

Remark 5.1. In most real implementations of such markets, it is possible that players will only be able to observe the result of their trade, but not the bids of other players (privacy is one of those reasons). This would limit the ability of players to reason about the game. In that case, player's beliefs will likely only consider prices and quantities, but not actions.

Strategic Behavior

There are two main types of strategic behavior that agents can engage in while playing the game. The first one consists on learning from other players' bidding patterns the expected trading prices and quantities at each stage and try to re-schedule their bidding in order to exploit the best deals. For example, if player i is a seller and she learns that there is a stage in which there is usually much more demand than supply, she will try to bid a higher quantity in that stage, expecting a higher selling price. The second kind of strategic behavior consists on submitting sub-optimal bids in the current stage, with the hope that it will produce a ripple effect in future trading periods which will result in an overall higher profit. An example of this would be not selling in the first stages with the hope of becoming the only seller in a future stage and make enough profit out of a "monopoly". Of course, this requires a very accurate knowledge of how much demand and supply there will be in future stages and how it depends on past trades. We believe that the first type of strategy will likely take place as it is easy to understand how to benefit from it 4 . On the contrary, the second kind of strategic behavior requires a deep understanding of the reaction that other players will have to specific outcomes of the game, which will most likely be unavailable. Coupled with the fact that outcomes of the market are random (MUDA splits players in a random fashion), it seems that the whole strategy is unlikely to succeed. Furthermore, it is possible that in the final implementation of LEMs, players will only have access to their market results, but not to the bid of other players. If that is the case, it will be almost impossible to implement strategic behavior of the second type.

Revisiting the Prosumer's Model

It should not come as a surprise to the reader that the model introduced in the previous section is quite difficult to solve. We do not try to solve such a model in this thesis. Instead, by writing the model down we expect to obtain insights that allow us to properly model the agents that participate in such markets,enabling us to produce realistic simulations as a result. To do so, we need to model rational agents in a simple way that is coherent with the game being played and that is able to capture some behaviors of interest.

Assumption 5.2. The actions of a player will affect her own state dynamics (through the operator f i t ), and those of other players through the results of the markets. Here, we assume a model of bonded rationality, in which players schedule their actions taking into account their private dynamics (charging a battery knowing that the energy will be available later) but do not take into account how those actions affect the private state of other agents. This is reflected in the fact that for a fixed instance of the decision problem of an agent, future market prices and maximum tradable quantities do not depend on the chosen action.

We believe that Assumption 5.2 is realistic and we do not expect real implementations of LEMs to behave any differently.

Remark 5.2. Even though the stochastic game as a whole might not be truthful, if as part of a player's strategy, she decides that she wants to buy q units at price at most p during time-slot t, then bidding truthfully in MUDA is the optimal action (in that time-slot).

As a consequence of Assumption 5.2 and Remark 5.2, the first-stage optimization problem can be substituted with the optimal amount of energy that each player wishes to buy or sell, given her beliefs about market prices. Indeed, the proposed model is exactly the optimal control of a battery with one addition: for each time-slot, apart from the price of the TEC, there is a market price with an associated maximum tradable quantity. This model extends the model introduced in Chapter 4 and for consistency we will use the same notation.

To keep the model simple, we assume that agents have a probabilistic belief about prices, but to avoid solving a stochastic optimization problem in each step, they use an unbiased estimator of each quantity instead.

As we mentioned earlier, each player has a belief in the form of a probability distribution about each market outcome qi t , pi t . Because players can be either buyers or sellers in the market, they will have a possibly different belief for each case. Let PB t , PS t , QB t , QS t be the priors for the buying price, selling price, buying quantity and selling quantities respectively for each time-slot t. The notation X will be used to denote an unbiased estimator of X.

For a player i that wishes to buy, the best option is to buy as much as possible in the market (at a better price than the TEC), and acquire the remaining energy with the TEC. The same holds for a player that wishes to sell. Substituting the player's beliefs about quantity and prices of the market in time-slot t, the cost (or profit) of player i during time-slot t associated with a net consumption profile of z i t is given by Equation (5.3).

F i t (z i t ) =            PB i t QB i t + β i t z i t if z i t > QB i t PB i t z i t if QB i t ≥ z i t ≥ 0 PS i t z i t if QS i t ≤ z i t ≤ 0 PS i t QS i t + γ i t z i t if z i t < QS i t (5.3)
With the above conventions, the decision problem faced by a player is given by optimization problem (5.4). Observe that this problem is exactly the same as in chapter 4 but with the change in the cost function.

min x i T t=1 F i t (z i t )
subject to:

x i ∈ X i z i t = l i t + max{x i t , 0} η c -max{-x i t , 0}η d , ∀t ∈ T (5.4)
Because the market is truthful, we find the bid of player i as

b i t = (z i, * t , PS t ) if z i, * t < 0 or b i t = (z i, *
t , PB t ) otherwise, where z i, * is the value of z in an optimal solution of (5.4).

Proposition 5.1. If the function F i t as defined in Equation (5.3) is convex (which happens as long as

PB i t ≥ β i t ≥ PS i t > γ i t , ∀t ∈ T )
, then optimization problem (5.4) is linear. The ideas of the proof are the same as the ones used for the corresponding proof in Chapter 4.

Post market After the market clears, based on the results of the market, players have to decide how much to finally trade with the TEC, if any. To do so, players should modify the cost function (5.3) by replacing their beliefs about market prices with the real results. Furthermore, if the player managed to trade in the market a quantity qi then an additional constraint

z i t ≥ qi t (if qi t > 0 or z i t ≤ qi t otherwise)
, should be added to guarantee that players will adhere to their commitments in the market. Here, we assume that players adhere willingly to their commitments with the market, but we could also envision a penalty for deviating from such result.

The value of g i t , the quantity to be traded with the TEC is be the difference between the new optimal value of z i t and qi t .

Modeling a Prosumer's Beliefs About Future Market Prices

As it was mentioned before, we assume that players have a belief (probability distribution) about future market prices and maximum tradable quantities. As these players play the game, they will observe new market's results and they will update their beliefs using the new information, improving their representation of the game. We model players' beliefs as conjugate distributions and use a Bayesian rule to update them. In theory, players could have up to 4T different beliefs where the 4 is because of the 2 quantities and 2 prices involved. In practice, players will reuse the same belief in time-slots in which they expect them market the behave similarly, for example because of time-of-day patterns. Each belief will be represented as normal probability distributions with unknown mean but known variance N 1 (µ i t , σ i t ). Furthermore, each player will have a belief about the value of the mean of such distribution in the form of a normal distribution

µ i t ∼ N 2 (υ i t , τ i t ).
After observing n outcomes of the variable of interest over time (which we will assume independent), the new values of υ and τ are given as:

τ = ( 1 τ 2 + n σ 2 ) -1 and υ = τ υ τ 2 + n 1 xi σ 2
, where (x i ) i=1,...,n are the observations of the variable in question.

If players use a different belief for each time-slot, they will only observe one outcome and that belief will never be used again. In contrast, if players reuse their beliefs across several days, updating them can be useful.

Players can change their beliefs about the game by doing two things. First, by deciding how to map time-slots to distributions (reuse), i.e., two time-slots can use the same distribution and the data of both time-slots updates the same prior. Secondly, players can change the initial value of their distribution to reflect their beliefs about the market before observing it.

Heuristic Action Profile for Prosumers

How should players participating in the multi-stage stochastic game described above bid in the individual markets?

In previous sections, we proposed a model for a player that participates in such games. There, we introduced the notion of a belief about future market prices and we explained how to incorporate them into the standard linear programming problem of each player.

In this section, we will offer some intuition on why such model makes sense. To do so, we will begin with a simpler problem that can be solved analytically, and we will discuss how a natural extension to this later problem results in the model described in the previous sections.

Case With No (Known) Future Market Prices

The simpler problem consists on a LEM in which there is a market only during the first time-slot. In other words, players can only trade among themselves during the first time-slots, and have to fall back to trade with the TEC for the rest of the game.

Assuming that this player which we shall call Ana for simplicity behaves according to the basic model described in Chapter 4, the value of the cost-to-go in the final time-slot Q i T is given by:

Q i T (s i T ) = min x i T β i T max{z i T , 0} -γ i T max{-z i T , 0} + 0
subject to:

x i T ∈ X i (S i T ) z i T = l i T + max{x i T , 0} η i c -max{-x i T , 0}η i d , (5.5)
Since in time-slots t = 2, . . . , T there are no unknowns surrounding other players, E[A] = A. Then, for time-slot T -1 we have that:

Q i T -1 (s i T -1 ) = min x i T -1 β i T -1 max{z i T -1 , 0} -γ i T -1 max{-z i T -1 , 0} + Q i T (s i T )
subject to:

x i T -1 ∈ X i (S i T -1 ) z i T -1 = l i T -1 + max{x i T -1 , 0} η i c -max{-x i T -1 , 0}η i d , (5.6)
Combining both (Equation (5.5) and (5.6) we obtain that:

Q i T -1 (s i T -1 ) = min x i T -1 ,x i T T t=T -1 β i t max{z i t , 0} -γ i t max{-z i t , 0}
subject to:

x i T -1 ∈ X i (S i T -1 ) S i T = S i T -1 + x i T -1 x i T ∈ X i (S i T ) z i T -1 = l i T -1 + max{x i T -1 , 0} η c -max{-x i T -1 , 0}η d , z i T = l i T + max{x i T , 0} η i c -max{-x i T , 0}η i d , (5.7)
Proceeding all the way until the first time-slot, we obtain that:

A(b i 1 , a -i 1 , w, s i 1 ) = min b i 1 ,x i pi 1 qi 1 + T t=1 β i t max{z i t , 0} -γ i t max{-z i t , 0}
subject to:

pi 1 , qi 1 = M(b i 1 , b -i 1 , w) x i t ∈ X i (S i t ), ∀t ∈ T S i t+1 = S i t + x i t , ∀t ∈ T z i t = l i t + max{x i t , 0} η i c -max{-x i t , 0}η i d , ∀t ∈ T (5.8)
Interpreting optimization problem (5.8), we can understand how a player might reason about their optimal bid. First, observe that in general, pi 1 and qi 1 are unknown in advance since the bids of the other players and the random value w not available to the player at the time to submit the bid. In spite of this, we can do the following: assume that the market price is known (disregarding the exact bid of other players) and find what would have been the optimal (desired) traded quantity qi 1 . Clearly, if pi 1 > β i 1 (or pi 1 < γ i 1 for selling), then qi 1 should be 0 since it is better to trade with the TEC.

For the other cases pi 1 ∈ [0, β i 1 ), we can substitute β i 1 by pi 1 and obtain the desired quantity qi 1 associated with pi 1 (this will be equal to z i 1 ). The accuracy of the bid can be increased by repeating this process for each value in the interval.

Case with Unknown Future Market Prices

The deduction presented in the previous subsection depends on the assumption that future prices are known. This is the case when the local energy market only occurs during the first time-slot.

Unfortunately, the model of interest to us requires unknown future market prices and quantities traded in those markets.

The most simple extension, is to assume that those future market prices (and quantities) can be forecast somehow. If that is the case, and those forecasts are accurate enough, it is reasonable to assume that the same model we just derived can be used, replacing the known TEC prices with the forecast market ones.

If each player has a belief about market prices, they can use their expectation of such quantity in the model as their forecast. In doing this, we arrive to the model described in Section 5.5.

Key Outcomes and Conclusions

In this chapter we introduced the concept of a local energy market as will be studied throughout the thesis. We described two market mechanisms to implement such markets: MUDA and Huang's, that differ on how truthful they are. We described a mathematical model of a multi-stage stochastic game that described the interactions of agents among them and with the TEC, revealing the complexity of the problem in question. Later, we proposed a mathematical model for an agent that participates in a LEM and we provided some intuition on why such model is descriptive of the problem at hand. This chapter exposed the complexities that are inherent to local energy markets implemented as sequential auctions, in which individual players can have beliefs about future market prices (and act accordingly). In the next chapter, we will provide empirical evidence that such markets can destabilize the power grid, as they provide incentives to players that are not aligned with those of the grid operators. This will prompt us to continue searching for better incentives and mechanisms.

Chapter6

On Some Structural Problems with Local Energy Markets

In the previous chapter, we described in detail a mathematical model for local energy markets. In it, we described them as a multi-stage stochastic game and we provided a reference model of how a player might decide to play such game.

At the end of the chapter, we hinted that the complexities of those multi-agent systems might yield behaviors that are detrimental for the power grid but beneficial to the market participants.

In this chapter, those issues are explored in further detail. We will motivate the rest of the chapter with small examples and then we will provide large-scale numerical examples of the aforementioned "bad" behaviors.

The idea that market prices can lead to instabilities in the power system is not new. Roozbehani et al. [START_REF] Roozbehani | On the stability of wholesale electricity markets under real-time pricing[END_REF][START_REF] Roozbehani | Volatility of power grids under real-time pricing[END_REF] study the effects of passing down real time prices to end customers. From a simple model in which demand changes base on real time prices, the authors obtain several conditions under which the system is unstable. The thesis of Negrete-Pincetic [START_REF] Negrete-Pincetic | Intelligence by design in an entropic power grid[END_REF] also shed light in the variability of prices and consumption. As a key factor, they identified that energy markets treat energy as a single dimensional commodity and not as a multi dimensional product, which it is. To summarize, in this chapter we show how sequential local energy markets are no exception to the problems faced by customers faced with varying energy prices.

Motivation Through Small Examples

As a motivation for this section, we present some of the examples that we found during our study of sequential local energy markets, which prompted us in the search of different alternatives.

Example A: Artificial Peak Demand

There are 4 players (3 buyers and one seller) and 3 time-slots. The 3 buyers need one unit and can get it in any of the 3 time-slots. The buyers can buy their unit in the local market at a variable price, or from the TEC at price 2. The seller has one unit to sell, only in the 3rd time-slot. He can sell it in the market or to the TEC for a price of 1. There is a probability p that each buyer will have 0 demand instead of desiring one unit, those events being independent, and the other buyers know that.

All buyers have an incentive to wait until the last time-slot: there is no loss in doing so (the price of the TEC does not change) and the profit can be bigger (because of the possible lower price in the market, e.g., with probability p 2 (1 -p) they will be the only buyer facing the seller) Although this is not necessarily bad for players, this is a bad equilibrium for the grid: it creates a peak in the last time-slot. Also, this outcome is not flexibility-efficient: in the best scenario, each buyer consumes in a different time-slot and the peak is the smallest possible.

Example B: Missed Trade Opportunities

There are two time-slots, one buyer and two sellers.

The buyer needs one unit in any time-slot. Seller 1 has 1 unit available only in the first time-slot (always) and seller 2 has 1 unit available in the second time-slot with probability p.

The buyer pays 3 at the TEC for energy, seller 1 can sell to the TEC at 2 per unit and seller gets only 1 per unit when selling to the TEC.

57

The sellers have no actions, they try to trade in the market, or they sell to the TEC if they don't. The buyer has to decide whether to trade with seller 1 or to wait for the better price at seller 2 (guaranteed if seller 2 has demand, which is not known). If p is high enough, then the buyer will always prefer to wait. Of course, this means that with probability 1 -p the first seller and the buyer will both use the main grid as a battery, instead of trading among themselves.

Example C: Time-of-Use

After understanding that any solution involving sequential markets will require the usage of forecasts, it is natural to wonder whether such reliance can be problematic. Here, we do not look at the performance of a single agent, but at the aggregated behavior of the multi-agent system. Consider a LEM game with N players and T = 10 stages. Furthermore, for all players it holds that β i 1 , . . . , β i 5 = 1, β i 6 , . . . , β i 10 = 2, l i 1 , . . . , l i 5 = 0, l i 6 , . . . , l i 10 = 1 and S i = 5. A feasible solution of this problem that exploits the flexibility available to players in the absence of a LEM, is for them to buy one unit each during the first five time-slots. This is the desirable outcome for the DSO as the load profile is flat. Now, instead of the optimal outcome, each of them forecasts that PB i t = 1 2 , t = 1, . . . , 5 and QB i t ≥ 5. Unfortunately, they were wrong and no one gets to trade. Then, at the end of time-slot 5, they acquire all their demand before the change in price (from 1 to 2) of the TEC. This produces a peak of size 5N during time-slot 5. Such a peak might cause issues in the quality of electricity supply, and a huge cost on the DSO. On the other hand, it can easily be seen that the profit for each user is the same as if they had spread their consumption evenly. That is, they do not pay extra for waiting for the right market opportunity. Furthermore, this scenario is desirable from the agents' economic perspective as it allows them to delay their expenditure [START_REF] Brigham | Fundamentals of financial management[END_REF]. This goes to show that the naive design of LEMs can incentivize unwanted behaviour (regardless of the auction used).

Contributions The example presented above points to a gap in our understanding of local energy markets. This problem is related to the coexistence of LEMs with alternative ways of trading energy and, in particular, to how agents plan their schedule with respect to future prices, a topic usually not considered in the literature [START_REF] Mengelkamp | Intelligent agent strategies for residential customers in local electricity markets[END_REF], [START_REF] Guerrero | Decentralized p2p energy trading under network constraints in a low-voltage network[END_REF].

The work from Alabdullatif et al. [11] is closely related to our approach. They study a set of agents that participate in a LEM and have the option to trade with a TEC instead. They do not model the scheduling of each player's flexibility as an optimization problem nor they forecast trading prices in the market for more than one time-slot ahead. Because of that, some of the behaviors and shortfalls of LEMS described in this chapter cannot be captured by their approach, as they arise form the higher (but realistic) complexity of the system.

Numerical Experiments Showcasing the Problems of LEMs

In the previous section we presented undesirable examples of what can happen in a local energy market. In it, players had an incentive to behave in a way that resulted in unnecessary peaks of consumption or wasted flexibility.

In this section, we make use of the model of a prosumer participating in a LEM to replicate the behavior encountered in the motivating example. Our intention is to understand which characteristics of Example C can be observed with a large number of players and which of them are only an effect of the size of the example. To do so, we resort to computational simulations of the whole multi-agent system.

Methodology

Each simulation consists of 9 consecutive days in which prosumers optimize their energy storage and trade using the LEM. Each agent will use the mathematical model described in Section 5.5. Different simulations will reflect different load profiles and representations of the uncertainty.

Each prosumer owns a battery and some agents have access to solar generation. The demand of each agent is sampled from the Ausgrid dataset [START_REF] Ratnam | Residential load and rooftop PV generation: an australian distribution network dataset[END_REF]. The dataset contains samples every 30 minutes, yielding 48 time-slots (D = 48) per day. For each instance of the simulation, the demand of N = 50 prosumers is sampled out of the 127 available in the dataset.

Because the generation in the data is not enough to support a LEM (there is little surplus available for trading), for half of the users we generate extra surplus at random. To do so, for each time-slot t mod D ∈ {15, . . . , 30} and each selected agent i, we sample extra generation from a uniform distribution R i t ∼ U[-3 10 , 0] i.i.d such that the new demand is given by l t = l t + R t .

The battery characteristics are the same for each prosumer:

S = 13, δ = -δ = 5, η c = η d = 0.95.
Two price tariffs are used for our simulations: a flat rate and a time-of-use tariff. In the flat rate, the price of buying energy is constant with a value of 30. The time-of-use tariff consists of two periods: a cheap one and an expensive one. The first one spans the first 32 time-slots of every day and has a value of 20 while the latter spans the remaining time-slots of every day and has a value of 30. Both tariffs have the same price for selling for all time-slots: 10. All prices are in cents per kilo watt hour.

In this experiment, players find their actions using Algorithm 2 (described in chapter 4) a rolling horizon of length 48 time-slots (or 1 day) to solve optimization problem (5.4). Participants only implement the first decision out of the 48 that they obtain before moving to the next time-slot.

To solve optimization problem (5.4), prosumers need to know their load in the next day (because of the rolling horizon procedure). Players have perfect knowledge of their load in the first of the 48 time-slots, and use a forecast for the remaining 47. We adopt the AvgPast forecast as described in chapter 4.

All market mechanisms will be run using the auction mechanism MUDA as implemented in PyMarket [START_REF] Kiedanski | Pymarket -a simple library for simulating markets in python[END_REF] and the optimization problems involved in controlling the battery are solved with CPLEX. Simulations were run in parallel using GNU parallel [START_REF] Tange | Gnu parallel-the command-line power tool[END_REF].

Modeling Beliefs about Future Market Prices

In all our simulations, players learn from their own experience. That is, they update their belief based solely on the market prices and quantities they have been subject to, but not those of the other players. A player will update her beliefs about prices only if she gets to trade a positive quantity in the market.

We proceed to introduce the types of priors used to represent future market quantities in the experiment. First we describe three representations: Optimistic (OPT), Neutral (NEU) and Pessimistic (PES). The 3 representations map the 4 distributions (PB, PS, QB, QS) at each time-slot t ∈ T to the 4 representations of time-slot (t mod D), effectively keeping 4D beliefs: one for each quantity and price for each time-slot in a single day. This exploits the time-of-day effects. Their difference lies in the initial value of υ i (the mean). The OPT belief assumes that prices in the market are 30% better than the trading with the TEC (both, while buying and selling). The NEU belief assumes that prices are only 10% better than the TEC. Finally, PES assumes that the prices are the same as the TEC.

We consider two more types of beliefs. One named Solar (SOL) and one named Unique (UNQ). The SOL belief considers only two different distributions of each type. Time-slots that occur when the sun should be shining1 are mapped to a distribution and the rest are mapped to another one. The distribution associated with sun hours is initialized by assuming prices to be 20% better than the market while in the other belief prices are the same as the TEC. At last, the UNQ belief considers only 4 distributions, one for each price and quantity. In this case, the prices are considered to be 10% better than those offered by the TEC. All the beliefs about quantities for all the different representations are initialized at a large value, stimulating an initial participation in the market.

In addition to different representations, we also consider different frequencies to update beliefs: with every new observation of the market, every n observations of the market or the most extreme case of never updating the belief.

Proposed Evaluation

We conclude this section with the four metrics of interest studied in this chapter. First, we looked at the Social Cost (SC), the sum of the cost incurred by all players during all time-slots. This is an indicator of how well the market performs from the perspective of the players. Secondly, we look at two statistics, the maximum peak and the most negative peak of the aggregated net consumption profiles. We refer to them as max and min respectively. Finally, we look at the total amount of energy that gets matched locally (LM). This is an upper bound of the energy traded in the market since all the energy traded in the market gets matched locally, but it is possible for two prosumers to consume and inject at the same time without having traded2 . Because all of the above metrics are difficult to contextualize on their own, instead of presenting their corresponding value, we will always show the relative change of the metric with respect to the scenario without a market in which players optimize their battery independently of each other.

Results and Discussion

We begin our presentation of the results obtained by directing the attention of the reader to Table 6.1. In it, the average relative change with respect to the scenario without a market is presented. For SC, a negative value indicates that the market managed to decrease the total cost (a positive outcome). For both min and max, a negative value is desirable as it denotes that the maximum positive peak or the maximum negative peak were reduced. Finally, a higher and positive value of the locally matched energy LM is beneficial as this was one of the motivations to introduce energy markets.

In Table 6.1, the column Freq denotes the number of samples collected before updating the beliefs. In addition to not updating their beliefs, in simulations with a Freq value of 0, players did not trade in the market, they only used the belief to change their battery schedule planning. Solar Tariff as used in the legend of Figures 6.1 and 6.3 refers to the SOL belief in the Freq 0 scenario. The last column of the table shows the total amount of energy (in kWh) that was locally matched during the simulation. We can observe that the Solar Tariff is less effective when paired with a Flat tariff.

In Figures 6.1 and 6.3, each curve represents the difference between the aggregated net load of a simulation using the market and a set of beliefs and the aggregated load obtained without a market (for the same parameters). The x-axis coincides with the default aggregated load. In Figure 6.1 players are subject to a Time-of-Use tariff, while in 6.3 they use a Flat tariff.

By turning our attention to Figure 6.1 we notice two things. First, of all the beliefs plotted in red, most of them produce small deviations with respect of the default operation (close to 0) while one of them creates very high peaks. The belief producing the peaks in red is NEU. Interestingly, Table 6.1 indicates that for a ToU tariff, the NEU achieved a reduction in the social cost, while creating higher peaks and reducing the amount of energy locally matched. Indeed, we observe a misalignment of objectives. To explain why this behaviour emerges, we refer the reader to Figure 6.2. In it, we plotted the same curve producing the peaks of Figure 6.1 in blue. In red, we plotted the total amount of energy that participants asked in the market at the beginning of each time-slot. We observe that there is a mismatch between the two quantities. Moreover, players try to trade in the market until the change in price. At that point, they decide to buy all their required quantity. This is a consequence of the expected market price in the most expensive period: 30 * 0.9 = 27 > 20, is higher than the default low TOU price. We argue that this type of behaviour, only due to the beliefs of a player, can show up in real deployments of LEMs with dire consequences. Unfortunately, from an economic perspective, this behaviour is rational for agents as it allows them to delay their expenditure [START_REF] Brigham | Fundamentals of financial management[END_REF].

One might wonder why we do not see the same effect observed for NEU for OPT if we still have that 30 × 0.7 = 21 > 20. This is because the actual price of electricity is related to the round-trip efficiency of the battery. In this case, with a round-trip efficiency of 0.95 2 , the actual cost of charging during the low period and discharging in the most expensive period is 20/0.95 2 ∼ 22.16 > 21. This explains why players do not engage in the frenetic behaviour of consuming pre-peak: they still believe that trading in the market during the most expensive period will be cheaper.

The second thing to notice in Figure 6.1 is that the Solar Tariff that performs fairly well, performs quite badly when the default tariff is flat. This is important to notice as it reveals that a belief is not bad on itself, but only inasmuch as it is coupled with a default electricity tariff.

Our numerical findings support the hypothesis formulated around the example in Section 6.1. When players have access to an unlimited supply of energy outside the market, the tariff at which that energy can be bought should be carefully designed. Otherwise, it is possible that players (inadvertently) game the system, trying to increase their profits at the cost of the physical grid.

Flat rates offer less incentives than ToUs to change the patterns of consumption in ways that result in spikes, for most of the beliefs tested in this chapter. Nevertheless, they offer slightly less efficient environments. It is not unthinkable to imagine a trial in which the flat rate is replaced with incentives in special times of the day to increase local consumption. Such incentives should be designed carefully, as we showed that they could prove dangerous to the operation of the power grid.

Key Outcomes and Conclusions

Local Energy Markets are increasingly being proposed as a solution for distributed energy resource management on smart distribution grids. Nevertheless, several aspects of such programs related to their implementations and how they will alter the behaviour of their participants remains unknown and further analysis is required to understand all possible ramifications.

In this case study we presented simple pathological examples in which the equilibrium strategies of the players resulted in undesirable peaks for the grid operator. The examples were further validates with numerical examples in which the multi-agent system was simulated for 9 consecutive days.

Our experiments indicate that some strategies and beliefs of players can create peaks of consumption that would not exist without the market. Flat tariffs seem to be better adapted to be coupled with local energy markets, even though they provide lower energy matching capabilities.

The results in this section depend on who will be the decision-maker in practice: a human or an algorithm. In practice, the assumption of rationality is better suited for computers than for humans. There are advantages and disadvantages of both approaches. On the computational side, computers are more sensitive to prices: a price that is epsilon cheaper in the feature can lead to a drastic change in energy consumption whereas a human might have a slower reaction. Of course, this can be mitigated by a "learning rate of sorts". When it comes to the implementation of both approaches, computer algorithms might require more powerful hardware than what is usually deployed as Home Energy Management Systems. It is mostly because of this point that we assumed that players might be able to reason about the impact their action will have on their future consumption but not on the impact that they will have on the actions of others: it is unlikely that the computational power to do so will be available for such an application. On the other hand, humans can get tired of bidding, only updating their preferences every now and then and so limiting the usefulness of the market. In my personal view, automatic controllers (possibly implemented through threshold policies for efficiency) are the most likely implementation for prosumers in a local energy market.

Local energy markets will require the design of new mechanisms capable of dealing with existing tariffs or new tariffs capable of supporting markets while still providing efficient outcomes.

Looking at the examples at the beginning of this chapter we notice two things. First, some of the problems seem to arise from players "waiting" and secondly, not all the flexibility is properly exploited. In the next part we will study a new market mechanism that aims at solving these two problems.

Part III

Combinatorial Local Energy Markets for Local Energy Trading

Chapter7

A Model Based on a Combinatorial Auction For Local Energy Markets

So far, we have explored local energy trading through the lens of sequential local energy markets. We explained how such markets can operate, how agents can participate in them and some of the different problems associated. Specifically, in the previous chapter we also pinpointed some structural problems with sequential LEM that hinder their ability to properly exploit the flexibility in future Smart Grids.

In the following two chapters, we will discuss a different market model that, unlike the ones studied in previous chapters, clears day-ahead.

Motivation for Combinatorial Auctions

When designing markets, it is useful to exploit the characteristics of the environment in which they will be deployed. We can observe some of these ideas in the wholesale energy market, where special types of bids were designed to provide generators with a descriptive language in which they could express their preferences, tightly coupled to the start-up costs in their energy production processes.

As an example, in NordPool [123], they identify four types of block bids. A regular block order that is "all-ornothing" and can span several hours (this in sharp contrast with simple bids as seen in the previous part that can be partially accepted). Linked block orders in which the acceptance of a block is conditioned on the acceptance of its parent (think of a tree structure). Curtailable bids can be partially accepted up to certain threshold defined in the bid. Finally, there are also profile block orders.

Following the same spirit, namely the design of bids that are tailored to the participants, we propose a new market mechanism for local energy markets.

We put forward the design of an approach based on a combinatorial double auction [START_REF] Li | Pricing in combinatorial double auctionbased grid allocation model[END_REF], [START_REF] Samimi | A combinatorial double auction resource allocation model in cloud computing[END_REF], [START_REF] Xia | Solving the combinatorial double auction problem[END_REF] that improves the utility of all players and increases the total traded energy. Even though combinatorial auctions have already been proposed [START_REF] Penya | Combinatorial markets for efficient energy management[END_REF], [START_REF] Carlsson | A flexible model for tree-structured multi-commodity markets[END_REF] in the scope of local energy trading, the design presented here is the first to exploit the structure of flexible demand derived from energy storage.

We shall begin by briefly explaining what a combinatorial auction is. Then, we will present a small motivating example that depicts why it can be important to have a combinatorial auction for local trades. After discussing related work we will promptly delve into the proposed model.

Motivating example

Consider a setting with two time-slots, one seller and two buyers. The seller has one unit to sell in each time-slot (no flexibility). Buyer I needs one unit in the first time-slot (no flexibility). Buyer II needs one unit but is indifferent in which time-slot she gets it. Buyer II is willing to pay slightly more than buyer I. In a LEM implemented as a sequential auction, both buyers will try to trade in the first time-slot. The reason for this is that buyer II does not know the future (although she might have some beliefs about it), and would rather not risk loosing an opportunity to buy at a better price. In the end, Buyer II will trade with the seller in the first time-slot (because she offers a more competitive price), while Buyer I will be forced to buy from the TEC. In the second time-slot, the seller is forced to sell to the TEC as there are no buyers.

If the mechanism had allowed Buyer II to express her flexibility, Buyer I would have traded in the first time-slot and Buyer II in the second one, consuming all surplus locally and at a higher welfare for all participants involved.

To solve this problem we propose to design a mechanism that better integrates the available flexibility of the participants.

Overview of Combinatorial Auctions

Simply put, combinatorial auctions are auctions in which participants can express their preference for combination of items. For example, at an auction selling individual shoes, a participant might be interested in buying only matching pairs.

Combinatorial auctions have been used for a long time and continue to be in use today [START_REF] De | Combinatorial auctions: A survey[END_REF], [START_REF] Nisan | Algorithmic Game Theory[END_REF]. As early as 1976, Jackson [START_REF] Lee | Technology for spectrum markets[END_REF] proposed the use of a combinatorial mechanism to auction the rights for radio spectrum. Other uses of combinatorial auctions include but are not limited to: allocation of airport slots, trading of financial securities considering bundles and bus routes.

The day-ahead wholesale electricity market in Europe is indeed an example of a combinatorial auction.

Although very appealing in theory, combinatorial auctions have several practical drawbacks. First, having each player express her own preferences can be quite complex, since for N different items there are possibly 2 N different subsets. To tackle this, several bidding languages have been proposed to simplify the bidding process (such as OR, XOR, etc.). On the other hand, once all the bids are received, the winner determination problem (analogous to the clearing of the market) requires solving Mixed Integer optimization problems which belongs to the family of NP-Hard problems.

Related Work on Combinatorial Auctions for Local Energy Markets

Most of the recent treatment of Local Energy Markets focuses on sequential implementations where, every 15 to 30 minutes, players get to trade energy for the next time-slot [START_REF] Mengelkamp | Trading on local energy markets: A comparison of market designs and bidding strategies[END_REF] [108], [START_REF] Weinhardt | How far along are local energy markets in the DACH + region[END_REF], [START_REF] Guerrero | Decentralized p2p energy trading under network constraints in a low-voltage network[END_REF]. These implementations rely on double-sided auctions and peer-to-peer schemes to implement such markets.

The idea of implementing LEMs as combinatorial auctions is not new. For a survey on some of the methods that have been proposed, the reader is referred to [START_REF] Lopez-Rodriguez | Methods for the management of distributed electricity networks using software agents and market mechanisms: A survey[END_REF]. We briefly discuss two of those herein.

In [START_REF] Penya | Combinatorial markets for efficient energy management[END_REF], the authors propose the use of parallel reverse combinatorial auctions to implement LEMs. They explicitly avoid the use of exchanges (double-sided auctions), as proposed in the present chapter, because of the complexity that they entail. In their design, players can submit bids for single time-slots and correlation functions between time-slots that specify a change in the desired price of the correlated items if they were to be acquired together.

Carlsson and Andersson in [START_REF] Carlsson | A flexible model for tree-structured multi-commodity markets[END_REF] propose a double sided combinatorial auction using a tree-structured market. Their proposal allows for substitute buy and sell bids where players are indifferent regarding when they buy (or sell) a certain quantity. This is very similar to the approach of this chapter, with the difference that our mechanisms allow ramp constraints to be incorporated in the bids. Other applications of combinatorial auctions to local energy trading include the use of combinatorial auctions to divide the usage of a shared battery among participants [START_REF] Zhong | Multi-resource allocation of shared energy storage: A distributed combinatorial auction approach[END_REF], [START_REF] Bizzat Hussain Zaidi | Combinatorial auctions for energy storage sharing amongst the households[END_REF].

Finally, as we have already mentioned, the European day ahead market is by itself a combinatorial auction. A public description of the algorithm designed by N-SIDE can be found in [START_REF] Pxs | Euphemia public description. pcr market coupling algorithm[END_REF].

Proposed System and Mathematical Model

We put forward the design of a combinatorial double auction that exploits the flexibility available for players. Unlike the traditional auctions used for LEMs in which players bid the quantity they want to buy or sell for a single time-slot, we allow players express in their bids their desire to acquire specific profiles of energy spanning multiple periods. We proceed to explain the bidding format, the allocation problem formulation and the pricing rules.

Bidding format and allocation rule

In the proposed auction, each player expresses all her acceptable trading profiles and her utility associated with each one of them. To do so, each player bids a feasible set of actions X i (this can be done by bidding the battery capacity, initial state of charge and other battery characteristics) and her utility function ûi such as the one defined Chapter 4 (for which it suffices to include the prices for buying β i t and selling γ i t energy to the TEC). Here, we use the ĥ notation to emphasize that the bid needs not to be truthful. From the bids, we can obtain pi , the cost that player i can guarantee without trading in the local market, according to her reported information.

Regarding the allocation rule, it will be derived from the optimal solution of optimization problem (7.1a)-(7.1e). As the objective function of the allocation problem, we decided to use Equation (7.1a), which maximizes the value of all the local trades. The value of the local trades is defined to be the same as the price that players should had paid the TEC to buy (sell) the same amount of energy. That way, the clearing algorithm maximizes the amount of profit that can be distributed among the market participants. This is analogous to finding the clearing price in a double auction such as [START_REF] Huang | Design of a multi-unit double auction e-market[END_REF].

The optimization problem that follows uses three different groups of variables. The z variables represent the net load of each player and actually depend on the usage of the battery (x). The λ i group is a positive scaling factor that represents the proportional amount of net load of player i that gets traded in the market (the rest will have to be traded with the TEC). Finally, the P i variable denotes the payment of player i into the market and negative represents profit.

max z i ,λ i ,P i i∈N i∈T C i t λ i t z i t (7.1a)
subject to:

i∈N P i ≥ 0 (7.1b) P i + t∈T C i t 1 -λ i t z i t ≤ pi ∀i ∈ N (7.1c) i∈N λ i t z i t = 0 ∀t ∈ T (7.1d) z i ∈ X + l ∀i ∈ N (7.1e) λ i t ≥ 0 ∀i ∈ N , ∀t ∈ T (7.1f) (7.1g)
The first constraint (7.1b) ensures that if the equality holds, all the money is redistributed among the participants according to the market decisions, while if the inequality is strict, the market maker obtains a profit. Constraint (7.1c) guarantees that the auction is individually rational, i.e., each players is at least as good as if she had not participated in the local market. Observe that the cost of a participant in has two components: how much they pay in the market (P i ) and how much of their net load is not traded in the market and therefore is traded with the TEC. In this case, this is given by t∈T C i t 1 -λ i t z i t . Finally, the sum of both quantities has to be lower than their expected costs.

It's important to note that encoding all of the N constraints (7.1c) requires a total of 2N T additional binary variables. Equalities (7.1d) ensure that the amount of sold energy is equal to the energy bought in every time-slot. The last constraint guarantees that only feasible net consumption profiles are used. Finally, the amount of energy traded by player i at time-slot t is given by z i t λ i * , where λ i * and z i t are the optimal solutions of optimization problem (7.1a).

To provide insight into why constraints 7.1c require binary constraints we can proceed as follows. We introduce auxiliary variables

A i t = max{1 -λ i t z i t , 0} and
B i t = max{-1 + λ i t z i t , 0}
. So that we can re-write the constraint as:

P i + t∈T β i t A i t -γ i t B i t ≤ αi .
We are left only with providing the new constraints for A and B which we can do as follows

A i t ≥ 1 -λ i t z i t A i t ≥ 0 M b i,A t + 1 -λ i t z i t ≥ A i t M (1 -b i,A t ) ≥ A i t b i,A t ∈ {0, 1}
.

where M is an upper bound on 1 -λ i t z i t . Analogous for B. The resulting problem, mixed integer linear, is particularly hard to solve.

Before moving forward, we shall provide a short explanation on why this model is combinatorial. Indeed, it is not evidently clear where the "combinatorial" part comes from. For this, observe that instead of bidding in each time-slot for a specific amount of energy, players are expressing their preferences over consumption profiles that span the whole time-horizon. Furthermore, unlike the sequential case where bids are "ANDs" (a player wants to buy a certain amount on time-slot 1 AND a certain amount on time-slot 2, etc), players are expressing "XOR" preferences about the multiple consumption profiles -I am willing to consume following this profile at this cost (exclusive) OR that other profile at a different cost, etc.

Payment rule

As a payment rule, one alternative is to use the value of P i * in the optimal solution of (7.1a). For the cases in which the values of P i * will not be unique, a predefined rule can be used to choose among the possible values. One such rule could be to select the values of P i * that maximize a given fairness criterion.

We proceed to illustrate our proposal with an example.

A simple example

Let T = {1, 2, 3, 4} and consider only two players 1 and 2 such that: x 1 = (0, 0, -1, 0), x 2 = (0, 0, 0, 1),

β 1 = β 2 = (2, 2, 3, 3), γ 1 = γ 2 = (1, 1, 1, 1), S 1 = S 2 = 1, S 1 0 = S 2 0 = 0.
If player 1 does not trade in the market, she will sell all her energy at price 1, for a total utility of α 1 = 1, net consumption profile n 1 = (0, 0, -1, 0) and no need to user her battery s 1 = (0, 0, 0, 0). Analogously, player's 2 utility is -2 as she charges her battery during the first time-slot and discharges it in the last one (s 2 = (1, 0, 0, -1)) to obtain a net consumption profile n 2 = (1, 0, 0, 0).

We will now assume that the two players decide to participate in the auction and they do so truthfully. In the optimal solution of the allocation problem defined by their bids, it holds that n 1 = (0, 0, -1, 0) = -n 2 , λ 1 = λ 2 = (0, 0, 1, 0). Furthermore, the maximum value is attained at:

3 × (1) + 1 × (-1) = 2
Regarding the payments, we have that for player 1: P 1 * ≤ -1 and for player 2: P 2 * ≤ 2. Consequently, any payment from player 2 to player 1 in the interval P 2 * ∈ (1, 2) will leave both players better off than before.

We could also envision the same players taking part in a sequential auction as described in Part II of this thesis. In that case, player 1 cannot change her actions because she cannot change when she will be producing. On the other hand, player 2 can. If we imagine that he has a belief about market prices such as the ones that have already been described, she might believe that the trading price in the market is always 20% better than the TEC. If that is the case, she might not buy the unit she needs in the first time-slot because she believes she has a chance of buying it cheaper in time-slot 2. In spite of this, because the buying price in the market at time-slot 3 is still 3 × 0.8 = 2.4 > 2 higher than the TEC during the cheaper period, she will trade with the TEC during time-slot 2 instead of 1, but not improving the local trades.

Some Properties of the Proposed Model

First, observe that in (7.1a), the scenario without trades (P i t = λ i t = 0, ∀i ∈ N , ∀t ∈ T ) is always feasible and therefore, a solution always exists. This solution needs not to be unique, as discussed in subsection 3.2. Secondly, when all players bid truthfully, the proposed auction obtains the consumption and trading profiles that maximize the value of the trades. The obtained allocation can outperform the results obtained when players maximize their individually utility and attempt to trade later using sequential auctions. An example of this was given in the previous subsection. There, the total utility of players went from -1, had they tried to trade in sequential auctions using the net profiles that maximized their individual utilities, to 0 by trading in the proposed auction.

Interpreting Combinatorial Auctions as Local Energy Trading

Interpreting the proposed combinatorial auction in terms of the general framework proposed in Chapter 2 is quite simple.

First, recall that the partial consumption profile of a player was defined as the fraction of her consumption that gets traded locally. By definition (we defined it that way in the Sections above), λ i corresponds to the fraction of the energy that gets traded in the market. Hence, by definition, λ i z i is the partial consumption profile of player i as determined by this market clearing algorithm.

Furthermore, when looking at the winner determination problem, constraint 7.1c ensures that the price paid by each player (left hand side) is no greater than what they can guarantee by themselves (right hand side of the constraint). The left hand side is composed of two terms: the payment of the market P i and a second term that equals to the amount of energy that the participant could not trade in the market. These two terms are exactly the fraction of the total costs paid by player i and therefore constitute, her distribution of the costs.

Key Outcomes and Conclusions

Looking at the proposed model, it is not difficult to spot problems in its design, after all, it was more a concept model than one ready for production.

First of all, the winner determination problem is computationally expensive. Secondly, the amount of information that players need to reveal is the same as in a centralized implementation, which is quite high! Finally, there is no guarantee that the proposed model is strategy-proof, which can result in very bad solutions if the participants of the market do not report their true preferences (and they have no incentive to do so).

In the next chapter we explore a variation of the proposed mechanism that is more efficient (from a computational point of view) and for which we can prove some results regarding truthfulness.

Chapter8

COMBFLEX: A Tailored Combinatorial Auction for Trading Energy Flexibility Arising from Energy Storage

Improving on the Previously Proposed Combinatorial Auction

In the previous chapter we presented a first attempt at designing a local energy market based on combinatorial auctions. We concluded the chapter by pinpointing several areas of possible improvement in the proposed model. The goal of this chapter is to show how we can exploit the domain knowledge that we have about prosumers and their decision process when they own a battery to design an efficient bidding language and winner determination problem. That is, we manage to partially overcome several of the difficulties in the previous model by designing a market tailored to local energy trading among prosumers with batteries.

At the end of this chapter, we will compare the new designed mechanism with the previously presented sequential local energy markets by means of numerical simulations.

We show that the proposed mechanism cannot be "obviously manipulable" (a relaxation of strategy-proofness) and that the winner determination problem is of polynomial complexity.

A strategy-proof mechanism is one in which telling the truth (revealing the true type of each player) is a weakly dominant strategy [START_REF] Nisan | Algorithmic Game Theory[END_REF]. This is a desirable property as players do not need to employ complicated strategies and try to "game" the market. Among all mechanisms that are not strategy-proof, some are easier (obvious) to manipulate than others. A mechanism is obviously manipulable if there exists a manipulation (a strategy) θ whose best case or worst case is strictly better than telling the truth θ. A mechanism that has no obvious manipulations is said to be Non Obviously Manipulable (NOM). [START_REF] Troyan | Obvious manipulations[END_REF].

Indifference Regions: the Source of Flexibility

Most of the time, the solution to the linear programming problem that outputs the optimal schedule of the battery is not unique. In that case, we can derive, from the solution obtained (i.e., the plan), a set of solutions among which the player is indifferent. These sets will define the flexibility regions that the player can bid in the market. We assume that players have already solved their optimization problem using (A.12) with respect to the TEC prices and have found their respective plans (one of the optimal schedules).

Indifference regions for buying bids

Consider the load profile (blue curve) in Figure 8.1, an optimal battery trajectory (black curve) obtained by solving the appropriate LP, i.e., the plan ∆ and the corresponding net load in red. Negative consumption represents surplus of generation while a positive value of the battery curve stands for charging. The player owns a battery with charging and discharging efficiencies η c = η c = √ 0.8. The optimal strategy is to charge the battery during the "cheap" period and discharge it during the expensive one. The load in the expensive period is 4 units, so there needs to be 4 Consequently:

1 ∆ = (0, √ 0.8, √ 0.8, √ 0.8, √ 0.8, √ 0.8, -2 √ 0.8 , -2 √ 0.8 ).
We may observe that the charging during time-slots 2 to 4, could have been done also in time-slot 1. Furthermore, one time-slot could have charged more than others. With the above, we can define the net load indifference region Z B for buying as:

Z B = (z 1 , z 2 , z 3 , z 4 , 0, 0, 0, 0) : 4 1 z i = 3, z i ∈ [0, δ η c ] .
In the traditional literature of sequential LEMs, the player will offer to buy 1 unit in the market associated with time-slot 2 and 1 unit in the market associated with time-slot 3 and 1 unit for time-slot 4. In the proposed market, the player can submit the following bid b = (1, 4, 3, δ ηc , C 1 ), or more generally b = (t s , t e , Q, δ η c , β) (8.1)

In the equation above, the first two coordinates indicate the first and last time-slot (inclusive) of the interval for which the player is bidding. The third coordinate represents how much energy the player wishes to buy, the forth coordinate how much energy she is willing to buy per time-slot (no more than she can store, due to the ramp constraints) and finally, β is the maximum price she is willing to pay per unit.

Indifference regions for selling bids

In this subsection we describe an analogous scenario as the presented in the subsection above expect that, in this case, the player wishes to sell energy.

As it is the case with most feed-in-tariffs, we assume a flat rate structure. In this example, the battery's efficiencies are given by η c = η d = √ 0.8 but, unlike the previous case, where we did not care about the value of the ramp constraints, we have that δ = δ = 1.

The plan ∆, defined in Figure 8.2 is given by: ∆ 1

= (1, √ 0.8, 1, √ 0.8, 2(1 - √ 0.8), -1, -1, -1, -1)
The plan needs to add up to -S i 0 , as leaving energy in the battery at the end of the horizon is not optimal.

The battery cannot discharge more than 4 units during time-slots 6 to 9, which because of efficiency can only cover 4 √ 0.8 units of load. As a result, the player needs to acquire extra load during those time-slots, regardless of the battery. During the first time-slots, the objective is to store the 4 units required. To do so, the player will have to keep 4 √ 0.8 of her generated units. During time-slots 2 and 4, the battery charges at its maximum capacity and the remaining surplus has to be sold. During time-slots 3 and 5, the battery charges all generated energy without reaching its ramp constraint. Finally, during time-slot 6, the battery is underused and there is still surplus. The battery could have been charged more in time-slot 6 instead of one of the other time-slots.

We proceed to describe a set that contains the alternatives of the player that result in the same cost. Denote by z i t the amount offered by the player in the market. Clearly, z i t ≤ max{-l i t , 0} as the player cannot sell energy that she does not have. The energy kept is the energy not sold and is represented by l i t -z i t . Because of ramp constraints, the energy kept must be smaller than the maximum charging power taking the efficiency into account:

l i t -z i t ≤ δ ηc .
Finally, the energy kept has to be at least the amount of energy needed te t=ts l i t -z i t ≥ K, where K is the total amount of energy that need to be kept. In this example, K = 4 √ 0.8 . Putting it all together, the set of Z S of indifferent alternatives while selling can be described as:

Z S = z ∈ R T1 : te k=ts l i k -z k ≥ K, z k ∈ [ l i k - δ η c + , l i k ] .
with T 1 = es + 1. For this particular example, we have the following:

Z S = { 6 i=2 z i ≥ 4 √ 0.8 , z 3 , z 5 , z 6 ∈ [0, 1], z 2 ∈ [2 - 1 √ 0.8 , 2], z 4 ∈ [3 - 1 √ 0.8 , 3]}.
A player that wishes to participate in the market and sell, can submit a bid expressed as the indifference set Z S together with the maximum price she is willing to pay for each unit. The summarized selling bid can be described as: b = (t s , t e , K, l s , l s+1 , . . . , l e , δ η c , γ)

(8.2)
where γ is the minimum price per unit that the seller is willing to accept.

Mathematical Model of the Proposed Mechanism

In this section we will explain the market mechanism proposed in this chapter, which we shall call Combflex.

There are 4 key ingredients to the mechanism: the participants, described in Chapter 4, the bid format, the winner determination problem (WDP) and the payment rules.

Bid format

The bid format consists of the union of indifference sets for buying and selling. Under this definition, an arbitrary bid B can be defined as: B = ((Z 1 , p 1 ), . . . , (Z m , p m )), where Z j is an indifference region for buying or selling and p j is the associated reservation price per kWh with Z j . To submit the indifference regions, players can submit the corresponding summaries for buying and selling as described in Equations (8.1) and (8.2).

A bid B will be considered valid as long as the set of variables involved in each indifference region V (Z j ) are pairwise-disjoint. In other words, there is no overlap in time between blocks in the same bid.

Winner Determination Problem

The winner determination problem proposed in this chapter is an optimization problem that maximizes the value of the local trades, as in the previous chapter.

For each player i, let B i be the set of time-slots in which player i is buying and S i the set of time-slots in which player i is selling. We will omit variables in time-slots for which the player is not buying nor selling. With the above notation, the variable z i b , b ∈ B i represents the amount of energy that player i is buying at time-slot b while z i s represents the amount of energy that player i is selling during time-slot s.

The notation z i s ∈ B i (or equivalently z i b ∈ B i ) will denote that the variable is within the appropriate indifference set and satisfies the corresponding constraints. Observe that this can be done wlog. as each variable belongs to only one indifference set within B i .

Finally, observe that, even though a player might need to buy or sell a quantity at a given time-slot (as it is the case in the example for selling, where

z 2 ∈ [2 -1 √ 0.8 , 2]
), that quantity needs not be necessarily traded in the market (it can be settled with the TEC). To model such behavior in the market, for each variable z there will be a variable w such that 0 ≤ w ≤ z. The role of w in the WDP in this chapter is analogous to the role of λ in the previous chapter. By doing so, variables z will represent the point of consumption within the indifference region, while variables w will represent how much of that consumption (or surplus) gets traded in the market. We will make use of the convention that z i t = 0 and consequently w i t = 0 whenever t / ∈ B i ∪ S i . With the above conventions, the optimization problem that defines the WDP is given by Equation (8.3):

min z,w t∈T i∈N w i t p i t 1 t∈B i -w i t p i t 1 t∈S i (8.3)
subject to: (8.4)

z i b ∈ B i , ∀b ∈ B i , ∀i ∈ N (8.5) z i s ∈ B i , ∀s ∈ S i , ∀i ∈ N (8.6) i∈N w i t 1 t∈B i -w i t 1 t∈S i = 0 ∀t ∈ T (8.7) 0 ≤ w i t ≤ z i t , ∀i ∈ N , ∀t ∈ T (8.8)
where 1 • is the indicator function.

As mentioned above, the objective of optimization problem (8.3), is to maximize the value of the local trades. The first constraint guarantees that the variables involving buying in the market are constrained as defined in the buying indifference sets. Similarly, the second constraint guarantees that the selling variables are properly defined within the bid. The third group of constraints guarantees that the amount of energy bought and sold in every time-slot is the same. Finally, the last constraint guarantees that no player trades in the market above the selected consumption level.

Proposition 8.1. The WDP has a polynomial complexity in T as well as in N .

Proof. In optimization problem (8.3), the objective function and the constraints are linear (the indifference sets are defined as the intersection of half-spaces), so the problem can be written as a linear program (LP). The resulting problem scales linearly in the number of players and time-slots and it is known that LP in is P.

Payment Rule

The payment rule defines how much each player gets for each trade in the market.

For each solution of WDP, we will show that we can determine a price for buying and selling for each time-slot up to two parameters λ l and λ h . We refer to these as the clearing prices. Consequently each player will pay all the traded quantities at the clearing prices.

First, we shall show that such prices exist.

Theorem 8.1. In an optimal solution of the optimization problem (8.3), it holds that in every time-slot t, the maximum price asked by all players selling in that time-slot (w > 0) is smaller or equal than the lowest price offered by all participants that bought in that time-slot (w > 0). In other words, in every time-slot t, there is a nonempty interval of prices [p low t , p high t ] such that every participant that is trading in that time-slot (according to the optimal solution) is satisfied with it.

Proof. By way of contradiction. First, we show that the solution can be improved by removing the trades for which the theorem does not hold. Later, we show that the new solution is still feasible.

Suppose that there is a time-slot t in which the theorem does not hold and let w * be the optimal solution. There are some w * s for buying and some for selling (same quantity) for which the price of buying is lower than selling. Observe that the objective function can be improved by removing those quantities. We consider a new solution w by removing those quantities of w * . Because we did not change the values of z * and we only decreased w, it holds that w ≤ z * , so the solution is still feasible. This is a contradiction because we assumed that the solution was optimal.

We can change the mechanism by changing which value in the interval is used as the price. Furthermore, we could allow for a different price for buying than for selling. By doing so, the leftover money would go to the market maker, who might have to cover operational costs for running the market (it would be a weakly budget balanced mechanism).

Mechanism Variant Splitting the Market into Two

We have shown that we can define a price (or two) in every time-slot. With that in mind, we might envision a procedure to reduce the ability of participants to cheat (or game the market). Borrowing ideas from MUDA [START_REF] Segal-Halevi | MUDA: A truthful multi-unit double-auction mechanism[END_REF], we can split all the bids into two different markets, namely "left" and "right". Each market clears independently and we use the prices obtained in the other half, i.e., the right market uses the prices of the left one and vice versa.

We will call this variant of the Combflex mechanism Combflex Split. By splitting the market, players cannot influence their trading price at all. The proposed procedure comes at a cost, namely efficiency: when forcing players to trade at the clearing prices of another market, some trades are bound to be lost.

Furthermore, we can envision a market that clears by splitting into two markets with a probability p, and runs the efficient version with probability 1-p. This could incentivize participants to tell the truth without compromising efficiency in the long run.

Properties of the Proposed Mechanism

In this section we prove some properties about the mechanism introduced in the previous section. Theorem 8.2. In Combflex, reporting a smaller quantity (for a seller) or price (for a buyer) than desired can be profitable.

Proof. A seller offering a smaller quantity can shift the supply curve to the left, increasing the price. She might benefit overall from selling a smaller quantity at a higher price. The buyer whose bid intersects the supply curve can influence the trading price. By offering a lower buying price, he can reduce the clearing price of the time-slot at a profit. Theorem 8.3. In Combflex split a player cannot misreport her preferences to change the market prices.

Proof. This follows directly from the fact that all prices are determined exogenously.

Theorem 8.4. The variant of the mechanism that splits players into two groups is not obviously manipulable.

Proof. A strategy in which a buyer offers a smaller price and less quantity is dominated by telling the truth in the mechanism that splits players. Therefore, a profitable manipulation must offer at some time-slot more energy than desired or at a higher reservation price. For those strategies, there is a profile of the other players actions in which only the misreported quantity/price trades, and the player is worse off. The same holds for a seller.

So far, we have proposed a new market mechanism to implement local energy markets. In the previous sections we established some of its properties. To complement all of our previous results, in this section we compare the performance of the combinatorial auction presented in this chapter with traditional mechanisms used in the literature to implement LEMs, using numerical simulations.

Experiment Setup

For the experiments reported in this section, we considered an environment composed of 50 players. Each player was equipped with a battery with a maximum capacity of 13 kWh, charging and discharging efficiencies of 0.95 and a maximum charging and discharging ramp rate of 5 kW.

For the load profiles, we used data sampled every 30 minutes, which yielded 48 time-slots in a day. Real consumption data was obtained from the Ausgrid project [START_REF] Ratnam | Residential load and rooftop PV generation: an australian distribution network dataset[END_REF]. The profiles contain a small amount of renewable surplus, but not sufficient to justify a local energy market. For this reason, we augmented half of the profiles with additional renewable energy. To do so, we sampled a uniform random variable R i t ∼ U[-0.3, 0] i.i.d. for the time-slots when the sun should shine. The sampled variables were added to the player's profiles.

We considered two electricity tariffs: a flat rate and a Time-of-Use with two steps. The price of the flat rate was 14, while the ToU had prices: 12 (during the first half of the day) and 16 (during the second half). Both tariffs offered a constant Feed-in-Tariff at 10. All prices are in cents per kilo watt hour.

Half of the users with extra generation were subscribed to the flat rate and the other half to the ToU. The same was true for the players without additional generation.

We evaluated 7 different mechanisms: 4 obtained as variants of the mechanism presented in this chapter, and 3 other market algorithms often encountered in the literature and previously discussed in the context of this thesis.

The 4 variants of the proposed mechanism considered were: Combflex 1-0, Combflex .5-.5, Combflex S 1-0 and Combflex S .5-.5. We abbreviate Combflex split by Combflex S Finally, in the 1 -0 markets, buyers and sellers each pay the clearing price, with all the profit in the gap going to the market maker. In contrast, the .5 -.5 algorithms use the mean between the buying and selling clearing prices and all the profit stays with the participants.

The external market algorithms used as benchmark are: Auction M, Auction H, P2P. Auction M, stands for the strategy-proof auction MUDA [START_REF] Segal-Halevi | MUDA: A truthful multi-unit double-auction mechanism[END_REF]. Auction H, stands for the double auction proposed by Huang et al [START_REF] Huang | Design of a multi-unit double auction e-market[END_REF]. Finally, P2P stands for a simple competitive peer-2-peer trading algorithm. In it, players are matched randomly and trade if the buying price offered is higher than the asked price for selling. The mechanisms continues randomly matching all remaining players with tradeable quantities until no more trades are available or all players have been matched together.

The markets described above are used to trade energy for single time-slots, unlike the auction presented in this chapter where all the time-slots are traded at the same time. Unlike the mechanism introduced in this chapter, the three benchmarks were ran sequentially, with players trading only for the next time-slot. To stimulate the interaction of players in the market, we assume that players have a belief about market prices being more competitive during hours when the sun shines (as there is extra surplus). This is the same as the solar belief described in Part II of this thesis. This was achieved by changing the tariffs of players to include a markup of 10% during the corresponding time-slots. This implies that the buying price was 10% cheaper and the selling price was 10% higher. The implementations of MUDA, Huang et al., and the P2P algorithm were ran using the PyMarket library [START_REF] Kiedanski | Pymarket -a simple library for simulating markets in python[END_REF]. To solve the numerous optimization problems, in the simulations we used CPLEX and Pulp. Furthermore, simulations were ran in parallel using GNU parallel [START_REF] Tange | Gnu parallel-the command-line power tool[END_REF]. We simulated 100 different days. In each of them, all the mechanisms ran under the same conditions.

Results

We proceed to explore the numerical results obtained. Two metrics are of particular interest: the aggregated social cost and the total non-traded energy. The social cost is simply the sum of the cost of all players. The total nontraded energy is the absolute value (energy injected in the grid is negative) of the energy that could not be traded locally, and had to be consumed or injected from the main grid. It is a metric that describes the capability of a market to incentivize local trades.

For each simulated day, we obtained the desired metric when running each of the market algorithms as well as when players optimized their batteries individually, without a market.

In Figures 8.3 and 8.4 we plot the Social Cost and the Untraded Energy, respectively. Instead of showing the value of the metric, for each day we took the ratio of the corresponding metric when running the market divided by the metric when no market was in place. A lower value of both metrics is desirable and consequently, a ratio lower than 1 indicates a reduction with respect to the case without market.

Regarding the social cost, we observe that, when Combflex uses the middle price (.5 -.5), it outperforms traditional auction mechanisms, but the (1 -0) variant does not. Finally, all variants of Combflex result in a higher amount of locally traded energy than their traditional counterparts.

Key Outcomes and Conclusions

In this chapter, we presented a market mechanism for buying and selling energy among end-customers. The bidding format was designed to capture the natural flexibility available to players that own energy storage and photo-voltaic panels.

Even though the market introduced in this chapter is not strategy-proof, we have established that it is not obviously manipulable. The mechanism was evaluated using numerical simulations with realistic data. In these experiments, the proposed market outperformed traditional LEM implementations such as double auctions and peer to peer exchanges. We envision two possible directions for future work. First, to obtain analytical guarantees on the efficiency of the mechanism. Finally, to incorporate new bids into the mechanism such as allowing players to offer unused battery capacity.

Part IV

Energy cooperatives for Local Energy Trading

Chapter9

Preliminaries of Cooperative Game Theory

So far, we have studied Local Energy Trading implemented through non-cooperative games, namely auctions. The reader might call that in the introduction, we discussed not only competitive mechanisms, but also cooperative ones (such as solar sharing) as possible tools to ease the energy transition and the incorporation of more renewable energy resources into the power grid. One of the main ideas behind energy sharing and cooperation in general is that players will jointly maximize their local matching behind a net meter and then decide how to divide the total costs incurred, which are expected to be less than if acting independently. The natural framework to study such agreements is through the lens of cooperative game theory. In this chapter we will introduce basic notions of cooperative games that will be used throughout this part.

Standard Definitions in Cooperative Game Theory

A cooperative N -person game with transferable utility G = (N , v) is defined by a characteristic cost function v : 2 N → R that assigns to each subset S (coalition) of N = {1, 2, . . . , N } the cost incurred by players in S, v(∅) = 0. We specify that the game is of transferable utility to denote that players deal with goods (in this case money) that can be transfered from one player to the another within the same coalition S. This is in contrast to games where players have ordered preferences, and it is not obvious that those can be exchanged among the players. In the same way that we defined a characteristic cost function, a characteristic value function assigns to each coalition S the joint value generated by the players in it. Every characteristic cost function can be written as a value function by changing its sign (-v). Because of this, we will not make a lot of emphasis in if we are dealing with one or the other, as it should be clear from context. The set of all players N is referred to as the grand coalition.

The main goal in a cooperative game is to find a Payoff Vector (PV) x ∈ R N that specifies how much a player should pay (or earn, for value games), and that exhibits some desirable properties.

For example, a payoff vector x is said to be:

• Efficient if i∈N x i = v(N ). • Individually Rational (IR) if x i ≤ v({i}), ∀i ∈ N (x i ≥ v({i}) for value games). • Group Rational if i∈S x i ≤ v(S), ∀S ⊂ N . (≥ for value games). • Symmetric if v(S ∪ {i}) = v(S ∪ {j}), ∀S ⊂ N -{i, j} ⇒ x i = x j . Definition 9.
1. An imputation is a payoff vector that is efficient and individually rational.

Definition 9.2. An imputation x dominates another imputation y (denoted as: x > y) if there exists a subset of players S such that x i < y i , ∀i ∈ S (> for value games) and v(S) ≤ i∈S x i (≥, corresp.).

The interpretation of a dominated payoff is that a subset of players can be better of in the not-dominated (x), and they have a reason to demand a better outcome (they can still do better while looking at v(S)).

Definition 9.3. The core of the game is the set of all PVs that are efficient, individually rational and group rational. Equivalently: the core is the set of all imputations that are not dominated.

Remark 9.1. The core of a game can be empty.

Example 9.1. Consider a simple example with 3 players. Players 1 and 2 can produce a left shoe, while player 3 can produce a right shoe. This is a traditional example in the literature. The value of a single shoe of the same type is 0, nobody wants only left or only right shoes. The value of a pair of shoes is 1. From this, we can write down the characteristic value function of the game:

v({1}) = 0 v({2}) = 0 v({3}) = 0 v({1, 2}) = 0 v({1, 3}) = 1 v({2, 3}) = 1 v({1, 2, 3}) = 1 (9.1)
The result of such a game is a payoff vector x that specifies the gain of each player. If we were to try and find the core of the game, we will look for a vector x = (x 1 , x 2 , x 3 ) such that:

x 1 + x 2 + x 3 = v({1, 2, 3}) = 1 x 1 + x 2 ≥ v({1, 2}) = 0 x 1 + x 3 ≥ v({1, 3}) = 1 x 2 + x 3 ≥ v({2, 3}) = 1 x 1 ≥ v({1}) = 0 x 2 ≥ v({2}) = 0 x 3 ≥ v({3}) = 0 (9.2)
It is not difficult to see that the only solution to the above problem is given by x = (0, 0, 1). This tell us that the only stable solution of the game (in the core) is the one assigns all the value to player 3. Player 3 holds all the bargaining power! Definition 9.4. Shapley value. The Shapley value φ(v) ∈ R N is a PV satisfying:

φ i (v) = S⊆N -{i} |S|!(N -|S| -1)! N ! (v(S ∪ {i}) -v(S)) (9.
3) Definition 9.5. A game is sub-additive (super-additive for value games) if v(S∪V ) ≤ v(S)+v(V ), S∩V = ∅ (≥, corresp.).

Remark 9.2. An important characteristic of sub-additive of super-additive games is that from all possible partitions of players into different sub-coalitions, it is always optimal to join the Grand Coalition (two separate coalitions cannot do worse by joining). This simplifies the study of such games, since we only need to worry about the stability of the grand coalition. All the games studied in this part happen to satisfy this property.

Definition 9.6. A game is said to be concave (convex for value games) if its characteristic cost function is sub-modular (super-modular, corresp.), that is: v(S ∪ T ) + v(S ∩ T ) ≤ v(S) + v(T ), ∀S, T ⊆ N (≥, corresp.) Remark 9.3. It can be shown that the above condition is equivalent to:

v(S ∪ {i}) -v(S) ≤ v(T ∪ {i}) -v(T ), ∀i ∈ N , T ⊆ S ⊆ N \ {i}
for cost games and:

v(S ∪ {i}) -v(S) ≥ v(T ∪ {i}) -v(T ), ∀i ∈ N , T ⊆ S ⊆ N \ {i}
for value games [START_REF] Shapley | Cores of convex games[END_REF].

Remark 9.4. The Shapley value is unique, efficient, symmetric and linear. When the game is sub-additive (super-additive for value games), the Shapley value is an imputation. When the cost (value) function is concave (convex), the Shapley value is in the core [START_REF] Shapley | Cores of convex games[END_REF].

Example 9.2. We will calculate the shapely value for the game described above.

φ 1 (v) = 1 6 S∈{∅,{2},{3},{2,3}} |S|!(N -|S| -1)! (v(S ∪ {i}) -v(S)) = 1 6 [0!2!0 + 1!1!0 + 1!1!1 + 2!0!0] = 1 6 φ 2 (v) = 1 6 S∈{∅,{1},{3},{1,3}} |S|!(N -|S| -1)! (v(S ∪ {i}) -v(S)) = 1 6 [0!2!0 + 1!1!0 + 1!1!1 + 2!0!0] = 1 6 φ 2 (v) = 1 6 S∈{∅,{1},{2},{1,2}} |S|!(N -|S| -1)! (v(S ∪ {i}) -v(S)) = 1 6 [0!2!0 + 1!1!1 + 1!1!1 + 2!0!1] = 4 6
From this we can see that the Shapley value is more fair than the solution in the core. We can also observe that this game is not convex, since that would imply that the Shapley value is in the core. We can verify that:

=1 v({1, 3} ∪ {2, 3}) + =0 v({1, 3} ∩ {2, 3}) < =1 v({1, 3}) + =1 v({2, 3}) .
which should not happen for value games.

Definition 9.7. A game is Essential if N i=1 v({i}) > v(N ) (< for value games) and Inessential if N i=1 v({i}) = v(N ).
For all inessential games, the core is not empty and consists solely of the (trivial) payoff vector x = (v{1}, v{2}, . . . , v{N }) [START_REF] Lucas | Chapter 17 von neumann-morgenstern stable sets[END_REF]. As a consequence, cooperative game theory focuses on essential games. Definition 9.8. A characteristic function game is said to be (0, 1) -normalized if v({i}) = 0, ∀i ∈ N and v(N ) = 1. Remark 9.5. All essential games with a characteristic value function v can be transformed into an equivalent (0, 1) -normalized game with characteristic value function v [START_REF] Lucas | Chapter 17 von neumann-morgenstern stable sets[END_REF] where:

v(S) = v(S) -i∈S v({i}) v(N ) - N i=1 v({i}) (9.4)
There are many other solutions concepts that have been introduced in addition to the core and the Shapley value. We proceed to present two more. Definition 9.9. A set of imputations V is a Von Neumman and Morgenstern solution of a game if:

• No imputation in V dominates another imputation in V (internal stability).

• If y / ∈ V is an imputation, then there exists x ∈ V such that x dominates y. (external stability).

The set V is also referred to as a stable set [START_REF] Rapoport | N-person game theory: concepts and applications[END_REF].

Definition 9.10. The Nash Bargaining Solution (NBS) is the (only) payoff vector that complies with all the following:

• Invariant to affine transformations

• Pareto optimal • Symmetric
• Independent of irrelevant alternatives

• Individually Rational

For a complete definition of the above properties, the reader is referred to [START_REF] Rausser | The Nash Solution to the Bargaining Problem[END_REF]. If all players are risk-neutral, then their utilities can be modeled as linear functions of their costs. Denote by d i the disagreement point of player i, that is, what player i expects to pay (or gain) if negotiations break down. If the payoff space P is compact and convex, then the NBS is the only solution to the optimization problem defined in (9.5):

max x∈P N i=1 (x i -d i ) , s.t. x i ≥ d i (9.5)
Before moving into the next section, we need to present one more result of great importance regarding the core of a cooperative game.

Deciding whether the core is empty is in general an NP-complete problem [START_REF] Deng | On the complexity of cooperative solution concepts[END_REF]. A general tool to check if the core is empty or not is due to Bondareva and is presented in Theorem 9.1. Theorem 9.1. (Bondareva) A cooperative value game G = (N , v) has a nonempty core if and only if for every function α : 2 N \ {∅} → [0, 1] that satisfies Equation (9.6) also satisfies the condition stated in Equation (9.7) [START_REF] Shapley | On balanced sets and cores[END_REF]. ∀i ∈ N :

S⊆2 N \∅|i∈S α(S) = 1 (9.6) S∈2 N \∅ α(S)v(S) ≤ v(N ) (9.7)
Definition 9.11. A game cooperative game for which equations (9.6) and (9.7) hold, is said to be balanced. Furthermore, if very subgame G ⊂ G is balanced (a subgame is obtained by restricting the number of players), then the original game is said to be totally balanced.

A more detailed introduction to cooperative game theory can be found in [START_REF] Rapoport | N-person game theory: concepts and applications[END_REF].

Linear Programming Games

A particularly interesting family of cooperative games that will be used extensively in this part of the thesis called linear programming and linear production games [START_REF] Owen | On the core of linear production games[END_REF], [START_REF] Samet | On the core and dual set of linear programming games[END_REF].

Definition 9.12. Let N be the set of N players, A ∈ M (R) M ×N a constraint matrix, b {i} ∈ R M a vector of resources controlled by player i, B ∈ M (R) M ×N a matrix whose j-th column is the vector b {j} , c ∈ R N a cost vector common to all players and let t S be a binary vector whose j-th coordinate is 1 if j ∈ S.

A linear programming game (LPG) is a cooperative game G = (N , v) for which

v(S) = max{cx : Ax ≤ Bt S } (9.8)
Two important results regarding LPGs are summarized by the following theorem, due to Owen.

Theorem 9.2. Linear Programming Games are totally balanced [START_REF] Owen | On the core of linear production games[END_REF].

Proof. We shall prove the theorem by a direct application of Theorem 9.1.

The key idea behind the proof is to construct a solution that is feasible in the optimization problem associated with the grand coalition. Because the problem is a maximization one, it will hold that the value of this solution will be a lower bound of v(N ) and that will conclude the proof. Of course, for this to happen, the proposed solution multiplied by the cost vector should be equal to the left hand side of Equation (9.7). Let γ S be an arbitrary balanced function satisfying Equation (9.6). Our candidate solution will be given by

x * = S∈2 N \∅ x S * γ S
, where x S * is an optimal solution to v(S).

First, we show that the proposed solution is indeed feasible in the problem of the grand coalition. For this we need to show that:

S⊂N \∅ A (γ S x S * ) = S⊂N \∅ γ S (A x S * ) ≤ S⊂N \∅ γ S b S * ? = b N By construction, b S = i∈S b {i} . Consequently, S⊂N \∅ γ S b S * = i∈S b {i} S⊂N \∅,i∈S γ S = b N
Furthermore, the cost associated with the proposed solution (which we just show is feasible) is given by:

S γ S v S = S γ S ( c x S * ) = c S γ S x S *
Therefore S γ S v(S) ≤ v(N ), because the balanced sum S γ S x S * is feasible in the maximization problem, hence it is a lower bound and this concludes the proof. Theorem 9.3. In a linear programming game G = (N , c, A, B), the set defined as: DS = {y ∈ R N : y = uB, for some dual optimal vector u} (9.9) is a subset of the core.

Proof.

The main idea behind the proof is to observe that the feasible set of the dual of (9.8) does not depend on the coalition, hence the optimal solution of the problem associated with the Grand Coalition is also feasible in the problem associated with all the other coalitions S.

Let u S be the optimal solution of the dual of optimization problem (9.8) associated with coalition S.

Because of duality in linear programming problems it holds that:

u S b S = v(S).
Furthermore, the solution u S associated with coalition S is feasible in the dual of all other coalitions. By applying the definition of matrix product, we now that the searched solution y = u N B satisfies that

y i = u N , b {i} .
By taking the some on an arbitrary coalition S we have that

i∈S y i = i∈S u N , b {i} = u N b S (9.10)
From Equation (9.10), we can conclude the proof, as follows. First, when S = N , i∈S

y i = u N b N = v N
which implies that the payoff is efficient. Finally, we know that u N b S is the objective value obtained by substituting λ N in the dual associated with coalition S. Because the dual is a minimization problem, u N b S is an upper-bound on v(S). We can thus derive that i∈S y i ≥ v(S), thus concluding the proof.

It can be shown [START_REF] Kalai | Totally balanced games and games of flow[END_REF] that the family of totally balanced games coincides with the family of linear programming games.

Chapter10

Coalitional Storage Games: A Model Using Cooperative Game Theory for Sharing Energy Storage

General Context

We begin our study of cooperative models with a shared investment problem. We consider a set of players without generation (only with positive energy consumption) and without batteries. Clearly, there is no local energy trading available to them. These consumers, subject to a Time-of-Use tariff, could benefit from having energy storage, as they could charge it when the energy is cheaper and discharge it when it is more expensive. We study the benefit that these players can obtain from investing together into community storage, instead of doing so independently.

The main motivation behind this is that individual players might under utilize their energy storage systems, and that as more players participate in the operation of the battery, the more likely it is that it will be used up to its full potential.

To embed this model within the local energy trading framework, we can envision that a (virtual) player controls the shared battery, and that this player buys from the grid and sells the energy internally to the other participants using virtual trades.

Related Work

Energy storage systems (ESS) have been extensively proposed as solutions for a wide array of problems in smart grids: from increasing auto consumption of prosumers owning photovoltaic panels to providing ancillary services to the grid [START_REF] Shi | Using battery storage for peak shaving and frequency regulation: Joint optimization for superlinear gains[END_REF], [START_REF] Engels | Techno-economic analysis and optimal control of battery storage for frequency control services, applied to the german market[END_REF] or even stacking several of such services [START_REF] Engels | Combined stochastic optimization of frequency control and selfconsumption with a battery[END_REF]. One particularly widespread use of ESS is arbitrage: buying energy when it is cheap and using it when it is expensive. Typically, the literature that addresses such solutions does not take into account the cost of buying a battery, e.g., it assumes that the participants already own it [START_REF] Horta | Novel market approach for locally balancing renewable energy production and flexible demand[END_REF]. In practice, this can be far from the case as the present cost of an ESS is in the range of thousands of US dollars, an investment that many might not be willing to make. There is still a debate on whether batteries used for arbitrage are profitable and the answer appears to depend on the prices available for arbitrage [START_REF] Carpenter | The impact of electricity pricing schemes on storage adoption in ontario[END_REF] [62], the veracity of the model used [START_REF] Ghiassi-Farrokhfal | Toward a realistic performance analysis of storage systems in smart grids[END_REF] and how efficiently it is operated [7]. Moreover, while there is evidence that, in at least certain cases, energy arbitrage is profitable, it is unclear whether these incentives alone will be enough to motivate individual investments for the typical consumer.

The decision to buy an ESS is closely related to the problem of selecting optimal characteristics for such storage, namely capacity, output power, efficiency, etc. In [START_REF] Wang | Energy storage in datacenters: what, where, and how much?[END_REF], the problem of optimal storage investment for a datacenter is studied from the perspective of the datacenter owner.

An alternative to individual purchase, which is becoming increasingly popular, is collaborative consumption or Sharing Economy [START_REF] Hamari | The sharing economy: Why people participate in collaborative consumption[END_REF]. Although the most well-known success stories do not belong to the energy domain [START_REF] Oskam | Airbnb: the future of networked hospitality businesses[END_REF], there have already been proposals in the context of Smart Grids. For example, the problem of storage placement is dealt with in [START_REF] Gkatzikis | Collaborative placement and sharing of storage resources in the smart grid[END_REF], the shared collaborative management of ESS is treated in [START_REF] Wang | Active demand response using shared energy storage for household energy management[END_REF] and the shared of surplus renewable in [START_REF] Zhu | Sharing renewable energy in smart microgrids[END_REF] and [12].

When we consider the shared investment in a commodity, the main questions are how much to buy, how much every participant should pay and whether a payment plan is "stable". Here, stability means that, once the payment plan is agreed upon, no participant (nor group thereof) has an incentive to opt out of the investment. The natural framework for studying such scenarios is cooperative game theory, and indeed most of the existing work on the problem has focused on cooperative game models. For example, [5] studies the shared investment in photovoltaic panels, while [6] discusses the possibility of a snowball effect between the formation of communities and changes in electricity tariffs to accommodate such communities. Some other studies take a non-cooperative game perspective, e.g., [START_REF] Kalathil | The sharing economy for the electricity storage[END_REF], which considers firms investing in a shared ESS.

Our study is motivated by the analysis presented in [START_REF] Chakraborty | Sharing storage in a smart grid: A coalitional game approach[END_REF], which employed cooperative game theory to model the collective purchase of a battery. In this chapter, we improve their general model by including key constraints related with real storage systems and by dealing with realistic energy demand profiles. Furthermore, we describe classes of games where it is profitable to purchase storage only through shared investment. Of the several differences between [START_REF] Chakraborty | Sharing storage in a smart grid: A coalitional game approach[END_REF] and our study, we highlight the following. First and foremost, [START_REF] Chakraborty | Sharing storage in a smart grid: A coalitional game approach[END_REF] does not consider ramp constraints while our study does; in addition, while in [START_REF] Chakraborty | Sharing storage in a smart grid: A coalitional game approach[END_REF] the model of a user does not take into account inter-day consumption patterns, our model does. Since our study makes a first step in exploring the (evidently complex) impact of ramp constraints, we focus on deterministic consumption profiles, while [START_REF] Chakraborty | Sharing storage in a smart grid: A coalitional game approach[END_REF] considers probabilistic profiles.

The emphasis on the user representation is important for two reasons. First, ramp constraints cannot be appropriately modeled if the only information about the user is their net consumption. Second, we believe that new solutions in Smart Grids should take advantage of the enormous amount of data on user consumption patterns, thanks to the massive deployment of smart meters and open access datasets, e.g., Pecan Street. Some examples of this trend are [START_REF] Kurniawan Wijaya | Consumer segmentation and knowledge extraction from smart meter and survey data[END_REF], [START_REF] Omid Ardakanian | Computing electricity consumption profiles from household smart meter data[END_REF].

To the best of our knowledge, the present work is the first to study in depth the impact of ramp constraints in coalition storage games. Ramp constraints are included in the model of [6], yet they do not claim any results regarding the grand coalition (i.e., the set of all players) nor they study the impact of such constraints on the earnings of the coalitions, both of which we do in this study.

Our main contributions can be summarized as follows.

• For a realistic tariff structure (Time of Use), we show that it can be profitable to invest on shared storage; thus, providing a feasible alternative to individual purchase.

• We extend the cooperative game proposed in [START_REF] Chakraborty | Sharing storage in a smart grid: A coalitional game approach[END_REF] by including ramp constraints and proving the existence of stable solutions for an extensive part of such games that we characterize. In doing so, we show an equivalence between a subset of a well known class of cooperative games, namely "unitary glove market" games and an interesting class of storage games with ramp constraints.

• We expose the intrinsic relationship between ramp constraints and consumer profiles and indicate how it impacts the formation of coalitions.

• Through numerical studies with real data that exploit the insight of our theoretical analysis, we show the potential impact of ramp constraints on shared investments in energy storage systems.

The rest of the chapter is organized as follows. Next, in Section 10.3, the model employed in this study, as well as the considered problem, are formulated. The theoretical analysis and results are presented in Section 10.4. Then, a case study that is based on real data is described in Section 10.5. Section 10.6 presents a discussion about the implication of our model decisions and the applicability of our results. Finally, concluding remarks are presented in Section 10.7.

Mathematical Formulation of the Model

From now on, the terms coalition and community will be used interchangeably. The same holds for consumer and player.

Let N be a set of N consumers that are "closely located" (e.g., they reside in the same building) and that do not have generation capabilities, i.e., they are not prosumers. We assume that each of them is subject to a Time-of-Use (ToU) pricing with two levels: p l from midnight to t m (early) and p h from t m to midnight (t f , late), where p l < p h . Consumer i might benefit from having a battery if she can buy energy early, store it in the battery and consume it later. If X i denotes the above mentioned energy, then her savings amounts to (p h -p l )X i . More formally, the daily cost incurred by consumer i when having a battery B is given by Equation (10.1):

H i (B) = f (B) + p l tm 0 l i (s)ds + (p l -p h )X i + p h t f tm l i (s)ds (10.1)
where l i represents the consumption profile of player i, f (B) is the amortized cost of the battery B in a given day, the second term accounts for the energy already consumed earlier, the third term for the gain by shifting consumption to the early period and the last term for the energy than had to be consumed during the expensive period. Finally, X i is defined in Equation (10.2):

X i = min t f tm min{l i (s), R(B)}ds, R(B)t m , S (10.2)
In the above equation, R(B) denotes the ramp constraint, i.e., the maximum power at which the battery can charge or discharge (for simplicity, we assume that the value is the same in both directions, with opposite signs) and S denotes the battery capacity. Thus, the first term in Equation (10.2) stands for the maximum energy available to shift to the early period taking into account the ramp constraint; the second accounts for the fact that, if the two periods of the ToU are not of the same length, it might be impossible to discharge all the charged energy; and the last term is the capacity of the battery. We will not take into account battery efficiency but it can be shown that our results hold for this case (see Appendix C.1 for details). Consumer i wishes to find an optimal battery such that Equation (10.1) is minimized. We shall refer to this minimization problem as the optimal storage problem for player i.

Simplified model and problem

Let us consider a simplified version of the optimal problem for player i with the following additional assumptions.

(A1) |t m | = |t f -t m |.
By assuming that t m occurs in the middle of the day, we can omit the middle term in Equation (10.2):

t f tm min{l i (s), R(B)}ds < R(B)t m .
This does not change the results drastically, but makes the analysis less cumbersome.

(A2) Without loss of generality, we normalize time as follows: t f = 1, t m = 0. Moreover, and w.l.o.g., we shall also assume that the energy consumption in the early period is 0.

(A3) f (B) = πS. That is, the cost of the battery is linear in the capacity. In particular, this assumption allows us to compare our results with previous studies, namely: [START_REF] Chakraborty | Sharing storage in a smart grid: A coalitional game approach[END_REF] and [START_REF] Kalathil | The sharing economy for the electricity storage[END_REF]. In what follows, π shall denote the amortized cost of the battery in a single day.

(A4) The consumption of each player l i is piece-wise constant. Indeed, in most datasets, the consumption of a consumer is given by discrete values because of the sampling time frequency (typically, every second or every 15 minutes).

(A5) R(B) = δS, i.e, linear ramp constraints. This captures the potential of stacking several storage modules, thus increasing power output as well as capacity.

If we have that (t f -t m )γ = S, then γ is the power at which the battery charges completely in the period [t m , t f ]. With that in mind, we will use γ = δS to denote the ramp constraint. In this setting, δ = 1 means that the battery takes the whole interval to charge and δ = 2 is a battery that works twice as fast. δ shall always be fixed.

(A6) p h -p l = P > π. In the case that prices depend linearly on capacity, we can write: f (B) = πS. It is also clear that X i ≤ S, where S is the capacity of the battery. If π > P , then P X i < πS, and the battery cost is larger than the gain obtained through it. Thus, in such a case, the optimal action would be not to buy a battery.

Defining the Cooperative Game

Let S ⊆ N denote a coalition of consumers. The energy consumption of S is given by (10.3) and the total energy consumption during the time period will be denoted by E S .

l S : [0, 1] → R + , l S (x) = i∈S l i (x) (10.3)
Given the above model and assumptions, the cost of the community S having a battery B is given by Equation (10.4): Finally, the cost of a community S is given by Equation (10.5), and we will denote by S S * the optimal battery size at which the minimum is obtained.

v(S) = min S H S (B) (10.5)
Although it is not possible to obtain an analytic solution in general, in what follows we will derive several properties of Equation (10.4) that will greatly simplify the computation of v. We will refer to the optimization problem defined in Equation (10.5) as the optimal storage problem for coalition S.

A cooperative game in which the characteristic cost function v is defined using Equation (10.5) will be termed a storage game.

Specifically, it will be shown that, under certain conditions, the optimization problem defined by Equation (10.5) is convex and, furthermore, there exists a simple algorithm for obtaining the optimal solution.

Transforming the integral

Equations (10.4) and (10.5) shed little light on the solution of the optimal storage problem for coalition S. In part, this is due to the integral in (10.4), which integrates in time and not in battery size.

Let I C be a partition of the interval [0, 1] in which l S is constant and let {I 1 , I 2 , . . . , I m } denote the different intervals with the convention that the consumption of player i (l i ) is increasing in the sub-index. That is: I 1 is where l S takes the smallest positive value and, in general: l S (I k-1 ) < l S (I k ) < l S (I k+1 ) for all k (l S (I 0 ) = 0).

If F (S) = where α is such that l S (I α ) ≤ δS and l S (I α+1 ) > δS. For the example presented in Figure 10.1, α = 1.

Let G(S) = δS 0 g(x)dx where g is the piece-wise constant function defined in (10.7). Function g is plotted in Figure 10.1 C, while function G is plotted in Figure 10.1 D.

g(x) = j=m j=i |I j | x ∈ [l S (I i-1 ), l S (I i )) 0 x ≥ l S (I m ) (10.7)
Proposition The functions F and G are identical. Proof. We then can establish that F = G. Note that, in the above, the double summation in the second line is pseudo-telescopic and most terms cancel out, resulting in the first sum of the last line. Using Leibniz' rule, we can obtain a closed formula for the derivative of G, presented in equation (10.9). Specifically, it is equal to g multiplied by δ with dilated intervals. Observe that G is a non-increasing function, and denote by ∆ k the k-th value attained by it. These values are going to play a crucial role in determining the optimal size of the battery. Plot D in Figure 10.1 presents the function G, the different values ∆ and the identity function h : R → R, h(S) = S as a dashed line. Recalling that X S (B) is defined as the minimum between F and h (shown in blue), the derivative of X S with respect to S is either one of the ∆s or 1, depending on which function is larger.

G(S) =

G (x) = δ j=m j=i |I j | x ∈ [ l S (Ii-1) δ , l S (Ii) δ ) 0
x ≥ l S (I m ) (10.9)

We proceed with two additional observations: first, x ≥ l S (Im)

δ =⇒ G(x) = E S ; and second, if γ(l S ) ∆ = |{x : l S (x) > 0}| is the positive support of l S , then ∆ 1 = δγ(l N ).

Optimal Storage Size for a Community

The optimal cost of a community (defined in Equation (10.5)) involves minimizing (10.4), which is a convex minimization problem. This can be seen from the fact that -X S is convex and so is πS. The derivative of H S is given by equation (10.11). Observe that there exists a battery capacity S such that:

∀ S ≥ ba : min{G(S), S} = G(S) and G (S) = 0 ⇒ ∂H S ∂S > 0 (10.10)
This is because at some point, G stops increasing while h never does. Since H S is continuous, (10.10) is sufficient to guarantee that a minimum exist before S or at it.

∂H S ∂S = π -P min{G(S), S} = S π -P G (S) min{G(S), S} = G(S) (10.11)
The following proposition shall be used extensively in what follows.

Proposition 10.1. If P ∆ 1 < π, then the optimal battery capacity is C * B = 0.

Proof. P ∆ 1 < π ⇒ ∆ 1 < π P < 1.
This implies that min{F, h} = F and ∂H S ∂S = π -P ∆ 1 > 0. Then, the fact that H is increasing and H(0) = 0 implies that the minimum of H is attained at 0.

In the next section we shall establish several properties of the cooperative game defined above.

Game Theoretic Analysis

Case with Inactive Ramp Constraints

The first step in analyzing the game will be to study the case in which ramp constraints are inactive. Specifically, by "inactive ramp constraints" we mean that min{F (S), h(S)} = F (S) ⇐⇒ S ≥ E S

. Observe that this happens as long as δ is large enough.

It is quite simple to realize that, in this setting, the optimal battery size is exactly the energy consumed by the community (C S * B = E S ). Furthermore, the cost of the grand coalition J(N ) satisfies the following Equation (10.12):

v(N ) = p h E N + πE N -P E N = (π + p l )E N = (π + p l ) N i=1 E {i} = N i=1 v({i}) (10.12)
which implies that the game is inessential. Recall that inessential games have a non-empty core consisting of only one imputation, namely one that assigns to players what they can do on their own: v({i}). In our framework and in the absence of ramp constraints, it is the same to cooperate or not. As already mentioned in the Section 10.2, games without ramp constraints have already been studied in [START_REF] Chakraborty | Sharing storage in a smart grid: A coalitional game approach[END_REF]. It is of interest then to check whether the solutions of both games coincide. We then conclude our analysis of inactive ramp constraints by comparing between the optimal capacity of a battery in the model proposed by [START_REF] Chakraborty | Sharing storage in a smart grid: A coalitional game approach[END_REF] and ours. Since [START_REF] Chakraborty | Sharing storage in a smart grid: A coalitional game approach[END_REF] considers a probabilistic distribution of consumption and we do not, the comparison will be made for the case that such distribution is concentrated at a single point, i.e., the deterministic case.

In the model of [START_REF] Chakraborty | Sharing storage in a smart grid: A coalitional game approach[END_REF], the cost of a coalition is given by:

v(S) = πC + p h E f [(X -S) + ] + p l E f [min{S, X}] (10.13)
where X is the energy consumed. If X is concentrated at one point, then (10.13) becomes (10.14), a decreasing function if S ∈ [0, X] and increasing if S ∈ (X, ∞). S = X is trivially the minimum and, since X is the energy consumption, both models yield the same solution.

v(S) = (π -ph + p l )S + p h X S ≤ X πS + p l X S > X (10.14)

An example with an empty core

As shown above, if ramp constraints are inactive, the core is non-empty, and, furthermore, it consists of a rather trivial solution. However, with active ramp constraints, the results are sharply different. For example, consider a constant ramp constraint R(B) = δ (for subsection we drop assumption A5). That is, independently of the battery size, the maximum power that the battery can provide at any point in time is fixed. When players consume energy concurrently, they increase the required maximum peak power. Under certain circumstances, the optimal strategy is to buy a larger battery, yet sometimes consumers do better on their own. In those cases, the core will naturally be empty, as there is no incentive for the grand coalition to form. We proceed to present a simple example.

Let A and B be two consumers with the following properties: A consumes power at a constant rate K during an interval of length x A , and B consumes power at the same rate during an interval of length x B . Their detailed consumption is depicted in Figure 10.2. Furthermore, let the parameters of the game satisfy: 2) and (3) guarantee that, for both players, it is optimal to buy a battery. Together, the optimal action is to buy a battery of size x A + x B -x AB = γ(l N ).

The characteristic cost function of the game satisfies:

v({A}) = Kx A (p l +π), v({B}) = Kx B (p l +π), v({A, B}) = K(x A + x B -x AB )(p l + π) + p h Kx AB .
Any allocation (α, β) that is in the core should satisfy:

α + β = K(x A + x B )(p l + π) + Kx AB (P -π) α ≤ Kx A (p l + π) β ≤ Kx B (p l + π) (10.15)
Clearly, whenever x AB > 0 and P > π, the system equation ( 10.15) has no solution.

The intuition behind the above example is as follows: if players can use their battery at its maximum rate on their own and it is not possible to buy a faster battery together (because δ is capped), then they cannot do better acting together than by acting on their own.

The fact that our model is able to produce examples with an empty core while in [START_REF] Chakraborty | Sharing storage in a smart grid: A coalitional game approach[END_REF] it has been proven that the games considered there have a non-empty core, indicates that the incorporation of ramp constraints calls for special attention and analysis.

The Case with Two Players with Linear Ramp Constraints

As just shown,if ramp constraints are constant, then the core might be empty. We proceed to show that this is not the case with linear constraints1 .

Theorem 10.1. The 2-person storage game with linear ramp constraints has a non-empty core.

Proof. Assume that each of the two players, A and B, when acting on its own, finds it optimal to buy a battery of capacity S A and S B . The incurred cost will be πS A ( πS B ). Let d A and d B denote the usage rate of the battery bought by A and B, respectively, when acting alone. d A (s) ≤ l A (s) and d A (s) ≤ δS A and equivalently for B.

When acting together as a coalition, they can buy a battery of size S AB = S A + S B . The cost incurred by the players would be πS AB = S A π + S B π. Let d AB (s) = d A (s) + d B (s) be the usage of the battery at time s. d AB (s) ≤ δS A + δS B = δS AB .

Thus, when acting together, each of them can do as well as when acting alone. This in turn guarantees that v({1, 2}) ≤ v({1}) + v({2}) and, hence, it is a sufficient condition for the existence of the core in the two player case.

For the rest of the chapter we will assume again linear ramp constraints.

Unitary Glove-Market Games

So far it has been proven that, if ramp constraints are inactive, the resulting game is inessential, whereas if the ramp constraints are active, the core might be empty. In general, when the core is nonempty, it is hard to fully describe it. We proceed to formulate a class of games for which it will be shown that the core has a closed form solution. Furthermore, we will indicate that the core lacks fairness, hence we will explore the employment of Von Neumman and Morgenstern solution, which does provide both stability and fairness.

Let

A 1 , A 2 , . . . , A K ⊂ [0, 1] and A i ∩ A j = ∅, ∀i, j ∈ [K].
For each set A k define the consumption profile c k such that:

c k (A k ) = 1 c k ([0, 1]\A k ) = 0
we will refer to c k as the consumption profile of type k. Consider a N -person storage game where the consumption of player i is the same as the consumption profile of type k for some k, i.e, l {i} = c k . We will say that player i is of type k. Furthermore, we will impose that the game satisfies:

C1) 1 > P δ K i=1 |A i | > π C2) P δ K i=1,i =j |A i | < π, ∀j ∈ [K]
Conditions (C1) and (C2) imply that it is necessary to have a player of each type in order to buy a battery. Further discussion about both conditions can be found in Section 10.6.

We proceed to determine the characteristic cost function. For a given coalition S, denote by n i (S) the number of players of type i in S and m(S) = min{n i (S) : i ∈ [K]}. If m(S) = 0 there is not a player of every type and ∆ 1 = γ(l S ) < π, so it is not optimal to buy a battery. If m(S) > 0, P ∆ 1 > π but P ∆ 2 < π, so the optimal battery size is l S (I1)

δ = m(S) δ . v(S) = p h K i=1 n k (S)E k m(S) = 0 p h K i=1 n i (S)E i + π m(S) δ -P m(S) K i=1 E i m(S) > 0 (10.16)
The characteristic cost function v of such a game is given by (10.16) and it is equivalent to the normalized characteristic value function v that satisfies v(S) = 0 if m(S) = 0 and v(S) = m(S) m(N ) if m(S) > 0. The game with (0, 1)-normalized characteristic value function v is a well studied game in the literature, namely the "unitary glove-market" (UGM) game [START_REF] Apartsin | The core and the bargaining set in glove-market games[END_REF]. UGM is an extension of the "glove market", game in which N players produce either a left or right glove that they attempt to sell. The value of a left or right item is 0, but a matching pair can be sold for a unit of value. The extension, namely UGM, consists of considering K different types of commodities (for any K ≥ 2) and imposing that a only bundles that consist of one commodity of each type have positive value (and 0 value otherwise). The following theorem has been established for UGM games [START_REF] Apartsin | The core and the bargaining set in glove-market games[END_REF].

Theorem 10.2. Let v be a (0, 1)-normalized characteristic value function of an N player unitary glovemarket game, let T 1 , T 2 , . . . , T K denote the number of players of each type and K i=1 T i = N . Without loss of generality assume that: T 1 = T 2 = . . . T l < T l+1 ≤ . . . T K . Then, the core is non-empty and consists exactly of the following imputations α:

α(i, j) = u i i ≤ l 0 i > l (10.17)
where

k=l k=1 u k = 1 T 1 = 1 m([N ])
.

and α(i, j) is the cost of the j-th player of type i.

As it has been often claimed [START_REF] Lucas | Chapter 17 von neumann-morgenstern stable sets[END_REF], the core may not provide fairness. Indeed, for UGM games for which l = 1, the core is precisely the (single) imputation that assigns all the profit to players of type 1 and nothing to the rest. Even when l > 1, all the players of type k > l get nothing and players of types k 1 , k 2 ≤ l might get different payoffs.

A solution concept that has been claimed to be fairer that the core is the Shapley Value [START_REF] Rapoport | N-person game theory: concepts and applications[END_REF]. Yet, this concept too is not without shortcomings: it is computationally intensive to compute it and it might not be in the core, which implies lack of stability in some sense. Moreover, in our case the following holds.

Proposition 10.2. For any unitary glove-market game with l < K, the Shapley value is not in the core.

Proof. Consider a player p of type i with i > l and a coalition S with one player of each type except i. Then, v(S ∪ {p}) -v(S) = 1 T1 > 0. Looking at Equation ( 9.3), φ p (v) must be positive, but any payoff vector in the core will assign player p exactly 0, because she is of type i > l. This concludes that the Shapley value cannot be in the core if l < K.

Motivated by the negative result of Proposition 10.2, we proceed to analyze the Von Neumman and Morgerstern solution for such games. The existence of stable sets in which all players of types i > l get a positive reward is guaranteed for UGM games [START_REF] Apartsin | The core and the bargaining set in glove-market games[END_REF], [START_REF] Rapoport | N-person game theory: concepts and applications[END_REF]. To illustrate this result, we consider a concrete case for which we derive the precise Von Neumman and Morgerstern solution, as follows.

Proof.

To begin with, we note that, for UGM games, an imputation x can dominate an imputation y over a coalition S if and only if S contains at least a player of each type, otherwise both conditions in Definition 9.9 presented in Chapter 9 cannot be satisfied concurrently.

First, we prove that V is internally stable. Let x = (x 1 , x 2 , x 2 , x 4 , x 4 ) ∈ V and y = (y 1 , y 2 , y 2 , y 4 , y 4 ) ∈ V . Since it cannot be the case that x 1 > y 1 , x 2 > y 2 and x 4 > y 4 (because x, y are efficient), we may assume w.l.o.g. that x 4 < y 4 . Now, x cannot dominate y via a coalition in which a player of type III is present, because she would do worse.

Yet we have already shown that, if x were to dominate y, it should do it via a coalition with one player of each type, which is a contradiction. Hence, V is internally stable.

Next, we prove that V is externally stable. Let y be an imputation such that y 2 = y 3 . Since . Because y 2 = y 3 , > 0. Consider the imputation x defined as:

x = (y 1 + , m II + , m II + , m III + δ + , m III + δ + )
The imputation x is in V and it dominates y via the coalition S = {1, i, j} where i (j) was the player of type II (III) getting less with imputation y . Because there are not other imputations outside V , we conclude that V is externally stable, which concludes the proof.

The core of the game considered in Proposition 10.3 consists on the unique payoff vector (1, 0, 0, 0, 0) whereas the stable set V contains the payoffs (0, 1 4 , 1 4 , 1 4 , 1 4 ) and ( 1 5 , 1 5 , 1 5 , 1 5 , 1 5 ) among others. Even though the notion of stable set is not without its shortcomings, we argue that is a better fit for some of the storage games considered here, e.g., to the "unitary glove market" games.

Nonetheless, the concept of stable sets as a solution to cooperative games has also its drawbacks.In particular, it has been shown that some games admit infinitely many stable sets. This is in sharp contrast to the Shapley value, which provides a fair and unique solution. We thus proceed to investigate a class of storage games for which it is guaranteed that the Shapley value is also stable, i.e., it belongs to the core.

A class of concave games

In the previous subsection we introduced a class of storage games with non-empty core and we illustrated how the core might fall short of providing fairness. We proceed to introduce a class of concave games, for which it is known that the Shapley value belongs to the core, hence providing a solution that is fair and at the same time stable. Moreover, we shall establish an analytic expression of the corresponding Shapley value. We begin with the definition of some additional terminology.

A player is said to fill a time gap if ∀S, i ∈ S, γ(l N ) -γ(l N \{i} ) > 0. That is, she consumes energy at a point in time when no one else is consuming. To ease the notation we shall use tg i = γ(l N ) -γ(l N \{i} ) to denote the length of the time gap of player i.

The next theorem establishes that, if conditions similar to C1 and C2 (as specified in Subsection 10.4.4) hold and every player fills a time gap, then the game is concave. Then the game is concave and the solution is not trivial. a a Here, "trivial" means that no player buys a battery.

We shall refer to games satisfying H1, H2 and H3 as "time-filling" games. In the next section, we will show how "time-filling" games arise naturally and how real data satisfies the hypotheses (H1) -(H3).

Proof. For any coalition S ∈ 2 N , S = N , i.e, different from the grand coalition, γ(l S ) < γ(l N \{i}) for some i and H3 implies that the optimal battery capacity is 0:

v(S) = p h i∈S E i .
For the grand coalition, H2 guarantees that the optimal battery capacity is positive and let S N denote that value. Then, v(N ) = p h i∈S E i + S N π -P X N , with S N π -P X N < 0.

It is easy to check that, for every player i and every pair of coalitions

S ⊂ T ⊆ 2 N \{i} , it holds that v(S ∪ {i}) -v(S) = v(T ∪ {i}) -v(T ) = p h E i if T = N .

Minimal set of conditions

Although the conditions specified in the above theorem are sufficient yet not necessary, they are minimal in the sense that, in the absence of any of them, there is a counterexample in which the game is either not concave or else it admits a trivial solution. We begin by removing condition (H2) and showing that the resulting game is trivial. Observe that, if π ≤ P δγ(l N ) does not hold, then π > P δγ(l N ) = P ∆ 1 . By Proposition 10.1 the optimal action is not to buy a battery, i.e., a game with a trivial solution.

We proceed to consider the case in which some players do not fill a time gap, i.e., we drop (H1). Figure 10.3 describes one such scenario, in which only player 4 fills a time gap.

Observe that the game now is an instance of the UGM game with K = 2 (i.e., a basic "gloves market" game). Proposition 10.2 guarantees that, in such a game, the Shapley value cannot be in the core. Yet, if the game were to be concave, then the Shapley value would be in the core, i.e., a contradiction. Thus, the game is not concave.

Finally, let us observe what occurs when we drop condition H3. If this is the case, all players fill a time gap, for the grand coalition it is profitable to buy a battery, and there exists a coalition S = N such that C S * B > 0 (the optimal action is to buy a battery). An example of such case is given by Figure 10.4 with parameters satisfying: 25 12 , p h = 22 10 , p h = 2 10 and tg 0 = tg 1 = 1 10 . It is easy to check that all players fill a time gap (the light shaded area in the figure), P δγ(l N ) = 25 12 > π and P δγ(l {1,2} ) = 5 3 > π. The resulting cost value function is given by the following Equation (10.18).

x 0 = x 1 = 1 5 , x 2 = 3 10 = tg 2 , P = 2, π = 1, δ =
Finally: v([2]) -v({0, 2}) = 293 950 ≈ 0.308 > v({0, 2}) -v({2}) = 3 50 = 0.06.
Proving that the game is not concave. We thus conclude that each of H1, H2 and H3 are required to guarantee that the resulting game is concave and non-trivial.

Closed form of the Shapley Value

As mentioned, in a concave game, the Shapley value belongs to the core. Although in general, due to its combinatorial nature, the Shapley value is computationally intractable, this is not the case for the class of "time-filling" games, as shown next.

Theorem 10.4. The Shapley value of a "time-filling" game with characteristic cost function v is exactly

φ i (v) = p h E i - v(N )-j∈N p h Ej N Proof.
Since only in the grand coalition it is profitable to buy a battery, v(T ∪ {i}) -v(T ) = p h E i , ∀T = N . There are N ! ways in which the grand coalition will form. In (N -1)(N -1)! of those ways, i will not be the last player to join, and her contribution to the grand coalition will be exactly p h E i . In the other (N -1)! cases, because of her, the grand coalition will form and her contribution to it will be: v(N ) -j∈N ,j =i p h E j . The Shapley value of player i will be her expected contribution, that is:

φ i = (N -1)(N -1)! N ! p h E i + (N -1)! N !   v(N ) - j∈N ,j =i p h E j   = 1 N   (N -1)p h E i + v(N ) - j∈N ,j =i p h E j   = p h E i + 1 N   v(N ) - j∈N p h E j   (10.19)

Nash Bargaining Solution

We focus here on the particular case were the payoff space is precisely the set of efficient imputations and the disagreement point of player i is given by v({i}). Under the above setting, the solution to Equation (9.5) is given by Equation (10.20):

x i = v({i}) + v(N ) -i∈N v({i}) N (10.20)
The fact that the Nash Bargaining Solution and the Shapley value coincide for the class of concave "time-filling' games considered in Section 10.4.5, guarantees that the Shapley value also satisfies all the desirable properties of the NBS. Why both values coincide can be explained as follows: as opposed to the Shapley value, the Nash Bargaining Scheme uses information only of the grand coalition; however, in the class of "time-filling' games considered in Section 10.4.5, no other sub-coalition will be formed, and the information available to both payoffs is the same.

Case Study

In order to gain insight into the practical relevance of our analytic findings and their potential applicability into actual scenarios, we proceed to examine the problem considered in this study using real-world data.

We consider the following three widely used batteries: Powerwall 2 [4], BMZ ESS 7.0 [2] and Aquion [START_REF] Kiedanski | Strategy-proof local energy market with sequential stochastic decision process for battery control[END_REF]. In addition, in order to complement the results, we consider two "fake" batteries. Table 10.1 summarizes the properties of the different batteries. Our model uses two battery paramters, π and δ, which can be estimated from the ones in Table 10.1. If the battery is assumed to last 10 years, the amortized cost of the battery in a given day π can be calculated by dividing the total cost by 10 × 365. Estimating δ can be somewhat trickier. The number of hours a battery needs to fully charge (HFC) is given by the capacity divided by the output power. Recalling that δ represents how many times the battery can be fully charged in the considered interval (namely, 12 hours), δ is given by 12h/HF C. The results are presented in Table 10.2. For the consumption data, we employ that of Pecan Street [149]. In particular, we consider the energy usage of five users during the 1st of April of 2018, as shown in Figure 10.5a. Our main objective is to asses the financial viability of a shared investment in storage, and in particular check whether a grand coalition would be formed. Five different Time-of-Use tariffs were considered, as follows: P = 15, 18, 22, 30, 35c/kW h and p h = P + 20c/kW h.

Denoting by W (S) -the cost incurred by the players of coalition S when no battery is bought, we define the relative gain with respect to not buying a battery (RGnBB) as W (S) --v(S) W (S) - × 100%. For P = 18 and P = 30 this is plotted in Figures 10.6a and 10.6b respectively. Naturally, as P increases, so does the gain: this is reflected by the ranges of values plotted in the y-axis. Lighter shades of blue denote a cheaper battery, while darker shades correspond to the more expensive ones. In Figure 10.6a, there is a clear correlation between the gap in buying prices (P) and the amortized cost of the battery (π), yet not so much with ramp constraints. It is important to notice that, depending on the available tariffs, there might be technologies that are not profitable, e.g., Aquion and Slow Fake with P = 18 and therefore they do not appear in Figure 10.6a. They do appear in Figure 10.6b because with P = 30 both are profitable.

Analogously to the relative gain with respect to not buying a battery, Figure 10.7a (10.7b) depicts the relative gain with P = 18 (P = 30) obtained by the grand coalition in comparison with every player buying a battery on their own (RGBBO), that is:

|v(N ) - N i=1 v({i})|/ N i=1 v({i})
. Whereas Figures 10.6a, 10.6b inform us on the benefits of buying a battery, Figures 10.7a and 10.7b illustrate how much players benefit from acting as a coalition instead of acting alone. We note that here, ramp constraints do play a central role. First, they determine if the game is essential or not. For the two ESS batteries with very large ramp constraints, namely Super Tesla and BMZ, there is never a gain in participating in the coalition and players can do just as well on their own. In addition, when ramp constraints are active but not too loose, the value obtained increases with the number of players considered and the value of the ramp appears to play a significant role on the speed of the growth.

Finally, observe that the size of the whiskers is correlated with the value of the ramp constraint. This has the following explanation: when ramp constraints are tighter, the individual profile of each player becomes more relevant in determining the gain of the coalition.

We proceed to present a scenario in which the "time-filling" games studied in Section 4 can arise.

Reducing peak consumption

Depending on a contract, consuming energy at higher power might be detrimental. For example, consider an household that has to decide whether to sign a contract for a higher maximum power at a higher price. In such cases, the consumer might be interested in investing in a battery in order to satisfy their out-of-range peaks.

To assess the impact of our methodology in such scenarios, let us consider a peak tariff, where all power consumed above 3kW is charged at price p h = 35c kW h and the rest at p l = 10c kW h. The threshold, albeit very low, allow us to showcase the example for some of the consumers used at the beginning of the section, defined in Figure 10.5a. In practice, it will probably be a much larger value. For this particular scenario, Theorem 10.3 holds, the game is concave and the Shapley value is in the core. Table 10.3 compares the cost (in cents) of each player without batteries E i , as assigned by the Shapley value φ or by two arbitrary imputations in the core ζ 1 and ζ 2 . The rightmost three columns contain the relative gain obtained by each player with each of the allocations. It can be seen that the Shapley value is fairer than the other two imputations and, because the game is concave, it is not weakly dominated by the other two.

E i φ ζ 1 ζ 2 R.G φ R.G ζ 2 R.G

Discussion

While some of the hypotheses employed for the analysis might not be fully satisfied in practice, we believe that the analytical results do have practical value. Indeed, consider the player that decides to start the investment process: evidently, she has to search for possible partners. If the array of options is too wide, she might opt for approaching only some of them. Given the combinatorial nature of this problem, she would like to know beforehand with whom she is most likely to find a good solution. Our findings indicate that such solutions would try to combine as many heterogeneous profiles as possible and a similar number of participants of each "type". This can be motivated as follows: from the examples in Subsection 10.4.5, we know that, if players have quite similar consumption profiles, then the game is not concave. Furthermore, Theorem 10.2 suggests that, if there is a consumption "type" more popular than others, allocations in the core will make them pay more. Future work should focus on understanding to which extent these observations hold.

Another possible concern is that the applicability of a shared investment proposal based on a fixed consumption profile might not be reasonable. In this respect, we can suggest two possible workarounds. First, we propose to gather several consumption profiles for each user and analyze a best/worst scenario. This could help decide whether the investment is feasible or not. Second, although we described the solution as a shared investment, it might be even more realistic to consider that a third party (e.g., a utility) could offer storage as a service. In this case, the results provided by this study can be used as a tool to decide how much to charge consumers, as a function of their usage of the battery, in a post-settlement base (pricing policy). Using a payment that guarantees that the grand-coalition is stable can be interpreted as follows: consumers are satisfied with what they are paying and have no reason to form a "coalition" that would depart and choose a different provider. Observe that, in this case, the assumption of a fixed profile is not necessarily restrictive, since it is used to bill a "fixed consumption" that has already taken place.

Key Outcomes and Conclusion

We showed that the shared investment on local energy storage systems can be profitable for a realistic battery model and that the overall earnings of a coalition are sensitive to batteries' ramp constraints and to the intraday consumption profiles of consumers. Using real data, we demonstrated that, in several state-of-the art battery products, ramp constraints are tight and thus affect the outcome of the game, while in some other products there is no benefit from cooperation. We modeled the investment problem using cooperative game theory and for interesting classes of storage games we proved the existence of stable solutions. In particular, a family of storage games with ramp constraints was shown to be equivalent to a subset of the well-studied "glove market games". In the future, we plan to consider several important questions and extensions, including: stochastic consumption profiles; examining the impact of batteries degradation (batteries average lifetime) on the overall profitability; and taking into account that some of the players might be solar producers and hence the battery can be charged with their energy surplus.

We showed a large number of interesting families and cases, but the model is far from being deployable at this stage, since some of the assumptions are quite restrictive:

• The families studied are not comprehensive and finding a payoff in the core for the rest is intractable for large number of players.

• Considering a continuous battery size and deterministic load profile might not be an accurate enough model for a long term investment.

In the next chapter, we show how to deal with some of these constraints.

Chapter11

A model for Coalitional Storage Games Using Linear Programming Games

In the previous chapter, we introduced a cooperative game in which players can invest together in community storage and they benefit from it buy making the most out of their Time-of-Use tariff. Several properties were proved for such games, but one remained unanswered: is the core of such game always non-empty? At the time, we left the question unanswered. It took us almost an additional year to draw the connections with another family of cooperative games that allowed us to exploit the framework of cooperative investments to its full potential. Since this model is an extension of the one in the previous chapter, it fits within the LEC framework in the same way as the other one.

Formulating Coalitional Storage Games as Linear Programs

In the previous chapter, we defined the cost of a coalition in a coalitional storage game using the minimization problem present in Equation (10.5). We repeat it here for completeness

H S (B) = πS -P X S (B) + p h 1 0 l S (t)dt X S (B) = min 1 0 min{l S (t), δS}dt , S v(S) = min S H S (B) (11.1)
Even though it is succinct, the nested minimum structure in it does not make it amenable to the theoretical analysis that we wish to pursue in this chapter. Therefore, we seek to find an equivalent but more useful way to represent the cost of each coalition.

Before we continue, we introduce the following notation. Let B = (S, P, L, δ) be a battery with capacity B that costs P per kW h, has a life expectancy of L days and has a maximum discharge rate of δ (again, to simplify notation, we assume that the discharging capacity is -δ).

In cooperative game theory, one of the most well known studied games are linear production games (LPG) [START_REF] Borm | Operations research games: A survey[END_REF], [START_REF] Owen | On the core of linear production games[END_REF]. In this families of games, each player has some resources and the value of each coalition is determined by how much the coalition can produce with the collection of the resources of all its players. In LPG, the cost of each coalition is given by a linear program and by exploiting the duality of such linear programs it can be proved that all LPGs have non-empty cores. Inspired by these findings, we shall formulate the minimization problem in (11.1) as a linear program and frame it as an LPG.

The linear programming problem equivalent to formulation (11.1) that computes the cost of a coalition is given by Equation (11.2a). The formulation does not include the cost that is normally incurred during the cheap period. This value is always constant and additive, hence to simplify the notation, from now onward we will assume that it is always 0, i.e., l S t = 0, t = 1, . .

S + E + -E -= t (δS -e - t ) (µ), (11.2c)
S, E + , E -, e + , e -≥ 0 (11.2d)

The newly introduced variables are interpreted as follows:

1. e + : For each time slot, is the amount of energy required beyond the maximum ramp constraint.

2. e -: For each time slot is the amount of energy below the maximum ramp constraint.

3. E + : Is the total amount of energy consumed beyond the battery capacity, taking into account only the energy that could be stored, i.e, ignoring the energy that requires more power than the ramp constraint.

4. E -: Gap between the energy required and the total amount of energy in the battery.

5. λ t and µ are the dual variables associated with each of the equations.

Proposition 11.1. The optimization problems (11.1) and (11.2a) are equivalent.

Proof.

The proof can be found in Appendix C.3.

Proposition 11.2. The optimization problem (11.2a) can be written as:

min{c cd x : A cd x ≤ b cd (S), x ≥ 0}
, where the subscript cd denotes that it is the matrix notation of the continuous and deterministic version of the problem. The explicit formulation is in Appendix C.2.

Theorem 11.1. The cooperative game in which the cost of each coalition is given as the solution of the optimization problem (11.2a) satisfies the hypothesis of 9.1 and therefore it has a non-empty core.

Proof. The proof is quite simple. It suffices to show that we can re-write optimization problem with a fixed cost vector and matrix constraint that does not depend on the coalition. Indeed, this is the case as the constants that depend on the players (only l i t ) appear already as right-hand side constraints only. Therefore, it follows form an application of the Bondareva theorem.

Although the core of a game is probably the most well-studied equilibrium concept in cooperative game theory, its main limitation is that, in general, it is N Pcomplete to find an imputation in the core. This is a consequence of the fact that the number of constraints that define the core grows exponentially with the number of players.

Fortunately, borrowing ideas from the literature on linear programming games, we can obtain an element of the core by solving a simple optimization problem. The idea is captured in the following theorem.

Theorem 11.2. The payoff vector defined as y = (y 1 , y 2 , . . . , y N ) where u i is defined as in Equation (11.3) is in the core when the cost of each coalition is given by the optimization problem (11.2a).

y i = T t=1 l i t w * t (N ) (11.3)
where the dual of Optimization Problem (11.2a) is given by

P roblem D S ) maximize w b cd (S) T w (11.4a) subject to A T cd w ≤ c cd (11.4b)
and w * (N ) = (w * 1 (N ), . . . , w * T +1 (N ) denotes an optimal solution of the dual of the grand coalition.

Proof. The proof follows directly by applying theorem 9.3

Extending the model: Stochasticity

The model that we have explored so far allows players to share the cost of an investment in a battery using as a reference the consumption in a single day. The framework can also be used to analyze how much each player should pay a posteriori given their consumption profiles. Nevertheless, because of the stochastic nature of electricity consumption, it is reasonable to expect that the optimal battery capacity in the shared investment changes if the stochasticity of the load is taken into account. This observation is illustrated by the following example.

Example 11.1. Consider a single player that consumes energy during a single time-slot. There are two scenarios: Ω = {w 0 , w 1 }, which occur with probability p 0 = p 1 = 1 2 , respectively. In scenario w 0 , the consumption of the player is l = 0, while in the second scenario it is l = 3. Furthermore, let the electricity prices satisfy p l = 1, p h = 5 and the battery be B = (1, 3, 1, ∞).

In the deterministic version, we would solve the optimization problem using the average consumption, namely 3 2 , in which case the optimal cost is given by the following expression:

v(B) = min B 3B + min{B, 3 2 } + 5 max{ 3 2 -B, 0} (11.5)
The optimal solution of the problem in Equation 11.5 is B * = 3 2 and v(B * ) = 6. Analogously, we can find that, for scenario w 0 , the optimal battery size is 0 at a total cost of 0, while for scenario w 1 , the optimal battery size is 3 for a total cost of 12.

We are interested in the cost incurred by buying the battery obtained solving the average case, but experiencing the real consumption in each of the two scenarios.

For scenario ω 0 , the cost is:

C ω0 ∆ = 3 × 3 2 + min{ 3 2 , 0} + 5 max{0 -3 2 , 0} = 9 2 and for scenario ω 1 it is: C ω1 ∆ = 3 × 3 2 + min{ 3 2 , 3} + 5 max{3 -3 2 , 0} = 27 2 .
Hence, the average cost experienced is:

C ω0 p + C ω1 (1 -p) = 1 2 9 2 + 1 2 27 2 = 9.
However, the cost could have been decreased by buying instead a battery of size 0 for a total cost of 7.5 with a reduction of 1.5 (16%). This value, 1.5, is known by the Stochastic Programming community (see [START_REF] Kall | Stochastic programming[END_REF]) as the Value of the Stochastic Solution: it is a measure of how much can be gained by considering the stochastic problem instead of the deterministic one.

We proceed to formalize the concepts that were introduced in Example 11.1.

Two stage stochastic optimization

To better model the investment problem, we extend our previous formulation to include the typical stochasticity present in the consumers loads. To that end, we consider a two-stage stochastic programming formulation of the 108CHAPTER 11. A MODEL FOR COALITIONAL STORAGE GAMES USING LINEAR PROGRAMMING GAMES investment problem. In the first stage of the problem, the actual energy consumption is unknown and the coalition has to decide on which battery they should acquire. In the second stage of the problem, the actual consumption is revealed and the coalition decides how to operate the acquired battery so as to minimize their costs, i.e., buy as much energy as possible during the cheap period and discharge during the expensive period.

Preserving the notation from the previous section, we can write the first stage of the problem as follows.

First stage) minimize

S P L S + E ω∈Ω [g(S, w)] (11.6a)
subject to S ≥ 0 (11.6b)

Similarly, the second stage can be written as follows.

g(S, w) = minimize E + (w), E -(w), e + (w), e -(w) p l (S -E -(w)) + p h (E + (w) + t e + t (w)) (11.7a
)

subject to δS + e + t (w) -e - t (w) = i∈S l i t (w) ∀t ∈ T , , (11.7b)
S + E + (w) -E -(w) = t (δS -e - t (w)), (11.7c)
E + , E -, e + , e -≥ 0 (11.7d)
Observe that the second stage of the problem depends on the battery size, which is known at that point. Each element w ∈ Ω corresponds to one scenario and encodes the electricity consumption of every player, so that w ∈ R T ×N .

When the sample space Ω is finite, the expectation can be written as a sum, and the two-stage problem can be written as a very large LP described by the optimization problem (C.5), as detailed in Appendix C.4. For the investment problem, it is more likely that the distribution of consumption profiles has a continuous support. In that case, we construct a set of scenarios (outcomes) Ω = {w 0 , . . . , w m } with associated probabilities p w0 , . . . , p wm such that the two distributions, namely the original one and the new one are close with respect to some metric such as the Wasserstein-distance d 1 [START_REF] Ch | Scenario tree generation for multiperiod financial optimization by optimal discretization[END_REF]. In what follows, we shall assume that we are working with a discretized distribution function of the consumption profiles with finite support denoted by Ω.

Theorem 11.3. The cooperative game defined by using optimization problem (C.5) as the cost of each coalition is balanced and, therefore, it has a non-empty core.

Proof. See Appendix C.6.

Using the new model, we can solve Example 11.1. The optimal solution is not to buy a battery (B * = 0) for a total cost of 15 2 . As in the deterministic-continuous case, we do not need to find the cost of every coalition in order to find a vector in the core, and we can resort to the dual as in the previous case.

Theorem 11.4. The payoff vector y = (v 1 , v 2 , . . . , v N ) is in the core, where v i is given by:

v i = ω∈Ω T +1 t=1 x i t (ω)m * t (ω)(N ) (11.8)
where m * t (ω)(N ) is the optimal solution of the dual of (C.5) which is given by (11.9a) associated with scenario ω for the problem associated to the Grand Coalition in time-slot t.

D S ) maximize m b T cs m (11.9a) subject to X T m ≤ c cs (11.9b)
Proof. Analogous to the deterministic case.

On the construction of scenarios

As we mentioned at the end of Subsection 11.2.1, ultimately we need to deal with a discretization of the consumption profile distribution instead of the actual continuous one. This amounts to find a finite set Ω of scenarios with their respective probabilities that behaves similarly to the original distribution. There are many approaches on how to produce optimal scenarios such as [START_REF] Defourny | Scenario trees and policy selection for multistage stochastic programming using machine learning[END_REF] or [START_REF] Ch | Scenario tree generation for multiperiod financial optimization by optimal discretization[END_REF]. However, the generation of an optimal scenario is out of the scope of this chapter. Instead, we observe that we can use the consumption of each player during a single day as a suitable scenario. By doing so, the one thing left to be decided is how many scenarios should be employed. This question is addressed in Subsection 11.4.1, where we numerically assess the sensitivity of the solution on the number of scenarios.

If the model were to be extended by using a multi-stage stochastic optimization problem instead of a twostage model, the number of scenarios would increase drastically. One possible solution would be to manually craft the scenarios so that they capture the most representative aspects of the load patterns while maintaining a low complexity. We have not investigated such an approach yet.

Extending the model: Discreteness

After extending the model in order to cope with stochasticity, we next address the need to cope with the fact that batteries come in discrete sizes. That is, we would like to change the optimization problem of each coalition so that it considers only discrete battery sizes (i.e., multiples of the original battery size). Unfortunately, the technique we used to prove the existence of the core for the (deterministic and stochastic) continuous cases no longer works. This is due to the fact that it relied upon the duality gap, which cannot be guaranteed to be 0 in the discrete case.

Remark 11.1. Because we will need to refer and compare the characteristic value function of the continuous and discrete versions of the game simultaneously, we shall use the following notation: v c (S) shall denote the value of coalition S in the continuous version of the game while v d (S) will stand for the value in the discrete version.

Our analysis of the discrete model begins then with the observation that, if we know the solution of the continuous problem, then the solution of the discrete problem is not too different. Formally: Theorem 11.5. The optimal battery size for a coalition S in the discrete setting is given by B ↑ or B ↓ , where B ↑ is the smallest multiple of B greater than β and B ↓ is the largest positive multiple of B smaller or equal than β. In this context, β is the optimal battery size for the coalition in the continuous setting.

Proof.

See Appendix C.5.

From the above result we can derive the following lemma:

Lemma 11.1. 0 ≤ v d (N ) -v c (N ) ≤ f (B, p l , p h ).
That is, the integrality gap does not depend on the net consumption of players.

110CHAPTER 11. A MODEL FOR COALITIONAL STORAGE GAMES USING LINEAR PROGRAMMING GAMES Proof. Observe that, by changing the battery size by S kWh where S is the size of one unit of the battery, the cost will be affected by two factors: the change in the cost of the battery and the maximum energy than can be shifted from the expensive period to the cheap period. In either case, the bounds on those quantities depend only on the battery and the prices, but not on the consumption profiles.

Lemma 11.1 hints that, as the number of players grows, the relative cost of the rounding error becomes smaller.

Having introduced some general properties of discrete storage games, we proceed to show that the core of such a game may be empty, in contrast with the results obtained for continuous battery sizes.

An example with an empty core

Consider a coalitional storage game with a single time slot, and with a battery that can be bought in multiples of 1 kWh. Furthermore, assume that the ramp constraints are large enough so that they do not play a role, i.e., e + = 0, S + E + -E -= i∈S l i . Let X be the consumption of an arbitrary coalition; then, the cost of the coalition is defined as the solution of the optimization problem (11.10a).

P 1 minimize S, E + , E -, e + , e - P L S + p l (S -E -) + p h (E + + e + ) (11.10a
)

subject to δS + e + -e -= X, (11.10b)
S + E + -E -= δS -e -= X, (11.10c)
S ∈ Z + E + , E -, e + , e -≥ 0 (11.10d)
From (11.10c) and Theorem 11.5, we know that the only possible solution of the above optimization problem is either:

B = X, B ↑ , or B ↓ , with ∆ = B ↑ -X.
The total costs incurred in each of these three cases are summarized in Table 11.1. The first case, C 1 , only occurs if X ∈ Z. The second case will take place over the third one only if its associated cost is smaller, which occurs when the following condition is satisfied:

∆ < p h -P L -p l p h -p l .
Table 11.1: The cost in each of the three cases Case Battery size Total cost

C 1 B = X X(π + p l ) C 2 B = B ↑ X(π + p l ) + ∆π C 3 B = B ↓ X(π + p l ) + (1 -∆)(p h -π -pl)
Having detailed the solution for a simple game, let us consider an example. Example 11.2. Let N = 3 and T = 1, such that the consumption of each of the three players is given by: l 1 = 0.9, l 2 = 0.6, l 3 = 0.4. Furthermore, the prices satisfy p h = 0.55, p l = 0.2, P L = 0.3 In Table 11.2, the costs of each coalition for several battery sizes are shown, and the minimum cost, which coincides with the value of the coalition, is highlighted in blue.

For this game, the core is given as the vectors satisfying the following constraints: (11.11)- (11.15).

x 0 ≤ 0.48, x 1 ≤ 0.33, x 2 ≤ 0.22, (11.11)
x 0 + x 1 ≤ 0.775 (11.12)

x 0 + x 2 ≤ 0.665 (11.13)

x 1 + x 2 ≤ 0.5 (11.14)
x 0 + x 1 + x 2 = 0.98 (11.15)
Since x 1 + x 2 ≤ 0.5, we have x 0 ≥ 0.48 = 0.48. Substituting x 0 in Inequalities (11.12) and (11.13) we obtain: x 1 ≤ 0.295, x 2 ≤ 0.185. Now, it follows that x 0 + x 1 + x 2 ≤ 0.48 + 0.295 + 0.185 = 0.96 < 0.98 hence the problem is unfeasible.

Hence, for discrete battery sizes, the core may be empty. Furthermore, because the ramp constraints played no role in this case, the result is more general.

The -core of a discrete game

In view of the potentially empty core in the discrete case, we turn to consider a relaxed solution concept, namely the -core. The idea is that we allow each coalition to be unsatisfied with the payoff up to . In particular, with = 0 the solution coincides with the original concept of the core.

Because thecore is always non-empty for a sufficiently large value of , we are interested in bounding the value of for which we can guarantee that thecore is non-empty.

We begin by constructing an approximation of the core. This approximation will be used for obtaining a solution for the discrete version of the game, given that we cannot apply the dual approach for discrete battery sizes. Because we shall use this approximation as our solution, we are mostly interested in finding the value of associated with such a solution, rather than with a value that holds for every payoff.

Let y ∈ core(v c ) be a vector in the core of the continuous variant of the game, which is guaranteed to exit. For this vector we know that N y n = v c (N ), because every payoff in the core is efficient. We can then define the approximation of a vector in the core of the discrete game as:

ŷ = y v d (N ) v c (N ) (11.16)
If δ denotes the difference v d (N ) -v c (N ), we can rewrite (11.16) as:

ŷ = y + y δ v c (N ) (11.17)
This new vector is efficient in the discrete game, and in it, every player pays proportionally the same as in the continuous game. Yet, it might not satisfy group rationality for some coalitions (indeed, if the game has an empty core, it necessarily violates at least one of them). By defining M S (z) = i∈S z i -v d (S) to be the violation of the constraints associated with the coalition S, we can denote by M (z) = max S∈2 N M S (z) the largest violation of any constraint. It follows that y is in the core of the game if and only if M (y) ≤ 0. If M (y) > 0, by definition we know that y belongs to the M (y) -core of the game. Proposition 11.3. There exists satisfying

≤ v d (N ) -v c (N ) v c (N ) n∈N y n -min n y n ≤ δ 1 - min n y n v c (N )
such that the -core is nonempty, where y is a vector in the core of the continuous version of the game.

Proof.

We know that v d > v c . It follows that any vector that satisfies i∈S z i ≤ v c (S), also satisfies i∈S z i ≤ v d (S); hence, y satisfies all the inequality constraints that define the discrete core. When considering ŷ and the discrete core, we have that, for any coalition S, the inequality can be violated by at most i∈S y n δ vc(N ) is lower otherwise. Even though the Individual investment might seem strictly better than the Default one, it entails risks associated with it (buying storage without help) not included in the model. Consequently for cases in which players are risk-averse or the profits are small, the Default investment might be a better solution.

We obtained the consumption profile of each user from the AusGrid project [START_REF] Ratnam | Residential load and rooftop PV generation: an australian distribution network dataset[END_REF] 1 , which consists on data of 128 users between the 1st of July of 2012 and the 29th of June of 2013. The consumption is sampled every 30 minutes, so T = 48. In particular, because we are only interested in the consumption during the times when electricity is most expensive, which we assumed to be the second half of the day, we only keep the second half of the consumption of each day.

As mentioned above, we assume that the lower price spans the first half of the day, and the most expensive price spans the rest. As actual tariffs, we used reference prices from New Mexico [46]: p h = 18.9 cents per kWH and p l = 5.1 cents.

For the battery characteristics, we used Telsa's Powerwall 2 [4], for which we have B P W 2 = (13.5, 555.5, 365 × 15, 5), where we assumed a life expectancy of 15 years, slightly more than the warranty.

Unless explicitly stated otherwise, all games considered in what follows are of the discrete type.

Sensitivity to the size of the scenario set

We start by analyzing the impact of changing the look-back threshold in the individual payments as well as in the size of the battery. By look-back threshold we mean the number of scenarios used in the stochastic optimization problem. Recall that we assume that each day is equivalent to one scenario and that the probability of observing the load of any given day is the same as in any other day. We create 9 coalitional games with 15, 30 and 45 consumers by sampling the consumer's id and the period of the year from the dataset. For each of the 27 games, we vary the number of scenarios used from 1 to 45.

In Figure 11.1, the change in the optimal battery size for each coalition is plotted as a function of the number of scenarios considered and each line corresponds to one of the different games. We observe that, in most games, the optimal battery size tends to stabilize around 35 -40 scenarios. This observation shall be employed in future simulations in order to select the number of scenarios.

We proceed to consider the accuracy of the approximated solution for discrete games.

Accuracy of the approximation

One way to measure the accuracy of the approximation is to consider the empirical value of that defines the -core.

If we can compute the value of each of the 2 N -1 possible coalitions, then we could check if our approximation is within the core and if so, how far. We will proceed to do so, but only for small values of N . An important observation is that the absolute value of the violation does not satisfactorily capture the accuracy of the approximation: a value of = 1 when the order of magnitude is a million can be small, but if it is in the order of the tenths it might be too large. To account for this, we consider the value of the violation divided by the and the investment is made with consumption profiles that are not very representative of the average consumption of those users. One example of that is assigning the same probability to an extreme peak of consumption. For example, if the average consumption of a player is X kWh per day but the scenarios used to determine the cost division coincide with all the days in which he/she consumes above X, then the outcome will not be favourable for he/she. P1 P2 P3 P4 P5 P6 P7 P8 P9 P10 A thorough assessment of the performance is provided in Table 11.4. For each combination of the considered parameters, the table depicts the percentage of users (on average) that are better off by using one cost assignment technique over the other. As an example, consider the two highlighted cells in the table. The blue one indicates that, for the simulations with 45 consumers, 45 scenarios and 15 test days, only 12% of the consumers preferred the Individual investment over the Default one (all the rest preferred the Default). On the other hand, the red cell indicates that 60% of the players preferred the Keep Proportion technique versus doing nothing, in the games with 30 players, 30 test days and 45 scenario days. We observe that the Keep Proportion technique has very different outcomes: for about half of the players it is profitable, yet for the other half it is not. Nevertheless, we observe that not investing at all in storage is always worse than joining the grand coalition and paying a cost calculated using the Re-solving technique. Furthermore, we consider the increase in storage owned by the players while participating in the cooperative scheme. For the same scenarios as in Table 11.4, Table 11.5 depicts the average increase in the number of batteries and the percentage of change. It can be seen that cooperation duplicates or even triplicates the number of storage owned by players in consideration (with respect to buying storage individually). Furthermore, knowing that using the payment scheme Re-solving no player is worse than doing nothing, we conclude that coalitional storage games are capable of increasing the amount of storage in the community while satisfying individual needs.

Conclusions and Future Work

In this study we extended the coalitional storage game for modeling the shared investment in storage. We provided two extensions, namely, considering discrete battery sizes and accommodating a stochastic representation of the load. We believe that these extensions are important steps towards the deployment of the theoretical findings of this line of work in real-life settings. We have shown that the cooperative investment is always profitable when continuous batteries are considered and almost always profitable in the case of discrete batteries. Furthermore, we provided computationally-efficient algorithms for finding such solutions. These solutions specify how the costs should be split among players.

It follows from our theoretical model that the shared investment is profitable for players when their real consumption distribution is taken into account. Unfortunately, our numerical results indicate that using an approximation of such consumption profiles can lead to unsatisfied players in the long run. Thus, constructing better approximations of the considered profiles is one of our intended directions for future work.

Introduction and Problem Overview

In this chapter, we study the existence of stable cooperation agreements among households with renewable energy generation resources and, optionally, storage capacity. A coalition is considered to be stable if the distribution of the gains is satisfactory for all participants; which means that no strict subset of participants will benefit from leaving the coalition. Indeed, we consider that the main objective of forming coalitions is to optimize the cost of the energy that has to be exchanged with the traditional electricity company (usually, at any given time, these companies sell energy at a higher price than they buy locally generated energy). Our contributions are the following:

1. We prove that the centralized solution to the cooperative game introduced in [59] can be found efficiently using techniques from the theory of Linear Programming Games [START_REF] Samet | On the core and dual set of linear programming games[END_REF], instead of being NP-Compelete as previously believed.

2. We provide a distributed algorithm to compute the aforementioned solution, in which each participant of the game can retain its private information. Previous proposed approaches require players to submit private information to a centralized solver.

3. In our distributed algorithm, a communication network enables players to exchange their updated solutions. We evaluate numerically several families of such network topologies and conclude that faster convergence of our algorithms is obtained with topologies based on expander graphs.

The chapter is organized as follows. Section 12.2 introduces some required concepts and notation. In Section 12.3, we position our study within the literature. Our mathematical model is presented in Section 12.4. The distributed algorithm used to solve the problem is introduced in Section 12.5. Numerical results are presented and discussed in Section 12.6. A case study comparing the model with the previous non-cooperative energy trading mechanisms is presented in Section 12.7. Final remarks are presented in Section 12.8.

Preliminaries: Graph Theoretic Concepts

A graph G = (V G , E G ) is defined by a set of nodes V G and a set of edges E G . In this chapter we will assume that graphs are simple, weighted, undirected and have no loops. The adjacency matrix of G is denoted A G and its entries a vw are positive if and only if there is an edge between v and w, v, w ∈ V G . We define the degree of a node v ∈ V G as d v = w∈V G a vw . The degree matrix D G is a diagonal matrix with the degrees of the nodes in its diagonal. The neighbourhood of a node v ∈ V G , is defined as the set of adjacent vertices to it: N (v) = {w : a vw > 0} and the Laplacian of G by L G = D G -A G . We will be interested in the largest eigenvalue of L G , which we shall denote by µ G . We denote the complete graph in N nodes K N , the cycle graph in N nodes by C n , the line graph by P N , the wheel graph (namely, a cycle with a node connected to all other nodes) as W n and an arbitrary tree by T n .

We are particularly interested in expander graphs as these types of graphs have provided fast convergence for distributed algorithms in previous studies [START_REF] Duchi | Dual averaging for distributed optimization: Convergence analysis and network scaling[END_REF]. We require to additional definitions before defining expander graphs.

Let ∂A = {(v, w) : (v, w) ∈ E G , v ∈ A ⊆ V G , w ∈ V G \ A}.
be the edge boundary of a set A. We define the Cheeger constant as:

h(G) = min{ |∂A| |A| : A ⊂ V G , 0 < |A| ≤ 1 2 |V G |}.
[114]. With those definitions in place, a graph is said to be expander if it is sparse and its Cheeger constant is high. Intuitively, they are graphs that achieve good connectivity properties with only a few edges. Two types of expander graphs will be used in this chapter. First, a cycle chordal graph (denoted E n ) as defined in [START_REF] Lubotzky | Discrete groups, expanding graphs and invariant measures[END_REF]Theorem 4.4.2], which can be obtained by adding to a cycle, the edges that connect each node (numbered from 0 to p) to its inverso moduli p (p prime). Finally, we consider randomly sampled 4-regular graphs (which we will denote R 4,n ), as they are good expander graphs with high probability [START_REF] Friedman | On the second eigenvalue of random regular graphs[END_REF].

The reader is referred to Chapter 9 for cooperative game theory related concepts.

Related work

In [START_REF] Han | Incentivizing prosumer coalitions with energy management using cooperative game theory[END_REF], Han et al. propose the use of energy cooperatives among households with batteries and renewable energy generation with the target of minimizing the total cost paid to their Traditional Electricity Company (TEC). The authors model the energy cooperatives using cooperative game theory and prove that the considered game has a non-empty core. Their solution is based on the concept of nucleolus and on the Shapely value. Due to the complexity of solving the cooperative game, their simulations are run up to 14 players, which took some 500 minutes to solve.

In [START_REF] Han | Estimation of the shapley value of a peer-to-peer energy sharing game using coalitional stratified random sampling[END_REF], the authors study the same cooperative game as in [START_REF] Han | Incentivizing prosumer coalitions with energy management using cooperative game theory[END_REF] and propose a technique to estimate the Shapely value using stratified random sampling. They show that, using a high sampling rate, a game with 30 players can be solved in some 300 minutes, while a game with 50 players can be solved in the same amount of time by sacrificing some accuracy.

In [START_REF] Han | Improving the scalability of a prosumer cooperative game with k-means clustering[END_REF] the authors use K-means to cluster similar consumption profiles in order to reduce the number of players needed to solve the cooperative game introduced above. They found that the nucleolus obtained using the number of clusters instead of the number of players approximated reasonably well the nucleolus of the original game. With the proposed technique, they managed to solve a game with up to 200 players (albeit approximately).

In our study, we deal with the same cooperative game as in [START_REF] Han | Incentivizing prosumer coalitions with energy management using cooperative game theory[END_REF], [START_REF] Han | Estimation of the shapley value of a peer-to-peer energy sharing game using coalitional stratified random sampling[END_REF], [START_REF] Han | Improving the scalability of a prosumer cooperative game with k-means clustering[END_REF]. By using techniques based on Linear Production Games (LPG) [START_REF] Owen | On the core of linear production games[END_REF], we show that the core of the game is non-empty, and we derive a method for efficiently computing a payoff in the core that can scale up to several thousand players. Our second contribution consists in providing a distributed algorithm that computes such a payoff over any type of overlay communication network. In addition to the high efficiency, our solution has the key benefit that it does not require the disclosure by the households of local information.

In addition to the articles described above, there are several studies related to power systems from a cooperative approach. A review of different models and challenges associated with energy communities can be found in [START_REF] Prasad Koirala | Energetic communities for community energy: A review of key issues and trends shaping integrated community energy systems[END_REF]. In [START_REF] Moret | Energy collectives: A community and fairness based approach to future electricity markets[END_REF], the authors model energy communities that interface with the wholesale energy market via a community manager, but they do not analyze their problem through cooperative game theory nor their consumers have batteries. The authors in [START_REF] Darivianakis | A stochastic optimization approach to cooperative building energy management via an energy hub[END_REF] model a cooperative energy management system as a multi-stage stochastic optimization problem, but only focus on the optimization problem and not in the game theoretic aspects of cooperation. Kim et al. [START_REF] Kim | Direct energy trading of microgrids in distribution energy market[END_REF] model the trading of energy between microgrids using a Generalized Nash Bargaining solution. Their cooperative approach does not take into account the possibility of deviation of sub-coalitions.

Finally, cooperation for the shared investment in energy storage has also been extensively considered as discussed in the previous chapters.

Cooperative game model

We introduce now the model for the case when players collaborate. We model the system by a cooperative game formed by their ensemble. The main idea behind the cooperative game is that players in a coalition S ⊆ N will only be charged by their aggregated net load. That is, if players manage to synchronize the consumption in the coalition with their surplus, they will be able to reduce their costs. This is a direct application of our local energy trading framework in which the partial consumption profile of each players is chosen to be maximal and the distribution of the costs coincides with the core of the game.

In order to define a cooperative game, it is required to define the value of each coalition S. As explained above, in our case the value v S of a coalition S is (the minus of) the minimal electricity cost that players in S can achieve by cooperating.

Following [START_REF] Md | Optimal storage arbitrage under net metering using linear programming[END_REF], the value v S (same as v(S)) of a coalition S is given by the optimal solution of the linear maximization problem (12.1).

v S = max x i,+ ,x i,-: i∈S - T t=1 z S,+ t β t -z S,- t γ t (12.1)
subject to:

x i,+ , x i,-∈ X i ∀i ∈ S z S,+ t -z S,- t = i∈S x i,+ η i c -x i,-η i d + l i ∀t ∈ T x i,+ , x i,-≥ 0 ∀i ∈ S
where z S,+ t = max{z S t , 0}, z S,- t = max{-z S t , 0}, and z S t represents the net energy consumption of the coalition S at time-slot t. Although z S,+ , z S,-are deterministic functions of the charging and discharging profiles, we define them as variables for our formal treatment of the problem. We will prove that the optimal solutions of (12.1) are well behaved, i.e., at every time-slot x i,+ t and x i,- t cannot both be strictly positive. The same holds for z S,+ , z S,-.

Observation 12.1. The coalitional game whose characteristic value function is given by the value of (12.1) is not an LPG. This can be seen from the fact that the cost function is piece-wise linear and the fact that the variables z S,+ , z S,-are not associated with a single player, as required by Definition 9.12.

In spite of Observation 12.1, we shall be able to maintain the results obtained for LPGs (namely, Theorem 9.2) using the same techniques as in [START_REF] Owen | On the core of linear production games[END_REF]. First, we need to formulate the optimization problem defined in (12.1) as in Equation (9.8).

We shall use the notation V = (α|a)(β|b) . . . (ζ|z) to represent the vector V = ( With each player i, we will associate the vector b i defined as follows:

b i = (0|i4T -4T ) I (S i -S i s |T ) II (S i s |T ) III (δ i |T ) IV (δ i |T ) . . . (12.2)
. . .

(0|(N -i)4T ) (l i 1 |1) . . . (l i T |1) V (-l i 1 |1) . . . (-l i T |1) V I
The vectors b i shall be used as the right-hand side constraints of linear programs. Therefore, with each entry, there will be an associated dual variable. The roman numbers indicated in Equation (12.2) will be used to crossreference those dual variables with the corresponding entry.

Let B be the matrix whose columns are the vectors b i , b S = Bt S , x a vector of variables ordered as in Equation (12.3) and c the cost vector defined in Equation (12.4).

x =(z S,+ 1 , . . . , z S,+ T , z S,- 1 , . . . , z S,- T , x 1,+ 1 , . . . , x 1,+ T , x 1,- The main advantage of the new formulation is that the matrix A and the cost c do not depend on the coalition. Furthermore, the right hand sides of the formulation are additive.

With the formulation introduced above, the dual D S associated with the linear programming formulation of the optimization problem (12.1) is given by D S ) min{ λ S Bt S : A T λ S ≥ c, λ S ≥ 0} (12.5) We are ready to prove the main results of the chapter.

Theorem 12.1. The cooperative game G = (N , v) where v S is obtained by solving optimization problem (12.1) has a non-empty core.

Proof. This follows from a direct application of Theorem 9.2.

Theorem 12.2. The payoff vector y = (y 1 , . . . , y N ) is in the core, where u i is defined by Equation (12.6).

y i = T t=1 [ λ I,i,t N S i + λ III,i,t N δ i + λ IV,i,t N δ i + λ V,i,t N l i t -λ V I,i,t N l i t ] = λ N , b {i} (12.6)
where λ X,i,t S is the optimal dual variable of the problem of coalition S associated with the constraint of player i at time-slot t, labeled with the roman number X in the Equation (12.2).

Proof. This follows from a direct application of Theorem 9.3.

Distributed algorithm for the dual

In the previous section we showed that the cooperative game considered in this chapter has a non-empty core. Furthermore, we showed that a pay-off in the core could be built from the dual of the optimization problem associated with the value of the grand coalition v N , D N . In this section we will establish a distributed algorithm to compute a vector in the core by solving the dual D N in which players do not need to transmit their private information.

We assume that the players can exchange information using a communication network represented by a graph B with adjacency matrix B. Each player holds an estimate of the solution of the dual D N and in each iteration of the algorithm, they send their estimate to all their neighbours (as defined by the network B). The algorithm finishes when the players reach a consensus, i.e., their estimates of the solution coincides.

We begin by rewriting D N in a way that exposes the information available to each player (the private information of player i is the vector b i ). Let A i ∈ M 4T N +2T,2T denote the block matrix in A obtained by restricting A to only the columns associated with the variables x i,+ and x i,-. Then, the dual D N can be written as:

D N ) min λ i∈N b {i} λ = f i ( λ) (12.7)
subject to: (12.8) i∈N Ω i (12.9) λ ≥ 0 (12.10)

where

Ω i = { λ : A T i λ ≥ 0, λ V,t -λ V I,t ∈ [-β t , γ t ], ∀t ∈ T } (12.11)
Furthermore, denote by Π i : R 4T N +2T → Ω i the projection into the set Ω i . Our implementation of a distributed algorithm is based on the consensus protocol introduced in Liu et al. [START_REF] Liu | Constrained consensus algorithms with fixed step size for distributed convex optimization over multiagent networks[END_REF]. The following result can be derived from [START_REF] Liu | Constrained consensus algorithms with fixed step size for distributed convex optimization over multiagent networks[END_REF].

Theorem 12.3. Under the following assumptions: 1. the sets Ω i are closed and convex, 2. the functions f i are convex on Ω i , differentiable and its gradient is Lipschitz continuous on Ω i , 3. the communication between players occurs through an undirected and connected graph, the iteration:

       x i k+1 = Π i x i k -q i k+1 q i k+1 = α ∇f i (x i k ) + w i k + j∈N (i) b ij (x i k -x j k ) w i k+1 = w i k + j∈N (i) b ij (x i k+1 -x j k+1 ) (12.12)
with α < 1 µ B converges to the optimum. We used the notation x i k to represent the estimate of the solution held by player i at iteration k and b ij (the (i, j) entry of the adjacency matrix B) to represent the weight of the edge between i and j in the graph B.

In our setting, the sets Ω i are closed and convex as they are defined as the finite intersection of half-spaces. The functions f i are linear, so they are convex in Ω i , differentiable and their gradient is Lipschitz. Proposition 12.2. In each iteration, each player needs to solve a projection operation Π i . Because the feasible set is linear, this accounts to solving a quadratic programming problem. Furthermore, observe that it is possible to implement the projection step for each player in a way that does not depend on the number of players.

Proof.

Let r j i be the j-th row of matrix A i . The row r j i = 0 if and only if the variables x i,+ or x i,-have positive coefficients in it (because A i is restricted to only those variables). In the original matrix A, there are only 6T rows in which x i,+ , x i,-or both appear (those associated with entries denoted with the roman numbers from I to VI in Equation (12.2)). Therefore, there are only 6T non-zero rows in A i , which proves that the size of the problem Π i does not depend on the number of players.

In Proposition 12.2, it was shown that the projection step in each iteration does not depend on the number of players, but only on the number of time-slots used. We conclude that the time required to reach consensus in the distributed algorithm depends on the number of players only through the required number of iterations before convergence and the size of the neighbourhood of each player in the graph B, but not on the size of the problem that needs to be solved at each iteration. In spite of this, the memory requirements do scale linearly with the number of players as each vector x i k+1 ∈ R 4T N +2T .

Proposition 12.3. The structure of the projection step Π i is the same for all players and time-slots.

Proof. The matrix A i by definition is obtained by linearizing the constraints in F i , and so all the projection steps have the same constraints and objective. The difference between them is that the projection step affect different coordinates of the dual λ. Nevertheless, this can be surmounted by selecting the appropriate coordinates to feed into the projection step and updating only those.

E n R 4,n W n K n P n T n C n

Numerical Experiments

In this section we provide numerical evidence on the (efficient) performance of our proposed algorithms 1The numerical experiments shall focus on the running time of the algorithms and not on the economic benefits of the cooperation. Such benefits have already been studied in [START_REF] Han | Improving the scalability of a prosumer cooperative game with k-means clustering[END_REF], [START_REF] Han | Constructing prosumer coalitions for energy cost savings using cooperative game theory[END_REF], [START_REF] Han | Estimation of the shapley value of a peer-to-peer energy sharing game using coalitional stratified random sampling[END_REF], [START_REF] Han | Incentivizing prosumer coalitions with energy management using cooperative game theory[END_REF], [START_REF] Feng | Coalitional game based transactive energy management in local energy communities[END_REF].

In our experiments, we assumed that all players owned energy storage and we modeled the characteristics of those devices based on a Tesla's Powerwall 2. That is, we considered that S i = 13.5 kW h, S i s = 0, δ i = 5 = δ kW for all i ∈ N . In Proposition 12.3 it was shown that the structure of the projection steps is the same for all players and time-slots. This fact can be exploited by the projection algorithm, which can reuse the steps of previous executions to increase its speed. In particular, we use the OSQP solver [START_REF] Stellato | OSQP: An operator splitting solver for quadratic programs[END_REF], [START_REF] Banjac | Infeasibility detection in the alternating direction method of multipliers for convex optimization[END_REF], [START_REF] Banjac | Embedded code generation using the OSQP solver[END_REF] to improve the running time of the projection step of each player.

12.6.1 On the impact of the network structure

We begin our numerical study by comparing the performance of the distributed algorithm using different graph families. For the experiments in this section, we assumed T = 10 time-slots and we sampled the demand profiles of players from a uniform distribution l i t ∼ U[-3, 3] i.i.d2 . We compare seven families of graphs: the complete graph in n nodes, the cycle in n nodes, a wheel graph, a path graph, a randomly sampled 4-regular graph (R 4,n ), a random tree and a cycle chordal graph.

In Figure 12.1, the running times of the distributed algorithm for the different families of graphs are shown. It can be seen that the regular and chordal graphs provide the better performance, followed by the W n , T n , P n , C n . The worst performance in running time is achieved by the complete graph.

Our results regarding the performance of the different families of graphs mirror those in Duchi et al. [START_REF] Duchi | Dual averaging for distributed optimization: Convergence analysis and network scaling[END_REF]. In [START_REF] Duchi | Dual averaging for distributed optimization: Convergence analysis and network scaling[END_REF], the authors prove (for a similar but different algorithm, introduced to solve a different problem) that the fastest convergence of their distributed algorithm is achieved by expander graphs.

To study more in detail the reason behind the poor performance of complete graphs, we measured the number of iterations before convergence. Figure 12.2 provides the result of such an experiment. All simulations were stopped after 10000 iterations, a threshold that was consistently reached by P n , T n and often by C n . On the other hand, simulations using K n as the underlying communication graph managed to converge before the cut-off, in the neighbourhood of the 6000th iteration.

The observed performance can be explained as follows. The running time of each iteration of the algorithm is dominated by two factors: the time required to project into the sets Ω i and the average size of the neighbourhood of each node. Observe that there are two vector additions for each player for each one of her neighbours. In path graphs or cycles, the last value does not depend on N , whereas in K n it does. In summary, expander graphs require the least number of iterations to converge and provide the best running times. Furthermore, complete graphs converge in less iterations than cycles and trees, but, depending on the specific implementation, might take longer overall.

E n R 4,n W n K n P n T n C n

On the impact of players on performance

In this subsection we benchmark three different approaches to solve the cooperative game discussed in this study. The first approach consists of using the traditional definition of the core, i.e., computing the value of 2 N -1 coalitions and then finding a vector inside the set. The second approach consists of solving the dual of the problem associated with the grand coalition and building a vector in the core (in a centralized fashion), as described in Theorem (12.2). Finally, the third benchmark consists of running the distributed version of the algorithm, as described in the previous section. From the results in the previous subsection, we know that the performance of the distributed algorithm is better when the underlying topology in an expander graph, so that is the topology used in our experiments. Figure 12.3 presents the results obtained by running the algorithms on an increasing number of players. The Naive implementation is known to be exponential in the number of players as the results show, so we computed its value only up to 14 players. The centralized algorithm proposed in this chapter has excellent scaling properties, and its much faster than the other alternatives. Finally, the distributed algorithm performs considerably better than the naive implementation, although it is slightly slower than its centralized counterpart. This is due to the fact that, as the number of players increases, so does the required time to reach consensus.

It should be observed that our implementation of the distributed algorithm has not been optimized, so there is room for further improvement.

Benchmark against LEM

We will finish this chapter by describing a numerical benchmark between the cooperative framework and the non cooperative one. To do this, we simulated using the same data, the cooperative model described in this chapter against the sequential LEMs described in Part II of this thesis.

Setup

We consider a set N of 50 players modeled as described in Chapter 4. The objective of each agent is to minimize their costs during 5 consecutive days, which accounts for 240 time-slots of 30 minutes each (each days consists on 48 time-slots). Fifty independent simulations are used for each of the mechanisms presented in the previous section. Each agent's load is obtained by sampling consumption profiles from the Ausgrid dataset [START_REF] Ratnam | Residential load and rooftop PV generation: an australian distribution network dataset[END_REF]. The dataset consists on 127 players, so for each simulation, 50 where sampled at random. Because the amount of renewable generation in the dataset is not sufficient to justify the need for trading among participants, for half of the players we increased their generation by adding a random uniform value during the time-slots during sun-hours.

We assume that players will only be able to know perfectly their consumption in the next time-slot (30 minutes), but for all the remaining time-slots they will use a forcast instead. The forecast will be calculated using the average consumption in previous day at the same hour and day of the week. This forecast was shown to offer a good trade-off between accuracy and simplicity in [START_REF] Kiedanski | Sensitivity to forecast errors in energy storage arbitrage for residential consumers[END_REF].

For the non-cooperative scheme, we assume that players have a simple belief about future prices of the market. That is, PS t = 1.2γ t , PB t = 0.8β t if (t mod 48) ∈ [12,[START_REF] Adrian | Optimization of prosumers flexibility in electricity markets[END_REF] and QB t = QS t = K 0. In other words, they believe that during the time-slot when the sun shines the price in the market is 20% more competitive and there is enough capacity to trade. For the other time-slots, the price is exactly the same as offered by the TEC.

Players will be subscribed to an electricity tariff that will be held constant across the whole simulation. We will use two tariffs: a flat rate and a time-of-use tariff. Both of them have the same Feed-In-Tariff (price paid by the TEC to inject back into the grid) which is always lower than any of the buying prices.

Every player will have a battery. The characteristics of those batteries will be modelled after a Tesla's Power wall 2: S = 13.5 kW h, η c = η d = 0.95, δ = -δ = 5 kW .

Decisions will be taken in an iterative fashion, similar to Model Predictive control. In each time-slot, agents will run their optimization problem to obtain the set of optimal actions but will only implement the first one. This will be repeated for each time-slot. Because the procedure described above is computationally expensive, instead of solving the optimization problem until the end of the horizon, each player will use a rolling horizon (sliding window) of size 48.

In both, cooperative and non-cooperative schemes the players have access to the same information and take the same amount of decisions per time-slot.

Results

Before introducing the numerical results, we shall describe the two metrics studied. First, we look at the total cost incurred by the participants. This metric does not describe the gains of particular players, but captures how much capital is retained by the participants (in contrast to flowing to the TEC). Because the cooperative mechanisms minimizes the total cost by definition, we expect it to outperform the non-cooperative counterparts.

Secondly, we look at the total amount of energy exchanged with the main grid that was not dynamically locally matched. From the perspective of grid operators, this value represents the potential of communities to be selfsustainable. For a flat tariff, we know that the cooperative game should also minimize the amount of energy consumed externally. On the other hand, in the precedence of a ToU, the cooperative scheme might end up consuming more than the non-cooperative counterparts.

Because our implementation of the rolling horizon involves a cut-off, instead of a receding horizon in the last time-slots, players might end up with non-zero energy stored in their batteries. Let B denote the total amount of energy stored among all players. To normalize the results across simulations, we remove 0.95B kWh from the total traded energy with the main grid (because it will not have to be bought in the future) and 0.95B × F iT from the total cost (where F iT is the price of selling energy back to the grid). The later is a conservative measure, since it is less than the value saved by not buying energy in the future.

In images 12.4, we compare the cost and the exchanged energy with the main grid of four scenarios: a cooperative game with perfect information and one-shot solution (instead of using a rolling horizon), a cooperative game with imperfect information and a rolling horizon and the two non-cooperative games with imperfect information and a rolling-horizon. The values obtained are shown as a percentage of change with respect to a scenario without local energy matching rules in which players are independent of each other. A percentage equal to 0% indicates that the mechanisms performed the same as the case without mechanisms, a percentage lower than 0% indicates a positive reduction of the metric in question. Finally, a higher number indicates a degradation in performance. Indeed, we observe that our hypothesis are satisfied. The cost is greatly reduced by the cooperative schemes versus the non-cooperative ones, while the total exchanged load is barely the same between the different mechanisms.

Simulations were performed with varying number of batteries and players and the results remained similar. They are not presented here for lack of space.

Key Outcomes and Conclusion

In this chapter we study the formation of energy communities in which households with energy resources cooperate to minimize their electricity costs. We significantly enhance the results of previous works in this domain, making the solution scalable and providing a distributed algorithm for its implementation. These results are obtained by introducing an appropriate reformulation of the game used to model the system, which enabled us to harness techniques from the theory of Linear Programming Games. On these bases, we derive an efficient algorithm -with linear complexity in terms of the number of players -to find a vector in the core, that is, a distribution of the benefits among the players that guarantees that no subset of players could do better by leaving the cooperative. Furthermore, using consensus algorithms' results, we provide a distributed implementation of the algorithm in which players do not need to reveal their private information to any third party. Finally, through numerical simulations, we show that using expander graphs as the underling communication network provides the best running time for the distributed algorithm among several well-known families of graphs. Not only that, but we compared the model against its non cooperative counterpart and showed that it outperforms it by a wide margin.

Chapter13

A Joint Cooperative Model for Investment and Operation of Energy Storage Assets

Motivation for a New Model and Proposed System

In the three previous chapters, we went on to describe how cooperative games can be used to model the shared investment in energy storage by a group of players. Later, we also looked at how a similar approach can be used to control a fleet of batteries in a centralized or distributed fashion, while still providing sufficient incentives to players that guarantee that they will form the Grand Coalition.

When we looked at investment in batteries, the business model was only built around the idea that players who have contracted a Feed-in-Tariff can charge their battery during the cheap periods and discharging it during a more expensive one. We did not consider that players could have solar production of any kind, which would also decrease the cost that they would have to pay for their electricity. On the other hand, when we looked at the cooperative control of the fleet of batteries, we assumed that players already owned such devices, which need not to be the case.

In this chapter, we will show how both models can be merged into one: providing a mechanisms to invest in the technology needed, and operating it in a cooperative and efficient way. We will allow some players to have batteries and PVs of their own, while still investing in a cooperative one.

Looking back at the shared investments developed in previous chapters, the major weakness of model is the lack of specification of the origin of the capital needed to invest in such devices. To solve this issue, in this chapter we describe how the profits of cooperation can be separated in those arising from the hardware and those that are the result only of cooperation. In that way, we can envision that a third party will take care of the investment in hardware (if the return over investment is attractive enough), while the players themselves can enjoy the return over investment of cooperation alone (although nothing forbids the players in the coalition to partake in the hardware investment).

Contributions with Respect to the Previous Models

The contributions of this chapter can be summarized as follows. First, we extend the model for shared investment in batteries presented in chapter 11 in several ways. To do so, we allow players to invest in shared photovoltaic panels and not only on energy storage. Furthermore, we allow the tariff to be completely arbitrary as long as the price of buying is greater than the price of selling in every time-slot.

With respect to the model for cooperative control introduced in Chapter 12, we introduce one more device that needs to be controlled, the shared battery. In addition, we introduce local network charges that need to be paid even when trading inside the community. This represents a tax for the usage of the grid and would be modeled by the term Q in Equation (3.3).

Finally, as we explained above, we distinguish two kinds of return over investments, one for hardware (batteries and PVs) and one for cooperation. By doing so, we facilitate the process of exposing the possible profits that the different parties can guarantee.

Mathematical Model of the Join Cooperative Game

We will follow the conventions used so far to describe the feasible set of a player that owns a battery. Let X i be the feasible set of player i as described in Chapter 4. Furthermore, let X C (•) denote the feasible set of the shared battery, which depends on the size of the battery bought.

To model the investment in PV panels, we assume that there is certain solar irradiance / weather patterns that depend on each scenario and that the total production scales linearly with the size of the PV panels based on the external factors.

Putting it all together, the cost of a coalition is given by the following two-stage linear optimization problem:

v(S) = min B,V p BAT B + p P V V + E ω∈Ω [f (B, V, ω)]
subject to:

B, V ∈ N ( or R ≥0 ) (13.1) f (B, V, ω) = min x 1 ,...,x S ,x C T t=1 β S t max{z S t , 0} -γ S t max{-z S t , 0} + i∈S∪{C} T t=1 α|z i t |
subject to:

x i ∈ X i , ∀i ∈ S ∪ {C} z i t = l i t + max{x i t , 0} η c -max{-x i t , 0}η d , ∀t ∈ T , ∀i ∈ S ∪ {C} z S t = V I t + i∈S∪{C} l i t + max{x i t , 0} η c -max{-x i t , 0}η d , ∀t ∈ T (13.2)
with the convention that l C t = 0 and p BAT , p P V are the unitary price of the battery model and the PV model used respectively. The second term in the objective function are the local charges.

Theoretical Properties of the Join Model

We can proceed to recover most of the results that we have derived in previous chapters for the simpler versions of the model. Note that the neither the cost vector nor the constraint matrix depend on the coalition. We shall refer to that linear programming problem as (LP).

Proposition 13.2. The solution of (LP)coincides with the solution of (13.1).

Proof. The proof is almost analogous to the proof of Theorem 4.1. Observe that the new addition in the cost, the term α is constant as long as x i,+ -x i,-is constant, which results in the same conditions that we were in for the proof in the same chapter.

Naturally, the core of this new game is not empty and we can derive an efficient procedure to recover it.

Theorem 13.1. The core of the coalitional storage game in which the characteristic cost function is defined by the solution of the optimization problem (13.1) and (13.2) (real valued investment sizes) has a non-empty core and furthermore, the payoffs

y = b 1 , u , . . . , b N , u .
are in the core, where u is an optimal solution of the dual of (LP)associated with the Grand Coalition and

•, • denotes the inner product of two vectors.

Proof. This follows as a direct application of Theorem 9.2 and 9.3.

Because this new model is a generalization of the model presented in Chapter 11, we know that the core can be non-empty when considering discrete investments.

Finally, we can also recover results concerning the distributed computation of the core. In spite of this, this result will not be very useful since we will be performing MPC to compute the actual schedule of the battery and the distributed algorithm is amenable for solving one instance of the problem but not as many as we might require (it can be done, but it is not super efficient).

Computing Fair Payoffs After Observing the Data

One of the main challenges that we face when we try to apply the model described in this chapter is that the scenarios that we use to compute the core, need not to coincide exactly with the load observed by players, creating a mismatch between both values. Furthermore, it is unrealistic that players will have a perfect knowledge of their load. If that is the case, then they might use a forecasts instead (and a procedure similar to MPC), as described in Chapter 4. When doing so, the experienced costs (using forecasts), will be close to the optimal value (corroborated by the sensitivity experiments at the beginning of the thesis), but not exactly the same. In this regard, a meaningful question is how to compute a payoff for each player, in the light of the real experienced costs.

We begin by presenting an example.

Example 13.1. Consider a simple game with only two players and two time-slots. Each of this players has an energy consumption profile that is unknown. We consider that there are two possible scenarios and for each scenario there is one possible load profile for each player. Both scenarios are equally likely. We assume that players can invest in batteries and PVs. We assume a linear model for both where the cost and the capacity scale linearly.

For the battery, we will assume that the cost of 1 kWh of storage comes with 1kW ramp constraints, perfect efficiency and costs 1.

Regarding PVs, there is a production profile for each scenario: p a = p b = (1, 0) and 1 unit of profile costs 2 (buying units of PV scales linearly the production profile).

Players can buy 1 unit of battery and 1 unit of PV to obtain 1 unit of solar energy in the second time-slot at a cost of 3 (2 for the PV panel and 1 for the battery).

The price for buying energy in every time-slot is 4 and the price for selling energy is 0.

We summarize all the relevant information in Table 13.1

To derive the Table above, we made use of the fact that because the battery is perfectly efficient, the optimal investment can be found by solving:

min x 3x + 4 × 1 2 max{l a -x, 0} + 4 × 1 2 max{l b -x, 0}
where l a is the total load in scenario a, (analogous for b). In this case, for player 1 the optimal decision is to get a battery of battery and PV of size 0.2 resulting in a cost of 2.2 on average. For player 2, the optimal size is 0.5 and the cost is 2.5 on average.

On the other hand, if both players get together, their optimal decision is to get 1.2 units, reducing their total cost even further from 4.7 to 4.2 on average.

With this we can compute the characteristic cost function and find that the core is any payoff in which each player gets at most what they would get alone (2.2 and 2.5) and they share the extra 0.5 profit among them.

In this example, the core is given by all the vectors such that:

(2.2 - λ 2 , 2.5 - 1 -λ 2 ), λ ∈ [0, 1] 130CHAPTER 13.

A JOINT COOPERATIVE MODEL FOR INVESTMENT AND OPERATION OF ENERGY STORAGE AS

By taking λ = 0.5 we obtain a "fair" payoff in the core (1. 95, 2.25).

When analyzing the return of investments, we can consider two different categories of it: the profit obtained by investing in the hardware, ignoring the cooperation, and the profits obtain solely from cooperation.

We assume that external investors could get all the profit associated with the hardware ((2.4 -2.2) + (3 -2.5) = 0.7) and the players could get the remaining profit from cooperation, which is naturally the solution to the core (0.5 in this example). We can see the cooperative game described above as a tool to asses profitability (Return over investment) of a collaborative investment and the subsequent operation of the system.

In this subsection we explore what happens when we consider the actual experienced load profiles instead of the scenarios.

Example 13.2. Going back to Example 13.1, lets consider the case when scenario a occurs. The cost to both players if they cooperate is 3.6, the costs of each player when operating independently but investing in storage/PV are 0.6 and 3.5 and the costs without investment are 0.8 and 4.

One possibility to split the costs, is to find a payoff in the core of the original game and keep the same proportions with respect to the experienced cost. For example, if in the previous game we took λ = 0.5, the payoff of player 1 would have been 1.95 and of player 2 2.25. Then, when experiencing scenario a, we would assign a payoff of 1. 95 4.2 × 3.6 = 117 70 ∼ 1.6714 and for player 2: 2.25 4.2 × 3.6 = 27 14 ∼ 1.9285. We can observe that for such an instance, the ROI of the battery investment is still exactly 0.7, the ROI of cooperation is 0.5 but player 1 is paying a much higher cost than what she should have by being alone.

This goes to show that when experiencing specific scenarios, one player might "feel" that she would have been better without cooperation. In reality, if both scenarios are sampled effectively with probability 0.5, the expected payoff will be the same as the obtained theoretically.

Another interesting problem that players might face is that the real cost is greater than what it should be, because forecasts where used while operating the system. As an example, the final cost could be 3.6 + instead of only 3.6. Of course, if the information needed to compute the cost of the grand coalition is inaccurate, it has to be the case that the information used to compute the cost of sub coalitions is also inaccurate.

A Procedure to Allocate Empirical Payoffs

In the previous section, we presented a small example that dealt with inaccuracies at the moment of computing real payoffs of players. Here, we describe an heuristic procedure to tackle such problem.

First, we begin by the case in which the load is perfectly known (there are no errors while operating the batteries). Let y * be a core of the original game, using scenarios and let B * and V * be the optimal solutions of such game (discrete or continuous).

The vector y * will probably be used to estimate the savings of each player. Nevertheless, it does not reflect the actual costs that the players will incur as they consume energy in the days to come, as it is built on scenarios derived from the past.

What we propose to do is to extract from the payoff y * the proportion of it that corresponds to the investment in hardware, and use that throughout the future days, while recomputing the "core" associated with the specific load.

To obtain the payoff associated with the hardware investment, we can compute z * a payoff in the core of a variant of the original game. In this new game, the optimal PV and batteries quantities are fixed, and there costs is set to 0. By doing so, the payoff z * contains costs associated only with the operation of the system.

If we let = y * -z * , then represents one possible distribution of the costs associated solely to the hardware investment among players. We can later use i to charge each player the daily cost of the battery and PV bought.

Putting it all together, for a new day d, players can re compute z * d by replacing the different scenarios with only the real load experienced during that day and derive the final payoff y * d = z * d + for that day.

It is time for us to consider what happens when the load is not perfectly known. In those cases, players will have a forecast of it instead and will probably depend on a procedure such as MPC to control the system in question.

The value of each coalition with imperfect information v(S) = v(S) + δ(S) can be described as the value each coalition would have gotten with perfect information plus an error term that depends on the coalition.

One possible workaround for this would be to "scale" the core obtained using perfect information so that the total cost is the same as the cost with imperfect information. However, this new vector needs not to be in the core.

We propose a different approach inspired in the following result.

Theorem 13.2. If v and u are two cooperative games and y and w are two payoffs in the core of each game, then y + w = q is in the core of the game v + u = c.

Proof. First, it is easy to see that the new payoff is efficient.

i∈N q i = i∈N y i +w i = v(N )+u(N ) = c(N ).
Secondly, we show that all the required inequalities hold: i∈S q i = i∈S y i + w i ≤ v(S) + u(S) = c(S). This concludes the proof.

By virtue of Theorem 13.2, if we obtain a payoff of the "error term" game, we can obtain a payoff in the core of the imperfect information game by simply adding a payoff in the core of the perfect information game and the one from the error game.

Since in general finding a payoff in the core of a game is NP-complete, we will assume that the error terms behave nicely, so that we can model the error game and obtain a closed formula of a payoff in the core of that game.

One such example is as follows: is in the core of the error game.

To summarize, we need to obtain a payoff vector for each day after the initial investment is made. Following the ideas introduced in this section, we propose to build this payoff by using Algorithm 3. The first step (1) of the algorithm consists in solving the discrete version of the game and finding the optimal value of the Grand Coalition and the sizes of the PV as well as the battery. The second step (2) of the algorithm is to approximate a solution in the core of the discrete game with scenarios by solving the continuous version and scaling the vector so that it is efficient. In the third step the same problem but fixing first the variables to their optimal discrete sizes and changing their costs to 0. The cost corresponding to the hardware purchase assigned to each player h c is calculated then. We proceed to initialize the total payments of each player, which we will increment after observing the load of each day and calculating the corresponding payoffs. Step seven (7) solves the continuous game with fixed hardware sizes for the real error-free load in day d. The core of that game is calculated and the solution will be used to calculate the final payments of each player in that day. In the eight step (8) the core of the gap game is calculated using the errors between the value of each coalition as obtained doing MPC and the value that could have been obtained using the real load instead. To solve this problem we assume that the errors can be modeled following Proposition 13.3. Finally, the cost of each player in day d is calculated and added to his/her running bill.

Numerical Experiments

We conclude this chapter with the results of some numerical simulations. The objective of these numerical experiments will be to analyze on real data the ROI for hardware and cooperation of the methodology proposed. Secondly, we assess the Return over Investments for hardware and for cooperation in the different simulations. Table 13.2 depicts the percentage of change with respect to the cooperation ROI and the hardware ROI for different simulations. On top of this, the last column depicts relative importance of cooperation with respect to the total costs incurred by the players. From the first two columns we learn that the variance between the theoretical ROI and the experimental one are significant. This can be a problem as it makes it harder to asses the profitability of the shared investment. We conjecture that this is due to the very simple procedure used to compute the stochastic scenarios.

On the other hand, looking at the last column, we observe how as the price of batteries increases or as the number of batteries to control increases, the profit obtain by cooperation is higher. Not only this is to be expected, but also the profits attained ( 15%) are quite interesting.

Finally, we numerically assess the distance between the optimal sizes of the discrete and the continuous solutions. Figure 13.2 shows the distribution of the distances across all simulations. It can be seen the optimal battery only differers up to rounding errors, while the PV can change up to two units in some cases.

Key Outcomes and Conclusions

This chapter closes the part of the thesis that deals with the analysis of cooperative mechanisms for local energy trading. In this part we have introduced two main models: a shared investment in hardware (PVs and batteries) and a model for a centralized (or distributed) control of a fleet of batteries. This chapter showed that both models can complement each other and that the theoretical tools at our disposal still allow us to guarantee the existence of stable solutions.

Furthermore, this chapter addressed one of the biggest challenges behind the investment model: the origin of the capital required. A method was described to identify two sources of profit, hardware and cooperation, and it was argued that the profit in hardware can be given to an external party willing to provide the required sum of money. This is very similar in spirit to the concept of Citizen Energy Community introduced in the Clean Energy Package in Europe and that has seen many new projects in development.

From the questions that remain to be answered, probably the most important one is how much can the model improve if we increase the accuracy of the scenarios. This shall be the next focus of our research. In spite of this, I am confident in suggesting cooperative approaches in place of competitive ones to implement local energy trading in power grids transitioning to Smart Grids.

Finally, the model introduced in the part of the thesis assumes that the objective function of electricity prices and in particular a Time-of-Use tariff. Such tariff limits the interaction between the community and the grid operator, which will become increasingly important. Future work should find the way to integrate the grid operator in decision problem of the players.

Chapter14

General Conclusions

An Integrated Overview of Main Results Presented in this Thesis

The power system is evolving towards the Smart Grid paradigm, aiming to reduce its overall CO2 emissions and to improve its ability to host an increasing amount of renewable energy sources and to feed a rapidly changing electricity demand structure. This process will entail major changes in the way the system is constituted and operated. Among these several changes, it has been proposed to put consumers in a more central position within the system (figuratively). That is, move away from the historic structure in which consumers were simply buyers of energy to a new model in which they actively take part in the process of acquiring and producing their energy. This is partly motivated by the idea that a consumers, when faced with the choice of green, locally or self-produced energy versus "dirty", fossil-fuel-based, bulk produced energy, they will opt for the former.

These new structures can take many forms. The legislation in Europe is consistent with this [3], [START_REF] Hancher | The eu winter package: briefing paper[END_REF], where several definitions such as "Citizen Energy Communities" and "Renewable Energy Communities" have been created, but without a very specific definition of what they entail.

Throughout the thesis, we embarked in the study of some of these models, in which the participation and decision about consumption were delegated to an automatic device in charge of optimizing its owner's utility.

This allowed us to benefit from the mathematical formalism of game theory, in which agents are assumed to be rational. This assumption, although not very realistic when dealing with humans, fits the agents and players studied here quite well.

We explored two types of interactions among these players: one in which participants cooperated and others in which they did not.

When considering competing agents, we focused on auction mechanisms for trading energy among neighbors. First, we studied sequential double-sided auctions where we found that not only they increase the complexity of the decision problem faced by its participants, but they can also create incentives that result in large peaks of consumption. Later, we designed a new auction mechanism borrowing ideas from combinatorial auctions and drawing from our specific knowledge domain. As a result, we obtained a market capable of achieving higher welfare among prosumers.

Regarding cooperative models, we studied two problems: shared investments in energy storage and photovoltaic panels and the cooperative control of a fleet of batteries. For both problems we showed that stable solutions exists. Furthermore, we showed that both problems can be integrated into a single framework for assessing the profitability of a shared investment followed by a cooperative control of the shared assets. In a certain way, what we did was to show that the centralized operation of storage is better than its distributed counterpart (which is trivial), but also that incentives exists that promote such cooperative use.

For both types we have analyzed their advantages and disadvantages, obtaining valuable insights for the design of local energy trading systems. Comparatively we found that competitive environments are much easier to implement, they simply need to be created and players allowed to participate. On the other hand, cooperation requires the ability to create and enforce contracts, so that players are "forced to cooperate".

The literature on energy trading is mostly focused on the competitive implementations and models, whereas the new European regulation seems to favor a cooperative model. The later, as we have shown throughout the thesis, results in higher profits for the participants and improved welfare whenever they can be implemented.

That is not to say that every competitive environment is doomed to fail. If the is a lesson that can be taken from this thesis is that the behavioral changes induced by local energy trading are quite complex and sensitive to the many parameters that compose such systems (for example, the peaks before the change of ToU price introduced in Chapter 6). In a way, this thesis provides a hope and a warning. Hope in that we show how much these systems can be designed, tuned and built to improve the social welfare of its participants and a warning knowing that if those design are careless, the consequences can be quite damaging for the power system.

Discussion on the Potential Impact of the Results

If batteries were to mimic the narrative of photovoltaic panels, their prices (and therefore their massive adoption) will only drop considerably after large deployments are executed at very high prices (such as the current ones). Given the current prices of electricity, PV panels and energy storage systems, it is unlikely that market-based frameworks will provide enough incentives for end-customers to unilaterally invest in storage solely for the profit of it. Until then, shared investments (especially in buildings) will have the potential to increase the demand in batteries while the solo alternative is not interesting enough.

The cooperative models developed in the last part of this thesis show that it is possible to prove that cooperation is not only profitable but it is also amenable to tractable computational models. I don't believe that the models presented here will necessarily be the same as the models that will be implemented in the following years, there are yet challenges regarding the updating of contracts that need to be studied in more detail. What I do think is that they can serve as inspiration for innovative business models in which the core idea is cooperation, (such as in energy cooperatives which are becoming increasingly popular in France) and not competition.

In Appendix B, we present the outline of a benchmark for grid flexibility. I see the lack of a benchmark for flexibility as a major obstacle in the massive deployment of the markets and communities discussed throughout the thesis. Providing the legal framework for these frameworks and putting them into practice costs considerable amount of money that will translate in a more resilient and clean power grid. But for how much? How do these programs compare to a national program to upgrade "dumb" into smart water heaters ? I envision that some of this frameworks will face sever push backs from legislators once we realize that we can not quantify the benefits of the proposed changes in comparison to well known approaches (we might not be able to quantify the benefits of those either, but since they have been around for a longer time, they are "safer").

Implementation barriers for Energy Cooperatives Nowadays, the price at which consumers pay for their energy is mostly driven by taxes related to the usage of the power grid. It has been proposed that participants engaging in local energy trading should be subject to a special tariff that reflects the fact that, for example, they might use less the transmission grid. Even though such special tariffs, sometimes referred as wheel charges, are been actively discussed, they have not been deployed in most countries. Until such situation changes, the assumption introduced in Chapter 2 that the local charges will be small enough so that energy trades are more attractive than trading with the TEC is not really valid and the deployment of the solutions described in this thesis will remain on hold.

Future Work

There are several directions in which the work carried throughout this thesis can be extended.

Integration Additional Flexibility Sources, such as Electric Vehicles

In this thesis we assumed that all the flexibility of players was due to energy storage. This assumption, although not unrealistic, could be complemented by including in the decision model different appliances such as Water Heaters, Electric Vehicles and Fridges among others. Modeling the flexibility of such appliances will likely require the introduction of integer variables and it would be interesting to understand up to what extent the model can be extended while still being tractable (for example, for the case in which players still interact through sequential markets).

Integrating Advanced Tariff Structures such as Pricing Sensitive to Peak Consumption

The fixed electricity tariff was an assumption that was used throughout the different studies included in this thesis. This tariff, widely used, is what most of the residential customers are subscribed to. Nevertheless, it would be interesting to understand if the tariff could be adapted to take into account charges for maximum power consumed.

Such tariffs have added complexity, since a peak of consumption in the future might throw off-balance optimization that have been done in the past. This kind of tariff is particularly interesting as it could potentially avoid some of the problems faced by sequential LEMs when paired with Time-of-Use tariffs. If players have an incentive to reduce their peak, it is unlikely that they will charge at maximum power just before the change in price.

Improving Fairness of the Proposed Solutions

As we have seen in the chapters dealing with cooperative games, solutions obtained from the core can be "unfair" to some players. If there core is not unique, then there are solutions that are potentially more fair than others. Deriving a polynomial algorithm to find such solutions could make the cooperative scheme much more attractive. In the current version, this can be a problem since the solution obtain by the dual is a vertex of the feasible solution and it is therefore "extreme" in some sense.

Making the Models Suitable for an Evolving System

In part IV of this thesis in which we dealt with cooperative games, we assumed that the set of players was fixed as did not change in time. This might be unrealistic in practice, where new players (neighbors) might arrive or leave their home and therefore the energy community. Understanding how to extend the model to allow for dynamic changes in the coalition and payoffs remains a very interesting area of theoretical research.

Improving the efficiency of the proposed implementations

In the thesis we discussed a distributed algorithm to control a fleet of batteries among several players in a cooperative fashion. Such algorithm was useful for solving one-shot versions of the game, but it is not scalable if the problem has to be solved for every time-slot while implementing MPC.

It remains an area of further research to understand to improve such algorithm.

ChapterA

Exploiting Flexibility in Irrigation

This chapter describes a project in which the main goal was to understand if there was flexibility available in the irrigation of soy crops that could be used to reduce the energy cost of powering the pump required for such activity given that it accounts (at least in Uruguay) for a large percentage of the total operational costs of these farmers . This chapter summarizes the project, our solution and the results obtained.

A.1 Introduction

One of the most popular strategies for integrating DER is to dynamically exploit demand flexibility through Demand Response (DR) programs. These were initially reserved for major industrial players [START_REF] Cappers | Demand response in u.s. electricity markets: Empirical evidence[END_REF], but thanks to the application of Information and communication technologies under the smart grid paradigm, they have been recently extended to smaller commercial or residential consumers.

In this chapter we deal with a third, mostly unstudied, case, namely: demand response for medium consumers such as farmers with specific necessities, e.g. satisfying crop water requirements.

Indeed, participating in flexibility programs is particularly interesting for farmers, for whom the cost of electricity used in irrigation systems can account for up to 30% of the total production costs. Moreover, irrigation is mostly done without automatic control, or without considering both real time irrigation requirements and energy prices. This is an important problem, and several novel solution approaches have been considered [START_REF] Osroosh | Comparison of irrigation automation algorithms for drip-irrigated apple trees[END_REF] 1 . Furthermore, it has also been shown that irrigation increases crop yield, even in countries with temperate climates [START_REF] Montoya | Is irrigating soybean profitable in uruguay? a modeling approach[END_REF].

Irrigation can be an important source of flexibility [START_REF] Barooah | Spectral decomposition of demand-side flexibility for reliable ancillary services in a smart grid[END_REF] and crop productivity depends significantly on the structure of irrigation cycles [START_REF] Goksoy | Responses of sunflower (helianthus annuus l.) to full and limited irrigation at different growth stages[END_REF]. We point out that the flexibility provided by irrigation is of a special kind: the peak of inflexible load does not occur every single day, but is concentrated on some particular weeks of the crop growth cycle. For this reason, tariffs such as Time-of-Use with intra-day price are not well adapted.

In some countries with high penetration of renewables, the concept of rebates has been introduced. It is a discount in the electricity price to incentivize consumption when there is a surplus of generation.

The goal of this study is to investigate the extent of the benefits that can be obtained by exploiting flexibility in these cases. In doing so, we bridge the real practices in agriculture with innovative energy market models, in an effort to fully utilize latent flexibility.

The idea of decreasing the usage of electricity in irrigation is not new [START_REF] Rodríguez Díaz | Exploring energy saving scenarios for on-demand pressurised irrigation networks[END_REF], [START_REF] John L Merriam | Flexible irrigation systems: Concept, design, and application[END_REF]. In [START_REF] Rodríguez Díaz | Exploring energy saving scenarios for on-demand pressurised irrigation networks[END_REF], the authors optimize the water pressure of the irrigation system, while [START_REF] John L Merriam | Flexible irrigation systems: Concept, design, and application[END_REF] considers the flexible operation of an irrigation system. None of these studies takes into account neither the benefits of exploiting flexibility by the electricity operator nor the reduction in the electricity cost by properly utilizing electricity tariffs.

The contributions of this study are summarized as follows:

• We model the energy management problem with irrigation requirements as constraints and provide an algorithm to solve it. Evaluation is conducted on real data, and it indicates that significant reductions in the electricity cost incurred by farmers can be achieved while keeping the optimal level of productivity.

• We evaluate the effect of dynamic rebates based on renewable energy surplus in irrigation scheduling. It is shown that these can be beneficial to both the farmers and the TEC.

• Scheduling techniques are proposed to solve the problem of assigning a limited amount of surplus to an increasing number of participants in a fair manner.

A.2 Problem Definition

Nomenclature

We proceed to introduce the nomenclature used for formulating the model in the next section. This notation is specific to this chapter alone.

O

A.2.1 Crop Model

A task of vital importance in agriculture is the irrigation of fields, due to its positive impact on yields. The water supplied to the crops should be just enough to maximize the production (but no more than that). This turns out to be a difficult task, as the water requirement of a crop varies depending on the stage of growth. Climate conditions also play an important role in determining these requirements.

A plant absorbs most of its water requirements from its roots at a precise depth that varies with plant growth. The amount of water available in the ground varies between the maximum capacity of the field to retain water (FC) and the permanent wilting point (PWP) at which the plant dies. The total available water within these margins is (TAW), but the plant can only absorb a fraction (specified as the multiplicative constant L t , the tiredness fraction) that is known as Readily Available Water (RAW, Equation (A.2)). These values are depicted in The Root Zone Depletion (RZD, Equation (A.8)) is the level of water that has been depleted from the field capacity and is no longer available for the plant. It depends on its previous value, minus all the irrigation (U, Equation (A.4)), minus all the rain (G) plus the evaporation in the soil and the transpiration of the plant. These last two phenomena are represented together as the evapotranspiration (H, Equation (A.6)). The evapotranspiration is piece-wise linear and has two levels depending on the stress of the ground (percentage of RZD with respect to the PWP) [START_REF] Allen | Crop evapotranspiration: guidelines for computing crop water requirements[END_REF]. To keep the crop from under-performing (i.e., producing less kilograms per hectare), the Root Zone Depletion needs to be smaller than the Readily available Water Level. Otherwise, the water content in the ground would not be at a level available for the plant to easily reach, causing hydric stress and consequently yield reduction. In

A.3 Proposed Solution

A.3.1 Algorithm

We proceed to present an solution algorithm for the optimization problem (A.12). A key idea of our solution is to note that, to satisfy each of the constraints "O t ≤ M t ", only variables Y ij with i ≤ t could be used. That is, only the variables Y 1j affect O 1 , while for the constraint O 2 ≤ M 2 , only Y 1j , Y 2j can be used, etc. Therefore, we need to produce a feasible solution by using only those variables.

With that observation in mind, the algorithm is fairly simple, namely: consume the least amount of water to satisfy all the constraints and try to use the variables with the lowest cost associated with them. The motivation behind this idea is as follows: for small values of t, the amount of variables to consider is small, and therefore it is quite likely that we have found the optimal assignment. For larger values of t, although the number of variables to considered increases with t, by iterating in order, many of the irrigation variables Y ij , i < t will already be used, i.e, they do not have to be considered.

Algorithm 4 Optimization Algorithm

Input: F, G, H, J, K, L, C , Output: Y Y ij ← 0, ∀i, j 1: for i = 1 to T do 2: while O i > M i do 3: l, k ← select best variable , decrease O with least cost 4: Y lk ← Y lk + 5: update O i 6:
end while 7: end for 8: return Y To decide which variable should be used, Algorithm 5 is employed.

Algorithm 5 Selecting the best variable

Input: F, G, H, J, K, L, t, Output: z, w z, w, ∂ zw ← -1, -1, 0 1: for i = 1 to t do 2:

for j = 1 to 24 do else if α ij = 1 and α zw < 1 then end for 16: end for 17: return (z, w)

A.3.2 Opportunity assignation

The aims behind Opportunity Offers is to sell the surplus of renewable generation available.

As implemented thus far, a massive adoption of these tariffs could result in a peak of consumption greater than the original surplus. If such a case arises, more expensive units will have to be dispatched to satisfy the new demand, resulting in additional costs and a negative environmental impact.

In this subsection we propose four mechanisms inspired by the rich literature on scheduling to allocate the surplus of renewable energy without exceeding the available quantities. All the mechanisms follow the same time structure. For a given opportunity offer occurring at time t of day d, interest of buying from the consumers will be collected the day before (t, d -1) by the scheduler manager (SM). Intention to participate in an offer is expressed by the different participants as a pair (q i (t, d), c i (t, d)) where q(t, d) represents the consumption in kWh that the participant is requiring to buy and c i (t, d) is the cost (estimated by each participant) of not irrigating at all during the time t of day d.

Given the offers, the SM decides how much of the requested quantity each participant is allowed and communicates this data back to them. They are then free to reschedule their consumption as they see fit with the new change in tariff. It is important to note that losing participants are not forbidden to irrigate during hour t of day d. Although this might sound counter-intuitive, we are assuming that the proposed mechanism replaces the assignment of the opportunity offers but leaves the basic tariff structure unchanged and therefore, they are always allowed to use it.

The four evaluated mechanisms, namely: Least Served First (LSF), Most valuable first (MVF), Proportional (PR) and Fixed priority (FP), differ in how the SM assigns the opportunity offers, as follows. In LSF, the Scheduling Manager maintains an historic record of the energy assigned to each player in the past. Using this information, it sorts the received offers and assigns all the requested energy to the first participant. If there is any remaining surplus, it continues the allocation following the created order. Tiebreaks are handled arbitrarily. The second approach, Most valuable first, assigns as much energy as possible to the player that reported the highest cost c i (t, d), and if there is any remaining energy, it continues with the second highest cost, etc. The Proportional mechanism consists of distributing the available energy among all participants, proportionally to their submitted offers. Finally, FP is a very simple mechanism included for comparison purposes, in which there is a fixed order and energy is always distributed according to the same order. Algorithm 6 describes the common feature of the LSF, MVF and FP mechanisms. Their difference lies in the SortUsingMechanism function and how the permutation π is built. In Algorithm 6, a i is the assigned quantity to player i. For the Proportional case, the assigned quantity is a i = A(t,d)q i (t,d)

C

, where A(t, d) is the available quantity of surplus, q i (t, d) is the quantity asked by the farmer i, and C = j q j (t, d).

Algorithm 6 Scheduling mechanism

Input: (q 0 (t k , d j ), c 0 (t k , d j )), . . . (q N -1 (t k , d j ), c N -1 (t k , d j )) Input: A Output: a 0 (t k , d j ), . . . , a N -1 (t k , d j )

1: π ← SortUsingMechanism([N ]) 2: m ← 0 3: a l ← 0, l ∈ [N ] 4: while A > 0 and i < N do 5:

i ← π(m)

6:
if q i (t k , d j ) > A then 7:

a i ← q i 8: A ← Aq i (t k , d j )

12:

m ← m + 1 13: end while 14: return a 0 , a 2 , . . . , a N -1

A.4 Numerical Analysis

In this section, using real data, we shall demonstrate the benefits of implementing the proposed solutions.

We had access to real irrigation data from a soybean producer in Uruguay. The dataset contains the irrigation profile during 140 days, namely from 9th November 2017 to 28th March 2018, as well as all the other parameters required by the model2 . The data corresponds to one irrigation pivot used in a field of 75 hectares. The pump required to power the pivot consumes 77kW. We assume that there are no associated costs for starting or stopping the pump. Table A.1 summarizes the parameters used to instantiate the two cost functions except for the thresholds α ij , which where estimated out of historical data as the average consumption in the same day. First, we evaluate the net benefits of using Algorithm 4, when the tariff structure remains unchanged, i.e., only considering the three-tier ToU. Next, the benefits of including the opportunity offers are measured. Finally, the different mechanisms to assign such OOs are evaluated.

A.4.1 Optimization algorithm using real data

Figure A.3 depicts the evolution of the water content in the ground (WCG) for that period, given the irrigation pattern followed by the farmer. It can be seen that the WCG drops below the level of hydric stress, implying that some performance (in terms of kilograms per hectare) was lost. Moreover, the farmer did not manage to fully avoid irrigation during peak periods. in comparison with the ToU prices for our proposal. Observe that our irrigation strategy respects the constraints imposed by plant water requirements while avoiding the most expensive ToU period. Table A.2 summarizes the numerical comparison between the cases. Although our algorithm irrigates for a longer period of time (to satisfy the problem constraints), it has an overall decrease in the variable cost3 .

A.4.2 Optimizing for Opportunity offers

From the real data on electricity consumption and production surplus event, we obtained the α tj coefficients for the cost function C 2 defined in Equation (A.11). As expected, during most of the hours there were no surplus events, and thus no OOs. Figure A.7 depicts the hours at which the value of α tj was less than 1, i.e., there was an active rebate. Sub-figure A.7.A depicts the consumption of energy with respect to the active offers for the real past consumption. It should be observed that, although the farmer did not plan her irrigation around these rebates (as they are not currently available to farmers), there is some natural overlap between the two. The middle and bottom sub-figures (B and C) also depict the irrigation for those hours with active OOs, but for our algorithm, using C 1 and C 2 as the cost functions, respectively. Naturally, optimizing using C 2 as the cost function increases the irrigation whenever there is a surplus4 . The different costs of irrigation if Opportunity Offers were available are presented in Table A.3. We observe that optimizing for C 1 or C 2 uses the same irrigation time (and therefore the same amount of water), but reductions are bigger: a 15% reduction could be achieved in the costs if Opportunity offers were to be allowed to farmers. This is beneficial for farmers, because they decrease their costs, and also for the utility, because they have extra means to consume the surplus of renewable generation. This can foster the installation of new renewable energy sources in the grid.

A.4.3 Evaluation of opportunity assignments

In the previous subsection we showed that offering dynamic rebates to farmers can increase their profit and reduce the amount of renewable energy that need to be curtailed. As mentioned, offering this tariff to new actors can have an associated drawback: if the new peak in demand is greater than the original surplus, the utility company might suffer higher costs than from just curtailing renewable energy sources. To overcome this difficulty, we evaluate different scheduling mechanisms that aim to assign precisely all the surplus.

To asses the behaviour of the different mechanisms in combination with our algorithm, a synthetic dataset was created from real data. Under the assumption that farmers that are physically close experience similar weather conditions, we took the data we already had and added a small random noise to emulate spatial difference. For the OOs dataset, we used the same periods (t, d) as in the previous sections, yet now we generated the amounts of surplus. In each of these dates, a random quantity of excess of energy was considered in a way that reveals the effects of the different mechanisms: too much energy would satisfy all customers and the mechanisms would be indistinguishable and the same would happen if there is no surplus at all. Figure A.8 illustrates the results of running the different scheduling mechanisms with 20 participants. Each bar plot represents the relative gain of each player (cost with OOs divided by cost without OOs). The number in the legend shows the net cost of all the players averaged over all simulation runs. We seek a mechanism that minimizes the total cost while keeping the variance of different players small, i.e, maximizes fairness.

As expected, the Fixed Priority technique was the most unfair (highest variance), followed by the MVF. The least total welfare was obtained by Least Served First, closely followed by MVF. Overall, Least Served First yielded the best fairness and overall welfare.

A.5 Discussion and Final Remarks

We have proposed an architecture and specific irrigation scheduling and coordination mechanisms that enable to exploit electricity demand flexibility from irrigation and, at the same time, minimize the farmers' electricity costs. Indeed, the proposed scheduler manages to reduce the cost of irrigation electricity while keeping optimal levels of productivity, by exploiting existing ToU tariffs.

Based on real data, we showed that producers do not irrigate in an optimal way and they could greatly benefit from an automatic scheduler, such as the one proposed in this study, in order to boost their production and decrease their energy costs. Moreover, there is a clear potential to exploit opportunity offers, as shown by the decrease of cost in both instances of the algorithm. We demonstrated that the biggest barrier preventing the massive adoption of opportunity offers can be circumvented with our proposed allocation mechanisms.

Consequently, our proposed irrigation scheduler and flexibility allocation mechanisms increase the potential of the grid to host renewable energy sources while reducing the electricity costs in agriculture, which is of key economic relevance in several countries.

We conclude by mentioning some potential improvements that could be achieved through future work. First, the current algorithm is deterministic and requires a forecast of the temperature and precipitation to work with. A natural extension is to pair it with a forecast and use Model Predictive Control to correct the irrigation plan as the real information becomes available. Second, although a Time of Use together with the rebates can provide important incentives, further work should be carried to understand whether a local energy market can provide better results. Finally, it would be interesting to investigate and apply the set of techniques and approximations introduced in [START_REF] Schoonen | Optimal irrigation management for largescale arable farming using model predictive control * * the authors acknowledge the support of 'toeslag voor topcon-sortia voor kennis en innovatie' (tki htsm) from the ministry of economic affairs, the netherlands[END_REF] to this problem. be paid to baselines. While providing a flexibility service, the degree of change with respect to normal behavior is usually the object of the contract. How that usual behavior is measured is of critical importance. This needs not to be the same as the default operation mode, although they could be the same. Traditional approaches for baselineing include an average of past consumption and the consumption a few minutes before delivery of the contracted flexibility.

B.3.1 Detail level of the appliances

One key question in the design of the dataset is how realistic should the model of the appliances be. A very detailed description might make the benchmark too complicated for normal use whereas a too simplistic model might force users to add their own modifications, defeating the purpose of the benchmark altogether. For example, consider the case of battery storage. A description of such storage should include the maximum battery capacity, the minimum battery capacity and maximum and the minimum charging and discharging power. It should most likely also include the charging and discharging efficiencies. But should it consider a non-constant efficiency that depends on the state of charge? Should it include the likelihood of a random discharge? The most useful level of detail probably lies in the middle, where most researchers can feel comfortable about the realism of the model. Finally, to enable power flow calculations using the benchmark, appliances should include their power factor.

As another example, consider the thermal characteristics of households, used for modeling heating, ventilation, and air conditioning applications (HVAC). Should the complete blueprint of a household be provided or only the total area of the residence? What about the materials of the walls, their thickness or the orientation of the house? Again, the exact level of detail should probably be decided based on the feedback of the community that will use the benchmark.

B.3.2 Modularity

Not every project calls for the same level of detail. For example, authors in [START_REF] Chen | Distributed Control Design for Balancing the Grid Using Flexible Loads[END_REF] survey the use of distributed control of several kinds of appliances such as refrigerators, water heaters, air conditioning and swimming pools [START_REF] Mathias | Smart fridge/dumb grid? demand dispatch for the power grid of 2020[END_REF][START_REF] Meyn | Ancillary service to the grid from deferrable loads: The case for intelligent pool pumps in florida[END_REF][START_REF] Meyn | Ancillary service to the grid using intelligent deferrable loads[END_REF]. For such application, the need to deal with extra appliances might be seen as a reason not to use the benchmark ("it is too much for our problem"). A second example could be modeling the shared investment in storage by a collective of users without one, such as the one proposed in Chapter 10, 11 and 13 of this thesis. In that particular application, only the net load might be desired and having to deal with specific appliances might be seen as a drawback. This will be a key factor in the adoption of the benchmark: it should not be overly simplistic nor overly detailed for most users.

In that regard, the benchmark should be designed in a way that allows for some of its parts to be encapsulated and treated as black boxes if desired. In particular, for a deterministic dataset, the default operation (obtained in a pre-specified manner, see subsection B.3.4) can be distributed together with the original data. For the use-case of the distributed control of the pools, the interested user can fix all the appliances to behave as in the default scenario and deal only with the flexibility of the pools. By doing so, he/she can assess in a realistic scenario the added benefit of the distributed control mechanisms with respect to the normal performance.

B.3.3 Granularity

Some applications closer to the physical power grid level might require load samples every minute or second, while testing complicated game theoretical models might only allow for sampling at periods of 30 minutes or greater. An important quality regarding time granularity is to find a standardized way of aggregating time-slots. That way, if the benchmark is distributed at the 5 minute level, but an application requires data sampled hourly, it will be possible for them to aggregate it for their use and dis-aggregate it later, producing results in the standard format.

This seems to indicate that the smaller the granularity, the better, as we can always go to coarser profiles. Nevertheless, the computational complexity produced by a dataset sampled every milliseconds will not provide added benefits to the DR community. The sweet spot seems to be around 1 or 5 minutes, but is up for discussion.

B.3.4 Default operation

So far, we have discussed how to design the dataset and what information should be included in it. Unfortunately, this is still not sufficient to provide a reliable and reproducible benchmark. Central to the idea of a benchmark is the idea of comparing the performance of one technique to another one. This calls for a "default operational mode" (DOM), i.e., the behaviour of the system when no DR program is applied to it. Clearly, this default mode should be uniquely specified in the data. We want to point out that this is not trivial to achieve and that extra specification will be required to guarantee the existence of a unique DOM. The simplest approach to obtain a DOM would be to solve an optimization problem for each household that outputs a schedule of all the appliances, such that the total cost payed for electricity is minimized. We shall refer to this solution as the Default with Perfect Information (DwPI). There are two main problems with the DwPI. First, it can be computationally impossible to find. Consider a dataset containing samples with a resolution of 1 minute and a horizon of 1 year: there are more than half a million time-slots, each one of them with several discrete variables. Secondly, the result obtained will not be representative of a real settings in which agents have to forecast their load and even maybe their prices. One way to solve this problem is to use a forecast of the load and a rolling horizon Model Predictive Control technique to obtain the solution. If the length of the horizon and the forecasting technique are pre-specified, then a unique solution can be obtained4 : the Default with Forecast (DwF).5

B.3.5 Measuring grid flexibility

In a power grid where all the generation is dispatchable 6 , peak reduction has been traditionally the objective of demand response programs. This was motivated by the fact that the most complex task was to satisfy the higher peaks of demand. With the introduction of non-dispatchable generation, such as solar and wind, matching the produced energy with the consumption becomes ones of the most important problems to solve, as there is no benefit in installing non-dispatchable generation if there will be no consumption when there is generation. We believe that the matching between generation and consumption should be a central measure of grid flexibility. To obtain a concrete measure of it, we can define grid flexibility as the integral of the difference between renewable production and consumption. We can further distinguish between curtailment (generation is larger than consumption) and unmet demand (which requires extra generation capabilities to be dispatched). The later is arguably worse than the former, so we can envision a metric defined as the weighted average of the two quantities, with a higher emphasis on the unmet demand.

B.3.6 Valuation of load shedding

One of the traditional mechanisms for DR is load shedding. Properly modeling such a mechanism requires the valuation of agents for not consuming their required energy. For flexible loads, this value can be obtained by shifting around the load and trying to obtain a new, feasible allocation, possibly at a higher cost. For inflexible loads, or when the flexibility is not sufficient, the procedure described above will not provide the required answer. Instead, the dataset should specify an external valuation of that quantity, i.e, at what price will each household turn off their inflexible appliances. This value is intrinsically personal and depends in the socioeconomically situation of each agent. For example, a household in a rich neighbourhood might be willing to pay more to keep their swimming pool warm than a poor family will be willing to pay to keep their heating on during winter. There is no clear way to obtain a representative valuation for this. Some sort of valuation belonging to a specific family of functions could be assumed (lets say quadratic with sampled coefficients), but it will likely result in biased results towards DR techniques with similar assumptions (positively or negatively). The other alternative would be to limit the scope of the benchmark and decide that such demand response programs are not included, which in principle is undesirable given the important role of such mechanisms.

B.3.7 Benchmark's scope

We conclude this section with a brief discussion on the limitations of our approach. Demand response is a complex problem that encompasses a wide range: from technical capabilities of the power grid to be controlled in real-time to patterns in human behaviour that modify how households react to incentives. Given the complexity of modeling human behaviors in response to price incentives, we simplify the benchmark by focusing on the engineering aspect of it. Although idealized, we feel this approach creates immediate benefits while sidestepping potential biases and inherent uncertainties. We motivate our approach as follows. First, the capability of comparing the performance of different DR programs applied to the same reality will enhance our understanding of what it is required to properly implement them, even if such knowledge deals only with the technological aspects of DR. Secondly, to avoid adding biases about human behaviour to the benchmark, we restrict ourselves to the case in which all flexibility and change in consumption is enabled by smart appliances and does not require the active participation of household owners.

B.6.1 Deep learning on the raw dataset

With computational effort, it should be possible to approximate, for each of the aforementioned datasets, the default aggregated consumption without any DR. It should also be possible to compute the optimal flexibility profile that a centralized entity could achieve it it had control of all the appliances available. This would yield a labeled training set where each data point is one of the benchmarks and the label is the optimal profile that can be obtained. We envision that a deep learning algorithm could be trained to predict such performance by identifying the relevant features in the benchmark. For example, it might be that the total count of batteries plus water heaters with their corresponding electricity price is a good predictor of the net grid flexibility. In that case, the algorithm could learn the best predictors of performance and then be used to predict the grid flexibility capabilities of other regions of the grid. This could also be seen as a problem of Transfer Learning.

B.6.2 Reinforcement Learning

It is very likely that implementing and simulating a real-time HEMS will require solving large mixed integer optimization problems. Doing so for large grids and lengthy time horizons can prove intractable. In this regards, reinforcement learning (as well as other techniques) could be used to replace the computationally expensive decision process faced by each agent. This could provide an opportunity to evaluate ML for real time-control. Even closer to demand response, reinforcement learning could be applied to learn a model of how agents change their behavior from their default consumption profile to a different one in the presence of a DR program. This can be used to discover and quickly test DR techniques in a variety of scenarios. Under ideal conditions, such tests could be used as a first step, followed by a through evaluation of the most promising techniques in comprehensive simulations of the benchmark.

B.7 Conclusions

In this chapter we propose the design of a benchmark for demand response applications that will enable a systematic measurement of the grid flexibility available in different regions of the grid. These measurements are crucial in the adoption of demand response programs, without which, the massive deployment of renewable resources and the decarbonization of power systems will be hindered. Together with a specification of such benchmark, we provide the reader with three examples of how different demand response techniques can benefit from the benchmark. Furthermore, we present two potential applications of AI: predicting the maximum grid flexibility that could be achieved and learning how consumers will react to new demand response techniques.

C. 3

C.4 Stochastic LP formulation

In this appendix we formulate the large linear optimization problem (C.5) that is obtained by merging the optimization problem associated with the first stage (11.6a) and second stage (11.7a) of the stochastic formulation of the cost of a coalition. C.5 Proof of Theorem 11.5

Theorem 11.5 The optimal battery size for a coalition S in the discrete setting is given by B ↑ or B ↓ , where B ↑ is the smallest multiple of B greater than β and B ↓ is the largest positive multiple of B smaller or equal than β.

In this context, β is the optimal battery size for the coalition in the continuous setting.

Proof. Consider a parametric linear program min{cx|Ax = b + bλ, x ≥ 0}. It is known that if φ(λ) is the value of the optimal solution, then λ → φ(λ) is a convex piecewise linear continuous function [10]. Observe that the optimization problem considered so far can be written in this parametric form, where the parameter λ is precisely the battery size S (the reformulation requires to replace all the instances of S in the cost function by the appropriate combination of the rest of the variables). Since φ is continuous and convex in R + and, furthermore lim x→∞ φ(x) = ∞, φ has a global minimum that coincides with the battery size in the continuous case. Now, because φ is convex, it holds that, among the integer values, φ has to be minimized by either S ↑ or S ↓ . This implies that, instead of solving the mixed integer optimization problem, it suffices to solve the continuous one and check which of the two solutions is better.

C.6 Proof of Stochastic Balance

This appendix provides the proof of Theorem 11.3.

Theorem 11.3 The cooperative game defined by using optimization problem (C.5) as the cost of each coalition is balanced and, therefore, it has a non-empty core.

Proof. For the continuous and stochastic version of the problem we shall use the notation X , b cs , c cs to denote the components of the matrix formulation of the associated LP. Furthermore, let W = |Ω|.

The proof is quite similar to that of the deterministic case. Observe that the main difference between the feasible sets of optimization problems (C.5) and (11.2a) is that in the former, each constraint is repeated for each scenario w ∈ Ω. Therefore, we shall show that the same ideas used in the proof of Theorem 11.1 still hold when each constraint is also indexed by w. First, we show that the consumption profiles are still additive while multiplying them by the balanced coefficients (Equation (C.6)): . . .

l S T (w W ) 0                 =                
S α(S)l S 1 (w 0 )

. . . S α(S)l S T (w 0 ) S α(S)0 S α(S)l S 1 (w 1 ) . . . S α(S)l S T (w W ) S α(S)0

                =                 l N 1 (w 0 ) . . . l N T (w 0 ) 0 l N 1 (w 1 )
. . .

l N T (w W ) 0                 (C.7)

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A.4.1 Optimization algorithm using real data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A.4.2 Optimizing for Opportunity offers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A.4.3 Evaluation of opportunity assignments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A.5 Discussion and Final Remarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B Design of A Benchmark for Demand Response Applications B.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B.1.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B.1.2 The role of benchmarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B.2 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B.3 Benchmark specification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B.3.1 Detail level of the appliances . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B.3.2 Modularity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B.3.3 Granularity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B.3.4 Default operation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B.3.5 Measuring grid flexibility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B.3.6 Valuation of load shedding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B.3.7 Benchmark's scope . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B.4 A simple example: Consumer specification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B.5 Example applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B.5.1 Optimal Time-of-Use tariff design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B.5.2 Energy trading: Auctions and Peer-to-peer . . . . . . . . . . . . . . . . . . . . . . . . . . . . B.5.3 Optimal investment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B.6 Machine Learning applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B.6.1 Deep learning on the raw dataset . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B.6.2 Reinforcement Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B.7 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

2. 1 15 8. 4

 1154 Outline of the system in which local energy trading takes place . . . . . . . . . . . . . . . . . . . . . 3.1 An example of a consumption profile (black curve) and an associated partial consumption profile (blue curve). Notice how the partial consumption always has the same sign as the original consumption profile and its absolute value is always smaller. Each square represents a unit of energy. . . . . . . . 3.2 The three curves plot energy consumed per time-slot. Positive values denote energy consumption while negative values represent surplus of generation. . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.1 Total daily energy consumption obtain from the Ausgrid dataset. Each boxplot corresponds to a different player. The variance within each boxplot is due to different energy consumption in different days. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2 Percentage of error between the optimal value using real data and the forecast for the algorithms described above. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.1 An example of the clearing process in Huang's auction. It is almost a basic double auction (intersection of supply and demand) with the addition that the two players that define the clearing price do not trade. There is a different price for buying and for selling and all the profit resulting from the difference between both prices goes to the market maker. . . . . . . . . . . . . . . . . . . . . . . . . 5.2 The clearing algorithm of the muda mechanism. Each side is trading with the clearing price of the other group. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.3 Simplified overview of the decision flow of a participant in a LEM . . . . . . . . . . . . . . . . . . . . 5.4 Timeline of decision process and decision flow for a player in the LEM game . . . . . . . . . . . . . . 5.5 Cost function including the beliefs about market prices . . . . . . . . . . . . . . . . . . . . . . . . . . 6.1 All net profiles obtained by using different beliefs in one simulation with a ToU rate. . . . . . . . . . 6.2 Net load asked in the market versus the net load actually consumed. . . . . . . . . . . . . . . . . . . 6.3 All net profiles obtained by using different beliefs in one simulation with a flat rate. . . . . . . . . . 8.1 Example load profile with buying indifference. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.2 Example load profile with selling indifference. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.3 Cumulative distribution of the ratio between the social cost obtained when running a LEM and without it. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Cumulative distribution of the ratio between the untraded local energy obtained when running a LEM and without it. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.1 Different steps transforming the integral . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.2 2-person storage game with an empty core . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.3 Only player 4 fills a time gap . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.4 All players fill a time gap . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.5 Load profiles of the five players used in the experiment . . . . . . . . . . . . . . . . . . . . . . . . . . 10.6 Relative gain with respect to the case of no battery . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.7 Relative game with respect to the case of players acting alone . . . . . . . . . . . . . . . . . . . . . . 11.1 Changes in the optimal battery size in a community . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.2 The relative maximum violation of the approximated solution for a varying number of players . . . . 11.3 The total cost of individual players for 15 days after the investment using the different techniques to assign the cost. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.1 Comparison of the average running time of the distributed algorithm using different topologies for an increasing number of players. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.2 Comparison of the number of iterations before convergence of the distributed algorithm using different

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A.1 Illustration of the different parameters in the model. . . . . . . . . . . . . . . . . . . . . . . . . . . . A.2 The set of points satisfying O 1 ≤ M 1 for some combination of parameters. Pixels in white represent points in the set and in black, outside. This set is clearly non-convex. . . . . . . . . . . . . . . . . . A.3 Irrigation model on real data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A.4 Consumption across time of uses. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A.5 Algorithm maximizes TOU without taking into account opportunities. . . . . . . . . . . . . . . . . . A.6 Algorithm maximizes TOU without taking into account opportunities. Irrigated hours for each ToU price. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A.7 Consumption during the Opportunity offers for the three considered cases. . . . . . . . . . . . . . . . A.8 Gains of farmers under different assignment techniques. . . . . . . . . . . . . . . . . . . . . . . . . .

Figure 1 . 2 :

 12 Figure 1.2:A graphical example of the three scenarios described for a household with a solar panel and a smart meter that consumes and injects into the main grid. Depicted in the image are the three possible total costs of the participant in each of the scenarios at the end of the billing period. The arrows on the left represent the net energy in the time-slot after using as much solar as needed for self-consumption..

Figure 1 Figure 1 . 3 :

 113 Figure 1.3: Simple representation of a High -Medium and a Medium -Low Voltage transformers, the load of two households and the aggregated view of the load from the MV grid. The traditional electricity company sells energy from the HV/MV side to the LV side. The scope of the thesis is mostly limited to the Distribution Grid and in particular, to households behind the same Low voltage substation, such as the group in the right side of the image.

  PhotovoltaicPanel

Figure 2 . 1 :

 21 Figure 2.1: Outline of the system in which local energy trading takes place

Figure 3 . 1 :

 31 Figure 3.1: An example of a consumption profile (black curve) and an associated partial consumption profile (blue curve). Notice how the partial consumption always has the same sign as the original consumption profile and its absolute value is always smaller. Each square represents a unit of energy.

Figure 3 . 2 :

 32 Figure 3.2: The three curves plot energy consumed per time-slot. Positive values denote energy consumption while negative values represent surplus of generation.

Theorem 4 . 1 .

 41 If Assumption 4.4 holds, then optimization problem (4.3) is equivalent to a linear programming problem.

Figure 4 . 1 :

 41 Figure 4.1: Total daily energy consumption obtain from the Ausgrid dataset. Each boxplot corresponds to a different player. The variance within each boxplot is due to different energy consumption in different days.

Example 5 . 1 .

 51 If a player submits a buying bid given by B buying = ((1, 3), (3, 2), (5, 1)).

Figure 5 . 1 :

 51 Figure 5.1: An example of the clearing process in Huang's auction. It is almost a basic double auction (intersection of supply anddemand) with the addition that the two players that define the clearing price do not trade. There is a different price for buying and for selling and all the profit resulting from the difference between both prices goes to the market maker.

Figure 5 .

 5 2 depicts one round of the MUDA mechanism.

Figure 5 . 2 :

 52 Figure 5.2: The clearing algorithm of the muda mechanism. Each side is trading with the clearing price of the other group.

Figure 5 . 3 :

 53 Figure 5.3: Simplified overview of the decision flow of a participant in a LEM

Figure 5 . 5 :

 55 Figure 5.5: Cost function including the beliefs about market prices

Figure 6 . 1 :Figure 6 . 2 :Figure 6 . 3 :

 616263 Figure 6.1: All net profiles obtained by using different beliefs in one simulation with a ToU rate.

√ 0 . 8 4 √0. 8 - 2 √ 0. 8 need0. 8 = 3 ,

 08482883 units in the battery before the change in price. The two free units in time-slots 5 and 6 contribute to 2 √ 0.8 units in the battery. The remaining R = to be bought. This can be done by buying R √ because of the charging efficiency.

Figure 8 . 1 :

 81 Figure 8.1: Example load profile with buying indifference.

Figure 8 . 2 :

 82 Figure 8.2: Example load profile with selling indifference.

Figure 8 . 3 :

 83 Figure 8.3: Cumulative distribution of the ratio between the social cost obtained when running a LEM and without it.

Figure 8 . 4 :

 84 Figure 8.4: Cumulative distribution of the ratio between the untraded local energy obtained when running a LEM and without it.

90CHAPTER 10 .

 10 COALITIONAL STORAGE GAMES: A MODEL USING COOPERATIVE GAME THEORY FOR SHARINGH S (B) = πS -P X S (B) + p h

1 0F

 1 min{l S (x), δS}dx, then it is clear from the plots A and B in Figure10.1 that Equation (10.6) holds:I i ), δS}|I i | = α i=1 l S (I i )|I i | +

  (l S (I r ) -l S (I r-1 )) + j=m j=α+1 |I j |δS = r=α r=1 |I r |l S (I r ) + j=m j=α+1 |I j |δS (10.8)

l S (I 1 )Figure 10 . 1 :

 1101 Figure 10.1: Different steps transforming the integral

Figure 10 . 2 : 2 -

 1022 Figure 10.2: 2-person storage game with an empty core

Proposition 10 . 3 .

 103 Consider a 5-person"unitary glove market" game with K = 3. Player 1 is of type I, players 2 and 3 are of type II and players 4 and 5 are of type III. Consequently l = 1. Let V = {ζ ∈ R 5 : ζ 2 = ζ 3 , ζ 4 = ζ 5 } be a set of imputations where ζ i is the payoff corresponding to player i. The set V is a stable set, i.e., a solution in the sense of Von Neumman and Morgenstern.

y 2 =

 2 y 3 , y does not belong to V . No other restrictions are imposed on y. Let m II = min{y 2 , y 3 }, M II = max{y 2 , y 3 }, m III = min{y 4 , y 5 } and M III = max{y 4 , y 5 }. Define = M II -m II 5 and δ = M III -m III

2

 2

Theorem 10 . 3 .

 103 If the following hold: (H1) All players fill a time gap (H2) π < P δγ(l N ) (H3) π ≥ P δγ(l N \{i} ), ∀i ∈ N

Figure 10 . 3 :

 103 Figure 10.3: Only player 4 fills a time gap

Figure 10 . 4 :

 104 Figure 10.4: All players fill a time gap

Figure 10 . 5 :

 105 Figure 10.5: Load profiles of the five players used in the experiment

Figure 10 . 6 :

 106 Figure 10.6: Relative gain with respect to the case of no battery

Figure 10 . 7 :

 107 Figure 10.7: Relative game with respect to the case of players acting alone

45 Figure 11 . 1 :

 45111 Figure 11.1: Changes in the optimal battery size in a community

Figure 11 . 3 :

 113 Figure 11.3: The total cost of individual players for 15 days after the investment using the different techniques to assign the cost.

  a α, . . . , α, b β, . . . , β, . . . , z ζ, . . . , ζ).

1 . 1 ,

 11 . . . , x N,- T ) c = (-β 1 , . . . , -β T , γ 1 , . . . , γ T , 0, . . . , 0, . . . , 0) (12.4)Finally, let A ∈ M (R) 4T N +2T ×2T N +2T be the matrix obtained by writing all the constraints associated with the optimization problem(12.1) in the order associated with vectors b and x.

120CHAPTER 12 .

 12 INCENTIVES TO IMPLEMENT CONTROLLERS FOR A FLEET OF BATTERIES USING COOPERATIV Proposition 12.1. The optimization problem defined in (12.1) is equivalent to v S = min{ c x : A x ≤ Bt S , x ≥ 0}.

Figure 12 . 1 :

 121 Figure 12.1: Comparison of the average running time of the distributed algorithm using different topologies for an increasing number of players.

17

 17

Figure 12 . 2 :

 122 Figure 12.2: Comparison of the number of iterations before convergence of the distributed algorithm using different topologies for an increasing number of players. The simulations had a 10000 cut-off.

Figure 12 . 3 :

 123 Figure 12.3: Running time of the different algorithms with a varying number of players. The y-axis is in log scale.

124CHAPTER 12 .

 12 INCENTIVES TO IMPLEMENT CONTROLLERS FOR A FLEET OF BATTERIES USING COOPERATIV

Figure 12 . 4 :

 124 Figure 12.4: Cumulative distribution of the total cost

Proposition 13 . 1 .

 131 If the sample set Ω is finite, then the two-stage linear optimization problem defined by (13.1) and (13.2) can be re written as a large linear programming problem of the form v(S) = min{cx : Ax ≤ iinS b i , x ≥ 0}.

Proposition 13 . 3 .

 133 If δ(S ∪ T ) = µ(δ(S) + δ(T )) with S ∩ T = ∅ and µinR, then w = µ N -1 (δ({1}), . . . , δ({N })).

Figure 13 . 1 :

 131 Figure 13.1: Comparison of two different ways to assign to distribute the cost to the players. In one way, all the profits of cooperation are evenly distributed among the participants whereas in the other one, an approximation of the payoffs in the core as described in the previous section was used. The top figure plots the results with the corresponding outliers, whereas the bottom image zooms-in near the x-axis.

Figure 13 . 2 :

 132 Figure 13.2: Comparison of the distance between the optimal discrete solution and the optimal continuous solution with respect to the size of the investment.

  Figure A.1.

  Figure A.1, two different values of RZD and the corresponding WCG are represented. WCG values are shown as the fully grey shaded areas increasing upwards and the rectangle with the brick pattern represent the level below which water is not easily available for the plant. The red value of RZD is larger than the RAW value,

4 :

 4 if Y ij ≤ 1 -and d > 0 then 5: if α ij < 1 and α zw = 1 then 6: z, w, ∂ zw ← i, j, d 7:

  Figure A.3 depicts the evolution of the water content in the ground (WCG) for that period, given the irrigation pattern followed by the farmer. It can be seen that the WCG drops below the level of hydric stress, implying that some performance (in terms of kilograms per hectare) was lost. Moreover, the farmer did not manage to fully avoid irrigation during peak periods. Figure A.4 shows the distribution of irrigation hours along ToU price periods, where green, yellow and red stand for low, medium and high prices, respectively.

Figure A. 3 :

 3 Figure A.3: Irrigation model on real data

Figure A. 4 :

 4 Figure A.4: Consumption across time of uses.

Figure A. 5 :

 5 Figure A.5: Algorithm maximizes TOU without taking into account opportunities.

Figure A. 6 :Figure A. 7 :

 67 Figure A.6: Algorithm maximizes TOU without taking into account opportunities. Irrigated hours for each ToU price.

Figure A. 8 :

 8 Figure A.8: Gains of farmers under different assignment techniques.

(

  P S ) minimize S, E + , E -, e + , e - πS + w∈Ω p(w)C(S, E + , E -, e + , e -, w)subject to δS + e + t (w) -e - t (w) = n∈S x n t (w) ∀t ∈ T , ∀w ∈ Ω, S + E + (w) -E -(w) = t (δSe - t (w)), ∀w ∈ Ω,S, E + , E -, e + , e -≥ 0 , E + , E -, e + , e -, w) = p l (S -E -(w)) + p h (E + (w) + t e + t (w))

  show that the balanced solution is still feasible for the Grand Coalition problem (C.7).

  General Context and Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.2 Scope of the Thesis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.3 Methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.4 Contributions of the Thesis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.5 List of Publications Produced During the Thesis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . CONTENTS 13.7.2 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.8 Key Outcomes and Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 General Conclusions 14.1 An Integrated Overview of Main Results Presented in this Thesis . . . . . . . . . . . . . . . . . . . . 14.2 Discussion on the Potential Impact of the Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14.3 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14.3.1 Integration Additional Flexibility Sources, such as Electric Vehicles . . . . . . . . . . . . . . . 14.3.2 Integrating Advanced Tariff Structures such as Pricing Sensitive to Peak Consumption . . . . 14.3.3 Improving Fairness of the Proposed Solutions . . . . . . . . . . . . . . . . . . . . . . . . . . . 14.3.4 Making the Models Suitable for an Evolving System . . . . . . . . . . . . . . . . . . . . . . . 14.3.5 Improving the efficiency of the proposed implementations . . . . . . . . . . . . . . . . . . . . A Exploiting Flexibility in Irrigation A.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A.2 Problem Definition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A.2.1 Crop Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A.2.2 Electricity tariffs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A.2.3 Optimization Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A.3 Proposed Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

	Contents
	1 Introduction
	1.1

A.3.1 Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A.3.2 Opportunity assignation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A.4 Numerical Analysis .

Table 6 .

 6 1: Statistics obtain with different configurations. All these values are % of change.

				SC min max LM net LM
	Tariff Freq Belief
	Flat	0	SOL	-11	6	95	62	396
	TOU 1	NEU	-2	-54	57 -36	187
			OPT	1 -150	-40	-3	284
			PES	-1	0	40	-6	282
			SOL	-1	-7	7	-3	285
			UNQ	-1	-6	-7	13	337
		0	PES	0	0	0	0	297
			SOL	-11 -155	-1	50	435

Table 10 .

 10

		1: Batteries characteristics
		Price (USD) Capacity (kWh) Rated Power (kW)
	Powerwall 2	7250	14	5
	Super Tesla	7250	14	11.6
	BMZ ESS 7.0	7700	6.77	8
	Aquion	2200	2.2	0.68
	Fake	4380	6	0.5
	Table 10.2: Estimated parameters from batteries
		Estimated π (c/kW h) Estimated δ ( 1 h ) Color
	Powerwall 2	14	30 7
	Super Tesla	14	10
	BMZ ESS 7.0	31	14
	Aquion	27	3.7
	Fake	20	1

Table 10 .

 10 3: Imputations of a concave game

  ζ 2

	ID 871	2.84 2.79 2.69 2.84	1.78	5.63	0.00
	ID 8622 3.21 3.16 3.21 3.06	1.58	0.00	4.96
	ID 4298 2.11 2.06 2.11 2.11	2.39	0.00	0.00

  E + , E -, e + , e - P L S + p l (S -E -) + p h (E + +

	106CHAPTER 11. A MODEL FOR COALITIONAL STORAGE GAMES USING LINEAR PROGRAMMING GAMES
	P roblem P S )	minimize		e + t )	(11.2a)
	S, t
	subject to
	δS + e + t -e -t =	l i t	∀t ∈ T (λ t ),	(11.2b)
		i∈S
	. , T 2 .
			105

Table 11 .

 11 2: Value of all the coalitions for different battery sizes. Optimal battery sizes are highlighted and coincide with the derivation in Table11.1 using the fact that ∆ = 0.11428.

	Coalition / Battery size	0	1	2	3
	(0,)	0.495	0.480	0.78 1.08
	(1,)	0.330	0.420	0.72 1.02
	(2,)	0.220	0.380	0.68 0.98
	(0, 1)	0.825	0.775	0.9	1.2
	(0, 2)	0.715	0.665	0.86 1.16
	(1, 2)	0.550	0.5	0.8	1.1
	(0, 1, 2)	1.045	0.995	0.98 1.28

Table 13 .

 13

	Player 1 Player 1 Community: 1 and 2

1: Parameters of the example

Table 13 .

 13 2: Return over investment of hardware and cooperation for different number of simulations

			%ROI Coop Change %ROI Hrdw Change %ROI coop of total cost E
			MEAN	STD MEAN	STD MEAN	STD
	# Batteries $ Battery
	0.0	120.0	-757.18	6955.02	13.19	771.62	7.61	3.02
		150.0	29.04	35.08 -353.81	2380.45	8.57	1.62
		190.0	4.69	16.96	68.84	125.42	9.37	1.32
	2.0	120.0	49.90	80.26	46.21	452.99	12.97	3.51
		150.0	7.18	15.33	69.99	252.65	14.30	2.25
		190.0	2.68	11.60	29.64	67.53	14.95	1.90
	4.0	120.0	24.54	32.54	67.41	263.03	17.81	4.29
		150.0	0.63	9.78	18.26	79.92	19.98	3.16

Table A .

 A 1: Parameters of C 1 and C 2 .

	Variable	Value
	p l	2772
	p m	3.078
	p h	10.205
	T l	{0, . . . , 6}
	T m	{7, . . . , 17, 22, 23}
	T h	{18, . . . , 21}
	P	77
	β l	0.4
	β m	0.4
	β h	1

Table A .

 A

		2: Numerical Comparison Without OOs
	Optimized Irrigated Time (h) Cost (UYU)
	No	1163	324394
	Yes	1323 (+13%)	289075 (-11%)

Table A .

 A

		3: Numerical Comparison With OOs
	Optimizing Irrigated Time (h) Cost (UYU)
	No	1163	301771
	C 1	1323 (+13%)	268287 (-12%)
	C 2	1323 (+13%)	261679 (-15%)

  Proof of Proposition 11.1 Proposition 11.1 The optimization problems (11.1) and (11.2a) are equivalent.Proof. The basic idea is as follows: each variable e + t , e - t allows us to represent the minimum inside the summation (integral) of X S (B), while E + , E -represent the outer minimum.Specifically, by Equation (11.2b): min{δB, l S t } = min{δB, δB + e + t -e - t } = min{δB, δBe - t } = δBe - t . Doing some extra arithmetic we obtain that:= min{B, B + E + -E -} = B -E -

	min	 	B,	T	min{l S t , δB}	 	= min{B,
					t= T 2		
	v S (B) =	BP L	+ p h	2 t= T T	l S t -(p h -p l )(B -E -)
		=	BP L	+ p l (B -E -) + p h	t∈T	l S t -p h (B -E -)
		=	BP L	+ p

t δBe - t } l (B -E -) + p h t (δB + e + t -e - t ) -p h (( t (δBe - t ) -E -) = BP L + p l (B -E -) + p h (E + + t e + ) (C.4)

The DSO have other responsibilities as well

https://www.centrica.com/media/4609/the-future-of-flexibility-centrica-cornwall-lem-report.pdf

We use as an objective function the minimization of costs with respect to a Time-of-Use tariff but that cost could be exchanged for other metrics of Quality of Service.

Correlation is never 0, but at least very low.

French Heures Creuses and the Tarif d'Achat are examples of this.

We are aware that a bad controller can negatively influence the results obtained, but this is far from the case.

By finding the intersection of the supply and demand curves.

They might get to trade some energy, which changes their state of charge and consequently their future actions, but they do not schedule their load any differently because of the existence of the market, which was the goal in the first place

This is equivalent to the value function in reinforcement learning or dynamic programming.

Even though, implementing such strategies in a profitable way might be difficult.

(t mod D) ∈[12,[START_REF] Adrian | Optimization of prosumers flexibility in electricity markets[END_REF]

This is directly related to the partial consumption profiles introduced in chapter 2.

Linear ramp constraints have been defined in Section 3.1, Assumption 5 (A5)

https://www.ausgrid.com.au/Industry/Our-Research/Data-to-share/Solar-home-electricity-data

The code used for our numerical experiments can be found in https://github.com/gus0k/cdc20_code.

Since the load distribution does not impact the execution time, which is the metric we want to evaluate, we chose a simple one, although not realistic

13.7. NUMERICAL EXPERIMENTS

http://www.irricontrol.com.uy/Irricontrol

Some of these had to be estimated using the temperature and rain data during those days.

The cost of water for most farmers in Uruguay is negligible, particularly when compared with electricity costs and performance losses. Therefore, an increase in the irrigation time, and consequently in the water usage, does not impact the overall irrigation costs.

To solve for C 1 , we set α tj = 1, ∀t, j.

http://yann.lecun.com/exdb/mnist/

If the solution is not unique, additional information will have to be provided to select among them.

The forecasting technique should be deterministic and clearly specified for border cases.

Unless he/she needs to simulate the benchmark but even then, we envision that open source software will be available to simplify that task

1.0 0.50 0.000 (0, 1, 2)

1.9 0.95 0.030 (because we know that the constraint is satisfied in the continuous case). The maximum of all those violations is clearly achieved in a coalition with N -1 players. That completes the proof.

We proceed to show how Proposition 11.3 is reflected in the example introduced in the previous subsection.

Example 11.3. Continuation of the Example 11.2.

Recall the game introduced in Subsection 11.3.1 with associated coalitions whose costs are described in Table 11.2. The costs and optimal battery sizes of the continuous version of the game are given in Table 11. 3. The last column shows the difference in cost between the discrete problem and the continuous one.

Applying Theorem 11.2 and Equation (11.3), we obtain the vector y = (0.45, 0.3, 0.2) that is in the core of the continuous game. Then, the approximation of a payoff vector of the discrete game is obtained by applying Equation (11.16) with v d (N ) = 0.98, v c (N ) = 0.95 yielding ŷ = (0.4642, 0.3094, 0.2063)

From Proposition 11.3, we know that an upper-bound on is given by: 0.98-0.95 0.95 (0.45 + 0.3) ∼ 0.0236. We can verify that indeed M (ŷ) = M (1,2) (ŷ) = 0.5157 -0.5 = 0.0157 < 0.0236.

Numerical Simulations

In this section, we numerically evaluate the performance of coalitional storage games using real data, and we quantify the consumers' benefit obtained by implementing such a scheme.

We are interested in answering the following questions. What is the actual benefit for consumers of using the scheme? What is the optimal size of the scenario set and how is it reflected in the output? How bad (if at all) is in practice the approximated solution introduced in Section 11.3.2? And, finally, do we achieve an increase in the number of storage units in the grid while being profitable for end users?

A realistic assessment of the performance of the scheme should be made on an out-of-sample dataset, i.e., using consumption that corresponds to days after the purchase of the battery. Unfortunately, the studied model does not explicitly specify how to do that. Therefore, two alternatives, both related to the original problem, are considered and evaluated here, as follows.

First, let y ∈ core(v) be a vector in the core of the game, solved by generating scenarios from day 1 until day D = |Ω|. The first proposed technique is to obtain a new vector y d for each day in the future D + 1, D + 2, . . . so that vector y d is efficient for day d (the same of payment of each player is exactly the cost incurred during the day) and each player pays proportionally the same as they did in y. We shall call this technique Keep Proportions. The second proposed technique is to fix the battery size obtained by the shared investment (using a discrete battery size and stochastic load profiles), and then employ Equation (11.3) in order to obtain the new payments for each player (using the consumption of the new day). We shall call this technique Re-Solving. Finally, we employ two other cost assignments as benchmarks, as follows. First, we consider the Default, which is the cost incurred by each player if they decided not to invest in storage at all and pay the default price for their electricity. In addition, we consider the Individual cost, which consists of the cost paid by each of the players if they decide to buy a battery on their own if it is profitable. It coincides with the Default cost if the optimal battery size is 0, but it total cost of the coalition in which the maximum violation occurs, that is:

, where S * is the coalition for which the maximum violation occurs.

We generated 30 games for each number of players between 3 and 10 (inclusive) using 40 scenarios in each game. The box plots in Figure 11.2 depict the change in the relative violation of the constraints as the number of players changes. The values are only for the cases when the core of the discrete game is non-empty as it is impossible to measure the distance to the empty set.

As can be seen, as the number of players increases, the violation becomes less meaningful, in line with the results of Lemma 11.1. This supports the claim that it is better to have larger coalitions, as they are more robust in some sense.

Overall performance

We conclude the numerical evaluation by considering the overall performance of the model. A central question is whether the cooperative game can be an efficient method for increasing the number of storage devices in the power grid without outside incentives.

To do so, two measures are needed, namely: the economic benefit of each consumer of participating in the coalition, and the number of extra storage units that are installed.

We simulated 96 coalitional storage games, varying the number of players N ∈ {15, 30, 45}, the number of scenarios used for making the decision |Ω| = W ∈ {30, 45} and the number of days after the investment in which the decision was evaluated F ∈ {15, 30}. For each combination of parameters, 8 games were created by sampling the load profile of different users.

For the two techniques used to distribute the costs during the test days and the two benchmarks, Figure 11.3 depicts the total cost of 10 of the players for one such game. Although some of the players do much better by joining the coalition, others are worse off. In particular, observe that for player 4 the technique is very negative, while for player 6 it is highly profitable. This can happen if the scenarios do not adequately represent the loads of each user In our numerical simulations, the cooperative scheme achieved an increase between 100% and 250% in the amount of storage hosted in residential premises compared to the setting in which consumers invest individually, when it was profitable for them to do so. Accordingly, we believe that coalitional games offer solution concepts that are very well positioned to boost the number of distributed storage devices in Smart Grids. Furthermore, some consumers might consider the tasks of installing storage in their homes overly complicated. In such a case, opting to participate in collective storage (which might not require more than a simple agreement) may offer a more attractive approach to adopting storage.

It is important to note that a coalitional game cannot make a battery profitable if the gap between the high price and the low price of electricity is lower than the amortized cost of the battery. That is, the viability of buying storage first depends on the electricity tariffs and on the storage prices, and obviously our model (and its implied scheme) cannot get around this problem. Nevertheless, if storage is barely profitable, participating in a shared investment will provide higher margins of profit at a reduced risk (which is shared among the members of the coalition). Relatedly, we expect battery technologies to improve and decrease their costs in the future, hence increasing the applicability of the cooperative game approach proposed in this study.

Our study indicates several directions for future research that could further increase the benefits of the proposed solution. One such is to consider advance tariff models, for example those that include peak demand charges; another is to find efficient schemes to update an existing coalition once either new players wish to join or present ones wish to leave. Finally we are considering the substitution of the second-stage optimization problem in our model with a more detailed version, such as the one used in [START_REF] Md | Optimal storage arbitrage under net metering using linear programming[END_REF], [START_REF] Kiedanski | Sensitivity to forecast errors in energy storage arbitrage for residential consumers[END_REF].

Algorithm 3 Computing core payoffs

) fixing variables for hardware and changing cost to 0 4: h c = y 1 -y 2 5: p ← (0, . . . , 0) 6: for d ∈ 1, . . . , D do 7:

gap ← Core(δ) Error gap game 9:

p ← p + y d + gap + h c 10: end for 11: return p Furthermore, we will study the difference between the theoretical ROIs obtained by solving the analytical model with the stochastic scenarios and the real ROI obtained by simulating the system using the real load profiles experienced by the participants.

Setup

For the consumption data of the players, we will use the Ausgrid dataset that was presented in the previous chapters. Each simulation will randomly sample the consumption profile of 30 users. We will use the last W = 20 days of past consumption as the scenarios, where the scenario corresponding to day d contains all the information about the consumption of each of the players in that day ω d = N i=1 l i d . Furthermore, we simulate D = 7 days into the future for evaluation the empirical performance of the cooperation among players.

In our experiments, all players are subscribed to the same Time-of-Use tariff (this is required by the model, in which all participants have the same costs for electricity). The ToU consists of two periods, a cheap period that runs from 23hs to 7hs at a price of 0.138€ and an expensive price in the remaining hours at a price of 0.178 € per kilo watt hour of energy consumed. We assumed that there is a constant Feed-in-Tariff at which participants can inject their surplus: 0.05 € per kilo watt hour.

Participants can invest in energy storage and photovoltaic panels. For the investment in energy storage, the simulations were configured after the parameters of a Tesla's power wall with 13.5 kWh of capacity, 5kW of up and down ramp constraints and round trip efficiency of 0.95 2 . For the cost of the battery we simulated different scenarios with an amortized cost per day ranging from 1.2€ per day (4380€ in 10 years) up to 1.9€ per day (6935€ in 10 years).

Regarding the investment in photovoltaic panels, we assumed that participants could invest in multiples of 3kW photovoltaic panels. To simulate the generation of such panels we sampled a uniform random variable U[- 3 4 , 0] i.i.d for each time-slot between 6h and 18h, for an average generation of 9 kWh. The amortized cost of a single installation in a day was estimated to be 0.7€, which accounts for almost 4000€ in 15 years. An interesting remark is that since we did not impose constraints in the maximum size of the PV installation, if the generation in a day is sufficiently large, the optimal solution becomes buying infinite PV panels, and the optimization problem is unbounded. With the current set of parameters, the probability of that happening is quite low.

Results

We begin this section by looking at the improvement of the players' costs while cooperating and without cooperation. In Figure 13.1 we compare two methods for distributing the final costs among players. The first one is based on the core and was derived using Algorithm 3, while the second one consists on splitting equally among all players the reduction obtained while cooperating. We can observe that the core, even though stable, is "unfair" and assigns a large reduction to some players will making others worse off than individually (this is due to the approximation involved in using scenarios and a finite look forward). While the fair split reduced the costs of all players and no players is worse off (because they are literally obtaining a profit on top of acting individually), those payments might not be stable in the sense that a sub coalition might be willing to deviate. therefore some productivity will be lost. On the contrary, the green RZD value is smaller than the RAW level and there is no loss in production. An equivalent way of stating the problem, which will be used later, is the following: the water content in the ground (WCG, Equation (A.9)), which is the amount that is left after removing from the FC the water that has been depleted (RZD), has to be larger than the "refilling point", i.e, the amount of water that is left after removing from the FC the RAW. We denote this latter quantity as the level of hydric stress (N, Equation (A.3)) or as the refilling point.

In Equation (A.4), S t denotes the amount of hours used for irrigation on day t. Furthermore, we will use the variables Y tj ∈ [0, 1] to denote the percentage of the time that the irrigation system was working during the j hour of the day t. It naturally holds that

A.2.2 Electricity tariffs

First, we consider a three-period Time of Use (ToU) tariff as described in Equation (A.10). There, p l , p m , p h denote the low, medium and high prices (correspondingly), P is the power (kW) consumed by the pump used for irrigation and T l , T m , T h ⊂ [START_REF] Brigham | Fundamentals of financial management[END_REF] correspond to the hours in which each of the rates is available.

Second, we consider a family of rebates on top of the three periods ToU described by Equation (A.10), designed to incentivize consumption whenever there is surplus of generation. Equation (A.11) defines these rebates, which we will refer to as Opportunity Offers (OO) in the rest of the chapter.

Opportunity offers work as follows: for a given hour and day (i, j), if the consumption Y ij is greater than a threshold α ij that emulates average past consumption, the cost to be paid is a fraction β ∈ (0, 1] of the original cost. If no OO is available, α ij can be set to 1, which yields the same result. The variables P, p l , p m , p h , T l , T m , T h have the same interpretation as in Equation (A.10).

The motivation behind this tariff is that the TEC has contractual obligations to all the surplus of certain green generators but there are days in which such generation is larger than the actual consumption. In those days, they offer it at a lower price to anyone willing to consume more than they would normally do.

A.2.3 Optimization Problem

The optimization problem defined in Equation (A.12) seeks to find an irrigation assignment guaranteeing the required level of water in the ground and minimizing the cost of irrigation. The two different tariffs (Equations (A.10),(A.11), represented by C ) will be used as cost functions.

Observe that constraint A.12b is equivalent to {Q t ≥ N t }, The function O t : [0, 1] 24×D → R is not differentiable and, for some combinations of the parameters, the set {O t ≤ M t } is not convex.

ChapterB

Design of A Benchmark for Demand Response Applications

Demand response (DR) or grid flexibility (GF) encompasses the ability of end-customers to change their energy consumption in response to incentives, with the goal of improving the operation of the power grid. Typically, examples of DR include Time-of-Use tariffs, where consumers are offered different electricity prices at different times of the day (such having a cheaper electricity price during the night) or direct control of appliances such as water heaters by a central utility (without impacting the comfort of users). In this chapter we attempt to formulate some requirements that would be needed to implement an agnostic benchmark for demand response.

B.1 Introduction

B.1.1 Motivation

Climate change is one of the biggest challenges ever faced by humanity. A recent book by [START_REF] Hawken | Drawdown: The most comprehensive plan ever proposed to reverse global warming[END_REF], compiles a list of techniques to help reverse climate change. About Grid Flexibility, it mentions that the impact was not measured because the system is too complex to properly assess its benefits. Even though demand response is a vital tool for enabling the energy transition and the deployment of renewable resources, there seems to be no reliable and reproducible measure for the performance of such techniques. Consider a small low voltage (LV) grid with several households and no demand response program in place. One could wonder what is the most effective (in terms of consumption change and cost of implementation) DR program that could be deployed to attain certain level of flexibility. Should we encourage users to install smart appliances and a Home Energy Management System (HEMS)? If there are appliances, is it better to use dynamic pricing, create local energy markets or directly pay users to gain the control of their HEMS? Would the results change if there were electric vehicles in every household?

B.1.2 The role of benchmarks

Standardized datasets and benchmarks exist and are important in many STEM areas. For example, in the artificial intelligence community, image recognition is arguably one of the most developed areas of research. There are many reasons for this, but the fact that anyone can develop a machine learning algorithm, evaluate it on a dataset such as MNIST [START_REF] Deng | The mnist database of handwritten digit images for machine learning research [best of the web[END_REF] and know whether the implementation is working as expected is a major benefit. In particular, for the MNIST dataset there are leaderboards that contain the performance of several algorithms 1 . Even though the superior performance of one algorithm over another one for a specific dataset should not be sufficient to claim that one is better than the other, such comparison across different datasets might prove to be a good indicator. Benchmarks are not exclusive of the Image Recognition community. Indeed, examples in other areas include: community detection in graphs [START_REF] Lancichinetti | Benchmark graphs for testing community detection algorithms[END_REF] and natural language processing [START_REF] Wang | GLUE: A multi-task benchmark and analysis platform for natural language understanding[END_REF]. Even in the power system community, the IEEE X-BUS systems [START_REF] William | Radial distribution test feeders[END_REF], [START_REF] Subcommittee | Ieee reliability test system[END_REF] provide benchmarking capabilities for some applications, but not specifically for DR.

B.2 Related Work

There are many projects that are similar to what we are proposing but do not solve the problem quite right. For example...

B.3 Benchmark specification

Drawing a parallelism with the image processing community: the definition of an image is clear. It is a 3 dimensional matrix where each entry represents one of the RGB values of a pixel (for coloured images). Once everyone agreed on what an image is, many benchmarks (datasets) could be designed to solve different tasks: images with text, with objects, with faces, etc. There is not a clear analogous definition of what an "image" is in Smart Grids, in particular for DR applications. In this section we take the first steps towards a definition that will enable the systematic treatment of DR programs.

The requirements can be divided into four categories:

1. Energy Generation.

2. Power Grid Specification.

3. Consumer Specification.

Performance Metrics.

A brief description of each one of them can be found below.

Energy Generation

The amount of produced energy available for consumption, its sources and their respective location in the grid should be key components of a demand response benchmark. In particular, how much renewable energy is available at each point in time will be needed for measuring DR performance as the ability to match consumption and renewable generation. Another piece of information that might be relevant consists on weather information such as temperature and cloud cover. That kind of information will be needed in more precise studies dealing with seasonal effects of demand response and its correlation to meteorological effects. This could also be relevant for the Consumer Specification.

Power Grid Specification The power grid can be seen as a graph, where edges are transmission lines and loads as well as generators are connected at the nodes. A detailed specification of the physical characteristics of each component will be required to produce realistic simulations. Formats already exist to provide detailed information about the grid topology and a well designed benchmark should reuse already established specifications. One such example is OpenDSS [START_REF]OP Model and OpenDSS Storage Element. Opendss manual[END_REF] 2 .

Consumer Specification Consumers should be modeled in a manner that allows the users of the benchmark to derive the consumer change in behaviour in response to a change in the system. A simple way of doing so is by providing a set of appliances each consumer owns, together with their required usage. For example: nonflexible appliances such as TVs or lightbulbs should be paired with specific usage times, while washing machines or dishwashers (flexible appliances) could require only a start time and a completion deadline. The default electricity tariff to which each household is subscribed has to be included. Those prices, together with the list of appliances, their usage (and assuming that consumers act rationally to minimize their electricity bill) and possibly some additional details should provide enough information to derive each agent's electricity consumption. Examples of possible problem formulations can be found in [START_REF] Paterakis | Optimal household appliances scheduling under day-ahead pricing and load-shaping demand response strategies[END_REF], [START_REF] Chen | Uncertainty-aware household appliance scheduling considering dynamic electricity pricing in smart home[END_REF] and [9].

Performance Metrics Having a standardized measure to evaluate demand response programs is critical to the idea of the benchmark and we believe it should be part of its specification. In this regard, measuring the mismatch between renewable generation and energy consumption 3 seems to be a good choice, in contrast to the traditional peak reduction. Further discussion on this point can be found in subsection B.3.5. Finally, special attention should In this section we provide a minimal example of how a consumer can be modeled from the benchmark data. The appliance usage of one consumer, Camila, is provided in Table B.1. Consumption times are specified in hours, the minimal unit of time. Light bulbs are non-flexible appliances and the provided range is exactly the period of time in which they will be on. On the other hand, the washing machine is flexible, and the first pair of brackets defines the interval in which the consumer finds acceptable that the machine operates. The second pair of brackets indicates for how long it should (continuously) run once it starts. The electricity rate follows a Time-of-Use tariff with a cost of 15 ¢/ kWh between 14h -22h and 10 ¢/ kWh at other times. The default consumption of Camila during Day 0 can be found by solving the Optimization Problem as defined in (B.1a). In it, c t denotes the energy consumption at time-slot t, l t is a binary variable that indicates whether the light bulb is on or off at time-slot t and z t plays the same role for the washing machine. w t is an auxiliary variable that decides when the washing machine will turn on. An optimal solution can be found by turning the washing machine on before the change in price (for example, w 10 = 1) at a total cost of 23.5 ¢. This example contains many implicit assumptions such as the time resolution and the time horizon used to solve the optimization problem. A thorough discussion on such assumption was provided in Section B.

B.5 Example applications

In this section we will present how the benchmark can be used by different demand response programs. The examples are meant to be illustrative and by no means represent the full scope of what can be achieved with the benchmark.

B.5.1 Optimal Time-of-Use tariff design

In [START_REF] Yang | A game-theoretic approach for optimal time-of-use electricity pricing[END_REF], the authors describe how to design an optimal time-of-use tariff using Stackelberg games under the assumption that agents have flexibility. The cost of each agent (Equation (B.3)) is a function of her/his base consumption and how much they deviate from it.

where p i t is the cost of electricity for player i at time-slot t, d i t is her/his original consumption profile, l i t is the new consumption, and α, β are the parameters that modify the shape of her/his flexibility. For brevity, we omit the full formulation of the game.

From the benchmark, the coefficients α i t and β i t of such functions could be estimated to find the best fit. This could be achieved by comparing the costs paid by each agent with the old demand d i and the new one l i . With the obtained coefficients, the game can be solved and the optimal tariff determined. Finally, the obtained tariff can be used to simulate the benchmark and measure the real impact of the obtained tariff in the grid.

B.5.2 Energy trading: Auctions and Peer-to-peer

There is an important area of research dedicated to energy trading among consumers as we have studied throughout this thesis.

For trading across several periods, the user of the benchmark could gather the consumption and generation data from the DwF. However, if the application requires households to use a forecast of the market trading prices instead of the real ones, the benchmark could be be simulated using those forecasted prices instead of the observed prices that come with the benchmark. Having obtained the consumption profiles, we can envision at least two kinds of usages. In the first one, households trade their consumption in each time-slot, but the results of the market does not affect the energy consumed in future time-slots. In that case, at the end of the simulation, the total cost of each player would be the sum of the costs incurred in the market and the costs of the energy that could not be traded (and was paid at the normal price). Alternatively, the results of the market could change the behavior of each player. In that case, with the market results each player would re-run their optimization problem to find their new schedule for the remaining time-slots (that was the approach used in this thesis). Observe that the described application does not require the user of the benchmark to deal with the appliances level 7 , as he/she can simply extract the default consumption profile if needed.

If the trading environment changes the consumption profile of the players, then it can be compared against other techniques such as the optimal tariff design described in the previous subsection to understand which one achieves more grid flexibility. On the other hand, if the trading scheme does not change the consumption of agents, then it can be compared at the profit level alone.

B.5.3 Optimal investment

One interesting application of the benchmark is to understand what is the optimal investment with a fixed capital that can be made to improve the flexibility of the grid. For example, one possibility could be to invest in storage and distribute it in crucial points in the grid. Another option would be to offer a small sum of money to households in exchange for gaining the control of some of their appliances, while keeping their desired comfort level. For the later example, a possible way to determine how much money should be given to each household could be to decide on a sum that guarantees that each consumer obtains a reduction of X% of her/his original bill. With the introduced modifications in the grid, the benchmark could be simulated and the flexibility obtained compared to other techniques such as energy trading and optimal tariff design.

B.6 Machine Learning applications

Once benchmarks are established, there are many possible machine learning (ML) applications. Here, we provide two ideas on how ML can be useful in applications related to predicting the value of the unmet renewable energy generation. Observe that from the specification presented in Section B.3, it is possible to build a massive number of datasets by creating variations of the topology, the appliances available, the number of consumers, etc.

ChapterC

Energy cooperatives

C.1 Battery efficiency

Battery efficiency can be modeled by two constants η c , η d ∈ (0, 1] representing the efficiency loss for charging and discharging, respectively. In our model, the battery only charges during the cheap period, and only discharges during the expensive period. In the former, this affects the total energy that can be charged in the battery, or, in other words, the total available capacity: instead of having a battery with capacity C B , the available capacity will be η c C B . During the later period, the efficiency of the battery also affects the energy that is available for use. We can assume that, if there are discharging losses, the output power will remain the same yet the battery will discharge faster. Overall, this can be modeled as having a smaller battery with capacity η c η d C B . By introducing a change of variables D = η c η d C B we get back to our original problem.

C.1.1 Battery operation

Although we do not directly deal with battery operation, in the following we briefly discuss how this can be considered. A battery owned by the coalition S should be charged a quantity l S (Equation 10.2) during the cheap period. This can be done at a uniform (or any other) charging rate as long as the total energy in the battery is l S . During an arbitrary point t in the expensive period, the battery should be discharged at a rate of min{δS, l S t }. If additional energy is still needed, it should be bought from the utility.

C.2 Matrix formulation of the optimization problem

In this appendix we provide the matrices and vectors involved in the matrix formulation of the linear optimization problem (11.2a).