Biao Yin

Chen Zheng

Xinyi Liu

Haox- Uan Liu

Zhaoyu Liu

Dongdong Zhao

Zhixue Zhongliang Li

United Nations Environment Programme [UNEP, 2014] indicates that buildings use about 40% of global energy, 25% of global water, 40% of global resources, and they emit approximately 1/3 of Green House Gas (GHG) emissions. With the development of human society, environmental issues and energy crisis have drawn more and more attention. In this background, buildings can offer a great potential for achieving significant GHG emission reductions and energy saving. On the other hand, people spend greater part of their time in buildings. As the quality of indoor life is increasingly considered as of paramount importance, more and more occupants constantly seek to improve comfort in their indoor living and working spaces. Meanwhile, the popularization of the concept of home office makes the productivity in residential buildings economically significant. How to manage buildings in a proper way is therefore a subject of uttermost importance.

Corresponding to the increasing demands for environment, energy, and comfort, smart home has become an active research topic. Smart home, also known as automated home or intelligent home, is the residential extension of smart building. It is an automation system that incorporates a diverse of home appliances with ubiquitous computing, networking, and controlling abilities, forming an "Internet of Things", to provide better home life service and experience to residents [Ma et al., 2012, Ricquebourg et al., 2006]. One smart home system does not simply mean turning devices on and off automatically.

It can also monitor their working states and detect the environmental changes to make better decisions. To design and implement such a system, two important factors, comfort level and energy efficiency, should be considered carefully.

Generally, there are three basic factors that determine occupants' indoor quality of life: thermal comfort, visual comfort and indoor air quality [START_REF] Dounis | Advanced control systems engineering for energy and comfort management in a building environment-a review[END_REF]. Among them, thermal comfort attracts our major attention, because the overriding energy consumption in residential buildings is caused by using auxiliary space heating systems, which accounts for 45% of total residential energy consumption [BEDB, 2014]. Hence, one research point is to guarantee occupants' thermal comfort, and at the same time to minimize the energy cost based on advanced computing and control techniques. This can be regarded as a way to improve energy efficiency.

In addition, energy efficiency is also related to the performance of smart grid, which is a modernized electrical grid using information and communication technologies to improve the efficiency, reliability, economics, and sustainability of the production and distribution of electricity [DoE, 2015]. In this sense, smart homes can be treated as integral additions to the smart grid. Demand response is one of the key energy management strategy adopted by both of them. It refers to changes in electric usage by end-use customers from their normal consumption patterns in response to changes in the price of electricity over time, and helps to reduce peak demand, therefore tempering the need to operate highcost and high-emission generating units as well as cutting down residents' electricity bills [Siano, 2014]. Communication between smart homes and the smart grid allows flexible home appliances shifting working time or adjusting working power during on-peak periods to balance the usage of electricity. However, when the load management of a set of homes work selfishly, new peak-loads will occur during lower electricity price periods and they are called peak rebounds [START_REF] Safdarian | A distributed algorithm for managing residential demand response in smart grids[END_REF], which may damage the power grid, cause unforeseen disasters, and reduce the global profit. How to manage smart homes to avoid this negative situation to improve occupant comfort and energy efficiency is therefore the second research point.

1.2/ OBJECTIVES AND CONCERN OF THIS WORK

The objectives and major concern of this work could be summarized as:

(i) Propose an advanced control method for space heating system in smart home;

(ii) Design and implement a cooperative intelligent system for management of multiple smart homes.

On one hand, people spend a great part of time indoor, so their satisfaction with the thermal environment is important, because it influences not only habitants' health, that thermal discomfort has been known to lead to sick building syndrome symptoms, but also productivity of home workers [START_REF] Myhren | Flow patterns and thermal comfort in a room with panel, floor and wall heating[END_REF]. Hence, the first objective of this research is to propose an advanced method for space heating system to maintain optimal thermal comfort. After implementing this intelligent system, the indoor air temperature can fit occupant's states dynamically and automatically. Meanwhile, energy wasting through overheating is avoided.

On the other hand, a smart home always contains a variety of household appliances besides space heaters. Without coordination, selfishly operating these devices of multiple homes at the same time will make the load on the main grid extremely high. Even with the help of demand response enabled by time-varying prices, severe peak rebounds at periods with lower prices may happen. Therefore, the second objective of this research is to design a cooperative system to optimize the scheduling of the controllable devices in multiple smart homes, achieving the trade-off between minimum electricity bill and maximum consumer's satisfaction and the utility of main grid.

1.2.1/ AN INVERSE PMV OPTIMIZATION BASED HYBRID INTELLIGENT CON-TROL FOR THERMAL COMFORT IN SMART HOME

In literature, thermal comfort is the condition of mind that expresses satisfaction with the thermal environment and is assessed by subjective evaluation [ASHRAE, 2004]. A human being's thermal sensation is mainly related to the thermal balance of his or her body as a whole. This balance is influenced by physical activity and clothing insulation, as well as many environmental parameters. As a result of individual differences, specifying a thermal environment to satisfy everybody is impossible, for there will always be a percentage of dissatisfied occupants. But it is possible to specify environments predicted to be acceptable by a certain percentage of the occupants. The Predictive Mean Vote index (PMV) derived by P. O. Fanger [ISO7730, 2005] stands among the most recognized thermal comfort models, which predicts the mean value of the votes of a large group of persons based on the heat balance of the human body.

Based on the PMV thermal comfort model and advanced networking, computing and sensor technologies, improving thermal comfort in smart homes is a practical example of employing Cyber-Physical Systems (CPS). CPSs integrate computation, networking and physical processes, in which embedded computers and networks monitor and control the physical processes with feedback loops where physical processes affect computations and vice versa [Lee, 2006]. Just like the way that Internet has changed how human interact with each other, CPSs will change how human interact with the physical world around them [START_REF] Rajkumar | Cyberphysical systems: the next computing revolution[END_REF]. When there are occupants in a smart home, the proliferation of powerful but low-cost sensors make it possible to capture thermal comfort related parameters from the physical world. Through ubiquitous networking technologies the information can be exploited by widespread high-capacity computing units to intelligently control space heating devices in order to obtain superb thermal comfort for occupants.

Before bridging the PMV model with the heating system, more efforts are needed. The regular PMV computation is a function which takes environmental-dependent variables and personal-dependent variables as inputs, then calculates a scalar number indicating thermal comfort as the output. Nonetheless, in this research, we propose to inversely employ the PMV model, that is to say in this situation air temperature is set to be the output, while an optimal thermal comfort value with the other former inputs are set to be inputs. Therefore, by inversely computing the PMV model, an optimal air temperature preference based on the present environmental and personal states can be obtained.

However, because PMV is non-linear and non-convex [START_REF] Kelman | Analysis of local optima in predictive control for energy efficient buildings[END_REF], the inverse computation of PMV is also non-linear and non-convex. Hence, the idea of using meta-heuristic algorithms to search the desired solutions, which are optimal air temperatures in this work, is natural. The Particle Swarm Optimization (PSO) algorithm, which is originally proposed by [START_REF] Kennedy | Particle swarm optimization[END_REF], Shi et al., 1998] and derived from simulating social behavior as a stylized representation of the movement of organisms in a bird flock or fish school, has turned out to be an efficient solution. Based on this algorithm, an optimal or quasi-optimal indoor temperature can be acquired. By setting this temperature value as the preference for the space heating system, which uses Proportional-Integral-Derivative (PID) to regulate the heating power, the room temperature can be controlled and the thermal comfort can be guaranteed.

In order to validate the designed system and control method, a mathematical thermal model is used to simulate thermal variation of a smart home. Based on the heat balance rule [START_REF] Achterbosch | The development of a comvenient thermal dynamic building model[END_REF], the thermal dynamics of a room can be expressed as a thermal network, which is analogous to an electric circuit network by regarding heat flow as electric current, and temperature as electric potential respectively. Then temperature changes of different components of the room, including room air, wall, roof, and floor can be described by first order differential equations. Based on this model, a series of computer simulations are conducted. The results indicate the proposed CPS can both provide better thermal comfort and consume less energy compared with the systems adopting traditional control methods and intelligent control methods presented in the state of the art.

1.2.2/ AN ORGANIZATIONAL MULTI-AGENT SYSTEM BASED ON COOPERATIVE PARTICLE SWARM OPTIMIZATION FOR MULTI-HOME SCHEDULING

Towards further improving energy efficiency of individual homes and the grid utility, an architecture that includes a set of smart homes and a smart grid is considered. In fact, this architecture is a holarchy, which is a hierarchical structure composed of whole-part constructs, called holons, as substructures [Wilber, 1998]. By taking the advantage of smart grid, different homes can communicate with the grid so that bidirectional information exchange and energy flow are both possible between them. Each home is composed of a variety of electric appliances. These devices consume energy, and provide different services. According to their controllability, these appliances are divided into fixed devices, which have fixed load profiles such as electric oven, refrigerator, TV, etc., and controllable devices including power-shiftable devices and time-shiftable devices. Power-shiftable devices represent these devices whose working power is changeable to fit variations of environmental status, and at some circumstances they can decrease the power at the range that occupants can accept to reduce load on the grid. Time-shiftable devices are these appliances that can shift the power consumption time within a preferred working period and when they start to work they are in a constant power for a certain period.

The system considered is complex, since different smart homes are geographically distributed, each of them is an autonomous entity composed of diverse devices to fulfil occupants' various requirements, and moreover, to balance the total load on the grid it needs them to cooperate with each other. In order to model this system, a Multi-Agent System (MAS) paradigm is used, which is a computerized system composed of multiple interacting intelligent agents for modeling autonomous, distributed, dynamic and complex systems. In this paradigm, components of different architecture levels could be designed as agents, such as smart grid agent, smart home agent, space heating agent, washing machine agent, etc.. However, as Jennings and Wooldridge have pointed out, Agent Centered Multi-Agent Systems (ACMAS) may suffer some drawbacks, such as the patterns and outcomes of the interactions are inherently unpredictable and predicting the behavior of the overall system based on its constituent components is extremely difficult, when engineering large systems [Jennings, 2000]. Therefore, an Organization Centered Multi-Agent System (OCMAS) is preferred to model the system.

In a smart home, different electric appliances are operated to work or planed to work in order to satisfy occupants' requirements. When they work properly, occupants' comfort can be guaranteed, whereas running defectively or unreasonably may bring dissatisfaction.

Therefore one of the system objectives is to minimize the dissatisfaction caused by these appliances in each smart home. Furthermore, demand response enabled by time-varying prices can propel customers to change their consumption patterns for a more economical purpose. Hence, decreasing the household cost of electricity usage should be considered in the system. However, a non-coordinated response of customers for time-varying prices may lead to severe peak rebounds at periods with lower prices. Towards flattening the total load profile, the variance of energy consumption on the main grid need to be minimized. Corresponding to these three situations, the objective function of each household is to minimize the integration of these three sub-objectives.

Based on the objective function, device agents in this system need to find an optimal or quasi-optimal policy to control the corresponding physical appliances, for example, space heating agent should find the appropriate heating power and washing machine agent should find the suitable operation time. As the problem involves multiple households, and each of them contains multiple controllable appliances, using the standard PSO implies that a full dimensional vector should be constructed for each particle. This will cause the "two steps forward, one step back" problem [van den Bergh et al., 2004], which can make the convergence very hard. In order to solve this problem that involves a large number of decision variables, the Cooperative Particle Swarm Optimization (CPSO) algorithm proposed in [van den Bergh et al., 2004] is adopted and developed.

1.3/ PLAN OF THE DOCUMENT

After the brief overview of the proposals presented in this thesis, this section introduces the overall organization of this document. According to the objectives described in Section 1.2, this thesis is organized into three main parts. The various chapters in this document and their respective organizations are summarized in Figure 1.1.

The first part, including Chapter 2 and Chapter 3, presents the state of the art in terms of context, problem and solution.

The second part, including Chapters 4 and 5, presents a cyber-physical system with an inverse PMV optimization based hybrid intelligent control method to improve thermal comfort in smart home.

The third part, including Chapters 6, 7, and 8, presents an organization centered multiagent system adopting cooperative particle swarm optimization for multi-home scheduling.

The detail of each chapter is described below.

Chapter 2 presents the state of the art in terms of context and problem this work is covered, including the context related to thermal comfort and thermal comfort models.

When a set of smart homes are considered, the multiple home scheduling problem is introduced.

Chapter 3 presents the state of the art in terms of solutions for thermal comfort improvement and multiple home scheduling.

Chapter 4 presents our proposed cyber-physical system with a hybrid intelligent control approach for thermal comfort improvement.

Chapter 5 presents a building thermal model, which is based on the heat balance theory and expressed by an electric circuit network. By using this model, a series of simulations are conducted to verify the proposed system and control method.

Chapter 6 presents a holarchy of smart grid system, of which the two bottom layers attract our more attention and are modelled by an organization centered multi-agent system.

Chapter 7 describes the objective of the system, which belongs to a multi-objective optimization problem. A cooperative particle swarm optimization algorithm is proposed to solve this problem.

Chapter 8 presents the models of some controllable devices and a case study for the proposed system and method.

2.2/ PRINCIPLES OF THERMAL COMFORT

Although there has been much effort to define and measure thermal comfort, it is still far from fully understood. According to the American Society of Heating, Refrigeration and Air-Conditioning Engineers (ASHRAE), thermal comfort is the condition of mind that expresses satisfaction with the thermal environment and is assessed by subjective evaluation [ASHRAE, 2004]. In the following subsections the most important knowledge about thermal comfort will be introduced.

2.2.1/ THERMOREGULATION OF HUMAN BODY

Human being is an endotherm and his normal physiological life is maintained by a nearly constant internal temperature. Due to his metabolic activities, the human body continuously produces heat, which must be dissipated into the environment to prevent hyperthermia, whereas excessive heat loss may result hypothermia. Therefore, there must be a physiological mechanism to handle this. The hypothalamus is the central control organ as a thermostat for body temperature control, of which the core temperature T core is kept generally close to 37 • C while the skin temperature T skin (generally has an average value of 33 • C ), which can be easily affected by surrounding conditions, is more variable.

The hypothalamus can be informed the thermal stimulus by electrical impulses from various thermal receptors in the brain and the skin [Candas, 1999]. According to the sensory information, two general mechanisms, behavioural and physiological ones, used to keep the thermal equilibrium of the human body, can be triggered. Behavioural responses are rapid adaptive actions, such as keeping far away from heat sources, changing clothes, turning on/off or up/down space heating systems, etc.. Physiological responses are much more complex processes and occur as internal body regulation. They can be summarized as follows. ( 1) When the core temperature is higher than the setpoint, redundant heat inside the body will be brought to skin to dissipate with a much higher rate due to the increased blood flow caused by vasodilatation of vessels. Sweating occurs to cool down the skin by evaporations, which increases the heat exchange rate between the internal body and the skin. ( 2) When the body temperature drops below the setpoint, vasoconstriction, which is aroused by heart rate decreasing and blood flow reducing, and pore contraction can both minimize the internal heat loss, and at the same time the increasing of muscle tension, known as shivering, can produce internal heat. Obviously, thermal comfort of human body is decided by many factors.

2.2.2/ THERMAL COMFORT PARAMETERS

According to [ASHRAE, 2004], there are six parameters that mainly influence thermal comfort through heat exchange between human body and the environment, including air temperature, mean radiant temperature, humidity, air velocity, metabolic rate, and clothing insulation.

Air Temperature

Air temperature T air is the time and location averaged temperature of the occupant's surrounding compound [Kranz, 2011]. It can greatly influence the heat exchange rate on the skin, the thermal equilibrium, and then the thermal comfort of a person. In much work, air temperature is chosen as the unique index to express thermal comfort.

Mean Radiant Temperature

Mean radiant temperature (MRT) is a hypothetical parameter and is defined as "the temperature of an uniform black enclosure that exchanges the same amount of thermal radiation with the occupant as the actual enclosure" [ASHRAE, 2004]. Radiation plays an 32CHAPTER 2. THERMAL COMFORT AND ENERGY EFFICIENCY IN SMART HOMES important role in the heat exchange between the human body and the environment. It can be estimated from measured values of the temperature of the surrounding surfaces and their positions with respect to the occupant:

T 4 MRT = N i=1 T 4 s f i F p-i , (2.1)
where

T MRT is the mean radiant temperature in • C,

N

is the number of surfaces surrounding the occupant,

T s f i is the temperature of surface i in • C, F p-i
is the angle factor between a person and the surface i.

Humidity

Humidity refers to the moisture content of the air and it affects evaporation of water from sweating surfaces of the skin [ASHRAE, 2004]. Although it may indirectly influence occupant's comfort that too low level of humidity will cause dry nose, skin and eyes while too high level of humidity will lead to sensation of stickiness, it plays a minor role when an environment is close to thermal comfort, since dissipation through sweat evaporation can be neglected [START_REF] Devonshire | The effects of infrared-reflective and antireflective glazing on thermal comfort and visual performance: a literature review[END_REF]. For example, in general a 10% change in humidity has the same influence as a 0.3 • C change of air temperature, while at low air temperatures the thermal comfort can be even considered as independent of humidity.

Air Velocity

Air velocity can influence heat loss through the skin. Convective heat transfer is roughly proportional to the square root of mean air velocity [ASHRAE, 2004]. On one hand, a rise in air velocity can contribute to compensate for thermal discomfort caused by relatively higher air temperature as a pleasant breeze. On the other hand, it may also lead to local thermal discomfort. Draught, that is defined as an unwanted local cooling of the body by air motion, is the most common cause of local discomfort [ISO7730, 2005].

Metabolic Rate

Metabolism exists in a human body through lifetime. The metabolic rate, which varies over a wide range and is in unit of met (1 met = 58.1W/m 2 ), quantifies the transformation rate from chemical energy into heat, that is dissipated to the environment, and mechanical

Clothing Insulation

Clothing insulation is the thermal insulation provided by clothing, the adjustment of which directly affects on the heat balance of a person, and therefore has a substantial impact on thermal comfort. According to [ASHRAE, 2005], the intrinsic clothing insulation R cl can be calculated as follows:

R cl = T sk -T o Q - 1 h • f cl (2.2) with T sk is the skin temperature in • C, T o is the operative temperature in • C, Q is the body heat loss in W/m 2 , h is the radiative heat transfer coefficient in W/m 2 • K, f cl is the clothing area factor.
Operative temperature is the uniform temperature of an imaginary black enclosure in which an occupant would exchange the same amount of heat by radiation plus convection as in the actual non-uniform environment [ASHRAE, 2004]. Clothing insulation I cl can be calculated as I cl = R cl /0.155 in a unit of clo, so 1clo = 0.155m

2.3/ THERMAL COMFORT MODELS

After knowing the main parameters that influence thermal comfort, how to evaluate thermal comfort according to present conditions is also important. Thermal comfort modeling basically started in the 1960s mainly for use in military and aerospace applications [Kranz, 2011]. In this section, three thermal comfort models are introduced, namely predictive mean vote model, adaptive thermal comfort model, and simplified comfort model.

2.3.1/ PREDICTIVE MEAN VOTE MODEL

The Predictive Mean Vote index (PMV) derived by P. O. Fanger [ISO7730, 2005] stands among the most recognized thermal comfort models, which predicts the mean value of the votes of a large group of persons on the 7-point thermal sensation scale, based on the heat balance of the human body. The 7-point thermal sensation scale separately indicates -3 for cold, -2 for cool, -1 for slightly cool, 0 for neutral or comfortable, +1 for slightly warm, +2 for warm, and +3 for hot. Moreover, the International Organization for Standardization (ISO) recommends maintaining PMV at level 0 with a tolerance of 0.5 as the best thermal comfort. The calculation of PMV is complicated, as expressed from Equation 2.3 to Equation 2.6. It is affected by the six variables as we discussed separately in detail before. Among them, it includes four environment-dependent variables: air temper-ature, mean radiant temperature, humidity and air velocity, and two personal-dependent variables: metabolic rate and clothing insulation, as shown in Figure 2.1. When calculating PMV, a relative humidity of 50% is often chosen, since the influence of humidity on thermal sensation is small at moderate temperatures close to comfort.

PMV Calculation

0 (Neutral) -1 (Slightly Cool) -2 (Cool) -3 (Cold) Figure 2.1: PMV Calculation PMV =[0.303 • exp(-0.036 • M) + 0.028] • {(M -W) -3.05 • 10 -3 • [5733 -6.99 • (M -W) -p a ] -0.42 • [(M -W) -58.15] -1.7 • 10 -5 • M • (5867 -p a ) -0.0014 • M • (34 -T air ) -3.96 • 10 -8 • f cl • [(t cl + 273) 4 -(T MRT + 273) 4 ] -f cl • h c • (t cl -T air )} (2.3) t cl =35.7 -0.028 • (M -W) -I cl • {3.96 • 10 -8 • f cl • [(t cl + 273) 4 -(T MRT + 273) 4 ] + f cl • h c • (t cl -T air )} (2.4) h c =          2.38 • |t cl -T air | 0.25 for 2.38 • |t cl -T air | 0.25 > 12.1 • √ v air 12.1 • √ v air for 2.38 • |t cl -T air | 0.25 < 12.1 • √ v air (2.5) f cl =          1.00 + 1.290I cl for I cl 0.078 m 2 • K/W
The PMV predicts the mean value of the thermal sensation of a population exposed in the same environment. As people have different sensation of comfort, individual votes are scattered around this mean value, so it is more useful to consider whether or not people will be satisfied. The Predicted Percentage of Dissatisfied (PPD) is an index that establishes a quantitative prediction of the percentage of thermally dissatisfied people who feel too cool or too warm, as expressed below:

PPD = 100 -95 • exp(-0.03353 • PMV 4 -0.2179 • PMV 2 ) (2.7)
The relationship between PPD and PMV is shown in Figure 2.2.

2.3.2/ ADAPTIVE THERMAL COMFORT MODEL

Unlike the PMV model, which is developed by using the principle of heat balance and experimental data collected in a controlled climate chamber under steady state conditions, the adaptive thermal comfort model is derived from hundreds of field studies with the idea that occupants can adapt to their thermal environment in free-running buildings (i.e. not heated or cooled) to some extend through dynamically interactions. The adaptations have been summarized by [De Dear, 2004] into three categories: behavior adjustments, physiological adaptations, and psychological adjustments.

Behavioural adjustments imply occupants can control their thermal environment by means of clothing, operable windows, fans, sun shades, etc. [De Dear et al., 1998]. In short, if a change occurs such as to produce discomfort, people react in ways which tend to restore their comfort [START_REF] Nicol | Adaptive thermal comfort and sustainable thermal standards for buildings[END_REF].

Physiological adaptations refer to body mechanisms to survive in drastic temperature environment. For example, in a cold environment the body utilizes vasoconstriction, while in a warm environment the body utilizes vasodilatation.

Psychological adjustments mean reactions to sensory information due to past experience and expectations. Personal comfort setpoints are not constant, and habituation and expectation can alter thermal perceptions [De Dear, 2004].

38CHAPTER 2. THERMAL COMFORT AND ENERGY EFFICIENCY IN SMART HOMES

In order to achieve thermal comfort, occupants will have these adaptive approaches to adapt to their natural environment. Therefore, compared with the space with heating or cooling systems, people in free-running buildings can endure a broader range of temperature variation [START_REF] Yao | A theoretical adaptive model of thermal comfort-adaptive predicted mean vote (apmv)[END_REF]. The mechanism of the thermal comfort adaptive model is shown in Figure 2 [START_REF] Yao | A theoretical adaptive model of thermal comfort-adaptive predicted mean vote (apmv)[END_REF] Because occupants' behavioural, physiological and psychological reactions are exceedingly complicated, the "Black Box" method, which is widely used in cybernetics [Raven, 1978], is employed to estimate occupants' thermal sensation. The word "Black" here means the full operational details remain unknown. This method tries to build the logical and statistical relationships between information input and output of the box. Based on it, the complex system can be described and understood, and the output can be predicted from the view of the general reactions of the system.

Applying the "Black Box" theory in thermal comfort, the PMV model can be expressed as shown in Figure 2.4 and the PMV can be estimated by Equation 2.8.

PMV = G × δ (2.8)
where δ is the combination of the six parameters, and G is a estimation function. Based on the adaptive model, the PMV model and the "Black Box" theory, the authors of [START_REF] Yao | A theoretical adaptive model of thermal comfort-adaptive predicted mean vote (apmv)[END_REF] have proposed the Adaptive Predicted Mean Vote (aPMV) model to index thermal comfort in free-running buildings, considering that psychological adaptive self-regulation plays a significant role in determining occupants' thermal sensations [De Dear et al., 1998]. In this model, psychological and behavioural adaptations act as a negative adaptive feedback, denoted by K δ , while physiological adaptation is contained within the "Black Box" G, as can be seen in Figure 2.5.

G δ PMV

G

aPMV = G × (δ -aPMV × K δ ) (2.9) aPMV = G × δ 1 + G × K δ (2.10)
where K δ is the influence of occupants' adaptations.

Substituting Equation 2.8 into Equation 2.10, then 2.11 can be written as:

aPMV = PMV/(1 + PMV × K δ δ ) (2.11) Assuming λ = K δ δ , Equation
aPMV = PMV 1 + λ × PMV (2.12)
where λ is called "adaptive coefficient".

The advantage of the aPMV model is it not only considers the six parameters that the PMV model takes into account, but also occupants' behavioural and psychological adaptations which are corresponding to behaviour habit, lifestyle, local climate, culture, social background, etc.. Therefore, it is a more accurate index model. It can be seen that when λ is equal to zero, the aPMV is equal to PMV representing no adaptive action. However, on one hand the adaptive coefficient λ is difficult to estimate in reality, and on the other hand the PMV model can be applied to buildings with heating and cooling systems, while the adaptive model can be generally applied only to free-running buildings where no mechanical systems have been installed [ASHRAE, 2004]. Hence, the PMV model has higher practicality in both research fields and industrial applications.

2.3.3/ SIMPLIFIED COMFORT MODEL

To minimize the complexity of the control problem, it can depend on simplified but descriptive comfort indices with a few variables as inputs. One example is the thermal comfort index called "effective temperature", which is developed by ASHRAE to express the combined effect on comfort of air temperature, humidity, and air speed [Givoni, 1998]. Its extended version is named Standard Effective Temperature (SET), which normalizes hu- , 2004]. In real applications, SET has been widely used to index thermal comfort or discomfort [START_REF] Ye | A new approach for measuring predicted mean vote (pmv) and standard effective temperature (set)[END_REF].

Another example of simplified comfort model is defined as upper and lower bounds on detected indoor temperature, and this comfort index is used in [Ma et al., 2011, Nagai, 2002, Oldewurtel et al., 2010, Braun, 1990, Kelman et al., 2013]. ASHRAE has recommendations for operative temperature and acceptable range that can be regarded as upper and lower comfort bounds in summer and winter separately, as shown in Table 2.3. These conditions are assumed for a relative humidity of 50%, air speed < 0.15m/s, mean radiant temperature equal to air temperature and a metabolic rate of 1.2met. Clothing insulation is defined as 0.9clo in winter and 0.5clo in summer.

As can be seen, the simplified comfort models can be implemented easily, but they can not index thermal comfort precisely and dynamically with the change of personal states. Moreover, due to their impreciseness, energy dissipation may occur via overheating. Based on the analysis of these three types of comfort model, PMV outperforms the others and thus is selected as the thermal comfort model in our work.

2.4/ DEMAND RESPONSE

Although occupants continuously seek to improve their thermal comfort, they have to consider the economic factor. Moreover, as indicated in the preceding sections, people will accept or even adapt to an uncomfortable environment if the level of uncomfortableness is limited to acceptable scope. Therefore, a reasoned balance between comfort and cost should be taken into consideration. Actually a smart home contains a variety 42CHAPTER 2. THERMAL COMFORT AND ENERGY EFFICIENCY IN SMART HOMES of electric appliances providing various services for occupants not only space heating devices. These electric appliances can be divided into three categories: fixed device, power-shiftable device and time-shiftable device. By reasonably shifting working power and working time of the latter two types of devices, energy efficiency can be improved, which is beneficial to both customers and producers of electricity.

In order to motivate occupants change their consumption patterns to coordinate with generating features of electricity producers, a program named demand response, that changes the price of electricity over time, is established. Demand response ensures efficient energy consumption and production by decreasing the level of energy use during peak demand or high electricity price periods. This in turn guarantees grid reliability as well as protects the profit of consumers [START_REF] Barbose | A survey of utility experience with real time pricing[END_REF], Saele et al., 2011].

Thanks to the emergence of smart grid technology with the ability of bidirectional information and energy communication, Advanced Metering Infrastructure (AMI) installed in smart home, and dedicated embedded control systems [Hart, 2008, Giacomoni et al., 2011], demand response is playing the role as an efficient energy management strategy and has attracted much attention from research fields. Generally, there are different options of price-based demand response made by policymakers, and the most commonly implemented options are described as follows.

• Time-Of-Use (TOU), whereby electricity rates often vary by time of day, e.g. peak vs. off-peak period, and are typically pre-defined for a period of several months or years.

• Real-Time Pricing (RTP), whereby electricity prices fluctuate hourly reflecting the costs of generating and/or consuming power at the wholesale level, and are typically known to customers on a day-ahead or hour-ahead basis.

• Critical Peak Pricing (CPP), whereby TOU is in effect except for certain peak days when CPP events are triggered by system contingencies. [START_REF] Conejo | Real-time demand response model. Smart Grid[END_REF] for individual consumers to adjust their decisions in response to time varying electricity prices. To manage the data uncertainty that may introduce considerable distortion on the solution, a robust optimization approach is employed. [START_REF] Kumaraguruparan | Residential task scheduling under dynamic pricing using the multiple knapsack method[END_REF] map the electricity bill minimization problem under dynamic day-ahead price environment to the well known multiple knapsack problem which enables cheap and efficient solutions to the scheduling problem. In the work of [START_REF] Chen | Real-time price-based demand response management for residential appliances via stochastic optimization and robust optimization[END_REF], a real-time pricebased demand response management model for residential appliances is proposed. The applications of this management model can be embedded into smart meters, employing scenario-based stochastic optimization and robust optimization approaches to determine the optimal operation considering future electricity price uncertainties. The authors of [START_REF]Uncertainty-aware household appliance scheduling considering dynamic electricity pricing in smart home. Smart Grid[END_REF] propose a stochastic energy consumption scheduling algorithm, which takes the time-varying pricing information, distributed renewable generation, and the customer-defined target trip rate as inputs, and generates an operation schedule to minimize the monetary expense. A decision-support tool based on particle swarm optimization is presented in [START_REF] Pedrasa | Coordinated scheduling of residential distributed energy resources to optimize smart home energy services. Smart Grid[END_REF] to help residential consumers optimize their acquisition of electrical energy services. With this approach, the minimization of monetary expense and the maximization of user comfort are balanced based on userdefined monetary benefits for electric services.

However, all of them only think about the scheduling of electric devices in a single smart home case, that is to say the smart home with these approaches merely considers to optimize its profit by reducing the usage of flexible appliances during peak-load hours or deferring them to low price periods. In general, it ignores the states of its neighbours.

As indicated in [START_REF] Li | Automated residential demand response: Algorithmic implications of pricing models. Smart Grid[END_REF], it shows that when the load management of a set of homes work selfishly, new peak-loads will occur during lower electricity price periods and it is called peak rebounds [START_REF] Safdarian | A distributed algorithm for managing residential demand response in smart grids[END_REF] which may also damage the power grid, cause unforeseen disasters, and reduce the global profit. This accentuates the necessity of an effective global demand response management scheme to ensure the coordination among them. on the main grid to make decisions themselves to maximize their occupants' profits.

In a smart home, there are various types of electric appliances. Some of them are controllable, so when they plug in, the smart home should recognize them and take control of them; while some are not, but the smart home should also know some information about them, such as their working power. For the controllable devices, their working status need to be optimized to provide proper services for occupants and guarantee their comfort. All the smart homes in the district need to communicate with the grid to exchange information, like time-varying electricity prices. Based on the information, operation and scheduling of controllable devices can be balanced between increasing quality of service and reducing the household cost of electricity usage. Moreover, these smart homes need to cooperate to flatten the variance of total load on the main grid to improve the energy efficiency of generating units.

2.7/ CONCLUSION

This chapter has presented the state of the art in terms of context and problem that our work involves. For thermal comfort, it is mainly influenced by six parameters: air temperature, mean radiant temperature, humidity, air velocity, metabolic rate, and clothing insulation, and the PMV thermal comfort model has turned out to be better than the other models to evaluate it. When the problem is extended that contains multiple smart homes with each equipped a variety of electric devices, the non-coordinated single home scheduling based on demand response may cause peak rebounds. Hence, it should be considered as a multiple home scheduling problem which need a coordinated approach to solve. In the next chapter, solutions for thermal comfort control and multiple home scheduling will be discussed.

THERMAL COMFORT CONTROL AND MULTIPLE HOME SCHEDULING

3.1/ INTRODUCTION

Up to now, in order to improve thermal comfort, there are already various methods. We divide them into two categories: model-based and model-free. Model-free methods can be further classified into local control methods and optimal control methods. Through analysing their advantages and disadvantages, we propose to employ a model-free hybrid control method for thermal comfort improvement. Towards solving the multiple home scheduling problem, based on the analysis of the shortages of some existing methods as well as the goals of grid utility and each smart home, we identify it as a multi-objective optimization problem, which can be tackled by searching from the Pareto Front or optimizing the aggregation of multiple objectives with different weights. Furthermore, taking into account the characteristics of the system: distributed, heterogeneous, intelligent and autonomous, an organization centered multi-agent system paradigm is proposed to model

this complex system.
This chapter is organized as follows. Section 3.2 presents and compares control methods for thermal comfort improvement. Section 3.3 introduces existing methods for the multiple home scheduling problem. Section 3.4 introduces multi-objective optimization. Agent and multi-agent system are introduced in Section 3.5. Section 3.6 presents organization centered multi-agent system described by Role-Interaction-Organization meta-model.

3.2/ THERMAL CONTROL METHODS

In order to regulate temperature to obtain a high level of thermal comfort, different thermal control methods can be used. In this section, a number of popular thermal control methods are divided into two categories: namely model-based control and model-free control, which will be introduced respectively.

3.2.1/ MODEL-BASED CONTROL

Model-based control as its name indicate is a type of control methodology that needs a model to describe properties of the control system in advance. The model used can be a mathematical model, often expressed with differential equations, that is built based on physical and mathematical knowledges, or a black/grey box model which is driven by data. The most famous model-based control method that is employed in building system is Model Predictive Control (MPC), which can handle large-scale Multi-Input Multi-Output (MIMO) dynamically coupled systems subject to state and input constraints with good performance. It was firstly proposed by [Braun, 1990] to reduce energy costs of buildings.

Inspired by this work, a number of methods based on MPC are developed for building heating and cooling control [START_REF] Freire | Predictive controllers for thermal comfort optimization and energy savings[END_REF], Castilla et al., 2011, Castilla et al., 2014, Moros ¸an et al., 2011, Chen et al., 2013a].

The principle of MPC is to solve a finite time optimal control problem at each time step t, expressed as follows [Ma et al., 2012]:

min u k=N-1 k=0 J(x t+k|t , u t+k|t , d t+k|t ), (3.1
)

subject to x t+k+1|t = f (x t+k|t , u t+k|t , d t+k|t ), ∀k = 0, 1, ..., N -1 (3.2)
y t+k|t = g(x t+k|t , u t+k|t , d t+k|t ), ∀k = 0, 1, ..., N (3.3)
u t+k|t ∈ U, ∀k = 0, 1, ..., N -1 (3.4)
y t+k|t ∈ Y, ∀k = 0, 1, ..., N (3.5)
where the symbol v t+k|t is interpreted as "the variable v at time t + k predicted at time t", N is the prediction horizon, J is the cost function corresponding to the problem to solve, x is the system states which include air temperature, relative humidity, air speed, etc.

in thermal regulation of buildings, u is the control actions like a magnitude of heating or cooling power, d is disturbances of the system, f (x, u, d) is a state transfer function which allows to predict the future building states based on the present states, control actions, and disturbances, y is the outputs of the system that is calculated by the function g(x, u, d)

defined by MPC designers, U is the action space, and Y is the output space. The objective of MPC is to minimize the cost function based on the model obtained in advance while satisfying the constraints.

Although MPC avoids the drawbacks brought from employing traditional control techniques such as excessive parameter tuning, the lack of prediction capability, etc., it also has to face up some challenges, for example, MPC requires analytical building models at the designing stage which is difficult to obtain on one hand. On the other hand, buildings have different structures, components and heating and cooling devices. So in order to use MPC, different models have to be designed for each building, which will increase the use cost. Moreover, due to the model designing process, the development cycle for MPC is longer than other control methods. Last but not the least, MPC requires more powerful computation units to solve an optimization problem in real-time, which is scarcely possible for residential buildings. For these reasons, the acceptance of MPC in building 50CHAPTER 3. THERMAL COMFORT CONTROL AND MULTIPLE HOME SCHEDULING community has been limited to research purpose.

3.2.2/ MODEL-FREE CONTROL

The other type of control methodology is model-free control which does not need to build a model of the system. The controller can take actions based on the present states or conditions for building heating and cooling systems. We develop the classification by [START_REF] Wang | Supervisory and optimal control of building hvac systems: a review[END_REF], and obtain a new classification schematic for model-free control, as presented in Figure 3.1. In this figure, model-free control can be further subdivided into local control, optimal control and hybrid control.

Model-Free Control Local Control Hybrid Control

Pure Learning Approaches On/Off Expert Systems PID Optimal Control Local control is the basic control and automation that has been widely adopted in buildings' heating and cooling systems due to its simplicity to design and its low computational complexity to determine the control signals [START_REF] Wang | Supervisory and optimal control of building hvac systems: a review[END_REF]. The control target of this type of method is its control variable itself, i.e. the indoor temperature is both control target and variable for corresponding systems. The two typical methods in this category are On/Off control (or bang-bang control) and Proportional-Integral-Derivative (PID) control.

On/Off control, which is a feedback control switching abruptly between two states: on or off, are ubiquitously used in old building systems without digital con-trol ability [START_REF] Simoes | Improving energy efficiency of cyber physical systems using multi-agent based control[END_REF], Lim et al., 2006, Dounis et al., 2009, Burns et al., 1991, Chinnakani et al., 2011]. Although it has been adopted by a variety of industrial products because of its simplicity, fewer and fewer attention is drawn from recent research domain.

Due to its working mode, the indoor temperature can not be kept stable, which vibrates heavily and forms a bandwidth bringing unsatisfactory thermal comfort.

PID control is a feedback loop control mechanism extensively implemented in current building heating and cooling systems [START_REF] Palme | Dynamic sensation of comfort in buildings: the temperature changes effects[END_REF], Cline et al., 2004, Kulkarni et al., 2004]. It calculates an error value as the difference between a measured process variable and a desired setpoint and through computations of the three modules It has been found that the indoor temperature can be maintained well with this control method as proved by its widespread use in reality. Moreover, the work of [START_REF] Paris | Heating control schemes for energy management in buildings[END_REF] has shown that PID is already good enough for indoor temperature regulation comparing with other methods. However, thermal comfort does not only relate to the environmental conditions like room temperature, but also factors about people themselves. For example, a temperature that is comfortable for a man sitting may be uncomfortable for a man doing exercises; when people wear more clothes, turning down the 52CHAPTER 3. THERMAL COMFORT CONTROL AND MULTIPLE HOME SCHEDULING temperature may be a proper way to obtain good thermal comfort. With the development of society, these local control methods can not satisfy people's increasing eagerness about indoor thermal comfort. This tendency encourages the research on optimal control methods in smart homes.

Optimal control takes into account the ever-changing environmental and personal conditions as well as the characteristics of heating and cooling systems to provide satisfied indoor thermal comfort while minimizing the energy consumption. Optimal control allows an overall consideration of the system characteristics not only one local variable like the indoor temperature, and thus can provide services with a higher quality.

Pure learning approaches and expert systems are two popular implementations of the optimal control. By using pure learning approaches, the system attempts to improve its behaviour based on the results of previous actions. After some learning time, these approaches can find the optimal or near-optimal solutions for the control problem without any prior knowledge of the environment.

For example, in the work of [START_REF] Dalamagkidis | Reinforcement learning for energy conservation and comfort in buildings[END_REF], the authors developed a reinforcement learning control system to achieve thermal comfort in an individual smart home with minimal energy consumption, and the system adopts Q-learning algorithm for learning, which is described in Algorithm 1. The drawback of this kind of method is that the performance is sensitive to many factors, such as the selection of the state-action, learning parameters, etc.. These features make it almost impossible to implement in practice [START_REF] Henze | Evaluation of reinforcement learning control for thermal energy storage systems[END_REF], Liu et al., 2006].

Algorithm 1 Q-learning algorithm with -greedy policy

Input discount factor γ, exploration , and learning rate α

Initialize Q-function Q 0 ← 0 Measure initial state x 0 for every time step k = 0, 1, 2, ... do u k ← u ∈ arg max ū Q k (x k , ū) with probability 1 - randomly select u with probability apply u k , measure next state x k+1 and reward r k+1 Q k+1 (x k , u k ) ← Q k (x k , u k ) + α[r k+1 + γ max u Q k (x k+1 , u ) -Q k (x k , u k )] end for
Expert systems can determine the optimal control actions according to the given working conditions. The knowledge base in an expert system is derived from the specific knowledge of one or more human experts. The advantage of this kind of method is that it is easy to design, implement and manage. The work employing expert systems for thermal comfort control in smart homes can be found in [START_REF] Dounis | Design of a fuzzy system for living space thermal-comfort regulation[END_REF], Hamdi et al., 1998, Gouda et al., 2001, Kristl et al., 2008, Bruant et al., 2001].

The most popular expert system is Fuzzy Logic Control (FLC). In general, a FLC contains four parts: fuzzifier, rules, inference engine and defuzzifier (see Figure 3.3). Firstly, a crisp set of input data is gathered and converted to a fuzzy set using fuzzy linguistic variables, fuzzy linguistic terms and membership functions. This step is known as fuzzification.

Afterwards, an inference is made based on a set of rules. Lastly, the resulting fuzzy output is mapped to a crisp output using the membership functions in the defuzzification step. However, applications of expert system work well only if the knowledge base from experts is sound, if not, significant errors may occur. combines the advantages of the two former types of method together. The proposed hybrid control method takes into account the ever-changing environmental and personal parameters, and by inversely solving the PMV model is able to obtain an optimal temperature setpoint for a PID controller, which is good at regulating indoor temperature. The detailed design and implementation will be presented in the next chapter. The experimental simulations have proved that the proposed hybrid control method performs better 54CHAPTER 3. THERMAL COMFORT CONTROL AND MULTIPLE HOME SCHEDULING than the other methods presented before.

Fuzzifier

3.3/ EXISTING METHODS FOR MULTIPLE HOME SCHEDULING

Depending on an advanced control method, a high level of thermal comfort can be obtained. But a new question is asked then: does this mean the perfect solution? The answer is negative. As presented in the previous chapter, on one hand, there are various electric devices in a smart home fulfilling occupants' different requirements besides thermal comfort. On the other hand, reducing electricity bill and avoiding peak rebounds also need to be considered. Thus, an effective global demand response management scheme to ensure the coordination among them is necessary. It involves different electric devices in a smart home and a set of neighbouring smart homes in a district. The problem therefore belongs to multiple home scheduling.

Many recent studies have been focused on this multiple home scheduling problem. In the work of [START_REF] Kishore | Control mechanisms for residential electricity demand in smartgrids[END_REF], a distributed scheduling mechanism is proposed to reduce peak demand within a neighbourhood of homes. The mechanism provides homes a guaranteed base level of power and allows them to compete for additional power to meet their needs. The authors of [Mohsenian-Rad et al., 2010b] propose to use game theory and formulate the problem as an energy consumption scheduling game, where the players are the users and their strategies are the daily schedules of their household appliances and loads. In the case of a single utility company serving multiple customers, the global optimal performance in terms of minimizing the energy costs is achieved at the Nash equilibrium of the formulated energy consumption scheduling game. In the work of [START_REF] Safdarian | A distributed algorithm for managing residential demand response in smart grids[END_REF], a system-wide demand response management model to coordinate demand response provided by residential customers is presented to flatten the total load profile subject to minimum individual cost. The problem is formulated as a mixed integer linear programming, and an iterative distributed algorithm is used to solve it. Other work that considers to maximize various users' profits while avoiding severe peak-load rebounds can be found in [START_REF] Kahrobaee | A multiagent modeling and investigation of smart homes with power generation, storage, and trading features. Smart Grid[END_REF], Atzeni et al., 2013, Gatsis et al., 2012].

Although these reviewed articles have taken valuable steps towards realizing effective demand response for multiple homes, however there are some limitations: (1) in some of them, utility cost minimization is considered as the only objective function which is contradicted to habitants' objective -minimizing individual electric cost; (2) In almost all of them, they only focused on energy minimization and thus user comfort has not been considered sufficiently. For example, deferring the operation of washing machine until the midnight should be avoided, for the noise will disturb the sleeping residents.

In order to take both user's profit and utility's profit into account, the dissatisfaction caused by using electric appliances, the related cost of electricity usage in each smart home, and the variance of total energy consumption on the main grid should all be considered. Therefore, the problem involved is a multi-objective optimization problem.

3.4/ MULTI-OBJECTIVE OPTIMIZATION

The general global optimization problem can be defined as [START_REF] Coello | Handling multiple objectives with particle swarm optimization[END_REF]:

Definition 1: Given a function f : Ω ⊆ R n → R, Ω ∅, for x ∈ Ω, the value f * f ( x * ) > -∞
is called a global minimum if and only if

∀ x ∈ Ω : f ( x * ) ≤ f ( x) (3.6)
where x * is the global minimum solution, f is the objective function, and the set Ω is the feasible region.

When the problem contains multiple objectives, it becomes a multi-objective optimization problem:

Definition 2: Find the vector x * = [x * 1 , x * 2 , ..., x * 3 ]
T which will satisfy the m inequality constraints 3.8) and will optimize the vector function

g i ( x) ≥ 0 i = 1, 2, ..., m (3.7) the p equality constraints h i ( x) = 0 i = 1, 2, ..., p (
f ( x) = [ f 1 ( x), f 2 ( x), ..., f k ( x)] T (3.9) where x = [x 1 , x 2 , ...,
x n ] T is the vector of decision variables.

56CHAPTER 3. THERMAL COMFORT CONTROL AND MULTIPLE HOME SCHEDULING

For multi-objective problems, there are mainly two approaches to be used: the first is to search for the Pareto Optimal Set and then obtain the Pareto Front, according to which the most appropriate solution can be chosen; the second is to distribute a proper weight factor to each objective function and then aggregate them together to form a single composite objective function.

The first approach is based on Pareto Optimality. Formally a Pareto Optimality is defined as [START_REF] Coello | Handling multiple objectives with particle swarm optimization[END_REF]:

Definition 3: A point x * ∈ Ω is Pareto optimal if ∀ x ∈ Ω and I = {1, 2, ..., k} either ∀i ∈ I, f i ( x) = f i ( x * ) (3.10) or ∃i ∈ I, f i ( x) > f i ( x * ) (3.11)
This definition tells that x * is Pareto optimal if there is no feasible vector x to decrease some criteria without causing increasing at least one other criterion simultaneously. Other definitions associated with Pareto Optimality are Pareto dominance, Pareto optimal set, and Pareto front:

Definition 4: A vector u = (u 1 , u 2 , ..., u k ) is said to dominate v = (v 1 , v 2 , ..., v k ) (denoted as u v) if and only if u is partially less than v, i.e., ∀i ∈ {1, 2, ..., k}, u i ≤ v i ∧ ∃i ∈ {1, 2, ..., k}, u i < v i
Definition 5: For a given multi-objective problem f (x), the Pareto optimal set P * is defined as

P * := {x ∈ Ω|¬∃x ∈ Ω, f (x ) f (x)} (3.12)
Definition 6: For a given multi-objective problem f (x) and Pareto optimal set P * , the Pareto front PF * is defined as The second approach is based on the aggregation of the k multiple objective functions

PF * := { u = f = ( f 1 (x), f 2 (x), ... f k (x))|x ∈ P * } (3.13)
f = ( f 1 (x), f 2 (x), ... f k (x))
to generate a single composite objective function:

F(x) = k i (ω i • f i (x)) (3.14)
where ω i is the weight factor for each objective function. Choosing the weight factors is the most problematic for this method, because objectives always express different criteria and thus have different quantities. Except this, the aggregation approach is easier and simpler than Pareto optimality approach to implement, for it can be optimized by a singleobjective optimization algorithm, while Pareto optimality approach has to firstly obtain the Pareto optimal set by an algorithm and then select the most appropriate solution from it by another algorithm. Therefore, the aggregation method is adopted in this dissertation.

Based on this method, the three objectives of a smart home: to minimize the dissatisfaction, the electric cost, and the total power variance on the main grid can be aggregated into a single objective function. The first two sub-objectives can be realized within a smart home independently, while the realization of the last sub-objective requires the cooperation among neighbouring smart homes, that connect to the smart grid forming a smart micro-grid system. In addition, considering the characteristics of this smart microgrid system: distributed, heterogeneous, intelligent and autonomous as presented in the previous chapter, a Multi-Agent System (MAS) paradigm is proposed to be employed to model this complex system.

3.5/ AGENT AND MULTI-AGENT SYSTEM

This section concentrates on the introduction of agent and multi-agent system, in which general concepts, agent management reference model, agent communication language, and agent life cycle will be presented respectively.

3.5.1/ GENERAL CONCEPTS

The fundamental element of a multi-agent system is an agent. As Wooldridge [Wooldridge, 1999] defined, "an agent is a piece of software of a hardware entity that is situated in an environment and is able to autonomously react to changes in the environment." The environment is simply everything external to the agent. As can be seen in Figure 3.5, an intelligent agent can observe an environment through sensors, act upon the environment using actuators and direct its activities towards achieving goals. Moreover, an intelligent agent also has characteristics such as proactivity, reactivity, and social ability.

• Proactivity means agents are able to exhibit goal-oriented behaviours, by producing a plan. The goals are divided into local and global ones. An individual agent usually has local goals while a group of agents may have global goals.

• Reactivity indicates agents can perceive the changes of the environment, and they are expected to respond in order to satisfy their goals.

• Social ability allows agents to perform complex decision-making procedures to achieve global goals through cooperation, coordination, and negotiation.

Agent

Perception Action Environment In most real world applications and systems, such as e-commerce trading, network and resource management, telecommunications, supply chain management, production control, traffic monitoring, aircraft maintenance, manufacturing systems, decision support systems, power systems, etc., it is not sufficient to have only one single intelligent agent to achieve the system objective. At this moment, Multi-Agent Systems (MAS) can be used to model a large-scale complex system as a group of geographically distributed, autonomous and adaptive intelligent agents, and through the interactions of cooperation, coordination, and negotiation among these agents the global objective can be realized. The following definition formally explains what a MAS is: a MAS is a system composed of a collection of interacting entities called agents, evolving in an environment where they can autonomously perceive and act to satisfy their needs and objectives. [Wooldridge, 1999] The benefits of applying MAS to the smart home domain are mainly twofold: (1) MAS plays an important role in modeling the management of smart homes; (2) MAS is a suitable approach to build this complex system.

In terms of modeling, MAS is based on the object-oriented approach [Wooldridge, 2009],

where entities in a system are represented as objects. Therefore, it maintains the bene-60CHAPTER 3. THERMAL COMFORT CONTROL AND MULTIPLE HOME SCHEDULING fits of data encapsulation and methods protection. In specific, the data structure allows for associating object attributes being accessible through the object methods. The object methods can be called only when their properties are public. And if the method is called, it has no option but to execute. However, the agent paradigm augments this approach by realizing a higher layer of abstraction, where direct access to the methods is not possible. It has to request another agent to execute a certain method by communicating via message passing. Even if the agent has received the request, it retains autonomy to decide and schedule its own actions. Hence, deploying the MAS as a modeling approach can guarantee a certain degree of autonomy for different actors or agents in simulating complex systems. In this dissertation, different smart homes contain various types of controllable devices that can both be modelled as intelligent agents, and they have autonomy to operate without a central manipulation. That is to say, in order to balance the total loads on grid, the smart grid agent cannot manipulate the actions of those agents directly, but has to indirectly negotiate through sending requests.

In terms of building complex systems, MAS is regarded as a platform for distributed application based on the same notion of agent's autonomy. Due to the reason that various smart homes are naturally geographically distributed, MAS provides a suitable choice for this application. Furthermore, MAS enables building flexible, extensible and robust systems [Mihailescu, 2013]. Flexibility can be explained at the agent level and at the system level. At the agent level, heterogeneous agents have various abilities to achieve their individual objectives, while at the system level through interactions of cooperation, coordination, and negotiation among individual agents the global objective can be realized.

Extensibility refers to the openness of the MAS architecture, where agents are able to be added or deleted in the system and agents can obtain new functionalities. These are attributed to the modularity based on object-oriented programming and agent interactions via communication. Robustness means a certain level of fault is tolerable, and this is especially important for a distributed system.

3.5.2/ AGENT MANAGEMENT REFERENCE MODEL

In order to employ a MAS, first of all, how a MAS functions need to be known. Agent Management Reference Model is defined by the Foundation for Intelligent Physical Agents (FIPA), and according to [FIPA, 2004] [FIPA, 2004] This model consists six components:

1 An Agent Platform (AP) is the hardware and software entirety that needs to operate MAS, including computers, operating system, agent support software, FIPA agent management components, and agents.

2 An agent is a specific operational entity that has an objective to fulfil or provides some services. It is contained in an AP, and must have an identity called Agent Identifier (AID) to be differentiated from the other agents in the same AP. These agents communicate with each other using an Agent Communication Language (ACL) to cooperate, coordinate, and negotiate.

3 An Agent Management System (AMS) is the core component in an AP. Each

AP can only has one single AMS, which maintains a directory of AIDs recording transport addresses for agents registered with the AP, so it is a reification of the agent directory service. 6 Software describes all non-agent, executable collections of instructions that an agent may access, as if a person can use different tools to support his work.

3.5.3/ AGENT COMMUNICATION LANGUAGE

In order to facilitate communications among agents to make them cooperate and coordinate in a MAS, a standard language is needed. To this end, FIPA has proposed the Agent Communication Language (ACL) [FIPA, 2002a]. A FIPA ACL message should contain a set of message parameters, of which performative defines the type of communicative act of the message such as request, inform, propose, refuse, etc., participant in communication denotes which agents take part in the conversation, content of message includes the exact information of the message, description of content defines the language, encoding and ontology of the content expression, and control of conversation defines extra specific operations of the message. All message parameters can be seen in Table 3.1.

(request :sender (agent-identifier : name agent A) :receiver (set (agent-identifier : name agent B)) :content "What is your favorite sport?" :language fipa-sl) 3.7 shows an ACL message example. This message is sent from agent A to agent B, to ask the favorite sport of agent B. The performative in this case is a Request, which denotes the communicative act of the message. The FIPA semantic language (FIPA-SL) is a content language specification with a specific syntax and semantic, which is implemented in several MAS platforms [FIPA, 2002b]. Table 3.1: FIPA ACL Message Parameters [FIPA, 2002a] When agents are created physically on an AP, they will utilize the facilities offered by the AP for realizing their functionalities. In this context, an agent, as a physical software process, has a physical life cycle that has to be managed by the AP. During its lifetime, an agent transits from the initiated state to the active state, then may switch among the three states: waiting, suspended, and transit, and at last is destroyed (Figure 3.8):

Parameters

• In initiated state, the agent executes a set of instructions to prepare its operations.

• Active state is the normal state in which the agent offers some services or operates to realize its objective.

64CHAPTER 3. THERMAL COMFORT CONTROL AND MULTIPLE HOME SCHEDULING

• When entering waiting state, for example waiting for a message from another agent, the agent is pooling for an external event to wake it up.

• When the agent is suspended, it is halted from the active state.

• Transit state means the agent is physically moving from one platform to another. Multi-Agent System (ACMAS) has undesired properties, such as unconstrained communication, unlimited access, and single framework [START_REF] Ferber | From agents to organizations: An organizational view of multi-agent systems[END_REF]. As consequences, these undesired properties can decrease the security of systems and have poor modularity. In addition, as Jennings has pointed out [Jennings, 2000]: ACMAS may suffer some drawbacks when engineering large systems, for example, the patterns and the outcomes of the interactions are inherently unpredictable.

Initiated

To overcome these problems, it can depend upon Organization Centered Multi-Agent System (OCMAS). Compared with ACMAS, OCMAS is described by specific meta-models. The Role-Interaction-Organization (RIO) meta-model [START_REF] Hilaire | Formal specification and prototyping of multi-agent systems[END_REF], Rodriguez, 2005], which is one of many meta-models proposed to describe an OCMAS, is employed to design our OCMAS thanks to its simplicity and conciseness. In the RIO meta-model, an organization defines a set of roles and their interactions with each other.

A role, that is an abstraction of a behaviour of an agent, is like a class in object-oriented programming defining the behaviours of an object. The formal definitions of organization, role and interaction are provided as follows [Rodriguez, 2005]:

(a) Organization: An organization is defined by a set of roles, their interactions and common context. They define a specific pattern of interaction.

(b) Role: A role is the abstraction of a behaviour in a certain context and confers a status within the organization. The role gives the playing entities the right to exercise its capacities. Roles may interact with other roles defined in the same organization.

(c) Interaction: An interaction links two roles in a way that an action in the first role produces a reaction in the second.

In addition to its formal specification, RIO provides a graphical representation of orga-

nizations. An example of the graphic representation of a RIO diagram is presented in The following chapters will present our contributions on these two aspects. For the first aspect, we propose a cyber-physical system that employs a hybrid intelligent control method to improve thermal comfort. For the second aspect, the complex system is modelled by an organization centered multi-agent system, the composite objective function of each smart home is mathematically defined, and a cooperative particle swarm optimiza-tion algorithm is developed to optimize schedules of controllable devices in each smart home. Simulations and experiments are conducted to verify the proposed systems and methods.

4.1/ INTRODUCTION

In this chapter, based on the development of advanced networking, computing and sensor technologies, a cyber-physical system for thermal comfort improvement is presented. By analysing the advantages and disadvantages of local control and optimal control methods for thermal comfort, a hybrid intelligent control method is proposed for this cyber-physical system. This method contains two modules: namely the inverse PMV optimization module and the PID control module. The former is used to find an optimal setpoint temperature for the PID control module. The latter is used to regulate the indoor temperature to reach this setpoint. In order to find a suitable meta-heuristic algorithm for the former module, three popular algorithms: particle swarm optimization, genetic algorithm and simulated annealing are compared, and the best one is selected by experiments and analysis.

This chapter is organized as follows. Section 4.2 presents the proposed cyber-physical system for thermal comfort management. Section 4.3 presents the sensors and network that are needed to implement this system. The proposed hybrid intelligent control method adopted by the system is presented in Section 4.4.

4.2/ CYBER-PHYSICAL SYSTEM

Cyber-Physical System (CPS) can be considered as a confluence of embedded systems, real-time systems, distributed sensor systems and controls [START_REF] Rajkumar | Cyberphysical systems: the next computing revolution[END_REF]. They In this figure, the information of physical world is captured by sensors and transmitted through wired or wireless networks to cyber world. In cyber world, different computing units of many physical forms, constitute an intelligent whole to deal with a variety of objectives. After computing, the cyber world can affect the physical world or provide some services by actuators. , can be perceived by different sensors. These data are transmitted to Cyber World through the network. In Cyber World, a hybrid intelligent control method, which takes the advantages of both local control and optimal control introduced in the state of the art, is proposed to regulate the indoor temperature to maintain thermal comfort. More details about designing this CPS will be presented in the following sections.

4.3/ SENSORS AND NETWORKS

Sensors

Sensors play an important role in perceiving information from the physical world. Only when accurate data is perceived by corresponding sensors, can the CPS functions well.

In the proposed CPS, many thermal comfort related parameters need to be perceived from the physical world. The environmental-dependent parameters can be easily detected by specific sensors that have been widely deployed in many large commercial and residential buildings [START_REF] Zhao | A conceptual scheme for cyber-physical systems based energy management in building structures[END_REF], except for mean radiant temperature. On one hand, the measurement of this parameter needs many special instruments installing in different points in the room, which is time and money consuming ; on the other hand, the final mean radiant temperature is not very far from air temperature [START_REF] Bermejo | Design and simulation of a thermal comfort adaptive system based on fuzzy logic and on-line learning[END_REF].

Hence, it can be approximated with the same value of the perceived air temperature.

In addition, the personal-dependent parameters: metabolic rate and clothing insulation can both be detected by special sensors or be estimated. By surveying literatures, for metabolic rate, it can be estimated by heart rate [Green, 2011] that is easier to be perceived, or be detected directly by sensors, such as Suzuken LifeCorder [START_REF] Ito | Real-time mets estimation for effective indoor amenity control in hems[END_REF]; for clothing insulation, it can be estimated by clothing weight: Clo = 0.000558 • Weight + 0.062 empirically [Matsumoto et al., 2011], by correlating to ambient temperature [START_REF] Han | Occupancy and indoor environment quality sensing for smart buildings[END_REF] or seasonal average value [START_REF] Lee | Pmv calculation and intelligent control algorithm for location-based human adaptive airconditioner[END_REF], or be detected directly by sensors [Gauthier, 2011]. The typical values of these parameters can be seen in Table 2.1 and Table 2.2 of the state of the art.

Networks

Communication technology is extremely important in the implementation of a CPS, because it determines the connection between computing units from the cyber world and sensors and actuators from the physical world. The four most popular protocol standards for short-range wireless communications with low power consumption, that are suitable for home scale applications, are Bluetooth [START_REF] Bluetooth | Specification of the bluetooth system, version 1.1[END_REF], ZigBee [Alliance, 2006], Ultra-Wide Band (UWB) [START_REF] Di | [END_REF], and Wi-Fi [Bianchi, 2000]. The comparisons of these four protocol standards are listed in Table 4.1.

4.4/ HYBRID INTELLIGENT CONTROL METHOD

In order to improve thermal comfort in this CPS, a hybrid intelligent control method is designed and implemented. This control method contains two modules: namely the inverse PMV optimization module and the PID control module, as shown in Figure 4.2. This section will present how to implement these two modules respectively.

4.4.1/ INVERSE PMV OPTIMIZATION MODULE

As presented in the state of the art, the thermal comfort in smart homes can be evaluated by the PMV model, which takes six parameters -four environmental-dependent and two personal-dependent as inputs and then calculates a PMV value as the output to index thermal comfort for present conditions. The PMV calculation can be simply expressed as:

PMV = F(T air , T MRT , v air , h r , I cl , M) (4.1)
where F is the PMV calculation function the definition of which is presented in Section 2.3.1, the six input parameters denote indoor air temperature, mean radiant temperature, air velocity, humidity, clothing insulation, and metabolic rate respectively, and PMV here denotes the PMV value calculated from F.

In this work, we propose to inversely use the PMV model to find the optimal indoor temperature. Assume that the six parameters have been estimated or perceived by sensors from the present environment, denoted by Tair , TMRT , vair , hr , Īcl , M. By using Equation 4.

f (x) = abs(F(x, T MRT , v air , h r , I cl , M)) (4.3)
This fitness function takes the absolute value of the PMV calculation result. By using meta-heuristic algorithms to search for the indoor temperature x that minimizes the fitness function f (x), the setpoint temperature T * air can be found. is originally proposed and developed by [START_REF] Kennedy | Particle swarm optimization[END_REF], Shi et al., 1998]. It is a meta-heuristic algorithm that has been turned out to be powerful to solve complex nonlinear and non-convex optimization problems [Lin et al., 2009]. Moreover, it has several other advantages, such as fewer parameters to adjust, and easier to escape from the local optimal solutions. In PSO, a population of candidate solutions, here dubbed particles that include position vector x and velocity vector v, is randomly generated around the search space initially.

After that they are iteratively updated to simulate the movement around the search space according to mathematical formulae over the particles' position and velocity, as expressed below:

v k+1 i = w • v k i f irst part + c 1 • rand() • (p best k i -x k i ) second part + c 2 • rand() • (g best k -x k i )
third part (4.4)

x k+1 i = x k i + v k+1 i (4.5)
where w is inertia weight, c 1 and c 2 are acceleration constants, rand() generates random value between 0 and 1, p best k i is the local best position, g best k is global best position, i is the particle index, and k is the iteration index. For Equation 4.4, the first part expresses particle's inertia of previous action, the second part reflects particle's cognition that stimulates the particle to minish errors, and the third part is called social part, which indicates the information sharing and cooperation among particles. The standard PSO is described in Algorithm 2. Return the best chromosome as the found solution.

Crossover Point

4.4.2.3/ SIMULATED ANNEALING

Simulated Annealing (SA) is an optimization technique inspired by the annealing process in metallurgy that involves heating and controlled cooling of a material to increase the size of its crystals and reduce the defects [START_REF] Kirkpatrick | Optimization by simulated annealing[END_REF]. When heating a piece of metal, the high temperature means the particles within the metal have high energy to move around intensely, changing the structure of the metal. During the cooling process, as the temperature is decreasing, movements of particles are limited and finally stop to be steady states.

dE = f (x i ) -f (x 0 ) if dE <= 0 then x 0 = x i else if exp( -dE T ) > rand(0, 1) then x 0 = x i end if end if

Decrease temperature T end for

Return x 0 as the found solution.

4.4.2.4/ EXPERIMENTS AND COMPARISONS

In order to select a meta-heuristic algorithm to efficiently search for the temperature setpoint T * air from the PMV calculation function F, experiments are conducted and the performances of the different algorithms are analyzed in this part. 4) The running time of each execution is recorded as an evaluation of algorithm's performance.

The searching performances of these algorithms are presented in Figure 4.5 -4.7. From these figures, it can be found that (1) generally the searching performances of PSO and GA with different population sizes are better than SA. Because PSO and GA are both population-based optimization methods and each particle of PSO or each chromosome of GA represents a potential solution for the problem hence increasing their solution searching ability, while SA is not so it needs more iterations for trial. One exception is the GA algorithm with a population of 5, which has a terrible performance. This is because the differentiation of offspring chromosomes is dependant upon their parent chromosomes, so if population size is not big enough, the searching ability of GA will be greatly limited.

But it is not the same case for PSO, since each particle of PSO is relatively independent with each other. ( 2) For PSO and GA, the algorithms with a population of 20 perform the best. This is because the searching ability of these two population-based optimization methods is dependant upon the size of its population: the bigger population the more powerful searching ability in each iteration. This record comes from the computing environment with an Intel(R) Xeon(R) CPU E31225 @ 3.10GHz processor, a 8.00 GB RAM, and a 64-bit operating system. In this figure, for PSO and GA it can generally conclude that the bigger population the more time consumption. This phenomenon is opposite to their searching performances in which more population is advantageous. The reason is that the fitness function in this experiment is expensive to compute and for population-based PSO and GA the evaluation of each individual needs to calculate the fitness function once. Therefore, the algorithms with less population can find the best solution faster instead, albeit they need more iterations. The longer time consuming result of the GA algorithm with a population of 5 chromosomes is caused by its terrible searching ability, which makes it exceed the maximum iteration number without finding the best solution.

The comparison of failed times among these algorithms is showed in Figure 4.9. It can be seen that on average for a total of 50 executions SA fails 5 times, GA with 5 chromosomes fails 12 times, the others always succeed.

By conducting this experiment, the comparison results on searching performance, time performance, and total failed number of times indicate that the PSO algorithm with 5 particles is an efficient method to calculate the temperature setpoint T * air from the PMV calculation function.

4.4.3/ PID CONTROL MODULE

As the other part of the hybrid method, the PID control module is used to regulate the heating power regularly. There are three reasons that we choose it: firstly due to the huge thermal mass of smart homes, the indoor temperature change is a rather slow process; secondly the work of [START_REF] Paris | Heating control schemes for energy management in buildings[END_REF] has proved that PID is already good enough to regulate the indoor temperature comparing with other methods; thirdly, PID can be implemented easily and only needs to tune three scalar parameters. The PID controller is formally described as follows:

Q p (t) = K p e(t) + K i e(t)dt + K d de(t) dt (4.6)
where Q p (t) is the calculated heating power at time t via PID calculation, and e(t) is the error between the setpoint temperature T * air obtained from the inverse PMV optimization module and the perceived present indoor temperature. K p , K i and K d are the proportional parameter, the integral parameter and the derivative parameter respectively, that can be set empirically or tuned easily by experiments. After the inverse PMV optimization module has found the setpoint temperature T * air using a meta-heuristic algorithm, this setpoint temperature is sent to the PID control module, as shown in Figure 4.10. Based on the setpoint temperature and the perceived indoor temperature T air (t) of the present environment, the PID control module can continuously compute heating power for space heating system in the physical environment. By using this feedback loop control strategy, the indoor temperature can be controlled to reach the setpoint temperature T * air to guarantee thermal comfort for occupants.

Because the calculation of the inverse PMV optimization module is time consuming and is much slower than the calculation of the PID control module on one hand, and on the other hand there is no need to compute the setpoint temperature frequently, hence the two modules work asynchronously. For example, the PID control module acquires the present indoor temperature and calculates heating power every 10 seconds, while the execution of the PMV optimization module is event driven, that is to say only when some apparent changes of the parameters such as metabolic rate or clothing insulation are detected, can make this module execute to compute a new setpoint temperature for the PID control module.

5.1/ INTRODUCTION

In this chapter, in order to test the performance of the proposed cyber-physical system with the hybrid intelligent control method, experimental simulations are conducted. Before the simulations, a building thermal model, based on the heat balance theory and expressed by an electric circuit network, is introduced. In the simulations, the proposed method is compared with two traditional control methods: PID control and On/Off control, and two intelligent control methods: fuzzy logic control and reinforcement learning control in two different scenarios.

This chapter is organized as follows. Section 5.2 introduces the building thermal model, and Section 5.3 gives the experimental simulation results and analysis.

5.2/ BUILDING THERMAL MODEL

Although the proposed hybrid intelligent control method is a model-free and on-line method, in order to simulate the thermal variation in a building to test the proposed cyberphysical system and the control method for thermal comfort improvement, a mathematical thermal model need to be implemented.

Generally the room temperature is affected not only by auxiliary heating/cooling systems, but also by other factors, such as the ambient temperature. According to [START_REF] Achterbosch | The development of a comvenient thermal dynamic building model[END_REF], the heat balance of a building can be expressed as

φ h (t) + φ s (t) = φ t (t) + φ c (t) (5.1)
where φ h is the heat supplied by all internal heat sources; φ s is the heat gained by solar radiation; φ t is the heat loss through external contact; φ c is the heat retained by the building.

In order to analyze the thermal dynamics of a smart home, we can consider it as a thermal network that contains different components affecting the variation of indoor temperature.

For the reason that in physics thermodynamics and electricity have the similar expressions, for example we have Ohm's law I current = ∆U electric potential R electric resistance in electricity and we have Q heat f low = ∆T temperature R thermal resistance in thermodynamics, therefore we are able to employ electrical method to express thermodynamical models.

According to this approach, the thermal network can be analogous to an electric circuit network by regarding heat flow as electric current, temperature as electric potential and treating thermal resistance (R), which is the reciprocal of thermal transmittance (U): R = 1/U, and thermal capacitance (C) as electric resistance and electric capacitance respectively, as shown in Figure 5.1. The symbols and the subscripts are explained in Table 5.1 and Table 5.2.

In the figure, there are two nodes, T amb and T air , which represent the outdoor and indoor air temperature, and six sub-circuits which indicate physical components of the room, including room air, internal partitions, ceiling, floor, external walls and windows. We assume that there is no thermal capacitance for windows and no thermal resistance for room air.

φ = A × U × (T 1 -T 2 ) (5.2)
where φ is the heat transfer in watts, A is the area in square metres, U is the thermal transmittance, T 1 is the temperature on one side of an object and T 2 is the temperature on the other side of the object.

dT f dt = A f C f pQ s A f + U f (T air -T f ) (5.5) dT c dt = A c C c U c (T air -T c ) (5.6) dT ip dt = A ip C ip (1 -p)Q s A ip + U ip (T air -T ip ) (5.7) dT air dt = 1 C air Q p + Q e + (A g U g + U v )(T amb -T air ) +A w U wi (T w -T air ) + A f U f (T f -T air ) +A c U c (T c -T air ) + A ip U ip (T ip -T air ) (5.8)
In above equations, Q p heat supplied by the heating system in W, Q e heat gained by using electrical equipments in W, Q s solar radiation through glazing in W, A area in m 2 , p fraction of solar radiation entering floor. Above equations can be stacked using the state-space notation:

ẋ = Ax + Bu (5.9)
where ẋ is a vector of derivatives of temperatures of external walls(T w ), floor(T f ), ceiling(T c ), internal partitions(T ip ) and air inside(T ai ): ẋ = [ Ṫw , Ṫ f , Ṫc , Ṫip , Ṫai ], A, B are matrices of coefficients, x is a vector of states and u is the input vector, including Q p , Q e , Q s and T ao . Although Q e is defined as the heat gained by using electric equipments, it also contains the heat generated by occupants through metabolism.

The metabolic rate has been introduced in the state of the art. According to metabolism, the heat production of all occupants is determined by occupant's number, average body area and average metabolic rate:

Q e p = N × 58 × A D × M,
where Q e p is the heat produced by occupants, N is occupant's number, A D is the average surface area of an occupant which can be estimated by A D = 0.203H 0.725 W 0.425 (H is height and W is weight) and for an average size adult it is about 1.8m 2 , and M is the average metabolic rate. Some of the metabolic rates of typical activities have been listed in Table 2 and Reinforcement Learning (RL) in two different scenarios.

A =                                 -A w1 C w1 [U wi1 + U wo1 ] 0 0 0 0 A w1 U wi1 C w1 0 -A w2 C w2 [U wi2 + U wo2 ] 0 0 0 A w2 U wi2 C w2 0 0 -A f U f C f 0 0 A f U f C f 0 0 0 -A c U c C c 0 A c U c C c 0 0 0 0 -A ip U ip C ip A ip U ip C ip A w1 U w1 C a A w2 U w2 C a A f U f C a A c U c C a A ip U ip C a -A g U g -U v C a -• • • - A ip Uip C a                                 B =                               0 0 0 A w1 U wo1 C w1 0 0 0 A w2 U wo2 C w2 0 0 p C f 0 0 0 0 0 0 0 (1-p) C ip 0 1 C a 1 C a 0 A g U g +U v C a                               T amb 1/Ug1 1/Uwo1

5.3.1/ EXPERIMENTAL PREPARATION

The two simulation scenarios are assumed to take place in Oslo which represents a cold winter (the ambient temperature varies from 6 In PSO part of HIC, inertia weight w = 0.72, acceleration constants c 1 = c 2 = 1.49, particle number is 5, and maximum iteration number is 150. In PID control part, proportional gain is 2000, integral gain is 0.5, and derivative gain is 0, which are set empirically and tested by experiments. The maximum power of the electric heating radiator is 2000W.

For traditional PID and On/Off controls, occupants need to manually tune the setpoint for the system. When they feel hot or cold, they may turn down or turn up the setpoint of the heating system for 1 • C or 2 • C respectively or even probably do nothing for the uncomfortableness is acceptable or they have no time to do it. Hence, we simulate these behaviours as: if the absolute value of PMV exceeding 0.5 lasts for more than 10 minutes, occupants will have a probability of 0.6 to increase or decrease the setpoint separately. If they decide to do it, they will change the setpoint for 1 • C with a probability of 0.8 or 2 • C with a probability of 0.2.

Besides the traditional control methods, the two intelligent control methods also need to be set before running. In this work, the FLC, which is configured with the same inference engine proposed in [START_REF] Dounis | Design of a fuzzy system for living space thermal-comfort regulation[END_REF], takes the present PMV value and ambient temperature as inputs, and a heating power can be obtained as output via this expert system.

The inference surface of this FLC can be seen in Figure 5.3. The RL method also takes PMV and ambient temperature as inputs, discretizes the heating power range [0,2000] into 25 actions. The parameters are chosen as: exploration rate = 0.1, learning rate α = 0.1,and discount factor γ = 0.6.

-10 0 10 20 30 The simulations of occupant number, metabolic rate, and clothing insulation are shown in Figure 5.4, which depicts the general life of a person who works or studies during the daytime, has lunch at home sometimes, and returns home cooking, taking exercise, watching TV, etc. in the evenings regularly on weekdays, and invites friends to have a party at weekend.

5.3.2/ EXPERIMENTAL RESULTS AND COMPARISON IN SCENARIO 1

Figure 5.5 shows the simulation results of our proposed method in Scenario 1. The upperleft subfigure records the variations of indoor and outdoor air temperature, in yellow line and blue line respectively. When the occupant is at home, the system will compute an optimal setpoint according to the present conditions, and based on this setpoint it can tune the power of the heating system to have the room temperature access to the setpoint.

Because the occupant may do different activities like sitting, cooking, sleeping, etc. and wear different clothes with different insulation values, the room temperature has to change to obtain good thermal comfort. When the occupant leaves the room, the heating system will turn off to save energy. This makes the indoor temperature drop to about 10 • C . Due to the huge thermal mass of the building, although the outdoor temperature drops below 0 • C , the indoor temperature can be kept around 10 • C , and this is called thermal flywheel effect [Tsilingiris, 2003]. The upper-right subfigure shows the PMV record. It defines that when there is no one in the room, the PMV is set to 0. From the subfigure, it can be seen that the PMV can be kept between +0.5 and -0.5 to assure excellent thermal comfort, except a few minutes' slight cool or warm feeling, which are caused by ( 1) leaving and then returning home in extremely cold weather, in which case the starting heating system is unable to warm the room instantly; and ( 2) the arriving of great amount of visitors from whom the metabolism and thermal radiation make the room over warm. The two bottom subfigures indicate the output heating power and the intelligent setpoint picking during this simulation. or +1. In order to maintain room temperature around the setpoint, the On/Off control method turn on and off frequently, which forms bandwidths in recorded PMV. Through comparative analysis of setpoint records, it can be found that our intelligent system alters the most times, for it is reactive to the change of present states. With PID control and On/Off control, the setpoints are reset several times manually. It reflects that thermal comfort can not be guaranteed with a constant setpoint. The setpoint should be tuned to adjust physical and personal states to maximize the comfort. of them take ambient temperatures and PMV values as inputs while take the heating power as output. The result with FLC is smoother, because the output is continuous. But to have a good FLC, sound knowledge from experts is needed. For RL, the output is discretized, so the PMV oscillates to some extent. These two methods can control the heating system without manually setting temperature preferences, but the results are not as good as our proposed method, and even worse than the two conventional methods.

To improve the performance of these two intelligent control methods, more sophisticated techniques should be employed. However, they raise the cost and consume more time.

Table 5.4 lists the PMV performance with different control methods in scenario 1, in which our proposed method outperforms the others. Energy consumption comparison among these five methods are displayed in Figure 5.9. It shows HIC use the least energy.

5.3.3/ EXPERIMENTAL RESULTS AND COMPARISON IN SCENARIO 2

Figure 5.10 shows the simulation results of our proposed method in scenario 2. The upper-left subfigure records the variations of indoor and outdoor air temperature. Figure 5.11 is the PMV simulation results with PID, On/Off, FLC, and RL in this scenario. From the PMV records, it can be seen that although the much higher average ambient temperatures bring challenges for systems to avoid over-heating especially when amount of persons visit the room, the proposed HIC method outperforms the other four in general.

Detailed information about PMV performance in this scenario is listed in Table 5.5. In ad-dition, Figure 5.12 shows that HIC is also one of the methods consuming the least energy in this scenario.

5.4/ CONCLUSION

This chapter has presented a building thermal model with which the thermal variation in a building can be simulated. Based on this model, a series of experiments are conducted.

By these experiments, it indicates that the proposed control method outperforms the two traditional control methods and the two intelligent control methods. Therefore, with the proposed cyber-physical system, a high level of thermal comfort can be obtained. But this does not mean the perfect solution, because occupants not only seek to improve their thermal comfort, but also have to consider the economic factor. Moreover, a smart home contains a variety of electric appliances not just space heating devices. By reasonably optimizing the schedules of controllable devices and at the same time taking into account demand response and cooperation with the other neighbouring smart homes, a better solution can be obtained.

MULTIPLE SMART HOME MODELING

6.1/ INTRODUCTION

In this chapter, since smart homes belong to the end nodes of a smart grid system, in order to solve problems involved in smart homes, the smart grid system should also be taken into account. Therefore, the concepts of holon and holarchy are introduced to model the complex smart grid system, which can be expressed in several layers. However, only the two bottom layers: household layer and device layer which constitute a smart microgrid holon, have direct effect on user comfort, where this dissertation is focused. To model these two layers, an organization centered multi-agent system paradigm is employed, which is described by the Role-Interaction-Organization meta-model. In addition, the organizational structure of the multi-agent system, the behaviours of different roles, and the interactions among agents are described in detail. This chapter is organized as follows. Section 6.2 introduces the concepts of holon and holarchy and how to model a smart grid system as a holarchy. How to model a smart 109 micro-grid system as an organization centered multi-agent system is presented in Section 6.3.

6.2/ HOLON AND HOLARCHY

Smart homes have naturally tight relationship with smart grid, which is a modernized electrical grid using information and communication technologies to improve the efficiency, reliability, economics, and sustainability of the production and distribution of electricity [DoE, 2015]. On one hand, a smart home is treated as an extended component that belongs to one part of the smart grid. On the other hand, enabled by demand response, improving global energy efficiency requires multiple smart homes to communicate with the smart gird. Therefore, the smart gird need to be taken into consideration in this work.

Since a smart grid system is a complex and distributed system, a flexible method to model this system has been proposed in [START_REF] Negeri | Holonic architecture of the smart grid[END_REF] based on the concepts of holon and holarchy.

The term holon, which is a self-similar structure composed of holons as substructures, was coined by [Koestler, 1990]. A holon can be treated either as an autonomous, atomic, entity, or as an organization of holons from different observation level, so it is a whole-part construct. This duality is called the Janus effect, in reference to the two faces of a holon.

The concept of holon can be found everywhere. For example, an organ of the human body can be seen as a holon, which is a functional entity being a part of the body and at the same time contains a group of different tissues also seen as holons. A hierarchical structure of holons composed of holons as substructures is called a holarchy. The whole structure in Figure 6.1 is a holarchy, and the green circles represent holons. In a holarchy, each holon can work autonomously and distributedly to make the holarchy a distributed system. By communicating and cooperating with other holons, the individual objective of the holon and the global objective of the holarchy can be achieved. Moreover, a holarchy can dynamically change its structure by reorganizing the holons to adapt to a dynamic environment. These properties of the holonic concept make it very suitable to model large and complex systems. When a smart grid system is modelled as a holarchy, different types of components in this system, such as energy communities, micro-grids, households, etc., are modelled as holons in this holarchy. In addition to considering occupants' comfort, electric appliances in a household should also be taken into account. Because they provide various services to fulfil occupants' requirements and determine users' comfort. Therefore, we argue that electric appliances are functional entities and should also be modelled as holons in a smart grid holarchy. In this holonic system, the functional entities or holons that are able to autonomously manage their resources and provide services, tending to aggregate together to form a bigger cluster to facilitate power and information exchange and thereby to enable the scalable and flexible management of the smart grid system. For instance, a set of electric appliances can be clustered together to form an end energy consumption and service providing community, which is a household holon. Upwards, a group of neighbouring household holons can be clustered together to form a district energy community, in which through communication and cooperation the energy consumption can be optimized as a larger consumer. Likewise, a group of district energy communities can form a city scope energy community at the next aggregation level, and a province scope, and a state scope, and so on. It means the clustering can be recursively repeated at various aggregation levels to model a large complex system. Then from a top-down view, a complex system can be divided into several resolvable parts at different levels. Through this bottom-up modeling approach and the divide-and-conquer technique, the complex problem can be solved. the scalability of the holarchy, more complex framework and problems of smart grid can be modelled and solved step by step. Hence, in this dissertation concentration is on the device layer and the household layer which have great and direct impact on occupants' comfort.

6.3/ MODELING SMART MICRO-GRID SYSTEM AS AN OCMAS

In this section, we model the smart micro-grid system, which contains holons in the household layer and the device layer of the smart grid holarchy, as an Organization Centered Multi-Agent System (OCMAS) based on the Role-Interaction-Organization (RIO) meta-model. The organizational structure at agent level, behaviours of different roles, and the organizational dynamics will be presented respectively.

6.3.1/ ORGANIZATIONAL STRUCTURE AT AGENT LEVEL

OCMAS is described by specific meta-models. Among them, RIO is proposed to be employed to design the smart micro-grid system thanks to its simplicity and conciseness.

As indicate in Section 3.6, RIO provides a graphical representation of organizations, that at agent level can be expressed by a cheeseboard diagram. In the cheeseboard diagram, a group is described as an ellipse like a board, an agent is represented as a skittle that stands on the board, and a role is represented as a hexagon. An agent may belong to several groups when they goes through several boards and if an agent plays a specific role, there is an arrow-line connecting from the role to the agent. In the Device Group, controllable devices are divided into two categories: Power-Shiftable and Time-Shiftable, and each controllable device agent can play only one of the corresponding roles. Power-Shiftable devices represent these devices whose working power is changeable to fit variations of environmental status, and at some circumstances they can decrease the power at the range that occupants can accept to reduce the load on the grid. Time-Shiftable devices are these appliances that can shift the power consumption time within a preferred working period and when they start to work they are in a constant power for a certain period. Non-Shiftable devices are devices that can shift neither working power nor working time, and therefore are also called fixed devices.

6.3.2/ ROLE BEHAVIOUR DESCRIPTION

Behaviours define what actions or services an agent is able to do or provide to achieve its goal or the global goal of the system. The behaviours of each role defined in the organizational structure of the smart micro-grid system are described as follows.

For Smart Grid role, there are two sequences of behaviours (Figure 6.4):

• when a specific time point has come, the Smart Grid role will notice the Smart Home role the time-varying electricity prices.

• when the Smart Grid role has received the power schedules from all agents playing the Smart Home role, it will calculate the total load variance on the main grid, and then will send this data to the Smart Home role. For Smart Home role, two sequences of behaviours exist (Figure 6.5):

Idle

• after the Smart Home role has received the total load variance or time-varying electricity prices, it will forward the data to the other roles in Device Group.

• when the Smart Home role has received the power schedules from all agents playing various device roles in Device Group, it will sum all power schedules up, and then will send the total schedule data as the power schedule of this smart home to the Smart Grid role.

Idle

Forward it to all device roles in Device Group Sum all power schedules up

Receive power schedules from all agents playing device roles in Device Group

Send total power schedule to Smart Grid role

Receive total load variance OR electricity prices For Non-Shiftable role, there are two behaviours (Figure 6.6):

• when the fixed device is turned on, the Non-Shiftable role is able to monitor and record the power consumption of this device, while when the device is turned off, it will return back to the idle state.

• when the inquiring event is triggered, the Non-Shiftable role can send the historical record of power consumption to the Device Manager role.

For Power-Shiftable role, there are two behaviours (Figure 6.7):

• when the Power-Shiftable device is turned on, the Power-Shiftable role can control the device to work according to the power schedule.

• when the schedule updating event is triggered, the Power-Shiftable role is able to send the new power schedule after optimization to the Device Manager role. For Time-Shiftable role, there are also two behaviours (Figure 6.8):

Idle

• when the Time-Shiftable device is turned on, the Time-Shiftable role can control the device begin to work at the scheduled time.

• when the schedule updating event is triggered, the Time-Shiftable role can send the new scheduled time after optimization to the Device Manager role.

Idle

Power Distribution Group Device Group

Smart Grid Smart Home Device Manager When the Smart Grid agent is triggered by an event like a specific time point, it sends the day-ahead electricity prices to all Smart Home agents in the district. After receiving this data a Smart Home agent forwards it to all device agents. Based on this price information, the controllable device agents initially optimize their schedules by an optimization algorithm, and send them back to the Smart Home agent. The dissatisfaction and electric cost of the schedules can be evaluated by the controllable device agents themselves based on the dissatisfaction models defined by occupants and the electricity prices received, but they have to wait for the total load variance computed and notified by the Smart Grid agent to complete their evaluation of the schedules. The Non-Shiftable device agents just send their historical power consumption records as schedules back to the Smart Home agent. After obtaining schedules from all device agents, it sums them up as the power schedule of the smart home and then sends it to the Smart Grid agent. According to schedules from all Smart Home agents, the Smart Grid agent can calculate the total load variance on the grid. Afterwards, all controllable device agents will receive this important information to complete their evaluation and further optimize their schedules.

Power-Shiftable

Time-Shiftable

Non-Shiftable

6.4/ CONCLUSION

This chapter has presented how to model a smart micro-grid system containing holons in the two bottom layers of the smart grid holarchy by employing an organization centered multi-agent system. In this multi-agent system, two groups are defined: Power Distribution Group, which includes Smart Grid role and Smart Home role, and Device Group, that contains Device Manager role, Non-Shiftable role, Power-Shiftable role and Time-Shiftable role. Based on the research results of these two layers and the scalability of the holarchy, more complex framework and problems of smart grid can be modelled and solved in future. In the next chapter, the objectives of each Smart Home agent will be defined and the algorithm that is used to optimize the schedule of each smart home will be discussed.

RESIDENTIAL SCHEDULING OPTIMIZATION

7.1/ INTRODUCTION

In this chapter, as it has been described in the state of the art, the problem involves a district which contains a set of neighbouring smart homes connecting to the main grid and forming a smart micro-grid system to exchange information and energy. Each Smart Home agent has three objectives: to minimize the dissatisfaction and cost in the individual home, and the total power variance on the smart micro-grid. According to the problem description, the scheduling within a smart home is identified as a multi-objective optimization problem, and therefore the three objectives are mathematically defined. Then based on the aggregation approach, these three objectives are aggregated into a single composite objective function. In order to solve the multiple home scheduling problem, a cooperative particle swarm optimization is employed within each smart home.

121

This chapter is organized as follows. Section 7.2 presents the composite objective function of a smart home. A scheduling optimization algorithm for each smart home is presented in Section 7.3.

7.2/ OBJECTIVE DEFINITION AND AGGREGATION

As presented in Section 2.6, in a District A, there are N smart homes connecting to the main grid to exchange data and energy, as shown in Figure 7.1. In a smart home, different controllable appliances are scheduled to work in order to satisfy occupants' requirements.

When they function properly, occupants' comfort can be guaranteed, whereas running defectively or unreasonably can decrease occupants' satisfaction, for example in case of space over or under heating or scheduling a washing machine to work during midnight.

Therefore one of the smart home goals is to minimize the dissatisfaction caused by these appliances in each smart home. Furthermore, demand response can propel customers to change their consumption patterns for a more economical purpose, such as turning down some devices' working power during on-peak periods and deferring some devices' running time to off-peak periods.

OBJECTIVE DEFINITION AND AGGREGATION

123 Thus, decreasing the household cost of electricity usage should be considered in the system.

However, a non-coordinated or selfish response of customers for time-varying prices may lead to severe peak rebounds at periods with lower prices. For example, when all N customers in District A prefer to use air-conditioning, washing machine, dish washer, etc. during off-peak time selfishly, the total load on the main grid will turn to be extremely high.

This will increase the burden of generating units, do harm to the main grid, and bring unforeseen disasters. Towards flattening the total load profile, the variance of energy consumption on the main grid need to be minimized.

Corresponding to these three situations, we propose that the objective function of each household i is to minimize the integration of the three objectives based on the aggregation approach introduced in Section 3.4, where the dissatisfaction and the cost can be evaluated by its individual Smart Home agent while the variance is evaluated by the Smart Grid agent based on power scheduling data received from all smart homes in the district, and we define it as follows:

min Dissatis f action(i) + α • Cost(i) + β • Variance (7.1)
where α and β are two weight factors. We define the three sub-objectives as follows:

Dissatis f action(i) = T t=1 S j=1 [I i j (t) • U i j (t)] (7.2) Cost(i) = T t=1 [ρ(t) • S j=1 P i j (t)] (7.3) Variance = T t=1 [ N i=1 S j=1 P i j (t) - 1 |T | N i=1

7.3/ SCHEDULING OPTIMIZATION ALGORITHM

In this research, in order to reduce the dissatisfaction and electric cost of each individual smart home and at the same time make a set of neighbouring smart homes coordinate with each other to decrease the total load on the main grid, we use Cooperative Particle Swarm Optimization (CPSO) to search for the near-optimal schedules for controllable devices in a smart home, for example, the heating power of an electric heater and the starting time of a washing machine. We choose CPSO because of its straightforward implementation and demonstrated ability of optimization. CPSO is based on the original PSO, which has been described and employed in Section 4.4 of this dissertation.

7.3.1/ COOPERATIVE PARTICLE SWARM OPTIMIZATION

As presented in the previous section, the composite objective function of each smart home includes scheduling optimization of controllable devices within the home and coordination with other neighbouring smart homes. When using the canonical PSO, each particle has to contain schedules of all controllable devices in all smart homes so that the total load variance can be calculated as the third part of the aggregated objective function.

Due to the huge size of the particle dimension, this algorithm will encounter the convergence problem called "two steps forward, one step back" [van den Bergh et al., 2004].

Because each particle containing all components for calculating a potential solution is evaluated by a scalar value called fitness. In the iteration process of the algorithm, some movements of the components of a particle are towards bettering the fitness value while some may have detrimental effects. As long as the effect of the improvement outperforms the effect of detriment to make the overall fitness increase, the better vector representing a particle position will replace the old one. Hence, it is possible that the components that were formerly close to solution move further away, and their performances are falsely treated as improvements. In order to illustrate this problem, an numerical example is shown as follows.

Assume that the fitness function is f (x) = x -λ 2 , where λ = (10, 10, 10). This obviously implies that the global minimum is x * = (10, 10, 10). If at present g best k = x k i = (12, 40, 10), the fitness of the global best position is f (g best k ) = 904, then assuming at the next iteration 20,15,30), the fitness of this particle is f (x k+1 i ) = 525, which is better than the global best position. Thus, the global best position will be replaced by x k+1 i . Although the fitness of the global best position is improved thanks to the considerably improvement of the second component in particle i, the first and the third components move farther away from 12 to 20 and from 10 to 30 respectively. This undesirable situation is caused by the fact that the fitness function is computed only after all the components in a particle have been updated.

x k+1 i = (
In order to overcome this convergence difficulty, authors in [van den Bergh et al., 2004] proposed the CPSO algorithm, which utilizes a divide-and-conquer approach. The vector to be optimized is divided into several components. Each swarm attempts to optimize only one of the components. The fitness of a particle belonging to one of the swarms is calculated by concatenating it with the present global best particles of the other swarms.

The CPSO is described in Algorithm 5. The function c( j, z) returns an m-dimensional vector formed by concatenating z, which represents the position of the particle from swarm j denoted by s j , with the global best positions of all the other swarms. According to CPSO, for the smart micro-grid system, there could be the same number of swarms as the number of total controllable devices in all neighbouring homes. Each swarm is employed to optimize the schedule of one controllable device instead of all of them. As each swarm's fitness function, which is a subsection of the composite objective function of a smart home, is also composed of three parts: dissatisfaction, electric cost and total load variance, the first two parts can be evaluated by the device agent independently, while the third part need cooperate with the other swarms. However, at this time, it only need to know the summed result of global best schedules of the other swarms instead of every schedule candidate in traditional PSO.

7.3.2/ FIRST STAGNATION

Although the CPSO algorithm is typically able to solve any problem that the standard PSO can solve and usually is more efficient, this algorithm has the possibility to be trapped in a state where all the swarms cannot jump out to search better solutions. This convergence difficulty is indicated in [van den Bergh et al., 2004] as stagnation.

(0,0) x 1

x 2

x* (4,0) and then easily find the zero point of its searching space, which is the point (0,0) in the coordinates denoted by the black point. Note that the global searching result of CPSO is concatenated by the global best positions of all the swarms, and hence it is a strictly non-increasing sequence, which means the algorithm can not take temporarily bad steps by each swarm to make overall improvements globally. Therefore, after the point (0,0) has been reached, the algorithm will be stagnated and can not find the global minimum

x * any more. For this convergence difficulty caused by stagnation, a good news is that due to the stochastic initialization of particles of each swarm in the CPSO algorithm, it is unlikely that this algorithm will be stagnated every time. This phenomenon inspires us that increasing the randomness is able to improve robustness of the algorithm.

7.3.3/ SECOND STAGNATION

In addition, in our scheduling optimization problem, there exists another kind of stagnation, we name it as second stagnation. This is caused by dividing one day's power schedule of power-shiftable devices into several sequential parts, and each part is optimized by a swarm. For example, a twenty four hour schedule of the space heating system is optimized by four swarms. Each swarm is responsible for the optimization of a part of the schedule including six hours. We do this for the reason that if we use a single swarm to optimize the whole schedule of a power-shiftable device, each particle will have a large dimension of twenty four, which may also cause the "two steps forward, one step back" convergence problem.

In light of this partition approach, when these swarms try to cooperate to find a nearoptimal schedule solution for the power-shiftable device for one day, they work successively according to the time order. Consequently, the optimization result of the former swarm will affect the fitness function of the latter one. A tiny miss of the first swarm will cause severe variation of the fitness function for the last swarm. Note that generally particles in a swarm tend to fly to the global best position, so if the fitness function changes a lot, the algorithm will be stagnated at a wrong place.

For example, as shown in Figure 7.3, assume that before k = n iteration the global minimum vector evaluated by the fitness function is in region A, so the global best particle should be some particle in this region and other particles tend to fly to this region. In the diagram, the grey circles are particles, the black circle is the global best particle, and the arrows represent the direction and magnitude of particle velocities. At iteration k = n, all particles fly near to region A. However, at the next iteration, the fitness function changes a lot, that makes the global minimum vector occur in a new region B. But from then on the global best particle is still chosen from the particles that fly to region A, and other particles still fly following it. Hence no particle is able to search the new region B to find the global minimum.

8.1/ INTRODUCTION

In this chapter, in order to verify the proposed multi-agent system and the cooperative particle swarm optimization algorithm, experimental simulations are conducted. Each smart home is assumed to be composed of two power-shiftable devices and three time-shiftable devices, which are regarded as typical controllable devices according to [START_REF] Pedrasa | Coordinated scheduling of residential distributed energy resources to optimize smart home energy services. Smart Grid[END_REF], Tasdighi et al., 2014], and their mathematical models are introduced. Then a case study is described, where it integrates the scheduling optimization algorithm into the proposed multi-agent system. Based on this case study, three simulation results are compared and analysed.

This chapter is organized as follows. Section 8.2 presents the models of controllable appliances. A case study is described in Section 8.3. Experimental results and analysis are presented in Section 8.4.

8.2/ MODELS OF CONTROLLABLE APPLIANCES

In this section, for simulation purpose, the models of two power-shiftable devices: space heater and water heater, and three time-shiftable devices: washing machine, clothes dryer, and dish washer are presented.

8.2.1/ POWER-SHIFTABLE DEVICES

Power-shiftable devices represent these devices whose working power is changeable to fit variations of environmental status, and at some circumstances they can decrease the power at the range that occupants can accept to reduce load on grid. Two main powershiftable devices considered in this work are electric space heater and electric water heater. Two models for each of them need to be defined. One is the dissatisfaction model which expresses how occupants feel dissatisfied after using these devices. The other is the physical model that describes the dynamic of the object controlled by the corresponding device, for example, the dynamic of indoor air temperature which is controlled by the electric space heater.

We define the dissatisfaction by using the space heater as follows:

∆ sh (t) = T * in (t) -T in (t) (8.1) U sh (t) =          ∆ sh (t) × if 0 < ∆ sh (t) < ζ ∆ sh (t) 2 × 10 else (8.2)
where T * in (t) and T in (t) are the desired temperature and the actual indoor temperature at time t respectively; U sh (t) is the dissatisfaction caused by using the space heater at time t; is a small scalar number; ζ is a temperature difference limit that occupants can tolerate. It means the farther away the indoor temperature is from the desired temperature the more dissatisfaction comes, and if the temperature difference is under occupants' tolerance, the dissatisfaction is negligible. According to [ASHRAE, 2005], the discrete time model using one-hour time step for indoor temperature variations is given by

T in (t + 1) = T in (t)e -1/τ + R • P sh (t) • (1 -e -1/τ ) + T out (t) • (1 -e -1/τ ) (8.3)
where R is the thermal resistance of the building shell in For the dissatisfaction by using the water heater, we define it as follows:

∆ wh (t) = T * hot (t) -T hot (t) (8.4) U wh (t) =          ∆ wh (t) × θ if 0 < ∆ sh (t) < η ∆ wh (t) 2 × 10 else (8.5)
where T * hot (t) and T hot (t) are the desired hot water temperature and the actual hot water temperature at time t respectively; U wh (t) is the dissatisfaction caused by using the water heater at time t; θ is a small scalar number; η is a temperature difference limit of hot water that occupants can tolerate.

In modeling water heating, we assume that the storage of hot water is always full, which means the equivalent of cold water enters the storage immediately when some hot water is drawn to be used. Hence, based on the heat balance, the temperature of hot water at each hour is calculated from the following equation [START_REF] Pedrasa | Coordinated scheduling of residential distributed energy resources to optimize smart home energy services. Smart Grid[END_REF]:

T hot (t + 1) = V out f low (t) • (T cold (t) -T hot (t)) + V total • T hot (t) V total + P wh (t) V total • C water (8.6)
where C water is the specific heat of water in kWh/L • • C, P wh (t) is the heating power of the water heater at time t, V total is the total volume of the hot water storage, V out f low (t) and T cold (t) are the volume of hot water drawn to be used and the cold water temperature at time t separately.

8.2.2/ TIME-SHIFTABLE DEVICES

Time-shiftable devices are these appliances that can shift the power consumption time within a preferred working period and when they start to work they are in a constant power for a certain period. In this work, three time-shiftable appliances are considered: washing machine, clothes dryer, and dish washer. Because there is no distinct physical dynamic caused by employing these devices, thus only the dissatisfaction models are needed. We define the dissatisfaction caused by using these devices as the functions shown in Figure 8.1. It means the operation of washing machine and clothes dryer during the sleeping time will annoy the habitants due to the noise, and the dish washer should have cleaned the tableware and be off during the lunch and dinner time, or they will cause dissatisfaction for occupants.

8.3/ CASE STUDY

In this section, we use a case study to illustrate the proposed multi-agent system which has been presented in Section 6.3 and the proposed optimization algorithm that has been discussed in Section 7.3. In each simulation, five smart homes with each composed of five controllable devices, the models of which have been introduced in the previous section, are taken into account. Because it's a case study, the multi-agent system is simplified, where only controllable devices are considered and the three time-shiftable devices are monitored and optimized by a single agent called Time Shiftable Devices agent instead of three separate agents. Thus, for each simulation there are twenty-one agents in total, including a Smart Grid agent and five Smart Home agents with each containing two power-shiftable agents: a Space Heating agent and a Water Heating agent, and a Time Shiftable Devices agent.

8.3.1/ PARAMETER SETUP

For each controllable device agent, the proposed Cooperative Particle Swarm Optimization with particle Reinitialization (CPSO-R) is employed to optimize its working schedule.

The parameters of CPSO-R are set as: w = 0.72, c 1 = 1.49, and c 2 = 1.49 according to [START_REF] Eberhart | Comparing inertia weights and constriction factors in particle swarm optimization[END_REF] Temperature difference limit for space heater η = 5 In this case study, we take trade-off between the swarm number and the particle vector dimension of each swarm considering its efficiency and complexity. In detail, nine swarms are designed for each smart home, of which four swarms optimize the power schedule of space heater, four swarms optimize the power schedule of water heater, and one swarm optimizes the starting time schedule of the three time-shiftable devices (Figure 8.4).

total load variance can be obtained only after its Smart Home agent has communicated with the Smart Grid agent. The Smart Grid agent can compute the total load variance for the device agent based on the global best schedule records received from all Smart Home agents (see Figure 8.4). Then a fitness value of the schedule for the device agent is calculated by aggregating these three parts up with different weights.

8.3.3/ RUNNING ENVIRONMENT

To conduct simulations, the proposed agents are implemented in a platform called Java Agent Development Framework (JADE), which aims at developing multi-agent systems and applications conforming to FIPA standards. It provides a runtime environment for FIPA-compliant agents and a package to develop Java agents.

8.4/ EXPERIMENTAL RESULTS AND ANALYSIS

In order to show the impacts of demand response as well as coordinated actions of neighbouring smart homes in a district to avoid peak rebounds, three cases: basic case, case without coordination and case with coordination are simulated and compared. In each simulation, occupants are assumed at home except for 8 : 00 ∼ 12 : 00 and 14 : 00 ∼ 18 : 00 during which periods they are assumed to be out for working.

8.4.1/ BASIC CASE Figure 8.6 shows the experimental results of the basic case. In this case, demand response and coordination are not considered and the algorithm is only used to control the space heater and the water heater, so there is no intelligent scheduling and operation for controllable appliances. When people turn them on, they work immediately and maximize people's comfort. Figure 8.6(c) shows that all the three time-shiftable devices of the five households are used after occupants return home in the evening. This not only causes peak loads on the main grid, as can be seen in Figure 8.6(a), but also costs more money for the electricity prices during these periods are higher. Figure 8.6(b) and Figure 8.6(d) show that the mean indoor temperature and the mean hot water temperature of the 5 families reach the desired degrees when people are at home. Note that when occupants are not at home, the space heater is turned off and the desired temperature is not important which is set to be the same as the ambient temperature during these periods. In order to solve the peak rebounds and make trade-off between electrical budget and comfort, all homes in the district are assumed to be able to coordinate by aggregating their electricity information on the main grid, and a 2-degree of indoor temperature and a 5-degree of hot water temperature away from the desired temperatures are assumed to be tolerated for people.

The experimental results of the case 3 are shown in Figure 8.8. From Figure 8.8(a), we can see that the variance of total load on the main grid is minimized, and peak loads and peak rebounds are avoided. show that by employing our proposed system and method, power-shiftable devices can make good trade-off between user comfort and power consumption and time-shiftable devices can select proper starting time to guarantee their service quality while reduce the power variance on the grid. The statistical comparison of the three cases is listed in Table 8.2, and we can find that with demand response and coordination the proposed system and method can both save money and balance loads on the grid. (OCMAS) was used to model this large-scale complex system. Each smart home comprises a set of controllable electric devices. Some of them are power-shiftable, while the others are time-shiftable. From literatures, we have seen that selfish and non-coordinated operation of these devices for time-varying prices may lead to severe peak rebounds at periods with lower prices. Therefore, we proposed a meta-heuristic algorithm based on Cooperative Particle Swarm Optimization (CPSO) to optimize schedules of controllable devices in each smart home of the smart micro-grid system. The experimental results have verified our proposed system and algorithm. In addition, the problems solved and the system designed in this dissertation belong to the bottom layers of the smart grid holarchy. Hence, based on these research results and the scalability of the holarchy, more complex problems from higher layers of the holarchy can be tackled in future.

In order to understand the work presented in this thesis, we briefly review the path that we followed throughout this work.

In Chapter 2, we discussed the state of the art in terms of context and problem that our work covers. Firstly, the context related to thermal comfort were studied, which contains principles of thermal comfort and thermal comfort models. When considering multiple homes of a district that take advantage of demand response, non-coordinated scheduling may cause peak rebounds. Hence, the multiple home scheduling problem which need a coordinated approach to solve is introduced. In light of the context and problem, the state of the art in terms of solutions for thermal comfort improvement in a single smart home and solutions for the multiple home scheduling problem are investigated in Chapter 3.

Based on the studies and analysis of Chapter 3, we presented in Chapter 4 our proposed CPS with a hybrid intelligent control approach for the thermal comfort improvement in an individual smart home and we verified the proposed system and method in Chapter 5.

The main characteristics of the proposed CPS and the control method are summarized as follows:

• The CPS brings together the discrete logic of computing to control and monitor the continuous thermal dynamics of smart homes.

• The CPS can perceive required environmental and personal parameters by using advanced sensors and network.

• The proposed hybrid intelligent control method includes two modules: a inverse PMV optimization module employing PSO for setpoint selecting and a PID control module for indoor temperature regulating.

• Based on this system, thermal comfort is improved compared with two traditional control methods and two intelligent control approaches.

• Energy consumption is reduced through avoiding the overheating of electric heater.

In order to make multiple smart homes in a district not only reduce their individual electricity bill and dissatisfaction by using electric devices, but also decrease the variance of the total load on the main grid, an OCMAS was presented to model this complex system in Chapter 6 and a CPSO was developed to optimize schedules of controllable devices in Chapter 7. In Chapter 8, experimental simulations were conducted to verify the proposed system and method. The proposed OCMAS and CPSO have these characteristics:

• The proposed OCMAS can overcome the undesired properties such as unconstrained communication, unlimited access, and single framework, of classical MAS.

• The problem we deal with belongs to the bottom layers of the smart grid holarchy, so the developed MAS can be integrated into and reused by a more complex system from higher layers of the holarchy.

• Managing different controllable devices of each smart home in a district to reduce user dissatisfaction, electricity cost, and variance of total load, it involves a multiobjective optimization problem. In order to figure this problem out, we use the aggregation method which aggregates the three objectives to generate a single composite objective function.

• A novel CPSO algorithm with particle reinitialization is developed for optimization of scheduling and operation of electric devices in smart homes. This algorithm can solve the "two steps forward, one step back" problem of canonical PSO, decrease the possibility of the first stagnation problem and tackle the second stagnation problem.

• Based on the proposed system and approach, a coordinated management of a set of smart homes can be realized to increase benefits of both occupants and the grid utility.

9.2/ PERSPECTIVES AND FUTURE RESEARCH DIRECTIONS

The three main aspects presented in this section can be considered as extensions of this work. The first is to integrate green energy and storage systems into smart home systems. The second is to implement price negotiation ability for smart home agents participating in the electricity market to maximize their payoffs. The third is to implement intelligent and flexible grouping strategies for smart homes in a large district.

9.2.1/ INTEGRATION OF GREEN ENERGY AND STORAGE SYSTEMS INTO SMART HOME SYSTEMS

While the development of technologies of smart grid and green energy brings numerous opportunities for future power system and benefits for habitants, it also requires to solve important challenges for design and management problems. One of them comes from the usage of green energy resources, such as wind power and solar power. Although this sort of energy helps reduce carbon emissions, it is difficult to provide reliable and stable power as a result of uncertainties of the weather and other factors. Therefore, how to leverage the potential benefits of green energy resources while maintaining acceptable levels of reliability becomes a key point. One way to mitigate this problem is to take advantages of intelligent storage systems, i.e. battery and super-capacitor, to conveniently coordinate the electrical supply and demand in the smart grid. For example, the storage systems charge when there exists excess energy, and they discharge when green energy resources solely provide insufficient energy. In our future work, we plan to integrate green energy and storage systems to help reduce power consumption from the main grid, especially during peak load periods.

9.2.2/ IMPLEMENTING PRICE NEGOTIATION ABILITY FOR SMART HOME AGENT PARTICIPATING IN ELECTRICAL MARKET

As the electricity industry is moving toward an open and competitive market to replace the old regular form, all participants in the electrical market are allowed to negotiate so as to reach rational deals to maximize payoffs during buying and selling process [START_REF] Al-Agtash | An evolutionary computation approach to electricity trade negotiation[END_REF][START_REF] Prac ¸a | Intelligent agents for negotiation and game-based decision support in electricity markets[END_REF], Slavickas et al., 1999]. When smart homes are integrated with green energy resources, they are not only power consumers but also producers. When the green energy production is insufficient, the household need to purchase energy from the utility grid, while when the green energy resources produce more energy than the household demands, the smart home may sell the redundant energy back to the utility grid. During these processes, households and the utility grid need to negotiate the electricity prices to make the transaction successfully operate.

9.2.3/ IMPLEMENTATION OF INTELLIGENT AND FLEXIBLE GROUPING STRATE-GIES FOR SMART HOMES

In this dissertation, only a district that contains a set of smart homes is considered. However, different districts have different scales. Therefore, implementing intelligent and flexible grouping strategies for large districts which contain a great number of homes should be taken into account. In general, there are two reasons. On one hand, reducing the variance of total load on the main grid need smart home agents exchange their power consumption data with the grid agent. Hence, by using a divide-and-conquer method smart homes of a large district can be divided into small groups to decrease the computing time and waiting time. On the other hand, due to the high initial investment expense of renewable distributed generation (DG), like wind turbine, solar photovoltaic, and high capacity batteries, it is more profitable for a group of households to share the residential DG.

Abstract:

With the development of human society, smart homes attract more and more attention from the research fields, since they can offer a great potential for improving indoor comfort and energy efficiency. In this work, firstly great effort is concentrated on the thermal comfort improvement, not only for its dominant influence on indoor comfort, but also the overriding energy consumption in residential buildings spent on it. Thus a cyber-physical system , which adopts a hybrid intelligent control method, to regulate the indoor temperature is designed and implemented. Based on this system, thermal comfort and the energy efficiency can be improved compared with other popular methods. Towards further improving energy efficiency of individual homes and the grid utility, a set of neighbouring smart homes, each comprising a number of controllable devices, and a smart grid adopting demand response are considered. Moreover, an organization centered multi-agent system is employed to model this large-scale complex system. For the reason that selfish and non-coordinated scheduling of the controllable devices for time-varying prices may lead to severe peak rebounds, hence a meta-heuristic algorithm based on cooperative particle swarm optimization is proposed to optimize schedules of these devices. The experimental results have verified the proposed system and algorithm. In addition, the problems solved and the system designed in this dissertation belong to the two bottom layers of the smart grid holarchy. Based on these research results and the scalability of the holarchy, more complex problems from higher layers of the holarchy can be tackled in future.

R ésum é :

Avec le d éveloppement de la soci ét é humaine, les maisons intelligentes attirent de plus en plus d'attention dans les champs de recherche, car ils peuvent offrir un grand potentiel pour am éliorer le confort int érieure et l'efficacit é énerg étique. Dans ce travail, tout d'abord un grand effort est concentr é sur l'am élioration de confort thermique, non seulement pour son influence dominante sur le confort int érieur, mais aussi la consommation d' énergie dominante dans les b âtiments r ésidentiels pass é sur elle. Ainsi, un syst ème de cyber-physique, qui adopte une m éthode de contr ôle intelligent hybride, pour r éguler la temp érature à l'int érieur est conc ¸u et mis en oeuvre. Bas é sur ce syst ème, le confort thermique et l'efficacit é énerg étique peut être am élior ée par rapport à d'autres m éthodes populaires. Vers am éliorer encore l'efficacit é énerg étique des maisons individuelles et de l'utilit é de la grille, un ensemble de maisons intelligentes voisins, comprenant chacun un certain nombre de dispositifs contr ôlables, et un r éseau électrique intelligent adoptant la r éponse à la demande sont consid ér ées. En outre, un syst ème multi-agent organisation centr ée est utilis é pour mod éliser ce syst ème complexe à grande échelle. Pour la raison que la planification égoïste et non coordonn ée des dispositifs contr ôlables pour des prix variant dans le temps peut conduire à des rebonds de pointe graves, d'o ù un algorithme m éta-heuristique bas ée sur la optimisation coop érative par essaim de particules est propos ée pour optimiser les planifications des dispositifs. Les r ésultats exp érimentaux ont v érifi é le syst ème et l'algorithme propos é. En outre, les probl èmes r ésolus et le syst ème conc ¸u dans cette th èse appartiennent aux deux couches inf érieures de la holarchie de le r éseau électrique intelligent. Bas é sur ces r ésultats de recherche et l' évolutivit é de l'holarchie, des probl èmes plus complexes à partir des couches sup érieures de la holarchie peuvent être abord és à l'avenir.

6. 3

 3 The Organizational Structure of a Smart Micro-Grid using a Cheese-Board Notation at Agent Level . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.4 Role Behaviour Description of Smart Grid . . . . . . . . . . . . . . . . . . . 6.5 Role Behaviour Description of Smart Home . . . . . . . . . . . . . . . . . . 6.6 Role Behaviour Description of Non-Shiftable . . . . . . . . . . . . . . . . . . 6.7 Role Behaviour Description of Power-Shiftable . . . . . . . . . . . . . . . . 6.8 Role Behaviour Description of Time-Shiftable . . . . . . . . . . . . . . . . . 6.9 The Organizational Sequence Diagram of the Smart Micro-Grid Organization119 7.1 Smart Homes in a District . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.2 Convergence Difficulty Caused by Stagnation . . . . . . . . . . . . . . . . . 7.3 Convergence Difficulty Caused by Stagnation . . . . . . . . . . . . . . . . . 8.1 Dissatisfaction of Time-Shiftable Devices . . . . . . . . . . . . . . . . . . . 8.2 Time-Varying Price [Safdarian et al., 2014] . . . . . . . . . . . . . . . . . . 8.3 Outdoor Temperature of One Day . . . . . . . . . . . . . . . . . . . . . . . . 8.4 CPSO-R Implementation and Operation . . . . . . . . . . . . . . . . . . . . 8.5 Screenshot of Agents and Interactions in JADE . . . . . . . . . . . . . . . . 8.6 Experimental Results of Case 1 . . . . . . . . . . . . . . . . . . . . . . . . . 8.7 Experimental Results of Case 2 . . . . . . . . . . . . . . . . . . . . . . . . . 8.8 Experimental Results of Case 3 . . . . . . . . . . . . . . . . . . . . . . . . .

Figure 2 . 2 :

 22 Figure 2.2: Relationship between PPD and PMV

Figure 2 . 4 :

 24 Figure 2.4: PMV Black Box Model

Figure 2 . 5 :

 25 Figure 2.5: The aPMV Model

*

  Price-based demand response programs charge customers time-varying electricity rates that reflect the value and cost of electricity in different time periods. * Incentive-based demand response programs are designed such that customers can receive extra payments to reduce their loads during peak periods. Since the incentive-based demand response is too flexible that different specific programs can be designed according to features of different regions, price-based demand response draws more attention of academic research. So our work is based on the price-based demand response.

  -Based Control . . . . . . . . . . . . . . . . . . . . . . . . 48 3.2.2 Model-Free Control . . . . . . . . . . . . . . . . . . . . . . . Concepts . . . . . . . . . . . . . . . . . . . . . . . . . . 58 3.5.2 Agent Management Reference Model . . . . . . . . . . . . . . . 60 3.5.3 Agent Communication Language . . . . . . . . . . . . . . . . . . 62 3.5.4 Agent Life Cycle . . . . . . . . . . . . . . . . . . . . . . . . .

Figure 3 . 1 :

 31 Figure 3.1: Classification Schematic of Control Methods

(Figure 3 . 2 :

 32 Figure 3.2: A Block Diagram of PID Controller

Figure 3 . 3 :

 33 Figure 3.3: Fuzzy Logic Controller

Figure 3 .

 3 Figure 3.4 is an example of Pareto front. In this example, it contains two objective functions: f 1 and f 2 . Each point in the coordinates represents a feasible choice, and smaller values are preferred to larger ones. The Pareto front, indicated by the red line, are those

Figure 3 . 5 :

 35 Figure 3.5: Agent and Environment

62CHAPTER 3 . 5 A

 35 THERMAL COMFORT CONTROL AND MULTIPLE HOME SCHEDULING 4 A Directory Facilitator (DF) provides yellow pages services to other agents, that means agents may register their services with the DF or query the DF to find out what services are provided by other agents. Message Transport Service (MTS) enables messages to be transported from one agent to the other or even between different platforms. Messages are usually created in FIPA ACL message structure.

Figure 3 .

 3 Figure 3.7: An ACL Message Example

Figure

  Figure 3.7 shows an ACL message example. This message is sent from agent A to

Figure 3 . 8 :

 38 Figure 3.8: Life Cycle of an Agent, based on[FIPA, 2004]

Figure 3 .

 3 Figure 3.9. At the organizational level, there are two organizations: Organization 1 and Organization 2 respectively. Organization 1 contains two roles, denoted by Role 1 and Role 2. Organization 2 contains three roles: Role 3, Role 4 and Role 5. Only roles in the same organization can have interactions with each other.

  Figure 3.9: RIO Meta-Model Example

Figure 4 . 1 :

 41 Figure 4.1: Cyber-Physical System

Figure 4 . 2 :

 42 Figure 4.2: Cyber-Physical System Architecture for Thermal Comfort Control

Figure 4 .

 4 Figure 4.2 depicts the architecture of the proposed CPS for thermal comfort management. In this figure, there are three parts: Physical World, Network, and Cyber World. Physical World contains Sensor Domain and Actuator Domain. Different indoor and outdoor parameters, such as outdoor temperature, indoor temperature, air speed, metabolic rate etc., can be perceived by different sensors. These data are transmitted to Cyber World

Figure 4 . 3 :

 43 Figure 4.3: Inverse PMV Optimization

78CHAPTER 4 .

 4 A CYBER-PHYSICAL SYSTEM FOR THERMAL COMFORT IMPROVEMENT 4.4.2/ META-HEURISTIC ALGORITHMS In this subsection, three popular meta-heuristic algorithms, namely particle swarm optimization, genetic algorithm and simulated annealing are introduced, and the best algorithm for searching for the setpoint temperature T * air from the PMV calculation function F is selected by experiments and analysis. 4.4.2.1/ PARTICLE SWARM OPTIMIZATION Particle Swarm Optimization (PSO), which is derived from simulating social behavior as a stylized representation of the movement of organisms in a bird flock or fish school,

Figure 4 . 4 :Algorithm 3

 443 Figure 4.4: Crossover in a Typical Recombination Step

Figure 4 . 5 :

 45 Figure 4.5: Comparison of Searching Performance among SA, PSO-Pop-5, and GA-Pop-

Figure 4 .

 4 Figure 4.6: Comparison of Searching Performance among SA, PSO-Pop-10, and GA-Pop-10

Figure 4 . 7 : 20 Figure 4 .

 47204 Figure 4.7: Comparison of Searching Performance among SA, PSO-Pop-20, and GA-Pop-20

Figure 4 . 8 :

 48 Figure 4.8: Comparison of Time Performance among All Algorithms

Figure 4 .

 4 Figure 4.9: Comparison of Failed Times among All Algorithms

  Figure 4.10: PID Controller

U

  Figure 5.1: Thermal Network

  the thermal capacitance, ∆Q is the change of heat and ∆T is the change of temperature. Now the thermal model of a smart home depicted in Figure 5.1 can be expressed by differential Equations 5.4 -5.8, except the sub-circuit of windows, because we assume windows have not the property of thermal capacitance to store heat due to their tiny thermal mass. (T air -T w ) + U wo (T amb -T w )(5.4)

Figure 5

 5 Figure 5.2: Thermal Network of Multiple Rooms

Figure 5 . 4 :

 54 Figure 5.3: Surface of Fuzzy Logic Controller

Figure 5 . 5 :

 55 Figure 5.5: Simulation Result of HIC in Scenario 1

Figure 5 . 7 : 1 Figure 5 .

 5715 Figure 5.6: Simulation Result of PID Control in Scenario 1

Figure 5 . 8 : 1 Figure 5 .

 5815 Figure 5.8: PMV Results of FLC and RL in Scenario 1

Figure 5 .

 5 Figure 5.9: Energy Consumption Comparison in Scenario 1

Figure 5 .

 5 Figure 5.10: Simulation Result of HIC in Scenario 2

Figure 5 .

 5 Figure 5.12: Energy Consumption Comparison in Scenario 2

Contents 6 . 1

 61 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109 6.2 Holon and Holarchy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110 6.3 Modeling Smart Micro-Grid System as an OCMAS . . . . . . . . . . 113 6.3.1 Organizational Structure at Agent Level . . . . . . . . . . . . . . 113 6.3.2 Role Behaviour Description . . . . . . . . . . . . . . . . . . . . . 115 6.3.3 Organizational Dynamics Description . . . . . . . . . . . . . . . 118 6.4 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120

  Figure 6.1: Holon and Holarchy

Figure 6 . 2 :

 62 Figure 6.2: Holarchy of Smart Grid System

Figure 6 . 3 Figure 6 . 3 :

 6363 Figure 6.3 presents the organizational structure of the smart micro-grid system at agent level. There are two group boards: the Power Distribution Group and the Device Group.The Power Distribution Group has two roles: the Smart Grid role and the Smart Home role. In this group, the Smart Grid agent collects power schedules from all Smart Home agents, and informs them the time-varying prices and the total load variance. Each Smart Home agent is a Device Manager at the same time.

Figure 6 . 4 :

 64 Figure 6.4: Role Behaviour Description of Smart Grid

FigureFigure 6

 6 Figure 6.5: Role Behaviour Description of Smart Home

  Figure 6.7: Role Behaviour Description of Power-Shiftable

Figure 6 . 8 :

 68 Figure 6.8: Role Behaviour Description of Time-Shiftable

Figure 6

 6 Figure 6.9: The Organizational Sequence Diagram of the Smart Micro-Grid Organization

  Particle Swarm Optimization . . . . . . . . . . . . . 124 7.3.2 First Stagnation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126 7.3.3 Second Stagnation . . . . . . . . . . . . . . . . . . . . . . . . . . 128 7.4 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131

Figure 7 . 1 :

 71 Figure 7.1: Smart Homes in a District

Figure 7 . 2 :

 72 Figure 7.2: Convergence Difficulty Caused by Stagnation

Figure 7 . 3 :

 73 Figure 7.3: Convergence Difficulty Caused by Stagnation

Figure 8 . 1 :

 81 Figure 8.1: Dissatisfaction of Time-Shiftable Devices

  , and each swarm contains 50 particles. The time-varying electricity prices and one day's ambient temperature are shown in Figure 8.2 and Figure 8.3.The weight factors of sub-objectives and parameters of controllable appliance models are listed in Table8.1.

Figure 8 . 2 :

 82 Figure 8.2: Time-Varying Price [Safdarian et al., 2014]

  Figure 8.3: Outdoor Temperature of One Day

Figure 8 . 5 :

 85 Figure 8.5: Screenshot of Agents and Interactions in JADE

Figure 8 .

 8 Figure 8.5 shows the screenshot of all agents created in each simulation, including the three default system agents, two sniffer agents which are responsible for sniffing the interactions among agents, and twenty-one working agents that we design. The right part of the figure shows the interactions or messages sending among these twenty-one working agents in order to optimize the schedule of each controllable device.

  Figure 8.6: Experimental Results of Case 1

  Figure 8.7: Experimental Results of Case 2

Figure 8 . 8 :

 88 Figure 8.8: Experimental Results of Case 3

8. 5

 5 / CONCLUSIONThis chapter has presented the mathematical models of controllable devices in a smart home, and introduced a case study. The performance of the proposed system and algorithm has been demonstrated by conducting and comparing three case simulations. The experimental results indicate that selfish operation and optimization of individual home will make peak rebounds happen, which may damage the power grid, cause unforeseen disasters, and do harm to the global profit, while a coordinated management of a set of smart homes can get rid of this negative situation and increase the benefits for both occupants and the grid utility from demand response.CONCLUSION AND PERSPECTIVES9.1/ GENERAL CONCLUSION Along this thesis, we have worked on the improvement of occupant comfort and energy efficiency in smart homes. At first, thermal comfort, which is the condition of mind that expresses satisfaction with the thermal environment, is concentrated on. It is one of the three main factors that determine occupants' quality of life in a home environment. The residential thermal comfort is depicted by the Predicted Mean Vote (PMV) index, which stands among the most recognized thermal comfort models based on the heat balance of the human body. Besides the crucial effect of thermal comfort on occupants' indoor lives, it takes a great part of the total residential energy consumption. Based on these contexts, we designed and implemented a Cyber-Physical System (CPS), which adopts a hybrid intelligent control method, to regulate the indoor temperature. The controller in this CPS has two modules, the first module employs a Particle Swarm Optimization (PSO) algorithm to inversely obtain an optimal temperature from the PMV model as a setpoint, and the second module uses a PID controller to regulate the indoor temperature to reach this setpoint. Based on this CPS, the thermal comfort can be maintained and the energy efficiency can be improved.Towards further improving energy efficiency of individual homes and the grid utility, a number of neighbouring smart homes and a smart grid were considered. Due to their distributed and autonomous properties, an Organization Centered Multi-Agent System

. 41 2.5 Single Home Scheduling . . . . . . . . . . . . . . . . . . . . . . . . . . 44 2.6 Multiple Home Scheduling Problem

  thermal comfort models are introduced and compared. However, only thinking about thermal comfort is not enough. Towards further improving energy efficiency of individual homes and the grid utility, demand response is relayed on. Although much work has already been carried out to activate demand response potentials in individual smart home systems, severe peak rebounds may happen without a global consideration.

	THERMAL COMFORT AND ENERGY
	Therefore, a set of neighbouring smart homes in a district are concerned and the problem
	EFFICIENCY IN SMART HOMES of multiple home scheduling is described.
	This chapter is organized as follows. Section 2.2 gives the principal knowledge about
	thermal comfort. Section 2.3 introduces three popular thermal comfort models. Demand

Contents 2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 2.2 Principles of Thermal Comfort . . . . . . . . . . . . . . . . . . . . . . 30 2.2.1 Thermoregulation of Human Body . . . . . . . . . . . . . . . . . 30 2.2.2 Thermal Comfort Parameters . . . . . . . . . . . . . . . . . . . . 31 2.3 Thermal Comfort Models . . . . . . . . . . . . . . . . . . . . . . . . . 34 2.3.1 Predictive Mean Vote Model . . . . . . . . . . . . . . . . . . . . . 34 2.3.2 Adaptive Thermal Comfort Model . . . . . . . . . . . . . . . . . . 37 2.3.3 Simplified Comfort Model . . . . . . . . . . . . . . . . . . . . . . 40 2.4 Demand Response . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45 2.7 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46

2.1/ INTRODUCTION

Nowadays, smart home attracts more and more attention from research fields. When only considering an individual home, much effort of this work is made on the improvement of thermal comfort. Therefore, how human body reacts to environmental changes to maintain thermal comfort and what are the primary parameters affecting thermal comfort are studied. After that, in order to evaluate thermal comfort according to present conditions, three response is introduced in Section 2.4, and based on this scheme some related work of single home scheduling is presented in 2.5. In Section 2.6, the multiple home scheduling problem is introduced.

Table 2 .

 2

	1: Typical Met Levels for Various Activities [ASHRAE, 2005]
	Activity	(met)
	Sleeping	0.7
	Seated, quiet	1.0
	Standing, relaxed 1.2
	Filing, seated	1.2
	Filing, standing	1.4
	Walking about	1.7
	Walking (0.9 m/s) 2.0
	Cooking	1.6 to 2.0
	Housecleaning	2.0 to 3.4
	Exercise	3.0 to 4.0

work within an organism. Being different states and doing different work will lead to various metabolic rates. The exemplary characteristic metabolic rates for some selected activities are shown in Table

2

.1.

  2

Table 2 .

 2

	2: Thermal insulation for typical combinations of garments [ISO7730, 2005]
	Daily wear clothing	(clo)
	Panties, T-shirt, shorts, light socks, sandals	0.3
	Underpants, shirt with short sleeves,
	light trousers, light socks, shoes	0.5
	Panties, petticoat, stockings, dress, shoes	0.7
	Underwear, shirt, trousers, socks, shoes	0.7
	Panties, shirt, trousers, jacket, socks, shoes	1.0
	Panties, stockings, blouse, long skirt, jacket, shoes	1.1
	Underwear with long sleeves and legs, shirt,
	trousers, V-neck sweater, jacket, socks, shoes	1.3
	Underwear with short sleeves and legs, shirt,
	trousers, vest, jacket, coat, socks, shoes	1.5
	Underwear with short sleeves and legs, shirt,
	trousers, jacket, heavy quilted outer jacket, socks, shoes	2.0
	Underwear with long sleeves and legs, shirt,
	trousers, thermo-jacket, overalls with heave quilting, socks, shoes 2.5

  1.05 + 0.645I cl for I cl > 0.078 m 2 • K/W

		(2.6)
	where
	M	is the metabolic rate in W/m 2 ,
	W	is the effective mechanical power in W/m 2 ,
	I cl	is the clothing insulation in m 2 • K/W,
	f cl	is the clothing surface area factor,
	T air	is the air temperature in • C,
	T MRT is the mean radiant temperature in • C,
	v air	is the relative air velocity in m/s,
	p a	is the water vapour partial pressure in Pa,
	h

c is the convective heat transfer coefficient in W/(m 2 • K), t cl

is the clothing surface temperature in • C.

  .3.

	Outdoor Climates	Indoor Climates	Human Physiology & Thermal Sensation	Actual Thermal Body Sensation of Human
	Physical
	Stimuli
		Behavioural
		Adaptations
			Social, Economical
			and Cultural
			Background
		Psychological	Thermal
		Adaptations	Expectations
			Previous Thermal
			Experiences
			Adaptations
	Figure 2.3: The Adaptive Model Mechanism, from

Table 2 .

 2

	3: ASHRAE Standard Recommendations [ASHRAE, 2005]
		Operative temperature Acceptable range
	Summer 22 • C	20 -23 • C
	Winter	24.5 • C	23 -26 • C
	midity, air velocity as well as the two personal variables, clothing insulation and metabolic
	rate. Specifically, SET is the temperature of an imaginary environment at 50% relative
	humidity, < 0.1m/s air speed, and mean radiant temperature equal to air temperature,
	in which the total heat loss from the skin of an imaginary occupant with an activity level
	of 1.0met and a clothing level of 0.6clo is the same as that from a person in the actual
	environment, with actual clothing and activity level [ASHRAE

  it formally explains how a MAS is constructed and operated, as presented in Figure3.6.

	Software
		Agent Platform
	Agent	Agent Management System	Directory Facilitator
		Message Transport System
	Figure 3.6: Agent Management Reference Model

Table 4 .

 4 In the application domain, Bluetooth is designed for cordless devices as computer peripherals, such as mice, keyboards, printers, and a master Bluetooth device can communicate with a maximum of seven devices. UWB intends to provide high-speed wireless communication thanks to its bandwidth which can satisfy most of the indoor multimedia applications. Wi-Fi is usually used for computer-to-computer connections as an exten-

	Standard	Bluetooth UWB	ZigBee	Wi-Fi
	IEEE spec.	802.15.1	802.15.3a	802.15.4	802.11a/b/g
	Frequency band 2.4GHz	3.1 -10.6GHz 868/915MHz; 2.4GHz 2.4GHz; 5GHz
	Max signal rate	1Mb/s	110Mb/s	250Kb/s	54Mb/s
	Nominal range	10m	10m	10 -100m	100m
	Max cell nodes	8	8	> 65000	2007
	sion or substitution of cable networks. Zigbee, that possesses a powerful networking
	capability, strong network robustness and reliability, is designed for wireless monitoring
	and control networks, which is constituted of embedded devices requiring low power con-

1: Comparison of Bluetooth, UWB, Zigbee, and Wi-Fi Protocols, from

[START_REF] Lee | A comparative study of wireless protocols: Bluetooth, uwb, zigbee, and wi-fi[END_REF]

sumption and tolerating low data rates. So sensors and actuators in the designed CPS can use ZigBee for the data transmission with the computing units, as the dashed arrowlines: P2C Sensing and C2P Actuating indicated in Figure 4.2.

76CHAPTER 4. A CYBER-PHYSICAL SYSTEM FOR THERMAL COMFORT IMPROVEMENT

  1, a PMV value can be computed to index the thermal comfort of the present environment. Now assume that an inverse PMV function on indoor temperature T air can be found, denoted by G, which can calculate an indoor temperature value as the output by taking T MRT , v air , h r , I cl , M and a PMV value as inputs. Recall that in the 7-point thermal sensation scale of the PMV model, 0 represents neutral or comfortable. So based on the five parameters TMRT , vair , hr , Īcl , M estimated or perceived from the present environment and the PMV preference value PMV air means if the home is heated to this temperature level, occupants will feel comfortable in thermal aspect. Formally it is mathematically expressed as follows:T * air = G(PMV * , T MRT , v air , h r , I cl , M)(4.2)If we can find this inverse function G, then from this function we can calculate a temperature value based on the present environment as a setpoint for PID controller, which can regulate the real indoor temperature to reach this setpoint to obtain a high level of thermal comfort, as shown in Figure4.3.

			PMV Preference:
			PMV*
		Clothing Insulation,
	Physical Process	Metabolism Rate, MRT, Humidity, Air Velocity	Inverse PMV Optimization by Meta-Heuristic	Setpoint: T* air	PID Controller
			Algorithm

* (if set to be 0), by using the inverse PMV function G, an indoor temperature value T * air can be obtained. This value T *

  The basic SA algorithm maintains both a state which represents a solution to the optimization problem, and a temperature that will decrease from initially high level to zero during the execution of SA. At each iteration, the state is perturbed to produce a new solution. Both of the old and new states are evaluated by the objective function labelling as energy level. If the new state is no worse (less or equal energy) than the old one, it is selected. If not, it might be selected with a probability dependent upon both the current temperature and the energy difference between the two states. The cooling process is implemented in the SA algorithm as a decrease in the probability of accepting worse solutions as it explores the solution space. The SA algorithm is described in Algorithm 4

	Algorithm

4 SA algorithm Input initial state x 0 and temperature T Evaluate the energy of the state f (x 0 ) for each iteration i ∈ {1, 2, ..., n} do Perturb the state x 0 to get a new state x i Evaluate the energy of the new state f (x i )

Table 4 .

 4 2: Parameters of Inverse PMV Optimization TMRT • C vair m/s hr % Īcl clo M met Five experimental tests are conduced and Table 4.2 lists the typical values of the five parameters needed for PMV calculation. Since we consider estimating the mean radiant temperature with the same value of air temperature, there is no values for TMRT in the table. In each experimental test, the performances of seven meta-heuristic algorithms,

	Test 1	-	0.1	50	0.5	1.0
	Test 2	-	0.2	50	0.7	1.2
	Test 3	-	0.3	60	1.8	1.4
	Test 4	-	0.2	60	1.0	1.6
	Test 5	-	0.1	40	2.0	0.7

namely SA, PSO separately with a population of 5, 10, and 20 particles, and GA separately with a population of 5, 10, and 20 chromosomes, are compared. The parameters of PSO is listed in Table

4.3

. The GA and SA algorithms are already implemented in the Optimization Toolbox of Matlab R2013a, so we directly call these two algorithms. The parameters for them are just the default parameters in the Matlab Optimization Toolbox. within this limitation, this execution is regarded as being failed. (3) If the program has found the best solution within the maximum iteration number, it stops immediately. (

Table 5 .

 5

		2: Description of Subscripts
	w	external wall,
	wi	inside part of external wall,
	wo	outside part of external wall,
	f	floor,
	c	ceiling,
	ip	internal partition,
	amb ambient (outdoor) air,
	air	indoor air,
	v	ventilation,
	g	glazing

  .1 of the state of the art.This is an interesting model to describe thermal dynamics. Because on one hand this model captures the main factors that can affect room temperature. On the other side, this model is scalable. The thermal network above is used to represent thermal dynamics of a single room. If it needs to model thermal dynamics of multiple rooms, by connecting new components of thermal circuits to the single room circuit is able to fulfil this. An example of multiple room circuit is shown in Figure5.2. For the sake of convenience in simulation, we use the single room model in the experimental process. The physical component parameters of a smart home are listed in Table5.3, which can be obtained

from

[ASHRAE, 2005]

.

Table 5 .

 5 4: PMV Performance with Different Control Methods in Scenario 1

	Name Max	Min	Mean	Std	PMV in
						[-0.5, 0.5]
	HIC	0.6196 -0.7988 -0.0056 0.0335 99.96%
	PID	0.7002 -1.0195 -0.0186 0.2286 97.61%
	On/Off 0.9475 -1.1561 -0.0242 0.2071 96.91%
	FLC	1.1811 -0.9281 0.1221 0.2403 90.90%
	RL	1.8277 -0.9341 -0.1034 0.3314 87.42%

Table 5 .

 5 5: PMV Performance with Different Control Methods in Scenario 2

	Name Max	Min	Mean	Std	PMV in
						[-0.5, 0.5]
	HIC	0.6731 -0.7676 0.0024 0.0730 99.18%
	PID	0.7141 -1.1673 -0.0263 0.2553 96.01%
	On/Off 0.9431 -1.1158 0.0281 0.2040 97.06%
	FLC	1.3801 -0.6594 0.0984 0.2727 90.46%
	RL	2.1449 -1.0537 0.0156 0.4078 81.19%

  (t) working power of appliance j in household i at time t.

	t, T	index and set of time interval,
	i, N	index and set of household,
	j, S	index and set of electric appliance,
	I i j (t) binary variable denoting allowable working status of appliance j
		in household i at time t,
	U i j (t) dissatisfaction caused by operating appliance j
		in household i at time t,
	ρ(t)	electricity sale price at time t
	P i j
		T	S
			P i j (t)] 2	(7.4)
		t=1	j=1
		where

Algorithm 5

 5 Standard CPSO algorithm define: c( j, z) ≡ (s 1 .g best , s 2 .g best , ..., s j-1 .g best , z, s j+1 .g best , ..., s m .g best )Initialize each n particles of m swarms with random positions and zero velocities while any stopping criterion is not fulfilled do for each swarm j ∈ {1, 2, ..., m} do for each particle i ∈ {1, 2, ..., n} do if f (c( j, s j .x i )) < f (c( j, s j .p besti )) thens j .p besti = s j .x i end if if f (c( j, s j .p besti )) < f (c( j, s j .g best i )) then s j .g best i = s j .p besti end ifUpdate the particle velocity v i and position x i according to Equation Return the global best solution (s 1 .g best , s 2 .g best , ..., s m .g best ).

	end for
	end for
	end while

  Update the particle velocity v i and position x i according to EquationReturn the global best solution (s 1 .g best , s 2 .g best , ..., s m .g best ). With the increase of device and home number, the convergence of the algorithm will become a problem. Thus, a cooperative particle swarm optimization with particle reinitialization is proposed, which can solve the "two steps forward, one step back" problem of canonical PSO, decrease the possibility of the first stagnation problem and tackle the second stagnation problem. In order to verify the proposed system and algorithm, a case study will be introduced in the next chapter.

	7.4/ CONCLUSION
	This chapter has mathematically defined the objectives of each smart home: minimizing
	dissatisfaction, cost and power variance, and combined them to be a single composite
	objective function based on the aggregation approach. Because this composite objective function includes scheduling optimization of controllable devices in a smart home and coordination among a set of neighbouring smart homes, if we use the traditional PSO, a particle is a full dimensional vector that contains schedules of all controllable devices in all smart homes. SIMULATION AND ANALYSIS
	end for
	end for
	if mod(Index, N) == 0 then
	Reinitialize each n particles of m swarms with random positions and zero velocities
	end if
	end while

Algorithm 6 CPSO-R algorithm define: c( j, z) ≡ (s 1 .g best , s 2 .g best , ..., s j-1 .g best , z, s j+1 .g best , ..., s m .g best )

Initialize each n particles of m swarms with random positions and zero velocities Set the reinitialization interval N; Set Index = 0 while any stopping criterion is fulfilled do

Index = Index + 1

for each swarm j ∈ {1, 2, ..., m} do for each particle i ∈ {1, 2, ..., n} do if f (c( j, s j .x i )) < f (c( j, s j .p besti )) then

s j .p besti = s j .x i end if if f (c( j, s j .p besti )) < f (c( j,

s j .g best i )) then s j .g best i = s j .p besti end if Contents 8.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133 8.2 Models of Controllable Appliances . . . . . . . . . . . . . . . . . . . . 134 8.2.1 Power-Shiftable Devices . . . . . . . . . . . . . . . . . . . . . . . 134 8.2.2 Time-Shiftable Devices . . . . . . . . . . . . . . . . . . . . . . . 136 8.3 Case Study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137 8.3.1 Parameter Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . 137 8.3.2 CPSO-R Implementation within Smart Home . . . . . . . . . . . 138 8.3.3 Running Environment . . . . . . . . . . . . . . . . . . . . . . . . 140 8.4 Experimental Results and Analysis . . . . . . . . . . . . . . . . . . . 141 8.4.1 Basic Case . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141 8.4.2 Case without Coordination . . . . . . . . . . . . . . . . . . . . . 142 8.4.3 Case with Coordination . . . . . . . . . . . . . . . . . . . . . . . 143 8.5 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145

  • C/kW, C is the heat capacity of indoor air in kWh/ • C, τ = R • C, P sh (t) is the working power of the space heater at time t, and T out (t) is the outdoor temperature at time t.

Table 8 .

 8

		2: Comparison of The Three Cases
		mean cost mean power	std.
		(cent euro)	(kW)	(power on grid)
	Case 1	228.1	5.764	5.293
	Case 2	219.6	5.764	3.615
	Case 3	213.5	5.465	0.469

This chapter has presented a cyber-physical system with a hybrid intelligent method for thermal comfort improvement. The proposed cyber-physical system is based on the advanced networking, computing and sensor technologies, which makes the intelligent control of thermal comfort come true. The proposed hybrid intelligent method is composed of two modules. The inverse PMV optimization module adopts the particle swarm optimization with a population of 5 particles, the performance of which has been verified to be better than the other algorithms by experiments. The PID control module, which works asynchronously with the former module, has been turned out to be efficient for indoor temperature regulating by its widely industrial applications. In the next chapter, we will verify the proposed system and method by simulations and experiments.

ACKNOWLEDGMENTS

assistance in this work.

I also thank CSC -China Scholarship Council programme 1 . My research would not have been possible without its generous funding.

Algorithm 2 PSO algorithm

Initialize n particles with random positions and zero velocities while any stopping criterion are not fulfilled do for each particle i ∈ {1, 2, ..., n} do

Evaluate the fitness f (x i )

Update the local best position p besti and the global best position g best Update the particle velocity v i and position x i according to Equations 4.4 and 4.5 end for end while

Return the global best position g best as the solution.

4.4.2.2/ GENETIC ALGORITHM Genetic Algorithm (GA), which is a subset of Evolutionary Algorithm (EA), mimics the process of natural evolution first described by [START_REF] Darwin | The origin of species by means of natural selection[END_REF]. It contains features such as natural selection, survival of the fittest individual, recombination, and mutation, which were introduced by [Holland, 1975] and later developed by [Golberg, 1989] and [Michalewicz, 2013]. GA is widely used to search for global optimum of problems in various research fields, not only because its implementation is easy but also it does not impose any restrictions on the function to be optimized, like continuity, the existence of derivatives, or dimensionality.

The basic form of GA involves three types of operators: selection, crossover, and mutation:

Selection is used to pick chromosomes in the population for reproduction with the strategy that the fitter the chromosome the more times it is likely to be selected to reproduce.

Crossover, which mimics biological recombination between two single-chromosome or- The objective function of each household is to minimize the aggregation of the three subobjectives: dissatisfaction, electric cost, and total load variance. This objective can be achieved only when the schedule of each controllable device in the household is optimized. The dissatisfaction and the electric cost by using each controllable device can be evaluated by its corresponding agent after it has computed a power schedule as a power-shiftable agent or a scheduled starting time as a time-shiftable agent, while the