Acknowledgement

Research is indeed a social activity involved with discussions and exchanging ideas. This thought originates from the wonderful interactions that I want to acknowledge with the many contributors to this thesis.

As a PhD student, the initial research inspiration comes from supervisors. I am thrilled to have both Prof. Isabelle Chrisment and Dr. Abdelkader Lahmadi taking that role. Throughout my academic research journey, their guidance and professional insights have always been inspiring me.

Besides my supervisors, I would like to thank Dr. Mohamed Kaâniche and Dr. Stéphane Mocanu for being the reviewers of my thesis. Moreover, I want to thank the rest of my jury committee members: Prof. Ye-Qiong Song, Prof. Abdelmadjid Bouabdallah, and Dr. Ghita Mezzour. Likewise, I am grateful to Mme Souad Boutaguermouchet and Mme Sylvie Musilli who helped me with all the administrative tasks.

The sharing and discussion of ideas is an important aspect of research. In this sense, I am really appreciative of the wonderful visiting times spent both abroad and in France. I am thankful to Prof. Babak Nahid-Mobarakeh, Prof. Peng-Yong Kong, Dr. Lei Mo, and Dr. Qipeng Song for the great discussions we had in different academic occasions.

I would like to express my gratitude to all of my colleagues and co-authors for the excellent talks and discussions we had together. I owe a debt of gratitude to all of the promising ideas we have encountered and fought with... the ones that worked and the ones that did not, they have all been a valuable source of inspiration and knowledge. A special thanks to Pierre Marie Junges for all the fantastic ideas we exchanged and discussed, and to Thomas Lacour for all the kind help in the implementation of experimental testbeds.

I am very grateful for the wonderful atmosphere at LORIA. It is incredible how many interesting conversations and partnerships began during coffee breaks or on the way to lunch. Thank you to Prof. Ye-Qiong Song, who is also a Chinese working in LORIA, for all the kind help that adapts myself faster to the life in France and make me less homesick.

A special thanks to my former advisor and colleague in the Netherlands, Dr. André Teixeira, for helping me build my theoretical foundation of cyber-security research.

The work leading to this thesis has been funded by the French Government under grant FUI 23 PACLIDO (Protocoles et Algorithmes Cryptographiques Légers pour l'Internet Des Objets). All the related supports are greatly acknowledged.

Without the help of my family, none of this would have been possible. Special thanks to my parents Zhanping Ma and Xingying Liu and my brother Mingxu for their support and encouragement throughout my life. My dearest love to my wife, Liufang Chen, for her unflinching support and unselfish devotion.

Mingxiao Ma i "delay injection attack" on imbalanced dataset. . . . . . . . . . . . . . . . . . . . 5.6 ROC curves of the selected classifiers for multiclass classification with "normal", "MaR attack" and "delay injection attack" on imbalanced dataset. . . . . . . . . . 5.7 ROC curves of the selected classifiers for binary classification with "normal" and "MaR attack" on balanced dataset. . . . . . . . . . . . . . . . . . . . . . . . . . . 5.8 ROC curves of the selected classifiers for binary classification with "normal" and "delay injection attack" on balanced dataset. . . . . . . . . . . . . . . . . . . . . . 5.9 ROC curves of the selected classifiers for multiclass classification with "normal", "MaR attack" and "delay injection attack" on balanced dataset. . . . . . . . . . .

List of Figures

Un système de contrôle de micro-réseau avec des contrôleurs primaires et secondaires. Des réseaux de communication clairsemés sont utilisés pour l'échange de données entre les unités de production d'électricité voisines, par exemple pour transmettre des consignes de référence et des mesures (indiqués par l'exposant s).

Abstract

Modern low-voltage microgrid systems rely on distributed and cooperative control approaches to guarantee safe and reliable operational decisions of their inverter-based distributed generators (DGs). However, many sophisticated cyber-attacks can target these systems, deceive their traditional detection methods and cause a severe impact on the power infrastructure.

In this thesis, we systematically study the vulnerabilities and threats of distributed controlled microgrid systems. We design a novel attack named "measurement-as-reference" (MaR) attack and take it as a typical stealthy attack example to theoretically analyze the attack impact on the microgrid system and use numerical simulation results to verify the analysis. We provide mathematical models of possible false data injection (FDI) and denial of service (DoS) attacks in a representative distributed and cooperative controlled microgrid system. We propose a secure control framework with an attack detection module based on machine learning techniques. To validate the effectiveness of this framework, we implement two typical attacks, MaR attack and delay injection attack, on a hardware platform modeled after a microgrid system. We collect datasets from the platform and validate the performance of multiple categories of machine learning algorithms to detect such attacks. Our results show that tree-based classifiers (Decision Tree, Random Forest and AdaBoost) outperform other algorithms and achieve excellent performance in detecting normal behavior, delay injection and false data attacks.

Résumé

Les micro-réseaux électriques s'appuient sur des approches de contrôle distribuées et coopératives pour garantir des décisions opérationnelles sûres et fiables de leurs générateurs distribués (DG). Cependant, de nombreuses cyber-attaques sophistiquées peuvent viser ces systèmes, tromper leurs méthodes de détection traditionnelles et avoir des conséquences importantes sur l'infrastructure électrique.

Dans cette thèse, nous étudions les attaques ciblant les systèmes de contrôle associés à ces mircro-réseaux. Nous avons développé dans un premier temps une nouvelle attaque nommée

Cyber-Physical Security of Microgrids

New information and communication technologies (ICT) and various distributed generations (DGs), e.g., photovoltaic (PV) and wind turbine generators, are integrated into the power grid to build smart grids. They are essentially defined according to IEEE 2030-2011 standard, as a composition of three interoperability infrastructures [START_REF] Bushra Canaan | Microgrid cyber-security: Review and challenges toward resilience[END_REF], as outlined in Figure 1.1. This suggested interdependence has led to the problem of security becoming increasingly complex and impose additional challenges which could easily lead to damage to the fundamental security concerns as well as create new ones.

A new power generation paradigm where a mixed generation pool consists of conventional centralized plants and distributed generation units at lower voltage levels is emerging. The concept of microgrids is developed to tackle these challenges by expediting a local integration of renewable energy sources. Microgrids, composed of several DGs, energy storage elements, and controllable loads, are autonomous, and controllable small-scale power systems can operate in grid-connected and islanded modes [START_REF] Binod | Detection and mitigation of cyber-threats in the dc microgrid distributed control system[END_REF][START_REF] Teixeira | Voltage control for interconnected microgrids under adversarial actions[END_REF].

Microgrids utilize a hierarchical control structure, including primary, secondary, and tertiary control levels [START_REF] Binod | Detection and mitigation of cyber-threats in the dc microgrid distributed control system[END_REF][START_REF] Bidram | Distributed control systems for small-scale power networks: Using multiagent cooperative control theory[END_REF]. The primary control conventionally utilizes local droop controllers on distributed energy resources (DERs) to maintain microgrid voltage stability after the islanding. The secondary control of microgrid, which regulates the voltage to synchronize the average voltage of microgrid to the nominal setpoint, can be implemented through a centralized structure. In this structure, the microgrid secondary control center is responsible for determining and sending the voltage and power setpoints to the individual DERs through a complex two-way, high-bandwidth communication network [START_REF] Binod | Detection and mitigation of cyber-threats in the dc microgrid distributed control system[END_REF]. However, this control structure is vulnerable to the single-point-of-failure in the control center, which could reduce system reliability. Alternatively, distributed cooperative control of microgrid with a sparse communication network [START_REF] Binod | Detection and mitigation of cyber-threats in the dc microgrid distributed control system[END_REF][START_REF] Bidram | Distributed control systems for small-scale power networks: Using multiagent cooperative control theory[END_REF][START_REF] Ali Beg | Detection of false-data injection attacks in cyber-physical dc microgrids[END_REF] Figure 1.1: Smart grid architecture as a composition of three interoperability infrastructures in compliance with IEEE 2030 [START_REF] Bushra Canaan | Microgrid cyber-security: Review and challenges toward resilience[END_REF].

has been proposed recently to increase the reliability, scalability, and resilience of the microgrid control system.

These microgrids are evolving into cyber-physical systems (CPSs) with advanced softwarebased control and communication networks [START_REF] Ali Beg | Detection of false-data injection attacks in cyber-physical dc microgrids[END_REF]. These control techniques map DERs to nodes on a sparse communication digraph and ensure that all nodes reach an agreement on quantities of interests issued by the leader node (known as synchronization) by adopting the locally relevant information exchanged among neighboring DERs.

Such microgrids with a distributed nature are vulnerable to malicious cyber-attacks. There is no globally centralized entity to monitor activities of all DERs, and each DER only has access to its local data and partial data from neighboring nodes, resulting in a limited global situational awareness of cyber-attacks [START_REF] Ali Beg | Detection of false-data injection attacks in cyber-physical dc microgrids[END_REF]. Thus, it becomes more challenging to operate the power networks in a reliable and resilient mode in this new paradigm. To the best of our knowledge, the first known hacker-caused outage happened in 23 December 2015, hackers compromised information systems of three energy distribution companies in Ukraine and temporarily disrupted the electricity supply to consumers.

The awareness and concern over the security of CPS like microgrids have been increased. Modern SCADA systems have moved towards using standard communication technologies to enable access to remote devices and facilitate a smooth interface between devices from different vendors. Consequently, the number of possible attack points for malicious cyber agents to exploit has dramatically increased. Another common practice is the use of standard hardware and software platforms to decrease costs and improve flexibility. New vulnerabilities of these standard platforms may be discovered over their life-cycle, which significantly increases the risk of cyber threats to a large number of SCADA systems. In fact, the number of reported CPSrelated security incidents has significantly increased over recent years. As shown in Figure 1.2, total vulnerability instances amount of each CPS is mapped to the years from 2000 till 2019, considering common vulnerabilities and exposures (CVE) vulnerabilities since 1999 till now [START_REF] Jiang | Cyber-physical systems security based on a cross-linked and correlated vulnerability database[END_REF]. Compared to master terminal units (MTUs) and remote terminal unit devices (RTUs), more

Cyber-Physical Security of Microgrids

vulnerabilities of programmable logic controllers (PLCs) and HMI/SCADA have been disclosed, especially in the past 6 years [START_REF] Jiang | Cyber-physical systems security based on a cross-linked and correlated vulnerability database[END_REF]. Figure 1.2: CPS vulnerability totals trend by year [START_REF] Jiang | Cyber-physical systems security based on a cross-linked and correlated vulnerability database[END_REF].

Analogous to the situation in CPSs susceptible to various types of cyber-attacks, attacks on microgrid systems can be roughly grouped into two classes based on attack goals:: false data-injection (FDI) attacks and denial of service (DoS) attacks. FDI attacks, which is also named deception attacks in some literature [START_REF] Ding | security control and attack detection for industrial cyber-physical systems[END_REF], target the sensors and controllers and even the communication links, which in turn corrupt the transferred data and compromise the microgrid data integrity. DoS attacks prevent the controllers from receiving sensor measurements or the actuators from receiving control commands and thus compromise the availability of microgrid system services. In this thesis, both FDI and DoS attacks are of concern.

The cyber-attacks in microgrid control systems have been investigated in [START_REF] Teixeira | Voltage control for interconnected microgrids under adversarial actions[END_REF][START_REF] Ma | On the impact of synchronization attacks on distributed and cooperative control in microgrid systems[END_REF], where concrete DoS and FDI attack scenarios and impact analysis have been studied, but no attack detection and mitigation schemes are provided. To detect corrupted data in the Supervisory Control and Data Acquisition (SCADA) system of power grid, state vector estimation (SVE) is extensively used [START_REF] Liu | False data injection attacks against state estimation in electric power grids[END_REF][START_REF] Kosut | Malicious data attacks on the smart grid[END_REF]. The detector obtains the system state, which is estimated from the observed measurements and then computes the residual between the estimated and the observed measurements. If the residual is greater than the given threshold, a data injection attack is detected. However, the exact recovery of state vectors is a challenge for SVE-based methods in sparse networks [START_REF] Cotilla-Sanchez | Comparing the topological and electrical structure of the north american electric power infrastructure[END_REF][START_REF] Kosut | Malicious data attacks on the smart grid[END_REF], where the Jacobian measurement matrix is sparse, and the performance of this approach is restricted by the sparsity of the state vectors. Besides, if the false data injected vectors reside in the column space of the Jacobian measurement matrix and satisfy some sparsity conditions, then the false data injection attacks, called unobservable attacks, cannot be detected [START_REF] Liu | False data injection attacks against state estimation in electric power grids[END_REF][START_REF] Kosut | Malicious data attacks on the smart grid[END_REF]. Additionally, detection methods broadly applying state estimation processes, e.g., using Kalman filter [START_REF] Manandhar | Detection of faults and attacks including false data injection attack in smart grid using kalman filter[END_REF], sparse optimization [START_REF] Liu | Detecting false data injection attacks on power grid by sparse optimization[END_REF], graphical methods [START_REF] Bi | Graphical methods for defense against false-data injection attacks on power system state estimation[END_REF] to address FDI attack detection all require centralized communication structures. The FDI attack detection in distributed control systems deploying Kullback-Leibler (KL) divergence is addressed in [START_REF] Binod | Detection and mitigation of cyber-threats in the dc microgrid distributed control system[END_REF]. However, the above attack detection techniques only deal with a single type of cyber attack (FDI). A distributed state estimation method based on the alternating direction method of multipliers (ADMMs) is proposed to detect hybrid cyber-attacks (FDI and DoS attacks) in [START_REF] Du | Admm-based distributed state estimation of smart grid under data deception and denial of service attacks[END_REF]. However, this method mainly benefits the large-scale power transmission systems due to its heavy computation complexity.

The best practices and techniques from security are a sound first approach to increase the security and resilience of CPS. However, traditional IT security does not consider the interdependencies between the physical components and the cyber domain. A holistic approach is required to effectively handle the complex coupling between the physical process and the IT infrastructure.

Research Questions

This thesis addresses the problem of cybersecurity and resilience in the distributed control system of microgrids. Traditional cybersecurity does not consider the interdependencies between the physical components and the cyber systems. On the other hand, control theoretic approaches typically deal with independent disturbances and faults. Thus they are not tailored to handle cyber threats. Theory and tools to analyze and build control system resilience are, therefore, lacking and in need to be developed.

The recently coined cyber-physical system term integrates a physical infrastructure and a cyber framework to reduce the complexity and costs of traditional control systems in, e.g., industrial environments. Microgrid control systems are composed of sensors, actuators, and other field devices that interact with the physical processes. The technology evolution brings these systems towards a combination between a physical layer that encompasses the physical framework; and a cyber layer that encompasses the communication and computation framework. Figure 1.3 shows schematic and risk assessment concepts of a typical cyber-physical control system with a communication network that is vulnerable to adversaries.

The thesis focuses on the following groups of questions related to the distributed control system of microgrids:

• Reference Architecture. What core components should be considered in a reference architecture for cyber-secure and resilient microgrid control systems? What components could be vulnerable to cyber attacks?

• Modeling Framework. How can a malicious adversary be modeled from a controltheoretic perspective? What is the right level of modeling detail? What assumptions should be made to model the adversary considering system properties of microgrid control systems?

• Impact Assessment. What metrics could be used to assess and compare threats? What is the worst-case impact the threat could cause, and how to quantitatively measure the attack impacts? How can the metrics be used to prioritize attack scenarios and assess the effectiveness of defensive actions?

• Attack Detection and Response. What tools could be employed to detect the threats as effectively as possible? What tools and actions can be devised to increase the cybersecurity and resilience of microgrid control systems? Can such methods be implemented and used in real-life microgrids?

These questions are tackled throughout this thesis, contributing towards a framework to analyze, identify, and evaluate the consequences of vulnerabilities in microgrid control systems, as well as to propose and devise effective protection schemes.

Methodology and Contributions

In this thesis, we address security issues in microgrid control systems, which are typical cyberphysical systems. We focus on the protection between the cyber and physical layers of these systems by designing a secure framework, including an attack detection mechanism.

Threat modeling for complex systems is a four-stage process that begins with defining and diagramming the system, then identifying things that can go wrong, after which mitigation strategies can be developed. The entire process is validated to ensure accurate conclusions. In this thesis, we employ a methodology described in the Flowchart 1.4 to study the threat modeling and risk management problem of microgrid systems. We start with the security analysis of previous theoretical detection mechanisms work. We consider a more comprehensive control structure for a microgrid system, mainly a hierarchical control approach consisting of three control levels: primary, secondary and tertiary control [START_REF] Bidram | Distributed control systems for small-scale power networks: Using multiagent cooperative control theory[END_REF]. Specifically, we design a distributed cooperative hierarchical control system consisting of droop control as primary control and cooperative control as secondary control. We believe this control scheme is more realistic and reliable than the control architecture considered in [START_REF] Teixeira | Voltage control for interconnected microgrids under adversarial actions[END_REF] and [START_REF] Ma | Voltage control in distributed generation under measurement falsification attacks[END_REF], because of its better support for scalable integration of local loads and high integration of distributed generation.

Based on the above control model, we consider cyber attacks that may target the communication links between DGs, and we use risk assessment methods to quantitively analyze the impact of these attacks on voltage deviation at the primary control level and reference voltage synchronization at the secondary control level. Our impact analysis results are valuable for microgrid system designers to help them evaluate potential cyber threats targeting these systems.

We propose a resilient distributed cooperative control for microgrids that accommodates FDI and DoS attack detection. Compared to the existing attack detection methods based on state estimation [START_REF] Liu | False data injection attacks against state estimation in electric power grids[END_REF][START_REF] Kosut | Malicious data attacks on the smart grid[END_REF], statistical machine learning algorithms outperform them in detecting both observable and unobservable attacks [START_REF] Ozay | Machine learning methods for attack detection in the smart grid[END_REF]. The attack detection problem can be modeled as a typical classification problem to learn communication patterns at the network-level of the distributed control system and predict the class of a packet as attack or normal.

To validate the performance of machine learning and deep learning based attack detection, we give a detailed hyper-parameter tuning methodology to get the best hyper-parameters used for classification. We collect datasets from the platform and compare the performances of different machine learning and deep learning algorithms to detect measurement-as-reference (MaR) and delay injection attacks. We give a detailed analysis for the three classification problems on balanced and imbalanced dataset, respectively.

Thesis Outline

The thesis outline is as follows.

1) In Chapter 2, we review some of the main concepts of the microgrid systems, including the control structure and the communication networks used throughout the thesis. We present a brief summary of recent developments in the area of microgrid control systems, which are described and modeled in more detail and the cyber-physical security problems are reviewed, and various cyber-physical attacks grouped in different categories are summarized. At last, we give a comprehensive summary of the attack detection schemes of recent related work on cyber-secure control systems. The Chapter concludes with a brief description of the main applications and experimental setups considered in the thesis.

2) In Chapter 3, we consider a more comprehensive control structure for a microgrid system, mainly a hierarchical control approach consisting of three control levels: primary, secondary and tertiary control [START_REF] Bidram | Distributed control systems for small-scale power networks: Using multiagent cooperative control theory[END_REF]. Specifically, we design a distributed cooperative hierarchical control system consisting of droop control as primary control and cooperative control as secondary control. We believe this control scheme is more realistic and reliable than the control architecture considered in [START_REF] Teixeira | Voltage control for interconnected microgrids under adversarial actions[END_REF] and [START_REF] Ma | Voltage control in distributed generation under measurement falsification attacks[END_REF], because of its better support for scalable integration of local loads and high integration of distributed generation. Based on the above control model, we consider cyber attacks that may target the communication links between DGs, and we use risk assessment methods to quantitively analyze the impact of these attacks on voltage deviation at the primary control level and reference voltage synchronization at the secondary control level. Our impact analysis results are valuable for microgrid system designers to help them evaluate potential cyber threats targeting these systems.

3) In Chapter 4, we model and generalize both FDI and DoS attacks in distributed and cooperative control microgrid systems. We also give a general system stability condition analysis under FDI and DoS attacks. We propose a secure control framework with attack detection and response modules relying on machine learning and deep learning algorithms. Various groups of supervised machine learning and deep learning algorithms are studied to design the attack detection module against FDI attacks and DoS attacks under well-chosen performance metrics. Considering it is difficult to find publicly available datasets, we build an experimental platform emulating the distributed and cooperative controlled microgrid systems, and implement two typical FDI and DoS attacks hard to be detected by traditional residual comparison detectors. The two attacks are: measurement as reference (MaR) attack as typical FDI attack and delay injection attack as typical DoS attack. Balanced and imbalanced datasets are collected to under normal and attacked conditions to be further analyzed.

4) in Chapter 5, we validate the inefficiency of traditional threshold comparison method in detecting MaR and delay injection attacks. We describe how we tune the machine learning and deep learning hyper-parameters and list the best parameters of each classifier that we use for classification. At last, we show the experimental results and performance analysis for collected imbalanced and balanced dataset, respectively.

5) in Chapter 6, we give a general conclusion about this thesis and talk about the future work.

Publications

Early versions of the results covered in this thesis have been successfully reported in the following peer-reviewed publications.

Introduction

As a controlled small-scale power system, a microgrid can be operated in both islanded and grid-connected mode in a specified area in order to facilitate the provision of supplementary power and maintain a standard service [START_REF] Espina | Distributed control strategies for microgrids: An overview[END_REF][START_REF] Md Alamgir Hossain | Evolution of microgrids with converter-interfaced generations: Challenges and opportunities[END_REF]. It usually consists of small-scale distributed generators, energy storage elements, local loads, and controllers. Because of their advantages in reliability, flexibility, scalability and efficiency, microgrids are becoming increasingly popular in power systems. Microgrids can use alternate current (AC) or direct current (DC) energy or even AC/DC hybrid topologies, in which AC and DC power supplies and loads can both be considered [START_REF] Espina | Distributed control strategies for microgrids: An overview[END_REF].

Resilience is the potential to sustain sufficient operating levels in the face of unusual conditions [START_REF] Teixeira | Toward cyber-secure and resilient networked control systems[END_REF]. The microgrid infrastructure must have resiliency in relation to disturbances and faults, by developing stable and secure frameworks as a typical cyber-physical system. Not only should such a framework be fault-tolerant, but it should also secure the system from malicious adversaries. While the security of IT systems is commonly known in the literature, the same is uncertain in the case of microgrid control systems.

In this Chapter, we review some of the main concepts of the microgrid systems, including the control structure and the communication networks used throughout the thesis. Firstly, we present a brief summary of recent developments in the area of microgrid control systems, which are described and modeled in more detail in Chapter 3. Secondly, the cyber-physical security problems are reviewed, and various cyber-physical attacks grouped in different categories are summarized. At last, we give a comprehensive summary of the attack detection schemes of recent related work on cyber-secure control systems. This Chapter concludes with a brief description of the main applications and experimental setups considered in the thesis.

Microgrid Control Systems

Overview

Microgrids are designed to run in both grid-connected and islanded working modes, depending on whether or not they are connected to the main grid. The microgrid control system is responsible for regulating voltage and frequency in various operating modes and ensuring a proper load sharing among DGs, controlling the power flow between the main grid and the microgrid and minimizing the operations costs.

In current power networks, the security and reliability of operations are known as crucial problems. Modern information and communication technology (ICT) and distributed generation technologies, including photovoltaic (PV) and wind turbines, are introduced into the power grid. Thus there is a modern trend of power production, in which a hybrid generation pool is used, including of traditional clustered plants and distributed generator (DG) units at lower voltage levels. In this new paradigm, though, it becomes more challenging to run the power grids in a stable and robust manner. To overcome these problems, the idea of microgrid is built by the rapid local deployment of renewable sources. Autonomous, small-scale controllable power systems may operate in grid-connected and islanded modes, consisting of a variety of DGs, energy stock elements and controllable loads [START_REF] Binod | Detection and mitigation of cyber-threats in the dc microgrid distributed control system[END_REF][START_REF] Teixeira | Voltage control for interconnected microgrids under adversarial actions[END_REF].

Three critical roles are typically implemented through microgrid control systems: (i) control of distributed generators, often known as distributed energy resources (DER), (ii) energy management, and (iii) microgrid protection. Although microgrids can be run at medium voltage levels, there is no doubt that the microgrid is most widely used in low-voltage delivery systems. In Figure 2.1 the hierarchical control structure is typically used to control and handle the AC-microgrid, DC-microgrid or even a hybrid-microgrid.

In a hierarchical framework, the control mechanism is split down into three major layers: central, secondary and tertiary control loops (see Figure 2.1). The droop control system is generally used for the primary control loop, simulating a synchronous machine's actions. The secondary control loop aims at repairing the secondary (e.g., voltage and frequency) variables to their nominal values. Finally, the energy management system (EMS) is commonly used for maximum operation and congestion management in a micro-microgrid in the third-level control loop. In order to control current and voltage in the input/output of a power converter mounted on the microgrid, an internal (inner) control loop is typically used at the lower stage. The above internal control is generally named zero control level. They normally use resonant controllers, controllers implemented in the synchronous rotating d-q framework, predictive controllers for realization.

Figure 2.1: The hierarchical control structure of a microgrid consisting of primary, secondary and tertiary control levels as specified in [START_REF] Bidram | Distributed control systems for small-scale power networks: Using multiagent cooperative control theory[END_REF].

As for each control layer's implementation, three application methodologies may be implemented for each of the microgrid topologies discussed in this thesis. These topologies involve centralized, distributed and decentralized designs. Below [START_REF] Espina | Distributed control strategies for microgrids: An overview[END_REF] is a short overview of each implementation methodology:

1) Centralized Control. In this case, the microgrid requires a central controller to interact with the DERs in the microgrid. All information transmitted from the other components in the microgrid must be processable by the central controller. Centralized controllers are not very robust and will be addressed further in the following section.

2) Distributed Control. In this scenario, centralized control is not needed since the control effort is distributed and unified among all the microgrid modules. The autonomous agents cooperatively attempt to obtain the global goals. Distributed control systems increase the microgrid's scalability and provided that fault-tolerant mechanisms are implemented.

3) Decentralized Control. In this situation, only local measurements are used for the control scheme of a DER unit (agent). The methodologies for control are generally based on "voltage-reactive power" (V -Q) and "frequency-active power" (f -P ) droops, which subtract proportional parts of the output average reactive and active powers to the voltage amplitude and frequency of each DG unit. The microgrid load between the distributed generation (DG) units is exchanged through a physical connection by using droop controllers, according to their power capabilities. Notice that there are no communication networks in this approach, and this undoubtedly makes it impossible to implement secondary and tertiary control loops. However, in the literature such methods have been suggested, such as the use of high-pass "wash-out" filters [START_REF] Espina | Distributed control strategies for microgrids: An overview[END_REF].

Generally, secondary and tertiary control layers of the microgrid can be implemented utilizing centralized topologies, and it is cost-efficient to do so given that they have to measure all DERs throughout the microgrid to reach the control target. However, these control layers can be applied with greater efficiency and protection by the new advances in distributed control theory. In addition, the distributed architecture of communication is sparse and dependent on local controls, which provides an essential factor in DER control and collaboration during operations of the microgrid in order to achieve global objectives. The critical value of distributed control systems can be outlined as follows for microgrid applications: 1) Robustness. If a single feature ceases operating well or if there is any fault, none of the overall functions of the microgrid can be affected. On the other hand, the centralized control center appears to be vulnerable to common errors and have significant teamwork concerns.

2) Scalability. This is a control method that is flexible to use. Changes to the microgrid, such as introducing more DERs, battery energy system (BESS), and more loads, are much simpler to incorporate, without disrupting the operation of other microgrid components.

3) Flexibility. It enables the DERs to operate plug-and-play functions. This is an enticing microgrid feature.

In the mid of 2000s, the implementation of microgrids coupling with distributed control algorithms was initially suggested [START_REF] Dimeas | A multi-agent system for microgrids[END_REF], [START_REF] Prodanovic | High-quality power generation through distributed control of a power park microgrid[END_REF]. The potential implementation of agent control for the energy market was addressed in [START_REF] Dimeas | A multi-agent system for microgrids[END_REF]. In [START_REF] Prodanovic | High-quality power generation through distributed control of a power park microgrid[END_REF], the authors designed a strategy that integrates the activities of a centralized controller and a distributed local controller. Since then, discussions have been conducted about what kind of control scheme (centralized or distributed) is more effective for microgrid applications. A detailed comparative description of characteristics of the centralized and distributed control topologies is given in [START_REF] Espina | Distributed control strategies for microgrids: An overview[END_REF]. We further conclude that the distributed control topology has the following advantages over the centralized control topology:

1. Lower communication needs. Microgrids with the centralized control scheme continuously monitor a multitude of involved DERs that are vulnerable to communication delays or packet losses. Local DER controllers in the distributed control system share data only with neighbors that are resilient in response to communication delays or packet losses and relax strict communication requirements.

2. Lower computational burden. Microgrids in a centralized control scheme process data from a variety of contributing distributed energy resources, which results in substantial computational burdens on microgrids. Parallel data from neighbors will be handled by local DER controllers which engage in the distributed control system based on consensus protocols with a substantial reduction of the computational burdens.

3. Better scalability. The system scale of the centralized control system is constrained by the communicational and computational capacities of the microgrid, the communication topology, and the usable bandwidth. DERs involved in the distributed control framework only communicate through the sparse communication networks with their neighbors, with plug-and-play characteristics and increased scalability for possible modifications and expansions.

Better reliability.

When a communication connection fails in the system, the centralized control structure is highly vulnerable and unable to operate correctly. The whole power network may be shut down by the maintenance and improvement of the centralized control system. Thanks to the redundancy of the distributed control system, it can withstand a communication link failure due to the presence of alternate routes. The distributed control system could be maintained and upgraded without the whole system being shut down.

5. Better resilience. The centralized control device implemented in the star communication topology has minimal controller redundancy and is prone to single-point failures, which can cause the system to crash. The distributed control system implemented in the meshed communication network has improved controller redundancy to maintain robustness against single-point failures.

Provided that different fault tolerance mechanisms are implemented, it can be concluded that distributed techniques have the following benefits compared with centralized methods: they improve robustness and reliability, allow for flexibility and scalability, including operations with plug-and-play functions.

Hierarchical Control Layers of Microgrid Systems

For the hierarchy control structure shown in Figure 2.1, each control level has its own control goals because of operating in different timescales. The primary control operates on a fast timescale, and is responsible for the control of transients to stabilize the voltage and frequency of the microgrid during changing of load or generation, or subsequent to an islanding event. Secondary control is designed to compensate for voltage and frequency deviations caused by primary control in terms of fault conditions. Finally, tertiary control, as the highest and slowest control level, optimizes the operations in both operating modes and manages to control the power flow between the main grid and the microgrid.

Primary Control

The primary control is locally implemented at each DG, e.g., droop control for inverter-based distributed energy resources (DER). However, the secondary control usually relies on a centralized control structure. Central controllers are designed to issue global commands requiring information gathered from the whole system and thus a complex two-way communication network is also needed. This kind of communication network makes it vulnerable and may effect the system reliability. In [START_REF] Bidram | Distributed control systems for small-scale power networks: Using multiagent cooperative control theory[END_REF], the authors replace the existing standard centralized secondary control with an efficient distributed control structure. They consider the microgrid as a multi-agent system where each DG is an agent. A voltage source inverter (VSI) is employed to connect a DG to the microgrid. The VSIs are interconnected through the physical power network configuration.

The primary control conventionally utilizes local droop controllers on distributed energy resources (DERs) to maintain microgrid voltage stability after the islanding. Primary control responds quickly and is designed to stabilize the voltage and frequency of the microgrid. Subsequent to an islanding event, the microgrid may lose its voltage and frequency stability due to the mismatch between the power generated and consumed. Primary control offers the plug-and-play capability for DGs and properly shares the active and reactive power among them, preferably without any communication links. This control level must mitigate circulating currents that can cause the over-current phenomenon in the power electronic devices and damage the dc-link capacitor.

Primary control ensures the stability of the DG but may cause frequency and voltage deviations. Secondary control restores the microgrid voltage and frequency to their desired reference values ω ref and v ref and compensates for deviations caused by the primary control. In islanded mode, ω ref is set to the nominal angular frequency, and v ref is set to the nominal voltage of the microgrid. In the grid-connected mode, ω ref and v ref are dictated by the tertiary controller. The secondary control hierarchy has a lower dynamic response than that of the primary control. This timescale separation justifies considering the dynamics of the primary and secondary control loops to be decoupled and facilitates their individual designs [START_REF] Bidram | Distributed control systems for small-scale power networks: Using multiagent cooperative control theory[END_REF].

Primary control responds quickly and is designed to stabilize the voltage and frequency of the microgrid. After an islanding event, the microgrid may lose its voltage and frequency stability due to the mismatch between the power generated and consumed. Primary control offers the plug-and-play capability for DGs and appropriately shares the active and reactive power among them, preferably without any communication links. This control level must mitigate circulating currents that can cause an over-current phenomenon in the power electronic devices and damage the dc-link capacitor.

Primary control provides the reference points for the voltage and current control loops of the DGs. These inner control loops are commonly referred to as zero-level control, which is generally implemented in either active-reactive power (PQ) or voltage control modes. In the PQ control mode, the DG active and reactive powers are regulated on predetermined reference points. The control strategy is implemented with a current-controlled VSI (CCVSI). In the voltage control mode, the DG operates as a voltage-controlled VSI (VCVSI), where the reference voltage is determined by the primary control, conventionally via droop characteristics.

The droop control is an autonomous control that operates at each DG and does not require intercommunication links between the DGs. The conventional active power control (frequency droop characteristic) and reactive power control (voltage droop characteristic) are used for frequency and voltage control. The principles of the conventional droop techniques are discussed next, using an equivalent circuit model of a VCVSI connected to an ac bus. Neglecting the switching ripples and high-frequency harmonics, the VCVSI is modeled as an ac source, with voltage E ∠δ. It is assumed that the DG is connected to a microgrid bus with the voltage V com ∠0, and the DG output impedance and the line impedance are lumped as a single effective line impedance of Z ∠θ. The complex power delivered to the microgrid bus is calculated as

S = V com I * = V com E ∠θ -δ Z - V 2 com ∠θ Z (2.1)
from which the real and reactive powers are achieved, respectively, as

P = VcomE Z cos(θ -δ) -V 2 com Z cos(θ), Q = VcomE Z sin(θ -δ) -V 2 com Z sin(θ). (2.2)
If the effective line impedance, Z ∠θ is assumed to be purely inductive, θ = 90 • , then (2.2) can be reduced to

P = VcomE Z sin δ, Q = VcomE cos δ-V 2 com Z (2.3)
If the phase difference between the converter output voltage and the common ac bus, δ, is small enough, then, sin δ ≈ δ. and cos δ ≈ 1. Thus, the frequency and voltage droop characteristics can be used to fine-tune the voltage reference of the VCVSI, based on the droop control equations

ω = ω * -D P P , E = E * -D Q Q (2.4)
where E * and ω * are the primary control references. The droop coefficients, D P and D Q , can be adjusted either heuristically or by tuning algorithms (for example, particle swarm optimization).

In the former approach, D P and D Q are determined based on the converter power rating and the maximum allowable voltage and frequency deviations. For instance, in a microgrid with N DGs, the constraints

D P 1 P n1 = D P 2 P n2 = • • • = D PN P nN = ∆ω max D Q1 Q n1 = D Q2 Q n2 = • • • = D QN Q nN = ∆E max (2.5)
must be satisfied, where ω max and E max are the maximum allowable angular frequency and voltage deviations, respectively, and P ni and Q ni are the nominal active and reactive power of the ith DG. The droop equations (2.4) are a feedback control scheme giving the required frequency ω and voltage E of the DG depending on the current power conditions P and Q observed by DG and the prescribed primary control reference frequency ω * and voltage E * . The primary control reference frequency ω * and voltage E * are set by the secondary control level, as discussed below.

During the grid-tied operation of the microgrid, the DG voltage and angular frequency, E and ω, are enforced by the grid. The DG output active and reactive power references, P ref and Q ref can hence be adjusted through E * and ω * as

P ref = ω * -ω D P Q ref = E * -E D Q (2.6)
The conventional droop method can be implemented with no communication links, which improves reliability. However, this method has some drawbacks:

1) The conventional droop method is developed assuming highly inductive effective impedance between the VCVSI and the ac bus. However, this assumption is challenged in microgrid applications since low-voltage transmission lines are mainly resistive.

2) As opposed to the frequency, the voltage is not a global quantity in the microgrid. Thus, the reactive power control in (2.4) may adversely affect the voltage regulation for critical loads.

3) For nonlinear loads, the conventional droop method is unable to distinguish the load current harmonics from the circulating current. Moreover, the current harmonics distort the DG output voltage. The conventional droop method can be modified to reduce the total harmonic distortion of the output voltages.

These potential drawbacks are widely discussed in the literature. A comprehensive survey on primary control schemes that correct these issues is provided in [START_REF] Bidram | Hierarchical structure of microgrids control system[END_REF].

Secondary Control

Primary control ensures the stability of the DG but may cause frequency and voltage deviations. Secondary control restores the microgrid voltage and frequency to their desired reference values ω ref and v ref and compensates for deviations caused by the primary control. In islanded mode, ω ref is set to the nominal angular frequency, that is, 2π × 50 or 2π × 60 rad/s, and v ref is set to the nominal voltage of microgrid, v nominal . In the grid-connected mode, ω ref and v ref are dictated by the tertiary controller. Secondary control hierarchy has a lower dynamic response than that of the primary control. This timescale separation justifies considering the dynamics of the primary and secondary control loops to be decoupled and facilitates their individual designs. The frequency of the microgrid and the terminal voltage of a given bus are compared with the corresponding reference values, ω ref and v ref , respectively. Then the error signals are processed by individual controllers according to

δω = K P ω (ω ref -ω) + K I ω (ω ref -ω) dt + ∆ω s , δE = K PE (v ref -E ) + K IE (v ref -E ) dt, (2.7)
where K P ω , K I ω , K PE and K IE are the secondary controller parameters. The resulting signals δω and δE are sent to the primary controller as changes to ω * and E * in the droop control (2.4) of the DG primary controller to compensate for the frequency and voltage deviations [START_REF] Bidram | Hierarchical structure of microgrids control system[END_REF].

An additional term, ∆ω s , is considered in the frequency controller in (2.7) to facilitate synchronization of the microgrid to the main grid. In the islanded operating mode, this additional term is zero. However, during synchronization, a phase-locked loop (PLL) module is required to measure ∆ω s [START_REF] Bidram | Hierarchical structure of microgrids control system[END_REF]. During grid-tied operation, the voltage and frequency of the main grid [v ref and ω ref , in (2.7)] are considered as the references.

This standard secondary control (2.4) is implemented by a centralized node with access to all the required information in the microgrid. This implementation requires an extensive communication network, generally with two-way communication links. As such, this method is expensive, unreliable, and vulnerable to a single point of failure. This thesis describes how to replace (2.4) by distributed control structures using cooperative control techniques, which rely on sparse, inexpensive communication networks.

The secondary control of microgrid, which regulates the voltage to synchronize the average voltage of microgrid to the nominal setpoint, can be implemented through a centralized structure. In this structure, the microgrid secondary control center is responsible for determining and sending the voltage and power setpoints to the individual DERs through a complex two-way, high-bandwidth communication network [START_REF] Binod | Detection and mitigation of cyber-threats in the dc microgrid distributed control system[END_REF]. However, this control structure is vulnerable to the single-point-of-failure in the control center, which could reduce system reliability. Alternatively, distributed cooperative control of microgrid with a sparse communication network [START_REF] Binod | Detection and mitigation of cyber-threats in the dc microgrid distributed control system[END_REF][START_REF] Bidram | Distributed control systems for small-scale power networks: Using multiagent cooperative control theory[END_REF][START_REF] Ali Beg | Detection of false-data injection attacks in cyber-physical dc microgrids[END_REF] has been proposed recently to increase the reliability, scalability, and resilience of the microgrid control system.

Tertiary Control

Tertiary control is the highest (and the slowest) control level and is meant to optimize the economic performance of the microgrid and manage the power flow between microgrid and main grid [START_REF] Bidram | Hierarchical structure of microgrids control system[END_REF]. In the grid-tied mode, the power flow between microgrid and main grid can be managed by adjusting the amplitude and frequency of the DG voltages. The active and reactive output powers of the microgrid, P G and Q G , are compared with the corresponding reference values,

P ref G and Q ref G ,
   ω ref = K PP P ref G -P G + K IP P ref G -P G dt, v ref = K PQ Q ref G -Q G + K IQ Q ref G -Q G dt, (2.8)
where K PP , K I ω , K PE and K IE are the controller parameters. The values ω ref and v ref are further used as the reference values for the secondary control, as in (2.7).

Communication Networks of Microgrid Systems

Preliminaries of Graph Theory for Microgrid Communication Networks

The communication network of a multiagent cooperative system can be modeled by a directed graph (digraph). A generic network topology of a microgrid system is usually characterized by a directed graph (digraph) G = (V, E, A G ) with a nonempty finite set of N nodes V = {v 1 , v 2 , . . . , v N }, a set of edges E ⊆ V × V, and the associated adjacency matrix A G . The set N i = {j ∈ V : (i, j ) ∈ E} denotes the neighbor set of node i. In a microgrid, DGs are the nodes of the communication digraph. The edges of the corresponding digraph of the communication network represent the communication links. The weight is represented by a ij for the edge from node j to node i, and a ij = 0 if there is no data transfer from node j to node i. Considering N nodes on the graph, the adjacency matrix is defined as

A G = [a ij ] ∈ R N ×N .
(2.9)

We define the diagonal in-degree matrix D = diag {d i } ∈ R N ×N , where d i is the sum of communication weights from neighbors of node.

d i = j ∈N i a ij .
(2.10)

The Laplacian matrix of the graph is defined as L = D -A G . One property of the Laplacian matrix L is that the row sums of L are all zero, because the row sums of D and A G are equal.

A directed path from node i to node j is a sequence of edges depicted as {(v i , v k ), (v k , v l ), . . . , (v m , v j )}. A graph is considered to have a spanning tree if there exists a root node with a directed path from that node to every other node in this graph. A graph is strongly connected if there exists a directed path between every two nodes, i.e., there exists a spanning tree where every node is a root node [START_REF] Bidram | Distributed control systems for small-scale power networks: Using multiagent cooperative control theory[END_REF] [START_REF] Moayedi | Distributed tertiary control of dc microgrid clusters[END_REF].

Sparse Communication Topology to Implement Distributed Control

To implement the distributed control for microgrid systems, sparse communication topology is employed [START_REF] Bidram | Distributed control systems for small-scale power networks: Using multiagent cooperative control theory[END_REF][START_REF] Olfati | Consensus problems in networks of agents with switching topology and time-delays[END_REF][START_REF] Ren | Consensus seeking in multiagent systems under dynamically changing interaction topologies[END_REF]. The controllers designed in this thesis must be supported by a local communication network. This network can be implemented by a TCP/IP communication protocol. The sampling time of the secondary control can be chosen to be much larger than the sampling time of the communication network. Communication links contain an intrinsic delay (for example, around 1 ms in the work [START_REF] Yang | Communication infrastructures for distributed control of power distribution networks[END_REF]). Since the time scale of the secondary control is large enough, the communication link delays can be safely assumed to be zero. The software and hardware infrastructures required to implement the discussed distributed control protocols are quite modest. Voltage source inverters can be accompanied by commercial digital signal processors (DSPs) to implement internal voltage and current control loops [START_REF] Bidram | Distributed control systems for small-scale power networks: Using multiagent cooperative control theory[END_REF]. The distributed control protocols embedded on each DG would not require dedicated signal processing units and do not impose a heavy processing burden on the existing DSP. The distributed adaptive controllers can be deployed on the existing processors, with a slight software update to pre-existing codes.

The distributed controllers presented here can admit both time-invariant and time-varying communication networks. For time-invariant communication networks, the adjacency matrix A G is fixed. The communication topology must be a graph containing a spanning tree in which the controller of each DG only requires information about that DG and its immediate neighbors on the communication graph. Therefore, the communication requirements to implement the controller with a time-invariant communication network are rather mild. However, the controller could be susceptible to communication link failures or data loss. In time-varying communication structures, each DG can send its own information to the neighboring DGs intermittently. The communication network is piecewise constant; that is, the adjacency matrix A G changes intermittently at prespecified time instants. Therefore, the secondary control is robust against data loss and communication link failures [START_REF] Olfati | Consensus problems in networks of agents with switching topology and time-delays[END_REF][START_REF] Ren | Consensus seeking in multiagent systems under dynamically changing interaction topologies[END_REF][START_REF] Huanhai Xin | A self-organizing strategy for power flow control of photovoltaic generators in a distribution network[END_REF]. According to [START_REF] Huanhai Xin | A self-organizing strategy for power flow control of photovoltaic generators in a distribution network[END_REF], a time-varying communication network can provide synchronization if the sequential completeness condition is satisfied. The sequential completeness condition means that, given an infinite sequence of finite intervals, the resulting graph over each finite interval must contain a spanning tree. In the following chapters of this thesis, we study time-invariant communication networks for the sake of simplicity.

Communication Types for Microgrid Control

Architecture for the operation and control of microgrids requires an encrypted and reliable channel for successful operation. Encrypted communication across the microelements is the desired task. However, wired communication helps in short-distance data transmission; RS 232 and RS 385 are the most used ones, but with the increase in the size of the grid, wired data transmission is not advisable as it increases the complexity of the grid. So, opting for internet communication protocol based on ISO or open system interconnection, transmission control protocol/internet protocol, and UDP/IP protocol ensures security and fast transmission of the data. Table 2.1 summarizes the various communication links available for microgrids concerning coverage distance [START_REF] Yadav | Microgrid control, storage, and communication strategies to enhance resiliency for survival of critical load[END_REF][START_REF] Dondi | Network integration of distributed power generation[END_REF][START_REF] Singh | Effect of load models in distributed generation planning[END_REF][START_REF] Teng | Value-based distributed generator placements for service quality improvements[END_REF][START_REF] Zahedi | A review of drivers, benefits, and challenges in integrating renewable energy sources into electricity grid[END_REF].

Besides, the microgrid structure has been categorized based on the range of communication into home area network (HAN), local area network (LAN), and wide-area network (WAN). Microgrid control architecture has various power system devices embedded with Intelligent Electronic Devices (IEDs). IED receives the signals from the DER and transmits them to the microgrid controller. Then, the microgrid controller generates the reference and control signals of voltage, current, frequency, real and reactive power to IEDs, which send the control signal to DERs and the connected loads. The human-machine interface is used for monitoring, event recording, and controlling purposes.

The microgrid communication is affected by various parameters such as speed, topology, and congestion status of the network and computational capability of IEDs hardware. The second transmission method is the Power Line Carrier (PLC), in which power lines are used for the transfer of the data. It is efficient in terms of performance and cost. Some of the disadvantages of using PLC are noise, deviation, and signal attenuation risk.

Every component of the microgrid has some delay requirements. The efficiency of the system can be increased by decreasing the bottleneck, congestion, loss of data packets, etc.; to achieve the desired efficiency, selection of protocols is the primary step. If the system exceeds the specified delay range, it leads to the electric malfunctioning in the microgrid. In this regard, the Quality of Service (QoS) concept is provided, in which priority treatment is given for the system having critical delay so that information reached on proper time and reduced the loss of data. In order to achieve it, the components are layered based on the requirement as follows [START_REF] Yadav | Microgrid control, storage, and communication strategies to enhance resiliency for survival of critical load[END_REF].

1) Physical layer. This layer consists of sensors, actuators, smart plugs, smart meters, mi- crocontrollers and storages. Based on the input data given by sensors, actuators perform the desired task, smart meter monitors the data given by sensors and controls the smart plug for transferring the power from the grid elements to the appliances. The interaction between the sensors, actuators, smart meter and plug with the computer is achieved using microcontrollers, and storage units are used to store the processed data received from the sensors.

2) Network layer. It is used to transfer the data from the physical layer to the application layer. These layers can be either wired or wireless. Wired includes cabled or fiber optics, use of fiber optics prevents intervention by electromagnetic radiation. The wireless mode includes 3G, LTE, WiFi. The only disadvantage of the former concerning the latter is the loss of data; data loss might account for up to 30% of the total data.

3) Cloud layer. The encryption of the data, retrieval of the data, and data management occur in this layer. The necessity of this layer arouses due to an increase in the amount of the data, loss of the data, and invasion of the privacy of the data.

4) Application layer. It is the final layer, which the end-users see and use. This layer manages the consumption of energy and the pricing of the system. The functioning of this layer entirely depends upon the other three layers. Over the years, there has been significant development in wireless networks. Internet of Things (IoT) is one such element, which has been turning around most of the theoretical applications to possibilities of achieving. One such application where IoT can be used is communication between various components across microgrids. Using IoT, the most critical microgrid factors, namely, the efficiency of power generation and distribution is achieved. Since the components of a microgrid are mostly non-homogeneous, various protocols have been used to achieve the desired requirement.

To communicate between the layers of the communication channel, Internet Protocol base or non-Internet Protocols base is applied. Internet-based protocols include Constrained Application Protocol (CoAP), Message Queuing Telemetry Transport (MQTT), 3G, LTE, Wi-Fi, whereas non-Internet Protocols include ZigBee based, z-wave based and Bluetooth.

Cyber-Physical Security of Microgrid Control Systems

Cyber-Physical Security

Traditionally, security used to be achieved on two fundamental levels; short-term security that deals with the stability of the demand-supply procurers. The dynamism enables the energy system to adapt quickly to sudden changes in the grid loads. Moreover, long-term security focuses on investments that support economic and sustainable development requirements. With the arrival of smart grids, which are defined as a composition of three interoperability infrastructures (communication, power system and information technology), has led the security problem to grow in complex imposing supplementary challenges threatening to introduce easier ways of causing damage to the fundamental security concerns and creating new ones.

Consequently, recent security assessment has focused on identifying the potential vulnerabilities introduced by the cyber layer and analyzes the possible impacts on energy systems, which has given birth to a brand new research area called cyber-physical security. A cyber-physical system is co-engineered collaborating domains of physical and computational counterparts, in which the crucial system tasks are handled with its physical part. Simultaneously, informatically enhanced processes normally referred to as cyber-are responsible for maximizing the exploration of intelligent devices and applications.

The reason why academia recently chose to add the term "physical" to the equation is to shed light on the emerging threats imposed by connecting these two fundamentally different infrastructures, which practically may lead to problems that do not particularly belong to a failure of either system. In light of these assumptions, further investigation is still needed to confirm or deny the putative relationships.

The most indispensable objectives of security requirements considerations of any data transferring communication in the IT network security are known as CIA-triad, which stands for Confidentiality, Integrity, and Availability, respectively. Smart grid security is also built upon the previous trestles, but with a difference in priority order, where availability comes on top of the requirements, followed by integrity, accountability, and confidentiality. Other referencing emphasizes accountability as additional security criteria. This sequence of importance goes back to the severity of impacts resulting from tampering with these criteria.

Attackers can penetrate the smart grid communication systems using vulnerable entry points in the logical border surrounding a network, known as the Electronic Security Perimeter (ESP). Interventions may occur with the help of numerous mediums, such as the Universal Serial Bus (USB) thumb drive, viruses, and even software patches and updates. Despite the fact that cyber intrusions on cyber-physical systems (CPSs) can be found under different terms, such as bias injection attack, zero dynamics attack, denial of service (DoS) attacks, eavesdropping attack, replay attack, stealthy attack, covert attack, and dynamic false data injection attacks. These attacks can still be classified according to the one or multiple security criteria they are jeopardizing.

Intentionally introduced faults or malicious attacks triggered by the cyber layer leaving serious impacts on not only the technical aspects, but also on economic and social correlations in power network operations, are the focus of this research. Effects range from tampering with smart meters data or manipulating the forecasted load profiles to reach equipment damage or even complete blackouts.

Threats to the Microgrid Systems and Associated Vulnerabilities

It is essential to enhance the ability of decision-makers to assess and manage risks associated with microgrids that may change over time and vary geographically. Therefore, clear identification of threats and vulnerabilities and their likelihood and impact is the first step in building resilient microgrids [START_REF] Mishra | Microgrid resilience: A holistic approach for assessing threats, identifying vulnerabilities, and designing corresponding mitigation strategies[END_REF].

• Threats. Threats are anything that can damage, destroy, or disrupt utility grid or microgrid operation. In other words, threats are what we are trying to protect microgrid against.

They are typically natural or human-induced hazards that are not within the control of the site, such as wildfire, hurricane, cyberattacks, or physical attacks. Threats are identified by reviewing climate data and state hazard assessments, and stakeholder interviews with site staff or emergency management teams from the surrounding community.

• Vulnerabilities. Vulnerabilities are weaknesses within the microgrid either in infrastructure or processes. Unlike threats, they are within the control of the site and can be modified or mitigated to prevent or reduce the impact of a disruption. Vulnerabilities are identified through stakeholder interviews with energy managers, electrical engineers, maintenance staff, emergency managers, utilities, microgrid designers, operators, and end-users. They can also be identified by reviewing contingency response plans and after-action reports following disasters or disruptions.

• Risk. Assessing the risk essentially means finding a way to quantify the relative potential of damage that various threats in the environment can cause to the microgrid. Therefore, risk is a function of threats exploiting vulnerabilities to impact the operations and damage or destroy the assets. Thus, we define the risk factor metric based on the probability of a threat, the probability of a vulnerability's exploitation given that threat, and the impact of the vulnerability. This definition of risk factor has been extensively used in the literature.

A microgrid system-level risk assessment is the first step in building resilient microgrids.

Figure 2.2: Risk Factor Venn Diagram

To evaluate the relationship between threats and vulnerabilities, the threat likelihood score is multiplied by the vulnerability probability and impact scores to create a risk score for each specific threat-vulnerability combination. Risk is created when a threat can exploit an already present vulnerability in the microgrid. The magnitude of the risk is determined by the likelihood of the threat and vulnerability, as well as the scale of damage the vulnerability could cause if exploited. Therefore, risk factor is calculated as intersection of these three factors, as shown in Figure 2.2. The risk score enables analysis and ranking of the risks to prioritize mitigation actions. Threats and hazards change over time, so the risk assessment should be updated on a regular basis.

For a complex cyber-physical system like a microgrid, there are multiple dimensions of resilience. They can be classified into three domains:

• Physical resilience. Sustaining physical infrastructure (which includes assets and electrical cables connecting the assets) during a disruptive event and continuing or restoring operations rapidly after the event is over.

• Cyber resilience. Identifying and defending against various types of cyberattacks as well as accidental failures and maintaining safe performance during such an event.

• Communications resilience. Maintaining safe and stable operational performance during communications failures from the utility grid end or internal microgrids communications.

More detailed discussions about these threats are shown as follows.

Physical Threats

Physical threats include natural hazards (wildfires, hurricanes, floods), changing climate (more extreme temperatures), and human-induced attacks (including terrorist attacks, such as shooting substations) [START_REF] Mishra | Microgrid resilience: A holistic approach for assessing threats, identifying vulnerabilities, and designing corresponding mitigation strategies[END_REF].

• Natural hazards. Weather events like tropical cyclones, hurricanes, floods, earthquakes, and wildfires can cause damage to the utility grid, and if the epicenter of the impact is geographically close to the microgrid, then the impact can be of equivalent order.

• Changing climate. Increasingly extreme temperatures caused by the changing climate can put undue strain on a microgrid especially when it is operating in islanded mode. The spike in cooling and heating loads can cause harm in two potential ways: 1) excessive power demand can cause equipment and devices to operate above their rated capacity, resulting in permanent damage; and 2) increased demand can cause load-shedding in the microgrid. If load-shedding strategies are not automated, then there is an increased risk of a site-wide outage.

• Human-induced attacks. Physical attacks, such as bombings or shootings of major microgrid components like generators and storage assets, could cause serious damage and disable a microgrid. Electromagnetic pulse attacks can also take out generation assets and controllers in the microgrid. The attacks can be from one gunman shooting specific equipment to a massive bombing that can destroy the whole site.

Cyber Threats

Cyberattacks on microgrids can be classified into the following three types [START_REF] Wang | Cyber security in the smart grid: Survey and challenges[END_REF][START_REF] Li | Cybersecurity in distributed power systems[END_REF]: attacks on availability, integrity, and confidentiality.

• Availability. Attacks on availability attempt to limit or delay access to data. In a microgrid, this may be extended to include availability of power in addition to data. Attacks against availability are generally referred to as Denial of Service (DoS) attacks. DoS attacks in microgrids generally include not only blocking resource access but also delaying the timing of critical message exchange. For example, jamming communications channels between various parts of the microgrid.

• Integrity. Attacks on integrity interfere with the accuracy or trustworthiness of data. Spoofing, tampering, and elevation of privilege are all integrity violations. For the microgrid, interference with the quality of power might also be considered a violation of integrity. For example, an attacker uses a Domain Name System (DNS) hijack to feed inaccurate price data to the controller, showing diesel as cheaper than natural gas. The result is that the microgrid uses diesel instead of natural gas, costing the operator money. Faking a microgrid disconnect from the grid signal compromises the integrity. Other examples of integrity attacks include sending false commands to the microgrid controller, causing unwanted or premature islanding; and changing the setting groups of protection relays in a microgrid, causing the relays to trip erroneously.

• Confidentiality. Attacks on confidentiality are attempts to gain access to privileged information. This might include operational or billing data, or any other information that should not be accessible to unauthorized entities. For microgrids participating in markets, the bidding price data can be stolen.

In a microgrid system, each interface or trust boundary represents a potential attack surface that must be analyzed for vulnerabilities. As methods for identifying specific threats are extensively well-documented elsewhere, we will focus here on those attack surfaces that are common to microgrid systems in general, rather than on specific attacks, which must necessarily be identified on a case-by-case basis. The vulnerable attacks surfaces along with the types of threats that can impact the operations of microgrids are listed as Table 2.2 [START_REF] Mishra | Microgrid resilience: A holistic approach for assessing threats, identifying vulnerabilities, and designing corresponding mitigation strategies[END_REF].

For microgrids with advanced controls and analytics capabilities, maintaining cyber-reliability is a matter of concern as well. The more complex the control algorithm, the higher the risk associated with having unforeseen/blinding scenarios making it behave in an unintended manner. Moreover, the higher the sophistication of the software stack, the greater the risk of injecting unintended bugs while developing the stack further. Since these challenges are not imposed by an external entity, they are not categorized as threats against which resilience solutions should be built in the microgrid. But they do pose a challenge for maintaining the reliability of microgrid, which is a significant aspect as well.

Communications Failure Threats

The modern grid is envisioned to be highly integrated with multidirectional data exchange and information communication between all its parts. The vision is consistent with microgrids as well for deploying a more active and responsive infrastructure for faster demand response and more exhaustive energy management capabilities. It is indispensable for an intelligent microgrid to rely on communication networks. On the other hand, using sophisticated communication schema makes the microgrid vulnerable, due to the increased risk of communication failure, along with the microgrid's growing dependence on electric systems and public communication networks.

In the case of microgrids, if the communication failure happens during grid-connected operation, the microgrid's controller will lose access to grid-side information, resulting in its inability to maintain the commanded power and power-factor at the point-of-interconnection. The microgrid should be able to maintain stable operations and stop feeding the power to the grid if it has been doing demand management before the communication failure. In worst case scenarios, the microgrid needs to have the ability to island itself safely. Communication failure internal to the microgrid is more of a reliability concern, as it is not imposed by an external entity. If not addressed appropriately, it can adversely impact the operation of the microgrid and damage the assets if left unattended for longer durations especially when operating in islanded mode.

Threats Due to Interdependencies Between Various Systems

Electricity, oil, natural gas, transportation, telecommunications, and water systems are all interdependent critical infrastructure (CI) systems. The operations of power systems are in constant exchange with other CI systems. On one level, electric power systems support other CI systems in their operations, whereas, on the other level, electric power systems depend on other CI systems to be able to operate.

These tightly intertwined interdependencies between other CI systems and electric power systems increase their vulnerabilities by exposing them to second-order and third-order threats. Microgrids are also equally, if not more, vulnerable to the threats posed by such interdependencies. Microgrids operate as integrated energy systems where gas, water, distributed generation, and utility grid supplies are coordinated to supply various kinds of loads. Interruption in the supply of any of these critical utilities (electricity, natural gas, water, or heat) can negatively impact the ability of the integrated energy system to meet loads and affect the operational economy of the microgrid.

Attack Detection of Microgrid Systems

There are different methods to detect attacks for microgrid system, including traditional methods based on state estimation and also data driven methods like machine learning. We give a detailed description about these methods and elaborate their advantages and disadvantages in detection attacks targeting microgrid systems.

Attack Detection based on State Estimation

For smartgrid systems including microgrids, false data injection attacks are defined in the following model:

z = Hx + n, (2.11)
where x ∈ R D contains the voltage phase angles at the buses, z ∈ R N is the vector of measurements, H ∈ R N ×D is the measurement Jacobian matrix, and n ∈ R N is the measurement noise, which is assumed to have independent components. The attack detection problem is defined as that of deciding whether or not there is an attack on the measurements. If the noise is distributed normally with zero mean, then an state vector estimation (SVE) method can be employed by computing

x = H T ΛH -1 H T Λz, (2.12)
where Λ is a diagonal matrix whose diagonal elements are given by Λ ii = v -2 i and v 2 i is the variance of n i ∀i = 1, 2, . . . , N . The goal of the attacker is to inject a false data vector a ∈ R N into the measurements without being detected by the operator. The resulting observation model is z = Hx + a + n.

(2.13)

The false data injection vector, a, is a nonzero vector, such that a i = 0, ∀i ∈ A, where A is the set of indices of the measurement variables that will be attacked. The secure variables satisfy the constraint a i = 0, ∀i ∈ A, where A is the set complement of A.

To detect an attack, the measurement residual is examined in 2 -norm ρ = z -Hx 2 2 , where x ∈ R D is the SVE. If ρ > τ , where τ ∈ R is an arbitrary threshold, which determines the tradeoff between the detection and the false alarm probabilities, then the network operator declares that the measurements are attacked.

Limitations of SVE-based attack detection methods

One of the challenging problems of SVE-based attack detection methods is that the exact recovery of state vectors is a challenge for SVE-based methods in sparse networks [START_REF] Cotilla-Sanchez | Comparing the topological and electrical structure of the north american electric power infrastructure[END_REF][START_REF] Kosut | Malicious data attacks on the smart grid[END_REF], where the Jacobian measurement matrix is sparse, and the performance of this approach is restricted by the sparsity of the state vectors.

In addition, unobservable attacks can be constructed even if the network operator can estimate the state vector correctly [START_REF] Liu | False data injection attacks against state estimation in electric power grids[END_REF][START_REF] Kosut | Malicious data attacks on the smart grid[END_REF]. For instance, if a = Hc, where c ∈ R D is an attack vector, then the attack is unobservable using the measurement residual ρ .

Moreover, detection methods broadly applying state estimation processes, e.g., using Kalman filter [START_REF] Manandhar | Detection of faults and attacks including false data injection attack in smart grid using kalman filter[END_REF], sparse optimization [START_REF] Liu | Detecting false data injection attacks on power grid by sparse optimization[END_REF], graphical methods [START_REF] Bi | Graphical methods for defense against false-data injection attacks on power system state estimation[END_REF] to address FDI attack detection all require centralized communication structures, which is difficult to achieve for distributed microgrid control systems. The FDI attack detection in distributed control systems deploying Kullback-Leibler (KL) divergence is addressed in [START_REF] Binod | Detection and mitigation of cyber-threats in the dc microgrid distributed control system[END_REF]. However, all the above attack detection techniques only deal with a single type of cyber attack (FDI). A distributed state estimation method based on the alternating direction method of multipliers (ADMMs) is proposed to detect hybrid cyber-attacks (FDI and DoS attacks) in [START_REF] Du | Admm-based distributed state estimation of smart grid under data deception and denial of service attacks[END_REF]. However, this method mainly benefits the large-scale power transmission systems due to its heavy computation complexity.

Attack Detection based on Machine Learning Techniques

Machine learning is a form of artificial intelligence that enables computers to learn and improve without being explicitly programmed. Researchers have proposed different machine learning algorithms to enable FDI attacks detection. In this subsection, we first discuss the metrics used for the performance evaluation of machine learning-based FDI attacks detection algorithms. Leveraging these metrics, we review the existing literature and compare their performance.

Performance metrics

A multitude of metrics has been adopted to evaluate the performance of the detection methods. Accuracy, precision, recall, F1 score, and receiver operating characteristic (ROC) curve are among the most common metrics. With the true label of a measurement and its predicted label, the output of a detection model can be divided into true positive (TP): indicating a correct positive prediction, true negative (TN): a correct negative prediction, false positive (FP): an incorrect positive prediction, and false negative (FN): an incorrect negative prediction.

Accuracy is the ratio of the number of correct predictions to the number of total predictions Accuracy = TP + TN TP + TN + FP + FN .

(2.14)

Accuracy is meaningful when the measurement data is balanced (when the number of positive and negative measurement samples are equal). To evaluate the performance of the detection model with imbalanced data, precision, recall, and F1 score are often considered. Precision (also called positive predictive value) describes the capability of a model to identify an attack's overall true positive predictions [111]. It is represented as the ratio of the correct positive predictions to the number of samples labelled as positive Precision = TP TP + FP .

(2.15)

The recall (also called sensitivity) gives the model the capability to identify all attacks. Recall is described as the ratio of the number of correct positive predictions to the number of positive samples Recall = TP TP + FN .

(2.16)

From (2.15) and (2.16) , it can be observed that precision and recall are closely related. For a given model, a decrease in FP (precision) leads to an increase in FN (recall), and vice versa. To achieve an optimal trade-off between precision and recall, the F-Measure is used to combine these two metrics. To avoid being heavily impacted by extreme values of precision or recall, the F-Measure score is designed as the harmonic average of precision and recall, as shown below

F1 score = 2 × ( Recall × Precision ) Recall + Precision . (2.17)
For a given classifier, recall and precision may vary a lot with different sets of measurements, e.g., balanced data and unbalanced data, making it hard to evaluate the performance of a classifier. To have a stable representation of the classifier performance, the ROC curve is often used. In a continuous binary classifier, the output is a continuous variable ranging from 0 to 1. Thus, a threshold is leveraged to divide the outputs into positive and negative. Different thresholds result in different TP rates (TPR, equals to recall) and FP rates (FPR = FP/(FP + TN)). ROC curve plots TPR (y-axis) against FPR (x-axis) under different thresholds. The closer the ROC curve is to the upper left corner or coordinate (0, 1) (the larger the area under the curve), the better the performance. ROC illustrates how well the detection method distinguishes between the attacked and the secured measurements

Different machine learning methods for attack detection

Supervised learning

Machine learning algorithms can be classified into supervised learning, semi-supervised learning, and unsupervised learning. In supervised learning, inputs and desired outputs are provided to the machine to construct a function that maps the input to the desired output.

The attack detection problem can be modeled as a binary supervised machine learning problem. Detecting FDI attacks is considered a supervised binary classification problem. The objective of the binary classifier is to decide whether the given data s with m features is either z, a normal measurement vector (negative class) or z a = z + a, an attacked measurement vector (positive class) [START_REF] Joachims | Making large-scale svm learning practical[END_REF]. The output class labels are

y = +1 for a = 0 -1 for a = 0 (2.18)
where a is the attack vector. The common used supervised learning algorithms are perceptrons, support vector machines (SVMs), knearest neighbours (KNNs), and logistic regression.

• Perceptron. In perceptrons, a weight vector w ∈ R M Tr is trained such that the output label, y i , of a sample s i is predicted by the following classification function:

f (s i ) = sign (w • s i ) = 1 for w • s i ≥ 0(a = 0), -1 for w • s i < 0(a = 0). (2.19)
During the training phase, the weight vector is updated for each training sample as w (i + 1) = w (i) + ∆w , where ∆w = γ (y i -f (s i )) s i and γ is the learning rate. From the classification function, we can see that the convergence of the perceptron algorithm can be guaranteed when the samples are linearly separable. Therefore, it is suitable for FDIs detection only when a hyperplane can separate the measurements.

• Support Vector Machine (SVM). In SVMs, a hyperplane is constructed to separate two different classes. The hyperplane can be represented by a weight vector w and a bias value b. The decision boundaries for the linear separable data can be formulated as two parallel hyperplanes using (2.20)

w T s i + b = +1, if y i = +1 w T s i + b = -1, if y i = -1 (2.
y i w T • s i + b -1 + ξ i ≥ 0 ξ i ≥ 0∀i = 1, 2, 3, . . . , M Tr (2.21)
where ζ is the adjustable regularisation parameter, ξ i is the slack variable for the non-linear separable training set, and M Tr is the feature vector.

• k-Nearest Neighbor (KNN). KNN is another supervised learning algorithm that assigns labels to an unlabelled sample according to its KNNs. The Euclidean distance is used to determine the similarity between a given labeled sample, s i , and an unlabelled sample, s i . The set of KNNs for a given measurement sample can be determined using the Euclidean distance as follows :

s i -s i(1) 2 ≤ s i -s i(2) 2 ≤ . . . ≤ s i -s i(M Tr ) 2 ℵ (s i ) = s i(1) , s i(2) , . . . , s i(k ) (2.22)
Majority voting is one of the most commonly used methods for assigning labels from the set of KNNs of s i . KNN is easy to implement but it fails to work when the size of the data sample is smaller than the dimension of the feature vector.

• Logistic Regression. The logistic regression algorithm assumes that the distribution of the label y i of data s i follows the following logistic function:

P (y i | s i ) = 1 1 + exp (-y i (w • s i + b))
.

(2.23)

The weight vector w is estimated by maximizing the following cost function:

J (w ) = - 1 M Tr M Tr i=1 log (1 + exp (-y i (w • s i + b))) . (2.24)
A brief comparison of various machine learning methods is presented in [START_REF] Ozay | Machine learning methods for attack detection in the smart grid[END_REF]. The paper is one of the first research works to utilize supervised learning algorithms for FDI detection in smartgrid systems. The authors used a hierarchical network in which the measurements are grouped as clusters, and each cluster is regarded as a sample si . The false data is directly injected into the measurements before the measurements are grouped into clusters. The detection method is based on the observations made in [START_REF] Liu | Detecting false data injection attacks on power grid by sparse optimization[END_REF]. According to [START_REF] Liu | Detecting false data injection attacks on power grid by sparse optimization[END_REF], the distance between samples determines the attack vector:

s i -s j 2 =      z i -z j + a i -a j 2 , if a i , a j = 0 z i -z j + a i 2 , if a i = 0, a j = 0 z i -z j 2 , if a i , a j = 0 (2.25)
Therefore, by looking into the distance between two samples, it is possible to detect an FDI. In their experiment, the performance of different machine learning algorithms is evaluated against FDI attacks with different sparsity k/m (the ratio of measurements that the attacker has access to). Accuracy, precision, and recall are used as performance metrics. The results proved that the machine learning algorithms perform better than any other algorithm (e.g., state vector estimation approach) in detecting FDIs. Although SVMs achieved the highest prediction accuracy, they also present some limitations, such as the selection of the kernel and sensitivity to the sparsity of the system. KNN is very sensitive to system size and performed better for small-sized systems. Despite conducting plenty of experiments, the authors of [START_REF] Ozay | Machine learning methods for attack detection in the smart grid[END_REF] did not evaluate the performance of detection algorithms for stealthy FDIs. Furthermore, only the sparsity of injected data was considered. The magnitude of the injected data could potentially impact both the operation of the system and the performance of the detection methods. Last, the lack of attack data can result in imbalanced data samples during the detector training process affecting its classification accuracy.

Considering the limitations of [START_REF] Ozay | Machine learning methods for attack detection in the smart grid[END_REF], a similar work is conducted in [START_REF] Yan | Detection of false data attacks in smart grid with supervised learning[END_REF]. Both works utilize closely related system models. Two assumptions are taken into consideration when the attack vectors are created in the adversary model: (i) that the injected value a i is greater than the noise level and (ii) that the mean of the attack vector a i is larger than the variance of the attack vector. Attack vectors with different sparsity and variance (The variance reflects the magnitude of disturbances caused by false data.) are tested in their experiments. To solve the imbalanced data problem, they propose the extended nearest neighbor (ENN) algorithm. For each class, ENN measures the average ratio of the nearest neighbors belonging to the same class. Instead of using majority voting, the label of a sample was predicted by finding the class which presents the greatest ratio variability with the sample labeled in that class. The performance of SVMs, KNN, and ENN is then experimentally evaluated. Accuracy and F1 scores are used as performance metrics. SVMs outperformed KNN and ENN in most of the test cases. A critical range of sparsity was observed in which the accuracy and F1 score increased significantly. However, this is reasonable since the distance increases s i -s j when the sparsity increases, which leads to more distinct classes. The experiment was conducted on the IEEE 30 bus system. The detection performance of the algorithms in larger systems was not demonstrated.

Esmalifalak et al. [START_REF] Esmalifalak | Detecting stealthy false data injection using machine learning in smart grid[END_REF] proposed a distributed SVM algorithm. Each substation owned a training set and stealthy FDI attacks, which could bypass BDD methods based on their corresponding residuals. Before training, PCA is applied to the training set to reduce the feature dimension. To avoid a huge volume of data exchange, each substation is trained using a local classifier, and only the locally optimized weight vectors are exchanged. Their optimization problem, (2.21), is provided below

min w k ,ξ k ,b k w k 2 2 + ζ n k =1 Tr i=1 ξ ki Subject to: y ki w Tr k • s ki + b k -1 + ξ ki ≥ 0 ξ ki ≥ 0∀i = 1, 2, 3, . . . , m; k = 1, . . . , n (2.26)
where n is the number of substations and w k is the local optimization parameter. The alternating direction method of multipliers is used to solve this distributed optimization problem. Experiments are performed on the IEEE 118 bus system. The authors empirically verify the convergence of distributed SVM classifiers to centralized SVMs with different numbers of substations.

To recapitulate, supervised learning methods have achieved superior performance in comparison with traditional residual-based BDD methods. Among the aforementioned algorithms, SVMs have demonstrated to achieve the highest accuracy. According to Esmalifalak et al. [START_REF] Esmalifalak | Detecting stealthy false data injection using machine learning in smart grid[END_REF], the curse of the dimensionality problem can be solved by leveraging PCA, which significantly enhanced the efficiency of machine learning algorithms. Nevertheless, most of the attack data are often generated randomly in the experiments while a sophisticated adversary would deliberately choose attack vectors considering the system dynamics. The performance of the proposed methods against such sophisticated FDI attacks still remains unknown. Moreover, most of the prior works conducted simulation experiments. Thus, the efficiency of existing methods, if applied to real power system deployments, cannot be guaranteed.

In addition to FDI attacks, supervised machine learning techniques are also used for detecting other cyber attacks. In [START_REF] Meti | Detection of distributed denial of service attacks using machine learning algorithms in software defined networks[END_REF], the authors proposed DoS attack detection schemes based on SVM and Neural Network classifiers. In [START_REF] Iqbal | A classification framework to detect dos attacks[END_REF], the authors present a machine learning-based classification framework to detect the Denial of Service (DoS) attacks. The framework consists of five stages, including 1) selection of the relevant Dataset, 2) Data pre-processing, 3) Feature Selection, 4) Detection, and 5) reflection of Results.

Semi-supervised learning

In semi-supervised learning, the majority of the given data is unlabelled. Although semisupervised learning algorithms are the least common learning approaches applied for the detection of FDI attacks, we still introduce them in this survey work for completeness. An example of a semi-supervised learning algorithm is a semisupervised SVM (S 3 VM). S 3 VM assumes that samples with different labels are clustered into different groups and that the diameter of each cluster is small enough to avoid sub-clusters [START_REF] Chapelle | Optimization techniques for semi-supervised support vector machines[END_REF] The objective function of S 3 VM is defined as

min w ,b ζ   M Tr i=1 L Tr (s i , y i ) + M Ts i=1 L Ts s i   + w 2 (2.27)
where y = w T s i + b and ζ is the regularisation parameter, L Tr and L Ts are the loss function of the training and test samples, respectively. Foroutan and Salmasi [START_REF] Armina | Detection of false data injection attacks against state estimation in smart grids based on a mixture gaussian distribution learning method[END_REF] investigated FDI attack detection methods by using the S 3 VM based on Gaussian mixture distributions. According to Filho [START_REF] Gesteira | Mixture models for the analysis of gene expression[END_REF], a finite mixture distribution model, defined as a convex combination of two or more probability density functions, is capable of approximating any arbitrary distributions due to its flexibility in modeling complex data. The authors assume that all FDI attacks have the same amount of energy or c vector, where a = H c, and that they have the same mean squared error. In the adversary model, the attack vectors are designed based on the minimum energy residual attack and sparsest attack, introduced in [START_REF] Kosut | Malicious data attacks on the smart grid[END_REF][START_REF] Julien | Efficient computations of a security index for false data attacks in power networks[END_REF], respectively. In the training phase, a positive data set, i.e., a data set with attacked measurements, was used to build the Gaussian mixture model. Then, a mixture of data sets consisting of both positive and negative labels (attacked and normal measurements) determines the threshold. In the evaluation phase, the unlabelled data set used for testing and F1 score evaluates the performance of the results. PCA was applied to the data set to overcome measurement dimensionality issues. The authors demonstrate the performance of the proposed detection method on the IEEE 118 bus power system. To generate diversified datasets, different topological networks are constructed using Monte-Carlo simulations. The performance of the proposed detection method depends on the selection of a proper threshold. A high-threshold value reduces recall while a low-threshold value lowers precision. The impact of the detection algorithms is illustrated with a ROC curve. Although the proposed model demonstrates a high F1 score compared with other machine learning algorithms (e.g., SVMs and perceptrons), it performs well only when the attacked measurements and the real measurements lie in distinct regions of the feature space, i.e., the attacked data can be effortlessly isolated.

Another detection method based on the S 3 VM algorithm is proposed in [START_REF] Ozay | Machine learning methods for attack detection in the smart grid[END_REF]. The input samples are integrated into the cost function forming the following optimization problem:

min W 2 2 + ζ 1 M Tr i=1 L Tr (S i , y i ) + ζ 2 M Te i=1 L Te S i , ( 2.28)
where ζ 1 and ζ 2 are the cost parameters, L Tr and L Te are the loss functions for the training and testing samples. In the simulation, the authors use default values for the parameters as suggested in [START_REF] Joachims | Making large-scale svm learning practical[END_REF]. The experiments are conducted on IEEE 9, 57, and 118 bus systems, and the measurement matrix is generated using Matlab's Matpower toolbox. Compared with supervised learning algorithms, S 3 VM demonstrated improved robustness against data sparsity despite the fact that S 3 VM still remains sensitive to unbalanced data samples.

Unsupervised learning

Unsupervised learning algorithms group the unlabelled samples based on the similarities and differences between samples, without any prior training. Clustering is the most popular unsupervised learning method where the measurement samples are grouped based on the distance between samples in the feature space. Different distance metrics can be chosen (e.g., Euclidean distance). K -means (also called hard c-means) is an example of a clustering method, which divides data into k groups. k -means iteratively assigns each data point to one of the k groups, whose centroid has the minimum distance to the data point in the feature space. Each centroid in a cluster is a collection of features, which define a group. The centroids are updated at each round. Several techniques are used to validate the k -value including cross-validation and other information criteria. Fuzzy c-means (FCM) clustering is another type of k -means clustering, which assigns data points to two or more clusters [START_REF] Mohammadpourfard | Identification of false data injection attacks with considering the impact of wind generation and topology reconfigurations[END_REF]. Each point belongs to a cluster based on a corresponding probability value, rather than having a binary value as is the case of k -means clustering. In FCM, the clustering problem can be solved by minimizing the following equation:

J = N i=1 C j =1 u m ij x i -C j 2 , 1 ≤ m < ∞ (2.29)
where N is the number of data points, C = 2 is the number of the clusters (cluster of attacks and cluster of normal measurements), x i is the ith dimensional measured data, and C j is the center of the j th cluster, which is determined using

C j = N i=1 u m ij • x i N i=1 u m ij (2.30)
where u ij is the degree of membership of the ith measurement. The updated membership u ij computed by the following equation:

u ij = 1 C k =1 (( x i -C j ) / ( x i -C k )) 2/(m-1)
(2.31)

Mohammadpourfard et al. [START_REF] Mohammadpourfard | A statistical unsupervised method against false data injection attacks: A visualization-based approach[END_REF] presented a visualization based on the unsupervised anomaly detection method and FCM clustering to detect and locate FDI attacks. The authors also propose a localization method that helps in identifying the attack after topology reconfigurations and the integration of different resources. When FDI attacks occur, the probability distributions of system states deviate significantly from normal states, hence enabling FDI attack detection. First, the authors normalize the data, and then various statistical measures are applied to characterize the probability distribution of each state vector. PCA is applied to the new feature set to reduce the dimensionality of data and to visualize them in a two-dimensional space where the grid operators can determine whether an attack has occurred or not (using patterns of normal and abnormal data). FCM is used to detect outliers and locate the FDI attacks. Load data from the New York Independent System Operator are used for the simulations. FDI attacks data are generated on the IEEE 9 and 14 bus system with the assumption that the adversary decreases or increases a specific state variable by at least 6% of its original value. The proposed method is applied to two different FDI attack scenarios: detecting FDI attacks with and without topology changes. Compared to supervised learning algorithms such as SVMs and KNN, the proposed model achieves higher detection accuracy [START_REF] Mohammadpourfard | A statistical unsupervised method against false data injection attacks: A visualization-based approach[END_REF].

Yang et al. [START_REF] Yang | False data injection attacks detection in power system using machine learning method[END_REF] proposed three different anomaly detection approaches to detect FDI attacks: (i) local outlier factor, (ii) isolation forest, and (iii) robust covariance estimation. The local outlier factor is a density-based anomaly detection method that measures the local standard deviation of any given data point from its neighbors by comparing their local density [START_REF] Konstantinou | A data-based detection method against false data injection attacks[END_REF]. Isolation forest is an outlier detection technique based on decision trees that does not employ any distance or density measure and can handle large, high-dimensional datasets. Robust covariance estimation is another anomaly detection method based on the elliptic envelope fitting method, which assumes that the given data is a Gaussian distribution and defines the shape of the data. An IEEE 14 bus system case is used to evaluate the mentioned detection approaches. Attack vectors generated with Gaussian distributed non-zero elements have the same mean and variance as the original measurement set. The authors use PCA to reduce the data dimension from 41 to 2, to reduce noise, and simplify the detection problem. All proposed methods achieve high accuracy for FDI attacks detection. However, these three detection methods achieve high detection rates only when the contamination rate is known and small [START_REF] Yang | False data injection attacks detection in power system using machine learning method[END_REF] .

Deep neural network

Deep learning algorithms mimic the human brain structure, functions, and are one of the fastest developing artificial intelligence technologies. Although deep learning algorithms require time and large amounts of data for their training stage, they have been applied for FDI attacks detection achieving high-accuracy rates [START_REF] Hassabis | Neuroscience-inspired artificial intelligence[END_REF].

Multilayer perceptrons (MLPs), also called feed-forward neural networks, are deep learning models where information flows in only one direction, i.e., from the input through the hidden layers to the output, as shown in Figure 2.4. They consist of an input layer, which receives the input signals, one or more hidden layers to construct the approximation function, and an output layer that predicts the final decision based on the input and approximation function. Multiple studies where MLPs have been applied to detect FDI attacks have been reported in the literature [START_REF] Mohammadpourfard | A statistical unsupervised method against false data injection attacks: A visualization-based approach[END_REF][START_REF] Ganjkhani | A novel detection algorithm to identify false data injection attacks on power system state estimation[END_REF][START_REF] Tabakhpour | Neural network model for false data detection in power system state estimation[END_REF]. In these works, the FDI attack detection problem is formulated as a supervised classification problem. In MLP-assisted binary classification, a linear combination of an input weight vector produces a single output, as shown in the following equation:

y = ϕ n i=1 w i s i + b (2.32)
where y is the estimated output of the activation function, w is the weight, s is the input vector, b is the bias, and ϕ is the non-linear activation function. The activation function is an essential feature of the MLP architectures. It decides whether a neuron should be fired or not by calculating the weighted sum of inputs and adding a corresponding bias to it. Sigmoid, tanh, and rectified linear unit (RELU) are examples of activation functions. RELU is the most widely used function because it is fast and less computationally expensive. MLPs use back-propagation training algorithms and the weights are updated using gradient descent to minimize the error function.

Ashrafuzzaman et al. [START_REF] Ashrafuzzaman | Detecting stealthy false data injection attacks in power grids using deep learning[END_REF] proposed different MLP structures for the detection of FDI attacks in an AC static SE system topology. The paper assumes that partial knowledge of the system, including the H matrix and other parameters, is known to the attacker. A standard IEEE 14 bus system is used to conduct the simulation. The Matpower toolbox is used to generate the measurement vector z, which contains 122 measurement features (40 active and reactive power flows, 14 power injections, and 27 voltage measurements). The authors train the MLP using stochastic gradient descent (an optimization technique for the network parameters update) and tanh as the activation function. Four models with different network architectures are utilized for detection. The first model consists of one hidden layer with 100 neurons, and the second model consists of three hidden layers with 150 neurons. For the third and fourth models, the authors use the first and second models with a regularisation value of 0.0001. Regularisation is a technique used to reduce or prevent overfitting of a neural network. The models' detection performance is compared with other machine learning algorithms. Accuracy, precision, recall, and F1 score are used to evaluate the MLP detection. The results of the four discussed models are similar with an accuracy of around 98%.

Similarly, Foroutan and Salmasi [START_REF] Armina | Detection of false data injection attacks against state estimation in smart grids based on a mixture gaussian distribution learning method[END_REF] applied MLPs to detect FDI attacks and compared them with common machine learning detection models. The network consists of an input layer, one hidden layer, and an output layer. Tanh is used as an activation function. Although MLP produced higher detection accuracy than the other algorithms, their training process is slow. Ganjkhani et al. [START_REF] Ganjkhani | A novel detection algorithm to identify false data injection attacks on power system state estimation[END_REF], on the other hand, introduced a novel MLP algorithm leveraging a non-linear autoregressive exogenous (NARX) configuration, which takes into account the high correlation between power system measurements as well as the state variables. The NARX configuration is used for time series prediction and can predict step-ahead values of the states by factoring measurement values and historical data as input variables. In the experiment, NARX is constructed with an input and a hidden layer with different numbers of neurons and sigmoid linear activation functions. The historical data contained 6048 measurement vectors and state variables. The detection model is trained using 70% of the historical data and 30% for the testing and validation.

A recurrent neural network (RNN) is a sophisticated deep learning algorithm that uses internal memory or feedback loops, as shown in Figure 2.5. Unlike MLPs, RNNs use the information from past events for their predictions. A long short-term memory unit can be added to a standard RNN to solve the problem of vanishing gradient descent and store information for an extended period [START_REF] Hochreiter | Long short-term memory[END_REF]. RNNs formulate the FDI attacks detection problem as a sequence of prediction. Results from previous time steps are used for the prediction of the current output rendering RNNs efficient in detecting manipulated measurements. Ayad et al. [START_REF] Ayad | Detection of false data injection attacks in smart grids using recurrent neural networks[END_REF] utilized RNNs as a sequence classification algorithm for detecting FDI attacks in DC SE. Back-propagation through time, an extensive type of back-propagation, is applied to train the algorithm. The authors run the training algorithm multiple times to produce the optimal set of parameters that achieve the least error. Then, the optimal parameters are applied to the network for the prediction of the test data classes. Since the output ranged from 0 to 1, a threshold is set to determine the output class as either 1 or 0 (1 is compromised, and 0 is normal). The IEEE 30 bus test case is used for the experiments with 112 measurement vectors. The proposed model obtains outstanding detection results with an accuracy rate of 99%. James et al. [START_REF] James | Online false data injection attack detection with wavelet transform and deep neural networks[END_REF] proposed an RNN architecture for detecting FDI attacks in AC SE setups. The discrete wavelet transform (DWT) algorithm is used for the RNN model. The main goal of DWT is to extract the hidden time-frequency domain characteristics and features at every specific time. The proposed model is able to leverage dynamic temporal and spatial features for attack detection. The authors detect FDI attacks in AC SE with complete and incomplete system knowledge. For the incomplete knowledge case, the attacker only knew a few selected phase angles, power flows, and power injections for selected buses. The remaining buses' information is generated using the algebraic sum of the connecting buses. The RNN-based detection model is constructed with two types of neuron layers: gated recurrent unit and fully-connected dense layers. To tune the network hyper-parameters, a dropout approach is used. In dropout, the outputs of some layers are discarded according to predefined probabilities. Dropout solves the overfitting problem, eminent in extensive training datasets, and increases the accuracy for newly added test data. The proposed detection method's performance is assessed on the IEEE 118 bus and 300 bus test cases. Over 200k samples are generated to train the detection model, and which achieves a high detection rate of 93% [107]. However, the main challenge of this RNN type is to optimally tune the network hyper-parameters.

Deep belief networks (DBNs) consist of multiple layers of stochastic and latent variables [START_REF] Hinton | Deep belief networks[END_REF]. The latent variables are generally binary variables. DBNs are compositions of simple, unsupervised networks such as restricted Boltzmann machines or autoencoders [START_REF] Hinton | Deep belief networks[END_REF]. The authors in [START_REF] Mariam Mn Aboelwafa | A machine learning-based technique for false data injection attacks detection in industrial iot[END_REF] utilized autoencoder networks for the detection of FDI attack leveraging temporal and spatial sensor data correlations. He et al. [START_REF] He | Real-time detection of false data injection attacks in smart grid: A deep learning-based intelligent mechanism[END_REF] proposed a DBN and state vector estimator (SVE) for real-time detection of FDI attacks. The proposed model utilizes an extended DBN called conditional DBN which extracts temporal features in high-dimensions. SVE calculates the 2 -norm of the measurement residuals and compares them with a given threshold as follows [START_REF] He | Real-time detection of false data injection attacks in smart grid: A deep learning-based intelligent mechanism[END_REF]:

r = ẑ -H x 2 > τ, Attack alarm r = ẑ -H x 2 ≤ τ, No attack alarm (2.33)
The authors design the model based on the assumption that the topology of the power system does not change significantly within a small time-frame. For the simulation, the IEEE 118 bus test case is used to simulate four different attack scenarios. ROC curve is used to evaluate the detection scheme. Then, the detection results with a different number of attacked measurement k are compared with other detection algorithms such as MLPs and SVMs. The proposed model achieves the highest detection accuracy. However, training a DBN is extremely computationally expensive, since this process can take up to weeks even if specialized hardware exploiting graphics processing unit acceleration is used [START_REF] Sarikaya | Application of deep belief networks for natural language understanding[END_REF].

Wei et al. [START_REF] Wei | False data injection attacks detection with deep belief networks in smart grid[END_REF] proposed a different DBN-based model, where the detection process can be divided into three parts: (i) the pre-processing data stage, (ii) the training stage, and (iii) the testing stage. During the pre-processing data stage, measurement data including the attacked measurements are extracted using different IEEE standard nodes. The training process of the DBN is divided into the pretraining stage and reverse-trimming stage. In the pretraining stage, the authors use an unsupervised greedy learning algorithm from the bottom layer to the upper layer to extract the measurement features, train every layer, and share the measurement features with every layer. The Restricted Boltzmann Machine (RBM) is trained layer-by-layer and tuned using backpropagation to minimize prediction errors. After the training process, part of the measurement data is used to test and validate the performance of the detection model. The simulation results show that DBN-based detection achieves high accuracy in detecting FDI attacks (98%). 38

Summary

As shown in Table 2.3, the machine learning methods and related references are summarized.

Summary

This chapter gives a detailed review of the microgrid control systems and the related vulnerabilities and attack detection methods. Improving the cybersecurity of cyber-physical energy systems is vital for the efficient and resilient operation of the power grid. Cyber attacks can elicit severe physical and economic impacts on power systems. Researchers have thoroughly investigated FDI attacks and have proposed algorithms to detect these data integrity attacks. Among these algorithms, machine learning-based methodologies are gaining attention due to their superior detection performance.

In this chapter, we provide a comprehensive review of various cyber attack detection methods leveraging machine learning algorithms. The goal of this survey is to compare different machine learning cyber attack detectors employed in power systems. Our investigation concludes that supervised learning and deep learning methods achieve the highest detection rates. Our future work will explore how machine learning-based cyber attack detectors perform in DS which incorporate DERs (e.g. microgrids) and what modifications are essential. Also, we aim to develop detection algorithms leveraging generative adversarial networks to further improve cyber attacks detection performance against stealthy and more sophisticated attacks.

Chapter 3

Impact Analysis of Synchronization

Attacks on Microgrid Systems

Introduction

A microgrid is a small-scale low-voltage electrical network, consisting of generation units, loads and storage elements, where dedicated control systems enable them to provide guaranteed power quality for local loads and have a high integration of distributed generators (DGs), including photovoltaic (PV) and wind power generation systems. The stability of the control system in microgrids is a key element to guarantee safe and reliable operations. However, the widely use of communication networks in their control systems introduces new security challenges since they are vulnerable to malicious cyber attacks. A power outage incident caused by the malware "BlackEnergy" in Ukraine during 2015 has proved that cyber attacks could cause a major damage on power quality and devices. In [START_REF] Teixeira | Voltage control for interconnected microgrids under adversarial actions[END_REF], the authors studied cyber attacks on the voltage/frequency control systems of microgrid systems where they consider two types of attack scenario, reference signal attack and measurement routing attack, and study their impact on voltage stability and deviation in the droop-controlled DGs. Risk assessment methods to quantify the impact of measurement falsification attacks on a microgrid system are studied in [START_REF] Ma | Voltage control in distributed generation under measurement falsification attacks[END_REF]. To the best of our knowledge, these previous works only consider threat models targeting the primary control level of a microgrid system.

In this chapter, we consider a more comprehensive control structure for a microgrid system, mainly a hierarchical control approach consisting of three control levels: primary, secondary and tertiary control [START_REF] Bidram | Distributed control systems for small-scale power networks: Using multiagent cooperative control theory[END_REF]. Specifically, we design a distributed cooperative hierarchical control system consisting of droop control as primary control and cooperative control as secondary control. We believe this control scheme is more realistic and reliable than the control architecture considered in [START_REF] Teixeira | Voltage control for interconnected microgrids under adversarial actions[END_REF] and [START_REF] Ma | Voltage control in distributed generation under measurement falsification attacks[END_REF], because of its better support for scalable integration of local loads and high integration of distributed generation.

Based on the above control model, we consider cyber attacks that may target the communication links between DGs, and we use risk assessment methods to quantitively analyze the impact of these attacks on voltage deviation at the primary control level and reference voltage synchronization at the secondary control level. Our impact analysis results are valuable for microgrid system designers to help them evaluate potential cyber threats targeting these systems.

The rest of this chapter is organized as follows. Section 3.2 describes the detailed system model of the microgrid system under study. Section 3.3 shows details about the primary and secondary controllers that are employed to the microgrid system. Section 3.4 takes a specific measurement as reference (MaR) attack as an eample, and shows how do we model the attacks and theoretically analyze the attack impacts. Section 3.5 validates the correctness of the attack impact analysis with simulation results. At last, Section 3.6 summarizes this chapter.

System Model

In this section, we describe the model of a microgrid system in terms of its hierarchical control structure and communication network.

Preliminaries

In this section, we review several important definitions and properties with regard to certain classes of linear time-invariant (LTI) systems that will be useful in building our system model and running further theoretical analysis. Consider a state-space represented continuous LTI system:

ẋ (t) = Ax (t) + Bu(t) y(t) = Cx (t) + Du(t).
(3.1)

In the LTI system (3.1), x (t) ∈ R n , u(t) ∈ R m and y(t) ∈ R p are the state vector, the input vector, and the output vector at time t, respectively. And A, B , C and D are the dynamics matrix, input matrix, output matrix and feedthrough matrix respectively. Denoting a ij = [A] i,j as the entry of A in the i-th row and j -th column, the class of diagonally dominant matrices is defined as follows.

Definition 1 (Diagonally dominant matrices). A square matrix A is called to be row-diagonally dominant if its entries satisfy the conditions

|a ii | ≥ j =i |a ij | , ∀i ∈ {1, . . . , n} . (3.2)
Given Definition 1, the system (3.1) is called to be row-diagonally dominant if the dynamics matrix A is row-diagonally dominant.

Besides row-diagonally dominant systems, another important class of systems throughout this paper is that of positive systems. Next we describe the definition and properties of positive systems.

Definition 2 (Positive systems). The LTI system (3.1) is said to be (internally) positive if and only if its state x (t) and output y(t) are non-negative for every non-negative input u(t) and every non-negative initial state x (0).

Lemma 1 (Positivity). The LTI system (3.1) is positive if and only if

A is a Metzler-matrix, i.e., it has non-negative off-diagonal entries, and B , C and D are non-negative, i.e., they only have non-negative entries.

Lemma 2 ([53]

). If the system (3.1) is positive, the following statements are equivalent:

1) the matrix A is Hurwitz, i.e., every eigenvalue of A has strictly negative real part).

2) There exists a ξ ∈ R n such that ξ > 0 and Aξ < 0.

3) The matrix -A -1 exists and has nonnegative entries. These first two statements give a scalable stability verification and relate the stability of positive systems to a set of inequality constraints, which is helpful to derive stability conditions. The third statement contributes to the voltage magnitude deviation analysis for systems under cyber attacks.

Microgrid System with Hierarchical Control Structure

Microgrids are designed to work in both grid-connected and islanded operating modes, depending on whether connecting or not to the main grid. The task of the microgrid control system is to regulate voltage and frequency for different operating modes, to achieve proper load sharing among DGs, to control the power flow between the main grid and the microgrid, and to optimize the cost of its operations.

Currently, a hierarchical control structure is designed to achieve the above operating goals [START_REF] Bidram | Distributed control systems for small-scale power networks: Using multiagent cooperative control theory[END_REF]. As shown in Figure 3.1, the hierarchical control structure consists of primary, secondary and tertiary control levels. Each control level has its own control goals because of operating in different timescales. The primary control operates on a fast timescale, and is responsible for the control of transients to stabilize the voltage and frequency of the microgrid during changing of load or generation, or subsequent to an islanding event. Secondary control is designed to compensate for voltage and frequency deviations caused by primary control in terms of fault conditions. Finally, tertiary control, as the highest and slowest control level, optimizes the operations in both operating modes and manages to control the power flow between the main grid and the microgrid.

In this chapter, we mainly consider the primary and secondary control levels of microgrid with balanced loads. The primary control is locally implemented at each DG, e.g., droop control for inverter-based distributed energy resources (DER). However, the secondary control usually relies on a centralized control structure. Central controllers are designed to issue global commands requiring information gathered from the whole system and thus a complex two-way communication network is also needed. This kind of communication network makes it vulnerable and may effect the system reliability. In [START_REF] Bidram | Distributed control systems for small-scale power networks: Using multiagent cooperative control theory[END_REF], the authors replace the existing standard centralized secondary control with an efficient distributed control structure. They consider the microgrid as a multi-agent system where each DG is an agent. A voltage source inverter (VSI) is employed to connect a DG to the microgrid. The VSIs are interconnected through the physical power network configuration.

A sparse communication network, which overlays the physical power network, is integrated in the control system. The controllers use this communication network to only communicate in a distributed way with neighboring nodes. The secondary controller also uses cooperative multiagent control techniques to make all agents act as a one group to a common synchronization goal and follow cooperative decisions.

In this chapter, we consider a distributed cooperative control structure similar to [START_REF] Bidram | Distributed control systems for small-scale power networks: Using multiagent cooperative control theory[END_REF], but design our own primary and secondary control loops emulating real microgrid systems. The microgrid with N DG units is depicted in Figure 1. Each DG unit is represented by a bus. The lumped DERs and loads within one DG unit are respectively modeled as a single DER and load. We set the DG unit directly connected to main grid as DG Unit 1 . Assumption 1. In the power distribution system under study, we make the following assumptions:

1) The system has balanced three-phase power network, i.e., it can be represented as an equivalent single-phase system;

2) All N buses are inverter-based, and represented by V i and θ i for i = 1, . . . , N .

Let R ij and X ij be resistance and reactance of the transmission line between bus i and bus j , respectively, thus under Assumption 1, the active and reactive power injections at bus i is given respectively by

P i = V 2 i G i - j ∈N i V i V j (G ij cos(θ ij ) + B ij sin(θij )), Q i = -V 2 i B i - j ∈N i V i V j (G ij sin(θ ij ) -B ij cos(θij )), (3.3)
in which

G ij = R ij /(R 2 ij + X 2 ij ) ≥ 0 and B ij = -X ij /(R 2 ij + X 2 ij
) ≤ 0 are, respectively, the conductance and susceptance of the transmission line between bus i and bus j . Additionally, we define self-conductance and self-susceptance as

G i = G ii + j ∈N i G ij ≥ 0 and B i = B ii + j ∈N i B ij ≤ 0,
respectively. Note that we use θ ij = θ i -θ j to represent the angle difference between node i and j in the remainder of this paper.

Assumption 2. In the power distribution system under study, we assume the transmission line impedances have the same ratio

R ij /X ij = -G ij /B ij = ρ ≥ 0 for all lines (i, j ) ∈ E.
The line ratio reflects the inherent property of the power system. A small ratio ρ > 0 represents power systems with inductive transmission lines (R ij X ij ), while higher ratio ρ means the transmission lines are resistive, which is often the case for medium and low voltage distribution systems. We can draw the conclusion that, for systems with homogeneous transmission lines that have similar characteristics, Assumption 2 naturally holds because the line ratio ρ depends on the transmission line characteristics. Moreover, Assumption 2 is also commonly employed in the case of purely inductive lines (ρ = 0) [START_REF] Schiffer | Conditions for stability of droopcontrolled inverter-based microgrids[END_REF][START_REF] Simpson-Porco | Synchronization and power sharing for droopcontrolled inverters in islanded microgrids[END_REF].

Communication Network

In the microgrid system, DG units are considered as the nodes of the communication graph and the corresponding communication links represent its edges. Figure 1 describes a line network, however a generic network topology of a microgrid system is usually characterized by a directed graph (digraph) G = (V, E, A G ) with a nonempty finite set of N nodes V = {v 1 , v 2 , . . . , v N }, a set of edges E ⊆ V × V, and the associated adjacency matrix A G . The set N i = {j ∈ V : (i, j ) ∈ E} denotes the neighbor set of node i. The weight is represented by a ij for the edge from node j to node i, and a ij = 0 if there is no data transfer from node j to node i. Considering N nodes on the graph, the adjacency matrix is defined as

A G = [a ij ] ∈ R N ×N .
We define the diagonal in-degree matrix D = diag {d i } ∈ R N ×N , where d i is the sum of communication weights from neighbors of node i,

d i = j ∈N i a ij .
The Laplacian matrix of the graph is defined as L = D -A G . One property of the Laplacian matrix L is that the row sums of L are all zero, because the row sums of D and A G are equal.

A directed path from node i to node j is a sequence of edges depicted as {(v i , v k ), (v k , v l ), . . . , (v m , v j )}. A graph is considered to have a spanning tree if there exists a root node with a directed path from that node to every other node in this graph. A graph is strongly connected if there exists a directed path between every two nodes, i.e., there exists a spanning tree where every node is a root node [START_REF] Bidram | Distributed control systems for small-scale power networks: Using multiagent cooperative control theory[END_REF] [START_REF] Moayedi | Distributed tertiary control of dc microgrid clusters[END_REF].

Distributed Cooperative Control system

In this section, we rely on existing control models to propose a distributed cooperative control scheme for voltage dynamics. We give a detailed description of both the primary and secondary control of the microgrid system under study. For each DG unit, the voltage and phase-angle dynamics can be respectively modeled by a single integrator

Primary Control

τ i Vi (t) = u v i (t), τ θ i θi (t) = u θ i (t), (3.4)
where τ i > 0 and τ θ i > 0 are the inverter's time-constants and u V i (t) and u θ i (t) are the control signals computed by the droop controller at time t ≥ 0. As illustrated in Fig. 1, the measurements and reference signals are available to each controller from the architecture of the control system. Each DG unit is controlled by a droop controller based on the capabilities of the local inverter-based DERs. Each controller receives the reference signal computed remotely and measurements through the communication network. Let V * i be the reference voltage for the i-th bus and V j and θ j , be the voltage magnitude and voltage angle of the j -th bus, respectively. A suitable communication protocol is needed for the transmission of these data, e.g., IEC 61850.

In this chapter, we are mainly interested in the voltage dynamics of the power distribution system. So we consider the following assumption and neglect the phase-angle dynamics throughout the rest of the chapter. Assumption 3. The phase-angle difference θ ij between any neighboring nodes i and j is assumed to be constant.

Note that Assumption 3 could be employed in a local analysis when the phase-angles remain in the neighborhood of the original equilibrium point. In addition, we underline that the assumption is valid if there exists a time-scale separation between the voltage dynamics and the phase-angle.

In order to compute the control output signals, we refer to the voltage quadratic droop controller [START_REF] Simpson-Porco | Synchronization and power sharing for droopcontrolled inverters in islanded microgrids[END_REF] described by

u V i (t) = -κ i V c i (t)(V c i (t) -V c * i (t)) -Q c i (t), (3.5)
where κ i > 0 is the control gain of the droop controller. Additionally, V c i (t), V c * i (t) and Q c i (t) respectively represent the voltage measurement, voltage reference signal with respect to bus i and reactive power injection measurement. They are received by the droop controller, as illustrated in Fig. 1. Under nominal operation, these signals match the corresponding physical variables and reference signals, i.e.,

V c i (t) = V i (t), Q c i (t) = Q i (t), and V c * i (t) = V * i (t) (V * i (t)
is sent by a higher level controller from the substation). The closed-loop dynamics of the i-th DG unit under nominal operation are described by the differential equations

τ i Vi = -κ i V i (V i -V * i ) -Q i = -V i (κ i V i -κ i V * i + j ∈V l ij (θ)V j ), ∀i = 1, . . . , N , (3.6)
with the time argument omitted. Additionally, under the Assumption 2, the parameter l ij (θ) is described as

l ij (θ) = B ij (ρ sin(θ ij ) + cos(θ ij )), i = j -B i , i = j . (3.7) Denoting V = [V 1 . . . V N ] , τ = [τ 1 . . . τ N ] , κ = [κ 1 . . . κ N ]
, and [V ] as the diagonal matrix with V i as the i-th diagonal entry, we can get the voltage dynamics under the quadratic droop control in vector form:

[τ ] V = [V ]([κ]V * -([κ] + L(θ))V ), (3.8)
where the matrix L(θ) is defined as [L(θ)] ij = l ij (θ). Linearization of the voltage dynamics. For convenience of theoretic analysis of system stability and attack impact, we consider the Jacobian linearization of the power system (3.8) around an equilibrium point ( V , V c * ) such that -(

[κ] + L(θ)) V + [κ] V * = 0. Denote x (t) = V (t) -V and u(t) = V c * (t) -V c *
as the voltage and reference deviations, respectively. The corresponding linearized system is described by

ẋ (t) = Ax (t) + Bu(t), (3.9)
where

A = -[ V ][τ ] -1 ([κ] + L(θ)) and B = [ V ][τ ] -1 [κ].
For the sake of simplicity, we suppose that V = 1pu subsequently, where 1 represents a vector with all entries equal to 1.

System properties

In this subsubsection, necessary and sufficient conditions for the linearized power system to be positive are elaborated. These conclusions will play an important role in the impact analysis for the power system under attack in subsequent sections. Firstly, the following assumption is required to derive necessary and sufficient conditions for the linearized system (3.9) to be positive.

Assumption 4. The maximum phase difference between any two neighboring nodes, defined as

∆ θ := max (i,j )∈E |θ ij |, satisfies the inequality ∆ θ < π/2.
For any conventional power system, the constraint ∆ θ < π/2 is required for the stability of the phase-angle dynamics [START_REF] Schiffer | Conditions for stability of droopcontrolled inverter-based microgrids[END_REF]. Under the previous assumptions, we establish the following result for system positivity.

Lemma 3 ([63]

). Consider the power distribution system under study, having active and reactive power injections (3.3) at bus i with ∆ θ < π/2, and applying the quadratic droop controller (3.6) for each DG unit. Then a necessary and sufficient condition for the corresponding linearized system (3.9) to be positive is

ρ ≤ |cot(∆ θ )| . (3.10)
Note that several of the properties of positive systems stated in Subsection 3.2.1 have important consequences in the context of power systems and, in particular, the voltage dynamics. Letting the input u(t) be the voltage reference at one individual bus and x (t) and the voltages of all buses, the closed-loop system being positive implies that an increase in the voltage reference u(t) translates to an increase in all the voltages x (t). Hence, there is no contradictory effect where a desired increase in voltage at one bus inadvertently decreases the voltage in other buses. Additionally, positivity of the closed-loop system also reduces the voltage overshoots in response to step changes in the voltage reference.

Remark 1. While the latter discussion motivates positivity as a desirable system feature, Lemma 2 provides a design objective for the phase-angle controller that ensures positivity of the voltage dynamics, namely the inequality (3.10). In fact, since the line ratio ρ is a system parameter that depends solely on the transmission lines' characteristics, (3.10) can be interpreted as a bound on the phase-angle differences that is parameterized by the line ratio ρ. Rewriting (3.10) as |tan(∆ θ )| ≤ ρ -1 , we have that a resistive system with a large ρ yields a strict bound on the maximum phase-angle difference ∆ θ , while a purely inductive system with ρ = 0 does not constrain the phase-angle difference.

Lemma 4 ([63]

). Suppose the linearized system (3.9) is positive. The system (3.9) is rowdiagonally dominant if, and only if, the following inequality holds

κ i + |B ii | ≥ ( ρ 2 + 1 -1) j ∈N i |B ij | . (3.11)
Theorem 2. Consider the linearized dynamics of the power system (3.9)) and suppose the system is positive. Then the following statements hold: 1) the system is asymptotically stable if and only if there exist positive scalars ξ > 0 such that the following inequality holds for all i = 1, . . . , n;

ξ i |-κ i + B i | > j ∈N i ξ j |-B ij (ρ sin(θ ij ) + cos(θ ij ))| ;
2) the system is asymptotically stable if it is row-diagonally dominant, i.e., the following inequality holds for all i = 1, . . . , n:

|-κ i + B i | > j ∈N i |-B ij (ρ sin(θ ij ) + cos(θ ij ))| ;

Secondary Control

In this work, the goal of the secondary controller is to generate the voltage reference signal for the primary controller at each DG unit. We employ the distributed cooperative control of multi-agent systems [START_REF] Bidram | Distributed control systems for small-scale power networks: Using multiagent cooperative control theory[END_REF] to design the secondary voltage controller for our microgrid system. We simplify the microgrid system to show the topology of the system communication graph G in Fig. 3.3.

DGs are assumed to be able to communicate with each other through the communication network G. We assume that only one leader DG node i has access to the reference V * by a weight factor called the pinning gain g i . The pinning matrix G is defined to carry all the pinning gains of the graph:

G = diag {g i } ∈ R N ×N .
(3.12)

The computation of the global reference voltage V * is based on the load deferences between loads in this microgrid and its neighbors, as specified in [START_REF] Moayedi | Distributed tertiary control of dc microgrid clusters[END_REF].

The synchronization of reference values of DGs in a communication network G can be modeled as a tracking problem of multi-agent cooperative control systems [START_REF] Bidram | Distributed control systems for small-scale power networks: Using multiagent cooperative control theory[END_REF]. The goal of a cooperative tracking problem is to have consensus for the agent variables. However, unlike the synchronization problem, the consensus value is a control objective usually provided by the leader. One can state the tracking problem as x = 1 • x ref , where x ref is the reference value determined by the leader. The leader is connected to a limited number of agents. A pinned graph is used to model the tracking problem. In Fig. 3.3, the leader node is set to be DG 1 and the consensus is the reference value V * . To reach the consensus at the reference value, the graph should contain at least one pinned root node. Then, the control agents cooperate to hold

ẋ = -(L + G)(x -1 • x ref ) (3.13)
at the steady state. Since all eigenvalues of L+G have positive real values, equation (3.13) results in the desired tracking performance at the steady state. It is noteworthy that the reference value can also have dynamics. However, to have acceptable tracking performance for all agents, the dynamics of the tracking equation (3.13)should be much faster than those of the reference.

We assume that only one leader node i acting as the command generator has access to the reference V ref with pinning gain g i , and all other nodes synchronize to node i. The value V ref can be determined by keeping a critical load operating at the nominal voltage V nominal [START_REF] Bidram | Distributed adaptive voltage control of inverter-based microgrids[END_REF]. Hence the secondary controller computes V * as

V ref = κ p (V nominal -V critical ) + κ i (V nominal -V critical )dt (3.14)
where V critical is the critical load voltage and κ p and κ i are the controller gains. Note the voltage reference vector

V * = [V * 1 . . . V * N ] .
Consider the tracking problem in equation (3.13) and we set x = V * in the secondary control problem. Then the reference synchronisation process of all DG units follows

V * = -(L + G)(V * -1 • V ref ) (3.15)
at the steady state. Equation (3.15) leads to the desired tracking performance of each DG unit at the steady state. Note the adjacency matrix of the communication network

G is A G = [a ij ] ∈ R N ×N .
The voltage reference setpoint of each DG unit can be written as

V * i = j ∈N i a ij (V * j -V * i ) + g i (V ref -V * i ).
(3.16)

Cyber attacks on voltage control and Impact Analysis

In this section we consider the model of the microgrid system described in section 3.2 and we study the possible adversary actions on the reference signal transmitted by the communication network at the primary and secondary control levels.

It is natural to consider a naive attack where an attacker simply falsifies the reference voltage value V * i for DG i. However, it is easy to be detected by simple detection algorithms [START_REF] Isozaki | On detection of cyber attacks against voltage control in distribution power grids[END_REF] considering the admissible range

V * min ≤ V * i ≤ V * max , (3.17)
where V * min and V * max are respectively the lower and upper limit for the reference value. In this work, we identify a novel type of of attack scenarios: "measurement as reference " attack. Under the assumption that the attacker has knowledge about the hierarchical control structure, it targets the communication link between DG units and maliciously replaces the reference signal with the measurement of the previous node. Since the two signals have very close values and follow the same dynamic changes, this attack is "naturally stealthy", and is possible to cause a serious impact without being detected by traditional intrusion detection algorithms.

First we describe the considered attack scenario and we give its mathematical definition. Then we describe how the attack influences the voltage control systems. We characterize the attack impact by the maximum voltage magnitude deviation and inaccurate reference voltage synchronization it causes.

Measurement as reference attack

Without loss of generality, we assume the attacker replaces the reference signal V * i+1 for node i + 1 with the voltage measurement V s i of node i when defining the attack. The goal of measurement as reference attack is set to cause voltage fluctuations and irregular regulations to harm the loads without violating the admissible range (3.17). For convenience of analysis, we assume only one communication link is under attack. The definition of measurement as reference attack is given as follows. 50

Cyber attacks on voltage control and Impact Analysis

Definition 3 (measurement as reference attack). In a measurement as reference attack on the communication link between DG unit i and i + 1, the attacker manipulates the exchanged data by replacing the reference signal V * i+1 (t) for DG unit i + 1 with the voltage measurement V s i (t) of DG unit i , so that

V * i+1 (t) = V s i (t), (3.18)
Furthermore, the droop control signal at DG unit i + 1 under attack is given by

τ i Vi+1 (t) = -κ i+1 V c i+1 (t)(V c i+1 (t) -V s i (t)) -Q c i+1 (t) (3.19)

Attack Impact on Voltage Deviation

One impact of the measurement as reference attack is voltage magnitude deviations . Consider the resulting changes to the voltage magnitude at DG j in the network, i.e., V j . The resulting linearized system under attack can be written as (3.20) where A = -[τ ] -1 (κ + L(θ)) and e i ∈ R N is the i-th column of the N -dimensional identify matrix. The attack impact can be quantified as the maximum deviation of y j (t) caused by a corrupted input u(t) = e i x (t), which is bounded by |u(t)| ≤ δ, where V * min ≤ δ ≤ V * max . Computation of maximum voltage deviation. We use sup t≥0 |y j (t)| to represent the maximum voltage magnitude of DG j under a measurement as reference attack (3.18) on the link between DG i and i + 1. We know that the linearized system (3.20) under attack is a positive system [START_REF] Rantzer | Scalable control of positive systems[END_REF] since A is a Metzler matrix with non-negative off-diagonal entries and τ -1 i+1 κ i+1 e i+1 , e j are non-negative [START_REF] Teixeira | Voltage control for interconnected microgrids under adversarial actions[END_REF]. Let's define the transfer function of the system (3.20) as H (s) = Y (s)/U (s) = L {y(t)} /L {u(t)}, where L means Laplace transform. Thus we can compute

ẋ (t) = Ax (t) + τ -1 i+1 κ i+1 e i+1 u(t) y j (t) = e j x (t) u(t) = e i x (t),
H (s) = τ -1 i+1 κ i+1 e j (sI -A) -1 e i+1 . (3.21)
According to the properties of positive systems [START_REF] Rantzer | Scalable control of positive systems[END_REF], we know that the L ∞ -induced norm of system (3.20) is given as

H L∞-ind = sup u L∞ <∞ y L∞ u L∞ = H (0), (3.22)
which characterizes the maximum amplitude of the output signal y j (t) which can be achieved by an input u(t) with bounded amplitude

u L∞ := sup t≥0 |u(t)| ≤ δ. (3.23) So we can compute sup t≥0 |y j (t)| = H L∞-ind • u L∞ ≤ H (0) • δ = -τ -1 i+1 κ i+1 δe j A -1 e i+1 = τ -1 i+1 κ i+1 δ[-A -1 ] j ,i+1 , (3.24)
where [-A -1 ] j ,i+1 is the element of row j column i+1 from matrix -A -1 . So τ -1 i+1 κ i+1 δ[-A -1 ] j ,i+1 gives the upper bound of the voltage deviation caused by the attack. Since the δ value in (3.23) is determined by the system state, the upper bound in (3.24) is tight because the maximum is reachable. Therefore, we find that the impact on voltage deviation of each node depends on the structure of matrix A, which reflects the power system topology.

Impact on Reference Voltage Synchronization

Besides the voltage deviation, the attack also affects the reference synchronization regarding the secondary control level. Note that the reference of each DG unit will follow the differential equation (3.15) under nominal operations. So the problem of studying what is the attack impact on reference synchronization is mapped to seeking for the initial conditions of differential equation (3.15) caused by the attack.

Since the primary control is operating much faster than the secondary control, the voltage state of each DG node would become stable again very quickly and reaches a new equilibrium point when the secondary controller regulates the reference signals. To compute this equilibrium point, we let V = 0 in (3.8)

, i.e., [κ]V * i -([κ] + L(θ))V = 0.
According to the definition of measurement as reference attack V * i+1 (t) = V s i (t), we have

V * i+1 = V i = e i ([κ] + L(θ)) -1 [κ]V ref . (3.25)
So we can get the initial conditions of differential equation (3.15) under attack as

(e i ([κ] + L(θ)) -1 [κ] -e i+1 )V ref = 0. (3.26)
The differential equation (3.25) with initial condition (3.26) can be written as

Vref = -(L + G)(V ref -1 • V * ) (e i ([κ] + L(θ)) -1 [κ] -e i+1 )V ref = 0. (3.27)
Equation (3.27) describes the reference signal synchronization process under attack for each node. When the secondary controller regulates the set point V * , all references of the nodes relying on communication with node i +1 for synchronization, are not able to reach the consensus V * .

Simulation System and Validation

In this section, we use simulation tools to validate the theoretical analysis of the measurement as reference attack and its impact proposed in the previous section.

Simulation Settings

We use Matlab and Simulink tools to build the microgrid control system described in Figure 1 and model the measurement as reference attack provided in Definition 3.

As shown in Fig. 3.4, we build a simulink model with different modules to simulate the distributed and cooperative controlled microgrid system. The module microgrid contains derivative blocks to simulate the primary and secondary controllers as described in Section 3.3. The module x _out represents the observed state vector, i.e., the DG voltage measured by each sensor at the related DG unit. The module y_out represents the output vector. The signals V _ref , Q_load , P _load and Attack _ref represent the voltage reference signal, injected reactive power, injected positive power and attack signal, respectively. For the power system simulation settings, we consider a microgrid system with 4 DG units as shown in Figure 1, i.e., N = 4. We follow the model settings proposed in [START_REF] Teixeira | Voltage control for interconnected microgrids under adversarial actions[END_REF] and [START_REF] Ma | Voltage control in distributed generation under measurement falsification attacks[END_REF], where all power lines, DERs and loads are identical. The power system is characterized by the equation (3.3) with parameters: ρ = 0.5, B ij = -0.2, and G ij = -ρB ij for all edges (i, j ) ∈ E and B ij = -0.2 and G ii = -ρ |B ii | for all buses.

For the primary control, we set the parameters θ 12 = -0.01rad, θ 23 = -0.045rad, and θ 34 = -0.01rad, to make sure that the phase-angle differences between any neighboring nodes are constant. The quadratic droop control modeled by equations (3.8) and (3.5) is characterized by parameters τ i = 10 -4 , τ θ i = 10 -2 , and κ i = 0.2 for all buses.

The voltage dynamics under primary control are defined by the nonlinear differential equations (3.8). Through Jacobian linearization, the corresponding linearized dynamics characterized by equation (3.9) is given as

A = 10 -4 •      -4.01 1.88 0 0 2.1 -6.01 2.04 0 0 1.95 -6.01 1.88 0 0 2.1 -4.01      .
Clearly the above system is positive and the properties of positive systems [START_REF] Rantzer | Scalable control of positive systems[END_REF] are applicable. Finally, we choose typical parameters for the secondary control. As depicted in Figure 1, the network is a 4-node line topology and DG 1 is the only pinned root node. Considering the communication network topology and a proper synchronization rate, we set the adjacency matrix A G = [a ij ] carrying the communication weights of a 21 = 12, a 32 = 11, a 41 = 10 and all other weights equal to zero, and the pinning matrix G = diag {g i } carrying all pinning gains of g 1 = 10, g 2 = g 3 = g 4 = 0.

Simulation Results

Considering the above simulation settings, the voltage of each DG is in stable state and the global reference set point V * = 1pu. At time t = 2 × 10 -3 s, the attacker introduces the measurement as reference attack by replacing the reference signal V * 2 at node 2 with the voltage measurement V 1 at node 1. As shown in Figure 3.5, the attack cause voltage deviation at each DG unit, which is a short-term impact on the primary control. We can see the voltages reach a stable state again very quickly. Comparing the voltage deviation of the simulated non-linear system under attack with the linearization analysis in Section 3.4, we can find that the voltage deviation caused by the attack is within boundary computed in (3.24). Comparison result shows that the voltage deviation of the simulated non-linear system under attack is the same to the linearized system, which shows our linearization analysis in Section 3.4 is valid.

How the attack influences the reference synchronization is shown in Figure 2. At time t = 0.1s, the secondary controller regulates the reference set point from V * = 1pu to V * = 1.05pu. After about 0.5s, the reference signals finish the synchronization process. We find that DG 2 and DG 3 no longer reach the consensus V * , which could lead to a heavy impact on voltage regulation in a microgrid system.

Summary

In this chapter we present the design of a novel attack named Measurement as Reference (MaR) attack targeting the distributed control system of microgrid. In this attack that we instantiated on voltage control, the attacker introduces a reference voltage deviation by manipulating the synchronization data exchanged between the distributed controllers. We show the impact of this synchronisation attack by simulation and theoretical analysis. We use control-theoretic tools to derive the maximum voltage deviation introduced by this attack where the system will not synchronize to the correct setting point provided by the secondary controller. However, a distributed control system is also subject to other attacks including delay injection and denial of service attacks, that also severely impact the operations of this system. The detection of the attacks is still challenging because of the stealthy nature of some of them where the deviations from nominal values are very low. This challenge is the main motivation of the following chapter that proposes a more robust method based on Machine Learning algorithms to detect these attacks including the MaR attack.

Introduction

Several existing studies highlight the requirements for a comprehensive attack resilient control system for microgrids that simultaneously: (i) expedites the proper voltage stability and regulation; (ii) handles attacks on the distributed cooperative control systems; (iii) deals with both FDI and DoS attacks; (iv) is applicable to resource-limited small-scaled distributed controllers.

To this end, this chapter proposes a resilient distributed cooperative control for microgrids that accommodates FDI and DoS attacks detection. Compared to the existing attack detection methods based on state estimation [START_REF] Liu | False data injection attacks against state estimation in electric power grids[END_REF][START_REF] Kosut | Malicious data attacks on the smart grid[END_REF], statistical machine learning algorithms outperform them in detecting both observable and unobservable attacks [START_REF] Ozay | Machine learning methods for attack detection in the smart grid[END_REF]. The attack detection problem can be modeled as a typical classification problem to learn communication patterns at the network-level of the distributed control system and predict the class of a packet as attack or normal . We study various groups of supervised machine learning algorithms to design a comprehensive attack resilient control framework, in particular to detect FDI and DoS attacks in microgrids. In this chapter, we make the following contributions [START_REF] Ma | Detecting a stealthy attack in distributed control for microgrids using machine learning algorithms[END_REF]:

• We model and generalize both FDI and DoS attacks in distributed and cooperative control microgrid systems. We also give a general system stability condition analysis under FDI and DoS attacks.

• We propose a secure control framework with attack detection and response modules relying on machine learning and deep learning algorithms. Various groups of supervised machine learning and deep learning algorithms are studied to design the attack detection module against FDI attacks and DoS attacks under well-chosen performance metrics.

• Considering it is difficult to find publicly available datasets, we build an experimental platform emulating the distributed and cooperative controlled microgrid systems, and implement two typical FDI and DoS attacks hard to be detected by traditional residual comparison detectors. The two attacks are: measurement as referent (MaR) attack as typical FDI attack and delay injection attack as typical DoS attack. Balanced and imbalanced dataset are collected to under normal and attacked conditions to be further analyzed.

The rest of this chapter is organized as follows. Section 4.2 gives a general modelling of FDI and DoS attacks on the distributed and cooperative control microgrid system and formulate the problem under study. Section 4.3 proposes a detailed secure control framework. Section 4.4 describes the hardware testbed we designed to realize the distributed and cooperative control and implement the designed attacks. At last, Section 4.5 summarizes this chapter.

Attack Modeling and Problem Formulation

In this section, we describe a generalized attack model for the proposed distributed and cooperative control system described in Chapter 3. Both FDI and DoS attack scenarios are considered.

Modeling of FDI and DoS Attacks

To build general models for both FDI and DoS attacks with regard to the distributed and cooperative control system with quadratic nonlinearity, we adopt Jacobian linearization to study the droop controlled DG units (3.8) around an equilibrium point ( V , V * ). As described in equation (3.9), -(

[κ] + L(θ)) V + [κ] V * = 0 must hold at the equilibrium point. Note that state matrix A = -[ V ][τ ] -1 ([κ] + L(θ)) and input matrix B = [ V ][τ ] -1 [κ].
For the sake of simplicity, let V = 1pu subsequently, where 1 denotes a vector with all entries equal to 1.

To better describe the system dynamics of the microgrid system, we introduce additional terms ω(t) and y(t) to the linearized dynamic system (3.9):

ẋ (t) = Ax (t) + Bu(t) + ω(t) y(t) = Cx (t) (4.1)
where ω(t) ∈ R n is the Gaussian process noise representing the system errors. Furthermore, C ∈ R m×n is the output matrix and the output y(t) ∈ R m is observed by a sensor network with m sensors. FDI and DoS attacks are the two major types of attacks in CPS systems. FDI attacks target data integrity, where the adversary forwards false data from one or more corrupted sensors or controllers. The injected false data could cover incorrect measurements, timestamps, or sender/receiver IDs. FDI attacks can be launched by breaking the underlying cryptography system or compromising some sensors/controllers or the communication links between them. The MaR that we designed and validated through simulation in Chapter 2 is an example of this kind of attacks where the attacker modifies the transmitted reference value by the measurement value. We have shown that such attack deviates the voltage control and the system becomes unstable.

On the other hand, DoS attacks target system availability. The adversary makes the sensor measurements unavailable to the controllers or the control commands unavailable to the actuators. DoS attacks can be launched by jamming the communication links, delaying or dropping data packets, modifying the network routings, data flooding on the network, etc. A well known DoS attack in microgrids is delay injection attacks where the attacker introduces some communication delay to the travelling packets in the network to make the valid data received with time-delays. The introduced delay affects the control system behavior, which is time critical, and creates instability in the power system that may compromise its availability.

We propose a general framework to model both FDI and DoS attacks against the microgrid system by using additive changes to (4.1):

ẋ (t) = Ax (t) + Bu(t) + ω(t) + Γr (t) y(t) = Cx (t) + Ψz (t) (4.2)
where the attack vectors r (t) and z (t), and their respective coefficient matrices Γ and Ψ are to be determined by the FDI/DoS attack settings. We assume that the attack starts at time t = t 0 , and we introduce the unit step function 1 {t≥t 0 } to model this assumption, where 1 {t≥t 0 } = 0 for t < t 0 and 1 {t≥t 0 } = 1 for t ≥ t 0 .

For FDI/DoS attack on a subset U of control signals, Γ and r (t) in (4.2) are determined as

Γ = B r i (t) = (-u(t) + α i (t))1 {t≥t 0 } , ∀i ∈ U 0, ∀i / ∈ U (4.3)
where α i (t) is the corrupted control signal sent by the attacker. Note that α i (t) = 0 for FDI attack and α i (t) = 0 for DoS attack.

Similarly, for FDI/DoS attack on a subset Y of sensor nodes, Ψ and z (t) in (4.2) are determined as

Ψ = C z i (t) = (-x ( t) + β i (t))1 {t≥t 0 } , ∀i ∈ Y 0, ∀i / ∈ Y (4.4)
where β i (t) is the corrupted sensor measurements sent by the attacker. Note that β i (t) = 0 for FDI attack and β i (t) = 0 for DoS attack.

General Stability under Attack

Besides the general modeling of FDI and DoS attacks on the microgrid system, any concrete attack scenario can be further written as

ẋ (t) = A a x (t) + Bu a (t) + ω(t) (4.5)
where A a is the corrupted state-transition matrix and u a (t) is the manipulated input by the attacker.

Hereby we give the necessary and sufficient condition for the stability under a concrete attack (4.5).

Theorem 1. Consider the linearized dynamics under attack (4.5), and then the system is asymptotically stable if and only if the following conditions hold:

1) A a is a row-diagonally dominant Metzler matrix, i.e., |ã ii | ≥ j =i |ã ij | , ∀i, j ∈ {1, . . . , N },
where ãij = A ij , and ãij ≥ 0 for i = j .

2) There exists a ξ ∈ R n such that ξ > 0 and A a ξ < 0.

Proof. The two statements in Theorem 1 get A a is Hurwitz, i.e., every eigenvalue of A a has a strictly negative real part [START_REF] Ma | Voltage control in distributed generation under measurement falsification attacks[END_REF]. So the necessary and sufficient condition for stability under attack (4.5) follows directly from Lyapunov's first method of stability criteria.

Secure Control Framework

In this section, we propose a comprehensive secure control framework relying on statistical machine learning algorithms to detect attacks targeting microgrid systems. We explore various machine learning algorithms to design the attack detection module against observable/stealthy FDI attacks and DoS attacks modeled as (4.3) and (4.4). As shown in Fig. 3, we illustrate a secure control framework with attack detection and response modules at each DG unit. Instead of building a centralized attack detection and response system, the distributed and cooperative DG units can actively defend and maintain the integrity and availability of the microgrid system.

Architecture

A detailed description of the architecture and the components of the attack detection and response modules is shown in Fig. 4.2. The proposed architecture contains the following four main stages: 1) Data Collection. Data collection is the process of gathering and measuring information on targeted variables in the system. All incoming packets are captured and stored as sets of traffic flow. Each sample is described by a set of features and expressed in the vector space model.

2) Preprocessing. Data preprocessing is that step in which the data gets transformed, or encoded, to bring it to such a state that is easy to parse. In other words, the features of the data can now be easily interpreted by the algorithm. It extracts information about packet connections from data and constructs new statistical features. A feature is an individual measurable property or characteristic of a phenomenon being observed. Standardization is needed if there is a necessity to eliminate the effect of scale differences of features.

3) Classification. Dataset of particular class is split into subsets. Different machine learning and deep learning classifiers are trained by using labeled datasets. Cross-validation is used for manipulating training data to subdivide the training data into k disjoint subsets and to reconstruct training sets by leaving out some of the subsets.The performance of the classifiers are tested on online data. Different groups of machine learning and deep learning algorithms are chosen as the candidate of the "Classifier" module. Comparing the performances of each algorithm and the algorithm with the best performance in terms of certain metrics will be determined as the final classifier. As shown in Algorithm 1, we iteratively evaluated a candidate set of known ML techniques to identify the best technique that is able to detect the running attacks with high performance.

4)

Response. Train classifier i with S 1 ;

4:

Test classifier i with S 2 ;

5:

Calculate the F-Measure F 1 i ;

6: if F 1 i > F 1 then 7: F 1 = F 1 i ; 8: d * = i; 9:
end if 10: end for 11: return d * ; to update their respective filtering modules. Data packets from suspicious nodes available in this blacklist could be dropped to prevent possible damages to the microgrid system. As shown in Algorithm 2, we propose a detailed response mechanism triggered by each DG unit after receiving labeled packets steam from the previous the classification module. When the boolean value Drop_packet = True, it means the DG unit will simply drop the received packet. Similarly, when the boolean value Send _List_to_neighbours = True, it means the DG unit will spread the updated hash table to its neighbouring nodes.

Algorithm 2

The response mechanism of the DG unit after receiving labeled packets form the classification module. Input: A packets stream P = [P 1 , P 2 , P 3 , ...], where P i = {P i _value, P i _label , P i _ip_address, P i _port_number }, the hash table List; Output: The updated hash table List, the boolean values Drop_packet and Send _List_to_neighbours; 1: Initialize Hashtag = 0, Drop_packet = False, Send _List_to_neighbours = False; 2: for P i ∈ P do 3:

if P i _label = "Attack " then 4:
Hashtag = Hash(P i _ip_address, P i _port_number );

Selected Machine Learning algorithms

The classification module that we described in the previous subsection relies mainly on a classification method to detect attack packets from the network traffic exchanged between the DGs of the microgrid. From a statistical learning perspective, attack detection can be modeled as a typical classification problem that maps each network packet transmitted between DG units to a label, including normal or attack labels and the attack type. Many machine learning algorithms have been already used for the detection of attacks in communication networks of microgrid systems. They use network traffic, control and sensing features as input of their classification methods.Therefore, there is no best classification model for all applications but some will perform better than others in our case.

Since we have full knowledge of the FDI and DoS attack models as elaborated in (4.3) and (4.4), we can implement the attacks and collect labeled datasets of them, which is usually a difficult task to realize in real life. Thus, we can explore supervised machine learning algorithms and state-of-the-art deep learning algorithms to validate their effectiveness for attack detection in microgrid systems.

Considering that it is usually tricky to find one rule (or feature set) which outperforms other rules (or features) of particular classification methods, we also test ensemble learning algorithms. Therefore, we select seven different machine learning classifiers grouped into five categories, and also one deep learning algorithm represented by convolutional neural networks, as shown in

Bayesian Algorithms

The Naive Bayes classifier is a classical probabilistic classifier with strong independence assumptions between the features. It demonstrates how generative assumptions and parameter estimations can simplify the learning process [START_REF] Hastie | The Elements of Statistical Learning[END_REF]. The Naive Bayes classifier requires a number of linear parameters in the number of variables (features or predictors) in the learning problem, thus it is highly scalable. By evaluating a closed-form expression, the Naive Bayes classifier can do maximum-likelihood training in linear time, in contrast to expensive iterative approximation that is used by many other types of classifiers.

Instance-based Algorithms

This category of algorithms do not derive abstractions for the classification problem, instead they rely on the similarity of an instance to its nearest neighbours available in the training set to predict a class label. In this category, we used two algorithms: K-Nearest Neighbor (KNN), Support-Vector Machine (SVM).

• The KNN classifier finds k samples in the training data that are closest to the test sample, and labels the new sample with the most frequent label among these samples. The KNN classifier outputs a class membership. Based on the plurality vote of its neighbors, an object is classified to the most common class among its k nearest neighboring objects, where k is positive integer and typically small. When k = 1, then the classifier simply assigns the object to the class of its single nearest neighbour [START_REF] Hastie | The Elements of Statistical Learning[END_REF][START_REF] Shalev | Understanding machine learning: From theory to algorithms[END_REF].

• SVM is used for classification by constructing a hyperplane or set of hyperplanes in a high or infinite-dimensional space. An SVM model is a representation of the examples as points in space, mapped so that the examples of the separate categories are divided by a clear gap that is as wide as possible. New examples are then mapped into that same space and predicted to belong to a category based on the side of the gap on which they fall [START_REF] Ben-Hur | Support vector clustering[END_REF][START_REF] Shalev | Understanding machine learning: From theory to algorithms[END_REF]. To keep the computational load reasonable, the mappings used by SVM schemes are designed to ensure that dot products of pairs of input data vectors may be computed easily in terms of the variables in the original space, by defining them in terms of a kernel functionk (x , y) selected to suit the problem. The hyperplanes in the higherdimensional space are defined as the set of points whose dot product with a vector in that space is constant, where such a set of vectors is an orthogonal (and thus minimal) set of vectors that defines a hyperplane. The vectors defining the hyperplanes can be chosen to be linear combinations with parametersα i of images of feature vectors x i that occur in the data base. With this choice of a hyperplane, the points x in the feature space that are mapped into the hyperplane are defined by the relation i α i k (x i , x ) = constant. Note that if k (x , y) becomes small as y grows further away from x , each term in the sum measures the degree of closeness of the test point x to the corresponding data base point x i . In this way, the sum of kernels above can be used to measure the relative nearness of each test point to the data points originating in one or the other of the sets to be discriminated. Note the fact that the set of points x mapped into any hyperplane can be quite convoluted as a result, allowing much more complex discrimination between sets that are not convex at all in the original space.

Decision Tree Algorithms

Decision Tree has a structure like a flowchart, where each internal node denotes a test on a data attribute, each branch denotes the outcome of the test, and each leaf node denotes a class prediction. A decision tree is a predictor that predicts the label associated with an instance x by traveling from a root node of a tree to a leaf. At each node on the root-to-leaf path, the successor child is chosen on the basis of a splitting of the input space. Usually, the splitting is based on one of the features of x or on a predefined set of splitting rules. A leaf contains a specific label. A decision tree is built by splitting the source set, constituting the root node of the tree, into subsets. which constitute the successor children. The splitting is based on a set of splitting rules based on classification features. This process is repeated on each derived subset in a recursive manner called recursive partitioning. The recursion is completed when the subset at a node has all the same values of the target variable, or when splitting no longer adds value to the predictions [START_REF] Shalev | Understanding machine learning: From theory to algorithms[END_REF].

Artificial Neural Network (ANN) Algorithms

Multilayer Perceptrons (MLP) a class of feedforward ANN optimizing the weights for the activation function of neurons organized in a network architecture. An MLP consists of at least three layers of nodes: an input layer, a hidden layer and an output layer. Except for the input nodes, each node is a neuron that uses a nonlinear activation function. MLP utilizes a supervised learning technique called backpropagation for training. Back-propagation learning may be implemented in one of two basic ways, sequential mode and batch mode. In sequential mode, adjustments are made to the free parameters of the network on an example-by-example basis. The sequential mode is best suited for pattern classification. In batch mode, adjustments are made to the free parameters of the network on an epoch-by-epoch basis, where each epoch consists of the entire set of training examples. The batch mode is best suited for nonlinear regression [START_REF] Shalev | Understanding machine learning: From theory to algorithms[END_REF]. To address overfitting and improve the neural network, L2 regularization can be applied as regularization methods for neural networks.

Ensemble Algorithms

This category of algorithms relies on a set of estimators, in particular decision tree, to build a multi-stage classifier and predict the class label according to the output of this set of estimators. These algorithms apply averaging or boosting methods to produce the final prediction. in this category, we applied two algorithms: AdaBoost and RandomForest.

• AdaBoost classifiers implement boosting to combine "weak classifiers" into a single "strong classifier". AdaBoost classifier builds a strong classifier by combining multiple poorly performing classifiers so that you will get high accuracy strong classifier. The basic concept behind AdaBoost is to set the weights of classifiers and training the data sample in each iteration such that it ensures the accurate predictions of unusual observations. Any machine learning algorithm can be used as base classifier if it accepts weights on the training set. AdaBoost classifiers should be trained interactively on various weighed training examples. In each iteration, it tries to provide an excellent fit for these examples by minimizing training error [START_REF] Rojas | Adaboost and the super bowl of classifiers a tutorial introduction to adaptive boosting[END_REF][START_REF] Shalev | Understanding machine learning: From theory to algorithms[END_REF].

• Random Forest classifiers consist of a collection of decision trees, where each tree is constructed by applying an algorithm A on the training set S and an additional random vector, θ, where θ is sampled independent and identically distributed from some distribution [START_REF] Shalev | Understanding machine learning: From theory to algorithms[END_REF]. The prediction of the random forest is obtained by a majority vote over the predictions of the individual trees. As a meta estimator, Random Forest fits multiple decision tree classifiers on various sub-samples of the dataset and improves its predictive accuracy by averaging. Random Forest adds additional randomness to the model, while growing the trees. Instead of searching for the most important feature while splitting a node, it searches for the best feature among a random subset of features. This results in a wide diversity that generally results in a better model.Therefore, in Random Forest, only a random subset of the features is taken into consideration by the algorithm for splitting a node. We can even make trees more random by additionally using random thresholds for each feature rather than searching for the best possible thresholds (like a normal decision tree does).

Deep Learning algorithms

More recently, few work have applied [START_REF] Kravchik | Detecting cyber attacks in industrial control systems using convolutional neural networks[END_REF] deep learning algorithms, in particular Convolutional Neural Networks (CNNs) , for attack detection in cyber-physical systems. CNNs are feedforward neural networks that became popular in image processing. By applying convolutions to small regions of the input, CNNs can significantly improve the efficiency of neural networks by avoiding performing matrix multiplication over the entire image at once. Unlike two-dimensional (2D) CNNs used in image processing, 1D CNNs can be successfully used for time series processing because time series have a strong 1D (time) locality which can be extracted by convolutions. The work in [START_REF] Kravchik | Detecting cyber attacks in industrial control systems using convolutional neural networks[END_REF] shows that 1D CNNs can be effectively used for detecting cyber-attacks in complex multivariate ICS data. To compare with the above machine learning algorithms, we also selected 1D CNNs to evaluate its performance for detecting attacks in mircorgrids.

Performance metrics

The performance evaluation of the selected ML algorithms relies on a set of widely used metrics that we will explain in this subsection. The evaluation of these metrics will allow us to identify the best ML algorithms that are able to detect attack on microgrid systems, including the MaR attack that we described in Chapter 3. In binary classification, we indicate data under attack as positive, and normal data as negative. Denote tp as the number of true positives, fp as the number of false positives, tn as the number of true negatives, and fn as the number of false negatives. We measure the learning capabilities of the algorithms by three classical metrics which are: Precision, Recall and F-Measure. The Precision measures the performance in predicting positive samples, which is known as positive predictive value (PPV).

Precision = tp tp + fp

On the other hand, Recall measures the capability to identify all the positive samples, which is also called true positive rate (TPR).

Recall = tp tp + fn

Finally, F -Measure is the harmonic mean of Precision and Recall and captures both Precision and Recall properties as a combined single measure. Note that we choose F-Measure over another standard metric Accuracy as the total classification performance metric because F-Measure gives a better measure when the class distribution is imbalanced like in the attack detection problem.

F -Measure = 2 • Precision • Recall Precision + Recall
A classifier has a strong performance when the above measures are approaching 1. To evaluate the classification performance of each learning algorithm, we also consider receiver operating characteristic (ROC) curves by plotting the true positive rate (TPR) against the false positive rate (FPR), where TPR = tp tp+fn , FPR = fp fp+tn . For each ROC curve, its area under the curve (AUC) directly represents the classification performance where a better classifier has a larger AUC value.

When running the classifiers, we employ k -fold (e.g., k = 5) cross-validation techniques to protect against overfitting during training. We partition the dataset into k randomly chosen subsets of roughly equal size. We use one subset to validate the performance of the classifier model trained from the remaining subsets. This process is repeated k times to guarantee that each subset is used exactly once for validation.

Experimental platform and data collection

Our proposed framework relies on a ML algorithm for the detection of attacks which require the collection of several datasets of normal and attack samples to be able to evaluate the candidate ML algorithms and select the algorithms that perform better than the others. However, it is difficult to find publicly available datasets for the two proposed attacks. For that reason, we implemented two types of attacks targeting the communication links and instantiated them on a hardware platform built for emulating the distributed and cooperative controlled microgrid system. We used this platform to collect several datasets by running it in normal and attack modes.

Microgrid testbed

As shown in Fig. 4.4, we built a hardware testbed modeling a simplified microgrid platform consisting of 4 DG units. • a Raspberry Pi as the main control center of the DG unit;

• an Arduino with a motor shield runing a PID control algorithm to drive the DC motor A;

• two DC motors, where DC motor A drives DC motor A to generate electrical power;

• a voltage sensor measuring the voltage at each DG unit;

• a lightbulb serving as the domestic load.

For each DG unit, one Raspberry Pi and one Arduino board act as the primary controller, two motors A and B act as a power generator, and one lightbulb acts as an electrical load. The primary control equivalent to (3.6) is realized in the Arduino board with the help of PID settings. The Raspberry Pi devices are connected through a WiFi network and only communicate with its neighbor using unencrypted messages. DG 1 acts as the leader node and has access to the secondary control voltage instructions, and the remaining DG units exchange information with neighboring nodes and synchronize their voltage value to DG unit 1.

The distributed control system

In order to give the control instructions from a DG to another (concretely from a Raspberry Pi to its neighbouring Raspberry Pi), it is necessary to be able to run simultaneously a server and a client on the same DG. The server part receives the control instructions (the measurement and the reference values) whereas the client gives it to the next node of the tree topology.

The server listening on a specific port, waits for clients to connect and then handles the packets it gets from them. In our case, the server just accepts the connections of the clients and receives their messages. Figure 4.6 shows the different connections between the clients and the servers running the 4 DGs to propagate the control instructions between them and synchronise to the desired voltage. To inject a control instruction in the network which is in our case the desired voltage, an other kind of client is needed, which serves as the secondary controller. Its objectives are the same as the others but the main difference is that it does not wait for receiving an instruction from a previous node because it does generate the control instruction. The user is asked for the instruction value (it must be between 0 and 12V -0 is excluded because the Arduino will not accept it). The rest of the function is similar to what the client function does.

Attacks implementation

We implemented two specific attacks on our platform to illustrate the FDI and the DoS attacks modeled in Section 4.2.1:

• Measurement-as-reference (MaR) attack: the attacker intercepts the communication link between DG unit i and unit i + 1 to compromise the transmitted data packets, and maliciously substitutes the reference value V * i+1 (t) of DG unit i + 1 with the voltage measurement V i (t) of DG unit i.

• Delay injection attack: the attacker injects some delays according to a specific statistical distribution to the packets exchanged between DG unit i and unit i +1 to perturb the communication. In this work, we consider a Gaussian-distributed delay injection attack with a proper mean and variance to cause loss of system availability without being observed, because it is tricky to distinguish delay injection attack from naturally generated fault delay.

We use an open-source tool "Saboteur"1 to inject delays into the communication links.

MaR attack implementation with Man-in-the-Middle technique

For MaR attack, the reference value V * i+1 (t) is close to the voltage measurement V i (t), and they both transmit in the same link. MaR attack is naturally stealthy in front of traditional residual comparison detectors and can cause voltage fluctuation and synchronization errors [START_REF] Ma | On the impact of synchronization attacks on distributed and cooperative control in microgrid systems[END_REF]. We implemented the MaR attack by using a Man-in-the-Middle (MitM) technique where the attacker intercepts the packets exchanged between two successive DGs and substitues the reference value with the measurement. The attack tool is running on a specific machine connected to the same network as the DGs. To implement the attack tool, we relied on the Scapy library for packet capture and their modification before injecting them into the network. • ARP Poisoning. First, the intruder must trick the targets into thinking the one they want to talk to is him. To do so, he takes advantage of the ARP protocol to send counterfeit ARP packets to both targets : the sender and the receiver. The intruder modifies their ARP tables in order to make them send the packet to him when they want to contact their peer.

• Packets sniffing. Once the ARP Poisoning is launched, the intruder is in the middle of the conversation between the victims. He can observe the packets but of course alongside the packets sent through the socket, many other exist and are captured by the sniffer.

Filtering the packets is important to only stop the packets we are interested in. Basically, the filter checks the IP addresses and the protocol of the packet. If these fields correspond to the socket, we examine the packet.

• Packets inspection. If a packet is captured, it is processed to check if it contains synchro-nisation values. If there is an instruction, we must replace its value by the measured value also contained in the packet and the MAC addresses must also be modified because we are rerouting the packet. Eventually, we must force the recalculation of the checksum field of the TCP packet because we modified the payload. If no instruction was found in the packet, we must change the MAC addresses before sending it. These changes depend on the source and destination of the packet.

• Stopping and restoring ARP tables. When the intruder stops the attack, the tool modifies again the ARP tables of its victims to disturb the less possible the network when he stops playing the attack. The effect of MaR attack on the microgrid testbed. We observe that the brightness of the lightbulbs in the middle are lower than the two others.

The MaR attack process can be seen from Fig. 4.8, and the MaR attack effects on the testbed is shown in Fig. 4.9. Only one leader node (DG 1) gets access to the reference V * sent by the secondary controller. All other nodes will synchronize their corresponding reference values with the leader node while communicating with their neighbouring nodes. The brightness differences show that the synchronization between DG units has been interrupted due to our attack.

Delay injection attack implementation

We implemented the delay injection attack by using saboteur 2 which is an open source network fault injection tool for stability testing. Its core component is an agent that accepts commands over HTTP and configures its host's network stack for various common fault scenarios: total network partition, remote service dead (not listening on the expected port), delays, packet loss and TCP connection timeout (as often happens when two systems are separated by a stateful firewall). We run the saboteur tool on one DG to delay the packets targeting the TCP port used by the control system.

Considering the delay injection attack should be stealthy/unobservable, and at the same time could cause as much damage as possible, we consider a Gaussian-distributed delay with mean 2000ms and variance 1500ms.

Data collection and datasets description

We collected datasets under three different scenarios: normal operations without attack, MaR attack, and delay injection attack by using our experimental platform. Assuming the attacker targets the communication link between DG unit 1 and DG unit 2, we use tcpdump tool to capture the packets transmitted on the network and dissect them to extract five features:

• latency: the time difference between sending and receiving timestamps of the same packet from DG unit 1 to DG unit 2.

• ref _dg1: the reference value at DG unit 1.

• ref _dg2: the reference value at DG unit 2.

• mea_dg1: the voltage measurement value at DG unit 1.

• mea_dg2: the voltage measurement value at DG unit 2.

We run tcpdump on both DG1 and DG2 to capture the TCP control packets exchanged between them. Then, we extracted from the obtained PCAP files the 5 features described above. We labelled each collected dataset with three class labels: label 0 represents normal operations, label 1 represents system under MaR attack, and label 2 represents system under delay injection attack. A sample of the collected dataset is shown in Fig. 4.10. We collected 12297 samples in total where 8736 samples are labeled as "normal," 1836 samples are labeled as "MaR attack", and 1725 samples are labeled as "delay injection attack". This first dataset is imbalanced considering the fact that cyber-attacks happen much rarely than normal operations in real-life microgrid systems, so the collected dataset is usually with a larger proportion of "normal" samples and smaller proportions of "MaR attack" and "delay injection attack" samples. However, it is of great importance to have a balanced dataset to guarantee the performance of the classifiers based on machine learning and deep learning algorithms. Thus, for training the classifiers, we sampled the "normal" set to obtain a balanced dataset with 5552

Introduction

In this chapter, we validate the attack detection methods to be used by our secure control framework described in Section 4.3. The two typical attacks targeting the communication links, namely measurement as referent (MaR) and delay injection attack are implemented on the hardware testbed described in Chapter 4 Before studying the machine learning and deep learning based detection algorithms, we firstly show the inefficiency of traditional threshold comparison method in attack detection for the distributed microgrid system. Experimental results show that traditional threshold comparison method performs poorly in terms of precision, recall and accuracy.

To validate the performance of machine learning and deep learning based attack detection, we give a detailed hyper-parameter tuning methodology to get the best hyper-parameters used for classification. We collect datasets from the platform and compare the performances of different machine learning and deep learning algorithms to detect MaR and delay injection attacks. The classification problem will be divided into three categories: binary classification of "normal" and "MaR attack", binary classification of "normal" and "delay injection attack", and multi-class classification of "normal" , "MaR attack" and "delay injection attack". We give a detailed analysis for the three classification problems on balanced and imbalanced dataset, respectively.

The rest of this chapter is organized as follows. Section 5.2 validates the inefficiency of traditional threshold comparison method in detecting MaR and delay injection attacks. Section 5.3 describes how we tune the machine learning and deep learning hyper-parameters and list the best parameters of each classifier that we use for classification. Section 5.4 and Section 5.5 show the experimental results and performance analysis for collected imbalanced and balanced dataset, respectively. In the end, Section 5.6 gives a comprehensive discussion and analysis about the experimental results.

Detection using threshold comparison method

In this section, we make a first experiment to mainly motivate the choice of relying on ML algorithms in our secure control framework, described in the previous chapter, to detect attacks in microgrid systems.

For that, we evaluate the performance of a well known technique for corrupted data detection in such systems that relies on state vector estimation (SVE) method. The method obtains the system state, which is estimated from the observed measurements and then computes the residual between the estimated and the observed measurements. If the residual is greater than the given threshold, a data injection attack is detected.

We believe that threshold comparison based methods applied to (SVE) cannot be deployed directly in the microgrid system described in Section 4.2, because there is no globally centralized estimation system in the distribution network of power systems. However, we can still utilize the same threshold comparison idea to design a straightforward detection algorithm of our MaR attack described in Chapter 3. Since the voltage measurement value V i and reference value V * i at each DG follow the equation 3.5, we compute the residue r = |V i -V * i | and compare r with a certain threshold δ, where δ > 0. If r ≥ δ, we consider that the system is under attack.

To evaluate the performance of the above threshold comparison method, we ran the microgrid testbed described in Section 4.4, under both normal operations and under MaR attack described in Section 3.4.1, to get 2097 normal and 1040 attacked samples. We tune the δ value from 0.05V to 1.20V, with a step 0.05V, to observe how the accuracy, precision and recall changes when δ changes.

As shown in Fig. 5.1, we observe that when the threshold value δ is low, the attack detection achieves high recall and low accuracy. When we increase δ, the recall decreases rapidly and becomes close to 0 when δ = 1.20V , and the accuracy gets higher gradually. Overall, the precision of this detection algorithm is very low (lower than 0.4). Fig. 4 depicts the ROC curve of this detection method where the true positive rate and the false positive rate are computed under different thresholds. We observe that the performance of a classifier based on this method is quite bad and it is not better than random. So this method is not a good choice for attack detection in this system. These results show that such method, even its is simple to apply and realize, is inefficient to detect our synchronisation attack. Therefore, it is more appropriate to apply learning techniques and evaluate their performance for such attack and others.

Hyper-parameters tuning of the selected ML algorithms

In order to validate and evaluate the set of ML algorithms, described in the previous chapter (Section 4.3.2), on the collected datatsets, we use the scikit-learn library3 for traditional algorithms, and we use Keras and Tensorflow to build and evaluate a 1D CNN neural network. We also identified manually the optimal hyperparameters for each algorithm that provide the best performance according to the metrics that we described in subsection 4.3.3. In the next subsection, we illustrate our methodology for parameters tuning of Random Forest on a balanced dataset and then we detail the tuned parameters values of the selected ML algorithms.

Hyper-parameters tuning of Random Forest on a balanced dataset

To illustrate the method of hyper-parameters tuning, we consider the Random Forest classifier on binary classification of "normal" and "delay injection attack" as an example. We rely on validation curves that map different parameter values to a performance score and then visually check the potentially optimized values of the classifier model. A validation curve can be plotted on a graph to show how well a model performs with different values of a single hyper-parameter. As shown in Fig. 5.3, the three different hyper-parameters, number of trees, max _depth, and max _features, are plotted while varying their respective values to show how they affect the F-Measure metric of the Random Forest classifier model. We varied the n_estimators and max _depth parameters between 1 and 50 with a step of 1. We varied the max _features parameter between 1 and 5 with a step of 1. For each varied parameter, we plotted its F-measure scores during the training and also during the cross-validation to avoid the overfitting with a k-fold equal to 5. Through the different plots, we can easily observe that the appropriate values of these hyper-parameters are as follows: number of trees = 30, max _depth = 20, max _features = 3.

Selected hyper-parameter values

The optimal parameters of the machine learning and deep learning algorithms are listed in Table 5.1. The AdaBoost classifier uses 30 decision trees with a max depth of 20 for each of them. The KNN classifier is tuned with a number of k nearest neighbours equal to 10 of each query point. The MLP classifier is tuned with 100 neurons in the hidden layer and a number of epochs equal to 1000. For the Naive Bayes classifier, we used a Gaussian model with the default values 4 . The Random Forest classifier uses 30 decision trees with a maximum depth of 20 of each of them and with a maximum number of features equal to 3 to consider for the best split.

For the SVC polynomial classifier, we selected a degree of 2 and the other parameters shared with linear and RBF SVC classifiers have default values. The 1D CNN is implemented using a single convolutional layer containing 16 filters with a kernel size equal to 2 for each of them and an activation function of type relu. We add also to the neural network a dropout layer with a value of 0.5, a Max polling layer of size 2 and a dense layer with a relu activation function. The last layer used for the classification is relying on a sigmoid function. The model is using an Adam optimizer with a binary cross entropy loss function for binary classification and categorical cross entropy loss function for the multi class classification. The accuracy metric is used for model evaluation during training and testing.

Evaluation of ML algorithms with imbalanced dataset

In this section, we use the imbalanced dataset to train and test the selected machine learning and deep learning algorithms. Imbalanced dataset has more "normal" data samples compared to data samples of "MaR attack" and "delay injection attack". We have to note that this imbalanced dataset is more close to real-life datasets compared to balanced dataset, since the number of attack packets and samples is always lower than those generated during normal operations. Our goal is evaluate the selected ML algorithm with such imbalanced dataset described in Table 4.1 in chapter 4 and identify the effect of such imbalance on their performance. Therefore, we evaluated the attack detection performances into three different classification problems: binary classification of normal and MaR attack, binary classification of normal and delay injection attack, multi-class classification of the three labels (normal, delay inject and MaR).

Classification of "normal" and "MaR attack" with imbalanced dataset

The ROC curves of the selected classifiers for binary classification with "normal" and "MaR attack" are shown in Fig. 5.4. We can find that many classifiers perform well with a high AUC = 0.99. These classifiers are KNN, MLP, Random Forest, SVC Linear, SVC polynomial and SVC RBF. It means that these models can distinguish samples labeled as "normal" and "MaR attack" with a probability of 99%. Note that 1D CNN classifier has the worst AUC = 0.87, which shows that traditional machine learning outperforms deep learning in this specific classification problem. The evaluation using the different metrics measuring the performances of the selected algorithms are listed in Table 5.2. We can see that three tree based algorithms, Ada Boost, Decision Tree and Random Forest have the best performance in terms of the overall metric F -Measure = 0.95. These three algorithms also have good performances in Precision and Recall, especially for Random Forest, which has the highest Recall = 0.98. We can draw a conclusion that Tree based machine learning algorithms, especially for Random Forest, are the best classifiers in binary classification with "normal" and "MaR attack" on imbalanced dataset. Even though MLP and SVC RBF have the highest Precision = 1, the relatively lower Recall proves that they are not capable of detecting all possible MaR attacks. However, we observe also that 1D CNN has lower performance than the others with a F-Measure value of 0.8.

Classification of "normal" and "delay injection attack" with imbalanced dataset

The ROC curves of the selected classifiers for binary classification of "normal" and "delay injection attack" are shown in Fig. 5.5. We can find that three classifiers perform well with a high AUC = 0.99, which are Ada Boost, Decision Tree and Random Forest. It means that these three tree based models can distinguish samples labeled as "normal" and "MaR attack" with a probability of 99% on the imbalanced dataset. Note that all other classifiers including 1D CNN classifier have much lower AUC values compared to the previous classification problem (Normal and MaR attack) and also compared to the tree based methods. The evaluation of the metrics measuring the performances of different algorithms are listed in Table 5.3. We can see that three tree based algorithms, Ada Boost, Decision Tree and Random Forest still have the best performance in terms of the overall metric F -Measure = 0.99. These three algorithms also have good performances in Precision and Recall. And all other classifiers have much lower performances. Our finding that Tree based machine learning algorithms are the best classifiers in binary classification with "normal" and "delay injection attack" on imbalanced dataset.

Figure 5.5: ROC curves of the selected classifiers for binary classification with "normal" and "delay injection attack" on imbalanced dataset.

Classification of "normal", "MaR attack" and "delay injection attack" with imbalanced dataset

In a multi-class classification problem, we compute the micro, macro, and weighted averages of the three metrics (Precision, Recall, and F-Measure) to evaluate the performances of each classifier [START_REF] Van | Macro-and micro-averaged evaluation measures[END_REF]. To be more clear:

• Micro-averaged: all samples equally contribute to the final averaged metric.

• Macro-averaged: all samples equally contribute to the final averaged metric.

• Weighted-averaged.: each classes's contribution to the average is weighted by its size.

A high macro average indicates that the classifier performs well for each class, while a high micro average indicates that it performs well overall. Weighted average balances the effects of micro and macro averages.

The ROC curves of the selected classifiers for three-class classification with "normal", "MaR attack", and "delay injection attack" on imbalanced dataset are shown in Fig. 5.6. We find that the micro-average and macro-average ROC curves of the Ada Boost, Decision Tree and Random Forest classifiers have larger AUCs than other classifiers, especially for Random Forest, having the AUC for both micro-average and macro-average ROC curves, verifying that Random Forest has the best performance in detecting all three classes. 0.85 (+/-0.10)0.87 (+/-0.12)0.86 (+/-0.07)0.85 (+/-0.10)0.70 (+/-0.06)0.85 (+/-0.10)0.85 (+/-0.10)0.70 (+/-0.12)0.82 (+/-0.10) Random Forest 0.97 (+/-0.05)0.96 (+/-0.07)0.98 (+/-0.03)0.97 (+/-0.05)0.96 (+/-0.02)0.97 (+/-0.05)0.97 (+/-0.05)0.96 (+/-0.06)0.97 (+/-0.05) SVC Linear 0.84 (+/-0.10)0.88 (+/-0.11)0.86 (+/-0.07)0.84 (+/-0.10)0.67 (+/-0.05)0.84 (+/-0.10)0.84 (+/-0.10)0.67 (+/-0.11)0.80 (+/-0.09) SVC polynomial0.82 (+/-0.08)0.75 (+/-0.10)0.81 (+/-0.05)0.82 (+/-0.08)0.62 (+/-0.07)0.82 (+/-0.08)0.82 (+/-0.08)0.66 (+/-0.06)0.80 (+/-0.06) SVC RBF 0.86 (+/-0.00)0.91 (+/-0.02)0.88 (+/-0.01)0.86 (+/-0.00)0.68 (+/-0.01)0.86 (+/-0.00)0.86 (+/-0.00)0.71 (+/-0.02)0.83 (+/-0.01) 1D CNN 0.88 (+/-0.00)0.92 (+/-0.01)0.89 (+/-0.00)0.88 (+/-0.00)0.71 (+/-0.00)0.88 (+/-0.00)0.88 (+/-0.00)0.74 (+/-0.00)0.85 (+/-0.00)

The evaluation metrics measuring the performances of different machine learning algorithms are listed in Table 5.4 where for each metric we provide its micro, macro and weighted values. Besides Random Forest, Decision Tree and AdaBoost also outperform the rest of the classifiers in all three metrics. The rest of the classifiers have poor performances in both average Precision and average Recall, which means they can not only detect most of the positive samples under attack but also generate lots of false alarms.

Evaluation of ML algorithms with balanced dataset

In this section, we use the balanced dataset to train and test the selected machine learning and deep learning algorithms. In the balanced dataset the number of "normal" samples is close to those of the samples of "MaR attack" and "delay injection attack". Such balanced dataset is usually used to get more accurate models compared to imbalanced dataset, because when one set of classes dominate over another set of classes, the machine learning model will be more biased towards the majority class. Thus it causes poor classification of the minority classes.

Classification of "normal" and "delay injection attack" on balanced dataset

The ROC curves of the selected classifiers for binary classification with "normal" and "delay injection attack" are shown in Fig. 5.8. We can find that three classifiers perform well with a high AUC = 0.99, which are Ada Boost, Decision Tree and Random Forest. It means that these three tree based models can distinguish samples labeled as "normal" and "delay injection attack" with a probability of 99% on the balanced dataset. Note that all other classifiers including 1D CNN classifier have much lower AUC values. The evaluation metrics measuring the performances of different algorithms are listed in Table 5. [START_REF] Bi | Graphical methods for defense against false-data injection attacks on power system state estimation[END_REF]. We can see that three tree based algorithms, Ada Boost, Decision Tree and Random Forest have the best performance in terms of the overall metric F -Measure = 0.99. These three algorithms also have good performances in Precision and Recall. And all other classifiers have much poorer performances. We can draw a conclusion that Tree based machine learning algorithms are the best classifiers in binary classification with "normal" and "delay injection attack" on balanced dataset. normal data packets than data packets under attacks to be processed by DGs. The results of this experiment are shown in Table 5.12. We find that these three tree based algorithms still have the largest macro average F-Measures in all three classification problems, especially for the binary classification of "normal" and "delay injection" and multiclass classification. It proves that these three classifiers trained from a balanced dataset can successfully deal with imbalanced data in real-life microgrid systems. The training models are close enough to the behavior of the microgrid system and the studied attacks. To summarize the experimental results, we consider that the three tree based algorithms, Decision Tree, Ada Boost and Random Forest as the most suitable machine learning algorithms to design the attack detection module in Fig. 4.2 among all selected classifiers because of the following reasons:

• Their classification results do not make a large difference between imbalanced and balanced dataset, and classifiers trained from a balanced dataset can successfully deal with imbalanced data in real-life microgrid systems, which proves its training model is close enough to the behaviour of the microgrid system and also the studied attacks.

• They have better classification performances than the other machine learning and deep learning algorithms in both binary and multi-class classification results by having the largest Precision, Recall, F-Measure and AUC values. Among the three classifiers, Random Forest slightly outperforms Decision Tree and Ada Boost.

• They can handle various types of features and need very little preprocessing. This property ensures the attack detector to be capable of handling sophisticated attack scenarios involving various types of features. This thesis considered the security and attack detection of distributed controlled microgrid systems. Modern low-voltage microgrid systems are relying on distributed and cooperative control approaches to guarantee safe and reliable operations. However, many sophisticated cyber-attacks can target these systems, deceive their traditional detection methods and cause a severe impact on the power infrastructure. In this thesis, novel distributed control models of microgrids are presented. Attack models targeting the microgrid control system and their related impact analysis are discussed. A secure control framework with an attack detection module based on machine learning techniques is proposed. To validate the effectiveness of this framework, datasets considering false data injection (FDI) and denial of service (DoS) attacks are collected from the designed hardware platform that models the microgrid system. Different machine learning and deep learning classifiers are compared and analyzed based on the datasets collected.

A brief summary of the thesis contributions and possible future research directions are discussed below.

Conclusions

This thesis addressed several topics concerning security and attack detection of distributed controlled microgrids. The main contributions are as follows.

• Bridging Microgrid Control and Cyber Security. In this study, we study some of the key principles of the microgrid structures, including the control mechanism and the communication networks used in the research. Next, we include a short overview of recent developments in the area of microgrid control systems. Secondly, we study cyber-physical protection topics, and summarize a number of cyber-physical attacks in different categories. Finally, we offer an extensive overview of the recent relevant studies on cyber-secure control systems for attack detection schemes.

• Attack Modelling and Impact Analysis. In this study, we present a new attack, Measurement as Reference (MaR), directed at the distributed microgrid control system. The assailant introduces a reference voltage variance by manipulating the synchronization data shared between the distributed controllers in this attack, which we instantiated on voltage control. By simulation and theoretical analysis, we demonstrate the result of this synchronization attack. If the system does not synchronize to the proper location given by the secondary control system, the control theory tools extract the maximum voltage deviation introduced by this attack. However, other threats, such as delaying injections and denial of service attacks, also affect the operations of this system, are also triggered by a distributed controlled system. The detection of the attacks is still challenging because of the stealthy nature of some of them, where the deviations from nominal values are very low. This challenge is the inspiration behind this chapter, which proposes a better way of detecting such attacks, including the MaR attack, based on machine learning algorithms.

• Design of a Secure Framework with Attack Detection. We study various groups of supervised machine learning algorithms to design a comprehensive attack resilient control framework, in particular to detect FDI and DoS attacks in microgrids. We model and generalize both FDI and DoS attacks in distributed and cooperative control microgrid systems. We also give a general system stability condition analysis under FDI and DoS attacks. We propose a secure control framework with attack detection and response modules relying on machine learning and deep learning algorithms. Various groups of supervised machine learning and deep learning algorithms are studied to design the attack detection module against FDI attacks and DoS attacks under well-chosen performance metrics. Considering it is difficult to find publicly available datasets, we build an experimental platform emulating the distributed and cooperative controlled microgrid systems, and implement two typical FDI and DoS attacks hard to be detected by traditional residual comparison detectors. The two attacks are: measurement as referent (MaR) attack as typical FDI attack and delay injection attack as typical DoS attack. Balanced and imbalanced dataset are collected to under normal and attacked conditions to be further analyzed.

• Verification and Analysis of Various Attack Detection Algorithms. We validate the attack detection methods to be used by our secure control framework. The two typical attacks targeting the communication links, namely measurement as referent (MaR) and delay injection attack are implemented on the hardware testbed described in Chapter 4. Before studying the machine learning and deep learning based detection algorithms, we firstly show the incompetence of traditional threshold comparison method in attack detection for the distributed microgrid system. Experimental results show that traditional threshold comparison method performs poorly in terms of precision, recall and accuracy.

To validate the performance of machine learning and deep learning-based attack detection, we give a detailed hyper-parameter tuning methodology to get the best hyper-parameters used for classification. We collect datasets from the platform and compare the performances of different machine learning and deep learning algorithms to detect MaR and delay injection attacks. The classification problem will be divided into three categories: binary classification of "normal" and "MaR attack", binary classification of "normal" and "delay injection attack", and multi-class classification of "normal" , "MaR attack" and "delay injection attack". We give a detailed analyzation for the three classification problems on balanced and imbalanced dataset, respectively. By collecting datasets from an experimental microgrid platform running these attacks, we find that tree-based machine learning models, especially Random Forest, have the best performance in binary and three-class classification problems. In fact, Random Forest is quite a general machine learning algorithm to design the attack detection module proposed in our approach. It can handle various types of features and need very little preprocessing and has quick prediction/training speed. Compared to single Decision Tree and Adaboost, it is robust to outliers and nonlinear data; it can also handle imbalanced datasets, has low bias and moderate variance, and limits overfitting without substantially increasing error due to bias.

Future Work

Several steps remain to be taken concerning the opportunities for future study (due to the work begun in this thesis. It is worth noting that cyber-physical systems have a wide variety of issues to overcome. This thesis dealt with some of the security issues in the subject with restricted scope, giving specific attention to the identification of malicious actions concealed or mixed with faults and accidents. However, cyber-physical networks include many other aspects that need to be tackled together in order to enhance their resistance to threats and misuse.

There are also a variety of steps to be taken with respect to possible research perspectives (as a result of the study begun in this thesis). It should be remembered that cyber-physical devices face many obstacles. In this thesis, certain security issues have been dealt with within a restricted spectrum, with the identification of disruptive acts concealed or in conjunction with errors and injuries being a specific concern. Nonetheless, there are also other cyber-physical devices that need to be managed together to enhance their ability to strike and exploit.

As future works, we will study the scalability and transferability of our methodology for cyber-security problems in more complex and close-to-reality cyber-physical systems, like smarthome, smart-city, etc. More advanced machine learning techniques like deep learning and more sophisticated attack scenarios will also be explored.

Moreover, a bigger scope of security issues in terms of cyber-physical system could be investigated, e.g., the privacy of data transmitted among the communication links. The threat scenarios discussed in the study consisted mainly of data deception and denial of service attacks. In these cases, the attacker was attempting to interrupt the device by tampering with the data of the sensor and actuator. In addition to such situations, disclosure attacks that collect private information from the plant and control algorithms are also important. Methodologies for resolving privacy while maintaining an appropriate degree of monitoring and evaluation of the success of disclosure attacks are needed.

Résumé de la thèse en français

Modélisation et détection des attaques dans les systèmes de contrôle distribué et coopératif de micro-réseaux électriques 1 Introduction

La sécurité et la fiabilité des opérations sont considérées comme des questions critiques dans les systèmes de réseaux électriques modernes. Les nouvelles technologies de l'information et de la communication (TIC) et diverses générations réparties (DG), par exemple les générateurs photovoltaïques (PV) et les éoliennes, sont intégrées dans le réseau électrique. Ainsi, un nouveau paradigme de production d'électricité, dans lequel un parc de production mixte se compose de centrales conventionnelles centralisées et d'unités de production distribuées à des niveaux de tension inférieurs, est en train d'émerger. Cependant, il devient plus difficile d'exploiter les réseaux électriques de manière fiable et résiliente dans ce nouveau paradigme. Le concept de micro-réseaux est développé pour relever ces défis en accélérant l'intégration locale des sources d'énergie renouvelables. Les micro-réseaux utilisent une structure de contrôle hiérarchique pour contrôler et réguler, en particulier la tension électrique, y compris les niveaux primaire et secondaire de la structure de contrôle. Le contrôle primaire utilise conventionnellement des régulateurs de statisme locaux sur les ressources énergétiques distribuées (DER) pour maintenir la stabilité de la tension du micro-réseau après l'îlotage. Le contrôle secondaire du micro-réseau, qui régule la tension pour synchroniser la tension moyenne du micro-réseau avec le point de consigne nominal, peut être mis en oeuvre par une structure centralisée. Ces micro-réseaux évoluent vers des systèmes cyber physiques (CPS) avec des réseaux de contrôle et de communication basés sur des solutions logicielles [START_REF] Ali Beg | Detection of false-data injection attacks in cyber-physical dc microgrids[END_REF]. En revanche, à cause de cette intégration de ces solutions logicielles et de communication, ces réseaux sont sujets à de multiples cyber-attaques, en particulier sur leurs système de contrôle [START_REF] Teixeira | Voltage control for interconnected microgrids under adversarial actions[END_REF].

Dans cette thèse, notre contribution porte sur l'analyse et la modélisation de ces attaques dans des environnements de contrôle distribués associés à ces mircro-réseaux électriques. Dans la section 2, nous étudions l'impact d'une cyber attaque ciblant un réseau de contrôle pour altérer sa synchronisation. Dans le section 3, nous présentons une solution pour détecter et remédier à ces attaques, en particulier celles d'injection de fausses données et de déni de service. Dans la section 4, nous évaluons plusieurs algorithmes d'apprentissage automatique pour détecter ces attaques d'une façon précise. Enfin, une conclusion générale et les perspectives de recherche sont présentées.

Analyse de l'impact des attaques de synchronisation

Nous considérons, un système de contrôle distribué basé sur trois niveaux : primaire, secondaire et contrôle tertiaire. Nous estimons que ce schéma de contrôle est plus réaliste et plus fiable que l'architecture de contrôle centralisée, en raison de son meilleur support pour l'intégration évolutive des charges locales et des l'intégration de la production distribuée. Sur la base de ce modèle de contrôle, nous considérons des cyber attaques qui peuvent viser la communication entre les entités distribuées de génération (DG), et nous utilisons des méthodes d'évaluation des risques pour analyser quantitativement les l'impact de ces attaques sur la déviation de la tension au niveau du contrôle primaire et de la tension de référence de la synchronisation au niveau du contrôle secondaire. La Figure 1 présente le modèle du système considéré pour analyse l'impact d'une attaque de synchronisation sur le réseau de communication reliant les différentes DGs. Nous définissons une nouvelle attaque de synchronisation, nommée Measurement as Reference (MaR), ciblant ce système qui permet à un attaquant de remplacer la consigne par une mesure, en particulier celles pour réguler la tension électrique. Dans cette situation, l'attaquant remplace une consigne V * i+1 d'un noeud i + 1 avec une valeur de mesure V s i d'un noeud i. L'objectif de cette attaque est de provoquer des fluctuations de tension et des régulations irrégulières pour endommager les charges sans violer la plage admissible.

Nos simulations ont montré que cette attaque introduit des déviations de la consigne au niveau de chaque DG comme le montre la Figure2.

Un framework pour un contrôle distribué plus sécurisé

L'objectif de cette section est de proposer une méthode pour sécuriser un système de contrôle distribué d'un micro-réseau. Nous proposons un framework pour la détection des attaques ciblant ces micro-réseaux, en particulier celles d'injection de fausses de données, à l'image que l'attaque MaR présentée dans la section 2 et également des attaques de déni de service. La

Conclusion générale

Travail réalisé

L'intégration des système de contrôle distribués et coopératifs dans les micro-réseaux a introduit plusieurs problématiques de cyber sécurité, notamment, des attaques ciblant leurs réseaux de communication pour falsifier les données ou introduire des délais. Dans cette thèse, nous traitons ce problème en 3 étapes. Tout d'abord, nous avons étudié l'impact d'une attaque de synchronisation sur la régulation d'un micro-réseau. Cette attaque permet à une attaquant de remplacer la consigne par une mesure lors de leurs échanges entre deux entités de contrôle. Nous avons simulé cette attaques et nous avons démontré son effet sur la régulation d'une consigne de la tension du réseau électrique. Deuxièmement, nous avons proposé un framework pour offrir un contrôle distribué et sécurisé pour ces micro-réseaux. Notre framework s'appuie notamment sur la détection des attaques de falsification de données et l'injection des délais. Nous avons mis en oeuvre et développé une plate-forme expérimentale pour étudier ces attaques et collecter des jeux de données représentatifs de leurs modes opératoires. Troisièmement, nous avons évaluer différents algorithmes d'apprentissage pour classifier les paquets échangés sur le système de contrôle et détecter ces attaques. Nous avons identifié que les algorithmes basés sur les arbres offrent les meilleures performances en terme de précision et temps de détection.

1. 1

 1 Smart grid architecture as a composition of three interoperability infrastructures in compliance with IEEE 2030[START_REF] Bushra Canaan | Microgrid cyber-security: Review and challenges toward resilience[END_REF]. . . . . . . . . . . . . . . . . . . . . . . . . . . 1.2 CPS vulnerability totals trend by year [32]. . . . . . . . . . . . . . . . . . . . . . 1.3 Schematic and impact assessment concepts of a typical cyber-physical control system with a communication network that is vulnerable to adversaries. . . . . . 1.4 Methodology flowchart to study the threat modeling and risk management problem of microgrid control systems . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.1 The hierarchical control structure of a microgrid consisting of primary, secondary and tertiary control levels as specified in [7]. . . . . . . . . . . . . . . . . . . . . 2.2 Risk Factor Venn Diagram . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.3 SVM concept [50, 57]. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.4 MLP concept [57]. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .2.5 RNN concept [57]. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.1 Microgrid with a communication network. As a part of the power distribution network, microgrids are at the downstream of distribution substations. Sparse communication networks are employed for data exchange between neighboring DG units, e.g., transmitting reference signals and measurements. . . . . . . . . . 3.2 A microgrid control system with primary and secondary controllers. Sparse communication networks are employed for data exchange between neighboring DG units, e.g., transmitting reference signals and measurements (denoted by the superscript s). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3 The topology of the communication graph G of a microgrid system. . . . . . . . 3.4 The simulink model of the distributed and cooperative controlled microgrid system. 3.5 The impact of the measurement as reference attack on voltage deviation. . . . . . 3.6 The impact of the measurement as reference attack on reference synchronization. 4.1 Secure control framework with attack detection and response modules. . . . . . 4.2 Internals of attack detection and response modules in the secure control framework. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.3 Selected learning algorithms and their respective groups. . . . . . . . . . . . . . 4.4 The hardware microgrid platform consisting of 4 DG units that perform distributed and cooperative control strategy. . . . . . . . . . . . . . . . . . . . . . . 4.5 The hardware components of each DG. . . . . . . . . . . . . . . . . . . . . . . . 4.6 Illustration of the communication between the DGs acting as clients and server for the transmission of control values. . . . . . . . . . . . . . . . . . . . . . . . . 4.7 Steps of the Man-In-The-Middle technique to realize a MaR attack. . . . . . . . vii 4.8 Illustration of the execution of the MaR attack in our platform. . . . . . . . . . 4.9 The effect of MaR attack on the microgrid testbed. We observe that the brightness of the lightbulbs in the middle are lower than the two others. . . . . . . . . . . . 4.10 A set of samples extracted from the captured packets with their respective 5 feature values and class labels. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.1 Performance metrics (Accuracy, Precision and Recall) of the threshold comparison method. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.2 ROC curve of a classifier based on threshold comparison applied to detect the MaR attack. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.3 Validation curves for the hyper-parameters n_estimators (number of trees), max_depth and max_features of the Random Forest algorithm for the classification of delay attack and normal behaviour of the system. . . . . . . . . . . . . . . . . . . . . . 5.4 ROC curves of the selected classifiers for binary classification with "normal" and "MaR attack" on imbalanced dataset. . . . . . . . . . . . . . . . . . . . . . . . . . 5.5 ROC curves of the selected classifiers for binary classification with "normal" and

2 L

 2 'impact de l'attaque MaR sur la synchronisation d'une tension de référence entre 4 DGs. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 Un framework de contrôle sécurisé avec des modules de détection et de réaction aux attaques. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 Courbe ROC d'un classifieur basé sur la comparaison de seuils pour détecter l'attaque MaR. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . viii

3 1. 2 6 1. 3 7 1. 3 . 1 9 1. 3 . 2

 3263731932 -Physical Security of Microgrids . . . . . . . . . . . . . . . . . . Research Questions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Methodology and Contributions . . . . . . . . . . . . . . . . . . . . . Thesis Outline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Publications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

Figure 1 . 3 :

 13 Figure 1.3: Schematic and impact assessment concepts of a typical cyber-physical control system with a communication network that is vulnerable to adversaries.

Figure 1 . 4 :

 14 Figure 1.4: Methodology flowchart to study the threat modeling and risk management problem of microgrid control systems

20 )Figure 2 . 3 :

 2023 Figure 2.3: SVM concept [50, 57].

Figure 2 . 4 :

 24 Figure 2.4: MLP concept [57].

Figure 2 . 5 :

 25 Figure 2.5: RNN concept [57].

Figure 3 . 1 :

 31 Figure 3.1: Microgrid with a communication network. As a part of the power distribution network, microgrids are at the downstream of distribution substations. Sparse communication networks are employed for data exchange between neighboring DG units, e.g., transmitting reference signals and measurements.

Figure 3 . 2 :

 32 Figure 3.2: A microgrid control system with primary and secondary controllers. Sparse communication networks are employed for data exchange between neighboring DG units, e.g., transmitting reference signals and measurements (denoted by the superscript s).

Figure 1

 1 Figure 1 illustrates how the reference signals and measurements are available to each controller. Making use of the capabilities of inverter-based DERs, each DG unit is controlled by a droop controller. The droop controller receives reference signal through synchronization process between neighboring nodes and voltage measurements from local meters.For each DG unit, the voltage and phase-angle dynamics can be respectively modeled by a single integrator

Figure 3 . 3 :

 33 Figure 3.3: The topology of the communication graph G of a microgrid system.

Figure 3 . 4 :

 34 Figure 3.4: The simulink model of the distributed and cooperative controlled microgrid system.

Figure 3 . 5 :

 35 Figure 3.5: The impact of the measurement as reference attack on voltage deviation.

54 3Figure 3 . 6 :

 5436 Figure 3.6: The impact of the measurement as reference attack on reference synchronization.

4 . 1 4 . 2 59 4. 3 63 4. 3 . 3 66 4. 4 72 4. 5

 41425936333664725 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Attack Modeling and Problem Formulation . . . . . . . . . . . . . . . 4.2.1 Modeling of FDI and DoS Attacks . . . . . . . . . . . . . . . . . . . . . 58 4.2.2 General Stability under Attack . . . . . . . . . . . . . . . . . . . . . . . Secure Control Framework . . . . . . . . . . . . . . . . . . . . . . . . . 4.3.1 Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60 4.3.2 Selected Machine Learning algorithms . . . . . . . . . . . . . . . . . . . Performance metrics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Experimental platform and data collection . . . . . . . . . . . . . . . 4.4.1 Microgrid testbed . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67 4.4.2 The distributed control system . . . . . . . . . . . . . . . . . . . . . . . 68 4.4.3 Attacks implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . 69 4.4.4 Data collection and datasets description . . . . . . . . . . . . . . . . . . Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Figure 4 . 1 :

 41 Figure 4.1: Secure control framework with attack detection and response modules.

Figure 4 . 2 :

 42 Figure 4.2: Internals of attack detection and response modules in the secure control framework.

12 :

 12 return List, Drop_packet, Send _List_to_neighbours;

Fig. 4

 4 .3.

Figure 4 . 3 :

 43 Figure 4.3: Selected learning algorithms and their respective groups.

Figure 4 . 4 :

 44 Figure 4.4: The hardware microgrid platform consisting of 4 DG units that perform distributed and cooperative control strategy.

Figure 4 . 5 :

 45 Figure 4.5: The hardware components of each DG.

Figure 4 . 6 :

 46 Figure 4.6: Illustration of the communication between the DGs acting as clients and server for the transmission of control values.

Figure 4 . 7 :

 47 Figure 4.7: Steps of the Man-In-The-Middle technique to realize a MaR attack.

Figure 4 . 8 :

 48 Figure 4.8: Illustration of the execution of the MaR attack in our platform.

Figure 4 . 9 :

 49 Figure 4.9: The effect of MaR attack on the microgrid testbed. We observe that the brightness of the lightbulbs in the middle are lower than the two others.

5 . 1 5 . 2 5 . 3 5 . 4 5 . 5 5 . 6

 515253545556 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Detection using threshold comparison method . . . . . . . . . . . . . Hyper-parameters tuning of the selected ML algorithms . . . . . . . 5.3.1 Hyper-parameters tuning of Random Forest on a balanced dataset . . . 78 5.3.2 Selected hyper-parameter values . . . . . . . . . . . . . . . . . . . . . . 80 Evaluation of ML algorithms with imbalanced dataset . . . . . . . . 5.4.1 Classification of "normal" and "MaR attack" with imbalanced dataset . 81 5.4.2 Classification of "normal" and "delay injection attack" with imbalanced dataset . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83 5.4.3 Classification of "normal", "MaR attack" and "delay injection attack" with imbalanced dataset . . . . . . . . . . . . . . . . . . . . . . . . . . . 85 Evaluation of ML algorithms with balanced dataset . . . . . . . . . . 5.5.1 Classification of "normal" and "MaR attack" with balanced dataset . . . 88 5.5.2 Classification of "normal" and "delay injection attack" on balanced dataset 90 5.5.3 Classification of "normal", "MaR attack" and "delay injection attack" on balanced dataset . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92 Discussion and Analysis . . . . . . . . . . . . . . . . . . . . . . . . . .

Figure 5 . 1 :

 51 Figure 5.1: Performance metrics (Accuracy, Precision and Recall) of the threshold comparison method.

Figure 5 . 2 :

 52 Figure 5.2: ROC curve of a classifier based on threshold comparison applied to detect the MaR attack.

Figure 5 . 3 :

 53 Figure 5.3: Validation curves for the hyper-parameters n_estimators (number of trees), max_depth and max_features of the Random Forest algorithm for the classification of delay attack and normal behaviour of the system. 79

Figure 5 . 4 :

 54 Figure 5.4: ROC curves of the selected classifiers for binary classification with "normal" and "MaR attack" on imbalanced dataset.

Figure 5 . 6 :

 56 Figure 5.6: ROC curves of the selected classifiers for multiclass classification with "normal", "MaR attack" and "delay injection attack" on imbalanced dataset.

Figure 5 . 7 :

 57 Figure 5.7: ROC curves of the selected classifiers for binary classification with "normal" and "MaR attack" on balanced dataset.

Figure 5 . 8 :

 58 Figure 5.8: ROC curves of the selected classifiers for binary classification with "normal" and "delay injection attack" on balanced dataset.

Figure 1 :

 1 Figure 1: Un système de contrôle de micro-réseau avec des contrôleurs primaires et secondaires. Des réseaux de communication clairsemés sont utilisés pour l'échange de données entre les unités de production d'électricité voisines, par exemple pour transmettre des consignes de référence et des mesures (indiqués par l'exposant s).

102 3 .Figure 2 :

 32 Figure 2: L'impact de l'attaque MaR sur la synchronisation d'une tension de référence entre 4 DGs.

Figure 3 Figure 3 :

 33 Figure 3 présente l'architecture de notre framework pour à la fois détecter et répondre à ces attaques.

Figure 4 :

 4 Figure 4: Courbe ROC d'un classifieur basé sur la comparaison de seuils pour détecter l'attaque MaR.

  to obtain the frequency and voltage references, ω ref and v ref based on

Table 2 .

 2 1: Comparison of various communication channels

	Mode	Frequency Band	Signal Rate	Power Consump-	Expense	Application in the	Role Index in Re-
				tion		Microgrid		siliency Enhance-
								ment
	Weightless	subGHz	0.1-25mbps	Low	Low	Smart metering	Low
	LoRa	subGHz	< 50kbps	Low	Moderate	Utility metering	Low
	NB-IoT	subGHz	0-1mbps	Moderate	High	Facility manage-	Moderate
						ment services
	Wireless HART 2.4GHz	250kbps	Moderate	Moderate	Encryption	of	Moderate
						data
	WiFi	subGHz, 2.4GHz	0-50mbps	Moderate	Low	Intercommunication	High
						between	neigh-
						boring systems
	Z-wave	subGHz	30kbps	Low	Moderate	Smart metering	Low
	802.15.4	subGHz, 2.4GHz	40, 250kbps	Low	Low	Equipment main-	Moderate
						tenance
	SigFox	subGHz	<1kbps	Low	Low	Monitoring power	Moderate
						generation from
						the grid
	ZigBee	2.4GHz	230kbps	Low	Medium	Intra and inter	High
						communication
						between devices
	LTE Cat 0/1	Cellular	0-10mbps	Moderate	High	Transmission	High
						of preprocessed
						data across the
						microgrid
	Bluetooth	2.4GHz	1-3mbps	Low	Low	Communication	High
						across the com-
						ponents of the
						microgrid
	3G	Cellular	10-30mbps	High	High	Autonomous in-	High
						teraction across
						the components

Table 2 .

 2 2: Cybersecurity Risk Analysis

	Attack Surface	Examples			Threats/Vulnerabilities Potential Impact
	Wired Links	Ethernet, Phone lines,	Line may be cut or	Blocking resources De-
		RS-232, RS-422, RS-	unplugged,	causing	laying the timing of crit-
		485, I2C (often used for	DoS. An attacker may	ical message exchange
		sensors)			splice into the wires
					and eavesdrop, tamper
					with, and inject data
					into the medium
	Wireless Links	IEEE 802.11 (WiFi),	The broadcast nature	Confidentiality	lost
		Cellular networks, Zig-	of the medium enables	With jammed signals,
		Bee/XBee			anyone to listen to com-	SCADA system loses
					munications.	Signals	connection with micro-
					may be jammed, result-	grid's analysis layer,
					ing in DoS.		causing	sub-optimal
							operation.
	Unencrypted Communications	Most low-level serial	An attacker who can	Data	confidentiality
		protocols including I2C	access the medium can	lost?market participa-
		and SPI Unencapsu-	view, modify, and send	tion bidding strategy
		lated TCP and UDP,	meaningful data.	compromised.
		including HTTP, FTP,
		Telnet
	Unauthenticated Communications DNS, HTTP, most	Attacker can pretend	Can cause unwanted
		lowlevel serial protocols	to be legitimate data	islanding of microgrid
					source or user		or unwanted change to
							grid-connected	mode
							during island (can be
							a life-threating issue
							if maintenance work-
							ers are working on
							electrical components).
	Exposed Endpoints	Sensors, HMI, Remote	Attacker could falsify	Wrong price-signal or
		Servers, Anything the	actionable data.	At-	weather data will cause
		attacker can gain physi-	tacker could damage in-	suboptimal and noneco-
		cal or virtual access to	terface, causing DoS.	nomic operational dis-
							patch strategy of the
							microgrid.
	Human	Technicians,	admin-	Attacker gains real cre-	Unwanted	islanding
		istrators, contractors,	dentials from a legit-	Dispatch	strategy
		any user with the	imate user and can	causes	economics
		system	or	network	then access normally re-	losses in operations.
		access			stricted systems.	In-	Unwanted outages
					sider threat: a legiti-
					mate user exploits their
					access to cause damage

Table 2 .

 2 3: Summary of different references in attack detection using machine learning methods

	Reference	Supervised	Machine Learning Methods Semi-supervised Unsupervised	Deep Learning	Attack Type Core Feature	Core Metrics
	[50]	Perceptron, SVM, KNN, Logistic Regression	S3VM	K -means		FDI	Observed Measurements	Precision, Recall, Accuracy
	[17]	SVM				FDI	Observed Measurements	Precision, Recall
								Precision,
	[71]	SVM, KNN				FDI	Observed Measurements	Recall, Accuracy,
								F-Measure
	[44]	SVM, Naive Bayes			MLP	DoS	Certain Network Parameters	Precision, Recall, Accuracy
		Decision Tree,						Precision,
		Random Forest,						Recall,
	[29]	SVM,			MLP	DoS	Certain Network Parameters	Accuracy,
		Naive Bayes,						F-Measure,
		KNN						MCC
								Precision,
	[20], [19], [35], [25]		S3VM			FDI	Observed Measurements	Recall,
								Accuracy
								Precision,
	[48]			K -means, Fuzzy c-means		FDI	Observed Measurements	Recall, F-Measure,
								Detection Rate
	[72]			Local Outlier Factor, Isolation Forest, Robust Covariance Estimation		FDI	Observed Measurements	Precision, Accuracy
	[21], [62], [2]				MLP	FDI	Observed Measurements	Precision, Accuracy
								Precision,
	[27], [3]				RNN	FDI	Observed Measurements	Accuracy, Sensitivity,
								Specificity
	[31]				RNN	FDI	Observed Measurements	False Alarm Rate, Detection Rate
	[1], [68]	SVM			DBN	FDI	Observed Measurements	False Alarm Rate, Detection Rate
	[24]				DBN	FDI	Observed Measurements	Detection Accuracy

4 Cyber attacks on voltage control and Impact Analysis . . . . . . . . 50

Contents 3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41 3.2 System Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42 3.2.1 Preliminaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2.2 Microgrid System with Hierarchical Control Structure . . . . . . . . . . 3.2.3 Communication Network . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3 Distributed Cooperative Control system . . . . . . . . . . . . . . . . 46 3.3.1 Primary Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3.2 Secondary Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3.4.1 Measurement as reference attack . . . . . . . . . . . . . . . . . . . . . . 3.4.2 Attack Impact on Voltage Deviation . . . . . . . . . . . . . . . . . . . . 3.4.3 Impact on Reference Voltage Synchronization . . . . . . . . . . . . . . .

3.5 Simulation System and Validation . . . . . . . . . . . . . . . . . . . . 52 3

.5.1 Simulation Settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.5.2 Simulation Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54

  The detection system deployed in each DG unit maintains and updates a hash table of attack tags recording the IP address and port number of the suspicious nodes under attack. When a DG unit receives an attack tag, it verifies if it exists in its hash table. If not, it adds the attack tag to a blacklist and sends the updated attack tag to all cooperative nodes Algorithm 1 Selection of the best attack detection method for Classification module. Input: The training dataset S 1 and testing dataset S 2 with correct labels y i ∈ Ω = {0, 1, 2}; the set of various machine learning and deep learning classifiers D = {Ada_Boost, KNN , 1D_CNN , ..., etc.}; Output: The best performed classifier d * ∈ D in attack detection;

1: Initialize F 1 = 0; 2: for i ∈ D do 3:

Table 5 . 1 :

 51 Parameters of the selected machine learning algorithms.

	Classifier	Parameters
	Ada Boost	DecisionTree estimator with max_depth=20, number of estimators=30, learning rate=0.1
	Decision Tree	max_depth=20
	KNN	number of neighbors=10, metric='cosine'
	MLP	number of neurons=100, L2 penalty=1, max_iter=1000
	Naive Bayes	GaussianNB with default parameters
	Random Forest	number of trees=30, max_depth=20, max_features=3
	SVC Linear	kernel='linear', regularization parameter=1.0
	SVC polynomial	kernel='poly', degree=2, gamma='auto', regularization parameter=1.0
	SVC RBF	kernel='rbf', gamma='auto', regularization parameter=1.0
	1D CNN	filters=16, kernel_size=2, activation='relu'

Table 5 .

 5 2: Performances of different classifiers for binary classification with "normal" and "MaR attack" on imbalanced dataset.

	Classifier	Precision	Recall	F-Measure
	Ada Boost	0.93 (+/-0.23) 0.97 (+/-0.01) 0.95 (+/-0.13))
	Decision Tree	0.93 (+/-0.23) 0.97 (+/-0.01) 0.95 (+/-0.13)
	KNN	0.96 (+/-0.14) 0.91 (+/-0.05) 0.93 (+/-0.05)
	MLP	1.00 (+/-0.00) 0.89 (+/-0.06) 0.94 (+/-0.03)
	Naive Bayes	0.89 (+/-0.35) 0.98 (+/-0.02) 0.92 (+/-0.22)
	Random Forest	0.93 (+/-0.23) 0.98 (+/-0.01) 0.95 (+/-0.13)
	SVC Linear	0.91 (+/-0.37) 0.92 (+/-0.02) 0.90 (+/-0.22)
	SVC polynomial 0.92 (+/-0.31) 0.95 (+/-0.02) 0.93 (+/-0.18)
	SVC RBF	1.00 (+/-0.01) 0.87 (+/-0.05) 0.93 (+/-0.02)
	1D CNN	0.92 (+/-0.00) 0.70 (+/-0.00) 0.80 (+/-0.00)

Table 5 .

 5

	3: Performances of different classifiers for binary classification with "normal" and "delay
	injection attack" on imbalanced dataset
	Classifier	Precision	Recall	F-Measure
	Ada Boost	0.99 (+/-0.02) 1.00 (+/-0.01) 0.99 (+/-0.01)
	Decision Tree	0.98 (+/-0.02) 1.00 (+/-0.01) 0.99 (+/-0.01)
	KNN	0.96 (+/-0.02) 0.21 (+/-0.08) 0.34 (+/-0.11)
	MLP	1.00 (+/-0.00) 0.21 (+/-0.07) 0.34 (+/-0.10)
	Naive Bayes	0.77 (+/-0.64) 0.22 (+/-0.08) 0.33 (+/-0.19)
	Random Forest	0.99 (+/-0.02) 0.98 (+/-0.02) 0.99 (+/-0.01)
	SVC Linear	1.00 (+/-0.00) 0.21 (+/-0.08) 0.35 (+/-0.10)
	SVC polynomial 1.00 (+/-0.00) 0.21 (+/-0.08) 0.35 (+/-0.10)
	SVC RBF	1.00 (+/-0.00) 0.21 (+/-0.07) 0.34 (+/-0.10)
	1D CNN	1.00 (+/-0.00) 0.01 (+/-0.00) 0.02 (+/-0.00)

Table 5 .

 5 4: Performances of different classifiers for multiclass classification with "normal", "MaR attack" and "delay injection attack" on imbalanced dataset

	Classifier		Average Precision			Average Recall			Average F-Measure
		Mirco	Macro	Weighted	Mirco	Macro	Weighted	Mirco	Macro	Weighted
	Ada Boost	0.97 (+/-0.05)0.95 (+/-0.07)0.97 (+/-0.03)0.97 (+/-0.05)0.96 (+/-0.02)0.97 (+/-0.05)0.97 (+/-0.05)0.95 (+/-0.05)0.97 (+/-0.04)
	Decision Tree	0.97 (+/-0.05)0.95 (+/-0.07)0.97 (+/-0.03)0.97 (+/-0.05)0.96 (+/-0.02)0.97 (+/-0.05)0.97 (+/-0.05)0.95 (+/-0.05)0.97 (+/-0.04)
	KNN	0.86 (+/-0.03)0.85 (+/-0.06)0.85 (+/-0.03)0.86 (+/-0.03)0.69 (+/-0.03)0.86 (+/-0.03)0.86 (+/-0.03)0.71 (+/-0.06)0.83 (+/-0.04)
	MLP	0.87 (+/-0.01)0.92 (+/-0.02)0.88 (+/-0.01)0.87 (+/-0.01)0.70 (+/-0.03)0.87 (+/-0.01)0.87 (+/-0.01)0.72 (+/-0.03)0.84 (+/-0.02)
	Naive Bayes

Table 5 .

 5

	6: Performances of different classifiers for binary classification with "normal" and "delay
	injection attack" on balanced dataset
	Classifier	Precision	Recall	F-Measure
	Ada Boost	0.99 (+/-0.02) 1.00 (+/-0.00) 0.99 (+/-0.01)
	Decision Tree	0.99 (+/-0.02) 0.99 (+/-0.01) 0.99 (+/-0.01)
	KNN	0.62 (+/-0.06) 0.40 (+/-0.07) 0.49 (+/-0.05)
	MLP	0.99 (+/-0.03) 0.22 (+/-0.08) 0.36 (+/-0.10)
	Naive Bayes	0.98 (+/-0.06) 0.22 (+/-0.08) 0.36 (+/-0.11)
	Random Forest	0.99 (+/-0.01) 0.98 (+/-0.03) 0.98 (+/-0.02)
	SVC Linear	1.00 (+/-0.00) 0.21 (+/-0.07) 0.35 (+/-0.10)
	SVC polynomial 1.00 (+/-0.00) 0.21 (+/-0.08) 0.35 (+/-0.10)
	SVC RBF	0.88 (+/-0.16) 0.23 (+/-0.09) 0.36 (+/-0.11)
	1D CNN	1.00 (+/-0.00) 0.20 (+/-0.00) 0.33 (+/-0.00)

Table 5 .

 5 8: Summary of best performing ML algorithms based on the F-Measure for each classification problem with imbalanced dataset.

			Classification problem
		Normal and MaR Normal and Delay Injection Three classes
	Ada Boost	(0.95)	(0.99)	(0.95)
	Decision Tree	(0.95)	(0.99)	(0.95)
	KNN
	MLP
	Naive Bayes
	Random Forest	(0.95)	(0.99)	(0.96)
	SVC Linear
	SVC polynomial
	SVC RBF
	1D CNN
			Classification problem
		Normal and MaR Normal and Delay Injection Three classes
	Ada Boost	(0.97)	(0.99)	(0.95)
	Decision Tree	(0.97)	(0.99)	(0.95)
	KNN
	MLP	(0.97)
	Naive Bayes
	Random Forest	(0.98)	(0.98)	(0.96)
	SVC Linear
	SVC polynomial	(0.97)
	SVC RBF
	1D CNN

Table 5 . 9 :

 59 Summary of best performing ML algorithms based on the F-Measure for each classification problem with balanced dataset.

		Running time (seconds) of training and testing on balanced dataset
		Normal and MaR Normal and Delay Injection	Three classes
	Ada Boost	0.415	0.571	0.081
	Decision Tree	0.509	0.253	0.098
	KNN	1.239	0.747	0.334
	MLP	20.442	11.763	5.815
	Naive Bayes	0.485	0.232	0.047
	Random Forest	2.508	1.614	0.617
	SVC Linear	4.940	164.892	135.249
	SVC polynomial	12.489	3632.050	1621.737
	SVC RBF	6.177	14.180	6.581
	1D CNN	7.861	6.848	11.257

Table 5 .

 5 10: Summary of the running time of training and testing for each classifier on balanced dataset.with balanced or imbalanced datasets, and have also low running time. In a last experiment, we evaluated the robustness of these algorithms when training them with a balanced dataset and testing them with an imbalanced dataset. Such situation is more realistic, since the data packets transmitted in the real-life microgrid systems are imbalanced, i.e., there are many more

		Running time (seconds) of training and testing on imbalanced dataset (s)
		Normal and MaR Normal and Delay Injection	Three classes
	Ada Boost	0.576	0.574	0.167
	Decision Tree	0.410	0.386	0.141
	KNN	5.566	4.481	1.726
	MLP	41.014	28.703	16.448
	Naive Bayes	0.287	0.252	0.070
	Random Forest	3.243	3.791	1.298
	SVC Linear	10.639	1054.497	596.080
	SVC polynomial	22.829	21820.063	4384.425
	SVC RBF	19.768	155.710	26.540
	1D CNN	16.920	16.455	24.912

Table 5 .

 5 11: Summary of the running time of training and testing for each classifier on imbalanced dataset.

Table 5 .

 5 12: Summary of the Macro average F-Measure values of different classifiers for training on balanced dataset and testing on imbalanced dataset.

			Macro average F-Measure
		Normal and MaR Normal and Delay Injection Three classes
	Ada Boost	1.00	0.99	0.99
	Decision Tree	1.00	0.99	0.99
	KNN	0.97	0.61	0.71
	MLP	0.98	0.65	0.73
	Naive Bayes	0.97	0.64	0.71
	Random Forest	1.00	0.99	0.99
	SVC Linear	0.97	0.64	0.69
	SVC polynomial	0.98	0.64	0.67
	SVC RBF	0.97	0.63	0.72
	1D CNN	0.76	0.35	0.84

•

  They have fast prediction/training speed, and prediction speed is significantly faster than training speed because generated classifiers are saved for future uses. It is a crucial property for attack detection when DG units have limited resources. From Table5.10 and Table5.11, Decision Tree and Ada Boost are faster than Random Forest in training and testing. However, in real-life cases, Random Forest could also be faster to train than Decision Tree because only a subset of features in this model is utilized, so it works quickly with high dimensional data. Random Forest is also parallelizable, so the training process can be split into multiple machines to run, resulting in faster computation time than sequential AdaBoost.

	Chapter 6
	Conclusions and Future Work

Figure 5.9: ROC curves of the selected classifiers for multiclass classification with "normal", "MaR attack" and "delay injection attack" on balanced dataset.

Contents 6.1 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97 6.2 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99 1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101 2 Analyse de l'impact des attaques de synchronisation . . . . . . . . . 102 3 Un framework pour un contrôle distribué plus sécurisé . . . . . . . 102 4 Méthodes de détection par apprentissage automatique . . . . . . . . 104 5 Conclusion générale . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105 5.1 Travail réalisé . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105 5.2 Perspectives de recherche . . . . . . . . . . . . . . . . . . . . . . . . . . 106

MaR (Measurement as Reference) qui cible les consignes de synchronisation échangées entre les entités du système de contrôle. Nous avons analysé par simulation numérique l'impact de cette attaque sur la stabilité du micro-réseau et la convergence du système de contrôle vers une consigne commune. Nous avons également développé des modèles d'analyse des attaques de type injection de fausses données et déni de service sur ces systèmes pour étudier leurs impacts et leur détection. Ensuite, nous avons proposé un framework qui permet de détecter ces attaques, en se basant sur l'apprentissage automatique des caractéristiques des paquets réseau échangés entres les entités d'un système de contrôle distribué. Nous avons mis en oeuvre une plate-forme expérimentale représentative d'un micro-réseau électrique et son système de contrôle pour collecter des jeux de données et valider notre framework, en particulier son module de détection des attaques. Enfin, nous avons évalué les performances de différents algorithmes d'apprentissage automatique pour détecter les attaques que nous avons introduites sur la plate-forme expérimentale. Nos résultats montrent que les algorithmes basés sur les techniques d'arbres, à l'image des arbres de décision, les forêts aléatoire et AdaBoost offrent les meilleurs performance en termes de précision et de justesse pour détecter les différentes attaques et les distinguer.

https://github.com/tomakehurst/saboteur

https://github.com/tomakehurst/saboteur

https://scikit-learn.org/stable/index.html

https://scikit-learn.org/stable/modules/generated/sklearn.naive_bayes.GaussianNB.html

samples in total and 1991 samples labeled as "normal," 1836 samples labeled as "MaR attack", and 1725 samples labeled as "delay injection attack". Table 4.1 shows the number of samples for each scenario and for both imbalanced and balanced datasets.

- We will train and test the classifiers both on the balanced and imbalanced datasets. By comparing the performance differences of the classifiers on balanced and imbalanced datasets, we can better evaluate the performances of different algorithms as attack detectors. In particular, we are interested in the evaluation of the effect of the imbalanced dataset on the performance of the classifiers algorithms.

Summary

In this chapter, we have designed and described a secure control framework for microgrid systems. We have modeled both FDI and DoS attacks with regards to the distributed and cooperative control microgrid. The secure framework mainly contains four parts: data collection, preprocessing, classification and response. With help of various machine learning and deep learning algorithms, we further detail the classification module of the secure framework. To generate datasets containing normal and attack samples, we build a microgrid testbed to implement two typical attacks, measurement as reference attack and delay injection attack.

Classification of "normal" and "MaR attack" with balanced dataset

The ROC curves of the selected classifiers for binary classification with "normal" and "MaR attack" are shown in Fig. 5.7. We can find that many classifiers perform well with a high AUC = 0.99. These classifiers are MLP, Naive Bayes, Random Forest, SVC Linear, SVC polynomial, SVC RBF and 1D CNN. It means that these models can distinguish samples labeled as "normal" and "MaR attack" with a probability of 99%. Note that all other classifiers have also have high AUC values greater than 97%, which means all selected classifiers can handle the classification problem with good accuracy. The evaluation metrics measuring the performances of different algorithms are listed in Table 5. [START_REF] Ben-Hur | Support vector clustering[END_REF]. We can see that Random Forest has the best performance in terms of the overall metric F -Measure = 0.98. All traditional machine learning classifiers have also good performances in Precision, Recall and F-Measure. Note that 1D CNN has the lowest F -Measure = 0.85 and lowest Recall = 0.75, which means that traditional machine learning outperforms deep learning in binary classification with "normal" and "MaR attack" on balanced dataset.

Classification of "normal", "MaR attack" and "delay injection attack" on balanced dataset

The ROC curves of the selected classifiers for three-class classification with "normal", "MaR attack", and "delay injection attack" on balanced dataset are shown in Fig. 5.9. We can find that the micro-average and macro-average ROC curves of the Ada Boost, Decision Tree and Random Forest classifiers have larger AUCs than other classifiers, especially for Random Forest, having the AUC for both micro-average and macro-average ROC curves, verifying that Random Forest has the best performance in detecting all three classes. The evaluation metrics measuring the performances of different machine learning algorithms are listed in Table 5.7. Besides Random Forest, Decision Tree and AdaBoost also outperform the rest of the classifiers in all three metrics. The rest of the classifiers have poor performances in both average Precision and average Recall, which means they can not only detect most of the positive samples under attack but also generate lots of false alarms.

Discussion and Analysis

As shown in the previous sections, we find that Decision Tree, Random Forest (bagged trees), and Adaboost (boosted trees) outperform the other selected classification models. These algorithms can not only detect attack data packets but can also distinguish different attack types with high performance. We find also that the deep learning algorithm 1D CNN does not provide good performance and it is less performant than the others. Regarding the effect of balanced and imbalanced datasets, all the algorithms provide close performances for both of them. A summary of the best performing algorithms for each classification problem and for both balanced and imbalanced datasets is shown in Tables 5.9 and 5.8.

Besides, we evaluated the running time of different learning and testing tasks for each algorithm as shown in Table 5.10 and Table 5.11. Note that we measured the running time on a MacBook Pro PC with 2.9 GHz Dual-Core Intel Core i5 CPU and 8 GB 2133 MHz LPDDR3 RAM. Table 5.10 and Table 5.11 show that imbalanced dataset requires more time to train and test than balanced dataset. We can easily observe that the SVC polynomial classifier has larger running time in all cases, which means it is not a good choice if the DG units are resource limited in the microgrid system. SVC RBF has the least time cost in all three SVM based classifiers. The three tree based classifiers, Decision Tree, AdaBoost, and Random Forest have very fast training and testing speeds compared to other classifiers, where Decision Tree and AdaBoost are slightly faster than Random Forest.

The obtained results show that the learning models of the three algorithms: Decision Tree, Random Forest and AdaBoost provide good performance when training and validating either

Perspectives de recherche

Cette thèse ouvre de nombreuses perspectives visant à améliorer notre approche. Tout d'abord, il faudrait étudier la détection d'autres attaques et d'autres anomalies dues à des dysfonctionnement et développer des modèles capables de les distinguer. À un autre niveau, notre solution est seulement appliquée aux micro-réseaux et leurs système de contrôle, il est également intéressant d'adapter notre approche aux autres environnements à l'image de smart-home, smart-city, etc. Enfin, la méthode de réponse aux attaques, mérite d'être améliorée en reconfigurant par exemple le réseau de communication d'une façon dynamique, ce qui permet de réduire l'impact des attaques et réduire leurs surfaces.