Jean-Michel Portal
email: jean-michel.portal@im2np.fr

Marc Bocquet

Damien Deleruyelle

Christophe Muller

Non-Volatile Flip-Flop Based on Unipolar ReRAM for Power-Down Applications

Keywords: Low-power, Power-down, Flip-Flop, Non volatile memory, Resistive switching Memory, ReRAM

In this paper, we propose a new architecture of non-volatile Flip-Flop based on ReRAM unipolar resistive memory element (RNVFF). This architecture is proposed in the context of powerdown applications. Flip-Flop content is saved into ReRAM memory cell before power-down and restored after power-up. To simulate such a structure a compact model of unipolar ReRAM was developed and calibrated on best in class literature data. The architecture of the RNVFF, based on the insertion of a non-volatile memory block before a master-slave Flip-Flop, is detailed. The save and restore processes are described from the succession of four operating modes (normal, save, read, reset) needed by the save and restore processes. Finally, the structure is fully validated through electrical simulations, when the data to save is either '0' or '1'.

INTRODUCTION

A major challenge in nomad applications is the reduction of power consumption. The mainstream of power reduction is driven since many years by transistor downscaling and concomitant voltage reduction. A side effect of this reduction is the increase of leakage current in sub-threshold regime with more than 40% of active mode energy dissipation due to power leakage [START_REF] Kursun | Multi-Voltage CMOS Circuit Design[END_REF][START_REF] Sery | Life is CMOS: Why chase life after?[END_REF] of idle transistors.

To overcome this issue, solutions based on process changes have been proposed such as high- oxide associated with a metal gate [START_REF] Robertson | High dielectric constant gate oxides for metal oxide Si transistors[END_REF]. Another well-known solution to save power is to power-down subcircuits of System on Chip (SoC) during idle state. However, when sub-circuits are powered-down, the data saved in the Flip-Flops are lost with a subsequent high power budget required for saving/restoring their contents together with sub-threshold leakage current.

Numerous design solutions have been proposed to maintain Flip-Flop contents such as multithreshold voltages MOS transistors used with power gating techniques [START_REF] Jiao | Low-Leakage and Compact Registers with Easy-Sleep Mode[END_REF]. The basic principle to save the Flip-Flop's content during power-down relies on a retention circuit also known as balloon circuit [START_REF] Matsuya | A 1-V high-speed MTCMOS circuit scheme for power-down application circuits[END_REF]. The scheme of a retention Flip-Flop with balloon latch is reproduced in Fig. 1 [START_REF] Matsuya | A 1-V high-speed MTCMOS circuit scheme for power-down application circuits[END_REF]. Using this technique, the master-slave Flip-Flop is connected either to virtual ground or VDD while a balloon latch is connected to real ground and VDD. During power-down, the data of the slave latch in the Flip-Flop is memorized in the balloon latch while the Flip-Flop is disconnected from the ground or VDD thanks to a switch inserted between the real and the virtual ground line.

The integration of Non-Volatile Flip-Flop (NVFF) in SoC may also be a solution to lower power consumption. The recent emergence of innovative low voltage memory concepts paves the way for novel NVFF solutions as already demonstrated with either ferroelectric FeRAM [START_REF] Yan | A design of ferro-DFF for non-volatile systems[END_REF][START_REF] Wang | A Compare-and-write Ferroelectric Nonvolatile Flip-Flop for Energy-Harvesting Applications[END_REF] or magnetoresistive MRAM [START_REF] Zhao | Spin-MTJ based Non-Volatile Flip-Flop[END_REF][START_REF] Guillemenet | On the use of magnetic RAMs in field-programmable gate arrays[END_REF][START_REF] Yamamoto | Nonvolatile delay Flip-Flop based on spin-transistor architecture and its power-gating applications[END_REF] memories and recently with bipolar memristive devices [START_REF] Robinett | A memristor-based nonvolatile latch circuit[END_REF]. Fig. 2 depicts FeRAM-based NVFF [START_REF] Wang | A Compare-and-write Ferroelectric Nonvolatile Flip-Flop for Energy-Harvesting Applications[END_REF] solution, in which a non-volatile back-up module based on the insertion of two FeRAM memory cells is used to save and restore the Flip-Flop content during power-down. This back-up module is connected to the output of the slave stage of the Flip-Flop. The MRAM-based solution [START_REF] Zhao | Spin-MTJ based Non-Volatile Flip-Flop[END_REF] illustrated in Fig. 3 is based on the insertion of two MRAM cells in the master stage of the Flip-Flop. Here again, the modified non-volatile master stage enables storing data during the power-down phase. Even if both technologies are compatible with CMOS standard processes, they rely either on complex stack of magnetic layers for MRAM or on a high temperature crystallization ferroelectric oxide for FeRAM.

The main purpose of this paper is to show how emerging memory concept relying on unipolar resistive switching (namely ReRAM standing for Resistive Random Access Memory) may also represent an interesting solution for implementation in NVFF (RNVFF for ReRAM NVFF). This solution could benefit from the good compatibility between ReRAM and CMOS technologies. The paper is organized as follows. Section 2 is dedicated to unipolar ReRAM physical model description and its calibration on best in class literature data. In section 3, the save/restore processes of RNVFF are detailed together with the architecture of the Flip-Flop. Section 4 presents simulation results that validate the efficiency of the solution. Finally, section 5 gives concluding remarks.

UNIPOLAR RERAM PHYSICAL MODEL OVERVIEW

Introduction

NiO-based unipolar resistive switching device (ReRAM) is a good candidate for distributed memory applications due to its simple MIM (metal/Insulator/metal) structure, good compatibility with standard CMOS processes, low operating voltage (i.e. below 1 V in [START_REF] Kim | Electrical observations of filamentary conductions for the resistive memory switching in NiO films[END_REF]) and fast programming time (i.e. in the sub-10 ns range in [START_REF] Tsunoda | Low Power and High Speed Switching of Ti-doped NiO ReRAM under the Unipolar Voltage Source of less than 3 V[END_REF]).

For the particular class of ReRAM devices relying on thermochemical mechanisms, the memory effect is due to creation/destruction of conductive filaments (CF) within the oxide providing two resistive states named low resistance state (LRS) and high resistance state (HRS). In unipolar ReRAM, the same voltage polarity enables switching either from HRS to LRS (set) and from LRS to HRS (reset). The main drawback is the "electroforming" stage required to create initial CFs within a virgin dielectric oxide. In fact, this process requiring a higher voltage compared to set and reset voltages could be a strong issue when embedding ReRAM in CMOS logic. However, recent works have proposed technological solutions that enable reducing forming voltages to the level of set voltage, paving the way toward "forming-free" devices [START_REF] Bruchhaus | Memristive Switches with Two Switching Polarities in a Forming Free Device Structure[END_REF].

Unipolar ReRAM physical model description

The proposed RNVFF circuit relies on a compact model accounting for both set and reset operations in NiO-based unipolar resistive switching devices [START_REF] Bocquet | Self-consistent physical modeling of set/reset operations in unipolar resistive-switching memories[END_REF]. The initial physical model takes into account two mechanisms: redox reactions (i.e. electrochemical oxidation/reduction processes) and thermal diffusion. Set operation is governed by a local reduction process leading to the creation of CFs, whereas reset operation involves both oxidation reaction and thermal diffusion. Nevertheless, considering involved activation energies, the oxidation mechanism may be neglected, the diffusion process mainly governing the reset operation. The description of set and reset operations relies on a self-consistent kinetics equation (eq. 1) linking diffusion and reduction velocities diff and red respectively to the dimensionless concentration of metallic species CNi.

The local diffusion velocity diff (eq. 2) of the metallic species explains the thermal rupture of CF during reset operation [START_REF] Russo | Self-Accelerated Thermal Dissolution Model for Reset Programming in Unipolar Resistive-Switching Memory (ReRAM) Devices[END_REF]. In equation 2, Ea is the activation energy governing the thermally-assisted exodiffusion of metallic species, kdiff is the thermal diffusion rate and TCF represents the temperature of CF:

Ni T k Ea diff diff C e k CF b       (2)
Besides, equation 3 gives a simplified expression of the reduction velocity red (expressed by

diff red Ni dt dC     (1)
classical Butler-Volmer equation [START_REF] Bard | Electrochemical Methods: Fundamentals and Applications[END_REF]), in which  is the asymmetry factor, k0 is the reaction rate, E0 is the free energy of the reaction at equilibrium, VCell is the applied voltage and TOx is the oxide temperature:

  Ni T k V q E 0 red C 1 e k v Ox b Cell 0          (3)
In equation 2, the local CF temperature TCF(x) in x direction increases along with the applied voltage (VCell) due to Joule effect as described in equation 4. In this latter equation, CF is the CF conductivity, Kth is the CF thermal conductivity, tOx is the oxide thickness and Tamb is ambient temperature. Solving the 1D heat equation, the CF temperature is given by:

                        2 2 Ox 2 Ox Cell th CF amb CF x 4 t t V K T x T (4)
Finally, it must be underlined that the present physical model enables continuously accounting for both creation (set) and destruction (reset) of conductive filaments. This numerical feature is a key point for a model dedicated to be implemented in computer-aided design tools.

ReRAM model calibration and model card extraction

Before simulating circuits integrating ReRAM devices, the physical model was confronted to quasi-static and dynamic I(V) characteristics measured on actual devices [START_REF] Kim | Electrical observations of filamentary conductions for the resistive memory switching in NiO films[END_REF][START_REF] Cagli | Evidence for threshold switching in the set process of NiO-based ReRAM and physical modeling for set, reset, retention and disturb prediction[END_REF][START_REF] Lee | 2-stack 1D-1R Cross-point Structure with Oxide Diodes as Switch Elements for High Density Resistance RAM Applications[END_REF]. Fig. 4 shows quasi-static set and reset I(V) characteristics measured on NiO-based memory elements by several authors [START_REF] Kim | Electrical observations of filamentary conductions for the resistive memory switching in NiO films[END_REF][START_REF] Cagli | Evidence for threshold switching in the set process of NiO-based ReRAM and physical modeling for set, reset, retention and disturb prediction[END_REF][START_REF] Lee | 2-stack 1D-1R Cross-point Structure with Oxide Diodes as Switch Elements for High Density Resistance RAM Applications[END_REF]. The physical model shows an excellent agreement with experimental data for both set and reset operations, which demonstrates its flexibility to match electrical data reported on various technologies. Moreover, Fig. 5 reports experimental and simulated evolutions of reset current Ireset as a function of the maximum current ISetMax used in preceding set operation [START_REF] Nardi | Control of filament size and reduction of reset current below 10 μA in NiO resistance switching memories[END_REF]. The proposed model well catches the universal Ireset = f(ISetMax) trend observed on various NiO-based technologies and confirms the scalability of the reset current [START_REF] Nardi | Control of filament size and reduction of reset current below 10 μA in NiO resistance switching memories[END_REF]. Besides, as reported in ref. [START_REF] Bocquet | Self-consistent physical modeling of set/reset operations in unipolar resistive-switching memories[END_REF], the model is also able to fit the evolution of programming voltages along with ramp speed to describe the dynamic behavior of memory elements.

Among I(V) characteristics shown in Fig. 4, data published by Kim et al. [START_REF] Kim | Electrical observations of filamentary conductions for the resistive memory switching in NiO films[END_REF] exhibiting switching voltages compatible with 65 nm CMOS technology VDD (Fig. 4c) are selected to extract the model card for design purpose. This latter model card also fulfilled the condition of achieving set and reset operations in 10 ns under 1.2 V bias are reported in ref. [START_REF] Tsunoda | Low Power and High Speed Switching of Ti-doped NiO ReRAM under the Unipolar Voltage Source of less than 3 V[END_REF].

RNVFF ARCHITECTURE WITH SAVE/RESTORE PROCESS DESCRIPTION

Introduction

In power-down applications, Flip-Flop with non-volatile capability might be an alternative solution to power gating technique. The architecture of the non-volatile Flip-Flop is presented in section 3.2, while the different operating phases of the save and restore processes are described in section 3.3.

RNVFF architecture

Proposed RNVFF solution relies on the implementation of a non-volatile memory (NVM) block connected to the input of a conventional master-slave Flip-Flop. As illustrated Fig. 6 the NVM block is composed of routing components (input tri-states inverters and output multiplexer). In between lies a branch that connects a central point (MEM) to VDD through two serial PMOS (MP1 and MP2) on one hand and to ground through a ReRAM memory cell on the other hand. It has to be noticed that the area of this latter branch is reduced since ReRAM element may be processed in the back-end of line on the top of CMOS level. The input tri-states inverter enables connecting and isolating the test feature with a multiplexer on their input. Therefore the multiplexer of the NVM block could be mixed with the scanmultiplexer to introduce a minimal delay overhead.

To summarize, the structure of the NVM block is very simple with two routing elements (input tristates inverter and output multiplexer) and a branch with two transistors and one ReRAM element (i.e.

2T/1R structure). Considering that the output multiplexer may be mixed with a scan multiplexer, the area overhead introduced by the structure is one tri-states inverter and a 2T/1R branch.

Save and restore processes description

To save and restore Flip-Flop content, four successive operating modes are required:  The normal mode in which the Flip-Flop works in a conventional manner;  The save mode in which Flip-Flop content is saved in NVM block;  The read mode in which NVM block content is restored in the Flip-Flop;  The reset mode in which ReRAM memory cell turns back to a high resistance state while the Flip-Flop works in a conventional way.

Active path in normal operating mode is illustrated by solid red line in Fig. 7. In this mode the ReRAM cell is isolated from the input (IN) of the structure thanks to the input tri-states inverter, which is open (SAVE_EN = '0'). It has to be mentioned, that in this operating mode, the ReRAM cell remains in its initial high resistance state (i.e. HRS). The Flip-Flop works in a conventional manner by connecting the input (IN) of the structure to the input (D) of the Flip-Flop thanks to the output multiplexer (READ_EN = '1'). The delay overhead introduces is due to the extra-load of the tri-states input inverters and the routing through the output multiplexer.

Again, the actives paths of the save operating mode are illustrated by the solid red line in Fig. 8. In this mode the ReRAM cell is connected to the input (IN) of the structure thanks to the input tri-states inverter, which is turned on (SAVE_EN = '1'). The MEM central point is now the complement of the input (IN) value. When the tri-states inverter is activated, the ReRAM cell is switched to its low resistance state LRS (IN = '0' and MEM = '1') or remains in a high resistance state HRS (IN='1' and MEM='0'). The Flip-Flop continues to work in a conventional manner and store the value of the input (IN) thanks to the output multiplexer (READ_EN = '1'). So during the save mode, the ReRAM resistance state reflects the value of the Flip-Flop. Logic '1' corresponds to a high resistance state whereas logic '0' corresponds to a low resistance state. Right after the save mode, the Flip-Flop can be completely powered-down since ReRAM cell stores the information.

After power-up, a read mode is mandatory to restore Flip-Flop content. Active path in read operating mode is illustrated in Fig. 9. In this mode, the output multiplexer (READ_EN = '0') enables connecting the MEM node to the input (D) of the Flip-Flop, while a clock edge restores the content of the Flip-Flop. The resistive bridge (voltage divider) composed by MP1 (turned-on with RESET_EN = '0') and MP2 on one side and the ReRAM cell on the other side determines the voltage on the node MEM. If the ReRAM cell is in its LRS state then the MEM voltage is grounded and a '0' is restored in the Flip-Flop. In contrast, if the MEM voltage is closed to VDD, a '1' is restored in the Flip-Flop. MP2 is introduced in the branch to limit the voltage on the node MEM when close to VDD in order to initiate the switching of the ReRAM cell to a HRS state during the read mode, if needed (ReRAM cell in a LRS state).

Active paths of the reset operating mode are illustrated in Fig. 10. In this mode, the Flip-Flop works again in a conventional way thanks to the output multiplexer that connects the input (IN) of the structure to the input of the Flip-Flop (READ_EN = '1'). Doing so, the ReRAM cell is isolated from the rest of the circuit and can turn back to a HRS state, if needed, thanks to the application of a voltage close to VDD through MP1 (turned-on with RESET_EN = '0') and MP2. When RESET_EN = '1', the reset process is stopped and the whole structure turns back to a normal operating mode.

Conclusion

In conclusion, it is important to note that the standard functionality of the Flip-Flop is guaranteed in all modes, except during read mode. Moreover, the ReRAM state is HRS in all modes when the content to save is '1' with a minimal leakage current consumption. When the content to save is '0' then the current consumption is restricted to the save, read and reset modes with the switching of the ReRAM cell from HRS to LRS and respectively from LRS to HRS. It is also important to underline that no biasing is necessary during power-off to preserve ReRAM state.

RNVFF SIMULATION RESULTS

To validate the RNVFF functionality, the full structure is simulated under electrical simulator using a low power CMOS 65 nm design kit and the unipolar ReRAM compact model fitted on best in class literature data. VDD is nominal for the technology and set to 1.2 V during all operating modes except during power-off where it is set to 0 V. All operating modes are simulated for input values of '1' and '0'. MP2 has a minimal length (L=0.06 µm) and a double width (W=0.24 µm) and MP1 has minimal dimensions (L=0.06 µm and W=0.12 µm). The tri-states inverter is composed of NMOS and PMOS transistors with a double length (L=0.12 µm) to limit current during set process and standard width (WPMOS=0.15 µm, WNMOS=0.12 µm). The output multiplexer and the Flip-Flop are standard-cells from the library. i.e. normal mode, save mode, power-off, read mode, reset mode and again normal mode. Fig. 12 shows the chronograms of the input (IN) , the Flip-Flop input (D) and the current through the ReRAM cell (IReRAM) to save and restore a logic '0'. As described in the previous section, the ReRAM is set during the save mode with a current of 12 µA during 2 ns. During the read and reset modes, the ReRAM is reset during 5 ns with a current decreasing from 9 µA to 0 µA. During the read mode, the input IN is set to '1' while the read process forces a '0' value at the input D of the Flip-Flop, validating successfully the save and restore processes for a data equal to '0'. Fig. 13 shows the chronograms of the input (IN), the Flip-Flop input (D) and the current through the ReRAM cell (IReRAM) to save and restore a logic '1'. As previously described, the ReRAM cell remains in HRS during the save mode with a current below 0.1 µA during 2 ns. During the read and reset modes, the ReRAM cell remains in HRS during 5 ns with a current decreasing from 0.4 µA to 0 µA. During the read mode, the input IN is set to '0' while the read process enforces a '1' value at the input D of the Flip-Flop, validating successfully the save and restore processes of a data equal to '1'.

In conclusion, the simulation results validate successfully the functionality of the RNVFF in all operating modes. The simulation also demonstrates that the current consumption of this structure is restricted to the save and restore processes of a logic '0'. Indeed, the ReRAM cell remains always in HRS, when the data to save and restore is equal to '1'.

CONCLUSION

In this paper, a new architecture of non-volatile Flip-Flop based on unipolar Resistive RAM is proposed. This latter architecture is dedicated to power-down applications, in which the content of the Flip-Flop is saved as resistance states in a ReRAM device before power-down and restored after power-up. The overall save and restore processes are detailed together with the architecture of the proposed structure. One may notice that this architecture relies on a non-volatile memory block inserted at the front of a Flip-Flop. The first advantage of such a structure is a better compatibility between the ReRAM memory element and the CMOS level as compared to MRAM or FeRAMbased solutions. Moreover, the use of such a structure does not require any biasing during power-off in comparison to retention Flip-Flop employing a balloon latch. Another point to underline is the low power consumption during all operating modes, except when the cell is set or reset (corresponding to save and read/reset of a '0' content). Finally, the full structure is successfully validated with electrical simulation using a 65 nm CMOS design kit and the unipolar compact model calibrated on best in class data from the literature.

FIGURES AND TABLES

Figure 1. Architecture of a classical balloon latch used with power-gating technique (redrawn from [START_REF] Matsuya | A 1-V high-speed MTCMOS circuit scheme for power-down application circuits[END_REF]). ISet IReset [START_REF] Cagli | Evidence for threshold switching in the set process of NiO-based ReRAM and physical modeling for set, reset, retention and disturb prediction[END_REF] Simu. this work

V Cell (V)

ICell (A)

VCell (V)

Symbols: [START_REF] Kim | Electrical observations of filamentary conductions for the resistive memory switching in NiO films[END_REF] Lines: Simulations

  central point (MEM) to the input of the structure (IN). PMOS transistors of this inverter are adequately sized to provide a current limitation through the ReRAM element when connecting the input IN to the central point MEM. The two inputs multiplexer enables bypassing NVM block, when the signal READ_EN = '1', otherwise (READ_EN = '0') the output of the NVM block is connected to the input (D) of the Flip-Flop. It is worth noting that in most of SoC, Flip-Flops integrate a scan

Fig. 11

 11 Fig. 11 presents the chronograms of VDD and control signals, i.e. SAVE_EN, READ_EN and

Figure 2 .

 2 Figure 2. Flip-Flop architecture with a back-up module based on FeRAM memory for energy harvesting application (redrawn from[START_REF] Wang | A Compare-and-write Ferroelectric Nonvolatile Flip-Flop for Energy-Harvesting Applications[END_REF]).

Figure 3 .

 3 Figure 3. Architecture of a latch with a non-volatility capability based on two magnetic tunnel junctions MTJ (redrawn from[START_REF] Zhao | Spin-MTJ based Non-Volatile Flip-Flop[END_REF]).

Figure 4 .

 4 Figure 4. Experimental I(V) characteristics measured on a NiO-based memory structure reported in (a) [18], (b) [19], and (c) [12] and corresponding simulations using the presented ReRAM physical model.

Figure 5 .

 5 Figure 5. Maximum current during the reset operation (IReset) as a function of the maximum current during the preceding set operation (ISetMax). Experimental data were extracted from [20].

Figure 6 .Figure 7 .

 67 Figure 6. RNVFF architecture with Non-volatile block based on ReRAM cell connected to the Flip-Flop input D.

Figure 8 .Figure 9 .

 89 Figure 8. RNVFF active path during save operating mode.

Figure 10 .Figure 11 .

 1011 Figure 10. RNVFF active path during reset operating mode.

Figure 12 .Figure 13 .

 1213 Figure 12. Chronograms of data signals and ReRAM current during the save & restore processes using a RNVFF, with IN='0'.


Ali Castaings

Alain Bouscayrol
email: alain.bouscayrol@univ-lille1.fr

Walter Lhomme

R Trigui

Power Hardware-In-the-Loop simulation

Keywords: hardware-in-the-loop, energy management, supercapacitor, battery, fuel cell, vehicle

come

The urban travel demand is significantly growing. According to the International Energy Agency the 2012 concentration of CO2 was about 40% higher than in the mid-1800s (IEA, 2013). It is then important to find out alternatives to conventional thermal vehicles.

Several solutions have been depicted such as battery electric vehicles or Fuel Cell (FC) vehicles [START_REF] Chan | Electric, Hybrid, and Fuel-Cell Vehicles: Architectures and Modeling[END_REF]. However, each solution has some limitations. FCs have some power transfer issues [START_REF] Bernard | Fuel-Cell Hybrid Powertrain: Toward Minimization of Hydrogen Consumption[END_REF] while batteries have some lifetime issues [START_REF] Omar | Lithium iron phosphate based battery -Assessment of the aging parameters and development of cycle life model[END_REF]. Multi-sources vehicles represent an interesting alternative as they enable to take advantage of the properties of the different sources [START_REF] Ehsani | Modern Electric, Hybrid Electric, and Fuel Cell Vehicles: Fundamentals, Theory, and Design[END_REF]. However, they represent very complex systems. It is then difficult to manage such systems.

Several works has been done on Energy Management Strategies (EMSs) of multi-sources vehicles. Two approaches have been depicted [START_REF] Salmasi | Control Strategies for Hybrid Electric Vehicles: Evolution, Classification, Comparison, and Future Trends[END_REF][START_REF] Wirasingha | Classification and Review of Control Strategies for Plug-In Hybrid Electric Vehicles[END_REF], rule-based approach [START_REF] García | Operation mode control of a hybrid power system based on fuel cell/battery/ultracapacitor for an electric tramway[END_REF][START_REF] Thounthong | Energy management of fuel cell/battery/supercapacitor hybrid power source for vehicle applications[END_REF] and optimization-based approach [START_REF] Yu | An innovative optimal power allocation strategy for fuel cell, battery and supercapacitor hybrid electric vehicle[END_REF][START_REF] Odeim | Power Management Optimization of a Fuel Cell/Battery/Supercapacitor Hybrid System for Transit Bus Applications[END_REF]. The main issues are related to real-time applications. Moreover, the EMSs have to ensure the physical limitations (i.e. overcharge or depleting) of the sources for any driving condition. As a consequence, it is important to find testing procedures to assess the EMSs before their implementation in a real vehicle.

Power Hardware-in-The-Loop (P-HIL) simulation [START_REF] Bouscayrol | Hardware-in-the-loop simulation[END_REF] has been used in several applications for testing components before their implementation in a real system. P-HIL has thus been used for testing EMSs of hybrid and electric vehicles in real-time conditions [START_REF] Allègre | Flexible real-time control of a hybrid energy storage system for electric vehicles[END_REF][START_REF] Castaings | Practical control schemes of a battery/supercapacitor system for electric vehicle[END_REF][START_REF] Odeim | Power Management Optimization of an Experimental Fuel Cell/Battery/Supercapacitor Hybrid System[END_REF].

The objective of this paper is to present a P-HIL simulation of a FC-battery-Supercapacitors (SCs) vehicle. The developed platform enables to assess an EMS in real-time conditions (e.g. various driving conditions). The control organization of the P-HIL simulation is achieved by using Energetic Macroscopic Representation (EMR) [START_REF] Bouscayrol | Graphic Formalisms for the Control of Multi-Physical Energetic Systems[END_REF]. The second section is devoted to the description of the P-HIL simulation. The control organization is presented in the third section. The results are given in the last section before the conclusion.

II. P-HIL SIMULATION OF THE STUDIED SYSTEM

A. P-HIL principle

Hardware-In-the Loop simulation consists in adding some actual elements (hardware) in the simulation loop [START_REF] Bouscayrol | Hardware-in-the-loop simulation[END_REF]. In Power HIL, some power elements can be tested before their implementation on the real system. It is useful for testing the subsystem and its control in real-time conditions.

(Figure 1.a). In P-HIL simulation, the power part is split into two parts, the part under test (with its control) and the emulated part. An interface (interf. in ). The emulation system must have the same behavior than the simulated system. The control references of the emulation system come from the simulation of the emulated system.

Also, the interface must be faster than the emulated system to emulate without delay. The next part is devoted to the control organization of the P-HIL system. Different parts have to be interconnected. Indeed, there are the models simulations, the emulation subsystems with their control, the tested subsystems and their control. A graphical formalism, Energetic Macroscopic Representation (EMR) is used as a tool for achieving the subsystems interconnection. First, EMR is based on action-reaction principle. It enables to ensure a physical connection between the elements. Second, EMR approach is based on causality principle. It enables to deduce the control structure of the system and to use real-time models for the emulation subsystems.

III. CONTROL ORGANIZATION

A. Real part

The control of the system is achieved by using Energetic Macroscopic Representation (EMR). EMR highlights energetic properties of the components of a system to develop control schemes [START_REF] Bouscayrol | Graphic Formalisms for the Control of Multi-Physical Energetic Systems[END_REF]. There are several pictograms to represent the system model (see Appendix). By using EMR approach, the control part is organized in two levels, the "local control" part and the "global control" part (i.e. EMS). The main interest of using EMR is that the "local control" part of the system can be systematically deduced by "mirror" effect from its EMR. The EMR and the control part of the "real part" of the system are depicted Figure 4.

Local control part

The local control is represented by the light blue blocks in

Global control part

The global control part corresponds to the Energy Management Strategy (EMS). That aims to use the degrees of freedom of the control in the best way. There are two kinds of EMSs for multi-sources vehicles; rule-based EMSs and optimization-based EMSs [START_REF] Salmasi | Control Strategies for Hybrid Electric Vehicles: Evolution, Classification, Comparison, and Future Trends[END_REF] Figure 4: EMR and control organization of the "real part"

B. Emulated parts

The EMR and its control organization of the emulated parts are depicted Figure 5. The purple blocks correspond to the simulated part of the P-HIL simulation. As it can be noticed, the control references come from the simulation of the real components models (purple pictograms). Also this is a reduced-scale P-HIL simulation. As a result, some adaption coefficients are taken into account [START_REF] Allègre | Flexible real-time control of a hybrid energy storage system for electric vehicles[END_REF]. These coefficients enable to pass from the full-scale simulated models to the reference signals of the reduced-scale system (1). The reduced-scale coefficients values are given in Table 1. VALIDATION OF AN ENERGY MANAGEMENT STRATEGY

A. Principle

The experimental setup is presented Figure 6. A dSPACE 1005 card is used as an interface between the power part and the computer board. The EMS is an optimization-based strategy. It consists in minimizing the hydrogen consumption while improving the battery lifetime [START_REF] Castaings | Comparison of energy management strategies of a battery/supercapacitors system for electric vehicle under real-time constraints[END_REF].

The first test is achieved on a standard driving cycle (WLTC class 2, low velocity phase Figure 7) where the EMS parameters have been identified. This corresponds to "ideal" driving conditions. The second test is carried out using a real driving cycle (Figure 8) coming from results on the instrumented car (Tazzari Zero) [START_REF] Depature | Efficiency Map of the Traction System of an Electric Vehicle from an On-Road Test Drive[END_REF]. This test enables to assess the robustness of the EMS when varying the driving conditions. The parameters of the full scale and reduced scale systems are given in Table 1. This is interesting for its lifetime (Figure 9). As depicted in Figure 10 the EMS enables to respect the SCs voltage limitations. This is important for ensuring the system safety. This aspect has been assessed thanks to the P-HIL platform. The same trends can be noticed for the real driving cycle (Figure 11 and Figure 12). The key point is that the EMS still enables to reach interesting performances while ensuring the system safety. However, as the parameters were not computed on this driving cycle, the SCs tend to be discharged at the end of the driving cycle. This can cause some repeatability issues. Indeed, if the same driving cycle is repeated, the results won't be the same as the previous one.

Csc=19 F usc-M=44 V | usc-m=0.65usc-M uSC-0=0.9usc-M Battery 24 cells (3.3 V / 20 Ah / 820 W) SoCb-M=100 % | SoCb-m=90 % SoCb-0=95 % ki-sbat=1/17 ku-sbat=1

  Figure 1.a) is required for connecting the simulation signals and the power signals. The interface has then power and signals elements. (Figure 1.b

Figure 1

 1 Figure 1: P-HIL, (a) principle, (b) practical scheme

Figure 2 :

 2 Figure 2: Architecture of the studied vehicle P-HIL organization The objective of the P-HIL simulation is to assess the system controllability in real-time and to validate the FC and SCs behavior. In the presented work, the emulated parts are the battery branch and the traction part. The corresponding emulation systems are depicted Figure 3. For the battery branch, the battery is replaced by a SCs bank. The SCs bank has to reflect two battery characteristics  the battery SoC limitations : this depends on the SCs bank size  the battery voltage dynamics. The SCs voltage has higher dynamics than the battery voltage ones. If the battery model is accurate enough, the battery voltage dynamics can be reflected by the SCs. The traction part is emulated by a current source composed of a DC-DC converter, a smoothing inductor and a SCs bank. The main dynamics of the inverter current are taken into account in the traction model (cf. section III).

Figure 3 :

 3 Figure 3: P-HIIL system architecture

Figure 4 .

 4 It manages the system components to track the reference of the DC bus voltage. The right duty cycles of the converters (αb, αfc and αsc) are then defined. In addition, the local control points out the control requirements. In the studied case 4 sensors and 4 controllers (closed-loop control) are required as well. The inversion of an accumulation element is performed via a closed-loop control (crossed blue parallelogram). A conversion element is directly inverted with an open-loop control (blue parallelogram). The inversion of a coupling element depicts degrees of freedom that correspond to the output of the EMS (global control).

Figure 5 :

 5 Figure 5: EMR and control organization of the emulated parts

Figure 6 :

 6 Figure 6: Experimental setup

Figure 9 :

 9 Figure 9: FC branch and traction currents

  Figure 11: FC branch and traction currents


Samson Lasaulce
email: lasaulce@l2s.centralesupelec.fr

Sophie Tarbouriech
email: so-phie.tarbouriech@laas.fr

Information constraints in multiple agent problems with i.i.d. states

. In wireless communications, it may represent the state of the global communication channel. The approach used is to exploit Shannon theory to characterize the achievable long-term utility region. Two scenarios are described. In the first scenario, the number of agents is arbitrary and the agents have causal knowledge about the state. In the second scenario, there are only two agents and the agents have some knowledge about the future of the state, making its knowledge non-causal.

Chapter overview

This chapter concerns the problem of coordination among agents. Technically, the problem is as follows. We consider a set of K ≥ 2 agents. Agent k has a utility, payoff, or reward function u k (x 0 , x 1 , ..., x K ) where x k , k ≥ 1, is the action of Agent k while x 0 is the action of an agent called Nature. The Nature's actions correspond to the system state and is assumed to be non-controlled; more precisely, Nature corresponds to an independent and identically distributed (i.i.d.) random process. The problem studied in this chapter is to characterize the long-term utility region

under various assumptions in terms of observation at the agents. By long-term utility for Agent k we mean the following quantities:

U k (σ 1 , ..., σ K ) = lim T →+∞ 1 T E T ∑ t=1 u i (X 0 (t), ..., X K (t)) (1)
where σ k = (σ k,t ) t≥1 is a sequence of functions which represent the strategy of Agent k, x k (t) is the action chosen by Agent k at time or stage t ≥ 1, t being the time or stage index; concerning notations, as far as random variables are concerned, capital letters will stand for random variables whereas, small letters will stand for realizations. Note that, implicitly, we assume sufficient conditions (such as utility boundedness) under which the above limit exists. The functions σ k,t , k ∈ {1, ..., K}, map the available knowledge to the action of the considered agent. The available knowledge depends on the information assumptions made (e.g., the knowledge of the state can be causal or non-causal). We will distinguish between two scenarios.

In the first scenario, agents are assumed to have some causal knowledge (in the wide sense) about the state whereas, in the second scenario non-causal knowledge (i.e., some knowledge about the future) about the state is assumed. The second scenario is definitely the most difficult one technically, which is why only two agents will be assumed.

Remarkably, the long-term utility region, whenever available, can be characterized in terms of elegant information constraints. For instance, in the scenario of non-causal state information, determining the long-term utility region amounts to solving a convex optimization problem whose non-trivial constraints are the derived information-theoretic constraints.

Introduction

An important example, which illustrates well how the results reported in this chapter can be used, is given by the problem of power control in wireless networks (see Fig. 1). Each transmitter has to adapt its transmit power not only to the fluctuations of the quality of the link (or channel gain) between itself and its respective receiver but also to the transmit power levels of the other transmitters that uses the same radio resources (and therefore create interference). This problem is a multi-agent problem where the agents are the transmitters, the actions of the agents are their transmit power level, and the system state is given by the set of channel gains of the various links in presence; channel gains are typically non-controlled variables (they do not depend on the transmit power levels) and evolve in a random manner; in practice, each transmitter has a partial and imperfect knowledge of the system state. Now, if the agents (namely, the transmitters in the considered example) have a certain performance criterion, which will be referred to as a utility function for the general setup considered in the chapter, the important problem of knowing the best achievable utilities appears. For instance, a transmitter might be designed to maximize its communication rate. The best data rate of a given transmitter would be obtained if all the other transmitters would be silent (i.e., when they don't transmit) and when the transmitter perfectly adapts its power to the channel gain fluctuations of the link between itself and its intended receiver. Obviously, in the real life, several transmitters will transmit at the same time, hence the need to coordinate as well as possible, which leads to the problem of characterizing the best performance possible in terms of coordination. This precisely corresponds to the problem of characterizing the long-term utility region i.e., the set of possible achievable points (U 1 ,U 2 , ...,U K ) for a given definition for the strategies. In Sections. 3 and 4, we will consider two different definitions for the strategies, each of them corresponding to a given observation structure that is, to some given information assumptions.

Fig. 1 The problem of power control in wireless networks is a typical application for the results provided in this chapter. The agents are the transmitters, the agents' actions are given by the transmit power level, and the agent utility function may be its communication rate with its intended receiver.

General problem formulation

This chapter aims at describing a few special instances of a general problem which has been addressed in several recent works [START_REF] Larrousse | Coded Power Control: Performance Analysis[END_REF], [START_REF] Larrousse | Implicit coordination in two-agent team problems. Application to distributed power allocation[END_REF], [START_REF] Larrousse | Coordination in state-dependent distributed networks: The two-agent case[END_REF], [START_REF] Larrousse | Coordination in State-Dependent Distributed Networks[END_REF], [START_REF] Agrawal | A framework for decentralized power control with partial channel state information[END_REF], [START_REF] Larrousse | Coordination in distributed networks via coded actions with application to power control[END_REF], [START_REF] Treust | Joint empirical coordination of source and channel[END_REF].

We consider K ≥ 2 agents, where Agent k ∈ {1, . . . , K} produces time-t action x k (t) ∈ X k 1 for t ∈ {1, . . . , T }, T ≥ 1, the set X k representing the set of actions for Agent k. Each agent has access to some observations associated with the chosen actions and the realization of a random process {X 0,t } T t=1 = {X 0,1 , ..., X 0,T } ∈ X T 0 . In the motivating example described in the introduction, the random process was given by the global wireless channel state i.e., the set of qualities of all the links in presence. In a control problem, the random process may represent a non-controlled perturbation or some uncertainty. All agents' actions and the random process also affect the agents' individual stage or instantaneous utility functions u 1 , ..., u K where for all k ∈ {1, ..., K} the function u k writes:

u k : X 0 × X 1 × ... × X K → R (x 0 , x 1 , ..., x K ) → u k (x 0 , x 1 , ..., x K ) . (2)
One of the main goals of the chapter is to explain how to determine the set of feasible expected long-term utilities:

U (T ) k = E 1 T T ∑ t=1 u k (X 0,t , X 1,t , ..., X K,t ) , (3)
that are reachable by some strategies for the agents. The set of feasible utilities is fully characterized by the set of feasible averaged joint probability distributions on the (K +1)-uple {(X 0,t , X 1,t , ..., X K,t )} T t=1 . Indeed, denoting by P X 0,t X 1,t ...X K,t the joint probability distribution of the time (K + 1)-uple (X 0,t , X 1,t , ..., X K,t ), we have

U (T ) k = 1 T T ∑ t=1 E [u k (X 0,t , X 1,t , ..., X K,t )] = 1 T T ∑ t=1 ∑ x 0 ,...,x K P X 0,t X 1,t ...X K,t (x 0 , x 1 , ..., x K )u k (x 0 , x 1 , ..., x K ) = ∑ x 0 ,...,x K u k (x 0 , x 1 , ..., x K ) 1 T T ∑ t=1 P X 0,t X 1,t ...X K,t (x 0 , x 1 , ..., x K ).
Therefore the problem of characterizing the long-term utility region amounts to determining the set of averaged distributions

P (T ) (x 0 , x 1 , ..., x K ) = 1 T T ∑ t=1 P X 0,t X 1,t ...X K,t (x 0 , x 1 , ..., x K ) (4)
that can be induced by the agents' strategies. For simplicity, and in order to obtain closed-form expressions, we shall focus on the case where T → ∞ [START_REF] Gossner | Optimal use of communication resources[END_REF], [START_REF] Larrousse | Coded Power Control: Performance Analysis[END_REF]. We consider two types of scenarios with two different observation structures. In the first scenario, referred to as the non-causal state information scenario, the agents 1 Throughout the chapter, we assume that all the alphabets such as X k are finite.

observe the system states non-causally. That means, at each stage t ∈ {1, . . . , T } they have some knowledge about the entire state sequence X T 0 = (X 0,1 , . . . , X 0,T ). In the second scenario, referred to as the causal state information scenario, the agents learn the states only causally and therefore, at any stage t, the agents have some knowledge about the sequence X t 0 = (X 0,1 , . . . , X 0,t ), where throughout the chapter we use the shorthand notations A m and a m for the tuples (A 1 , . . . , A m ) and (a 1 , . . . , a m ), when m is a positive integer.

3 Coordination among agents having causal state information

Limiting performance characterization

Firstly, we define the information structure under consideration. At every instant or stage t, Agent k is assumed to have an image or a partial observation S k,t ∈ S k of the nature state X 0,t with respect to which all agents are coordinating. In the case of the wireless power control example described in the introduction, this might be the knowledge of local channel state information, e.g., a noisy estimate of the direct channel between the transmitter and the associated receiver. The observations S k,t are assumed to be generated by a memoryless channel. By memoryless it is meant that the joint conditional probability on sequences of realizations factorizes the product of individual conditional probabilities. Denoting by k the transition probability for the observation structure of Agent k, the memoryless condition can be written as:

P(s T K |x T 0 ) = T ∏ t=1 k (s k (t)|x 0 (t)). (5)
The strategy or the sequence of decision functions for Agent k, σ k,t , is defined by:

σ k,t : S t k -→ X k (6) (s k (1), s k (2), ..., s k (t)) -→ x k (t) (7)
where S k is observation alphabet for Agent k.

As mentioned in Section 2, the problem of characterizing the long-term utility region amounts to determining the achievable correlations measured in terms of joint distribution, hence the notion of implementability for a distribution.

Definition 1. (Implementability) The probability distribution Q(x 0 , x 1 , ..., x N ) is im- plementable if there exist strategies (σ 1,t ) t≥1 , ..., σ K,t ) t≥1 such that as T → +∞, we have for all x ∈ X , 1 T T ∑ t=1 P X 0,t ...X K,t (x 0 , ..., x K ) -→ Q(x 0 , ..., x K ) (8)
where P X 0,t ...X K,t is the joint distribution induced by the strategies at stage t.

The following theorem is precisely based on the notion of implementability and characterizes the achievable long-term utilities that are implementable under the information structure [START_REF] Agrawal | A framework for decentralized power control with partial channel state information[END_REF]; for this, we first define the weighted utility function w as a convex combination of the individual utilities u k :

w = K ∑ k=1 λ k u k . ( 9
)
Theorem 1.

[5] Assume the random process X 0,t to be i.i.d. following a probability distribution ρ and the available information to the transmitters S k,t to be the output of a discrete memoryless channel obtained by marginalizing the joint conditional probability . An expected payoff w is achievable in the limit T → ∞ if and only if it can be written as:

w = ∑ x 0 ,x 1 ,...x N , u,s 1 ,...s N ρ(x 0 )P U (u) (s 1 , ..., s K |x 0 )× ∏ K k=1 P X k |S k ,U (x k |s k , u) w(x 0 , x 1 , ..., x K ). ( 10
)
where U is an auxiliary variable, which can be optimized, and P X k |S k ,U (x k |s k , u) is the probability that Agent k, chooses action x k after observing s k , u.

The auxiliary variable U is an external lottery known to the agents beforehand, which can be used to achieve better coordination e.g., in presence of individual constraints or at equilibrium. Theorem 1 allows us find all the achievable utility vectors (U 1 , ...,U K ). Indeed, the long-term utility region being convex (this readily follows from a time-sharing argument), its Pareto boundary can be found by maximizing the weighted utility w. Of course, remains the problem of determining the strategies allowing to operate at a given arbitrary point of the utility region. Since, this problem is non-trivial and there does not exist any methodology for this, we provide an algorithm which allows one to find a suboptimal strategies. Indeed, the associated multilinear optimization problem is too complex to be solved and to overcome this we resort to an iterative technique which is much less complex but is suboptimal.

An algorithm to determine suboptimal strategies

One of the merits of Theorem 1 is to provide the best performance achievable in terms of long-term utilities when agents have an arbitrary observation structure. However, Theorem 1 does not provide practical strategies which would allow a given utility vector to be reached. Finding "optimal" strategies consists in finding good sequences of functions as defined per [START_REF] Agrawal | A framework for decentralized power control with partial channel state information[END_REF], which is an open and promising direction to be explored. More pragmatically, the authors of [START_REF] Agrawal | A framework for decentralized power control with partial channel state information[END_REF] proposed to restrict to stationary strategies which are merely functions of the form f k : S k → X k . This choice is motivated by practical considerations such as computational complexity and it is also coherent with the current state of the literature. The water-filling solu-tion is a special instance of this class of strategies. To find good decision functions, the idea, which is proposed in [START_REF] Agrawal | A framework for decentralized power control with partial channel state information[END_REF], is to exploit Theorem 1. This is precisely the purpose of this section.

The first observation we make is that the best performance only depends on the vector of conditional probabilities P X 1 |S 1 ,U , ..., P X K |S K ,U and the auxiliary variable probability distribution P U , the other quantities being fixed. It is therefore relevant to try to find an optimum vector of lotteries for every action possible and use it to take decisions. Since this task is typically computationally demanding, a possible and generally suboptimal approach consists in applying a distributed algorithm to maximize the expected weighted utility. The procedure proposed in [START_REF] Agrawal | A framework for decentralized power control with partial channel state information[END_REF] is to use the sequential best response dynamics (see e.g., [START_REF] Lasaulce | Game Theory and Learning for Wireless Networks: Fundamentals and Applications[END_REF]). The idea is to fix all the variables (that are probability distributions here) expect one and maximize the expected weighted utility with respect to the only possible degree of freedom. This operation is then repeated by considering another variable. The key observation to be made is then to see that when the distributions of the other agents are fixed, the best distribution for Agent k boils down to a function of s k , giving us a candidate for a decision function which can be used in practice.

To describe the algorithm of [START_REF] Agrawal | A framework for decentralized power control with partial channel state information[END_REF] (see also Fig. 2), we first rewrite the expected weighted utility in the following manner:

W = ∑ x 0 ,x 1 ,...x K ,u,s 1 ,...s K ρ(x 0 )P U (u)× (11)
Γ (s 1 , ..., s K |x 0 , x 1 , ..., x K )× (12)
K ∏ k=1 P X k |S k ,U (x k |s k , u) w(x 0 , x 1 , ..., x K ) = ∑ i k , j k ,u δ i k , j k ,u P X k |S k ,U (x k |s k , u) (13)
where i k , j k , u are the respective indices of x k , s k , u and

δ i k , j k ,u = ∑ i 0 ρ(x i 0 )Γ k (s k |x i 0 ) ∑ i -k u k (x i 0 , x i 1 , ...x i K )× ∑ j -k ∏ k =k Γ k (s j k |x 0 ) ∏ k =k P X k |S k ,U (x k |s k , u) P U (u) (14)
where i -k , j -k are the indices which represent i k , j k being constant, while all the other indices are summed over. To make the description of the algorithm clearer, we have also assumed the independence of the observation channels as well as independence of the signal with the strategies chosen by the agents, i.e., Γ (s 1 , ...,

s K |x 0 , x 1 , ..., x K ) = Γ 1 (s 1 |x 0 ) × . . . ×Γ K (s K |x 0 )
. Written under this form, for every agent, optimizing the expected weighted utility in a distributed manner im-plies giving a probability 1 for the optimal coefficient δ i k , j k ,u , and every player does that turn by turn.

! Fig. 2 Pseudo-code of the Algorithm proposed in [START_REF] Agrawal | A framework for decentralized power control with partial channel state information[END_REF] to find suboptimal strategies.

To conclude, note that the above algorithm always converges. This can be proved e.g., by induction or by calling for an exact potential game property (see e.g., [START_REF] Mondrere | Potential games[END_REF], [START_REF] Lasaulce | Game Theory and Learning for Wireless Networks: Fundamentals and Applications[END_REF]).

Coordination between two agents having noncausal state information

Limiting performance characterization

As explained previously, the problem of characterizing the utility region in the case where the state is known non-causally to the agents is much more involved technically. Even in the case of two agents, one may have to face with an open problem, depending on the observation structure assumed for the agents. Here, we consider an important case for which the problem can be solved, as shown in [START_REF] Larrousse | Coordination in State-Dependent Distributed Networks[END_REF]. Therein, the authors consider an asymmetric observation structure. In the case of non-causal state information, agents' strategies are sequences of functions that are defined as follows. For Agent 1 the strategy is defined by:

σ 1,t : S T 1 × Y t-1 1 -→ X 1 (15) (s 1 (1), ..., s K (T ), y 1 (1), ..., y 1 (t -1)) -→ x 1 (t) (16)
and for Agent 2 the strategy is defined by:

σ 2,t : S T 2 × Y t-1 2 -→ X 2 (17) (s 2 (1), ..., s 2 (t), y 2 (1), ..., y 2 (t -1)) -→ x 2 (t) (18)
where y k (t) ∈ Y k is the observation Agent k has about the triplet (x 0 (t), x 1 (t), x 2 (t)) whereas s k (t) ∈ S k is the observation Agent k has about the state x 0 (t). Note that distinguishing between the two observations s k and y k is instrumental. Indeed, it does not make any sense physically speaking to assume that an agent might have some future knowledge about the actions of the other agents, which is why the feedback signal is strictly causal. On the other hand, assuming some knowledge about the future of the non-controlled state x 0 perfectly makes sense, as motivated the chapter abstract and the works quoted in the list of references. More precisely the observation is assumed to be the output of a memoryless channel whose transition law is denoted by Γ :

Pr Y 1 (t) = y 1 (t),Y 2 (t) = y 2 (t) X t 0 = s t 0 , X t 1 = x t 1 , X t 2 = x t 2 ,Y t-1 1 = y t-1 1 ,Y t-1 2 = y t-1 2 = Γ (y 1 (t), y 2 (t)|sx 0 (t), x 1 (t), x 2 (t)). (19)
We now provide the characterization of the set of implementable probability distributions both for the considered non-causal strategies. Theorem 2. [START_REF] Larrousse | Coordination in State-Dependent Distributed Networks[END_REF] The distribution Q is implementable if and only if it satisfies the following condition2

I Q (S 1 ;U) ≤ I Q (V ;Y 2 |U) -I Q (V ; S 1 |U). (20)
where U and V are auxiliary random variables and Q is any joint distribution that factorizes as

Q(x 0 , s 1 , s 2 , u, v, x 1 , x 2 , y 1 , y 2 ) = ρ(x 0 ) (s 1 , s 2 |x 0 )P UV X 1 |S 1 (u, v, x 1 |s 1 )P X 2 |US 2 (x 2 |u, s 2 )Γ (y 1 , y 2 |x 0 , x 1 , x 2 ) (21)
In practice, to plot the utility region, one typically has to solve a convex optimization problem. To be illustrative, we consider the special case of [START_REF] Larrousse | Coded Power Control: Performance Analysis[END_REF] namely, Y = X 1 .

Denoting by H the entropy function, the problem of finding the Pareto-frontier of the utility region exactly corresponds to solving the following optimization problem: minimize

-∑ x 0 ,x 1 ,x 2 Q(x 0 , x 1 , x 2 )w(x 0 , x 1 , x 2 ) subject to H Q (X 0 ) + H Q (X 2 ) -H Q (X 0 , X 1 , X 2 ) ≤ 0 -Q(x 0 , x 1 , x 2 ) ≤ 0 -1 + ∑ x 0 ,x 1 ,x 2 Q(x 0 , x 1 , x 2 ) = 0 -ρ(x 0 ) + ∑ x 1 ,x 2 Q(x 0 , x 1 , x 2 ) = 0 .
The above problem can be shown to be convex (see [START_REF] Larrousse | Coded Power Control: Performance Analysis[END_REF]). In the next section we exploit this result to assess the performance gain brought by implementing coordination for distributed power control in wireless networks.

Application to distributed power control

Here we apply the results of the previous section to the wireless power control problem.

A flat-fading interference channel (IC) with two transmitter-receiver pairs is considered. Transmissions are assumed to be time-slotted and synchronized; the timeslot or stage index is denoted by t ∈ N * . For k ∈ {1, 2} and " = -k" (-k stands for the terminal other than k), the signal-to-noise plus interference ratio (SINR) at Receiver k on a given stage writes as SINR k = g kk x k σ 2 +g k x -k where x k ∈ X IC i = {0, P max } is the power level chosen by Transmitter k, g k represents the channel gain of link k , and σ 2 the noise variance. If Transmitter 1 is fully informed of x 0 = (g 11 , g 12 , g 21 , g 22 ) for the next stage and Transmitter 2 has no transmit CSI while both transmitters want to maximize the average of a common stage payoff which is

w IC (x 0 , x 1 , x 2 ) = ∑ 2 k=1 f (SINR k (x 0 , x 1 , x 2 )
), there may be an incentive for Transmitter 1 to inform Transmitter 2 what to do for the next stage; a typical choice for f is f (a) = log(1 + a). Since Transmitter 1 knows the optimal pair of power levels to be chosen on the next stage, say (x * 1 , x * 2 ) ∈ arg max

(x 1 ,x 2 )
w(x 0 , x 1 , x 2 ), a simple coded power control (CPC) policy for Transmitter 1 consists in transmitting on stage t at the level Transmitter 2 should transmit on stage t + 1. Therefore, if Transmitter 2 is able to observe the actions of Transmitter 1, power levels will be optimally tuned half of the time. Such a simple policy, which will be referred to as semicoordinated PC (SPC), may outperform (in terms of average payoff) pragmatical PC policies such as the one for which the maximum power level is always chosen by both Transmitters ((x 1 , x 2 ) = (P max , P max ) is the Nash equilibrium of the static game whose individual utilities are u k = f (SINR k )).

The channel gain of the link between Transmitter k and Receiver is assumed to be Bernouilli distributed: g k ∈ {g min , g max } is i.i.d. and Bernouilli distributed g k ∼ B(p k ) with P(g k = g min ) = p k . The utility function is either f (a) = log(1 + a) or f (a) = a. We define SNR[dB] = 10 log 10 P max σ 2 and set g min = 0.1, g max = 1.9, σ 2 = 1. The low and high interference regimes (LIR for low interference regime, HIR and for high interference regime) are respectively defined by (p 11 , p 12 , p 21 , p 22 ) = (0.5, 0.9, 0.9, 0.5) and (p 11 , p 12 , p 21 , p 22 ) = (0.5, 0.1, 0.1, 0.5). At last, Y ≡ X 1 and we define two reference PC policies : full power control (FPC) policy x k = P max for every stage ; the semi-coordinated PC (SPC) policy x 2 = P max , x † 1 ∈ arg max x 1 w IC (x 0 , x 1 , P max ). Fig. 3 and 4 depict the relative gain in % in terms of average payoff versus SNR[dB] which is obtained by costless optimal coordination and information-constrained coordination. Compared to FPC, gains are very significant whatever the interference regime and provided the SNR has realistic values. Compared to SPC, the gain is of course less impressive since SPC is precisely a coordinated PC scheme but, in the HIR and when the communication cost is negligible, gains as high as 25% can be obtained with f (a) = log(1 + a) and 45% with f (a) = a.

Conclusion

In this chapter, we have described an information-theoretic framework to characterize the limiting performance of a multiple agent problem. More precisely, the theoretical performance analysis has been conducted in terms of long-term utility region. We have seen that the problem amounts to finding the set of implementable joint distribution over the system state and actions. Both in the scenarios of causal and non-causal state information, auxiliary random variables appear in the characterization of implementable joint distribution. To be able to assess numerically the limiting performance for given utility functions, an optimization problem has to be solved. In the causal state information scenario, the problem is multilinear and the challenge is due to the dimension of the vectors involved. In the non-causal state information scenario, the problem to be solved is a convex problem; more precisely, the information constraint function which translates the agent capabilities in terms of coordination is a convex function of the joint distribution. Note that although the state is not controlled and evolves randomly, the general problem of characterizing the utility region for any number of agents is not trivial. Of course, the problem Fig. 4 The difference with Fig. 3 is that the reference power control policy is the semi-coordinated power control policy (SPC), which is already a CPC policy. Additionally, the top curve is obtained with f (a) = a. is even more difficult in the case of controlled states, which therefore constitutes one possible non-trivial extension of the results reported in this chapter. Another interesting research direction would be to consider the case where the state and actions are continuous. A first attempt to this has be made in [START_REF] Agrawal | Implicit coordination in two-agent team problems[END_REF]. Interestingly, the corresponding problem can be shown to be strongly connected to the famous Witsenhausen problem [START_REF] Witsenhausen | A counterexample in stochastic optimum control[END_REF], [START_REF] Sahai | Demystifying the Witsenhausen counterexample[END_REF], which is a typical decentralized control problem where control and communication intervene in an intricate manner.

Fig. 3

 3 Fig.3Relative gain in terms of expected payoff ("CPC/FPC -1" in [%]) vs SNR[dB] obtained with CPC (with and without communication cost) when the reference power control policy is to transmit at full power (FPC).

  expected payoff & HIR Costless communication case & LIR Information-constrained expected payoff &LIR Costless communication & HIR & f(a)=a

The notation I Q (A; B) indicates that the mutual information should be computed with respect to the probability distribution Q.


J Grampeix

F Buckley

J.-P Martin

M Colonna

G Gély

G Pananakakis

B Ghibaudo

De Salvo

M Bocquet
email: marc.bocquet@cea.fr

G Molas

H Grampeix

J Buckley

F Martin

J P Colonna

M Gély

G Ghibaudo

B De Salvo

S Deleonibus

Intrinsic fixed charge and trapping properties of HfAlO interpoly dielectric layers

  L'archive ouverte pluridisciplinaire

Introduction

In order to meet the performance requirements of future generations of Flash memory [1], one of the nearest major improvements will concern the scaling of the InterPoly Dielectric (IPD) stack. For the 45nm and 35nm nodes, in order to compensate the loss of the vertical sidewalls of the poly-Si floating gate and keep high the coupling ratio [START_REF] Alessandri | Proc. Of the 208th ECS Meeting[END_REF][START_REF] Houdt | [END_REF], the IPD thickness should be reduced. In a previous work [START_REF] Molas | Proc. of ESSDERC[END_REF], we proposed HfAlO high-k materials to replace the nitride layer in ONO interpoly dielectric stacks for future Flash memories, arguing the advantages both in terms of coupling and insulating properties. We showed that the leakage current was strongly governed by the trapping in the high-k layer, with a strong temperature activation. Indeed, a Poole-Frenkel conduction, probably assisted by the traps in the HfAlO layer, was identified.

In this paper, we further developed the analysis of HfAlO high-k materials, embedded in OHO stacks, by focusing on the trapping properties and fixed charges. In particular, we investigate: (1) the intrinsic negative fixed charge in the oxides, (2) the trapping phenomena which take place in the high-k dielectrics with various compositions and thicknesses during a gate stress, (3) the retention properties of these layers, and (4) finally, we present simulations based on a Shockley Read Hall (SRH) approach which allow us to model the electron trapping in the OHO stacks.

Experimental results

Sample description

The schematic of the triple layer capacitors studied in this work is shown in Figure 1. The high-k films were sandwiched between two HTO (High Thermal Oxide) deposited at 730°C, with a thickness of 4nm. Three different high-k materials, deposited by ALCVD, were studied: Hafnium Oxide (HfO 2 ), Aluminum Oxide (Al 2 O 3 ) and Hafnium Aluminate (HfAlO). In HfAlO films, the Hf concentrations were controlled by the HfCl 4 :Al(CH 3 ) 3 deposition cycle ratio, which are respectively: 9:1 (94% of Hf), 1:4 (31% of Hf), and 1:9 (27% of Hf). Different high-k physical thicknesses ranging between 3nm and 9nm were fabricated by controlling the number of ALD deposition cycles.

Intrinsic negative fixed charge

In this section we investigate the intrinsic fixed charge of the HfAlO layers. We can observe that the flatband voltages of OHO samples are shifted compared to the 10nm-thick HTO reference sample, due to the presence of intrinsic fixed charge in the high-k layers. The shift increases monotonically when the Al concentration of the HfAlO alloy increases.

The origin of the intrinsic negative fixed charge in HfAlO materials is nowadays still not clear [START_REF] Lee | [END_REF]. In amorphous Al 2 O 3 layer, it was suggested that the Al 2 O 3 could be dissociated into (AlO 4/2 ) -and Al 3+ [6] and that, at the SiO 2 /Al 2 O 3 interface, the charge compensation does not take place.

The inset of Figure 2 shows that V FB is a linear function of the HfAlO thickness, which suggests a surface rather than a volume distribution of the fixed charge according to Equation 1, being in agreement with previous works reported in the literature [7][8][9].

(

)

H H H H H SiO T fb t t t t V ε ρ ε σ ρ σ ε ⋅ ⋅ + ⋅ + ⋅ + ⋅ = ∆ 2 2 2
(1) with :

: charge density at high-k/Bottom HTO interface. : volume charge density in high-k. t H : thickness of high-k layer. t T : thickness of HTO top layer. 2) localised at the bottom HTO /High-k interface.

ε H : high-k dielectric constant. ε SiO2 : SiO 2 dielectric constant.

Trapping properties

In this section, we investigate in more details the charge trapping phenomena of OHO samples during a gate stress.

The gate current density as a function of the electric field is reported in Figure 3. On the same graph is also reported the time evolution of the gate current at constant gate bias (J G -Time) on virgin devices. The hysteretic behaviour, and the continuous decreasing of the leakage current with the elapsing time demonstrate that trapping phenomena take place in the high-k materials. Note that the charge trapping could be an issue for IPD applications, as it may degrade the reliability of the memory, generating threshold voltage instabilities. To evaluate more precisely the trapping capabilities of the interpoly stacks, we monitored the evolution of the flatband voltages as a function of time when the devices were submitted to different gate stresses (Figure 4). A continuous V FB shift is observed, showing the progressive electron trapping in the stack as the stress time increases. It clearly appears that for a given stress condition, the trapping capability increases with the Hf concentration. As already reported in the literature [START_REF] Molas | Proc. of ESSDERC[END_REF]10], this result could be correlated with the crystalline structure of the high-k materials: the larger the Hf concentration, the more crystalline the layer, and hence, the higher the trapping capability.

Based on these measurements, we extracted the charges trapped in the gate stack after programming. We assume that the charges are localized at the interface between the Bottom HTO and the HfAlO layer. Indeed, the further the traps are from the cathode, the slower the charging kinetics. Making this assumption, it appears that the extracted trapped charge value does not depend on the HfAlO thickness (Figure 5). This confirms (see Equation 1) that charges are mainly trapped at the high-k interface and that the bulk contribution is negligible at the first order. The stressing conditions are performed at constant V G /EOT. We assume that the charges are localized at the interface between the Bottom HTO and the HfAlO layer.

Retention characteristics

In this section, we investigate the dynamic discharging of the charges previously trapped in the OHO layers by a writing stress. Figure 6 plots the room temperature retention characteristics of OHO samples with different compositions of the HfAlO layer. We observe that for the Hf-rich sample, the electron discharging rate is quite fast, whereas the HfAlO 1:4 and 1:9 keep most of the charge after 10 6 s.

Figure 6 also shows the strong temperature activation. Indeed, the charge loss is strongly accelerated at 85°C (within a factor 2 for the 9:1 sample). Nevertheless, the trend observed at room temperature is still conserved, i.e.: the 1:4 and 1:9 HfAlO layers exhibit the same charge decay, while for the 9:1 HfAlO sample, only 50% of trapped charge remains after 10 6 s. These observations are consistent with experiments performed on SONOS-like structures with high-k trapping layers [11].

Modelling

In this part, we introduce an analytical model to qualitatively explain the trapping characteristics of our IPD stacks. To this aim, we use the SRH model presented in [10], and we focus on the Hf-rich (9:1) OHO samples.

The simulations are performed assuming that:

The trapped charge is localized at the bottom HTO / HfAlO interface, which is consistent with our experimental data. Note that in a more realistic approach, we should take into account the charge trapped in the HfAlO bulk, characterised by slower trapping time constants.

The thermalization of electrons in the SiO 2 layer close to the cathode is neglected. In fact, in our range of programming voltages, the electron paths in the conduction band of the HTO, after tunneling, is inferior to 1nm (Figure 7), which is indeed shorter than the mean free path of electrons in SiO 2 [12]. In other words, we assume that the electron energy remains constant till the trapping in HTO/HfAlO interface states happens.

The gap (E G =5.65eV) and the permittivity (εr=17) of 9:1 HfAlO were extracted by ellipsometry and by C-V G measurements, respectively. We also consider that the bottom and top HTO are 4.5nm and 5nm thick respectively, which is in agreement with TEM observations.

The shift between the conduction band of Si and HfAlO, E C , and the trapping cross section, , were fixed based on literature data: E C =2eV [13,14], =10 -18 cm 2 [START_REF] Fernandes | Proc. of ESSDERC[END_REF]10]. N st , the trap density, is adjusted to fit the experimental saturation level. Fig. 8: Modeling of the trapping characteristics of OHO stack (HfAlO thickness is 6nm, concentration is 9:1), based on structure and parameters reported in Figure 7.

Based on these assumptions, we use the following equations to simulate the programming characteristics:

st N ft Vth Ct × ∆ = ( ) ( ) ( ) 1 dft ft cn ep ft en cp dt = -× + -× +
cn/en and cp/ep are the electron and hole capture/emission rates which govern carrier exchanges between the traps and substrate. ft: trap occupation probability according to Shockley-Read-Hall statistics model [START_REF] Ielmini | [END_REF].

Ct: trap to gate coupling capacitance.

Figure 8 shows the experimental and the modelling trapping characteristics. We observe a very good correlation between the simulation and the experimental data for the three programming voltages, which validates our theoretical approach.

Conclusions

In this paper we investigated the intrinsic fixed charge and trapping phenomena happening under stress of HfAlO based interpoly dielectric stacks. We demonstrated that the fixed charge content increases with the Al concentration of the HfAlO layer. We showed that the trapping capability when the device is submitted to a constant voltage stress increases as the Hf ratio of the compound increases. Based on programming measurements, we proved that in our devices the electron trapping mainly occurs at the first interface, between the bottom HTO and the HfAlO layer, rather than in the volume of the high-k dielectrics. We also observed that the discharging rate of the previously trapped charges is more important for Hf-rich alloys. Finally, an analytical model based on a SRH approach allowed us to fit our experimental data and to extract the main trapping parameters of HfAlO high-k materials.

Fig. 1 :

 1 Fig. 1: Schematic showing the capacitor device stack studied in this work. Various high-k were investigated: HfO 2 , Al 2 O 3 and HfAlO with different Hf:Al ratios (9:1, 1:4 and 1:9).

Figure 2 Fig. 2 :

 22 Figure 2 plots the capacitance-voltage characteristics of the studied OHO triple layers in the virgin state.

Fig. 3 :

 3 Fig.3: Current density J G versus equivalent electric field E G of HTO /HfAlO 9:1 -9nm/HTO stack. J G -time measurements on virgin devices are also represented for different electric fields. Inset: J G -time measurements performed at different applied electric field (1MV/cm, 6MV/cm and 10MV/cm) as a function of time.

Fig. 4 :Fig. 5 :

 45 Fig. 4: Programming characteristics of OHO samples with various HfAlO compositions. The HfAlO thickness is 6nm.

Fig. 6 :

 6 Fig. 6: Room temperature and 85°C retention characteristics of OHO samples with various compositions of HfAlO. The HfAlO thickness is 6nm. The programming conditions are fixed to have an initial flatband voltage shift of 1.5V for each sample.

2 Fig. 7 :

 27 Fig. 7: Energy band diagram of OHO stack at V G =10V simulated in this work. Fitting parameters are indicated. The HfAlO thickness is 6nm, the concentration is 9:1. The charges trapped in the HTO-HfAlO interface are responsible for the different electric field values in the bottom and top oxide layers.

Table 1

 1 summarizes the number of equivalent fixed charge localised at the bottom HTO/High-k interface, calculated from the V FB shifts.

	HfAlO	Number of fixed charge
	composition	nb/cm 2
	9:1	3•10 12
	1:4	3.5•10 12
	1:9	4•10 12

Table 1 :

 1

Number of equivalent fixed charge (extracted from the characteristics reported in Figure

Acknowledgments

Part of this work was supported by the MEDEA+ NEMESYS project.