Karima Djabella

Mayer Landau

Michel Sorine

BIFURCATION ANALYSIS OF THE CONTROL MECHANISM OF PACEMAKER ACTIVITY IN A MATHEMATICAL CARDIAC CELL MODEL

Keywords: cardiac pacemaker cell, nonlinear dynamics, oscillation control, bifurcation analysis, computer simulation

teaching and research institutions in France or abroad, or from public or private research centers.

Bifurcation analysis of the control mechanism of pacemaker activity in a mathematical cardiac cell model

INTRODUCTION

The pacemaker activity of the sinoatrial (SA) node cells is an important subject in cardiac electrophysiology as the spontaneous activity controls the heart rate and is itself under the control of the autonomic nervous system (ANS) through different ionic channels.

The precise ionic currents that control membrane potential (V) through gate mechanisms (g) during diastole and then the heart rate, are still a matter of debate [START_REF] Kodama | Are we lost in the labyrinth of the sinoatrial node pacemaker mechanism?[END_REF]. [START_REF] Difrancesco | Pacemaker mechanisms in cardiac tissue[END_REF] argues that the hyperpolarization activated current (i f ) is the only current that can generate and control the slow depolarization of pacemaker cells. This current is normally carried by Na + and K + . [START_REF] Guo | A sustained inward current activated at the diastolic potential range in rabbit sinoatrial node cells[END_REF] reported another current, called the sustained inward current i st , where the major charge carrier is believed to be Na + . Also a Ca 2+ "window" current has been observed in rabbit sinoatrial node cells [START_REF] Boyett | Control of the pacemaker activity of the sinoatrial node by intracellular ca 2+ . experiments and modelling[END_REF]. It is possible that any of these currents, or a combination of them, is responsible for membrane depolarization during diastole. In our simulations using a new reduced order model of the SA node cell [START_REF] Djabella | A differential model of controlled cardiac pacemaker cell[END_REF], we concluded that all the currents directly or indirectly contribute to pacemaker depolarization, with the background calcium current (I b,Ca ) being the most dominant inward current during diastole. Thus, the roles of all individual currents (like pump and exchanger) in pacemaker activity must be considered.

The aim of this study is to precise the role of each current in the normal pacemaking of SA node pacemaker cells. To do that the nonlinear dynamics of the mathematical model of the pacemaker cell of [START_REF] Djabella | A differential model of controlled cardiac pacemaker cell[END_REF] is studied by a bifurcation analysis, as well as by numerical simulations. We construct bifur-cation diagrams by calculating stationary points, periodic orbits, stability of the equilibrium points, Hopf bifurcation points as functions of bifurcation parameters (maximal conductances) to characterize the dynamics properties of each individual ionic components in the SA node pacemaker cell model.

A salient point of this numerical study is the illustration of the usefulness of bifurcation analysis to get insight into the mechanisms of the SA node cell pacemaker activity. The Ca 2+ , K + , Na + /K + and Na + /Ca + currents appear to be essential for pacemaker activity because blocking one of these currents abolishes spontaneous activity. Furthermore, in the vicinity of the control conditions, the cycle length (CL) is specially more sensitive to the background calcium current than to the others. In contrast, in our model, the Na + current would be inessential to pacemaker activity, as it only modulates action potential waveforms and does not significantly affect pacemaker frequency: reducing conductance of this current does not cause bifurcation to quiescence and its effect on the stability, dynamics and bifurcation structure of the cell model much smaller than the others. Remark that the same kind of analysis has been conducted for more complex models of SA node pacemaker cells as in [START_REF] Kurata | Roles of L-type Ca2+ and delayedrectifier K+ currents in sinoatrial node pacemaking: insights from stability and bifurcation analyses of a mathematical model[END_REF], based on the model of [START_REF] Kurata | Dynamical description of sinoatrial node pacemaking: improved mathematical model for primary pacemaker cell[END_REF] which has 29 state variables and 18 ionic currents. The simplicity of the present model with its 8 state variables and 6 ionic currents makes possible to push further the analysis.

The paper is organized as follows: the model and the theoretical and numerical methods for bifurcation analysis are described in Section 2. Section 3 shows some simulation results. A discussion and conclusions are presented in Sections 4 and 5 respectively. The model equations are recalled in an appendix.

THEORY AND METHODS

Mathematical Model

We use our model of SA pacemaker cell which can reproduce the dynamic properties and bifurcation structures of real SA node cells (see fig. 2 andfig. 3) despite its reduced order. The model equations and notations are recalled in the APPENDIX.

The complete model for the normal pacemaking cell includes five membrane currents, among which I K,t , I Na,t , I Ca,t , represent the sums of all the currents through the K + , Na + , Ca 2+ channels respectively. The membrane currents system also includes the exchanger and the pump currents denoted as I NaCa and I NaK respectively [START_REF] Endresen | A theory for the membrane potential of living cells[END_REF], charging the membrane capacitance C m . The background current I b,Ca is included in the expression of the I Ca,t current.

As for ventricular cells, the model takes into account the main processes that regulate intracellular Calcium concentration: release and uptake by the sarcoplasmic reticulum (SR), buffering in the SR by calsequestrin [START_REF] Ten Tusscher | A model for human ventricular tissue[END_REF] and in the bulk cytosol [START_REF] Shannon | A mathematical treatment of integrated ca dynamics within the ventricular myocyte[END_REF] where several buffers of Calcium are in competition with the myofilament protein troponin-C. In the case of ventricular cells, this competition regulates the contractile activity. To reduce the number of variables, only two buffers have been selected (calmodulin and troponin).

Stability and Bifurcation Analysis

A bifurcation is a change of the number or type of attractors in a nonlinear dynamical system when some system parameters are changed. It is accompanied by a change of the stability of an attractor. At a bifurcation point, at least one eigenvalue of the Jacobian gets a zero real part. The bifurcation phenomena observed in cardiac myocytes include cessation or generation of pacemaker activity and occurrence of abnormal (irregular) dynamics [START_REF] Guevara | Three ways of abolishing automaticity in sinoatrial node: ionic modeling and nonlinear dynamics[END_REF]). We will be mainly concerned by Hopf bifurcations occuring when the Jacobian matrix has a complex conjugate pair of eigenvalues. That change of the sign of the real parts, making a limit cycle to appear or disappear. Structural stability of a dynamical system is an equivalence property of systems with sufficiently close parameters (the topological description of the dynamics do not change under small perturbations) and fails when bifurcations occur (see e.g. [START_REF] Kuznetsov | Elements of applied bifurcation theory[END_REF][START_REF] Wiggins | Introduction to applied nonlinear dynamical systems and chaos[END_REF] for bifurcation theory and numerical methods for bifurcation analysis). This concept may help to understand the generic behaviour of dynamical systems in state spaces of small dimensions (this is the case for continuous time dynamical systems on compact 2-dimensional manifolds that can be precisely characterised, but already false in the 3-dimensional case). A practical way to study this stability is to analyse bifurcations, that is in general a difficult task for bifurcations depending on more than 3 parameters (the number of parameter necessary to tune, to get the bifurcation is its co-dimension).

We examine here how the stability and dynamics of our 8-state-variables cell model alter with changes in bifurcations of co-dimension 1 and construct associated bifurcation diagrams for one parameter while keeping all the other parameters at their standard values. The bifurcation parameters chosen in this study are the maximum conductances of six membrane currents.

We calculate the equilibrium points (EP) and periodic orbits in the state space and also determine the asymptotic stability of an EP by computing 8 eigenvalues of the 8 × 8 Jacobian matrix derived from the linearization of the nonlinear system around the EP. Then, EPs (steady state branches) and local extrema of periodic orbits (periodic branches) are plotted against the bifurcation parameter. Hopf bifurcation (HB) points where the stability analysis of an EP reverses were detected by the stability analysis as described above.

Numerical Integration

We use the software XPP (also called XPPAUT) running under X11 and Windows which is a tool for analyzing dynamical systems [START_REF] Ermentrout | Simulating, analyzing, and animating dynamical system[END_REF].

Dynamic behavior of the cell model is determined by solving the simultaneous nonlinear ordinary differential equations (ODE) numerically. We use CVODE algorithm which is a solver for stiff and non-stiff ODE systems, that includes variable-order methods and variable-step methods. The maximum relative error tolerance for our integration method is set to 10 -6 . The action potential (AP) amplitude, the voltage difference between the maximum diastolic (MDP) and peak overshoot potential (POP), as well as the CL, are determined for each cycle. The continuationbifurcation algorithms of XPP are used for the fixed points and periodic solutions of the ODE.

RESULTS

Figure 2 shows the spontaneous action potentials together with the underlying five membrane currents in the cell model; I Na,t , I Ca,t , I K,t , I NaK and I NaCa . In Figure 3, the modelled action potential is shown together with an experimental curve of [START_REF] Baruscotti | A ttx-sensitive inward sodium current contributes to spontaneous activity in newborn rabbit sino-atrial node cells[END_REF]. The curve are identical in shape, but the modelled curve is adjusted somewhat (we multiplied the voltage amplitude by a factor 1.26, without changing the minimum value) to obtain the same voltage amplitudes.

To identify the key currents responsible for generation and control of pacemaker activity, we explore the effects of changing the conductances of I Na,t ( ĪNa ), I Ca,t ( ĪCa ), I b,Ca ( Īb,Ca ), I K,t ( ĪK ), I NaK ( ĪNaK ) and I NaCa ( ĪNaCa ) currents on EP stability and oscillation dynamics of the cell model.

Figure 1 shows the bifurcation diagrams constructed for ĪK , ĪCa , ĪNaK , ĪNaCa and Īb,Ca , depicting the steady state potential (V 0 ) at an EP (steady state branch) and local potential extrema of MDP and POP (periodic branches), as well as CL, as functions of bifurcation parameters. The steady state potential is negatively or positively shifted with decreasing ĪK , ĪCa and ĪNaK while it is approximately constant with decreasing Īb,Ca or ĪNaCa (V 0 ≃ -25 mV ). Decreasing ĪK or ĪNaCa leads to stabilization of the EP: HB points are at ĪK = 14.08 pA and ĪNaCa = 0.70 pA respectively, with a cessation of limit cycle oscillations. When changing ĪCa , the amplitude of a limit cycle gradually decreases and finally the oscillation disappears at the HB points. The HB points are located at ĪCa = 42.3 pA and at ĪCa = 2.56 pA. From the second one, an unstable branch emerging goes in the same direction as the steady state solution which loses stability (to the right). We note The same behaviour as for ĪCa is found when changing ĪNaK : a stable limit cycle gradually decreases and finally disappears at the Hopf bifurcation points located at ĪNaK = 7.8 pA and at ĪNaK = 64.45 pA. We have find also two limit points at ĪNaK = 0.73 pA and at ĪNaK = 0.86 pA, the curve of EP along the middle branch is unstable. When increasing Īb,Ca , a stable limit cycle gradually decreases to desappear at the HB point Īb,Ca = 0.1899 pA.

The cycle length variations are also shown as functions of the bifurcation parameters (see the right panel of the fig. 1). In the vicinity of the control conditions, the CL is especially sensitive to the change of Īb,Ca but relatively insensitive to the others. This background calcium current has a very small conductance value, Figure 4 shows the bifurcation diagrams constructed for ĪNa , depicting the steady state potential (V 0 ) at an EP (steady state branch) and local potential extrema of MDP and POP (periodic branches), as well as CL, as functions of the bifurcation parameter. Changing ĪNa does not stabilize the EP or abolish spontaneous activity and the CL is insensitive to its variation.

DISCUSSION

Following the approach in [START_REF] Kurata | Roles of L-type Ca2+ and delayedrectifier K+ currents in sinoatrial node pacemaking: insights from stability and bifurcation analyses of a mathematical model[END_REF], we have investigated some dynamical mechanisms of the SA node pacemaker activity by a stability and bifurcation analysis of our reduced order model of pacemaker cell. The pacemaker activity corresponds here to a stable limit cycle around an unstable EP, so that the most important roles of the sarcolemmal currents in pacemaking, are their abilities to drive to the instability of an EP or contribute to maintain it, avoiding the quiescent state of the system at stable EPs [START_REF] Guevara | Three ways of abolishing automaticity in sinoatrial node: ionic modeling and nonlinear dynamics[END_REF] as in the case of the stable resting potential of atrial or ventricular myocytes. Our results are compatible with the qualitative behaviour of the more complex model of [START_REF] Kurata | Roles of L-type Ca2+ and delayedrectifier K+ currents in sinoatrial node pacemaking: insights from stability and bifurcation analyses of a mathematical model[END_REF]. Initiation or arrest of normal pacemaking appear as Hopf bifurcation phenomena controlled by dedicated currents.

We have considered the roles of I Na,t , I Ca,t , I b,Ca , I K,t , I NaK and I NaCa currents in SA node pacemaker cell by exploring how these currents affect the bifurcation structure of the cell model. Each of I Ca,t , I b,Ca , I K,t , I NaCa and I NaK is able to induce to EP instability and pacemaker generation via a Hopf bifurcation. Furthermore, I b,Ca appears to be indispensable for pacemaker activity in that it is the main responsible for cycle length sensitivity (fig. 1). EP instability and oscillation dynamics are not affected by the I Na,t current (Fig. 4).

CONCLUSION

In this paper, stability and bifurcation analysis is used to investigate the mechanisms of pacemaking in the SA node cell represented by a reduced order mathematical model. This nonlinear dynamical approach is known to be useful for general understanding and systematic description of the mechanisms of normal or abnormal pacemaker activity [START_REF] Guevara | Three ways of abolishing automaticity in sinoatrial node: ionic modeling and nonlinear dynamics[END_REF]. This study demonstrate that bifurcation theory also provides a convenient way of determining the roles of individual currents in pacemaker activity of SA node cells and to compare qualitative behaviours of models of various complexity (here 8 state variables against 29 in [START_REF] Kurata | Dynamical description of sinoatrial node pacemaking: improved mathematical model for primary pacemaker cell[END_REF]). In our reduced order model of SA node cell, pacemaker activity depends on the conductivity ranges of I Ca,t , I b,Ca , I K,t , I NaCa and I NaK which are responsible for equilibrium points instability via a Hopf bifurcation. Furthermore, I b,Ca allows fine tuning of the SA node rate.

This work also suggests that for some applications where reduced order models of SA node cell are required (e.g. long-term or multi-cell simulations, analysis or identification...), it is possible to check that the models used have the required qualitative behaviours (i.e. bifurcation structures) of larger SA node cell models validated against sub-cellular experiments.

APPENDIX

The model of SA pacemaker cell is the system (1) of eight ODEs (notations are defined in table 1):

                                           dV dt = - I K,t +I Na,t +I Ca,t +I NaK +I NaCa C m dK i dt = 2I NaK -I K,t FV C dNa i dt = - I Na,t + 3I NaK + 3I NaCa FV C dCa i dt = 2I NaCa -I Ca,t 2FV C +J leak +J rel -J up -∑ b∈I B B b dθ b dt , I B = {T n,Cal} , dg X dt = g X∞ -g X τ g X , X ∈ {Na, K} dθ b dt = k on b |Ca i | + (1 -θ b ) -k o f f b θ b , b ∈ I B (1)
The gate dynamics are defined by

g Na∞ = 1 2 1-tanh V -V g Na RT /2F , g X∞ = 1 2 1+tanh V -V g X RT /2F , X ∈ {K, d, m} 1 τ g X = 1 τ X cosh V -V g X RT /2F , X ∈ {Na, K} (2)
where X = d, m represent fast Ca, Na activation gating, denoted as usual d ∞ = g d∞ , m ∞ = g m∞ . Setting

V X = RT z X F log X e X i , X ∈ {Ca, Na, K} , (3)
the currents through the membrane are then:

I K,t = ĪK g K sinh V -V K 2RT /F (4) I Na,t = ĪNa g Na m ∞ sinh V -V Na 2RT /F (5) I Ca,t = [ ĪCa (1-g K )d ∞ + Īb,Ca ] sinh V -V Ca RT /F (6)
I NaK = ĪNaK tanh V + 2V K -3V Na -V AT P 2RT /F (7)

I NaCa = ĪNaCa sinh V + 2V Ca -3V Na 2RT /F (8)
Finally, the calcium-induced calcium release (CICR) mechanism is described using

J rel = K rel d ∞ (Ca SR -Ca iJCT ) (9)
J leak = K leak (Ca SR -Ca iJCT ) (10

Fig. 3 .Fig. 1 .

 31 Fig. 3. Experimentally recorded and scaled (by a factor of 1.26) model-generated rabbit sinoatrial action potential waveform.

Fig. 4 .

 4 Fig. 4. Effects of changing current conductance ĪNa . Notations are similar to fig. 1.

Table 1 .

 1 Notations used in the text Pump current taking up calcium in the SR J leak Leakage current from SR to the cytoplasm X i Intracellular concentration of the free ion X X e External concentration of the ion free X Ca T Total calcium concentration in the cell Ca SR Free calcium concentration in the SR θ T n Fraction of Troponin-C sites bound with Ca θ Cal Fraction of Calmodulin sites bound with Ca Ca iJCT SR Calcium buffered in the junction

	Notations	Definitions
	V	Membrane potential
	g X	X activation gating
	d ∞	Fast Calcium activation gating
	m ∞	Fast Sodium activation gating
	J rel	CICR current
	J up

)

  Ca iJCT = B JCT |Ca i | + |Ca i | + + K JCT (11)Ca SR + B SR |Ca SR | + |Ca SR | + + K SR =

	V C V SR	b∈I B Ca T -Ca i -∑ B b θ b -Ca iJCT (12)
	Ca T = Ca e +	C m 2FV C	V +	1 2	(Na

e -Na i +K e -K i ) (13)


Emmanuel Chambon

Laurent Burlion

Pierre Apkarian

Overview of LTI interval observer design: towards a non-smooth optimisation-based approach

Some applications in control require the state vector of a system to be expressed in the appropriate coordinates so as to satisfy to some mathematical properties which constrain the studied system dynamics. This is the case with the theory of linear interval observers which are trivial to implement on cooperative systems, a rather limited class of control systems. The available literature shows how to enforce this limiting cooperativity condition for any considered system through a statecoordinate transformation. This article proposes an overview of the existing numerical techniques to determine such a transformation. It is shown that in spite of being practical, these techniques have some limitations. Consequently, a reformulation of the problem is proposed so as to apply non-smooth control design techniques. A solution is obtained in both the continuous-and discretetime frameworks. Interestingly, the new method allows to formulate additional control constraints. Simulations are performed on three examples.

1. G is Input-to-State Stable;

Introduction

To be applicable, many observation and control strategies applied to linear systems are required to comply with some constraints on the designed dynamics. These constraints are expressed through mathematical properties which should be satisfied by the resulting system state-space representation matrices. For example, considering the case of interval observers [START_REF] Gouzé | Interval observers for uncertain biological systems[END_REF], a constructive cooperativity condition [START_REF] Smith | Monotone dynamical systems: an introduction to the theory of competitive and cooperative systems[END_REF] on the observation error dynamics is looked for, which is linked to properties of the resulting state matrix. In the continuous-time case, this matrix should be Metzler which means that its off-diagonal elements should be positive. For a given system, this property is rarely satisfied in the original state coordinates. However, a change of coordinates can be calculated so that cooperativity is satisfied in the new coordinates [START_REF] Mazenc | Interval observers for linear time-invariant systems with disturbances[END_REF][START_REF] Raïssi | Interval state estimation for a class of nonlinear systems[END_REF]. Various techniques -which are detailed in this papercurrently exist to determine such a transformation. Their advantages lie in their simplicity. However, they rely on "first guess" and are hardly compatible with additional constraints related to control performance. In the case of interval observers, it may be expected that the error dynamics should satisfy to additional constraints like disturbances rejection. To the authors opinion, the fact that the observer gain and the state-coordinate transform cannot be simultaneously computed to satisfy to the Metzler property and to an additional disturbance rejection constraint is a limit to these methods.

An overview of the aforementioned existing techniques is presented in this article. Their limitations motivate a reformulation of the problem. This is presented in this article both in the continuousand discrete-time frameworks. This leads to an optimization problem which is solved using nonsmooth optimization techniques usually dedicated to control design. Using such techniques, additional control constraints can be taken into account in the determination of the appropriate state-coordinate transformation. This article extends the preliminary results presented in [START_REF] Chambon | Metzler matrix transform determination using a non-smooth optimization technique with an application to interval observers[END_REF]. This approach was motivated by the interval observers cooperativity requirement but it is expected to be generalizable to any problem requiring specific structures of the matrices of the studied system. It is important to understand that no theoretical addition is brought to the existing interval observer framework but rather on a computation-related sub-problem. Note that the use of time-varying changes of coordinates [START_REF] Mazenc | Interval observers for linear time-invariant systems with disturbances[END_REF] is not considered since our method is not applicable in this case.

The core of the technique is to reformulate the mathematical conditions on a matrix -like the Metzler requirement on the state-matrix -into a stabilization control problem. Using recent theoretical results [START_REF] Apkarian | Tuning controllers against multiple design requirements[END_REF] of the structured controller synthesis field, a control law synthesis is then performed on multiple models in view of the specified control requirements. The approach is applied to various examples of interval observer problems. Numerical and simulation results are compared to the ones obtained using the existing techniques.

This paper is organized as follows. The notations and definitions required to understand the motivations and the resulting computation-related problems are presented in Sect. 2. The theory of interval observers is recalled in Sect. 3. In the same section, an overview of the existing techniques to compute the necessary state-coordinate transformation is performed. This raises a difficult numerical problem which is described in Sect. [START_REF] Raïssi | Interval state estimation for a class of nonlinear systems[END_REF]. A solution based on multiple models control law synthesis is proposed in Sect. 5 for both the continuous-and discrete-time frameworks. The approach is then applied on various examples in Sect. 6 before concluding in Sect. 7.

Notations and definitions

The Laplace transform is denoted s and the Z-transform is denoted z. Let denote Z the set of integers, R the set of real numbers and R + the set of positive real numbers. Let Z + = Z ∩ R + . If not otherwise stated, the indices (i, j, k, l, m, n) are integers in Z + . The H 2 -norm of a transfer function T (s) (resp. T (z)) is denoted T (s) 2 (resp. T (z) 2 ) and is defined as

T (s) 2 = 1 2π ∞ -∞ tr [T (jω) T (jω)] dω resp. T (z) 2 = 1 2π
π -π tr T (e jω ) T (e jω ) dω, where "tr" is the trace operator and T (jω) the conjugate transpose of T (jω). The elements of a matrix A ∈ R n×m are referred to using the lowercase notation a ij where i ≤ n and j ≤ m. The ordering operators >, <, ≤ and ≥ are understood componentwise when applied to multi-dimensional signals. Known (or computed) time-varying bounds on a multi-dimensional signal x ∈ R n are denoted x(t) and x(t):

∀t, x(t) ≤ x(t) ≤ x(t) (1)
In this article, only linear time-invariant (LTI) systems are considered. In the continuous-time case, let an LTI system (G) be defined by

(G)    ẋ = Ax + B 1 u + B 2 d y = Cx x(0) = x 0 (2)
where x ∈ R n , u ∈ R, d ∈ R and matrices are of appropriate dimensions.

Definition 1 (Interval observer, continuous-time). Suppose quantities x 0 , x 0 and signals d(t), d(t) are known such that x 0 ≤ x 0 ≤ x 0 and d(t) ≤ d(t) ≤ d(t), ∀t. Then, the dynamical system

G        ż = A z + B 1 u + B 2 d d ∈ R 2n x x = C z with rank C = 2n (3)
associated with the initial condition z 0 = C C -1 x 0 x 0 , where the newly defined matrices are of appropriate dimensions, is a linear interval observer of the system (G) in Eq. (2) if, see [

(t) = d(t), ∀t ∈ R + then x(t) -x(t) → 0.
A similar definition is obtained in the discrete-time case. This definition means that any system can be an interval observer as long as the two conditions are satisfied. However, this is quite hard in general to find the required matrices A , B 1 , B 2 and C . To do so, some constructive assumptions are gradually made on the considered systems structures so as to easily satisfy both conditions. For a given matrix A ∈ R n×m , let define A + = max(A, 0) and A -= A + -A. To determine matrix B 2 and C , the following Lemma will be used.

Lemma 1. (see [START_REF] Efimov | Interval state observer for nonlinear time varying systems[END_REF]Lemma 1] and the associated proof

) Let x ∈ R n s.t. x(t) ≤ x(t) ≤ x(t) and A ∈ R m×n a constant matrix s.t. A = A + -A -. Then ∀t, A + x(t) -A -x(t) ≤ Ax(t) ≤ A + x(t) -A -x(t) (4)
The following definitions will be useful to define A and obtain interval observers on linear systems.

Definition 2 (Metzler matrix). Let A = (a ij ) ∈ R n×n . The matrix A is said to be Metzler if ∀i = j, a ij ≥ 0 (5) Definition 3 (Non-negative matrix). Let A d = (a ij ) ∈ R n×n . The matrix A d is said to be non-negative if ∀ (i, j) , a ij ≥ 0 (6)
The theory of cooperative systems is extensively used when designing interval observers. Such systems have interesting properties as far as trajectory ordering is considered. This is used to fulfil the second condition in Def. 1 straightforwardly. Definition 4 (Cooperative system, continuous-time). An autonomous continuous-time linear system (G) is said to be cooperative if its state matrix A is a Metzler matrix.

Remark 1. In the case of a continuous-time linear system with inputs (and D = 0), it is said to be cooperative if A is Metzler, B ∈ R n×l + and C ∈ R m×n + . A cooperative continuous-time system is thus a positive system. Of course, discrete-time systems can also be cooperative under slightly different conditions. Definition 5 (Cooperative system, discrete-time). An autonomous discrete-time linear system (G d ) is said to be cooperative if its state matrix A d is a non-negative matrix.

Motivations

In many control applications, it is required to enforce specific properties on a system state-space representation. This is the case when designing interval observers as defined in Def. 1.

Motivating problem

The linear interval observer framework and existing techniques to compute a state-coordinate transformation such that the state-matrix is Metzler or non-negative are now presented.

Interval observer literature

The theory of interval observers was first introduced in [START_REF] Gouzé | Interval observers for uncertain biological systems[END_REF] and [START_REF] Rapaport | Parallelotopic and practical observers for nonlinear uncertain systems[END_REF] as a deterministic state "framing" method in the case of cooperative systems [START_REF] Smith | Monotone dynamical systems: an introduction to the theory of competitive and cooperative systems[END_REF] disturbed by unknown disturbances with known bounds. When the original system is not cooperative, a state-coordinate transformation is considered. In [START_REF] Mazenc | Asymptotically stable interval observers for planar systems with complex poles[END_REF] the authors show (on a precise class of systems) exponential stability of a time-varying interval observer obtained using a time-varying state-coordinate change. This is applied in [START_REF] Mazenc | Interval observers for linear time-invariant systems with disturbances[END_REF] on a broader class. A similar transformation is also proposed in [START_REF] Combastel | A stable interval observer for LTI systems with no multiple poles[END_REF]. The approach is extended in [START_REF] Thabet | An effective method to interval observer design for time-varying systems[END_REF] to Linear Time Varying (LTV) systems. In other works, a more classical approach leads to time-invariant linear interval observers with weaker stability guarantees but broader system classes. In [START_REF] Raïssi | Interval state estimation for a class of nonlinear systems[END_REF], a time-invariant state coordinate transform is used to obtain a cooperative observation error dynamics on which an interval observer is designed. This is extended to a class of nonlinear and LPV1 /LTV systems in [START_REF] Efimov | Control of nonlinear and LPV systems: interval observerbased framework[END_REF] and [START_REF] Efimov | Interval state observer for nonlinear time varying systems[END_REF]. Another formulation is proposed in [START_REF] Dinh | Interval observer composed of observers for nonlinear systems[END_REF]. An extensive study of these techniques is available in [START_REF] Efimov | Design of interval observers for uncertain dynamical systems[END_REF] and [START_REF] Dinh | Interval observer and positive observer[END_REF].

The theory of interval observers applied to discrete-time systems [START_REF] Efimov | On interval observer design for timeinvariant discrete-time systems[END_REF] is very similar to the continuous-time case. Extensions to discrete-time LPV systems have been proposed in [START_REF] Efimov | Estimation and control of discrete-time LPV systems using interval observers[END_REF]. An extension to continuous-time systems with discrete measurements is proposed in [START_REF] Dinh | Interval observer and positive observer[END_REF].

Note there exists other approaches based on Internal Positive Representations of systems [START_REF] Cacace | A new approach to design interval observers for linear systems[END_REF][START_REF] Efimov | On interval observer design for timeinvariant discrete-time systems[END_REF] In this paper, the considered systems are linear time-invariant systems and the results in [START_REF] Raïssi | Interval state estimation for a class of nonlinear systems[END_REF] are used.

Interval observer design using a classical observer

Let consider an LTI continuous-time system with a state-space representation as in Eq. ( 2) where u ∈ R is the control signal, d ∈ R is an unknown state disturbance and x 0 ∈ R n is the initial condition. The following hypotheses are considered Assumption 1. Couples (x 0 , x 0 ) and d, d are known s.t.

x 0 ≤ x 0 ≤ x 0 and ∀t ≥ 0, d(t) ≤ d(t) ≤ d(t) (7)
Assumption 2. The pair (A, C) in Eq. (2) is detectable.

To fulfil the first condition in Def. 1, a classical observer of the system in Eq. ( 2) is used:

(G obs )    ˙ x = A x + B 1 u + L (y -C x) y = C x x(0) = x 0 (8)
where L ∈ R n×m is defined such that A -LC is Hurwitz. The underlying estimation error dynamics is deduced from both Eqs. ( 2) and ( 8):

(G e ) ė = (A -LC) e + B 2 d e 0 = x 0 -x 0 (9)
where e = xx. If A -LC is not Metzler, (G e ) is not cooperative and it is not possible to fufil the second condition in Def. 1 without a state coordinate change. Under Hyp. 1 one obtains bounds on the initial error e 0 : e 0 = x 0x 0 and e 0 = x 0x 0 .

Assumption 3. A state coordinate transformation P ∈ R n×n is known s.t. M = P (A -LC) P -1 is Metzler.
The error dynamics in Eq. ( 9) expressed in the new coordinates becomes

(G ez ) ėz = M e z + B 2 d e z (0) = P e 0 ( 10
)
where e z = P e and B 2 = P B 2 . According to Def. 1 and using Lemma 1 and Hyp. 3, an interval observer G on the system in Eq. ( 10) is defined by the following dynamic system

G ez        ėz = M e z + B + 2 d -B - 2 d ėz = M e z + B + 2 d -B -
2 d e z (0) = P + e 0 -P -e 0 e z (0) = P + e 0 -P -e 0 [START_REF] Combastel | A stable interval observer for LTI systems with no multiple poles[END_REF] Let T = P -1 . To obtain bounds in the original coordinates e = T e z , the following relations are used

e = T + e z -T -e z e = T + e z -T -e z (12)
which preserve state ordering under Lemma 1. Thus, for any e 0 ∈ [e 0 , e 0 ] the following framing is obtained:

∀t ≥ 0, e(t) ≤ e(t) ≤ e(t) (13)
which in turn leads to

∀t ≥ 0, x(t) = x(t) + e(t) ≤ x(t) ≤ x(t) + e(t) = x(t) (14)
Using an appropriate state coordinate transformation, the state of a generic disturbed linear timeinvariant continuous-time system can thus be bounded.

In the discrete-time case, the formalism is very close to the continuous-time case with a slight difference on the properties to be satisfied by the state matrix, since it should be Schur-stable and non-negative, as stated in Def. 5:

Assumption 4. A state coordinate transformation P ∈ R n×n is known s.t. M = P (A -LC) P -1 is non-negative.

An overview of existing techniques to find an appropriate state coordinate transform

In the framework presented in Sect. 3.1.2, Hyp. 3 and 4 are strong hypotheses. In the case of high-order systems, choosing a satisfying state coordinate transformation matrix P is not obvious:

Problem 1. (continuous-time case) Find P ∈ R n×n and L ∈ R n×m such that M = P (A -LC) P -1 is Hurwitz Metzler. Problem 2. (discrete-time case) Find P ∈ R n×n and L ∈ R n×m such that M = P (A -LC) P -1 is
Schur-stable and non-negative.

In this section, the existing methods to determine a solution to Pbs. 1 and 2 are presented along with their advantages and limitations.

Real-constrained pole placement (trivial solution)

This trivial solution is obtained by noticing that diagonal matrices satisfy to Def. 2. Supposing that the pair (A, C) is observable (or that the unobservable eigenvalues are real negative, which is stronger than simple detectability), the matrix L is chosen such that A -LC only has negative real eigenvalues. Then, choosing P as the matrix of the right-hand eigenvectors means M = P (A -LC) P -1 is diagonal hence is a Metzler matrix.

The advantage of this technique is its simplicity since it only requires a pole placement and an eigenstructure determination algorithm. Three main limitations can however be mentioned:

• placing real poles can result in very large observation gains L resulting in sensitivity problems;

• an observer with complex poles may achieve better results;

• due to floating-point computations precision errors, the computed matrix M is Metzler up to the machine precision.

Remark 2. In the discrete-time case, the poles have to be placed on real values between 0 and 1 to comply with the definitions of Schur-stability and non-negativity.

Resolution of a Sylvester equation

To find a solution to Pb. 1, another approach can be used

• choose M ∈ R n×n to be a Metzler matrix;

• choose Q ∈ R n×m , this will represent the matrix product P L;

• using algorithms for example proposed in [START_REF] Golub | A Hessenberg-Schur method for the problem AX+XB=C[END_REF] or [START_REF] Bartels | Solution of the matrix equation AX+XB=C[END_REF], solve the following Sylvester equation in P :

-M P + P A = P LC = QC (15)
If M and A have no common eigenvalues, this equation has a unique solution for any Q.

One thus obtains a Metzler matrix M = P (A -LC) P -1 where the observer gain is defined by L = P -1 Q. This method was proposed in [START_REF] Raïssi | Interval state estimation for a class of nonlinear systems[END_REF] along with a constructive lemma [4, Lemma 1, p. 261] to solve the equation using observability criteria. However this Lemma is difficult to apply in case complex eigenvalues are looked for since there is no trivial (i.e. diagonal or triangular) Metzler matrix M with complex eigenvalues. Numerical errors may also occur in case of badly-conditioned matrices. A solution to the Sylvester equation can be obtained for example using [START_REF] Bartels | Solution of the matrix equation AX+XB=C[END_REF].

Remark 3. In the discrete-time case, Remark 2 is still valid here.

Conclusions on the interests of an alternative approach

Both techniques proposed in the literature bring the advantage of simplicity and only use built-in functions of any numerical computation software. This will be sufficient in most considered cases.

However, in more complex cases -high-order systems, ill-conditioned system matrices, additional constraints like observer error dynamics decay rate optimization, etc. -it seems more difficult to obtain satisfying results due to numerical precision problems as well as to the method philosophy. Indeed, the chosen matrices M and Q could lead to non-optimal results in view of other control constraints. Moreover, it seems difficult to obtain an expression of a Metzler matrix from expected complex eigenvalues.

An alternative technique is hereby proposed which could complement the existing methods by taking these more complex cases into account.

Problems statement

In this section, the different problems which arise in the proposed approach are exposed. As far as the state coordinate transformation determination is concerned, a reformulation into a control law synthesis problem is proposed, as exposed in [START_REF] Chambon | Metzler matrix transform determination using a non-smooth optimization technique with an application to interval observers[END_REF]. A numerical approach to find a solution simultaneously satisfying to these problems is then proposed in Sect. 5. It uses existing controller synthesis techniques as permitted by the reformulation.

State coordinate transformation determination

In Sect. 3, the design of interval observers has motivated the need for state coordinate transformation determination methods. This article proposes a new approach based on solving a control design problem.

Initial problem

Let recall the initial state coordinate transformation problem as stated in Pb. 1. Problem 3. (Mathematical formulation) Given (A, C) detectable, determine simultaneously P ∈ R n×n and L ∈ R n×m such that M = P (A -LC) P -1 is • Metzler and Hurwitz (continuous-time case);

• non-negative and Schur-stable (discrete-time case).

In other words, finding a solution to Pb. 3 in the continuous-time case is equivalent to finding P and L under n(n -1) positivity requirements on the off-diagonal elements of M = (m ij ) ∈ R n×n . This directly comes from the definition 2 of a Metzler matrix:

m ij ≥ 0, ∀i = j ⇔ -m ij ≤ 0, ∀i = j
In terms of control theory, these last inequalities are reminiscent of the stabilisation condition on a "fictitious" mono-dimensional system having its real pole located at -m ij .

Reformulation into a control design problem

A reformulation of Pb. 3 into a control design problem is done in the following proposition.

(G c ) ẋ1 = (A -LC) x 1 ẋij = -m ij (P, L) x ij ∀i = j ( 16
)
is Hurwitz-stable for given P and L then P (A -LC) P -1 is Metzler and Hurwitz.

Proof. Straightforward considering the definition of a Metzler matrix, see Def. 2, and the definitions of the coefficients m ij (P, L). In particular, ensuring positiveness of m ij (P, L) is equivalent to stabilizing ẋij = -m ij (P, L) x ij .

The states x ij ∈ R are called "fictitious" states since they do not have any physical meaning. For an initial matrix A ∈ R n×n , n (n -1) "fictitious" states are considered. Remark 4. Prop. 1 is reminiscent of a structured control design problem which is NP-hard in general. It is usually solved using algorithms with local optimality certificates which are known to work in practice. There is however no guarantee that the locally optimal solution stabilizes Eq. (16).

Remark 5. In the discrete-time case, n 2 "fictitious" states are considered since the diagonal elements of matrix M should also be non-negative. However, due to the definition of Schur-stability, using Prop. 1 would lead to -m ij (P, L) ∈ [-1, 1] , ∀(i, j) which does not comply with the definition of nonnegativity. In that case, the system in Eq. ( 16) is replaced by the following system:

(G c,d ) x 1,t+1 = (A -LC) x 1,t x ij,t+1 = f (-m ij (P, L)) x ij,t ∀ (i, j) (17)
where f : R → R is a linear function which maps -m max ij , 0 onto [-1, 1] with m max ij > 0 a design parameter which will be useful later:

f (x) = 2 m max ij x + 1 ( 18
)
Using this function has proved to be effective in practice.

Adding control requirements: resulting optimization problem

As already mentioned in Sect. 3.3, choosing P and L simultaneously such that additional control constraints are fulfilled is difficult. For example, considering the observation error dynamics in Eq. ( 9), it may be beneficial in terms of estimation quality to minimize a normalized value of the H 2 -norm of the transfer from the disturbance d to the observation error e: min P,L max { W (s)T d→e (s, L) 2 , T 2 (s, P, L) 2 , T 3 (s, P, L) ∞ , . . .} subject to G c (P, L) (Eq. ( 16)) is stable.

where W (s) is an appropriate weighting and T 2 (s, P, L) and T 3 (s, P, L) are additional design transfer functions. Note that the cost function is nonsmooth. This type of optimization problem is typical of the kind of problem encountered when synthesizing structured controllers against multiple requirements over multiple models. The idea is thus to use such formalism to find at least a local minimizer to this problem.

Structured controller synthesis

The synthesis of full-order controllers through H ∞ methods has been widely studied and used in the past two decades. Solutions to the problem of H ∞ synthesis in the case of MIMO systems were provided for example in [START_REF] Doyle | State-space solutions to standard H 2 and H ∞ control problems[END_REF] and [START_REF] Scherer | The Riccati inequality and state-space H ∞ -optimal control[END_REF]. When considering controllers with a fixed order much lower than the original plant, the problem of finding a controller is a non-convex optimization problem.

A local solution to this problem was proposed in [START_REF] Apkarian | Nonsmooth H ∞ synthesis[END_REF] and implemented in consecutive works [START_REF] Gahinet | Structured H ∞ synthesis in MATLAB[END_REF].

Other implementations for computing fixed-order controllers include [START_REF] Burke | HIFOO -a MATLAB package for fixed-order controller design and H ∞ optimization[END_REF]. More recent works [START_REF] Gahinet | Frequency-domain tuning of fixed-structure control systems[END_REF][START_REF] Apkarian | Tuning controllers against multiple design requirements[END_REF][START_REF] Apkarian | Multi-model, multi-objective tuning of fixed-structure controllers[END_REF] consider the case of finding a controller satisfying to multiple requirements on multiple models.

Considering these techniques, it appears possible to solve the minimisation problem in Eq. ( 19).

Nonsmooth optimization-based approach

In this section, the structured controller synthesis approach is detailed to solve the optimization problem in Sect. 4.2. An illustration of the technique to determine an interval observer state coordinate transformation is also proposed, which is then applied to three examples in Sect. 6.

Structured controller synthesis algorithm

As mentioned in Sect. 4.2, it appears that the reformulation proposed in Prop. 1 joined to additional control requirements leads to a structured controller synthesis problem. As already mentioned, this problem has been solved in [START_REF] Apkarian | Tuning controllers against multiple design requirements[END_REF][START_REF] Apkarian | Nonsmooth H ∞ synthesis[END_REF]. In this article, the systune function -along with the TuningGoal structure -from the Matlab Robust Control Toolbox [28] is used to solve the problem. This function uses nonsmooth optimization algorithms to solve nonsmooth non-convex optimization problems. The method is provably convergent and computes certified locally optimal solutions from any remote starting point.

In the following sections, the models and requirements needed to solve the optimization problem in Sect. 4.2 using these tools are detailed.

Models definition

As was already mentioned, recent control synthesis techniques allow to set multiple requirements on multiple models, be they describing an existing physical system or mathematical constraints, as is the case with the "fictitious" states used in Prop. 1. In the continuous-time case, let define n(n -1) fictitious systems:

G ij M ẋij = -m ij (P, L) x ij , ∀i = j (20)
and, in the discrete-time case, let define the following n 2 fictitious systems:

A =         -1 1 0 0 -1 0 -1 -2 0 -1 0 1 -2 0 -3 -2 0 0 -1 0 -2 -3 0 1 -1 0 2 0 -4 0 -1 -1 0 1 0 -1         , B 1 =         -18 -13 -5 -4 -10 22         , C = 1 0 0 0 0 0 ( 23
)
The system is stable and it is initialized at x 0 = 20 10 6 20 30 40 . The Luenberger observer is initialized at x 0 = 0 and the initial state is supposed to lie in [-x 0 , x 0 ] where x 0 = 50 1 1 1 1 1 1 . Thus, the initial observation error e 0 lies in [e 0 , e 0 ] = [-x 0x 0 , x 0x 0 ].

The observation error dynamics is defined by

ė = (A -LC)e -Ld e 0 = x 0 -x 0 (24)
which is not cooperative. Under Hyp. 3, the following system is an interval observer for the observation error dynamics in Eq. ( 24), where P + L = max(P L, 0) and

P - L = P + L -P L.        e z = M e z + P + L d -P - L d e z
= M e z + P + L d -P - L d e z (0) = P + e 0 -P -e 0 e z (0) = P + e 0 -P -e 0 [START_REF] Burke | HIFOO -a MATLAB package for fixed-order controller design and H ∞ optimization[END_REF] Reverting back to the initial coordinates is done using Eq. ( 12). The three following methods are considered to determine P and L:

• real pole placement allowing state matrix diagonalization;

• resolution of a Sylvester equation;

• application of the nonsmooth control-based method.

Existing techniques

The numerical results obtained using existing techniques are presented here.

Real pole placement

The following pole placement gives rather good results in simulation as far as the convergence speed of the observer is concerned

p = -4.7 -4.6 -2.1 -1.2 -1.1 -0.3 (26)
Using the pole placement and diagonalization approach in Sect. 3.2.1, one obtains the following P and L matrices

P =         -2.
        , L =         -0.0000 -1.2981 -1.1800 -1.0496 -0.0981 -0.3113         (27)
which leads to the following matrix M = P (A -LC) P -1

Table 1: Control-based approach parameters for the example described in Sect. 6.1, p (A-LC) refers to the poles of the observer error dynamics ( 24)

Syn. model Requirement Settings (C1-CT) (R1-CT) - (C2-CT) (R2-CT) m max ij = 200 (C3) (R3)-1 Re(p A-LC ) ∈ -∞, -3.10 -3 (R3)-2 1 0.4 T d→e (s) ≤ 1 M =         -4.7000
0.0000 -0.0000 0.0000 0.0000 -0.0000 -0.0000 -4.6000 -0.0000 0.0000 0.0000 -0.0000 -0.0000 0.0000 -2.1000 0.0000 0.0000 0.0000 0.0000 -0.0000 -0.0000 -1.2000 -0.0000 0.0000 -0.0000 0.0000 0.0000 0.0000 -1.1000 -0.0000 -0.0000 0.0000 -0.0000 -0.0000 -0.0000 -0.3000

        (28)
Due to numerical errors, the observation error dynamics is however not certified to be cooperative.

Sylvester equation

The approach in Sect. 3.2.2 is applied to the considered system. The following upper-triangular Metzler matrix M is proposed

M =         -4
.7000 1.0000 1.0000 1.0000 1.0000 1.0000 0 -4.6000

1.0000 1.0000 1.0000 1.0000 0 0 -2.1000 1.0000 1.0000 1.0000 0 0 0 -1.2000 1.0000 1.0000 0 0 0 0 -1.1000 1.0000 0 0 0 0 0 -0.3000

        (29)
along with Q = 1 2 3 4 5 6 . Solving Eq. ( 15) for these values, one obtains

P =        
        , L =         0.0000 -1.2981 -1.1800 -1.0496 -0.0981 -0.3113         (30)
which leads to the following matrix

M = P (A -LC) P -1 M =         -4
.7000 1.0000 1.0000 1.0000 1.0000 1.0000 0.0000 -4.6000

1.0000 1.0000 1.0000 1.0000 -0.0000 -0.0000 -2.1000

1.0000 1.0000 1.0000 0.0000 -0.0000 0.0000 -1.2000 1.0000 1.0000 -0.0000 -0.0000 0.0000 -0.0000 -1.1000 1.0000 -0.0000 -0.0000 0.0000 -0.0000 0.0000 -0.3000

        (31)
In this case, numerical errors also appear in the final result.

Control-based approach

The method proposed in Sect. 4 and 5 is used. The synthesis models and requirements used are recalled in Table 1 and refer to notations used in Sect. 5.

Using the control law synthesis algorithm presented in Sect. 5.1, a locally optimal solution is found in 287s after 3 restarts and requiring 1107 iterations. The following results are obtained, where M = P (A -LC)P -1 is Hurwitz Metzler.

Overview of LTI int. obs. design: towards a non-smooth optimisation-based approach, IET CTA, author version In blue, simulation results for the system in Eq. ( 22) and in magenta (resp. black/red), for the classical observer in Eq. ( 8) obtained using the observer gain L in Eq. (32) (resp. Eq. ( 27) or Eq. ( 30)) produced by the control-based approach (resp. real pole placement or Sylvester equation resolution).

M =         -0.
        , L =         -0.0862 0.0583 -0.1903 0.0349 0.0366 -0.1631         (32)

Simulations and comparisons

The system is initialized at x 0 and simulated over 40s using u(t) = sin(t) and a random-numbergenerated signal d(t) ∈ [-2, 1]. Results are shown on Fig. 1 and2.

The results obtained using the control-based approach are highly satisfying. Although the obtained interval observer seems slower than the one obtained using real pole placement, it converges closer to the system state. Additional requirements could be used to deal with the convergence speed. The results obtained using the Sylvester equation approach are very bad due to the difficulty of selecting appropriate matrices M and Q. Moreover, P is used to determine L while this is not the case in the other methods.

Overview of LTI int. obs. design: towards a non-smooth optimisation-based approach, IET CTA, author version In blue, simulation results for the system in Eq. ( 22) and in magenta (resp. black/red), bounds of the interval observer obtained using the control-based approach (resp. real pole placement/Sylvester equation resolution). The lower bounds are represented using a dashed line. The plots on the left are the same as the plots on the right but with limited ordinate excursion.

Continuous-time unstable third-order example

The following example is adapted from [START_REF] Raïssi | Interval state estimation for a class of nonlinear systems[END_REF]. The considered system dynamics is given by

ẋ = Ax + Bd y = Cx (33)
where x ∈ R 3 and d(t) is an unknown but bounded state disturbance with

d(t) = -1 ≤ d(t) ≤ 1 = d(t), ∀t.
The signal d(t) = sin(2t) will be used in simulation. The system matrices are defined as follows

A =   2 0 0 1 -4 √ 3 -1 - √ 3 -4   , B =   -1 0 1   , C = 1 0 0 ( 34
)
This system is unstable and detectable. The initial state is given by x 0 = -2 1.5 -1 and the initial observer state by x 0 = 0. Bounds on the initial state are supposed to equal x 0 = -2 -2 -2 and x 0 = -x 0 . The initial observation error e 0 lies in [e 0 , e 0 ] = [x 0x 0 , -x 0x 0 ]. The observation error dynamics is defined by

ė = (A -LC) e + Bd e 0 = x 0 -x 0 (35)
which is not cooperative. Under Hyp. 3, the following system is an interval observer for the observation error dynamics in Eq. ( 35), where P + B = max (P B, 0) and

P - B = P + B -P B.        e z = M e z + P + B d -P - B d e z
= M e z + P + B d -P - B d e z (0) = P + e 0 -P -e 0 e z (0) = P + e 0 -P -e 0 (36) Reverting back to the initial coordinates is done using Eq. [START_REF] Thabet | An effective method to interval observer design for time-varying systems[END_REF]. The two following methods are considered to determine P and L:

• application of [4, Lemma 1, p. 261];

• application of the nonsmooth control-based method.

Application of the constructive Lemma

In this section, Lemma 1 in [4, p. 261] is used as a practical method to solve the Sylvester equation presented in Sect. 3.2.2. At first, one has to choose the observer gain so that A -LC is Hurwitz, for example L = 3 0 0 . Then, a targeted matrix M is selected such that it is Metzler and has the same eigenvalues as A -LC: In this section, the approach proposed in Sect. 4 and 5 is used. The synthesis models and requirements used, along with the settings values, are presented in Table 2.

M =   -3 2 0 0 -3 2 2 0 -3   (
Using the control law synthesis algorithm presented in Sect. 5.1, a locally optimal solution is found after 5 restarts requiring less than 400 iterations each. The execution time rises to 15s. The result is given below. where M = P (A -LC) P -1 with no numerical error.

P =   0.

Simulations and comparisons

The system initialized at x 0 is simulated over 10s with d(t) = sin(2t). As expected, the system state diverges. However, considering the observation error e rather than the state x, the bounds e and e obtained using the interval observer in Eq. ( 36) are expected to converge. Simulation results are displayed on Fig. 3 and4.

In this particular example, the results obtained using the proposed control-based approach are better. The obtained matrix M is certified to be Hurwitz Metzler. By adding control requirements and simultaneously synthesizing P and L, the disturbance can be better rejected.

Discrete-time example

This example is adapted from [START_REF] Efimov | Estimation and control of discrete-time LPV systems using interval observers[END_REF]. The following discrete-time system with sample time T s = 1s is considered

x t+1 = A d x t + B d d t y t = C d x t (40)
where x t ∈ R 5 and d t is an unknown disturbance bounded by [-1, 1] and which is simulated by a random number generator on a limited range and with a time sampling of T s . The numerical values used for the system matrices are

A d =       -0.
     , B d =       -1 0 0 0 1       , C d = 1 0 0 0 0 0 0 0 1 0 (41)
The system is Schur-stable. It is initialized at x 0 = -0.3 -0.5 0.6 0.9 -0.2 . The Luenberger observer is initialized at x 0 = 0 and the initial state is supposed to lie in [-x 0 , x 0 ] where x 0 = 1 1 1 1 1 . The initial observation error e 0 lies in [e 0 , e 0 ] = [-x 0x 0 , x 0x 0 ]. The observation error dynamics is defined by 35) and in solid black, bounds of the interval observer obtained using the control-based approach. The results obtained using Lemma 1 in [START_REF] Raïssi | Interval state estimation for a class of nonlinear systems[END_REF] are represented in dashed lines. = P + e 0 -P -e 0 e z,0 = P + e 0 -P -e 0

Reverting back to the initial coordinates is performed using Eq. ( 12). In that case, the Sylvester method proposed in Remark 3 yields very poor results with the chosen arbitrary pole placement and non-negative matrix M d . The two remaining methods: real pole placement as described in Remark 2 and the approach proposed in this article are compared.

Real pole placement and diagonalization

A pole placement is used as described in Remark 2 in Sect. 3.2.1. The desired poles are chosen arbitrarily as real numbers inferior to 1 and positive p 3 = 0.1 0.3 0.5 0.7 0.9 (44)

The following P and L matrices are thus obtained

P =       -0.
= P (A -LC)P -1 M =      
0.9000 0.0000 -0.0000 0.0000 -0.0000 -0.0000 0.7000 -0.0000 0.0000 0.0000 -0.0000 -0.0000 0.1000 -0.0000 0.0000 0.0000 0.0000 0.0000 0.5000 -0.0000 -0.0000 -0.0000 0 -0.0000 0.3000

      (46)
Due to numerical errors, the matrix M is not non-negative up to the machine precision.

Control-based approach

The approach proposed in Sect. 4 and 5 is used, in the case of discrete-time systems. The synthesis models and requirements used are recalled in Table 3.

Using the control law synthesis algorithm presented in Sect. 5.1, a local optimal solution is found after 3 restarts and 400 iterations at maximum. The execution time rises to 41s. The following matrices are obtained where M d = P (A d -LC d )P -1 is Schur-stable and non-negative as required.

Simulations and comparisons

The disturbance d t used for the simulation is randomly generated with d t ∈ [-1, 1]. The simulation results are shown on Fig. 5. The results obtained using the control-based approach are very satisfactory. The simultaneous synthesis of P and L leads to a more precise observer. An arbitrary choice of the poles is not required anymore since it is included in the optimization process.

Conclusions

In this article, a new approach to compute an appropriate change of coordinates for a system to be cooperative in these new coordinates has been introduced following preliminary results in [START_REF] Chambon | Metzler matrix transform determination using a non-smooth optimization technique with an application to interval observers[END_REF]. Both the continuous-and discrete-time cases have been considered. An overview of the existing techniques to determine such transformation has been performed. The new approach has been applied on three examples and compared with the existing techniques.

The proposed method is based on structured controller synthesis techniques [START_REF] Apkarian | Tuning controllers against multiple design requirements[END_REF][START_REF] Apkarian | Nonsmooth H ∞ synthesis[END_REF]. It is used to enforce specific properties on the state-space matrices of a system. The main addition of the method in comparison with existing ones is the possibility to satisfy additional constraints dealing with the dynamics of the observer. Also, in the case of interval observers, matrices P and L are determined simultaneously while other methods require to choose a targeted Metzler matrix M with the same eigenvalues as A -LC, which is not an easy task from the authors point of view.

The current limitation of the method is the computation time which increases with the order of the system. Also, it uses algorithms which were not specifically designed for this task. The consequence is a loss in computational efficiency. As far as the theory of interval observers is concerned, the proposed method currently only applies to determining a time-invariant change of coordinates in the linear time-invariant case. A perspective of this work could be to extend the approach to the time-varying case for which theoretical results have been proposed [START_REF] Efimov | Interval state observer for nonlinear time varying systems[END_REF].

Proposition 1 .

 1 (Equivalent control design problem, continuous-time case) Let (A, C) detectable and m ij (P, L) = P (A -LC) P -1 ij for P ∈ R n×n and L ∈ R n×m . If the following system

20 Figure 1 :

 201 Figure1: In blue, simulation results for the system in Eq. (22) and in magenta (resp. black/red), for the classical observer in Eq. (8) obtained using the observer gain L in Eq. (32) (resp. Eq. (27) or Eq. (30)) produced by the control-based approach (resp. real pole placement or Sylvester equation resolution).

Figure 2 :

 2 Figure2: In blue, simulation results for the system in Eq. (22) and in magenta (resp. black/red), bounds of the interval observer obtained using the control-based approach (resp. real pole placement/Sylvester equation resolution). The lower bounds are represented using a dashed line. The plots on the left are the same as the plots on the right but with limited ordinate excursion.

37) Using [ 4 , 2 =

 42 Lemma 1, p. 261] with e 1 = 1 0 1 and e -0.8660 0.5000 -0.0915 0.0000 -1.0000   , M = P (A -LC) P -1 =

Figure 4 :

 4 Figure4: In solid magenta, simulation results over 20s for the system in Eq. (35) and in solid black, bounds of the interval observer obtained using the control-based approach. The results obtained using Lemma 1 in[START_REF] Raïssi | Interval state estimation for a class of nonlinear systems[END_REF] are represented in dashed lines.

10 Figure 5 :

 105 Figure5: Left: in blue, simulation results for system (40) and in magenta (resp. black), for the discrete-time formulation of the classical observer in Eq. (8) obtained using the observer gain L produced by the control-based approach (resp. real pole placement). Right: bounds of the interval observer compared to the simulated state.

Table 2 :

 2 Control-based approach parameters for the example described in Sect. 6.2, p (A-LC) refers to the poles of the observer error dynamics(35)

	Syn. model Requirement	Settings
	(C1-CT)	(R1-CT)		-
	(C2-CT)	(R2-CT)	m max ij	= 100
	(C3)	(R3)-1 (R3)-2	Re(p A-LC ) ∈ -10, -3.10 -3 1 0.8 T d→e (s) ≤ 1
	6.2.2 Control-based approach

  In blue, simulation results over 2s for the system in Eq. (33) and in magenta (resp. black), bounds of the interval observer obtained using the control-based approach (resp. Lemma 1 in[START_REF] Raïssi | Interval state estimation for a class of nonlinear systems[END_REF]).

		200
	x 1	0
	-200	0	0.2	0.4	0.6	0.8	1	1.2	1.4	1.6	1.8	2
		50
	x 2	0
		-50	0	0.2	0.4	0.6	0.8	1	1.2	1.4	1.6	1.8	2
		50
	x 3	0
		-50	0	0.2	0.4	0.6	0.8	1	1.2	1.4	1.6	1.8	2
								Time (s)
	e 3 e 2 e 1 Figure 3: Time (s) 0 2 4 6 8 10 0 2 4 6 8 10 0 2 4 6 8 10	12 12 12	14 14 14	16 16 16	18 18 18	20 20 20

Table 3 :

 3 Control-based approach parameters for the example described in Sect.[START_REF] Apkarian | Tuning controllers against multiple design requirements[END_REF].3, T dt→et (z) refers to the transfer function from e t to d t , stability of the observation error dynamics is also ensured M d e z,t + P + B d t -P - B d t e z,t+1 = M d e z,t + P + B d t -P - B d t e z,0

	Syn. model Requirement		Settings
	(C1-DT)	(R1-DT)	m max ij	= 100
	(C3)	(R3)-1	Stab. Eq. (42)
		(R3)-2	T dt→et (z) 2 ≤ 1
				e t+1 = (A d -LC d )e t + B d d t e 0 = x 0 -x 0	(42)
	which is not cooperative for any value of L. Under Hyp. 4, the following system is an interval observer
	for the observation error dynamics defined in Eq. (42), where P + B = max(P B d , 0) and P -B = P + B -P B d .
			   	e z,t+1 =
			  

Département de Commande des Systèmes et Dynamique du Vol -Onera, Centre de Toulouse,

Avenue ÉdouardBelin,

Toulouse, France, emmanuel.chambon@onera.fr

Linear Parameter Varying

The required synthesis models are enumerated here in the form of collections of models. Note that some collections only apply to the continuous-(CT) or discrete-time case (DT).

(C1-CT) n(n -1) unidimensional fictitious systems G ij M i,j,i =j as defined in Eq. ( 20);

as defined in Eq. ( 21) with f a linear function mapping -m max ij , 0 onto [-1, 1] where m max ij is a design parameter playing the same role as in (C2-CT) (see Remark 5);

This helps to restrict the set of acceptable solutions when 0 < m max ij < +∞. The initialising variables excursion is thus limited around a smaller set of potential local optima;

(C3) any other model in state-space representation, describing the control or observation problem, thus including tunable controller or observer parameters.

A series of requirements are now expressed on each collection.

Requirements

For each collection in Sect. 5.2, a set of requirements (or constraints, see Sect. 4.3) is expressed.

(R1-CT) a requirement on the closed-loop poles location is expressed on (C1-CT) in the form of a constraint on the minimum decay rate. This ensures negativity of -m ij (P, L) hence positivity of m ij , ∀i = j (CT case);

(R1-DT) similar requirement to (R1-CT) but expressed on the n 2 models in (C1-DT);

(R2-CT) similar to previous requirements sets but applied on (C2-CT). This ensures negativity of m ijm max ij , ∀i = j;

(R3) any control requirement to enforce on (C3) like disturbance rejection, closed-loop poles, etc.

Applications

In this section, the method proposed in Sect. 5 is applied to three examples, including an unstable system, using the interval observers formalism presented in Sect. 3. A comparison with the methods proposed in the literature is also provided.

Continuous-time stable sixth-order example

The formalism of interval observers in the continuous-time case is presented in Sect. 3.1.2. The method presented in this article proposes to enforce Hyp. 3 by computing P and L such that M is Hurwitz Metzler.

The following example is inspired from [START_REF] Mazenc | Interval observers for linear time-invariant systems with disturbances[END_REF]. The considered system dynamics is given by ẋ

where x ∈ R 6 , u(t) = sin(t) and d(t) is an unknown but bounded measurement disturbance with d = -2 ≤ d(t) ≤ 1 = d, ∀t. A random number generator with limited range is used in simulation.

The system matrices are numerically given by


Alexis Brenon
email: alexis.brenon@imag.fr

Franc ¸ois Portet
email: francois.portet@imag.fr

Michel Vacher
email: michel.vacher@imag.fr

Preliminary Study of Adaptive Decision-Making System for Vocal Command in Smart Home

In smart homes, prediction and decision are often defined a priori and require tuning from the user, which can be tedious, and complex. However, these smart homes have the ability to analyze the user behavior and to modify their decisions automatically. We present a preliminary study that tests a decision system from voice command and the user's context, which is modified by reinforcement learning. The system ran on a realistic corpus, which shows the interest of such an adaptation.

I. INTRODUCTION

The smart home aims to improve the user control and experience. To provide this enhanced control, these systems perceive their environment and make decisions about it. This decision can take place reactively -after a user command -or proactively -to alert about a risky situation. This perception is useful not only to make a decision but also to adapt the decision to the context, characterized by the current conditions.

Last years, the community has shown a new keen interest in voice controlled smart homes [START_REF] Cristoforetti | The DIRHA simulated corpus[END_REF]. This kind of control is based on a vocal user interface, which allows a 'natural' communication with the system and, which is well adapted for disabled people or emergency situation (hands-free and distant interaction) [START_REF] Vacher | Evaluation of a context-aware voice interface for Ambient Assisted Living: qualitative user study vs. quantitative system evaluation[END_REF]. In this framework, vocal commands may be 'turn on the light', 'check the door', 'call my daughter', etc. and the context must be used to remove the ambiguity of the command. If the command 'turn on the light' is pronounced in a bedroom with many lamps, the user doesn't precise which lamp(s) to turn on because he expects that his counterpart would guess it. Moreover, it would not be natural to ask the user to specify any details of a command; it is up to the system to check the context to take the most relevant decision.

The state of the art shows some approaches deploying a decision-making system in a smart home. For example Moore et al. [START_REF] Moore | Rule strategies for intelligent context-aware systems: The application of conditional relationships in decision-support[END_REF] has developed a system which uses a set of fuzzy rules to find the best decision based on the context. Kofler et al. [START_REF] Kofler | A semantic representation of energy-related information in future smart homes[END_REF] use description logic to define the behavior of a contextaware system while Leong et al. [START_REF] Leong | A rule-based framework for heterogeneous subsystems management in smart home environment[END_REF] model the behavior of a perceptive system with ECA rules (Event-Condition-Action). Other approaches based on Bayesian networks have been implemented [START_REF] Lee | Fusion of modular bayesian networks for context-aware decision making[END_REF] to reflect the uncertainty of the data. However, in these proposals, the system adapts to the current situation and does not take into account changes that may occur. Indeed, depending on the season, time of day, the arrival of a guest or a disability, patterns and contexts of use may change. In addition, to reflect the preferences of the user, these systems need to be configured, which can be very complex [START_REF] Shafti | Inferring eca-based rules for ambient intelligence using evolutionary feature extraction[END_REF]. Moreover, some misinterpretations of a vocal command or of the context may be identified by the system that could automatically correct it to avoid the call for a fine and tedious parameterization. These constraints militate for a flexible, scalable system that can adapt to the changing behavior of individuals.

This was the case in the ACHE project (Adaptive Control of Home Environment) [START_REF] Mozer | The Neural Network House: An Environment hat Adapts to its Inhabitants[END_REF] in which a home automation control system was learned, observing how the user deals with the system decisions. Actions were predicted by a neural network and the usefulness of the action was modified using reinforcement learning techniques. However, this project does not seem to have been brought to completion and did not explicitly uses the context to make its decision.

The aim of the study presented in this article is to develop a system which adapts itself to the user and the environment in the long term. The article presents in section II the Q-learning method and how it is particularly well suited to the problem of decision-making following a voice command. Section III describes the testing of this method on a corpus acquired in realistic condition. The article ends with a discussion of the results and the perspectives for improvement.

II. METHOD

We make a brief introduction to Q-learning and how it was implemented for decision-making system in a smart home.

A. Reinforcement Learning & Q-Learning

Reinforcement learning is a machine learning technique that allows an agent to learn its behavior through a feedback from its environment. Thus, a reinforcement learning problem is based on three main components [START_REF] Harmon | Reinforcement learning: A tutorial[END_REF]: (1) the environment on which agent acts; (2) the reinforcement function which defines the objective of the agent, assigning a reward for each state-action pair; (3) the value function associating the expected reward until final state (where the system succeeds or fails), to each state of the agent.

The goal of the agent is to determine a strategy, based on the value function, by a series of trial-and-error interactions. The first approaches were based on the principles of dynamic programming to modify the value function (usually represented as an association table) until convergence.

In 1989, Watkins published an extension to the dynamic programming approach called the Q-Learning [START_REF] Watkins | Learning from delayed rewards[END_REF]. He introduced the notion of Q-Value not associated with a system state but with a state-action pair. Similarly, the notion of value function was replaced by the Q-function. It also defined the equation for updating the values of the Q-function, according to the Bellman equation used for dynamic programming.

Q n+1 st,at = Q n st,at + α r(s t , a t ) + γ max at+1 Q n st+1,at+1 (1)
where Q n denotes the Q-function at step n, s t the state of the environment at time t, a t an action at time t, α the learning factor used to vary the learning speed of the agent, r(s t , a t ) the reward received for action a t , γ the discount factor and max at+1 Q(s t+1 , a t+1 ) the value of Q for the best action a t+1 in the next state s t+1 . This new approach limited the computing cost of reinforcement learning for non-deterministic environments. While the use of dynamic programming implies the computation of a sum of a theoretically infinite number of interactions, the Q-Learning only requires the values of the current state and of the next one to update the Q-function. However, the Q-value associated with a state is provided to converge to the optimal value after enough interactions, as in previous approaches.

B. Voice Controlled Smart Home

A smart home is a place that integrates sensors and actuators (from home automation) and is capable of perceiving the environment to act on it reactively or proactively [START_REF] Chan | Smart homescurrent features and future perspectives[END_REF].

A voice control application illustrates this need for perception and action. For example, if a user in a smart home pronounces 'Turn on the light', another person will easily interpret this request and take action. A computer system must infer alone the lamp that is implicit in the statement (if there are several lamps) and its intensity (in the case where it would be adjustable). This missing information must be retrieved from the knowledge of the context: the system must infer the user's location to turn on the light in the right room and must also deduct, from his activity, the location and the light intensity.

In our study, we consider that possible actions are: turn on/off the light/radio, open/close the blinds/curtains, tell the temperature/time, request a video/phone/emergency call.

These actions are a subset of possible actions defined after a user study [START_REF] Portet | Design and evaluation of a smart home voice interface for the elderly: acceptability and objection aspects[END_REF]. Of course, this set of actions should be tailored to each user and housing, but this predefined list was useful for the evaluation of the system. This study also identified voice commands using a simple grammar as shown in Figure 1. Each command begins with a unique keyword, 'Nestor'. This kind of environment seems particularly suited for Qlearning as the sets of states and actions, though potentially great, are finite and discrete.

III. EXPERIMENT

This section presents the realized scenarios and realistic data from a previous study, details how learning has been done and the evaluation of the experiment.

A. Scenario of the use of a voice controlled smart home

We consider in this study the DOMUS smart home designed by Grenoble Informatics Laboratory (LIG) [START_REF] Gallissot | Une plateforme usage pour l'intgration de linformatique ambiante dans l'habitat. l'appartement domus[END_REF]. This 30 square meters home, includes a bathroom, a kitchen, a bedroom, and a study. All these rooms are equipped with sensors and actuators such as infrared motion sensors, touch sensors, video cameras (only used for annotation purposes), etc. In addition, seven microphones were placed in the ceiling for audio capture. The apartment is fully usable and can accommodate a resident for several days. More than 150 sensors are managed in the apartment to provide different services (e.g. light, opening/closing shutters, media management, etc.).

To collect decision data in context, several participants were recruited to play scenarios of the daily life in the apartment. Participants were asked to issue voice commands to activate the actuators in the smart home. The objective of this experiment was to test a smart controller [START_REF] Chahuara | Making Context Aware Decision from Uncertain Information in a Smart Home: A Markov Logic Network Approach[END_REF] in real situations corresponding to voice commands spoken by the user. The situations that we considered in the study were: (1) to clean up the apartment, (2) prepare and eat a meal, (3) converse via video conference (4) do leisure activities (reading), ( 5) take a nap. To guide the participants in the realization of the experiment, the grammar voice command was provided (see Figure 1) with a scenario of everyday scenes (Figure 2). This scenario was designed to last about 45 minutes, however, there was no constraint on the execution time. The first four parts put the user in situations of daily activities while speaking a voice command. Each participant received a list of actions to make and voice commands to pronounce. Each participant had to use a voice command, repeating up to 3 times in case Go to the kitchen, ask for the temperature:

Nestor donne la température Take a snack, put the dish in the sink, ask for the time:

Nestor donne moi l'heure You realized it's late, you need to go shopping Before leaving, you want to turn off the lights:

Nestor éteins la lumière You also want to close the blinds: Nestor baisse les stores Figure 2. A sample of an example scenario that a participant had to do.

of system failure. A Wizard of Oz was used in the case of persistent failure. A total of 15 people (9 women, 6 men) participated in the experiment to record 11 hours of data. The average age of participants was 38 ± 13.6 (19-62). All experiments were video recorded but only for annotation purposes. In the study presented in this article, the home automation control corpus is composed of a set of situations (human activity, location, condition of the apartment), voice control statements and the corresponding home automation commands. This data set is extracted from the Sweet-Home corpus [START_REF] Vacher | The Sweet-Home speech and multimodal corpus for home automation interaction[END_REF].

B. Learning

Our model is learned on a first corpus, called train, before to be evaluated using cross-validation technique on a test corpus.

1) Corpus description: Each of the data sets is one or more text files following this scheme: The focus is on taking a decision when the user calls the system through a vocal command. Thus, each file entries necessarily contains a valid usr cmd field as the usr loc field which corresponds to the user location when he interacts with the system. However, the usr act field, which symbolizes the activity of the user, can take a null value that means that we do not know what activity is being performed.

Each state of the system (which here corresponds to the current context) is then associated with an expected output consisting of two pieces of information: the desired command -turn on the lights -and the place of the action -the ceiling or bedside lamp. This information is defined, for the train corpus, by a set of rules and in the case of the test corpus by a domain expert.

The train corpus is a corpus which should enable us to learn a first model of our environment. For this, it must be relatively large. As there is little decision data corpus, the training corpus was generated automatically by following the method used in [START_REF] Chahuara | Making Context Aware Decision from Uncertain Information in a Smart Home: A Markov Logic Network Approach[END_REF]. For this, we used a script that runs through all possible states of our system and deducts the expected decision from a set of logical rules. From the 300 states of our system, about 400 samples are generated.

The test corpus is based on real data extracted and annotated from experiments performed during the Sweet-Home project [START_REF] Vacher | The Sweet-Home speech and multimodal corpus for home automation interaction[END_REF]. During cross-validation, we leave data from one subject for evaluation, tuning our model using data from fifteen subjects. Each subject did 25 interactions on average leading to a total of 407 interactions covering 37 states.

2) Experiment: The experiment is conducted in 2 phases: training, and cross-validation. Cross-validation is executed on a sixteen folds corpus. Each phase is broken down into 10 steps at the end of which the awards received are integrated for learning. This allow us to track the performances all along the experiment.

For the training phase, we use the train corpus to simulate many interactions. We try to ensure that all states have been met several times allowing the system to explore different actions outcomes. Then, we get a Q-values table representing a first model of the environment.

We use this model as a base for the evaluation phase. For each fold of the test corpus, the system adapts itself using data from fifteen subjects before being evaluated on the left out one. The results of the experiment are saved to extract metrics to evaluate the performance of our system.

In both cases, learning process in the same. We provide a state to the system which returns an action. If expected and returned actions are the same, the system is rewarded and we provide a new state. Else, the system is punished and it stays in the same state. With the train corpus, the system can try infinitely to find the right action whereas with the test corpus, to simulate the user giving up, it has only three tries. For the moment, we use a Minimum time to goal [START_REF] Harmon | Reinforcement learning: A tutorial[END_REF] reward function, but more custom functions will be used to take into account the apartment layout or to severely punish the system if it is too long to find the right action.

3) Evaluation and metrics: During the experiment, we save data allowing us to compute different metrics like the F1-score and the mean reward.

The first one is a classical classification metrics. Each decision making is seen as a classification of the state. We can then compute the precision and the recall to compute the F1-score. This reflects the ability of the system to choose the right decision based on the current state.

The mean reward is a more custom metrics which gives the mean reward obtained between two reward integrations for learning (the 10 steps explained above). Depending on the reward function used, this metric, with the F1-score, allow us to determine if the system fails often but with a small mistake (it lights the ceiling instead of the bedside lamp) or rarely but with a big one (it turns on the radio instead of closing the store).

C. Results

We ran the protocol explained above, and summarized the classification results in the confusion matrix shown in Figure 3. The fact that the matrix diagonal is well identified, as well as the good overall accuracy of about 70%, show that the learning process associate the right actions to the right state.

Nevertheless, you can see two erroneous behaviors. The first one is a confusion between actions 19 and 20, 'turning on the kitchen ceiling light' and 'turning on the kitchen sink light'. This is an understandable confusion as some states are very ambiguous even for a human annotator. The second error appears on actions 11 and 12, 'turn off all the lights of the kitchen' and 'turn off the kitchen ceiling light', classified as 'tell time in the kitchen'. We are still investigating to understand what is going on, but this seems to be cause by the lack of representative samples for both actions. The experiment reported in this article shows that adapting a system to a user are possible even with a simple reinforcement approach as the Q-learning. However, a number of points have not been taken into account. The information generated by the home automation system is often used by decision systems. But in this work the decision is based on the information of location and activity that are the result of an inference. So, it is necessary to work on the transfer of negative reward that can either be due to a bad decision or a wrong inference. For this, we have to check whether the theoretical frameworks taking uncertainty into account can be applied to this double problem of uncertainty and reinforcement, as in the work of Hagras et. al. [START_REF] Hagras | An incremental adaptive life long learning approach for type-2 fuzzy embedded agents in ambient intelligent environments[END_REF] in which a fuzzy reasoning based system adapts to the person while taking into account the uncertainty. The Partially Observable Markov Decision Process (POMDP) also seems particularly suited to this type of problem [START_REF] Zaidenberg | Reinforcement Learning of User Preferences for a Ubiquitous Personal Assistant[END_REF]. Finally, it is worth checking whether an MLN-based system [START_REF] Chahuara | Making Context Aware Decision from Uncertain Information in a Smart Home: A Markov Logic Network Approach[END_REF] or a DNN-based [START_REF] Mnih | Human-level control through deep reinforcement learning[END_REF] one can be adapted to include a strengthening mechanism.

Another important point not considered in the study is the identification of persons in the home. Indeed, a behavior is associated with a user and so rewards may be different. This problem can be included in the previous problem, characterizing a limited range of users that should be identified together with the rest of the environment.

Finally, we would like to validate our approach. A long-term adaptation requires a large amount of data that is extremely expensive to implement and raises verification problems (e.g. differentiate essential versus minor adjustments). Our strategy first is to test the system against radical adaptations like a person or habitat change that will provide information on system ease of 'installation' in a new environment. Moreover, we will test a long-term adaptation in less expensive environments, potentially less intrusive than the habitat, and with little identification problem, through smartphone applications and actions in an intelligent office.

Figure 1 .

 1 Figure 1. Vocal commands grammar sample (in French).

  usr_cmd usr_loc usr_act -> expctd_cmd expctd_loc Examples: blind -open kitchen none ->blind -open kitchen light -on kitchen cook ->light -on kitchen -sink

Figure 3 .

 3 Figure 3. Confusion matrix for the decision-making system