S Aberkane

J C Ponsart

D Sauter

Multi-Objective Output Feedback Control of a Class of Stochastic Hybrid Systems with State Dependent Noise

Keywords: 2000 Mathematics Subject Classification: 93E03, 93E15, 93E20, 34D20

This paper deals with dynamic output feedback control of continuous time Active Fault Tolerant Control Systems with Markovian Parameters (AFTCSMP) and state-dependent noise. The main contribution is to formulate conditions for multi-performance design, related to this class of stochastic hybrid systems, that take into account the problematic resulting from the fact that the controller only depends on the FDI (Fault Detection and Isolation) process. The specifications and objectives under consideration include stochastic stability, H 2 and H ∞ (or more generally, Stochastic Integral Quadratic Constraints) performances. Results are formulated as matrix inequalities. The theoretical results are illustrated using a classical example from literature.

Introduction

Control engineers are faced with increasingly complex systems where dependability considerations are sometimes more important than performance. Sensor, actuator or process (plant) failures may drastically change the system behavior, resulting in performance degradation or even instability. Thus, fault tolerance is essential for modern, highly complex control systems. Fault Tolerant Control Systems (FTCS) are needed in order to preserve or maintain the performance objectives, or if that turns out to be impossible, to assign new (achievable) objectives so as to avoid catastrophic failures. FTCS have been a subject of great practical importance, which has attracted a lot of interest for the last three decades. A bibliographical review on reconfigurable fault tolerant control systems can be found in [START_REF] Zhang | Bibliographical review on reconfigurable fault-tolerant control systems[END_REF]. Active fault tolerant control systems are feedback control systems that reconfigure the control law in real time based on the response from an automatic fault detection and identification (FDI) scheme. The dynamic behaviour of active fault tolerant control systems (AFTCS) is governed by stochastic differential equations (because the failures and failure detection occur randomly) and can be viewed as a general hybrid system [START_REF] Srichander | Stochastic stability analysis for continuous-time fault tolerant control systems[END_REF]. A major class of hybrid systems is jump linear systems (JLS). In JLS, a single jump process is used to describe the random variations affecting the system parameters. This process is represented by a finite state Markov chain and is called the plant regime mode. The theory of stability, optimal control and H 2 /H ∞ control, as well as important applications of such systems, can be found in several papers in the current literature, for instance in [START_REF] Boukas | On Stabilization of Uncertain Linear Systems with Jump Parameters[END_REF][START_REF] Boukas | Linear Discrete-time Systems with Markovian Jumps and Mode Dependent Time-delay: Stability and Stabilizability[END_REF][START_REF] Boukas | H ∞ Constant Gain State Feedback of Stochastic Hybrid Systems with Wienner Process[END_REF][START_REF] Boukas | Stabilization of Stochastic Nonlinear Hybrid Systems[END_REF][START_REF] Boukas | Static Output Feedback Control for Stochastic Hybrid Systems: LMI Approach[END_REF][START_REF] Costa | Continuous-time state-feedback H 2 -control of Markovian jump linear systems via convex analysis[END_REF][START_REF] De Farias | Output Feedback Control of Markov Jump Linear Systems in Continuous-Time[END_REF][START_REF] De Souza | H ∞ Control For Linear Systems With Markovian Jumping Parameters[END_REF][START_REF] Ji | Controllability, stabilizability, and continuous-time Markovian jump linear quadratic control[END_REF][START_REF] Ji | Jump linear quadratic Gaussian control in continuous time[END_REF]. To deal with AFTCS, another class of hybrid systems was defined, denoted as active fault tolerant control systems with Markovian parameters (AFTCSMP). In this class of hybrid systems, two random processes are defined: the first random process represents system components failures and the second random process represents the FDI process used to reconfigure the control law. This model was proposed by Srichander and Walker [START_REF] Srichander | Stochastic stability analysis for continuous-time fault tolerant control systems[END_REF]. Necessary and sufficient conditions for stochastic stability of AFTCSMP were developed for a single component failure (actuator failures). In [START_REF] Mahmoud | Analysis of the Stochastic Stability for Active Fault Tolerant Control Systems[END_REF], the authors proposed a dynamical model that takes into account multiple failures occurring at different locations in the system, such as in control actuators and plant components. The authors derived necessary and sufficient conditions for the stochastic stability in the mean square sense. The problem of stochastic stability of AFTCSMP in the presence of noise, parameter uncertainties, detection errors, detection delays and actuator saturation limits has also been investigated in [START_REF] Mahmoud | Analysis of the Stochastic Stability for Active Fault Tolerant Control Systems[END_REF][START_REF] Mahmoud | Stochastic Stability Analysis of Active Fault-Tolerant Control Systems in the Presence of Noise[END_REF][START_REF] Mahmoud | Stability of Fault Tolerant Control Systems Driven By Actuators With Saturation[END_REF][START_REF] Mahmoud | Active Fault Tolerant Control Systems: Stochastic Analysis and Synthesis[END_REF]. Another issue related to the synthesis of fault tolerant control laws was also addressed by [START_REF] Mahmoud | Optimal Control Law for Fault Tolerant Control Systems[END_REF][START_REF] Shi | H ∞ -Control for Markovian Jumping Linear Systems with Parametric Uncertainty[END_REF][START_REF] Shi | Robust disturbance attenuation for discretetime active fault tolerant control systems with uncertainties[END_REF]. In [START_REF] Mahmoud | Optimal Control Law for Fault Tolerant Control Systems[END_REF], the authors designed an optimal control law for AFTCSMP using the matrix minimum principle to minimize an equivalent deterministic cost function. The problem of H ∞ and robust H ∞ control (in the presence of norm bounded parameter uncertainties) was treated in [START_REF] Shi | H ∞ -Control for Markovian Jumping Linear Systems with Parametric Uncertainty[END_REF][START_REF] Shi | Robust disturbance attenuation for discretetime active fault tolerant control systems with uncertainties[END_REF] for both continuous and discrete time AFTCSMP. The authors defined a single failure process to characterize random failures affecting the system, and they showed that the state feedback control problem can be solved in terms of the solutions of a set of coupled Riccati inequalities. The dynamic/static output feedback counterpart was treated by [START_REF] Aberkane | Output Feedback Stochastic Stabilization of Active Fault Tolerant Control Systems[END_REF][START_REF] Aberkane | H ∞ Stochastic Stabilization of Active Fault Tolerant Control Systems: Convex Approach[END_REF][START_REF] Aberkane | Output Feedback H ∞ Control of a Class of Stochastic Hybrid Systems with Wiener Process via Convex Analysis[END_REF]4] in a convex programming framework. Indeed, the authors provide an LMI characterization of dynamical/static output feedback compensators that stochastically stabilize (robustly stabilize) the AFTCMP and ensures H ∞ (robust H ∞ ) constraints. In addition, it is important to mention that the design problem in the framework of AFTCSMP remains an open and challenging problematic. This is due, particulary, to the fact that the controller only depends on the FDI process i.e. the number of controllers to be designed is less than the total number of the closed loop systems modes by combining both failure an FDI processes. The design problem involves searching feasible solutions of a problem where there are more constraints than variables to be solved. Generally speaking, there lacks tractable design methods for this stochastic FTC problem. Indeed, in [START_REF] Aberkane | Output Feedback Stochastic Stabilization of Active Fault Tolerant Control Systems[END_REF][START_REF] Aberkane | H ∞ Stochastic Stabilization of Active Fault Tolerant Control Systems: Convex Approach[END_REF][START_REF] Mahmoud | Active Fault Tolerant Control Systems: Stochastic Analysis and Synthesis[END_REF][START_REF] Shi | H ∞ -Control for Markovian Jumping Linear Systems with Parametric Uncertainty[END_REF][START_REF] Shi | Robust disturbance attenuation for discretetime active fault tolerant control systems with uncertainties[END_REF], the authors make the assumption that the controller must access both failures and FDI processes. However, this assumption is too restrictive to be applicable in practical FTC systems. In this note, and inspired by the work of [START_REF] De Souza | Mode-Independent H ∞ Filters for Hybrid Markov Linear Systems[END_REF] on mode-independent H ∞ filtering for Markovian jumping linear systems, the assumption on the availability of failure processes, for the synthesis purposes, is stressed. The results are based on a version of the well known Finsler's lemma and a special parametrization of the Lyapunov matrices. This note is concerned with dynamic output feedback control of continuous time AFTCSMP with state-dependent noise. The main contribution is to formulate conditions for multi-performance design, related to this class of stochastic hybrid systems, that take into account the problematic resulting from the fact that the controller only depends on the FDI process. The specifications and objectives under consideration include stochastic stability, H 2 and H ∞ (or more generally, Stochastic Integral Quadratic Constraints) performances. Results are formulated as matrix inequalities. A coordinate descent-type algorithm is provided and its running is illustrated on a VTOL helicopter example. This paper is organized as follows: Section 2 describes the dynamical model of the system with appropriately defined random processes. A brief summary of basic stochastic terms, results and definitions are given in Section 3. Section 4 considers the H 2 /H ∞ control problem. In Section 5, a coordinate descent-type algorithm is provided and its running is illustrated on a classical example from literature. Finally, a conclusion is given in Section 6.

Notations. The notations in this paper are quite standard. R m×n is the set of m-by-n real matrices. A T is the transpose of the matrix A. The notation X ≥ Y (X > Y , respectively), where X and Y are symmetric matrices, means that X -Y is positive semi-definite (positive definite, respectively); I and 0 are identity and zero matrices of appropriate dimensions, respectively; E{•} denotes the expectation operator with respect to some probability measure P ; L 2 [0, ∞) stands for the space of square-integrable vector functions over the interval [0, ∞); • refers to either the Euclidean vector norm or the matrix norm, which is the operator norm induced by the standard vector norm; • 2 stands for the norm in L 2 [0, ∞); while • E 2 denotes the norm in L 2 ((Ω, F, P ), [0, ∞)); (Ω, F, P ) is a probability space. In block matrices, indicates symmetric terms:

A B B T C = A B T C = A B C .

Dynamical Model of AFTCSMP with State Dependent Noise

To describe the class of linear systems with Markovian jumping parameters that we deal with in this paper, let us fix a complete probability space (Ω, F, P ). This class of systems owns a hybrid state vector. The first component vector is continuous and represents the system states, and the second one is discrete and represents the failure processes affecting the system. The dynamical model of the AFTCSMP with Wiener Process, defined in the fundamental probability space (Ω, F, P ), is described by the following differential equations:

ϕ :      dx(t) = A(ξ(t))x(t)dt + B(η(t))u(y(t), ψ(t), t)dt + E(ξ(t), η(t))w(t)dt + v l=1 W l (ξ(t), η(t))x(t)d l (t) y(t) = C2x(t) + D2(ξ(t), η(t))w(t) z(t) = C 1 x(t) + D 1 (η(t))u(y(t), ψ(t), t) (1)
where x(t) ∈ R n is the system state, u(y(t), ψ(t), t) ∈ R r is the system input, y(t) ∈ R q is the system measured output, z(t) ∈ R p is the controlled output, w(t) ∈ R m is the system external disturbance, ξ(t), η(t) and ψ(t) represent the plant component failure process, the actuator failure process and the FDI process, respectively. ξ(t), η(t) and ψ(t) are separable and mesurable Markov processes with finite state spaces Z = {1, 2, ..., z}, S = {1, 2, ..., s} and R = {1, 2, ..., σ}, respectively.

(t) = [ 1 (t) . . . v (t)] T is a v-dimensional standard Wiener process on a given probability space (Ω, F, P ), that is assumed to be independent of the Markov processes. The matrices A(ξ(t)), B(η(t)), E(ξ(t), η(t)), D 2 (ξ(t), η(t)), D 1 (η(t)) and W l (ξ(t), η(t)) are properly dimensioned matrices which depend on random parameters. In AFTCS, we consider that the control law is only a function of the mesurable FDI process ψ(t). Therefore, we introduce a full order dynamic output feedback compensator (ϕ d ) of the form:

ϕ d : dv(t) = A c (ψ(t))v(t)dt + B c (ψ(t))y(t)dt u(t) = C c (ψ(t))v(t) (2)
Applying the controller ϕ d to the AFTCSMP ϕ, we obtain the following closed loop system:

ϕ cl :      dχ(t) = Λ(ξ(t), η(t), ψ(t))χ(t)dt + Ē(ξ(t), η(t), ψ(t))w(t)dt + v l=1 W l (ξ(t), η(t))χ(t)d l (t) ȳ(t) = C2 (ψ(t))χ(t) + D2 (ξ(t), η(t))w(t) z(t) = C1(η(t), ψ(t))χ(t) (3)
where:

χ(t) = [x(t) T , v(t) T ] T ; ȳ(t) = [y(t) T , u(t) T ] T ; Λ(ξ(t), η(t), ψ(t)) = A(ξ(t)) B(η(t))Cc(ψ(t)) B c (ψ(t))C 2 A c (ψ(t)) ; Ē(ξ(t), η(t), ψ(t)) = E(ξ(t), η(t)) B c (ψ(t))D 2 (ξ(t), η(t)) ; C2 (ψ(t)) = C 2 0 0 C c (ψ(t)) ; D2(ξ(t), η(t)) = D 2 (ξ(t), η(t)) 0 ; C1(η(t), ψ(t)) = C1 D1(η(t))Cc(ψ(t)) ; W l (ξ(t), η(t)) = W l (ξ(t), η(t)) 0 0 0 .
Our goal is to compute dynamical output feedback controllers ϕ d that meet various specifications on the closed loop behavior. The specifications and objectives under consideration include stochastic stability, H 2 performance and H ∞ performance (or more generally, Stochastic Integral Quadratic Constraints (SIQC)).

The FDI and the Failure Processes ξ(t), η(t) and ψ(t) being homogeneous Markov processes with finite state spaces, we can define the transition probability of the plant components failure process as [START_REF] Mahmoud | Active Fault Tolerant Control Systems: Stochastic Analysis and Synthesis[END_REF][START_REF] Srichander | Stochastic stability analysis for continuous-time fault tolerant control systems[END_REF]:

   p ij (∆t) = π ij ∆t + o(∆t) (i = j) p ii (∆t) = 1 - i =j π ij ∆t + o(∆t) (i = j)
The transition probability of the actuator failure process is given by:

   p kl (∆t) = ν kl ∆t + o(∆t) (k = l) p kk (∆t) = 1 - k =l ν kl ∆t + o(∆t) (k = l)
where π ij is the plant components failure rate, and ν kl is the actuator failure rate. Given that ξ = k and η = l, the conditional transition probability of the FDI process ψ(t) is:

   p kl iv (∆t) = λ kl iv ∆t + o(∆t) (i = v) p kl ii (∆t) = 1 - i =v λ kl iv ∆t + o(∆t) (i = v)
Here, λ kl iv represents the transition rate from i to v for the Markov process ψ(t) conditioned on ξ = k ∈ Z and η = l ∈ S. Depending on the values of i, v ∈ R, k ∈ Z and l ∈ S, various interpretations, such as rate of false detection and isolation, rate of correct detection and isolation, false alarm recovery rate, etc, can be given to λ kl iv [START_REF] Mahmoud | Active Fault Tolerant Control Systems: Stochastic Analysis and Synthesis[END_REF][START_REF] Srichander | Stochastic stability analysis for continuous-time fault tolerant control systems[END_REF]. For notational simplicity, we will denote

A(ξ(t)) = A i when ξ(t) = i ∈ Z, B(η(t)) = B j and D 1 (η(t)) = D 1j when η(t) = j ∈ S, E(ξ(t), η(t)) = E ij , D 2 (ξ(t), η(t)) = D 2ij and W l (ξ(t), η(t)) = W lij when ξ(t) = i ∈ Z, η(t) = j ∈ S and A c (ψ(t)) = A ck , B c (ψ(t)) = B ck , C c (ψ(t)) = C ck when ψ(t) = k ∈ R. We also denote x(t) = x t , y(t) = y t , z(t) = z t , w(t) = w t , ξ(t) = ξ t , η(t) = η t , ψ(t) = ψ t and the initial conditions x(t 0 ) = x 0 , ξ(t 0 ) = ξ 0 , η(t 0 ) = η 0 and ψ(t 0 ) = ψ 0 .

Definitions and Basic Results

In this section, we will first give a basic definition related to stochastic stability notion and then we will summarize some results about exponential stability in the mean square sense of the AFTCSMP.

Stochastic Stability

For system (1), when u t ≡ 0 for all t ≥ 0, we have the following definition.

Definition 1: System (1) is said to be internally exponentially stable in the mean square sense (IESS), if there exist positive constants α and β such that the solution of

dx t = A(ξ t )x t dt + v l=1 W l (ξ t , η t )x t d lt satisfies the following inequality E x t 2 ≤ β x 0 2 exp [-α(t -t 0 )] (4)
for arbitrary initial conditions (x 0 , ξ 0 , η 0 , ψ 0 ).

The following theorem gives a sufficient condition for internal exponential stability in the mean square sense for the closed loop system (3).

Theorem 1: The closed loop system (3) is IESS for t ≥ t 0 if there exists a Lyapunov function ϑ(χ t , ξ t , η t , ψ t , t) such that K 1 χ t 2 ≤ ϑ(χ t , ξ t , η t , ψ t , t) ≤ K 2 χ t 2 (5)
and

Lϑ(χ t , ξ t , η t , ψ t , t) ≤ -K 3 χ t 2 (6)
for some positive constants K 1 , K 2 and K 3 , where L is the weak infinitesimal operator of the joint Markov process {χ t , ξ t , η t , ψ t }.

Remark 1: The proof of the Markovian property of the joint process {χ t , ξ t , η t , ψ t } is given for example in [START_REF] Khasminskii | Stochastic Stability of Differential Equations[END_REF][START_REF] Mahmoud | Active Fault Tolerant Control Systems: Stochastic Analysis and Synthesis[END_REF][START_REF] Srichander | Stochastic stability analysis for continuous-time fault tolerant control systems[END_REF].

A necessary condition for internal exponential stability in the mean square sense for the closed loop system ϕ cl is given by theorem 2.

Theorem 2: If the system (3) is IESS, then for any given quadratic positive definite function W (χ t , ξ t , η t , ψ t , t) in the variables χ t which is bounded and continuous ∀t ≥ t 0 , ∀ξ t ∈ Z, ∀η t ∈ S and ∀ψ t ∈ R, there exists a quadratic positive definite function ϑ(χ t , ξ t , η t , ψ t , t) in χ t that satisfies the conditions in theorem 1 and is such that

Lϑ(χ t , ξ t , η t , ψ t , t) = -W (χ t , ξ t , η t , ψ t , t).
Remark 2: The proofs of these theorems follow the same arguments as in [START_REF] Mahmoud | Active Fault Tolerant Control Systems: Stochastic Analysis and Synthesis[END_REF][START_REF] Srichander | Stochastic stability analysis for continuous-time fault tolerant control systems[END_REF] for their proposed stochastic Lyapunov functions, so they are not shown in this paper to avoid repetition.

The following proposition gives a necessary and sufficient condition for internal exponential stability in the mean square sense for the system (3).

Proposition 1: A necessary and sufficient condition for IESS of the system (3) is that there exists symmetric positive-definite matrices P ijk , i ∈ Z, j ∈ S and k ∈ R such that:

ΛT ijk P ijk + P ijk Λijk + v l=1 W T lij P ijk W lij + h∈Z h =i π ih P hjk + l∈S l =j ν jl P ilk + v∈R v =k λ ij kv P ijv < 0 (7)
∀i ∈ Z, j ∈ S and k ∈ R, where

Λijk = Λ ijk -0.5I     h∈Z h =i π ih + l∈S l =j ν jl + v∈R v =k λ ij kv     (8)
Proof: The proof of this proposition is easily deduced from theorems 1 and 2.

Proposition 2:

If the system (3) is IESS, for every w = {w t ; t ≥ 0} ∈ L 2 [0, ∞), we have that χ = {χ t ; t ≥ 0} ∈ L 2 ((Ω, F, P ), [0, ∞)), i.e., E ∞ 0 χ T t χ t dt < ∞, for any initial conditions (χ 0 , ξ 0 , η 0 , ψ 0 ).
Proof: The proof of this proposition follows the same lines as for the proof of proposition 4 in [START_REF] Aberkane | Output Feedback H ∞ Control of a Class of Stochastic Hybrid Systems with Wiener Process via Convex Analysis[END_REF].

We conclude this section by recalling a version of the well known Finsler's lemma that will be used in the derivation of the main results of this paper.

Lemma 1 [START_REF] De Souza | Mode-Independent H ∞ Filters for Hybrid Markov Linear Systems[END_REF]: Given matrices

Ψ ijk = Ψ T ijk ∈ R n×n and H ijk ∈ R m×n , ∀i ∈ Z, j ∈ S and k ∈ R, then x T t Ψ ijk x t < 0, ∀x t ∈ R n : H ijk x t = 0, x t = 0; (9)
if and only if there exist matrices L ijk ∈ R n×m such that:

Ψ ijk + L ijk H ijk + H T ijk L T ijk < 0, ∀i ∈ Z, j ∈ S, k ∈ R. ( 10
)
Note that conditions [START_REF] Boyd | Linear Matrix Inequalities in System and Control Theory[END_REF] remain sufficient for [START_REF] Boukas | Static Output Feedback Control for Stochastic Hybrid Systems: LMI Approach[END_REF] to hold even when arbitrary constraints are imposed to the scaling matrices L ijk .

4 The Control Problem

H ∞ Control

Let us consider the system (3) with

z t = z ∞t = C ∞1 x t + D ∞1 (η t )u(y t , ψ t , t)
z ∞t stands for the controlled output related to H ∞ performance.

In this section, we deal with the design of controllers that stochastically stabilize the closed-loop system and guarantee the disturbance rejection, with a certain level µ > 0. Mathematically, we are concerned with the characterization of compensators ϕ d that stochastically stabilize the system (3) and guarantee the following for all w ∈ L 2 [0, ∞):

z ∞ E 2 = E ∞ 0 z T ∞t z ∞t dt 1/2 < µ w 2 (11)
where µ > 0 is a prescribed level of disturbance attenuation to be achieved. To this end, we need the auxiliary result given by the following proposition.

Proposition 3: If there exists symmetric positive-definite matrices P ∞ijk , i ∈ Z, j ∈ S and k ∈ R such that

ΛT ijk P ∞ijk + P ∞ijk Λijk + v l=1 W T lij P ∞ijk W lij + CT ∞1jk C∞1jk + µ -2 P ∞ijk Ēijk ĒT ijk P ∞ijk + h∈Z h =i π ih P ∞hjk + l∈S l =j ν jl P ∞ilk + v∈R v =k λ ij kv P ∞ijv < 0 (12)
∀i ∈ Z, j ∈ S and k ∈ R. then the system (3) is IESS and satisfies

z ∞ E 2 = E ∞ 0 z T ∞t z ∞t dt 1/2 < µ w 2 (13)
Proof The proof of this proposition follows the same arguments as in [START_REF] Aberkane | H ∞ Stochastic Stabilization of Active Fault Tolerant Control Systems: Convex Approach[END_REF].

Remark 3: Given fixed matrices U ≥ 0, V = V T and Q, the previous characterization extends to more general quadratic constraints on w t and z t of the form

J SIQC = E t f t 0 z t w t T U Q Q T V z t w t dt < 0 (14)
These constraints are known as stochastic quadratic integral constraints [START_REF] Cheng | Tao Stability and Performance of the Stochastic Fault Tolerant Control Systems[END_REF].

The following proposition gives a sufficient condition to ( 14) to be hold.

Proposition 4: If there exists symmetric positive-definite matrices

P ijk , i ∈ Z, j ∈ S and k ∈ R such that   Θ ijk P ijk E ijk + C T 1jk Q C T 1jk ∆ V 0 -Σ   < 0 (15)
where

     U = ∆Σ -1 ∆ T Θ ijk = ΛT ijk P ijk + P ijk Λijk + v l=1 W T lij P ijk W lij + h∈Z h
J SIQC = E t f t 0 χ t w t T Φ(ξ t , η t , ψ t ) + C 1 (η t , ψ t ) 0 0 I T U Q Q T V C 1 (η t , ψ t ) 0 0 I χ t w t dt -E t f t 0 Lϑ(χ t , ξ t , η t , ψ t )dt (18)
where

Φ(ξ t , η t , ψ t ) = Θ(ξ t , η t , ψ t ) P(ξ t , η t , ψ t E(ξ t , η t , ψ t 0
From Dynkin's formula, we have

E ϑ(χ t f , ξ t f , η t f , ψ t f ) -ϑ(χ t 0 , ξ t 0 , η t 0 , ψ t 0 ) = E t f t 0 Lϑ(ξ t , η t , ψ t )dt (19)
Assuming, without loss of generality, that ϑ(χ t 0 , ξ t 0 , η t 0 , ψ t 0 ) = 0, it follows from ( 18) and ( 19) that if

Φ(ξ t , η t , ψ t ) + C 1 (η t , ψ t ) 0 0 I T U Q Q T V C 1 (η t , ψ t ) 0 0 I < 0 (20)
then ( 14) holds. Finally, by factorizing U ≥ 0 as

U = ∆Σ -1 ∆ T
and by Schur complement property, ( 20) is equivalent to [START_REF] De Souza | Mode-Independent H ∞ Filters for Hybrid Markov Linear Systems[END_REF]. Hence, the proof is complete.

The H ∞ output feedback fault tolerant control method to be developed in this paper is based on Lemma 1 and with the following parametrization of the Lyapunov matrices P ∞ijk :

P ∞ijk = M T k N -1 ∞ijk M k > 0 (21)
where N ∞ijk are symmetric positive definite matrices and M k are nonsingular matrices. This parametrization is inspired by the work of [START_REF] De Souza | Mode-Independent H ∞ Filters for Hybrid Markov Linear Systems[END_REF] on mode-independent H ∞ filtering for Markovian jumping linear systems. The next proposition presents the main result of this section, which is derived from proposition 3 with P ∞ijk as in [START_REF] Iwasaki | The dual iteration for fixed order control[END_REF] and using Finsler's lemma together with appropriate parametrization of matrices

M k and the dynamical controller matrices.

Proposition 5: If there exists matrices R k , S k , D k , X k , Y k , Z k and symmetric matrices N ∞ijk , i ∈ Z, j ∈ S and k ∈ R such that       Ω k + Ω T k A ijk -N ∞ijk δ ijk N ∞ijk 0 E T ijk -µ 2 I C ijk 0 0 -I Ξ ijk 0 0 0 -Γ ijk       < 0 (22)
where

Ω k = R k 0 S k + D k D k ( 23
)
A ijk = R k A i + R k B j Z k R k B j Z k S k A i + Y k C 2 + X k + S k B j Z k X k + S k B j Z k ( 24
)
E ijk = R k E ij S k E ij + Y k D 2ij (25)
C ijk = C ∞1 + D ∞1 Z k D ∞1 Z k ( 26
)
Γ ijk = diag { 1ijk , 2ijk , 3ijk , 4ijk } (27)                                                                  1ijk = N ∞1jk , . . . N ∞(i-1)jk , N ∞(i+1)jk , . . . N ∞zjk 2ijk = N ∞i1k , . . . N ∞i(j-1)k , N ∞i(j+1)k , . . . N ∞isk 3ijk = N ∞ij1 , . . . N ∞ij(k-1) , N ∞ij(k+1) , . . . N ∞ijr 4ijk = [N ∞ijk , . . . , N ∞ijk ] Ξ ijk = [ 1ijk , 2ijk , 3ijk , 4ijk ] T 1ijk = √ π i1 Ω T k , . . . , √ π i(i-1) Ω T k , √ π i(i+1) Ω T k , . . . , √ π iz Ω T k 2ijk = √ ν j1 Ω T k , . . . , √ ν j(j-1) Ω T k , √ ν j(j+1) Ω T k , . . . , √ ν js Ω T k 3ijk = λ ij k1 Ω T k , . . . , λ ij k(k-1) Ω T k , λ ij k(k+1) Ω T k , . . . , λ ij kr Ω T k 4ijk = Φ T 1ijk , . . . , Φ T vijk Φ lijk = R k W lij 0 S k W lij 0 δ ijk = -    h∈Z h =i π ih + l∈S l =j ν jl + v∈R v =k λ ij kv    (28)
and 4ijk , 4ijk contain v elements. Then, the system (3) is IESS and satisfies [START_REF] De Farias | Output Feedback Control of Markov Jump Linear Systems in Continuous-Time[END_REF]. Moreover, the transfer functions matrices of suitable controllers are given by

H k (s) = Z k sI -D -1 k X k -1 D -1 k Y k , ∀k ∈ R. ( 29
)
Proof: It will be shown that if the inequalities (22) hold, then the controllers (29) ensure that conditions (12) of proposition 3 are satisfied with a matrix P ∞ijk > 0 as in [START_REF] Iwasaki | The dual iteration for fixed order control[END_REF]. First, note that with a matrix P ∞ijk > 0 as in [START_REF] Iwasaki | The dual iteration for fixed order control[END_REF], the inequalities ( 12) are equivalent to

Π T ijk Υ ijk Π ijk < 0 (30)
where

Π ijk =       I 0 0 0 N -1 ∞ijk M k 0 0 0 0 I 0 0 0 0 I 0 0 0 0 I       (31)
Υ ijk =        0 0 C T ∞1jk R ∞ijk M k Λ ijk δ ijk N ∞ijk M k E ijk 0 0 -µ 2 I 0 0 -I 0 -S ∞ijk        (32)                                                R ∞ijk = [R ∞1ijk , R ∞2ijk , R ∞3ijk , R ∞4ijk ] R ∞1ijk = √ π i1 M T k , . . . , √ π i(i-1) M T k , √ π i(i+1) M T k , . . . , √ π iz M T k R ∞2ijk = √ ν j1 M T k , . . . , √ ν j(j-1) M T k , √ ν j(j+1) M T k , . . . , √ ν js M T k R ∞3ijk = λ ij k1 M T k , . . . , λ ij k(k-1) M T k , λ ij k(k+1) M T k , . . . , λ ij kr M T k R ∞4ijk = W T 1ij M T k , . . . , W T vij M T k S ∞ijk = diag [S ∞1ijk , S ∞2ijk , S ∞3ijk , S ∞4ijk ] S ∞1ijk = N ∞1jk , . . . N ∞(i-1)jk , N ∞(i+1)jk , . . . N ∞zjk S ∞2ijk = N ∞i1k , . . . N ∞i(j-1)k , N ∞i(j+1)k , . . . N ∞isk S ∞3ijk = N ∞ij1 , . . . N ∞ij(k-1) , N ∞ij(k+1) , . . . N ∞ijr S ∞4ijk = [N ∞ijk , . . . , N ∞ijk ] (33)
S ∞4ijk contains v elements.

Define n × n nonsingular matrices U k and V k such that U k V k = D k and let the matrix M k be parameterized as follows

M k = R -1 k R -1 k V T k -U -1 k S k R -1 k U -1 k V T k -U -1 k S k R -1 k V T k -1 (34)
Moreover, let the transformation matrix

T k = R T k S T k 0 U T k ( 35
)
Note that

M -T k T k = ג k = I 0 V k V k ( 36
)
and M k and T k as defined above are nonsingular. Indeed, from [START_REF] Ji | Controllability, stabilizability, and continuous-time Markovian jump linear quadratic control[END_REF] it follows that Ω k + Ω T k < 0, which implies that the matrices R k and D k are nonsingular. Therefore, in view of the definition of the matrices U k and V k , the matrices T k and M -T k T k are nonsingular and thus M -T k are nonsingular matrices as well. Next, introduce the matrices

F k = diag ג k , ג k , I, I, ג k ( 37
)
ג k = ג{ k , . . . , ג k } ( 38
)
where ג k contains ((z -1) + (s -1) + (r -1) + v) blocks ג k . Performing the congruence transformation

F T k (•)F k on Υ ijk , inequality (30) is equivalent to κ T F T k Υ ijk F k κ < 0, κ = F -1 k Π ijk ς, ς = 0 (39)
considering that

κ : κ = F -1 k Π ijk ς, ς = 0 = {κ : H ijk F k κ = 0, κ = 0} (40)
where

H ijk = M k -N ∞ijk 0 0 0 (41)
we have that ( 39) is equivalent to

κ T F T k Υ ijk F k κ < 0, ∀κ = 0 : H ijk F k κ = 0 (42)
By lemma 1, (42) holds iff the following inequalities are feasible for some matrices L ijk of appropriate dimensions

F T k Υ ijk F k + L ijk H ijk F k + F T k H T ijk L T ijk < 0 (43)
Without loss of generality, let the matrices L ijk be rewritten as

L ijk = F T k L ijk , then (43) is equivalent to F T k Υ ijk + L ijk H ijk + H T ijk L T ijk F k < 0 (44) Setting L ijk = I 0 0 0 0 (45) inequalities (44) become F T k Υ ijk F k < 0 (46)
where

Υ ijk =        M k + M T k M k Λ ijk -N T ∞ijk δ ijk N ∞ijk 0 E T ijk M T k -µ 2 I C ∞1jk 0 0 -I R T ∞ijk 0 0 0 -S ∞ijk        (47)
Consider the following state-space realization for the controllers ( 2)

A ck = V k D -1 k X k V -1 k , B ck = V k D -1 k Y k , C ck = Z k V -1 k ( 48
)
and let the matrix N ∞ijk be defined as

N ∞ijk = T T k M -1 k N ∞ijk M -T k T k ( 49
)
By performing straightforward matrix manipulations, it can be easily shown that

T T k Λ ijk M -T k T k = A ijk , T T k E ijk = E ijk , C ∞1jk M -T k T k = C ijk (50) T T k M -T k T k = Ω k , ג T k R ∞ijk ג k = Ξ T ijk (51) ג T k S ∞ijk ג k = Γ ijk (52)
Next, taking into account ( 37), ( 38) and ( 49)-( 52), it can be readily verified that (46) is identical to [START_REF] Ji | Controllability, stabilizability, and continuous-time Markovian jump linear quadratic control[END_REF]. Thus ( 12) is satisfied with

P ∞ijk = M T k N -1 ∞ijk M k .
Finally, the controller transfer matrix of ( 29) is readily obtained from (48).

From practical point of view, the controller that stochastically stabilizes the AFTCMP and at the same time guarantees the minimum disturbance rejection is of great interest. This controller can be obtained by solving the following optimization problem:

O ∞ :                            min τ >0, N ∞ijk =N T ∞ijk >0, R k , S k , D k , X k , Y k , Z k τ s.t :         Ω k + Ω T k A ijk -N ∞ijk δ ijk N ∞ijk 0 E T ijk -τ I C ijk 0 0 -I Ξ ijk 0 0 0 -Γ ijk         < 0
where the matrices inequalities in the constraints are obtained from ( 22) by replacing µ 2 by τ . This leads to the following Corollary:

Corollary 1: Let τ > 0, N ∞ijk = N T ∞ijk > 0, R k , S k , D k , X k , Y k , Z k
be the solution of the optimization problem O ∞ . Then, the controller (2) stochastically stabilizes the AFTCSMP we are considering and moreover the closed loop system satisfies the disturbance rejection of level √ τ . ♦

Remark 4: The above result can be easily extended to general SIQ constraints. This is illustrated by the following corollary:

Corollary 2: If there exists matrices R k , S k , D k , X k , Y k , Z k and symmetric matrices N ijk , i ∈ Z, j ∈ S and k ∈ R such that       Ω k + Ω T k A ijk -N ijk δ ijk N ijk C ijk E T ijk V C ijk 0 0 -Σ Ξ ijk 0 0 0 -Γ ijk       < 0 ( 53
)
where

C ijk = Q T C ijk C ijk = ∆ T C ijk
Then, the SIQ contraints ( 14) are verified. Moreover, the transfer functions matrices of suitable controllers are given by

H k (s) = Z k sI -D -1 k X k -1 D -1 k Y k , ∀k ∈ R. ( 54
) ♦ 4.2 H 2 Control
Before introducing the main results of this section, let us consider the following definition which represents a generalization of the H 2 -norm from Markovian jump linear systems [START_REF] Costa | Continuous-time state-feedback H 2 -control of Markovian jump linear systems via convex analysis[END_REF][START_REF] Dragan | Stability and Robust Stabilization to Linear Stochastic Systems Described by Differential Equations with Markov Jumping and Multiplicative White Noise[END_REF][START_REF] Dragan | H 2 Optimal Control for Linear Stochastic Systems[END_REF] to AFTCSMP. Let us consider the system (3) with

z t = z 2t = C 21 x t + D 21 (η t )u(y t , ψ t , t)
z 2t stands for the controlled output related to H 2 performance.

Definition 2:

We define the H 2 -norm of the IESS system (ϕ cl ) as

ϕ cl 2 2 = m d=1 i,j,k µ ijk z d,i,j,k 2 E 2
where z d,i,j,k represents the output {z t ; t ≥ 0} when: a) the input is given by w t = {w t ; t ≥ 0}, w t = e d δ t , δ t the unitary impulse, and e d the mdimensional unitary vector formed by 1 at the dth position and zero elsewhere; b) χ 0 = 0, ξ 0 = i, η 0 = j, ψ 0 = k and µ = (µ 111 , . . . , µ szr ) is the initial distribution of the joint Markov process.

From the definition above and using the same arguments as in [START_REF] Costa | Continuous-time state-feedback H 2 -control of Markovian jump linear systems via convex analysis[END_REF][START_REF] Dragan | Stability and Robust Stabilization to Linear Stochastic Systems Described by Differential Equations with Markov Jumping and Multiplicative White Noise[END_REF][START_REF] Dragan | H 2 Optimal Control for Linear Stochastic Systems[END_REF], we can state the following corollary.

Corollary 3: Assume that (ϕ cl ) is IESS then i) ϕ cl 2 2 = i,j,k µ ijk tr(E T ij P oijk E ij )
, where P o = {P o111 , . . . , P oszr } denotes the observability Gramian, i.e., P oijk are the unique positive semidefinite solutions of the following equations

ΛT ijk P oijk + P oijk Λijk + v l=1 W T lij P oijk W lij + h∈Z h =i π ih P hjk + l∈S l =j ν jl P ilk + v∈R v =k λ ij kv P ijv + C T 21 C 21 = 0 (55) ∀i ∈ Z, j ∈ S and k ∈ R. ii) ϕ cl 2 2 < i,j,k µ ijk tr(E T ij P 2ijk E ij )
, where P 2ijk is a positive definite solution of the following matrix inequality

ΛT ijk P 2ijk +P 2ijk Λijk + v l=1 W T lij P 2ijk W lij + h∈Z h =i π ih P 2hjk + l∈S l =j ν jl P 2ilk + v∈R v =k λ ij kv P 2ijv +C T 21 C 21 < 0 (56) ∀i ∈ Z, j ∈ S and k ∈ R.
iii) If there exists positive definite matrices P 2ijk , and matrices A ck , B ck and C ck such that i,j,k

µ ijk tr(E T ij P 2ijk E ij ) < γ 2 ΛT ijk P 2ijk +P 2ijk Λijk + v l=1 W T lij P 2ijk W lij + h∈Z h =i π ih P 2hjk + l∈S l =j ν jl P 2ilk + v∈R v =k λ ij kv P 2ijv +C T 21 C 21 < 0 ∀i ∈ Z, j ∈ S and k ∈ R.
Then ϕ dk are stabilizing controllers such that ϕ cl 2 < γ.

iv) The H 2 output feedback control problem is solved by the following optimization problem

               min i,j,k µ ijk tr(Z ijk ) s.t: E T ij P 2ijk E ij < Z ijk ΛT ijk P 2ijk + P 2ijk Λijk + v l=1 W T lij P 2ijk W lij + h∈Z h =i π ih P 2hjk + l∈S l =j ν jl P 2ilk + v∈R v =k λ ij kv P 2ijv + C T 21 C 21 < 0 (57)

♦

From definition 2 and corollary 3, we can state the following result which solves the H 2 output feedback control problem.

Proposition 6: The H 2 output feedback control problem is solved by the following optimization problem

O 2 :                                  min Z ijk , N 2ijk , R k , S k , D k , X k , Y k , Z k i,j,k µ ijk tr(Z ijk ) s.t : Z ijk E ijk N 2ijk > 0       Ω k + Ω T k A ijk -N 2ijk δ ijk N 2ijk 0 0 C ijk -I 0 Ξ ijk -Γ ijk       < 0 (58) ∀i ∈ Z, j ∈ S and k ∈ R. If (58) hold then ϕ cl 2 2 < i,j,k µ ijk tr(E T ij P 2ijk E ij ).
Moreover, the transfer functions matrices of suitable controllers are given by

H k (s) = Z k sI -D -1 k X k -1 D -1 k Y k , ∀k ∈ R. ( 59
)
Proof: The proof of this proposition follows the same arguments as for the proof of proposition 5.

Remark 5:

We can see from definition 2 that the considered H 2 -norm depends on the the initial distribution of the joint Markov process. However, these distributions are, in general, unknown. This inconvenient could be avoided by replacing, for example, the performance criteria by the one as in [START_REF] Hassibi | A path-following method for solving BMI problems in control[END_REF] in [START_REF] Dragan | H 2 Optimal Control for Linear Stochastic Systems[END_REF] (when the input is the unit variance white noise). Proposition 6 could be then easily applied for this H 2 -norm definition. The only difference is that, in the optimization problem (58), the initial distribution µ ijk is replaced by ε ijk = l,h,v p(lhv)(ijk) where

P = [p (lhv)(ijk) ] = lim t→∞ P (t)
and P (t) is the probability transition matrix of the joint Markov process. Indeed, the H 2 -norm definition considered in this case is given by

ϕ cl 2 2 = lim T →∞ i,j,k E T 0 z s 2 ds | ξ 0 = i, η 0 = j, ψ 0 = k
Using the same arguments as in [START_REF] Dragan | H 2 Optimal Control for Linear Stochastic Systems[END_REF], the following characterization of the H 2 -norm in terms of observability Gramian can be given:

ϕ cl 2 2 = i,j,k ε ijk tr(E T ij P oijk E ij ) (60)

Multi-Objective Synthesis

The multi-objective synthesis problem amounts to find common controllers that stochastically stabilize the system and ensure H 2 /H ∞ performances. This multi-performance synthesis problem can be stated as follows:

Given positive scalars α 2 and α ∞ , find stabilizing dynamic output feedback controllers ϕ d that solve the following constrained optimization problem

       min γ 2 ,γ ∞ ,A ck ,B ck ,C ck α 2 γ 2 + α ∞ γ ∞ s.t : z ∞ E 2 < γ ∞ w 2 , ϕ cl 2 < γ 2 (61)
The result is straightforward. It amounts to the collection of all related matrix inequality constraints.

Remark 6: Notice that the developed synthesis conditions are only sufficient. This is due to the fact that the controllers only depend on the FDI process, i.e., the number of controllers to be designed is less than the total number of the closed loop system modes by combining both failures an FDI processes.

Computational Issues and Example

The inequality conditions in propositions 5 and 6 are not linear in the variables and it is difficult to verify these conditions directly. However, the characterizations given in these propositions have the following nice properties:

i) The given parametrization enables us to express these norm minimization problems in closed form, i.e., all variables explicitly appear in constraints ( 22) and (58), which is not possible with projection-like conditions;

ii) The H 2 /H ∞ synthesis problem with fixed matrices C c (ψ t ) (synthesis problem similar to the one considered by [START_REF] Tokunaga | Multi-Objective Robust Control with Transient Specifications[END_REF] in the case of linear time invariant systems) is a LMI problem;

iii) With Z k fixed, conditions ( 22) and ( 58) are LMIs in

Z ijk , N 2ijk , N ∞ijk , R k , S k , D k , X k , Y k .
iV) With R k and S k fixed, conditions ( 22) and ( 58) are LMIs in

Z ijk , N 2ijk , N ∞ijk , D k , X k , Y k , Z k .
From the two last properties, one can see that the synthesis problem is expressed as a BMI (Bilinear Matrix Inequalities) problem. BMI problems are known to be generally nonconvex and NP-hard [START_REF] Toker | On the NP-hardness of solving bilinear matrix inequalities and simultaneous stabilization with static output feedback[END_REF]. This means that any algorithm which is guaranteed to find a global optimum cannot be expected to have a polynomial time complexity. There exists different approaches to the solution of this problem, which can be classified into global [START_REF] Fukuda | Branch-and-cut algorithms for the bilinear matrix inequality eigenvalue problem[END_REF][START_REF] Tuan | Low nonconvexity-rank bilinear matrix inequalities: Algorithms and applications in robust controller and structure designs[END_REF][START_REF] Tuan | A new Lagrangian dual global optimization algorithm for solving bilinear matrix inequalities[END_REF] and local [START_REF] Ibaraki | Rank minimization approach for solving BMI problems with random search[END_REF][START_REF] Iwasaki | The dual iteration for fixed order control[END_REF][START_REF] Hassibi | A path-following method for solving BMI problems in control[END_REF]. Most of the global algorithms to the BMI problem are varaiations of of the Branch and Bound algorithm [START_REF] Fukuda | Branch-and-cut algorithms for the bilinear matrix inequality eigenvalue problem[END_REF][START_REF] Tuan | Low nonconvexity-rank bilinear matrix inequalities: Algorithms and applications in robust controller and structure designs[END_REF]. Although the major focus of global search algorithms is the computational complexity, none of them is polynomial time due to the NP-hardness of the problem. As a result, these approaches can currently be applied only to problems with modest size. Most of the existing local approaches, on the other hand, are computationally fast but, depending on the initial condition, may not converge to the global optimum. The simplest local approach makes use of the fact that by fixing some of the variables, x, the BMI problem becomes convex in the remaining variables y, and vice versa, and iterates between them [START_REF] Iwasaki | The dual iteration for fixed order control[END_REF]. The algorithm (coordinate descent-type algorithm) used in this paper belongs to this class of methods. Nevertheless, these types of algorithms, called coordinate descent methods in [START_REF] Iwasaki | The dual iteration for fixed order control[END_REF], alternating SDP method in [START_REF] Fukuda | Branch-and-cut algorithms for the bilinear matrix inequality eigenvalue problem[END_REF], and the dual iteration in [START_REF] Iwasaki | The dual iteration for fixed order control[END_REF], are not guaranteed to converge to a local solution [START_REF] Fukuda | Branch-and-cut algorithms for the bilinear matrix inequality eigenvalue problem[END_REF]. Such an algorithm is given by Algorithm 1. For the purposes of its initialization, we adopt the same methodology as in [START_REF] Kanev | Robust Output-Feedback Controller Design via Local BMI Optimization[END_REF]. These can be summarized as follows:

Algorithm 1.

• Step 0. Initialization: Set q = 0. Design state feedback matrices Z kq that solves the following optimization problem

       min γ 2q ,γ ∞q ,Z kq α 2 γ 2q + α ∞ γ ∞q s.t : z ∞ E 2 < γ ∞q w 2 , ϕ cls 2 < γ 2q (62)
where ϕ cls is the closed loop system obtained by applying the controller ϕ s given by: ϕ s : u t = Z kq x t to the system (1).

• Step 1. Fix the matrices Z kq and search for a solution to the multi-objective control problem, defined in equation ( 60), in terms of the remaining unknown matrices Z ijkq , N 2ijkq , N ∞ijkq , R kq , S kq , D kq , X kq , Y kq . Set the current cost value υ q = α 2 γ 2q + α ∞ γ ∞q .

• Step 2. Fix the matrices R kq and S kq and search for a solution to the multi-objective control problem, defined in equation ( 60), in terms of the remaining unknown matrices Z ijkq , N 2ijkq , N ∞ijkq , D kq , X kq , Y kq , Z kq . Set the current cost value ω q = α 2 γ 2q + α ∞ γ ∞q .

• Step 3. If υ q -ω q < , > 0, Stop. Otherwise, set q ← q + 1 and go back to Step 1.

As in the usual coordinate descent methods, the above algorithm generates a non-increasing sequence of the objective function values, and thus the convergence is guaranteed. Note, however, that the limit of the sequence may not be optimal.

A VTOL Example

In this section, the proposed multi-objective dynamic output feedback control of AFTCSMP is illustrated using a VTOL helicopter model adapted from [START_REF] De Farias | Output Feedback Control of Markov Jump Linear Systems in Continuous-Time[END_REF]. Consider the nominal system with

    W 1 =     0.1 0 0 0 0 0.1 0 0 0 0 0.1 0 0 0 0 0.1     , C 2 = 1 0 0 0 0 1 0 0 , D 2 = 0.1 0 0 0.1 , C∞1 = 0 1 0 0 0 0 0 1 , D∞1 = 1 0 0 1 , C21 = 1 0 0 0 0 0 1 0 , D21 = 0 0 0 1 .
The state vector x t ∈ R 4 is composed by the following:

x 1 : longitudinal velocity;

x 2 : vertical velocity;

x 3 : rate of pitch;

x 4 : pitch angle.

and the components of command vector are:

u 1 : general cyclic command; u 2 : longitudinal cyclic command.
For illustration purposes, we will consider two faulty modes: i) Mode 2: A 50% power loss on the first actuator;

ii) Mode 3: A 50% power loss on both actuators.

From above, we have that S = {1, 2, 3}, where the mode 1 represents the nominal case. The failure process is assumed to have Markovian transition characteristics. The FDI process is also Markovian with three states R = {1, 2, 3}. The actuator failure rates are assumed to be: The state trajectories of the closed loop system resulting from the obtained controllers are shown in Figure 3. These trajectories represent a single sample path simulation corresponding to a realization of the failure process η t and the FDI process ψ t given by Figure 1 and Figure 2 respectively. Figure 4 represents the evolution of the controlled outputs z ∞t . It can be seen that the closed-loop system is stochastically stable and that the disturbance attenuation is achieved.

[π ij ] =   -0.

Magnitude

x t [START_REF] Aberkane | Output Feedback Stochastic Stabilization of Active Fault Tolerant Control Systems[END_REF] x t [START_REF] Aberkane | H ∞ Stochastic Stabilization of Active Fault Tolerant Control Systems: Convex Approach[END_REF] x t [START_REF] Aberkane | Output Feedback H ∞ Control of a Class of Stochastic Hybrid Systems with Wiener Process via Convex Analysis[END_REF] x t (4)

Conclusion

In this paper, the dynamic output feedback multi-objective control of continuous time AFTCSMP was considered within a framework that allows to take into account the problematic resulting from the fact that the controller only depends on the FDI process. The specifications and objectives considered, include stochastic stability, H 2 and H ∞ performances. The main results were derived using a version of the well known Finsler's lemma and a parametrization of the Lyapunov matrices. The numerical resolution of the obtained results was done using a coordinate descent-type algorithm. The effectiveness of the developed method was illustrated on a VTOL helicopter example.

Figure 1 :

 1 Figure 1: Failure modes

  Figure 2: FDI modes

Figure 3 :

 3 Figure 3: State variables evolution: single sample path simulation

Figure 4 :

 4 Figure 4: Evolution of the variables z ∞t : single sample path simulation

  =i π ih P hjk + Proof: Let us consider the following quadratic Lyapunov function ϑ(χ t , ξ t , η t , ψ t ) = χ T t P(ξ t , η t , ψ t )χ t , η t , ψ t ) P(ξ t , η t , ψ t E(ξ t , η t , ψ t 0

					l∈S	ν jl P ilk +	v∈R	λ ij kv P ijv
					l =j		v =k
	then
					J SIQC < 0
							(16)
	then:	Lϑ(χ t , ξ t , η t , ψ t ) =	χ t w t	T	Θ(ξ t χ t w t	(17)
	adding and subtracting E

t f t 0 Lϑ(χ t , ξ t , η t , ψ t )dt to J SIQC , we get

  For the above AFTCSMP, and using Algorithm 1 with α 2 = α ∞ = 1, we obtain the following H 2 /H ∞ performances from w t to z 2 and z ∞ respectively: γ 2 = 1.3813, γ ∞ = 7.0430. The corresponding dynamical controllers are given as follows:

	The FDI conditional transition rates are: [λ 1 ij ] =   -0.02 0.01 0.01 1.00 -1.01 0.01   , [λ 2 ij ] =  	-1.01 0.01	1.00 -0.02	0.01 0.01	  , [λ 3 ij ] =	 	-1.01 0.01	0.01 -1.01	1.00 1.00	  .
	1.00	0.01	-1.01		0.01	1.00	-1.01				0.01	0.01	-0.02
	       	-4.5605 1.2505 -0.8586 1.7167 -0.5951 -0.1871 6.1141 -13.949 9.0361 0.8173 0.0519 0.8206	-0.7661 -2.0089 2.1057 2.8241 -3.6110 -6.1882 0.8173 0.8173 0.3333 0.4416 -0.6220 -1.5290	4.6913 -2.0707 10.8691 -4.9146 4.1034 -6.6348 -1.8753 -1.2901 0	       	ψ t =1
	               	-1.8554 -0.8810 -0.0395 -0.5837 -2.7850 -8.6549 4.1078 3.8934 0.2180 2.2321 -2.8481 -4.2791 1.3433 3.3313 1.1581 0.1551 -0.6729 0.0591 0.1496 0.2176 2.4528 1.4950 -2.1517 -3.1308 -2.0190 -0.9464 0.0742 -0.4221 -0.7222 -6.3567 2.2273 1.2006 -0.5881 0.9657 -2.1838 -3.2807 1.3238 3.2111 1.2362 0.1852 -0.7787 0.0063 0.2350 0.3908 1.8829 1.2999 -1.9038 -3.0831	1.6635 -1.7314 3.0771 -1.4002 -3.4934 0.9851 5.2828 -1.2354 0 1.7047 0.9938 -1.3945 4.5383 2.9401 -0.5600 -1.3741 -3.4574 0	               	ψ t =2 ψ t =3
	where
						0
					002 0.0010 0.0010 0.0010 -0.002 0.0010	 
					0.0010 0.0010 -0.002

ψt=i is a realization of the controller (ϕ d ) for ψ t = i.


Synthesis of Complex Humanoid Whole-Body Behavior: a Focus on Sequencing and Tasks Transitions

Joseph Salini, Vincent Padois and Philippe Bidaud

Abstract-We present a novel approach to deal with transitions while performing a sequence of dynamic tasks with a humanoid robot. The simultaneous achievement of several tasks cannot be ensured, so we use a strategy based on weights to represent their relative importance. The robot interacts with a changing environment, and the input torques are different depending on whether the robot performs tasks in a constrained state (e.g. in contact) or not. We develop a solution with smooth weights variations and transitional tasks which avoids sharp torque evolutions. In order to validate this approach, simulations are carried out on a virtual iCub robot which is assigned the realization of a complex mission involving various changing tasks.

I. INTRODUCTION

Challenges in the control of complex robotic systems are evolving from the derivation of efficient controllers dedicated to elementary tasks such as walking ( [START_REF] Kajita | Biped walking pattern generation by using preview control of zero-moment point[END_REF], [START_REF] Dimitrov | An optimized linear model predictive control solver for online walking motion generation[END_REF]) or multiple contact control ( [START_REF] Park | Robot multiple contact control[END_REF]) to the blending of such controllers within higher level control architectures for dealing with complex activities.

These new control architectures are expected to provide complex robotic systems with the ability to reactively perform sequences of complex tasks in unstructured environments and in dynamically changing contexts where unexpected physical interactions may occur between the robot and its environment. Besides, achieving complex behaviors may also lead to perform several, potentially antagonistic tasks at the same time, while having critical importance.

Humanoid robots (physical ones or avatars) are good examples of such complex systems. As a matter of fact the scientific literature in this domain recently proposed control architectures to tackle these new challenges.

Among this literature, the work of Sentis et al. ( [START_REF] Sentis | Whole-body dynamic behaviour and control of human-like robots[END_REF], [START_REF] Sentis | Compliant control of multi-contact and center of mass behaviors in humanoid robots[END_REF]) provides analytical methods to solve the humanoid control problem in cases where walking and compliant reaching tasks are involved while constraints such as equilibrium, collision and joint limits avoidance have to be satisfied. Tasks are hierarchically achieved and the proposed framework fully takes into account dynamic couplings. While elegant and physically meaningful, this type of method leads to a rather complex problem formulation which induces an extensive use of computationally expensive inversion techniques. Also this approach supposes the definition of a tasks hierarchy All authors are with: Université Pierre et Marie Curie Institut des Systèmes Intelligents et de Robotique -CNRS UMR 7222 Pyramide Tour 55, Boite Courrier 173 4 Place Jussieu F-75005 Paris email: {salini,padois,bidaud}@isir.upmc.fr and, most importantly, it does not allow to account for unilateral constraints (contacts for example) in a straightforward way.

In order to obtain more generic control problem formulations as well as to take unilateral constraints into account simply, optimization techniques have recently been used extensively in the robotic context. More specifically, Linear Quadratic Programming (LQP) has been used in different contexts. For example, Abe et al. ( [START_REF] Abe | Multiobjective control with frictional contacts[END_REF]) make use of LQP in the framework of animation of humanoid avatars achieving various tasks. Colette et al. ( [START_REF] Collette | Robust balance optimization control of humanoid robots with multiple non coplanar grasps and frictional contacts[END_REF]) pursue the same goal, using similar tools with a focus on robust balancing. The same kind of approach is being used in the work of Escande et al. ( [START_REF] Escande | Fast resolution of hierarchized inverse kinematics with inequality constraints[END_REF]) at the inverse velocity kinematics level with efficient implementation on a humanoid robot in mind and allowing to enforce priorities both at the tasks and constraints levels. Finally, even though not applied to the humanoid case directly, the work of Decré et al ( [START_REF] Decré | Extending itasc to support inequality constraints and non instantaneous task specification[END_REF]) makes use of cascading LQPs to formulate complex control problems with priorities between tasks as well as unilateral constraints.

Even though very promising, the optimization approach to control as it is implemented in these recent contributions still lacks two interesting features. The first one is the possibility to resort to soft hierarchies of tasks. Indeed, tasks hierarchy is of high interest but it is not always simple to define strict priorities between tasks especially in highly varying contexts where there is no guarantee that all tasks can be achieved simultaneously. Secondly, keeping the implementation of these controllers on real humanoid robots in mind, control input smoothing and saturation is very important and sequences of tasks of different nature (contact → free-space is one example) should ensure soft transitions, being them planned or reactively activated.

As a tentative to propose a control framework addressing these two challenges, the work presented in this paper is an extension of our previous work ( [START_REF] Salini | LQP-Based Controller Design for Humanoid Whole-Body Motion[END_REF]). We propose the derivation of a humanoid robot controller based on LQP that allows the description of pre-planned tasks sequences including transition conditions and authorizing the reactive insertion of new tasks to ensure a proper behavior of the system in dynamically changing environments. Several tasks can be specified simultaneously and achieved based on a soft hierarchy implemented through a generalized task weighting strategy. This weighting strategy is also used when a transition occurs in order to ensure smoothed, realistic control inputs.

The paper is organized as follows. In the first section, the formulation of control objectives in terms of operational accelerations and forces is described and task servoing is discussed. Higher level tasks are are also introduced among which impedance and predictive ZMP control. Then, in the second section, the retained constraints are presented and an emphasis is put on unilateral contact constraints. The third section synthesizes the LQP controller based on the chosen tasks and constraints formulation. Results on a simulated iCub robot are then proposed in the fourth section through the presentation of different scenarii. These results are discussed in conclusion where some insights on possible future work are also given.

II. CONTROL OBJECTIVES

A humanoid robot can be modeled as an under-actuated tree-structure composed of rigid bodies linked together with revolute joints. The equation of motion of such a system can be derived from the Euler-Lagrange formalism M (q)q + N (q, q) = g(q) + Sτ + J c (q) t w c

(1

)
where q, q, q are respectively the generalized coordinates, velocity and acceleration vectors, M, N, g the generalized inertia matrix, the Coriolis and non-linear effects matrix and the gravity vector, S, τ the actuation matrix and the input torque vector, and finally J c , w c the contact points Jacobian and the contact wrench. S allows to account for the fact that some degrees of freedom are not actuated (typically the free-floating base). This equation is nonlinear and generally linearized around the state (q, q) and M, N, g, S, J c are thus supposed to be known. As a matter of fact, a task can either be described at the acceleration level (q) or at the wrench level (w c ), τ being the control variable of the system. In the remaining of this paper, χ = [q, w c , τ ] T is called the action variable of the system.

A. Generic Task Definition

A task can be defined as the servoing of a specific 3D frame attached to the robot body to a desired goal value. This control objective can either be described in terms of twist (denoted t) and/or wrench ( [START_REF] Duindam | Port-based modeling and control for efficient bipedal walking robots[END_REF]). In the case of a twist (velocity) task, one needs to relate t to q which can be done using the task jacobian J ṫ = J(q)q + J(q, q) q.

(2) It can also be described in term of a wrench and one needs to relate w to τ which can similarly be done using the dual relation

τ = J(q) t w. (3)
Generally, the achievement of the task cannot be guaranteed because of the constraints acting on the system. This leads to define a task as an error to minimize with respect to a desired goal. Thus, a task can be written as

min x des -x (4)
where x can represent a set of accelerations ṫ and/or wrenches w and the superscript des stands for desired.

This formulation allows to describe cartesian space tasks: reaching, obstacle avoidance, Center of Mass (CoM) control, contact force control, etc. Joint space tasks (posture control, joint limit avoidance, torque minimization, etc.) can easily be described using the same formalism, the corresponding jacobian matrices becoming trivial. This multiplicity of tasks is illustrated on Figure 1.

B. Task Servoing

In order to ensure some robustness with respect to the robot model, as well as to enforce a given task behavior (e.g. a virtual spring-damper system), a controller can be derived at the task level. When dealing with twists during a tracking task, this type of controller is often written

ṫdes = ṫgoal + K p p + K d ˙ p ( 5
)
where p is the pose error which can be trivially computed when dealing with position and requires to resort to a nonminimal representation of the orientation such as quaternions ( [START_REF] Altmann | Rotations, Quaternions and double groups[END_REF]). Reaching is a particular case of the tracking task where the goal trajectory is reduced to one point and ṫgoal = t goal = 0.

An equivalent wrench formulation leads to

w des = w goal + K f p w + K f i w dt ( 6
)
where w is the wrench error.

C. Higher level tasks 1) Impedance Controller: Impedance control is a way to unify twist and wrench control ( [START_REF] Hogan | Stable execution of contact tasks using impedance control[END_REF]). Instead of directly controlling forces or accelerations, impedance control rather aims at inducing a proper behavior of the system with respect to contacts with its environment. A typical approach consists in setting a reference position to a frame attached to the robot and to control the reaction wrench induced by a disturbance.

It allows the use of a unified controller for non-constrained and constrained motions, e.g. when in contact.

A simple impedance controller can be modeled using a spring-damper system linking the controlled part of the system to a virtual reference. Hence, the choice of the reference position, the stiffness and damping coefficients determines the behavior of the reaction which may change, for instance to speed up the completion of the task, or to increase the contact wrench of the end-effector at contact point.

2) Zero Moment Point Controller: An approximation of the Zero Moment Point (ZMP) ( [START_REF] Kajita | Biped walking pattern generation by using preview control of zero-moment point[END_REF]) can be controlled in order to perform a walking or a balancing task, through the control of the Center of Mass (CoM). It is computed as follow c = J com (q)q + Jcom (q) q

z = c - h g c (7)
where J com is the jacobian of the CoM, z, c are respectively the horizontal position of the ZMP and of the CoM, h is the height of the CoM and g is the gravity value. The direct control of z may be unstable, so an approach which takes into account the prediction of the future position must be implemented. As described in [START_REF] Wieber | Trajectory free linear model predictive control for stable walking in the presence of strong perturbations[END_REF], the initial configuration of the CoM is denoted C 0 = [c, ċ, c] T and the prediction of the ZMP along a time horizon

H is denoted Z H = [z 1 , z 2 , ..., z H ] T .
Z H = P x C 0 + P u ... CH
The goal is to minimize the norm of the difference between a ZMP reference (Z ref computed a priori) and the predicted trajectory. This minimization problem is unconstrained, so it can be solved directly, which gives ...

c H = -(P T u .P u + R.I N ) -1 .P T u .(P x .X com -Z ref )
where R is a ratio between the state and the command error, and I N is the identity matrix. The first row of this input matrix is used to control the CoM and ZMP trajectories.

III. CONSTRAINTS

The tasks completion is bounded by the environment of the robot and by its own technological limits. This section presents the major constraints faced by the system, but this list is not exhaustive and can be extended with custom constraints.

A. Equation of motion

The first constraint is the one due to the law of physics which imposes the evolution of χ according to equation [START_REF] Kajita | Biped walking pattern generation by using preview control of zero-moment point[END_REF].

B. Technological limits

The characteristics of the actuators bound their range of action. Generally, the range of motion, velocity, acceleration and torque are bounded

τ min ≤ τ ≤ τ max (8) qmin ≤ q ≤ qmax (9) qmin ≤ q ≤ qmax (10) q min ≤ q ≤ q max (11)
In order to describe all constraints in terms of χ, the action variable of the system, constraints [START_REF] Salini | LQP-Based Controller Design for Humanoid Whole-Body Motion[END_REF] and [START_REF] Duindam | Port-based modeling and control for efficient bipedal walking robots[END_REF] are respectively replaced by

qmin ≤ q + qh 2 ≤ qmax (12) q min ≤ q + qh 1 + q(h 1 ) 2 /2 ≤ q max ( 13
)
where h 1 and h 2 are anticipation coefficients set to predict the future value of the state (q, q) given the generalized acceleration q of the system.

C. Interaction 1) Frictional Contact:

The robot interacts with its environment through a set of contact points linked to its bodies. Restricting, for simplicity, twists and wrenches to their linear velocities and forces components respectively, each contact point i can be described by its linear velocity v ci = J ci q ∈ R 3 where J ci is the jacobian 1 of the contact point and by its force denoted f ci ∈ R 3 . Several types of interaction can be imagined such as frictional contact, ball and socket, sliding, etc. Focusing on frictional contacts, four cases have to be distinguished: c 1 the contact is persistent, v ci = 0 and f ci lies inside the Coulomb cone; c 2 the contact is lifting, v ci .n > 0 and f ci = 0; c 3 there is no contact, v ci ∈ R 3 and f ci = 0; c 4 the contact is sliding v ci × n = 0 and f ci = 0; where n is the normal vector of contact.

The contact force can be decomposed into two components, a normal force f n ci = f ci • n and a tangential force

f t ci = f ci -(f ci •n)n.
Hence, the Coulomb cone constraint is written:

f t ci ≤ µ i f n ci ( 14
)
where µ i is the friction coefficient of contact i. A polygonal approximation of the cone, lying in the real Coulomb cone, may be used to linearize equation [START_REF] Wieber | Trajectory free linear model predictive control for stable walking in the presence of strong perturbations[END_REF]. Given C f i the representation of this linearized cone, the inequality becomes

C f i f ci ≤ 0 ∀i. ( 15
)
Within the framework of this work, only the two first cases are treated. The third one is trivially taken into account without writing any specific constraint equation. The fourth one is out of scope for this work.

2) Kinematic-Loop: A kinematic loop links two points in the tree-structure of the system, which results in the following equality on their respective joint velocities. Thus, considering each kinematic loop j, we have

(J j1 -J j2 ) q = 0 ∀j. (16)
Again, these equations do not directly depend on the action variable χ described in section II. Rewriting these equations, we get the following interaction constraints equations

c 1 v cip = 0 : J cip q + Jcip q = 0 C f ip f cip ≤ 0 ∀i p (17)
c 2 v ci l .n > 0 : J ci l q + Jci l q.n > 0 f ci l = 0 ∀i l (18) (J j1 -J j2 )q + ( Jj1 -Jj2 ) q = 0 ∀j ( 19
)
where i p and i l respectively describe persistent and lifting contacts.

IV. OPTIMIZATION PROBLEM FORMULATION

Tasks and constraints being formalized as a set of functions to minimize and as a set of inequalities respectively, optimization techniques offer a strong and very general mathematical framework to solve the control problem. Given the quadratic form of the tasks and the linearity of the constraints described in sections II and III, Linear Quadratic Program (LQP) is a good candidate, especially because of its computational efficiency. While computationally more costly, Second Order Cone Program (SOCP) could also be used to handle constraints related to the friction cones without simplification. The implementation of LQP done in this work could easily be converted into a SOCP ( [START_REF] Park | Convex optimization algorithms for active balancing of humanoid robots[END_REF]).

A. Generic LQP

A LQP solves the following problem

min (χ) 1 2 χ T P χ + p T χ (20)
s.t. : Gχ h Aχ = b
where χ is the vector to optimize, P, p represent the quadratic cost function, G, h define the inequality constraints, and A, b define the equality constraints.

As mentioned in section II, tasks are described as a norm to minimize which is a function of q , w c or τ , depending on their nature. Having introduced χ as the concatenation of these three variables, one can, without loss of generality formulate any task k in the following manner

T k (χ) = E k χ + f k 2 (21)
where E k and f k are properly computed to represent the respective error to minimize, and the norm is squared to fit the LQP. For instance, considering the case of task inducing the control of the acceleration of one end-effector (EF) of the robot, we have E = [J EF 0 0] and f EF = JEF qqdes .

As for the constraints, they are given by equations ( 1), ( 8), ( 9), ( 12), ( 13), ( 17), ( 18) and [START_REF] Barthélemy | Arboris for Matlab[END_REF]. It is important to notice that these constraints may not always be compatible inducing an empty set of solutions. This is not acceptable in the context of real time robot control. To solve this difficulty, a first strategy would require to resort to constraints relaxation. This does not really make sense given the type of problem solved. A second strategy consists in rewriting the constraints set in order to ensure compatibility and thus the existence of at least one solution. This second strategy was not implemented within the framework of this work but interested reader can refer to the recent work of Rubrecht et al. ( [START_REF] Rubrecht | Constraints Compliant Control: constraints compatibility and the displaced configuration approach[END_REF]) who treat this problem at the velocity kinematics level.

B. Tasks priorities: Hierarchy vs Weighting

When two or more tasks have to be performed at the same time, conflicting situations may occur. Some tasks must have more importance to ensure their achievement at the expenses of other ones. Two strategies are possible which are either based on a strict hierarchy or relative weighting between tasks. Before giving more details about these two approaches, it is important to notice here that in both cases, tasks may not be compatible with the constraints or tasks may be incompatible one another, in which case they cannot be achieved perfectly.

1) Hierarchy: To enforce such priorities, the most common strategy is to resort to strict hierarchies of tasks ( [START_REF] Kanoun | Prioritizing linear equality and inequality systems: Application to local motion planning for redundant robots[END_REF]). A strict hierarchy is an ordered list of tasks where the lower ones cannot interfere with the higher ones. This solution is relatively straightforward and just requires relative orders. From an algorithmic point of view, strict hierarchies require the projection of low priority tasks jacobians in the null-space of higher priority tasks jacobians.

In this case, the optimization problem is solved recursively. Given a ordered set of n tasks T i (χ) = E i χ + f i 2 , the algorithm becomes for i=1..n:

min (χ) 1 2 (T i (χ)) 2 + (w 0 T 0 ) 2 s.t. : Gχ h A i χ = b i A i+1 ← A i ∪ E i b i+1 ← b i ∪ (E i χ * i ) where χ * i is the solution of program i, A 1 = A, b 1 =
b and T 0 is a task dedicated to the minimization of the whole optimization variable. This is required when the control problem has many solutions in order to ensure uniqueness and more specifically to provide a solution that minimizes accelerations, contact forces and actuation torques. This task is given, using a weight w 0 , a very small importance compared to others.

While providing convincing results, this priority strategy has several drawbacks. The first one lies in the fact that this algorithm solves as many LQP as the number of tasks per call which increases the overall computation time. Moreover the extension of the constraints set at each step increases the risks of not finding any solution including the complete set of tasks. In other words, low priority tasks may not be achieved at all. The second disadvantage of this method is that it requires to define relative orders between tasks which is never obvious in most robotics contexts. Moreover, this ordered list is not always feasible (e.g. a loop in the relative orders), and a transition or a replacement in the stack of tasks may involve discontinuities in the control signal, which must be avoided on real systems.

2) Weighting: The approach retained in this paper relies on weights. This weighting strategy associates each task with a coefficient thats sets its importance with respect to others (a task with a very high relative weight gets the highest priority). As a consequence priorities are not strict and all tasks are achieved according to the trade-off defined by the weights.

Given a set of tasks with their relative weights

(T i , w i ) i ∈ [1.
.n], the LQP becomes:

min (χ) 1 2 i ((w i .T i (χ)) 2 ) + (w 0 T 0 ) 2 s.t. : Gχ h Aχ = b
where w 0 w i ∀i. This program is solved only one time per call and the time of computation depends mainly on the dimension of the set of constraints. Practically speaking, a task can be considered much more important if its weight is ten times higher than the weights of other tasks.

Even though defining the value of the weights for all tasks may not be a simple problem, it leaves more flexibility and lower priority tasks are never completely abandoned even in the worst cases. Also, similarly to the previous strategy, if an exact solution exists (i.e. a solution that minimizes exactly all tasks), it will be found, i.e. all tasks will be perfectly achieved. Finally, this method allows to change the weights continuously in order to generate smooth transitions. This is an important feature when facing the implementation on a real system, especially when dealing with complex robotic missions where tasks can be reactively activated based on the environment dynamics.

3) Transitions: The knowledge of the past solution is not given to the optimization program, so discontinuities may happen if two successive problems are not close enough. Sharp transitions in χ are obviously not desirable at least from a practical point of view (e.g. a sharp transition in terms of contact wrench w c would either mean a sudden lost of support or a strong impact).

These discontinuities are very likely to occur in the case where the relative importance of tasks is suddenly modified or when constraints appear or disappear. To deal with such discontinuities, one can

• add a constraint related to the continuity of the control signal (e.g. bound the values of the derivative of the action variable) -this possibility has not been implemented in this paper;

• ensure continuity using a properly weighted pseudo inverse jacobian as proposed in [START_REF] Mansard | Continuity of varying-feature-set control laws[END_REF] -this is not possible when using optimization-based control approaches (e.g. LQP) since no inverse is computed explicitly; • ensure continuity using a continuous variation of the weights associated to each task -this is the approach retained in this paper. Indeed, continuous change of the tasks weights should imply smooth transition. Besides, a planned transition from a constraint state to another one (for instance the lost of support of a chair when standing) can be achieved with the use of a transitional task whose weight gains importance during a short period of time.

Of course, the choice of the tasks weights and their evolutions should be automated to generate an efficient generic whole-body motion controller. Here, the authors use a relatively sequential approach. Each task has triggers for its start and its end. A supervisor evaluates its completeness and modifies its weight according to the mission context and with respect to the criticality of the other tasks2 . As a matter of fact, the tasks weights can be modified automatically without requiring any complex tuning where w prev and w next are the weights value of task i respectively for the previous and the next mission contexts, t e and t s are the trigger time respectively for the end of the previous mission context and the start of the next one. The time between t e and t s is thus naturally defined as the transition time.

w i =      w prev , t
This should lead to a good overall behavior even if complex cases.

V. EXPERIMENTS & RESULTS

The experiments have been carried out using Arboris-Python ( [START_REF] Barthélemy | Arboris for Matlab[END_REF]), an open-source dynamic simulator developed at ISIR with the Python programming language. The implementation of LQP relies on CVXOPT/CVXMOD, two free Python packages dedicated to convex optimization ( [START_REF] Dahl | -python software for convex optimization[END_REF]). The simulated robot is a rather accurate model of the iCub robot which is present at ISIR in a version that does not allow torque control ( [START_REF] Sandini | The icub cognitive humanoid robot: An open-system research platform for enactive cognition[END_REF]). 38 degrees of freedom are simulated (32 + 6 floating joints to locate the root in space), and 4 contact points at each foot, as shown in Figure 1.

A. Impedance Controller

As recalled in section II, impedance control provides a mean to handle the interaction with a changing environment: adapt to disturbances (e.g. an impact) or anticipate reactions (e.g. to push, grab, etc.).

In this simulation, iCub stands in front of a moving table which prints vertical oscillating motion with a period of 3 s. The robot puts its hands on the table and applies a pressure using an impedance controller. Each hand has a reaching goal 10 cm below the point of contact, and a virtual springdamper system linking the hand to its goal maintain the pressure on the table (stiffness and damping are respectively set to 10 s -2 and 2.

√ 10 s -1 ). The robot keeps the pressure with its hands and the torques induced by this simulation have no discontinuity as shown on Figure 3. Fig. 3. An iCub robot applies a pressure on a moving table using an impedance controller.

B. Sequence of tasks

A complex scenario composed of a sequence of various complex tasks has been developed. This scenario describes iCub standing up from a chair, walking toward a table, grabbing a box and moving it to an other position, as shown on Figure 2. Using their respective weights, consecutive tasks are blended during a short period of time in order to ensure smooth transitions. For example, the walking task starts just before the end of the standing up task, and the same happens with the grabbing task. The mission is globally achieved with a good performance level (cf. the video submission attached to this paper) and rather than describing the entire mission in details, we hereafter focus on specific issues.

1) Hierarchy vs Weighting: In order to compare the weighting and hierarchy priority rules, the grabbing part of the scenario is performed using both strategies. In both cases four tasks (COM, hands, back, posture control) are used and associated either to strict priority values or to weights. Given the chosen tasks, the robot needs to bend over to achieve the box grabbing and this requires a modification of the relative tasks importances. This change is illustrated on Figure 4. The resulting input torso torques are shown on Figures 5 and6. This result illustrates the positive impact of using smooth transitions in tasks priorities on the continuity of the control input. 2) Transition between constrained and free motions: We focus here on the sequencing of tasks inducing a transition between a constraint and a non-constraint state. The retained example is the standing behavior which leads to the disappearance of constraints when contacts vanish. A standing strategy involving a transition task is compared to a strategy where no transition is planned.

The input torques of the robot with no transition between constraint and non-constraint states is shown on Figure 7. The discontinuities are very important, and such a variation cannot be handled with a real system.

To cope with this issue, the constraints of the considered contacts are transformed into a transition task and then smoothly removed. In other words, if a contact disappears, a force control task is created with low but increasing priority over a short period of time, until the force becomes null. At this time, the constraint on the contact point acceleration must be replaced by an acceleration control task with a high but smoothly decreasing importance over a short period of time. If a contact appears, this set of operations is applied backward. The resulting input control torque is shown on Figure 8. Here again, the smooth insertion of transition tasks ensure smooth control inputs.

VI. CONCLUSIONS AND FUTURE WORKS

A. Conclusions

In this paper, we present a method to perform a sequence of dynamic tasks. We use an optimization program to solve the problem of computing the proper torque control given a set of tasks subject to a set of equality and inequality constraints.

The main contribution of this paper is the development of a strategy to define soft priorities between tasks that have to be achieved simultaneously. The evolution of the tasks importance is ensured with weights and allows to easily perform smooth sequencing between changing tasks and, as a consequence, to synthesize complex behaviors.

This weighting strategy as well as the insertion of transitional tasks also endow the system with the capability to Fig. 7. Inputs torques applied to the robot without a constraint breaking strategy: a torque peak appears when contact vanishes. Fig. 8. Inputs torques applied to the robot with a constraint breaking strategy: the control torque is increased as expected but in a smooth manner. switch smoothly between constrained and non-constrained states and ensure that continuous input torques are fed to the robot.

Simulation results on a virtual iCub robot are presented to illustrate the realization of a complex mission as well as some specific aspects related to tasks transitions.

B. Future Works

A simple extension to this work will consist in melding both weighting and strict hierarchy strategies. Indeed, even though the soft priority strategy is shown to be more versatile, exhibits good properties from the control standpoint and is computationally less expensive, one can imagine cases where very specific tasks cannot be handled as constraints but needs to be achieved almost perfectly (e.g. balance tasks on complex grounds).

Also, the proposed framework was not tested in the case of collision avoidance tasks. We are confident that this type of tasks can be implemented in a straightforward manner and this will be part of near future work.

From the supervision point of view, automatic tasks weighting and transition insertions can probably be improved in order to rely on a less had oc strategy and provide a more general framework.

Finally, a new version of iCub allowing torque control will soon be available at ISIR and will allow us to implement and validate this control algorithm on a real robot.

Fig. 1 .

 1 Fig.1. Illustration of real and virtual iCub robots with prescribed tasks of different natures.

Fig. 2 .

 2 Fig.2. Transformation of the scenario into tasks and triggering events which feed the generic controller.

Fig. 4 .Fig. 5 .

 45 Fig. 4. Evolution of the tasks importance while grabbing the box.

Fig. 6 .

 6 Fig.6. Evolution of the input torques with a weighting strategy.

  ] T and one can compute the state and input matrices P x , P u to obtain the relation

	Hence, the input matrix is composed of the future CoM jerks ... CH = [ ... c 0 , ... c 1 , ..., ... c H-1

  < t e

	exp	log (wnext) -log (wprev) ts -te	t , t ∈ [t e , t s ]	(22)
	w next , t > t s

Of course, this rather simple supervisor can be coupled to or replaced by an higher level reasoning unit but this is out of scope of this work.