Florian De Mortain

Audine Subias

Louise Travé-Massuyès

Vincent De Flaugergues

Towards Active Diagnosis of Hybrid Systems leveraging Multimodel Identification and a Markov Decision Process

Keywords: Active diagnosis, hybrid systems, multimodel identification, Markov Decision Process Active diagnosis, hybrid systems, multimodel identification, Markov Decision Process Active diagnosis, hybrid systems, multimodel identification, Markov Decision Process Active diagnosis, hybrid systems, multimodel identification, Markov Decision Process Active diagnosis, hybrid systems, multimodel identification, Markov Decision Process Active diagnosis, hybrid systems, multimodel identification, Markov Decision Process Active diagnosis, hybrid systems, multimodel identification, Markov Decision Process Active diagnosis, hybrid systems, multimodel identification, Markov Decision Process

Active diagnosis is defined as the association of fault detection and isolation algorithms with the execution of control plans that optimize fault research performance. This paper addresses active diagnosis of hybrid systems. It proposes to associate a diagnosis method based on multimodel identification and a framework for optimal conditional planning relying on a Markov decision process (MDP). The multimodel diagnosis algorithm identifies the most probable fault by measuring a distance between residual vectors generated from the test system and a set of reference fault models. Moreover a criterion called the correct diagnosis rate (CDR) is set up to evaluate the accuracy of the diagnosis results depending on the applied operation sequence. Conditional planning is formulated as a MDP, which is a model mixing a discrete structure and probabilistic variables. It is based on a reward function weighing diagnosis accuracy and the cost of actions and the optimal conditional plan is characterized thanks to the recursive Bellman function. An application to a diesel engine airpath model is presented so as to illustrate the diagnosis and planning methods in practice.

INTRODUCTION

Car diagnosis is challenged by the unceasing evolution of car technologies. Technicians diagnosing car failures in repair workshops are helped in their task by decision support tools that need to be continuously enhanced. One opportunity to improve them is to combine fault detection and isolation (FDI) algorithms, which monitor the system behavior, with the application of relevant control laws, meant at boosting fault research performance. Such a mix of control and diagnosis is known as active diagnosis.

The first objective of our work is to set up an active diagnosis solution. It is a method integrating both a diagnosis algorithm and a conditional planning method that finds optimal sequences of actions based on the past observations. The method has been designed with the aim of being applicable to a family of hybrid systems, which consist of interactions between continuous and discrete dynamics, and to an industrial system: a diesel engine airpath. A third key objective is to integrate techniques belonging to two a priori distinct worlds of the literature, which are active diagnosis of continuous systems (CS) and test sequencing of discrete event systems (DES). The approach is thus built likewise [START_REF] Bayoudh | Active diagnosis of hybrid systems guided by diagnosability properties[END_REF].

The literature of active diagnosis of CS is firstly composed of methods based on multimodel identification. On the one hand, diagnosis is formulated as determining from a set of models, each corresponding to a nominal or fault situation, the one that best fits the system behavior. On the other hand, planning for diagnosis is achieved, e.g. in [START_REF] Blackmore | Finite horizon control design for optimal discrimination between several models[END_REF], through quadratic optimization on linearized systems and in [START_REF] Šimandl | Active fault detection and control: Unified formulation and optimal design[END_REF] through an generic control framework where an input generator interacts with a diagnosis module. The authors use a criterion balancing trajectory tracking and fault detection objectives and the optimal input is characterized thanks to the Bellman function. This framework includes the notion of Markov chain. In a similar spirit, [START_REF] Gholami | Active fault diagnosis for hybrid systems based on sensitivity analysis and ekf[END_REF]) bases its method on parameter estimation where optimal inputs are the ones that maximize the sensitivities of the parameters. Finally, [START_REF] Eriksson | A method for quantitative fault diagnosability analysis of stochastic linear descriptor models[END_REF]) contributes to the active diagnosis literature, even if not explicitly stated. The goal is to analyze the effect of uncertainties and control inputs on the capacity to distinguish two fault models from each other thanks to a bank of residual. A distinguishability measure, based on the Kullback-Leibler norm, is set up so as to carry this analysis out.

In the literature oriented towards DES, [START_REF] Bayoudh | Active diagnosis of hybrid systems guided by diagnosability properties[END_REF]) presents a method of active diagnosis of hybrid systems cast in a DES framework. The system model, in the form of a hybrid automaton, is transformed into a purely discrete automaton and then into a diagnoser that integrates signature events obtained from residual signals with thresholds. A minimax search algorithm applied to the diagnoser finds conditional trajectories of modes that optimize fault discrimination. Besides, test selection for hybrid systems is addressed in [START_REF] Pons | Iterative hybrid causal model based diagnosis: Application to automotive embedded functions[END_REF]. The paper details an algorithm using consistency based diagnosis principles. Then

INTRODUCTION

Car diagnosis is challenged by the unceasing evolution of car technologies. Technicians diagnosing car failures in repair workshops are helped in their task by decision support tools that need to be continuously enhanced. One opportunity to improve them is to combine fault detection and isolation (FDI) algorithms, which monitor the system behavior, with the application of relevant control laws, meant at boosting fault research performance. Such a mix of control and diagnosis is known as active diagnosis.

The first objective of our work is to set up an active diagnosis solution. It is a method integrating both a diagnosis algorithm and a conditional planning method that finds optimal sequences of actions based on the past observations. The method has been designed with the aim of being applicable to a family of hybrid systems, which consist of interactions between continuous and discrete dynamics, and to an industrial system: a diesel engine airpath. A third key objective is to integrate techniques belonging to two a priori distinct worlds of the literature, which are active diagnosis of continuous systems (CS) and test sequencing of discrete event systems (DES). The approach is thus built likewise [START_REF] Bayoudh | Active diagnosis of hybrid systems guided by diagnosability properties[END_REF].

The literature of active diagnosis of CS is firstly composed of methods based on multimodel identification. On the one hand, diagnosis is formulated as determining from a set of models, each corresponding to a nominal or fault situation, the one that best fits the system behavior. On the other hand, planning for diagnosis is achieved, e.g. in [START_REF] Blackmore | Finite horizon control design for optimal discrimination between several models[END_REF], through quadratic optimization on linearized systems and in [START_REF] Šimandl | Active fault detection and control: Unified formulation and optimal design[END_REF] through an generic control framework where an input generator interacts with a diagnosis module. The authors use a criterion balancing trajectory tracking and fault detection objectives and the optimal input is characterized thanks to the Bellman function. This framework includes the notion of Markov chain. In a similar spirit, [START_REF] Gholami | Active fault diagnosis for hybrid systems based on sensitivity analysis and ekf[END_REF] bases its method on parameter estimation where optimal inputs are the ones that maximize the sensitivities of the parameters. Finally, [START_REF] Eriksson | A method for quantitative fault diagnosability analysis of stochastic linear descriptor models[END_REF]) contributes to the active diagnosis literature, even if not explicitly stated. The goal is to analyze the effect of uncertainties and control inputs on the capacity to distinguish two fault models from each other thanks to a bank of residual. A distinguishability measure, based on the Kullback-Leibler norm, is set up so as to carry this analysis out.

In the literature oriented towards DES, [START_REF] Bayoudh | Active diagnosis of hybrid systems guided by diagnosability properties[END_REF]) presents a method of active diagnosis of hybrid systems cast in a DES framework. The system model, in the form of a hybrid automaton, is transformed into a purely discrete automaton and then into a diagnoser that integrates signature events obtained from residual signals with thresholds. A minimax search algorithm applied to the diagnoser finds conditional trajectories of modes that optimize fault discrimination. Besides, test selection for hybrid systems is addressed in [START_REF] Pons | Iterative hybrid causal model based diagnosis: Application to automotive embedded functions[END_REF]. The paper details an algorithm using consistency based diagnosis principles. Then

INTRODUCTION

Car diagnosis is challenged by the unceasing evolution of car technologies. Technicians diagnosing car failures in repair workshops are helped in their task by decision support tools that need to be continuously enhanced. One opportunity to improve them is to combine fault detection and isolation (FDI) algorithms, which monitor the system behavior, with the application of relevant control laws, meant at boosting fault research performance. Such a mix of control and diagnosis is known as active diagnosis.

The first objective of our work is to set up an active diagnosis solution. It is a method integrating both a diagnosis algorithm and a conditional planning method that finds optimal sequences of actions based on the past observations. The method has been designed with the aim of being applicable to a family of hybrid systems, which consist of interactions between continuous and discrete dynamics, and to an industrial system: a diesel engine airpath. A third key objective is to integrate techniques belonging to two a priori distinct worlds of the literature, which are active diagnosis of continuous systems (CS) and test sequencing of discrete event systems (DES). The approach is thus built likewise [START_REF] Bayoudh | Active diagnosis of hybrid systems guided by diagnosability properties[END_REF].

The literature of active diagnosis of CS is firstly composed of methods based on multimodel identification. On the one hand, diagnosis is formulated as determining from a set of models, each corresponding to a nominal or fault situation, the one that best fits the system behavior. On the other hand, planning for diagnosis is achieved, e.g. in [START_REF] Blackmore | Finite horizon control design for optimal discrimination between several models[END_REF], through quadratic optimization on linearized systems and in [START_REF] Šimandl | Active fault detection and control: Unified formulation and optimal design[END_REF] through an generic control framework where an input generator interacts with a diagnosis module. The authors use a criterion balancing trajectory tracking and fault detection objectives and the optimal input is characterized thanks to the Bellman function. This framework includes the notion of Markov chain. In a similar spirit, [START_REF] Gholami | Active fault diagnosis for hybrid systems based on sensitivity analysis and ekf[END_REF]) bases its method on parameter estimation where optimal inputs are the ones that maximize the sensitivities of the parameters. Finally, [START_REF] Eriksson | A method for quantitative fault diagnosability analysis of stochastic linear descriptor models[END_REF]) contributes to the active diagnosis literature, even if not explicitly stated. The goal is to analyze the effect of uncertainties and control inputs on the capacity to distinguish two fault models from each other thanks to a bank of residual. A distinguishability measure, based on the Kullback-Leibler norm, is set up so as to carry this analysis out.

In the literature oriented towards DES, [START_REF] Bayoudh | Active diagnosis of hybrid systems guided by diagnosability properties[END_REF]) presents a method of active diagnosis of hybrid systems cast in a DES framework. The system model, in the form of a hybrid automaton, is transformed into a purely discrete automaton and then into a diagnoser that integrates signature events obtained from residual signals with thresholds. A minimax search algorithm applied to the diagnoser finds conditional trajectories of modes that optimize fault discrimination. Besides, test selection for hybrid systems is addressed in [START_REF] Pons | Iterative hybrid causal model based diagnosis: Application to automotive embedded functions[END_REF]. The paper details an algorithm using consistency based diagnosis principles. Then

INTRODUCTION

Car diagnosis is challenged by the unceasing evolution of car technologies. Technicians diagnosing car failures in repair workshops are helped in their task by decision support tools that need to be continuously enhanced. One opportunity to improve them is to combine fault detection and isolation (FDI) algorithms, which monitor the system behavior, with the application of relevant control laws, meant at boosting fault research performance. Such a mix of control and diagnosis is known as active diagnosis.

The first objective of our work is to set up an active diagnosis solution. It is a method integrating both a diagnosis algorithm and a conditional planning method that finds optimal sequences of actions based on the past observations. The method has been designed with the aim of being applicable to a family of hybrid systems, which consist of interactions between continuous and discrete dynamics, and to an industrial system: a diesel engine airpath. A third key objective is to integrate techniques belonging to two a priori distinct worlds of the literature, which are active diagnosis of continuous systems (CS) and test sequencing of discrete event systems (DES). The approach is thus built likewise [START_REF] Bayoudh | Active diagnosis of hybrid systems guided by diagnosability properties[END_REF].

The literature of active diagnosis of CS is firstly composed of methods based on multimodel identification. On the one hand, diagnosis is formulated as determining from a set of models, each corresponding to a nominal or fault situation, the one that best fits the system behavior. On the other hand, planning for diagnosis is achieved, e.g. in [START_REF] Blackmore | Finite horizon control design for optimal discrimination between several models[END_REF], through quadratic optimization on linearized systems and in [START_REF] Šimandl | Active fault detection and control: Unified formulation and optimal design[END_REF] through an generic control framework where an input generator interacts with a diagnosis module. The authors use a criterion balancing trajectory tracking and fault detection objectives and the optimal input is characterized thanks to the Bellman function. This framework includes the notion of Markov chain. In a similar spirit, [START_REF] Gholami | Active fault diagnosis for hybrid systems based on sensitivity analysis and ekf[END_REF] bases its method on parameter estimation where optimal inputs are the ones that maximize the sensitivities of the parameters. Finally, [START_REF] Eriksson | A method for quantitative fault diagnosability analysis of stochastic linear descriptor models[END_REF]) contributes to the active diagnosis literature, even if not explicitly stated. The goal is to analyze the effect of uncertainties and control inputs on the capacity to distinguish two fault models from each other thanks to a bank of residual. A distinguishability measure, based on the Kullback-Leibler norm, is set up so as to carry this analysis out.

In the literature oriented towards DES, [START_REF] Bayoudh | Active diagnosis of hybrid systems guided by diagnosability properties[END_REF]) presents a method of active diagnosis of hybrid systems cast in a DES framework. The system model, in the form of a hybrid automaton, is transformed into a purely discrete automaton and then into a diagnoser that integrates signature events obtained from residual signals with thresholds. A minimax search algorithm applied to the diagnoser finds conditional trajectories of modes that optimize fault discrimination. Besides, test selection for hybrid systems is addressed in [START_REF] Pons | Iterative hybrid causal model based diagnosis: Application to automotive embedded functions[END_REF]. The paper details an algorithm using consistency based diagnosis principles. Then

INTRODUCTION

Car diagnosis is challenged by the unceasing evolution of car technologies. Technicians diagnosing car failures in repair workshops are helped in their task by decision support tools that need to be continuously enhanced. One opportunity to improve them is to combine fault detection and isolation (FDI) algorithms, which monitor the system behavior, with the application of relevant control laws, meant at boosting fault research performance. Such a mix of control and diagnosis is known as active diagnosis.

The first objective of our work is to set up an active diagnosis solution. It is a method integrating both a diagnosis algorithm and a conditional planning method that finds optimal sequences of actions based on the past observations. The method has been designed with the aim of being applicable to a family of hybrid systems, which consist of interactions between continuous and discrete dynamics, and to an industrial system: a diesel engine airpath. A third key objective is to integrate techniques belonging to two a priori distinct worlds of the literature, which are active diagnosis of continuous systems (CS) and test sequencing of discrete event systems (DES). The approach is thus built likewise [START_REF] Bayoudh | Active diagnosis of hybrid systems guided by diagnosability properties[END_REF].

The literature of active diagnosis of CS is firstly composed of methods based on multimodel identification. On the one hand, diagnosis is formulated as determining from a set of models, each corresponding to a nominal or fault situation, the one that best fits the system behavior. On the other hand, planning for diagnosis is achieved, e.g. in [START_REF] Blackmore | Finite horizon control design for optimal discrimination between several models[END_REF], through quadratic optimization on linearized systems and in [START_REF] Šimandl | Active fault detection and control: Unified formulation and optimal design[END_REF] through an generic control framework where an input generator interacts with a diagnosis module. The authors use a criterion balancing trajectory tracking and fault detection objectives and the optimal input is characterized thanks to the Bellman function. This framework includes the notion of Markov chain. In a similar spirit, [START_REF] Gholami | Active fault diagnosis for hybrid systems based on sensitivity analysis and ekf[END_REF]) bases its method on parameter estimation where optimal inputs are the ones that maximize the sensitivities of the parameters. Finally, [START_REF] Eriksson | A method for quantitative fault diagnosability analysis of stochastic linear descriptor models[END_REF]) contributes to the active diagnosis literature, even if not explicitly stated. The goal is to analyze the effect of uncertainties and control inputs on the capacity to distinguish two fault models from each other thanks to a bank of residual. A distinguishability measure, based on the Kullback-Leibler norm, is set up so as to carry this analysis out.

In the literature oriented towards DES, [START_REF] Bayoudh | Active diagnosis of hybrid systems guided by diagnosability properties[END_REF]) presents a method of active diagnosis of hybrid systems cast in a DES framework. The system model, in the form of a hybrid automaton, is transformed into a purely discrete automaton and then into a diagnoser that integrates signature events obtained from residual signals with thresholds. A minimax search algorithm applied to the diagnoser finds conditional trajectories of modes that optimize fault discrimination. Besides, test selection for hybrid systems is addressed in [START_REF] Pons | Iterative hybrid causal model based diagnosis: Application to automotive embedded functions[END_REF]. The paper details an algorithm using consistency based diagnosis principles. Then

INTRODUCTION

Car diagnosis is challenged by the unceasing evolution of car technologies. Technicians diagnosing car failures in repair workshops are helped in their task by decision support tools that need to be continuously enhanced. One opportunity to improve them is to combine fault detection and isolation (FDI) algorithms, which monitor the system behavior, with the application of relevant control laws, meant at boosting fault research performance. Such a mix of control and diagnosis is known as active diagnosis.

The first objective of our work is to set up an active diagnosis solution. It is a method integrating both a diagnosis algorithm and a conditional planning method that finds optimal sequences of actions based on the past observations. The method has been designed with the aim of being applicable to a family of hybrid systems, which consist of interactions between continuous and discrete dynamics, and to an industrial system: a diesel engine airpath. A third key objective is to integrate techniques belonging to two a priori distinct worlds of the literature, which are active diagnosis of continuous systems (CS) and test sequencing of discrete event systems (DES). The approach is thus built likewise [START_REF] Bayoudh | Active diagnosis of hybrid systems guided by diagnosability properties[END_REF]).

The literature of active diagnosis of CS is firstly composed of methods based on multimodel identification. On the one hand, diagnosis is formulated as determining from a set of models, each corresponding to a nominal or fault situation, the one that best fits the system behavior. On the other hand, planning for diagnosis is achieved, e.g. in [START_REF] Blackmore | Finite horizon control design for optimal discrimination between several models[END_REF], through quadratic optimization on linearized systems and in [START_REF] Šimandl | Active fault detection and control: Unified formulation and optimal design[END_REF] through an generic control framework where an input generator interacts with a diagnosis module. The authors use a criterion balancing trajectory tracking and fault detection objectives and the optimal input is characterized thanks to the Bellman function. This framework includes the notion of Markov chain. In a similar spirit, [START_REF] Gholami | Active fault diagnosis for hybrid systems based on sensitivity analysis and ekf[END_REF]) bases its method on parameter estimation where optimal inputs are the ones that maximize the sensitivities of the parameters. Finally, [START_REF] Eriksson | A method for quantitative fault diagnosability analysis of stochastic linear descriptor models[END_REF]) contributes to the active diagnosis literature, even if not explicitly stated. The goal is to analyze the effect of uncertainties and control inputs on the capacity to distinguish two fault models from each other thanks to a bank of residual. A distinguishability measure, based on the Kullback-Leibler norm, is set up so as to carry this analysis out.

In the literature oriented towards DES, [START_REF] Bayoudh | Active diagnosis of hybrid systems guided by diagnosability properties[END_REF]) presents a method of active diagnosis of hybrid systems cast in a DES framework. The system model, in the form of a hybrid automaton, is transformed into a purely discrete automaton and then into a diagnoser that integrates signature events obtained from residual signals with thresholds. A minimax search algorithm applied to the diagnoser finds conditional trajectories of modes that optimize fault discrimination. Besides, test selection for hybrid systems is addressed in [START_REF] Pons | Iterative hybrid causal model based diagnosis: Application to automotive embedded functions[END_REF]. The paper details an algorithm using consistency based diagnosis principles. Then

INTRODUCTION

Car diagnosis is challenged by the unceasing evolution of car technologies. Technicians diagnosing car failures in repair workshops are helped in their task by decision support tools that need to be continuously enhanced. One opportunity to improve them is to combine fault detection and isolation (FDI) algorithms, which monitor the system behavior, with the application of relevant control laws, meant at boosting fault research performance. Such a mix of control and diagnosis is known as active diagnosis.

The first objective of our work is to set up an active diagnosis solution. It is a method integrating both a diagnosis algorithm and a conditional planning method that finds optimal sequences of actions based on the past observations. The method has been designed with the aim of being applicable to a family of hybrid systems, which consist of interactions between continuous and discrete dynamics, and to an industrial system: a diesel engine airpath. A third key objective is to integrate techniques belonging to two a priori distinct worlds of the literature, which are active diagnosis of continuous systems (CS) and test sequencing of discrete event systems (DES). The approach is thus built likewise [START_REF] Bayoudh | Active diagnosis of hybrid systems guided by diagnosability properties[END_REF]).

The literature of active diagnosis of CS is firstly composed of methods based on multimodel identification. On the one hand, diagnosis is formulated as determining from a set of models, each corresponding to a nominal or fault situation, the one that best fits the system behavior. On the other hand, planning for diagnosis is achieved, e.g. in [START_REF] Blackmore | Finite horizon control design for optimal discrimination between several models[END_REF], through quadratic optimization on linearized systems and in [START_REF] Šimandl | Active fault detection and control: Unified formulation and optimal design[END_REF] through an generic control framework where an input generator interacts with a diagnosis module. The authors use a criterion balancing trajectory tracking and fault detection objectives and the optimal input is characterized thanks to the Bellman function. This framework includes the notion of Markov chain. In a similar spirit, [START_REF] Gholami | Active fault diagnosis for hybrid systems based on sensitivity analysis and ekf[END_REF]) bases its method on parameter estimation where optimal inputs are the ones that maximize the sensitivities of the parameters. Finally, [START_REF] Eriksson | A method for quantitative fault diagnosability analysis of stochastic linear descriptor models[END_REF]) contributes to the active diagnosis literature, even if not explicitly stated. The goal is to analyze the effect of uncertainties and control inputs on the capacity to distinguish two fault models from each other thanks to a bank of residual. A distinguishability measure, based on the Kullback-Leibler norm, is set up so as to carry this analysis out.

In the literature oriented towards DES, [START_REF] Bayoudh | Active diagnosis of hybrid systems guided by diagnosability properties[END_REF]) presents a method of active diagnosis of hybrid systems cast in a DES framework. The system model, in the form of a hybrid automaton, is transformed into a purely discrete automaton and then into a diagnoser that integrates signature events obtained from residual signals with thresholds. A minimax search algorithm applied to the diagnoser finds conditional trajectories of modes that optimize fault discrimination. Besides, test selection for hybrid systems is addressed in [START_REF] Pons | Iterative hybrid causal model based diagnosis: Application to automotive embedded functions[END_REF]. The paper details an algorithm using consistency based diagnosis principles. Then

INTRODUCTION

Car diagnosis is challenged by the unceasing evolution of car technologies. Technicians diagnosing car failures in repair workshops are helped in their task by decision support tools that need to be continuously enhanced. One opportunity to improve them is to combine fault detection and isolation (FDI) algorithms, which monitor the system behavior, with the application of relevant control laws, meant at boosting fault research performance. Such a mix of control and diagnosis is known as active diagnosis.

The first objective of our work is to set up an active diagnosis solution. It is a method integrating both a diagnosis algorithm and a conditional planning method that finds optimal sequences of actions based on the past observations. The method has been designed with the aim of being applicable to a family of hybrid systems, which consist of interactions between continuous and discrete dynamics, and to an industrial system: a diesel engine airpath. A third key objective is to integrate techniques belonging to two a priori distinct worlds of the literature, which are active diagnosis of continuous systems (CS) and test sequencing of discrete event systems (DES). The approach is thus built likewise [START_REF] Bayoudh | Active diagnosis of hybrid systems guided by diagnosability properties[END_REF]).

The literature of active diagnosis of CS is firstly composed of methods based on multimodel identification. On the one hand, diagnosis is formulated as determining from a set of models, each corresponding to a nominal or fault situation, the one that best fits the system behavior. On the other hand, planning for diagnosis is achieved, e.g. in [START_REF] Blackmore | Finite horizon control design for optimal discrimination between several models[END_REF], through quadratic optimization on linearized systems and in [START_REF] Šimandl | Active fault detection and control: Unified formulation and optimal design[END_REF] through an generic control framework where an input generator interacts with a diagnosis module. The authors use a criterion balancing trajectory tracking and fault detection objectives and the optimal input is characterized thanks to the Bellman function. This framework includes the notion of Markov chain. In a similar spirit, [START_REF] Gholami | Active fault diagnosis for hybrid systems based on sensitivity analysis and ekf[END_REF]) bases its method on parameter estimation where optimal inputs are the ones that maximize the sensitivities of the parameters. Finally, [START_REF] Eriksson | A method for quantitative fault diagnosability analysis of stochastic linear descriptor models[END_REF]) contributes to the active diagnosis literature, even if not explicitly stated. The goal is to analyze the effect of uncertainties and control inputs on the capacity to distinguish two fault models from each other thanks to a bank of residual. A distinguishability measure, based on the Kullback-Leibler norm, is set up so as to carry this analysis out.

In the literature oriented towards DES, [START_REF] Bayoudh | Active diagnosis of hybrid systems guided by diagnosability properties[END_REF]) presents a method of active diagnosis of hybrid systems cast in a DES framework. The system model, in the form of a hybrid automaton, is transformed into a purely discrete automaton and then into a diagnoser that integrates signature events obtained from residual signals with thresholds. A minimax search algorithm applied to the diagnoser finds conditional trajectories of modes that optimize fault discrimination. Besides, test selection for hybrid systems is addressed in [START_REF] Pons | Iterative hybrid causal model based diagnosis: Application to automotive embedded functions[END_REF]. The paper details an algorithm using consistency based diagnosis principles. Then in [START_REF] Chanthery | An AO*-like algorithm implementation for active diagnosis[END_REF] an application of active diagnosis of DES is developed based on an AO* heuristic search in a AND/OR graph derived from a diagnoser automaton. Besides, in [START_REF] Pattipati | Application of heuristic search and information theory to sequential fault diagnosis[END_REF]) the authors formulate and solve a test sequencing problem based on a Markov Decision Process.

The method developed in this paper is, first of all, based on a simplifying hypothesis. The considered hybrid system is considered to be remaining into a limited operation range called mode q, where discrete events do not occur and its behavior consists only of continuous dynamics. That is why the approach exploits a nonlinear model, typically used to represent continuous systems.

The first part of the method presents a diagnosis process based on multimodel identification, also called here multimodel diagnosis. The method explains how to generate residuals with multiple models and how to find the most probable fault by comparing the system residuals with the fault models ones, by means of a distance measure. Furthermore a new criterion, called the Correct Diagnosis Rate (CDR), is presented. Its function is to rate the confidence of a diagnosis depending on the uncertainty level and on the past sequence of actions. The multimodel diagnosis process is presented in section 2. The second part of the method outlines a framework for conditional planning for active diagnosis formulated as a Markov Decision Process (MDP). A new reward function based on the CDR and the cost of actions is proposed. Section 3 is dedicated to this MDP formulation. Finally, section 4 deals with the application of the method on an industrial model of a diesel engine airpath system. It illustrates its complexity and shows how to generate residuals, how to compute the CDR and finally how to solve a simple conditional planning scenario. Conclusions and perspectives are given in section 5.

MULTIMODEL DIAGNOSIS PROCESS

The first stage of the approach is dedicated to the design of a diagnosis algorithm along with a way to rate the relevance of its results. The process of multimodel diagnosis involves three steps which are the building of multiple fault models, the generation of residual sequences for the system and each fault model and finally the selection of the fault model whose residuals best match the system ones. Furthermore, a quantitative criterion called the Correct Diagnosis Rate (CDR) is introduced. Its role is to help guiding the process of active diagnosis by indicating how much confidence can be assigned to a diagnosis depending on the past sequence of actions.

System, control framework and multiple fault models h

m o dia no i a h id m constrained into a limited operation mode q, where its dynamics are purely continuous. Its model is given, for each time t n ϵ T = {t 0 ,t 1 t NT }, by the following discrete-time stochastic state space representation: ) , , , (

n n n n 1 n w f u x g x (1) ) , , , ( n n n n n v f u x h y (2)
where g n and h n are nonlinear vector functions. x n ℝ Nx is the continuous state of the system, u n ℝ Nu is the input, y n ℝ Ny is the output and f ℝ Nf is the fault parameter. w n ℝ Nw and v n ℝ Nv are respectively the process and measurement noise variables. They are modeled by zero mean Gaussian probability density vector functions p(w n ) and p(v n ). 1), where it is connected with a con oll . Hence the system behavior is more robust to uncertainties and in the specific case of automotive control, it helps preventing the engine to stall or to be overspeeding. The controller is fed in a discrete-time approach by control actions a ϵ Ω, where Ω is the finite set of N Ω control actions. A sequence of N A consecutive actions a is denoted A = {a 0 a N A -1 } N A , while its associated time sequence is TA. x 0 is the initial state of the system.

The essence of multimodel diagnosis is to anticipate the system fault behaviors by means of fault-dedicated models. The process of building fault models starts by defining a list of fault parameters. They represent the faults cases which may occur and that have not yet been discarded by other diagnosis means. The finite set of (N F +1) fault parameters

f i ℝ N f is denoted F = {f 0 f N F }.
f 0 accounts for the nominal case. The single fault hypothesis holds, hence only one element of a parameter vector f i ϵ F deviates from zero at a time. Moreover, various fault parameters can refer to the same fault, when different fault amplitudes are modeled. For example, biased measurement faults of 5% and 10% of a specific sensor can be modeled by two different fault parameters f i and f j ϵ F.

A set of fault-dedicated models is finally obtained by replacing the variable f in the equations ( 1) and ( 2) by a fault parameter f i ϵ F, resulting in stochastic models denoted s f i . This set of multiple fault-dedicated models is denoted in a synthesized way, S DIAG = {s f i } f i ϵF . The set S DIAG thus represents in a whole range of anticipated fault situations.

Residual generation

Now that each fault has its model, the motivation here is to generate the data on which to base the comparison between the system and the fault models.

N r R n f n f A i i 0
for each f i ϵ F. The process of residuals generation in a multimodel scheme thus results in sequences o idual o h m and o ach aul model s f i ϵ S DIAG . This paves the way for electing the most probable fault with the diagnosis test.

Diagnosis test

The role of the diagnosis test is to identify the fault parameter f i ϵ F that best explains the system behavior. It relies on a distance measure that compares residuals from the system and the fault models with each other. These residuals are then seen as data points in a multidimensional algebraic space. Moreover, as it is thoroughly explained in [START_REF] Eriksson | A method for quantitative fault diagnosability analysis of stochastic linear descriptor models[END_REF], model uncertainties are key to interpret residuals values. In order to simplify the uncertainty definition, the following approximation is made. The noise on the residual value resulting from the process noise and measurement noise is considered to be a white Gaussian noise of variance matrix I . Even if this hypothesis is not strictly true for nonlinear systems, it is reasonable in the context of our study. The variance matrix I is designed based on the analysis of the gap between experimental behavior data of the system and its modeled behavior. Let us now define a distance measure that evaluates the dissimilarity between the residuals sequences of the m and a fault model s f i ϵ S DIAG :

A n t t r r I r r R R ) ( ) ( ) , ( 1 i i i f n n f n n f A A (3)
Based on this distance measure, the next definition specifies how to determine the most probable fault, i.e. the diagnosis.

Definition 1 Given an action sequence A, the fault parameter f

i ϵ F is the diagnosis if: ) , ( min arg j f A A F f i R R f j (4)
This definition means that the diagnosis is considered to be the fault parameter whose residuals are the most similar, in m o h di anc m a u , with the system ones.

Correct diagnosis rate

The diagnosis algorithm returns the most probable fault after a given action sequence A. But due to model uncertainties, the system residuals may be not closer to the theoretical residuals from the fault the system is affected by, and the distance measure diagnosis may be wrong. This is illustrated in figure 5 (mostly presented in part 4) where noisy fault residual vectors, which are spread in a large area, mingle with other residuals from other faults. Thus there is an ambiguity on the diagnosis result due to uncertainties.

This ambiguity also depends on the applied actions. For example, as illustrated in figure 4 (mostly presented in part 4 as well), the nominal residuals are similar to the intake leakage residuals after the control action a 1 , while they are dissimilar to both the intake leakage and the airflow sensor residuals after the control action a 3 . As a result, there is a greater ambiguity on a diagnosis concluding that the system is nominal (with no fault) after the action {a 1 } rather than after the action sequence {a 3 } or {a 1, a 3 }, because there is less chance to confuse the nominal case with the leakage.

So as to quantify the concept of ambiguity of a diagnosis, and help in the selection of the best action sequences, a criterion called the Correct Diagnosis Rate (CDR) is defined.

It relates to the ambiguity as the less the diagnosis is ambiguous, the greater the CDR. The criterion refers to both a diagnosis f i ϵ F and a past sequence of actions A. So as to pave the way for the definition of the MDP, a new concept of diagnosis state is as well defined so as to synthetize A and f i in a single mathematical object.

Definition 2 The system i in h dia no i a i f A d if the
last available diagnosis obtained by the application of the action sequence A N A , is the fault parameter f i ϵ F.

The CDR is computed offline, based on a test sample of noisy fault models, denoted S TEST . This test set is built by duplicating all models of the initial set S DIAG , into a large number of clone models, and replacing the measurement and noise parameters, v n and w n , by randomly instantiated values according to their probabilistic models. All fault parameters f i ϵ F appear the same number of times in S TEST . The CDR is defined by the following probabilistic expression:

i TEST i f A i S f A p d f d CDR (5) SAFEPROCESS 2015 September 2-4, 2015. Paris, France
The CDR is a way to compute the probability of true positives in a multiple fault framework. This stage has thus specified a multimodel diagnosis algorithm. Also to prepare the design of the best action sequences, a criterion rating the ambiguity of a diagnosis has been set up.

OPTIMAL CONDITIONNAL PLANNING FORMULATED AS A MARKOV DECISION PROCESS

The second and last stage of the approach consists in formulating a problem of optimal conditional planning for active diagnosis. The objective here is to design an optimal policy of actions, i.e. to determine which is the best next action based on the past observations. After any sequence of actions, the resulting diagnosis always carries some ambiguity, because of model uncertainties. Therefore the optimal policy of actions should minimize this ambiguity by driving the system to the most informative operation points. Also, the diagnosis should stop when no significant reduction of the diagnosis ambiguity is expected to occur. In the approach, a MDP model is introduced so as to represent this problem and its optimal solution is characterized with the help of the Bellman equation.

Definition of the Markov Decision Process

Optimal conditional planning is modeled as a Markov Decision Process, illustrated in figure 6, that consists of the following elements: when the additional action a is applied. Moreover, the transition function satisfies

F f a A f A i i j f d a d 1 , , T .
The role of the reward function is to orient the choice of action by associating a reward value to each state transition. In the approach it is based on the correct diagnosis rate CDR and a cost of action C, which is constant and independent of the action a . The reward function is defined as follows:

C j i i j f A f a A f a A f A ) CDR( ) CDR( , , R d d d a d (7)
The first two terms of the function consist of the improvement of the diagnosis accuracy. The actions that better the confidence of a diagnosis are then favored compared to other. The third and last term is the cost of action. Its role is to disadvantage the application of new actions compared to stopping the diagnosis, in the case they do not significantly improve the CDR.

Optimal conditional planning is a decision process in the sense that next actions are decided based on the past observations. Furthermore, this is a Markovian decision process provided it respects the property of process without memory. This property says that based upon the present state, the future and the past are independent. In a mathematical form, it is summed up as:

n n n t 1 n 1 n t 1 n -1 n -1 n t n n n t 1 n 1 n t t f A f A Start t f A t f A f A a d d P d a d a d d P t t t t t , ,..., , , , (8)
The hypothesis is made here that the decision process respects this property. In the context of active diagnosis, the present diagnosis, realized based on all available residuals (from time t 1 to current time t n ) is the only relevant one, and all past diagnosis are obsolete. Therefore the property of process without memory is consistent with the problem. As a result, the formalism of the MDP is now fully set up and next step presents its optimal solution.

Optimal solution

The objective here is to characterize the optimal policy * of the MDP. The concept of utility function is firstly introduced.

Definition 3 i n a olic , the utility function

) ( f A V d is
) ( V* ) , , R( . , , T max ) ( * i i j i i j j f a A f a A f A F f a A f A f A d d a d d a d d V f a (9)
The computation of the optimal policy based on the Bellman equation requires enumerating all the states and computing the rewards and transition probabilities for all cases. Due to SAFEPROCESS 2015 September 2-4, 2015. Paris, France the process size, which consists of N D states, this solution is intractable in the general case. Heuristic techniques, such as the AO* algorithm, may however address this issue by quickly identifying suboptimal policies.

APPLICATION TO A DIESEL ENGINE AIRPATH MODEL

This last part exemplifies the multimodel diagnosis and conditional planning methods on an industrial model of a Diesel engine airpath. It is a dynamic, nonlinear and greybox model made of both physical equations and data-based maps. The design of optimal control actions for diagnosis is simplified by taking account only static operation points. It fits with the framework developed in part 2 and 3 with the condition that time steps are taken very large compared to the system dynamics. The key ideas of this part are to show how the diagnosis method is setup in practice, how it takes into account model uncertainties with the CDR and finally how to select the best next actions based on the MDP.

System description, fault modeling and residual design

The studied airpath system is illustrated in fig. 3. It is composed of 11 sensors, e.g. pressure and temperature variables, and 4 actuators: the fuel injection u Inj , turbocharger valves position u Turb , EGR valve position u Egr and throttle u Thrt . Moreover a PI controller is used to control u Inj according to a reference for the engine speed N Eng . Many faults can affect the system such as clogging, leakage, sensor and actuator faults. In the case study, yet 3 fault models are considered: the nominal case, a leakage in the intake manifold and a positive bias of the airflow sensor, respectively denoted f Nom , f LkInMfd and f SnsQair+ . They are modeled according to the equations in [START_REF] Eriksson | A method for quantitative fault diagnosability analysis of stochastic linear descriptor models[END_REF]. The section of the leakage is set at 2% of a standard engine pipe section and the level of the sensor bias is 5% of the real value of the measured variable. Moreover, the standard deviation of the model uncertainties encompassing process and sensor noise is set at 5% on the output variables.

Then a bank of 2 residuals is designed. They are the intake manifold pressure simulation error r P and the airflow estimation error r Q . The estimation of the airflow is reconstructed thanks to other measured variables. where P InMfd is the intake manifold pressure, Q Air is the airflow, V Cyl is the cylinders volume, Vol is the volumetric efficiency, r InMfd is the gas constant in the intake manifold and T Exch is the temperature after the exchanger. The exponents meas and est respectively refer to measured and estimated variables.

Distance measure diagnosis & CDR

The operation points of the case study are the following ones: a 1 = {N Eng = 3000rpm, u turb = 0%}, a 2 = {N Eng = 3000rpm, u turb = 100%} and a 3 = {N Eng = 2000rpm, u turb = 100%}. The inputs u egr and u thrt are set to fixed values. The residuals generated by simulation for the three fault models associated to f Nom , f LkInMfd and f SnsQair+ on the 3 operation points are illustrated on figure 4. The x-axis refers to the residuals r P and r Q while the y-axis refers to their value. This graph illustrates how residuals vary depending on the fault and the operation point. Let us zoom now on the first operation point a 1 . Figure 5 represents the 2 residuals on a 2D view. The large crosses are the responses of the fault models without noise. The disks represent the standard deviation of the process and measurement noise (5% on all residuals). A real system would generate residuals represented by a data point on this graph. The diagnosis by distance measure is equivalent to finding the fault model whose response is the closest in this graph to the real system data point. S TEST is composed in this case study of 3000 noisy fault models and is not fully represented on the graph for sake of simplicity. The CDR is also computed for each fault and each action sequence. On the operation point a 1 , the CDRs for f Nom and f SnsQair+ are . Indeed, as shown in figure 5, the nominal residuals are close to the intake leakage residuals on a 1 , rather than the airflow sensor bias distant from those two first fault responses. Therefore there is more ambiguity (lower CDR) on a diagnosis giving f Nom , as there is a higher risk of mixing with f LkInMfd , and less ambiguity (higher CDR) on a diagnosis giving f SnsQair+ .

MDP based conditional planning

now illustrate the conditional planning mechanism by considering a situation where the system has already been driven on the operation point a 1 and the diagnosis process has identified the nominal case f Nom as the most probable fault. Figure 6 shows the MDP and the considered subpart (in bold) starts from state Nom a1 d . There are 3 next possible decisions: applying the control action a 2 , applying a 3 or no applying any additional action. The utility function is computed for the 3 cases, by choosing a cost of action C = 0.05. This computation uses the values of the transition probabilities and the CDRs, stored in table 1, themselves evaluated thanks to S TEST . The resulting utility values are given in table 2. They show that, with respect to the chosen reward function, applying action a 2 is the best decision SAFEPROCESS 2015 September 2-4, 2015. Paris, France compared to applying a 3 or stopping the diagnosis. This is consistent with figure 4 as on the operation point a 2 all the 3 fault signatures are rather dissimilar with each other rather than on a 3, the residuals for f LkInMfd and f SnsQair+ are rather similar. Hence a 2 should be preferred to a 3. In the same analysis between a 2 and a 1, one can conclude that after having applied a 1 and reaching the state Nom a1 d , applying a 2 is a better option than stopping the diagnosis there. This shows how to compute utility values for a one-step horizon. A perspective for this approach is to use a heuristic technique so as to make the computation of the utility fast, as for a longer horizon the number of states to explore grows exponentially and the problem becomes intractable. The main perspective of this work is to apply heuristic techniques, such as the AO* algorithm, to address the complexity issue of the MDP and to quickly build suboptimal conditional plans.

  a n ic clo d-loop control architecture, shown in figure (

  ocia d o a n ic con oll .

  in definition 2, are associated to both a diagnosis f i ϵ F and a past sequence of actions A. They also include an initial state Start d that precedes the start of the active diagnosis session. The number of states N D equals the number of combinations of (f i, A), i.e. a. In practice, these transition probabilities are computed offline, based on the test set S TEST , according to the definition below.

  , evaluating transition probabilities consists in counting the proportion of models s f k ϵ S TEST that are in the state j f A d after the control sequence A, and reach the state i d f a A

  the expected sum of future rewards starting in state f A d and then acting according to the olic until reaching a final state.The optimal policy, denoted *, is the one that maximizes the utility of the initial state. According to the Bellman principle, * also maximizes the utility starting from any state of the MDP. The Bellman equation translates this principle in a mathematical form. It characterizes the optimal utility

Fig. 3 .

 3 Fig. 3.Illustration of the diesel engine airpath system

Fig. 4 .

 4 Fig. 4. Stem view of residual vectors for 3 faults f Nom , f LkInMfd and f SnsQair+ and 3 static control actions a 1, a 2 and a 3 . The y axis has no unit because residuals values are relative.

Fig. 5 .

 5 Fig. 5. 2D view of residual vectors without noise (+) and with noise (*), for the 3 faults and the static control action a 1 .5. CONCLUSIONSThis paper addressed the design of an active diagnosis solution. The scope includes hybrid systems whose operation is constrained into a mode and an industrial application that is a Diesel engine airpath system. The originality of the research approach lies in the mix of techniques dedicated to CS and DES. A multimodel diagnosis process is set up and introduces a specific distance measure and a criterion called the CDR. The CDR helps rating the confidence assigned to a diagnosis depending on the uncertainties level and on the past sequence of actions and consists of a first contribution. Then conditional planning for active diagnosis is formulated as a MDP with a reward function weighing the diagnosis accuracy (CDR) and the cost of additional actions. This

Fig. 6 .

 6 Fig. 6. Partial Markov Decision Process for the 3 faults and the final operation sequences {a 1 ,a 2 } and {a 1 ,a 3 }, with a highlight on the subpart starting from state Nom a1 d .

  Francedeviate from zero at least during a transient, when at least one of the faults they are sensitive to, affects the system.The general manner to generate residuals is to identify in the system model, a set of equations and known variables that reconstructs a given variable in two different ways. The known variables are the input u n , output y n and control actions a n , as shown in fig.2.

	SAFEPROCESS 2015
	September 2-4, 2015. Paris, Residual generator action a n input u n output y n		residuals r n
	Fig. 2. Generic residual generation scheme.
	In the application part (cf. section 4.2), two residual
	generation techniques are specified, which are estimation
	error residuals and simulation error residuals. The following
	developments are independent of the residual generation
	technique.
	In the active diagnosis case, a sequence of residuals is
	evaluated online by drivin h		m			i h h ac ion
	sequence A. The sequence is denoted	R	A	r n	0	n	N	A	.
	Besides, it is also sought to generate residuals specific to
	each fault. Therefore, in an offline process, each fault-
	dedicated model s f i of the set S DIAG is used to generate
	residual sequences by reproducing in simulation the
	condi ion	a li d	o	.	The		sequence	is
	denoted	A
										Most contributions in the
										active diagnosis literature do it by means of input-output
										data; see (Blackmore & Williams 2006) and (Šimandl &

[START_REF] Šimandl | Active fault detection and control: Unified formulation and optimal design[END_REF]

. However, a more generic alternative, widespread in the classical FDI literature, consists in using residuals instead. Residuals are signals resulting from a processing of the input-output behavior data of the system. Residuals are theoretically zero when there is no fault and

Table 1 .

 1 Transitions probabilities and CDR for the subpart of the MDP starting from Nom

	0.23	0.17	0

a1 d .

Table 2 .

 2 Utility values for actions a 2 , a 3 , and stop from Nom

a1 d .