Résumé en français

Introduction

De nombreux systèmes critiques doivent satisfaire une spécification donnée. Cette spécification peut comprendre des critères de sécurité, des mesures de rendement ou d'autres conditions. Le processus de vérification, dont le but est de vérifier que le système satisfait une spécification, peut être réalisé de différentes manières. Chacune ayant ses avantages et inconvénients, le choix de la méthode à utiliser dépend à la fois du système qui est étudié et de la spécification. Une des méthodes de vérification possibles est de réaliser des batteries de tests sur le système. Cette méthode est notamment utile lorsqu'on ne connait pas le fonctionnement interne d'un système (le code d'un programme par exemple). Le choix des tests est alors basé sur la spécification [START_REF] Gao | Testing and quality assurance for component-based software[END_REF]. Au contraire, si l'on a accès au système complet, un modèle formel et opérationnel du système peut être construit. Ce modèle peut ensuite être étudié avec des méthodes dédiées. La complexité de certains systèmes peut rendre la construction du modèle difficile. Celle-ci peut être accompli grâce à une analyse du code du système, en réalisant des tests spécifiques dans certaines configurations du système permettant de déterminer comment il évolue (par exemple en surchargeant le processeur d'un ordinateur afin d'observer comment le programme réagit face à ce type de pression), etc. Quand elle est possible, cette approche a de nombreux avantages :

• Lors de la conception d'un système, si le prototype actuel n'accompli pas les objectifs voulus, il doit être modifié. Construire de nouveaux prototypes jusqu'à en obtenir un qui soit satisfaisant est coûteux. Il est plus économique et simple, de modifier un modèle jusqu'à ce que celui-ci satisfasse la spécification, et seulement alors de construire le système associé.

• Un modèle est souvent conçu afin de vérifier plusieurs propriétés. Si l'on désire vérifier de nouvelles propriétés à une date ultérieure, utiliser le modèle existant peut suffire. Dans le cas contraire, il n'est pas forcément nécessaire de construire entièrement un nouveau modèle. Il peut être suffisant de raffiner le modèle actuel, en y incorporant les informations appropriées. Utiliser un tel raffinement réduit fortement la complexité de la nouvelle étude.

• Enfin, si le modèle est suffisamment proche de la réalité, il permet une analyse précise du système. Ce n'est pas le cas lors d'utilisation de batteries de tests, qui ne couvrent qu'une partie des situations possibles. Le même souci existe pour d'autres méthodes telles que la vérification statistique de modèle [START_REF] Barbot | Acceleration for statistical model checking[END_REF].

Il existe de nombreux formalismes permettant de représenter un système. Plus celuici est complexe (représentation du temps, de l'aléatoire. . . ), plus on peut modéliser de systèmes et de spécifications et plus l'étude de ce formalisme est difficile. Notamment, les formalismes possédant une composante probabiliste tels que les chaînes de Markov [START_REF] John | Finite Markov chains[END_REF] ont de nombreuses applications. En effet, certains systèmes nécessitent des probabilités pour être représentés de façon précise. C'est le cas des systèmes contenant des comportements intrinsèquement probabilistes, par exemple un programme utilisant l'aléatoire afin de briser les symétries. Le hasard peut aussi être une conséquence de l'interaction du système avec l'environnement dont le comportement n'est pas entièrement prévisible. De plus, les probabilités permettent de représenter les incertitudes d'un modèle construit de façon approchée, par analyse statistique par exemple. Enfin, l'utilisation de probabilités élargit l'ensemble des propriétés qui peuvent être spécifiées en permettant de les quantifier. Par exemple, si un système (qui n'est pas un système critique) peut commettre des erreurs, mais que celles-ci ont peu de chances d'avoir lieu, ceci peut suffire pour satisfaire la spécification.

Le choix du formalisme utilisé détermine également quelles sont les informations accessibles aux utilisateurs : lorsque l'on construit un modèle, les différents évènements qui peuvent avoir lieu dans le système et leurs effets sont décrits ; certains de ces évènements ont lieu de façon interne et ne sont donc pas observables par un utilisateur extérieur. Le contrôle de l'information transmise par un système a vu son importance augmenter ces dernières années à cause de l'omniprésence des instruments électroniques communicants. Certaines informations du système doivent être maintenues secrètes (les mots de passe par exemple) alors que d'autres doivent être rendues publiques (les erreurs du système par exemple). Les problèmes liés à l'observation partielle peuvent être groupés en trois familles selon le type d'objectif à accomplir : (1) la planification sous observation partielle, (2) la dissimulation d'information à l'observateur et (3) la récupération d'information.

Le diagnostic est l'un des problèmes principaux de cette troisième catégorie. Le terme diagnostic vient du domaine médical dans lequel il désigne l'identification d'une maladie à partir de symptômes. Dans la communauté des systèmes à évènements discrets, cette identification est appliquée à des systèmes dynamiques (les centrales électriques, les chaînes de production. . . ). Dans cette approche, une exécution du système est observée et on essaie de détecter si un évènement particulier, appelé la faute, a eu lieu. La faute ne représente pas forcément une défaillance du système, cependant cette terminologie est utilisée principalement car une irrégularité est l'un des évènements les plus importants à détecter durant une exécution. En effet, celles-ci menacent la sureté et la disponibilité du système. Ceci peut provoquer des dégâts catastrophiques à la fois en termes économiques et humains. L'étude des fautes est également justifiée du fait que tout système peut, et en fait va, faire une erreur. En effet, les systèmes que l'on construit sont de plus en plus complexes et ont des interactions de plus en plus importantes avec l'environnement. Il est donc extrêmement difficile de ne pas introduire d'erreurs lors de la conception d'un système et il est presque impossible de prédire toutes les actions que l'environnement aura sur le système. Enfin, des fautes auront lieu à cause du vieillissement des composants du système.

Comme les fautes sont dangereuses, inévitables et potentiellement difficiles à identifier, une méthode automatique de détection est nécessaire. Par ailleurs, cette méthode doit être précise car stopper un système à cause d'un faux positif est coûteux et elle doit être réactive de façon à ce que les fautes soient repérées avant tout dommage sérieux. En réaction à une faute, on peut soit (1) essayer d'optimiser le comportement du système durant la période préalable à la faute [START_REF] Ehlers | Risk-averse control of Markov decision processes with ω-regular objectives[END_REF], ce qui est particulièrement utile pour des systèmes dont les composants sont régulièrement remplacés, ou (2) essayer de détecter la faute de façon à réagir à son occurrence. Comme l'on désire réagir aussi vite que possible, prédire l'occurrence de la faute permettrait de réagir avant même que le système ne commette l'erreur. Cette question est étudiée dans le contexte des problèmes de prédiction [START_REF] Genc | Predictability of event occurrences in partiallyobserved discrete-event systems[END_REF]. Cependant, il est rare qu'un système permette une prédiction efficace des fautes. Détecter les fautes a posteriori est plus plausible. L'étude du diagnostic soulève deux problèmes importants : comment décider si un système est diagnostiquable, ce qui est appelé diagnostiquabilité, et, dans le cas positif, comment construire un diagnostiqueur la fonction réalisant le diagnostic et qui satisferait potentiellement des conditions supplémentaires sur la taille de la mémoire utilisée, le délai de détection, etc. Dans le domaine des systèmes à évènements discrets, le diagnostic a été défini initialement pour des systèmes finis tels que les systèmes à transitions étiquetées partiellement observables [SSL + 95] puis a été étendu à de nombreux modèles complexes (e.g. les réseaux de Petri [START_REF] Cabasino | A new approach for diagnosability analysis of petri nets using verifier nets[END_REF][START_REF] Bérard | The complexity of diagnosability and opacity verification for Petri nets[END_REF], les systèmes à pile [START_REF] Morvan | Diagnosability of pushdown systems[END_REF], etc.) et cadres (e.g. décentralisés [START_REF] Debouk | Coordinated decentralized protocols for failure diagnosis of discrete event systems[END_REF], distribués [START_REF] Holloway | Time templates for discrete event fault monitoring in manufacturing systems[END_REF]). De plus, plusieurs travaux rassemblés sous le nom diagnostic actif étudient comment contrôler le système pour en assurer la diagnostiquabilité [START_REF] Sampath | Active diagnosis of discreteevent systems[END_REF][START_REF] Thorsley | Active acquisition of information for diagnosis and supervisory control of discrete-event systems[END_REF][START_REF] Cassez | Fault diagnosis with static and dynamic observers[END_REF][START_REF] Chanthery | Monitoring and active diagnosis for discreteevent systems[END_REF].

Notre but dans ce document est d'étudier le diagnostic de systèmes probabilistes. Par conséquent, la première question qui se doit d'être abordée est le choix du formalisme (probabiliste) à utiliser. On doit notamment déterminer si, en plus des probabilités, le modèle est partiellement contrôlable, s'il peut représenter une infinité d'états différents ou s'il doit décrire efficacement des comportements concurrents. Par ailleurs, comme le diagnostic est un problème d'observation partielle, le modèle doit indiquer quelle observation est associée à une exécution.

Dans un deuxième temps, il va nous falloir établir les différents problèmes que nous allons étudier. De nombreuses notions de diagnostic ont déjà été définies, chacune accomplissant un objectif différent. Nous devons donc présenter un ensemble cohérent de notions qualitatives et quantitatives appropriées englobant les définitions importantes déjà établies. De plus, les définitions formelles des problèmes que nous étudions doivent être choisies prudemment. En effet, comme ces problèmes mélangent observation partielle, probabilités et, dans certains cas, contrôle, ils ont de fortes chances d'être indécidables. Une petite modification dans la définition peut faire la différence entre un problème pouvant se résoudre efficacement et un problème indécidable.

Une fois les notions de diagnostic définies, notre but sera d'établir les complexités précises de la diagnostiquabilité et de la synthèse des diagnostiqueurs pour chaque notion, ceci, dans les différents formalismes que nous aurons choisis. Nous chercherons également à déterminer comment modifier un système afin qu'il devienne diagnostiquable. Ceci donne deux approches: une approche passive, de vérification, et une active, de contrôle.

Définitions du modèle et du diagnostic

Le modèle le plus utilisé dans ce document est celui des systèmes probabilistes à transitions étiquetées partiellement observables (dont le nom anglais est abrégé en pLTS). Un pLTS est formellement défini par un tuple Q, q 0 , Σ, T, P où Q est un ensemble dénombrable d'états, q 0 est l'état initial, Σ est un ensemble d'évènements pouvant avoir lieu dans le système, T est un ensemble de transitions indiquant comment un évènement de Σ affecte l'état courant du système et P donne la probabilité de chaque transition. Afin de représenter l'observation partielle du système, l'ensemble d'évènements Σ est partitionné entre évènements observables Σ o et évènements inobservables Σ u . Une exécution ρ du système est une suite d'états et de transitions liant deux états consécutifs. Grâce à P, on peut attribuer une probabilité à toute exécution finie. Ensuite, et en utilisant des résultats de théorie de la mesure, on peut définir une mesure de probabilité sur l'ensemble des exécutions infinies. Par ailleurs, à toute exécution on peut associer une observation qui est la projection sur Σ o de la séquence d'évènements étiquetant ses transitions. On suppose que le pLTS est convergent ce qui signifie que toute exécution infinie possède une observation infinie.

Example 0.1. Considérons le pLTS représenté dans la figure 0.1. Une utilisation normale de la machine à café est donnée par exemple par l'exécution ρ = q 0 pièce q 1 sucre q 1 café q 0 . Cependant dans l'état q 1 une erreur représentée par l'évènement 'f ' peut avoir lieu, menant à l'état f 1 à partir duquel on ne peut plus obtenir de café. Cet évènement a cependant une faible probabilité d'avoir lieu. L'exécution normale ρ a pour probabilité le produit des probabilités des transitions empruntées, c'est-à-dire 1 × 0.29 × 0.7 = 0.203. Un comportement fautif de la forme ρ = q 0 pièce q 1 sucre q 1 f f 1 a probabilité 0.0029 d'avoir lieu.

Comme dans l'exemple précédent, les exécutions du modèle peuvent être fautives ou correctes. Ceci est indiqué par la présence (ou absence) de la faute (l'évènement f ) à l'intérieur de celle-ci. Le but du diagnostic est d'utiliser l'observation d'une exécution pour déterminer si celle-ci est correcte ou fautive. Comme plusieurs exécutions différentes peuvent posséder la même observation, une exécution est surement fautive (resp. surement correcte) si toute les exécutions partageant la même observation sont fautives (resp. correctes). Sinon, l'exécution est ambiguë. Si une exécution est surement correcte ou surement fautive, un verdict peut être rendu. Le souci vient des exécutions ambiguës. Plusieurs définitions de diagnostiquabilité peuvent être proposées. Pour des systèmes non probabilistes, la définition originelle de diagnostiquabilité requiert que toute séquence fautive devienne finalement surement fautive [SSL + 95]. Ainsi, toute faute est finalement détectée. Cette condition est trop forte pour des systèmes probabilistes car un système pourrait être déclaré non diagnostiquable à cause d'une exécution de probabilité nulle. Une adaptation possible est de demander qu'avec probabilité 1 une exécution fautive devienne surement fautive [START_REF] Thorsley | Diagnosability of stochastic discrete-event systems[END_REF]. Nous appelons cette notion FFdiagnotiquabilité. Cette notion ignore l'ambiguïté des exécutions correctes. Le système peut donc rester ambigü infiniment, ce que l'on peut vouloir éviter. La diagnostiquabilité peut être étendue aux exécutions correctes en requérant que la probabilité des séquences ambigües (fautives et correctes) converge vers 0. Cette notion se nomme FAdiagnostiquabilité (le A signifie "all" alors que F signifie fautif). Pour ces deux notions de diagnostiquabilité, l'ambiguïté est résolue sur des exécutions finies (le premier F signifi-ant justement fini). Si l'on autorise l'ambiguïté à être résolue lorsque la séquence devient infinie, la FFet la FA-diagnotiquabilité deviennent la IFet la IA-diagnotiquabilité. Les quatre notions sont résumées dans la figure 0.2.

q 0 q 1 f 1 f , 0.01
Ces notions de diagnostiquabilité ne permettent aucune erreur de verdict. Ce sont des notions dites exactes. Il peut cependant être intéressant d'affaiblir cette condition. Considérons le pLTS de la figure 0.3. Toute exécution fautive est ambiguë. Cependant, de par le choix des probabilités, une exécution fautive a plus de chances de produire un 'b' qu'un 'a' et inversement pour les exécutions correctes. Par conséquent, en comparant le nombre de 'b' et de 'a' dans l'observation, on peut déduire avec forte probabilité si l'exécution est correcte ou fautive. Nous considérons donc plusieurs notions de diagnostiquabilité dites approchées dans ce document (l'une d'entre elle ayant été introduite dans [START_REF] Thorsley | Diagnosability of stochastic discrete-event systems[END_REF] et pour laquelle seule une condition suffisante avait été donnée). La formalisation du modèle et des notions de diagnostiquabilité est réalisée dans le chapitre 2. Nous expliquons maintenant comment ces notions de diagnostiquabilité peuvent être étudiées.

q 0 q f q c f , 1 2 u, 1

Vérification de la diagnostiquabilité

L'étude d'un problème commence par une analyse sémantique (développées dans le chapitre 3) de façon à bien le comprendre. Dans le cas de la diagnostiquabilité, cette analyse sémantique prend tout d'abord la forme d'une étude des liens entre les différentes notions de diagnostiquabilité. Certains sont assez clairs. Par exemple, les notions de FAet IA-diagnostiquabilité considèrent les exécutions fautives ainsi que les exécutions correctes alors que FFet IF-diagnostiquabilité ne considèrent que les notions fautives. Donc un système FA-diagnostiquable (resp. IA-) est FF-diagnostiquable (resp. IF). Similairement, observer des exécutions infinies donne plus d'informations que leurs préfixes finis, donc les notions de diagnostiquabilité finies impliquent leur équivalent infini. De façon intéressante, si on ne s'intéresse qu'aux exécutions fautives, on a une réciproque partielle : si le système est à branchement fini, la FF-diagnostiquabilité est équivalente à la IF-diagnostiquabilité.

La deuxième étape d'une analyse sémantique est de déterminer des caractérisations efficaces des notions étudiées. Avoir des contraintes sur le système étudié permet d'avoir des caractérisations plus simple. Nous étudions donc d'abord le cas des systèmes ayant un nombre d'états fini.

Étude des systèmes finis

Les systèmes représenté par un modèle ayant un nombre d'états fini possèdent des propriétés extrêmement utiles pour le diagnostic. Notamment, on sait qu'avec probabilité 1, une exécution atteint une composante strictement connexe terminale (CSCT) du graphe induit par le pLTS. Comme la diagnostiquabilité s'intéresse à des comportements "avec probabilité 1", l'étude peut se concentrer sur les CSCT du système. Par ailleurs, il existe une méthode simple de déterminisation de l'automate induit par le pLTS. Cette déterminisation est très utile pour caractériser l'ambiguïté d'une exécution. En effet, l'automate déterminisé associe à chaque séquence d'observations l'ensemble d'états du pLTS pouvant être atteint par des exécutions associées à cette séquence. Par conséquent, en supposant sans perte de généralité que les états du pLTS sont partitionnés entre états fautifs (atteint par une exécution fautive) et états corrects, une exécution surement fautive est une exécution dont la séquence d'observation mène à un état de l'automate déterminisé ne possédant que des états fautifs. Observons ceci sur un exemple. La figure 0.4 représente un pLTS qui est FF-diagnostiquable ainsi que IA-diagnostiquable mais qui n'est pas FA-diagnostiquable. En effet, toute faute est suivie finalement par un 'b' révélant la faute. L'observation a ω est donc au contraire associée à une exécution surement correcte, mais tout préfixe fini de cette exécution est ambigu. Donc toute exécution correcte finie est ambiguë. Le pLTS de la figure 0.4 respecte la partition entre états fautifs (les f i ) et états corrects (les q i ) mentionnée plus tôt. Nous représentons l'automate déterminisé induit en figure 0.5. Un état de cet automate atteint par l'observation w contient deux ensembles : nous séparons les états du pLTS pouvant être atteint par une exécution correcte de ceux atteints par une exécution fautive, de plus on ne considère que les exécutions terminant par un évènement observable. Les états doublement entourés ne contiennent soit aucun état correct, soit aucun état fautif. Les observations menant à ces états correspondent donc à des exécutions non ambiguës.

q 0 q 2 f 1 f 2 q 1 u, 1 2 f , 1 2 a, 1 2 u, 1 2 a, 1 b, 1 b, 1
En observant cet automate et au vu de notre remarque antérieure sur les CFCT, on pourrait penser que le diagnostic de ce système est simple étant donné que le seul état pour lequel le verdict ne peut être rendu ne fait pas parti d'une CFCT de l'automate déterminisé. Cependant, si les CFCT sont atteintes avec probabilité 1 dans le pLTS, ce n'est pas forcément le cas dans l'automate déterminisé. Afin de regagner cette propriété tout en conservant les informations données par l'automate déterminisé, on réalise le produit synchronisé sur les évènements observables du pLTS et de l'automate. Celui-ci est représenté dans la figure 0.6. q 0 , s 0 q 2 , s 0 q 2 , s 1 Le produit synchronisé conserve le comportement probabiliste du système. C'est-àdire, qu'il y a une bijection entre les exécutions du pLTS et celles du produit, les deux exécutions ayant la même observation, correction et probabilité. Par conséquent, la diagnostiquabilité du pLTS est équivalente à celle de son produit synchronisé. Par contre, ce dernier possède plus d'informations car la composante des états correspondants à l'automate déterminisé indique si une exécution finissant dans cet état est surement fautive, surement correcte ou ambigüe. Observons les CFCT de ce système. Il y en a 3, toutes réduites à un état: (q 1 , s 1 ), (f 2 , s 2 ) et (f 2 , s 4 ). Les ensembles s 2 et s 4 impliquent que les exécutions atteignant la deuxième et troisième CFCT sont surement fautives. s 1 en revanche, montre une ambiguïté dans la première CFCT. Comme s 1 est associé à un état correct q 1 , cette ambiguïté n'existe que pour des exécutions correctes. Ce produit synchronisé montre donc que le pLTS est bien FF-diagnostiquable, mais pas FA-diagnostiquable.

{q 0 }, ∅ s 0 {q 1 , q 2 }, {f 2 } s 1 ∅, {f 1 , f 2 } s 2 ∅, {f 1 } s 3 ∅, {f 2 } s 4
f 1 , s 0 f 1 , s 1 f 1 , s 2 f 1 , s 3 f 2 , s 1 f 2 , s 2 f 2 , s 4 q 1 , s 0 q 1 ,
En utilisant le produit synchronisé, une caractérisation peut être établie pour toutes les notions de diagnostiquabilité exacte1 . Cette caractérisation peut ensuite être vérifiée en espace polynomial. La principale source de complexité de l'algorithme est la déterminisation qui produit un automate de taille au plus exponentielle en la taille du pLTS. Utilisant une réduction du problème de l'universalité du langage généré par un automate non déterministe, on montre que les notions de diagnostiquabilité exacte sont PSPACEdifficiles. Elles sont donc PSPACE-complètes pour les pLTS finis.

En ce qui concerne le diagnostic approché, seule une notion, construite en modifiant subtilement celle introduite dans [START_REF] Thorsley | Diagnosability of stochastic discrete-event systems[END_REF], est décidable. La décidabilité est montrée en réduisant le problème à un nombre au plus quadratique d'instances du problème de la distance 1 de deux chaînes de Markov étiquetées, problème qui a été montré décidable en PTIME dans [START_REF] Chen | On the total variation distance of labelled Markov chains[END_REF]. Ceci donne in fine un algorithme PTIME. Les résultats d'indécidabilité quant à eux sont montrés grâce à des réductions du problème du vide des automates probabilistes [START_REF] Paz | Introduction to probabilistic automata[END_REF].

Construction des diagnostiqueurs

Le but de l'étude du diagnostic est la détection automatique de la faute. Cette détection est réalisée par un diagnostiqueur qui observe le système et donne son verdict. Formellement, un diagnostiqueur est une fonction D : Σ * o → {?, , ⊥}. Un verdict ? ne fournit aucune information, un verdict déclare l'exécution actuelle fautive et un verdict ⊥ fournit une information relative à la correction de l'exécution. Un diagnostiqueur a trois caractéristiques principales: verdict, sureté et réactivité. Le verdict formule la nature de l'information que le diagnostiqueur doit fournir au cours d'une exécution (détection de fautes uniquement, ou aussi de la correction de l'exécution par exemple). La sureté formule quand le diagnostiqueur peut émettre son verdict. Dans le cas du diagnostic exact, la sureté requiert que si le diagnostiqueur produit un verdict, celui-ci est correct. Ce n'est pas forcément le cas dans le cadre du diagnostic approché. La réactivité exprime à quelle régularité le diagnostiqueur doit fournir des informations sur le statut de l'exécution courante.

Les notions de diagnostiquabilité ayant été présentées sous la forme d'un problème de décision, la troisième étape de l'étude sémantique consiste à établir le lien entre chaque notion de diagnostiquabilité et l'existence d'un diagnostiqueur avec un verdict, une sureté et une réactivé donnés. La preuve permettant d'établir le lien entre diagnostiquabilité et existence d'un diagnostiqueur est constructive. Par conséquent, nous disposons d'un algorithme permettant de construire automatiquement un diagnostiqueur pour chaque système diagnostiquable. Cependant, le diagnostiqueur en question utilise une mémoire non bornée. En vue d'une possible implémentation, il est préférable de pouvoir se limiter à des diagnostiqueurs à mémoire finie. Un tel diagnostiqueur est représenté par un automate déterministe sur Σ o enrichi d'un verdict (M, Σ o , m 0 , up, D fm ) où M est un ensemble d'états de la mémoire avec m 0 l'état initial, up est la fonction de tran-sition, mettant à jour la mémoire du diagnostiqueur et D fm : M → {?, , ⊥} associe un verdict à chaque état de la mémoire. La taille d'un tel diagnostiqueur à mémoire finie est le nombre d'états de la mémoire qu'il possède. La fonction de mise à jour peut être étendue à des séquences d'observations de façon inductive : pour ε le mot vide w ∈ Σ * o et a ∈ Σ o , up(m, ε) = m et up(m, wa) = up(up(m, w), a). Si un diagnostiqueur à mémoire finie n'est pas un diagnostiqueur selon la définition établie plus haut, il en induit un qui est définit par D(w) = D fm (up(m 0 , w)) pour tout w ∈ Σ * o .

Example 0.2. La figure 0.7 représente un diagnostiqueur à mémoire finie qui ne fournit aucune information (verdict ?) initialement puis déclare une faute (verdict ) dès qu'un 'b' est observé. Considérons le pLTS de la figure 0.4. La faute est identifiée dès qu'un 'b' est observé. Par conséquent, le diagnostiqueur induit par le diagnostiqueur à mémoire finie de la figure 0.7 peut être utilisé pour détecter les fautes de ce pLTS. Ce diagnostiqueur ne détecte pas les exécutions correctes, il correspond à la notion de FF-diagnostiquabilité. On l'appelle donc un FF-diagnostiqueur. Pour toute notion de diagnostic exact, un diagnostiqueur peut être construit à partir de l'automate déterminisé qui a été utilisé pour vérifier la diagnostiquabilité. Le diagnostiqueur peut donc avoir une taille exponentielle. C'est malheureusement inévitable : certains pLTS n'admettent pas de diagnostiqueurs de taille sous-exponentielle. Pour le diagnostic approché, aucune borne sur la taille de la mémoire n'existe dans le cas général. En d'autres mots, on ne peut pas toujours construire de diagnostiqueur à mémoire finie. Pire encore, déterminer s'il existe un diagnostiqueur à mémoire finie est un problème indécidable.

Ces résultats, reposant fortement sur la limitation à un nombre d'états finis pour le modèle, sont rassemblés dans le chapitre 4. Nous allons maintenant discuter de ce qui peut être fait pour lever cette restriction.

Étude des systèmes infinis

De nombreux systèmes réels nécessitent un nombre infini d'états pour être décrit de façon précise. Afin de rendre une analyse possible, on ne peut pas utiliser directement un pLTS infini. On a besoin d'un modèle de plus haut niveau, capable de représenter de façon finie un pLTS infini. De nombreux formalismes permettent ceci. Les automates à pile et les réseaux de Petri notamment représentent deux classes orthogonales de LTS infinis. Le choix du formalisme est très important car plus un formalisme est expressif, plus les problèmes seront compliqués à résoudre. Les automates à pile et les réseaux de Petri sont trop puissants par exemple, toutes les notions de diagnostiquabilité exacte étant indécidables dans ces deux formalismes.

Nous nous sommes donc intéressés à un formalisme légèrement plus faible, plus précisément une sous-classe des automates à pile : les automates à pile visibles. Les automates à pile sont des automates enrichis d'une pile qui leur permet de conserver de l'information au cours d'une exécution. Les transitions de l'automate disponibles dépendent de l'état courant ainsi que de la tête de la pile. Une fois sélectionnée, une transition peut soit (1) modifier la tête de la pile (transition locale), (2) ajouter un nouveau symbole en tête de pile (transition d'empilement) ou (3) retirer la tête de pile actuelle, s'il y en a une (transition de dépilage). La sémantique d'un automate à pile probabiliste est un pLTS infini dont les états représentent l'état actuel de l'automate à pile ainsi que le contenu de la pile. Ce pLTS peut être infini car la taille de la pile n'est pas bornée. La restriction aux automates à pile visible requiert que l'ensemble des évènements est partitionné selon le type de transitions auquel ils correspondent, Σ = Σ ∪ Σ ∪ Σ . Les évènements de Σ , Σ et Σ sont respectivement associés à des transitions d'empilement, de dépilage et des transitions locales. De plus, Σ ∪ Σ ⊆ Σ o . Un observateur voit donc quand un élément est ajouté ou retiré de la pile. La taille de la pile est donc connue à tout moment, son contenu par contre peut être inconnu.

Example 0.3. La figure 0.8 donne un exemple d'automate à pile probabiliste. Une exécution démarre dans l'état q 0 avec pour contenu de pile le symbole ⊥ 0 . Celui-ci est appelé élément de fond de pile et ne peut pas être retiré ou modifié. Le reste de la pile n'est composé que d'un certain nombre de γ. La seule transition d'empilement est in et les transitions de dépilage sont out et abort.

Ce système reçoit un certain nombre d'ordres qu'il note dans la pile. Puis il commence à servir ses clients, chaque ordre reçu reçoit donc une réponse soit sous la forme d'un out, soit de façon fautive sous la forme d'un abort. Enfin, il retourne à son point initial. Les deux exécutions données en exemple présentent un comportement correct et fautif pour deux ordres reçus.

Il y a une claire partition entre transitions d'empilement, de dépilage et locales. Par conséquent si in, out et abort sont observables, cet automate à pile probabiliste est visible.

La restriction aux automates à pile probabilistes visibles limite peu l'expressivité du formalisme, mais donne des propriétés supplémentaires utiles au modèle, notamment elle permet de réaliser une déterminisation de l'automate à pile. On ne peut cependant pas faire comme dans les systèmes finis et étudier les CFCT du produit du modèle et de son automate déterminisé. Notamment car ces CFCT peuvent ne pas exister (cas où aucune borne sur la taille de la pile n'existe). Une autre forme de caractérisation de la diagnostiquabilité est donc nécessaire. Pour ce faire, une logique nommée pathL a été introduite et des formules caractérisant plusieurs notions de diagnostiquabilité exacte ont été établi. Ensuite, en utilisant notamment l'automate à pile déterminisé, ces formules ont pu être traduites en des formules de pLTL, une logique connue et pour laquelle des algorithmes de vérification existe pour les automates à pile. En utilisant ces résultats nous avons obtenu des algorithmes EXPSPACE pour les notions de diagnostiquabilité caractérisées (la borne inférieure prouvée étant EXPTIME). La FA-diagnostiquabilité, qui requiert la détection des séquences fautives et correctes en temps fini n'a cependant pas pu être caractérisée. En fait, des résultats de non-expressivité ont été établi pour montrer que cette notion ne pouvait pas être exprimée en pathL.

q 0 q 1 f 1 1 2 • γ, serve, γ 1 • ⊥ 0 , empty, ⊥ 0 1 2 • γ, f , γ 1 • ⊥ 0 , reset, ⊥ 0 1 2 • γ, in, γγ 1 • ⊥ 0 , in, ⊥ 0 γ 1 2 • γ, out, ε 1 • γ, abort, ε (q 0 , ⊥ 0 ) (q 0 , γ ⊥ 0 ) (q 0 , γ γ ⊥ 0 ) (q 1 , γ γ ⊥ 0 ) (q 1 , γ ⊥ 0 ) (q 1 , ⊥ 0 ) (q 0 , ⊥ 0 ) (f 1 , γ γ ⊥ 0 ) (f 1 , γ ⊥ 0 ) (f 1 , ⊥ 0 ) (q 0 , ⊥ 0 )
Ces résultats sont développés dans le chapitre 5.

Contrôle d'un système

Les pLTS donnent une représentation passive d'un système. Ainsi, étudier le diagnostic sur des pLTS est purement un travail de vérification. Il ne permet donc pas de questionner efficacement comment modifier le système afin de le rendre diagnostiquable. Afin d'étudier ce genre de problème, on ajoute une forme de contrôle dans le pLTS. Le formalisme obtenu, appelé CLTS partitionne l'ensemble des évènements observables en évènements contrôlables Σ c et évènements incontrôlables Σ e . Après chaque observation, un contrôleur choisit un ensemble d'évènements autorisés excluant potentiellement certains évènements contrôlables. Ceci limite donc les transitions pouvant être prise par le système.

Example 0.4. Un exemple de CLTS est représenté dans la figure 0.9. Initialement, deux évènements de poids 1 sont possibles et forcément autorisés par le contrôle. Par conséquent chacune a une probabilité 1 2 d'être prise. En q 1 , 'b' peut être interdit par le contrôleur. Si c'est le cas, la transition étiquetée par 'a' est choisie avec probabilité 1.

q 0 q 1 q 2 q 3 f 1 u, 1 b, 1 f , 1 f , 1 a, 1 b, 1 a, 1 a, 1
Ainsi, si le contrôleur autorise tous les évènements en permanence, l'exécution q 0 uq 1 aq 1 bq 2 a probabilité 1/8. S'il interdit 'b' en permanence, cette exécution a probabilité 0. Finalement, s'il n'autorise un 'b' qu'après l'observation d'un 'a', l'exécution a probabilité 1/4.

Le contrôle est formellement défini par l'utilisation de stratégies. Une stratégie π : Σ * o → Dist(2 Σ ) est une fonction associant à une séquence d'observations une distribution probabiliste sur les ensembles d'évènements autorisés. Par ailleurs, si l'ensemble Σ • est sélectionné par la stratégie, on a Σ u ∪ Σ e ⊆ Σ • . En d'autres mots, la stratégie ne peut exclure que des évènements contrôlables. Un CLTS C équipé d'une stratégie π génère un pLTS infini dénoté C π .

Example 0.5. Considérons le CLTS C représenté dans la figure 0.9. Il y a deux ensembles d'évènements possibles à autoriser: Σ et Σ \ {b} que nous abrégeons en Σ -. Définissons la stratégie π par π(a n ) = p n • Σ -+ r n • Σ avec p n + r n = 1 pour tout n ∈ N et π(w) = 1•Σ sinon. C'est-à-dire, après avoir observé a n , avec probabilité p n l'ensemble Σ -est autorisé par la stratégie et avec la probabilité complémentaire Σ est autorisé. Le pLTS généré C π est infini. Une partie de celui-ci est représenté en figure 0.10.

Expliquons la distribution de probabilité à la sortie de la configuration (ε, q 1 , Σ). Les deux transitions sortant de q 1 ont le même poids, comme elles sont toutes deux autorisées par la stratégie, la probabilité de chaque transition est 1 2 . Comme 'a' et 'b' sont observables, un nouveau contrôle est choisi. Si un 'b' est observé, de par la définition de π, le nouveau contrôle est Σ. Par contre, si un 'a' est observé, le nouveau contrôle est Σ -avec probabilité p 1 et Σ sinon. Il y a donc trois transitions sortantes de (ε, q 1 , Σ) ayant pour probabilité respectivement 0.5, 0.5p 1 et 0.5r 1 .

Nous discutons maintenant des questions abordées pour les CLTS.

q 0π ε, q 0 , Σ - ε, q 0 , Σ ε, q 1 , Σ - ε, f 1 , Σ - ε, f 1 , Σ ε, q 1 , Σ b, q 2 , Σ a, q 1 , Σ - a, q 1 , Σ a, f 1 , Σ - a, f 1 , Σ b, f 1 , Σ • • • • • • • • • • • • • • • u, p 0 u,

Diagnostic actif et dégradation

Lorsque l'on étudie le diagnostic d'un CLTS, la question n'est plus si le CLTS est diagnostiquable (ce qui n'a techniquement pas de sens en soi) mais s'il existe une stratégie telle que le pLTS induit est diagnostiquable. Cette problématique a été étudiée pour des systèmes probabilistes dans [BFH + 14]. Afin de déterminer s'il existe une stratégie satisfaisant la notion de diagnostiquabilité qu'ils étudient, ils traduisent le problème du diagnostic en une condition de Büchi pour un processus de décision Markovien partiellement observable. Le problème peut ensuite être résolu avec des techniques connues. La stratégie obtenue a de bonnes propriétés : elle est notamment ce qu'on appelle "basée sur la croyance", ce qui implique entre autres que le pLTS généré est fini. Leur travail soulève un souci important : afin de rendre le système diagnostiquable, la stratégie peut faire des choix problématiques comme forcer l'occurrence d'une faute. Bien que ceci permette de détecter le mauvais comportements du système, cela va à l'encontre du but initial du diagnostic qui est de pouvoir utiliser un système fonctionnel. Ils introduisent donc le problème du diagnostic sûr. Celui-ci demande s'il existe une stratégie satisfaisant à la fois le diagnostic et assurant une probabilité positive aux exécutions correctes. Cette notion est malheureusement indécidable dans le cas général et un algorithme NEXPTIME est donné dans le cadre limité des stratégies à mémoire finie (i.e. pour lesquelles le pLTS engendré est fini).

Continuant sur cette idée, nous avons introduit de nouvelles notions permettant de mesurer la dégradation du système. Ces notions ne s'assurent pas que le système reste correct infiniment avec une probabilité positive comme le diagnostic sûr, mais demandent que, si faute il y a, celle-ci puisse être retardée fortement. Le diagnostic longtemps correct et le diagnostic fortement résistant sont deux telles notions, mesurant différemment le délai à imposer à l'occurrence de la faute. Ces deux notions sont impliquées par le diagnostic sûr et sont différentes quand appliquées à des pLTS infinies (i.e. il existe des pLTS infinis satisfaisant chacune des notions sans satisfaire l'autre). Parmi les notions de dégradation que nous avons introduites, certaines sont quantitatives, d'autres, comme le diagnostic longtemps correct et le diagnostic fortement résistant, sont qualitatives. Les notions quantitatives sont toutes indécidables, même limitées à des stratégies à mémoire finie. Au contraire, des algorithmes ont pu être établis pour les notions qualitatives. Ceux-ci procèdent en deux étapes. Tout d'abord et en enrichissant les états du CLTS comme fait dans le cas passif grâce à une déterminisation du CLTS, on identifie l'ensemble des états du CLTS enrichi que l'on peut visiter tout en respectant la diagnostiquabilité. Cette méthode permet en fait de construire la stratégie la plus permissive assurant la diagnostiquabilité du système. Dans un second temps, on étudie le CLTS réduit aux états accessibles sous cette stratégie et on identifie comment restreindre la stratégie afin de rester suffisamment longtemps dans une exécution correcte. Cette étude peut se réaliser en EXPTIME. Le diagnostic actif étant EXPTIME-difficile, les notions qualitative de dégradation introduites sont donc EXPTIME-complètes dans le cas général. Nous avons également montré comment réduire à EXPTIME la complexité du diagnostic sûr limité aux stratégies à mémoire finie. Sous cette restriction, le diagnostic sûr est donc EXPTIME-complet également.

Ces travaux sont présentés dans le chapitre 6.

Assurer l'opacité d'un système

Dans cette thèse, le diagnostic est le problème d'observation partielle auquel nous avons prêté le plus d'attention. D'autres problèmes d'observation partielle sont également intéressants à étudier, notamment l'opacité que nous étudions formellement dans le chapitre 7. Le but de l'opacité est de cacher une information à l'observateur. En conséquent, sur bien des aspects cette notion apparait comme un dual du diagnostic. Formellement, un système possède deux types d'exécutions : publiques ou secrètes. Pour déterminer, si une exécution est secrète, on pourrait faire comme pour le diagnostic et utiliser un évènement particulier qui, s'il est présent dans une exécution, la rend secrète. De façon équivalente, ceci peut être représenté en partitionnant l'ensemble des états du système en états publics et états secrets et en considérant ces derniers absorbants. Une exécution est secrète ici si elle visite un état secret. C'est cette seconde option que nous utilisons pour l'opacité. Parmi les exécutions secrètes, certaines révèlent le secret. Ce sont celles pour lesquelles toute exécution ayant la même observation est secrète. Notons le parallèle avec le diagnostic : en considérant les exécutions secrètes comme fautives, une telle exécution serait appelée surement fautive. Lorsqu'on étudie l'opacité d'un système, nous désirons mesurer à quel point le secret est révélé, c'est-àdire quelle est la mesure de probabilité des exécutions révélant le secret. Cette mesure est appelée la révélation.

L'opacité a été étudiée dans un contexte passif, ainsi que dans un contexte actif. Le contrôle utilisé dans l'étude de l'opacité est fortement différent de celui utilisé pour le diagnostic. En effet, le contrôleur est une fonction associant à une exécution (et non à une séquence d'observations) une distribution sur un ensemble d'action. Chaque action correspond à une distribution de probabilité sur un ensemble de transitions. Ce formalisme donne beaucoup plus de puissance au contrôleur. Il connait précisément quelle est l'exécution actuelle, et son choix n'est pas forcément limité à quels évènements contrôlables il autorise, mais à un ensemble d'actions décrivant potentiellement des choix plus complexes.

Autre différence avec les CLTS, dans le cadre de l'opacité, les observations ne sont plus mises sur les transitions, mais sur les états. Pour noter ces différences, on parlera d'OMC pour les systèmes passifs et d'OMDP pour les systèmes contrôlables.

Example 0.6. Considérons l'OMC représenté en figure 0.11. L'observation associée avec chaque état est indiquée à côté de celui-ci. Les états secrets sont indiqués en grisé. En supposant que o 1 et o 2 sont deux observations autre que ε, toute exécution contenant au moins 3 observations révèle le secret. La révélation est donc de 1. Considérons maintenant l'OMDP de la figure 0.12. Dans l'état initial q 0 , deux actions sont possibles. Si l'action 'a' est choisie, l'exécution entre en q 1 avec probabilité 1 2 et en q 2 avec la même probabilité. Si 'b' est choisie, tous les états ont une probabilité 1 3 d'être atteint.

q 0 o 2 q 0 bq 0 o 2 q 0 bq 2 o 2 q 0 bq 1 o 1 q 0 bq 0 aq 2 o 2 q 0 bq 0 aq 1 o 1 1 3 1 3 1 3 1 2 1 2
Définissons la stratégie π choisissant initialement l'action 'b', puis toujours l'action 'a'. L'OMC induit par ce OMDP contrôlé par la stratégie π est celui représenté dans la figure 0.11. Ainsi, en utilisant la stratégie π, on assure une révélation de 1. En utilisant une stratégie π qui sélectionne 'b' à tout les coups, la révélation n'aurait été que de 1 2 . Intuitivement, lorsqu'on contrôle un système pour augmenter la révélation, on réalise une analyse du pire cas pour le système. Dans la réalité, ce pire cas est atteint si, par exemple, le contrôle est effectué par un virus ayant accaparé certaines fonctionnalités du système. L'inverse, quand le contrôle cherche à minimiser la révélation, est aussi intéressant à étudier. Cela représente, par exemple, le cas où un concepteur possède quelques degrés de liberté dans son système et désire choisir l'option qui maximisera l'opacité du système. Ces deux problèmes semblent symétriques au premier abord, mais leur analyse est en fait extrêmement différente.

q 0 o 2 q 2 o 2 q 1 o 1 a, 1 2 | b, 1 3 a, 1 2 | b, 1 3 b, 1 3 a, 1 a, 1
Pour la maximisation de la révélation, une simplification est possible au niveau des stratégies : les stratégies déterministes sont suffisantes. C'est-à-dire, afin de maximiser la révélation, on peut se contenter de considérer des stratégies qui associent à chaque exécution finie non pas une distribution sur les actions, mais directement une action. Malgré cette simplification cependant, presque tous les problèmes sont indécidables. La seule question importante que l'on peut résoudre est : étant donné une OMDP M, existe t-il une stratégie π telle que l'OMC induite M π a une révélation de 1. Ce problème peut se traduire en un problème d'accessibilité avec probabilité 1 dans un processus de décision Markovien partiellement observable, problème pour lequel des algorithmes efficaces existent. Le processus construit est de taille exponentielle et l'algorithme résolvant le problème d'accessibilité est en EXPTIME. Par conséquent une application directe donne un algorithme 2EXPTIME. Cependant, une analyse précise montre qu'une seule exponentielle est nécessaire. En effet, la complexité de l'algorithme vient principalement de l'utilisation d'une forme de déterminisation du processus de décision Markovien partiellement observable. Hors, celle-ci est déjà nécessaire à la transformation de l'OMDP vers le processus et n'a donc pas à être répétée.

Conclusion

Cette thèse présente principalement une analyse des problèmes liés au diagnostic de systèmes probabilistes. Sa première contribution est de rassembler en un tout cohérent les différentes définitions existantes sur ce problème. Ceci permet à la fois de donner une base solide à la recherche présentée ici, et de servir de fondations à toute recherche future sur ce sujet. En deuxième point, cette thèse explique comment vérifier les notions de diagnostiquabilité définies pour différents systèmes, que ceux-ci soient constitués d'un nombre d'états fini ou infini. Pour les systèmes infinis, la décidabilité de certaines notions reste ouverte et certains algorithmes ne sont pas prouvés optimaux. Il reste donc du travail

Part I

Chapter 1

General Introduction

Model-based verification. Many critical systems must fulfil a given specification. This specification may include security criteria, efficiency measures or other kinds of requirements. The verification process, which checks if the system respects the specifications, can be performed in several manners. Each one having its pros and cons, the choice of the best method to use strongly depends on the kind of systems to be verified and on the specification. One of the possible method of verification is to perform tests on the system. If one wants to test a program for example, yet does not have access to the internal structures or working of it, the tests can be realised through specificationbased testing [START_REF] Gao | Testing and quality assurance for component-based software[END_REF]. In this method, one considers the program as a black-box and focuses on the specification in order to determine which inputs are the most likely to show a failure of the system. If one has access to the content of the program, one can build a formal and operational model of the system. Then this model can be analysed via dedicated methods.

Building the model may be difficult for some complex systems. It can be done by analysing the code of the system, by making specific tests in its different states to understand its evolution (for example by overloading the CPU to see how a program reacts faced to this kind of stress), etc. When possible, this approach has many benefits:

• When designing a system, if the current prototype does not satisfy our goals, it must be modified. Building iteratively a new prototype until one gets a good result is expensive. It is easier and cheaper to modify a model of the system until it satisfies the requirements and only then implement the system.

• A model is often built for checking several properties. If one wishes to check for additional properties at a later date, verifying the existing model may be sufficient.

If not, one does not necessarily have to build a fully new model. One only needs to refine the existing one with the appropriate, missing information, which strongly reduces the complexity.

• Finally, if the model is close enough to reality, then it allows for an accurate analysis of the system. This is not the case when using arrays of tests which only

General Introduction

partially cover the range of possibilities. The same issue exists for other methods such as statistical model checking [START_REF] Barbot | Acceleration for statistical model checking[END_REF].

Probabilistic models. There exist several different formalisms for representing a system. The more complete the formalism is (by adding time, multiple players. . . ), the more systems and specifications can be described in it, but also the more complex it is to study.

In particular, stochastic models such as Markov chains [START_REF] John | Finite Markov chains[END_REF] or Markov decision processes [START_REF] Puterman | Markov decision processes: discrete stochastic dynamic programming[END_REF] have many applications. There are some systems that require probabilities in order to be accurately represented. For example, they can be used to represent systems that contains inherent random behaviours. This occurs in any program using randomisation in order to break symmetries for instance, such as the algorithms dealing with the consensus problem [START_REF] Aguilera | Stumbling over consensus research: misunderstandings and issues, in Replication: theory and practice[END_REF]. The randomisation also appears in the processes used in the consensus problem as one might represent the possibility that these processes fail with some probability. Another example of application of stochastic models is the case of systems that face unpredictable behaviours from the environment. This can be the case for a server that receives requests. These requests have randomised content and their timing of arrivals can also be random. The latter requires to mix probabilities and time in the model, as in stochastic timed automaton [BBB + 14]. Moreover, probabilities can also be used in the model to represent the uncertainty created when the modelling is done through a statistical analysis.

Using probabilities also enlarges the set of properties that can be specified by giving a measure on the runs of the system. For example, if a non-critical system possesses failures, yet one can determine that they are not likely to occur, this may be enough. Let us also consider a security example: if an attacker tries a password and discovers that it is wrong, he technically gets an information, however this information is not important enough to be worrisome. With probabilities, the specification can quantify the properties of the system the designer wants to verify. Moreover, even a qualitative quantification is useful as it allows to neglect behaviours that are present in the model, yet have a zero probability of occurring.

Paradigms of partial observation. Another important component of a system that can be modelled is related to the observation available to the users: when one builds a model, one describes the different actions that can be taken by the system, however, these actions may be internal and are not necessarily visible by an external observer. Managing the information exchanged with a system has shown increasing importance in recent years due to the omnipresence of communicating electronic devices. Some of the actions of the system may need to be kept private (passwords) while others must be made public (failures). The problems raised by partial observation can be grouped in three families thanks to the different types of goals the system has: (1) planning under partial observation, (2) hiding information from the observer and (3) getting information from the system.

This first category appears for example when studying games. In a game of poker, a player has to select a decision based on some observations (his cards) and on some partial information (his opponents choices), moreover probabilities are involved in order to determine the likelihood to draw a specific card. Such a case falls under the study of works such as [START_REF] Bertrand | Qualitative determinacy and decidability of stochastic games with signals[END_REF] where the authors look for almost-surely winning or positively winning strategies in stochastic games. Partially observable Markov decision process (POMDP) is the stochastic one-player (also called one and a half player) special case of the above (See [START_REF] Chatterjee | Qualitative analysis of partially-observable Markov decision processes[END_REF] for algorithms to achieve qualitative objectives in POMDP). POMDP have been extensively used in the IA community, for example in order to plan the actions of a moving robot [START_REF] Kaelbling | Planning and acting in partially observable stochastic domains[END_REF].

Many works focus on hiding information from an attacker. For instance code obfuscation [BGI + 01], albeit in a complete observation setting. In a partial observation setting, many theoretical hiding problems are gathered under the general name "opacity". An opaque system hides an information by ensuring the observation given by a secret behaviour of the system will be identical to the observation triggered by a non-secret behaviour. A practical example is given in [START_REF] Andrés | Geo-indistinguishability: Differential privacy for location-based systems[END_REF] where the authors investigate how to hide the position of a cellphone user (by randomising the position declared by the phone) while getting relevant answers to location-based requests. In a theoretical and stochastic setting, [START_REF] Bryans | Towards quantitative analysis of opacity[END_REF][START_REF] Bérard | Quantifying opacity[END_REF] defined probabilistic measures of the opacity of a model. The studied model is passive: once defined, one cannot modify its behaviour in order to ensure better properties. A form of control is quickly introduced in [START_REF] Bérard | Quantifying opacity[END_REF] and expanded in [START_REF] Bérard | Probabilistic opacity for Markov decision processes[END_REF][START_REF] Bérard | Preserving opacity on interval Markov chains under simulation[END_REF][START_REF] Bérard | Opacity for linear constraint Markov chains[END_REF]. More precisely, the authors of [START_REF] Bérard | Probabilistic opacity for Markov decision processes[END_REF] investigate Markov decision processes (MDP) with or without partial observation and secrets given by the infinite language of an automaton (using various accepting conditions). In [START_REF] Bérard | Preserving opacity on interval Markov chains under simulation[END_REF][START_REF] Bérard | Opacity for linear constraint Markov chains[END_REF], a model in between MDP and POMDP is used: the control is realised as in an MDP (thus the controller uses complete information) but the winning condition (opacity) uses partial observation and is thus more related to POMDP problems. One issue with these approaches is that they all rely on the black-box hypothesis: it is assumed that the opponent does not know how the non-determinism of the system is resolved. This hypothesis simplifies the problem, but is unrealistic in many cases. For example, the control within the system could be the result of a virus implanted by the attacker. In this case, it is natural for the attacker to know how the virus is implemented.

On the opposite, if the goal is to get information from the system, the first question to answer is to determine the kind of information that must be detected. One possibility is to determine, given a set of partially observable systems, which system is producing the current observation. In [START_REF] Chen | On the total variation distance of labelled Markov chains[END_REF], the authors investigate how far two labelled Markov chains are one to the other in terms of the probabilities of the observed behaviours. They use a distance to measure the importance of the difference between the two models and approximate (or in some cases compute) this distance. The identification of a system can be applied to other questions such as the identification of its initial state, the equivalence of Markov chains (when the distance is equal to 0, see [START_REF] Doyen | Equivalence of labeled markov chains[END_REF]) or the monitoring of hidden Markov chains (HMC). In this last example, one monitor observes a random run of one of two given HMC and must determine which HMC is the origin with appropriately high probability. The case of the monitor required to be correct with probability 1 on infinite sequences was introduced in [SZF11] and solved using the distance of [START_REF] Chen | On the total variation distance of labelled Markov chains[END_REF] in [START_REF] Kiefer | Distinguishing hidden Markov chains[END_REF]. Instead of identifying the current system, one can wish to obtain a specific information from the observations of the current system. We develop this kind of problems in the next part due to their importance within this thesis.

Diagnosis. Diagnosis, from the greek "διάγνωσις" which means "to distinguish" or "to discern", is by the definition of the wiktionary the "identification of the nature and cause of something (of any nature)". This describes many different problems in various domains. In medicine, a doctor analyses the symptoms to deduce the illness causing them. There has been multiple works in order to automatise this kind of diagnosis such as the rule-based expert system MYCIN [START_REF] Buchanan | Rule based expert systems: the MYCIN experiments of the stanford heuristic programming project[END_REF]. Another approach in the medical domain can be found in computer-aided diagnosis [DMK + 99] where the computer analyses medical image such as radios of a patient and points towards the abnormalities it detects. Due to the non-negligible number of false positives and negatives, these works are far from replacing the experts opinion. Forms of diagnosis can also be found in network management for example, where it is more often called fault management. More precisely, fault management has two aspects. The first one, passive, consists in receiving messages from the devices on the network and if an alarm was sent, to understand the cause and react to it. The second one takes an active step by considering that a failing device may not be able to detect its own fault and warn the system. Thus, the fault manager will interact regularly to check the behaviour of the devices. Fault managing therefore requires to do tests, diagnosis and possibly reparation. These diagnosis notions deal with systems that can be extremely complex but that are most often static. One wants to identify the current status of the system from what it is emitting currently, the evolution of the system is not monitored. Diagnosis, as seen in the discrete event systems community, focus in contrast on dynamic systems.

Diagnosis of discrete event systems. For many systems (power systems, manufacturing systems. . . ) one needs to take into account the evolution of the system when analysing it. Such systems can be analysed with the approach from the discrete event systems community. In this approach, while the system is running, one follows a run of the system and tries to deduce the occurrence (or absence of) of a specific event called the fault. While one may want to detect any kind of important action of the system, the term "fault" is chosen both to correspond to the name "diagnosis" which, as shown in the previous examples, is mostly used to detect failures, and because faults are often one of the most important elements to detect within a run. They threaten the safety and availability of the system. In many of the systems listed above, a safety issue may lead to catastrophic damages both in terms of economic and human loses. The study of faults in particular is also justified by the fact that every system may, and will, fail. Indeed, the systems we build are increasingly complex and have increasingly intricate interactions with the environment. It is thus extremely difficult when designing a system not to introduce errors and it is almost impossible to predict every reaction the environment will have to the system. Finally, at the very least, failures will occur because of components ageing.

As faults are dangerous, unavoidable and potentially hard to detect (especially in large-scale complex systems), one needs an automated way to detect them. Moreover this method has to be accurate as stopping a system due to a false positive is costly and it must be reactive so that the failure is detected before too many damages were done. In order to react to the fault, one may either (1) wish to optimise the behaviour of a system in the delay before the occurrence of a fault [START_REF] Ehlers | Risk-averse control of Markov decision processes with ω-regular objectives[END_REF], which is particularly useful for systems which components are automatically replaced on a regular basis, thus hopefully before the occurrence of any fault, or (2) try to detect the fault. As one wants to react quickly to the fault, predicting its occurrence before the system even enters a faulty behaviour would be very efficient. This view is the one studied in prediction problems [START_REF] Genc | Predictability of event occurrences in partiallyobserved discrete-event systems[END_REF]. However, enabling prediction is a very strong requirement for a system. Detecting the fault a posteriori is more likely. The study of diagnosis raises two important issues: deciding whether the system is diagnosable which is called diagnosability and, in the positive case, synthesising a diagnoser possibly satisfying additional requirements about memory size, detection delays, etc. In the discrete event system context, diagnosis was first defined for finite systems such as partially observable Labelled Transition Systems [SSL + 95] then was extended to numerous more complex models (e.g. Petri nets [START_REF] Cabasino | A new approach for diagnosability analysis of petri nets using verifier nets[END_REF][START_REF] Bérard | The complexity of diagnosability and opacity verification for Petri nets[END_REF], pushdown systems [START_REF] Morvan | Diagnosability of pushdown systems[END_REF], etc.) and settings (e.g. decentralised [START_REF] Debouk | Coordinated decentralized protocols for failure diagnosis of discrete event systems[END_REF], distributed [START_REF] Holloway | Time templates for discrete event fault monitoring in manufacturing systems[END_REF]). Also, several contributions, gathered under the generic term of active diagnosis, focus on enforcing the diagnosability of a system [SLT98, TT07, CT08, CP09].

Useful techniques. By observing the previously mentioned works on diagnosis, it appears that some methods and results are recurrent. Let us mention and explain some of them here.

Diagnosability is an hyper property, it cannot be checked by analysing every run of the system separately. On the contrary, some runs are faulty (contain the fault) while the others are correct and we want to compare the observations triggered by faulty runs to the ones produced by correct runs. A key object (e.g. see [START_REF] Jiang | A polynomial algorithm for testing diagnosability of discrete-event systems[END_REF][START_REF] Yoo | Polynomial-time verification of diagnosability of partially observed discrete-event systems[END_REF]) used to decide diagnosability is the twin-plant: a new model is built by making the product of the initial model with itself. A run of the twin-plant consists of a pair of runs of the model. As one wants to compare the observations of the runs, the product is made so that the two runs that are followed simultaneously have the same sequence of observations. This way, one can determine if there exist two runs with the same sequence of observations and some appropriate properties only by checking a single run of the twin-plant. Another often used construction (see [SSL + 95]) is the belief construction. This construction can be seen as an expanded twin-plant or as a form of determinisation of the model: instead of following a pair of runs, the belief automaton instead follows every possible run. More precisely, we keep every state that could be reached with the current sequence of observations, sometimes enhanced with some additional information. This construction is more useful than the twin-plant as it keeps much more information, however it is of exponential size w.r.t. the size of the original model while the twin-plant is only quadratic. The belief automaton by itself gives some information, but can also be used to enrich the initial model. For example, assume the belief automaton of a stochastic model was built, one can now build the product of this model with its belief automaton [START_REF] Vardi | Probabilistic linear-time model checking: An overview of the automata-theoretic approach[END_REF][START_REF] Baier | Principles of model checking[END_REF]. The result, thanks to the determinism of the belief automaton, has the same stochastic behaviour as the original model, however configurations now contain two information: (1) the current state of the associated run in the initial model and (2) the current belief, i.e. the set of states that could be reached with the current sequence of observations. Enriching a model this way is very useful to apply model-checking results. Model checking consists in, given a model of a system, verifying if it satisfies a property, often given by a logical formula. The two most famous logics used are LTL (linear temporal logic) [START_REF] Pnueli | The temporal logic of programs[END_REF] and CTL (computational tree logic) [START_REF] Emerson | Handbook of theoretical computer science[END_REF]. The first one focuses on properties of individual run while the second is mostly interested in branching properties. As diagnosis is not a branching property we only discuss LTL here. In LTL, one can encode formulae about the future of a run, e.g., a condition will eventually be true, a condition will remain true until another one becomes true, etc. The basic components of a formula are propositional variables whose truth value depends, in our framework, on the current state of the model. The more information is contained within a state, the more precise the use of propositional variables can be. How to verify that a "simple" model satisfies an LTL formula is known for a long time [START_REF] Vardi | An automata-theoretic approach to linear temporal logic[END_REF] 1 . Complications occur however when the model is more complex or when the property one wants to check requires more expressive power than what LTL can offer. For stochastic specifications, LTL was extended to pLTL. The extension allows to quantify the measure of the paths satisfying a given LTL formula (the probabilistic operator cannot be nested in the formula). Verifying these formulae is more difficult in terms of complexities. It has been studied both for finite systems [START_REF] Courcoubetis | The complexity of probabilistic verification[END_REF] or infinite ones [START_REF] Etessami | Model checking of recursive probabilistic systems[END_REF]. One can refer to [START_REF] Baier | Principles of model checking[END_REF]Chapter 10] for details about the model checking of probabilistic systems. One important point is that the main source of the complexity of the algorithms is the size of the formula. For example, in [START_REF] Etessami | Model checking of recursive probabilistic systems[END_REF], the qualitative model checking of a recursive Markov chain (a model of infinite-state stochastic system) is PSPACE in the size of the model (and can drop to PTIME under some restrictions) but is EXPTIME in the formula. When studying diagnosis, most problems can be expressed with a simple and fixed formula, which means that the part of the complexity depending on then size of the formula is not our concern.

Another set of techniques that can be used are the results known for POMDP. They have two main interests. First, when studying diagnosis on active systems, some problems can be translated into POMDP problems for which there exist efficient algorithms (as done in [BFH + 14]). The second interest of POMDP comes in fact mostly from probabilistic automata (PA), which are a subclass of POMDP. Many problems are known to be undecidable for PA [START_REF] Paz | Introduction to probabilistic automata[END_REF][START_REF] Gimbert | Probabilistic automata on finite words: Decidable and undecidable problems[END_REF] and due to the simplicity of the PA model, these problems are often easier to use to prove undecidability than problems for POMDP (such as the policy-existence problem under the infinite-horizon average reward criterion [START_REF] Madani | On the undecidability of probabilistic planning and related stochastic optimization problems[END_REF]).

Finally, as there exist many problems using partial observation, the first thing to do when studying a new notion is to check if there already exists a similar problem for which an analysis was realised. In the positive case, one only has to establish a translation. A relevant example of this was mentioned earlier when we stated that [START_REF] Chen | On the total variation distance of labelled Markov chains[END_REF] was used to solve a monitoring problem. The authors of [START_REF] Chen | On the total variation distance of labelled Markov chains[END_REF] established a polynomial algorithm in order to determine if the distance of the language of two labelled Markov chains is equal to 1. In other words, to decide if, almost surely, given an infinite observed sequence, one can determine which labelled Markov chain emitted it. The algorithm relies on the existence of algorithms to detect when two systems have exactly the same language (distance 0) and that if the models are not at distance 1, then a "part" of them is at distance 0. While diagnosis focuses mostly on finite runs and this problem considers infinite runs, strong links can be identified.

Challenges and objectives.

As our global goal is to perform model-based verification, the first question that needs to be tackled in this thesis is the choice of the formalism. This formalism has to include probabilities as we want to be able to quantify the specification. But some points are still open: we must determine whether the model incorporates non-determinism, represents infinitely many states or expresses efficiently concurrent behaviours for example. Moreover, we intend to work on partial observation problems. This requires the model to select what observation is associated with a run.

Our second issue lies in the choice of the problems to focus on. Many notions of diagnosis have been defined over the years with different goals in mind. We have to find a set of appropriate qualitative/quantitative diagnosis notions that encompasses the important, already known, notions that focus on realistic relevant issues, and that is coherent as a whole. Moreover, the formal definitions of the problems we work on must be carefully chosen. Indeed, mixing partial observation, probabilities and control quickly leads to undecidability results (as is the case for PA, mentioned earlier). A slight modification of the definitions may strongly modify the complexity. For example we will see a case, where inverting two quantifiers turn a problem of complexity PTIME to an undecidable one.

Once the properties are defined, our goal is to establish precisely the complexities of verifying if the model satisfies the chosen specification. We are also interested in determining how to modify the system so that it satisfies the properties. This gives two main approaches, a passive one associated with observation and an active one associated with observation and control.

Outline. This thesis is organised as follows.

• In Chapter 2, we introduce notations useful all along the document. While it does not contain any result per se, it includes the definitions of the notions of diagnosis that we introduced. The choice of appropriate definitions is already a contribution. In the second part of this chapter, we present a state of the art on the diagnosis problem.

• In Chapter 3, we realise a semantical analysis of the problems of diagnosability we defined in Chapter 2. More precisely, we first present the notion of diagnoser, i.e. the function realising the diagnosis, and prove the equivalence between the existence of a diagnoser and the diagnosability of a system. We also establish the relations between the various notions of diagnosability and, when possible, characterise these notions. The semantical analysis of a problem as done here is very important as understanding the problems is the first step to solving them. This chapter is based on [START_REF] Bertrand | Foundation of diagnosis and predictability in probabilistic systems[END_REF][START_REF] Bertrand | Accurate approximate diagnosability of stochastic systems[END_REF][START_REF] Bertrand | Diagnosis in infinite-state probabilistic systems[END_REF].

• In Chapter 4, we focus on our simplest model, representing finite stochastic systems. Using the finiteness, we strengthen our characterisations of the diagnosability notions and use them in order to establish algorithms to decide the problems when possible or to prove undecidability in the opposite case. We also show how to build diagnosers using finite memory. This chapter develops contributions from [START_REF] Bertrand | Foundation of diagnosis and predictability in probabilistic systems[END_REF][START_REF] Bertrand | Accurate approximate diagnosability of stochastic systems[END_REF].

• In Chapter 5, we turn to systems with infinitely many states. We cannot use the characterisations obtained in Chapter 4, but the results of Chapter 3 still hold. We study different models and clearly observe the increase in difficulty compared to finite-state models. We still manage to obtain decidability results for one model. This chapter extends [START_REF] Bertrand | Diagnosis in infinite-state probabilistic systems[END_REF].

• In Chapter 6, we consider controllable systems. Using the control, one can ensure properties for the system. However, controlling the system with one objective in mind can have negative side-effects. For example, ensuring diagnosability can increase the likelihood of faults within the system. We are therefore interested in combining multiple objectives, one of them being diagnosability. This chapter is based on [START_REF] Bertrand | Diagnostic et contrôle de la dégradation des systèmes probabilistes[END_REF].

• In Chapter 7, we instead focus on another partial observation problem: opacity, which, on many aspects, appears like a dual of diagnosability. We introduce the notion and explain the impact opacity has on the choice of the framework: we consider here active systems as is done in Chapter 6, however the type of control is different as the controller is not interpreted in the same manner. We define multiple measures of opacity and explain how to maximise or minimise them when possible. The results of this chapter were published in [START_REF] Bérard | Probabilistic disclosure: Maximisation vs. minimisation[END_REF].

Other works. In order to limit the number of frameworks and problems to define and to give a better coherence to the thesis, some of the works we realised are not detailed in this document. We give a short description of these results here. The results of [START_REF] Bertrand | Foundation of diagnosis and predictability in probabilistic systems[END_REF] serve as a foundation to our analysis of diagnosability. They are thus, for the most part, necessary for this thesis and are therefore developed in Chapter 3 and Chapter 4. However, this work also contains contributions on prediction and prediagnosis. Prediction describes the ability to detect the fault before its occurrence. It had been shown to be in NLOGSPACE for logical systems [START_REF] Genc | Predictability of event occurrences in partiallyobserved discrete-event systems[END_REF]. While using probabilities usually increases the difficulty, we gave an NLOGSPACE algorithm for the prediction problem in stochastic systems. The authors of [START_REF] Chen | Stochastic failure prognosability of discrete event systems[END_REF] present a similar result with a notion called "prognosis". While prediction is limited to the detection of faults before their occurrence, diagnosis is itself limited to their detection after the occurrence. That is why we also introduced prediagnosis where one is allowed to either predict or diagnose the fault. Any predictable or diagnosable system is thus prediagnosable, but the converse does not hold. We showed that prediagnosability (i.e. the problem of deciding if a system is prediagnosable) is PSPACE-complete.

For the diagnosability notions studied in this thesis, faults are permanent: once a fault is triggered, any following behaviour is faulty. However, one may want to consider faults that are only temporary. For example, a model could contain the possibility of a reparation. This raises many new problems: one may wish to detect the fault before the reparation, to count the number of faults occurring within the system, etc. These questions were investigated in [START_REF] Fabre | Diagnosability of repairable faults[END_REF] in a non-stochastic setting. While diagnosability of permanent faults is known to be in NLOGSPACE for logical systems, we showed that with repairable faults, it becomes PSPACE-complete. We also discussed multiple methods to count faults and presented among other things an NLOGSPACE algorithm to decide if one can count the number of faults while having a delay of at most one count of fault.

One of the non-stochastic framework where diagnosability was studied is Petri nets (PN) [START_REF] Cabasino | A new approach for diagnosability analysis of petri nets using verifier nets[END_REF][START_REF] Bérard | The complexity of diagnosability and opacity verification for Petri nets[END_REF]. For bounded PN, the usual method to solve diagnosability is to build the reachability graph of the net, which is an automaton structure representing the behaviour of the PN and then to use existing results on this kind of models [START_REF] Jiang | A polynomial algorithm for testing diagnosability of discrete-event systems[END_REF][START_REF] Yoo | Polynomial-time verification of diagnosability of partially observed discrete-event systems[END_REF]. The issue with this method is the size of the reachability graph. In order to face this issue, many works try to abstract the graph in order to reduce its size while keeping the relevant information. In a partial observation setting, this led to the introduction of the basis reachability graph [START_REF] Cabasino | Diagnosability of bounded Petri nets[END_REF]. In [START_REF] Lefaucheux | Basis coverability graph for partially observable Petri nets with application to diagnosability analysis[END_REF], we showed how to extend this abstraction to unbounded PN, calling the new object "Basis coverability graph". We established some results about the properties of the basis coverability graph and explained how to use it to solve diagnosability of unbounded PN.

Chapter 2

Preliminaries

The first step of the theoretical analysis of a problem is the definition of the framework. The chosen framework possesses different properties depending on the specificities of the system one wishes to study. Some frameworks are thus better adapted to represent concurrency, to express infinite-state systems. . . In the first section of this Chapter, we introduce the main definitions that are used throughout the thesis. More precisely, in Subsection 1.1, we recall some definitions and results of descriptive set theory. We then define in Subsection 1.2 the main probabilistic model used in this document and define a probabilistic measure. In Subsection 1.3, we explain how partial observation can be added to this model. These definitions give a framework for various problems linked to partial observation. In this section, we also give definitions related to diagnosis, which is the main partial observation problem studied during this thesis. Diagnosis corresponds to a family of questions, focusing on the identification of a faulty behaviour within the system. We explain in Subsection 1.4 how the notion of fault can be formalised in our model. Finally, in Subsection 1.5, we discuss different notions of diagnosability. These notions are used to capture when and how the faulty (or correct) behaviour of the system must be identified, and with which accuracy. Section 2 finally presents a state of the art on diagnosis.

Framework

Let us start with a few general notations. We denote by N the set of natural numbers, Q the set of rational numbers and R the real numbers. For a finite alphabet Σ, we denote by Σ * (resp. Σ ω ) the set of finite (resp. infinite) words over Σ. ε represents the empty word.

Descriptive set theory

Descriptive set theory [Mos80, Chapter 1] defines and studies classes of "well-behaved" sets. These sets having good properties, they have applications in many areas. In this thesis, they have two main applications. First, they are used to evaluate the complexity of some problems. This is achieved using a hierarchy ranking these sets: the higher a Preliminaries set is in the hierarchy, the more complex it is. Therefore if we can express a property with a set, the problem complexity can be related to the set complexity. Secondly, these sets form a building block of the formal definition of the stochastic behaviour of our model.

Let us first recall some standard facts about Borel sets.

Definition 2.1. Given a space X, the set Y is a topology on X if

• X ∈ Y and ∅ ∈ Y , • Y is stable by union, i.e. given (O i ) i∈I a family of elements of Y , ∪ i∈I O i ∈ Y , • Y is stable by finite intersection, i.e. given (O i ) i∈I a finite family of elements of Y , ∩ i∈I O i ∈ Y ,
We call the sets in Y the open sets.

Example 2.1. Consider the space of infinite words over two letters {a, b} ω . On such a space, the usual topology uses the notion of cylinder. Given a finite word w of {a, b} * , the cylinder of w is the set of infinite words extending w, Cyl(w) = w{a, b} ω . One obtains a topology by choosing the set of cylinders as its basic components and by adding the sets created through union and finite intersection. {a, b} ω belongs to this topology as it is the cylinder of ε, the empty set can be obtained as the intersection of the cylinders of 'a' and of 'b'. Under this topology, the set of words containing at least k 'a', for k ∈ N is an open set as it is a countable union of cylinders.

Given a space X and a topology Y on X, we define the Borel hierarchy (represented in Figure 2.1) as the three classes of sets Σ 0 n , Π 0 n and ∆ 0 n obtained inductively by

• Σ 0 1 = Y is the set of open sets; • ∀n ≥ 1, a set belongs to Π 0 n if its complement belongs to Σ 0 n • ∀n ≥ 2, O is in Σ 0 n if there exists a family (O i ) i∈I of elements of Π 0 n-1 such that O = ∪ i∈I O i ; • ∀n ≥ 1, ∆ 0 n = Π 0 n ∩ Σ 0 n .
The sets in Π 0 1 are called closed, the sets Σ 0 2 and Π 0 2 are respectively called F σ and G δ . A Borel set is a set belonging to some level of the Borel hierarchy. The set of Borel sets is denoted B.

Example 2.2. Continuing Example 2.1, the set composed of the single word a ω is closed. Indeed, its complement is ∪ n∈N Cyl(a n b) which is an open set. The set of words with infinitely many 'a' is neither open, nor closed. It however belongs to G δ . Indeed, it is the complement of the set of words ending by b ω . As this set of words is an infinite union of closed set (each closed set containing a single infinite word), it belongs to F σ , thus its complement is a G δ set.

Σ 0 1 Σ 0 2 • • • ∆ 0 1 ∆ 0 2 • • • Π 0 1 Π 0 2 • • • Σ 0 α • • • ∆ 0 α ∆ 0 α+1 • • • Π 0 α • • • Figure 2
.1: Representation of the Borel hierarchy.

In our models, we need to measure the probabilities of certain events. This is done thanks to Carathéodory's extension theorem [START_REF] Ash | Probability and measure theory[END_REF]. Before stating this theorem, we need to introduce some definitions used in measure theory.

Definition 2.2. Given a space X, a ring of sets R of X is a subset of the powerset of X satisfying:

• ∅ ∈ R; • R is closed under pairwise union, ∀A, B ∈ R, A ∪ B ∈ R; • R is closed under relative complements, ∀A, B ∈ R, A \ B ∈ R.
Definition 2.3. Given a space X, a σ-algebra S of X is a subset of the powerset of X satisfying:

• ∅ ∈ S; • S is closed under countable union, ∀(A i ) i∈N ∈ S, i∈N A i ∈ S; • S is closed under complement, ∀A ∈ S, S \ A ∈ S.
By definition, a σ-algebra is a ring of sets. Observe that the set of Borel sets B is a σ-algebra. More precisely, it is the σ-algebra generated by the open sets (i.e. it is the smallest σ-algebra containing the open sets).

Definition 2.4. Given a ring of sets R on a space X, a pre-measure µ on R is a function µ : R → [0, +∞] such that:

• µ(∅) = 0;
• for all countable family of sets of R pairwise disjoint, (A i ) i∈N , we have

µ ∞ n=1 A n = ∞ n=1 µ(A n ). A pre-measure µ is called σ-finite if there exist a countable number of sets A 1 , A 2 • • • ∈ R such that X = ∞ k=1 A k and for all k ∈ N, µ(A k ) < ∞. If R is a σ-algebra, then µ is called a measure.
A measure µ is called inner regular if for every set E, we have

µ(E) = sup{µ(F ) | F ⊆ E ∧ F is a closed set}.
In the current work, we require a measure, yet only a pre-measure can efficiently be defined. Fortunately, Carathéodory's extension theorem allows to bridge the gap.

Theorem 2.1 ([ADD99]

). Let R be a ring on a space X, µ be a pre-measure on R. There exists a measure µ extending µ on the σ-algebra generated by R. Moreover, if µ is σ-finite, then µ is unique and also σ-finite.

Example 2.3. Let us continue Example 2.1 and consider the ring of sets R generated by the topology based on the cylinders. We define the pre-measure µ on this ring as the only pre-measure satisfying ∀n ∈ N, w ∈ {a, b} n , µ(Cyl(w)) = 1 2 n . One could interpret the represented system as an infinite number of coin flips, each result ('a' or 'b') having 1 2 probability. µ gives the probability of cylinders (a given finite number of flips) and finite unions of them.

The pre-measure µ is σ-finite as µ({a, b} ω ) = 1. According to Carathéodory's extension theorem, there is thus an unique measure µ extending µ on the σ-algebra generated by R: the Borel sets. Observe that µ is inner regular.

Probabilistic Labelled Transition Systems

We now define the model that represent the system. The choice of the model depends on the properties one wishes to have. Petri nets [START_REF] Diaz | Petri nets: fundamental models, verification and applications[END_REF] for example, efficiently represent concurrent systems. Another possibility is to use automata structures as in the seminal work of [SSL + 95] formally defined by: Definition 2.5. A labelled transition system (LTS) is a tuple A = Q, q 0 , Σ, T where:

• Q is a countable set of states with q 0 ∈ Q the initial state;

• Σ is a finite set of events;

• T ⊆ Q × Σ × Q is a set of transitions;
Informally, the states of Q represent the different configurations the system can be in, an event of Σ is an action that can be taken by the system (sending a request, activating a component. . . ) and T describes how this action affects the system.

Formally, we write q a -→ q when there exists a transition (q, a, q ) ∈ T ; this transition is then said to be enabled in state q. We assume all LTS we consider are live, i.e. in every state of the LTS at least one transition is enabled. This ensures the system will not reach a deadlock position and stop activating events. A run ρ of an LTS A is a (finite or infinite) sequence ρ = q 0 a 0 q 1 . . . such that for all i ≥ 0, q i ∈ Q, a i ∈ Σ and when q i+1 is defined, q i a i -→ q i+1 . A run thus represents the evolution of a system over time. The notion of run can be generalised, starting from an arbitrary state q. Given an LTS A, we write Ω A for the set of all infinite runs starting from q 0 . We only write Ω when the LTS A is clear from context. When it is finite, ρ ends in a state that we denote last(ρ) and its length, denoted by |ρ|, is the number of events occurring in it. Given a finite run ρ = q 0 a 0 q 1 . . . q n and a (finite or infinite) run ρ = q n a n q n+1 . . . starting in last(ρ), we call concatenation of ρ and ρ the run ρρ = q 0 a 0 q 1 . . . q n a n q n+1 . . . . The run ρ is then a prefix of ρρ , which we denote by ρ ρρ . The cylinder generated by a finite run ρ consists of all the infinite runs that extend ρ:

Cyl(ρ) = {ρ ∈ Ω | ρ ρ }.
The sequence associated with ρ = qa 0 q 1 . . . is the word σ ρ = a 0 a 1 . . ., and we write indifferently q ρ = ⇒ or q σρ = ⇒ (resp. q ρ = ⇒ q or q σρ = ⇒ q ) for an infinite (resp. finite) run ρ starting in q (resp. and ending in q ). A state q is reachable (from the initial state q 0 ) if there exists a run ρ such that q 0 ρ = ⇒ q, which we alternatively write q 0 = ⇒ q. The language of an LTS A consists of all infinite words that label runs of A and is formally defined as

L ω (A) = { σ ∈ Σ ω | ∃ q 0 σ = ⇒ }. A bottom strongly connected component (BSCC) of
an LTS is a strongly connected component from which no state outside of the BSCC are reachable.

Example 2.4. Consider the LTS represented in Figure 2.2. It contains three states. q 0 is the initial state, representing the machine waiting to receive an order. When such an order occurs the run takes the transition labelled by the 'coin' event and enters the second state. In this second state, the machine is preparing the coffee, it has the possibility to add sugar, and after a certain amount has been added, it returns to its initial state by giving a coffee. In this operating state, the machine can also commit an error, event 'f ', leading to a faulty state f 1 . In f 1 , the machine cannot serve coffee any more and sends an 'out of order' signal. A normal use of this system by a consumer is given for example by the run ρ = q 0 coin q 1 sugar q 1 coffee q 0 . q 0 q 1 f 1 f coin coffee sugar out of order Figure 2.2: An LTS representing a coffee machine. q 0 is the initial state, which is represented by the incoming arrow. Transitions between two states are labelled by the event associated with the transition.

In order to represent the unpredictability of the environment and to neglect events that have a null probability of occurring, we want to represent stochastic behaviours. To do so, the model must be enriched using probabilities. More precisely, this is achieved by adding a probability matrix indicating how the transitions, labelled by events, are randomly chosen.

Definition 2.6. A probabilistic labelled transition system (pLTS) is a tuple A = Q, q 0 , Σ, T, P where:

• A = Q, q 0 , Σ, T is an LTS;
• P is the transition matrix from T to Q >0 fulfilling for all q ∈ Q:

(q,a,q )∈T P[q, a, q ] = 1 .

The LTS A is called the underlying labelled transition system of A.

Note that since we assume the state space to be at most countable, a pLTS is by definition at most countably branching: in every state q, only countably many transitions are enabled, so that the summation (q,a,q )∈T P[q, a, q ] is well-defined. The definitions introduced for LTS are naturally lifted to pLTS. Given a countable set Z, a distribution on Z is a mapping µ :

Z → [0, 1] such that z∈Z µ(z) = 1. The support of µ is Supp(µ) = {z ∈ Z | µ(z) > 0}. If Supp(µ) = {z} is a single element, µ
is a Dirac distribution on z written 1 z . We denote by Dist(Z) the set of distributions on Z. The transition matrix defines in every state q a distribution on the transitions whose support are exactly the transitions enabled in q.

Example 2.5. Consider the pLTS represented in Figure 2.3. Its underlying LTS is represented in Figure 2.2. The difference is thus that probabilities were added on the transitions so that the sum of the probabilities exiting any state is equal to one. The run ρ that was described as being a normal use of the system in Example 2.4 can now be associated with a probability, which is the product of the probability of the events it triggered. Here, 1 × 0.29 × 0.7 = 0.203. This result, being low, seems to point out that, in this representation, most consumers do not take exactly one unit of sugar in their coffee or do not even get their coffee.

q 0 q 1 f 1 f , 0.01 coin, 1 coffee, 0.7
sugar, 0.29 out of order, 1 We now use the probabilities within the system to formally define the probability measure that are used on runs. This is done using the descriptive set theory presented in Subsection 1.1. The construction of the measure uses the Carathéodory's extension theorem, similarly to what is done in Example 2.3.

Here, the set space X we are interested in is the set of all infinite runs Ω. The open set Σ 0 1 is built from the cylinders: Σ 0 1 is the smallest set containing ∅, Ω, such that for all finite run ρ, Cyl(ρ) ∈ Σ 0 1 , and such that Σ 0 1 is stable by union and finite intersection. The complement of a cylinder is a finite union of cylinders, the complement of an (potentially infinite) union is an (potentially infinite) intersection and the complement of a finite intersection is a finite union. A set F is thus closed if and only if F = n∈N O n where O n is a union of cylinders. Therefore an F σ set F can be written as

F = m∈N n∈N O m,n
where O m,n is a union of cylinders whose associated paths have length n. Without loss of generality, the sequence of closed sets may be chosen as a non-decreasing sequence. The cylinders are thus used as the basis to build a Borel hierarchy on Ω.

Given a pLTS A, in order to have a probabilistic measure spanning all the sets of infinite runs of the Borel hierarchy generated by the cylinders, we define a pre-measure P A on the ring of sets generated by the open sets by: for all finite run q 0 a 0 q 1 . . . q n , P A (Cyl(q 0 a 0 q 1 . . .

q n )) = P[q 0 , a 0 , q 1 ] • • • P[q n-1 , a n-1 , q n ]
and for all finite sequence of pairwise disjoint cylinders A 1 , . . . , A n ,

P A (∪ i=1,...,n A i ) = i=1,...,n P A (A i ) .
According to Carathéodory's extension theorem, this pre-measure can be extended uniquely on the whole σ-algebra generated by this ring. We still write P A for the obtained measure. Moreover, when A is fixed, we may omit the subscript. To simplify, for ρ a finite run, we sometimes abuse notation and write P(ρ) for P(Cyl(ρ)). If R is a (countable) set of finite runs such that no run is a prefix of another one, we write P(R) for ρ∈R P(ρ) which is consistent since all intersections of associated cylinders are empty.

Partial observation

In partial observation problems, one considers an observer of the system (a user, an attacker. . . ) who perfectly knows the model, yet only partially observes its behaviours. In this case, one needs to formalise which information can be observed. This allows to associate with every run of the system, an observation or sequence of observations. The first question is to determine what part of a run gives an information: events, states or both. In fact, these three options are equivalent in terms of expressibility. Thus, for every problem, one selects the option that is the more efficient at representing the specificities of the problem. For example, when studying diagnosis, we try to detect a faulty action of the system. As this faulty action is an event, observations are put on events. This is the option we follow in most chapters of this thesis, see Chapter 7, page 195, for a model with observations on states.

Events represent actions taken by the system. A formalisation of the observation could be made by distinguishing internal and external actions. This would mean that some actions occur within the system and are thus unobservable while others are done publicly and are thus observable. Formally, one partitions the set of events Σ into two disjoint sets Σ o and Σ u , the sets of observable and unobservable events, respectively.

Example 2.6. Consider the pLTS of Figure 2.3. Clearly, introducing a coin, adding sugar to the cup and receiving the coffee are observable actions. The other two are more subject to discussion. Depending on the fault and the sensors within the system, f could be considered observable or unobservable. Indeed, if f stands for the explosion of the machine, it would clearly be observable, however if f is an internal error, detecting it requires the existence of an appropriate sensor within the system. The status of the last event also depends on which fault occurred. For our example, let us assume f is unobservable, but the failure of the system is detected, allowing the machine to publicly send an 'out of order' message. In this case, the run "q 0 coin q 1 sugar q 1 f f 1 out of order" produces the sequence of observations "coin sugar out of order".

A more sophisticated method than this partition would be to equip the pLTS with a mask function. This mask function associates every event with an observation, taken from an observation alphabet. This function can map an event to ε meaning that the event is unobservable or project multiple events onto the same observation, making them indistinguishable. When using a mask function, Σ o is the observation alphabet and Σ u is the set of unobservable events (i.e. events which observation is ε).

Example 2.7. Consider the pLTS of Figure 2.3 again. We use the observation alphabet Σ o = {coin, coffee, beep} and the mask function P such that P(coin) = coin, P(coffee) = coffee, P(f ) = ε and P(sugar) = P(out of order) = beep. Here, we see that two of the events are indistinguishable as they share the same observation beep. When a beep is produced, a user does not know whether the machine is out of service or if it is adding more sugar. In other words, the infinite runs "q 0 coin q 1 f f 1 (out of order f 1 ) ω " and "q 0 coin q 1 (sugar q 1 ) ω " share the same observation sequence "coin sugar beep ω ".

Observe that the mask function setting generalises the partition discussed above. Indeed, the partition is mimicked by the mask function which projects every unobservable event to ε and every observable event onto itself. We now define multiple notations using the mask function formalism. Due to the previous remark, these definitions can easily be applied to the partition setting. In the future chapters, we mostly use partitions for simplicity. We state explicitly when we use mask functions.

Given an observation alphabet, a mask function is a mapping P : Σ → Σ o . It is extended to words from Σ * inductively by: P(ε) = ε and P(σa) = P(σ)P(a). We write |σ| o for the observable length of σ, that is |P(σ)|. The observable length of a run ρ, denoted |ρ| o , is the observable length of its associated sequence. Given a run ρ and its sequence σ ρ we sometimes use P(ρ) for P(σ ρ ). When σ is an infinite word over Σ, its projection (resp. observable length) is the limit of the projections (resp. observable length) of its finite prefixes. Given a ∈ Σ o , |σ| a is the number of occurrences of a in σ. As usual the mask function P is extended to languages: for

L ⊆ Σ * ∪ Σ ω , P(L) = {P(σ) | σ ∈ L}.
With respect to the mask function P, a pLTS A is said convergent if there is no infinite sequence of unobservable events from any reachable state:

L ω (A) ∩ Σ * Σ ω u = ∅.
When A is convergent, for every σ ∈ L ω (A), P(σ) ∈ Σ ω o . In the rest of the thesis we assume that pLTS are convergent. We refer to a sequence for a finite or infinite word over Σ, and to an observed sequence for a finite or infinite word over Σ o . The projection of a sequence onto Σ o is thus an observed sequence. The prefix of length n ∈ N of an observed sequence w is denoted w ≤n .

We now define the notion of signalling runs. They correspond to the finite runs which last event was observable (i.e. finite runs q 0 a 0 q 1 • • • a n-1 q n such that P(a n-1 ) = ε). Signalling runs are precisely the relevant runs w.r.t. partial observation issues since each observable event provides additional information about the execution to an external observer. In the sequel, SR(A) denotes the set of signalling runs of the pLTS A, and SR n (A) the set of signalling runs of observable length n. The pLTS is dropped from the notation when it is clear from context. Since we assume that the pLTS are convergent, for every n > 0, SR n is equipped with a probability distribution defined by assigning measure P(ρ) to each ρ ∈ SR n . Given ρ a finite or infinite run, and n ≤ |ρ| o , ρ ↓n denotes the unique prefix of ρ that belongs to SR n . For convenience, the empty run q 0 is defined as the single signalling run of null length. For an observed sequence w ∈ Σ * o , we define its cylinder Cyl(w) = wΣ ω o and the associated probability P(Cyl(w)) = P({ρ ∈ Ω | P(ρ ↓|w| ) = w}) = P({ρ ∈ SR |w| | P(ρ) = w}), often shortened as P(w).

Fault and ambiguity

We now give definitions and notations for a partial observation problem, which is particularly of interest to us, diagnosis. Diagnosis focuses on the detection of a special unobservable event called the fault f ∈ Σ u thanks to the observations received from the system. Let us now classify runs depending on whether they contain a fault or not. A run ρ is faulty if its associated sequence σ ρ contains f , otherwise it is correct. For n ∈ N, we write F n (resp. C n ) for the set of infinite runs whose signalling prefix of observable length n is faulty (resp. correct). We further define the sets of all finite faulty and correct signalling runs F and C and the sets of infinite faulty and correct runs

F ∞ = n∈N F n and C ∞ = n∈N C n .
A run ρ is a minimal faulty run if it is a faulty run and there does not exist a prefix ρ of ρ that is a faulty run. We write, for all n ∈ N, minF n for the set of minimal faulty runs of length n and minF = n∈N minF n for the set of all minimal faulty runs.

Given two states q and q and an observation a ∈ Σ o , we write q ⇒ a q if there exists a run ρ = q 0 a 0 q 1 . . . q n with q 0 = q, q n = q , ρ ∈ SR 1 and P(ρ) = a. We also write q ⇒ a f q (resp. q ⇒ a c q ) if there exists a faulty (resp. correct) run ρ = q 0 a 0 q 1 . . . q n with q 0 = q, q n = q , ρ ∈ SR 1 ∩ F (resp. ρ ∈ SR 1 ∩ C) and P(ρ) = a.

Except explicit mention of the opposite, we assume that the state space Q of A is partitioned into correct states and faulty states: Q = Q f Q c such that faulty (resp. correct) states, i.e. states in Q f (resp. Q c ), are only reachable by faulty (resp. correct) runs. This can be done without loss of generality. Indeed, considering a pLTS A = Q, q 0 , Σ, T, P , we can build the pLTS A = Q , (q 0 , ⊥), Σ, T , P where:

• Q = Q × {⊥, },
• ((q, i), a, (q , j)) ∈ T if (q, a, q ) ∈ T , and either a = f and i = j or a = f and j = ,

• for ((q, i), a, (q , j)) ∈ T , P ((q, i), a, (q , j)) = P(q, a, q ).

Denoting Q f = Q × { } and Q c = Q × {⊥},
the pLTS A has the same behaviour as A and verifies the partition mentioned above as a run enters Q f if and only if its last transition was a fault and can never go back to Q c . While the correct or faulty status of the current run may not be known to the observer, the observed sequences carry some information about them. An infinite (resp. finite) observed sequence w ∈ Σ ω o (resp. Σ * o ) is called ambiguous if there exists a correct infinite (resp. signalling) run ρ and a faulty infinite (resp. signalling) run ρ such that P(ρ) = P(ρ ) = w. Otherwise, it is either surely faulty, or surely correct depending on whether P -1 (w) ∩ SR ⊆ F or P -1 (w) ∩ SR ⊆ C. A run is ambiguous, surely correct or surely faulty if its observed sequence is ambiguous, surely correct or surely faulty respectively. We write Sf ∞ (resp. Sc ∞ ) for the set of infinite surely faulty (resp. correct) runs. In addition Sf n (resp. Sc n ) is the set of infinite runs whose signalling prefix of observable length n is surely faulty (resp. correct).

Example 2.8. Consider the pLTS of Figure 2.3 associated with the mask function P such that P(f ) = ε, P(out of order) = P(sugar) = beep and every other event is projected on itself.

First observe that this pLTS satisfies the partition between faulty and correct states. Indeed, a run ends in f 1 iff it is faulty.

The observed sequence "coin beep" is ambiguous as it can be generated by the correct run "q 0 coin q 1 sugar q 1 " and the faulty signalling run "q 0 coin q 1 f f 1 out of order f 1 ". Extending this observed sequence with other observations of beep maintains the ambiguity. Extending it with 'coffee' however makes it surely correct. There does not exist any surely faulty observed sequence in this pLTS.

Which diagnosis for pLTS?

The goal of diagnosis is the automatic detection of the fault event. This detection is performed by a diagnoser, a function observing the system and giving its verdict. Formally, a diagnoser is a function D : Σ * o → {?, , ⊥} assigning to every finite observed sequence a verdict. Informally, when a diagnoser outputs ? it does not provide any information, while means that the diagnoser announces a fault and ⊥ that the diagnoser provides some information about the correctness of the current run. Multiple notions of diagnoser, and thus of diagnosis, can be defined depending on the properties that we require. In logical systems, three main features of the diagnoser are considered: verdict, correctness and reactivity. Verdict specifies the nature of the information the diagnoser provides along the run: it may only be related to detection of faults or may also assert that (some prefix of) the run does not include a fault. Correctness specifies that when the diagnoser outputs a verdict, this verdict holds. Reactivity expresses the regularity at which the diagnoser must provide information about the status of the run. The aim of this section is to define appropriate verdict, correctness and reactivity requirements for probabilistic systems. We start with informal explanations that also motivate the need for considering different variants of diagnosis. We present these variants as decision problems which are intuitively easier to understand and simpler to use. We make the link with diagnosers in Chapter 3.

In seminal works about probabilistic systems, the verdict is limited to fault detections and the reactivity is usually relaxed by requiring that when a fault occurs, a diagnoser almost surely detects it after a finite delay [START_REF] Thorsley | Diagnosability of stochastic discrete-event systems[END_REF]. Let us look at the pLTS of Figure 2.4. One cannot detect that the run q 0 f (f 1 a) ω is faulty due to the correct run q 0 u(q 1 a) ω with same observed sequence a ω . However with probability 1, a faulty run will produce a 'b' and thus almost all faulty runs are unambiguous, so that faults are almost surely detected. On the other hand, one cannot provide any information about the single correct run q 0 u(q 1 a) ω since its observed sequence is ambiguous as well as any of its prefix. Observe that the notion of ambiguity described here is qualitative: the observation of the correct run is considered ambiguous even though the probability to be faulty, conditioned on the observation, converges to 0.

q 0 f 1 f 2 q 1 f a u a b a
Figure 2.4: Detecting faults but not correct runs. When probabilities are not specified, we assume uniform distributions. Dashed edges are used for unobservable transitions. For observable transitions, the observation given by the mask function labels the edge.

In order to examine which verdict could be provided about correct runs, let us look at the pLTS of Figure 2.5. The sequence a n is ambiguous. However up to the n -1 th observation, all the runs that correspond to this observed sequence were correct which is a useful information for instance to restart later the system from a correct state. Along the (surely correct) observed sequence a ω , the observer can always deduce that longer and longer prefixes of the run were correct while never being able to assert that the current run is correct. The correctness requirement may be specified in different ways. For an exact diagnosis, we ask that a fault can be claimed only when a fault surely happened (as it is the case in non-probabilistic systems). However it may be necessary to weaken the correctness requirement as illustrated by the pLTS of Figure 2.6. Since all observed sequences are ambiguous no exact diagnosis can be provided. However it is clear that when in a long enough observed sequence the ratio between occurrences of 'b' and 'a' is close to 3, the probability that the corresponding run is faulty is close to 1. Let us fix any ε > 0 and only require that the probability for the verdict to be erroneous should be less than ε. Then using the strong law of large numbers, (approximate) fault detection is possible in this pLTS. To formalise later the correctness of an approximate diagnoser, with every observed sequence w ∈ Σ * o we associate a correctness proportion

q 0 q 2 f 1 f 2 q 1 u f a u a b b a
q 0 q f q c f , 1 2 u, 1 2 a, 1 4 b, 3
CorP(w) = P({ρ ∈ C ∩ SR |w| | P(ρ) = w}) P({ρ ∈ SR |w| | P(ρ) = w})
, which is the conditional probability that a signalling run is correct given that its observed sequence is w.

The standard way to specify reactivity in probabilistic systems for fault detection is to require that whatever the minimal faulty run, almost surely the diagnoser will output its (faulty) verdict. We may also consider uniform reactivity which strengthens reactivity by requiring that the (random) delay is independent of the minimal faulty run. More formally, uniform reactivity ensures that given any positive probability threshold α > 0 there exists a delay n α independent of the considered minimal faulty run such that the probability to exceed this detection delay is bounded by α. Let us illustrate these reactivity features with the pLTS of Figure 2.7 for which only approximate diagnosis is possible. Fix some ε > 0 and consider the minimal faulty run q 0 uq 1 (aq 1 ) m f q f . After a certain number of occurrences of 'b' (say n), the correctness proportion of the observed sequence a m b n will be less than ε and thus the diagnoser can output its verdict. However due to the probabilities of an occurrence of 'a' in correct and faulty runs respectively equal to 3 4 and 3 16 , n must depend on m and so this reactivity cannot be uniform. This can be mathematically seen through the definition of CorP:

q 0 q 1 q f q c u, 1 2 f , 13
for n ≥ 1, CorP(a m b n ) = 3 m 4 m+n 3 m 4 m+n + ( 3 16 ) m × 15 16 = 1 1 + 15 16 × 4 n-m In order to have CorP(a m b n ) ≤ 1/2, one needs n > m. Therefore uniform diagnosis is not possible.
In order to formalise the different requirements discussed above, we first define several sets of runs related to ambiguity. Definition 2.7 (Ambiguous runs). Let A be a pLTS, ε ≥ 0 and n ∈ N >0 . Then:

• FAmb ∞ is the set of infinite faulty ambiguous runs of A;

• CAmb ∞ is the set of infinite correct ambiguous runs of A;

• FAmb n is the set of infinite runs of A whose signalling prefix of observable length n is faulty and ambiguous;

• CAmb n is the set of infinite runs of A whose signalling prefix of observable length n is correct and ambiguous.

• FAmb ε n is the set of infinite faulty ambiguous runs of A whose observed sequence of length n, w fulfils: CorP(w) > ε.

By definition, for all n ∈ N, FAmb 0 n = FAmb n . Observe that, for all n ∈ N, and ε ≥ 0, CAmb n , FAmb n and FAmb ε n are open sets, thus measurable. However, CAmb ∞ and FAmb ∞ are not Borel sets in the general case (e.g. see Chapter 3, Section 3).

We propose five specifications of exact diagnosability for probabilistic systems based on three discriminating criteria: whether the unambiguity requirement holds for faulty runs only or for all runs, whether ambiguity is defined at the level of infinite runs or for longer and longer finite signalling prefixes, and whether the delay before detection of minimal faulty runs is uniform. These notions are summarised in Figure 2.8 except for uniformity postponed to next figure.

Definition 2.8 (Exact diagnosability). Let A be a pLTS.

• A is IF-diagnosable if P(FAmb ∞ ) = 0. • A is IA-diagnosable if P(FAmb ∞ CAmb ∞ ) = 0.

IF-diagnosis IA-diagnosis

FA-diagnosis FF-diagnosis

Infinite runs

Finite prefixes

All runs Faulty runs Verdict Reactivity Figure 2.8: Summarising the variants of exact diagnosis.

• A is FF-diagnosable if lim sup n→∞ P(FAmb n ) = 0. • A is FA-diagnosable if lim sup n→∞ P(FAmb n CAmb n ) = 0.
• A is uniformly FF-diagnosable if for all α > 0 there exists n α ∈ N such that for all n ≥ n α and all minimal faulty run ρ ∈ minF

P({ρ ∈ FAmb n+|ρ|o | ρ ρ }) ≤ α • P(ρ) .
Uniform and/or approximate diagnoses are defined for FF-diagnosis as summarised in Figure 2.9. We chose FF-diagnosis as it corresponds to the classical notion of diagnosis. Moreover there is no clear intuition on what would be the meaning of uniformity and approximation for the other variants. εFF-diagnosability allows the diagnoser to claim a fault when the correctness proportion does not exceed ε, and accurate approximate diagnosability denoted by AFF-diagnosability corresponds to εFF-diagnosability for arbitrary ε > 0.

Definition 2.9 (Approximate diagnosability). Let A be a pLTS, and ε ≥ 0.

• A is εFF-diagnosable if for every minimal faulty run ρ ∈ minF and all α > 0 there exists n ρ,α such that for all n ≥ n ρ,α :

P(Cyl(ρ) ∩ FAmb ε n+|ρ|o ) ≤ α • P(ρ).
• A is uniformly εFF-diagnosable if for all α > 0 there exists n α such that for all minimal faulty run ρ ∈ minF and all n ≥ n α : • A is (resp. uniformly) AFF-diagnosable if it is (resp. uniformly) εFF-diagnosable for all ε > 0.

P(Cyl(ρ) ∩ FAmb ε n+|ρ|o ) ≤ α • P(ρ
When studying diagnosis, we are interested in the following problems. First, for every notion of diagnosability, we want to determine if it is possible to automatically decide whether a given system is diagnosable. Moreover, in the positive case, we want to establish the exact complexity of the problem. Then, given a diagnosable system, we want to build a diagnoser satisfying the corresponding verdict, correctness and reactivity features. When possible, we represent the diagnoser using a finite-state automaton to express that only a finite memory is necessary. The problems can vary depending on the framework. For example, when the system is controllable, we want to decide if one can control the system in a way ensuring diagnosability.

State of the art on diagnosis

Diagnosis of finite LTS. For LTS, diagnosability requires that the occurrence of unobservable faults can be deduced after a finite delay from the sequence of observable events occurring before and after the fault [SSL + 95]. Using the definitions we introduced in this chapter, an LTS is diagnosable iff n∈N m≥n FAmb m = ∅. Diagnosability of finite LTS was shown to be decidable in NLOGSPACE [START_REF] Jiang | A polynomial algorithm for testing diagnosability of discrete-event systems[END_REF][START_REF] Yoo | Polynomial-time verification of diagnosability of partially observed discrete-event systems[END_REF]. The algorithm relies on the twin-plant described page 29. An LTS is not diagnosable iff there exists a reachable cycle in the twin-plant in which each state is a pair composed of a faulty and a correct state. Detecting such a cycle can be done in non-deterministic logarithmic space in the twin-plant, which is quadratic in the size of the original LTS, hence the result. Surprisingly, while deciding diagnosability is easy, the construction of the diagnoser can require exponential time. This construction is based on the belief construction quickly presented page 29.

Diagnosis of infinite LTS. An LTS with infinitely many states must be represented by a higher level model in order to be studied. This can be done using Petri nets (PN) for example. The semantics of a PN is called the reachability graph and can be represented by an LTS. This LTS is finite iff the PN is bounded. Cabasino et al. studied diagnosis of both bounded and unbounded PN [START_REF] Cabasino | A new approach for diagnosability analysis of petri nets using verifier nets[END_REF]. In order to solve diagnosability in the unbounded case they first build the verifier net, which is a construction similar to the twin-plant, then construct the coverability graph (a finite abstraction of the reachability graph) of the verifier net and finally analyse the cycles of the coverability graph. This final analysis requires some additional properties of the cycles compared to the analysis in the twin-plant for finite LTS. This algorithm however has a complexity that depends on the size of the coverability graph, which may be Ackermanian in the size of the description of the PN. An algorithm with better complexity was developed in [START_REF] Bérard | The complexity of diagnosability and opacity verification for Petri nets[END_REF]. It still uses the verifier net, but transforms the diagnosability problem into an LTL formula and uses model-checking results to obtain an EXPSPACE upper bound. Interestingly, this paper also studies opacity, confirming the intuition that opacity and diagnosis are two close problems. See [START_REF] Basile | Overview of fault diagnosis methods based on Petri net models[END_REF] for a presentation of the usual techniques used for fault diagnosis in PN. LTS with infinitely many states can also be represented by pushdown systems. Morvan and Pinchinat showed that diagnosability in the general case is undecidable [START_REF] Morvan | Diagnosability of pushdown systems[END_REF]. However, for a large subclass called visibly pushdown automata, diagnosability can be decided in PTIME. This is done by building once again a form of twin-plant, this time making the product of the visibly pushdown automata. Thanks to this product, they can define an appropriate Büchi condition on the twinplant so that diagnosability can be deduced from the emptiness of the obtained Büchi pushdown automaton (checking the emptiness can be done in PTIME). The restriction to visibly pushdown automata, which we discuss in Chapter 5, is required in order to build the twin-plant.

Diagnosis of stochastic systems. When diagnosability was adapted to stochastic systems by Thorsley and Teneketzis in [START_REF] Thorsley | Diagnosability of stochastic discrete-event systems[END_REF], two notions of diagnosability were initially defined: A-diagnosability and AA-diagnosability. In finite pLTS, A-diagnosability corresponds to our uniform FF-diagnosability and AA-diagnosability corresponds to our uniform AFF-diagnosability. For A-diagnosability, they gave a necessary and sufficient condition based on the belief construction: they first build a diagnoser similarly to what was done for logical systems, then test for the recurrence of ambiguous states of the belief (i.e. states that can contain both correct and faulty states). The complexity of the algorithm checking this characterisation is not mentioned however. For AA-diagnosability, they only give a sufficient condition, leaving the general problem open. The questions left opened by [START_REF] Thorsley | Diagnosability of stochastic discrete-event systems[END_REF] were tackled by Chen and Kumar who gave algorithms with PTIME complexities to answer both diagnosability decision problems [START_REF] Chen | Polynomial test for stochastic diagnosability of discrete-event systems[END_REF]. Their algorithm for AA-diagnosability is particularly interesting as they translate the problem into a question of language equivalence (in terms of probability of words) for the original pLTS in a specific initial distribution. Unfortunately, these two algorithms are erroneous (see Chapter 4 for details on these two problems).

When a pLTS is not diagnosable, one interesting question that can be raised is how far from diagnosability it is. If there is only a very little chance that the fault is not detected, the system may be "diagnosable enough". This direction was studied in [START_REF] Nouioua | A probabilistic analysis of diagnosability in discrete event systems[END_REF] where Nouioua and Dague consider an exact notion of diagnosability and wish to measure the probability of the faulty ambiguous runs. To realise this measure, they make the product of the pLTS with its belief construction (thus a method that we already presented page 29). Then they measure the asymptotic probability to be in each state of this product pLTS. As, thanks to the belief, states contain the relevant information to determine if they were reached by a faulty ambiguous run, they can determine the probability to be faulty and ambiguous at the limit. This approach is continued in [START_REF] Bazille | Diagnosability degree of stochastic discrete event systems[END_REF] where Bazille et al. introduce a notion of k-diagnosability degree, which is defined as the probability to detect a fault at most k steps after it occurs, conditioned to the occurrence of a fault. They measure this degree by (1) building the product of the pLTS with the belief construction and (2) using polynomial time algorithms that compute the sum of the probabilities of the runs that reach a target state set (in this case, the states which belief component show they were reached by faulty ambiguous runs). Using the computation of k-diagnosability degrees for different values of k, they also investigate the average speed of detection of a fault.

Active diagnosis.

One can enrich an LTS by allowing a form of control. This is done by introducing non-determinism, that is resolved at every step by a controller. One possibility of control is done through restriction of the enabled events: at every step, the controller selects a set of observable events Σ • ⊆ Σ o and the next transition taken by the LTS is labelled either by an unobservable event or by an observable one which observation belongs to Σ • . Some observable events can also be considered uncontrollable and are enabled no matter the choice of the controller. In this framework, the diagnosability of a system depends on the choice of the controller. Finding a controller such that the controlled system is diagnosable is the goal of active diagnosis. This problem was introduced in [START_REF] Sampath | Active diagnosis of discreteevent systems[END_REF]. Sampath, Lafortune and Teneketzis then solve the question by building the most permissive controller through a complicated iterative procedure which complexity is not given (and seems to be doubly exponential). Later, a planning-based approach via a twin-plant construction was proposed in [START_REF] Chanthery | Monitoring and active diagnosis for discreteevent systems[END_REF]. The exact complexities were finally established in [START_REF] Haar | Optimal constructions for active diagnosis[END_REF][START_REF] Haar | Optimal constructions for active diagnosis[END_REF], where the active diagnosis problem was shown to be EXPTIME-complete and finite-memory controllers (most permissive and optimal in memory size) are given. This is done by translating the active diagnosis problem into a Büchi game (using a variant of the product with the belief) and then solving the Büchi game, which gives an optimal strategy which can be translated back into a controller. This analysis was extended to controllable stochastic systems [BFH + 14]. Here, instead of translating the problem to a Büchi game, Bertrand et al. use partially observable Markov decision processes and show that the problem of stochastic active diagnosis is also EXPTIME-complete. They also study a safe notion of stochastic active diagnosis where the controller is required to keep a positive probability of infinite correct runs. This second problem is then showed to be undecidable in the general case and EXPTIME-complete when limited to finite-memory strategies.

The control on the system is not necessarily related to the events, it can also be applied on the observations. The observations of a system are given by sensors. In order to detect an event, one needs to have a sensor at the appropriate position and for it to be switched on. In [START_REF] Cassez | Fault diagnosis with static and dynamic observers[END_REF] and [START_REF] Thorsley | Active acquisition of information for diagnosis and supervisory control of discrete-event systems[END_REF] the authors investigate in slightly different frameworks, how to limit the number of sensors needed and how to build a controller which chooses at every step which sensor is switched on or off. The main differences between the two papers are twofold: (1) [START_REF] Thorsley | Active acquisition of information for diagnosis and supervisory control of discrete-event systems[END_REF] considers both logical and stochastic systems while [START_REF] Cassez | Fault diagnosis with static and dynamic observers[END_REF] only focuses on logical systems and (2) [START_REF] Cassez | Fault diagnosis with static and dynamic observers[END_REF] establishes that a most permissive finite-state controller can be computed in doubly exponential time, using a game-theoretic approach while [START_REF] Thorsley | Active acquisition of information for diagnosis and supervisory control of discrete-event systems[END_REF] does not give the exact complexity of their algorithm.

See [START_REF] Zaytoon | Overview of fault diagnosis methods for discrete event systems[END_REF] for a survey on diagnosis mainly describing results for logical systems, but discussing also timed, stochastic and active systems.

Chapter 3

Semantical analysis of diagnosability

As explained in Chapter 2, diagnosers observe the system to determine if its behaviour is correct or not. They are formalised as functions giving a verdict to each sequence of observation produced by a run of the system. There exist many different ways to define a diagnoser, depending on the properties a system designer may want. We identified as the most important features of a diagnoser its verdict, correctness and reactivity.

• The verdict determines what information is given by the diagnoser. Consequently, changing the verdict literally means modifying the purpose of the diagnoser. For example, when testing a car, if a component malfunctions, the company needs to detect it. The diagnoser thus only needs to detect the faulty behaviour of the system. However, the verdict of a diagnoser could also require to detect the correct behaviour of a system. For example, when following online a critical system like a power plant, the technicians needs to know that the system is correct. If the system can be faulty, they may need to shut it down in order not to take any risk, which can have an important cost.

• The correctness determines if the diagnoser is allowed to make an erroneous claim and, if so, how accurate must its claim be. It is of course better to have a diagnoser that does not make any error, but this restriction is not always realistic. Consider a program simulating dice throws. If the program outputs 4 a high number of times in a row, this may be a correct behaviour as such a throw has a positive probability. However, the longer this streak of 4 continues, the more likely it is caused by a malfunction of the program.

• The reactivity determines how quickly and how often the diagnoser must output a verdict. One possibility would be to require that if a fault occurs, after a bounded delay this fault will be detected. This is the reactivity that is often required in nonprobabilistic systems represented by an automaton [SSL + 95]. This requirement is too strong however for probabilistic systems. Indeed, in probabilistic systems, one can have for example an event which can occur at every step after a fault with some probability, say 1/2, and which allows the detection of the fault. In expectation, the fault is thus detected after two steps, but no bound can be given on the maximum delay. This is a situation occurring in every system where, after a fault, the system can, with some probability, continue to act normally. When studying probabilistic systems, the reactivity requirement must thus be adapted. This can be done, for example, by requiring that with probability 1 the fault will be detected. This version of reactivity allows to ignore runs that have a zero probability of occurring.

The choice of the verdict, correctness and reactivity notions of a diagnoser thus depends on what information the designer of the system desires and on which guarantees are demanded. The decisions that are taken also affect the complexity of determining the diagnosability and of building the diagnoser. They thus also affect the capacities required from the computer that will carry out the diagnosis. Studying as many relevant notions as possible and clearly establishing their complexity is thus necessary for an appropriate application of diagnosis.

In Chapter 2, multiple notions of diagnosability were defined for probabilistic systems based on the ambiguity of specific sets of runs. This allowed us to establish diagnosability analysis as a decision problem which is easier to use in proofs and more intuitive. However, the end goal of analysing the diagnosability of a system being to build a diagnoser, one could also use the following definition: given notions of verdict, correctness and reactivity, a system is called diagnosable if there exists a diagnoser of the system achieving these features. We will show in Section 1 how to reconcile these two approaches: we will associate a diagnoser with each diagnosability notion previously defined. Having defined these associated diagnosers will allow us to see how the notions of diagnosability translate on an actual run of the system.

Once the notions of diagnoser have been clearly defined, we must determine how the different variants of diagnosability relate to one another. Establishing links between multiple notions is a classical part of the semantical analysis of a problem. Through this analysis, one can establish that two definitions that are syntactically different, are in fact equivalent. In this case, one can choose to use either definition in a proof for example. Implications between two notions are also useful. If a system verifies a stronger property, we will not have to check for the weaker ones. For diagnosability, it allows the system designer to select the strongest available diagnoser without having to check every notion. Or at least one of the strongest notions as some of them may be incomparable. Therefore, establishing the links between the notions of diagnosability defined previously will be the focus of Section 2

The last goal of this chapter, presented in Section 3 will be to give, when possible, a characterisation of the notions of diagnosability. As we wish to have the simplest possible characterisation, an important question is to determine what information is needed to characterise diagnosability. For example, can we restrict ourselves to studying the structure of the system or do probabilities matter? And if they do, in what way? In fact, in the general case, a characterisation relies both on the structure of the system and on its probabilities.

• The structural part of the characterisation is based on descriptive set theory recalled in Chapter 2, Section 1.1. We will define a logic generating sets of runs belonging to a low level of the Borel hierarchy and associate, when possible, a formula with every notion of diagnosability.

• The probabilistic part then consists in measuring the probability of the set of runs identified above. The system is then diagnosable if and only if this measure verifies a qualitative requirement.

Having such a characterisation has multiple advantages. For example, using modelchecking techniques, one could use these characterisations to solve diagnosability as we explain in Chapter 5. However this is not necessarily the optimal method as shown in Chapter 4. As another example, each Borel set is associated with a level of the Borel hierarchy. The higher this level, the more complicated the set is. This complexity reflects a complexity to measure the probability of the set but also a complexity of understanding the meaning behind this set. Having a characterisation with a set belonging to a low level of the Borel hierarchy thus makes it easier for the system designer to understand the associated diagnosability notion. This chapter presents and extends some of the results given in [START_REF] Bertrand | Foundation of diagnosis and predictability in probabilistic systems[END_REF][START_REF] Bertrand | Accurate approximate diagnosability of stochastic systems[END_REF][START_REF] Bertrand | Diagnosis in infinite-state probabilistic systems[END_REF].

Diagnoser and diagnosability

In this section we focus on the synthesis of diagnosers for the notions of diagnosis defined in Chapter 2. Recall the definition of diagnosers: Definition 3.1. A diagnoser is a function D : Σ * o → {?, , ⊥} assigning to every finite observed sequence a verdict.

Multiple verdicts can be required for the diagnosers. Intuitively, ? does not provide any information, claims the occurrence of a fault and ⊥ provides information about the correctness of the current run.

Diagnosability as defined in Chapter 2 considers infinite behaviours: either by focusing on the ambiguity of infinite runs, or by requiring that the probability of a set of finite runs converge to 0 when their length diverges to infinity. On the contrary, diagnosers are built to react and give an information after a finite number of observations. There is therefore no easy direct link between diagnosability and diagnosers.

Due to its definition, a diagnoser may use infinite memory or more precisely, unbounded memory. While infinite memory is not achievable in real systems, unbounded memory is. It however raises a question on how to implement this memory and how much information has to be kept by the diagnoser. For example, if a diagnoser only needs to remember how many observations occurred, it may rely on a counter which, although unbounded, is easy to represent and to modify. When implementing a diagnoser (which will be done in Chapter 4), it is still natural to limit oneself to finite memory. We therefore define now the notion of finite-memory diagnosers. Definition 3.2. A finite-memory diagnoser is given by a tuple (M, Σ o , m 0 , up, D fm ) where:

• M is a finite set of memory states,

• m 0 ∈ M is the initial memory state,

• up : M × Σ o → M is a memory update function, • D fm : M → {?, , ⊥} is a diagnoser function.
A finite-memory diagnoser (M, Σ o , m 0 , up, D fm ) can be seen as a deterministic automaton over Σ o where the set of states is M , the initial state is m 0 and the transition function is up. Moreover the states of this automaton are labelled by an element of {?, , ⊥} which is given by the function D fm . The update up is extended into a function up : M × Σ * o → M defined inductively by up(m, ε) = m and up(m, wa) = up(up(m, w), a). The size of a finite-memory diagnoser is given by its number of memory states. A finite-memory diagnoser is not a diagnoser as defined in Chapter 2 and recalled in Definition 3.1, yet it induces the diagnoser D defined by D(w) = D fm (up(m 0 , w)).

Example 3.1. Consider the finite-memory diagnoser (M, {a, b}, m 0 , up, D fm ) (represented in Figure 3.1) where

• M = {m 0 , m b }, • up(m 0 , a) = m 0 , up(m 0 , b) = up(m b , a) = up(m b , b) = m b , • D fm (m 0 ) =? and D fm (m b ) = .
It induces a diagnoser D which makes no claim as long as it only observes 'a' and claims a fault as soon as a 'b' is observed. It then commits to this choice and keeps claiming a fault whatever is observed next. Each following subsection focuses on one notion of diagnosability. They are ordered from the easiest to the most difficult definition of diagnoser. Due to the relations that will be established in Section 2, we won't detail every notion of diagnosability here.

FF-diagnosers

We will start by defining the FF-diagnosers of a pLTS, which is the diagnoser associated with FF-diagnosability. Recall that FF-diagnosability requires the probability of the set of faulty ambiguous finite runs to converges to 0 (i.e. lim n-→∞ P(FAmb n ) = 0). These diagnosers only provide information about faulty runs, they therefore never raise a ⊥ verdict. However they need to raise a verdict almost surely after a finite delay in a faulty run. We can thus restrict their verdict to the set {?, }. We require from diagnosers associated with exact diagnosability notions that they satisfy an additional property,commitment, which means that when it claims a fault it will persistently claim it in the future. This can be done thanks to the permanence of the fault, i.e. a faulty run will remain faulty. Let us comment on this definition. The commitment property ensures that if the diagnoser outputs at some point it will always output . The correctness property forbids the diagnoser to claim a fault during the observation of a correct run. This reflects that FF-diagnosability is a notion of exact diagnosability. Thus the FF-diagnoser is exact too. The limit in the reactivity condition of the above definition is well defined. Indeed, note that if is produced, due to the commitment property, the limit is . Otherwise the diagnoser always outputs ? so that the limit is also a ? verdict.

q 0 f 1 f 2 q 1 f a u a b a Figure 3.2: An FF-diagnosable pLTS.
Example 3.2. For the pLTS of Figure 3.2, we define the diagnoser D such that D(ε) = ?, for all n ∈ N, D(a n ) =? and for every word w ∈ a * , D(w) = . This diagnoser is in fact the one induced by the finite-memory diagnoser of Example 3.1. Such a diagnoser is indeed an FF-diagnoser: commitment Once a 'b' is observed, every subsequent observation is also a 'b', therefore once D produces , it will keep outputting .

correctness If a has been produced, b was observed. The only transition labelled by a 'b' is the self-loop on f 2 which can only be reached by faulty runs. Therefore any observed sequence for which D outputs is surely faulty.

reactivity Let ρ be a minimal faulty run. It ends either in f 1 or in f 2 . If it ends in f 1 , then with probability 1 a run extending ρ takes the transition to f 2 . In f 2 , it will read a 'b' in the next step. In other words, with probability 1, a faulty run will trigger a 'b' and thus is raised by the diagnoser. Due to the commitment property, given a faulty run ρ, the probability that the diagnoser outputs ? infinitely often during the observation of a run extending ρ is thus 0.

Observe also that this pLTS is indeed FF-diagnosable as for n ≥ 1, we have

P(FAmb n ) = P({ρ ∈ F n | P(ρ ↓n ) = a n }) = 1 2 n .
As suggested by the above example, there exists an FF-diagnoser if and only if the system is FF-diagnosable. We now prove this formally.

Proposition 3.1. A pLTS is FF-diagnosable if and only if it admits an FF-diagnoser.

This proof is done in the following way. Assuming there exists an FF-diagnoser we study a family of sets of faulty runs FD n that corresponds to runs where the fault is claimed after n observations. We compute the probability of this set and link this value to the probability of FAmb n . This then allow us to show that the probability of FAmb n converges to 0, proving thus the FF-diagnosability of the pLTS. Assuming the pLTS is FF-diagnosable, we present a diagnoser and then show it is an FF-diagnoser by proving the properties one by one.

Proof. Let A be a pLTS, and assume there exists an FF-diagnoser D for A. For every n ∈ N, we define FD n = {ρ ∈ F ∞ | D(P(ρ ↓n )) = } the set of faulty runs that are diagnosed faulty after n observed events. We will start by showing that lim n-→∞ P(FD n ) = P(F ∞ ). As a consequence of the commitment property, the sequence (FD n ) n∈N is non-decreasing and for every faulty run

ρ ∈ F ∞ , D(P(ρ)) = lim n→∞ D(P(ρ ↓n )) =? is equivalent to ρ / ∈ n∈N (FD n ), i.e. lim n-→∞ FD n = n∈N (FD n ) = {ρ ∈ F ∞ | D(P(ρ)) =?}.
Since D is reactive, for every minimal faulty run ρ ∈ minF, we have P({ρ ∈ Ω | ρ ρ ∧D(P(ρ )) = ?}) = 0. As every faulty run is prefixed by an unique minimal faulty run, we have

P({ρ ∈ F ∞ | D(P(ρ )) =?}) = ρ∈minF P({ρ ∈ Ω | ρ ρ ∧ D(P(ρ )) =?}) = 0. Thus, P( n∈N (FD n )) = P({ρ ∈ F ∞ | D(P(ρ )) =?}) = P(F ∞ ) -P({ρ ∈ F ∞ | D(P(ρ )) =?}) = P(F ∞ ). Moreover since D is correct, for every n ∈ N, FD n ⊆ Sf n . Therefore, for every n ∈ N, P(FAmb n ) = P(F n ) -P(Sf n ) ≤ P(F n ) -P(FD n ) and lim n-→∞ P(FAmb n ) ≤ lim n-→∞ P(F n ) -P(FD n ) = 0.
This shows that A is FF-diagnosable.

Assume now that A is FF-diagnosable. We define the function

D : Σ * o → { , ?} by D(w) =
if and only if w is a surely faulty observed sequence. Let us check that D is an FF-diagnoser. As a surely faulty ambiguous sequence cannot become ambiguous again, D fulfils the commitment property. Moreover, since D(w) = iff w is a surely faulty sequence, D is correct. Now, let ρ be a minimal faulty run.

P({ρ ∈ Ω | ρ ρ ∧ D(P(ρ )) =?}) = lim n→∞ P({ρ ∈ FAmb n+|ρ|o | ρ ρ }) . For every n ∈ N, we have {ρ ∈ FAmb n+|ρ|o | ρ ρ } ⊆ FAmb n+|ρ|o and, as A is FF- diagnosable, lim n→∞ P(FAmb n ) = 0. Therefore P({ρ ∈ Ω | ρ ρ ∧ D(P(ρ )) =?}) = 0 and D is reactive.
The notion of FF-diagnoser we defined is therefore appropriate for FF-diagnosability.

FA-diagnosers

FA-diagnosability and IA-diagnosability not only consider the diagnosis of faults but also of correct runs. Indeed, recall that they require respectively that the probability of the set of ambiguous finite runs converges to 0 (i.e. lim n-→∞ P(FAmb n ∪ CAmb n ) = 0) and that the probability of infinite ambiguous runs is equal to 0 (i.e. P(FAmb ∞ ∪CAmb ∞ ) = 0). Contrary to FF-diagnosers, FAand IA-diagnosers have three possible verdicts: , related to faulty sequences, ⊥, linked with correctness, and ? when no information can be derived from the observation. We consider the partial order ≺ on these values defined by ? ≺ and ? ≺ ⊥. This order is natural as ? gives less information than the other verdicts. Although we consider the detection of correct and faulty runs, note that the situation is not symmetric given that the faults are persistent while correct runs may become faulty.

Definition 3.4. An FA-diagnoser for a pLTS A is a function D : Σ * o → { , ⊥, ?} such that: commitment For every w w ∈ Σ * o , if D(w) = then D(w ) = .
correctness For every w ∈ Σ * o ,

• if D(w) = then w is surely faulty;

• if D(w) = ⊥ then w is surely correct. reactivity P({ρ ∈ Ω | D(P(ρ)) =?}) = 0 where for w ∈ Σ ω o , D(w) = lim inf n→∞ D(w ≤n ).
Let us comment on this definition. The commitment property is similar to the one of the notion of FF-diagnoser. In particular, there is no commitment for a ⊥ verdict. This is natural as a fault could appear later, forcing the diagnoser to change its verdict. The correctness property is also similar to the FF-diagnoser for a verdict. For the ⊥ verdict, it is the dual, the diagnoser cannot output ⊥ while observing a faulty run. This diagnoser is 'exact'. The limit in the reactivity requires the partial order. While if a is claimed, the limit will be , a ⊥ verdict can be followed by a ? verdict. Due to the correctness property, the limit is equal to ⊥ if the observed sequence is, after a finite number of observations, always surely correct. commitment Once a 'b' is observed, every subsequent observation is also a 'b', therefore once D produces , it will keep outputting .

q 0 f 1 f 2 q 1 f a u
correctness If a has been produced, 'b' was observed previously. The only transition labelled by a 'b' is the self-loop on f 2 which can only be reached by faulty runs. Therefore any observed sequence for which D outputs a is surely faulty. Similarly, when D outputs ⊥ it means that a 'c' was observed previously and such an observation can only be made in q 1 which is a correct state from which no fault can ever be triggered. Thus the observed sequence is surely correct.

reactivity The set of infinite observed sequences w for which D(w) =? is restricted to {a ω }. There exist exactly two runs corresponding to this observed sequence ρ 1 = q 0 u(q 1 a) ω and ρ 2 = q 0 f (f 1 a) ω . Moreover, the probability of each of these two runs is 0 as firing 'a' in q 1 or in f 1 only has probability 1/2. Thus with probability 1, a 'b' or a 'c' will be observed, ensuring the reactivity of D.

Observe also that this pLTS is indeed FA-diagnosable as for every n ≥ 1, we have

FAmb n = {q 0 u(q 1 a) n q 1 , q 0 f (f 1 a) n f 1 , q 0 f (f 1 a) n-1 f 1 af 2 }. Thus P(FAmb n ) = 3 2 n+1 .
We now want to establish the link between the existence of an FA-diagnoser and FA-diagnosability. However, there is no equivalence in the general case. Indeed, let us consider the pLTS of Figure 3.5. The probability of the set of ambiguous runs of length n ≥ 1 starting by u 1 is the probability to have read a 'b' on the n -1'th observation. This is equal to 1 n+1 . Moreover, runs initially starting by u 2 will almost surely trigger a 'c', removing the ambiguity. Thus it is FA-diagnosable. However, a run starting with u 1 will almost surely trigger infinitely many 'b's. Because of the correctness property, we have for every diagnoser D, P({ρ ∈ Ω | D(P(ρ)) =?}) ≥ P({ρ ∈ Ω | q 0 u 1 q 1 ρ}) = 1 2 . Thus, D is not reactive. Fortunately, the equivalence holds when restricted to finite pLTS. We postpone the proof of the following proposition to Chapter 4 (more precisely to Proposition 4.14, page 126) which focuses on finite pLTS. Proposition 3.2. A finite pLTS A is FA-diagnosable if and only if it admits an FAdiagnoser.

q 0 q f f 1 q 1 q 2 . . . 1 2 • u 2 1 2 • u 1 1 3 • f 1 3 • a 1 3 • b 1 2 • b 1 2 • c 1/2 • a 1/2 • b 2/3 • a 1/3 • b
As a conclusion, the definition of FA-diagnosers is similar to the one of FF-diagnosers, but with additional requirements to deal with the correct ambiguous runs. This fits the definition of FA-diagnosability which becomes equivalent to the one of FF-diagnosability when the set of correct ambiguous runs can be neglected. However, due to the complexity created by the fact that being correct is not a permanent status of runs (contrary to being faulty), the link between existence of an FA-diagnoser and FA-diagnosability cannot be established in the general case.

IA-diagnosers

The problem in defining IA-diagnosers is that IA-diagnosability defines ambiguity for infinite runs while a diagnoser must give its verdict after a finite observation. As a consequence, the information an IA-diagnoser gives after a finite run is weaker than the information an FA-diagnoser has to give.

Definition 3.5. An IA-diagnoser for A is a function D : Σ * o → { , ⊥, ?} such that: commitment For every w w ∈ Σ * o , if D(w) = then D(w ) = . correctness For every w ∈ Σ * o • if D(w) = , then w is surely faulty; • if D(w) = ⊥, letting |D(w)| ⊥ = |{0 < n ≤ |w| | D(w ≤n ) = ⊥}|, then for every signalling run ρ such that P(ρ) = w, ρ ↓|D(w)| ⊥ is correct. reactivity P({ρ ∈ Ω | D(P(ρ)) =?}) = 0 where for any observed sequence w ∈ Σ ω o , D(w) = lim sup n→∞ D(w ≤n ).
Let us comment on the definition. Commitment is once again focused only on the verdict. Correctness is usual for but quite different for ⊥. Indeed, the correctness of FA-diagnosers requires that a ⊥ verdict means the observed sequence is surely correct. The interpretation of D(w) = ⊥ for IA-diagnoser is that the diagnoser ensures that any signalling subrun of length |D(w)| ⊥ ≤ |w| of a signalling run for w is correct. Of course it may deduce this information from the last |w| -|D(w)| ⊥ observations. This does not reveal if the current run is correct or not. However, if the diagnoser outputs ⊥ infinitely often along an observed sequence w, lim n→∞ |D(w ≤n )| ⊥ = ∞. Therefore the infinite observed sequence w is surely correct. The reactivity condition uses a limit superior as we only need ⊥ to be claimed infinitely often but not necessarily without ? in between. correctness Observing a 'c' can only be made for faulty run, thus D correctly raises a . For the correct runs the idea is the following. After observing any sequence waa or wab, with w ∈ {a, b} * , the diagnoser knows a posteriori that one step before, that is after the observation of wa, the run was necessarily correct. Indeed, observing the suffix aa is not possible after a fault, yet wba is not surely correct.

q 0 q 1 f 1 f 2 f u a b c a, b a, b
After a run ρ with such an observation we thus know that ρ ↓|P(ρ)|-1 is correct and |D(w)| ⊥ is at most equal to |P(ρ)| -2 as no ⊥ can be claimed after 0 or 1 observation. Thus D is correct.

reactivity With probability 1 a faulty run will trigger a 'c' (raising a verdict) and a correct run will trigger infinitely many 'a's (raising infinitely many ⊥ verdicts).

Moreover this pLTS is IA-diagnosable as CAmb ∞ ∪ FAmb ∞ = P -1 ({a, b} * b ω ) which has probability 0 as in every state there is a positive probability to trigger an action whose observation is different than b. We now aim at establishing the link between IA-diagnosability and the existence of an IA-diagnoser. Since IA-diagnosability gives an information about infinite runs, we need a way to translate it to finite runs in order to establish the link with the diagnoser. Hence, we first introduce a lemma linking the sets FAmb n and FAmb ∞ for n ∈ N. This lemma will be reused in the next section when establishing the link between FF-diagnosability and IF-diagnosability.

Lemma 3.1. Let A be a pLTS. Then

lim n→∞ P(FAmb ∞ \ FAmb n ) = 0. Moreover, if A is finitely branching, then lim n→∞ P(FAmb n \ FAmb ∞ ) = 0.
The main difficulty of the following proof is in the second point. There, using the finitely-branching assumption and invoking König's lemma, we show that if the prefixes of a finite run are never surely faulty, then we can build an infinite correct run with the same observed sequence.

Proof. Observe that Ω admits the following partitions

Ω = FAmb ∞ C ∞ Sf ∞ and for all n ∈ N, Ω = FAmb n C n Sf n . Thus, for all n ∈ N, FAmb ∞ \ FAmb n = (C n Sf n ) ∩ FAmb ∞ = (C n Sf n ) \ (C ∞ Sf ∞ ) ⊆ (C n \ C ∞ ) (Sf n \ Sf ∞ ).
Since for all n, Sf n ⊆ Sf ∞ , one gets:

FAmb ∞ \ FAmb n ⊆ C n \ C ∞ .
{C n } n∈N is a non-increasing family of sets and we claim that C ∞ = n∈N C n . Indeed an infinite run ρ is correct if and only if f does not occur in it i.e. if and only if all its signalling subruns are correct. Thus,

lim n→∞ P(C n \ C ∞ ) = 0 which implies lim n→∞ P(FAmb ∞ \ FAmb n ) = 0 .
For the other direction, using again the two partitions we obtain:

FAmb n \ FAmb ∞ = (C ∞ Sf ∞ ) ∩ FAmb n = (C ∞ Sf ∞ ) \ (C n Sf n ) ⊆ (C ∞ \ C n ) (Sf ∞ \ Sf n ).
Since for all n, C ∞ ⊆ C n , one gets:

FAmb n \ FAmb ∞ ⊆ Sf ∞ \ Sf n Let us show that, under the assumption that A is finitely branching, Sf ∞ ⊆ n∈N Sf n . Let ρ / ∈ n∈N Sf n .
We build a tree as follows:

• Nodes at level n correspond to the correct signalling runs whose observed sequence is P(ρ ↓n );

• The node at level n+1 associated with ρ is a child of the node at level n associated with ρ if ρ ρ .

Since ρ / ∈ n∈N Sf n , for all n ∈ N, there exists a correct run with observed sequence P(ρ ↓n ), so that the above-defined tree is infinite. Since the pLTS is finitely branching and convergent, the tree is also finitely branching. By König's lemma, it must contain an infinite branch, thus there exists an infinite correct run whose observed sequence is P(ρ). As a consequence ρ is not surely faulty

: ρ / ∈ Sf ∞ . This establishes that Sf ∞ ⊆ n∈N Sf n . Thus lim n→∞ P(Sf ∞ \ Sf n ) = 0 implying lim n→∞ P(FAmb n \ FAmb ∞ ) = 0 which concludes the proof.
We can now establish the following proposition.

Proposition 3.3. A finitely-branching pLTS A is IA-diagnosable if and only if it admits an IA-diagnoser.

Proof. Assume first that there exists an IA-diagnoser D for A, and let ρ be an infinite run. By reactivity, almost surely D(P(ρ)) ∈ { , ⊥}. If D(P(ρ)) = then there exists some n such that D(P(ρ ↓n )) = . By correctness, ρ ↓n is surely faulty and thus ρ is surely faulty. If D(P(ρ)) = ⊥, we claim that ρ is surely correct. Observe that the diagnoser infinitely often outputs ⊥, so by correctness, for all n, P(ρ ↓n ) is surely correct and thus in particular ρ ↓n is correct. Assume there exists an infinite faulty run ρ with P(ρ ) = P(ρ). There exists a n such that for all m ≥ n, ρ ↓n is faulty. Thus by correctness there can be no more than n ⊥ verdicts for P(ρ) contradicting the fact that D(P(ρ)) = ⊥. Thus with probability 1, an infinite run is unambiguous.

Conversely, assume that A is IA-diagnosable. Given an observed sequence w, we denote by N w the largest integer such that Cyl(P -1 (w)) ∩ F Nw = ∅, i.e. the largest integer such that every run with observation w was correct after N w observations. We define the diagnoser D such that D(ε) =? and for every observed sequence w and observation a ∈ Σ o , if wa is surely faulty, then

D(wa) = , if N wa > N (w), then D(wa) = ⊥ else D(wa) =?.
• Commitment is direct from the definition of D.

• Correctness is achieved as is raised for surely faulty runs and ⊥ is raised when N w (for appropriate observed sequence w) increased, thus for all w ∈ Σ * o , |D(w)| ⊥ ≤ N w which implies that for a run ρ ∈ P -1 (w), ρ ↓|D(w)| ⊥ is correct.

• Reactivity, however is a bit more complicated, we need the result of Lemma 3.1.

Let

ρ ∈ FAmb ∞ ∪ CAmb ∞ .
-If ρ is correct, then suppose that there exists n 0 such that for all n ∈ N, there exists a run ρ f ∈ F n 0 with P(ρ f ↓n ) = P(ρ ↓n ). Then using König's lemma and a construction similar to the one of Lemma 3.1, there exists ρ f ∈ F n 0 such that P(ρ) = P(ρ f ) which would mean ρ ∈ CAmb ∞ and raise a contradiction. Thus, as for each n 0 there exists n ∈ N such that N P(ρ ↓n ) ≥ n 0 , N P(ρ) = lim n-→∞ N P(ρ ↓n ) = ∞. As every time this value increases, ⊥ is produced by D, D outputs infinitely many ⊥, thus D(P(ρ)) = ⊥.

-If ρ is faulty, according to Lemma 3.1,

lim n→∞ P(FAmb n \ FAmb ∞ ) = 0.
Therefore, with probability one, there exists

n 0 ∈ N such that ρ ∈ FAmb n for n ≥ n 0 . Let n 1 such that ρ ∈ F n 1 and n 2 = max(n 0 , n 1 ), then by definition of D, D(P(ρ ↓n 2 )) = D(P(ρ)) = .
Therefore D is reactive.

Thus, D is an IA-diagnoser.

IA-diagnosers are thus appropriately associated with IA-diagnosability. They manage to give information after a finite amount of time about infinite ambiguity. We start to see a complexity hierarchy between the various exact diagnosability notions. FFdiagnosability appears as the simplest notion as its equivalence with the existence of an FF-diagnoser was established in the general case. Second comes IA-diagnosability for which we needed to restrict our framework to finitely-branching pLTS. Finally, the most difficult notion is FA-diagnosability for which the equivalence only holds for finite pLTS. This hierarchy is however only an intuition for now as it could be the result of an inappropriate choice of diagnosers definitions.

εFF-diagnosers

We now define the εFF-diagnosers corresponding to εFF-diagnosability for ε > 0. Recall that εFF-diagnosability requires that, after any minimal faulty run ρ, the probability of the set of faulty runs extending ρ whose observed sequence has a correctness proportion (CorP, defined page 46) above ε converges to 0 (i.e. lim n-→∞ P(FAmb ε n ) = 0)1 . Given ε ≥ 0, εFF-diagnosers are similar to FF-diagnosers in the sense that both only consider faulty runs and thus never output a ⊥ verdict that would give information about correct runs. However, εFF-diagnoser may make errors: when they give a verdict, then the probability that the claim is wrong is at most ε.

Definition 3.6. Let ε ≥ 0. An εFF-diagnoser for A is a function D : Σ * o → { , ?} such that: correctness For every w ∈ Σ * o , if D(w) = then CorP(w) ≤ ε; reactivity lim sup n→∞ P({ρ ∈ F ∩ SR n | D(P(ρ)) =?}) = 0.
Let us comment on this definition. This diagnoser is no longer exact. Given an observed sequence w, it is allowed to output , if the probability of error is below ε as shown by the requirement CorP(w) ≤ ε. There is no longer a notion of commitment, allowing the diagnoser to go back from a verdict to a ? verdict. This absence of commitment is one of the differences between the definition of εFF-diagnosers and the one of monitors for distinguishability of hidden Markov chains [START_REF] Sistla | Monitorability of stochastic dynamical systems[END_REF][START_REF] Kiefer | Distinguishing hidden Markov chains[END_REF]. See the discussion after Lemma 4.3, page 108, for more details about the links between monitoring and diagnosability. Finally, the reactivity condition as defined here is different from what was done for the other diagnosers. This choice was made in order to be closer to the corresponding diagnosability notion as approximate notions of diagnosis are harder to handle than exact ones.

One could introduce a uniform variant of this definition that would correspond to uniform εFF-diagnosability. However, this definition would not follow the same structure as the other ones as a uniform reactivity would have to be defined on individual runs instead of on the global conditions we used here.

Example 3.5. Let us observe the pLTS on the left of Figure 3.8. We define the diagnoser D such that given an observed sequence w ∈ Σ * o , then D(w) = iff w contains at least as many 'b' than 'a', else D(w) =?. Clearly, this diagnoser does not satisfy any

q 0 q f q c f , 1 2 u, 1 2 a, 1 4 b, 3 4 a, 3 4 b, 1 4 q 0 q f q c f , 1 2 u, 1 2 a, 1 4 b, 3 4 a, 1 4 b, 3 4 Figure 3
.8: Left: an 1/2FF-diagnosable pLTS. Right: a pLTS which is not εFFdiagnosable for any ε > 0.

commitment property, but it is not required for εFF-diagnosers. Moreover, this diagnoser cannot be translated into a finite-memory diagnoser as we need to keep the difference between the number of occurrences of 'a' and 'b'. We give a visual representation, using an infinite number of states, of this diagnoser in Figure 3.9. Such a diagnoser is a 1/2FF-diagnoser:

correctness Given an observed sequence w, if D(w) = , then CorP(w) = 3 |w|a 3 |w|a +3 |w| b . As by definition of D, |w| a ≤ |w| b , CorP(w) ≤ 1/2, thus D is correct.
reactivity Let α > 0, as faulty runs have a probability 3/4 to raise a 'b' at each step, according to the weak law of large number there exists n 0 ∈ N such that for all n ≥ n 0 , P({ρ

∈ F n | |P(ρ)| a > |P(ρ)| b }) < α.
Let ρ be a faulty run such that

D(P(ρ)) =?. Thus by definition of D, ρ ∈ {ρ ∈ F n | |P(ρ )| a > |P(ρ )| b }. Therefore, for n ≥ n 0 , P({ρ ∈ F ∩ SR n | D(P(ρ)) =?}) ≤ P({ρ ∈ F n | |w| a > |w| b }) < α. Thus D is reactive.
Observe also that this pLTS is indeed 1/2FF-diagnosable as for n ≥ 1, P(FAmb The diagnoser used in Example 3.5 requires unbounded memory. In fact, there is no finite-memory 1/2FF-diagnoser for the pLTS on the left of Figure 3.8. Proposition 3.4. There exists a 1/2FF-diagnosable pLTS that admits no finite-memory 1/2FF-diagnoser.

1/2 n ) = P({ρ ∈ F n | |P(ρ ↓n )| a > |P(ρ ↓n )| b }) which converges to 0 according to the weak law of large numbers. m 0 m 1 ? m 2 ? . . . m -1 m -2 . . .
Proof. Consider the 1/2FF-diagnosable pLTS on the left of Figure 3.8 and assume there exists a 1/2FF-diagnoser with m states. After any sequence a n for n ≥ 1, it cannot claim a fault as CorP(a n ) = 3 n 3 n +1 > 1 2 . So there exist 1 ≤ i < j ≤ m + 1 such that the diagnoser is in the same memory state after observing a i and a j . Consider the faulty run ρ = q 0 f q f (aq f ) i . Due to the reactivity requirement, there must be a run ρρ for which the diagnoser claims a fault. Thus for all n, the diagnoser also claims a fault after ρ n = q 0 f q f (aq f ) i+n(j-i) ρ but lim n→∞ CorP(P(ρ n )) = 1, which contradicts the correctness requirement.

We now establish the link between εFF-diagnoser and εFF-diagnosability.

Proposition 3.5. Let ε ≥ 0. A pLTS A is εFF-diagnosable if and only if it admits an εFF-diagnoser.
This proof has more computations than the previous ones due to the approximate notion of correctness, however the main ideas are similar: from an εFF-diagnoser we relate FAmb ε n to {ρ ∈ F n | D(P(ρ )) =?} in order to show that the probability of FAmb ε n converges to 0. In the other direction, assuming A is εFF-diagnosable, we build an εFF-diagnoser that is reactive for the runs that do not belong to FAmb ε n infinitely often.

Proof. Let A be a pLTS and ε ≥ 0. Assume that there exists an εFF-diagnoser D for A. Let ρ be a minimal faulty run and α > 0. Since D is reactive, there exists n ρ,α ∈ N such that for all n ≥ n ρ,α ,

P({ρ ∈ F ∩ SR n | D(P(ρ )) =?}) ≤ α • P(ρ) .
Thus for all n ≥ n ρ,α :

P({ρ ∈ SR n+|ρ|o | D(P(ρ )) =? ∧ ρ ρ }) ≤ P({ρ ∈ F ∩ SR n+|ρ|o | D(P(ρ )) =?}) ≤ α • P(ρ) . Since D is correct, Cyl(ρ) ∩ FAmb ε n+|ρ|o ⊆ Cyl({ρ ∈ SR n+|ρ|o | D(P(ρ )) =? ∧ ρ ρ }). Thus P(Cyl(ρ) ∩ FAmb ε n+|ρ|o ) ≤ α • P(ρ)
. This establishes that A is εFF-diagnosable. Conversely assume that A is εFF-diagnosable. Let D be the diagnoser defined by: for all w ∈ Σ * o , D(w) = iff CorP(w) ≤ ε. Such an εFF-diagnoser is correct by definition. Let α > 0. Since (F n ) n∈N is a non-decreasing sequence converging to F ∞ , there exists n 0 ∈ N such that for all n ≥ n 0 , P(F n \ F n 0 ) < α/2. By εFF-diagnosability of A, for all ρ ∈ k≤n 0 minF k , there exists n ρ such that for all n ≥ n ρ

P(Cyl(ρ) ∩ FAmb ε n+|ρ|o ) ≤ α 2 • P(ρ).
Define n max = max ρ∈ n≤n 0 minFn n ρ2 . Then for n ≥ n 0 + n max we have

P(FAmb ε n ) ≤ P(FAmb ε n ∩ F n 0 ) + P(FAmb ε n \ F n 0 ) ≤ ρ∈ k≤n 0 minF k P(Cyl(ρ) ∩ FAmb ε n ) + P(F n \ F n 0 ) ≤ ρ∈ k≤n 0 minF k α 2 • P(ρ) + α 2 ≤ α.
So we have established that

lim n→∞ P(FAmb ε n ) = 0. By definition of D, FAmb ε n = {ρ ∈ F n | D(P(ρ)) =?}. Thus D is reactive.
For every ε ≥ 0 we have thus an appropriate notion of εFF-diagnoser associated with εFF-diagnosability. As a pLTS is AFF-diagnosable if it is εFF-diagnosable for every ε > 0, we directly obtain the following corollary.

Corollary 3.1. A finite pLTS A is AFF-diagnosable if and only if for all ε > 0 it admits an εFF-diagnoser.

In other words, when a pLTS is AFF-diagnosable, the designer can choose the accuracy they want for the diagnoser.

Relationships between diagnosability notions

In this section, we establish the links between the multiple notions of diagnosability defined in Section 1.5. We gave in Section 1 diagnosers associated with the various notions of diagnosability. Thus, intuitively requiring a stronger version of one feature (verdict, correctness or reactivity) defines a diagnoser that gives more information and thus which is less likely to exist. For example, the difference between FF-diagnosability and FA-diagnosability is that FA-diagnosability must identify faulty and correct runs, while FF-diagnosability only cares about faulty runs. Thus FA-diagnosability implies FF-diagnosability, which can be formally proven immediately since for all n, FAmb n ⊆ FAmb n CAmb n . The two notions being entirely distinct as FF-diagnosability does not imply FA-diagnosability as shown by the pLTS of Figure 3.10: there is a single ambiguous observed sequence for every n ∈ N, a n , this sequence can be observed with a probability 1/2 of correct runs and by a probability 1/2 n of faulty runs, i.e. ∀n ∈ N, P(CAmb n ) = 1/2 and P(FAmb

n ) = 1/2 n therefore it is FF-diagnosable without being FA-diagnosable. Similarly, as FAmb ∞ ⊆ FAmb ∞ CAmb ∞ , IA-diagnosability implies IF-diagnosability
The converse is not true however as the pLTS of Figure 3.103 is IF-diagnosable while it is not IA-diagnosable.

An interesting case is the link between IF-diagnosability and FF-diagnosability. Intuitively, as for IF-diagnosability we can observe the infinite run before giving a verdict

q 0 f 1 f 2 q 1 f a u a b a Figure 3.10: An FF/IF-diagnosable pLTS which is not FA/IA-diagnosable.
while FF-diagnosability only allows for a finite observation, FF-diagnosability implies IF-diagnosability. This is indeed true. Using the first direction of the Lemma 3.1 we know that

lim n-→∞ P(FAmb ∞ \ FAmb n ) = 0. Thus if a pLTS is FF-diagnosable, lim n-→∞ P(FAmb n ) = 0, therefore lim n-→∞ P(FAmb ∞ ) ≤ lim n-→∞ P(FAmb ∞ \ FAmb n ) + lim n-→∞ P(FAmb n ) = 0.
Hence the pLTS is IF-diagnosable. Moreover, the implication is strict. Observe the pLTS of Figure 3.11, it is IF-diagnosable (and even IA-diagnosable in fact) as every correct run ends with an infinity of b which cannot be observed with a faulty run and the only faulty run only triggers the observation a, thus FAmb ∞ CAmb ∞ = ∅. However, it is not FF-diagnosable as for all n ∈ N, P(FAmb n ) = 1/2. Indeed, for n ∈ N, there exists a transition from q 0 to a state q n1 labelled by an a and with probability 1/2 n+1 , the one correct run ρ of observable lengthn starting by this transition has observation P(ρ) = a n , thus the only faulty run of observable length n, q 0 f (f 1 a) n f 1 is ambiguous. Moreover this faulty run has probability 1/2. Although we have a strict implication, the pLTS used to prove the strictness has an infinite branching. This is in fact necessary: given a finitely-branching convergent pLTS, IF-diagnosability is equivalent to FF-diagnosability. Given a pLTS A, the implication from IF-diagnosability to FF-diagnosability is shown using once again Lemma 3.1, which, assuming finite branching, says that lim n-→∞ P

(FAmb n \ FAmb ∞ ) = 0. Thus if A is IF-diagnosable, lim n-→∞ P(FAmb n ) = lim n-→∞ P(FAmb n \ FAmb ∞ ) + P(FAmb ∞ ) = 0. Thus A is FF-diagnosable. q 0 q 11 q 21 q 22 f 1 . . . a, 1/4 a, 1/8 a f , 1/2 b b a Figure 3.11: An infinitely-branching pLTS that is IA-diagnosable but not FF- diagnosable.
Although the relationship between IA-diagnosability and FA-diagnosability is the same as the one between IF-diagnosability and FF-diagnosability, the same link cannot be established. Indeed, even for finite pLTS, IA-diagnosability does not imply FAdiagnosability. Let us observe the pLTS of Figure 3.12. It is IA-diagnosable, indeed, every infinite correct run have observed sequence a ω while the observed sequence of every infinite faulty run is of the form

a n b ω for n > 0, thus CAmb ∞ FAmb ∞ = ∅.
Consider however the infinite correct run ρ = q 0 uq 1 (aq 1 ) ω . It has probability 1 2 , and all its finite signalling subruns are ambiguous since their observed sequence is a n , for some n ∈ N which is the observed sequence of the faulty signalling run q 0 u(q 2 a) n-1 q 2 f f 1 af 2 . Thus for all n ≥ 1, P(CAmb n ) ≥ 1 2 , so that this pLTS is not FA-diagnosable.

q 0 q 2 f 1 f 2 q 1 u f a u a b b a Figure 3.12: An IA-diagnosable pLTS which is not FA-diagnosable.
The next theorem summarises the connections between the different diagnosability notions.

Theorem 3.1. The diagnosability notions for pLTS are related according to the diagram below, where arrows represent implications. All implications, except the one from IFdiagnosability to FF-diagnosability and the one from FF-diagnosability to uniform FFdiagnosability, hold for arbitrary infinite-state pLTS. The latter implications holds for finitely-branching pLTS and finite pLTS respectively. Implications that are not depicted do not hold, already in the case of finite-state pLTS.

FA-diagnosable IA-diagnosable uniformly FF-diagnosable FF-diagnosable IF-diagnosable uniformly AFF-diagnosable uniformly εFF-diagnosable AFF-diagnosable εFF-diagnosable 0FF-diagnosable
for finitely branching pLTS for finite pLTS for all ε > 0 for all ε > 0 Let us first describe this diagram. Omitting 0FF-diagnosability which is equivalent to FF-diagnosability, the first two columns correspond to exact diagnosis (thus diagnosis where the diagnoser cannot claim a fault if there is a probability that the fault did not occur) while the last two rows correspond to approximate diagnosis. The lower row contains the notions of diagnosis considering infinite runs, the middle row the ones considering finite runs and the upper rows the ones considering finite runs and requiring uniformity on the speed of reactivity of the diagnoser. From any notion of diagnosability, the notion above it, if there is one, has a more restrictive reactivity and the one on its left requires a better correctness or has a verdict extended to more elements.

FF-diagnosability plays a central role in the diagram as the notion of uniformity and approximate diagnosis was built from this notion. This notion was chosen as it is the traditional notion of diagnosis [START_REF] Thorsley | Diagnosability of stochastic discrete-event systems[END_REF], moreover there is no clear intuition of what approximation and uniformity means for infinite runs.

Proof. We prove all implications that were not already stated in the beginning of this section. Most are pretty straightforward. For all ε > 0, the implications from AFFto εFF, uniform AFFto uniform εFF-and uniform εFF-to εFF-are direct by definitions. The implications from uniform AFFto AFF-, uniform FFto FF-, FFto AFFand uniform FFto uniform AFFcomes partially from definitions and from application of other implications (mostly the equivalence between FFand 0FF-proven below). The most complicated implications are (1) the equivalence between FFand 0FF-diagnosis which is a careful inspection of sets of runs, taking account the possibly infinite branching, (2) the implication from FAto IA-diagnosis which is inspired by Lemma 3.1 and (3) the implication from FFto uniform FF-diagnosability for finite pLTS which requires the characterisation of FF-diagnosability for finite pLTS which will be established later and is thus postponed (See Proposition 4.3, page 95).

FF ⇔ 0FF.

Let A be a 0FF-diagnosable pLTS and ε > 0. Since (F n ) n∈N is a non-decreasing sequence converging to F ∞ , there exists n 0 ∈ N such that for all n ≥ n 0 , P(F n \ F n 0 ) < ε 2 . By 0FF-diagnosability of A, for all ρ ∈ k≤n 0 minF k , there exists n ρ such that for all n ≥ n ρ

P(Cyl(ρ) ∩ FAmb n+|ρ|o ) ≤ ε 4 • P(ρ).
Notice that, because the pLTS may be infinitely branching, the set k≤n 0 minF k may be infinite. We therefore define n max such that P({ρ

∈ k≤n 0 minF k | n ρ > n max }) ≤ ε 4 .
Thus, only a small portion of runs ρ in k≤n 0 minF k have n ρ > n max . Then for n ≥ n 0 + n max we have

P(FAmb n ) ≤ P(FAmb n \ F n 0 ) + P(FAmb n ∩ F n 0 ) ≤ P(FAmb n \ F n 0 ) + P({ρ ∈ k≤n 0 minF k | n ρ > n max }) + P({ρ ∈ FAmb n | ∃ρ ∈ k≤n 0 minF k , ρ ρ , n ρ ≤ n max }) ≤ ε 2 + ε 4 + ε 4 P({ρ ∈ k≤n 0 minF k | n ρ ≤ n max }) ≤ ε .
Let A be a FF-diagnosable pLTS. Consider ρ ∈ minF and α > 0. There exists n 0 ∈ N such that for all n ≥ n 0 , P(FAmb n ) ≤ α • P(ρ). Thus for all n ≥ n 0 :

P(Cyl(ρ) ∩ FAmb n+|ρ|o ) ≤ P(FAmb n+|ρ|o ) ≤ α • P(ρ). FA ⇒ IA.
For all n ∈ N, define CAmb n,∞ the set of correct ambiguous runs that admit an observationally equivalent run which is faulty before its n th observable event.

Observe that the sequence of sets {CAmb n,∞ } n∈N is non-decreasing and that

CAmb ∞ = n∈N CAmb n,∞ . Moreover, by definition, CAmb n,∞ ⊆ CAmb n . As- sume that lim sup n→∞ P(FAmb n CAmb n ) = 0. By Lemma 3.1, P(FAmb ∞ ) = 0.
For all ε > 0, there exists n 1 ∈ N such that for all n ≥ n 1 , P(CAmb n ) < ε and thus P(CAmb n,∞ ) < ε. On the other hand, there exists

n 2 ∈ N such that for all n ≥ n 2 , P(CAmb ∞ ) -P(CAmb n,∞ ) < ε. Combining these two inequalities for n = max(n 1 , n 2 ), one obtains P(CAmb ∞ ) < 2ε. As ε is arbitrary, P(CAmb ∞ ) = 0.
We now provide counter-examples for the implications that do not hold and which were not already developed at the beginning of the section. The most interesting example is the one establishing the difference between FA-diagnosis to uniform εFF-diagnosis as it requires an infinite pLTS.

uniform AFF

IF. Consider the pLTS depicted on the left of Figure 3.8. All infinite faulty runs are ambiguous, and the probability of faulty runs is 1 2 , thus this pLTS is not IF-diagnosable. Fix some ε > 0 and 0 < α < 1. There are two minimal faulty runs ρ a = q 0 f q f aq f and ρ b = q 0 f q f bq f . Consider first ρ a and let ρ be the random variable of a signalling run of length n that extends ρ a . One can express the correctness proportion of ρ in terms of the number of a's in its observed sequence, written |ρ| a :

CorP(ρ) = ( 3 4 ) |ρ|a ( 1 4 ) |ρ|-|ρ|a ( 3 4 ) |ρ|a ( 1 4 ) |ρ|-|ρ|a + ( 1 4 ) |ρ|a ( 3 4 ) |ρ|-|ρ|a .

Simplifying this expression, we obtain:

CorP(ρ) = 1 1+3 |ρ|-2|ρ|a
. Now, by the strong law of large numbers, for any η > 0, there exists n η such that for every n ≥ n η , P(|4|ρ| a -|ρ|| > η) < α. So with probability at least 1 -α, the correctness proportion of ρ is bounded by

1 1+3 η+|ρ| 2
. For a sufficiently large η, this value is smaller than ε, so that

P(CorP(ρ) ≤ ε) ≥ 1 -α.
A similar reasoning applies to ρ b , and one can then take the maximum of the two integers n η to prove that the pLTS is uniformly AFF-diagnosable.

q 0 q 1 q f q c u, 1 2 f , 13 16 u, 1 2 a, 3 16 b, 1 a, 3 4 b, 1 4 Figure 3
.13: An AFF-diagnosable pLTS which is not uniformly εFF-diagnosable for ε < 3/4.

AFF uniform εFF.

Consider the pLTS of Figure 3.13. Fix some 0 < ε < 3 4 , 0 < α < 1 and n α . Consider the minimal faulty run ρ = q 0 uq 1 (aq 1 ) nα+1 f q f bq f . Let ρ be the signalling run of length 2n α + 2 such that ρ ρ . P(ρ ) = a nα+1 b nα+1 . Thus,

CorP(P(ρ )) ≥ 3 4 . So P(Cyl(ρ) ∩ FAmb ε 2nα+2 ) = P(ρ) > α • P(ρ).
Thus the pLTS is not uniformly εFF-diagnosable.

Let ρ be a minimal faulty run. Then P(ρ) = a n 0 b for some n 0 . For all n, let ρ n be the single the single signalling run of observable length |ρ| + n that extends ρ.

It fulfils P(ρ n ) = a n 0 b n+1 and P(ρ n ) = P(ρ). The single correct signalling run ρ n with P(ρ n ) = P(ρ n ) fulfils P(ρ n ) = 3 n 0 2•4 n 0 +n+1
. Thus lim n→∞ CorP(P(ρ n )) = 0. So the pLTS is εFF-diagnosable for all ε > 0 and thus AFF-diagnosable.

FA

uniform εFF when considering infinite pLTS.

Let us consider the pLTS of Figure 3.14. It is FA-diagnosable as almost surely a faulty (resp. correct) run contains a b (resp. c) after a finite number of steps that cannot be mimicked by a correct (resp. faulty) run. We claim that it is not uniformly εFF-diagnosable for all ε such that 0 < ε < 1 2 . Note that for all n ∈ N, CorP(a n ) ≥ 1 2 . Fix some 0 < α < 1 and n α ∈ N. Consider the minimal faulty run ρ = q 0 uf 1 af 2 . . . af nα f f nα . The shortest extension of ρ that is not ambiguous (i.e. contains a b) contains n α + 1 observable events more than ρ does. Therefore,

P({ρ ∈ FAmb ε nα+|ρ|o | ρ ρ }) = P(ρ) > α • P(ρ). q 0 f 0 f 1 f 2 . . . f k f k+1 . . . f 1 f 2 . . . f k f k+1 . . . q 1 q 2 u a a a a a u a, 1 4 f f f f b a, 3 4 c Figure 3
.14: An infinite FA-diagnosable pLTS that is not uniformly εFF-diagnosable.

uniform εFF AFF. Consider the pLTS of Figure 3.15. There is a single signalling minimal faulty run q 0 f q f aq f . Any observed sequence of length at least 1 is ambiguous and corresponds with equal probability to a signalling correct or a faulty run. Consequently it is not AFF-diagnosable, yet it is uniformly εFF-diagnosable for ε = 1 2 .

This concludes the proof.

q 0 q f q c f u a a Figure 3.15: A uniform 1 2 FF-diagnosable pLTS, that is not AFF-diagnosable.

Characterisation of diagnosability

Our goal in this section is to establish "simple" characterisations of the diagnosability notions for a pLTS. More precisely, we aim at studying whether there exists a Borel set B ∈ B that only depends on the underlying LTS such that almost surely a random run belongs to B if and only if the pLTS is diagnosable. Allowing B to depend on the probabilities of the pLTS would give more expressivity, but strongly increases the complexity of the information contained in the set which goes against the goal of a "simple" characterisation. Similarly, as much as possible, we look for a set B belonging to a low level of the Borel hierarchy. Approximate notions of diagnosability heavily depend on the probabilities of the transitions. This can be seen on the two pLTS of Figure 3.8, page 69 for example: while the one on the left is εFF-diagnosable for every ε > 0, the one on the right, which is obtained simply by swapping two probabilities, is not εFF-diagnosable system for all ε > 0. We thus focus here on FA/FF/IA/IF-diagnosis. For these notions, the characterising Borel sets cannot be obtained directly from the definitions and require some machinery. Indeed, the notions of FFand FA-diagnosability are expressed by the limit of the probability of a family of open sets and the notions of IFand IAdiagnosability are expressed by a set which is not a priori a Borel set.

We will specify these Borel sets using a logic called pathL. Logics are tools that are efficient at giving specifications for a system and, thanks to model checking, at deciding if these specifications are satisfied by the system. The complexity of model checking a logic depends on the kind of systems considered (finite, probabilistic,. . . ) and on the power of expressivity of the logic. This makes thus yet another argument for requiring simple characterisations of the diagnosability notions.

In this section, we define the logic pathL in Subsection 3.1. Then, in Subsection 3.2 we give, whenever possible, a characterisation of the different diagnosability problems. Finally in Subsection 3.3, we provide impossibility results that justify why characterisations were not given for some of the diagnosability notions.

The logic pathL

We define the logic pathL in this section. Similarly to Probabilistic Linear Time Logic (PLTL) [START_REF] Courcoubetis | The complexity of probabilistic verification[END_REF], pathL first defines a specification, then specifies a probabilistic condition over this specification. The main difference with PLTL is that pathL is based on the notion of path formulae instead of using atomic propositions. A path formula p is a predicate over finite prefixes of runs. Before defining their syntax formally, let us first give some examples of path formulae.

Example 3.6. Given a finite run ρ = q 0 a 0 q 1 . . . q k , let f be defined by f(ρ) = true if a i = f for some i < k. This path formula characterises the faulty finite runs.

Let U be defined by U(ρ) = true if there exists a correct signalling run ρ with P(ρ) = P(ρ ). If this path formula is false, the current run is surely faulty.

Let us introduce a more intricate path formula. For σ ∈ Σ * o , we define firstf(σ) as the smallest value such that there exists a faulty run with observation σ such that its prefix of length firstf(σ) is faulty, i.e. firstf(σ) = min{k | ∃ρ signalling run P(ρ) = σ ∧ ρ ↓k is faulty} with the convention that min(∅) = ∞. Then the path formula W is defined by: W(ε) = false and W(q 0 a 0 . . . q n+1 ) = true if firstf(P(q 0 a 0 . . . q n+1 )) = firstf(P(q 0 a 0 . . . q n )) < ∞. Every time this path formula is false, we increased the size of the greatest prefix that we are sure is correct.

As shown by the path formula U for example, path formulae are extremely strong as they may depend on the global structure of the system, here by depending on the other existing runs with the same observation. This is far stronger than atomic propositions used in PLTL for example that only depend on the current state of the run.

Formally, a path formula is either generated by a context sensitive grammar or equivalently its acceptance is decided by a linear bounded automaton [START_REF] Kuroda | Classes of languages and linear-bounded automata[END_REF]. In other words, one has to be able to determine the truth of a path formula in linear space.

Example 3.7. Among the examples of path formulae given in Example 3.6, the most difficult one is W. Let us show how one can compute the truth of this path formula in linear space. We first define for σ ∈ Σ * o and q ∈ Q, the restriction of firstf to q: m(σ, q) = min{k | ∃ρ ∈ SR, last(ρ) = q ∧ P(ρ) = σ ∧ ρ ↓k is faulty} with the convention that min(∅) = ∞.

Let ρ be a finite run, σ its observed sequence. If σ = ε, W(ρ) = false. Else σ = σ 1 . . . σ n For every q ∈ Q, we compute the values m(σ, q) and m(σ 1 . . . σ n-1 , q) with the following algorithm:

Algorithm 1 Computing the values of m 1: Input: pLTS A, σ ∈ Σ * o 2: Output: (m(σ ≤k , q)) k≤|σ|,q∈Q 3: for q ∈ Q do 4: m(ε, q) ← ∞ 5: for i = 1 to n do 6: for q, q ∈ Q do 7: m(σ 1 . . . σ i , q) ← ∞ 8: if q ⇒ σ i f q then 9: m(σ 1 . . . σ i , q ) → min(m(σ 1 . . . σ i-1 , q ), i) 10: if q ⇒ σ i q then 11: m(σ 1 . . . σ i , q ) → min(m(σ 1 . . . σ i-1 , q ), m(σ 1 . . . σ i-1 , q))
This algorithm uses linear space in n. Moreover, we have an equivalence between W(ρ) = true and max q∈Q {m(σ 1 . . . σ n-1 , q)} < max q∈Q {m(σ, q)} < ∞. Thus the truth of W can be decided by a linear bounded automaton. It is therefore a path formula.

We now define the syntax of pathL.

Definition 3.7. The syntax of a pathL formula is:

φ ::= p | ¬φ | φ 1 ∧ φ 2 | 3φ
where p is a path formula. In the sequel we use the standard shortcut 2φ ≡ ¬3¬φ.

A formula is evaluated at some position k of a run ρ = q 0 a 0 q 1 . . .. The prefix ρ ≤k of ρ is defined by ρ ≤k = q 0 a 0 q 1 . . . q k . The semantics of pathL is inductively defined by:

• ρ, k |= α if α(ρ ≤k ), note that only the past of ρ is used; • ρ, k |= ¬φ if ρ, k |= φ; • ρ, k |= φ 1 ∧ φ 2 if ρ, k |= φ 1 and ρ, k |= φ 2 ; • ρ, k |= 3φ if there exists k ≥ k such that ρ, k |= φ.
Finally ρ |= φ if ρ, 0 |= φ. Due to the presence of path formulae (with no restriction) this language subsumes LTL and more generally any ω-regular specification language, i.e. any language that can be recognized by an ω-automaton such as a Rabin automaton.

Proposition 3.6. The language generated by pathL subsumes ω-regular languages.

Proof. The language of a deterministic Rabin automaton is determined by a finite family of pair of sets (E i , F i ). It consists of the set of runs ρ for which there exists i ∈ N such that ρ visits finitely often the states of E i and infinitely often the states of F i . We define the path formulae E i and

F i such that E i (ρ) = true iff last(ρ) ∈ E i and F i (ρ) = true iff last(ρ) ∈ F i .
The runs accepted by the Rabin automaton equivalently satisfy the formula i (32(¬E i ) ∧ (23F i )). The language accepted by the Rabin automaton is thus generated by pathL.

In order to reason about the probabilistic behaviour of a pLTS, we introduce the notion of qualitative probabilistic formulae: Definition 3.8. The syntax of a qualitative probabilistic formula of pathL is: P p (φ) with ∈ {<, >, =}, p ∈ {0, 1} and φ ∈ pathL. The semantics is obvious:

A |= P p (φ) if and only if P({ρ ∈ Ω | ρ |= φ}) p.
The set of Borel sets defined by pathL is closed by complementation since the complement of the set of runs generated by a formula φ is generated by the formula ¬φ. Therefore given a pLTS A and a formula φ, A |= P =1 (φ) iff A |= P =0 (¬φ) and A |= P <1 (φ) iff A |= P >0 (¬φ). The qualitative probabilistic formulae can thus be restricted to P =0 (φ) and P >0 (φ).

Logical characterisation of diagnosability

We now exhibit qualitative probabilistic formulae ψ such that a pLTS A is diagnosable iff A |= ψ.

FF-diagnosability. We start with FF-diagnosability. This notion seems to be the easiest one as it focuses on faulty runs and its definition already uses a family of Borel sets. Indeed, we can prove a simple characterisation of FF-diagnosability using the path formulae f and U that we defined in Example 3.6.

Proposition 3.7. Let A be a pLTS. Then A is FF-diagnosable iff A |= P =0 (32(f ∧ U)).
Once a run becomes surely faulty, it cannot become ambiguous. Thus, informally, this formula means that a pLTS is FF-diagnosable if the measure of runs that are infinitely often faulty and ambiguous is equal to 0. These runs are the faulty runs for which the fault can never be claimed by the FF-diagnoser. It is thus natural to require their probability to be equal to 0.

Proof. We write E = {ρ ∈ Ω | 32(f ∧ U)} for the set of runs we are interested in. We further define, for every ρ ∈ minF,

E ρ = {ρ ∈ Ω | ρ ρ ∧ ρ |= 2U} and for every n ∈ N, E n ρ = {ρ ∈ Ω | ρ ρ ∧ ρ |= 2 n U} where ρ |= 2 n φ if for every k ≤ n, ρ, k |= φ. As ρ ∈ minF and ρ , n |= f ∧ U implies ρ [0, n]
is faulty and ambiguous, we have for all n ≥ 0 E n ρ ⊆ FAmb n+|ρ| . Observe that E = ρ∈minF E ρ and that E ρ = ∩ n∈N E n ρ . Thus P(E) = ρ∈minF P(E ρ ) and lim n→∞ P(E n ρ ) = P(E ρ ). • Assume first that P(E) > 0. Then, there exists ρ ∈ minF such that P(E ρ ) > 0. By definition, for every n > |ρ| o P(FAmb n ) ≥ P(E ρ ). Thus, A is not FF-diagnosable.

• Assume now that P(E) = 0. So, for every ρ ∈ minF, P(E ρ ) = 0. Let us pick some ε > 0. Since F = n∈N F n , there exists n 0 such that for every n ≥ n 0 ,

P(F\F n ) ≤ ε 3 . Let R = {ρ ∈ minF | |ρ| o < n 0 }. Pick a finite subset R of R such that ρ∈R\R P(ρ) ≤ ε 3 . Define K = |R |. Let n 1 be such that for every n ≥ n 1 and every ρ ∈ R , P(E n ρ ) ≤ ε 3K . Observe now that for every n ≥ n 0 , FAmb n ⊆ (F \ F n ) ∪ ρ∈R\R C(ρ) ∪ ρ∈R E n ρ . Thus, for every n ≥ n 1 , P(FAmb n ) ≤ ε 3 + ε 3 + K ε 3K = ε. Since ε is arbitrary, A is FF-diagnosable.
IF-diagnosability. IF-diagnosability focuses on infinite runs and FAmb ∞ is not per se a Borel set. Obtaining a characterisation is thus more difficult. Thanks to Theorem 3.1 however, in finitely-branching pLTS the above characterisation also holds for IF-diagnosability.

Corollary 3.2. Let A be a finitely-branching pLTS. Then A is IF-diagnosable iff A |= P =0 (32(f ∧ U)).

IA-diagnosability.

The assumption of finitely-branching pLTS is also needed in order to characterise IA-diagnosability. In addition to the path formula U we also use the path formula W defined in Example 3.6, page 78. Proposition 3.8. Let A be a finitely-branching pLTS. Then A is IA-diagnosable iff

A |= P =0 (32(U ∧ W)).
Intuitively, as a faulty run cannot become correct, if a run does not satisfy U once, then ρ |= 32(¬U). Thus, ρ |= 32(U ∧ W) if either (1) it does not satisfy U at least once, meaning ρ is surely faulty, or (2) does not satisfy W infinitely often. In the latter case, the intuition is the following. Infinitely often firstf has increased. If there exists a faulty run ρ f with P(ρ) = ρ f , then firstf is bounded, on the prefixes of P(ρ), by the length at which ρ f becomes faulty. Thus no faulty run exists with observation P(ρ) and ρ is surely correct.

Proof. In order to prove formally the adequacy of the formula, it is enough to show that ρ ∈ Ω is ambiguous if and only if ρ |= 32(U ∧ W). We focus below on correct runs; the case of faulty runs can be treated in a similar, and even simpler, way.

• Let ρ ∈ CAmb ∞ . Since ρ is ambiguous, there exists a faulty run ρ such that P(ρ ) = P(ρ). Let k 0 be such that ρ ↓k 0 is faulty. Thus for all k ≥ k 0 , firstf(P(ρ ↓k )) ≤ k 0 and in addition it is non-decreasing. So there exists some k 1 ≥ k 0 such that for all k ≥ k 1 , firstf(P(ρ ↓k )) is constant. We thus obtain ρ |= 32W. Moreover, since ρ |= 2 U, we conclude that ρ |= 32(U ∧ W).

• Conversely, let ρ be a correct run such that ρ |= 32(U ∧ W). Thus there is a position k 0 such that for all k ≥ k 0 , ρ, k |= W. In particular, by definition of W, for all k ≥ k 0 , there is a finite signalling run ρ (k) such that P(ρ (k) ) = P(ρ ↓k ) and ρ (k) ↓k 0 is faulty. Consider the tree of these runs ρ (k) by merging the common prefixes. This tree is finitely branching and infinite. By König's lemma, it must admit an infinite branch, corresponding to a run ρ with P(ρ ) = P(ρ) and ρ ↓k 0 faulty. We deduce that ρ is ambiguous. The pLTS of Figure 3.16 illustrates the need for the finitely-branching assumption in Proposition 3.8. The set of unobservable events is {u, f }. Observation b occurs in every infinite correct run, while the observed sequence of the single infinite faulty run is a ω . This pLTS is thus IA-diagnosable. However, it does not satisfy P =0 (32(U ∧ W)) since the unique infinite faulty run has probability 1 2 and satisfies at the same time 2W, by unicity, and 2U. Indeed for every n ∈ N, there is a correct signalling run with observed sequence a n . Observe that the sets of runs specified by the characterisations of FF-diagnosability (32(f ∧ U)) and IA-diagnosability (32(U ∧ W)) are F σ sets, i.e. countable unions of closed sets.

q 0 q f q 1 q 2 q 3 . . . • • • 1 2 • f 1 4 • u 1 8 • u 1 16 • u 1 • a 1 • a 1 • a 1 • b

Non-expressivity results

We chose to use a logic where the structural and probabilistic aspects are separated. This brought some advantages, but also weakens the expressibility. As a consequence, approximate notions of diagnosability for which the probabilities are important to define the ambiguity cannot be characterised. We prove this formally by showing that there is no Borel set E and F such that neither A |= P =0 (E) nor A |= P >0 (F ) characterises 1/2FF-diagnosability. Proposition 3.9. There exists a finitely-branching LTS A such that for every Borel sets E and F of runs, there exists a pLTS A = (A, P) such that:

• either A is 1/2FF-diagnosable and P A (E) > 0;
• or A is not 1/2FF-diagnosable and P A (E) = 0. and

• either A is 1/2FF-diagnosable and P A (F ) = 0; • or A is not 1/2FF-diagnosable and P A (F ) > 0.
This proposition is proved by constructing a family of pLTS whose underlying LTS is the same, thus the Borel set we construct is the same for every member of this family. However the probabilities can be appropriately chosen in order for the pLTS to model or not the formula in contradiction with its diagnosability.

Proof. Consider the LTS A = Q, q 0 , Σ, T defined as follows and let the set of unobservable events be Σ u = {f , u}:

• Q = {q 0 , q f , q c }, • Σ = {a, b, f , u},
• T = {(q 0 , u, q c ), (q 0 , f , q f ), (q c , a, q c ), (q c , b, q c ), (q f , a, q f ), (q f , b, q f )}.

We consider a family of pLTS, represented in Figure 3.17 with underlying LTS A. Given a pair of probabilities (p 1 , p 2 ), we define the pLTS A (p 1 ,p 2 ) = (A, P (p 1 ,p 2 ) ) in which

P (p 1 ,p 2 ) (q 0 , f , q f ) = P (p 1 ,p 2 ) (q 0 , u, q c ) = 1/2, P (p 1 ,p 2 ) (q c , a, q c ) = p 1 , P (p 1 ,p 2 ) (q c , b, q c ) = 1 -p 1 , P (p 1 ,p 2 ) (q f , a, q f ) = p 2 and P (p 1 ,p 2 ) (q f , b, q f ) = 1 -p 2 . First note that A (p 1 ,p 2 ) is 1/2FF-diagnosable iff p 1 = p 2 .
This can be established similarly to what was done in Example 3.5, page 68. In fact one can show that

A (p 1 ,p 2 ) is AFF-diagnosable iff p 1 = p 2 . q 0 q f q c f , 1 2 u, 1 2 a, p 2 b, 1 -p 2 a, p 1 b, 1 -p 1 Figure 3
.17: A family of pLTS whose underlying LTS has no appropriate characterisation for 1/2FF-diagnosability.

Let E be an arbitrary Borel set over the set of runs of A. If there exists a probability value p such that A (p,p) |= P =0 (E) we have the first part of the result. Else, let p and p be two probabilities with p = p . As the probabilities of the runs of E in A (p,p ) do not depend simultaneously on p and on p ,

P A (p,p ) (E) + P A (p ,p) (E) = P A (p,p) (E) + P A (p ,p ) (E).
Moreover, for every probability p , A (p ,p ) |= P =0 (E), thus either P A (p,p ) (E) > 0 or P A (p ,p) (E) > 0 which concludes the first part of the proof.

Let F be an arbitrary Borel set. If there exists a probability p such that A (p,p) |= P >0 (E) we have the first part of the result. Else, let p and p be two probabilities with p = p . As before, P A (p,p ) (F ) + P A (p ,p) (F ) = P A (p,p) (F ) + P A (p ,p ) (F ) = 0. Thus both A (p,p ) |= P =0 (F ) and A (p ,p) |= P =0 (E) which concludes the second part of the proof.

For exact notions of diagnosability, intuitively, FA-diagnosability would be in between FF-diagnosability and IA-diagnosability in terms of complexity. Surprisingly we showed that FA-diagnosability does not admit such a characterisation: there is no F σ set E such that a pLTS A is FA-diagnosable if and only if A |= P =0 (E). Proposition 3.10. There exists a finitely-branching infinite LTS A such that for every F σ set E of runs, there exists a pLTS A = (A, P) such that:

• either A is FA-diagnosable and P A (E) > 0; • or A is not FA-diagnosable and P A (E) = 0.
This proposition is proved in a similar fashion as Proposition 3.9, it is only more involved. The family of pLTS we construct has infinitely many states and infinitely many parametric probabilities. These probabilities can be chosen in order to give a positive value to either the limit of the probability of CAmb n or to the given F σ set.

Proof. Consider the LTS A = Q, q 0 , Σ, T defined as follows:

• Q = {f 1 , q f } ∪ {q i | i ∈ N}; • Σ = {a, b, c, u, f };
• T = {(q 0 , u, q f ), (q 0 , u, q 1 ), (q f , a, q f ), (q f , b, q f ), (q f , f , f 1 ), (f 1 , b, f 1 ), (f 1 , c, f 1 )} ∪ {(q i , a, q i+1 ), (q i , b, q i+1 )} i≥1 ;

• Σ u = {f , u}. We consider a family of pLTS, represented in Figure 3.18, with underlying LTS A.

q 0 q f f 1 q 1 q 2 . . . 1 2 • u 1 2 • u 1 3 • f 1 3 • a 1 3 • b 1 2 • b 1 2 • c (1 -p 1 ) • a p 1 • b (1 -p 2 ) • a p 2 • b
For p = (p n ) n≥1 a sequence of probabilities, we define the pLTS A p = (A, P p ) in which for every n ≥ 1 the probability that 'b' occurs from state q n is P p (q n , b, q n+1 ) = p n , the probability that 'a' occurs from state q n is P p (q n , a, q n+1 ) = 1 -p n and all other probabilities are independent of p: P p (q 0 , u, q f ) = P p (q 0 , u, q 1 ) = P p (f 1 , b, f 1 ) = P p (f 1 , c, f 1 ) = 1 2 , P p (q f , a, q f ) = P p (q f , b, q f ) = P p (q f , f , f 1 ) = 1 3 . Observe that a faulty run almost surely produces a 'c', so that lim n→∞ P(FAmb n ) = 0. Moreover, as the ambiguous runs are the ones ending by a 'b', the probability to be ambiguous after n observations on the leftmost part is 1 2 2 n-1 3 n and on the rightmost part is 1 2 p n P(CAmb n ) = p n + 2 n-1 3 n . Therefore, A p is FA-diagnosable iff lim n-→∞ p n = 0. Let E be an arbitrary F σ set. We pick some FA-diagnosable A p i.e. such that lim n-→∞ p n = 0 and write P p for the probability measure it induces. If P p (E) > 0 we are done. Assume thus that P p (E) = 0. In order to define a second pLTS, via p , consider an infinite increasing sequence {n j } j≤1 and let for n / ∈ {n j } j≤1 , p n = p n and for n ∈ {n j } j≥1 , p n = 1 2 . Due to the sub-sequence p n j = 1 2 , A p is not FA-diagnosable. The sequence {n j } j≤1 depends on P p and will be defined after some preliminary observations. Let F = {ρ | q 0 uq 1 ρ}. Denoting P p the probability measure of the second pLTS, observe that

P p (E \ F ) = P p (E \ F ) = 0. Using the above discussion, the F σ set E ∩ F = m∈N n∈N O m,n where for all m, n, O m,n is a disjoint union of cylinders C(ρ) with |ρ| = n, O m,n+1 ⊆ O m,n and O m,n ⊆ O m+1,n . Denote F m = n∈N O m,n For all m, lim n→∞ P p (O m,n ) = P p (E ∩ F m ) ≤ P p (E ∩ F ) = 0.
• n 1 is chosen such that for all n ≥ n 1 , p n ≤ 1 2 . Observe now that for all n j ,

p n j = 1 2 = 1 2p n j p n j and 1 -p n j = 1 2 ≤ 1 -p n j ≤ 1 2p n j (1 -p n j ).
By definition of P p , since O m,n is a disjoint union of cylinders C(ρ) with |ρ| = n, applying inductively the previous inequalities, for all n such that n k < n ≤ n k+1 (denoting n 0 = 0):

P p (O m,n ) ≤ P p (O m,n ) 2 k 1≤j<k p n j . (3.1)
• Assume that we have chosen n 1 , . . . , n k . Since lim n→∞ P p (O k,n ) = 0, there exists n k+1 > n k such that P p (O k,n k+1 ) ≤ 1≤j≤k p n j . We choose such an index.

Equation (3.1) now implies that for all m ≤ k,

P p (O m,n k+1 ) ≤ P p (O k,n k+1 ) ≤ 1 2 k .
Thus for all m,

P p (F m ) = lim k→∞ P p (O m,n k+1 ) = 0. Since E ∩ F = m∈N F m , P p (E ∩ F ) = 0 and so P p (E) = 0.
Beyond Proposition 3.10, we conjecture that the impossibility also holds for arbitrary Borel sets.

Proposition 3.10 only shows the non-existence for characterisations that requires a null probability of the given set. There could thus still exists a characterisation asking for a positive probability. In fact, such a characterisation does not exist. This impossibility is even stronger than the one of Proposition 3.10 as we show that a positive probability characterization cannot exist whatever the Borel set (and not only F σ ).

Proposition 3.11. There exists a finitely-branching LTS A such that for every Borel set E of runs, there exists a pLTS A = ((A, P), Σ o , P) such that:

• either A is FA-diagnosable and P A (E) = 0;

• or A is not FA-diagnosable and P A (E) > 0.
This proof is similar to the previous one, albeit with an even more complex family of pLTS. Indeed, here, instead of having a parametric probability p i for every i ∈ N we have one for every word w ∈ {a, b} * .

Proof. Consider the LTS A = Q, q 0 , Σ, T defined as follows:

• Q = {f 1 , q f , q 0 } ∪ {q w | w ∈ (a + b) * }; • Σ = {a, b, c, u, f };
• T = {(q 0 , u, q f ), (q 0 , u, q 1 ), (q f , a, q f ), (q f , b, q f ), (q f , f , f 1 ), (f 1 , b, f 1 ), (f 1 , c, f 1 )} ∪ {(q w , a, q wa ), (q w , b, q wb )} w∈(a+b) * ;

• Σ u = {f , u}.

We consider a family of pLTS, represented in Figure 3.19, with underlying LTS A, parametrised by a mapping p : (a + b) * → (0, 1). Let A p = ((A, P p ), Σ o , P) be the pLTS such that the probability that 'b' occurs from state q w is P(q w , b, q wb ) = p(w), the probability that 'a' occurs from state q w is P(q w , b, q wa ) = 1 -p(w) and all other probabilities are independent from p: P p (q 0 , u, q f ) = P p (q 0 , u, q 1 ) =

P p (f 1 , b, f 1 ) = P p (f 1 , c, f 1 ) = 1 2 , P p (q f , a, q f ) = P p (q f , b, q f ) = P p (q f , f , f 1 ) = 1 3 .
In the sequel, for convenience, we also write p(w, b) for p(w), and define p(w, a) = 1 -p(w), so that P(q w , a, q wa ) = p(w, a). Word w can be decomposed into letters w = w[1] . . . w[n], and we give notations for factors:

q 0 q f f 1 q ε q w q wa q wb q a q b w . . . . . . 1 2 • u 1 2 • u 1 3 • f 1 3 • a 1 3 • b 1 2 • b 1 2 • c ( 1 -p ( ε ) ) • a p ( ε ) • b ( 1 -p ( w ) ) • a p ( w ) • b
w[1, k] = w[1] . . . w[k] with the convention that w[1, 0] = ε. Finally we define p p (w) = 1≤k≤n p(w[1, k -1], w[k]
), as the probability to read w from q ε . Since

lim n→∞ P(FAmb n ) = 0 and P(CAmb n-1 ) = w||w|=n-1 p(w, b) + 2 n-1 3 n , we deduce that A p is FA-diagnosable iff lim n-→∞ |w|=n-1 p(w, b) = 0.
Let E be an arbitrary measurable set. Pick some pLTS A p which is FA-diagnosable, i.e. with lim n-→∞ |w|=n-1 p(w, b) = 0. If P p (E) = 0 where P p is the probability of this pLTS, we are done. Assume therefore that P p (E) > 0. Let F = {ρ | q 0 uq ε ρ} be the set of runs starting with a u-transition to q ε . Denoting P p the probability measure of any other pLTS A p , observe that P p (E \ F ) = P p (E \ F ). So, if P p (E \ F ) > 0, then by picking any non FA-diagnosable (A, P p ), we are done. So assume P p (E \ F ) = 0 which implies P p (E ∩ F ) > 0. The probability being an inner regular measure (recall Definition 2.4, page 37), there exists a closed set G ⊆ E ∩ F with P p (G) > 0.

If G = F then P p (G) = P p (G) = 1 2 .
In this case, we can therefore conclude by picking any non FA-diagnosable pLTS A p .

Assuming G F , since G is closed, there is some cylinder C(ρ) with ρ = q 0 uq ε . . . q w such that G ∩ C(ρ) = ∅. Then we define the pLTS A p as the pLTS A p except that for every w w and every x ∈ {a, b}, p (w , x) = 1 2 . Thus for every n ≥ |w|,

|w |=n p (w , b) ≥ Pp(ρ) 2 . So A p is not FA-diagnosable. On the other hand, P p (E ∩ F ) ≥ P p (G) = P p (G) > 0.
With Proposition 3.10 and Proposition 3.11 FA-diagnosability appears to be the most complicated of the exact diagnosability notions. Indeed, it cannot be characterised by any set of at least the first two level of the Borel hierarchy contrary to the other notions. This confirms the intuition that was raised when defining the diagnosers associated with each notion of diagnosability. For approximate notions of diagnosability, the nonexpressibility result of Proposition 3.9 clearly shows that studying these notions requires a characterisation that intertwines structural and probabilistic elements.

Conclusion

In Section 1, appropriate diagnosers were associated with notions of diagnosability. This gave another point of view on the notions of diagnosability and raised the question of the memory that is necessary for a diagnoser. We showed that for approximate notions of diagnosability, unbounded memory may be necessary even for finite systems. However for exact notions of diagnosability, every example we gave used finite memory.

In Chapter 4, we study diagnosability of finite pLTS and establish what memory is needed for each notion of exact diagnosability.

In Section 2, the links between the various diagnosability notions were established. This showed multiple interesting facts. In non-probabilistic systems, diagnosability only focuses on faulty runs as, for finitely-branching systems, if there is an infinite faulty ambiguous run, there also exists an infinite correct ambiguous run. However, when probabilistic models are considered, this symmetry is broken. Caring about both faulty and correct runs make for an entirely new notion. Moreover, probabilities do not affect correct and faulty runs in the same way concerning ambiguity: while IFand FFdiagnosability are equivalent for finitely-branching systems, IAand FA-diagnosability are not equivalent even for finite systems. Another interesting point, the uniformity requirement is not as important for exact notions of diagnosis than for approximate ones. Last, while AFF-diagnosability could appear to be close to FF-diagnosability as it forces an arbitrary high accuracy, the two notions are still very different.

In Section 3, characterisations composed of a probabilistic requirement on a set of paths defined by a pathL formula were established, when possible, for the notions of diagnosability. The notions which could not be characterised where the approximate notions of diagnosability (this is due to the sort of characterisation we were looking for) and FAdiagnosability. The latter is quite surprising as the definition of FA-diagnosability seems to be in between the ones of FFand IA-diagnosability. Using model-checking techniques, one could derive an algorithm to decide a notion characterised by a logical formula. The absence of characterisation for approximate diagnosability and FA-diagnosability does not however mean that there is no algorithm for these problems. Indeed, we show in Chapter 4 that for finite pLTS, one can give a more specific characterisation of the exact diagnosability notions, including the FA-diagnosability. These characterisations have the same descriptive complexity and are used to obtain an algorithm. In Chapter 5, we study diagnosability for infinite-state systems and base our approach on the characterisations proved in the current section.

Chapter 4

Algorithmic analysis of the diagnosability of finite pLTS

In Chapter 3, we studied the notions of diagnosability in a general setting. More precisely, we showed that a diagnoser could be associated with each notion of diagnosability, we established the links between the different diagnosability definitions and gave a logical characterisation of multiple versions of exact diagnosability. With additional assumptions, stronger results can be established: characterisations can be refined and efficient algorithms can be designed. One of the usual restriction is the finiteness of systems. Many real systems can be represented with finitely many states (a vending machine, a Pac-Man game, . . . ). In fact, most of the systems which interaction with the environment only requires to keep in memory a finite number of events would fit in such a framework. For example, let us consider a server that receives and processes requests. If the server uses a finite memory, then after memorising a fixed number of yet unprocessed requests, the next request might be discarded without being processed due to a stack overflow. Such servers can be represented with finitely many states. Servers able to memorise an unbounded number of requests is dealt with in Chapter 5. When considering finite systems, we immediately get some results as consequences of the ones of Chapter 3. For example, any finite system being finitely branching, according to Theorem 3.1, page 73, IF-diagnosability and FF-diagnosability, two diagnosability notions focused only on faulty runs, are equivalent. As another example, every logical characterisation given in Section 3 of Chapter 3 applies.

The first step is to establish what additional results can be obtained for diagnosability when we only study systems with finitely many states. These systems can easily be represented and have important properties that do not hold in the general case. For example, there is a finite number of probability values in the system since there is a finite number of transitions. This is not true in the general case as can be seen in the example of Figure 3.11, page 72. Using these additional properties, we establish new characterisations of the diagnosability notions in Section 1. These characterisations are not given as probabilistic logical formulae but as conditions on the structure or on the probabilities of the language observed in the system. More precisely, the characterisa-tions of the exact diagnosability notions are purely structural (i.e the probability values do not matter): they rely on a variant of the determinisation of the system. For the approximate diagnosability notions, the probabilities matter, and cannot be removed from the characterisation. The given characterisation relies on the comparisons of the probability of sets of observed sequences that can be observed from pairs of states.

Once the characterisations are established, we apply them to get as many algorithmic and complexity results as possible. One such application, is to provide an algorithm to decide the diagnosability of a system. Indeed, given a "simple" characterisation, one can easily check if it holds on a given system. Checking a characterisation can be done with many different algorithms, some being more efficient than others. The efficiency of an algorithm is described by the complexity class the problem belongs to, in computational complexity theory. When possible we provide algorithms that are optimal with respect to the standard computational classes. For the other diagnosability problems, we establish that the associated decision problem is undecidable. This is done in Section 2.

When a system is diagnosable, there exists a diagnoser for this system. The possible diagnosers use different amount of memory, give their verdict more or less quickly, etc. In Section 3, we show how to automatically construct a diagnoser for systems that are exactly diagnosable. The diagnosers we build have optimal memory and give their verdict as soon as possible. The constructions of the exact diagnosers are based on the characterisations given in Section 1. This strengthens the importance of these characterisations.

This chapter develops and extends some of the results from [START_REF] Bertrand | Foundation of diagnosis and predictability in probabilistic systems[END_REF][START_REF] Bertrand | Accurate approximate diagnosability of stochastic systems[END_REF].

Characterisations of diagnosability

In this section, we establish characterisations for the different notions of exact diagnosability and for one notion of approximate diagnosability. These characterisations strongly rely on the restriction to finite state systems. Therefore they are easier to express and check, but in general cannot be adapted to more general cases as will be seen in Chapter 5. As a direct consequence of the finite-state restriction and of Theorem 3.1, page 73, FF-diagnosability and IF-diagnosability coincide. So we only consider FF-diagnosability in the rest of this chapter. For all the exact diagnosability notions, the methodology is similar. We first construct an ad hoc deterministic automaton which gathers all the information needed for the diagnosis, by tracking possible correct and faulty executions. Secondly, we build the product of the original pLTS with this deterministic automaton1 . Diagnosability can then be characterised on the product by graph-based properties.

For approximate diagnosability, we show that the diagnosability notions can be characterised relying on the distance 1 problem for labelled Markov chains (LMC). This problem, recalled in Chapter 2, receives as input two LMC and asks for the existence of an event, that is almost sure in one LMC, and has null measure in the other. It was shown to be decidable in PTIME [START_REF] Chen | On the total variation distance of labelled Markov chains[END_REF].

Exact diagnosis

The deterministic automata we build are variants of the deterministic Büchi automaton introduced in [START_REF] Haar | Optimal constructions for active diagnosis[END_REF], that accepts the infinite unambiguous observed sequences. The latter tracks the subsets of possible states reached by signalling runs associated with an observed sequence. It looks like the on-the-fly determinisation of A viewing unobserved events as silent transitions. However, in view of the forthcoming characterisations, the subsets of correct and faulty states are divided in three sets: U , V and W . The intuitive meaning of these sets is the following one:

• A state q belongs to U , if there is a correct signalling run with the current observed sequence ending in q;

• A state q belongs to V ∪ W if there is a faulty signalling run with the current observed sequence ending in q.

• The partition between V and W ensures that for all q ∈ V , q ∈ W and ρ a faulty run ending in q, there exists a faulty run ρ ending in q with an earlier fault than the fault of ρ. In other words, V and W contain the states reached by faulty runs, while W keeps track of the runs that have been faulty for the longest time.

W corresponds to the set of faulty states for which the ambiguity with the correct states of U has to be resolved (when both are not empty), while V corresponds to a waiting room of states reached by faulty runs that will be examined when the current ambiguity is resolved. Before giving the definition of the deterministic automaton associated with a pLTS and for sake of readability, we define for two sets of states U and V and an observation a ∈ Σ o the set of states

updatefaulty(U, V, a) = {q | ∃q ∈ U, q ⇒ a f q} ∪ {q | ∃q ∈ V, q ⇒ a q}.
This set contains the states reached from U by a faulty signalling run of observation a and the ones reached from V by a signalling run of observation a. Definition 4.1. Given a pLTS A, the deterministic automaton associated with A is Obs(A) = {S, s 0 , ∆, F } where

• s 0 = ({q 0 }, ∅, ∅) is the initial state of Obs(A);
• the states and transitions of the deterministic Büchi automaton Obs(A) are inductively defined by: Given (U, V, W ) a state of Obs(A) and a ∈ Σ o , there exists a state (U , V , W ) ∈ S and a transition (U, V, W ) a -→ (U , V , W ) in ∆ as soon as:

1. {q ∈ U ∪ V ∪ W | q ⇒ a } = ∅, 2. U = {q | ∃q ∈ U, q ⇒ a c q}, 3. If W = ∅ then V = ∅ and W = updatefaulty(U, V, a), 4. If W = ∅ then W = updatefaulty(∅, W, a) and V = updatefaulty(U, V, a)\ W ;
• the set F of accepting states consists of all triples (U, V, W ) with U = ∅ or W = ∅.

When U = ∅, the current signalling run is surely faulty, since U tracks the possible states after a correct run. When W = ∅ the current signalling run may be ambiguous (if V = ∅) but the "oldest" possible faulty runs under scrutiny have been discarded. Hence, any infinite observed sequence of A passing infinitely often through F is not ambiguous (either it is surely faulty, or ambiguities are resolved one after another).

Example 4.1. Let A be the pLTS of Figure 4.1. We represent the associated deterministic automaton Obs(A) in Figure 4.2, where accepting states for the Büchi condition are doubly framed.

q 0 q 2 f 1 f 2 q 1 u f a u a b b a Figure 4.1: An IA and FF-diagnosable pLTS which is not FA-diagnosable.
Let us consider the observed sequence a ω which has probability 1/2 in the pLTS. The initial state of Obs(A) is ({q 0 }, ∅, ∅} meaning that the initial state of the pLTS is q 0 and no fault occurred so far. Then, observing 'a' we reach ({q 1 , q 2 }, ∅, {f 2 }) meaning that from q 0 , we can reach q 1 and q 2 with correct signalling run of observation 'a' and f 2 with a faulty signalling run of observation 'a'. As this faulty run is the oldest one, it is added to W . The second observed 'a' leads to the state ({q 1 , q 2 }, {f 2 }, ∅). The run that ended in f 2 previously cannot be extended by observing an 'a', thus the set W which tracked this run (as the only oldest faulty run) is empty. However a new fault can be created ending in state f 2 , this fault being made while W was tracking other faults, it is not considered "old" and thus is added to V . This state is therefore an accepting state. Observing yet another 'a' leads back to ({q 1 , q 2 }, ∅, {f 2 }). The run tracked by V disappeared, a new fault was created and as W was empty it is considered a "old" fault and f 2 is put in W . As the path of Obs(A) corresponding to the observed sequence a ω alternates between these two states infinitely often, it visits infinitely often a state where W = ∅, thus a ω is accepted.

For the other observed sequences, a 'b' can only be observed in a faulty state, therefore any observed sequence containing a 'b' is surely faulty. This can be seen in Obs(A) as any path containing a 'b' ends in one of the five rightmost states, all of which have an empty set U (the first component corresponding to the correct reachable states). As ure 4.1 is accepted by the associated deterministic automaton. According to Proposition 4.1, this means that there is no infinite ambiguous sequence in this pLTS, thus it is FFand FA-diagnosable.

{q 0 }, ∅, ∅ s 0 {q 1 , q 2 }, ∅, {f 2 } s 1 {q 1 , q 2 }, {f 2 }, ∅ s 1 ∅, {f 1 }, {f 2 } s 2 ∅, ∅, {f 1 , f 2 } s 2 ∅, ∅, {f 1 } s 3 ∅, ∅, {f 2 } s 4 ∅, {f 2 }, ∅

FF-diagnosability

As explained earlier, for each diagnosability notion, we consider a variant of Obs(A). For FF-diagnosability, we only need to remove the ambiguity for faulty runs So we can omit the faulty sets of states V and W . We write FF(A) for the resulting simplified automaton, called FF-automaton, obtained from Obs(A) by only considering the Ucomponent of states. To recover the stochastic behaviour of A which is not reflected in FF(A), we now define the pLTS A FF = A × FF(A) as the product of A and FF(A) synchronised over observed events. Definition 4.2. Given a pLTS A = Q, q 0 , Σ, T associated with the FF-automaton FF(A) = {S, s 0 , ∆, F }, we define A FF = Q , (q 0 , {q 0 }), Σ, T where:

{q 0 } s 0 {q 1 , q 2 } s 1 ∅ s 2 a b b a, b a
• Q = Q × S;
• ((q, B), a, (q , B)) ∈ T iff (q, a, q ) ∈ T and

-either a ∈ Σ u and B = B , -or a ∈ Σ o and there exists a transition B a -→ B in ∆.
Since FF(A) is deterministic and complete, A FF is still a pLTS, with the same stochastic behaviour as A. More precisely, there is a bijection between the runs of A and the runs of FF(A). A run and its image by the bijection have the same observation and the same probability. In addition, the U -component of a state (q, U ) of A FF stores the relevant information w.r.t FF-diagnosability of the observed sequence so far.

Example 4.4. Carrying on with the example pLTS of Figure 4.1, Figure 4.4 shows the resulting product pLTS. Observe that it has two bottom strongly connected components (BSCC), consisting of the absorbing states (q 1 , s 1 ) and (f 2 , s 2 ). q 0 , s 0 q 2 , s 0 In a finite pLTS almost all runs ends in a BSCC [START_REF] Baier | Principles of model checking[END_REF], and FF-diagnosability is a property of runs expressed as an almost sure event. So, the characterisation of FF-diagnosability can be stated on the BSCC of A FF . Proposition 4.2. Let A be a finite pLTS. Then A is FF-diagnosable if and only if A FF has no BSCC containing a state (q, U ) with q ∈ Q f and U = ∅.

q 2 , s 1 f 1 , s 0 f 1 , s 1 f 1 , s 2 f 2 , s 1 f 2 , s 2 q 1 , s 0 q 1 , s 1 u f f a
The proof relies on two elements. First, a run will almost surely reach a BSCC of the pLTS. Second, for a faulty run to be ambiguous, the set U contained in the state in which the run ends must not be empty. Therefore if every BSCC corresponds to either correct runs (i.e. q ∈ Q c ) or faulty unambiguous runs (i.e. U = ∅), the pLTS is FF-diagnosable.

Proof. Suppose first that there exists a reachable BSCC C of A FF and a state s = (q, U ) in C such that q ∈ Q f and U = ∅. Let ρ be a signalling run leading from the initial state s 0 of A FF to s. Now, for every state s = (q , U ) ∈ C, necessarily q ∈ Q f and U = ∅, because C is strongly connected2 . So for every signalling run ρ that extends ρ, writing s = (q , U ) for the state ρ leads to, there exists a correct signalling run ρ such that P(ρ ) = P(ρ ) and q 0 ρ -→ q with q ∈ U . As a consequence the observed sequence P(ρ ) is ambiguous in A FF , and for every n ≥ |ρ| o , P(FAmb n ) ≥ P(ρ). As A and A FF have the same ambiguous observed sequences and the associated runs have the same probabilities, A is not FF-diagnosable.

Suppose now that for every state s = (q, U ) of a BSCC C, either q ∈ Q c , or U = ∅. This property is in fact uniform by BSCC: for every BSCC C, either for every state (q, U ) ∈ C, q ∈ Q c , or, for every state (q, U ) ∈ C, U = ∅. This is a straightforward consequence of C being strongly connected. Moreover, if a run ρ reaches a pair (q, U ) then q ∈ Q c implies U = ∅. Indeed, let ρ be the greatest signalling run prefix of ρ. ρ ends in a pair (q , U ) where U = U as P(ρ) = P(ρ ). Moreover if q ∈ Q c , then q ∈ Q c , therefore q ∈ U implying that U = ∅. Therefore in A FF the BSCC are partitioned in correct ones, in case all q-components of states in C are correct, and faulty ones, in case all U -components of states in C are empty ensuring unambiguity of faulty runs ending in a BSCC. Since runs almost surely leave the transient states and reach a BSCC, this implies that lim n-→∞ P(FAmb n ) = 0.

As a consequence of this characterisation, we establish the equivalence between FFand uniform FF-diagnosability for finite pLTS, claimed in Theorem 3.1.

Proposition 4.3. Let A be a finite pLTS. If A is FF-diagnosable, then it is uniformly FF-diagnosable.
In a finite FF-diagnosable pLTS, thanks to the characterisation given in Proposition 4.2, we know that faults can be detected at worst when the run reaches a BSCC. The proof consists in showing and using that the speed at which a BSCC is reached is uniform from any state.

Proof. Let A be an FF-diagnosable pLTS. Given a run ρ of A, let ρ FF be the corresponding run in A FF : the states in ρ FF extend the states appearing along ρ by subsets of possible correct states after the corresponding prefix of the observed sequence P(ρ). Let S BSCC denotes the set of states of A FF that belong to a BSCC. Last, for every state (q, U ) of A FF and every n ∈ N, denote by SR q,U n the set of signalling runs in A FF of length n starting at (q, U ).

Let α > 0. Our objective is to get n α such that for every n ≥ n α and every minimal faulty run ρ ∈ minF:

P A ({ρ ∈ SR n+|ρ|o | ρ ρ ∧ CorP(P(ρ)) > 0}) ≤ α • P(ρ) .
We first exploit the almost sure convergence towards BSCC in A FF . For every state (q, U ) of A FF , the measure of runs starting in (q, U ) and avoiding all BSCC during n steps tends to 0, when n goes to infinity. Thus, given α, for every reachable (q, U ), there exists n q,U ∈ N such that for every n ≥ n q,U ,

P A FF ({ρ FF ∈ SR q,U n | last(ρ FF ) ∈ S BSCC }) ≤ α.
We define n α as the maximum of n q,U over all states (q, U ). Now let ρ be a minimal faulty run of A, and define (q, U ) = last(ρ FF ). Since

n α ≥ n q,U , P A FF ({ρ FF ∈ SR q,U nα | last(ρ FF ) ∈ S BSCC }) ≤ α.
Therefore, as A and A FF have the same probabilistic behaviour,

P A ({ρ ∈ SR nα+|ρ|o | ρ ρ ∧ last(ρ FF ) ∈ S BSCC }) ≤ α • P(ρ).
Thanks to the characterisation of Proposition 4.2, all states in BSCC reachable from (q, U ) in A FF necessarily are of the form (q , ∅). Therefore, if a finite run ρ FF reaches such a BSCC, ρ FF admits no correct run with same observed sequence, and hence CorP(P(ρ FF )) = 0. Equivalently, CorP(P(ρ )) > 0 implies last(ρ FF ) / ∈ S BSCC . Thus

P A ({ρ ∈ SR nα+|ρ|o | ρ ρ ∧ CorP(P(ρ )) > 0}) ≤ α • P(ρ)
which shows that A is uniformly FF-diagnosable. For FA-diagnosability, we again start from Obs(A). Here, we need information about the ambiguity of both faulty and correct runs. Yet, we still do not need to keep all the information given by Obs(A). Indeed, we can gather the V and W components into a unique set, that we again call V . In other words we keep the information on which faulty states could be reached, but not the distinction between "old" and "new" faulty runs. The resulting simplified automaton is denoted by FA(A). As we did for the FF case, we now define the pLTS A FA = A × FA(A) as the product of A and FA(A) synchronised over observed events (the definition has the same structure as the one of Definition 4.2, only using FA(A) instead of FF(A)). A FA is still a pLTS with same stochastic behaviour as A augmented with the relevant information of the observed sequence w.r.t FA-diagnosability.

1.1.2 FA-diagnosability {q 0 }, ∅ s 0 {q 1 , q 2 }, {f 2 } s 1 ∅, {f 1 , f 2 } s 2 ∅, {f 1 } s 3 ∅, {f 2 } s 4
Example 4.6. Again, FA-diagnosability is characterised through the BSCC of A FA .

q 0 , s 0 q 2 , s 0 q 2 , s 1 f 1 , s 0 f 1 , s 1 f 1 , s 2 f 1 , s 3 f 2 , s 1 f 2 , s 2 f 2 , s 4 q 1 , s 0 q 1 , s 1 u f f
Proposition 4.4. Let A be a finite pLTS. A is FA-diagnosable if and only if A FA has no BSCC that:

• either contains a state (q, U, V ) with q ∈ Q f and U = ∅;

• or contains a state (q, U, V ) with q ∈ Q c and V = ∅.

Note that the characterisation of FA-diagnosability is symmetric for correct states and V -component (resp. faulty states and U -component). This reflects the symmetry of the definition of FA-diagnosability.

The main difference between this proof and the one of Proposition 4.2 is that the second item is not uniform inside a BSCC: there may exists a BSCC containing two states

(q 1 , U 1 , V 1 ) and (q 2 , U 2 , V 2 ) with q 1 , q 2 ∈ Q c , V 1 = ∅ and V 2 = ∅.
As a consequence it is harder to show that if a BSCC verifies the second item, then the pLTS is not FAdiagnosable. Instead of having every extension of the run to be correct ambiguous, the extensions only are ambiguous when they visit some particular states. However, the probability that a run ends in such a state converges towards the steady state distribution of this state which is positive as the state belongs to a BSCC, contradicting the FA-diagnosability.

Proof. To prove the left-to-right implication, we proceed by contraposition. If one assumes the first item holds, the same argument as in the proof of Proposition 4.2 applies. Precisely, suppose that there exists a reachable BSCC C of A FA and a state s = (q, U, V ) in C such that q ∈ Q f and U = ∅. Let ρ be a signalling run leading from the initial state s 0 of A FA to s. Now, for every state s = (q , U , V ) ∈ C, necessarily q ∈ Q f and U = ∅, because C is strongly connected. So for every signalling run ρ that extends ρ, writing s = (q , U , V ) for the state ρ leads to, there exists a correct signalling run ρ such that P(ρ ) = P(ρ ) and q 0 ρ -→ q with q ∈ U . As a consequence the observed sequence P(ρ ) is ambiguous, and for every n ≥ |ρ| o , P(FAmb n ) ≥ P(ρ), so that A is not FA-diagnosable. Suppose now that there exists a reachable BSCC C of A FA and a state s = (q, U, V ) in C such that q ∈ Q c and V = ∅. Since the pair (U, V ) is unchanged by unobservable transitions, w.l.o.g we assume that s is the successor of some state of C by an observable event and we denote C the set of such states. Observe that a signalling run that reaches s is ambiguous. Denote π i (s ) the probability that a random run of length i ends in a state s . In a finite DTMC, for every state s of a reachable BSCC the Cesaro-limit π ∞ (s ) = lim n→∞ 1/(n + 1) n i=0 π i (s ) exists and is greater than 0. For s ∈ C denote by p s ,s the probability of an observable transition from s to s. Then

0 < s ∈C π ∞ (s )p s ,s ≤ lim inf n→∞ 1 n + 1 n i=0 α i (s)
where α i (s) is the probability that a random signalling run of length i ends in s. α i differs from π i by only considering signalling runs. From time 0 to time n, a run can be a signalling run at most n + 1 times. Thus:

1 n + 1 n i=0 α i (s) ≤ 1 n + 1 n i=0 P(CAmb i ) which implies that 0 < lim inf n→∞ 1 n + 1 n i=0 P(CAmb i ) ≤ lim sup n→∞ P(CAmb n ) .
In this case also, we conclude that A is not FA-diagnosable.

The proof of Proposition 4.2 has established that a signalling run reaching a BSCC C where for every state s = (q, U, V ), q is faulty and U = ∅, is surely faulty. Similarly a signalling run that reaches a BSCC where for every state s = (q, U, V ), q is correct and V = ∅, is surely correct. Thus an ambiguous signalling run must only visit transient states. Since runs almost surely leave the transient states and reach a BSCC, this implies that:

lim sup n→∞ P(FAmb n ) + P(CAmb n ) = 0 ,
and therefore, the pLTS is FA-diagnosable.

Let us emphasise that although there does not exist a simple logical characterisation of FA-diagnosability, in finite pLTS, it enjoys a characterisation that is similar to the one of FF-diagnosability.

IA-diagnosability

IA-diagnosability is the notion of exact diagnosability for which we need to use Obs(A) with no simplification. However, to stick to the presentation for the other diagnosability notions, we write here IA(A) for Obs(A). As before, to come up with a characterisation, one builds A IA = A × IA(A), the product of A and IA(A) synchronised over observed events.

Example 4.7. Figure 4.7 shows the synchronised product corresponding to the pLTS depicted in Figure 4.1. Among the BSCC, all the faulty ones ( i.e. the ones reached after a faulty event) have U = ∅, while {(q 1 , s 1 ), (q 1 , s 1 )}, the single one that is reached by a correct run, has a state (q 1 , s 1 ) with W = ∅. We now establish a characterisation of IA-diagnosability on A IA .

Proposition 4.5. Let A be a finite pLTS. A is IA-diagnosable if and only if A IA has no BSCC such that:

• either, all its states (q, U, V, W ) fulfil q ∈ Q f and U = ∅;

• or all its states (q, U, V, W ) fulfil q ∈ Q c and W = ∅.

Proof. This proof relies on Proposition 4.1. An infinite run is not ambiguous if its observation satisfy the Büchi condition of Obs(A), therefore A IA will be IA-diagnosable if the pLTS almost surely satisfies the Büchi condition. As a run almost surely reaches a BSCC, every BSCC must contain a state that allows the diagnosis. Assume first that A IA has a BSCC with (at least) some state (q, U, V, W ) with q ∈ Q f and U = ∅. Using Proposition 4.2, A is not FF-diagnosable and thus not IA-diagnosable either, due to Theorem 3.1. Assume now some BSCC C of A IA has all its states (q, U, V, W ) with q ∈ Q c and W = ∅. In particular none of these states are accepting for the deterministic Büchi automaton IA(A). Let ρ be a finite signalling run that hits C. By Proposition 4.1, any infinite run ρ that extends ρ is ambiguous. From q ∈ Q c we deduce that P(CAmb ∞ ) ≥ P(ρ) > 0. Therefore A is not IA-diagnosable.

Assume now A IA has no BSCC such that either, all its states (q, U, V, W ) fulfil q ∈ Q f and U = ∅, or all its states (q, U, V, W ) fulfil q ∈ Q c and W = ∅. First observe that in case some BSCC of A IA contains some state (q, U, V, W ) with q ∈ Q f and U = ∅, then all its states satisfy the same constraints. Moreover, if some state (q, U, V, W ) of a BSCC has q ∈ Q c , then all states of this BSCC have their first component in Q c . Therefore, the condition can be reformulated as follows: all BSCC C of A IA satisfy:

• either all states (q, U, V, W ) of C fulfil q ∈ Q f and U = ∅;

• or all states (q, U, V, W ) of C fulfil q ∈ Q c and some state (q, U, V, W ) of C fulfils W = ∅.

Whatever the case, all BSCC contain (at least) an accepting state for the Büchi condition of IA(A). Since all runs almost surely end in a BSCC and visit each of its states infinitely often, using Proposition 4.1, almost all runs of A IA are unambiguous. This proves that A is IA-diagnosable.

Surprisingly, while in general FA-diagnosability could not be characterised by a logical formula contrary to IA-diagnosability, restricted to finite systems, the characterisation of IA-diagnosability is the more involved one.

Approximate diagnosis

We now turn to the characterisation of approximate diagnosis and particularly of AFFdiagnosability. The reason why we only consider AFF-diagnosability here will become clear in Subsection 2.2.1 where we show that all other approximate diagnosability notions are undecidable. Our characterisation of AFF-diagnosability relies on the notion of distance between two Markov chains with labels on the transitions. A labelled Markov chain (LMC) is a pLTS where every event is observable: Σ = Σ o . In order to exploit results of [START_REF] Chen | On the total variation distance of labelled Markov chains[END_REF] on LMC in our context of pLTS, we introduce the mapping M that computes in polynomial time the probabilistic closure of a pLTS w.r.t. unobservable events and produces an LMC. Informally, the probabilities of all paths of A from state q to state q with same observed sequence a ∈ Σ o are gathered to obtain the probability in M(A) to move from q to q with label a. The transformation is formally defined below. For sake of simplicity, we denote by A q , the pLTS A where the initial state has been substituted by q.

Definition 4.3. Given a pLTS A = Q, q 0 , Σ, T, P with Σ = Σ o Σ u , the labelled Markov chain M(A) = Q, q 0 , Σ o , T , P is defined by:

• T = {(q, a, q ) | ∃ρ = q • • • aq ∈ SR 1 (A q )} (and so a ∈ Σ o ).
• for every (q, a, q ) ∈ T , P (q, a, q

) = P({ρ ∈ SR 1 (A q ) | ρ = q • • • aq }).
Example 4.8. The LMC associated with the pLTS of Figure 4.1 is represented in Figure 4.8. The transition from q 0 to q 1 which was unobservable has been replaced by a transition labelled by 'a'. The new transition has probability 1/2 which is the product of the probability of the replaced unobservable transition (of value 1/2) and of the transition labelled by 'a' that followed (of value 1). A transition from q 0 to f 2 appeared, it replaces the run q 0 uq 2 f f 1 af 2 which has probability 1/8. Let E be a prospect 3 of Σ ω o (i.e. a measurable subset of Σ ω o for the standard measure), we denote by P M (E) the probability that prospect E occurs in the LMC M. Given two LMC M 1 and M 2 , the (probabilistic) distance between M 1 and M 2 generalises the concept of distance for distributions. Given a prospect E, |P M 1 (E)-P M 2 (E)| expresses the absolute difference between the probabilities that E occurs in M 1 and in M 1 . The distance between M 1 and M 2 is defined as the supremum over the prospects: Definition 4.4. Let M 1 and M 2 be two LMC over the same alphabet Σ o . Then d(M 1 , M 2 ) the distance between M 1 and M 2 is:

q 0 q 2 f 1 f 2 q 1 a, 1/4 b, 1/4 a, 1/2 a, 1/2 a, 1 b, 1 b, 1/2 a, 1/2 b, 1/8 a, 1/8 a, 1/4
d(M 1 , M 2 ) = sup{|P M 1 (E) -P M 2 (E)| | E prospect of Σ ω o } .
Example 4.9. Consider the LMC of Figure 4.8, called M 1 , and the one of Figure 4.9, called M 2 . The prospect E = {a ω } has probability 1/2 in M 1 and 0 in M 2 . Thus d(M 1 , M 2 ) ≥ 1/2. Moreover, M 2 can be obtained from M 1 by deleting the state q 1 (and the associated transitions) and merging q 0 with q 2 . As a consequence, for any prospect E such that a ω ∈ E, P M 2 (E) = 2.P M 1 (E). We thus have

P M 1 ({a ω }) -P M 2 ({a ω }) = 1/2, P M 2 (Σ ω o \ {a ω }) -P M 1 (Σ ω o \ {a ω }) = 1/2
, and for any other prospect E, the difference is smaller or equal to 1/2. Therefore

d(M 1 , M 2 ) = 1/2. q 2 f 1 f 2 b, 1/4 a, 1/2 b, 1 b, 1/2 a, 1/2 a, 1/4
Figure 4.9: An example of LMC.

The distance 1 problem asks, given two LMC M 1 and M 2 , whether d(M 1 , M 2 ) = 1. The next proposition summarises the results of Chen and Kiefer on LMC, that we use later.

Proposition 4.6 ([CK14]).

• Given two LMC M 1 , M 2 , there exists a prospect E such that:

d(M 1 , M 2 ) = P M 1 (E) -P M 2 (E).
• The distance 1 problem for LMC is decidable in polynomial time.

The first item of this proposition states that the supremum is reached (and thus is a maximum). In fact, given two LMC M 1 , M 2 , the authors show that one prospect

reaching the maximum is E = {w ∈ Σ ω o | lim n-→∞ P M 1 (w ≤n ) P M 2 (w ≤n ) ≥ 1}.
We now use the notion of distance 1 to characterise AFF-diagnosability. Let us first consider a subclass of pLTS called initial-fault pLTS. Informally, an initial-fault pLTS A consists of two disjoint pLTS A f and A c and an initial state q 0 with an outgoing unobservable correct transition leading to A c and a transition labelled by f leading to A f (see Figure 4.10). Moreover no faulty transitions occur in A c . In other words, if a fault occurs during a run of an initial-fault pLTS, it does so on the very first transition.

Definition 4.5 (Initial-fault pLTS).

A pLTS A = Q, q 0 , Σ, T, P is an initial fault pLTS if there exist two disjoint pLTS

A f = Q f , q f , Σ, T f , P f and A c = Q c , q c , Σ \ {f }, T c , P c such that: q 0 q c q f f , 1 2 u, 1 2 A c (Σ \ {f }) A f (Σ)
Figure 4.10: Schematic representation of an initial-fault pLTS.

• Q = {q 0 } Q f Q c ; • T = T f T c {(q 0 , u, q c ), (q 0 , f , q f )} with u ∈ Σ u ;
• for every t ∈ T f , we have P(t) = P f (t) and for every t ∈ T c , we have P(t) = P c (t), and P((q 0 , u, q c )) = P((q 0 , f , q f )) = 1/2.

We denote such a pLTS by A = q 0 , A f , A c .

Example 4.10. The pLTS of Figure 4.11 is a simple initial-fault pLTS. Under the initial-fault restriction, we can get a simple characterisation for AFFdiagnosability as established in the next lemma. The idea of this characterisation is then extended to every pLTS.

q 0 q f q c f , 1 2 u, 1 2 a,
Lemma 4.1. Let A = q 0 , A f , A c be an initial-fault pLTS. Then A is AFF-diagnosable if and only if d(M(A f ), M(A c )) = 1.
We write P, P f and P c for the probability measures of pLTS A, A f and A c . By construction of M(A f ) and M(A c ), for every observed sequence σ, P M(A f ) (σ) = P f (σ) and similarly P M(A c ) (σ) = P c (σ). In words, the mapping M leaves unchanged the probability of occurrence of an observed sequence.

AFF-diagnosability is a property of identification (decide whether the run is faulty) of the finite runs that start by a fault with high probability, using the probability P. The distance 1 between M(A f ) and M(A c ) is also a property of identification (decide which LMC the run that produces the infinite observed sequence belongs to) of infinite observed sequences, using the probabilities P f and P c . As mentioned above, these three probability measures are probabilities of parts of the pLTS A and are thus related. The proof therefore mostly consists in understanding the links between the two identification properties: how to translate the set of finite runs identified by AFF-diagnosability into a set of infinite sequences and reciprocally. The intuition to establish the relation is that the prefixes of an infinite faulty run reveals the fault with arbitrarily high accuracy if and only if the associated infinite observed sequence "reveals" that the run belongs to M(A f ).

Proof. Let us prove the equivalence, starting with the left-to-right implication.

• Assume A is AFF-diagnosable. Then, for every ε > 0 and every minimal faulty run ρ:

lim n→∞ P({ρ ∈ SR n+|ρ|o | ρ ρ ∧ CorP(P(ρ )) > ε}) = 0. (4.1)
Pick some 0 < ε < 1. Applying Equation (4.1) on the minimal faulty run ρ f = q 0 f q f with |P(ρ f )| = 0, there exists some n ∈ N such that:

P({ρ ∈ SR n | ρ f ρ ∧ CorP(P(ρ)) > ε}) ≤ ε .
Let S be the set of observed sequences of faulty runs with observable length n and correctness proportion not exceeding threshold ε:

S = {σ ∈ Σ n o | ∃ρ ∈ SR n , P(ρ) = σ ∧ ρ f ρ ∧ CorP(σ) ≤ ε} .
We define E = Cyl(S) to be the prospect consisting of the infinite suffixes of these sequences. Let us show that P c (E) ≤ ε/(1 -ε) and P f (E) ≥ 1 -2ε. We have:

P f (E) = 1 -2 P({ρ ∈ SR n | ρ f ρ ∧ CorP(P(ρ)) > ε}) ≥ 1 -2ε
where the factor 2 comes from the probability 1/2 in A to enter A f that P f does not take into account contrary to P. Moreover, for every observed sequence σ ∈ S, CorP(σ) ≤ ε. Using the definition of CorP:

CorP(σ) = P({ρ ∈ C ∩ SR n | P(ρ) = σ}) P({ρ ∈ SR n | P(ρ) = σ}) = P c (σ) P c (σ) + P f (σ) ≤ ε. Thus, P c (σ) ≤ ε 1-ε P f (σ).
Hence:

P c (E) = σ∈S P c (σ) ≤ σ∈S ε 1 -ε P f (σ) = ε 1 -ε P f (E) ≤ ε 1 -ε . Therefore d(M(A c ), M(A f )) ≥ P f (E) -P c (E) ≥ 1 -ε(2 + 1 1-ε ).
Since ε was arbitrary, taking the limit when ε goes to 0, we obtain the desired result: d(M(A c ), M(A f )) = 1. Note that we did not exhibit the prospect that reaches the maximum but only a prospect ε-close to it. The proof could be modified to use this maximum prospect, but it makes the proof unnecessarily more complicated.

• Conversely assume that d(M(A f ), M(A c )) = 1. Thanks to Proposition 4.6, there exists a prospect E ⊆ Σ ω o such that P f (E) = 1 and P c (E) = 0. For every n ∈ N, let S n be the set of prefixes of length n of the observed sequences of E:

S n = {σ ∈ Σ n o | ∃σ ∈ E, σ σ }.
For every ε > 0, we also define S ε n as the subset of S n consisting of sequences whose correctness proportion exceeds threshold ε:

S ε n = {σ ∈ S n | CorP(σ) > ε}.
From n∈N Cyl(S n ) = E, we derive that lim n→∞ P c (S n ) = P c (E) = 0. Thus lim n→∞ P c (S ε n ) = 0. On the other hand, for every n ∈ N,

P c (S ε n ) = σ∈S ε n P c (σ) > σ∈S ε n ε 1 -ε P f (σ) = ε 1 -ε P f (S ε n ) .
Since ε is fixed, we have

P f (S ε n ) < 1-ε ε P c (S ε n ), thus lim n→∞ P c (S ε n ) = 0 implies that lim n→∞ P f (S ε n ) = 0.
Let ρ be a minimal faulty run and α > 0. There exists n α ≥ |ρ| o = 1 such that for all n ≥ n α , P f (S ε n ) ≤ α. Let n ≥ n α , and S n be the set of observed sequences of length n triggered by a run with prefix ρ and whose correctness proportion exceeds ε:

S n = {σ ∈ Σ n o | ∃ρ ∈ SR n , ρ ρ ∧ P(ρ ) = σ ∧ CorP(σ) > ε} .
Let us prove that P( S n ) ≤ α. On the one hand, since

P f (S n ) ≥ P f (E) = 1, P f ( S n ∩ (Σ n o \ S n )) = 0. On the other hand, since P f (S ε n ) ≤ α, P f ( S n ∩ S n ) ≤ P f (S ε n ) ≤ α. Thus, P f ( S n ) = P f ( S n ∩S n )+P f ( S n ∩(Σ n o \S n )) ≤ α.
Because α was taken arbitrary, we obtain that lim n→∞ P f ( S n ) = 0. Observe now that

P({ρ ∈ SR n | ρ ρ ∧ CorP(P(ρ )) > ε}) = 1 2 P f ( S n ). Therefore, lim n→∞ P({ρ ∈ SR n | ρ ρ ∧ CorP(P(ρ )) > ε}) = 0. In conclusion A is AFF-diagnosable.
This characterisation shows that, for initial-fault pLTS, AFF-diagnosability can be reduced to the distance 1 problem. As one can perform the closure w.r.t. unobservable events and check the distance 1 in polynomial time, AFF-diagnosability for initial-fault pLTS belongs to PTIME.

Example 4.11. Consider the initial-fault pLTS of Figure 4.11 q 0 , A f , A c and the prospect

E = {σ ∈ Σ ω o | lim sup n∈N |σ ≤n | b n ≥ 1 2 }.
As a 'b' has a probability 3/4 to be observed at each step in A f and 1/4 in A c , P f (E) = 1 and P c (E) = 0 where P f and P c are the probability measures of A f and A c . Therefore this initial-fault pLTS is AFFdiagnosable. In fact, as this pLTS has a single minimal faulty run, it is even uniformly AFF-diagnosable.

Remark 4.1. There exists a single minimal faulty run in every initial-fault pLTS. As a consequence, AFF-diagnosability and uniform AFF-diagnosability are equivalent for initial-fault pLTS.

In order to understand why characterising AFF-diagnosability for general pLTS is more involved, consider the pLTS A presented in Figure 4.12. Recall that A is AFFdiagnosable as shown in the proof of Theorem 3.1, page 73.

q 0 q 1 q f q c u, 1 2 f , 13 16 u, 1 2 a, 3 16 b, 1 a, 3 4 b, 1 4 Figure 4
.12: An AFF-diagnosable pLTS where the distance 1 characterisation cannot be applied in a simple way.

Let us look at the distance between pairs of a correct and a faulty states of A that can be reached by runs with the same observed sequence. On the one hand, we have d(M(A q 1 ), M(A q f )) ≤ 3/16 since for any prospect E either (1) b ω ∈ E implying

P M(Aq f ) (E) = 1 and P M(Aq 0 ) (E) ≥ 13/16 or (2) b ω / ∈ E implying P M(Aq f ) (E) = 0 and P M(Aq 0 ) (E) ≤ 3/16. On the other hand, d(M(A qc ), M(A q f )) = 1 since P M(Aq f ) (b ω ) = 1 and P M(Aq c ) (b ω ) = 0.
Intuitively, the pair (q 1 , q f ) is irrelevant, since the correct state q 1 does not belong to a BSCC of the pLTS, while (q c , q f ) is relevant since q c belongs to a BSCC triggering a "recurrent" ambiguity. The next theorem characterises AFF-diagnosability for general pLTS, establishing the soundness of this intuition.

Theorem 4.1. Let A be a pLTS. Then, A is AFF-diagnosable if and only if for every correct state q c belonging to a BSCC and every faulty state q f reachable by a faulty run ρ f such that q c is reachable by a run with same observed sequence, d(M(A qc ), M(A q f )) = 1.

The proof of Theorem 4.1, due to its complexity and length, is divided into two lemmas, Lemma 4.2 and Lemma 4.3 given below, each of them stating one implication of the equivalence. Lemma 4.2. Let A be a pLTS. If there exists q c ∈ Q c belonging to a BSCC,

q f ∈ Q f such that d(M(A q f ), M(A qc )) < 1 and runs q 0 ρc = ⇒ q c and q 0 ρ f = ⇒ q f such that P(ρ c ) = P(ρ f ), then A is not AFF-diagnosable.
This lemma is the easiest of the two. It is proved by contraposition. Assume there exist two states in A, q c ∈ Q c belonging to a BSCC and q f ∈ Q f reachable resp. by ρ c and ρ f with P(ρ c ) = P(ρ f ), and with d(M(A qc ), M(A q f )) < 1. Applying Lemma 4.1 to the initial-fault pLTS A = q 0 , A q f , A qc where q 0 is a new state, one deduces that A is not AFF-diagnosable. First we relate the probabilities of runs in A and A . Then we show that considering the additional faulty runs with same observed sequence as ρ f does not make A AFF-diagnosable.

Proof. Let A be a pLTS, assume there exists q c ∈ Q c belonging to a BSCC, q f ∈ Q f such that d(M(A q f ), M(A qc )) < 1 and runs q 0 ρc = ⇒ q c and q 0 ρ f = ⇒ q f such that P(ρ c ) = P(ρ f ). Let us introduce some notations:

σ 0 := P(ρ f ) = P(ρ c ), p f := P A (ρ f ), p c := P A (ρ c ) .
Let p g (≥ p f ) be the probability of the faulty runs with observed sequence σ 0 :

p g = P A ({ρ ∈ SR |σ| | P(ρ) = σ 0 , and ρ is faulty}) .
For all n ≥ |σ|, let S n be the set of observed sequences of length n "extending" ρ f :

S n = {σ ∈ Σ n o | ∃ρ ∈ SR n , ρ f ρ ∧ P(ρ) = σ} .
Given σ ∈ S n , we refine p f , p c and p g as follows.

•

p σ f = P A ({ρ ∈ SR n | ρ f ρ ∧ P(ρ) = σ}); • p σ c = P A ({ρ ∈ SR n | ρ c ρ ∧ P(ρ) = σ}); • p σ g = P A ({ρ ∈ SR n | ρ is faulty and P(ρ) = σ}).
We introduce the initial-fault pLTS A = q 0 , A q f , A qc for some new state q 0 . It is well-defined since q c belongs to a BSCC so that A qc does not trigger faults. We write P for the probability measure in A . Since d(M(A q f ), M(A qc )) < 1, due to Lemma 4.1, there exist positive reals α , ε ≤ 1 such that for all n 0 ∈ N there exists n ≥ n 0 :

P A {ρ ∈ SR n | q 0 f q f ρ ∧ CorP(P(ρ)) > ε} > α .
This entails the following inequality for A:

P({ρ ∈ SR n | ρ f ρ ∧ p P(ρ) c p P(ρ) c + pc p f p P(ρ) f > ε }) > 2p f α .
Indeed in A , the probability of the set of faulty (resp. correct) run with observed sequence P(ρ) is

p P(ρ) f 2p f (resp. p P(ρ) c 2pc
): the probability in A to go in q f (resp. q c ) initially, 1/2, times the probability in A of the runs extending ρ f (resp. ρ c ) with observation P(ρ), p

P(ρ) f (resp. p P(ρ) c
), divided by the probability of ρ f (resp. ρ c ), p f (resp. p c ). Finally the 2p f factor of the lower bound takes into account the fact that the probability of reaching q f in A is 1/2 while the probability of ρ in A is p f . Observe that

p P(ρ) c p P(ρ) c + pc p f p P(ρ) f > ε is equivalent to p P(ρ) c p P(ρ) c +p P(ρ) f > ε pc ε pc+(1-ε )p f . So defining ε = ε pc ε pc+(1-ε )p f ≤ 1 and α = 2p f α ≤ 2,
the previous inequality can be rewritten:

P({ρ ∈ SR n | ρ f ρ ∧ p P(ρ) c p P(ρ) c + p P(ρ) f > ε}) > α .
Let S n be the subset of observed sequences of S n whose correctness proportion is greater than ε when only considering extensions of ρ f , but smaller than ε * = αε 4 when considering all faulty runs:

S n = {σ ∈ S n | p σ c p σ c + p σ f > ε ∧ p σ c p σ c + p σ g ≤ ε * }. Let σ ∈ S n , p σ f < 1-ε ε p σ c and p σ c ≤ ε * 1-ε * p σ g . Therefore p σ f < (1-ε)ε * (1-ε * )ε p σ g . Summing over all sequences of S n : σ∈S n p σ f < (1-ε)ε * (1-ε * )ε p g . Since p g ≤ 1: σ∈S n p σ f ≤ (1-ε) α 4(1-αε 4 ) ≤ α 2 . Thus, P({ρ ∈ SR n | ρ f ρ ∧ p P(ρ) c p P(ρ) c + p P(ρ) g > ε * }) ≥ P({ρ ∈ SR n | ρ f ρ ∧ p P(ρ) c p P(ρ) c + p P(ρ) f > ε}) - σ ∈S n p σ f > α - α 2 = α 2 . Observe that given σ ∈ S n , CorP(σ) ≥ p σ c p σ c +p σ g
, since we ignore correct runs ρ with P(ρ) = σ that do no extend ρ c . So defining ε = ε * and α = α/2, for all n 0 ∈ N there exists n ≥ n 0 :

P({ρ ∈ SR n | ρ f ρ ∧ p P(ρ) c p P(ρ) c + p P(ρ) g > ε}) > α .
Let ρ 0 be the minimal faulty run such that ρ 0 ρ f . We observe that Cyl(ρ f ) ⊆ Cyl(ρ 0 ), so that

P({ρ ∈ SR n | ρ 0 ρ ∧ p P(ρ) c p P(ρ) c + p P(ρ) g > ε}) > α
which establishes that A is not AFF-diagnosable.

Lemma 4.3. Let A be a pLTS. If for all q 0 ρc = ⇒ q c and q 0

ρ f = ⇒ q f with P(ρ c ) = P(ρ f ), q f ∈ Q f and q c ∈ Q c belonging to a BSCC, d(M(A qc ), M(A q f )) = 1, then A is AFF- diagnosable.
Let ρ 0 be a minimal faulty run, α > 0, ε > 0, σ 0 = P(ρ 0 ) and n 0 = |σ 0 |. Before developing the proof, we sketch its structure and illustrate it in Figure 4.13. First, we extend the runs with observed sequence σ 0 by n b observable events where n b is chosen in order to get a high probability that the runs end in a BSCC. Let σ ∈ Σ n b o be such an observed sequence. We partition the possible runs with observed sequence σ 0 σ into three sets R F σ , R C σ and R T σ . R F σ is the subset of faulty runs while R C σ (resp. R T σ ) is the set of correct runs ending (resp. not ending) in a BSCC. At first, we do not take into account the transient runs in R T σ . We apply Lemma 4.1 to obtain an integer n σ such that from R F σ and R C σ , we can diagnose with (appropriate) high probability and low correctness proportion after n σ observations. Among the runs that trigger diagnosable observed sequences, some will exceed the correctness proportion, ε, when taking into account the runs from R T

σ . Yet we show that the probability of such runs is small when cumulated over all extensions σ leading to the required upper bound α.

q 0 . . . R C σ R T σ R F σ i n i t i
Let S = {σ ∈ Σ n b o | ∃ρ ∈ SR n 0 +n b ρ 0 ρ ∧ P(ρ) = σ 0 σ}.
Pick some σ ∈ S and define:

• R F σ = {ρ ∈ SR n 0 +n b | P(ρ) = σ 0 σ ∧ last(ρ) ∈ Q f }; • R C σ = {ρ ∈ SR n 0 +n b | P(ρ) = σ 0 σ ∧ last(ρ) ∈ Q c and belongs to a BSCC}; • R T σ = {ρ ∈ SR n 0 +n b | P(ρ) = σ 0 σ∧last(ρ) ∈ Q c and does not belong to a BSCC}.
Temporarily, we ignore the runs of

R T σ . Let Q σ c = {last(ρ) | ρ ∈ R C σ } and Q σ f = {last(ρ) | ρ ∈ R F σ }. For every pair (q f , q c ) ∈ Q σ f × Q σ c
, consider the initial-fault pLTS A = q 0 , A q f , A qc for some new state q 0 , and denote P its associated probability measure. Due to Lemma 4.1, for all α > 0, ε > 0, there exists n q f ,qc such that for all n ≥ n q f ,qc :

P {ρ ∈ SR n | q 0 f q f ρ ∧ p P(ρ) c p P(ρ) c + p P(ρ) f > ε } ≤ α where p P(ρ) c (resp. p P(ρ) f
) is the probability in A of a correct (resp. faulty) run with observed sequence P(ρ). and p

P(ρ) f ≤ 2 p f p σ 0 σP(ρ) f
. Thus for all n ≥ n 0 + n b + max(n q f ,qc ):

P{ρ ∈ SR n | ∃ρ ∈ R F σ ∧ ρ ρ ∧ p P(ρ) c p P(ρ) c + pc p f p P(ρ) f > ε } ≤ 2α
where the factor 2 takes into account the first transition in A .

Choosing ε = εp f εp f +(2-ε)pc and α = α 4|S| , after algebraic operations the previous inequality can be rewritten:

P{ρ ∈ SR n | ∃ρ ∈ R F σ ∧ ρ ρ ∧ p P(ρ) c p P(ρ) c + p P(ρ) f > ε 2 } ≤ α 2|S|
.

Let

n σ = n 0 + n b + max(n q f ,qc | (q f , q c ) ∈ Q σ f × Q σ c )
and n 1 = max(n σ | σ ∈ S) and consider n ≥ n 1 . Ignoring the runs of R T σ , one could detect the fault done in ρ 0 with good accuracy and high probability n 1 steps after it occurred.

We now take into account the runs of

R T σ . Let ρ ∈ {ρ ∈ SR n | ∃ρ ∈ R F σ ∧ ρ ρ}. Define p P(ρ) t
to be the probability of runs (1) with observed sequence P(ρ) and (2) extending runs of R T σ . Since a correct run with observed sequence P(ρ) must have a prefix in R T σ or in R C σ :

CorP(P(ρ)) ≤ p P(ρ) c + p P(ρ) t p P(ρ) c + p P(ρ) t + p P(ρ) f .
Consider the following set of runs:

R n σ = {ρ ∈ SR n | ∃ρ ∈ R F σ ∧ ρ ρ ∧ p P(ρ) c + p P(ρ) t p P(ρ) c + p P(ρ) t + p P(ρ) f > ε ∧ p P(ρ) c p P(ρ) t + p P(ρ) f ≤ ε 2 } .
For ρ ∈ R n σ , one gets by algebraic operations,

2p P(ρ) t ε > p P(ρ) f . Thus P( R n σ ) < 2P(R T σ ) ε and σ∈S P( R n σ ) < 2 σ∈S P(R T σ ) ε .
Due to the choice of n b , σ∈S P(R T σ ) < η, and we derive σ∈S P(

R n σ ) < 2η ε = α 2 .
Summarising for all n ≥ n 1 :

P{ρ ∈ SR n | ρ 0 ρ ∧ CorP(P(ρ)) > ε} = σ∈S P{ρ ∈ SR n | ρ 0 ρ ∧ σ 0 σ P(ρ) ∧ CorP(P(ρ)) > ε} ≤ σ∈S P{ρ ∈ SR n | ∃ρ ∈ R F σ ∧ ρ ρ ∧ p P(ρ) c p P(ρ) c + p P(ρ) f > ε 2 } +P{ρ ∈ SR n | ∃ρ ∈ R F σ ∧ ρ ρ ∧ p P(ρ) c p P(ρ) c + p P(ρ) f ≤ ε 2 ∧ p P(ρ) c + p P(ρ) t p P(ρ) c + p P(ρ) t + p P(ρ) f > ε} ≤|S| α 2|S| + α 2 = α
which establishes the AFF-diagnosability of A.

As an alternative to the proof of Theorem 4.1, one could mimic the approach by Kiefer and Sistla [START_REF] Kiefer | Distinguishing hidden Markov chains[END_REF] for monitorability. The idea would be, from a pLTS A to derive two hidden Markov chains, say H c and H f representing respectively the observed sequences for correct and faulty runs of A. However, to establish that distinguishability of H c and H f corresponds to AFF-diagnosability essentially relies on the same arguments we used in the above proof (and so this alternative approach would not simplify it). The difficulty lies in that the properties one conditions by to obtain H c and H f , namely always correct or eventually faulty, anticipate on the future behaviour of the system; in contrast, the correctness proportion appearing in the definition of AFF-diagnosability only reasons about the possible behaviours up to the last observation.

Verification of the diagnosability

We now study the decidability of the different diagnosability notions for finite pLTS and in the positive case provide the complexity. The characterisations given in Section 1 play an important role in this study. Indeed, when a simple characterisation exists, the diagnosability problem is decidable (an algorithm consists in checking this characterisation). Conversely, when we did not exhibit a characterisation, we show that the problem is undecidable.

Decidability results and upper bounds

We start by showing how to check the characterisations defined in Section 1, therefore providing upper bounds to some of the diagnosability problems. We first consider exact diagnosability notions, and establish that they can all be solved in PSPACE. In all cases, to obtain the PSPACE upper-bound, we avoid building explicitly the exponential size product pLTS (that is used in the characterisations) and only explore it on-the-fly.

The three results have similar proofs. As a consequence we first develop the case of FF-diagnosability then we simultaneously deal with both FA and IA-diagnosability.

Proposition 4.7. The FF-diagnosability problem is decidable in PSPACE.

The proof consists in designing a PSPACE algorithm to check the characterisation given in Proposition 4.2. This algorithm exploits Savitch's Theorem [START_REF] Savitch | Relationships between nondeterministic and deterministic tape complexities[END_REF] which establishes that PSPACE = NPSPACE. This theorem allow us to use non-determinism in our decision procedure.

Proof. To obtain a PSPACE algorithm, we cannot build explicitly the product pLTS A FF , which is exponential in the size of A. Given two states s, s of A FF , one can check in polynomial space in the size of A whether s can be reached from s. Indeed, reachability of a state is known to be in non-deterministic logarithmic space in the size of the system, thus here NPSPACE, which is equal to PSPACE thanks to Savitch's Theorem. Using this procedure, we can check whether a state s is not in a BSCC by guessing another state s such that s is reachable from s but s is not reachable from s . Here again we apply Savitch's Theorem. Thus the procedure that decides whether A is not FF-diagnosable consists in guessing a state s = (q, U ) with q ∈ Q f and U = ∅, checking that it is reachable from s 0 and whether s belongs to a BSCC.

We state below similar results for FA and IA-diagnosability problems.

Proposition 4.8. The FAand IA-diagnosability problems are decidable in PSPACE.

The two proofs are similar to the proof of Proposition 4.7: we design a decision procedure which uses non-determinism to check the characterisation given in the previous section and the non-determinism is removed using Savitch's Theorem.

Proof. We first check the characterisation of FA-diagnosability given in Proposition 4.4 without explicitly building the product pLTS A FA . First given a state (q, U, V ) of A FA we can check in polynomial space whether it belongs to a BSCC (as in the proof of Proposition 4.7). We can also check in polynomial space whether some state (q , U , V ) with U = ∅ or V = ∅ can be reached from (q, U, V ) by guessing such a state and then checking the reachability condition. Combining the two, this provides a polynomial space algorithm to check whether (q, U, V ) belongs to a BSCC in which no state (q , U , V ) fulfils U = ∅ and V = ∅. Thus the procedure that decides whether A is not FA-diagnosable consists in guessing a state s = (q, U, V ), checking that it is reachable from s 0 and belongs to a BSCC where all states (q , U , V ) fulfil U = ∅ and V = ∅.

We use the characterisation of IA-diagnosability given in Proposition 4.5 without building explicitly the product pLTS A IA . First, given a state (q, U, V, W ) of A IA , we can check in polynomial space that it belongs to a BSCC (as in the proof of Proposition 4.7). We can also check in polynomial space whether it is coreachable from a state (q , U , V , W ) that fulfils U = ∅ or W = ∅ by guessing such a state. Combining the two procedures, we can check in polynomial space whether (q, U, V, W ) belongs to a BSCC where all states (q , U , V , W ) of the BSCC fulfil U = ∅ and W = ∅. Thus the procedure that decides whether A is not IA-diagnosable consists in guessing a state s = (q, U, V, W ), checking that it is reachable from the initial state s 0 and belongs to a BSCC where all states (q , U , V , W ) fulfil U = ∅ and W = ∅.

For approximate diagnosability, we focus on AFF-diagnosability and establish a complexity upper-bound, relying on the characterisation from the previous section.

Theorem 4.2. The AFF-diagnosability problem is decidable in PTIME for pLTS.

Proof. The decidability and complexity results rely on the characterisation of AFFdiagnosability showed in Theorem 4.1. Reachability of a pair of states with the same observed sequence is decidable in NLOGSPACE by an appropriate "self-synchronised product" of the pLTS that we detail below. Since there are at most a quadratic number of pairs to check, and given that the distance 1 problem can be decided in polynomial time due to [START_REF] Chen | On the total variation distance of labelled Markov chains[END_REF] (as recalled in Proposition 4.6), the decidability and PTIME upperbound follow.

We now define the appropriate "self-synchronised product" of the pLTS mentioned above. Given a pLTS A = Q, q 0 , Σ, T, P we build the product LTS A ⊗ A = Q × Q, {q 0 , q 0 }, Σ, T where ((q 1 , q 2 ), a, (q 1 , q 2 )) ∈ T if

• if a ∈ Σ u then either (q 1 , a, q 1 ) ∈ T and q 2 = q 2 or (q 2 , a, q 2 ) ∈ T and q 1 = q 1 ;

• else (a ∈ Σ o ), (q 1 , a, q 1 ) ∈ T and (q 2 , a, q 2 ) ∈ T .

A pair of state (q, q ) is reachable in A ⊗ A from (q 0 , q 0 ) if and only if there exist two runs ρ and ρ of A such that last(ρ) = q, last(ρ ) = q and P(ρ) = P(ρ ). Therefore, A is AFF-diagnosable if for any pair of states reachable in A ⊗ A, (q, q ), with q correct, belonging to a BSCC of A, and q faulty, then d(M(A q ), M(A q )) = 1. All tests can be checked in PTIME.

We thus have decidability of every notions of exact diagnosability and of one notion of approximate diagnosability, AFF-diagnosability. Surprisingly, AFF-diagnosability, which definition seems more complicated as it depends on the exact probabilistic values of the transitions contrary to the definitions of the exact notions of diagnosability, has a lower upper bound.

Hardness of Diagnosability

We gave upper bounds on the complexity of diagnosability in Subsection 2.1. We now provide tight lower bounds: on the one hand we establish undecidability of the approximate diagnosability notions that were not characterised, and on the other hand we provide a PSPACE lower bound for the exact diagnosis.

Undecidability results

After having previously proved that AFF-diagnosability can be solved in polynomial time, we now establish that all other specifications of approximate diagnosability are undecidable. This result could be expected for εFF-diagnosability and uniform εFFdiagnosability since it is often the case for problems mixing probabilities, partial ob-servation and quantitative requirement (here represented by ε)4 . On the contrary, the undecidability of the uniform AFF-diagnosability problem is at first sight surprising since it is a slight variation of the AFF-diagnosability problem. In fact the reduction for the latter problem is more intricate than the one for the εFF-and uniform εFFdiagnosability. We reduce the emptiness problem for probabilistic automata [START_REF] Paz | Introduction to probabilistic automata[END_REF] to both problems. Let us first details this problem. A probabilistic automaton is an automaton enhanced with probabilities on the transitions so that given a state and a letter, the sum of the probabilities of the transition exiting each state and labelled by the given letter is 1.

Example 4.12. Figure 4.14 represents a probabilistic automaton which initial state is q 0 and set of accepting states is {q 2 }. The sum of the probability of the transitions exiting q 0 is 2 as there is a transition labelled by a 'a' and a transition labelled by a 'b'. The word bab has probability 1/2 to end in q 2 and 1/2 to end in q 1 . Definition 4.6. A probabilistic automaton is a tuple A = Q, δ 0 , Σ, (P a ) a∈Σ , F where

q 0 q 1 q 2 q 3 b, 1 a, 1/2 a, 1/2 a, 1 a, 1/2 b, 1 a, 1/2
• Q is a set of states with F ⊆ Q, the set of final states;

• δ 0 ∈ Dist(Q) is the initial distribution;
• Σ is an alphabet;

• For every a ∈ Σ, P a is a stochastic Q × Q matrix.

When P a [q, q ] > 0, there is a transition from q to q labelled by a and P a [q, q ]. Given a word w = a 1 . . . a n ∈ Σ * , the acceptance probability of w, P A (w) is defined by P A (w) = q 0 ∈Q δ 0 (q 0 ) q∈F P w [q 0 , q] where P w = P a 1 • • • P an . Given a rational threshold 0 < ε < 1, the language L A,ε is defined by L A,ε = {w ∈ Σ * | P A (w) > ε}. For a probabilistic automaton A and a threshold ε, the emptiness problem asks whether L A,ε = ∅. This problem is undecidable even for a fixed 0 < ε < 1 [START_REF] Paz | Introduction to probabilistic automata[END_REF]. The problem is also undecidable for the language defined by {w ∈ Σ * | P A (w) ≥ ε}, i.e. when the inequality is not strict. One can also restrict oneself to automata such that every word w satisfies 1/4 ≤ P A (w) ≤ 3/4. This can be ensured by a simple construction. Given a probabilistic automaton A with initial distribution δ 0 and a threshold 0 < λ < 1, we build the probabilistic automaton A that contains the states of A and two additional states q a and q r . The new initial distribution goes in q a and q r with probability 1/4 and with probability 1/2 it uses the initial distribution of A. In the last case, the behaviour is the one of A, from q r and q a everything can be observed with a self-loop and q a is a final state. For every word w, we have

P A (w) = P A (w) 2 + 1 4 .
This is the sum of the probability to be accepted after starting initially in the A component of A plus the probability to go in q a . As a consequence, every word is accepted in A with probability between 1/4 and 3/4 and a word is above the λ threshold in A iff it is above 1/4 + λ/2 in A . Thus the emptiness is also undecidable with this restriction.

Note that this assumption relies on the use of an initial distribution δ 0 instead of an initial state q 0 . Indeed, with an initial state, the word ε would have probability 0 or 1. When this assumption is not needed, we use an initial state instead of an initial distribution. Another important undecidable problem that is used in a latter chapter is the value 1 problem. It asks for a probabilistic automaton A if for all ε > 0 L A,1-ε = ∅.

In other words, does there exists words of arbitrarily high probability? This problem is known to be undecidable [START_REF] Gimbert | Probabilistic automata on finite words: Decidable and undecidable problems[END_REF].

Theorem 4.3. For any rational 0 < ε < 1, the εFF-diagnosability and uniform εFFdiagnosability problems are undecidable for pLTS.

We make a reduction from the emptiness problem of probabilistic automata. From a given probabilistic automaton A = Q, δ 0 , Σ, (P a ) a∈Σ , F , we build a pLTS which produces with probability 1 runs whose observed sequence belongs to Σ * ω (where ∈ Σ) and for all n ≥ 2, the correctness proportion CorP of w n , with w ∈ Σ * , satisfies CorP(w n ) = P A (w). In other words, if a word w is accepted with probability greater than ε, then the ambiguity of the word w 2 is greater than ε and every faulty run with this observation will remain ambiguous.

Proof. Let A = Q, δ 0 , Σ, (P a ) a∈Σ , F be a probabilistic automaton. W.l.o.g we assume 1/4 ≤ P A (w) ≤ 3/4 for every w ∈ Σ * . Define the pLTS A = Q , q 0 , Σ , T , P as follows:

• Σ = Σ { , f , u}, Σ u = {f , u}; • Q = Q ∪ {q 0 , q c , q f , f }; • T = {(q 0 , u, q) | q ∈ Q, δ 0 (q) > 0} ∪ {(q, a, q) | q, q ∈ Q, a ∈ Σ, P a [q, q ] > 0} ∪ {(q, , q c | q ∈ F } ∪ {(q, , q f | q ∈ Q \ F } ∪ {q c , , q c } ∪ {q f , f , f } ∪ {f , , f }
• P is defined by:

-For all q ∈ Q such that δ 0 (q) > 0, P (q 0 , u, q) = δ 0 (q); -For all q ∈ Q and a ∈ Σ, P (q, a, q ) = Pa[q,q ] 1+|Σ| ; -For all q ∈ F , P (q, , q c ) = 1 1+|Σ| ; -For all q ∈ Q \ F , P (q, , q f ) = 1 1+|Σ| ;

-P (q f , f , f ) = P (f , , f ) = P (q c , , q c ) = 1.
This reduction is illustrated in Figure 4.15. In each state, the sum of the probabilities of the exiting transitions correctly sum to 1. For instance, let q ∈ F and a ∈ Σ, then q ∈Q P a [q, q ] = 1, thus:

(q,a,q )∈T P (q, a, q

) = a∈Σ q ∈Q P a [q, q ] 1 + |Σ| + P (q, , q c ) = |Σ| 1 + |Σ| + 1 1 + |Σ| = 1. q 0 q 1 A q q q f q c f u, δ 0 (q 1 ) a, Pa[q 0 ,q] 1+|Σ| f , 1 , 1 1+|Σ| , 1 1+|Σ| , 1 , 1
Figure 4.15: From probabilistic automata to pLTS.

We claim that the following three assertions are equivalent:

1. A is εFF-diagnosable; 2. A is uniformly εFF-diagnosable; 3. L A,ε = ∅.
Given that uniform εFF-diagnosability entails εFF-diagnosability, we only show that item 1 implies item 3, and item 3 implies item 2. The first implication is proved by contraposition.

1 implies 3 Assume that there exists a word w ∈ Σ * such that P A (w) > ε. Consider the set of signalling correct runs with observed sequence w n+2 . By construction, its probability is P A (w) (1+|Σ|) |w|+1 . Similarly, the set of signalling faulty runs with observed sequence w n+2 has probability 1-P A (w) (1+|Σ|) |w|+1 . Thus CorP(w n+2 ) = P A (w) > ε. By assumption on A, P A (w) ≤ 3/4 < 1, so that the set of faulty runs with observed sequence w n+2 is non-empty. Pick ρ a minimal faulty run with observed sequence w . Using the above probability values, for every n ≥ 0:

P({ρ ∈ SR n+|ρ|o | ρ ρ ∧ CorP(P(ρ )) > ε}) = P(ρ) .
Thus A is not εFF-diagnosable.

3 implies 2 Assume that for every word w ∈ Σ * , Pr A (w) ≤ ε. Let ρ be a minimal faulty run of A. By construction, its observed sequence is of the form w 2 with w ∈ Σ * . Using the same reasoning as above, for every ρ ρ : CorP(P(ρ )) = Pr A (w), and thus CorP(P(ρ )) ≤ ε. Therefore, for any α > 0, choosing n α = 0, one gets:

P({ρ ∈ SR nα+|ρ| | ρ ρ ∧ CorP(P(ρ )) > ε}) = 0 . So A is uniformly εFF-diagnosable.
This completes the proof that εFF-diagnosability and uniform εFF-diagnosability are undecidable.

Uniform AFF-diagnosability is also shown to be undecidable by a reduction from the emptiness problem for probabilistic automata.

Theorem 4.4. The uniform AFF-diagnosability problem is undecidable for pLTS.

As this reduction is more involved, we start by a developed sketch of proof and then give the full proof. We proceed by a reduction from the emptiness problem for probabilistic automata where w.l.o.g. one assumes that the acceptance probability of any word lies between 1/4 and 3/4. Given such a probabilistic automaton one builds a pLTS as follows.

• With probability 1/2 one enters one of the two copies of the automaton whose probabilities are modified in a similar way as in the previous proof.

• In a non-final (resp. final) state of the first (resp. second) copy, one may exit the copy of the automaton by taking a transition labelled by (resp. f ) and enter a terminating state. In a final state (resp. non-final) state of the first (resp. second) copy, one may "restart" the copy of the automaton by taking a transition labelled by which lead to the initial state of the copy.

• The terminating state of the first copy iteratively outputs with probability 1/2 or while the terminating block of the second copy endlessly outputs .

Due to the behaviour of the terminating blocks, the correctness proportion of a faulty run goes to 0 as its length increases. Thus the pLTS is AFF-diagnosable. The element that will depend on the probabilistic automaton is the uniformity of the convergence.

Observe that the language of the observed sequences of minimal faulty runs extended by one transition is (Σ * ) * Σ * .

Assume there exists a word w with acceptance probability strictly greater than 1/2. Then in A, the correctness proportion of (w ) n fulfils: lim n→∞ CorP((w ) n ) = 1. Due to this property (and the behaviours of the terminating blocks), the pLTS is not uniformly AFF-diagnosable. If no such word exists, then for any w = w 1 w 2 . . . w k , CorP(w) ≤ 3/4. Due to this property (and the behaviours of the terminating blocks), the pLTS is uniformly AFF-diagnosable.

We now develop the full proof.

Proof. Let A = Q, δ 0 , Σ, (P a ) a∈Σ , F be a probabilistic automaton such that the acceptance probability of any word lies between 1/4 and 3/4. Define the pLTS A = Q , q 0 , Σ , T , P as follows.

• Σ = Σ { , , u, f }, Σ uo = {u, f }; • Q = {q u , q f | q ∈ Q} ∪ {q 0 , b u , b f }; • T = {(q 0 , u, q u ) | δ 0 (q) > 0} ∪ {(q 0 , u, q f ) | δ 0 (q) > 0} ∪ {(q u , , q u 0 ) | q ∈ F, δ 0 (q) > 0} ∪ {(q f , , q f ) | q ∈ Q \ F, δ 0 (q) > 0} ∪ {(q u , , b u ) | q ∈ Q \ F } ∪ {b u , , b u } ∪ {b u , , b u } ∪ {b f , , b f } ∪ {(q f , f , b f ) | q ∈ F } ∪ {(q u ,
a, q u ), (q f , a, q f ) | q, q ∈ Q, a ∈ Σ, P a [q, q ] > 0}

• P is defined by: -For all q ∈ Q with δ 0 (q) > 0), P (q 0 , u, q u ) = P (q 0 , u, q f ) = δ 0 (q) 2 ; -For all (q u , a, q u ) ∈ T , P (q u , a, q u ) = Pa[q,q ] 1+|Σ| ;

-For all (q f , a, q f ) ∈ T , P (q f , a, q f ) = Pa[q,q ] 1+|Σ| ; -For all (q u , , q u 0 ) ∈ T , P (q u , , q u 0 ) = δ 0 (q 0 ) 1+|Σ| ; -For all (q f , , q f 0 ) ∈ T , P (q f , ,

q f 0 ) = δ 0 (q 0 ) 1+|Σ| ; -For all (q u , , b u ) ∈ T , P (q u , , b u ) = 1 1+|Σ| ; -For all (q f , f , b f ) ∈ T , P (q f , f , b f ) = 1 1+|Σ| ; -P (b u , , b u ) = P (b u , , b u ) = 1 2 ; -P (b f , , b f ) = 1.
This reduction is illustrated in Figure 4.16. In each state, the sum of the probabilities of the exiting transitions correctly sum to 1. For instance, let q ∈ Q, (q f ,a,q )∈T P (q f , a, q

) = a∈Σ q ∈Q P a [q, q ] 1 + |Σ| + 1 1 + |Σ| = |Σ| 1 + |Σ| + 1 1 + |Σ| = 1. q 0 q u 1 q u 2 b u u, δ 0 (q 1 ) 2 u, δ 0 (q 2 ) 2 a, Pa[q 1 ,q 2 ] 1+|Σ| a, Pa[q 2 ,q 1 ] 1+|Σ| , 1 1+|Σ| , 1 2 , δ 0 (q 1 ) 2 , 1 1+|Σ| , δ 0 (q 2 ) 1+|Σ| q f 1 q f 2 b f u, δ 0 (q 1 ) 2 u, δ 0 (q 2 ) 2 , δ 0 (q 2 ) 1+|Σ| a, Pa[q 1 ,q 2 ] 1+|Σ| a, Pa[q 2 ,q 1 ] 1+|Σ| , δ 0 (q 1 ) 1+|Σ| , 1 f , 1 1+|Σ| Figure 4
.16: From probabilistic automata to pLTS: rectangles surround the two copies of the state space of the probabilistic automaton.

We claim that A is uniformly AFF-diagnosable if and only if L A,1/2 = ∅.

Observe first that for all q ∈ Q, L ω (A q f ) ⊆ L ω (A q u ) so that all faulty runs are ambiguous.

• Assume that there exists a word w ∈ Σ * such that P A (w) > 1/2. We prove that A is not uniformly 1 2 FF-diagnosable. So we pick arbitrary 0 < α < 1 and n α . Consider the observed sequence σ n = (w ) n for some n to be fixed later. As every word is accepted with positive probability by A, it is ambiguous. Let

γ n = P({ρ ∈ C | P(ρ ) = σ n }) P({ρ ∈ F | P(ρ ) = σ n })
.

Since P A (w) > 1/2, γ n fulfils lim n→∞ γ n = ∞.
Let ρ n be a minimal faulty run with P(ρ n ) = σ n . Let ρ be a signalling run extending

ρ n with |ρ| o = |ρ n | o + n α . Then P(ρ) = σ n nα
. By a straightforward examination of A one gets:

P({ρ ∈ C | P(ρ ) = σ n nα }) P({ρ ∈ F | P(ρ ) = σ n nα }) = γ n 2 -nα 1 + γ n 2 -nα .
Choosing n such that γ n 2 -nα > 1, one gets: CorP(ρ) > 1/2. So:

P({ρ ∈ SR nα+|ρn|o | ρ n ρ ∧ CorP(P(ρ)) > 1 2 }) = P(ρ) > αP(ρ) .
Thus A is not uniformly 1 2 FF-diagnosable.

• Conversely assume that for every word w ∈ Σ * , P A (w) ≤ 1/2. Combining this assumption with the hypothesis that P A (w) ≥ 1/4, one deduces that for every observed sequence σ ∈ (Σ ∪ { }) * , CorP(σ) ≤ 3/4. On the other hand, for every minimal faulty run ρ, P(ρ) ∈ (Σ ∪ { }) * . Pick any positive ε, α and consider an arbitrary minimal faulty run ρ. The observed sequence σ of a faulty run ρ that extends ρ fulfils σ = P(ρ) n for some n. After a new occurrence of the fraction between the probability of correct runs with observed sequence σ over the probability of faulty runs with observed sequence σ is halved.

Thus choosing n α such that 3•2 -nα 1+3•2 -nα ≤ ε, for all n ≥ n α : P({ρ ∈ SR n+|ρ|o | ρ ρ ∧ CorP(P(ρ)) ≤ ε) = P(ρ) ≥ (1 -α)P(ρ) .
Thus A is uniformly εFF-diagnosable and since ε was chosen arbitrarily, A is uniformly AFF-diagnosable.

The undecidability of (uniform) εFF-and of uniform AFF-diagnosability are due to different reasons. For (uniform) εFF-diagnosability, it is mainly caused by the use of a quantitative requirement (shown through the use of ε). For the uniform AFFdiagnosability, the problem arises as the detection speed of the fault strongly depends on the behaviour of the system before the occurrence of the fault. Limiting the behaviour of the system before the occurrence of a fault can raise decidability results (as in initialfault pLTS where uniform AFF-diagnosability is diagnosable as it is equivalent to AFFdiagnosability).

Thanks to Proposition 3.5, page 70, and the definition of AFF-diagnosability, we know that a system is AFF-diagnosable iff for all ε > 0 there exists an εFF-diagnoser of the system. Limiting oneself to finite memory can sometimes bring better decidability or complexity results (in [BFH + 14] for example). It is therefore natural to question if such a restriction would make uniform AFF-diagnosability decidable. There is several ways to add the finite-memory restriction to AFF-diagnosability: 1. a system is AFF-diagnosable with finite memory if for all ε > 0 there exists a finite-memory εFF-diagnosers; 2. a system is AFF-diagnosable with finite memory if there exists λ > 0, such that for all 0 < ε < λ there exists a finite-memory εFF-diagnosers; 3. a system is AFF-diagnosable with finite memory if there exists a sequence (ε n ) n∈N such that ∀n, ε n > 0, ε n n-→∞ ----→ 0 and for all n ∈ N there exists a finite-memory ε n -diagnosers; These three notions are however equivalent since if D is a finite-memory εFF-diagnosers for some ε > 0, then D is a finite-memory ε -diagnoser for all ε > ε. Surprisingly, this restriction complexifies the problem as stated in the following proposition.

Proposition 4.9. The AFF-diagnosability with finite memory problem is undecidable for pLTS.

The proof is obtained thanks to the reduction made in the proof of Theorem 4.4. Indeed, in this particular case, we can show that the uniform property is equivalent to the finite-memory restriction.

Proof. Given a probabilistic automaton A, let A be the pLTS built in the reduction of the proof of Theorem 4.4. We know that A is uniformly AFF-diagnosable if and only if L A,1/2 = ∅. We show that A is uniformly AFF-diagnosable iff for every ε > 0 there exists a finite-memory εFF-diagnoser of A, which establishes the undecidability.

Assume that A is uniformly AFF-diagnosable. Let ε > 0. By definition of uniform AFF-diagnosability, there exists n 0 ∈ N such that for all minimal faulty run ρ ∈ minF and all n ≥ n 0 , P(Cyl(ρ) ∩ FAmb ε n+|ρ|o ) ≤ P(ρ) 2 . As Cyl(ρ) is a single infinite run, this means that P(Cyl(ρ) ∩ FAmb ε n+|ρ|o ) = 0. Let D be the diagnoser that counts how many were observed and outputs iff this number is above n 0 and no was observed after the first . This diagnoser can be represented with finite memory as it only needs to count up to a fixed value n 0 . Moreover it is an εFF-diagnoser: correctness. if D(w) = for some w ∈ Σ o ∪ { , } * , then there exists w ∈ Σ o ∪ { } * such that w = w n with n ≥ n 0 . As in A, every word is accepted with positive probability, there exists a minimal faulty run ρ with observation w . Let ρ f be the infinite run of Cyl(ρ). By uniformity, ρ f ∈ FAmb ε n +|ρ|o . Moreover, the prefix of length n + |ρ| o of ρ f has observation w. Thus CorP(w) ≤ ε.

reactivity. The n observations following a fault are , and no can be observed after the first in a faulty run. Thus, for all m ≥ n, P(ρ ∈ F ∩ SR m | D(P(ρ)) =?) = 0 which implies reactivity.

Conversely, assume that A is not uniformly AFF-diagnosable. There thus exists ε > 0 such that A is not uniformly εFF-diagnosable. Suppose there exists a finite-memory εFF-diagnoser with m memory states. As A is not uniformly εFF-diagnosable, there exists a minimal faulty run ρ such that for all n ≤ m + 1, P(Cyl(ρ) ∩ FAmb ε n+|ρ|o ) > 0. Since there exists only one infinite run ρ f extending ρ, this means ρ f ∈ FAmb ε n+|ρ|o . Consider the observation P(ρ) m+1 . The last m + 1 memory states visited in the finitememory diagnoser while reading this observation are denoted s 1 , . . . s m+1 . None of these memory state can claim a fault by correctness of the diagnoser and as ρ f ∈ FAmb ε n+|ρ|o for n ≤ m + 1. Moreover, as the finite-memory diagnoser has m states, there exists i, j ≤ m + 1 such that s i = s j . there thus exists a cycle labelled by a number of in the finite-memory diagnoser. By determinism of the diagnoser, it means that for all n ∈ N, D(P(ρ) m+1 ) =? which contradict the reactivity requirement. There thus does not exists a finite-memory εFF-diagnoser.

PSPACE-hardness of exact diagnosability

In order to establish a lower bound for the complexity of exact diagnosability, we introduce a variant of language universality. Definition 4.7. A language L over an alphabet Σ is said eventually universal if there exists a word v ∈ Σ * such that v -1 L = Σ * .

Several variants of the universality problem were shown to be PSPACE-complete in [START_REF] Rampersad | The computational complexity of universality problems for prefixes, suffixes, factors, and subwords of regular languages[END_REF] but, to the best of our knowledge, eventual universality has not been considered.

Because of our diagnosis framework, we focus on live non-deterministic finite automata (NFA). Similarly to pLTS, an NFA is live if from every state there is at least one outgoing transition. The language of an NFA A, denoted L(A), is defined as the set of finite words that are accepted by A.

Proposition 4.10. Let A be a live NFA where all states are terminal. Then deciding whether L(A) is eventually universal is PSPACE-hard.

Proof. We reduce the universality problem for NFA, which is known to be PSPACEcomplete [START_REF] Meyer | The equivalence problem for regular expressions with squaring requires exponential space[END_REF] to the eventual universality problem. Let A = (Q, q 0 , Σ, T, F ) be an NFA. Starting from A, one builds in polynomial time the NFA A = (Q , q 0 , Σ , T , Q ) where Σ = Σ { }, Q = Q {s}, and

T = T ∪ {(q, , q 0 ) | q ∈ F } ∪ {(s, a, s) | a ∈ Σ} ∪ {(q, a, s) | a ∈ Σ, q → A }
with q → A meaning that there is no transition exiting q in A. The additional state s and the associated transitions are added to ensure that A is live, they do not alter the accepted language.

• Assume that L(A) = Σ * . Any word w over the alphabet Σ can be decomposed into w = w 1 w 2 . . . w n with w i ∈ Σ * . For each factor w i , since A is universal, there exists a run ρ i on w i ending in some terminal state q i ∈ F . Therefore w is accepted in A by the run ρ 1 ρ 2 . . . ρ n . Hence A is universal, and thus eventually universal: ε -1 L(A ) = Σ * .

• Conversely assume that A is eventually universal and let v ∈ Σ * be such that v -1 L(A ) = Σ * . Given w ∈ Σ * , we consider the word w = v w . Since A is eventually universal with witness v, w ∈ L(A ) and there exists an accepting run that can be decomposed as: ρ ρ q 0 . As a can only be read in a final state and leads to q 0 , the run ρ , which corresponds to the word w, has q 0 as initial state, ends in a final state of A, and by construction of A only uses transitions of A. So ρ is a run of A that accepts w. Therefore w ∈ L(A), and A is universal. Now that we established that universal eventuality is PSPACE-hard, we can use it to establish a complexity lower bound for the different exact diagnosability problems.

Proposition 4.11. The FF-diagnosability, FA-diagnosability and IA-diagnosability problems are PSPACE-hard.

Proof. The proof is by reduction from the eventual universality problem. Let A be a live NFA over Σ, in which all states are final. One builds in polynomial time the initial-fault pLTS A = q 0 , A f , A c depicted in Figure 4.17 where Σ o = Σ { }, Σ u = {u, f } and all transitions outgoing a state have the same probability. A f consists of a single state on which any letter of Σ can be read with a self loop. A c is a copy of A to which we add a new state q to which one can access by a transition labelled by from any state of the copy of A.

We establish the following two implications (note that they do not use the same diagnosability notion):

• A is not FA-diagnosable implies A is eventually universal; • A is eventually universal implies A is not FF-diagnosable.
Since FA-diagnosability implies IA-diagnosability, which implies IF-diagnosability which is equivalent to FF-diagnosability according to Theorem 3.1, page 73, this proves that all three notions are at least as hard as eventual universality. • Assume that A is not FA-diagnosable. By Proposition 4.4, either A FA contains a reachable BSCC C with some state s = (q, U, V ) ∈ C such that q ∈ Q f and U = ∅ or A FA contains a reachable BSCC C with some state s = (q, U, V ) ∈ C such that q ∈ Q c and V = ∅. The latter case is excluded since the only correct state belonging to a BSCC of A FA contains q as first component and q is only reachable by a transition labelled by . As this observation cannot occur in a faulty run, q = q implies V = ∅. Consider the former case: obviously q = f 0 . Since C is a BSCC and f 0 is a sink state in A, for every state s = (q , U , V ) ∈ C, one has q = f 0 and U = ∅. Since in f 0 all events of Σ are enabled, this implies that for all w ∈ Σ * , there is a correct run ρ 1 in A starting from some state of q ∈ U with observed sequence w. Consider an observed sequence v ∈ Σ * labelling a run in A FA from the initial state to s. Then there is correct run in A from q 0 to q with observed sequence v. So the run ρ = ρ 0 ρ 1 has vw as observed sequence. Since ρ = q 0 uρ with ρ a run of A starting from q 0 , vw ∈ L(A). This holds for any word w, thus v -1 L(A) = Σ * and A is eventually universal.

q 0 f 0 q 0 q q f Σ u A
• Assume that there exists a word v ∈ Σ * such that v -1 L(A) = Σ * . Of course, any word extending v is also a witness that A is eventually universal. Let v ∈ Σ * be such that some faulty run with observed sequence vv ends in a BSCC C of A FF . Since (vv ) -1 L(A) = Σ * , all states of C are of the form (f 0 , U ) with U = ∅. Therefore, by Proposition 4.2, A is not FF-diagnosable.

Since the lower bounds matches the upper bounds, the different notions of exact diagnosability are PSPACE-complete for finite pLTS.

Diagnoser construction

When a system is shown to be diagnosable, the next step is to build a diagnoser. A diagnoser that works in every case would keep track of every run compatible with the observed sequence and give a verdict depending on the nature of this set of runs. This diagnoser, by nature, uses unbounded memory. For implementation purpose, we are rather interested in finite-memory diagnosers as defined in Section 1 of Chapter 3. We explain how to automatically build a finite-memory diagnoser for a diagnosable system. This is not possible for every notion of diagnosability however. Indeed, we showed in Proposition 3.4, page 69, that εFF-diagnosers may need infinite memory. Therefore we do not develop approximate diagnosability here, and focus on exact diagnosability.

FF-diagnoser

We start with FF-diagnosers. These diagnosers only provide information about faulty runs. In the sequel we fix A a finite pLTS. Proposition 4.12. If A is an FF-diagnosable pLTS with n correct states, one can build an FF-diagnoser for A with at most 2 n memory states.

Proof. The idea of this proof and of all the following proofs for constructing a finitememory diagnoser is to use the characterisation given in Section 1. For example, in order to construct the FF-diagnoser, we first build the FF-automaton of A. Then we define the finite-memory diagnoser on its structure. Finally we show that the constructed diagnoser is indeed an FF-diagnoser thanks to the FF-diagnosability of the system.

For an FF-diagnosable pLTS A with FF(A) = (S, s 0 , ∆, F ), its deterministic and complete FF-automaton, we define the finite memory diagnoser (S, Σ o , up, s 0 , D fm ) with up(s, a) = s if (s, a, s ) ∈ ∆ and D fm (U ) = iff U = ∅. Let us show that the induced diagnoser D is indeed an FF-diagnoser, and that it has at most 2 n memory states, where n is the number of correct states of A.

commitment When U is empty, it remains empty forever which implies commitment.

correctness When D outputs the verdict , FF(A) is in the state associated with ∅. As U contains the set of correct states reachable with the current observed sequence, the observed sequence is surely faulty.

reactivity If an infinite faulty run ρ is such that D(P(ρ)) =? then, by construction of FF(A) and definition of D, for every length n ∈ N, there exists a finite correct signalling run ρ n ∈ SR n such that P(ρ n ) = P(ρ ↓n ). Using König's lemma, since A is finitely branching, one can extract an infinite correct run ρ ∞ such that P(ρ ∞ ) = P(ρ), so that ρ ∈ FAmb ∞ . Moreover P(FAmb ∞ ) = 0 as A is FFdiagnosable. Putting everything together, for every minimal faulty run ρ, P({ρ ∈ Ω | ρ ρ ∧ D(P(ρ )) =?}) = 0.

size The memory states are states of FF(A), which are themselves subsets of correct states of A. Therefore, D uses at most 2 n memory states, with n = |Q c |.

We now show that the size order of the previous FF-diagnoser is optimal.

Proposition 4.13. There is a family {A n } n∈N of FF-diagnosable pLTS such that A n has 2n + 2 correct states and it admits no FF-diagnoser with less than 2 n states.

A n Proof. This proof is inspired by a similar result of lower bounds on controllers established in [START_REF] Haar | Optimal constructions for active diagnosis[END_REF]. Consider the example of Figure 4.18 where Σ o = {a, b, c} and the initial state is q 0 . Consider a finite faulty run including a c event. Its observed sequence belongs to L = {a, b} * a{a, b} n-1 c + . Since any finite correct run has an observed sequence belonging to L = {a, b} * ∪ {a, b} * b{a, b} n-1 c + and L ∩ L = ∅, FAmb n CAmb n ⊆ {ρ | P(ρ) ∈ {a, b} n }. Since lim n→∞ P({ρ | P(ρ) ∈ {a, b} n }) = 0, the pLTS is FA-diagnosable and so IA-diagnosable and FF-diagnosable.

q 0 l 0 l 1 l 2 . . . l n r 1 r 2 . . . r n f a a,
Intuitively, when a c is observed, any FF-diagnoser must have remembered the observable event that happened n steps earlier to know if the run is faulty or not. Thus, it must remember the last n observed events, in case a c event occurs.

More formally, assume there exists a diagnoser D = (M, Σ, m 0 , up, D fm ) with less than 2 n memory states. Then there exist two distinct words w 1 ∈ {a, b} n and w 2 ∈ {a, b} n leading to the same memory state: up(m 0 , w 1 ) = up(m 0 , w 2 ). The words w 1 and w 2 differ at least from one letter say w 1 [i] = b and w 2 [i] = a. Consider for k ≥ 1, the signalling correct run ρ 1,k corresponding to observed sequence w 1 a i-1 c k whose sequence of visited states is q i 0 r 1 . . . r k+1 n and the signalling faulty run ρ 2,k corresponding to observed sequence w 2 a i-1 c k whose sequence of visited states is q i 0 l 0 l 1 . . . l k+1 n . They also lead to the same memory state. By correctness, D(w 1 a i-1 c k ) =?. Thus for all suffixes ρ of ρ 2,1 , D(ρ) =? contradicting the reactivity of D.

FA-diagnoser

We now turn to FA-diagnosability which not only considers the diagnosis of faults but also of correct runs. We build the FA-diagnoser from the FA-automaton similarly to what was done in Proposition 4.12.

Proposition 4.14. A pLTS A is FA-diagnosable if it admits an FA-diagnoser. Moreover, if A is a FA-diagnosable pLTS with n states, one can build an FA-diagnoser with at most 2 n memory states.

Proof. Let A be a pLTS. Assume first that there exists an FA-diagnoser D for A. For every n ∈ N, we define FD n = {ρ ∈ Ω | D(P(ρ ↓n )) = } the set of runs that are diagnosed faulty after n observed events, and symmetrically CD n = {ρ ∈ Ω | ∀m ≥ n, D(P(ρ ↓m )) = ⊥} the set of runs that are persistently diagnosed correct after n observed events. The sequences (CD n ) n∈N and (FD n ) n∈N are non-decreasing. As ? ≺ ⊥ and ? ≺ , for every run ρ ∈ Ω,

D inf (P(ρ)) =? is equivalent to ρ / ∈ n (FD n ∪ CD n ). Thus we have n∈N (FD n ∪ CD n ) = {ρ ∈ Ω | D inf (P(ρ)) =?}. Since D is reactive, P({ρ ∈ Ω | D inf (P(ρ)) =?}) = 1. Moreover, since D is correct, for every n ∈ N, FD n ⊆ Sf n and CD n ⊆ Sc n . Thus for every n ∈ N, P(FAmb n ∪ CAmb n ) = 1 -P(Sf n ∪ Sc n ) ≤ 1 -P(FD n ∪ CD n ) and lim n-→∞ P(FAmb n ∪ CAmb n ) ≤ 1 -lim inf n-→∞ P({ρ ∈ SR n | D(P(ρ)) =?}) = 0. This shows that A is FA-diagnosable.
Assume now that A is FA-diagnosable and has n states. From FA(A) = (S, s 0 , ∆, F ) the FA-automaton of A, we define D = (S, Σ o , s 0 , up, D fm ) the finite-memory diagnoser where up(s, a)

= s if (s, a, s ) ∈ ∆, D fm ((U, V )) = iff U = ∅ and D fm ((U, V )) = ⊥ iff V = ∅.
Let us check that D is an FA-diagnoser, and that its size is at most 2 n if n denotes the number of states of A.

commitment When U is empty, it remains empty forever which implies commitment.

correctness Let w ∈ Σ * o be an observed sequence. If (U, V ) is the state in FA(A) reached after reading w, then recall that U (resp. V ) is the set of states in A that can be reached by correct (resp. faulty) signalling runs labelled by w. By construction, if D(w) = then w is surely faulty, and if D(w) = ⊥ then w is surely correct. size D has at most 2 n memory states because every state of FA(A) consists of a pair

(U, V ) with U ⊆ Q c and V ⊆ Q f .
As the pLTS of Figure 4.18 is FA-diagnosable, and since any FA-diagnoser is also an FF-diagnoser, using Proposition 4.13 we obtain the following lower bound for the size of FA-diagnosers.

Proposition 4.15. There is a family {A n } n∈N of FA-diagnosable pLTS such that A n has 2n + 2 states and it admits no FA-diagnoser with less than 2 n memory states.

IA-diagnoser

We end with IA-diagnosability and build an IA-diagnoser similarly as to what was done in Proposition 4.12.

Proposition 4.16. If A is a IA-diagnosable pLTS with n c correct states and n f faulty states, one can build an IA-diagnoser with at most 2 nc 3 n f states.

Proof. Let A be an IA-diagnosable pLTS. From IA(A) = (S, s 0 , ∆, F ) the IA-automaton of A, we define D = (S, Σ o , s 0 , up, D fm ) the finite-memory diagnoser where up(s,

a) = s if (s, a, s ) ∈ ∆, D fm ((U, V, W )) = iff U = ∅ and D fm ((U, V, W )) = ⊥ iff W = ∅. Let us prove that D is indeed an IA-diagnoser for A.
commitment. When U is empty, it remains empty forever which implies commitment. correctness. For any word w ∈ Σ * o , we denote by (U w , V w , W w ) the state in IA(A) reached after reading w. For any word w, if U w = ∅, by construction of IA(A), w is surely faulty. Assume now that W w = ∅ and U w = ∅. Let w be the longest proper prefix of w such that W w = ∅. Let ρ be any signalling run with P(ρ) = w. Assume that ρ ↓|w | is faulty. Thus the states visited by ρ ↓n for |w | < n ≤ |w| were always in W ρ ↓n . Since W w = ∅, this is not possible and so ρ ↓|w | is correct. Thus every time a state with W = ∅ is visited, the length of the greatest prefix, for which all signalling subruns corresponding to this prefix are correct, is increased. This establishes correctness.

reactivity. Let ρ be an infinite run such that D sup (P(ρ)) =?. Due to the characterisation of Proposition 4.5, either (1) the strongly connected component of A IA that ρ infinitely often visits is not a BSCC or (2) ρ does not visit infinitely often all the states of this strongly connected component. The probability of such runs is null which establishes the reactivity.

size D has at most 2 nc 3 n f memory states because every state of IA(A) consists of a triple (U, V, W ) with U ⊆ Q c and V ∪ W ⊆ Q f . Moreover, one does not keep in V the states that are tracked in W , ensuring V ∩ W = ∅.

The following lower bound can be derived from the proof of Proposition 4.13, since the pLTS of Figure 4.18 is IA-diagnosable, and because any IA-diagnoser is also an FF-diagnoser.

Proposition 4.17. There is a family {A n } n∈N of IA-diagnosable pLTS such that A n has 2n + 2 states and it admits no IA-diagnoser with less than 2 n memory states.

The construction of an exact diagnoser can thus require exponential time, which is one class above the verification of exact diagnosability. This exponential time is only necessary if we want to build the full diagnoser though. Another possibility would be to update the state of the diagnoser on-the-fly during a run. One would only need to keep the current memory state (which has linear space), but the update process would take a polynomial time, instead of the constant time obtained when using a fully-built diagnoser.

Conclusion

In Section 1, we gave characterisations of the different notions of exact diagnosability and of one notion of approximate diagnosability for finite systems. The characterisations of the notions of exact diagnosability are of the same descriptive complexity, something that is impossible in the general case as shown in Section 3 of Chapter 3, page 77. The characterisation of the notion of approximate diagnosability has two important differences compared to the ones of exact diagnosability: (1) it depends on the exact probabilities of the system and (2) it only requires a comparison between pairs of states.

In Section 2, we gave matching upper and lower bounds for the various diagnosability problems for finite systems. These results heavily relied on the characterisations given in Section 1. Thus, as the characterisations of every notion of exact diagnosability have the same descriptive complexity, it is not surprising that they end up having the same complexity. The results on the approximate diagnosability notions are more surprising. AFF-diagnosability, the notion for which a characterisation was obtained, is decidable in polynomial time, a better complexity than what is needed for the exact diagnosability notions. This gain in complexity is obtained as, contrary to exact diagnosability where one needs to follow sets of states, for AFF-diagnosability, only pairs of states have to be compared. The comparison however depends on the exact values of the probabilities of the system, which brings a different kind of difficulty to the analysis. While this difficulty could be solved for AFF-diagnosability, it is the main reason of the undecidability of all the other notions of approximate diagnosability: (uniform) εFFand uniform AFF-diagnosability. On this point, uniform AFF-diagnosability is equivalent to the notion of AA-diagnosability introduced in [TT05] which decidability was left open (only necessary conditions were given). Our undecidability result thus answers negatively to this question.

In Section 3, we gave automatic methods to construct finite-memory diagnosers for systems that are exactly diagnosable. We also showed that the sizes of the generated diagnosers are asymptotically optimal. For approximate notions of diagnosability, there does not necessarily exist finite-memory diagnosers and deciding the existence of such a diagnoser is undecidable. When such a diagnoser does not exist, one may need unbounded memory. Depending on the form of the unbounded memory required, it can be more or less manageable (for example a counter can easily be implemented).

Chapter 5

Algorithmic analysis of the diagnosability of infinite pLTS

In Chapter 4, we showed how to solve diagnosability for systems that can be represented by a pLTS with finitely many states.While this encompasses many kinds of systems, this is far from being exhaustive. Often, in order to satisfy its specification, a system will require unbounded memory: for example, when the system receives and records information or requests from the environment. Observe that an infinite number of states does not mean an infinite memory per se, but only an unbounded one. Stacks and queues are instances of such dynamic data structures.

While allowing an infinite amount of states increases the expressive power of the model, it increases the difficulty of its study. First, the studied systems must possess a finite representation. This can be done by assuming only a finite part of the system needs to be studied (e.g. infinite systems with finite attractors [START_REF] Baier | A note on the attractor-property of infinite-state markov chains[END_REF]) or by using a higher level model whose semantics is an infinite-state system (Petri nets [Mur89, Dia09, CGS14], well structured transition systems [START_REF] Finkel | Well-structured transition systems everywhere![END_REF], pushdown automata [AM04, KEM06, MP09, HS10, EY12]). We study such a formalism in this chapter.

Diagnosability has already been studied for infinite-state non-stochastic systems: represented by pushdown automata [START_REF] Morvan | Diagnosability of pushdown systems[END_REF] or by Petri nets [START_REF] Cabasino | A new approach for diagnosability analysis of petri nets using verifier nets[END_REF][START_REF] Bérard | The complexity of diagnosability and opacity verification for Petri nets[END_REF]. Partially observable visibly pushdown automata is a subclass of partially observable pushdown automata for which diagnosability was studied in [START_REF] Morvan | Diagnosability of pushdown systems[END_REF]; for such models, diagnosability is decidable (using the determinisation procedure of [START_REF] Alur | Visibly pushdown languages[END_REF]). With a restriction on the unobservable subnet akin to our convergence assumption, [START_REF] Cabasino | A new approach for diagnosability analysis of petri nets using verifier nets[END_REF] gives a decidable characterisation of a non-stochastic notion of diagnosability for partially observable Petri nets. However the algorithm is non-primitive recursive. The authors of [START_REF] Bérard | The complexity of diagnosability and opacity verification for Petri nets[END_REF] extend this work by considering different classes of Petri nets and reducing the complexity (EXPSPACE for the general case). However to the best of our knowledge diagnosis of probabilistic infinite-state systems has not yet been studied.

So we extend these works by considering the probabilistic variants of diagnosability. In Section 1, we study the stochastic diagnosability of partially observable probabilistic pushdown automata (POpPDA). As diagnosability is already undecidable for non-stochastic systems [START_REF] Morvan | Diagnosability of pushdown systems[END_REF], the decidability of the stochastic variants is unlikely. However the notations introduced here will be used in Section 2 where we consider a restriction of POpPDA called partially observable probabilistic visibly pushdown automata (POpVPA). This restricted class has many advantages. It naturally benefits from the numerous results which are known for POpPDA [START_REF] Brázdil | Analyzing probabilistic pushdown automata[END_REF], especially the ones on model-checking algorithms [START_REF] Kučera | Model checking probabilistic pushdown automata[END_REF][START_REF] Etessami | Model checking of recursive probabilistic systems[END_REF]. Moreover, the authors of [START_REF] Alur | Visibly pushdown languages[END_REF] gave an algorithm for the determinisation of a POpVPA. PopVPA generates an infinitestate pLTS and the efficient characterisations given in Chapter 4 strongly rely on the finiteness of the models. They thus cannot be used any more. So, we use the characterisations from the Section 3 of Chapter 3 based on the pathL logic. However the model-checking algorithm cannot be directly applied to the formulae of the pathL logic. Some tricky machinery will be required to "encode" the path formulae of the pathL logic in the system in order to use the results of [START_REF] Etessami | Model checking of recursive probabilistic systems[END_REF]. Finally, in Section 3, we study the diagnosability of partially observable stochastic Petri nets (POSPN), we mimic the restriction used on POpPDA in Section 2, and discuss the case of partially observable stochastic visible Petri nets (VSPN).

This chapter develops and extends some of the results from [START_REF] Bertrand | Diagnosis in infinite-state probabilistic systems[END_REF][START_REF] Lefaucheux | Basis coverability graph for partially observable Petri nets with application to diagnosability analysis[END_REF].

Diagnosability of probabilistic pushdown automata

In this section we study infinite-state pLTS generated by probabilistic pushdown automata (pPDA). First, we define pPDA and the infinite-state pLTS generated by a pPDA. The pPDA model being very expressive, we show that diagnosability of pPDA is undecidable.

Probabilistic pushdown automata

A pPDA randomly generates infinite behaviours using a stack. This stack contains letters of a stack alphabet with the most recently added letter put at the top. Transitions of the pPDA can be conditioned by the top of the stack. Moreover, a transition can push a new element onto the stack, pop one element off it or modify the top of the stack. Let us first see an example of an infinite state pLTS, that we will be able to represent by a pPDA.

Example 5.1. The pLTS of Figure 5.1 represents a server that accepts jobs (event in) until it randomly decides to serve the jobs (event serve). When a job is done the result is delivered (event out). When all jobs are done, the server waits for a new batch of jobs. However randomly, the server may trigger a fault (event f ) and then abort all remaining jobs (event abort). Afterwards, the server is reset (event reset).

The infinite number of states in this system comes from the unbounded number of jobs the server can receive before he starts serving them. To see this system using a stack, one could use a stack letter to represent a job and add one such letter to the stack every time a new job comes in. Dealing with a job (event out) or aborting it (event abort) removes an element of the stack.

q 0 q 10 f 10 q 01 q 11 f 11 q 02 q 12 f 12 . . . . . . . . . in, 1 in, 1 2 in, 1 2 out, 1 2 out, 1 2 out, 1 2 empty, 1 serve, 1 2 serve, 1 2 abort, 1 abort, 1 abort, 1 f , 1 2 f , 1 2 reset, 1
Figure 5.1: An infinite-state pLTS that can be represented by a pPDA.

We now define pPDA similarly to what can be found in [START_REF] Kučera | Model checking probabilistic pushdown automata[END_REF].

Definition 5.1. A probabilistic pushdown automaton (pPDA) is defined by a tuple V = (Q, q 0 , Σ, Γ, δ, P) where:

• Q is a finite set of states with q 0 the initial state;

• Σ is a finite alphabet of events;

• Γ is a finite alphabet of stack symbols including a set of bottom stack symbols Γ ⊥ with initial symbol ⊥ 0 ∈ Γ ⊥ ;

• δ ⊆ Q×Γ×Σ×Q×Γ * is the set of transitions such that for every (q, γ, a, q, w) ∈ δ,

|w| ≤ 2, γ ∈ Γ ⊥ implies w ∈ Γ ⊥ (Γ \ Γ ⊥ ) * and γ / ∈ Γ ⊥ implies w ∈ (Γ \ Γ ⊥ ) * ;
• P is the transition probability function fulfilling for every q ∈ Q and γ ∈ Γ:

(q,γ,a,q ,w)∈δ P[(q, γ, a, q , w)] = 1.

A pPDA may be viewed as a pLTS equipped with a stack. The transitions of the pPDA can depend on the top symbol of the stack and modify it. The definition ensures that the stack is never empty: the bottom stack symbols Γ ⊥ are never removed. Moreover symbols of Γ ⊥ never occurs elsewhere in the stack. Let T = (q, γ, a, q , w) ∈ δ be a transition of a pPDA. If |w| = 1 (resp. |w| = 2, |w| = 0) then T is said to be a local (resp. push, pop) transition. A local transition can update the top symbol and a push transition can modify the top symbol and add another symbol on top of it. Notions such as runs are defined on pPDA analogously to what was done for pLTS. We call configuration the pair composed by a state and a stack.

The semantics of a pPDA is the (potentially infinite) pLTS representing its behaviour. The states of this pLTS are pairs consisting of a state and a stack contents. They therefore contains all the information that are necessary in the pPDA to determine the available transitions and their probabilities. Definition 5.2. A probabilistic pushdown automaton pPDA V = (Q, q 0 , Σ, Γ, δ, P) defines a pLTS A V = (Q V , (q 0 , ⊥), Σ, T V , P V ) where:

• Q V = {(q, z) | q ∈ Q ∧ z ∈ ⊥Γ * };
• T V = {((q, zγ), a, (q , zw)) | zγ ∈ ⊥Γ * ∧ (q, γ, a, q , w) ∈ δ};

• For every ((q, zγ), a, (q , zw)) ∈ T V , P V [((q, zγ), a, (q , zw))] = P[(q, γ, a, q , w)].

As a pPDA has a finite number of states, the associated pLTS is finitely branching.

Example 5.2. Figure 5.2 gives an example of a pPDA whose semantics is the pLTS from Figure 5.1. Indeed, the stack alphabet has only one letter. We could thus replace it by a counter giving the number of element in the stack. The pLTS of Figure 5.1 does exactly that by representing the configuration (q 1 , γ n ) of the semantics of the pPDA by the state q 1n and similarly for the other configurations. A transition t = (q, γ, a, q , w) is represented by an edge from state q to state q and labelled by P[t] • γ, a, w.

Observe that in this example the set of states is not partitioned between faulty and correct states as from the state f 1 reached by a faulty run, one can go back to the initial state with the reset event (event r). As for pLTS, we enlarge pPDA with partial observation features. As discussed in Section 1.3 of Chapter 2, this can be done either by partitioning the alphabet of events into observable and unobservable events or by providing a mask function associating with every event an observation. In the previous chapters we used the partition of the alphabet of events. Here, we use the mask function which is more appropriate for our needs.

q 0 q 1 f 1 1 2 • γ, serve, γ 1 • ⊥ 0 , empty, ⊥ 0 1 2 • γ, f , γ 1 • ⊥ 0 , reset, ⊥ 0 1 2 • γ, in, γγ 1 • ⊥ 0 , in, ⊥ 0 γ 1 2 • γ, out, ε 1 • γ, abort, ε (q 0 , ⊥ 0 ) (q 0 , γ ⊥ 0 ) (q 0 , γ γ ⊥ 0 ) (q 1 , γ γ ⊥ 0 ) (q 1 , γ ⊥ 0 ) (q 1 , ⊥ 0 ) (q 0 , ⊥ 0 ) (f 1 , γ γ ⊥ 0 ) (f 1 , γ ⊥ 0 ) (f 1 , ⊥ 0 ) (q 0 , ⊥ 0 ) in in
Definition 5.3. A partially observable pPDA (POpPDA) is a tuple V, Σ o , P consisting of a pPDA V equipped with a mapping P : Σ → Σ o ∪ {ε} where Σ o is the set of observations.

A POpPDA is diagnosable according to a notion of diagnosability if the pLTS it generates is diagnosable. As the generated pLTS are finitely branching, thanks to Theorem 3.1 a POpPDA is FF-diagnosable iff it is IF-diagnosable.

Example 5.3. Consider the pPDA V of Figure 5.2, we define Σ o = {in, out, loc, reset} and two observation masks P 1 and P 2 with P 1 (in) = in, P 1 (serve) = P 1 (empty) = P 1 (reset) = loc, P 1 (abort) = P 1 (out) = out and P 1 (f ) = ε, P 2 (reset) = reset and for every event t = reset, P 2 (t) = P 1 (t). V, Σ o , P 1 and V, Σ o , P 2 are two POpPDA differentiated only by the observation of the event reset. As a faulty run will inevitably contain a reset and a correct run that leaves q 0 will contain a serve, the POpPDA V, Σ o , P 2 which distinguishes these two events is diagnosable for every non-uniform exact notion of diagnosability. However, the POpPDA V, Σ o , P 1 is not diagnosable as serving the requests and going back to q 0 has the same observation as making a fault, aborting the requests and going back to q 0 .

Undecidability of diagnosability for POpPDA

Unfortunately, for every notion of diagnosability, the diagnosability problem for POp-PDA is undecidable. The undecidability can be obtained by adapting the proof for diagnosability of non-probabilistic pushdown automata [START_REF] Morvan | Diagnosability of pushdown systems[END_REF]. However, in order to show how robust the result is, we rather reduce from the Post Correspondence Problem (PCP). An instance of PCP is given by an integer n ∈ N and two families of non-empty words {v i } i≤n and {w i } i≤n on the alphabet {a, b}. The following question is undecidable [START_REF] Post | A variant of a recursively unsolvable problem[END_REF]: does there exist k > 0 and i 1 , . . . i k ∈ {1, . . . , n} such that

w i 1 . . . w i k = v i 1 . . . v i k ?
We show in Theorems 5.1 and 5.2 that undecidability already holds for two (incomparable) subclasses of POpPDA with restriction on what is observable and on the number of phases of any run. A phase is a portion of a run in which the stack either never decreases or never increases.

Theorem 5.1. The diagnosability problems are undecidable for POpPDA even when (1) a local transition does not update the top of the stack, (2) every event labelling a push transition is fully observable and corresponds to the pushed symbol, and (3) every run has at most two phases.

The use of the stack is obviously central to the proof of the undecidability. Moreover conditions (1) and (2) limit the ability of push and local transitions to silently manipulate the stack. There is no such limitation for pop transitions however as the proof of undecidability heavily rely on hiding when the pop transitions are performed.

Let us sketch the proof. We reduce the PCP problem to the diagnosability problem. To do so, the POpPDA we build, from a PCP instance (n, {v i } i≤n , {w i } i≤n ), is divided in three parts. In the first one, it will select and push onto its stack a sequence of numbers, i 1 . . . i k , with ∀j ≤ k, i j ≤ n. Then it goes randomly to one of the two other parts of the POpPDA, one part being accessed by a faulty transition. Each of these two parts is associated with a family of words of the PCP instance. Once it reached one of these parts, say the one associated with the w i , the run will read the words of w i induced by the numbers pushed on the stack. The resulting observation is w = w i 1 . . . w i k . On the other part, the observation is similarly v = v i 1 . . . v i k . The fault can be detected if and only if v = w. Having the pop transition undetected is fundamental as it allows to hide when the word w i j starts being read.

Proof. Let (n, {v i } i≤n , {w i } i≤n ) be a PCP instance In this proof, we let i (resp. m i ) be the length of v i (resp. w i ). Also, given a word w and k ≤ |w| we use w[k] to denote the k th -letter of w.

We build a pPDA V = (Q, Σ, Γ, δ, P) as follows:

• Q = {q 0 , q c , q s , f s } ∪ {q k i | 1 ≤ i ≤ n, 1 ≤ k ≤ i } ∪ {f k i | 1 ≤ i ≤ n, 1 ≤ k ≤ m i } ;
• Σ = {1, . . . , n, , u, r, f , a, b};

• Γ = {1, . . . , n, ⊥ 0 } with Γ ⊥ = {⊥ 0 };

• δ consists of the following transitions:

{(q 0 , ⊥ 0 , x, ⊥ 0 x, q c ) | 1 ≤ x ≤ n} ∪ {(q c , x, y, xy, q c ) | 1 ≤ x, y ≤ n} ∪ {(q k i , z, v i [k], z, q k+1 i ) | 1 ≤ i ≤ n, 1 ≤ k < i , z ∈ {⊥ 0 , 1, . . . , n}} ∪ {(f k i , z, w i [k], z, f k+1 i ) | 1 ≤ i ≤ n, 1 ≤ k < m i , z ∈ {⊥ 0 , 1, . . . , n}} ∪ {(q i i , z, v i [ i ], z, q s ) | 1 ≤ i ≤ n, z ∈ {⊥ 0 , 1, . . . , n}} ∪ {(q s , x, r, ε, q 1 x ) | 1 ≤ x ≤ n} ∪ {(f m i i , z, w i [m i ], z, f s ) | 1 ≤ i ≤ n, z ∈ {⊥ 0 , 1, . . . , n}} ∪ {(f s , x, r, ε, f 1 x ) | 1 ≤ x ≤ n} ∪ {(q c , x, u, x, q s ), (q c , x, f , x, f s ) | 1 ≤ x ≤ n} ∪ {(q s , ⊥ 0 , , ⊥ 0 , q s ), (f s , ⊥ 0 , , ⊥ 0 , f s )}.
• P assigns arbitrary positive probabilities to transitions in δ: P(q, γ, a, q , w) > 0 ⇔ (q, γ, a, q , w) ∈ δ and (q,γ,a,q ,w)∈δ P[(q, γ, a, q , w)] = 1.

We further consider the POpPDA V, Σ o , P with Σ o = Σ \ {r, u, f }, and the masking function satisfies P(u) = P(r) = P(f ) = ε and P(x) = x for any other event x. This POpPDA is represented in Figure 5.3.

Let us prove that the instance of the PCP is positive if and only if the POpPDA is IF-, IA-, FAand AFF-diagnosable.

First, observe that almost surely occurs in an infinite run of the pPDA V. Thus, for any ε > 0, there exists N ε ∈ N such that the measure of signalling runs with observable length N ε that reach configurations (q s , ⊥ 0 ) or (f s , ⊥ 0 ) by an event is at least 1 -ε.

• Assume that there exists a solution i 1 , . . . , i k to the PCP instance (n, {v i } i≤n , {w i } i≤n ). Consider in the POpPDA the faulty run:

ρ f = q 0 (i j q c ) j≤k f (f s r(f p i j w i j [p]) p≤m i j ) j≤k (f s ) ω ,
and the correct run: These two runs have the same observed sequence:

ρ c = q 0 (i j q c ) j≤k u(q s r(q p i j v i j [p]) p≤ i j ) j≤k (q s ) ω . q 0 q c f s q s q 1 1 . . . q 1 1 . . . q 1 n . . . q n n f 1 1 . . . f m 1 1 . . . f 1 n . . . f mn n ⊥ 0 , x, ⊥ 0 x x, f , x x, u, x x, y, xy 1, r, ε z, v 1 [1], z z, v 1 [ 1 -1], z z, v 1 [ 1 ], z n, r, ε z, v n [1], z z, v n [ n -1], z z, v n [ n ], z ⊥ 0 , , ⊥ 0 1, r, ε z, w 1 [1], z z, w 1 [m 1 -1], z z, w 1 [m 1 ], z n, r, ε z, w n [1], z z, w n [m n -1], z z, w n [m n ], z ⊥ 0 , , ⊥ 0
P(ρ f ) = P(ρ c ) = i 1 . . . i k w ω with w = w i 1 . . . w i k = v i 1 . . . v i k .
Therefore, ρ f is an infinite ambiguous faulty run. Given that P(ρ f ) > 0, we deduce that the POpPDA V, Σ o , P is not IF-diagnosable. From Theorem 3.1, it is also neither IA-diagnosable nor FA-diagnosable. Moreover, after the occurrence of a fault, there is no probabilistic choice. As a consequence the correctness proportion is either 0 or 1/2. As the correctness proportion of the faulty prefixes of ρ f is never 0 as seen above, the POpPDA is not AFF-diagnosable.

• Conversely, assume that the PCP instance (n, {v i } i≤n , {w i } i≤n ) has no solution. Let ε > 0, let N ε ∈ N be the integer obtained with our earlier observation. Consider a correct run ρ c with observable length N ε , ending in (q s , ⊥ 0 ) and containing at least an occurrence of . Its observed sequence is of the form

P(ρ c ) = i 1 . . . i k v i 1 . . . v i k
m for some i 1 , . . . , i k , m. Due to the fact that (n, {v i } i≤n , {w i } i≤n ) has no solution, no faulty run can have the same observed sequence. Therefore, ρ c is surely correct. Symmetrically, any faulty run ending in (f s , ⊥ 0 ) after an occurrence of is surely faulty. We thus conclude that, for any ε > 0, there exists N ε ∈ N such that P(FAmb Nε CAmb Nε ) ≤ ε. As a consequence, the POpPDA V, Σ o , P is FA-diagnosable. By Theorem 3.1 it is also IA-diagnosable, IF-diagnosable and AFF-diagnosable.

A similar undecidability result holds for a class of POpPDA in which pop events are fully observable, and the number of phases is constant:

Theorem 5.2. The diagnosability problems are undecidable for POpPDA even when (1) a local transition does not update the top of the stack, (2) every event labelling a pop transition is fully observable and corresponds to the popped symbol, and (3) every run has at most two phases.

Proof. The proof follows the same line as the one for Theorem 5.1. The difference is that instead of choosing first the words that will be read by pushing them on the stack and later popping them off discreetly, the pPDA reads the words and silently push on the stack which words were read and at the end pop them off and verify if the same sequence could indeed be used for both family of words. From an instance (n, {v i } i≤n , {w i } i≤n ) of PCP, let us define a pPDA V = (Q, Σ, Γ, δ, P) where:

• Q = {q 0 , q s , f s , q e , f e }∪{q k i | 1 ≤ i ≤ n, 1 ≤ k ≤ i }∪{f k i | 1 ≤ i ≤ n, 1 ≤ k ≤ m i };
• Σ = {1, . . . , n, , u, c, f , a, b};

• Γ = {1, . . . , n, ⊥ 0 } with Γ ⊥ = {⊥ 0 };
• δ consists of the following transitions:

{(q 0 , ⊥, u, ⊥, q s ), (q 0 , ⊥, f , ⊥, f s ), (q e , ⊥, , ⊥, q e ), (f e , ⊥, , ⊥,

f e )} ∪ {(q k i , z, v i [k], z, q k+1 i ) | 1 ≤ i ≤ n, 1 ≤ k < i , z ∈ {⊥, 1, . . . , n}} ∪ {(f k i , z, w i [k], z, f k+1 i ) | 1 ≤ i ≤ n, 1 ≤ k < m i , z ∈ {⊥, 1, . . . , n}} ∪ {(q i i , z, v i [ i ], z, q s ) | 1 ≤ i ≤ n, z ∈ {⊥, 1, . . . , n}} ∪ {(f m i i , z, w i [m i ], z, f s ) | 1 ≤ i ≤ n, z ∈ {⊥, 1, . . . , n} ∪ {(q s , z, c, zx, q 1 x ) | z ∈ {⊥, 1, . . . , n}, x ∈ {1, . . . , n}} ∪ {(f s , z, c, zx, f 1
x ) | z ∈ {⊥, 1, . . . , n}, x ∈ {1, . . . , n}} ∪ {(q s , x, x, ε, q e ) | x ∈ {1, . . . , n}} ∪ {(f s , x, x, ε, f e ) | x ∈ {1, . . . , n}} ∪ {(q e , x, x, ε, q e ) | x ∈ {1, . . . , n}} ∪ {(f e , x, x, ε, f e ) | x ∈ {1, . . . , n}}.

• P assigns arbitrary positive probabilities to transitions in δ.

We further consider the POpPDA V, Σ o , P with Σ o = Σ \ {c, u, f }, and the masking function satisfies P(u) = P(c) = P(f ) = ε and P(x) = x for any other event x. This POpPDA is represented in Figure 5.4. Let us prove that the instance of the PCP is positive if and only if the POpPDA is IF-, IA-, FAand AFF-diagnosable.

q 0 f s f e q s q e q 1 1 . . . q 1 1 . . . q 1 n . . . q n n f 1 1 . . . f m 1 1 . . . f 1 n . . . f mn n ⊥ 0 , f , ⊥ 0 ⊥ 0 , u, ⊥ 0 z, c, z1 1, v 1 [1], 1 1, v 1 [ 1 -1], 1 1, v 1 [ 1 ], 1 z, c, zn n, v n [1], n n, v n [ n -1], n n, v n [ n ], n ⊥ 0 , , ⊥ 0 x, x, ε x, x, ε z, c, z1 1, w 1 [1], 1 1, w 1 [m 1 -1], 1 1, w 1 [m 1 ], 1 z, c, zn n, w n [1], n n, w n [m n -1], n n, w n [m n ], n ⊥ 0 , , ⊥ 0 x, x, ε x, x, ε
First, observe that almost surely occurs in an infinite run of the pPDA V. Thus, for any ε > 0, there exists N ε ∈ N such that the measure of signalling runs with observable length N ε that reach configurations (q e , ⊥ 0 ) or (f e , ⊥ 0 ) by an event is at least 1 -ε.

• Assume that there exists a solution i 1 , . . . , i k to the PCP instance (n, {v i } i≤n , {w i } i≤n ). Consider the faulty run:

ρ f = q 0 f f s (c(f p i j w i j [p]
) p≤m i j f s ) j≤k (i j f e ) j≤k ( f e ) ω , and the correct run:

ρ c = q 0 uq s (c(q p i j v i j [p]
) p≤ i j q s ) j≤k (i j q e ) j≤k ( q e ) ω .

These two runs have the same observed sequence: P(ρ f ) = P(ρ c ) = wi 1 . . . i k ω with w = w i 1 . . . w i k = v i 1 . . . v i k . Therefore, ρ f is an infinite ambiguous faulty run. Given that P(ρ f ) > 0, we deduce that the POpPDA A, Σ o , P is not IF-diagnosable. From Theorem 3.1, it is also neither IA-diagnosable nor FA-diagnosable. Moreover, after reaching the state f e or q e , there is no probabilistic choice. As a consequence the sequence of the correctness proportion of the faulty prefixes of ρ f is stationary. As it is never 0 as seen above, the POpPDA is not AFF-diagnosable.

• Conversely, assume that the PCP instance (n, {v i } i≤n , {w i } i≤n ) has no solution. Let ε > 0, let N ε ∈ N be the integer obtained with our earlier observation. Consider a correct run ρ c with observable length N ε ending in (q e , ⊥ 0 ) and with an occurrence of . Its observed sequence is of the form v i 1 . . . v i k i 1 . . . i k m for some i 1 , . . . , i k , m. Due to the fact that (n, {v i } i≤n , {w i } i≤n ) has no solution, no faulty run can have the same observed sequence. Therefore, ρ c is surely correct. Symmetrically, any faulty run ending in (f e , ⊥ 0 ) by an occurrence of is surely faulty. We thus conclude that, for any ε > 0, there exists N ε ∈ N such that P(FAmb Nε CAmb Nε ) ≤ ε. As a consequence, the POpPDA V, Σ o , P is FA-diagnosable. By Theorem 3.1 it is also IA-diagnosable, IF-diagnosable and AFF-diagnosable.

Diagnosability of probabilistic visibly pushdown automata

As diagnosability is undecidable for pPDA, we now turn to a more restrictive model: probabilistic visibly pushdown automata (pVPA) [START_REF] Alur | Visibly pushdown languages[END_REF]. While keeping a significant expressive power, pVPA is a natural subclass of pushdown automata that is more tractable than the general model and which language has many of the desirable properties that regular languages have. In particular, there exists a method for the determinisation of a non-deterministic visibly pushdown automaton [START_REF] Alur | Visibly pushdown languages[END_REF].

We formally define pVPA and describe how to build a diagnosis-oriented determinisation of a pVPA. Then, we give a decision procedure for diagnosability and study the hardness of the diagnosability problems.

Probabilistic visibly pushdown automata and diagnosis-oriented determinisation

A pVPA is a pPDA where events are partitioned into three sets depending on if they correspond to push, pop, or local transitions.

Definition 5.4. A probabilistic visibly pushdown automaton (pVPA) is a pPDA V = (Q, q 0 , Σ, Γ, δ, P) whose event alphabet is partitioned into local, push and pop events Σ = Σ Σ Σ and such that for every transition T = (q, γ, a, q , w) ∈ δ, T is a local (resp. push, pop) transition iff a is a local (resp. push, pop) event.

A pVPA without the transition probability function P is called a visibly pushdown automata (VPA).

The definitions of pPDA carry on to pVPA in particular the semantics of a pVPA is an infinite-state finitely-branching pLTS.

Example 5.4. Consider the pPDA of Figure 5.2. This pPDA is a pVPA as shown by the partition of events given by Σ = {in}, Σ = {out, abort} and Σ = {serve, empty, reset, f }.

To define partially observable pVPA, we equip a pVPA with a mask function and require that only local events may be unobservable, and that pushes and pops can still be distinguished. As a consequence, given the observed sequence of one run, one can deduce the height of the stack as the difference between pushes and pops, plus one (the bottom symbol).

Definition 5.5. A partially observable pVPA (POpVPA) is a tuple V, Σ o , P consisting of a pVPA V equipped with a mapping P : Σ → Σ o ∪ {ε} such that:

• Σ o = Σ o, Σ o, Σ o, is the set of observations; • P(Σ ) ⊆ Σ o, ∪ {ε}, P(Σ ) ⊆ Σ o, and P(Σ ) ⊆ Σ o, .
When we aimed to verify the diagnosability of finite pLTS in Chapter 4, one of the first step was to build a diagnosis-oriented determinisation of the pLTS (see Definition 4.1, page 91). While this could not be done for pPDA, a determinisation for pVPA was established by Alur and Madhusudan [START_REF] Alur | Visibly pushdown languages[END_REF]. Following the same approach, we now explain how to adapt the determinisation of [START_REF] Alur | Visibly pushdown languages[END_REF] for diagnosability. For a pVPA V, its determinisation is called the estimate VPA of V and is denoted A(V). As in the finite case, we need tags that reflect the category of runs (faulty or correct) given an observed sequence with a distinction between "old" and "young" faulty runs. Due to its technicality, we postpone the formal definition of A(V): we first explain some features of the construction and illustrate them on an example (represented in Figure 5.5).

States and stack symbols. The VPA A(V) tracks all runs with same observation in parallel memorising their status w.r.t. faults. More precisely to the current set of runs corresponds the symbol on the top of the stack which is a set of tuples where each tuple is written as a fraction γ,X,q γ -,X -,q -. Let us describe the meaning of this tuple: • q is the current state of the run and γ is the symbol on the top of its stack;

• X ∈ Tg = {U, V, W} is the status of the run: U for a correct run, V for a young faulty run and W for an old faulty run;

• The denominator (γ -, X -, q -), is related to the configuration just after the last push event of the run: γ -is the stack symbol under the top symbol, while X -is the status of the run reaching this configuration and q -the state of this configuration.

A priori, a single state run would be enough. However the simulation of a pop event in the original VPA is performed in two steps requiring some additional states that we explain later.

Example 5.5. The initial configuration of the VPA A(V) of Figure 5.5 (run, { ⊥ 0 ,U,q 0 ⊥ 0 ,U,q 0 } ) corresponds to the empty run represented by a singleton. The denominator of bottom stack symbols is by convention (⊥ 0 , U, q 0 ) and is irrelevant for specifying the transitions of A(V). Tag updates. Let us explain how the tag X of an item γ,X,q γ -,X -,q -of the current stack symbol is determined. If this item corresponds to a correct run then X = U. When after a transition of A(V) a (tracked) correct run becomes faulty, there are two cases. Either there was no tag W in (the numerators of items of) the top stack symbol of the stack then the run is tagged by W. Otherwise it is tagged by V meaning that it is a young faulty run. The tag V (young) becomes W (old) when, in the previous state, there was no tag W in the top stack symbol. A tag W is unchanged along the run. Local transitions. Given an observed local event o ∈ Σ o, , from the state run with top stack symbol bel, there is a local transition (run, bel, o, run, bel ) in A(V) looping over run that encodes the possible signalling runs with observation o in V. More precisely for every transition sequence (q, α) o = ⇒ (r, β) in V (i.e. a sequence of unobservable local events ended by an event e with P(e) = o) and α,X,q α -,X -,q -∈ bel one inserts β,Y,r α -,X -,q -in bel . The value of Y follows the rules of tag updates.

a X 0 = { ⊥,X,q 0 ⊥,X,q 0 }, a X 1 = { γ,X,q 0 ⊥,X,q 0 }, a X ∞ = { γ,X,q 0 γ,X,q 0 }, b X 1 = { γ,X,q 1 ⊥,X,q 0 }, b X ∞ = { γ,X,q 1 γ,X,q 0 } c X 0 = { ⊥,X,q 1 ⊥,U,q 0 , ⊥,X,f 1 ⊥,U,q 0 }, c X 1 = { γ,X,q 1 ⊥,X,q 0 , γ,X,f 1 ⊥,X,q 0 }, c X ∞ = { γ,X,q 1 γ,U,q 0 , γ,X,f 1 γ,U,q 0 }, X ∈ {U, W} run { U,q 1 ⊥ 0 ,U,q 0 , W,f 1 ⊥,U,q 0 } { W,q 1 ⊥ 0 ,W,q 0 , W,f 1 ⊥,W,q 0 } { U,q 1 γ,U,q 0 , W,f 1 γ,U,q 0 } { W,q 1 γ,W,q 0 , W,f 1 γ,W,q 0 } b U 1 , pop, ε c U 1 , pop, ε a U 0 , ε, c U 0 b W 1 , pop, ε c W 1 , pop, ε a W 0 , ε, c W 0 b U ∞ , pop, ε c U ∞ , pop, ε a U 1 , ε, c U 1 a U ∞ , ε, c U ∞ b W ∞ , pop, ε c W ∞ , pop, ε a W 1 , ε, c W 1 a W ∞ , ε, c W ∞ a X 1 , loc, b X 1 a X ∞ , loc, b X ∞ c X 0 , loc, a X 0 c X 0 , reset, a W 0 a X 0 , in, a X 0 a X 1 a X 1 , in, a X 1 a X ∞ a X ∞ , in, a X ∞ a X ∞ (run, { ⊥ 0 ,U,q 0 ⊥ 0 ,U,q 0 } ) (run, { γ,U,q 0 ⊥ 0 ,U,q 0 } { ⊥ 0 ,U,q 0 ⊥ 0 ,U,q 0 } ) (run, { γ,U,q 0 γ,U,q 0 } { γ,U,q 0 ⊥ 0 ,U,q 0 } { ⊥ 0 ,U,q 0 ⊥ 0 ,U,q 0 } ) (run, { γ,U,q 1 γ,U,q 0 } { γ,U,q 0 ⊥ 0 ,U,q 0 } { ⊥ 0 ,U,q 0 ⊥ 0 ,U,q 0 } ) ({ U,q 1 γ,U,q 0 , W,f 1 γ,U,q 0 }, { γ,U,q 0 ⊥ 0 ,U,q 0 } { ⊥ 0 ,U,q 0 ⊥ 0 ,U,q 0 } ) (run, { γ,U,q 1 ⊥ 0 ,U,q 0 , γ,W,f 1 ⊥ 0 ,U,q 0 } { ⊥ 0 ,U,q 0 ⊥ 0 ,U,q 0 } ) ({ U,q 1 ⊥ 0 ,U,q 0 , W,f 1 ⊥ 0 ,U,q 0 }, { ⊥ 0 ,U,q 0 ⊥ 0 ,U,q 0 } ) (run, { ⊥ 0 ,U,q 1 ⊥ 0 ,U,q 0 , ⊥ 0 ,W,f 1 ⊥ 0 ,U,q 0 } ) (run, { ⊥ 0 ,U,q 0 ⊥ 0 ,U,q 0 } ) (run, { ⊥ 0 ,W,q 0 ⊥ 0 ,U,q 0 } ) in in
Example 5.6. In the VPA A(V) of Figure 5.5 there are several transitions corresponding to the transition (q 0 , γ, serve, q 1 , γ) of V including (run, { γ,U,q 0 γ,U,q 0 }, loc, run, { γ,U,q 1 γ,U,q 0 }). For example, the runs represented in Figure 5.5 use this transition. Push transitions. Given an observed push event o ∈ Σ o, , from the state run with top stack symbol bel, there is a push transition (run, bel, o, run, bel bel ) in A(V) looping over run that encodes the possible signalling runs with observation o in V. More precisely for every transition sequence (q, α) o = ⇒ (r, β -β) in V and α,X,q α -,X -,q -∈ bel one inserts β -,Y,r α -,X -,q -in bel and β,Y,r β -,Y,r in bel . The value of Y follows the rules of tag updates.

Example 5.7. In Figure 5.5 several transitions of A(V) correspond to the transition (q 0 , ⊥ 0 , in, q 0 , ⊥ 0 γ) of V, including (run, { ⊥ 0 ,U,q 0 ⊥ 0 ,U,q 0 }, in, run, { ⊥ 0 ,U,q 0 ⊥ 0 ,U,q 0 }{ γ,U,q 0 ⊥ 0 ,U,q 0 }) and several transitions of A(V) correspond to the transition (q 0 , γ, in, q 0 , γγ) of V, including (run, { γ,U,q 0 ⊥ 0 ,U,q 0 }, in, run, { γ,U,q 0 ⊥ 0 ,U,q 0 }{ γ,U,q 0 γ,U,q 0 }). Here, the specification of the tag updates is straightforward since it does not involve faulty runs. The runs represented in Figure 5.5 use these two transitions from the initial state.

Pop transitions. Given an observed pop event o ∈ Σ o, , from the state run with top stack symbol bel, the "pop operation" is performed by a sequence of two transitions: a pop transition labelled by o reaching another state that contains some information. This information is then used by the next (local) transition labelled by ε to move back to state run with a consistent stack symbol. Given an intermediate stack symbol, there is exactly one possible such transition. Thus despite these transitions, A(V) is still deterministic. The first transition (run, bel, o, , ε) in A(V) is specified as follows. The next state is a set of items of the following shape X,q α -,X -,q -. More precisely for every transition sequence (q, α) o = ⇒ (r, ε) in V (i.e. a sequence of unobservable local events ended by an event e with P(e) = o) and α,X,q α -,X -,q -∈ bel one inserts Y,r α -,X -,q -in . The value of Y follows the rules of tag updates. A transition ( , bel, ε, run, bel ) is specified as follows. For every X ,q γ,X,q in and γ,X,q γ -,X -,q -in bel (i.e. the denominator of the first fraction and the numerator of the second fraction match), one inserts γ,X ,q γ -,X -,q -in bel .

Example 5.8. Let us describe how the pop is performed by two transitions in the runs of the VPA of Figure 5.5 from the state reached after event serve. From q 1 with γ as top of the stack there are two transitions whose observation is pop: (q 1 , γ, out, q 1 , ε) and (q 1 , γ, abort, f 1 , ε). Thus starting from run with top stack symbol { γ,U,q 1 γ,U,q 0 }, one reaches state = { U,q 1 γ,U,q 0 , W,f 1 γ,U,q 0 }. The faulty run is tagged with W as there was no tag W in the former top stack symbol. In the next configuration, the top stack symbol is { γ,U,q 0 ⊥ 0 ,U,q 0 }. So the transition labelled by ε moves back to state run with updated top stack symbol { γ,U,q 1 ⊥ 0 ,U,q 0 , γ,W,f 1 ⊥ 0 ,U,q 0 }.

We now give the definition of the estimate VPA A(V) associated with a given POpVPA V. Let µ ∈ {g, c, f } we write (q, γ) o = ⇒ µ (q , w) with o ∈ Σ o if when µ = g (resp. c, f ), there exists a (resp. correct, faulty) run of transitions starting from (q, γ) to (q , w) such that all transitions are unobservable except the last one labelled by e with P(e) = o. Let ρ be such a run then we also write (q, γ) ρ = ⇒ µ (q , w) All transitions of such runs are local except the last one whose type depends on the type of o.

Definition 5.6. Given V, P, Σ o a POpVPA with V = (Q, Σ, Γ, δ, P), its estimate VPA is the deterministic VPA A(V) = (Q e , Σ o , Γ e , δ e ) defined by:

• Q e = {run} (2 Γ×(Tg×Q) 2 \ ∅) is the set of states with initial state q e 0 = run;

• Γ e = 2 (Γ×Tg×Q) 2 \ ∅ is the stack alphabet with set of bottom stack symbols Γ e ⊥ = 2 Init \ ∅ where Init = { ⊥ 0 ,X,q ⊥ 0 ,U,q 0 | (X, q) ∈ Tg × Q} and initial stack symbol ⊥ e 0 = { q 0 ,U,⊥ 0 q 0 ,U,⊥ 0 };

• The transition relation δ e is defined as follows.

local transitions (run, bel , o, run, bel ) ∈ δ e if:

• β,U,r α -,U,q -∈ bel iff there exists α,U,q α -,U,q -∈ bel and (q, α)

o = ⇒ c (r, β).
• If W occurs in bel , β,W,r α -,X,q -∈ bel iff there exists α,W,q α -,X,q -∈ bel and (q, α)

o = ⇒ g (r, β). • If W occurs in bel , β,V,r
α -,X,q -∈ bel iff (1) there exists α,U,q α -,U,q -∈ bel and (q, α)

o = ⇒ f (r, β) or
(2) there exists α,V,q α -,X,q -∈ bel and (q, α) o = ⇒ g (r, β).

• If W does not occur in bel , β,W,r α -,X,q -∈ bel iff (1) there exists α,U,q α -,X,q -∈ bel and (q, α)

o = ⇒ f (r, β) or
(2) there exists α,V,q α -,X,q -∈ bel and (q, α)

o = ⇒ g (r, β).
push transitions (run, bel , o, run, bel bel ) ∈ δ e if:

• β -,U,r α -,U,q -∈ bel and β,U,r β -,U,r ∈ bel iff there exists α,U,q α -,U,q -∈ bel and (q, α)

o = ⇒ c (r, β -β).
• If W occurs in bel , β -,W,r α -,X,q -∈ bel and β,W,r β -,W,r ∈ bel iff there exists α,W,q α -,X,q -∈ bel and (q, α)

o = ⇒ g (r, β -β). • If W occurs in bel , β -,V,r
α -,X,q -∈ bel and β,V,r β -,V,r ∈ bel iff (1) there exists α,U,q α -,U,q -∈ bel and (q, α)

o = ⇒ f (r, β -β) or
(2) there exists α,V,q α -,X,q -∈ bel and (q, α)

o = ⇒ g (r, β -β).
• If W does not occur in bel , β -,W,r α -,X,q -∈ bel and β,W,r β -,W,r ∈ bel iff (1) there exists α,U,q α -,U,q -∈ bel and (q, α)

o = ⇒ f (r, β -β) or
(2) there exists α,V,q α -,X,q -∈ bel and (q, α)

o = ⇒ g (r, β -β).
pop transitions (run, bel , o, , ε) ∈ δ e with ∈ Q e \ {run} if:

• U,r α -,U,q -∈ iff α,U,q α -,U,q -∈ bel and (q, α) o = ⇒ c (r, ε).
• If W occurs in bel , W,r α -,X,q -∈ iff there exists α,W,q α -,X,q -∈ bel and (q, α)

o = ⇒ g (r, ε). • If W occurs in bel , V,r
α -,X,q -∈ iff (1) there exists α,U,q α -,U,q -∈ bel and (q, α)

o = ⇒ f (r, ε) or
(2) there exists α,V,q α -,X,q -∈ bel and (q, α) o = ⇒ g (r, ε).

• If W does not occur in bel , W,r α -,X,q -∈ iff (1) there exists α,U,q α -,U,q -∈ bel and (q, α)

o = ⇒ f (r, ε) or
(2) there exists α,V,q α -,X,q -∈ bel and (q, α)

o = ⇒ g (r, β -β). ε-transitions ( , bel , ε, run, bel ) ∈ δ e if:
α,X ,r α -,X -,q -∈ bel iff there exists α,X,q α -,X -,q -∈ bel and X ,r α,X,q ∈ .

We say that a configuration is stable if its associated state is run.

Example 5.9. Let us explain the runs given in the Figure 5.5. It starts in the initial configuration (run, { ⊥ 0 ,U,q 0 ⊥ 0 ,U,q 0 } ) which represents the empty run. From q 0 there exists only one path of observation in in the POpVPA. As this path is correct, by reading in on the estimate VPA we reach (run, { γ,U,q 0 ⊥ 0 ,U,q 0 } { ⊥ 0 ,U,q 0 ⊥ 0 ,U,q 0 } ). The new element of the stack { γ,U,q 0 ⊥ 0 ,U,q 0 } means that the stack of the possible run has head γ and its current state is q 0 after a correct run, moreover the run entered q 0 when it pushed this γ and it does not have a second non-terminal element in our stack. Reading a second in is still doable by a single run, we reach (run, { γ,U,q 0 γ,U,q 0 } { γ,U,q 0 ⊥ 0 ,U,q 0 } { ⊥ 0 ,U,q 0 ⊥ 0 ,U,q 0 } ) which modifies one information compared to before: we know from the bottom part of the head stack that the stack has at least a second γ.

Reading a serve then is possible as there exists a correct signalling run from q 0 to q 1 with observation serve. The estimate VPA modifies the head stack so as to represent that the run we follow is now in q 1 but without modifying anything else.

Reading a pop event is more involved: from q 1 with head of stack γ, triggering a pop can be done by a correct run staying in q 1 or by a faulty run going in f 1 . To represent this and the popping of the stack, we go in two steps. In the first step, we go to the state { U,q 1 γ,U,q 0 , W,f 1 γ,U,q 0 } which keeps the information of the two possibilities of current configuration and we pop the stack. In the second step, we deterministically take an ε transition that transfer this information from the state to the stack. In order to transfer the information, the estimate VPA checks which of the current possible runs (represented by U,q 1 γ,U,q 0 and W,f 1 γ,U,q 0 ) corresponds to each of the new head of stack. This is done by comparing the bottom part of the run with the top part of the head of stack, here γ, U, q 0 in every cases. Reading a second pop realises a similar process reaching (run, { ⊥ 0 ,U,q 1 ⊥ 0 ,U,q 0 , ⊥ 0 ,W,f 1 ⊥ 0 ,U,q 0 } ). An empty would lead to (run, { ⊥ 0 ,U,q 0 ⊥ 0 ,U,q 0 } ) as there is a correct run from q 1 to q 0 labelled by empty but no run from f 1 with such label. Similarly a reset cannot be taken from q 1 but it can be read from f 1 , thus we reach (run, { ⊥ 0 ,W,q 0 ⊥ 0 ,U,q 0 } ).

The following proposition links runs of V and observed sequences of A(V).

Proposition 5.1. Let σ be an observed sequence of A(V) and ρ * be its corresponding finite run with successive stable configurations (run, w 0 ) . . . (run, w n ). Let w n = bel 1 . . . bel h and for i < n, bel (i) be the top stack symbol of w i . Then:

• For all γ h ,X h ,q h γ h-1 ,X h-1 ,q h-1 ∈ bel h , there exists a sequence ( γ i ,X i ,q i γ i-1 ,X i-1 ,q i-1 ) 0<i<h such that for all i, γ i ,X i ,q i γ i-1 ,X i-1 ,q i-1 ∈ bel i and a signalling run ρ of V such that P(ρ) = σ that reaches configuration (q h , γ 1 . . . γ h ). In addition:

• if X h = U then ρ may be chosen correct;

• if X h = U then ρ may be chosen faulty;

• if X h = W then there exists 0 < k ≤ n, such that ρ ↓k is faulty and W does not occur in bel (k-1) .

• Conversely, let ρ be a signalling run of V such that P(ρ) = σ reaching configuration (q h , γ 1 . . . γ h ), there exists a sequence ( γ i ,X i ,q i γ i-1 ,X i-1 ,q i-1 ) 0<i≤h such that for all i, γ i ,X i ,q i γ i-1 ,X i-1 ,q i-1 ∈ bel i . In addition:

• if ρ is correct then X h = U;
• if ρ is faulty then X h = U;

• if there exists 0 < k ≤ n, such that ρ ↓k is faulty and W does not occur in bel (k-1) then X h = W.

The difficulty of this proof is the number of cases that have to be studied: what is the tag and which kind of transition (local, push or pop) is considered. As a consequence, we only detail the most involved case (when the event is a pop and the tag is W. The result is obtained by induction on the size of the observed sequence and mostly consists in understanding the definitions of A(V) and especially of the tag updates.

Proof. We prove the result by induction on |σ|. The basis case is straightforward. For the inductive step, we only detail the most involved case: σ[n] ∈ Σ o, . For the properties related to tags, we only detail the ones related to W.

Denote σ = σ[1] . . . σ[n -1] and w n-1 = bel 1 . . . bel h bel h+1 . • Let γ h ,X h ,q h γ h-1 ,X h-1 ,q h-1 ∈ bel h . By construction, there exists γ h+1 ,X h+1 ,q h+1 γ h ,X h ,q h ∈ bel h+1 with γ h = γ h , a signalling run (q h+1 , γ h+1 ) ρ = ⇒ (q h , ε) with proj(ρ ) = σ[n], γ h ,X h ,q h γ h-1 ,X h-1 ,q h-1 ∈ bel h where (γ h-1 , X h-1 , q h-1 ) = (γ h-1 , X h-1 , q h-1
) and X h is obtained by updating X h+1 w.r.t. bel h+1 and ρ . In particular if X h = W then:

(1) X h+1 = W, or (2) W does not occurs in bel h+1 and (a) X h+1 = V or (b) X h+1 = U and ρ is faulty.

By inductive hypothesis, there exists a sequence (

γ i ,X i ,q i γ i-1 ,X i-1 ,q i-1
) 0<i≤h such that for all i,

γ i ,X i ,q i γ i-1 ,X i-1 ,q i-1
∈ bel i and a signalling run ρ of V such that P(ρ ) = σ reaching configuration (q h+1 , γ 1 . . . γ h+1 ). Consider the signalling run ρ = ρ ρ ; it reaches configuration (q h , γ 1 . . . γ h ). Since for all i < h, bel i = bel i , the sequence (

γ i ,X i ,q i γ i-1 ,X i-1 ,q i-1
) 0<i<h and the run ρ are appropriate. The three additional properties follow from the rules of tag updates. In particular, if X h = W, then:

• the assertion (1) holds and then the property comes from the inductive hypothesis, or • the assertion (2) holds which implies that W does not occur in bel h+1 and ρ is faulty.

• Let ρ be a signalling run of V such that P(ρ) = σ which reaches configuration (q h , γ 1 . . . γ h ). Let us write ρ = ρ ↓n-1 ρ with (q h+1 , γ h+1 ) ρ = ⇒ (q h , ε). By the inductive hypothesis, there exists a sequence (

γ i ,X i ,q i γ i-1 ,X i-1 ,q i-1
) 0<i≤h+1 such that for all i,

γ i ,X i ,q i γ i-1 ,X i-1 ,q i-1 ∈ bel i and for all i ≤ h, γ i = γ i . By construction, γ h ,X h ,q h γ h-1 ,X h-1 ,q h-1
∈ bel h for some X h . Since bel i = bel i for all i < h, we obtain the required sequence of items.

The three additional properties follow from the rules of tag updates. In particular, assume there exists 0 < k ≤ n, such that ρ ↓k is faulty and W does not occur in bel k-1 .

• If ρ ↓n-1 is correct then, as ρ is faulty, ρ is faulty and W does not occur in bel n-1 = bel h+1 . So by construction X h = W.

• If ρ ↓n-1 is faulty then:

• either X h+1 = W and by construction X h = W,

• or X h+1 = V. By induction hypothesis there does not exist 0 < k ≤ n -1, such that ρ ↓k is faulty and W does not occur in bel k-1 . So W does not occur in bel n-1 = bel h+1 . Therefore X h = W.

Decidability of diagnosability for POpVPA

Our goal in this subsection is to use the characterisations from the Section 3 of Chapter 3 to decide the diagnosability of POpLTS generated by POpVPA. To do so, we face the difficulty that the Borel sets that characterise IF-, IAand FF-diagnosability are not a priori regular, even in the finitely-branching case. Yet, for POpVPA, we circumvent this problem, and manage to specify these sets by pLTL formula on a product of the POpVPA with its estimate VPA, the tags are used to define the atomic propositions. The decidability of the qualitative model checking for recursive probabilistic systems [START_REF] Etessami | Model checking of recursive probabilistic systems[END_REF] then yields the decidability of the above three diagnosability notions for POpVPA. The first step is to build the product of the POpVPA and its estimate VPA. From this point on, we assume that the set of states of the POpVPA is separated between correct and faulty states Q = Q c ∪ Q f . This can be done without loss of generality: the transformation ensuring this is similar to the one shown for pLTS in Section 1.4 of Chapter 2. We build V A(V) = V × A(V) the product automaton of V and A(V) synchronised on the alphabet of observed events Σ o . The transitions of V labelled by unobservable events do not change the second component of the state and the transitions of A(V) labelled by ε do not change the first component of the state. Due to the determinism of A(V), V A(V) has the same probabilistic behaviour as the one of V except that it memorises additional information along the run. More precisely, let ρ be a run of V, then ρ, a run of V A(V) , is obtained from ρ by following the same transitions and adding the single transition firable after any pop transition. One immediately gets P V A(V) (ρ) = P V (ρ). Formally we have: Definition 5.7. Given V, P, Σ o a POpVPA V = (Q, Σ, Γ, δ, P) and its estimate VPA A(V) = (Q e , run, Σ o , Γ e , δ e ), their synchronised product is the pVPA V A(V) = (Q A , Σ ∪ { }, Γ A , δ A , P A ) where:

• Q A = Q × Q e is
the set of states with initial state q A 0 = (q 0,c , run);

• Γ A = Γ × Γ e is the stack alphabet with Γ ⊥ × Γ e
⊥ the set of bottom stack symbols and ⊥ A 0 = (⊥ 0 , ⊥ 0 ,U,q 0 ⊥ 0 ,U,q 0 ) the initial symbol;

• The transition relation δ A consists of: local transitions.

• For all (q, γ, a, q , γ ) ∈ δ with a unobservable and bel ∈ Γ e , we have ((q, run), (γ, bel ), a, (q , run), (γ , bel )) ∈ δ A ;

• For all (q, γ, a, q , γ ) ∈ δ and (run, bel, o, run, bel ) ∈ δ e with P(a) = o, we have ((q, run), (γ, bel ), a, (q , run), (γ , bel )) ∈ δ A ;

• For all ( , bel , ε, run, bel ) ∈ δ e , q ∈ Q and γ ∈ Γ, we have ((q, ), (γ, bel ), , (q, run), (γ, bel )) ∈ δ A ; push transitions.

• For all (q, γ, a, q , γ γ ) ∈ δ and (run, bel, o, run, bel bel ) ∈ δ e with P(a) = o, we have ((q, run), (γ, bel ), a, (q , run), (γ , bel )(γ , bel )) ∈ δ A ; pop transitions.

• For all (q, γ, a, q , ε) ∈ δ and (run, bel, o, , ε) ∈ δ e with P(a) = o, we have ((q, run), (γ, bel ), a, (q , ), ε) ∈ δ A ;

• The transition probability function P A is defined by:

-P A ((q, run), (γ, bel ), a, (q , run), (γ , bel )) = P(q, γ, a, q , γ ); -P A ((q, run), (γ, bel ), a, (q , run), (γ , bel )(γ , bel )) = P(q, γ, a, q , γ γ ); -P A ((q, run), (γ, bel ), a, (q , ), ε) = P(q, γ, a, q , ε);

for ∈ Q e \ {run}, P A ((q, ), (γ, bel ), , (q, run), (γ, bel )) = 1.

Example 5.10. The product POpVPA contains the current run of the POpVPA and the information given by the estimate POpVPA. Let us consider the faulty run given as example in the Figure 5.2. This POpVPA does not follow the separation between correct and faulty states. Here we write q c (resp. q f ) if the state q was reached by a correct (resp. faulty) run. After reading in, we are in state (q 0,c , run) meaning that the state of the possible run is q 0 , it was reached by a correct run and our estimate VPA is in state run, the head of stack is (γ, { γ,U,q 0,c ⊥ 0 ,U,q 0,c } { ⊥ 0 ,U,q 0,c ⊥ 0 ,U,q 0,c } ), meaning our real head is γ and the rest is the head of the estimate VPA. If we follow the faulty run until after the first pop, we reach the state (f 1,f , { U,q 1 γ,U,q 0,c , W,f 1,f γ,U,q 0,c }), we are thus in f 1 with a faulty run and the estimate VPA is in one of the temporary states. In order to leave this state, we read a which leads to the state (f 1,f , run). is an event only affecting the part of the POpVPA corresponding to the estimate VPA, allowing it to realise the ε transition that follows the observation of a pop event.

Given a finite run ρ of V, we inductively define the run ρ of V A(V) as follows. First (q 0 , ⊥ 0 ) = (q A 0 , ⊥ A 0 ). Let ρ of length n ≥ 1, a ∈ Σ and q ∈ Q and γ 1 , . . . , γ h ∈ Γ such that ρ = ρ a(q, γ 1 . . . γ h ). If a ∈ Σ then ρ = ρ a((q, run), (γ 1 , bel 1 ) . . . (γ h , bel h )) where (run, bel 1 . . . bel h ) is the configuration reached by P(ρ) in A(V). If a ∈ Σ then ρ = ρ a((q, ), (γ 1 , bel 1 ) . . . (γ h , bel h )) ((q, run), (γ 1 , bel 1 ) . . . (γ h-1 , bel h-1 )(γ h , bel h )) where ( , bel 1 . . . bel h ) is the configuration reached by P(ρ) in A(V) and (run, bel 1 . . . bel h-1 bel h ) is the single next configuration reached by an ε transition. As previously observed, P(ρ) = P(ρ).

In order to prove decidability of diagnosability for a POpVPA V, one wants to check whether the formulae characterising diagnosability defined in Chapter 3 hold on V. Let us first recall the relevant results of Chapter 3. We defined three path formulae:

• f: for every run ρ, f(ρ) = true if ρ is faulty;

• U: for every run ρ, U(ρ) = true if there exists a correct signalling run ρ with P(ρ) = P(ρ );

• W: W(ε) = false and W(q 0 a 0 . . . q n+1 ) = true if firstf(P(q 0 a 0 . . . q n+1 )) = firstf(P(q 0 a 0 . . . q n )) < ∞

where firstf(σ) = min{k | ∃ρ signalling run P(ρ) = σ ∧ ρ ↓k is faulty} with the convention that min(∅) = ∞.

Using these path formulae, given a pLTS A, we obtained the following results:

• A is FF-diagnosable iff A |= P =0 (32(f ∧ U)); • if A is finitely branching, A is IA-diagnosable iff A |= P =0 (32(U ∧ W)).
The paths formulae f, U and W depends on the past of the run and not only on the current configuration. We therefore transform the pathL formulae into pLTL properties that are checked on V A(V) . First, for each path formula we define an atomic propositions on the pairs ((q, run)(γ, bel )) consisting of a state of V A(V) together with a top stack contents.

Definition 5.8. Let bel ⊆ 2 (Γ×Tg×Q) 2 , we say that the tag X occurs in bel if there exists γ,X,q γ -,X -,q -∈ bel .

The atomic propositions ν f , ν u and ν w corresponding to the path formulae f, U and W are defined by: • ν f ((q, run)(γ, bel )) = true if and only if q ∈ Q f ;

• ν u ((q, run)(γ, bel )) = true if and only if U occurs in bel ;

• ν w ((q, run)(γ, bel )) = true if and only if W occurs in bel .

We extend ν f , ν u and ν w over configurations cf = ((q, ), w)) with = run by

ν f (cf ) = ν u (cf ) = ν w (cf ) = true.
The atomic propositions ν f and ν u perfectly reflect the paths formula f and U, and ν w is eventually forever true if and only if W is.

Proposition 5.2. Let ρ be an infinite run of V. Then:

• For all k ∈ N, f(ρ ↓k ) ⇔ ν f (last(ρ ↓k )) and U(ρ ↓k ) ⇔ ν u (last(ρ ↓k )); • ρ |= 32W ⇔ ∃K∀k ≥ K. ν w (last(ρ ↓k )) = true.
The second component of V A(V) representing A(V), one can use the results of Proposition 5.1 to link the tags and the runs associated with the observed sequence. This is what we do here. The most complicated (and interesting) case being the link between W and W. The idea is the following. When the tag W disappears after following an observation in A(V), let n be the observed length of the last time W was not tagging any state, then the oldest fault in the current run occurred after the n'th observation. Thus every time W is not present, the longest prefix of the run that is surely correct increased, ensuring that W is f alse. Of course, W can be f alse more often than the absences of W. However, if after the n'th observation, for n ∈ N, W always tag a state of the belief, it means that there exists a run consistent with the observation for which a fault occurred at most at the n'th step. Therefore firstf is bounded, which implies that W will eventually become forever true.

Proof. First, note that f and ν f obviously coincide: they both express that a fault occurred.

To prove the second item, about U and ν u , we use the link between observed sequences and the tag U in V A(V) . Let σ be an observed sequence triggered by a run of V. Then bel σ is the top stack symbol of the stable configuration in A(V) reached by the run accepting σ (so ending by an ε-transition if the last event is a pop event). Due to Proposition 5.1, U occurs in bel σ iff there is a correct signalling run of V with observed sequence σ. According to the definition of ν u , we thus deduce that for any finite signalling run ρ of V, ν u (last(ρ)) = true iff U(ρ) = true. We now establish the link between W and ν W . To show the left-to-right implication, let ρ ∈ Ω and K 0 ∈ N be such that ρ, K 0 |= 2W. By definition of W, firstf(P(ρ ↓k )) is constant and bounded by K 0 for k ≥ K 0 . For all k ∈ N, let bel k be the top stack symbol reached in A(V) after reading the observed sequence P(ρ ↓k ). If for all k ≥ K 0 , W occurs in bel k , then for all k ≥ K 0 , ν w (last(ρ ↓k )) = true. Otherwise there exists K 1 ≥ K 0 such that W does not occur in bel K 1 . Let k > K 1 , as firstf(P(ρ ↓k )) ≤ K 0 , there exists a faulty run ρ of V A(V) such that P(ρ ) = P(ρ ↓n ) and ρ ↓K 0 is faulty. W does not occur in bel K 1 and ρ ↓K 1 +1 is faulty. Thus by Proposition 5.1, W occurs in bel k . Therefore for all n > K 1 , ν w (last(ρ ↓n )) = true. Let us show the right-to-left implication. Let ρ ∈ Ω and K ∈ N be such that for all k ≥ K, ν w (last(ρ ↓k )) = true. By definition of ν w for all k ≥ K, W occurs in bel k (defined as above). Let k ≥ K, by Proposition 5.1, there exists a run ρ of V A(V) such that P(ρ ) = P(ρ ↓k ) and there exists n ≤ k such that ρ ↓n is faulty and W does not occur in bel n-1 . Thus n ≤ K. Therefore for all k ≥ K, firstf(P(ρ ↓k )) ≤ K. Since beyond K, firstf is bounded, it is non decreasing and then ultimately constant. Let K such that for all k ≥ K , firstf(P(ρ ↓k )) = firstf(P(ρ ↓k-1 )). So ρ, K |= 2W and thus ρ |= 32W.

Thanks to the relationships between the path formulae, and the atomic propositions, and using the characterisations from Section 3 of Chapter 3, we reduce the FF-, IFand IA-diagnosability to the model checking of a pLTL formula on the product VPA V A(V) . Model checking qualitative pLTL for probabilistic pushdown automata is doable in polynomial space in the size of the model [START_REF] Etessami | Model checking of recursive probabilistic systems[END_REF]. In our case, V A(V) is exponential in the size of V. We thus obtain the decidability and a complexity upper-bound for the diagnosability problems for POpVPA.

Theorem 5.3. FF-diagnosability, IF-diagnosability and IA-diagnosability are decidable in EXPSPACE for POpVPA.

Proof. Thanks to the Propositions 5.2 and 5.1 and the characterisations of Propositions 3.7 (page 80) and 3.8 (page 81), we can derive pLTL characterisations of diagnosability for POpVPA. Namely, for V a POpVPA, as V and V A(V) have the same probabilistic behaviour,

• V is FF-diagnosable iff V A(V) |= P =0 (32(ν f ∧ ν u )); • V is IA-diagnosable iff V A(V) |= P =0 (32(ν u ∧ ν w )).
Moreover, since POpPDA have finitely many states, the POpLTS they generate are finitely-branching. Therefore, IF-diagnosability coincides with FF-diagnosability according to Theorem 3.1. The two above qualitative pLTL formulae can be checked on probabilistic pushdown automata thanks to [START_REF] Etessami | Recursive Markov chains, stochastic grammars, and monotone systems of nonlinear equations[END_REF]. More precisely, one can transform V A(V) into a recursive Markov chain (the transformation is linear) [START_REF] Etessami | Model checking of recursive probabilistic systems[END_REF]. Then, the model checking of qualitative pLTL on recursive Markov chains is doable in PSPACE in the size of the Recursive Markov Chain and EXPTIME in the size of the formulae [START_REF] Etessami | Recursive Markov chains, stochastic grammars, and monotone systems of nonlinear equations[END_REF]. In our case, the product VPA V A(V) is exponential in the size of V and the size of the formulae is constant. This yields an EXPSPACE algorithm for checking diagnosability of POpVPA.

EXPTIME-hardness of the diagnosability for POpVPA

While the notions of diagnosability we studied in the previous section are decidable in EXPSPACE (Theorem 5.3), this is not necessarily optimal. Here, we only show an EXPTIME lower bound on the complexity. This lower bound is obtained by reducing the universality problem for VPA, which is known to be EXPTIME-complete [START_REF] Alur | Visibly pushdown languages[END_REF]. This reduction also applies to FA-diagnosability for which the decidability status is unknown. Theorem 5.4. FF-, IF-, FAand IA-diagnosability are EXPTIME-hard for POpVPA.

Proof. Let us start with FF-diagnosability. The proof is done by reduction from the universality problem for VPA, which is known to be EXPTIME-hard [START_REF] Alur | Visibly pushdown languages[END_REF]. Recall the universality problem for VPA: given a VPA A and a set of final states Q F , do we have

P({ρ ∈ SR | last(ρ) ∈ Q F }) = Σ ω o ?
Starting from a VPA A we build a pVPA V (see Figure 5.6) with two components: one correct and one faulty, both reachable in one step from the initial state. The correct component is a copy of A with a positive probability of making a reset (emptying its stack and going back to the initial state of A) in a final state. Every reset starts by a new observable event ,followed by some pop event and ends by a second . In the faulty component, one can read any observation of Σ * o and also has the possibility to produce and in a way that mimics a reset. If a is read, then the faulty component triggers some and a as would be done in the correct component. This way, the observation associated with a reset does not give any information on the correctness of a system. What matters is after which observed sequence can a reset occur. If an observed sequence cannot end in an accepting state of A, then in a faulty run, with probability 1 this observed sequence will be read in between two resets, revealing the fault. Reciprocally, if A is universal, everything that can be observed on a faulty run can also be observed in the correct component establishing that V is not diagnosable.

Formally, from a VPA A = (Q, Σ, Γ, δ) and a subset of accepting states Q F ⊆ Q, we build a pVPA V = (Q , Σ , Γ , δ , P ) as follows:

• Q = Q ∪ {f 0 , f , q 0 , q } and q 0 is the initial state;

• Σ = Σ {f , u, , }; • Γ = Γ {B} and Γ ⊥ = Γ ⊥ ;
• Writing δ , resp. δ and δ for the set of local resp. push and pop transitions of V, δ consists of the following transitions:

local δ ∪ {(q 0 , ⊥ 0 , u, ⊥ 0 , q 0 ), (q 0 , ⊥ 0 , f , ⊥ 0 , f 0 ), (f 0 , γ, , γ, f ) | γ ∈ Γ ∪ {⊥ 0 }} ∪ {(q, γ, , γ, q ) | q ∈ Q F , γ ∈ Γ ∪ {⊥ 0 }} ∪ {(f 0 , γ, a, γ, f 0 ) | a ∈ Σ , γ ∈ {B, ⊥ 0 }} ∪ {(q , ⊥ 0 , , ⊥ 0 , q 0 ), (f , ⊥ 0 , , ⊥ 0 , f 0 )}; push δ ∪ {(f 0 , γ, a, γB, f 0 ) | a ∈ Σ , γ ∈ {B, ⊥ 0 }}; pop δ ∪ {(f 0 , B, a, ε, f 0 ) | a ∈ Σ } ∪ {(f , B, , ε, f )} ∪ {(q , γ, , ε, q ) | γ ∈ Γ};
• P is such that for every γ ∈ Γ, P (f 0 , γ, , γ, f ) = 1 2 , and assigns arbitrary positive probabilities to the other transitions in δ .

We further consider the POpVPA V , Σ o , P with Σ o = Σ ∪ { , } and the masking function satisfies P(u) = P(f ) = ε and P(x) = x for any other event x ∈ Σ . This construction is illustrated in Figure 5.6. The figure uses the following shortcuts:

a ∈ Σ , a ∈ Σ , a ∈ Σ , γ ∈ Γ, γ ∈ {B, ⊥ 0 } and z ∈ Γ \ {⊥ 0 }. q 0 f 0 q 0 f q q f ⊥ 0 , f , ⊥ 0 B, a , ε γ , a , γ γ , a , γ B ⊥ 0 , u, ⊥ 0 γ , , γ B, , ε ⊥ 0 , , ⊥ 0 γ, , γ z, , ε ⊥ 0 , , ⊥ 0 V Figure 5.6: A POpVPA for the EXPTIME-hardness of FF-diagnosability.
The observed sequences corresponding to correct runs in V , Σ o , P are either of the form w 1 k 1 w 2 . . . k n-1 w n or of the form w 1 k 1 w 2 . . . w n-1 m . In these decompositions, w i , for i < n, is a sequence corresponding to a run of V starting in q 0 and ending in some accepting state q f ∈ Q F , k i is the number of elements in the stack after reading w i in V and also in V (apart from the bottom stack symbol ⊥ 0 ), w n is the sequence associated with a run of V starting in q 0 , and m is at most the number of elements in the stack after reading w n-1 in V. Note that k i only depends on w i , and does not depend on the exact run over w i , since V is a VPA. The observed sequences corresponding to faulty runs in V , Σ o , P are less constrained. They are of one of the two forms presented above, however the words w i for i ≤ n can be any word of Σ * .

Let us show that V is not universal if and only if V , Σ o , P is FF-diagnosable. First assume that V is not universal. Then there exists a word w ∈ Σ * such that no run of V reading w ends in an accepting state q f . However, the observed sequence of any faulty run almost-surely contains the factor w . Indeed, faulty runs almost surely visit infinitely often the configuration (f , ⊥ 0 ), and from there, the probability λ to read w is positive. Let ρ be an infinite faulty run. Its observed sequence is of the form P(ρ) = w 1 k 1 w 2 k 2 w 3 . . . with k i ≤ |w i | for every i. If there exists i ≤ n such that w i = w then ρ is surely faulty, since it has no corresponding correct run. The latter statement can be refined. For n ≥ |w|, if, for every i ≤ n, |w i | ≤ n and there exists i ≤ n such that w i = w then ρ ↓2n 2 +n is surely faulty. Indeed, |w i k i | ≤ 2n + 1, w occurs at the latest for i = n, and once it occurs the prefix is surely faulty. Let us therefore consider faulty runs that do not satisfy this property. We let

Avoid n = {ρ ∈ F | P(ρ) = w 1 k 1 w 2 k 2 w 3 • • • ∧ (∀i ≤ n w i = w ∨ ∃i ≤ n |w i | > n)}.
By construction, FAmb 2n 2 +n ⊆ Avoid n . Moreover, using standard union-sum inequalities,

P(Avoid n ) ≤ (1 -λ) n + n
2 n (recall that λ is the probability to read w from (f 0 , ⊥ 0 )). Thus lim n→∞ P(Avoid n ) = 0 and hence lim n→∞ P(FAmb n ) = 0 so that V , Σ o , P is FF-diagnosable. Assume now that V is universal. Let ρ be an infinite surely faulty run of V , Σ o , P . We write ρ for the greatest ambiguous prefix of ρ and a ∈ Σ o ∪ { , } such that ρ a is again a prefix of ρ. Observe that a cannot be since the number of 's between two 's, whether on the left or right-hand-side of V , is entirely determined by the word of Σ * o read before the first . For the same reason, if a = , P(ρ ) ends with a word of Σ * o (i.e. the number of 's in P(ρ ) is even). Let w be the greatest suffix of P(ρ ) contained in Σ * o . If a = , we deduce that there is no run starting in q 0 with observed sequence w and ending in an accepting state of V. Therefore, V is not universal. Similarly, if a ∈ Σ o , then there is no run starting in q 0 and with observed sequence wa. In that case also, V is not universal. We hence conclude that there is no infinite surely faulty run in V , Σ o , P . As the probability to generate faulty runs is positive, this implies that V , Σ o , P is not IF-diagnosable. Now, IF-diagnosability is equivalent to FF-diagnosability for finitely-branching POpLTS (see Theorem 3.1), and so V , Σ o , P is not FF-diagnosable.

Let us now argue for the EXPTIME-hardness of FA-diagnosability and IA-diagnosability. The reduction is very similar to the previous one: it only requires an additional state in the correct component that ensures that almost surely any correct run will be identified as being correct. Therefore the problem may only come from the faulty runs which are dealt with exactly as above. From the VPA V = (Q, Σ, Γ, δ) and pVPA V = (Q , Σ , Γ , δ ) defined above, we construct a pVPA V = (Q , Σ , Γ , δ , P ) such that

• Q = Q ∪ {q c }
and q 0 is the initial state;

• Σ = Σ ∪ {f , u, , α};

• Γ = Γ; • δ = δ ∪ {(q, α, γ, q c ) | γ ∈ Γ ∪ {⊥ 0 }, q ∈ Q ∪ {q c }};
• P assigns arbitrary positive probabilities to transitions in δ .

We further consider the POpVPA V , Σ o , P with Σ o = Σ \ {f , u}, and the masking function satisfies P(f ) = P(u) = ε and P(x) = x for any other event x. The construction is illustrated in Figure 5.7, where we use the shortcuts:

a ∈ Σ , a ∈ Σ , a ∈ Σ , γ ∈ Γ, γ ∈ {B, ⊥ 0 } and z ∈ Γ \ {⊥ 0 }. q 0 f 0 q 0 f q q f q q c ⊥ 0 , f , ⊥ 0 B, a , ε γ , a , γ γ , a , γ B ⊥ 0 , u, ⊥ 0 γ , , γ B, , ε ⊥ 0 , , ⊥ 0 γ, , γ z, , ε ⊥ 0 , , ⊥ 0 γ, α, γ γ, α, γ γ, α, γ V Figure 5
.7: A POpVPA for EXPTIME-hardness of FA-diagnosability and IAdiagnosability.

V is a slight modification of V : from any state of V (accepting or not), reading the new letter α leads to the sink state q c . As a consequence, for any correct run of V , Σ o , P , there is a positive probability at each step to perform event α and become surely correct. This implies lim n→∞ P(CAmb n ) n∈N = 0. Observe that the above proof for V also applies to V : V is not universal if and only if V , Σ o , P is FF-diagnosable. Now, since lim n→∞ P(CAmb n ) n∈N = 0, FF-diagnosability, FA-diagnosability and IAdiagnosability coincide for V , Σ o , P . We conclude that V is not universal if and only if V , Σ o , P is diagnosable (for any notion of diagnosability).

While diagnosability was undecidable for POpPDA, the situation is more complex with POpVPA. Some notions become decidable, although we may not have the exact complexity, while for others decidability remains open. The method used here requires a characterisation that can be translated in pLTL. This is not possible for FAdiagnosability or the approximate notions of diagnosability which cannot be expressed in pathL. Recall that pathL is more expressive than LTL according to Proposition 3.6, page 79.

Diagnosability of infinite pLTS represented by stochastic Petri nets

In this section we study infinite-state pLTS generated by stochastic Petri nets (SPN). This model is incomparable to pPDA and therefore generates different kinds of pLTS. In Subsection 3.1 we formally define SPN and the infinite-state pLTS generated by an SPN. We then show in Subsection 3.2 that, as for pPDA, diagnosability is undecidable in SPN, and that a restriction similar to what pVPA are to pPDA is not enough to regain decidability.

Stochastic Petri nets

A Petri net contains places and transitions. In each place there are tokens and a transition consumes tokens from input places and produces tokens in output places.

From a manufacturing point of view, these tokens can be seen as different items that are received, assembled with other items, processed and the final product can be exported. Due to the locality of transitions, Petri nets are appropriate for modelling concurrent systems. The infinite behaviour comes from the potentially unbounded number of tokens inside the net. The sets of infinite-state pLTS that can be generated by pushdown systems and Petri nets are incomparable.

Definition 5.9. A Petri net (PN) is a structure N = (P, M 0 , T, P re, P ost), where P is a set of m places; M 0 is the initial marking, i.e. a vector M : P → N that assigns to each place of a PN a non-negative integer number of tokens; T is a set of n transitions; P re : P × T → N and P ost : P × T → N are the pre-and post-incidence functions that specify the arcs. We also define C = P ost -P re as the incidence matrix of the net.

For Mat ∈ {P re, P ost, C} and t ∈ T , we write Mat(• , t) for the column vector which, for every i ∈ N, contains at the row i the value Mat(i, t). A transition t is enabled from M iff M ≥ P re(• , t) and may fire yielding the marking M = M + C(• , t). One writes M [σ to denote that the sequence of transitions σ = t j 1 • • • t j k is enabled from M , and M [σ M to denote that the firing of σ yields M . One writes t ∈ σ to denote that a transition t is contained in σ. The length of the sequence σ (denoted |σ|) is the number of transitions in the sequence, here k.

Example 5.11. Consider the PN of Figure 5.8. The initial marking is M 0 = [2, 0, 0, 0, 0]. Two tokens are needed to fire t 2 , one in place p 1 and one in place p 2 . In order to take the manufacturing analogy again, this means two items must be assembled here. In p 1 two items are already here, ready for assembling, however p 2 is empty. Firing t 1 delivers one item to p 2 , enabling the transition t 2 . Once t 2 was fired, one token in p 1 and one token in p 2 are consumed and one token (the assembled product) is produced in p 3 . There two transitions can be fired. For example, the token can be consumed by t 4 producing a new token in p 4 which enables t 6 . This last transition consumes a token without creating any new one, so it could correspond to the finished product being sent, and thus removed from consideration by this system. A sequence of transitions corresponding to the arrival of new products, their processing and removing from the system is σ = t 0 t 1 t 2 t 4 t 6 . Firing this sequence uses exactly the tokens that are created inside it. Therefore it can be repeated: σ k is enabled from M 0 for all k ∈ N.

The set of all sequences that are enabled at the initial marking M 0 is denoted L(N ), i.e., L(N Example 5.12. Consider again the PN of Figure 5.8. Firing t 1 k times in M 0 leads to the marking M 1 = [2, k, 0, 0, 0]. Therefore the place p 2 is not bounded.

) = {σ ∈ T * | M 0 [σ }. A marking M is reachable in N iff
Probabilities are added to a PN by adding a fire rate to every transition in the following way.

Definition 5.10. A Stochastic Petri Net (SPN) is a pair N = (N, µ) where N is a PN and for all t ∈ T, µ(t) ∈ R + is the rate of firing of transition t.

The usual interpretation of rates is that, in a given marking, a delay is computed for every enabled transition t with an exponential probability distribution function of parameter µ(t), i.e. the probability distribution function for the delay of transition t is f t : x ∈ R + → µ(t)e -µ(t)x . Multiple time semantics [START_REF] Haddad | Stochastic Petri nets[END_REF] can be chosen in an SPN to decide how these delays are used to determine which transition is fired. For instance, one could use (a) a single server policy: each transition can only be fired once by a given marking, (b) a race policy: the transition whose firing delay elapses first is assumed to be the one that will fire next and (c) a resampling memory policy: at the entrance in a marking, the remaining delays associated with all transitions are forgotten. Observe that as we use an exponential probability distribution, whether the delays are forgotten or not does not modify the probabilistic semantic. Using these choices, one could then define the semantics of the SPN as a continuous time Markov chain. However, as we only focus on discrete-time semantics here, we simplify the definition of the probabilistic behaviours of the SPN. We remove the time consideration from the semantics, and only keep the discrete time Markov chain induced by the continuous time Markov chain. This semantics keeps enough information to answer questions expressed for example by pLTL or pCTL formulae, but cannot address time-related issues such as mean reaction time.

Using the simplified interpretation of rates, as for pLTS, a probability measure can be defined on the sequences of transitions of a PN. Given a sequence σ ∈ T * , we write C(σ) for the set of infinite sequences prefixed by σ, C(σ) = {σ ∈ T ω | ∃σ ∈ T ω : σ = σσ }. The set of infinite sequences is the support of a probability measure defined by Caratheodory's extension theorem from the probabilities of the cylinders: the probability of the cylinder starting by the empty sequence ε is equal to 1 and, for σt a sequence, the probability of C(σt) in M 0 , written P(σt), satisfies

P(σt) = P(σ) × µ(t) t ∈T,M 0 [σt µ(t )
As for pPDA, we enhance SPN with a mask function.

Definition 5.11. A partially observable SPN (POSPN) is a tuple N , Σ o , P consisting of an SPN N equipped with a mapping P : T → Σ o ∪ {ε} where Σ o is the set of observations.

From now on, we assume that there does not exist a marking M reachable from M 0 and an infinite sequence σ ∈ T ω such that P(σ) = ε and M [σ . This assumption corresponds to the assumption of convergence that was made for pLTS. The observed sequence w of observations associated with the sequence σ is w = P(σ). Note that the length of a sequence σ is always greater than or equal to the length of the corresponding observed sequence w (denoted |w|). Given a word w ∈ L * , we write P(w) = σ∈P -1 e (w) P(σ). Thanks to our earlier assumption, this sum is finite. Example 5.13. Consider again the PN N of Figure 5.8. We define the POSPN (N, µ), {a, b, c}, P such that for all t ∈ T, µ(t) = 1 and P(t 0 ) = P(t 1 ) = b, P(t 2 ) = a, P(t 3 ) = P(t 4 ) = ε and P(t 5 ) = P(t 6 ) = c. The observed sequence bac corresponds to the sequences t 1 t 2 t 3 t 5 and t 1 t 2 t 4 t 6 , each of which has a probability 1 72 . Therefore P(bac) = 1 36 . The (potentially infinite) pLTS associated with a POSPN is based on the reachability graph of the PN: every state corresponds to a reachable marking. Definition 5.12. A POSPN N , Σ o , P defines a pLTS A V = (Q N , M 0 , T, T N , P N ) where:

• Q N = R(M 0 ); • T s pn = {(M, t, M ) | M [t M }; • For every (M, t, M ) ∈ T N , P N [(M, t, M )] = µ(t) t ∈T,M [t µ(t ) .
This pLTS is infinite when the reachability set is infinite. This happen iff the PN is not bounded. If the PN is k-bounded, for k ∈ N, then the size of the generated pLTS is exponential in the size of the PN and in k. A POSPN is diagnosable according to a notion of diagnosability if the pLTS it generates is diagnosable.

In order to mirror the POpVPA restriction of POpPDA, we introduce the notion of visible POSPN. Definition 5.13. A visible POSPN (VSPN) is a POSPN such that • an unobservable transition does not modify the number of tokens in the system;

• for every pair of two transitions t 1 and t 2 with P(t 1 ) = P(t 2 ) P ost(• , t 1 )1 v -P re(• , t 1 )1 v = P ost(• , t 2 )1 v -P re(• , t 2 )1 v where 1 v is the vector with 1 in every position.

This second condition means that the number of tokens is modified similarly by t 1 and t 2 . An observer of a VSPN thus knows at all time how many tokens are present in the system.

Undecidability of diagnosability for stochastic Petri nets

The exact diagnosability problems for k-bounded Petri nets are decidable as the generated pLTS is exponential and the exact diagnosability problems for finite pLTS are decidable. Moreover deciding if a Petri net is bounded is also decidable [START_REF] Rackoff | The covering and boundedness problems for vector addition systems[END_REF]. For unbounded Petri nets however, while non-stochastic variants of diagnosability are known to be decidable on Petri nets, this is not the case for the stochastic notions of exact diagnosability. In order to show the undecidability, we reduce the problem of the language inclusion for Petri nets, namely: given two PN N 1 and N 2 , an observation alphabet Σ o and a mask function P does P(L(N 1 )) ⊆ P(L(N 2 )) hold? This problem is known to be undecidable(see the survey [START_REF] Esparza | Decidability issues for Petri nets -a survey[END_REF]).

Theorem 5.5. The FF-, IAand FA-diagnosability problems of POSPN are undecidable.

Given two PN N 1 and N 2 , we build an SPN where the initial transition (which can be faulty), produces tokens in one among two components. This component corresponds to an enhanced copy of one of the two given PN. Then, a sequence of this PN is triggered. At any moment during this sequence, a transition starting a reset operation can be taken. This reset operation removes all the tokens from the PN then produces the tokens corresponding to the initial marking so that a new sequence can be read by the PN. The goal is that the fault is detected iff an observed sequence that can only be triggered from N 1 is observed in between two resets operation.

The difficulty of this reduction lies in the reset operation as one cannot test directly whether the places of the PN were correctly emptied. This information however, can be encoded in the observation. Let us now describe what happens in a reset. The reset starts and ends by an observable and, in between, produces a certain number of . Each of these removes a token that was left inside the PN, so that at any moment, the observer knows precisely the number of tokens within the system. If there is still at least one token in the system when the second occurs, a gadget is used to allow the system to trigger any observed sequence so that no information is given to the observer. In other words, the observed sequence in between two reset operations give an information on the system iff the previous reset had correctly emptied the PN.

Proof. Let N 1 = (P 1 , M 1 0 , T 1 , P re 1 , P ost 1 ) and N 2 = (P 2 , M 2 0 , T 2 , P re 2 , P ost 2 ) be two PN, with the mask function P and the observation alphabet Σ o .

Without loss of generality, we assume that the initial marking M i 0 has a single token in a place p i 0 for i = 1, 2, that every transition is observable and that there exists an integer k ∈ N such that the number of tokens in the system is equal to k times the length of the sequence plus 1. This last assumption could be ensured by choosing k as the maximum number of tokens added by a transition (this means number of tokens created minus number of tokens consumed) and adding an additional place where the unnecessary tokens are put (i.e. if a transition adds k tokens with k < k, then this new place receives k -k tokens).

p 0 p 1 0 p 2 0 Box 1 Box 2 p 1 run p 2 run t 1 in , f t 2 in ,
Figure 5.9: Reduction from language inclusion. The Figure 5.10 represents the content of the box Box 1 , it is similar for Box 2 . Transitions are labelled by their name and their observation.

We build the POSPN (P, M 0 , T, P re, P ost, µ, Σ o ∪ { , }, P ) (represented in Figure 5.9) where:

• P = P 1 ∪ P 2 ∪ {p 0 } ∪ {p i emp , p i run , p i err | i = 1, 2}; • T = T 1 ∪ T 2 ∪ {t i in , t i , t i rese , t i resn | i = 1, 2} ∪ {t i a | a ∈ Σ, i = 1, 2} ∪ {t p emp , t p err | p ∈ P 1 ∪ P 2 };
• for i ∈ {1, 2}, p ∈ P i , t ∈ T i , P re(p, t) = P re i (p, t) and P re(p i run , t) = 1, P re(p 0 , t i in ) = 1, P re(p i emp , t i ) = 1, P re(p i run , t i resn ) = 1, P re(p i err , t i rese ) = 1, for a ∈ Σ, P re(p i err , t i a ) = 1, P re(p, t p emp ) = P re(p i emp , t p emp ) = 1, P re(p, t p err ) = P re(p i emp , t p err ) = 1. When undefined, P re(p, t) = 0;

• for i ∈ {1, 2}, p ∈ P i , t ∈ T i , P ost(p, t) = P ost i (p, t) and P ost(p i run , t) = 1, P ost(p i 0 , t i in ) = P ost(p i run , t i in ) = 1, P ost(p i run , t i ) = P ost(p i 0 , t i ) = 1, for a ∈ Σ, P ost(p i err , t i a ) = 1, P ost(p i 0 , t i a ) = k, P ost(p i emp , t i resn ) = P ost(p i emp , t i rese ) = 1, P ost(p i emp , t p emp ) = 1, P ost(p i 0 , t p err ) = 2, P ost(p i err , t p err ) = 1. When undefined, P ost(p, t) = 0;

• P extends P on N by, for p ∈ P 1 ∪ P 2 , i ∈ {1, 2}, a ∈ Σ, P (t i in ) = , P (t i ) = P (t p err ) = P (t i rese ) = P (t i resn ) = , P (t i a ) = a, P (t p emp ) = ;

• for i ∈ {1, 2}, p ∈ P i , µ(t 1 rese ) = µ(t 1 resn ) = µ(t p emp ) = 2k(|Σ| + |T 1 |) (assuming |T 1 | ≥ 1)
and for every other transition t µ(t) = 1. Moreover, t 1 in = f is the fault transition. We show that the system is FF-diagnosable iff P(L(N 1 )) ⊆ P(L(N 2 )). First note that the set of observed sequences associated with the infinite sequences starting by the transition t i in , denoted L i , contains exactly the words of the form w 1 n 1 . . . w k n k . . . where for all 1 ≤ j ≤ k, (1)

p 1 run t p 1 0 p N 1 p 1 emp t 1 resn , t p emp , t 1 , p 1 err t p err , t 1 rese , t 1 a , a
w j ∈ Σ * o , (2) j m=1 k|w m | + 1 ≥ j m=1 n m and (3) j-1 m=1 k|w m | + 1 = j-1
m=1 n m implies w j ∈ P(L(N i )). Suppose that P(L(N 1 )) ⊆ P(L(N 2 )). Let σ be an infinite faulty sequence. As σ is faulty, it initially fired t 1 in , thus P(σ) ∈ L 1 . Thanks to the above remark on the languages L i , and as P(L(N 1 )) ⊆ P(L(N 2 )), P(σ) ∈ L 2 , therefore there exists a sequence σ starting by the transition t 2 in with same observation as σ. Moreover this transition is not faulty as it did not fire t 1 in initially and cannot fire it after the first transition. Therefore P(σ) is not surely faulty. As this is true for every faulty sequence, the system is not IFdiagnosable. The pLTS generated by this POSVN being finitely branching, according to Theorem 3.1, this implies that the POSPN is not FF-diagnosable.

Suppose now that P(L(N1 )) ⊆ P(L(N 2 )). There thus exists a word w such that w ∈ P(L(N 1 )) \ P(L(N 2 )). The observed sequences of L 1 such that there exists i ∈ N with i-1 m=1 k|w m | + 1 = i-1 m=1 n m and w i = w are surely faulty as they do not belong to L 2 . We denote SL 1 the set of these observed sequences. Let us show now that with probability 1 an infinite faulty sequence belongs to SL 1 . While a token is in p 1 err or p 1 run , every transition taken with observation other than produces k tokens in the copy of N 1 . Moreover, there are at most |Σ| + |T 1 | such transitions, each with rates 1. As the transition triggering has rate 2k(|Σ| + |T 1 |), the expectation of the number of tokens produced before a is below 1 + 2k. During a reset operation, as for all places p ∈ P 1 , µ(t p emp ) = 2k(|Σ| + |T 1 |) and the two transitions producing have rates 1, the expectation of the number of token removed from the copy of N 1 , assuming there are enough tokens within the system, is greater than 1 + k(|Σ|+|T 1 |) 1 . Thus, with probability 1, a faulty sequence will infinitely often remove all the tokens from P 1 . Therefore with probability 1, the observation of an infinite faulty sequence will be of the form w

1 n 1 . . . w k n k • • • ∈ L 1 with infinitely many i ∈ N such that i-1 m=1 k|w m | + 1 = i-1 m=1 n m .
There is a probability p > 0 that for any such i, w i = w as w ∈ P(L(N 1 )). Therefore with probability 1, there exists i ∈ N such that w i = w. Hence with an infinite faulty sequence almost surely belongs to SL 1 . This implies that the POSPN is IF-diagnosable and thus FF-diagnosable.

In order to reduce the problem to FA-diagnosability and IA-diagnosability, we proceed similarly to the proof of Theorem 5.4: we add another place p c and a transition t c that takes a token from p 2 run and puts it in p c . This transition has firing rate 1 and observation . This is thus the only transition with this observation. As a consequence, taking this transition ensures the run is surely correct and remains that way. As a run entering Box 2 almost surely infinitely often contains a token in p 2 run , every run almost surely either becomes faulty or surely correct. Therefore lim n-→∞ CAmb n = 0. This implies that in this POSVN, FF-diagnosability is equivalent to FA-diagnosability and IA-diagnosability. The rest of the proof above then applies.

An interesting feature of this proof is that the number of tokens in the POSPN used in the reduction can be deduced from the observation at all time. Therefore it is a VSPN. This gives the following result.

Corollary 5.1. The FF-, IAand FA-diagnosability problems of VSPN is undecidable.

Thus, a restriction similar to what allowed us to regain decidability in POpPDA is not enough for POSPN.

Conclusion

The study of diagnosability for infinite-state pLTS depends heavily on the model used to finitely represent such a pLTS. Choosing a model that is too powerful leads quickly to undecidability. This has been shown with the undecidability proofs established for restricted classes of POpPDA and POSPN. These proof contains important differences. For instance, while undecidability is proven for every notion of stochastic (and in fact even non-stochastic) diagnosability in POpPDA, it is only proven for the exact notions of diagnosability in POSPN. Moreover, it is known that non-stochastic diagnosability is decidable in PN. In this sense, PN is a model for which there is still hope to get decidability results. We did not use the notion of coverability graph here, which gives a finite over-approximation of the reachability graph. Maybe an analysis of its language coupled with a study of the pathological behaviours (due to the over-approximation) may help in solving AFF-diagnosability. Moreover, the restriction that was used for POSPN was chosen to mimic the one used on POpPDA, but is not necessarily the most suited to the model.

Even if the POpPDA model has the strongest undecidability results, the appropriate restriction allows to regain decidability. For POpVPA, we could use model-checking methods to verify pLTL formula equivalent to the logical characterisation of Section 3 of Chapter 3. This only gave decidability of the notions for which a logical characterisation was known. Many questions are still left open partially as a consequence.

First, it would be interesting to find ways to close the complexity gap between our upper and lower bound for the decidable diagnosability notions. The complexity of the current decision procedure comes from an exponential determinisation and the use of a PSPACE model-checking result. As we are interested in specific simple formula, there may be a way to verify them in PTIME instead. The exponential of the determinisation seems harder to remove.

Second, we would want to determine the decidability status of FA-diagnosability. If it is undecidable, it would confirm the difference in complexity with the other notions of exact diagnosability that the logical characterisations showed. However, this difference was shown for infinite-state pLTS in general, not for pLTS generated by POpVPA. It is possible that, as for finite systems, FA-diagosability could be decided for POpVPA with the same complexity as the other exact diagnosability notions.

Finally, one may be interested in considering the case of the approximate diagnosability notions. The method used here cannot be applied. Moreover, it is unclear now what the POpVPA restriction simplifies for approximate notions of diagnosability. Recall that in the finite case, no determinisation were used to solve these notions, allowing for a PTIME algorithm. We conjecture that this notion remains undecidable.

Part III Chapter 6

Control of the degradation in probabilistic systems

Embedded systems are often equipped with one (or more) controller(s) that can modify the behaviour of the system in reaction to the environment. Controllers can, for example, be used in order to maintain some vital functionalities of the system when facing a failure of a component. As controllers need to detect failures to react efficiently, it is tempting to add to controllers a diagnosis task. In other words, the system will contain some choices that can be made and which will alter the behaviour of the system while satisfying its specification. Controllers will then resolve these choices in order to render the system diagnosable. Controllers can be formalised in multiple different ways. For example, controllers could be within the system and thus have have full knowledge of the behaviour of the system or they could rely on partial observation similarly to diagnosers. Since the goal is for controllers to deduce the existence of a fault, we cannot assume they know exactly the state of the system, and thus it must rely on partial observation. Formally, some of the observable events are controllable and, considering its current observation, the controller chooses which subset of events the system can trigger. A system is then said to be actively diagnosable if there exists a controller ensuring its diagnosability. In [START_REF] Sampath | Active diagnosis of discreteevent systems[END_REF], the authors showed that the active diagnosability problem is decidable in doubly exponential time for non-probabilistic systems. Then in [START_REF] Haar | Optimal constructions for active diagnosis[END_REF], the authors designed a single exponential time algorithm and proved this complexity to be optimal. In the probabilistic case, the controllable system can be represented by a weighted transition system in the active case. This weighted transition system, coupled with a controller, produces a pLTS that can have infinitely many states (depending on the memory required by the controller). Thus, unsurprisingly, the active probabilistic diagnosability is more complicated than the corresponding passive problem: exact diagnosability is PSPACE-complete in pLTS (see Chapter 4) while it is EXPTIME-complete (see [BFH + 14]) for controllable weighted LTS (a controllable variant of pLTS).

However the choices performed by the controller ensuring active diagnosis may have a pernicious effect: to detect faults, controllers sometimes could favour the occurrence of these faults! Forcing a fault in the system easily ensures diagnosability but contradicts the initial goal of trying to maintain important functionalities of the system. Additional requirements can thus be made to controllers in order to manage the degradation of the system. Thus, a controller ensures safe active diagnosability if the controlled system is diagnosable and there is a positive probability that an infinite run is correct. In other words, the controller is allowed to increase the probability of a fault in order to ensure diagnosability, however it must maintain a positive probability of correct behaviours. A quantitative version of this requirement fixes a threshold ε to the probability of correct runs that the controller must achieve. Unfortunately, safe active probabilistic diagnosability is undecidable [BFH + 14]. However, when limited to finite-memory controllers, the problem becomes decidable in NEXPTIME [BFH + 14]. Safe active diagnosability may be too strong a requirement for some real systems. Indeed, systems age and whatever control is applied, their components will eventually fail. Thus, in many cases, the fault can be considered unavoidable by the system. As a consequence, some systems are designed to behave correctly for a long period of time at the end of which they will be replaced by a new system. Instead of trying to force runs to stay correct, a controller could try to slow the speed at which the system fails. This expresses a different kind of requirements for the degradation control of a system. We formalise the framework and these requirements in Section 1, establishing a few semantical results along the way. Then, in Section 2.1 we present the algorithmic results.

This chapter develops and extends some of the results from [START_REF] Bertrand | Diagnostic et contrôle de la dégradation des systèmes probabilistes[END_REF].

1 Degradation of a probabilistic system

In this section, we give formal definitions of the degradation of a system. These degradation notions have to be satisfied by the system simultaneously to diagnosability, ensuring that any fault is detected and the system does not produce faults too often or too quickly. As this combination depends on the notion of diagnosability chosen and our focus here is more on degradation, we only use FF-diagnosability (which is the simplest notion of exact diagnosability that we introduced in Chapter 2).

In terms of observation, we use in this chapter a partition between observable and unobservable events (see discussion of Section 1.3 of Chapter 2).

In Subsection 1.1, we give the definitions of degradation for pLTS. Then, in Subsection 1.2, we show how to add a form of control and state the problems we are interested in.

Degradation in passive systems

When protecting a system from degradation, we want that it has a sufficient probability not to trigger a fault, or at least, that if a fault has to occur, it can be postponed as much as possible. We study different notions of the degradation of a system: safety, fault freeness and resiliency.

A pLTS is safe [BFH + 14] if it guarantees a positive probability of infinite correct runs. A pLTS that is not safe is thus doomed to trigger a fault with probability 1. The probability to stay correct could however be arbitrarily low. So we can quantify the notion in order to refine it: for ε > 0, a pLTS is ε-safe if this probability is greater or equal to ε.

Definition 6.1. Let A be a pLTS, ε > 0. A is ε-safe if P(C ∞ ) ≥ ε. It is safe if P(C ∞ ) > 0.
As pointed out in the introduction, in some cases, safety is a too strong requirement. We formalise now two alternatives: fault freeness and resiliency. Fault freeness aims at quantifying the period of time during which the pLTS is correct. We introduce a discount factor γ ≤ 1 on duration in order to vary the importance given to the length of the correct runs. When γ is chosen small, only the beginning of the runs matter. This focus on the short-term is useful for systems that are regularly replaced for example. A greater γ will on the opposite be chosen if one wants the system to be correctly performing for a longer time. The expectation of this discounted value is then compared to a threshold v. Definition 6.2. Let A be a pLTS,

0 < γ ≤ 1 and v ∈ [0, ∞]. • A is (γ, v)-fault free if n≥1 P(C n )γ n ≥ v. • A is lasting fault free if it is (1, ∞)-fault free.
Clearly, for any fixed value of γ, the greater v is, the better the system. Remark also that for γ < 1, the sum n≥1 P(C n )γ n is finite and smaller than 1 1-γ . For γ = 1, n≥1 P(C n )γ n is the mean observable length of the maximal correct signalling prefix of a random run, which can be infinite. This justifies the name lasting fault free when the expectation is infinite.

The notion of resiliency is an alternative measure of degradation based on a factor of degradation ratio per time unit α < 1. A pLTS is α-resilient if the proportion of finite correct runs which stays correct on the next occurrence of an observable event is asymptotically greater than α. This requirement has two qualitative variants: strong resiliency (resp. weak resiliency) requires α-resiliency for every (resp. for at least one) α < 1. In other words, a system is weakly resilient if asymptotically, the probability to be in a correct run of observable length n is greater than an exponential α n . And a system is strongly resilient if this probability is asymptotically greater than all such exponential. Definition 6.3 (Resilient pLTS). Let A be a pLTS.

• Let 0 < α < 1. A is α-resilient if lim sup n→∞ α n P(Cn) = 0; • A is strongly resilient if for all 0 < α < 1, A is α-resilient;
• A is weakly resilient if there exists 0 < α < 1 such that A is α-resilient.

Example 6.1. Let us consider the pLTS A of Figure 6.1. We give examples of the different notions of degradation by studying some choices of probabilities (p i ) i∈N .

q 0 q 1 q 2 q 3 f 1 • • • • • • a, p 1 a, p 2 a, p 3 f , 1 -p 1 f , 1 -p 2 f , 1 -p 3 b Figure 6.1: An example of infinite pLTS with parametric probabilities (p i ) i∈N .
A has a single correct run ρ = q 0 aq 1 aq 2 . . ., with observation a ω while every faulty run contains an infinite number of 'b'. A is thus FF-diagnosable. Moreover, the probability of ρ is n≥1 p n and the probability of its prefix of length n is r n = i≤n p i . Consequently, A is safe iff lim n→∞ r n > 0. This can be achieved by choosing

p i = 1 -1 2 i for example.
Also, by direct application of the definition, A is lasting fault free iff n≥1 r n = ∞. Let us consider different values of (p i ) i∈N . • Let p i = i i+1 . Then r n = 1 n+1 . Thus A is not safe but is lasting fault free. For every α < 1, lim n→∞ (n + 1)α n = 0. Thus A is also strongly resilient.

• Let p i = i 2 (i+1) 2 . Then r n = 1 (n+1) 2 .
Thus A is neither safe nor lasting fault free. For every α < 1, lim n→∞ (n + 1) 2 α n = 0. Thus A is strongly resilient.

• We inductively define two sequences m k and n k by: n k = 2 j<k m j (hence n 0 = 1) and m k = n k + j<k m j + n j . We also define the intervals:

• I k = [n k + j<k m j + n j , j≤k m j + n j [; • J k = [ j≤k m j + n j , n k+1 + i≤k m j + n j [. When i ∈ I k , we choose p i = 1 2 . When i ∈ J k , we choose p i = 1. Observe that for all n ∈ J k , r n = 2 -j≤k m j . Consequently n≥1 r n ≥ k≥0 n∈J k r n = k≥0 2 j≤k m j 2 -j≤k m j = ∞.
Thus A is lasting fault free.

Let k ∈ N and n = j≤k m j + n j . Consequently,

r n = 2 -j≤k m j . Fix α = 1 √ 2 . α n r n = 2 j≤k m j ( √ 2) -j≤k m j +n j ≥ 2 m k ( √ 2) -2m k = 1.
Therefore A is not α-resilient.

The next theorem establishes the precise links between the qualitative versions of the three degradation notions for pLTS. Theorem 6.1. Let A be a pLTS.

• If A is safe then A is lasting fault free and strongly resilient;

• If A if finite then:
A is safe iff A is lasting fault free iff A is strongly resilient;

• There exists a lasting fault free pLTS that is not strongly resilient;

• There exists a strongly resilient pLTS that is not lasting fault free.

The first assertion is quickly obtained from the definitions and the last two come directly from the previous examples. The second one requires a bit more development. As the pLTS A is finite, one can use the notion of bottom strictly connected component used in Chapter 4 to characterise the diagnosability notions for finite pLTS. A notable difference is that, in Chapter 4, we had to consider the BSCC of an enriched pLTS. Here, we show that every notion of degradation is equivalent to the existence of a reachable correct BSCC of the pLTS A.

Proof. Let A be a safe pLTS. There exists ε > 0 such that for all n, P(C n ) ≥ ε. Thus,

n≥1 P(C n ) ≥ n≥1 ε = ∞. Moreover, for all α < 1, lim n→∞ α n P(Cn) ≤ lim n→∞ α n ε = 0.
Thus A is both lasting fault free and strongly resilient.

Let A be a finite pLTS. Observe that every BSCC of A contains either only correct states or only faulty states. Accordingly we can speak of faulty BSCC or correct BSCC. As A is a finite pLTS, we know that almost surely an infinite run reaches a BSCC and that the mean time to reach a BSCC is finite (see e.g. [START_REF] Baier | Principles of model checking[END_REF]). Due to the first result, A is safe iff there exists a reachable correct BSCC.

Suppose that A is not safe. • Every reachable BSCC are faulty which implies that the mean time to reach a faulty BSCC if finite. This mean time is an upper bound on the mean observable length of the maximal signalling prefix of a correct run. Thus A is not lasting fault free.

• We note m = |Q|. For all q ∈ Q c , there exists ρ q a run starting in q composed of an elementary run from q to a faulty BSCC followed by an elementary run (or circuit) in the BSCC of which only the last event is observable (by convergence). This run has an observable length smaller or equal to m. We note µ q , the probability of that run and µ = min q∈Qc µ q . Consider a signalling run ρ of observable length n for an arbitrary n and ending in q ∈ Q c . From the existence of ρ q , P({ρ

∈ SR n+m ∩ C | ρ ρ }) ≤ (1 -µ)P(ρ). Thus P(C n+m ) ≤ (1 -µ)P(C n ). So, P(C n ) ∈ O((1 -µ) n m ). Choosing α = (1 -µ) 1 m , A
is not α-resilient and thus not strongly resilient.

Controlled systems

Extending the pLTS formalism in order to express control requires to fix at least two features of this formalism: the nature of the control and the distribution of probabilities of the controlled system. Intuitively, we want the control and the diagnosis to be realised by the same device: from its observations, it restricts the system in order to diagnose it and limit its degradation. The control is thus done with partial observation. So we recall the Controllable Labelled Transition System (CLTS) from [BFH + 14]. In this model, in order to specify the control, a subset of observable events is considered controllable. The controller forbids a subset of controllable events depending on the sequence of observations it has received. Thus the controller cannot modify its choice between two observations. The transitions of the system are no more labelled by (rational) probabilities but by (integer) weights which measure their relative possibility of occurrence. Given a state and a set of forbidden controllable actions, the weights of the transitions exiting this state and labelled by uncontrollable or allowed controllable actions are normalised to obtain a probability distribution. If the controller does not introduce any deadlock, the controlled system is a live pLTS. Definition 6.4. A Controllable Labelled Transition System (CLTS) is a tuple C = Q, q 0 , Σ, T where:

• Q is a set of states with an initial state q 0 ∈ Q;

• Σ = Σ o Σ u
is a finite set of events partitioned into the set of observable events Σ o containing controllable events Σ c ⊆ Σ o and the set of unobservable events Σ u containing the fault f ;

• T : Q × Σ × Q → N
is the transition function that associates an integer weight with each transition.

A CLTS induces a labelled transition system which transition relation is defined by q a -→ q if T (q, a, q ) > 0. The extended relation = ⇒ and the other usual definitions are defined as for pLTS. We assume that the CLTS is convergent and live. Example 6.2. A CLTS C is represented in Figure 6.2. If the control enables every event, the run q 0 uq 1 aq 1 bq 2 has probability 1/8. If the control always forbids 'b', this same run has probability 0. And if it only allows 'b' after observing one 'a', it has probability 1/4.

q 0 q 1 q 2 q 3 f 1 u b f f a, b a a
We now formalise the ingredients necessary to define how to control CLTS. Let Σ • ⊆ Σ and q ∈ Q, let us write G Σ • (q) for the sum of the weights of the transitions exiting q and labelled by an event of Σ • . Using this sum, we define a normalisation of the transition relation restricted to the events of Σ • by:

T Σ • (q, a, q ) = T (q,a,q ) G Σ • (q) if a ∈ Σ • and G Σ • (q) > 0 0 otherwise. A strategy of a CLTS C is a function π : Σ * o → Dist(2 Σ ) such that for all w ∈ Σ * o and all Σ • ∈ Supp(π(w)), Σ \ Σ c ⊆ Σ • .
In other words, given an observation, a strategy chooses (possibly with randomisation) a set of allowed events that contains the uncontrollable events. Let C be a CLTS and π be a strategy, we consider the configurations of the form (w, q, Σ • ) ∈ Σ * o × Q × 2 Σ with w the observed sequence, q the current state and Σ • the set of allowed events by π after observation of w. We inductively define the set Reach π (C) of the reachable configurations under π by:

• for all Σ • ∈ Supp(π(ε)), we have (ε, q 0 , Σ • ) ∈ Reach π (C);
• for all (w, q, Σ • ) ∈ Reach π (C) and all a ∈ Σ u such that q a -→ q , we have (w, q , Σ • ) ∈ Reach π (C), and the corresponding transition is denoted by (w, q, Σ • ) a -→ π (w, q , Σ • );

• for all (w, q, Σ • ) ∈ Reach π (C), all a ∈ Σ o ∩ Σ • such that q a -→ q and all Σ • ∈ Supp(π(wa)), we have (wa, q , Σ • ) ∈ Reach π (C), and the corresponding transition is denoted by (w, q, Σ • ) a -→ π (wa, q , Σ • ).

A strategy π is called live if for every configuration (w, q, Σ • ) ∈ Reach π (C), we have G Σ • (q) = 0. Only the live strategies are relevant as the other strategies create deadlocks. We are now in a position to introduce the semantics of a CLTS controlled by a live strategy π in terms of a live pLTS. Its set of states is Reach π (C) augmented by an initial state to randomly choose the initial control according to π(ε). The probability distributions are based on T Σ • if the current control is Σ • combined with the random choice of π in case of an observable event occurrence. Definition 6.5. Let C be a CLTS and π be a live strategy, the pLTS C π induced by the strategy π on C is defined by C π = Q π , Σ, q 0π , T π , P π where:

• Q π = {q 0π } ∪ Reach π (C); • for every (ε, q 0 , Σ • ) ∈ Reach π (C), (q 0π , u, (ε, q 0 , Σ • )) ∈ T π ; • for every (w, q, Σ • ), (w , q , Σ • ) ∈ Reach π (C), (w, q, Σ • ), a, (w , q , Σ • ) ∈ T π iff (w, q, Σ • ) a -→ π (w , q , Σ • ); • for every (ε, q 0 , Σ • ) ∈ Reach π (C), P π (q 0π , u, (ε, q 0 , Σ • )) = π(ε)(Σ • );
• for every ((w, q, Σ • ), a, (w, q , Σ • )) ∈ T π and every a ∈ Σ u , P π ((w, q, Σ • ), a, (w, q , Σ • )) = T Σ • (q, a, q );

• for every (w, q, Σ • ), a, (wa, q , Σ • ) ∈ T π and every

a ∈ Σ o ∩ Σ • , P π (w, q, Σ • ), a, (wa, q , Σ • ) = T Σ • (q, a, q ) • π(w.a)(Σ • ).
Example 6.3. Consider the CLTS C depicted in Figure 6.2. There are two possible enabled subsets: Σ and Σ \ {b} that we denote Σ -. Let us define the strategy π by π(a n ) = p n • Σ -+ r n • Σ with p n + r n = 1 for all n ∈ N and π(w) = 1 Σ otherwise. The generated pLTS C π is infinite. A part of it is represented in Figure 6.3. Let us develop the distribution of probabilities exiting the configuration (ε, q 1 , Σ). The two transitions exiting q 1 are enabled with equal probabilities, thus normalised to 0.5. Since 'a' and 'b' are observable, the new control is chosen, in the case where a 'a' is observed, by a probabilistic choice p 1 •Σ -+r 1 •Σ while if a 'b' is observed, there is a deterministic choice 1 Σ . This results in three transitions with probability 0.5p 1 , 0.5r 1 and 0.5 respectively. In Definition 3.2, page 58, we introduced finite-memory diagnosers. Similarly, one can formally define finite-memory strategies for CLTS using a set of memory states, a memory update function indicating how observations modify the memory state and a decision function mapping every memory state to a choice of the strategy. The size of the memory is the number of memory states. If the size of the memory of a strategy π of a CLTS C is finite, then C π is also finite.

q 0π ε, q 0 , Σ - ε, q 0 , Σ ε, q 1 , Σ - ε, f 1 , Σ - ε, f 1 , Σ ε, q 1 , Σ b, q 2 , Σ a, q 1 , Σ - a, q 1 , Σ a, f 1 , Σ - a, f 1 , Σ b, f 1 , Σ • • • • • • • • • • • • • • • u, p 0 u, r 0 u, 0.5 f , 0.5 f , 0.5 u, 0.5 a, p 1 a, r 1 a, p 1 a, r 1 a, 0.5p 1 a, 0.
Let us define the problems of active diagnosis in the context of the degradation control. Roughly speaking, given a CLTS, one asks whether there exists a strategy such that the associated pLTS is FF-diagnosable and satisfies the required property related to degradation. We distinguish, as usually done, the quantitative problems and the qualitative ones (such as safety, lasting fault freeness and strong/weak resiliency).

Definition 6.6 (Quantitative problems). Given a CLTS C, 0 < ε, α < 1, 0 < γ ≤ 1 and v ∈ [0, ∞]:
• The ε-safe active diagnosis problem consists in deciding if there exists a strategy π such that C π is FF-diagnosable and ε-safe;

• The (γ, v)-fault free active diagnosis problem consists in deciding if there exists a strategy π such that C π is FF-diagnosable and (γ, v)-fault free;

• The α-resilient active diagnosis problem consists in deciding if there exists a strategy π such that C π is FF-diagnosable and α-resilient.

Definition 6.7 (Qualitative problems). Given a CLTS C:

• The safe active diagnosis problem consists in deciding if there exists a strategy π such that C π is FF-diagnosable and safe;

• The lasting fault free active diagnosis problem consists in deciding if there exists a strategy π such that C π is FF-diagnosable and lasting fault free;

• The strongly resilient active diagnosis problem consists in deciding if there exists a strategy π such that C π is FF-diagnosable and strongly resilient;

• The weakly resilient active diagnosis problem consists in deciding if there exists a strategy π such that C π is FF-diagnosable and weakly resilient.

When tackling problems on strategies, the first step is to wonder if one can restrict the strategies that are considered. For example, can we use strategies with finite memory. This cannot be done as shown in the following example.

Example 6.4. In order to illustrate the impact of taking into account infinite memory strategies, let us examine the CLTS C of Figure 6.2. The only ambiguous observed sequence is a ω . A strategy π thus makes it FF-diagnosable iff the probability of faulty runs with this observed sequence in C π is 0. This is done by allowing 'b' often enough so that it occurs with probability 1. However, the only correct run is ρ = q o u(q 1 a) ω with observation a ω . Thus, C is not actively safely diagnosable.

Let us denote, as in Example 6.3, by p n the probability to forbid 'b' after the observed sequence a n given by the strategy π. Then P Cπ (q o u(aq

1 ) n ) = 1 2 i≤n 1+p i
2 . Thus, by choosing p n = 1 -1 n+1 , C π is FF-diagnosable, lasting fault free and strongly resilient. On the other hand, no finite-memory strategy could achieve this goal since otherwise by Theorem 6.1, C would be actively safely diagnosable.

Restricting one-self to finite-memory strategies is thus a loss of generality. It can however be useful to regain decidability of difficult problems as we will see later.

Algorithmic analysis of degradation

In this section we answer the problems listed above by establishing if they are decidable and in the positive case by giving their exact complexities. We start in Subsection 2.1 by proving the undecidability of the quantitative problems. As a consequence, in Subsection 2.2, we focus on the qualitative problems and prove the EXPTIME-completeness of all of them except the safe active diagnosis problem. Lastly, in Subsection 2.3, we see that the safe active diagnosis problem is more difficult but can still be decided efficiently when restricted to finite-memory strategies.

Undecidability of the quantitative problems

The quantitative problems turn out to all be undecidable. The proofs of these results are obtained by reductions from the emptiness problem of probabilistic automata 1 .

We start by showing the result for the ε-safe diagnosis problem, with ε > 0.

Proposition 6.1. The ε-safe active diagnosis problem is undecidable.

The idea of this proof is the following. Given a probabilistic automaton A with alphabet Σ, one builds a CLTS C composed of two independent parts each one initially entered with probability 1 2 by an unobservable transition. The unobservable event leading to the first part is the fault f which can only be detected almost surely if the observable event / ∈ Σ occurs with probability 1. The second part is constituted of a CLTS version of A augmented by exiting transitions. One can exit A by allowing a . When this happens, if the system was in a final state of A it goes to a correct BSCC of the CLTS, ensuring the run will remain correct. Else a fault is triggered on the next step. This construction ensures the following properties. If there exists a word w with an acceptance probability at least 2ε, the strategy which consists in forcing the observed sequence w ensures a probability of the set of infinite correct runs of at least ε. In the opposite case, we show that no strategy can achieve this threshold.

Proof. Let 0 < ε < 1/2. We proceed here by reduction from the problem of the existence of a word w such that P A (w) ≥ 2ε. We consider a probabilistic automaton A = Q, q 0 , Σ, (P a ) a∈Σ , F for which w.l.o.g. we assume that: (1) Σ ∩ {u, f , , } = ∅ and (2) the probabilities are fractions n d with fixed denominator d ∈ N. One builds the CLTS C = Q , q 0 , Σ , T described in Figure 6.4 and defined by:

• Q = Q ∪ {q 0 , q c , q f , f 1 , f 2 }; • Σ = Σ ∪ {f , u, , }, Σ u = {f , u} and Σ c = Σ ∪ { };
• the transition function T is defined as follows.

1.

T (q 0 , f , f 1 ) = T (q 0 , u, q 0 ) = T (q c , , q c ) = T (q f , f , f 2 ) = T (f 2 , , f 2 ) = T (f 1 , , f 2 ) = 1; 1 see page 115.
2. for every a ∈ Σ, T (f 1 , a, f 1 ) = 1; 3. for every s, s ∈ Q and every a ∈ Σ, T (s, a, s ) = d • P a (s, s ); 4. for every s ∈ F , T (s, , q c ) = 1 and for every s ∈ S \ F , T (s, , q f ) = 1; 5. for every other triplet, T is equal to 0.

As detailed above, the probabilities in A are all multiplied by their common denominator d, to obtain integer weights, and we write d • A in the figure to represent this scaling.

q 0 f 1 s 1 s 2 d • A q f f 2 q c f , 1 u, 1 , 1 , 1 f , 1 , 1 Σ, 1 , 1 , 1 Figure 6
.4: Reduction to ε-safe diagnosability.

Let us show that there exists a strategy π such that C π is ε-safe and FF-diagnosable iff there exists a word w accepted in A with probability at least 2ε.

Remark first that, for π an arbitrary strategy, C π is FF-diagnosable iff occurs almost surely in a run. Indeed, an observed sequence w ∈ Σ * is ambiguous. On the other hand every faulty run ρ triggering a will produce a removing the ambiguity.

• Assume there exists a word w = a 1 . . . a k ∈ Σ * such that P A (w) ≥ 2ε. We define the deterministic strategy π by:

• π(w) = {f , u, , }; • for all 0 ≤ i < k, π(a 1 . . . a i ) = {f , u, a i+1 , };
• π(w ) = Σ for any other word w .

Observe that after at most k + 1 observable events, any run leaves Q ∪ {f 1 } and thus occurs almost surely implying that C π is FF-diagnosable. Moreover, the probability of correct runs with observation w is equal to P A (w) 2 : it is the probability to take u initially times the probability to end the observation of w in an accepting state of A. As P A (w) ≥ 2ε, this ensures that C π is ε-safe.

• Assume now that for all w ∈ Σ * , P A (w) < 2.ε. Let π be a strategy such that C π is FF-diagnosable, thus with probability 1 an infinite run contains a . Moreover, this run is correct iff the first is followed by a second . Then we have:

P π (C ∞ ) = w∈Σ * P π (w ) = w∈Σ * P π (w ) • P A (w) 2 < ε w∈Σ * P π (w ) = ε.
Therefore, C π is not ε-safe, which concludes the reduction and proves undecidability of the ε-safe active diagnosis problem.

We now turn to the (γ, v)-fault free active diagnosis problem. It is done once again by reduction from the emptiness problem of PA. In fact, it has many similarities with the previous proof, but instead of reaching a state q c where the run will stay correct, being accepted by the PA only postpones the fault by one step. Proposition 6.2. The (γ, v)-fault free active diagnosis problem is undecidable.

The idea of this proof is the following. Given a probabilistic automaton A with alphabet Σ, one builds a CLTS C composed of two independent parts each one initially entered with probability 1 2 by an unobservable transition. The unobservable event leading to the first part is the fault f which can only be detected almost surely if the observable event / ∈ Σ occurs with probability 1. The second part is constituted of a CLTS version of A augmented by exiting transitions. One exits A with probability 1 2 at every step towards a faulty sub-part except if the event is triggered. In this case, if the system was in a final state of A it leaves the states of A and postpones the occurrence of a fault by one time step compared to if it stayed in A. This construction ensures the following properties. If there exists a word w with an acceptance probability at least 1 2 , the strategy which consists in forcing the observed sequence w as long as the run stays in A ensures an average observable length (without discount) of the maximal correct signalling prefix greater or equal to 1. In the opposite case, we show that no strategy can achieve this threshold.

Proof. We proceed here by reduction from the problem of the existence of a word w such that P A (w) ≥ 1 2 . We consider the probabilistic automaton A = Q, q 0 , Σ, (P a ) a∈Σ , F for which w.l.o.g. we assume that: (1) Σ ∩ {u, f , , } = ∅ and (2) the probabilities are fractions n d with fixed denominator d. One builds the CLTS C = Q , q 0 , Σ , T described in Figure 6.5 and defined by:

• Q = Q ∪ {q 0 , q 1 c , q 2 c , q 3 c , f 1 , f 2 }; • Σ = Σ ∪ {f , u, , }, Σ u = {f , u} and Σ c = Σ ∪ { };
• the transition function T is defined as follows.

1.

T (q 0 , f , f 1 ) = T (q 0 , u, q 0 ) = T (q 1 c , , q 3 c ) = T (q 3 c , ,

q 3 c ) = T (q 3 c , f , f 2 ) = T (q 2 c , f , f 2 ) = T (f 2 , , f 2 ) = T (f 1 , , f 2 ) = 1;
2. for every a ∈ Σ, T (f 1 , a, f 1 ) = 1;

3. for every s, s ∈ Q and every a ∈ Σ, T (s, a, s ) = d•P a (s, s ) and T (s, a, q 2 c ) = d;

4. for every s ∈ F , T (s, , q 1 c ) = 1 and for every s ∈ S \ F , T (s, , q 2 c ) = 1; 5. for every other triplet, T is equal to 0.

Here again, the probabilities in A are multiplied by the constant d, which we abbreviate in the figure by d • A. Let us show that there exists a strategy π such that C π is (1, 1)-fault free and FFdiagnosable iff there exists a word w accepted in A with probability at least 1 2 . Remark first that, for π an arbitrary strategy, C π is FF-diagnosable iff occurs almost surely in a run. Indeed an observed sequence w ∈ Σ * is ambiguous. On the other hand every run ρ leaving Q ∪ {f 1 } almost surely reaches f 2 where occurs and, whatever ρ, a fault has occurred.

q 0 q 0 f 1 s 1 s 2 d • A q 1 c q 2 c f 2 q 3 c f , 1 u, 1 , 1 Σ, d Σ, d ∪ {( , 1)} f , 1 f , 1 , 1 , 1 Σ, 1 ,
• Assume that there exists w = w 1 . . . w k ∈ Σ * such that P A (w) ≥ 1 2 . We define the deterministic strategy π by:

• π(w) = {f , u, , }; • for all 0 ≤ i < k, π(w 1 . . . w i ) = {f , u, w i+1 , };
• π(w ) = Σ for any other word w .

Observe that after at most k + 1 observable events, any run leaves S ∪ {f 1 } and thus occurs almost surely implying that C π is diagnosable.

By definition of C and π, a correct signalling run ρ such that P(ρ) = w 1 . . . w i for i < k has probability 1 2 of staying correct at the next step depending on if the current state is q 2 c or belongs to Q. Similarly, a correct signalling run ρ such that P(ρ) = w 1 . . . w k has a probability P A (w) of being at the next step in q 1 c and 1 -P A (w) in q 2 c . Moreover, in state q 3 c , a correct signalling run has a probability 1 2 of staying correct and in q 3 c at the next step. Therefore for all n ∈ N, we have n ≤ k implies P(C n ) = ( 12 ) n and n > k

implies P(C n ) = ( 1 2 ) n-1 P A (w) ≥ ( 1 2 ) n . Finally: ∞ n=1 P(C n ) ≥ ∞ n=1 ( 1 2 ) n = 1.
• Assume that for all w ∈ Σ * , P A (w) < 1 2 . Let π be a strategy such that C π is diagnosable. Observe that (using a slight and understandable abuse of language):

P π (C n ) = w∈Σ n P π (w ∧ C) + w∈Σ n-1 P π (w ∧ C) + 1<k≤n w∈Σ n-k P π (w k ∧ C).
Let us show that P π (C n+1 ) ≤ Pπ(Cn) 2 with a strict inequality if there exists w ∈ Σ n-1 with P π (w ) > 0.

P π (C n+1 ) = w∈Σ n x∈Σ∪{ } P π (wx ∧ C) + w∈Σ n-1 P π (w 2 ∧ C)+ 1<k≤n w∈Σ n-k P π (w k+1 ∧ C)
Let us examine the three terms.

• A correct run ρ with observed sequence w has a conditional equiprobability that last(ρ) ∈ Q or last(ρ) = q 2 c . Thus, w∈Σ n x∈Σ∪{ } P π (wx) = 1 2 w∈Σ n P π (w). • A correct run ρ with observed sequence w k such that k > 1 verifies last(ρ) = q 3 c . Thus, 1<k≤n w∈Σ n-k P π (w k+1 ∧ C) = 1 2 1<k≤n

w∈Σ n-k P π (w k ∧ C) • A correct run ρ of observed sequence w has a conditional probability P A (w) that last(ρ) = q 1 c and 1 -P A (w) that last(ρ) = q 2 c . Thus:

w∈Σ n-1 P π (w 2 ∧ C) = w∈Σ n-1 P A (w)P π (w ∧ C) ≤ 1 2 w∈Σ n-1 P π (w ∧ C)
with a strict inequality if there exists a word w ∈ Σ n-1 with P π (w ) > 0.

By assumption, C π is diagnosable. Thus, according to our characterisation of a strategy ensuring FF-diagnosability, there exists a word w such that P π (w ) > 0. As a consequence,

∞ n=1 P(C n ) < ∞ n=1 ( 1 2 ) n = 1, thus A is not (1, 1) fault free.
Remark 6.1. A straightforward adaptation of the proof shows that for every 0 < γ < 1, A is (γ, γ 2-γ ) fault free iff there exists a word w such that P A (w) ≥ 1 2 .

We end with the α-resilient active diagnosis problem. The construction of the reduction is a bit simpler. This is due to the fact that the system is FF-diagnosable for any arbitrary strategy. In other words, the reduction only relies on the α-resilient property to establish undecidability. Proposition 6.3. The α-resilient active diagnosis problem is undecidable.

This time, given a probabilistic automaton A with alphabet Σ, one transforms A into a CLTS, augmented by two states and some transitions. This CLTS is called C and its initial state is the initial state of A. At each step, when reading an event of Σ, with probability 1/2 we exit A and will commit a fault in the next step. When a is read after a word w 1 . . . w k with for all i ≤ k w i does not contain , either we go back to the initial state of A or we will trigger a fault on the next turn depending on the probability to accept w k . If a strategy can regularly trigger a word with acceptance probability greater than 1/2, it can slow the speed at which the runs become faulty.

Proof. We proceed here by reduction from the problem of the existence of a word w such that P A (w) > 1 2 . We consider a probabilistic automaton A = Q, q 0 , Σ, (P a ) a∈Σ , F for which we assume w.l.o.g. that: (1) Σ ∩ {u, f , , } = ∅ and (2) the probabilities are fractions n d with d ∈ N fixed. One builds the CLTS C = Q , q 0 , Σ , T represented in Figure 6.6 (with some shortcuts to ease readability) and defined by:

• Q = Q ∪ {q 1 , f 1 }; • Σ = Σ ∪ {f , , }, Σ u = {f } et Σ c = Σ ∪ { };
• the transition function T is defined by:

1. T (q 1 , f , f 1 ) = T (f 1 , , f 1 ) = 1;
2. for every s, s ∈ Q, a ∈ Σ, T (s, a, s ) = d • P a (s, s ) and T (s, a, q 1 ) = d; 3. for every s ∈ F , T (s, , s 0 ) = 1 and for every s ∈ S \ F , T (s, , q 1 ) = 1; 4. for every other triplet, T is equal to 0.

Once again, the probabilities in A are multiplied by the constant d, which we abbreviate in the figure by d • A.

As every fault is followed by a , whatever the strategy π, C π is FF-diagnosable.

• Assume there exists w = w 1 . . . w k ∈ Σ * such that P A (w) > 1 2 . We denote v = P A (w). We define the deterministic strategy π by:

• π((w ) * w) = {f , , };

• for all 0 ≤ i < k, π((w ) * w 1 . . . w i ) = {f , , w i+1 };

• π(w ) = Σ for any other word w .

q 0 s 1 s 2 d • A q 1 f 1 Σ, d Σ, d ∪ {( , 1)} , 1 f , 1 , 1
Figure 6.6: Reduction to α-resilient active diagnosability.

Under strategy π, the observed sequence of a correct run ρ is some (w ) m w 1 . . . w i with 0 ≤ i ≤ k.

• If P(ρ) = (w ) m w 1 . . . w i with 0 < i then with conditional equiprobability, last(ρ) ∈ Q or last(ρ) = q 1 . Thus with probability 1 2 , the run will be correct after the next observation.

• If P(ρ) = (w ) m then with conditional probability v, last(ρ) = q 0 and with probability 1 -v, last(ρ) = q 1 . Thus with probability v, the run will be correct after the next observation.

Consider an arbitrary n and write the Euclidian division of n -1 by k + 1 as n -1 = m(k + 1) + i with i ≤ k. One has 2 -(n-1) P π (C n ) = (2v) m . Hence 2 (n-1) Pπ(Cn) = 1 2v n-1 k+1

implying lim n→∞

-n

Pπ(Cn) = 0. So C π is 1 2 -resilient. • Assume now that for every word w ∈ Σ * , P A (w) ≤ 1 2 . Let π be an arbitrary strategy. The observed sequence of a correct run ρ is some u 1 . . . u m such that for all i, u i ∈ Σ * .

• If u m = ε with 0 < i then with conditional equiprobability, last(ρ) ∈ Q or last(ρ) = q 1 . Thus with probability 1 2 , the run will be correct after the next observation. • If u m = ε then with conditional probability P A (u m-1 ), last(ρ) = q 0 and with probability 1 -P A (u m-1 ), last(ρ) = q 1 . Thus with probability P A (u m-1 ) ≤ 1/2, the run will be correct after the next observation.

Summarising, one has:

P π (C n ) ≤ 2 -(n-1) implying lim sup n→∞ 2 -n Pπ(Cn) ≥ 1 2 . So C π is not 1 2 -resilient.

Decidability of the Qualitative Problems

In contrast to the quantitative notions, and to the notable exception of the safe active diagnosis problem, all the qualitative problems of diagnosability under degradation constraints we introduced are decidable and EXPTIME-complete. The simplest case is the one of weak resilient active diagnosability. The proof idea is common to all cases: starting from a construction that gives an efficient characterisation of active diagnosability (inspired from [BFH + 14] and detailed below), we establish a necessary and sufficient condition for the existence of a control strategy that ensures the given notion of diagnosability under a degradation constraint.

Let us start by defining a construction inspired from [BFH + 14]2 . This construction is the adaptation to the active setting of the FF-automaton used in Subsection 1.1.1 of Chapter 4, page 93. This construction uses the notion of belief. The initial belief is {q 0 }, and given a current belief B and an observed event b, the belief obtained after b has been observed is defined by:

∆(B, b) = {q ∈ Q | ∃q ∈ B, ρ ∈ SR 1 , q ρ = ⇒ q ∧ P(ρ) = b} .
∆(B, b) is thus the set of states a partially observable systems may be in, given that the previous belief was B and observation O occurred. Importantly, it does not depend on the strategy as every controllable event is observable. The set of beliefs of a CLTS C is denoted Bl C and we drop the subscript when there is no risk of confusion. Beliefs are of importance since they formalize the discrete information an observer has on the current state of the system. Thus, to decide FF-diagnosability of a CLTS, the states of the CLTS are enriched with two sets U and V that correspond, respectively, to the subset of correct, or faulty, states, that are reachable by a signalling run corresponding to the current observed sequence, i.e. to the set of correct, or faulty, states of the current belief. Such a pair of sets (U, V ) is therefore called a separated belief. As we study FF-diagnosability here, one could wonder why we do not only use a set U as we did in Chapter 4. In fact, in some of the constructions that we make later, we need to know the full belief. For example, forgetting a faulty state could result in a controller making a choice that creates a deadlock in this faulty state. By using U and V , we have the information pertaining to FF-diagnosis (U ) and to the current belief (U ∪ V ).

Formally, from a CLTS C = Q, q 0 , Σ, T , we define its belief version on the same event alphabet C B = Q B , q B 0 , Σ, T B by:

• Q B = Q × 2 Q × 2 Q and q B 0 = (q 0 , {q 0 }, ∅); • for every (q, U, V ) ∈ Q × 2 Q × 2 Q , for every a ∈ Σ, and every q ∈ Q -if a / ∈ Σ o , T B ((q, U, V ), a, (q , U, V )) = T (q, a, q ); -if a ∈ Σ o , letting U = ∆(U, a) ∩ Q c and V = ∆(U ∪ V, a) ∩ Q f , then T B ((q, U, V ), a, (q , U , V )) = T (q, a, q ).
for every other triplet ((q, U, V ), a, (q , U , V )), T is equal to 0.

The size of the belief CLTS C B is exponential in the size of C. For the properties we are interested in, they have the same behaviour. We introduce Θ, a discrete version of T B , extended to observed sequences. For w ∈ Σ * o , (q , U , V ) ∈ Θ((q, U, V ), w) as soon as there exists a run ρ such that P(ρ) = w and (q, U, V ) ρ = ⇒ (q , U , V ). We now construct Win the set of all separated beliefs (U, V ) such that, starting from any (q, U, V ) with q ∈ U ∪ V , C B is actively diagnosable. This set is computed as a greatest fixpoint. We let Win 0 = 2 Qc × 2 Q f and for n ∈ N, Win n+1 is the set of the separated beliefs (U, V ) of Win n such that for all state q ∈ U ∪V , there exists a sequence of sets of allowed events (Σ • i ) 1≤i≤k and an observed sequence w = o 1 . . . o k with o i ∈ Σ • i verifying:

• there exists a run ρ starting in (q, U, V ) with P(ρ) = w and reaching (q * , U * , V * ) with q * ∈ Q c (i.e. the current state is correct) or U * = 0 (the fault is claimed);

• Consider a state q i reached from q ∈ U ∪ V by a run with observed sequence o 1 . . . o i with 0 ≤ i < k, i.e. (q i , U i , V i ) ∈ Θ((q , U, V ), o 1 . . . o i ) for a separated belief (U i , V i ). then:

1. the control induced by Σ • i+1 does not create any deadlock: G Σ • i+1 (q i ) = 0; 2. Every new separated belief obtained by an observable step o ∈ Σ • i+1 starting in q i belongs to

Win n : ∀o ∈ Σ • i+1 , ∀(q o , U o , V o ) ∈ Θ((q i , U i , V i ), o), (U o , V o ) ∈ Win n .
The computation of Win is in polynomial time in the size of C B , given that in every non-terminal iteration at least one separated belief is removed. The correctness of Win is established in the richer context of IA-diagnosability in [BFH + 14], and π * a (deterministic finite-memory) strategy ensuring diagnosability consists in, given a separated belief (U, V ) ∈ Win choosing the greatest set Σ • such that every possible separated belief reached on the next step still belongs to Win. Thus, π * is the most permissive strategy ensuring active diagnosability.

To decide weakly (resp. strongly) resilient active diagnosability, and lasting fault free active diagnosability, we build on the belief CLTS construction. Theorem 6.2. Weakly resilient active diagnosability is EXPTIME-complete.

Analysing the set of separated beliefs Win gave a condition for the active diagnosability and in the positive case a deterministic finite-memory strategy π * ensuring it. We show in this proof, that in order for a CLTS to be weakly resilient active diagnosable, it needs (1) to be actively diagnosable and (2) C B must contain a reachable cycle of correct states associated with separated beliefs of Win. The idea is that if such a cycle exists, playing a strategy permissive enough (for example π * ), there is a fixed probability to stay within this cycle and this probability can be used to establish a lower bound to the speed at which the system becomes faulty.

The lower bound is straightforward considering that active diagnosability was already proven to be EXPTIME-hard [BFH + 14].

Proof. We first establish the membership in EXPTIME. Given a CLTS C, its belief CLTS C B , and the strategy π * , we derive a pLTS A. It is obtained from C B by restricting it to the states with separated belief in Win and controlled by π * . We claim that C is actively diagnosable with guarantee of weak resiliency iff there exists in A a reachable cycle such that the first component of every state along the cycle is a correct state of C. • Suppose first that such a cycle exists in A. We let α > 0 be the probability of this cycle, n 1 its length, n 0 the observed length of the shortest run reaching a state of the cycle and µ the probability of this run. For all n ≥ n 0 ,

P A (C n ) ≥ µα n-n 0 n 1
. As a consequence, A is α -resilient for all α < α. A is thus weakly resilient. Therefore, C π * , which has the same probabilistic behaviour as A is weakly resilient too.

• Conversely, suppose that there is no such cycle in A. Let π be a (live) strategy such that C π is FF-diagnosable. This strategy can be mimicked in C B , ignoring the separated belief information. The reachable states of C B π are associated with separated beliefs of Win (due to the characterisation recalled above). As π * is the most permissive strategy ensuring to stay in Win, there does not exist any such cycle in C B π either. Consequently, there exists n f ∈ N such that every run ρ in C B π with |ρ| ≥ n f ends in a state which first component is faulty.

Thus P C π (C n f ) = P C B π (C n f ) = 0, which means that C π is not weakly resilient.
The complexity lower-bound is obtained by reduction from the active diagnosability problem for CLTS, which is known to be EXPTIME-hard [BFH + 14]. From C = Q, q 0 , Σ, T a CLTS, we define the CLTS C = Q ∪ {q 0 , q s }, q 0 , Σ ∪ { }, T with a fresh observable event, and such that T (q 0 , , q 0 ) = T (q 0 , , q s ) = T (q s , , q s ) = 1, for every q, q ∈ Q and a ∈ Σ, T (q, a, q ) = T (q, a, q ) and for every other triplet T (q, a, q ) = 0. Clearly enough, C is actively diagnosable iff C is actively diagnosable. Moreover, C is safe by construction, and thanks to Theorem 6.1(a), it is strongly resilient, and thus weakly resilient.

The proof of the next theorem also relies on the set of separated beliefs Win. We build a subset of Win, called WinK. A separated belief (U, V ) of Win belongs to WinK if there exists a strategy π such that from every distribution with support U ∪ V , π guarantees to stay in Win, and to give a positive probability to the set of infinite correct runs. The CLTS is actively diagnosable with guarantee of strong resiliency iff from the initial belief one can reach a belief of WinK while staying in Win. The strategy π defined with the construction of WinK does not necessarily allows to diagnose the system. So the winning strategy consists in cleverly combining the strategy used to make the system FF-diagnosable and π. Theorem 6.3. Strongly resilient active diagnosability is EXPTIME-complete.

Proof. Let C be a CLTS. As in the construction preliminary to Theorem 6.2, we build C B , Win and π * . We then define WinK U ⊆ 2 Q ×Win by a greatest fix point computation. For (U , (U, V ) ∈ WinK U , (U, V ) is a separated belief for which there exists a strategy allowing to a set of runs starting in U ∪ V to stay in the states of C B associated with a belief of Win, and if the run started in U , it stayed correct. WinK U is obtained as the limit of a non-increasing sequence (WinK n ) n∈N defined inductively by: WinK 0 = {(U , (U, V )) | (U, V ) ∈ Win∧∅ = U ⊆ U } and for n ∈ N, WinK n+1 is the set of elements (U , (U, V )) of WinK n such that there exist a set of allowed events Σ • verifying:

• Σ • does not create a deadlock: ∀q ∈ U ∪ V, G Σ • (q) = 0;
• under the control Σ • no run starting in a state of U will make a fault before the next observation:

∀q c ∈ U , ∀ρ ∈ SR 1 , q c ρ = ⇒ q ∧ P(ρ) ∈ Σ • ⇒ q ∈ Q c ;
• every triplet reached by an observable step o ∈ Σ • belongs to WinK n : ( Ũ , ( Ũ , Ṽ )) ∈ WinK n with:

1. Ũ = {q c ∈ Q c | ∃q c ∈ U 1 , ∃ρ ∈ SR 1 , q c ρ = ⇒ q c ∧ P(ρ) = a}; 2. Ũ = ∆(U, o) ∩ Q c and Ṽ = ∆(U ∪ V, o) ∩ Q f .
From WinK U , we define the set WinK ⊆ Win by keeping only the second component of

WinK U : WinK = {(U, V ) ∈ Win | ∃U , (U , (U, V )) ∈ WinK U }.
Let us state some of the properties of this construction.

• By induction, if (U , (U, V )) / ∈ WinK n then for every (live) strategy and q ∈ U , there exists a faulty run starting in q of observable length n;

• If ∅ = U ⊆ U then (U , (U, V )) ∈ WinK U implies (U , (U, V )) ∈ WinK U . Thus, if (U, V ) / ∈ WinK, for all q ∈ U , ({q}, (U, V )) / ∈ WinK U .
We also define PreWin the set of states of C B of the form Q × Win from which a state (q, U, V ) with (U, V ) ∈ WinK is reachable. Let us show that C is diagnosable and strongly resilient iff the initial state of C B belongs to PreWin.

• Suppose that the initial state belongs to PreWin. Let (U , (U, V )) be an element of WinK U . We define π (U ,(U,V )) the strategy that ensures to stay in WinK U . This strategy immediately derives from the fixpoint definition of WinK U . For (U, V ) ∈ WinK, we also define π (U,V ) = π (U ,(U,V )) for an arbitrary U such that (U , (U, V )) ∈ WinK U . Finally, we let π 0 be the following strategy working in three successive phases which may not all be triggered. 1. First π 0 mimics π * until a separated belief (U, V ) ∈ WinK is reached; 2. Then, at every observed sequence w, π 0 chooses to apply π (U,V ) with probability p w = |w| |w|+1 , and to switch to the third phase with probability 1 -p w ;

3. Finally, π 0 behaves forever as π * .

We observe that C π 0 is FF-diagnosable. Indeed, on the one hand, the events allowed by π 0 are included in those allowed by the maximally permissive strategy π * , and on the other hand almost-surely, π * is applied from some moment on. Therefore every fault will almost surely be detected. Moreover, let us prove that it is strongly resilient. Indeed, by definition of PreWin, there exists a run ρ starting in the initial state and reaching a state (q, U, V ) such that (U, V ) belongs to WinK. Let U ⊆ U the one chosen arbitrarily when defining π (U,V ) . Without loss of generality, we suppose that ρ reaches a state of U . As a fault can only be created after ρ if π 0 switches to its third phase, for n ≥ |ρ| o we have

P π 0 (ρ ∈ C n | ρ ρ) ≥ P π 0 (ρ) n i=|ρ| i i + 1 = P π 0 (ρ) |ρ| n + 1 .
Thus, for every 0 < α < 1, similarly to nα n , α n Pπ 0 (Cn) converges to 0. • Conversely, suppose that the initial state does not belong to PreWin. Let π be a strategy ensuring diagnosability. For every state (q, U, V ) with q ∈ U reachable by a run ρ 0 with π, (U, V ) ∈ WinK and due to one of our observations ({q}, (U, V )) ∈ WinK U . Let K be the number of iterations in the fixpoint computation of WinK. Then, for every sequence of K random choices under π, there exists a faulty run ρ ∈ F, compatible with these choices, starting in (q, U, V ) and of observable length smaller than K. Adding up the probabilities of runs corresponding to every sequence of choices of π we obtain

P π (ρ ∈ F |ρ 0 |o+K | ρ 0 ρ) ≥ λ K|Q| P π (ρ 0 )
where λ = min q ∈Q 1 G Σ (q ) . Thus, for every n ∈ N, P π (C n+K )

≤ P π (C n )(1 -λ K|Q| ). Letting α = (1 -λ K.|Q| ) 1 K , we obtain lim n→∞ α n
Pπ(Cn) > 0, so that C π 0 is not strongly resilient.

To conclude the proof, we observe that the EXPTIME-hardness derives from the same reduction as in the proof of Theorem 6.2.

It turns out that this same combination of strategies can be used to ensure lasting fault freeness and FF-diagnosability. In fact, the following theorem establishes that the characterisation of the strongly resilient active diagnosability also applies to the lasting fault free active diagnosability. Theorem 6.4. Lasting fault free active diagnosability is equivalent to strongly resilient active diagnosability.

We show here that the characterisation given in the proof of Theorem 6.3 for a CLTS to be actively diagnosable with guarantee of strong resiliency also characterises the fact that the CLTS is actively diagnosable with guarantee of lasting fault freeness. This shows the equivalence of the two notions in the active case.

Proof. We reuse the definitions from the proof of Theorem 6.3. Let us show that C is actively diagnosable with guarantee of lasting fault freeness iff the initial state of C B belongs to PreWin.

• Suppose that the initial state belongs to PreWin. Then, as discussed in the proof of Theorem 6.3, C π 0 is diagnosable and there exists a finite run ρ such that P(ρ ∈ C n | ρ ρ) ≥ P(ρ) |ρ| n+1 . Thus:

∞ n=1 P(C n ) ≥ ∞ n=|ρ| P(ρ ∈ C n | ρ ρ) ≥ P(ρ)|ρ| ∞ n=|ρ| 1 n + 1 = ∞.
• Conversely, if the initial state does not belong to PreWin. Let π be a strategy ensuring diagnosability. For every n ∈ N, P(C n+K ) ≤ P(C n )(1 -λ K.|Q| ). Thus:

∞ n=1 P(C n ) ≤ K ∞ n=1 (1 -λ K.|Q| ) n ≤ K.|Q B |. 1 λ K.|Q| < ∞.
Given the equivalence of strong resiliency and lasting fault freeness, from Theorem 6.3 we derive: Corollary 6.1. Lasting fault free active diagnosability is EXPTIME-complete.

Safe active diagnosis problem under finite-memory strategies

Contrary to the other qualitative problems, safe active diagnosability is known to be undecidable [BFH + 14]. In order to regain decidability, one can restrict the strategies so that they only use finite memory. Note first that decidability is not immediate even if the strategies are assumed to be finite-memory, since no a priori bound on the memory is known3 . This restriction was studied in [BFH + 14] where the authors give an NEXPTIME algorithm. However, the known lower bound is only EXPTIME, leaving a gap. We refine here this complexity result by proving that safe active diagnosis can be solved in EXPTIME when restricting to finite-memory strategies.

To do so, we prove a more general result in the context of a well-known model, quite popular in artificial intelligence and more recently in formal methods, that combines partial observation, probabilities and control, namely Partially Observable Markov Decision Processes (POMDP) [START_REF] Åström | Optimal control of markov processes with incomplete state information[END_REF][START_REF] Kaelbling | Planning and acting in partially observable stochastic domains[END_REF]. We establish that the existence of finite-memory schedulers that ensure a Büchi objective with probability 1 and a safety objective with positive probability in a POMDP is decidable in EXPTIME. We then reduce the safe active diagnosis of a CLTS C restricted to finite-memory strategies to the existence of a finite-memory scheduler in a POMDP M C ensuring at the same time a Büchi objective with probability 1 and a safety objective with positive probability. Definition 6.8. A partially observable Markov decision process (POMDP) is a tuple M = Q, q 0 , Obs, Act, T where • Q is a finite set of states with q 0 the initial state;

• Obs : Q → O ∪ { } assigns an observation O ∈ O to each state.
• Act is a finite set of actions;

• T : Q × Act → Dist(Q) is a partial transition function. Letting Ena(q) = {a ∈ Act | T (q, a)
is defined} the set of enabled actions in state q, we assume that:

for all q ∈ Q, Ena(q) = ∅, and whenever Obs(q) = Obs(q ), then Ena(q) = Ena(q ) and slightly abusing our notation, we denote by Ena(O) the set of events enabled in every state with observation O.

A decision rule of a POMDP is a distribution from Dist(Act) that resolves one nondeterminism choice by randomization. A scheduler for a POMDP maps histories of observations to decision rules. Formally, a scheduler is a function τ : O + → Dist(Act)

such that for every O 1 • • • O i , Supp(τ (O 1 • • • O i )) ⊆ Ena(O i ).
Given a scheduler τ , a POMDP M yields a stochastic process. This stochastic process can be represented by an infinite state pLTS, denoted M(τ ) in which states are histories of observations. One denotes by P M τ (Ev) the probability that an infinite observed sequence of Ev is realized in this pLTS.

Similarly to what was said for strategies, one can define finite-memory schedulers for POMDP. The notion of belief can be adapted to POMDP. As for CLTS, it is a nonempty set of states that represents the current state estimate, i.e. the set of states the system may be in, given the actions (which affect the reachable set of states contrary to what is done for CLTS) and observations so far. The initial belief is {q 0 }, and given a current belief B, a decision rule δ and an observation O, the belief obtained after δ has been applied and O has been observed is defined by:

∆(B, (δ, O)) = q∈B, a∈Supp(δ) Supp(T (q, a)) ∩ Obs -1 (O) .
Aiming at providing a POMDP M C for the safe active diagnosis problems of a CLTS C, we face several difficulties. First, in a CLTS the observations are related to events while in a POMDP they are related to states. As a consequence, we need to label the states by the latest observation made by the system. Secondly, our objectives are not based on states but on observed sequences. Fortunately, the relevant information pertaining to the observations, namely the information about ambiguity of observed sequences, is available in the belief. Thus (with two exceptions) the states are triples formed of a state q, an event 'a' and a belief B of the CLTS. A third adaptation concerns the control mechanism. In C, the control is performed by choosing (possibly randomly) a subset of allowed controllable events. Thus actions of M C are subsets of events that include the uncontrollable events. Given some control decision Σ • , to define the transition probability of M C from (q, a, B) to (q , a , B ), one must consider all runs in C labelled by events of Σ • from q to q such that the last event, labelled by 'a ', is the only observable one. The probability of any such run is obtained by the product of the individual step probabilities. The latter are then defined by the normalization of weights w.r.t. Σ • . Finally, there cannot be infinite runs of unobservable events due to the convergence of C. However some runs can reach, via unobservable events, a state from which no event of Σ • is enabled. In other words, the control Σ • applied in (q, a, B) may have a positive probability to reach a deadlock (i.e. the chosen decision rule leads to a strategy for the CLTS which is not live). In order to capture this behaviour and to obtain a non defective probability distribution, we add an additional state lost, that corresponds to such deadlocks. The next definition formalizes our approach. Definition 6.9. The POMDP M C = Q M C , q M C 0 , Obs, Act, T M C derived from a CLTS C = Q, q 0 , Σ, T is defined by:

• Q M C = Q × Σ o × Bl C {(q 0 ,
ε, {q 0 }), lost} with q M C 0 = (q 0 , ε, {q 0 });

• the set of observations is O = Σ o ∪{lost}, with Obs(lost) = lost and for (q, a, B) ∈ Q M C , Obs((q, a, B)) = a;

• Act = {Σ • ⊆ Σ | Σ • ⊇ Σ \ Σ c }; • for every (q 1 , a, B) ∈ Q M C and Σ • ∈ Act, T M C ((q 1 , a, B), Σ • ) = µ ∈ Dist(Q M ) where for b ∈ Σ • ∩ Σ o : -µ((q , b, ∆(B, b))) = q 1 a 1 -→q 2 ••• an -→q n+1 b - →q a 1 •••an∈Σ • ∩Σu n i=1 T Σ • (q i , a i , q i+1 ) • T Σ • (q n+1 , b, q ); -µ(lost) = q 1 a 1 -→q 2 ••• an -→q n+1 a 1 •••an∈Σ • ∩Σu G Σ • (q n+1 )=0 n i=1 T Σ • (q i , a i , q i+1 ); • for every Σ • ∈ Act, T M C (lost, Σ • ) = 1 lost .
Given C, the construction of M C , which is of size in 2 O(|Q|+|Σ|) , can be done in exponential time. Also, the probability distributions over next states (µ in Definition 6.9) are presented as sums over runs of C, but they can be computed in polynomial time by matrix operations.

A CLTS C and its associated POMDP M C are closely related. In particular, strategies in C and schedulers in M C are in a one-to-one correspondence. First, let us explain how to naturally derive a strategy π for C from a scheduler τ in M C . For an observed sequence

a 1 • • • a n ∈ Σ * o , we set π(a 1 • • • a n ) = τ (a 1 • • • a n )
Notice that the strategy π obtained that way is not necessarily live: for example, if after a 1 • • • a n the choice of τ leads with positive probability to lost, then π is not live. However, as soon as τ ensures to avoid state lost, then the corresponding strategy π is live. Similarly, to a live strategy π for C, we can associate a scheduler τ in M C that always avoids lost: given a sequence of observations that does not contain lost, thus of the form

a 1 • • • a n , with a i ∈ Σ o for all i, we set τ (a 1 • • • a n ) = π(a 1 • • • a n ).
Moreover, if (π, τ ) is a pair of live strategy and corresponding scheduler (that always avoids lost), the probability measures P Cπ and P M C τ are essentially equivalent. More precisely, the product in M C with the observation and the belief does not change the probability measure defined by C π .

We now show how to decide for POMDP the existence of a finite-memory scheduler that ensures a Büchi objective with probability one and a safety objective with positive probability. We use LTL notations to denote sets of runs in a POMDP, such as 3, 2 and 23 for eventually, always and infinitely often respectively (given a state q, 23q thus represents the set of runs containing q infinitely often).

Theorem 6.5. The problem whether, given a POMDP M with subsets of states F and I, there exists a finite-memory scheduler τ such that P M τ (23F ) = 1 and

P M τ (2I) > 0 is EXPTIME-complete.
there is a positive probability, under τ to reach a pair of beliefs out of W k-1 , and thus out of Win =1 by induction hypothesis. As the sequence of action and observations was chosen so that one can reach F from q, the second case implies that the first case holds with our selected sequence of actions and observations. For the third case, let

(B 3 , B 3 ) = ∆ (B , B), (α 1 , O 1 ) • • • (α i-1 , O i-1 )(α i , O i ) .
Either there exists q ∈ B 3 such that (q, B 3 , B 3 ) / ∈ W k-1 , then it is treated similarly to the first case. Else B 3 / ∈ Bl ⊆I . Observe that, in this case, the second requirement on τ is not satisfied since

P δ 0 π (2I) < 1.
Thanks to Lemma 6.1, Win =1 can be computed in EXPTIME. Let us now define Lose as the set of beliefs that are clearly losing:

Lose = {B ∈ Bl | ¬∃τ ∀δ 0 with Supp(δ 0 ) = B, P δ 0 τ (23F ) = 1} .
As established e.g. in [START_REF] Bertrand | Qualitative determinacy and decidability of stochastic games with signals[END_REF] in the more general framework of 2-player stochastic games with signals, Lose can also be computed in EXPTIME.

Informally, we now consider the set of beliefs from which one can reach, while staying in I, and not risking to fall in Lose, some belief B such that there exists B = ∅ with (B , B) ∈ Win =1 . In order to easily represent what staying in I means, we assume without loss of generality that the set of states Q \ I is absorbing4 . Formally, let Win be the following set of beliefs:

Win = {B 0 ∈ Bl | ∃(B , B) ∈ Win =1 s.t. B = ∅ and ∃α 1 • • • α n , ∃O 1 • • • O n , ∆ B 0 , (α 1 , O 1 ) • • • (α n , O n ) = B ∀i ≤ n, ∀O i , ∆ B 0 , (α 1 , O 1 ) • • • (α i-1 , O i-1 )(α i , O i ) / ∈ Lose}.
The set Win characterizes winning beliefs, that is, beliefs from which there exists a finitememory scheduler (called a winning scheduler) ensuring at the same time, the Büchi objective 23F almost-surely, and the safety objective 2I with positive probability. Formally:

Lemma 6.2. B 0 ∈ Win if and only if for every δ 0 with Supp(δ 0 ) = B 0 , there exists a finite-memory scheduler τ such that P δ 0 τ (23F ) = 1 and P δ 0 τ (2I) > 0.

Proof of Lemma 6.2. Assume first that B 0 ∈ Win. We design a finite memory scheduler τ that is winning from any initial distribution δ 0 with support B 0 . In a first mode, τ aims at reaching a pair of beliefs (B , B) ∈ Win =1 from B 0 . More precisely, τ plays the sequence of actions that leads with positive probability from B 0 to some B ∈ Bl such that there exists B = ∅ with (B , B) ∈ Win =1 . If this succeeds, τ then switches to another mode, where it behaves as the winning scheduler that starts from (B , B) in Lemma 6.1. If it fails, the play ends in a belief B 1 / ∈ Lose (by definition of Win), and from there τ plays to ensure visiting F infinitely often with probability 1. All in all, τ ensures almost surely visiting F infinitely often, and with positive probability (the probability of the prefix leading to B, times the probability that the play is in B at that time point) to stay in I. Note that the size of the memory τ uses is in O(|Bl| 2 ). Indeed, in its first phase, it tries to reach a belief by using a set of actions of length smaller than |Bl| as it does not need to visit the same belief twice. If it fails to reach the target belief, then ensuring the Büchi requirement can be done with a belief-based scheduler, i.e. a scheduler that only remembers the current belief, thus with memory of size |Bl|. If it reaches its target however, it needs to remember pairs of beliefs as done in Lemma 6.1, thus requires a memory of size |Bl| 2 .

Let now δ 0 be an initial distribution with support B 0 , and assume that there exists a finite-memory scheduler τ such that P δ 0 τ (23F ) = 1 and P δ 0 τ (2I) > 0. We consider M(τ ) the pLTS generated by τ , with finite state space Q × Mem, where Mem is a finite set of memory states. Without loss of generality, we iteratively tag each state of M(τ ) with its associated belief. Since τ is winning and almost surely a run reach a BSCC, there must exist a BSCC C in M τ , reachable from some (q 0 , m 0 ) via an I-run ρ (a run where all state are included in I), and such that all states (q, m) ∈ C satisfy q ∈ I, and there exists a state (q f , m f ) ∈ C such that q f ∈ F . Let (q, m) ∈ C be the state reached by run ρ, B be the belief obtained after observing ρ. From (q, m), under scheduler τ , all plays stay in I. Moreover, for any q ∈ B, from (q , m), under scheduler τ , almost all runs visit F infinitely often. As a consequence, by the definition of Win =1 , ({q}, B) ∈ Win =1 . Then, we conclude that B 0 ∈ Win, exploiting the I-run ρ, and the fact that τ ensures 23F almost-surely, and thus always avoids Lose.

Win characterizes the winning beliefs, and can be computed in EXPTIME. We thus showed the computability in EXPTIME of the set of supports B from which for every distribution δ 0 with Supp(δ 0 ) = B there exists a finite-memory scheduler τ such that P δ 0 τ (23F ) = 1 and P δ 0 τ (2I) > 0.

Now the safe active diagnosis restricted to finite-memory strategies can be reduced to the existence for POMDP of a finite-memory scheduler that ensures a Büchi objective almost surely, and a safety objective with positive probability. As M C is exponential in the size of C and the algorithm on the POMDP is in EXPTIME, we obtain a 2EX-PTIME complexity upper-bound. Fortunately, in order to avoid a doubly exponential blowup and to establish the EXPTIME complexity, we observe that the exponential comes in both cases from the computation of beliefs depending only on the original CLTS. This implies that the safe active probabilistic diagnosis problem is in EXPTIME when restricted to finite-memory strategies.

Corollary 6.2. The safe active diagnosis problem restricted to finite-memory strategies is decidable in EXPTIME.

Proof. Given a CLTS C, we build M C and decide if there exists a scheduler τ ensuring P δ 0 τ (23F ) = 1 and P δ 0 τ (2I) > 0 with I = {(q, a, B)

| q ∈ Q c } and F = {(q, a, B) | B ⊆ Q f ∨ q ∈ Q c }
and δ 0 is the Dirac distribution of support {q M C 0 }. Due to the link between M C and C, this choice of F corresponds to runs that are either correct or surely faulty in C and this choice of I corresponds to runs that are correct. Thus there exists a finite-memory scheduler τ as defined above iff the corresponding strategy π in C ensures safe active diagnosis. Moreover, as explained above the corollary, deciding the existence of this scheduler can be done in EXPTIME.

A matching lower-bound is already known from the literature: Proposition 6.5 ([BFH + 14]). The safe active diagnosis problem restricted to finitememory strategies is EXPTIME-hard.

Obviously, this lower bound also holds for the more general problem: on POMDP, whether there exists a finite-memory strategy ensuring a Büchi objective almost-surely and a safety objective with positive probability.

Conclusion

Degradation of a controllable probabilistic system combines two objectives. The system must satisfy at the same time a diagnosability and a degradation condition. Interestingly and as shown first in [BFH + 14], having to satisfy both conditions at the same time increases the difficulty: safe active diagnosability combines two decidable problems yet ends up being undecidable. In order to regain decidability, we introduced two new degradation notions both in a qualitative and a quantitative ways. While the quantitative versions are undecidable, the qualitative ones brings interesting results. Indeed, on the one hand, they are close to safe active diagnosability as two of the notions are equivalent to it for finite pLTS. On the other hand, they are decidable in EXPTIME. As EXPTIME is the lower bound of the complexity of active diagnosability, it is unsurprisingly also a lower bound of the complexity of the combination of active diagnosability and of a degradation condition. Therefore we can test the combination of active diagnosability with a degradation condition without reaching a new complexity class.

This analysis however can result in diagnosers requiring infinite memory. When restricted to finite-memory controllers, many differences appear. First, as the pLTS obtained by controlling a CLTS with a finite-memory strategy is finite, then according to Theorem 6.1, safety, strong resiliency and lasting fault freeness are equivalent. Studying the safe active diagnosability, we showed it to be EXPTIME-complete. Thus, the restriction to finite memory helped regain decidability. For weakly resilient active diagnosability, the restriction to finite memory is not necessary as, as shown in Theorem 6.2, if the system is weakly resilient active diagnosable, there exists a strategy with finite memory.

The notions of degradation introduced here were inspired from the notion of safe active diagnosability. One could be interested in other notions of degradations representing different forms of failures within the system. For example, in our framework, the notion of faulty run is a Boolean one; once a fault occurred, the run is faulty. The fault is thus seen as a definitive and complete damage of the system. But a fault could only represent a small degradation of the system which would still be partially available. In this alternative framework, the degradation to be evaluated would be the evolution of the number of faults in a run w.r.t. its length. Another possible direction of research would be to take a more general approach to the notion of combined objectives. It would be interesting to determine which pairs of objectives can be studied separately and which ones lead to undecidability.

Chapter 7

Opacity

This thesis consists in a study of the control of the information in probabilistic systems, mainly focusing on diagnosis. While the goal of diagnosis is to analyse an observation in order to reveal a hidden information (the fault), one could be interested in asking a dual question: can we limit the amount of hidden information that is revealed by the system. This question belongs to an important domain of partial observation issues called opacity. While the two notions can appear similar, the motivations behind them are different. Diagnosability is a notion of safety of a system while opacity is one of security. This difference in motivations implies that the questions asked for opacity are not be the same as the ones asked for diagnosis. Moreover, the model itself can differ in order to possess different properties. In this introduction, we first present informally opacity and some questions related to this domain of research: in the passive framework then in the active one. Finally, we discuss the form of control used in the active opacity framework, emphasising the differences with active diagnosability.

Opacity problems for passive systems. Given a set of secret runs, a run discloses the secret if every run with the same observed sequence is secret. With this definition, the disclosure of the secret is akin to exact diagnosability. One can also define a notion of disclosure that would resemble approximate diagnosability: for ε > 0, a secret run ε-discloses the secret if the probability of runs with the same observed sequence that are not secret conditioned on the probability of the observed sequence is at most ε. For non-probabilistic systems, opacity boils down to detecting if there exist a run disclosing the secret to an observer. For probabilistic systems, we are interested in quantifying the opacity of the system [BMS15, SH14, BKM12]. For instance we would want to determine if the measure of disclosing runs of a given length is positive, if it is above a certain threshold, and how this measure evolves with the observed length of the runs. This precise quantification of the measure of disclosing runs, called disclosure, does not appear in the diagnosability notions we studied. Indeed, a fault was considered a dangerous event which cannot, in any case, be missed. For the disclosing of the secret however, it is more usual to tolerate that part of the secret may be leaked, as long as it is a limited amount.

Opacity

Opacity problems for active systems. The focus on disclosure takes a whole new meaning when considering active systems. Indeed, a controller will have an effect on the system, and depending on its choices, this disclosure will increase or decrease. Existing works study the case where the controller maximises the disclosure [START_REF] Bérard | Probabilistic opacity for Markov decision processes[END_REF][START_REF] Bérard | Opacity for linear constraint Markov chains[END_REF]. This corresponds to a worst-case analysis of the system, i.e. for the worst possible control. This kind of analysis aims at representing the case where the control is done by an attacker that is observing the system. This control can be obtained by the attacker by using a virus for example. The opposite direction, where the control tries to minimise the disclosure is also worthy of analysis. Indeed, for example, if a system has been designed in order to satisfy a specification, yet there are still some liberties within the system, some choices that are possible and that do not affect the specification, then these choices can be made in order to optimise opacity of the system. In this case, the control is realised during the design of the system and is made in order to minimise the disclosure of the system. Thus, both maximisation and minimisation corresponds to real issues.

Formalisation of the control. When we studied active diagnosis in Chapter 6, the control was exterior to the system: from the observations it received, it was able to stop some controllable actions from occurring. The controller and the observer had the same information and thus could be thought as the same mechanism. In the examples given earlier (the virus for maximisation and the system design for the minimisation), the control comes from within the system. There is thus a clear separation between the controller and the observer/attacker. To formalise this opposition, the control is realised with a full knowledge of the system: it knows what is the exact run that is followed and especially what is the current state and make its decision based on this. However, as the attacker is not himself within the system a run only discloses the secret with its observation. In other words, the controller will try to minimise or maximise a set of runs satisfying a condition that is based on their observations. Moreover, we assume the attacker is aware of how the controller makes its choices. Indeed, the security of the system should not be based on the black box hypothesis (that an attacker is lacking information). Especially when considering cases such as the virus example: the virus could very well have been implanted in the system by the attacker, ensuring he is aware of how the virus works.

In Section 1, we establish the specifications and important questions of opacity that we consider throughout the chapter. These definitions present two different horizons over which to consider opacity: a given fixed horizon and an unbounded yet finite horizon. In Section 2 and Section 3, we study opacity over finite horizon for maximisation and minimisation respectively. These two sections echo one another emphasising the differences between the two. Finally in Section 4 we detail our results for opacity over fixed horizon for both maximisation and minimisation.

This chapter develops and extends some of the results from [START_REF] Bérard | Probabilistic disclosure: Maximisation vs. minimisation[END_REF].

Specification for Opacity

The notion of opacity is very similar to the one of diagnosis. They both consider the information revealed by runs of partially observable systems. The difference is that the goal of diagnosis is to reveal an information about the current state of the system, while opacity tries to hide an information. Despite these similarities, the framework of opacity contains important differences with the one of diagnosis. We first define formally the framework of opacity for passive systems (Subsection 1.1), then extend the definitions to allow a control of the system (Subsection 1.2).

Opacity for Markov chains

Labelled Markov chains, as introduced in Chapter 4, are pLTS where every event is observable. We now define another kind of Markov chains called observable Markov chains which are pLTS where the observation is associated with the state instead of the transition. This labelling of states thus describes what an external observer can see and is given by an observation function1 . We use this new framework as, while diagnosis aimed to detect an event (the fault), opacity consists in hiding that the system is currently in a secret behaviour represented by its state. This could however easily be translated as the detection of a transition triggered when entering a secret state. In fact, the equivalence of associating events with transitions or with states is a folk result. Figure 7.1 gives the informal idea of how to push events from states to transitions and Theorem 7.1 starts by a modification of the system that is close to the usual transformation allowing to associate the labels with the states rather than with the transitions. We write p(s |s) instead of p(s)(s ) to emphasise the fact that the probability of going to state s is conditioned by being in state s. Given a distribution µ 0 on S, we Opacity denote by M(µ 0 ) the Markov chain with initial distribution µ 0 . The definitions of runs, observed sequences, probability measure, . . . can easily be adapted to OMC. To give an example, an infinite run of M(µ 0 ) is a sequence of states ρ = s 0 s 1 . . . ∈ S ω such that µ 0 (s 0 ) > 0 and for each i ≥ 0, p(s i+1 |s i ) > 0. The observed sequence of this infinite run is

O(ρ) = O(s 0 )O(s 1 )... ∈ Σ ∞ o .
The observation function is called non erasing if O(S) ⊆ Σ (all states are visible).

As in opacity one aims to hide a secret behaviour of the system, a way to represent what is secret is needed. There are various ways to define it depending on if we want the secret to be permanent, intermittent, described within the system. . . We consider here the case where the secret is permanent and given by a subset of states Sec ⊆ S of the model: a (finite of infinite) run s 0 s 1 . . . is secret if s i ∈ Sec for some i, otherwise it is public. Under this choice, the secret itself behaves very similarly to the fault. As for pLTS where we could make the partition between faulty and correct states without loss of generality, we assume here that the set of secret states Sec is absorbing. To show this can be done without loss of generality, a new Markov chain M = (S , p , O ) is defined from M by: S = (S × {0, 1}), where (s, 0) represents state s where the secret has not been visited while (s, 1) represents the opposite situation. The transitions are then duplicated accordingly: (1) p ((s , i)|(s, i)) = p(s |s) for all s ∈ S, s ∈ S \ Sec, and i ∈ {0, 1}, (2) p ((s , 1)|(s, i)) = p(s |s) for all s ∈ S, s ∈ Sec, and i ∈ {0, 1}. The new observation function is defined by O ((s, i)) = O(s) for all s ∈ S and i = 0, 1 and the new set of secrets is S × {1}. There is a one-to-one probability-preserving correspondence between the runs in M and the ones in M .

Example 7.1. Consider the OMC of Figure 7.2 with initial distribution 1 q 0 . The observation associated with a state by the observation function is displayed next to it. The secret state is shaded. Assuming o 1 = ε and o 2 = ε, every state is associated with an observation different than ε, therefore the observation function is non-erasing.

q 0 o 1 q s o 1 q 11 o 2 q 21 o 1 q 22 o 2 . . . q i1 o 1 . . . i -2 steps q i(i-1) o 1 q ii o 2 . . . 1 1 1 1 1 2 1 4 1 8 1 2 i+1 1 1 1 1 Figure 7
.2: An infinitely-branching OMC with Sec = {q s }.

One quantitative way to define the disclosure of a system is to consider that a run discloses the secret if the probability that the current run belongs to the secret, conditioned over the observation of the run, is greater than some given threshold ε > 0.

Definition 7.2. Given an OMC M = (S, p, O), an initial distribution µ 0 , Sec ⊆ S and an observation w ∈ Σ * , the proportion of secret runs with observation w is:

Psec M(µ 0 ) (w) = P M(µ 0 ) ({ρ ∈ O -1 (w) | ρ is secret}) P M(µ 0 ) (w)
.

For ε > 0, w is ε-min-disclosing if Psec M(µ 0 ) (w) > 1 -ε and no prefix of w satisfies this inequality. Writing D ε min for the set of ε-min-disclosing observations, the εdisclosure is defined by Disc ε (M(µ 0 )) = w∈D ε min P M(µ 0 ) (w). The positive ε-disclosure problem consists in deciding whether Disc ε (M(µ 0 )) > 0.

To establish a parallel with diagnosability, positive ε-disclosure is similar to εFFdiagnosability. Indeed, in εFF-diagnosability (resp. the positive ε-disclosure problem) one considers the fault (resp. secret) revealed if the likelihood of the fault (resp. secret) conditioned on the observation is above 1 -ε. The difference is that for opacity, we ask whether the measure of the set of runs disclosing the secret is positive while for diagnosability, we require this probability to be equal to the probability of faulty runs. In other words, considering a run to be faulty iff it is secret, the system is εFF-diagnosable iff Disc ε (M(µ 0 )) = P(F ∞ ).

In this chapter, we aim at studying active notions of opacity. While being the most realistic notion of probabilistic disclosure, ε-disclosure is unfortunately a too complex notion. Indeed, the problem is already undecidable for OMC:

Theorem 7.1. The positive ε-disclosure problem is undecidable for OMC.

To establish this undecidability result, we reduce the emptiness problem for probabilistic automata. The reduction itself is pretty straightforward. Note however that the reduction requires to first translate the PA into an OMC such that the probability of acceptance of a word w in the PA is equal to the probability to end in a secret state in the Markov chain knowing that the run has observed sequence w.

The emptiness problem and the value 1 problem 2 are undecidable for PA already with a two-letter alphabet [START_REF] Paz | Introduction to probabilistic automata[END_REF][START_REF] Gimbert | Probabilistic automata on finite words: Decidable and undecidable problems[END_REF]. Hence in the various reductions we use the alphabet {a, b}.

Proof. Given a PA A = (Q, q 0 , {a, b}, (P a , P b ), F ) that we suppose complete without loss of generality, we first transform A into an incomplete OMC  where {a, b, } is the observation alphabet (an illustration is given in Figure 7.3). The set of states is Q = Q ∪ {q tag | q ∈ Q ∧ tag ∈ {a, b}}, with initial distribution 1 q 0 . The observation function Ô is defined by Ô(q) = and Ô(q c ) = c for q ∈ Q and c ∈ {a, b}. The transition function p is defined for q, q ∈ Q and c ∈ {a, b} by p(q | q c ) = P c (q, q ) and p(q c | q) = 1 4 . This OMC is incomplete as for every state q ∈ Q, the sum of the probabilities exiting q in  is 1/2.

2 These notions were defined before Theorem 4.3, page 115 Opacity A : We now build the OMC M A = (S, p, O) over alphabet {a, b, , } by adding two states to complete  (see Figure 7.4 where the doubly circled state q f is a final state of A):

q q 2 q 1 q 3  : q a q a b q b q 1 q 2 q 3 a, 1 2 a, 1 2 b, 2 3 b, 1 3 1 4 1 4 1 2 1 2 2 3 1 3 Figure 7.3: From PA A to incomplete OMC Â. q 0 q q f  s pub s sec 1 1 1 2 1 2
• S = {s pub , s sec } ∪ Q, with Sec = {s sec };
• The function p is obtained from p by adding the transitions: For every q ∈ F , p(s sec | q) = 1 2 , for every q ∈ Q \ F , p(s pub | q) = 1 2 , and p(s pub | s pub ) = p(s sec | s sec ) = 1;

• O extends Ô by O(s sec ) = O(s pub ) = .
We now prove that, given ε ∈]0, 1[, A accepts a word with probability strictly greater than 1 -ε iff Disc ε (M(µ 0 )) > 0. First assume that there exists a word w = a 1 . . . a n ∈ {a, b} * with P A (w) > 1 -ε. Then w corresponds to a non secret run with observed sequence ŵ = a 1 . . . a n in M A and Psec M(µ 0 ) ( ŵ ) = P A (w) > 1 -ε, which implies Disc ε (M(µ 0 )) > 0.

Opacity

Example 7.2. Consider the infinitely-branching OMC of Figure 7.2. The single secret run is SecRuns = {q 0 q ω s } and its observation is o ω 1 . Moreover, the observations of the public (as opposed to secret) infinite runs is O(SecRuns) = o + 1 o ω 2 . As a consequence, the infinite observed sequence of the secret run discloses the secret, Disc ω = 1 2 . However, no finite observation is disclosing: ∀n ∈ N, Disc n = Disc = 0.

This shows that disclosure and ω-disclosure differ.

However, both notions coincide for convergent finitely-branching OMC.

Lemma 7.1. Let M = (S, p, O) be a Markov chain, µ 0 an initial distribution and Sec ⊆ S. For SecRuns = Reach(Sec), Disc(M(µ 0 )) ≤ Disc ω (M(µ 0 )) with equality when M is convergent and finitely branching.

Proof. We first establish the following claim: if M is convergent and finitely branching, then the set of runs ρ such that O(ρ) has length n is finite for any n > 0.

We first prove the claim for signalling runs. We proceed by induction on the observable length. There exist finitely many signalling runs of length 0: by convention they are the runs that (1) do not contain any event and (2) start in an unobservable state of Supp(µ 0 ) ∩ O -1 (ε). Let us assume the hypothesis holds for n ∈ N. For every signalling run ρ 0 with |ρ 0 | o = n we consider the tree formed by the set O n+1 = {ρ ∈ SR | ρ 0 ρ ∧ |O(ρ)| = n + 1} by sharing common prefixes. Internal nodes of this tree correspond to unobservable states while all leaves are observable. Since the OMC is finitely branching, the tree is of bounded degree. By contradiction, assume that the tree is infinite. König's lemma yields an infinite branch containing only unobservable states, which contradicts the convergence hypothesis. Therefore there exist only finitely many signalling runs of observable length n + 1 extending ρ 0 . As there exist finitely many signalling runs of observable length n according to the induction hypothesis, one deduces that there exist finitely many signalling runs of observable length n + 13 . This concludes the induction. The result can then be extended to every runs as, from the convergence hypothesis, for every n ∈ N and every run ρ of observable length n, there exists a signalling run ρ ∈ SR n+1 such that ρ ρ . As |SR n+1 | < ∞, there are finitely many runs of observable length n.

We now prove that the set of infinite runs V = ∪ w∈D ∪ ρ∈O -1 (w) Cyl(ρ) is contained in SecRuns \ O -1 (O(SecRuns)). Let ρ 1 be an infinite run in V. Then there is a disclosing observation w 1 ∈ Σ * and a signalling prefix ρ 1 of ρ 1 such that O(ρ 1 ) = w 1 and ρ 1 is secret. For any infinite run ρ 2 such that O(ρ 1 ) = O(ρ 2 ), the observation w 1 is also a prefix of O(ρ 2 ), hence there is a finite signalling prefix ρ 2 of ρ 2 such that O(ρ 2 ) = w 1 . Since w 1 is disclosing, ρ 2 is also secret, hence ρ 1 belongs to SecRuns \ O -1 (O(SecRuns)) and Disc(M(µ 0 )) ≤ Disc ω (M(µ 0 )).

For the converse inclusion, let ρ be an infinite run in SecRuns\O -1 (O(SecRuns)) with observation O(ρ) = w = o 1 o 2 . . . ∈ Σ ω . We prove by contradiction that there is a finite disclosing prefix ŵ of w and a signalling prefix ρ of ρ such that ρ ∈ Cyl(ρ) and O(ρ) = ŵ.

Otherwise, for any n ≥ 1, w n = o 1 . . . o n is not disclosing and there exists a signalling run ρ n such that O(ρ n ) = w n but ρ n is not secret. The set T = {ρ ∈ SR | ∃n ρ ≤ ρ n } of all signalling prefixes of the ρ n 's form a tree: the root of the tree is ε and the nodes at level k are the prefixes with observation w

k , {ρ ∈ T | |O(ρ )| = k}. A node ρ is a child of ρ if |O(ρ )| = w k , |O(ρ )| = w k+1 for some k and ρ ρ .
From the claim, we know that T is of bounded degree. Assuming that it is infinite, König's lemma again yields an infinite branch ρ ∞ such that each prefix of length k is not secret and has observation w k . Hence ρ ∞ is not secret and has observation O(ρ ∞ ) = w, which is a contradiction.

In the following we only consider finitely-branching convergent systems. As a consequence, we will only focus on disclosure over fixed or finite horizon.

Opacity for Markov Decision Processes

We now want to add control to the system. The form of control we define here differs from the one used in Chapter 6. Indeed, in the context of diagnosability, the controller observed the system and from this observation, they chose some controllable actions that they blocked until the next observation. Thus the control had to make decisions without an exact knowledge of the state of the system. For opacity we are interested by a control acting with full knowledge. Therefore, the control can be more accurate. We use Markov decision processes where in each state there is a set of possible actions and the controller chooses one of them. This action induces a probability distribution on the next state reached. Definition 7.5. An observable Markov Decision Process (OMDP) over alphabet Σ is a tuple M = (S, Act, p, O) where S is a finite set of states, Act = ∪ s∈S A(s) where A(s) is a finite non-empty set of actions for each state s ∈ S, p : S × Act → Dist(S) is a (partial) transition function defined for (s, a) when a ∈ A(s) and O : S → Σ ∪ {ε} is the observation function.

The difference with POMDP described in Definition 6.8, page 6.8, beside some syntactic modifications is on how the control is defined. As for OMC, we write p(s |s, a) instead of p(s, a)(s ). We use the same kind of definitions as usual for runs, observed sequences, . . . For example, given an initial distribution µ 0 , an infinite run of M is a sequence ρ = s 0 a 0 s 1 a 1 . . . where µ 0 (s 0 ) > 0 and p(s i+1 |s i , a i ) > 0, for s i ∈ S, a i ∈ A(s i ), for all i ≥ 0. We denote by M(µ 0 ) the OMDP M with initial distribution µ 0 . For decidability and complexity results, we assume that all probabilities occurring in the model (transition probabilities and initial distribution) are rational.

Example 7.3. Consider the OMDP of Figure 7.5. From the initial state q 0 , two actions are possible. If action a is chosen, the system moves to q 1 with probability 1/2 and to q 2 with probability 1/2. If b is chosen, every state has a probability 1/3 to be reached.

The OMDP model uses both non-deterministic choice (the choice of the action) and probabilistic choice (the induced distribution). The non-determinism is where the Given a strategy σ, a run ρ = s 0 a 0 s 1 a 1 . . . of M is σ-compatible if for all i, a i ∈ Supp(σ(s 0 a 0 s 1 a 1 . . . s i )).

Opacity q 0 o 2 q 2 o 2 q 1 o 1 a, 1 2 | b, 1 3 a, 1 2 | b, 1 3 b, 1 3 a, 1 a, 1
In order to apply the strategies as defined here one requires to remember the whole run that occurred. Moreover, the strategies are allowed to choose randomly between the different allowed actions. All of this may not always be necessary however. We are thus interested specifically in strategies satisfying specific properties. A strategy σ is deterministic if σ(ρ) is a Dirac distribution for each finite run ρ. In this case, we denote by σ(ρ) the single action a ∈ A(last(ρ)) such that σ(ρ) = 1 a . A strategy σ is observation-based if for any finite run ρ, σ(ρ) only depends on (1) the observed sequence O(ρ) and (2) the current state last(ρ), i.e. given ρ such that O(ρ) = O(ρ ) and last(ρ) = last(ρ ), we have σ(ρ) = σ(ρ ). We then write σ(O(ρ), last(ρ)) for σ(ρ).

Let σ be a strategy and ρ be a σ-compatible run. We define B σ ρ the belief of ρ w.r.t. σ about states as follows:

B σ ρ = {s ∈ S | ∃ρ σ-compatible, O(ρ ) = O(ρ) ∧ s = last(ρ ) ∧ O(s) = ε}.
The belief B σ ρ contains the set of states that can be reached under the strategy σ and with observation O(ρ). A strategy σ is belief-based if for all finite run ρ, σ(ρ) only depends on the belief B σ ρ and the current state last(ρ), i.e. given ρ such that B σ ρ = B σ ρ and last(ρ) = last(ρ ), we have σ(ρ) = σ(ρ ). Observe that a belief-based strategy is observation-based since B σ ρ only depends on w = O(ρ). So we also write B σ w for B σ ρ . A strategy σ is memoryless if σ(ρ) only depends on last(ρ) for all ρ.

The semantics of a OMDP M with initial distribution µ 0 under the strategy σ is a (possibly infinite)observable Markov chain M σ (µ 0 ) where each state is associated with a finite σ-compatible run of M(µ 0 ), that can be equipped with the observation function mapping O(last(ρ)) to the state associated with the finite run ρ. The transition function p σ is defined for ρ a finite run and ρ = ρas by p σ (ρ |ρ) = σ(ρ)(a)p(s |s, a) and we denote by P Mσ(µ 0 ) (or P σ for short when there is no ambiguity) the associated probability measure. Writing V σ (µ 0 ) for the set of runs disclosing the secret in M σ (µ 0 ), we have Disc(M σ (µ 0 )) = P Mσ(µ 0 ) (V σ (µ 0 )). We assume all OMDP considered are convergent (there is no cycle of unobservable states), which implies the convergence of all OMC induced by strategies.

Example 7.4. Consider the OMC of Figure 7.6. It represents the semantics of the OMDP of Figure 7.5 with the strategy σ choosing the action b initially, then always choosing action a. σ is observation-based as the only run for which it does not select a is the empty run, which is the only run with observed sequence o 2 . It is also belief-based as the empty run is the only run with belief {q 0 }. Indeed, after some observations, the current belief is either {q 0 , q 2 } or {q 1 }. It is however not memoryless as the empty run and q 0 bq 0 both ends in q 0 but the same action is not chosen in both cases. After three observations, under σ, the system cannot be in q 0 any more, it is thus necessarily in a secret state. Therefore, Disc(M σ (µ 0 )) = 1. The control can be either adversarial or cooperative with respect to the system: it Opacity can try to either maximise or minimise the opacity. We therefore define the disclosure value of an OMDP according to the type of the strategies. We only consider ε-disclosure for fixed horizon in light of the undecidability result of Theorem 7.1. Definition 7.7. Given an OMDP M = (S, Act, p, O), an initial distribution µ 0 and a secret Sec ⊆ S,

q 0 o 2 q 0 bq 0 o 2 q 0 bq 2 o 2 q 0 bq 1 o 1 q 0 bq 0 aq 2 o 2 q 0 bq 0 aq 1 o 1 1 3 1 3 1 3 1 2 1 2
for disc ∈ {Disc, Disc n , Disc ε n }, n ∈ N and 0 < ε < 1, the maximal dis- closure of Sec in M is disc max (M(µ 0 )) = sup σ disc(M σ (µ 0 )) and the minimal disclosure of Sec is disc min (M(µ 0 )) = inf σ disc(M σ (µ 0 )).
Note that the construction ensuring that once a secret state is visited, the run remains in a secret state forever, extends naturally from OMC to OMDP. We only consider OMDP of this form in the rest of this chapter.

Example 7.5. Consider the OMDP of Figure 7.5. As soon as the strategy selects action a, the system enters a secret state and discloses the secret with probability 1. Therefore Disc max (M(µ 0 )) = 1. If the strategy only selects action b however, observing a 'o 1 ' clearly shows the system is in q 1 , thus disclosing the secret, while after observing at least two 'o 2 ', the belief is {q 0 , q 2 } which does not disclose the secret. The probability to observe at some point 'o 1 ' being equal to 1/2, Disc min (M(µ 0 )) = 1.

We study the following problems for OMDP over finite or fixed horizon:

• Computation problems.

-The value problem: compute the disclosure; -The strategy problem: compute an optimal strategy whenever it exists.

• Quantitative decision problems. Let = ≥ for maximisation and = ≤ for minimisation.

-The disclosure problem: Given M and a threshold θ ∈ [0, 1], decide if disc(M) θ;

-The strategy decision problem: decide if there exist a strategy σ such that disc(M σ ) θ.

• Qualitative decision problems.

-The limit-sure disclosure problem: the disclosure problem when θ = 1 for maximisation and θ = 0 for minimisation;

-The almost-sure disclosure problem: the strategy decision problem when θ = 1 for maximisation and θ = 0 for minimisation.

For the complexity results regarding a fixed horizon n, we assume that n is written in unary representation or bounded by a polynomial in the size of the model where the polynomial is independent of the model as done in classical studies (see for instance [START_REF] Papadimitriou | The complexity of Markov decision processes[END_REF]).

As said earlier, the whole power of the strategies we defined may not be necessary to answer the above problems. Restricting ourself to a subset of strategies that gives the same disclosure values can help simplify the proofs and the representation in practice of these strategies. Moreover, it helps understanding what is important in the control of an OMDP to optimise the disclosure. We thus show that for disclosure problems we can restrict strategies to observation-based ones.

Proposition 7.1. Given an OMDP, a secret and a strategy σ, there exists an observationbased strategy σ such that for disc ∈ {Disc,

Disc n , Disc ε n }, disc(M σ (µ 0 )) = disc(M σ (µ 0 )).
For this proof, from an arbitrary strategy σ, we build an observation-based strategy σ with the same disclosure value. The strategy σ is randomised and is obtained by choosing, after an observed sequence w, a distribution on the different choices made by σ on runs with observed sequence w. This is done so that the probability of choosing an action after observing w is the same for both strategies.

We then prove that σ meets the same disclosure value as σ. More precisely, we establish that the probability to reach a state with a given observation is the same for both strategies. This is done by induction on the length of the observed sequence and on an ordering of the unobservable states. Two cases have to be considered, depending on if the last state is observable or not. However, each case is dealt with in the same way (both for the initialisation and for the induction step). Thus we only detail the first one.

Proof. Let M = (S, Act, p, O) be an OMDP with initial distribution µ 0 , and let σ be a strategy. For an observation w ∈ Σ * and a state s ∈ S, we define the sets (note that these are finite sets given the claim in Lemma 7.1) R(w, s) = {ρ finite run of M

σ (µ 0 ) | O(ρ) = w ∧ last(ρ) = s}.
We now define a mapping σ from Σ * × S to Dist(Act) by

σ(w, s) = 1 ρ∈R(w,s) P σ (ρ) ρ∈R(w,s) P σ (ρ)σ(ρ).
σ(w, s) corresponds to the average choice made by σ after a run with observed sequence w and ending in s. Using σ, we define the new strategy σ for a finite run ρ by σ (ρ) = σ(O(ρ), last(ρ)). We claim that P σ (R(w, s)) = P σ (R(w, s)) for any observation w and any state s, which entails equality of disclosure.

Partitioning the set of states into S = S o S u where S u = O -1 (ε), we can assume a topological sort on the subgraph obtained by removing all edges in S × S o (this subgraph is acyclic due to the hypothesis of convergence). This means that there exists a numbering η of the states so that if η(s ) > η(s), there is no transition from s to s . We proceed to prove the above claim by a joint induction on the pairs (w, s) using |w| and η(s).

For the base cases, we need to establish the property for w = ε with s ∈ S u , and for w ∈ Σ with s ∈ S o , where µ 0 (s) > 0 in both cases.

As seen in the previous proof, erasing observations leads to technical and cumbersome developments. In order to avoid them in the design of procedures for the finite-horizon case, we apply the preliminary transformation that ensures the observation is non-erasing described in the next proposition. We precisely state the size of the obtained OMDP in view of complexity results.

Proposition 7.2. Given an OMDP M = (S, Act, p, O), an initial distribution µ 0 and a secret Sec, one can build in exponential time an OMDP M = (S , Act , p , O ), an initial distribution µ 0 and a secret Sec where O is non-erasing and for disc ∈ {Disc min , Disc max } disc(M(µ 0 )) = disc(M (µ 0 )). In addition, the size of S , p and µ 0 is polynomial w.r.t. the ones of S, p and µ 0 . The size of Act is polynomial w.r.t. the size of Act and exponential w.r.t. the size of S.

The main idea of the construction is that every time a run visits an observable state, an observation-based strategy can fix a set of action for the current state and for every unobservable state. It will then keep this choice until the run visits a new observable state. Once such a set of actions is fixed, one can easily compute the probability distribution to reach the next observable state. Unobservable states can thus be removed from the system. We also add a new state to deal with the possibility that an unobservable state had a positive probability in the initial distribution.

Proof. We first build the new OMDP and then explain the correspondence between strategies in both models, which induces the relationship between disclosures. Construction of the OMDP. We start from OMDP M = (S, Act, p, O) with Act = ∪ s∈S A(s), observation alphabet Σ, and a set of secret states Sec ⊆ S. Choosing a fresh observation symbol and a fresh state s , we build an OMDP M = (S , Act , p , O ) with set of states S = {s } ∪ (S \ O -1 (ε)), and observation alphabet Σ ∪ { }, where the initial distribution is 1 s . The observation function O is defined by O (s ) = and O (s) = O(s) otherwise. Note that all states have non-trivial observation. The set of actions of M is Act = DR where DR is the set of vectors of deterministic decision rules δ over S, i.e. such that δ(s) ∈ A(s). The intuition of DR is that the actions associated with the current state and any unobservable state by the strategy after an observation is fixed until the next observable state, so we can gather this set of action into a single action.

We now define the transition probabilities, starting by the transitions exiting s . For a run ρ = s 0 a 1 . . . a n s n , we write π(ρ) = n i=1 p(s i |s i-1 , a i ) and first(ρ) = s 0 . Given an observable state s ∈ S and δ ∈ DR, the set Ê(s , δ, s) contains the finite runs ρ = s 0 a 1 s 1 . . . a n s n starting from some s 0 ∈ Supp(µ 0 ) and ending in s n = s such that a i = δ(s i-1 ) for all i, 1 ≤ i ≤ n and all states s 0 , . . . , s n-1 are unobservable. Observe that the intermediary states are all distinct due to the convergence of the OMDP. We set p (s | s , δ) = ρ∈ Ê(s , δ,s) µ 0 (first(ρ))π(ρ). If s ∈ Supp(µ 0 ), the set Ê(s , δ, s) contains the run reduced to ρ = s.

We turn to the transitions exiting the other states. It is easier as we do not need to take the initial distribution into account. Given a state s = s , an action δ ∈ DR and an observable state s , we consider the finite set Ê(s, δ, s ) of signalling runs of Opacity M ρ = s 0 a 1 . . . a n s n starting in s 0 = s and ending in s n = s such that for each i, 1 < i ≤ n, a i = δ(s i-1 ), and all intermediate states are unobservable. We set p (s |s, δ) = ρ∈ Ê(s, δ,s ) π(ρ). Note that Ê(s, δ, s ) may include runs like ρ = s δ(s)s .

In order to efficiently compute the transition function of some δ, one uses a topological sort of the unobservable states thanks to the convergence hypothesis, and then compute the probability from observable states to reach first the unobservable states topologically sorted and then the observable states. This gives a polynomial time computation of the transition function of δ. Thus, the size of S , p and µ 0 is polynomial w.r.t. the ones of S, p and µ 0 . Moreover, the size of Act is polynomial w.r.t. the size of Act and exponential w.r.t. the size of S.

Correspondence between strategies. The above construction ensures that any run

ρ = s δ 1 s 1 . . . δ k s k of M corresponds to the set of runs ρ = ρ 1 s 1 , . . . ρ k s k of M containing the sequence s 1 . . . s k of observable states, with ρ 1 ∈ Ê(s , δ 1 , s 1 ) and ρ i ∈ Ê(s i-1 , δ i , s i ) for 1 < i ≤ k. All runs in the set have the same observation w = O(s 1 ) . . . O(s k ) with O (ρ ) = w.
To show that disclosure over finite horizon is the same in both OMDP, we establish correspondences between the strategies of M and M and the associated disclosure value. From Proposition 7.1, we can restrict to observation-based strategies.

• Let σ be an observation-based strategy of M , defined on Σ * × S . Given an observation w ∈ Σ * there exists δ such that for every state s ∈ S , we have σ ( w, s) = δ. We define σ(w, s) = δ(s). Then, writing P σ (resp. P σ ) instead of P Mσ(µ 0 ) (resp. P M σ (µ 0 ) ), and defining for w ∈ Σ * and s ∈

S \ O -1 (ε), R(w, s) = {ρ ∈ SR Mσ(µ 0 ) | O(ρ) = w ∧ last(ρ) = s} and R (w, s) = {ρ ∈ SR M σ (µ 0 ) | O(ρ ) = w ∧ last(ρ ) = s}, we have P σ (R(w, s)) = P σ (R (w, s)).
• Conversely, given an observation-based strategy σ of M, we build an observationbased strategy σ of M as follows: Given w ∈ Σ * , we define the mapping σ ( w) : S → Dist(DR) by σ ( w)(s) = σ(w, s) for any s ∈ S. Then, using the same notations as above, we have P σ (R(w, s)) = P σ (R (w, s)).

Therefore, defining the set of secret states of S by Sec = Sec ∩ S , as the set of secret states is absorbing, the disclosures over finite horizon are equal for σ and σ . Thus for finite horizon, one can restrict oneself to non-erasing observation functions with some care on the complexity of the actions. Also, on fixed horizon, we assume the observation function is non-erasing. Thus in both cases we are able to use this assumption.

Maximisation with finite horizon

We start the study of the disclosure problem with the maximisation objective over finite horizon. In other words, here the strategy tries to maximise the disclosure of the secret after an arbitrarily long, yet finite, amount of time.

In Subsection 2.1, we show how to restrict the study to deterministic strategies without loss of generalities. We show that most of the notions are unfortunately un-decidable in Subsection 2.2, and prove the decidability of the almost-sure disclosure in Subsection 2.3.

Deterministic strategies are sufficient

We showed in Proposition 7.1 that one can limit oneself to observation-based strategies. In fact, for maximisation problems, one can do even better. Indeed, the additional power given by randomisation is not useful and thus observation-based deterministic strategies are sufficient.

Proposition 7.3. Given an OMDP M, a secret Sec and a disclosure notion disc ∈ {Disc, Disc n , Disc ε n }, for any observation-based strategy σ there exists a deterministic observation-based strategy σ such that disc(M σ (µ 0 )) ≤ disc(M σ (µ 0 )).

This proof strongly uses Lemma 1 of [START_REF] Chatterjee | Randomness for free[END_REF] (or alternatively [START_REF] Goubault-Larrecq | Random measurable selections[END_REF]) which establishes, in an active stochastic setting, that deterministic strategies are sufficient to optimise an objective defined by a set of infinite runs. This Lemma does not directly give the result we want as, contrary to the objectives used in their paper, the choice of the strategy modifies which runs are disclosing. However, as a disclosing run for a randomised strategy is also a disclosing run for a deterministic strategy that does not introduce new runs, we can use parts of their proof to show our result.

Proof. In the proof of Lemma 1 of [START_REF] Chatterjee | Randomness for free[END_REF], the authors show that a randomised observation-based strategy can be seen as a convex combination of a family of deterministic observation-based strategy. As a consequence, in our framework, given an observation-based strategy σ and a disclosure notion disc, there exists an observationbased deterministic strategy σ det such that for every finite run ρ, Supp(σ det (ρ)) ⊆ Supp(σ(ρ)) and

P Mσ det (µ 0 ) (V σ (µ 0 )) ≥ P Mσ(µ 0 ) (V σ (µ 0 )).
The second property is not enough to conclude, as a disclosing run under σ is not necessarily a disclosing run under σ . However, thanks to the first property we can obtain that V σ (µ 0 ) ∩ Ω Mσ det (µ 0 ) ⊆ V σ det (µ 0 ). Indeed, as σ is more permissive than σ det , ΩM σ det (µ 0 ) ⊆ ΩM σ (µ 0 ). This implies that, given a run ρ, if O(ρ) discloses the secret with the strategy σ then either O(ρ) discloses the secret with the strategy σ det or O(ρ) cannot be observed with σ det .

This implies:

P σ det (V σ (µ 0 )) = P σ det (V σ (µ 0 ) ∩ Ω Mσ det (µ 0 ) ) ≤ P σ det (µ 0 ) (V σ det (µ 0 )). Therefore, disc(M σ det (µ 0 )) = P σ det (V σ det (µ 0 )) ≥ P σ det (V σ (µ 0 )) ≥ P σ (V σ (µ 0 )) and the result holds since P σ (V σ (µ 0 )) = disc(M σ (µ 0 )).
Observe that this proof shows that the restriction to deterministic strategy does not decrease the disclosure. However, it does not necessarily keep the same disclosure as before contrary to the proof used to restrict to observation-based strategies. Therefore it cannot be applied for minimisation problems. Opacity 2.2 Undecidability of the disclosure and limit-sure disclosure problems

As mentioned in the previous proof, one of the difficulties of opacity is that the set of disclosing runs depends on the strategy: a transition can be completely blocked by some strategy, modifying the set of disclosing observations. This was illustrated in Figure 7.5, where choosing action a in state q 0 removes the edge to q 0 . This situation was excluded in the computation of the disclosure presented in [START_REF] Bérard | Preserving opacity on interval Markov chains under simulation[END_REF][START_REF] Bérard | Opacity for linear constraint Markov chains[END_REF] where the authors study a restricted form of Interval Markov Chains [START_REF] Jonsson | Specification and refinement of probabilistic processes[END_REF]. The disclosure problem for the general class of OMDP was left open. We answer negatively to the general problem by proving undecidability of the disclosure problem, hence the disclosure cannot be computed in general. Undecidability also holds for limit-sure disclosure. Writing I for the set of intervals in [0, 1], an interval Markov chain (IMC) over an alphabet Σ is a tuple M = (S, s init , I, O) where S is the set of states, s init is the initial state, I : S → I S associates with every state s ∈ S a mapping from S to I, and O : S → Σ ∪ {ε} is the observation function. We abuse notations by writing µ ∈ I(s) to denote any distribution µ : S → [0, 1] such that for all s ∈ S, µ(s ) ∈ I(s | s). The notion of run ρ is the same as for an OMC but a transition from s = last(ρ) to some successor requires the choice of a distribution µ ∈ I(s). A strategy of IMC M is thus a mapping σ associating with each finite run ρ with s = last(ρ) a distribution σ(ρ) ∈ I(s). In other words, an IMC is a OMDP where the chosen action represents a set of probabilities satisfying the interval conditions set by I and summing to 1. In fact, an IMC can be transformed into an (exponentially larger) OMDP where actions are the basic feasible solutions of the linear program specified by the constraints associated with intervals [START_REF] Sen | Model-checking Markov chains in the presence of uncertainties[END_REF][START_REF] Chatterjee | Model-checking omega-regular properties of interval Markov chains[END_REF]. Thus undecidability results for IMC also hold for OMDP.

Example 7.6. Consider the IMC of Figure 7.7. From the initial state, the strategy must attach a probability p 1 ∈ [0, 1 2 ] to the transition to s 1 and a probability p 2 ∈ [ 1 4 , 1] to the transition to s 2 . As, in order to obtain a distribution, we require that p 1 + p 2 = 1, p 2 is de facto restricted to the interval [ 1 2 , 1]. If p 1 = 1 4 is selected, then p 2 = 3 4 and the run has a probability p 1 to move to s 1 and p 2 to move to s 2 . In these two states, the only exiting transition is labelled by the interval {1} which we simplified by removing the braces in the figure. Thus, the run then loops indefinitely on the state.

s 0 o 1 s 1 o 1 s 2 o 2 [0, 1 2 ] [ 1 4 , 1] 1 1 Figure 7.7: Example of IMC.
Theorem 7.2. The maximal finite-horizon disclosure problem is undecidable for OMDP, even when the secret is reached with probability 1 and for a non-erasing observation function.

The maximal finite-horizon disclosure problem when restricted to finite-memory strategies is undecidable (even with the same additional assumptions).

Starting from a PA A, we build an IMC M A = (S, s 0 , I, O) such that there exists a word w ∈ {a, b} * with P F A (w) > 1 2 if and only if Disc max (M A ) > 1 4 . The proof of Theorem 7.2 is more involved than the proof of Theorem 7.1 because the strategies must be taken into account. The goal is to have the strategy choose a single word then "plays" it in the IMC and the probability to disclose the secret is half of the probability of the selected word. However, just with this, nothing would prevent the strategy to switch words during the run if it realises that the current run will not disclose the secret. We add a second component that ensures that if the strategy deviated from the single selected word the current run will not disclose the secret. Doing so, the strategy loses the advantage of knowing the current state of the run.

Proof. We first give the construction in two steps. The first step is very similar to what was done in the proof of Theorem 7.1. Starting from a PA A = (Q, q 0 , {a, b}, T, F ) that is supposed complete, we build an IMC Â = ( Q, q 0 , Î, Ô) where Σ = {a, b} is the observation alphabet. The set of states is Q = Q ∪ {q c | q ∈ Q ∧ c ∈ {a, b}}, with initial state q 0 . The observation function Ô is defined by Ô(q) = ε and Ô(q c ) = c for q ∈ Q and c ∈ {a, b}. The interval mapping Î : Q → I Q is defined for q, q ∈ Q and c ∈ {a, b} by:

• Î(q | q c ) = T (q | q, c) is a point interval; • Î(q c | q) = [0, 1].
Compared to the illustration given in Figure 7.3, this construction amounts to replacing all by ε (making the states non observable) and the probabilities 1 4 from original states to new ones by the interval [0, 1].

However, the construction of the complete IMC M A = (S, s 0 , I, O) from A is more involved and requires to add a supplementary gadget limiting the power of the strategy. This is why we first use an observation function which can erase states and explain at the end how to relax this hypothesis. The construction is illustrated in Figure 7.8 with some conventions to avoid too many edges, a final state from A (e.g. like q f ) is doubly circled.

• S = {s 0 , s 1 , q 1 , q 2 , q , q s } ∪

Q ∪ {s c | c ∈ {a, b, }} ∪ {r c | c ∈ {a, b, , }}; • I(s 1 | s 0 ) = I(q 0 | s 0 ) = 1
2 and the restriction of

I to Q is Î. For all c ∈ {1, a, b, }, c ∈ {a, b, }, I(s c | s c ) = 1 6 and I(r c | s c ) = [0, 1 4 ], for all c, c ∈ {a, b, , }, I(r c | r c ) = 1
5 and I(q s | r c ) = 1 5 . For all q ∈ Q \ F , I(q 1 | q) = [0, 1], for all q ∈ F , I(q 2 | q) = [0, 1], and I(q s | q 1 ) = I(q | q 2 ) = I(q s | q ) = I(q s | q s ) = 1.

• O extends Ô by: O(s 0 ) = O(s 1 ) = ε, O(q 1 ) = O(q 2 ) = O(q s ) = , O(q ) = ,
ε q f ε q ε q 1 q 2 q q s 1 1 2 1 2 [0, 1] [0, 1] 1 1 1 1 6 [0, 1 4 ] 1 6 1 5 1 5 1 5 1 5 1 5 [0, 1 4 ] 1 5 Â Figure 7
.8: Reduction from the strict emptiness problem of PA to the maximal finitehorizon disclosure problem. An edge outgoing from a dotted box should be duplicated to originate from all states in the box and an edge entering a dotted box from a state s should be duplicated from s to any state in the box. Hence a loop on a dotted box represents a complete graph inside the box (including self-loops).

Informally, for Sec = {q s }, the upper gadget ensures that for any strategy σ there is at most one word w ∈ {a, b} * such that the observation w discloses the secret. The lower gadget allows one to generate secret runs of observation w with half the probability as the one assigned by the PA to w.

We now formally prove that there exists a word w ∈ {a, b} * with P F A (w) > 1 2 if and only if Disc max (M A ) > 1 4 . First suppose there exists a word w = a 1 . . . a n ∈ {a, b} * accepted with probability greater than 1 2 in A. We define the strategy σ for a finite run ρ in both parts of M A (when relevant) as follows:

• In the upper part, assume that ρ ends in a state s c with c ∈ {1, a, b, }. If there exists i < n such that O(ρ) = a 1 . . . a i then σ(ρ)(r a i+1 ) = 0, leaving no choice for the rest of the distribution: In order for the sum of probabilities to be equal to 1 we have for b = a i+1 , σ(ρ)(r b ) = 1 4 . If O(ρ) = w, then σ(ρ)(r ) = 0, which also leaves no choice for the rest of the distribution.

• In the bottom part, we can assume that ρ ends in a state q ∈ Q. If there exists i < n such that O(ρ) = a 1 . . . a i then σ(ρ)(q a i+1 ) = 1. Finally, if O(ρ) = w then σ(ρ)(q 2 ) = 1 if q ∈ F , and σ(ρ)(q 1 ) = 1 otherwise.

At the beginning the system will move with probability 1 2 to Â, where the strategy ensures that the word w is observed. This leads to the state q 1 with probability 1 2 P F A (w) and thus the next observations belong to * and the runs with observations in w + belong to the secret. On the other hand, the system can also go to s 1 with probability 1 2 from where, due to the decisions of the strategy, a run with observation w ends in s (the decisions of the strategy ensure that either the run does not have observation w , or it could not go in a r state). Moreover, from s , cannot be observed. This implies that w is a min-disclosing observation in

M A,σ , hence Disc(M A,σ ) ≥ 1 2 P F A (w) > 1 4 . Since Disc max (M A ) = sup σ Disc(M A,σ ), we can conclude that Disc max (M A ) > 1
4 . Conversely, suppose that the disclosure is strictly greater than 1 4 and let σ be a strategy such that Disc(M A,σ ) > 1 4 . Then, σ must forbid states in {r c | c ∈ {a, b, }}, otherwise there would be no disclosing observation since every observation can be simulated once a state r c is reached. Writing Σ = {a, b, }, we inductively define the word w ∈ Σ ∞ ∪ Σ * by a sequence (w i ) i≥0 of non-decreasing prefixes of w:

• We start with w 0 = ε;

• Assume w i is built and let ρ i be a run ending in state s x for some

x ∈ {1, a, b, }, with O(ρ i ) = w i . If σ(ρ i )(r c ) = 0 for some c ∈ {a, b, }, then w i+1 = w i c, otherwise w i+1 = w i .
The set of ambiguous observations (i.e. corresponding to both secret and non-secret runs) are the ones reaching the set of states {r c | c ∈ {a, b, , }}:

w i x = w i+1 x = w i x(Σ ∪ { }) * ω .
Hence, the set of disclosing observations is reduced to either w ω , where w is the largest prefix of w in {a, b} * if occurs in w, and empty otherwise. Since the disclosure is greater than 0, we obtain w as the single min-disclosing observation with Disc(M A,σ ) = P M A,σ (w ). Since P F A (w) ≥ 2.P M A,σ (w ), we can conclude that P F A (w) > 1 2 . The proof can be extended with a non-erasing observation function by replacing ε with a fresh symbol (like in the proof of Theorem 7.1). This requires to slightly modify the parts of the IMC corresponding to the sets of states {s c | c ∈ {a, b, }} and {r c | c ∈ {a, b, }} in order to ensure alternation of letters from {a, b} and this new symbol.

The undecidability result holds even when restricted to finite-memory strategy as the strategy defined in the first direction of the proof only uses finite memory.

Opacity

If we could compute the maximal finite-horizon disclosure, we could solve the associated decision problem. Thus, as a consequence of the previous theorem, we obtain:

Corollary 7.1. The maximal finite-horizon disclosure of an OMDP cannot be computed.

We now turn to the qualitative disclosure problems, and using a reduction from the value 1 problem in PA, we also have:

Theorem 7.3. The maximal finite-horizon limit-sure disclosure problem is undecidable for OMDP.

Proof. The reduction from the value 1 problem for PA done here is similar to the one of the proof of Theorem 7.2. The difference is that any run initially moving from s 0 to s 1 (thus moving to the part of the IMC which was used to limit the power of the strategy) will now almost-surely disclose the secret. More precisely, the construction of M A depicted in Figure 7.8 for the proof of Theorem 7.2 is slightly modified as follows (see Figure 7.9): a new state q with O(q ) = is added in the upper part just before reaching the secret state q s . In this case, the runs reaching the secret in the upper part disclose the secret as they end with ω .

The disclosure on the bottom part is performed as before. As a consequence, if a word w is "selected" by the strategy, the finite-horizon disclosure will be equal to 1/2.(1 + P F A (w)). This value can be arbitrarily close to 1 iff A accepts words with probabilities arbitrarily close to 1, which yields the result.

• • • • • • =⇒ q s 1 1 5 1 5 • • • • • • q q s 1 1 5 1 5 1 Figure 7
.9: Modification of Figure 7.8 for limit disclosure.

Decidability of the almost-sure disclosure problem

Fortunately the maximal finite-horizon almost-sure disclosure problem is decidable. The proof relies on results for partially observable MDP (POMDP) described in Definition 6.8, page 186.

Theorem 7.4. The maximal finite-horizon almost-sure disclosure problem in OMDP is EXPTIME-complete. Moreover, if the system is almost-surely disclosing, one can build a belief-based strategy with disclosure 1.

We reduce the almost-sure disclosure problem for maximisation in OMDP to almostsure reachability in POMDP. The POMDP we build is exponential in the size of the original OMDP and the algorithm to solve almost-sure reachability is exponential in the size of the POMDP [START_REF] Chatterjee | Qualitative analysis of partially-observable Markov decision processes[END_REF]. A naive application of the two successive results would give a 2-EXPTIME algorithm. However, a finer analysis yields EXPTIME complexity of our algorithm as these two exponentials do not stack. The hardness is obtained by a reduction from the safety problem in games with imperfect information that was shown to be EXPTIME-complete in [START_REF] Berwanger | On the power of imperfect information[END_REF]. The reduction for the lower bound is similar to the one of Theorem 7.2.

Proof. We start by giving the construction of the POMDP, then we show that solving almost-sure reachability in the POMDP is equivalent to the finite-horizon almost-sure disclosure problem for maximisation. Finally we prove the hardness. Construction of the POMDP. We start from an OMDP M = (S, Act, p, O) with Act = ∪ s∈S A(s), observation alphabet Σ o , and a set of secret states Sec ⊆ S. Thanks to Proposition 7.2, we can assume O to be non-erasing (the potentially exponential blow up of the number of actions does not affect the complexity result as we will see later). Let µ 0 be an initial distribution. We assume w.l.o.g. that µ 0 is a Dirac distribution on some state s 0 ∈ S. We build a POMDP M = Q, q 0 , Obs, Act, T with set of states Q = S ×2 S , with q 0 = (s 0 , {s 0 }) and observation alphabet Σ Correspondence between strategies. To show that M is almost-surely disclosing for Sec iff Sec × 2 Sec can almost-surely be reached in M , we establish a correspondence between the strategies of M and the scheduler of M . From Proposition 7.3, we can restrict to deterministic observation-based strategies for M, and from [START_REF] Chatterjee | Qualitative analysis of partially-observable Markov decision processes[END_REF], we also restrict to deterministic schedulers for M .

In both direction of the equivalence, given a strategy σ and a scheduler τ , we write P σ (resp. P τ ) instead of P Mσ(µ 0 ) (resp. P M (τ ) ), and define, for w ∈ Σ Now let Reach(Sec × 2 Sec ) be the set of runs reaching Sec × 2 Sec in M (τ ). Then we claim that Disc max (M σ (µ 0 )) = P τ (Reach(Sec × 2 Sec )). Indeed, an observation w discloses the secret under strategy σ iff all observable states reachable with observed sequence w belong to the secret, i.e. iff B σ w ⊆ Sec. Thus the runs ρ in M with a disclosing observation for M are the ones for which last(ρ ) ∈ Sec × 2 Sec . Therefore,

• S = {s 0 , 0 , q 1 , q 2 , q , q 1 s , q s | r c , a) > 0. For all a, a ∈ Σ, p(q 1 s | q 1 , a) = p(q | q 2 , a) = p(q 1 s | q , a) = p(q 1 s | q 1 s , a) = p(q 2 s | q 2 s , a) = 1.

• O(s 0 ) = O( 0 ) = obs( 0 ); For z ∈ L, s, r, a ∈ Σ, o ∈ O, O(z a,o ) = (a, o); For o ∈ { , }, O(z a,o ) = o, and O(q 1 ) = O(q 2 ) = = O(q 1 s ), O(q ) = , O(q 2 s ) = .

The initial distribution is µ 0 (s 0 ) = 1/2 = µ 0 ( 0 ) and the secret is Sec = {q 1 s , q 2 s }. This proof being similar to the one of Theorem 7.2, we only detail here the differences. A run starting in s 0 will almost surely trigger a and disclose the secret. A run starting in 0 will almost surely reach q 1 s as after any action in the copy of G there is a positive probability to reach q 1 or q 2 . In order for a finite run starting in 0 to disclose the secret, it cannot go through q 1 and should not share its observed sequence with a run ending in a state r c . Given a strategy σ of M, if there exists a σ-compatible run ρ visiting a state c with an observation O( c ) ∈ Σ × F , then there is a σ-compatible path ρ visiting q 1 , therefore a set of runs with positive probability do not visit the secret. Thus a deterministic strategy almost surely disclosing the secret in M never visits a state triggering an observation of the form Σ × F . Moreover such a strategy does not take the current state into account. Indeed, let ρ and ρ be two runs such that O(ρ) = O(ρ ) and ending in two states c and s c . If σ(ρ) = a and σ(ρ ) = a are two actions in Σ with a = a then there exists o ∈ O such that ρa a,o is a σ-compatible run. Since a = a , ρ ar a,o is also a σ-compatible run with same observation than ρa a,o . Hence no observation prefixed by O(ρa a,o ) would disclose the secret.

Therefore, similarly as in Theorem 7.2, Control has a winning strategy iff there exists a deterministic strategy considering only the observed sequence that almost-surely discloses the secret. This implies EXPTIME-hardness of the maximal finite-horizon almost-sure disclosure problem.

In this section on the maximisation of the disclosure over finite horizon, we saw that although we could restrict ourselves to deterministic strategies, most of the finitehorizon problems are very complicated (all but one are undecidable).

Minimisation with finite horizon

We now turn to minimisation over finite horizon where strategies try to hide the secret from an observer, and thus to minimise the disclosure.

Opacity

This section shows a surprising result. Indeed, in Subsection 3.1 we show that for minimisation we cannot assume strategies to be deterministic any more, thus the problem seems more complex. However, the disclosure value can be computed as we establish in Subsection 3.2.

Deterministic strategies are not enough

After the proof of Proposition 7.3, we remarked that the proof showed that deterministic strategies do not decrease the disclosure, thus the proof could only work for maximisation. In fact, the result itself is limited to maximisation as randomisation may be necessary to minimise the disclosure. Let us see that on an example. Consider the OMDP depicted in Figure 7.10 with Sec = {q 2 , q 3 }. There are two deterministic strategies, choosing respectively a or b in q 0 . In both cases, the disclosure is 1 2 . On the other hand, for randomized strategies σ p such that σ p (q 0 ) = pa + (1 -p)b with 0 < p < 1, there are no disclosing observations, hence the disclosure is 0. While we cannot restrict ourselves to deterministic strategies, we still use in the decision procedures a restricted class of strategies. These strategies are called families of almost-deterministic strategies and are based on ε-decision rules. Definition 7.9. Let δ be the deterministic decision rule for state s selecting action a ∈ A(s). Then, for ε > 0, the ε-decision rule δ ε ∈ Dist(A(s)) is defined by: In other words, given a strategy σ we define a family of strategies that have an increasingly high probability to play like σ as the run goes on, yet always allow the other actions with positive probability. We will see that strategies of this form are dominant in the sense that they are sufficient to compute the disclosure value. However, if there exists a strategy that minimises the disclosure, this strategy does not always belong to a family of almost-deterministic strategy.

o 0 q 0 o 1 q 1 o 2 q 2 o 1 q 3 o 2 q 4 a, 1

The minimal disclosure value is computable

Using Proposition 7.2, we assume in the following that the observation function we consider is non-erasing. The complexity of the transformation does not affect the results since the polynomial complexity in the number of actions is dominated by the exponential complexity in the number of states.

In order to compute the minimal disclosure value, we build from an OMDP M, another OMDP M min which is a "correct abstraction" (as is stated by Proposition 7.4) for reducing minimal disclosure problems to minimal reachability problems. Intuitively, in M min , the states are enlarged by the maximal belief that can occur independently of the action that has been selected. NextMax is intuitively the belief obtained with a strategy allocating some probabilities to every action. Observe that given a family of almost deterministic strategies {σ ε } and a run ρas of M with O(s) = o, one has for all ε > 0, B σε ρas = NextMax(B σε ρ , o). Then M min is formally defined as follows:

• S min , the set of states, is defined by: S min = {(s, B) | s ∈ B ⊆ O -1 (O(s))};

• for every (s, B) ∈ S min , A(s, B) = A(s); We focus now on fixed-horizon problems for both maximal (Subsection 4.1) and minimal (Subsection 4.2) disclosure. In both cases, the algorithms and hardness results have similarities.

Maximal disclosure

In order to compute the value of the maximal disclosure within a fixed horizon, one could build the POMDP described in the proof of Theorem 7.4, then use pre-existing results on POMDPs. This would result in an EXPTIME algorithm, whereas we provide below an algorithm with a better complexity in PSPACE.

Theorem 7.7. The fixed-horizon maximal value (when the horizon n is described in unary representation) is computable in PSPACE and the fixed-horizon maximal disclosure problem is PSPACE-complete.

Due to the complexity of the proof, we separate the algorithm computing the value from the hardness of the decision problem in two separate proofs.

In order to compute the value, we first order the observation alphabet Σ. Then, a non-deterministic decision procedure operating in PSPACE enumerates every observed sequence of length n while maintaining the sets of states that were possible after every prefix of this observation, the actions that were chosen non-deterministically in these states and values used in the computation of the disclosure. The information kept is of polynomial size and when every observation has been read, one of the values computed are exactly the disclosure of the system at time n. This provides an NPSPACE algorithm which can be turned in to a PSPACE one using Savitch's Theorem [START_REF] Savitch | Relationships between nondeterministic and deterministic tape complexities[END_REF]. In order to get the value we observe that we can compute the polynomially sized denominator of this value and then we make iterative calls to the decision algorithm.

Proof. We first present a non-deterministic procedure that decides in NPSPACE the disclosure problem. It can then be determinised using Savitch's Theorem [START_REF] Savitch | Relationships between nondeterministic and deterministic tape complexities[END_REF].

From an arbitrarily ordered observation alphabet Σ, the procedure operates as follows for horizon n:

• It maintains a disclosure value v, a sequence of observations o 0 • • • o i with i ≤ n, a sequence of sets of states B 1 • • • B i with B j ⊆ O -1 (o j ) for all j ≤ i, an action a j,s ∈ A(s) for all (j, s) with j < i and s ∈ S j , and for all (j, s) with j ≤ i and s ∈ B j the probability p j,s to reach s after the sequence of observations o 0 • • • o i ;

• Initially v = 0, o 0 is the smallest observation in O(Supp(µ 0 )), where µ 0 is the initial distribution, B 0 = Supp(µ 0 ) ∩ O -1 (o 0 ) and p 0,s = µ 0 (s) for s ∈ B 0 ;

• If i < n then for all s ∈ B i , the procedure guesses an action a i,s ∈ A(s). Let o i+1 be the smallest observation such that there exists a state s ∈ B i and a state s ∈ O -1 (o i+1 ) with p(s |s, a i,s ) > 0. Then B i+1 is set to {s ∈ O -1 (o i+1 ) | ∃s ∈ B i p(s |s, a i,s ) > 0} and for all s ∈ B i+1 , p i+1,s = s∈B i p i,s p(s |s, a i,s );

• If i = n, the procedure examines B n . If B n ⊆ Sec then v = v + s∈Bn p n,s otherwise v is unchanged. Afterwards it "backtracks" to the greatest 0 < i ≤ n such that there exists o i > o i with some s ∈ B i-1 and a state s ∈ O -1 (o i ) with p(s |s, a i-1,s ) > 0. Then B i and the p i,s 's are updated accordingly and the procedure carries on. If there is no such i, the procedure returns to i = 0 and similarly looks for some o 0 > o 0 , where the initialisation step is again performed except for the value of v which is unchanged. When the maximal observation in Σ ∩ O(Supp(µ 0 )) is handled, the procedure terminates by comparing v to the threshold.

The correctness of the procedure follows from the fact that there exists an optimal deterministic strategy where the selection of the action for the current state depends only on the sequence of observations (and not on the sequence of visited states).

The space complexity of the procedure is in O(n|S|(log(|A|) + nb)) where b is the maximal number of bits used to represent a transition probability of the OMDP.

Observe now that, since the maximal value is obtained by a deterministic strategy, one knows a denominator of this value: it is d n where d is the least common multiple of the denominators of the probabilities occurring in the model. Its bit size is polynomial w.r.t. the size of the model. So by iteratively solving the disclosure problem for i d n for increasing values of i, one computes the maximal value in PSPACE.

As can be seen in the proof, the optimal strategy could be computed when solving the value problem. However the size of this strategy may be exponential due to the beliefs and thus this strategy is computable in EXPTIME.

For the hardness result, we reduce the validity of Quantified Boolean Formulae (QBF). Recall that QBF extends propositional formulas by allowing quantification over the Boolean variables. Syntactically, the formulae are described by the following grammar: The idea of the reduction is the following. Given φ a closed QBF (w.l.o.g. in 3CNF with n variables and m clauses), we build an OMDP M such that φ is true iff the disclosure of M is greater or equal to 1 2 2n in 2(n + m) + 3 steps. In fact, 1 2 2n is exactly the measure of runs reaching the secret in 2(n + m) + 3 steps, thus every path reaching the secret must be disclosing. Such a run discloses the secret iff some Boolean variable of φ and its negation (x and ¬x for example) do not occur in its observation.

In M, during the first 2n steps, an assignment is 'given' to each Boolean variable: (i) for each existentially quantified Boolean variable x, the strategy chooses whether x or ¬x occurs in the observation and (ii) for each universally quantified Boolean variable y, by a random choice with probability 1 2 . During the last 2m steps, the strategy must trigger a Boolean variable in every clause of φ so that if a clause is not satisfied by the current assignment, then a Boolean variable is observed as both true and false during the run. Thus the observation would not disclose the secret.

Proof. We reduce the validity of a quantified Boolean formula: Given a closed QBF in 3CNF φ = ∃x 1 ∀y 1 ∃x 2 . . . ∀y n ψ with ψ = i=1...m (z i 1 ∨ z i 2 ∨ z i 3 ), we build an OMDP M such that φ is true if and only if Disc 2(n+m)+3,max (M) ≥ 1 2 2n . The OMDP M = (S, Act, p, O) (depicted in Figure 7.11 with some conventions to avoid having too many edges) is defined by:

• S = {s init , s end , s }∪{s z i | z ∈ {x, y}, i = 1 . . . n}∪{s ¬z i | z ∈ {x, y}, i = 1 . . . n}∪ {s i t | i ∈ {1, . . . , m}} ∪ {s z i j | i ∈ {1, . . . , m}, j ∈ {1, 2, 3}} ∪ i∈{1,...n} (S x i ∪ S y i ∪ S ¬x i ∪ S ¬y i ) where for z i one of the Boolean variable, S z i = {s z i ,1 a | a ∈ { , z i , ¬z i | z ∈ {x, y}, i ∈ {1, . . . , n}}} ∪ {s z i ,2 a | a ∈ { , , z i , ¬z i | z ∈ {x, y}, i ∈ {1, . . . , n}}}. Similar for the box S ¬z i of the negation of a variable.

• A(s init ) = {x 1 , ¬x 1 }, and for all i < n, A(s y i ) = A(s ¬y i ) = {x i+1 , ¬x i+1 }. For i ∈ {1, . . . , m}, A(s i t ) = {1, 2, 3} and A(s) = {next} for all other states (even in boxes). The initial distribution µ 0 is 1 s init and the set of secret states is Sec = {s end }.

We show that φ is true iff the disclosure of M for observations of length 2(n + m) + 3 is greater than or equal to 1 2 2n . First observe that for any strategy, the measure of runs reaching state s end with observation of length 2(n+m)+3 is exactly 1 2 2n . Indeed, during each of the first 2n actions, whatever the choices of the strategy, there is a probability 1 2 to go in one of the boxes and 1 2 to advance to the next choice, thus a probability 1 2 2n to reach the state s . From there every run reaches s end in 2(m + 1) steps. If the strategy is such that some variable and its negation are read on the way to s end , then there exists a run with same observation reaching the second part of a box where every observation can be triggered, and thus the run reaching s end will not disclose the secret. Intuitively, during the first 2n steps, every Boolean variable is assigned a value: Opacity a smaller value. Then, since the support are unchanged, the decision rule informally described as (1 + ε)δ -εδ for small enough ε would still provide a better value, meaning that the support A cannot be used to find an optimal strategy. Like for the case of maximisation, the hardness of the fixed-horizon minimal disclosure problem is obtained by a reduction from the validity of a quantified Boolean formula. Let φ = ∃x 1 ∀y 1 ∃x 2 . . . ∀y n ψ with ψ = i=1...m (z i 1 ∨ z i 2 ∨ z i 3 ) a closed QBF where we assume w.l.o.g. that in every clause the literals are distinct. We build the OMDP M = (S, Act, p, O) (represented in Figure 7.13) where:

• S = {s init , s end , s } ∪ {s z i | z ∈ {x, y}, i = 1 . . . n} ∪ {s ¬z i | z ∈ {x, y}, i = 1 . . . n} ∪ {s i t | i ∈ {1, . . . , m}} ∪ {s z i j | i ∈ {1, . . . , m}, j ∈ {1, 2, 3}} ∪ {s z i end | z ∈ {x, y}, i = 1 . . . n}∪ i∈{1,...n} (S x i ∪S y i ∪S ¬x i ∪S ¬y i ) where for z i one of the Boolean variable, S z i = {s z i a | a ∈ { , z i , ¬z i | z ∈ {x, y}, i ∈ {1, . . . , n}}} ∪ {s z i , s z i f }. Similar for the box S ¬z i of the negation of a variable.

• A(s init ) = {x 1 , ¬x 1 }, and for all i < n, A(s y i ) = A(s ¬y i ) = {x i+1 , ¬x i+1 }. For i ∈ {1, . . . , m}, A(s i t ) = {1, 2, 3} and for every other state (even in boxes), A(s) = {next}. t | s , next) = 1. For all i = 1 . . . m, j ∈ {1, 2, 3}, p(s z i j | s i t , j) = 1, and if i < m, p(s i+1 t | s z i j , next) = 1. Finally p(s end | s zm j , next) = 1, and for all z ∈ {x, y}, i = 1 . . . n, p(s z i end | s end , next) = 1/(2n). We now describe p for the box S x 1 other boxes being similar. For all a ∈ { , x 1 , z i , ¬z i | z ∈ {x, y}, i ∈ {2, . . . , n}}, b ∈ { , , z i , ¬z i | z ∈ {x, y}, i ∈ {1, . . . , n}} p(s x 1 b | s x 1 a , next) = 1/(4n + 2), and p(s The initial distribution µ 0 is 1 s init and the secret runs are the ones visiting s end (Sec = {s end } ∪ {s z i end | z ∈ {x, y}, i = 1 . . . n}). In a similar fashion as what was done in the hardness part of the proof of Proposition 7.7, we show that φ is true iff the disclosure of M is equal to 0 in 2(n + m + 2) steps. First observe that a run ρ reaching s end can be extended for all j ∈ {1, . . . , 2n} in a run ρ j such that O(ρ j ) = O(ρ)j. Moreover, ρ 1 discloses the secret iff x 1 and ¬x 1 both occur in O(ρ 1 ) (and similarly for the other ρ j s). Indeed a run reaching S x 1 or S ¬x 1 cannot have triggered both observations x 1 and ¬x 1 and also end with observation 1.

x 1 ¬x 1 | s x 1 ¬x 1 , next) = p(s x 1 f | s x 1 , next) = p(s x 1 f | s x 1 f , next) =
Intuitively, during the first 2n steps, all Boolean variables are be assigned a value: either chosen by the strategy as it chooses whether x i or ¬x i occurs in the observation not disclose the secret because there exists j ∈ {1, 2, 3} such that z i j appears in the previous 2m observations while its negation (written ¬z i j here) appears in the first 2n + 1 observations. Let ρ of length 2(n + m + 2) extending ρ by ending in s z i j end . There is no other run with the same observation and ρ is a secret run, thus ρ discloses the secret. Therefore the disclosure of M is positive.

Observe that this reduction also works for finite horizon since no further disclosure may occur after the first occurrence of a state in {s x 1 end , . . . , s xn end , s y 1 end , . . . , s yn end }.

Contrary to the case of maximisation, the above proof implies PSPACE-completeness for the limit-sure and almost-sure problem for disclosure minimisation.

Remark 7.2. As for maximisation, the proof of hardness can be adapted for ε-disclosure and the algorithm for membership cannot be directly applied. The minimal fixed horizon ε-disclosure could however be computed by minimising an exponential system of equations, resulting in an exponential time algorithm.

Conclusion

To our knowledge, the opacity of probabilistic systems had only been studied in order to maximise the disclosure of the system. Moreover, these studies always restricted the framework so that the strategy that is chosen does not modify if an observed sequence is disclosing or not, leaving the general case open. In the context of the previous studies, only maximisation was considered, which is understandable as maximisation and minimisation of disclosure are similar: they both consists in the optimisation of a fixed event. We, however, focused on the general case, both for maximisation, and for minimisation. In our framework, maximisation and minimisation present a strong asymmetry. Indeed, when considering finite horizon, most maximisation problems are undecidable although deterministic strategies are optimal. In contrast, minimisations problems are decidable, but good strategies often require randomisation. Note that a complexity gap (PSPACE-hard versus in EXPTIME) remains to fill for the finitehorizon minimisation problem. For fixed horizon, there is still an asymmetry between maximisation and minimisation that clearly appears in some parts of the proofs. But it is not as strong as in finite horizon and algorithms with good complexities can be obtained for both.

Although we used a variant of Markov decision processes enriched with observation) to represent our models, opacity is not an usual MDP problem. Indeed, opacity is an hyper property as the disclosure depends on a set of paths linked by their observation. This gives a partial observation flavour to opacity. Opacity as seen here is therefore a problem in between OMDP and POMDP. For this kind of problems, as seen in this chapter, it is important to determine whether the problem can be translated to an MDP or a POMDP problem in order to use the results known on these models. Here, maximisation of the disclosure was closer to POMDP problems while minimisation was closer to OMDP problems.

Opacity

A promising research direction to consider is the approximate notion of opacity. It is the most natural notion of opacity. Indeed, if an attacker knows there is a 99% chance to be in the secret, the secret could be considered to be disclosed. As we have shown in Section 1, the most naive definition of approximate opacity is undecidable even without control. For diagnosability, we showed in Chapter 4 that AFF-diagnosability, an elaborate notion of approximate diagnosability was decidable for passive systems. A natural question is thus to determine if we can define a similar notion of opacity. A notion that would measure the set of infinite secret runs which disclose with arbitrarily high probability the secret for example. As it is close to AFF-diagnosability, we conjecture that it should be decidable for observable Markov chains. However, in active system, the finite-horizon maximisation/minimisation disclosure problem are likely to be undecidable.

Chapter 8

Conclusion

Contributions

This thesis constitutes part of the work towards a theoretical analysis of partial observation problems in a stochastic framework. More specifically it focused on the problem of diagnosis. Diagnosis had already been studied for stochastic systems [TT05, CK13, BFH + 14], however the definitions used varied and many central issues had been left open. The first step to set solid foundations for the analysis of diagnosis in probabilistic systems was thus to define precise and realistic notions of diagnosis which would encompass the ones already established. This was done in Chapter 2. Before focusing on any specific framework, we performed in Chapter 3 a semantical analysis of the different notions of diagnosability. While some intuitions on the relations between the notions could be obtained directly from the definitions, the analysis allowed, among other things, to establish formally these links, with a few surprises due mostly to the distinction between finite systems, finitely-branching systems and infinitely-branching systems.

We then turned to multiple specific frameworks and developed methods to decide diagnosability with optimal complexity. First, we focused on passive systems. Moreover, in Chapter 4, we restricted ourselves to finite systems. This important restriction pushed us to make once again some semantical analysis in order to obtain refined results exploiting the finite number of states. This gave us precise characterisations of the decidable notions of diagnosis, allowing us to establish the exact complexities of the problems. We also showed how to automatically build diagnosers associated with each notion of diagnosability. In Chapter 5, we extended our analysis to infinite-state systems. This immediately raised one important issue that we did not have for finite systems: how to represent such systems. In consequence, we studied several possible representations, one of which yielded multiple decidability results. These decidability results were obtained in large part thanks to the analysis made in Chapter 3. This emphasizes the importance of a good understanding of a notion and how to characterise it as a preamble to study the problem.

We then considered active systems. As for stochastic infinite systems, many different

Conclusion

frameworks may be studied. But contrary to stochastic infinite systems, diagnosis had already been studied for stochastic active systems [BFH + 14], providing a framework our developments. The latter revealed an issue with the control of a system: ensuring diagnosability could be at the expense of the correct performance of the system. We studied in Chapter 6 how to limit the degradation of the system while preserving diagnosability. More precisely, we defined notions of the degradation of a system and showed the decidability and precise complexity for some of them, and established the undecidability of the others.

In the last chapter, Chapter 7, we switched our focus to opacity, another partial observation problem. In active systems, we showed how, when possible, one could develop strategies maximising or minimising the opacity of a system. The main element that made this analysis successful is the understanding of the forms that the optimal strategies would take.

Our contributions, while providing a good foundation for the diagnosis of probabilistic systems and many interesting results, are far from giving the whole picture. In the next section we provide a list of remaining open questions and research directions extending the thesis.

Perspectives

The current thesis opens quite a few perspectives, some of which were already given in each chapter conclusion and are partially repeated here. We classify these ideas depending on whether they are short-term, mid-term or long-term objectives. This decomposition represents how direct the link between the current work and the perspective is. We start with the short-term perspectives, i.e. the problems immediately raised by the works presented here.

• The most immediate perspectives are the ones given by the gaps within our results:

notions for which we could not establish the decidability status, complexities that are not tight, etc. For example, the algorithms given to decide the exact notions of diagnosability in probabilistic visibly pushdown automata are in EXPSPACE while the proven lower bound is only EXPTIME. Another open question is the exact complexity of computing the minimal disclosure of the opacity (PSPACE lower bound versus EXPTIME upper bound). The main open question however is the decidability status of the FA-diagnosability in pVPA. We showed that this notion was harder than the other notion of exact diagnosability by studying its membership in the Borel hierarchy, but could not give an algorithm nor an undecidability proof. We conjecture that this notion is decidable and, in fact, with an algorithm of the same complexity than the other. Indeed, the proofs of the the non-expressivity results that limited the study of FA-diagnosability uses systems that cannot be expressed by pVPA. There could exists a pathL formula that would characterise FA-diagnosability when restricted to pVPA. An ongoing work seems to confirm this to be true.

• Another immediate perspective is raised by the introduction of the pathL logic. It was used in Chapter 5 in order to decide some notions of exact diagnosability. This logic may be useful for example to test properties such as the uniformity of the speed of diagnosis, the boundedness of the mean detection time of a fault (or mean time before an information about the correctness of a run), etc. Moreover, if the pathL logic cannot be used directly, one could define an enriched version of this notion with greater expressive power. This enriched version must be carefully designed so that the generated formulae can be checked.

• The last immediate perspective relates to the active framework. Whether it be for diagnosis or opacity, we only focused on exact notions. It would be natural to tackle the, usually harder, approximate notions. This is in fact an ongoing work.

The current results seem to point toward decidability for AFF-diagnosability in active systems while similar approximate notions are undecidable for opacity, both for maximisation and minimisation.

We now turn to mid-term perspectives. They correspond to problems that are strongly connected to this thesis, while not being immediate.

• In our active framework, the observations are clearly given by the model. This represents in reality sensors within the system. Using a sensor has a cost. Therefore, instead of having fixed sensors, one could have a list of potential sensors associated with costs. The goal would then be to obtain diagnosability while minimising the cost. A cost could also be given to having the sensor turned on, forcing the optimal strategy to decide when it needs to have the sensor operating. Some works were already done on this subject, see [START_REF] Cassez | Fault diagnosis with static and dynamic observers[END_REF] and [START_REF] Thorsley | Active acquisition of information for diagnosis and supervisory control of discrete-event systems[END_REF].

• Faults, as defined in this document, are a boolean property: a system is either faulty or correct. Moreover, they are permanent. Once a fault occurred, we did not consider as important to decide if more faults would be created later for example. No matter the number of faults, the run is deemed faulty. One could envision a different idea for the fault. A fault could represent a partial degradation of the system, the failure of one of its non-vital component or something that can be repaired (see [START_REF] Fabre | Diagnosability of repairable faults[END_REF] for a study of repairable faults in a non-stochastic framework). Seen this way, many new questions arise. In passive systems, this means defining measures of correctness for a system and testing properties, for example on the delay of the fault counter (one can test if this delay is bounded, if it is unbounded but in o(n) where n is the length of the run, in O(n). . . ).

In active systems, we would wish to find controllers that optimise the measures of interests or that ensure good delays of detection for the fault counter. This would obviously only be an interesting study for systems where many faults will be triggered. However, this is not an unrealistic assumption as every system is slowly degraded due to time elapsing.

• Our diagnosability algorithms currently only gives a Boolean answer. In order for system designers to modify its system so that it becomes diagnosable for example, they need to know what is the cause of the missed fault. As a consequence, it would be interesting to design algorithm able to give a counter-example to the diagnosability of the system whenever it exists. This notion of counter-example needs to be made precise. In an LTS, a counter-example can be given by an ambiguous cycle of the system. As with probabilistic systems, this cycle may have a zero probability, the counter-example must thus be able to describe a set of runs with positive probability. For example, by giving a finite faulty run such that any extension of it is ambiguous. However, this kind of counter-example does not necessarily exist, in pushdown systems for example. The generation of counter-examples in stochastic systems has been studied, see [ÁBD + 14], but for different objectives than diagnosability.

• The main formalism we chose, probabilistic labelled transition systems, has its limitations. One could be interested in studying higher-level models such as stochastic Petri nets, stochastic process algebra (with PEPA for example [Hil96, Chapter 3]), etc. can be more appropriate to represent some real life systems. Indeed, high-level formalisms usually are (often exponentially) more concise than low level formalisms, hence a greater comfort for designers. Moreover, high-level formalisms usually have a structure, which allows to conceive more efficient algorithms as the generated systems (pLTS, MDP, etc.) benefit from additional properties. For example, when a system is a synchronised product of several components, the transition matrix or the infinitesimal generator can be computed using tensor products of the matrix representing the different components (see e.g. [START_REF] Haddad | Evaluation of high-level Petri nets by means of aggregation and decomposition[END_REF][START_REF] Haddad | Asynchronous composition of high level Petri nets: A quantitative approach[END_REF]).

• During this thesis, we designed many algorithms. In parallel to the previous item, it would be useful to implement these algorithms. This tool could then be used to solve the diagnosability problems for pLTS or for some higher-level models whose semantics is an appropriate pLTS. This implementation should be integrated within an existing tool to benefit from the possibilities offered while enriching it. A good candidate would be COSMOS [BBD + 15], a statistical model checker for the hybrid automata stochastic logic.

We now end with the long-term perspectives, more remote to our current work.

• We established many results within this thesis. Some of which used methods that are quite usual, some required to use new ideas such as the pathL logic. These new ideas may be useful in order to tackle other issues related to partial observation such as identification problems, stochastic games. . . It could therefore be interesting to see which kind of problems, in these other frameworks, would benefit from our approach.

• Another direction comes from another interpretation of faults. Let us proceed through an example. When someone is sick, multiple symptoms appear or do not appear in the body. These symptoms each correspond to a failure of the human body, so each are a fault that must be detected. However, in order to cure the patient one needs to link the patterns to a common cause: an illness.

In other words, one wants to deduce a meta-information from the behaviour of the system. Finding the origin of the fault has been studied under the term causality, but, as explained in [START_REF] Gößler | Counterfactual causality from first principles?[END_REF], this approach focused on static systems.

In dynamic systems, this meta-information may be seen as a pattern that must be detected [START_REF] Jeron | Supervision patterns in discrete event systems diagnosis[END_REF]. This objective can be seen as an extension of diagnosis, but it goes into the long-term category as it corresponds to what seems to be a far more general question. It can also be linked to questions of identification of complex behaviour, possibly similar to what was studied in [START_REF] Piel | Online event flow processing for complex behaviour recognition[END_REF].

• Diagnosis is a research domain with clear applications. As a consequence, it would gain a lot from being studied in cooperation with the industry. This would allow researchers to better understand the industry's need. For example, there could exists definitions of diagnosability that we did not focus on, yet have relevance from an industrial point of view. Moreover, while our methods are efficient in theory, they may raise practical issues that we did not consider. There are already some cooperations such as [START_REF] Hounkonnou | Empowering self-diagnosis with selfmodeling[END_REF] where the authors investigate the problem of building a model appropriate for diagnosis out of a real system, in their case, a network.

• The last perspective discussed here is of a different nature: it is not a research direction. However, it could still have a great impact on the domain of research. There exist many contributions which either establish a known result or have an erroneous proof for a theorem that was already proven false (trying to give a PTIME algorithm for a PSPACE-hard problem for example). This clearly shows the difficulty for researchers to know the current state of the research, even for a specific domain such as diagnosis. To tackle this issue, one possibility may be to try to build a cooperative website that would gather all the results on diagnosis, something in the spirit of the POMDP webpage [POM]. This way, one could efficiently find the current state of the art on the domain and this would save a lot of useful time to many researchers. This obviously has some issues. To be useful, the existence of such a website has to be spread and people must keep it up to date. It would also raise many questions of organisation due to the width of the domain, even when restricted to diagnosis (the many existing frameworks, diagnosability notions, methods of approach,...). The various surveys on diagnosis issues [START_REF] Zaytoon | Overview of fault diagnosis methods for discrete event systems[END_REF][START_REF] Basile | Overview of fault diagnosis methods based on Petri net models[END_REF] would help dealing with this point. But most of all, building such a website is extremely time-consuming.

Figure 0

 0 Figure0.1: Un pLTS représentant une machine à café. q 0 est l'état initial, ce qui est représenté par la flèche entrante. Les transitions entre les états sont étiquetées par l'évènement provoquant cette transition ainsi que par la probabilité que cette transition soit prise.

Figure 0 . 3 :

 03 Figure 0.3: Quand un diagnostic approché est nécessaire.

Figure 0 . 4 :

 04 Figure 0.4: Un pLTS qui est IA et FF-diagnostiquable mais n'est pas FA-diagnostiquable.

Figure 0

 0 Figure 0.5: L'automate déterminisé associé au pLTS de la figure 0.4.

Figure 0 . 6 :

 06 Figure 0.6: Produit synchronisé du pLTS de la figure 0.4 et de son automate déterminisé. Les probabilités sont omises pour faciliter la lisibilité.

Figure 0 . 7 :

 07 Figure 0.7: Exemple de diagnostiqueur à mémoire finie. Le verdict donné par D fm dans un état de la mémoire est indiqué sous l'état.

Figure 0 . 8 :

 08 Figure 0.8: Un automate à pile probabiliste et deux de ses exécutions finies.

Figure 0 . 9 :

 09 Figure 0.9: Un exemple de CLTS. Les probabilités sont remplacées par des poids. Le seul évènement contrôlable est 'b'.

Figure 0

 0 Figure 0.10: Un pLTS obtenu en contrôlant le CLTS de la figure 0.9.

Figure 0

 0 Figure 0.11: Un exemple d'OMC.

Figure 0 .

 0 Figure 0.12: Un example d'OMDP. Les transitions sont étiquetées par des paires d'action et de la probabilité de prendre cette transition si cette action est choisie.

Figure 2 .

 2 Figure 2.3: A pLTS representing a coffee machine. The probability of a transition is given next to the event labelling it.

Figure 2 . 5 :

 25 Figure 2.5: Detecting correctness for longer and longer prefixes of correct runs.

Figure 2 . 6 :

 26 Figure 2.6: When approximate diagnosis is necessary.

Figure 2

 2 Figure 2.7: When reactivity cannot be uniform.

Figure 3 . 1 :

 31 Figure 3.1: The finite-memory diagnoser of Example 3.1. The verdict given by D fm in a memory state is written below the state.

Definition 3. 3 .

 3 An FF-diagnoser for a pLTS A is a function D : Σ * o → { , ?} such that: commitment For every w w ∈ Σ * o , if D(w) = then D(w ) = . correctness For every w ∈ Σ * o , if D(w) = then w is surely faulty. reactivity For every ρ ∈ minF, P({ρ ∈ Ω | ρ ρ ∧ D(P(ρ )) =?}) = 0 where for w ∈ Σ ω o , D(w) = lim n→∞ D(w ≤n ).

Figure 3

 3 Figure 3.3: An FA-diagnosable pLTS.

Figure 3

 3 Figure 3.4: The finite-memory diagnoser of Example 3.3.

Figure 3 . 5 :

 35 Figure 3.5: An infinite FApLTS which does not accept any FA-diagnoser.

Figure 3 . 6 :

 36 Figure 3.6: An IA-diagnosable pLTS.

Figure 3 . 7 :

 37 Figure 3.7: The finite-memory diagnoser of Example 3.3.

Figure 3 . 9 :

 39 Figure 3.9: An automaton representing the diagnoser of Example 3.5.

Figure 3 .

 3 Figure 3.16: An infinitely-branching IA-diagnosable pLTS which does not satisfy P =0 (32(U ∧ W)).

Figure 3 .

 3 Figure 3.18: A family of pLTS whose underlying LTS has no appropriate characterisation of FA-diagnosability.

Figure 3 .

 3 Figure 3.19: Another family of pLTS whose underlying LTS has no appropriate characterisation of FA-diagnosability.

  Figure 4.2: The deterministic automaton associated with the pLTS of Figure 4.1. The states that are framed twice are accepting for the Büchi condition.

  Proposition 4.1 ([HHMS13]). Let A be a finite pLTS. Then the deterministic Büchi automaton Obs(A) accepts exactly the infinite unambiguous observed sequences of A. Example 4.2. As seen in Example 4.1, every observed sequence of the pLTS of Fig-

  Example 4.3.

  Figure 4.3 illustrates this construction on the pLTS of Figure 4.1. This automaton reflects that once b happens, the current signalling run is surely faulty. Thus the set of possible correct states is empty (state s 2 ).

Figure 4

 4 Figure 4.3: The FF-automaton of pLTS of Figure 4.1.

Figure 4

 4 Figure 4.4: The synchronised product of the pLTS of Figure 4.1 and its FF-automaton.

Figure 4

 4 Figure 4.5: The FA-automaton of the pLTS of Figure 4.1.

  Example 4.5.

  Figure 4.5 illustrates this construction on the pLTS of Figure 4.1. As expected, the FA-automaton is a refinement of the FF-automaton: the U -component of a state in FA(A) corresponds to a state in FF(A). For instance, state s 2 of Figure 4.3 is split here into s 2 , s 3 and s 4 .

  Figure 4.6 continues Example 4.5 and shows the synchronised product for the pLTS of Figure 4.1.

  Figure 4.6: The synchronised product of pLTS of Figure 4.1 and its FA-automaton.

Figure 4

 4 Figure 4.7: The synchronised product of the pLTS of Figure 4.1 and its IA-automaton.

Figure 4

 4 Figure 4.8: The LMC obtained from the pLTS of Figure 4.1.

Figure 4

 4 Figure 4.11: A uniformly AFF-diagnosable initial-fault pLTS.

Figure 4

 4 Figure 4.13: Illustration of the proof of Lemma 4.3.

  the probability of a correct (resp. faulty) run with observed sequence P(ρ), p f = min(P(ρ) | ρ ∈ R F σ ) and p c = ρ∈R C σ P(ρ). By a worst-case reasoning, one gets pP(ρ)

Figure 4 .

 4 Figure 4.14: An example of probabilistic automaton.

Figure 4 .

 4 Figure 4.17: A reduction for PSPACE-hardness of IF-, FAand IA-diagnosability.

Figure 4

 4 Figure 4.18: An FF-diagnosable pLTS requiring an FF-diagnoser with exponential size.

reactivity

  Let ρ be a signalling run such that D(P(ρ)) =?. Due to the characterisation of Proposition 4.4, the strictly connected component of A FA that ρ has reached cannot be a BSCC. So given some n ∈ N, P({ρ ∈ Ω | ∃m ≥ n D(P(ρ ↓m ) =?}) ≤ P({ρ ∈ Ω | ρ ↓n does not reach a BSCC}). Thus P({ρ ∈ Ω | D inf (P(ρ)) =?}) = lim n→∞ P({ρ ∈ Ω | ∃m ≥ n D(P(ρ ↓m ) =?}) ≤ lim sup n→∞ P({ρ ∈ Ω | ρ ↓n does not reach a BSCC}) = 0 .

Figure 5

 5 Figure 5.2: A pPDA generating the pLTS from Figure 5.1 and two finite of its runs.

Figure 5 .

 5 Figure 5.3: A POpPDA for the proof of Theorem 5.1.

Figure 5 .

 5 Figure 5.4: A POpPDA for the proof of Theorem 5.2.

Figure 5

 5 Figure 5.5: The VPA A(V) associated with the POpVPA V, Σ o , P 2 of Example 5.3 with two runs. The tag V was ignored to remove redundancy and simplify the figure.

Figure 5 .

 5 Figure 5.10: Content of the box Box 1 .

Figure 6 . 2 :

 62 Figure 6.2: An example of CLTS. Weights are all equal to 1 and omitted on the figure. The only controllable event is b.

Figure 6 . 3 :

 63 Figure 6.3: An example of controlled CLTS.

1 , 1 Figure 6 . 5 :

 165 Figure 6.5: Reduction to (γ, v)-fault free active diagnosability.

Figure 7 . 1 :

 71 Figure 7.1: Pushing observations from states to transitions.

Figure 7 . 4 :

 74 Figure 7.4: Reduction to the positive ε-disclosure problem.

Figure 7 . 5 :

 75 Figure 7.5: An example of OMDP where the initial distribution is a Dirac distribution on q 0 . Transitions are labelled by a set of pairs of actions and of the probability to take this transition under that action.

Figure 7

 7 Figure 7.6: The OMC induced by the strategy choosing the action b initially, then always choosing action a for the OMDP of Figure 7.5.

  for all c ∈ {a, b, , }, O(r c ) = c, and for all c ∈ {a, b, }, O(s c ) = c.

  o . The observation function Obs is defined by Obs((s, B)) = O(s). The set of actions of M is Act = DR where DR is the set of vectors of deterministic decision rules δ over S. Given a state (s, B) ∈ S , an action δ ∈ DR and an observable state s , we have T ((s, B), δ)(s , B ) = |s, δ(s)), for B = {s | O(s ) = O(s )∧ ∃ŝ ∈ B, p(s |ŝ, δ(ŝ)) > 0} 0, otherwise.

  * o and s ∈ S, the sets of finite runs R(w, s) = {ρ finite run of M σ (µ 0 ) | O(ρ) = w ∧ last(ρ) = s} and R (w, s) = {ρ finite run of M (τ ) | O(ρ ) = w ∧ last(ρ ) = (s, B σ w )}. • Let τ be a scheduler of M , defined on Σ * o . We define σ for any observation w ∈ Σ * o and state s ∈ S by σ(w, s) = τ (w)(s, B σ w ). Note that a τ -compatible run ρ of M ends in a state (s, B) where B = B σ w (the belief w.r.t. σ) if O(ρ ) = w. We have P σ (R(w, s)) = P τ (R (w, s)).

  2 s }∪{ c | ∈ L, c ∈ Σ×O}∪{s c | c ∈ Σ×(O∪{ })}∪{r c | c ∈ Σ × (O ∪ { , })}; • Act = Σ; • For all a ∈ Σ, o ∈ O, c ∈ S, ∈ obs -1 (o), p( a,o | c , a) > 0 iff ( , a, ) ∈ ∆. If ∈ obs -1 (F ) then p(q 1 | c , a) > 0 and if ∈ obs -1 (F ) then p(q 2 | c , a) > 0. For all a ∈ Σ, c ∈ {0} ∪ (Σ × (O ∪ { })), (b , o ) ∈ Σ × (O ∪ { }), p(s (b ,o ) | s c ,a) > 0 and if b = a, p(r (b ,o ) | s c , a) > 0. For all c, c ∈ Σ × (O ∪ { }), p(r c | r c , a) > 0 and p(q 2

Figure 7

 7 Figure7.10: With Sec = {q 2 , q 3 }, deterministic strategies are not sufficient for minimisation.

  1. If |A(s)| > 1 then δ ε (a) = 1 -ε and for all b ∈ A(s) \ {a}, δ ε (b) = ε |A(s)|-1 ; 2. Else δ ε = 1 a . δ ε is said to favour a.ε-decision rules are used to define approximations of deterministic strategies. Definition 7.10. Let σ be an observation-based deterministic strategy. Then {σ ε } ε>0 is a family of observation-based almost-deterministic strategies defined for any state s and w ∈ Σ n by: σ ε (w, s) = σ(w, s) 2 -n ε .

  Given a set of potential current states B and a new observation o, we define the maximal set of potential next states NextMax(B, o) over decision rules applied to B by: NextMax(B, o) = {s ∈ O -1 (o) | ∃s ∈ B ∃a ∈ A(s) p(s |s, a) > 0}.

•

  for every (s, B), (s , B ) ∈ S min , p((s , B )|(s, B), a) = p(s |s, a) if B = NextMax(B, O(s )), 0 otherwise; • for every (s, B) ∈ S min , O(s, B) = O(s). Given µ 0 an initial distribution over S, the associated initial distribution µ min over S min is defined by µ min (s, Supp(µ 0 ) ∩ O -1 (O(s))) = µ 0 (s) and µ min (s, B) = 0 for all other B. We define the subset Avoid(Sec) ⊆ S min by Avoid(Sec) = {(s, B)|B ⊆ Sec}. Proposition 7.4. The minimal disclosure value for Sec in M(µ 0 ) is equal to the minimal probability to reach Avoid(Sec) in M min (µ min ). Furthermore it is asymptotically reached by a family of belief-based almost deterministic strategies. Opacity 4 Fixed-horizon problems

φ

  ::= ψ | ∃x.φ | ∀x.φ ψ ::= x | ψ ∧ ψ | ψ ∨ ψ | ¬ψ | true A QBF is closed ifevery Boolean variable is bound by a quantifier. Deciding if a closed QBF is valid (i.e. equivalent to true) is PSPACE-hard [Sip06].

Figure 7

 7 Figure 7.11: Reduction of the validity problem to the disclosure on a fixed horizon. The box S x 1 is represented in Figure 7.12.

Figure 7

 7 Figure 7.12: Representation of the box S x 1 . We use the convention of Figure 7.8.

Figure 7

 7 Figure 7.13: Reduction of the validity problem to the disclosure on a fixed horizon. The box S x 1 is represented in Figure 7.14

Figure 7 .

 7 Figure 7.14: Representation of the box S x 1 (with the conventions of Figure 7.8).

  1. • O(s end ) = , O(s b a ) = a and O(s a ) = a when a is a Boolean variable, its negation or . For i = 1 . . . n, O(s x i end ) = O(s x i f ) = O(s ¬x i f ) = 2i -1 and O(s y i end ) = O(s y i f ) = O(s ¬y i f ) = 2i, and for any other state s, O(s) = .

  FF-diagnosers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.2 FA-diagnosers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.3 IA-diagnosers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.4 εFF-diagnosers . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 Relationships between diagnosability notions . . . . . . . . . . . . . . . .

	I Introduction
	1 General Introduction
	2 Preliminaries
	1	Framework . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.1 Descriptive set theory . . . . . . . . . . . . . . . . . . . . . . . . 1.2 Probabilistic Labelled Transition Systems . . . . . . . . . . . . . 1.3 Partial observation . . . . . . . . . . . . . . . . . . . . . . . . . . 1.4 Fault and ambiguity . . . . . . . . . . . . . . . . . . . . . . . . . 1.5 Which diagnosis for pLTS? . . . . . . . . . . . . . . . . . . . . . 2 State of the art on diagnosis . . . . . . . . . . . . . . . . . . . . . . . . . II Analysing information in passive systems 3 Semantical analysis of diagnosability 1 Diagnoser and diagnosability . . . . . . . . . . . . . . . . . . . . . . . . 1.1 3 Characterisation of diagnosability . . . . . . . . . . . . . . . . . . . . . . 3.1 The logic pathL . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2 Logical characterisation of diagnosability . . . . . . . . . . . . . . 3.3 Non-expressivity results . . . . . . . . . . . . . . . . . . . . . . . 4 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

  , on ne peut pas se limiter à des stratégies déterministes. La capacité du contrôleur à agir de façon randomisée est importante pour rendre le système opaque. Pour autant, les problèmes sont beaucoup plus facile à résoudre : la révélation exacte de l'OMDP peut être calculée. En effet, bien que l'on ne puisse pas utiliser de stratégies déterministes, on peut se limiter à un type de stratégies particulier, que nous avons nommé quasi-déterministe. Ces stratégies choisissent une action et lui associent une probabilité proche de 1, puis partagent le reste de la probabilité sur toutes les autres actions.

Le problème est également EXPTIME-difficile (réduction depuis les jeux de sécurité à information partielle), il est donc EXPTIME-complet. Pour la minimisation, la situation est différente, d'une façon surprenante. Tout d'abordN'utiliser que des stratégies de cette forme permet de réduire le problème de la minimisation de la révélation à un problème d'accessibilité dans les processus de décision Markovien. Contrairement au cas de la maximisation, ceux-ci sont totalement observables et plus de problèmes sont décidables dans ce cas. Notamment, minimiser la probabilité d'accessibilité lorsque l'observation est complète peut être réalisé en temps polynomial. Donc comme le processus que l'on construit est de taille exponentielle, on obtient un algorithme EXPTIME. Le problème n'est cependant pas prouvé EXPTIMEdifficile. La meilleure borne inférieure dont l'on dispose est PSPACE et est obtenue par réduction de la validité d'une formule booléenne quantifiée (QBF).

  ).

		Reactivity
		uniform	uniform	uniform
	Uniform	εFF-diagnosis	AFF-diagnosis	FF-diagnosis
	Simple	εFF-diagnosis	AFF-diagnosis	FF-diagnosis
				Correctness
		ε approximate	Accurate approximate	Exact
		Figure 2.9: Summarising the approximate variants of FF-diagnosis.

  there exists a firing sequence σ such that M 0 [σ M . The set of all markings reachable from M 0 defines the reachability set of N and is denoted R(N ). Given k ∈ N, a place p of a PN N is Figure 5.8: A Petri net. Circles are places and rectangles are transitions. In the initial marking, p 1 has two tokens represented by the two black dots.

	t 0	p 1	t 2	p 3	t 4	p 4	t 6
	t 1
			t 3			t 5
		p 2			p 5

k-bounded if for all M ∈ R(N ), M (p) ≤ k. It is bounded if there exists k ∈ N such that p is k-bounded. A PN is bounded (resp. k-bounded) iff

all of its places are bounded (resp. k-bounded).

Pour la IA-diagnostiquabilité, une information supplémentaire est nécessaire. L'automate déterminisé que l'on construit possède un troisième ensemble permettant de partitionner les états fautifs en deux groupes, selon le moment où la faute a été commise.

à réaliser dans cette direction. Le cas des systèmes contrôlables a enfin été étudié. Pour ceux-ci, un autre angle de questionnement a été utilisé : il ne s'agit plus seulement de déterminer la diagnostiquabilité du système. En premier lieu, il s'est agi de combiner le diagnostic avec une limitation de la dégradation du système. Combiner ces deux problèmes ne fait sens que pour des systèmes actifs : pour un système passif, les deux problèmes peuvent être vérifiés séparément, au contraire, dans des systèmes actifs, il peut exister pour chaque propriété une stratégie la vérifiant, mais aucune stratégie ne satisfait les deux simultanément. Dans un second temps, il a été question de l'analyse de l'opacité, une autre notion de contrôle de l'information produite par un système.

One technique is to obtain a Büchi automaton that is equivalent to the model and another one that is equivalent to the negation of the property. The intersection of the two non-deterministic Büchi automata is empty if the model satisfies the property.

This definition is written slightly differently from the one of Definition

2.9, page 48. Yet, one can quickly see that they are equivalent. However, we cannot easily express uniformity with a similar definition.

Note that n≤n 0 minFn is finite due to A being finitely branching and convergent.

This pLTS was already displayed in Figure3.2.

Using such a product to enrich the initial model was mentioned as an usual technique page

There, the deterministic automaton was called belief automaton.

Recall that the set Q f is absorbing

q 0 , s 0 q 2 , s 0 q 2 , s 1 q 2 , s 1 f 1 , s 0 f 1 , s 3 f 1 , s 1 f 1 , s 2 f 1 , s 1 f 1 , s 2 f 2 , s 1 f 2 , s 1 f 2 , s 2 f 2 , s 2 q 1 , s 0 q 1 , s 1 q 1 , s 1 f 2 , s 4 f 2 , s 4 u f

The term used in the literature is event. We differ here as we already use event for the letters labelling the transitions as established in Definition 2.5, page 38

one of the most famous example is the undecidability of the emptiness problem for probabilistic automaton[START_REF] Paz | Introduction to probabilistic automata[END_REF] that we detail below, see also[START_REF] Madani | On the undecidability of probabilistic planning and related stochastic optimization problems[END_REF] for some examples of undecidable problems for partially observable Markov decision process, a formalism which also contains a form of control of the system

This value is obtained by analysing the case where only one place contains tokens.

The difference with the construction of [BFH + 14] is that we focus here on FF-diagnosability rather than IA-diagnosability which simplifies the writing of the proofs. The results of this chapter and of [BFH + 14] however hold for both notions of diagnosability.

In the case of Proposition

4.9, page 120, the restriction in fact made the problem more difficult.

This can be ensured similarly to what was done for the set Q f in Subsection 1.4 of Chapter 2, page 43.

The equivalent of the observation function for pLTS was called mask function.

For the case n = 0 one must also consider the signalling runs obtained from runs starting in states of Supp(µ0) \ O -1 (ε). There are finitely many such runs too.

Note that since observations are not useful for this reachability objective, observations could be removed from the OMDP Mmin, yielding a Markov decision process, which is the model studied in[START_REF] Baier | Principles of model checking[END_REF].

Part II

Analysing information in passive systems

Controlling Information in Active Systems

Theorem 6.5 derives from Propositions 6.4 and 6.5 below, that state, respectively, the upper bound in the general case, and the lower bound in a particular case, namely for the safe active diagnosability under finite-memory strategies. Proposition 6.4. Given a POMDP M with subsets of states F and I, one can decide in EXPTIME whether there exists a finite-memory scheduler τ such that P M τ (23F ) = 1 and P M τ (2I) > 0.

Due to the complexity of this proof, we decompose it using two lemmas. The idea is the following. We first define a set Win =1 of pairs of beliefs (B, B ) with B ⊆ B such that there exists a scheduler that ensures with probability 1 to stay in I from any state of B and to reach F from any state of B . As B ⊆ B , this implies that if one starts with a distribution which support is B , there is a scheduler satisfying both the Büchi objective with probability one and the safety objective with positive probability. Such a belief B is a "winning" belief. However, there are "winning" beliefs that cannot be obtained directly from Win =1 . In Lemma 6.1, we show how to compute efficiently Win =1 through a greatest fixed point algorithm. Using Win =1 , we then build a set of beliefs Win, which contains intuitively the beliefs from which there exists a scheduler that can reach a belief that corresponds to the second component of a pair in Win =1 with positive probability while never reaching a "losing" belief (a belief from which we cannot satisfy the Büchi requirement). Finally, in Lemma 6.2 we show that Win contains exactly the set of "winning" beliefs. Thus, there exists a scheduler satisfying the two objectives iff the initial belief {q 0 } belongs to Win.

Proof. In this proof, the POMDP M = Q, q 0 , Obs, Act, T is fixed, and we use notation P δ 0 τ (Ev) to denote the probability of Ev under scheduler τ assuming that instead of q 0 , the initial state in M is given by the distribution δ 0 ∈ Dist(Q).

Let us first explain how to compute the following set of pairs of beliefs:

∀δ 0 with Supp(δ 0 ) = B, P δ 0 τ (23F ) = 1, and ∀δ 0 with Supp(δ 0 ) = B , P

Intuitively, Win =1 denotes pairs of beliefs such that there exists a scheduler that ensures a Büchi objective almost-surely from the larger belief, and a safety objective almostsurely from the smaller one. Note that, (1) in the definition of Win =1 , we do not require the scheduler τ to be finite-memory and (2) as schedulers associate a decision rule to every sequence of observation, the same choices are taken after the same sequence of observations for the Büchi and the safety objective although the initial distribution differs. Given that we consider pairs of beliefs, we introduce the following notation: ∆((B , B), O 1 ) = ∆(B , O 1 ), ∆(B, O 1 ) , and similarly for sequences of actions and observations. Also, for X ⊆ Q a subset of states, we denote by Bl ⊆X = {B ∈ Bl | B ⊆ X} the set of beliefs contained in X.

We now show how to efficiently compute Win =1 .

Control of the degradation in probabilistic systems

Lemma 6.1. Let Win ∞ be the greatest fixed point starting from {(q, B , B) ∈ Q × Bl × Bl | q ∈ B, B ⊆ B, B ⊆ I} of the following operator:

We have

Proof of Lemma 6.1. To establish that Win =1 corresponds to the projection on the pair of beliefs of Win ∞ , we first assume that for all q ∈ B, (q, B , B) belongs to Win ∞ , and exhibit a scheduler τ that witnesses (B , B) ∈ Win =1 . Let us define τ as follows. The scheduler τ has finite memory Bl × Bl. From memory state (B , B), τ dictates to play uniformly all actions α such that for every observation O and every q ∈ ∆(B, α, O), we have (q, ∆((B , B), α, O)) ∈ Win ∞ . Note that this set of "safe" actions is necessarily non empty because (q, B , B) ∈ Win ∞ . If α is played, and O is observed, the memory state of τ is updated to ∆((B , B), α, O), which is still in Win ∞ , by assumption on α. The scheduler τ then continues similarly with memory state ∆((B , B), α, O). So defined, let us show that τ witnesses (B , B) ∈ Win =1 . First, let δ 0 be a distribution with support B. The scheduler τ ensures to stay (surely) in Win ∞ . Moreover, for every q ∈ B, with a positive probability, say p (q,B ,B) > 0, the sequence

actions and observations leading to F that derives from the fixpoint definition, happens from q. There are finitely many p (q,B ,B) , all are positive, so they are lower bounded by some positive value p. Playing τ forever thus ensures visiting F almost surely, and iterating this reasoning, even visiting F infinitely often with probability 1. Now, assuming B = ∅ let δ 0 be a distribution with support B . Any action picked by τ ensures that, whatever the observation, the first belief-component remains in I. Therefore, surely, from distribution δ 0 the plays stay in the invariant I.

Let us now assume that the triplet (q, B , B) is removed during the iterative computation of the fixed point W ∞ . We prove, by induction on k, that if (q, B , B) is removed at iteration k, then, (B , B) / ∈ Win =1 . If k = 0, the pair is removed at initialization, hence B ⊆ I or B ⊆ B, and obviously (B , B) / ∈ Win =1 . Otherwise it happens at the kth iteration, for some k ≥ 1. Assume, towards a contradiction, that there exists a scheduler τ , witnessing that (B , B) ∈ Win =1 . In particular, there exists a sequence of pairs of actions and observations allowed by the scheduler (α 1 , O 1 ) • • • (α n , O n ) so that there exists q 0 . . . q n ∈ Q with q 0 = q, q n ∈ F, ∀i < n, T (q i , α i+1 )(q i+1 ) > 0, ∀1 ≤ j ≤ n and Obs(q j ) = O j . Because the triple (q, B , B) was removed at iteration k, it must be that, either (1) for 1) and starting in q ends in F or (3) there exists an index i and an observation O i such that for

In the first case, it means that Conversely, if Disc ε (M(µ 0 )) > 0, then there exists an observation w in ({a, b, , }) * such that Psec M(µ 0 ) (w ) > 1 -ε. In this case, w is of the form a 1 . . . a n * where, letting w = a 1 . . . a n , we have Psec M(µ 0 ) (w ) = Psec M(µ 0 ) ( a 1 . . . a n ) = P A (w). Therefore L >1-ε (A) is not empty. This undecidability result leads us to consider the simpler case where the disclosure is the probability of the set of runs surely leaking the secret, i.e., such that all runs with the same observation are secret. One such disclosure notion, the ω-disclosure (used in [BCS15, BMS15, BKMS16]), was defined for a Markov chain M = (S, p, O) with initial distribution µ 0 by considering a measurable set of secret runs SecRuns ⊆ Ω M(µ 0 ) . In our context, as mentioned earlier, SecRuns is Reach(Sec), the set of infinite runs visiting a state from Sec. Moreover an infinite observation w ∈ Σ ω discloses the secret if all runs ρ ∈ O -1 (w) are secret. Setting SecRuns = Ω M(µ 0 ) \ SecRuns, we define: Definition 7.3. For an OMC M = (S, p, O), an initial distribution µ 0 and a subset Sec ⊆ S, with SecRuns = Reach(Sec), the ω-disclosure of Sec in M is:

The downside of this definition is that it only considers infinite observed sequences. In reality, an attacker will only have access to finite observed sequences before having to deduce if the system is in a secret state. To obtain measures directly related to the finite observation of a potential attacker, we assume that M = (S, p, O) is convergent: each infinite run ρ has an infinite observation O(ρ) ∈ Σ ω . Two measures can then be defined: using fixed or finite horizon. In the fixed-horizon case, the attacker observes the system for a fixed amount of time and has to make his deduction at the end of this observation. In order to link the amount of time the attacker observes the system and the number of observations they receive, in this case, we only consider non-erasing observation functions O. In the finite-horizon case, the attacker can wait as long as they want, as long as it is a finite amount of time. Note that if D is the set of disclosing observations, and V(µ 0 ) = ∪ w∈D ∪ ρ∈O -1 (w) Cyl(ρ) the set of runs disclosing the secret, then Disc(M(µ 0 )) equals P M(µ 0 ) (V(µ 0 )).

As waiting longer gives more information, the disclosure with finite horizon is always at least as large as the disclosure with fixed horizon. In fact, (Disc n (M(µ 0 ))) n∈N is a non-decreasing sequence with limit Disc(M(µ 0 )).

Opacity

Case 1. By induction on η(s), we consider a state s ∈ S u such that η(s) = min s ∈Su (η(s )).

Assuming the property holds for (ε, s) with η(s) ≤ n, we prove it for s with η(s ) = n + 1. We have:

and using the definition of σ yields:

Applying the induction hypothesis on (ε, s)

Case 2. We now consider w = o ∈ Σ and s ∈ S o , hence O(s ) = o. Then:

and a reasoning similar as above yields the result.

For the induction step, we first need to prove the property for (w, s ) with s ∈ S u , assuming it holds for all (w, s) with s ∈ S o and for all (w, s) with s ∈ S u and η(s) < η(s ). Then we have:

and we can conclude along the same lines as above.

Finally, we consider (w , s ) with w = wo ∈ Σ * Σ o and s ∈ S o , with:

ρ∈R(w,s)

which again implies the desired result.

Opacity

thanks to the earlier remark, we have that the probability of reaching Sec × 2 Sec in M under strategy τ is also the probability of disclosing Sec in M under strategy σ.

• Conversely, given an observation-based strategy σ of M, we build a scheduler τ of M as follows: we define the mapping τ (w) : Σ * o → Dist(Act) by τ (w)(s, B σ w ) = σ(w, s) for any s ∈ S. With the same reasoning as above, we immediately get that the probability of reaching Sec × 2 Sec in M under strategy τ is also the probability of disclosing Sec in M under strategy σ.

We can conclude that M is almost-surely disclosing if and only if the runs of M reach almost-surely the set Sec × 2 Sec . Moreover, if M almost-surely reaches the set Sec × 2 Sec , we can build a scheduler τ doing so. Using the transformation described above and the results from [START_REF] Chatterjee | Qualitative analysis of partially-observable Markov decision processes[END_REF], we extract from τ a belief-based strategy σ of M that almost-surely discloses the secret.

Let us argue that the whole algorithm is in EXPTIME. The exponential in the algorithm of [START_REF] Chatterjee | Qualitative analysis of partially-observable Markov decision processes[END_REF] comes from a determinisation of the system, which is already done in our transformation from M to M , and thus not required a second time. Moreover, the non-erasing assumption on the observation could have created exponentially many new actions, which are exactly the ones built by the use of a vector of decision rules in our construction. Thus the exponentials do not stack.

The hardness is shown with a reduction from safety games with imperfect information.

Definition 7.8. A safety game with imperfect information is defined by a tuple G = (L, 0 , Σ, ∆, O, F, obs) where • L is a finite set of locations with initial location 0 ∈ L;

• Σ is a finite alphabet;

• ∆ ⊆ L × Σ × L is the transition relation such that for all ∈ L and a ∈ Σ there exists at least one with ( , a, ) ∈ ∆;

• O is a finite set of observations, and F ⊆ O are the final observations;

• obs : L → O is the observation mapping.

A safety game with imperfect information G is a turn-based game played by two players Control and Environment. It starts in location 0 with Control to play. In the first round, Control chooses a letter a 0 ∈ Σ, then Environment chooses a location 1 such that ( 0 , a 0 , 1 ) ∈ ∆ and Control only observes o 1 = obs( 1 ). The next rounds are played similarly and Control wins if for all i, o i ∈ F . The problem of existence of a winning strategy for Control is EXPTIME-complete [START_REF] Berwanger | On the power of imperfect information[END_REF]. We now describe a reduction from this problem to the almost-sure disclosure problem of OMDP.

The reduction is similar to the one in the proof of Theorem 7.2 except that we replace the probabilistic automaton by a safety game G = (L, 0 , Σ, ∆, O, F, obs) with imperfect information and directly build an OMDP M = (S, Act, p, O) over alphabet

Opacity

We show this result in two steps. First we show that, using families of belief-based almost-deterministic strategies, one can obtain a disclosure value in M arbitrarily close to the minimal reachability probability in M min . This ensures that the disclosure value is below this probability. Second, we show that the disclosure obtained by an arbitrary strategy is greater or equal to the probability to reach Avoid(Sec).

Proof. We know that the minimal reachability probability for Avoid(Sec) in M min (µ min ) is obtained by a memoryless deterministic strategy σ min that selects some action a s,B in state (s, B) (see e.g. [START_REF] Baier | Principles of model checking[END_REF]). Consider {σ ε } the family of belief-based almostdeterministic strategies defined by favouring a s,B in state s after a run ρ such that B σε ρ = B. Given a run ρ = s 0 a 0 . . . a n-1 s n in M(µ 0 ) we inductively define the run b(ρ) = (s 0 , S 0 )a 0 . . . a n-1 (s n , S n ) in M min (µ min ) by: S 0 = Supp(µ 0 ) ∩ O -1 (O(s 0 )) and S i+1 = NextMax(S i , O(s i+1 )). Due to the observation given when introducing NextMax, with strategy σ ε , the observation of run ρ discloses the secret iff b(ρ) reaches Avoid(Sec). Consider, under strategy σ ε , the probability to disclose the secret with runs ρ such that b(ρ) includes at least once an action not selected by σ min . By construction, at each step i, the probability of not choosing the action favoured by σ ε is ε 2 i , hence the probability of this set of runs is i≥0 (1 -ε) i ε 2 i ≤ 2ε. Consider now a finite run s 0 a 0 . . . a n-1 s n such that b(ρ) is σ min -compatible. Then the probability of the original run is less than or equal to the probability of its corresponding run. So we deduce that the minimal disclosure value of M(µ 0 ) is bounded above by ν +2ε where ν is the minimal reachability probability for Avoid(Sec) in M min (µ min ). Since this holds for all ε > 0, we obtain that the minimal disclosure value of M(µ 0 ) is bounded above by the minimal reachability probability for Avoid(Sec) in M min (µ min ).

Conversely consider an arbitrary strategy σ in M(µ 0 ). This strategy may also be applied in M min (µ min ) by forgetting the second component of the state, defining a strategy σ . For any run s 0 a 0 . . . s n in M σ (µ 0 ), there is a single run (s 0 , S 0 )a 0 . . . (s n , S n ) in M min (µ min ) under σ with the same probability. Given the run s 0 a 0 . . . s n , consider the successive associated subsets of beliefs according to σ, B 0 , . . . , B n . By induction (and definition of M min ) it is straightforward to show that B i ⊆ S i . So s 0 a 0 . . . s n does not disclose the secret in M under σ implies that (s 0 , S 0 ) . . . (s n , S n ) does not reach Avoid(Sec). This entails that the reachability probability of Avoid(Sec) in M min (µ min ) under σ is less than or equal to the disclosure probability in M(µ 0 ) under σ.

Since minimal reachability probability in OMDP can be computed in polynomial time(see e.g. [BK08]) 4 we immediately obtain the first part of the next theorem. We establish the second part (PSPACE-hardness) in the proof of Theorem 7.8 as the proof holds also for disclosure over fixed-horizon.

Theorem 7.5. The minimal disclosure value of M(µ 0 ) can be computed in EXPTIME. The associated decision problem is PSPACE-hard.

We now turn to the existence of a strategy that achieves the minimal value. As remarked earlier, we cannot assume such a strategy belongs to an almost-deterministic family of strategies. We establish that it can be analysed without additional complexity.

Theorem 7.6. The existence of a strategy that achieves the minimal disclosure value can be decided in EXPTIME. In the positive case, this strategy can be computed in EX-PTIME.

The main ingredient of the proof is an elimination algorithm that removes iteratively the beliefs from which no strategy can reach the maximal disclosure. Once al these beliefs were removed, if the initial belief was not deleted, then there exists a beliefbased strategy minimising the disclosure and this strategy plays in order to stay within the beliefs kept by the algorithm.

Proof. Let us first introduce multiple notations that are used within the proof. We define disc * (M(s, B)) as the minimal disclosure value when starting in M in state s with belief B. Given some belief B and some decision rule vector δ over B we introduce the possible successors of B when applying δ:

The algorithm simultaneously solves the existence and the synthesis problem. First, using Proposition 7.4, the algorithm computes for all (s, B) ∈ S min , disc * (M(s, B)). Then it maintains a set Win of beliefs initially set to all beliefs from which it iteratively eliminates items and stops when no more elimination is possible. Given B ∈ Win, it looks for a decision rule vector δ over B such that:

If such a δ does not exist then B is eliminated from Win. In other words, a belief is eliminated if there does not exist a decision rule that meets the minimal disclosure value. Each iteration can be performed in polynomial time w.r.t. |S min | and the number of iterations is at most |S min |. Observe that when a belief is eliminated, it should not be "reached" by a strategy that obtains the minimal disclosure value. So the elimination is sound. When the elimination stops, the algorithm answers positively iff for all o ∈ O(Supp(µ 0 )), Supp(µ 0 ) ∩ O -1 (o) ∈ Win. Thus, by the soundness of the elimination step, if the answer is negative there is no optimal strategy for minimal disclosure value. If the answer is positive, let us consider the belief-based strategy σ defined by applying the decision rules obtained during the last iteration of the algorithm. On the one hand, under σ when visiting a state s with belief B such that disc * (M(s, B)) = 0, one never leaves such kind of pairs of states and beliefs. So the secret is never disclosed, showing that the disclosure value obtained by σ for such (s, B) is null. Under σ, the disclosure value of all the other pairs of states and beliefs fulfil the equations of the elimination step. It is known that the single solution of this system is the vector of minimal reachability probabilities of Avoid in M min (µ min ) (see [START_REF] Baier | Principles of model checking[END_REF] for instance) which yields the result.

Opacity

either chosen by the strategy as it chooses whether x i or ¬x i occurs in the observation for all 1 ≤ i ≤ n, or randomly as y i and ¬y i both have equal chance of being triggered. During the last 2m steps, the strategy must trigger a Boolean variable in every clause of the disjunction so that if a clause is not satisfied by the current assignment, then a Boolean variable is observed as both true and false during the run. Thus the observation would not disclose the secret. In order for a measure of 1 2 2n of runs to disclose the secret, for every assignment of the y i the controller must force the run reaching s end to disclose the secret.

Suppose that φ is equivalent to true. Thus there exist functions (f i ) i=1...n (expressing the choices for x 1 , . . . , x n ) such that for every set of assignments (a 1 , . . . , a n ) of the variables y 1 , . . . , y n the Boolean formula ψ[f 1 (), a 1 , f 2 (a 1 ), . . . , f n (a 1 , . . . , a n-1 ), a n ] is true. We choose a strategy σ such that for every possible set of assignments (a 1 , . . . , a n ) for the variables y 1 , . . . , y n , for all i ∈ {0, . .

The choice of the strategy is arbitrary in the other cases. Such a strategy can be defined since the formula ψ[f 1 (), a 1 , f 2 (a 1 ), . . . , f n (a 1 , . . . , a n-1 ), a n ] is true and thus every clause is satisfied by this choice of assignments.

With this strategy, the fixed-horizon disclosure in 2(n + m + 1) steps is 1 2 2n . In other words, all the runs reaching the secret disclose it. Indeed let ρ be a secret run of length 2(n + m + 1). There exists an assignment a 1 , . . . , a n ∈ {y 1 , ¬y 1 , . . . y n , ¬y n } such that O(ρ) = f 1 ()a 1 f 2 (a 1 ) . . . a n z 1 i 1 z 2 i 2 . . . z m im . By choice of σ, if, for z ∈ {x, y} and i ∈ {1, . . . n}, z i appears in the observation of ρ, ¬z i does not appear, and vice versa. Therefore as can be read either in s end or in a state reachable only by runs observing a Boolean variable and its negation, ρ discloses the secret.

Conversely, suppose that φ is not equivalent to true and let σ be a strategy, which can be assumed to be deterministic thanks to Proposition 7.3. We build partial functions f i : Σ 2i → Act consistent with σ: for every observation w ∈ Σ 2i of some run ρ, if σ chooses action a ∈ A(last(ρ)) for ρ (i.e., σ(ρ)(a) = 1) then f i (w) = a. As φ is not equivalent to true, there exists an assignment (a 1 , . . . , a n ) for the variables y 1 , . . . , y n such that the Boolean formula ψ[f 1 (), a 1 , f 2 (a 1 ), . . . , f n (a 1 , . . . , a n-1 ), a n ] is false. We now build a run with non null probability, reaching the secret but not disclosing it.

By construction, there exists ρ such that O(ρ) = f 1 ()a 1 f 2 (a 1 ) . . . f n (a 1 . . . a n-1 )a n , with last(ρ) = s and P σ (ρ) > 0 (where again P σ stands for P Mσ(µ 0 ) ). Let i ∈ N be an integer such that the clause z i 1 ∨ z i 2 ∨ z i 3 is not true under the assignment [f 1 (), a 1 , f 2 (a 1 ), . . . , f n (a 1 , . . . , a n-1 ), a n ], in other words such that the negations of z i 1 , z i 2 and z i 3 were chosen as assignment. Let ρ be the run of length 2(n + m) + 2 extending ρ and ending in s end . Then ρ does not disclose the secret: indeed, there exists j ∈ {1, 2, 3} such that z i j appears in its last 2m observations while its negation (written ¬z i j here) appears in the first 2n + 1 observations. Thus there exists a run with same observation leading to the second part of the box S ¬z i j which is outside the secret and where every observation is possible. As the total measure of runs reaching the secret is 1 2 2n and at least a subset of measure P σ (ρ) of the runs reaching the secret do not disclose it, the disclosure of M is strictly smaller than 1 2 2n in 2(n + m) + 3 observation steps.

The existence of an optimal strategy in the first part of the proof implies that the limit-sure and the almost-sure problem are equivalent. Moreover, the secret being revealed with probability 1 in a given number of steps, every run must reach the secret in this number of steps. Testing if there exists a strategy such that every run reaches a set of target states in a given number of steps in an OMDP can be solved in polynomial time.

Remark 7.1. The proof of hardness can be adapted for maximal fixed-horizon ε-disclosure, but the algorithm for membership cannot be directly applied. The ε-disclosure could however be computed by maximising an exponential system of equations, resulting in an exponential time algorithm.

Minimal disclosure

The proof of the next theorem is similar to the proof of Theorem 7.7 on the fixed-horizon maximal disclosure.

The hardness result is obtained once again using a reduction from the validity of a QBF. Many of the ideas used in the proof of Theorem 7.7 reappears here: the run is still composed of two parts, in the first one it gives an assignment to the Boolean variables and in the second one the strategies goes through the clauses of the formula and verify it can satisfy them. We however give the full proof due to non-negligible differences. Now the strategy must satisfy every clause of the formula in order for the run not to disclose the secret.

For the strategy decision problem, contrary to the maximisation case, due to the randomisation, there does not necessarily exists an optimal strategy. In order to get the same complexity for the strategy decision problem, we establish that when a randomised decision rule must be selected in the optimal strategy, it can always be uniformly distributed over its support.

Theorem 7.8. The fixed-horizon minimal value is computable in PSPACE. The fixedhorizon minimal disclosure problem is PSPACE-complete. In addition, the strategy decision problem is also decidable in PSPACE.

Proof. The procedures for the first two problems are very similar to the ones used in Theorem 7.7. There are only two differences. First, given B i the current belief and o i+1 one computes B i+1 = NextMax(B i , o i+1 ) (independently of the guessed actions a i,s ). Second, the computation procedure operates by decreasing values of i when the value is less or equal than i d n . In order to decide whether a strategy exists that provides the minimal value, one guesses this strategy in PSPACE as before. However there is an additional difficulty since the (possible) optimal strategy may be randomised. Thus during the procedure, given some belief B and some state s, one guesses the support A ⊆ A s of the decision rule and one defines the decision rule say δ as a uniform choice over A . We claim that this restriction is sound. Assume another decision rule δ with same support would provide Opacity for each 1 ≤ i ≤ n, or randomly as y i and ¬y i both have half a chance of being triggered. During the last 2m+1 steps, the strategy must choose a Boolean formula in every clause so that if a clause is not satisfied by the current assignment, then a Boolean variable is observed as both true and false during the run. The last step then triggers randomly the observation j for j ∈ {1, . . . 2n}.

Suppose that φ is equivalent to true. Then there exist functions (f i ) i=1...n such that for every set of assignments (a 1 , . . . , a n ) for the variables y 1 , . . . , y n the Boolean formula ψ[f 1 (), a 1 , f 2 (a 1 ), . . . , f n (a 1 , . . . , a n-1 ), a n )] is true. We choose a strategy σ such that for every possible set of assignments (a 1 , . . . , a n ) for the variables y 1 , . . . , y n , and for all i, 0 ≤ i ≤ n -1, σ( f 1 ()a 1 f 2 (a 1 ) . . . a i ) = f i+1 (a 1 , . . . , a i ). Moreover for k ∈ {1, . . . , m} and i 1 , . . . , i k ∈ {1, 2, 3}, there exists

. The choice of the strategy is arbitrary in the other cases. Since ψ[f 1 (), a 1 , f 2 (a 1 ), . . . , f n (a 1 , . . . , a n-1 ), a n )] is true, every clause is satisfied by this choice of assignments, hence it is possible to define such a strategy. With this strategy, the fixed-horizon disclosure in 2(n + m + 2) steps is 0. In other words, none of the runs reaching the secret discloses it. Indeed let ρ be a secret run of length 2(n + m + 2), then there exist a 1 , . . . , a n ∈ {y 1 , ¬y 1 , . . . y n , ¬y n } and j ∈ {1, . . . , 2n} such that O(ρ) = f 1 ()a 1 f 2 (a 1 ) . . . a n z 1 i 1 z 2 i 2 . . . z m im j. By choice of σ, if, for z ∈ {x, y} and i ∈ {1, . . . n}, z i appears in the observation of ρ, ¬z i does not, and vice versa. Therefore as j can be read either from s end or in a box state outside of the secret reachable only by runs that do not observe a Boolean variable and its negation, ρ does not disclose the secret.

Conversely, suppose that φ is not equivalent to true and let σ be an arbitrary strategy. We first build a deterministic strategy σ with smaller or equal disclosure. The first choice concerns {x 1 , ¬x 1 } and the next observation in a run corresponds to that choice. Consider σ 1 (resp. σ 1 ) the strategy that selects x 1 (resp. ¬x 1 ) and then plays like σ. Due to the fact that observations are distinct, the disclosure value w.r.t. σ is a convex combination of the ones of σ 1 and σ 1 . So one substitutes σ by the one with smaller or equal disclosure. A similar pattern applies for every choice until reaching the horizon. Thus by iterating this transformation we obtain a deterministic strategy. So we assume now that σ is deterministic. Since there is a finite number of such strategies for fixed horizon, it only remains to prove that the disclosure value under σ is positive. We build partial functions f i : Σ 2i → Act consistent with σ: for every observation w ∈ Σ 2i of some run ρ, if σ chooses action a ∈ A(last(ρ)) for ρ, then we set f i (w) = a. As φ is not equivalent to true, there exists an assignment (a 1 , . . . , a n ) for the variables y 1 , . . . , y n such that the Boolean formula ψ[f 1 (), a 1 , f 2 (a 1 ), . . . , f n (a 1 , . . . , a n-1 , a n )] is false.

We now build a run disclosing the secret. By construction, there exists ρ such that O(ρ) = f 1 ()a 1 f 2 (a 1 ) . . . f n (a 1 . . . a n-1 )a n , leading to last(ρ) = s with P σ (ρ) > 0. Let i ∈ {1, . . . , m} such that the negations of z i 1 , z i 2 and z i 3 were chosen as assignment hence z i 1 ∨z i 2 ∨z i 3 is not true under the assignment [f 1 (), a 1 , f 2 (a 1 ), . . . , f n (a 1 , . . . , a n-1 ), a n )]. Let ρ be a run of length 2(n + m) + 3 extending ρ and ending in s end . Then ρ does

Title : Controlling Information in Probabilistic Systems

Keywords : Model checking, probabilistic systems, diagnosis, Markov chains, partial observation Abstract : The control of the information given by a system has recently seen increasing importance due to the omnipresence of communicating systems, the need for privacy, etc. This control can be used in order to disclose an information of the system, or, oppositely, to hide one. Diagnosis for instance tries to determine from the observation produced by the system whether a fault occurred within it or not. In this PhD, we study the diagnosis of stochastic systems through a model-based approach. The goal is to establish the decidability and optimal complexity of the decision problems and to build the adequate diagnosers. We consider these problems in multiple frameworks (finite/infinite, passive/active).