Victory, after all, I suppose ! Well, it seems a very gloomy business.

-J.R.R. Tolkien "The Hobbit" J'ai pu lire quelque part qu'une thèse est un aquarium, et les remerciements des poissons rouges. J'aurais plutôt tendance à comparer la thèse à un marais, avec ses zones boueuses où l'on peut s'enfoncer, ses cours d'eau, certains vifs et décidés, d'autres réduits à de simples bras-morts que toute activité a abandonnée. Peut être aussi avec ses mangroves, se nourrissant avidement des courants marins qu'elles peuvent capter.

Logiquement, dans un tel environnement, les remerciements sont des piranhas. Ils se déplacent par bancs, sont typiques de l'image du milieu bien qu'ils n'en soient qu'une toute petite partie, et viennent se coller aux visiteurs pour les accueillir. Il paraît que cela a bon goût aussi. Certes, ce sont des saletés attirées par l'odeur du sang, mais bon dans une métaphore, on ne peut pas tout avoir ! Je tiens à remercier Thierry et Nicolas pour leur encadrement, et pour ne pas avoir sauté en marche lorsque j'ai proposé de défricher un vieux cours d'eau quelque peu effrayant loin des courants habituels. L'équipe SUMO en général pour son accueil et son ambiance, et notamment les permanents, Nathalie, Blaise, Loïc, Hervé, Ocan, Éric et Éric, pour leur aide, leurs points de vue et quelques très bonnes discussions scientifiques. En remerciant l'équipe je ne peux que laisser une place à part à Laurence et à son support si salutaire pour moi qui suit facilement submergé par des difficultés administratives (et d'autant plus émerveillé de les voir disparaître sans que le ciel ne me soit tombé sur la tête).

Merci aussi à Sophie et François, qui m'ont accompagné de bien des manières et sur le temps long, en réussissant souvent à trouver du temps qu'ils ne semblaient pas avoir. Pour m'avoir tous les deux trouvé une place dans d'enseignant dans leurs cours aussi, j'y ai beaucoup appris en patience (un peu) et en pédagogie (beaucoup).

Un grand merci aussi aux membres de mon jury de thèse, qui acceptent de prendre le temps de m'écouter, et particulièrement à Étienne et Frits, pour avoir lu ces pages et avoir proposé un bon nombre d'améliorations.

Je remercie bien sûr tous les non permanents du département que j'ai croisé, ainsi que les anciens de ma promo dont j'ai continué de partager le parcours. Particulièrement Emily, avec qui beaucoup de thé a été versé, et à qui je dois un merci particulier, pour avoir accepté de lire en avant-avant première les piranhas, ce qui mord un peu et gâche la surprise. Arthur, Tristan et Hugo également pour de nombreuses discussions (pas forcément toutes scientifiques). Victor et Choufi, pour la camaraderie. Mathias et Adrian, en particulier pour l'impressionnante énergie qu'ils réussirent à dispenser. J'ai aussi une pensée pour Nicolas, Lily et Camille, avec qui j'ai pu passer du bon temps, et échanger quelques bons mots. Clément enfin, pour beaucoup de bons moments et de nombreuses soirées de jeux permettant de sortir la tête de l'eau.

Ensuite je tiens à remercier ma famille, qui m'a beaucoup soutenu à travers cette thèse et les études précédentes, alors même que j'ai bien moins pu la voir pendant ce temps. En particulier mes parents, qui ont en plus fait l'effort méritoire de s'intéresser aux détails de ma thèse et de traverser quelques fois la France.

Enfin un grand merci à Solène, qui m'a supporté pendant tout ce temps, en première ligne et par tous temps.

RÉSUMÉ

Les ordinateurs, et plus généralement les systèmes automatiques, sont de plus en plus répandus à la fois dans notre vie courante (téléphones, logiciels de paie, jusque dans des choses aussi simples qu'une machine à café) et dans nos réalisations les plus complexes et avancées (fusées, médecine de pointe, analyse météorologique. . .). Les solutions aux problèmes de société, d'ingénierie ou scientifiques sont de plus en plus couramment recherchées avec l'aide de l'informatique, créant d'autant plus de défis pour les systèmes informatiques et au final, les informaticiens.

Conséquemment à cette situation une quantité impressionnante de systèmes liés à l'informatique existe, tant en terme de purs nombres que de taille ou de diversité. De par leurs interactions variées avec le monde, les systèmes informatiques acceptent des entrées de natures totalement différentes et s'attachent à des facteurs dissimilaires dans leurs calculs. Il peut s'agir de données quantitatives ou continues telles que le temps, le poids ou l'énergie, d'un environnement réactif, de probabilités, de changements de comportement brusques. . . Les résultats attendus sont eux aussi de natures variables, telle que des décisions opérationnelles, le résultat de fonctions mathématiques ou de l'affichage.

Exemple. Pensez à une entrée vidéo : une suite d'images se succédant rapidement dans le temps, couplée avec les informations de différents senseurs (vitesse, pression, température) et une voix qui doit être analysée depuis une piste audio et transformée en décisions et changements de comportement accompagnés d'une sortie vocale présentant les décisions.

Ceci peut parfaitement correspondre à l'interface d'une machine-outil, d'un robot, ou (plus à la mode) d'une voiture autonome. Afin de traiter ces situations et besoins variés, et à cause de la progression très active de la science informatique, les systèmes informatiques ont des principes et des structures extrêmement dissimilaires.

Remarque. Cette dissimilarité est reflétée (et peut être même magnifiée) dans la variété des domaines et communautés de recherche en informatique, dont les formations, la méthodologie et même les méthodes de falsifiabilité varient grandement.

Il faut ajouter à cela que les systèmes informatiques sont interconnectés par nature, créant des réseaux hautement plus complexes que la somme de leurs composants. Prédire le comportement d'un (composant d'un) système une fois interconnecté dans un tel environnement est donc devenu une tâche d'une épouvantable difficulté pour un humain seul. Cela met en lumière l'intérêt de méthodes et d'outils d'analyse automatique.

Pour certains de nos systèmes informatiques, une défaillance peut être relativement bénigne, et peut parfois être corrigée. Pensez par exemple à la corruption d'un (fragment d'un) pixel d'une image, ou à une machine à laver qui flanche. Néanmoins, pour un grand nombre de systèmes une erreur peut avoir un coût terrible, qu'il soit monétaire et affecte l'efficacité (un site internet ne répondant plus, la paralysie d'un réseau ferroviaire, des erreurs dans un logiciel de paie, un banque perdant des données. . .) ou en vies humaines (le le système de contrôle du frein d'un train n'arrivant pas à activer les freins, une machineoutil réalisant des mouvements imprévus en présence de travailleurs, la défaillance d'un pilote automatique sur un avion. . .). Prévenir des erreurs dans de tels systèmes est donc d'une importance cruciale, alors qu'ils sont souvent parmi les plus élaborés, ou immergés dans des environnements hautement complexes.

Exemple. Un exemple assez connu d'une telle erreur est l'échec du lancement de la première fusée Ariane 5. Cette fusée a explosé moins d'une minute après son décollage à cause d'une exception matérielle déclenchée par un dépassement provenant de la conversion de l'accélération horizontale d'entiers à virgule flottante 64 bits en entiers signés 16 bits.

Cette erreur, aussi simple qu'elle puisse paraître une fois découverte, a été causée par l'usage de connexions légèrement inadaptées entre des composants matériels autrement corrects et la décision de ne pas protéger la valeur contre les conversions pendant le processus de création, laquelle découlait d'hypothèses basées sur la fusée Ariane 4. Elle était donc difficile à repérer et put passer inaperçue à travers de multiples vérifications, simulations et phases de test.

Cet échec, heureusement non létal, causa la perte des quatre satellites transportés par la fusée, entraînant un déficit de plus de 370 millions de dollars états-uniens. Il devient connu comme l'erreur logicielle la plus coûteuse de l'Histoire (jusqu'alors) et joua un rôle important dans la reconnaissance et le développement des recherches assurant la fiabilité des composants critiques.

La conséquence de ces situations (dépendance toujours croissante envers les systèmes informatiques ; importante dissimilarité de buts et de moyens ; interconnection de systèmes ; existence de systèmes informatiques critiques pour la sécurité) est qu'un nombre toujours croissant de systèmes informatiques sont créés et connectés entre eux et avec des composants plus anciens. Une défaillance de certains de ces systèmes pourrait avoir de désastreuses conséquences. Dans le même temps, l'analyse et le diagnostic ou la vérification à la (seule) main est devenu irréaliste.

Un besoin vital de modèles

Comme expliqué précédemment, la plupart des systèmes sont trop complexes ou trop grands pour être analysés directement par des humains, particulièrement les systèmes dont la fiabilité est la plus cruciale. Cela est dû à des facteurs variés, notamment le grand nombre de lignes de codes ; les nombreux langages informatiques différents qui peuvent être utilisés ; la variété des applications et des expertises associées ; l'interconnexion avec des composants écrits dans des langages historiques, à peine utilisés mais toujours fonctionnels ; l'interconnexion avec des composants propriétaires qui ne peuvent pas être complètement analysés ; la dépendance au matériel. . . entre autres facteurs. Pour cette raison, l'analyse automatique des systèmes est nécessaire et son domaine d'application ne cesse de s'étendre. Les méthodes formelles, c'est-à-dire les techniques mathématiquement rigoureuses pour la spécification et la vérification, constituent l'un des principaux candidats à une telle analyse. L'utilisation de méthodes mathématiquement correctes et la possibilité d'obtenir des résultats prouvés permettent une grande confiance dans l'analyse effectuée.

Toutes ces méthodes nécessitent un modèle (pour des raisons d'efficacité et pour obtenir une représentation commune) et sont des exemples de l'action des modèles sur la réalité : elles utilisent des modèles pour raisonner sur un objet réel, agir sur eux (surveillance, test) et prendre des décisions à leur sujet (propriétés).

Pour cette raison, elles dépendent de l'existence d'un modèle adapté. Pourtant, les modèles formels de qualité, intégrant les bonnes informations et abstractions, sont rares dans la nature : il est donc utile de pouvoir créer des modèles à partir de systèmes réels.

Apprentissage : un levier pour obtenir des modèles

La plupart des méthodes mentionnées plus haut (notamment le contrôle, le test formel et la vérification) s'appuient sur un modèle mathématique du système et/ou des propriétés à vérifier.

Exprimer de tels systèmes et propriétés avec des modèles peut être difficile. Notamment, formaliser correctement une situation à la main nécessite un expert à la fois de la situation et du modèle, afin de prendre en compte les subtilités des deux. Même pour un tel expert, cela prend du temps et est sujet à de nombreuses erreurs Pire encore il est parfois totalement impossible ou prohibitivement coûteux d'obtenir un modèle. Cela peut être dû à des composants non-observables, par exemple dans une approche boite-noire où il est seulement possible d'interagir avec le système (ou certains composants) sans observer sa dynamique interne (cela peut arriver par exemple pour les systèmes propriétaires) ; à des réseaux constitués d'un grand nombre de composants différents (cas le modèle d'un réseau est souvent d'une taille bien supérieure à celle des modèles de ses composants) ; à des systèmes ne pouvant être arrêtés pour analyse.

Plus généralement, certains algorithmes peuvent ou doivent utiliser des informations déraisonnables à demander à un expert du domaine (par exemple les probabilités de différents comportements du système lors de son utilisation).

Ces problèmes peuvent être résolus en apprenant un modèle pour un système ou une propriété à partir d'exemples de ses comportements (et parfois de comportements défectueux). Apprendre automatiquement de tels modèles à partir d'observations externes est l'objet de l'apprentissage de modèles, un sous-domaine de l'apprentissage automatique.

L'apprentissage de modèles est un processus allant de la réalité vers les modèles : il acquiert des observations de systèmes réels et les synthétise à l'aide d'un formalisme cible.

Entre modèles et réalité

Dans les sections précédentes, nous avons présenté la vérification formelle (c'est-à-dire l'action de modèles sur la réalité) et l'apprentissage de modèles (c'est-à-dire l'action de la réalité sur les modèles). Ces interactions ont d'importants bénéfices : les modèles dirigent les actions sur la réalité, ce qui aide à viser des comportements spécifiques, dont les observations peuvent être utilisées pour raffiner ou construire les modèles.

Au coeur de cette opération se trouve l'abstraction d'évènements réels par des formalismes mathématiques. Cette abstraction sous-tend la définition des modèles et aide à organiser l'apprentissage en visant des comportements d'intérêt. Par exemple, cela permet de détecter quelles observations sont utiles, et lesquelles correspondent à des parties déjà explorées de l'abstraction. Les interactions entre modèles et réalité forment une spirale vertueuse. Par exemple, les tests et l'apprentissage sont essentiellement le même processus observé sous des angles différents : tous deux créent des entrées à proposer à un système et observent ses actions. Lors des tests, le modèle utilisé pour générer ces entrées est supposé correct et le système est « en cours de test ». En apprentissage, le système est la référence, et nous visons à construire un modèle le décrivant. En pratique, les premières phases de test sont souvent utilisées pour détecter les erreurs de modélisation et affiner le modèle, ce qui peut être qualifié d'apprentissage. Pour une discussion plus approfondie sur ce parallèle, voir [START_REF] Aichernig | Model Learning and Model-Based Testing[END_REF].

Modèle Réalité

La boucle d'interaction générale entre les modèles et la réalité est esquissée dans la figure 1.

Cette boucle, aussi bénéfique qu'elle soit, vient avec ses propres défis, qui surgissent des différences entre des modèles mathématiquement parfaits et la réalité, où les mesures sont floues et les systèmes durs à contrôler et observer. Cela nécessite des méthodes robustes et prenant en compte la contrôlabilité et l'observabilité limitées d'un système.

La robustesse n'est pas abordée dans cette thèse, bien que la section 2.2.3 donne quelques indications bibliographiques. Elle revient à prendre en compte des erreurs de mesure (par exemple temps d'action d'un capteur qui induit une erreur sur la mesure du temps ; imprécision d'une information GPS sur la localisation) et des imprécisions de contrôle (par exemple, en robotique : différence entre le mouvement planifié et l'effet des moteurs ; pour le temps : un activateur qui peut agir à tout moment dans un intervalle de temps au lieu d'un instant précis où la commande a été donnée).

L'observabilité peut être gérée au niveau des modèles, en transformant un modèle avec des actions non observables (par exemple, les communications internes entre les composants) en un équivalent où toutes les actions sont observables. Un autre élément central de l'observabilité est la distinguabilité : un modèle peut-il faire la distinction entre deux observations, ou font-elles partie du même comportement ? Des comportements indiscernables peuvent être utilisés pour détecter des erreurs de modélisation ou pour diminuer le nombre d'observations à générer.

Des problèmes de contrôlabilité surviennent dans les systèmes où un contrôleur ou un testeur ne peut pas toujours appliquer un comportement ciblé. Dans ce cas, le contrôleur doit s'appuyer sur le système lui-même pour atteindre son objectif, et les méthodes correspondantes doivent caractériser ce qui est en dehors du contrôle de l'agent et planifier en conséquence.

Comme déjà mentionné, les abstractions sont au coeur de la boucle entre les modèles et la réalité. Concrètement, différents niveaux d'abstractions sont nécessaires :

• des exécutions des modèles à leurs traces observées par un agent extérieur, des abstractions de différents niveaux d'observations sont nécessaires, ajoutant de plus en plus d'informations sur l'état interne.

• d'exécutions uniques aux ensembles d'exécutions définissant les modèles, en passant par des ensembles minimaux non distinguables, différents niveaux de précisions sont utilisés pour analyser ou construire un modèle.

Dans le contexte de cette thèse, ces abstractions sont les plus importantes lorsque l'on discute des méthodes d'apprentissage ou des méthodes d'approximation d'état, car elles s'intéressent aux abstractions inverses, c'est-à-dire à la déduction d'exécutions de modèles à partir d'observations et à celle d'ensembles (infinis) d'exécutions à partir de quelques exemples. Un soin particulier sera donc apporté à travers tous les chapitres de cette thèse à la définition de ces différentes abstractions et de leurs relations.

Ajouter le temps au mélange

Le temps est un aspect important d'un grand nombre d'applications telles que les transports, la fabrication, la sécurité. . . où les contraintes temporelles sur les actions font partie intégrante du comportement recherché.

Exemple. Considérons une simple machine à café, un objet banal sans doute critique pour le bon fonctionnement d'un laboratoire de recherche. Sa spécification (de haut niveau) pourrait être donnée par un automate simple, comme le montre la figure 2, avec des entrées données par l'utilisateur préfixées par " ?" et les sorties de la machine préfixées par " !". Selon cette spécification, tout utilisateur demandant un café, le payant et confirmant l'achat est sûr de finalement recevoir un café. L'absence de toute contrainte temporelle rend ce résultat peu attrayant (personne n'apprécierait attendre des minutes ou des heures pour un café sans aucune garantie de temps). Des contraintes temporelles sont nécessaires pour encoder le comportement désiré.

La prise en compte du temps apporte un épanouissement de nouveaux défis, notamment lorsque l'on considère que le temps est continu, ce qui est nécessaire pour obtenir une plus grande expressivité sur les contraintes temporelles. En particulier, le temps continu nécessite un nouveau modèle et de nouvelles abstractions, ainsi que des moyens de mettre en oeuvre de telles représentations : encoder une valeur continue dans un système numérique est impossible, et l'approximer peut être hasardeux.

Les automates temporisés se sont imposés comme l'un des principaux modèles pour le temps continu, principalement grâce à l'intéressant compromis qu'ils représentent entre efficacité et expressivité (ils sont parmi les modèles les plus expressifs dont l'accessibilité est décidable). Par conséquent, les travaux théoriques sur les automates temporisés sont à la fois étendus et applicables, en partie grâce aux multiples sous-classes d'automates temporisés qui ont été identifiées.

Exemple. La figure 3 représente un automate temporisé spécifiant la machine à café de la figure 2. L'ajout d'une horloge (notée c) mesurant le temps écoulé et remise à 0 (ce que l'on note {c}) par certaines transitions permet de spécifier des contraintes de temps (en secondes pour cet exemple). Notez que, par rapport à la version non chronométrée de la spécification, il est possible de spécifier des délais d'attente comme c'est le cas avec les transitions étiquetées par !annuler. Cette spécification garantit qu'un utilisateur reçoit un café en moins de 20 secondes après avoir confirmé son achat.

Dans cette thèse, nous nous concentrons sur des algorithmes et des structures de données prenant le temps en compte, avec les automates temporisés comme modèle de référence.

Contributions

Les contributions de cette thèse sont centrées sur les interactions entre modèles et réalité, dans le cas de systèmes temporisés modélisés avec des (variantes d')automates temporisés. Les principaux axes sont la contrôlabilité, l'observabilité et la distinguabilité des comportements. Un certain soin est également apporté à la disposition des différentes abstractions nécessaires à la poursuite des discussions techniques.

Le chapitre 1 présente les définitions centrales et les notions clefs qui sont ré-utilisées dans toutes les contributions. Le chapitre 2 détaille l'état de l'art des différents domaines de contribution. Les chapitres 3 à 5 détaillent les contributions.

Les principales contributions sont résumées ci-dessous :

• Le chapitre 3 présente nos travaux sur la contrôlabilité, appliqués aux automates temporisés de test de conformité (avec entrées et sorties). Il traite des jeux difficiles où aucune stratégie gagnante n'existe et discute d'un moyen de s'appuyer sur le système pour progresser -tout en minimisant cette dépendance.

Plus précisément, lors du test de systèmes complexes, il n'est pas rare de faire face à des systèmes qui ne peuvent pas être complètement contrôlés par l'agent. Pourtant, bien que les cas de test soient modélisés comme des stratégies sur des arènes de jeu, le système n'est pas l'adversaire du testeur : il ne combat pas activement les objectifs du test mais a simplement les siens, qui peuvent ou non correspondre à ceux du testeur. Dans ce cadre, il est logique de discuter de la dépendance vis-à-vis des comportements du système (selon une certaine forme d'équité) tout en quantifiant cette dépendance. Cela se fait via une notion de rangs caractérisant la distance à un objectif d'accessibilité en termes de pertes de contrôle et de stratégies de baisse de rang, qui utilisent ces rangs pour progresser vers l'objectif.

Ce travail a été présenté pour la première fois dans [START_REF] Henry | Control strategies for offline testing of timed systems[END_REF] et est présentement soumis sous forme d'article de revue. La section 3.6 étend l'approche du chapitre et discute de l'utilisation des stratégies introduites précédemment à une configuration très générale où le recours à l'équité du système n'est pas suffisant pour assurer la victoire. Il n'est pas encore soumis.

• Le chapitre 4 propose une structure généralisant les automates temporisés et permettant de suivre l'état du système après une observation donnée dans des modèles partiellement observables -un problème que nous montrerons être à la fois difficile et central pour de nombreuses méthodes formelles, notamment de diagnostic, de test formel et de surveillance. Le coeur de la structure est d'obtenir une représentation symbolique pour les ensembles des configurations du modèle, qui est à la fois finiment représentable et close par les opérations habituelles, c'est-à-dire les actions observables et laissant le temps s'écouler avec d'éventuelles transitions inobservables. Nous montrons que cela peut être réalisé pour des automates temporisés avec une seule horloge, et posons les bases théoriques pour obtenir des résultats similaires pour les automates temporisés généraux, bien que nous n'identifions pas la classe finiment représentable adéquat. Ce travail a été présenté dans [START_REF] Bouyer | Diagnosing timed automata using timed markings[END_REF] au cours de cette thèse, et d'abord développé dans [START_REF] Bouyer | Efficient timed diagnosis using automata with timed domains[END_REF] avant elle.

• Le chapitre 5 généralise une méthode d'apprentissage actif antérieure à une sousclasse beaucoup plus large d'automates temporisés qui présentent une sémantique inobservable de haute dimension. Pour cette raison, des suppositions sur les différentes formalisations possibles des observations doivent être formulées, et écartées si nécessaire, grâce à une nouvelle notion que nous appelons invalidité.

Plus précisément, nous étudions une sous-classe nouvellement introduite d'automates temporisés dont les remises à zéro des horloges ne sont pas liées aux observations. Il est donc nécessaire de deviner les remises à zéro de l'horloge. La notion d'invalidité caractérise les choix de remise à zéro des horloges qui ne peuvent pas expliquer l'ensemble actuel d'observations et peuvent donc être écartées en toute sécurité. Une version antérieure de ce travail a été présentée dans [START_REF] Henry | Active learning of timed automata with unobservable resets[END_REF] au cours de cette thèse, tandis que la version actuelle propose des algorithmes mis à jour et une discussion beaucoup plus approfondie sur l'invalidité.

INTRODUCTION

Misery me! I have heard songs of many battles, and I have always understood that defeat may be glorious. It seems very uncomfortable, not to say distressing. I wish I was well out of it.

-J.R.R. Tolkien "The Hobbit"

Computers and automated systems become ever more prevalent, both in our common life (smartphones, payroll softwares, even things as simple as coffee machines) and in our most complex and advanced realizations (rockets, state of the art medicine, weather analysis. . . ). New societal, engineering or scientific obstacles are approached more and more often with the help of computers, creating all the more challenges for computer systems, and ultimately, computer scientists. This situation leads to the existence of an impressive amount of computer related systems, both in term of sheer size, numbers and diversity. By interacting in disparate ways with the world, computer systems accept as input informations of completely different kinds, and take dissimilar factors into account in their computations. These can be quantitative or continuous data, such as time, weight or energy, a reactive environnement, probabilities, abrupt changes in behaviours. . . The expected outputs are themselves of varying nature, such as operative decisions, results of (mathematical) functions or displays (e.g. image or video).

Example. Think of a video input, which has images coming rapidly through time, coupled with different sensor informations (speed, pressure, temperature) and a voice that has to be analysed through an audio feed, and translated into decisions and changes in behaviour as well as an audio output of a voice presenting those decisions.

This can very well correspond to the interface of a machine-tool, a robot, or, in more trendy fashion, an autonomous car.

To handle these different situations and needs, and because of the active progression in computer science, computer systems are extremely dissimilar in design and structure.

Remark. This dissimilarity is reflected (and perhaps is easier seen) on the variety of the research domains and communities in computer science, which vary greatly in formalism, methodology, and even falsifiability.

Furthermore, by nature computer systems are interconnected and the resulting networks are highly more complex than their original components. To predict the behaviour of a (component of a) system once interconnected with such an environment has hence become a horrendous task for a human to approach directly. This emphasizes the interest of automated analysis methods and tools.

For some of the computer systems we use, failure is relatively benign, and sometimes can be corrected. Think of a corruption of (part of a) pixel of an image, or a washing machine breaking. Yet for a great number of systems, errors have a terrible cost, either monetary and in efficiency (a website going down, a paralysis of a train system, a payroll software making mistakes, a bank loosing data. . . ) or in human lives (control of a train break failing to activate the brake, machine tool making an unplanned movement around workers, auto-pilot failure on a plane).

Preventing errors in such systems is thus of the outmost importance, while they are often on the most elaborate end of the spectrum, or immersed in highly complex environments.

Example. A somewhat well known example of such error is the launch failure of the first Ariane 5 rocket. That rocket exploded after less than a minute of flight because of a hardware exception triggered by an overflow originating from the conversion of horizontal acceleration from 64-bit floating points integer to 16-bit signed ones.

That error, as trivial as it may seem once discovered, was due to the use of somewhat unadapted connections between otherwise valid hardware components and the decision not to protect that value conversion during the design process, which resulted from assumptions based on the previous Ariane 4 rocket. Hence, it was intricate to discover, and was able to pass unnoticed through multiple verifications, simulations and test phases.

That failure, fortunately not fatal, caused the waste of four satellites the rocket transported, resulting in a loss of more than US$370 millions. It became known as the most expensive software bug in history (so far) and played a great role in the recognition and development of research ensuring the reliability of safety-critical components.

The result of these situations-ever growing reliance on computer systems; great dissimilarity of purposes and means; interconnection of systems; existence of safety-critical computer systems-is that an ever increasing number of computer systems are created and connected between themselves and with older components. For some of these systems a failure would have dire consequences. Yet analyzing, diagnosing or checking these systems (only) by hand has become unrealistic.

A dire need for models

As explained previously, most systems are too complex and/or too large to be analysed by humans, especially those whose reliability is the most required. This is due to different factors, notably the sheer number of lines of codes; the numerous different languages that can be used; the variety of applications and associated expertises; the interconnection with components written in historic languages, barely used anymore but still running in practice; the interconnection with proprietary components that cannot be completely analysed; the dependence on hardware. . . between others. Because of this, automatic analysis of systems is necessary, and its application range ever increasing. One of the main contenders for such analysis is formal methods i.e. mathematically rigorous techniques for the specification and verification. The use of sound methods and the possibility to derive proved results allows for a great confidence in the performed analysis.

Due to their precision, formal methods can be both very complex for an expert to design and computationally expensive to run. Hence, it is necessary to rely on fitting abstractions to concentrate the expressivity on the relevant aspects of a system, while simplifying the task at hand. For critical systems, different kinds of complementary insurances are sought: formal verification leads to proofs of high-level properties, but relies on an often very abstract model; program analysis works directly on the code, which must be available and the properties derived are generally less complex and of different kind; testing cannot conclude to positive properties (e.g. a bug will not happen) but can interact with the final product (including the physical implementation and the environment); monitoring at run-time can rectify or stop faulty behaviours as they happen but is more limited in its options (e.g. sometimes has to stop the system, which can be a problem).

All these methods require a model of some sort (for efficiency and to obtain a common representation), and are examples of actions of the models on reality: they use models to reason upon real objects, act on them (monitoring, testing), and take decision about them (properties).

As such, they depend on the existence of a fitting model to be employed. Yet, good formal models integrating the correct informations and abstractions are rare in the wild: it is hence valuable to be able to create models from real life systems.

Learning: leveraging model availability

Most of the aforementioned approaches-notably control, formal testing and verificationrely on a mathematical model of the systems and/or properties to be verified.

Expressing such systems and properties with models can be difficult. Notably, correctly formalizing a situation by hand requires an expert in both the situation and the model, so as to take into account the subtleties of both. Even for such an expert, it is time consuming, and error prone.

Even worse, sometimes obtaining a model is completely impossible or prohibitively difficult. This can be due to unobservable components i.e. in a black-box approach where one can only interact with the system (or some components) without observing its internals (e.g. proprietary systems); networks constituted of a great number of different components (as model of the network is often of far greater size than the models of its components); system that cannot be taken offline for analysis.

More generally, some algorithms can or must use information that is unreasonable to ask from a domain expert (e.g. probabilities of different behaviours of the system in practice).

These problems can be solved by learning a model for a system or a property from examples of its behaviours (and sometimes of faulty behaviours). To automatically learn such models from external observations is the object of model learning, a sub-domain of machine learning.

Model learning is a process that typically goes from reality to the models: it acquires observations from real systems and synthesizes them using a targeted formalism.

Between models and reality

In the previous sections, we presented formal verification i.e. actions from the models upon reality and model learning i.e. actions from reality on models. These interactions have important benefits: models direct the action on reality, which in turns helps to target specific behaviours whose observations can be used to refine or build them.

At the core of this interaction is the mathematical abstraction of real events. That abstraction underlies the definition of models, and helps to organize the learning toward behaviours of interest. For example, it allows to detect which observations are useful, and which correspond to abstractions already explored. The interactions between models and reality form a virtuous spiral. For example, testing and learning are essentially the same process, except observed from a different perspective: both create inputs to propose to a system and observe its actions. In testing, the model used to generate these inputs is supposed correct, and the system is under test. In learning, the system is the reference, and we aim at constructing a model describing it. In practice, the first testing phases are often used to detect modeling errors and refine the model, which can be argued to be learning. For a more in depth discussion on this parallel, see [START_REF] Aichernig | Model Learning and Model-Based Testing[END_REF].

Model Reality

The general interaction loop between models and reality is sketched in Figure 4.

This loop, as beneficial as it is, comes with its own challenges, which arise from the differences between mathematically perfect models and reality, where measures are fuzzy, and systems hard to control and observe. This calls for methods that are robust, and take into account the limited controllability and observability of a system.

Robustness is not addressed in this thesis, although Section 2.2.3 gives some bibliographic pointers. It boils down to taking into account errors in measures (e.g. reaction-time of a captor that induces an error in the measure of time; imprecision of a GPS information on localisation) and imprecisions in the control (e.g. in robotics: difference between the planned movement and the effect of the motors; for time: an activator which can act at any point in an interval of time instead of a precise instant where the command was given).

Observability can be handled at the level of the models, by transforming a model with unobservable actions (e.g. internal communications between components) into an equivalent one where all actions are observable. Another central element of observability is distinguishability: i.e. can a model distinguish between two observations, or are they part of the same behaviour. Undistinguishable behaviours can be used to detect modeling errors or to diminish the number of observations to generate.

Controllability issues arise in systems where a controller or a tester cannot always enforce a targeted behaviour. In this case, the controller has to rely on the system itself to achieve its goal, and corresponding methods should characterize what is outside of the control of the agent, and plan around it.

As already mentioned, abstractions are at the core of the loop between models and reality. Specifically, different levels of abstractions are necessary:

• from model executions to their traces observed by an exterior agent, abstractions of different levels of observations are necessary, adding more and more information about the internal state.

• from single executions to sets of executions defining the models, passing by minimal undistinguishable sets, different levels of precisions are used to analyse or construct a model.

In the context of this thesis, these abstractions are most important when discussing learning methods or state approximation methods, as these are interested in inverse abstractions i.e. deducing model executions from observations and (infinite) sets of executions from few examples. A special care will thus be given through all chapters of this thesis to the definition of these different abstractions and their relations.

Adding time to the mix

Time is a significant aspect in a number of applications-such as e.g. transportation, manufacturing, security-where time constraints on actions are part of the intended behaviour.

Example. Consider a simple coffee machine, a mundane object arguably critical for the good functioning of a research laboratory. Its (high level) specification could be given through a simple automaton, as depicted in Figure 5, with inputs given by the user prefixed by "?" and outputs of the machine prefixed by "!". Following this specification, any user asking for a coffee, paying it and confirming the purchase is ensured to obtain a coffee eventually.

The lack of any timing constraint makes that result unappealing (no one would appreciate waiting minutes or hours for a coffee without any timing guarantees). Time constraints are necessary to encode the intended behaviour.

Taking time into account brings a flourish of new challenges, notably when considering that time is continuous, which is necessary to obtain a greater expressivity on time constraints. In particular, continuous time requires a new model and abstractions, as well as ways to implement such representations: encoding a continuous value in a digital system is impossible, and approximating it can be hazardous. Timed automata have imposed themselves as one of the main models for continuous time, mainly because of the interesting compromise they represent between tractability and expressivity (they are the among the most expressive models for which reachability is decidable). Hence, theoretical work on timed automata are both far reaching and applicable, thanks in part to the multiple sub-classes of timed automata that have been identified.

Example. Figure 6 represents a timed automaton specifying the coffee machine from Figure 5. The addition of a clock (noted c) measuring the time elapsing and reset to 0 (noted {c}) by some transitions allows to specify time constraints (in seconds for this example). Notice that, compared to the untimed version of the specification, it is possible to specify time-outs as done with the transitions labeled by !cancel.

This specification ensures that a user gets a coffee in less than 20 secondes after confirming its purchase.

In this thesis, we focus on algorithms and data structures that take time into account, with timed automata as a reference model.

Contributions

This thesis contributions are centered on the interactions between models and reality, in the case of timed systems modeled with (variants of) timed automata. The main focuses are on controllability, observability and distinguishability of behaviours. A certain care is also given to the lay out of different abstractions necessary to carry on the technical discussions.

Chapter 1 presents the central definitions and key notions that are re-used in all contributions. Chapter 2 details the state of the art of the different domains of the contributions. Chapter 3 to 5 detail the contributions.

The main contributions can be summarized in the following way:

• Chapter 3 presents our work on controllability, applied to conformance testing timed automata (with inputs and outputs). It deals with difficult games where no winning strategy exists and discusses a practical way to rely on the system for progress-while minimizing that reliance.

More precisely, when testing complex systems, it is not unusual to face systems that cannot be completely controlled by the agent. Yet, although test cases are modeled as strategies on game arenas, the system is not the tester adversary: it does not actively fight the test objectives but simply has its own, that may or may not align with the tester's. In this framework, it makes sense to discuss reliance on the system behaviours (according to some fairness) while quantifying that reliance. This is done via a notion of ranks characterizing the distance to a reachability objective in term of control losses and rank-lowering strategies, that use these ranks to progress toward the objective.

This work was first presented in [START_REF] Henry | Control strategies for offline testing of timed systems[END_REF], and is now submitted as a journal paper. Section 3.6 extends the approach of the chapter, and discusses the use of the strategies introduced previously to a very general setup where reliance on the fairness of the system is not enough to ensure victory. It is not yet submitted.

• Chapter 4 proposes a structure generalizing timed automata and allowing to track the state of the system after a given observation in partially observable models-a problem that we will show to be both difficult and central for numerous formal methods, notably diagnostic, formal testing and monitoring. The core of the structure is to obtain a symbolic representation for sets of the model configurations, that is both finitely representable and closed by the usual operations i.e. observable actions and letting time elapse with possible unobservable transitions. We show that this can be achieved for timed automata with one clock, and lay the theoretic bases to obtain similar results for general timed automata, although we do not identify the fitting finitely representable class. This work was presented in [START_REF] Bouyer | Diagnosing timed automata using timed markings[END_REF] during this thesis, and first developed in [START_REF] Bouyer | Efficient timed diagnosis using automata with timed domains[END_REF] before it.

• Chapter 5 generalizes a previous active learning method to a far broader subclass of timed automata that crucially displays a high-dimensional unobservable semantics.

Because of this, guesses on different possible formalizations of the observations have to be formulated, and ruled out when necessary, through a new notion that we call invalidity.

More specifically, we study reset-optional event recording automata, a newly introduced subclass of timed automata whose clock resets are not tied to observations. It is hence necessary to guess the clock resets. Invalidity characterizes clock resets that cannot explain the current set of observations and can thus be safely discarded. An earlier version of this work was presented in [START_REF] Henry | Active learning of timed automata with unobservable resets[END_REF] during the course of this thesis, while the current version proposes updated algorithms and a much more thorough discussion on invalidity.

Publications

The publications realized during this thesis are as follows: -J.R.R. Tolkien "The Hobbit"

Time representation

In the rest of this thesis, time is modelled using clocks, that measure the amount of time elapsed since an arbitrary origin, and are considered perfect (no error appears in the stored value, reading them is instantaneous and does not introduce errors) and infinitely precise. These clocks will later be reset by timed automata, as described in Section 1.2. Time is measured in arbitrary time units abstracting the level of precision required, that vary from application to application. The value of time units is an important parameter in the construction of models, as we will only compare time to integer numbers of those time units.

Example 1.1.1. If one considers a computer-based system, the time unit can be the computer tick, or any multiple of it. In an interactive system with sensors and activators, the time unit size can be dictated by the precision of those components. In general, determining the time unit is a compromise between precision, that allows more expressivity, and efficiency (most applications having an exponential time dependency on the maximal number of time units considered by a model).

Formally, given a finite set C of variables named clocks, a valuation of C is a function

c 1 c 2 c 3 c 4 {c 1 } {c 2 } {c 3 } {c 4 } v v(c 1 ) = 4 i=1 t i v(c 2 ) = 4 i=2 t i v(c 3 ) = 4 i=3 t i v(c 4 ) = t 4 t 1 t 2 t 3 t 4
A clock constraint is an expression of the form c 1 -c 2 ∼ n or c 1 ∼ n, for c 1 , c 2 ∈ C, ∼ ∈ {<, ≤, =, ≥, >} and n ∈ Z. A valuation v satisfies a constraint c 1 -c 2 ∼ n (resp. c ∼ n) if and only if v(c 1 )-v(c 2 ) ∼ n (resp. v(c) ∼ n).
We call zone over C [START_REF] Daws | Model checking of real-time reachability properties using abstractions[END_REF] any finite conjunction of such constraints, and write Z C (or sometimes simply Z) for the set of zones over C. Given a valuation v and a zone z, we write v |= z when v satisfies all the constraints in z.

Remark 1.1.4. As for valuations, a zone can be considered as a set of R |C| ≥0 defined by the valuations satisfying its constraints. This justifies notations such as v ∈ z. We will sometime identify a zone and its set of valuations when it eases the discussion, as done below.

Given a zone z, we define its future -→ z = v∈z -→ v and the effect of resetting a subset

C ⊆ C z [C ←0] = {v [C ←0] | v ∈ z}.
Notice that these are also zones.

We call guard over C any finite conjunction of clock constraints of the form c ∼ n, and write G C (or simply G) for the set of guards. As guards are zones, we extend every operation and definition made on zones to guards.

Example 1.1.5. In Figure 1.2 a representation of the valuations for the set of clocks

C = {c 1 , c 2 } is displayed.
The valuation 0 is displayed with the result of some operations:

v 1 = 0 + 0.7, v 2 = v 1[{c 2 }←0] and v 3 = v 2 + 0.5.

The leftmost highlighted polygon is the zone defined as

1 ≤ c 1 ≤ 2 ∧ -1 ≤ c 1 -c 2 ≤ 1 while the rightmost is the guard defined as c 1 ≥ 2. c 1 c 2 1 2 3 1 2 0 v 1 v 2 v 3

Timed automata

Now that a representation of time is defined with basic operations, automata handling this representation can be introduced.

Definition 1.2.1. [AD94] A timed automaton (TA) is a tuple

A = (L, init , Σ, C, I, E) such that:
• L is a finite set of locations, and init ∈ L is the initial location;

• Σ is a finite alphabet of discrete actions;

• C is its finite set of clocks;

• I : L → G C is a function associating an invariant to each location;

• E ⊆ L × G C × Σ × 2 C × L is a set of transitions. For a transition e = ( , g, a, C , ),
we call g = guard(e) its guard, a = act(e) its action, C = reset(e) its reset, = src(e) its source and = tgt(e) its target. We extend these notations to sequences of transitions with src((e i ) 1≤i≤n ) = src(e 1 ) tgt((e i ) 1≤i≤n ) = tgt(e n ) and act((e i ) 1≤i≤n ) = (act(e i )) 1≤i≤n .

In a timed automaton A we write K A (c) to denote the maximal constant of c ∈ C, i.e. the maximal constant to which c is compared in A.

Remark 1.2.2. Notice that a clock which is never tested can safely be removed from a timed automaton. If one wants to define the maximal constant for such a clock, a recurring convention is to use 0.

We call maximal constant of A, denoted K A , the maximum of the K A (c) for c ∈ C. A partial path π = (e i ) 1≤i≤n is a finite sequence of consecutive transitions (i.e. for all 1 ≤ i < n, src(e i+1 ) = tgt(e i )). A path is a partial path starting in init .

Example 1.2.3. A simple timed automaton is depicted in Figure 1.3. In this automaton, init

L = { init , 1 , 2 }, Σ = {a, b}, C = {c 1 , c 2 }, K(c 1 ) = 2 and K(c 2 ) = 3.
c 1 ≤ 3 start 1 c 2 ≤ 2 2 true e 1 = (true, a, ∅) e 2 = (c 1 ≤ 2, b, {c 1 }) e 3 = (c 1 ≤ 2 ∧ c 2 > 3, a, ∅) Figure 1.3: A simple timed automaton.
Because clocks take real values, and the availability of a transition depends on valuations (as the clock values have to satisfy the transition guard to enable it), the internal state (that we call configuration) of a timed automaton is the pair of its location and the current clock valuation. Thus, timed automata are a finite representation of infinite-state transition systems, that we use to express their semantics. Definition 1.2.4. With a TA A = (L, init , Σ, C, I, E), we associate the infinite-state transition system T A = (S, s init , Γ, →) where

• S = {( , v) ∈ L × R C ≥0 | v |= I( )}
is the set of configurations of the timed automaton and s init = ( init , 0) is the initial configuration;

• Γ = R ≥0 E is the set of transition labels;

• → ⊆ S × Γ × S is the transition relation defined as the union of the delay transitions, which contains all triples

(( , v), t, ( , v + t)) ∈ S × R ≥0 × S 1 ;
the action (or discrete) transitions, consisting of all triples

(( , v), ( , g, a, r, ), ( , v [r←0] )) ∈ S × E × S such that v |= g. We note enab(s) = {e ∈ E A | s e -
→} the set of transitions of the timed automaton enabled in a configuration s.

A partial run of A is a (finite or infinite) sequence ρ = ((s i-1 , p i , s i )) 1≤i<n of transitions in T A , with n ∈ N ∪ {+∞}. We write first(ρ) for s 0 and, when n ∈ N, last(ρ) for s n . A run is a partial run starting in the initial configuration s init . The duration of ρ is dur(ρ) = p i ∈R ≥0 p i . We write Ex(A) for the set of runs of A and pEx(A) the subset of partial runs.

A configuration s is said reachable from a configuration s when there exists a partial run starting in s and ending in s. We write Reach(A, S ) for the set of states that are reachable from some configuration in S , and Reach(A) for Reach(A, {s init }).

A (partial) run ρ is said to belong to a (partial) path π, denoted ρ ∈ π when its sequence of discrete transitions is π. A path is said feasible if and only if some run belongs to it. Remark 1.2.5. Notice that a run ρ = ((s i-1 , p i , s i )) 1≤i<n can also be represented as (s i-1 p i -→ s i ) 1≤i<n using the notation → defining the transitions of the semantics.

Example 1.2.6. Consider again the automaton of Figure1.3. A run of this automaton is

ρ = s init 1.5 -→ ( init , ( 1.5 1.5 )) e 1 -→ ( 1 , ( 1.5 1.5 )) e 2 -→ ( init , ( 0 1.5 )) e 1 -→ ( 1 , ( 0 1.5 )) 1 - → 1 - →( 1 , ( 2 3.5 )) e 3 -→ ( 2 , ( 2 3.5 )) .
As ρ ∈ π 2 = e 1 • e 2 • e 1 • e 3 this shows that π 2 is feasible. On the contrary, π 1 = e 1 • e 3 is unfeasible. Indeed, as a run starts from s init and e 1 does not reset c, it is impossible to satisfy the guard of e 3 after the first e 1 .

The (partial) signature associated with a (partial) run ρ = ((s i-1 , p i , s i )) 1≤i<n is sig(ρ) = (proj(p i )) 1≤i<n , where proj(p) = p if p ∈ R ≥0 , and proj(p) = (a, C ) if p = ( , g, a, C , ). We write pSig(A) = proj(pEx(A)) and Sig(A) = proj(Ex(A)) for the sets of (partial) signatures of A. We write s µ -→ s when there exists a (partial) finite run ρ such that µ = proj(ρ), first(ρ) = s and last(ρ) = s , and write dur(µ) = Σ γ i ∈R ≥0 γ i . Note that as expected, when sig(ρ) = µ then dur(ρ) = dur(µ). We write s µ -→ when s µ -→s for some s .

Notice that in the above definitions, runs and signatures could have several successive delay transitions, and several successive discrete transitions. This formulation is useful when a fine control on the progression of the system is necessary. In the context of this thesis, this formulation is used notably for game theory. It corresponds to a point of view from inside the model, when one wants to be able to "stop time" to take a decision before letting it elapse anew.

Yet, from an outside approach, only the total amount of time elapsed between two discrete actions is of interest. In this context, concerned with models for the languages it generates or reads, manipulating words that alternate strictly between delays and discrete events eases the discussion.

We say that a run (resp. signature) is alternating when it starts with a time elapse transition and alternates strictly between time elapses and discrete actions. Note that it is always possible to convert a non-alternating run / signature into an alternating one in a canonical way by summing consecutive delays and adding 0 delays between consecutive actions. In the following, we call alternating signatures ending by a discrete action (i.e. in (R ≥0 × Σ × 2 C ) * ) timed words with resets.

The trace of a (partial) signature corresponds to what can be observed by the environment, namely delays and discrete actions. The trace of a signature is the limit of the following inductive definition, for t, t ∈ R ≥0 , a, b ∈ Σ, C ⊆ C, and partial signatures µ 1 , µ 2 :

trace(ε) = 0 trace(t) = t trace((a, C )) = a trace(µ 1 • µ 2 ) = trace(µ 1 ) • trace(µ 2 )
with the concatenation of traces being defined with two special cases as follow:

t • t = t + t a • b = a • 0 • b .
Remark 1.2.7. This definition ensures that traces are always alternating, starting from a delay. The interest is to enforce a structure corresponding to an observation. Notably, the separations between consecutive delays in (non alternating) signatures can correspond to informations about the signature construction that should not be available by an outside observer.

We say that a (partial) trace ending with a discrete action is a timed word (notably the trace of a timed word with resets is a timed word). We denote timed words as elements of (R ≥0 × Σ) * .

We also define resets(µ) as the sequence of clock resets appearing in µ and define the operator ⊗ that from a timed word w t = ((t i .a i )) 1≤i<n and a reset sequence of same length r = (r i ) 1≤i<n builds the timed word with resets w ⊗ r = ((t i , a i , r i )) 1≤i<n .

Remark 1.2.8. The ⊗ operation could be defined for general signatures and traces, but we only use it for timed words, hence the use of this simpler definition.

The above definitions are summed up in the Figure 1.4, where the syntax and different levels of observation (semantics, signatures and traces) are highlighted.

Example 1.2.9. Considering again the run from Example 1.2.6:

ρ = s init 1.5 -→ ( init , ( 1.5 1.5 )) e 1 -→ ( 1 , ( 1.5 1.5 )) e 2 -→ ( init , ( 0 1.5 )) e 1 -→ ( 1 , ( 0 1.5 )) 1 - → 1 - →( 1 , ( 2 3.5 )) e 3 -→ ( 2 , ( 2 3.5 ))
one can construct an alternating run

ρ = s init 1.5 -→ ( init , ( 1.5 1.5 )) e 1 -→ 0 - →( 1 , ( 1.5 1.5 )) e 2 -→ 0 - →( init , ( 0 1.5 )) e 1 -→ ( 1 , ( 0 1.5 )) 2 - →( 1 , ( 2 3.5 )) e 3 -→ ( 2 , ( 2 3.5 )).
The signature of ρ is Notice that the timed word with resets µ constructed from µ by making it alternate is sig(ρ ).

µ = sig(ρ) = 1.5 • (a, ∅) • (b, {c 1 }) • (a, ∅) • (b, {c 1 }) • 1 • 1 • (a, ∅). TA: A = (L, init , Σ, C, I, E) Path: π = (e i ) 1≤i<n ∈ E * e i = ( i-1 , g i , a i , C i , i ) T A = (S, s init , Γ, →) (Partial) Run: ρ ∈ pEx(A) ⊆ → * ρ = (( i-1 , v i-1 ) p i -→ ( i , v i )) 1≤i<n (Partial) Signature: µ = (proj(p i )) 1≤i<n ∈ (R ≥0 ∪ (Σ × 2 C )) *

Timed word with resets

w tr = ((t i , a i , C i )) 1≤i<n ∈ (R ≥0 × Σ × 2 C ) * (Partial) Trace: w t ∈ (R ≥0 × Σ) * • (R ≥0 ∪ ε) Resets: r = (C i ) 1≤i<n ∈ (2 C ) * Timed word: w t = ((t i , a i )) 1≤i<n ∈ (R ≥0 × Σ) *

Observations

µ = 1.5 • (a, ∅) • 0 • (b, {c 1 }) • 0 • (a, ∅) • 2 • (a, ∅).
Finally, we can construct the trace w = trace(µ) = trace(µ ):

w = 1.5 • a • 0 • b • 0 • a • 2 • a.
For a run ρ, we write trace(ρ) = trace(sig(ρ)). In the same way, for a timed automaton A, we write Traces(A) = µ∈Sig(A) trace(µ) the set of traces corresponding to its runs and pTraces(A) the set of traces corresponding to partial runs. Note that all traces are alternating, as they are meant to encode the words that can be observed. Two timed automata are said trace equivalent if they have the same set of traces.

Remark 1.2.10. From the point of view of the language of timed words generated by a model, timed automata without invariants are as expressives as timed automata with invariants. Indeed, starting from a TA A, one can intersect all the the guards of the outgoing transitions of a given location with its invariant, i.e. replace g in ( , g, a, C , ) by g ∧ I( ), and suppress the invariants of the automaton. The only difference between the runs of A and the resulting automaton A is the appearance of greater final delays in A which do not appear in timed words (as they end with discrete actions).

While different from a control point of view as new configurations can be reached in

A these timed automata have the same language of timed words. For this reason, we will sometimes omit invariants entirely when focusing on the languages of timed words generated by timed automata.

We define the notion of determinism for timed automata.

Definition 1.2.11. A timed automaton A is said

• deterministic if for all e 1 = ( , g 1 , a, C 1 , 2 ) and e 2 = ( , g 2 , a, C 2 , 2 ) either e 1 = e 2 or g 1 ∩ g 2 = ∅;
• determinizable if there exists a trace-equivalent deterministic TA.

Remark 1.2.12. The notion of determinism defined above is syntactic. It entails semantic determinism: for all w ∈ T races(A), there is a unique ρ ∈ Ex(A) such that trace(ρ) = w. The converse is not true, as semantic determinism gives no constraints on unreachable valuations.

We chose the syntactic approach because it makes proofs easier when reasoning at the syntactic level and can be checked at a lesser cost as it only requires to compare transitions of the timed automaton, as opposed to its sets of runs and traces.

An important properties of timed automata is that they are not all determinizable. Even more, the determinizability of a timed automaton is an undecidable problem [START_REF] Finkel | Undecidable Problems About Timed Automata[END_REF][START_REF] Tripakis | Folk theorems on the determinization and minimization of timed automata[END_REF]. We write DTA for deterministic timed automata (and will write D-to emphasize that we restrict to deterministic models when manipulating other related classes).

Example 1.2.13. Figure 1.5 represents a non-deterministic, non-determinizable timed automaton.

Intuitively, this automaton requires that a pair of actions must be separated by exactly one time unit. A deterministic TA should thus keep the timing information of every action that happened less than one time unit ago, to be able to detect wether those actions happened exactly one time unit ago.

This cannot be done with a finite set of clocks, as unboundedly many actions could occur [START_REF] Alur | A Theory of Timed Automata[END_REF].

Equivalence relations

As explained in Section 1.2, the semantics T A of a timed automaton A is an infinite transition system. While it is possible to use such semantics for abstract reasoning, a finite representation is necessary, notably for implementation purposes.

Most of these representations are based on zones with states of the form ( , z) ∈ L × Z. We will often abuse the notation and write s ∈ ( , z) when s = ( , v) and v |= z. We first define minimal abstractions, based on equivalence relations on valuations: region equivalence and K-equivalence. Definition 1.3.1. For a TA A equipped with the set of clocks C, consider the following relation ∼ ∼ ∼reg [START_REF] Alur | A Theory of Timed Automata[END_REF] between valuations: v ∼ ∼ ∼reg v when the following conditions are met:

1. ∀c ∈ C, v(c) > K A (c) ⇔ v (c) > K A (c) ; 2. ∀c ∈ C, v(c) ≤ K A (c) ⇒ ( v(c) = v (c) ∧ ( v(c) = 0 ⇔ v (c) = 0)) ; 3. ∀c, c ∈ C, v(c) ≤ K A (c) ∧ v(c ) ≤ K A (c ) ⇒ ( v(c) ≤ v(c ) ⇔ v (c) ≤ v (c ) ) .
where • is the integer part operator and • the fractional part operator. ∼ ∼ ∼reg is an equivalence relation and we call regions its equivalence classes. We write R A for the set of regions of a timed automaton A.

The ∼ ∼ ∼reg relation is a time-abstract bisimulation relation i.e. if two valuations v, v are in a same region reg, there exists t ∈ R ≥0 such that v + t is in a region reg if and only if there exists t ∈ R ≥0 such that v + t ∈ reg . The name of time-abstract bisimulation comes from the fact that from v, one can simulate the behaviour of v and conversely (bisimulation), but this may require to choose different delays (time-abstract).

Definition 1.3.2. For a TA A equipped with the set of clocks C, consider the relation ≈ K between valuations defined by restricting to points 1. and 2. of Def. 1.3.1.

We call the equivalence classes of ≈ K K-closed zones and write Kz(v) for the one containing v. K-equivalence is coarser than region equivalence. Indeed, it only encodes direct distinguishability by a guard of A but is not a bisimulation relation. Notice that there is a finite number of regions and K-closed zones in any timed automaton thanks to the limitation by the automaton's maximal constants. In this chapter, an overview of the state of the art of some research domains related to this thesis topic is proposed. Most of the literature cited in this chapter is closely related to one or more of the thesis axes, but some subsections will be identified as general culture and are proposed for the reader's interest. They can safely be skipped for the purposes of the technical developments.

Timed models

Real-time behaviours often play an important role for modelling and specifying correctness of computer systems. Discrete models, such as finite-state automata, are not adequate to model real-time aspects of a behaviour or encode constraints on them. For this reason, different communities have proposed variants of their models taking time into account.

Timed automata and related models

A variety of different models exist to take continuous time into account. The most ancient state-machine including (continuous) time is the time Markov decision process that goes back to the 1960's [START_REF] Ronald | Dynamic programming and markov processes[END_REF][START_REF] Bruce | Finite state continuous time Markov decision processes with a finite planning horizon[END_REF]. In this model, the time elapsed before a selected action happens is sampled from a continuous random variable. There are however no constraints on timing embedded in the model. Timing constraints have been considered in two concomitant extensions of Petri nets to continuous time: Merlin's time Petri nets [START_REF] Meir | A Study of the Recoverability of Computing Systems[END_REF] and Ramchandani's timed Petri nets [START_REF] Ramchandani | Analysis of Asynchronous Concurrent Systems by Timed Petri Nets[END_REF]. Both models feature explicit time intervals that have to be respected by executions, but are otherwise not equivalent.

The study of continuous time system has boomed during the 1990's with the development of several models. By far the most expressive, hybrid dynamical systems, or hybrid systems for short [Alu+92; Nic+93; Alu+95; Hen96], combine discrete automata-like jumps between control states and continuous behaviours where quantities evolve over time as dictated by differential equations. As a side effect of hybrid systems expressivity, many related decision problems are undecidable. Notably reachability is in general undecidable for hybrid systems, even as "simple" as stopwatch automata where differential equations are of the form ẋ = 0 and ẋ = 1, although decidable subclasses of hybrid systems are identified [START_REF] Henzinger | What's Decidable about Hybrid Automata?[END_REF].

Timed automata have been introduced in 1994 by Alur and Dill [START_REF] Alur | A Theory of Timed Automata[END_REF] 1 . Less expressive than hybrid systems, their reachability problem is decidable (although PSPACEcomplete) [START_REF] Alur | A Theory of Timed Automata[END_REF]. Yet they remain highly complex models in general. For example, not all timed automata are determinizable and determinizability is undecidable [START_REF] Finkel | Undecidable Problems About Timed Automata[END_REF][START_REF] Tripakis | Folk theorems on the determinization and minimization of timed automata[END_REF]; the universality problem (Are all words accepted by a given model?) and the language inclusion problem are decidable only for deterministic timed automata.

For this reason, a number of subclasses have been studied over the years, by reducing the number of clocks, or linking clock resets to discrete actions as in e.g. event recording automata [START_REF] Alur | Event-Clock Automata: A Determinizable Class of Timed Automata[END_REF], a subclass first introduced because it is determinizable. This subclass (and an extension of it proposed during the course of this thesis) will be further discussed and introduced in Chapter 5. The number of subclasses is one of the appeals of timed automata: any general algorithm can be declined for a great number of less complex use cases.

Extensions of timed automata also exist. Some add new considerations to timed automata, such as timed automata with energy that add a constraint in the form of a positive "energy" variable that has to be maintained above 0 through a run [START_REF] Bouyer | Infinite Runs in Weighted Timed Automata with Energy Constraints[END_REF][START_REF] Bacci | Optimal and Robust Controller Synthesis Using Energy Timed Automata with Uncertainty[END_REF], timed game automata [Asa+98; Cas+05] that add controllability aspects in the form of a (2-player) game or timed automata with inputs and outputs [START_REF] Krichen | Conformance testing for real-time systems[END_REF] (this last will be presented in context in Section 2.2 and further discussed in Chapter 3).

Others are simpler variants on the definition, such as adding invariants (as presented Section 1.2), first introduced in Timed Safety Automata [START_REF] Henzinger | Symbolic model checking for real-time systems[END_REF], a model slightly different from modern timed automata (based on [START_REF] Alur | A Theory of Timed Automata[END_REF]) that deviated from the first timed automaton [START_REF] Alur | Model-checking for real-time systems[END_REF]. Another classic variant authorizes diagonal constraints i.e. constraints of the form c 1 -c 2 ∼ n with c 1 , c 2 ∈ C, n ∈ Z and ∼ ∈ {<, ≤, =, ≥, >}. Diagonal constraints have been first discussed in [START_REF] Alur | A Theory of Timed Automata[END_REF]. It is known that a timed automaton with diagonal constraints can be converted into a classic timed automaton (i.e. without diagonal constraints), although at the cost of an exponential blow-up in locations, depending on the number of diagonal constraints [START_REF] Bérard | Characterization of the Expressive Power of Silent Transitions in Timed Automata[END_REF]. Interestingly, the usual reachability algorithm for TAs, which is zone based, has been found to be faulty in the case of timed automata with diagonal constraints [Bou03; BY03] and a modified algorithm has been proposed [START_REF] Bouyer | Diagonal Constraints in Timed Automata: Forward Analysis of Timed Systems[END_REF].

More recently, some models including continuous time have been built upon Mealy machines, such as time delay Mealy machines [START_REF] Caldwell | Learning Time Delay Mealy Machines From Programmable Logic Controllers[END_REF] and their generalization Mealy machines with a single timer [START_REF] Frits | Learning Mealy Machines with One Timer[END_REF]. Time delay Mealy machines propose to use a single countdown timer, forcing an action to be taken instantly when it reaches 0. The model dictates that the timer is reset at each transition, which limits its expressivity. Mealy machines with a single timer remove that reset constraint.

In the following of this thesis, we will focus our technical developments on timed automata, although related approaches based on other timed models are often part of the relevant literature.

Behaviour abstraction

In the first introduction of timed automata, reachability of a given configuration has been proven to be PSPACE-complete by the mean of the region graph associated with a timed automaton, that is a finite (untimed) automaton whose states are pairs of locations and clock regions.

Despite this challenging worst case, a great amount of work has been put into more efficient algorithms to solve reachability and other related problems for timed automata.

Most of these algorithm rely primarily on zones (presented in Section 1.1) and their implementations with Difference-Bound Matrices (DBMs), defined in [START_REF] Berthomieu | An Enumerative Approach For Analyzing Time Petri Nets[END_REF] with an application for Time Petri nets and introduced to timed automata by [START_REF] David | Timing assumptions and verification of finite-state concurrent systems[END_REF].

Difference-bound matrices for zones upon the set of clocks C = {c 1 , . . . , c n } are (n + 1) × (n + 1) matrices such that the coefficient m i,j of a matrix at the i-th line and j-th column is a pair (c, ≺) with n ∈ Z and ≺ ∈ {<, ≤}. It is interpreted as the constraint c i-1 -c j-1 ≺ n with a special clock c 0 that is always evaluated to 0, serving to encode guard constraints (i.e. c ∼ n). Difference-bound matrices come with operations corresponding to the usual operations on zones and a canonical form uniquely representing zones, see e.g. [START_REF] Bengtsson | Timed Automata: Semantics, Algorithms and Tools[END_REF][START_REF] Sankur | Robustness in Timed Automata: Analysis, Synthesis, Implementation[END_REF].

Zones and DBMs, as well as other finite representations (for a discussion and related algorithms, see e.g. [START_REF] Roussanaly | Efficient Verification of real-time systems[END_REF]), are the basis of several scientific and industrial tools for timed automata, which have adopted always more algorithms to gain efficiency in practice. Amongst the best-known tools handling timed automata are: Kronos [START_REF] Daws | The Tool KRONOS[END_REF][START_REF] Bozga | Kronos: a model-checking tool for real-time systems[END_REF] the historic first tool implementing verification of timed automata and a forward reachability algorithm, now outclassed by successors 2 ;

Uppaal [START_REF] Behrmann | Uppaal 4.0[END_REF] is an academic and industrial tool building upon timed automata and expending the model in various ways (e.g. with game semantics, different type of actions, notions of urgency or parameters). Sadly, the software code is kept by its designers, which has complicated its interaction with the research community Because of the predominance of those representations, most algorithms introduced for timed automata (especially practical algorithms) manipulate a zone-based representation of the automaton behaviours.

Model verification

Offline verification. Because of the wide range of applications of computer systemsnotably their use in numerous safety critical systems-and of their increasing complexity, the use of formal methods for checking their correct behaviours has become essential [START_REF] Henzinger | The Embedded Systems Design Challenge[END_REF][START_REF] Clarke | Model checking: algorithmic verification and debugging[END_REF]. It is even one of the main motivations (if not the main) for the introduction of formal models. Numerous approaches have been introduced and extensively studied over the last 40 years, and mature tools now exist and are used in practice, as those mentioned in Section 2.1.2 for timed automata, see e.g. [START_REF] Bengtsson | Timed Automata: Semantics, Algorithms and Tools[END_REF][START_REF] Behrmann | Uppaal 4.0[END_REF].

Most of these approaches rely on building mathematical models, such as automata and extensions thereof, in order to represent and reason about the behaviours of those systems; various algorithmic techniques are then applied in order to ensure correctness of those behaviours, such as model checking [START_REF] Clarke | Model checking[END_REF][START_REF] Clarke | Handbook of Model Checking[END_REF] or deductive verification [START_REF] Antony | An axiomatic basis for computer programming[END_REF][START_REF] Filliâtre | Deductive software verification[END_REF].

These techniques focus on assessing properties of a model in an offline manner i.e. reasoning on the system model, without executing the system itself. With powerful properties such as the correctness of all of the system behaviours, this can lead to very costly algorithms, in terms of computation and memory. This is sometimes a too demanding requirement, especially with highly expressive timed models. Furthermore, it is sometimes desirable to not only check the correctness of a model of the system but to verify properties of its implementation in its environment, in which bugs could appear independently of the model correctness e.g. due to implementations errors (software) or faulty components (hardware), or interactions between them.

Online verification.

To leverage this issue, a set of online (or runtime) verification approaches have been crafted, that interact with the running system to assess its properties [START_REF] Leucker | A brief account of runtime verification[END_REF].

Fault diagnosis is a prominent problem in runtime verification: it consists in monitoring real executions of a partially-observable system, and detect at runtime, as early as possible, whether some property holds (e.g. whether some unobservable fault has occurred) 8 [START_REF] Sampath | Failure diagnosis using discrete-event models[END_REF]. This is usually done by synthesizing a diagnoser i.e. a formal model used to monitor the executions w.r.t. the properties of interest. For finite-state models, a diagnoser can usually be built by determinizing a model of the system, using the powerset construction; it will keep track of all possible states that can be reached after each (observable) step of the system, thereby computing whether a fault may or must have occurred. The related problem of prediction, a.k.a. prognosis, (that e.g. no faults may occur in the near future) [START_REF] Genc | Predictability of event occurrences in partially-observed discrete-event systems[END_REF], is also of particular interest in runtime verification, and can be solved using similar techniques [START_REF] Thierry Jéron | Predictability of Sequence Patterns in Discrete Event Systems[END_REF].

Conformance testing is another online verification technique whose aim is to check whether the behaviours of a (black-box) running system conform to its specification. To do that a tester runs test cases which apply inputs to the system and observe that outputs conform to the specification. In its model-based declination [START_REF] Tretmans | Conformance Testing with Labelled Transition Systems: Implementation Relations and Test Generation[END_REF], the specification is formal, e.g. an automaton, and since the system and specification are partially observable, the generation of tests (online or offline) may also rely on determinization to know the effect of any given action of the system in order to control it.

The following subsections discuss more in depth the model-based testing of timed systems and focus on the state estimation problem underlying determinization.

Model-based testing

Real-time reactive systems are open systems interacting with their environment and subject to timing constraints. Such systems are encountered in many contexts, in particular in critical applications such as transportation, control of manufacturing systems, etc. Their correctness is then of prime importance, but is also very challenging due to multiple factors: combination of discrete and continuous behaviours, concurrency aspects in distributed systems, partial observability and limited controllability of open systems.

To assess the correctness of such systems, testing remains the most-used validation technique, with variations depending on the design phase. Conformance testing is one of those variations, consisting in checking whether a real system correctly implements its specification, which serves as a reference. Those real systems, also called implementations under test are considered as black boxes, thereby offering only partial observability to the tester, for various reasons (e.g. because sensors cannot observe all actions, or because the 8. Although the termed "diagnosis" is coined by the community, fault detection may be more accurate, as suggested by S. Tripakis [START_REF] Tripakis | Fault Diagnosis for Timed Automata[END_REF], as it is not concerned with the root cause of a failure.

system is composed of communicating components whose communications cannot all be observed, or because of intellectual property of peer software). Controllability is another well-known issue when the system makes its own choices upon which the environment, and thus the tester, has a limited control. This is the main focus of the ideas developed in Chapter 3 around strategies to control test cases.

One of the most challenging activities of conformance testing is the design of test cases that, when executed on the real system, produce meaningful verdicts about the conformance of the system at hand with respect to its specification. Formal models and methods are good candidates to help test-case synthesis, both in terms of productivity gain and increased confidence in their verdict [START_REF] Tretmans | Conformance Testing with Labelled Transition Systems: Implementation Relations and Test Generation[END_REF]. Observability and controllability problems are central issues to overcome in this task.

In formal testing, it is adequate to refine the TA model by explicitly distinguishing (controllable) inputs and (uncontrollable) outputs, giving rise to the model of TAIOs (Timed Automata with Inputs and Outputs) [START_REF] Krichen | Conformance testing for real-time systems[END_REF]. Other variations of the TA model have been equipped with inputs and outputs and used in the context of testing, such as TIOA [SVD01; Kay+03], or TIOTS [START_REF] Kim Guldstrand Larsen | Online Testing of Real-time Systems Using Uppaal[END_REF].

Test-case synthesis from timed automata. Test-case synthesis from TAs has been extensively studied during the last 20 years (see [CG98; CKL98; SVD01; EDK02; NS03; BB04; LMN04; KT09; Ber+12], to cite a few). When testing offline, the test cases are first computed, stored, and later executed on the implementation. They should thus anticipate all specified outputs after an observed trace. But then, one of the difficulties comes from partial observation. In the untimed framework, this is tackled by determinizing the specification. Unfortunately, this is not always feasible for TAIO specifications since determinization is not possible in general. The solution is then either to perform online testing, where a subset construction is made on the current execution trace [START_REF] Kim Guldstrand Larsen | Online Testing of Real-time Systems Using Uppaal[END_REF], or to restrict to determinizable sub-classes (see e.g. [START_REF] Nielsen | Automated test generation from timed automata[END_REF] for ERAs). While the latter approach limits the class of models that can be used, the former suffers from the multiplication of possible configurations after a given trace.

Partial observation is addressed in [START_REF] David | Testing Real-Time Systems under Uncertainty[END_REF] with a variant of the TA model where observations are described by observation predicates, composed of a set of locations together with clock constraints. Test cases are then synthesized as winning strategies, if they exist, in a game between the specification and its environment that tries to guide the system to satisfy the test purpose. More recently, some advances were obtained in [START_REF] Bertrand | Offline test selection with test purposes for non-deterministic timed automata[END_REF] by the use of an approximate determinization procedure using a game approach [START_REF] Bertrand | A game approach to determinize timed automata[END_REF] that preserves tioco conformance and is exact for most of the known determinizable sub-classes of TAs when sufficient resources (i.e. number of clocks and maximal constant) are provided. Controllability of a system under test. The problem of testing is often informally presented as a game between the environment and the system under test (see e.g. [START_REF] Yannakakis | Testing, Optimization, and Games[END_REF]). But in reality very few papers effectively take into account the controllability of the system in this game.

In the context of timed testing, a game approach has been studied in [START_REF] David | A Game-Theoretic Approach to Real-Time System Testing[END_REF], where test cases are synthesized as winning strategies of a reachability game. But this work is restricted to deterministic models, and controllability problems are not really mastered. In fact, like in [START_REF] David | Testing Real-Time Systems under Uncertainty[END_REF], when no winning strategies are found, the game is abandoned and it is suggested to modify the test purpose. This is mitigated in [START_REF] David | Cooperative Testing of Timed Systems[END_REF] with the use of cooperative strategies, which rely on the cooperation of the system under test to win the game. The strategy can furthermore distinguish between cooperating states, where it needs to rely on the system to win, and winning states, where it can win despite the system best effort.

A more in-depth approach to the controllability problem is the one of [START_REF] Ramangalahy | Strategies for comformance testing[END_REF] in the untimed setting, unfortunately a scarcely-known work. Test selection is modelled as a game where the tester tries to satisfy a test purpose while detecting non-conformance, but faces control losses, i.e., states where the system proposes uncontrollable but correct outputs that moves it away from its objective. The computed strategies could quantify on the number of control losses. Interestingly, this allows to minimize both the reliance on the system decisions and the distance to the next control loss.

Conformance testing of timed automata is discussed in the Chapter 3 of this thesis. Notice that Uppaal has an extension named TIGA (TImed GAmes) that implements timed games algorithms [START_REF] Cassez | Efficient On-the-fly Algorithms for the Analysis of Timed Games[END_REF].

State estimation

As mentioned before, efficient offline algorithms for the analysis of timed automata have been proposed and implemented. Despite these successes, the state estimation problem remains one of the most challenging limitations that all methods have to encounter. This problem is concerned with the detection of the possible configurations of a system after a given trace. For offline methods, it takes the form of determinization and closure with respect to silent transitions. Sadly, for timed automata, determinizability is undecidable [START_REF] Finkel | Undecidable Problems About Timed Automata[END_REF][START_REF] Tripakis | Folk theorems on the determinization and minimization of timed automata[END_REF] and silent transitions strictly increase expressivity [START_REF] Bérard | On the power of nonobservable actions in timed automata[END_REF].

For online methods, one can try to efficiently keep tracks of the possible configurations [START_REF] Grez | The monitoring problem for timed automata[END_REF]. For example, Baier et al. have proposed a method to construct a deterministic timed tree from a timed automaton, with the limitation that the tree may be infinite [START_REF] Baier | When Are Timed Automata Determinizable?[END_REF]. This method has been declined for diagnosis [START_REF] Tripakis | Fault Diagnosis for Timed Automata[END_REF], and implemented in UPPAAL TRON [START_REF] Hessel | Testing Real-Time Systems Using UPPAAL[END_REF] (one of UPPAAL's libraries) and IF TTG [START_REF] Krichen | Conformance testing for real-time systems[END_REF] for online testing of timed input-output systems. This approach is computationally very expensive, as one step consists in maintaining the set of all configurations that can be reached by following (arbitrarily long) sequences of unobservable transitions; moreover, the set of possible configurations is updated only when a new event is observed (or after a timeout), which may significantly delay the detection of a fault.

Another possibility is to impose some restrictions on the constructed model. Bouyer, Chevalier and D'Souza studied a restricted setting, only looking for diagnosers under the form of deterministic timed automata with limited resources [START_REF] Bouyer | Fault Diagnosis Using Timed Automata[END_REF]. Similarly, Krichen et al. have proposed to build a deterministic timed automaton from a non-deterministic one, by fixing a set of clocks and possibly making some approximations in the behaviour [START_REF] Krichen | Conformance testing for real-time systems[END_REF]. Bertrand et al. later generalized and improved this approach using games [START_REF] Bertrand | A game approach to determinize timed automata[END_REF]. Both works have then been used for offline test generation from timed automata models [KT09; Ber+12; HJM18].

Finally, automata over timed domains [START_REF] Bouyer | On the determinization of timed systems[END_REF], a larger determinizable class of models comprising timed automata has been created to offer a determinization procedure, but with a great expressivity-and hence complexity.

Robustness(es) of models

This subsection is for scientific culture, as no contribution is made on robustness on this thesis. We include it to acknowledge the importance of robustness in model-reality interactions and provide (a few) bibliographic pointers on that matter.

The robustness of a model is generally understood as its ability to maintain good properties under small perturbations. This take is obviously vague, notably depending on what one consider a "good property" and a perturbation-which can both vary. In the case of timed models, an important emphasis is given to timing errors that can arise from imprecise measure of time, unexpected runtime behaviours or imprecisions in actuator controls.

Precise definitions and modeling vary. Robust timed automata [START_REF] Gupta | Robust Timed Automata[END_REF] first proposed a topological semantics for timed automata and consider that a run is only accepted if an open tube around it is accepted too (i.e. it excludes isolated behaviours). Timed automata with clock drifts were also considered [START_REF] Puri | Dynamical Properties of Timed Automata[END_REF]. A classic syntactic approach is to consider automata with enlarged or shrinked guards by a small 0 < < 1 [De +04]. A survey of this approach and related works and results for the verification, implementation and monitoring of such system can be found in [START_REF] Bouyer | Robustness in timed automata[END_REF].

Game based approaches also exist, that try to propose optimally robust strategies for reachability [START_REF] Clement | Computing Maximally-Permissive Strategies in Acyclic Timed Automata[END_REF].

Model learning

Machine learning is the automatic improvement of agents through experience and interactions with an environment, which can consist of provided data or be an open word. It is one of the core component of the so-called artificial intelligence domain that tries to design machines that can mimic (or approximate) cognitive functions associated with the human or animal mind, typically learning and problem solving.

Concept learning is a sub-domain of machine learning interested in the automatic inference and refinement of high-level concepts from the environment. In the context of interactions with formal methods, it is interesting that the representation of those concepts takes the form of formal models (state machines, logic formulae. . . ). This specific branch of research is called model learning.

In this section we first propose a quick discussion about the various passive model learning methods in existence (Section 2.3.1), then a presentation of active learning, the learning method most relevant to this thesis (Section 2.3.2).

Quick dive in passive model learning

This subsection is mostly general culture, as it presents roots of model learning and some research efforts that differ greatly from the technical developments of this thesis which are based on active learning.

One of the theoretical backbones of model learning is the work of Anil Nerode on the so called Nerode congruence [START_REF] Nerode | Linear automaton transformations[END_REF], an equivalence relation between states of a deterministic finite automaton allowing to define the minimal deterministic automaton recognizing a given language.

The first historical approach to model learning is a passive learning approach, where the agent learns from predetermined set of examples, either positive (i.e. part of the language to be learned) or negative (i.e. out of the language to be learned).

It is long known that learning from positive examples only is far less expressive. For instance, primitive recursive languages can be identified in the limit-roughly, after an unbounded but finite number of examples-by passive learning, but only finite languages can be identified from positive examples only [START_REF] Gold | Language Identification in the Limit[END_REF]. Notably, negative examples are necessary to learn the class of regular languages. Gold then investigated the learning from a finite set of given data, and found that DFA learning 9 is NP-hard in this framework [START_REF] Gold | Complexity of Automaton Identification from Given Data[END_REF]. Angluin gave a characterization of the learnability from text [START_REF] Angluin | Inductive Inference of Formal Languages from Positive Data[END_REF] and investigated some ways to extend the class of learnable languages. This research effort lead to active learning (see Section 2.3.2) and showed that relaxing the learning condition to learning with probability one enhances that expressivity [START_REF] Angluin | Identifying languages from stochastic examples[END_REF].

Learning of cyber-physical systems. Model learning was in part motivated by the need to obtain models to test cyber-physical systems. Obtaining negative examples of a cyber-physical system behaviours is costly, if not outright infeasible (as it corresponds to security issues or require to break the system). Due to the restriction on learning from positive observations only, researchers have sought passive learning methods for probabilistic automata from positive examples only.

One of the best known methods is ALERGIA [START_REF] Rafael | Learning stochastic regular grammars by means of a state merging method[END_REF]. This algorithm first constructs a tree representing the available examples and then merge its states when their sub-trees have similar languages (i.e. the same language with a high probability). This approach has been developed with a global criterion for merging in the algorithm MDI (Minimal Divergence Inference) [START_REF] Thollard | Probabilistic DFA Inference using Kullback-Leibler Divergence and Minimality[END_REF] that allows to obtain a bound on the (Kullback-Leibler) divergence between the samples and the model. A state-merging method based on Kullback-Leibler divergence is also proposed in [START_REF] Clark | PAC-learnability of Probabilistic Deterministic Finite State Automata[END_REF]. The article is more theoretical and does not provide an implementation, but gives an in-depth review of their complexity results. It notable proves that probabilistic deterministic automata are Probably Approximately Correct (PAC)-learnable [START_REF] Valiant | A Theory of the Learnable[END_REF], PAC-learning being an important theoretical framework for learning, that links together the probability of error, the precision of the model learned 9. and minimal DFA learning and the number of required data.

When a model integrating time is required, it is necessary to add a splitting operator to distinguish between different behaviours depending on time. The BUTLA (Bottom Up Timing Learning Algorithm) [START_REF] Maier | Anomaly Detection in Production Plants using Timed Automata -Automated Learning of Models from Observations[END_REF] is a first work on this topic, learning a probabilistic timed automaton. The authors rely on domain specific knowledge to advocate a split performed to separate modes in the probability distribution of a transition rather than a difference in the sub-automata. This idea is re-exploited in [START_REF] Maier | On the Learning of Timing Behavior for Anomaly Detection in Cyber-Physical Production Systems[END_REF] to propose a preprocessing of the data to separate the different modes, suppressing the splitting operation. A general scheme for offline learning of deterministic real-time automata is presented in [VWW08; VWW12] with statistical tests to decide merge and splits.

All the aforementioned algorithms are offline, working only on a given set of observations. Yet, online techniques for passive learning have also been investigated, forcing to rethink the general scheme of the algorithm, as a tree of all available observations could not be built. OTALA [START_REF] Maier | Online passive learning of timed automata for cyberphysical production systems[END_REF] is an online adaptation of BUTLA, allowing online passive learning from positive examples only in an incremental way. It learns a deterministic real time automaton (only one clock, reset at every transition). It has been extended to a parallel framework [START_REF] Windmann | Learning parallel automata of PLCs[END_REF].

An overview of cyber-physical systems [START_REF] Niggemann | On the Diagnosis of Cyber-physical Production Systems: State-of-the-art and Research Agenda[END_REF] identifies the following key-points for the domain:

• A general learnable model is needed; • this model should deal with timed and hybrid constraints;

• more work should be done at the level of the component, instead of always focusing on root causes: early identification of a faulty component can be more valuable and easier to obtain to root causes.

Learning software. Another longstanding application of formal methods is software analysis. In the case of model learning, the specificity of software is that it allows to leverage some black-box hypotheses by making assumptions on the structure of the observed object. One interesting point is that this specificity can be used to learn more complex models (such as automata with multiple clocks).

When learning software models, the two main models of interest are specifications describing the language of a program and normal use models that describe how the software is used in practice, focusing not on its expressivity or design but on its interactions with actual users.

A wide array of applications exist for these models, ranging from test generation, anomaly identification to detection of inefficiencies or debugging assistance.

One of the early methods is the k-Tail algorithm, first introduced theoretically [BF72] as a variation on the work of Nerode and then reformulated and implemented [START_REF] Cook | Discovering Models of Software Processes from Event-Based Data[END_REF]. This algorithm takes a local approach and compare nodes based on they "k-future" i.e. the next k steps in their executions. This approach is especially efficient at detecting loops, which are a central focus of software learning. It has been extended to models with parameters [START_REF] Lorenzoli | Automatic generation of software behavioral models[END_REF] with the gk-Tail algorithm and to real time models with the Timed k-Tail algorithm [START_REF] Pastore | Timed k-Tail: Automatic Inference of Timed Automata[END_REF]. Interestingly, Timed k-Tail is able te learn automata with multiple clocks, which are notoriously difficult for passive learning, by using a learning bias and targeting nested behaviours (e.g. nested loops or function calls).

An interesting research direction is implemented in the TAUTOKO tool that combines learning and testing. In its earliest version [START_REF] Dallmeier | Generating test cases for specification mining[END_REF] this tool generates a first model using the results of a test suite and enrich it using mutations on the test suite. This work has been extended to avoid the need for an initial test suite and generate different types of new tests [START_REF] Dallmeier | Automatically Generating Test Cases for Specification Mining[END_REF]. This extension allows to iterate on the learning / testing loop and generate new tests. The comparison of these methods is performed in the latter article. The methodology of this work is of great interest to interleave test and learning even in other frameworks, notably the use of a coverage criterion to direct tests at each learning step.

Let us mention that works exist that learn non automata-based models. One example of this is the TREM tool implementing a set of methods to learn regular expressions as specifications [START_REF] Schmidt | TREM: a tool for mining timed regular specifications from system traces[END_REF].

Other applications These works are more loosely related to ours, as our setting greatly differs from positive inference. Applications of model learning are numerous and are not limited to the somewhat classical formal-method application domains of cyber-physical systems and software. We give some examples of different applications.

A combination of learning, abstraction, testing and modeling has been applied to the learning of large scale software with a focus in keeping an expert user in the center of the loop [START_REF] Xiao | Software Testing by Active Learning for Commercial Games[END_REF]. This approach has been demonstrated on commercial games, where the purpose is to maximise a loosely defined enjoyability metric, that can only be evaluated by experts. This makes the learned model central for visualization purposes and calls for an interactive method instead of a monolithic tool.

Model learning is also used in medicine. This is not new, with the important use of Markov's models. In a recent thesis [START_REF] Schmidt | Machine learning of timed automata[END_REF] the modeling is pushed further with the addition of an explicit modeling of time. The author use probabilistic real time automata (and subclasses of these) to model the evolution of a disease, taking into account both timing and discrete (the different symptoms) aspects.

Active learning

Active learning [START_REF] Angluin | Learning Regular Sets from Queries and Counterexamples[END_REF] is a type of learning in which a teacher assesses the learner's progress and directs the learning effort toward meaningful decisions. The learner can request information from the teacher via membership queries, asking about a specific observation, and equivalence queries, proposing to compare the current hypothesis to the correct model; in the latter case, the teacher either accepts the hypothesis or returns a counter-example witnessing mispredictions of the learner's hypothesis.

It was introduced by D. Angluin to leverage the restrictions found on learning from random given samples by E. Gold [Gol67;[START_REF] Gold | Complexity of Automaton Identification from Given Data[END_REF]. The author argues that in a human learning process you can assume the learner to be "helpful", and to interact with the learner as a professor would do for a student. They take the example of a human specialist trying to train an expert system. To quantify this helpfulness would not make a lot of sense, but Angluin proposes a "minimally adequate teacher" (MAT) that would be able to answer membership and equivalence queries as a reference. The MAT structure is illustrated in The MAT allows Angluin to define the L * algorithm, that can learn regular languages represented by a (minimal) deterministic finite automaton. This algorithm identifies nodes by using Nerode's congruence and stands as a proof of concept establishing a well-studied framework [Ang87a; Ang87b; Ang90] which allows for sound proofs of correctness and complexity of learning algorithms. The L * was afterward refined to deal with input-output systems [START_REF] Rivest | Inference of Finite Automata Using Homing Sequences[END_REF] or to obtain a better data structure based on trees [START_REF] Michael | An Introduction to Computational Learning Theory[END_REF]. Extensions have then been made to languages of infinite words ( [START_REF] Maler | On the Learnability of Infinitary Regular Sets[END_REF] for a first work on sub-classes and [START_REF] Farzan | Extending Automated Compositional Verification to the Full Class of Omega-Regular Languages[END_REF] for the general class of ω-regular languages), as well as non-deterministic finite automata [START_REF] Bollig | Angluinstyle learning of NFA[END_REF]. Other models have also been studied such as Mealy machines [Nie03; SG09], I/O automata [START_REF] Aarts | Learning I/O Automata[END_REF], cover automata [START_REF] Ipate | Learning finite cover automata from queries[END_REF] or extended finite state machines (i.e. automata with data) [START_REF] Cassel | Learning Component Behavior from Tests: Theory and Algorithms for Automata with Data[END_REF].

Some work have focused in the addition of continuous time in the active learning framework, notably to some subclasses of timed automata: deterministic TAs with only one clock [An+20] and deterministic event-recording automata (DERA) [GJP06; Gri08; GJL10; Lin+11], which have as many clocks as actions in the alphabet, and where each clock encodes exactly the time elapsed since the last corresponding action was taken. These classes of automata present the advantages of having a low-dimensional continuous behaviour (for 1-clock TAs) and to allow to derive the resets of the clocks directly from the observations (for DERA). Generalization to non-deterministic models has been studied in the case of real-time automata (1-clock automata whose clock is reset at each transition) [An+21].

Timed automata have also been learned via genetic programming [START_REF] Tappler | Time to Learn -Learning Timed Automata from Tests[END_REF], a search method different from active or passive learning that generates (guided) mutations on a model to obtain the desired result.

Interestingly, methods combining learning and formal methods have already been used for DERAs in the case of assume-guarantee model-checking [START_REF] Shang-Wei Lin | Learning Assumptions for CompositionalVerification of Timed Systems[END_REF]. In modular model-checking, building the model of a multi-component system is avoided by the mean of assumptions on components. Such assumptions have then to be proven correct on the component themselves. The main limitation of this method is that expert-knowledge is required to build the assumptions. In [START_REF] Shang-Wei Lin | Learning Assumptions for CompositionalVerification of Timed Systems[END_REF], the authors propose to use active-learning of DERAs [START_REF] Shang-Wei Lin | An Efficient Algorithm for Learning Event-Recording Automata[END_REF] to automatically learn the assumptions.

Introduction

As explained in Section 2.2.1 of the state of the art, real-time systems are prevalent in numerous critical applications, and their correctness is often ensured in practice through testing. The main challenges that an automated test generation must overcome are the partial observability and partial controllability of implementations under test. Namely, some actions cannot be observed, which makes state estimation after a given trace complex, and as some actions are controlled by the system, notably its outputs, it may not always be possible to reach or avoid a given set of states. In the context of formal testing, inputs and outputs of the system are separated in the models and alphabets, giving rise (notably) to the model of timed automata with inputs and outputs (TAIO). This separation is used to encode the controllability issue as a formal game between tester and implementation.

Testing timed systems

In the present chapter, TAIOs will be used for most testing artifacts, namely specifications, implementations, and test cases. Since completeness of testing is hopeless in practice1 , it is helpful and usual to rely on test purposes that focus on some behaviours that need to be tested. We use an extension of TAIOs called Open TAIOs (or OTAIOs) [START_REF] Bertrand | Offline test selection with test purposes for non-deterministic timed automata[END_REF] to formally specify them. OTAIOs play the role of non-intrusive observers of actions and clocks of peer OTAIOs, thus allowing test purposes to identify those behaviours of the specification to be tested. Formal testing also requires to define conformance as a relation between models of specifications and their possible implementations. In the timed setting, the classical tioco relation [START_REF] Krichen | Conformance testing for real-time systems[END_REF] states that, after an observable timed trace of the specification, the outputs and delays of the implementation should be possible in the specification. This testing framework is sketched in Figure 3.1. The general problem we address is to synthesize test cases from a specification, that directs the implementation towards the behaviours targeted by test purposes, with the intention that verdicts issued during the executions of the test cases on the implementation are consistent with the actual conformance between the implementation and the specification. This formal framework will be described more precisely later on. The following example motivates the controllability problem we want to address. gages arrive on the conveyor belt, and after some time they reach an automated sorting area, where the system may dispose of a luggage. If it is not removed, the luggage reaches an operator, who can choose to route it towards one of two planes. If the operator fails to decide, after some time the luggage loops on the belt and restarts the whole process. When testing such system from the operator's point of view, the tester would have no control over the sorting of the luggage by the belt. Possible test cases could verify that the system is able to dispose of a luggage, or that the conveyor speed is as expected ( i.e. that time constraints are met during the system execution). When testing for such properties, the choice operated by the system may be problematic; for instance, the automatic sorting may choose not to dispose of a given luggage, although it is able to, or on the contrary it could dispose of a luggage used to check time constraints, which would not allow to verify them.

Contributions and related works

The present chapter extends the work presented in [START_REF] Henry | Control strategies for offline testing of timed systems[END_REF]. It adapts the game approach proposed in [START_REF] Ramangalahy | Strategies for comformance testing[END_REF] to the timed context using the framework developed in [START_REF] Bertrand | Offline test selection with test purposes for non-deterministic timed automata[END_REF]. Precisely, we develop rank lowering strategies inspired by [START_REF] Ramangalahy | Strategies for comformance testing[END_REF] on top of the testing framework of [START_REF] Bertrand | Offline test selection with test purposes for non-deterministic timed automata[END_REF]. The latter work culminated in the construction of a game between the tester and the system under test, but it did not construct the strategies of the tester to solve the game, i.e., the inputs and delays that could control as much as possible the system through behaviours targetted by the test purpose.

Compared to [START_REF] Ramangalahy | Strategies for comformance testing[END_REF], the model of TA is much more complex than finite transition systems, the test purposes are also much more powerful than simple sub-sequences of the specification considered in that work, thus even if the approach is similar, the game has to be completely revised. Furthermore, we present some fairness assumptions that identify a reasonable restriction of the implementation behaviours, under which our strategies are winning. Our model is a bit different to the one of [START_REF] David | Testing Real-Time Systems under Uncertainty[END_REF], since we do not rely on observation predicates, but partial observation comes from internal actions and non-determinism. While our approach handles non-deterministic specifications and test purposes (thanks to the determinization game presented in [START_REF] Bertrand | A game approach to determinize timed automata[END_REF]), at some point in this work we do require exact determinization to ensure some of the test-case properties. We also assume the constant reachability to a subset of so-called restart transitions that send the model back in its initial configuration. We will however explain what happens when relaxing these assumptions. In comparison, [START_REF] David | Testing Real-Time Systems under Uncertainty[END_REF] avoids determinizing TAs, relying on the determinization of a finite state model, thanks to a projection on a finite set of observable predicates. Cooperative strategies of [START_REF] David | Cooperative Testing of Timed Systems[END_REF] have similarities with our fairness assumptions, but their models are assumed deterministic, and their strategies cannot count the number of control losses that will be faced, just detecting that some are needed. Our approach takes controllability into account in a more complete and practical way with the reachability game and rank-lowering strategies.

An other related line of work is the thesis [START_REF] Van Den | Coverage and Games in Model-Based Testing[END_REF], that discusses the formal relationships between games and model-based testing in the untimed case. Notably, it coins the joker-n strategies, that are winning when the system under tests behaves as the tester needs at n well chosen states along the execution.

The chapter is organized as follows. Section 3.2 introduces basic models: TAIOs, OTAs, OTAIOs, and then timed game automata (TGA). Section 3.3 is dedicated to the testing framework with hypotheses on models of testing artifacts, the conformance relation and the construction of the objective-centered tester that denotes both non-conformant traces and the goal to reach according to a test purpose. Section 3.4 constitutes the core of the chapter and the main contribution. The test-synthesis problem is interpreted as a game on the objective-centered tester. Rank-lowering strategies are proposed as candidate test cases, and a fairness assumption is introduced to make such strategies win. Then properties of test cases with respect to conformance are proved. Section 3.5 presents the algorithms used to compute a machine-compatible symbolic representation of a strategy and some interesting properties of these algorithms. Section 3.6 defines a notion of resistance of configurations and strategies and explains how rank-lowering strategies can be defined and used in the most general framework, where no restrictions on the models are proposed. Notably, in this setting, rank-lowering strategies are not winning.

Finally, Section 3.7 concludes the chapter with some future works and take-away.

Timed automata and timed games

In this section, we introduce our models for timed systems and for concurrent games on these objects, along with some useful notions and operations.

Timed automata with inputs and outputs

In order to adapt timed automata to the testing framework, we consider TAs with inputs and outputs (TAIOs), in which the alphabet is split between input, output and internal actions (the latter being used to model partial observability). We present a variation of TAs (and TAIOs) called open TAs (and open TAIOs) [START_REF] Bertrand | Offline test selection with test purposes for non-deterministic timed automata[END_REF], in which a distinguished subset of observed clocks is only observed and cannot be controlled with resets. This will be useful to specify test purposes describing those behaviours of a TA (or TAIO) that require testing. TAs (and TAIOs) will be viewed as particular cases with no observed clocks.

Definition 3.2.1. An open timed automaton (OTA) is a timed automaton

A = (L A , A init , Σ A , C A = C A p C A o , I A , E A )
where the set of clocks is partitioned into proper clocks C A p and observed clocks C A o so that only proper clocks may be reset along transitions.

Formally E A ⊆ L A × G(C A ) × Σ A × 2 C A p × L A .
We will often ommit the A superscripts when the referenced timed automaton is clear from context.

Intuitively, proper clocks C p are controlled by A through resets, while observed clocks C o can only be observed by A through guards and invariants, but belong to (are proper clocks of) another OTA B that controls them, and with which A is synchronized by product (see below).

Definition 3.2.2. An Open Timed Automaton with Inputs and Outputs (OTAIO) is an OTA in which Σ

A = Σ A ? Σ A ! Σ A
τ is the disjoint union of input actions in Σ A ? (denoted by ?a, ?b, ...), output actions in Σ A ! (denoted by !a, !b, ...), and internal actions in Σ A τ (denoted by τ 1 , τ 2 , ...). We write Σ obs = Σ ? Σ ! for the alphabet of observable actions. We call TAIO an OTAIO with no observed clocks.

TAIOs will be sufficient to model most objects of the testing framework, but the ability of OTAIOs to observe other clocks will be essential to specify test purposes (see Section 3.3.2), which need to synchronize with the specification to focus on behaviours that need to be tested.

The semantics of TAs need to be expanded for OTAs to take into account the clock resets of observed clocks, that are not specified by the OTA but will be observed along with discrete actions.

Definition 3.2.3. Let A = (L A , A init , Σ A , C A p C A o , I A , E A
) be an OTA. Its semantics is defined as an infinite-state transition system T A = (S A , s A init , Γ A , → A ) that differs from the one of a timed automaton in the following ways:

• Γ A = R ≥0 (E A × 2 C A o
) integrates observed clock resets to the transitions labels;

• the transitions corresponding to discrete moves take observed clocks into account i.e.

(( , v), (e In the sequel, we only consider infinite runs that are non-Zeno i.e. have an infinite duration. This restricts the set of theoretically possible runs of some timed automata, but only suppresses runs that have no interest for test-generation: either a Zeno run expresses a finite delay as an infinite sum of delays converging to that value -which is clearly a mathematical artifact -or realises an infinite number of discrete actions in finite time, which is impossible for most systems under test in practice.

A , C A o ), ( , v [C p ∪C o ←0] )) ∈ S A × (E A × 2 C A o ) × S A .
Example 3.2.5. A simple Zeno run is ρ = (s i , 1 2 i , s i+1 ) 1≤i<∞ that simply corresponds to a delay of 1 time unit. As will be explained later in this chapter, separations between consecutive delays in a run will come from so called "decision points" for a strategy. Having an infinite number of them in finite time is thus impossible, as it would require an infinite amount of computations.

Similarly, the timed automaton in Figure 3.4 excluding Zeno runs, ensure that every run ends in 2 , while infinite Zeno runs could oscillate between init and 1 by playing a an infinite number of times in less than 1 time unit.

A finite run is accepted in a set of locations F ⊆ L A if its last configuration belongs to F × R ≥0 . We denote by Ex F (A) the subset of runs accepted in F (note that Ex(A) = When discussing signatures for OTAIOs, the resets of observed clocks is preserved:

Ex L A (A)).
proj(( , g, a, C p , ), C o )) = (a, C p C o ).
The intuition is that signatures should only abstract informations linked to the locations and guards, while preserving everything necessary to identify the behaviour, and observed clocks resets are necessary to deduce the valuations.

Finally, when constructing traces of runs of OTAIOs, and more generally automata with internal actions, these actions are ignored, since they cannot be observed by the environment. For t 1 , . . . , t k ∈ R ≥0 , τ ∈ Σ τ C ⊆ C and a partial signature µ:

trace(t 1 , . . . , t k • (τ, C ) • µ) = ( k i=1 t i ) • trace(µ)
We furthermore define, for an OTAIO A, a trace σ and a configuration s:

• A after σ = {s ∈ S | ∃µ ∈ Sig(A), s init µ - → s ∧ trace(µ) = σ}
is the set of all configurations that can be reached when the trace σ has been observed from s A init ;

• elapse(s) = {t ∈ R ≥0 | ∃µ ∈ (R ≥0 ∪ (Σ τ × 2 C )) * , s µ - → ∧ dur(µ) = t}
is the set of delays that can be observed from location s without observing any action;

• out(s) = {a ∈ Σ ! | ∃e ∈ enab(s), act(e) = a} ∪ elapse(s) is the set of possible outputs and delays that can be observed from s. For S ⊆ S, we write out(S ) = s∈S out(s);

• in(s) = {a ∈ Σ ? | ∃e ∈ enab(s), act(e) = a} is the set of possible inputs that can be proposed when arriving in s. For S ⊆ S, we write in(S ) = s∈S in(s).

The notion of enabled transitions and delay transitions are extended to regions as follows: for a region reg, we let enab(( , reg)) = enab(( , v)) for any v in reg 3 ; for any transition e, we let next(e, reg) be the unique region reg such that for all s ∈ reg, s e -→s implies s ∈ reg , and we write SucTemp(reg) for all strict time-successor regions reg of reg. We also extend the notion of execution to regions 4 . The notion of determinism and determinizability defined for TAs is generalized to OTAIOs, and we define some more useful sub-classes of OTAIOs. An OTAIO A is said

• complete if any action can be played from any configuration, formally S = L × R C ≥0 (i.e., all invariants are always true) and for any location ∈ L and action a ∈ Σ, ( ,g,a,C , )∈E A g = true;

• input-complete if any input can be taken from any configuration, formally for any location ∈ L and action a ∈ Σ ? , ( ,g,a,C , )∈E A g = true;

• non-blocking if it does not block time waiting for an input, i.e. for any s ∈ S A and any non-negative real t, there is a partial run ρ from s involving no input actions (i.e., proj(ρ) is a sequence over

R ≥0 ∪ (Σ ! ∪ Σ τ ) × 2 C
) and such that dur(ρ) = t;

• repeatedly observable if from any state some action can be observed, formally for any s ∈ S, there exists a partial run ρ from s such that trace(ρ) / ∈ R ≥0 .

Remark 3.2.6. As for determinism in the case of TAs, (input-)completeness is defined in a syntactic way, with the semantic versions being:

Completeness: from any reachable state, any delay and action can be taken, i.e., S = L × R C ≥0 ( i.e., all invariants are always true) and for any state s ∈ Reach(A) and any action a ∈ Σ, it holds s

(a,C ) ---→ for some C ⊆ C; Input-completeness: for any s ∈ Reach(A), in(s) = Σ ? .
As for determinism, the semantic notions are implied by the syntactic ones, but not equivalent.

The product of two OTAIOs extends the classical product of TAs.

Definition 3.2.7. Given two OTAIOs

A = (L A , A init , Σ ? Σ ! Σ τ , C A p C A o , I A , E A ) and B = (L B , B init , Σ ? Σ ! Σ τ , C B p C B o , I B , E B ) over the same alphabets, their product is the OTAIO A × B = (L A × L B , ( A init , B init ), Σ ? Σ ! Σ τ , (C A p ∪ C B p ) ((C A o ∪ C B o \ (C A p ∪ C B p )), I, E) where E : ( 1 , 2 ) → I A ( 1 ) ∧ I B ( 2 ) and E is the (smallest) set such that for each ( 1 , g 1 , a, C 1 p , 1 ) ∈ E A and ( 2 , g 2 , a, C 2 p , 2 ) ∈ E B , E contains (( 1 , 2 ), g 1 ∧ g 2 , a, C 1 p ∪ C 2 p , ( 1 , 2 )).
Intuitively, proper clocks of the product are proper clocks of one of the operands, while observed clocks are observed clocks of an operand which are not proper of the other. Two transitions carrying a common action synchronize if guards intersect and only proper clocks of the product can be reset. Notice that if a proper clock of A is an observed clock of B, its evolution is observed by guards of B, and vice versa. Then it is not difficult to see that the product of two OTAIOs corresponds to the intersection of the signatures of the individual OTAIOs, i.e. Sig(A × B) = Sig(A) ∩ Sig(B) [START_REF] Bertrand | A game approach to determinize timed automata[END_REF]. Moreover if TAs are equipped with accepting locations

F A ⊆ L A and F B ⊆ L B , we also get Sig F A ×F B (A × B) = Sig F A (A) ∩ Sig F B (B).

Timed games

We introduce timed game automata [START_REF] Asarin | Controller Synthesis for Timed Automata[END_REF], which we later use to formalize the interactions between the tester and the system under test.

Definition 3.2.8. A timed game automaton (TGA) is a timed automaton

A g = (L, init , Σ co Σ uco , C, I, E) where Σ = Σ co Σ uco is

partitioned into actions that are controllable by the player (Σ co ), and those that are not (Σ uco ).

Intuitively, the tester is the player for which we will try to find winning strategies against its opponent, the system under test. All the notions of runs and signatures defined previously for TAs are extended to TGAs, with the interpretation of Σ co as inputs controlled by the environment and Σ uco as outputs controlled by the system.

Notice that in this work we consider games that are built observable and deterministic. We can thus define strategies in terms of runs. Indeed, any system output can only correspond to a unique transition from any given configuration. Hence, one can always know in which configuration the tester is (because our strategies are deterministic too) and it is sufficient to define strategies on runs and configurations (instead of set of runs / configurations for non-deterministic games). Definition 3.2.9.

Let A g = (L, init , Σ co Σ uco , C, I, E) be a TGA. A strategy for the player is a partial function f : Ex(A g ) → R ≥0 × (Σ co ∪ {⊥}) \ {(0, ⊥)} such that for any finite run ρ, letting f (ρ) = (t, a), t ∈ elapse(last(ρ)
) is a possible delay from last(ρ), and there is an a-transition available from the resulting configuration (unless a = ⊥).

The special action ⊥ is used to model situations where the player only wants to spend some delay: when waiting for the adversary to take a move, or when the player has already won. We do not allow strategies to output (0, ⊥), as it would amount to instantly recompute a strategy, and would only loop until time delays, which is not desirable for a decision-making function. Strategies give rise to sets of runs of A g , defined as follows: Definition 3.2.10. Let A g = (L, init , Σ co Σ uco , C, I, E) be a TGA and f be a strategy over A g . The set of outcomes of f from s init , denoted by Outcome(f, s init ) (we might omit to mention s init when it is clear from the context), is the smallest subset of partial runs starting from s init containing the empty partial run (whose last configuration is s init ), and s.t. for any ρ ∈ Outcome(f ), letting f (ρ) = (t, a) and last(ρ) = ( , v), we have

• ρ • (( , v), t , ( , v + t )) • (( , v + t ), e, ( , v )) ∈ Outcome(f ) for any 0 ≤ t ≤ t and act(e) ∈ Σ uco such that (( , v + t ), e, ( , v )) ∈ pEx(A g ); • and -either a = ⊥, and ρ • (( , v), t, ( , v + t)) ∈ Outcome(f ); -or a ∈ Σ co , and ρ • (( , v), t, ( , v + t)) • (( , v + t), e, ( , v )) ∈ Outcome(f ) with act(e) = a.
An infinite partial run is in Outcome(f ) if infinitely many of its finite prefixes are.

The outcomes of f characterize the subset of runs of A g that can be observed while following the strategy f . The first point corresponds to the system taking an uncontrollable transition while the tester waits. It also includes a race for actions between the system and the tester, when t = t. The second point adds the runs corresponding to the actions selected by the tester, when the system does not interfere.

In this chapter, we will be interested in reachability winning conditions (under particular conditions). In the untimed setting, the set of winning configurations can be computed iteratively, starting from the target locations and computing controllable predecessors in a backward manner. In a timed setting, the computation can be performed on regions, so that it terminates (in exponential time) [START_REF] Asarin | Controller Synthesis for Timed Automata[END_REF][START_REF] Cassez | Efficient On-the-fly Algorithms for the Analysis of Timed Games[END_REF]. In Section 3.4 we adapt this approach to our test-generation framework, after presenting it in Section 3.3. The main idea of this framework is to build a fully-observable deterministic TAIO synthesizing all the information from the specification and the test purpose, which we call objective-centered tester. This tester is then interpreted as a game in order to apply the backward strategy computation.

Testing framework

We now present the testing framework, defining (i) the main testing artifacts, i.e., specifications, implementations, test purposes, and test cases, along with the assumptions we put on them; (ii) a conformance relation relating implementations and specifications. The combination of the test purposes and the specification and the construction of an approximate deterministic tester is explained afterwards.

Framework overview

We first give in Figure 3.5 an overview of the framework detailed in the rest of this section. Conformance testing aims at checking whether some black-box implementation conforms to its specification. In formal testing, the specification is given by a formal model and it is assumed that implementations, which are real objects, behave like some unknown model. Conformance is formalized by a relation linking implementations to specifications. This conformance relation is checked by test cases, which interact with the implementation through controls and observations. Test cases are also given by formal models, synthesized from the specification and from test purposes that target some particular behaviours one would like to test. It is then important that some properties can be proved about the accuracy of test cases to detect (non-)conformance. In the following, a particular formalization is presented in the context of timed systems.

Test context

We use TAIOs as models for specifications and implementations, TGAs and strategies for test cases, and OTAIOs for test purposes. This allows us to define expressive test purposes (as we have a timed automaton to describe objectives and conditions) while remaining abstract with respect to some of the system details given in the specification. On a technical side, it gives a unity to the objects we manipulate. In formal testing, the main objective is to detect non-conformances, which correspond to Fail verdicts. If a test purpose is given, another objective, if no non-conformances are detected, is to exercise behaviours targetted by the test purpose, which are figured out by the Pass verdict. But due to lack of control of the tester upon the implementation, one may miss those targetted behaviours, giving rise to Inconclusive verdicts. In order to enforce the occurrence of conclusive verdicts (i.e., Fail or Pass), we equip specifications with restart transitions, corresponding to a kind of system shutdown and restart, and assume that from any (reachable) configuration, a restart transition is always reachable.

Definition 3.3.1. A specification with restarts (or simply specification) on (Σ

? , Σ ! , Σ τ ) is a non-blocking, repeatedly-observable TAIO S = (L S , S init , (Σ ? ∪ {ζ}) Σ ! Σ τ , C S p , I S , E S ), where ζ / ∈ Σ ? is the restart action. We let Restart S = E S ∩ (L S × G M S (C S ) × {ζ} × {C S p } × { S
init }) be the set of ζ-transitions, and assume that from any reachable configuration, there exists a finite partial execution containing ζ, i.e. for any s ∈ Reach(S), there exists µ s.t.

s µ•ζ -→ s S init .
The non-blocking hypothesis rules out "faulty" specifications having no conformant physically-possible implementation. Indeed, a blocking implementation should be able to stop time to wait for an input that it does not control. As this is impossible, implementations are non-blocking, and so should their specifications be. Repeated-observability will be useful for technical reasons, when analyzing exhaustiveness of test cases. It intuitively ensures that some output will always eventually be visible, allowing to detect the current configuration of the system. Our assumption on ζ-transitions entails: Proposition 3.3.2. Let S be a specification with restarts. Then Reach(T S ) is stronglyconnected.

Proof. Let s be a configuration of T S reachable from s S init . By hypothesis, there exists a finite partial execution starting in s whose trace contains ζ. This trace leads to the configuration s S init hence any reachable configuration of T S is reachable from s, and we conclude that the reachable part of T S is strongly-connected.

Being strongly connected ensures that there is always a possibility to reach any (reachable) configuration, and will help us ensure that a strategy always leads to a conclusive verdict, as it cannot be stuck in a situation where no path exists to a reachability goal. We discuss the consequences of releasing the strong connectivity hypothesis in Remark 3.4.11.

We now introduce the formalization of test purposes. In practice, test purposes are used to describe the intention behind test cases, typically behaviours one wants to test because they describe basic functionalities that must be correct and/or because an error is suspected. In our formal testing framework, we describe them with OTAIOs that observe the specification (its actions and clocks) and use accepting locations to define behaviours to be tested.

Definition 3.3.3. Given a specification S = (L S , S init , (Σ ? ∪ {ζ}) Σ ! Σ τ , C S p , I S , E S Restart), a test purpose for S is a pair (T P, Accept T P ) where T P = (L T P , T P init , Σ ? ∪ {ζ} Σ ! Σ τ , C T P p C S
p , I T P , E T P ) is a complete OTAIO and Accept T P ⊆ L T P is a subset of accepting locations; it is required that transitions carrying restart actions ζ in T P are of the form ( , g, ζ, C T P p , T P init ), i.e., they reset all proper clocks and return to the initial state.

In the following, we may simply write T P in place of (T P, Accept T P ). Notice that we force test purposes to be complete because they are non-intrusive observers: they should never constrain the runs of the specification they observe, but should only label those accepted behaviours to be tested. Test purposes observe the clocks of the specification through guards, but cannot reset them. They have their own proper clocks that they can reset; those clocks may serve e.g. to count unspecified delays. The condition on restart transitions in test purposes encodes the fact that we do not want to test ζ (no test cases will require to make a certain number of restart to complete), and forces the test purpose to be strongly connected.

Example 3.3.4. Figure 3.6 is a test purpose for our conveyor-belt example. It aims to test that it is possible to ship a package to destination 2 in less than 5 time units, while avoiding to visit Waste. The Accept set is limited to one location, named Accept. The test purpose has a proper clock c 2 , and no observed clocks. We denote by oth (for otherwise) the set of transitions that reset no clocks, and are enabled for an action other than ζ when no other transition is possible for this action in this location. This set serves to complete the test purpose.

In practice, conformance testing links a mathematical model, the specification, and a black-box implementation, that is a real-life physical object observed by its interactions with the environment. In order to formally reason about conformance, one needs to bridge the gap between the mathematical world and the physical world. We then assume that the real implementation has the same behavior as an unknown TAIO that we call implementation, and has the same interface as the specification S. The hypotheses made on implementations are not restrictions, but model real-world contingencies: the environment might always provide any input and the system cannot alter the course of time.

Having defined the necessary objects, it is now possible to introduce the timed inputoutput conformance (tioco) relation [START_REF] Krichen | Conformance testing for real-time systems[END_REF]. Intuitively, it can be understood as "after any specified trace, outputs and delays of the implementation should be specified". Definition 3.3.6. Let S be a specification and I ∈ I(S). We say that I conforms to S for tioco, and write I tioco S when: ∀σ ∈ Traces(S), out(I after σ) ⊆ out(S after σ)

Combining specifications and test purposes

Now that the main objects are defined, we explain how the behaviours targetted by the test purpose T P are characterized on the specification S by the construction of the product OTAIO P = S × T P. Since S is a TAIO and the observed clocks of T P are exactly the clocks of S, the product P is actually a TAIO. Furthermore, since T P is complete, Sig(P) = Sig(S). By defining accepting locations in the product by Accept P = L S × Accept T P , we get that signatures accepted in P are exactly signatures of S accepted by T P. Formally: Proposition 3.3.7. Let S be a specification and T P a test purpose on this specification, P = S × T P their product. Then Sig(P) = Sig(S) and Sig Accept P (P) = Sig(S) ∩ Sig Accept T P (T P)

Proof. Remember that the set of signatures of the product of two OTAIOs is the intersection of the signatures of the two original OTAIOs [START_REF] Bertrand | A game approach to determinize timed automata[END_REF], so that Sig(S × T P) = Sig(S) ∩ Sig(T P); since T P is complete (it cannot prevent any signature of S), Sig(T P) = (R ≥0 ∪ (Σ × 2 Cp∪Co )) * , we conclude that Sig(S × T P) = Sig(S).

We also have Sig L S ×Accept(T P) (S×T P) = Sig L S (S)∩Sig Accept T P (T P) thus Sig Accept(P) (P) =

Sig(S) ∩ Sig Accept T P (T P)

By projection on traces, we immediately get: Corollary 3.3.8. Let S be a specification, T P a test purpose, and P their product. S and P are trace-equivalent. This entails that I tioco S if, and only if, I tioco P. Observe in particular that ζ of S synchronize with ζ of T P, which are available everywhere. This induces that the product is also strongly-connected, as shown below.

Corollary 3.3.9. Let S be a specification, T P a test purpose, P their product and T P its associated timed transition system. The reachable part of T P is strongly-connected.

Proof. This proof is derived from the proof of Prop. 3.3.2. Although they are really close, we have to do it again as the product does not ensure any relation on the semantics.

Let (( 1 , 2 ), v) be a reachable configuration of T P . There exists a finite partial execution of S starting in ( 1 , v) whose trace contains ζ, and thus by Corollary 3.3.8 there exists a finite partial execution starting in (( 1 , 2 ), v) whose trace contains ζ. Hence this transition leads to the configuration s init = (( S init , T P init ), 0). It comes that there exists a finite partial execution from (( 1 , 2 ), v) to s init . Hence any reachable configuration of T P is reachable from (( 1 , 2 ), v). It can then be concluded that the reachable part of T P is stronglyconnected. We make one final hypothesis: we consider only pairs of specifications S and test purposes T P whose product P can be exactly determinized. This restriction is necessary for technical reasons: if the determinization is only approximated (i.e. the deterministic product DP a is not trace-equivalent to the product P), we cannot ensure that restarts are still reachable in general and thus lose strong connectivity. As noted in Remark 3.4.11 our approach can still be of interest in this general case, but can benefit of some refinements. This is later explored in Section 3.6. One possible method to achieve determinization is to use the determinization game presented in [START_REF] Bertrand | A game approach to determinize timed automata[END_REF]. This determinization is known to be exact for several classes of timed automata, such as strongly-non-Zeno automata, integer-reset automata, or event-recording automata. Furthermore this game always constructs a deterministic timed automaton, and when the set of traces is approximated, it preserves tioco-conformance in the following sense: if an implementation I conforms to its specification S (equivalently it conforms to P), then it also conforms to the approximate determinization DP a ; in other words, non-conformances with respect to the approximation DP a are still non-conformances with respect to the specification S.

Example 3.3.10. Figure 3.7 represents the product of the conveyor-belt specification of Figure 3.3 and the test purpose of Figure 3.6. All nodes are named by the first letters of the corresponding states of the specification (first) and of the test purpose (second). The only accepting location is (D

2 , A). D 1 , St true Bo, St c ≤ 3 D 2 , St true D 2 , A true So, St c ≤ 1 Wa, Wa c ≤ 1 St, St c ≤ 2 start c ≤ 2 τ {c} true !waste {c} true τ {c} true, ?ship 1 , {c} c 2 > 5, ?ship 2 , {c} c 2 ≤ 5, ?ship 2 , {c} c = 3 !past {c} c = 1, !end 1 , {c} c = 1 !end 2 {c} c = 1, !end 2 , {c} true ζ {c, c 2 } true ζ {c, c 2 } true ζ {c, c 2 } c = 1 τ {c} true ζ {c, c 2 }
Given the product P = S×T P, let DP be its exact determinization. Then Traces(DP) = Traces(P), and the reachability of ζ transitions is preserved. We also realize the closure by Σ τ to obtain an observable model5 . Moreover the traces leading to Accept DP and Accept P are the same.

3 ≤ c ≤ 6, !past, {c} c = 1, !end 1 , {c} c = 1 !end 2 {c} c = 1, !end 2 , {c} true, ζ, {c, c 2 } true, ζ, {c, c 2 } true, ζ, {c, c 2 } true ζ {c, c 2 }

Accounting for failure

At this stage of the process, we dispose of a deterministic and fully-observable TAIO DP having exactly the same traces as the original specification S, and equipped with a subset of locations labelled as Accept DP which identifies traces of runs of S accepted by the test purpose. From this TAIO, we aim to build a tester, that can monitor the implementation, feeding it with inputs and selecting verdicts from the returned outputs.

In order to also explicitly model tioco-faulty behaviours (unspecified outputs and delays after a specified trace), we complete DP with respect to its output alphabet, by adding an explicit Fail location. We call this completed TAIO the objective-centered tester. Definition 3.3.12. Given a deterministic TAIO DP = (L DP , DP init ,

Σ ? Σ ! Σ τ , C DP p , I DP , E DP ), we construct its objective-centered tester OT = (L DP ∪ {Fail}, DP init , Σ ? Σ ! Σ τ , C DP p , I OT , E OT ) where I OT (l) = true.
The set of transitions E OT is defined from E DP by:

E OT = E DP ∪ ∈L DP a∈Σ DP ! {( , g, a, ∅, Fail) | g ∈ G a,l } ∪ (Fail, true, a, ∅, Fail) | a ∈ Σ DP
where for each a and , G a,l is a set of guards complementing the set of all valuations v for which an a-transition is available from ( , v) (notice that G a,l generally is non-convex, so that it cannot be represented by a single guard).

Verdicts are defined on the configurations of OT as follows:

• Pass = ∈Accept DP ({ } × I DP ( )); • Fail = {Fail} × R ≥0 ∪ ∈L DP { } × R Cp ≥0 \ I DP ( ) .
Pass corresponds to behaviours accepted by the test purpose, while Fail corresponds to non-conformant behaviours. Notice that we do not define the usual Inconclusive verdicts (i.e. configurations in which we cannot conclude to non-conformance, nor accept the run with respect to the test purpose) as we will enforce the apparition of Pass or Fail thanks to the preservation of strong connectivity.

In the following, we present some of the useful properties of objective centered testers, notably what properties are kept from DP.

First, the ζ-transitions are preserved in OT . Lemma 3.3.13. Let OT be an objective-centered tester. For any location in L OT \ {Fail}, there exists a finite partial execution ρ ∈ pEx(OT ) starting in and containing a ζtransition.

Proof. The same result holds from any state in S, and Traces(S) = Traces(S × T P). The result follows by exact determinizability of the product S × T P. This, as for previous objects, is used to ensure the strong connectivity of the semantics. Corollary 3.3.14. Let OT be an objective-centered tester and T OT its associated timed transition system. Then Reach(T OT ) \ Fail is strongly-connected.

Proof. The proof is the same as the one of Prop. 3.3.2, using Lemma 3.3.13. Lemma 3.3.13 is the reason our method assumes exact determinizability: we cannot ensure in general that a restart transition will remain reachable: if determinization is approximated, some traces might be lost, and the lemma would not hold anymore.

Observe that OT and DP model the same behaviours. Obviously, their sets of traces differ, but the traces added in OT precisely correspond to runs reaching Fail. We now define a specific subset of runs, signatures and traces corresponding to traces that are meant to conform to the specification. Definition 3.3.15. A run ρ of an objective-centered tester OT is said conformant if it does not reach Fail. We write Ex conf (OT ) for the set of conformant runs of OT , and Sig conf (OT ) (resp. Traces conf (OT )) the corresponding signatures (resp. traces). We write Ex fail (OT ) = Ex(OT ) \ Ex conf (OT ) and similarly for the signatures and traces.

The conformant traces are exactly those traces specified by DP, i.e. Traces(DP) = Traces conf (OT ) and correspond to executions that are tioco-conformant with the specification; on the other hand, Ex fail are runs where a non-conformance is detected. We prove this in the following. Proposition 3.3.16. Let DP be the exact determinization of the product P between a specification and a test purpose, and OT be its associated objective-centered tester. Then

Traces(DP) = Traces conf (OT ).
Proof. An execution is in Ex conf (OT ) if it avoids the Fail verdict. This amounts to avoiding location Fail and respecting the invariants of DP. By construction of OT , this corresponds exactly to the runs of DP.

We can furthermore ensure that no approximation is made, by proving that Traces conf (OT ) and Traces(Ex fail (OT )) are disjoint. As we know by construction that the executions differ, this result highlights that no confusion is made when abstracting from execution to traces. Lemma 3.3.17. Given an objective-centered tester OT , we have

Traces conf (OT ) ∩ Traces(Ex fail (OT )) = ∅.
Proof. Let ρ ∈ Ex fail (OT ). Consider the longest prefix ρ of ρ that does not reach Fail, and let γ be the transition taken after ρ in ρ. Two cases should be considered:

• If γ is a delay transition, then it violates the invariant of the location in DP.

By determinism of DP, DP after trace(ρ ) is a singleton set of states. Hence the same delay is not available after ρ in DP; • If act(γ) ∈ Σ ! then this output is not specified in the current location of DP.

By determinism of DP, DP after trace(ρ ) is a singleton. Hence transition γ is not possible after ρ in DP.

In both cases trace(ρ) / ∈ Traces conf (OT ). As this holds for any run of Ex fail (OT ) we have the desired property.

It remains to say that OT is repeatedly-observable, except for configurations in Fail.

Lemma 3.3.18. For a repeatedly-observable specification S, Reach(OT )\Fail is repeatedlyobservable.

Proof. We know that Traces(S) = Traces(P), hence out(P after σ) = out(S after σ) for all σ ∈ Traces(P). As Traces(DP) = Traces(P) by assumption, we also know that for all σ ∈ Traces(DP), out(P after σ) ⊆ out(DP after σ). It comes ∀σ ∈ Traces(OT ), out(S after σ) ⊆ out(OT after σ) as OT only adds traces to DP. Hence for all s ∈ Reach(S OT ) \ Fail, there exists µ ∈ Sig(OT ) s.t. s µ -→ and trace(µ) / ∈ R ≥0 . Indeed, there exists σ ∈ Traces conf (OT ) such that s = OT after σ (as OT is deterministic outside of Fail) and for s ∈ S after σ, there exists µ such that s µ -→ and trace(µ ) / ∈ R ≥0 . It suffices to take µ ∈ pSig(OT ) such that trace(µ) = trace(µ ), and by the previous trace-inclusion property, such a trace exists.

In this section we explained the construction of the objective-centered tester OT from the specification and a test purpose and some of its properties. It represents the most general behaviours of testers that detect both non-conformance to the specification S and acceptance by the test purpose T P. In the next section, we go further and explain how to tackle controllability problems by interpreting OT as a game whose winning strategies will be test cases that try to avoid control losses.

Translating objective-centered testers into games

In this section, we interpret objective-centered testers as games between the tester and the implementation, and propose strategies that try to avoid control losses. We then introduce a scope in which the tester always has a winning strategy, and discuss the properties of the resulting test cases (i.e. game structure and built strategy).

We want to enforce conclusive verdicts when running test cases, i.e. either the implementation does not conform to its specification (Fail verdict) or the awaited behaviour appears (Pass verdict). We thus say that an execution ρ is winning for the tester if it reaches a Fail or Pass configuration and denote by Win(A g ) the set of such executions. Observe however that Fail is entirely controlled by the implementation, which may or may not reveal non-conformances. We will thus only target Pass (while monitoring Fail).

In the following, we consider the TGA

A OT g = (L OT , OT init , Σ OT ? Σ OT ! , C p , I OT , E OT )
where the controllable actions are the inputs Σ co = Σ OT ? and the uncontrollable actions are the outputs Σ uco = Σ OT ! . Indeed, we place ourselves on the tester side, from which inputs of the system are controlled and outputs are observed.

Rank-lowering strategies

In this part, we restrict our discussion to TGAs where Pass configurations are reachable (when seen as plain TAs). Indeed, when this is not the case (we will discuss the fact that the proposed method can detect this), trying to construct a strategy seeking a Pass verdict is hopeless. This is a natural restriction, as it only rules out test purposes that are unsatisfiable by the specification.

To discuss partial controllability we introduce control losses (that will be formalised later in this section) as configurations where the system under test can propose uncontrollable outputs that move its configuration further away from Pass. We define a hierarchy of configurations, depending on their "distance" to Pass, based on the number of control losses one has to suffer to reach Pass, and the number of transitions toward the next control loss. This uses a backward algorithm, for which we define the predecessors of a set of configuration.

Given a set of configurations S ⊆ S of A OT

g , letting V denote the complement of V , we define three kinds of predecessors of S :

• discrete predecessors by a sub-alphabet Σ ⊆ Σ

Pred Σ (S ) = {( , v) | ∃a ∈ Σ , ∃( , a, g, C , ) ∈ E, v |= g ∧ ( , v [C ←0] ) ∈ S }
• timed predecessors, while avoiding a set V of configurations:

tPred ≤ (S , V ) = {( , v) | ∃t ∈ R ≥0 , ( , v + t) ∈ S ∧ ∀ 0 ≤ t ≤ t. ( , v + t ) / ∈ V }
with a less constraining variant:

tPred < (S , V ) = {( , v) | ∃t ∈ R ≥0 , ( , v + t) ∈ S ∧ ∀ 0 ≤ t < t. ( , v + t ) / ∈ V \ S }
We furthermore write tPred(S ) = tPred ≤ (S , ∅) = tPred < (S , ∅);

• final timed predecessors are defined for convenience (see below):

ftPred(S ) = tPred < (Fail, Pred Σuco (S )) ∪ tPred(Pred Σ (S ))
The first two sets are the easier to understand: Pred Σ (S ) is the set of states from which a transition carrying an action in Σ leads to S ; tPred ≤ (S , V ) and tPred < (S , V ) are the sets of states from which delaying leads to S without entering V , with a discussion on whether reaching S ∩ V is accepted (it is for tPred < (S , V )).

Remark 3.4.1. For tPred < (S , V ), the condition ( , v + t ) / ∈ V \ S is necessary for open zones: without it, for any open S and V , tPred < would be equivalent to tPred ≤ .

The final timed predecessors correspond to situations where the system is cornered into reaching S unless it choses non-conformance or to remain indefinitely idle: on the left side of the union, tPred(Fail, Pred Σuco (S )) is the set of states from which the system under test only has the choice between taking an uncontrollable transition to S (as no uncontrollable transition to S will be available) or reach Fail. On the right side are the states from which there are no transitions to anywhere but S in the future, hence the system will end up playing a transition to S . Intuitively, this is because the system will not remain infinitely idle when it can act (this will be formally enforced by a fairness hypothesis). Such situations are not considered as control losses, as the system can only take a beneficial transition for the tester (either by going to S or to Fail). Observe that tPred ≤ (resp. tPred < ) and ftPred need not return convex sets, but are efficiently computable using Pred and simple set constructions [START_REF] Cassez | Efficient On-the-fly Algorithms for the Analysis of Timed Games[END_REF]. Now, using these notions of predecessors, a hierarchy of configurations based on the distance to Pass is defined. Definition 3.4.2. The sequence (W j i ) j,i of sets of configurations is defined as:

• W 0 0 = Pass; • W j i+1 = π(W j i ) with π(S ) =tPred < S , Pred Σuco (S ) ∪tPred ≤ Pred Σco (S ), Pred Σuco (S ) ∪ftPred(S ) ; • W j+1 0 = π (W j ∞ ) with π (S ) = tPred(S ) ∪ Pred Σ (S ) and W j ∞ the limit 6 of the sequence (W j i ) i .
In this hierarchy, j corresponds to the minimum number of control losses the tester has to go through (in the worst case) in order to reach Pass, and i corresponds to the minimal number of steps before the next control loss (or to Pass). The states in W j+1 0 are considered control losses as the implementation might take an output transition leading to an undesirable configuration (higher in the hierarchy). On the other hand, in the construction of W j i+1 , the tester keeps full control, as it is not possible to reach such bad configurations with an uncontrollable transition. The computation of (W j i ) i corresponds to finding the controllable zones, the least fix points W j ∞ , and then taking a control-loss step W j+1 0 . Finally, the hierarchy iterates this process and terminates on its least fix point, effectively finding all configurations coreachable from Pass and the minimal number of control losses.

Remark 3.4.3. The use of tPred < and tPred ≤ in the definition of W j i+1 corresponds to the following intuition: for tPred < (S , Pred Σuco (S )), it is undesirable to force conditions on S , which is W j i and has been handled by its own definition. On the other hand, tPred ≤ (Pred Σco (S ), Pred Σuco (S )) enforces that no control loss can happen in Pred Σco (S ).

Notice that the sequence (W j i ) is an increasing sequence of pairs of regions and locations, and hence can be computed in time exponential in the size of C OT and linear in the size of L OT . We then have the following property saying that the computation of (W j i ) covers j=0 j=1 j=2

...

i, j ∈ N such that (Reach(A OT g ) \ Fail) ⊆ W j i .
Proof. Let s ∈ Reach(A OT g ) \ Fail be a reachable configuration. Since i) Pass is reachable from s T init (by hypothesis); ii) there is a partial run from s back to the initial configuration (Corollary 3.3.14), then Pass is reachable from s. Moreover, there is such a partial run with length bounded by the number of regions.

For each s ∈ Reach(A OT g ) \ Fail, we fix a finite partial run to Pass, and reason by induction on the length n of this partial run in order to prove that conf ∈ W n 0 :

• case n = 0: in this case s ∈ Pass = W 0 0 ;

• inductive case: we assume that the result holds for n, and take s with a partial run to Pass of length n + 1. Then s γ -→ s for some γ with act(γ) ∈ Γ, and there is a partial run from s to Pass of length at most n, so that s ∈ W n 0 . Hence in the worst case s ∈ W n+1 0 . This proves our result.

As explained above, this property is based on the assumption that the Pass verdict is reachable. Nevertheless, if this were not the case, it would be detected during the hierarchy construction, which would then converge to a fixpoint not including s Ag init . As all the configurations in which we want to define a strategy are covered by the hierarchy, we can use it to define a preference relation (i.e. a total preorder) that will guide the tester to better places in terms of control and distance to Pass. Definition 3.4.5. Let s ∈ Reach(A OT g ) \ Fail. The rank r(s) of s is the pair

(j s = arg min j∈N (s ∈ W j ∞ ), i s = arg min i∈N (s ∈ W js i ))
When r(s) = (j s , i s ), it holds s ∈ W js is and j s is the minimal number of control losses before reaching an accepting state, and i s is the minimal number of steps in the strategy before the next control loss. We write s s when r(s) ≤ N 2 r(s ), where ≤ N 2 is the lexical order on N 2 . Intuitively, it is easier to drive trajectories to Pass from s than from s . Proposition 3.4.6.

is a total preorder on Reach(A OT g ) \ Fail.

Proof. inherits transitivity and reflexivity from ≤ N 2 . It is not antisymmetric because several configurations may have the same rank, for example, all configurations of Pass have rank (0, 0). As by Prop. 3.4.4 there exists j, i ∈ N 2 such that W j i covers Reach(A OT g ) \ Fail, all these configurations have a rank and can be compared, hence is total.

Example 3.4.7. In Figure 3.10, the (W j i ) are represented on the game constructed from the OT corresponding to the deterministic TA presented in Figure 3.8. For clarity, the Fail location and the transitions leading to it are not represented. In this example,

W 0 0 = (D 2 , A) × R ≥0 , W 1 0 = W 0 0 St × (c ≤ 5) and W 1 1 = W 1 0 St × (5 < c ≤ 6) ∪ {Wa, (D 1 , St), (D 2 , St)} × R ≥0 . The Fail verdict corresponds to Fail ∪ (St × (c ≥ 6)). D 1 , St true St true D 2 , St true D 2 , A true Wa true c ≤ 3 !waste {c} true, ?ship 1 , {c} c 2 > 5, ?ship 2 , {c} c 2 ≤ 5, ?ship 2 , {c} 3 ≤ c ≤ 6, !past, {c} c = 1, !end 1 , {c} c = 1 !end 2 {c} c = 1, !end 2 , {c} true, ζ, {c, c 2 } true, ζ, {c, c 2 } true, ζ, {c, c 2 } true ζ {c, c 2 } Figure 3
.10: The ranks of A g Example 3.4.8. This notion of rank can be applied to many difficult games i.e. games where you cannot always win, even out of the testing framework. For example, the game in Figure 3.11 has the following ranks, with all configurations in 6 considered as accepting: The notion of rank and its order relation define a preference relation on configurations, with configurations in Pass being the minimal elements. We use it to define a strategy trying to decrease the rank during the execution, i.e. decrease in each state both the distance and the number of control losses. For any s ∈ S, we write r -(s) for the largest rank such that r -(s) < N 2 r(s), and W -(s) for the associated set in (W j i ) j,i . We (partially) order pairs (t, a) ∈ R ≥0 × Σ according to t. Definition 3.4.9. A strategy f for the tester is rank-lowering if, for any finite run ρ with last(ρ) = s = ( , v), it selects the smallest delay satisfying one of the following constraints:

(0, 0) in 6 × R ≥0 , (0, 1) in 5 × R ≥0 , (0, 2) in s 4 × (c > 4), (1, 0) in 3 × R ≥0 ∪ 4 × (c ≤ 4), (1, 1) in 2 × (c < 3), (1, 2) in 2 × (c ≥ 3), (2, 0) in 0 × (c ≤ 3) ∪ s 1 × R ≥0 and (2, 1) in 0 × (c > 3).
• if s ∈ tPred(Pred Σco (W -(s))), then f (ρ) = (t,
• if s ∈ tPred(Pred Σuco (W -(s))), then f (ρ) = (t, ⊥) with t such that s t - → s / ∈ Pred Σuco (W -(s)), ∃t < t s t - → s ∈ Pred Σuco (W -(s)
) and t is minimal if such a t exists. Else t is maximal in the same sense as above (maximal delay successor region);

• if s ∈ tPred(W -(s)),then f (ρ) = (t, ⊥) such that s t - → s with s ∈ W -(s)
, and t is minimal in the same sense as above;

• otherwise f (ρ) = (t, ⊥) where t is maximal in the same sense as above (maximal delay-successor region).

The first three cases follow the construction of the (W j i ) and propose the shortest behaviour leading to W -. The fourth case corresponds either to a configuration of Pass, where W -is undefined, or to a ftPred leading to Fail, cases in which the tester has won. Interestingly, (possibly several) rank-lowering strategies always exist. The constraints of rank-lowering strategies allow the creation of memoryless strategies, i.e. strategies that only rely on local information without keeping the history in memory. Indeed, they only use local information, aggregated during the incremental construction of sets (W j i ). Hence, a straightforward implementation only using the sets (W j i ) is memoryless.

Example 3.4.10. An example of a rank-lowering strategy on the automaton of Figure 3.10 is: in (D 2 , A), play ⊥ (as W 0 0 has been reached); in St, play (0, ship 2 ?) from W 1 0 , otherwise play (1, ⊥). In any other state, play (0, ζ). The omission of Fail in Figure 3.10 does not impact the strategy representation: Fail is a winning set of configurations, but is not targetted by the strategies. Remark 3.4.11. It is worth noting that even in a more general setup where the models are not equiped with ζ-transitions, as in [START_REF] Bertrand | A game approach to determinize timed automata[END_REF], rank-lowering strategies may still be useful: as they are defined on the co-reachable set of Pass, they can still constitute test cases, and the configurations where they are not defined are exactly the configurations corresponding to a Fail verdict or to an Inconclusive verdict, i.e. no conclusions can be made since an accepting configuration cannot be reached.

Yet, rank-lowering strategies would gain to be refined in this case in order to avoid Inconclusive verdicts as much as possible. Indeed, as defined in this chapter, a rank lowering strategy would choose a path that is shorter but more prone to inconclusivity over a longer but safer path. This can be handled by adding a new layer to the ranks encoding the distance to an inconclusive verdict ( i.e. the minimum number of uncontrollable actions required to reach one) over the path to Pass. Strategies are defined in a similar fashion, but in this general case we cannot ensure that they win, as they could sometimes reach Inconclusive.

We detail this generalization in Section 3.6.

Making rank-lowering strategies win

A rank-lowering strategy is generally not a winning strategy: it relies on the implementation fairly exploring its different possibilities and not repeatedly avoiding an enabled transition. In this section, we introduce a notion of fairness, and prove that the rank-lowering strategies are winning under this fairness assumption.

The following lemma ensures that we cannot end in a situation where no transitions can be taken, forcing the system to delay indefinitely. It will be used with the support of fairness, and will be the key to ensuring victory on fair executions. Lemma 3.4.12. If OT is repeatedly-observable, then for any run ρ = ((s i , γ i , s i+1 )) i∈N ∈ Ex(A g ) ending with an infinite sequence of delays, if ρ does no reach Fail, there is an infinite number of states in ρ where some transition e is enabled, formally,

7 ρ / ∈ Ex fail (A g ) ⇒ ∃e ∈ E A g , ∞ ∃ i ∈ N, e ∈ enab(s i ).
Proof. We show this lemma by contradiction. Assume that for some ρ ∈ Ex(A g ), we have

ρ / ∈ Ex fail (A g ) ∧ ∀e ∈ E A g , ∞ ∀ i ∈ N, e / ∈ enab(s i ).
Let ρ max be the shortest prefix such that no transition is enabled after this prefix along ρ (it exists because E is finite and there is only a finite number of these prefixes per element of E). Consider any prefix ρ of ρ strictly containing ρ max ; there is no partial signature µ such that last(ρ ) µ -→ and trace(µ) / ∈ R ≥0 , as there is no time successor of last(ρ ) with an enabled transition. This contradicts the repeated-observability of OT out of Fail (as A g and OT are the same automaton).

In order to introduce our notion of fairness, we define three notions of the infinite support of a run. The first one characterizes the set of regions encountered infinitely often, while the other two distinguish the regions from which a discrete action was taken infinitely often from the regions left by elapsing time infinitely often. This distinction will help us precisely define the behaviour expected from the implementation in each case. Definition 3.4.13. Let ρ be an infinite run, its infinite regions support Inf(ρ) is the set 7. In this expression, ∞ ∃ i ∈ N, φ(i) means that φ(i) is true for infinitely many integers. In the same way, ∞ ∀ i ∈ N φ(i) means that φ(i) is true for all but finitely many integers.

of regions appearing infinitely often in ρ:

Inf((s i , γ i , s i+1 ) i∈N ) = {reg ∈ R | ∞ ∃ i ∈ N, s i ∈ reg ∨ (γ i ∈ R ≥0 ∧ ∃s i ∈ reg, ∃t i < γ i , s i t i - →s i )}
Its infinite transitions support Inf E (ρ) is the set of pairs (reg, e) of R × E Ag such that the transition e is taken infinitely often from the region reg:

Inf E ((s i , γ i , s i+1 ) i∈N ) = {(reg, e) ∈ Inf(ρ) × E A g | ∞ ∃ i ∈ N, s i ∈ reg ∧ γ i = e}
and its infinite waiting support is the set of regions left infinitely often by delaying along ρ.

Inf t ((s i , γ i , s i+1 ) i∈N ) ={reg ∈ Inf(ρ) | ∞ ∃ i ∈ N, γ i ∈ R ≥0 ∧ ∃ t ≤ γ i , s i t - →s i ∈ reg ∧ s i+1 ∈ reg ∈ SucTemp(reg)}
Using these notions of infinite support, we will define the fairness of a run. The intuition behind it is as follows: if a behaviour is implemented, then it will be ultimately displayed if repeatedly requested. Formally, we reason on the game and not on the implementation, as we do not know its model, making our fairness assumption stronger than what is intuitively expected.

To ease the reading of the definition we define the set of transitions with uncontrollable actions E uco = {e | act(e) ∈ Σ uco } and similarly for controllable actions E co = {e | act(e) ∈ Σ co }. We furthermore define the set of controllable enabled transitions for a region reg: enab co (reg) = enab(reg) ∩ E co and similarly the set of conformant uncontrollable enabled transitions (i.e. that do not lead to Fail):

enab uco,conf (reg) = enab(reg) ∩ E uco \ {e | next(e, reg) ⊆ Fail}.
Definition 3.4.14. An infinite run ρ in a TGA A g = (L, init , Σ uco Σ co , C, I, E) (with timed transitions system T = (S, s init , Γ, → T )) is said to be fair when: 8 8. See definitions of enab, next, SucTemp in subsection 3.2.1.

∀reg ∈ Inf(ρ),                {reg} × enab uco,conf (reg) ⊆ Inf E (ρ) ∧ (3.1)      ∨ {e | (reg, e) ∈ Inf E (ρ)} ∩ E co = ∅ ∨ reg ∈ Inf t (ρ) enab co (reg) = ∅ ∧ SucTemp(reg) ⊂ Fail      (3.2)
We denote by Fair(A g ) the set of fair runs of A g .

Fair runs model restrictions on the system runs corresponding to strategies of the system. The first part (3.1) of the definition ensures that for each region visited infinitely often, each tioco-conformant enabled action of the implementation will be played infinitely often from that region. Intuitively, it means that the implementation explores all of its options infinitely often. The second part (3.2) ensures that the implementation will infinitely often let the tester play in this region, by saying that either a discrete action (selected by the tester) has been played infinitely often from this region (first disjunctive case) or that it has been possible to leave this region by waiting infinitely often (second disjunctive case). This is limited by the third disjunctive case, when there is no controllable action enabled and the strict timed successors of the region are either empty (maximal delay successor region) or included in Fail. This definition of fairness is related to the "strong fairness" notion used in model checking. Restricting to fair runs is sufficient to ensure a winning execution when the tester uses a rank-lowering strategy. Intuitively, combined with Lemma 3.4.12 and the repeated-observability assumption, it assures that the system will keep providing outputs until a verdict is reached, and allows to show the following property. Proposition 3.4.15. Rank-lowering strategies are winning on Fair(A g ) (i.e., all fair outcomes are winning).

In terms of testing, this means that if the implementation is fair, a tester following a rank-lowering strategy will reach a conclusive verdict. In order to carry out this proof, we reason on regions. We exploit the fact that a region is included in any W j i it intersects. Proof. Let T = (S, s init , Γ, → T ) be the timed transition system associated with A g = (L, init , Σ uco Σ co , C, I, E). Let f be a rank-lowering strategy. We want to prove that Outcome(f ) ∩ Fair(A g ) ⊆ Win(A g ). We proceed by contradiction.

Suppose there exists an infinite run ρ ∈ Outcome(f ) ∩ Fair(A g ) such that ρ / ∈ Win(A g ). We consider the set of prefixes of ρ ending in a "decision point" i.e. where the strategy proposed a new pair (delay,action). Observe that as there are infinitely many such prefixes, there exist some regions where infinitely many of them end. These regions are a subset of Inf(ρ). We denote by r min the minimal rank obtained in these regions and consider one of these regions reg such that r(reg) = r min . By definition of reg there are infinitely many prefixes ν of ρ ending in reg such that the strategy proposed a new pair. We write

f (last(ν)) = (t f ν , a f ν ).
As there are four possible ways to propose an action for a rank lowering strategy, at least one of them appeared infinitely often. We distinguish these four cases in our reasoning:

• a f ν ∈ Σ co , i.e. last(ν) ∈ tPred(Pred Σco (W -(last(ν)))
). In this case, there exists e ∈ E A g such that act(e) = a f ν ∧ last(ν)

t f ν -→ s ν e - →s ∈ W -(last(ν)).
Because of the minimality constraint on t f ν there exists a unique region reg such that for all ν, s ν ∈ reg . If

last(ν) ∈ reg then last(ν) ∈ Pred {a f ν } (W -(last(ν)))
and by the minimality constraint on t f ν no rank lowering strategy can delay out of reg , hence reg / ∈ Inf t (ρ). Furthermore e ∈ enab c (reg). It follows from fairness that there exists a controllable transition e such that (reg, e) ∈ Inf E (ρ). As this can only be a transition labeled by a f ν by definition of Outcome(f ) and

A g is deterministic, next(e, reg) ∈ Inf(ρ)∩W -(last(ν)) = Inf(ρ) ∩ W -(reg)
and f will take a new decision once arriving in this region after each of the infinitely many transitions, contradicting the minimality of r min . Else, when last(ν) / ∈ reg we have reg ∈ SucTemp(reg) \ Fail. Hence by construction of a rank lowering strategy and definition of Outcome(f ), there is no controllable transition taken in reg along ρ. Hence by fairness reg ∈ Inf t (ρ) and the first delay-successor region of reg is in Inf(ρ). By induction on the number of regions between reg and reg , using the case reg = reg as the base case and the previous remark as induction step, we know that reg ∈ Inf(ρ). We can then conclude using the previous discussion;

• a f ν = ⊥ and last(ν) ∈ tPred(Pred Σuco (W -(last(ν)))). In this case there exists e ∈ E A g such that act(e) ∈ Σ uco ∧ ∃t ≤ t f ν , last(ν) t - → s ν e - → s ∈ W -(last(ν))
. Plus, because of the minimality constraint on t f ν there exists a unique region reg such that for all ν last(ν)

t f ν -→∈ reg .
As in the previous case, if reg = reg then by fairness (reg, e) ∈ Inf E (ρ) as e is uncontrollable and thus next(e, reg) ∈ Inf(ρ) ∩ W -(reg) and f will take a new decision once arriving in this region after each of the infinitely many transitions, contradicting the minimality of r min . Else reg ∈ SucTemp(reg) \ Fail and as a rank lowering strategy will never play a discrete transition in reg by minimality constraint on t f ν we have the same induction as in the previous case and we can conclude using the case reg = reg ;

• a f ν = ⊥ and last(ν) ∈ tPred(W -(last(ν))). In this situation the simple induction on time successors regions is enough to conclude that we reach infinitely often a region in W -(last(ν)) in which (by minimality of the delays proposed by a rank lowering strategy) the strategy will take a decision infinitely often. This contradicts once again the minimality of r min ;

• otherwise, either W -(last(ν)) is undefined and hence r min = (0, 0), contradicting the fact that ρ is not winning, or there is no partial execution from last(ν) to W -(last(ν)) meaning that the system can only delay. By construction of the (W j i ) this corresponds to a timed predecessor of Fail. Hence the system is cornered and can only delay to Fail, thus ρ should be winning.

Under the hypothesis of a fair implementation, we thus have identified a test-case generation method, starting from the specification with restarts and the test purpose, and constructing a test case as a winning strategy on the game created from the objectivecentered tester. The complexity of this method is exponential in the size of DP. More precisely: Proposition 3.4.16. Given a deterministic product DP, the objective-centered tester OT can be linearly computed from DP, the construction of a strategy relies on the construction of the (W j i ), and is hence exponential in the size of C DP and linear in the size of L DP .

Observe that if DP is obtained from P by the game presented in [START_REF] Bertrand | A game approach to determinize timed automata[END_REF], then L DP is doubly-exponential in the size of C S C T P C DP (in the setting of [START_REF] Bertrand | A game approach to determinize timed automata[END_REF], C DP is a parameter of the algorithm).

Properties of the test cases

Having constructed strategies for the tester, and identified a scope of implementation behaviours that allows these strategies to enforce a conclusive verdict, we now study the properties obtained by the test generation method presented above. We call test case a pair (A g , f ) where A g is the game corresponding to the objective-centered tester OT , and f is a rank-lowering strategy on A g . We denote by T C(S, T P) the set of possible test cases generated from a specification S and a test purpose T P, and T C(S) the set of test cases for any test purpose. Recall that it is assumed that the test purposes associated with a specification are restricted to those leading to a determinizable product.

We first define parallel runs as the possible outcomes of a test case combined with an implementation, which thus model their parallel composition. Definition 3.4.17. Given a test case (A g , f ) and an implementation I, their parallel runs are the sequences

((s i , s i ), (γ i , γ i ), (s i+1 , s i+1 )) i such that (s i , γ i , s i+1 ) i is an outcome of f in A g , ((s i , γ i , s i+1 )) i is a run of I, and for all i, either γ i = γ i if γ i ∈ R ≥0 or act(γ i ) = act(γ i ) otherwise.
We write ParRun(A g , f, I) for the set of parallel runs of the test case (A g , f ) and of the implementation I.

We say that an implementation I fails a test case (A g , f ), and write I fails (A g , f ), when there exists a run in ParRun(A g , f, I) that reaches Fail. Our method is sound, that is, a conformant implementation cannot be detected as faulty.

Proposition 3.4.18. The test-case generation method is sound: for any specification S, it holds

∀I ∈ I(S), ∀(A g , f ) ∈ T C(S), (I fails (A g , f ) ⇒ ¬(I tioco S)).
Proof. Let S be a specification, I ∈ I(S) and (A g , f ) ∈ T C(S). Suppose that I fails (A g , f ), we will prove that ¬(I tioco S).

Since I fails (A g , f ), there is a finite run ρ of ParRun(A g , f, I) such that last(ρ) ∈ Fail × S I and it is the first configuration of ρ in this set. Let σ = trace(ρ). By construction of Fail, either σ = σ • t (if the configuration of Fail reached corresponds to a faulty invariant) or σ = σ •a with a ∈ Σ ! (and Fail is reached). In both cases out(I after σ ) out(DP after σ ), and by definition ¬(I tioco DP).

As Traces(P) = Traces(DP) by exact-determinizability hypothesis, ¬(I tioco P). Finally, as Traces(P) = Traces(S), we have ¬(I tioco S), which concludes the proof.

The proofs of this property and the following one are based on the exact correspondence between Fail and the faulty signatures of S, and use the trace equivalence of the different models (DP, P and S) to conclude. As they exploit mainly the game structure, fairness is not used, and they do not rely on the strategy being played. They are in fact true for any strategy and not only for rank-lowering ones.

We first state that if a test case exercises a non-conformant trace, then it produces the Fail verdict. Proof. Let S be a specification, I ∈ I(S) and (A g , f ) ∈ T C(S). Suppose that ¬(ParRun(A g , f, I) tioco S). We want to show that I fails (A g , f ). By definition of ¬(ParRun(A g , f, I) tioco S), there exist σ ∈ Traces(S) and a ∈ out(ParRun(A g , f, I) after σ) \ out(S after σ) Observe that once again, the properties of the strategy are not used. This method also enjoys a precision property: traces leading the test case to Pass are exactly traces conforming to the specification and accepted by the test purpose. The proof of this property uses the exact encoding of the Accept states and the definition of Pass. As the previous two, it then propagates the property through the different test artifacts.

with out(ParRun(A g , f, I) after σ) = {a ∈ Σ ! ∪ R ≥0 | ∃ρ ∈ ParRun(A g , f, I), trace(ρ) = σ •
Proposition 3.4.20. The test case generation method is precise: for any specification S and test purpose T P it can be stated that

∀(A g , f ) ∈ T C(S, T P), ∀σ ∈ Traces(Outcome(f )), A g after σ ∈ Pass ⇔ (σ ∈ Traces(S) ∧ T P after σ ∩ Accept T P = ∅)
Proof. Let σ be in Traces(Outcome(f )). Then A g after σ ∈ Pass if, and only if, the run ρ such that trace(ρ) = σ (which is unique by determinism of A g outside Fail) is such that last(ρ) ∈ Pass, i.e. ρ ∈ Ex(DP) and last(ρ) ∈ Accept DP . Hence DP after σ ∈ Accept DP and as the determinization is exact, σ ∈ Traces(P) and P after σ ∈ Accept P , which gives by definition σ ∈ Traces(S) ∧ T P after σ ∩ Accept T P = ∅.

The proof uses only properties of the game, and once more does not rely on the precise strategy used.

Lastly, this method is exhaustive in the sense that for any non-conformance, there exists a test case that allows to detect it, under fairness assumption. Proposition 3.4.21. The test generation method is exhaustive: for any exactly determinizable specification S and any implementation I ∈ I(S) making fair runs

¬(I tioco S) ⇒ ∃(A g , f ) ∈ T C(S), I fails (A g , f ).
To demonstrate this property, a test purpose is tailored to detect a given nonconformance, by targeting a related conformant trace.

Proof. Let S be a specification, and I ∈ I(S) a non-conformant implementation. By definition of ¬(I tioco S), there exists σ ∈ Traces(S) and a ∈ R ≥0 ∪ Σ ! such that a ∈ out(I after σ) and a / ∈ out(S after σ). As S is repeatedly-observable, there exists

t ∈ R ≥0 and b ∈ Σ S obs such that σ • t • b ∈ Traces(S).
Because S is also non-blocking, if a is a delay, we can take b ∈ Σ S ! . Indeed, otherwise there would be no trace controlled by the implementation for any finite time (say, for time a).

It is possible to build a test purpose T P that accepts exactly the trace σ•t•b. It suffices to direct every transition that is not part of this trace to a sink location. As σ •t•b ∈ Traces(S), it is also a trace of the product P = S × T P. As S is exactly determinizable and T P is deterministic, P is exactly determinizable by allowing enough resources (number of clocks and maximal constant) to DP. We thus obtain Traces(DP) = Traces(P) and

σ • t • b ∈ Traces(DP). Hence, the minimal elements of Pass are OT after σ • t • b.
From OT a test case (A g , f ) can be built, with f a rank-lowering strategy. By assumption, the implementation is playing fair runs, hence f is winning. So there exists ρ ∈ Outcome(f ) such that trace(ρ) = σ • t • b, and thus there exists ρ ∈ Outcome(f ) such that trace(ρ ) = σ. By assumption, σ • a ∈ Traces(I), and depending on the nature of a: The properties obtained in this last part correspond exactly to the properties already obtained in [START_REF] Bertrand | Offline test selection with test purposes for non-deterministic timed automata[END_REF] in the determinizable case, our results only adapt the formalism to the game formulation, making it more precise, especially on the interaction between the test strategy and the implementation, as expected from the game formulation. The construction of rank-lowering strategies allows to keep the interesting properties of the test cases, while adding an equally important information about the control of the tests: the test strategies are winning in finite time (i.e. an outcome of the strategy cannot loop infinitely without reaching a winning configuration) against all fair implementations.

• if a ∈ Σ ! then σ • a ∈ Outcome(s Ag init , f ) as A g is complete on Σ ! . Hence σ • a ∈ ParRun(A g ,

Implementing Rank Lowering Strategies

The construction of a practical algorithm to implement a rank lowering strategy mainly boils down to the computation of the (W j i ) hierarchy on a symbolic representation of the game semantic. Although the construction points toward a backward implementation, we chose a forward one in order to allow to (over) approximate the ranks or to use a partial result while the exact ranks are being computed online. Furthermore, we argue that extending this algorithm to the more general setting where Inconclusive configurations exist is less complex than for a backward one.

Algorithm

In order to practically use the rank lowering strategies, an algorithm has to be devised to deal with the necessary symbolic representation of locations and zones. For this we develop on the algorithm proposed in [START_REF] Cassez | Efficient On-the-fly Algorithms for the Analysis of Timed Games[END_REF], that presents an efficient on-the-fly approach to construct winning strategies for timed games, using a zone-based representation. Our symbolic algorithm presented in Algorithm 1 is inspired from this work, and uses at its core a rank updating function detailed in Algorithm 2. In order to present these functions, we first define (j, i) -as the maximal rank strictly lower than (j, i) in order to avoid the distinction between (j, i -1) and (j -1, ∞) in the algorithms. We also define, for a zone z, a location , an action α ∈ Σ, the set of states reached from ( , z) after α and some delay:

Post α (( , z)) = {s | ∃s ∈ ( , z), ∃t ∈ R ≥0 s α.t -→ s }.
As A g is deterministic, this set of states corresponds to a zone in a unique location.

Observe that the (W j i ) operators π and π (defined in Def. 3.4.2) use information about the game structure, and are not restricted to the configurations explored by the algorithm.

In the following, we heavily rely on the properties of π and π , the two functions used to compute the sets (W j i ). We quickly formalize them bellow.

Proposition 3.5.1. For p a function in {π, π } and S a set of configurations, p(S ) ≥ S and p is increasing ( i.e. S ≥ S ⇒ p(S ) ≥ p(S )).

Proof. For the first property, p(S ) ≥ S is implied by the inclusion of null delays in tPred < .

For the second property, notice that Pred Σ and tPred are increasing, and for π that S decreases when S increases.

These properties transfer to the computation of (W j i ) and are used to ground the algorithms. Although it was not formally stated before, it was already highlighted in Figure 3.10.

To maintain coherence and ease the notations, we present the set of configurations computed by the algorithm as unions of products of a location with a zone of valuations. Of course an actual implementation of the algorithm should suppress the information about locations, that is already referenced in the symbolic states.

The computation of an RLS realized by Algorithm 1 is based on a set Visited of encountered symbolic states and a queue Waiting of transitions to be processed9 . For each symbolic state, two informations are stored: a list Depend of incoming transitions and a dictionary W of zones associated with different ranks in the state. The algorithm is based on a method updateRank that will be presented later and correctly updates the estimates W (s) according to the available information. After an initialization taking care of the timed successors of the initial configuration, denoted by ( init , -→ 0 ) (lines 1 to 9), the main loop starts in line 10 and iterates until the reachable part of the automaton is explored, unless the initial configuration is a final one. In this loop, at each iteration a transition is taken from Waiting and processed. There are two cases to consider, presented in Figure 3.12, where the snake arrow represents the call to an edge, forward or backward. If s / ∈ Visited the transition was stored to explore and the first branch of the if executes (line 13). It records the new s and analyses the local information as presented on the left side of Figure 3.12. If new information is acquired, its backpropagation is planed (line 19). The second situation corresponds to a backpropagation triggered by a previous call to line [START_REF] Tappler | Time to Learn -Learning Timed Automata from Tests[END_REF]. In this case we are mostly interested in updating the ranks in the origin of the transition, and backpropagate again if something has been updated. This corresponds to the right side of Figure 3.12. Observe that line 25 is there to account for an exploration that leads to a pre-existing state 10 . Our algorithm is based on a data structure approximating the (W j i ). It takes the form of a dictionary W (s) for each symbolic state s. In order to reduce the memory cost of W (s) we chose to keep in memory only the ranks (j, i) that increase strictly the corresponding zone e.g. such that W j i W j i-1 . The ranks that do not have their zone explicitly stored hence refer to the greatest zone of lesser rank. If no such zone exists, then the zone is empty.

Algorithm 2 describes the updateRank function. Its purpose is for a given state s to update W (s)[(j, i)] according to W (.) of its successors. The general structure of the algorithm is a while loop on the ranks. In order to compute more efficiently, and to avoid the complex discussion about the unbounded number of i for a given j (and of j in general), it relies on an identification of active ranks through active(j, i), i.e. ranks in which W (s )[(j, i)] strictly increases in an s of interest (either s or a successor). We distinguish between local activity (in act loc ,) which has to be updated directly, and activity in a successor (in act suc ), that is of interest for the next rank. Furthermore, we identify the maximal active rank in r max 11 . This is realized in the initialization (until line 8). In the following we use the notations r.i and r.j to denote the components of a rank. The rest of the algorithm is a loop in the ranks, that uses the jumpNext method to update the current rank according to active. The loop terminates when we went far enough to ensure that no more updates can be performed. Lines 10 to 13 compute the current estimation of W j i 10. It corresponds to the case where this transition is not triggered by a backpropagation call. 11. The maximal rank may evolve during the algorithm execution as ranks are added to or removed from active.

according to the W (s ). Then, if this zone is strictly greater than the current approximate, it is updated (starting line 19). Else, the data structure is cleared for this rank (lines 14 to 17). In both cases, local is updated.

The jumpNext method is used in updateRank to correctly select the next rank of interest. Two main discussions are handled by Algorithm 3. First, local and distant ranks have a different behaviour. Indeed, if something is stored in W (s)[(j, i)], it has to be updated and thus we target (j, i). Else, we go to (j, i + 1) as the construction of a zone only depends on the zones of strictly lesser rank. This distinction is performed using the act loc subset of active. Second, when the next active rank is after a control loss (i.e. j increases), we have to know if that control loss can increase the zone. If the control loss operator (π ) has not been applied to the current zone yet, it could lead to an increase of the zone and we go to (j + 1, 0) to test it. Else we can go directly to the next active rank (with the discussion according to its location).

Example 3.5.2. In Figure 3.13 an example of execution of updateRank is presented. On the first row active(j, i) is represented, with the local subset being denoted by (l). On the second row the action of the while loop is represented. The algorithm first looks at rank (0, 1) which is both local and a successor of an active one. It is updated, and we consider for this example that the region has strictly increased. Hence it becomes active and the next iteration goes to (0, 2). This one is not increased and hence the next jump targets the next active rank, (0, 4). As it is a local rank, we stay at (0, 4) and try to update it. Say that it does not correspond to a greater region than the new (0, 1). In this case, it is suppressed from the dictionary and the rank is not local anymore. For this example, we consider that it is not active anymore. It leads to a call to jumpNext that targets the next control loss, i.e. (1, 0). The example corresponds to the case where this control loss does not augment the zone, and we jump just after the next active rank as we know that a control loss would not help (it was just tried without success) and the next rank is not local. If (2, 2) does not yield a strict zone increase, then updateRank terminates and the only key remaining in

W (s) is (0, 1).
Using a forward algorithm instead of a purely backward one allows to compute approximate strategies (by requiring only the initial state to be in W (s init )[(j, i)]), and avoids (most) unreachable states while making it possible to extend the algorithm to prune a (preprocessed) set of losing states (e.g. inconclusive states). Observe that if the strategies are approximated, the (W j i ) are under-approximated i.e. the ranks are over approximated.

Algorithm 2: updateRank 1 updateRank Input: a symbolic state s Output: a boolean denoting whether an improvement has been performed 2 let act suc ((j, i)) be true iff W (s )[(j, i)] = W (s )[(j, i) -] for s being a successor of s. 3 and act loc ((j, i)) be true iff W (s)[(j, i)] = W (s)[(j, i) -] 4 we write active((j, i)) = act suc ((j, i)) ∨ act loc ((j, i)) 5 and r max = max (j,i) active((j, i)) 6 T emp j = -1 // the last j for which a set has been improved 7 T emp W = W (s)[(0, 0)] // the current set 8 j = 0; i = 1 9 while (j, i) ≤ (r max .j + 1, 0) do // We try to improve the set of rank (j, i)

if i = 0 then // then we add a control loss W ← π (∪ s ∈Visited W (s )[(j, i) -]) ∩ s else W ← π(∪ s ∈Visited W (s )[(j, i) -]) ∩ s if W ⊆ T emp W then
// this set does not need to be explicitly stored erase W (s)[(j, i)] // handles the data structure Furthermore it may be necessary to refine the approximation online if a control loss leads to an unexplored part of the system.

act loc (j, i) ← f alse jumpNext else if W (s)[(j, i)] W then // the set was improved W (s)[(j, i)] ← W act loc ((j, i)) ← true T emp j = j T emp W = W i := i + 1 return T emp j ≥ 0
Most importantly, the core of the computation happens inside updateRank, making it easier to adapt the algorithm to different ways to construct the symbolic graph, both forward and backward.

Properties

We prove some interesting properties of the algorithm by constructing them from properties of updateRank and invariants of the main algorithm while loop.

The following lemma ensures that updateRank preserves the soundness of the approximation of the W j i . Informally, it states that from an under-approximation of W j i updateRank cannot overestimate it (i.e. overestimate the rank).

Lemma 3.5.3. The following property is an invariant of updateRank:

∀s ∈ Visited, ∀ i, j ∈ N, W (s)[(j, i)] ⊆ W j i .
Proof. Suppose that the property is satisfied. We show that the property is preserved by induction on (j, i). Let

P (j m , i m ) = ∀j ≤ j m , i ≤ i m , ∀s ∈ Visited, W (s)[(j, i)] ⊆ W j i .
• for W (s)[(0, 0)] nothing is updated so P (0, 0) remains true;

• fix j, i ∈ N and suppose P (j, i). For a given s, either W (s)[(j, i + 1)] is not updated and the property trivially holds, or

W (s)[(j, i + 1)] = π(∪ s ∈Visited W (s )[(j, i)]) ∩ s. By hypothesis, ∪ s ∈Visited W (s )[(j, i)] ⊆ W j i . Furthermore, π is an increasing function. Hence W (s)[(j, i + 1)] = π(∪ s ∈Visited W (s )[(j, i)]) ∩ s ⊆ π(W j i ) ∩ s = W j i+1 ∩ s.
It comes that P (j, i + 1) is satisfied after the call to updateRank;

• fix j ∈ N. We denote by i j the minimal i such that (∀i > i, W j i = W j i ) holds. Suppose P (j, i j ). For a given s, if W (s)[(j + 1, 0)] is not updated then the property is trivially preserved. Else, with the same arguments as in the previous case,

W (s)[(j + 1, 0)] = π (∪ s ∈Visited W (s )[(j, i j )]) ∩ s ⊆ W j+1 0
. It comes that after updateRank P (j + 1, 0) is satisfied.

The following lemma allows to say that W (s) is increasing as a function of the rank, when it corresponds to a coherent approximation. Lemma 3.5.4. For s ∈ Visited, (j, i) > (j , i ), when updateRank is not processing a rank between these two, we have that

W (s)[(j, i)] ⊇ W (s)[(j , i )].
Proof. The proof can be done by a direct induction using the fact that either

W (s)[(j, i)] is not explicitly stored, which means that W (s)[(j, i)] = W (s)[(j, i) -] or W (s)[(j, i)] = π(∪ s ∈Visited W (s)[(j, i) -]) ∩ s ⊇ ∪ s ∈Visited W (s)[(j, i) -] ∩ s = W (s)[(j, i) -]
as π(S) ⊇ S, or the same with π .

We can now use the previous result to state that updateRank correctly updates the approximation.

Lemma 3.5.5. The application of updateRank ensures the two following properties:

• if before the call, ∀s ∈ Visited, ∀i, j ∈ N, W (s)[(j, i + 1)] ⊆ π(∪ s ∈Visited W (s )[(j, i)]) ∩ s then, after the call, ∀s ∈ Visited, ∀i, j ∈ N, W (s)[(j, i + 1)] = π(∪ s ∈Visited W (s )[(j, i)]) ∩ s; • if before the call, ∀s ∈ Visited, ∀i, j ∈ N, W (s)[(j + 1, 0)] ⊆ π (∪ s ∈Visited W (s )[(j + 1, 0) -]) ∩ s then, after the call, ∀s ∈ Visited, ∀j ∈ N, W (s)[(j + 1, 0)] = π (∪ s ∈Visited W (s )[(j + 1, 0) -]) ∩ s.
Proof. We prove the first property. The same proof can be used for the second one with π instead of π. Fix j, i ∈ N and s ∈ Visited. Suppose that before the call to updateRank, ∀s ∈ Visited, ∀i, j ∈ N, W (s)[(j, i + 1)] ⊆ π(∪ s ∈Visited W (s )[(j, i)]) ∩ s.

• if (j, i) is processed, then when the counter of the function are equal to (j, i) we discuss according to the condition W ⊆ T emp W . If it is satisfied, W (s)[(j, i)] is suppressed from the data structure, meaning that W (s)[(j, i + 1)] = T emp W . We furthermore have W ⊇ T emp W as W ⊇ W (s)[(j, i)] as π(S) ⊇ S and same for π , and W (s)[(j, i)] ⊇ T emp W by Lemma 3.5.4. It comes that W (s)[(j, i + 1)] = W and the property is satisfied. If the condition W ⊆ T emp W is not met, either the test W (s)[(j, i + 1)] W is satisfied, and in this case the value of W (s)[(j, i + 1)] is updated to the correct value, or it is not, and by hypothesis, we already have

W (s)[(j, i + 1)] = W = π(∪ s ∈Visited W (s )[(j, i)]) ∩ s; • if (j, i) is not processed by updateRank, then it means that π(∪ s ∈Visited W (s )[(j, i)]) ∩ s = π(∪ s ∈Visited W (s )[(j, i) -]) ∩ s.
Indeed it means that for all s that matter for s, there is no increase in the region between these two ranks. In this case, we know that (j, i) is not a key of W (s) as the rank is not local (else it would have been updated). It comes that

W (s)[(j, i)] = W (s)[(j, i) -]. By induction W (s)[(j, i) -] = π(∪ s ∈Visited W (s )[(j, i) -]) ∩ s (the in- duction is correctly initialized as (0, 1) is processed). It comes that W (s)[(j, i)] = π(∪ s ∈Visited W (s )[(j, i)]) ∩ s.
Now that the properties of the auxiliary function are stated, we discuss of the while loop invariants in the main algorithm. Lemma 3.5.6. The following properties are true at the beginning and end of every while loop iteration:

• ∀ s ∈ Visited, ∀ α ∈ Σ, noting s = Post α (s): (s, α, s ) ∈ Waiting ∨ (s ∈ Visited ∧ (s, α, s ) ∈ Depend(s )); • ∀s ∈ Visited, ∀ i, j ∈ N, W (s)[(j, i)] ⊆ W j i ; • ∀s ∈ Visited, ∀i, j ∈ N, W (s)[(j, i + 1)] ⊆ π(∪ s ∈Visited W (s )[(j, i)]) ∩ s ∧ (W (s)[(j, i + 1)] = π(∪ s ∈Visited W (s )[(j, i)]) ∩ s ∨ ∃(s, α, s ) ∈ Waiting, s ∈ Visited) ; • ∀s ∈ Visited, ∀j ∈ N, W (s)[(j + 1, 0)] ⊆ π (∪ s ∈Visited W (s )[(j + 1, 0) -]) ∩ s ∧ (W (s)[(j + 1, 0)] = π (∪ s ∈Visited W (s )[(j + 1, 0) -]) ∩ s ∨ ∃(s, α, s ) ∈ Waiting, s ∈ Visited).
Proof. We prove each point independently.

• we prove the first invariant by induction. Before the loop execution, Visited = {s init } and Waiting = {(s init , α, s (2) {(s , α, s ) | s = Post α (s )} is added to Waiting, ensuring the property of s . In both cases, the property is preserved;

) | s = Post α (s init )}.
• we will show the property by induction:

before the loop start, the property is satisfied. Indeed, either W (s init )[(0, 0)] is empty, and in this case all the W (s init )[(j, i)] are empty, or W (s init )[(j, i)] = s init ∩ Pass ⊆ Pass = W 0 0 . In this case, observe that this satisfies the property, and we know by Lemma 3.5.3 that updateRank preserves it;

with the same argument as in the base case, we know that W (s)[(0, 0)] ⊆ W 0 0 . Hence, by induction hypothesis, the property is always satisfied before the call to updateRank, and this call preserves it by Lemma 3.5.3.

• the property is satisfied when first entering the loop. Indeed, if s init ∩ Pass = ∅, W (s init ) is empty and the property trivially holds. Else, notice that

W (s init )[(j, i + 1)] = W (s init )[(0, 0)] ⊆ π(W (s init )[(j, i)]
) ∩ s before the call to updateRank as only W (s init )[(0, 0)] adds some states (i.e.implicitly, all other indices have equal sets). By Lemma 3.5.5 the property is ensured upon entering the loop. During the loop execution, when s is a new state, for any state previously in Visited the property is ensured by induction hypothesis. For s , if there is no successor of s in Visited then the same proof as for s init ensures that the property is satisfied at the end of the loop iteration. Else, such a (s , α, s ) is in Waiting with s ∈ Visited, and W (s ) is empty, and thus W (s )[j, i + 1] is included in every set, and in π(∪ s ∈ Visited W (s )[(j, i)]) ∩ s. This is enough to ensure the property. When s is not a new state, the property is ensured for Visited \ {s} by induction hypothesis. For s, we know that W (s)[(j, i + 1)] ⊆ π(∪ s ∈Visited W (s )[(j, i)]). Hence by Lemma 3.5.5 the call to updateRank ensures the property;

• this property is proven as the previous one, using the second case of the same lemma.

For the following property, we restrict A to its reachable part. This allows to state the third equation without intersecting W j i with Reach(A) or ∪ s∈Visited s.

Proposition 3.5.7. Upon termination of the algorithm, the following properties hold:

∀s ∈ Visited, ∀i, j ∈ N, W (s)[(j, i)] ⊆ W j i (A), Waiting = ∅ ⇒ ∀q ∈ Reach A (( init , 0)), ∃s ∈ Visited, q ∈ s, Waiting = ∅ ⇒ ∀s ∈ Visited, ∀ j, i : s \ W (s)[(j, i)] ⊆ s \ W j i (A) .
Proof. The first property is a direct consequence of Lemma 3.5.6 second point.

For the second property, we reason by induction on an execution. Consider q = ( , v) ∈ Reach(( init , 0)). There exists a path q i a i -→ q i+1 in A with q 0 = ( init , 0) and q n = q. We associate an element of Visited to each q i by induction as follows: q 0 ∈ s init by construction. If a i ∈ R, s i+1 = s i . As the elements of Visited are closed by delays, we have q i+1 ∈ s i+1 . Else s i+1 = Post a i (s i ). Observe that s i+1 is guaranteed to be in Visited by the first point of Lemma 3.5.6 as Waiting = ∅. By induction, we have s n ∈ Visited such that q ∈ s n . In order to prove the third property, we suppose that Waiting = ∅ and prove the property by induction:

• for (0, 0) we have that

W (s)[(0, 0)] = s ∩ Pass = s ∩ W 0 0 . So s \ W (s)[(0, 0)] = s \ (s ∩ W 0 0 ) = s \ W 0 0 ;
• for (j, i + 1) suppose that the property is satisfied for (j, i). s \ W (s)[(j, i + 1)] = s \ π(∪ s ∈Visited W (s )[(j, i)]) by the third point of Lemma 3.5.6. Plus, the induction hypothesis implies that

W (s )[(j, i)] ⊇ W j i ∩s . It comes that π(∪ s ∈Visited W (s )[(j, i)]) ⊇ π(∪ s ∈Visited (s ∩W j i )) as π is an increasing function. Hence, s\π(∪ s ∈Visited W (s )[(j, i)]) ⊆ s \ π(∪ s ∈Visited (s ∩ W j i )). Furthermore s \ π(∪ s ∈Visited (s ∩ W j i )) = s \ π(W j i ∩
(∪ s ∈Visited s )). By using the second equation and restricting A to its reachable part, we have (since

Waiting = ∅): s \ π(W j i ∪ s ∈Visited s ) = s \ π(W j i ) = s \ W j i+1 .
In the end, we have that s \ W (s)[(j, i + 1)] ⊆ s \ W j i+1 ;

• for (j + 1, i) we follow the same ideas using the fourth point of Lemma 3.5.6 instead of the third, and the properties of π instead of those of π.

Together, those three properties ensure that either the algorithm terminates early, and then the W j i are under approximated (i.e.the rank is over approximated) or it computes exactly the W j i (on the reachable part).

Generalizing Rank-Lowering Strategies

This section discusses the use of rank-lowering strategies in the most general case where the reachable configurations of the testing game are not all co-reachable from Pass. In this case, Inconclusive verdicts reappear and, given that we consider difficult games where it is necessary to let the system take (a possibly unbounded number of) uncontrollable transitions to reach our targets, we cannot ensure that our strategies are winning (i.e. eventually reach Pass or Fail). Still it is possible to define resistant strategies that try to avoid the Inconclusive verdict as much as they can -while pursuing a conclusive one.

We define resistent strategies through the notion of k-resistance. Intuitively, a configuration is considered k-resistant if there is a path from it to the goal along which the system needs to play more than k well chosen uncontrollable actions to reach Inconclusive. Bluntly speaking "it remains more than k hurtful events away from Inconclusive configurations".

In the following, we first present the notion of resistance in Section 3.6.1, and then describe a general notion of resistant strategies, before specializing it for resistant ranklowering strategies 3.6.2. Remark 3.6.1. We choose the term of resistance over the one of robustness as this section treats of pure game theory. In this thesis, robustness is always meant with respect to reality, as opposed to "perfect" formal models. Specifically, we consider that a method or model is robust when it copes well with errors in measures and control.

k-Resistance

First, we need to define the notion of inconclusive verdict. A configuration s is said to be inconclusive with respect to a target set Pass when s / ∈ coReach(Pass), i.e. a configuration is inconclusive as soon as we cannot construct a strategy leading to the accepting verdict. Thus, formally Inconclusive = coReach(Pass).

As we now authorize the presence of inconclusive configurations, the conditions on specifications and test purposes can be relaxed: we can consider specifications without restart transitions and products of specifications and test purposes that are not (exactly) determinizable.

The notion of k-resistance is defined similarly to the ranks of rank-lowering strategies, by considering that a "hurtful" uncontrollable transition (or delay) is a control loss. The main difference is that, as this is a safety matter and not an optimization problem, the number of controllable steps separating a configuration from a resistance loss is irrelevant. Based on the prior intuition of resistance, we have that the set of (at least) 0-resistant configurations is coReach(Pass).

Using this, we define the k-resistant sets by induction. Definition 3.6.2. For a TGA A g with a set of target configurations Pass, we define the k-resistant sets R k for k ∈ N inductively as:

R 0 = coReach(Pass) R k+1 = {s | ∃µ, s µ - →Pass ∧ ∀µ 1 µ 2 = µ, s µ 1 -→ s ⇒ s / ∈ Pred Σuco (R k ) ∪ tPred < (R k , Pred Σco (R k )) \ Pass} R ∞ = lim k→∞ R k
The construction of R k+1 relies on the following intuition: a configuration can resist to k + 1 control losses if there exists a partial run from it to Pass such that all uncontrollable deviations from this partial run are k resistant. Uncontrollable delays (i.e. delays that cannot be avoided with a controllable actions) are considered as control losses with the additions of tPred < (R k , Pred Σco (R k )) in the set of configurations to avoid in the definition of R k+1 .

Remark 3.6.3. It is not explicitly required that s / ∈ R k in the definition of R k+1 as this condition is already ensured by s /

∈ tPred < (R k , Pred Σco (R k )).
The following lemma gives some insight on the structure of (R k ) k∈N , which is reminiscent of the (W j i ) structure.

Lemma 3.6.4. For any k ∈ N, we have that

R k+1 ⊆ Pred Σuco (R k ) ∩ tPred < (R k , Pred Σco (R k )) ∪ Pass and R k+1 ⊆ R k .
Proof. We first prove the first property. Consider s ∈ R k+1 . By applying the definition, either s ∈ Pass and we can conclude, or there exists µ such that s

µ - →Pass ∧ ∀µ 1 µ 2 = µ, s µ 1 -→ s and either s ∈ Pass or s / ∈ Pred Σuco (R k ) ∪ tPred < (R k , Pred Σco (R k )). By taking µ 1 = , we can conclude that s / ∈ Pred Σuco (R k ) ∪ tPred < (R k , Pred Σco (R k )), i.e. s ∈ Pred Σuco (R k ) ∩ tPred < (R k , Pred Σco (R k ))
, which implies our first property.

Using this first property we obtain that

R k+1 ⊆ R k by noticing that ∀k ∈ N, Pass ⊆ R k and tPred < (R k , Pred Σco (R k )) ⊆ R k . This last inclusion comes by applying the complement to tPred < (R k , Pred Σco (R k )) ⊇ R k .
The idea of the R k hierarchy is to propose a lower bound to the number of actions that can be taken by an implementation to reach an Inconclusive verdict, by ensuring that such an incontrollable action cannot decrease k by more than 1. As long as k ≥ 0 we know that it is possible to reach a Pass verdict if we play a correct strategy.

Remark 3.6.5. We construct R k with a forward-completion approach. This is useful to avoid "local maxima" of resistance that would trap strategies or require expensive computations to counter, but it also has some flaws, the main one being that if an unsafe set of configurations is unavoidable on the partial run to Pass, then all previous configurations cannot be distinguished, as illustrated in Figure 3.14.

It is possible to leverage this issue by considering "resistance up to a point" to distinguish such configurations but it would either require an enormous computational and memory effort (if every e.g. region is considered) or (more reasonably) expert knowledge to identify bottlenecks and ask for a refinement. This is not discussed here.

To ensure that the hierarchy of k-resistant sets can be computed, a first step is to prove that despite being in theory infinite, only a finite number of different k appear for a given game and reachability objective. Proposition 3.6.6. For a TGA A g and a reachability objective Pass, the number of different elements of (R k ) k∈N∪{∞} is finite. Proof. It is easy to see that a R k contains every region it intersects, as it depends on possible transitions. Hence, if there is an infinite number of different (R k ) k∈N (we can omit R ∞ without loss of generality as it is a unique element) we can find an infinite sequence of location-region pair ( i , reg i ) i∈N and an increasing mapping function m : i → k associating a k to each element of the sequence such that ( i , reg i ) ⊆ R m(i)+1 \ R m(i) . By Lemma 3.6.4 we know that (R k ) k∈N is a decreasing sequence. Thus for each infinitely many k > 0 such that R k = R k-1 , m -1 (k) points to new pairs of locations and regions.

R ∞ R 0 Inconclusive c? u! R 0
As there are only finitely many such pairs, this is impossible.

The previous definition of resistance is intuitive (mathematically speaking), but does not hint toward an algorithm to compute the sets. The following characterization will point toward a computation. Proposition 3.6.7. Consider a TGA A g . An equivalent definition of R k is:

R -1 = L Ag × R |C Ag | ≥0 R k+1 = lfp S →Pass ∪tPred < S , Pred Σuco (R k ) ∪ tPred < (R k , Pred Σco (R k )) ∪tPred ≤ Pred Σ (S ), Pred Σuco (R k ) ∪ tPred < (R k , Pred Σco (R k )) R ∞ = lfp[k → R k ]
where lfp is the least fix point operator.

Remark 3.6.8. As for the (W j i ), it is necessary to distinguish Pred Σ (S ), on which we want to implement some constraints (and thus use tPred ≤ ) and S that should not be constrained (and hence handled with tPred < ). For example, constraining S could be problematic when starting from Pass if there exists uncontrollable transitions from Pass to undesirable configurations.

Proof. The proof is made by induction. First, notice that since R -1 is the set of all configurations, R -1 = ∅. Thus the characterization of R 0 is exactly the definition of co-reachability of Pass as a fix-point and it matches the definition. Now consider for k ∈ N that the definition and characterization of R k match. We make the proof for k+1 by double inclusion. Let us note D i the definition of R i and F i the fixpoint characterization. We will furthermore note

V i = Pred Σuco (R i ) ∪ tPred < (R i , Pred Σco (R i )) when it is known that D i = F i . D k+1 ⊆ F k+1 Consider s ∈ D k+1
. By definition there exists s µ -→ s with s ∈ Pass. If s ∈ Pass ⊆ F k+1 we have our result. Else, consider a partial run ρ = s 0 γ 1 ...γ n s n corresponding to µ. We show by induction that all s i are in F k+1 . Without loss of generality, we consider that there exists 0 ≤ j < n such that for all 0 ≤ i ≤ j, s i / ∈ Pass and conversely for j < i ≤ n, s i ∈ Pass. First, for all such i > j,

s i ∈ Pass ⊆ F k+1 . Next, for i ≤ j, considering s i+1 is in F k+1 , we discuss on γ i+1 ∈ E ∪ R ≥0 . • If γ i+1 ∈ E then s i ∈ Pred Σ (s i+1 )\V k by D k+1 . Thus s i ∈ tPred ≤ Pred Σ (s i+1 ), V k . It comes that s i ∈ F k+1 . • If γ i+1 ∈ R ≥0 then s i ∈ tPred < (s i+1 ∪ Pass, V k ).
Indeed we get the restriction by applying the constraint of D k+1 for µ 1 = γ 1 . . . γ i γ i+1 with 0 ≤ γ i+1 < γ i+1 .

For each such signature, either it ends in Pass or in V k , hence the result. It comes that s i ∈ F k+1 .

By induction, it comes that s = s 0 ∈ F.

F k+1 ⊆ D k+1 It suffices to prove that D is a fixpoint of S → Pass ∪ tPred(S ∪ Pred Σ (S ), R k ∪ Pred Σuco (R k ) ∪ tPred < (R k , Pred Σco (R k )))
that we will note f here. First note that for any sets S and V , tPred < (S , V ) ⊇ S , such that f (D k+1 ) ⊇ D k+1 . It remains to prove that f (D k+1 ) ⊆ D k+1 . We discuss according to the cases of f :

• Pass ⊆ D k+1 ;

• for s ∈ tPred < (D k+1 , V k ) we know that there exists t ∈ R ≥0 such that s Now to conclude on R ∞ it is enough to remark that by Prop. 3.6.6, R ∞ is a R k for k ∈ N.

Remark 3.6.9. The function iterated is close to the π function defining W j i in its construction, but its semantic differs greatly: allowing predecessors by uncontrollable actions and even more importantly avoiding a fix set makes a greatly wider fix point.

This characterization could equivalently start from R 0 = coReach(Pass). This definition however highlights the inductive nature of R 0 and unites the computation for k = 0 and k > 0. We can use it to insist on the forward-completion of the k-resistant sets.

Corollary 3.6.10. For any k ∈ N and s ∈ R k , there exists a partial run ρ such that s ρ -→Pass and for any s reached from s after a prefix of ρ, s ∈ R k .

Proof. The characterization function f constructs such a partial run iteratively.

Example 3.6.11. Consider the timed game automaton over C = {c 1 , c 2 } depicted in Figure 3.15 where Pass is 5 (true guards and empty resets are omitted for the sake of readability). The inconclusive verdicts and different safety ranks are displayed in the table below in the form of location/zone pairs.

Inconclusive 3 × true, 2 × c 1 ≥ 3 R 0 \ R 1 2 × c 1 < 3, 8 × (c 1 ≤ 1 ∧ c 2 < c 1 ) R 1 \ R 2 1 × c 1 < 3, 8 × c 1 > 1, 8 × c 1 ≤ c 2 R 2 \ R ∞ 6 × true, 7 × true R ∞ 4 × true, 5 × true, 1 × c 1 ≥ 3
Notice that if most of control losses defining ranks correspond to uncontrollable transitions, we have an example in 2 or 8 of a control loss due to an uncontrollable delay: in 2 for

c 1 < 3, the condition on tPred < (R k , Pred Σco (R k ))
is the only thing stopping the zone to be in R 1 . The same goes for the zone Finally, we point out that choosing to enhance the resistance (i.e. go from a k to a k + 1 resistant configuration) cannot be done without taking risks. Lemma 3.6.12. Let k ∈ N and s ∈ R k \ R k+1 . Then for any partial run ρ, s

(c 1 < 1 ∧ c 2 ≤ c 1 ) in 8 . 1 2 3 4 5 8 6 7 !a c 1 < 3 !a c 1 < 3 !a c 1 ≥ 3 !a c 1 ≥ 3 ?b ?c c 1 ≤ 2 !a c ≥ 1 {c 1 } ?b ?b ?b !a {c 1 } !a {c 2 } !a c 1 = 1 ∧ c 2 < 1 ?b c 1 > 1
ρ - → s ∈ R k+1 ⇒ ∃ρ ∈ fPref(ρ), s ρ - → s ∈ R k ∪ Pred Σuco (tPred < (R k , Pred Σco (R k )) with fPref(ρ) the set of prefixes of ρ.
Proof. We conduct this proof by contradiction. Consider ρ such that s ρ -→s ∈ R k+1 and for all prefixes ρ , s

ρ - →s / ∈ R k ∪ Pred Σuco (R k ) ∪ tPred < (R k , Pred Σco (R k ))
. Then, by induction, iterating the function f defined in Prop. 3.6.7 will prove that s ∈ R k+1 . This impossibility to enhance resistance without taking risks entails that it should not be a safety decision to enhance it. Instead, the optimization part of the strategy should weight it with its own criterions e.g. is it worth to take that risk, or should another one be preferred?

Combining resistance and optimization

In this section, we explain how to combine safety with respect to the Inconclusive verdict -which is expressed by the mean of so-called safety first strategies-and optimization with respect to a reachability objective.

Safety first strategies

The k-resistant sets do not retain enough information to decide what a player should do to achieve the reachability objective, but they advise about some actions that should not be proposed to avoid as much as possible the Inconclusive verdict. Using them, we define a set of safety first strategy that restrain their choices to actions that are considered "safe" by the resistance hierarchy. For that, we extend the usual notion of strategy with an aim token. This will extend strategies in the case where it relies on its adversary to take a move, by specifying a set of possible actions of the adversary that are expected. Although the player cannot control these actions, the aim is used to prove that the strategy aims for a safe configuration. Definition 3.6.13. A strategy aim for a strategy f is a partial function a f from runs to sets of transitions that is coherent with the strategy, i.e. when f

(ρ) = (t, a) if a ∈ Σ co then a f (ρ) = {e ∈ E | act(e) = a ∧ ρ t•e -→} and when a = ⊥, a f (ρ) ⊆ {e ∈ E | act(e) ∈ Σ uco ∧ ∃t ≤ t, ρ t •e
-→}. We take the convention that an empty aim corresponds to a delay only.

Aims can be used to complement the strategy definition, to add additional constraints on its behaviour. We use it to specify safety-first strategies. Definition 3.6.14. A strategy f in a deterministic TGA A g with a reachability objective Pass is considered to be safety first when for all finite ρ such that last(ρ) / ∈ Pass, with f (ρ) = (t, a),

• if a ∈ Σ co : last(ρ) ∈ R k ⇒ last(ρ) t•e
-→ s ∈ R k with e the unique transition enabled such that act(e) = a;

• if a = ⊥ and a f (ρ) = ∅: last(ρ) ∈ R k ⇒ ρ t •e
-→ s ∈ R k for all e ∈ a f (ρ) and all t ≤ t such that e is enabled after t ;

• if a = ⊥ and a f (ρ) = ∅: last(ρ) ∈ R k ⇒ ρ t - →s ∈ R k .
The set of safety first strategies of a game A g is noted SF Ag .

Remark 3.6.15. This definition of safety-first strategies is tailored for reachability objectives but could easily be adapted for other objectives. We only enforce that the strategy is only safety-first until the game is won to relieve technical discussion about actions after the objective is reached.

These strategies are called safety first because if there exists a safe partial run (as safe as the current configuration at least) they will try to follow it regardless of the efficiency cost. As emphasized by Lemma 3.6.12 this is the strongest requirement one can make on strategies.

Safety first rank lowering strategies A simple combination of (R k ) and (W j i ) proposing to simply play "safety first" and optimize in the set of safest moves fails to generate effective strategies. Indeed, although they both recommend somewhat related runs, these ones can differ, and taking a local decision can be difficult. Figure 3.16 (discussed in Example 3.6.16) proposes a timed automaton that traps a memoryless strategy in a loop. Notice that this kind of difficulty does not depend on the timed nature of the automaton, but arises from the conflict between optimization and safety. To highlight it, no clock constraint is put in this example.

Example 3.6.16. In the automaton displayed in Figure 3.16 a memoryless strategy trying to reach A using only local information about R k and W j i would either be unsafe (and ignore the safest path 1-2-3) or get stuck between locations 1 and 4 . Indeed, after 1 , all directly reachable locations are in R ∞ and the best W j i is (1, 1) in 4 . But once in 4 , prioritizing safety means avoiding the risky location 5 and going back to 1 .

This issue arises from a form of dependency tracking problem: 4 is safe only because 1 is, so there is no point in going in 4 from 1 : this does not progress along a safe path.

In order to prevent such blocking situations, we first compute the (R k ), and use them to define ( k W j i ) and a safety first equivalent of rank lowering strategies.

Definition 3.6.17. Given the sequence (R k ) k∈N∪{∞} , we define the sequence ( k W j i ) j,i∈N as:

• k W 0 0 = Pass for all k; • k W j i+1 = π k ( k ≥k k W j i ) where π k (S ) = π(S ) ∩ R k+1 ; • k W j+1 0 = π k ( k ≥k k W j ∞ ) where π k (S ) = π (S ) ∩ R k+1 ;
Here the intersection of the operators defining W j i (π and π ) with R k+1 comes from the backward nature of the computation: by ensuring that k does not increase when computing the predecessors, we ensure that it does not decrease when playing forward. Using this ( k W j i ), we can now find back some of the useful results we have on (W j i ).

W j i (0, 0) (3, 0) (2, 0) (1, 0) (1, 1) (1, 0) - R k ∞ ∞ ∞ ∞ ∞ 0 -1 Figure 3
.16: A TA trapping a memoryless strategy Proposition 3.6.18. There exists i, j ∈ N such that

Reach(A g ) \ (Inconclusive ∪ Fail) ⊆ k∈N k W j i Proof.
Let s ∈ Reach(A g ) \ (Inconclusive ∪ Fail) be a reachable configuration. We have that s ∈ coReach(Pass) as s / ∈ Inconclusive, and hence there exists a maximal k such that s ∈ R k by finiteness of the number of such k (Prop. 3.6.6). Thus by Corollary 3.6.10 there exists a partial run ρ from s to Pass that stays in R k . By noting n the length of this partial run, we prove that s ∈ k W n 0 by induction on ρ. When n = 0, s ∈ Accept = ∞ W 0 0 and the property holds. If we assume that the result holds for all partial runs of length n and consider s ∈ R k such that its corresponding partial run ρ = (s, γ, s ).ρ has length n+1.

Then by induction hypothesis, as

s is in R k , s ∈ k W n 0 and s ∈ π k ( k W n 0 ), i.e. s ∈ k W n+1 0 .
This result ensures that we have a rank information available everywhere it can be used. We can thus rank the configurations using this information. Definition 3.6.19. Let s ∈ Reach(A g ) \ (Inconclusive ∪ Fail). The rank of s is:

r(s) = (k s = arg max k∈N∪{∞} s ∈ R k , j s = arg min j∈N s ∈ ks W j ∞ , i s = arg min i∈N s ∈ ks W js i )
We can equip the ranks with an order induced by the lexical order on (≥, N)×(≤, N) 2 , i.e. (k, j, i) (k , j , i ) if and only if k > k or k = k and either j < j or j = j and i ≤ i . We note s s when r(s) r(s ).

Proposition 3.6.20.

is a total preorder on Reach(A g ) \ (Inconclusive ∪ Fail).

Proof. inherit transitivity and reflexivity from . It is not antisymmetric as different configurations can have the same rank 12 . It is total thanks to Prop.3.6.18 that assures us that Reach(A g ) \ (Inconclusive ∪ Fail) is covered by the ( k W j i ). We define ≺ as the strict relation corresponding to , and we write r -(s) for the greatest rank such that r -(s) ≺ r(s), and W -(s) for the associated k W j i . Then using this new notion of W -(s), a safety first rank lowering strategy can be defined exactly in the same way that it was when using only W j i .

Definition 3.6.21. We call safety-first rank-lowering strategy a rank lowering strategy defined using W -(s) on the ranks induced by ( k W j i ). We define the aim of these rank lowering strategies in the following way (according to the cases of Definition 3.4.9), with s = last(ρ):

• if s ∈ tPred(Pred Σco (W -(s))) then f (ρ) = (t, a)
with a ∈ Σ co and a f (ρ) = {e} the unique e enabled after ρ • t such that act(e) = a;

• if s ∈ tPred(Pred Σuco (W -(s))), then f (ρ) = (t, ⊥) with t such that s t - → s / ∈ Pred Σuco (W -(s)), ∃t < t, s t - → s ∈ Pred Σuco (W -(s)) and s e - → s ∈ W -(s)
with act(e) ∈ Σ uco . We take a f (ρ) to be the set of such e.

• in the two last cases a f (ρ) = ∅.

The name of safety-first rank lowering strategy is justified by the following proposition. Proposition 3.6.22. Let f be a safety first rank lowering strategy computed in a deterministic TGA A g . Then f ∈ SF Ag .

12. Even configurations coming from different regions or zones can have the same rank.

Proof. We discuss according to the case in the definition of rank-lowering strategy as given in Definition 3.4.9. Noting s = last(ρ) and k we discuss according to f (ρ).

• In the first case, s ∈ tPred(Pred Σco (W -(s))) and then f (ρ) = (t, a) such that there exist e ∈ E with act(e) = a satisfying s t•e -→ s ∈ W -(s). In this case, by definition of W -s ∈ R ks with k s the maximal k such that s ∈ R k . We thus have that for all k such that last(ρ

) in R k , last(ρ) t•a -→ s ∈ R k .
• In the second case, s ∈ tPred(Pred Σuco (W -(s))) and by the definition ∃t < t s t -→s

a f (ρ) ---→ s ∈ W -(s)
. By definition of W -we again have our result.

• In the third case a f (ρ) = ∅ and by Definition 3.4.9 we know that f (ρ) = (t, ⊥) with last(ρ) t -→s ∈ W -(s) and by definition of W -we again have our result.

• In the last case last(ρ) ∈ Pass and there is no condition to satisfy.

Note that safety-first rank-lowering strategies correspond to rank-lowering strategies when Inconclusive = ∅. In this setting they are thus winning under the aforementioned fairness assumption. In general however they are not winning, unless the initial configuration happens to be in R ∞ .

Conclusion

This chapter proposes a game approach to the controllability problem for conformance testing from timed automata (TA) specifications. It defines a test synthesis method that produces test cases whose aim is to maximize their control upon the implementation under test, while detecting non-conformance.

Test cases are defined as strategies of a game between the tester and the implementation, based on the distance to the satisfaction of a test purpose, both in terms of number of transitions and potential control losses. Fairness assumptions are used to make those strategies winning and are proved sufficient to obtain the exhaustiveness of the test synthesis method, together with soundness, strictness and precision.

A symbolic algorithm is proposed to effectively compute these strategies, paving the way to an implementation.

Finally, an extension is proposed for the strategies in the case where the Inconclusive verdict cannot be avoided. Although in this setting the strategies are not winning, they offer a basis to approach the control problem for test cases and related situations. This chapter opens numerous directions for future work. First, we intend to tackle partial observation in a more complete and practical way. One direction consists in finding weaker conditions under which approximate determinization [START_REF] Bertrand | A game approach to determinize timed automata[END_REF] preserves strong connectivity, a condition for the existence of winning strategies. One could also consider a mixture of our model and the model of [START_REF] David | Testing Real-Time Systems under Uncertainty[END_REF] whose observer predicates are clearly adequate in some contexts. Quantitative aspects could also better meet practical needs. The distance to the goal could also include the time distance or costs of transitions, in particular to avoid restarts when they induce heavy costs but longer and cheaper runs are possible.

The fairness assumption could also be refined. For now it is assumed on both the specification and the implementation. If the implementation does not implement some outputs, a tester could detect it with a bounded fairness assumption [START_REF] Ramangalahy | Strategies for comformance testing[END_REF], adapted to the timed context (after sufficiently many experiments traversing some region all outputs have been observed), thus allowing a stronger conformance relation with equality of output sets. A natural extension could also be to complete the approach in a stochastic view, for example by constructing a probabilistic approximation of the implementation behaviour during the test execution, allowing to use this information to improve efficiency or to gradually adapt our fairness expectation and thus the test strategy. Fundamentally, refining the fairness assumption requires to learn the implementation behaviours and adapt the assumption and strategies to them, raising the question of the interleaving of (formal) testing and (formal) learning.

Notably, stochastic information is required in the general setting of Section 3.6 to prove some form of optimality of the strategies in term of resistance to the Inconclusive verdict.

Chapter 4

TIMED MARKINGS

Go back? No good at all! Go sideways? Impossible! Go forward?

Only thing to do! On we go! -J.R.R. Tolkien "The Hobbit"

Introduction

As discussed in Section 2.2, the state estimation problem (i.e. knowing in which state a system is after a given trace) is a major component of formal verification as well as an important limitation for offline verification methods, that circumvent it using approximations or limitations to sub-classes of models.

For timed automata, this problem is especially challenging as both silent transitions and non-determinism strictly augment the expressivity of the models. In order to tackle it, it is thus interesting to consider intuitions coming from a larger determinizable class of systems, namely automata over timed domains [START_REF] Bouyer | On the determinization of timed systems[END_REF] from which we import the notion of system state as sets of configurations as well as the transitions between such sets.

In this chapter, we develop a technique (first presented in [START_REF] Bouyer | Efficient timed diagnosis using automata with timed domains[END_REF]) for efficiently computing, at runtime, the set of all possible configurations in which a partially-observable one-clock timed automaton can be and discuss its extension to general timed automata with multiple clocks -although that extension is not fully developed.

The main ingredient of our approach is the notion of linear timed sets: intuitively, a timed set is a set of valuations that evolves over time (formally, for a one-clock timed automaton, it is a mapping f : t ∈ R ≥0 → f (t) ⊆ R ≥0 ). Linear timed sets form a restricted class of timed sets, where a valuation v in the set is transformed in v + t by a time elapse of t and then filtered with respect to some minimal constant r.

A linear timed set can be defined as and r i ∈ Q ≥0 for all i (see Figure 4.3 on page 121 for an example of a linear timed set). Linear timed sets are well-suited to represent sets of clock valuations for one-clock timed automata, and compute their evolution along time, which is needed to update state estimations, as we illustrate on the following example.

f : t ∈ R ≥0 → i (X i + t) ∩ [r i ; +∞), with X i ⊆ R
Example 4.1.1. Consider the one-clock timed automaton of Figure 4.1: in this automaton, the transition from 1 to 2 is observable (labelled with b), while the transitions from 0 to 1 are unobservable (labelled with the silent action τ ). Assume that this automaton starts from the initial configuration ( 0 , c = 0). As long as no b-action is observed, we have no way of knowing whether the automaton is in 0 or 1 . Now, assume that a b-action takes place at time 1.3: then we know that one of the transitions from 0 to 1 has occurred; moreover, it cannot be the transition guarded with 3 ≤ c ≤ 4, since only 1.3 time units have elapsed in total. One easily checks that, if we observe a b-transition at time 1.3, then the automaton is in state 2 with c ∈ [1; 1.3].

Figure 4.2 (left) represents this set of valuations.

Similarly, if, starting from the initial configuration, we observe a b-transition at time 4.6, then both transitions from 0 to 1 may have taken place; in this situation, it can be checked that if the top transition has been taken, then the automaton can be in 2 with c ∈ [2.6; 4.6], while if the bottom transition has been taken, the automaton can be in 2 with c ∈ [1; 1.6]. This set is represented on Figure 4.2 (right).

For such an example, our algorithm would first compute the linear timed set

t ∈ R ≥0 → (([-4; -3] ∪ [-2; 0]) + t) ∩ [0; +∞),
corresponding to all clock valuations that can be obtained in 1 after a delay t, as long as no transition has been observed. Since the b-transition can only take place if c ≥ 1 (and does not reset clock c), the linear timed set reached when observing b is the intersection of this linear timed set with [1; +∞), namely: In order to deal with all locations of a timed automaton, we use markings, which associate a set of valuations with each location of the automaton; similarly, linear timed markings associate a linear timed set with each location of the automaton: while sets and linear timed sets represent valuations, markings and linear timed markings represent sets of configurations.

t ∈ R ≥0 → (([-4; -3] ∪ [-2; 0]) + t) ∩ [1; +∞).

This linear timed set is represented at Figure 4.3. It provides a way of representing, as a single object, all possible configurations that can be reached in 2 right after observing transition b at time t, for any

t ∈ R ≥0 . -4 -3 -2 -1 0 1 2 3 4 5 r = 1 X f (0) = ∅ -4 -3 -2 -1 0 1 2 3 4 5 X + 1.3 r = 1 f (1.3) = [1; 1.3] -4 -3 -2 -1 0 1 2 3 4 5 X + 4.6 r = 1 f (4.6) = [1; 1.6] ∪ [2.6; 4.6]
Our algorithm consists in computing linear timed markings representing all configu- rations that can be reached from a given initial configuration after a given sequence of timed observations (alternations of observable transitions and delay transitions possibly including an arbitrary number of unobservable transitions). Given such a linear timed marking M , when an observation a is received at time t, we can easily compute the marking m containing all possible configurations in which the automaton can end up just after an a-transition is taken. From there, our algorithm computes the set of configurations that can be reached when time elapses (this is a linear timed marking), and the sets of configurations that can be reached by following any possible sequence of silent transitions. We prove that this can be effectively computed and finitely represented as a regular timed marking, which we call the τ -closure of m. Finally, in Section 4.5 we start extending our technique to timed automata with an arbitrary number of clocks: we prove in particular that linear timed markings are still sufficient to represent the configurations that can be reached after a given sequence of timed observations. But we have not been able to extend our notion of regular timed markings to a notion that could be used to properly represent and compute all reachable configurations in the n-clock setting; this is part of our future work.

Our work is most tied with [START_REF] Bouyer | On the determinization of timed systems[END_REF] from which automata over timed domains are borrowed, and [START_REF] Tripakis | Fault Diagnosis for Timed Automata[END_REF] for the idea of computing sets of states after some finite execution -although we use a precomputation by opposition to Tripakis's fully online method.

Preliminaries

In this part (focusing on one-clock timed automata), we heavily use sets and intervals of reals with rational bounds, and especially unbounded ones. We first introduce these (and more generally notations for sets of reals with bounds in any subset of R), and then the model of timed automata and related notions.

Sets and intervals of real

Let X and Y be two subsets of R, we define

X + Y = {x + y | x ∈ X, y ∈ Y } and X -Y = {x -y | x ∈ X, y ∈ Y }.
For t ∈ R ≥0 , we write X + t (resp. X -t) as a shorthand for X + {t} (resp. X -{t}), used to shift X forward (resp. backward) by t.

For any subset K of R, we write I K for the set of intervals of R with bounds in K ∪ {-∞, +∞}, and I K ≥0 for the set of intervals with bounds in K ≥0 ∪ {+∞}.

For r ∈ K, we define the following sets of I K :

→ r = [r; +∞) → r = (r; +∞) → r = (-∞; r) → r = (-∞; r].
We write

K ≥0 = { → r, → r | r ∈ K ≥0 }
for the set of upward-closed intervals in I K ; in the sequel, elements of K ≥0 are denoted with r. Similarly,

K ≥0 = { → r, → r | r ∈ K ≥0 } ∪
{R}, and we use notation r for intervals in K ≥0 .

The following results are straightforward, and will be useful to ground intuition and in the sequel: Lemma 4.2.1. Let v ∈ K ≥0 , r and s in K ≥0 , and t in R ≥0 . Then

• if v / ∈ r, then r ⊆ → v; • r ∩ s ∈ { r, s};
• if t intersects both r and s, then r ∩ s ∩ t = ∅.

One-clock timed automata

We consider in this chapter one (and later n-) clock timed automata without invariants (i.e. all invariants are set to true) and equipped with a set of final transitions F that is used to define their language. We thus note a timed automaton A = (L, Σ {τ }, { init },

C, E, F) with F ⊆ L and omitting the invariant function. Σ is an alphabet of observable actions, while τ is an unobservable one. Notice that the automaton is not forced to be deterministic.

As the automata are equipped with a single clock we can partition E by separating E id , the set of non-resetting transitions, i.e., having ∅ as their reset, and E 0 for the complement set of resetting transitions.

As for a transition e = ( , g, a, C , ), the guard g is an interval, it can be written in a unique way as the intersection of an element e ∈ Q ≥0 and an element e ∈ Q ≥0 . In the sequel of this chapter, the guard of a transition e will often be written e ∩ e. Remark 4.2.2. This chapter was originally written with rational guards instead of integer ones. We kept it this way to showcase that the method developed here does not depend on the value of the time unit. For this reason, intervals and (representation of) semantic notions often use Q ≥0 instead of N. This does not impact most of the discussion, except for Section 4.4.3 (where it is discussed).

In this chapter we will consider only (partial) runs (resp. signatures) with a strict alternation between delay transitions and action transitions, but not always ending by Σ. As explained in chapter 3, τ does not appear in traces. We write ( , v) t -→ π ( , v ) when such a partial run ρ exists with dur(ρ) = t and π its partial path (i.e. in E * ).

For any sequence of observable actions λ ∈ Σ * 1 and any t ∈ R ≥0 , we write ( , v) t → λ ( , v ) whenever there exists a partial run π such that λ = act(π) |{τ } and ( , v)

t - → π ( , v ).
The untimed language L(A) of A is the set of words λ ∈ Σ * such that ( init , 0) t → λ ( , v ) for some ∈ F, some valuation v and some delay t ∈ R ≥0 .

In the sequel, we heavily use markings, which map locations of A to sets of clock valuations, thereby representing sets of configurations of A. In our context, a marking of a one-clock timed automaton

A = (L, Σ {τ }, { init }, C, E, F) is a function m : L → 2 R ≥0 representing the set of configurations {( , v) | v ∈ m( )}. We note M = {m | m : L → 2 R ≥0 }
the set of markings of A (we omit to reference A in the M symbol as it will always be clear from context).

For any t ∈ R ≥0 , we define the function O t : M → M by letting, for any m ∈ M and any ∈ L,

O t (m) : ∈ L → {v ∈ R ≥0 | ∃ ∈ L. ∃v ∈ m( ). ( , v) t → ε ( , v )},
representing the configurations reached from m after observing a delay of t time units.

1. Remember that consider automata over Σ {τ } to distinguish observable and unobservable actions.

we cannot know when this occurred. Actually, the bottom transition requires 3 ≤ c ≤ 4, so it cannot have been used during the first 1.3 time units. In the end, it is not hard to check that

O 1.3 (m 0 ) : 0 → {1.3} 1 → [0; 1.3] 2 → ∅
The b-transition can be taken from the configurations in 1 where c ≥ 1. This amounts to applying O b to the marking above, which results in a marking m 1.3,b mapping l 0 and 1 to the empty set, and

2 to [1; 1.3].
Now, what if, instead, we observe b at time 4.6? Computing O 4.6 (m 0 ) can be done as above, but now taking both transitions between 0 and 1 into account. This results in

O 4.6 (m 0 ) : 0 → {4.6} 1 → [0.6; 1.6] ∪ [2.6; 4.6] 2 → ∅
Then taking transition b is similar to the previous situation: it corresponds to applying O b , which results in a marking m 4.6,b mapping 0 and 1 to the empty set, and 2 to [1; 1.6] ∪ [2.6, 4.6]. We recover the markings represented on Figure 4.2.

The two following sections will be devoted to the computation of O t (and O a ). First Section 4.3 introduces the notion of linear timed sets, which we use to represent sets of valuations that evolve over time, and its subclass of regular timed sets which allows a finite representation in the case of one-clock timed automata. In Section 4.4 we lift these notions to sets of configurations and their evolution over time, by the notions of linear timed markings and regular timed markings. We define a τ -closure operator on linear timed markings, and, show that in the case of one-clock timed automata, it can be efficiently computed using regular timed markings, at the expense of some precomputations.

Regular timed sets

In this section, we define the notion of linear timed set to represent sets of clock valuations and their evolution over time. Most important to us is the subclass of regular timed sets, that uses regularity (as we will define just below) to ensure a finite representation, and will be key to efficiency in the computation of closure, as will be seen in the next section.

Regular unions of intervals

Before defining linear and regular timed sets, we introduce our notion of regularity for sets.

Definition 4.3.1. A regular union of intervals is a 4-tuple R = (I, J, p, q) where I and J are finite unions of intervals in I Q ( e.g. intervals of R with bounds in Q ∪ {-∞, +∞}), p ∈ Q ≥0 is the period, and q ∈ N is the offset. It is required that J ⊆ (-p; 0] and

I ⊆ → (-q • p).
The regular union of intervals R = (I, J, p, q) represents the subset of

I Q S(R) = I ∪ +∞ k=q (J -k • p) (where J -k • p is the interval obtained by shifting J by -k • p).
A regular union of interval offers an efficient -or at least finite -representation of a structured infinite set of intervals.

Example 4.3.2. Figure 4.5 shows an example of a regular union of intervals.

There, the period is 1, the offset is 3, and the sets I and J are as displayed on the figure. Proof. We now prove the result for S(R): writing S(R) = I ∪ +∞ k=q J -k • p, thanks to the constraints imposed on I and J, we have

S(R) = ( → (-q • p) \ I) ∪ +∞ k=q ((-p; 0] \ J) -k • p.
For the union we write R = (I, J, p, q) and R = (I , J , p , q ), so that

S(R) = I ∪ +∞ k=q J -k • p S(R ) = I ∪ +∞ k=q J -k • p .
We can write p = a b and p = a b , where a, b, a and b are positive integers. Let

N = min{n ∈ N | n • b • a ≥ q and n • b • a ≥ q }.
Then we can write

S(R) =   I ∪ N •b•a -1 k=q J -k • p   ∪ +∞ k=N     b•a -1 r=0 J -r • p   -k • a • a   .
and

S(R ) =   I ∪ N •b •a-1 k=q J -k • p   ∪ +∞ k=N     b •a-1 r=0 J -r • p   -k • a • a   . Since J ⊆ (-p; 0], we have b•a -1 r=0 J -r • p ⊆ (-b • a • p; 0] = (a • a ; 0]
, and similarly for J . Taking the union of the equalities above, we get an expression of S(R) ∪ S(R ) under the form I ∪ +∞ k=N (J -k • (a • a )), which proves that S(R) ∪ S(R ) can be represented as a regular union of intervals.

The proof for S(R)+S(R ) follows similar ideas: as for S(R)∪S(R ), we first rewrite S(R) and S(R ) in such a way that they have the same period. Hence we assume w.l.o.g. R = (I, J, p, q) and R = (I , J , p, q ). Since I and J are finite unions of intervals, we can write

S(R) = k i I k i ∪ k j ( +∞ k=q J k j -kp)
where I k i and J k j are intervals. Similarly for S(R ). This way, we can simply consider the following cases, and apply our previous result for union to prove that we end up with a regular union of intervals:

• I k i + I k i ; • I k i + ( +∞ k=q J k j -kp); • ( +∞ k=q J k j -kp) + ( +∞ k=q J k j -kp).
The first case is trivial. The second and third cases are easy to handle.

Notice that a regular union of intervals is finitely representable.

Linear and regular timed sets

We introduce linear timed sets as a way to represent sets of clock valuations (and eventually markings), and how they evolve over time. Informally, a linear timed set represents a mapping from the non-negative reals to the set of subsets of R ≥0 . We begin with the definition of atomic timed sets, which form a special case.

Definition 4.3.5. An atomic timed set is a pair T = (X; r) where X ⊆ R and r ∈ Q ≥0 . With such a pair T = (X; r), we associate a mapping f T : R ≥0 → 2 R ≥0 defined as f T (t) = (X + t) ∩ r. The set f T (t) represents the actual valuations after t time units. We call the second component r a filter.

A linear timed set T is a countable set {T k | k ∈ K} (sometimes also denoted with k∈K T k ) of atomic timed sets. With such a linear timed set, we again associate a mapping f T : R ≥0 → 2 R ≥0 defined as f T (t) = k∈K f T k (t). A linear timed set is finite when K is. We write T (R) for the set of linear timed sets of R.

For a timed set T after a delay t, we will often call actual valuations the elements of f T (t) = (X + t) ∩ r by contrast to potential valuations i.e. elements of (X + t) \ r that are not valuations for this particular delay, but model valuations that will appear for greater delays.

Remark 4.3.6. Notice that potential and actual valuations heavily depend on the delay t as a potential valuation for a given valuation is an actual valuation for a greater delay.

Example 4.3.7. Notice that those sets correspond to the markings reached in the situations of Example 4.1.1 (see Figure 4.2). Actually, T can be used to represent all clock valuations that may be reached in state 2 in the automaton depicted on Figure 4.1 (starting from the initial configuration ( init , 0)), depending on the date at which b is observed.

Given two linear timed sets T and T , we write T T whenever f T (t) ⊆ f T (t) for all t ∈ R ≥0 . This is a pre-order relation; it is not anti-symmetric as for instance ({1}; → 0) ({1}; → 1) and ({1}; → 1) ({1}; → 0). We write T ≡ T whenever T T and T T .

Clearly, cycles in timed automata may generate linear timed sets where the set X is infinite. Think of a self-loop silently resetting the clock whenever it reaches 1: the resulting linear timed set would be (-N; → 0). We will prove that for keeping track of all the configurations of any one-clock timed automaton, it is always sufficient to use finite unions of atomic timed sets in which the first component has a simple shape, namely that of a regular union of intervals.

Definition 4.3.8. A regular timed set is a finite timed set T = {(X k ; r k ) | k ∈ K} such that for all k ∈ K, the set X k ⊆ R is S(R k ) the image by S of a regular union of intervals R k = (I k , J k , p k , q k ).
This defines an adequate structure for representing and manipulating sets of configurations of one-clock timed automata and their evolution over time. In the sequel, we extend linear timed sets into linear timed markings, explain how to compute them, and show that regular timed markings are (necessary and) sufficient for representing all reachable configurations of partially-observable one-clock timed automata.

Closure under delay and silent transitions

In this section, we fix a one-clock timed automaton A = (L, Σ {τ }, { init }, C, E, F), with a unique silent action τ and aim at computing the functions O a for any a ∈ Σ and O t for any t ∈ R ≥0 . Computing O a (m) for a ∈ Σ is not very involved: for a given location ∈ L, for each location ∈ L and each transition e labelled with a with source and target , it suffices to intersect m( ) with the guard e ∩ e, and add the resulting interval (or the singleton {0} if the intersection with the guard is non-empty and e is a resetting transition) to O a (m)( ), i. .

From now on, we only focus on computing O t , for t ∈ R ≥0 . For this, it is sufficient to only consider silent transitions of A: we let U = U 0 U id be the subset of E containing exactly the transitions labelled with τ , partitioned into those transitions that reset the clock (in U 0 ), and those that do not (in U id ). We write A τ for the restriction of A to silent transitions, and only consider that automaton in the sequel. In the following, we first describe the effect of (silent) transitions on markings and use it to define the τ -closure in Section 4.4.1 and then propose an operator to explicitly compute this closure in Section 4.4.2, using linear timed sets. Finally we prove that this computation and its result can be finitely represented and effectively computed with regular timed sets in Section 4.4.4. This process is summarized in Figure 4.6.

(M ) : U * → (L → (R ≥0 → 2 R ≥0 )) T = (X, r) R = (I, J, p, q) f T S(R)

Linear timed markings and their τ -closure

We use markings to represent sets of configurations; in order to compute O t , we need to represent the evolution of these sets over time. For this, we introduce timed markings. A timed marking is a mapping M : L → (R ≥0 → 2 R ≥0 ). For any delay t ∈ R, we may (abusively) write M (t) for the marking represented by M after delay t (so that for any ∈ L and any d ∈ R ≥0 , both notations M (t)( ) and M ( )(t) represent the same subset of valuations in R ≥0 ).

For any ∈ L and any t ∈ R ≥0 , M ( )(t) is intended to represent all clock valuations that can be obtained in after a delay of t time units from the marking M (0).

A special case of timed marking are those timed markings M that can be defined using linear timed sets, i.e., for any , M ( ) is a mapping f T for some linear timed set T ; timed markings of this kind will be called linear timed markings in the sequel. Atomic (resp. finite, regular) timed markings are linear timed markings whose values can be defined using atomic (resp. finite, regular) timed sets (we may omit to mention linearity in these cases to alleviate notations). The structure of these different classes of timed markings can be found in Figure 4.7. As we prove below, regular timed markings are expressive enough to represent how markings evolve over time in one-clock timed automata. Definition 4.4.1. We define the union of timed markings M 1 and M 2 as the timed marking such that Two timed markings are said equivalent when for all locations and delays t, M 1 ( )(t) = M 2 ( )(t). We write M 1 ≡ M 2 when this is the case.

M 1 ∪ M 2 ( )(t) = M 1 ( )(t) ∪ M 2 ( )(

t). The intersection of two timed markings is

Proposition 4.4.2. The union of two linear (resp. finite, regular) timed markings is a linear (resp. finite, regular) timed marking.

Proof. This can be seen directly by the stability of timed sets and regular unions of intervals by union, which comes respectively by definition and Prop. 4.3.4.

As we prove below, regular timed markings are expressive enough to represent how markings evolve over time in one-clock timed automata.

We use (linear) timed markings to dispose of a representation in the form of linear timed sets, that can store both actual valuations, and potential valuations that do not exist yet but will exist after some time elapses, as displayed for example in Figure 4.3. This representation has the advantage over markings to be time-independent as a single timed marking represents the set of valuations for all possible delays.

With any marking m, we associate a linear timed marking, which we write -→ m, defined as -→ m( )(t) = {v + t | v ∈ m( )}. This timed marking is linear since it can be defined e.g. as -→ m( ) = f (m( ); → 0) . It can be used to represent all clock valuations that can be reached from marking m after any delay t ∈ R ≥0 , without taking any transition.

To go further, we define the effect of a partial path π ∈ U * of silent transitions on a marking m as the following timed marking m π associating to each delay the marking of reachable configurations:

m π : → t → {v ∈ R ≥0 | ∃ ∈ L.∃v ∈ m( ). ( , v) t - → π ( , v )}.
The set m π ( )(t) corresponds to all configurations reachable in after reading π from a configuration in m with a delay of exactly t time units. By definition of the transition relation → π , for m π (t) to be non-empty, π must be a sequence of consecutive transitions. Notice that m ε = -→ m as one would expect 2 (and hence is linear). Going further, we can define the τ -closure of m as the timed marking m τ such that m τ ( )(t) = π∈U * m π ( )(t). By definition of O t , for any delay t ∈ R ≥0 , we have O t (m) = m τ (t) for any marking m.

This formalism is easily extended from markings to timed markings by taking for a timed marking M and a partial path of silent transitions π ∈ U * :

M π : → t → {v ∈ R ≥0 | ∃ ∈ L.∃t 0 ≤ t. ∃v ∈ M ( )(t 0 ). ( , v) t-t 0 --→ π ( , v )} and defining M τ (t)( ) = π∈U * M π (t)( ).
Remark 4.4.3. Notice that the definition of M π differs from the one of m π only by the addition of an initial delay t 0 ; this takes into account the fact that some potential configurations may become actual in M . By seeing M (t 0 ) as a marking, we have

M π : → t → t 0 ≤t M (t 0 ) π ( )(t -t 0 ).
The following lemmas prove that this extension is self consistent and coherent with what was defined on markings. First, the effect of the empty sequence on a linear timed marking leaves it unchanged: Lemma 4.4.4. For any linear timed marking M , it holds M ε ≡ M .

Proof. Since M is linear, for any , M ( ) is a linear timed set, so that M ( )(t) = k∈N (X k + t) ∩ r k . On the other hand,

M ε ( )(t) = {v ∈ R ≥0 | ∃ ∈ L. ∃t 0 ≤ t.∃v ∈ M ( )(t 0 ). ( , v) t-t 0 --→ ε ( , v )} = {v ∈ R ≥0 | ∃t 0 ≤ t.∃v ∈ M ( )(t 0 ). ( , v) t-t 0 --→ ε ( , v )} = {v ∈ R ≥0 | ∃t 0 ≤ t.∃v ∈ M ( )(t 0 ). v = v + (t -t 0 )} = {v ∈ R ≥0 | ∃t 0 ≤ t. ∃k ∈ N.∃v ∈ (X k + t 0 ) ∩ r k . v = v + (t -t 0 )} = {v ∈ R ≥0 | ∃t 0 ≤ t. ∃k ∈ N.v ∈ (X k + t) ∩ r k } = k∈N (X k + t) ∩ r k .
2. Remember that in m ε , ε corresponds to the empty sequence of transitions.

Second, applying a partial path of silent transitions to a marking or to its corresponding linear timed marking yields the same result: Lemma 4.4.5. For any marking m and any partial path π ∈ U * , it holds ( -→ m) π ≡ m π .

Proof. For any t ∈ R ≥0 and ∈ L, we have

( - → m) π (t)( ) = {v ∈ R ≥0 | ∃ ∈ L. ∃t 0 ≤ t.∃v ∈ - → m(t 0 )( ).( , v ) t-t 0 --→ π ( , v )} = {v ∈ R ≥0 | ∃ ∈ L. ∃t 0 ≤ t.∃v ∈ m( ).( , v) t 0 -→ ε ( , v ) t-t 0 --→ π ( , v )}
On the other hand,

m π (t)( ) = {v ∈ R ≥0 | ∃ ∈ L. ∃v ∈ m( ).( , v) t - → π ( , v )}. It follows that any v ∈ ( - → m) π (t)( ) is in m π (t)( ), since ( , v) t 0 -→ ε ( , v ) t-t 0 --→ π ( , v ) implies ( , v) t - → π ( , v ). Conversely, any v ∈ m π (t)( ) is in ( - → m) π (t)( ), since if ( , v) t - → π ( , v ), then taking t 0 = 0, we have ( , v) t 0 -→ ε ( , v) t-t 0 --→ π ( , v ).
This result can be generalized in the following way, linking even more the operations m π (defined on markings) and M π (defined on linear timed markings): Corollary 4.4.6. For any marking m and any two silent partial paths π 1 and π 2 in U * , it holds (m π 1 ) π 2 ≡ m π 1 .π 2 . Proof. For any m, π 1 and π 2 :

(m π 1 ) π 2 = (( - → m) π 1 ) π 2 (by lemma 4.4.5) = ( - → m) π 1 .π 2 (by definition) = m π 1 .π 2 (by lemma 4.4.5)
Finally, we formally define what we will consider as τ -closures in the sequel:

Definition 4.4.7. Let M be a timed marking. A timed marking N is a τ -closure of M if N ≡ M τ . The timed marking M is said τ -closed if it is a τ -closure of itself.
Our aim in this section is to compute (a finite representation of) a τ -closure of any given initial marking (defined using regular unions of intervals). . Write M for this timed marking; M is not closed under silent-transitions, as for instance configuration ( 1 , 0) is reachable; however, this configuration cannot be reached after any delay: it is only reachable after delay 0, or after a delay larger than or equal to 2 time units. In the end, it can be checked that a τ -closed timed marking for this automaton is M τ ( 0 ) = M ( 0 ), and

0 1 c < 1; τ c ≥ 2; τ ; c := 0 1 -3 -2 -1 0 1 2 r = 0 0 -3 -2 -1 0 1 2 r = 0
M τ ( 1 ) = ((-∞; -2] ∪ [0; 0]; → 0).

Computing τ -closures

In this subsection, we show how to compute τ -closures. For this, we rely on linear timed sets as a mean to represent M ( ) for the timed markings M and locations encountered during the operation.

We will define an operator computing the effect of a silent transition on a linear timed set, and then extend it to sequences and languages of silent transitions on one hand and to linear timed markings on the other hand. We will also prove that this operator corresponds to the semantic operations on linear timed markings M π and M τ used to compute the τ -closure. The approach we have followed so far is summarized in Figure 4.9, with links to the different sections.

Core to the definition of the operator is the gauge, that computes the effect of a resetting transition on a set of (potential) valuations. markings: Definition 4.4.9. Let X and Y be two subsets of R. We define their gauge as the set

m : L → 2 R ≥0 linear timed sets: T = k∈K (X k ; r k ) linear timed markings: M ( ) = f T timed markings: M : L → (R ≥0 → 2 R ≥0 ) f T - → m, m π M π , M τ
X Y = (X -Y ) ∩ R ≤0
This operation can be characterised in the following ways: Proposition 4.4.10. Let X and Y be two subsets of R. Then

X Y = {-t | t ∈ R ≥0 ∧ (Y -t) ∩ X = ∅} = {-t | t ∈ R ≥0 ∧ (X + t) ∩ Y = ∅} Proof. We observe that X Y = {x -y | x ∈ X, y ∈ Y s.t. x ≤ y} = {-t | t ∈ R ≥0 ∧ ∃t ∈ X. ∃y ∈ Y. t = y -x} = {-t | t ∈ R ≥0 ∧ ∃y ∈ Y. y -t ∈ X} = {-t | t ∈ R ≥0 ∧ (Y -t) ∩ X = ∅}.
The other equality is proven similarly. These characterizations can be read as "X Y is the inverse of the set of delays after which valuations in X satisfy the guard Y ". It corresponds to the intuition that after waiting such a delay, the transition can be taken and as it is a resetting one, the clock value becomes 0.

Remark 4.4.11. These characterizations are specific to the one clock setting. Indeed, X Y is by definition a set of valuations and only when there is a unique clock do valuations and delays correspond.

0 1 g = 1 ≤ c ≤ 2, τ c := 0 -5 -4 -3 -2 -1 0 1 2 X g -5 -4 -3 -2 -1 0 1 2 X g
In general, X Y is the set of valuations that will be in (X + tY ) [{c}←0] ( i.e. {0}) after t time units. This is the definition used for n-clocks, and it is proven (in Section 4.5) that it corresponds to the current one.

Example 4.4.12. Consider the simple transition in Figure 4.10, with guard g = [1, 2]. When entering in 0 with a set of potential configurations corresponding to X = {0} ∪ [-3, -2] displayed on the bottom of the figure, the result of X g is the set of configurations [-5, -3] ∪[-2, -1]. To see this, imagine a right shift of X, and consider the delays t during which the two intervals composing X meet g, i.e. [1, 2] and [3,5]. Each such t is a date at which a configuration ( 1 , c = 0) can be spawned (because the transition resets the clock); therefore, -t is a potential valuation in 1 . In the end, (X g; → 0) = ([-5, -3]∪[-2, -1]; → 0) is the linear timed set of all potential valuations in 1 in this situation.

In the following, we use the gauge to define the operator on linear timed sets and extend it to linear timed markings, while proving that it corresponds to the semantic operations M π and M τ defined on general timed markings. Proposition 4.4.13. The following simple statements will be useful in the sequel:

• If X ≤ Y (that is, if for any x ∈ X and any y ∈ Y , it holds x ≤ y; this is the case in particular if X ⊆ R ≤0 and Y ⊆ R ≥0 ), then X Y = X -Y ;
• if X > Y ( i.e., if for any x ∈ X and any y ∈ Y , it holds x > y), then X Y = ∅;

• If X and Y are two intervals, then X Y is an interval;

• If R is a regular union of intervals and Y is an interval, then S(R) Y can be represented as a regular union of intervals noted R Y ;

• If Y ⊆ R ≥0 , then (X Y ) Y = (X Y ) -Y .
Proof. The first two claims are trivial from the definition of X Y . The third claim follows from the fact that X -Y is an interval if X and Y are. The fourth claim follows from Prop. 4.3.4. Finally, the last claim is a consequence of the first one (because X Y ⊆ R ≤0 ).

We now define a mapping : T (R) × U * → T (R), intended to represent the linear timed set that is reached by performing sequences of silent transitions from some given linear timed set. Intuitively, we want to replace semantic operations, such as M → M π : M × U * → M by computations based on their representations (i.e. linear timed sets). This will serve to compute a representation of the closure of any linear timed set.

We first consider atomic timed sets, and the application of a single silent transition.

Definition 4.4.14. For an atomic timed set (X; r) of T (R) and a transition e = ( , e ∩ e, τ, C , ) of U :

((X; r), e) =                        (∅; → 0) if r ∩ e = ∅ (X ∩ e; r ∩ e) if r ∩ e = ∅ and C = ∅ (X ( r ∩ e ∩ e); → 0) if r ∩ e = ∅ and C = C
The intuition behind these three cases is as follows (see also Figure 4.11 for a graphical illustration, and the proof of Lemma 4.4.17 for the formal arguments):

• if r ∩ e = ∅, then the transition cannot be taken: it means that the upper bound e of the guard is smaller than the smallest clock value in r that can be reached;

• if r ∩ e = ∅, then there is some range of delays t such that f (X; r) (t) ∩ e ∩ e is not empty and the transition can be taken. Furthermore, the transition can be taken from any potential valuation in X ∩ e: for valuations in this set the transition will be possible after some delay, while valuations out of e will never go back in this set.

Then after the transition:

if the transition does not reset the clock, then the value of the clock is not changed. Then X ∩ e can represent the set of potential valuations. In order for a valuation to be actually reachable, it needs to be actual in (X; r) (i.e. in r) and if the transition resets the clock, then the set of reachable values for the clock in after delay t 1 corresponds to the set of delays that can be spent in (after the clock reset), i.e., the difference between t 1 and the delays t that can be spent in before taking the transition. Those delays that can be spent in before taking the transition can be seen to precisely correspond to X ( r ∩ e ∩ e).

c ∈ [a, b] C • if r ∩ → b = ∅: (X; r) in a b • if r ∩ → b = ∅: (X; r) in a b (∅; → 0) in • if C = ∅: (X∩ → b; → a∩ r) in • if C = C: ((X-r∩[a, b])∩R ≤0 ; → 0) in
Notice that r ∩ e ∩ e corresponds to the set of actual valuations of f (X; r) (t) satisfying the guard.

Example 4.4.15. Consider a linear timed set T = (X = [-3, -2] ∪ {0}; → n) with n ∈ N representing the configurations reachable in the state 0 of the automaton in Figure 4.10. If n = 3, then for all delays t, f T (t) ≥ 3 and it is clear that the transition cannot be taken. Hence, as expected, (T, ( 0 ,

→ 1 ∩ → 2, τ, {c}, 1 )) = ∅ (

this corresponds to the first case of the definition).

In contrast, if n = 1 then all configurations satisfying the guard can be reached after a fitting delay, and (T, ( 0 ,

→ 1∩ → 2, τ, {c}, 1 )) = (X g, → 0) with X g = [-5, -3]∪[-2, - 1
] as displayed on the figure. Notice that the new filter → 0 corresponds to the intuition: all positive clock values can be reached (after a certain delay), as the transition can be taken and it resets the clock. Definition 4.4.14 (cont.). We extend to partial paths inductively as follows:

((X; r), ε) = (X; r)
and, for π ∈ U * and e ∈ U ,

((X; r), π • e) =      ( ((X; r), π), e) if tgt(π) = src(e) (∅; → 0) otherwise.
Example 4.4.16. Consider a linear timed set ([-3, -1]; → 2), intended to represent the configurations that are reachable in a state : then after 3 time units, only configuration ( , 2) can be observed, and after 4.5 time units, the set of reachable configurations in is

{( , c) | 2 ≤ c ≤ 3.5}.

Now, assume that there is a resetting transition from to , guarded with c ∈ [1; 4]. The set of configurations reachable in (originating from the above linear timed set and transition) then is

([-3; -1] ( → 2 ∩ → 1 ∩ → 4); → 0) = ([-3; -1] [2; 4]; → 0) = ([-7; -3]; → 0).
In particular, assuming we depart from the linear timed set ([-3; -1]; → 2) in , the transition can only take place between 3 and 7 time units. Definition 4.4.14 (cont.). We extend this definition to linear timed sets as follows:

({T k | k ∈ K}, π) = { (T k , π) | k ∈ K}
The operator is now extended to all linear timed sets and sequences of transitions. We now prove that it indeed corresponds to the effect of taking transitions from a given timed set. For this we do not reason directly on the linear timed sets but on their associated functions f T . The following lemma states that (informally) (T, π) corresponds to the effect of following π from T . It is the technical core of the section, linking the semantics and the operator. Lemma 4.4.17. Let T be a linear timed set and π ∈ U * . Then for any t ∈ R ≥0 and

any v ∈ R ≥0 , v ∈ f (T,π) (t) ⇔ ∃0 ≤ t 0 ≤ t. ∃v ∈ f T (t 0 ).(src(π), v ) t-t 0 --→ π (tgt(π), v).
Proof. We carry the proof for the case where T is an atomic timed set. The extension to unions of atomic timed sets is straightforward.

The proof is in two parts: we begin with proving the result for π = ε, then for a single transition, and finally proceed by induction to prove the full result.

If π = ε, then v ∈ f (T, π)(t) is equivalent to v ∈ f T (t), which entails the right-hand- side of the equivalence for t 0 = t. Conversely, if v = v + (t -t 0 ) for some v ∈ f T (t 0 ), then, writing T = (X; r), we have v ∈ (X + t 0 ) ∩ r, so that v ∈ (X + t) ∩ r, i.e. v ∈ f T (t).
Now, assume that π is a single transition e = ( , e ∩ e, τ, C , ), for which we assume (w.l.o.g.) that e ∩ e = ∅. In case T is empty (i.e. X = ∅), then also (T, e) is empty, and the result holds. We now assume that T = (X; r) is not empty (i.e. X = ∅), and consider three cases, corresponding to the three cases of the definition of (T, e):

• if r ∩ e = ∅, then (T, e) = (∅; → 0). On the other hand, for any t 0 and any v ∈ f T (t 0 ), it holds v ∈ r, so that v / ∈ e, and the transition cannot be taken from that valuation (even after some delay). Hence both sides of the equivalence evaluate to false, and the equivalence holds.

• now assume that r ∩ e = ∅, and consider the case where e does not reset the clock.

Write p = r ∩ e, and pick v ∈ f (T,π) (t) = f (X∩ e; p) (t) (assuming one exists). Then -t; v). Moreover, the fact that both r ∩ e and e ∩ e are non-empty implies that also p ∩ e = r ∩ e ∩ e is non-empty.

v ∈ (X + t) ∩ ( e + t) ∩ p; this implies p ∩ e ∩ [v -t; v] = ∅: indeed, -if v ∈ e or v -t ∈ p, then the result is trivial; -otherwise, v / ∈ e implies e ⊆ → v, and v -t / ∈ p implies p ⊆ → v -t, so that p ∩ e ⊆ (v
Then for any v in that set p∩ e∩[v -t; v], letting t 0 = v -(v -t), we have v ∈ X +t 0 . In the end, v ∈ f T (t 0 ), and v ∈ e ∩ e, so that (src(e), v )

t-t 0 --→ e (tgt(e), v). Conversely, if t 0 ∈ [0; t] and v ∈ f T (t 0 ) exist such that (src(π), v ) t-t 0 --→ π (tgt(π), v), then v ∈ X + t 0 ∩
r, and for some t 1 ≤ t -t 0 , v + t 1 ∈ e ∩ e. Then v = v + t -t 0 since t does not reset the clock; also, since v ∈ X + t 0 ∩ r, we have v ∈ (X + t) ∩ r; finally, from v + t 1 ∈ e ∩ e, we get v ∈ e + (t -t 0 -t 1 ) ⊆ e and v ∈ e + (t -t 0 -t 1 ) ⊆ e + t. In the end, v ∈ ((X + e) + t) ∩ ( r ∩ e) = f (T,e) (t).

• we finally consider the case where r ∩ e = ∅ and e resets the clock. In this case, v ∈ f (T,π) (t) means that v ∈ → 0 and v -t ∈ X ( p ∩ e), which rewrites as 0 ≤ v ≤ t and (X + t -v) ∩ ( p ∩ e) = ∅ (by Prop. 4.4.13). Let t 0 = t -v. The property above entails that 0 ≤ t 0 ≤ t, and that there exists some v ∈ (E + t 0 ) ∩ ( p ∩ e), so that 0 ≤ t 0 ≤ t, v ∈ f T (t 0 ) and (src(e), v ) t-t 0 --→ e (tgt(e), v). Conversely, if those conditions hold, then for some 0 ≤ t 1 ≤ t -t 0 , we have v + t 1 ∈ e ∩ e, and v = t -(t 0 + t 1 ) ≥ 0 (remember that e resets the clock). Then v + t 1 ∈ X + (t 0 + t 1 ) ∩ r ∩ e ∩ e, so that

-(t 0 + t 1 ) ∈ X ( p ∩ e), and finally v ∈ f (T,π)(t) .
We now consider the case of π • e, assuming that the result holds for π ∈ U + . In case tgt(π) = src(e), the result is trivial. Otherwise, first assume that v ∈ ( (T, π • e))(t), and let T = (T, π). Then v ∈ f (T ,e) (t), thus there exist 0 ≤ t 0 ≤ t and v ∈ f T (t 0 ) s.t. (src(e), v)

t-t 0 --→ e (tgt(e), v ). Since v ∈ f T (t 0 ), there must exist 0 ≤ t 1 ≤ t 0 and v ∈ f T (t 1 ) such that (src(π), v ) t 0 -t 1 ---→ π (tgt(π), v). We thus have found 0 ≤ t 1 ≤ t such that (src(π), v ) t-t 1 --→ π•e (tgt(e), v ).
Conversely, if (src(π), v ) t-t 1 --→ π•e (tgt(e), v ) for some 0 ≤ t 1 ≤ t and v ∈ f T (t 1 ), then we have (src(π), v )

t 0 -t 1 ---→ π (tgt(π), v) t-t 0
--→ e (tgt(e), v ) for some t 0 ∈ [t 1 ; t] and some v. In that case, we prove that v ∈ f (T,π) (t 0 ): indeed, we have t 1 ∈ [0; t 0 ], and v ∈ f T (t 1 ) such that (src(π), v ) t 0 -t 1 ---→ π (tgt(π), v), which by induction hypothesis entails v ∈ f (T,π) (t 0 ). Thus we have 0 ≤ t 0 ≤ d and v ∈ f T (t 0 ), where T = (T, π), such that (tgt(π), v) t-t 0 --→ e (tgt(e), v ), which means v ∈ f (T ,e) (t), and concludes the proof. This results is the first link between and the semantics of timed automata, and the main technical result of the section, the following ones stemming from it. Thanks to this characterization of (T, π), we get: Corollary 4.4.18. For any sequence π of silent transitions, and any two equivalent linear timed sets T and T , the linear timed sets (T, π) and (T , π) are equivalent.

This corollary ensures that we do not depend on the representation of the structure for our results since the mapping defined on linear timed sets preserves the equivalence of markings that they represent.

Finally, we extend to linear timed markings in the expected way: Definition 4.4.14 (cont.). Given a linear timed marking M , and a sequence π of transitions, we let:

(M, π) : → f (T M (src(π)) ,π) if = tgt(π), → f (∅; → 0)
otherwise.

with T M (src(π)) a timed set such that f T = M (src(π)).

Since (T 1 T 2 , π) = (T 1 , π) (T 2 , π), we also have (M 1 M 2 , π) ≡ (M 1 , π) (M 2 , π
) when applied to linear timed markings. Then, we can extend the link between and the semantics to linear timed markings. This is the main result of the section, as it formally establishes the correspondence between the semantic computation of the effect of silent transitions and the operator that we build to compute it. Proof. For t ∈ R ≥0 and ∈ L, we have

M π (t)( ) = {v ∈ R ≥0 | ∃ ∈ L. ∃t 0 ≤ t.∃v ∈ M (t 0 )( ). ( , v ) t-t 0 --→ π ( , v)}. Hence clearly M π ( ) ≡ f (∅; → 0) if = tgt(π). When = tgt(π), we have M π (t)( ) = {v ∈ R ≥0 | ∃t 0 ≤ t.∃v ∈ M (t 0 )(src(π)). (src(π), v ) t-t 0 --→ π ( , v)}.
By Lemma 4.4.17, this corresponds to (M, π)( )(t).

Finally, we extend from sequences of transitions to languages. It follows that the τ -closure of any marking (in particular the initial marking) can be represented as a linear timed marking. However, this linear timed marking is currently defined as an infinite union over all sequences of consecutive silent transitions. We make the computation more effective (and representable) in the Section 4.4.4.

Necessity of regular timed sets

The next subsection will prove that regular timed sets are enough to express (and effectively compute) the set of configurations reached by a one-clock timed automaton. To complete this result, we prove that any regular timed marking can appear in the τ -closure of a one-clock timed automaton.

Remark 4.4.22. In this subsection, we consider timed automata with guards in Q ≥0 . This is necessary to encode the regular timed markings, as their intervals and periods can be rationals.

Conversely, if it is desired to limit the discussion to automata with integer guards, it suffices to consider regular timed markings with integer bounded intervals and integer periods.

Proposition 4.4.23. Given a regular timed set T , there exists a one-clock timed automaton

A such that T represents the τ -closure of A.

Proof. The proof is first made for an atomic regular timed set T = (X; r) defined using a regular union of intervals R = (I, J, p, q). The generalization to finite unions of atomic regular timed sets is then proposed at the end. We prove this result by constructing a timed automaton. For this, we first explain how to encode intervals with bounds in Q ≤0 ∪{-∞}, regular repetitions, finite unions, and how to add a filter r. Then we construct an automaton for T .

• Consider an interval I with bounds in Q ≤0 ∪ {-∞}. In the automaton represented in Figure 4.12, we start from the initial (atomic regular) timed marking M init associating f ({0}; with 1 . Indeed, by definition of , the closure is computed as: • Consider a rational p ∈ Q ≤0 , an integer q, an interval J ∈ (-p; 0] with rational bounds, and the automaton depicted in Figure 4.13. As argued in the previous case, starting from M init , the effect of the transition init → 1 through is represented by (J -p • q; → 0). Then it is easy to see that the τ -closure is represented by ( ∞ k=q J -k • p; → 0) in 1 . Indeed, the effect of one application of the self-loop has the form • {p}, effectively repeating the pattern shifted by -p. The fixpoint then is • The general method to combine a finite number of atomic timed sets representing different behaviours ending in the same configuration is to regroup them in a finite timed set (i.e., a finite collection of atomic timed sets). When all the atomic timed sets share the same constraint r, an equivalent method is to take the explicit union of their first part. As argued in Prop. 4.3.4, when the atomic timed sets are regular, this yields an atomic regular timed set. A simple example of such unions is shown in Figure 4.14. • Adding a stricter filter r to a linear timed set through a transition is straightforward: it is the exact effect of a non-resetting transition of guard r.

({0} -I ∩ → 0; → 0) = ({0} -I; → 0) = (I; → 0). init 1 -I, τ, {c}
( ∞ k=q J -k •p; → 0) in 1 . init 1 -(J -q.p), τ, {c} p, τ, {c}
Using these components, we can build an automaton using T = (S(R); r) to encode the τ -closure of the initial timed marking. A possible example is depicted in Figure 4.15 for R = ( n I k=1 I k , n J k=1 J k , p, q), where the dotted line is meant to represent the n I and n J transitions. In this automaton, the closure can be represented by (S(R); It is clear to see that for a regular timed set T = {(X k ; r k ) | k ∈ K}, as K is by definition finite, building all components necessary to accept the (X k ; r k ) with the same initial and final location yields a finite timed automaton with 1 clock requiring T to represent its τ -closure.

→ 0) in R and T is f . init 1 n J R f -I n I , τ, {c} -I 1 , τ, {c} -(J 1 -q.p), τ, {c} p, τ, {c} Q ≥0 , τ, ∅ -(J n J -q.p), τ, {c} p, τ, {c} Q ≥0 , τ, ∅ r, τ, ∅
Remark 4.4.24. Notice that we rely on the intervals defining a regular union of intervals having bounds in Q ≤0 , in order to define automaton guards with their additive inverse.

Finite representation of the closure

In Section 4.4.2, we demonstrated how the τ -closure of any linear timed marking (and hence of any marking) can be computed as a linear timed marking using the operator. To complete this result, we prove in this subsection that when starting from regular timed markings, the τ -closure can be computed with regular timed markings. This presents two main interests:

• First, it corresponds to the semantics of timed automata, as the initial timed marking is clearly regular and we have proved that any regular timed set can appear in a τ -closure (Prop. 4.4.23),

• second, linear timed markings rely on linear timed sets, which in general use infinite unions of sets of R. In order to get an effective algorithm, we need a finite representation of sets, which regular unions of intervals provide.

The following theorem is the general result we prove in the following discussion. We call language of transitions of a timed automaton A, noted L E (A), the language obtained by reading the names of the transitions instead of the labels.

Theorem 4.4.25. Consider a timed automaton A and a regular language L ⊆ L E (A) included in its language of transitions. Given a regular timed marking M , the timed marking (M, L) is regular.

Thanks to this theorem and the fact that the timed marking corresponding to the initial valuation is regular, we can deduce the following facts: Corollary 4.4.26. Given a timed automaton A with τ transitions and its silent fragment A τ :

• the initial (timed) marking of A τ is regular and

• the τ -closure of any regular (timed) marking is regular. This result entails that regular timed markings are indeed enough to represent and compute the τ -closures.

The remaining of this subsection is a proof of the previous Theorem 4.4.25. We first prove in the Paragraph 4.4.4 that the linear timed marking (M, L) is a finite timed marking, by discussing the set of possible filters. Then we prove that it is a regular timed marking in Paragraph 4.4.4.

Finiteness

Finiteness is achieved by separating the initial timed marking per location, and then proving that there is only a finite amount of possible filters.

Let M be a regular timed marking: then M can be written as the finite union of atomic regular timed markings M ( ,X, r) , defined as

M ( ,X, r) ( ) = f (X; r) and M ( ,X, r) ( ) = f (∅; → 0)
for all = . Thus (M, L) is the function associating to an input the union of the corresponding outputs of (M ( ,X, r) , L) and it suffices to compute the result for atomic regular timed markings M ( ,X, r) .

We write 1 ((X; r), π) ⊆ R and 2 ((X; r), π) ∈ R ≥0 for the first and second components of ((X; r), π). Notice that 2 ((X; r), π) does not depend on X (so that we may denote it with 2 ( r, π) in the sequel). In particular,

• 2 ((X; r), π) = → 0 if π ∈ U * × U 0 is a sequence of consecutive transitions ending with a resetting transition;

• 2 ((X; r), π) = r ∩ k<n e k if π = e 1 . . . e n ∈ U id
* is a sequence of consecutive non-resetting transitions.

Letting J r = { → 0, r} ∪ { e | e ∈ U id }, it follows that 2 ((X; r), π) ∈ J r for any (X; r) and any π (because by Lemma 4.2.1, r ∩ e is either r or e, for any r and e in R ≥0 ). Hence (M ( ,X, r) ), and thus also (M, L), can be written as a finite union of atomic timed markings as J r is a finite set. By definition, (M, L) is thus a finite timed marking.

Regularity

In the following, we reason on each M ( ,X, r) separately. To prove regularity, we first introduce some more notations and state Lemma 4.4.27, that allows to rewrite using the properties of the gauge. Then we separate (M ( ,X, r) , L) as we did for M by separating first the ending locations and then the ending filters. For this we separate L = , ∈L, r, r ∈J r [L( , )] r r . The proof that (M ( ,X, r) , [L( , )] r r ) is indeed regular is encapsulated in Lemma 4.4.28. First, some more formalism:

• we let G id = { e | e ∈ U id } and G id = { e | e ∈ U id }. We thus have J r = { → 0, r} ∪ G id ;
• for r ∈ R ≥0 and e ∈ U , we write Φ( r, e) for the interval r ∩ e ∩ e;

• we define a mapping J r : U * → N J r ×U 0 that counts the number of occurrences of certain timing constraints at resetting transitions along a path: precisely, it is defined inductively as follows (where represents addition of an element to a multiset):

J r (ε) = {0} J r ×U 0 J r (π • e) = J r (π) {( 2 ( r, π), e)} if e ∈ U 0 J r (π • e) = J r (π) if e ∈ U id .
The idea behind J r is the following: consider a timed set (X; r), and a resetting silent transition guarded with c ∈ e ∩ e. Resetting clock c when it is in r ∩ e ∩ e amounts to substracting to c some value in that interval. The function J r precisely counts the number of occurrences of each of those intervals along a sequence π of silent transitions. Since there is only one clock, the order of the transitions is not important. Lemma 4.4.27 formalizes this intuition: Lemma 4.4.27. Let (X; r) be an atomic timed set with X ⊆ R ≤0 , and π ∈ U * . Then either 1 ((X; r), π) = ∅, or

1 ((X; r), π) = X - J=( y,e)∈J r ×U 0 J r (π)(J) × Φ( y, e) = x - J∈J r ×U 0 J r (π)(J) • y J x ∈ X and y J ∈ Φ(J) for all J ∈ J r × U 0 .
Proof. The result is straightforward for π = . Now, assume the result holds for some π ∈ U * , and consider π = π • e.

Assuming that 1 ((X; r), π) = ∅, we first consider the case where e ∈ U id : then 1 ((X; r), π • e) = 1 ((X; r), π) ∩ e (by Definition 4.4.14 of ). Since 1 ((X; r), π) ⊆ R ≤0 ⊆ e, we have 1 ((X; r), π • e) = 1 ((X; r), π). Since J r (π • e) = J r (π) when e ∈ U id , our result follows.

Now assume e ∈ U 0 : we have

1 ((X; r), π • e) = 1 ((X; r), π) ( 2 ( r, π) ∩ e ∩ e).
Since 1 ((X; r), π) ⊆ R ≤0 and 2 ( r, π) ⊆ R ≥0 , Prop. 4.4.13 entails

1 ((X; r), π • e) = 1 ((X; r), π) -( 2 ( r, π) ∩ e ∩ e).
Then, since Φ( 2 ( r, π), e) = 2 ( r, π)∩ e∩ e, substracting 2 ( r, π)∩ e∩ e precisely corresponds to the effect of adding {( 2 ( r, π), e)} to the multiset J r (π).

Let M ( ,X, r) be an atomic regular timed marking. As regular timed markings are defined using linear timed sets, we extend the notations 1 and 2 to regular timed markings as the functions returning resp. mappings from locations to sets (that we want to define using regular unions of intervals), and elements of R ≥0 , such that for any regular timed marking M , we have (M, π) : → f ( 1 (M,π)( ); 2 (M,π)( )) . For any state of A τ , we let L( , ) be the set of all sequences of consecutive transitions from to in A τ . Then (M ( ,X, r) ) τ = ∈L (M ( ,X, r) , L( , )).

Hence it suffices to prove that (M ( ,X, r) , L( , )) is regular.

For any set L of sequences of consecutive transitions, and for any r and r in R ≥0 , we let L r r = {π ∈ L | r = 2 ( r, π)}. One easily observes that for any r ∈ R ≥0 and any L, it holds L = r ∈J r L r r , so that (M ( ,X, r) , L( , )) can be written as

r ∈J r → f 1 (M ( ,X, r) ,[L( , )] r r )( ); r .
We can thus focus on 1 (M ( ,X, r) , [L( , )] r r ). The following key lemma entails that this set is a regular union of intervals: Lemma 4.4.28. Let M ( ,X, r) be an atomic regular timed marking. Let r and r be two elements of R ≥0 , and L ⊆ L(A τ ) be a regular language. Then 1 (M ( ,X, r) , L r r ) can be defined using a regular union of intervals.

Proof. The proof of this result is in two parts: we first express L r r as the language of a finite automaton, and then-by a tedious proof-express 1 (M ( ,X, r) , L r r ) using regular unions of intervals.

Finite automaton for L r r . We assume that L is accepted by some automaton B = (L, Σ, init , C, U, F) (in particular, L( , ) would be obtained from A τ by imposing an initial state and a single accepting state ). In order to derive a finite automaton accepting L r r , we have to keep track of the value of 2 along runs. For this, we take the product of B with J r : we define the automaton This automaton accepts words in ( R ≥0 × U × R ≥0 ) * . For any r ∈ R ≥0 , we define

B r r = (L × J r , R ≥0 × U × R ≥0 , ( init , r), C, U , F × { r }) over the extended alphabet R ≥0 × U × R ≥0 (
κ r : U → ( R ≥0 × U × R ≥0 ) as κ r (e) =      ( r, e, r ∩ e) if e ∈ U id ( r, e, → 0) if e ∈ U 0
and extend this to U * as κ r (e • π) = κ r (e) • κ r (π) where r is such that κ r (e) = ( r, e, r ). Then:

Lemma 4.4.29. For any r and r , we have κ r (L(B) r r ) = L(B r r ).

Proof. Pick a finite word π = ( r k , e k , r k ) 0≤k≤n in ( R ≥0 × U × R ≥0 ) * accepted by B r r . By construction of B r r , it holds r 0 = r, and r k = κ r k (e k ) for all k. Hence π ∈ κ r (L(B) r r ). Conversely, pick a partial path π = (e k ) 0≤k≤n in L(B) r r , and consider the word π = κ r (π) = ( r k , e k , r k ) 0≤k≤n . Again by construction of B r r , any accepting run of B on π can be transformed into an accepting run of B r r on π, which entails our result. Writing P U : R ≥0 × U × R ≥0 → U for the projection on the second element of this alphabet (and extending it to sequences in the natural way), we get P U (L(B r r )) = L(B) r r .

Defining 1 (M ( ,X, r) , [L( , )] r r ) as a regular union of intervals. We now focus on 1 (M ( ,X, r) , [L( , )] r r ): this function maps to 1 ((X, r), [L( , )] r r ) (which we write η in the sequel for the sake of readability), and any other state to the empty timed set. To define η as a regular timed set, we progress in three steps: first, we decompose [L( , )] r r according to the presence of a reset in each sequence of transitions, then we quickly conclude for non-resetting sequences. Finally we discuss the resetting ones.

• Separating resetting and non-resetting transitions. For any g ∈ J r = { → 0, r} ∪ G id and any g ∈ G id , we define

W id g, g = {π = ( g 1 , e 1 , g 1 ) • • • ( g n , e n , g n ) |
g n = g and min 1≤k≤n e k = g and e k ∈ U id for all 1 ≤ k ≤ n}.

We decompose [L( , )] r r as the disjoint union of

g ∈ G id [L( , )] r r ∩ P U (W id r , g )
(all runs in [L( , )] r r that do not contain any resetting transition) and

g ∈ G id ( g,e)∈J r ×U 0 [L( , )] r r ∩ P U W id g, g × {( g, e, → 0)} × (J r × U × J r ) *
where the right-hand side of the intersection contains (the projection of) all paths containing at least one resetting transition labelled ( g, e, → 0). We write Z for the set

W id g, g × {( g, e, → 0)} × (J r × U × J r ) * .
Using the decomposition above, we get

η = g ∈ G id π∈[L( , )] r r ∩P U (W id r , g ) 1 ((X, r), π) ∪ g ∈ G id ( g,e)∈J r ×U 0 π∈[L( , )] r r ∩P U (Z)
1 ((X, r), π) .

• Conclusion for non-resetting transitions. In the first part, any path π ∈ [L( , )] r r ∩ P U (W id r , g ) contains only non-resetting transitions, so that we have 1 ((X, r), π) = X ∩ g . Hence the first part is a finite union of regular intervals (because M ( ,X, r) is a regular timed marking). This can be represented by a regular union of intervals of the form (I, ∅, 0, 0).

• Discussion on the non-resetting transitions. The non-resetting transitions are handled in the following way. Lemma 4.4.27 and the regularity of L(l, l ) r r , that we obtained by constructing a finite timed automaton recognising it, are used to reformulate the expression enough to use Parikh's theorem. Then, using the new form arising from the theorem, a final discussion is conducted to separate the case where the closure uses a finite union of sets, and the case where there is an infinite (but regular) union.

For the second part of η, i.e.

g ∈ G id ( g,e)∈J r ×U 0 π∈[L( , )] r r ∩P U (Z) 1 ((X, r), π)
decomposing π ∈ P U (Z) as π 1 • e • π 2 according to the above unions, we have Since π 1 only contains non-resetting transitions, we have 1 ((X, r), π 1 ) = X ∩ g , and since π 1 ∈ W id g, g we have 2 ((X, r), π 1 ) = g. Then, since e is a resetting transition, we get

1 ((X, r), π) = 1 (((X ∩ g ) Φ( g, e), → 0), π 2 ).
We then have

π∈[L( , )] r r ∩P U (Z) 1 ((X, r), π) = 1 ((X ∩ g ) Φ( g, e), → 0), Q where Q = P U (W id g, g × {( g, e, → 0)})\[L( , )] r r is the left-quotient of [L( , )] r r by P U (W id g, g × {( g, e, → 0)}), i.e.
the set of all finite words β ∈ U * for which there exists a finite word

α in P U (W id g, g × {( g, e, → 0)}) such that α • β is in [L( , )] r r .
Hence it remains to prove that for all g ∈ G id and all ( g, e) ∈ J r × U 0 , the set

η = 1 ((X ∩ g ) Φ( g, e), → 0), Q ,
is a regular union of intervals.

Write X e, g, g = (X ∩ g ) Φ( g, e) ⊆ R ≤0 . From Lemma 4.4.27, we derive, for any π 2 ∈ Q:

1 ((X e, g, g , → 0), π 2 ) = X e, g, g - J∈J → 0 ×U 0 J → 0 (π 2 )(J) × Φ(J). Hence η = π 2 ∈Q X e, g, g - J∈J → 0 ×U 0 J → 0 (π 2 )(J) × Φ(J)
= X e, g, g -

π 2 ∈Q J∈J → 0 ×U 0 J → 0 (π 2 )(J) × Φ(J)
For any π 2 ∈ Q, we write p(π 2 ) for the Parikh vector of κ → 0 (π 2 ), i.e. the function mapping each letter λ of J → 0 ×U ×J → 0 to the number of occurrences of λ along κ → 0 (π 2 ). We write p(Q) for the set of all such vectors. Then for all ( n, e) ∈ J → 0 × U 0 , we have

J → 0 (π 2 )( n, e) = n ∈J → 0 p(π 2 )( n, e, n ). It follows: η = X e, g, g - P ∈p(Q) ( n,e)∈J → 0 ×U 0 n ∈J → 0 P ( n, e, n ) × Φ( n, e).
Now, the set

Q = P U (W id g, g × {( g, e, → 0)})\[L( , )] r r is regular, so that by Parikh's theorem, p(Q) is a semi-linear set; it can be written p(Q) = d k=1 (p k 0 + d k k =1 p k k ×N), where p k k ∈ N J → 0 ×U 0 ×J → 0 for all 1 ≤ k ≤ d and 0 ≤ k ≤ d k . Then: η = X e, g, g - d k=1 ( n,e)∈J → 0 ×U 0 n ∈J → 0 p k 0 ( n, e, n ) × Φ( n, e) + (γ k ) k ∈N d k d k k =1 ( n,e)∈J → 0 ×U 0 n ∈J → 0 γ k • p k k ( n, e, n ) × Φ( n, e) .
We write Γ e, g, g for the second term, so that η = X e, g, g -Γ e, g, g . For each 1 ≤ k ≤ d and 0 ≤ k ≤ d k , we define the interval

I k k = ( n,e)∈J → 0 ×U 0 n ∈J → 0 p k k ( n, e, n ) × Φ( n, e), so that Γ e, g, g = d k=1 I k 0 + (γ k ) k ∈N d k d k k =1 γ k • I k k .
Notice that I k k ⊆ R ≥0 for all k and k. We then consider two cases:

first assume that for some k 0 and k 0 ≥ 1 and some ( n 0 , e 0 ) ∈ J

→ 0 × U 0 , it holds p k 0 k 0 ( n 0 , e 0 , → 0
) > 0 and Φ( n 0 , e 0 ) has positive length. Consider the interval

L γ = I k 0 0 + γ • I k 0 k 0
for any γ ∈ N. Then clearly L γ ⊆ Γ e, g, g for all γ. Moreover, by definition of k 0 and k 0 , the length of I k 0 k 0 is positive, so that the length of γ •I k 0 k 0 tends to infinity with γ. It follows that for some α ∈ Q ≥0 , → α ⊆ γ∈N L γ ⊆ Γ e, g, g . Then Γ e, g, g ∩ → α has finite granularity, so that it is a finite union of intervals of the form I k k ∩ → α. It follows that Γ e, g, g is a finite union of intervals, and can hence be represented by a regular union of intervals. Now, by hypothesis, X can be represented as a regular union of intervals. By Prop. 4.4.13, so can X e, g, g . Then by Prop. 4.3.4, η = X e, g, g -Γ e, g, g can be represented as a regular union of intervals.

-We now assume that for all k and k ≥ 1 and all ( n, e) ∈ J r × U 0 , either p k k ( n, e, → 0) = 0, or Φ( n, e) has length 0. Then for all 1 ≤ k ≤ n and 1 ≤ k ≤ d k ,

I k k contains a single element, which is a rational. Then -Γ e, g, g = {-γ | γ ∈ Γ e, g, g } can be represented as a regular union of intervals. By Prop. 4.3.4, we get that η can also be represented by a regular union of intervals in that case.

In the end, η is the union of two sets that can be represented as regular unions of intervals, thus by Prop. 4.3.4, it can also be represented as a regular union of intervals.

Towards efficient diagnosis for n-clocks timed automata

In this section, we extend (part) of our formalism to n-clock timed automata. Precisely, we define a generalized gauge ( ) operator, and prove that it can be used to compute an function. As for one-clock automata, we show that it is enough to consider linear timed markings, that handles. However we do not propose an extension of regular unions of intervals and thus do not prove that we can furthermore restrict the study to regular timed markings.

We first extend the basic definitions in Section 4.5.1, and then use them to define a generalized -operator and prove that it can be used to compute τ -closures for any number of clocks (Section 4.5.2).

Valuations for multiple clocks

In this subsection, we present the approach taken on n-clocks timed automata, and most importantly their valuations, and how the generic notations of this chapter are lifted from 1 to n ∈ N clocks.

Recall that a clock valuation is usually defined as a function v ∈ R C ≥0 . As we use negative values to store future clock values, we sometimes consider functions to R instead.

To simplify furthermore the technical discussions, we fix an (arbitrary) order on the clocks, and hence allow ourselves to consider valuations as vectors in R n (more heavily than in other chapters). Using this, we generalize the notions introduced in 4.2.1 for intervals to intersections of constraints on different dimensions. For a subset K of R, we consider r ∈ K n ≥0 and r ∈ K n ≥03 . We define the sum of a subset X of R n with an integer t ∈ R as the translation of the set on all dimensions: X + t = {(x 1 + t, .., x n + t) | (x 1 , .., x n ) ∈ X}. This corresponds to t time units elapsing. Using these definitions, we can extend atomic, finite and linear timed sets to n-clocks simply by considering sets and constraints in n dimensions. We note T (R n ) the set of linear timed sets of R n . We extend the clock resets operations to sets elementwise, i.e. for C ⊆ C,

X [C ←0] == {x [C ←0] | x ∈ X}.
To discuss timed predecessors and successors of a set of clock valuations we note respectively Pre(X)

= t∈R ≥0 (X -t) and Post(X) = t∈R ≥0 (X + t).
The notion of sequences of transitions and its associated notations does not depend on the number of clocks and can be directly reused. We extend markings to elements of

M = (2 R n ≥0
) L and use similar definitions for O a for an observable action a ∈ Σ:

O a (m) : ∈ L → {v ∈ R n ≥0 | ∃ ∈ L. ∃v ∈ m( ). ∃e ∈ enab(( , v)). act(e) = a ∧ ( , v) 0 - → e ( , v )}.
and O t :

O t (m) : → {v ∈ R n ≥0 | ∃ ∈ L. ∃v ∈ m( ).( , v) t → ε ( , v )}.

τ -closures

Using the previous definitions, we can extend timed markings to n dimensions and get back every definition and result of Section 4.4.1 by just considering valuations in R n ≥0 instead of R ≥0 . In this context, → 0 has to be read as the intersection of the corresponding one dimensional constraints on all dimensions. We hence focus here on how to compute the τ -closures using linear timed markings, as done in Section 4.4.2. For this, we first define and formalize the operator transforming a set of (potential) configurations by the effect of a guard and a set of clock resets.

c 1 c 2 -4 -3 -2 -1 1 2 3 -1 0 1 2 3 X Y X {c 1 } Y v v + t v v + t Figure 4.16: Effect of the gauge on c 1 for X = -1 ≤ c 1 ≤ 0 ∧ c 2 ≥ 0 ∧ c 2 -c 1 ≤ 1 and Y = 1 ≤ c 1 ≤ 2.
X C Y = {v ∈ R n | ∃t ≥ 0, v ∈ ((X + t) ∩ Y ) [C ←0] -t} (4.1)
Remark 4.5.2. A valuation v in X C Y is an anticipation ( i.e. a potential valuation corresponding to) an element of ((X + t) ∩ Y ) [C ←0] : the intersection of f ( X, → 0)(t) 4 with the guard Y . This can be better seen by rephrasing the definition as

v + t ∈ ((X + t) ∩ Y ) [C ←0] . Hence v ∈ X C Y if and only if there exists v ∈ X and t ∈ R ≥0 such that v + t ∈ Y and v + t = v + t [C ←0]
as can be seen in Figure 4.16.

We give some characterizations of C , that can both help to represent the resulting set of valuations and obtain a zone-based computation.

Proposition 4.5.3. For two sets X and Y (a guard) and C ⊆ C,

X C Y = {v ∈ R n | ∃t ≥ 0, v ∈ (X ∩ (Y -t)) [C ←0] -t × 1 C } (4.2)
with 1 C the vector with ones in the dimensions corresponding to clocks of C and zeros 4. The filter is taken as → 0 because it is not part of the definition.

c 1 c 2 -4 -3 -2 -1 1 2 3 -3 -2 -1 0 1 2 3 X Y X {c 1 } Y X {c 2 } Y Figure 4.17: Effect of the gauge on individual clocks for X = -1 ≤ c 1 ≤ 0 ∧ c 2 ≥ 0 ∧ c 2 -c 1 ≤ 1 and Y = 1 ≤ c 1 ≤ 2. otherwise. Similarly, letting I = {t ∈ R ≥0 | (X + t) ∩ Y = ∅}: X C Y = ((X + I) ∩ Y ) [C ←0] -I ∩ X [C ←0] -1 (4.3) X C Y = (X ∩ (Y -I)) [C ←0] -I × 1 C ∩ X [C ←0] -1 (4.4)
with X [C ←0] -1 the zone obtained from X by suppressing all constraints (guard and diagonals) including clocks in C .

The characterization 4.2 can be used to more easily handle examples of C : it behaves as a filter on X where Y is sent back by an increasing delay -t for t ∈ R ≥0 and the projection on C \ C is used on the sub-vector space -t × 1 C . The characterization 4.3 allows to express the gauge using only operations on sets (of delays and valuations). Its variant 4.4 proposes the same for 4.2.

Example 4.5.4. Consider a transition with guard

Y = 1 ≤ c 1 ≤ 2 in a timed automaton over the two clocks C = {c 1 , c 2 }. We take the set of potential valuations X = -1 ≤ c 1 ≤ 0 ∧ c 2 ≥ 0 ∧ c 2 -c 1 ≤ 1.
Both X and Y are depicted in Figure 4.17, together with X {c 1 } Y and X {c 2 } Y and the constructions necessary to obtain them through characterization 4.3. Notice that in this case, I = [1, 3].

Finally, we construct p 3 in the following way:

∀c ∈ C, p 3 (c) =      p 2 (c) if c / ∈ C s(c) if c ∈ C .
We now prove that p 3 ∈ (K -t 2 ) ∩ X:

• p 3 is in Y . Indeed Y being a guard it suffices to verify that for each clock c, p 3 (c) satisfies the guard (i.e. non-diagonal) constraints of Y . This is ensured by definition of p 3 as p 2 , s ∈ Y ;

• p 3 is in v + t 2[C ←0] -1 . Indeed p 2 ∈ P ⊆ v + t 2[C ←0] -1
and the projections on C \ C of p 2 and p 3 are equal;

• p 3 is in X + t 2 . We prove this by showing that it satisfies the different constraints;

p 3 is on the segment between p 2 and s, which both satisfy the diagonal constraints of X + t 2 . As these constraints are convex, p 3 satisfies them too;

for all guard constraints on c ∈ C , s ∈ (X + t 2 ) and p 3 (c) = s(c) ensures the satisfaction of the constraint;

similarly for all guard constraints on c / ∈ C , p 3 (c) = v + t 2 (c) = q(c) and q ∈ (X + t 2 ).

Hence p 3 ∈ X + t 2 as it satisfies all its constraints. We thus have our result, as p 3 is a witness that v ∈ X C Y .

Remark 4.5.5. There is no delay-based characterization in the n-clock case as delays are not of the same type than valuations anymore -by opposition to the 1 clock case.

This operator generalizes the operator we defined for the one-clock case: indeed, for C = {c}:

X C Y = {v ∈ R | ∃t ≥ 0, v ∈ ((X + t) ∩ Y ) [{c}←0] -t} = {v ∈ R | ∃t ≥ 0, ((X + t) ∩ Y ) [{c}←0] = ∅ ∧ v ∈ {0} -t} = {-t | t ∈ R ≥0 ∧ ((X + t) ∩ Y ) = ∅}
which is equal to X Y (the one-clock operator) by Prop. 4.4.10.

Y = e ∩ e C • if Post(X) ∩ r ∩ Y = ∅: (X, r) in c 1 c 2 X Y • if Post(X) ∩ r ∩ Y = ∅: (X, r) in c 1 c 2 X Y (∅, → 0) in c 1 c 2 • if C = ∅: c 1 c 2 • if C = {c 1 }: x y Figure 4
.18: Representation of the effect of silent transition ( , Y, τ, C , ) in three cases.

Thanks to this operator, we can define the effect of a transition on an atomic timed set: Definition 4.5.6. For an atomic timed set T = (X, r) of T (R n ) and a transition e = ( , g = e ∩ e, τ, C , ) of U :

((X, r); e) =            (∅; → 0) if Post(X) ∩ r ∩ g = ∅ (X C ( r ∩ g); ( r ∩ e) [C ←0] ) if Post(X) ∩ r ∩ g = ∅.
Remark 4.5.7. We can separate the case C = ∅ in the expression of to obtain X ∩ Pre( r ∩ g) = X ∅ ( r ∩ g). As the computation is then far less involved it can be interesting to separate this case in an implementation. One can notice that for the 1-clock case C = ∅ and C = C were separated and no general expression was given. The reason is that both cases can be expressed with far be taken from that valuation. Hence both sides of the equivalence evaluate to false, and the equivalence holds.

• now assume that Post(X) ∩ r ∩ Y = ∅, and consider the case where C is empty. We proceed by double inclusion. We first take a valuation v ∈ f (T,π) (t) and exhibit a predecessor of this configuration in f T (t 0 ) ∩ Y for a t 0 to be constructed.

v ∈ f (T,π) (t) means that v ∈ (X ∩ Pre( r ∩ Y )) + t ∩ r ∩ e. Equivalently, v ∈ (X + t) ∩ Pre( r + t ∩ Y + t) ∩ r ∩ e.
As e encodes the lower bounds on individual clocks of Y and Pre( r

+ t ∩ Y + t) keeps all diagonal constraints of r ∩ Y , we have that v ∈ (X+t)∩Pre( r+t∩Y +t)∩Post( r∩Y ). Hence ∃t ∈ [0, t], v-t ∈ X+t-t ∩ r∩Y . By taking t 0 = t-t and v = v-t+t 0 = v-t we can write ∃t 0 ∈ [0, t], ∃v ∈ f T (t 0 )∩ r∩Y .
The first inclusion then holds by definition of π.

Conversely, if t 0 ∈ [0; t] and v ∈ f T (t 0 ) exist such that (src(π), v ) t-t 0 --→ π (tgt(π), v), then v ∈ X + t 0 ∩ r and as there is no reset on π, v = v + t -t 0 , v ∈ X + t ∩ r + t -t 0 ⊆ X + t ∩ r. Furthermore, ∃t 1 ≤ t -t 0 such that v + t 1 ∈ Y . More precisely, v + t 1 ∈ r ∩ Y as v ∈ r
and this set has no upper bound. As v = v + t -t 0 , we have v ∈ Pre( r ∩ Y ) + t and v ∈ e as t 1 ≤ t -t 0 . By aggregating the constraints, we obtain v ∈ (X + t) ∩ Pre(( r ∩ Y ∩) + t) ∩ r ∩ e i.e. the left-hand side of the equivalence.

• Consider Post(X) ∩ r ∩ Y = ∅ and C = ∅. In this case v ∈ f (T,π) (t) means that ∃t 2 ≥ 0, v ∈ ( r ∩ Y ∩ (X + t 2 )) [C ←0] -t 2 + t. Notice that as v ∈ R n ≥0 , it comes that t -t 2 ≥ 0. Let v [C ←0] = v + t 2 -t ∈ (f T (t 2 ) ∩ Y ) [C ←0] It comes that we can construct v ∈ f T (t 2 ) ∩ Y from v [C ←0] 5 . By taking t 0 = t 2 we have that 0 ≤ t 0 ≤ t, v ∈ f T (t 0 ) such that v t-t 0 --→ π v + t -t 0 = v.
Conversely if for a given 0 ≤ t 0 ≤ t and a given v ∈ f T (t 0 ), we have that (src(π), v )

t-t 0 --→ π (tgt(π), v), then there exists 0 ≤ t 1 ≤ t -t 0 such that v + t 1 ∈ Y and the transition is taken from v . By definition of f T , v + t 1 ∈ f T (t 0 + t 1 ) ∩ Y . We have that v = (v + t 1 ) [C ←0] + t -t 0 -t 1 ∈ (f T (t 0 + t 1 ) ∩ Y ) [C ←0] + t -t 0 -t 1 . Hence by taking t 2 = t 0 + t 1 we ensure that v ∈ (X C Y ) + t. Furthermore, t -t 0 -t 1 ≥ 0 as t 1 ≤ t -t 0 thus ∀c ∈ C , v(c) ≥ 0, and v |C ∈ ( r ∩ e) |C as v |C ∈ Post(v + t 1 ) |C (t 1 ≤ t -t 0 ) and v + t 1 ∈ f T (t 0 + t 1 ) ∩ Y ⊆ r ∩ Y ⊆ r ∩ e.
This proves the left part of the equivalence.

5. Possibly several such v exists, we consider one of them.

We now extend this result to sequences of transitions. The case where π = ε is straightforward. Now assume that the result holds for some word π, and consider a word π • e. In case tgt(π) = src(e), the result is trivial.

The case of single transitions has been handled just above. We thus consider the case of π • e with π ∈ U + . First assume that v ∈ f (T,π•e) (t), and let T = (T, π). Then v ∈ f (T ,e) (t), thus there exist 0 ≤ t 0 ≤ t and v ∈ f T (t 0 ) s.t. (src(e), v)

t-t 0 --→ e (tgt(e), v ). Since v ∈ f T (t 0 ), there must exist 0 ≤ t 1 ≤ t 0 and v ∈ f T (t 1 ) such that (src(π), v ) t 0 -t 1 ---→ π (tgt(π), v).
This way, we have found a value t 1 with 0 ≤ t 1 ≤ t such that (src(π), v )

t-t 1 --→ π•e (tgt(e), v ). Conversely, if (src(π), v ) t-t 1 --→ π•e (tgt(e), v ) for some 0 ≤ t 1 ≤ t and v ∈ f T (t 1 ), then we have (src(π), v ) t 0 -t 1 ---→ π (tgt(π), v) t-t 0
--→ e (tgt(e), v ) for some t 1 ≤ t 0 ≤ t and some v. We prove v ∈ f (T,π) (t 0 ): indeed, we have 0 ≤ t 1 ≤ t 0 ], and v ∈ f T (t 1 ) such that (src(π), v )

t 0 -t 1 ---→ π (tgt(π), v), which by induction hypothesis entails v ∈ f (T,π) (t 0 ). Thus we have 0 ≤ t 0 ≤ t and v ∈ f T (t 0 ), where T = (T, π), such that (tgt(π), v) t-t 0 --→ e (tgt(e), v ), which means v ∈ f (T ,e) (t

), and concludes the proof.

As in the one-clock case, this semantic characterization of (T, π) entails the following, ensuring that the result of a computation by does not depend on the representation. Corollary 4.5.10. For any sequence π of transitions, and any two equivalent linear timed sets T and T , the linear timed sets (T, π) and (T , π) are equivalent.

We can once more extend to linear timed markings in the same way and generalize Theorem 4.4.19 to n clocks. The proof is the same for v ∈ R n ≥0 , using Lemma 4.5.9. By considering the extensions of to languages, we immediately get: Corollary 4.5.11. For any linear timed marking M , it holds (M ) ≡ M τ . This proves that our operator (and the underlying gauge) is a correct representation of the semantic of a (silent) timed automaton, and that linear timed markings are sufficient to represent the semantics of timed automata.

Example 4.5.12. Consider the timed automaton with two clocks depicted in Figure 4.19 (the τ labels are omitted). The construction with 0 , 1 and 2 allows to obtain the atomic timed set

(c 1 ∈ [-1, 0] ∧ c 2 ∈ [-1, 0]; c 1 , c 2 ≥ 0) as input for 3
6 . The closure of 3 is depicted on the side with the two guards generating it. It is composed of several infinite 6. We slightly abuse the construction to take the union of sets for two timed sets sharing the same filter.

behaviours. Notice that there is some regularity in the structure, but that it can be complex: several nested infinite repetitions appear.

0 1 2 3 c 1 ≤ 1 c 1 := 0 c 2 ≤ 1 c 2 := 0 c 2 ≤ 1 c 2 := 0 c 1 ≤ 1 c 1 := 0 c 1 = 2 c 1 := 0 c 2 = 2 c 1 , c 2 := 0 c 1 c 2 -8 -7 -6 -5 -4 -3 -2 -1 1 2 3 -7 -6 -5 -4 -3 -2 -1 1 2 3 {c 1 } {c 1 , c 2 }

Stability of a representable class

We do not define regular timed sets and markings for multiple clocks, but in the following, we prove that finite timed markings are enough to represent the closure. Proposition 4.5.13. Consider a timed automaton A and a regular language L ⊆ L E (A) included in the language of transitions. Given a finite timed marking M , (M, L) is a finite timed marking.

Proof. Let M be a finite timed marking: then M can be written as the finite union of atomic timed markings M ( ,X, r) , defined as M ( ,X, r) ( ) = f (X; r) and M ( ,X, r) ( ) = f (∅; → 0) for all = .

In the end, any regular timed marking M can be written as the finite union k∈K M k of atomic timed markings. Thus we have (M ) ≡ k∈K (M k ) and it is enough to prove the property for atomic timed markings.

We write 1 ((X; r), π) and 2 ((X; r), π) for the first and second components of ((X; r), π), and use the same generalization to timed markings than in the one-clock case.

Note that:

• 2 ((X; r), π) = r ∩ k<n e k if π = e 1 . . . e n ∈ U id * is a sequence of consecutive non-resetting transitions.

• 2 ((X; r), π • e) = ( 2 ((X; r), π) ∩ e) [C ←0] if e is a resetting transition for C .

We define the set of upper bounds of guards appearing in U as

G = { e | ∃ e = ( , Y = e ∧ e, C , , ) ∈ U } and using this set J r = { g ∈ R n ≥0 | ∀i ∈ [1, n], ∃ e ∈ G ∪ { r, → 0}, e i = g i }.
It is easy to see that J r is a finite set and 2 ((X; r), π) ∈ J r . Hence a finite number of atomic timed sets is sufficient to describe the closure of a finite timed marking, i.e. the closure is a finite timed set. Indeed, there is a finite number of filters r in the representation of M (one per M k ), no more than |J r | different atomic timed sets are needed per location of the automaton for each r, and there is a finite number of locations.

Conclusion and future works

In this chapter, we presented a novel approach to solve the state estimation problem, and in particular to efficiently compute the τ -closure in the case of partially observable one-clock timed automata; it builds on a kind of powerset construction for automata over timed domains, using our new formalism of linear timed sets to represent the evolution of the set of reachable configurations of the automaton. We prove that the semanticsand in particular the τ -closure -of a one-clock timed automaton can be computed using regular timed sets, a finitely representable subclass of linear timed sets. We furthermore show that the full class of regular timed sets is needed to represent general one-clock timed automata.

We extend the basis of our approach to timed automata with multiple clocks, and prove that finite timed sets are again enough to compute the τ -closure.

An implementation demonstrating the approach for 1-clock automata, in the case of diagnosis, and comparing it to the technique proposed in [START_REF] Tripakis | Fault Diagnosis for Timed Automata[END_REF] has been realized. It is not reproduced in this thesis as the author did not take part in its creation, but can be found at http://people.irisa.fr/Nicolas.Markey/download/DOTA.zip. It is furthermore commented in [START_REF] Bouyer | Diagnosing timed automata using timed markings[END_REF].

A natural extension of these results is to introduce a (representable) notion of regularity for multiple clocks automata and to prove that it is large enough to represent the semantics and τ -closure.

Conclusion and future works

This is by no means immediate, as first the presence of multiple clocks allows involved behaviours, and second (and most importantly), the proofs proposed in the one-clock case rely on the structure of the operator, and the ability to simply "count" resetting transitions (i.e. the order of transitions only slightly matters). This is not true for multiple clocks, where the orders of the resets is central to the dynamics.

Another possible direction of research could target priced timed automata, with the aim of monitoring the cost of the execution in the worst case.

In this chapter, we propose to generalize active learning to a class of timed automata enjoying both several clocks and different possible resets that cannot be inferred directly from observations. This allows us to design and prove algorithms that handle all the main difficulties that arise in deterministic TAs, making this contribution an important first step towards active learning for generic deterministic TAs.

To our knowledge, the closest works to ours are Grinchtein's thesis on active learning of DERA [START_REF] Grinchtein | Learning of Timed Systems[END_REF], the T L * algorithm [START_REF] Shang-Wei Lin | An Efficient Algorithm for Learning Event-Recording Automata[END_REF], that proposes the most efficient algorithm to learn DERAs using observation tables, and the paper proposing to learn one clock TAs [An+20]. The work of Grinchtein et al. [START_REF] Grinchtein | Inference of Event-Recording Automata Using Timed Decision Trees[END_REF] is the most related to ours, as we use some of the data structures they developed and keep the general approach based on timed decision trees. The main difference between our work and this one is that we handle the inference of resets in a class of models in which they cannot be directly deduced from observations. The authors of this work have later proposed T L * s , a completely different and more efficient algorithm to learn DERAs. This approach relies on the region-graph construction, which uses the knowledge of clock values to minimize the number of queries. As clock values depend on clock resets, we could not use this argument and build upon the method proposed in T L * s . The approach reported in [An+20] proposes to deal with reset guessing, but makes it in a somewhat "brute force" manner, by directly applying a branch-and-bound algorithm and jumping from model to model. In order to be able to deal with larger dimensions, e.g. to handle TAs with more than one clock, we need to be more efficient by exploiting the theory built around TAs and detecting invalid models as early as possible. Finally, T L * [Lin+11] is thematically very close to our work (both because it learns DERAs and enhances upon Grinchtein's thesis). Yet it is a different technical stem, that uses specificities of the ERAs to optimize the learning method in a fashion similar to T L * s , while our approach goes beyond ERAs. It is furthermore based on a table data-structure, while our work reasons on trees.

Preliminaries

Timed automata

For the rest of this chapter, we fix a finite alphabet Σ of actions. We are mostly interested in the language generative approach of timed automata. Because of this we consider timed automata without invariants and equipped with a subset of accepting locations that will be used to define their language.

As a consequence, in this chapter we define timed automata as A = (L, init , C, E, Accept) without precising the alphabet Σ, the invariant function I : L → {true} but adding the accepting subset Accept ⊆ L. We define the set of accepting configurations as

Accept T = Accept × R C ≥0 .
We say that a path (resp. run) is accepting when the target of the last transition is in

Accept (resp. in Accept T ).
As this chapter focuses on languages and not controllability, we consider only finite alternating runs, signatures and traces ending in Σ (as final delays are by nature unobservable in this setting). As mentioned in Section 1.2 (Figure 1.4), we will refer to signatures as timed words with resets and traces as timed words. Furthermore, for such words one can group delays and discrete transitions. We will thus represent timed words with resets as w tr = (t i , a i , r i ) 1≤i≤n ∈ (R ≥0 × Σ × 2 C ) * and timed words as sequences of pairs in (R ≥0 × Σ) * . Remark 5.2.1. These denominations accentuate the language recognition point of view of learning. We wish to learn a model from a timed language that we query as needed, as such we are interested in timed words and add semantics and behavioural informations, such as resets.

Event recording automata (ERA) [START_REF] Alur | Event-Clock Automata: A Determinizable Class of Timed Automata[END_REF] form a subclass of TAs in which there is one clock c a per action a of the alphabet Σ, and such that for each transition ( , g, a, r, ), it holds r = {c a }. Since clock resets are determined by actions, they can be retrieved from observed timed words, and this class of TAs is easier to learn than general TAs (although more complex than DFAs).

A TA is said complete when from any configuration ( , v) and any letter a, there is a transition of the form ( , g, a, r, ) such that v |= g. Clearly enough, any timed automaton can be turned into a complete timed automaton accepting the same timed language.

Remark 5.2.2. This definition is equivalent to the definition of completeness given in Chapter 3 (p.65) but stated in a more semantic manner.

In a given deterministic timed automaton, a timed word (resp. timed word with resets) can be the trace (resp. signature) of at most one (alternating) run (ending in Σ). This unique run ρ such that trace(ρ) = w t is then denoted trace -1 run (w t ). As a consequence, a timed word w t is accepted if and only if the run trace -1 run (w t ) is (defined and) accepted.

In this chapter, we extend ERAs as reset-optional ERAs (RERAs), by allowing transitions to not reset their clock: Definition 5.2.3. A reset-optional event recording automaton (RERA) over Σ is a TA A = (L, init , C, E, Accept) such that for all transitions ( , g, a, r, ) ∈ E, it holds r ⊆ {c a }.

While not as expressive as generic timed automata, RERAs are more complex than ERAs in a crucial way with respect to learning: the clock resets along a run ρ of an unknown RERA cannot be inferred directly from its observed timed word w t = obs(ρ). Furthermore, RERAs are not determinizable in general. Indeed, the classical example of non-determinizable 1-clock TA proposed in Figure 1.5 and adapted to the current setting in Figure 5.1, is in fact a RERA. In the rest of this chapter, we will restrict our study to deterministic RERAs.

Active learning for timed automata

The general principle of (untimed) active learning is to learn a model from observations acquired by membership queries and equivalence queries made to a teacher, as displayed in Figure 5.2.

In membership queries, a word is provided to a teacher, who in return provides the observation, i.e. its membership to the target language. In an equivalence query, a hypothesis (a model) is proposed to the teacher; she either agrees it if it is equivalent to the model we wish to learn, or otherwise provides a counterexample, i.e. a word and its observation that separates the language of the target model and that of the hypothesis. The set of observations, which grows with counter-examples and results from membership queries, is formalized as a partial function Acc mapping words to acceptance status (+ or -). To build a model, classical active learning algorithms then want to identify a prefix-closed subset U of Dom(Acc) such that for all letters a in the alphabet and words u ∈ U , u.a ∈ Dom(Acc) and either u.a ∈ U , or there is another word u ∈ U having the same observed behaviour as u.a. This is usually done by a data structure that can store observations, makes it possible to check some important properties on the set U and can be transformed into a hypothesis when the current set indeed has these properties. This is the role of the observation table in Angluin's L * algorithm [START_REF] Angluin | Learning Regular Sets from Queries and Counterexamples[END_REF].

A high level pseudo-code of a generic active learning approach is presented in Algorithm 4. Its principle is the following: starting from the empty hypothesis (i.e. a model accepting the empty language), an equivalence query is made, and the counterexamples processed into the data structure, i.e. membership queries are made to help the structure get back its set of "good" properties. This is then repeated until an equivalence query returns yes. In the setting of timed automata, the problem can be expressed in the very same way, with the definitions of the previous section: the learner makes membership queries about timed words, and aims at reconstructing a timed automaton matching those observations. In this setting, the acceptance function is a partial function Acc : (R ≥0 × Σ) * → {+, -} associating acceptance status to timed words that have been observed. We then define what it means for a timed automaton to model a set of observations: Definition 5.2.4. Let Acc be an acceptance function and A be a deterministic timed automaton. We say that A is a model for Acc when for all w t ∈ Dom(Acc), w t is accepting if, and only if, Acc(w t ) = +.

Remember that in a deterministic TA, there is at most a unique run ρ = trace -1 run (w t ) of A such that trace(ρ) = w t , and the acceptance of w t by A is defined by the acceptance of ρ by A. If there is no such run, then by definition w t is not accepted.

Recall that the objective of an active learning agent is not to learn a model corresponding to a fixed set of observations, but rather to acquire enough information via adapted requests and hypotheses, so as to build an automaton equivalent to the teacher's model.

There are two main challenges when considering the timed-automata setting:

• the first one is the uncountable number of possible delays, and more generally the uncountable number of configurations of the automaton. This is a classical issue in the area of timed automata, and finite abstractions have already been studied extensively to cope with this;

• the second challenge lies in the fact that timed words do not contain enough information to recover the full configuration of the timed automaton they are learning: clock resets have to be inferred as well, proposing hypotheses and discarding them on-the-fly when they are contradicted by new observations.

In this chapter, we present a method to perform the active learning of deterministic reset-optional event recording automata. Precisely, we consider that the teacher has a deterministic RERA model in mind. This objective makes us face the two challenges mentioned above, as RERAs have both a high dimensionality and unobservable clock resets. We propose to deal with them by • introducing relevant abstractions to group the (infinitely) many behaviours in a finite number of sets. These will be used to describe the different levels of precision that we need, namely answering the following two questions: how large can we afford our guards to be? What is the acceptance status of elementary (non distinguishable) sets of behaviours?

• using our abstractions to infer from the observations the relevance of different sequences of resets to explain the behaviours. Precisely, we detect reset combinations that do not allow to separate observations despite their disagreement on acceptance, and prune them as soon as they appear.

Section 5.3 introduces more formalism to abstract observations and behaviours of timed automata at different levels, according to our needs for the rest of the chapter.

In Section 5.4, our data-structures and their important properties are described. They separate decision making (akin to the construction of U ) and reset guessing, as these two tasks require different levels of abstraction. In Section 5.5, we describe our version of the active learning algorithm and the properties verified by each function. Finally in Section 5.6, the construction of a hypothesis from the data structures is discussed.

Abstraction

In this section, we will detail several layers of abstractions that are useful to understand timed automata behaviours and that will be used in this chapter.

We extend the notions defined in Chapter 1 to obtain representations of single observations, minimal undistinguishable sets of observations, and large sets of observations that will serve to define the model behaviours (i.e. paths). These abstractions are central to our work on model-reality interactions, as they serve to define different languages encoding the information inferred from observations or induced from the models, e.g. clock dynamics. One could see a connection with abstract interpretation in the crucial importance of different representations (abstract domains) used to organise the information. The main difference probably being that we are only interested in exact representation, whereas abstract interpretation often serves to discuss the balance between precision and efficiency.

Two orthogonal kinds of abstractions arise.

Precision level: A run does not describe only the atomic behaviour that a timed automaton can encode, as if a run is accepted by an automaton, usually an infinite number of related runs are also accepted. Infinite sets of runs can be regrouped in so-called zone runs. Using region equivalence and K-equivalence relations, the atomic behaviour of timed automata can be finitely represented by specific zone runs called region runs and K-closed runs.

Another interesting level of precision is that of paths, as they define the behaviour of the model by fixing the guards. This gives us three decreasingly-precise levels: runs, region or K-closed runs and paths that are respectively related to single executions, atomic behaviours, and construction of the language.

Observation level: In Section 1.2, runs of timed automata and their corresponding traces, which are obtained by projecting away unobservable aspects, have been introduced. In many applications, such as learning, one intuitively wants to retrieve runs from traces, i.e., invert the projection. However runs depend on a specific model (notably on the locations and transitions names) that can not be learned. Hence, we will introduce model-agnostic intermediate notions. First, abstract runs will be defined, based on the information available in timed words with resets, which forget model-specific information. In the same way, but at a less precise level, zone words with resets and guarded words with resets will be defined as model-agnostic behaviours of zone runs and paths, and at the same time as infinite sets of abstract runs or timed words with resets.

The different notions that will be introduced in this section are summarized in Figure 5.3, along with the two axis of abstraction: precision and observation level.

Zone runs, region runs and K-closed runs

Alternating runs serve as representations of single executions, but we still lack a notion of minimally distinguishable set of runs, which we now define using K-equivalence.

We start by defining (alternating) zone runs, i.e., runs in which all valuations are generalized to zones. Formally, a (partial) zone run is an element η

= (( i-1 , z i-1 ) δ - → ( i-1 , z i-1 ) e i - → ( i , z i )) 1≤i≤n of ((L × Z) × {δ} × (L × Z) × E) * × (L × Z) such that for all 1 ≤ i ≤ n, z i-1 ⊆ - → z i-1
and for e i = ( i-1 , g, a, r, i ), we have z i-1 ∩ g = ∅ and

z i = (z i-1 ∧ g) [r←0] .
Here, δ is a symbolic representation of time elapse, as any precise delay may lead to different zones from different valuations in the same zone. We say that a partial zone run is a zone run when z 0 is the zone limited to 0. We say that an alternating (partial) run ρ = (( i-1 , v i-1 )

t i - →( i-1 , v i-1 ) e i - →( i , v i )) 1≤i≤n belongs to a zone run η = (( i-1 , z i-1 ) δ - →( i-1 , z i-1 ) e i - →( i , z i )) 1≤i≤n , noted ρ ∈ η when for 0 ≤ i ≤ n: i = i , v i ∈ z i and for i > 0, e i = e i .
Similarly, we say that a partial zone run η 1 is included in a partial zone run η 2 of the same length n, noted η 1 ⊆ η 2 , when for any 0

≤ i ≤ n, 1 i = 2 i , z 1 i ⊆ z 2 i , z 1 i ⊆ z 2 i and e 1 i = e 2 i if i = 0.
Remark 5.3.1. Notice that once we know that z 1 0 ⊆ z 2 0 it is only necessary to check that z 1 i ⊆ z 2 i . Notably, for zone runs this is ensured by definition. We call region run a zone run in which all zones are regions and K-closed run a zone run in which all zones are K-closed zones. Notice that for any run ρ, there is a unique K-closed run η (resp. region run η) such that ρ ∈ η. Furthermore, any other run ρ belonging to η is accepting if, and only if, ρ is.

TA:

A = (L, init , C, E, Accept) Path: π = (e i ) 1≤i≤n e i = ( i-1 , g i , a i , r i , i ) T = (S, s init , Γ, →) Run: ρ ∈ Ex(A) alternating ρ = (( i-1 , v i-1 ) t i - → ( i-1 , v i-1 + t i ) e i - →( i , v i ))
1≤i≤n

Zone run:

η = (( i-1 , z i-1 ) δ - → ( i-1 , z i-1 ) e i - →( i , z i ))
1≤i≤n

Abstract run:

σ = (v i-1 t i - → v i-1 +t i (a i ,r i ) ---→ v i ) 1≤i≤n
Timed words with resets:

w tr = ((t i , a i , r i )) 1≤i≤n ∈ (R ≥0 × Σ × 2 C ) *
Timed words:

w t = ((t i , a i )) 1≤i≤n ∈ (R ≥0 × Σ) *
Abstract zone run:

ζ = (z i-1 ) δ - → z i-1 (a i ,r i ) ---→ z i ) 1≤i≤n
Zone word with resets:

w zr = ((z i-1 , a i , r i )) 1≤i≤n ∈ (Z × Σ × 2 C
) * Guarded word with resets:

w gr = ((g i , a i , r i )) 1≤i≤n ∈ (G × Σ × 2 C
) *

Resets: We say that two runs are region (resp. K)-equivalent when they belong to the same region (resp. K-closed) run.

r = (r i ) 1≤i≤n ∈ (2 C ) * Observations Behaviours Semantics Syntax
Interestingly, although K-closed runs concentrate on undistinguishability in the present, one can deduce a unique region run from a K-closed run, thanks to the interaction between the constraints and the resets.

Lemma 5.3.2. Pairs of runs are K-equivalent if and only if they are region-equivalent.

Proof. We can easily see that pairs of region-equivalent runs are K-equivalent, as regionequivalence adds strictly more constraints.

Consider a K-closed run η and two runs ρ 1 , ρ 2 ∈ η. We show that ρ 1 and ρ 2 are region-equivalent. We make the proof by induction on the runs, noting ρ j = (( i-1 , v j i-1 )

t j i - → ( i-1 , v i-1 j ) e i - →( i , v j i )) 1≤i≤n for j ∈ {1, 2} and η = (( i-1 , z i-1 ) δ - →( i-1 , z i-1 ) e i - →( i , z i ))
1≤i≤n . The central fact underlying the proof is that a region has the same constraints as a K-closed zone plus constraints on the order on the fractional parts of the clock values, i.e. to be in the same region, the fractional parts of the clock values must be in the same order (with equalities at the same places).

First, v 1 0 = v 2 0 = 0 with all clocks evaluated to 0. These are clearly region-equivalent. After t 1 , v 0 1 ≈ K v 0 2 and for both valuations, all (fractional parts of the) clock values are equal. Hence they are region-equivalent. Inductively, starting from two region-equivalent valuations

v 1 i-1 ∼ ∼ ∼reg v 2 i-1 , for j ∈ {1, 2} v j i = v i-1 j [r i ←0] , hence v i 1 ∼ ∼ ∼reg v i 2
as resets preserve region-equivalence. After t i the valuations v i j are K-equivalent and the order of the fractional parts of the clock valuations in v i j is equal to the one of v j i up to a permutation induced by the delay. We show that both runs are affected by the same permutation. Indeed, the permutation depends on the integer constants crossed by the clock values (i.e. the K-closed zones with constraints c = n with n ∈ N and c ∈ C). Starting from the same region and ending in the same K-closed zone implies that both runs cross the same constants (those in the intersection of the past of the K-closed zone and the future of the region). Hence

v i 1 ∼ ∼ ∼reg v i 2 .
By induction this gives us our result.

Remark 5.3.3. This result obviously does not hold for partial K-closed and region runs as it relies on the initial valuation 0 to ensure that first valuations are region equivalent.

Remark 5.3.4. In this chapter K-closed runs (and related notions) are abundantly used instead of region runs. This is because we are mostly interested in the question "given two runs, could a model have separated them?" which is encoded by K-equivalence (for timed automata without diagonal constraints).

Signatures and Behaviours

Now that all semantic notions are defined, we need a way to capture the corresponding behaviours that is model-agnostic, similarly to how timed words with resets capture alternating runs.

Between runs and timed words with resets, we define abstract runs, that keep the valuations of a run, and only abstract the locations of the timed automaton. For an alternating run ρ = (( i-1 , v i-1 )

t i - →( i-1 , v i-1 ) e i - →( i , v i )) 1≤i≤n we define its behaviour behav as the abstract run σ = behav(ρ) = (v i-1 t i - →v i-1 (a i ,r i ) ---→ v i ) 1≤i≤n .
We define the signature of an abstract run σ = (v i-1

t i - → v i-1 (a i ,r i ) ---→ v i ) 1≤i≤n
as the projection on timed words with resets; i.e. sig(σ) = (t i , a i , r i ) 1≤i≤n . Notice that this definition is coherent with the previous definition of signatures for runs as sig(behav(ρ)) = sig(ρ) for any run ρ.

From a timed word with resets, we have enough information to infer the clock values after each delay and action/reset pair, starting from the initial valuation. Formally, sig is bijective from abstract runs to timed words with resets. We note sig -1 behav its inverse. Considering a timed word with reset w tr = (t i , a i , r i ) 1≤i≤n we have sig -1 behav (w tr ) = (v i-1

t i - → v i-1 + t i (a i ,r i ) ---→ v i ) 1≤i≤n with v 0 = 0 the initial valuation and v i = (v i-1 + t i ) [r i ←0] .
Remark 5.3.5. As we have defined several signature functions (from runs and from abstract runs), we differentiate the inverse functions by precising their target: sig -1 behav for abstract runs ( i.e. behaviours) and sig -1 run for runs. Despite similar notations, these two functions are very different in nature. Notably sig -1 behav is always known while sig -1 run depends on the model.

We define similar abstractions and (inverse) projections for zone runs under the name of abstract zone runs and zone words with resets in (Z

× Σ × 2 C ) * . For a zone run η = (( i-1 , z i-1 ) δ - → ( i-1 , z i-1 ) e i - → ( i , z i )) 1≤i≤n , its behaviour is defined as the abstract zone run ζ = behav(η) = (z i-1 δ - → z i-1 (a i ,r i ) ---→ z i ) 1≤i≤n with e i = ( i-1 , g i , a i , r i , i ).
The signature of that abstract zone run is the zone word with resets sig(ζ) = (z i-1 , a i , r i ) 1≤i≤n . Notice that only the zone z i-1 is taken into account, as we want to know the zone after the observed delay but before the action transition and possible reset is taken.

Again, we define sig(η) = sig(behav(η)) and sig -1 behav the inverse projection starting from z 0 = {0}.

We call K-closed word (resp. region word) (with reset) a zone run with resets where all zones are K-closed zones (resp. regions).

For a path π = (e i ), the abstraction has to keep the guards, as they are important to define the behaviours. For this, a specific kind of zone words with resets, named guarded words with resets (in (G × Σ × 2 C ) * ) is used. Precisely, for transitions e i = ( i-1 , g i , a i , r i , i ), we define the signature of the path as sig((e i ) 1≤i≤n ) = (g i , a i , r i ) 1≤i≤n .

Remark 5.3.6. We do not define abstract paths as we have no use for them, but they could be defined similarly to abstract zones with resets.

We extend the "belongs-to" relation of abstract runs with respect to abstract zone runs and zone words with resets (resp. the inclusion to pairs of abstract zone runs and zone words with resets) by checking that at all indices, the actions and resets correspond, and that the valuation after delay belongs to the zone, i.e. v i-1 + t i ∈ z i-1 (resp. that at each index, the zone of the first word is included in the corresponding zone of the second one). This "belongs-to" relation can also be extended to timed words with resets with respect to zone runs with resets by w tr ∈ w zr if sig -1 behav (w tr ) ∈ w zr . We extend the notation Kz (used to define the K-closed zone containing a given valuation) to timed words with resets, so that for a timed word with resets w tr , Kz(w tr ) is the unique K-closed word for which w tr ∈ Kz(w tr ).

We can also extend the resets projection to all these notions (run, abstract run, zone runs with resets, zone words with resets, guarded words with resets) by projecting away everything but resets.

Finally, we say that a timed word w t is compatible with a zone word with resets w zr and write w t w zr , if there exists a timed word with resets w tr ∈ w zr with trace(w tr ) = w t . In other words, letting r = resets(w zr ), we have w t w zr whenever w tr = w t ⊗ r belongs to w zr .

Remark 5.3.7. The abstractions defined in this section span the semantics (with zone runs) and behaviours (with abstract runs, zone and guarded words with resets) but not the observations. Indeed there is no point in generalizing observations: without information about clocks, one would not know what to deduce out of a single observation.

Manipulations on words

In this subsection, some manipulations on timed words and K-closed words useful in the rest of the paper are presented.

Linear combinations of timed words

In our learning process, we will manipulate linear combinations of timed words. For two timed words w 1 t = ((t 1 i , a i )) 0≤i≤n and w 2 t = ((t 2 i , a i )) 0≤i≤n with the same untimed projection, we define their λ-weighted sum

w 3 t = λ • w 1 t + (1 -λ) • w 2 t ,
as the timed word

w 3 t = ((λ • t 1 i + (1 -λ) • t 2 i , a i ) 0≤i≤n
). Such linear combinations have the following property: Proposition 5.3.8. For any two timed words w j t = (t j i , a i ) 0≤i≤n for j ∈ {1, 2} with the same untimed projection, for any 0 ≤ λ ≤ 1 and for any reset word r = (r i ) 0≤i≤n :

sig -1 behav ((λ • w 1 t + (1 -λ) • w 2 t ) ⊗ r) = λ • sig -1 behav (w 1 t ⊗ r) + (1 -λ) • sig -1 behav (w 2 t ⊗ r).
This means that for any valuation v i reached in sig -1 behav ((λ

• w 1 t + (1 -λ) • w 2 t ) ⊗ r) and any clock c ∈ C, v i (c) = λ • v 1 i (c) + (1 -λ) • v 2 i (c) with v j i the corresponding valuation in sig -1 behav (w j t ) ⊗ r.
Proof. The proof is made by induction on 0 ≤ i ≤ n considering the valuations v i in

λ • w 1 t + (1 -λ) • w 2 t ) ⊗ (r i ) i . For i = 0, the only valuation encountered is v 0 = 0 = λ • v 1 0 + (1 -λ) • v 2 0 .
For i > 0, assume that we have the property for all valuations reached along the prefixes (i.e. for i < i), and especially that v

i-1 (c) = λ • v 1 i-1 (c) + (1 -λ) • v 2 i-1 (c). Then we have that v i-1 (c) + λ • t 1 + (1 -λ) • t 2 = λ • (v 1 i-1 + t 1 ) + (1 -λ) • (v 2 i-1 + t 2
) and by applying the resets commended by r we obtain the result desired for the i-th valuation.

Offsets on zones and directions

We generalize the notion of offset from valuations (e.g. v + t for t ∈ R ≥0 ) to zones, with integer (positive or negative) offsets affecting subsets of clocks in the following way.

∀ i, j ∈ {1, . . . , |C|}, c i ≺ k + n i and c i -c j ≺ k + n i -n j . c 1 c 2 1 2 3 1 2 3 ( 1 -1 ) ( -1 0 )
Figure 5.4: The effect of some offsets on a zone.

Example 5.3.10. In Figure 5.4, an open zone is presented, together with the effect of multiples offsets on it. Offsets are noted ( n 1 n 2 ) with n 1 the offset on clock c 1 and n 2 the offset on clock c 2 .

Finally, it will be useful to consider the effect of modifying the delay at a given index in a word on the rest of the word. This is formalized through the notion of direction. To ease the notations, we will note 1 C for C ⊆ C the vector with ones in the dimensions of clocks in C and zeros everywhere else. Definition 5.3.11. Consider a sequence of resets r = (r k ) 1≤k≤n . For two indices 1 ≤ i ≤ j ≤ n we define the direction at rank i on rank j, noted dir j i (r), as

dir j i (r) = 1 C with C = {c ∈ C | ∀i ≤ k < j, c / ∈ r k }.
We generalize directions to j < i by fixing dir j i (r) = 1 ∅ and by defining directions on all words with resets ( i.e. time, zone and guarded words with resets).

We will use directions to build offsets. To explain the convention for j < i, we give a characterization of directions for timed words with resets.

Proposition 5.3.12. Consider a timed word with resets w tr = (t i , a i , r i ) 1≤i≤n and its corresponding abstract run σ = sig -1 behav (w tr ) = (v i-1

(t i ,a i ,r i ) ----→ v i ) 1≤i≤n
. Then for any 1 ≤ k, j ≤ n, dir j k (w tr ) corresponds to the effect on v j-1 + t j of adding one time unit to t k . Formally, noting

w tr = (t i , a i , r i ) 1≤i<k • (t k + 1, a k , r k ) • (t i , a i , r i ) k<i≤n , and σ = sig -1 behav (w t r) = (v i-1 (t i ,a i ,r i ) ----→ v i ) 1≤i≤n , it holds v j-1 + t j = (v j-1 + t j ) + dir j k (w tr ).
Proof. For any clock c ∈ C, (v j-1 + t j )(c) is the sum of delays since the last index l at which c was reset. If l ≥ k then adding time to t k does not modify the valuation of c. By definition we have dir j k (w tr )(c) = 0, and the equality holds. Conversely, if l < k, or if clock c has not been reset, then the modification directly impacts the valuation by the same amount, and we have dir j k (w tr )(i) = 1, matching the addition of 1 time unit.

Example 5.3.13. Consider the timed word with resets (for C = {c a , c b , c c })

w tr = (1.2, a, c a )(0.3, b, c b )(1.6, a, c a )(2.6, c, c c )(0.1, c, ∅)(0.1, c, ∅).
Then dir 6 6 (w tr ) = dir 6 5 (w tr ) = 1 C , dir 6 4 (w tr ) = 1 {ca,c b } , dir 6 3 (w tr ) = 1 {c b } and dir 6 2 (w tr ) = dir 6 1 (w tr ) = 1 ∅ .

Observation structures

In this section, we describe the structures used to represent and process the timed observations acquired during learning (and stored via the acceptance function Acc), and the decisions we make based on those observations. We begin with defining a generic structure, which we call timed decision graphs, which are tree-shaped structures encoding (abstractions of) timed words, as well as all possible combinations of resets. We then define two special instances of these graphs:

• timed language graphs are timed decision graphs in which we reason more specifically about guards that have to be assigned to transitions. Timed language graphs will be very close to the candidate timed automata we aim at building;

• timed observation graphs are timed decision graphs in which we reason more specifically about reset sequences. We use them to detect invalid sequences of resets.

Remark 5.4.1. From now on, we will rely on the targeted class of automata: RERAs. Notably, we will replace resets denoted by 2 C by { , ⊥} where denotes that the clock corresponding to the transition action is reset and ⊥ that it is not. This will notably serve in the construction of the timed decision graphs and in the different languages to simplify the expressions.

We did not apply these restrictions before as the abstractions are fitting for any timed automaton and thus need not be restrained in this way.

Timed decision graphs

Definition 5.4.2. A timed decision graph (TDG) is a labelled graph D = (S, E) where S is partitioned into two sets S o and S d as follows:

• S o ⊆ (Z × Σ × { , ⊥}) * is
-ζ(s 0 o ) = {0}; -ζ(w zr • (z, a, r)) = ( ----→ ζ(w zr ) ∩ z) [r←0]
(where we consider that r = {c a } when r = and r = ∅ if r = ⊥). Completeness means that all observations are represented in the TDG, while wellgroundedness means that every observation state has a non-empty set of compatible observations. A labelled TDG is said to implement an acceptance function Acc when it is both complete and well-grounded w.r.t. Acc.

• S d ⊆ S o × Z × Σ
We will use two instantiations of these structures in our learning process:

• timed language graphs (TLG) are timed decision graphs with the following extra requirements:

all zones appearing in a TLG are guards (i.e., they do not involve diagonal constraints);

-for any s o , writing Z = {z | (s o , z, a) ∈ S d }, if Z is non-empty, then Z partitions the set of clock valuations R C ≥0 .
As we explain below, TLGs will be used to refine guards, eventually representing (the unfolding of) a candidate timed automaton modelling the current observation.

• timed observation graphs (TOG) are timed decision graphs in which all zones that appear in the definitions of the states are K-closed zones;

all decision states have two successors, by both and ⊥.

As we explain below, these will be useful for characterizing (im)possible sequences of clock resets. In a TOG, we note s the root state, as it corresponds to the empty word.

Notice that in general, different labelled TDGs can implement the same acceptance function Acc, because different zones can appear in the zone words defining the states of the TDG. This is the case for TLGs. However there is unicity for TOGs, as proven below.

Proposition 5.4.3. Given an acceptance function Acc, there is a unique TOG M implementing it.

Proof. Consider M and M implementing Acc. By definition of TOGs, all zones appearing in these graphs are K-closed, and thus all the zone words are K-closed words. Notice that some states (here all leaves) are labelled with bothand +, indicating that this graph (which can be seen as a timed automaton) is not a good candidate for our learning situation, and has to be refined. The TOG implementing Acc is displayed in Figure 5.5c. Notice that the rightmost leaves are both accepting and non-accepting, as both (0.7, a)(0.9, a) and (0.7, a), (1.2, a) satisfy their constraints. As the guards of a TOG are minimal, we know that these two words are equivalent and that any model for those observations should reset the clock c a after the first transition (at least for 0 < c a < 1). Timed language graphs and timed observation graphs provide two different views of a set of observations, with opposite levels of abstraction. Timed observation graphs are precise in the sense that they distinguish the behaviours as finely as possible for a deterministic RERA, using K-equivalence as a base.

Timed language graphs are made to be as general as possible, only featuring guards that are necessary to model the observations, i.e. a transition labelled by a guard g is only split in g 1 and g 2 when it is necessary to distinguish accepting and non-accepting behaviours.

Remark 5.4.5. This notion of necessity is local: from a given TLG, if it is necessary to add a guard this does not mean that there exist no fitting TLGs without that guard: perhaps a different one could have been added, or changing another guard would make the addition unnecessary.

We introduce a function α : S M o → S N o that links these two models, both in order to display the strong links between the two graphs and for later algorithmic use. We prove below that α is well-defined, and some of its basic properties: Proof. We prove the different properties independently.

• To ensure that α is well-defined, it suffices to check that for every s M o ∈ S M o there is a unique covering guarded word s N o in S N o . The unicity comes by induction by the properties of the observation states of a TLG: there is no overlapping between guards corresponding to the same action after a given language state (by definition of a TLG); the existence comes from the well-groundedness of M w.r.t. Acc and the completeness of the observation structure: by well-groundedness, for each observation state s N o there is an observation w t , and by completeness, that observation w t is covered by the TLG.

Consistency and validity

In the learning process we will construct TLGs, i.e. basically prefix-closed sets of guarded words with resets associated with their sets of compatible observed timed words. These guarded words with resets are the premises of paths in the targeted TA. However only some of them can form those paths in a deterministic TA modeling Acc, since their image by Acc must be unique for all compatible observed timed words. This is formalized by the notion of consistency: Detecting and handling inconsistencies is central to our algorithms, as it characterizes the need to introduce new guards to split observation nodes in the timed language graphs. For this, we use the following characterization of inconsistencies of observation states of a TLG.

Proposition 5.4.9. Consider a labelled TLG N implementing an acceptance function Acc. An In the subtree to the right, however, it can be checked that no guards (only involving clock x a and integer constants) can resolve the inconsistencies in this subtree: this indicates that resetting clock x a in the first decision state is the only solution in this example.

We claim that detecting impossible reset decisions on the TLG, as we do in Example 5.4.10, is not an efficient approach. Indeed, detecting such an impossibility requires discarding every possible guard that could separate the observations. This can be done either by refining guards until no refinements are possible (which may require numerous membership queries, see Algorithm 9 in Section 5.5.2 for more insight), or by checking on any pair of words leading to an inconsistency, whether there is a guard that can separate them, which again may require a large number of queries. As exemplified above, TOGs are composed with K-closed words with resets, that cannot be refined. However some of them may not be possible in any deterministic TA modeling the acceptance function Acc. This is formalized by the notion of invalidity:

Definition 5.4.12. Let Acc be an acceptance function. Let w zr be a K-closed word with resets. We say that w zr is valid for Acc if there exists a deterministic complete RERA A that models Acc such that all timed words with resets in w zr are signatures of runs of A.

Otherwise, w zr is said invalid.

We generalize (in)validity to TLGs and say that an observation state s o of a TLG is invalid when there exists an invalid K-closed word with resets included in s o . We call s o valid otherwise. We say that the TLG N is valid when it has no invalid observation state. We call the valid part of a TLG its restriction to valid observation states and intermediate decision states.

In the particular case of TOGs, the following proposition characterizes invalid reset combinations in the (unique) TOG implementing Acc; this will allow us to identify and prune them: Proposition 5.4.13. Let Acc be an acceptance function, and M Acc the TOG implementing Acc. The set of marked observation states of M Acc is defined by the least fixpoint of the following marking: Let us first consider that s o is marked and prove that w zr is invalid by induction on the definition of marked states.

•
First assume that |label(s o )| = 2; then by definition of label and words, there exist two timed words w t and w t compatible with w zr , such that Acc(w t ) = Acc(w t ). Take the two corresponding timed words with resets w tr = w t ⊗ resets(w zr ) and w tr = w t ⊗ resets(w zr ), implementing the resets of w zr along w t and w t . If a timed automaton A contains w tr as a run, it also contains w tr (because they correspond to the same K-closed word with resets). Since A must be deterministic, either both w tr and w tr correspond to accepting runs, or they both correspond to non-accepting runs. By Definition 5.2.4, A cannot be a model for Acc. Second, suppose s o = w zr and there is an action a and a K-closed zone z such that both s o = w zr • (z, a, ) and s o = w zr • (z, a, ⊥) are marked. By induction hypothesis, they are invalid.

Assume that some complete deterministic RERA A contains a timed word with resets w tr ∈ w zr as a run, then by completeness it would also contain w tr • (v, a, ) or w tr • (v, a, ⊥) for some v ∈ z, contradicting our induction hypotheses as these are invalid.

We thus have proven that all marked states are invalid. Now, assume that s o = w zr is a successor of a marked observation state s o = w zr . Hence w zr is invalid. By definition, no timed automata containing a timed word with resets w tr ∈ w zr may be a model for Acc. Then, this directly extends to w zr (as all automata containing w zr contain its prefixes), and w zr is invalid.

Conversely, we consider a zone word with resets w zr invalid for the acceptance function Acc, and try to prove that the observation state s o = w zr has a marked prefix in the TOG M Acc . The proof follows the following scheme: we first show how TAs can be extracted from the TOG. We then discuss on such TAs containing paths for w zr , and use the fact that they cannot be models to find states that cover accepting and non accepting observations. Finally, the definition of marked states is used to reach the conclusion. The difficulty comes from the fact that the pair of observations that we have may not correspond to w zr in the first place (we have no guarantees except that the TA is not a model). Hence we have to show that other labels of size two can be handled by changing the reset choices (thus changing the TA extracted from the TOG) without modifying the path containing w zr . This enforces that, in the end, a pair relevant for w zr will appear.

By definition of invalidity any deterministic RERA A containing signatures in w zr is not a model of Acc. This is the case in particular for any (tree-shaped) RERA obtained from the TOG M Acc by keeping only observation states, freely choosing whether or not to reset the clock on each transition, and making leaves accepting or not depending on label (both choices are possible when |label(s o )| = 2). Take such a TA A M containing the path π corresponding to w zr in the TOG. It cannot be a model of Acc, so by Definition 5.2.4 there must be w t ∈ Dom(Acc) such that Acc(w t ) disagrees with the acceptance of w t in A M . Notice that, for Acc(w t ) not to agree with the acceptance of the target location, it has to end in a location in A M built from an observation state s o with |label(s o )| = 2. Hence, there is another timed word w t in words(s o ) such that Acc(w t ) = Acc(w t ).

We will now prove that w zr has a marked predecessor. We consider two cases:

• first, assume that for any reset choices, w t and w t cannot be separated by any guard. Formally: for any sequence of reset choices r ∈ { , ⊥} |wt| , for any index 0 ≤ j ≤ |w t |, and for any guard g, writing v i and v i for the clock valuations reached at step i along the abstract runs sig -1 behav (w t ⊗ r) and sig -1 behav (w t ⊗ r) obtained from w t and w t by resetting clocks according to r, we have v i |= g if, and only if, v i |= g.

In this case, all leaves in the TOG M Acc that can be reached by reading w t or w t are labelled with both + and -, and are marked by our marking procedure. By our hypothesis, all intermediary valuations visit the same K-closed zones, and all corresponding states in the TOG are marked. In the end, the root of the TOG is marked, and then all states have a marked predecessor. In particular, it is true for s o = w zr ;

• now, assume that there exist two timed words with resets w tr = w t ⊗r and w tr = w t ⊗r, obtained from w t and w t by adding a same sequence of resets r, that can be separated at some step i by a guard g. This sequence of resets r might not be the one considered in A M but we can modify the reset sequence of A M along the branch corresponding to w t and w t (i.e. by changing the reset choices and using different branches of N ).

-First assume that we can modify A M so that w tr and w tr correspond to runs of A M , without modifying the path π corresponding to w zr ; in that case, we build a new TA A M from A M accordingly, and add the guard g at step i, so as to separate w tr and w tr . In that case, we can repeat the arguments above with the TA A M , as we assumed that no TA containing w zr can be a model for Acc.

By furthermore forbidding to re-use a reset combination already discarded (e.g. to return to A M ) and since there are only finitely many observations, and they are of finite length, this situation cannot occur indefinitely. Notice that if there is no way to separate two observations without coming back to a previously discarded reset combination, we can conclude to the invalidity of the root as previously.

-If the modification inevitably leads to excluding w zr from the runs of A M , then there is a prefix w zr of w zr (which is also a prefix of w tr ) after which no guard can distinguish between (the suffixes of) w t and w t . Applying the same arguments as above for the root, all leaves in the subtree after w tr corresponding to all reset sequences for suffixes of w t and w t are labelled with + and -; the state corresponding to w zr must also be marked, and so s o = w zr has a marked predecessor.

Example 5.4.14. In the TOG of Example 5.4.11, both leaves in the right-hand-side subtree are invalid, which indicates that the clock has to be reset when reading the first a (at least when it occurs after a delay in (0, 1)).

Remark 5.4.15. We could propagate marks to suffixes to mark exactly all invalid states, but as the TOGs are always explored from the root it is unnecessary (a marked prefix will be reached beforehand) and constitutes a simple optimization.

When marking a state of the TOG, we are able to aggregate a set of pairs of words that will characterize an invalidity. For unmarked successors of marked states, we use their predecessor characteristic sets. For an invalid observation state s o , we call W so a characteristic set of its invalidity.

One can see that a characteristic set of an invalidity is sufficient to characterize it, as it matches the definition of marking.

Using the marked states introduced in Prop. 5.4.13, the TOG will be used to detect invalid resets, while the purpose of the TLG is to construct possible guards for candidate timed automata. In order to be able to construct as many different hypotheses as possible, all (valid) combinations of resets must be considered, while the invalid ones should be pruned to avoid any overhead. For this we define a notion of maximal TLG, where no valid resets are pruned. The algorithms proposed in the following sections will construct and maintain maximal and valid TLGs. Notice that as we suppose that the system we try to learn can be modelled by a RERA, ensuring the maximality of a TLG implementing Acc suffices to ensure that we will find a model (as there is a valid RERA model and we do not discard valid choices). It can be noticed that the root of this TOG is valid, but that it is required for a model to distinguish between the cases where the first a occurs before time 2 and the cases where it occurs after time 3. For example, the TLG N displayed to the right of Figure 5.8 is maximal and implementing Acc. Indeed, as there is no guard on the first a-transition, the decision state s d leads only to invalid successors, which are thus pruned.

Example 5.4.18 is a situation where a guarded word with resets (here ) is not invalid in the TLG, but all its successors for a given action and guard (in this case a and true, corresponding to the decision state s d in the TLG of Figure 5.8) are. In such situations, two different K-closed words with resets make the successors invalid, and a guard has to be added in the TLG to separate them. This corresponds to a situation where Acc displays two different behaviours, one that require a clock reset to model, and another one that requires not to reset this very clock. Thus, in order to model Acc it is necessary to introduce a guard to separate these two behaviours (identified by two different invalidities).

Updating observation structures

The algorithms presented in this section are the operational core of our work. They are used to update the previously-defined data structures and ensure their good properties by membership queries and their processing, thus corresponding to line 4 of Algorithm 4 (Section 5.2.2).

We present the details of our algorithms as a set of subfunctions, connected as displayed on Figure 5.9. They work in three phases: • First, starting from a new observation (obtained from a counterexample or via a membership query), FindPath (Algorithm 5) updates the structures (TLG and TOG) while keeping most of their good properties, except possibly consistency. It uses AddWord (Algorithm 6) on prefixes when new states have to be added in either tree. This may require calls to Request (Algorithm 7) on prefixes, which provides new observations (via membership queries) and thus new calls to FindPath on those prefixes when necessary.

When an invalidity is detected in the TOG during this process, FindPath finds the root of the invalidity and prunes the corresponding subtree of the TLG as it is itself invalid (this approach is further discussed in Remark 5.5.1). These algorithms are presented in Section 5.5.1.

• When an inconsistency is detected by FindPath, a call to AdjPair (Algorithm 9) is made. This algorithm introduces a separating guard for the inconsistency, using a binary search timed words to find a fitting separation. Similarly, when a (valid) decision state is left without successors, InvalidityGuard (Algorithm 12) is called (through a preprocessing via Init-InvalidityGuard (Algorithm 8)). Its process is similar to AdjPair, but it reasons on K-closed words with resets, making it more complex.

These algorithms request new observations through Request, possibly leading to new calls to FindPath. Both these algorithms add guards to symbolic set of guards that can be used, but do not modify directly the structures. AdjPair is described in Section 5.5.2 and InvalidityGuard in Section 5.5.3.

• Finally, after the cycle of new observations and guard storing finishes, the relevant part of the models are rebuilt using these new guards by Rebuild (Algorithm 15) using calls to FindGuard (Algorithm 16) to find relevant guards. These algorithms are presented in Section 5.5.4.

Adding a new observation

Adding a new observation in the TLG N and in the TOG M is performed by function FindPath (Algorithm 5): this function recursively descends into M, following the input timed word w t and exploring both resetting and non-resetting edges. It is called as

FindPath( , w t , 0, s ).

• If the descent gets stuck by lack of new successors before exhausting w t , then new nodes have to be added in the TOG (and possibly corresponding nodes in the TLG). This is performed by AddWord (Algorithm 6), making new membership queries via Request (Algorithm 7) if needed to ensure prefix-closure. Each call to AddWord adds two observation states (corresponding to the two reset choices) in the TOG and possibly two observation states in the TLG (unless they already exist), as well as the intermediary decision states.

• If the descent exhausts w t , then it reaches a node corresponding to the K-closed timed word containing w t , whose words (and label, as a side-effect) function can be updated (as well as those of the associated node in the TLG). Invalidities and inconsistencies can be detected and resolved, respectively by scheduling calls to InvalidityGuard (Algorithm 12) and AdjPair (Algorithm 9). These two functions will be We now prove the rest of the property by induction on the calls to both FindPath and AddWord. We use the following induction hypothesis: A call to AddWord/FindPath creates subgraphs of N and M that implement the restriction of Acc to the subset of its domain belonging to s o the observation state given in argument.

• Basic case for FindPath. Here we have w t = . In this case the complete word to add was read before along the path. α(s o )) constitute, by the hypothesis on the initial structure, a subgraph implementing Acc (except that the guard g of the considered transition is not covered), except for the new word that may not be covered. But by unicity, they cannot correspond to paths satisfying the new word and thus we have our properties.

• Inductive case for AddWord. This case works exactly as the base case, except that calls to Request ensure that the new states have non-empty words, and the completeness with respect to w t is ensured by the induction hypothesis.

Dealing with inconsistency

An inconsistency arises when a language state of the TLG contains both accepting and non-accepting observations. It entails that a guard must be added somewhere in the structure in order to separate these observations.

For this we search for a pair of adjacent timed words with resets, which intuitively identify the boundary between accepting and non-accepting behaviours. We then build a finite set of differences between adjacent timed words with resets, each of which corresponds to a possible guard. This procedure is described in the AdjPair algorithm (Algorithm 9).

We use K-equivalence to define the notion of adjacency. Intuitively adjacent timed words with resets have the same projection on actions and resets, and their valuations either are K-equivalent, or materialize a boundary between the accepted and non-accepted words.

As these boundaries rely on clock values, we define adjacency on abstract runs and transfer it to timed words with resets using the sig bijection. Definition 5.5.4. For two abstract runs sharing the same length, the same actions and resets

σ = (v i-1 t i ,a i ,r i ---→ v i ) 1≤i≤n and σ = (v i-1 t i ,a i ,r i ---→ v i ) 1≤i≤n ,
we say that σ is adjacent to σ when for all i ∈ [1, n] and c ∈ C:

• if v i-1 (c) + t i ∈ N then |(v i-1 (c) + t i ) -(v i-1 (c) + t i )| < 1, • otherwise, v i-1 (c) + t i ≈ K v i-1 (c) + t i .
A timed word with resets w tr is said adjacent to a timed word with resets w tr when sig -1 behav (w tr ) is adjacent to sig -1 behav (w tr ).

Notice that adjacency is not a symmetric relation. However, we will sometimes say that an ordered pair (w tr , w tr ) is adjacent to mean that w tr is adjacent to w tr . We will often abuse the term and call (w tr , w tr ) an adjacent pair.

We use adjacency to identify differences between the timed words with resets as possible new guards that resolve the inconsistency. Definition 5.5.5. Given a pair of adjacent abstract runs sharing the same length, the same actions and resets σ

= (v i-1 t i ,a i ,r i ---→ v i ) 1≤i≤n and σ = (v i-1 t i ,a i ,r i ---→ v i ) 1≤i≤n , their difference noted diff(σ, σ ) is the set of quadruples defined as follows: if for a clock c ∈ C, v i-1 (c) + t i = k ∈ N, then if v i-1 (c) + t i < k, (i, c, ≥, k) ∈ diff(σ, σ ), and if v i-1 (c) + t i > k, (i, c, ≤, k) ∈ diff(σ, σ ).
For a pair of adjacent timed words with resets (w tr , w tr ) we define diff(w tr , w tr ) = diff(sig -1 behav (w tr ), sig -1 behav (w tr )).

Using these definitions, we can derive from two adjacent abstract runs or timed words with resets a set of candidates to make a new guard, that we call consistency guards. These guards will be used to reconstruct a language graph that is consistent with respect to the adjacent pair. Definition 5.5.6. For an adjacent pair (w tr , w tr ), a clock constraint c ≺ k with ≺ ∈ {<, ≤, ≥, >}, (i, c, ≺, k) ∈ diff(w tr , w tr ) is a consistency guard if there is no (j, c, ≺ , l) ∈ diff(w tr , w tr ) such that j < i or j = i and l < k.

The consistency guards are taken on the first difference, so as to ensure that they cannot be overwritten later (as there are no guards that can separate the pair before the consistency guard), and to avoid large constants as much as possible. Notice that we cannot always infer a unique guard from an adjacent pair, as multiple clocks can be different at the same time.

Remark 5.5.7. We choose to introduce a bias on guards by choosing a consistency guard with a constant as small as possible. In practice, one could replace this bias by another (expert-designed) one more fitted to the application. Our algorithms do not depend on it.

We define a set G Cons of consistency guards that will be used to create the guards in the updated TLG afterward.

AdjPair makes membership queries on linear combinations of the two initial observations to perform a binary search until the clock values of the pair have less than 1 time unit of distance. Then it forces every non-K-equivalent pair of clock values to have one of its elements be an integer with more linear combinations. Finally, in order to ensure that only one of the two timed words with resets have such integer distinctions, it compares them with their mean. This gives an adjacent pair, from which consistency guards are extracted and added to G Cons . Proposition 5.5.8. The AdjPair algorithm constructs an adjacent pair using at most O(m|Σ| log(K)) membership queries.

Proof. We refer the reader to the proof of Theorem 5.8 in [START_REF] Grinchtein | Inference of Event-Recording Automatausing Timed Decision Trees[END_REF], a long version of [START_REF] Grinchtein | Inference of Event-Recording Automata Using Timed Decision Trees[END_REF] appearing in O.Grinchtein's thesis [START_REF] Grinchtein | Learning of Timed Systems[END_REF].

The AdjPair algorithm proposes a guard that separates, as one would expect, the accepting and non-accepting observations given as an argument, as well as the accepting and non-accepting observations made during the algorithm execution.

Proposition 5.5.9. Let w 1 tr , w 2 tr be two timed words with resets sharing the same length n, the same actions and resets such that Acc(trace(w 1 tr )) = + and Acc(trace(w 2 tr )) = -. Consider W 1 the set of accepting observations requested during the call to AdjPair(w 1 tr , w 2 tr ) and W 2 the set of negative ones. By taking a consistency guard (i, c, ≺, k) for ≺ ∈ {<, ≤, =, ≥, >}, at depth 1 ≤ i ≤ n in the set of guards added to G Cons by AdjPair(w 1 tr , w 2 tr ), we have the following:

There is a guard g being either x ≺ k, or its complement such that, for any

w 1 ∈ W 1 ∪{trace(w 1 tr )} (resp. w 2 ∈ W 2 ∪{trace(w 2 tr )}), noting sig -1 behav (w 1 ⊗resets(w 1 tr )) = (v j-1 t j - → v j-1 +t j (a j ,r j ) ---→ v j ) 1≤j≤n (resp. sig -1 behav (w 2 ⊗resets(w 2 tr )) = (v j-1 t j - →v j-1 +t j (a j ,r j ) ---→ v j ) 1≤j≤n ), we get v i-1 + t i ∈ g and v i-1 + t i ∈ g.
Proof. First, notice that every call to Request is made on (the trace of) a linear combination of w 1 tr and w 2 tr . This can be seen by induction on the while and for loops: at the beginning Algorithm 9: Adding to G Cons consistency guards corresponding to an inconsistency.

1 AdjPair Input: w tr , w tr two timed words with resets of same length n, with same actions and resets, such that Acc(trace(w tr )) = + and Acc(trace(w tr )) = -. Output: G Cons a modified set of adjacent pairs tr and w tr = w 2 tr , and w tr is clearly constructed as a linear combination of both. Afterward, either w tr or w tr becomes w tr . As linear combinations of linear combinations of w 1 tr and w 2 tr are linear combinations of w 1 tr and w 2 tr , we have our result. By Prop. 5.3.8 we know that the clock values on the abstract run of these timed words with resets are linear combinations of the clock values of sig -1 behav (w 1 tr ) and sig -1 behav (w 2 tr ). This tells us that for a given depth 1 ≤ j ≤ n, the valuations v j-1 + t j (i.e. the valuations before the j-th transition) of the different words are on a segment between the corresponding valuations of w 1 tr and w 2 tr . We will now show that the following property is an invariant of the while and for loops: for any depth 1 ≤ j ≤ n, the valuations before the j-th transition of any positive observation's abstract run are between the ones of sig -1 behav (w 1 tr ) and sig -1 behav (w tr ) (w tr being -as defined in the algorithm -the current positive timed word with resets). Similarly, the valuation of any negative observation's abstract run are between the ones of sig -1 behav (w 2 tr ) and sig -1 behav (w tr ). At the beginning of the algorithm, this is clear, as w 1 tr = w tr , w 2 tr = w tr and no other Request has been made yet. Now suppose that it is true before a (for or while) loop iteration. w tr is a linear combination of w tr and w tr , so as previously, by Prop. 5.3.8, the valuation reached in sig -1 behav (w tr ) before the j-th transition is in the segment between the ones of sig -1 behav (w tr ) and sig -1 behav (w tr ). As w tr takes the name of the member of {w tr , w tr } it shares its acceptance with, the property is preserved at the end of the iteration.

2 Note sig -1 behav (w tr ) = (v i-1 (t i ,a i ,r i ) ----→ v i ) 1≤i≤n and sig -1 behav (w tr ) = (v i-1 (t i ,a i ,r i ) ----→ v i ) 1≤i≤n . 3 while not(∀i ∈ [0, n], ∀c ∈ C, |v i (c) + t i -(v i (c) + t i )| < 1 ∨ ((v i (c) + t i > K) ∧ (v i (c) + t i > K))) do
if v i (c) + t i = v i (c) + t i ∧ v i (c) + t i ∈ N ∧ v i (c) + t i ∈ N then 11 if v i (c) + t i < v i (c) + t i then 12 λ := ( v i (c) + t i -(v i (c) + t i ))/(v i (c) + t i -(v i (c) + t i )) 13 else 14 λ := ( v i (c) + t i -(v i (c) + t i ))/(v i (c) + t i -(v i (c) + t i ))
Finally, by definition of a consistency guard at depth i, it separates the valuations of the adjacent pair before their i-th transition. As the adjacent pair used is either (w tr , w tr ) or (w tr , w tr ) according to w tr acceptance status, as w tr is a linear combination of w tr and w tr and by using the previously proved facts, we have our result.

Dealing with decision states with no successors

As explained earlier, a marked observation state in the TOG indicates an invalidity: it points to a combination of resets that is unable to explain a precise set of observations. Invalidity is first simply dealt with by pruning the invalid parts of the TLG, as done in line 11 of FindPath (Algorithm 5). But sometimes a challenge may arise, as exemplified in Example 5.4.18: it can occur that all successors of a decision state of the TLG following a valid observation state are pruned, due to invalidities; this is because the guard leading to that decision state is not strong enough, and it has to be reinforced so as to split Introducing any guard on c a separating the behaviours for c a < 2 and c a > 3 would create two decision states which would each have one valid successor. For example, the TLG after the introduction of the guard c a > 3 is presented in Figure 5.10. Note that the valid leaves are inconsistent, as could be expected from the TOG. We would then have to call AdjPair as described in Section 5.5.2. Remark 5.5.11. In such situation, the introduction of a guard (called validity guard) is the only possible solution. Indeed, pruning a valid node of the TLG would contradict the definition of a (maximal) implementation of Acc, and leaving a valid node without successors does not allow to use it to construct an hypothesis. Remark 5.5.12. This situation is a consequence of our somewhat lazy handling of invalidities. Indeed, when detecting an invalidity in the TOG, one could decide to "cut" it out of the TLG by introducing guards modelling it and splitting the corresponding node. However, this would require isolating a K-closed zone from its complement, which in turn requires introducing a large number of subtrees, since the complement of a K-closed zone can in general only be represented by a disjunction of a large number of guards.

On the contrary, we chose to directly prune the subtree, and only split nodes (by introducing a validity guard as above) once we have pruned all possible resets. This approach remains correct, as the TLG remains maximal and we add separations as soon as a decision state is left without successors.

The constant in the guard that is introduced is taken in the set of integer constants separating the two invalid K-closed words with resets in the TOG. This is formalised by the notion of validity guard, akin to the consistency guard introduced previously. Remark 5.5.14. The validity guards are introduced as deep as possible as opposed to consistency guards that are introduced high in the tree. The reason for this is that invalidity is not primarily used to introduce guards to the model. In addition, it can be harder to achieve a precise guard separating K-closed words due to their structure and the difficulty to reproduce invalid K-closed words with their characteristic sets, which is discussed in the rest of this subsection.

We define validity guards for sets of words because we sometimes want to enforce a validity guard deduced from a pair upon other pairs (as we have more insight on its utility) while it is not formally a validity guard for the other pairs (i.e. it separates them but does not meet all criteria).

As for inconsistencies, it is desirable to introduce guards that correspond as closely as possible to the behaviours of the teacher's model. To this aim, we try to identify relevant validity guards by making new membership queries that "replicate" an invalidity by shifting its characteristic set (the set of pairs of timed words with resets witnessing the invalidity, see Definition 5.4.16).

Example 5.5.15. In Examples 5.4.18 and 5.5.10, several guards could have been introduced. In order to identify a good candidate, a solution is to replicate the characteristic set of (one of the) invalidities with x a ∈ (2, 3). For this, membership queries are made for observations stemming from the characteristic set of one of the invalidities. For example, we can shift the observations corresponding to the subtree with x a ∈ (3, 4), which may result in the following new observations:

- - + + -- - ± ± - ± ± - + + -- - ± ± - + + -- a,(1,2) a,[1] ⊥ a,(1,2) ⊥ ⊥ a,( 2
Acc((2.7, a)) = - Acc((2.7, a)(1.1, a)) = + Acc((2.9, a)(1.1, a)) = -
We end up with the TOG of Figure 5.11.

Since the subtrees of the TOG for guards x a ∈ (1, 2) and x a ∈ (2, 3) are different, we know that there is a change of behaviour at x a = 2. Here, we cannot determine which behaviour prevails for x a = 2, as explained later in Remark 5.5.16.

Remember that the characteristic set of an invalidity at depth n is made of pairs of timed words with resets (w 1 tr , w 2 tr ), each pair sharing the same length at least equal to n, the same actions and reset sequence resets(w 1 tr ) = resets(w 2 tr ), but diverging on the acceptance of their observations: Acc(trace(w 1 tr )) = Acc(trace(w 2 tr )). Introducing a guard precisely separating two invalidities is more complex than for inconsistencies for three reasons: (i) first, any attempt to detect the span of an invalidity of depth n should be able to replicate its characteristic set targeting a new K-closed word with resets of length n. Moreover, replication should not modify the K-equivalence structure of the characteristic set even beyond the invalidity at depth n: valuations reached by replicas should be K-equivalent if and only if they were initially.

(ii) second, the reset sequence r cannot explain the observations of the characteristic set, but a yet unknown one r could. When replicating invalidities we should thus consider all pairs of timed words with resets (trace(w1 tr ) ⊗ r , trace(w 2 tr ) ⊗ r ) for any sequence of resets r and also try not to modify their K-equivalence structure.

(iii) third, as for inconsistencies, it is important that the guards introduced by the algorithm separate all the K-closed words corresponding to each invalidity encountered during the execution. Yet, K-closed zones have a more complex structure than valuations. On the other hand, it is possible that failing to satisfy constraint (ii) still keeps the invalidity: changing the behaviour for other resets may lose the invalidity, but is not guaranteed to do so, especially if all reset combinations are not affected.

We found that no simple algorithm was satisfactory to solve the above sketched problem. The rest of this subsection is organized as follows: we first formalize the problem to be solved, then we review some approaches to solve it and point their shortcomings. Finally, we propose an algorithm that produces a validity guard, and explain the choice of our approach.

A formalization of constraints

Consider an invalid K-closed word with reset w zr = (z i-1 , a i , r i ) 1≤i≤n and a pair (w 1 tr , w 2 tr ) of timed words with resets in the characteristic set of w zr . We note, for j ∈ 1, 2, w j tr = (t j i , a i , r i ) 0≤i<n 1 .

Trying to replicate the invalidity in a K-closed word w zr = (z i-1 , a i , r i ) 1≤i≤n requires to construct a new characteristic set containing pairs (w 1 tr , w 2 tr ) solving the following constraints, by noting sig -1 behav (w j tr ) = (v j i-1

(t j i ,a i ,r i ) -----→ v j i ) 1≤i≤n . ∀1 ≤ i ≤ n. v j i-1 + t j i ∈ z i-1 (i.e., w j tr [1, n] ∈ w zr ) (5.1) ∀0 ≤ l < l ≤ |w 1 tr |. l<i≤l t 1 i ≈ K l<i≤l t 2 i iff l<i≤l t 1 i ≈ K l<i≤l t 2 i (5.2)
Here (5.1) corresponds to targeting w zr (part of constraints (i)), expressing that w 1/2 tr ∈ w zr , and (5.2) encodes (ii) (and the corresponding part of (i)), i.e., the preservation of the equivalence relation between the words for any reset sequence. Indeed, the clock valuations for any reset sequence are sums of consecutive delays. Together, these equations encode (i) and (ii).

Remark 5.5.18. Notice that the constraints in (5.2) can be relieved in the following way: one only has to consider reset combinations for which both words of the pair are valid, as if they reach invalid states we know that the reset combination does not happen in the model we wish to learn. We do not give the details of this optimization here, as it would not help to construct a general algorithm and would make equations (5.2) much less readable.

The constraint (iii) can be stated in the following way: consider two K-closed words with resets w 1 zr = ((z 1 i , a i , r i ) 1≤i≤n ) and w 2 zr = ((z 2 i , a i , r i ) 1≤i≤n ) corresponding to two different invalidities. When one tries to replicate the invalidity of w 1 zr in w 1 zr = ((z 1 i , a i , r i ) 1≤i≤n ) and w 2 zr in w 2 zr = ((z 2 i , a i , r i ) 1≤i≤n ), the following property should hold: for any guard c ∼ n with c ∈ C, n ∈ N and ∼ ∈ {<, ≤, ≥, >} such that z 1 i ∈ g and z 2 i ∈ g, it holds z 1 i ∈ g and z 2 i ∈ g.

Different approaches

Different approaches are now listed, together with counter-examples showing the properties that they would violate. A first simple approach would be to reason at the level of observations inside the invalidities characteristic sets and construct linear combinations of timed words with resets. The problem of this approach is that it is very well possible that the combinations are not This makes such direct method impossible to use, as observations that are not Kequivalent can never detect an invalidity.

Another possible approach is to apply offsets to zones in the invalid K-closed words based on their current distances i.e. define a "distance" between zones and use it on the zones z 1 i and z 2 i to make them closer in the sense of that distance. The problem with that approach is that the distance between two zones at depth i is not affected only by operations at depth i but by all operations on depth 0 < j < i. Indeed if a zone is modified at depth j, the effect of that modification has to be taken into account on all the future zones to ensure that the resulting word is feasible.

For this reason applying modifications at each depth i simultaneously based on distances between z 1 i and z 2 i would be meaningless: the effect of the modifications on the first zones would invalidate the next ones. Notably, the resulting K-closed word would not always be feasible.

To avoid this, one could chose to work only on a given depth i at a time, and loop through the word. Sadly, such modifications would (sometimes) either invalidate condition

x a 0 1 2 3 4 5 w 1 zr δ 1 δ 2 x a 0 1 2 3 4 5 w 2 zr δ 1 δ 2
(a) w 1 zr and w 2 zr .

x a 0 1 2 3 4 5

δ 1 δ 2 δ 2 x a 0 1 2 3 4 5 δ 1 δ 2 δ 2 (
(z, z ) = max{n ∈ N | z ∈ ---→ z + n}
This means that ∆ is the maximal integer constant you can add to a valuation in z while steel having a successor in z . ∆ will then be used as an abstraction of delays for K-closed words with resets.

Proof. First, notice that indeed z ∈ --------→ z + ∆(z, z ) as for any constraint c ≺ k in z + ∆(z, z ) with ≺ ∈ {<, ≤}, we have by definition of ∆ and offsets that k = n + ∆(z, z ) ≤ n with n the corresponding constraint in z and n in z . Furthermore the diagonal constraints are unchanged and the future operator lifts all upper bounds. Hence for all

v |= z, v |= --------→ z + ∆(z, z ).
On the other hand for N ≥ ∆(z, z ) + 1 by considering an argmin constraint c ≺ n in z (i.e. a constraint that minimizes the |n -n|), as there is no useless constraint in z there are v |= z infinitely close to that constraint (else it could be suppressed). For such v we have that v(c) < n + 1 = n + ∆(z, z ) + 1 = n + N and hence v |= ---→ z + N . Thus we have our result.

The necessary elements to build a binary search for K-closed words with resets have now been introduced. Remark 5.5.23. Notice that this kind of binary search can only work if applied to the whole word. Applying it to just a given depth would in general lead to the exact situation of Example 5.5.20 (when the offset is propagated to future zones).

Sadly, this approach fails to satisfy constraint (iii) because of the variety of forms of the zones, as shown in Example 5.5.24.

Example 5.5.24. Consider the K-closed words presented in Figure 5.14a:

w 1 zr = (0 < c a < 1 ∧ 0 < c b < 1, b, {c b }).((c a = 1 ∧ 0 < c b < 1, a, ∅)) w 2 zr = (2 < c a < 3 ∧ 2 < c b < 3, b, {c b }).((2 < c a < 3 ∧ c b = 0, a, ∅)) .
In order to apply a binary search, one has to reason on the K-closed runs associated with these words, so as to deduce the delays ∆ taken. We note {0} for the initial zone corresponding to this set.

sig -1 behav (w 1 zr ) = {0} δ - →0 < c a < 1 ∧ 0 < c b < 1 (b,{c b }) ----→ 0 < c a < 1 ∧ c b = 0 δ - → c a = 1 ∧ 0 < c b < 1 (a,∅) --→ c a = 1 ∧ 0 < c b < 1 sig -1 behav (w 2 zr ) = {0} δ - →2 < c a < 3 ∧ 2 < c b < 3 (b,{c b }) ----→ 2 < c a < 3 ∧ c b = 0 δ - → 2 < c a < 3 ∧ c b = 0 (a,∅) --→ 2 < c a < 3 ∧ c b = 0
Using this, one can see that at depth 1 the "delays" built from ∆ are 0 for w 1 zr and 2 for w 2 zr . At depth 2 both words have delay 0. Thus the result of a binary search would give the K-closed words presented in Figure 5.14b.

One can see that the zones at depth 2 have exchanged positions with respect to e.g. the guard c a < 2. Remark 5.5.25. Notice that the issue raised in Example 5.5.24 does not really depend on ∆. It would occur for any pseudo distance used, precisely because we do not have a valid distance on general zones.

In the example, adding different offsets to the zones at depth 2 would not satisfy constraint (iii). Proposition 5.5.27. Consider w zr = ((z i , a i , r i )) 1≤i≤n an invalid K-closed word and W its characteristic set. Then for

< min (w 1 tr ,w 2 tr )∈W min ((t i ,a i ,r i )) 1≤i≤m ∈{w 1 tr ,w 2 tr } {1 -j l=i t l | 1 ≤ i ≤ j ≤ m} n
Enlarge(w zr , W, ) terminates and returns w zr = (z i , a i , r i ) and W such that

z i = z i if z i is open and z i = z i
else. Furthermore all pairs in W respect the constraints of Equation 5.1 ( i.e. their elements are in w zr ).

The value of epsilon proposed is upper bounded so that it cannot make a clock valuation reach a new integer value in an observation, and so for any reset combination.

Proof. First, as Enlarge features only for loops on the (finite) length of w zr and the (finite) number of pairs in W , it terminates. Next, the constraint on z i compared to z i is clearly satisfied as the z i are initialized equal to z i and then replaced by their direct time successors if and only if they are not open.

It remains to show that the observations of the characteristic set W are indeed in w zr .

Consider w tr an element of a pair of W . We note sig

-1 behav (w tr ) = (v i-1 t i - →v i-1 + t i (a i ,r i ) ---→ v i )
and consider the corresponding

w tr in W sig -1 behav (w tr ) = (v i-1 t i - → v i-1 + t i (a i ,r i ) ---→ v i ).
We show by induction on 1 ≤ i ≤ |w zr | that v i-1 + t i ∈ z i . For i = 1, either z 1 was open; in which case v 0 + t 1 = v 0 + t 1 ∈ z 1 = z 1 ; or z 1 was not and z 1 = z 1 . In this case, v 0 + t 1 = v 0 + t 1 + and for all guards c = k in z 1 , k < (v 0 + t 1 )(c) < k + 1 as

< 1. Furthermore for all guards k < c < k + 1 in z 1 , k < (v 0 + t 1 )(c) < k + 1. Indeed, < 1 -v 0 + t 1 by definition. Hence v 0 + t 1 ∈ z 1 .
Suppose that the property is verified up to a depth i (included). We show the result for depth i + 1. If z i+1 is open then z i+1 = z i+1 and v i + t i+1 = v i + t i+1 . We want to show that the clock values in v i + t i+1 have the same integer part than the ones of v i + t i+1 . As both have a non-zero fractional part (v i + t i+1 because it is in an open zone, and v i + t i+1 because it only adds positive delays, hence without changing its integer part it cannot reach a zero fractional part), this suffices to prove that the other hand, for closed constraints (i.e. equality constraints) in z i+1 , the addition of in the last delay ensures that it reach the next open constraint (i.e. from c = c to c < c) and k. < 1 thus (v i + t i+1 )(c) < c + 1. This corresponds exactly to the constraints of z i+1 = z i+1 , hence we have our result.

v i + t i+1 ∈ z i+1 = z i+1 . Notice that for c ∈ C, (v i + t i+1 )(c) = (v i + t i+1 )(c) + k. with k ∈ N, k ≤ i.
Remark 5.5.28. The Enlarge algorithm is built simple. It could be possible to add at fewer steps by tracking if the previous were enough to push the valuations in z i . This in turn would allow for slightly more permissive constraints on . This could be done for example by checking if has already been added since the last time the clocks which have closed constraints were reset.

We choose to present the algorithm in a simple way to enhance readability at the price of this optimization.

With respect to constraint (iii), for two K-closed zones z 1 , z 2 there are guards that separate z 1 and z 2 but not z 1 and z 2 . An example is given in Example 5.5.29.

Example 5.5.29. Consider the zones z 1 : 0 < c a < 1 ∧ c b = 2 and z 2 : c a = 1 ∧ 1 < c b < 2. They are represented on Figure 5.16 together with their direct time successors z 1 and z 2 . Notice that the guard g : c a ≤ 3 accepts z 1 , z 1 and z 2 but not z 2 .

Despite this difficulty, we can use the fact that the zones have been enlarged toward the future to select guards that do not distinguish between non-open zones and their direct time successors, allowing to preserve constraint (iii). Proof. This result comes from the fact that these guards cannot distinguish between v(c) = k ∈ N and k < v(c) < k + 1, which are the only differences between a non-open K-closed zone and its direct time successor, as shown in Lemma 5.5.26.

We use these guards to separate open K-closed zones in BinSearch. Notice that if there exists a guard separating K-closed open zones, then there exists an enlargement closed one that separates them.

Finally, the Enlarge algorithm replicates characteristic sets while guaranteeing that pairs of K-equivalent words for any sequence of resets remain equivalent.

Proposition 5.5.31. For w 1 tr , w 2 tr elements of pairs of a characteristic set W, considering w 1 tr , w 2 tr the corresponding words in W constructed by a call to Enlarge(w zr , W, ) with an satisfying the constraint of Proposition 5.5.27 we have that

∀ 1 ≤ i ≤ j ≤ min(|w 1 tr |, |w 2 tr |) j k=i t 1 k ≈ K j k=i t 2 k =⇒ j k=i t 1 k ≈ K j k=i t 2 k .
where t m k is the k-th delay in w m tr and t m k is the k-th delay in w m tr .

Proof. The proof uses the same arguments as the proof of Proposition 5.5.27 but generalised to all sums of consecutive delays and not only clock values for the sequence of resets considered.

This property assures that the structure of the characteristic set is not broken (i.e. pairs remain K-equivalent). Nevertheless this approach does not respect all the constraints of Equation 5.2. Indeed, adding sometimes makes equivalent two previously non-equivalent delay sequences, as shown on Example 5.5.32. As explained in that example, this is not a failure of that particular algorithm but a larger problem with any zone enlargement.

Example 5.5.32. Consider the sequences of delays and clock resets displayed in Table 5.1 for a set of clocks C = {c a , c b }. The three words w 1 , w 2 and w 3 are K-equivalent and reach at depth 5 a non-open zone with constraint c a = 1. Hence any algorithm looking to enlarge the zone should add (or remove) an at the fourth or fifth delays (Enlarge does it at the fifth). Now consider the alternative sequence of resets where c b is not reset by the second transition. For that reset sequence, the three words are not K-equivalent. Notably, at the end of the fifth delay, the valuation of c b is 2.1 for w 1 , 2 for w 2 and 1.9 for w 3 . Hence adding (resp. removing) an to the total elapsed time makes the valuations of w 1 and w 2 equivalent (resp. w 3 and w 2 ), which violates the constraints of equation 5.2. Hence to satisfy these constraints, an should be removed (resp. added) from the delays 1 2 or 3. Yet, the same problem occurs at depth 3 as the valuation of c b at depth 3 (without the clock reset of c b ) is 1.1 for w 1 , 1 for w 2 and 0.9 for w 3 . 1 2 3 4 5 w 1 0.5 0.4 0.2 0.3 0.7 w 2 0.4 0.4 0.2 0.5 0.5 w 3 0.3 0.4 0.2 0.7 0.3 2.1 + 2.0 + 1.9 + {c a } {c b } {c a } Table 5.1: An example of enlargement making timed words with resets equivalent.

Hence we have here an example of words that cannot be "enlarged" without making some of their previously non-equivalent valuations equivalent.

The enlargement step is the first performed by InvalidityGuard (Algorithm 12), that uses Enlarge to create open replicas of its initial K-closed zones and performs membership queries on the resulting set. InvalidityGuard makes calls to MakeGuard (Algorithm 11) to introduce all necessary guards when terminating. That algorithm takes an ordered list of pairs of K-closed words, and adds validity guards to separate them when necessary.

To ease the algorithms notations, we define the predicate IsOpen that returns true if and only if its parameter is an open K-closed word.

s

(a 1 , z 1 )

r 1 ¬r 1 (a 2 , z 2 ) r 2 ¬r 2
Candidates for Rs 1 (w zr ).

Figure 5.17: The K-closed words tested for Rs 1 w zr with w zr = ((a i , z i , r i )) 1≤i≤2 .

enlarged zones from the original ones by a guard (consistency guard for example). In such case, it is important that we know that a validity guard must still be introduced to separate the pair. A more in depth discussion is made in Section 5.5.4 for the proof of Lemma 5.5.51. We order the pairs from those that are separated by few guards i.e. a non-open K-closed word and its enlargement or a pair of large K-closed words on which we were able to apply a binary search to pairs on which we pick from a greater set of guards-and thus have a lower probability of obtaining a meaningful one.

InvalidityGuard first discusses wether or not the invalidities are open, and in the case they are not enlarges them. Then it discusses on the validity of reset combinations for the enlarged K-closed words with resets. To ground the discussion, we define the set of valid resets after a prefix of a word. Definition 5.5.33. Consider an acceptance function Acc. For a timed word with resets w tr and 0 < i ≤ |w tr |, we call reset set at depth i, noted Rs i (w tr ), the set of valid resets for the i-th transition after the prefix of size i -1 of w tr . Formally, f ∈ { , ⊥} is in Rs i (w tr ) if and only if w tr [1, i -1].(t, a, f ) is valid, with t, a the i-th delay and action of w tr .

We generalize this definition to K-closed words and to a set of timed words with resets which prefix(es) of size i belongs to the same K-closed word.

An illustration of the valid reset is given in Figure 5.17. In the case where the two original K-closed words with resets are open, a binary search is performed to introduce a guard separating them in the form of a validity guard for their linear combinations (see BinComb and BinSearch for more details).

Else, an enlargement is performed and the algorithm discusses according to the validity of the resets at depth |s o |+1 of the enlarged words. Notice that in this case, as enlargements are not linear combinations, we may have to introduce several validity guards or to make Yet, it means forcing an abstract delay of 0 instead of 1 on a K-closed word with resets. This raises a problem for its characteristic set. Suppose we choose to transform w 2 zr in w 1 zr by diminishing its abstract delay at depth two (and augmenting it at depth three) and consider the following (prefix of a) pair of its characteristic set: Diminishing their two first delays to end in c∈C 0 < c < 1 without resets requires to make them equivalent for the reset sequence while they are currently not equivalent.

Remark that this discussion is independent of any algorithm.

Proposition 5.5.36. Let w 1 zr and w 2 zr be two (invalid) K-closed words with resets of same size and actions, with equal resets except for the final one and W 1 , W 2 their characteristic sets. Then w 1 zr , W 1 , w 2 zr , W 2 = BinComb(w 1 zr , W 1 , w 2 zr , W 2 ) respect the constraints (i) (ii).

Proof. We will make the proof using Equations 5.1 and 5.2 instead of constraints (i) and (ii). As already argued this is correct. First notice that all delays given to characteristic set observations are non negative. Indeed, when delays are modified, either the last transition was a resetting one or δ ≥ 1. In both cases we show that the new modified delays are non negative. The modification of a delay is of the form t = t + δ -∆ i with ∆ i = ∆(z i , z temp ) the abstract delays between the zones of the word at index i. If the last transition was resetting, t ≥ ∆ i (as the final zone is open). As furthermore δ ≥ 0 (it is the mean of two natural integers with a floor or ceil approximation) we have t ≥ 0. Else, notice that t ∈ [∆ i -1, ∆ i + 1]. Thus, as we suppose that δ ≥ 1 we have that t ≥ 0.

We start by Equation 5.2. Either BinComb returns the arguments, and the result trivially holds (as the delays are not modified), or it adds the same integer delay to all elements of W 1 (resp. W 2 ) to create W 1 (resp. W 2 ). This cannot affect K-equivalence as the fractional parts are not modified and the integer parts are modified in the same way.

We prove that Equation 5.1 is satisfied using the delay based characterization of zones given in Lemma 5.5.34. Note j ∈ {1, 2}, δ i the value of δ during the i-th iteration of the BinComb's main for loop and ∆ j i = ∆(z j i , z j temp ) during this iteration.

v i-1 + t i = v i-1 + t i + δ i -∆ j i = v i-1 + t i + i k=1 (δ k -∆ j k ) × dir i k (w 1 zr )
by separating the delays added by the algorithm and the pre-existing ones. We hence can say, noting z j i the i-th zone in w 1 zr :

v i-1 + t i ∈ z j i + i k=1 (δ k -∆ j k ) × dir i k (w 1 zr ) = {0} + i k=1 (δ k )
= z 3 i by applying Lemma 5.5.34 twice. We thus have our result by the induction principle.

Finally we prove that the constraints (iii) are satisfied.

The following lemma is the technical core ensuring that BinComb (and BinSearch) respect the constraints (iii).

Lemma 5.5.37. Consider w 1 zr and w 2 zr be two (invalid) K-closed words with resets of same size and actions, with equals resets except for the final one and W 1 , W 2 their characteristic sets. Consider the call w 1 zr , W 1 , w 2 zr , W 2 = BinComb(w 1 zr , W 1 , w 2 zr , W 2 ) and suppose that the result of the function is not its arguments. We note z j i-1

∆ j i -→ z i-1 j (a i ,r i )
---→ z j i the abstract runs for j ∈ {1, 2, 3}, j ∈ {1, 2} corresponding to w j zr and j = 3 corresponding to the behaviours of both w 1 zr and w 2 zr (we abuse the notation as we will not use the final zone and reset).

We have that for any i ≤ i :

i k=i ∆ 3 k =     i k=i ∆ 1 k + ∆ 2 k 2    
if the first approximation made after i is • and

i k=i ∆ 3 k =     i k=i ∆ 1 k + ∆ 2 k 2     otherwise.
Furthermore, in both cases

i k=i ∆ 3 k = i k=i ∆ 1 k + ∆ 2 k 2
if and only if the last approximation made is different from the first one (or no approximation has been made).

Proof. We make the proof by induction on i ≥ i. We suppose that the first approximation is • . The proof for • is similar. For i = i we have ∆ 3 i equals to δ at the step i of the for loop of BinComb. Thus ∆ 3 i =

∆ 1 i +∆ 2 i 2
if no approximation is made (i.e. iff

∆ 1 i +∆ 2 i 2 ∈ N). Else,
as the first approximation is • we have ∆ 3 i =

∆ 1 i +∆ 2 i 2
. In both cases, the property holds.

For the inductive case, suppose that the property holds up to i ≥ i. We prove that it holds for i + 1. For this we discuss according to the last approximation made and the need for an approximation at this index.

If the last approximation made was

• then i k=i ∆ 3 k = i k=i ∆ 1 k +∆ 2 k 2 . Precisely i k=i ∆ 3 k = i k=i ∆ 1 k +∆ 2 k 2
-1 2 . Suppose that no approximation is necessary at index i +1, i.e.

∆ 3 i +1 = ∆ 1 i +1 +∆ 2 i +1

2

∈ N. Then we have our result as one can pass integers under the ceil function and no new approximation can correct the sum to make it equal to the approximated one. Now suppose that an approximation is necessary. Then the approximation applied is • as approximations are alternating. Thus we have that

i +1 k=i ∆ 3 k = i +1 k=i ∆ 1 k +∆ 2 k 2 as ∆ 3 i +1 = ∆ 1 i +1 +∆ 2 i +1
2 + 1 2 . The case where the last approximation made is • follows the same ideas. This lemma is used afterward to show that we do not sum approximation mistakes but always keep an integer approximation of the mean of the original delays.

Combining this wih the chracterization by delays of zones given in Lemma 5.5.34, we can show that the result of a call to BinComb satisfies constraints (iii) with respect to the arguments.

Proposition 5.5.38. Let w 1 zr and w 2 zr be two (invalid) K-closed words with resets of same size and actions, with equals resets except for the final one and W 1 , W 2 their characteristic sets. Consider the call w 1 zr , W 1 , w 2 zr , W 2 = BinComb(w 1 zr , W 1 , w 2 zr , W 2 ) and j, k ∈ {1, 2}. Any separating atomic guard g at depth i for w j zr and w k zr such that w j zr passes in g at depth i verifies that w 3-j zr does not pass in g at depth i.

Proof. We first consider the case where the returns of the BinComb call are its arguments.

In this case the result trivially holds. Indeed, no guard can separate w j zr and w k zr = w k zr for k = j and for k = 3 -j any guard separating w j zr and w 3-j zr indeed separates them. We now reason on the case where a modification is made by BinComb. Consider that g : c ≺ n. Then noting z j i-1 (resp. z j i-1 ) the K-closed zones appearing before the i-th transition in w j zr (resp. w k zr ) and ∆ j i (resp. (∆ k i ) ) the abstract delays leading to it we have by Lemma 5.5.34:

z j i-1 = {0} + i l=1
∆ j l × dir i l (resets(w j zr ))

z j i-1 = {0} + i l=1
∆ k l × dir i l (resets(w j zr )) .

Furthermore we know that dir i l for a given clock c and a fixed i is always 0 until a first l 0 and always 1 afterward.

It comes that for v j i-1 ∈ z j i-1 and v k i-1 ∈ z k i-1 :

v j i-1 (c) ∈ ( i l=l 0 ∆ j l , i l=l 0 ∆ j l + 1) v k i-1 (c) ∈ ( i l=l 0 ∆ k l , i l=l 0 ∆ k l + 1) .
By Lemma 5.5.37 we can conclude, as we know that i l=l 0 ∆ k l is an integer approximation of the mean of i l=l 0 ∆ j l and i l=l 0 ∆ 3-j l .

Remark 5.5.39. Our algorithm is non-deterministic because of the approximation problem: in practice, a branch and bound approach is to be used, that we do not detail here to avoid a more convoluted discussion. Furthermore, to avoid making the algorithm more complex or longer we interrupt the computation as soon as we detect that modifying fractional parts of delays may be necessary. That necessity could be checked in the characteristic set: if all delays are greater than one it is not necessary. Finally, one could choose to still carry on the computation and recreate a characteristic set, even without satisfying (ii). Indeed, we simply need to replicate the invalidity, and it may happen that not completely satisfying (ii) still replicates it.

Remark 5.5.40. Notice that for RERAs with only one clock, the alternation can be reset after each resetting transition as clock values are put back to 0, hence having a nul fractional part.

The algorithm BinSearch is depicted in Algorithm 14 and uses BinComb to generate new K-closed words and corresponding characteristic sets that it turns into membership queries. It then iterates the process to find a pair of K-closed words that are as close as possible (while keeping different invalidities). For that, initial K-closed words are replaced by the results of BinComb, depending on the valid resets at maximum depth. The decisions depending on the valid resets of the replicas are summarized in Table 5.3. Remark 5.5.41. We do not distinguish between the stop case (there is no valid reset) and the return case (both resets are valid) in the implementation to avoid discussing the existence of a return which would make InvalidityGuard longer. In practice, if there is no valid reset, then a predecessor has become invalid and will be pruned, hence there is no point in finding a separating guard (the state without successors will be pruned). Hence it is important to stop the call to InvalidityGuard in order not to waste the computations.

Notice that we only reason on Rs |w 1 zr | (w 1 zr ) because they are the same as Rs |w 1 zr | (w 2 zr ) as these two K-closed words with resets only differ on the last reset choice. The optional argument of BinSearch is used when one wants to return, on top of the initial pair, the first time w 1 zr (resp. w 2 zr ) is modified during the execution. This in turn is used by InvalidityGuard in the case where the two initial invalidities have to be enlarged and the enlargement have different valid resets at depth |s o | + 1. This is because w 1 zr must be separated from w 1 zr and other linear combinations, but as the enlargement is not a linear combination itself, a different guard may be necessary for strict linear combinations. A simple example is given in Example 5.5.42. We show that every subsets of W and W ⊥ are separated by the guards introduced. For any w ∈ W strict and w ⊥ ∈ W ⊥ strict , by Lemma 5.5.45 the guard separating w 1 and w 2 separates them (as w 1 and w 2 are the last linear combinations constructed through BinSearch). The same goes for w 1 zr and w 2 zr as well as for the pair (w 1 zr , w 2 zr ). In the same way, w 1 zr is separated from W ⊥ strict by the guard separating w 1 and w 2 . w 2 zr is separated from W strict in the same way. w 1 zr is explicitly separated from w 1 zr and w 1 first by MakeGuard, and in the same way w 2 zr is explicitly separated from w 2 zr and w 2 first . Hence W and W ⊥ are indeed separated by the guards introduced by InvalidityGuard.

The proposed method relying on a binary search on open zones is thus satisfying, as inconsistencies and decisions states without successors in the subtree have been handled. To verify this would require a prior check.

In practice it may be more efficient to rebuild without that verification, thus handling all detected issues but rebuilding a potentially inconsistent subtree, or with some decisions states having no successors. This would function as a detection method, allowing to treat those (newly discovered) issues and rebuilding again.

We prove in Lemma 5.5.51 that any validity or consistency guard added in G Cons , G V al indeed separates all the observations and K-closed words that must be separated, independently of other guards introduced by the algorithm.

Lemma 5.5.51. Consider a call to AdjPair (resp. InvalidityGuard) in which two sets of timed words with resets W + and W -respectively accepting and non-accepting (resp. W and W ⊥ sets of K-closed words with resets respectively invalid for a sequence of resets terminated by and ⊥) have been requested. Then independently of the other guards in G Cons no pairs in W + × W -can end in the same language state in the subtree rebuilt by Rebuild. Respectively independently of the other guards in G V al no pairs in W × W ⊥ can end in language states child to the same decision state in the subtree rebuilt by Rebuild.

Proof. We separate the proofs for consistency and validity. In the case of AdjPair we know that all valuations in words W + ∪ W -are linear combinations of valuations in the arguments of AdjPair (by Proposition 5.3.8). Thus any guard separating W + ∪ W -either separates exactly W + and W -or separates only one of those two sets in two and the half that stays with the other set contains the observation that carries the consistency guard (as it is the limit linear combination). By Proposition 5.5.9 we have that the sets are separated.

In the case of InvalidityGuard, we first assume that the call did not end with s o the predecessor of the argument s d being invalid. We then know by Lemma 5.5.34 that we can characterize the zones in the K-closed words reached during the binary search as linear combinations (based on ∆) of the initial enlarged zones. We thus have the same result as for AdjPair for the open-zones only. But notice that either a specific validity guard is introduced between the two initial (non-open)K-closed words or they are added to the set for which the validity guard must be raised. Thus in any case, as the validity guard indeed separates pairs of W × W ⊥ by Proposition 5. Notice that the condition for invalidities not to be covered by the children of a given decision state corresponds to the condition of not being in the same language state for inconsistencies. Indeed the resets of these words are different at the last depth only.

Proposition 5.5.52. Consider a valid and consistent language state s N l of the TLG. After adding consistency (resp. validity) guards to G Cons (resp. G V al ) to separate every pair causing an inconsistency (resp. forcing the pruning of all successors of a decision state), if every guard added is at depth strictly greater than |s N l | running Rebuild(s N l ) constructs a subtree rooted in s N l argument that is maximal, complete, well-grounded, consistent with respect to the current acceptance function Acc and well-guarded. Notably, no decision states are left without successors.

Proof. First notice that the subtree built is indeed a TLG subtree as it respects the alternation of decision and observation states, the type of edges, the constraints of guards (i.e. a call to FindGuard returns a partition of its original guard and it is always called on true) and all leaves are observation states. Indeed, by Lemma 5.5.51 we know that no pair encountered during an call to InvalidityGuard can end in both children of a decision state. By hypothesis all such pairs have been treated hence we have our result.

We show each property independently.

Maximality Maximality comes directly from the structure of Rebuild where all successors of a constructed decision state are considered for building and only abandoned if invalid.

Completeness Consider an observation w t ∈ Acc that passes the root s N l of the subtree. As the Rebuild function continuously calls itself as long as an observation passes the current word and no decision state is left without successors, there is by induction a language state s l such that w t ∈ words(s l ).

-closedness is used to construct a RERA by folding the successors of U into comparable states of U . -uniqueness is useful to bound the number of states in U and thus the size of the resulting automaton.

The following lemma (Lemma 6.2 in [START_REF] Grinchtein | Inference of Event-Recording Automatausing Timed Decision Trees[END_REF], a long version of [START_REF] Grinchtein | Inference of Event-Recording Automata Using Timed Decision Trees[END_REF] appearing in O.Grinchtein's thesis [START_REF] Grinchtein | Learning of Timed Systems[END_REF]) ensures that there always exists a satisfying set of states U . For its constructive proof, we refer the reader to the original paper. Lemma 5.6.4. Let be a height-monotone preorder on states in a resulting graph π(N ). Then there exists a -closed and -unique prefix-closed subset of the language states of π(N ).

Using such a subset, we can fold the resulting graph into a RERA as follows: Definition 5.6.5. Let N be a consistent TLG for Acc, π an admissible reset strategy and a preorder on language states of π(N ). Consider a -unique, -closed and prefixclosed subset U of π(N ). Then a U -merging of N according to π is a RERA (U, , C, E, Accept) such that Accept = {u ∈ U | label(u) = {+}} and for any observation state u.(g, a, r) of π(N ) with u ∈ U , there is exactly one edge of the form (u, (g, a, r), u ) ∈ E with u.(g, a, r) u . Notice that, by the second condition, a U -merging RERA is deterministic.

Furthermore, if the observation structure is complete, a U -merging generalizes the observations obtained so far.

Constructing a candidate RERA. Using the results of the previous subsections, we can now construct a candidate RERA from our observation structure. All admissible reset strategies can be constructed by branch and bound. Then a merging is constructed for each resulting graph, and equivalence queries are launched.

For each of the RERA constructed by merging, either a counter-example will be returned by the equivalence query, or the candidate is deemed correct. In the latter case, we return this RERA; in the former case, we include the counter-example in our observation structure and repeat the process.

Conclusion

In this chapter, we propose an active learning method for deterministic reset-optional event recording automata. We add a key feature to the state of the art: invalidity, that allows to detect incorrect guesses of resets when they are not tied to observations. This required to rework all the data structures and algorithms involved to handle invalidity on-the-fly. Most importantly, this brings the lacking notion to scale up to the class of deterministic timed automata (DTAs). Interestingly, the algorithmic handling of invalidity highlights the complex dynamics of undistinguishable classes of valuations and observations, formalized as K-closed zones and K-closed words with resets.

A clear future work is to generalize this method to actually handle DTAs. This mostly requires to handle resets of sets of clocks instead of single ones. As the complexity would be greatly increased, this calls for some optimization. A promising addition would be to use an implicit structure. Instead of storing all possible reset configurations, only storing a small set of them at the same time would decrease the memory cost. As the models are built directly from observations, and not from previous states, the computational overhead may be limited. Another interesting trail for future development is to find a way to build a timed automaton from the observation structure that exploits the different admissible reset strategies without building all of them. Works on approximate determinization of timed automata through games [START_REF] Bertrand | A game approach to determinize timed automata[END_REF] deal with similar problems and offer interesting leads. Finally, in [START_REF] Grinchtein | Inference of Event-Recording Automata Using Timed Decision Trees[END_REF], the authors propose to refine the adjacent pairs into critical pairs, that have a minimal set of differences. This allows to better identify the guards to be added, and thus can have a positive effect on both the size of the constructed models and the computational cost. Sadly, no precise procedure is given to construct the pairs, so creating one would be beneficial to the approach. More generally, studying the efficiency of this algorithm and of the variants proposed as future work could help better understand the applicability and bottlenecks of the approach.

An interesting branch would be to try to integrate invalidity and its characterization to T L * . This could allow to benefit from the algorithm efficiency. One of the main difficulties in this line of research would probably be to integrate the abstractions we propose into T L * that uses the ERA structure to remain at the level of guarded words with resets even for membership queries. In a similar fashion, the T L * algorithm relies on prior learning of the untimed language of the timed automaton. Yet, the untimed language construction is existential (i.e. an untimed word is accepted if it is the projection of an accepted timed word), and the implementation of untimed equivalence of membership queries could be difficult (or require a far greater complexity) in a timed setting, which is not discussed in the original paper. representation of a system set of configurations that can be reached after a given observation, at least for 1-clock automata. We extend the theoretical basis of this structure to general (n-clock) timed automata, but do not propose a sufficient finitely representable class for them. This contribution has a double interest: it allows to avoid the undecidable determinisation problem (in the 1-clock case) by defaulting to a sufficient representation and gives some insight on the structure of time dynamics in timed automata.

• An active learning method for deterministic reset-optional event recoding automata (DRERAs), a new quite general subclass of deterministic timed automata that displays both a high dimensionality and unobservable clock resets (and thus time dynamic). The key of this contribution is the notion of (in)validity that allows to characterize reset choices that can or cannot be part of a model of a set of observations. On top of that notion, we propose data structures to handle the learning of DRERAs, focusing on the representation of their behaviours, and the algorithms necessary to construct, update and exploit them. We believe that the notion of invalidity is the last previously missing piece to handle the learning of deterministic timed automata.

Modeling as abstract interpretation

A guiding thread of the different axes of this thesis is the correct formalisation of the various abstractions underpinning timed automata in term of level of observation-syntax with paths; semantics with runs; behaviours with abstract runs and timed words with resets; observations with timed words-and precisionfrom simple runs (and other observation levels) to minimally distinguishable sets (region or K-equivalent) to general sets defined by paths. These different representations, and the abstraction functions that link them, are presented from the beginning (see Figure 1.4) and further developed in the case of learning (see Figure 5.3). The abstraction functions play an important role in the first chapters of the thesis, while the invert-abstractions are at the core of the discussion on active learning.

Even more generally, and perhaps more notably, laying out these abstractions and their relations is akin to abstract interpretation as the correct representation (i.e. abstract domain) and their properties are central to state estimation (which builds on a new abstract domain) and learning (that sees paths and equivalence classes as an abstract framework that constrains the observations interpretation).

One could see the interactions between models and reality as an interpretation of the "real semantic" (i.e. what is happening in the real world) into specific sound approximations (i.e. formal objects) that both abstract parts of the reality that are deemed irrelevant for the task at hands and organize the relevant information.

Perspectives

The different contributions of this thesis open multiple perspectives for future developments. Firstly, our active learning method is limited to DRERAs, but as discussed earlier, reunites all the key elements to learn deterministic timed automata, thus that extension should be made, in regard for the interest of having a formal learning technique general enough for all learning models. Developing other learning methods for (subclasses of) timed automata could also be of interest, notably reinforcement learning, that is simultaneously historically model-based, linked to formal communities and extremely in vogue among machine learning specialists.

Completing the generalization of the state estimation method of Chapter 4 to general timed automata could also have great practical interests, although it would require new ideas that are not introduced here, as discussed at the end of the chapter.

Finally, developing formal game-based test generation for timed systems would both be interesting for its theoretical implications and practical consequences, due to the far reaching use of tests in the industry. The author believes that further developing such methods greatly depends on the association of testing and learning in mixed methods that would allow to start testing from an initial (partial) model and carry on with increasingly complex and accurate models synthesized through learning. This would allow both to verify properties of the implementation and to construct a model of the system (hopefully of great quality) that helps understanding and analyzing it.

Related to this last point, but outside the scope of timed models, the abstraction based approach supported by this thesis could be used in other domains to intertwine formal learning and acting (i.e. testing, control, diagnostic. . . ), especially for systems that have rich behaviours such as e.g. models with data, a great number of agents (potentially parametric and/or dynamic) or monads. Abstract: This thesis deals with formal methods based on timed automata, their actions upon reality, and the informations that can be learned from it. It proposes different contributions in three separate domains: game theory and formal test generation, seen as a way to control a system using formal methods; state estimation, that deduce the possible configurations of a system from observations by the mean of a formal construction; active model learning, that propose to construct a formal model of a system by requesting observations out of it, directing them as needed for the task.

As diverse as they may seem, these contri-butions are linked by the underlying formalism, abstractions and preoccupations that characterizes interactions between formal models and reality. Furthermore, they benefit from one another in practice, forming a virtuous loop: the capacity to learn from reality allows for better models, that in turn permit a finer control of the real systems, which helps the learning processes.

We link these different contributions together based on the grounds above, and advocate for a greater rapprochement between learning and formal methods and communities.

Figure 1 :

 1 Figure 1 : Boucle entre modèles et réalité

Figure 2 :

 2 Figure 2 : Automate décrivant une simple machine à café.

Figure 3 :

 3 Figure 3 : Un automate temporisé spécifiant une machine à café ; temps en secondes.

Figure 4 :

 4 Figure 4: Reality-Model loop

Figure 5 :

 5 Figure 5: Automaton describing a simple coffee machine.

Figure 6 :

 6 Figure 6: A timed automaton specifying a coffee machine with time in seconds.

Figure 1 . 1 :

 11 Figure 1.1: Clocks encoding the timed elapsed since their last reset in a valuation v.

Figure 1 . 2 :

 12 Figure 1.2: Two zones including a guard, and some valuations.

  The invariants and transitions are represented in the figure. Two paths of this automaton are π 1 = e 1 • e 3 and π 2 = e 1 • e 2 • e 1 • e 3 .

  Figure 1.4: The central notions of timed automata executions.

  Figure 1.5: A 1-clock non-determinizable timed automaton.

Example 1.3. 3 .

 3 In Figure1.6 the equivalence classes for region equivalence and Kequivalence are depicted for two clocks C = {c 1 , c 2 } and maximal constants of 1.Note that both regions and K-closed zones are zones. For a region or a K-closed zone z we define enab( , z) = enab( , v) for any v |= z. Furthermore for a region reg and a transition e ∈ E A , we define next(e, reg) as the unique region reg such that ∀s ∈ ( , reg), s e -→s ∈ reg .

  Figure 1.6: Equivalence classes.

FigureFigure 2 . 1 :

 21 Figure 2.1: The basic active learning framework.

  Figure 3.1: The testing framework.

Example 3.1. 1 .

 1 Figure 3.2: An airport conveyor belt 2 .

Figure 3 .

 3 Figure 3.3: A TAIO specifying a conveyor belt.

Figure 3

 3 Figure 3.4: A timed automaton displaying Zeno-runs with infinitely many discrete actions.

  Figure 3.5: The testing framework.

Figure 3

 3 Figure 3.6: A test purpose for the conveyor belt.

  Definition 3.3.5. Let S = (L S , S init , (Σ ? ∪ {ζ}) Σ ! Σ τ , C S p , I S , E S ∪ Restart) be a specification. An implementation of S is an input-complete and non-blocking TAIO I = (L I , I init , (Σ ? ∪ {ζ}) Σ ! Σ I τ , C I p , I I , E I ). We denote by I(S) the set of possible implementations of S.

Figure 3 . 7 :

 37 Figure 3.7: Product of the conveyor belt specification and the presented test purpose.

  Example 3.3.11. The automaton in Figure 3.8 is a deterministic approximation of the product presented in Figure 3.7. The internal transitions have collapsed, leading to an augmented Start location. true, ?ship 1 , {c} c 2 > 5, ?ship 2 , {c} c ≤ 5, ?ship 2 , {c}

Figure 3

 3 Figure 3.8: A deterministic approximation of the product.

Figure 3

 3 Figure 3.9: A representation of the outcomes of a rank lowering strategy

  Figure 3.11: The ranks in a complex game

  Proposition 3.4.19. The test generation method is strict: given a specification S, ∀I ∈ I(S), ∀(A g , f ) ∈ T C(S), ¬(ParRun(A g , f, I) tioco S) ⇒ I fails (A g , f )

  a} the extension of outputs after a trace to parallel runs. Since DP is an exact determinization of P we have the following equalities: Traces(S) = Traces(P) = Traces(DP) = Traces conf (OT ). Since a ∈ R ≥0 ∪ Σ ! , σ • a ∈ Traces(OT ) as invariants have been removed, and the automaton has been completed on Σ ! with transitions to Fail). Hence σ • a ∈ Traces(Ex fail (OT )). Thus, for ρ ∈ ParRun(A g , f, I) such that trace(ρ) = σ • a, last(ρ) ∈ Fail and I fails (A g , f ).

  f, I) and as σ • a / ∈ Traces(S) and the determinization is exact, σ • a / ∈ Traces conf (OT ) and A g after σ • a ∈ Fail. Hence I fails (A g , f ); • if a is a delay, then a > t, and b ∈ Σ ! . As b is controlled by the implementation, and there is no invariant in A g , σ • a ∈ Outcome(f ). Hence σ • a ∈ ParRun(A g , f, I) and as σ • a / ∈ Traces(S) and the determinization is exact, σ • a / ∈ Traces conf (OT ) and A g after σ • a ∈ Fail. Hence I fails (A g , f ).

Figure 3

 3 Figure3.12: The two kind of edges popped from Waiting.

Figure

  Figure 3.13: An execution of updateRank

Figure 3 .

 3 Figure 3.14: A game with undistinguishable states for the R k hierarchy.

  k+1 . Furthermore, by definition of D k+1 , s µ -→Pass. By aggregating the constraints on V k on t and µ it comes that s tµ -→ Pass proves that s ∈ D k+1 ; • for s ∈ tPred ≤ (Pred Σ (D k+1 ), V k ), we can similarly construct s eµ -→ Pass to prove that s ∈ D k+1 .

Figure 3 .

 3 Figure 3.15: A TGA with two clocks, considering Pass = 5 .

c

  Figure 4.1: A one-clock timed automaton where only the b-transition between 1 and 2 is observable.

Figure 4 . 2 :

 42 Figure 4.2: Two sets (in location 2 ) representing the reachable configurations in the automaton of Figure 4.1 after observing transition b at dates 1.3 (left) and 4.6 (right).

Figure 4 . 3 :

 43 Figure 4.3: The linear timed set f : t → (([-4; -3] ∪ [-2; 0]) + t) ∩ [1; +∞) representing all reachable valuations that can be reached in 2 when observing transition b at time t. The fact that f (0) = ∅ indicates that transition b cannot be taken at time 0.

  Figure 4.4: Markings, linear timed markings, and related operations

  Figure 4.4 is a graphical representation of those different concepts and operations.

Figure 4 . 5 :

 45 Figure 4.5: Example of a regular union of intervals.

  Figure 4.3 displays an example of an atomic timed set T = (X; → 1), with X = [-4; -3] ∪ [-2; 0]. The picture displays the sets f T (0) = ∅, f T (1.3) = [1; 1.3], and f T (4.6) = [1; 1.6] ∪ [2.6; 4.6].

  e.: O a (m)( ) = ( , e∩ e,a,r, )∈E id m( ) ∩ ( e ∩ e) ∪ ( , e∩ e,a,r, )∈E 0 m( ) ∩ ( e ∩ e) [C←0]

Figure 4 . 6 :

 46 Figure 4.6: Relation between the objects and their representations.

  Figure 4.7: The different classes of timed markings.

Figure 4 .

 4 Figure 4.8: A silent timed automaton and its reachable configurations.

Figure 4 . 9 :

 49 Figure 4.9: Link between the different objects and plan of the 1-clock discussion.

Figure 4 .

 4 Figure 4.10: Effect of the gauge operator.

Figure 4 .

 4 Figure 4.11: Representation of the effect of silent transition ( , [a, b], τ, C , ) in three cases.

  Theorem 4.4.19. (M, π) ≡ M π for all π ∈ U * and all linear timed marking M .

  Definition 4.4.20. For a timed marking M and a language L ⊆ U * , we let(M, L) = π∈L (M, π) and we note (M ) = (M, U * ).From this definition and the previous statements, we immediately get: Corollary 4.4.21. For any linear timed marking M , it holds (M ) ≡ M τ . It comes that correctly implements se semantics.

→ 0 )

 0 with the initial location init and f (∅; → 0) with any other location. The τ -closure M τ init = (M init ) of M init associates f ({0}; → 0) with init , and f (I; → 0)

Figure 4 .

 4 Figure 4.12: An automaton whose closure in 1 is (I, → 0).

Figure 4 .

 4 Figure 4.13: An automaton whose closure in 1 is ( ∞ k=q J -k.p; → 0).

init 1 -

 1 Figure 4.14: An automaton having (I 1 ∪ I 2 ; → 0) as closure in 1 .

Figure 4 .

 4 Figure 4.15: An automaton using T to encode the closure in f .

  this will be useful for technical reasons), where U = {([ , g], e ∩ e, ( g, e, → 0), ∅, [ , → 0]) | e = ( , e ∩ e, ∅, τ, ) ∈ U and g ∩ e = ∅} ∪ {([ , g], e ∩ e, ( g, e, g ), C, [ , g ]) | e = ( , e ∩ e, C, τ, ) ∈ U and g ∩ e = ∅ and g = g ∩ e}.

  Definition 4.5.1. For two sets X and Y (a guard) and C ⊆ C, we let

Figure 4 .

 4 Figure 4.19: A simple timed automaton and its associated timed set.

  Figure 5.1: A one-clock non-determinizable RERA.

Figure 5 . 2 :

 52 Figure 5.2: The basic active learning framework.

Algorithm 4 :145

 4 A high-level pseudo-code of an active learning algorithm. Start from the structure representing U = ∅ and the hypothesis H = ∅. 2 while H is not equivalent to the model we want to learn do 3 Process counterexample Make and process membership queries to obtain good properties on U . Fold the data structure into a model H. 6 Return H

  Figure 5.3: Words, runs, models and their relations.

Definition 5.3. 9 .

 9 Consider a zone z defined by a set of constraints of the form c i ≺ k and c i -c j ≺ k with c i , c j ∈ C, ≺ ∈ {<, ≤, =, ≥, >} and k ∈ N. For a vector of integers N = n 1 ... n |C| ∈ Z |C| we note z + N the zone defined by the constraints

d

  and E M = E M . Thus M = M . Example 5.4.4. Consider the RERA of Figure 5.5a with the final location being the only accepting one. If the learner issues requests for the timed words , (0.7, a), (0.7, a)(0.9, a) and (0.7, a)(1.2, a), they end up with the following acceptance function: Acc( ) = -Acc((0.7, a)(0.9, a)) = + Acc((0.7, a)) = -Acc((0.7, a)(1.2, a)) = -The graph of Figure 5.5b represents a labelled TLG implementing this acceptance function Acc; in that figure (and all similar figures in this chapter), circle states are observation states and diamond states are decision states. In that example, all zones (which we see here as guards) contain all clock valuations.

  start a, true, {c a } a, c a ≤ 1, ∅ (a)The RERA providing observations. The TOG implementing Acc.

Figure 5 . 5 :

 55 Figure 5.5: An active-learning setting.

Definition 5.4. 6 .

 6 Consider a timed language graph N and a timed observation graph M. For two observation states s M o = w zr ∈ S M o and s N o = w gr ∈ S N o , we define α(s M o ) = s N o when w zr ⊆ w gr .

Figure 5

 5 Figure 5.6: The TLG obtained after handling an inconsistency in the TLG of Figure 5.5b.

Figure 5 . 7 :

 57 Figure 5.7: The timed observation graph implementing Acc for Example 5.4.10.

  Definition 5.4.16. Let Acc be an acceptance function and M the TOG implementing Acc. When marking the observation states s o of M, construct the following set of pairs of timed words with resets W so :• if |label(s o )| = 2, W so = {(w 1 , w 2 )} for a pair w 1 , w 2 in words(s o ) such that Acc(w 1 ) = Acc(w 2 )• when marking a state s o with two marked successors s o • (z, a ), s o • (z, a⊥), W so = W so•(z,a ) ∪ W so•(z,a⊥) .

Figure 5 . 8 :

 58 Figure 5.8: On the left: the TOG for Example 5.4.18, with its invalid nodes (in grey); on the right: a corresponding maximal TLG.

Figure 5 . 9 :

 59 Figure 5.9: The structure of the update algorithms.

4ww tr := w tr 7 else 8 w

 8 tr := 0.5w tr + 0.5w tr 5 if Request(obs(w tr )) then 6 tr := w tr 9 for i ∈ [0, n], c ∈ C do 10

15w

  tr := λ.w tr + (1 -λ).w tr if Request(trace(w tr )) = + then 16 w tr := w tr 17 else 18 w tr := w tr 19 w tr = 0.5w tr + 0.5w tr 20 if Request(trace(w tr )) = + then 21 Add to G Cons the consistency guards from diff(w tr , w tr ) 22 else w tr = w 1

Figure 5

 5 Figure 5.10: TLG of Figure 5.8 after the addition of a guard splitting the two invalidities. Invalid pruned nodes in grey.

Figure 5

 5 Figure 5.11: TOG after the creation of a new branch, obtained by shifting the observations in the (3, 4) branch in the (2, 3) time interval.

KFigure 5

 5 Figure5.12: Two pairs of timed word with resets that cannot be handled by a direct binary search.

  b) Effect of offsets at depth 1 with and without propagation at depth 2.

Figure 5 .

 5 Figure 5.13: One depth modification.

  The K-closed words before modification.

  At depth two.(b) The K-closed words after modification.

Figure 5 .

 5 Figure 5.14: Issue with binary search on general zones.

  By definition of , k + is not enough to change the integer part of a clock value, as clocks values are sum of consecutive delays. Hence we have our result.If z i+1 is not open, we get our result in the same way as(v i +t i+1 )(c) = (v i +t i+1 )(c)+k. with k ∈ N, 1 ≤ k ≤ i + 1.First for open constraints in z i+1 as before we know that adding k. does not change the integer part and thus cannot make the fractional part null. On

Figure 5

 5 Figure 5.16: Non-open K-closed zones and their direct timed successors.

Lemma 5.5. 30 .

 30 For any K-closed zone z, and any guard of the form g : c ≺ n with n ∈ N, c ∈ C and ≺ ∈ {≤, >} we have that z ∈ g if and only if z ∈ g. Such guard is called enlargement closed guard.

Figure 5 .

 5 Figure 5.18: Two problematic close K-closed words with resets.

(

  w t = (0.7, a)(0.7, a)(0.1, a), w t = (0.3, a)(1.1, a)(0.1, a)) .

Figure 5

 5 Figure 5.20: An example of W and W ⊥ in the case where the enlarged timed words with resets have the complementary valid resets of their initial ones.

  5.46 we have our result. Now consider the case where s o became invalid. In that case, it means that an open K-closed word w zr of size |s o | was found invalid for a given continuation (z, a) and both resets , ⊥. In that case if some of the open zones in W ∪ W ⊥ appear in the subtree it means that a guard has been introduced separating it from w zr . As w zr was the last linear combination constructed when the algorithm stopped, it means that it is separated from the open zones that do not agree with its Rs |so|+1 . Hence again we have our result for the open-zones only. Notice that even when s o 's invalidity is detected, guards are added to G V al to separate non-open K-closed words with resets between them and with open K-closed words. We hence have our result.

Titre:

  Histoire d'un aller et retour : méthodes formelles et apprentissage de modèles pour les systèmes en temps réel Mot clés : Automates temporisés, Théorie des jeux, Apprentissage actif, Méthodes formelles, Estimation d'état Résumé : Cette thèse traite des méthodes formelles utilisant les automates temporisés, de leurs actions sur la réalité, et des informations que l'on peut apprendre grâce à ces observations. Elle propose différentes contributions dans trois domaines distincts : la théorie des jeux et la génération de tests, vue comme un moyen de contrôler un système à l'aide de méthodes formelles ; l'estimation d'état, qui déduit les configurations possibles d'un système à partir d'observations au moyen d'une construction formelle ; l'apprentissage actif de modèles, qui propose de construire un modèle d'un système en lui demandant des observations, en les orientant selon les besoins de la tâche. Aussi diverses qu'elles puissent paraître, ces contributions sont liées par le formalisme sous-jacent, les abstractions utilisées et les préoccupations qui caractérisent les interactions entre les modèles formels et la réalité. De plus, elles bénéficient les unes des autres dans la pratique, formant un cercle vertueux : la capacité d'apprendre de la réalité permet de meilleurs modèles, qui à leur tour permettent un contrôle plus fin des systèmes, ce qui favorise les processus d'apprentissage. Nous relions ces différentes contributions entre elles sur la base des motifs ci-dessus, et nous plaidons pour un rapprochement entre les méthodes et les communautés d'apprentissage de modèles et de méthodes formelles. Title: There and back again : formal methods and model learning for real-time systems Keywords: Timed automata, Game Theory, Active learning, Formal methods, State estimation

Example 3.2.4. Figure

	c = 1					c = 1
	!end 1 {c}	true ?ship 1			true ?ship 2	!end 2 {c}
	Dest 1	{c}	Boarding		{x}	Dest 2
	true		c ≤ 3			true
			true
	c = 3 !past		τ {c}			c = 1 τ
	{c}			true		{c}
			!waste
	true ?ζ		Sort c ≤ 1	{c}	Waste c ≤ 1	true ?ζ
	{c}					{c}
		c ≤ 2	true
		τ			?ζ
		{c}	{c}
		start	Start c ≤ 2
	3.3 is an example of TAIO specifying a conveyor belt such as
	the one of Example 3.1.1. It uses one proper clock c (no observed one) that is used in
	invariants to bound the sojourn time in locations. It has as inputs ?ship 1 and ?ship 2 , and a
	special restart action ?ζ (see later), outputs !end 1 , !end 2 , !past, !waste, and internal action
	τ . After a maximum of 2 time units in location Start (depending for example on their
	weight), packages reach a sorting point in location Sort, where they are automatically
	sorted between packages to reject and packages to ship. Packages to reject go to location
	W aste, while packages to ship are sent to a boarding platform (location Boarding), where
	an operator can send them to two different destinations Dest 1 or Dest 2 . If the operator
	takes more than 3 time units to select a destination, the package goes past the boarding
	platform and restarts the process. The restart action ?ζ also allows to go back to Start
	from several locations.

  Hence the property is satisfied. During the loop execution, the property is preserved. Indeed, if e = (s, α, s ) is the considered edge, if s ∈ Visited the only array taken out of Waiting is e and it is added to Depend(s ). Furthermore no state is added to Visited, hence the property is preserved. Else, s / ∈ Visited and (1) e is added to Depend(s ) which ensures the property for Visited \ {s };

  between 1 and 2 is observable.

	values in [1; 1.3] can be reached in 2 if we observe b at date 1.3	values in [1; 1.6] ∪ [2.6; 4.6] can be reached in 2 if we observe b at date 4.6
	0	1	2	3	4	5	0	1	2	3	4	5

  the set of observation states: it is a non-empty prefixclosed set of zone words with resets. We write s 0 o for the empty word, which always belongs to S o . With any state s o ∈ S o , we associate the zone ζ(s o ) reached after the zone word with reset s o , and defined inductively as follows:

  is the set of decision states; a state s d = (s o , z, a) represents the situation where we observe the occurrence of action a, and have to decide whether clock c a will be reset. We require that for anys d = (s o , z, a), it holds ζ(s o ) ∩ z = ∅; -for any s o , for any a, if both (s o , z, a) and (s o , z , a) are in S d , then z = z or z ∩ z = ∅.Edges are defined as follows: we haveE = E od E do ⊆ S × ((Z × Σ) ∪ { , ⊥}) × S,such that for any s o ∈ S o and any s d ∈ S d : • there is an edge (s o , f, s d ) ∈ E od if, and only if, f is of the form (z, a) and s d = (s o , z, a); • there is an edge (s d , f, s o ) ∈ E do if, and only if, f ∈ { , ⊥} and, writing s d = (s o , z, a), we have s o = s o • (z, a, f ). that by our definition, for all s d = (s o , z, a), the state s o belongs to S o . Symmetrically, we require that for all s o = s o • (z, a, f ), the state (s o , z, a) ∈ S d . We use timed decision graphs to represent and manipulate observations stored in the Acc function (i.e. in Dom(Acc)). Fix such a function Acc, and a TDG D. With any observation state s o ∈ S o of D, corresponding to a zone word with resets w zr , we associate the (possibly empty) set of timed words that are observation-compatible with it, formally: words(s o ) = {w t ∈ Dom(Acc) | w t w zr }. ) = {Acc(w t ) | w t ∈ words(s o )}. The labelled TDG associated with D and Acc is then defined as the pair D Acc = (D, label). all labelled TDGs are relevant to handle acceptance functions. A labelled TDG D Acc is said • complete (w.r.t. Acc) when Dom(Acc) ⊆ so∈So words(s o ); • well-grounded (w.r.t. Acc) if for any observation state s o ∈ S o it holds words(s o ) = ∅.

	We also let
	label(s o Not
	Notice This way, our timed
	decision structures always are connected, tree-shaped graphs.
	Finally, we require that all decision states have at least one successor, so that all leaves
	are observation states.

  We will first prove that S M o ⊆ S M o . As M is well-grounded w.r.t. Acc, for any s o ∈ S M o . there exist some w t ∈ Dom(Acc) ∩ words(s o ). And as M is complete, there also exists a state s o ∈ S M o such that w t ∈ words(s o ). Furthermore, since by definition of TOGs all decision states in M have two successors, one can choose s o with the same sequence of resets as s o , i.e. r = resets(s o ) = resets(s o ). Now, given w t and r, the K-closed word with resets w zr such that (w t ⊗ r) ∈ w zr is unique, hence we get s o = w zr = s o .

	It comes that S M o ⊆ S M o . We can show the opposite inclusion by symmetry, and thus
	S M o = S M o . By the definition of TOGs, this implies that S M d = S M

  Consider a complete TLG N for Acc and a TOG M implementing Acc. Then α is well-defined. It is surjective if, and only if, the TLG N is well-grounded (thus implements Acc). Furthermore, for all s M

	Proposition 5.4.7. o	∈ S M o , words(s M o ) ⊆ words(α(s M o )), (and
	consequently label(s M o ) ⊆ label(α(s M o ))).

•

  If the TLG is well-grounded, then for any s N o ∈ S N o , there is a timed word w t of Acc in words(s N o ). By completeness of M w.r.t. Acc, there exists some observation state By well-groundedness of M, there is w t ∈ Acc such that w t ∈ words(s M o ) and by definition of α, we get w t w gr . Well-groundedness of N then follows by definition of words.

	s M o ∈ S M o , with w t ∈ words(s M o ), and thus α(s M o ) = s N o . Thus α is surjective.
	Conversely, suppose that α is surjective. Then for any s N o = w gr ∈ S N o , there exists
	s M o ∈ S M o such that α(s M o ) = s N o . • The inclusion words(s M o ) ⊆ words(α(s M o )) comes from the fact that any observation
	that belongs to s M o also belongs to α(s M o ).

  Definition 5.4.8. A guarded word with resets w gr is said consistent with respect to an acceptance function Acc when for all w t , w t ∈ Dom(Acc), if w t and w t are compatible with w gr then Acc(w t ) = Acc(w t ). Otherwise, w gr is said inconsistent.

  observation state s o ∈ S N o is inconsistent if and only if |label(s o )| = 2. Proof. By definition, s o is inconsistent if and only if there exists two timed words w t , w t in Dom(Acc) compatible with s o , but Acc(w t ) = Acc(w t ). By definition of labelled TLGs, words and label, this is equivalent to w t , w t ∈ words(s o ) such that Acc(w t ) = Acc(w t ), which is equivalent to |label(s o )| = 2. Then, we say that a timed language graph N Acc is consistent when for any s o ∈ S o , we have |label(s o )| = 1; it is said inconsistent otherwise.

  any observation state s o with |label(s o )| = 2 is marked; • if, for some observation state s o , there is an action a and a K-closed zone z such that both observation states corresponding to s o • (z, a, ) and s o • (z, a, ⊥) exist and are marked in M Acc , then s o is marked; A K-closed word with resets w zr ∈ S M Acc Consider a TOG M Acc implementing Acc and an observation state s o = w zr .

	o	is invalid w.r.t. Acc if, and only if it has a
	marked predecessor.
	Proof.

  Definition 5.4.17. A TLG N is said maximal if for any decision state s d = (s o , z, a) ∈ S d and for r ∈ { , ⊥}, there is (s d , r, s o • (z, a, r)) ∈ E if, and only if, s o • (z, a, r) is valid.

  Algorithm 5: Adding a new observed timed word p t • w t in N and M 1 def FindPath (p t , w t , v, s o ): Input: p t : prefix timed word; w t : suffix timed word; v: valuation after p t ; s o : state of TOG after p t := s o .(Kz(v + t), a, ) and s o := s o .(Kz(v + t), a, ⊥) in S M o create transitions (s o , (Kz(v + t), a), s d ), (s d , , s o ) and (s d , ⊥, s o ) in E M if ∃s c = (α(s o ), g, a) ∈ S N d such that v + t ∈ g then // Handling N let s l , s l in S N o such that (s c , , s l ) and (s c , ⊥, s l ) in E N

	2	if w t = then	// end of word reached
	3	add p t to words(s o ) and words(α(s o ))
	4	if |label(s o )| = 2 then	// invalidity detected in TOG
		s o .invalid := true
		s

d := parent(s o ) while both successors of s d are invalid do // seek invalidity root s o := parent(s d ) s o .invalid := T rue s d := parent(s o ) delete transition entering α(s o ) and subtree rooted at α(s o ) from M schedule a call to Init-InvalidityGuard(s o ) 13 else if |label(α(s o ))| = 2 then // inconsistency detected in TLG pick w t and w t in words(α(s o )) s.t. Acc(w t ) = Acc(w t ) schedule a call to AdjPair(w t ⊗ (resets(s o )), w t ⊗ (resets(s o ))) 17 else // w t not empty 18 (t, a) • w t := w t 19 if there exist z and s d s.t. (s o , (z, a), s d ) ∈ E M and v + t ∈ z then for (s d , r, s o ) ∈ E M do FindPath(p t • (t, a), w t , (v + t) [r←0] , s o ) 22 else AddWord(p t , w t , v, s o ) Algorithm 6: Extending M and N to include a new timed word def AddWord (p t , w t , v, s o ): Input: p t : prefix timed word; w t : suffix timed word; v: valuation after p t ; s o : state of TOG after p t (t, a) • w t := w t create state s d := (s o , Kz(v + t), a) in S M d // Handling M create states s o else create state s c := (α(s o ), true, a) in S N d create states s l := α(s o ).(true, a, ) and s l := α(s o ).(true, a, ⊥) in S N l create transitions (α(s o ), (true, a), s c ), (s c , , s l ) and (s c , ⊥, s l ) in E N define α(s o ) := s l and α(s o ) := s l Request (p t • (t, a)) // get status of p t • (t, a) add p t • (t, a) to words(s o ), words(s o ), words(s l ), and words(s l ) if |label(s l )| = 2 then // inconsistency detected at node s l (and s l ) pick w t and w t in words(s l ) s.t. Acc(w t ) = Acc(w t ) schedule a call to AdjPair(w t ⊗ (resets(s o )), w t ⊗ (resets(s o ))) if w t = then // proceed recursively with the suffix for (s d , r, s o ) ∈ E M do 20 AddWord(p t • (t, a), w t , (v + t) [r←0] , s o ) Algorithm 7: Requesting an observation. def Request (w t ): Input: w t : timed word if w t ∈ Dom(Acc) then return Acc(w t ) else Make a membership query on w t and set its result o to Acc(w t ) FindPath( , w t , 0, s ) return o

  The past p t storing that word is added to words of both s o and α(s o ). Hence the subgraphs are complete with respect to the new observation (by adding delays and actions no other reachable states can correspond to that same word) and every new observation requested during the recursive calls (as they do not reach it). No other states are created, hence the hypothesis on the initial structures suffice to conclude that the subgraphs are complete and well-grounded with respect to the valid part of Acc , as by Proposition 5.5.2 only invalid states w.r.t. Acc are pruned. The edges constructed by this call agree with the definition, and by the hypothesis on the initial observation graph, the other successors of s o are complete (except with respect to the new word, but their sequence of letters and K-closed zones do not match) and well grounded, hence in all cases we obtain a subgraph of M that is complete (as s l and s l have no successors) and well grounded.We now discuss the properties of N . There are two possibilities. Either a decision state covering (t, a) from α(s o ) exists, in which case the behaviour of AddWord mimics the one of FindPath, and we can conclude using the previous point, or there is no such decision state. In this case, by definition of a TLG, we know that no decision state for letter a exist as successors of α(s o ). Hence the new decision and observation states created in N correspond to the definition of a TLG. Furthermore, the observation states are well-grounded, as p t .(t, a) is added to their set of words, and this, together with the initial hypothesis on N implementing Acc, ensures that the subgraph is complete w.r.t. Acc .• Inductive case of FindPath. We consider that w t = (t, a).w t . Thus we enter the else part in line 2. If the else case is called in line 22, we only make a call to AddWord on w t hence by induction hypothesis, we have the desired properties. Else, as there is only one guard such that w t can go through that guard (by unicity of the K-closed set containing a timed word with resets), all successors satisfying a prefix of w t are reached by the recursive calls and by induction hypothesis they lead to implementing subgraphs for both the TOG and TLG. Furthermore, other successors of s o (resp.

• Basic case for AddWord. We first discuss the properties of M. As AddWord is never called on empty words, we have w t = (t, a). The call to AddWord adds a new decision state s d = (s o , a, Kz(v + t)) and two new language nodes s o and s o corresponding to the effect of resetting or not c a after the action. As AddWord was called in FindPath, we know that no successor covers (t, a) in s o . As we are in the base case, words of s o and s o are augmented with the observation p t • (t, a), making them complete with respect to Acc (as no other timed word in Dom(Acc ) reaches this state) and well-grounded.

  , r i ) 1≤i≤n sharing the same length, actions and resets. Then c ≺ k, with c ∈ C, ≺∈ {<, ≤}, and k ∈ N, is called a validity guard at depth 1 ≤ i ≤ n if, and only if, for all j > i, it holds z 1 j ≈ K z 2 j and c ≺ k separates z 1

	Definition 5.5.13. Consider two K-closed words with resets w 1 zr = (z 1 i , a i , r i ) 1≤i≤n and
	w 2 zr = (z 2 i , a i i and z 2 i ( i.e., either z 1 i ∈ c ≺ k and
	z 2 i ∈ c ≺ k, or the converse). We then say that (c ≺ k, i, w 1 zr , w 2 zr ) is a validity guard.
	We extend the definition of validity guard for sets of K-closed words by saying that
	(c ≺ k, i, W 1 zr , W 2 zr ) is a validity guard if there is w 1 zr ∈ W 1 zr and w 2 zr ∈ W 2 zr such that
	(c ≺ k, i, w 1 zr , w 2

zr ) is a validity guard and for all pairs in W 1 zr × W 2 zr , c ≺ k separates their zones at depth i.

  Notably, K-closed zones can have different forms, being open or closed in different dimensions (i.e. with constraints of the form c = n or n < c < n + 1), which will make preserving this property more challenging. The second point-having to preserve K-equivalence for other sequences of resets-induces that it is highly difficult to replicate the characteristic sets in K-closed words that have finer constraints than the initial K-closed word ( i.e. equality constraint replacing a pair of inequalities). In Example 5.5.15 for instance, determining the behaviour for c

	Remark 5.5.16.

a = 2 requires to collapse (2.7, a)(1.1, a) and (2.9, a)(1.1, a) which both transform into (2, a)(1.1, a). Thus no invalidity could be detected.

Remark 5.5.17. Points (i) and (iii) are absolutely necessary: without always satisfying them, an algorithm becomes meaningless, as it either fails to displace the observations or to introduce meaningful guards separating the sets of K-closed words with resets.

À cause de leur précision, les méthodes formelles peuvent être à la fois très complexes à utiliser et calculatoirement coûteuses. De ce fait, il est nécessaire d'utiliser des abstractions appropriées afin de concentrer l'expressivité sur les aspects importants d'un système, tout en simplifiant la tâche à accomplir.Dans le cas des systèmes critiques, plusieurs types d'assurances complémentaires sont recherchées : la vérification formelle donne des preuves de propriétés de haut-niveau, mais dépend de modèles souvent très abstraits ; l'analyse de programmes travaille directement sur le code, qui doit donc être disponible, et dérive des propriétés souvent moins complexes et de type différent ; le test ne peut assurer de propriétés positives (par exemple qu'une erreur ne va pas arriver) mais peut interagir avec le produit final (incluant l'implémentation physique et l'environnement) ; la surveillance au moment de l'exécution peut rectifier ou arrêter les comportements défectueux au fur et à mesure qu'ils se produisent, mais ses options sont plus limitées (par exemple, elle doit parfois arrêter le système, ce qui peut être un problème).

A first conference version was published in 1990.

See this chapter's epigraph.

Briefcase icon, commercial airplane icon and person icon by Delapouite under CC BY

3.0 from https://game-icons.net/.

This definition is valid because the two first conditions in Def. 1.3.1 ensure that guards cannot distinguish between valuations in a given region. Hence the enabled transitions are the same.

Observe that writing reg t -→ reg for a delay t is execution-specific, as that delay may lead to a region reg = reg from some configurations in reg.

The determinization procedure in[START_REF] Bertrand | A game approach to determinize timed automata[END_REF] also realizes the closure.

The sequence (W j i ) i is non-decreasing, and can be computed in terms of clock regions; hence the limit exists and is reached in a finite number of iterations[START_REF] Cassez | Efficient On-the-fly Algorithms for the Analysis of Timed Games[END_REF].

The specific order of computation of the transitions does not impact termination or correctness, but can be of great importance for the efficiency.

((X, r), π) = 1 ( ( ((X, r), π 1 ), e), π

).

Both open and closed intervals may appear in the same element on different dimensions

Add to G Cons the consistency guards from diff(w tr , w tr )

The length n of the timed words is greater than the length of w zr .

Then O a (m) is the marking representing the set of configurations that can be reached after taking an a-transition from the configurations represented by m.

Remark 4.2.3. O a (m) does not represent the set of configurations that can be reached after observing an a-transition, as this would include the possibility of taking unobservable transitions instantly before or after the a-transition. The corresponding marking is

). We choose this definition because it concentrates the challenges related to silent transitions exclusively in the computation of O t .

When all transitions in E are observable (i.e. are labelled by observable letters in Σ), the operations O a and O t can be easily computed. O a consists in taking all transitions labelled by a that are enabled in some item of the marking, while O t amounts to adding t to each item of the marking (in other terms, for any marking m, any state ∈ L, and any v ∈ R ≥0 , we have v ∈ m( ) if, and only if, v + t ∈ O t (m)( )).

When timed automata contain a silent (i.e. unobservable) action τ , the computation becomes more complex. Now ( , v) t → ε ( , v ) indicates a sequence of zero or more silent transitions in t time units; in that case we may have = .

The function O t cannot be computed anymore by just shifting valuations by t. In its raw form, the function O t can be obtained by the computation of the set of reachable configurations in a delay t by following (arbitrarily long sequences of) silent transitions. This is analogous to the method proposed by Tripakis [START_REF] Tripakis | Fault Diagnosis for Timed Automata[END_REF] for keeping track of the set of all possible configurations the automaton can be in after observation λ ∈ Σ * and delay t. In [START_REF] Tripakis | Fault Diagnosis for Timed Automata[END_REF], the set of possible configurations is updated each time a new action is observed (or after a time out, if no new observations occur); this approach may increase the delay for detecting the occurrence of faulty actions.

Example 4.2.4. Consider again the one-clock timed automaton of Figure 4.1, and the (initial) marking m 0 which maps 0 to the single valuation v 0 such that v 0 (x) = 0, and locations 1 and 2 to the empty set. This marking corresponds to a single valuation.

Assume that we observe transition b after 1.3 time units. Obviously, the automaton must have taken one of the transitions from 1 to 2 , but since they are unobservable, Notice how by adding delays t ∈ R to these sets we can see that the gauge corresponds to the resets of clocks for valuations in X + t ∩ Y .

Proof. The characterization 4.2 is simply obtained from the definition 4.1 by manipulating the +t and considering valuations and zones as vectors in R n . 4.4 is obtained similarly from 4.3.

Hence, it remains to prove 4.3. The core of the work is to show that the two occurrences of t in 4.1 can be made independent by adding the constraint v ∈ X [C ←0] -1 . First, we can see that

as by definition t ∈ I. Furthermore, X C Y ⊆ X [C ←0] -1 as by definition of the gauge for v ∈ X C Y and c ∈ C \ C , there exists v ∈ X such that v(c) = v (c) (as argued in Remark 4.5.2). We now show that

. By definition, there exists

It remains to prove that (K -t 2 ) ∩ X is not empty. For this, we translate the problem and prove that K ∩ (X + t 2 ) is not empty.

The rest of the proof is geometric and relies heavily on the structure of zones and guards. It constructs an element of K ∩ (X + t 2 ). We note

By definition of t 2 ∈ I we know that these three sets are not empty. We will use this to construct an element of K ∩ (X + t 2 ). By 4.3 there is

We then have that p 2 ∈ P and respects all diagonal constraints of X + t 2 (as p 1 ∈ X + t 2 and diagonal constraints are invariant by time elapsing).

simpler expressions than the general case (which corresponds to ) C . Remark 4.5.8. Unlike in the 1-clock case, we do not separate the guard g between upper and lower constraints ( e and e). Indeed, the interactions between the upper and lower ones induce diagonal constraints that would not appear with only the upper (resp. lower) ones and are necessary to correctly model the behaviours.

The definition of is illustrated in Figure 4.18 for a two-clock automaton ( r is represented by the non-grayed-out areas). Definition 4.5.6 (cont.). As in the one-clock case, we extend to sequences of transitions inductively by letting ((X; r), ε) = (X; r) and, for π ∈ U + ,

Finally we extend this definition to linear timed sets by letting

Using these definitions, we can prove that corresponds to the effect of a sequence of transitions from a timed set. The following lemma extends Lemma 4.4.17. Notice that the proof has to be adapted, not only to the new gauge definition but also to the diagonal constraints arising in timed automata with multiple clocks. Lemma 4.5.9. Let T be a linear timed set and π ∈ U * . Then for any t ∈ R ≥0 and any

Proof. We carry the proof for the case where T = (X; r) is an atomic timed set. The extension to unions of atomic timed sets is straightforward. We begin with the case where π is a single transition e = ( , Y = e ∩ e, τ, C , ). In case X is empty, then also f (T,e) (t) is empty, and ∀t 0 , f T (t 0 ) = ∅, so the result holds. We now assume that X is not empty, and consider three cases:

• if Post(X) ∩ r ∩ Y = ∅, then (T, e) = (∅; → 0). On the other hand, for any t 0 and any v ∈ f T (t 0 ), it holds v ∈ Post(X) ∩ r, so that v / ∈ Y , and the transition cannot Chapter 5

ACTIVE LEARNING

Never laugh at live dragons, Bilbo you fool! -J.R.R. Tolkien "The Hobbit"

In this chapter, we focus on model learning of a new sub-class of timed automata: reset-optional event recording automata. Fundamentally, learning corresponds to the construction of the inverse functions trace -1 and sig -1 that would allow to deduce a run (of a given model) from an observation.

The work displayed in this chapter focuses on the inference, representation and management of the information necessary to the construction of the inverse functions, i.e. to find and represent the semantics of a timed language under a form fitting a timed automaton.

Introduction

As discussed in Section 2.3.2, active learning is a fitting learning framework for timed automata, as it can overcome passive learning expressivity limitations. For this reason, some contributions have already been made in this domain, although limited to somewhat simple models: automata with one clock displaying low-dimensional behaviours and eventrecording automata where clock resets can be directly inferred from observations. Furthermore, by its interacting nature, active learning is a method of interest to ground a general theoretical framework for a learning-acting loop between (timed) models and reality.

Remark 5.1.1. It is interesting to note that reinforcement learning, a kind of learning stemming from the artificial-intelligence community, would also be a good candidate for such loop. Yet, reinforcement learning theory usually relies on stochastic models and have been less developed (to the author's knowledge) for timed automata, potentially due to the complex nature of probabilistic timed automata [START_REF] Sutton | Reinforcement Learning: An Introduction[END_REF]. explained in the next sections. Notice that they are not run until FindPath terminates; this is precisely to simplify the termination proof of FindPath.

Remark 5.5.1. Notice the way FindPath handles invalidities: it finds the root of the invalidity and marks it in the TOG. In the TLG the corresponding observation state is pruned. This approach is somewhat lazy as it chooses to prune a node that covers potentially more than the invalidity (and possibly valid observations), without reintroducing ones that cover its valid observations. This is made to avoid both the computational cost and the blow-up of the model (as it would require a great number of new states). This pruning is not problematic as long as any decision states has at least one un-pruned successor. When all successors are pruned, we use algorithm InvalidityGuard (Algorithm 12) to introduce a guard and create new decision states that have valid successors, as discussed in Section 5.5.3.

Function AddWord recursively creates the nodes in the TOG and TLG as needed. It launches new calls to Request for each intermediary word, which in turn calls FindPath in order to add the new information in all corresponding nodes and checks for inconsistencies and invalidities.

The following statements express soundness of our algorithms. They ensure that the good properties of the structures are preserved by the call to FindPath. Note that we do not consider the consequences of the scheduled calls to InvalidityGuard and AdjPair in the following properties. They will be discussed in the following subsections.

Proposition 5.5.2 ensures that the pruning made during the execution of the FindPath algorithm prunes exactly the invalid states of the TLG (ensuring maximality) and that the corresponding states of the TOG are marked. Proposition 5.5.3 states that FindPath keeps the implementation properties of the TOG and TLG with respect to the updated set of observations. Notice that these algorithms do not handle consistency, but only schedule a call to AdjPair that will handle it later.

We note NewPref M (w t ) the set of prefixes of a timed word that is not covered by the TOG M, i.e.

o } where a timed word is compatible with a set of zone word with resets if and only if it is compatible with one of its elements. Proof. Invalidity is detected along the calls to FindPath, and the propagation of the invalid tag follows the definition of marks for all ascendant states. No descendant are tagged, but this does not matter as they cannot be reached without passing by marked states as M is a tree. A call to SearchPrune is then made, that targets exactly the root of the invalid subtree that has been detected, and prune it in N . As this is made for all detected invalidities and the subtrees are detected, when the procedure terminates, no language state invalid because of an invalidity detected in FindPath can be reached. Furthermore only the invalid subtree is suppressed, hence no valid states are made unreachable (as by definition all descendant of invalid states are invalid).

To conclude, it suffices to notice that every new membership query gives rise to a corresponding call to FindPath, leaving no invalidities undetected.

Proposition 5.5.3. Starting from an acceptance function Acc, a TLG N and a TOG M implementing the valid part of Acc and a timed word w t / ∈ Dom(Acc), we consider Acc be such that Dom(Acc ) = Dom(Acc) ∪ {w t } ∪ NewPref M (w t ) and for all w t ∈ Dom(Acc),

A call to FindPath( , w t , 0, s ) with s the root of M modifies M and N in such a way that they implement the valid part of Acc .

Proof. First of all, a call to FindPath terminates, as recursive calls are in finite number and on words of strictly decreasing length, there is at most one call to AddWord or SearchPrune from FindPath along each explored path and recursive calls to AddWord are in finite number and on words of strictly lower length. Thus, there is a finite number of calls to Request, and if these calls make calls to FindPath for new observations, the timed words given as arguments are of strictly lesser length.

An algorithm trying to reduce the distance between them at depth one without breaking constraint (iii) at this depth would likely propose to replace their first zones by 2 < c a < 3 ( i.e. an offset of 2 × 1 ca for w 1 zr and -2 × 1 ca for w 2 zr ). If the effect of this modification is not propagated at depth two, then w 1 zr would be transformed in

which cannot be satisfied by any timed word, as it would require a negative delay. On the other hand, applying the effect of this transformation With the effect of this modification on the zones at depth 2 the same offset would be applied on the zone at depth two, resulting in the words

Because of this, the guard g : c a > 3 separates the zones at depth two in a different way for (w 1 zr , w 2 zr ) on one hand ( i.e. z 1 2 / ∈ g and z 2 2 ∈ g) and (w 1 zr , w 2 zr ) ( i.e. z 1 2 ∈ g and z 2 2 / ∈ g).

The zone words after modification for both choices (propagating the offsets or not) are represented in Figure 5.13b.

A possibility is to mimic the binary search employed for inconsistency: use delays between subsequent zones (or an abstract representation of such delays at least) as the metric we want to converge on. For this we define a pseudo-distance ∆ on zones as follows.

Definition 5.5.21. Consider two K-closed zones z 1 and z 2 . We define ∆(z 1 , z 2 ) as: All the aforementioned approaches cannot be used, as they either fail to construct meaningful K-closed words with resets or to satisfy constraint (iii). In the following, we propose a method that will always satisfy constraints (i) and (iii), but will fail to satisfy constraint (ii) in a specific step. We argue that this is more desirable as loosing the invalidity only impacts efficiency (the introduced guard is less precise and may have to be refined) instead of correctness by e.g. introducing guards that do not separate all targeted behaviours i.e. failing constraint (iii).

Binary search on open zones

We build on the above binary search method, but try to avoid the difficult handling of

K-closed zones of different form (open or closed in different dimensions).

The solution we propose is based on a binary search on open zones, which satisfies all three constraints (as proven below). That binary search is initialized with a zone enlargement phase. This step respects constraints (i) and (iii) but not condition (ii) (a counter example is shown below).

The general algorithm is InvalidityGuard, described in Algorithm 12. It takes as input the decision state of the TLG which successors have been pruned, along with the characteristic sets of the invalidities, and adds a set of validity guards to the set G V al of validity guards using algorithm MakeGuard (Algorithm 11). The algorithm depends on BinSearch (Algorithm 14) that handles the binary search on open zones and Enlarge (Algorithm 10) that proposes open K-closed words replicas of the initial invalidities. BinSearch itself depends on BinComb to propose new K-closed words replicas (and their characteristic sets). The algorithmic structure is summarized in Figure 5.15.

We first describe the properties of the enlargement phase, then of the binary search on open zones and finally of InvalidityGuard in general.

The Enlarge algorithm is depicted in Algorithm 10. Its idea is to replace equality constraints in the K-closed words by large ones (i.e. k < c < k + 1), based on the following fact.

Lemma 5.5.26. Any K-closed zone z that is not open has one unique direct time successor K-closed zone, that we note z . For any constraint

Proof. We first prove that z is well defined. For this consider a zone z that is not open and take a valuation v in that zone. Then there is some clocks

. This being true for any valuation of z, and the constraints on the new zone depending uniquely on the constraints of z, we have that z is well-defined and open.

The Enlarge algorithm takes as parameters an invalid K-closed word with resets w 1 zr , its characteristic set W and a positive real .

It simply replaces all non-open zones in w 1

zr by their direct time successors, and add to the delays of the words. We show that for a fitting (small) value of this algorithm respects equation 5.1.

Algorithm 11: Creates validity guards to separate an ordered list of pairs of K-closed words with resets.

MakeGuard Input: Two sets W 1 and W 2 of K-closed words with resets that have to be separated by validity guards. Output: A set of validity guards separating the two sets.

The main purpose of MakeGuard is to avoid to add validity guards to G V al when another validity guard already separates a pair of incompatible invalidities. In this case, we nevertheless add the pair to the set of the validity guard. This is a consequence of the enlargement phase: as enlargement is not a linear combination, it is possible to separate validity guards depend on sets of words to ensure that the guards will be available when needed when reconstructing the TLG (this is discussed in the proof of Lemma 5.5.51). The discussion in the case where both initial invalidities have to be enlarged is summarized in Table 5.2.

After the enlargement step, InvalidityGuard either terminates or calls BinSearch that performs a binary search between two open K-closed words. The procedure constructing new K-closed words and characteristic sets from given ones, called BinComb, is described in Algorithm 13.

The BinComb algorithm constructs new K-closed sets and characteristic sets that meet constraints (i), (ii) and (iii). The core of its work is to find the middle (in the sense of ∆) between the abstract delays (i.e. the ∆ between the zone after the (i -1)-th reset and the zone before i-th transition) of the two K-closed words with resets given as arguments and construct both the new K-closed word using these middle zones and a characteristic set for it. The main difficulty lies in the nature of distances between (open) zones: they must remain in N. Hence it is sometimes necessary to round the mean up or down.

The fact it is sufficient to reason on abstract delays comes from the following lemma characterizing zones from delays, resets and directions. Lemma 5.5.34. Let ((z i-1 , a i , r i )) 1≤i≤n be an open K-closed word with resets of behaviours

Algorithm 13: Replication of open invalidities.

BinComb Input:

an open invalid K-closed word with its characteristic set W 1 and same for w 2 zr = ((z 2 i , a i , r 2 i )), W 2 with both invalid words being of same size and having the same actions and resets except fro the last reset. Output: w 1 zr , W 1 , w 2 zr , W 2 replicating the inputs such that w 1 zr and w 2 zr differ only on their last reset.

Then, noting {0} : c∈C 0 < c < 1:

Proof. We make the proof by induction. For i = 1, there is n ∈ N such that for any clock c ∈ C, the constraint n < c < n + 1 is in z 0 . By definition of ∆, n = ∆(z 0 , z 0 ) and we have our result. For i > 1, by induction The fact that BinComb respects (i) simply comes from correct manipulation of delays, while respecting (iii) requires alternating between rounding δ up and down (when an approximation is required) to avoid stacking approximation errors.

Because of this, we happen to round δ down. This is problematic in the case where it is rounded down to 0 in a delay transition between two open zones (i.e. when the previous discrete transition did not reset any clock). In this case one may have to manipulate the fractional values of the time elapses in the characteristic sets observations, which would contradict constraints (ii). To avoid this we cut the computation and return the arguments in place of the new K-closed word with resets.

Example 5.5.35. Consider two (prefixes of) K-closed words with resets depicted in Figure 5.18 (we represent them with |C| = 2 but one clock suffice):

There is no problem at depth one as both words have an abstract delay of 0 (from {s init } to its direct successor). At both depth two and three however one word has an abstract delay of 1 and one of 0. Hence the mean of those delays is 0.5 and must be approximated (as abstract delays between zones are always integers). One can see that an upper approximation at both steps would create an un-fitting resulting word, as it would reach zone c∈C 2 < c < 3 at depth three, falsifying constraint (iii). Hence using a lower approximation is necessary.

Algorithm 14: Finding a guard to separate two invalidities. BinSearch Input: two invalid K-closed words with resets w 1 zr , w 2 zr of same size and W 1 W 2 their two characteristic sets. An optional parameter f irst defaulting to false. Output: A pair (w 1 , w 2 ) of limit linear combinations of w

In this case a guard has to be introduced between each initial invalidity and its enlarged version.

Furthermore, a binary search is made between w 1 zr and w 2 zr . Suppose that it leads to

Then w 1 zr and w 1 f irst must be separated by a guard, which cannot be the same than the one separating w 1 zr and w 1 zr (in the example depicted on Figure 5

.19). Thus a new guard is necessary.

We know by Propositions 5.5.27, 5.5.31 and 5.5.36 that InvalidityGuard respects constraints (i) and (ii) except for collapsing K-equivalent classes during the enlargement phase as explained in Example 5.5.32. It remains to show that this algorithm terminates and respects constraint (iii). We first prove termination.

Proposition 5.5.43. Consider s d a decision state of a TLG N succeeding to a valid observation state, such that both its successors have been pruned due to invalidities characterised by the set W 1 (resp. W 2 ) corresponding to the K-closed word w 1 zr (resp. w 2 zr ). Then a call to

are the largest characteristic sets built during the algorithm, δ max is the maximal difference between delays ∆(z 1 i-1 , z 1 i-1 ) in behav(w 1 zr ) and behav(w 2 zr ) and n max is the maximal length of observations in W 1 ∪ W 2 .

Such call makes up to

membership queries before termination.

Proof. The initial enlargement phase has a time complexity O(|W 1 ∪W 2 |×n max ) dominated by the cost of the copy of the characteristic sets in the calls to Enlarge and makes |W 1 ∪ W 2 | calls to Request, resulting in (up to) the same amount of membership queries. After that step, each loop iteration of the main while loop in BinSearch has the same O(|W 1 ∪ W 2 | × n max ) complexity (due to the copies in BinComb) and makes up to |W 1 ∪ W 2 | membership queries, with the current values of W 1 and W 2 in the algorithm. For any iteration of that loop, either the algorithm terminates or exactly one of the two replicas replace its original invalid K-closed word (and is not equal to it). Hence as the difference of distances is divided by 2 (by the choice of δ) we have our result.

Remark 5.5.44. Notice that the size of W 1 , W 2 is bounded during the execution of the algorithm because new characteristic sets are built of replica of ancient ones. In the worst case, a W can have 2 nmax-|so| pairs with n max being constant through the algorithm and s o the predecessor of s d .

To show that InvalidityGuard respects the constraints (iii) we first show that BinSearch respects them (using the related properties on BinComb). This will then be used to conclude. Lemma 5.5.45. At any iteration of the While loop of BinSearch, any (enlargement closed) validity guard (g, i, w j zr , w 3-j zr ) with j ∈ {1, 2} is also a separating guard for (w j zr , w 3-j zr ) such that z j i ∈ g if and only if z j i ∈ g with z j i (resp. z j i ) the i-th zone of w j zr (resp. w j zr ).

Proof. This property holds by induction by using Lemma 5.5.38 as w j zr is taken from the results of a call to BinComb passing w 1/2 zr as arguments. The next property shows that the guards introduced by InvalidityGuard indeed separate the different invalidities, as a corollary of Lemma 5.5.45 and Lemma 5.5.30. We note W (resp. W ⊥ ) the set of timed words with resets which prefixes are invalid after the sequence of resets resets(s o ). (resp. resets(s o ).⊥) encountered during the call to

Upon the termination of a call InvalidityGuard(s d , W 1 , W 2 ), a set of validity guards is added to G V al that separate W and W ⊥ . Proof. During this proof, we use the fact that MakeGuard introduces guards that indeed separate its arguments, and are enlargement closed if the pair to be separated is made of open K-closed words with resets.

We divide the proof in cases depending on wether w 1 zr and w 2 zr are open and on the different cases of the algorithm.

• If both w 1 zr and w 2 zr are open, InvalidityGuard defaults to a call to BinSearch. By Lemma 5.5.45 and using the fact that we replace invalidities by invalidities of the same type, we know that this call separates correctly the invalidities encountered.

it indeed solves the problem by introducing fitting separating guards and handling the intricate structures of K-closed zones. It yet has some short-comings, namely the risk of collapsing K-equivalence classes for different resets during the enlargement phase and the possibility to stop the binary search when a specific manipulation on small distances between K-closed zones happens. As argued in the previous discussion and shown in examples, these are consequences of the structure of equivalence classes of valuations and would require a completely different approach to be handled (or rather avoided). Now that the search of all different necessary guards has been presented (through AdjPair and InvalidityGuard) we discuss the introduction of those guards in the TLG and the rebuilding of subtrees that ensues.

Rebuilding the graph

To rebuild a subtree of a TLG is to introduce new guards using consistency and validity guards only when necessary, and re-propagate the informations in the new guarded words with resets they satisfy. We use Rebuild (Algorithm 15) for this. Intuitively, Rebuild only introduces guards "when needed", which is formalized by the following well-guardedness property. Rebuild is described in Algorithm 15. It is called on a valid and consistent observation state of the TLG and erases the subtree rooted in it, before reconstructing a valid and consistent subtree with respect to the current observation function. It uses calls to FindGuard (Algorithm 16) that handles the guard creation.

Remark 5.5.48. In the rebuild function, we use Request on guarded words with resets instead of timed word. The extension is quite simple thanks to the resets, as searching in M if an observation modelling the argument exists is only a dive in the tree, and if none is found, making an membership query from the last guess is the same. Remark 5.5.49. As written, Rebuild completely erases the subtree and then reconstructs it. An obvious optimization is to only suppress transitions and nodes when necessary to avoid invalidity or inconsistency. We do not develop this here to keep the algorithm short and simple.

Algorithm 15: Rebuilds a subtree of N to handle consistency.

Rebuild

Input: A (valid) language state s l 2 Suppress recursively all successors of s l 3 for a ∈ Σ such that there is at least an observation passing s l . ( Rebuild constructs a complete, consistent and well-guarded subtree if it is called high enough in the tree. This is proven in Proposition 5.5.52 that assumes all sources of Well-groundedness The construction of any new language state stops as soon as no observation continues from the current language state. As Dom(Acc) is prefix-closed in the sense of K-closed words (this is ensured by the TOG) we know that all previously constructed language states have an observation in their words (as the last one has one).

Consistency Consistency is ensured by Lemma 5.5.51 and the hypothesis that all pairs of words that can create an inconsistency have led to a call to AdjPair that separated them.

Well-guardedness This property is ensured by construction, as all guards introduced by FindGuard are consistency or validity guards.

This proposition tells us that we can keep the timed language graph up-to-date with respect to observations (i.e., complete and consistent) while preserving the good properties that were ensured by the previous algorithms. This concludes the presentation of the algorithms updating the data-structure (TLG and TOG). It remains to show how a candidate timed automaton can be constructed from this structure.

Building a candidate timed automaton

Following the active learning approach, our purpose is to identify a subset of nodes in the language graph that will correspond to locations of the automaton, and then fold transitions according to an order on the remaining nodes. [START_REF] Grinchtein | Inference of Event-Recording Automata Using Timed Decision Trees[END_REF] discusses such orders when resets are fixed. To handle RERA we first have to fix a reset strategy before applying the original method. This gives as many hypotheses as we have strategies.

Reset selection. We present the general framework but do not discuss good strategies in the following. Such strategies would rely on heuristics. Definition 5.6.1. A reset strategy over a timed language graph N is a mapping π : S d → { , ⊥}, assigning a decision to each decision states.

A reset strategy π is said admissible if for any state s d , there is a language state s l such that (s d , π(s d ), s l ) ∈ E.

An admissible reset strategy is used to prune the language graph in such a way that only one reset combination is considered for each transition. The effect of an admissible reset strategy π on its timed language graph N is the TLG π(N ) defined from N by keeping only outgoing transitions from decision states that agree with π. We call this TLG the resulting graph of π.

It can be seen quite directly that a resulting graph always has exactly one successor to each decision state. Using this, we can notice that those resulting graphs are very close to timed decision trees of [START_REF] Grinchtein | Inference of Event-Recording Automata Using Timed Decision Trees[END_REF], in which no decision states exist and the transitions from language states to language states directly hold the (only possible) reset.

Having an admissible strategy thus is the basis of the construction of a hypothesis. The way our data structure is handled suffices to ensure that once all observations are integrated and the TLG is rebuilt, an admissible strategy always exists.

Proposition 5.6.2. In a timed language graph constructed using the FindPath and Rebuild algorithms and where every scheduled call to Rebuild has been done, there always exists at least one admissible reset strategy. Proof. To ensure that an admissible reset strategy exists, one only needs to check that every decision state has at least one successor. We only prune the graph in the SearchPrune algorithm, and this algorithm schedules a call to Rebuild when no successors exist for a decision state. As Rebuild constructs a subtree where all decision states have at least a successor (thanks to the FindGuard function that explicitly checks for this), we have our property.

Orders and folding. Once an admissible reset strategy is fixed, it is possible to fold the resulting graph into a RERA. This is made through the use of a preorder on states: we want to find a maximal subset for this order.

We define the height of a language state s l , noted height(s l ), as the height of the subtree it is the root of. A preorder on language states is said height-monotone when s l s l implies height(s l ) ≤ height(s l ).

Definition 5.6.3. Let N be a timed language graph and a preorder on its language states. A prefix-closed subset U of N is called -closed if s l U for all successors of U and -unique if for all s l , s l ∈ U , s l = s l ⇒ ¬(s l s l ).

CONCLUSION

So comes snow after fire, and even dragons have their ending.

-J.R.R. Tolkien "The Hobbit"

This thesis has tackled different problems related to the interactions between formal models of timed systems (as timed automata and some variants) and sheer reality. Chronologically, the research effort started from a discussion of the action of model-based agents on reality, through formal game theory. This lead to the realization that some informations lack in any reasonable model of a system. It thus makes sense to try to recover those information from the real system we are already interacting with, which can take the form of various (model) learning techniques. The hazard (and the discovery of an exciting body of research compiled during Olga Grinchtein's thesis [START_REF] Grinchtein | Learning of Timed Systems[END_REF]) led the author to primarily consider active learning.

Contributions

The contributions of this thesis focus on a very formal approach of reality-models interactions, with their upsides, such as a fine control and understanding of the behaviours, and their downsides, mainly the complexity that can arise and the formal (or perhaps mathematical) limitations that sprout from the model choices. The main contributions are three-fold:

• A model-based conformance testing method for timed automata with inputs and outputs that relies on game theory, and specifically so-called rank lowering strategies for hard games, allowing to plan even in situations where victory cannot be achieved without the opponent's cooperation. These strategies allow a quantitative take on such games, minimizing the reliance on cooperation and planing around it. We discuss their interest both in a restricted "fair" setting where they can be proved to be winning and in very general situations where the game difficulty forbids to hope for an assured victory. In this case, these strategies can nevertheless offer an interesting way toward the objective while trying to resist to failure.

• A discussion on a timed markings, a structure allowing to pre-compute a finite