Abstract

In the past decades, the analysis of complex critical systems subject to uncertainty has become more and more important. In particular the quantitative analysis of these systems is necessary to guarantee that their probability of failure is very small. As their state space is extremly large and the probability of interest is very small, typically less than one in a billion, classical methods do not apply for such systems.

Model Checking algorithms are used for the analysis of probabilistic systems, they take as input the system and its expected behaviour, and compute the probability with which the system behaves as expected. These algorithms have been broadly studied. They can be divided into two main families: Numerical Model Checking and Statistical Model Checking. The former computes small probabilities accurately by solving linear equation systems, but does not scale to very large systems due to the space size explosion problem. The latter is based on Monte Carlo Simulation and scales well to big systems, but cannot deal with small probabilities.

The main contribution of this thesis is the design and implementation of a method combining the two approaches and returning a confidence interval of the probability of interest. This method applies to systems with both continuous and discrete time settings for time-bounded and time-unbounded properties.

All the variants of this method rely on an abstraction of the model, this abstraction is analysed by a numerical model checker and the result is used to steer Monte Carlo simulations on the initial model. This abstraction should be small enough to be analysed by numerical methods and precise enough to improve the simulation. This abstraction can be build by the modeller, or alternatively a class of systems can be identified in which an abstraction can be automatically computed.

This approach has been implemented in the tool Cosmos, and this method was successfully applied on classical benchmarks and a case study.

Résumé

Ces dernières années, l'analyse de systèmes complexes critiques est devenue de plus en plus importante. En particulier, l'analyse quantitative de tels systèmes est nécessaire afin de pouvoir garantir que leur probabilité d'échec est très faible. La difficulté de l'analyse de ces systèmes réside dans le fait que leur espace d'état est très grand et que la probabilité recherchée est extrêmement petite, de l'ordre d'une chance sur un milliard, ce qui rend les méthodes usuelles inopérantes.

Les algorithmes de Model Checking quantitatif sont les algorithmes classiques pour l'analyse de systèmes probabilistes. Ils prennent en entrée le système et son comportement attendu et calculent la probabilité avec laquelle les trajectoires du système correspondent à ce comportement. Ces algorithmes de Model Checking ont été largement étudiés depuis leurs créations. Deux familles d'algorithmes existent : le Model Checking numérique réduit le problème à la résolution d'un système d'équations. Il permet de calculer précisément des petites probabilités mais souffre du problème d'explosion combinatoire; le Model Checking statistique est basé sur la méthode de Monte-Carlo qui se prête bien à l'analyse de très gros systèmes mais qui ne permet pas de calculer de petites probabilités.

La contribution principale de cette thèse est le développement d'une méthode combinant les avantages des deux approches et qui renvoie un résultat sous forme d'intervalles de confiance. Cette méthode s'applique à la fois aux systèmes discrets et continus pour des propriétés bornées ou non bornées temporellement.

Cette méthode est basée sur une abstraction du modèle qui est analysée à l'aide de méthodes numériques, puis le résultat de cette analyse est utilisé pour guider une simulation du modèle initial. Ce modèle abstrait doit à la fois être suffisamment petit pour être analysé par des méthodes numériques et suffisamment précis pour guider efficacement la simulation. Dans le cas général, cette abstraction doit être construite par le modélisateur. Cependant, une classe de systèmes probabilistes a été identifiée dans laquelle le modèle abstrait peut être calculé automatiquement.

Cette approche a été implémentée dans l'outil Cosmos et des expériences sur des modèles de référence ainsi que sur une étude de cas ont été effectuées, qui montrent l'efficacité de la méthode.

Remerciements

Je remercie mes encadrants de thèse Claudine Picaronny et Serge Haddad pour leur patience et leur gentillesse, ainsi que pour les conseils et les encouragements qu'ils m'ont prodigués tout au long de ces trois années. Grâce à leurs qualités scientifiques et humaines, ils m'ont permis de développer mon esprit critique et mon autonomie et de découvrir les différentes facettes du métier d'enseignant-chercheur. Je remercie en particulier Serge pour m'avoir fait rencontrer un grand nombre de chercheurs de différents horizons géographiques et scientifiques et pour m'avoir encouragé à travailler avec eux.

Je tiens également à remercier les deux rapporteurs de cette thèse Pieter-Tjerk De Boer et Gerardo Rubino qui ont accepté d'évaluer ce manuscrit et m'ont fait part de remarques très pertinentes pour la suite de mes travaux, ainsi que Patrice Moreaux, David Parker et Nihal Pekergin pour avoir accepté de participer à mon jury de thèse.

Je remercie tous les membres du LSV qui m'ont chaleureusement accueilli et avec qui j'ai pu discuter et échanger sur mes travaux pendant ces trois ans. Je remercie Paolo Ballarini, Hilal Djafri et Nihal Perkergin, pour leurs collaborations sur l'outil Cosmos qui m'ont permis de participer à l'implémentation d'un outil de vérification. Je remercie également Elvio Amparore, Marco Beccuti, Susanna Donatelli et Giuliana Franceschinis ainsi que les membres du département informatique de l'université de Turin pour m'avoir accueilli dans leur laboratoire lors de mon séjour en Italie. Je tiens à mentionner que participer au projet CosyVerif qui m'a sensibilisé au problème de la visibilité de la recherche a été une exprérience très enrichissante. Merci à tous les participants de ce projet de m'avoir intégré dans leur équipe et en particulier à Maximilien Colange, Clément Desmoulin et Alban Linard avec qui j'ai le plus étroitement collaboré.

Je tiens à remercier tous les membres du LSV qui ont rendu l'ambiance si conviviale et en particulier ceux avec qui j'ai collaboré sur des enseignements Cesar Rodriguez, David Baelde, Claudine Picaronny et Micheèle Sebag. Un grand merci à l'équipe administrative et technique qui m'a permis de travailler dans des conditions optimales. Je remercie ceux qui ont partagé mon bureau : Aiswarya, Benjamin, Patrick, Jérémie, Asalé et Baptiste pour l'ambiance studieuse et amicale qu'ils y ont instauré. Merci aussi à tous les nageurs du LSV qui m'ont aidé à garder la forme, j'ai en particulier une pensée pour Alban, Virginie, Paul, Thida, Sophie, Claudine et Francis. Je remercie tous les doctorants du LSV pour leurs camaraderie. Je tiens à remercier chaleureusement Aiswarya et Benjamin dont les 'chamailleries' ont animé ma vie au LSV, ainsi que Cesar que je n'ai jamais pu convaincre des bienfaits de la cuisine maison.

J'aimerai enfin remercier toute ma famille, mes parents pour m'avoir motivé, soutenu et donné le goût d'apprendre, Paul pour m'avoir le premier initié à l'informatique et Antoine pour être un si bon conteur. Pour finir je remercie Elisa pour son aide, son amour et pour m'avoir aidé à garder confiance en moi.

Benoît Barbot Oxford, Novembre 2014

Chapter 1

Introduction

When one is interested in the analysis of a given system, the usual approach consists in building a mathematical model of this system and in performing the analysis on the model rather than on the initial system. The benefits of such an approach are multiple and can be seen in many fields:

In engineering, modelling can very useful during the whole lifetime of a product. For instance, during the design of a system, the model can be modified to explore several design options or formal analysis and simulation can be conducted to ensure the correct behavior of the system. As these tests do not require to build any actual prototype, they are indeed much cheaper and faster than the traditional ones. During the exploitation of the system, the model can also be used to help maintenance or understand fault so that the next generation will be more effective. The use of models has been successful in several fields of engineering, as for example in mechanical engineering, electrical or software engineering.

In research,in most scientific fields, models are nowadays used to help with the comprehension of a system. In particular, models are used to understand or predict a specific behavior of a system without dealing with its whole complexity. In fact, the complete analysis could be either too expensive or too difficult but also unnecessary. The difficulty is then to abstract the system in a "smart" way in order to build a minimal pertinent mathematical model of the system for the property of interest.

Although applied in different fields and on different systems, similar trends can be found among the methods used for system analysis. The most common ones are simulation where the model is executed as the real system so that observations can be conducted on the simulated system; verification where all the states of a modeled system are explored to ensure that no faulty state can be reached; synthesis where parts of a model constrained by other parts are automatically build; and evaluation where the efficiency of a model is estimated.

The study and development of efficient methods for system analysis constitutes therefore the object of an active research in computer science. Within this thesis, we will focus on one particular method, namely Model Checking.

Model Checking

In Model Checking [START_REF] Emerson | Characterizing correctness properties of parallel programs using fixpoints[END_REF], one is interested in checking if a software or hardware system meets its specification. In this case, the system is modeled as a transition system while its specification is modeled as a formula in a given logic. An exhaustive exploration of the state space can then be performed in order to determine whether or not a faulty state is reached. This approach has been broadly studied for the last decades, leading to the development of several effective tools. Historically, model checking has been first applied to non-probabilistic programs using several specification languages like LTL [START_REF] Pnueli | The temporal logic of programs[END_REF] or CTL [START_REF] Emerson | Temporal and modal logic[END_REF]. However, some systems are subject to unsettled behaviors which may be governed by probabilistic laws. In those systems, standard model checking is not very helpful as it can return only a boolean (this state is reached) but gives no information about the frequency of this event to actually occur. In order to get access to this information, one then has to use quantitative model checking instead.

Two main sources of randomness can be identified:

• A system can explicitly contain randomness. For example, a network protocol often includes a randomized algorithm to break symmetry [START_REF] Lynch | Distributed algorithms[END_REF].

• Randomness may also be used to model unknown parts of a system. For instance, one can model the physical part of a system or its interaction with its environment with probabilities. In the case of a Cyber-physical system, a deterministic algorithm is then entangled with a physical part interacting with the environment which is modeled as a stochastic process [START_REF] Clarke | Statistical model checking for cyber-physical systems[END_REF].

Quantitative model checking stands therefore at the junction between model checking and performance evaluation. It has been applied to a broad range of probabilistic systems. However, some particular cases still requires special attention as for instance rare events.

Rare events

Rare events are events that appear with a very small probability, like the simultaneous failure of two redundant components in a critical system or an overflow in a stable telecommunication system. Indeed human intuition is far from correct when it comes to the estimation of such small probabilities.

For example, the report on the Challenger space shuttle accident by Richard

Feynmann [START_REF] Feynman | Personal observations on the reliability of the shuttle[END_REF] starts with the following sentences "It appears that there are enormous differences of opinion as to the probability of a failure with loss of vehicle and of human life. The estimates range from roughly 1 in 100 to 1 in 100,000.", Richard Feynmann notes that even experts on a system cannot estimate accurately small probabilities and are prone to underestimation. In the context of mathematical finance, Nassim Nicholas Taleb developped the Black Swan theory in [START_REF] Taleb | The Black Swan: The Impact of the Highly Improbable Fragility[END_REF] claiming that rare events play a crucial role and that their probability of occurrences are often underestimated to the point that they are not even considered by experts of the domain. It is therefore crucial to have reliable methods for computing small probabilities of failure.

Classical Methods for Probability Estimation

To estimate probabilities, they are two main approaches: numerical analysis and Monte Carlo simulation. In numerical analysis, the problem of computing the probability for a model to have a specific behavior may for instance be transformed into a system of linear equations. This system is then solved with classical linear algebra techniques. The main drawback of these methods is that the time and memory consumptions greatly increase when models become too complex, either because of the size of the state space which is known as the state space explosion problem, or because it features complex probability distributions. In Figure 1.1 experiments on a classical communication system composed of several queues in tandem are reported. This example is a well-known benchmark for rare event methods and is formally defined and studied in this manuscript. The state space of the system greatly increases with the number of queues. The time for estimating a probability of overflow before returning to an idle state in systems of variable sizes is plotted. On this figure, the time for numerical computation becomes intractable for large models. This situation is not specific to this example and very large system are intractable due to memory or time limitations. An alternative to numerical computations is Monte Carlo simulation. In this setting, the system is simulated a large number of times and an estimation of the probability of interest is computed by counting the number of trajectories of the system that contains the behavior of interest. In the model-checking setting, this method is also known as statistical model checking. In Figure 1.1, a simulation-based method is used to estimate a probability close to 1 with a constant number of trajectories. This method takes almost constant time on the tandem queues system with respect to the size of the models. The simulation time mainly depends on the number of branchings in the system. It is constant in the tandem queue example and, most of the time, increases less than logarithmically with the system size. The simulation time also depends on the length of the trajectories which may not be directly linked to the size of the system. Moreover, simulation-based Unfortunately, simulation-based methods do not handle efficiently the estimation of rare-event probability. Figure 1.2 shows the time required for the estimation of a small probability such that the estimation error is a constant fraction of the probability of interest which can be formalized using confidence intervals. On this example, when the probability goes to zero, the simulation time becomes intractable whereas the time required by numerical method is almost constant.

The limitations of numerical methods and Monte-Carlo simulation have been heavily studied. In particular, several methods to deal with the rareevent problem have been developed, the two main ones being splitting and importance sampling. These two methods modify how the simulation is conducted by making copies of trajectories in the case of splitting or biasing probability distributions in the case of importance sampling. They have been successfully applied in various domains like nuclear physic or queuing network. However two main limitations remain to apply them to modelchecking problems: they require manual setting and they can only provide asymptotical guarantee on their result, in some particular cases they can even worsen the problem.

Contributions

This thesis presents contributions to simulation techniques in three directions:

• A theoretical method to use importance sampling which guarantees the result by providing a conservative confidence interval not relying on some asymptotical hypotheses has been developed [IV, V, VI]. This method relies on an abstraction of the system under study which is small enough to be analyzed numerically and which is used to build an efficient importance-sampling algorithm. It combines the advantages of both numerical and simulation-base methods for the estimation of probabilities of occurrence of rare events in large systems. This method can be used by manually defining an abstraction of the model. A theoretical framework in which the abstraction of the model is automatically computed is also presented.

• Software developments on the tool Cosmos have been conducted.

Cosmos is a statistical model checker which takes Petri nets as input for models, and performs quantitative model checking using the HASL logic. The theoritical methods for handling rare events that we have developed have been integrated in this tool.

The input language of the tool has been extended to a broader class of Petri nets, namely Stochastic Symmetric Nets [I].

Recurrent difficulties with verification tools appear when one wants to exchange data between tools or interact with them in a uniform way. The CosyVerif platform [II] addresses these issues and provides a common interface to various tools including Cosmos.

• Several experiments using the rare-event methods have been conducted as well as one case study on a biological system [III] to assess the efficiency of the proposed methods.

Organization

In a first part, Chapter 2 recalls the formal notation and statistical results used throughout this thesis as well as classical algorithms used for numerical analysis of models. This chapter also recalls some properties of coupling relations. Chapter 3 presents in more details the rare-event problem for Monte Carlo simulations as well as classical methods to circumvent it, namely splitting and importance sampling. In the second part, theoretical results are presented. Chapter 4 presents a reliable theoretical method to estimate small probabilities for big systems. This method describes an algorithm based on an abstraction of the model. The design of this abstraction is a manual step. In Chapter 5, a framework is described in which abstractions can automatically be computed. The third part presents more applied results. Chapter 6 presents software contributions and benchmarks. In this chapter, the efficiency of the methods presented in Chapter 4 is demonstrated, as well as the efficiency of the statistical model checker Cosmos. Chapter 7 presents a biological case study where the methods for rare events previously described are used. Finally Chapter 8 contains the conclusions and perspectives of this thesis.

Part I

Preliminaries

Chapter 2

Technical Background

Introduction

This chapter presents known results used all along this thesis. They are presented in three parts:

• In a first part, we present the different stochastic models that we will consider later. We recall their semantics and some useful related properties.

• In the second part, we collect classical statistical results for estimating the expected value of a random variable. In particular, we introduce the notion of confidence interval and recall different methods to compute some.

• The last part is devoted to model-checking : we present logics to specify properties of stochastic systems and algorithms to verify them.

Models

Discrete Event Dynamic System

A general setting to describe probabilistic system is to suppose that the state of the system evolves in discrete steps, which implies that, in any time interval, the number of reached state is countable. This does not allows to model hybrid systems where the state of the system evolves continuously. Discrete events system are modeled formally by defining successive states of a system as sequence of random variables. The time between successive events is also described by a sequence of random variables.

Definition 1 (Discrete Event Dynamic System (DEDS))

A discrete event dynamic system is a tuple: (S, S 0 ,E,(E n ) 1 0 , (T n ) 1 0 ) with:

• S is a discrete set of states,

• S 0 is the random variable of the initial state taking value in S,

• E is a set of events,

• δ : S ⇥ E ! S is the transition function.

• (E n ) 1 0 is a sequence of random variables over the set of events E.A realization of (E n ) 1 0 is sequence of events in the system. The sequence of states is inductively defined as S n+1 = δ(S n ,E n ) from the initial state S 0 .

• (T n ) 1
0 is a sequence of stopping times which are random variables taking values in R + representing time interval. The variable T 0 is the time before the first event. Variable T n denotes the time between event E n-1 and E n .

Running example. A tandem queues system is used in this section to illustrate the different model formalisms. This system is shown on Figure 2.1. It consists in two queues : event λ represents the arrival of a client in the first queue. Once served in the first queue, the client goes to the second queue, represented by event ⇢ 1 . Once served in the second queue, the client leaves the system with event ⇢ 2 . The first event can occur at any time while the two others require the corresponding queue to contain at least one client.

n 1 n 2 λ ρ 1 ρ 2 Figure 2.1: Tandem Queues System
Each of the three events occurs according to a probability distribution. The first event is distributed according to a Poisson process and the two services ⇢ 1 and ⇢ 2 are distributed with exponential distributions. This system is described as M/M/1 with two queues in Kendall's notation.

As the distribution are Markovian at any time the system is fully described by the number of clients in the two queues. The pair (n 1 ,n 2 ) is thus used to describe the state of the system where there are n 1 clients in the first queue and n 2 clients in the second queue. As the queues are not bounded, the state space of the system is N ⇥ N. A Zeno trajectory in DEDS is a realization of a DEDS for which P 1 i=0 T i converges. Systems in which the probability of Zeno paths is not null are more difficult to analyzed and often avoided. The model of Markov chains which is heavily used in this thesis is not subject to the problem of Zeno path.

Several analysis can be performed on such a system. Stationary analysis studies whether the system reaches an equilibrium and in this case the behavior of the system in this equilibrium.

Definition 2 (Stationary distribution)

Let denoted by ⇡(s) s2S the probability distribution of states of the DEDS after an infinite time. It is defined when it exists as

⇡(s)= lim n!1 P(S n = s)
Transient analysis studies the behavior of a system in some fixed bounded time.

Definition 3 (Transient distribution)

Let π ⌧ (s) s2S be the distribution of state at time τ . It exists as soon as the set of Zeno paths has null probability in the system. It is defined as:

π ⌧ (s)= 1 X n=0 P S n = s | T n  τ<T n+1 P T n  τ<T n+1
As the transient distribution depends of the initial state, π ⌧,s ,w i t hs 2 S is defined as the distribution at time τ of a DEDS where the initial state have been replaced by s.

Discrete Time Markov Chain

A discrete time Markov chain (DTMC) is a stochastic process fulfilling the discrete Markov property. The discrete Markov property ensures that the process is memoryless after each event, that is all future stochastic behaviors only depend of the current state.

Definitions Definition 4 (Discrete Markov Property for DEDS)

Let D =( S, S 0 ,E,(E n ) 1 0 , (T n ) 1 0 )b eaDEDS.T h eDEDS D has the discrete Markov property if for all n 2 N: 8s 2 S, P(S n+1 = s | S 0 ,T 0 ,S 1 ,T 1 ,...,S n ,T n )=P(S n+1 = s | S n ) and 8t 2 R + , P(T n+1 >t| S 0 ,T 0 ,S 1 ,T 1 ,...,T n ,S n+1 )=P(T n+1 >t| S n+1 )

In DTMC, the passing of time is represented only by the successive states of the system. The time between events is irrelevant and thus assumed to be always equal to one time unit. Consequently the state space of a DTMC is countable. Thus it can be represented as a possibly infinite directed graph where arcs are labeled by probabilities. This can be defined formally by the following:

Definition 5 (Discrete Time Markov Chain (DTMC))

A Discrete Time Markov Chain (DTMC) C is a tuple (S, s 0 , P)w i t h :

• S is a possibly infinite set of states,

• P = P(s, s 0 ) (s,s 0 )2S⇥S is the transition probability matrix, This matrix P is stochastic which means that all its elements are non negative and the sum of each line is one, i.e. 8s 2 S, P s 0 2S P (s, s 0 ) = 1.

Remark 2

The function P is called a matrix despite the fact that it has infinite rows and columns when S is infinite.

DTMC takes as semantics a DEDS with the same state space, where the random variable S 0 takes for unique value s 0 . The sequence of states is defined as follows: 8n>0, 8s 2 S, P(S n = s)= X s 0 2S P(S n-1 = s 0 )P(s 0 ,s).

The sequence of times (T n ) is equal to 1 for all n. By definition DTMCs fulfill the Markov property.

The embedded graph of a Markov chain is defined as a directed graph whose vertices are the states of the Markov chain and the transitions of this graph are s ! t where P(s, t) > 0. Furthermore transitions of the graph may be labeled by their probabilities. The figure 2.4 represents a DTMC of a tandem queues system. The number of clients in the first queue is represented on the horizontal axis and the number of clients in the second one is represented on the vertical axis. In the initial state s 0 , the two queues are empty. Here this model is presented as a discrete one and thus the probability distribution of events is replaced by choices between events. Given a state s =(n 1 ,n 2 ),a new client comes in the first queue with probability λ, a client leaves the first queue for the second one with probability ⇢ 1 and a client leaves the second queue and exits with probability

λ ρ 1 ρ 2 λ ρ 1 ρ 2 λ ρ 1 ρ 2 λ ρ 2 λ ρ 1 ρ 2 λ ρ 1 ρ 2 λ ρ 1 ρ 2 λ ρ 2 λ ρ 1 ρ 2 λ ρ 1 ρ 2 λ ρ 1 ρ 2 λ ρ 2 λ ρ 1 ρ 2 λ ρ 1 ρ 2 λ ρ 1 ρ 2 λ ρ 2 ρ 2 ρ 2 ρ 2 ρ 2 ρ 1 ρ 1 ρ 1 ρ 1 s 0 n 1 0 1 2 3 4
⇢ 2 (λ + ⇢ 1 + ⇢ 2 =1).
The transition probability matrix is as follows: 8s =(n 1 ,n 2 ) 2 S,

P (n 1 ,n 2 ), (n 1 +1,n 2 ) = λ if n 1 > 0 P (n 1 ,n 2 ), (n 1 -1,n 2 + 1) = ⇢ 1 if n 2 > 0 P (n 1 ,n 2 ), (n 1 ,n 2 -1) = ⇢ 2 if n 2 > 0 P (0,n 2 ), (0,n 2 ) = ⇢ 1 if n 1 > 0 P (n 1 , 0), (n 1 , 0) = ⇢ 2 P (0, 0), (0, 0) = ⇢ 1 + ⇢ 2 otherwise P (n 1 ,n 2 ), (n 0 1 ,n 0 2 ) =0
For our purpose, we enrich the definition of Markov chains with events. This definition should not be confused with the more classical definition of labeled Markov chain. In a labeled Markov chain, a labeling function assigns labels to states of the Markov chain. Here events are added on the transitions of the chain. It is possible to define enriched labeled Markov chain as the two extensions do not interfere one with the other.

Definition 6

An enriched discrete time Markov chain C is defined by:

• a set of states S including an initial state s 0 ; • a finite set of events E;

1 ρ 2 ρ 2 ρ 2 ρ 2 ρ 1 ρ 1 ρ 1 ρ 1 1 s - s + s 0 λ λ λ λ λ λ λ λ λ λ λ λ λ λ ρ 1 ρ 1 ρ 1 ρ 1 ρ 1 ρ 1 ρ 1 ρ 1 ρ 1 ρ 1 ρ 2 ρ 2 ρ 2 ρ 2 ρ 2 ρ 2 ρ 2 ρ 2 ρ 2 ρ 2
• a successor function δ : S ⇥ E ! S;

• a function p : S ⇥ E ! [0, 1] such that for all s 2 S, P e2E p(s, e) = 1.

We define the transition probability matrix P of size S ⇥ S by: 8s, s 0 2 S, P(s, s 0 )= X δ(s,e)=s 0 p(s, e)

Running example. The tandem queues example has three events: a is the arrival of a client, e 1 is the end of a service in queue 1 and e 2 is the end of a service in queue 2. For each state s, p(s, a)=λ, p(s, e 1 )=⇢ 1 and p(s, e 2 )=⇢ 2 .

In the context of simulation we will use bounded DTMC (BDTMC) as chain equipped with two special states. The terminology will be explained later.

Definition 7 (BDTMC)

A bounded DTMC is a tuple (S, s 0 , P,s -,s + )w h e r e :

• (S, s 0 , P)i saDTMC.

• s -2 S and s + 2 S and these two states are absorbing.

i.e. P(s -,s -)=P(s + ,s + ) = 1.

• The only bottom strongly connected components are {s -} and {s + }.

This definition is equivalent to say that states s -and s + are reached with probability one and are absorbing. This definition is independent from the one of enriched DTMC and thus it makes sense to consider enriched bounded DTMC. The figure 2.5 shows a BDTMC.ABDTMCis ergodic as the only recurrent states are s + and s -and there exists p 2 [0, 1], the probability to reach s + such that π(s + )=p, π(s -)=1-p.

Methods

Qualitative analysis Qualitative analysis of a DTMC is done by analyzing its embedded graph. From the embedded graph of any DTMC,t h etree decomposition in strongly connected components can be computed. A Strongly Connected Component (SCC) is a subset of the graph for which their exists a path between all pairs of states and maximal for this property. The tree decomposition in strongly connected components is computed by taking as root the SCC containing the initial state. Then inductively a SCC A is the parent of a SCC B if there exists a path from a state of A to a state of B. SCC which are not connected to the root does not appear in the tree decomposition. SCC which are leaves of the tree are called Bottom Strongly Connected Components (BSCC).

BSCC play a crucial role in qualitative analysis. For any state s in a BSCC, the probability that s is visited infinitely often by a trajectory of the Markov chain is strictly greater than zero whereas for any state which is not in a BSCC this probability is equal to zero.

A state s of a SCC is called aperiodic if the gcd of lengths of all cycles on s is 1. Finite Markov chains are ergodic if all BSCC are aperiodic. A SCC is aperodic if it contains an aperiodic state.

Computing reachability probability Let π s denote the stationary distribution of a DTMC where the initial state has been replaced by s.T h e stationary distribution of a DTMC C exists and is unique as soon as C is ergodic. In the particular case of a BDTMC, π s (s + ) is the probability to reach s + from a state s and is denoted by µ(s). This stationary distribution can additionally be defined by the following equation:

Definition 8 (Reachability)

Given a DTMC C and a set of states S + ⇢ S, we define the vector of probabilities µ 2 [0, 1] S as the smallest solution of the following system of equations:

µ(s)= 8 > < > : 1i f s 2 S + X s 0 2S P(s, s 0 )µ(s 0 )i f s/ 2 S +
This definition allows to computes effectively the value of the stationary distribution µ by solving a linear system of equations.

Computing time-bounded reachability probability For any delay u 2 N,w ed e fi n eµ u (s) as the probability to reach the state s + in u time units starting from state s, that is defined as

µ u (s)=P(S u = s + )
with S 0 = s. The vector µ whose components are µ u (s), for s 2 S is called the time-bounded reachability vector with time-bound u. Similarly to µ a more effective definition of µ u can be stated: Definition 9 (Time-Bounded Reachability for DTMC) Given a DTMC C, a set of states S + ⇢ S and a positive integer u we define the vector of probabilities µ u 2 [0, 1] S as:

µ u (s)= 8 > > > < > > > : 1i f s 2 S + 0i f u =0^s/ 2 S + X s 0 2S P(s, s 0 )µ u-1 (s 0 )i f u>0 ^s/ 2 S +
This definition can be used to compute effectively time-bounded reachability using matrix-vector multiplications.

Coupling Methods The coupling method [START_REF] Lindvall | Lectures on the coupling method[END_REF] is a classical method for comparing two stochastic processes. It can be applied in various contexts (establishing ergodicity of a chain, stochastic ordering, bounds, etc.). In this thesis coupling methods are used to establish an order on the set of states of a DTMC toward the reachability of a state s + . A coupling between two Markov chains is also a Markov chain whose state space is a subset of the product of the two spaces. This subset is called the coupling relation. A coupling must satisfy that that its projection on any of its components behaves like the original corresponding chain. This means that the product chain contains the behaviors of its two components. Depending of the context for which the coupling is used, additional constraints are imposed. For our needs, we only define the coupling of a chain with itself and we add a constraint on the set of states to characterize an order on the DTMC states. Higher states in this order have higher probability to reach a state s + .

Definition 10

Let C =(S, P) be a Markov chain and s + 2 S be an absorbing state of C. A coupling of C with itself is a DTMC C ⌦ =(S ⌦ , P ⌦ ) such that :

• S ⌦ ✓ S ⇥ S • 8s 6 = t 2 S, 8(s, s 0 ) 2 S ⌦ , P (s,t)= P t 0 2S P ⌦ ((s, s 0 ), (t, t 0 )) and 8s 0 6 = t 0 2 S, 8(s, s 0 ) 2 S ⌦ , P(s 0 ,t 0 )= P s 0 2S P ⌦ ((s, s 0 ), (t, t 0 )) • 8(s, t) 2 S ⌦ ,s= s + ) t = s +
The set S ⌦ defines a coupling relation with a reachability goal s + .

The following proposition allows to compare transient and stationary reachability probabilities without any numerical computation. Recall that µ(s) denotes the probability to reach the state s + in C starting from state s.

Theorem 1

Let C ⌦ be a coupling of C, with a reachability goal s + Then, for all (s, s 0 ) 2 S ⌦ ,w eh a v e : µ(s)  µ(s 0 ) 8u>0,µ u (s)  µ u (s 0 )

Proof:

Let σ be a finite random trajectory ending in a bottom strongly connected component in the coupled chain starting from (s, s 0 ). We define the random variables 1 s + , 1 0 s + by:

• 1 s + = 1 if the first component of the ending state of σ is s + . • 1 0 s + = 1 if the second component of the ending state of σ is s + .
Let (s f ,s 0 f ) 2 S ⌦ be the final state of σ. Using that 8(s, t) 2 S ⌦ ,s= s + ) t = s + we have

s f = s + ) s 0 f = s +
Then for all σ:

1 s + (σ)  1 0 S + (σ)
By taking expected values, we have:

E(1 s + )  E(1 0 s + )
Which can be rewritten as: µ(s)  µ(s 0 ). By taking conditional expectation over the length of the path we obtain:

8u>0, E(1 s + |σ|u)  E(1 0 s + |σ|u)
This can be rewritten as: 8u>0,µ u (s)  µ u (s 0 ).

⇤

The following proposition specifies a special kind of coupling adapted to our needs. We point a local property, at the level of transitions, which whenever satisfied makes a relation a coupling. Precisely: For any pair of states (s, s 0 ) in the relation S ⌦ , for any events e 2 E we require that the pair (δ(s 1 ,e),δ(s 2 ,e)) is also in the relation, in other words, we require that the following diagrams holds.

8(s, s 0 ) 2 S, 8e 2 E : s s 0 S ⊗ t e t 0 e S ⊗
When this holds the relation S ⌦ is a coupling relation.

Theorem 2

Let C =(S, E, δ, p) be an enriched BDTMC with a reachability goal state s + . Let S ⌦ ⇢ S ⇥ S be a relation on C such that:

1. 8e 2 E, s 1 ,s 2 2 S ⌦ ,p (s 1 ,e)=p(s 2 ,e) 2. 8(s 1 ,s 2 ) 2 S ⌦ ,e 2 E, (t 1 ,t 2 ) 2 S 2 (δ(s 1 ,e)=t 1 ^δ(s 2 ,e)=t 2 ) ) (t 1 ,t 2 ) 2 S ⌦
We define δ ⌦ ((s 1 ,s 2 ),e)=( δ(s 1 ,e),δ(s 2 ,e)) and p ⌦ ((s 1 ,s 2 ),e)=p(s 1 ,e).

Then the DTMC C ⌦ =(S ⌦ ,E,δ ⌦ ,p ⌦ ) and S ⌦ is a coupling relation on C 2 .

Proof:

The second hypothesis asserts that the chain is well defined. Furthermore, for s, t in S such that s 6 = t, we have, using Definition 6:

8s 0 2 S, (s, s 0 ) 2 S ⌦ ) X t 0 2S P ⌦ ((s, s 0 ), (t, t 0 )) = X e|δ(s,e)=s 0 p(s, e)=P (s,t). 1 1 λ ρ 1 ρ 2 λ ρ 1 ρ 2 λ ρ 1 ρ 2 λ ρ 1 ρ 2 λ ρ 2 ρ 1 λ ρ 1 ρ 2 λ ρ 1 ρ 2 λ ρ 1 ρ 2 ρ 2 ρ 2 ρ 2 ρ 1 ρ 1 ρ 1 λ λ λ s • - s • + s • 0 Figure 2.
{(n 1 ,n 2 ) | 0  n 1 + n 2  N ^n2  R}
by merging all states (n 1 ,n 2 ) such that n 1 + n 2 = N . s + corresponds to this merging and any pair (n 1 ,n 2 ) such that n 1 + n 2 = N denotes s + . The goal set contains only s + . s -=(0, 0). Let us define

S ⌦ = {((n 1 ,n 2 ), (n 0 1 ,n 0 2 )) | n 1 + n 2  n 0 1 + n 0 2 ^n1  n 0 1 }
and prove that S ⌦ is a coupling relation. The first and the third conditions of proposition 2 are satisfied by construction. Let us check that for all couples ((n 1 ,n 2 ), (n 0 1 ,n 0 2 )) in the relation, all successors are also in the relation. We examine the three events of the system:

1. For event a, the successor of

((n 1 ,n 2 ), (n 0 1 ,n 0 2 )) is ((n 1 +1,n 2 ), (n 0 1 + 1,n 0
2 )) which is inside the relation.

2.

For event e 1 , the successor of

((n 1 ,n 2 ), (n 0 1 ,n 0 2 )) is ((n 1 -1 {n 1 >0^n 2 <R} ,n 2 + 1 {n 1 >0^n 2 <R} ), (n 0 1 -1 {n 0 1 >0^n 0 2 <R} ,n 0 2 + 1 {n 0 1 >0^n 0 2 <R}
)) the first condition defining S ⌦ is satisfied as the sums are unchanged.

• If n 1  n 0 1 then n 1 -1 {n 1 >0^n 2 <R}  n 0 1 -1 {n 0 1 >0^n 0
2 <R} and the second condition defining S ⌦ is satisfied.

• Else n 1 = n 0 1 and using the first condition we have n 2  n 0 2 ; then

n 0 1 > 0 ^n0 2 <R) n 1 > 0 ^n2 <Rwhich implies: n 1 -1 {n 1 >0^n 2 <R}  n 0 1 -1 {n 0 1 >0^n 0 2 <R} ,

3.

For event e 2 , the successor of ((n 1 ,n 2 ), (n 0

1 ,n 0 2 )) is ((n 1 ,n 2 -1 {n 2 >0} ), (n 0 1 ,n 0 2 -1 {n 0 2 >0} )).
As the first component is unchanged, the second condition holds.

• If n 1 + n 2 <n 0 1 + n 0 2 then n 1 + n 2 -1 {n 2 >0}  n 0 1 + n 0 2 -1 {n 0 2 >0} • Else n 1 + n 2 = n 0 1 + n 0 2 and using the second condition n 2  n 0 2 then n 2 -1 {n 2 >0}  n 0 2 -1 {n 0 2 >0} ,s ot h efi r s tc o n d i t i o nh o l d s .
Then S ⌦ is a coupling relation.

Continuous Time Markov Chain

Definitions

When the time between events of a Markov chain is not discrete, continuous distribution of the time are used and the continuous Markov property should hold between each event.

Definition 11 (Continuous Markov Property for DEDS)

Let D =( S, S 0 ,E,(E n ) 1 0 , (T n ) 1 0 )b eaDEDS.
T h eDEDS D has the continuous Markov property if it has the discrete Markov property and for all n 2 N:

8t, t 0 2 R + , P(T n >t+ t 0 | T n >t 0 )=P(T n >t)
The discrete Markov property ensures that at each event the system is memoryless. The continuous Markov property ensures furthermore that between two events, the waiting time is also memoryless.

The only continuous distribution satisfying the continuous Markov property is the negative exponential distribution. Thus continuous time Markov chains are defined by adding a rate which is a positive real to each state. The waiting time in each state is distributed exponentially with this rate.

Definition 12 (CTMC)

A Continuous Times Markov Chain is a tuple: (S, s 0 , P)w i t h :

• S is a set of states, • s 0 2 S is the initial state, • Q =( Q(s, s 0 )) (s,s 0 )2S⇥S
is the infinitesimal generator matrix, all elements of this matrix are non negative except for the diagonal ones, which are negative. Moreover the sum of coefficients for each line is equal to zero, i.e. 8s 2 S,

P s 0 2S Q(s, s 0 )=0, Q(s, s)=- P s 0 6 =s2S Q(s, s 0 ).
Other (equivalent) definitions of Continuous time Markov chain exist. The infinitesimal generator matrix Q can be replaced by the transition rate matrix R. For the same CTMC matrices Q and R are equal except on the diagonal where R can take arbitrary non negative values corresponding to loops. The infinitesimal generator can be split in two, a probability transition matrix P and an exit rate function E : S ! R ≥0 . For two states s, s 0 2 S, the transition rate matrix R(s, s 0 ) is equal to E(s)P(s, s 0 ) These three representations describe the same stochastic process but with different points of view. The infinitesimal generator is used in the Kolmogorov equation describing the behaviors of the chain. The rate transition matrix R is more used for modeling as the loops are treated like other transitions. Finally splitting the stochastic behavior in waiting time and probability transition matrix reveals the discrete behaviors of the system.

A CTMC takes its semantics as a DEDS with the same state space S,a unique initial state s 0 , 8n>0, 8s 2 S, P(S n = s)= X

s 0 2S\{s} Q(s 0 ,s)P(S n-1 = s 0 ), 8n>0, P(T n  x)=1-e Q(Sn,Sn)x .
Similarly to DTMCs, enriched CTMCs can be defined:

Definition 13

An enriched continuous time Markov chain C is a tuple (S, s 0 ,E,δ,λ) defined by:

• a set of states S including an initial state s 0 ;

• a finite set of events E;

• a successor function δ : S ⇥ E ! S;

• a function λ : S ⇥ E ! R +
We define the infinitesimal generator matrix Q of size S ⇥ S by:

8s 6 = s 0 2 S, Q(s, s 0 )= X δ(s,e)=s 0 λ(s, e) 8s 2 S, Q(s, s)=- X δ(s,e)6 =s λ(s, e)

Methods

Computing reachability probability The embedded DTMC of a CTMC is defined as: D =(S, E, δ, p)w h e r ep is defined as follow:

8s 2 S, e 2 E, p(s, e)= λ(s, e) P s 0 2S λ(s 0 ,e)
.

The distribution of time (T n )i nt h eDEDS is replaced by a sequence always equal to 1. As reachability probabilities do not depend of the waiting time, reachability probabilities in a CTMC are equal to reachability probabilities in its embedded DTMC.

Computing time-bounded reachability probability Let µ ⌧ the vector of probabilities to reach s + in ⌧ time units be defined as µ ⌧ (s)=⇡ ⌧,s (s + ).

Using that the evolution of time is distributed with negative exponential distribution, the following definition for µ ⌧ holds: Definition 14 (Bounded Reachability for CTMC) Given a CTMC C, a set of states S + ⇢ S and a positive real ⌧ we define the vector of probabilities µ ⌧ 2 [0, 1] S as the smallest solution to the following system of equations:

µ ⌧ (s)= 8 > < > : 1i f s 2 S + Z ⌧ 0 X s 0 2S,s 0 6 =s Q(s, s 0 )e Q(s,s)u µ ⌧ -u (s 0 )du if s/ 2 S +
While in a CTMC the differential system can be numerically solved, a more efficient way to obtain these probabilities consists in uniformising the chain.

A chain is said to be uniform when for each state s, the exit rate λ = λ s is independent from s. When a chain is uniform the following holds. Given a uniform chain, the transient distribution π ⌧ is obtained by the following formula:

π ⌧ (s)= X n≥0 e -λ⌧ (λ⌧ ) n n! P n (s 0 ,s)
Indeed using the uniform hypothesis, e -λτ (λ⌧ ) n n!

is the probability that n transitions take place in interval [0,⌧] and P n (s 0 ,s) is the probability to be in state s after n transitions. The term e -λτ (λ⌧ ) n n! is known as Poisson probability.

Given a non uniform chain with bounded rates, it is possible to transform it in a uniform chain with the same distribution ⇡ ⌧ (see figure 2.7). It consists in selecting some upper bound of the outgoing rates (say λ), consider λ as the uniform transition rate and set a transition matrix P (u) defined by:

8s 6 = s 0 2 S P (u) (s, s 0 )= λ s λ P (u) (s, s 0 ) P (u) (s, s)=1- X s 0 6 =s P (u) (s, s 0 )
Running example.

Observe that uniformization does not require the CTMC to be finite. For instance, the rates of the infinite CTMC corresponding to the queuing tandem are bounded by With the same argument, when the chain is uniform with rate λ, µ ⌧ (s) fulfills:

⇢ 0 +⇢ 1 +⇢ 2 .A s s u m i n gt h a t⇢ 0 +⇢ 1 +⇢ 2 = 1,
µ ⌧ (s)= X n≥0 e -λ⌧ (λ⌧ ) n n! µ n (s)
This expression for µ ⌧ is useful when it come to numerically compute the value of µ ⌧ as [START_REF] Jansen | Understanding Fox and Glynn's "computing poisson probabilities[END_REF] present efficient algorithms for truncating the infinite sum and computing Poisson probabilities with exact framing of the numerical errors.

Coupling methods For continuous time Markov chain, we can strengthen the result on coupling:

Theorem 4
Let C =(S, E, δ, λ) be an enriched CTMC with absorbing state s + . Let S ⌦ ⇢ S ⇥ S be a relation on C such that:

1. 8e 2 E, s, s 0 2 S ⌦ , λ(s, e) >λ(s 0 ,e) ) (δ(s, e),s 0 ) 2 S ⌦ and λ(s, e) <λ(s 0 ,e)

) (s, δ(s 0 ,e)) 2 S ⌦ 2. 8(s, s 0 ) 2 S ⌦ ,e 2 E, (t, t 0 ) 2 S 2 (δ(s, e)=t ^δ(s 0 ,e)=t 0 ) ) (t, t 0 ) 2 S ⌦ 3. S ⌦ \ (S f ⇥ S) ⇢ (S f ⇥ S f )
We define the pro duct CTMC, C ⌦ =(S ⌦ ,E ⌦ ,δ ⌦ ,λ ⌦ ) as follow:

• E ⌦ = E ] Ẽ,w h e r e
Ẽ is a copy of E.

• 8(s, s 0 ) 2 S ⌦ , δ ⌦ ((s, s 0 ),e)=(δ(s, e),δ(s 0 ,e))

and δ ⌦ ((s, s 0 ), ẽ)=

> < > :

(δ(s, e),s 0 )i f ( δ(s, e),s 0 ) 2 S ⌦ (s, δ(s 0 ,e)) if (s, δ(s 0 ,e)) 2 S ⌦ (s, s 0 ) otherwise • 8(s, s 0 ) 2 S ⌦ , λ ⌦ ((s, s 0 ),e)=min(λ(s, e),λ(s 0 ,e)) and λ ⌦ ((s, s 0 ), ẽ)=|λ(s, e)λ(s 0 ,e)|

Then the CTMC C ⌦ is a coupling over C 2 and

8⌧ 2 R + ,µ ⌧ (s)  µ ⌧ (s 0 ) Proof:
The second hypothesis asserts that the chain is well defined. Furthermore, for all s, t in S such that s 6 = t, and for all s 0 2 S, such that (s, s 0 ) 2 S ⌦ we have, using definition 6:

X t 0 2S Q ⌦ ((s, s 0 ), (t, t 0 )) = X e2E ⌦ |δ ⌦ ((s,s 0 ),e)=(t,t 0 ) λ ⌦ ((s, s 0 ),e)
We can split the sum over transitions of E and transitions of Ẽ. = X e2E|δ(s,e)=t λ ⌦ ((s, s 0 ),e)+ X ẽ2 Ẽ|δ(s,e)=t^(δ(s,e),s 0 )2S ⌦ λ ⌦ ((s, s 0 ), ẽ)

Rates of transitions of E can be replaced by theirs definitions. Rates of transitions of Ẽ are null if not in (δ(s, e),s 0 ) 2 S ⌦ = X e2E|δ(s,e)=t min(λ(s, e),λ(s 0 ,e)) + X ẽ2 Ẽ|δ(s,e)=t max(λ(s, e)λ(s 0 ,e), 0)

The two sum can be merged together. = X e2E|δ(s,e)=t min(λ(s, e),λ(s 0 ,e)) + max(λ(s, e)λ(s 0 ,e), 0)

Finally we obtain. = X e2E|δ(s,e)=t λ(s, e)=Q (s,t)

The other claim for the coupling is symmetric. The proof that

8⌧ 2 R + ,µ ⌧ (s)  µ ⌧ (s 0 )
is obtain by rewriting the proof of Theorem 1 with conditional expectation on the time duration of paths. ⇤

Higher Level Model

Markov Chains are well suited for the theoretical analysis of probabilistic systems but not so well suited for modeling systems. To build a Markov chain modeling a system, one has to enumerate all states of the system and build the transition probability matrix. To ease the modeler's work, higher level models are used, which allow to express in a compact way complex systems.

From such a compact representation, a Markov chain can be extracted and give a stochastic semantics to the system. In this thesis, we use stochastic Petri nets and stochastic symmetric nets as higher level models.

Stochastic Petri Net

Stochastic Petri net is a well known formalism for modeling some stochastic process. It relies on the usual formalism of Petri net, well adapted to the modeling of the control flow of concurent system. Let us first recall this formalism that have been heavily studied (see for example [START_REF] Diaz | Petri Nets: Fundamental models, verification and applications[END_REF])

Definition 15 (Petri net)

A Petri net or Place/Transition net is a tuple N =(P, T, W -,W + ,m 0 ) with P \ T = ; where

• P is a finite set of places,
• T is a finite set of transitions,

• W -: P ⇥ T ! N is the pre incidence matrix,

• W + : P ⇥ T ! N is the post incidence matrix,

• m 0 2 N P is the initial marking.

We call a marking m 2 N P of a Petri net N a vector assigning an integer to each place of the net. A transition t 2 T is fireable in a marking m if 8p 2 P, m(p) -W -(p, t) ≥ 0. The firing of a firable transition t from a marking m leads to marking m 0 denoted m t -! m 0 ,wherem 0 is defined as 8p 2 Let us first recall the different way time can be introduce in the behaviors of Petri net (see for example [START_REF] Bérard | Comparison of different semantics for time Petri nets[END_REF] for a more detailed comparison of Petri nets equipped with time) Two alternative approaches exist:

P, m 0 (p)=m(p)-W -(p, t)+W + (p, t). Let σ = σ 1 •••σ n 2 T ⇤ be
1. Time Petri Net [START_REF] Berthomieu | Modeling and verification of time dependent systems using time Petri nets[END_REF] add clocks and guards on transitions of the net. [START_REF] Abdulla | Timed Petri nets and BQOS[END_REF] add clocks to tokens and guards on input arcs of the net.

Timed Petri Net

The first approach is more suitable to model a stochastic system, as it makes a correspondence between transitions of the net and transitions in the reachability graph. Thus stochastic Petri nets have been defined with probability distributions on the transitions. Several choices remains on how the time is taken into account in transitions, when dealing with time Petri net all the choices are non deterministic:

• In Duration semantics, a transition is chosen among all the fireable ones, the tokens in the input places are removed and a waiting time is chosen with respect to the guard of the transition. When this time has elapsed the token are released in the output places. This semantic is not commonly used because marking reached by the net before the end of a transition may not belongs to the set of reachable marking of the plain Petri net.

• In Delay semantics, as soon as a transition is fireable a time is chosen and the transition is said to be activated. The transition is fired only when the sampled time is reached. This semantics preserves concurrence as several concurrent transitions can be activated at the same time, for each of them a time is chosen and only the transition with the smallest time is fired.

The second semantics is the most used one. Even with this semantics choices remains which are called policy.

• Memory policy determines the behaviors of transitions when a transition is activated, deactivated by concurrent transitions and activated again. Using no memory policy a time is chosen each time the transition is activated. Using age memory policy when a transition is deactivated the remaining time before firing is stored, when the transition is activated again, the firing time is set according to the remaining time before deactivation.

• Server policy determines if a transition for which there are enough tokens to fire it several times can effectively be activated several time.

In single server policy such a behavior is forbidden. In infinite server policy the transition is activated as many times that there are enough tokens in the input places. In multiple server policy the maximal number of activations is fixed and specified for each transition.

In Stochastic Petri nets all choice of time are replaced by exponential probabilistic distribution and there is no guard on transitions. Moreover single server policy is used.

Definition 16 (Stochastic Petri Net(SPN))

A Stochastic Petri net is a tuple N =( P, T, W -,W + ,m 0 , Λ) where (P, T, W -,W + ,m 0 ) is a Petri net and Λ : N P ⇥ T ! R is the rate function which associates a rate to each marking and transition.

Due to the choice of policy, stochastic Petri nets take naturally as semantics an enriched CTMC: From N =( P, T, W -,W + ,m 0 , Λ) a CTMC C =(S, s 0 ,E,δ,λ) is defined as follow:

• S = Reach(N ,m 0 ) • s 0 = m 0 • E = T • δ(m, e)=m 0 s.t. m e - ! m 0 • λ(m, e)=Λ(s, e)
Running example. Figure 2.9 shows a SPN representing the tandem queues system. Two places (Q 1 ,Q 2 ) are used to encode the two queues and transitions encode events.

Q 1 Q 2 λ ρ 1 ρ 2 Figure 2
.9: A SPN representing the tandem queues system A common extension of SPN are generalized stochastic Petri nets which authorized to use immediate transition which follow Dirac distributions with parameter 0. Weight and/or priority are added to resolve concurrent firing of immediate transitions. As immediate transitions are memoryless the semantic of generalized Petri net is still a CTMC. Adding such immediate transitions is really convenient for modeling stochastic systems and may reduce the size of the reachable marking [START_REF] Kartson | Modelling with generalized stochastic Petri nets[END_REF]. In this setting Zeno paths are possible if there is a cycle of immediate transitions. Such cycles can be detected by analyzing the structure of the net.

Stochastic Symmetric Net

When a model contains several instantiations of a same pattern, it can be expressed in a compact way using Stochastic Symmetric Net (SSN). Intuitively instead of using indistinguishable tokens as in a Petri net, sets of colors are used to distinguish them. When the obtained system is unchanged when color classes are permuted, the lumped embedded Markov chain is much smaller and can be computed directly [START_REF] Chiola | Stochastic well-formed colored nets and symmetric modeling applications[END_REF]. Whether color classes can be permitted or not can be obtained by syntactic analysis of the system.

As the stochastic symmetric net formalism is an extension of the symmetric net formalism, we first recall this formalism. Symmetric net where also named well formed net. Compare to plain Petri nets which model the control flow of concurrent systems, symmetric nets can be used to model also data structure by distinguishing between tokens. In Petri net firing a transition is analogous to calling a function with no argument, in symmetric net a transition has a set of variables which are instantiated to some color. When the transition is fired it is analogous to calling a function that takes as arguments the bindings of colors to variables.

Definition 17 (Symmetric Net)

As y m m e t r i cn e ti sat u p l eN =( P, T, W -,W + ,m 0 , C, Domain, X) where

• P and T are set of places and transitions such that P \ T = ;.

• C = {C 1 , ••• ,C k } is
W -= 0 B B @ 0000 x 000 00x 0 0 x 0 x 1 C C A ,W + = 0 B B @
x 0 x 0 0 x 00 000x 0000

1 C C A .
Then stochastic symmetric net are defined by adding a distribution to each transition of the net which parameters may depends of the binding color of the transition.

Definition 18 (Stochastic Symmetric Net(SSN))

A stochastic symmetric net is a tuple N = (P, T, W -,W + ,m 0 , C, Domain, X, Λ) where • (P, T, W -,W + ,m 0 , C, Domain, x)i sas y m m e t r i cn e t Color variables are declared globally instead of locally to each transition to simplify notation, thus there is only one variable x taking value in C 1 .

In this example the pattern of queues is factorized in one place. It allows to parametrize the model by the color class C 1 . This example is not symmetric by every permutation of colors in the color class, thus for this

C 1 = {1,...,d} x : C 1 Q : C 1 < 1 > λ <d> ρ(d) <x> <x+1> ρ(x)
x 6 = d This model implements a crude protocol to ensure the integrity of files across different databases using global mutexes for each file. The workflow is as follows:

1. If a user on site s wants to use a file f it takes transition Start. Then he waits for the common mutex on this file to be available.

2. As soon as the mutex is available it is acquired with transition Acquire.

Then the user can start to edit the file.

3. When modifications to the file are finished, transition Change is fired and messages are sent to all sites distinct from s.

4. Each message arrives to its site with transition SendMessage.

5. Each site starts to update files according to messages as soon as possible with transtion Update.

6. When the update is finished each site sends a reply with transition EndUpdate 7. Each reply arrives to the site that initiates the change of the file with transition SendReply.

8.

As soon as all replies, one from every site, are received the mutex of the file is released and the site returns to an idle state with transition Release.

In this SSN any permutation in S and F does not change the behavior thus this system can be analyzed efficiently using symbolic representation taking into account these symmetries. Symmetries are syntactically checked: no arc valuation distinguishes color tokens, the model contains no guard and the initial state is invariant by all permutations of colors.

Statistical Methods

Monte Carlo Method

The Monte Carlo method computes an estimation of E(X)w h e nX is a random variable following an unknown distribution law L, defined as the result of a computation on a trajectory of a probabilistic system such that V(X)i sfi n i t e . I fX follows a known distribution up to a parameter, a Monte Carlo method can be used to estimate this parameter. A Monte Carlo method contains two distinct parts.

• The first part is a simulator that produces realizations of X. Ideally the simulator must produce independent realizations of this law. To do this a simulator requires a source of randomness that produces independent realizations of a known law. In general such a source of randomness is approximated by a pseudo random number generator producing real numbers chosen uniformly in [0; 1] like Mersenne-Twister [START_REF] Matsumoto | Mersenne twister: A 623dimensionally equidistributed uniform pseudo-random number generator[END_REF]. The produced samplings are not really randomly chosen and are not independent because they are produced by a deterministic algorithm but the number of required samplings to distinguish such a sampling from a random sampling is huge.

The simulator takes as input a description of the system that one wants to simulate. The system is usually described as a stochastic process.

For each simulated trajectory the algorithm returns a value, usually a real number, which is a realization of L.

• The second part is a deterministic algorithm which takes as input a sequence of realizations and produce an estimation of the result that one wants to compute. This part is the Monte Carlo estimator. As the goal is to estimate an expectancy, the best estimator is the mean of all the values. The estimator is itself a random variable. After N simulations of the system, the estimator posses N independent identically distributed copies of X noted (X i ) N i=1 . The estimator is

Z = 1 N P N 1 X i .
Basic probabilistic property shows that this estimator is an unbiased estimator of E(X), meaning that its expected value is equal to E(X). Moreover it's variance is equal to V(X) N .

Confidence interval

The goal of a Monte Carlo method is to compute an estimation of E(X) for some variable X, when the exact value of this expected value can not be computed, one wants to know how far this approximation is from the actual value. It is not possible to guarantee that the numerical value we obtain is at a given distance to the actual value. That is after a number of sample, with a non-zero probability our simulation may have ignored a parts of the system. This leads to incorrect value for the estimation.

For example if one wants to check that a dice is well balanced, one can throw it 60 times; if the dice is indeed well balanced we can expect each face to show around 10 times. But even if one face never shows up, we can only suspect that the dice is biased, without certainty.

Intuitively, as the variance of Z tends to 0 when N tends to infinity, for afi x e dN and a fixed real δ>0, the variance of Z is smaller than that of X; thus, a realization p of Z will be more likely to be in the interval [E(X)δ; E(X)+δ] than a realization of X.

The notion of confidence interval allows to have some probabilistic guarantee on a result obtained using random algorithms. This guarantee takes the form of confidence on the fact that the actual value is close enough to the realization. In a more general setting a confidence interval is defined as follows:

Definition 19 (Confidence Interval) Let (X i ) N
1 be independent random variables following a common distribution including a parameter θ. Let 0 <γ<1 be a confidence level. Then a confidence interval for θ with level at least γ is given by two random variables l(X 1 ,...,X N ) and u(X 1 ,...,X N ) such that for all θ:

P [l(X 1 ,...,X N )  θ  u(X 1 ,...,X N )] ≥ γ Theorem 5 (Markov Inequality)
Given a positive random variable Y the following equality holds:

8β>0,β• P [Y ≥ β]  E(Y )
In the context of Monte Carlo method the parameter that one wants to frame is E(X). An easy way to build a confidence interval without making any new assumption on the distribution law of X is to use the inequality of Chebyshev.

Theorem 6 (Chebyshev's inequality)

Given a random variable X such that E(X) and V(X) are finite. The following inequality holds:

8α>0, P h |X -E(X)|≥α p V(X) i  1 α 2
Using the Chebyshev inequality on the random variable Z using N samples, one can choose α such that 1 -1 ↵ 2 is equal to the confidence level 1ε. The inequality now states that for a realization p of Z we have:

P " E(X) 2 " p - r V(X) Nε , p + r V(X) Nε ## ≥ 1 -ε which is a confidence interval.
Several remarks can be made on this confidence interval:

• The speed of convergence of the Monte Carlo method with respect to

N is in O ⇣ 1 p N ⌘
. This is not a fast convergence but the only hypothesis that we make is that the variance is finite.

• The width of the confidence interval depends highly on the variance of the distribution law. Classical methods to improve the speed of convergence of Monte Carlo algorithm are based on the reduction of this variance.

• The variance of the distribution is unknown. To use this confidence interval, one has to estimate or bound this variance. This fact contributes greatly to the rare event problem of Monte Carlo methods.

To obtain tighter confidence intervals or to reduce the require number of simulations, one can make stronger assumptions on the shape of the distribution of X and use more advanced statistical results to build confidence intervals.

Functionalities Statistical procedures have different kinds of inputs and outputs:

• The output can be a boolean corresponding to the comparison between a probability and a threshold. Corresponding statistical techniques are called hypothesis testing.

• The input can be an expression involving the expectations of random variables and the output is an estimation given as a confidence interval framing the value.

Probabilistic Guarantee As the result is obtained by a statistical estimation, there are two kinds of probabilistic guarantees. Denoting by p the confidence level,

• The probability that the result is wrong is upper bounded by p. Here the result is either the comparison of a probability with a threshold or the framing of a parameter inside a confidence interval.

• The probability that the result is wrong is asymptotically upper bounded by p. The limit can be taken when the number of samples goes to infinity or when the width of the confidence interval goes to 0. In this case only asymptotic confidence intervals are obtained.

Static versus sequential sampling

• A static procedure takes as input or initially computes the number of simulations before performing these simulations.

• A sequential procedure both performs successive simulations and estimates the stopping criterion which depends on the simulation outputs.

Gaussian Confidence Interval

In the general case, the classical approach is based on the hypothesis that the normal law is a good approximation of the simulated law. A discussion on the validity of this approximation for the estimation of a rare event is made in Section 3.2. Thus, given a number of paths N and a confidence level 1ε,themethod produces a confidence interval.

Theorem 7

If X follows a normal distribution a confidence interval can be computed for a confidence level of 1ε. Let z be such that 2φ(z) -1=1-".T h e following equation holds:

P ⇣ E(X) 2 h p -z p V(X), p + z p V(X) i⌘ =2φ(z) -1
Where φ is the error function sometime written erf defined by

φ(x)= 1 p 2⇡ Z x -1 e -u 2 2 du
As the mean of normally distributed random variables still follow a normal law, this result can be used with N i.i.d. copies of X. We then obtain the following equality:

P p 2 " p -z p V(X) p N , p + z p V(X) p N #! =2φ(z) -1
To use this theorem one should find a suitable z. Fortunately such a z only depends on the confidence level. The value z is computed only once using implementation of the φ -1 function. Indeed, this function is well known and precise approximation exists and are implemented in standard library of most programming language.

As mentioned for the Chebyshev's inequality for a fixed distribution of X and a fixed confidence level, the confidence interval width is still proportional to 1 p N . To apply this result one needs to know V(X). Fortunately there exists an unbiased estimator of V(X).

Theorem 8

Given N i.i.d. random variables (X i ) N i=1 . Let V be a random variable equal to 1 N -1 N X i=1 (X i -Z) 2
The random variable V is an unbiased estimator of V(X) If X does not follow a normal law but the sample is large enough and both the mean and the variance of X are finite, we can use the central limit theorem:

Theorem 9 (Central Limit Theorem) Let (X i ) N
1 a sequence of i.i.d. random variables of finite expectancy E(X) and finite variance V(X)t h e nw h e nN tends to the infinity the empirical mean converges in distribution to a normal law.

p N 1 N N X i=1 X i ! -E(X) ! d !N(0, V(X))
This method can be used in a static way by simulating N samples and applying the Gaussian confidence interval method. Or one can use results from [START_REF] Chow | On the asymptotic theory of fixed-width sequential confidence intervals for the mean[END_REF] to build a sequential procedure.

Let x 1 ,x 2 ,... an infinite sequence of realizations of X. We are interested in computing a confidence interval of a given width 2d with a confidence level 1ε, using the least number of terms in (x i ). Let a = φ -1 1 -" 2 Let N ≥ 1 the number of sample be the smallest integer such that:

1 N N X i=1 x i -x N + 1 N  d 2 a 2 where x N = n X i=1 x i N
The following theorem ensure that a Monte Carlo estimator with N samples produce a confidence interval a width smaller than 2d.

Theorem 10 (Chows and Robbin)

If 0 < V(X) < 1 then 1. lim d!0 d 2 N a 2 V(X) = 1 a.s., 2. lim d!0 P (p 2 [x N -d, x N + d]) = 1 -ε 3. lim d!0 d 2 E(N ) a 2 V(X) =1
Compare to Gaussian analysis, this theorem allows to sequentially compute a confidence interval of a given width with the additional requirement that the width of the confidence interval tends to zero.

Chernoff-Hoeffding Bounds

When the distribution of the random variable X has a bounded support, Chernoff-Hoeffding's bounds [START_REF] Hoeffding | Probability inequalities for sums of bounded random variables[END_REF] allow to compute a tighter conservative confidence interval.

Theorem 11 (Chernoff-Hoeffding inequality)

Let X 1 ,...,X N be N independent random variables. Assume there exists (a i ) N i=1 and (a i ) N i=1 such that 8i, P(X i 2 [a i ,b i ]) = 1 then

P 1 N N X i=1 X i -E(X i ) ≥ z !  2exp - 2N 2 z 2 P N i=1 (b i -a i ) 2

!

Which can be written as

P 0 @ E(X) 2 2 4 p -z s P N i=1 (b i -a i ) 2 N , p + z s P N i=1 (b i -a i ) 2 N 3 5 1 A ≥ 1-2e -2z 2
Written like this, the confidence interval can be compared to the Chebyshev one. The variance have been replaced by 2 which is an upper bound on the variance. The speed of convergence with respect to N is also

P N i=1 (b i -a i )
1 p N .

Confidence Interval for Binomial law

In the case where a random variable X follows a binomial law for which we want to estimate the parameter θ, that is X takes values in {0, 1} and θ = P(X = 1), a static confidence interval can be computed on the distribution. Let x 1 ,x 2 ,...x N be a sequence of N independent realization of X. Let x = P N i=1 x i . A confidence interval for the binomial law with confidence level 1ε can be expressed as:

n θ | P Bin(N, θ)  x > ε 2 ^P Bin(N, θ) ≥ x > ε 2 o
where Bin(N, θ) denotes a random variable following a binomial distribution with N trials and a probability of success θ. This confidence interval is known as Clopper-Pearson confidence interval [START_REF] Clopper | The use of confidence or fiducial limits illustrated in the case of the binomial[END_REF]. It can be expresses using quantiles of the Beta distribution (denoted B) as:

h B ⇣ ε 2 ,x,N +1-x ⌘ ,B ⇣ 1 - ε 2 , 1+x, N -x ⌘i
The Beta distribution is a continuous function for which a good approximation exists. Therefor it is possible to numerically compute this confidence interval.

Hypothesis testing

When one is not interested in the actual value of a statistic but rather wants to decide whether this value is above or below a threshold, hypothesis testing methods can be used.

Let us focus on a binomial distribution of unknown parameter p.G i v e na threshold probability p 0 we want to know whether p>p 0 or p<p 0 with high confidence. It is not possible to decide this when p 0 is arbitrary closed to p thus we define an indifference region: ]p 0 ,p 1 [ containing p 0 and we defined two hypothesis:

• H 0 is the hypothesis that p  p 0 , • H 1 is the hypothesis that p ≥ p 1 .
Given two confidence level α and β we want that the probability to accept hypothesis H 1 whereas H 0 holds is less than α and the probability to accept hypothesis H 0 whereas H 1 holds is less than β.I fp lies in the indifference region both hypothesis are acceptable.

Given α, β, p 0 and p 1 , the Sequential Probability Ratio Test (SPRT) [START_REF] Wald | Sequential tests of statistical hypotheses[END_REF] is an optimal sequential test for deciding whether H 0 or H 1 holds.

This test is as follows: after N>0 samples, let x be the number of successful samples. We define

p 1,N p 0,N = p x 1 (1 -p 1 ) N -x p x 0 (1 -p 0 ) N -x
There are three cases:

• If p 1,N p 0,N  β 1-↵ then accept hypothesis H 0 . • If p 1,N p 0,N ≥ 1-β ↵ then accept hypothesis H 1 . • Otherwise 1-β ↵ < p 1,N p 0,N < β 1-↵ .
We cannot conclude without further simulation.

Remark 4 To avoid effectively computing p 1,N p 0,N which is likely to be numerically unstable for large value of N , the logarithm of this expression is computed:

log ✓ p 1,N p 0,N ◆ = x log ✓ p 1 p 0 ◆ +(N -x) log ✓ 1 -p 1 1 -p 0 ◆ and compared to log ⇣ 1-β ↵ ⌘ and log ⇣ β 1-↵ ⌘ .

Model Checking

Model checking has been widely used since its introduction in [START_REF] Emerson | Characterizing correctness properties of parallel programs using fixpoints[END_REF]. It allows to formally check that a system satisfies a specification in a fully automated way. More often, a model checking algorithm checks if the specification of a system holds by exploring all its possible states. This requires the construction of the reachability graph. The simplicity of this approach allows automation whereas other methods of verification often require human intervention. But for large or infinite systems it is not possible to build and explore the reachability graph. Several techniques are then used to reduce the state space. For example it is possible to generate the state space on the fly or to use symbolic representation of the states space.

For probabilistic system one can be interested in the quantitative model checking. In this setting the probabilities that a trajectory satisfy a particular property is computed, or whether this probability is above some threshold.

Specification

There are two main families of specification languages for probabilistic systems. The first is based on the Linear Temporal Logic (LTL) and specifies accepted paths in the system. The probability for a path of a system to be accepted is specified, thus this logic does not allowed nested probabilistic operator. The other one is based on the Computational Tree Logic (CTL) where only state formula are considered. In this logic probabilistic operator and until operator can be nested but have to alternate.

PLTL

The Probabilistic Linear Temporal Logic (PLTL) allows to specify whether the probability of paths in a system satisfying an LTL formula. A PLTL formula if of the form P ./ p (φ)w h e r e./ 2{<, >}, p 2 [0, 1] and φ is an LTL formula given by the following grammar:

φ ::= q |¬φ | φ _ φ | φ ^φ | φ U I φ | X I φ
Where q is an atomic proposition of the system, ¬, _ and ^have there usual meanings. LTL is a path-based logic, that is the semantics of all operators is defined on a path of the system. The U and X operator are equipped with an interval I ⇢ R + . The until operator means that along a path, the formula of the left member must hold until a state where the formula of the right member holds. The X operator means that in the next discrete step the formula must hold. The formal semantics is as follows: given a non-Zeno realization of a DEDS σ =( (

s i ) 1 i=0 , (t i ) 1 i=0
) and a time t,l e tn such that P n i=0 t i  t< Generalization of this logic consists in replacing the LTL formula by some kind of automaton specifying which are accepting paths.

P n+1 i=0 t i σ, t |= q if s n |= q σ, t |= ¬φ if σ, t 6 |= φ σ, t |= φ 1 ^φ2 if σ, t |= φ 1 and σ, t |= φ 2 σ, t |= φ 1 _ φ 2 if σ, t |= φ 1 or σ, t |= φ 2 σ, t |= X I φ if with t 0 = P n+1 i=0 t i ,t 0 -t 2 I and σ, t 0 |= φ σ, t |= φ 1 U I φ 2 if

PCTL

The PCTL logic [START_REF] Hansson | A logic for reasoning about time and reliability[END_REF] allows to specify properties using nested probabilistic operator thus branching properties. The PCTL logic is state based logic but it contains path based subformula. It is described by the following grammar:

::= q |¬ | _ | ^ | P ./ p (φ) φ ::= U I | X I
where denotes a state based formula and φ denotes a path based formula. A PCTL formula always starts by a state based fromula. LTL operators have their usual meaning: P ./ p 1 U 2 specify whether the probability for 1 U 2 to holds is above or below the threshold p in the initial state. The semantics of path based operator is the same as in LTL. The formal semantics of state base operators is as follows, with s 2 S,

s |= q if s |= q s |= ¬φ if s 6 |= φ s |= φ 1 ^φ2 if s |= φ 1 and s |= φ 2 s |= φ 1 _ φ 2 if s |= φ 1 or s |= φ 2 s |= P ./ p if P ⇣ (S i ) 1 i=0 , (T i ) 1 i=0 , 0 |= φ | S 0 = s ⌘ ./ p
A generalization of this logic is the Continuous Stochastic Logic (CSL) with an extra operator allowing to specify properties of the steady state probability.

Numerical Model Checking

Numerical models are best suited for PCTL type formula. Considering the formula as a tree, they are evaluated starting from the leaf. Given a sub-formula consisting of one probabilistic operator at the root and no nested probabilistic operator, the evaluation of U and X formulas can be reduced to time-bounded or time unbounded reachability problems as sub-formulas are state formulas. These reachability problems are solved using Definition 9, 14 or Theorem 3. Once the sub-formula is evaluated on each state it can be replaced by a proposition in the upper sub-formula until the root is reached. One can refer to the chapter 10 of [START_REF] Baier | Principles of model checking[END_REF] for details about model checking of probabilistic systems.

For a linear specification, the formula is translated into an automaton, the synchronized product of the stochastic system with the automaton is then built. This product is still a stochastic system, where the model checking problem reduced to time-unbounded reachability. If the formula is large the size of the automaton and the size or the product may become problematic.

Most numerical model checking algorithms rely on matrix vector multiplication, usually taking advantage of the sparsity of transition probability matrices. However these computations are difficult to parallelized. The tool Marcie [START_REF] Heiner | MARCIE -Model checking And Reachability analysis done effiCIEntly[END_REF] features a parallel implementation of this method but still a non negligible part of the computation cannot be performed in parallel. Due to Amdahl's law [START_REF] Amdahl | Validity of the single processor approach to achieving large scale computing capabilities[END_REF], the gain of parallel computing is limited.

Strong probabilistic hypotheses on the stochastic system to be analyzed are mandatory. The easiest case is to suppose that the system is Markovian and have a finite state space. Even with theses strong hypotheses the system of linear equations required by the computation can be huge. In that case, the computation of solutions becomes difficult and therefore requires iterative methods. Only on approximated solution of such a system is then obtained.

For a more complex linear specification like one given by a time automaton, efficient techniques exist only if there is at most one clock in the automaton see for example [START_REF] Donatelli | Model checking timed and stochastic properties with CSL TA[END_REF] and [START_REF] Barbot | Efficient ctmc model checking of linear real-time objectives[END_REF].

This approach can be adapted to deal with non determinism [START_REF] Vardi | Automatic verification of probabilistic concurrent finite state programs[END_REF] and to tackle model checking problem on Markov decision processes.I nt h i ss e t t i n g probabilistic transitions are alternated with non deterministic ones.

This approach has been efficiently implemented in several tools with different types of Markovian systems. For example Prism [START_REF] Kwiatkowska | Prism: Probabilistic symbolic model checker[END_REF] can deal with Discrete Time Markov Chains, Continuous Time Markov Chains, probabilistic automata and Markov decision processes. Uppaal [START_REF] Behrmann | Developing uppaal over 15 years[END_REF] deals with various models of automata. GreatSPN [START_REF] Chiola | Greatspn 1.7: Graphical editor and analyzer for timed and stochastic Petri nets[END_REF] and Marcie [START_REF] Heiner | MARCIE -Model checking And Reachability analysis done effiCIEntly[END_REF] deal with Stochastic Petri nets. Others tools exist: Mrmc [START_REF] Katoen | The ins and outs of the probabilistic model checker MRMC[END_REF], . . .

Statistical Model Checking

The alternative to numerical approach when dealing with large systems is to use statistical methods. The statistical model checking relies on a Monte Carlo algorithm to estimate the probability of interest. Precisely, on a linear specification, a random variable X is defined which takes 1 as value on a trajectory which satisfies the specification and 0 otherwise. Thus this variable follows a Benouilli law and we compute its expected value E(X).

This method is more adapted to linear specifications. The Monte Carlo algorithm simulates a large number of trajectories, say N , and counts the number of those trajectories satisfying the specification. An estimation of the probability is obtained as the ratio of the number of successful trajectories on the total number of trajectories. The random variable Z is defined as the mean of N independent copies of X:

Z = 1 N N X i=1 X i
Statistical model checking can be parallelize very easily: it suffices to run several simulators of the system on different processors or different machines and take the mean result of trajectories of all simulators. Particular attention is required on the random number generator to ensure that all generated trajectories are independent one from each others. As the only sequential operation in this approach is to compute the mean value and possibly a confidence interval, there is almost no cost in using parallel computing.

Statistical method can be naturally extended to performance evaluation. Instead of computing a probability, the expected value of a random variable those values depend on trajectories of the system is computed.

Comparison Between Numerical and Statistical Model Checking

Statistical methods have several advantages compared to numerical ones.

1. The memory required to simulate one trajectory in the system is usually very small. Thus the memory requirement of a statistic model checker is very low. Whereas the memory is often the bottleneck of numerical methods.

3. As all trajectories are independent, it is straightforward to parallelize the simulation whereas as we already mentioned, it is much more difficult for numerical computation.

4. Complex linear properties like the one expressed in the Hybrid Automaton Stochastic Logic (HASL) [START_REF] Ballarini | HASL: An expressive language for statistical verification of stochastic models[END_REF] can be efficiently evaluated.

However, the statistical method has several drawbacks:

1. If precise results are required, which means that we want a tight confidence interval, the time of computation can become big. In general the width of the confidence interval decreases according to the square root of the number of simulations. This means that in order to divide by two the width of the confidence interval we need to perform four time more simulations.

2. Difficulties can occur when the simulation is used to compute a stationary distribution. The difficulties lie in the fact that a stationary distribution is defined as a limit of distributions of states after an infinite number of time unit. As simulations are finite, the initial distribution introduces a bias which can be difficult to correct.

3. Statistical model checking is not well suited to evaluate logic based on state formula because it requires to perform simulations starting from each state which is too costly.

4. For properties expressed as nested CSL probabilistic operator, the difficulty relies on the fact that nested probabilistic operators require to estimate first the most nested operator and then to evaluate the formula bottom-up. For each nesting new simulations of the system is needed. Careful computations of confidence interval are required to take into account the errors at each level of nesting. Details can be found in [START_REF] Younes | Statistical probabilistic model checking with a focus on time-bounded properties[END_REF].

5. Statistical model checking cannot evaluate unbounded until operator φ U : if there is a bottom strongly connected component in the system where φ holds, as the simulator only evaluates finite trajectories, it cannot decide if the formula holds. A workaround is described in [START_REF] He | A bounded statistical approach for model checking of unbounded until properties[END_REF]. Tools rely on more pragmatic approaches: either the unbounded until is not available in the specification language (ex Uppaal) or the simulator does not terminate when simulating a trajectory always satisfying φ (ex Cosmos).

6. Finally crude Monte Carlo estimator is unsuitable to compute very small probabilities as for the probability of rare event. This problem is the subject of the next chapter. Contrary to statistical model checker, numerical one are not subject to rare events. In some case, reachability of a rare event may lead to computations using ill-conditioned transition probability matrices. In this case the convergence time of iterative method may increase. Chapter 3

Rare Events

Introduction

The main drawback of statistical methods is the rare-event problem. This problem occurs when the probability that a trajectory satisfies a formula is very small, say less than 10 -9 . The satisfaction of such formula is then called a rare event with respect to the model. In order for statistical methods to work in this case, a large number of trajectories must be generated to obtain an accurate estimation of the result. The time required to produce such a large number of trajectories makes the statistical approach in the rare-event setting unfit. To enhance Monte Carlo methods, two main families of algorithms tackling this problem have been developed, namely splitting and importance sampling. In Section 3.2, the rare-event problem is studied in details. In Section 3.3, splitting methods are presented and importance sampling methods are discussed in Section 3.4.

Rare-Event Problem

Let us illustrate the rare-event problem with an example. Assume we want to estimate the probability of an event occurring with a probability smaller than 10 -15 with a confidence level of 0.99. Suppose we are able to compute up to a billion trajectories. Let us describe the results of plain Monte Carlo simulation depending on all possible outcomes.

• With a large probability, approximately 1 -10 -6 , the rare event does not occur in any trajectory. In this case, the corresponding confidence interval is [0, 7 10 -9 ]. This confidence interval contains the value we are looking for, but the width of this confidence interval is large. The precision of the result is thus very low.

• With a small probability, smaller than 10 -6 , the rare event occurs in one trajectory and the corresponding confidence interval is [7 10 -10 , 2 10 -9 ]. This confidence interval does not contain the expected value.

• With an even smaller probability, the rare event occurs in more than one trajectory and the confidence interval does not contain the expected value either.

In all cases, the number of trajectories required to get an accurate estimate of the probability with the plain Monte Carlo statistical method is intractable.

In this example the result is not satisfactory. By modeling the occurrence of a rare event by a Bernouilli variable X with the unknown parameter p, another way to illustrate the rare-event problem is to look at the variance of the random variable Z associated with the Monte Carlo estimator of the rare event. This random variable follows a binomial law whose variance can be computed by the following formula V(Z)= p(1-p) N with p = E(X) the probability of the event to occur. When p is small, the variance can be approximated by p N as the term in O(p 2 )i s negligible. A confidence interval may be computed using for instance the Chebyshev Inequality. The confidence interval width is then proportional to the standard deviation up to a factor depending of the confidence level. As the standard deviation is the square root of the variance, the confidence interval width is proportional to p p p N .W h e np is small, the width of the confidence interval is big compared to p, making the estimation not accurate. Other more involved methods (like Clopper-Pearson confidence interval) producing a confidence interval reduce the factor depending of the confidence level. However the width of the confidence interval remains proportional to

p p p N .
Experimentally, this can be observed by simulating a parametrized system where the probability of the event of interest can be made arbitrary small using a parameter. Then, by estimating the value of this probability with a confidence interval of fixed relative width (i.e., the ratio of the confidence interval width on the probability is always below some threshold) with a sequential method, one observes that the simulation time quickly increases when the probability tends to zero. Figure 3.1 illustrates this on a tandem queue system. When the probability becomes tiny, we observe that the simulation time explodes.

Most methods that compute confidence intervals use the variance, but this variance is unknown. Only estimations of the variance are available when one is computing the probability of a rare event. For instance, suppose that we want to estimate the probability of occurrence of a rare event with a probability less than 10 -6 but not equal to zero. In general two approximations are usual for the computation of the confidence interval. The first one assumes that the estimator follows a normal law, its approximation being justified by the central limit theorem. The second one consists in replacing V(X) by its estimation in the computation of the confidence interval. This second approximation is not correct when dealing with a rare event as the estimator for V(X) is also subject to rare event problem.

Suppose we are able to run a thousand simulations: in most cases, the event will not occur in these thousand simulations. Thus the empirical variance computed with this estimator is equal to 0. A direct application of Theorem 7 with any confidence level returns a confidence interval equal to the singleton [0, 0]. This result is a underestimation of the true value. Underestimation of the value of interest is a characteristic of rare events. It may occur even when specific methods to deal with rare events are used. Some empirical rules exist to indicate whether or not one can invoke the convergence to the normal law. For instance, one rule states that both N E(X) and N (1 -E(X)) should be greater than 5. Most of them do not apply to rare events as E(X) is too small. Historically, this problem has been first studied in particle transmission problem [START_REF] Kahn | Estimation of particle transmission by random sampling[END_REF], where one is interested in computing the probability for a single particle to get across a material. If the material is thick enough, the probability is very small but as the initial number of particles can be large, the number of particles going through can still be significant. In this context, a small number of particles crossing the material can cause radiation poisoning so that getting an accurate analysis is critical. This problem has then been analyzed in a large variety of contexts. In general, estimating a tiny probability is important when the impact of a single occurrence of the event is tremendous and/or the process that contains this event is iterated a large number of times.

• In biology, rare-event probability problems can occur for estimating the speed of some molecular reactions. As the number of molecules is huge, even small probabilities of reaction lead to observable consequences.

• In telecommunication, probabilities of overflow or loss of packet could be small but the number of packets handled by a telecommunication system could be huge.

• In insurance, the probability of a stock market to crash should stay small to avoid drastic consequences.

• The dependability of a mechanical system could be quantified with probabilities. For critical system, this probability should be small. The probability of failure should be small not just for a given short period of time but for the whole lifetime of a system which may be very long.

Splitting

A classical method to deal with rare events with Monte Carlo simulation is the splitting technique [START_REF] Harder | Rare events, splitting, and quasi-Monte Carlo[END_REF]. A detailed description can be found in Chapter 3 of [START_REF] Rubino | Rare Event Simulation using Monte Carlo Methods[END_REF] or in [START_REF] Garvels | The splitting method in rare event simulation[END_REF]. According to [START_REF] Kahn | Estimation of particle transmission by random sampling[END_REF], Dr. Von Neumann proposed the splitting technique in the late forties. It is usually defined by a sequence of successively embedded subsets of the state space of a Markov chain γ k ⇢•••⇢γ 0 ,w h e r e γ k contains exactly the states satisfying the rare event and γ 0 contains all states of the system except a sink state. The probability of interest p is the probability to reach a state of γ k before returning to the sink state. In more general settings, this method can be used to estimate the probability to reach γ k before an arbitrary stopping time. Each time a trajectory reaches the next subset for the first time, this trajectory is split into several independent trajectories. Each trajectories carry a weight which is also split uniformly in each subtrajectories. Trajectories which start in the initial state have a weight of 1. Let n i be the number of trajectories in which a trajectory reaching γ i is split, n 0 being the number of trajectory starting from the initial state. With H i being the number of trajectories reaching γ i (H 0 = 1), the following is an unbiased estimator of p:

H k n 0 n 1 •••n k-1 .
This can be seen by introducing p i = P(γ i |γ i-1 ) the probability that a trajectory entering for the first time γ i-1 reaches a state of γ i before reaching the sink state. Then the following equalities hold:

E ✓ H k n 0 n 1 •••n k-1 ◆ = n 0 n 1 ...n k X i=0 E ✓ H k H k-1 n k-1 H k-1 ◆ iP(H k-1 = i) n 0 n 1 •••n k-2 = n 0 n 1 ...n k-1 X i=0 P(γ k |γ k-1 ) iP(H k-1 = i) n 0 n 1 •••n k-2 = p k E ✓ H k-1 n 0 n 1 •••n k-2 ◆ = ••• = p 1 p 2 ...p k = p

These equalities hold since

H i H i-1 n i-1 is an unbiased estimator of P(γ i |γ i-1 ).
As trajectories coming close to the rare event are split, the probability to have a trajectory reaching the rare event is larger than in a crude Monte-Carlo simulation.

γ 3 γ 4 γ 2 γ 1 γ 0 1 2 1 4 1 8 1 8 Split Split Split s 0 1 1 2 1 4
Sink Example 5 To illustrate the splitting method, a simple example can be built from a coin tossing problem. Suppose we want to estimate the probability for an unbiased coin to return head 25 times consecutively. This probability is 1 2 25 ⇡ 3 10 -8 . Using Monte-Carlo simulation, the rare-event problem arises. This example can be formalized in a BDTMC where the state space is {0, 1,...,25,s -},t h ei n i t i a ls t a t ei s0, the accepting state is 25,a n ds - is the rejecting state. States 25 and s -are absorbing, each probability to go from states i to state i +1 is 0.5 and the probability to go to s -is 0.5. The probability for a trajectory to reach state 25 is 1 2 25 which is rare. A splitting technique can be used here. The embedded subsets are chosen as γ 0 = {0, 1,...,25} γ 1 = {5, 6,...,25}, γ 2 = {10, 11,...,25},... γ 5 = {25}. The number of replications (n i ) is constant and equals to 50.

In this setting, the probability p 1 = p 2 = ...p 5 can be computed and is equal to 1 -1 2 5 ⇡ 0.03. The probability of having at least one trajectory reaching 25 is crudely under-approximated by the probability that there is one trajectory reaching γ i+1 from γ i for each 1  i  5 :

(1 -(p 1 ) 50 ) 5 ⇡ 0.32
The mean number of trajectories reaching the state 25 is (n 0 p 1 ) 5 ⇡ 9.3. These calculations show that using splitting, observing a trajectory reaching the target state is no longer a rare event and the mean number of trajectories to be generated is tractable. By comparison in plain Monte-Carlo, to obtain a probability of 0.32 to observe a trajectory reaching 25, a number of trajectories around 12 10 6 is required.

In a simplified setting, a simple expression for the variance of this estimator can be obtained. We make the assumption that for any subset of γ i , the probability to reach γ i+1 is independent of the state of entry in γ i and is independent of the trajectory history. This is equivalent to say that there is only one entry state for each subset and this state fully characterizes the stochastic process. In this setting, for all i  k, the random variable H i follows a binomial law with parameters n i-1 H i-1 and p i . The variance is computed as follows:

V (H k )=E (V (H k |H k-1 )) + V (E (H k |H k-1 )) = E (n k-1 H k-1 p k (1 -p k )) + V (n k-1 H k-1 p k ) = n 0 p 1 n 1 p 2 ...n k-1 p k (1 -p k )+n 2 k-1 p 2 k V (H k-1 ) = ... = k X i=1 n 2 i p 2 i+1 ...n 2 k-1 p 2 k n 0 p 1 ...n i-1 p i (1 -p i ) = p 2 1 ...p 2 k n 2 0 ...n 2 k-1 k X i=1 1 -p i n 0 p 1 ...n i-1 p i . Thus V ✓ H k n 0 n 1 ...n k-1 ◆ = p 2 k X i=1 1 -p i n 0 p 1 ...n i-1 p i (3.1)
In the optimal case, for 0 <i<k, n i are chosen such that n i p i+1 ⇡ 1, thus the number of trajectories in each subset is roughly constant, the expression of the variance becomes:

p 2 P k i=1 (1 -p i ).
This function takes its minimal value for p 1 = p 2 = p k = k p p where it becomes p 2 k(1k p p), which converges toln(p)w h e nk goes to the infinity.

Example 6

The coin tossing problem fulfills the requirement for using Equation (3.1). The variance is around 1. 53 10 -15 . The number of trajectories required by the Monte-Carlo estimator to achieve the same variance is around 19 10 6 .

The main difficulty in the splitting method is to choose the sequence of subsets. In general, an importance function [START_REF] Garvels | On the importance function in splitting simulation[END_REF] is defined on the states of the system and the sequence of subsets is defined as the sequence of sets of states where the importance function exceeds a threshold.

Another crucial parameter is the number of offsprings at each split. If it is too small no trajectory reaches the rare event and if it is too large the number of trajectories may become unmanageable. In an optimal setting, the number of offspring at each split is inversely proportional to the probability to reach the next subset. In this setting, the total number of trajectories in any subset is constant. This optimal setting is theoretical as the probability to reach the next subset is unknown. Different strategies exist to keep the number of trajectories large enough but still manageable, which dynamically compute the number of splits n i . In this case, each trajectory is annotated by the number of siblings at each split.

Additionally to splitting trajectories on transitions coming closer to the goal, trajectories taking a transition going away from the goal may be killed, using a russian roulette, to reduce the number of paths to manage. It is also possible to get rid of the sequence of subsets and decide for each transition whether to split or kill the trajectory according to the evolution of the importance function by this transition. For example, the RESTART algorithm [START_REF] Garvels | The splitting method in rare event simulation[END_REF] is a splitting technique that aims at keeping the number of trajectories manageable and preventing death of all trajectories.

This method requires to deal with numerous parameters: the sequence of subsets, the number of splittings of the trajectories, etc. The number of parameters helps the user to find a good parametrization but in return makes the automation difficult. Moreover, variance reduction is not easy to guarantee theoretically.

As the system is unmodified by splitting methods, these methods can be used in a black box setting as long as it is possible to make a copy of the system. Such techniques are efficient when some expert knowledge on the system is available. In particular, to design the sequence of embedded subsets, one needs to know which part of the state space is close to the rare event. These methods work the best when the probability to reach the rare event is uniformly distributed on the boundary of each set γ i ,l i k e in the setting used for the computation of the variance. They have been successfully used in various settings, for example in the setting of particle transport (see for instance Chapter 10 of [START_REF] Rubino | Rare Event Simulation using Monte Carlo Methods[END_REF] ), in chemical physics [START_REF] Allen | Simulating rare events in equilibrium or nonequilibrium stochastic systems[END_REF], in telecommunication [START_REF] Akin | Efficient simulation of TCP/IP networks characterized by non-rare events using DPR-based splitting[END_REF], in dependability [START_REF] Jegourel | Importance splitting for statistical model checking rare properties[END_REF], etc.

Importance Sampling

Importance sampling [START_REF] Glynn | Importance sampling for stochastic simulations[END_REF] is based on a different idea. It relies on the modification of the probability distribution in the system. This modification introduces a bias which will increase the probability that a rare event occurs. During the simulation of a trajectory, it is possible to keep track of the introduced bias with a so called likelihood ratio. By weighting the trajectories with their likelihood, it produces an unbiased estimator of the probability of the rare event. The difficulty is then to find a correct bias for the system. It is easy to prove that there exists an optimal modification of the system which computes the probability of the rare event with only one simulation, In this case, the estimator has a zero variance. But an erroneous modification of the system can, on the contrary, gives an estimator which is "worse" than the initial one. The efficiency of an importance sampling is often evaluated by looking at its variance: the smaller the variance is, the faster the convergence to the expected value will be.

In this thesis, we only consider importance sampling on DTMC.I ti s possible to define it for more general settings with arbitrary continuous distributions. In this case, the likelihood ratio is defined in terms of Radon-Nikodym derivative dP/d P where P is the initial probability distribution and P is the biased one.

Definition

Recall that in a BDTMC, all trajectories starting from s 0 reach the state s + or state s -with probability 1. The problem we are focusing on in the following is that, "Given a BDTMC, what is the probability to reach s + ?". Formally, we introduce the random variable V s 0 such that for all trajectories σ = s 0 ,s 1 ,...,s n ,s ± ending in s + or s -of a BDTMC,

V s 0 = ⇢ 1i f σ ends in state s + 0i f σ ends in state s -.
We are interested in estimating E(V s 0 ). The formal definition of importance sampling is as follows:

Definition 20 (Importance Sampling)

Given a BDTMC C =(S, s 0 , P,s -,s + ), and a biased transition probability matrix P 0 such that:

8s, t 2 S, P(s, t) > 0 ^t 6 = s -) P 0 (s, t) > 0 (3.2)
The random variable W s 0 is defined such that for all trajectories σ = s 0 ,s 1 ,...,s n ,s ± ending in s + or s -of the biased BDTMC (S, s 0 , P 0 ,s -,s + ),

W s 0 = ( P(s 0 ,s 1 ) P 0 (s 0 ,s 1 ) • P(s 1 ,s 2 ) P 0 (s 1 ,s 2 ) • ... • P(sn,s + ) P 0 (sn,s + ) if σ ends in state s + 0i f σ ends in state s -
A statistical method which estimates W s 0 in order to estimate V s 0 is called an importance sampling.

The value taken by the random variable W s 0 when it reaches the state s + is the likelihood.

Remark 5 Condition 3.2 ensures that the introduced bias cannot remove transitions that have not s -as target. This is necessary to ensure that all trajectories in the original model are valid trajectories in the biased model.

This condition allows one to add new transitions in the bias distribution but trajectories that take such transitions in the bias model have a likelihood of zero and thus behave as trajectories reaching s -which reduces the probability to reach s + .

Thus when one is building a biased model, only DTMC with the same underlying graph should be considered. This is not very restrictive as one can still choose all the probability transitions in this graph.

The following proposition establishes the correctness of the method.

Theorem 12

Given a BDTMC C =(S, s 0 , P,s -,s + ), and a biased transition probability matrix P 0 , such that P 0 defines an importance sampling on C with associated random variable W s 0 then

E(W s 0 )=E(V s 0 ).

Proof:

In all states, the probability to reach s -or s + is equal to 1. Then thanks to Equation 8, the expected value of the random variable V s is the unique solution of the following system of equations:

8 > > > < > > > : E(V s -)=0 E(V s + )=1 8s/ 2{s -,s + } E(V s )= X s 0 6 =s - P(s, s 0 )E(V s 0 ) (3.3)
We now write the corresponding system for P 0 with correction factor:

8 > > > > < > > > > : E(W s -)=0 E(W s + )=1 8s/ 2{s -,s + } E(W s )= X s 0 6 =s - ^P0 (s,s 0 )>0 P 0 (s, s 0 ) ✓ P(s, s 0 ) P 0 (s, s 0 ) ◆ E(W s 0 )
Thanks to the restriction of Equation 3.2, the two systems are equal after simplification, and we have E(W s 0 )=E(V s 0 )=p. ⇤

A good choice of P 0 should reduce the variance of W s 0 w.r.t. to the variance of V s 0 .

Running example.

As the tandem queue system is parametrized by the three probabilities λ, ⇢ 1 and ⇢ 2 an importance sampling can be built by changing these parameters to λ 0 ,⇢ 0 1 and ⇢ 0 2 . In this example we are interested in estimating the probability to reach a state with at least 5 clients before reaching the state where the system is empty. This probability is equal to 0.0089. An importance sampling is built by increasing the value of the probability of an incoming client λ 0 and reducing the probability of a client leaving the system ⇢ 0 2 . Figure 3.3 shows the evolution of the variance when λ 0 increases.

We observe that starting from λ 0 = λ =0.1, the variance decreases when λ 0 increases until a minimal value is reached, then the variance increases. This shows the difficulty to find a good importance sampling where the variance is significantly smaller than in the initial model.

Zero Variance Importance Sampling

The following proposition shows that for any BDTMC, there exists a matrix P 0 which leads to a null variance. We denote the probability to reach s + starting from s by µ(s). Let P 0 be defined by:

• For all s such that µ(s) 6 = 0, P 0 (s, s 0 )= µ(s 0 ) µ(s) P(s, s 0 )

• For all s such that µ(s) = 0, P 0 (s, s 0 )=P(s, s 0 ) Then for all s,w eh a v eV(W s ) = 0.

Proof: If µ(s) = 0 then all trajectories starting in s end in s -. Therefore the variance is null.

If µ(s) 6 = 0, thanks to the equation:

µ(s)= X s 0 |µ(s 0 )>0
P(s, s 0 )µ(s 0 ) P 0 (s, -) is a distribution. A trajectory starting from a state s with µ(s) > 0 only visits states s 0 with µ(s 0 ) > 0, so it ends in s + . Denoting by s = u 0 ,...,u l = s + such a trajectory, the value L is equal to

µ(u 0 ) µ(u 1 ) µ(u 1 ) µ(u 2 ) ... µ(u l-1 ) µ(u l ) = µ(s).
⇤ This result has ap r i o r ino practical application since it requires the knowledge of µ for all states, whereas we only want to estimate µ(s 0 ). Several importance sampling methods including the one presented in this thesis are based on approximating the vector µ and plugging this approximation into the zero-variance importance sampling biased matrix.

Infinite Variance Importance Sampling

Unfortunately a poor choice of P 0 can lead to an importance sampling with infinite variance. Let us illustrate this on an example.

Let us define a BDTMC C with three states s 0 , s + and s -,w h e r es 0 is the initial state, s + is an absorbing state and s -is not reachable from s 0 . Let P be its transition probability matrix. P(s 0 ,s 0 )= 1 2 = P(s 0 ,s + ), P(s + ,s + ) = 1 and P(s + ,s 0 ) = 0. We are interested in the probability to reach s + , this probability is indeed equal to 1.

s + 1 s 0 1 2 1 2 s - 1 Figure 3.

4: Markov Chain illustrating the infinite variance importance sampling

We can use an importance sampling on this Markov chain by defining a new transition probability matrix P 0 , P 0 (s 0 ,s 0 )= 1 4 , P 0 (s 0 ,s + )= 3 4 and P 0 = P for other transitions. Let W s 0 be the random variable associated to this importance sampling. Let L n be the likelihood of a path reaching s + in n steps.

L n = n-1 Y i=1 P(s 0 ,s 0 ) P 0 (s 0 ,s 0 ) ! P(s 0 ,s + ) P 0 (s 0 ,s + ) = n-1 Y i=1 1 2 4 1 ! 1 2 4 3 = 2 n 3 .
The probability of such a path is p 0

n p 0 n = n-1 Y i=1 P 0 (s 0 ,s 0 ) ! P 0 (s 0 ,s + )= n-1 Y i=1 1 4 ! 3 4 = 3 4 n .
We compute the variance of the random variable W s 0 .

V(W s 0 )=E(W 2 s 0 ) -E 2 (W s 0 )= 0 @ X n≥1 L 2 n p n 1 A -1 V(W s 0 )= 0 @ X n≥1 ✓ 2 n 3 ◆ 2 3 4 n 1 A -1= 0 @ X n≥1 1 3 1 A -1
As this series does not converge the variance of W s 0 is infinite. This example shows that a poor choice of importance sampling may lead to an infinite variance estimator. In this case, there is no guarantee that the estimator converges or even worse it could seem to converge to a value which is not the probability that we want to compute.

Distribution of W s 0

The major drawback of importance sampling is that the random variable V s 0 which follows a Bernoulli law is replaced by the random variable W s 0 whose distribution is unknown. This is a problem for computing a confidence interval. Indeed all methods to compute a confidence interval rely either on the range of values taken by the random variable or on its variance. When the distribution is unknown there is no bound on the values taken by the random variable and the variance may be difficult to estimate.

When importance sampling is used, trajectories may use transitions whose probability is smaller in the biased model than in the initial one. Such transitions are unlikely but carry a large likelihood. The case of infinite variance is the worst instance of this phenomenon. It may happen with high probability that after a consequent number of simulations no such trajectories have been observed, thus the expected value of W s 0 is underestimated. Moreover as the variance of W s 0 is also underestimated, confidence intervals computed to frame E(W s 0 ) will be wrong. This problem is well known and frequently happens. Results of an importance sampling algorithm are usually taken as underestimations of the probability of interest.

Running example. In Figure 3.3, an estimation of the variance of the random variable W s 0 is plotted, the analysis can be enhanced by plotting the probability density function of the random variable W s 0 for several values of λ 0 . Figure 3.5 shows such a plot for the tandem queue system. For λ 0 = λ, W s 0 = V s 0 and thus it follows a Bernoulli law. For other values of λ 0 , the probability density function is continuous and when λ 0 increases, the probability for a trajectory to have a large likelihood drastically increases.

Efficiency of Importance sampling

To assert the quality of importance sampling, classical approaches consider that the model is parametrized by ε and that the probability of the event of interest converges to 0 as ε goes to 0. Then the behaviors of the importance sampling for the rare event is studied for value of ε close to zero. One measure is the relative error :

Definition 21

The relative error of an estimator is half the width of the Gaussian confidence interval divided by the mean value.

An unbiased estimator Z is said to have Bounded Relative Error (BRE) if the relative error remains bounded when ε tends to 0. It is equivalent to say that the ratio E(Z 2 ) E 2 (Z) is bounded. This property is very strong and difficult to check in practice. A weaker property is the asymptotic optimality. An unbiased estimator Z is asymptotically optimal with respect to ε if lim

"!0 log E(Z 2 ) log E(Z) =2.
These two efficiency properties can be strengthened to take into account higher moments of the estimator [START_REF] Blanchet | Asymptotic robustness of estimators in rare-event simulation[END_REF].

State of the Art on Importance Sampling

There are two main categories of importance sampling: state-dependent and state-independent importance samplings, depending of the level on which the definition of the importance sampling takes place.

State-dependent importance sampling

The state-dependent importance sampling is defined at the level of DTMC (or CTMC in a more general setting). If the DTMC is huge or infinite, it becomes difficult to manage a representation of the new distribution. Moreover it is unfeasible to manually define the biased transition probability matrix as there are as many parameters as transitions in the DTMC. Several works rely on decomposing transitions of the DTMC into transitions that bring the system closer to the rare event and those that make the system further from it. In [START_REF] Shahabuddin | Importance sampling for the simulation of highly reliable Markovian systems[END_REF], a threshold γ is chosen, then in each state of the Markov chain, outgoing transitions are weighted such that, the sum of probabilities of all transitions that bring the system closer to the rare event is at least 1γ.

The partition of transitions in the set of transitions that bring the system closer to the rare event and the set of transitions that bring it further from the rare event is manually defined. By making γ small enough, the event of interest is no longer a rare event in the biased model. Approximations of the zero-variance importance sampling by approximating the probability to reach the target state µ have been extensively studied for example in [START_REF] Juneja | Rare-event simulation techniques: an introduction and recent advances[END_REF]. In [START_REF] Ecuyer | Approximating zero-variance importance sampling in a reliability setting[END_REF], the zero-variance importance sampling is approximated by using most typical trajectories to reach the rare event. This method is extended in [START_REF] Reijsbergen | Automated rare event simulation for stochastic Petri nets[END_REF] to stochastic Petri nets by analyzing the structure of the underlying Markov chain.

Adaptive methods for learning the vector µ during the simulation to produce better and better approximations have also been studied [START_REF] Halton | Sequential Monte Carlo[END_REF]. The algorithm works as follows 1. Choose an initial probability vector µ 1 .S e ti = 1.

2. Build a transition probability matrix P i from the vector µ i using the optimal importance sampling formula which is normalized to obtain a stochastic matrix:

P i (s, s 0 )= 1 P s 00 2S µ i (s 00 ) µ i (s) P(s, s 00 ) µ i (s 0 ) µ i (s) P(s, s 0 )
3. For each state s, estimate µ i+1 (s), the probability to reach s + from s using the importance sampling induced by P i . [START_REF] Allen | Simulating rare events in equilibrium or nonequilibrium stochastic systems[END_REF]. Set i = i + 1. Go back to step (2) until a stopping criterion is reached.

The process iterates until the obtained importance sampling is efficient enough. Under some regularity assumptions, it is proven in [START_REF] Kollman | Adaptive importance sampling on discrete Markov chains[END_REF] that the sequence of vectors (µ i ) converges exponentially toward µ.

State-independent importance sampling

The state-independent importance sampling is an importance sampling defined on a high-level model using parameters. In high-level models, transitions of the Markov chain are not defined independently. All transitions of the Markov chain depend on a small number of parameters. For instance in the tandem queue example, there are three parameters which are the rate of arrival and the rate of the services. In the corresponding Markov chain, several transitions have the same rate. State-independent importance sampling consists in modifying the probabilities or rate of transitions at a high level and not for each state of the Markov chain independently. Importance sampling where the probability of transition depends on the states but only relies on a few parameters can also be called state independent. They have the same properties as a real state-independent importance sampling. For example, consider a queue with an infinite server policy, that is the probability of a client to be served increases linearly with the number of waiting clients. This system is only parametrized by the probability to serve a single client. An importance sampling in this setting can only change this parameter and modify all transition probabilities in a different way. However as all transition probabilities depend on the same parameter, the biasing of transition probabilities does not really depend on each state.

As the number of parameters is usually small, the associated importance sampling is also easy to define. Some of these methods are:

• Trying several values of each parameter and searching experimentally for good values. This is feasible only if the number of parameters is very small and their values are bound.

• The parameters may have some meaning to the modeler who can use his knowledge on the system to design the biased transition probability matrix.

• Optimization methods can be used to find the best parameter values that minimize the variance. Among optimization methods, the crossentropy method is detailed below.

Unfortunately if no values of parameters characterize the optimal importance sampling which is frequent, there is no guarantee to find an efficient importance sampling. Indeed, the number of degrees of freedom is equal to the number of parameters whereas in the case of state-dependent importance sampling, the number of degree of freedom is the number of transitions in the Markov chain. The problem of finding an efficient importance sampling then relies on finding a correct parametrization of the system, and, when using iterative optimizing methods, on finding suitable initial parameters.

Running example. We illustrate the limit of state-independent importance sampling in the tandem queue system which has been heavily studied. By choosing as parameters the three probabilities λ, ⇢ 1 and ⇢ 2 , the optimal stateindependent importance sampling is obtained by exchanging the value of λ with the largest value among ⇢ 1 and ⇢ 2 [START_REF] Parekh | A quick simulation method for excessive backlogs in networks of queues. Automatic Control[END_REF]. It is referred in the following as PW importance sampling. This importance sampling is induced by results in large deviation theory. In this setting, an importance sampling is asymptotically efficient if the number of simulations required to obtain a given relative error grows less than exponentially with the overflow level N .I n [START_REF] Sadowsky | Large deviations theory and efficient simulation of excessive backlogs in a GI/GI/m queue. Automatic Control[END_REF], it is shown that only PW importance sampling may be asymptotically efficient. In [START_REF] Glasserman | Counterexamples in importance sampling for large deviations probabilities[END_REF], it is proven that for some values of the parameters, PW importance sampling is not asymptotically efficient and its variance may even be infinite [START_REF] Boer | Analysis of state-independent importance-sampling measures for the two-node tandem queue[END_REF].

For some values of the parameters of the importance sampling, there is no efficient state-independent importance sampling whereas there are efficient state-dependent ones [START_REF] Dupuis | Dynamic importance sampling for queueing networks[END_REF] which can be efficiently computed.

Cross-entropy method

Among optimization methods, the cross-entropy method has recently received a lot of attention [START_REF] Rubinstein | Optimization of computer simulation models with rare events[END_REF][START_REF] Clarke | Statistical model checking for cyber-physical systems[END_REF][START_REF] Jégourel | Cross-entropy optimisation of importance sampling parameters for statistical model checking[END_REF]. In practice it relies on a parametrization of the model, even if it is defined at the level of the Markov chain. Among all the importance samplings defined by changing the value of the parameters, it computes the one that minimizes the distance to the optimal importance sampling using the cross-entropy distance [START_REF] Kullback | On information and sufficiency[END_REF]. Furthermore, this minimization is done iteratively by successive simulations of the model. An introduction to the cross-entropy method can be found in [START_REF] Boer | A tutorial on the cross-entropy method[END_REF] where it is shown that this method can be used to solve also classical optimization problems unrelated to rare events, like the traveling salesman problem. More formally in the context of BDTMC with an initial transition probability matrix P, let call f P the probability density function over paths of the Markov chain: f P (s 0 ,s 1 ,s 2 ...,s n )=P(s 0 ,s 1 )P(s 1 ,s 2 ) ...P(s n-1 ,s n ). Let Ω be the set of all finite paths ending in s + or s -. Recall that X(σ)i sa random variable which value is 1 for path ending in s + and zero otherwise. The probability density function of the zero-variance importance sampling is

f ⇤ (σ)= f P (σ)X(σ) E(X)
. Given two probability transition matrices P 1 and P 2 , the cross-entropy distance is defined as:

CE(f P 1 ,f P 2 )= Z σ2Ω f P 1 (σ) log f P 1 (σ) f P 2 (σ) dσ.
This quantity is not formally a distance but is sufficient for our purposes. The idea of this method is to find a change of measure that minimizes the cross-entropy to the zero-variance importance sampling:

CE(f ⇤ ,f P 0 )= Z σ2Ω f ⇤ (σ) log f ⇤ (σ) f P 0 (σ) dσ = Z σ2Ω (f ⇤ (σ) log f ⇤ (σ) -f ⇤ (σ)log(f P 0 (σ)))dσ = Z σ2Ω f ⇤ (σ) log f ⇤ (σ)dσ - Z σ2Ω X(σ)f P (σ) E(X) log(f P 0 (σ))dσ
By keeping only terms depending of P 0 , minimizing the cross-entropy is equivalent to maximizing the right term: max

P 0 Z σ2Ω X(σ)f P (σ) log(f P 0 (σ))dσ
which can be rewritten as:

max P 0 Z σ2Ω f P 0 (σ)L P 0 (σ)X(σ) log(f P 0 (σ))dσ (3.4)
where L P 0 = f P f P 0 is the likelihood corresponding to the change of measure with transition probability matrix P 0 .

Using Equation (3.4), one can define an iterative algorithm as follows:

1. Choose an initial transition probability matrix P 1 .S e ti = 1.

2. Simulate n i trajectories of the Markov chain using P i as transition probability matrix, yielding a set of trajectories

{σ k } n i k=1 3. Compute P i+1 = max P 0 P n i k=1 L P i (σ k )X(σ k ) log(f P 0 (σ k )) 4. Set i = i + 1.
Go back to step (2) until a stopping criterion is reached.

Suppose the system is parametrized by ε<1 and that E(X) goes to 0 as ε goes to 0. Let P ce be the matrix obtained by the last iteration of the algorithm. Under the additional strong constraint that there exist two finite constants K 1 and K 2 such that

K 1  CE(f ⇤ ,f P ce )  K 2
for any ε, then the cross-entropy method is asymptotically optimal [START_REF] Ridder | Asymptotic optimality of the cross-entropy method for Markov chain problems[END_REF] In order to implement such a method, a parametrization of the transition probability matrix is often necessary to make the optimization step tractable. Remark that in general the result of asymptotical optimality does not hold with a parametrization. As each intermediate step produces an importance sampling, the algorithm can be stopped as soon as the produced importance sampling is efficient enough. The initial transition probability matrix is crucial as the followings steps require that some trajectories reach s + . Here the cross-entropy method is described in the setting of DTMC but it is also used in a more general setting.

Conclusion

The reliable estimation of a rare-event probability is crucial for the analysis of some stochastic systems. However Monte-Carlo simulations are ineffective to deal with this kind of probabilities. Methods that deal with the rare-event problem in Monte Carlo simulation have been intensively studied, mostly in some specific domain like computational physics. Recently, a new interest on statistical model checking stimulates research for methods dealing with the rare-event problem in more general settings.

The existing methods, mainly importance sampling and splitting, cannot produce confidence intervals for the value of interest except by using asymptotical results whose validity is difficult to assert in practice. Moreover all these methods are parametrized and often require the modeler to guess these parameters.

Part II Theoretical Contributions

Chapter 4

Guaranteed Variance Reduction 4.1 Introduction

In a reachability probability problem, one wants to compute the probability p to reach a final state s + , from the initial state s 0 of a given model. When the size of the model becomes too large for numerical analysis and when the probability of interest p is too small for direct statistical simulation, rare event acceleration methods become necessary. A possible approach to tackle this problem is to build an importance sampling scheme for which a statistical simulation becomes possible. In order to be useful, this importance sampling has to be built carefully to ensure a reduction of the variance.

In this chapter, we propose a method to design such efficient importance sampling. This method relies on the construction of an abstraction of the initial model, on which the distribution of the importance sampling is computed by numerical analysis. This abstraction is performed by lumping some states and possibly modifying distributions in order to obtain a smaller model with a similar behavior but more manageable for numerical computations. We call this model the reduced model.

In order to ensure that the variance is reduced within this approach, we define sufficient conditions on the reduced model in Section 4.2, together with methods to check that they are satisfied by a given reduced model. In the three following sections, we use such reduced models to define importance samplings allowing efficient estimations of either time-unbounded or time-bounded reachability probabilities.

A New Approach for Importance Sampling

This section formalizes the idea of abstracting a model in a way that allows one to build an importance sampling with sufficient property to reduce the variance. We present two kinds of hypotheses on the construction ensuring importance samplings satisfying different interesting properties. These hypotheses are the same for models taking as semantics a DTMC or a CTMC and for time-bounded or unbounded reachability problems. In the following we consider only DTMCs. Similar definitions are given in Section 4.5 for CTMCs.

Principle of the Method

In this section, we consider a model M taking as semantics a DTMC with two absorbing states s + and s -reached with probability one. Our goal is to efficiently estimate the probability to reach s + with statistical methods whereas this probability is small. The principle of the method is summarized in Figure 4.1. Roughly speaking, the computation is split in two parts. A crude approximation of the probability of interest is computed with a numerical computation, then using this approximation, a simulation with importance sampling computes a precise value. The numerical computation is performed on an abstraction M • of the model M using a function f that maps states of M to states of M • . As the transition probability matrix P 0 of the optimal importance sampling(see Section 3.4.2) is defined by

P 0 (s, s 0 )= µ(s 0 ) µ(s) P(s, s 0 ),
where µ(s) is the probability to reach the state s + from s,w ed e fi n ei na similar way the matrix by

P 0 (s, s 0 )= µ • (f (s 0 )) µ • (f (s)) P(s, s 0 ),
where the probability vector µ has been replaced by µ The aim is that the likelihood under this importance sampling will be close to µ • (f (s 0 )), whereas in the optimal case the likelihood is equal to µ(s 0 ). Moreover µ • (s • ) is a reachability probability in a smaller DTMC, it can be computed numerically. However, some adaptations are required in P 0 due to the fact that µ • • f 6 = µ, to obtain a transition probability matrix. These adaptations lead to different important samplings which are defined in Sections 4.3, 4.4 and 4.5.

Reduced models must achieve three possibly conflicting goals:

1. The size of the reduced model has to be significantly smaller than the size of the initial one in order to perform a numerical analysis on it.

2. The reachability probabilities of the event of interest with and without reduction (µ • (f (s 0 )) and µ(s 0 )) should be no more different than about three orders of magnitude. Otherwise, the importance sampling obtained from the reduced model will be useless because the event of interest will still be rare.

3. The random variable W s 0 obtained from the importance sampling we produce should have a small variance. In particular, we want to ensure that we do not fall in the case described in Section 3.4.3 where the variance is infinite.

An example of reduction for the example of tandem queues is given in the next section. Chapter 5 and 6.7 present and discuss several examples of reduced models.

Reduced Model Definition 22 (Reduction of Model)

Let C be a BDTMC with two absorbing states s + and s -.ADTMC C • is called a reduction of C by a function f that maps S to S • , the state space of C • , if, denoting s • -= f (s -) and s • + = f (s + ), the following assertions are satisfied:

s + s - C s 0 s • + s • - C • f (s 0 ) f Figure 4.2: Definition of the reduced model • f -1 (s • -)={s -} • Let s • 2 S •
, and denote by µ • (s • ), the probability to reach s • + starting from s • , then for all s 2 S,

µ • (f (s)) = 0 ) µ(s)=0, .
The first assertion entails that only the state s -is mapped to s • -. The second one entails that if a state is mapped to a state that cannot reach s • + ,t h e n it also cannot reach s + . Observe that C • is a BDTMC. These assumptions ensure that trajectories in the model can be mapped to trajectories in the reduced model as transitions between state are preserved. This is necessary for the existence of the objects built in the next section. The efficiency of the method relies on additional properties of this reduced model.

In the following, everything marked with a "•"l i k es • + is related to the reduced model.

Running example.

Let us recall the example of the tandem queues. Figure 4.3 shows on the upper left part a SPN representing the tandem queues. The state of the system is represented by the pair (n 1 ,n 2 ) of the number of tokens in each queue. The property of interest is that a state where the sum of the number of tokens in the two queues exceeds N is reached before the initial state where the system is empty. This property will later be referred as "Global overflow in tandem queues" and can be expressed in LTL as X (

n 1 + n 2 > 0 U n 1 + n 2 ≥ N ).
On the bottom left part of Figure 4.3, the DTMC corresponding to this model where all states where n 1 + n 2 ≥ N have been lumped together into an absorbing state s + and the state where n 1 = n 2 =0have been made absorbing Constraints can be added on the reduced model to bound the random variable W s 0 . In this case confidence intervals build from Chernoff-Hoeffding bounds can be computed. Moreover, if the distribution of W s 0 is bounded and known up to some parameters, tighter confidence intervals can be constructed. In the next section, we define properties of reduced models that will be used to produce an important sampling yielding distribution of the likelihood which are bivaluated.

Q 1 Q 2 λ ρ 1 ρ 2 f Q 1 Q 2 R AQ 2 λ ρ 1 ρ 2 1 ρ 2 ρ 2 ρ 2 ρ 2 ρ 1 ρ 1 ρ 1 ρ 1 1 s - s + s 0 λ λ λ λ λ λ λ λ λ λ λ λ λ λ ρ 1 ρ 1 ρ 1 ρ 1 ρ 1 ρ 1 ρ 1 ρ 1 ρ 1 ρ 1 ρ 2 ρ 2 ρ 2 ρ 2 ρ 2 ρ 2 ρ 2 ρ 2 ρ 2 ρ 2 f 1 1 λ ρ 1 ρ 2 λ ρ 1 ρ 2 λ ρ 1 ρ 2 λ ρ 1 ρ 2 λ ρ 2 ρ 1 λ ρ 1 ρ 2 λ ρ 1 ρ 2 λ ρ 1 ρ 2 ρ 2 ρ 2 ρ 2 ρ 1 ρ 1 ρ 1 λ λ λ s • - s • + s • 0
f (n 1 ,n 2 )= ⇢ (n 1 ,n 2 ) if n 2  R (n 1 + n 2 -R,

Guaranteed Variance Reduction

In Section 3.4, it was shown that an importance sampling should increase the probability of transitions toward the goal. This is reflected by the fact that the function f should ensure that µ(s 0 )  µ • (f (s 0 )) and more generally for an arbitrary state that µ(s)  µ • (f (s)). This can be used either in a weak sense, as in most of the states this condition should hold, or in a strong sense, as in all states this condition should hold.

At the same time, we will see in the next section, that when we use reduced models to build an importance sampling, the probability to reach the state s + from s 0 under the importance sampling is smaller than µ(s 0 )/µ • (f (s 0 )) thus µ • (f (s 0 )) should not be too big compared to µ(s 0 ). These two constraints highlight difficulties to produce suitable reduced models as the reduced model should increase the probability to reach the goal but not too much. We study here the strong case where:

8s, µ(s)  µ • (f (s)). (4.1)
We want to substitute to this condition a sufficient structural one in order to check it by an analysis of the model and not to evaluate it on each state by numerical computation of µ.

Let us look at a particular case. Suppose that the Markov chain C does not have any cycle such that it induces a topological order. The condition holds for the states s + and s -. An induction over the states of S starting from s + and s -following the reverse topological order may be done, the induction hypothesis is:

µ(s)= X s 0 2S P(s, s 0 ) • µ(s 0 )  X s 0 2S P(s, s 0 ) • µ • (f (s 0 )).
In order to conclude that µ(s)  µ • (f (s)), the additional hypothesis 8s 2 S, X

s 0 2S P(s, s 0 ) • µ • (f (s 0 ))  µ • (f (s))
is required Using the additional hypothesis of this particular case, we define reduction with guaranteed variance in the general case as follows:

Definition 23 (Reduction with Guaranteed Variance)

Let C be a BDTMC and C • a reduction of C by f .T h e DTMC C • is a reduction with guaranteed variance by f if, for all s 2 S such that µ • (f (s)) > 0, it holds that:

X s 0 2S P(s, s 0 ) • µ • (f (s 0 ))  µ • (f (s)).
(4.2)

s 0 s 1 " s 2 1 -" s - " 1 -" s + " 1 -" f s • 0 s • 12 1 s • -1-p s • + p Figure
P(s 1 ,s + ) • µ • (f (s + )) + P(s 1 ,s -) • µ • (f (s -)) = 1 -ε  p, thus µ • (f (s 0 )) ≥ 1 -ε.I fp =1-ε,
the probability for the simulation to reach s + under the importance sampling induced by the reduced model is 2ε. For a small value of ε,s a y10 -9 , this probability is still a rare event.

This example illustrates that a reduced model should not lump together states with very different probabilities to reach the state s + . It is straightforward in this example that states s 1 and s 2 have very different behaviors but in more realistic settings, we have to merge a large (or even infinite) number of states together, and some of them may have different behaviors.

Running example. In the tandem queues example, the reduced model is a reduction with guaranteed variance. As the proof requires additional tools, we only provide some intuition here. Aggregated states in the reduced model have the same behaviors when there are less than R clients in the second queue (n 2  R). In states where n 2 ≥ R,t h et r a n s i t i o nρ 1 is replaced by a self-loop which prevents the system from emptying too fast. This difference between the two models slows down clients in the reduced model and it increases their number. Thus µ(s)  µ • (f (s)) in all states.

Structural Guarantee

Unfortunately, Equation (4.2) is difficult to check. The numerical verification that this equation holds is a two-step procedure. The first step is to compute µ • for every state of C • . This step is required later for computing the importance sampling and is not problematic if the size of C • is sufficiently small which is already supposed. The second step is to check that for any state in C, the equation holds. This supposes to enumerate all states of C. As the size of C can be huge, this is in general unfeasible. Moreover, most models have scaling parameters (like N or R for the tandem example) and we want to prove that the property holds for any value of these parameters.

For these reasons, we define a stronger structural requirement for the reduction which will ensure that Equation (4.2) is satisfied by construction. This is expressed in the next proposition using enriched BDTMC instead of BDTMC for the sake of clarity.

Proposition 1

Let C be an enriched BDTMC, C • be a reduction of C by f . Assume there exists a function g : S ⇥ E ! E • such that for all s 2 S, e 7 ! g(s, e)i s an injection which maps each transition of the model to a transition in the reduced model, such that if: 8s 2 S, 8e 2 E :

1. p(s, e)=p • f (s),g(s, e) 2. p(s, e) > 0 ) µ • f (δ(s, e))  µ • δ • (f (s),g(s, e))
then C • is a reduction of C by f with guaranteed variance. Proof: For all s 2 S, starting from the left hand side of Equation (4.2):

X s 0 2S P(s, s 0 ) • µ • (f (s 0 )) Using the definition of P in enriched DTMC = X e2E p(s, e) • µ • (f (δ(s, e))) Using hypothesis 1 = X e2E p • (f (s),g(s, e)) • µ • (f (δ(s, e))) Using hypothesis 2  X e2E p • (f (s),g(s, e)) • µ • (δ • (f (s),g(s, e)))
Using that g is an injection

 X e • 2E • p • (f (s),e • ) • µ • (δ • (f (s),e • )))
Using the definition of µ

• = µ • (f (s)) ⇤ M M • s t 1 e1 t 2 e 2 t 3 e 3 f (s) t • 1 e1 t • 2 e 2 e 3 ≤ f µ • ≤ f µ •
f µ • define by s  f µ • t if µ • (f (s))  µ • (t)
. This condition can also be seen as follows:

8s 2 S, 8e 2 E : s f (s) f δ(s, e) δ e δ • (f (s),g(s, e)) δ • g(s,e) f (δ(s, e)) f  µ •  f µ •
where δ e stands for the function s 7 ! δ(s, e) and s

•  µ • t • if µ • (s • )  µ • (t • ).
For any state s,i fs is mapped into C • by f and the transition labeled by g(s, e) is taken (bottom left path of the diagram), it is more probable to reach s • + from the obtained state δ • f (s),g(s, e) than from the state f δ(s, e) which would be reached if the event e is taken first, and its result is mapped to C • by f (upper right path of the diagram). Thus the reduced model is going faster to s • + than the initial one is going to s + . Most of the time, Condition 2 is proven using a coupling (defined in Section 2.2.2). We therefore restate Proposition 1 in this context: Running example. For the tandem example, the function g is chosen to be equal to the identity. Thus Condition 1 of Proposition 2 holds. For Condition 2, the coupling

S ⌦ = {((n 1 ,n 2 ), (n 0 1 ,n 0 2 )) | n 1 +n 2  n 0 1 +n 0 2 ^n1  n 0
1 } defined at page 29 is used. For any state (n 1 ,n 2 ) 2 C • such that n 2 <R, the function f behaves like the identity and δ e and δ • e are identical. Thus the diagram commutes and the property holds.

We now focus on the a state s =(n 1 ,n 2 ) such that n 2 ≥ R:

• For event λ: δ λ and δ • λ behave the same, in the sense that they both increase the first component n 1 by one, thus f (δ(s, λ)) = δ • (f (s),λ) and the property holds.

• For event ⇢ 1 : δ ⇢ 1 and δ • ⇢ 1 perform self-loops and thus the property holds.

• For event ⇢ 2 : The case where n 2 = R is similar to the previous cases.

In the case where n 2 >R, the behaviors of δ ⇢ 2 and δ • ⇢ 2 are different:

.

f (δ ⇢ 2 (n 1 ,n 2 )) = (n 1 + n 2 -R -1,R) . δ • ⇢ 2 (f (n 1 ,n 2 )) = (n 1 + n 2 -R, R -1)
.

As (n 1 + n 2 -R -1,R), (n 1 + n 2 -R, R -1) 2 S ⌦ the property holds.
Using Proposition 1, the reduction proposed for the tandem is thus a reduction with guaranteed variance reduction.

We now weaken Condition 1 in order to enlarge the field of application of the method.

Proposition 3

Let C be an enriched BDTMC, C • be a reduction of C by f . Assume there exists a function g : S ⇥ E ! E • such that 8s, e 7 ! g(s, e) is an injection which maps each transition of the model to a transition in the reduced model, such that:

1. 8s 2 S, 8e 2 E, p • f (s),g(s, e) >p(s, e) ) µ • f (s)  µ • ⇣ δ • f (s),g(s, e) ⌘ (1.a) p • f (s),g(s, e) <p(s, e) ) µ • f (s) ≥ µ • ⇣ δ • f (s),g(s, e) ⌘ (1.b) 2. 8s 2 S, 8e 2 E s.t. p(s, e) > 0, µ • ⇣ f δ(s, e) ⌘  µ • ⇣ δ • f (s),g(s, e)
⌘ then C • is a reduction of C by f with guaranteed variance.

Proof:

Let us give the idea underlying this proof: when a transition has different probabilities in the reduced and the initial models, it can be split into two parts by introducing loops. The first part of the transition is the same in both models and corresponds to the smallest probability. The second part is the probability difference between the original and reduced models. However, in the model with the smallest original transition probability, it is introduced as a self-loop, while in the other one, its target state is left unchanged. The chain D is indeed a DTMC. Similarly, we define

D • =(S • ,E ] Ẽ, δ• , p• )
with for all s 2 S:

8e 2 E, δ• f (s),g(s, e) = δ • f (s),g(s, e)
and p• f (s),g(s,e) = 1 h(s) min

⇣ p(s,e),p • f (s),g(s,e) ⌘ , 8e 2 Ẽ, if p • f (s),g(f (s),e) >p(s,e)then δ• f (s),g(s,e) = δ • f (s),g(s,e) , if p • f (s),g(f (s),e)  p(s, e)t h e n δ• f (s),g(s, e) = f (s), p• f (s),g(s, e) = 1 h(s) p(s, e) -p • f (s),g(s, e) .
The above figure illustrates how transitions are split to create transitions with equal probabilities. An edge from left to right (respectively right to left) sends a state on another state with higher(respectively lower) probability to reach s + .

As µ (respectively µ • ) expresses unconstrained time reachability probabilities, its value is identical on C and D (respectively on C • and D • ). The two models differ only by loops and transitions that can be split or not.

We can now prove that D • is a reduction with guaranteed variance of D by checking that it fulfills the hypotheses of Proposition 3. By construction the first condition holds. For all s 2 S and e 2 E ] Ẽ there are two cases:

1. If e 2 E as δ(s, e)=δ(s, e), the condition holds by hypothesis.

µ • (f ( δ(s, e))) = µ • (f (s))  µ • (δ • (f (s),g(s, e))) = µ • ( δ• (f (s),g(s, e))).
(c) p(s, e)= 1 h(s) p(s, e)p • (f (s),g(s, e)) : this case is depicted on the right part of the figure, and we have δ(s, e)=δ(s, e) and δ• (f (s),g(s, e)) = f (s). Using hypothesis (1.b) and (2), we have:

µ • ( δ• (f (s),g(s, e)) = µ • (f (s)) ≥ µ • (δ • (f (s),g(s, e))) ≥ µ • (f (δ(s, e))) = µ • (f ( δ(s, e))). ⇤ Remark 7
In this proposition, the requirement that transitions in relation by g should have equal probabilities in the two models is replaced by hypotheses (1.a) and (1.b): their probability can be greater in the reduced model if the transition leads to a state with higher probability to reach the state s + and can be smaller if the transition leads to a state with lower probability to reach the state s + . These hypotheses can be checked with the coupling relation required for hypothesis (2).

Reachability Analysis for Markov Chain

In this section, an importance sampling for unbounded reachability problems using reduced models is defined. Given a stochastic model M taking as semantics a DTMC or a CTMC which can be infinite, we are interested in computing the probability to reach a state s + before reaching the state s -, supposing that these two states are reached with probability one, so that the problem input is a BDTMC.

General Importance Sampling

In the following, efficient important samplings based on reduced models (Definition 22) are constructed. Due to the fact that

µ 6 = µ • • f , for s 2 S such that µ • (f (s)) > 0thesumh(s)= P s 0 2S µ • (f (s 0 ))
µ • (f (s)) P(s, s 0 ) of the outgoing probabilities is not necessarily equal to 1. When h(s) > 1, the only reasonable choice for the construction of the transition probability matrix P 0 is to build an intermediate transition matrix summing to h(s) for state s and then divide each probability by h(s). When h(s) < 1, there are two possibilities. Either one normalizes by h(s), or the remaining probability 1h(s) leads to the sink state s -. These two importance samplings are described more precisely in the following definition.

Definition 24 (General Importance Sampling with Normalization)

Let C be a BDTMC and C • a reduction of C by f . For s 2 S,l e t h(s)= P

s 0 2S µ • (f (s 0 ))
µ • (f (s)) P(s, s 0 ). We define a transition probability matrix P 0 =(P 0 (s, s 0 )) (s,s 0 )2S⇥S by the following rules, for all s 0 2 S:

• if µ • (f (s)) > 0t h e nP 0 (s, s 0 )= µ • (f (s 0 )) h(s)µ • (f (s)) P(s, s 0 ) • if µ • (f (s)) = 0 then P 0 (s, s 0 )=P(s, s 0 ).
Remark 8 This importance sampling makes the state s -unreachable: the probability to go to a state s 0 is proportional to µ • (f (s 0 )), the probability to reach a state where µ • (f (s 0 )) = 0 is also null. The definition of the reduced model (Definition 22) implies that if µ • (f (s 0 )) = 0 then µ(s 0 )=0thus s + is no longer reachable from s 0 .

Remark 9

The second case of this definition is required for the completeness of the definition but is irrelevant as soon as state s + is reachable from the initial state. States where µ • (f (s)) = 0 are not reachable and cannot reach state the s + thus trajectories visiting such a state always end in s -.

Definition 25 (General Importance Sampling with Sink State)

Let C be a BDTMC and C • a reduction of C by f . For s 2 S,l e t h(s)= P

s 0 2S µ • (f (s 0 ))
µ • (f (s)) P(s, s 0 ). We define a transition probability matrix P 0 =(P 0 (s, s 0 )) (s,s 0 )2S⇥S by the following rules:

• if µ • (f (s)) > 0 and h(s)  1 then for all s 0 2 S \{s -}, P 0 (s, s 0 )= µ • (f (s 0 )) µ • (f (s))
P(s, s 0 ) and P 0 (s, s -)=1-h(s)

• if µ • (f (s)) > 0 and h(s) > 1 then for all s 0 2 S,

P 0 (s, s 0 )= µ • (f (s 0 )) h(s)µ • (f (s)) P(s, s 0 )
• if µ • (f (s)) = 0 then for all s 0 2 S, P 0 (s, s 0 )=P(s, s 0 )

Remark 10 Again the last case is irrelevant and only required for completeness.

By construction, in the two definitions P 0 is a transition probability matrix. It complies to Conditions (3.2).T h u sP 0 defines an importance sampling.

We are now in position to describe the whole method for a model M and a reachability goal s + .T h i si sat h r e e -s t e pp r o c e d u r e :

1. Specify a model M • with associated DTMC C • , and a function f such that C • is a reduction of C by f .

Compute function µ

• with a numerical model checker applied on the DTMC C • .

3. Estimate µ(s 0 ) with a statistical model checker applied on C using one of the importance samplings of Definition 24 and 25.

Steps 2 and 3 are automatic and can be performed by tools.

Step 1 requires to manually design the reduced model. Figure 4.1 shows a schematic description of this method. The choice between the two importance sampling is discussed in Chapter 6.7.

With the importance sampling using normalization (Definition 24), the random variable W s 0 (Definition 20) can take in general any positive real value.

In the next proposition we study the set of values taken by the distribution of the random variable W s 0 induced by the importance sampling with sink state (Definition 25).

Proposition 4

Let C be a BDTMC and C • its reduction. The importance sampling of Definition 25 has the following property: for all s such that µ(s) > 0, W s is a random variable which takes its values in {0}[[µ • (f (s)), 1[.

Proof:

Let σ =(s = u 0 ,u 1 ,...,u l = s + ) be a trajectory starting in s ending in s + . As the trajectory avoids s -, its value is

µ • f (u 0 ) max(h(u 0 ), 1) µ • f (u 1 ) ... µ • f (u l-1 ) max(h(u l-1 ), 1) µ • f (u l ) = µ • f (s) 0 B B @ Y 0i<l, h(u i )>1 h(u i ) 1 C C A ≥ µ • f (s) ⇤ Corollary 5
Let C be a BDTMC and C • its reduction.

P(W s 0 > 0)  µ(s 0 ) µ • (f (s 0 )) Proof:
The probability P(W s > 0) is the probability for a trajectory to ends up in state s + starting from s 0 , under this importance sampling. The expected value of W s 0 can be written:

E(W s 0 )=P(W s 0 > 0) • E(W s 0 |W s 0 > 0). As E(W s 0 )=E(V s 0 )=µ(s 0 ) and E(W s 0 |W s 0 > 0) ≥ µ • (f (s 0 )). ⇤
Even if this proposition gives some clue on the values taken by W s 0 , the shape of its distribution is unknown and its variance may be infinite. Therefore, the shape of this distribution will be experimentally studied in Chapter 6.7.

Importance Sampling with Guaranteed Variance

In Section 4.3.1, a general importance sampling was defined. However, in this case, the shape of the distribution of the random variable W s 0 was unknown and its variance possibly infinite.

In this section, the reduced models with guaranteed variance defined in Section 4.2 are used to construct important sampling for which the shape of the distribution has bounded support and the reduction of the variance of W s 0 compare to V 0 is guaranteed.

Actually, in the general case, the normalization with h(s) > 1 increases the likelihood and may induce a large variance of W s 0 . For a reduction model with guaranteed variance, for any s 2 S, h(s)  1, therefore the definition of an important sampling with normalization remains unchanged (cf Definition 24) and Definition 25 of importance sampling with sink state can be rewritten as:

Definition 26 (Importance Sampling with Guaranteed Variance)

Let C be a BDTMC and C • be a reduction of C by f with guaranteed variance. Let P 0 be a transition matrix on S defined by: Let s be a state of S,

• if µ • f (s) = 0 then for all s 0 2 S, P 0 (s, s 0 )=P(s, s 0 ) • if µ • f (s) > 0 then for all s 0 2 S \{s -}, P 0 (s, s 0 )= µ • f (s 0 ) µ • f (s)
P(s, s 0 ) and P 0 (s, s -)=1-X s 0 2S P 0 (s, s 0 ).

The reduction of the variance for such importance samplings is proven in the following proposition using the conditions defined earlier for reduced models with guaranteed variance, first for the normalization case, and then for the sink state case.

Proposition 6

Let C be a BDTMC and C • be a reduction with guaranteed variance by f . The importance sampling based on the matrix P 0 of Definition 24 has the following properties:

• For all s 2 S such that µ(s) > 0, W s is a random variable which has value in [0,µ • f (s) ]. • µ(s)  µ • f (s) and V(W s )  µ(s)µ • f (s) -µ 2 (s).

⇤

This proposition shows that Inequation (4.2) is a sufficient condition for obtaining an importance sampling with guaranteed variance reduction.

Remark 11 Since µ(s 0 ) ⌧ 1, V(V s 0 ) ⇡ µ(s 0 ).I fµ(s 0 ) ⌧ µ • (f (s 0 )),w e obtain V(W s 0 ) ⇡ µ(s 0 )µ • (f (s 0 ))
, so the variance is reduced by a factor µ • (f (s 0 )). In the case where µ(s 0 ) and µ • (f (s 0 )) have same order of magnitude, the reduction of variance is even bigger.

Running example. We have already seen that the tandem example is a reduction with guaranteed variance. The importance sampling produced by the reduced model behaves as follows: for all states s =( n 1 ,n 2 ) such that n 2 <Rwe can compute

P s 0 2S µ • (f (s 0 )) µ • (f (s)) P(s, s 0 ) = P e2E µ • (f (δ(s,e))) µ • (f (s)) p(s, e) Since n 2  R, f (δ(s, e)) = δ • (f (s),e)) = P e2E µ • (δ • (f (s),e))p • (f (s),e) µ • (f (s))

=1

This shows that when the reduced model behaves like the original one (i.e. f (δ(s, e)) = δ • (f (s),e))) the probability to take the transition to s -is equal to 0.

We are now in position to describe the whole method for guaranteed variance importance sampling for a model M and an unbounded reachability problem. It is a four-step procedure:

1. Specify a model M • with associated DTMC C • and the functions f and g.

2.

Prove using a coupling on M • that Proposition 3 holds.

Compute function µ

• with a numerical model checker applied on C • .

4. Estimate µ(s 0 ) with a statistical model checker applied on C using importance samplings of Definition 24 or Definition 26.

As in the general case, the last two steps are done by tools. The first step is the specification of M • which requires to manually study M. The second step is in also done by hand (see the proofs of correctness in the examples). In this section the previous method is generalized, to the setting of bounded reachability probabilities in discrete time Markov chains.

The main motivation for developing a dedicated method to analyze bounded reachability probabilities is that an event can be frequent or even recurrent in a system, even if the time of its first occurrence is very large. Figure 4.6 shows an example of a DTMC where the probability to reach a state labeled by b without time constraint is 1. However, if the transition probability p is sufficiently small and if a time bound is set for the maximal number of steps, this problem might become a rare event one and an importance-sampling method is required. The methods presented in Section 4.3 where designed to estimate time-unbounded probabilities, even if the simulator is adapted to reject trajectories exceeding the time bound, the method is not efficient. In fact, in the reduced models built by these methods, the probability to reach the goal state should be greater than in the initial one. However, in this case, if the time bound is not taken into account this probability is already equal to 1 for any state which makes the importance sampling built by the method useless.

In the unbounded setting, the computation of a reachability property probability over a continuous time Markov chain is done by looking at the embedded DTMC. The formalism is the same for the two settings. In the time-bounded reachability setting, a distinction has to be made between the discrete and continuous cases: the two formalisms are different and will be handled separately. This section only focuses on DTMC.

Adapting Reachability Analysis

Given a model M taking as semantics a DTMC, C we are interested in computing the probability to reach a state s + in u steps starting from the initial state s 0 .

In order to adapt the method from the time-unbounded reachability case, the cross product between the Markov chain C and a counter is build. The counter counts the remaining number of transitions in the Markov chain before reaching the horizon. Trajectories that do not reach the state s + after u steps end up in state s -. Given a DTMC C, a reachability goal s + and a time bound u,w ed e fi n e the Markov chain C u = S u , (s 0 ,u), P u by:

s + s - ••• ••• S |s+ ⇥{ u} ρ2 λ S |s+ ⇥{ u-1} λ ρ 1 ρ 2 S |s+ ⇥{i} ρ2 λ (n 1 ,n 2 ) (0, 1) (n 1 ,n 2 ) S |s+ ⇥{i-1} λ ρ 1 ρ 2 (n 1 +1,n 2 ) (n 1 -1,n 2 + 1) (n 1 ,n 2 -1) S |s+ ⇥{2} ρ2 λ S |s+ ⇥{1} λ ρ 1 ρ 2
• S u = S\{s + }⇥[1,u] [{s -,s + } • P u (s -,s -)=P u (s + ,s + )=1(s -,s + are absorbing states)
• 8s, s 0 2 S\{s + }, 81 <v u P u (s, v), (s 0 ,v-1) = P(s, s 0 ) P u (s, v),s + = P(s, s + )

• 8s 2 S\{s + }, P u ((s, 1),s + )=P(s, s + ) P u ((s, 1),s -)=1-P u ((s, 1),s + )

• Other transition probabilities are null. From a theoretical point of view, the importance sampling method developed in the previous section can now be used, on a Markov chain C u ,b u t Definition 29 (Guaranteed Variance for Bounded Property)

Let C be a DTMC and C • a reduction of C by f . C • is a bounded reduction with guaranteed variance if for all v>1, for all s 2 S such that µ • v (f (s)) > 0 we have :

X s 0 2S P(s, s 0 ) • µ • v-1 (f (s 0 ))  µ • v (f (s)) (4.4)
Similarly to the time-unbounded case (Proposition 1) structural requirements can be define to guarantee the reduction of variance. The first, more restrictice, case can be rewritten as:

Proposition 8
Let C be an enriched DTMC, C • be a reduction of C by f . Let u be a positive integer. Assume there exists a function g : S ⇥ E ! E • such that 8s 2 S, e 7 ! g(s, e) is an injection which maps each transition of the model to a transition in the reduced model, such that: 81 <v u, 8s 2 S, 8e 2 E :

1. p(s, e)=p • (f (s),g(s, e)) 2. p(s, e) > 0 ) µ • v (f (δ(s, e)))  µ • v (δ • (f (s),g(s, e)))
then C • u is a reduction of C u by f with guaranteed variance.

The second, more general, case (Proposition 3) cannot be adapted to time-bounded setting. In fact, the proof requires to add self-loops to the Markov chain which modify its transient behaviors.

However, when a coupling on the states of M • is available, another proof of the guaranteed variance reduction can be established. This proof is based on the same hypotheses as the variance reduction in the unbounded case.

Corollary 9

Let C be an enriched DTMC, C • be a reduction of C by f . Let u be an integer. Assume there exists a function g : S ⇥ E ! E • such that 8s 2 S, e 7 ! g(s, e) is an injection which maps each transition of the model to a transition in the reduced model.  C is a relation over the states of C • such that: 8s 2 S, 8e 2 E :

1. p(s, e)=p • (f (s),g(s, e)) In this setting, we only need to build one reduced model with guaranteed reduction for both the time-unbounded and time-bounded reachability settings.

2. p(s, e) > 0 ) f (δ(s, e))  C δ • (f (s),

Algorithmic Considerations

As C • u is bigger than C • , computing µ • over C • u may be too expensive. More precisely, let n be the size of C • . The size of C • u is un. Using Gaussian elimination to compute µ • requires Θ (un) 3 operations. Using iterative or more efficient matrix inversion methods leads to algorithms taking more than O (un) 2 operations. Instead we compute µ • u over C • , which is the probability to reach s • + in at most u steps. We use Equation 8 which requires u matrix vector multiplications. The computation only requires Θ un 2 operations. If the transition probability matrix of C • is sparse (which is usually the case) we can decrease furthermore this complexity. We denote by d the maximum outdegree of the states of C • . A simulation takes at most u steps going through states (s u ,u),...,(s 1 , 1),s ± where s u = s 0 and s ± 2{ s + ,s -}. In state (s v ,v), the computation of the distribution P 0 u ((s v ,v), -) (cf. Definition 26) requires the computation of the values of µ • v (f (s)) and µ • v-1 (f (s 0 )), for each possible target state s 0 from s v , where there are at most d target states. Therefor, the vectors {µ • v } 0<vu may be computed iteratively one from the other with complexity Θ(ndu). Let us remark that in typical modelings, d is very small compared to n. For example d = 3 in the running example. In most of the examples shown in this thesis, in which the state space size depend on a parameter, the number d is independent of the parameter.

More precisely, we derive from P • the transition probability matrix of C, the matrix P • 0 , a square (sub stochastic) matrix, indexed by S\{s -} and defined by 8s, s 0 2 S\{s -}:

• P • 0 (s, s 0 )=P • (s, s 0 ), • P • 0 (s + ,s + )=1, P • 0 (s + ,s 0 ) = 0. Then µ • v = P • 0 • µ • v-1 and µ • 0 is null except µ • 0 (s + ) = 1.
For large values of u, the space complexity to store the vectors µ • v becomes intractable and the

• For i 0 <i , the vectors w i,v-1 are stored along 2 i 0 -1 -1 iterated matrix-vector products starting from vector w i 0 ,v-1 :

w(j, v -1) = P • 0 2 j w(j +1,v -1)
.

The computation associated with v requires 1+2+•••+2 i 0 -1 ,i.e. Θ(nd2 i 0 ) matrix-vector products. Observing that the bit i is reset at most u2 -i times, the complexity of the whole computation is

k X i=1 2 k-i Θ(nd2 i )=Θ(ndu log(u)).
Algorithm 3:

Precomputation(u, µ • 0 ,P • 0 ) Data: w,l,i 0 Result: L /* L fulfills L(i)=µ • P l j=i a u,j 2 j */ 1 l blog 2 (u)c + 1; w µ • 0 ; L(l + 1) w 2 for i from l downto 0 do 3 if a u,i =1then 4 for j from 1 to 2 i do 5 w P • 0 w 6 L(i) w 7 Stepcomputation(v, P • 0 ,L) /* L is updated accordingly to v -1 */ 8 i 0 min(i | a v,i = 1); w L(i 0 + 1); L(i 0 ) w 9 for i from i 0 -1 downto 0 do 10 for j =1to 2 i do 11 w P • 0 w 12 L(i) w Fourth Algorithm
The last algorithm is also a naive approach. It does not store any vector and performs the whole computation at each step as shown in the bottom right corner of Figure 4.8. This algorithm is not effective as the time complexity is extremely high. Otherwise stated, it provides a lower bound on the memory requirement i.e. 2n, the space needed for a matrix-vector multiplication without taking into account the space required to store the matrix. The space memory of 2n is also the space needed by a numerical model checker computing µ u (s 0 ).

Algorithm 4:

Stepcomputation

(v, µ • 0 ,P • 0 ) Result: w, w' /* Vector w equals to µ • v */ w µ • 0 for i =1to u do w' P • 0 w w w
The four algorithms are numbered according to their decreasing space complexity. The corresponding space-time trade-off is summarized in Table 4.1, where the space requirement only takes into accounts the manipulated vectors and is expressed as a number of reals. Other memory requirements are negligible as n and u grow.

More on simulation.

Simulating sequentially trajectories is really inefficient since the additional computations during the simulation stage are repeated during every simulation. Thus, for Algorithm 2, 3 and 4 we proceed with a bunch of trajectories simultaneously simulated step by step. This requires additional memory as each simulation needs space for the state of the system and the event queue. The impact of the batch size on the simulation time and space requirement is investigate in more detail in Chapter 6.7.

To summarize the overall method for evaluating time-bounded reachability in the discrete setting: considering a discrete model M, having as semantics

Confidence interval generation

Figure 4.9: Schematic summary of the method for DTMC a DTMC C, the probability to reach a target state s + in u steps can be estimated as follows:

1. Exhibit a suitable reduced model M • having as semantics a DTMC C • , such that C • is a reduction of C by f .

(Optional)

Prove that this reduction guarantees the variance using a coupling and Proposition 9.

3. Choose one of the algorithms implementing the time-memory trade-off. Alternatively, start with Algorithm 1 and if it fails due to memory requirement, use a less memory consuming algorithm.

4. Perform the pre-computation of the chosen algorithm.

Use the distributions µ •

v to perform importance sampling on a batch of simulations of model M. At each step for each trajectory, take a transition, then update the vector µ • v according to the chosen algorithm.

Steps 4 and 5 are automatic.

Step 3 could also be automatized by trying each algorithm starting from the first one. As for the unbounded case, the first and second steps require human intervention. Figure 4.9 shows a scheme of this method.

Time-Bounded Reachability for CTMC

In this section, our method is extended to compute time-bounded reachability probabilities in CTMC. This is done by reducing the problem on the CTMC to a time-bounded reachability problem over its embedded DTMC (c.f. Definition 13).

The problem studied in this section takes as input a CTMC C and a positive real τ to compute the probability to reach a state s + in τ time units starting from state s 0 .

Transient Analysis

Recall that for any CTMC with a bound on the outgoing rates of all states, a uniform CTMC (Definition page 33) can be built such that bounded reachability probabilities are equals in the two chains.

In a uniform CTMC with outgoing rate λ, the probability to reach state s + in τ time units starting from state s is given by µ ⌧ (s) (see Definition 14) which can be expressed as a weighted sum of the probabilities to reach state s + starting from s in the embedded DTMC with the following formula:

µ ⌧ (s)= X u2N e -λ⌧ (λτ ) u u! µ u (s).
This sum is weighted by Poisson probabilities e -λτ (λ⌧ ) u u!

, which are plotted on Figure 4.10 along with the probabilities µ u (s 0 ) for the tandem example. One can see that e -λτ (λ⌧ ) u u! tends quickly to 0 when u tends to 0 or to the infinity.

As this sum is infinite, bound are necessary, in [START_REF] Fox | Computing poisson probabilities[END_REF], given some rate λτ and accuracy parameters α, β, a numerical method computes indexes u -and u + and coefficients c u such that for u - u  u + :

c u (1 -α -β)  e -λ⌧ (λτ ) u u!  c u , X u<u - e -λ⌧ (λτ ) u u!  α, X u>u + e -λ⌧ (λτ ) u u!  β.

Proposition 10

Let C be an enriched CTMC, C • be a reduction of C by f . Assume there exists a function g : S ⇥ E ! E • such that 8s 2 S, e 7 ! g(s, e) is an injection which maps each transition of the model to a transition in the reduced model, such that:

1. 8s 2 S, 8e 2 E, 8u 2 N ⇤ λ • f (s),g(s, e) >λ(s, e) ) µ • u f (s)  µ • u ⇣ δ • f (s),g(s, e) ⌘ (1.a) λ • f (s),g(s, e) <λ(s, e) ) µ • u f (s) ≥ µ • u ⇣ δ • f (s),g(s, e) ⌘ (1.b) 2. 8s 2 S, 8e 2 E s.t. λ(s, e) > 0,µ • u ⇣ f δ(s, e) ⌘  µ • u ⇣ δ • f (s),g(s, e)
⌘ then the embedded DTMC of C • is a bounded reduction of the embedded DTMC of C by f with guaranteed variance for bounded property.

Proof:

We mimic the proof of Proposition 3. The same splitting of transitions can be performed to build two CTMC where all corresponding transitions have the same rate. As this transformation is done on a CTMC, the additional loops do not change the transient behaviors of the model. This was not the case for DTMC where the transient behaviors where affected by loops. In the previous section, we describe a theoretical method to produce a confidence interval framing of µ ⌧ (s 0 ). Even if our system is a CTMC,t h e distribution required for the importance sampling method comes from the embedded DTMC analysis. Estimating µ ⌧ (s 0 ) requires to apply (u +u -) times the algorithm for a bounded DTMC.W h e n⌧λ is big this becomes intractable.

u + u - 0 v ... µ u + (s 0 ) k ... µ u + -1 (s 0 ) ... µ u + -2 (s 0 ) ... ... µ u -(s 0 ) 1 1

Simultaneous Simulation

Trajectories for all u between u -and u + are simultaneously simulated step by step in order to use the same vector of probability µ • v at each step. This allows µ • v to be computed only once and used for all trajectories devoted to the computation of µ u (s 0 )w h e r eu ≥ v. The simulation starts with the generation of k trajectories devoted to the computation of µ u + (s 0 ). After one step of simulation of these k trajectories, we generate k new trajectories devoted to the computation of µ u + -1 (s 0 ). We resume the simulation with these 2k trajectories and so on until the generation of the trajectories devoted to the computation of µ u -(s 0 ). Figure 4.14 illustrates this point.

Reuse of Simulation

Further optimization improvement is possible in order to reduce the width of confidence interval of each µ u (s 0 ).

One can use the same simulation to estimate several µ u (s 0 ). Given a trajectory that reaches state s + or s -in v steps, all estimators of µ v 0 (s 0 ) with 0 <v 0  u can take into account this trajectory. More precisely each estimator with v 0 <vadds a realization with likelihood equal to 0 and each estimator with v 0 ≥ v adds a realization with likelihood equal to W s 0 .

In an ideal setting, one only have to perform the simulation for estimating µ u + (f (s 0 )) to estimate all (µ u (f (s 0 )) u + u -. Experimentally this idealistic case can happen (see Section 7 for an example), but in general there is a difficulty in the fact that values of µ v (s 0 ) for small v compared to u + can correspond

From Model Checking to Reachability

Previous sections focus on time-bounded and time-unbounded reachability problems. In this section, we show how more general model checking problems can be transformed into such reachability problems. To achieve this goal we extend the definition of a Markov chain with a set of atomic propositions AP and a labeling function L : S ! 2 AP .

From Until Formula to Reachability

Consider a model checking problem on model M given as a DTMC or a CTMC against a formula φ = a U u b. Where the constant u can either denote 1 or an integer if M is a DTMC, or a real if M is a CTMC. Here we are interested in transforming this model checking problem to a reachability problem.

The Markov chain M is transformed by lumping together all states where b holds (S b ) in a new state that we called s + . All states where ¬a ^¬b holds (S ab ) are lumped together in state s -. States s + and s -are made absorbing. All remaining states (S ab ) are left unchanged. If a state is no longer reachable from the initial state due to this transformation, it is removed from the model. In the case u = 1, the formula φ can be rewritten as φ = a U b.T h e corresponding DTMC C φ is then given by: Definition 30 (From Until Formula to Reachability) Given a DTMC or a CTMC C and a formula φ = a U b, the Markov chain C φ is defined by:

• S φ = S ab [{s -,s + } • s -,s + are absorbing states: P φ (s -,s -)=P φ (s + ,s + )=1
• 8s 2 S ab ,s 0 2 S φ , P φ (s, s 0 )= 8 < : P(s, s 0 )i f s 0 2 S ab P s 00 2S ab P(s, s 00 )i f s 0 = s - P s 00 2S b P(s, s 00 )i f s 0 = s + This definition applies both to DTMC and CTMC: as states lumped together are absorbing ones, their exit rate is irrelevant. When u = 1, in order to use the method described in section 4.3 an additional hypothesis is required: M| = P =1 [Fb]t h e nC φ is a BDTMC and the method applies.

When u is an integer or a real (C φ ,u) is an instance of time bounded reachability problems that can be solved using methods of Sections 4.4 and 4.5.

From Model Checking Against Finite State Automaton to Reachability

We adapt our method to model checking problems expressed by an automaton.

Let M be a model taking as semantics a discrete or continuous time Markov chain. Let AP be the set of atomic propositions labeling the states of M and E the set of events labeling the transitions of M. Let A be a finite-state automaton over the alphabet AP ⇥ E. Let u be a time bound taking values in N [ {1},i fM has discrete time semantics, and values in R + [ {1} otherwise.

The following definition is a classical definition of automaton.

Definition 31 (Finite State Automaton)

A finite state automaton A is a tuple (Q, Q 0 , Σ,T,F), where

• Q is a finite set of locations, • Q 0 ⇢ Q is the non empty set of initial locations, • Σ is a finite alphabet, • T ⇢ Q ⇥ Σ ⇥ Q is a set of transitions,
• F ⇢ Q is the set of final locations, and 8q 2 F, a 2 Σ, 9q 0 2 Fs . t .(q, a, q 0 ) 2 T, i.e., the set of final locations is absorbing and complete.

We define the synchronized product of the Markov chain underlying model M and the automaton A as follows: as the automaton is not deterministic this synchronization build the powerset construction of the automaton, Definition 32 (Synchronized Product with an Automaton)

Given an enriched DTMC or a CTMC C =(S, s 0 ,E,δ,p), a labeling function L : S ! 2 AP and a finite state automaton A =(Q, Q 0 ,E ⇥ 2 AP ,T,F), the enriched synchronized Markov chain C A =(S ⌦ ,s ⌦ 0 ,E ⌦ ,δ ⌦ ,p ⌦ )isdefined by:

• S ⌦ =(S ⇥ 2 Q ) [{s + ,s -} • s ⌦ 0 =(s 0 ,Q 0 ) • E ⌦ = E • δ ⌦ (s -, -)=s -,δ ⌦ (s + , -)=s + δ ⌦ ((s, Q 1 ),e)= 8 < : s + if (Q 1 ⇥{(e, L(s))}⇥Q f ) \ T 6 = ; s -if (Q 1 ⇥{(e, L(s))}⇥Q) \ T = ; (δ(s, e), (Q 1 ⇥{(e, L(s))}⇥Q)) otherwise • p ⌦ ((s, Q 1 ),e)=p(s, e)
The probability to reach s + in the synchronized product C A is equal to the probability that a trajectory of C is accepted by the automaton.

When u = 1,i fC A is BDTMC, then the method described in Section 4.3 applies. When u is finite, the corresponding time bonded method applies as well.

The model checking of Markov chain against such an automaton can be related to the model checking of asCSL as defined in [START_REF] Baier | Model checking Markov chains with actions and state labels[END_REF]. In asCSL, one computes the probability of a path to be accepted by a finite automaton given as a regular expression, in a given time interval. Compared to asCSL, the logic we present (1) restricts the interval of the path operator to intervals containing 0 and (2) disallows the nesting of probabilistic operator. These restrictions are due to the difficulty to deal with steady state probabilities, nested operators and infinite paths in statistical model checking.

Conclusion

In this chapter, we present a theoretical framework allowing to estimate probabilities of rare events using importance samplings. Using numerical computations on a reduced model, important samplings are produced which, with additional hypotheses, ensure the reduction of the variance. This method applies to discrete and continuous models in time-bounded and time-unbounded settings and allows to compute reachability properties or more general probabilistic model checking problems. It is the first method to ensure the reduction of variance and producing confidence intervals which do not relies on asymptotic hypotheses.

To this point, the reduced model is build manually which can be tedious. In the following chapter, an automated method is proposed for some classes of systems. Moreover, experiments are conducted in Chapter 6.7 to assess the efficiency of these importance samplings. Finally, in Chapter 7, a case study is performed on a biological system using this method to estimate the avancement of a reaction.

Chapter 5

Patterns for Stochastic Bounds 5.1 Introduction

In Chapter 4 we described a method for building an importance sampling that guarantees a variance reduction and produces a confidence interval. The main drawback of this method is that it requires a human intervention to design an abstraction of the studied model. In this chapter, we present a class of systems for which a stochastically bounding model with smaller state space can be automatically built and can be used as a reduced model.

Stochastic bound relies on an ordering of the state space of the system to be studied. In case of a DTMC or a CTMC we assume that the state space S is equal to {1,...,n} where 1 and n are the two absorbing states. We are interested in computing the probability to reach 1 or n from a non absorbing state in time bounded or time unbounded setting. Bounding models are Markov chains where the probability to reach n (respectively 1) is greater (respectively smaller) than in the initial model from a non absorbing state. Moreover, these properties are ensured without computing the transient or steady state distributions. Bounding models satisfy two conditions, stochastic ordering and stochastic monotony. Stochastic ordering is a property between the original model and the bounding one, whereas stochastic monotony is an intrinsic property of the bounding model. Both of them can be checked by studying the transition probability matrix or the infinitesimal generator of the Markov chain.

Bounding models are used to analyze systems with large state spaces when the properties of interest can not be computed either numerically due the size of the system or statistically due to the rare event problem or difficulties to estimate steady state probabilities. Bounding models are built with additional constraints that make the bounding model lumpable, yielding a smaller state space. Numerical methods are applied on the bounding model to compute an upper or a lower bound of the value of interest.

Bounding models are classically used to analyze a Markov chain with a large state space. In [START_REF] Abu-Amsha | An algorithm to bound functionals of Markov chains with large state space[END_REF][START_REF] Fourneau | Algorithms for an irreducible and lumpable strong stochastic bound[END_REF][START_REF] Haddad | Sub-stochastic matrix analysis for bounds computation -theoretical results[END_REF] algorithms computing bounding models from the transition probability matrix of a Markov chain are presented. These algorithms take also as input an equivalence relation over the state space, states in this relation are aggregated in the bounding model. In [START_REF] Plateau | On the stochastic structure of parallelism and synchronization models for distributed algorithms[END_REF] a stochastic system is defined as a tensor product of several Markov chains yielding a compact representation for a large Markov chain. These representations are used (for example in [START_REF] Fourneau | An algorithmic approach to stochastic bounds[END_REF]) to build bounding models by analyzing each component instead of the whole system.

The systems considered in this chapter are instantiations of a same pattern. All instances share a common structure but can have different stochastic behaviors. States of the pattern are totally ordered and this order induces a partial order on states of the global system. A pattern that stochastically bounds all the instantiations is built yielding a bounding model for the whole system. The interaction between the instances are constrained to ensure the monotonicity of the bounding model. Furthermore this bounding model is a reduced model that can be used to build importance sampling with guaranteed variance reduction.

In Section 5.2, the class of systems that we consider is defined. In Section 5.4, a coupling on the state space of systems is defined. In Section 5.5, this coupling is used to build reduced models with guaranteed variance.

Framework

Our model is a parallel composition with partial synchronization of a finite number of copies of one automaton that we call a pattern. This pattern is divided in a finite numbers of zones. Several processes move each on its own pattern.

Syntax

We first define the class of automata we shall use for that purpose.

Definition 33 (Pattern)

A pattern A = hTp,Σ,L,Z,V,δ,{! k } k2Tp i is defined by:

• Tp = {1,...,K} is a finite set of process types of cardinal K.

• Σ=Σ f ] Σ b is the set of labels partitioned into two subsets.

• L = {1,...,n} is a finite set of local states of cardinal n.

• Z =(t i ) ⇣+1
i=1 with ⇣ positive integer, (t i ) ⇣+1 i=1 an increasing sequence of states such that t 1 = 1 and t ⇣+1 = n. This sequence divides the set of local states in subsets of contiguous states called zones : z 2i-1 = {t i } and z 2i = {t i ,...,t i+1 }. We define the boundary of a zone z, denoted by @z, as @z = z = {t i },i fz = z 2i-1 and @z = {t i ,t i+1 } if z = z 2i . Thus ⇣ is the number of even zones.

• δ : L ⇥ Σ ! L is the local transition function. It satisfies : 8l 2 L, 8e 2 Σ
1. e 2 Σ f ) δ(l, e) ≥ l and we say that e labels a forward moves. e 2 Σ b ) δ(l, e)  l and we say that e labels a backward moves.

2.

81  i  ⇣, l 2 z 2i \ z 2i+1 ^e 2 Σ f ) δ(l, e) 2 z 2i . l 2 z 2i \ z 2i-1 ^e 2 Σ b ) δ(l, e) 2 z 2i . • V =(v k l ) l2L,k2Tp
is a finite set of integer variables.

• 8k 2 Tp ! k : L ⇥ Σ ! (N V ! R ≥0
) is a "rate parameter" function. It associates to a transition a function from variables to non-negative real numbers.

The pattern defines a dynamic system where several processes are moving, each on its own automaton. This automaton is depicted as a line whose states are indexed from the leftmost 1 to the rightmost n and whose transition function is δ. Transitions with same label go in the same direction (condition 1); if it goes to the right, we call it a forward move and if it goes to the left, we call it a backward move. This line is divided in ⇣ contiguous zones (the even ones). Condition 2 ensures that a process in a even zone cannot leave it without passing by the boundary of this zone, which corresponds to a zone with odd index.

Condition 2 ensures that a transition keeps a process in its current zone or moves it to an adjacent one. The stochastic behavior of the whole system is a continuous time Markov chain where each process is in concurrence with the others. The stochastic behavior on each process is defined by the rate function ! k which returns the rate of a transition. This rate depends on several parameters:

z 1 t 1 z 2 z 3 t 2 z 4 z 5 t 3 z 6 z 7 t 4 1 2 3 4 5 6
1. the type of the process which is local and constant for a process, 2. the location of the pattern and the label of the transition which are local to the process, 3. The valuation of the set of variables V which is global to the whole system.

To build the stochastic semantics, the value of a variable v k l in the set V is set to the number of processes of type k which current location is greater than l in the whole system. These will formally be defined in Section 5.2.2.

A process interacts with other processes in two ways: First through synchronization at zone boundaries, second by the means of the set of variables V . After each transition in the whole system, the value of a variable v k l 2 V is set to the number of processes of type k which current location is greater than l. These two interactions will formally be defined in Section 5.2.2.

Remark 16

For this definition the pattern must be a complete automaton, i.e. for all l 2 L and all e 2 Σ, δ(l, e) is defined. A model can be completed by adding self-loops on states where transitions labeled by e are not defined. The rate of such a self-loop can be arbitrary, for instance the minimal rate of transitions of same label for the same process (!(l, e)=min l 0 2L (!(l, e))). This transformation does not affect the stochastic behavior of the system.

z 1 = {1}, z 2 = {1, 2, 3, 4} , z 3 = {4}, z 5 = {4, 5} , z 6 = {5, 6,
7} and z 7 = {7}.I t is easy to check that the second condition on transitions holds: for every transition there is an even zone that contains its source and destination states.

Operational Semantic

In order to build a fully stochastic model from the pattern defined in the previous section one needs to add additional information to the model. The number of processes and their type must be specified. In each even zone processes are joined together in block, the number of processes per block and the total number of blocksin each region is specified. On boundaries of zone, processes join to form a block when they enter a zone and they disjoint when they exit a zone. In an odd zone the size of blocks are always of 1. Formally this is defined as follows:

Definition 34 (instance)

An instance is a pattern specified with the following list of parameters:

• I = {1,...,m} denotes the set of processes in the system.

•  : I ! Tp denotes the mapping from the set of processes to the set of types of processes. We denote by {m i } 1iK the number of processes with type i,w i t h P 1iK m i = m the total number of processes.

• {a 2i } 1i⇣ where a 2i is the cardinality of each processes per block in zone z 2i . By convention, for all 0  i  ⇣, a 2i+1 = 1. For any zone indexed by i we say that its multiplicity is a i .

• {b 2i } 1i⇣ where b 2i is the maximal number of allowed blocks in zone z 2i . By convention, for all 0  i  ⇣, b 2i+1 = m.

• f : P(I) ⇥ I ⇥{1,...,⇣}! P(I) ! [0, 1] is the probabilistic block assigning function. It ensure that if f = f(E, i, z),i 2 E, z is an even number of zone and |E|≥a z then the sum of all f (F )o v erF such that i 2 F and |F | = a z is equal to 1. For all F such that i 6 2 F or |F |6 = a z then f (F ) = 0. That is f is a discrete probabilistic distribution over the possible blocks.

An instance yields an enriched CTMC.T h eCTMC of an instance is built as a transition system based on the product of copies of the pattern A, one for each process in I with additional constraints due to synchronization between processes. Inside an even zone (let say z 2i ), a process is associated with a 2i -1 other processes, all in z 2i , in some way that we shall explain later. We call the set of these associated processes a block. Processes in a block inside zone z 2i cannot enter or leave the zone without being synchronised; this synchronization happens when all processes in the block are located on the same boundary of the zone. There cannot be more than b 2i blocks in zone z 2i , therefore when the zone is full, processes waiting to form a new block and enter the zone must wait on the boundary.

Definition 35 (Uniform block assigning function)

A block assigning function f is called uniform if for all 0 <g ⇣,E 2 P(I) such that |E|≥a 2g ,i 2 E, F ⇢ E such that i 2 F and |F | = a 2g the following holds:

f(E, i, 2g)(F )= 1 |E|-1 a 2g -1
That is the probability distribution f(E, i, 2g) is uniform over all valid blocks. 2 is an example of pattern for a distributed database example with synchronization. The system contains N databases. When a database receives an update it must synchronize it to all the others. Initially each database is in an idle state that is location 1, then one database receives a modification request with label a and moves to location 2. This request may be denied with transition c. Otherwise the database enters the critical zone z 4 and commits the change in two steps. Finally the database synchronizes with the others by entering zone z 6 . Transitions labeled by b in location 1 and 2 are used by a database which did not receive a modification request to reach the location 4 where they can synchronized together. At the end of the synchronization databases reach state 6.

t 1 t 2 z 1 z 2 z 3 z 4 3 t 4 z 5 z 7 z 6 t 3
There are several database types depending on the frequency at which they receive modification requests. To simplify we suppose that type 1 database receive a high amount of modification request while type 2 receive a low amount of modification. We also suppose that half of the database are of type 1 and half of type 2. The rate parameter function is as follows:

! 1 (1,a)=1 ! 2 (1,a)=0.5 ! 1/2 ( ,a)=1 ! 1/2 (1,b) = 10 min 1,v 1 3 + v 2 3 ! 1/2 (2,c)=0.5 -0.5min 1,v 1 3 + v 2 3 ! 1/2 (3,b) = 10 ! 1/2 (2,b) = 10 min 1,v 1 3 + v 2 3
The multiplicity of all zones except z 4 is 1. The multiplicity of z 4 is a 4 = N . The maximal number of allowed blocks is N in each zone except z 6 where b 6 =1.

The block assigning function is the uniform one.

From an instance A,I,,{a 2i } 1i⇣ , {b 2i } 1i⇣ , the annotated CTMC M =(S, Σ, ∆, Ω, f) is defined as follows:

processes: A process cannot enter a zone z of multiplicity µ>1 without being associated with µ -1 other processes in a block, these processes being all in the same local state in @z. Symmetrically, a process cannot leave this zone unless all processes in the corresponding block are ready to leave, which requires that they all are in the same local state in @z.

Due to synchronization at the creation of blocks the successor function returns a set of states. When there is no creation of block the set is a singleton, otherwise the set contains a state for each choice of blocks.

The function f : P(I) ⇥ I ⇥{1,...,⇣}!P (I) ! [0, 1] specifies how processes are chosen to build a block. This function takes as first argument a subset of I corresponding to processes ready for synchronization, as second argument the index of the process initiating the transition and the index of the zone in which the new block is created. The function returns a discrete distribution probability over the set of subsets of the suitable cardinality.

From a global state s =( l j ,z j ,p j ) j2I a global transition with label e 2 Σ and process i 2 I maps to a set of state ∆(s, (e, i)) by applying the local transition δ of the pattern on state l i with label e if possible and applying synchronization rules to update the block structure. These rules are precisely described in table 5.1 for processes lying in an even zone in this case ∆(s, (e, i)) is a singleton, and in table 5.2 for processes lying in an odd zone, in this case ∆(s, (e, i)) is set of state. If a transition is not possible due to synchronization, we say that the corresponding process is blocked and the transition is replaced by a self-loop. For the sake of conciseness we use the notation ∆(s, e, i) instead of ∆(s, (e, i)). We denote z the zone z i , containing l and the global state s + =∆(s, e, i)=(l + j ,l + j ,p + j ) j2I .

The rate function Ω.

The rate function gives the stochastic behavior of the system by specifying rates for each transition of each process. As the model is a CTMC the waiting time in each global states follows an exponential distribution whose rate is equal to the sum of all rates of transitions of each process. The probability to take a transition is its rate divided by the sum of all rates of other transitions. When there is a creation of block, several output states are possible whose probability are defined by f. The Ω function only specifies the rate for individual processes the final rate matrix is defined latter.

The global rate function is based on the local rate functions ! k by evaluating a valuation for the variables of the pattern on a global state s. The variables of the pattern are evaluated as follows:

8l 2 L, v t l ((l j ,z j ,p j ) j2I )=#{j 2 I | l j ≥ l ^(j)=t}
This means that the variable v t l takes for value the number of processes of type t in a state bigger than l. in a same state behave the same way. This will be formalized below by specifying an equivalence relation over states where two global states are equivalent if they are identical up to the processes numbering, that is each local state gets the same number of processes and if there is a one to one correspondence between the blocks. Then we will show that a symmetric instance is strongly lumpable on this equivalence relation [START_REF] Kemeny | Finite Markov chains[END_REF]. Afterward one builds the quotient of the state space S by the equivalence relation and thus obtains a smaller system with the same stochastic behavior.

Definition 37 (Equivalence relation)

Let s =((l i ,z i ,p i ) i2I ) and s 0 =((l 0 i ,z 0 i ,p 0 i ) i2I ) be two global states. Let σ be in S m We say that s ⇠ σ s 0 if:

8i 2 I,l i = l 0 σ(i) ,z i = z 0 σ(i) ,σ (p i )=p 0 σ(i)
The equivalence relation s ⇠ s 0 is defined by 9σ 2 S m ,s⇠ σ s 0 .

Lemma 2

In a symmetric instance instance M =(S, Σ, ∆, Ω, f) the following holds: Let s be a global state, e 2 E an event and i 2 I ap r o c e s st h e n 8s 1 ,s 2 2 ∆(s, e, i),s 1 ⇠ s 2 .

Proof:

For all s, s 0 2 ∆(s, e, i), let F = {j 2 I, z j = z i }. Let F 1 ,F 2 ⇢ F such that s 1 =∆ ⇤ (s, e, i, F 1 ) and s 2 =∆ ⇤ (s, e, i, F 2 )l e tσ 2 S m such that σ(i)=i, σ(F 1 )=F 2 and σ |I\F is the identity. Then s 1 ⇠ σ s 2 holds. ⇤

Lemma 3

A symmetric instance M =( S, Σ, ∆, Ω, f) satisfies the following: Let σ 2 S m and s = ((l i ,z i ) i2I , {I 1 ,...,I u }), s 0 = (l 0 i ,z 0 i ) i2I , {I 0 1 ,...,I 0 u 0 } be two global states such that s ⇠ σ s 0 .T h e n :

• (v 1 (s),...,v n (s)) = (v 1 (s 0 ),...,v n (s 0 ))
• For all i 2 I, Ω(s, e, i)=Ω(s 0 ,e,σ(i)).

Proof:

The first claim is obtained from the definition of the variables. By hypothesis there is only one process type in M ; therefore Ω(s, e, i)= !(l i ,e)(v 1 (s),...,v n (s)) and Ω(s 0 ,e,σ(i))) = ! 1 (l 0 σ(i) ,e)(v 1 (s 0 ),...,v n (s 0 )). Thus as l i = l 0 σ(i) ,Ω ( s, e, i)=Ω(s 0 ,e,σ(i)). ⇤

Proposition 12

Let M =( S, Σ, ∆, Ω, f) be a symmetric instance then it is strongly lumpable w.r.t. the relation ⇠.

Proof:

Let s = (l j ,z j ,p j ) j2I =((l i ,z i ) i2I , {I 1 ,...,I u }) and s 0 = (l 0 j ,z 0 j ,p 0 j ) j2I = (l 0 i ,z 0 i ) i2I , {I 0 1 ,...,I 0 u 0 } , such that s ⇠ σ s 0 . Let i 2 I be a process and e 2 Σ an event, F 2P(I) a subset of process and let i 0 be equal to σ(i), let

F 0 = σ(F ). Let s + =∆ ⇤ (s, (e, i, F )) = ⇣ (l + j ,z + j ,p + j ) j2I ⌘ and s 0+ =∆ ⇤ (s 0 , (e, i 0 ,F 0 )) = ⇣ (l 0+ j ,z 0+ j ,p 0+ j ) j2I ⌘ .
We have to prove that P (e 0 ,i 0 ,F 0 )|∆ ⇤ (s,(e 0 ,i 0 ,F 0 ))⇠s + Ω ⇤ (s, (e 0 ,i 0 ,F 0 )) = P (e 0 ,i 0 ,F 0 )|∆ ⇤ (s 0 ,(e 0 ,i 0 ,F 0 ))⇠s + Ω ⇤ (s 0 , (e 0 ,i 0 ,F 0 )). We first prove that s + ⇠ s 0+ For the sake of simplicity we will assume that e labels a forward move. The proof is symmetric when e labels a backward move.

• If the transition preserves the block structure of the global state s then we have s + ⇠ s 0+ using the permutation σ.

• If the block containing i is disjoint by the transition, l + i = t z i +1 and 8g 2 p i \{i},l g = t z i +1 .B l o c kp i is disjoint to produce a z i blocks of size 1, namely {g},w i t hl + g = t z i +1 , 8g 2 p i . Same conditions are fulfilled by the state s 0 Therefore s + ⇠ σ s 0+ .

• If the block containing l i is a singleton (p i = {l i }), there are at least a z i +1 other blocks of size one at the same position, and the number of blocks in the target zone z + i is strictly smaller than b z i +1 ,t h e nt h e transition joint a z i +1 blocks including p i in one block. Let q be the union of these a z i +1 blocks of size 1. Partition of blocks in state s + is equal to: {p j ,p j 6 ⇢ q}[{p j 1 [•••[p ja z i +1 } where the new block takes position t + 1. As s ⇠ σ s 0 there exists a z i +1 in s 0 which are singletons in position l i . The number of blocks in the target zone in s 0 is equal to the one in s thus the maximal number of allowed blocks in s 0 is not reached. The set

q 0 = p 0 σ(j 1 ) [•••[p 0 σ(ja z i +1 ) thus the partition of blocks in s 0+ is equal to {p 0 j ,p 0 j 6 ⇢ q 0 }[{p 0 σ(j 1 ) [•••[p 0 σ(ja z i +1 ) } Finally s + ⇠ σ s 0+ .
We now compare the two sets {(e 0 ,i 0 ,F 0 )|∆ ⇤ (s, (e 0 ,i 0 ,F 0 )) ⇠ s + } and {(e 0 ,i 0 ,F 0 )|∆ ⇤ (s 0 , (e 0 ,i 0 ,F 0 )) ⇠ s + }. Recalls that s ⇠ σ s 0 , for all (e 0 ,i 0 ,F 0 ) 2 Σ,I,P(I) such that ∆ ⇤ (s, (e 0 ,i 0 ,F 0 )) ⇠ s + , it holds that ∆ ⇤ (s 0 , (e 0 ,σ(i 0 ),σ(F 0 ))) ⇠ s + using s + ⇠ σ s 0+ . Reciprocally for all (e 0 ,i 0 ,F 0 ) 2 Σ,I,P(I) it holds that ∆ ⇤ (s 0 , (e 0 ,i 0 ,F 0 )) ⇠ s + then ∆ ⇤ (s, (e 0 ,i 0 ,F 0 )) ⇠ s + and thus the two sets are equal. Using lemma 2 we have

X (e 0 ,i 0 ,F 0 )| ∆ ⇤ (s,(e 0 ,i 0 ,F 0 ))⇠s + Ω ⇤ (s, (e 0 ,i 0 ,F 0 )) = X (e 0 ,i 0 ),9F | ∆ ⇤ (s,(e 0 ,i 0 ,F 0 ))⇠s + Ω(s, (e 0 ,i 0 )) = X (e 0 ,i 0 ),9F | ∆ ⇤ (s,(e 0 ,i 0 ,F 0 ))⇠s + Ω(s 0 , (e 0 ,σ(i 0 ))) = X
(e 0 ,i 0 ,F 0 )| ∆ ⇤ (s 0 ,(e 0 ,i 0 ,F 0 ))⇠s + Ω ⇤ (s 0 , (e 0 ,i 0 ,F 0 ))

⇤

To simplify the notation on symmetric instances, as ⇠ is a lumping relation (Proposition 12) and as when a block is created all the choices of processes leads to equivalent state (Lemma 2), in the following, we use ∆ in place of ∆ ⇤ and Ω in place of Ω ⇤ .

Every equivalence class of the relation ⇠ contains states with the same stochastic behavior up to ⇠. The following definition selects some representatives of the equivalence class which are easier to reason with.

Informally a state is correctly ordered if two conditions are fullfilled : processes in a same block are adjacently enumerated and the enumeration respects the partial ordering induced by the linear ordering of zones.

Definition 38 (Correctly Ordered)

We say that a state s =(l i ,z i ,p i ) i2I is correctly ordered if 8i, j 2 I, z i >z j ) i<j 8i, j 2 I, i 2 p j )8 k 2{i, . . . , j},k 2 p j Lemma 4

Let s =( l i ,z i ,p i ) i2I be a global state, there exists a correctly ordered global state s =( li , zi , pi ) i2I such that s ⇠ s.

Proof:

We build a permutation iteratively. Let I ⇣ = {i, z i = ⇣}. Let c be the cardinality of I ⇣ .I fc 6 = 0, we build a permutation from {1,...c} onto I ⇣ such that 8i, j 2 I ⇣ ,i2 pj )8 k 2{i, . . . , j},k 2 pj . Then we proceed the iteration from the ⇣ -1 zone. ⇤

Coupling

In this section we define a new relation  C on states of a symmetric instance.

Informally, given two states, this relation defines whether one of them is "closer" to the final state s f than the other. We formalize this idea by showing that  C is a coupling relation and that s f is the maximal element for this relation. We use this relation only on symmetric instances, therefore for the sake of simplicity, we restrict the definition to this context in this whole section.

We need an additionnal constraint on the pattern to build the coupling relation:

Definition 39 (Symmetric Monotonic Pattern)

A pattern A =(Tp,Σ,L,Z,V,δ,{ω})i ssymmetric monotonic if |Tp| =1 and for all e 2 Σ, for all couple of states l, l 0 2 L with l  l 0 , the following conditions are fulfilled: (C1). δ(l, e)  δ(l 0 ,e).

(C2).

⇢

δ(l, e) >l 0 ) ω(l, e)  ω(l 0 ,e) δ(l 0 ,e) <l ) ω(l, e) ≥ ω(l 0 ,e)

(C3). For all pairs of instantiation of variables (v

i ) n i=1 , (v 0 i ) n i=1 such that 8k 2{1,...,n},v k  v 0 k the following condition holds: ⇢ e 2 Σ f ) ω(l, e)(v 1 ,...,v n )  ω(l, e)(v 0 1 ,...,v 0 n ) e 2 Σ b ) ω(l, e)(v 1 ,...,v n ) ≥ ω(l 0 ,e)(v 0 1 ,...,v 0 n )
Condition (C1) on a monotonic pattern ensures that each transition with the same label, either forward or backward, preserves the order of states.

The pattern depicted in figure 5.1 satisfies this constraint. Condition (C2) ensures that if a process in state l can overtake a process in state l 0 ≥ l by a forward transition e then the rate of e in l is smaller than its rate in l 0 (and similarly for a backward transition). Condition (C3) ensures that for two states containing a process in the same location, the rate of a forward transition is bigger for the state closer to the final state (and similarly for a backward transition).

Checking conditions C1,C2 and C3 can be done by enumeration. These conditions concern the pattern whose number of locations is small compared to the size of the state space of the instance. Each one of these conditions compares behaviors of two locations. As long as the rate expressions are simple enough, say piece-wise linear, all the checking can be done automatically.

Remark 17 An instance built from a symmetric monotonic pattern is indeed symmetric because all processes are of type 1.

Example 9 In the database example, if all the databases are of type 1,t h e pattern is symmetric monotonic. Conditions C1, C2 and C3 hold.

Definition 40 (Coupling relation)

Let M =(S, Σ, ∆, Ω, f) be an instance, σ be in S m and s =(l i ,z i ,p i ),s 0 = (l 0 i ,z 0 i ,p 0 i ) be two states. The relation s  σ C s 0 is defined by: 8i 2 I, z i <z 0 σ(i)

or z i = z 0 σ(i) ^σ(p i )=p 0 σ(i) ^li  l 0 σ(i) s  C s 0 if there exists σ 2 S m such that s  σ C s 0 .
For two states s, s 0 , s  C s 0 means that s 0 is closer to the final state than s in the sense that there is a one to one mapping which ensures that each process of s is mapped to a process of s 0 closer to n. It also requires that if ap r o c e s si ns is mapped to a process in the same zone, its whole block is mapped to a block in s 0 .

The two next lemmas are technical properties between ⇠ and  C .T h e y are required to prove that the relation we have built is indeed a coupling relation.

The next lemma shows that the relation ⇠ and  C are "compatible". More formally, if s  C s 0 any state in the equivalence class of s is in relation with any state of the equivalence class of s 0 . Lemma 5 8s, s 0 , s, s0 2 S;( s

 C s 0 ) ^(s ⇠ s) ^(s 0 ⇠ s0 ) ) (s  C s0 ) Proof: Let s =( l i ,z i ,p i ) i2I ; s 0 =( l 0 i ,z 0 i ,p 0 i ) i2I ; s =( li , zi , pi ) i2I and s0 = ( l0 i , z0 i , p0 i ) i2I . Let σ 1 ,σ 2 and σ 3 be permutations such that s ⇠ σ 1 s, s  σ 2 C s 0 and s 0 ⇠ σ 3 s0 . Let σ be equal to σ 3 • σ 2 • σ 1 . Let us prove that s  σ C s0 . For all i 2 I,l e t i 1 = σ 1 (i); i 2 = σ 2 (i 1 ) and i 3 = σ 3 (i 2 ). If z i 1 <z 0 i 2 , as zi = z i 1 and z0 i 3 = z 0 i 2
we have zi < z0 σ(i) .O t h e r w i s e : z i 1 = z 0 i 2 and due to the first equivalence we have: σ 1 (p i )=p i 1 and li = l i 1 . Due to the relation  C we have: σ 2 (p i 1 )=p 0 i 2 and l i 1  l 0 i 2 . Due to the second equivalence we have: σ 3 (p 0 i 2 )=p 0 i 3 and l 0 i 2 = l0 i 3 . By combining all these hypothesis we have:

zi = z i 1 = z 0 i 2 =z 0 i 3 =z 0 σ(i) σ(p i )=σ 3 (σ 2 (p i 1 )) = σ 3 (p 0 i 2 )=p 0 i 3 =p 0 σ(i) li = l i 1  l 0 i 2 = l0 i 3 = l0 σ(i)

⇤

In the following we pick for any pair of states (s, s 0 ) such that s  C s 0 a fixed permutation σ such that s  σ C s 0 . This ensures the unicity of the product CTMC defined below. Due to the previous lemma product CTMC built with different sets of permutations are equivalent by the relation ⇠ and thus their respective lumpings coincide.

Lemma 6

8s, s 0 2 S; s  C s 0 )9s, s0 2 S correcly ordered s.t.

s ⇠ s and s 0 ⇠ s0 and s  Id C s0 Proof: We assume that s et s 0 are correctly ordered. Let s =( l i ,z i ,p i ) i2I and

s 0 =(l 0 i ,z 0 i ,p 0 i ) i2I . Let σ such that s  σ C s 0 .
Let z be the rightest zone reached by s and L = {i, l i 2 z} ; we denote by q its cardinal, which is a multiple of a z . Let L 0 be the set of processes in s 0 in zones on the right of z, that is

L 0 = {i, z 0 i ≥ z}. Let us split L 0 in L 0 > = {i, z 0 i >z}, L 0 = = {i, z 0 i = z}. Thanks to lemma 1, |L 0 > |, |L 0 = | are multiples of a z .
As s  σ C s 0 , L 0 contains σ(L), thus we can also split L 0 as the union of σ(L) and M = {i, i / 2 L; z 0 σ(i) > z}. Remark that |M |, is also a multiple of a z .

The permutation σ induces a bijection from M [ L onto L 0 , and thus a bijection from I \ (M [ L)={i; z 0 σ(i) <z} onto I \ L 0 = {i; z 0 i <z}. We are going to construct three permutations µ, µ 0 and ⌧ such that : let s =( li , zi , pi ) i2I and s0 =( l0 i , z0 i , p0 i ) i2I such that s ⇠ µ s, s0 ⇠ µ 0 s 0 s and s0 are correctly ordered and s  ⌧ C s0 ,w i t h8i 2 L, ⌧ (i)=i. We then conclude recursively on the number of zones containing processes of s.

Let first suppose that |L||L 0 > |.T h e nw ed e fi n e;µ = µ 0 = Id, ⌧ (i)=i, for i 2 L [ M (in that case, we do not change states s and s 0 but just re-affect the correspondance of processes between local states). The property li < l0 i = l 0 i results from

z i = z<z 0 i , for i 2 L [ M .
Suppose now that |L| > |L 0 > | ; some of the blocks of s in L correspond via σ to some blocks of s 0 in L. Let q be such that qa z = |L|-|L 0 > |.T h e r e are q blocks p i in L such that p 0 σ(i) is also in L,s a yE the set of these blocks. Remark that blocks of s in L are listed from 1 to L,b l o c kb yb l o c k: {l 1 ,..,l az } that we may call the first block, {l az+1 ,..,l 2az } that we may call the second block, and so on until {l (r-1)az+1 ,..,l raz } the last one, with |L| = ra z . We replace s by the equivalent state s such that the q blocks of s in E defined just above, are at the end of the list of indices in L. This is done using a permutation µ that fixes all indices in I \ L,t h u ss is correctly ordered. We also change the numerotation of the processes of s 0 which are in z in the same way, just permuting the blocks, using the previous numerotation. This defines a permutation µ 0 that only moves the indices in L such that σ(i) 2 L and replace s 0 by the equivalent state s0 ⇠ µ 0 s 0 .P r e c i s e l y:

-for i from 1 to |L|-qa z , the property li < l0 i = l 0 i results from z i = z<z 0 i .

-for i from |L|-qa z +1 to |L|, the property li < l0 i results from the choice of the numerotation that ensures pi =p 0 i . ⇤

We are now in position to define the coupling Markov chain of a symmetric instance with itself:

Definition 41 (Product CTMC)

Let M =(S, Σ, ∆, Ω, f) be a symmetric instance We define the product CTMC of M, as the chain C ⌦ =( C , Σ ⌦ , ∆ ⌦ , Ω ⌦ ) where:

• Σ ⌦ =Σ] Σ, where Σ is a copy of Σ. • 8(s, s 0 ) 2 S 2 , s  σ C s 0 , 8i 2 I, 8e 2 Σ ∆ ⌦ ((s, s 0 ),e,i)=(∆(s, e, i), ∆(s 0 ,e,σ(i))) and ∆ ⌦ ((s, s 0 ), ẽ, i)= 8 > < > : (∆(s, e, i),s 0 )i f ∆ ( s, e, i)  C s 0 (s, ∆(s 0 ,e,σ(i)) if s  C ∆(s 0 ,e,σ(i)) (s, s 0 ) otherwise • 8s  σ C s 0 ,Ω ⌦ ((s, s 0
),e,i)=min(Ω(s, e, i), Ω(s 0 ,e,σ(i))) and Ω ⌦ ((s, s 0 ), ẽ, i)=|Ω(s, e, i) -Ω(s 0 ,e,σ(i))|

In order to make this definition consistent, we have to prove that the effect of a transition preserves the coupling relation. This is the purpose of the next lemma.

Lemma 7

Let M =(S, Σ, ∆, Ω, f) be a symmetric instance. Then its product CTMC is preserved by transitions, that is: 8s  σ C s 0 , 8e 2 Σ, 8i 2 I, ∆(s, e, i)  C ∆(s 0 ,e,σ(i))

Proof: Let s =( l i ,z i ,p i ) i2I and s 0 =( l 0 i ,z 0 i ,p 0 i ) i2I be two states in the relation s  σ C s 0 . By lemma 6 we may suppose that s and s 0 are correctly ordered and σ = Id. Let e 2 Σ and let h be the process in I which is moved by transition e. We denote by s + =( l + j ,z + j ,p + j ) j2I (respectively s 0+ =( l 0+ j ,z 0+ j ,p 0+ j ) j2I ) the successor of s (respectively s 0 ) by transition (e, h): s + =∆ ( s, e, h) (respectively s 0+ =∆(s 0 ,e,h)). There are several excluding cases:

• When z + h <z 0 + h the property trivially holds.

• When z h = z + h = z 0 h = z 0 + h ,
that is, there is no change of zone neither for s nor for s 0 then condition C1 ensures that the property holds.

• When z h <z + h = z 0 + h , z + h is odd and e is a forward move, then z + h = z h + 1 and the block p h splits and reaches the boundaries of z h .T h e n 8i 2 p h \{h},l i = l + i = t z h /2+1  l 0 + i , using C1, l + h = t z h /2+1  l 0 + h , thus the property holds.

• When z 0 h >z 0 + h = z + h , z 0 + h is odd and e is a backward move, this case is symmetrical to the previous one by exchanging s and s 0 .

• When z h <z + h = z 0 + h , z + h is even and e is a forward move then z + h = z h + 1, processes in z h joint together to build a new block p + h . There is at least a z h +1 processes in zone z h in s. We can order the processes in s such that p h contains successive processes in z h : Let σ 2 S m such that s ⇠ σ s and 8i 2 I, z i 6 = z h ) σ(i)=i, and there exists H = {h 1 ,h 2 ,...,h a z h +1 }, σ(H)=p h , σ(h)=h and h 1 is a multiple of a z h +1 , this is possible because s is correctly ordered. Let us consider the set of processes in p + h , there are two cases:

. If z 0 h = z h +1 then 8i 2 H, z 0 i = z h +1 due to the multiplicity of zones and because s 0 is correctly ordered. Then by using C1, l + h  l 0 + h and 8i 2 H\{h},l

+ i = t z h +1 2  l 0 + i and thus s +  σ -1 C s 0 + .
. If z 0 h = z h then 8i 2 H, z 0 i = z h due to the multiplicity of zones and because s 0 is correctly ordered. Then processes join to

l 0 h to build a new block in s 0+ . Let σ 0 2 S m such that σ 0 (H)=p 0 + h then s +  σ 0 •σ -1 C s 0 + . • When z 0 h >z 0 + h = z + h , z 0 +
h is even and e is a backward move, this case is symmetrical to the previous one by exchanging s and s 0 .

⇤ Lemma 8

Let s be a global state and e the label of a forward transition. For i 2 I, we have s  σ C ∆(s, e, i)w i t hσ the identity.

Proof:

Let s =((l j ,z j ,p j ) j2I )

and let s + =∆(s, e, i)=

⇣ (l + j ,z + j ,p + j ) j2I ⌘
.T h e nl + i =∆(l i ,e) ≥ l i and for all j 6 = i, l + j = l j thus for all j 2 I, l 0 j ≥ l j and s  Id C s + . ⇤

Lemma 9

Let s be a global state and e the label of a backward transition. For i 2 I,w eh a v e∆ ( s, e, i)  σ C s with σ the identity.

Proof:

Similar to the one of the previous lemma. ⇤

Lemma 10

Let M =(S, Σ, ∆, Ω, f) be an instance built from a symmetric monotonic pattern. Then for all e 2 Σ, i 2 I and s  σ C s 0 :

( if Ω(s, e, i)  Ω(s 0 ,e,σ(i)) then s  C ∆(s 0 ,e,σ(i)) if Ω(s, e, i) ≥ Ω(s 0 ,e,σ(i)) then ∆(s, e, i)  C s 0

Proof: First we assume that Ω(s, e, i)  Ω(s 0 ,e,σ(i)). We want to prove that s  C ∆(s 0 ,e,σ(i)). As the pattern is symmetric we have Ω(s, e, i)=! 1 (l, e) and Ω(s 0 ,e,σ(i)) = ! 1 (l 0 ,e)t h u s! 1 (l, e)  ! 1 (l 0 ,e). As s  σ C s 0 we also have l  l 0 . Using the monotonicity of the pattern we know that l  δ(l 0 ,e). There are now several cases depending of e's transition type.

• e labels a forward move then s  C s 0  C ∆(s 0 ,e,σ(i)) using lemma 9.

• e labels a backward move which does not change the block structure, then as l i  δ(l 0 σ(i) ,e) and the others processes do not move we have s  σ C ∆(s 0 ,e,σ(i)).

• e labels a backward move disjoining a block, then as 8j 6 = i, l j  l 0 σ(j)

and l i  δ(l 0 σ(i) ,e), processes coupled with the block disjointed by the transition i.e. {j 2 I|σ(j) 2 p 0 σ(i) } are in a zone z i such that z i <z 0 σ(i) , thus s  σ C ∆(s 0 ,e,σ(i)).

• e labels a backward move joining a blocks. Let s 0+ =(l 0+ j ,z 0+ j ,p 0+ j ) j2I be equal to ∆(s 0 ,e,σ(i)). Then we have: 8j 6 = i, l j  l 0 σ(j) and l i  δ(l 0 σ(i) ,e),then processes coupled with blocks joint by the transition i.e. {j 2 I|σ(j) 2 p 0+ σ(i) } are in a zone z i such that z i <z 0 σ(i) ,t h u s s  σ C ∆(s 0 ,e,σ(i)).

Similarly if we assume Ω(s, e, i)  Ω(s 0 ,e,σ(i)) a symmetric proof allows to have ∆(s, e, i)  C s 0 using lemma 8.

⇤

Using Proposition 4 the product CTMC of M is a coupling.

Application to Guaranteed Variance Reduction

This framework can be used to prove guaranteed variance reduction in two ways:

1. For a system which is not symmetric one can define its symmetrized counterpart and define the reduced model as the lumping of the symmetric model. In the following a method to compute the symmetrized version of a system is presented yielding a fully automatic method.

2. When the reduced model can be defined as a symmetric monotonic instance this framework ensures that  C is a coupling relation. Then using theorem 10 the guaranteed variance reduction can be proved.

One still have to define a reduction function f and the labels function g.

We now describe the symmetrization of an instance.

Definition 42 (Symmetrization)

Let M =(S, Σ, ∆, Ω, f) be an instance. We define the symmetrization of M denoted by Sym(M)=(S, Σ, ∆,Sym(Ω), f)b y : 8s 2 S, e 2 Σ,i 2 I, Sym(Ω)(s, e, i)= ⇢ max s 0 ⇠s Ω(s 0 ,e,i) if e is a forward move min s 0 ⇠s Ω(s 0 ,e,i) if e is a backward move

The block assigning function is unchanged. By construction the resulting instance is symmetric.

Example 10 For the database example the symmetrization is obtaied by replacing databases of type 2 by databases of type 1. To build a reduced model from a pattern which is not monotonous we need to replace the pattern by a bounding pattern which is monotonous. This bounding pattern must fulfill some condition to ensure that we obtain a reduced model with guaranteed variance. We define an order relation over the set of patterns as follows:

Definition 43 (Order on pattern)

Let A =( Tp,Σ,L,Z,V,δ,{ω}) and A 0 =( {1}, Σ,L,Z,V,δ 0 , {ω 0 })t w o pattern with the same set of locations, events and zones. A 0 is bounding A if A 0 is symmetric monotonic, and for all k 2 Tp, l 1 ,l 2 2 L the following holds:

8e 2 Σ f ,δ 0 (l 1 ,e)  δ(l 1 ,e) ^ω0 k (l 1 ,e)  ω(l 1 ,e) 8e 2 Σ b ,δ 0 (l 1 ,e) ≥ δ(l 1 ,e) ^ω0 k (l 1 ,e) ≥ ω(l 1 ,e)
Several algorithms computing bounding models for CTMC exist [START_REF] Abu-Amsha | An algorithm to bound functionals of Markov chains with large state space[END_REF][START_REF] Fourneau | Algorithms for an irreducible and lumpable strong stochastic bound[END_REF][START_REF] Haddad | Sub-stochastic matrix analysis for bounds computation -theoretical results[END_REF]. they can be applied to patterns to obtain a symmetric monotonic pattern bounding the initial pattern. The bounding pattern is suitable to build a reduced model by lumping its equivalent states.. Remark that the symmetrization of an instance M of a monotonous pattern is a bounding M.

The following proposition proves that a bounding model could be used as a reduced model in the method described in Chapter 4.

Proposition 13

Let M and M 0 be two instances such that the pattern of M 0 is bounding the pattern of M and M 0 is symmetric monotonic then M 0 is a reduction with guaranteed variance of M.

Proof:

We have to prove that the hypothesis of Proposition 10 holds. Let M = (S, Σ, ∆, Ω, f) and M 0 =(S, Σ, ∆ 0 , Ω 0 , f).

Recall that the transition function and rate function that we use for the initial and reduced model must be ∆ ⇤ and Ω ⇤ when applying the proposition.

The reduction function f is defined as the identity. Recall that the set of event E in the initial model is equal to Σ ⇥ I ⇥P(I). The family of functions g : S ⇥ E ! E is defined as the identity on the labels: g(s, e)=e.

For all s 2 S, e 2 Sigma, i 2 I and F 2P(I), there are two cases to prove Condition 1: • If e 2 Σ b the proof is symmetric.

• If e 2 Σ f , ∆ 0 ⇤ (f (s),g(s, (e, i, F ))) = ∆ 0 ⇤ (s, (e, i, F ))) =∆ 0 (s, (e, i))f(s, F ) = ! 0 (l i ,e)f(s, F ) ≥ ! (i) (l i ,e)f(s, F ) =∆ ( s, (e, i))f(s, F ) =∆ ⇤ (
Condition 2 holds due to:

f (∆ ⇤ (s, (e, i, F ))) = ∆ ⇤ ( s, (e, i, F )) =∆ ⇤ (f (s), (e, i, F )) =∆ ⇤ (f (s),g(s, (e, i, F )))
⇤ Example 11 Proposition 13 applies to the database example. Thus, a reduced model with guaranteed variance can be built. Using 200 databases, 100 of each type, the size of the CTMC is 53, 075, 700.T h e s i z e o f t h e reduced model however is 60, 900 which can be efficiently handled by numerical methods. We compute the probability that a whole transaction takes place in 3 time units, that is the state where all processes are in location 7 is reached within 3 time units. The result of the computation using the importance sampling methods described in Chapter 4 and the tool Cosmos described in Chapter 6 are that this probability is in the interval [3.09 10 -38 , 1.17 10 -37 ] with a confidence level of 0.99.W i t h300, 000 trajectories the computation takes 2 hours and uses 4GB of memory.

Conclusion

In this chapter we have developed a framework for which a bounding model can be built automatically. Moreover this bounding model is suitable to be used as a reduced model with guaranteed variance. The framework structurally sets for such a model the monotonicity of the bounding model. In a future work, we would like to model more complex systems. In particular, in this framework constraints are given to control the interaction between several processes moving on a same pattern to ensure monotonicity, it would be interesting to weaken these constraints. For example it should be possible to let processes fork in a constant number of processes when entering a zone and join back into a single process when leaving the zone. An other future work is to extend this framework to open systems where processes can be created or destroyed. The method described in this chapter could be automatized but to make it completely automatic, it remains to build a tool that implements all steps of this method. There is no theoretical difficulty to build such a tool but they could be technical ones as the condition on rate covers expression containing variables and not only reals.

Chapter 6 Cosmos

Introduction

In Chapter 4, a rare event acceleration method was proposed. We chose to implement it in the tool Cosmos which is a statistical model checker of stochastic Petri nets with general distributions against the hybrid automata stochastic logic (HASL) and has been initially developed by Paolo Ballarini and Hilal Djafri [START_REF] Ballarini | HASL: An expressive language for statistical verification of stochastic models[END_REF][START_REF] Ballarini | Cosmos: A statistical model checker for the hybrid automata stochastic logic[END_REF][START_REF] Djafri | Numerical and Statistical Approaches for Model Checking of Stochastic Processes[END_REF]. The implementation of rare event handling required to modify how probability distributions were sampled. In order to evaluate these modified probability distributions, facilities to compute numerically steady states and transient distributions were also implemented such that in total, the rare event handling required around 3600 new lines of C++ code. We also provided two main improvements for Cosmos:

• We extended the input formalism to Symmetric Stochastic Nets (SSN). This is a join work with Elvio Amparore, Marco Beccuti, Susanna Donatelli and Giuliana Franceschinis. This extension required heavy modifications of the simulator (around 2500 lines of code) to implement efficient simulation of SSN. We also extended the specification language of Cosmos to take advantage of the SSN formalism [I].

• We integrated Cosmos in the CosyVerif [II] platform (around 1800 lines of parser in Cosmos and around 1500 lines of Java and XML in CosyVerif). CosyVerif provides a uniform way to deal with models, as well as tools to edit and analyze them. Before this integration, CosyVerif dealt only with Petri nets, symmetric nets and hierarchical Petri nets. Thus support for stochastic models and automata had to be added. Contributions to the CosyVerif platform are join work with Étienne André, Clément Démoulins, Lom Messan Hillah, Francis Hulin-Hubard, Fabrice Kordon, Alban Linard and Laure Petrucci.

When one wants to analyze a particular model, one needs to choose the formalism, the method and the tool for the modeling. It is thus important that the performances of the different methods and tools are benchmarked against one another in order to evaluate tool adequacy and efficiency. In this chapter, we perform different experiments:

• runtime comparisons between Cosmos and several statistical model checkers: Plasma [START_REF] Jégourel | A platform for high performance statistical model checking -plasma[END_REF], Prism [START_REF] Kwiatkowska | PRISM 4.0: Verification of probabilistic real-time systems[END_REF], Uppaal [START_REF] Bulychev | Uppaal-smc: Statistical model checking for priced timed automata[END_REF], and Ymer [START_REF] Younes | Ymer: A statistical model checker[END_REF],

• several experimentations and benchmarks of our importance sampling method presented in Chapter 4 against other importance sampling methods and numerical methods,

• runtime benchmarks for the new SSN simulator of Cosmos compared to its plain Petri net simulator. Section 6.2 describes the general architecture of Cosmos. Section 6.3 describes the implementation of the extension of Cosmos which deals with importance sampling. Section 6.4 describes the contribution to the CosyVerif platform and the integration of Cosmos. Section 6.5 describes the extension of Cosmos to SSN as well as experiments using this extension. Section 6.6 compares Cosmos with some other statistical model checkers. Section 6.7 presents the experiments conducted on importance sampling. Section 6.8 contains our conclusions.

Description of the Tool

Architecture

Cosmos is a statistical model checker accepting as input several types of stochastic Petri nets with general distribution and a HASL formula. A HASL formula is described by a Linear Hybrid Automaton (LHA). The main algorithm of this tool randomly simulates the Petri net according to its stochastic semantics and synchronizes with the execution of the automaton. During the synchronization, it evaluates HASL expressions. A statistical procedure decides when to stop the simulation and produces a confidence interval for a HASL expression.

The tool Cosmos consists of about 17000 lines of C++ code and is freely available at [START_REF] Barbot | [END_REF] under the Gnu General Public License version 3 (GPLv3). The tool relies on code generation to perform efficient simulation. It is divided into three main parts:

1. The parsing and code generation part reads the input files and the command line in order to build data structures for the net and the automaton. Then C++ code simulating both the behaviors of the net and the automaton is generated. The resulting code is compiled by a C++ compiler and linked with the simulator library. The resulting binary is the complete simulator program.

2. The simulator part is a library implementing the algorithm synchronizing the net and the automaton. It also implements the stochastic generation of event using the pseudo random number generator provided by the BOOST library and handle these events in an event queue. Rare event algorithms are implemented in the simulator and modify the way events are produced.

3. The server part launches several copies of the simulator and aggregates their results. According to statistical parameters, a procedure decides whether enough trajectories have been simulated and stops all simulators when needed. Then, HASL expressions are evaluated and several output files are produced according to options. The computation of confidence intervals uses the BOOST library for the computation of quantiles of the normal distribution function and binomial distribution.

Figure 6.1 shows a detailed overview of Cosmos internal architecture. This schematics details all files with which Cosmos interacts. Input files can be in different formats. File formats .gspn and .lha are the legacy input languages for Cosmos described in the Chapter 6 of [START_REF] Djafri | Numerical and Statistical Approaches for Model Checking of Stochastic Processes[END_REF]. The file format .grml is used by the CosyVerif platform. More details on this format are provided in Section 6.4. The files spn.cpp, markingImpl.cpp and LHA.cpp contain the generated code implementing the simulator. The markingImpl.cpp file is used to implement colored tokens when simulating SSN. The file Result.res contains the result of the computation (a copy of this file is also displayed on the command line). Several other files can be generated using some options. For the rare event part, files are generated to store the vector of probabilities µ • and the transition probability matrix P • .T h efi l emodelUnfold.grml contains an unfolded version of a model when the input is a SSN. Data files Rawdata.dat and Result.dat store data produced by the simulator. Depending on several options, they may contain the result of the simulation over time, the probability density function or the cumulative density function of an expression, the step-by-step trace of simulations or a sampled trace of simulations. Their format allows them to be read by Gnuplotto produce a graph. When using importance sampling for rare events, the reduction function is implemented in file lumpingfun.cpp.

Several tools interact with Cosmos either by calling it or by being called by it:

• A C++ compiler is required to build the simulator from the generated code. Until now, GCC and Clang have been used.

• When the Petri net is Markovian and the automaton has no clock, Prism can be called by the simulator. In this case a state space generator is used and the CTMC of the product between the model and the automaton is given to Prism. The vector of probabilities computed • The plotter GnuPlot can be called by Cosmos to produce a graph from the various output files providing a better visualization of the results.

• A small OCaml tool called AutomataGen which is part of Cosmos builds a simple automaton from a LTL style formula. This provides an easier use of the tool for simple properties where the full expressive power of HASL is not required.

• Several OCaml scripts shipped with Cosmos perform benchmark, testing and plotting.

Moreover Cosmos has been integrated into the platform CosyVerif. Alligator, the server part of CosyVerif can call Cosmos and parse its result. Coloane, the client part of CosyVerif can call Cosmos through Alligator and displays the result. It can also produce .grml files for the Petri net or the automaton that can be used through a command line.

The HASL Logic

Cosmos uses the HASL logic as a specification language [START_REF] Ballarini | HASL: An expressive language for statistical verification of stochastic models[END_REF]. In this section, we describe this logic. A formula in this logic contains two parts, a Linear Hybrid Automaton (LHA) and a set of HASL expressions. The LHA is a deterministic automaton. It can be synchronized with a discrete event stochastic process (DESP), described as a Petri net, in order to select some of its trajectories. HASL expressions are formulas on the variables of the LHA, that are evaluated along each accepted trajectory of the system. The synchronized product between the DESP and the automaton contains two kinds of transitions: either the transition is initiated by the DESP and the automaton performs a synchronization transition; or the transition is initiated by the automaton which performs an autonomous transition which does not affect the DESP. These two types of transitions are described in more detailed below. To ensure that the synchronization can efficiently be computed, some limitations on the synchronized product are required:

• The automaton is deterministic in the sense that for any trajectory of the DESP, there is at most one run of the automaton.

• For any finite trajectory of a DESP, the run of the automaton is also finite, that is, the automaton cannot take an infinity of autonomous transitions which are not synchronized with the DESP.

• In order to deal efficiently with the hybrid part of the automaton, variables evolve linearly with time in the implementation.

More formally, in order for an LHA to synchronize with a Petri net, several common objects are used in the definition of an LHA. Given a SPN M:

• The set of events E is a subset of the set of transitions T of M.

• The set Ind is defined as the set of indicator functions which are realvalued functions whose values depend only on the marking of M in a given state.

• The set Prop is a set of boolean propositions where atoms are equalities or inequalities of indicator functions in Ind.

The formal definition of a LHA is as follows: Definition 44 (Linear Hybrid Automaton (LHA))

A Linear Hybrid Automaton is a tuple A = E, L, Λ,L 0 ,L f ,X,flow, ! where:

• E is a finite alphabet of events;

• L is a finite set of locations;

• Λ:L ! Prop is a location labeling function;

• L 0 , the set of initial locations is a subset of L,;

• L f , the set of final locations is a subset of L;

• X =(x 1 ,...x n )i san-tuple of data variables;

• flow : L 7 ! Ind n is a function which associates each location to one indicator per data variable representing the evolution rate of the variable in this location. flow i denotes the projection of flow on its i th component.

• The transition relation ! is a subset of L ⇥ (2 E ⇥ Const) ] ({]}⇥lConst) ⇥ Up ⇥ L.
This is a set of edges, where:

. The letter ] is a special event which labels autonomous transitions of the automaton. . Const denotes a set of constraints defined as boolean combinations of equalities or inequalities over Ind and (x i ) n i=1 , . lConst denotes a set of left-closed linear constraints which are boolean combinations of expressions of the form

n X i=1 ↵ i x i o n c,
where ↵ i ,c 2 Ind and o n 2 {, ≥, =}.

. Up = Up 1 ,...,Up n denotes the sequence of updates. Each Up i corresponds to the update of the variable x i and is an expression over Ind and (x i ) n i=1 .

The notation l E 0 ,γ,U ----! l 0 is defined as (l, E 0 ,γ,U,l 0 ) 2!.

Furthermore the LHA A fulfills the following conditions.

• Initial determinism: 8l 6 = l 0 2 Init,Λ ( l) ^Λ(l 0 ) , false.T h i s must hold whatever the interpretation of the indicators occurring in Λ(l) and Λ(l 0 )i s .

• Determinism on events:

8E 1 ,E 2 ✓ E s.t.E 1 \E 2 6
= ;, 8l, l 0 ,l 00 2 L, if l 00 E 1 ,γ,U ----! l and l 00 E 2 ,γ 0 ,U 0 -----! l 0 are two distinct transitions, then either Λ(l) ^Λ(l 0 ) , false or γ ^γ0 , false. Again this equivalence must hold whatever the interpretation of the indicators occurring in Λ(l), Λ(l 0 ), γ and γ 0 is.

• Determinism on ]: 8l, l 0 ,l 00 2 L, if l 00 ],γ,U
---! l and l 00 ],γ 0 ,U 0 ----! l 0 are two distinct transitions, then either Λ(l) ^Λ(l 0 ) , false or γ ^γ0 , false.

• No ]-labeled loops: For all sequences

l 0 E 0 ,γ 0 ,U 0 -----! l 1 E 1 ,γ 1 ,U 1 -----! • • • E p-1 ,γ p-1 ,U p-1 ----------! l p such that l 0 = l p ,t h e r e exists i  p such that E i 6 = ].
Given a Petri net M and an LHA A, a state of the synchronized product is a tuple (m, l, X, t)w h e r em is a marking of M, l is a location in L of the automaton A, X is the valuation of the data variable and t is the current time. This synchronized product evolves with two different kinds of transitions. When a transition is fired in the Petri net, the automaton matches this transition with a transition of its own. The automaton performs autonomous transition as soon as some timed guards are satisfied. The Petri net has a fully-stochastic semantics and the automaton is deterministic yielding a fully-stochastic semantics for the synchronized product. More precisely the synchronized product evolves as follows:

• Due to its initial determinism, the LHA for the synchronized product has only one initial state: (m 0 ,l 0 , (0) n i=1 , 0) where m 0 is the initial marking of the Petri net and l 0 2 Init is the initial location of the automaton satisfying the initial determinism.

• The Petri net M fires a transition e at time t 1 such that m e,t 1 --! m 0 .I f there exists a synchronized transition in A such that l e,γ,U ---! l 0 where m 0 ,X +(t 1t) • flow(l) |= γ and m 0 |=Λ ( l 0 ) and there is no valid ] transition in the automaton that can occur before e, that is:

8t 2 2 [t, t 1 ], 8l 2 2 L, γ 2 lConst,U2 Up, l ],γ,U ---! l 2 ) m, X +(t 2 -t) • flow(l) 6 |= γ , then a synchronized transition occurs leading to state (m 0 ,l 0 , Up i [m 0 ,X +( t 1 -t) • flow(l)] n i=1 ,t 1 ) in the synchronized product.
• There is a valid autonomous transition labeled by ] in the automaton, that is

9t 0 ≥ t, l 0 2 L, γ 2 lConst,U2 Up, l ],γ,U ---! l 0 ) m, X +(t 0 -t) • flow(l) |= γ .
If there is no scheduled transition in M before time t 0 and there is no ] transition which can occur before time t 0 , then a autonomous transition occurs leading to state m, l

0 , Up i [m 0 ,X +(t 0 -t) • flow(l)] n i=1 ,t 1 .
• The synchronization fails when the automaton does not reach an accepting state and one of the following events occurs:

. The system M is in deadlock and no ] transition is available in the automaton. . The system takes a transition which cannot be matched with any transition of the automaton.

In this case the trajectory is rejected.

• As soon as an accepting state of the automaton is reached, the trajectory is stopped and accepted.

Definition 45 (HASL Expressions)

HASL expressions are given by the term Z in the following grammar:

Z ::= c | P | E[Y ] | HYPOTHESIS[c, c] | Z + Z | Z -Z | Z ⇥ Z | Z/Z Y ::= c | Last(y) | Min(y) | Max(y) | Int(y) | Mean(y) | Y + Y | Y -Y | Y ⇥ Y | Y/Y y ::= c | x | y + y | y ⇥ y | y/y,
where c denotes a real constant, x denotes a variable of the automaton and the arithmetic operators have their usual meaning. Expressions derived from y are computed at each simulation step of a trajectory. Expressions derived from Y are computed at the end of an accepted trajectory, where:

• Last(y) denotes the value of y in the last state of the trajectory,

• Min(y) and Max(y) denote the minimal and maximal values taken by y along a trajectory,

• Int(y) is the integral over time of the value of y along a trajectory,

• Mean(y) is the average value taken by y along a trajectory.

Expressions derived from Z are HASL expressions and are evaluated on a set of trajectories where:

• P denotes the probability of acceptation of trajectories by the automaton.

• E[Y ] is the expected value of Y ,

• HYPOTHESIS[c 1 ,c 2 ] denotes a pair of hypotheses: whether the probability for a path to be accepted by the automaton is above or below c 1 with indifference zone of width c 2 .

Expressions derived from Y are efficiently computed using that the hybrid part of the automaton is piecewise linear. Expressions derived from Z are evaluated as confidence intervals using statistical methods described in Section 6.2.3.

Running example.

Figure 6.2 depicts a HASL formula whose first expression Expr 1 characterizes the probability of global overflow in the tandem queues, that is there are more than N clients in the two queues before reaching a state where the queues are empty. Furthermore Expr 2 estimates the mean time from the initial state to reach a state with more than N clients without visiting a state with no client.

All the transitions of the automaton are labeled by E, >, {} meaning that they can synchronize with any transition of the Petri net, the guard is always true and there is no update. The determinism is ensured by the invariant of location.

Statistical Procedures

We now detail the different statistical procedures proposed by Cosmos for evaluating HASL expression depending on several criteria. The statistical results on which they rely are recalled in Section 2.3.

• Sequential hypothesis testing [START_REF] Wald | Sequential tests of statistical hypotheses[END_REF]. This procedure checks whether a probability is above a threshold. Parameters of this procedure are the t =1

E, >, {} • Chernoff-Hoeffding bounds [START_REF] Hoeffding | Probability inequalities for sums of bounded random variables[END_REF]. This static method requires three related parameters, each of them can be determined by the two others. These parameters are the interval width, the confidence level and the number of samples. It outputs a confidence interval whose width satisfies the requirement and where the probabilistic guarantee is exact. It applies to estimate the expectation of a bounded random variable.

(Q1 + Q2 <N) &( Q1 + Q2 > 0) E , > , { } Q1 + Q2 ≥ N Q1 + Q2 =0 E , > , { } X = {t} Expr 1 : P Expr 2 : E[Last(t)]
• Chow-Robbins bounds [START_REF] Chow | On the asymptotic theory of fixed-width sequential confidence intervals for the mean[END_REF]. This sequential method requires two parameters: the interval width and the confidence level. It outputs an interval whose width satisfies the requirement and where the probabilistic guarantee is asymptotic w.r.t. the width of the interval. It applies to estimate the expectation of a random variable, when no known bound is available.

• Gaussian approximation. This static method requires two parameters. The number of samples has to be given. The second parameter is either the confidence level or the interval width, one of these determining the other one. It ouputs an interval whose width satisfies the requirement and where the probabilistic guarantee is asymptotic w.r.t. the number of samples. It applies to estimate the expectation of a random variable. It is based on the central limit theorem.

• Clopper-Pearson bound [START_REF] Clopper | The use of confidence or fiducial limits illustrated in the case of the binomial[END_REF] This static method computes confidence intervals for binomial distributions. It takes as input three parameters, the total number of samples, the confidence level and the number of successful samples and outputs a confidence interval for the probability of a sample to be successful.

Integration of Importance Sampling

Here the integration of algorithms presented in Chapter 4 is detailed.

Distribution Parameters

Cosmos is designed to simulate discrete event stochastic systems. To achieve this, the time of occurrence of each event is sampled according to its distribution. A queue of events stores the times at which each event will occur.

The main loop of the simulation engine takes the first event in this queue, executes it and if needed adds or removes events from the queue. In this setting it is easy to simulate CTMCs. At each step of the CTMC simulation, a time is computed for each transition according to its rate and added to the queue. Consequently Cosmos does not simulate directly a DTMC,b u taCTMC whose rates are equal to the probability distribution of the DTMC.T h i s approach has some impact on the implementation of the importance sampling algorithm.

In a general case we assume that vectors of probability µ • u , u 2 N [ {1} required to compute rate of the importance sampling are stored as an array. The reduction function f is given by the user as a C function. Algorithm 5 is used to compute each simulation step. Starting from a state s 2 S and a current likelihood L 2 R +⇤ , the algorithm computes the next step t 2 S and updates the likelihood. Function λ is the rate function of the CTMC. Function λ 0 is the rate function under the importance sampling, it is computed onthe-fly and is implemented as an array. The two float variables Acc and Acc 0 are used to store the sum of the rates. The data structure EQ is the event queue, it is implemented as a heap to ensure that an insertion can be done in logarithmic time. The event ? is a special event representing all the events leading to the state s -.

Algorithm 6 is the plain algorithm used in Cosmos when importance sampling is not required. In this algorithm the computation of the set {s 0 2 E | λ(t, e 0 ) 6 = λ(s, e 0 )} is done efficiently using the structure of the model. As Cosmos takes as input a stochastic Petri net, the structure of the net is analyzed to precompute this set. This set is in general much smaller than the set E. The event queue must be initialized at the start of the trajectory.

When importance sampling is required, Cosmos uses Algorithm 5 which is more costly. In this algorithm, the event queue is emptied and filled again at each step of the simulation. In this case the function µ • is not local while without importance sampling the firing of a transition of the Petri net is local and only affects places to which it is connected.

Experiments simulating a fixed number of trajectories show that the ratio of speed between Algorithm 6 and Algorithm 5 under an importance sampling where µ • = 1 is around 5.5 slower on the example of tandem queues. Transition probability matrices are stored as sparse matrices. Steady state analysis is done by exporting the transition probability matrix to Prism and retrieving the result. Prism state space generator is not used because Cosmos needs to identify states of the reduced model to apply the reduction function. Moreover it would require the reduced model to be given in the Prism language whereas the initial model is a Petri net. Transient analysis is performed by Cosmos using sparse matrix vector multiplication. This transient analysis is performed using algorithms described in Chapter 4.

Fox-Glynn Algorithm and Uniformization

The computation of Poisson probabilities is performed using the freely available implementation of Fox-Glynn algorithm described in [START_REF] Jansen | Understanding Fox and Glynn's "computing poisson probabilities[END_REF]. The uniformization of the original Petri net model is done by adding a special transition which implements loops on the underlying CTMC. The loop rate is computed after each firing of a transition.

CosyVerif

Description

CosyVerif [II] is a software environment for the verification of systems. The goal is to provide an environment allowing to handle and modify models given in various formalisms and to interface many verification tools. This allows a user to build and modify models in an easy-to-use graphical interface and to then use various tools in the same interface. One may also edit a model on a personal computer while running verification tools on more powerful machines using a client-server architecture. To handle various kinds of formalisms, CosyVerif uses a hierarchy of formalisms sharing common concepts. One part of this hierarchy concerns automata, the other one Petri nets. In each of these parts, variants of formalisms are defined incrementally, favoring the reuse of portions of formalisms. Figure 6.5 shows the hierarchy as it is implemented in CosyVerif.

Personal Contributions

CosyVerif is the successful result of a collaboration of experts of different formalisms. It consists of around 20K lines of code for the server and 60K lines of code for the client. It is mainly written in Java and XML. In this section, I describe my personal contributions to this platform.

• CosyVerif has been initially built for handling Petri nets and symmetric nets. In order to integrate Cosmos in the platform, several new formalisms had to be built in and lead to the current hierarchy of formalisms of CosyVerif 6.5:

. A formalism for handling general arithmetic and boolean expressions used in all formalisms of the platform. It also handles parameters in expressions in a uniform way. . Stochastic extensions of Petri nets defining stochastic nets with general distributions from Petri nets and symmetric stochastic nets from symmetric nets. . Several variants of automata, and in particular linear hybrid automata which are used by Cosmos.

Abstract formalisms are used in this hierarchy to denote partial implementation of formalisms. For example, abstractPN-Core defines a notion of Petri nets as a bi-parted graph without marking or valuation. This formalism is extended later in P/T Net when markings and valuations are integers and in Symmetric-Net when markings and valuations consist of colored tokens.

• The graphical editor of CosyVerif allows one to design models of each formalism of the hierarchy. This required to implement parser handling of all expressions appearing in the various formalisms and producing an AST which is encoded in GrML. • Direct simulation consists in using a simulator which takes colors into account and which simulates directly the semantic of SSNs. As rules for firing transitions become more complex, the simulation time can be increased by this approach.

• Reduction to stochastic Petri nets consists in unfolding aS S Ni n t o a stochastic Petri net with the same semantics and then using a stochastic Petri net simulator. This approach allows one to use all the optimizations of the Petri net simulators. However, the size of the unfolded Petri net may become huge which can be problematic on some simulators. This approach also requires to "unfold" the specification formula to a formula about stochastic Petri nets.

The two approaches have been implemented in Cosmos and this section provides experimental results which compare them.

Extension of HASL

In this section, we present an extension of the HASL logic to SSNs. As SSNs use colored tokens, the HASL logic had to be extended to be able to deal with them.

The HASL logic was extended in several ways, mainly by extending the definition of LHA:

1. Automata variables can now have three different types:

• flow variables take values in R and correspond to the original variables of HASL. They evolve continuously with a constant rate with respect to the time between two transitions.

• stable variables take values in R but are only altered by transitions updates. On a single trajectory they take only a countable number of values. As they do not vary between transition firings, they can be used as constants in linear expressions.

• color variables take colors as value and are modified only on transition updates. A color variable can be seen as a subtype of a stable variable.

Additionally, these three type of variables can be indexed by a color or an integer. In this case they behave like an array.

2. An event read by the LHA is now a pair of a transition identifier and a binding of the SSN color variables. This allows the LHA to synchronize over specific colors of the net. The value of binding can be compared to or stored into color variables.

3. An expression involving some marking of the net specifies a filter on the colors of tokens which can be color variables.

4. Linear expressions of variables are linear combinations of flow variables where coefficients are expressions that can contain stable variables and marking expressions.

The extension of HASL allows one to specify interesting complex properties like property D1 given in Figure 6.7. This property applies to the database model presented in Figure 2.12 on page 41. It computes the CDF of the time for an update to be performed when the system is in a nominal state: the number of concurrent updates is small and the communication speed between the sites is high.

More precisely, the property D1 describes the cumulative distribution function of the time required for a change request to complete, under the condition that the time to acquire the mutex on the file to be changed does not exceed a given threshold and the time to send all messages from the active site which modified the file to the passive sites is below a given threshold. The LHA expressing formally D1 starts by waiting in location l 0 until T time units elapse, then waits in location l 1 for the transition Start to occur. The binding of the color variables of the SSN is copied in the LHA color variable. Using the color variable, the automaton waits for the transition Acquire in l 2 with the same binding as for the Start transition. This waiting time is bounded by the constant threshold. The automaton continues to synchronize with the SSN until the transition Release occurs and the final state l 6 is reached. All trajectories in which a time bound is exceeded are rejected. Other examples of properties using the extension of HASL on different models are given in [I].

Property D1 has been evaluated with Cosmos providing the graph depicted in Figure 6.8.

Implementation of SSN in Cosmos

The implementation of SSN in Cosmos has been influenced by the specification language of symmetric nets in the CosyVerif platform. In particular, binding variables are defined globally to the model and not locally for each transitions.

Let us first describe two possible data structures to store colored tokens:

• Each place marking can be implemented as a dynamic structure containing a list of colored tokens with their number of occurrences. This data structure is well suited for places such that the number of tokens for any marking is small but the cardinality of the domain is big. The memory consumption is linear in the number of tokens. Within this approach, the time required for checking the presence of a token with a particular color is linear whereas emptiness of a place can be checked in constant time.

• Alternatively, each place marking can be implemented as an array of integers where each possible colored token is assigned to a cell of the array. This data structure comes with an overhead in the memory consumption when the number of colors is too big. The time required to check the presence of tokens of a particular color is constant whereas checking for emptiness required a time proportional to the size of the domain of the place.

The second approach is implemented in Cosmos as it allows faster simulation when the number of colors is not excessively large, which is usually the case. This also extends naturally the original implementation of Cosmos for the marking of uncolored tokens and fits well within the code generation algorithm of the tool. Specific classes are generated to handle each color domain and color class. These classes handle color tokens of each domain.

An optimization could be to implement both approaches, and choose for each place how to implement the marking, based on an analysis of the net providing the size of the domain and a possible bound on the maximal number of tokens.

In the SSN implementation of the simulator, events are additionally labeled by the bindings of the color variables of the SSN. Code is generated to enumerate fireable bindings for a given pair of a colored marking and a transition.

The implementation of the extension of HASL in Cosmos is quite straight-Figure 6.8: Result of the estimation of D1 on the distributed database example forward, the automaton is synchronized with information on the binding of variables instead of only synchronizing over transitions.

Runtime Comparison of Symmetric Net Simulator vs Petri net Simulator

Given a SSN, a Stochastic Petri net can be built such that the set of reachable markings of both nets are in a one-to-one mapping. This transformation known as unfolding is implemented in Cosmos. In this section, the time and memory consumptions of Cosmos on a model and on its unfolded counterpart are discussed. The comparison of some performance evaluation measures also asserts the correctness of the SSN simulator.

The number of places and transitions is usually much bigger in the unfolded Petri net than in the symmetric one. For instance on the database example, the place wait mutex whose domain is S ⇥ F needs to be replaced by |S|•|F | places in the unfolded model. Similarly, each transition is replaced by a transition for each possible binding.

As Cosmos relies on code generation to simulate efficiently Petri nets, some code is generated for each place and transition of the net. Thus the time and memory required for the building phase before the simulation depends on the size of the net. When the number of places and transitions is small, this building time is negligible. However when this number becomes bigger than 1000, it may become the bottleneck of the computation. Figure 6.9 represents a client-server system where each query of a client requires three independent computations. This model uses colored tokens to keep track of the different queries. This client-server system is used with the database example to benchmark the symmetric net simulator. Figure 6.10 shows the comparison of the time consumption of both the SSN and SPN simulators on the client server example (on the right) and the overall computation.

On the database example, the firing rules are more complicated. In particular, arcs labeled by < Alls, f > are naively handled by Cosmos, yielding a large overhead in the simulation time. Even as the building time increases for SPN, for properties which require a large number of trajectories, the additional time required to used SSN is not negligible.

Comparison Between Statistical Model Checkers

In this section, we compare Cosmos to other statistical model checkers with respect to different criteria:

• Focusing on the inputs of the tools, we compare the expressiveness and the conciseness of the formalisms for modeling a system.

• Similarly, we discuss the expressiveness and the conciseness of the language for the specification of a logical property.

• As these tools generate a large number of samples, we compare the time to generate a (random) trajectory in Section 6.6.2.

• We present the statistical procedures included in each of these tools and their impact on the whole computation time.

The following comparison is related to the one of [START_REF] Jansen | How fast and fat is your probabilistic model checker? an experimental performance comparison[END_REF] where the authors performed experimental comparisons of numerical and statistical model checkers. Recently, new tools performing SMC have been released justifying new comparisons.

Expressiveness Comparison

Numerous tools are available for performing SMC, some of them also performing numerical model checkers. Here is a non-exhaustive list of tools freely available for universities: Cosmos, Plasma [START_REF] Jégourel | A platform for high performance statistical model checking -plasma[END_REF], Prism [START_REF] Kwiatkowska | PRISM 4.0: Verification of probabilistic real-time systems[END_REF], Uppaal [START_REF] Bulychev | Uppaal-smc: Statistical model checking for priced timed automata[END_REF], Apmc [START_REF] Herault | Apmc 3.0: Approximate verification of discrete and continuous time Markov chains[END_REF], Ymer [START_REF] Younes | Ymer: A statistical model checker[END_REF], Mrmc [START_REF] Katoen | The ins and outs of the probabilistic model checker MRMC[END_REF] and Vesta [START_REF] Sen | VESTA: A statistical modelchecker and analyzer for probabilistic systems[END_REF]. We have not found any updates of Apmc since 2006. Since 2011, Mrmc has not been updated and the corresponding team seems to use Uppaal. Finally, the link for downloading Vesta is not valid anymore. So we focus on the following tools: Ymer,P rism,U ppaal,P lasma and, Cosmos.

Ymer

Ymer is a statistical model checker taking as input CTMCs or generalized semi-Markov processes, described in the Prism language. Its specification language is a fragment of CSL (an adaptation of CTL with probabilistic operators replacing path operators and adding bound to time operators) without the steady-state operator but including the unbounded until.

Prism

Prism performs numerical and statistical model checking on probabilistic models. The numerical part of Prism is dedicated to discrete and continuous Markov chains, Markov decision processes and probabilistic timed automata.

The statistical part deals with discrete and continuous Markov chains. The Prism language defines a probabilistic system as a synchronized product between modules with finite state space and guarded transitions. Using this representation, this language describes big systems in a compact way. The verification procedures of Prism take as input a wide variety of languages for the specification of properties. Most of them are based on CSL or PCTL (a formalism close to CSL).

Uppaal

Uppaal is a verification tool including many formalisms: timed automata, hybrid automata, priced timed automata, etc. It supports automata-based and game-based verification techniques. Large scale applications have been analyzed with Uppaal. It has recently been enriched with a statistical model checker engine. The corresponding formalism is based on probabilistic timed systems. The probabilistic extension of Uppaal defines distributions as follows: exponential for transitions having guards without upper time bound, and uniform otherwise. The specification language is (P)LTL (i.e. an adaptation of LTL with path operators substituted for quantifiers) with bounded until.

Plasma

Plasma is a platform dedicated for statistical model checking, designed as flexible. The plugin system allows a developer to add extensions and can also be integrated in another software in a library. Prism and languages dedicated to the modeling of biological systems are supported. Prism language is extended with more general distributions. The specification language is a restricted version of PLTL with a single threshold operator.

Discussion

Formalisms are characterized by different features. First, they can be programming languages oriented like Prism or formal model oriented like Cosmos. In general, formalisms take advantage of the concurrency present in the model. Uppaal combines both approaches and allows one to specify timing requirements in the system. Application-based languages are also proposed (e.g. for biological systems in the case of Plasma). Property specifications are either defined by some timed probabilistic temporal logic or by combining hybrid automata with appropriate expressions. Whatever the choice, the main distinctive features are the following ones: presence of the unbounded until, nesting of probabilistic operators and expressiveness of time requirements. Beyond boolean properties and probability computations, Cosmos provides an expressive way to specify performance indices.

Runtime Comparison of Statistical Model Checkers

In this section, we perform runtime comparisons of several statistical model checkers. We choose examples taken from classical probabilistic system benchmarks. These examples are easily modeled in the different formalisms of tools.

Tandem queues

The first example is a variation of the tandem queue example used as the running example and is given in Figure 6.11. The Tandem Queues System (TQS) is a M/Cox 2 /1 queue composed with a M/M/1 queue, it is available on the Prism web page [START_REF] Kwiatkowska | PRISM 4.0: Verification of probabilistic real-time systems[END_REF] (where the Cox distribution is detailed). Please note that this example is different from the tandem example used in other sections of this thesis. Here, the two queues are bounded by a constant N . The specification property, denoted φ TQS ,i s : The two queues are full before T time units,w h e r eT is some time bound. This property is expressible in all specification languages of the tools. One may be interested either in the satisfaction probability or only in knowing whether it is above 0.3. For the experiments, we use N = 5, λ = 20, µ 1 =0.2, µ 0 1 =1.8, µ 2 = 2 and  = 4 as in [START_REF] Kwiatkowska | PRISM 4.0: Verification of probabilistic real-time systems[END_REF].

n 1 n 2 λ µ 1 ,µ 0 1 ,µ 2  Figure 6.11: Tandem Queues System (TQS)

Dining Philosophers

The Dining Philosopher Model (DPM) is a standard benchmark problem for discrete event systems. We have adapted it in a probabilistic setting. In the DPM, N philosophers eat and think around a table. While thinking, a philosopher decides after some random time to pick its right fork, then after some more time, he decides to take its left fork and to start to eat. Later on, the philosopher starts thinking again and puts back his two forks. A fork is shared between two neighbours at the table. In the probabilistic setting, all the times are exponentially distributed. The property that we study is the following one: a deadlock occurs before they have been N lunches (possibly by the same philosopher). The single deadlock occurs when all philosophers have taken their right fork. The number of states in this system is exponential with respect to the number of philosophers. This benchmark is thus a typical case were SMC is the only way to perform analysis of the system due to the state space explosion problem. For the experiments, we choose 10 for the (common) rate of the exponential distributions.

Experiment settings

We have set the following statistical parameters: the confidence level is 0.95 and the width of confidence interval is 0.005, the probability of error is 0.005 for the hypothesis testing and the width of the indifference region is 0.001. These parameters have been chosen in order to obtain a important number of trajectories. The other parameters have been set to their default value. Most tools can take advantage of parallelisation but for simplicity we use only one processor1 for the comparison.

Prism, Uppaal and Plasma support Chernoff-Hoeffding and hypothesis testing methods. In addition, Prism provides two sequential methods for confidence intervals where the stopping criterion is unknown to us. Cosmos provides Chernoff-Hoeffding, Chow-Robbins and Gaussian methods. Ymer offers the hypothesis testing method.

DPM experiments.

Figure 6.12 refers to the runtime for the DPM as a function of the number of philosophers. The results are also reported in Table 6.1. Among the tools using Chernoff-Hoeffding bounds, Cosmos is the fastest. For 100 philosophers, Plasma is 1.95 times slower, Uppaal is 1.5t i m e ss l o w e r and Prism-APMC is 2.5 times slower. Among the tools using sequential procedure, the two versions of Prism have similar runtime and Cosmos is up to 1.95 times faster.

TQS experiments with confidence intervals.

Results about the runtime comparison with different time bounds T are reported in Figure 6.13. The results are also reported in Table 6.2. There are two kinds of behaviors for tools depending on the applied method. For the first kind which corresponds to the Chernoff-Hoeffding method, the Table 6.1: Runtime comparison for the DPM simulation time increases with respect to the time bound T . About 295000 trajectories are required to obtain the specified confidence interval. For the second kind which corresponds to sequential confidence interval methods, the required number of samplings decreases when the time bound goes to infinity. This phenomenon is a consequence of the evolution of the satisfaction probability of φ TQS that goes to 1 when T goes to infinity. Among the tools using Chernoff-Hoeffding method, Cosmos is again the fastest. When the time bound is 200, Plasma is 2.58 times slower, Uppaal is 4.18 times slower and Prism-APMC is 6.47 times slower. Among the tools using sequential procedure, when the time bound is 40, the two versions of Prism have similar runtime and Cosmos is up to 2.85 time faster. We performed experiments on classical benchmarks using these tools.

They have shown that the choice of the statistical test has a big impact on the simulation time. Thus tool developers should provide as many tests as possible to let users choose which one best fits their needs. For the same statistical procedure, we observed that differences of speed can be as big as six times faster. What remains to be done is to perform these comparisons on large scale case studies, contrary to the benchmark we choose here. However some difficulties may arise for the modeling of such case studies in every formalism.

6.7 Importance Sampling Benchmark

Global overflow in tandem queues

The first example we study is the running example of the tandem queue system. Recall that the running example consists of two waiting queues in tandem (M/M/1), where client arrivals are distributed following a Poisson distribution of parameter λ. Clients are served in the first queue with rate ⇢ 1 and in the second queue with rate ⇢ 2 . We are interested in computing the probability of a global overflow, that is the probability to reach a state where the total number of clients is larger than a threshold N , starting from a state with one client in the first queue, without reaching a state where the system is empty. As we are interested in rare-event probability, we suppose that the system is stable, that is ⇢ 1 <λand ⇢ 2 <λ. We extend this system to d queues in tandem, the probability of interest is unchanged and the reduced model is built by bounding all the queues except the first one by R. Figure 6.15 shows Petri nets modeling the system and its reduction. The mapping function f iteratively maps the exceeding clients of each queue to the previous one starting from the last one. This system is a classical benchmark for importance sampling methods on queuing network. Even if this example is simple to describe, several issues arise when designing importance sampling for it. In [START_REF] Parekh | A quick simulation method for excessive backlogs in networks of queues. Automatic Control[END_REF], an empirical state-independent importance sampling is designed. It consists in exchanging the arrival rate and smallest serving rate. It was proved later in [START_REF] Sadowsky | Large deviations theory and efficient simulation of excessive backlogs in a GI/GI/m queue. Automatic Control[END_REF] that this importance sampling is optimal among the state-independent important samplings. In [START_REF] Boer | Analysis of state-independent importance-sampling measures for the two-node tandem queue[END_REF], an exhaustive analysis of state-independent important sampling shows that some values of λ, ⇢ 1 ,⇢ 2 in the importance sampling lead to infinite variance. In [START_REF] Dupuis | Dynamic importance sampling for queueing networks[END_REF], a method to build efficient state-dependent importance sampling is described. This method is asymptotically optimal when some of its parameters tend to zero.

Q 1 Q 2 Q d λ ρ 1 ... ρ d f Q 1 Q 2 R AQ 2 Q d R AQ d λ ρ 1 ... ρ d
In Table 6.4, we compare our method to several of these importance samplings. Experiments have been performed for several values of d and N . The confidence interval has been computed with a confidence level of 0.95. The following table shows the parameters used depending of the value of d, the second column indicates the parameter R used to bound the queues in For the sake of readability, the exponent has been factorized for each line and is reported in the third column.

• The fourth column reports results computed numerically with the tool Prism.

• Columns 5 and 6 report the result of our method using the tool Cosmos.

• Columns 7 and 8 report experiments performed in [START_REF] Dupuis | Dynamic importance sampling for queueing networks[END_REF] using an asymptotically optimal importance sampling.

• The last column reports experiments performed using the importance sampling described in [START_REF] Parekh | A quick simulation method for excessive backlogs in networks of queues. Automatic Control[END_REF], which is an optimal state-independent importance sampling.

We observed that in all cases our method provides tighter confidence intervals than the ones of DSW (at least 5 times tighter and at most 70) and contains the real value. For PW, we observed a classical caveat of importance sampling, that is underestimation of the value. As the value is underestimated, confidence interval computed in this case were not correct and thus were not reported. This caveat will be discussed in more details on experiments where guaranteed variance reduction cannot be obtained.

We now analyze the memory consumption of the different variants of our method compared to numerical computations. The number of states in the model is bounded by N d and the number of states in the reduced model is bounded by N • (R + 1) d-1 . Table 6 The size of the DTMC grows exponentially with d as well as the time and memory required by Prism to perform the computation. For d = 9, the required memory exceeds the maximum of 42GB available on the computer used for the experimentation. The size of the reduced model also grows exponentially with d but at a much slower pace. The computation of µ • is therefore always tractable. The time required by Cosmos for the simulation also increases but is much smaller than the time required by Prism when the model becomes big (d ≥ 6). Times required for the importance sampling with sink state and with normalization are similar. Table 6.6 and 6.7 show statistical results. The first column represents the number of queues. The exponent has been factorized to the second column to save space. The third column reports numerical results when available. The fourth column reports the value of µ • (s • 0 ) computed with Prism on the reduced model. For the two importance samplings, the mean value, the confidence interval and the confidence interval width are reported. For the importance sampling with sink state, confidence intervals are computed using bounds on Bernouilli law. For the importance sampling with normalization, Chernoff-Hoeffding bounds are used and intervals computed using Gaussian analysis are also provided. The observed confidence intervals are always very tight around the mean value and contain the numerical value when it is available. The first line d = 1 is reported for completeness but is not relevant as the reduced model is equal to the initial model.

On this model, the importance sampling with normalization with Chernoff-Hoeffding bounds is less effective than the one with sink state. This is due to the Chernoff-Hoeffding bounds which are very conservative. Asymptotical confidence intervals computed using Gaussian analysis are also provided. On this set of experiments, the Gaussian intervals are tighter than the Chernoff-Hoeffding and Bernouilli ones, and still contain the numerical value when available.

Local Overflow in Tandem Queues

We consider the tandem queue system with two queues with a different property to check: The second queue contains N clients (n 2 = N ) before the second queue is empty (n 2 =0 ). The state space is S = N ⇥ [0..N ] with initial state (0, 1). Contrary to the first example, C is now infinite but C • must be finite in order to apply the numerical model checker.

The reduced model behaves as the original one until the first queue contains R clients. Then the model behaves as if there was an infinite number of clients in the first queue. The Petri net is depicted in Figure 6.16(a). The corresponding reduction function f (whose effect on the original chain is represented in Figure 6.16(b)) is defined by: We found by running experiments on small values of N and R that for R ≥ 0.74 ⇥ N we have µ(s 0 ) ≥ µ • (f (s 0 ))/10. This example shows that we can apply our method on an infinite model subject to the specification of a finite reduced model. Observe that computation time reductions w.r.t. standard statistical model checking are still impressive.

f (n 1 ,n 2 )= ⇢ (n 1 ,n 2 )i f n 1  R (R, n 2 ) otherwise dE x p µ(s 0 ) µ • (s • 0 ) Mean Chernoff-
Q 1 R AQ 1 Q 2 λ ρ 0 1 R ρ 1 ρ 2 (a) Reduced model s - s + s 0 (b) Reduction chain

Bottleneck in Tandem Queues

We consider the tandem queue system with another property to check: The second queue is full (n 2 = N ) before the first one (n 1 = N ). The reduced model is obtained by considering that the second queue is full when it contains N -R clients or in an equivalent way that the second queue always contains at least R clients. The corresponding Petri net is depicted in Figure 6.17(a). The reduction function (whose effect on the original chain is represented in Figure 6.17(b)) is defined by:

f (n 1 ,n 2 )= ⇢ (n 1 ,R)i f n 2  R (n 1 ,n 2 ) otherwise
However, the experimental results are not satisfactory since µ(s 0 ) ⌧ µ • (f (s 0 )) when R is small compared to N . This shows that designing a reduced model with relevant computation time reduction is sometimes tricky (and remains to be done for this example).

Parallel Random Walk

The Petri net depicted in Figure 6.18 models a parallel random walk of N walkers. A walk is done between position 1 and position L starting in position L/2 and ends up in the extremal positions. At every round, some random walker can randomly go in either direction. However when walkers i and i + 1 are in the same position, walker i can only go toward 1. We

Q 1 R Q 2 λ ρ 1 R +1 R ρ 2 (
R i,1 A i,2 R i,2
... Transition A i,j (resp. R i,j-1 ) corresponds to a step toward L (resp. 1). This model is a paradigm of failure tolerant systems in which each walker represents a process which finishes its job when it reaches position 1. Failures may occur and drive the process away from its goal. When position L is reached, the job is aborted. We want to evaluate the probability that a majority of walkers reaches position L.

A i,j-1 R i,j-1 A i,j R i,j A i,j+1 R i,j+1 P i,j P i,j P i,j+1 P i,j+1 ... A i,L-2 R i,L-2 A i,L-1 P i+1,j P i+1,j P i+1 ,j+1 P i+1,j+1
This model has L N states. In order to get a reduced model, we remove all synchronisations between walkers. Behaviors of all walkers are now independent and thus a state of the reduced system is now defined by the number of walkers in each position. The size of the reduced system is

N +L-1 L-1
. Proposition 3 holds for this reduced model. Intuitively, removing synchronisation between walkers increases the probability to reach position L.

Table 6.9 shows the experimental results with the following parameters p =0 .3,q =0 .7, L = 15. We stop the simulation when the confidence interval width reaches one tenth of the estimated value. Our method handles whereas the standard statistical model checking and numerical model checking fail due to either the low probability or the size of the system.

The Dining Philosophers

The last example is the usual dining philosophers problem which illustrates a problem of concurrency for sharing resources. The transition rate for a philosopher to take his right fork is λ 1 , the one to take its left fork is λ 2 and the one to put back both forks on the table and start to think again is ⇢. We are interested in the following property: All the philosophers are blocked because they have all taken their right fork, before r meals are eaten. The model of the system is a Petri net depicted in Figure 6.19(a) in the particular case of four philosophers. The place in the center of the net (Ns) is used to compute the number of eaten meals (r).

In the reduced model, philosophers are not sitting while they think. When a philosopher wants to eat, he sits down on the left of another philosopher and takes his right fork, if no philosopher is already sitting, he sits down anywhere. Only the leftmost philosopher is then able to take his left fork. This model is depicted Figure 6. 19(b). In this model, the deadlock is more frequent than in the original one. We define the rate as f 1 = m(Th) ⇥ λ 1 , f 2 = min(m(Wa),m(Fo)) ⇥ λ 2 and e = m(Ea) ⇥ ⇢ where m is the current marking. The number of states in the original model is at least 2 n • n whereas the number of states in the reduced model is at most n 3 . There is an exponential gap in between the size of the two models which allows one to easily compute µ • even on big models. For this example, our method returns good approximations for small values of N but for larger values the likelihood associated to some rare trajectories is large as can be seen on Histogram 6.20(b) where the impact of trajectories are there frequency multiplied by there likelihood. Thus in this histogram one can observe that a single trajectory with a likelihood of 10 -6 contribute more to the observable average value of W s 0 than the 5000 trajectories with a likelihood of 2 • 10 -12 .

Conclusion

In this chapter, we provided some implementation details and the results of some benchmarks and experiments for the tool Cosmos. From these observations, we can state that Cosmos is well adapted for the analysis of stochastic Petri nets and higher formalisms as it offers most functionalities users are interested in. Its performances in terms of simulation speed are among the best available on typical benchmarks. Thanks to its integration in the verification platform CosyVerif, it benefits from a graphical user interface and can be run as a virtual machine over all common operating systems.

We performed experiments on the importance sampling methods described in Chapter 4 and found that they are often more efficient than other importance samplings methods. For different kinds of models where the state space is too large to be analyzed with numerical methods, these importance sampling methods efficiently estimate the rare-event probability of interest. The case study we consider in this chapter is an example of biological regulatory networks which models how several chemicals species, more specifically proteins and genes, interact to regulate a biological function. These network models systems with continuous behaviors but they can be accurately modeled as discrete systems [START_REF] Richard | Formal methods for modeling biological regulatory networks[END_REF]. As a discrete system they have been widely studied using computer science techniques [START_REF] Paulevé | Refining dynamics of gene regulatory networks in a stochastic ⇡-calculus framework[END_REF][START_REF] Herajy | Towards a computational steering and Petri nets framework for the modelling of biochemical reaction networks[END_REF] Signaling processes is an example of biological regulatory networks playing a crucial role for the regulatory behavior of living cells. They mediate input signals, i.e. the extracellular stimuli received at the cell membrane, to the cell nucleus, where they enter as output signals the gene regulatory system. Understanding signaling processes is still a challenge in cell biology. To approach this research area, biologists design and explore signaling networks, which are likely to be building blocks of the signaling networks of living cells. Among them are the Ras/Raf/MEK/ERK pathway which has been extensively studied (see for example this review [START_REF] Kolch | Meaningful relationships: the regulation of the Ras/Raf/MEK/ERK pathway by protein interactions[END_REF]). This pathway is involved in the control of cell proliferation, differentiation and apoptosis.

A signaling cascade is a set of reactions which can be grouped into levels. At each level a particular enzyme is produced (e.g. by phosphorylation); the level generally also includes the inverse reactions (e.g., dephosphorylation). The system constitutes a cascade since the enzyme produced at some level is the catalyzer for the reactions at the next level. The catalyzer of the first level is usually considered to be the input signal, while the catalyzer produced by the last level constitutes the output signal. The transient behavior of such a system presents a characteristic shape, the quantity of every enzyme increases to some stationary value. In addition, the increases are temporally ordered w.r.t. the levels in the signaling cascade. This behavior can be viewed as a signal traveling along the levels, and there are many interesting properties to be studied like the traveling time of the signal, the relation between the variation of the enzymes of two consecutive levels, etc.

In [START_REF] Heiner | Petri nets for systems and synthetic biology[END_REF], it has been shown how such a system can be modeled by a Petri net which can either be equipped with continuous transition firing rates leading to a continuous Petri net which determines a set of differential equations or by stochastic transition firing rates leading to a stochastic Petri net. This approach emphasizes the importance of Petri nets which, depending on the chosen semantics, permit to investigate particular properties of the system. In this paper we wish to explore the influence of stochastic features on the signaling behavior, and thus we focus on the use of stochastic Petri nets.

Analysis of stochastic Petri nets can be performed either numerically or statistically. The former approach is much faster than the latter and provides exact results up to numerical approximations, but its application is limited by the memory requirements due to the combinatory explosion of the state space.

In this chapter we consider three families of properties for signaling cascades that are particularly relevant for the study of their behavior and that are (depending on a scaling parameter) potentially rare events. From an algorithmic point of view, this case study raises interesting issues since the combinatorial explosion of the model quickly forbids the use of numerical solvers and its intricate (quantitative) behavior requires elaborated and different abstractions depending on the property to be checked.

Due to these technical difficulties, the signaling cascade analysis has led us to substantially improve our method and in particular the way we obtain the final confidence interval. From a biological point of view, experiments have pointed out interesting dependencies between the scaling parameter of the model and the probability of satisfying a property.

In Section 7.2 we present the biological background, the signaling cascade under study and the properties to be studied. Then in Section 7.3, we model signaling cascades by stochastic Petri Nets (SPN). Then in Section 7.4 we report and discuss the results of our experiments. Finally in Section 7.5, we conclude and give some perspectives to this work.

Biological Background

In technical terms, signaling cascades can be understood as networks of biochemical reactions transforming input signals into output signals. In this way, signaling processes determine crucial decisions a cell has to make during its development, such as cell division, differentiation, or death. Malfunction of these networks may potentially lead to devastating consequences on the organism, such as outbreak of diseases or immunological abnormalities. Therefore, cell biology tries to increase our understanding of how signaling cascades are structured and how they operate. However, signaling networks are generally hard to observe and often highly interconnected, and thus signaling processes are not easy to follow. For this reason, typical building blocks are designed instead, which are able to reproduce observed input/output behaviors. The case study we have chosen for our paper is such a signaling building block: the mitogen-activated protein kinase (MAPK) cascade [START_REF] Levchenko | Scaffold proteins may biphasically affect the levels of mitogen-activated protein kinase signaling and reduce its threshold properties[END_REF]. This is the core of the ubiquitous ERK/MAPK network that can, among others, convey cell division and differentiation signals from the cell membrane to the nucleus. The description starts at the RasGTP complex which acts as an enzyme (kinase) to phosphorylate Raf, which phosphorylates MAPK/ERK Kinase (MEK), which in turn phosphorylates Extracellular signal Regulated Kinase (ERK). We consider RasGTP as the input signal and ERKPP (activated ERK) as the output signal. This cascade (RasGTP ! Raf ! MEK ! ERK) of protein interactions is known to control cell differentiation, while the strength of the effect depends on the ERK activity, i.e. concentration of ERKPP.

Raf

The scheme in Figure 7.1 describes the typical modular structure for such a signaling cascade, see [START_REF] Chickarmane | Oscillatory dynamics arising from competitive inhibition and multisite phosphorylation[END_REF]. Each layer corresponds to a distinct protein species. The protein Raf in the first layer is only singly phosphorylated. The proteins in the two other layers, MEK and ERK respectively, can be singly as well as doubly phosphorylated. In each layer, forward reactions are catalysed by kinases and reverse reactions by phosphatases (Phosphatase1, Phosphatase2, Phosphatase3). The kinases in the MEK and ERK layers are the phosphorylated forms of the proteins in the previous layer. Each phosphorylation/dephosphorylation step applies mass action kinetics according to the pattern A + E ⌦ AE ! B + E. This pattern reflects the mechanism by which enzymes act: first building a complex with the substrate, which modifies the substrate to allow for forming the product, and then disassociating the complex to release the product; for details see [START_REF] Breitling | A structured approach for the engineering of biochemical network models, illustrated for signalling pathways[END_REF].

Figure 7.2 depicts the evolution of the mean number of proteins with time. At time zero there is a only hundred RasGTP proteins, then we observe the transmission of the signal as successively the number of RafP MEKPP and ERKPP increases. Having the wiring diagram of the signaling cascade, a couple of interesting questions arise whose answers would shed some additional light on the subject under investigation. Among them are an assessment of the signal strength in each level, and specifically of the output signal. We will consider these properties in Sections 7.4.1 and 7.4.2. The general scheme of the signaling cascade also suggests a temporal order of the signal propagation in accordance with the level order. What cannot be derived from the structure is the extent to which the signals are simultaneously produced; we will discuss this property in Section 7.4.3.

Petri net modeling

We now explain how to model our running case study in the Petri net framework. The signaling cascade is made of several phosphorylation/dephosphorylation steps, which are built on mass/action kinetics. Petri net component depicted in Figure 7.3. The mass action kinetics is expressed by the rate of the transitions. The marking-dependent rate of each transition is equal to the product of the number of tokens in all its incoming places up to a multiplicative constant given by the biological behavior (summing up dependencies on temperature, pressure, volume, etc.).

The whole reaction network based on the general scheme of a three-level double phosphorylation cascade, as given in Figure 7.1, is modeled by the Petri net in Figure 7.4. The input signal is the number of tokens in the place RasGTP, and the output signal is the number of tokens in the place ERKPP.

This signaling cascade model represents a self-contained and closed system. It is covered with place invariants (see the appendix), specifically each layer in the cascade forms a P-invariant consisting of all states a protein can undergo; thus the model is bounded. Assuming an appropriate initial marking, the model is also live and reversible; see [START_REF] Heiner | Petri nets for systems and synthetic biology[END_REF] for more details, where this Petri net has been developed and analyzed in the qualitative, stochastic and continuous modeling paradigms. In our paper we extend these analysis techniques for handling properties corresponding to rare events.

We introduce a scaling factor N to parameterize how many tokens are spent to specify the initial marking. Increasing the scaling parameter can be interpreted in two different ways: either an increase of the biomass circulating in the closed system (if the biomass value of one token is kept constant), or an increase of the resolution (if the biomass value of one token inversely decreases, called level concept in [START_REF] Heiner | Petri nets for systems and synthetic biology[END_REF]). The kind of interpretation does not influence the approach we pursue in this paper. Increasing N means to increase the size of the state space and thus of the CTMC, as shown in Table 7.1 which has been computed with the symbolic analysis tool Marcie [START_REF] Heiner | MARCIE -Model checking And Reachability analysis done effiCIEntly[END_REF]. As expected, the explosion of the state space prevents numerical model checking for higher N and thus calls for statistical model checking.

Furthermore, increasing the number of states means to actually decrease the probabilities to be in a certain state, as the total probability of 1 is fixed. With the distribution of the probability mass of 1 over an increasingly huge number of states, we obtain sooner or later states with very tiny probabilities, and thus rare events. Neglecting rare events is usually appropriate when focusing on the averaged behavior. But they become crucial when certain jump processes such as mutations under rarely occurring conditions are of interest.

Experiments

We have analysed three properties, the last two are inspired by [START_REF] Heiner | Petri nets for systems and synthetic biology[END_REF]. Recall that the initial marking of the model is parametrized by a scaling factor N . For the first two properties, the reduced model is the same model but with local smaller scaling factors on the different layers of phosphorylation. Every state of the initial model is mapped (by f ) to a state of the abstract model which has the "closest" proportion of chemical species. For instance let N = 4 which corresponds to 16 species of the first layer, a state with 6 tokens in Raf and 10 tokens in RafP is mapped, for a reduced model with N = 3, to a state with 4 = b6 ⇥ 3/4c tokens in Raf and 8 = d10 ⇥ 3/4e tokens in RafP (see the appendix for a specification of f ).

All statistical experiments have been carried out with our tool Cosmos [START_REF] Ballarini | HASL: An expressive language for statistical verification of stochastic models[END_REF]. Cosmos is a statistical model checker for the HASL logic [START_REF] Ballarini | HASL: An expressive language for statistical verification of stochastic models[END_REF]. It takes as input a Petri net (or a high-level Petri net) with general distributions for transitions. It performs an efficient statistical evaluation of the stochastic Petri net by generating a code per model and formula. In the case of importance sampling, it additionally takes as inputs the reduced model and the mapping function specified by a C function and returns the different confidence intervals.

All experiments have been performed on a machine with 16 cores running at 2 GHz and 32 GB of memory both for the statistical evaluation of Cosmos and the numerical evaluation of Marcie.

Maximal peak of the output signal

The first property is expressed as a time-bounded reachability formula assessing the strength of the output signal of the last layer: "What is the probability to reach within 10 time units a state where the total mass of ERK is doubly phosphorylated?", associated with probability p 1 defined by: p 1 = P(True U 10 (ERKPP =3N ))

The inner formula is parametrized by N , the scaling factor of the net (via its initial marking). The reduced model that we design for Cosmos uses different scaling factors for the three layers in the signaling cascade. The first two layers of phosphorylation which are based on Raf and MEK always use a scaling factor of 1, whereas the last layer involving ERK uses a scaling factor of N . The second column of Table 7.2 shows the ratio between the number of reachable states of the original and the reduced models.

Experimental Results.

We have performed experiments with both Cosmos and Marcie.T h et i m e and memory consumptions for increasing values of N are reported in Table 7.2. For each value of N we generate one million trajectories with Cosmos.W e observe that the time consumption significantly increases between N =3 and N = 4. This is due to a change of strategy in the space/time trade-off in order to not exceed the machine memory capacity. Marcie exponential increase w.r.t. both time and space resources. When N = 3, it is slower than Cosmos and it is unable to handle the case N = 4.

Table 7.3 depicts the values returned by the two tools: Marcie returns a single value, whereas Cosmos returns three confidence intervals (discussed above) with a confidence level set to 0.99. We observe that confidence intervals computed by the Gaussian analysis neither contain the result, the ones computed by Chernoff-Hoeffding do not contain it for N = 3, and the most conservative ones always contain it (when this result is available). An analysis of the likelihood L s 0 is detailed in the appendix.

Figure 7.5 illustrates the dependency of p 1 with respect to the scaling factor N . It appears that the probability p 1 depends on N in an exponential way. The constants occurring in the formula could be interpreted by biologists.

Mapping function.

We describe here formally the reduction function f . The reduction function must map each marking of the Petri net to a marking of the reduced Petri net.

First we observe that the signaling cascades SPN contains three places invariants of interest:

• The total number of tokens in the set of places {Raf,Raf RasGTP,RafP Phase1,RafP, MEK RafP, MEKP RafP} is equal to 4N .

ERKPP being zero, the concentration of RafP rises above some level L while the concentrations of MEKPP and ERKPP remain at zero, i.e. RafP is the first species to react?". While this property has its focus on the beginning of the signaling cascade, it is obvious how to extend the investigation by further properties covering the entire signaling cascade. This formula is parametrized by L. Due to the lack of space only some values of L in [0, 4N [ are reported. The results for increasing N and L are given in Table 7.4.3. As can be observed, the probability to satisfy this property is not a rare event thus no importance sampling is required. Instead results are obtained by a plain Monte Carlo simulation generating 10 millions of trajectories. For N>3 Marcie requires more than 32GB of memory thus the computation was stopped. On the other hand, the memory requirement of Cosmos is around 50MB for all experiments.

We also observed that as expected the probability exponentially decreases with respect to L.

Conclusion

We have studied rare events in signaling cascades with the help of an improved importance sampling method implemented in Cosmos. As demonstrated by means of our scalable case study, our method has been able to cope with huge models that could not be handled neither by numerical computations nor by standard simulations. In addition, analysis of the experiments has pointed out interesting dependencies between the scaling parameter and the quantitative behavior of the model.

In future work we intend to incorporate other types of quantitative properties, such as the mean time a signal needs to exceed a certain threshold, the mean traveling time from the input to the output signal, or the relation between the variation of the enzymes of two consecutive levels. We also plan to analyse other biological systems for which the evaluation of tiny probabilities might be relevant like mutation rates in growing bacterial colonies [START_REF] Gilbert | Colouring spacea coloured framework for spatial modelling in systems biology[END_REF].

Chapter 8

Conclusion and Perspectives

Conclusion

In this thesis, a new method to deal with the quantitative model checking of rare events was presented. This method was thus designed to compute extremely small probabilities with a conservative confidence interval which is of particular interest for critical systems. The rare-event problem was recalled in Chapter 3. To achieve this computation, this method relies on the combination of two well-know methods: Monte Carlo simulations and numerical model checking which were both presented in Chapter 2. This combination is performed by first applying numerical methods over an abstraction of the model and then using the outcoming results to bias the Monte Carlo simulation with importance sampling as shown in Chapter 4. Contrary to other methods using Monte Carlo simulations, the method presented provides given some hypotheses, a conservative confidence interval. Additionally, it also guarantees the reduction of the variance with respect to a plain Monte Carlo simulation so that the width of this conservative confidence interval is also smaller.

This method can be used directly on any finite discrete or continuous Markov chain. However, the abstraction of the model has then to be defined manually which can be extremely tedious as the reader might have noticed. To circumvent this difficulty, a framework for probabilistic models was provided in Chapter 5 in which the abstraction is derived automatically.

This method has been implemented in the tool Cosmos.E x p e r i m e n t s and a case study have then been performed to demonstrate its efficiency in comparison with numerical methods, plain Monte Carlo simulation and other rare-event methods for Monte Carlo simulation. Details of the implementation and the results of these analyses can be found in Chapters 6 and 7.

In this thesis, we have conducted the design, implementation and testing of an alternative method to deal with the rare-event problem. We have provided a tool which is freely available, and is integrated in the CosyVerif platform and has been thoroughly tested. Indeed, this work is not finished and many additional features could be introduced. In the following, I provide a non-exhaustive list of these improvements.

Perspectives

Several perspectives were presented all along this manuscript either concerning the theoretical developments, the implementation or the experiments.

Concerning the theoretical aspects, the method with guaranteed variance reduction detailed in Chapter 4 presents several limitations either on its range of application or on its ease of use:

• In its actual formulation, the model to study can be infinite but its reduced model must be finite. This restriction was introduced in order to be able to use numerical model checking algorithms based on linear algebra on the abstraction. However, there exist methods to compute reachability probabilities in infinite systems like quasibirth-death processes. Using these methods would therefore enable the handling of infinite reduced models and therefore produce better importance sampling.

• Moreover, the models are requested to be Markovian while it is often not the case of real-life systems. As Monte Carlo simulations can deal with non-Markovian models, it will thus be very useful to extend the method to this framework. However, one should ensure that although the initial model is not Markovian, the reduced one is, in order to take advantage of efficient numerical methods on Markovian models.

Conditions on the reduced model should then be adapted to maintain the guaranteed variance reduction.

• Finally, from a pragmatic point of view, designing the reduced model and establishing the proof that it satisfies the guaranteed variance property is in practice a difficult and tedious task. Methods to help the construction and/or the proof of such reduced models are thus strongly needed in order for these methods to be used more widely. One solution to this issue was presented in Chapter 5.A natural perspective is to extend the range of this framework by, for instance, letting processes fork and join when changing zone or by allowing processes to be created or destroyed as it is the case in open systems.

Concerning the implementation of the method detailed in Chapter 6, several technical issues still need to be solved. Initially, the tool Cosmos was designed as a simulator. Numerical model-checking methods have been introduced afterwards to deal with rare events. However, they use a naive representation of the underlying Markov chain while tools dedicated to numerical model checking like Prism or Marcie rely on more involved symbolic methods derived from binary decision diagrams and are thus much more space efficient. For time-unbounded reachability problems, the numerical and simulation parts of the computation are successive. Therefore, the numerical part of the computation can be done by a numerical tool and its results can then be given as input to Cosmos for the simulation part. For time-bounded reachability problems however, the simulation and numerical computation must be performed at the same time and cannot be performed by two different tools. Improving the numerical model checker of Cosmos or developing its interface with an external numerical model checker is thus critical in order to be able to efficiently conduct these calculations. However, both of them will require significant development efforts.

Moreover, experiments presented in Chapter 6 and 7 showed that, even though the methods described in this manuscript can in principle compute probabilities of rare events which were out of reach of classical simulationbased or numerical methods, the design of the reduced model is in practice challenging in order to ensure the guaranteed variance property. Without this property, one can only compute asymptotical confidence intervals that are wider and not conservative. Additional experiments and case studies should be conducted to establish in which domain the method applies the best. Moreover, using the framework described in Chapter 5, additional testing would provide guidelines to decide which kind of generalizations of the framework would be helpful or required to model different types of systems.

Finally, for these methods to reach their full potential, they should be used not only by researchers in computer science but also by other scientists both in academics or in the industry. In order for that to become possible, it implies that much effort should be put into the automation of the critical steps and to the development of a user-friendly interface. A first step has been done in this direction with the development of the CosyVerif platform but much remains to be done. It will require theoretical efforts to design new algorithms, their implementation in efficient and well-documented tools, and also the development of an environment that will help modelers to build systems.

Figure 1 . 1 :Figure 1 . 2 :

 1112 Figure 1.1: Numerical and simulation based computation time for increasing state space on a tandem queue example

Figure 2 . 2 Figure 2 . 2 :

 2222 Figure2.2 shows a realization of a DEDS specifying the tandem queues system. The initial state is (1, 0) and the set of events is {λ, ⇢ 1 ,⇢ 2 }.I t s formal definition is given in next section.

Example 1 Figure 2 . 3 :

 123 Figure 2.3: A simple DTMC

Figure 2 . 4 :

 24 Figure 2.4: DTMC for the tandem queues

Figure 2 . 5 :

 25 Figure 2.5: DTMC with absorbing states

  s ot h es e c o n dc o n d i t i o n holds.

Figure 2 . 7 :

 27 Figure 2.7: Two equivalent CTMCs: a non uniform one and its uniformisation

Figure 2 .

 2 4 also represents the embedded DTMC of the uniform version of the tandem (with uniform rate 1).

Example 2 Figure 2 . 8 :

 228 Figure 2.8: A simple Petri net

C 1 =Figure 2 . 10 :

 1210 Figure 2.10: A simple example of symmetric net

Figure 2 . 11 :

 211 Figure 2.11: SSN representing a tandem queues system with d queues

Figure 2 . 12 : 4 Figure 2 .

 21242 Figure 2.12: SSN representing the distributed database example

Figure 3 . 1 :

 31 Figure 3.1: Simulation time as a function of the probability for fixed relative width on a tandem queues system.

Figure 3 . 2 :Figure 3 . 2

 3232 Figure 3.2: Evolution of trajectories using splitting

λ 1 Variance of W s 0 Figure 3 . 3 :

 1033 Figure 3.3: Evolution of the variance of W s 0 for the tandem queue system

8 λ 0 16 Figure 3 . 5 :

 801635 Figure 3.5: PDF of W s 0 for increasing values of λ 0

Figure 4 . 1 :

 41 Figure 4.1: Schematic representation of the method

Figure 4 . 3 :

 43 Figure 4.3: Reduction of the tandem queues as an SPN and as a DTMC.

  R) otherwiseThis reduced model is shown on the bottom right of Figure4.3. We see that the function f lumps together all states where n 1 + n 2 is the same when n 2 ≥ R. The intuition behind this reduction is to forbid clients in the first queue to enter the second one, as soon as there are R clients in the second queue. This increases the probability of a global overflow. The corresponding SPN is shown on the upper right of Figure4.3.

Figure 4 . 5 :

 45 Figure 4.5: Structural conditions for guaranteed variance.

Corollary 2 Let

 2 C be an enriched BDTMC, C • be a reduction of C by f . Assume there exists a function g : S ⇥ E ! E • such that 8s 2 S, e 7 ! g(s, e)i s an injection which maps each transition of the model to a transition in the reduced model.  C is a relation over the states of C • such that if: 8s 2 S, 8e 2 E, 1. p(s, e)=p • (f (s),g(s, e));2. p(s, e) > 0 ) f (δ(s, e))  C δ • (f (s),g(s, e));3.  C is a coupling relation with target state s + . Then C • is a reduction of C by f with guaranteed variance.

p

  • f (s),g(s, e) -p(s, e) s p(s, e) δ(s, e) f (s) p(s, e) p • f (s),g(s, e) -p(s, e) δ • f (s),g(s, e) p(s, e) <p • f (s),g(s, e) M M • p(s, e) >p • f (s),g(s, e) s p • f (s),g(s, e) p(s, e)p • f (s),g(s, e) δ(s, e) p(s, e)-p • f (s),g(s, e) f (s) p • f (s),g(s, e) δ • f (s),g(s, e) Formally, we define a new enriched DTMC D =( S, E ] Ẽ, δ, p)w h e r e Ẽ is a copy of E. Let h(s)= P e2E max ⇣ p(s, e),p • f (s),g(s, e) ⌘ . For all s 2 S: 8e 2 E, δ(s, e)=δ(s, e) and p(s,e)= 1 h(s) min ⇣ p(s,e),p • f (s),g(s,e) ⌘ , 8e 2 Ẽ, if p • f (s),g(f (s),e) >p(s, e)t h e n δ(s, e)=s, if p • f (s),g(f (s),e)  p(s, e)t h e n δ(s, e)=δ(s, e), p(s, e)= 1 h(s) p(s, e)p • f (s),g(s, e) .

2 .

 2 If e 2 Ẽ, then there are three cases: (a) p(s, e) = 0 : the condition trivially holds. (b) p(s, e)= 1 h(s) p • (f (s),g(s, e))p(s, e) : this case is depicted on the left part of the figure, and we have δ(s, e)=s and δ• (f (s),g(s, e)) = δ • (f (s),g(s, e)). Using hypothesis (1.a), we have:

1 Figure 4 . 6 :

 146 Figure 4.6: Rarity triggered by time bound

Figure 4 . 7 :

 47 Figure 4.7: "Unfolding C u " for the example of the tandem queues, here S |s + stands for S\{s + } Definition 27 (Time bounded Reachability Problem for DTMC)

Figure 4 . 10 :

 410 Figure 4.10: Repartition of Poisson and µ u (s 0 ) Probabilities as a function of discrete time

Figure 4 . 13 :

 413 Figure 4.13: Bounding upper part of Poisson and µ u (s 0 ) Probabilities

Figure 4 . 14 : 4 . 5 . 3

 414453 Figure 4.14: Parallel simulation of trajectories estimating (µ u (s 0 )) u + u=u -

  Figure 5.1: A pattern

Example 7

 7 An example of pattern is depicted on Figure 5.1. Alphabets for this pattern are: Σ f = {a} and Σ b = {b}, thus all forward jumps are labeled by a and all backward jumps are labeled by b. The set of states is {1, 2,...,7}. Zones are delimited by the sequence Z = {1, 4, 5, 7} and zones are:

Figure 5 . 2 :

 52 Figure 5.2: pattern for a database example

  s, (e, i, F )) Thus Condition (1.b) holds. Due to Lemma 8, s  C ∆(s, (e, i)) and thus Condition (1.a) holds.

Figure 6 . 1 :

 61 Figure 6.1: Overview of Cosmos architecture

Figure 6 . 2 :

 62 Figure 6.2: A HASL formula for the tandem global overflow property

  0 (s, ?)) e argmin e2E (EQ[e]) if e = ? then // If e = ? the system is moved to state s -. // The likelihood is set to 0.

Figure 6 . 4 :

 64 Figure 6.4: User formalism and model complying with FML and GrML

Figure 6 .•

 6 Figure 6.6 shows a screen-shot of the graphical editor Coloane displaying a trace of a Cosmos simulation through the CosyVerif platform.• All tools integrated to the CosyVerif platform are launched through small programs called services which specify the list of parameters required by a tool and call the tool with those parameters. Tools allowing different kinds of computation which require different inputs are integrated with several services, which is the case of Cosmos.

Figure 6 . 5 :Figure 6 . 6 :

 6566 Figure 6.5: Hierarchy of formalism in CosyVerif

1 Figure 6 . 7 :

 167 Figure 6.7: HASL formula for the property D1

Figure 6 . 12 :

 612 Figure 6.12: Runtime for a Probability Measure of the Philosophers Model

Figure 6 . 13 :

 613 Figure 6.13: Runtime for a probability measure of the TQS model

Figure 6 . 15 :

 615 Figure 6.15: Tandem Queues with several queues

Figure 6 . 16 :

 616 Figure 6.16: Petri net for the local overflow in tandem queues (R =3,N = 5)

Figure 6 . 17 :

 617 Figure 6.17: Petri net for the tandem queues (R =2,N = 5)

Figure 6 . 18 :

 618 Figure 6.18: The Petri net for the parallel random walk

Figure 6 . 20 :

 620 Figure 6.20: Histogram of the value of trajectory for 10 philosophers

Figure 7 . 1 :

 71 Figure 7.1: The general scheme of the considered three-level signaling cascade; RasGTP serves as input signal and ERKPP as output signal.

Figure 7 . 2 :

 72 Figure 7.2: Transmission of the signal in the signalling cascade

3 Figure 7 . 3 :

 373 Figure 7.3: Petri net pattern for mass action kinetics A + E ⌦ AE ! B + E.

Figure 7 . 4 :

 74 Figure 7.4: A Petri net modelling the three-level signaling cascade given in Figure 7.1; k i are the kinetic constants for mass action kinetics, N the scaling parameter.

p 3 =

 3 P((MEKPP = 0) ^(ERKPP = 0)) U (RafP >L))

  a set of color classes where each C i are pair-wise distinct and are finite set of colors. Each C i can furthermore be strictly ordered.• Domain : P ! C i 1 ,C i 2 ,...,C i k where 80  j  k, i j is the domain function on places which assigns a sequence of color classes to each place.• Domain : T ! C i 1 ,C i 2 ,...,C i k where 80  j  k, i j is the domain function on transitions which assign a sequence of color classes to each transition. This function specifies the domain of each color variable. For each transition a set {x i 1 ,x i 2 ,...,x i k } of variables is defined. Each x i takes value in C i .

	• W -,W + are the input and output arc functions, for all p 2 P and t 2 T , W -(p, t) and W + (p, t) are functions that map valuations of colored variables in Domain(t) to multisets of elements in Domain(p).
	• X is the guard function. It associates to each transition a boolean function taking as input the valuation of color variables.
	Example 3 The symmetric net depicted on Figure 2.10 is an extension of
	the plain Petri net of Figure 2.8. Sets of places and transitions are the same
	as for the plain Petri net. The set of color classes is the singleton {C 1 }. The Domain function maps all places to the color class C 1 and, for each
	transition, assign variable x to take values in C 1 . All the guards are always

  there exists t 0 ≥ t s.t. t 0t 2 I and 8t 00 2 [t, t 0 [,σ ,t 00 |= φ 1 and σ, t 0 |= φ 2

Table 2 .

 2

		Numerical	Statistical
	Result	Exact (up to numerical er-ror)	Probabilistic framing
	Memory	Store transition matrix and a vector of probability	Store one state of the system
	Parallelism Difficult	Free (almost)
	Models Markovian	General distribution
	Determinism Handle non-determinism	Requires fully stochastic models
	Logic	PCTL and PLTL (The sec-ond increase the state space)	PLTL, automata based logic, no nested operator
	Unbounded until Stationary Distribution Rare events Some numerical instability Supported Exact	Require hypothesis on the model Biased by initial distribution Intractable

2: Summary of comparisons between statistical and numerical model checking.

  • • f . This vector µ • is defined as a reachability probabilities vector in an abstraction of the model M i.e. the reduced model M • . The function f maps states of the original model to states of the reduced model. Such a function is called a reduction function.Figure 4.2 illustrates the behavior of the function f .

  4.4: Example of reduced models Remark 6 This condition may be difficult to ensure when building a reduced model. Figure 4.4 illustrates this difficulty. Suppose that ε is a tiny probability and that in order to build the reduced model we decide to lump together states s 1 and s 2 in a new state s • 12 . We need to choose the probability p to go from s • 12 to s • + . One can check that µ(s 0 )=2 ε(1ε) and µ • (s 12 )=p. The definition of reduced models only implies that p>0. The principle that the reduced model should increase the probability to reach s + implies that p>2ε(1ε). Equation (4.2) requires additional constraints on p.I fw eu s e this equation on state s 1 , we have that

  Running example. Figure 4.4.2 describes the Markov chain C u associated with the example. The state space of C is replicated u times. States wheren 1 + n 2 ≥ N (respectively n 1 + n 2 =0) are merged across all the copies into s + (respectively s -). The initial state is ((1, 0),u).O nt h el a s tc o p y ,o n l y transition λ in the states where n 1 + n 2 = N -1 lead to s + , all others lead to state s -.

Remark 12 Note that by construction C u is a BDTMC: observe that the probability to reach s + or s -from any state is equal to 1. Moreover by construction, µ(s, v)=µ v (s) where µ(s, v) is the probability to reach s + in C u and µ v (s) is the probability to reach s + in C in v steps (see

Definition 9)

.

  Using that  C is a coupling relation with target state s + implies that 8v>1, 8(s,s 0 ) 2 S ⇥ S, s  C s 0 ) µ v (s)  µ v (s 0 ),which is the hypothesis of Proposition 8. ⇤ Remark 15 This corollary has exactly the same hypotheses as Corollary 2.

	3.  C is a coupling relation with target state s + ,
	then C • u is a reduction of C u by f with guaranteed variance.
	Proof:
	g(s, e))
	96

Table 4 .

 4

	1: Compared complexities of the four algorithms
	Complexity Space	Algo. 1 Algo. 2 nu 2n p u	Algo. 3 n log u	Algo. 4 2n
	Time
	for the	Θ(ndu) Θ(ndu)	Θ(ndu)	0
	precomputation
	Additional time
	for the	0	Θ(ndu) Θ(ndu log(u)) Θ(ndu 2 )
	simulation

Table 5 .

 5 3 reports the size of the CTMC obtained from the initial model with N databases and the size of the CTMC obtain from the symmetric model. A drastic reduction of the size of the CTMC is observed.

	N Initial model Symmetric model
	20	8,821	690
	40	107,141	2,580
	60	492,961	5,670
	80	1,484,281	9,960
	100	3,519,101	15,450
	120	7,155,421	22,140
	140	13,071,241	30,030
	160	22,064,561	39,120
	Table 5.3: Number of states in the database example before and after
	symmetrization and lumping.

  Scheme of CosyVerif architectureTo achieve these goals, CosyVerif relies on a common file format to exchange any kind of graph-based model between its tools. This file format is structured using the notion of formalism.Aformalism is a meta-model for a formal notation. Tool developers must then describe formalisms accepted by their tools. When a model is created, one chooses the formalism to which this model belongs. The platform CosyVerif matches then formalisms of the models and tools. It provides users with a list of available tools for a given model. To achieve this, a language for formalism has been developed and is called Formalism Markup Language (FML). It describes how a model is structured. All models described by a FML are graph based and are encoded in the Graph Markup Language (GrML). Both FML and GrML rely on XML as file format. Figure6.4 illustrates the relation between FML, GrML formalisms and models.

	Web Client and Interface Eclipse Client and Interface: Coloane Command Line Client Meta-Meta Meta Instance is an instance of Network User Formalism User Model Figure 6.3: FML is an instance of	Virtual Machine Cosmos Service Cosmos Service Cosmos Service 1 Tool 1 Server: Alligator Service 2 Tool 2 GrML complies with is structured by specialises	…

Table 6 . 2

 62 Results on hypothesis testing are reported in Table6.3. Each value is the mean of 100 experiments. The threshold value for the hypothesis is always very close the numerical value in order to increase the number of trajectories performed by tools. Results confirm that hypothesis testing methods are faster than confidence interval based methods. All tools generate a similar number of trajectories. In most cases Cosmos is the fastest, Ymer being up to 1.8t i m e ss l o w e r ,U ppaal2.8 times slower, and Prism 6t i m e ss l o w e r .

	Cosmos (Chow and Robbins)	Prism-ACI
		Cosmos (Chernoff)	Prism-APMC
			Uppaal		Plasma
			Prism-CI
					Time Bound
	Time Cosmos Cosmos Uppaal Prism Prism Prism Plasma
			C-H		CI	ACI	APMC
	20	9	16	59	40	39	81	14
	40	14	24	95	66	72	138	25
	80	12	34	133	65.6	64	213	46
	200	3	38	159	7	6	246	98
	SMC Comparison Conclusion
	While the formalisms for modeling the system are quite different and address
	specific kinds of applications, specifications of properties are somewhat

: Runtime comparison for the TQS TQS experiments with sequential hypothesis testing.

Table 6 . 3

 63

: Runtime comparison for the TQS for Sequential Testing similar.

Table 6 .

 6

	4: Comparison of different importance samplings for computing the
	overall overflow probability on tandem queues
	the reduced model:
	dR 24 0 .10 .45 λ ⇢ 1 = ⇢ 2 = ••• = ⇢ d Number of trajectories 20, 000
	430 .04	0.24	20, 000
	920 .01	0.11	100, 000

  .[START_REF] Amdahl | Validity of the single processor approach to achieving large scale computing capabilities[END_REF] reports results of experiments with

	d	Size of C	Prism Mem. Time	Reduce Model, Prism Size Time Mem. Time S Time N Mem. Cosmos
	1	51	57	0.052	51	2.0	57	7	8	67
	2	1,326	59	0.22	198	1.7	58	11	12	69
	3	23,426	68	0.73	768	1.7	60	17	20	70
	4	316,251	92	1.4	2,976	1.7	75	27	29	72
	5	3,478,761	181	39	11,520	1.7	102	47	51	73
	6	32,468,436	747	580	44,544	2.0	187	101	109	74
	7 264,385,836 5,174	6,635	172,032	4.1	199	341	372	76
	81 .9 • 10 9 36,736 75,940 9 12.6 • 10 9 Out of Mem. 10 75.4 • 10 9 11 418.1 • 10 9	663,552 2,555,904 9,830,400 37,748,736	12 50 217 964	229 279 10,470 1,666 313 51,377 787 284,726 256,165 6,432 1,773 103 11,536 388 51,840 1,491

Table 6 .

 6 5: Time of memory consumption for the computation of global overflow for the tandem queues N = 50,R = 3 and with increasing values of d. The number of samples used by Cosmos is equal to 100, 000. The second column reports the size of the DTMC. Columns 3 and 4 report the memory (in megabytes) and time (in seconds) required by Prism to perform the computation. Column 5 reports the size of the DTMC of the reduced model. Columns 6 and 7 report the time and memory required by Prism to compute the vector µ • . Finally columns 8, 9 and 10 report the time and memory used by Cosmos to perform the simulations using either important sampling with sink state (S) or important sampling with normalization (N). As the memory consumption of these two importance samplings are equal, it is reported only once.

Table 6 .

 6 6: Numerical and statistical results for the computation of global overflow for the tandem queues using importance sampling with sink state.

	dE x p µ(s 0 ) µ • (s • 0 )	I.S. with Sink State
			Mean	CI	Width
	1 10 -52 8.519	8.52	8.519 [8.518, 8.519] 0.001
	2 10 -50 4.251	5.29	4.256 [4.238, 4.273] 0.034
	3 10 -48 1.082	1.35	1.081 [1.077, 1.086] 0.009
	4 10 -47 1.871	2.27	1.866 [1.858, 1.873] 0.014
	5 10 -46 2.474	2.93	2.471 [2.463, 2.480] 0.017
	6 10 -45 2.667	3.09	2.669 [2.661, 2.678] 0.017
	7 10 -44 2.440	2.78	2.440 [2.433, 2.448] 0.015
	8 10 -43 1.948	2.18	1.947 [1.942, 1.953] 0.011
	9 10 -42	1.53	1.383 [1.379, 1.387] 0.007
	10 10 -42	9.76	8.904 [8.882, 8.927] 0.045
	11 10 -41	5.70	5.243 [5.231, 5.256] 0.025

Table 6 .

 6 7: Numerical and statistical results for the computation of global overflow for the tandem queues using importance sampling with normalization.

		Hoeffding	Gaussian
	CI	width	CI	width

Table 6 .

 6

	8: Experimental results for the local overflow in tandem queues R T(s) Size of Cosmos Exp µ • (f (s 0 )) C • C • µ(s 0 ) C.I. T(s) Nb Traj. 12 ⇡ 0 338 11.6 1.48 0.283 2 10 -6 29 ⇡ 0 1530 13 10 -11 29.8 3.81 0.719 66 1.44 6767 17 10 -20 18.7 4.22 0.734 37 500 370 1770 185871 10 -91 10.3 6.63 0.805 N 25 50 100 1000 740 24670 741741 10 -179 324 3.95 4.0 180

Table 6 .

 6 9: Experimental results for the parrallel random walk

	N	size of	Prism Num.	Prism Stat.			Cosmos
	C 15 57 .5 • 10 5 1 61 .1 • 10 7 71 .7 • 10 8 2248 2.93E-12 Time µ(s 0 )T i m e ⇡ 0 0.00113 12 6 1.88E-9 21 127 1.14E-12 No Accepted Path. µ(s 0 ) C.I. Nb Traj. T C • (s) T(s) 1.15E-3 1E-4 1 ⇡0 ⇡0 0 # 18000 0.5 13 53000 1 57 50000 2.8 186 Out of Mem. 145000 7.9 1719 1.86E-15 1.86E-16 µ(s 0 ) Conf. Int. 0.00113 0 1.94E-9 1.89E-10 1.17E-12 1.17E-13 2.92E-12 2.89E-13 82 .0 • 10 9 128000 24 3800 4.7E-15 4.75E-16 93 .8 • 10 10 371000 71 26000 3.12E-18 3.11E-19 10 5.7 • 10 11 11 8.0 • 10 12 321000 228 67000 7.90E-18 7.89E-19
			Ns	n Th	f 1	Wa	f 2	Ea
					n 2	Fo
					e		Ns
			(a) M with N =4		(b) the reduce model M

• Figure 6.19: Petri net for the dining philosophers huge models (with size up to 8 10 12 ) with very small probabilities (8 10 -18 )

Table 6 .

 6 10: Experimental results for the philosophers

	N 3 5 10 7.3 • 10 4 2.358 • 10 -11 size of µ(s 0 ) C 56 5.822 • 10 -4 492 1.590 • 10 -6 15 8.8 • 10 6 3.402 • 10 -16 18 1.4 • 10 8 30 9.4 • 10 12	T (s) numerical 0.007 0.028 3.45 845	size of C • 24 72 396 1152 3.979 • 10 -17 5.868 • 10 -19 µ • (f (s 0 )) µ(s 0 )T ( s ) simulation 5.822 • 10 -4 0.22 5.822 • 10 -4 9.950 • 10 -7 705 1.604 • 10 -6 1400 1.853 • 10 -12 6.006 • 10 -12 4150 6400 1900 1.416 • 10 -19 2.212 • 10 -22 7936 1.566 • 10 -27 1.051 • 10 -38 16000

Table 7 .

 7 Each step follows the pattern A + E ⌦ AE ! B + E and is modeled by a small 1: Development of the state space for increasing N .

	N	number of states	N	number of states
	1	24,065 (4)	6	769,371,342,640 (11)
	2	6,110,643 (6)	7	5,084,605,436,988 (12)
	3	315,647,600 (8)	8	27,124,071,792,125 (13)
	4	6,920,337,880 (9)	9 122,063,174,018,865 (14)
	5 88,125,763,956 (10) 10 478,293,389,221,095 (14)

Table 7 .

 7 2: Computational complexity related to the evaluation of p 1 .75•10 -27 ,5.88•10 -26 ] [3.75•10 -27 ,4.54•10 -25 ] [3.75•10 -27 ,1.57•10 -23 ] 8.18•10 -26 3 [4.34•10 -42 ,1.72•10 -39 ] [4.34•10 -42 ,1.82•10 -38 ] [4.43•10 -42 ,1.87•10 -37 ] 2.56•10 -39 4 [1.54•10 -57 ,8.54•10 -56 ] [1.54•10 -57 ,1.98•10 -55 ] [1.78•10 -57 ,7.05•10 -55 ] Out of Memory 5 [3.97•10 -73 ,2.33•10 -70 ] [3.97•10 -73 ,7.30•10 -70 ] [5.44•10 -73 ,2.24•10 -69 ]

	suffers an

Table 7 .

 7 3: Numerical values associated with p 1

As long as we have an operational semantics of a probability distribution (i.e. we have a probabilistic algorithm whose outputs follow this distribution law), the statistical method can be used to analyze any system using this law.

These experiments have been executed on a MacBook Pro, with processor

2.4 GHz Intel Core 2 Duo.

• A confidence interval for this importance sampling can be computed using Chernoff-Hoeffding bounds.

Proof:

Let σ =(s = u 0 ,...,u l = s + ) be a trajectory starting in s ending in s + . Its value is:

We know that E(W s )=µ(s), then P

.

This implies that µ(s)  µ • f (s) and V(W s )=µ(s)µ • f (s)µ 2 (s).

The variance is finite and as W s 0 takes bounded values, Chernoff-Hoeffding bounds apply and we can compute confidence interval (see 2.3.4). ⇤

Similarly for the importance sampling with sink state the following proposition holds.

Proposition 7

Let C be a BDTMC and C • be a reduction with guaranteed variance by f . The importance sampling based on the matrix P 0 of Definition 26 has the following properties:

• For all s such that µ(s) > 0, W s is a random variable which has value in {0,µ • f (s) }.

• µ(s)  µ • f (s) and V(W s )=µ(s)µ • f (s)µ 2 (s).

• A confidence interval for this importance sampling can be computed using classical method on Bernouilli distribution.

Proof:

Let σ =(s = u 0 ,...,u l = s + ) be a trajectory starting in s ending in s + .

As the trajectory avoids s -, its value is:

We know that E(W s )=µ(s), then P

.

This implies that µ(s)  µ • f (s) and V(W s )=µ(s)µ • f (s)µ 2 (s).

The variance is finite and as W s 0 is a Bernouilli law, it takes only two values and it is possible to compute a confidence interval (see 2.3.5).

as the size of C u is u times bigger than the one of C, the relation between C and C u must be used as much as possible, for the sake of scalability.

As the design of a reduced model relies on the dynamics of the system, we will define it on C.W ed e fi n ear e d u c e dm o d e lC • of C by a reduction function f . We then define as reduced model for C u ,t h eBDTMC C • u by applying Definition 27 to C • . For all 0  v  u, the function f u is an extension of f defined by f u (s, v)=(f (s),v). Moreover Remark 12 applies to distributions (µ • v ) u v=0 where 8s

) is the probability of reaching s • + in C • in v steps. In this context Definition 25 can be reformulated as: Definition 28 (Time Bounded Importance Sampling for DTMC)

Let C be a DTMC, C • be a reduction of C by f and u a positive integer. Then P 0 u is the transition matrix on S u the state space of C u . Let s be a state of S\{s + } and 0 <v u,

P(s, s 0 ). P 0 u is defined by:

1. if µ • v (f (s)) = 0 then for all s 0 2 S u , P 0 u ((s, v),s 0 )=P(s, s 0 ) 2. if µ • v (f (s)) > 0 and h v (s)  1t h e n -I fv>1t h e n8s 0 2 S\{s + },

Remark 13 This importance sampling uses s -as a sink state. The importance sampling with normalization can be defined similarly to Definition 24

Remark 14 Similarly to the unbounded reachability case, suppose h v (s)  1 holds for all s 2 S and v 2{ 1,...,u}, which means that Case 3 of Definition 28 never occurs, C • u is a reduction with guaranteed variance of C u by f and an importance sampling with guaranteed variance is obtained.

As the reduction of C u to C • u is obtained as an unfolding of a reduction of C to C • , the guaranteed variance reduction property (Definition 23) can be rewritten for time-bounded properties as Algorithm 1 Algorithm 2 challenge is to obtain a suitable space-time trade-off. This problem of space complexity is due to the fact that the computation of vectors µ • v is performed iteratively starting from the end (v = 0) whereas the simulation starts from the initial state (v = u) and decreases v at each step.

We propose four methods to implement this space-time trade-off. The methods consist of a precomputation stage and a simulation stage. Their difference lies in the information stored during the first stage and the additional numerical computations during the second stage. In the precomputation, the three first methods compute iteratively the u vectors µ

) for v from 1 to u. The fourth one does not have a precomputation stage. An overview of the four algorithms is shown in Figure 4.8. The respective time and space complexities of the four algorithms are shown in Table 4.1.

First Algorithm

The first algorithm correspond to the naive implementation. It consists in storing all the vectors (µ • v ) u v=1 during the precomputation stage and then proceeding to the simulation without any additional numerical computations as shown in the top left corner of Figure 4.8. The precomputation stage is described in Algorithm 1 where list L is the only memory requirement, and takes Θ(nu) memory space. The precomputation requires u matrix vector multiplications taking Θ(ndu) elementary operations.

Algorithm 1:

Second Algorithm

The second algorithm is described in the top right corner of During the simulation stage, in a state (s, v), with v = ml, the vector µ

stored in L to compute iteratively all vectors µ • l(m-1)+i = P •i (µ • l(m-1) ) for i from 1 to l -1 and stores them in K (see the step computation stage of Algorithm 2). Then it proceeds to l consecutive steps of simulation without additional computations. The integer l is chosen close to p u in order to minimize the space complexity of the algorithm.

Third Algorithm

The third algorithm is shown in the bottom left corner of Figure 4.8. Let l = blog 2 (u)c + 1. In the precomputation stage, it only stores l + 1 vectors in L. More precisely,

• initially using the binary decomposition of u (u = P l i=0 a u,i 2 i ), the list L of l + 1 vectors consists of w i,v = µ • P l j=i a v,j 2 j , for all 1  i  l +1 (see the precomputation step of algorithm 3).

• During the simulation stage in a state (s, v), with the binary decompo-

Let us describe the updating of the list performed by the stepcomputation of Algorithm 3. Let i 0 be the smallest index such that a v,i 0 = 1. Then, for i>i 0 , a v-1,i = a v,i and a v-1,i 0 = 0, and for i<i 0 , a v-1,i = 1. The new list L is then obtained as follows.

Algorithm 2: We are now interested in estimating:

Confidence interval generation

The method described in Section4.4 allows to compute µ u (s)i naDTMC, for all u between u -and u + . Figure 4.11 summarizes how the different algorithms fit together to build the importance sampling. Additional analysis is required to bound the errors made by the different parts of the computation. Each estimation of µ u (s) is subject to a statistical error controlled by a confidence interval I u and a confidence level " u such that P µ u (s) / 2 I u  " u holds. The error coming from the truncation of the sum is controlled, for the Poisson probabilities term, by ↵ and β. It remains to bound µ u (s) on N\{u -,...,u + }. As µ u (s) is increasing with respect to u, a simple bound is µ u -(s) for the lower part and µ 1 (s)=µ(s) for the upper part. These two probabilities can be estimated using previous methods which return confidence intervals whose upper bound are up u -and up 1 with confidence level " 0 uand " 0 1 such that P

Finally the sum between 0 and u -is bounded by ↵up u -and the sum after u + is bounded by ↵up 1

and the confidence interval I is defined by:

When µ(s 0 ) is close to 1, the upper bound is not accurate enough. A workaround is to cut this sum into two parts:

Let γ be a bound on the sum of Poisson probabilities from u ++ to infinity.

Guaranteed Variance for CTMC

To obtain guaranteed variance reduction in the setting of bounded reachability problem for a CTMC, we could require that the embedded DTMC of the

We define a new enriched CTMC D =(S, E ] Ẽ, δ, λ)w h e r e Ẽ is a copy of E. For all s 2 S: 8e 2 E, δ(s, e)=δ(s, e) and λ(s, e)=min(λ(s, e),λ

We can now prove that the embedded DTMC of D • is a reduction with guaranteed variance of the embedded DTMC of D by checking that it fulfills hypotheses of Proposition 8. Let Λ(s) be equal to P

Ẽ λ(s, e 0 ). By construction, the first condition of Proposition 8 holds. For all u 2 N ⇤ ,s2 S and e 2 E ] Ẽ there is two cases:

1. If e 2 E, as δ(s, e)=δ(s, e), the condition holds by hypothesis.

(2) we have:

.15: Parallel simulation of trajectories estimating (µ u (s 0 )) u + u=u -with reuse of trajectories to probabilities of rare events and in that case are not accurately estimated with importance sampling probability distribution computed for µ u + .

Recall that µ u (s 0 ) is an increasing function of u.T h u si fv is a lot smaller than u then µ v (s 0 ) can be a lot smaller than µ u (s 0 ). The likelihood of a trajectory simulated under an importance sampling built for estimating

Thus the probability for a trajectory to reach s + knowing it reaches s + or s -in v steps is smaller than µ v (s 0 )/µ • u + (f (s 0 )) which can be too small. This mean that almost all trajectories will not reach s + in v steps and thus the estimation of µ v (s 0 ) is still subject to the rare event problem.

A more elaborate approach is necessary to prevent this issue. The set of integers {u -,...,u + } can be split into several parts. Let l be an integer smaller than u + . Instead of simulating k trajectories for each u 2 {u -,u -+1,...,u + }, we can perform k • l simulations for each u 2{u -,u -+ l...,u + }. The total number of simulations is unchanged but each µ u (s 0 ) receive contribution from at most k • l trajectories.

Guaranteed Variance Reduction while Reusing Simulation

In the guaranteed reduction setting, some adaptations are required to maintain guaranteed variance reduction while reusing simulation. Consider an importance sampling with sink state build for estimating µ u (s 0 ). When one of its trajectory reaches the state s + in v steps with v  u,thelik elihoodW s 0 of this trajectory is equal to µ

The distribution of W s 0 is still bivaluated when simulations are reused and takes value in {0,µ • u f (s 0 ) }. Therefore, confidence intervals can still be build. Similarly, when an importance sampling with normalization build for estimating µ u (s 0 ) is used, the random variable W s 0 of a trajectory reaching s + in v takes value in [0; µ • u f (s 0 ) ]. Chernoff-Hoeffding bounds still apply and confidence intervals can be computed. The state space S.

Let Part(I) be the set of partitions of I. The set of global states S is the subset of (L⇥{1,...,2⇣ +1}) I ⇥Part(I) of tuples s = ((l i ,z i ) i2I , {I 1 ,...,I u }) such that:

• 8i 2 I, l i 2 z z i that is z i is the index of a zone containing local state l i .

• 81  j  u, 8i, i 0 2 I j ,z i = z i 0 .T h e s u b s e t I j fulfills |I j | = a z i . Given i 2 I there is a unique j such that i 2 I j .T h ei th process is synchronized with a z i -1 other processes which are all in the same zone, these processes being those of I j \{i}. We denote by p i the block containing i:w eh a v ep i = I j .

We also use the redundant notation for state s,( l i ,z i ,p i ) i2I where p i is the block containing i. Depending of the context one notation can be more convenient than the other.

For further use, we introduce two special states.

• s 0 =((1, 1) i2I , {{1}, {2},...,{m}}) so is the initial global state.

• s f =((n, 2⇣ + 1) i2I , {{1}, {2},...,{m}}) so is the final global state.

The partition {I 1 ,...,I t } is the set of equivalence classes of the equivalence relation defined by synchronizations.

The set of events E.

Events in our transition system are pairs in Σ ⇥ I of an event of the pattern automaton and the index of the process performing the event.

The successor function ∆.

A transition in such a system proceeds most of the time in an asynchronous way by applying a local transition of the pattern on a single process. This is true as long as the moving process remains inside a zone. But boundaries of zones, which are zones of length zero, force synchronization between some Disjoining blocks with a forward move z i is even:

Disjoining blocks with a backward move z i is even:

No change of zone, z i is even:

The move is a self-loop None of the previous conditions are satisfied l + i j = l Table 5.

1: Table of transitions for even zones

For a global state s 2 S, an event e 2 Σ and a process i 2 I,

Like for the successor function we use also the notation Ω(s, e, i) as Ω(s, (e, i)).

The transition rate matrix

The transition rate matrix is obtained by combining the Ω function and the f function. For all s =( l j ,l j ,p j ) j2I 2 S, e 2 Σ, i 2 I and F 2P(I)t h e Ω ⇤ (s, (e, i, F )) is defined as follows:

Joining blocks with a forward move z i is odd:

Joining blocks with a backward move z i is odd:

The move is a self-loop None of the previous conditions are satisfied l + i j = l Table 5.2: Table of transitions for odd zones

Similarly the ∆ ⇤ (s, (e, i, F )) is defined as follows:

• Otherwise ∆ ⇤ (s, (e, i, F )) = {(l 0 j ,l 0 j ,p 0 j ) j2I ) 2 ∆(s, (e, i)) s.t.

Then the rate transition matrix is defines by

1 {s 0 =∆ ⇤ (s,(e,i,F ))} Ω ⇤ (s, (e, i, F ))

The following lemma shows some arithmetic property of an instance.

Lemma 1

For a global state s =(l i ,z i ,p i ) i2I reachable from the initial state and a zone indexed by z of multiplicity a z , the following holds:

By induction on the length of a path starting from the initial state: In the initial state 8z>1, #{i, z i ≥ z} =0 and for z =1,a 1 = 1. Let s be a state satisfying the induction hypothesis. For any z, a transition from s satisfies:

• A block is created in the zone of multiplicity a z by moving a process forward. In this case the cardinality of {i, z i ≥ z} is increased by a z .

• A blocks of multiplicity a z disappears by moving a process backward, then the cardinality of {i, z i ≥ z} is decreased by a z .

• Otherwise the cardinality of {i, z i ≥ z} is unchanged.

⇤ Corollary 11

If there exists an execution from the initial state to the final state then m is a multiple of multiplicity a i for any i 2 I.

Symmetries

Definition 36 (Symmetric instance )

In this case we omit the type index for the rate functions and variables.

This definition makes sense because if there is only one type of processes the rate function Ω(s, e, i) is equal to !(l i ,e)(v

SampleExponential(λ 0 (s, e))

7 if Acc 0  Acc then // Add a special transition with the remaining probabilities.

Algorithm 6: Simulation Step

SimulationStep(s, EQ, λ) Result: t, EQ 1 e argmin e2E (EQ[e]) 2 EQ[e] 1

3 t δ(s, e) 4 for e 0 2{s 0 2 E | λ(t, e 0 ) 6 = λ(s, e 0 )}[{e} do // e 0 loops over transitions altered by the last transition.

SampleExponential(λ(s, e 0 ))

EQ[e 0 ] 1 // If e 0 is no longer firable its firing time is set to 1 This is acceptable as in rare event setting Algorithm 6 does not work at all.

State-Space Generation and Numerical Computation

In order to apply the methods described in Chapter 4, the tool has to generate the state space and the transition probability matrix of the reduced model and to perform numerical computation on it. As this part of the tool is only supposed to work with small reduced models, it is not as much optimized as the simulation part.

The workflow for using the importance sampling of Cosmos is in two parts:

1. Cosmos is run on the reduced model and if required with a "reduced" property. The tool generates the state space of the reduced model and performs the required numerical computation for the time-unbounded case. Results of this computation and the probability transition matrix are exported into a file.

Cosmos is run on the initial model. It compiles the reduction function

with the model and imports the file containing the results of numerical analysis or the transition probability matrix to compute the importance sampling.

The state space generator is built on top of the simulator. From a state of the system, all possible transitions are fired and the state obtained after each firing is copied in a stack of states to be explored. Tools dedicated to state-space generation usually rely on methods based on decision diagrams and are much more efficient. database example (on the left) as a function of the size of the color class. The time for building the simulator is also reported. The memory consumption in the building phase is proportional to the time consumption and thus is omitted in the graph for the sake of clarity. The memory consumption in the simulation phase is negligible and thus is also omitted. In both examples, the mean number of tokens of a specific color in a specific place has been evaluated (Property C2 for the client-server). The results of all simulations are consistent, with intersecting confidence intervals on SSN and SPN which confirms the correctness of the SSN simulator implementation.

The two examples show different behaviors. The client-server example features simple firing rules for the transitions. In this case, the expressions on the arcs are simple combinations of the binding variables. Therefore, these firing rules are simulated efficiently and the simulation time in the SSN is only slightly bigger than in the SPN. At the same time, as the SPN size increases drastically with the number of clients, the building time for the SPN simulator becomes intractable. The construction time on the SSN is always negligible. Thus on this example, the use of SSN speeds up the • The number of tokens in the set of places {ERK, ERK RafP, ERKP Phase2, ERKP, ERKP RafP, ERKPP Phase2, ERKPP} is equal to 3N

We also introduce three sequences of places one for each layer of phosphorylation.

Let us remark that a marking of the SPN N is uniquely determined by its values on places in S 1 , S 2 and S 3 . We define a function g such that: for all positive integer m, positive real number p and vector of integers of size k

1 ,d e fi n e db y :

One can see that the g is properly defined and that the sum of the components of u are equal to m. The reduction function f for the two properties is a mapping from the set of states of SPN N to the set of states of the reduced SPN N • .T h i s function takes as input the marking of a sequence of places that uniquely define the state. This sequence can be decompose on the three layers of phosphorylation, that is S 1 for the first layer, S 2 for the second layer and S 3 for the last layer.

Recall that layers are not independent one from the others because proteins of one layer are used to activate the following layer; this can be seen on the P-invariant that contains places of the following layer. The mapping function that we construct preserve these P-invariants.

Roughly, on each layer S i , this function f applies a function of the form g(p i ,m i , -).

Precisely, given a scaling factor N and a scaling factor for each of the three layers of the reduced model, respectively N 1 ,N 2 and N 3 , the reduction function f maps the marking m on the marking m • defined as follow:

One can see that the three P-invariants are preserved in the reduced model by f . We choose N 1 = N 2 = 1 and N 3 = N .

Experimental analysis of the likelihood.

We describe here some technical details of the simulation done for evaluating probability p 1 . Recall the likelihood of a trajectory requires the distribution of the random variable W s 0 (see subsection 4). Proposition 4 ensures that

Values taken by L s 0 are taken by W s 0 when at the end of a successful trajectory, therefore these values are in [µ • n + (f (s)), 1[. We simulate the system for the first formula with N = 2 and a discrete horizon of 615 (615 is the right truncation point given by Fox-Glynn algorithm). The result of the simulation is represented as an histogram shown in Figure 7.6. The total number of trajectories is 69000, 49001 of them are not successful. We observe that most of the successful trajectories end with a value close to 2.10 -35 , and that a few trajectories have a value close to 10 -32 . This is represented by an histogram which is shown as the green part of Figure 7.6 (with a logarithmic scale for the abscissa). We also represent the histogram of the contribution of the trajectories for the estimation of the mean of L s 0 , that is the red part of the figure (with a logarithmic scale for the ordinate). We observe that the contribution to this mean is almost uniform. Thus a trajectory ending with a likelihood close to 10 -32 have a larger impact than one ending with a likelihood close to 10 3 4. This means that an estimator of the mean value of L (s 0 ,u) will underestimate the expectation of L (s 0 ,u) .T o produce a framing of the result, one has to use a very conservative method to avoid underestimating the result.

Conditional maximal signal peak

The network structure of each layer in the signaling cascade presents a cyclic behavior, i.e. phosphorylated proteins, serving as signal for the next layer, can also be dephosphorylated again, which corresponds to a decrease of the signal strength. Thus an interesting property of the signaling cascade is the probability of a further increase of the signal strength under the condition that a certain strength has already been reached. We estimate this quantity for the first layer in the signaling cascade, i.e. RafP, and ask specifically for the probability to reach its maximal strength, 4N : "What is the probability of the concentration of RafP to continue its increase and reach 4N ,w h e n starting in a state where the concentration is for the first time at least L?". This is a special use case of the general pattern introduced in [START_REF] Heiner | Petri nets for systems and synthetic biology[END_REF].

where ⇡ is the distribution over states when satisfying for the first time the state formula RafP ≥ L (previously called a filter).

This formula is parametrized by threshold L and scaling factor N .T h e results for increasing N and L are reported in Table 7.4 (confidence intervals are computed by Chernoff-Hoeffding method). As before, Marcie cannot handle the case N = 3, the bottleneck being here the execution time.

It is clear that p 2 is an increasing function of L. More precisely, experiments point out that p 2 increases approximatively exponentially by at least one magnitude order when L is incremented. However this dependency is less clear than the one of the first property.

The reduced model is the one used for the first property except for the values of the following parameters: here we choose N 1 = 1, N 2 = N and N 3 = 0.

Signal propagation

To demonstrate that the increases of the signals are temporally ordered w.r.t. the layers in the signaling cascade, and by this way proving the traveling of the signals along the layers, we explore the following property: "What is the probability that, given the initial concentrations of RafP, MEKPP and