INTRODUCTION

Assessing the dependability of a critical system generally relies on Boolean models (Fault Trees models, Reliability block diagrams, etc.) for:

• qualitative analysis, such as the determination of the sets of component's failures that cause a critical or undesired system situation (it is defined as the cut set),

• quantitative analysis to evaluate the probabilities of undesired events or RAMS indicators, that can be based on the Sylvester-Poincaré Theorem or Binary Decision Diagrams technique [START_REF] Dutuit | Approximate estimation of system reliability via fault trees[END_REF][START_REF] Ibanez-Llano | A reduction approach to improve the quantification of linked fault trees through binary decision diagrams[END_REF]. This paper explores the use of probabilistic languages introduced by [START_REF] Garg | A Probabilistic Language Formalism for Stochastic Discrete-Event Systems[END_REF]) as a formal framework for qualitative/quantitative analyze of the event sequences. Before to describe the probabilistic languages concept, we will present some reasons that motivate the choice of events sequences in dependability studies.

Evaluation using cut analysis requires a full independence of the events involved in the cut set. However, this assumption cannot be stated in the case of dynamic reparable systems due to the order relationship between events occurrence:

• occurrence of event may depend on previous occurrence of other events; e.g. in the dynamic controlled systems event occurrence may be prohibited by its control according to a previous occurrence, or not, of some other events,

• impact of event sequence on the system failure may be different according to the scheduling of the event occurrence within the sequence; e.g. event e 1 followed by event e 2 leads to an undesired event while e 2 followed by e 1 has no impact. In these cases, the value of the obtained probability, based on cut-set calculus, represents an overstatement of the real probability that is searched, corresponding no longer with the actual needs of dependability studies.

Moreover, the cut-based analysis approaches fit well to the non-reparable systems. However, in dynamic reliability context, it is necessary to take in consideration the changes between the different modes of functioning/failure or when the systems turns back into a functioning state after having passed in a failure state. In this case the systems must be considered like reparable and only dynamic models are able to capture system reconfigurations.

For all these reasons, the cut-set based analysis needs to be enriched by the determination and assessment of event sequences within the cut-set. This evolution justifies the use of state space models to capture the system dynamic and the impact of component's failures and reparation on the system state. Some approaches have been recently developed to determine the critical sequences of events and some basic properties such as minimality and consistency has been proposed for dynamic reparable system [START_REF] Bouissou | A new formalism that combines advantages of fault-trees and Markov models: Boolean logic Driven Markov processes[END_REF][START_REF] Chaux | Towards an unified definition of Minimal Cut Sequences[END_REF]. However, these approaches are based on deterministic language theory and focus on the identification of a set of events sequences but present some limitations and divergences for qualitative and quantitative assessment due to modeling and sequence calculus assumptions.

The paper is organized as follows. Section 2 presents the state of the art and highlights the current Assessment of the critical events sequences of systems by means of probabilistic languages

D. Ionescu !,! , N. Brînzei !,! & J-F. Pétin !,!
1 Université de Lorraine, CRAN, UMR 7039 2, avenue de la forêt de Haye, Vandoeuvre-lès-Nancy, Cedex, 54518, France 2 CNRS, CRAN, UMR 7039 2, avenue de la forêt de Haye,Cedex,54518,France ABSTRACT: In dependability studies of dynamic systems it is important to assess the probability of occurrence for the events sequences which describe the system evolution or which are critical for the mission of the system or for the humans and environment safety. In this paper we use the probabilistic languages framework in order to realize the quantitative assessment and we start by modeling the system as a finite state automaton. This is ulterior transformed in a probabilistic automaton using the embedded discrete time Markov chain. The determination of the languages afferents at each state of the automaton enable to calculate the probability of occurrence for every events sequence that can be subtract from these languages. limitations of existing approaches for sequences assessment. Section 3 introduces the theoretical framework of the probabilistic languages [START_REF] Garg | A Probabilistic Language Formalism for Stochastic Discrete-Event Systems[END_REF] and the used notations. In section 4 is introduced the case study which is used like a support for the rest of the paper. Section 5 develops a theoretical framework based on probabilistic languages for quantitative assessment of events sequences. This proposed approach is illustrated using the case study in the section 6. Finally, section 7 presents the conclusions of this work and identifies some future research directions.

STATE OF THE ART

The existing approaches in quantitative assessment area can be classified in two categories:

• Boolean models where the failure of the system is expressed using the Boolean structure function;

• state space models where the system behavior is represented by states and transitions and his evolution is described by the associated language.

Boolean models

The most important Boolean models are: Event Trees and Fault Trees (classics/dynamics). Event Trees (Papazoglou 1998) are graphical models that can be discretized according to their possible effects or distinction in a series of simple events. In the same time Event Trees are adapted for modeling and assessment of the events sequences for nonreparable systems such as safety or protection systems. Instead, the reparable systems or instrumentation and control systems can not be modeled by Event Trees. Fault Trees are built according to an undesired event that is decomposed into basic events till this decomposition becomes impossible or judged useless [START_REF] Dutuit | Approximate estimation of system reliability via fault trees[END_REF]. These models are efficient for cut-based analysis but have limitations for sequence-based analysis because of the static point of view they consider.

State space models

The most currently used state space models in the context dependability studies are [START_REF] Cassandras | Introduction to Discrete Event System[END_REF]: Finite State Automata and Language theory, Petri Nets and Markov chains. Other models can also be mentioned in the context of this paper. Boolean logic Driven Markov Process (BDMP) developed by [START_REF] Bouissou | A new formalism that combines advantages of fault-trees and Markov models: Boolean logic Driven Markov processes[END_REF] replaces the basic events of a fault tree with Markov chains; this combination introduces dynamic features to cover the order and impact of events occurrence. Events sequences are determined by exploration of the model starting with the initial state of the system [START_REF] Bouissou | Efficient determination of minimal failure scenario for sequential systems[END_REF]. Stochastic Hybrid Automa-ton is a stochastic temporized finite state automaton with embedded differential equations for each mode [START_REF] Perez Castaneda | Stochastic hybrid automata model for dynamic reliability assessment[END_REF]; this model enables to determine sequences of events and their probabilities thanks to Monte-Carlo simulation, but it is not possible to obtain analytical solution.

All these models can be used, according to their deterministic or stochastic features for event sequences analysis [START_REF] Bouissou | A new formalism that combines advantages of fault-trees and Markov models: Boolean logic Driven Markov processes[END_REF][START_REF] Chaux | Towards an unified definition of Minimal Cut Sequences[END_REF] with two kinds of problem solving: determination and probability assessment of the event sequences. For the determination problem, approaches based on the language theory are efficient to compute some event sequences leading to a given state. Calculus is possible only for coherent systems and helps to determine the minimal set of sequences aiming to reconstruct the whole failure language [START_REF] Chaux | Towards an unified definition of Minimal Cut Sequences[END_REF]. This approach does not allow quantitative assessment for the determined events sequences.

For the assessment of sequence probability, some limitations are encountered by the existing approaches:

• state space models such as Markov chains are efficient for determining a state probability. Consequently, a sequence needs to be assimilated to its final state that must be considered as an absorbing state. In this case, probability of this state represents the probability of the language reaching this state.

• the sequence probability calculus is based on the events occurrence probability; in existing approaches, the value of this probability remains the same whatever is the state from which the event occurs. However, in dynamic system, a given event may have different occurrence probability in different system states even if its meaning -failure or reparation of the component c i -remains the same in all sequences. Probabilistic Languages [START_REF] Garg | A Probabilistic Language Formalism for Stochastic Discrete-Event Systems[END_REF], that takes advantages from both languages theory and stochastic process theory, appears to be an interesting way towards a formal framework for the analysis of event critical sequences. The next section presents the theory foundation of the probabilistic languages concept.

PROBABILISTIC LANGUAGES FRAMEWORK

The theory of probabilistic languages was developed by [START_REF] Garg | A Probabilistic Language Formalism for Stochastic Discrete-Event Systems[END_REF] in order to model the stochastic discrete event systems (DES) behavior.

To simplify the formal definition of the probabilistic language, a special event called "termination event", noted ∆ , is used to represent the fact that the state of the system obtained after the occurrence of a sequence is a terminal state. A system state is terminal if it represents a special interest for the study (e.g. system mission achievement, dangerous failure state). Thus the system behavior is given by the set Ω of finite length sequences followed or not by the termination event:

Ω = Σ * ( ∆ + ) = Σ * ∆ ∪ Σ * (1)
In this equation Σ is the set of all events called alphabet, Σ * is the set of all finite events sequences of the alphabet Σ (* is the iteration operation called also Kleene-closure) and represents an empty sequence of events.

To determine the occurrence probability for an events sequence s, should be considered not only the independent occurrence of this sequence but also the occurrence of all sequences that starts with s (all the sequences which have s like prefix). The set of all the sequences having a given prefix sequence s is given by:

< >= {| ∈ Ω} (2)
Definition 1 [START_REF] Garg | A Probabilistic Language Formalism for Stochastic Discrete-Event Systems[END_REF]): Consider the measurable space (Ω, ), where Ω = Σ * (e ∆ + ) and F is the σ-algebra generated by {< > | s ∈ Ω}. Then a probabilistic language (p-language) ℙ is a probability measure on the measurable space (Ω, ).

Thereby the termination probability for a sequence s (the probability that the evolution of a system ends after the occurrence of s) is given by the following expression:

ℙ Δ = ℙ - ℙ !∈Σ , ∀ ∈ Σ * (3)
where ℙ(s) represents the occurrence probability of s and ℙ()

!∈Σ
represents the probability of continuous operation beyond s.

The probabilities of termination corresponding to different sequences are mutually exclusive, so the cumulative probability that the system arrives in a terminal state can be obtained by adding the individual probabilities for all possible sequences:

ℙ Syst Δ = ℙ s Δ !∈! * (4)
Sometimes it is easier to describe a p-language using an associated automaton that can recognize this language. Definition 2: A probabilistic automaton (or pautomaton) over the alphabet (set of events) Σ is defined by the following quintuple:

! = (, , , ℙ, ! ) (5)
where:

• X is a finite set of states;

• is a finite set of events called alphabet;

• : × → is the transition function that associates to each initial state and each event an arrival state;

• ℙ ∶ ×× ⟶ 0, 1 is the probability transition function affecting to each transition a discrete occurrence probability which satisfies the following relation:

ℙ( ! , , ! ) !∈Σ ! ! ∈! ≤ 1, ∀ ! ∈ (6)
• ! represents the initial state.

The evolution of a p-automaton is the following: if the system is in the state ! , the transition e to the state ! is done with the probability ℙ ( ! , , ! ). The probability transition function can be extended to the paths ⊂ X ΣX * in the p-automaton ! (a path is obtaining by concatenating the transitions, where the end state and the initial state of two consecutive transitions coincide). The probability of the paths is defined by the next equation:

! ∈ ∶ ℙ ! = ℙ ℙ ! ! (7)
Each p-automaton defines a p-language and inversely each p-language can be represented by a pautomaton. Definition 3 [START_REF] Garg | A Probabilistic Language Formalism for Stochastic Discrete-Event Systems[END_REF]): The p-automaton ! = (X, Σ, , ℙ, ! ) with:

∀, ∈ Σ * , ∈ Σ ∶ ℙ , , = ! ! ! ! , if t = 0, otherwise . (8)
generates a L p-language.

PRESENTATION OF THE CASE STUDY

In order to present the application of probabilistic languages we start by considering an oven temperature control system (Figure 1). It operates in the following manner: the oven temperature can be controlled by a proportional integral controller (PI) or by an On/Off controller.

Figure 1. Structural diagram of the temperature control system of an oven.

The proportional integral controller (PI) has the role of keeping the temperature at a given target value and is characterized by a failure rate !" and by a probability of failure on demand !"#$% . The role of On/Off controller is to maintain the temperature in a given range and is also characterized by a failure rate !"/!"" and by a probability of failure on demand !"#$%/!"" . When the controller PI fails, the On/Off controller provides the temperature control if this one is not failing on demand. In the same manner if the On/Off controller fails the PI controller will provide the temperature control (if it not fails on demand).

When either of the controllers is failed it will be repaired, the repair process being characterized by a repair rate μ !" respectively μ !"/!"" . This operating the operating mode (obtained by removing the continuous part of the system describing the temperature evolution) of the case study defined in [START_REF] Perez Castaneda | Stochastic hybrid automata model for dynamic reliability assessment[END_REF].

values for the different rates that characterize the system transitions are:

!" = 3.5

• 10 !! ℎ !! , !"/!"" = 2 • 10 !! ℎ !! , !" = 8 • 10 !! ℎ !! , !"/!"" = 10 • 10 !! ℎ !! ,
!"#$% = 0.03, !"#$%/!"" = 0.05. The automaton that describes this system is presented in Figure 2.

PROPOSED APPROACH

The proposed approach is decomposed into four steps.

Step 0: system modelling

The approach based on probabilistic languages theory proposed by Garg et al. needs to give a priori the p-language (the set of sequences and for each sequence s the occurrence probability of sequences sub set that have s like prefix). In other words, the planguage is built from equation (3). This knowledge (the whole set of sequences and their probabilities) is not aware in the dependability studies. In dependability studies, the problem is rather reverse: giving a system, we need to determine the events sequences and, after that, to calculate their occurrence probabilities. Thus, we propose to work with, as initial data, the p-automaton.

Step 1: events probability determination

The p-automaton needs to be provided with event occurrence discrete probability for each transition (as in definition 2). As in dependability studies the probability laws describing the stochastic phenomena are rather continuous, we propose to determine discrete probabilities required by pautomata theory using embedded Discrete Time Markov Chain (DTMC) (Figure 3) in a continous time stochastic process. This embedded DTMC is obtained by considering only the instants of transition firing between states. Using embedded DTMC ensures that the system reaches a stationnary probabilities distribution of states. Consequently, the probability !" of a transition !" that starts from a given state s i to another state s j is determined as the ratio between the rate of this transition and the sum of the rates for all transitions starting from state s i . This probability is given by the following relation:

ℙ !" = !" = !" ( !" ) !!! (9)
Figure 3. Discrete Time Markov Chain representing the p-automaton for the case study.

For example on the case study, the occurrence probabilities for the transitions !" , !! and !" are:

ℙ !" = !" = !"/!"" 1 -!"#$%/!"" !"/!"" + !" ℙ !! = !! = !"/!"" !"#$%/!"" + !" !"#$% !"/!"" + !" ℙ !" = !" = !" 1 -!"#$% !"/!"" + !" !" * , !"/!""! ! 1! ! !",!!!"/!"" * ! ! 2! !", !"/!""! ! 3! !" * , !"/!""! ! 4! ! !" 1 -! !"#$%/!"" ! ! !" 1 -! !"#$% ! ! !" ! ! !" 1 -! !"#$% ! ! ! !"/!"" ! ! !"/!"" ! ! !" ! !"#!"/!"" ! ! ! !"/!"" 1 -! !"#$%/!"" ! ! !" ! !"#$% ! ! !" ! !"#$% + ! !"/!"" ! !"#!"/!"" ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! !" * , !"/!""! ! 1! ! !",!!!"/!"" * ! ! 2! !", !"/!""! ! 3! !" * , !"/!""! ! 4! ! !" ! !" ! ! !" ! !! ! ! !" ! !" ! ! ! !" ! !" ! ! ! !" ! !" ! ! ! !" ! !! ! ! ! !" ! !" ! ! ! !" ! !! ! ! ! !! ! !! ! ! !! ! !! ! !
Note that the sum off all the transitions, which can start from a given state, is equal to one.

Step 2: sublanguages determination

After event probability determination, next step is about probability sequence calculus.

Considering the case study, we focus on the analysis of the sequences that reach the state 3 (dangerous state). Indeed, in this state, the system is completely failed and the temperature oven is uncontrolled.

To use the equation ( 3), which gives the expression of the probability of these sequences, state 3 must be considered as a terminal state. Thus for a sequence, which terminates in state 3, considering first that the occurrence probability of a sequence is equal with the product between transitions probabilities (equation 7) and also considering the events !" , !! and !" , which allowed the system to continue his evolution beyond the state 3, the termination probability for the sequence s it is:

ℙ Δ = ℙ - ℙ = ℙ -ℙ !" + ℙ !! + ℙ !" = ℙ -ℙ ℙ !" + ℙ ℙ !! + ℙ ℙ !" = ℙ -ℙ ℙ !" + ℙ !! + ℙ !" = ℙ -ℙ !" + !! + !" = ℙ -ℙ • 1 = 0
This approach is proposed in the p-languages theory and it was developed in the context of synthesis of control strategy by supervision [START_REF] Wang | A language measure for performance evaluation of discrete event supervisory control systems[END_REF]. The obtained result shows that, in dependability studies, the calculus of events sequences probability is not possible if this p-language is coupled with embedded DTMC without any cautions (if the terminal state of a sequence is not an absorbent state, its probability is always equal to zero).

To avoid this, we propose to determine firstly the whole set of events sequences of the system, that is represented by the language L !"# . This language can be represented as the union of sublanguages associated with the states of the system. A sublanguage associated to a state ! is defined as the set of all events sequences, which lead the system from its initial state to the considered state.

= (10)

To determine the sublanguages we propose to use the theory of rational languages and more precisely the Arden lemma [START_REF] Carton | Langages formels, calculabilité et complexité[END_REF]. Arden Lemma: Being two languages A and B and being the equation:

! = ! + (11)
where ! represents the unknown language: 1. if ∉ , the only solution of the equation is ! = * . 2. if ∈ , the solutions have the form ! = + * where ⊆ Σ * . The lemma is mainly used in the case where ∉ and the language ! = * is the unique solution.

For each state ! of the system the equation ( 11) can be written, considering the sequences starting from all the other states ! ≠ ! and arriving in the state ! by only one transition. The set of n equations ( 11) (where n is the number of system states) allow to obtain all the sublanguages ! .

Having a system represented by a p-automaton and using this approach we can determine all the events sequences without model exploring.

Step 3: sequences probability calculation

Each events sequence of the system = !" !" … !!! ! that starts from the initial state ! and arrives in the state ! can be extracted from the language !"# otained in the previous step. Using the probability for each transition of the p-automaton obtained in step 1, we propose to use the following equation in order to obtain the termination probability for the sequence = !" !" … !!! ! :

ℙ ! = ℙ( !" ) ! !" ∈! , ∀ ∈ * (12)
The meaning of this equation is that the probability of a events sequence is equal with the product of the probabilities of all of its events e ij .

Figure 4 summerizes the different steps of our proposed approach to formally determine the whole set of events sequences and to assess their probability using the probabilistic languages.

APPLICATION ON THE CASE STUDY

Application of the proposed approach

Step 0: Figure 3 shows the embedded Discrete Time Markov Chain representing the p-automaton of the case study.

Step 1: The transition probabilities !" are calculated using the equation ( 9).

Step 2: The sublanguages associated to each state represent the sub set of the sequences that lead the system from the initial state 1 to one of his states. Considering each state of the automaton as terminal state (not absorbing) we obtain the next equations:

! = ! !" + ! !" (13) ! = ! !" + ! !! + ! !" (14) ! = ! !" + ! !" + ! !! + ! !"
(15) ! = ! !" (16) For example the first equation ( 13) denote the sublanguage associate to state ! as being the all events sequences arriving in this state by only one transition: from state ! (all the events sequences previously terminated in ! : ! ) followed by the transition !" or from state ! (all the events sequences previously terminated in ! : ! ) followed by the transition !" (according to Figure 3). Using the Arden lemma, the equations (13-16) give the next expressions for the sublanguages:

• to state 1:

! = !" + !" !! + !" !" * !" !! + !" !! + !" !" * !" * !" !! + !" !" * !" !" !" + !" !! + !" !" * !" + !" * !" * !" + !! + !" !" * !" !" * (17)
• to state 2:

! = ! !" + !" !! + !" !" * !" !! + !" !! + !" !" * !" * (18)
• to state 3:

! = ! !" + ! !" !! + !" !" * (19)
• to state 4:

! = ! !" (20)
Step 3: Having the sublanguage defined for each state of the considered system we can obtain firstly the probability for a sublanguage and secondly the probabilities for events sequences belonging to this sublanguage. To illustrate the sequence probabilities assessment, the results obtained for two sublanguages ( ! and ! ) are presented in this paper.

For the simplification we will use the following notations:

!" = ; !" = 1 -; !" = ! ; !" = ! ; !! = 1 -! -! ; !" = ! ; !" = ! ; !! = 1 -! -! ; !" = ! ; !" = 1 -! .
Probabilities assessment for the sublanguage L ! Using the expression ( 17) we can calculate the probabilities of different iterations that compose the regular expression of the sublanguage ! and, after that, we obtain the final analytic expression:

ℙ ! = ℙ ! = !∈! * ! ! ! ! ! ! ! ! ! ! !! ! ! ! !! ! ! ! = 1 (21)
This result, the probability of the sublanguage ℙ ! , is compliant with the expected one: indeed, for the p-automaton of the Figure 3, the probability of all sequences starting from state 1 and reaching back to state 1 must be obviously equal to one. Nevertheless, the approach is able to provide the probability of all individual sequences reaching to this state. To illustrate this fact, the occurrence probability for some particular sequences, extracted from ! and that lead the system in the state 1, are obtained doing the numerical calculus. Their values are presented in Table 1. Firstly we can observe that the sum of probabilities of considered events sequences is approximately equal with 1 and this corresponds to analytical solution of the sublanguage probability (eq. 21). Secondly we can identify the most relevant events sequences (by the greatest values of their probability).

Probabilities assessment for the sublanguage !

The sublanguage ! describes infinity sequences which bring the system in the failure state 3 from the initial state 1:

ℙ ! = ! ! !! ! !!! ! ! ! ! ! !! ! ! ! !! ! ! ! ! ! (22)
Calculating the numerical values for some sequences we obtain the values from Table 2. The state 3 represents the dangerous state because the oven temperature is uncontrolled and thus we can

Step 0. Modeling the system as a finite states automaton (FSA)

Step 1. Obtaining the p-automaton using embedded discrete time Markov chain (DTMC) technique p-automaton ≡ DTMC

ℙ !" = !" = !" !" !!!
Step 2. Determine the sublanguages associated to a state ! (rules of Arden)

Step 3. Calculate the probability of one or more sequences s

∈ L ! ℙ ! = ℙ( !" ) ! !" ∈!
identify the most critical events sequences (with the greatest values of their probability) of the system.

ℙ ! = ℙ ! ≅ 0.0037 ! ! ______________________________________________ 6.

Analytic validation of the results

In order to check the validity of the results (sequences probabilities), we propose the next approach: 1. To determine the stationary distribution of states probabilities for embedded DTMC [π ! π ! π ! π ! ] by using the equations:

= • and • 1 = 1 (23)
where the vector represents the state probability distribution, M is the matrix of transitions probabilities and 1 is a vector with all elements 1. Using the equations ( 23) we determined analytic form for the 2. To probabilities for different sublanguages, obtained by the equations (17-20), the following relation is to be validated:

ℙ ! • ℙ ! !∈! ! ! ! = ℙ ! • ℙ !→! = ℙ ! (24)
It means that the probability to reach state ! is equal to the product of the probability of the initial state ! and the probability of the sublanguage from ! to ! .

• for ! :

! • ℙ ! = ! • for ! : ! • ℙ ! = ! • for ! : ! • ℙ ! = ! • for ! : ! • ℙ ! = ! 7 CONCLUSIONS
In this paper, we presented the use of the theory of probabilistic language for the assessment of the occurrence probability of events sequences that describe the system evolutions. In order to be able to apply this theory, for a system modelled by a finite state automaton, we proposed a 3 steps approach. The use of the embedded Discrete Time Markov Chain in a continuous stochastic process for determining the events probability makes assumption that the system is in a stationary state characterizing by stationary distribution probabilities over its states. But the embedded DTMC is not limited to Continuous Time Markov Chain; a DTMC can also be defined from semi-Markov or under some hypothesis from more generally stochastic processes. Another advantage to use the DTMC to obtain the events probability is that the probability of an event is not the same during the system evolution, but can depends on the state where it occurs (in other words the same event can be characterized by different occurrence probabilities). The use of the Arden lemma permits to formally determine the whole set of events sequences, without model exploring. Finally, the probability occurrence for relevant or critical events sequences and for a sublanguage is determined.

The approach is applied on a control system temperature. This case study shows the ability of the proposed approach to take into account together stochastic phenomena characterized by continuous laws of probability (such as failures and reparations processes) and phenomena characterized by discrete laws of probability (such as failures on demand).

This work opens several ways of research in our future activity. First of all, it will be interesting to propose an approach that allows calculating the probabilities of events sequences in a transient system state before reaching the stationary distribution. Secondly, we are interested to apply this approach in more complex systems including components ageing with the more generally underlying stochastic continuous process. Also, an extension of the operations provided by the p-languages theory should be useful for taking in account different types of systems (non-deterministic systems, reconfigurable systems, systems composed of several sub-systems) in dependability studies.

Figure 2 .

 2 Figure 2. Continuous Time Markov Chain of the case study.

Figure 4 .

 4 Figure 4. Approach proposed for the probabilistic assessment of events sequences.

Table 1 .

 1 Occurrence probabilities for few sequences that lead the system in the state 1. = e 12 e 23 e 33 e 32 e 21 4.9961e-06 s 4 = e 12 e 22 e 23 e 32 e 23 e 32 e 21 4.8080e-10 s 5 = e 12 e 23 e 34 e 43 e 32 e 21 1.8331e-08 s 6 = e 12 e 22 e 23 e 33 e 34 e 43 e 32 e 21 2.2602e-11 s 7 = e 12 e 23 e 34 e 41 1.0233e-04 s 8 = e 13 e 34 e 41 0.0215 s 9 = e 13 e 33 e 34 e 41 8.8586e-04 s 10 = e 13 e 34 e 43 e 34 e 41 3.2502e-06 s 11 = e 13 e 32 e 21 0.0256 s 12 = e 13 e 33 e 32 e 22 e 23 e 32 e 21

	______________________________________________
	Sequence !	Sequence probability ℙ !
	______________________________________________
	s 1 = e 12 e 21	0.9213
	s 2 = e 12 e 22 e 21	0.0276
	s 3 4.1624e-09 ______________________________________________
		ℙ ! =	! !	ℙ ! ≅ 1
	______________________________________________

Table 2 .

 2 Occurrence probabilities for few sequences that lead the system in the state 3. = e 12 e 22 e 23 e 32 e 23 e 33 = e 13 e 33 e 34 e 43 e 32 e 23

	______________________________________________
	Sequence !	Sequence probability ℙ !
	______________________________________________
	s 1 = e 12 e 23	2.3744e-04
	s 2 = e 12 e 22 e 21 e 13	0.0014
	s 3 = e 13 e 33	0.0021
	s 4 3.8620e-11
	s 5 = e 12 e	1.4724e-09
	s 6 4.0899e-11 ______________________________________________


N Brînzei

G Deleuze

N Villaume

J F Pétin

Common cause failures modelling by means of coloured Petri nets for dependability assessment of a control system of nuclear power plant

This paper presents an approach to assess the effects of common cause failures (CCF) on dependability of digital systems. Independent failures of system components and partial or lethal shocks are considered in a global CCF model, the Atwood model. The Coloured Petri nets (CPN) are used to modelize the digital system and the common cause failures. Based on the CPN, the parameters of Atwood model are estimated analytically and by Monte-Carlo simulation. Thereafter, the Atwood model of CCF is modified in order to represent the dominant failures propagation on some system components in the case of partial shocks. The assessment of system dependability, in the presence of CCF failures, becomes possible. This approach is applied to a representative instrumentation and control system of a nuclear power plant. The system is large with a high level of redundancy.

INTRODUCTION

Digital Instrumentation and Control systems (I&C) systems have a major role in the control and protection of nuclear power plants. These systems are characterized by a large size, a high level of redundancy and a complex logic of vote. Although digital systems and their components are more reliable than the analog systems they replace, some characteristics raise specific issues on the modelling and assessment of common cause failures (CCF). A CCF can occur in operational or on demand modes and affect groups of identical or similar redundant components having the same function and operating under comparable conditions.

The β-factor model is the most widely used model for taking into account CCF within all types of systems, like nuclear power plants ( US-NRC 1987) and, more generally, in the field of power system [START_REF] Bricman-Rejc | Estimating common-cause failures parameters within power system reliability analysis[END_REF]. It involves the failure of whole set of components when a common cause event occurs. This model is used when the system is composed of only a few components. However when the system is composed of dozens of identical or similar components, the assumption of failure of whole set of components, when a CCF occurs, is very conservative.

The concepts of partial and lethal shocks are very well adapted to represent the potential effects of stress factors on electronic hardware. In the next section we introduce the Atwood model of CCF that takes into account independent failures of components and CCF failures due to shocks that affect all or only some components. For dependability assessment, the CCF model must be integrated within a system model. Also, the system model must integrate the dynamic behaviour of the digital I&C systems. The Markov chains or Petri nets are such types of models and the β-factor model has been integrated in Markov chains [START_REF] Lilleheier | Analysis of common cause failures in complex safety instrumented systems[END_REF] and in the classical Petri Nets [START_REF] Signoret | Make your petri nets understandable: Reliability block diagrams driven petri nets[END_REF]. The main drawback of these models is the combinatorial explosion of their size when the modelled system is large. To overcome this drawback, we propose to use the Coloured Petri Nets (CPN) for I&C systems modelling. The section three of this paper develops a CPN model for an I&C system and, also, for the CCF Atwood model. In the same section, firstly, we propose two approaches for parameters estimation of Atwood model and, secondly, we modify this model to represent asymmetry in CCF propagation oriented on some types of components. If for I&C modelled system only the hardware architecture is considered, the oriented propagation of CCF on this architecture takes into account the software and human aspects of functional safety analysis. Section four presents the probabilistic assessment of dependability indicators (PFD, MTTF) or of the impact of the oriented CCF. Finally, some conclusions and future outlook are presented.

MODELLING OF COMMON CAUSES FAILURES (CCF)

The Atwood model

The model introduced by Atwood (1980) considers that the system components are subject to two types of failures: independent failures and shock failures. Two kinds of shock failures are defined: lethal shocks and partial (or non-lethal) shocks. In a large redundant systems with N components, a shock is assumed to be non-lethal when it affects k components among N with 1 ≤ k ≤ N . A shock is lethal when it affects all components. In the case of a non-lethal shock, only the failure of some components is considered.

Each component has then a conditional probability p of failure. Individual failures, non-lethal and lethal shocks are assumed to follow independent processes. The occurrence frequencies of shocks (denoted µ for non-lethal shocks and ω for lethal shocks) are assumed to be constant.

The failure rate of a specific component in a group of N elements, due to an independent failure or to a non-lethal shock is:

λ N 1 = λ i + µ.p.(1 -p) N -1 (1)
The failure rate of a group of k components from N with 1 ≤ k ≤ N due to a non-lethal shock is:

λ (N ) k = µ.p k .(1 -p) N -k (2)
The failure rate of N components due to a nonlethal and lethal shock is:

λ N N = µ.p N + ω (3)
For a specific component in a group of N components, the total failure rate is given by:

λ T OT = λ IN D + ω + µ. N k=1 C k-1 N -1 p k .(1 -p) N -k (4)
The Atwood model is considered representative for the phenomena leading to multiple failures in the large digital I&C systems. It allows to represent CCF affecting only a part of the all components of the system. However, it introduces three parameters to estimate p, µ and ω. The results will be sensitive, of course, to the values used, usually given by expert opinion or by using default values. The default values usually used for these parameters are the following:

• α = µ λ T OT = 0.405: the rate of non-lethal shocks. • p = 0.2 ou 0.33 ou 0.5 : conditional probability of component failure in a non-lethal shock.

• β letal = ω λ T OT = 5.10 -3 : the rate of lethal shocks. Later, in this work, we propose to determine the values of these parameters by two approaches: an analytical one, or a second approach based on results of Monte-Carlo simulations.

The Coloured Petri nets (CPN) model

In order to asses the impact of CCF in the dependability study of digital I&C systems, it is necessary to have a model of this studied system that is able to take into account the system dynamics. A widely used model in the dependability studies of dynamic systems is represented by the Petri nets [START_REF] Signoret | Make your petri nets understandable: Reliability block diagrams driven petri nets[END_REF]. Since I&C systems are complex and large, we choose to use the coloured Petri nets (CPN) type. CPN [START_REF] Jensen | Coloured Petri nets: modeling and validation of concurrent systems[END_REF][START_REF] Jensen | Coloured Petri Nets: Basic Concepts, Analysis Methods and Practical Use[END_REF]) is a discrete-event modelling language combining the capabilities of Petri nets with the capabilities of a highlevel programming language. The main difference between a classical Petri Net and a CPN is that the CPN tokens can have different colours representing data types (e.g. Boolean, integer or more complex data structure). The formal definition of CPN is as follows.

A Coloured Petri Net is a 9-uplet CP N = (P, T, A, Σ, V, C, G, E, I), where:

1. P is a finite set of places.

2. T is a finite set of transitions, P ∩ T = ∅.

3. A ⊆ P × T ∪ T × P is a set of directed arcs.
4. Σ is a finite set of non-empty colour sets.

5. V is a finite set of typed variables such that Type[ν] ∈ Σ for all variables ν ∈ V .

6. C : P → Σ is a colour set function that assigns a colour set to each place.

7. G : T → EXP R V is a guard function that assigns a guard condition to each transition t such that Type[G(t)] = Bool, bool standing for Boolean data type.

8. E : A → EXP R V is an arc expression function that assigns an arc expression to each arc a such that T ype[E(a)] = T ype[C(p)], where p is the place connected to the arc a. Individual CPN models can be hierarchically related to each other in a formal way, i.e. with a welldefined semantics. CPN model hierarchy is realized through substitution transitions. The idea is to associate a transition to a more complex CPN (a module), which gives a more precise and detailed description of the activity represented by the substitution transition (represented by a double rectangle, e.g. in figure 2). The places connected to a substitution transition transmit a given marking from a high level (level of substitution transition) to a low level (level of module) and vice versa. CPN concept of hierarchy allows us to propose a modular modelling approach for a complex systems, based on generic modules that can be instantiated as often as needed.

Additionally, the probabilistic dependability assessment requires the time evolution of the system. For this, the CPN must take into account the time aspect. In a timed CPN [START_REF] Jensen | Coloured Petri nets: modeling and validation of concurrent systems[END_REF][START_REF] Jensen | Coloured Petri Nets: Basic Concepts, Analysis Methods and Practical Use[END_REF]), the time is given by a global clock. In addition to their colour, the tokens contain a time value, also called a time stamp. When a transition is enabled, it is fired and changes the time stamps of tokens which are deposited in its output places. In these places, the tokens remain frozen and can not be used to enable other transitions until the current model time (given by the global clock) is smaller than their time stamps. As soon as the time stamp of the tokens is greater than or equal to the current time model, these tokens can enable other transitions which are instantly fired. In other words, the time stamp describes the earliest model time from which a token can be used.

The following drawbacks can be cited: the low readability (which can be put into perspective by using hierarchical CPN) and the difficulty in verifying a Petri net model. This last drawback can be offset by using a verification of CPN properties. This properties verification is supported by the state space method. The basic idea underlying the state space method is to compute all reachable states and state changes of the CPN model and to represent them as a directed graph, where nodes represent states and arcs represent occurring events. From a constructed state space, it is possible to answer a large set of questions concerning the behaviour of the system, such as absence of deadlocks, a possibility to be able to reach a given state. We have used this formal verification of a CPN to study and validate some specific safety properties of an I&C systems [START_REF] Pinna | Deterministic and stochastic dependability analysis of industrial systems using Coloured Petri Nets approach[END_REF]), but a Monte-Carlo simulation can be also realized without this type of verification. In this paper, we propose to integrate the Atwood model into a CPN model of an I&C system. The obtained model allows the assessment of the CCF impact on the system dependability. In order to realize this assessment, in the next section, we introduce a case study coming from a nuclear power plant.

CASE-STUDY PRESENTATION AND ITS MODELLING

The studied system is a digital protection system. It is a part of the defence in depth of a nuclear power plant.

System architecture

This protection I&C system contains four divisions, which are strictly identical, see figure 1. These divisions are physically separated. Each division is composed of five "Acquisition & Processing Units" (APU). The APU 0, 1 and 2 represent the subsystem A (SSA). The APU 3 and 4 represent the subsystem B (SSB). The gathering of APU into subsystems takes into account the allocation of control functions of the I&C system. For the same control function, there is one implementation in an APU of SSA and one implementation in an APU of SSB with different inputs and treatment and their outputs must be identical in the normal operating mode. Different electronic boards C1 et C2 are included in each APU. Each APU contains one C1 board. The APU 0 and 1 contain four C2 boards, and the APU 3, 4 and 5 contain three C2 boards. These electronic boards are used for reception, treatment and emission of signals.

A second type of partition is defined: the groups of APU (GAPU). A group of APU contains all of the APU i (i [0,4]) of the four divisions.

Modelling assumptions

Assumptions for the electronic boards

modelling A constant failure rate is considered for boards.

The C2 boards of an APU are considered as a series system.

The board failures can be detected by a set of selftests. When a failure is detected by a self-test (SA), Figure 1: Architecture of the case study I&C system for a nuclear power plant the detection time is considered null. When a failure is not detected by a self-test (NSA), then it is detected offline during a periodical-test. For a given division, these periodical-tests take place at each 18 months. So every quarter of this period, a division is tested during the periodic tests. After the periodic tests, the failed boards are repaired.

According to the supplier of electronic boards, their coverage rate of self-tests (t c ) is 100% , but this value seems to be ambitious. We add to the model the

Assumptions for global system modelling

The hazardous event is represented by the failure of the protection I&C system.

The occurrence of this hazardous event is based on the voting logic of the APU:

• An APU fails when a card C1 or C2 is failed.

• A group of APU (GAPU) fails when 3 out of 4 APU is failed (3oo4).

• A subsystem (SSA or SSB) fails when a GAPU is failed.

• The I&C system fails when a subsystem is failed (1oo2) or when two subsystems are failed (2oo2).

We assume that the mission time of the protection I&C system is ten years. Indeed, this system is retrofitted only during the decennial maintenance operations of the nuclear plant. System unavailability can occur during the ten years. The system changes from unavailable to available state without being as good as new as it is not fully retrofitted on this period of ten years. Some electronic boards may be still failed when the system becomes available.

CPN modelling

We propose to use a modular approach for system modelling. Thus, the CPN model (shown in figure 2) is composed by the following modules:

• CCF generating (blue box)

• System representation (red box)

• State system description (green box) The firing of init temps transition allows to determine the occurrence time of the non-lethal shock using the function floor(exponential(!mu)+0.5) and, in the same time, to specify if it is detected or not by a self-test. This is done using the function detect(). This function draws a random value using an uniform distribution law of probabilities in the interval [0, 1]. If the value is lower than coverage rate of self-tests (t c ), the non-lethal shock is detected. In this case the function return the value 1, otherwise 0. The firing of the transition dcc assigns the occurrence time of CCF and the variable of failure detection at each token of the system that is shock sensitive. The transition no def dcc allows to remove the tokens representing boards that are not shock sensitive. The transition new dcc allows to generate the next occurrence The Atwood model introduces three parameters to estimate: p, µ and ω. Two approaches, analytic or by simulation, can be used for doing their estimation.

Analytical approach for estimation of CCF model parameters. From the expressions of α, β letal and λ T OT , we obtain the following equations system:

(1 -α N k=1 p k (1 -p) N -k )µ + αω = αλ IN D (5) (-β N k=1 p k (1 -p) N -k )µ + (1 -β)ω = βλ IN D (6)
The value of conditional probability p is considered known (e.g. at classical default values 0.2, 0.33 or 0.5). The independent failure rate for an electronic board is λ IN D = 2.35 10 -6 h -1 . The equations ( 5) and ( 6) give the values of occurrence frequencies of nonlethal shocks µ and of lethal shocks ω. The obtained results are presented in Table 1.

Table 1: The values of occurrence frequencies µ and ω for nonlethal and lethal shocks as a function of p using analytical approach p µ in h -1 ω in h -1 0.2 9.52 10 -7 1.18 10 -8 0.33 9.52 10 -7 1.18 10 -8 0.5 9.52 10 -7 1.18 10 -8

Monte-Carlo simulation approach for estimation of CCF model parameters. We introduce the ratio between the occurrence rate of independent failures λ IN D and the total failure rate λ T OT of a component (an electronic board):

γ = λ IN D λ T OT = 1 -α. N k=1 C k-1 N -1 p k .(1 -p) N -k -β letal (7) γ = 1 -α.p -β letal (8)
We introduce also E i that represents the number of independent failure events realized in 10 years of simulations. This number is obtained by simulating only the independent failures in Petri net model and their occurrences are counted. It allows to obtain from the equation 9, the average independent failure rate λ IN D for a board in the system. The different rates can be estimated from the following equations:

λ IN D = E i 10 ans * N (9) µ = α.λ IN D γ (10) ω = β letal .λ IN D γ (11)
The expected value of E i is equal to 8.28 and the value obtained for independent failures rate is:

λ IN D = 2, 36 E -6 h -1 .
The results obtained by applying this approach are presented in Table 2.

The occurrence frequencies µ and ω for non-lethal and lethal shocks obtained by applying these approaches (analytical and simulation approaches) are almost identical. This result allows, firstly, to validate the proposed CPN modelling approach for the Atwood shock model. Secondly, the Monte-Carlo simulation approach could be used to revisit some assumptions supporting the Binomial Failure Rate model underlying the Atwood shock model, especially in the case of digital I&C systems including software.

Table 2: The values of occurrence frequencies µ and ω for nonlethal and lethal shocks as a function of p using Monte-Carlo simulation approach p γ λ T OT µ ω 0.2 0.995 2.37 10 -6 9.62 10 -7 1.18 10 -8 0.33 0.995 2.37 10 -6 9.62 10 -7 1.18 10 -8 0.5 0.995 2.37 10 -6 9.62 10 -7 1.18 10 -8

Introduction of asymmetry in CCF propagation.

For a realistic modelling of common causes failures (CCF) and of their impact on the systems dependability, it is interesting to assume that some nonlethal shocks predominantly affect some k components among the all N components of the system, with 1 ≤ k ≤ N (e.g. only some types of electronic boards belonging to the I&C systems). We propose to modify the generation of non-lethal CCF for simulating the asymmetries in their propagation, in other words considering "oriented" CCF.

In a system with N components, a non-lethal shock affects the boards with a conditional probability p. The expected number of affected components is N * p. The set of N components can be divided in two sub-sets A (SSA) and B (SSB), containing respectively a et b components, such as a + b = N . Let be x (respectively y) the probability that a component of SSA (respectively SSB) is affected by a non-lethal shock. From reasoning on mathematical expectation, we have:

ax + by = N p (12)
To determine x and y we choose how a CCF will affect the SSA (p 1 ) and SSB (p 2 ), such as p 1 + p 2 = 1. Thus, we obtain:

x = N.p.p 1 a (13)
y = N.p.p 2 b ( 14
)
The solutions of these equations, x and y, are the probabilities of non-lethal shock for the CPN tokens that represent respectively the SSA and SSB components.

System modelling by CPN

The hierarchical and modular aspects of Coloured Petri nets are exploited to develop the CPN model of I&C system. Thus, each of its divisions is modelled by means of a substitution transition at the high level of the system, in the Figure 2. A division includes five APU, each of them includes the correct number of electronic boards C1 and C2. The CCF are transmitted to the APU and to the boards by means of CPN places (socket places). The state of an APU is determined by the state of its boards (available or unavailable). Once the state of a board changes, the new state of APU is sent to the specification that determines the state of the whole I&C system. An electronic board represented in the Figure 5 has three possible tangibles states: operational ("Marche" place), failed and non-detected ("detection" place) and in reparation ("reparation" place). When the board is operational, it can change its state if one of the following four events occurs:

• independent failure detected online by a self-test (modelled by the purple Petri sub-net on the right side of the model)

• independent failure detected offline in periodical-tests (modelled by the purple sub-Petri net on the left side of the model)

• non-lethal CCF detected online and lethal CCF (modelled by the tokens received by "AR DCC" place from the CCF models of the Figures 3 and4)

• non-lethal CCF detected offline in periodicaltests (modelled also by the tokens received by "AR DCC" place from the CCF model of the Figure 3)

If the failures are detected during the periodic tests, the "rep" transition is fired and the board changes its state to the reparation state. If the failures are detected online, the board changes its state immediately from the operational to the repair state. The repair time is calculated using the function floor(exponential(1/!MTTR)+1.0). As soon as the board is repaired, its state changes immediately to the operational state. The next occurrence times of the independent failures detected online and offline are also calculated. This CPN model of electronic boards are generic. For all boards, only the numerical values of parameters (failure and repair rates) are different.

System state modelling by CPN

Using the information about the state (available/unavailable) of the APU i (i ∈ [0, 4] of the division j (j ∈ [1, 4])), it is possible to determine the state of the I&C system during the Monte-Carlo simulation using the CPN of Figure 6. The system state is represented by a token whose colour is composed of five booleans, each of them representing the state of one APU. The different configurations on the transitions' guards allow to define the conditions of availability/unavailability of the system.

The entire CPN model obtained for the I&C system has 685 transitions and 504 places. Even if the model size is large enough, the use of hierarchy and colours concepts have resulted in a modular and readable model obtained through the instantiation of generic templates. We can note, that an equivalent classical Petri net model for the same I&C system should have Since the I&C system is a protection system (its mission is to protect the nuclear power plant), we are interested to asses only the Probability of Failure on Demand (PFD). This is in accordance with the IEC-61508 (2010) standard and its declination IEC-61513 (2011) that address the functional safety of digital systems. We define two kinds of PFD:

• the real PFD that represents the I&C system unavailability starting with the moment where the electronic boards are not anymore in their operational state;

• the visible PFD that represents the system unavailability only starting with the moment where the boards failure is detected. The time between the occurrence of non-detected failures and their detection by a periodical-test is not considered.

The definition of the two kinds of PFD allows to estimate the difference between the real state of the system and the state observed by plant operators during operation.

The PFD is estimate using Monte-Carlo simulation by the following equation:

P F D = System unavailability time M ission time (15)
The results presented in the Table 3 are obtained after 10,000 histories during 10 years. The 95% confidence interval for all of the results presented in Table 3 is also estimated and its value is < 10 -6 .

According to IEC-61513 (2011) "there is not an equivalent scheme to the reliability/risk reduction SIL levels proposed in IEC 61508 in common use in the nuclear sector". Consequently, we do not associate a SIL level for the system under study.

If the conditional probability of failure of one component in a non-lethal shock p increases, there will be more components affected by a shock and the PFD also increases. The difference between the real PFD and the visible PFD is about 10 to 100. To asses the impact of asymmetric propagation of non-lethal CCF, we consider the orientation of CCF on the two sub-systems SSA and SSB, using the approach presented in section 3.3.1.

The conditional probability of failure of one component in a non-lethal shock is p = 0.2. The frequency of non-lethal shocks is arbitrary fixed at one shock per year, i.e. µ = 1.14 10 -1 h -1 . The lethal shocks are not represented. 10,000 trajectories during 10 years are also realized and the occurrence of system failure ends the trajectory simulation. Table 4 shows the results. The sum of failures' combinations (columns) is equal with the total number of histories (10,000) for a fixed CCF orientation. We can observe that the more a sub-system is preferred (according to p 1 and p 2 ), the more the MTTFF increases. The logic vote explains this phenomenon. We observe also that independent failures combinations without CCF rarely leads to system failure, due to the high level of redundancy. Independent failures detected offline lead more easily to system failure than the ones detected online. This aspect can be improved by increasing the frequency of periodic tests or the coverage rate of self-tests.

CONCLUSIONS

The coloured Petri nets allow to represent the digital I&C systems and to assess the dependability indicators considering CCF. The shock Atwood model has been transposed in Petri net. It allows to take in account the independent failures, the lethal CCF, but also the non-lethal CCF. An extension of this model was proposed to represent the asymmetric propagation of non-lethal CCF on privileged axes. This allows to relax the assumption of the random repartition of non-lethal shocks on the all components and to represent the effects of diversity. The proposed approach has been applied to a representative case of I&C system of a nuclear power plant.

The extremely low volume of operating experience related to the protection I&C systems makes it difficult to estimate the α factors representing the conditional probability of failures on demand of the k components in a group of N components when a CCF occurs. This CPN model could be used to simulate operating experience for obtaining more data, which make a more accurate estimation of these factors.

Another interesting axis concerns the estimation of the three parameters of the Atwood model. The proposed Monte-Carlo simulation approach for the estimation of these parameters has been validated by an analytical approach. This Monte-Carlo simulation approach could be used to revisit the assumptions supporting the Binomial Failure Rate model underlying the Atwood model.

  9. I : P → EXP R ∅ is an initialisation function that assigns an initialisation expression to each place p such that Type[I(p)] = T ype[C(p)].

  nondetected failures by self-test (NSA failures) in order to take into account the errors due to operation (different parameters or installation of boards from the nominal conditions specified by the provider). Thus the coverage rate is reduced at 85%. The global failure rate of the boards remains identical (λ IN D = λ SA + λ N SA ). The rates of detected failures (λ SA ) and non-detected failures(λ N SA ) are adjusted by the following equations: λ SA = t c .λ IN D and λ N SA = (1 -t c ).λ IN D .

Figure 2 :

 2 Figure 2: High level coloured Petri net of the I&C system

Figure 3 :

 3 Figure 3: CPN sub-net modelling the non-lethal CCF

Figure 4 :

 4 Figure 4: CPN sub-net modelling the lethal CCF

Figure 5 :

 5 Figure 5: CPN sub-net of an electronic board

Figure 6 :

 6 Figure 6: CPN sub-net to determine the state of the whole I&C system (available or unavailable)

Table 3 :

 3 Real and visible PFD of the protection I&C system for different values of p

	Indicators	p	Average
	real PFD	0.2	2.5 10 -5
	visible PFD 0.2	< 10 -6
	real PFD	0.33 2.73 10 -4
	visible PFD 0.33 1.0 10 -6
	real PFD	0.5	5.4 10 -4
	visible PFD 0.5	4.0 10 -6
	4.2 Influence of the oriented non-lethal CCF

Table 4 :

 4 Combinations of failed boards leading to the system downtime depending on p 1 and p 2

	CCF orientation p 1 SSA	0.1	0.2	0.3
	p 2 SSB	0.9	0.8	0.7
	only CCF	9404	9723	9787
	indep. SA failures	3	0	1
	indep. NA failures	21	9	9
	indep. SA and NSA	77	46	42
	CCF & indep. SA failures	8	6	1
	CCF & indep. NSA failures 487	216	160
	MTTFF (in h)	49586 28921 22056


Luka Stanisic

Arnaud Legrand

Eective Reproducible Research with Org-Mode and Git

In this article we address the question of developing a lightweight and eective workow for conducting experimental research on modern parallel computer systems in a reproducible way. Our workow simply builds on two well-known tools (Org-mode and Git) and enables to address issues such as provenance tracking, experimental setup reconstruction, replicable analysis. Although this workow is perfectible and cannot be seen as a nal solution, we have been using it for two years now and we have recently published a fully reproducible article [3], which demonstrates the eectiveness of our proposal.

Introduction

In the last decades, both hardware and software of modern computers became so complex that even experts have troubles fully understanding their behavior. Therefore, it could be argued that these machines are no longer deterministic, especially when measuring execution times of programs running on a large distributed computer systems or hybrid platforms. Controlling every relevant sophisticated component during such measurements is almost impossible, making the full reproduction of the experiments extremely difficult. Consequently, studying computers has become very similar to studying a natural phenomena and it should thus use the same principles as other scientific fields that had them defined centuries ago. Although many conclusions are based on experimental results in this domain of computer science, surprisingly articles generally poorly detail the experimental protocol. Left with insufficient information, readers have generally trouble to reproduce the study and possibly build upon it. Yet, as reminded by Drummond [START_REF] Drummond | Replicability is not reproducibility: Nor is it good science[END_REF], reproducibility of experimental results is the hallmark of science and there is no reason why this should not be applied to computer science as well.

Hence, a new movement promoting the development of reproducible research tools and practices has emerged, especially for computational sciences. Such tools generally focus on replicability of data analysis [START_REF]Implementing Reproducible Research. The R Series, Chapman and Hall/CRC[END_REF]. Although high performance computing or distributed computing experiments involve running complex codes, they do not focus on execution results but rather on the time taken to run a program and on how the machine resources were used. These requirements call for different workflows and tools, since such experiments are not replicable by essence. Nevertheless in such cases, researchers should still at least aim at full reproducibility of their work.

There are many existing solutions partially addressing these issues, but none of them was completely satisfying our needs and therefore we developed an alternative approach that is based on two well-known and widely-used tools: Org-mode and Git. We present our contributions in Section 3, where we first describe a specific use of Org-mode for doing the provenance tracking of the entire projects. Then, we propose a unique way to use Git for keeping synchronized experiment results and code that generated them. Finally, it will be demonstrated in Section 4 how the proposed methodology helped us conducting two very different studies in the High Performance Computing (HPC) domain. We will also state limits of our approach, together with some open questions.

Related Work

In the past few years, the field of reproducibility research tools has been very active, various alternatives emerging to address diverse problematic. However, in the HPC domain most of them are concentrated on platform accessibility, setting up environments and running the experiments on large clusters. Even though such tools are very useful in general, we could not benefit from them due to the specific nature of our research projects. The machines we needed to study are recent prototypes, with ever-changing libraries, set up by expert administrators and we do not have neither the permission nor the interest to do any modifications to the environment.

However, when conducting experiments and analysis, there are more aspects that need to be considered. We detail the ones related to software, methodology and provenance tracking, which are often neglected by researchers in our community.

Accessibility It is widely accepted that tools like Git or svn are indispensable in everyday work on software development. Additionally, they help at sharing the code and letting other people contribute. Making data accessible, file hosting services, such as Dropbox, Google Drive and many others, became very popular among all scientists that want to collaborate. There is another group of services that is more oriented on making data publicly available and easily understandable to everyone, e.g., figshare1 . Provenance tracking Knowing how data was obtained is a complex problem.

The first part involves collecting meta-data, such as system information, experimental conditions, etc., and it is often managed by experimental engines.

The second, frequently forgotten in our domain, part is to track any applied transformation to the data, i.e., moving the objects from one state to another. Moreover, there is a question of storing both data and meta-data. Classical approach to solve these issues involves using a database. However, this solution has its limits, as managing source codes or comments in a database is not convenient and is handled in much better way by using version control systems and literate programming.

Documenting While provenance tracking is focused on how data was obtained, it is not concerned with why the experiments were run and what the observations on the results are. These things have to be thoroughly documented, since even the experimenters tend to quickly forget all the details. One way is to encourage users to keep notes when running experiment (e.g., in Sumatra [5, chap. 3]), while the other one consists in writing a laboratory notebook (e.g., with IPython2 ). Extendability It is hard to define good formats for all project components in the starting phase of the research. Some of the initial decisions are likely to change during the study, so system has to be flexible. In such a volatile context, integrated tools with databases, such as Sumatra, are too cumbersome for everyday extentions and modifications. To address the previous problems, we propose to rely on a minimalist set of simple, lightweight, and well-known tools. We use Org-mode [START_REF] Schulte | A multi-language computing environment for literate programming and reproducible research[END_REF], initially an Emacs mode for editing and organizing notes, that is based on highly hierarchical plain text files which are easy to explore and exploit. Org-mode has also been extended to allow combining plain text with small chunks of executable code (Org-babel snippets). Such features follow the literate programming principles introduced by Donald Knuth three decades ago, and for which there is a renewed interest in the last years. In addition, for version control system we decided to rely on Git, a distributed revision control tool that has an incredibly powerful branch management system.

Tips and Tricks for Reproducible Research

In this section, we provide guidelines on best practices and hints on pragmatic ways to implement them. First, we illustrate how to handle provenance tracking issues with literate programming, in particular with Org-mode. The approach we propose is lightweight, to make sure the experiments are performed in a clean, coherent and hopefully reproducible manner without being slowed down by a rigid framework. However, it is tempting to sometimes break one of these rules, which hinders reproducibility in the end. This is why we harden these guidelines with a particular usage of Git that forces the user to keep data and code synchronized.

Provenance Tracking Through Literate Programming

As described in Section 2, there are many tools that can help to automatically capture environment parameters, to keep track of experimentation process, to organize code and data, etc. However, it is not easy to understand how they work exactly, and additionally each of them creates new dependencies on specific libraries and technologies. Instead, we propose a solution based on plain text files, written in the spirit of literate programming, that are self-explanatory, comprehensive and portable. We do not rely on a huge cumbersome framework, but rather on a set of simple, flexible scripts, that address the following challenges.

Environment Capture Environment capture consists in getting all the details about the code, used libraries and system configuration. It is necessary to gather as much useful meta-data as possible, to allow to compare experiment results with the previous ones and inspect if there were any changes to the experimental environment. This process should not be burdensome, but automatic and transparent to the researcher. Additionally, it should be easy to extend or modify, since it is generally difficult to anticipate relevant parameters before performing numerous initial experiments.

Once meta-data is captured, it can be stored either individually or accompanying results data. Some prefer keeping these two separated, making their primary results unpolluted and easier to exploit, but they soon run into difficulties when they need to retrieve information from meta-data. Therefore, we strongly believe that the experiment results should stay together with the information about the system they were obtained on. Moreover, keeping them in the same file makes the access straightforward and simplifies the project organization, as there are less objects to handle. Consequently, even if data sustains numerous movements and reorganizations, one would never doubt which environment setup corresponds to which results.

In order to permit users to easily examine any of their information, these files have to be well structured. The Org-mode format is a perfect match for such requirements as its hierarchical organization is simple and can be easily explored. A good alternative might be to use the yaml format, which is typed and easy to parse but we decided to stay with Org-mode (which served all our needs) to keep our framework minimalist.

A potential issue of this approach is raised by large files, typically containing several hundreds of MB and more. Opening such files can temporary freeze a text editor and finding a particular information can then be tedious. We haven't yet met with such kind of scenario, but obviously it would require some adaptations to the approach.

Note that all the data and meta-data are gathered automatically using scripts, finally producing a read-only Org-mode document. Why the experiments were performed and what are the observations on its results is stored elsewhere, more precisely in a laboratory notebook of the project.

Laboratory Notebook A paramount asset of our methodology is the laboratory notebook (labbook), similar to the ones biologist, chemists and scientist from other fields use on a daily basis to document the progress of their work. For us, this notebook is a single file inside the project repository, shared between all collaborators. The main motivation for keeping a labbook is that anyone, from original researchers to external reviewers, can later use it to understand all the steps of the study and potentially reproduce and improve it. This self-contained unique file has multiple purposes. Indeed, the labbook should not only serve as journal but also play the following software development roles to ensure that it can be exploited by others: the project in general inside a log section. Since all modifications are already captured and commented in Git commits, the log section should offer a much more coarse grain view of the code development history. There should also be a list with descriptions of every Git tag in the repository as it helps finding the latest stable, or any other specific, version of the code. 5. Experiment results: Every set of experiment should be carefully noted here, together with the key input parameters, the motivation for running such experiment and finally observations on the results. Inside the descriptive conclusions, Org-mode allows to use both links and git-links connecting the text to the actual files in the Git repository. These hyperlinks point to the crucial data and the analysis reports that illustrate a newly discovered phenomena.

Managing efficiently all these different information in a single file requires a solid hierarchical structure, which once again motivated our use of Org-mode. We also took advantage of the Org-mode tagging mechanism, which allows to easily extract information, improving labbook's structure even further. Org tags can be used to distinguish which collaborator conducted a given set of experiments or to list expertise requests. Although many of these information may already be present in the experiment files, having it at the journal level revealed very convenient. Experiments can also be tagged to indicate on which machine they were performed and whether the results were important or not. Again, although such tagging is not required it is very handy in practice and make the labbook much easier to understand and exploit.

Several alternatives exist for taking care of experiment results and progress on a daily basis. We think that a major advantage of Org-mode compared to many other tools is that it is just a plain text file that can thus be read and modified on any remote machine without requiring to install any particular library. Using a plain text file is also the most portable format across different architectures and operating systems.

Data File Organization Having a clear, coherent and hierarchical organization of all the files is a good practice for a proper scientific work, especially when external collaborators are involved. Once again, the approach we propose is lightweight and flexible but is motivated by the three following important points:

1. There should never be any critical information in file organization. Important information should be in the files themselves. Indeed, we could as well have blobs rather than files but managing data and extracting important information would probably not be very convenient. Thus, we do not recommend to impose much on file organization so that users can organize their data in a way that is natural to them. We think this lack of rules is not an issue as long as this organization is explained in the labbook. 2. The file organization should be flexible enough to be changed and adapted as the experimental data set grows. Such reorganization could seemingly break the labbook hyperlinks. However, as we briefly mentioned, we recommend to use git-links in the labbook, which are Org-mode hyperlinks that store links to specific revisions of files (typically when they were created). So reorganizing the data files will not break the labbook information. 3. The naming convention should not impede the activity of the researcher, so here we used almost no convention at all. According to our experience, all experiment results could simply be saved in folders, each of them representing one set of measurements and having a unique characteristic name, e.g., the name of the machine on which it was performed. Inside a folder, file names could be prefixed by additional key characteristics of the experiment set, followed by an ordinal number indicating in which order experiments were run. This idea seemed the most natural one to adopt, but we are reconsidering alternatives for the future projects.

Conclusion

The approach we described implicates a partial redundancy of some data and meta-data, typically saved in both experiment result files and in the laboratory notebook. However, such information are never entered twice manually. Most data should always be automatically tracked, although when some data convey key information, they should be manually added to other places as well, since it provides a better overview of the whole project. We think that following the proposed guidelines is sufficient to conduct a clean, comprehensible and reproducible research while having a very fluid workflow. However, not all scientists are rigorous enough to always follow such conventions and even those who are, occasionally have the need to bend the rules in order to quickly get some results in a dirty way. This sometimes pollutes the whole project organization, often breaking the chains of the workflow processes and making some parts incoherent.

In order to force researchers to be more disciplined and help them doing their work in a reproducible manner, we propose to combine the previous approach with a particular usage of Git.

Using Git for Improving Reproducible Research

Even when the project is well organized, meta-data tracked, all the collaborators follow the conventions and take notes in the laboratory notebook, several practical issues may still arise:

1. Although a svn revision or a Git sha1 of the source code is captured, this does not guarantee that the experiment was run correctly and that the results can easily be reproduced. There could exist some uncommitted differences between the tracked and the current version of the code or the compilation could be out-of-date, i.e., code was compiled with an old revision. A solution proposed for example by Davison [5, chap. 3] and which we applied as well, is to force recompilation and systematically verify that everything is fully committed before running any experiment. The only exception to this rule are the tests performed to validate the workflow. 2. Even with the complete and correct meta-data and code revisions, it is not always easy to reconstruct the experimental setup, especially if it consists of the code from numerous external repositories. Multiplying repositories hinders provenance tracking, coherency and experimental reproduction. The solution we propose is to increase the reproducibility confidence level by using only revision control and a collection of simple scripts that automatically track information. Additionally, we suggest to store both code and experimental data in the same Git repository, so that they are always perfectly synchronized, which eases the obtainment of the code that produced a particular data set. Nevertheless, this introduced the following new challenges. 3. Unlike source code, data files can be large, thus keeping them together in the same repository can rapidly increase its size. Moreover, doing code modifications, analysis and experiments in the same Git branch complicates its history and makes experimental setup reproduction slightly ambiguous. 4. Another difficulty comes from managing several external beta source codes that should now coexist in the same repository. Since these codes are also under development, they are regularly upgraded by their developers and these changes need to be propagated to local project as well. Additionally, these codes typically have their own revision control, which rises many issues with potential local code modifications and commits that concern now both local and external projects.

Proposal To solve aforementioned problems we propose an approach that uses Git with two parallel interconnected branches, displayed in Figure 1. The first branch, named src, includes only the source code, i.e., code and scripts for running the experiments and the analysis. The second, data, branch consists of all the source code as well as all the data and analysis reports (statistical analysis results, figures, etc.). These two branches live in parallel and are interconnected through a third type of branches, the exp# branches, where the experimentation is performed. All these together form a "ladder like" Git repository, depicted in Figure 1(c).

We now explain the typical workflow usage of such branching scheme:

1. Development phase: Researchers work on a code development inside the src branch, committing the changes, as shown on Figure 1(a). These modifications can impact local or external code, analysis or even the scripts for running the experiments. Later, such modifications should be tested and the correctness of the whole workflow should be validated. Only then can one start doing real useful experiments. 2. Experimentation phase: Researcher creates a new branch from the src branch containing and a new folder to store the results. We used the convention that these two (branch and directory) should always have equal names, which eases the usage of both Git and labbook. In the example of Figure 1(a), this new branch for doing the measurements is called exp1. Next, new experiments are executed running the scripts and generate results. The resulting data, together with the captured environment meta-data, are then committed to the Git. After that, one might want to do some basic analysis of the data, investigating the results, which may later trigger another round of experimentation and so on, as it is showed on the Figure 1(b). Finally, only when all desired measurements are finished, exp1 will be merged with the data branch. 3. Merging and reports phase: All experimental exp# branches are in the end merged with data, as it can be seen on Figure 1(c). In addition, result observations for each exp# branch are written to the labbook. Afterwards, comparison of different experiments can be performed by generating figures, tables and clear explanations, to describe the newly discovered phenomena. Since the changes to the source code from src branch are also propagated through exp#, the head of the data branch will always contain the latest code together with all the data. Nevertheless, the older version of code responsible for producing a particular data or analysis can always be found in the Git history.

A peculiar situation occurs when there are source modifications inside the experimental branches. They have to be committed (as measurements are never done with an uncommitted code), even though in most cases they represent an ad hoc change, specific to an individual machine and its installation. These minor, local hacks would pollute the data branch and thus it has to be ensured that they are not propagated. It is done by using a special script for merging the branches instead of classical Git merge command. At the end of the exp# branch, all source code changes (not the data) have to be invalidated by using git revert, i.e., the "anti-commit" of all the previously committed modifications inside that branch. This way modifications remain local for that exp# branch and the experimental setup can still be reproduced by pulling the revision before the revert operation, i.e., the one used to generate the data.

If the researcher eventually realizes that some of the source code modifications done inside exp# branch could be useful for the whole project, there are two ways to inserted them in the src branch. The first one involves rewriting Git history and it is not advised as it leads to incoherences between Git repositories. The second option is to cherry-pick the desired commits. Although this approach produces some redundancy, it is very easy and safe and keeps the Git history clear and coherent.

External software One more challenge arises when there are external software repositories imported inside a local project. For example, one could have external source code B that is a part of a bigger local project A. Since B is also under development, one occasionally needs to pull the updates from its server, which can cause conflicts with local modifications to the code. Resolving these conflicts manually can sometimes be very tedious.

Even bigger problem occurs if one wants to push such local changes, as they can be committed to either our project A, or external project B, or even to both of them together. We decided to propagate, by default, these modifications only to the project A, keeping the Git sha1 of A always valid and up-to-date. Later if necessary, they can be pushed to B as well, but this has to be done by explicitly calling the necessary commands.

Dealing with described challenges is error-prone, thus we started using git-subrepo tool for cleaner and semi-automated management of external Git projects inside our local one. However, we still have to do everything manually when working with codes that are using other version control systems, notably svn.

Analysis

We now analyze the proposed solution and investigate how it addresses the stated problematic.

First, a complete synchronisation of code, data and analysis is ensured. The convention to use the same name for Git experiment branches and folders containing experiment results, additionally carefully noting it in the labbook, makes exploration of project history very smooth. This way one can easily find when a particular data or a figure was created, pull the revision used to generate it, inspect the code, reconstruct the environment and finally reproduce the object. Using experimental branches also allows some local source modifications, that are specific for the remote machine or some other part of the experimentation setup. These changes stay local for that exp# branch avoiding to pollute main src and data branches, elegantly making the project easier to read while still keeping it coherent. Additionally, Git permits to put tags on certain commits, which can be used to annotate an important revision, such as the ones with the stable source code or the ones with some specific adjustments. With the evolution of the study, Git history becomes large and harder to explore, thus these tags can help to quickly find a desired state of the project.

By using Git as proposed, it is extremely easy to set up an experimental platform on a remote machine by pulling only the head of the src branch. This solves the problem of long and memory consuming retrieving of the entire data and Git history, as the src branch is typically very small.

On the other hand, one might want to gather all the experimental data at once, which can be easily done by pulling only the data branch. This is the case for the researchers that are not interested in the experimentation process, but only at the analysis of the whole set of results. For them, src and exp# branches are completely transparent, as they will retrieve only the latest version of the source code (including needed analysis scripts) and the entire data.

Another use case is when someone wants to write an article or a report based on the experiment results. A completely new branch can be created from data, selecting from the repository only the data and analysis code needed for the publication and deleting the rest. This way complete history of the study behind the article is preserved for the reviewers.

Holding external projects inside a local one allows to do git pull or svn checkout in sub-directories, keeping them up-to-date. Small problems arise when some modifications to the external code are done. These changes de facto influence both external and local repository but our solution ensures that they are committed only to the local revision control. Therefore in the meta-data part of the experiment files, tracked revision of the external code corresponds to the version pulled before any of the local changes, which is not strictly legitimate. Nevertheless, this small issue is not critical, since the revision in local project is stored in meta-data as well, and this value is always perfectly correct. Pulling this version will bring the right code, keeping the research reproducible.

Evaluation

We used the described methodology in two very different use cases. The first one is a part of the study of CPU cache performance on various Intel and ARM micro-architectures [START_REF] Stanisic | Performance analysis of hpc applications on lowpower embedded platforms[END_REF]. The developed source code was very simple, containing only a few files, but there were numerous input parameters to be taken into account. Probably the critical part of this study is about the environment setup, which proved to be unstable, and thus, responsible for many unexpected phenomena. Therefore, it was essential to capture, understand and easily compare as much meta-data as possible. Although it did not lead to a reproducible article as we were only discovering such tools, we used this workflow and can still track the whole history of these experiments. The second use case [START_REF] Stanisic | Modeling and Simulation of a Dynamic Task-Based Runtime System for Heterogeneous Multi-Core Architectures[END_REF] aims at providing accurate performance predictions for dense linear algebra kernels, using the StarPU runtime on top of the SimGrid simulator. By contrast, input parameters and environment here are fixed, but the source code is very complex and in constant evolution. Moreover, we had to manage code from two external repositories as well many of our own scripts.

The proposed solution proved generally successful in both use cases. We have determined several good and bad sides, while for some aspects still remain rather uncertain.

Pros Our approach is fast and efficient for a daily usage. It provides reasonable boundaries without taking away too much flexibility from the users. It offers good code modification isolation, which is important for ad hoc changes. Perfect provenance tracking, which was painless to extend and explore, was crucial for the cache measurement study. Although these two use cases are very different, most of the captured environment meta-data is the same for both projects: date and time, hostname, Linux and gcc version, users logged on the machine, environment variables, used external libraries, code revisions, memory hierarchy of the machine, CPU governor and frequency, compilation outputs, etc. Since all the source code and data is in Git repository, reconstructing experimentation setup is straightforward. One could argue that not all elements are completely captured, since operating system and external libraries can only be reviewed but not reconstructed. To handle this, researchers would have to use virtual machines to run the experiments, which would introduce many new performance issues. Finally, after applying such a methodology throughout the whole research process, it was extremely easy to write an article [START_REF] Stanisic | Modeling and Simulation of a Dynamic Task-Based Runtime System for Heterogeneous Multi-Core Architectures[END_REF] in Org-mode that was completely reproducible as well. Along with the text, this Org-mode document contains all the analysis scripts and the raw data is provided as the article companion [START_REF]Companion of the StarPU+SimGrid article[END_REF] and can be inspected by reviewers.

Cons The biggest disadvantage of our approach is that it has many not so common conventions along with a steep learning curve workflow, hence it is difficult for new users. Moreover, it requires an expertise in Org-mode, preferably using Emacs text editor, together with a good understanding of Git. However, we believe that these tools provide benefits that are worth investing time.

Additionally, we find current way of managing external source repositories slightly cumbersome, and we are searching for a better solution. One path could be to use recipes or experiment engines, that would do the checkout of external sources for us and would only apply the right modifications before compiling.

The problem of storing large data files in repositories is well-known to the community. It has been already solved for the Mercurial revision control tool, but even after an thorough research we could not find a satisfactory solution for Git. Git repositories can quickly reach a few Gigabytes, which does not hinder daily committing but significantly slows down rebase operations to move back to previous experimental conditions of a specific dataset.

Open questions It is still unclear how this approach would scale for multiple users working simultaneously, doing code modifications and experiments in parallel. In theory it should work if everyone has sufficient experience of the tools and workflow, but we have never tried it with more than two persons.

Another interesting feature that we haven't yet experienced is collaboration with external users. These researchers could clone our project, work on it on their own, try to reproduce the results and build upon our work, potentially improving the code and contribute data sets back. Even though such utilization should work smoothly, there could be some pitfalls that we haven't anticipated.

These are only few of the unknown, and as there are certainly many more, we are hoping for the audience suggestions and remarks.

Conclusion

In this paper, we did not intend to propose new tools for reproducible research, but rather investigate whether a minimal combination of existing ones can prove useful. The approach we described is a good example of using well-known, lightweight, open-source technologies to properly perform a very complex process like computer science experimentation. Although our methodology is undoubtedly improvable and similar results could be obtained with other frameworks, we nonetheless find it very smooth for a daily usage and extremely beneficial to our work. We can only encourage people to build on such simple workflows to conduct their own studies, as it is clearly a very effective way to produce a reproducible research.

Fig. 1 .

 1 Fig. 1. Dierent phases in git workow

  1. README: The labbook should explain ideas behind the whole project purpose and methodology, i.e., what the workflow for doing experiments is and how the code and data are organized in folders. It should state the conventions on how the labbook should be used. This part serves as a starting point for newcomers and is also a good reminder for experienced users. 2. Documentation: The labbook should detail what are the different programs and scripts, and what is their purpose. This documentation concerns source code for the experimentation as well as tools for manipulating data and analysis code for producing plots and reports. Additionally, there should be explanations on the revision control usage and conventions. 3. Examples: The labbook should contain example usages of how to run scripts, displaying the most common arguments and format. Although such information might seem redundant with the previous documentation part, in practice such examples are indispensable even for everyday users, since some scripts require lots of environment variables, arguments and options. 4. Log: It is important to keep track of big changes to the source code and

http://gshare.com

http://ipython.org