J.-F Pommaret
email: jean-francois.pommaret@wanadoo.fr

GENERAL RELATIVITY AND GAUGE THEORY: BEYOND THE MIRROR

Keywords: General Relativity, Riemann tensor, Weyl tensor, Ricci tensor, Einstein equations, Elastic waves, Gravitational waves, Lie groups, Lie pseudogroups, Differential Galois theory, Spencer operator, Janet sequence, Spencer sequence, Differential modules, Algebraic analysis, Homological algebra, Extension modules, Biduality, Split exact sequence

This paper is a Summary Note sketching in a rather self-contained but condensed way the results recently presented through a series of lectures at the Albert Einstein Institute (AEI, Berlin/Postdam), october, 23-27, 2017. The initial motivation for studying the methods used in this paper has been a 1000$ challenge proposed in 1970 by J. Wheeler in the physics department of Princeton University while the author of this paper was a student of D.C. Spencer in the closeby mathematics department: Is it possible to express the generic solutions of Einstein equations in vacuum by means of the derivatives of a certain number of arbitrary functions like the potentials for Maxwell equations ?. After recalling the negative answer we already provided in 1995, the main purpose of this paper is to use again these new techniques of biduality in order to revisit the mathematical foundations of the concepts and equations leading to gravitational waves.

1) INTRODUCTION

The purpose of these (self-contained but difficult) lectures is to revisit general relativity (GR) and gauge theory (GT) in view of the latest mathematical developments existing today in group theory, system theory and module theory, namely:

• Systems : The order of the Weyl-type tensor and its Bianchi-type identities highly depend on the dimension, a result recently confirmed by A. Quadrat (INRIA) for 3,4,5 with computer algebra.

• Groups : The Ricci tensor has only to do with the second order jets of the conformal group, called elations by Cartan (1922).

• Modules : In contrast with the Maxwell equations, the Einstein equations are parametrizing the Cauchy stress equations but cannot be parametrized themselves. Such a result is not coherent with a "reflexivity " test in homological algebra involving the so-called first and second extension modules. As a byproduct, the Cauchy stress equations must not be confused with the divergencetype condition for the Einstein tensor obtained by contracting the Bianchi identities.

We briefly recall the historical framework leading to these new results. The concept of "group" has been introduced in mathematics for the first time by E. GALOIS (1830) and slowly passed from algebra to geometry with the work of S. LIE on Lie groups (1880) and Lie pseudogroups (1890) of transformations. The concept of a finite length differential sequence, now called Janet sequence, has been described for the first time as a footnote by M. JANET (1920). Then, the work of D. C. SPENCER (1970) has been the first attempt to use the formal theory of systems of partial differential equations in order to study the formal theory of Lie pseudogroups. However, the linear and nonlinear Spencer sequences for Lie pseudogroups, though never used in physics, largely supersede the "Cartan structure equations " (1905) and are quite different from the "Vessiot structure equations " (1903), introduced for the same purpose but still not known today because they have never been acknowledged by E. CARTAN or successors.

CART AN

-→ SP EN CER ր GALOIS → LIE COSSERAT + W EY L ց V ESSIOT -→ JAN ET
Meanwhile, mixing differential geometry with homological algebra, M. KASHIWARA (1970) created "algebraic analysis ", in order to study differential modules and double duality by using differential modules and their corresponding extension modules.

By chance, unexpected arguments have been introduced by the brothers E. and F. COSSERAT (1909) in order to revisit elasticity and by H. WEYL (1918) in order to revisit electromagnetism through a unique differential sequence only depending on the structure of the conformal group. However, while the Cosserat brothers were only using translations and rotations, Weyl has only been dealing with dilatation and elations as we shall explain.

2) PARAMETRIZATION

Starting with the well known linear map C :

S 2 T * → S 2 T * : R ij → E ij = R ij -1
2 ω ij tr(R) between symmetric covariant tensors, where ω is a metric with det(ω) = 0 and tr(R) = ω ij R ij , we may introduce the linear second order operators Ricci : Ω → E and Einstein : Ω → E obtained by linearization over ω and we have the relation Einstein = C • Ricci where C does not depend on any conformal factor. We recall the method used in any textbook for studying gravitational waves, which "surprisingly " brings the same map C : Ω → Ω = Ω -1 2 ω tr(Ω) in order to introduce the key unknown composite operator X : Ω → Ω → E, having therefore Einstein = X • C.

My first goal will be to prove that only the use of algebraic analysis, a mixture of differential geometry (differential sequences, formal adjoint) and homological algebra module theory, biduality, extension modules) totally unknown by physicists, is able to explain why the Einstein operator (with 6 terms) defined above is useless as it can be replaced by the Ricci operator (with 4 terms) in the search for gravitational waves equations. Indeed, using the fact that the Einstein operator is self-adjoint (with a slight abuse of language) when ω is the Minkowski metric (contrary to the Ricci operator as we shall see) and taking the respective (formal) adjoint operators, we get:

ad(Einstein) = ad(C) • ad(X ) ⇒ Einstein = C • ad(X ) ⇒ ad(X ) = Ricci ⇒ X = ad(Ricci)
Meanwhile, the Riemann operator can be considered as an operator describing the (second order) compatibility conditions (CC) for the Killing operator ξ ∈ T → L(ξ)ω = Ω ∈ S 2 T * with standard notations where L is the Lie derivative. In this new framework, we shall prove that we no longer need to use the Bianchi operator as the first order CC for the Riemann operator. Also, we shall prove that the relative parametrization with div-type differential constraints needed in order to keep only the Dalembert operator in the wave equations has nothing to do with any gauge transformation in the corresponding adjoint differential sequence, but with the search for a minimal parametrization, exactly like Maxwell did in 1870 for elasticity.

EXAMPLE: When n = 2, the stress equations become ∂ 1 σ 11 + ∂ 2 σ 12 = 0, ∂ 1 σ 21 + ∂ 2 σ 22 = 0.
Their second order parametrization σ 11 = ∂ 22 φ, σ 12 = σ 21 = -∂ 12 φ, σ 22 = ∂ 11 φ has been provided by George Biddell Airy (1801-1892) in 1863. It can be simply recovered in the following manner:

∂ 1 σ 11 -∂ 2 (-σ 12 ) = 0 ⇒ ∃ϕ, σ 11 = ∂ 2 ϕ, σ 12 = -∂ 1 ϕ ∂ 2 σ 22 -∂ 1 (-σ 21 ) = 0 ⇒ ∃ψ, σ 22 = ∂ 1 ψ, σ 21 = -∂ 2 ψ σ 12 = σ 21 ⇒ ∂ 1 ϕ -∂ 2 ψ = 0 ⇒ ∃φ, ϕ = ∂ 2 φ, ψ = ∂ 1 φ
When constructing a long prismatic dam with concrete, we may transform a problem of 3dimensional elasticity into a problem of 2-dimensional elasticity by supposing that the axis x 3 of the dam is perpendicular to the river with Ω ij (x 1 , x 2 ), ∀i, j = 1, 2 and Ω 33 = 0 because of the rocky banks of the river are supposed to be fixed. We may introduce the two Lamé constants (λ, µ) in order to describe the usual constitutive relations of an homogeneous isotropic medium as follows, passing from the standard case n = 3 to the restricted case n = 2 just by setting:

σ = 1 2 λ tr(Ω) ω + µ Ω, tr(Ω) = Ω 11 + Ω 22 ⇒ µ Ω = σ - λ 2(λ + µ)
tr(σ) ω, tr(σ) = σ 11 + σ 22 even though σ 33 = 1 2 λ(Ω 11 + Ω 22 ) = 1 2 λtr(Ω). Let us consider the right square of the diagram below with locally exact rows, where any vector bundle is simply denoted by its fiber dimension:

2 Killing -→ 3 Riemann -→ 1 → 0 . . . ↓↑ . . . 0 ← 2 Cauchy ←- 3 Airy ←- 1
Taking into account the linearization of the only component of the Riemann tensor over the Euclidean metric ω when n = 2 and substituting the Airy parametrization, we obtain:

tr(R) ≡ d 11 Ω 22 + d 22 Ω 11 -2d 12 Ω 12 = 0 ⇒ µ tr(R) ≡ λ + 2µ 2(λ + µ) ∆∆φ = 0 ⇒ ∆∆φ = 0
where the linearized scalar curvature tr(R) is allowing to define the Riemann operator in the previous diagram, namely the only compatibility condition (CC) of the Killing operator. It remains to exhibit an arbitrary homogeneous polynomial solution of degree 3 and to determine its 4 coefficients by the boundary pressure conditions on the upstream and downstream walls of the dam. The Airy potential φ has nothing to do with the perturbation Ω of the metric ω and the Airy parametrization is nothing else but the formal adjoint of the Riemann operator, that is Airy = ad(Riemann).

EXAMPLE: When n = 3, we may now use the left square of the following diagram with locally exact rows:

3 Killing -→ 6 Riemann -→ 6 . . . ↓↑ . . . 0 ← 3 Cauchy ←- 6 Beltrami ←- 6
where the self-adjoint operator Beltrami = ad(Riemann) has ben introduced by E. Beltrami in 1892. We may substitute the 3-dimensional constitutive relations with Lamé constants (λ, µ) in the Cauchy stress equations and get, when f = g (gravity) is now the right member:

(λ + µ) ∇( ∇. ξ) + µ∆ ξ = f ∇ ⇒ (λ + 2µ)∆tr(Ω) = 0 ⇒ ∆tr(Ω) = 0 ⇒ ∆tr(σ) = 0
We discover at once that the origin of elastic waves is shifted by one step backwards, from the right square to the left square of the diagram. Indeed, using inertial forces f = ρ ∂ 2 ξ/∂t 2 for a medium with mass ρ per unit volume in the right member of Cauchy stress equations because of Newton law and the vector identity ∇ ∧ ( ∇ ∧ ξ) = ∇( ∇. ξ) -∆ ξ, we discover the existence of two types of elastic waves A exp i ( k. xωt) with wave vector k, period T , pulsation ω = 2π/T along standard notations, namely the longitudinal and transversal waves with different speeds v T < v L , which are really existing because that are responsible for earthquakes:

   ∇. ξ = 0 ⇒ k. A = 0 ⇒ µ△ ξ = f ⇒ v T = µ ρ ∇ ∧ ξ = 0 ⇒ k ∧ A = 0 ⇒ (λ + 2µ)△ ξ = f ⇒ v L = λ+2µ ρ
These comments pushed the author to a systematic use of the formal adjoint of an operator.

Let us explain the origin of the definition of extension modules in homological algebra by means of an elementary example. With

∂ 22 ξ = η 2 , ∂ 12 ξ = η 1 for D, we get ∂ 1 η 2 -∂ 2 η 1 = ζ for the CC D 1 . Then ad(D 1 ) is defined by µ 2 = -∂ 1 λ, µ 1 = ∂ 2 λ while ad(D) is defined by ν = ∂ 12 µ 1 + ∂ 22 µ 2 but the CC of ad(D 1 ) are simply generated by ν ′ = ∂ 1 µ 1 + ∂ 2 µ 2 .
Using operators, we have the two differential sequences:

ξ D -→ η D1 -→ ζ ν ad(D) ←-µ ad(D1) ←-λ ւ ν ′
where D 1 generates the CC of D in the upper sequence but ad(D) does not generate the CC of ad(D 1 ) in the lower sequence, even though D 1 • D = 0 ⇒ ad(D) • ad(D 1 ) = 0, contrary to what happens in the Poincaré sequence for the exterior derivatve used in electromagnetism for exhibiting the Maxwell equations when n = 4. We shall see that this comment brings the need to introduce the first extension module ext 1 (M ) of the differential module M = coker(D) determined by D.

In the general case, using the same notation for a vector bundle and its set of local sections,

when E D -→ F is a given operator, its formal adjoint is ∧ n T * ⊗ E * ad(D)
←-∧ n T * ⊗ F * where E * and F * are respectively obtained from E and F by inverting the transition matrices, like T and T * .

Before going ahead, let us prove that there may be mainly two types of differential sequences, the Janet sequence introduced by M. Janet in 1920, having to do with the tools we have studied, and a different sequence called Spencer sequence introduced by D. C. Spencer in 1970 with totally different operators. For this, if E is a vector bundle over the base X, we shall introduce the q-jet bundle J q (E) with sections ξ q : (x)

→ (ξ k (x), ξ k i (x), ξ k ij (x), ...) transforming like the sections j q (ξ) : (x) → (ξ k (x), ∂ i ξ k (x), ∂ ij ξ k ( x), ...).
The Spencer operator D : J q+1 (E) → T * ⊗ J q (T ) allows to compare these sections by considering the differences

(∂ i ξ k (x) -ξ k i (x), ∂ i ξ k j (x) -ξ k ij (x), .
..) and so on. When ω is a nondegenerate metric with Christoffel symbols γ and Levi-Civita isomorphism j 1 (ω) ≃ (ω, γ) while T = T (X) is the tangent bundle to X, we consider the second order involutive system R 2 ⊂ J 2 (T ) defined by considering the first order Killing system L(ξ)ω = 0, adding its first prolongation L(ξ)γ = 0 and using ξ 2 instead of j 2 (ξ). Looking for the first order generating compatibility conditions (CC) D 1 of the corresponding second order operator operator D just described, we may then look for the generating CC D 2 of D 1 and so on. We may proceed similarly for the injective operator

T j2 -→ C 0 (T ) = J 2 (T ), finding successively C 0 (T ) D1 -→ C 1 (T )
and C 1 (T ) D2 -→ C 2 (T ) induced by D. When n = 2 and ω is the Euclide metric, we have a Lie group of isometries with the 3 infinitesimal generators {∂ 1 , ∂ 2 , x 1 ∂ 2x 2 ∂ 1 }. If we now consider the Weyl group defined by L(ξ)ω = Aω with A = cst and L(ξ)γ = 0, we have to add the only dilatation x 1 ∂ 1 + x 2 ∂ 2 . Collecting the results and exhibiting the induced kernel upper differential sequence, we get the following commutative fundamental diagram I where the upper down arrows are monomorphisms while the lower down arrows are epimorphisms Φ 0 , Φ 1 , Φ 2 :

0 -→ Θ j2 -→ 4 D1 -→ 8 D2 -→ 4 -→ 0 0 -→ Θ j2 -→ 3 D1 -→ 6 D2 -→ 3 -→ 0 Spencer ↓ ↓ ↓ 0 -→ 2 j2 -→ 12 D1 -→ 16 D2 -→ 6 -→ 0 ↓ Φ 0 ↓ Φ 1 ↓ Φ 2 0 -→ Θ -→ 2 D -→ 9 D1 -→ 10 D2 -→ 3 -→ 0 Janet 0 -→ Θ -→ 2 D -→ 8 D1 -→ 8 D2 -→ 2 -→ 0
It follows that "Spencer and Janet play at see-saw ", the dimension of each Janet bundle being decreased by the same amount as the dimension of the corresponding Spencer bundle is increased.

3) DIFFERENTIAL MODULES

Let A be a unitary ring, that is 1, a, b ∈ A ⇒ a + b, ab ∈ A, 1a = a1 = a and even an integral domain (ab = 0 ⇒ a = 0 or b = 0) with field of fractions K = Q(A). However, we shall not always assume that A is commutative, that is ab may be different from ba in general for a, b ∈ A.

We say that

M = A M is a left module over A if x, y ∈ M ⇒ ax, x + y ∈ M, ∀a ∈ A or a right module M B over B if the operation of B on M is (x, b) → xb, ∀b ∈ B.
If M is a left module over A and a right module over B with (ax)b = a(xb), ∀a ∈ A, ∀b ∈ B, ∀x ∈ M , then we shall say that M = A M B is a bimodule. Of course, A = A A A is a bimodule over itself. We define the torsion submodule t(M ) = {x ∈ M | ∃0 = a ∈ A, ax = 0} ⊆ M and M is a torsion module if t(M ) = M or a torsion-free module if t(M ) = 0. We denote by hom A (M, N ) the set of morphisms f : M → N such that f (ax) = af (x). We finally recall that a sequence of modules and maps is exact if the kernel of any map is equal to the image of the map preceding it.

When A is commutative, hom(M, N ) is again an A-module for the law (bf )(x) = f (bx) as we have (bf )(ax) = f (bax) = f (abx) = af (bx) = a(bf )(x). In the non-commutative case, things are more complicate and, given A M and A N B , then hom A (M, N ) becomes a right module over B for the law (f b)(x) = f (x)b.

DEFINITION:

A module F is said to be free if it is isomorphic to a (finite) power of A called the rank of F over A and denoted by rk A (F ) while the rank rk A (M ) of a module M is the rank of a maximum free submodule F ⊂ M . It follows from this definition that M/F is a torsion module. In the sequel we shall only consider finitely presented modules, namely finitely generated modules defined by exact sequences of the type F 1 d1 -→ F 0 p -→ M -→ 0 where F 0 and F 1 are free modules of finite ranks m 0 and m 1 often denoted by m and p in examples. A module P is called projective if there exists a free module F and another (projective) module Q such that P ⊕ Q ≃ F .

PROPOSITION: For any short exact sequence 0 → M ′ f -→ M g -→ M " → 0, we have the important relation rk A (M ) = rk A (M ′ ) + rk A (M "), even in the non-commutative case. As a byproduct, if M admits a finite length free resolution ...

d2 -→ F 1 d1 -→ F 0 p -→ M → 0, we may define the Euler-Poincaré characteristic χ A (M ) = r (-1) r rk A (F r ) = rk A (M ).
The following classical proposition will be used later on for exhibiting the Ricci tensor and the Weyl tensor from the Riemann tensor: PROPOSITION: We shall say that the following short exact sequence splits if one of the following equivalent three conditions holds:

0 -→ M ′ u ←- f -→ M v ←- g -→ M ′′ -→ 0
• There exists a monomorphism v : M ′′ → M called lift of g and such that g

• v = id M ′′ .
• There exists an epimorphism u : M → M ′ called lift of f and such that u • f = id M ′ .

• There exist isomorphisms ϕ = (u, g) : M → M ′ ⊕ M ′′ and ψ = f + v : M ′ ⊕ M ′′ → M that are inverse to each other and provide an isomorphism

M ≃ M ′ ⊕ M ′′ with f • u + v • g = id M and thus ker(u) = im(v).
These conditions are automatically satisfied if M " is free or projective.

Using the notation M * = hom A (M, A), for any morphism f : M → N , we shall denote by

f * : N * → M * the morphism which is defined by f * (h) = h • f, ∀h ∈ hom A (N, A) and satisfies rk A (f ) = rk A (im(f )) = rk A (f * ), ∀f ∈ hom A (M, N ).
We may take out M in order to obtain the deleted sequence ...

d2

-→ F 1 d1 -→ F 0 -→ 0 and apply hom A (•, A) in order to get the sequence ...

d * 2 ←-F * 1 d * 1 ←-F * 0 ←-0.
PROPOSITION: If we define the extension modules Introducing the morphism ǫ : M → M * * such that ǫ(m)(f ) = f (m), ∀m ∈ M, ∀f ∈ M * and defining the differential module N from ad(D 1 ) exactly like we defined the differential module M from D, we finally notice that any operator is the adjoint of a certain operator because ad(ad(P )) = P, ∀P ∈ D and we get: THEOREM: (reflexivity test) In order to check whether M is reflexive or not, that is to find out a parametrization if t(M ) = 0 which can be again parametrized, the test has 5 steps which are drawn in the following diagram where ad(D) generates the CC of ad(D 1 ) and D ′ 1 generates the CC of D = ad(ad(D)) while ad(D -1 ) generates the CC of ad(D) and D ′ generates the CC of D -1 : EXAMPLE: When n = 3, the div operator can be parametrized by the curl operator which can be itself parametrized by the grad operator. However, using (ξ 1 , ξ 2 , ξ 3 = 0), we may obtain the new minimal parametrization

ext 0 A (M ) = ker(d * 1 ) = hom A (M, A) = M * and ext i (M ) = ext i A (M ) = ker(d * i+1 )/im(d * i ), ∀i ≥ 1,
η ′ ζ ′ 5 D ′ ր D ′ 1 ր 4 φ D-1 -→ ξ D -→ η D1 -→ ζ 1 3 θ ad(D-1) ←- ν ad(D) ←-µ ad(D1) ←- λ 2 D 1 parametrized by D ⇔ D 1 = D ′ 1 ⇔ ext 1 (N ) = 0 ⇔ ǫ injective ⇔ t(M ) = 0 D parametrized by D -1 ⇔ D = D ′ ⇔ ext 2 (N ) = 0 ⇔ ǫ surjective COROLLARY: In the differential module framework, if F 1 D1 -→ F 0 p -→ M → 0 is
-∂ 3 ξ 2 = η 1 , ∂ 3 ξ 1 = η 2 , ∂ 1 ξ 2 -∂ 2 ξ 1 = η 3 ⇒ ∂ 1 η 1 + ∂ 2 η 2 + ∂ 3 η 3 = 0 which cannot be again parametrized.
We shall finally prove below that the Einstein parametrization of the stress equations is neither canonical nor minimal in the following diagrams:

4 Killing -→ 10 Riemann -→ 20 Bianchi -→ 20 -→ 6 → 0 ↓ ↓ 10 Einstein -→ 10 div -→ 4 → 0 0 ← 4 Cauchy ←- 10 Beltrami ←- 20 ←- 20 ↑ 10 Einstein ←- 10
obtained by using the fact that the Einstein operator is self-adjoint (with a slight abuse of language), where by Einstein operator we mean the linearization of the Einstein equations over the Minkowski metric, the 6 terms being exchanged between themselves. Indeed, setting

E ij = R ij -1 2 ω ij tr(R) with tr(R) = ω ij R ij , it
is essential to notice that the Ricci operator is not self-adjoint because we have for example:

λ ij (ω rs d ij Ω rs ) ad -→ (ω rs d ij λ ij )Ω rs
and ad provides a term appearing in -ω ij tr(R) but not in 2R ij because we have:

tr(Ω) = ω rs Ω rs ⇒ tr(R) = ω rs d rs tr(Ω) -d rs Ω rs
The upper div induced by Bianchi has nothing to do with the lower Cauchy stress equations, contrary to what is still believed today while the 10 on the right of the lower diagram has nothing to do with the perturbation of a metric which is the 10 on the left in the upper diagram. It also follows that the Einstein equations in vacuum cannot be parametrized as we have the following diagram of operators recapitulating the five steps of the parametrizability criterion (One can provide a computer algebra exhibition of this result):

Riemann 20 ր 4 Killing -→ 10 Einstein -→ 10 4 Cauchy ←- 10 Einstein ←- 10
As a byproduct, we are facing only two possibilities, both leading to a contradiction: 1) If we use the operator S 2 T * Einstein -→ S 2 T * in the "geometrical " setting, the S 2 T * on the left has indeed someting to do with the perturbation of the metric but the S 2 T * on the right has nothing to do with the stress.

2) If we use the adjoint operator ∧ n T * ⊗ S 2 T Einstein ←-∧ n T * ⊗ S 2 T in the "physical " setting, then ∧ n T * ⊗ S 2 T on the left has of course something to do with the stress but the ∧ n T * ⊗ S 2 T on the right has nothing to do with the perturbation of a metric.

These purely mathematical results question the origin and existence of gravitational waves.

We shall finally prove below, from what has been said in the beginning of this Summary Note, that the parametrization of the Cauchy operator by ad(Ricci) is neither canonical nor minimal in the following diagrams:

←-10

COROLLARY: When n ≥ 4, as the extension modules are torsion modules, each component of the Weyl tensor is a torsion element killed by the Dalembert operator whenever the Einstein equations in vacuum are satisfied by the metric. Hence, there exists a second order operator Q such that we have an identity describing the so-called Lichnerowicz wave equations (in France !):

✷ • W eyl = Q • Ricci 4) COSSERAT VERSUS WEYL
Computing the formal adjoint ad(D 1 ) of the first Spencer operator D 1 induced by D for the group of rigid motions when n = 2, we get:

σ 11 (∂ 1 ξ 1 -ξ 1,1 ) + σ 12 (∂ 2 ξ 1 -ξ 1,2 ) + σ 21 (∂ 1 ξ 2 -ξ 2,1 ) + σ 22 (∂ 2 ξ 2 -ξ 2,2 ) + µ 12,r (∂ r ξ 1,2 -ξ 1,2r )
Integrating by parts with ξ 1,1 = 0, ξ 1,2 + ξ 2,1 = 0, ξ 2,2 = 0, we obtain at once and exactly the Cosserat couple-stress equations:

∂ 1 σ 11 + ∂ 2 σ 12 = f 1 , ∂ 1 σ 21 + ∂ 2 σ 22 = f 2 , ∂ r µ 12,r + σ 12 -σ 21 = m 12
allowing to have now a nonsymmetrical stress with a new first order parametrization:

σ 11 = ∂ 2 φ 1 , σ 12 = -∂ 1 φ 1 , σ 21 = -∂ 2 φ 2 , σ 22 = ∂ 1 φ 2 , µ 12,1 = ∂ 2 φ 3 + φ 1 , µ 12,2 = -∂ 1 φ 3 -φ 2 with a potential (φ 1 , φ 2 , φ 3 ) ∈ ∧ 2 T * ⊗ ∧ 2 T ⊗ R * 2 ≃ R * 2 with dim(R 2 ) = 3
. These equations can be extended by adding the only dilatation with infinitesimal generator x i ∂ i in order to provide the virial equations. Similarly, going along the idea pioneered by Weyl in 1918, we obtain:

J i (∂ i ξ r r -ξ r ri ) + F ij (∂ i ξ r rj -∂ j ξ r ri )
An integration by parts brings the operator ∂ r F ir -J i and there is thus no conceptual difference at all between the Cosserat equations and the second set of Maxwell equations.

As a next crucial step, let us consider the Lie group of transformations of X described by the action of a Lie group G with local coordinates (a τ ), identity e ∈ G and Lie algebra G = T e (G), on X with infinitesimal generators θ τ = θ k τ (x)∂ k and introduce the section

ξ q = λ τ (x)j q (θ τ ) with λ ∈ ∧ 0 T * ⊗ G. We have thus (ξ k (x) = λ τ (x)θ k τ (x), ξ k i (x) = λ τ (x)∂ i θ k τ (x), .
..) and get at once D 1 ξ q = Dξ q+1 = (dλ τ )j q (θ τ ) where d : ∧ 0 T * ⊗ G → ∧ 1 T * ⊗ G is the exterior derivative, a result proving that the Spencer sequence is (locally) isomorphic to the tensor product by G of the Poincaré sequence for d, in a coherent way with the second Example of the Introduction. As the extension modules of a module M do not depend on the resolution of M , it follows that, if D r+1 generates the CC of D r in a Janet sequence like in the Introduction, then ad(D r ) generates the CC of ad(D r+1 ) while, if D r+1 generates the CC of D r in the corresponding Spencer sequence, then ad(D r ) generates the CC of ad(D r+1 ), though all these operators are quite different, a result not evident at all that Lanczos and followers could have not even been able to imagine.

It remains to prove that, in this new framework, the Ricci tensor only depends on the symbol ĝ2 ≃ T * ⊂ S 2 T * ⊗ T of the first prolongation R2 ⊂ J 2 (T ) of the conformal Killing system R1 ⊂ J 1 (T ) with symbol ĝ1 ⊂ T * ⊗ T defined by the equations ω rj ξ r i + ω ir ξ r j -2 n ω ij ξ r r = 0 not depending on any conformal factor. In the next general commutative diagram covering both situations while taking into account that the PD equations of both the classical and conformal Killing operator are homogeneous, the Spencer map δ is induced by -D and all the sequences are exact but perhaps the left column:

0 0 0 ↓ ↓ ↓ 0 → g 3 → S 3 T * ⊗ T → S 2 T * ⊗ F 0 → F 1 → 0 ↓ δ ↓ δ ↓ δ 0 → T * ⊗ g 2 → T * ⊗ S 2 T * ⊗ T → T * ⊗ T * ⊗ F 0 → 0 ↓ δ ↓ δ ↓ δ 0 → ∧ 2 T * ⊗ g 1 → ∧ 2 T * ⊗ T * ⊗ T → ∧ 2 T * ⊗ F 0 → 0 ↓ δ ↓ δ ↓ 0 → ∧ 3 T * ⊗ T = ∧ 3 T * ⊗ T → 0 ↓ ↓ 0 0
THEOREM: Introducing the δ-cohomologies H 2 (g 1 ) at ∧ 2 T * ⊗ g 1 and H 2 (ĝ 1 ) at ∧ 2 T * ⊗ ĝ1 while taking into account that g 1 ⊂ ĝ1 , we have the commutative and exact "fundamental diagram II ":

0 ↓ 0 S 2 T * ↓ ↓ 0 -→ Z 2 (g 1 ) -→ H 2 (g 1 ) -→ 0 ↓ ↓ ↓ 0 -→ T * ⊗ ĝ2 δ -→ Z 2 (ĝ 1 ) -→ H 2 (ĝ 1 ) -→ 0 ↓ ↓ ↓ 0 -→ S 2 T * δ -→ T * ⊗ T * δ -→ ∧ 2 T * -→ 0 ↓ ↓ 0 0
showing that we have the splitting sequence 0 → S 2 T * → F 1 → F1 → 0 providing a totally unusual interpretation of the successive Ricci, Riemann and Weyl tensors. It follows that dim( F1 ) = n(n + 1)(n + 2)(n -3)/12 and the W eyl-type operator is of order 3 when n = 3 but of order 2 for n ≥ 4. Similar results could be obtained for the Bianchi-type operator.

5) JANET VERSUS SPENCER

Whenever R q ⊆ J q (E) is an involutive system of order q on E, we may define the Janet bundles F r for r = 0, 1, ..., n by the short exact sequences:

0 → ∧ r T * ⊗ R q + δ(∧ r-1 T * ⊗ S q+1 T * ⊗ E) → ∧ r T * ⊗ J q (E) → F r → 0
We may pick up a section of F r , lift it up to a section of ∧ r T * ⊗ J q (E) that we may lift up to a section of ∧ r T * ⊗ J q+1 (E) and apply D in order to get a section of ∧ r+1 T * ⊗ J q (E) that we may project onto a section of F r+1 in order to construct an operator D r+1 : F r → F r+1 generating the CC of D r in the canonical linear Janet sequence:

0 -→ Θ -→ E D -→ F 0 D1 -→ F 1 D2 -→ ... Dn -→ F n -→ 0
If we have two involutive systems R q ⊂ Rq ⊂ J q (E), the Janet sequence for R q projects onto the Janet sequence for Rq and we may define inductively canonical epimorphisms F r → Fr → 0 for r = 0, 1, ..., n by comparing the previous sequences for R q and Rq . A similar procedure can also be obtained if we define the Spencer bundles C r for r = 0, 1, ..., n by the short exact sequences:

0 → δ(∧ r-1 T * ⊗ g q+1 ) → ∧ r T * ⊗ R q → C r → 0
We may pick up a section of C r , lift it to a section of ∧ r T * ⊗R q , lift it up to a section of ∧ r T * ⊗R q+1 and apply D in order to construct a section of ∧ r+1 ⊗ R q that we may project to C r+1 in order to construct an operator D r+1 : C r → C r+1 generating the CC of D r in the canonical linear Spencer sequence which is another completely different resolution of the set Θ of (formal) solutions of R q :

0 -→ Θ jq -→ C 0 D1 -→ C 1 D2 -→ C 2 D3 -→ ... Dn -→ C n -→ 0
However, if we have two systems as above, the Spencer sequence for R q is now contained into the Spencer sequence for Rq and we may construct inductively canonical monomorphisms 0 → C r → Ĉr for r = 0, 1, ..., n by comparing the previous sequences for R q and Rq . When dealing with applications, we have set E = T and considered systems of finite type Lie equations determined by Lie groups of transformations and ad(D r ) generates the CC of ad(D r+1 ) while ad(D r ) generates the CC of ad(D r+1 ). We have obtained in particular C r = ∧ r T * ⊗ R q ⊂ ∧ r T * ⊗ Rq = Ĉr when comparing the classical and conformal Killing systems, but these bundles have never been used in physics. Therefore, instead of the classical Killing system R 2 ⊂ J 2 (T ) defined by Ω ≡ L(ξ)ω = 0 and Γ ≡ L(ξ)γ = 0 or the conformal Killing system R2 ⊂ J 2 (T ) defined by Ω ≡ L(ξ)ω = A(x)ω and Γ ≡ L(ξ)γ = (δ k i A j (x) + δ k j A i (x)ω ij ω ks A s (x)) ∈ S 2 T * ⊗ T , we may introduce the intermediate differential system R2 ⊂ J 2 (T ) defined by L(ξ)ω = Aω with A = cst and Γ ≡ L(ξ)γ = 0, for the Weyl group obtained by adding the only dilatation with infinitesimal generator x i ∂ i to the Poincaré group. We have R 1 ⊂ R1 = R1 but the strict inclusions R 2 ⊂ R2 ⊂ R2 and we discover exactly the group scheme used through this paper, both with the need to shift by one step to the left the physical interpretation of the various differential sequences used. Indeed, as ĝ2 ≃ T * , the first Spencer operator R2 The main result we have obtained is thus to be able to increase the order and dimension of the underlying jet bundles and groups, proving therefore that any 1-form with value in the second order jets ĝ2 (elations) of the conformal Killing system (conformal group) can be decomposed uniquely into the direct sum (R, F ) where R is a section of the Ricci bundle S 2 T * and the EM field F is a section of ∧ 2 T * . The mathematical structures of electromagnetism and gravitation only depend on second order jets.

  they do not depend on the resolution chosen and are torsion modules for i ≥ 1.We now turn to the operator framework with modules over the ringD = K[d 1 , ..., d n ] = K[d] of differential operators with coefficients in a differential field K with n commuting derivations (∂ 1 , ..., ∂ n ),also called D-modules. DEFINITION: If a differential operator ξ D -→ η is given, a direct problem is to find generating compatibility conditions (CC) as an operator η D1 -→ ζ such that Dξ = η ⇒ D 1 η = 0. Conversely, given η D1 -→ ζ, the inverse problem will be to look for ξ D -→ η such that D 1 generates the CC of D and we shall say that D 1 is parametrized by D if such an operator D is existing.

  a finite free presentation of M = coker(D 1 ) with t(M ) = 0, then we may obtain an exact sequence of free differential modules where D is the parametrizing operator. However, there may exist other parametrizationsF 1 D1 -→ F 0 D ′ -→ E ′ called minimal parametrizations such that coker(D ′) is a torsion module and we have thus rk D (M ) = rk D (E ′ ).

D1-→

  T * ⊗ R2 is induced by the usual Spencer operator R3 D -→ T * ⊗ R2 : (0, 0, ξ r rj , ξ r rij = 0) → (0, ∂ i 0ξ r ri , ∂ i ξ r rj -0)and thus projects by cokernel onto the induced operator T * → T * ⊗ T * . Composing with δ, it projects therefore onto T * d → ∧ 2 T * : A → dA = F as in EM and so on by using the fact that D 1 and d are both involutive, or the composition of epimorphisms:Ĉr → Ĉr / Cr ≃ ∧ r T * ⊗ ( R2 / R2 ) ≃ ∧ r T * ⊗ ĝ2 ≃ ∧ r T * ⊗ T * δ -→ ∧ r+1 T *


Zhiguo Zeng

Enrico Zio
email: enrico.zio@ecp.fr

A classification-based framework for trustworthiness assessment of quantitative risk analysis

Keywords: Quantitative Risk Analysis (QRA), validity, reliability, trustworthiness, Naive Bayes classifer Highlights

In this paper, we develop a classification-based method for the assessment of the trustworthiness of Quantitative Risk Analysis (QRA). The QRA trustworthiness is assumed to be determined by the quality of the QRA process. Six quality criteria, i.e., completeness of documentations, understanding of problem settings, coverage of accident scenarios, appropriateness of analysis methods, quality of input data, accuracy of risk calculation, are identified as the factors most influencing the trustworthiness. The assessment is, then, formulated as a classification problem, solved by a Naive Bayes Classifier (NBC) constructed based on a set of training data, whose trustworthiness is given by experts. NBC learns the expert's assessment from the training data: therefore, once constructed, the NBC can be used to assess the trustworthiness of QRAs other than the training data. Leaveone-out cross validation is applied to validate the accuracy of the developed classifier. A stochastic hypothesis testing-based approach is also developed to check the consistency of the training data. The performance of the developed methods is tested for ten artificially generated scenarios. The results demonstrate that the developed framework is able to accurately mimic a variety of expert behaviors in assessing the trustworthiness of QRA.

A classification-based framework for trustworthiness assessment of quantitative risk analysis

I. INTRODUCTION

Since its first application in nuclear power plants in 1975 [START_REF] Apostolakis | How useful is quantitative risk assessment?[END_REF], Quantitative Risk Analysis (QRA) has been widely applied in various fields to support safety-related decision making [START_REF] Zio | Industrial disasters: Extreme events, extremely rare. some reflections on the treatment of uncertainties in the assessment of the associated risks[END_REF], e.g., chemical process industry [START_REF] Khakzad | Vulnerability analysis of process plants subject to domino effects[END_REF], oil & gas industry [START_REF] Abimbola | Risk-based safety analysis of well integrity operations[END_REF], maritime transportation [START_REF] Goerlandt | Maritime transportation risk analysis: Review and analysis in light of some foundational issues[END_REF], nuclear installations [START_REF] Zio | Estimation of the functional failure probability of a thermalhydraulic passive system by subset simulation[END_REF], etc. Various methods have been developed for QRA [START_REF] Khan | Dynamic risk management: a contemporary approach to process safety management[END_REF]. According to Khan et al. [START_REF] Khan | Methods and models in process safety and risk management: Past, present and future[END_REF], in general, QRA methods are evolving from semiqualitative analysis to detailed quantitative analysis. For example, in the 1990s, QRA in the process industries were primarily based on semi-qualitative methods like hazard operatability (HAZOP) analysis [START_REF] Khan | Opthazopan effective and optimum approach for hazop study[END_REF], while recent QRAs are mainly based on detailed quantitative analysis methods, such as Bayesian network [START_REF] Yu | A flexible hierarchical bayesian modeling technique for risk analysis of major accidents[END_REF], bow-tie model [START_REF] Khakzad | Dynamic safety analysis of process systems by mapping bow-tie into bayesian network[END_REF], etc. How to compare the trustworthiness of different QRA methods, then, becomes an essential problem in QRA.

Trustworthiness of a QRA refers to the degree that a decision maker can trust the results of QRA [START_REF] Rosqvist | On the validation of risk analysis: A commentary[END_REF][START_REF] Rae | Fixing the cracks in the crystal ball: A maturity model for quantitative risk assessment[END_REF][START_REF] Pasman | Is risk analysis a useful tool for improving process safety?[END_REF]. This question is of paramount importance in practice: only a trustworthy QRA can be useful to support decision making. Trustworthiness assessment of QRA has been discussed by many researchers [START_REF] Suokas | Quality control in safety and risk analysis[END_REF][START_REF] Aven | Foundational issues in risk assessment and risk management[END_REF][START_REF] Goerlandt | On the reliability and validity of shipship collision risk analysis in light of different perspectives on risk[END_REF], although sometimes using different concepts and terminologies, e.g., evaluation [START_REF] Goerlandt | On the reliability and validity of shipship collision risk analysis in light of different perspectives on risk[END_REF], validation [START_REF] Goerlandt | Validity and validation of safety-related quantitative risk analysis: A review[END_REF], verification [START_REF] Graham | Verifiability isn't everything[END_REF],

quality assurance [START_REF] Suokas | Quality control in safety and risk analysis[END_REF], credibility assessment [START_REF] Busby | Credibility in risk assessment[END_REF], etc. Goerlandt et al. present a thorough survey on the status quo of the trustworthiness assessment of QRA [START_REF] Goerlandt | Validity and validation of safety-related quantitative risk analysis: A review[END_REF]. According to their survey, existing methods on QRA trustworthiness assessment can be broadly classified into four categories: benchmark exercise, reality check, independent peer review and quality assurance.

Benchmark exercise methods rely on the comparisons among several parallel analyses of QRA to determine its trustworthiness. Usually, two quality characteristics, i.e., reliability and validity (see [START_REF] Goerlandt | Validity and validation of safety-related quantitative risk analysis: A review[END_REF] for a detailed discussion), are considered in the comparisons. For example, the trustworthiness of the QRA on an ammonia storage facility is assessed by comparing seven benchmark exercises in terms of their outcomes, methodologies, data and models [START_REF] Lauridsen | Assessing the uncertainties in the process of risk analysis of chemical establishments: Part 1 and 2[END_REF]. Reality check methods assess the trustworthiness of the QRA by comparing the results with real data or operating experience of the same system or process [START_REF] Goerlandt | Validity and validation of safety-related quantitative risk analysis: A review[END_REF]. A typical example is presented in [START_REF] Sornette | Exploring the limits of safety analysis in complex technological systems[END_REF],

where statistical data of real accidents and incidents are compared to the risk indexes calculated by QRA, to evaluate its trustworthiness. In independent peer review methods, the process of QRA and its results are reviewed by independent experts, based on a series of predefined quality requirements, and the trustworthiness of the QRA is determined by the experts accordingly [START_REF] Goerlandt | Validity and validation of safety-related quantitative risk analysis: A review[END_REF]. Reference [START_REF] Garrick | On pra quality and use[END_REF] presents a typical example of independent review methods, where QRA is reviewed and its trustworthiness is determined by experts considering the following factors, i.e., objective and statement of purpose, project plan and scope of work, figures of merit, methodology, data base, results, implementation and application and verification of selected results. Quality assurance methods apply quality control techniques on each phase of the QRA process, aiming at ensuring the quality of the QRA process [START_REF] Goerlandt | Validity and validation of safety-related quantitative risk analysis: A review[END_REF]. It is assumed that a high-quality QRA process will yield trustworthy risk assessment results. For example, in [START_REF] Suokas | Quality control in safety and risk analysis[END_REF], Suokas and Rouhiainen summarized common flaws in each phase of the QRA process and developed a check-list-based approach to ensure their quality.

Among the four methods, benchmark exercise and reality check take a retroactive perspective on trustworthiness assessment, in the sense that their assessments are primarily done by comparing the results of the analysis with either parallel analyses or field data and experience; independent peer review and quality assurance, on the other hand, are proactive in the sense that instead of directly assessing the results, these methods evaluate the capability of the QRA process (in terms of its quality) and predict the trustworthiness of the assessment results based on the quality of the QRA process. In a sense, the retroactive perspective is preferred as it is more trustable, since it directly assesses the trustworthiness of the results of the analysis [START_REF] Aven | Model output uncertainty in risk assessment[END_REF]. However, two major shortcomings might limit its applicability. First, the retroactive methods are normally more difficult and expensive to implement, due to the requirements on field data or parallel analyses [START_REF] Goerlandt | Validity and validation of safety-related quantitative risk analysis: A review[END_REF]. Second, the retroactive methods tell us little about the contributing factors to the trustworthiness, which, limit the ability to guide improvements of the QRA process for improved trustworthiness [START_REF] Wang | Identification of protective actions to reduce the vulnerability of safety-critical systems to malevolent acts: a sensitivity-based decision-making approach[END_REF]. Since in practice, the strict requirements of the retrospective methods on field data or parallel analyses are always hard to fulfill, in this paper, we focus only on the proactive methods.

In the proactive methods, the trustworthiness is assessed (in fact, predicted) based on the capability (in terms of quality) of the QRA process. Two issues are essential when developing such methods:

• how to evaluate the capability (in terms of quality) of a QRA process?

• how to relate the trustworthiness of QRA to the capability of the QRA process?

The first issue has been addressed relatively well in literature: the major influencing factors for the quality of a QRA process have been widely discussed in literature [START_REF] Aven | Reliability and validity of risk analysis[END_REF][START_REF] Fenner-Crisp | Key elements for judging the quality of a risk assessment[END_REF][START_REF] Rouhiainen | Importance of the quality management of safety analysis[END_REF] and various methods have been developed for the assessment [START_REF] Pinto | Ensuring the quality of occupational safety risk assessment[END_REF][START_REF] Vergison | A quality-assurance guide for the evaluation of mathematical models used to calculate the consequences of major hazards[END_REF][START_REF] Rouhiainen | Quasa: A method for assessing the quality of safety analysis[END_REF]. The second issue, however, is not so well explored. In most existing researches ( [START_REF] Pinto | Ensuring the quality of occupational safety risk assessment[END_REF][START_REF] Vergison | A quality-assurance guide for the evaluation of mathematical models used to calculate the consequences of major hazards[END_REF][START_REF] Rouhiainen | Quasa: A method for assessing the quality of safety analysis[END_REF], for example), the relationship between the trustworthiness and the quality of the QRA process is treated as a black box and the experts are asked to directly construct a mapping from the process capability to the trustworthiness of QRA. Usually, this is done by a simple conformance/non-conformance-based framework: the conclusion of whether the QRA is trustworthy or not is made by comparing the number of the conformed quality criteria to a predefined threshold value [START_REF]Regulatory guide 1.200: An approach for determining the technical adequacy of probabilistic risk assessment results for risk-informed activities[END_REF]. Such a process is subject to several uncertainties, primarily due to the opacity in the elicitation process and lack of procedures to assess the accuracy of such assessments [START_REF] Wang | Assessing the performance of a classificationbased vulnerability analysis model[END_REF]. A formal and quantitative method is, therefore, needed for trustworthiness assessment.

In fact, trustworthiness assessment can be viewed as a classification problem (more broadly, supervised learning, see [START_REF] Bishop | Pattern Recognition and Machine Learning[END_REF] for details): train a classifier, which is a mapping from the capability of the QRA process to the trustworthiness of its results, based on a set of training data that are pre-assessed by experts. Hence, in this paper, we develop a classification-based framework, using Naive Bayes Classifier (NBC), for a formal and quantitative assessment of the trustworthiness of QRA. NBC is a simple but effective classifier widely applied in machine learning applications, e.g., text classification [START_REF] Mccallum | A comparison of event models for naive bayes text classification[END_REF], tumor diagnosis [START_REF] Kononenko | Inductive and bayesian learning in medical diagnosis[END_REF], etc. Although classification-based frameworks have been developed to assess other qualitative factors, e.g., vulnerability and safety criticality of nuclear power plants [START_REF] Wang | An empirical classification-based framework for the safety criticality assessment of energy production systems, in presence of inconsistent data[END_REF], prediction capability of prognostic methods [START_REF] Zeng | A hierarchical decision making framework for the assessment of the prediction capability of prognostic methods[END_REF], etc., to the best of our knowledge, it is the first time that a classification-based framework is developed for the assessment of QRA trustworthiness and that NBC is used for that purpose. It should be noted that in this paper, the classifier is not directly used to assess the trustworthiness. Rather, it is used as a tool for constructing the evaluation criteria used for determining the trustworthiness.

The rest of the paper is organized as follows. A general classification-based assessment framework of QRA trustworthiness is developed in Section II. NBC is applied in Section III to assess the trustworthiness of QRA.

In Section IV, we develop a method to check the consistency of the experts that generate the training data. Ten numerical case studies are considered and, then, an application is presented regarding a real trustworthiness assessment of QRA in Section V. The paper is concluded in Section VI, with a discussion on potential future developments.

II. ASSESSMENT FRAMEWORK

In this section, we present a general framework to support classification-based trustworthiness assessment of QRA. Let T represent the trustworthiness of QRA. We take a proactive perspective on trustworthiness assessment and assume that T is determined by the quality of the QRA process. According to Rae et al. [START_REF] Rae | Fixing the cracks in the crystal ball: A maturity model for quantitative risk assessment[END_REF], a typical QRA process involves eight sub-processes, as shown in Figure 1. To ensure the quality of a QRA process, all the eight sub-processes should be conducted with high quality [START_REF] Rae | Fixing the cracks in the crystal ball: A maturity model for quantitative risk assessment[END_REF]. A framework for trustworthiness assessment is, then, developed in Figure 2 by considering the quality requirements on the eight sub-processes in Figure 1. Fig. 1. A typical QRA process [START_REF] Rae | Fixing the cracks in the crystal ball: A maturity model for quantitative risk assessment[END_REF] In Figure 2, the trustworthiness of QRA is characterized in terms of six criteria, i.e., completeness of documentations (x 1 ), understanding of problem settings (x 2 ), coverage of accident scenarios (x 3 ), appropriateness of analysis methods (x 4 ), quality of input data (x 5 ), accuracy of risk calculation (x 6 ), which reflect the quality requirements on the QRA process. Each criterion is evaluated into three grades, i.e., problematic (x i = 0), acceptable (x i = 1) and satisfactory (x i = 2), i = 1, 2, • • • , 6, based on a set of predefined scaling rules in Table A.1-A.6. Three discrete levels of T , i.e., T ∈ {0, 1, 2}, are considered in this paper. The levels are distinguished in Table I based on their reliability, which concerns the repeatability of the risk analysis [START_REF] Aven | Reliability and validity of risk analysis[END_REF] Fig. 2. Trustworthiness assessment framework and validity, which concerns whether the risk analysis addresses the "right problem" [START_REF] Aven | Reliability and validity of risk analysis[END_REF]. The problem of trustworthiness assessment is, then, formulated as a classification problem: given the states of the six criteria

x 1 , x 2 , • • • , x 6
, determine an appropriate category for the trustworthiness T. It should be noted that both the assessment framework in Figure 2 and the scaling rules in Table A.1-A.6 are constructed for illustrative purposes.

They are defied in a general form that allows them to be adapted for capturing the problem-specific features in practical applications.

Levels of trustworthiness Descriptions

T = 0: Unreliable • The result of the QRA is unrepeatable.

• No further judgements can be made on the trustworthiness of the QRA.

• Such QRA should not be used to support any decision making.

T = 1: Reliable but invalid • The result of QRA is repeatable but
• some critical hazards are not identified and analyzed by the QRA or

• some important risks (and their uncertainties) are not accurately quantified by the QRA.

• Such QRA can be used to support decision making, but not for safety-critical decisions.

T = 2: Reliable and valid • The result of the QRA is repeatable and

• all critical hazards are identified and analyzed by the QRA;

• all important risks (and their uncertainties) are accurately quantified by the QRA.

• Such QRA can be used to support critical decision making.

III. TRUSTWORTHINESS ASSESSMENT BASED ON NAIVE BAYES CLASSIFIER

In this section, we first review some preliminaries on NBC-based classification in Subsection III-A and, then, develop a NBC-based method to assess the trustworthiness of QRA in Subsection III-B.

A. Naive Bayes classifier

Let us define x = [x 1 , x 2 , • • • , x n ] ∈ X to be the input feature vector of the classification problem, where X is the feature space. A NBC is a function f NBC that maps input feature vectors x ∈ X to output class labels T ∈ {0, 1, • • • , C} [START_REF]Naive bayes, guassian distributions and practical applicationss[END_REF]. Usually, the feature vector also takes discrete values, so that we have

x i ∈ {0, 1, • • • , n i }, i = 1, 2, • • • , n.
Given a feature vector x, a NBC classifies it into the class with the maximum posterior probability [START_REF]Naive bayes, guassian distributions and practical applicationss[END_REF]:

T = arg max T P r(T | x). (1)
The posterior probability in ( 1) is calculated using Bayes rule [START_REF]Naive bayes, guassian distributions and practical applicationss[END_REF]:

P r(T | x) = P r(x, T ) P r(x) = P r(x | T )P r(T ) C T =0 P r(x | T )P r(T ) . (2)
If we further assume that the elements x i , i = 1, 2, • • • , n of the input feature vector x are independent, the nominator of (2) becomes:

P r(x | T )P r(T ) = P r(T ) n i=1 P r(x i | T ). (3)
Note that the denominator in ( 2) is the same for all possible values of T . Therefore, (1) can be simplified:

T = arg max T P r(T ) n i=1 P r(x i | T ). (4)
In order to apply the NBC, the P r(T ) and P r(x i |T ) in ( 4) should be estimated from training data. Training data are a set of samples whose correct classes are already known. Suppose we have

N training training data, denoted by (x (q) , T (q) ), q = 1, 2, • • • , N training .
Then, the required probabilities are estimated by:

P r (T = k) = Ntraining q=1 1 T (q) = k N training , (5)
P r

(x i = j | T = k) = Ntraining q=1 1 x (q) i = j, T (q) = k Ntraining q=1 1 T (q) = k , (6)
where 1(•) is the indicator function and

i = 1, 2, • • • , n, j = 0, 1, • • • , n i , k = 0, 1, • • • , C.
There is one potential problem for ( 5) and [START_REF] Zio | Estimation of the functional failure probability of a thermalhydraulic passive system by subset simulation[END_REF]. Suppose that due to statistical variations, for some specific values of j and k, we have Ntraining q=1

1 x (q) i = j, T (q) = k = 0. In this case, P r (x i = j | T = k) = 0,
which, according to (3), results in P r(x | T ) = 0, regardless of the posterior probabilities for other features.

Misclassification often happens in such situations. To avoid such a problem, a technique called Laplacian correction is often applied when estimating P r (T = k) and P r

(x i = j | T = k) [39]: P r (T = k) = Ntraining q=1 1 T (q) = k + γ N training + (C + 1) • γ , ( 7
) P r (x i = j | T = k) = Ntraining q=1 1 x (q) i = j, T (q) = k + γ Ntraining q=1 1 T (q) = k + (n i + 1) • γ , (8)
where γ ∈ (0, 1] is an adjustment factor introduced to compensate for the possible zero probabilities; C + 1 and n i + 1 are the number of possible values for T and x i , respectively.

B. Trustworthiness assessment

In this section, we apply the NBC to develop a classifier for the trustworthiness assessment problem in Figure 2. In this case, we have six features, i.e., x = [x 1 , x 2 , • • • , x 6 ] T . Each feature has three discrete levels, i.e., 2) Construction of the classifier: The procedures for constructing the NBC is summarized in Figure 4. In the preparation phase, the sample size of the training set and the training data collection scheme are determined using the methods discussed previously. The training data (x (q) , T (q) ), q = 1, 2, • • • , N training are, then, collected by expert judgements following the scaling rules in Table A.1-A.6. In the training phase, the NBC is constructed by estimating P r(T ) and P r(x i |T ) from the training data, using [START_REF] Khan | Dynamic risk management: a contemporary approach to process safety management[END_REF] and [START_REF] Khan | Methods and models in process safety and risk management: Past, present and future[END_REF], respectively. In the evaluation phase, the constructed NBC is applied to replace the role of the experts and determine the trustworthiness of a new QRA. By reviewing the related documents, the value for the feature vector x of the QRA is determined first, based on the scaling rules defined in Table A.1-A.6. Its trustworthiness is, then, determined based on the constructed NBC using (4). [START_REF] Wang | Assessing the performance of a classificationbased vulnerability analysis model[END_REF][START_REF] Bishop | Pattern Recognition and Machine Learning[END_REF]. The procedures of implementing LOOCV is summarized in Figure 5, where (x (q) , T (q) ), q = 1, 2,

x i ∈ {0, 1, 2}, i = 1, 2, • • • , 6. Hence, X = {0, 1, 2} × • • • × {0, 1, 2} = {0,
•
In ( 9), H 0 and H 1 are the null hypothesis and alternate hypothesis, respectively. If the observed data can support us to reject the null hypothesis, the training data is believed to be consistent and can be used to train the NBC;

otherwise, we cannot trust the consistency of the training data and a reevaluation of the training data is required.

The CR CV estimated by LOOCV is chosen as the test statistic:

CR CV = Ntraining i=1 1 T (i) pred = T (i) N training , (10)
where

T (i)
pred and T (i) are the predicted and true classes of the ith cross validations, respectively. The empirical distribution of the test statistic under null hypothesis can be approximated using randomly generated training data. Suppose for a given significance level α, p 1-α denotes the (1 -α) percentile of the empirical distribution of the CR CV calculated using the randomly generated training data. Then, decisions on the consistency of the training data can be made by comparing p 1-α and CR data , which is calculated using the real training data [START_REF] Chen | Validation methodology for distribution-based degradation model[END_REF][START_REF] Devore | Probability and statistics for engineering and the sciences[END_REF]:

• If CR data > p 1-α , reject H 0 , the expert is consistent;

• otherwise, cannot reject H 0 , the expert is inconsistent.

The physical meaning of the significance level α is the probability that an inconsistent expert is mistakenly judged consistent by the test [START_REF] Devore | Probability and statistics for engineering and the sciences[END_REF]. Depending on the confidence requirements, two values of α are commonly used: α = 0.01 and α = 0.05.

April 12, 2017 DRAFT Algorithm 1 below presents a pseudo-code for the developed consistency-checking method. In Algorithm 1, n random is the sample size used to approximate the distribution of the test statistic. In principle, we prefer a large value of n random since it helps to reduce the uncertainties in the decision caused by the approximation errors in estimating p 1-α . However, balance is needed between the accuracy of the estimation and the required computational costs. If Algorithm 1 returns IsConsist = 1, we conclude that the training data under evaluation is consistent under the significance level α; otherwise, we cannot reach the conclusion that the training data are consistent and a re-evaluation of the training data is required.

Algorithm 1 Consistency verification based on hypothesis testing

Inputs: α, n random , x (q) , T (q) , q = 1, 2

T (q) random ← GENRNDSAMPLE, q = 1, 2, • • • , N training ; 3: CR (i) random
← Do LOOCV using the randomly generated training data (x (q) , T

random ), q = 1, 2, • • • , N training ; 4: end for 5: CR sort ← Sort CR random in ascending order; In this section, we first test the performance of the developed methods under ten artificially generated scenarios in Subsection V-A. Then, the methods are applied in Subsection V-B to assess the trustworthiness of a real-world methanol QRA.

6: p 1-α ← CR (nrandom•(1-α)) sort ; 7: CR (i) data ← Do LOOCV using real training data (x (q) , T (q) ), q = 1, 2, • • • , N training ; 8: if CR data > p 1-α then

A. A numerical case study

Ten scenarios are artificially generated to test the performance of the developed algorithms. For each scenario, we assume that the behavior of the experts when assessing the trustworthiness is consistent and known to us.

Ten different expert behaviors are used to generate the testing scenarios. We present one of the assumed expert behaviors in Figure 6 for illustration purposes.

Let x (i)

f ull represent all the possible states in the feature space X, where in this case, we have i = 1, 2, • • • , 729.

The true trustworthiness for each x (i) f ull , denoted by T (i) f ull , can, then, be determined by the same known assessment behavior. According to the designed training data collection scheme, a fraction of the x

(i) f ull , T (i) f ull
is selected as the training data and the remaining elements are used as test data to assess the performance of the developed NBC. Let the test data be x

(i) test , T (i) test , , i = 1, 2, • • • , N test .
For each artificially generated scenario, two numerical metrics are calculated:

• CR CV , calculated by LOOCV using the training data set, based on (10);

• CR test , the classification correctness rate, calculated using the test data set:

CR test = Ntest i=1 1 T (i) pred = T (i) test N test , (11)
The influence of the size of the training data set on the performance of trustworthiness assessment is also This is because when N training = 9, the training data set is too small for accurate estimation of the posterior probabilities using ( 7) and [START_REF] Khan | Methods and models in process safety and risk management: Past, present and future[END_REF]. To avoid such a problem, it is suggested to ensure that N training ≥ 18 in practical applications. QRAs. The quality criteria of the methanol QRA is evaluated by reviewing its final report, which is available online from [START_REF]Quantitative risk assessment: Final report ([END_REF]. The application follows the procedures in Figure 4 and the main results are summarized as follows.

1) Training data collection scheme: In this step, we design the training data collection scheme. From the discussion in Subsection V-A, we can see that N training = 54 can, in general, yield good classification accuracy.

Therefore, we choose N training = 54. The row-exchange algorithm in Matlab R2015b is used to design the training data collection scheme. The resulting x (q) , q = 1, 2, • • • , N training are listed in Table II. It can be verified that the training data collection scheme in Table II is an orthogonal design. The values of x (q) , q = 1, 2, • • • , N training correspond to the levels of the quality criteria in Table A.1-A.6. II represents a pseudo QRA, characterized by specific quality criteria. An expert is asked to assess the trustworthiness for these pseudo QRAs, for generating the training data. Take the first row in Table II as an example. To generate the training data, the expert is asked the following question: if the quality of a QRA process is as depicted in Table III, which level of trustworthiness in Table I do you think the QRA has? Table III is generated by relating the values of x (q) , q = 1, 2, • • • , N training to the corresponding quality criteria in Table A.1-A.6. The procedures are repeated for the other rows in Table II.

2) Training data collection: Each row in Table

The training data generated by the expert are also listed in Table II.

3) Consistency verification: The consistency of the expert is checked using Algorithm 1. In this case, we choose α = 0.01 and n random = 10 4 . The Empirical Cumulative Distributive Function (ECDF) of CR random is presented in Figure 10, where p 1-α = 0.519. The procedures in Figure 5 are used to calculate CR data using the real training data. Since CR data = 0.852 > p 1-α , according to Algorithm 1, we can conclude that the expert provided the training data consistently under the significance level α = 0.01.

(q) 1 x (q) 2 x (q) 3 x (q) 4 x (q) 5 x (q) 6 T (q) Runs x (q) 1 x (q) 2 x (q) 3 x (q) 4 x (q) 5 x (q) T (q) 1 0 0 0 0 1 0 0 28 1 1 1 2 1 1 2 0 0 0 0 2 2 0 29 1 1 2 1 0 1 3 0 0 1 2 1 1 0 30 1 1 2 2 1 1 4 0 0 1 2 2 2 0 31 1 2 0 1 2 1 5 0 0 2 1 0 1 0 32 1 2 0 2 0 1 6 0 0 2 1 1 1 0 33 1 2 1 0 1 1 7 0 1 0 1 0 0 0 34 1 2 1 0 1 1 8 0 1 1 0 2 1 0 35 1 2 2 0 2 1 9 0 1 1 2 0 0 0 36 1 2 2 2 0 2 10 0 1 2 0 0 0 0 37 2 0 0 0 0 0 11 0 1 2 0 1 2 0 38 2 0 0 2 1 0 12 0 1 2 2 2 0 0 39 2 0 1 1 0 1 13 0 2 0 1 1 1 0 40 2 0 1 2 2 1 14 0 2 0 1 1 2 0 41 2 0 2 0 0 0 15 0 2 0 2 2 0 0 42 2 0 2 2 2 2 16 0 2 1 0 2 1 0 43 2 1 0 0 0 0 17 0 2 1 2 0 2 0 44 2 1 0 1 2 1 18 0 2 2 1 0 2 0 45 2 1 0 2 1 1 19 1 0 0 0 1 2 0 46 2 1 1 1 1 1 20 1 0 0 2 0 1 0 47 2 1 1 1 2 1 21 1 0 1 1 0 2 1 48 2 1 2 0 1 1 22 1 0 1 1 1 0 1 49 2 2 0 0 0 0 23 1 0 2 0 2 1 1 50 2 2 1 0 2 1 24 1 0 2 1 2 0 1 51 2 2 1 1 0 1 25 1 1 0 1 2 1 1 52 2 2 2 1 1 1 26 1 1 0 2 2 2 1 53 2 2 2 2 1 2 27 1 1 1 0 0 0 0 54 2 2 2 2 2 2
4) Classifier construction: The training data are used to construct the NBC, following the procedures The constructed NBC can also help to explain the expert's behavior in assessing the trustworthiness. For Completeness of documentation Some the following elements are missing in the documentations:

• scopes and objectives of the QRA;

• descriptions of the system under investigation and related references;

• accounts of the adopted analysis methods;

• presentation of source data needed for the analysis;

• report of the analysis results.

Understanding of problem settings

The analysts are unaware of the problem settings of the QRA due to the presence of all the following flaws:

• the purposes of the QRA are not clearly understood;

• the systems of interests are not well defined;

• the resources constraints (e.g., time, computational resources, etc) are not clearly defined.

Coverage of accident scenarios Some critical accident scenarios are highly likely to be missed by the identification process:

• the coverage of the identified accident scenarios is not validated;

• the validation shows that some critical accident scenarios might be missing.

Appropriateness of analysis methods

• The features of the selected analysis method satisfy the requirements of the problem and

• successful applications in similar problems can justify the choice of the method.

Quality of input data

• There is no sufficient statistical data and the input data is purely based on expert judgements;

• epistemic uncertainty in the expert-generated input data is not considered.

Accuracy of risk calculation

• Only errors from the calculation process itself (e.g., the accuracy of Monte Carlo simulations) might exist and

• the uncertainties caused by the errors are properly modeled.

P r (T = 0 | x 1 = 0) = P r (x 1 = 0 | T = 0) • P r(T = 0) P r(x 1 = 0) = P r (x 1 = 0 | T = 0) • P r(T = 0) 3 i=1 P r (x 1 = 0 | T = i) • P r(T = i) (12)
= 0.9896

That is, if x 1 equals to zero, the expert tends to judge the QRA as unreliable. This is a natural result, since x 1 denotes the completeness of documentations. If the QRA process is not well-documented, it is unlikely to be repeatable: therefore, the associated QRA is unreliable according to the criteria in Table I.

5) Comparison to existing methods:

In traditional proactive trustworthiness assessment methods, e.g., [START_REF]Regulatory guide 1.200: An approach for determining the technical adequacy of probabilistic risk assessment results for risk-informed activities[END_REF],

expert knowledge is elicited to develop a simple conformance/non-conformance-based framework that relates the quality criteria to the trustworthiness of the QRA. That is, the conclusion of whether the QRA is trustworthy or not is made by comparing the number of the conformed quality criteria to a predefined threshold value n th .

In this paper, we assume that a quality criterion i is conformed when x i = 2. Table V shows a comparison between the classification-based framework and the conformance/non-conformance-based framework, using the training data in Table II. It can be seen that in general, the existing conformance/non-conformance-based framework cannot accurately model the complex expert knowledge expressed in the empirical data in Table II.

The developed method, on the other hand, is capable of capturing the complex behavior of expert judgement in assessing the trustworthiness of the QRA.

6) Trustworthiness assessment:

To assess the trustworthiness of the methanol QRA using the developed NBC, its six quality criteria are first evaluated based on the QRA report [START_REF]Quantitative risk assessment: Final report ([END_REF] and following the scaling rules in Table A.1-A.6:

• The scaling rule for completeness of documentation (x 1 ) is listed in Table A.1. In general, the methanol QRA report contains sufficient information on the scope and objective of the analysis, the system under investigation and the adopted analysis methods. However, according to Table A.1, the presentation of the analysis results is incomplete, since no accounts of uncertainty are given in the report. Therefore, we have

Methods

x 1 = 1.
• The scaling rule for understanding of problem setting (x 2 ) is listed in Table A.2. As reflected in the report, the analyst understood well the purposes of the QRA, the system under investigation and the resources constraints of the analysis. Therefore, we have x 2 = 2.

• The scaling rule for coverage of accident scenario (x 3 ) is listed in Table A.3. The methanol QRA identifies accident scenarios by an initial HAZard IDentification (HAZID) workshop. The identified accident scenarios are verified by expert reviews, conducted by experts from the QRA analysis team, the operator of the methanol plant and other organizations in related fields [START_REF]Quantitative risk assessment: Final report ([END_REF]. However, no field data are used to verify the accident scenarios. Therefore, we have x 3 = 1.

• The scaling rule for appropriateness of analysis method (x 4 ) is listed in Table A.4. The QRA is mainly based on Event Tree Analysis (ETA), which assumes that the occurrence probabilities of the intermediate events dop not change with time. In practice, however, the safety barriers of the methanol plant might be time-dependent due to the degradation of critical components or systems. ETA is not able to capture such time-dependence. Therefore, we have x 4 = 0.

• The scaling rule for quality of input data (x 5 ) is listed in Table A.5. The input data used in the QRA come from expert judgements based on handbooks (parts count reliability prediction standards and TNO "purple book" [START_REF]Quantitative risk assessment: Final report ([END_REF]). Epistemic uncertainty in the expert judgements on the input data is not considered.

Therefore, we have x 5 = 0.

• The scaling rule for accuracy of risk calculation (x 6 ) is listed in Table A.6. The risk calculation is based on ETA, which is an analytical method. Therefore, its accuracy can be ensured and we have x 6 = 2.

By running the NBC with the input feature vector x = [1, 2, 1, 0, 0, 2], we can calculate the posterior probabilities from (4), as shown in Figure 12. We can conclude that T = 1 for the QRA of the methanol plant, which means, according to Table I, that the QRA of the methanol plant is reliable but invalid. Such a QRA can be used to support decision making, but not for safety-critical decisions. An inherent assumption of NBC is that the features (criteria) are independent among one another. In practice, however, various dependencies might exist among the features. How to consider these dependencies needs to be explored in future researches in order to derive a more accurate classifier. Furthermore, uncertainties might affect the evaluation of the criteria x 1 , x 2 , • • • , x 6 , since their assessments involve a lot of subjective judgements. Future researches can be carried out to integrate these uncertainties in the assessment framework, with mathematical theories dealing with subjective uncertainties, e.g., evidence theory [START_REF] Shafer | A mathematical theory of evidence[END_REF], possibility theory [START_REF] Zadeh | Fuzzy sets as a basis for a theory of possibility[END_REF], uncertainty theory [START_REF] Liu | Uncertainty Theory[END_REF], etc.

Levels Descriptions

x 3 = 0 Some critical accident scenarios are highly likely to be missed by the identification process:

• the coverage of the identified accident scenarios is not validated;

• the validation shows that some critical accident scenarios might be missing.

x 3 = 1 • Validation reveals that most critical accident scenarios are covered by the identification process but

• the validation is conducted based on peer review rather than using real data.

x 3 = 2 • Validation reveals that most critical accident scenarios are covered by the identification process and

• the validation is reliable by using field data. Levels Descriptions

x 4 = 0
The features of the selected analysis method cannot satisfy the requirements of the problem:

• the methods cannot capture some features of the problem (e.g., time-dynamics, dependencies, etc.) or

• the methods require more resources (e.g., data, computational power, etc.) than that can be provided.

x 4 = 1

• The features of the selected analysis method satisfy the requirements of the problem but

• the conclusion is drawn based on expert experience.

x 4 = 2

• The features of the selected analysis method satisfy the requirements of the problem and

• successful applications in similar problems can justify the choice of the method.

Levels Descriptions

x 5 = 0 • There is no sufficient statistical data and the input data is purely based on expert judgements;

• epistemic uncertainty in the expert-generated input data is not considered.

x 5 = 1 • There is no sufficient statistical data;

• the input data are based on expert judgements with fully consideration of epistemic uncertainty.

x 5 = 2 • Sufficient statistical data can be used for risk analysis.

Levels Descriptions

x 6 = 0 • The process of risk calculation contains major flaws;

• large errors might exist in the calculated risks.

x 6 = 1

• The process of risk calculation does not contain major flaws;

• only errors from the calculation process itself (e.g., the accuracy of Monte Carlo simulations) might exist but

• the uncertainties caused by the errors are not modeled.

x 6 = 2 • Only errors from the calculation process itself (e.g., the accuracy of Monte Carlo simulations) might exist and

• the uncertainties caused by the errors are properly modeled.

1 , 2} 6 .

 16 The trustworthiness also takes three values, i.e., T ∈ {0, 1, 2}. In general, three steps are involved in the development of the classifier, as shown in Figure 3.

Fig. 3 .

 3 Fig. 3. Procedures of constructing the NBC for trustworthiness assessment

Fig. 4 .

 4 Fig. 4. NBC construction procedure for QRA trustworthiness assessment

Fig. 5 .

 5 Fig. 5. Procedures of implementing LOOCV

Fig. 6 .

 6 Fig. 6. Illustration of the assumed expert rationale

  investigated. For this, nine levels of N training , i.e., N training = 9, 18, 27, 36, 45, 54, 63, 72, 81 are considered and the row-exchange algorithm in Matlab R2015b is used to design the training data collection scheme for each value of N training .

Figure 7 and

 7 Figure 7 and Figure 8 present the values of CR CV and CR test , evaluated under different scenarios, using different numbers of training data. It can be seen from the two Figures that both CRCV and CR f ull are improved as the number of training data N training increases. Hence, we can improve the accuracy of the developed classifier by choosing a larger N training .Although the classification accuracies are affected by uncertainties arising from the difference in the expert judgement behaviors adopted in different testing scenarios, the developed NBC can, in general, achieve satisfactory accuracies even for small values of N training . As demonstrated in Figure7: the average CR test exceeds 0.9 for N training greater than 18. However, when N training = 9, the classification accuracy is relatively poor.

Fig. 7 .

 7 Fig. 7. CRtest evaluated under different scenarios, with different numbers of training data

Fig. 8 .Fig. 10 .

 810 Fig. 8. CR CV evaluated under different scenarios, with different numbers of training data

1 in Figure 4 .

 14 The estimated P r (T = k) and P r (x i = j | T = k) are presented in TableIV and Figure11, 2 respectively. The accuracy of the constructed classifier is evaluated by the correct classification rate and we 3 have CR = 0.944. Therefore, the constructed NBC can be used to represent the expert judgements and provide 4 reasonable assessment of the trustworthiness of QRA.

5

 5

6 example, from Figure 11 , 7 P r (x 1 = 2 | 8 April 12

 611712812 we notice that P r (x 1 = 0 | T = 0) = 0.6882, P r (x 1 = 0 | T = 1) = 0.0041, T = 0) = 0.0233. From Bayes theorem,

(a) T = 0 (b) T = 1 (c) T = 2 Fig. 11 .

 01211 Fig. 11. Estimated P r (x i = j | T = k)

  Correct classification rate Classification-based method CR = 0.944 Conformance-based method CR = 0.130, for n th = 0 CR = 0.315, for n th = 1 CR = 0.463, for n th = 2 CR = 0.556, for n th = 3 CR = 0.500, for n th = 4 CR = 0.500, for n th = 5 CR = 0.482, for n th = 6

Fig. 12 .

 12 Fig. 12. Posterior probabilities for each value of T

  , • • • , N training

	Onputs: IsConsist

1: for i = 1 : n random do 2:

TABLE V A

 V COMPARISON TO EXISTING METHODS

TABLE A

 A

	.3
	SCALING RULES FOR x 3

TABLE A

 A

	.4
	SCALING RULES FOR x 4

TABLE A

 A

	.5
	SCALING RULES FOR x 5

TABLE A

 A

	.6
	SCALING RULES FOR x 6

April 12, 2017 DRAFT

Levels Descriptions

x 1 = 0 Some the following elements are missing in the documentations:

• scopes and objectives of the QRA;

• descriptions of the system under investigation and related references;

• accounts of the adopted analysis methods;

• presentation of source data needed for the analysis;

• report of the analysis results.

x 1 = 1 At least one of the following flaws present in the documentations:

• descriptions of scopes and objectives are incomplete or ambiguous;

• descriptions of the system under investigation are unclear;

• no sufficient references on the system under investigation are given;

• descriptions of the adopted methods are unclear;

• presentations of the results are incomplete (e.g., no uncertainty is considered) or ambiguous.

The documentation of the QRA process contains sufficient information for its repetition:

• the documentation contains all the necessary parts;

• no flows in level x 1 = 1 present.

Levels Descriptions

x 2 = 0 The analysts are unaware of the problem settings of the QRA due to the presence of all the following flaws:

• the purposes of the QRA are not clearly understood;

• the systems of interests are not well defined;

• the resources constraints (e.g., time, computational resources, etc) are not clearly defined.

The analysts misunderstand part of the problem settings due to some of the following flaws:

• the analysts fail to clearly understand the purposes of the QRA;

• the analysts fail to clearly understand the systems of interests;

• the analysts fail to clearly understand the resources constraints.

x 2 = 2

The analysts clearly understand the problem settings: no flaws in levels x 2 = 0 and x 2 = 1 occur.