Piero Baraldi

Enrico Zio

A comparison between probabilistic and Dempster-Shafer Theory approaches to Model Uncertainty Analysis in the Performance Assessment of Radioactive Waste Repositories

Keywords: MODEL UNCERTAINTY, DEMPSTER-SHAFER THEORY, RADIOACTIVE WASTE REPOSITORIES

Model uncertainty is a primary source of uncertainty in the assessment of the performance of repositories for the disposal of nuclear wastes, due to the complexity of the system and the large spatial and temporal scales involved.

This work considers multiple assumptions on the system behaviour and corresponding alternative plausible modelling hypotheses. To characterize the uncertainty in the correctness of the different hypotheses, the opinions of different experts are treated probabilistically or, in alternative, by the belief and plausibility functions of the Dempster-Shafer theory.

A comparison is made with reference to a flow model for the evaluation of the hydraulic head distributions present at a radioactive waste repository site. Three experts are assumed available for the evaluation of the uncertainties associated to the hydro-geological properties of the repository and the groundwater flow mechanisms.

Acronyms

INTRODUCTION

Model uncertainty is considered a primary source of uncertainty in the assessment of the performance of repositories for the disposal of nuclear wastes. By definition, a model is a representation of a real system and uncertainties inevitably arise whenever there are possible alternative interpretations of the system and its phenomena, which are all plausible in light of the current knowledge of the system.

Methods to deal with model uncertainty include prediction expansion, model set expansion and formal Bayesian approaches. In prediction expansion [START_REF] Zio | Two methods for the structured assessment of model uncertainty by experts in performance assessments of radioactive waste repositories[END_REF] , a single model is chosen as the best one to represent the system. However, it is recognized that this model has drawbacks and may represent only some characteristics of the system better than others. Sensitivity studies are performed on the various assumptions to analyze the effects on the model output. The associated uncertainty is dealt with by applying a random adjustment factor to the model results. The adjustment factor may be multiplicative or additive, or both. However, it is not clear if the technical exercise of quantifying the gap between reality and the model (i.e., the quantification of the adjustment factor distribution) is feasible in practical analyses.

Furthermore, a question arises on the reasoning behind this approach: if information exists that leads the expert to say something about the error in the prediction of a model, its correction can be simply considered as a modification of the original model [START_REF] Apostolakis | A Commentary on Model Uncertainty[END_REF][START_REF] Buslik | A Bayesian Approach to Model Uncertainty[END_REF] .

In model set expansion, the characteristics of the system under consideration are analyzed and models are created in an attempt to emulate the system based on goodness-of-fit criteria [START_REF] Reinert | Including model uncertainty in risk-informed decision making[END_REF][START_REF] Winkler | Model Uncertainty: Probabilities for Models?[END_REF] .

The models may use different assumptions and require different inputs. Each model has its own advantages and disadvantages, including limitations on and ranges of applicability.

These models are then combined to produce a meta-model of the system.

Several methods have been proposed regarding the construction of this meta-model. They include mixture [START_REF] Buslik | A Bayesian Approach to Model Uncertainty[END_REF] , the NUREG-1150 approach [START_REF] Keeney | Eliciting Probabilities from Experts in Complex Technical Problems[END_REF] , the joint US/EC Probabilistic Accident Consequence Uncertainty Analysis (PACUA) approach [START_REF]US Nuclear Regolatory Commision and the Commision of European Communities, Probabilistic Accident Consequence Uncertainty Analysis[END_REF] , and the Technical Facilitator-Integrator approach [START_REF] Budnitz | Recommendations for Probabilistic Seismic Hazard Analysis: Guidance on Uncertainty and Use of Experts[END_REF] . All of these methods rely on expert opinion. This is particularly inevitable in the case of interest here of the performance assessment of radioactive waste repositories, for which the predictive capabilities of the models cannot be verified over the time frames and spatial scales which they are required to apply to. As a consequence, the analyst cannot obtain empirical confirmation of the validity of a model from observations, so that the evaluation of the model must rely exclusively on the subjective interpretation of the information available at the time of the analysis. This leads to the conclusion that any attempt to address the issue of model uncertainty in a quantitative manner will rely on the use of expert judgment. In the mixture approach, the set of plausible models and their probabilities of being correct are agreed upon by the experts. The output distributions of the models are then linearly combined, with weights corresponding to the probabilities of correctness. The model distributions should be presented to the analysts before they are combined, allowing a transparent look at the range of variability that are combined into the meta-model. In some works, the model set expansion approach has been framed within a Bayesian approach to the estimation of the required parameters and combination of the individual model outputs [START_REF] Pulcini | Accounting for model uncertainty in reliability analysis of mechanical repairable equipment[END_REF][START_REF] Groen | Foundations of probabilistic inference with uncertain evidence[END_REF][START_REF] Droguett | Bayesian methodology for model uncertainty using model performance data[END_REF] .

These approaches are theoretically very attractive due to their mathematical rigor and ability to incorporate both objective and subjective information in a probabilistic representation.

On the other hand, it has been argued that since a model is a simplification of reality, it is possible to conclude that "every model is definitely false" [START_REF] Morgan | Uncertainty, A Guide to Dealing With Uncertainty in Quantitative Risk and Policy Analysis[END_REF] ; therefore, it seems inappropriate to evaluate the probability that a model produces outputs that replicate perfectly the reality, because this is impossible by definition: the probability of a model being more or less correct can only be interpreted in relative terms, i.e. as the probability that the model produces outputs more or less close to reality, in comparison with other models. In this view, the alternative models can be included into a meta-model parameterized with an index parameter  whose values (1,2,…n) are associated to the different plausible models (M 1 , M 2 ,…, M n ) and the uncertainty about the model can be converted into uncertainty about the value of  , represented by a given probability distribution. In practice, probability distributions can be assessed over the appropriate model structure and reinterpreted as associated to the model index parameter; this also allows comparing the impact of the uncertainty about the model with other uncertainties.

An alternative to this probabilistic representation of the uncertainty on the model is offered by a representation based on the belief and plausibility functions of the Dempster-Shafer Theory (DST). This approach requires an expert to provide for each plausible model an interval of values, limited by a lower bound, called belief, representing the amount of belief that directly supports a given model and an upper bound, called plausibility and measuring the fact that the model could possibly be true "up to that value" because there is only so much evidence that contradicts it. The Dempster rule of combination can then be used for the aggregation of the uncertainties of different experts on the correctness of the models. Actually, given that the models emerge from multiple modelling assumptions on the various aspects of system behaviour, the uncertainty analysis is made at the hypothesis level and then propagated into the uncertainty characterizing the entire model.

Finally, more powerful, fully formal Bayesian approaches have also been proposed for the treatment of model uncertainty. For example, in [START_REF] Mosleh | Model uncertainty and its implications in probabilistic risk assessment[END_REF] the available models are treated as sources of information that can be used to predict the true value of the model output through the application of Bayes theorem. The proposed methodology allows for utilizing various types of information, including experimental data and subjective evidence and for updating the prior distribution of the model output when new information becomes available [START_REF] Droguett | Bayesian methodology for model uncertainty using model performance data[END_REF] .

In the present work, the model set expansion approach to model uncertainty has been embraced to handle a situation in which multiple assumptions are made on the behaviour of

MODEL SET EXPANSION APPROACH

Consider the objective of expressing the uncertainty associated with the model prediction of the uncertain scalar output y of a system, due to the existence of a set of n models  

The family of distributions { ) ( y F l

} represents the uncertainty in the unknown y due to uncertainty in the models' structure; it can be probabilistically combined in a summary measure by means of a standard Bayesian approach [START_REF] Apostolakis | A Commentary on Model Uncertainty[END_REF][START_REF] Apostolakis | The Concept of Probability in Safety Assessments of Technological Systems[END_REF] . Indicating with ) ( l M p the epistemic probability which expresses the analyst's confidence in the set of assumptions underpinning the l -th model (here the term epistemic is used to characterize the uncertainty due to incomplete state of knowledge), the unconditional aleatory distribution ) (y F is given by the standard Bayesian estimator:

   n l l l M p y F y F 1 ) ( ) ( ) ( (1)
This average value has often encountered objections when employed for decision-making.

The argument is that the decision maker should be aware of the full epistemic uncertainties

    l M p
. Their average value (1) can lead to erroneous decisions, particularly when the epistemic uncertainty is very large, since the average can be greatly affected by high values of the variable even though they may be very unlikely. In general, the entire distribution of the uncertainties should be presented to the decision-maker who may then choose his/her own criteria upon which to base the decisions.

Furthermore, the model set expansion approach is based on two fundamental assumptions: mutual exclusiveness and collective exhaustiveness of the set of models. While the first assumption can be often accepted in practice, excluding the case in which one of the model is a special case of another model, the second is often not met in practice because it requires that a perfect model not only exists but that it also be one of the n models considered. In general, the complexity of the phenomena is such that the list of plausible models considered is necessarily incomplete. Moreover, progress in understanding the physical laws underpinning the process under analysis and the increasing computational capabilities are such that models evolve in time.

REPRESENTATION OF MODEL UNCERTAINTY

In this work, the probabilistic framework for the representation of the uncertainty regarding the probability of model correctness is compared with the Dempster-Shafer framework.

The probabilistic approach to model uncertainty representation

Within a classical probabilistic approach, the uncertainty on the correctness of the alternative models l M is modelled by the distributions of values

  l l M P p 
, each one representing the probability that the l-th alternative is the correct one, n l ,..., 1 

. This representation can be mathematically formalized through the introduction of a categorical random variable described by the distribution f is often referred to as second-order uncertainty [START_REF] De Rocquigny | Uncertainty in Industrial Practice, a guide to quantitative uncertainty management[END_REF] .

The model set expansion approach of Section 2 can be modified to include this new source of epistemic uncertainty. In particular, the standard Bayesian estimator of the output distribution (eq.1) becomes:

1 2 1 1 2 1 1 ... ) ,..., , ( ) ( ) (             n n n l l l dp dp dp p p p f p y F y F (2)

The Dempster-Shafer approach to model uncertainty representation

In this Section the DS evidence theory [START_REF] Shafer | A Mathematical Theory of Evidence[END_REF] is considered for the representation of the epistemic uncertainty affecting the expert knowledge of the probability   , measures the fact that l M could be the correct model "up to that value" because there is only so much evidence that contradicts it.

From a general point of view, contrary to probability theory which assigns the probability mass to individual elementary events, the theory of evidence makes basic probability assignments (bpa)   A m on sets A (the focal sets) of the power set   Z P of the event space Z , i.e. on sets of outcomes rather than on single elementary events.

In more details,   A m expresses the degree of belief that a specific element x belongs to the set A only, and not to any subset of A . The bpa satisfies the following requirements:

          1 ; 0 , 1 , 0 :       Z P A A m m Z P m (3)
Also, probability theory imposes more restrictive conditions on the specification of the likelihood of events as a result of the requirement that the probabilities of the occurrence and non-occurrence of an event must sum to one.

As a result, while in probability theory a single probability distribution function is introduced to define the probabilities of any event or proposition, represented as a subset of the sample space, in evidence theory there are two measures of likelihood, belief and plausibility: the belief about events and propositions is represented as intervals, bounded by two values, belief and plausibility. The belief in a proposition (set) A is quantified as the sum of the probability masses assigned to all sets enclosed by it:

     A B B m A Bel ) ( (4)
Hence, it is a lower bound representing the amount of belief that directly supports the proposition at least in part. Plausibility is the sum of the probability masses assigned to all sets whose intersection with the proposition is not empty:

      0 ) ( A B B m A Pl (5)
Hence, it is an upper bound on the possibility that the proposition could be verified, i.e. it measures the fact that the proposition could possibly be true "up to that value" because there is only so much evidence that contradicts it.

THE MULTIPLE-ASSUMPTIONS MULTIPLE-EXPERTS MODEL UNCERTAINTY PROBLEM

Let us consider a model M that consists of a set of k assumptions i H , i=1,…k where each assumption can assume n i alternative plausible hypotheses

i i j n j H ,..., 1 , 
. Combining all the n i possible alternative hypotheses for each i-th assumption,

   k i i n n 1 model structures n l S l ,..., 1 , 
in principle arise. For example, for a model for computing the neutron flux distribution in a nuclear reactor, there can be an assumption [START_REF] Zio | Two methods for the structured assessment of model uncertainty by experts in performance assessments of radioactive waste repositories[END_REF] H on the spatial discretization of the reactor core in homogeneous regions, with hypothesis H corresponding to N regions, and an assumption 2 H on the number of groups of precursors and delayed neutrons, with 1

2 1  H and 5 2 2  H . This gives rise to 4 2 2 2 1        k i i m n model structures: ) , ( ), , ( ), , ( ), , ( 2 2 1 2 4 2 2 1 1 3 2 1 1 2 2 2 1 1 1 1 H H S H H S H H S H H S     . Suppose that i Q different experts i i q Q q E ,..., 1 , 
give their opinions on the correctness of the alternative hypotheses for each i-th assumption, i=1,...,k. The objective of this Section is to compare the probabilistic and the Dempster-Shafer approaches in the estimation of the probability of model correctness based on the expert opinions.

A systematic framework is proposed for the representation of the uncertainty associated to the alternative model structures, as evaluated by the model experts with reference to the multiple alternative hypotheses. The framework consists in the following two steps:

i) the aggregation of the opinions of the different experts on the correctness of a given hypothesis of assumption;

ii) the propagation of the uncertainties arising from the expert evaluation of the individual assumptions into the uncertainty characterizing the entire model structure.

Aggregation of expert opinions and propagation of uncertainties in the probabilistic framework

Assume that the i Q experts have evaluated the probabilities of correctness of the i n alternative hypotheses regarding assumption i H in the model. Different approaches exist for the combination of multiple expert opinions. Axiomatic approaches such as the linear opinion pool [START_REF] Stone | The opinion pool[END_REF] and the logarithmic opinion pool [START_REF] Clemen | Combining Probability Distributions From Experts in Risk Analysis[END_REF] are intuitive, easy to understand and calculate;

for these reasons, they are often preferred in practice when there is no clear evidence justifying the adoption of a different pooling.

In this work the logarithmic opinion pool is adopted since, contrary to the linear opinion pool, it satisfies the principle of external Bayesianity.

According to the logarithmic opinion pool, the combined probability distribution  

i j H p is:          i q Q q w i j q i j H P k H P 1 i n j ,..., 1  (6)
where   i j q H P is the expert q -th probability distribution for assumption i H , k a normalizing constant and q w the weight used to represent the relative quality of the different experts. In the simplest case, the experts are viewed as equivalent and the combined distribution is proportional to the geometric mean of the individual distributions. Supposing that 3 i Q  experts are available, the results of the aggregation is the same whether formula ( 6) is directly applied to all 3 experts probabilities   i j q H P or it is firstly applied to the probabilities given by two experts and then the obtained probability is combined with the remaining expert probability.

Although axiomatic approaches have been largely used for aggregation of expert opinions, it has been pointed out that they cannot satisfy simultaneously a number of seemingly compelling desiderata [START_REF] French | Group consensus probability distributions: A critical survey[END_REF] and no foundationally-based method for determining the weights in their formula [START_REF] Genest | Allocating the weights in the linear opinion pool[END_REF] have been proposed. For these reasons, in the last decades attention has shifted from the axiomatic approaches to the development of combination methods based on a clear Bayesian paradigm for aggregating information from experts [START_REF] Clemen | Combining Probability Distributions From Experts in Risk Analysis[END_REF] . However, Bayesian approaches are not considered in this work since the property of external Bayesianity of the logarithmic opinion pool aggregation method is considered sufficient for the purpose of a fair

      k i il k i i l l i i p H P M P 1 1 ) ( ) ( (7)
In case the experts are uncertain on the exact values of the probabilities ij p and thus provide the joint probability density functions ) ,..., , (

1 2 1  i n i i i p p p f
, then also the probability of

correctness of model ) ( , l l l M P p M
 is uncertain. From eq.( 7), the joint probability density function ) ,..., , (

1 2 1  n p p p f
can be obtained numerically.

Aggregation of expert opinions and propagation of uncertainties in the Dempster-Shafer framework

Within the DST, the assignment by the i Q independent experts can be aggregated through the Dempster Rule of combination [START_REF] Shafer | A Mathematical Theory of Evidence[END_REF][START_REF] Yager | On the Dempster-Shafer Framework and New Combination Rules[END_REF] .

For simplicity, let us consider two experts whose evidence on the value of an uncertain variable is expressed in terms of two sets of basic probability assignments (bpa's),

) ( ), ( 2 1 A m A m on the sets A of the power set ) (Z P of Z .
Assuming that the bpa functions on the frame of discernment are based on independent arguments and bodies of evidence [START_REF] Shafer | A Mathematical Theory of Evidence[END_REF] , aggregation of this evidence into a joint bpa ) ( 12 A m can be obtained by means of Dempster rule (23) :

        0 0 0 1 12 2 1 12         m A K C m B m A m A C B (8)
where the complementary normalization factor K is given by

        0 2 1 C B C m B m K (9)
According to [START_REF] Reinert | Including model uncertainty in risk-informed decision making[END_REF] and [START_REF] Winkler | Model Uncertainty: Probabilities for Models?[END_REF] above, the degree of evidence ) . This way of combining evidence sources is analogous to the way in which in probability theory joint probability density functions (pdfs) are calculated from two independent marginal pdfs, and is thus justified on the same grounds. However, some intersections

i i C B 
of different focal elements i B and i C , from the first and second 18 expert, may result in the same set i A so that one must sum their product contribution to obtain

) ( 12 A m
. Furthermore, some of the intersections may be the empty set, for which 0 ) 0 ( 12  m .

Then, introducing K as the sum of products

    C m B m
   ) ( 1 ) ( Z P A A m
) is obtained by dividing by

K  1
. As K is determined by the sum of products of the bpa's of all sets where the intersection is null [START_REF] Winkler | Model Uncertainty: Probabilities for Models?[END_REF] , it represents the basic probability mass associated with conflict; it is a measure of the amount of conflict between the two mass sets and the normalization factor

K  1
has the effect of completely ignoring conflict by attributing any mass associated with conflict to the null set [START_REF] Yager | On the Dempster-Shafer Framework and New Combination Rules[END_REF] . Consequently, counterintuitive results are obtained in the face of significant conflict among experts, which has raised serious criticism to the formula [START_REF] Yager | Quasi-Associative Operations in the Combination of Evidence[END_REF] . Other assignment rules have been proposed to more properly represent the conflict [START_REF] Yager | Quasi-Associative Operations in the Combination of Evidence[END_REF][START_REF] Ferson | Representation, Propagation and Aggregation of Uncertainty[END_REF] ; this, however, is out of the scope of the present work.

If 2  i Q
experts are available, then their opinion can be aggregated by applying 1  i Q times the Dempster Rule (eq.( 4)). Notice that, since the Dempster-rule satisfies the associativity property

      C B A C B A     
, the order in which it is applied does not influence the final result [START_REF] Dempster | Upper and Lower Probabilities Induced by a Multi-valued Mapping[END_REF] .

The problem of the propagation of the uncertainty in the assumptions onto the uncertainty in the model structure is here addressed by considering the general problem of aggregating uncertainties on independent variables. In this respect, let us introduce k independent uncertain variables i z defined on their UODs

k i Z i ,..., 1 , 
and let us suppose that there are k experts, the i-th of them providing the bpa

  i i A m for each ) ( i i Z P A 
. According to the DST, the joint belief and plausibility of an element

  k A A A A ,..., , 2 1  of the power set of k Z Z Z Z × … × × = 2 1
are defined by:

  k i i i k A Bel A A A Be A Bel 1 2 1 ) ( = ) , … , , l( = ) ( (10)
  k i i i k A Pl A A A P A Pl 1 2 1 ) ( = ) , … , , l( = ) ( (11)
From the obtained belief and plausibility functions it is possible to obtain the basic probability

assignment     k A A A m A m ,..., , 2 1
 by ( 25):

       A B B A card B Bel A m ) ( 1 ) ( ) ( (12)
An alternative way that permits to directly compute the bpa of an element ) ,..., , (

2 1 k A A A A  of   Z P
is based on the application of the Dempster rule to the cylindrical extensions of the power sets elements [START_REF] Haenni | Implementing belief function computations[END_REF] . The cylindrical extension

 i A of an element i A of i Z over Z is
defined by considering for the variables

k i i x x x x x
,..., , ,..., ,

1 1 2 1   their entire UODs:   k i i i i Z Z A Z Z A ,..., , , ,..., 1 1 1     (13)
Given that the knowledge of the i-th expert is assumed to be only on the variable i z , he/she is not able to assign specific masses to any subsets j A of i Z , i j  . For this reason, it is intuitive for the i-th expert to associate to the cylindrical extension  i A defined on Z the same mass given to i A on the UOD i Z :

    i i i i A m A m   ( 14
)
whereas all the other elements of   Z P that are not cylindrical extensions of an element of i Z have null mass:

  0  A m i if   i i Z P A  such that A A i   (15)
Finally, for the aggregation of the masses given by different experts, it is possible to apply the Dempster rule to the cylindrical extensions. For example, considering two discrete independent variables that, due to eqs.( 14) and ( 15), becomes:

                     0 2 1 ) , (
                          0 2 2 1 1 ) , ( 2 2 1 1 2 1 12 2 1 2 1 2 1 1 ) , ( C B A A C B C m B m C m B m A A m ( 17
)
Furthermore, since in the case here discussed the set   (18) leading to the same results of the direct product of the belief and plausibility functions (eqs.( 10) and ( 11)).

        2 2 1 1 2 2 1 1 2 1 12 ) , ( A m A m A m A m A A m      
Notice, however, that eq.( 17) can be applied also to the case in which the two experts assign a bpa on overlapping UODs whereas both its simplified version of eq.( 18) and the method of eqs.( 10) and (11) are not applicable to this case. For example, if Expert 1 assigns a bpa to the elements   can then be seen as an element of the Cartesian product of the UODs i X , i.e.

 

n k 2 1 M M M X × … × X × X = X ,..., , 2 1 
. If we suppose that there are k experts, the i-th of them providing the bpa  

i i A m for each   i i X P A 
, it is then possible to compute the belief and plausibility of the model l M with structure ) ,..., , (

2 1 2 1 k l l l l k H H H S 
by applying eqs.( 10) and ( 11):

      k i i l i l i H Bel = M Bel 1 (19)     k 1 i i l i l ) H ( Pl = M Pl i 22
or the mass by eq.( 17):

    k 1 i i l i l ) H ( m = M m i (20)
Notice that by using eqs.( 10), (11) or ( 18) also the belief, plausibility and masses of the elements of the power set of formed by more than one model can be directly obtained.

CASE STUDY

Various conceptual models have been proposed for describing the groundwater flow and solute transport at a radioactive waste repository site in unsaturated, fractured tuff formations.

The analysis of the differences in the models has been mainly concerned with the fundamental assumptions underpinning their structures and their implications on the overall assessment by the models [START_REF] Gallegos | Preliminary Assessment of the Impact of Conceptual Model Uncertainty on Site Performance, High Level Radioactive Waste Management[END_REF][START_REF] Parsons | Conceptualization of a Hypothetical High-Level Nuclear Waste Repository Site in Unsaturated Fractured Tuff[END_REF] . In the analysis, all of the models have been considered based on the same system geometry. A spatially uniformly distributed flux at the repository horizon equal to 0.1 mm/yr is assumed as boundary condition for all models. Steady-state groundwater flow through a one-dimensional system, occurring from the base of the repository to the water table, is also assumed. Only four radionuclide species, Tc-99, I-129, Cs-135 and Np-237, are assumed to be released from the repository and transported as solute. The transport model is based on the assumption of a single, dominant, non-branching transport path. Radionuclide retardation factors were based on the distribution coefficient K d for each radionuclide. No other chemical reactions were included. Gaseous phase transport was not included either. The quantity considered as output of the model in this work is the hydraulic head distribution. In the analysis only the uncertainty due to the model structure is considered while the uncertainty on the model parameters is completely ignored.

With respect to the assessment of groundwater flow, the hypotheses made on two main, general assumptions drive the modeling: 1 H = "hydraulic and geological properties" and 2 H = "groundwater flow mechanisms". Tables I and II report the alternative hypotheses considered for the two assumptions on groundwater flow [START_REF] Zio | Two methods for the structured assessment of model uncertainty by experts in performance assessments of radioactive waste repositories[END_REF] . As a first case, consider that two experts have been asked to quantify the probabilities of correctness of the alternative hypotheses of the two assumptions 1 H and 2 H . In particular, Expert 1 gives a judgment only on the correctness of the alternative hypotheses of the first assumption [START_REF] Zio | Two methods for the structured assessment of model uncertainty by experts in performance assessments of radioactive waste repositories[END_REF] H whereas Expert 2 on the probability of correctness of the alternative hypothesis for the second assumption 2 H . Since the opinions elicited from the experts regard two different assumptions of the model, there is no overlapping between their knowledge and thus one needs only to tackle the problem of propagating the uncertainty in the assumptions onto the model. Let us also assume that the expert does not possess sufficiently refined knowledge of the system to express his/her belief on the correctness of the alternative hypotheses by discrete probability values, but that he/she feels more confident in giving an interval of possible values for the probabilities of correctness. For example, in evaluating the two different alternative hypotheses for the assumption 1 H , Expert 1 judges that the probability   2 H P is between 0.03 and 0.10.

Application of the probabilistic framework

For the assumption 1 H , the categorical random variable is binomial with possible realizations 1 1 H and 1

2

H whose probabilities are:

  11 1 1 1 p H P  (21)   12 1 2 1 p H P 
The expert assignment to the only parameter

  1 1 1 11 H P p 
of the binomial distribution is affected by epistemic uncertainty; in particular, the expert suggests only the two limiting values of 0.6 and 0.95 for 11 p , with no additional preference for values within the range, so that it seems reasonable to represent the uncertainty by a uniform probability density function   11 p f on the range defined by the limiting values (Figure 1). With respect to the assumption 2 H , the probability, assigned by Expert

            2 3 1 2 6 2 3 1 1 5 2 2 1 2 4 2 2 1 1 3 2 1 1 2 2 2 1 1 1 1 , , , , , , , , , , , H H M H H M H H M H H M H H M H H M      
Assuming the independence of the two assumptions [START_REF] Zio | Two methods for the structured assessment of model uncertainty by experts in performance assessments of radioactive waste repositories[END_REF] H and 2 H , it is possible to compute the probability of correctness of the 6 models which emerge from the combination of the 2·3 alternative hypotheses, by simply multiplying the probability of correctness of the constituting alternative hypotheses: From the obtained joint probability density function  

      1 21 11 2 1 2 1 1 1 1 12 p p p H P H P M P    (23)         2 21 11 2 1 2 1 2 1 2 12 1 p p p H P H P M P           3 22 11 2 2 2 1 1 1 3 12 p p p H P H P M P            4 22 11 2 2 2 1 2 1 4 12 1 p p p H P H P M P             5 22 21 11 2 3 2 1 1 1 5 12 1 p p p p H P H P M P               6
5 4 3 2 1 , , , , p p p p p f
, it is then possible to estimate the probability of correctness of the models by:

      5 ,..., 1 , , , , | ) ( 5 4 3 2 1 5 4 3 2 1      i dp p f p dp dp dp dp dp p p p p p f p M P M P i i p i i i i i (24)      

Application of the DST framework

The UOD 1 X of the assumption 1 H is formed by two elements, ; therefore, the definition of the corresponding bpa requires assigning a probability mass to all the three elements of the power set of 1

X , i.e.       1 2 1 1 1 2 1 1 1 , , , H H H H X P  .
The opinion provided by the expert that the probability of the correctness of alternative 1

1

H is a value between 0.6 and 0.95 leads to consider:

  60 . 0 1 1 1  H Bel (26)   95 . 0 1 1 1  H Pl
From this assignment, using the relation:

    1   A Pl A Bel ( 27
)
the belief and plausibility of 1 2 H are obtained:

  05 . 0 1 2 1  H Bel (28)   95 . 0 1 2 1  H Pl
From eq.( 12) the following masses of the elements of the power set   1 X P are then obtained:

60 . 0 ) ( 1 1  H M m (29) 05 . 0 ) ( 2 1  H M m 35 . 0 ) , ( 2 1 1  H H M M m
The second assumption [START_REF] Apostolakis | A Commentary on Model Uncertainty[END_REF] H is characterized by a UOD 2 X formed by the three alternatives (30) whereas in order to respect the constraints of a bpa, it is assumed that:

. 0 ) ( , 03 . 0 ) ( 80 . 0 ) ( , 60 . 0 ) ( 2 2 2 2 2 2 2 1 2 1 1 2     H Pl H Bel H Pl H Bel
35 . 0 ) ( , 15 . 0 ) ( 2 3 2 2 3 2   H Pl H Bel ( 31
)
These values of belief and plausibility of the elements of the frame of discernment lead to the following bpa:

60 . 0 ) ( 2 1 2  H m (32) 03 . 0 ) ( 2 2 2  H m 15 . 0 ) ( 2 3 2  H m 02 . 0 ) , ( 2 2 2 1 2  H H m 15 . 0 ) , ( 2 3 2 1 2  H H m 02 . 0 ) , ( 2 3 2 2 2  H H m 03 . 0 ) , , ( 2 3 2 2 2 1 2  H H H m
The probability masses elicited from Experts 1 and 2 can be finally combined according to eq.( 18). Table III reports bpa, belief and plausibility of the subsets of the frame of discernment  

6 5 4 3 2 1 , , , , , M M M M M M
formed by a single element.

Notice that there is a considerable mass of

      6 1 46 . 0 1 i i M m
assigned to the remaining subsets, i.e. a mass that cannot be attributed to a single model. It is also interesting to note that models

  2 1 1 1 1 , H H M  and   2 1 1 2 2 , H H M 
are characterized by a large difference between their belief and plausibility. This is caused by the fact that assumption 1 H is characterized by a large amount of mass not allocated to a single hypothesis. For example, Expert 1 judges that the set   , H H has a mass of 0.35; this means that there is a considerable evidence that one of these two models is correct but it is not possible to specify which one. For this reason, the strength of the belief in the models 1 M and 2 M is limited by the fact that it contains only the mass that can be attributed to the individual model, whereas the plausibility has an higher value because it includes all the mass that is not against such model and so it contains also the relatively large mass attributed to  

2 1 1 1 , H H .
This effect of large uncertainty on the beliefs in the correctness of the models is less evident for the other models because the masses on the alternatives H are lower than

) ( 2 1 H m .
Finally notice that

    6 3 M Bel M Bel  whereas     6 3 M Pl M Pl 
. Also this effect depends on the fact that there is a mass of 0.15 that Expert 2 assigns to  

Prediction of the hydraulic head

In order to further compare the probabilistic and the DS frameworks, the estimation of the model output, i.e. the hydraulic head at d=90 meters is here considered. The output values of

models 6 5 3 1 , , , M M M M
have been taken from [START_REF] Zio | Two methods for the structured assessment of model uncertainty by experts in performance assessments of radioactive waste repositories[END_REF] , whereas, since models 2 M and 4 M have not actually been developed, their predictions have been arbitrarily set. The case in which some models, based on particular combinations of assumptions, are not developed will be further addressed in Section 5.4. Here, since by observing the predictions of models   of the corresponding model i M (eq.( 25)). Figure 4 reports the discrete cumulative distribution representing the probability that the hydraulic head distribution at 90 metres is minor or equal to z . Furthermore, the expected value of the hydraulic head can be computed by: (Table III). In particular, it is interesting to compute, using eqs.( 4) and ( 5), the belief and plausibility of the sets   6 ,..., 1 ,   i z z i which can be interpreted as limiting cumulative density distributions (Figure 5).

      817 , , , , , , , , |
Notice that the probabilities   i z z P  obtained by applying the probabilistic framework are always within the interval of the corresponding belief and plausibility values. Thus, although the two approaches differ in the representation and propagation of the expert uncertainty on the probability of the model to be correct, in this case they lead to coherent results.

Finally, notice that synthetic measures of the expected value of the hydraulic head obtained from the belief and possibility distributions have been proposed in [START_REF] Smets | Constructing the pignistic probability function in a context of uncertainty[END_REF] , but they are not investigated in this work since decision makers usually demand to know the entire distribution of the uncertainties.

Model uncertainty in presence of non developed models

In this Section, the problem of aggregating the uncertainties of different experts on the correctness of the models is addressed on a case in which some possible models are not developed, although they have non zero probability of being correct for some experts. This case may arise when it is decided that some theoretically plausible models should not be developed due, for example, to the opinion of an expert or other practical limitations.

In particular, the previous case study is re-considered with the addition of the opinion of a third expert that judges models 2 M and 4 M surely not plausible. Furthermore, it is assumed that Expert 3 does not give any information to specify which of the remaining models

6 5 3 1 , , , M M M M
is the correct one.

Probabilistic framework

According to the probabilistic framework, Expert 3 opinion is represented by introducing a multinomial variable with possible realization (36) where the value of 6 is the normalization constant. From this a-priori distribution of the parameters of our multinomial distribution, the following estimation of the probabilities of the six models are obtained: According to this interpretation, the total lack of information on which model between

            25
6 5 3 1 , , , M M M M
is correct translates in the assignment of equal probabilities to the four models.

For the aggregation of the probabilities on the correctness of the 6 models given by Expert 3 with the probabilities obtained by combining the opinions of Expert 1 and Expert 2 reported in eq.( 25), the logarithmic opinion pool method is used, assigning in eq.( 6) equal weights to () PM, leading to the paradox that a non-zero probability is assigned to a model that is actually not developed.

          6 5 4 3 2 1 6 5 3 1 3 6 5 3 1 3 , , , , , , , , 0 1 , , , M M M M M M P M M M M A A m M M M M m      (39)
Notice that this representation of Expert 3 uncertainty on the models correctness is completely coherent with the information that he/she gives. In particular, through eqs.( 4) and ( 5), it is possible to show that it is equivalent to assign belief zero and plausibility 1 to all the subsets containing one of the four models ) , , , (

6 5 3 1 M M M M
.

By combining 3 m with 12 m using eq.( 8), a new bpa 123 m characterized by mass equal to 0 for all the subsets that contain 2 M and/or 4 M is obtained. M . Thus, analogously to the probabilistic case of the previous Section, the four models

6 5 3 1 , , , M M M M
form an exhaustive set of models and the previous observation made on the limitation of the exhaustiveness hypothesis still arises. In this respect, it is interesting to point out the possibility, within the DST framework, to assume that the frame of discernment of an uncertain quantity is formed by all its known possible realizations and by an unknown element (30), (31). This approach, that seems very promising to address the hypothesis of exhaustiveness of the models, will be further investigated in future works. It is also interesting to consider that within the DST approach, contrary to the probabilistic approach, Expert 3 opinion causes an increase of 63% of the masses assigned to 1 M by Experts 1 and 2. This is due to the fact that a considerable part of the mass With respect to the future development of the work, notice that the model set expansion approach is based on the two assumptions of mutual exclusiveness and collective exhaustiveness of the set of models: while the first assumption may be acceptable, upon verification, the second is certainly not met in practice because it requires that the perfect model is one of the models taken into account, which is not realistic. This aspect deserves further study in the future; in this respect, it seems worth considering modifications to the

X

  th model developed within the model set expansion approach n total number of possible models y quantity to be predicted by the model x input of the model (vector sign omitted for ease of representation) on which the model M is based i H i-th assumption on which a model is based i Universe Of Discourse of the assumption i H i n number of available alternative hypotheses for the i-th assumption i j H j-th alternative hypothesis for the i-th assumption

  the system, giving rise to corresponding alternative plausible modelling hypotheses whose correctness is judged by different experts; a comparison of the classical probabilistic and Dempster-Shafer theory of Evidence frameworks of uncertainty representation is performed. Formal Bayesian methods are not considered in the comparison since the focus is on the proper setting of the Dempster-Shafer representation and it is assumed that all the information on the correctness of the hypotheses is available at the moment of the analysis, with no need of subsequently updating the probability distribution of the model output. With respect to the application of the Dempster-Shafer theory of Evidence framework within the model set expansion approach, the belief and plausibility functions of the Dempster-Shafer theory of Evidence are used for the representation of the uncertainty in the correctness of the different hypotheses; then, the Dempster rule of combination is used for i) the aggregation of the uncertainties of different experts on the correctness of the hypotheses and ii) the propagation of the uncertainties regarding the individual assumptions into the uncertainty characterizing the entire model. The proposed approach is applied to a groundwater flow model for the evaluation of the hydraulic head distributions present in the far field region of a radioactive waste repository. Two classes of model assumptions are required, regarding the hydro-geological properties of the media in which the repository is located and the groundwater flow mechanisms. Three fictitious experts evaluate the associated uncertainties: the first one gives an evaluation of the correctness of the plausible alternative hypotheses for the first assumption on the geological properties, the second one for the second assumption on the groundwater flow mechanisms and the third one gives directly an opinion on the correctness of the overall groundwater flow models developed. The paper is organized as follows. In Section 2, a brief overview of the model set expansion is given. Section 3 discusses the representation of model uncertainty by the probabilistic and the Dempster-Shafer (DS) frameworks. In Section 4, the problems of aggregating the uncertainties of different experts and of propagating the uncertainties in the model assumptions onto the uncertainty characterizing the entire model are addressed by a probabilistic and a DS approach. In Section 5 the proposed uncertainty representation, aggregation and propagation techniques are applied to a groundwater flow model for the evaluation of the hydraulic head distributions present at a radioactive waste repository. Some conclusions on the advantage and limitation of the proposed method are drawn in the last Section.

S

  , l=1,...,n are based on different hypotheses all plausible in light of the existing information on the system behaviour. of y provided by the generic model, , is conditional on the structure of the model as well as on the values of its internal parameters.

  to the assumptions of exclusivity and exhaustivity of the set of models. The values of the

M

  , l=1,...,n be correct. In the DS framework, a lower and an upper bound are introduced for representing the uncertainty associated to   amounts of belief that directly supports l M at least in part, whereas the upper bound, called plausibility, ) ( l M Pl

1 1 H

 11 corresponding to the homogeneous core model of only 1 region and 1 2

  joint basic assignment 12 m (as required by

  of eq.(12) to the cylindrical extensions   the uncertainty of the experts by assigning a mass to the elements   of interest of this work, the i-th uncertain assumptions i H has an UOD i X defined by the set of the alternative hypotheses,

Figure 1 .

 1 Figure 1. Probability density function representing the epistemic uncertainty on 11 p .

Figure 2 .

 2 Figure 2. Probability density functions representing the epistemic uncertainty on 21 p and 22 p .

Figure 3 .

 3 Figure 3. Marginal probability density functions of the parameters 5 4 3 2 1 , , , , p p p p p defining the multinomial distribution of M.

1 H and 2 2 H : 10

 1210 a power set of 8 elements. Since the development of a full-fledged technique for the elicitation of the bpa from the expert knowledge is outside the scope of this work, here we limit ourselves to assigning directly the limiting values to the belief and plausibility of[START_REF] Apostolakis | A Commentary on Model Uncertainty[END_REF]

  the plausibility of model 6 and not of model 3.

Figure 4 .

 4 Figure 4. Discrete cumulative distribution of the hydraulic head distribution.

  framework, the basic probability assignment on the frame of discernment of the model output values  

Figure 5 .

 5 Figure 5. Belief (circles), plausibility (squares) of the sets   i

  methodological set up has considered the case in which multiple assumptions are made for the development of the model, giving rise to corresponding alternative plausible modelling hypotheses on whose correctness the opinions of different experts is sought.Through the illustration of an example concerning the assumptions underpinning a model of groundwater flow, the DST has been shown a proper framework for effectively representing the uncertainty on the correctness of the different hypotheses by means of two limiting values, belief and plausibility, overtaking the difficulties that experts encounter in characterizing their uncertainty using single probability values or probability density functions. The added value of the approach is particularly evident in the case in which some possible models are not developed, for any reason, a situation which may lead to counterintuitive and non conservative results if addressed probabilistically. Furthermore, within the DST, the Dempster rule of combination offers a natural basis for both the aggregation of the uncertainties of different experts on the correctness of the hypotheses and the propagation of the uncertainties regarding the individual assumptions onto the uncertainty characterizing the entire model. This is regardless of the limitations of the rule, which ignores conflicting expert opinions, which can be overcome by more elaborate aggregation rules not introduced here as the issue is out of the scope of the present work.

Figure 1 .Figure 2 .)Figure 3 .Figure4.

 123 Figure 1. Probability density function representing the epistemic uncertainty on 11 p .

Figure 5 .

 5 Figure 5. Belief (circles), plausibility (squares) of the sets   i z z ,...., 1 obtained by the DST approach and cumulative density probability values (dots) obtained by the Bayesian approach.

Figure 6 .

 6 Figure 6. Belief (circles), plausibility (squares) of the sets   i z z ,...., 1 obtained by the DST approach and cumulative density function values (dots) obtained by the Bayesian approach.

  comparison with the DST aggregation; still, future work will be devoted to the analysis of Bayesian approaches for the aggregation of expert opinions in the context of model uncertainty here of interest, which concerns also the propagation of uncertainties on the individual assumptions onto the uncertainty characterizing the entire model structure.

		p			P	H	i	j , 	n ,..., 1	. Assuming complete
		ij					j			i
	independence of the hypotheses, the probability correct is given by:	P	(	M	l	)	of model	l S 	(	H	1 1 l	,	H	2 2 l	,...,	H	k l k	)	to be

Let us consider, first of all, the case in which there is no epistemic uncertainty on the probability distribution of the i n alternative hypotheses i H of the i-th assumption characterized by i n discrete values parameters  

Table I . Alternatives for assumption 1 H "hydraulic and geological properties" Table II. Alternatives for assumption 2 H "groundwater flow mechanism"

 I

  2, of the possible alternative hypotheses are:

	two parameters are independent, i.e.			f	p	21	,	p	22		p	21	,	p	22			f	1 p	    2 1 2 p f p p	. In particular,
	uniform distributions are again assumed for		p	21 , p	22	.
					2 P	  21 2 1 p H 			U	] 80 . 0 , 60 . 0 [
					2 P	  22 2 2 p H 			U	] 10 . 0 , 03 . 0 [	(22)
							2 P	  2 3 H		1		p	21		p	22
	Notice that, since	p	21	 p	22		1	, the two parameters 21 p and 22 p of the multinomial
	distribution are not independent, i.e. a given value of 21 p constrains 22 p to be lower than
	1 p 	21	. However, in the case here considered	21 p 	p	22	is always less than 1 and thus the

Table III . Basic probability assignment, belief and plausibility of the six model hypotheses.

 III

Table IV . Estimation of the hydraulic head at d=90 meters

 IV

	According to the probabilistic framework, the probability   i z p	that the hydraulic head
	takes the value i z is:
	p	(	z	i	)			p		z	i	|	1 p	,	p	2	,	p	3	,	p	4	,	p	5	  f	1 p	,	p	2	,	p	3	,	p	4	,	p	5	 dp 1	dp	2	3 dp	dp	4	5 dp	(33)
	and thus it is equal to the probability  

i M P

  Table VI represents bpa, belief and plausibility of the subsets formed by a single model while Table VII compares the bpas before and after the aggregation of Expert 3 opinion.

	Notice that the DST combination of Expert 3 opinion with Expert 1 and 2 opinions results in a
	bpa with mass 0 for all the subsets that contain	2 M or	4

Table I .

 I Alternatives for assumption[START_REF] Zio | Two methods for the structured assessment of model uncertainty by experts in performance assessments of radioactive waste repositories[END_REF] H "hydraulic and geological properties"

		H	1 1	"15 hydrogeological vertical layers "
		H	1 2	"4 hydrogeological vertical layers"
	Table II. Alternatives for assumption 2 H "groundwater flow mechanism"
	H	2 1		"Groundwater flow in porous matrix only"
	H	2 2		"Groundwater flow in fractures only"
	H	2 3		"Groundwater flow in matrix and fracture simultaneously"

Table III . Basic probability assignment, belief and plausibility of the six model hypotheses.

 III

	1.5
	1
	0.5
	0	0	0.1	0.2	0.3	0.4	0.5	0.6	0.7	0.8	0.9	1
	Model			bpa			belief			plausibility
		M	1			0.36			0.36			0.76
		M	2			0.03			0.03			0.32
		M	3			0.02			0.02			0.10
		M	4			0.00			0.00			0.04
		M	5			0.09			0.09			0.33
		M	6			0.01			0.01			0.14

ACKNOWLEDGMENT

This work has been funded by the Ente per le Nuove Tecnologie, L'Energia e l'Ambiente (ENEA) within the Framework Program Agreement with the Italian Ministry of Economic Development.

Table V. Probabilities of correctness of the six models obtained by combining the three experts opinions (second column), only Expert 1 and 2 opinions (third column) and relative variation obtained by considering Expert 3 opinion (last column).

Note that the effect of the aggregation of Expert 3 opinion with Experts 1 and 2 opinions is to set to zero the probabilities 123 P of models 2 M and 4 M . Furthermore, due to the constraint that the sum of the probabilities of all the models must be one, the probability that is not counterbalanced by the renormalization of the probabilities.

DS framework

According to the DST framework, it is possible to represent Expert 3 judgment on the 6 models by the bpa:

are not able to specify to which of the two models belongs, is assigned by eq.( 8) to 1 M , since

. Analogously, the mass assigned to the other single model subsets increases due to the fact that they collect all the mass assigned by Experts 1 and 2 to subsets containing 2 M and/or 4 M . Finally, Figure 6 reports the belief and plausibility values obtained within the DST approach for the hydraulic head z and the corresponding cumulative density values obtained by the probabilistic approach. Due to the different aggregation mechanism of Expert 3 opinion, the cumulative distribution is far below the DS limiting distribution. For example, the probability that be lower than or equal to 810 is 0.46 according to the probabilistic framework and between 0.59 and 0.79 for the DST framework.

CONCLUSIONS

In this paper, the problem of representing and propagating model uncertainty within the model set expansion approach has been considered by the Dempster-Shafer Theory of Evidence compared to a classical probabilistic framework, with reference to a case study of a model of groundwater flow of interest for the performance assessment of radioactive waste DST which assume that the frame of discernment of an uncertain quantity is formed by all its known possible realizations and by an unknown element which covers lack of completeness.

Another direction of development of the work deemed necessary regards the case in which new information on the model assumption correctness, including experimental data and subjective evidence, becomes available after the initial analysis and, thus, updating of the output distribution is required; in this respect, formal Bayesian approaches to model uncertainty seem very fit to the purpose due to their ability to update the distribution of the model output and should be compared to their counterparts within the DST framework of analysis.

Finally, the characterization of the model output uncertainty here analyzed, is expected to be used within the Probabilistic Risk Assessment (PRA) framework for the performance assessment of radioactive waste repositories: whereas the output probability distribution provided by a probabilistic approach to model uncertainty can be directly used in the PRA framework, the DST produces lower and upper cumulative density functions which can be used to bound the value of the probability that the output is below a given value, an informative metric that can be used in the PRA framework.

, H H M  812m