Jon C Helton
email: jchelto@sandia.gov

Cédric J Sallaberry

Uncertainty and Sensitivity Analysis: From Regulatory Requirements to Conceptual Structure and Computational Implementation

Keywords: Aleatory uncertainty, Epistemic uncertainty, Performance assessment, Regulatory requirements, Sensitivity analysis, Uncertainty analysis 1

An approach to the conversion of regulatory requirements into a conceptual and computational structure that permits meaningful uncertainty and sensitivity analyses is descibed. This approach is predicated on the description of the desired analysis in terms of three basic entities: (i) a probability space characterizing aleatory uncertainty, (ii) a probability space characterizing epistemic uncertainty, and (iii) a model that predicts system behavior. The presented approach is illustrated with results from the 2008 performance assessment for the proposed repository for high-level radioactive waste at Yucca Mountain, Nevada.

Introduction

An approach to the conversion of regulatory requirements into a conceptual and computational structure that permits meaningful uncertainty and sensitivity analyses is descibed. This approach is predicated on the description of the desired analysis in terms of three basic entities: (i) a probability space characterizing aleatory uncertainty, (ii) a probability space characterizing epistemic uncertainty, and (iii) a model that predicts system behavior. The presented approach is illustrated with results from the 2008 performance assessment (PA) for the proposed repository for high-level radioactive waste at Yucca Mountain (YM), Nevada, carried out by the U.S. Department of Energy (DOE) to assess compliance with regulations promulgated by the U.S. Nuclear Regulatory Commission (NRC) [START_REF]Total System Performance Assessment Model/Analysis for the License Application[END_REF][START_REF]Yucca Mountain Repository Safety Analysis Report[END_REF][START_REF]Yucca Mountain Repository License Application[END_REF].

Example: DOE's Licensing Requirements for YM Repository

The NRC's licensing requirements for the YM repository provide a good example of the challenges that are present in the conversion of regulatory requirements into the conceptual structure and associated computational implementation of an analysis that establishes compliance (or noncompliance) with those requirements [4; 5].

The following two radiation protection requirements for a reasonably maximally exposed individual (RMEI) are at the core of the NRC's requirements for the YM repository ( [START_REF] Nrc | 10 CFR Part 63: Implementation of a Dose Standard after 10,000 Years[END_REF], p. 10829): "(a) DOE must demonstrate, using performance assessment, that there is a reasonable expectation that the reasonably maximally exposed individual receives no more than the following annual dose from releases from the undisturbed Yucca Mountain disposal system: (1) 0.15 mSv (15 mrem) for 10,000 years following disposal; and (2) 1.0 mSv (100 mrem) after 10,000 years, but within the period of geologic stability. (b) DOE's performance assessment must include all potential environmental pathways of radionuclide transport and exposure." In addition, the following elaboration on the preceding dose requirements for the RMEI is also given ( [START_REF] Nrc | 10 CFR Part 63: Implementation of a Dose Standard after 10,000 Years[END_REF], p. 10829): "Compliance is based upon the arithmetic mean of the projected doses from DOE's performance assessments for the period within 1 million years after disposal".

The preceding dose requirements indicate (i) that dose results must be determined for long time periods into the future and also for many different potential modes of exposure and (ii) that some type of averaging process is to be used to determine the dose values to which the regulatory requirements apply. The indicated averaging process (i.e., "arithmetic mean of projected doses") is vague and thus particularly challenging to the design of an analysis to assess compliance with the indicated bounds on (mean) dose. However, of necessity, implementation of this averaging process requires some form of a probabilistic representation of uncertainty.

Additional detail on what is desired in assessing compliance with the indicated dose requirements is provided by the NRC in the following definition for PA ( [START_REF] Nrc | Disposal of High-Level Radioactive Wastes in a Proposed Geologic Repository at Yucca Mountain, Nevada; Final Rule[END_REF], p. 55794): "Performance assessment means an analysis that: [START_REF]Total System Performance Assessment Model/Analysis for the License Application[END_REF] Identifies the features, events, processes (except human intrusion), and sequences of events and processes (except human intrusion) that might affect the Yucca Mountain disposal system and their probabilities of occurring during 10,000 years after disposal, (2) Examines the effects of those features, events, processes, and sequences of events and processes upon the performance of the Yucca Mountain disposal system; and (3) Estimates the dose incurred by the reasonably maximally exposed individual, including the associated uncertainties, as a result of releases caused by all significant features, events, processes, and sequences of events and processes, weighted by their probability of occurrence."

The preceding definition makes very clear that a PA used to assess regulatory compliance for the YM repository must (i) consider what could happen in the future, (ii) assign probabilities to what could happen in the future, (iii) model the effects of what could happen in the future, (iv) consider the effects of uncertainties, and (v) weight potential doses by the probability of the occurrence of such doses. Of particular interest and importance to the design of an analysis to assess compliance is the indicated distinction between "uncertainty" and "probability of occurrence". This is a distinction between what is often called epistemic uncertainty and aleatory uncertainty [8; 9]. Specifically, epistemic uncertainty derives from a lack of knowledge about the appropriate value to use for a quantity that is assumed to have a fixed value in the context of a particular analysis, and aleatory uncertainty derives from an inherent randomness in the properties or behavior of the system under study.

The NRC further emphasizes the importance of an appropriate treatment of uncertainty in assessing regulatory compliance for the YM repository in the following definition of reasonable expectation ( [START_REF] Nrc | Disposal of High-Level Radioactive Wastes in a Proposed Geologic Repository at Yucca Mountain, Nevada; Final Rule[END_REF], p. 55813): "Reasonable expectation means that the Commission is satisfied that compliance will be achieved based upon the full record before it. Characteristics of reasonable expectation include that it: (1) Requires less than absolute proof because absolute proof is impossible to attain for disposal due to the uncertainty of projecting long-term performance; (2) Accounts for the inherently greater uncertainties in making long-term projections of the performance of the Yucca Mountain disposal system; (3) Does not exclude important parameters from assessments and analyses simply because they are difficult to precisely quantify to a high degree of confidence; and (4) Focuses performance assessments and analyses on the full range of defensible and reasonable parameter distributions rather than only upon extreme physical situations and parameter values." As the preceding definition makes clear, the NRC intends that a thorough treatment of uncertainty is to be an important part of assessing compliance with licensing requirements for the YM repository.

Similar requirements to the NRC's requirements for the YM repository, either by explicit statement or implication, underlie requirements for analyses of other complex systems, including (i) the NRC's safety goals for nuclear power stations [START_REF] Helton | Calculation of Reactor Accident Safety Goals[END_REF], (ii) the U.S. Environmental Protection Agency's certification requirements for the Waste Isolation Pilot Plant [11; 12], and (iii) the National Nuclear Security Administration's mandate for the quantification of margins and uncertainties in assessments of the nation's nuclear stockpile [START_REF]Evaluation of Quantification of Margins and Uncertainties for Assessing and Certifying the Reliability of the Nuclear Stockpile[END_REF][START_REF] Helton | Guest Editorial: Quantification of Margins and Uncertainties[END_REF][START_REF] Helton | Quantification of Margins and Uncertainties: Conceptual and Computational Basis[END_REF]. Three recurrent ideas run through all of these examples: (i) the occurrence of future events (i.e., aleatory uncertainty), (ii) prediction of the consequences of future events (i.e., the modeling of physical processes), and (iii) lack of knowledge with respect to appropriate models and associated model parameters (i.e., epistemic uncertainty). The challenge in each case is to define a conceptual model and an associated computational implementation that appropriately incorporates these ideas into analyses supporting compliance determinations.

Conceptual Structure and Computational Implementation

The needed conceptual structure and path to computational implementation is provided by viewing the analysis of a complex system as being composed of three basic entities: (i) a probability space (,, p A ) characterizing aleatory uncertainty, (ii) a probability space (, , p E ) characterizing epistemic uncertainty, and (iii) a model that predicts system behavior (i.e., a function f(t|a, e), or more typically a vector function f(t|a, e), that defines system behavior at time t conditional on elements a and e of the sample spaces  and  for aleatory and epistemic uncertainty). In the context of the three recurrent ideas indicated at the end of the preceding section, the probability space (,, p A ) defines future events and their probability of occurrence; the functions f(t|a, e) and f(t|a, e) predict the consequences of future events; and the probabil-ity space (,, p E ) defines "state of knowledge uncertainty" with respect to the appropriate values to use for analysis inputs and characterizes this uncertainty with probability.

In turn, this conceptual structure leads to an analysis in which (i) uncertainty in analysis results is defined by integrals involving the function f(t|a, e) and the two indicated probability spaces and (ii) sensitivity analysis results are defined by the relationships between epistemically uncertain analysis inputs (i.e., elements e j of e) and analysis results defined by the function f(t|a, e) and also by various integrals of this function. Computationally, this leads to an analysis in which (i) high-dimensional integrals must be evaluated to obtain uncertainty analysis results and (ii) mappings between high-dimensional spaces must be generated and explored to obtain sensitivity analysis results. In general, f(t|a, e) is just one component of a high dimensional function f(t|a, e). It is also possible for f(t|a, e) and f(t|a, e) to be functions of spatial coordinates as well as time.

In general, the elements a of  are vectors

1 2 m a ,a ,...,a      a (1)
that define one possible occurrence in the universe under consideration. In practice, the uncertainty structure formally associated with the set  and the probability measure p A is defined by defining probability distributions for the individual elements a i of a. Formally, this corresponds to defining a density function d Ai (a i ) on a set  i characterizing aleatory for each element a i of a (or some other uncertainty structure such as a cumulative distribution function (CDF) or a complementary CDF (CCDF) when convenient). Collectively, the sets  i and density functions d Ai (a i ), or other appropriate uncertainty characterizations, define the set  and a density function d A (a) for a on , and thus, in effect, define the probability space (, , p A ).

Similarly, the elements e of  are vectors

  1 2 A M n , e ,e ,...,e      e e e (2)
that define one possible set of epistemically uncertainty analysis inputs, where the vector e A contains uncertain quantities used in the characterization of aleatory uncertainty and the vector e M contains uncertain quantities used in the modeling of physical processes. As in the characterization of aleatory uncertainty, the uncertainty structure formally associated with the set  and the probability measure p E is defined by defining probability distributions for the individual elements e i of e. Formally, this corresponds to defining a density function d Ei (e i ) (or some other uncertainty structure such as a CDF or CCDF when convenient) on a set  i characterizing epistemic uncer-tainty for each element e i of e. Collectively, the sets  i and density functions d Ei (e i ), or other appropriate uncertainty characterizations, define the set  and a density function d E (e) for e on , and thus, in effect, define the probability space (, , p E ). In practice, the distributions for the individual elements of e are often obtained through an extensive expert review process (e.g., [START_REF] Hora | Expert Opinion in Risk Analysis: The NUREG-1150 Methodology[END_REF]).

The model, or system of models, that predict analysis results can be represented by

    for a single result M M y t | , f t | ,  a e a e (3)
and

        1 2
for multiple results,

M M M M t | , y t | , , y t | , ,... t | ,       y a e a e a e f a e (4)
where t represents time. In practice, f(t|a, e) and f(t|a, e) are very complex computer models and may produce results with a spatial as well as a temporal dependency.

In concept, the probability space (, , p A ) and the function y(t|a, e) = f(t|a, e) are sufficient to determine the expected value E A [y(t|a, e)] of y(t|a, e) over aleatory uncertainty conditional on the values for uncertain analysis inputs defined by an element e = [e A , e M ] of  (i.e., risk in the terminology of many analyses and expected dose in the terminology of the NRC's regulations for the YM repository). Specifically,

            1 | , | | ,
A M A M A A nS j j M j A A j j j M A j A j E y t y t d j n S y t nS d y t p j                 
nS j nS                (5)
with the inclusion of "|e A " in d A (a|e A ) and p A ( j |e A ) indicating that the distribution (i.e., probability space) for a is dependent on epistemically uncertain quantities that are elements of e A . Similarly, the probabilities that define CDFs and CCDFs that show the effects of aleatory uncertainty conditional on a specific element e = [e A , e M ] of  are defined by

      | , | | , | d A M A y M A A p y t y y t d             a e e a e a e   (6)
and

      | , | | , | d , A M A y M A A p y y t y t d             a e e a e a e   (7)
respectively, where

      1 for | , | , 0 for | , M y M M y t y y t y y t             a e a e a e (8)
and

        0 for | , | , 1 | , 1 for | , . M y M y M M y t y y t y t y y t                     a e a e a e a e (9)
The integrals in Eqs. ( 6) and ( 7) can be approximated with procedures analogous to the sampling-based procedures indicated in Eq. ( 5).

The integrals in Eqs. ( 5)-( 7) must be evaluated with multiple values of e = [e A , e M ] in order to determine the effects of epistemic uncertainty. As illustrated in Sect. 4, the indicated multiple values for e = [e A , e M ] are often obtained with a Latin hypercube sample (LHS)

., ,

k A k M k k k n k e e e k nLHS            e e e (10)
of size nLHS from the sample space  for epistemic uncertainty due to the efficient stratification properties of Latin hypercube sampling [17; 18]. This sample provides the basis for both (i) the numerical estimation of the effects of epistemic uncertainty and (ii) the implementation of a variety sensitivity analysis procedures [START_REF] Helton | Survey of Sampling-Based Methods for Uncertainty and Sensitivity Analysis[END_REF][START_REF] Storlie | Multiple Predictor Smoothing Methods for Sensitivity Analysis: Description of Techniques[END_REF][START_REF] Storlie | Implementation and Evaluation of Nonparametric Regression Procedures for Sensitivity Analysis of Computationally Demanding Models[END_REF].

Just as expected values, CDFs and CCDFs related to aleatory can be defined as indicated in Eqs. ( 5)- [START_REF] Nrc | Disposal of High-Level Radioactive Wastes in a Proposed Geologic Repository at Yucca Mountain, Nevada; Final Rule[END_REF], similar quantities can be defined that summarize the effects of epistemic uncertainty. Several possibilities exist: (i) epistemic uncertainty in a result y(t|a, e M ) conditional on a specific realization a of aleatory uncertainty, (ii) epistemic uncertainty in an expected value over aleatory uncertainy, and (iii) epistemic uncertainty in the cumulative probability p A [y(t|a, e M )≤y|e A ] or exceedance (i.e., complementary cumulative) probability p A [y<y(t|a, e M ) |e A ] for a specific value y of an analysis result.

For a result y(t|a, e M ) conditional on a specific realization a of aleatory uncertainty, the expected value, cumulative probability and exceedance probability over epistemic uncertainty are given by

        1 | , | , d | , / , nLHS E M M E M M k k E y t y t d y t nLHS         a e a e e a e   (11)
        1 | , | , d | , / , E M y M E M nLHS y M k k p y t y y t d y t nLHS                     a e a e e a e   (12)
and

        1 | , | , d | , / , E M y M E M nLHS y M k k p y y t y t d y t nLHS                     a e a e e a e   (13)
respectively, where (i)  corresponds to the subspace of  that contains only the vectors e M and (ii) the vectors e Mk are part of the LHS in Eq. [START_REF] Helton | Calculation of Reactor Accident Safety Goals[END_REF].

For an expected result E A [y(t|a, e M )|e A ] over aleatory uncertainty, the expected value, cumulative probability and exceedance probability over epistemic uncertainty are defined analogously to the corresponding results in Eqs. ( 11)- [START_REF]Evaluation of Quantification of Margins and Uncertainties for Assessing and Certifying the Reliability of the Nuclear Stockpile[END_REF]. Specifically,

              1 | , | | , | d d | , | d / , E A M A M A A E nLHS Mk A Ak k E E y t y t d d y t d nLHS              
              1 | , | | , | d d | , | d / , E A M A y M A A E nLHS y M k A A k k p E y t y y t d d y t d nLHS                       (14)
and

              1 | , | | , | d d | , | d / , E A M A y M A A E nLHS y M k A A k k p y E y t y t d d y t d nLHS                      
where, in general, the inner integrals over  will have to be evaluated with some appropriate integration procedure as indicated in Eq. ( 5). For a cumulative probability p A [y(t|a, e M )≤y|e A ] or an exceedance (i.e., complementary cumulative) probability p A [y<y(t|a, e M ) |e A ] over aleatory uncertainty for a specific value y of an analysis result, the expected value, cumulative probability and exceedance probability over epistemic uncertainty are defined analogously to the corresponding results in Eqs.( 14)- [START_REF] Hora | Expert Opinion in Risk Analysis: The NUREG-1150 Methodology[END_REF]. For example, the expected value and cumula-tive probability for p A [y<y(t|a, e M )|e A ] that derive from epistemic uncertainty are defined by

                1 | , | | , | d d | , | d / E A M A y M A A E nLHS y M k A A k k E p y y t y t d d y t d nLHS                      
and

                  1 | , | | , | d d | , | d / , E A M A p y M A A E nLHS p y M k A A k k p p y y t p y t d d y t d nLHS                         
respectively.

The conceptual structure and computational procedures described in this section are illustrated in the next section with results from the 2008 YM PA [START_REF]Total System Performance Assessment Model/Analysis for the License Application[END_REF].

where, for a time interval [a, b] (e.g., [0, 10 4 yr] or [0, 10 6 yr]), nEW = number of early waste package (WP) failures, nED = number of early drip shield (DS) failures, nII = number of igneous intrusive (II) events, nIE = number of igneous eruptive (IE) events, nSG = number of seismic ground (SG) motion events, nSF = number of seismic fault (SF) displacement events, a EW = vector defining the nEW early WP failures, a ED = vector defining the nED early DS failures, a II = vector defining the nII igneous intrusive events, a IE = vector defining the nIE igneous eruptive events, a SG = vector defining the nSG seismic ground motion events, and a SF = vector defining the nSF seismic fault displacement events. The definition of the probability space (, , p A )

for aleatory uncertainty was completed by defining probability distributions for the individual elements of a (see [START_REF]Total System Performance Assessment Model/Analysis for the License Application[END_REF], App. J). In the 2008 YM PA, elements of the sample space  are referred to as scenarios, and elements of the set  are referred to as

where, as examples, the following quantities are elements of e: DSNFMASS = scale factor used to characterize uncertainty in radionuclide content of defense spent nuclear fuel; IGRATE = frequency of intersection of the repository footprint by a volcanic event (yr 1 ); MICTC99 = groundwater biosphere dose conversion factor (BDCF) for 99 Tc in modern interglacial climate; SCCTHRP = residual stress threshold for stress corrosion cracking nucleation of Alloy 22 (as a percentage of yield strength in MPa); SZGWSPDM = logarithm of scale factor used to characterize uncertainty in groundwater specific discharge (dimensionless); and WDGCA22 = temperature dependent slope term of Alloy 22 general corrosion rate (K). Distributions characterizing epistemic uncertainty were assigned to the individual elements of e and, in effect, defined the probability space (, , p E ) for epistemic uncertainty. A complete listing of the 392 elements of e and sources of additional information on these variables and the development of their distributions are given in Table K3-3 of Ref. [START_REF]Total System Performance Assessment Model/Analysis for the License Application[END_REF]. A very complex system of models was used to predict a large number of timedependent results related to evolution of the repository, including (i) the release of radionuclides from WPs, (ii) the transport of radionuclides away from the engineered component of the repository, and (iii) human exposure to released radionuclides (see [START_REF]Total System Performance Assessment Model/Analysis for the License Application[END_REF], Table K3-4, for a listing of the analysis results selected for study). An overview description of these models and extensive sources of additional information are available in Ref. [START_REF]Total System Performance Assessment Model/Analysis for the License Application[END_REF]. As an example, a high-level overview of the models used in the analysis of seismic ground motion events is given in Fig. 1. The models indicated in

will be used as an example to illustrate results of the form indicated in Sect. 3. It is in the generation of such results where the challenge of bringing conceptual structure and computational implementation together arises. Bluntly put, it is not possible to evaluate integrals of the form indicated in Sect. 3 for D(t|a, e M ) defined by modeling systems of the complexity shown in Fig. 1 without a carefully designed computational strategy that makes efficient use of what will almost always be a limited number of detailed, mechanistic calculations. Such a strategy will be analysis specific and designed to take advantage of particular properties of the models in use.

A core quantity in the NRC's regulatory requirements for the YM repository is the expected value E A [D(t|a, e M )|e A ] of D(t|a, e M ) over aleatory uncertainty, with E A [D(t|a, e M )|e A ] being an example of the expected value formally defined in Eq. ( 5). Specifically, where the subscripts contained in the set  are used to denote doses to the RMEI from individual scenario classes with N designating the scenario class in which no disruptions occur and the remaining subscripts designating scenario classes associated with the correspondingly subscripted aleatory occurrences in Eq. ( 19). Results associated with dose D C (t|a, e M ) for scenario class C in Eq. ( 22) are assumed to be calculated with only those elements of a and e = [e A , e M ] that are related to scenario class C. The approximation to E A [D(t|a, e M )|e A ] in Eq. ( 22) can be justified on the basis of tradeoffs between the effects of high probability-low consequence scenario classes and low probability-high consequence scenario classes.

                  | , | | , | d | , | d , , , , , , , | , | d

| , | ,

A M A M A A C M A A C C M A A C A C M A C E D t D t d D t d N EW ED II IE SG SF D t d E D t                          
Epistemic uncertainty is propagated in the 2008 YM PA with an LHS

1 2 3 9 2 , , ,..., , 1, 2,..., 300, k
A k M k k k k e e e k nLHS             e e e (23)
from the sample space  for epistemic uncertainty. In turn, the approximation

    ˆ| , | | , | A M k A k A C M k A k C E D t E D t          
A M k A k
E D t a e e to be valid. As an example, analysis results are presented for seismic ground motion events occurring in the time interval [0, 210 4 yr] (i.e., for what is called the seismic ground motion scenario class in the 2008 YM PA). This restriction reduces the elements of the sample space  for aleatory uncertainty to

1 1 1 2 2 2
, , , , , , ,..., , , ,

nSG nSG nSG nSG t v A t v A t v A      a (25)
where (i) nSG = number of seismic ground motion events in 20,000 yr, (ii) t i = time (yr) of event i, (iii) v i = peak ground velocity (m/s) for event i, (iv) A i = damaged area (m 2 ) on individual WPs for peak ground velocity v i , (v) the occurrence of seismic ground motion events is characterized by a hazard curve for peak ground velocity, and (vi) damaged area is characterized by distributions conditional on peak ground velocity.

To evaluate results of the form defined in Sect. 3 for the seismic ground motion scenario class, it is necessary to integrate the function D SG (t|a, e M ) over the sample space  for aleatory uncertainty with a defined as indicated in Eq. ( 25). In full detail, D SG (t|a, e M ) is defined by the model system shown in Fig. 1. Evaluation of this system is too computationally demanding to permit its evaluation 1000's of times for each element e k = [e Ak , e Mk ] of the LHS in Eq. ( 23). This is a common situation in analyses of complex systems, where very detailed physical models are developed which then turn out to be too computationally demanding to be naively used in the propagation of aleatory uncertainty. In such situations, it is necessary to find ways to efficiently use the results of a limited number of model evaluations to predict outcomes for a large number of different possible realizations of aleatory uncertainty.

For the seismic ground motion scenario class and the time interval [0, 210 4 yr], the needed computational efficiency was achieved by evaluating D SG (t|a, e Mk ) at a sequence of times (i.e., 100, 1000, 3000, 6000, 12000, 18000 yrs) and for a sequence of damaged areas areas (i.e., 10 8+s (32.6 m 2 ) for s = 1, 2, …, 5 with 32.6 m 2 corresponding to the surface area of a WP) at each time (Fig. 2a). This required 65 = 30 evaluations of the system indicated in Fig. 1 for each LHS element in Eq.( 23). Once obtained, these evaluations can be used with appropriate interpolation and additive procedures to evaluate D SG (t|a, e Mk ) for different values of a for each LHS element e k = [e Ak , e Mk ].

The individual CCDFs in Fig. 2b are defined by probabilities of the form shown in Eq. ( 7) with (i) D SG (t|a, e Mk ) and e Ak replacing y(t|a, e M ) and e A and (ii) t = 10 4 yr. Numerically, the integrals that define exceedance probabilities for the individual CCDFs are approximated with (i) random sampling from the possible values for a as indicated in Eq. ( 5) and (ii) estimated values ˆ( | , ) SG j Mk D t a e for D SG (t|a, e Mk ) constructed from results of the form shown in Fig. 2a. Specifically,

    1 | , | | , / , nS A S G M k A k y S G j M k j p y D t D t nS              a e e a e (26)
with the a j , j = 1, 2, …, nS, sampled in consistency with the density function d A (a| e Ak ) for vectors of the form shown in Eq. ( 25). The mean and quantile curves in Fig. 2b are (i) defined and approximated as indicated in Eqs. ( 17) and ( 18) and (ii) provide a summary of the epistemic uncertainty present in the estimation of exceedance probabilities (i.e., p A [y < D SG (10 4 |a, e M )|e A ]) for D SG (10 4 |a, e M ).

As indicated in Eq. ( 5), the expected value 14) , (ii) a CDF as defined by the cumulative probabilities in Eq. ( 15), or (iii) a CCDF as defined by the complementary cumulative probabilities in Eq. ( 16). The indicated mean defined in Eq. ( 14) and illustrated in Fig. 2c is the outcome of reducing all the information in Fig. 2b to a single number. The approximation process for a CDF also provides the basis for obtaining specific quantile values (e.g., q = 0.05, 0.5 ~ median, 0.95) as indicated in Fig. 2c.

In the NRC's regulatory requirements for the YM repository, bounds apply over time to expected dose to the RMEI. Thus, the analysis results of greatest interest are expected dose and the uncertainty in expected dose as a function of time (Fig. 2d). Specifically, expected doses for individual LHS elements correspond to the lighter lines in Fig. 2d, and quantile and mean values for expected dose that summarize the effects of epistemic uncertainty correspond to the darker dashed and solid lines. The results on Fig. 2d at 10,000 years correspond to the results shown in more detail in Fig. 2c. For reasons of computational efficiency, the individual expected dose curves in Fig. 2d were estimated with a quadrature procedure as described in Sect. J8.3 of Ref. [START_REF]Total System Performance Assessment Model/Analysis for the License Application[END_REF] rather than with a sampling-based procedure as illustrated in Fig. 2b.

Sensitivity analysis is an important component of the 2008 YM PA and contributes to an establishment of "reasonable expectation" by supporting a detailed examination of the operation of the models that predict dose and expected dose to the RMEI and many other analysis outcomes of interest. Specifically, sensitivity analysis in the 2008 YM PA was based on an exploration of the mapping between elements of the LHS indicated in Eq. ( 23) and analysis results of interest (e.g., dose and expected dose to the RMEI) with a variety techniques including stepwise rank regression (Fig. 2e) and time-dependent partial rank correlation coefficients (PRCCs) (Fig. 2f) [START_REF] Helton | Survey of Sampling-Based Methods for Uncertainty and Sensitivity Analysis[END_REF]. In stepwise rank regression, variable importance is indicated by the order in which variables are selected in the stepwise process, the incremental changes in R 2 values as variables are added to the regression model, and the values of the standardized rank regression coefficients (SRRCs) in the regression model. The indicated results with R 2 values and SRRCs provide a measure of the amount of epistemic uncertainty in the dependent variable under consideration that derives from the epistemic uncertainty in individual analysis inputs (i.e., elements of e); in contrast, PRCCs provide a measure of the strength of the monotonic relationship between individual epistemically uncertain analysis inputs and the dependent variable under consideration after removal of the monotonic effects of all other epistemically uncertain analysis inputs. Definitions for selected variables appearing Figs. 2e and 2f are given after Eq. [START_REF] Storlie | Multiple Predictor Smoothing Methods for Sensitivity Analysis: Description of Techniques[END_REF].

Additional detailed information on uncertainty and sensitivity analysis for the seismic ground motion scenario class in the 2008 YM PA is available in Sects. J8 and K7 of Ref. [START_REF]Total System Performance Assessment Model/Analysis for the License Application[END_REF]. Included in this material are extensive examples of results conditional on individual realizations of aleatory uncertainty as defined in Eqs. ( 11)- [START_REF]Evaluation of Quantification of Margins and Uncertainties for Assessing and Certifying the Reliability of the Nuclear Stockpile[END_REF].

As indicated in Eq. ( 22), expected dose results are obtained for individual scenario classes and then summed to determine expected dose over all scenario classes (i.e., over all futures as described by the vectors defined in Eq. ( 19)). The results of this summation are shown in Figs. 3a and3c for the time intervals [0, 210 4 yr] and [0, 10 6 yr], respectively. The individual expected dose curves in Fig. 3c are not as smooth as the individual expected dose curves in Fig. 3a. This difference results because the complexity of the calculations for the [0, 10 6 yr] time interval required the use of a sampling-based procedure to approximate expected dose from seismic ground motion events (see [START_REF]Total System Performance Assessment Model/Analysis for the License Application[END_REF], Sect. J8.4); in contrast, the quadrature procedures used to determine expected dose for the individual scenarios for the [0, 210 4 yr] time interval resulted in the smoother expected dose curves in Fig. 3a.

Ultimately, the NRC decided that their expected (i.e., "arithmetic mean") dose requirements of 15 mrem/yr and 100 mrem/yr for the time intervals [0, 10 4 yr] and [10 4 , 10 6 yr], respectively, applied to the mean (i.e., solid line) doses in Figs. 3a and3c (i.e., to results of the form defined in Eq. ( 14)). However, it is the spread of the individual dose curves in Figs. 3a and3c that provide the NRC requested uncertainty information and thus a basis for a "reasonable expectation" that the requirements are being met despite the presence of substantial epistemic uncertainty. Further, sensitivity analyses of the form illustrated in Figs. 3b and 3d also contribute to "reasonable expectation" by enhancing understanding of the overall analysis and thus confidence in the numerical implementation of the analysis. Definitions for selected variables appearing Figs. 3b and 3d are given after Eq. [START_REF] Storlie | Multiple Predictor Smoothing Methods for Sensitivity Analysis: Description of Techniques[END_REF].

Additional detailed information on uncertainty and sensitivity analysis in the 2008 YM PA is available in Apps. J and K of Ref. [START_REF]Total System Performance Assessment Model/Analysis for the License Application[END_REF].

Concluding Message

Everyone cannot be expected to agree on the details of an analysis of a complex system, but everyone should be able to know what those details are. Without such knowledge, it is not possible to have informed and meaningful discussions involving the views of all parties interested in the analysis.

A necessary starting point for the design, computational implementation and, ultimately, communication of an analysis for a complex system is a clear conceptual structure. As described in this presentation, this structure for many, if not most, analyses, can be based on three basic entities: a probability space (,, p A ) characterizing aleatory uncertainty, a probability space (, , p E ) characterizing epistemic uncertainty, and a model that predicts system behavior (i.e., a function f(t|a, e), or more typically a vector function f(t|a, e), that defines system behavior at time t conditional on elements a and e of the sample spaces  and  for aleatory and epistemic uncertainty). As illustrated with results from the 2008 YM PA, this conceptual view of the analysis of a complex system provides (i) a way to formally describe that analysis and then (ii) a clear path from formal description to computational implementation to written documentation.

  scenario classes. With this usage, scenarios and scenario classes correspond to what are called elementary events and events, respectively, in the usual terminology of probability theory. The individual elements of the sample space  for epistemic uncertainty in the 2008 YM PA are vectors of the form

  Fig. 1 correspond to part of what is very simplistically represented by f(t|a, e) in Eq. (4).Owing to its central role in the NRC's regulatory requirements for the YM repository, /yr) to the RMEI at time (yr) conditional on and ,

Fig. 1 .

 1 Fig. 1. Information transfer between the model components and submodels for the seismic scenario class in the 2008 YM PA ([1], Fig. 6.1.4-6).

  element e k of the LHS in Eq. (23), where ˆ[ a e e , respectively. Because of the occurrence of the same elements of e M in the evaluation of D C (t|a, e M ) for different values of C (i.e., for different scenario classes), it is essential that the doses D C (t|a, e M ) in Eq. (24) be evaluated for the same elements of the LHS in Eq. (23) for the indicated approximation to [ ( | , )| ]

Fig. 2 .

 2 Fig. 2. Example results for dose (mrem/yr) to RMEI for seismic ground motion scenario class: (a) dose for seismic events occurring at different times and causing different damaged areas on WPs ([1], Fig. J8.3-3a), (b) CCDFs for dose at 10,000 yr ([1], Fig. J8.3-10a), (c) CCDF for expected dose at 10,000 yr ([1], Fig. J8.3-5c), (d) time-dependent expected dose ([1], Fig. J8.3-6), (e) stepwise rank regression for expected dose at 10,000 yr ([1], Fig. K7.7.1-2a), and (f) time-dependent PRCCs for expected dose ([1], Fig. K7.7.1-1c).mates are the result of reducing each CCDF in Fig.2bto a single number. As indicated in Eqs. (14)-(16) and illustrated in Fig.2c, epistemic uncertainty associated E A [D SG (t|a, e M )|e A ] can be summarized by (i) an expected (mean) value E E {E A [D SG (t |a, e M )|e A ]} over epistemic uncertainty as defined in Eq. (14) , (ii) a CDF as defined by the cumulative probabilities in Eq. (15), or (iii) a CCDF as defined by the complementary cumulative probabilities in Eq. (16). The indicated mean defined in Eq.

Fig. 3 .

 3 Fig. 3. Expected dose to RMEI: (a, b) Expected dose and associated PRCCs for [0, 210 4 yr] ([1], Fig. K8.1-1[a]) and (c, d) Expected dose and associated PRCCs for [0, 10 6 yr] ([1], Fig. K8.2-1[a]).

4 Example: 2008 PA for YM Repository

	The individual elements of the sample space  for aleatory uncertainty in the 2008
	YM PA are vectors of the form
	a	, nEW nED nII nIE nSG nSF , , , ,   	,	, aaa a aa , , , , EW ED II IE SG	SF	 	,

Acknowledgements. Work performed at Sandia National Laboratories (SNL), which is a multiprogram laboratory operated by Sandia Corporation, a Lockheed Martin Company, for the U.S. Department of Energy's (DOE's) National Nuclear Security Administration under Contract No. DE-AC04-94AL85000. The United States Government retains and the publisher, by accepting this article for publication, acknowledges that the United States Government retains a non-exclusive, paid-up, irrevocable, world-wide license to publish or reproduce the published form of this article, or allow others to do so, for United States Government purposes. The views expressed in this article are those of the authors and do not necessarily reflect the views or policies of the DOE or SNL.