Piero Baraldi

Francesca Mangili

Enrico Zio

Investigation of uncertainty treatment capability of model-based and datadriven prognostic methods using simulated data

Keywords: Prognostics, uncertainty, particle filtering, bootstrap ensemble, turbine blade, creep

We look at different prognostic approaches and the way of quantifying confidence in equipment Remaining Useful Life (RUL) prediction. More specifically, we consider: 1) a particle filtering scheme, based on a physics-based model of the degradation process; 2) a bootstrapped ensemble of empirical models trained on a set of degradation observations measured on equipments similar to the one of interest; 3) a bootstrapped ensemble of empirical models trained on a sequence of past degradation observations from the equipment of interest only.

The ability of these three approaches in providing measures of confidence for the RUL predictions is evaluated in the context of a simulated case study of interest in the nuclear power generation industry and concerning turbine blades affected by developing creeps.

The main contribution of the work is the critical investigation of the capabilities of different prognostic approaches to deal with various sources of uncertainty in the RUL prediction.

Introduction

In prognostics the current system condition is projected in time by a predictive model [START_REF] Hess | Challenges, issues, and lessons learned chasing the 'Big P': real predictive prognostics Part 1[END_REF][START_REF] Tang | Methodologies for uncertainty management in prognostics[END_REF]. Since the prediction of the Remaining Useful Life (RUL) of degrading equipment is performed in the absence of future measurements concerning equipment degradation and operational conditions, the prognostic task is necessarily affected by large uncertainty. In this work, the sources of uncertainty affecting the RUL prediction are classified in three categories:

A. Randomness in the future degradation of the equipment. This intrinsic uncertainty in the degradation process has several causes such as the unknown future load profile, and operation and environmental conditions. B. Modeling error, i.e., inaccuracy of the prognostic model used to perform the prediction. In model-based prognostic approaches, this source of uncertainty takes into account the assumptions and simplifications made on the form and structure of the model, and the uncertainty on the model parameters. In datadriven approaches, it relates to the incomplete coverage of the data set used to train the empirical model. C. Uncertainty in current and past equipment degradation data, which are used by the prognostic model to elaborate the RUL prediction. These data are usually acquired by sensors with some measurement noise or derived from diagnostic systems assessing the equipment health state with some degree of uncertainty.

Other possible sources of uncertainty, which are not considered in this work, are the imperfect knowledge of the value of degradation beyond which the equipment can no longer perform its functions (failure threshold), or the time at which the degradation process starts (degradation initiation). Furthermore, in this work we assume that the equipment degradation is caused by a single degradation mechanism, not considering the uncertainty on the degradation caused by the onset of other, possibly competing, mechanisms [START_REF] Li | An inspection-maintenance model for systems with multiple competing processes[END_REF].

The challenge of managing uncertainties associated with prognostics has been recently addressed in [START_REF] Hess | Challenges, issues, and lessons learned chasing the 'Big P': real predictive prognostics Part 1[END_REF][START_REF] Tang | Methodologies for uncertainty management in prognostics[END_REF][START_REF] Liu | The modeling method on failure prognostics uncertainties in maintenance policy decision process[END_REF][START_REF] Saxena | Evaluating prognostics performance for algorithms incorporating uncertainty estimates[END_REF]. Uncertainty management in prognostics entails to identify, classify and analyze uncertainty sources with the aim of associating to the RUL predictions provided by a prognostic model an estimate of its uncertainty [START_REF] Liu | The modeling method on failure prognostics uncertainties in maintenance policy decision process[END_REF][START_REF] Saxena | Evaluating prognostics performance for algorithms incorporating uncertainty estimates[END_REF][START_REF] Urbina | Quantification of margins and uncertainties of complex systems in the presence of aleatoric and epistemic uncertainty[END_REF][START_REF] Zio | Prognostics and health management of industrial equipment[END_REF], i.e., a measure of the expected degree of mismatch between the real and predicted equipment failure time. This information, provided in the form of a probability distribution of the equipment RUL, can be used by the decision maker to confidently plan maintenance actions, according to the desired risk tolerance [START_REF] Tang | Methodologies for uncertainty management in prognostics[END_REF].

In this context, the objective of the present paper is to contribute to the way of investigating the capabilities of different prognostic approaches to deal with the uncertainty in the RUL prediction. To this aim, the analysis is performed with respect to three previously developed approaches [START_REF] Baraldi | Model-based and data-driven prognostics under different available information[END_REF].

A first approach, hereafter named "approach 1", is based on a mathematical model of the degradation process for the RUL prediction [START_REF] Luo | Model-based prognostic techniques applied to a suspension system[END_REF], embedded in a filtering method capable of accounting for the stochasticity of the process (source of uncertainty A) and the noise affecting the measurements (source of uncertainty C). Most filtering approaches rely on Bayesian methods and provide the probability distribution of the RUL [START_REF] Doucet | Sequential Monte Carlo methods in practice[END_REF][START_REF] Myötyri | Application of stochastic filtering for lifetime Prediction[END_REF]. The exact Kalman filter has been largely used in case of linear state space models and independent, additive Gaussian measurements and modeling noises, whereas analytical or numerical approximations of the exact solution (such as the Extended Kalman filter, the Gaussian-sum filters or the approximate grid-based filters [START_REF] Arulampalam | A tutorial on particle filters for online nonlinear/non-Gaussian Bayesian tracking[END_REF]) have been applied in cases where the dynamics of degradation is nonlinear and/or the associated noises are non-Gaussian [START_REF] Anderson | Optimal filtering[END_REF]. Numerical approximations based on the Monte Carlo sampling technique have gained popularity for their flexibility and ease of design [START_REF] Kitagawa | Non-Gaussian state-space modeling of nonstationary time series[END_REF][START_REF] Crisan | A survey of convergence results on particle filtering methods for practitioners[END_REF][START_REF] Djuric | Particle filtering[END_REF][START_REF] Pitt | Filtering via simulation: Auxiliary particle filters[END_REF]. Among them, Particle Filtering (PF) is often considered a state-of-the-art technology in the prognostic field and used as a term of comparison for newer approaches. The model-based particle filter approach here considered was firstly applied to state estimation for diagnostics [START_REF] Marseguerra | Monte Carlo simulation for model-based fault diagnosis in dynamic systems[END_REF][START_REF] Cadini | Model-based Monte Carlo state estimation for condition-based component replacement[END_REF] and then applied to prognostics [START_REF] Cadini | Monte Carlo-based filtering for fatigue crack growth estimation[END_REF][START_REF] Zio | Particle filtering prognostic estimation of the remaining useful life of nonlinear components[END_REF]. According to the particle filtering scheme proposed in [START_REF] Cadini | Monte Carlo-based filtering for fatigue crack growth estimation[END_REF], the RUL distribution prediction is performed by considering the stochastic model of the degradation process and the on-line observations of the equipment degradation.

In [START_REF] Zio | Evaluating maintenance policies by quantitative modeling and analysis[END_REF], this particle filtering-based prognostic approach is discussed with respect to the design of a predictive maintenance strategy, whose advantages are then compared with those of other maintenance strategies.

We consider also two data-driven approaches [START_REF] Schwabacher | A survey of data-driven prognostic[END_REF], based on statistical models that 'learn' trends from historical data. In particular, we consider bootstrap ensemble approaches [START_REF] Heskes | Practical confidence and prediction intervals in Advances[END_REF][START_REF] Raviv | Bootstrapping with noise: An effective regularization technique[END_REF], which are based on the aggregation of multiple model outcomes and have gained interest due to their ability of estimating the uncertainty in the predictions. These approaches allow estimating the model uncertainty (source of uncertainty B) by considering the variability in the predictions of the diverse models of the ensemble [START_REF] Heskes | Practical confidence and prediction intervals in Advances[END_REF].

On the other hand, the estimate of the uncertainty due to the stochasticity of the degradation process (source A) and the input noise (source C), requires to investigate the relation between the input and the error of the prognostic model based on its performance on a validation dataset. In what we will refer to as "approach 2", a bootstrap ensemble model is built to estimate the equipment RUL based on sequences of observations of evolution to failure of a set of similar equipments operating under similar conditions; in what we will refer to as "approach 3", a bootstrap ensemble model is built based on a sequence of degradation observations only of the equipment whose RUL we want to predict. Although approaches 2 and 3 are both based on the development of an ensemble of bootstrap models, they differ for the type of model used. Whereas in approach 2 we can directly model the RUL as a function of the observed parameters, in approach 3 we have to model the degradation evolution as a function of time, since direct RUL observations are not available.

The three approaches are investigated with reference to the creep growth process in the turbine blades of a Gas Turbine Modular Helium nuclear Reactor (GT-MHR) [START_REF] Couturier | High temperature alloys for the HTGR gas turbine: Required properties and development needs[END_REF][START_REF] Baraldi | A Kalman filter -based ensemble approach with application to turbine creep prognostics[END_REF]. The data used in this case study have been numerically simulated using a traditional model of the creep growth. Artificial data have been used in order to allow testing the three approaches on a large number of different blade degradation trajectories and thus evaluate their capability of correctly estimating the uncertainty on the provided RUL prediction.

The remainder of the paper is organized as follows: Section 2 presents the terminology used in the paper and the problem setting; in Section 3, the decomposition of the prediction error variance into three terms corresponding to the randomness in the future degradation of the equipment, the modeling error, and the uncertainty in current and past equipment degradation measures is reported; in Section 4, the three considered prognostic approaches are described; in Section 5, the problem of blade creeping in high temperature turbines is illustrated and the capability of uncertainty management of the three prognostic approaches are discussed; finally, in Section 6 some conclusions are drawn and potential future work suggested.

Terminology and problem setting

In this work, we assume that the equipment is subject to a single degradation mechanism described as a random process; we do not consider the effects that other competing degradation mechanisms can have on the equipment degradation. Also, we assume that degradation cannot exceed a maximum acceptable level, hereafter referred to as "failure threshold", d th , which is fixed and identical among similar equipment. Notice that in prognostics the failure threshold does not necessarily indicate complete failure of the system, but, for safety margins, it is often set at a conservative value of the degradation limit beyond which the risk of complete failure exceeds tolerance limits or the performance of the system does not fulfill the requirements [START_REF] Saxena | Metrics for offline evaluation of prognostic performance[END_REF]. Since the failure threshold is usually derived from expert knowledge or from experimental measurements of the equipment degradation at failure, its estimate is typically affected by uncertainty which contributes to increase the RUL prediction error variance. Methods to deal with the uncertainty on the failure threshold have been proposed in [START_REF] Usynin | Uncertain failure thresholds in cumulative damage models[END_REF][START_REF] Wang | Reliability and degradation modeling with random or uncertain failure threshold[END_REF], and, for this reason, this aspect is not considered in the present work.

We indicate by d i the equipment degradation level at time t i and we assume that its direct measure is not available, but some physical observations z i related to it are obtained. We indicate by z 1:i =(z Furthermore, in practice, the 'exact' model, g, of the equipment degradation process is not available (source of uncertainty B) and the degradation d i at time t i is not exactly known (source of uncertainty C). In this setting, the complete distribution of i RUL cannot be derived and prognostics is limited to estimating:

• the expected value of RUL i

• the variance of the prediction error as a measure of the accuracy with which the estimated expected value predicts the actual RUL value.

With respect to the estimate of the expected value of RUL i , it will be indicated by rûl i and considered as our RUL prediction. The prognostic model which generates at time t i the estimate rûl i of RUL i on the basis of the observations z i will be referred to as f, i.e. rûl i =f(z i ).

Prediction error variance

According to [START_REF] Heskes | Practical confidence and prediction intervals in Advances[END_REF], the prediction error variance 2 ˆi l u r σ can be decomposed into two terms: 2 A σ , the variance related to the uncertainty on the future degradation of the equipment whose degradation at time t i is d i (source of uncertainty A), and

( ) [ ] ( ) [ ] 2 | 2 | 2 2 2 2 ˆ) ( ] ) [( i d RUL d RUL i C B A i i l u r f E RUL E l u r RUL E i i i i i z - + - = = + = - = + µ µ σ σ σ (2)
In some applications, it can be useful to distinguish the uncertainty due to the modeling error (source B) from that due to the noise on the input data (source C). To this aim, we introduce the quantity

( ) [ ] ( ) [ ] ( ) [ ] ( ) [ ] 2 2 2 | | 2 | 2 | | | 2 | 2 ) ( ) ) ( ) ( C B d RUL RUL RUL i d RUL RUL RUL i d RUL i C B i i i i i i i i i i i i i i E f E f E f E σ σ µ µ µ µ µ µ µ σ + = - + - = - + - = - = + z z z z z z z (3)
Combining eqs. ( 2) and (3), one obtains:

( ) [ ] ( ) [ ] ( ) [ ] 2 2 2 2 | | 2 | 2 | 2 2 ˆ) ( ] ) [( C B A d RUL RUL RUL i i d RUL i i l u r i i i i i i i i i E f E RUL E RUL l u r E σ σ σ µ µ µ µ σ + + = - + - + - = = - = z z z (4)
Notice that these results have been obtained by assuming that the different components of the prediction error are independent and thus the expected values

))] ( )( [( | | i d RUL d RUL i f RUL E i i i i z - - µ µ and )] )( ) ( [( | | | i i i i i i d RUL RUL RUL i f E µ µ µ - - z z z
in eqs. ( 2) and (3), respectively, are zero.

Modeling approaches for RUL prediction

This Section illustrates briefly the three modeling approaches considered for RUL prediction.

Approach 1: Particle Filtering

In approach 1, a Monte Carlo-based filtering technique, called particle filtering [START_REF] Doucet | Sequential Monte Carlo methods in practice[END_REF][START_REF] Arulampalam | A tutorial on particle filters for online nonlinear/non-Gaussian Bayesian tracking[END_REF] z is obtained by a recursive computational procedure divided into successive prediction and update stages [START_REF] Arulampalam | A tutorial on particle filters for online nonlinear/non-Gaussian Bayesian tracking[END_REF]. In the prediction stage, supposing that the probability distribution function (pdf) Kalman Filter, the Gaussian sum filter, and grid-based methods [START_REF] Kotecha | Gaussian sum particle filtering[END_REF][START_REF] Saha | Comparison of prognostic algorithms for estimating remaining useful life of batteries[END_REF]. Also, Monte Carlo sampling techniques have become of increasing interest. Among these, particle filtering provides a solution by approximating the integrals in the Bayesian recursive procedure with weighted summations over a high number of samples called particles [START_REF] Doucet | Sequential Monte Carlo methods in practice[END_REF][START_REF] Arulampalam | A tutorial on particle filters for online nonlinear/non-Gaussian Bayesian tracking[END_REF].

) | Pr( 1 : 1 1 - - i i d z at time 1 - i t is
The application of the particle filtering procedure to the estimation of The sampling importance resampling (SIR) algorithm is used to avoid the degeneracy problem of the particle filtering algorithm, which consists in having all but one of the importance weights close to zero after several weight updates [START_REF] Arulampalam | A tutorial on particle filters for online nonlinear/non-Gaussian Bayesian tracking[END_REF]. This algorithm requires sampling, after one or more updates of the particle weights, a new set of particles from the old one with probability for a particle to be sampled proportional to its weight (see pseudo-code in Figure 2). New degradation trajectories have to be sampled starting from the degradation state p i d of each particle resampled at the observation time i t and new values of the particles duration of life p L are recorded. For more details the interested reader may refer to the specialized literature (e.g., [START_REF] Doucet | Sequential Monte Carlo methods in practice[END_REF][START_REF] Arulampalam | A tutorial on particle filters for online nonlinear/non-Gaussian Bayesian tracking[END_REF]). Notice that by this approach the distribution ) | Pr(

) | Pr( : 1i i RUL z is detailed in the pseudo-code given in
: 1i i RUL z is estimated, which is different from the distribution ) | Pr( i i d RUL
of the equipment RUL at time t i , given that the equipment has degradation d i at that time. However, in the Bayesian framework, this is the maximum information we can have on RUL i .

As for the uncertainty in the RUL prediction, in this approach the randomness of the degradation process (source of uncertainty A) is described by the model, whereas the observation equation accounts for the observation noise (source of uncertainty C). Thus, these two causes of uncertainty are accounted for in the RUL prediction through the procedure of particle sampling and weights updating, respectively. On the contrary, the contribution of model uncertainty to the RUL prediction uncertainty is not directly considered (source of uncertainty B), since it is assumed that the degradation dynamics model and the observation equation are exactly known. The effects of this uncertainty on the RUL prediction will be further discussed in Section 4.3.1. Notice, however, that if the uncertainty on the model parameters can be quantified and a probability distribution assigned to the value of the uncertain model parameter, the PF approach can be adjusted to handle also this source of uncertainty [START_REF] Lopes | Online Bayesian learning in dynamic models: An illustrative introduction to particle methods[END_REF]. D can then be used to train an empirical model based on one among the many data-driven modeling methods existing today (e.g., polynomial regression, nonparametric regression, neural networks, etc.). In their basic form, these methods provide in output a point prediction i l u r ˆ of the RUL without any information on the uncertainty of the estimate [START_REF] Wang | A probabilistic-based airframe integrity management model[END_REF]. To overcome this limitation, the bootstrap method for estimating the accuracy in the prediction of a stochastic output whose mean value and variance are unknown functions of the input is used in this work. Under the hypothesis that the model is as an unbiased estimator of

Approach 2: bootstrapped ensemble of empirical models trained on sequences of degradation observations and life time data

i i RUL z | µ , i.e., i i RUL i f E z z | )] ( [ µ = , the model error variance 2 B
σ can be rewritten as follows [START_REF] Heskes | Practical confidence and prediction intervals in Advances[END_REF][START_REF] Zio | A Study of the bootstrap method for estimating the accuracy of artificial neural networks. Predicting nuclear transient processes[END_REF]:

( ) [ ] ( ) [ ] ( ) [ ] ( ) [ ] 2 2 | 2 2 | 2 )] ( [ ) ( )] ( [ )] ( [ ) ( ) ( ) ( i i RUL i i i RUL i i B f E f E f E E f E f E f E i i i i z z z z z z z z z - = - + - = - = µ µ σ (6)
An estimate of the model error variance, With respect to the estimate of the remaining part of the RUL prediction variance, which is caused by the randomness of the degradation process and the observation noise (sources of uncertainty A and C ), i.e.

residuals s i r : ) ( ) ( , 2 2 s val i B s i s i s i rul l u r r z σ - - = (7)
The set of input/output pairs obtained by associating to the observations

s val i , z , val S s ,..., 1 = , i=1,…,N s , in val o i / D the corresponding residuals s i r is used for training an empirical model ) ( ) ( 2 i C A i z z + = σ χ
of the residual variance approximating the unknown relation between the input i z and the variance of the residuals [START_REF] Heskes | Practical confidence and prediction intervals in Advances[END_REF][START_REF] Carney | Confidence and prediction intervals for neural network ensembles[END_REF].

When a new observation i z is collected, the following procedure is applied in order to obtain the estimate i l u r ˆ of the equipment RUL and of the corresponding variance 2 ˆi l u r σ :

• Compute the output h i l u r ˆ of each models ) | ( / h o i i h f D z
of the ensemble;

• Compute the point estimate of the RUL:

i i l u r f ) ( = z ∑ = = H b h o i i h i f H l u r 1 / ) | ( 1 ˆD z (8)
• Compute the RUL prediction uncertainty as follows:

) ( )] | ( var[ ) ( ) ( ˆ/ 2 2 2 ˆi h o i i h i C A i B l u r f i z D z z z χ σ σ σ + = + = + (9)
Then, 2 ˆi l u r σ accounts for all three sources of uncertainty listed in Section 1.

However, the degradation measurements depend on the entire past trajectory of degradation, which means that training and validation data taken from the same trajectory are not independent, causing an underestimation of the variance. For this reason, the validation dataset The approach differs from approach 1 in that the stochastic model describing the dynamics of the degradation process is not available and actually the point is to develop it empirically. Coherently, the estimate of the prediction error variance ) (

∑ = = H h h i i l u r H l u r 1 1 ˆ ( 13
) ) ( )] | ( ṽar[ ) ( ) ( ˆ, , 2 2 2 ˆth i h th i h i C A i B l u r i d D d z z χ η σ σ σ + = + = + (14)
As for approach 2, all three sources of uncertainty listed in Section 1 are taken into account in the estimate 2 ˆi l u r σ .

Numerical application

The three different approaches presented in Section 4 are verified with respect to the RUL prediction of a simulated turbine blade undergoing degradation. The application focuses on the turbine of a generation IV high temperature gas reactor, which is characterized by rather extreme turbine operational conditions such as working temperatures exceeding 900°C. The predominant damage mechanisms affecting turbines operating at such elevated temperatures include creep deformation, corrosion and fatigue [START_REF] Carter | Common failures in gas turbine blades[END_REF]. The interaction of these and other mechanisms generates a degradation process that leads to crack initiation which rapidly leads to failure due to the quick accumulation of stress cycles caused by the high rotational speed. Notice that a turbine undergoing this degradation process can experience the loss of its blades, one of the most feared failure modes of turbomachinery since it is accompanied by abrupt changes in the power conversion equipment and in the reactor flow conditions [START_REF] Dunn | Preliminary safety evaluation of the gas turbine-modular Helium reactor (GT-MHR)[END_REF]. Figure 3 shows an example of high-pressure turbine deblading occurred in a German power plant [START_REF] Saez | Analysis of the turbine deblading in an HTGR with the CATHARE code[END_REF]. Also, fracture in rotary machines can result in turbine missiles, i.e., irregularly shaped projectiles travelling at high velocities which can impact on barriers in nuclear power plants causing severe damages to the facilities, and threatening public safety [START_REF] Made | Local damage assessment of turbine missile impact on composite and multiple barriers[END_REF]. This and the high cost of turbine blade replacement are strong reasons for performing prognostics on creeping turbine blades. In [START_REF] Flotow | Health monitoring and prognostics of blades and disks with blade tip sensors[END_REF] it is observed that long-term trends in creep-related blade deformation is anticipated analysis and diagnosis is analyzed and a proposed. In [START_REF] Dowell | Progress in turbomachinery prognostics and health management via eddy-current sensing[END_REF] ealth monitoring and prognostics may allow scheduling maintenance actions before the blade degradation evolves into cracks. In this context, the dimensionless quantity ε, defined as longation of the turbine blade in the longitudinal direction with respect to its original length an indicator of the blade degradation state [START_REF] Goel | Health risk assessment and prognosis of gas turbine blades by simulation and statistical methods[END_REF]. The blade is discarded when the accumulated determined value, namely the failure threshold, which assures that the risk of blade failure is 45] various sensing technologies for measuring blade deformations are possible application to turbine blade and disk health monitoring are blade tip clearance (BTP), which defines the gap between the tip of blades appear to be promising for measuring blade plastic elongation. In this respect, the main available today are based on capacitive measurements, eddy current or microwave sensing. 47] are already on the market [48-49] and provide clearance and vibration monitoring sensors to gas turbines used in the power generation and aerospace industries. frequency eddy currents actively induced in passing blades, is tested on field trials is demonstrated the capability of generating online clearance measureme blade. Similar results are obtained in [START_REF] Kwapisz | application of microwave sensing to blade health monitoring[END_REF] using a microwave sensing system, which is claimed to be capable of performing blade monitoring in the harsh environment of the first turbine stages. term trends in BTP can be measured and its future application to monitoring deformation is anticipated. In [START_REF] Goel | Health risk assessment and prognosis of gas turbine blades by simulation and statistical methods[END_REF], the possibility of using BTP in blade failure risk analysis and diagnosis is analyzed and a blade prognostic approach based on BTP linear regression is a general framework for PHM of turbine blades is proposed, considering blade tip other damage indicators such as vibrations, blade angular position, etc. advancements in blade tip clearance sensing technology, a sufficient amount of data for training and validation of prognostic models can be expected to be available in the notice that the amount of real data necessary to validate the prognostic approaches deve e-out cross-validation procedure [START_REF] Polikar | Comparative multiresolution wavelet analysis of ERP spectral bands using an ensemble of classifiers approach for early diagnosis of Alzheimer's disease[END_REF].

Degradation model

Modeling the degradation of a turbine blade is a hard task, especially if one needs to take into account all mechanisms involved and their interactions. For the purpose of this work, we limit ourselves to considering the accumulation of creep damage. Creep is an irreversible deformation process affecting materials exposed to a load below their elastic limit for a protracted length of time and at high temperatures. In the high pressure turbine first stage, blades creep is a major problem due to the high operational temperatures, and is often the life-limiting process [START_REF] Carter | Common failures in gas turbine blades[END_REF]. Blade elongation, ε, is taken as a measure of the blade creep strain.

In this work, the creep evolution is modeled using the Norton Law discretized with a step = ∆t 5 days, assuming that the dependence from the temperature follows the Arrhenius law [START_REF] Baraldi | A Kalman filter -based ensemble approach with application to turbine creep prognostics[END_REF][START_REF] Swindeman | A comparison of creep models for nickel base alloy for advanced energy systems[END_REF]:

[ ] t K RT Q A n j j j j j ∆ ⋅ + ⋅         - ⋅ + = + δϕ ω ε ε 2 1 exp , 0 0 = ε (15)
where j ε is the creep strain at time j t , Q is the activation energy, A and n are material inherent characteristics varying from one blade to another, K is a constant relating the load to the rotational speed j ω , R is the ideal gas constant, j T is the blade operating temperature and The values of the parameters j T , j ω and K have been set with reference to the helium gas turbine of a Gas Turbine Modular Helium Reactor (GT-MHR) developed by an international consortium, with a targeted 286MWe generation per module [START_REF] Couturier | High temperature alloys for the HTGR gas turbine: Required properties and development needs[END_REF]; the material inherent characteristics A and n are taken assuming that the blade is made of Ni-base cast Superalloy 713LC [START_REF] Couturier | High temperature alloys for the HTGR gas turbine: Required properties and development needs[END_REF]. The distributions used for the parameters are reported in Table II. Eq. ( 15) represents a stochastic process whose unknowable future evolution (cause A, Section 1) produces an irreducible uncertainty in the RUL prediction. Parameters A and n instead represent an uncertainty in the model (source B, Section 1). In fact, to a specific blade correspond fixed parameters A and n but their exact values are not known in practice; to include this source of uncertainty in the model, we assume to know with a certain precision the range of values of these parameters and associate to them a probability distribution (Table II).

For simplicity, it is assumed that it is possible to directly measure at inspection time t j the value of the creep strain ε j . Thus, the observation equation is:

j j j υ ε + = z ( 16
)
where j υ is a white Gaussian measurement noise with standard deviation σ υ =0.02. Then, the likelihood The failure threshold for creep strain th ε is set equal to the value of 1.5%.

Given the unavailability of real experimental data, a sequence of creep strain measurements i : 1

z on the blade of interest, hereafter called 'test trajectory', is simulated using eq. ( 15). The variation in time of the rotational speed ω, the gas temperature T and the stress fluctuations δφ are simulated by sampling their values ω j , T j and δφ j from the relative distributions (Table II) at each time instant t j . Every 30 days a measurement j z , corresponding to the creep strain j ε , is simulated by using eq. ( 16). A total number of 87 creep strain measurements have been simulated for a turbine blade with parameters A=3•10 -4 and n=6.

In order to verify the performance of the prognostic approaches, the simulation of the test trajectory has been conducted until the time L at which the creep strain reaches the failure threshold. The difference between L and the time t i at which the prognosis is performed is the RUL of the turbine blade; it will be referred to as "true RUL", and represented by the notation i rul .

Also, a number S=13 of historical creep growth trajectories of similar blades have been simulated using eq.

(15). To induce variability in the behavior of the similar blades the values of the characteristic parameters A

and n from one blade to another have been sampled from normal distributions (Table II) at the beginning of each new simulated degradation trajectory. Some examples of simulated creep growth trajectory are shown in Figure 4.

For each trajectory, a sequence of N s direct creep strain measurement s N : 1

z

, one every 30 days, are simulated according to eq. ( 16).

Results

During the life of the turbine blade, at every time i t , the set of observations i : 1

z is assumed to be available; the objective of the analysis is to predict at time i t , i=1,…,87, the RUL distribution for the test trajectory.

Three situations have been artificially constructed for the turbine blade case study described in the previous Section 4.1, corresponding to the three prognostic approaches of Section 3.

In the PF approach 1, the model of eq. ( 15) is used to simulate a number 1000 = P of particles starting from 0 0 = ε . Particle resampling is performed once every 5 measurements. Note that the particle filter has been preferred to the Kalman filter since the distribution of the process noise is not Gaussian as a consequence of the combination of speed, temperature and stress fluctuations in the creep growth process described by eq. ( 15).

In the bootstrapped ensemble approach 2, which uses multiple sequences of degradation observations, S trn =10 trajectories among the S = 13 totally available are used for building an ensemble of H=25 linear least square models

i h h h o i i h D l u r z z ⋅ + = 1 0 / ) | ( ˆα α
, whereas the remaining S val =3 trajectories are used to validate the ensemble and build the training dataset for the least squares model 25%. An ensemble of H=25 linear least square models

j h h h j h t t ⋅ + = 1 0 ) | ( β β η D is built and the models h j j h j j h 1 ' , ' , / ] | [ ~β η z D z ∆ = ∆
are derived from it. Notice that in a linear process, the time needed to increase the degradation level from j ε to ' j ε is proportional to the degradation increment This interval is obtained as follows:

j j j j ε ε ε - = ∆ ' ' ,
• In approach 1, ) ( can be computed according to the theory of the bootstrap method [START_REF] Heskes | Practical confidence and prediction intervals in Advances[END_REF] as: The prediction intervals provided by approach 3 are characterized by large oscillations and low accuracy, especially at the beginning of the trajectory, i.e., when few training data are available. Furthermore, the RUL prediction itself is noisy. This effect can be reduced by properly filtering the predictions. To this purpose, since the time evolution of the RUL is a linear process

i l u r conf i i c l u r C înf ) ( σ α α - = and i l u r conf i i c l u r C ŝup ) ( σ α α + = (17)
( 1 ) 1 ( ) ( - - = t rul t rul
), and assuming a Gaussian noise of the prediction, Kalman filtering can be applied [START_REF] Baraldi | A Kalman filter -based ensemble approach with application to turbine creep prognostics[END_REF].

RUL distribution and prediction interval

The objective of this Section is to determine whether the estimates of the prediction intervals provided by the three approaches properly describe the uncertainty in the RUL predictions. In practice, we want to know whether the estimates 2

RUL i True RUL (rul i ) Predicted RUL (rûl i ) C i inf/sup (0.32) C i sup (0 .32) Approach 1 Approach 2 Approach 3 ( ) [ ] ( )
∑ = - ≅ - = P p p i p i i i l u r P rul l u r RUL l u r E i 1 2 2 2 ˆσ (18)
Similarly, the computation of the true value of

( ) [ ] 2 | 2 i RUL A RUL E p i i - = ε µ σ is approximated by: ∑ = - ≅ P p p i RUL A rul P i i 1 2 | 2 ] [ 1 ε µ σ (19)
where the RUL expected value

Prediction interval provided by the PF approach 1

The specific blade undergoing the creep degradation process is characterized by fixed values of the parameters A and n in eq. [START_REF] Crisan | A survey of convergence results on particle filtering methods for practitioners[END_REF], which in general are not known. In this Section, in order to evaluate the PF performance in the estimate of the prediction uncertainty, the approach is firstly applied assuming to know the exact value of these parameters. In Figure 7 Differently from approach 1, the ensemble approach 2 is not able to learn the true values of A and n of the current test trajectory. Furthermore, this limitation of the model is not properly described by the prediction interval provided by the ensemble.

The reason for which the proposed bootstrap approach 2 is not able to correctly model the evolution of the error made by the model for a specific test trajectory is that the assumption that the predictive model ) ( i f z is an unbiased estimator of the RUL expected value i RUL µ , is not fully verified. In fact, if we build several ensemble models trained with different randomly chosen datasets and the RUL prediction with each one of them, we notice that the average RUL prediction These results confirm that the bootstrap approach 2 can actually provide a satisfactory estimates of

) ( i C A ε σ + , ) ( i B ε σ and ) ( ˆi l u r i ε σ
for a population of different turbine blades, and thus the proposed approach correctly quantifies the uncertainty of the prediction produced by the prognostic model for a generic creeping blade.

Prediction interval provided by approach 3

Figure 15 shows the analogous results of Figure 9 To understand the reason for which the proposed bootstrap approach 3 tends to underestimate the model error variance in this case, it must be pointed out that, from a probabilistic point of view, the single available trajectory used for training the models is only one of an infinite number of possible trajectories, which may be drawn from the creep growth process we wish to model. Thus, bootstrap sampling of creep strain measurements from a single degradation trajectory does not account for the variability of all possible degradation trajectories.

In the case of a linear process, it is possible to overcome this limitation, by considering, instead of the sequence of creep strain measurements i z :

1 , the set of independent creep strain increments for time unit

) /( ) ( 1 1 j j j j j t t z z z - - = ∆ + + , 1 ,..., 1 - = i j
. In this way, the variability of the training data is increased, and a better representation of the intrinsic variability of the process is provided. An accurate model of the process can still be achieved by estimating the parameters β h , h=1,…,H, of the models of the ensemble . Figure 16 shows that a more accurate estimate of B σ and l u r σ is achieved using this new ensemble of models.

h j j j j h β ε ε η / ) ( ~' , ' , ∆ = ∆

Conclusions

Three prognostic approaches have been investigated, particularly with respect to the treatment of the uncertainty in the predicted equipment RUL. Quantitative considerations have been made with regards to a simulated case study concerning the creep growth process in a high temperature turbine blade. The results show that the particle filtering approach provides a good approximation of the exact distribution of the equipment RUL in the case in which an accurate model reproducing the equipment degradation process is available. A limit of particle filtering is particle impoverishment, which relates to the failure of maintaining the diversity of particles and is caused by the resampling approach adopted to avoid particle degeneracy.

Particle impoverishment implies the impossibility of the particles to correctly represent all possible evolutions of the degradation process, which include, for example, changes in time due to the variation of operating conditions. In this context, different resampling methods, such as the one proposed in [START_REF] Li | Deterministic resampling: Unbiased sampling to avoid sample impoverishment in particle filters[END_REF], which samples particle considering not only the particle weight but also their spatial distribution (state values), can be considered. When using model-based approaches, imprecision of the model in the reproduction of the degradation process due to simplifications, incorrect model structure or assumptions on the equipment specific geometries or material properties, etc. can be amplified over time, causing unreliable estimates of the RUL distribution. Using particle filtering, it is possible to include model parameters in the state vector and, thus, perform model adaptation in conjunction with state tracking. In any case, it is very difficult for a physicsbased model to account for all aspects of a degradation process; for example, it is common to neglect some of the interactions between different degradation mechanisms or the possible existence of self-healing mechanisms which can reverse the degradation process and are likely to increase the uncertainty of the future degradation evolution. All these non-modeled phenomena can be accounted for by adding further noise to the process model which will result in a larger confidence interval associated to the RUL estimate. Further research is needed to quantify the impact of modeling errors on the final prediction of model-based approaches.

In the bootstrap approaches 2 and 3 considered, it has been shown that a reliable prediction of the equipment RUL with a correct quantification of its uncertainty can be obtained. With respect to the ensemble of bootstrapped models trained with historical sequences of observations in approach 2, the main limitation is that it is not able to learn the peculiar characteristics of the equipment of interest but it tends to reproduce an 'average' behavior. To overtake this problem, a different modeling approach could be used, such as that based on the idea of fuzzy similarity [START_REF] Zio | A data-driven fuzzy approach for predicting the remaining useful life in dynamic failure scenarios of a nuclear system[END_REF], or a procedure for updating the ensemble with the information conveyed by new observations [START_REF] Baraldi | A Kalman filter -based ensemble approach with application to turbine creep prognostics[END_REF].

The application of the bootstrap ensemble in the time series scheme of approach 3, in which only direct measurements of the degradation experienced by the equipment of interest are available, has shown the importance of injecting diversity into the bootstrapped models by using independent training data, in order to correctly quantify the modeling error. The case study considered is characterized by a linear degradation process, so that independent training data can be obtained by considering the creep strain increments between consecutive measurements; on the contrary, this would not be feasible for non-linear degradation processes. Furthermore, in this case of very little information available, the bootstrap method requires building an empirical model for the RUL variance estimate which is then used outside the region covered by the training data. Although good extrapolations have been obtained in the linear creep growth case study, the feasibility of the approach on more complex models should be verified.

Contrarily to physics-based models, we expect that data-driven methods can automatically learn from data the effects on the equipment RUL of phenomena influencing the degradation process, such as self-healing and interactions between different degradation mechanisms. The capability of data-driven methods of providing correct estimates of the RUL and its uncertain distribution depends on the availability in the training set of examples of the phenomena that we want to represent.

In this work, the problem of detecting the initiation of a degradation process, which is usually achieved by using properly developed diagnostic systems, has not been addressed. Although none of the approaches presented in this work requires knowing the exact time at which degradation has initiated, late detection of an ongoing degradation process will reduce the number of degradation measures available for prognostics;

this is expected to reduce the RUL prediction accuracy and increase its uncertainty especially in approaches 1 and 3 which, contrarily to approach 2, generate their RUL prediction on the basis of past degradation measurements. . Since only artificial data have been used in the case study considered in this work, conclusions about the successful application of these approaches in the field cannot be directly drawn. The analyses performed have shown the potential of these methods in performing RUL prediction with adequate management of its uncertainty; in this sense, they hold promises for future research aimed at confirming this potential in the application to real data.

Finally

  variance related to the imprecision of the model ) ( i f z (source of uncertainty B) and the noise on the data i z (source of uncertainty C):

Figure 1 .-

 1 The P particles P p ,..., 1 = are future degradation trajectories built by recursively sampling the particle degradation state p j d at time j t from the transition probability distribution derived from the degradation model, until the failure threshold th d is exceeded and the length of life p L of the particle is recorded. The value p i rul of the particle RUL at time step i t by an histogram of the P weighted values p i rul of the particle RULs at time i t ; the weighted average and the weighted standard deviation of the values

Figure 1 :

 1 Figure 1: Particle filtering operative procedure for RUL estimation [8].

FOR 10 .Figure 2 :

 102 Figure 2: Procedure for performing resampling at time i t [8].

Approach 2 : 1 z 1 z

 211 is based on the development of an empirical model f representing the relationship between the degradation observations i z available at time and the corresponding equipment RUL. The empirical model is built considering the observations of a set of S trajectories s N s , s=1,…,S, of similar equipments which have each reached failure in a time L s discretized in N s steps. The empirical model receives in input the observations i z and produces as output the RUL prediction, i l u r ˆ. In order to develop the model, a dataset of input/output pairs } is extracted from the set of observations s N s : by associating to the observations s i z at time i t along the s-th trajectory to failure the corresponding

1 ˆ

 1 to the observations in the validation dataset val o i / D . The obtained RUL predictions , s=S trn +1,…,S are used to calculate, for each validation observation s val i , z , the prediction

4 . 3 1 z 1 z

 4311 replacement from the total amount of input/output pairs in trn o i / D . Approach 3: Bootstrapped ensemble of empirical models trained on a sequence of past degradation observations from the equipment of interest only Approach 3 is based on the development of an empirical model of the degradation process based on the time series of its past observations i : and used for identifying the time at which the degradation will exceed the failure threshold. For simplicity of illustration the observations i : are assumed to be direct measures of the degradation d 1:i , eventually affected by noise.

Figure 3 :

 3 Figure 3: Deblading in a high pressure turbine

  a general framework clearance and other damage indicators such as vibrations, blade ang Notice that, given the advancements in for training and validation of prognostic models notice that the amount of real data necessary to validate the reduced by resorting to the leave-on : Deblading in a high pressure turbine [42].

j

  δϕ is a random variable modeling the fluctuations in the stress applied to a specific blade, which are due to fabrication defects, aging and corrosion of the blade, vibrations of the equipment or turbulences of the gas flow. Oscillations of the rotational speed j ω , and of the blade operating temperature j T , are represented by considering their deviations from the mean values ω µ and T µ as noises. In practice, with mean values zero and assigned standard deviations. Thus, the noise vector j γ in eq. (5) can be set equal to

  of uncertainty (source C, Section 1) in the final RUL prediction.

Figure 4 :

 4 Figure 4: Examples of creep growth trajectories.

  ensemble approach 3, which uses the time series of degradation observations, the prognostic model has been developed only after time[START_REF] Wang | Reliability and degradation modeling with random or uncertain failure threshold[END_REF] t in order to have available a dataset } i=30 direct creep strain measurements. This dataset has been partitioned into a training dataset D trn containing the first 75% of the available measurements and a validation dataset D val containing the remaining

,.

  and does not depend on the initial and final degradation values. The ensemble of models is tested on the validation dataset made of input/output pairs j=N trn +1,…,i-1, j'=j+1,…,i and the prediction residuals ' , j j r obtained are used to train the linear model ' variance of ∆t. After time 30 t , each time i t , i=31,…,87, a new measurement becomes available, a new ensemble of models h η ~ and a new model χ for the prediction error variance are built. The predictions from the models h η ~ of the ensemble and from model χ in correspondence of the input Since the data used for training model χ concern creep strainincrements which for the first two thirds of the trajectory are smaller than the increment an input region not described by the training data.In all three approaches, in correspondence of each prediction e., the interval expected to contain the true RUL value i rul with a probability of 1-α.

  , of the RUL distribution estimated by Particle Filtering.• In approaches 2 and 3, assuming that the prediction error has a Gaussian distribution, the value of

  Student's t-distribution with number of degrees of freedom equal to the number H of bootstrap models.

Figure 5

 5 Figure5shows the evolution of the true value rul i of the blade RUL (continuous thick line), its estimated value

Figure 5 :

 5 Figure 5: true rul i (continuous thick line) of a turbine blade, predicted value i l u r ˆ (dots) and prediction interval

2 Aσ , the model error, 2 ˆBσ

 22 the sum of three terms, due, respectively, to the process randomness, , and the noise on the observations, 2 C σ . It is also of interest to consider the term three approaches proposed do not estimate these two terms separately. The computation of the true value of 2 ˆi l u r σ would ideally require the availability of an infinite number P of equipment degradation trajectories which at time t i are in the degradation state ε i . Since in the case study here considered we can artificially generate degradation trajectories, an high number P=1000 of degradation trajectories has been used to numerically approximate the variance 2 ˆi l u r σ . For the p-th simulated trajectory, we have computed: L p being the equipment life duration along the p-th trajectory, 2) the equipment RUL prediction, p i l u r ˆ, provided by the prognostic model in correspondence of the observations

  by considering the P=1000 equipment degradation trajectories which at time t i are in the degradation state ε i and for which the observations p i z have been collected, and computing: by simulating P′=1000 new degradation trajectories, each one starting from a different degradation state on the approach. In Figure 6 (left), the true RUL distribution in Figure 6 (right) the true values of the standard deviations A σ (continuous line) and C A+ σ (dots) are reported as a function of the blade creep strain level i ε .

Figure 6 A σ and 2 Cσ

 62 Figure 6: pdf ) | Pr( i i RUL ε of the RUL of a turbine blade at three different instances of the degradation trajectory (left) and evolution of the RUL standard deviations A σ (continuous line) and C A+ σ

  Figure 7: comparison of the pdf ) | Pr( i i RUL ε (left, continuous line) and the standard deviation C A+ σ

Figure 8 :

 8 Figure 8: comparison of the pdf ) | Pr( i i RUL ε (left, continuous line) and the standard deviation C A+ σ

Figure 11 :

 11 Figure 11: comparison of the bootstrap estimates (dashed line) and true (continuous line) values of B σ during the life of

Figure 12 :.

 12 Figure 12: comparison of the distributions of the prediction

Figure 13 :

 13 Figure 13: comparison of the distributions of the prediction

Figure 14 :

 14 Figure 14: comparison of the bootstrap estimates (dashed line) of

Figure 15 :

 15 Figure15shows the analogous results of Figure9for approach 3. Notice that good estimates of ) ( i C A ε σ +

  as the average value of the creep strain increments of the bootstrap replicate h incr

Figure 16 :

 16 Figure 16: comparison of the bootstrap estimates (dots) and true values (continuous line) of C A+ σ

Figure

  Figure 1A: procedure for approximating ) ( 2 i B ε σ in approaches 2 and 3.

Table I : nomenclature

 I

	Symbol Description	Symbol	Description
	t i			i-th time instant	h						reference index of the ensemble models
	d i			equipment degradation at time t i	H					number of empirical models
	z i			physical observations related to d i	i r	s			prediction residual obtained in correspondence
	RUL i	random variable (rv) representing system RUL							of the input s i z
				at time t i	χ						empirical model for the prediction of the
	δ i			difference between RUL i and its expected value							residuals variance
				E[RUL i ]	d ˆ	i			prediction of d i at time t i
	rul i	realization of RUL i	η(t i )	empirical model for the prediction of d i
	f(z i )	prognostic model for the prediction of rul i	d j,j'	vector of the degradation states at t j and t j'
	rûl i	prediction of rul i	∆t j,j'	rv representing the time interval to evolve
	σ	2 ˆi l u r	prediction error variance							from d j to d j'
	σ	2 A	prediction error variance due to randomness in the future degradation of the equipment	( ~' , j j d η ' , ˆj j t ∆	)	empirical model for the prediction of ∆t j,j' estimate of ∆t j,j'
	σ	2 B	prediction error variance due to modeling error	δ		∆	t	j	, j	'	difference between ∆t j,j' and its mean value
	σ	2 C	prediction error variance due to uncertainty in	r j,j'		prediction residual obtained in correspondence
				equipment degradation data							of the input d j,j'
	s			index of the equipment	ε j						creep strain at time t j
	S			number of equipments	Q					creep activation energy
	L s			failure time of the s-th equipment	ω j			turbine rotational speed
	d th	failure thresholds	K					constant relating the load to ω j
	Pr(x)	probability distribution function of the rv x	R						ideal gas constant
	Pr(x|y)	conditional probability distribution function of	T j					blade operating temperature at time t j
				the rv x given y	δϕ	j	stress fluctuations
	µ x		mean value of the rv x	υ j						creep strain measurement noise
	σ x			standard deviation of the rv x	ε th			creep strain failure threshold
	g			transition function of the degradation state	α		h , 1 0 / β	h , 1 0	coefficients of the h-th empirical model of
	γ j			process noise vector							approach 2/3
	2 ˆx σ	estimate of 2 x σ	γ	0	, 1 ,	2	/	λ	coefficients of the empirical model of the
	p			index of the Monte Carlo sampled particle							residual variance χ of approach 2/3
	P			number of particles	∆ε j,j'	degradation increment from t j to t j'
	w	p i	weight of the p-th particle at time t i	∆	z	j	, j	'	observed degradation increment from t j to t j'
	D			dataset made by the observations z i	α						significance level of the confidence interval
	D i/o	dataset of the input/output pairs ( s i z , s i rul )	C	inf/ i	sup	( α	)	inferior/superior bound of the (1-α)-
	trn/val	apex/subscript indicating training/ validation							confidence interval for the RUL prediction
				sets of data	c	α conf	(1-α/2) percentile of a Student's t-distribution
	N trn/val	number of training/validation patterns							with H degrees of freedom

Sample a particle k with probability equal to its weight

	At time i t
	1. Compute		rul	p i		and	p i wn as in Figure 1
	FOR p=1:P
	2. k i wn
	3. set	p i d =	d	k i	, j=i and	j t t =	i
	WHILE	j d <	d	th
	4. j=j+1;	t	j	t = -1 j	+	t ∆
	5. sample	p j d ~) | Pr( 1 p j p j d d -
	END WHILE
	6. Register the new particle failure time	p f t =	t	j
	END FOR
	7. Assign equal weights to each particle	wn P i : 1 =	1	P
	8.

Build the RUL probability density function and compute its moments

	realization of RUL i , i.e.,	rul	s i	=	s L	-	t	i	. The dataset	i /	o
	i l u r ˆ and	σ	2 ˆl u r	as in Figure 1.
										t	i

Table II : type of distribution, mean value and standard deviation used for the creep growth model parameters

 II

	Variable	Symbol	Distribution	Units	Parameters of the distribution
	Activation energy	Q	Deterministic kJ/mol	Q=290
	Norton Law parameters	A	Normal	(N/m 2 ) -n /day	µ A =7.2•10 -3 ; σ A =5%
		n	Normal	-	µ n =6; σ n =0.2%
	Operating temperature	T j	Normal	K	µ T =1100; σ T =1%
	Rotational speed Load parameter	ω j K	Normal Deterministic Kg/m rpm	µ ω =3000; σ ω =1% ρ =1068
	Stress fluctuations	δφ	Gamma	MPa	θ=2; k=10

P. Baraldi is with the Politecnico di Milano, Dipartimento di Energia, via Ponzio 34/3,

Milano, Italy (phone:+39 02 23996355; fax: +39 02 23996309; e-mail: piero.baraldi@polimi.it).

for the equipment whose RUL we want to predict. In

Acknowledgments

The work of Francesca Mangili has been supported by a PhD grant of the Institutt For Energiteknikk (IFE), OECD Halden Reactor Project.

The participation of Enrico Zio in this research is partially supported by the China NSFC under grant number 71231001.

Finally, the authors wish to thank the reviewers for their constructive comments which have allowed improving the paper.

It can be noticed that the estimated prediction error standard deviation 15)), the observation equation (eq. ( 6)) and the A and n parameters distributions (Table II).

Prediction interval provided by approach 2

In order to estimate the real uncertainty affecting the RUL prediction of an ensemble of models, it is necessary to add the real model uncertainty 2

The continuous line in Figure 10 shows the real values of The dotted lines in Figure 9 represent the estimates of these quantities provided by the bootstrap ensemble.

The standard deviation

C A+ σ estimated by the bootstrap ensemble is significantly larger than its real value.

As in approach 1, this is due to the fact that the training trajectories have different values of parameters A and n and, thus, the empirical model learns the variance of a population of different blades instead of that of the specific blade with fixed values of A and n . In Figure 10, the estimate of the model error variance 2 B σ appears to be not very accurate. Figure 11 shows that the inaccuracy can be even more remarkable if other test trajectories are considered, characterized by values of A and n far away from the mean value of their respective distributions (