Hoang-Phuong Nguyen
email: hoang-phuong.nguyen@centralesupelec.fr

Jie Liu

MINES Enrico Zio
email: enrico.zio@mines-paristech.fr

Memory neural network with automatic parameter optimization by Tree-structured Parzen Estimator and applied to time-series data of NPP steam generators

Keywords: prognostics and health management, time-series forecasting, multi-step ahead prediction, long-short term memory, nuclear power plant prognostics, steam generator. Nomenclature AI Artificial intelligence ANN Artificial neural network

des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

A long-term prediction approach based on long short-term memory neural networks with automatic parameter optimization by Tree-structured Parzen Estimator and applied to time-series data of NPP steam generators

Hoang-Phuong Nguyen, Jie Liu, Enrico Zio

To cite this version:

Hoang-Phuong Nguyen, Jie Liu, Enrico Zio. A long-term prediction approach based on long shortterm memory neural networks with automatic parameter optimization by Tree-structured Parzen Estimator and applied to time-series data of NPP steam generators. (l) output vector of the hidden layer l of the LSTM network ỹ (l) thinned output vector of the hidden layer l obtained by using dropout z (l) input vector of the hidden layer l of the LSTM network In recent years, prognostics and health management (PHM) has attracted increasing attention from academic researchers and practitioners in different industrial sectors. The primary characteristic of PHM is that it can enable estimation and prediction of the health state of components and systems, by making use of past, present and future knowledge, information and data on their operations, and this capability can be used to identify malfunctions and anticipate failure patterns [START_REF] Zio | Prognostics and Health Management of Industrial Equipment[END_REF]. This allows estimating the remaining useful life (RUL) of components and systems, and scheduling the maintenance interventions for the most opportune and convenient instances. By so doing, the availability and reliability of the assets can be maximized, with reduced unscheduled shutdowns and maintenance costs.

Developing models for efficient PHM is a challenging task, with several issues to be addressed.

Among them, determining an appropriate horizon for the prediction, i.e. how far into the future the model should predict and with what accuracy, is crucial and application dependent, in the sense that it depends on the use that is made of the prediction, typically for taking some decisions [START_REF] Gouriveau | Connexionist-systems-based long term prediction approaches for prognostics[END_REF]. For instance, the selected horizon should be suitably long to allow that maintenance actions be timely carried out.

This often requires long-term predictions in practice. However, long-term predictions are known to suffer from increasing uncertainties, which may arise from the accumulation of prediction errors or from the complex interactions and correlations in the underlying process at different time steps. This has challenged and somewhat limited the research on long-term prognostics for many years [START_REF] Sorjamaa | Methodology for long-term prediction of time series[END_REF], [START_REF] Weigend | Time Series Prediction: Forecasting The Future And Understanding The Past[END_REF]. To address this problem, the main focus of this paper is the development of a prognostic framework for the long-term prediction of parameters relevant to the operation of the steam generators (SGs) in nuclear power plants (NPPs).

Depending on the information and data available for the model development, prognostic approaches can be divided into two main categories: model-based and data-driven approaches [START_REF] Lee | Prognostics and health management design for rotary machinery systems -Reviews, methodology and applications[END_REF].

Model-based approaches predict the degradation evolution by formalizing it into physical analytical or q * q computational models. These approaches are used in applications where the model of the degradation process exists and is not too complicated, e.g. models of fatigue crack growth [START_REF] Nguyen | Ensemble of Models for Fatigue Crack Growth Prognostics[END_REF], [START_REF] Nguyen | Dynamic-weighted ensemble for fatigue crack degradation state prediction[END_REF], of capacity degradation in Lithium-ion batteries [START_REF] Hu | A particle filtering and kernel smoothing-based approach for new design component prognostics[END_REF], [START_REF] Liao | A hybrid framework combining data-driven and model-based methods for system remaining useful life prediction[END_REF]. Alternatively, data-driven approaches utilize condition monitoring data collected from sensors to learn and predict the component or system behavior and degradation via statistical and artificial intelligent (AI) models, such as autoregressive integrated moving average (ARIMA) [START_REF] Nguyen | A Data-Driven Approach for Predicting the Remaining Useful Life of Steam Generators[END_REF], artificial neural network (ANN) [START_REF] Xiao | A novel neural-network approach of analog fault diagnosis based on kernel discriminant analysis and particle swarm optimization[END_REF]- [START_REF] Sanz | Gear dynamics monitoring using discrete wavelet transformation and multi-layer perceptron neural networks[END_REF], neuro-fuzzy (NF) [START_REF] Gouriveau | Connexionist-systems-based long term prediction approaches for prognostics[END_REF] and support vector machine (SVM) [START_REF] Liu | A Novel Dynamic-Weighted Probabilistic Support Vector Regression-Based Ensemble for Prognostics of Time Series Data[END_REF]- [START_REF] Li | Enhancing the Lithium-ion battery life predictability using a hybrid method[END_REF]. Due to the data-adaptive nature, data-driven approaches are quite appropriate for prognostic real-world applications where models are not available whereas obtaining condition monitoring data is becoming convenient with smart sensors.

When applying data-driven approaches to prediction, models like ANN, NF and SVM are usually limited in extracting and utilizing the temporal information of the given data which is necessary for prediction purposes. More specifically, these approaches consider each time step independently and make the prediction as a static mapping, which often takes into account only the current state of the process [START_REF] Heimes | Recurrent neural networks for remaining useful life estimation[END_REF]. Recently, a connectionist neural network model called recurrent neural network (RNN) has been proposed to account for the dynamics [START_REF] Angeline | An Evolutionary Algorithm that Constructs Recurrent Neural Networks[END_REF]. RNN is a network with feedback connections from the hidden and output layers to the preceding ones, by which the dynamics of sequential data can be captured and the memories of the previous patterns are retained via cycles in the network. In the last decade, RNNs have been extensively investigated for a variety of prognostic applications, including engine systems [START_REF] Li | An ensemble learning-based prognostic approach with degradation-dependent weights for remaining useful life prediction[END_REF]- [START_REF] Elsaid | Optimizing long short-term memory recurrent neural networks using ant colony optimization to predict turbine engine vibration[END_REF], lithium-ion batteries [START_REF] Zhang | Long short-term memory recurrent neural network for remaining useful life prediction of lithium-ion batteries[END_REF]- [START_REF] Liu | An integrated probabilistic approach to lithium-ion battery remaining useful life estimation[END_REF], rolling element bearings [START_REF] Cheng | Machine Health Monitoring Using Adaptive Kernel Spectral Clustering and Deep Long Short-Term Memory Recurrent Neural Networks[END_REF]- [START_REF] Malhi | Prognosis of defect propagation based on recurrent neural networks[END_REF] and fuel cells [START_REF] Ma | Data-driven proton exchange membrane fuel cell degradation predication through deep learning method[END_REF], [START_REF] Javed | Prognostics of Proton Exchange Membrane Fuel Cells stack using an ensemble of constraints based connectionist networks[END_REF]. Zhang et al. [START_REF] Zhang | Long short-term memory recurrent neural network for remaining useful life prediction of lithium-ion batteries[END_REF] utilized a RNN to extract the long-term dependencies underlying in the battery capacity degradation process. The obtained results showed that RNN outperformed the classical data-driven models in prediction robustness and accuracy. In [START_REF] Malhi | Prognosis of defect propagation based on recurrent neural networks[END_REF], the RNN model was modified with an incremental learning technique, which was then applied for predicting the long-term propagation of rolling element bearing degradation to failure. RNN has also been used in the construction of health indicators for generator bearings of wind turbines [START_REF] Guo | A recurrent neural network based health indicator for remaining useful life prediction of bearings[END_REF]. The obtained RNN-based health indicator showed its effectiveness for improving the prediction performance of the bearing RULs.

In this paper, a variational model of RNN, which is called long short-term memory (LSTM), is employed for developing a prognostic framework for SGs in NPPs. An important feature of the proposed framework is the ability to deal with a long-term prediction horizon. A multi-input multioutput (MIMO) prediction strategy and LSTM network are integrated to predict the equipment health conditions for multiple steps ahead. The proposed framework also handles two practical problems in prediction model development. On one hand, the performance of the prediction model depends on the time-series data acquired from sensors: any anomalous or missing data that can degrade the performance should be dealt with. On the other hand, an optimal model setting for different available datasets is crucial to successfully apply the prediction model to practical problems. The effective handling of these two issues is another contribution of this work.

In summary, the main contributions of this paper are as follows:

(1) A data preprocessing module consisting of an outlier removal and a missing data imputation methods is introduced for filtering and preparing the data for the prediction task.

( The rest of the paper is organized as follows. Section 2 presents a brief introduction of time series prediction approaches. Section 3 introduces the LSTM neural network. The proposed multi-step ahead prediction model is presented in Section 4. Section 5 describes the experimental case study, and the obtained results and their discussion are presented in Section 6. Finally, Section 7 concludes the paper.

Time series prediction: background and related work

Time series is a sequence of observations collected over time from a particular measured variable of an engineering component or system. In general, the main objectives of time series analysis are:

characterization, modeling and prediction (also called forecasting) [START_REF] Gershenfeld | The future of time series[END_REF]. Firstly, characterization aims to extract inherent structural characteristics of the measured variable, e.g. temporal trend, variance and seasonality. Then, the extracted information may be used to formulate an appropriate model for capturing long-term behavior of the system (modeling), or to estimate the evolution of the variable in the future (prediction). This section presents a brief introduction of time series prediction approaches and further discusses the strategies for multi-step ahead prediction.

Time series prediction

The beginning of time series prediction might be set in 1927 when Yule [START_REF] Yule | On a Method of Investigating Periodicities in Disturbed Series, with Special Reference to Wolfer's Sunspot Numbers[END_REF] introduced the first autoregressive technique for predicting the annual number of sunspots. In that original work, the prediction of the next time step was estimated as a weighted sum of previous observations of the time series. This idea has become the basis of data-driven approaches for time series prediction since then.

Among data-driven approaches, statistical models which attempt to express the future values as a linear function of the historical data have been popular and widely used in many applications of time series prediction, such as wind energy generation [START_REF] Erdem | ARMA based approaches for forecasting the tuple of wind speed and direction[END_REF]- [START_REF] Zuluaga | Short-term wind speed prediction based on robust Kalman filtering: An experimental comparison[END_REF], weather forecasting [START_REF] Wu | Prediction of short-term wind and wave conditions for marine operations using a multi-step-ahead decomposition-ANFIS model and quantification of its uncertainty[END_REF]- [START_REF] Mariño | Forecasting of reference crop evapotranspiration[END_REF], market demand forecasting [START_REF] Ohyver | Arima Model for Forecasting the Price of Medium Quality Rice to Anticipate Price Fluctuations[END_REF], [START_REF] Wang | A novel hybridization of nonlinear grey model and linear ARIMA residual correction for forecasting U.S. shale oil production[END_REF] and nuclear component prognostics [START_REF] Nguyen | A Data-Driven Approach for Predicting the Remaining Useful Life of Steam Generators[END_REF]. Erdem and Shi [START_REF] Erdem | ARMA based approaches for forecasting the tuple of wind speed and direction[END_REF] used an autoregressive moving average (ARMA) for predicting wind speed and direction. Kavasseri and Seetharamen [START_REF] Kavasseri | Day-ahead wind speed forecasting using f-ARIMA models[END_REF] proposed a variant model of ARIMA which was called fractional-ARIMA in order to extract the long dependency features of the time series data and enhance the prediction accuracy over long-term horizons. In a nuclear application, Nguyen et al. [START_REF] Nguyen | A Data-Driven Approach for Predicting the Remaining Useful Life of Steam Generators[END_REF] applied an ARIMA model to predict the long-term evolution of the tube supporting plate clogging degradation of NPP SGs for the first time, in which the predictions were performed up to 3 months ahead.

Although statistical models have shown their notable prediction accuracy and flexibility in different time series applications, one of their major drawbacks is the presumed linear form of the associated data, which has limited their applicability to many modern dynamic systems where the collected data are usually nonlinear and non-stationary [START_REF] Qin | Red tide time series forecasting by combining ARIMA and deep belief network[END_REF], [START_REF] Wang | Multi-step ahead wind speed prediction based on optimal feature extraction, long short term memory neural network and error correction strategy[END_REF]. To address this problem, several machine learning algorithms have been employed in the time series prediction area, such as SVM [START_REF] Berbić | Application of neural networks and support vector machine for significant wave height prediction[END_REF]- [START_REF] Wang | Short-Term Wind Speed Prediction Using Signal Preprocessing Technique and Evolutionary Support Vector Regression[END_REF], ANNs [START_REF] Chang | An improved neural network-based approach for short-term wind speed and power forecast[END_REF]- [START_REF] Li | On comparing three artificial neural networks for wind speed forecasting[END_REF] and fuzzy interference system (FIS) [START_REF] Catalão | Hybrid wavelet-PSO-ANFIS approach for short-term wind power forecasting in Portugal[END_REF], [START_REF] Özger | Prediction of wave parameters by using fuzzy logic approach[END_REF]. Unlike statistical approaches, machine learning models can automatically learn arbitrary complex mappings between inputs and outputs directly from the historical data and perform accurate predictions without any assumption about the mapping functions required. In addition, another advantage of machine learning approaches is the recently rapid advancements of information science technologies, particularly Big Data and deep learning techniques, which are offering opportunities for new developments in time series analysis. Kuremoto et al. [START_REF] Kuremoto | Time series forecasting using a deep belief network with restricted Boltzmann machines[END_REF] proposed a deep belief network (DBN) with restricted Boltzmann machines (RBMs) to address problems of initialization and local optima in chaotic time series forecasting, which was shown to outperform conventional shallow learning models. Wang et al. [START_REF] Wang | Deep learning based ensemble approach for probabilistic wind power forecasting[END_REF] presented a prediction model for probabilistic wind power prediction based on a convolutional neural network (CNN) model to automatically extract deep invariant structures and hidden nonlinear features exhibited at separated frequency bands of the data. A specialized kind of deep neural networks proposed for sequential data analysis is RNNs, which aim to capture the dynamics of sequential data and be able to retain the memories of the previous patterns via cycles of feedback connections between the network layers [START_REF] Angeline | An Evolutionary Algorithm that Constructs Recurrent Neural Networks[END_REF].

Wang and Li [START_REF] Wang | Multi-step ahead wind speed prediction based on optimal feature extraction, long short term memory neural network and error correction strategy[END_REF] presented a hybrid model integrating a RNN model and an optimal feature extraction technique for multi-step ahead wind speed prediction. Likewise, Li et al. [START_REF] Li | Multi-step wind speed forecasting using EWT decomposition, LSTM principal computing, RELM subordinate computing and IEWT reconstruction[END_REF] utilized LSTM RNN for predicting 5 steps ahead of the wind speed time series.

Multi-step ahead prediction strategies

Given a univariate time series of the observations collected up to time t, , the main goal is to predict the H next observations , which can be formulated as below:

, ( 1
)
where f is the prediction model and d is the embedding dimension (or the number of lagged values).

{ } 1 2 , ,..., t x x x { } ˆ, [1, ] t h x h H + Î { } 1 2 1 1
ˆˆ, ,..., ( , ,..., )

t t t H t t t d x x x f x x x + + + - -+ =
Depending on the desired horizon H, a prediction method can be classified into short-, medium-, or long-term prediction. As aforementioned, the further in the future one attempts to predict, the harder it is to achieve an accurate prediction due to the increasing uncertainty and accumulation of errors. To address this problem, there are three popular prediction strategies, namely recursive, direct and MIMO predictions, which are described as follows [START_REF] Gouriveau | Connexionist-systems-based long term prediction approaches for prognostics[END_REF].

Recursive prediction strategy

The recursive strategy attempts to train a model focused solely on one-step ahead prediction:

(

) 2
where fR is the one-step ahead prediction model.

After the model is trained, the predictions are recursively estimated. In other words, intermediate predictions are used as inputs for predicting next values until the prediction at the time horizon H, , is obtained:

(3)

An advantage of the recursive strategy is its low computational cost since only one single model is required for training. However, the prediction errors of the previous steps can easily accumulate in the next predictions, resulting in the decrease of accuracy in the long run. Besides, this prediction strategy does not take into account the data dependencies among time steps.

Direct prediction strategy

In contrast to the recursive strategy which uses a single model, the direct strategy [START_REF] Cox | Prediction by Exponentially Weighted Moving Averages and Related Methods[END_REF] constructs a set of H different models for different time steps and the same input data are used for feeding all the models as below:

(4)

1 1 1 ˆ( , ,..., ) t R t t t d x f x x x + - -+ = ˆt H x + 1 1 1 2 1 2 1 2 1 ˆ( , ,..., ) ˆ( , ,..., ) ˆˆˆ( , ,..., ) t R t t t d t R t t t d t H R t H t H t H d x f x x x x f x x x x f x x x + - -+ + + -+ + +- +- +-+ = = = ! ! 1 , 1 1 1 2 , 2 1 1 , 1 1
ˆ( , ,..., ) ˆ( , ,..., ) ˆ( , ,..., )

t D t t t d t D t t t d t H D H t t t d x f x x x x f x x x x f x x x + - -+ + - -+ + - -+ = = = ! !
where is the direct prediction model tuned to perform the prediction at time .

In the direct strategy, each prediction model is trained and dedicated to a certain horizon, so the error accumulation can be avoided. However, training different prediction models will greatly increase the prediction complexity and time consumption, and, like the recursive strategy, the direct strategy does not take into account the dependencies among time-series observations.

MIMO prediction strategy

Unlike the recursive and direct approaches, the MIMO approach is a multiple output strategy, in which the output of the prediction model is a vector of future values predicted by using only one model [START_REF] Ben Taieb | Long-term prediction of time series by combining direct and MIMO strategies[END_REF]:

(

) 5
where fMIMO is the multiple output prediction model. In this sense, the objective function during the model training is to simultaneously minimize the prediction errors on different horizons. By so doing, the MIMO strategy can preserve the temporal stochastic dependencies of sequential data, addressing the limitation of the recursive and direct approaches. On the other hand, the computational cost of the MIMO approach is less than that of the direct approach because it requires only one model to be trained.

3 Long short-term memory (LSTM)

Network architecture

LSTM is a variant of RNNs developed for dealing with the long-term dependency problem, which is also known as "vanishing gradients" or "exploding gradients" problem [START_REF] Hochreiter | Long Short-Term Memory[END_REF]. In general, an LSTM consists of a chain of repeating network modules, in which each module contains four interacting layers, as illustrated in Fig.

1 [START_REF] Olah | Understanding LSTM Networks[END_REF].

, D h f ˆt h x + , [1, ] t h h H + Î { } 1 2 1 1
ˆˆ, ,..., ( , ,..., )

t t t H M I M O t t t d x x x f x x x + + + - -+ = Fig. 1.
The schematic of a repeating network module in a LSTM network.

The key element of a LSTM network is the cell state C, which is depicted as the horizontal line running through the top of the diagram in Fig. 1. This cell state plays a role as a network memory, where information is added or removed via regulated structures called gates, which can optionally let information through. They are composed of a sigmoid neural network layer and a pointwise multiplication operation. An LSTM consists of three gates, including forget, input and output gates, in order to protect and control the cell state. Details on the LSTM procedure are described as follows.

At time t, an input is fed to the network. The forget gate first decides which information from the previous output is discarded or kept, and then the output of the forget gate is calculated as: ,

where are the input weights and bias of the forget gate, respectively, is a nonlinear function (e.g. sigmoid function) and " . " means matrix multiplication.

The next step is to determine which new information will be stored in the cell state, leading to the two following calculations. First, the input gate decides which states will be updated; then, a tanh layer generates a vector of new values that could be added to the cell state, as follows: ,

, [START_REF] Hu | A particle filtering and kernel smoothing-based approach for new design component prognostics[END_REF] where and are the input weights and bias of the input gate and the cell state layer, respectively. The outputs obtained from the forget gate, input gate and tanh layer are, then, used to

t x 1 t h - [ ] ( ) 1 , t f t t f f W h x b s - × = + ( , ) f f W b s t C ! [ ] ( ) 1 , i t t i t i W h x b s - × = + [ ] ( ) 1 tanh , t C C t t W h x b C - × = + ! ( , ) i i W b ( , ) C C W b
update the new cell state :

.

Finally, the network output is generated by the output gate and a tanh function, as: ,

, [START_REF] Xiao | A novel neural-network approach of analog fault diagnosis based on kernel discriminant analysis and particle swarm optimization[END_REF] where are the input weights and bias of the output gate, respectively.

Dropout regularization

A well-known and critical problem of deep neural networks such as LSTM is overfitting [START_REF] Zaremba | Recurrent Neural Network Regularization[END_REF]. That is, when the training data is limited, complicated mappings between the inputs and outputs that are learned by the network might be the result of sampling noise, which only exist in the training set but not in the real test set. One way to regularize such a network is averaging the outputs of all possible configurations of the parameters, in which each configuration is weighted by its posterior probability given by the training data [START_REF] Sutskever | Dropout : A Simple Way to Prevent Neural Networks from Overfitting[END_REF]. This method can be applied only for simple or small networks. With large neural networks, the computation for training many different network architectures or training one architecture on different data sets is very expensive. Dropout is a technique that addresses this issue [START_REF] Sutskever | Dropout : A Simple Way to Prevent Neural Networks from Overfitting[END_REF].

A motivation for dropout comes from a theory of sexual reproduction [START_REF] Livnat | Sex, mixability, and modularity[END_REF], in which new genes are naturally selected to spread throughout the population based on their competitiveness and less coadaptation which may reduce the chance of a new gene improving the fitness of an individual. Likewise, dropout aims to train each hidden unit in a neural network with a randomly chosen sample of other units. By dropping a unit out, we temporarily remove it from the network along with all its connections during the training process as illustrated in Fig. 2, in order to prevent units from high co-adaptation. By so doing, each hidden unit becomes more robust and is able to create useful features on its own without relying on other units, which helps the network avoid overfitting.

C 1 t t t t t C C C f i - = * + * ! t h [ ] ( ) 1 , o t t o t o W h x b s - × = +
*tanh( ) Consider a neural network with L hidden layers, in which the input and output vectors of layer l (for 𝑙 ∈ {1, … , 𝐿}) are denoted as z (l) and y (l) , respectively. w (l) and b (l) are the weights and biases of layer l, respectively. For a standard neural network, the feed-forward operation can be described as: ,

t t t h o C = ( , ) o o W b (a) (b)
, ( (12)
) 13
where f is the activation function and i denotes the index of hidden unit, as illustrated in Fig. 3(a).

With a dropout network (Fig. 3(b)), a vector of independent Bernoulli random variables r (l) with probability p is used at each hidden layer l to generate the thinned outputs ỹ (l) as follows:

,

, [START_REF] Liu | A Novel Dynamic-Weighted Probabilistic Support Vector Regression-Based Ensemble for Prognostics of Time Series Data[END_REF] where * denotes an element-wise product. The thinned outputs are, then, used as inputs to the next layer of the feed-forward operation: The dropout technique was shown to significantly reduce overfitting and improve the performance of standard neural networks in a wide variety of application domains, including handwriting recognition, speech recognition, image processing, object classification and computational biology [START_REF] Sutskever | Dropout : A Simple Way to Prevent Neural Networks from Overfitting[END_REF]. In this paper, dropout is used in the input and hidden layers of the proposed LSTM model in order to prevent overfitting and quantify the uncertainty information of the multi-step ahead predictions, which is further described in Section 4.3.

, ( 16
) , ( 17
)
z i ( l+1) = w i ( l+1) y l + b i ( l+1) y i ( l+1) = f z i ( l+1) ( ) r j ( l ) ~Bernoulli( p) ! y ( l ) = r ( l ) * y ( l ) z i ( l+1) = w i ( l+1) ! y ( l ) + b i ( l+1) y i ( l+1) = f z i ( l+1) ( ) (

Proposed LSTM-based prognostic framework

In this section, we present a prognostic framework for the multi-step ahead prediction of the timeseries data from SGs, as illustrated in Fig. 4.

Data preprocessing

As mentioned in Section 1, the quality of the observation data for training is one of the most important factors for the successful performance of a prediction model. Due to the errors during sensor measurements or signal transmission, the acquired observations may include missing and anomalous data points, e.g. outliers, which can negatively impact the model performance. In this study, we adopt

{ } 1 2 , ,..., t x x x { } ˆ, [1, ] t h x h H + Î
a raw data preprocessing module focusing on the two following tasks: 1) detecting and removing outliers; 2) imputing missing data points, the number of which may increase after removing outliers.

The first problem is addressed by using the Isolation Forest, an outlier detection technique built on the basis of decision trees [START_REF] Liu | Isolation forest[END_REF]. This technique is based on an assumption that outliers are few, different and susceptible to a mechanism called isolation. In comparison with conventional distance and density measures, isolation has been proved to be a much more effective indicator to detect anomalies. In addition, Isolation Forest also requires a small linear time complexity. Further details on the algorithm of Isolation Forest can be found in [START_REF] Liu | Isolation forest[END_REF]. Once outliers are reduced, a local polynomial regression technique is used to reconstruct missing data samples and reduce noises. The preprocessed data is later used for training and testing the prediction model in the following stages.

Model selection 4.2.1 Prediction horizon

Several research works have been carried out on determining an optimal horizon of prediction in order to provide predictions accurately and timely, and to ensure the usefulness of the prognostic model.

However, to the authors' knowledge, there is no general rule reported for dealing with this issue. We have carried out a review on the horizons selected in recent prediction studies for industrial applications during 2015-2019 [START_REF] Liu | A Novel Dynamic-Weighted Probabilistic Support Vector Regression-Based Ensemble for Prognostics of Time Series Data[END_REF], [START_REF] Li | Multi-step wind speed forecasting using EWT decomposition, LSTM principal computing, RELM subordinate computing and IEWT reconstruction[END_REF], [START_REF] Zhang | A combined model based on CEEMDAN and modified flower pollination algorithm for wind speed forecasting[END_REF]- [START_REF] Wang | Multi-step ahead wind speed forecasting using an improved wavelet neural network combining variational mode decomposition and phase space reconstruction[END_REF] and the result is summarized in Fig. 5. The result shows that multi-step ahead prediction has been less studied than single-step ahead prediction, and that most of the works were carried out with horizons ranging from 3 to 6 steps ahead. To demonstrate the effectiveness of the proposed model, a prediction horizon of 15 steps ahead is investigated in this study.

Hyperparameter optimization

In machine learning, hyperparameters define the model architecture and control the learning process, e.g. the number of hidden layers, activation function type and learning rate. Automatic hyperparameter optimization is playing a fundamental role in the development of machine learning models, including the recent deep neural networks, e.g. LSTM, whose learning performance greatly depends on a number of hyperparameter choices [START_REF] Feurer | Hyperparameter Optimization[END_REF]. Automatic hyperparameter optimization has several important advantages, such as: 1) reduction of the human effort in deploying machine learning, which is important in application because different hyperparameter configurations are needed for different datasets [START_REF] Kohavi | Automatic Parameter Selection by Minimizing Estimated Error[END_REF]; 2) improvement of the performance of machine learning models, by choosing the most appropriate (according to specified objectives) hyperparameters values for the target application at hand [START_REF] Snoek | Practical Bayesian optimization of machine learning algorithms[END_REF], [START_REF] Melis | On the state of the art of evaluation in neural language models[END_REF]; 3) increase of the reproducibility of results, as automatic hyperparameter optimization is clearly more reproducible than manual tuning by human and allows fair comparisons between different models by giving them the same level of tuning for the specific application [START_REF] Bergstra | Making a science of model search: Hyperparameter optimization in hundreds of dimensions for vision architectures[END_REF].

In this study, we implement a variant of Bayesian optimization (BO), called Tree-structured Parzen Estimator (TPE) [START_REF] Bergstra | Algorithms for Hyper-Parameter Optimization[END_REF], to automatically optimize the hyperparameters of the proposed prediction model.

A common advantage of BO approaches is that they require less function evaluations than other classical optimization approaches, such as grid search or RS. This is because these approaches learn and select the best hyperparameter sets based on their distributions describing the fitness scores in the previous iterations. Thus, the number of samples drawn from the hyperparameter search space is probabilistically guided and reduced, allowing for proper evaluations of the most promising candidates for hyperparameter choices.

Recently, TPE has been put forward to address the limitation of the conventional BO approaches in working with categorical and conditional parameters, and, thus, to improve the hyperparameters selection process [START_REF] Bergstra | Algorithms for Hyper-Parameter Optimization[END_REF]. It has, then, been widely used to tune machine learning models in various applications, such as image processing [START_REF] Bergstra | Making a science of model search: Hyperparameter optimization in hundreds of dimensions for vision architectures[END_REF], [START_REF] Ramachandram | Bayesian optimization on graph-structured search spaces: Optimizing deep multimodal fusion architectures[END_REF]- [START_REF] Chevtchenko | A convolutional neural network with feature fusion for real-time hand posture recognition[END_REF], electricity price forecasting [START_REF] Lago | Forecasting day-ahead electricity prices in Europe: The importance of considering market integration[END_REF], solar irradiance forecasting [START_REF] Lago | Short-term forecasting of solar irradiance without local telemetry: A generalized model using satellite data[END_REF], rail defect prediction [START_REF] Mohammadi | Exploring the impact of foot-by-foot track geometry on the occurrence of rail defects[END_REF], occupational accident prediction [START_REF] Kang | Predicting types of occupational accidents at construction sites in Korea using random forest model[END_REF].

Parzen-window density estimation, which is also known as kernel density estimation, is a nonparametric way to build a probability density function from empirical data. In the TPE algorithm, each sample from the empirical data defines a Gaussian distribution with a mean equal to the hyperparameter value and a specified standard deviation. At the start-up iterations, a random search is employed to initialize the distributions by sampling the response surface )𝜃 (,) , 𝑦 (,) , 𝑖 = 1, … , 𝑁 ,2,3 4, where denotes the hyperparameter set, y is the corresponding value on the response surface, i.e. the validation loss or the fitness value, and Ninit is the number of start-up iterations. Then, the hyperparameter space is divided into two groups, namely good and bad samples, based on their fitness values and a predefined threshold value y* (usually set to 15% [START_REF] Feurer | Hyperparameter Optimization[END_REF]), as follows: [START_REF] Heimes | Recurrent neural networks for remaining useful life estimation[END_REF] where and are the probabilities that the hyperparameter set is in the good and bad groups, respectively. Fig. 6 illustrates an example of the TPE initialization process for the hyperparameter distributions, with y* = 15% and Ninit = 100. The red points are the samples with the lowest fitness values after evaluation, thus being classified into the good group whereas the others form the bad group . In this way, the selection of optimal hyperparameters does not rely on the best observation, but on a set of best observations and their distributions. Then, the more iterations one used for initialization, the better distribution we get at the beginning. An Expected Improvement (EI) is, then, calculated as follows:

(19) q p(θ | y) = Pr good (θ ) if y < y * Pr bad (θ ) if y ≥ y * ⎧ ⎨ ⎪ ⎩ ⎪ Pr good

Pr bad q

Pr good Pr bad

Pr ( ) ( )

Pr ( ) good bad EI q q q = Fig. 6. Samples classification from the TPE initialization process.

At each iteration, the hyperparameter configuration that maximizes the EI is chosen. Fig. 7 shows the flowchart of the TPE optimization procedure.

Multi-step ahead prediction

In the testing stage, the MIMO prediction strategy introduced in Section 2.2.3, is used to predict the future values. As mentioned in Section 4.2.1, the prediction horizon h is set to 15-step ahead in this study, as shown in Fig. 8. To further assess the prediction performance, we adopt a Monte Carlo (MC) dropout technique [START_REF] Gal | Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning[END_REF] in order to capture the uncertainty information of the multi-step ahead predictions of the proposed model. It is important to note that the standard LSTM network is not capable to quantify the prediction uncertainty itself. In the MC dropout technique, a dropout probability is applied to all the weight layers in the network, which represents the network weights drawn from a Bernoulli distribution. Thus, the prediction uncertainty can be quantified by running several forward passes through the network. In this study, we perform NMC=100 stochastic forward passes, in which network units of each layer are randomly dropped out, and obtain the mean and confidence interval of the predictions.

* q Initial configuration {! (i) ,y (i) ,

Experimental study

SG data

In this paper, the prediction performance of the proposed model is evaluated on the SG data of French NPPs. SGs in pressurized water reactors (PWRs) are heat exchangers which use the heat from the primary reactor coolant to produce steam in the secondary side and, thus, drive the turbine generators. In addition, the SGs act as a safety barrier between the radioactive primary side and the non-radioactive secondary side. Due to their critical role in NPPs, any degradation mechanism in SGs should be monitored and prevented at the early stages of propagation. A widely used method of degradation monitoring is the analysis of the wide range level (WRL) dynamic behavior recorded by control sensors [START_REF] Corredera | Fouling and TSP Blockage of Steam Generators on EDF Fleet : Identified Correlations with Secondary Water Chemistry and planned Remedies[END_REF], [START_REF] Girard | Physical and Statistical Models for Steam Generator Clogging Diagnosis[END_REF].

WRL is one of the condition monitoring variables measured from the NPP SGs. It is estimated from the difference between the pressure measured at two difference heights, i.e. the dome and the bottom of the downcomer, as illustrated in Fig. 9 (label 18) [START_REF] Girard | Physical and Statistical Models for Steam Generator Clogging Diagnosis[END_REF]. Due to its nature, WRL is very sensitive to

x t-d+1 x t-d+2 x t {f,[!]} " # t+1 " # t+2 " # t+15 … …

Trained prediction model

the temperature, the flow rate of the feed-water and the circulation ratio of the SG. Usually, WRL variations are monitored during slow transients and during manual control at low power load [START_REF] Girard | Physical and Statistical Models for Steam Generator Clogging Diagnosis[END_REF].

Among critical SG degradation mechanisms, clogging is a phenomenon where the flow holes of the tube support plates are partially or completely blocked by deposits, leading to the reduction of the circulation flow rate in the SGs [START_REF] Girard | Physical and Statistical Models for Steam Generator Clogging Diagnosis[END_REF]. Clogging in SG is a slow process which may take several years.

In [START_REF] Luca | Development of Unsupervised and Semi-Supervised Clustering-Based Methods for Degradation Assessment of Nuclear Power Plant Steam Generators[END_REF], it has been shown that the WRL of a SG is closely related to the clogging degradation. Thus, the predictions of WRL can be converted to the clogging degradation state.

Data preprocessing

Before being used for the model development, the raw SG data are preprocessed by using the Isolation Forest and local regression approaches described in Section 4.1. Fig. 11 shows the results of applying the Isolation Forest for reducing outliers in the data of SG 1 of plant No. 2. In Fig. 11

Results and discussion

After the preprocessing stage, each SG data series is divided into a training set and a testing set.

The data for the first 11 years, from July 1992 to December 2002, which include a total of 1230 samples at a 3-day interval, are selected to train the proposed prediction model and the next 5-year data with 510 samples are employed to test the model performance.

Before constructing the proposed model, we employ the false nearest neighbor (FNN) algorithm [START_REF] Kennel | Determining embedding dimension for phase-space[END_REF] to determine the appropriate embedding dimension d of the data series. The main idea of the FNN algorithm is to find the minimum dimension where the distances between the nearest neighbors in the time series do not significantly change in the next higher dimensional embedding. Fig. 13 shows the result of applying FNN to the data of SG 1 of plant No. 1. A threshold for identifying the minimum embedding dimension is set to 0. In this Figure, the minimum embedding dimension value is found at 12. We summarize the optimal embedding dimensions identified for all the SGs data series in Table 1.

, [START_REF] Chen | Gated recurrent unit based recurrent neural network for remaining useful life prediction of nonlinear deterioration process[END_REF] , [START_REF] Zio | Failure and reliability predictions by infinite impulse response locally recurrent neural networks[END_REF] where N is the number of testing observations, x and 𝑥 6 are the observed and predicted values, respectively. ( )

2 1 1 N i i i RMSE x x N = = - å 1 1 100% N i i i i x x MAPE N x = - = ǻ 1 2 1 1 1 N i i N i j i j x x MASE N x x N = = ae ö ç ÷ - ç ÷ = ç ÷ - ç ÷ - è ø å å

Automatic hyperparameter optimization

The proposed prediction model is constructed with one LSTM layer with 64 neurons. Four major hyperparameters of the model are to be tuned, including dropout rate, activation function type, optimizer type and learning rate. The details of the hyperparameter search space are shown in Table 2. In Table 3, we show the performance comparison between TPE and RS, in terms of their obtained best training loss for all SGs. The results obviously show that the optimal configurations found by TPE generally outperform the best ones found by RS in the considered case studies. Thus, the optimal hyperparameter configurations found by TPE are used for prediction in the next stage.

Dropout regularization

In this section, a comparison is carried out between the proposed prediction model and a model with the same architecture but trained without dropout. The other hyperparameters are kept identical between the two models, as described in Section 6.1. The probability of the used dropout is automatically optimized by TPE. We employ all the six SG datasets to comprehensively evaluate

Performance evaluation

The WRL measurements of the six SGs are used for validating the developed prediction model for multi-step ahead prediction. It is important to remind that the prediction horizon used in this study is The prediction results obtained by the proposed model are, then, evaluated with respect to the four benchmark models, i.e. SVR-REC, SVR-MIMO, MLP-REC and MLP-MIMO, in terms of prediction accuracy. For a fair comparison, the hyperparameters of the compared models are optimized by using TPE with 30 trials. The details of the hyperparameter search spaces of the compared models are shown in Table 4.

Table 4. Hyperparameters of the benchmark models.

The comparative results of the proposed model and the four benchmark models for multi-step ahead predictions are shown in Fig. 18. Table 5 summarizes the prediction results in terms of the three accuracy indicators for different SG data. As can be seen in Fig. 18 and Nvidia Tesla K40m graphic cards (with 12 GB of GRAM). It is important to note that SG data used in this paper were recorded at an interval of 3 days. After being trained, the proposed model can be used to perform a 15-step ahead prediction, which is equivalent to 45 operating days ahead of the SGs. Due to this reason, the proposed prediction framework can be applied for a real-time time series prediction of the considered application.

The authors have tested the proposed framework on data from several nuclear power plants, with satisfactory results. Unfortunately, for industrial confidentiality, the data cannot be disclosed and shared.

Conclusion and Future Work

This paper presents an original multi-step ahead prediction framework for PHM applications. The framework integrates three consecutive steps: (1) data preprocessing, (2) adaptive model building and

(3) multi-step ahead prediction. Initially, the problems of abnormal outliers and missing data samples are addressed by employing two preprocessing techniques: Isolation Forest and local regression. Then, a LSTM RNN is constructed for making predictions over a long-term horizon, in which the network hyperparameters are automatically optimized by a TPE algorithm. A dropout regularization and a crossvalidation techniques are applied to address the overfitting problem during the training phase. Finally, the performance of the proposed model is evaluated for multi-step ahead predictions with a MIMO prediction strategy employed. A MC dropout is adopted to quantify the prediction uncertainty.

The proposed multi-step ahead prediction framework can be used for the predictions of time series of NPP operating parameters. A case study concerning the real WRL measurements of SGs which were acquired from different NPPs in France over a period of 16 years is carried out for validating the proposed framework. The experimental results show that the developed prediction framework is able to adaptively estimate the optimal setting for its architecture and capture the underlying long-term dependencies inherent in the given data, for achieving accurate predictions over a long horizon, up to 45 days ahead, outperforming conventional prediction approaches.

However, for the application of NPP SGs used in this study, sufficient information and data for performing a multivariate prediction are not provided, e.g. the information of the interdependency between measured variables and degradations (or failures), the interdependency within the variables, and the maintenance reports of the NPP SGs. Future research will be performed to develop a multivariate time series prediction model and integrate the proposed framework within a RUL estimation task for PHM and predictive maintenance.

  weight and bias of the LSTM cell state, respectively weight and bias of the LSTM forget gate, respectively weight and bias of the LSTM input gate, respectively weight and bias of the LSTM output gate, respectively a network layer in the repeating network module of a LSTM

)( 3 )

 3 Automatic hyperparameter optimization based on the Tree-structured Parzen Estimator (TPE) algorithm is performed. The obtained results are compared to the conventional random search (RS) algorithm with respect to different data scenarios. Dropout regularization and Monte Carlo (MC) techniques are integrated to assess the prediction uncertainty of the proposed model. (4) A case study using the data of SGs in French NPPs measured during the period 1992-2007 is carried out to evaluate the performance of the proposed LSTM-based framework for long-term prediction. To the authors' knowledge, this is the first study using LSTM for the long-term prediction and used on NPP SGs. Other prediction models are considered for comparison purposes.

  t

Fig. 2 .

 2 Fig. 2. An example of a dropout network model [64]: (a) A fully connected 2-hidden layers network; (b) The network obtained by applying dropout. Crossed units are excluded.

Fig. 3 .

 3 Fig. 3. Comparison of the basic operations of a standard and dropout network [64].

Fig. 4 .Algorithm 1 . 6 .

 416 Fig. 4. The flowchart of the proposed multi-step ahead prediction framework for SGs.

Fig. 5 .

 5 Fig. 5. Prediction horizons of recent studies.

Fig. 7 .

 7 Fig. 7. Flowchart of the TPE optimization procedure.

Fig. 8 .

 8 Fig. 8. Multi-step ahead prediction procedure.

Fig. 9 . 2 Fig. 10 .

 9210 Fig. 9. The front-cut schematic of a 51B-model SG [108].

Fig. 11 . 2 Fig. 12 .

 11212 Fig. 11. Applying the Isolation Forest to the data of SG 1 of plant No. 2.

Fig. 13 .

 13 Fig. 13. FNN result for SG 1 of plant No. 1.

For a fair

  comparison, the TPE and RS algorithms are evaluated by using the same model configurations and hyperparameter search space. The number of optimization trials is selected as 30 for the two algorithms. In addition, a k-fold cross-validation (k = 3 in this study) is adopted to prevent overfitting during training the model. The mean square error (MSE) is used as the objective function for model selection. In other words, at each optimization trial, the hyperparameter configuration with the lowest average prediction error evaluated by cross-validation is chosen. To achieve the training convergence, the number of training epochs is set to 100.

Fig. 14 Fig. 14 .Fig. 15 .

 141415 Fig.14shows the comparison of the TPE and RS hyperparameter searches over 30 trials for SG 1

Fig. 16 .

 16 Fig. 16. Training and test errors for the network architecture trained without and with dropout.

2 Fig. 17 .

 217 Fig. 17. Multi-step ahead prediction results by the proposed model for all SGs.

2 Fig. 18 .

 218 Fig. 18. Multi-step ahead predictions using different models for all SGs.

Table 1 .

 1 Minimum embedding dimensions for all SGs.In this study, we carry out three comparisons to evaluate the performance of the proposed prognostic model. The first comparison is conducted to analyze the viability of TPE in tuning the proposed model during the training stage. As a standard optimization approach, RS is considered for benchmarking purposes. Another comparison is, then, carried out to specifically validate the efficacy of dropout in the proposed prediction framework. In the third comparison, four hybrid prediction models, including single-output support vector regression using recursive strategy (SVR-REC), multioutput support vector regression using MIMO strategy (SVR-MIMO), single-output multilayer perceptron neural network using recursive strategy (MLP-REC) and multi-output multilayer perceptron neural network using MIMO strategy (MLP-MIMO), are employed as the benchmark models for comparison with the proposed model in multi-step ahead predictions. In this performance evaluation,

	three prediction accuracy metrics are considered, including root mean square error (RMSE), mean
	absolute percentage error (MAPE) and mean absolute scaled error (MASE). Their definitions are given
	as follows:

,

Table 2 .

 2 Hyperparameters of the proposed prediction model.

Table 3 .

 3 The best training loss obtained by TPE and RS in hyperparameter tuning for all SGs.

	Plant		No. 1			No. 2
	SG	1	2	3	1	2	3
	Random search	0.0487	0.0479	0.0307	0.0358	0.0321	0.0343
	TPE	0.0440	0.0370	0.0319	0.0350	0.0314	0.0270

Table 5

 5

	Model	Hyperparameter	Value set or Range
	SVR	Kernel function	{Linear, RBF, Poly, Sigmoid}
	(including SVR-REC
	and SVR-MIMO)	Degree (of the polynomial kernel function)	[2, 4]
		Regularization parameter (C)	[0.01, 100]
		Kernel coefficient (gamma)	[0.01, 10]
	MLP	Hidden layer size	[1, 5]
	(including MLP-REC
	and MLP-MIMO)	Activation function	{Logistic, Tanh, ReLU}
		Optimizer	{LBFGS, SGD, Adam}
		Learning rate	{Constant, Invscaling, Adaptive}
		Regularization parameter (alpha)	[0.0001, 0.01]

(values in bold), the proposed prediction model outperforms the four other benchmark models and achieves higher accuracy for all SGs. The results indicate the accurate and efficient learning of the proposed prediction model for the long-term dependencies of the SG data. (a) SG 1 of plant No. 1 (b) SG 2 of plant No. 1

Table 5 .

 5 Comparison of the prediction performance in multi-step ahead predictions for all SGs.The average computational time of training the proposed prediction model is 3.2 hours, on a GPGPU node comprising 2 Intel Xeon CPU E5-2695 (24 cores at 2.40 Hz with 32 GB of RAM) and 2

				SG 1			SG 2			SG 3
		Method
			RMSE	MAPE	MASE	RMSE	MAPE	MASE	RMSE	MAPE	MASE
		SVR-REC	0.0382	2.0775	15.2484	0.0333	2.1970	10.2221	0.0508	3.3173	12.1521
		SVR-MIMO	0.0283	1.6511	12.2085	0.0331	2.3878	10.8281	0.0640	5.1793	19.2863
	Plant No. 1	MLP-REC	0.0597	2.7824	20.8682	0.0656	3.3349	15.3398	0.0577	3.3597	13.4886
		MLP-MIMO	0.0339	1.7074	11.8887	0.1888	15.1554	62.2885	0.1867	15.2453	52.7662
		Proposed model	0.0212	1.0950	8.6166	0.0239	1.6973	5.8214	0.0426	2.7230	4.0846
		SVR-REC	0.0572	3.6462	12.8555	0.0906	4.7909	7.0354	0.0242	1.5005	6.1132
		SVR-MIMO	0.0401	3.1774	11.8732	0.0849	4.7575	7.4570	0.0247	1.7842	7.5984
	Plant No. 2	MLP-REC	0.0751	3.0309	11.2367	0.0862	4.7819	7.4403	0.0734	3.4535	14.2511
		MLP-MIMO	0.0607	4.8530	17.4741	0.0888	5.4251	7.5992	0.0499	4.1168	17.2689
		Proposed model	0.0281	2.0117	8.6455	0.0791	4.4033	9.3923	0.0206	1.3992	7.9604

reconstruction using a geometrical construction," Phys. Rev. A, 1992.

Acknowledgement

The authors would like to thank the PRISME department of Électricité de France (EDF) R&D for providing the data used in this paper.