P Baraldi

G Gola

E Zio

D Roverso

M Hoffmann

A randomized model ensemble approach for reconstructing signals from faulty sensors

Keywords: Sensor monitoring, Signal reconstruction, Ensemble, Diversity, Random feature selection, Bagging

On-line sensor monitoring aims at detecting anomalies in sensors and reconstructing their correct signals during operation. The techniques used for signal reconstruction are commonly based on auto-associative regression models. In full scale implementations however, the number of sensors to be monitored is often too large to be handled effectively by a single reconstruction model. In this paper we propose to tackle the problem by resorting to a pool (ensemble) of reconstruction models, each one handling an individual group of signals. This approach involves two main technical steps: firstly, a procedure for constructing signal groups, and secondly a procedure for combining the outputs of the reconstruction models associated to the groups. For the signal grouping step, a wrapper optimization search is proposed to identify the optimal number of groups in the ensemble and the size of the groups. For the model output aggregation step, a simple arithmetic average is adopted. Ensemble accuracy and robustness is achieved by promoting diversity between the signal groups through the use of the Random Feature Selection Ensemble (RFSE) technique in combination with the Bootstrapping AGGregatING (BAGGING) technique for training data selection. The individual reconstruction models are based on Principal Components Analysis (PCA). The proposed approach has been applied to a real case study concerning 215 signals monitored at a Finnish nuclear pressurized water reactor. The results obtained have been compared with those achieved by an equivalent ensemble of models based on a grouping directly optimized by a Multi-Objective Genetic Algorithm (MOGA).

Introduction

Sensors contribute to the safe and efficient operation of modern plants by conveying information on the plant state to the automated controls and the operators. To avoid misleading information which may lead to unsafe and/or inefficient states of operation, it is important to detect sensor malfunctions and possibly reconstruct the incorrect signals. This requires monitoring the sensor performance and has the potential benefits of reducing unnecessary sensor maintenance and increasing confidence in the recorded values of the monitored parameters, with important consequences on system operation, production and accident management [START_REF] Hoffmann | On-line Monitoring for Calibration Reduction[END_REF][START_REF] Hoffmann | Signal Grouping Algorithm for an Improved On-line Calibration Monitoring System[END_REF].

The problem of validating the signals recorded by sensors can be tackled by means of empirical models based on fuzzy logic [START_REF] Heger | Fuzzy Associative Memories for Instrument Fault Detection[END_REF][START_REF] Holbert | Fuzzy Logic for Power Plant Signal Validation[END_REF] and neural networks [START_REF] Wang | A Neural Network Realization of Linear Least-Square Estimate for Sensor Validation[END_REF][START_REF] Holbert | Neural Networks for Signal Validation in Nuclear Power Plants[END_REF]. In particular, auto-associative models have been applied to the validation of nuclear signals [START_REF] Holbert | An Integrated Signal Validation for Nuclear Power Plants[END_REF][START_REF] Fantoni | On-line Monitoring of Instrument Channel Performance in Nuclear Power Plant using PEANO[END_REF][START_REF] Fantoni | Multiple-Failure Signal Validation in Nuclear Power Plants using Artificial Neural Networks[END_REF]. Nevertheless, the single-model approaches typically used for signal validation can only handle a limited number of signals whereas practical applications often deal with a very large number of signals [START_REF] Hoffmann | On-line Monitoring for Calibration Reduction[END_REF][START_REF] Hoffmann | Signal Grouping Algorithm for an Improved On-line Calibration Monitoring System[END_REF].

In this work, a procedure is proposed for the reconstruction of signals coming from faulty sensors among a large set. The procedure is tailored for realistic applications where the number of measured signals is too large to be handled effectively by a single reconstruction model [START_REF] Hoffmann | Signal Grouping Algorithm for an Improved On-line Calibration Monitoring System[END_REF][START_REF] Roverso | Solutions for plant-wide on-line calibration monitoring[END_REF][START_REF] Zio | Genetic Algoritms for Grouping of Signals for System Monitoring and Diagnostics[END_REF][START_REF] Baraldi | Genetic algorithms for signal grouping in sensor validation: a comparison of the filter and wrapper approaches[END_REF]. The approach is based on the subdivision of the set of signals into overlapping groups, the development of a reconstruction model for each group of signals and the combination of the outcomes of the models within an ensemble approach [START_REF] Perrone | When Networks Disagree: Ensemble Methods for Hybrid Neural Networks[END_REF][START_REF] Krogh | Neural network ensembles, cross-validation and active learning[END_REF][START_REF] Sharkey | On combining artificial neural nets[END_REF][START_REF] Gola | Signal Grouping for Sensor Validation: a Multi-Objective Genetic Algorithm Approach[END_REF][START_REF] Baraldi | Reconstruction of faulty signals by an ensemble of principal component analysis models optimized by a multi-objective genetic algorithm[END_REF][START_REF] Gola | Reconstructing signals for sensor validation by a GAoptimized ensemble of PCA models[END_REF][START_REF] Rooney | Dynamic Integration of Regression Models[END_REF][START_REF] Ratsch | Sparse Regression Ensembles in Infinite and Finite Hypothesis Spaces[END_REF][START_REF] Brown | Managing Diversity in Regression Ensembles[END_REF][START_REF] Yu | Cocktail Ensemble for Regression[END_REF] (Figure 1).

An additional advantage of adopting ensembles of diverse models is an increased robustness of the ensemble-aggregated output [START_REF] Baraldi | Reconstruction of faulty signals by an ensemble of principal component analysis models optimized by a multi-objective genetic algorithm[END_REF][START_REF] Brown | Managing Diversity in Regression Ensembles[END_REF][START_REF] Yu | Cocktail Ensemble for Regression[END_REF][START_REF] Breiman | Bagging predictors[END_REF][START_REF] Polikar | Ensemble based systems in decision making[END_REF][START_REF] Tsymbal | Ensemble feature selection with dynamic integration of classifiers[END_REF][START_REF] Tsymbal | Diversity in search strategy for ensemble feature selection[END_REF]. Indeed, the conjecture is that, when performing the ensemble signal reconstruction, if the signal predictions obtained by the individual models are diverse (e.g. the reconstruction errors are different in magnitude and sign), their opportune aggregation will provide a more accurate and robust signal reconstruction [START_REF] Breiman | Bagging predictors[END_REF][START_REF] Polikar | Ensemble based systems in decision making[END_REF][START_REF] Tsymbal | Diversity in search strategy for ensemble feature selection[END_REF]. Theoretical studies have investigated the concept of diversity amongst the models of an ensemble [20-23, 25, 26] and the ways of appropriately aggregating the outcomes of the diverse models [START_REF] Rooney | Dynamic Integration of Regression Models[END_REF][START_REF] Yu | Cocktail Ensemble for Regression[END_REF][START_REF] Tsymbal | Ensemble feature selection with dynamic integration of classifiers[END_REF]. This work intends to contribute a practical application of theoretical concepts of empirical ensemble modeling to a largescale problem of monitoring and reconstructing a large amount of signals recorded by sensors at a nuclear power plant.

Figure 1. The multi-group ensemble approach to signal reconstruction Two issues are central to the ensemble approach: (1) the construction of the signal groups and (2) the combination of the outcomes of the individual models developed on the basis of the groups.

In practice, the aspects most relevant to the grouping of signals are:

(i) the groups size: groups made of a reasonably small number of signals are both more accurate and easier to handle [START_REF] Roverso | Solutions for plant-wide on-line calibration monitoring[END_REF][START_REF] Zio | Genetic Algoritms for Grouping of Signals for System Monitoring and Diagnostics[END_REF][START_REF] Baraldi | Genetic algorithms for signal grouping in sensor validation: a comparison of the filter and wrapper approaches[END_REF][START_REF] Gola | Signal Grouping for Sensor Validation: a Multi-Objective Genetic Algorithm Approach[END_REF][START_REF] Baraldi | Reconstruction of faulty signals by an ensemble of principal component analysis models optimized by a multi-objective genetic algorithm[END_REF][START_REF] Gola | Reconstructing signals for sensor validation by a GAoptimized ensemble of PCA models[END_REF];

(ii) the ensemble size: limiting the number of groups in the ensemble helps reducing the computational cost;

(iii) the diversity of the groups and of the associated models: ensembles based on diverse models are generally more reliable and robust [START_REF] Baraldi | Reconstruction of faulty signals by an ensemble of principal component analysis models optimized by a multi-objective genetic algorithm[END_REF][START_REF] Rooney | Dynamic Integration of Regression Models[END_REF][START_REF] Brown | Managing Diversity in Regression Ensembles[END_REF][START_REF] Breiman | Bagging predictors[END_REF][START_REF] Polikar | Ensemble based systems in decision making[END_REF][START_REF] Tsymbal | Ensemble feature selection with dynamic integration of classifiers[END_REF][START_REF] Tsymbal | Diversity in search strategy for ensemble feature selection[END_REF].

In this work, the groups and ensemble sizes are derived by means of a wrapper optimization. To promote group diversity, signals groups are randomly generated resorting to the Random Feature Selection Ensemble (RFSE) technique [START_REF] Bryll | Attribute bagging: improving accuracy of classifiers ensembles by using random feature subsets[END_REF]. The groups thereby created are then used to develop a corresponding number of signal reconstruction Principal Component Analysis (PCA) [START_REF] Jolliffe | Principal Component Analysis[END_REF][START_REF] Diamantaras | Principal component neural networks: theory and applications[END_REF][START_REF] Scholkopf | Kernel principal component analysis[END_REF][START_REF] Moore | Principal component analysis in linear systems: Controllability, observability, and model reduction[END_REF] models. The models are then trained on different data sets randomly generated using the Bootstrapping AGGregatING (BAGGING) approach [START_REF] Breiman | Bagging predictors[END_REF][START_REF] Polikar | Ensemble based systems in decision making[END_REF][START_REF] Bryll | Attribute bagging: improving accuracy of classifiers ensembles by using random feature subsets[END_REF] in order to inject further diversity in the ensemble of models. The methods for both signal grouping and reconstruction modelling have been chosen simple, consistently with the idea that simplicity is an added value in nuclear safety, where possible. Other more sophisticated methods to generate different training data sets exist, e.g. AdaBoost [START_REF] Polikar | Ensemble based systems in decision making[END_REF], which seem worth consideration in future research for nuclear applications.

Regarding the integration of the outcomes of the individual models, many techniques can be adopted ranging from the simple arithmetic average to weighted average [START_REF] Baraldi | Reconstruction of faulty signals by an ensemble of principal component analysis models optimized by a multi-objective genetic algorithm[END_REF], local fusion [START_REF] Bonissone | Fast Meta-Models for Local Fusion of Multiple Predictive Models[END_REF] and dynamic integration techniques [START_REF] Rooney | Dynamic Integration of Regression Models[END_REF][START_REF] Ratsch | Sparse Regression Ensembles in Infinite and Finite Hypothesis Spaces[END_REF], with increasing computational costs. In practice, there is no single combination rule that is universally recognized better than the others and it is still not clear that more sophisticated and complicated aggregation techniques are actually beneficial to the ensemble performance; then, for the same reason stated above and in absence of any other prior information, the simple arithmetic average is used as a valid choice for combining the models predictions [START_REF] Polikar | Ensemble based systems in decision making[END_REF][START_REF] Kuncheva | Classifier Ensembles for Changing Environment[END_REF][START_REF] Kittler | On Combining Classifiers[END_REF].

The paper is organized as follows. Section 2 presents the problem of signal grouping in general terms. In Section 3, the wrapper randomized approach for generating diverse groups of signals is described in details. An application is illustrated in Section 4, with regards to a real case study concerning the reconstruction of a data set of 215 signals measured at a Finnish nuclear Pressurized Water Reactor (PWR) located in Loviisa. Two more case studies have been considered for verification: the first concerns 84 signals measured at a Swedish nuclear Boiling Water Reactor (BWR) situated in Oskarshamn, the second the reconstruction of 920 simulated signals of the Swedish Forsmark-3 Boiling Water Reactor (BWR). For the Loviisa case study, a comparison is made with a Multi-Objective Genetic Algorithm (MOGA) optimization procedure [START_REF] Holland | Adaptation in natural and artificial systems: an introductory analysis with application to biology[END_REF][START_REF] Goldberg | Genetic algorithms in search, optimization, and machine learning[END_REF][START_REF] Chambers | Practical handbook of genetic algorithms: applications[END_REF][START_REF] Sawaragy | Theory of multiobjective optimization[END_REF] for ensemble group signals selection developed in [START_REF] Baraldi | Reconstruction of faulty signals by an ensemble of principal component analysis models optimized by a multi-objective genetic algorithm[END_REF][START_REF] Gola | Reconstructing signals for sensor validation by a GAoptimized ensemble of PCA models[END_REF]. A discussion on the advantages and limitations of the proposed procedure is offered in the last Section. Finally, in an attempt to make the paper selfconsistent an Appendix reports a brief synthesis of the basic concepts of Principal Components Analysis for signal reconstruction.

Grouping signals for diversity and optimal ensemble performance

Given a set of available

n m k  signals, K k ,..., 2 , 1 
, with given required characteristics.

The selection of the signals to insert in each group should be driven by both the individual properties of the groups and the global properties of the ensemble. Properties individually related to a group are, for example, the mutual information content of the signals in the group and the reconstruction performances of the associated model [START_REF] Zio | Genetic Algoritms for Grouping of Signals for System Monitoring and Diagnostics[END_REF][START_REF] Baraldi | Genetic algorithms for signal grouping in sensor validation: a comparison of the filter and wrapper approaches[END_REF][START_REF] Gola | Signal Grouping for Sensor Validation: a Multi-Objective Genetic Algorithm Approach[END_REF]; global properties of the groups ensemble are, for example, the diversity among the groups and a good redundancy of the signals in the ensemble, i.e., an adequate number of diverse groups containing a same signal [START_REF] Hoffmann | Signal Grouping Algorithm for an Improved On-line Calibration Monitoring System[END_REF][START_REF] Roverso | Solutions for plant-wide on-line calibration monitoring[END_REF][START_REF] Baraldi | Reconstruction of faulty signals by an ensemble of principal component analysis models optimized by a multi-objective genetic algorithm[END_REF][START_REF] Gola | Reconstructing signals for sensor validation by a GAoptimized ensemble of PCA models[END_REF].

Heuristic methods such as Multi-Objective Genetic Algorithms (MOGAs) [START_REF] Sawaragy | Theory of multiobjective optimization[END_REF] have proved effective in scanning the large search space of possible groups 1 to optimize their individual properties, e.g. by maximizing the correlation of the signals in the groups [START_REF] Zio | Genetic Algoritms for Grouping of Signals for System Monitoring and Diagnostics[END_REF][START_REF] Gola | Signal Grouping for Sensor Validation: a Multi-Objective Genetic Algorithm Approach[END_REF][START_REF] Baraldi | Reconstruction of faulty signals by an ensemble of principal component analysis models optimized by a multi-objective genetic algorithm[END_REF][START_REF] Gola | Reconstructing signals for sensor validation by a GAoptimized ensemble of PCA models[END_REF] and minimizing their reconstruction errors [START_REF] Baraldi | Genetic algorithms for signal grouping in sensor validation: a comparison of the filter and wrapper approaches[END_REF]. MOGA approaches have however shown some limitations in guaranteeing the mentioned global ensemble properties, e.g. diversity among the groups, adequate signal inclusion and redundancy, at the basis of the optimality and robustness of the performance of the ensemble [START_REF] Gola | Reconstructing signals for sensor validation by a GAoptimized ensemble of PCA models[END_REF]. Furthermore, the high computational cost required to run a MOGA search renders the method unfeasible for large-scale applications involving thousands of signals.

In this work, the RFSE technique is exploited to ensure good global ensemble properties of group diversity, signal coverage, and redundancy. The group size parameters, i.e. the number of signals in the groups and the number of groups in the ensemble, are determined by means of a wrapper search aimed at maximizing the ensemble performance in terms of minimum reconstruction error. PCA signal reconstruction models are trained on data sets constructed by BAGGING to inject further diversity in the models themselves.

The randomized wrapper approach to signal grouping

As previously stated, high diversity in the ensemble of models is beneficial to its performance of signal reconstruction. To this aim, in this work diversity is imposed onto the PCA models by randomizing the features of the groups upon which they are built with the RFSE technique [START_REF] Bryll | Attribute bagging: improving accuracy of classifiers ensembles by using random feature subsets[END_REF] and the data upon which they are trained with the BAGGING technique [START_REF] Breiman | Bagging predictors[END_REF][START_REF] Polikar | Ensemble based systems in decision making[END_REF][START_REF] Bryll | Attribute bagging: improving accuracy of classifiers ensembles by using random feature subsets[END_REF]. Optimization of the group size m and ensemble size K , is also carried out to improve the performance of the ensemble.

Injecting and verifying (input) diversity in the signal groups by RFSE

The RFSE technique consists in randomly sampling, with replacement, from the n available signals K subsets k F , 1, 2,..., kK 

, each constituted by m signals. This guarantees high signal diversity in the groups upon which the PCA models are built and provides a much faster construction of the signal groups compared to the optimality-driven searches, e.g. MOGA-based [START_REF] Baraldi | Reconstruction of faulty signals by an ensemble of principal component analysis models optimized by a multi-objective genetic algorithm[END_REF][START_REF] Gola | Reconstructing signals for sensor validation by a GAoptimized ensemble of PCA models[END_REF]). Indeed, RFSE is a completely random technique in which no optimization of the composition of the individual groups is sought, i.e. no relevance is given, for example, to the correlation between the signals in the groups (as in the filter MOGA search presented in [START_REF] Zio | Genetic Algoritms for Grouping of Signals for System Monitoring and Diagnostics[END_REF][START_REF] Gola | Signal Grouping for Sensor Validation: a Multi-Objective Genetic Algorithm Approach[END_REF][START_REF] Baraldi | Reconstruction of faulty signals by an ensemble of principal component analysis models optimized by a multi-objective genetic algorithm[END_REF][START_REF] Gola | Reconstructing signals for sensor validation by a GAoptimized ensemble of PCA models[END_REF]) or to their capability of efficiently reconstructing one another (as in the wrapper MOGA search presented in [START_REF] Baraldi | Genetic algorithms for signal grouping in sensor validation: a comparison of the filter and wrapper approaches[END_REF]). The coverage of all the signals is not a priori guaranteed by the random nature of the RFSE approach itself; nevertheless, since the probability that a signal does not appear in any group is

K nm n    
, a reasonable choice of the values of the ensemble parameters m and K can in practice guarantee coverage of all the signals in the ensemble with adequate redundancy, as shown in Section 4.

As previously mentioned, an optimal signal grouping structure should ensure on one side a good signal redundancy in the groups (i.e., for any signal there is a suitable number of groups containing it) but, on the other side, a diverse composition of the groups in terms of the signals contained. In other words, the groups must partially overlap (in order to have each signal included in more than one group), while being sufficiently diverse among each other.

An empirical measure is here proposed to verify the diversity injected by the RFSE in the resulting ensemble grouping structure in terms of the diversity in the signal composition of the different groups (the so-called input diversity

   n m m n m n 1 )! ( ! ! . ) 6 12 exp( 1 1 2 1 2 1 , ,    k k com k k in div  (1) where   2 1 2 1 2 1 , max , , k k k k com k k com m m n   is the normalized fraction of signals ( 2 1 ,k k com n
) in common between the two groups. The measure takes the form of a reversed sigmoid function on the compact support

  1 , 0 2 1 ,  k k com



, as shown in Figure 2. It is such that, high pairwise input diversity values are assigned to those pairs of groups whose fraction of common signals is relatively low (e.g. lower than 30%), whereas it penalizes group pairs with too many signals in common (e.g. more than 50%). ) if the two groups contain the exact same signals and The ensemble input diversity in  is, then, simply computed as the average of the signals diversities:

   n i i in in d n 1 1  (3)

Injecting diversity in the training data sets through BAGGING

Further diversity can be injected in the ensemble of models by training them on different data sets. In this work, this is achieved by the BAGGING technique which has proved successful in many applications [START_REF] Breiman | Bagging predictors[END_REF][START_REF] Bryll | Attribute bagging: improving accuracy of classifiers ensembles by using random feature subsets[END_REF].

First of all, the data set X of N The total amount of diversity injected in the ensemble, hereby called output diversity, is the result of the combination of the RFSE and BAGGING randomizations.

Verifying (output) diversity in the ensemble of models

To verify the amount of output diversity effectively injected in the ensemble, a measure is here proposed, which is directly related to the signal reconstruction performances obtained by the models of the ensemble. Let us consider a generic signal i included in i K groups, 1,2,..., in  . For a given test pattern t , different from those used for the training of the i K models, each group k provides the individual reconstruction ˆ() k i ft of the signal ( ), 1, 2,..., , 1, 2,...,

i i tst f t k K t N 
. For an accurate and robust reconstruction of the test values, the groups' reconstructions, or, analogously, their associated reconstruction errors, must be diverse. In this sense, let

) ( ) ( ) ( t f t f t i k i k i    , i K k ,..., 2 , 1  , tst N t ,..., 2 , 1 
, be the reconstruction error on the t -th test pattern of signal i by group k . If the errors ( ), 1, 2,...,

k ii t k K  
have all the same magnitude and sign, then their aggregation (i.e., the ensemble reconstruction of the t -th point of signal i ) will be affected by a bias and will not be accurate; on the other hand, if errors are diverse in magnitude and sign (i.e. if the groups provide diverse reconstructions), their combination provides a more accurate estimation of the test pattern of the signal.

In practice, this means requiring errors to have different signs and magnitudes distributed around zero, so that, from a mathematical point of view, this favourable situation for the generic signal i could be described in terms of a quasi-normal Gaussian distribution, with mean equal to zero and standard deviation  , of the i K groups' reconstruction errors of each t -th test value, i.e.   12 ( ), ( ),..., ( )

i K kk i i i t t t    ~) , 0 (  N .
On this basis, we define a procedure to measure the output diversity of the i K groups in reconstructing the generic point t of signal i . First, the empirical cumulative distribution function (cdf)

) ( t K i F of the groups' reconstruction errors ) (t k i  , i K k ,..., 2 , 1 
, is computed:

    i k i i K k t i t K I K F 1 ) ( 1 ) (    (4)
where

   ) (t k i I is the indicator function equal to 1 if    ) (t
) i i t k k t K i G i kK D F t F t    ,   1 , 0  t D
, between the two cdf's is computed (Figure 4) 2 . A small distance t D corresponds to a good distribution of the reconstruction errors, whereas a large distance reveals the presence of a bias in the reconstruction of the t -th signal's value. The pointwise output diversity between the i K groups in reconstructing the t -th value of the i -th signal is simply computed as

t t i D div   1
. The output diversity i out d for signal i is taken as the average of the pointwise diversities:

   tst N t t i tst i out div N d 1 1
(5)

Finally, the ensemble output diversity is the average of the signals output diversities:

   n i i out out d n 1 1  (6)

Determining the group size and number

In this work, a wrapper optimization of the ensemble performance is carried out in order to determine the group size opt m (i.e. the optimal number of signals to have in each group, equal for all groups in the ensemble) and the ensemble size opt K (i.e. the number of groups in the ensemble, each one constituted by opt m signals).

The optimal group size opt m is a problem-dependent parameter [START_REF] Bryll | Attribute bagging: improving accuracy of classifiers ensembles by using random feature subsets[END_REF], which is chosen within a pre-defined range of group sizes, i.e.

 

max min , m m m opt 

. A crude, exhaustive wrapper approach is here adopted. Operatively, for each candidate group size

  max min , m m m
, an ensemble of K groups of m signals is generated by randomly sampling the signals of the groups by RFSE, K corresponding PCA-based reconstruction models are trained on different data sets obtained by BAGGING and the ensemble reconstruction performance is computed. Since the scope of the work is to provide a robust ensemble of groups, the ensemble performance is computed also on a set of test signals artificially disturbed (see Section 4.1 for details on the disturbance procedure). The optimal group size opt m is the one corresponding to the ensemble of groups providing the best reconstruction performances on both undisturbed and disturbed signals. Notice that searching for an optimal group size equal for all groups is necessary in practice to reduce the number of parameters to be optimized.

A relevant issue for robust signal reconstruction is that each signal i be included in a suitable number of groups 1  i K , i.e. to have appropriate redundancy of signals representation in the ensemble. The average signal redundancy in the groups of the ensemble can be computed as:

   n i i K n R 1 1 (7)
     n i i K k k K m 1 1 (8)
In the case in which each group includes the same number of signals (i.e., m m k  , k  ) and each signal is included in the same number of groups (i.e., R K i  , i  ), Eq. ( 8) becomes:

R n m K    (9)
For a given (optimal) group size opt m , the (optimal) number of groups opt K in the ensemble can be determined by fixing the desired (optimal) signal redundancy opt R , viz.: Figure 5 sketches the scheme of the overall algorithm for devising and verifying the ensemble of models for signal reconstruction.

Figure 5. Sketch of the ensemble algorithm for signal reconstruction

Combination of the outcomes of the models in the ensemble

The combination of the outcomes of the ensemble of models is performed by simple averaging [START_REF] Perrone | When Networks Disagree: Ensemble Methods for Hybrid Neural Networks[END_REF][START_REF] Yu | Cocktail Ensemble for Regression[END_REF][START_REF] Breiman | Bagging predictors[END_REF]. This way of combining the models outputs can be seen as an extension to a regression problem of the Simple Voting (SV) technique adopted in classification problems to combine the class assignments of the single classifiers constituting the ensemble [START_REF] Krogh | Neural network ensembles, cross-validation and active learning[END_REF][START_REF] Jolliffe | Principal Component Analysis[END_REF].

When the

   i K k k i i E i t f K t f 1 ) ( 1 ) ( ˆ (11)
To evaluate the ensemble performance, first the absolute signal reconstruction error is computed as3 :

1 ( ) ( ) TST N EE i i i TST t f t f t N     (12)
Then, an ensemble performance index is computed as the average of the absolute signal reconstruction errors:

1 1 n EE i i n     (13)

Application

The ensemble approach has been applied to a real case study concerning the validation and reconstruction of 215 groups is generated by randomly sampling m signals in each group. The ensemble performance is then computed following the scheme illustrated in Figure 5 and the ensemble error E  is computed as Eq. [START_REF] Perrone | When Networks Disagree: Ensemble Methods for Hybrid Neural Networks[END_REF].

A robust ensemble of models is expected to be able to reconstruct the signals from faulty sensors, e.g. due to random noises, offsets or drifts. In other words, when a faulty signal is sent in input to the PCA models which include that signal, their ensemble should still provide in output a good estimate of the true value of the signal, by exploiting the information coming from the non-faulty signals in the groups of the ensemble. shows the best compromise between accuracy and robustness and is chosen as the ensemble groups' optimal size. Concerning the ensemble size, in general, high signal redundancy and, correspondingly, large numbers of groups guarantee better accuracy and robustness. Nevertheless, using a large number of groups in the ensemble leads to a considerable increase in the computational cost of training and testing the models, with only a slight improvement of the ensemble performances.

Indeed, the error scales are small (especially for the undisturbed signals,) and the error bars (which have not been inserted for visual clarity) are superposed. Nevertheless, from an operative point of view, a decision on how many groups to use and which bagging fraction to adopt must be made. For this reason, 25

 opt R
is the value chosen for the desired signal redundancy and, correspondingly, 142

 opt K
is the fixed ensemble size. This choice allows having good ensemble performances with a low computational cost and furthermore it allows having an honest comparison with the genetic algorithm-based optimized grouping [START_REF] Gola | Reconstructing signals for sensor validation by a GAoptimized ensemble of PCA models[END_REF] which is presented in Section 4.

BAGGING

This Section intends to illustrate specific situations in which Bagging is and is not effective in terms of the output diversity. Indeed, the BAGGING procedure is intended to enhance the output diversity of the ensemble. According to the adopted output diversity measure for the generic signal i , the models built using the signals of its i K groups are diverse if they make different errors in reconstructing the signal. More precisely, it is conjectured that the most beneficial diversity is achieved if the reconstruction errors for each signal's test point are distributed as a quasi-normal Gaussian function (Section 3.3).

In order to show the effects of BAGGING, two test patterns of signal 19 included in 25 [START_REF] Rooney | Dynamic Integration of Regression Models[END_REF]  K groups are here analyzed in details: the first (#196) is a signal measurement taken during normal plant operation, whereas the other (#731) corresponds to a measurement during plant shut-down. In Figure 8, the effects of BAGGING on the errors distribution of the 25 groups including signal 19 on the two test patterns are reported in both cases of undisturbed (top graphs of Figure 8) and disturbed (bottom graphs of Figure 8) patterns. When the pattern is disturbed, performing BAGGING allows obtaining more diverse groups predictions (i.e. the groups' errors distribution is more similar to that of a quasi-normal Gaussian) and thus a more accurate and robust reconstruction of the pattern's correct value. On the other hand, if the pattern is undisturbed, performing BAGGING can degrade the ensemble performance by reducing the groups diversity in the reconstruction of that pattern, i.e. by negatively affecting the distribution of the groups' errors and rendering it quite dissimilar to the Gaussian cumulative distribution, as for pattern 196 (top-left graph).

Furthermore, looking at the results for pattern 731 (corresponding to the shut-down transient) one sees that the ensemble trained without BAGGING does not become aware of the disturbance affecting it (bottom-right graph) and the errors distribution remains almost unaltered, as evident from the comparison between the top-right and bottom-right graphs; instead, BAGGING allows recognizing the occurrence of the disturbance and the distribution of the models' errors is adjusted to become more similar to a quasi-Gaussian distribution. In this view, the superior performance in the reconstruction of pattern 731 without disturbance obtained with BAGGING (top-right) can be explained by considering the transient part of the signal as disturbances, on whose reconstruction BAGGING has certainly positive effects.

In this sense, bagging has proved effective in improving the performances on disturbed data more than on the undisturbed ones.

Comparison with the MOGA approach to signal grouping

In order to evaluate the advantages and limitations of the proposed ensemble approach, a comparison to the signal grouping based on Multi-Objective Genetic Algorithm (MOGA) optimization investigated in [START_REF] Gola | Reconstructing signals for sensor validation by a GAoptimized ensemble of PCA models[END_REF] is proposed in this Section. The comparison is made on an ensemble of 150 groups.

In the MOGA approach, the search for the set of 150 MOGA K  optimal groups of signals has been iteratively carried on by optimizing (i.e. maximizing) the correlation between the groups' signals and the input diversity between the groups (the interested reader may refer to [START_REF] Gola | Reconstructing signals for sensor validation by a GAoptimized ensemble of PCA models[END_REF] for a fully detailed explanation). The comparable ensembles size allows for an honest comparison of the two techniques.

Concerning the input diversity in  (Eq. 3), i.e. the average degree of group overlapping in terms of signals, the RFSE shows a considerable improvement with respect to the MOGA approach ( 8884 . 0

 RFSE in  , whereas 6750 . 0  MOGA in



). This proves that, according to the adopted measure, randomly selecting the signals in the groups corresponds to a high injected diversity. As illustrated in Figure 2 Enhancing input diversity with the RFSE approach improves the robustness of the ensemble on disturbed signals ( * E  , Figure 9) and corresponds to an increase of the output diversity on disturbed signals ( * out  , Figure 10). Nevertheless, the output diversity on undisturbed signals ( out  , Figure 10) of the RFSE groups without BAGGING is lower and correspondingly the performance is worse ( E  , Figure 9). This performance degradation can be explained by the fact that, while in the MOGA the signals in the groups are selected based on their mutual correlation (a characteristic conjectured to be related to their capability of regressing one another [START_REF] Zio | Genetic Algoritms for Grouping of Signals for System Monitoring and Diagnostics[END_REF][START_REF] Baraldi | Genetic algorithms for signal grouping in sensor validation: a comparison of the filter and wrapper approaches[END_REF][START_REF] Gola | Signal Grouping for Sensor Validation: a Multi-Objective Genetic Algorithm Approach[END_REF][START_REF] Baraldi | Reconstruction of faulty signals by an ensemble of principal component analysis models optimized by a multi-objective genetic algorithm[END_REF][START_REF] Gola | Reconstructing signals for sensor validation by a GAoptimized ensemble of PCA models[END_REF]), the signals in the RFSE groups are randomly selected regardless of their mutual correlation or any other criterion for optimizing their reconstruction capabilities.

The robustness of both RFSE and MOGA ensembles ( * E  , Figure 9) is considerably improved when BAGGING is applied as indicated by the increase of the corresponding ensemble output diversities on disturbed signals ( * out  , Figure 10). Nevertheless, when performing BAGGING on the RFSE ensemble, the increase of the output diversity on undisturbed signals ( out  , Figure 10) is not followed by an increase on the reconstruction performances on undisturbed data ( E  , Figure 9). In general, performing BAGGING degrades the model accuracy in reconstructing undisturbed signals.

Finally, notice that BAGGING slightly contributes to RFSE at increasing the undisturbed error E  , whereas the effect on E  due to RFSE is more evident when comparing RFSE with MOGA. This means that too much diversity randomly introduced in the groups does not improve the capability of reconstructing undisturbed signals, which instead comes from having highly correlated signals (as in the MOGA groups); on the other hand, a robust signal reconstruction in case of disturbances is due to high models diversity (as in the RFSE and MOGA approaches with BAGGING).

The Figure 11 shows the results of the reconstruction of signal 214 (electrical power) obtained by the RFSE and the MOGA approaches both trained with BAGGING procedure ( 0.02

B  

). When the signal is undisturbed the highly correlated signals of the (less diverse) MOGA groups allow for a more accurate reconstruction, whereas as soon as the drift begins the more diverse RFSE groups are capable of providing a more robust signal reconstruction.

Finally, in order to give an overview of the advantages and limitations of the two approaches, Table 2 summarizes the characteristics of the signal grouping structure and the ensemble reconstruction performances obtained by the RFSE and MOGA approaches, respectively.

Signal grouping

Optimized parameters

Group size, signal redundancy and number of groups (wrapper approach).

Intra-group signals correlation and intergroup signals diversity (filter approach).

Group size, m

Controlled by wrapper optimization accounting for accuracy and robustness.

The same [START_REF] Sawaragy | Theory of multiobjective optimization[END_REF] for all groups.

Neither controlled, nor optimized. Groups of different sizes ranging from 8 to 147 signals.

Signal redundancy, R

Controlled a priori and optimized based on accuracy and robustness. Maintained on average after selecting randomly the groups' signals and evenly distributed among the signals.

Not imposed a priori, but indirectly optimized by maximizing signals diversity in the groups. Unevenly distributed among the signals.

Number of groups (ensemble size), K

Set as a function of the optimized group size and signal redundancy.

Set a priori according to the dimensions of the solution search space. Strongly influencing the computational cost of the optimization.

Computational cost

Medium-low; dependent on the type of regression model adopted to run the wrapper optimization of the ensemble dimension parameters, opt m and opt K .

High; strongly related to the ensemble dimension parameters, i.e. the number of signals and groups involved in the search.

Ensemble signal validation

Ensemble input diversity

Very high; due to signals random selection.

Medium-high; optimized by enhancing diversity between the groups, but indirectly reduced by maximizing signals correlation.

Accuracy (reconstruction of undisturbed signals)

Low; due to lack of optimization of groups individual properties (e.g. mutual correlation).

High; due to high signals correlation in the groups achieved during the optimization.

Ensemble output diversity

High; mostly due to model diversity injected with the bagging technique, but also increased by the combined injection of signals diversity (with random selection).

Medium; due only to the BAGGING procedure.

Robustness (reconstruction of disturbed signals)

High; due to the presence of diverse information both in the groups' signals (RFSE) and in the training data sets (BAGGING).

Medium; due only to the BAGGING procedure.

Table 2. Summary of the characteristics and performances of the RFSE and MOGA approaches

Validation of the approach on two additional case studies

The overall RFSE approach has been verified on two different case studies for validation. Table 3 briefly reports the characteristics of the data sets and the grouping parameters set with the same wrapper optimization hereby proposed. Notice that in order to limit the number of groups in the ensemble which greatly affects the computational cost opt R for the Forsmark-3 case study has been set very low. Notice also that the value of the optimal B  is higher for these case studies. This is due to the fact that data contain a smaller number of spurious measurements (especially the simulated data). The results of the ensemble signal reconstruction in terms of disturbed and undisturbed errors with and without BAGGING are also reported in Table 3; the performances resemble those achieved in the previous case study. Table 3. Characteristics, parameters and ensemble performances of the Oskarshamn and Forsmark-3 case studies.

Oskarshamn case study

Finally, the results of the Oskarshamn case study without Bagging can be compared with those achieved by using the Multi-Objective Genetic Algorithm approach to signal grouping presented in [START_REF] Baraldi | Reconstruction of faulty signals by an ensemble of principal component analysis models optimized by a multi-objective genetic algorithm[END_REF]. The MOGA has been aimed at searching for 100 optimal groups of signals by maximizing the correlation between the groups' signals and the input diversity between the groups. The reconstruction errors on undisturbed ( E  ) and disturbed ( * E  ) signals obtained by the approach are 9.39 × 10 -4 and 16.7 × 10 -3 , respectively. Even though the comparison has been carried on a noncomparable ensemble size (51 and 100 groups for the RFSE and MOGA, respectively), still the results confirm the higher robustness of the RFSE approach in reconstructing disturbed signals.

In general, even though the methodology proposed in this work is obviously problem-dependent with respect to the optimal grouping parameters which must be discerned for the specific data set, these results show that it is applicable to any problem involving a large number of signals to be validated and reconstructed for which training a single model entails convergence problems (such as for the iterative training of Auto-Associative Neural Networks) and shows significantly less robustness.

Conclusions

In this work, a novel approach to the reconstruction of the signal values from faulty sensors has been proposed, based on an ensemble of PCA models. The set of sensor signals, too large to be handled effectively with one single reconstruction model, is subdivided into small, overlapping groups and a PCA-based reconstruction model is developed for each group. The outcomes of the models are then combined by simple averaging to obtain the ensemble signal reconstruction.

The fundamental characteristic of diversity in the ensemble models has been generated by randomly sampling the signal features in the groups (by the RFSE procedure) and a fraction of the patterns for training the corresponding reconstruction models (by the BAGGING technique). The dimension parameters of the ensemble (i.e., the size of the groups and the number of groups in the ensemble) have been optimized by a direct wrapper approach.

The overall modelling scheme has been applied to the reconstruction of signals collected at a Finnish nuclear pressurized water reactor. Two additional case studies have been analyzed for validation purposes. The performances of the proposed approach have been compared with those of an equivalent ensemble obtained by means of a multi-objective genetic algorithm aimed at maximizing the intra-group signal correlation and the inter-group signal diversity. Considering the difficulties of calibrating the ensemble dimension parameters and of maintaining the group signals diversity in the MOGA optimization, and its not negligible computational cost, the RFSE approach has proved more efficient for large-scale applications involving hundreds of signals.

The approach proposed has shown considerable robustness in reconstructing signals when in presence of disturbances and drifts. At the same time, it allows to control and optimize the ensemble dimension (number of groups and of signals in the groups), which is fundamental for practical applications. On the other hand, not considering in any way the optimization of the models reconstruction capability is paid by a (small) loss of accuracy in reconstructing undisturbed signals. Finally, the performances of the overall RFSE approach have been verified on two different case studies providing similar satisfactory results and, in general, the methodology here presented has proved to be fast and robust and is currently object of further improvements.

The significance of the findings is to be framed within the practical problem tackled and goes beyond the numerical values of the results obtained. The ensemble approach proposed provides a feasible way for handling the validation and reconstruction of a set of a large number of signals,which cannot be handled by a single model. The method has proved its effectiveness, its low computational cost (an essential aspect for the application of any on-line sensor monitoring system) and its applicability to different signal sets. Finally, even though the numerical values show sometimes little improvements with respect to the reconstruction error, these must be seen in terms of enhanced robustness and plant production and safety when the validated and reconstructed signals are effectively used during operation. In this respect, notice the criticality to have a robust and accurate reconstruction of those signals that are used by the plant control systems, for which a small error in the reconstruction can lead to wrong control actions, possibly with significant production losses and safety threats: this last aspect is part of currently ongoing research.

Appendix. Principal Component Analysis (PCA) for signal validation and reconstruction

In this Appendix, we briefly sketch the procedural steps of Principal Component Analysis (PCA) as presented in [START_REF] Marseguerra | Lecture notes on Principal Components Analysis (PCA)[END_REF].

The central idea of PCA is to reduce the dimensionality of a data set consisting of a large number of interrelated variables, while retaining as much as possible of the variation present in the data set. This is achieved by transforming to a new set of variables, the principal components (PCs), which are uncorrelated and ordered so that the first few retain the most of the variation present in all of the original variables [START_REF] Jolliffe | Principal Component Analysis[END_REF]. , where m I is the unit matrix of order m .

In
  m N f f f f f f f f f N Nm N N m m , ... ... ... ... ...
                               f f f X (A1) so that ti ti f X  , m i N t ,...,
In the orthonormal basis, let ti u be the component of The data set has now two representations: when intended in the original basis, the t -th pattern is the vector t f with components ti f ; when intended in the orthonormal P basis, the same t -th pattern is a vector t u with component ti u along i p . In this view, once the orthonormal P basis has been fixed, Eq. (A5) provides t u as a function of t f . To get the inverse relation, we right multiply by T P and we obtain the data set X in the original basis, viz.:

t f along T i p , so that     m i ij ti tj p u f 1 (A3) and   T t m i T m T T tm t t T i ti t u u u u P u p p p p f                       1 2 1 2 1 ... ... ( A4
   m m m N T T N N , , ... ... 2 1 2 1                                   P U P u u u f f f X (A7)
In this view, Eqs. (A6) and (A7) represent the transformation laws of the observation patterns X between the original reference system and the orthonormal basis. Notice that up to this point the equations are exact and the data values are transformed in both senses without any loss of information.  m in which m  has been divided. In terms of the above submatrices, Eq. (A7) can be rewritten as:

  T m m T m m T T m T m                                 P U X P U P U P P U U X ~ (A8)
where we define X as:

  m N u u u u u u u u u N T T T N N N T , ... ... ... ... ... ... ~2 1 2 1 2 1 2 22 21 1 12 11                                                    f f f P P P P U X       (A9)
The t -th row of X , namely t f ~, is the orthonormal projection of t f onto   and then it may be expressed by as a linear combination of the vectors of the  P basis, viz.:  m which is postulated to be independent of t , viz.,

  T i i ti T T T t t t t u u u u P P P P f                         1 2 1 2 1 ... ... ~ (A10) and      1 ~i ij ti tj p u f (A11) is the j -th component, m j ,..., 2
      m i T i i t appx t b 1 ~ p f f (A12)
The best residual vector is that one which, on the average, minimizes the absolute value of the square error between the real   t f and approximated  

appx t f ~ data patterns, i.e.:     N t appx t t E 1 2 2 1 f f (A13)
By combining Eqs. (A4), (A10) and (A12), the error between the two vectors can be written as: where the vector f is the arithmetic average of the observation vectors, i.e. the average value of the signals. In particular, the i -th component of f is the arithmetic average of the i -th column of X . Then, from Eq. (A12), the expression for the PCA approximation of the data pattern t f is given by:

         m i T i i ti
      m i T i i t appx t 1 ~ p p f f f (A19)
or in matrix form,

T m m appx         P P X X X ~ (A20)
Coming to the problem of model learning, the Principal Components Analysis exploits the information in the mdimensional data set to generate an orthonormal basis P in m  constituted by m eigenvectors. These represent the result of the learning phase of the PCA model.

In this respect, let V be the covariance matrix of the data set. The problem to tackle at this point is how to choose an orthonormal basis P in m  and how to select among the m columns i p of P the  vectors which constitute the basis of   . As demonstrated and explained in details in [28, 29, 39], by substituting Eqs. (A4) and (A18) into Eq. (A16), we can write the minimum error corresponding to the coefficients V represents the covariance matrix of X eventually positive definite (so that its eigenvalues are real and positive) and can be written as,

                    N t T t T t m N N m 1 , , X X X X f f f f V (A22)
In order to find the  vectors which will constitute  P , let us minimize min E with respect to Finally, coming to the problem of signal validation and reconstruction by means of a PCA-based model, in order to simplify the calculations, the time trends of the signals have been previously normalized so that their mean is zero and their standard deviation equals 1. This allows to skip the computation of the residuals, since 0 f  and, according to Eq. (A19), Furthermore, coming to the problem of using the PCA as a signal reconstruction model, for each group k constituted by k m signals, the eigenvectors constituting the orthonormal basis  P , have been obtained by Eq. (A24) from the covariance matrix V of the pairwise signal correlations between the k m signals in the group.

Figure 2 .

 2 Figure 2. Group pairwise input diversity function, Eq. (1)In the special case that two groups have the same number of signals, i.e.,

Figure 3 

 3 Figure 3 illustrates the combination of the RFSE and BAGGING techniques to inject diversity in the ensemble. As a result of the RFSE, K signal-diverse groups are identified, upon which K PCA models are constructed; BAGGING then proceeds to sampling (with replacement) a number B NK  of different training sets k B trn , X , 1, 2,..., kK  , each constituted

  generic k -th model based on the signals of group k , randomly selected by RFSE, is trained with the set of data k B trn , X , randomly sampled by BAGGING, for 1, 2,..., kK  .

Figure 3 .

 3 Figure 3. Combined scheme of the RFSE and BAGGING techniques to inject diversity in the ensemble

GF

  empirical (real) cdf of the reconstruction errors is compared with the (ideal) cdf of a quasi-normal Gaussian distribution ()  of mean zero and given  and the maximum absolute distance 1

Figure 4 .2

 4 Figure 4. Comparison in terms of maximum absolute distance ) ( t D between the empirical ) ( t K i F and Gaussian ) ( G F

  one constituted by opt m signals, are then generated resorting to the RFSE technique and the corresponding PCA-based reconstruction models are BAGGING-trained.

For

  verification purposes, a partially faulty test set has been generated by introducing anomalies in a fraction of the patterns. More precisely, the signal values of a generic test pattern are altered by a random noise (setting them equal to the offset value of the corresponding sensor (97 they are not affected at all. The ensemble of groups is then tested on this test set of disturbed signals and the corresponding performance * E  is computed as Eq.[START_REF] Perrone | When Networks Disagree: Ensemble Methods for Hybrid Neural Networks[END_REF].Actually, to account for the variability of the training and test sets, five ensembles (each constituted by 150  K groups with randomly sampled signals) have been generated for each group size m and for each of the five ensembles the computation of the accuracy and robustness indicators has been 5 times cross-validated. The overall ensemble performance associated to the generic group size m is finally computed as the average over all cross-validations of all different ensembles.

Figure 6

 6 Figure 6 shows the results of the analysis. In general, ensembles built on large groups are more accurate on undisturbed signals (smaller E  values, Figure 6 left), whereas those built using both small   30  m and large   60  m groups have

Figure 6 .

 6 Figure 6. Performances of the ensemble for different group sizes ( m ) on undisturbed signals ( E  , left) and disturbed signals ( * E  , right). Cross-validation statistical errors (one standard deviation above and below the average) are also reported in the Figure.



  Once the optimal group size opt m has been fixed, it is possible to proceed to determine the optimal ensemble size opt K and BAGGING fraction B  . As explained in Section 3.4, the ensemble size is obtained by the empirical relation of Eq. (9), for a given desired signal redundancy opt R . Eight candidate values for opt R have been chosen, based on an expert engineering evaluation; correspondingly, eight candidate ensemble sizes opt K have been identified. These values are reported in

optK

  and actual redundancies R . The results of the joint optimization of opt K and B  are reported in Figure 7 in terms of ensemble accuracy, i.e. reconstruction errors on undisturbed signals (top graph), and robustness, i.e. reconstruction errors on disturbed signals (bottom graph).

Figure 7 .

 7 Figure 7. Results of the joint optimization for determining the optimal ensemble size

4 .

 4 Concerning the BAGGING procedure, in general the results show that injecting diversity by training the models on different data sets improves the ensemble performances. Furthermore, the reconstruction errors decrease (especially for disturbed signals) if the fraction of sampled training patterns B trn N in the bagging training sets becomes smaller. In fact, by so doing, many BAGGING training sets are completely disjoint and therefore highly diverse. The minimum reconstruction error is obtained for 1 . 0  B  in the case of undisturbed signals and 02 . 0  B  in the case of disturbed data. The remarkable decrease of the ensemble error on the disturbed signals when choosing 02 fraction, despite a slight worsening of the ensemble performances on the undisturbed set. Such a low value can be explained by the fact that excluding many training patterns from the signal data set allows removing with high probability those spurious signal measurements (e.g. spikes) that usually compromise the reconstruction of all the signals in the single groups.Finally, if the fraction becomes too small (e.g., ensemble performances considerably worsen due to the lack of data representation in the BAGGING training sets.

Figure 8 .

 8 Figure 8. Effects of BAGGING on the distribution of groups errors in reconstructing test pattern number 196 (left graphs) and 731 (right graphs), when undisturbed (top graphs) and disturbed (bottom graphs).

  robustness of the RFSE and MOGA ensembles has been then specifically tested for comparison on the reconstruction of faulty signals in case of multiple sensor failures. Ten signals have been chosen as objects of the analysis. Approximately, the first half of the signals has been left undisturbed as in the normal operation, while, in order to simulate a sensor failure, a linear drift decreasing the values of the signals up to 25% of their real values has been introduced in the remaining test values. The validation set has been linearly divided into training and test only once, i.e. without crossvalidation.

Figure 9 .Figure 10 .

 910 Figure 9. Comparison of the MOGA and RFSE approaches to signal grouping in terms of the ensemble reconstruction errors on undisturbed ( E  ) and disturbed ( * E  ) signals, with and without performing BAGGING

Figure 11 .

 11 Figure 11. RFSE (dots) and MOGA (stars) ensemble reconstruction of signal 214 (light line) when partly affected by a linear drift (dark line)

Forsmark- 3

 3

2 1 .P

 21 Correspondingly, the matrices P and U are partitioned as follows: constituted by the first  columns of P and U constitute the bases of the two mutually orthogonal subspaces   and  

  best constants are those that minimize the error and are determined by the conditions depend on t , using Eq. (A4) we can write,

.

  the Lagrange multiplier approach[START_REF] Jolliffe | Principal Component Analysis[END_REF]. The purpose is to find those   m vectors which minimize min E subject to the constraint of being orthonormal. The Lagrange function in terms of the submatrix Notice that since the eigenvalues have been supposed simple, the eigenvectors are orthogonal and may be normalized.By substituting (A24) into (A23) we obtain that the required minimum of the Lagrangian is but from Eq. (A25) it appears that the best choice is to select the smallest   m eigenvalues among the m possible eigenvalues in V . To this aim, we rank the m eigenvalues in decreasing order so that m

2 1

 2 The eigenvectors are correspondingly ranked and we choose for the basis  P of   the first  eigenvectors and for the basis amount of information lost by considering appx X ~ instead of X may be quantified for the individual observations by the differences

two sets of signals are completely disjoint; if, instead, the sizes of the two groups differ, then the maximum number of signals in common for the two groups is

  12 ,

															kk com	is 0 (and thus	kk in div		1 1e 		1	) if the
								12 , , kk com MAX n		min	 m m 12 , kk		and thus		2 MAX , com 1 , k k		1	. This well represents the fact
	that even if a group 1 k is completely included in another one 2 k , i.e.	2 k  , their pairwise input diversity is not zero due 1 k
	to the presence in 2 k of some signals not included in the smaller set 1 k .
	To compute input diversity at the level of the ensemble of groups, first the diversity for each signal	i		,..., 2 , 1	n	is
	calculated. Considering the generic signal	i included in	K groups, the signal input diversity i d is taken as the average
														i	in
	of its	i K groups' pairwise diversities	div	1 ,k k in	2	,	k 1	k , 2		,..., 2 , 1	K	i	, 12 kk  , viz.:
		d	i in		12 ,  12 21 11 11 1 ii KK kk in kk ii kk div KK        				(2)

  is large, the individual BAGGING training sets overlap significantly and the probability of not including a training pattern in any of the BAGGING training sets is very small, so that all the training patterns are likely to appear in at least one BAGGING training set and some patterns will appear multiple times in a given set; instead, if the fraction is small, some BAGGING training sets can be completely disjoint and some training patterns might not appear in any BAGGING training set.

						patterns available is partitioned into a training set	X TRN (made of	TRN N	patterns) and a
	test set	X TST (made of	TST N	patterns). The former is used to train the individual models, whereas the latter is used to
	verify the ensemble performance in the reconstruction task.
	In general, BAGGING amounts to generating a number	N of bootstrapped replicates	h X , B ,	h		, 1	,..., 2	N	of the training
							B			trn	B
	set	X	trn	by randomly sampling (with replacement) for each replicate a fraction		B		  1 , 0	of the total number of training
	patterns	N . If the fraction
				trn

B



  Considering that different overlapping groups have some signals in common, or, dually, different signals have some groups in common, the total number of signals (with repetitions) in the ensemble of groups can be expressed either by summing the number of signals k m

	constituting each group	k		, 1	,..., 2	K	(i.e., the groups' sizes) or by summing the number of
	groups i K including each signal	i		,..., 2 , 1	n	(i.e., the signals' redundancies), viz.:

  February 28, 2006 to November 1, 2007 from a corresponding number of sensors and include the measurements related to two plant outages occurred in the periods from June 26 to October 1, 2006 and from September 8 to September 23, 2007, respectively. A set . The PCA models (see Appendix) have been constructed with the code http://lib.stat.cmu.edu/multi/pca, adapted to perform the signal reconstruction task of interest here.

										signals
	measured at a nuclear Pressurized Water Reactor (PWR) located in Loviisa, Finland. A total number	N =12713 of
	215-dimensional patterns is available, 1 f t f t 2 ( ), ( ),..., ( ),..., i f t	f	215	( ) t ,	t		,..., 2 , 1	N	. Data signals have been sampled every
	hour from m X constituted by		6000
			4.1 Determination of	m
										opt
	The optimal size	opt m of the ensemble's groups is sought in the range of values	m		 20 , 40 	. The group dimensions in this
	range are expected to lead to accurate and robust regression models, while being at the same time easy to handle from a
	computational viewpoint. For verification purposes, the search has been further stretched up to group sizes max 70 m  .
	Operatively, for each candidate group size	m 	20,21,...	, an ensemble constituted by	K		150

m N patterns is used for the wrapper determination of the groups' optimal size opt m ; a set RB X constituted by the remaining 6713  RB N patterns is used for the wrapper determination of the ensemble size opt K and the BAGGING fraction B  . Notice that the patterns of the transient related to the first outage are included in m X , while those of the transient of the second outage are in RB X

Table

Table 1 .

 1 1, together with the actual mean signal redundancy (computed as Eq. 7) and standard deviation obtained after the groups have been created with the RFSE procedure. For the BAGGING fractions, seven values have been chosen, namely 1, 0.8, 0.6, 0.4, 0.1, 0.02 and 0.005; the case of no BAGGING procedure (in which all the Pre-defined, desired values for opt R , corresponding ensemble sizes

	K	models are trained using the same
	opt

  simulated with a sampling rate of one hour, under conditions of reactor start up, normal operation and shut down.

The first concerns 84 signals measured at a Swedish nuclear Boiling Water Reactor (BWR) situated in Oskarshamn, the other the reconstruction of 920 simulated signals of the Swedish Forsmark-3 Boiling Water Reactor (BWR). The data available for the measured 84 signals have been sampled every 10 minutes from May 31, 2005 to January 5, 2006 from a corresponding number of sensors, providing a total amount of 30080 N  time samplings. Regarding the 920 signals, the sensors measurements have been

  observations, i.e. the m signal values at the time instant t , viz.:

	this view, let 	f i	t (	),	t		, 1	,..., 2	N	,	i		, 1	,..., 2	 m	be a set of N observations in an m -dimensional space m  . The purpose
	of the PCA is to identify a  -dimensional (			m	) subspace	 			m	in which the most of the data set variation is
	retained and the least information is lost.
	From a mathematical point of view, let	X		  m N,	be the data set matrix whose rows	f	t		  m , , 1	t		, 1	,..., 2	N	, are the
	patterns of the m

  according to an optimality criterion to be defined later and representing an orthonormal basis for the data set:

												,..., 2 , 1		,		2 , 1	, is the i -th component of	f in the original basis.
																					t
	Let	P		(	m ,	m )			m	be a matrix constituted by m orthonormal column vectors	p		(	m , 1 ),	i		, 1	,..., 2	m	built from the
																					i
	data set X  p p p p p p p p p mm m m m m ... 2 1       ... ... 1 12 11     ... 2 22 21       p P	1	p	2	...	p	m	   m m , 	(A2)
	so that	P		p	,	i		, 1	,..., 2	m ,	j		, 1	,..., 2	m	is the	j -th component of	p in the original basis and, for the orthonormality of
				ij		ij														i
	the basis vectors,		p	T	 p				or	P	T		P		I
											i			j				ij			m

  Without loss of generality, assume that the basis vectors are ordered in such a way that the selected  vectors are the first ones in P , i.e.  

	The PCA approximation consists in mapping the observation vectors t f in a subspace	 			m	identified by			m
	vectors chosen according to a criterion explained later among the	p	i , 	, 1	,..., 2	m	.
						i
	p	p	...	p	

, 6

In the application that follows, each signal of the validation set has been previously normalized in the range [0.2, 1], for convenience.