Prof Mohamed Ichchou

Prof Alexandre Saidi

Prof Abdel-Malek Zine

Prof Xiaopin Zhong

Dr Elyes Mrabet

Dr Y U Hang

Dr Yin Xiangnan

Dr L V Ying

Dr Christophe Droz

Prof Abdelkhalak Elhami

Prof Tawfiq Imad

Prof Sergio De Rosa

Ms Ms M Belhadi

Ms M Badalan

S Stamkulov

Dr Ms Chaouki

Dr. CHAI Wenqi Ravi

Dr Lou Mingyang

Dr Luo Lingkun

Dr Huang Xingrong

Dr Yang Yifan

Ms. SUN Cheng

Dr Y I Kaijun

Dr Wang Xiaofang

Dr G Mazzeo

Weizhen You

Keywords: Structural reliability, Uncertainties, Statistical learning General Introduction

During my three-year PhD research, I received help

Résumé

L'étude de la fiabilité structurelle concerne principalement l'évaluation et la prédiction du risque de violation de l'état limite d'une structure d'ingénierie à n'importe quelle étape de sa vie. L'évaluation de la fiabilité permet d'améliorer la conception de la structure et la qualité des produits, ce qui est d'une grande importance pour les entreprises et les consommateurs. Le contrôle des vibrations est une technique pour réduire l'énergie d'une structure vibrante lorsqu'elle est excitée par des forces externes. Cette technique est largement utilisée dans divers systèmes, tels que les bâtiments, les ponts et les véhicules. La prévision de fiabilité aide les entreprises à planifier la production et à mettre en une maintenance préventive.

Pour faire l'évaluation de la fiabilité, un modèle d'analyse théorique est d'abord développé. En raison de facteurs intérieurs et extérieurs complexes, les incertitudes des propriétés structurelles ainsi que celles de l'excitation stochastique ont rendu l'analyse de la fiabilité plus difficile à appliquer. L'analyse de fiabilité traditionnelle est généralement basée sur des connaissances professionnelles ou des fonctions de performance explicites, ce qui était irréaliste pour les systèmes d'aujourd'hui qui sont plus complexes et non linéaires. En fait, l'analyse de fiabilité implique des estimations de la soi-disant probabilité de défaillance conditionnelle (CFP) qui peut être considérée comme un problème de régression prenant les incertitudes structurelles en entrée et les CFP en sortie. Dans cette situation, une attention croissante a été accordée aux théories de l'apprentissage statistique non paramétrique. La prédiction de la CFP peut être réalisée par des modèles d'apprentissage automatique, tels que les arbres de décision, les machines à vecteurs de support, les méthodes d'apprentissage d'ensemble, etc. Cette recherche vise à construire un cadre théorique qui intègre les théories du machine learning dans l'analyse de la fiabilité structurale et à explorer leurs performances sur différentes structures. Ce cadre est d'une efficacité et d'une précision élevées pour les systèmes structuraux qui présentent des incertitudes à la fois en termes de propriétés structurelles et d'excitation.

Mots

General

Reliability describes the ability of a system or component to function under stated conditions for a specified period of time [1]. Benefiting from more than half a century's development, 'reliability' has become one of the key roles that determine the product performance. Nowadays, increasingly competitive markets drive companies to produce more advanced systems to meet various requirements of the customers.

Meanwhile, the complexity of the products increase drastically and the failure mechanisms become more difficult to analyze. We notice that various 'smart' products are becoming even closer to human life, making larger influence on their comfort and safety. These conditions make reliability evaluations more and more important to apply. Structural reliability aims at quantifying the risk of failure of systems due to uncertainties in their design, manufacturing and environmental conditions. Vibration control is a technique to reduce the energy of a vibrating structure when it is excited by external forces. This technology is widely used in various systems, such as buildings, bridges, machine tools and vehicles. Reliability research in structural

systems remains an open problem due to uncertainties and different failure mechanisms. This dissertation presents contributions to structural reliability analysis, modeling and prediction in passive control mode.

Background

Vibration control of structural systems

Depending on the locations, certain civil structured facilities can be subjected to dynamic loads due to gusty wind fronts or strong ground motion associated with earthquake events of different intensity/severity during their life service. At high levels of intensity these naturally occurring dynamic loads may induce permanent structural damage and, in extreme cases, total structural failure/collapse. During the last three decades the incorporation of various devices such as energy dissipation equipment (e.g. viscous dampers, friction dampers, etc.), and tuned-mass dampers have been considered by various researchers and has been applied in practice to passively control the vibratory motion of structures maintaining its amplitude below certain acceptable thresholds ( [2], [3], [4], [5]). Typically, such "non-conventional" means of mitigating the hazard posed to structures due to the action of winds and earthquakes are applied to protect critical civil infrastructure such as high-rise buildings, hospitals, and long-span bridges. Furthermore, the employment of such passive devices is commonly considered to reinforce existing structures to meet the contemporary safety criteria and to retrofit damaged structures in the aftermath of severe seismic events. These practical applications have sustained the important and active research field of passive vibration control for various structures. Admittedly, it is noted that the improved structural performance can also be achieved by using active/semi-active control solutions relying on the integration of sensors, controllers and real-time data processing ( [3], [START_REF] Yoshioka | âsmartâ base isolation strategies employing magnetorheological dampers[END_REF], [START_REF] Osman E Ozbulut | Adaptive control of base-isolated structures against near-field earthquakes using variable friction dampers[END_REF]). However, due to reliability issues and the installation cost of such solutions, the use of active control systems is not as wide spread as the passive control solutions. In the context of passive vibration control, the concept of the dynamic vibration absorber is historically one of the first and most widely used strategies for passive vibration mitigation of dynamically excited mechanical and civil engineering structures and structural components ( [START_REF] Frahm | Device for damping vibrations of bodies[END_REF], [START_REF] Liu | Optimal design of damped dynamic vibration absorber for damped primary systems[END_REF]). It relies on attaching an additional free-to-vibrate mass to the structural system (primary or host structure) whose motion is to be suppressed via certain mechanical devices. These devices are appropriately designed (or "tuned") such that a resonant out-of-phase motion of the attached mass is achieved compared to the primary structure. Arguably, the most commonly used dynamic vibration absorber is the so-called "tuned mass-damper" (TMD). In its simplest form, the TMD considers a linear spring and a viscous damper to link the additional mass to the primary structure. The effectiveness of this classical TMD relies on tuning its stiffness and damping properties such that significant kinetic energy is transferred from the vibrating primary structure to the TMD mass and is absorbed through the viscous damper. Especially during the recent one decade, most of the researchers have been focusing on passive control using tuned mass dampers (TMD) in randomly driven linear structures ( [START_REF] Li | Crowdinduced random vibration of footbridge and vibration control using multiple tuned mass dampers[END_REF], [START_REF] Marano | A comparison between different robust optimum design approaches: application to tuned mass dampers[END_REF], [START_REF] Marano | Constrained reliability-based optimization of linear tuned mass dampers for seismic control[END_REF], [START_REF] Marano | Robust optimum design of tuned mass dampers devices in random vibrations mitigation[END_REF], [START_REF] Morga | Optimization criteria of tmd to reduce vibrations generated by the wind in a slender structure[END_REF], [START_REF] Omer | Optimum vibration absorber (tuned mass damper) design for linear damped systems subjected to random loads[END_REF], [START_REF] Yu | Reliability based robust design optimization for tuned mass damper in passive vibration control of deterministic/uncertain structures[END_REF]). The use of the TMDs in passive linear structural control is increasingly demanded in the field of engineering because of its efficiency and low cost. Indeed, the TMDs, belonging to the general class of vibration dampers, are very economical and efficient devices to mitigate the structural vibrations caused by external dynamic loadings.

Uncertainties in structures

The incorporation of uncertainty into systems analysis is an on-going topic of significance. For realistic systems, uncertainties can be involved in the design stage, in the manufacturing progress and during the service. In the design stage, uncertainties are derived from mathematical modeling process or incomplete knowledge about the system. Uncertainties in the manufacturing progress are reflected by the manufac-turing tolerance, material scatter on account of the limited precision in tools and processes or the lack of advanced technologies. During the service, uncertainties in the excitations (such as seismic loads, waves, temperature changes and any other kind of environmental loads) and boundary conditions as well as human factors are the major concerns. As any structure has a limited age, uncertainty is usually induced from the deterioration of material properties. In this aspect, uncertainties may be imposed upon geometry tolerances, material properties or excitations. These uncertainties can influence the performances of the system during its lifetime. Uncertainty analysis, or in other words, how to mathematically quantify uncertainties, has become an indispensable task that should be carefully accomplished in different stages of the structural system. During the last two decades, engineers and scientists have paid much attention to uncertainties of various types in the system. Various representations and theories for uncertainties exist, each of which is characterized by a distinct mathematical model based on the information on hand. Primarily, uncertainties can be classified as three variants: aleatory uncertainty, epistemic uncertainty and error. The variants as well as their explanations are illustrated in

Uncertainty

Aleatory uncertainty

(Variability, irreducible, random, inherent, or stochastic)

Inherent variation of the system or environment;

Induced by Irreducible variation of property ranging over time or population;

It can be modeled using various probability theories.

Epistemic uncertainty

(Reducible, subjective, state-of-knowledge, model form or simply uncertainty)

Incomplete information

Induced by Some level of ignorance It can be modeled using fuzzy set theory, evidence theory, possibility theory, or convex model, imprecise probability etc.

Error (Numerical uncertainty)

A recognizable deficiency in modeling and simulation

It is not caused by lack of knowledge

It should be identifiable through examination

It could be avoided by an alternative approach with limited validity of the applied numerical methodology As is shown in Figure 1.1, aleatory uncertainty can also be understood as variability, irreducible uncertainty, inherent uncertainty or stochastic uncertainty. The term 'aleatory' is used to specialize the inherent characteristics of the variations related to the physical system or the environment of interest. Aleatory uncertainty is a kind of uncertainty that generally has priori-known statistical properties. Accordingly, the probability theories can be applied. In contrast, epistemic uncertainty is also termed as reducible uncertainty, subjective uncertainty or model form uncertainty.

Epistemic uncertainty is generally induced by incomplete information, some level of ignorance of the system or the environment. The associated uncertainty analysis (UA) method depends mainly on non-probabilistic theories that are commonly known as generalized information theory. It mainly includes possibility theory, fuzzy set theory and evidence theory. As to the last term error, it could be reduced or even avoided by careful examinations or advanced methods.

Failure mechanisms

Failure mechanism is a physical process in which stresses cause damage to the elements comprising the system, ultimately leading to system failure [START_REF] Dasgupta | Material failure mechanisms and damage models[END_REF]. According to [START_REF] Dasgupta | Material failure mechanisms and damage models[END_REF] and [START_REF] Wallace R Blischke | Reliability: modeling, prediction, and optimization[END_REF], the failure mechanisms can be classified into two subgroups: overstress failures (such as large elastic deformation, yield and fracture) and wear-out failures (such as wear, corrosion and fatigue). In the first case, a failure event occurs when the stress to which the structure is subjected exceeds the allowable maximum strength. If the stress is below the threshold value, the stress has no permanent effect on the structure. In the second case, the stress causes damage that generally accumulates irreversibly. The accumulated damage does not reduce when the stress is removed, although sometimes annealing is possible. The cumulative damage does not cause any performance degradation when it is below the endurance limit. Once this limit is reached, the structure fails. As to random vibration, similar failure mechanisms can also be found in [START_REF] Stephen | Random vibration in mechanical systems[END_REF]: (1) Failure may occur at the first time that the random response exceeds a certain level; (2) Failure occurs when the accumulated damages reaches a fixed amount. The estimation of first-passage (or first excursion) probability is usually in conjunction with the former case; while the aging engineering belongs to the latter one. In this work, we concentrate on the over-stress or the first-passage failure mechanism. Simply, when the structure is subject to static loads, the over-stress mechanism is considered; when the structure is under dynamic loads, the first passage problem is regarded.

Structural reliability

Structural reliability is concerned with the rational treatment of uncertainties in structural engineering and with the methods for assessing the safety and serviceability of civil engineering and other structures [START_REF] Thoft-Cristensen | Structural reliability theory and its applications[END_REF]. It is a subject that has grown rapidly during the last decade and has evolved from being a topic for academic research to a set of well-developed or developing methodologies with a wide range of practical applications. Reliability is commonly defined as [START_REF] Thoft-Cristensen | Structural reliability theory and its applications[END_REF] "the probability of a device performing its purpose adequately for the period of time intended under the operating conditions encountered". There are four elements to the definition that must be considered. First, probability refers to the likelihood that a device or structural component will work properly. These terms imply acceptance of some degree of uncertainty. The second element refers to adequate performance. In order to determine whether a component has performed adequately, a standard is needed to define what is meant by adequate performance. The third element is the intended period of time. This is the mission endurance or lifetime of the structure under consideration. The final element of the definition is the operating conditions.

Environmental conditions play a significant role in determining structural reliability.

Simply stated, structural reliability is a yardstick of the capability of a structure to operate without failure when put into service. As implied in the definition, structural failure and, hence, reliability, is influenced by many factors. In its simplest form, the measure of reliability is made by comparing a component's stress to its strength. Failure occurs when the stress exceeds the strength. The larger this gap, the higher the reliability and the heavier the structure. Conversely, the smaller the gap, the lower the reliability, but the lighter the structure. The gap between stress and strength, enforced by the factor-of-safety, generally produces adequate although unmeasured reliability. Generally, the structural reliability changes in different peri-

Basic contents of structural reliability analysis

ods of its life, based on various interior and exterior factors. There are mainly three factors listed as below.

• Static strength A structure's capability to sustain operational loads is commonly assessed by comparing material performance parameters to limit or ultimate loads. Limit loads are generally defined as the maximum load expected during the life of the structure.

• Environmental effects Environmental factors of major importance include a combination of humidity and temperature. Many studies have been conducted

to investigate moisture absorption as well as the reduction of mechanical properties due to temperature and moisture exposure. Some of the approaches used to account for environmental factors is to define exposures that are extremes and selectively evaluate by test the effects on material properties. These extremes are then considered to be invariant during the lifetime of the structure.

Strength values are reduced to coincide with the environmental extremes.

• Fatigue is the gradual deterioration or progressive structural damage of a material that occurs when the material is exposed to repeated loads.

Basic contents of structural reliability analysis

One of the problems in stochastic mechanics is the estimation of the probability density of one or several structural responses. This kind of problem can be termed full probabilistic analysis. A derived, more practical problem is to find a specific probability of exceeding a response threshold that can be considered critical for the serviceability or the safety of the system. This is the basic task of reliability analysis. Notice that, in principle the second problem could be solved after the first, as the probability of failure can eventually be obtained by integration of the probability density function of the observed response. However, most reliability methods developed in the last decades attempt to estimate directly the failure probability or related reliability indices. Actually, the failure probability is calculated from an integration within the failure region that is defined by a particular performance function.

Performance function In the reliability problem, the input random variables are collected in a vector X, whose deterministic counterpart is the vector x. This defines the input space. The perf ormancef unction G(x) (also named limit-state function, LSF) is defined in such a way that G(x) > 0 means that the sample x is in the safe domain S = x : G(x) > 0; G(x) < 0 implies that x is in the failure domain F = x : G(x) < 0. Specially, G(x) = 0 is the boundary between the two domains.

For example, the performance function of a structural component can be defined as

G = R -S (1.1)
where S is a load effect on the component; R is the strength capacity to withstand it.

The safety limit state will be violated whenever G(x) < 0. The probability that this occurs for any single load application is the probability of limit state violation, or simply the probability of failure P f . Generally, P f is represented by the overlap area of the probability distributions of the two variables (see Figure 1.2). Theoretically,

P f = P (G(x)) = G(x)<0
f RS (r, s)drds (1.2) Since this overlap may vary with the time, so P f may be a function of time. More generally, assume that x = [x 1 , x 2 , ..., x n ] is the vector of n-dimension variables. x consists of the fundamental variables that define and characterize the behavior and safety of a structure. Accordingly, the probability of failure is denoted as [START_REF] Martin | A monte carlo method for reliability-based design opitmization[END_REF]]

P f = • • • G(X)≤0 f X (x 1 , x 1 , ..., x n )d x 1 ...d xn (1.3)
where f X (.) denotes the joint probability density function (pdf) of input vector . If the basic variables are all independent, f X (.) is simplified as:

f X (x) = n i=1 f X (x i ) = f X (x 1 ) * f X (x 2 )... * f X (x 3 ) (1.4)
Here f X (x i ) is the marginal probability density function for the basic variable x i .

The region of integrand G(X) ≤ 0 in (1.3) denotes the hyper-space where the limit

Methods for reliability analysis

The methods based on the limit-state function can be grouped into two families named as analytic and synthetic, according to whether the random variable set and its effects are treated with the tools of probability theory or with those of statistics (see Figure 1.3). In the first case we have two kinds of methods based on Taylor-series expansion of the limit state function, which are known as FORM (First-Order Reliability method) and SORM (Second-Order Reliability Method). As is well known, the FORM method requires information on the value of the performance function and its gradient in the vicinity of the so-called design point and the SORM method needs in addition the knowledge of the second-order derivatives. In the second case, sample sets of the basic variables are generated and processed by a numerical solver (generally a finite element code) of the structural model in order to obtain a population of structural responses over which statistical analyses can be conducted. This is called Monte Carlo simulation (MCS). For the reliability assessment, MCS approximates the reliability by the frequency that the generated samples drop into the safe region ( [START_REF] Padmanabhan | A study using monte carlo simulation for failure probability calculation in reliability-based optimization[END_REF], [START_REF] Papadrakakis | Reliability-based structural optimization using neural networks and monte carlo simulation[END_REF], [START_REF] Marek | Probabilistic assessment of structures using monte carlo simulations[END_REF]).The Monte Carlo method is distinguished by its entire generality.

Methods of analysis

Nonetheless, it is a well-known fact that it requires a large computational labor.

With respect to FORM and SORM it requires random samples instead of deterministic ones and its scaling with dimensionality is rather weak. Indeed, together with its generality, this later characteristic is a positive advantage. However, its computational cost remains a problem. This is not only due to the fact that it requires the many calls of the numerical solver for the assessment of any statistical quantity but also that in using it this quantity becomes a random variable. In order to diminish the computing labor of Monte Carlo methods, research efforts oriented to a reduction of the number of samples and the spread of the estimates obtained at each simulation epoch have been exerted in the last years. Sometimes, however, these developments are accompanied by a sacrifice of one or more of the attractive properties of the basic Monte Carlo method. For instance, some of these variance-reduction methods (for example, importance sampling) require an exploration for locating the design point or the estimation of an optimal density function, which is a task greatly affected by the curse of dimensionality. Anyhow, these research efforts evidence the need of reducing the very high cost of MCS especially for the case of large structures. As the complexity increases drastically and the failure mechanisms became more difficult to recognize, new theories were derived by the researchers.

Physical system

(difficult to conduct experiments on) niques for exploring the relationships between independent variables and response variables [START_REF] Raymond H Myers | Response surface methodology: process and product optimization using designed experiments[END_REF]. Its basic idea is to find a surrogate polynomial function of the basic variables that approximates the implicit performance function. The surrogate is then employed for most of the samples needed in the entire MCS. For this purpose, an experimental design is generated for calculating the undetermined coefficients of the surface so as to minimize the error of approximation, particularly in the region around the design point. The surrogate so obtained can also be connected to FORM and SORM methods. The development of RSM ( [START_REF] Wong | An adaptive response surface method for reliability analysis of structures with multiple loading sequences[END_REF], [START_REF] Byeng | A new response surface methodology for reliability-based design optimization[END_REF]) opened a new area of structural reliability research that can be labeled as the investigation on solver surrogate methods. The evolution from the physical models to surrogate models are ( [START_REF] Wiener | The homogeneous chaos[END_REF], [START_REF] Sakamoto | Polynomial chaos decomposition for the simulation of non-gaussian nonstationary stochastic processes[END_REF]) and Kriging model [START_REF] Gaspar | Assessment of the efficiency of kriging surrogate models for structural reliability analysis[END_REF]. Note that taking into account the refinement and sophistication of finite element models, the practical application of MCS could be greatly favored by the availability of such a substitute to the actual deterministic solution, since only some finite element solver calls would be needed for building the surrogate. In recent years, however, several important findings in this direction have been achieved. For example, some serious deficiencies of the RSM that make it unreliable for practical use have been highlighted. Fortunately, the development of Statistical Learning Theory and algorithms, such as Random Forests (RF) [START_REF] Yoshioka | âsmartâ base isolation strategies employing magnetorheological dampers[END_REF],

Artificial Neural Networks (ANN) ( [START_REF] Kutyłowska | Neural network approach for failure rate prediction[END_REF], [START_REF] Jamal | Artificial neural network application of modeling failure rate for boeing 737 tires[END_REF]) and Support Vector Machines (SVM) ( [START_REF] Hong | Predicting engine reliability by support vector machines[END_REF], [START_REF] Li | Support vector machine for structural reliability analysis[END_REF], [START_REF] Guo | Application of least squares support vector machine for regression to reliability analysis[END_REF]), afforded new ways of obtaining the surrogate of the performance function. Attention has been paid by researchers to the fact that structural reliability problems can be treated as a statistical classification task determined by the LSF. For this purpose, the modern pattern recognition methods become useful.

RF is an ensemble learning method that use multiple learning algorithms to obtain better predictive performance than could be obtained from any of the constituent learning algorithms alone. RF is comprised of many individual trees called classification and regression tree (CART). Each tree is built from a random sample set (i.e. bootstrap sample) generated from the training set. A tree learning process is actually a recursive data partitioning process based on the principle of minimal sum of squared errors (SSE). Other representative tree-based methods include Gradient Boosting (GB) [START_REF] Yoshioka | âsmartâ base isolation strategies employing magnetorheological dampers[END_REF] and Extremely Randomized trees (ETs) [START_REF] Geurts | Extremely randomized trees[END_REF]. Stacked Generalization (Stacking) involves training a meta-learner from the predictions of several base learners. First, all of the base learners are trained using the available data, then a meta-learner is trained from the predictions of the base learners. Stacking typically yields performance better than any single trained models [41]. It has been successfully applied on both regression [START_REF] Breiman | Stacked regressions[END_REF] and classification. An ANN consists of some artificial neurons and connections between them. An artificial neuron is a mathematical function that receives one or more inputs and sums them to produce an output. Usually each input is separately weighted, and the sum is passed through a non-linear function known as an activation function. Training of ANN involves adjusting the parameters, or the weights and biases, of the model in order to minimize error. Back propagation is used to make those weigh and bias adjustments relative to the error, and the error itself can be measured in a variety of ways, including by root mean squared error (RMSE).

Similar to ANN, support vector machine (SVM) is an intelligent learning method for pattern recognition. However, the theoretical bases of SVM and ANN are different. ANN is based on empirical risk minimization (ERM) principle, while SVM's foundation is structural risk minimization (SRM) principle. ANN can obtain the minimization risk for the training samples due to the application of the ERM principle, but for the un-trained samples ANN may give unreliable estimation. SRM principle can improve generalization ability of learning machine by minimizing the total of empirical risk and confidence bound. Theoretically, other machine learning tools such as k-nearest neighbors (kNN) are also an alternative. According to the current research state, we believe that the explorations are far from enough, but more and more researchers are trying to introduce these methods into structural reliability domain. This dissertation devotes itself totally to this topic.

Structural reliability analysis process

Generally, the reliability analysis of an engineering structure includes three parts.

Determine the basic variables that construct the structural model; express the probability distributions of these variables; define the performance function for the object structure. Based on these three tasks, the basic procedure of engineering structural reliability analysis is shown in Figure 1.6.

Structural mechanics analysis (Finite element code)

Structural performance function evaluation

Structural reliability analysis

Experimental design (data collection)

Engineering structure (sample space) In chapter 2, the fundamental knowledge of reliability analysis and reliability modeling techniques for structural systems are introduced. In this chapter, we begin with the classical reliability analysis theories that provide the basic knowledge of reliability analysis. Uncertainty analysis is subsequently followed, in which the basic probability theories are introduced and the associated methods are reviewed. They constitute the foundation of this dissertation.

In Chapter 3, the theory of uncertainty quantification is introduced and analyzed in detail.The concept of conditional failure probability is developed. Machine learning models are introduced as surrogates to model conditional failure probabilities.

In Chapter

General

In the chapter of introduction, we have emphasized the necessary and the importance of reliability engineering in the structural system. In this chapter, we begin with fundamental knowledge of reliability analysis, including classic analysis methods as well as modern statistical learning models. Among the structural reliability analysis methods, we introduce the mostly-used ones, such as first/second order reliabil-

General principle of structural reliability analysis

A structural system is defined as a structure that is required to provide specific functionality under well-defined safety constraints. Such constraints need to be taken into account during the system design phase in view of the expected environmental/ operating loads it will be subject to. In the presence of uncertainties in the physical properties of the system (e.g. due to tolerances in the manufacturing), in the environmental loads (e.g. due to exceptional weather conditions), or in the operating conditions (e.g. traffic), it can occur that the structure operates outside of its nominal range. In such cases, the system encounters a failure. Structural reliability analysis deals with the quantitative assessment of the probability of occurrence of such failures (probability of failure), given a model of the uncertainty in the structural, environmental and load parameters. Following the formalism introduced in [START_REF] Sudret | Uncertainty propagation and sensitivity analysis in mechanical models-contributions to structural reliability and stochastic spectral methods[END_REF], this part is intended as a theoretical introduction and of the available tools in structural reliability analysis. All the algorithms presented follow a black-box philosophy, e.g. they rely on the point-by-point evaluation of a computational model, without knowledge about its inner structure. Let us denote by X the vector of basic random variables. When considering models of mechanical systems, these variables usually describe the randomness in the geometry, material properties and loading [START_REF] Sudret | Uncertainty propagation and sensitivity analysis in mechanical models-contributions to structural reliability and stochastic spectral methods[END_REF]. They can also represent model uncertainties. This set also includes the variables used in the discretization of random fields, if any. The model of the system yields a vector of response quantities Y = M (X). In a mechanical context, these response quantities are e.g. displacements, strain or stress components, or quantities computed from the latter.

Limit-state function

A limit state can be defined as a state beyond which a system no longer satisfies some performance measure. Regardless on the choice of the specific criterion, a state beyond the limit state is classified as a failure of the system. Consider a system whose state is represented by a random vector of variables X ∈ D X ⊂ R M . One can define two domains D s ; D f ⊂ D X that correspond to the saf e and f ailure regions of the state space D X , respectively. In other words, the system is failing if the current state x ∈ D f and it is operating safely if x ∈ D s . This classification makes it possible to construct a limit -statef unction g(X) that assumes positive values in the safe domain and negative values in the failure domain:

x ∈ D s ⇔ g(x) > 0 x ∈ D f ⇔ g(x) < 0 (2.1)
The hyper-surface in M dimensions defined by g(x) = 0 is known as the limitstate surface, and it represents the boundary between safe and failure domains. A graphical representation of D s , D f and the corresponding limit-state surface g(x) = 0 is given in Figure 2.1.

Failure Probability

If the random vector of state variables X is described by a joint probability density function (PDF) f X (x), then one can define the failure probability P f as:

P f = P (G(X) < 0) (2.2)
This is the probability that the system is in a failed state given the uncertainties of the state parameters. The failure probability is then calculated as follows:

{ : ( ) 0} f D g   x x { : ( ) 0} s D g   x
x

Failure region

Safe region

( ) 0 g  x

Limit state boundary

P f = D f f X (x)dx = x:g(x)<0 dx (2.3)
Note that the integration domain in Eq. (2.3) is only implicitly defined by Eq. (2.2), hence making its direct estimation practically impossible in the general case. This limitation can be circumvented by introducing the indicator function of the failure domain, a simple classifier given by:

1 D f (x) =      1, g ( x) < 0 0, g ( x) > 0 (2.4)
In other words, 1 D f (x) = 1 when the input parameters x cause the system to fail and 1 D f (x) = 0 otherwise. This function allows one to cast Eq. (2.3) as follows:

P f = D 1 D f (x)f X (x)dx = E[1 D f (x)] (2.5)
where E[•] is the expectation operator with respect to the PDF f X (x). This reduces the calculation of P f to the estimation of the expectation value of 1 D f (x).

In this section, we investigate various methods to cope with the calculation of Eq.

(2.5). The methods are classified into three main categories: analytical approximation techniques and simulation-based methods. Besides these two categories we can also find some other not commonly-used methods out of our scope, such as stochastic finite element method (SFEM) for structural reliability which is time-consuming.

Furthermore, the time-variant or time-dependent reliability attracts more and more attention. We therefore also review time-variant modeling methods. On the other hand, when no performance function but test data exists, reliability analysis has to rely on statistical methods, such as Bayesian reliability.

Strategies for the estimation of P f

From the definition of 1 D f (x) in Section 2.2 it is clear that determining whether a certain state vector x ∈ D X belongs to D s or D f requires the evaluation of the limit-state function g(x). In the general case this operation can be computationally expensive, e.g. when it entails the evaluation of a computational model on the vector x. For a detailed overview of standard structural reliability methods and applications, see e.g. Ditlevsen and Madsen [START_REF] Ditlevsen | Structural reliability methods[END_REF]; Melchers [START_REF] Robert | Structural reliability analysis and prediction[END_REF]; Lemaire [START_REF] De | Fiabilité des structures: Couplage mécano-fiabiliste statique[END_REF]. In the following, three strategies are discussed for the evaluation of P f , namely approximation, simulation and adaptive surrogate-model based methods, see the classification in Figure 2.2.

Analytical approximation methods

Approximation methods are based on approximating the limit-state function locally at a reference point (e.g. with a linear or quadratic Taylor expansion). This class of methods can be very efficient (in that only a relatively small number of model evaluations is needed to calculate ), but it tends to become unreliable in the presence of complex, non-linear limit state functions. Two approximation methods are introduced in this dissertation, i.e. FORM and SORM.

First Order Reliability Method (FORM)

The first order reliability method aims at the approximation of the integral in eq.(1.3) with a three-step approach: 1) An iso-probabilistic transform of the input random vector x ∼ f X (x) into a standard normal vector u ∼ N (0, I n ); 2) A ( ( ) 0) ( )

f g P P g f d      X x x x
x x Initial question Figure 2.2: Classification of strategies to estimate probability of failure ( [START_REF] Nikolaidis | Engineering design reliability handbook[END_REF], [START_REF] Bilal | Probability, statistics, and reliability for engineers and scientists[END_REF], [START_REF] Engelund | A benchmark study on importance sampling techniques in structural reliability[END_REF]) search for the most likely failure point in the standard normal space (SNS), known as the design point u * ; 3) A linearization of the limit-state surface at the design point u * and the analytical computation of the resulting approximation of P f .

Iso-probabilistic transform

The first step of the FORM method is to transform the input random vector x ∼ f X (x) into a standard normal vector T (x) ∼ N (0, I n ). This transform can be used to map the integral in eq.(1.3) from the physical space of X to the standard normal space of U:

P f = D f f X (x)dx = u∈R n :G(u)<0 φ n (u)du (2.6)
where G(u) = g(T -1 (u)) is the limit-state function evaluated in the standard normal space and φ n (u) is the standard multivariate normal PDF given by: This PDF is maximal at the origin and decreases exponentially with ||u|| 2 . Thus the points that contribute at most to the integral in eq.(2.6) are those of the failure domain that are closest to the origin of the space. Thus the second step in FORM consists in determining the so-called design point , i.e. the point of the failure domain closest to the origin in the standard normal space. This point is obtained by solving an optimization problem:

φ n (u) = (2π) M/2 exp(- 1 2 (u 2 1 + u 2 2 + ... + u 2 n )) (2.7)
u * = argmin(||u|| 2 ), s.t.G(u) < 0, u ∈ R n . ( 2
P f = Φ(-β HL )
, where Φ is the standard normal cumulative density function.

The estimation of P f in the FORM algorithm is based on approximating the limitstate function as the hyperplane tangent to the limit-state function at the design point. Figure 2.3 illustrates this approximation graphically for the two-dimensional case.

To search for the design point in the general non-linear case, eq.(2.8) may be cast as a constrained optimization problem with Lagrangian:

L(u, λ) = 1 2 ||u|| 2 + λG(u) (2.9)
where λ is the Lagrange multiplier. The related optimality conditions read:

u L(u * , λ * ) = 0 ∂L ∂λ (u * , λ * ) = 0 (2.10)
The standard iterative approach to solve this nonlinear constrained optimization problem is given by the Rackwitz-Fiessler algorithm [START_REF] Kuschel | Time-variant reliability-based structural optimization using sorm[END_REF].

Second Order Reliability Method (SORM)

The SORM is an improved version of the FORM for P f estimate. After the design point U * is identified by FORM, the failure probability is approximated by a tangent hyper-paraboloid defined by the second order Taylor expansion of G(U * ) given by:

G(U ) ≈ T |u * • (u -u * ) + 1 2 (u -u * ) T H(u -u * ) (2.11)
where H is the Hessian matrix of the second derivatives of G(U ) evaluated at U * .

The failure probability in the SORM approximation can be written as a correction factor of the FORM estimate that depends on the curvatures of the hyperhyperboloid in eq. (2.11). To estimate the curvatures, the hyper-paraboloid is firstly cast into canonical form by rotating the coordinates system such that one of its axes is the vector. Usually the last coordinate is chosen arbitrarily for this purpose. A rotation matrix Q can be built by setting α as its last row and by using the Gram-Schmidt procedure to orthogonalize the remaining components of the basis. Q is a square matrix such that Q T Q = I. The resulting vector V satisfies:

U = QV (2.12)
In the new coordinates system and after some basic algebra (see e.g. [START_REF] Breitung | Asymptotic approximations for the outcrossing rates of stationary gaussian vector processes[END_REF]), one can rewrite eq.(2.11) as:

G(V ) ≈ || G(u * )|| • (β -V n ) + 1 2 (V -V * ) T QHQ T (V -V * ) (2.13)
where β is the Hasofer-Lind reliability index calculated by FORM,

V n = α T (Q T V )
and V * = 0, ..., β T is the design point in the new coordinates system. By dividing eq.(2.13) by the gradient norm || G(U * )|| and introducing the matrix

A = QHQ T /|| G(U * )||.
One obtains:

G(V ) ≈ β -V n + 1 2 (V -V * ) T A(V -V * ) (2.14)
where

G(V ) = G(V )/|| G(U * )||.
After neglecting second-order terms in V n and diagonalizating the A matrix via eigenvalue decomposition one can rewrite eq. (2.14) explicitly in terms of the curvatures κ i of an hyper-paraboloid with axis α:

G(V ) ≈ β -V n + 1 2 M -1 i=1 κ i V i (2.15)
For small curvatures κ i < 1, the failure probability P f can be approximated by the Breitung formula [START_REF] Breitung | Asymptotic approximations for the outcrossing rates of stationary gaussian vector processes[END_REF]:

P f,SORM ≈ Φ(β H L) M -1 i=1 (1 + β H Lκ i ) -1/2 , κ i < 1 (2.16)
Note that for small curvatures the Breitung formula approaches the FORM linear limit. The accuracy of eq.(2.16) decreases for larger values of κ i , sometimes even if

κ i < 1.
A more accurate formula is given by the Hohenbichler formula [START_REF] Hohenbichler | New light on first-and second-order reliability methods[END_REF].

Moment-based methods

Obviously, the FORM and SORM need to search for the design point by iteration which is time consuming. The moment-based methods [START_REF] Ditlevsen | Structural reliability methods[END_REF] do not need the design point not to mention the iteration to find the design point. Using moments is the simplest way to describe a distribution. After transformed from the variable space to the standardized normal space, the simplified representation of the distribution facilitates the determination of failure probability, i.e. no design point needs to be found any more.

Second moment is a very popular one due to its simplicity. It considers only the first two moments (mean and variance) but not higher order moments, such as skewness and flatness. However only the normal distribution can be represented exactly by second moment. Hence the second moment methods work under the assumption that the random variable obeys a normal distribution. When the second moment density approximation and the 'first-order' approximation are brought together, we have the so-called first-order second moment (FOSM) reliability method [START_REF] Michael | First order analysis of uncertainty in numerical models of groundwater flow part: 1. mathematical development[END_REF]. When considering the non-linear performance, the advanced first-order second moment (AFOSM) [START_REF] Mahadevan | Probability, reliability and statistical method in engineering design[END_REF] may be an appropriate method. The basic random variable is transformed into a reduced coordinate system and the Taylor's expansion is performed at design point rather than the mean value in FOSM.

As stated, the FOSM is based on the assumption of normal distribution. The higher order moments are suggested for other cases of more-complicated distribution. For example, the first-order third moment method (FOTM) was developed by Tichy [START_REF] Tichỳ | First-order third-moment reliability method[END_REF] and the higher-order moment methods are reviewed by Zhao [START_REF] Zhao | Moment methods for structural reliability[END_REF]. Those methods involves the skewness, flatness or higher order moment that has been ignored by the standard FOSM. However, it is obviously a drawback for the moment based methods that the process of obtaining statistical moments is very inefficient because that involves also numerical integrations.

Simulation based methods

Simulation methods are based on sampling the joint distribution of the state variables X and using sample-based estimates of the integral in eq.(2.5). At the cost of being computationally very expensive, they generally have a well-characterized convergence behavior that can be exploited to calculate confidence bounds on the resulting Pf estimates. Three sampling-based algorithms are introduced here, i.e.

Monte Carlo simulation, Importance Sampling and Subset Simulation.

Monte Carlo Simulation

Monte Carlo (MC) simulation is used to directly compute the integral in eq. (1.3) by sampling the probabilistic input model. Given a sample of size N of the input random vector X, X = x (1) , ..., x (N ) , the unbiased MCS estimator of the expectation value in Eq. (1.4) is given:

P f,M C ≈ Pf = 1 N N k=1 1 D f (x (k) ) = N f N (2.17)
where N f is the number of independent samples such that g(x) < 0. In other words, the Monte Carlo estimate of the failure probability is the fraction of samples that belong to the failure domain over the total number of samples. An advantage of Monte Carlo simulation is that it provides an error estimate. Indeed the indicator function 1 D f (x) follows by construction a Bernoulli distribution with mean µ 1 D f = P f and variance σ 2

1 D f = P f (1 -P f ).
Hence, Pf has the variance σ2 P f given by

var( Pf ) = 1 N 2 var( N k=1 1 D f (x (k) )) = 1 N 2 N k=1 var(1 D f (x (k) ) = N • σ 2 1 D f N 2 = σ 2 1 D f N (2.18)
Typically such plots will show that these measures are reduced as the number of samples increases and that a degree of stability is reached at a sufficiently high number of samples. The confidence intervals on Pf can be given as follows [START_REF] Reuven | Simulation and the Monte Carlo method[END_REF]:

Pf ∈ [ Pf + σP f Φ -1 (α/2), Pf + σP f Φ -1 (1 -α/2)] (2.19)
where Φ(x) is the standard normal cumulative distribution function (CDF) and α ∈ [0, 1] (significance level) is a scalar such that the calculated bounds correspond to a confidence level of 1 -α. An important measure for assessing the convergence of a MCS estimator is given by the coefficient of variation (COV ) defined as:

COV = σ P f Pf = (1 -Pf ) N • Pf (2.20)
The coefficient of variation of the MCS estimate of a failure probability therefore decreases with √ N and increases with decreasing P f . To give an example, to es- The MCS method is powerful, when applicable, due to its statistically sound formulation and global convergence. However, its main drawback is the relatively slow converge rate that depends strongly on the probability of failure. The variance reduction techniques, such as importance sampling [START_REF] Melchers | Importance sampling in structural systems[END_REF] and subset simulations [START_REF] Au | Estimation of small failure probabilities in high dimensions by subset simulation[END_REF],

have been proposed.

Importance Sampling

Importance sampling (IS) is an extension of the FORM and MCS methods that combines the fast convergence of FORM with the robustness of MCS. The basic idea is to recast eq. (1.3) as:

P f = Dx 1 D f (x) f X (x) φ(x) φ(x)dx = E φ [1 D f (x) f X (x) φ(x) ] (2.21)
where φ(x) is an M-dimensional importance distribution (also called instrumental /proposal distribution) and E φ denotes the expectation value with respect to the same distribution. The estimate of P f given a sample set X = x (1) , ..., x (n) drawn from φ is therefore given by:

Pf = Êφ [1 D f (X) f X (X φ(X) ] = 1 N N i=1 [1 D f (x (k) )w(x (k) )] (2.22)
where the weight w(x (k) ) = f X (x (k) )/φ(x (k) ). Briefly speaking, the following procedures are executed to estimate Pf : 1) Take N samples i.e. X = x (1) , ..., x (n) from the proposal distribution φ(x); 2) Calculate the target density f X (x (k) ) and proposal density φ(x (k) ); f X (x (k) ) is the joint PDF value of the uncertain parameters;

3) Calculate the weight w(x (k) ) and the indicator value 1 D f (x (k) ) for each sample x (k) ; 4) Estimate the value Pf according to eq. (2.22). The most important task is to find a proposal distribution φ(X). The proposal distribution can be constructed from the design point u * . In standard normal space, eq.(2.22) can be rewritten as:

Pf = Êφ [1 D f (T -1 (u)) φ M (u) Φ(u) ] (2.23)
When the results from a previous FORM analysis are available, a particularly efficient sampling distribution in the standard normal space is given by [START_REF] Melchers | Importance sampling in structural systems[END_REF]:

Φ(u) = φ M (u -u * ) (2.24)
where u * is the estimated design point. Given a sample set U = u (1) , ..., u (N ) of φ(u), the estimate of P f becomes:

Pf = 1 N exp(-β 2 HL /2) N i=1 1 D f (T -1 u i )exp(-u (k) • U * ) (2.25)
with corresponding variance:

σ2 P f = 1 N 1 N -1 N i=1 1 D f (T -1 (u (k) )) φ(u (k) ) Φ(u (k) ) (2.26)
The coefficient of variation and the confidence bounds can be calculated analogously to eqs (2. [START_REF] Dasgupta | Material failure mechanisms and damage models[END_REF]) and (2.20), respectively, and can be used as a convergence criterion to adaptively improve the estimation of Pf .

Subset Simulation

Monte Carlo simulation requires a large number of limit-state function evaluations to converge with an acceptable level of accuracy when P f is small. Subset simulation is a technique introduced by Au and Beck [START_REF] Au | Estimation of small failure probabilities in high dimensions by subset simulation[END_REF] that aims at offsetting this limitation by solving a series of simpler reliability problems with intermediate failure thresholds.

Consider a sequence of failure domains and Support vector machine. Besides, the author find that, the machine learning models such as ensemble learning models and ANNs are attracting more and more attention recently. Therefore, these methods have large potentials. In the following, a detailed description of each of the methods is given.

D 1 ⊃ D 2 ⊃ ... ⊃ D m = D f such that D f = ∩ k = 1 k D k . With
Pf = P (D m ) = P (∩ k = 1 m )P (D k ) = P (D 1 ) m-1 i=1 P (D i+1 |D i ) (2.

Response surface methods

For complicated components, it is hard to find the close-form limit state functions.

The moment based methods mentioned above cannot be applied directly. Some researchers ( [START_REF] Hoon | Response surface augmented moment method for efficient reliability analysis[END_REF], [START_REF] Byeng | A new response surface methodology for reliability-based design optimization[END_REF]) proposed to use response surface to determine a substitute of the implicit performance functions. The basic idea is to suppose a simple parametric expression with respect to random variables then interpolate within some running results to estimate those parameters such that the interpolation error is minimized.

The approximated performance function ĝ(x) is called response surface function which is usually in polynomial form. For example, a second-order polynomial takes the form

ĝ(x) = a 0 + n i=1 a i x i + n i=1 n j=1 a ij x i x j (2.28) or ĝ(x) = a 0 + n i=1 a i x i + n i=1 n j=1 a ii x 2 i (2.29)
where a i , a ii and a ij are the coefficients that need to be determined; n is the number of random variables. For a complete second-order polynomial (see eq.(2.28)), the number of coefficients to be estimated is

K = 2n + 1 + n(n -1)/2.
Therefore, the size of the training data should be larger than K. Based on this approximation the FORM/SORM and all of the moment-based methods can be used after then.

Polynomial chaos expansion

The original PCE, also termed as the homogenous chaos expansion and Hermite polynomial chaos expansion (HPCE), was developed by Wiener [START_REF] Wiener | The homogeneous chaos[END_REF]. With this expansion, any second-order random variable or stochastic process, i.e. the quantities with finite variance, may be expanded as follows:

u = û0 H 0 + ∞ i 1 =1 ûi 1 H 1 (ξ i 1 )+ ∞ i 1 =1 i 1 i 2 =1 ûi 1 i 2 H 2 (ξ i 1 , ξ i 2 )+ ∞ i 1 =1 i 1 i 2 =1 i 2 i 3 =1 ûi 1 i 2 i 3 H 3 (ξ i 1 , ξ i 2 , ξ i 3 )+... (2.30)
where H p (ξ i 1 , ..., ξ ip ) denotes the multivariate Hermite polynomial chaos of order p in terms of standard normal vector ξ = [ξ i 1 , ..., ξ ip ] and ûi 1 ...ip is the associated coefficient. For notational convenience, eq.(2.30) can be rewritten as

u = ∞ i=1 u i Φ i (ξ) (2.31)
In this expression, there is a one-to-one correspondence between the polynomial basis functions Φ i (ξ) and H p (ξ i 1 , ..., ξ ip ), also the deterministic coefficients u i and ûi 1 ,...,ip . Two categories of methods are distinguished to determine the coefficients the intrusive method [START_REF] Au | Estimation of small failure probabilities in high dimensions by subset simulation[END_REF] and non-intrusive method [START_REF] Roger | Stochastic finite elements: a spectral approach[END_REF]. When the random inputs are standard normal, a possible Hilbertian basis is the multivariate Hermite polynomial chaos basis.

Time-variant reliability

Most mechanical components experience changing the material properties and the running environments over time. In this case the performance is also a function of time or a stochastic process, denoted by g t (x, t). So does the reliability. Similarly the failure probability function of time is defined as

P f (t) = P r [g t (x, t) ≤ 0] = gt(x,t)≤0 f (x, t)dx (2.32)
where f (x, t) is the density function of random variable and time. It is almost impossible to exploit the whole failure function of time. However the time variant failure probability is reduced to a time-invariant problem for a fixed time. All methods mentioned for time-invariant reliability can be applied. Sometimes and in some applications, the failure probability in a time interval is of more interest than the general time-variant failure probability. Suppose the time interval is denoted by

T in = [t 1 , t 2 ]
, then the time-interval failure probability is defined as

P f (T in ) = P r [g t (x, t) ≤ 0], t ∈ T m (2.33)
It is still difficult to compute the probability due to the continuity of time. This can be solved by various techniques, such as the work of Breitung [START_REF] Breitung | Asymptotic approximations for the outcrossing rates of stationary gaussian vector processes[END_REF], Li [START_REF] Li | Mean out-crossing rate of nonlinear response to stochastic input[END_REF], Kuschel [START_REF] Kuschel | Time-variant reliability-based structural optimization using sorm[END_REF] and of Andrieu-Renaud [START_REF] Andrieu-Renaud | The phi2 method: a way to compute time-variant reliability[END_REF]. The basic idea is to use the outcrossing rate over the limit state of the structure. The outcrossing event at time t is defined as the event that at time t the performance g t (x, t) > 0 and at time t + dt, g t (x, t + dt) ≤ 0

, where dt) is a small time increment. Thus the mean value of the outcrossing rate at time t is estimated as follows

v c (t) = lim δt→0 P r [g t (x, t) > 0 ∩ g t (x, t) ≤ 0] δt (2.34)
Usually it is a small probability that the random process g t (x, t) out-crosses the bound 0. In this case we can assume that two arbitrary outcrossing events are independent of each other. In other words, we assume that the total number of the outcrossing events in T in obeys a Poisson distribution [START_REF] Grimmett | Probability and random processes[END_REF]. Finally we can get an approximation of the failure probability by using Poisson process model,

P f (T in ) ≈ exp{- t 2 t 1 v c (t)dt} (2.35)
For a component composed of multiple response processes, its failure probability is a joint probability of the individual events, i.e.

P f (T in ) = P r [ i∈I lim min t∈T in g (i) t (x, t) ≤ 0] (2.36)
where g

(i)
t (x, t) is ith limit state function under consideration and I lim is the index set of the limit state functions. If the response processes are independent, the joint probability can be evaluated straightforwardly. Further, Song [START_REF] Song | Joint first-passage probability and reliability of systems under stochastic excitation[END_REF] investigated how to consider the reliability of a component who consists of multiple interdependent response processes or a vector random process.

Statistical learning models

When the performance functions are implicit as often encountered in real world, the analytical approximation techniques cannot do anything to find them and the Monte Carlo simulations require a million times of calculation on LSF. This is impractical in the complex cases because of the large consumption of time. One of the strategy to facilitate this difficulty is the use of polynomial function which approximates the implicit performance and is also known as response surface method.

However it may be time-consuming to use the polynomial function when the components are complex and the number of random variables is large [START_REF] Martins | Comparison of response surface and neural network with other methods for structural reliability analysis[END_REF]. Moreover the approximate performance function is lack of adaptivity and flexibility and we cannot guarantee that it is sufficiently accurate to the true one. To address this issue, researchers proposed to use the statistical classifiers as an additional treatment or an approximation of performance. These methods are called classification-based methods. The mostly-used ones for reliability analysis are the methods using neural networks ( [START_REF] Hurtado | Neural-network-based reliability analysis: a comparative study[END_REF], [START_REF] Deng | Structural reliability analysis for implicit performance functions using artificial neural network[END_REF]) and those using support vector machine [START_REF] Hurtado | Filtered importance sampling with support vector margin: a powerful method for structural reliability analysis[END_REF]. The readers are also referred to [START_REF] Hurtado | Structural reliability: statistical learning perspectives[END_REF] in which Hurtado has investigated detailedly the structural reliability in statistical perspective and [START_REF] Richard O Duda | Pattern classification 2nd edition[END_REF] the detailed aspects of the classifiers.

Support Vector Machine

Support vector machines [START_REF] Cortes | Support-vector networks[END_REF] are supervised learning models with associated learning algorithms that analyze data used for classification and regression analysis. Given a set of training examples, each marked as a binary target value corresponding to one of the two categories, a SVM model is built that will mark a new example as one target value or the other. In this algorithm, each example is plotted as a point in the n-dimension space (n is the number of features in each example). Then a hyperplane is found that differentiate the two classes optimally, meanwhile a margin is determined (see Figure 2

1 2 ||w|| 2 , s.t.y i (w • x i + b) ≥ 1, ∀i (2.37)
In regression case, SVM is called Support Vector Regression, or SVR. As the output is a continuous value, it becomes difficult to predict as there are infinite possibilities.

To handle this, a margin of tolerance ε is preset such that the optimization problem is formulated as (see Figure 2.8),

min w,b 1 2 ||w|| 2 , s.t.|y i -(w • x i + b)| > ε, ∀i (2.38)

Artificial Neural Networks

Artificial neural networks (ANN) [START_REF] Friedman | The elements of statistical learning[END_REF] are one of the main tools used in machine learning. An ANN is an interconnected group of nodes, inspired by a simplification of neurons in a brain. It generally consist of input and output layers, as well as a hidden layer consisting of units that transform the input into something that the output layer can use, see Figure 2.9. The central idea of ANNs is to extract linear combinations of the inputs as derived features, and then model the target as a nonlinear function of these features. A basic unit of an ANN (i.e. a neuron) is shown

2 || || w 1 wx b   0 wx b   1 wx b   
f x   ( ) f x wx b   ( ) f x       0 Support

k-Nearest Neighbor (k-NN) method

The idea behind the k-NN algorithm is to build a classification method using no assumptions about the form of the function y = f (x 1 , x 2 , ..., x p ) that relates the dependent variable, y to the independent variables x 1 , x 2 , ..., x p . The only assumption we make is that it is a 'smooth' function. This is a non-parametric method because it does not involve estimation of parameters in an assumed function form.

This algorithm is used for classification and regression. There are two important problems to solve in the process of learning. The value k is generally a positive, small integer which can directly affect the prediction results. Cross-validations can be applied to determine the optimal k. For two points x (1) = (x (1) (1), ..., x (1) (n))

and x (2) = (x (2) (1), ..., x (2) (n)), the distance is calculated using one of the following measures: In classification, an object is classified by a majority vote of its neighbors, with the object being assigned to the class the most common among its k nearest neighbors.

• Euclidean Distance: d(x, y) = n k=1 (x (1) (k) -x (2) (k)) 2 ; • Minkowski Distance: d(x, y) = ( n k=1 |x (1) (k) -x (2) (k)| p ) 1/p ; • Mahalanobis Distance: d(x, y) = n k=1 |x (1) (k) -x (2) (k)| .
Figure 2.12 illustrates how k-NN is employed to make predictions in classification case. Similarly, in regression case, the output is the average of the values of its k nearest neighbors. Both for classification and regression, a useful technique can be to assign weight to the contributions of the neighbors, so that the nearer neighbors contribute more to the average than the more distant ones.

Bayesian inference method

Bayesian inference is based on Bayes' Theorem given by [START_REF] Sudret | Stochastic finite element methods and reliability: a state-of-the-art report[END_REF]

p(θ|D) = c -1 L(θ; D)p(θ)
, where θ is the vector of unknown parameters; D is a set of available data; L(θ; D) is the likelihood of θ when we consider D, it is calculated by values of all trees (regression case). More details will be provided in Chapter 3.

L(θ; D) = p(D|θ), p (
1 2 1  1 2 1  3 8  3 8  3 5  3 5 

Tree methods: Gradient Boosting

Gradient Boosting is a machine learning meta-algorithm. It can be used in conjunction with many other types of learning algorithms to improve their performance.

The output of the other learning algorithms ('weak learners') is combined into a weighted sum that represents the final output of the boosted classifier. AdaBoost is adaptive in the sense that subsequent weak learners are tweaked in favor of those instances misclassified by previous classifiers. The individual learners can be weak, but as long as the performance of each one is slightly better than random guessing, the final model can be proven to converge to a strong learner. Weak classifiers can be Decision tree/Decision stump, Neural Network, Logistic regression or even SVM.

To determine the coefficient for each classifier, we initialize the weight distribution of the samples in the dataset by a uniform distribution, and update this distribution by considering the misclassified samples in each loop. Meanwhile we calculate the coefficient for the current classifier.

Finally, we have a certain number of weak classifiers as well as their weights, then the strong classifier is obtained as the weighted sum of these classifiers,

f (x) = M i=1 α t h t (x) (2.39)
where α t is the coefficient for the 'weak' classifier h t (x). For GB, we use all training data, instead of bootstrapping, to build each tree.

The first tree is built from an initialized constant model (i.e. a constant value or a one-node tree) [START_REF] Friedman | The elements of statistical learning[END_REF]. From the second tree, to build each tree, we need to calculate the errors of the former trees (as a weighted sum) on all training points until the stopping criteria (number of trees, error threshold, • • • ) are satisfied. See Figure 2.16.

REPTree and Random Tree

REPTree (Reduced Error Pruning Tree) is a fast decision tree learner that builds a classification/regression tree by information gain/variance as the splitting criterion, and prunes it using reduced error pruning method [START_REF] Quinlan | Simplifying decision trees[END_REF]. We assume there is a training set and a test set. REPTree generates a single decision tree from the training set.

The tree is grown in the standard top-down manner, by finding a split for the current node (if entropy/square error can be reduced by splitting the data further), dividing the data based on the split, recursively generating corresponding successor nodes for each node. Once the tree has been grown, it is pruned using Quinlan's reducederror pruning method with back-fitting [START_REF] Quinlan | Simplifying decision trees[END_REF], making pruning decisions by measuring classification error/mean squared error on the hold-out set.

Random tree is a method for constructing a tree that performs no pruning [START_REF] Martin | A monte carlo method for reliability-based design opitmization[END_REF].

For classification problem, Random Tree grows a tree by selecting splits based on information gain, the attribute with the maximum information gain is chosen to split the current node. For regression problem, it selects the split that minimizes squared error (locally, for the data at the node being split, consider the mean as the predictor for each candidate subset to calculate squared error). Different from REPTree that chooses an attribute among all to split the node, a Random Tree considers a subset of randomly chosen attributes for splitting at each node. It performs multi-way splits on nominal attributes, with one branch for each value. (b). the way to find the splitting point. In RF, a decision tree firstly finds the best splitting point for each candidate feature, then chooses the best one; however, in an ET, a random splitting point is chosen for each candidate feature, then the best one

1 x    1 n x  2 x    s x A

Chapter summary

In this chapter, we have presented the fundamental knowledge of reliability. Traditional methods such as FORM, SORM and response surface methods are introduced in detail. Simulation-based methods including MCS and advanced MCS are described. Besides, the statistical machine learning methods are extensively introduced that include SVM, ANN, k-NN and trees-based methods. The tree-based machine learning methods are the research focus in this dissertation, and they are going to be explored and applied in structural reliability analysis in the later chapters.

Introduction

The design and optimization of industrial products (e.g. automobiles, bridges, airplanes• • • ) have made the computing tools inevitable. Despite the increasingly accurate models and the fast calculation of computers, the complexity of the real world problems is difficult to handle comprehensively. Generally, the calculations rely upon some simplified assumptions about the system such that the computational models are possible to build and analyzed. Moreover, model responses are induced by feeding the values of the input parameters into the model. However, due to complex environmental factors as well as the design defects, the input values are always difficult determine. In other words, they are characterized with uncertainties.

The input uncertainties finally result in the response uncertainties of the system.

This part mainly introduces the general framework to deal with the uncertainty propagations within the system.

General framework

Probabilistic engineering aims at describing the uncertainties existing in the physical systems and studying the influence of these uncertainties on the system response. In this thesis, methods for applying uncertainty analysis in physical systems are taken into account. The main steps for such analysis are summarized in Figure 3.1.

Step B Probabilistic Input model Quantification of sources of uncertainty

Step A Physical Model Model of the system/Assessment criteria

Models of physical system

Step A consists of building the mathematical model from the physical process analysis of the system and setting the assessment criteria (e.g. failure criteria) that will be used to evaluate the physical system of interest. This step gathers all the ingredients used for a classical deterministic analysis of the physical system to be analyzed. Let X denote the vector containing all basic random input variables.

When considering models of mechanical systems, these variables usually describe the randomness in the structural, environmental and load parameters. The physical model can be seen as a black-box function, i.e. as an unknown map from the space of input parameters to that of output quantities [START_REF] Lataniotis | Uqlab user manual-the model module[END_REF]:

Y = F (X) (3.1)
where X is a random vector that parametrizes the variability of the input parameters (typically through a joint probability density function, PDF) and Y is the vector of model responses quantities. In a mechanical context, these response quantities can be displacements, stress components, etal. The computational model can be deterministic in the sense that evaluating it repeatedly for a given input realization x 0 will always give the same result y 0 = F (x 0 ). However, the uncertainty in the input variables causes Y to be a random variable/vector as well. In this case, for each realization of X, the response vector can be determined through pointwise evaluations y (i) = F (x (i) ), where the input vector x (i) is a realization of X. Indeed, one of the main applications of uncertainty quantification is that of propagating the randomness in the input X to the output Y . Commonly used models in advanced application often comprise both random and deterministic parameters. When this is the case, the following notation is introduced to clarify such distinction:

Y = F (X, P ) (3.2)
where P is a set of deterministic parameters that are used to properly configure a model (e.g. configuration options, fixed values for parametric functions, etc.).

Analytic functions can be considered as a computational model that is normally known in their closed-forms. However, the vast majority of physical phenomena cannot be approximated by closed-form equations. In the domain of stochastic dynamics with random vibrations, numerical models, such as finite element schemes, are always adopted to calculate the model response.

Quantification of sources of uncertainty

A designed system generally provides a theoretical framework where the input parameters are selected in order that design criteria related to the purpose of the system are satisfied. In contrast, a practical system is a physical object that is built according to a given design. In fact, a real system cannot fully match the initial design. On one hand, the material properties of the real system may differ slightly from the designed properties; on the other hand, the loadings of the designed system are always idealized such that they can only roughly describe the characteristics of those of the real system.

For all the reasons above, a probabilistic model of the input parameters becomes necessary. The uncertainties that are related to the input parameters may be of different kinds, but they are mainly classified into two classes, i.e. aleatoric uncertainty and epistemic uncertainty. The details of the two classes are introduced in Figure 1.2 of Section 1.2.2. In many cases, it is difficult to distinguish clearly between aleatoric and epistemic uncertainty, since both types may be present in the same system. In this thesis, no distinction is made between the two kinds of uncertainties in their modelling, and the probabilistic model of the latter will be built to characterize uncertainties in the input parameters. It is direct to estimate a probabilistic model that represents the scattering of the available data since it is merely a problem of statistics. Note that in some cases (e.g. in the design stage of the object system) where no data is available, a prior probabilistic model can be pre-assumed by expert experience. Then Bayesian statistics can be applied to combine the priori model and the scarce data. In general, step B results in a specification of a random vector of input parameters X in terms of its joint PDF. When spatial variability is modelled by random fields, the discretization of the random fields will result in some basic random variables as an addition to the input random vector.

Uncertainty propagation

Let us consider a random response Y = F (X), where the input X is random vector.

The most important content in the field of probabilistic engineering mechanics is to study the probabilistic content of Y , namely its joint PDF f Y (y). Practically, this function is not directly computable except in special cases. Therefore, techniques for uncertainty propagation have to be developed. Based on the specific information on the random response that is studied, these techniques can be classified into three categories, see Figure 3.2.

Response variability methods are mainly about computing the mean and variance values of the model response. These indices only give some information on the central part of the response PDF, so higher order moment methods can be investigated. Structural reliability methods [START_REF] Robert | Structural reliability analysis and prediction[END_REF] essentially investigate the tails of the response PDF and compute the probability of exceeding a prescribed threshold (i.e. probability of failure). Spectral methods characterize the complete randomness of the response in an intrinsic way by using suitable tools of functional analysis. Also known as stochastic finite element methods in the context of computational mechanics [START_REF] Roger | Stochastic finite elements: a spectral approach[END_REF], spectral methods have been a popular research subject during recent years.

These methods aim at constructing an expansion from a suitable basis of functions so that researchers are allowed to solve problems of second moment or structural reliability by a straightforward post-processing of the expansion coefficients. In this thesis, structural reliability methods are the main concern.

Conditional failure probability

Basic concepts and definition

The object structure studied in this research is a linear dynamic system with uncertainties existing in the structural properties, such as mass, damping and stiffness.

Besides, the structure is subject to stochastic loading modelled as a Gaussian process. Hence, the assessment of failure probability involves the uncertainties in both structural parameters and excitations. As introduced in section 1.2.3, for the first excursion probability problem, the performance function is equal to or smaller than zero whenever the response exceeds the prescribed threshold. Taking into account the definition of performance function in Section 2.2.1, the first excursion probability is defined by means of a multi-dimensional integral:

P f = g(x,z)≤0 p X (x)p Z (z)dxdz = x∈Ω X ,z∈Ω Z I f (x, z)p X (x)p Z (z)dxdz (3.3)
where I f (x, z) is the indicator function which equals to 1 when the performance function g(x, z) ≤ 0 and 0 otherwise. It is noted that the calculation in Eq. (3.3) considers uncertainty in both structural parameters and excitations. In fact, it is possible to define the first excursion probability conditioned on a particular realization of the uncertain structural parameters. The latter probability is denoted as

P C f (x (i)
) and is defined as

P C f (x (i) ) = g(x,z)≤0 p Z (z)dz = z∈Ω Z I f (x (i) , z)p Z (z)dz (3.4)
where the superscript C in P C f (x (i) ) denotes a conditional failure probability (CFP) with respect to a realization of the uncertain structural parameters. It is noted that the characterization of the integrals in Eqs. (3.3) and (3.4) may involve hundreds of random variables in the context of dynamical systems under stochastic loading. Therefore, the current reliability problem forms actually a high dimensional problem.

Considering the relative independence between the uncertainties of the structural properties and those of the excitations, the integral in Eq.(3.3) can be reformed as follows,

P f = x∈Ω X [ z∈Ω Z I f (x, z)p Z (z)dz]p X (x)dx = x∈Ω X P C f (x)p X (x)dx = E[P C f (x)] (3.5)
where E[•] denotes the mathematical expectation.

Estimation of Conditional failure probability

Practically, Eq.(3.5) can be approximated by analyzing a certain number of realizations of the random vector X, say x (1) , x (2) , ..., x (Nx) . For each realization, a conditional failure probability is estimated. The Monte-Carlo estimator of P f is given by the empirical mean,

Pf,MCS = 1 N x Nx i=1 P C f (x (i) ) (3.6)
Besides, the conditional failure probability in Eq.(3.4) is denoted as

P C f (x (i) ) = E[I f (x (i) , z)] (3.7)
Eq.(3.7) can be approximated by analyzing a certain number of realizations of the random vector Z, say z (1) , z (2) , ..., z (Nz) . The Monte-Carlo estimator of

P C f (x (i)
) is given by

P C f,M CS (x (i) ) = 1 N z Nz j=1 I f (x (i) , z (j) ) (3.8)
While in principle both approximations of probabilities P f and P C f (x) can be carried out through MCS, the demanded computational burden would become quite heavy for the practical engineering problems. This is because the aforementioned probabilities are generally quite small and to ensure the accuracy, a large number of samples

Machine learning methods as surrogates

From Section 2.3.2.1, it is clear that there is a mapping from the space of random vector X to the space of conditional failure probability P C f (X). If there is a model that explicitly provide us this mapping relationship, most of the MCS procedures can be avoided so that the efficiency of the estimation could be largely improved. Based on this idea, surrogate models have been explored this thesis, and the statistical learning models have been extensively studied and employed to accomplish such a task. Figure 3.3 illustrates the theoretical framework to approximate the conditional failure probability P C f (X) by the statistical learning models. In Figure 3.3, a sample set consists of a certain number of cases each of which is a realization of the random vector X. Then by MCS a conditional failure probability

P C f (x (i)
) is estimated. From Eq.(3.8) it is known that in each loop of the MCS, a random realization of the stochastic excitation process is obtained as the input of the structural system and the response is then analyzed, see Figure 3.4 for the details.

As is shown in Eq.(3.8), in practical simulations, the excitations are denoted by a discretized form, i.e. a random vector Z. Thus, the structural responses are obtained also in a discretized form, refer to Figure 3.5. e i and y i is the ith realization of the From the framework illustrated in Figure 3.3, just a limited number of samples are needed from the uncertainty space so that a statistical learning model is built.

                       e e e 1, 1 1,2 1, 2,1 2,2 2, ,1 ,2 , ( ... ) ( ... )
               1 2 N             y y y Structural model
With the induced model the conditional failure probabilities for other samples will be directly estimated. Therefore, a large amount of computation time can be saved.

Chapter 4 and Chapter 5 are devoted to building statistical learning models to approximate the conditional failure probabilities. The performance of these models are also analyzed in various aspects.

Evaluation of very small conditional failure probabilities

As is mentioned in chapter 2, the general MCS can be treated as a standard reference. However, it is not computationally efficient for estimating very small failure probabilities (for example, < 1%) since the number of samples required to achieve a given accuracy is inversely proportional to the scale of Pf. In other words, estimating small probabilities requires information from rare samples that induce system failures. On average it requires many samples before one such failure sample occurs.

In view of this, the importance sampling method should be implemented to compute P f . In principle, an IS method tries to adjust the sampling density so that more samples from the failure region F can be obtained. The efficiency of the method relies on the construction of the importance sampling density (ISD), for which the knowledge about the failure region is inevitably required.

In the framework of MCS, each sample taken as input of the system is analyzed and the system response is obtained. The classic finite element codes is experimentally proved low efficient. Besides, the input excitations are generally discretized according to the time sampling step. Therefore, the response process can be calculated by a convolution integral based on the structural impulse response and the excitations. In this article, we assume the stochastic excitation process for the structure is modelled as a Gaussian process. In this aspect, some representation methods in the random field domain can be applied to describe discretized Gaussian process.

This subsection is devoted to the theories about discrete representation of random processes, the convolution integral based structural response calculation and an IS method to improve the estimation efficiency of conditional failure probabilities.

K-L decomposition of Gaussian excitation process

f (t) = µ(t) + n i=1 u i s i (t) = µ(t) + s(t) T u (3.9)
where u = [u 1 , u 2 , ..., u n ] T is a vector of standard normal variables, s(t) =

[s 1 (t), ..., s n (t)] T is a vector of deterministic basis functions dependent on the covariance structure of the excitation process, and n is a measure of resolution of the representation. To carry out this task, several discretization methods have been developed since the 1980s, a comprehensive review and comparison of these methods is presented in [START_REF] Sudret | Stochastic finite element methods and reliability: a state-of-the-art report[END_REF]. Series expansion methods are mostly considered. In this article, K-L expansion is employed for Gaussian process representation.

K-L expansion method

In the K-L expansion method, the random process can be decomposed as,

f (x) = µ(x) + ∞ i=1 λ i u i φ i (x) (3.10)
where µ(x) is the mean function and u i , i ∈ N + , are zero-mean orthogonal variables.

The eigen-pair {λ i , φ i } are the solutions of the eigenvalue problem:

B C HH (x, x )φ i (x )dx = λ i φ i (3.11)
which is a Fredholm integral equation of second kind. The kernel C HH (x, x ) being an autocovariance function, is symmetric and positive definite. The set of eigenvalues is moreover real, positive and numerable. It is possible to order the eigenvalues λ i in a descending series converging to zero. Truncating the ordered series in eq.(3.10)

after the M -th term gives the K-L approximation:

f (x) ≈ µ(x) + M i=1 λ i u i φ i (x) (3.12)
when the random field under consideration is Gaussian, the set of u i are independent standard normal variables. Eq.(3.11) can be solved analytically only for few auto-covariance functions and geometries of B. Detailed closed form solutions for triangular and exponential covariance functions for one-dimensional homogeneous fields can be found in [START_REF] Roger | Stochastic finite elements: a spectral approach[END_REF]. Consider a discrete representation of time t such that

∆t = T /(n T -1)
where n T is the number of time points such that the time instants of analysis are T k = (k -1)∆t, k = 1, ..., n T . Thus, the discrete representation of f (t) at time t k is:

f (t k ) ≈ f 0 (t k ) + n KL i=1 λ i u i φ i (t k ) = f 0 (t k ) + n KL i=1 f KL i (t k )u i (3.13)
where u i , i = 1, ..., n KL are independent, identically distributed standard Gaussian random variables with joint PDF p u (u), f 0 (t k ) and f KL i (t k ) denote the mean function and the sth component of the K-L vector at time t, and n KL , is the order of truncation of the series expansion. Note that according to Eq.(3.13) the force at the kth time instant is actually a function of the vector of uncertain variables u = (u 1 , u 2 , ..., u n KL ) T . The K-L vector f KL 1 (t k ), ..., f KL n KL (t k ) is determined by solving an eigen-problem of the corresponding covariance matrix of the discrete stochastic process. In this dissertation, it is assume, without loss of generality, that the excitation is a zero-mean Gaussian process, i.e. f 0 (t k ) = 0, k = 1, ..., n T .

An exemplary implementation of K-L expansion

This part introduces a simple implementation of K-L expansion. Assume that the output of a system, according to time t, is denoted as In this simulation, the five largest eigenvalues and their corresponding eigenvectors are calculated. In Figure 3.8, it is found that the eigenvalues are largely different, the largest is about 35.0 while the smallest one is close to 0. Therefore, the choice of n KL = 5 ensures the accuracy of K-L expansion. Figure 3.9 displays the five eigenfunctions related to the five largest eigenvalues. Figure 3.10 shows the five scaled eigenfunctions, refer to eq. (3.13). Notice that the value of the ith scaled eigenfunction at time t k is calculated as

f (t) = 1.4118 + [b * c/( √ π(t -s)] * exp [-c 2 * (log|t -s| -d) 2 ] ( 3
√ λ i φ i (t k ) = f KL i (t k ).

Convolution integral to calculate structural responses

Consider a classically-damped linear elastic structural system represented by an appropriate model (e.g. a finite element model) comprising a total of n DOFs. In addition, consider a vector of random variables of dimension grouping the uncertain structural parameters. The system is subjected to a Gaussian process excitation with zero mean, f (t), which is represented in the discrete form in Eq. (3.13). Then the response of the system can be evaluated by the convolution integral below [START_REF] Der | The geometry of random vibrations and solutions by form and sorm[END_REF],

y i (t, x) = t 0 h i (t -τ, x)f (τ )dτ (3.16)
where h i (t -τ, x) is the unit impulse response function for the ith DOF of the structure at time t due to a unit impulse applied at time τ . x is a vector that consists of the structural properties. Without loss of generality, the zero initial conditions at t = 0 is assumed. As a result of linearity, the response of interest is actually a linear combination of the contributions from each input f (τ ). Substituting for f (τ ) in eq.(3.13) and then changing the orders of summation and integration, we obtain

y i (t, x) = t 0 h i (t -τ, x) n KL l=1 u l f KL l (τ )dτ = n KL l=1 u l a l (t, x) = a(t, x) T u (3.17)
where a(t, x) = [a 1 (t, x), ..., a n KL (t, x)] T and

a l (t, x) = t 0 h i (t -τ, x)f KL l (τ )dτ, l = 1, ..., n KL (3.18)
The response process is found to be the scalar product of two vectors: the random vector u and the deterministic basis function vector a(t, x), whose elements are convolutions of the basis functions f KL l (t) of the excitation and the unit impulse response function h i (t -τ, θ) of the system. The geometric interpretations made earlier for the Gaussian process f (t) apply to the response process y i (t, θ) as well.

For the simulation purpose, the convolution integral in eq.(3.18) is re-written in its discrete form, i.e.

a l (t, x) = ∆t k j=1 h i (t k -t j , x)f KL l (t j ) (3.19)

Failure probability at any time t

Consider the set of realizations of f (t) that lead to the event y(t 0 ) ≥ y 0 at time t = t 0 , where y 0 is a selected threshold. These correspond to realizations of u that satisfy the condition,

y 0 -a(t 0 ) T u ≤ 0 (3.20)
In the space of u, these lie in a half space bounded by the hyper-plane y 0 -a(t 0 ) T u = 0 having the unit normal α(t 0 ) = a(t 0 )/||a(t 0 )|| and distance β(y 0 , t 0 ) = y 0 /||a(t 0 )|| from the origin. This is illustrated in Figure 3.12 in the plane formed by the coordinate u 1 and the vector α(t 0 ). According to the theory of structural reliability

0 1 u 2 u 0 0 ( , ) y t  * 0 0 ( , ) y t u 0 0 ( ) 0 T y a t   u 0 ( ) t   0 0 ( ) 0 y y t   Figure 3
.12: Illustration of response at time t 0 in a geometric way [START_REF] Der | The geometry of random vibrations and solutions by form and sorm[END_REF] [44], g 0 (u) = y 0 -a(t 0 ) T u is the limit-state function, g 0 (u) = y 0 -a(t 0 ) T u = 0 is the limit-state surface, α(t 0 ) is the unit outbound normal vector which towards the failure region, and β(y 0 , t 0 ) is the reliability index for the event g 0 (u) ≤ 0. From a well known result for the probability content of a half space in the standard normal space, the probability of failure is denoted as

P (g 0 (u) ≤ 0) = Φ(-β(y 0 , t 0 )), (3.21)
where Φ(.) denotes the standard normal cumulative probability function.

Among all realizations of u that give rise to the event y(t 0 ) ≥ y 0 , the one that has the highest likelihood is the one nearest to the origin. This point, known as the 'design point' in the theory of structural reliability, is given by

u * (y 0 , t 0 ) = β(y 0 , t 0 )α(t 0 ) = y 0 a(t 0 ) ||a(t 0 )|| (3.22)
The corresponding 'design point' excitation and response, according to eq.(3.9), eq.(3.17) and eq. (3.22), are

f * (t) = s(t) T u * (y 0 , t 0 ) = y 0 s(t) T a(t 0 ) ||a(t 0 )|| 2 = y 0 f KL (t)a(t 0 ) ||a(t 0 )|| 2 (3.23) y * (t) = a(t) T u * (y 0 , t 0 ) = y 0 a(t) T a(t 0 ) ||a(t 0 )|| 2 (3.24)
It is noticed that y * (t 0 ) = y 0 corresponds to the critical state at time t 0 . For a linear system subject to Gaussian excitations, the design point realization is proportional to the threshold y 0 . That means, if the threshold y 0 changes, then the design point realization will be changed with the same proportion. Of most interest is the design point excitation f * (t), which is the specific realization of f (t) that has the highest likelihood to give rise to the event y(t 0 ) ≤ y 0 . This realization is of particular interest from the viewpoint of design, as one can assure safety by providing adequate capacity against this particular deterministic excitation [START_REF] Der | The geometry of random vibrations and solutions by form and sorm[END_REF].

Formulation of conditional failure probability

In the perspective of first-passage probability, it is general to compare the responses of interest y i (t), i = 1, ..., n r against acceptable threshold levels y * i within the time duration T of the stochastic excitation. A failure takes place whenever the response y i exceeds its corresponding threshold. From this point of view, failure implies not meeting the predefined conditions (this does not necessarily imply collapse).

According to subsection 3.3, the failure event F can be defined in terms of x and u:

F = {x ∈ R nx , u ∈ R n KL : g(x, u) ≤ 0} (3.25)
where g(x, u) is the so-called performance function. The value of g(.) is equal or smaller than zero whenever a response exceeds its prescribed threshold. Considering the different DOFs of the structure as well as the sampled time points, the performance function g(x, u) is formulated as

g(x, u) = 1 -max i=1,...,nr ( max k=1,...,n T ( |y i (t k , x, u)| y * i )) (3
P C f (x) = g(x,u)≤0 p u (u)dxdu = u∈Ωu I F (u)p u (u)du (3.28)
It is noticed that the integrals in eq.(3.28) may involve hundreds or thousands random variables related to the stochastic loading. Therefore, the corresponding reliability problem constitute a high dimensional problem. While in principle P C f could be estimated using MCS, the associated computational burden would be quite considerable for problems of engineering involving very small failure probabilities. This is because, for small failure probabilities, a very large number of samples are required to ensure a reliable estimation result. In order to improve the computational efficiency, a very efficient importance sampling strategy [START_REF] Au | First excursion probabilities for linear systems by very efficient importance sampling[END_REF] for estimating the conditional failure probability is introduced in the next subsection.

Elementary failure events

According to the definition of first excursion problem, failure occurs whenever any of the responses of interest y i exceeds its designed threshold y * i within the time duration of the excitation process. The occurrence of failure at the kth time step due to the ith system response is termed elementary failure event [START_REF] Au | First excursion probabilities for linear systems by very efficient importance sampling[END_REF] and is denoted as F i,k (x), i = 1, ..., n r , k = 1, ..., n T . Therefore, the failure event F (θ) is the union of these elementary failure event, i.e.

F (x) = nr i=1 n T k=1 F i,k (x) (3.29)
where

F i,k (x) = {u ∈ R n KL : |y i (t k , x, u)| ≥ y * i } (3.30)
The elementary failure event F i,k (x) can be decomposed as the union of two disjoint events [START_REF] Au | First excursion probabilities for linear systems by very efficient importance sampling[END_REF]: up-crossing event where the response up-crosses its threshold , i.e.

F + i,k (x) = {u ∈ R n KL : a i,k (x) T u ≥ y * i } (3.31)
and down-crossing event where the response down-crosses its threshold , i.e.

F - i,k (x) = {u ∈ R n KL : -a i,k (x) T u ≥ y * i } (3.32)
For the out-crossing event

F i,k (x) = F + i,k (x) ∪ F - i,k (x), since F + i,k (x) and F - i,k (x) are disjoint, F i,k (x) has two design points corresponding to F + i,k (x) and F - i,k (x)
. The failure probability for F i,k (x) is the sum of those of the two out-crossing cases.

According to eq.(3.18), their hyper-planes have the unit normals with inverse directions. Moreover, they have the same distance from the origin, i.e., β + i,k (x) = β - i,k (x) Therefore, the two regions are symmetric, the probability related to F i,k (x) is simply 2Φ -1 (-β i,k (x)). Figure 3.13 provides an illustration of an elementary failure region

F i,k (x) with n r = 1, n T = 1, n KL = 2.

IS technique to evaluate P C f

To apply IS technique to evaluate the P C f , the integral in eq.(3.28) is re-written as

P C f (x) = u∈Ωu I F (u) p u (u) p IS,u (u) p IS,u (u)du = 1 N N v=1 I f (x, z v ) p u (u (v) p IS,u (u ( v)) (3.33)
where p IS,u (u) is the Importance Sampling density (ISD) function and u (v) , v = 1, ..., N are samples of uncertain vector U obtained via the ISD p IS,u (u). The most important part for implementing the IS procedures described in eq. (3.33) is the design of the ISD function that is able to obtain more samples in the failure region, meanwhile ensuring a low variability of the estimated probability, i.e. failure samples are drawn frequently while the variability of the ratio involving p u (u) and p IS,u (u)

0 1 u 2 u 1 ,1 ( )    1 ,1 ( )    * 1 ,1 ( ) u   * 1 ,1 ( ) u   * 1 ,1 1 ( ) T a u y    * 1 ,1 1 ( ) T a u y   1 ,1 ( ) F   1 ,1 ( ) F   Figure 3
.13: Illustration of an symmetric elementary failure region is low [START_REF] George L Ang | Optimal importancesampling density estimator[END_REF]. For the particular case of linear structures subject to Gaussian loading, [START_REF] Au | First excursion probabilities for linear systems by very efficient importance sampling[END_REF] proposes a very efficient ISD which is defined as a weighted sum of probability density functions conditioned on the elementary failure events described previously:

p IS,u (u) = nr i=1 n T k=1 w i,k (x)p u (u|F i,k (x)) (3.34)
where p u (u|F i,k (x)) is the probability distribution of U conditioned on the elementary failure event F i,k . w i,k is the weight associated with the elementary failure

domain F i,k (x)
and is defined such that [START_REF] Gaspar | Assessment of the efficiency of kriging surrogate models for structural reliability analysis[END_REF]:

w i,k (x) = Φ(-β i,k (x)) nr i=1 n T k=1 Φ(-β i,k (x)) (3.35)
Inserting eq.(3.34) and eq.(3.35) into eq.(3.33) and representing p u (u|F i,k (θ)) by the Bayes' theorem yields the following estimator for the failure probability [START_REF] Au | First excursion probabilities for linear systems by very efficient importance sampling[END_REF]:

P C f (x) = 1 N N v=1
Pf,s (x)

nr i=1 n T k=1 I F i,k (x, z (v) ) (3.36)
where

I F i,k (x, z
) is an indicator function equal to one in case the (i, k)th failure event takes place and zero, otherwise; and Pf,s (x) is the sum of the probabilities of occurrence of the elementary failure events and is defined as:

Pf,s (x) = 2( nr i=1 n T k=1 Φ(-β i,k (x))) (3.37)
As a summary, the estimation of the failure probability using the estimator in eq.(3.33) requires the characterization of the elementary failure domains through the design points and reliability indices. Samples of the vector U distributed according to the ISD in eq.(3.34) are required as well. For details on how to generate these samples, see Table 3.1. We called this K-L expansion based IS method KL-IS method.

Comparisons between standard MCS and KL-IS

To compare the efficiency of standard MCS and KL-IS method, the CPU time is firstly employed as the index. The object structure is a ten-story shear building shown in Figure 3.14, excited by stochastic ground excitations. This system has been studied by Hadi and Arfiadi in [START_REF] Muhammad | Optimum design of absorber for mdof structures[END_REF] and by Lee et al.in [START_REF] Lee | Optimal design theories and applications of tuned mass dampers[END_REF] and, for the sake of comparison, the structural parameters used in the present work are the same as those used by the afore-mentioned authors.

The nominal values of structural parameters are assumed to be m i = 360 × 10 3 kg, The Gaussian white noise process with the standard deviation σ = 2πS 0 /T s is assumed as the excitation process for the structure, where the power spectral density S 0 = 0.031m 2 /s 3 . In standard MCS, we record the average CPU time consumed Input: Deterministic basis function vectors a i,k (x), the set of weights {w i,k (x)} nr,n T 1,1

k i =
.

Output: N samples, i.e. {z (v) } N v=1 .

Step1: Set v = 1. Draw a pair of indices (I, K) from the set {(i, k)}, i ∈ {1, ..., n r }, k ∈ {1, ..., n T }. The pair (i, k) is sampled according to its weight w i,k (x).

Step2: Generate a sample u from the distribution p u (u).

Step3: Draw randomly two numbers c 1 , c 2 from [0, 1] uniform distribution.

Step4:

Calculate α = -Φ -1 ((1 -u 1 )Φ(-β I,K (x)))
, where Φ -1 (•) is the inverse CDF of the standard normal distribution.

Step5: Compute the vector γ I,K = u * I,K (x)/||u * I,K (x)||.

Step6: Set the sample z (v) as z

(v) = sign(0.5 -c 2 ) * [u + (α -u T γ I,K )γ I,K ],
where sign(0.5 -c 2 ) equals 1 if c 2 ≤ 0.5 and -1 otherwise.

Step7: Judge the value v. If v < N , v = v + 1, return to step1 ; otherwise, stop and output the IS samples z (v) , v = 1, ..., N .

to analyze one sample of the excitation process, then, by eq.(2.22) in chapter 2, we can calculate the number of samples needed to calculate a particular value of P f . It is assumed the COV is 10%, which means that the range scope of the estimated P f around the mean is 10%. To apply the KL-IS method, the number of terms in the K-L expansion is set as n KL = 300, and the number of importance samples generated is set as N = 2000. t * 100 P f after some simple algebra from eq.(2.22). Here t is the average CPU time spent to analyze one sample of the excitation process. According to the simulation results, t is calculated as 7.55 × 10 -4 s. As an example, if the actual P f value is 10 -4 , then at least 100 10 -4 = 10 6 samples are needed to analyze, resulting in T = 7.55 × 10 -4 * 100 10 -4 = 755s spent according to the accuracy demand. However, if the actual P f value is 10 -1 , then only 100 10 -1 = 10 3 samples are needed, which consumes T = 7.55 × 10 -4 * 100 10 -1 = 0.755s. Therefore, the standard MCS is only practically suitable to calculate relatively larger P f (for example larger than 1%).

In contrast, Figure 3.16 shows a short but almost constant value of CPU time. The reasons are as follows. Firstly, the KL-IS method employs discrete representation method, i.e. K-L expansions, to represent the excitations for the structure. The K-L expansion decomposes the Gaussian random process into deterministic part and uncertain part. The deterministic part, together with the structural impulse responses (see Figure 3.17). Because the failure probabilities are calculated from the relative response of different DOFs, these responses are also calculated, see Figure 3.18, is used to calculate the convolution integrals so that a coefficient matrix is obtained that comprises the vectors a i,k (x), where i = n r , k = 1, ..., n T (see eq.

(3.17)). It is noticed that once this coefficient matrix is determined, the failure probability of any DOF at any sampled time point can be directly calculated via eq.(3.19) (see results in Figure 3. [START_REF] Dasgupta | Material failure mechanisms and damage models[END_REF]).

Afterwards, we just need to generate the standard Gaussian variables (with dimension n KL ) and feed them into the coefficient matrix to calculate the structural responses. According to the simulation records, these procedures is not time consuming (about 4s). Secondly, in the generating process of importance samples, with the aide of the already calculated coefficients (see eq. (3.17)) the importance samples are very efficient to obtain. This part consumes less than 0.5s.

Even though the number of importance samples are increased, the time consumed does not change so much. The special mechanism of KL-IS method makes it very powerful to calculate very small values of P f without large sacrifice of time. Therefore, the KL-IS method is very suitable to calculate small P f (for example less than 1%). Moreover, the instant failure probabilities of different DOFs within the observed time interval are calculated from their coefficient matrices, see Figure 3.19.

It is found that, from the 1st DOF to the top DOF the failure probabilities decrease

Chapter summary

In this chapter, we have presented the fundamental knowledge of uncertainty quantification. The concept of conditional failure probability is also introduced. This chapter describes the basic procedures to handle the structural uncertainties via machine learning models. Besides, the KL-IS method has been presented in detail due to its clear advantage over the standard MCS method in evaluating very small failure probabilities. In chapter 4, the machine learning theories will be extensively explored and studied in conditional failure probability modeling and prediction.

Introduction

Theoretically, evaluation of the probability of failure can be accomplished by solving a multivariate integral analytically or numerically within the failure domain [START_REF] Chakraborty | A semi-analytical framework for structural reliability analysis[END_REF].

But this is only computationally practical for low-dimension (generally less than five) problems. Thus, much of the efforts have been devoted to developing alternative approaches, such as surrogate models and simulation methods.

Surrogate models were developed to deal with highly non-linear or implicit LSFs.

They are introduced to reduce computational burden in reliability analysis. Response surface method (RSM) is among the most popular surrogate models. Generally, researchers combine RSM with other approaches to refine the model parameters [START_REF] Hadidi | Efficient response surface method for high-dimensional structural reliability analysis[END_REF]. Support vector machine (SVM) has also gained much attention recently. Pan

and Dias [START_REF] Pan | An efficient reliability method combining adaptive support vector machine and monte carlo simulation[END_REF] combined an adaptive SVM and Monte-Carlo simulation to solve nonlinear and high-dimensional problems in reliability analysis. According to Dai et.

al [START_REF] Dai | A multiwavelet support vector regression method for efficient reliability assessment[END_REF], a new multi-wavelet linear programming SVM for the reliability analysis.

Other surrogate models have been proposed by the other researchers [START_REF] Dubourg | Metamodel-based importance sampling for structural reliability analysis[END_REF], [START_REF] Xue | A new unbiased metamodel method for efficient reliability analysis[END_REF].

In terms of simulation methods, Monte-Carlo simulation (MCS) technique is representative to solve high-dimensional and non-linear complex problems. It is dimension-independent, and often regarded as a benchmark to evaluate the other probabilistic methods due to its versatility and robustness. Zio [START_REF] Zio | System reliability and risk analysis[END_REF] described in detail the basic theories about MCS techniques used for reliability analysis in his recent book. Jose et al. [START_REF] Coit | A monte-carlo simulation approach for approximating multi-state two-terminal reliability[END_REF] proposed a MCS methodology to estimate the reliability of a multi-state network, and was able to obtain accurate approximations of the reliability. Daoud and Mahmoud [START_REF] Daoud | Monte carlo simulation-based algorithms for estimating the reliability of mobile agent-based systems[END_REF] proposed to use MCS to obtain the task route reliability of the mobile agent based systems (MABS), and the simulation results showed its robustness. Padmanabhan et. al [START_REF] Padmanabhan | A study using monte carlo simulation for failure probability calculation in reliability-based optimization[END_REF] used Monte Carlo Simulation techniques for Reliability-Based Optimization (RBO), they combined limit state approximations with MCS to improve the efficiency of reliability analysis. Some researchers tried to improve the efficiency of the analysis by combining MCS with other methods, especially surrogate models such as SVM [START_REF] Hurtado | Filtered importance sampling with support vector margin: a powerful method for structural reliability analysis[END_REF], artificial neural networks (ANN) [START_REF] João B Cardoso | Structural reliability analysis using monte carlo simulation and neural networks[END_REF] and [START_REF] Su | A gaussian process-based dynamic surrogate model for complex engineering structural reliability analysis[END_REF]Kriging.

Seen as a classification/ regression procedure, the prediction task can be realized by machine learning models among which SVM [START_REF] Hong | Predicting engine reliability by support vector machines[END_REF] and [START_REF] Kutyłowska | Neural network approach for failure rate prediction[END_REF]ANN were widely considered in recent articles. However, they cannot avoid shortcomings in all situations.

The disadvantages in ANNs mainly include the complex architecture optimization, low robustness and enormous training time [START_REF] Nitze | Comparison of machine learning algorithms random forest, artificial neural network and support vector machine to maximum likelihood for supervised crop type classification[END_REF]. SVM is time consuming for largescale applications and sometimes shows large error in sensitivity calculation of reliability index. Naive Bayes model learns the a priori distribution from the observed data and build the posterior distribution for new observations. But it is based on the conditional independence assumption, and for the continuous variables it needs to discretize them [START_REF] George | Estimating continuous distributions in bayesian classifiers[END_REF]. Bayesian networks (BNs) employ conditional probability tables (CPT) to deal with inter-dependent variables, but the results highly rely on assumptions about the target system behavior [START_REF] Garcia-Teodoro | Anomaly-based network intrusion detection: Techniques, systems and challenges[END_REF]. Besides, the rules encoded by BNs are difficult to interpret efficiently [START_REF] Janssens | Integrating bayesian networks and decision trees in a sequential rule-based transportation model[END_REF].

Random Forest is a tree method distinguished for its robustness and high accuracy in modeling and predictions [START_REF] Raczko | Comparison of support vector machine, random forest and neural network classifiers for tree species classification on airborne hyperspectral apex images[END_REF]. However, its application to reliability prediction is still limited in current literatures. This part therefore aims to explore the feasibility of RF as well as examine its performance in reliability modeling and prediction of passive controlled structures. The remainder of the paper is organized as follows.

Section 4.2 provides the response analysis of multi-TMD structures subject to seismic excitations. Section 4.3 introduces in detail the framework of the proposed method. Section 4.4 provides the numerical simulations and performance analysis.

Section 4.5 concludes this article with comments.

Structural response analysis

An ideal mechanical TMD system (see Figure 4.1) is composed of a main structure represented by a N degree-of-freedom (N -DOF) system, and a TMD structure with n degree-of-freedoms (n-DOF). All the DOFs of the TMDs are arranged in parallel to absorb the vibration energy of the main structure.

In case of the TMD system excited by a base acceleration ÿb , the first (n-

m S i • ÿS i + c S i • ( ẏS i -ẏS i-1 ) + k S i • (y S i -y S i-1 ) + c S i+1 • ( ẏS i -ẏS i+1 ) + k S i+1 • (y S i -y S i+1 ) = -m S i • ÿb (4.1)
where i = 1, 2, ..., n -1, y S 0 = 0. After some algebra, it is equivalent to

m S i • ÿS i + (c S i + c S i+1 ) • ẏS i -c S i • ẏS i-1 -c S i+1 • ẏS i+1 -(k S i + k S i+1 ) • y S i -k S i • y S i-1 -k S i+1 • y S i+1 = -m S i • ÿb (4.2)
For the nth degree of freedom, the motion equation is

m Sn • ÿSn + c Sn • ( ẏSn -ẏS n-1 ) + k Sn • (y Sn -y S n-1 ) + c T 1 • (y Sn -y T 1 ) + k T 1 • (y Sn -y T 1 ) + ... + c Tm • ( ẏSn -ẏTm ) + k Tm • (y Sn -y Tm ) = -m Sn • ÿb (4.3)
Eq.( 4.3) is equivalent to

m Sn • ÿSn + c Sn • ẏSn + k Sn • y Sn -c Sn • ẏS n-1 -k Sn • y S n-1 + m i=1 c T i • ( ẏSn -ẏT i ) + m i=1 k T i • (y Sn -y T i ) = -m Sn • ÿb (4.4)
Therefore, we obtain the motion equations for all DOFs. For all the TMDs, the motion equations are

m T 1 • ÿT 1 + c T 1 • ( ẏT 1 -ẏSn ) + k T 1 • (y T 1 -y Sn ) = -m T 1 • ÿb m T 2 • ÿT 2 + c T 2 • ( ẏT 2 -ẏSn ) + k T 2 • (y T 2 -y Sn ) = -m T 2 • ÿb ... m Tm • ÿTm + c Tm • ( ẏTm -ẏSn ) + k Tm • (y Tm -y Sn ) = -m Tm • ÿb
C ii =              c S i + c S i+1 , i = 1, ..., n -1 c S i + m i=1 c T i , i = n c T i-n , i = n + 1, ..., n + m
K ii =              k S i + k S i+1 , i = 1, ..., n -1 k S i + m i=1 k T i , i = n k T i-n , i = n + 1, ..., n + m (4.8)
In a similar way,

K = diag[K S , K T ].
Here, K S consists of the first n rows and the first n columns of K; K T consists of the last m rows and the last m columns of K.

Assume z 0 = [y S 1 , . . . , y Sn , y T 1 , . . . , y Tm , ẏs 1 , . . . , ẏsn , ẏT 1 , . . . , ẏTn ] T , we obtain

ż0 = A 0 z 0 + r 0 ÿb , (4.9)
where

A 0 = [O n+m , Λ n+m ; K U , K D ].
O n+m and Λ n+m are zero matrix and identity matrix respectively, the sizes of which are all

(n + m) × (n + m). r 0 = [0, ..., 0, -1, ..., -1] T 1×(2n+2m) . K U = [K 0 , A 1 , C 0 , A 2 ],
where

K 0 = -M -1 S × K S (4.10)
C 0 = -M -1 S × C S (4.11) A 1 = [[0] (n-1)×m ; [k T 1 , ..., k Tm ]/m Sn ] (4.12
)

A 2 = [[0] (n-1)×m ; [c T 1 , ..., c Tm ]/m Sn ] (4.13)
K D = [B 1 , K 1 , B 2 , C 1 ],
where

K 1 = -M -1 T × K T (4.14) C 1 = -M -1 T × C T (4.15) B 1 = [[0] m×(n-1) , [ k T 1 m T 1 , ..., k Tm m Tm ] T ] (4.16) B 2 = [[0] m×(n-1) , [ c T 1 m T 1 , ..., c Tm m Tm ] T ] (4.17)
Applying Kanai-Tajimi (KT) model [START_REF] Tajimi | A statistical method of determing the maximum response of a building structure during an earthquake[END_REF] to simulate the base acceleration ÿb ,

     ÿf + 2ε f ω f ẏf + ω 2 f y f = -w(t) ÿb = ÿf + w(t) = -(2ε f ω f ẏf + ω 2 f y f ) (4.18)
Here w(t) is zero mean Gaussian white noise, i.e. the excitation to the system. ω f and ε f are the natural frequency and damping ratio of the filter, y f is the relative response. We build the global state space vector of the system by, Z = [y s 1 , . . . , y sn , y T 1 , . . . , y Tm , y f , ẏs 1 , . . . , ẏsn , ẏT 1 , . . . , ẏTm , ẏf ] T .

Then the motion of the system becomes

Ż = AZ + f, (4.19)
where f is a column vector that contains zeros except the last element -w(t). The matrix A is extended from A 0 by considering the state y f . By now, the standard state space is built as

ẋ = Ax + Bu, y = Cx + Du (4.20)
where the input vector x corresponds to Z in eq.(4.19); the output y corresponds to the response of certain DOFs. Obviously, Bu = f, Du = 0, C is a diagonal matrix to decide the output response of interest.

Basic thought of the proposed methodology

The basic thought of the proposed methodology is schematically described in Figure 4.2. In deterministic case where the structural properties are known, the risk of failure can be quantified as the failure probability.

However, the situation changes when uncertainties are associated with the material properties. In the framework of uncertainty, a number of samples are taken from This model can be used to do reliability predictions whenever new observations of the structural properties are available.

Expression of uncertainties

We assume the uncertainties only exist in the mass (m), the damping factor (c) and the stiffness (k) of the primary system. In the present paper, the uncertainties of the parameters adopt the forms of normal distributions [START_REF] Venanzi | Robust optimal design of tuned mass dampers for tall buildings with uncertain parameters[END_REF],

m ∼ N m 0 , σ 2 m , c ∼ N c 0 , σ 2 c , k ∼ N k 0 , σ 2 k .
Here m 0 , c 0 and k 0 are the nominal mass, damping factor and the stiffness, and σ 2 m ,σ 2 c , σ 2 k are the corresponding variances.

Reliability evaluation

First-passage probability In vibration analysis of structural systems subjected to uncertain excitation modeled as stochastic process, it is generally required to estimate the probability that the system's response will stay within safe, prescribed limits, within a specified time span. The determination of such a probability is usually called 'first-passage problem' [START_REF] Roberts | First-passage probabilities for randomly excited systems: diffusion methods[END_REF].

Mathematically, the first-passage problem can be expressed as: to find the probability, P (t), that any one of the response states of interest, Y i (t), i = 1, • • • , m of a system exceeds in magnitude a critical barrier at least once in a defined time interval [0, T ] (see Figure 4.3),

P f = P rob(F ) = P (|Y i (t)| > c, t ∈ [0, T ]) (4.21)
Where F denotes a failure event of the structure. We note that the definition of the failure criterion is of importance to perform reliability estimation. In the framework of first-passage theory, the structure failure probability can be understood as the risk that the structure responses exceed a predefined limit within a certain interval of time. For the proposed seismic model, the Gaussian white noise process is selected as the source of excitation.

MCS for reliability evaluation

In simulation methods, the stochastic excitation is specified as a certain number of input random variables [START_REF] Au | First excursion probabilities for linear systems by very efficient importance sampling[END_REF], x = (x 1 , x 2 , . . . , x n ) which can be simulated as a generation of the excitation. Thus, the output response will be a function of these variables. To apply MCS for reliability evaluation, N samples of the multi-dimensional random vector x will be obtained from a multi-variable Gaussian distribution. Each sample is seen as a white noise process. Obviously, the n variables correspond to n sample of time points in [0, T ]. In terms of joint probability density function (PDF) f x (x) for input random variables, the failure probability can be analytically expressed as the integral,

P f = F f X (x) d x = ... G(X)≤0 f X (x 1 , x 2 , ..., x n )dx 1 dx 2 ...dx n (4.22)
where F : G (x) ≤ 0 is the failure region in n-dimensional space of X. It is practically impossible to carry out direct calculation of this integration. Thus, Monte-Carlo estimation is used as a numeric integration approach,

Pf = E [I F (x)] = 1 N N i=1 I G x (i) ≤ 0 (4.23)
where I F (•) is the indicator function:

I F (x) = 1 if x ∈ F and I F (x) = 0 otherwise;
x (i) is the ith sample of the joint probability distribution.

The failure criterion is defined as the following: y max > y limit , 'failure'; y max ≤ y limit , 'function'. Here, y max = max (| y s |) is the maximum displacement of the response among all the degrees of the base structure. More intuitively, the Monte Carlo estimator of the structure failure probability is written as [START_REF] Yu | Reliability based robust design optimization for tuned mass damper in passive vibration control of deterministic/uncertain structures[END_REF],

Pf = 1 N mc Nmc i=1 I f x (i) (4.24)
where x (i) is the ith sample of the excitation process; N mc is the number of samples used in MCS, and I f (•) is the indicator function that satisfies

I f (•) =     
1, y max > y limit 0, y max ≤ y limit (4.25)

Different ML methods for reliability modeling

Several machine learning models have been explored in our research. Some of them demand prior knowledges about the data while others just focus on the data itself.

During the research, we tested some representative methods on the data, including

Bayesian inference [START_REF] Kypraios | A tutorial introduction to bayesian inference for stochastic epidemic models using approximate bayesian computation[END_REF], k-nearest neighbor (k-NN) [102], Gradient Boosting (GB) [START_REF] Jerome H Friedman | Greedy function approximation: a gradient boosting machine[END_REF] and Random Forest. Some brief introductions of them are listed below.

• Bayesian Inference method is based on Bayes' theorem. It operates by firstly • Gradient Boosting (GB) is a generalization of Adaptive-Boosting (AdaBoost).

It is a kind of ensemble learning method that combines multiple 'weak learners' as a weighted sum so that the final output is determined. The weak learners can be decision trees, decision stumps et.al.

• Random Forest. Even though RF and GB belong to the same category where the decisions are made by combining the outputs of all individual trees, they are largely different in two aspects. Firstly, the way a tree is built. In GB, the trees are built one by one with each tree being built in the background of the previous one; In RF, trees are induced independently, and the randomness in the tree building process helps reduce the correlation among trees. Secondly, the way to make predictions. GB makes predictions by calculating the weighted sum of the decisions of all trees, while RF just averages the decisions.

The exploration of machine learning techniques in structural reliability research is far from enough, and we are the first to introduce RF in reliability analysis. Besides, our research shows that the RF model outperforms the other three models on our data. The following part will introduce RF model in detail.

RF for Reliability modeling

Basic principles of the model Random Forest is an ensemble learning method that use multiple learning algorithms to obtain better predictive performance than could be obtained from any of the constituent learning algorithms alone. RF is comprised of many individual trees called classification and regression tree (CART An underlying assumption of the ensemble process is that the base learners are independent. As the trees become more correlated (less independent), the model error tends to increase. Randomization helps to reduce the correlation among decision trees in order that the model accuracy is improved. Two kinds of randomness exist in the tree learning process.

• Bootstrap sampling Bootstrap sampling is applied to generate a sample set for each tree, which results in about one-third of the samples left out for each bootstrap training set.

• Node splitting For each node splitting, a subset of variables is generated by randomly choosing a certain number of variables from the variable set. Then one of these variables is selected, according to an evaluation criterion, to split the node. Generally, a subset contains sqrt(n) of the attributes.

Tree growing process A tree has a binary recursive structure containing several nodes and branches. Each node specifies a variable and a branch specifies the possible values it takes. A tree learning process is actually the node splitting process (see Figure 4.5), beginning with the root node that corresponds to the whole learning set. Each split results in two subsets of the data that falls into this node so that the resulting child nodes are the 'purest'. As to a regression tree, it is grown starting from the root node by repeating the following steps on each node.

Step1 Find each variable's best split. Randomly choose (n) variables as the candidate variable set. For each candidate variable, sort its values in an ascending order. For the sorted variable, examine all candidate split points to find the one that maximize the goodness of split when the node is split according to it.

Step2 Find the node's best split. Among the best splits found in step1, choose the one that maximizes the splitting criterion.

Step3 Node splitting. Split the node using its best split found in step2 if the stopping rules are not satisfied, otherwise, the node is taken as a leaf whose value is determined by averaging the decisions falling into this node.

In m is the number of samples from the uncertainty space; y is the target variable that has m target values here.

Splitting criterion At each node, we perform a split: we choose a variable x i and its value x i,j that minimizes the sum of squared errors (SSE), i.e.

(y i -c 1 ) 2 + x∈R 2 (y i -c 2 ) 2 ] (4.27)
where R 1 = {x|x i ≤ x i,j } is a the region that satisfies the condition

x i ≤ x i,j , R 2 = {x|x i > x i,j
} is a the region that satisfies x i > x i,j . Besides, c m is the average of the target values falling into the region

c m = E(y|x i ∈ R m ), m = 1, 2.
Definition (impurity) Assuming h(t) is the sample set that falls into node t, a is a sample in h(t), then at node t the impurity is defined as

i(t) = 1 N (t) a∈h(t) (y a - _ y (t)) 2 (4.28)
where N (t) is the number of samples falling into node t, y a is the decision value for sample a , _ y (t) is the average of all decision values in h(t). At node t, the best splitting point s will maximize

∆i(s, t) = i(t) -p 1 i(t 1 ) -p 2 i(t 2 ) (4.29)
where t 1 and t 2 are the two child nodes resulting from the splitting point s. p 1 and p 2 are the portions of samples that fall into t 1 and t 2 respectively. We notice that minimizing the SSE is equivalent to maximizing the decrease of impurity (DI) ∆i(s, t).

Stopping rules The splitting will be stopped if one or more of the following conditions is met: a). the impurity decrement is smaller than the threshold value; b). the current tree depth reaches the preset limit; c). the size of a node is less than the preset minimum size; d). all cases in a node have identical values for each attribute.

Training data preparation

f = 1 k k i=1 f i (x new ) (4.30)
where k is the number of trees within RF, f i (x new ) is the prediction by the ith tree, refer to The red dotted arrow lines constitute the rule in accordance with the values in , so the predicted value for y is 0.0674.

Co -coef = N i=1 (y i -y)(ŷ i -ỹ) N i=1 (y i -y) 2 N i=1 (ŷ i -ỹ) 2 (4.31) RM SE = 1 N N i=1 (ŷ i -y i ) 2 (4.32) RAE = N i=1 |ŷ i -y i | N i=1 |y -y i | (4.33)
where y i , ŷi are respectively the target values and predicted values, and y, ỹ are the corresponding mean values. N is the number of sample cases.

As can be seen, the correlation coefficient between the estimated values and observed values increase drastically when more training data are available. Meanwhile, the two error indices RMSE and RAE decrease clearly before they converge at about 10000 of the data size. The cross-validation results tell that the proposed RF model behaves very well in fitting the data. The test results show that the model gives accurate predictions, so it didn't over-fit the data. This is the evidence that the randomness in the RF model inducing process can effectively avoid over-fittings [START_REF] Breiman | Random forests[END_REF]. Besides, the size of the training data plays an important role in determining the accuracy of RF model.

deviations σ m s1 = σ m s2 = 1 , σ c s1 = σ c s2 = 10 , σ k s1 = σ k s2 = 300 .
The failure criterion is defined as, any degree of the main structure exceeds the assumed threshold value of displacement. Here the threshold is set 1.9 × 10 -2 m. Considering the increase of the number of uncertain properties, the sizes of data sampled from the assumed distributions range from 1000 cases to 29000 cases as the training data.

Another 1000 cases are sampled to test the model performance in prediction ability.

See the simulation results in Figure 4.9. In Figure 4.9, the three evaluation indices change in a similar way to those in Example 1, even though the convergence values of the error indices are higher. This is mainly because when the number of variables increases, the space dimension of the variable becomes higher, thus the uncertainties become more difficult to handle.

Undoubtedly, more observations will improve the model performance.

Parameter influence analysis

Besides the size of the sample that influences the model performance, the hyperparameters in the proposed method may also make contributions to the evaluation results. In this part, the authors consider three factors: the variances of the random variables, the number of trees in the RF model and the number of candidate features for each splitting in tree inducing process.

Variance of the variables

The 1-DOF structure in Example 1 is employed in this section. Considering the intuitivity of standard deviation and its direct relationship with variance, we use standard deviation instead of variance as the measure of dispersion of the samples.

Knowing that there are three parameters of the main structure, i.e. m s , c s , k s , we focus on one parameter and change its standard deviation in the samples. Meanwhile we fix the other two parameters, i.e. keep their standard deviations unchange. Based on the nominal values of the properties of the structure, we range the standard deviation of m s from 1 × 10 -2 to 1 × 10 -1 , with step 1 × 10 -2 . In a similar way, we range the standard deviation of c s from 3 × 10 -4 to 3 × 10 -3 , with step 3 × 10 -4 ;

for k s , the standard deviation ranges from 5 × 10 -1 to 5, with step 5 × 10 -1 . See

M RE = 1 N N i=1 |ŷ i -y i | y i (4.34)
where y i , ŷi are respectively the target values and predicted values. N is the number of samples to test the model. In contrast with RMSE that describes the absolute differences between predicted values and target values, MRE is designed to measure the percentage error of the predictions with respect to the target values. Figure 4.11 and 4.12 show how the model performance is affected by the changing variances of the uncertain properties. we know that with the increase of trees, the RF model tends to be more accurate.

SD1 SD2 SD3 SD4 SD5 SD6 SD7 SD8 SD9 SD10 2.5 3 3.5 4 4.5 5 5.5 6 6.5 RMSE 10 -3 CV
-m Test-m CV-c Test-c CV-k Test-k
Moreover, the RMSE begins to converge when the number of trees exceeds 120.

As the computational cost increases linearly with k, k = 120 is a good choice to make a trade-off between computational complexity and accuracy [START_REF] Hohenbichler | New light on first-and second-order reliability methods[END_REF]. This is a direction to determine the value of k. Therefore, based on the available data, the best k should be the minimum number achieving the accuracy level with a reasonable computational cost. It is also found that increasing the number of trees effectively reduces RAE, meanwhile increasing the correlation coefficient.

In the same simulation, we consider the influence of the number of candidate features and change it in each splitting procedure. Considering that there are 9 uncertain attributes to process, we range the number of candidate features from 2 to 5, and see the influences on different evaluation indices. See results in Figure 4.13. The simulation results show that the model accuracy reaches its highest when the number of features is taken as 4. This result is close to the conclusion in Leo Breiman [START_REF] Breiman | Random forests[END_REF].

This simulation shows the evidence that the number of candidate features is another key point to consider in RF model selections.

Feature importance analysis

Feature importance is a way to quantize the contributions of the features(variables)

to the overall model's predictive performance. For high-dimension problems, it helps filter the variables that have little influence on the response so that the computational efficiency is improved. The feature importance is reflected by the change in OOB error for each feature when that feature is randomly permuted in the OOB observations. The higher the difference, the more important a feature is, vice versa. This is called out-of-bag feature importance. We firstly introduce the concept of OOB error. To build a RF, bootstrap sampling is carried out to generate a certain number of bootstrap datasets, each of which has a corresponding OOB dataset. For each observation X i (i = 1, 2, ..., N ) in the training set, we estimate its out-of-bag prediction by averaging over the predictions of the trees (in the ensemble) that take

X i as out-of-bag data. That is, F (X i ) = 1 N i X i ∈S i f (X i ; T i,j ), (4.35)
where S i = {S i,j } N i j=1 denotes all the Bootstrap datasets that take X i as out-of-bag data. S i,j is the jth(j = 1, 2, ..., N i ) dataset within S i . T i,j is the tree grown from S i,j . f (X i ; T i,j ) is the predicted target value for X i by the tree T i,j . The out-of-bag error for X i can be determined by comparing the predicted target value with the true target value in a square error sense, i.e.

OOBE(X

i ) = (F (X i ) -y i ) 2 (4.36)
Then the OOBE for the RF model is calculated by taking the root average of OOBEs of all the observations in the training set. In order to calculate feature importances, the procedures are carried out on each tree in the RF. We denote {T i } k i=1 as the RF that consists of k random trees and S D N ,θ i as the OOB dataset corresponding to T i . Firstly, we calculate the root average value of OOBEs of the observations within

S D N ,θ i , that is ε T i = mean[OOBE(X)] given that X ∈ S D N ,θ i .
Then the values of the jth variable x j (j = 1, ..., n) in the OOB data are randomly permuted, meanwhile keeping other variable values unchanged. In a similar way, we use the altered OOB observations to calculate the new OOB error ε j,T i , see the procedures in Figure 4.14.

Generally, ε j,T i should be larger than ε T i . We calculate the difference between the two OOB errors, i.e. d j,T i = ε j,T i -ε T i , which can be seen as the importance index of x j via the tree T i . We repeat the procedures in Figure 4.14 by each tree in the forest, finally we obtain a matrix that consists of the feature importance indices induced by all trees in the RF, denoted below,

F I =             d 1,T 1 d 2,T 1 ... d n,T 1 d 1,T 2 d 2,T 2 ... d n,T 2 ... ... ... ... d 1,T k d 2,T k ... d n,T k             (4.37)
where k is the number of trees in the RF, and n is the number of features in the training set. Accordingly, the importance measure of x i is obtained as the average F I value over all trees, i.e.,

F I_i = ave{d i,T 1 , d i,T 2 , ..., d i,T k } (4.38)
the standard deviation (SD) of the estimation of F I of variable x i (i = 1, ..., n) is computed as

SD(F I_i) = var{d i,T 1 , d i,T 2 , ..., d i,T k } (4.39)
To make comparison more intuitive, all variable importances are scaled in order that they sum up to 1. We consider the same dataset analyzed in section 4. The feature importances in Figure 4.15 illustrate that the mass of each degree of the main structure are the principle elements to determine the model outputs. In contrast, the stiffness and damping coefficients are much less influential especially for the second and third degrees.

Comparisons with other machine learning methods

To compare the RF model with other models, three representative machine learning methods are selected, i.e. Bayesian inference method, k-NN method and GB method. We take the three different structures in section 4.1 as the object struc- The evaluation indices in Table 4.5 tell that the RF model behaves very well in reliability prediction compared with the other three models. For the single DOF structure, even though GB model behaves the best, the RF model achieved almost the same accuracy (only 5 × 10 -4 in difference). For the 2-DOF and 3-DOF structures, the RF model gives the most accurate predictions compared with the other three. Bayes method give the lowest accuracy in reliability prediction of all the three structures. Generally, Bayesian inference method requires prior knowledge or assumptions on the data distributions while k-NN does not. This makes k-NN tend to be more accurate than Bayes method when no prior information is available.

The GB method provides close, though less accurate, predictions to those by RF.

As an ensemble learning method, GB is different from RF in the aspect that it combines multiple 'weak learners' as a weighted sum to make predictions. It builds one tree at a time, and each new tree is built to reduce the errors of the previously built tree. The way each tree is built makes this model prone to overfit the data. This is one of the reasons that GB is less accurate than RF in some situations.

An illustrative example of the prediction results

This section provides the training and prediction results of the RF model based on the data employed in section 4.2.2. Some parameter configurations are listed in Table 4.6. Besides, the PSD takes 0.031, the displacement threshold takes 2.5 × 10 -2 m, and the first-passage time span is 20s. In order to present the simulation results more visibly, 100 samples are randomly selected from the training result and another 50 samples are obtained in the same way from the test result. The RF contains 160 trees, and the number of candidate features is set as 4. enough, but the RAE values is a little higher than those for 1-DOF structures.

Index m s i c s i k s i m T j c T j k T j
The increase of the predicting error is due to the weak concept of local neighborhood as the input space dimensionality grows [START_REF] Xu | Application of neural networks in forecasting engine systems reliability[END_REF], but more samples and suitable configurations of the RF model can help reduce the error.

Two case studies on multi-DOF structural system

In the subsections 4.4.1, the reliability analysis of three different DOF structures (i.e., 1-DOF, 2-DOF and 3-DOF) have been studied with the aide of Random Forest. It is noticed that the three structures do not have high-dimension uncertain structural parameters. Random Forest is known to be powerful in handling highdimension uncertainties, thus in this subsection, the multi-DOF structural system is studied. Two simulations are carried out in this subsection. In the first simulation, the object structure is multi-DOF one introduced from Elyes et.al [START_REF] Mrabet | New approaches in reliability based optimization of tuned mass damper in presence of uncertain bounded parameters[END_REF]. The RF performance is analyzed and compared with standard MCS results. In the second simulation, the RF model is tested on a famous benchmark problem introduced by G.I. Schueller [START_REF] Gerhart | Benchmark study on reliability estimation in higher dimensions of structural systems-an overview[END_REF]. It is also compared with standard MCS, Subset simulations (3 variants) [START_REF] Sk Au | Application of subset simulation methods to reliability benchmark problems[END_REF], complex modal analysis (CMA) [START_REF] Hector | Reliability analysis of linear dynamical systems using approximate representations of performance functions[END_REF], spherical subset simulation (S 3 ) [START_REF] Ls Katafygiotis | Application of spherical subset simulation method and auxiliary domain method on a benchmark reliability study[END_REF], auxilary domain method (AMD) [START_REF] Katafygiotis | Auxiliary domain method for solving multi-objective dynamic reliability problems for nonlinear structures[END_REF] and line sampling [START_REF] Hj Pradlwarter | Application of line sampling simulation method to reliability benchmark problems[END_REF].

TMD-based ten-DOF shear building

The system is a 10-storey shear building, excited by earthquakes, see Figure 4.17.

Different from Figure 3.14 where the studied structure has deterministic properties, the structure in Figure 4.17 has uncertainties in its mass, stiffness and damping indices (except for the TMD structure). In subsection 3.4.6, this structure has been employed to compare the efficiency of KL-IS method with standard MCS method.

However, this part devotes itself to conditional failure probability modeling by Random Forest as well as the overall failure probability considering uncertainties both in structural properties and in ground excitations. The structural parameters have nominal values that are the same as those in Figure 3.8 [START_REF] Mrabet | New approaches in reliability based optimization of tuned mass damper in presence of uncertain bounded parameters[END_REF]. Besides, the parameters of the TMD (also the KT model) are deterministic and the same to those in Figure 3.14. The nominal values and their uncertainties are listed in From Table 4.9, it is found that the estimated failure probability Pf,RF by RF is close regression trees is trained. Each tree in the RF has the parameter maxFeature= 5.

The estimated failure probabilities (as well as their standard deviations) resulting from different y lim values are listed in Table 4.10. The reference values calculated by standard MCS are also provided to make comparisons. In principle, the failure probability of the object structure decreases when the threshold y lim increases. In As a benchmark problem introduced in G.I. Schueller [START_REF] Gerhart | Benchmark study on reliability estimation in higher dimensions of structural systems-an overview[END_REF], the ten-DOF Duffing type oscillator has been widely used by researchers in structural reliability domain.

In this study, we focus on the linear random structures under stochastic excitations.

The governing equation is given by

Mÿ(t) + C ẏ(t) + Ky(t) = F(t) (4.40)
where M, C, K are the mass matrix, damping matrix and stiffness matrix respectively. y is the structural response; F(t) = u(t)[m 1 , m 2 , ..., m 10 ] T is the force on the structure. For more details about the dynamic process, refer to section 4.2. The statistical properties of the structural parameters and their constraints are listed in

p(t) = Ω 2 1g v f 1 (t) + 2ζ 1g Ω 1g v f 2 (t) -Ω 2 2g v f 3 (t) -2ζ 2g Ω 2g v f 4 (t) (4.41)
where the state space function with respect to the state vector

v f (t) = [v f 1 (t), v f 2 (t), v f 3 (t), v f 4 (t)] T of the filter is, vf (t) =             0 1 0 0 -Ω 2 1g -2ζ 1g Ω 1g 0 0 0 0 0 1 Ω 2 1g 2ζ 1g Ω 1g -Ω 2 2g -2ζ 2g Ω 2g             v f (t) + [0, w(t), 0, 0] T (4.42)
Here, w(t) stands for a modulated Gaussian white noise with auto-correlation function E(w(t)w(t + τ )) = Iδ(τ )h 2 (t) and I denotes the intensity function of the white noise. h(t) has the following form, The failure criterion is defined by the maximum relative displacements between two consecutive DOFs over the time interval [0.0s, 20.0s]. The sampling time step is set as ∆t = 0.05s. The failure probabilities are calculated for different threshold values listed in Table 4.12. We are interested in the first excursion probability that the relative maximum displacement of the first DOF is greater than the threshold 0.057m and 0.073m; besides, the probability that the relative maximum displacement between the 9th DOF and 10th DOF exceeds the threshold 0.013m and 0.017m is considered. In this simulation, the estimated failure probability values are very small (below 10 -4 ), therefore KL-IS method is employed to calculate the conditional failure probabilities. In applying K-L expansions, 3000 samples of the filtered white noise processes are generated from the filter, then the eigenvalue problem is solved (refer to eq. (3.11)). The number of K-L terms kept is n KL = 300. The relative response of each DOF can be directly calculated when the structure experiences impulse response. Figure 4.20 shows the relative responses assuming that the structure is having its nominal configurations.

h(t) =                      0, t = 0, t/
In the framework of MCS, 300 samples of the structural properties are analyzed, resulting in 300 conditional failure probabilities. A RF model with 30 rgression trees is then trained from the training set that consists of 300 cases and 3 × 10 = 30 variables in each case. For each threshold value in Table 4.12, similar Monte-carlo procedures are carried out. Once a RF model is trained, it is employed to make predictions on a newly generated data set that contains 2000 random samples of the structural properties. By eq.(3.5), the overall failure probability is estimated that is concerned with both structural uncertainties and excitation uncertainties. As already mentioned before, the proposed method is compared with other methods (the results are provided by those authors), see the results in Table 4.13.

A general conclusion drawn from Table 4.13 is that, a number of methods and their variants exist by which high-dimensional reliability problems with respect to linear or nonlinear stochastic dynamics can be solved very efficiently when compared to standard Monte Carlo. It is noticed that the estimated probabilities are very small (e.g. P f = 10 -6 ), which implies that standard Monte Carlo is practically not 4.13, it is found that the number of samples analyzed in the proposed method is very small (e.g. 300), but the finally estimated failure probabilities are highly accurate. This result tells that, RF model is powerful in handling high-dimension reliability problems.

Chapter summary

In this chapter, the Random Forest algorithm is investigated in reliability model- Moreover, the feature importance analysis tell that the mass is the most important features in reliability modeling. The comparisons between RF and other machine learning methods shows the evidence that the RF model is an alternative way to study structural reliability.

takes the majority/average as the final prediction. Boosting updates weight distribution in each round, and learns base models accordingly, then combines them according to their corresponding accuracy. Different from these two approaches, Stacking [41] learns a high-level model on top of the base models (classifier/regressor). It can be regarded as a meta-learning approach in which the base models are called firstlevel models and a second-level model is learnt from the outputs of the first-level models. A short description of all three methods is introduced below.

• Bagging method generally builds several instances of a black-box estimator from bootstrap replicates of the original training set and then aggregates their individual predictions to form a final prediction. This method is employed as a way to reduce the variance of a base estimator (e.g., a decision tree) by introducing randomization into its construction process.

• Boosting is a widely used ensemble approach, which can effectively boost a for an illustration. First, all of the other algorithms are trained using the available data, then a combiner algorithm is trained to make a final prediction using all the predictions of the other algorithms as additional inputs. Stacking typically yields performance better than any single trained models [41]. It has been successfully applied on both regression [START_REF] Breiman | Stacked regressions[END_REF] and classification. It was reported to outperform Bayesian model-averaging [START_REF] Clarke | Comparing bayes model averaging and stacking when model approximation error cannot be ignored[END_REF].

To understand why the generalization ability of an ensemble is usually much stronger than that of a single learner, Dietterich [START_REF] Tg Ditterrich | Machine learning research: four current direction[END_REF] gave three reasons by viewing the na-ture of machine learning as searching a hypothesis space for the most accurate hypothesis. The first reason is that, the training data might not provide sufficient information for choosing a single best learner. For example, there may be many learners perform equally well on the training data set. Thus, combining these learners may be a better choice. The second reason is that, the search processes of the learning algorithms might be imperfect. For example, even if there exists a unique best hypothesis, it might be difficult to achieve since running the algorithms result in sub-optimal hypotheses. Thus, ensembles can compensate for such imperfect search processes. The third reason is that, the hypothesis space being searched might not contain the true target function, while ensembles can give some good approximation. There are many theoretical studies on ensemble methods, yet it is far from a clear understanding of their underlying mechanisms [START_REF] Thomas G Dietterich | Ensemble learning. The handbook of brain theory and neural networks[END_REF]. For example, empirical observations show that Boosting often does not suffer from overfitting even after a large number of rounds, and sometimes it is even able to reduce the generalization error after the training error has already reached zero. The bias-variance decomposition is often used in studying the performance of ensemble methods. It is known that Bagging can significantly reduce the variance, so it is better to be applied to learners (such as decision trees) suffered from large variance. Boosting can significantly reduce the bias in addition to reducing the variance.

Principle of Stacking

Stacking is an ensemble learning technique that learns a meta-learner based on the ). We construct a corresponding example (x i , y i ) in the new data set, where x i = h (h 1 (x i ), h 2 (x i ), ..., h T (x i )). Thirdly, learn a second-level (meta) learner from the newly constructed data. Any learning method could he applied to learn the meta learner.

General Stacking procedures

Table 5.1: Pseudo-code for Stacking [START_REF] Charu | Data classification: algorithms and applications[END_REF] Input: Training data D = {x i , y i } m i=1 , x i ∈ X, y i ∈ y; a Stacking algorithm.

Output: A meta learner H.

Step1: Induce T base learners, i.e. h 1 , h 2 , ..., h T , from the training set.

Step2: Construct a new dataset D , where D = {(x i , y i )} m i=1 . Here

x' i = [h 1 (x i ), h 2 (x i ), ..., h T (x i )].
Step3: Build a meta-learner H from D .

Once the second-level learner is generated, for an test example x, its predictions are h (h 1 (x), h 2 (x), ..., h T (x)), where (h 1 , h 2 , ..., h T ) are first-level learner and h is the second-level learner. We can see that Stacking is a general framework. We can plug in different learning approaches or even ensemble approaches to generate first or second level learner. Compared with Bagging and Boosting, Stacking "learns" how to combine the base learner instead of voting.

Cross-Validations in Stacking

In Table 5. The models RF, ETs and GB are all tree-based methods, but they have different principles. The combination of them helps offset their weakness and improve the prediction accuracy. Interestingly, the model Stacking1 does not achieve better performance than GB when the size of training data is less than 6000. One reason is that GB is not a base model of Stacking1. If GB is added into Stacking1, resulting in Stacking2, the performance will be largely improved. This phenomenon also indicates that the choice of base models is important. In view of this, section 5.2 is devoted to the choice of base models.

Choice of base models and meta-model

As is shown in Section 5.1, the Stacking models generally achieve better performance than the single base learners. We now assume that the Stacking model has two base learners chosen from the set RF, GB, ETs, and the meta-learner as the remaining one from this set, for example RF, GB as the base learners and ETs as the metalearner. Besides, we take SVR(Support Vector Regression) as a candidate of the meta-learner. The same datasets (as in Section 4.1) are employed to build the Stacking models. The results are shown in Figure 5.5.

As can be seen from Figure 5.5, the choice of base learners makes difference. For the meta-model(if it is not SVR), maxFeatures=2. when the SVR model is employed as the third base learner, the Stacking performance largely improves. As an example, when the SVR is added as a third base learner to 'GB&ETs-RF', the RMSE is reduced much at any point. However, this does not mean that adding more base learners always result in better predictions, see a special case in Figure 5.7.

From Figure 5.7, it is seen that the blue solid line has much larger error than the red dashed line. Actually this is not the only case, there are several other cases that have is found that when GB model is taken as the meta-learner, the model performance will always be improved. For example, in Figure 5.5, the model 'RF&ETs-GB' achieves higher accuracy than 'RF&ETs-SVR'; in Figure 5.8, the model 'SVR&RF-GB' and 'SVR&ETs-GB' achieves the highest accuracy among the Stacking models explored; in Figure 5.6, the model 'RF&ETs&SVR-GB' is also among the most accurate models, but it consumes more time than the models 'SVR&RF-GB' and 'SVR&ETs-GB'. As we know, GB model is good at reducing bias. This is a reason why the Stacking models having GB as the meta-learner always result in higher accuracy. See the simulation results in Figure 5.9.

In Figure 5.9, it is found that the number of random trees in RF has little effect on the performance of the Stacking model. In contrast, the number of trees in GB model has great influence on the Stacking performance (see Figure 5.10). Besides, the RMSE value converges when the number of trees in GB is 50. For SVR model as one of the base learners, the kernel function is an important parameter. In this aspect, four kernel types are examined, i.e. 'Linear', 'Polynomial', 'Radial basis function (rbf)', 'Sigmoid'. In Figure 5.11, it is found that the Stacking model reaches its best performance when the kernel function of SVR is 'rbf'.

As is known that each random tree in RF (or GB) is trained in a recursive, binary splitting way. For each splitting, a random subset of features in the training data is sampled. The number of features is an important index that determines the model performance. In Figure 5.12, the RMSE is plotted according to different sizes of this subset. Obviously, the number of sampled features for each splitting has little effect on the prediction accuracy, because the RMSE value changes very small even though it has a trend of decrease. In contrast, for the mete-model GB, the number of sampled features for each splitting plays an important role. As the Stacking model 'SVR&RF-GB' has two base learners, the training data for the meta-model only has two features. From Figure 5.13, it is clear that the RMSE of the model reaches its lowest when the number of features sampled is n = 2.

Stacking models always outperforms the individual models, and their performance can be further improved by adjusting the hyper-parameters. But the intrinsic causes

Bias-variance analysis of Stacking model

Bias-variance decomposition [START_REF] Frahm | Device for damping vibrations of bodies[END_REF] is an important general tool for analyzing the performance of learning algorithms. Given a learning target and the size of training set, it divides the generalization error of a learner into three components, i.e., intrinsic noise, squared bias and variance. For a regression problem where f is the unknown target function to be learned, f maps elements in an input space X into numeric values. Let D be a probability distribution over X such that a random example,

x ∈ X, is drawn with probability D(x). Let L = {(x, f (x) + ε)|x ∈ X} be a learning set of m cases. For each case, the input x is drawn from D and the output value y = f (x) + ε is corrupted by Gaussian noise with zero mean and standard deviation σ. From L, an algorithm A will output an hypothesis h L . For a given test point x 0 , the predicted output is ŷ = h L (x 0 ).

E L,ε [(y -ŷ) 2 ] = E L,ε [(y -h) + (h -ŷ)] 2 = E L,ε [(y -h) 2 + (h -ŷ) 2 + 2(y -h)(h -ŷ)] = E L,ε [(y -h) 2 + (h -ŷ) 2 ] + 2E L,ε [y(h -ŷ) -h(h -ŷ)] = E L,ε [(y -h) 2 ] + E L,ε [(h -ŷ) 2 ]
(5.1)

The term

E L,ε [y(h -ŷ) -h(h -ŷ)] = 0, because E L,ε [y(h -ŷ)] = E L,ε [f (x) + ε] • [E L [h L (x)] -h L (x)] = E ε [f (x) + ε] • E L E L [h L (x)] -h L (x) = E ε [f (x) + ε] • E L [h L (x)] -E L [h L (x)] = 0 (5.2) E L,ε [h(h -ŷ)] = E L,ε E L [h L (x)] • [E L [h L (x)] -h L (x)] = E L [h L (x)] • E L E L [h L (x)] -h L (x) = E L [h L (x)] • E L [h L (x)] -E L [h L (x)] = 0 (5.3)
The term E L,ε [(y -h) 2 ] is further decomposed as follows,

E L,ε [(y -h) 2 ] = E L,ε (y 2 -2yh + h 2 ) = E L,ε (y 2 ) -2E L,ε (yh) + E L,ε (h 2 ) = E ε (y 2 ) -2hE ε (y) + h 2 = E ε [y -f (x)] 2 + f 2 (x) -2hf (x) + h 2 = E ε (ε 2 ) + [h -f (x)] 2 = σ 2 + [h -f (x)] 2 (5.4)
Therefore,

E L,ε [(y -ŷ) 2 ] = σ 2 + [h -f (x)] 2 + E L [(h -ŷ) 2 ] = noise 2 + bias 2 (ŷ) + var(ŷ) (5.5)
The squared noise describes how much y varies from f (x). It is due to the problem at hand and cannot be avoided. It is the lower bound of the expected error of any learning algorithm on the target. The squared bias is the difference between the average estimate and the target. The variance describes the magnitude of the fluctuation of the estimate (for different training sets of the same size). Since the noise is intrinsic and difficult to estimate, it is often subsumed into the bias term [START_REF] Zhou | Ensemble methods: foundations and algorithms[END_REF]. Thus, the generalization error is broken into bias and variance terms, i.e.

E L,ε [(y -ŷ) 2 ] = (h -y) 2 + E L [(h -ŷ) 2 ] = bias 2 [ŷ] + var[ŷ] (5.6)
Generally, if the model is 'too global', then its bias will be high, the model is underfitting the data; If the model is 'too local', then its variances will be high, the model is over-fitting the data. In practice, we have only one dataset to train the model.

To measure bias and variance, bootstrap sampling is generally applied to simulate multiple training sets. The pseudo-code is shown in Table 5.4. Output: Bias and variance of the model.

Step1: Generate n bootstrap replicates of D, i.e. {D i } n i=1 .

Step2: Apply learning algorithm to each D i and obtain an hypothesis h i .

Step3: Let T i be the OOB points for D i , compute predicted values h i (x) for

x i ∈ T i , By now, each x i ∈ D has several predictions.

Step4: For x i , estimate the bias and variance according to eq.(5.6).

Step5: Output the averaged results from Step4 over all points in D.

The bias-variance analysis is applied on the model 'SVR&RF-GB' which has base models SVR, RF and meta-model GB. The training set is taken from section 5.5. to high bias and low variance; when K becomes larger, random effects are weaker, resulting in low bias but high variance. For all three models, too small K leads to a too large bias, for which the variance cannot compensate. For RF, the optimal trade-off is at K = 5; for GB, K = 4; for ETs, K = 6. data to train each tree. Therefore, the time complexity to build a RF is [41],

T RF = O(M 1 * K 1 * (0.632 * 0.9N )log 2 (0.632 * 0.9N )) = 0.57 * O(M 1 * K 1 * N log 2 (0.57N )) = 0.57M 1 K 1 * O(N log 2 N ) (5.7)
Then, 10% of the training data L 1 is used to make predictions. The time complexity to make predictions is To make it clearer, the time complexities of the models are collected in Table 5.6. (5.17)

Therefore, the total time complexity is denoted as (notice that N ≈ 0.632N 1 )

T = T 1 + T 2 = (5.7M 1 + 9M 3 -9)K 1 * O(N log 2 N ) + (M 4 -1)K 2 * O(N log 2 N ) = (5.7M 1 K 1 + 9M 3 K 1 ) -9K 1 + M 4 K 2 -K 2 ) * O(N log 2 N ) (5.18)
To conclude, for the Stacking2 model, the total time complexity is denoted as

T = (2.5M 1 K 1 + 4M 3 K 1 -4K 1 + M 4 K 2 -K 2 ) × O(N log 2 N ) (5. 19
)
where N is the size of the original training set. We see that the time complexity T calculated above approximates a linear relationship with respect to N . Now we have calculated the time complexity of Stacking2 model. For Stacking1 model, the time complexity can be calculated in a similar way. Figure 5.17 It is noticed that only one threshold value is studied in Table 5.7. In contrast, multi-threshold values are considered here. Firstly, the failure criterion is defined as y 1 lim = 0.057m ∪ y 10 lim = 0.013m, which means that the failure takes place when the 1st DOF exceeds 0.057m or the 10th DOF exceeds 0.013m; Secondly, in a similar way, the failure criterion is defined as y 10 lim = 0.073m ∪ y 10 lim = 0.017m. Then the failure estimations by the two methods are calculated, see Table 5.8. It is found that, in these two settings, the two methods result in similar failure probability estimations. num of samples 800 1100

NonGaussian structural properties

As is known that all the simulations before are based on the assumption of Gaussian random structural parameters. In this part, non-Gaussian variables are studied.

Three situations are discussed here. 1) Lognormal: all random parameters follow lognormal distributions; 2) Gamma: all random parameters follow Gamma distributions; 3) Mixed: the first 5 DOFs follow Lognormal distributions while the last 5

DOFs follow Gamma distributions. Table 5.9 shows the results of failure probability estimations in the first situation. The 1st DOF or 10th DOF are considered to define the failures. It is found in Table 5.9 that, both methods achieve relatively high accurate estimations of failure probability. In the first case where y 1 lim = 0.057cm, the Stacking method behaves better than RF; while in the second case where y 10 lim = 0.013cm, both methods result in estimations very close to the reference values estimated by standard MCS. In the second situation, all parameters follow Gamma distributions. The same thresholds are studied as those in Table 5.9. We find that the two methods all give very close results to the refered values. In Table 5.11, the types of parameter distributions are mixed. The two methods still give high accurate estimation results. num of samples 1000 900

Chapter summary

In this chapter, Stacking method is explored for structural reliability modeling. The

Stacking method builds the model in a hierarchical way, combining different base learners to produce a meta-learner that generally outperforms any of the base learners. Numerical simulations are performed from various perspectives such that the proposed model is examined in a systematic way. The simulation results show that, with a price of a limited CPU time increment, the Stacking model obtains a significant advantage over its base learners in terms of structural reliability predictions.

• Conditional failure probability is well defined and studied. By considering the uncertainties both in the ground excitations and the structure properties, the overall failure probability is actually a high-dimension (generally several hundred) integral that is analytically impossible. By exploring the relationship between deterministic analysis and uncertainty analysis, we found that the uncertainty analysis can be approximated by a limited number of deterministic analysis which belong to a certain probability density distribution. The conditional failure probability is defined by assuming that the structural is deterministic, i.e. a sample of the uncertain structural properties is obtained from their joint probability density distribution, and then a failure probability is calculated for this 'deterministic' structure. The conditional failure probability is denoted as a high-dimension integral according to the distribution of all possible deterministic analysis based on the sample space of the excitation process. This task can be accomplished in the theoretic framework Monte-carlo simulations (MCS).

• The KL-IS method is employed to handle very small conditional probabilities. To overcome the shortcomings of standard MCS in approximating very small failure probabilities (for example, less than 1%), the KL-IS method is introduced that is much more efficient than the general MCS. In this method, the Karhunen-Loeve expansion is used to decompose the Gaussian excitation process into deterministic part (i.e. the K-L vectors) and uncertain part (i.e. multi-dimension standard normal vector). Under the assumption that the structure is 'deterministic', the structural response can be directly calculated as the convolution integral of its impulse responses and the excitation process that it is subjected to. Besides, to improve the sampling efficiency, an importance sampling technic has been detailed and used. The KL-IS scheme has been compared with the standard MCS and showed its overwhelming advantage in small failure probability estimations.

• Random Forest model has been used to predict conditional failure probabilities. In machine learning perspective, the estimation of conditional failure probability is related to the regression problem. Random Forest (RF) is a powerful machine learning tool to accomplish this task. To learn a RF model, • In this research, we have successfully introduced machine learning theories into structural reliability assessment domain. Various other machine learning tools that are designed for regression problems can be readily used.

a

Future work

The following issues need further research.

• In this research, a basic assumption is that the object structure is a linear system so that the KL-IS method can be applied. However, for non-linear structures, this method may not be well suited. Although linearization techniques can be used to convert nonlinear problems into linear problems, sometimes strong nonlinearity cannot be ignored. Therefore, extension of this method into non-linear domain is an important direction in the future research.

• Correlated random variables have not been studied. The stochastic excitations are modeled as filtered Gaussian white noise process, which means that the excitation values at different time points are relatively independent. However, in reality there may be some correlations between these excitations values. Similar situations happen to the uncertain structural properties that are assumed relatively independent by now. Extension of this assumption into correlated random variables will be meaningful in future research.

• The parameter tuning of Random Forest still needs more study. From the simulation results, it is known that, according to different parameter configurations the Random Forest model has clearly different performances. The parameters to be tuned constitute a 'random vector' that may include the number of trees, the number of features randomly sampled on each node, the depth of the trees and the scale of the sum errors within the leaf nodes.

Optimization methods may be considered to accomplish such a task.

• For the Stacking model, except for the parameter tuning problem introduced in Random Forest, the choice of base models and meta-models is still not of high efficiency. This makes the Stacking model consume more time as various base models are involved in the learning process. The organization of these different level models needs further study.

List of abbreviations

Figure 1

 1 Figure 1.1.

Figure 1 . 1 :

 11 Figure 1.1: Classification of uncertainties [[17], [18]]

Figure 1 . 2 :

 12 Figure 1.2: Space of two random variable (r, s) and their joint density function f RS (r, s);their marginal density functions f R and f S ; the failure domain D.[START_REF] Robert | Structural reliability analysis and prediction[END_REF]

Figure 1 . 4 :

 14 Figure 1.4: Evolution from physical model to surrogate model

Figure 1 . 5 :

 15 Figure 1.5: Basic procedures to build a surrogate model

Figure 1 . 6 :

 16 Figure 1.6: Basic procedure of engineering structural reliability analysis

Figure 2 . 1 :

 21 Figure 2.1: Safe and failure regions

Figure 2 . 3 :

 23 Figure 2.3: Graphical representation of the linearization of the limit-state function around the design point at the basis of the FORM estimation of P f . ([43])

Figure 2 . 4 :

 24 Figure 2.4: Some descriptions of failure probability convergence by MCS

Figure 2 .

 2 5 shows the convergence (with 95% confidence interval, or CI in short) of Pf with respect to different size of sample set in the MCS, and Figure2.6 shows the corresponding coefficient of variation of Pf .

Figure 2 . 6 :

 26 Figure 2.6: Coefficient of variation of the estimated P f

  .7). New examples are then mapped into that same space and predicted to belong to a class based on which side of the margin they fall. This hyper-plane generally takes the form f (x) = w • x + b. It is constructed by solving the optimization problem below, min w,b

Figure 2 . 9 :Figure 2 . 10 :

 29210 Figure 2.8: SVM for regression

Figure 2 . 11 :

 211 Figure 2.11: Sigmoid function

Figure 2 .

 2 Figure 2.12: k-NN for predictions

Figure 2 . 13 :Figure 2 . 14 :

 213214 Figure 2.13: Process of bootstrap aggregating

Figure 2 . 15 :

 215 Figure 2.15: Example classification tree

Figure 2 .Figure 2 . 16 :

 2216 Figure 2.15 shows a classification tree induced from a gender data. The gender is judged by the height and weight of the person, for example, the gender is male when

Figure 3 . 1 :

 31 Figure 3.1: Illustration of the general framework for uncertainty quantification[START_REF] Sudret | Uncertainty propagation and sensitivity analysis in mechanical models-contributions to structural reliability and stochastic spectral methods[END_REF]

Figure 3 . 2 :

 32 Figure 3.2: Classification of uncertainty propagation methods[START_REF] Sudret | Uncertainty propagation and sensitivity analysis in mechanical models-contributions to structural reliability and stochastic spectral methods[END_REF]

Figure 3 . 4 :

 34 Figure 3.4: Structural responses induced by different random excitations

Figure 3 . 5 :

 35 Figure 3.5: Numerical representations of the process in Figure 3.4

. 14 )Figure 3 . 6 :

 1436 Figure 3.6: The nominal output

Figure 3 Figure 3 . 7 :

 337 Figure 3.7: Ten sampled outputs according to the uncertainties of the coefficients

Figure 3 . 8 :Figure 3 . 9 :Figure 3 . 10 :Figure 3 . 11 :

 3839310311 Figure 3.8: The first five eigenvalues of the Fredholm integral equation of the second kind

  .26) where | • | denotes absolute value. Taking into account the definition of performance function introduced in Eq.(3.26), the first excursion probability is defined by means of a multidimensional integral:P f = g(x,u)≤0 p x (x)p u (u)dxdu = x∈Ω θ ,u∈Ωu)≤0 I F (x, u)p θ (θ)p u (u)dxdu (3.27) where I F (•) denotes the indicator function which is equal to 1 in case the performance function g(•) ≤ 0 and 0 otherwise. Note the definition in eq.(3.27) allows calculating the failure probability considering uncertainty in both structural parameters and excitation. Actually, for a particular realization of the uncertain structural parameters x, the conditional failure probability is denoted as

  650 × 10 6 N/m, and c i = 6.2 × 10 6 N s/m[START_REF] Mrabet | New approaches in reliability based optimization of tuned mass damper in presence of uncertain bounded parameters[END_REF], where i = 1, ..., 10. The natural frequency and damping ratio of the TMD are 2πrad/s and 3% respectively. The observation time period is [0, 20s], with time step T s = 0.05s. The threshold values for each DOF are set the same, i.e. y lim = 0.025m. Noticed that the threshold here denotes the relative response between two consecutive DOFs.

Figure 3 . 14 :

 314 Figure3.14: Ten storey shear building under earthquake excitations[START_REF] Mrabet | New approaches in reliability based optimization of tuned mass damper in presence of uncertain bounded parameters[END_REF]

Figure 3 .Figure 3 . 15 :

 3315 Figure 3.15: Evolution of CPU time according to different values of P f (by standard MCS)

Figure 3 . 16 :

 316 Figure 3.16: Evolution of CPU time according to different values of P f (by KL-IS)

Figure 3 . 17 :Figure 3 . 18 :

 317318 Figure 3.17: Impulse (absolute) response of different DOFs

Figure 3 . 19 :

 319 Figure 3.19: Evolution of failure probabilities of different DOFs

(4. 5 )

 5 By assuming the state space y = [y S 1 , ..., y Sn , y T 1 , ..., y Tn ], we express the motion of the system as Mÿ(t) + C ẏ(t) + Ky(t) = -Mrÿ b (4.6) where r = [1, 1, ..., 1] T consists of (n + m) elements. The mass matrix is M = diag[m S 1 , ..., m Sn , m T 1 , ..., m Tm ] = diag[M S , M T ]. Here, M S = diag[m S 1 , ..., m Sn ], M T = diag[m T 1 , ..., m Tn ]. The damping matrix C is a square matrix whose non-zero elements are

(4. 7 )

 7 Similar to M, C = diag[C S , C T ]. Here, C S consists of the first n rows and the first n columns of C; C T consists of the last m rows and the last m columns of C. The stiffness matrix K is a square matrix whose non-zero elements are

Figure 4 . 2 :

 42 Figure 4.2: The schematic of the proposed method

Figure 4 . 3 :

 43 Figure 4.3: A response process containing exceedance events

  assigning a prior distribution π(θ) for the parameters. Then, considering the experimental data D, π(θ) is updated by the likelihood π(D|θ). The posterior distribution π(θ|D) is obtained by re-normalizing the updated π(θ), i.e. π(θ|D) = π(D|θ)π(θ) θ π(D|θ)π(θ) ∝ π(D|θ)π(θ) (4.26) • k-NN is a type of instance-based learning where the function is only approximated locally. In regression case, the input consists of the k closest training examples in the feature space, and the output is the average of target values of the k neighbors. The value k can be determined by cross-validation, and the 'closeness' can be measured by Euclidean distance. We can also use weighted k-NN where the neighbors are weighted, according to their distances, to determine the output.

Figure 4 .

 4 Figure 4.4 illustrates how a Random Forest is learned and then employed to do predictions. The 'OOB' data can be used to estimate the model error. When the model is built, we are able to do predictions for future observations, either by majority voting or averaging the predicted values of all trees.

Figure 4 . 5 ,

 45 x 1 , x 2 , • • • , x n are the variables in the n-dimension uncertainty space;

  As RF belongs to supervised learning category, the training data should contain the input values and output values within each case. The reliability data is obtained as the training data. According to section 4.3.1, a set of samples of the uncertain variables are generated within their predefined uncertainty frameworks. Then, based on the section 4.3.2, the Monte-Carlo simulations are applied to estimated the failure probabilities for these samples.

Figure 4 . 6 :

 46 Figure 4.6: Prediction by RF

Figure 4 . 6 .

 46 Specifically, f i (x new ) = T D N ,θ i (x new ) where T D N ,θ i is the random tree learned from the ith bootstrap sample. θ i is an independent random seed that helps generate a bootstrap sample from the training set D N . In the process of prediction by an individual tree, x new is pushed through the tree (starting from the root), according to the splitting rules, until it reaches a certain leaf. Then the prediction value is obtained as the leaf, see the illustration in Figure 4.7. In Figure 4.7, it is a regression tree learned from the training data. Each observation in the training data consists of four input values [x 1 , x 2 , x 3 , x 4 ] and an output value y.

Figure 4 . 7 :Example 1 Table 4 . 1 :

 47141 Figure 4.7: Illustrative example: prediction by a single tree

Figure 4 . 8 :Example 2

 482 Figure 4.8: Model evaluation on different data(×1000) for 1-DOF structure

Figure 4 . 9 :

 49 Figure 4.9: Model evaluation on different data(×1000) for 2-DOF structure

Example 3

 3 The main structure is 3-DOF and the TMD structure is 2-DOF. Each degree of the structure, including the main structure and the TMD structure, has the same parameter configurations as those in the example 2. The threshold of the failure criterion is 2.5 × 10 -2 m. The simulation results are shown in Figure4.10.

Figure 4 .

 4 Figure 4.10 shows that the correlation coefficients of both cross-validation and prediction are still staying in a high level. The co-efficient and the error indices change in a similar trends to those in Example 2 except that the convergence values (es-pecially RAE) are larger than those in the last example. This is an evidence that when the dimension of uncertain space is high, the data will be more difficult to model. Intuitively, we need more samples to train the model, but there are some other factors that may possibly affect the model performance. Therefore, Section 4.2 will explore the possible influence factors on the model.

Figure 4 . 10 :

 410 Figure 4.10: Model evaluation on different data(×1000) for 3-DOF structure

Figure 4 . 11 :

 411 Figure 4.11: Simulation result based on different standard deviations

Figure 4 .Figure 4 . 12 :

 4412 Figure 4.11 and Figure 4.12 show that, the variance of the uncertain property m s makes clear influence on the RF model performance, while the variances of c s and k s do not. In Figure 4.11(12), 'CV-m', 'Test-m' respectively mean cross-validation results and test results when we study the variance of m s ; 'CV-c', 'Test-c', 'CV-k'

Figure 4 . 13 :

 413 Figure 4.13: Influence of the number of features and number of trees

Figure 4 .

 4 Figure 4.14: F I evaluation of jth feature via Tree i in RF

2 . 2 m 1 m 2 c 1 m 3 k 1 c 2 k 2 k 3 c 3

 223 with the same standard deviations, and implement the feature importance analysis. The RF contains 150 trees. The number of candidate features to split a node is 3. See results in table 4.4 and Figure 4.15.

Figure 4 .

 4 Figure 4.15: F Is of the structural properties (with SDs)

  tures, and the same datasets are considered as the training data. The main structures have the number of DOFs ranging from 1 to 3. For the first system with single DOF main structure, 8000 samples are chosen to train the model, while for the second and third structure, 15000 and 60000 samples are respectively chosen as the training data. Within each training data, 1000 samples are randomly chosen as the test set to test the model. To reduce the influence of the randomness of the data on the models, we repeat the test procedures for 50 times and average the evaluation results. RMSE is chosen as the index to evaluate the model performance. See the simulation results in Table4.5.

  Figure 4.16 show that the RF model prediction values are in good accordance with the actual values. The RMSE in the simulation results is small

Figure 4 . 16 :

 416 Figure 4.16: Simulation results of the object structure via RF

Figure 4 . 20 :

 420 Figure 4.20: Relative response of different DOFs of the structure

  ing and prediction of passive controlled structures. A comparative study of the predictive performance of the model on different degree-of-freedom structures are presented. Results from three numerical examples showed that the use of Random Forest model in failure prediction of passive controlled structures is a meaningful exploration. The analysis of influence factors on the model performance provides some possibilities to improve the prediction results when the samples are limited.

  set of weak classifiers to a strong classifier by iteratively adjusting the weight distribution of samples in the training set and learning base classifiers from them. At each round, the weight of misclassified samples are increased and the base classifiers will focus on these more. This is equivalent to inferring classifiers from training data that are sampled from the original data set based on the weight distribution. • Stacking (Stacked Generalization) involves training a learning algorithm to combine the predictions of several other learning algorithms. See Figure 5.1

Figure 5 . 1 :

 51 Figure 5.1: General framework of Stacking

Figure 5 . 2 :

 52 Figure 5.2: Diagram of Stacking method (include model evaluation on test data)

1 ,

 1 we use the same data set D to train first-level classifiers and prepare training data for second-level classifiers, which may lead to over-fitting. To solve this problem, the idea of cross validation is incorporated in stacking. K-fold cross validation (CV) is the most commonly used technique to evaluate classification performance. To evaluate the prediction ability of a learning algorithm, we conduct the following K-fold cross validation procedure: We partition training data into K disjoint subsets and run the learning algorithm K times. Each time we learn a classifier from K -1 subsets and use the learnt model to predict on the remaining one subset and obtain the predictions for this subset. After K runs, all the K subsets obtain their predictions. See Figure5.3. Each time a classifier is learnt, it is applied on the test set. Therefore the predictions are repeated K times on the test data. The final prediction on the test data is obtained by averaging the predictions of the K runs. The averaged predictions can be seen as a newly-created feature used to train the 2nd-level classifier.

Figure 5 . 3 :

 53 Figure 5.3: CVs (K=5) to create 2nd-level data. In practice, K=10 is used

Figure 5 .Figure 5 . 5 :

 555 Figure 5.6 shows the performances of the Stacking models that commonly have three base learners. Compared with the Stacking performances in Figure 5.5, the Stacking models in Figure 5.6 result in much better performances. It is found that,

Figure 5 . 6 :Figure 5 . 7 :

 5657 Figure 5.6: Performances of Stacking models that have three base learners and a metalearner. The form 'A&B&C-D' denotes that A, B and C are base learners and D the meta-learner

Figure 5 . 8 :

 58 Figure 5.8: Stacking models that have two base learners

Figure 5 . 13 :

 513 Figure 5.13: RMSE in terms of the number of sampled features for each splitting in GB

Figure 5 . 14 :

 514 Figure 5.14: Bias-variance of single models in terms of maxFeatures in each split

Figure 5 .

 5 Figure 5.14 shows how the bias and variance of a single model changes with respect to different K values in that model. As the plot in Figure 5.14 shows, the three models have similar trends of change. The plots with * denote the MSE of the

Figure 5 . 15 :

 515 Figure 5.15: Bias-variance of Stacking model in terms of K in each split in base learner RF

Figure 5 . 16 :

 516 Figure 5.16: Bias-variance values in terms of K in each split in meta-model

T( 5 . 9 )T 1 GB + T 2 GB= 0 . 5 *

 591205 RF = O(0.1N * M 1 log(0.632 * 0.9N )) = 0.1M 1 * O(N log N ) (5.8)For ETs, it is noted that in each loop of CV, all cases in the training data are used to train the trees in ETs. The time complexities to build the model and make predictions areT ET s = O(M 2 * K 1 ) * (0.9N ) log(0.9N )) = 0.9M 2 K 1 * O(N log N ) ET s = O(0.1N * M 2 log(0.9N )) = 0.1M 2 * O(N log N ) (5.10)For GB, each tree (from the 2nd) is built from the weighted sum of the prediction errors of the former trees. So the time complexity of training a GB model is divided into two parts: time complexity of building the trees and time complexity of making predictions by the trees. It is noted that the first and last trees are not used to make predictions in the training process of GB. The first tree is a constant which is seen as a 1-node tree[START_REF] Friedman | The elements of statistical learning[END_REF]. It is calculated as the average value of the targets in the training set. For the first tree, the time complexity is ignored, so we just consider the other (M 3 -1) trees. In GB, the whole training set is used to train each tree. The time complexity to build the trees in GB model isT 1 GB = O((M 3 -1)K 1 (0.9N )log 2 (0.9N )) = 0.9(M 3 -1)K 1 * O(N log 2 N ) (5.11)Let's see how the time complexity of making predictions is calculated during the inducing process of GB model. In the inducing process of GB model, to build the 3rd tree, we need the predictions of the 2nd tree, the time complexity to make predictions by the 2nd tree is O(log(0.9N )); to build the 4th tree, the predictions of the 2nd and 3rd trees are needed, the time complexity is 2O(log(0.9N )); • • • ; to build the M 3 th tree, the predictions of the 2nd until (M 3 -1)th trees are needed, the time complexity is (M 3 -2) × O(log(0.9N )). Therefore, the total time complexity to make predictions within GB model learning process isT 2 GB = O(log(0.9N )) M 3 -2 i=1 i = 0.5 * (M 3 -2)(M 3 -1) * O(log N )(5.12)In total, the time complexity needed for building a GB model isT GB = T (M 3 -2)(M 3 -1) * O(log N ) +0.9(M 3 -1)K 1 * O(N log 2 N ) (5.13) Then the GB model is used to make predictions on the 10% of the training set. The time complexity is T GB = O(0.1N (M 3 -1) log(0.9N )) = 0.1(M 3 -1) * O(N log N ) (5.14)

T 1 = 10 •

 110 1M 2 * O(N log N ) GB [ 0.9(M 3 -1)K 1 * O(N log 2 N ) +0.5 * (M 3 -2)(M 3 -1) * O(log N ) ] 0.1(M 3 -1) * O(N log N )In the 1st-level training process, RF, ETs and GB are trained respectively. 10-fold CVs are applied where 9 folds are used to train the model and 1 fold is used to make predictions. The time complexity is denoted asT 1 = 10 • (T RF + T RF + T ET + T ET + T GB + T GB ) (5.15)where T , T are the time complexity to build the model and make predictions respectively.(T RF + T RF + T ET + T ET + T GB + T GB ) = 10 • [0.57M 1 K 1 * O(N log 2 N ) + 0.1M 1 * O(N log N ) +0.9M 2 K 1 * O(N log N ) + 0.1M 2 * O(N log N ) +0.9(M 3 -1)K 1 * O(N log 2 N ) + 0.5 * (M 3 -2) * (M 3 -1) * O(log N ) + 0.1(M 3 -1) * O(N log N )] = (5.7M 1 + 9M 3 -9)K 1 * O(N log 2 N )(5.16) In the 2nd-level training process, a GB model is trained, using the features created by the base learners and the decisions from the original training set, T 2 = (M 4 -1)K 2 * O(N log 2 N ) + 0.5 * (M 4 -2)(M 4 -1) * O(log N ) = (M 4 -1)K 2 * O(N log 2 N )

Figure 5 . 17 :

 517 Figure 5.17: CPU time of the models employed in the simulations

5. 6 6 The

 66 Comparisons between Stacking and RF 5.6.1 The benchmark problem in section 4.4.Stacking method is compared with RF in the benchmark problem of 10-DOF oscillator introduced in section 4.4.6.2. The base learners of the Stacking model are SVM and RF (with 30 trees), the meta-learner is ETs (with 30 trees). For each threshold examined, the two methods use the same size of training set, see

Figure 5 . 19 :

 519 Figure 5.19: RMSE of Stacking model 'SVR&RF-GB' and single models

  training dataset is firstly generated. In the training set, the input is the samples of the structural properties obtained by random sampling from the assumed joint probability density function (PDF, assuming the variables are relatively independent). The output for each sample is induced by MCS or KL-IS method, depending on the scale of the failure probability to be estimated. Once the RF model is built, it can be used to make predictions on any other samples outside the training set. Finally the overall failure probability can be directly estimated. The RF model has shown its accuracy in conditional failure probability modeling and predictions.• The Stacking model has been explored and introduced to model the conditional failure probability. In principle, a Stacking model tries to improve the prediction performance by combining various models (called base models) in a special way, then constructing a higher-level model (called meta-model). The base models can be a single learner, such as k-nearest neighbor (k-NN), support vector machine (SVM); it also can be an ensemble learner, such as RF, Gradient boosting (GB) or Extra-Tress (ETs). The meta-model is induced from the predictions of the base models together with the original outputs of the training set. Here, the predictions, by the base models, are made on the input samples of the training set. Cross-validations are carried out to make these predictions in order that the overfittings can be avoided. The simulations have been performed from various perspectives such that the proposed model is examined in a systematic way. The simulation results show that, with a price of a limited CPU time increment, the Stacking model obtains a significant advantage over its base learners in terms of structural reliability predictions.

  -clés: La fiabilité structurelle, Incertitudes, Apprentissage statistique General . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

	Chapter 1
	Introduction
	Contents
	1.1

1 1.2 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 1.2.1 Vibration control of structural systems . . . . . . . . . . . . . 2 1.2.2 Uncertainties in structures . . . . . . . . . . . . . . . . . . . . 3 1.2.3 Failure mechanisms . . . . . . . . . . . . . . . . . . . . . . . . 5 1.2.4 Structural reliability . . . . . . . . . . . . . . . . . . . . . . . 6 1.3 Basic contents of structural reliability analysis . . . . . . . . 7 1.4 Methods for reliability analysis . . . . . . . . . . . . . . . . . 9 1.5 Structural reliability analysis process . . . . . . . . . . . . . 14 1.6 Research scope and objectives . . . . . . . . . . . . . . . . . . 14 1.7 Dissertation organization . . . . . . . . . . . . . . . . . . . . . 15

  from different perspectives. Bias-variance analysis of the prediction error is performed to evaluate the model in a more general way. Time complexity of the prediction model is also introduced and analyzed in detail.

		Chapter 2
	Reliability assessment of
	In Chapter 6, some conclusions are drawn on this dissertation research and provide
	perspectives for future work.	structural system: the
		fundamentals

4, to deal with passive controlled structures subject to earthquake induced vibrations, we propose a novel reliability modeling method based Random Forest and Monte-Carlo Simulation . Random Forest is distinguished due to its robustness and high accuracy in modeling and prediction work. Therefore, this part devotes to exploring the feasibility of RF and examining its performance in modeling and prediction of structural reliability. In this part, different complexity of structures are studied. We also test other representative machine learning methods, and compare them with RF in different aspects. In Chapter 5, Tree-based ensemble methods, such as RF, always behave well, but further studies are needed to improve the prediction performance. Stacking method tries to build the prediction model in a hierarchical way, resulting in a meta-learner induced from the predictions of various base learners. Recent research shows that, with a relatively small price of CPU time, the Stacking model can largely improve reliability predictions compared with individual base learners. The prediction model is learned on a training set that is provided by Monte-Carlo simulations. In numerical simulations, different Stacking models are explored and their performances are compared and analyzed Contents 2.1 General . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

2.2 General principle of structural reliability analysis . . . . . . 18 2.2.1 Limit-state function . . . . . . . . . . . . . . . . . . . . . . . 19 2.2.2 Failure Probability . . . . . . . . . . . . . . . . . . . . . . . . 19 2.3 Strategies for the estimation of P f . . . . . . . . . . . . . . . 21 2.3.1 Analytical approximation methods . . . . . . . . . . . . . . . 21 2.3.2 Moment-based methods . . . . . . . . . . . . . . . . . . . . . 25 2.3.3 Simulation based methods . . . . . . . . . . . . . . . . . . . . 26 2.3.4 Surrogate model based methods . . . . . . . . . . . . . . . . 32 2.3.5 Time-variant reliability . . . . . . . . . . . . . . . . . . . . . 34 2.3.6 Statistical learning models . . . . . . . . . . . . . . . . . . . . 35 2.4 Chapter summary . . . . . . . . . . . . . . . . . . . . . . . . . 46

Table 2 .

 2 1: characteristics of different types of single trees (regression case)

	Item	CART	RandomTree	REPTree
	Bootstrap	yes	no	no
	Split type	binary	binary	binary
	Split	square error	square error	square error
	principle
	Pruned	no	no	yes
	Split	the best one from	the best one from	the best one
	attribute	a random subset	a random subset	among all

attributes

A comparison of the different types of trees CART, RandomTree and REPTree is shown in Table

2

.1. We assume the problem that we are facing is a regression problem.

2.3.6.8 Extra-Trees

Extremely randomized trees, Extra-Trees or ETs, is so named because the ETs have further randomness in node splitting compared with RF. In principle, RF and ETs are similar. Both of them consist of multiple decision trees. In the tree inferring process, both of them use a random subset of the features as the candidate splitting features. The mainly different operations are listed below. (a). the way to prepare the training set for each tree. RF uses a bootstrap replica to train each tree. In contrast, ETs employ the whole training set to train each tree;

  Figure 2.17: Comparison of RF and ETs in node splitting is chosen. See Figure 2.17.

		b e s t s p l i t s
		1
	In RF	  
			b e s t s p lit s
	x
	2
	random
		b e s t s p l i t s
		m
	subset
	( size: m)
		. rand split s
		1
	In ET	  	best split s	'
		. rand split s
		m
	x
	n

  Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48 3.2 General framework . . . . . . . . . . . . . . . . . . . . . . . . 48 3.2.1 Models of physical system . . . . . . . . . . . . . . . . . . . .

	Chapter 3
	Uncertainty quantification of
	structural systems
	Contents
	3.1 . .	63
	3.4.4 Formulation of conditional failure probability . . . . . . . . .	65
	3.4.5 IS technique to evaluate P C f . . . . . . . . . . . . . . . . . . .	67
	3.4.6 Comparisons between standard MCS and KL-IS . . . . . . .

49 3.2.2 Quantification of sources of uncertainty . . . . . . . . . . . . 50 3.2.3 Uncertainty propagation . . . . . . . . . . . . . . . . . . . . . 51 3.3 Conditional failure probability . . . . . . . . . . . . . . . . . 52 3.3.1 Basic concepts and definition . . . . . . . . . . . . . . . . . . 52 3.3.2 Estimation of Conditional failure probability . . . . . . . . . 53 3.3.3 Machine learning methods as surrogates . . . . . . . . . . . . 54 3.4 Evaluation of very small conditional failure probabilities . . 56 3.4.1 K-L decomposition of Gaussian excitation process . . . . . . 56 3.4.2 Convolution integral to calculate structural responses . . . . 62 3.4.3 Failure probability at any time t . . . . . . . . . . . . . .

  Several discrete representation methods are available to describe such randomness in terms of a finite set of standard normal random variables. For a second-order Gaussian process f (t) with mean function µ(t), the existing discrete representation methods all result in the following form,

Let f (t) be a scalar function that represents the excitations acting over the object structure during a time period t ∈ [0, T ]. It is assumed this excitation process can be approximated by a Gaussian process, i.e. for each finite set of time realizations t 1 , t 2 , ..., t n T ∈ [0, T ], the n T random variables f (t 1 ), f (t 2 ), ..., f (t n T ) follow a joint Gaussian distribution.

Table 3 . 1 :

 31 Sampling scheme according to ISD proposed in eq.(3.34)[START_REF] Au | First excursion probabilities for linear systems by very efficient importance sampling[END_REF]

  ). Each tree is built from a random sample set (i.e. bootstrap sample) generated from the training set. As we use bootstrap sampling to obtain the samples, we therefore just obtain a subset of the training data which we call 'Inbag'; the remaining samples are called 'OOB (out-of-bag)' which is the complementary set of the 'Inbag'.

			Training Data
			[	X	N n 	,	N Y	1 	]
		Bootstrap sampling 1					Bootstrap sampling 2	...	Bootstrap sampling k
	InBag1	OOB 1	InBag 2		OOB 2	...	InBag k	OOB k
	Tree 1		Tree 2			...	Tree k
			Random Forest		Error estimation of RF
	A new
	observation
			Prediction
		(Majority votes/Average)

Figure 4.4: Random Forest: from modeling to prediction. D N is the training data, X N ×n is the input matrix that contains N observations of the input variables, n is the number of input variables, Y N ×1 is a column vector that contains the output values

Table 4

 4 .2.

	Table 4.2: Nominal values of the structure
	Parameters m(kg) c(N • s/m) k(N/m)
	1st DOF	4.6	62.0	6500
	2nd DOF	4.6	62.0	6500
	TMD	1.38	38.997	1.8327
	The uncertainties of structure properties are normally distributed with standard

table 4 .

 4 3. In Table4.3, SD means 'standard deviation'; SD1 is the first value of SD of the variable. For example, SD1 for m s is 0.01.

Table 4 . 3 :

 43 Standard deviations of the uncertain propertiesIn each simulation for a variable that takes a specific SD in Table 3, we use a training set with 7000 samples to build the RF model, and another 1000 sample as the test set. As before, we implement 10-fold cross-validation to evaluate the model. Two indices are employed to evaluate the model performance, i.e. RMSE and MRE. Here MRE means 'mean relative error' which is calculated as

	Items	SD1	SD2	SD3	SD4	SD5	SD6	SD7	SD8	SD9	SD10
	m s	0.01	0.02	0.03	0.04	0.05	0.06	0.07	0.08	0.09	0.10
	c s	0.0003 0.0006 0.0009 0.0012 0.0015 0.0018 0.0021 0.0024 0.0027 0.003
	k s	0.5	1.0	1.5	2.0	2.5	3.0	3.5	4.0	4.5	5.0

Table 4 .

 4 4: List of feature importances

	Feature	ms 1	ms 2	cs 1	ms 3	ks 1	cs 2	ks 2	ks 3	cs 3
	F I value	0.3340 0.2934 0.1301 0.0934 0.0900 0.0298 0.0163 0.0067 0.0064
	SD of F I 0.0055 0.0038 0.0029 0.0051 0.0019 0.0012 0.0029 0.0006 0.0006

Table 4 . 5 :

 45 RMSEs of different methods

	Structural DOF 1-DOF 2-DOF 3-DOF
	Bayes method	0.0208	0.0709	0.0952
	k-NN method	0.0180	0.0517	0.0486
	GB method	0.0051 0.0631	0.0462
	RF method	0.0056 0.0433 0.0451

Table 4 . 6

 46

: Structure parameters (i = 1, 2, 3; j = 1, 2)

Table 4

 4 .7. Notice that, in Table4.7, the SD values are taken as 10% of the corresponding nominal values. The structural failure takes place when a certain DOF of the structure at a certain time point has the response exceeding a prescribed threshold value. Here, for simplicity, the same threshold value is assumed for all DOFs within the observed time interval.

Table 4 . 7 :

 47 Statistical properties of the structural parametersIn this simulation, the threshold is set as y lim = 0.28m; the observed time interval is [0, 20s] with the time step T s = 0.05s after discretization. To prepare the training data for learning the Random Forest model, 1000 samples of the structural properties are randomly obtained according to their PDF (assuming normal distributions with the statistical moments displayed in Table4.7), then the conditional failure probability is estimated for each sample. In the framework presented in Figure4.2,The parameters in the Random Forest model are shown in Table4.8. 'nTrees' is the Figure4.17: Ten-DOF uncertain structure subjected to stochatic excitations[START_REF] Mrabet | New approaches in reliability based optimization of tuned mass damper in presence of uncertain bounded parameters[END_REF] number of random trees in Random Forest; 'nFeatures' is the number of features (variables) in the training set; 'maxFeatures' is the number of sampled features at each node. The RMSE is evaluated on the test set, see Table4.9. Then the learned RF is employed to make predictions on a new set of 2000 cases. According to eq.(3.6), the structural failure probability is estimated considering uncertainties both in structural properties and excitations, See the reults in Table4.9.

	Variables	Mean	SD
	m s1 , ..., m s10	360 × 10 3 kg	36 × 10 3 kg
	c s1 , ..., c s10	620 × 10 4 N s/m	62 × 10 4 N s/m
	k s1 , ..., k s10	650 × 10 6 N/m	65 × 10 6 N/m
	a Random Forest model is induced from the training set containing 1000 cases each
	of which has 30 input variables (i.e. structural parameters) and an output value
	(i.e. the value of conditional failure probability). Besides, a test set of 1000 cases is
	obtained to test the performance of the learned Random Forest model.

Table 4 . 9 :

 49 Simulation results by RF Pf,MCS obtained from standard MCS. As the value of y lim has direct influence on the estimated failure probability, different y lim have been studied in this simulation. The y lim ranges from 0.28m to 0.37m with a discretization step 0.01m. For each y lim , the KL-IS method is implemented to prepare the training set. 200 cases are generated in each training set. Then a RF model consisting of 30

	Train set	Test set	RMSE	Pf,RF	Pf,MCS
	1000	1000	0.0265	0.0789	0.0701
	to the reference value

Table 4 .

 4 10, both the simulation results of MCS and those of RF have shown this phenomenon. By comparisons of the two simulation results, it is found that the RF model achieves close estimation results to those achieved by standard MCS.

	0.08
	0.07
	0.06
	0.05
	Pf
	0.04
	0.03
	0.02
	0.01
	0
	0.28 0.29	0.3	0.31 0.32 0.33 0.34 0.35 0.36 0.37
			ylim(m)
	Figure 4.18: P f estimations (with error bars) by RF

Table 4 .

 4 [START_REF] Marano | A comparison between different robust optimum design approaches: application to tuned mass dampers[END_REF]. Notice that the damping ratio c i = 2ζ i √ m i k i in this case is the proportional viscous damping. Moreover, the Gaussian random variables for structural properties are censored to avoid the deviation from the mean being larger than five times the standard deviation. In Table4.11, r = SD/µ. The stochastic excitation p(t) is modeled by a modulated filtered Gaussian white noise:

Table 4 .

 4 10: Comparisons between standard MCS and RF results

	y lim (m)	Pf,RF	SD RF ( * 10 -3 ) Pf,MCS	SD M CS ( * 10 -4 )
	0.28	0.0789	2.036	0.0701	2.547
	0.29	0.0545	1.577	0.0496	2.172
	0.30	0.0391	1.334	0.0349	1.834
	0.31	0.0278	0.882	0.0241	1.534
	0.32	0.0183	0.778	0.0164	1.270
	0.33	0.0126	0.602	0.0110	1.042
	0.34	0.0084	0.436	0.0073	0.852
	0.35	0.0056	0.279	0.0048	0.689
	0.36	0.0036	0.188	0.0030	0.546
	0.37	0.0023	0.181	0.0019	0.436

Table 4 .

 4 is the Dirac delta function that equals to +∞ at t = 0 and 0 at t = 0. The values Ω 1g = 15.0rad/s, ζ 1g = 0.8, Ω 2g = 0.3rad/s, ζ 2g = 0.995 and I = 0.08m 2 /s 3 are used to model the filter. The input excitation of the filter is a shot noise that consists of a series of independent normally distributed impulses arranged at each time step[START_REF] Lin | Probabilistic theory of structural dynamics(book on probabilistic theory of structural dynamics, analyzing responses of structures to random vibrations)[END_REF]. The magnitude of the impulse at time t k = k∆t has mean 0 and standard deviation h(t) I/∆t. It is noticed that, in G.I. Schueller[START_REF] Gerhart | Benchmark study on reliability estimation in higher dimensions of structural systems-an overview[END_REF], the expression of 11: Statistical properties of the structural parameters

			2,	t ∈ [0, 2s]
						(4.43)
		1,		t ∈ [2s, 10s]
		exp(-0.1(t -10)), t ∈ [10s, 20s]
	δ(t) the standard deviation 'h(t)	√	I∆t' is wrong, however, some other researchers still
	use this wrong expression in their published papers.
	Variables	Mean(µ)			SD	Ratio(r)	Range scope
	m 1 , ..., m 10	10 × 10 3 kg		1.0 × 10 3 kg	0.1	µ ± 5µr
	k 1 , k 2 , k 3	40 × 10 6 N/m	4.0 × 10 6 N/m	0.1	µ ± 5µr
	k 4 , k 5 , k 6	36 × 10 6 N/m	3.6 × 10 6 N/m	0.1	µ ± 5µr
	k 7 , k 8 , k 9 , k 10	32 × 10 6 N/m	3.2 × 10 6 N/m	0.1	µ ± 5µr
	ζ 1 , ..., ζ 10	620 × 10 4 N s/m	62 × 10 4 N s/m	0.1	µ ± 5µr

Table 4 .

 4 12: Thresholds of interest to evaluate failure probability[START_REF] Gerhart | Benchmark study on reliability estimation in higher dimensions of structural systems-an overview[END_REF]

	Failure defined by	Res. threshold1	Res. threshold2
	First DOF	0.057m	0.073m
	Tenth DOF	0.013m	0.017m

Table 4 .

 4 13: Reliability estimation results from different thresholds

	Method	1stDOF	1stDOF	10thDOF	10thDOF
		0.057m	0.073m	0.013m	0.017m
	Standard MCS	1.06e -4	8.07e -7	4.88e -5	2.52e -7
	num of samples	2.98e 7	2.98e 7	2.98e 7	2.98e 7
	SubsetSim/MCMC	1.20e -4	1.00e -6	6.60e -5	4.70e -7
	num of samples	1850	2750	2300	2750
	SubsetSim/Hybrid	1.10e -4	1.10e -6	5.90e -5	3.20e -7
	num of samples	2128	3163	2645	3680
	Complex Modal Ana.	1.00e -4	9.80e -7	6.00e -5	4.60e -7
	num of samples	300	300	300	300
	Spherical SubsetSim	9.20e -5	8.80e -7	4.60e -5	5.30e -7
	num of samples	3070	4200	3250	4900
	Line sampling	9.80e -5	9.70e -7	6.00e -5	4.60e -7
	num of samples	360	3600	360	360
	Random Forest	7.6e -5	1.0e -6	4.2e -5	1.1e -7
	num of samples	800	1000	800	900

Table 5 .

 5 1 shows the general procedures of Stacking, which demands three steps. Firstly, learn first-level (base) learners based on the original training data. There are several ways to learn base learners. We can apply Bootstrap sampling technique to learn independent learners or adopt the strategy used in Boosting. Secondly, 5.4. Cross-Validations in Stacking 123 construct a new data set based on the output of base learner. Assume that each example in D is (x i , y i

Table 5 . 2 :

 52 Structure parameters (i = 1, 2, 3; j = 1, 2)In Figure5.4, the accuracy of two Stacking models as well as three single models i.e. RF, ETs and GB, are compared. It is found that the Stacking models always outperform the individual base models when enough training data is available. The Stacking1 model has smaller RMSEs than its base learners RF and ETs. Similarly, the Stacking2 model is more ac-curate than its base learners RF, ETs and GB all the time. By comparison of the two Stacking models, it is evident that more base models will probably result in more accurate predictions.

	Index	m s i c s i	k s i	m T j c T j	k T j
	Nominal 4.6 62 6500 1.38 1.83 39.0
	SD	1	10 300	-	-	-
	such as RF, Extra-trees (ETs) [40] or GB. The meta-model is GB [6]. Root mean
	square error (RMSE) is used to evaluate the model.

Figure 5.4: Model evaluations. Both Stacking1 and Stacking2 take GB as meta-model. Stacking1 takes RF, ETs as base models; Stacking2 takes RF, ETs, GB as base models

  The parameters of the base models are listed in Table5.3. 'nTrees' is the number of random trees in a base model; 'nFeatures' is the number of features in the training set; 'maxFeatures' is the number of sampled features at each node.

The

Stacking model 'RF&ETs-GB' achieves the best performance while 'RF&GB-ETs' ranks the last. Even though the base learners are the same, the Stacking performances differ due to the choice of the meta-learner. For ex-ample, the models 'RF&GB-ETs' and 'RF&GB-SVR' share the same base learners RF, GB but different meta-learners (i.e. ETs vs SVR). They result in largely different prediction accuracies. Similar cases are 'ETs&GB-RF' vs 'ETs&GB-SVR' and 'RF&ETs-GB' vs 'RF&ETs-SVR'.

Table 5 . 3 :

 53 Hyper-parameters of the base models

	Hyper-parameters	RF GB ETs
	Number of random trees	50	50	50
	Total num. of features	9	9	9
	Num. of features to sample	3	3	3

  Change of RMSE with respect to the number of trees in RF Change of RMSE with respect to the number of trees in GB

	0.01548 0.04
	0.01547 0.035
	0.01546 0.03
	0.01545 0.025
	0.01544
	0.02
	0.01543
	0.01542 0.015
	0.01541 0.01
	0.0154 0.005
	0.01539 0
	1 1 1 0.01542 0.01 0.02 0.03 0.04 0.05 0.06 0.07 0.08 0.09 RMSE 0.01544 0.01546 0.01548 0.0155 0.01552 0.01554 Figure 5.10: linear 2 3 2 3 2 Figure 5.12: RMSE in terms of the number of sampled features in each splitting in RF 4 5 6 7 8 9 10 Num of trees in RF (x10) 4 5 6 7 8 9 10 Num of trees in GB (x10) poly rbf sigmoid Kernel type 3 4 5 6 7 8 9 Num of sampled features in RF RMSE

RMSE

Figure 5.9: of these improvements are still not studied. Researchers always employ bias-variance analysis to see how different components of the error are changed by a design of a RMSE Figure 5.11: RMSE with respect to the kernel type of SVR new model. Therefore, section 4.4 devotes itself to bias-variance analysis of the Stacking model.

Table 5 . 4 :

 54 Pseudo-code of bias&variance calculation

Input: Training data D = {x i , y i } m i=1 , x i ∈ X, y i ∈ Y ; a Stacking algorithm.

  1, containing 10000 cases. Based on the procedures in Table5.4, the bias, variance and MSE (also RMSE) is computed, see Table5.5. The base learners SVR takes 'rbf' as its kernel function, RF has the same parameters as listed in Table5.3. 10-fold CVs is applied in the 1st-level training of Stacking. 100 bootstrap datasets are generated to represent the variations of the training data. In Table5.5, the analysis results of the single models, i.e. SVR, RF and GB, are also provided and compared with those of the Stacking model. Obviously, the Stacking model achieves much lower squared bias than any single models; its variance is larger than SVR, but it is smaller than RF and GB. However the sum of the two terms, i.e. MSE, is much smaller than any single model. Generally, the bias-variance decomposition results vary when a single model (RF or GB) changes its value of K, i.e. number of sampled features in each split. This is because the model complexity changes when K is set differently, which possibly results in overfitting or underfitting.

Table 5 . 5 :

 55 Bias-variance decomposition result

	Models	bias 2	variance	M SE	RM SE
	SVR&RF-GB 2.82e-04 1.13e-05 2.93e-04 1.71e-02
	SVR	4.00e-03	3.28e-6	4.00e-03	6.32e-02
	RF	4.15e-03	5.48e-4	4.69e-03	6.85e-02
	GB	6.50e-03	4.01e-4	6.90e-03	8.31e-02

Table 5 . 6 :

 56 Time complexities (average) of the base models inStacking2 K 1 * O(N log 2 N ) 0.1M 1 * O(N log N ) ETs 0.9M 2 K 1 * O(N log N ) 0.

	Models	Model building	Predicting
	RF	0.57M 1

Table 5 . 7 .

 57 Notice that the failure probability values are calculated in a mean sense. Form Table Figure 5.18: CPU time of Stacking model 'SVR&RF-GB' and single models

		80
					SVR		RF		GB		SVR&RF-GB
		70
		60
	CPU time	40 50
		30
		20
		10
		0
		0	2	4	6	8	10	12	14	16	18	20
					Size of training set (*1000)

Table 5 . 7 :

 57 Compare Stacking and RF in reliability estimations

	Method	y 1 lim = 0.057m y 1 lim = 0.073m y 10 lim = 0.013m y 10 lim = 0.017m
	Standard MCS	1.06e -4	8.07e -7	4.88e -5	2.52e -7
	num of samples	2.98e 7	2.98e 7	2.98e 7	2.98e 7
	Random Forest	7.6e -5	1.0e -6	4.2e -5	1.1e -7
	num of samples	800	1000	800	900
	Stacking	1.0e -4	9.2e -7	4.3e -5	1.1e -7
	num of samples	800	1000	800	900

Table 5 . 8 :

 58 Compare Stacking and RF when multi-thresholds are assumed

	Method	y 1 lim = 0.057m ∪ y 10 lim = 0.013m y 1 lim = 0.073m ∪ y 10 lim = 0.017m
	Standard MCS	1.12e -4	1.00e -6
	num of samples	1.0e 6	1.0e 6
	Random Forest	1.11e -4	1.10e -6
	num of samples	800	1100
	Stacking	1.20e -4	1.18e -6

Table 5 . 9 :

 59 Compare Stacking and RF when the structural parameters are all lognormal

	Method	y 1 lim = 0.057m	y 10 lim = 0.013m
	Standard MCS	8.8e -5	4.0e -5
	num of samples	1.0e 6	1.0e 6
	Random Forest	7.1e -5	4.0e -5
	num of samples	1600	1600
	Stacking	8.0e -5	4.1e -5
	num of samples	1600	1600

Table 5 .

 5 10: Compare Stacking and RF when the structural parameters are all Gamma

	Method	y 1 lim = 0.057m	y 10 lim = 0.013m
	Standard MCS	9.8e -6	3.7e -6
	num of samples	2.0e 7	2.0e 7
	Random Forest	9.7e -6	3.6e -6
	num of samples	500	800
	Stacking	9.6e -6	3.7e -6
	num of samples	500

Table 5 .

 5 11: Compare Stacking and RF when the structural parameters are a mixture of

	Lognormal and Gamma
	Method	y 1 lim = 0.057m	y 10 lim = 0.013m
	Standard MCS	3.1e -5	4.0e -6
	num of samples	2.0e 7	2.0e 7
	Random Forest	3.0e -5	4.1e -6
	num of samples	1000	900
	Stacking	3.2e -5	4.1e -6

3.5 Chapter summary . . . . . . . . . . . . . . . . . . . . . . . . . 77

Stacking method is compared with RF on the benchmark problem in chapter 4.

Non-Gaussian structural random variables are also studied in structural reliability estimation. The structural reliability assessment always demands appropriate characterization of input information such as loading, material parameters, etc. However, these parameters are often not determined uniquely as they are affected by various uncertainties due to interior and exterior factors. Hence, the output of virtual simulation models cannot be specified by a single value. Uncertainties in inputs are propagated to output quantities generally by simulation techniques. Uncertainty quantification is most useful for handling uncertainties and making evaluations on levels of structural safety, maintenance and repair activities. Developments in the field of data mining have provided various powerful tools for predicting performance of structural systems in the framework of virtual simulation models. Therefore, machine learning technics have been extensively explored in this dissertation for structural reliability assessment.

Dissertation contributions

In summary, this research contributes to methodologies of structural reliability analysis and machine learning theories in structural reliability estimation. The reliability analysis is performed in a new framework of ensemble learning models that models structural parameter uncertainties and map them into failure probabilities. The following list includes the contributions made in the dissertation.

List of publications