Thierry Gonon

Céline Helbert

Christophette Blanchet-Scalliet

Bruno Demory

Keywords: Metamodel, Kriging, Gaussian process regression, High dimension, Infinite conditioning, Multifidelity, Nested spaces, Variable-size design space problems

Metamodels are widely used in industry to predict the output of an expensive computer code. As industrial computer codes involve a large amount of input variables, creating directly one big metamodel depending on the whole set of inputs may be a very challenging problem. Industrialists choose instead to proceed sequentially. They build metamodels depending on nested sets of variables (the variables that are set aside are fixed to nominal values), i.e. the dimension of the input space is progressively increased. However, at each step, the previous piece of information is lost as a new Design of Experiment (DoE) is generated to learn the new metamodel. In this paper, an alternative approach will be introduced, based on all the DoEs rather than just the last one. This metamodel uses Gaussian process regression and is called seqGPR (sequential Gaussian process regression). At each step n, the output is supposed to be the realization of the sum of two independent Gaussian processes Y n-1 + Z n . The first one Y n-1 models the output at step n -1. It is defined on the input space of step n -1 which is a subspace of the one of step n. The second Gaussian process Z n is a correction term defined on the input space of step n. It represents the additional information provided by the newly released variables. Z n has the particularity of being null on the subspace where Y n-1 is defined so that there is a coherence between the steps. Firstly, some candidate Gaussian processes for (Z n ) n 2 are suggested, which have the property of being null on an infinite continuous set of points. Then, an EM (Expectation-Maximization) algorithm is implemented to estimate the parameters of the processes. Finally, the metamodel seqGPR is compared to a classic kriging metamodel where the output is assumed to be the realization of one second order stationary Gaussian process. The comparison is made on two analytic examples, a first one with two steps, up to dimension 4, and a second one with three steps, up to dimension 15. The introduced methodology is also tested on an industrial example which goes from dimension 11 to dimension 15. In all these test cases, seqGPR performs better than, or at least as well as kriging.

Bon, et bien je crois que le moment est venu. C'est un événement sacrément spécial que la fin d'une thèse, de trois années de recherche et d'approfondissement d'un sujet. Il est temps pour moi de remercier les principaux acteurs de la mienne, qui ont fait en sorte qu'elle se passe le mieux possible. Préparez-vous à la lecture des remerciements les plus longs de l'histoire des remerciements.

Merci aux membres du jury d'avoir accepté votre mission : Bertrand Iooss, Clémentine Prieur, Luc Pronzato, et Olivier Roustant. En particulier merci à Luc Pronzato et Bertand Iooss pour avoir accepté de rapporter ma thèse.

Merci à Céline, Christophette, et Bruno, mes très chers encadrants. Merci Céline pour ton énergie permanente, pour tes idées à foison, pour m'avoir accordé ta confiance en m'accompagnant en projet, en stage, puis en thèse. Merci pour ton aide bibliographique, ta connaissance dans le domaine, et ton investissement dans la communauté française de la quantification d'incertitude. Merci ensuite Christophette, pour avoir accepté de me superviser avec Céline, pour ton engagement dans ma thèse, ton ardeur à réfléchir aux questions que je t'ai posées, tes remarques toujours pertinentes et tes éclairs de génie, et nos partages de calculs et de réflexions qui m'ont passionné et ont été une vraie source de motivation dans ma thèse. Merci à Bruno pour m'avoir suivi toutes les deux semaines avec assiduité, pour ton enthousiasme sans égal envers ton travail qui me fait vouer un respect sans borne en ta personne. Merci pour toujours avoir gardé de vue les enjeux industriels, une vision globale qui nous a été extrêmement utile pour nous recadrer dans un contexte, et parfois nous réaligner sur les bons enjeux, lorsque nous nous perdions un peu dans des développements mathématiques.

Merci au labo de maths dans son ensemble, pour la bonne ambiance qui y règne et pour l'intégration des jeunes. En particulier, merci Isabelle pour ta gentillesse, ton sens du service, pour m'avoir aidé à m'y retrouver dans les méandres administratifs. Merci Francis, pour avoir réglé tous les soucis informatiques que j'ai rencontrés, et pour m'avoir conseillé des outils très utiles dans ma vie de tous les jours. Merci Martin pour avoir consacré du temps à m'expliquer le fonctionnement du cluster de Centrale et m'avoir aidé à me servir de ton code 1D. Merci à Benoit Fabrèges et Roland Denis, responsables du cluster de l'ICJ, vous que j'ai sollicités à de nombreuses reprises (probablement de façon excessive) et qui avez pris le temps de répondre à mes questions relatives au cluster ou même pour des problèmes de code (bien que cela dépasse le cadre de votre fonction). Cela m'a débloqué plusieurs fois et je vous en suis très reconnaissant.

Merci à ceux qui ont collaboré de près ou de loin à la thèse. Merci à Manuel et Nicolas, vous qui avez oeuvré côté Valeo pour la naissance de ma thèse, qui avez gardé un oeil sur moi tout au long de celle-ci, et que je ne remercierai jamais assez. Merci à l'équipe de CFD des systèmes thermiques de Valeo, à laquelle j'ai parfois été rendre visite durant ces trois ans. Merci Xavier Bay pour nous avoir accueillis aux Mines de Saint-Etienne, pour avoir longuement discuté avec nous de ma thèse, et défriché très pédagogiquement les portions de votre travail que nous avons utilisées. Merci à Olivier Roustant et Jean-Yves Welschinger pour avoir accepté d'être membres de mon comité de suivi, pour avoir surveillé le déroulement de ma thèse avec un vrai soucis de me faire réussir, et en particulier Olivier d'un point de vue scientifique pour nous avoir apporté votre expertise dans le domaine et nous avoir proposé des pistes.

Un grand merci aux membres de la vénérable cathestr, JC, Paypouze, Jacky, et Hippo, pour m'avoir accueilli dans votre coloc les bras ouverts au début de ma thèse. En particulier, merci à toi Hippolyte, à qui j'ai coûtume de dire que je dois ma thèse. Merci d'avoir été me chercher pour faire un projet de recherche et m'avoir fait connaître Céline par la même occasion.

Un merci tout particulier à mes chers cobureaux, avec qui j'ai eu le plaisir de passer mes journées, et parfois aussi mes soirées ou week end. Merci à Mona, tout d'abord, pour avoir passé avec moi mes premières longues journées d'hiver au labo, pour ta gentillesse à toute épreuve, et pour ta formidable réactivité au dooble. Merci à Nicolas, ensuite, toi qui dès le premier jour m'a demandé : "Dis-moi Thierry, est-ce que tu sais jouer au tarot ?". Merci pour les footings dans les pentes de la Croix-Rousse que tu m'as emmené faire pour mon plus grand plaisir (même si je n'avais pas le niveau), pour la sortie ski à laquelle tu m'as convié et que je qualifierais de journée des descentes de l'extrême. Merci à Laura, ou Madame Tatouille, notre mère à tous bien sûr, une source d'inspiration pour beaucoup de choses, pour avoir contribué à l'ambiance du labo avec les regrettés vendredis pâtisserie, pour le sujet de stage que tu as fait avec Mathilde et que je vous ai honteusement piqué, pour toutes les activités que tu nous as organisées : balades, escape game, et j'en passe, pour mon article et ma présentation au séminaire des doctorants que tu as relus avec une minutie qui m'a impressionné, dont je n'aurais pas été capable pour d'autres. Merci à Angèle, toi qui est la cobureau avec laquelle j'ai passé le plus de temps pendant la thèse, merci pour les problèmes mathématiques que tu nous as exposés et auxquels nous avons réfléchis devant le tableau, ton éthique de travail et ta conscience professionnelle qui je pense m'ont tiré vers le haut, merci pour ton incroyable énergie, enrhumée ou non, pour avoir été la précurseure des allerretour à vélo, et la plus assidue je le reconnais volontiers, merci pour les footings sur le temps du midi, pour tous les jeux que nous avons faits quand nous n'étions que deux ou avec d'autres (peut-être rattrapera-t-on un jour le score de Laura et Mathilde au hanabi), pour enfin nous avoir invités à ton magnifique mariage. Merci à Mathilde, informaticienne pour la thèse mais mathématicienne dans le coeur, merci pour ta bienveillance envers moi, j'ai vraiment apprécié ta compagnie, nos discussions en voiture après les quelques soirées que nous avons faites ensemble, ainsi que notre visionnage de Forest Gump en temps de grève qui restera inoubliable, je me réjouis grandement de ton heureux événement récent avec tout de même un léger pincement au coeur qu'il ne soit pas arrivé le 23 septembre, histoire que ce jour soit définitivement éligible à la fériérisation. Merci Mélina, mon illustre prédécesseure de thèse, pour avoir pris le temps de répondre à mes questions à certains moments de ma thèse quand j'en ai eu besoin, qui ont été charniers dans le déroulement de celle-ci et qui ont notamment conduit à des choix sur le format de mes codes R largement inspirés des tiens. Merci également pour les quelques fois où je t'ai rencontrée dans un cadre plus festif où j'ai pu découvrir ton sens de l'humour dévastateur qui fait mouche sur moi à tous les coups. Merci à Antoine, avec qui j'ai de commun nos années d'élèves ingénieurs, merci pour nous avoir fait découvrir ce jeu exceptionnel qu'est le hanabi, qui est devenu une véritable affaire d'état, vecteur de rapports passionnés sur le temps du midi, merci pour avoir assisté en ma compagnie et celle de Reda à la dernière grande victoire de Roger Federer (une demi-finale de Wimbledon d'anthologie contre Nadal), merci également pour m'avoir inclus au groupe des restos avec Lucas et Antonio. Merci à vous trois pour avoir traversé nos thèses côte à côte, pour nos soirées de débats mathématiques, politiques, sociétaux et sportifs enfiévrés. Merci Thibault pour avoir accepté les yeux fermés de faire le RAID avec moi, pendant lequel on s'est je pense bien marrés, nous les MItigés, les combattants, les talentueux, ou, pour reprendre tes mots, les chaudières. Merci à Hugues, toi qui as une place particulière en tant que dernier rescapé des compagnons du flunch à être resté dans l'enceinte de l'école, merci pour les soirées de réflexion intense autour d'énigmes mathématiques, et pour les parties endiablées de race for the galaxy (moi l'élève a encore du travail pour rattraper le niveau du maître). Merci Reda, le célèbre auteur de la fonction de Reda, comme aime à l'appeler Noé, merci pour ta tranquillité en toute circonstance, pour tes compétences avancées de recherche bibliographique, pour les confs que j'ai passées à tes côtés, ce qui m'a permis de me sentir moins seul, enfin pour notre goût commun du foot et les parties que nous avons jouées ensemble (une technique de jeu et un contrôle de balle qui méritent tout mon respect). Merci enfin à ceux qui ont fait un bout de chemin avec moi en se rajoutant à la formidable bande : Benjamin, Tania, Nicolas (le post-doc), Matthieu, Josselin et bien sûr la prometteuse relève : Noé et Benoit.

Un grand merci à la compagnie élargie du flunch : Hugues en premier artilleur, Natan le plus sympathique des malchanceux, Thomas saboteur dans l'âme, Thibault digne héritier d'Audiard et de Coluche, Clément frère de maillot (allez Paris) et meilleur binôme en 1A, Antonio le plus français des espagnols et solide compère des TD de la mort en 3A. Notre esprit de groupe a perduré malgré la perte tragique de notre vénéré flunch, sa disparition nous a rendus plus forts et plus soudés et je vous remercie pour les nombreux skype que nous avons faits, pour les parties acharnées de pictionnary, et pour les parties de saboteur qui auront révélé une absence totale de mauvaise fois au sein du groupe, sans aucune réserve possible.

Un mot sur ma famille qui a joué un rôle important. Merci papy pour m'avoir transmis ton goût des maths, de leur histoire (que je ne connais que trop peu et j'essaie d'y remédier), les livres que tu m'as passés, et pour m'avoir fait découvrir Mickael Launay. Merci à toi et mamie pour nos appels parfois longs au téléphone les week end, qui me tiennent compagnie. Merci papa et maman pour m'avoir soutenu dans ma décision audacieuse de me lancer dans un doctorat. Merci à vous ainsi que Stéphane et Laura. Merci Stéphane pour les crises de fou rire que tu déclenches chez moi, pour nos nombreux délires qui désespèrent notre entourage, pour tes anecdotes croustillantes et hors du commun, pour ta vie sociale riche et saine qui m'inspire dans mes propres relations. Oui c'est le petit frère qui sert d'exemple au grand. Merci Laura, toi qui sera toujours tendrement "ma petite Laura" même si tu me répètes souvent que tu n'as plus 8 ans tout de même, experte reconnue en films de noël, que je vois grandir et s'épanouir à vue d'oeil. Si tu peux me rendre un service, ça serait de ne jamais t'arrêter de rire, car c'est ma plus grande source de joie. Même si à mon grand regret, je vous ai dégouté des maths, j'épie avec curiosité la direction que prendront vos études qui seront de toute façon nouvelles pour la famille et source d'ouverture vers d'autres domaines. Merci papa pour ton entrain, ta verve, tes anecdotes de pêche, toutes les vacances que tu nous organises et qui sont de véritables bulles d'air pour moi. Merci ma petite maman, pour ta tendresse sans faille, ton soucis de notre bien-être permanent et ton goût pour la lecture dont j'essaie de m'inspirer. Merci Michèle et Jojo pour votre compagnie toujours appréciée et pour avoir occupé mon confinement avec vos superbes cours de danse en ligne. Merci à toute ma famille dans son ensemble, et en particulier la family tribe, Hélène, Pascal, Benoît, Claire, Geoffroy, Lié, et surtout Philippe, toi qui a pris du temps pour relire mon article avec soin en faisant valoir ton expertise de la grammaire anglaise.

Je remercie pour finir trois professeurs qui m'ont profondément marqué et ont également grandement influencé ma scolarité : d'abord Monsieur Chériaux (avec un x parce qu'il y en a plusieurs), mon formidable professeur de CM1-CM2, un des hommes les plus cultivés et passionnants qu'il m'ait été donné de rencontrer, ensuite Monsieur Consigli qui a laissé une trace indélébile sur mes années collège. Je dirais que si Monsieur Chériaux m'a fait aimé l'école, Monsieur Consigli m'a fait aimé les maths. Merci également à Monsieur Jaffrenoux, qui a révolutionné mon année de terminale, qui m'a absolument convaincu de faire une classe prépa, ce qui a indirectement conduit à ma thèse.

Je termine cette première et remuante expérience des remerciements avec une petite larme à l'oeil et souhaite à présent une bonne lecture de la suite aux plus courageux d'entre vous.

Résumé

Les métamodèles sont très largement utilisés dans l'industrie pour prédire la sortie des codes de calcul coûteux. Comme ces codes de calcul font intervenir une grande quantité de variables d'entrée, créer directement un grand métamodèle dépendant de l'ensemble des entrées apparait trop ambitieux. Les industriels choisissent par conséquent de procéder séquentiellement. Ils réalisent des études en plusieurs étapes avec des métamodèles se concentrant sur des ensembles de variables de plus en plus grands. Les variables non prises en compte sont fixées à une valeur nominale. La dimension de l'espace des entrées grandit à chaque étape. Cependant, l'information obtenue aux étapes précédentes est perdue car un nouveau plan d'expérience est généré pour construire le métamodèle. Dans cette thèse, une approche alternative est introduite, utilisant tous les plans d'expériences générés depuis le début plutôt que seulement celui de l'étape en cours. Ce métamodèle utilise la régression par processus Gaussiens et est appelé seqGRP (sequential Gaussian process regression). A chaque étape, la sortie est modélisée par la somme de deux processus : le processus qui modélisait la sortie à l'étape précédente et un processus correctif. Le premier est défini sur le sous-espace d'entrée de l'étape précédente tandis que le deuxième est défini sur le sous-espace de l'étape en cours. Le processus correctif représente l'information apportée par les variables libérées à l'étape concernée. Il a la particularité d'être nul sur le sous-espace de l'étape précédente pour assurer la cohérence de la modélisation entre les étapes. Premièrement, des candidats pour les processus correctifs sont proposés, qui ont la particularité d'être nuls sur un continuum de points. Ensuite, un algorithme d'EM (Expectation-Maximization) est implémenté pour estimer les paramètres des processus. Enfin, le métamodèle seqGPR est comparé à un métamodèle de krigeage classique qui modélise la sortie par un processus Gaussien stationnaire. La comparaison est faite sur deux exemples analytiques, un en deux étapes allant jusqu'à la dimension 4, un autre en trois étapes allant jusqu'à la dimension 15. La méthodologie introduite est également évaluée sur un exemple industriel allant de la dimension 11 à la dimension 15. Dans tous ces cas test, le métamodèle seqGPR a de meilleures performances, ou tout du moins est aussi bon que le krigeage. En parallèle, une méthodologie est proposée pour construire les échantillons d'entraînement du métamodèle. Enfin, deux problèmes complémentaires sont abordés : la présence de plusieurs plans d'expérience sur différents sous-espaces à chaque étape, et l'enrichissement des plans d'expérience.

Mots clés : Métamodèle, Krigeage, Régression par processus Gaussiens, Grande dimension, Conditionnement infini, Multifidélité, Espaces imbriqués, Problèmes d'espace d'entrées de taille variable.

Abstract

Metamodels are widely used in industry to predict the output of an expensive computer code. As industrial computer codes involve a large amount of input variables, creating directly one big metamodel depending on the whole set of inputs may be a very challenging problem. Industrialists choose instead to proceed sequentially. They build metamodels depending on nested sets of variables (the variables that are set aside are fixed to nominal values), i.e. the dimension of the input space is progressively increased. However, at each step, the previous piece of information is lost as a new Design of Experiment (DoE) is generated to learn the new metamodel. In this thesis, an alternative approach is introduced, based on all the DoEs rather than just the last one. This metamodel uses Gaussian process regression and is called seqGPR (sequential Gaussian process regression). At each step, the output is supposed to be the realization of the sum of two independent Gaussian processes. The first one models the output at the previous step. It is defined on the input space of the previous step which is a subspace of the one of the current step. The second Gaussian process is a correction term defined on the input space of the current step. It represents the additional information provided by the newly released variables. The correction term has the particularity of being null on the subspace of the previous step so that there is a coherence between the steps. Firstly, some candidate Gaussian processes for the correction terms are suggested, which have the property of being null on an infinite continuous set of points. Then, an EM (Expectation-Maximization) algorithm is implemented to estimate the parameters of the processes. Finally, the metamodel seqGPR is compared to a classic kriging metamodel where the output is assumed to be the realization of one second order stationary Gaussian process. The comparison is made on two analytic examples, a first one with two steps, up to dimension 4, and a second one with three steps, up to dimension 15. The introduced methodology is also tested on an industrial example which goes from dimension 11 to dimension 15. In all these test cases, seqGPR performs better than, or at least as well as kriging. A methodology is suggested to build the samples used for the seqGPR metamodel. Two complementary issues are tackled: the presence of multiple designs in different subspaces at each step, and the enrichment of the training samples.

keywords: Metamodel, Kriging, Gaussian process regression, High dimension, Infinite conditioning, Multifidelity, Nested spaces, Variable-size design space problems.

Chapter 1 Introduction

This thesis is part of a collaboration between Institut Camille Jordan and Valeo. This collaboration was originally materialized by the ANR project PEPITO (Plans d'Experience Pour l'Industrie du Transport et l'Optimisation) which led to a first paper [START_REF] Henner | Sampling strategies for metamodel enrichment and automotive fan optimization[END_REF] (see appendix 9.5.1) about enrichment strategies for metamodels. A part of the thesis work has been the implementation of R routines using the package RcppArmadillo, and the submission of another paper [START_REF] Gonon | Gaussian process regression on nested spaces[END_REF] (see appendix 9.5.2). This chapter describes the industrial problem which motivates the work (see section 1.1), the choosen model to answer the problem (see section 1.2) and the issues that it creates (see section 1.3).

Industrial motivation

In Valeo, numerical codes are used to model physical phenomena involved in manufacturing. The industrial product which motivates this thesis work is the fan system. It is part of the car engine cooling system (see figure 1.1). This cooling system is composed of a cold fluid circulating in part in the car engine to regulate its temperature and in part in a radiator where it is itself cooled down. When the car moves, the wind generated by the car speed and reaching the radiator is sufficient to evacuate the heat from the fluid. When the car is motionless, the fan is activated to replace the wind.

Computer experiments make the search for performance easier and cheaper than physical experiments. However, computer codes are confronted to some limitations. As they model complex physical phenomena, they are often computationally expensive. One simulation of the computer code modelling the aerodynamical outputs of the fan system lasts 1h to 1h30 on 256 CPU. For the accoustical outputs, the simulation goes to 3 to 5 days on 1500 CPU. This problem of expensive time duration is solved by the use of metamodels [Forrester et al., 2008], [Sacks et al., 1989]. A metamodel is a simpler statistical model, like radial basis neural network [Cheng and Titterington, 1994], kriging model [Santner et al., 2003a], [Roustant et al., 2012], support vector regression [Clarke et al., 2005]. This simpler model is fit on a sample of well-chosen runs, also called design of experiments (DoE) [Pronzato and Müller, 2012], [Dupuy et al., 2015]. Metamodels provide fast approximations of the code outputs. Besides, they have a proactive role as they help to select relevant simulations to be run [Jones et al., 1998]. The computer code is then used more efficiently. Therefore, to study the fan system, Valeo experts use a mixed approach which combines the use of a computational fluid mecanics (CFD) computer code with designs of experiments and metamodels. Figure 1.2: Visualization of the environment in which is modeled the fan system in the simulation code. The fan system is isolated from the engine cooling system. An air generator imposes a flowrate upstream from the fan. [Valeo, ] Building surrogate models can be difficult when numerical codes involve lots of variables. The output considered in this example is the Pressure difference (∆P ) between the upstream and the downstream of the air flow crossing the fan. The fan system is modeled independently from the cooling system context, as shown in figure 1.2. Among the variables studied on the fan system, one input is the flowrate (Q), which is, in this context, not delivered by the fan itself, but imposed upstream from it. The others are of geometric nature. The fan blade geometries are supposed identical to each other. The geometry is monitored at 5 different sections (see figure 1.3a). At each section, the stagger angle and the chord length are controlled (see figure 1.3b). The chord is the line formed by the two borders of the blade at the considered section. The stagger angle is the angle between the chord and the rotation axis. The sweep is manipulated at each section and shown in panel 1.3c. It is defined as the distance between the black line, formed by the rotation axis and the leading edge (right border on the figure) at the first section, and the leading edge at the considered section. At section 1, it is always equal to 0 and therefore not kept as an input. Other variables are involved like thickness, or cambers, more sections can be defined etc. Dimension reduction is a common approach to deal with a great number of variables. It consists in considering only the influential inputs. The influential inputs are often detected by sensitivity analysis methods ( [START_REF] Saltelli | Sensitivity Analysis[END_REF], [Sobol, 1993], [Iooss and Prieur, 2019]). However, this approach leads to a vicious circle. Indeed, on one side, the variables selected by sensitivity analysis are relevant only if the metamodel is sufficiently accurate, and on the other side, the dimension reduction is used to increase the metamodel accuracy.

Valeo experts chose an alternative method that consists in a study of several steps, which was defined in the PEPITO ANR project. The first step focuses on a small amount of variables deemed as important based on expert knowledge. The other variables are set to nominal values. A metamodel is created and used only for these variables until the understanding of their influence is sufficient. Then, to obtain better performances, new steps are proceeded with, in which new variables are added progressively. For example, in the case of the fan system, a first step of the study is done only on the flowrate, staggers and chords. Then, the second step also takes into account the sweeps. Further steps can be carried out with the thicknesses, cambers etc. and so on up to 30 variables.

The classical metamodeling approach for this kind of problem is to build a metamodel on a new independent space-filling design in the wider input space at each new step of the study. Yet, this approach may be criticized as the information provided by the previous DoE's and metamodels is disregarded. The question is then, what metamodel can be built to use optimally these ancient training samples. The idea is to use the information brought by the designs of all steps to provide a richer metamodel than the metamodel built on the design of the last step only.

Model

In the present work, an original metamodel is introduced, which takes into account the information from all the steps. The chosen approach consists in creating dependent metamodels from one step to another. The metamodels are based on Gaussian process regression [Santner et al., 2003a] [ Williams and Rasmussen, 2006] (see section 2.1 in chapter 2).

One way of taking into account the different designs could be to use a multifidelity model for which the levels are not defined on the same input space. That is what is done in [Hebbal et al., 2021] with deep Gaussian processes (see last paragraph of subsection 2.1.3 in chapter 2). Nevertheless, this model is really expensive to learn. Furthermore, the multifidelity context is not suitable since the runs obtained at the previous steps have the same accuracy level. Another way could be to define a virtual categorical input, equal to the number of the step, that influences the choice of the input variables to consider. [Pelamatti et al., 2021] defines a Gaussian process whose input space varies dynamically with this virtual categorical input. This modelization does not take into account the fact that the input sets are entwined from one step to another. This particularity can lead to a simpler modelization. The approach in this work, called seqGPR (sequential Gaussian process regression), is based on a recursive statistical model inspired by the autoregressive multifidelity model [Kennedy and O'Hagan, 2000] (see subsection 2.1.3 in chapter 2) but with the same accuracy for all the runs and with an input space of increasing dimension.

The suggested metamodel is based on a probabilistic model of the function to approximate.

It involves Gaussian processes built on nested subspaces and connected by additive relations. At a given step, the Gaussian process modeling the output on the current subspace is equal to the sum of the Gaussian process of the previous step, defined on a smaller subspace, and a correction process. As the two processes model the same function, they must coincide on the smaller subspace from the previous step. Therefore, the correction process must be null on the input subspace of the previous step, which is an infinite continuous set of points

Problematics and plan of the thesis

As regards the probabilistic model described in the previous section, one first difficulty is to build an appropriate correction term, null on a continuum of points with a numerically computable covariance kernel. One idea could be to enjoin the prediction to verify the nullity property. For example, relevant points from the concerned subspace can be added to the DoE. This technique is used in [Da Veiga and Marrel, 2012] in the context of monotonicity, boundedness, convexity constraints. One drawback is the greediness of that method. Besides, the nullity cannot be verified in the whole subspace. The correction process could be defined in a finite dimensional way, with adequate basis functions, as it is done in [Maatouk and Bay, 2017], [Lopera, 2019], [Bachoc et al., 2020], to ensure some properties of monotonicity, boundedness, convexity. Then the prediction could be computed by a Monte-Carlo scheme based on simulations using rejection sampling. Yet, this approach is greedy as the number of basis functions increases with the dimension. Instead of establishing the nullity in retrospect, it can be verified at first glance, as an intrinsic property of the correction term. In this thesis, the Gaussian process introduced in [START_REF] Gauthier | Spectral approach for kernelbased interpolation[END_REF], which uses an extension of the conditional expectation to an infinite continuous set of points, is retained. A tractable kernel is sought for this process which is then compared to other candidates verifying the nullity condition.

Once a tractable candidate for the correction term is proposed, another difficulty lies in the estimation of the hyperparameters. One way is to optimize the likelihood. This task can be numerically difficult, since the likelihood involves several sets of parameters for the different processes. One way to reduce the dimension of the optimization space is to propose nested designs as it is done in [Le Gratiet and Garnier, 2014]. In this thesis an alternative based on the expectation maximization algorithm [Friedman et al., 2001] is introduced. It allows the reduction of the dimension with limited constraints on the designs.

Special designs must be generated to be used by the seqGPR metamodel. At a given step, the design must be space-filling in the current subspace and be distant from the previous subspace, in order for the covariance matrices at stake to be invertible. A methodology must be defined to built such DoE's, inspired from the simulated annealing algorithm optimizing an LHS design for a given space-filling criterion (as in [START_REF] Morris | Exploratory designs for computational experiments[END_REF]), and taking into account the constraints proper to the metamodel seqGPR.

Chapter 2 defines the literature tools on which is based the method. The definition of the probabilistic model and the issue of building the correction processes null on infinite continuous sets of points are tackled in chapter 3. The estimation of the model parameters and the evaluation of the seqGPR method on analytical and industrial test cases are detailed in chapter 4. Adequate sampling strategies are developed in chapter 5. Chapter 6 describes some enhancements tried on the method.

Chapter 2

State of the art

This chapter details the different tools used in this thesis. Section 2.1 presents the kriging metamodel (or Gaussian process regression metamodel), from its definition to the associated results and methodologies. Section 2.2 lists the different answers existing in literature to impose some constraints on the kriging metamodel.

Generalities on the kriging metamodel

The present section defines the kriging metamodel as it is defined in [Williams and Rasmussen, 2006] and [Santner et al., 2003a]. First, some results about Gaussian vectors and Gaussian processes are recalled, that led to the definition of the kriging metamodel. Then, the parameter estimation by maximum likelihood is explained, and the EM algorithm (Expectation-Maximization) is detailed. The Multifidelity model is presented. The Sobol index used in sensibility analysis is defined. Finally, the usual sampling strategies associated with a metamodel are developped.

From Gaussian vectors to Gaussian process regression

Here are first described some theoretical results about Gaussian Vectors that are used in a kriging context.

Definition 1 (Equality in L 2 (Ω)) Let (Ω, F, P) denote a probability space. L 2 (Ω) is the set of all random variables X defined on Ω such that E[X 2 ] < +∞. Two variables X, Y ∈ L 2 (Ω) are said equal in L 2 (Ω) (X L 2 (Ω) = Y ) if E[(X -Y ) 2 ] = 0.
In the following, every variable is defined on the probability space (Ω, F, P) and all equalities between variables are equalities in L 2 . The notation

X L 2 (Ω) = Y is simplified in X = Y . Definition 2 (Gaussian Vector) A random vector Y = Y 1 • • • Y n is a Gaussian vec-
tor if every linear combination of its components is Gaussian, i.e. every

n i=1 λ i Y i (with λ i ∈ R)
is a Gaussian Variable.

Definition 3 (Conditional expectation) Let X (resp. Y ) be a Gaussian variable (resp. Gaussian vector). The expectation of X conditioned by Y , denoted by

E [X | Y ] is the random variable U solution of min U ∈span(1,Y ) E (X -U ) 2 ,
where span(1, Y ) denotes the set of all linear combinations of (1, Y ). In particular, if X and Y are centered, the conditional expectation is the orthogonal projection of X in span(Y ).

The definition of the conditional expectation can be extended if X is a vector by applying it component by component.

Proposition 1 (Explicit formula of the conditional expectation) Let V be a Gaussian vector split in two subvectors V = V 1 V 2 . The mean vector and covariance matrix of V can be defined by blocks:

   E [V ] = (m 1 , m 2 ) Cov (V , V ) = Σ 11 Σ 12 Σ 21 Σ 22 with m 1 = E[V 1 ], m 2 = E[V 2 ], Σ 11 = cov(V 1 , V 1 ), Σ 22 = cov(V 2 , V 2 ), Σ 12 = cov(V 1 , V 2 ), Σ 21 = cov(V 2 , V 1 ). The expectation of V 1 conditioned by V 2 is equal to E [V 1 | V 2 ] = m 1 + Σ 12 Σ + 22 (V 2 -m 2 ) , with Σ + 22 the Moore-Penrose generalized inverse of Σ 22 . In particular, if Σ 22 is positive defi- nite, Σ + 22 = Σ -1 22 .
It is a Gaussian vector of mean and covariance matrix:

E [E [V 1 | V 2 ]] = m 1 , Cov (E [V 1 | V 2 ] , E [V 1 | V 2 ]) = Σ 12 Σ + 22 Σ 21 .
Proof A proof of this proposition, entirely rewritten by hand, is suggested in appendix 9.2.1, as it was not encountered in the literature for the general case (this proposition is usually proven for the case Σ 22 positive-definite).

Proposition 2 (Conditional vector) Following the notations of proposition 1, the vector

[V 1 | V 2 = v] is equal to [V 1 | V 2 = v] = m 1 + Σ 12 Σ + 22 (v -m 2 ) + V 1 -E [V 1 | V 2 ] .
It is a Gaussian vector of mean and covariance matrix given by

E [V 1 | V 2 = v] = m 1 + Σ 12 Σ + 22 (v -m 2 ) cov ([V 1 | V 2 = v] , [V 1 | V 2 = v]) = Σ 11 -Σ 12 Σ + 22 Σ 21 Proof
The proof of this proposition was not found in the literature and one written by hand is suggested in appendix 9.2.2.

Proposition 3 (Multiple conditioning) Let (U, V , W ) be a Gaussian vector with U (resp. V and W ) a Gaussian variable (resp. Gaussian vectors). Let denote by

X = [U | V = v] (resp. Y = [W | V = v]) the variable U (resp. W ) conditioned by V = v, then: [X | Y = w] = [U | V = v, W = w]
Proof The proof of this proposition was not found in the literature and one written by hand is suggested in appendix 9.2.3 This last result on Gaussian vectors means that the conditioning can be carried out in any order, in one go or step by step. The next part is dedicated to Gaussian processes.

Definition 4 (Gaussian process)

A Gaussian Process Y : Ω × X → R verifies that for all n 1, for all x 1 , • • • , x n in X , Y (x 1 ) • • • Y (x n ) is a Gaussian vector.
Caracterization Y is entirely determined by its mean function E[Y (x)] = m(x) and its covariance kernel cov(Y (x), Y (t)) = k(x, t).

Usual covariance kernels

The following stationary tensor product kernels are often used in practice in the kriging context:

• Exponential kernel: k(x, t) = σ 2 i exp - |x i -t i | θ i • Matern 5 2 kernel: k(x, t) = σ 2 i 1 + √ 5 |x i -t i | θ i + 5(x i -t i ) 2 θ 2 i exp - √ 5 |x i -t i | θ i • Gaussian kernel: k(x, t) = σ 2 i exp - (x i -t i ) 2 θ 2 i
Definition 5 (Gaussian space) Let (Y (x)) x∈X a Gaussian process indexed by X . The Gaussian space engendered by Y is the closure of all linear combinations of the Y (x) (x ∈ X ).

Let f : X → R denote the output of an expensive computer code. A metamodel is used in order to approximate f . It is a simpler model quicker to evaluate. The kriging metamodel, or Gaussian process regression metamodel, is based on the assumption that the output f is the realization (a path) of a Gaussian Process Y . The Gaussian process is usually chosen second-order stationary: with constant mean m and stationary covariance kernel k

(such that k(x + h, t + h) = k(x, t)).
Then, given a sample of points X =   

x (1) . . .

x (n)    ⊂ X for
which we know the output value (f (X) = y), the prediction mean and variance of the kriging metamodel are given by:

ŷ(x) = E [Y (x) | Y (X) = y] = m + k(x, X)k(X, X) -1 (y -m1 X ), v(x) = V ar (Y (x) | Y (X) = y) = k(x, x) -k(x, X)k(X, X) -1 k(X, x), (2.1)
where 1 X is the column vector full of ones of size equal to the size of X. These formulae are a direct application of the formulae seen in proposition 1, considering the Gaussian vector Y (x) Y (X) .

Maximum Likelihood estimation and EM (Expectation -Maximization) algorithm

Knowing the distribution parameters of Y and some training data, the mean and variance of prediction of the corresponding kriging metamodel can be computed. But the parameters of Y are unknown a priori, so they first need to be infered from the training data. The parameter estimation can be done by maximum likelihood or cross-validation. In this thesis, the retained estimation method is the maximum likelihood. The method is described in this subsection, along with the EM algorithm which helps optimizing it in particular contexts.

Maximum Likelihood estimation

The parameters of Y need to be estimated from the training data Y (X) = y. To do that, the likelihood, which is a function of the parameters η, is maximized. In rough terms, maximizing the likelihood can be interpreted as finding the set of parameters that maximizes the probability that the training data occurs. The likelihood taken at the set of parameters η (noted L(η; y)) is equal to the density of the vector Y (X) (assuming Y has the parameters η), applied at the observed values y.

L(η; y) = h Y (X);η (y).

In the usual case, the output f is assumed to be a simulation of the Gaussian process

Y ∼ GP(m * , (σ * ) 2 r θ * (x, x )) of parameters η * = (m * , (σ * ) 2 , θ * ).
Denoting by n X the number of points (rows) of X, and by 1 X the unit column vector of size n X , the likelihood is equal to:

L(η; y) = 1 (2πσ 2 ) n X 2 √ |r θ (X,X)| exp -1 2 (y-m1 X ) r θ (X,X) -1 (y-m1 X ) σ 2
, where |r θ (X, X)| denotes the determinant of the matrix r θ (X, X). Taking -2 log(L) and keeping only the terms that depend on η, the quantity that needs to be minimized is:

l(η; y) = n X log(σ 2 ) + log |r θ (X, X)| + (y -m1 X ) r θ (X, X) -1 (y -m1 X ) σ 2 .
The optimal values of m and σ 2 are:

       m(θ) = 1 X r θ (X, X) -1 y 1 X r θ (X, X) -1 1 X σ 2 (θ) = (y -m(θ)1 X ) r θ (X, X) -1 (y -m(θ)1 X ) n X .
Finally, finding (m, σ 2 , θ) maximizing the likelihood is equivalent to finding θ minimizing:

l(θ; y) = n X log σ 2 (θ) + log |r θ (X, X)| .
This optimization is done numerically.

EM algorithm

The likelihood is often maximized numerically. This paragraph describes the EM algorithm as presented in [Friedman et al., 2001], which is an algorithm used to maximize the likelihood in the case where some observations are missing (which is of interest in the thesis).

The following notations are used:

• Z a random variable corresponding to the observed data and z, a scalar corresponding to its observation (which is known). The distribution of Z is parameterized by η * .

• Z m the missing data and z m the corresponding observation (which is unkown).

• T = (Z, Z m ) the complete data and t = (z, z m ) the corresponding observation.

It is assumed that there exists a σ finite measure µ such that all T laws admit a density f T (t) with respect to µ. So Z admits a density f Z (z) with respect to µ. The loglikelihood of the observed data at η is given by: l(η; z) = log(h Z;η (z)).

This loglikelihood can be difficult to optimize. The loglikelihood of the complete data is rather used, it is given by: l 0 (η; t) = log(h T ;η (t)). As t is not known entirely, the random variable l 0 (η; T ) = log(h T ;η (T )) is used instead of l 0 (η; t). The expectation part of the EM algorithm consists in defining the expectation of this variable conditionally to Z = z with the assumption that the Z distribution is parameterized by η:

Q(η, η) = E η [l 0 (η, T ) | Z = z]
The Maximization part of the EM algorithm consists in maximizing this quantity with respect to η.

• Iteration 0: Generate initial guess η(0) for η.

• Iteration k + 1: Compute η(k+1) as solution of max η Q(η, η(k) ) Proposition 4
The likelihood increases at each iteration of the EM algorithm Proof The proof, which is let in exercise in [Friedman et al., 2001], is detailed in appendix 9.2.4.

Multifidelity

This subsection describes the multifidelity metamodel in the context of Gaussian process regression (see [Kennedy and O'Hagan, 2000]). The multifidelity metamodel takes into account different levels of accuracy of the computer code. The low levels (resp. high levels) of accuracy are called low fidelity levels (resp. high fidelity levels), they are generally cheaper to evaluate (resp. more expensive to evaluate).

Metamodel For sake of simplicity, it is assumed that two levels of fidelity of the computer code are available. The low level is denoted by f 1 : X → R , and the high level by f 2 : X → R . The level of fidelity can result from the degree of refinement of the mesh in a finite element code, or the number of iterations in a Monte-Carlo based simulation.

f 1 (resp. f 2 ) is assumed to be the realization of a Gaussian process Y 1 : Ω × X → R (resp. Y 2 : Ω × X → R ). Y 2 is linked to Y 1 by the following formula: Y 2 (x) = g(x)Y 1 (x) + Y c (x), ∀x ∈ X . g : X → R is a known deterministic function and Y c : Ω × X → R is a Gaussian process, independent of Y 1 ,
and acting as a correcting term. The model is also valid if g = i β i f i with (β i ) i parameters to estimate and (f i ) i deterministic functions.

The training data for the kriging metamodel is composed of one sample X 1 ⊂ X on which the low fidelity level is observed f 1 (X 1 ) = y 1 , and another X 2 ⊂ X on which the high fidelity level is observed f 2 (X 2 ) = y 2 . The mean and variance of prediction at x ∈ X of the multifidelity metamodel are the moments of the variable

[Y 2 (x) | Y 1 (X 1 ) = y 1 , Y 2 (X 2 ) = y 2 ]: ŷ(x) = E [Y 2 (x) | Y 1 (X 1 ) = y 1 , Y 2 (X 2 ) = y 2 ] v(x) = V ar (Y 2 (x) | Y 1 (X 1 ) = y 1 , Y 2 (X 2 ) = y 2 )
Maximum likelihood estimation of the parameters The following notations are used:

• η 1 = (m 1 , θ 1 ) denotes the set of parameters of Y 1 ∼ GP (m 1 , k θ 1 (x, x )). η c = (m c , θ c )
denotes the set of parameters of Y c ∼ GP (m c , k θc (x, x )). η = (η 1 , η c ) denotes the total set of parameters.

• n 1 and n 2 are respectively the sizes of X 1 and X 2 .

•

Σ θ 1 is the covariance matrix of Y 1 on X 1 ∪ X 2 . Σ θc is the covariance matrix of Y c on X 2 :    Σ θ 1 = k θ 1 (X 1 , X 1 ) k θ 1 (X 1 , X 2 ) k θ 1 (X 2 , X 1 ) k θ 1 (X 2 , X 2 ) Σ θc = k θc (X 2 , X 2 )
• 1 n denotes the unit column vector of size n. diag(u) denotes the diagonal matrix whose diagonal components are the components of the vector u. 0 nm is the null matrix with n rows and m columns.

If the designs are nested (X 2 ⊂ X 1 ), then g(X 2 ) • Y 1 (X 2 ) = u 1 is observed. The likelihood becomes L(η; y 1 , y 2 ) = h Y 1 (X 1 ),g(X 2 )•Y 1 (X 2 )+Yc(X 2 ) (y 1 , y 2 ) = h Y 1 (X 1 ),u 1 +Yc(X 2 ) (y 1 , y 2 ) = h Y 1 (X 1 ),Yc(X 2 ) (y 1 , y 2 -u 1 ) = h Y 1 (X 1 ) (y 1 ) =L 1 (η 1 ;y 1 ) h Yc(X 2 ) (y 2 -u 1 ) =Lc(ηc;y 2 -u 1 )
as

Y 1 ⊥ Y c
The likelihood is decoupled. η 1 and η c can be estimated seperately by maximizing respectively L 1 (η 1 ; y 1 ) and L c (η c ; y 2u 1 ). Similarly, the prediction is also decoupled:

ŷ(x) = E [Y 1 (x) | Y 1 (X 1 ) = y 1 ] + E [Y c (x) | Y c (X 2 ) = y 2 -u 1 ] v(x) = V ar (Y 1 (x) | Y 1 (X 1 ) = y 1 ) + V ar (Y c (x) | Y c (X 2 ) = y 2 -u 1 ) If X 1 ∩ X 2 = ∅, neither Y 1 (X 2 ) nor Y c (X 2 )
is observed. This time, the prediction and the likelihood are not decoupled. The joint likelihood must be maximized:

L(η; y 1 , y 2 ) = h Y 1 (X 1 ),g(X 2 )•Y 1 (X 2 )+Yc(X 2 ) (y 1 , y 2 )
[ Zertuche, 2015] suggests to use an EM algorithm (see subsection 2.1.2) in this case.

Proposition 5 In the case of multifidelity with not nested designs, the EM algorithm is the following

• Iteration 0: Generate initial guess η(0) = (η (0) 1 , η(0) c ). • Iteration k + 1: -Compute η(k+1) 1 as solution of max η 1 Q 1 (η 1 , η(k) ) -Compute η(k+1) c as solution of max ηc Q c (η c , η(k) ) with                            Q 1 (η 1 , η ) = n 1 +n 2 2 log(2π) -1 2 log(|Σ θ 1 |) -1 2 tr (k θ 1 (X 2 , X 2 ) -k θ 1 (X 2 , X 1 )k θ 1 (X 1 , X 1 ) -1 k θ 1 (X 1 , X 2 )) -1 ∼ Σ η -1 2 (y 1 -m 1 1 n 1 ) ∼ µ η -m 1 1 n 2 Σ -1 θ 1 y 1 -m 1 1 n 1 ∼ µ η -m 1 1 n 2 Q c (η c , η ) = -n 2 2 log(2π) -1 2 log(|Σ θc |) -1 2 tr Σ -1 θc diag(g(X 2 )) ∼ Σ η diag(g(X 2 )) -1 2 Y 2 -g(X 2 ) • ∼ µ η -m c 1 n 2 Σ -1 θc Y 2 -g(X 2 ) • ∼ µ η -m c 1 n 2 and                                              ∼ µ η = m 1 1 n 2 +       k θ 1 (X 2 , X 1 ) k θ 1 (X 2 , X 2 )diag(g(X 2 )) × k θ 1 (X 1 , X 1 ) k θ 1 (X 1 , X 2 )diag(g(X 2 )) diag(g(X 2 ))k θ 1 (X 2 , X 1 ) diag(g(X 2 ))k θ 1 (X 2 , X 2 )diag(g(X 2 )) + k θ c (X 2 , X 2 ) × Y 1 -m 1 1 n 1 Y 2 -m 1 g(X 2 ) -m c 1 n 2       ∼ Σ η = k θ 1 (X 2 , X 2 ) -        k θ 1 (X 2 , X 1 ) k θ 1 (X 2 , X 2 )diag(g(X 2 )) × k θ 1 (X 1 , X 1 ) k θ 1 (X 1 , X 2 )diag(g(X 2 )) diag(g(X 2 ))k θ 1 (X 2 , X 1 ) diag(g(X 2 ))k θ 1 (X 2 , X 2 )diag(g(X 2 )) + k θ c (X 2 , X 2 ) × k θ 1 (X 1 , X 2 ) diag(g(X 2 )k θ 1 (X 2 , X 2 )       
Proof The proof of this proposition, given in [Zertuche, 2015], is recopied in appendix 9.2.5.

Multifidelity with different input spaces for each level

This paragraph describes a multifidelity model whose levels of fidelity are not defined in the same input space. Let X 1 denote the input space of f 1 , and X 2 the input space of f 2 . A multi-fidelity model using deep Gaussian processes has been proposed in [Hebbal et al., 2021]. The output is modelled by

Y (x) = Y 2 (Y 1 (x)), if x ∈ X 1 Y 2 (Y 1 (Ψ 2→1 (x))), if x ∈ X 2 Y 1 : X 1 → R is a Gaussian process representing the low fidelity level. Y 2 : X 2 → R
is a Gaussian process representing the high fidelity level. Ψ 2→1 : X 2 → X 1 is a multioutput Gaussian process acting as a mapping between the two input spaces. This approach is very complex to implement as it requires a heavy computational machinery.

Sobol index

Sensitivity analysis (see [Da Veiga et al., 2021]) aims at classifying variables by influence on the output. To do so, a well-known criterion is the Sobol index. If f (x 1 , • • • , x d ) is the output, the Sobol index of the variable x i is defined by:

S i = V ar (E [f (X 1 , • • • , X d ) | X i ]) V ar(f (X 1 , • • • , X d )) ,
where X 1 , • • • , X d are independent random variables of uniform distribution in [0, 1]. The Sobol index S i computes the proportion of the output variance explained by the input X i . This index has another formula which is easier to implement numerically with a Monte-Carlo scheme.

Proposition 6

The Sobol index S i can be rewritten:

S i = Cov(f (X), f ( ∼ X)) V ar(f (X))
,

where X = (X 1 , • • • , X i-1 , X i , X i+1 , • • • , X d ) and ∼ X = ( ∼ X 1 , • • • , ∼ X i-1 , X i , ∼ X i+1 , • • • , ∼ X d ).
The variable ∼ X j is independent of same law than X j .

Proof The proof of this proposition is given in [Le Gratiet, 2013a] and recopied in appendix 9.2.6 for the reader. , a usual criterion to use is the RMSE (Root Mean Square Error) defined by:

Design of Experiments

RM SE = ntest i=1 (y (i) test -ŷ(x (i) test )) 2 n test (2.
2)

The initial sample on which is built the kriging model must be space filling in order to optimize the learning of the response surface. Different types of DoE's (from [Santner et al., 2003b]) are presented in what follows, with constraints and criteria to optimize.

Latin Hupercube Sampling A very commonly used type of design is the LHS (latin hypercube sample). The purpose of such a design is that the projections of its points in every direction are sufficiently spaced and space filling in the corresponding direction. Basically an LHS design is of the form

X =   π 1 (u 1 ) π d (u 1 ) | • • • | π 1 (u n ) π d (u n )   , where u = (u 1 , • • • , u n ) is an equispaced discretization of [0, 1] and π 1 , • • • , π d are permuta- tions on u. For example, based on the following discretization u = u 1 = 1 6 , u 2 = 1 2 , u 3 = 5 6 , the following LHS can be built on X = [0, 1] 2 : X 1 =   u 1 u 1 u 2 u 2 u 3 u 3   , X 2 =   u 1 u 1 u 2 u 3 u 3 u 2   , X 3 =   u 1 u 3 u 2 u 1 u 3 u 2   .
All LHS are not necessarily space filling. The choice of the permutations π i can be restricted to have better space-filling properties Orthogonal Latin Hypercube (OLH) Sampling An OLH is an LHS which is subject to constraint. The input space is divided in subspaces of same size (the 4 corners in the 2D example below). The points of the OLH must be in the same proportion in all the subspaces.

LHS OLH

Those kind of samples are difficult to built. A simpler way to build a space-filling LHS is to choose the permutations π i by optimizing an additional space filling criterion. The optimization is usually done by simulated annealing.

Space filling criteria

• Maximin: the goal is to maximize the minimal distance between two points of the design max X min

x,t∈X

xt 2

• Generalized maximin: this criterion, which has to be minimized, takes into account the distances between all pairs of points in the design:

min X x,t∈X 1 x -t 2 p 1 p .
It is reduced to the maximin criterion when p → +∞.

• Minimax: the goal is to minimize the biggest distance between a point of X and a point of X :

min X max x∈X,t∈X
xt 2

• Discrepancy: this criterion, which has to be minimized, measures the uniformity of the distribution of the points of X:

min X sup J∈J card(X ∩ J) n -λ(J)
where J is the set of all hyperrectangles included in X , and λ is the Lebesgues measure.

The discrepancy and minimax criteria are very expensive to compute for a given sample so their optimization is too complex. In practice, the maximin and generalized maximin are used. Alone, they tend to select points at the border of X especially when the dimension of X is high (because of the curse of dimensionality), but combined with the latin hypercube constraint, they give suitable designs.

After having trained a kriging model on an initial sample, new points can be added to improve the accuracy of the metamodel. There exists methods to add points relevant to solve problems of optimization [Jones et al., 1998] or inversion [START_REF] Picheny | Adaptive designs of experiments for accurate approximation of a target region[END_REF]. However, here are only presented enrichment methods to improve the accuracy of the metamodel in the whole input space. The procedure consists in adding one or several points optimal for some criterion.

MSE (Mean Squared Error)

The additional point maximizes the variance of prediction (defined in equation (2.1)):

max x∈X v(x)
It is a space filling criterion. As the variance of prediction increases as the point is far from the DoE, this procedure will add points that will complete the wholes in the DoE. The problem of this method is that the points selected to enrich the sample are often located in the corners of the input space. In the paper [START_REF] Henner | Sampling strategies for metamodel enrichment and automotive fan optimization[END_REF] (see appendix 9.5.1), a methodology is proposed, that selects points optimal for the MSE criterion in a restricted zone around the center point (the input space is [-1, 1] d and the restricted zone is [-0.5, 0.5] d ).

IMSE (Integrated Mean Squared Error)

The point chosen is the one which, when added, minimizes the integrated variance

min x X v∪{x} (t)dt
with v∪{x} (t) the variance of prediction of the kriging metamodel enriched with the point x.

LOO-CV The two previous criteria are only spatial criteria (favouring points far from the training sample) as they are based on the prediction variance. An interesting idea would be to have a criterion that takes into account both the space and the output evolution.

[ [START_REF] Gratiet | Multi-fidelity Gaussian process regression for computer experiments[END_REF] suggests an enrichment procedure for the DoE, based on the maximization of the following criterion:

max x∈X v(x) • 1 + n i=1 (y i -ŷ-i (x (i) )) 2 v-i (x (i) ) 1 x∈V i
This criterion is the product of two criteria. The first factor is the prediction variance v(x).

The second factor is associated to the output evolution, it favors points in the zones where the output behavior is not correctly predicted by the metamodel. y i is the observation of the output at the training point x (i) . ŷ-i and v-i are respectively the mean and variance of prediction of the kriging metamodel trained on the sample without this point X\{x (i) }. V i is the Voronoï cell corresponding to x (i) , i.e. the points that are closer to x (i) than to the other training points. At a given point x ∈ X , the criterion is equal to v

(x) 1 + (y i -ŷ -i (x (i) )) 2 v-i (x (i) )
with i the index of the training point x (i) which is the closest to x. This criterion can be seen as an adjusted kriging variance which better estimates the prediction error of the kriging model. Indeed, in the Voronoï cells where the variance supposedly underestimates the prediction error, that means when

(y i -ŷ-i (x (i) )) 2 >> v-i (x (i) ), the term 1 + (y i -ŷ -i (x (i) )) 2 v-i (x (i) )
automatically increases the value of the criterion, which as a result is superior to the initial variance. Conversely, in the zones where the prediction error is small:

(y i -ŷ-i (x (i) )) 2 << v-i (x (i)
), the criterion is close to the initial variance v(x).

kriging under constraints : Gaussian process null on any subset

The goal of this section is to review the literature handling directly or indirectly the definition of a Gaussian process null on a part of the input space. Let Z denote such a Gaussian process, defined on X Z and null on a subset D ⊂ X Z :

Z(x) = 0, ∀x ∈ D ⇔ Z(D) = 0. (2.3)
Three methods of the literature to impose that constraint (or similar ones) are presented in this section. Two methods that are applied for more general constraints (monotonicity, convexicity, boundedness) are described and a method directly dedicated to the nullity constraint is presented.

Imposing the constraint a posteriori

The purpose of the kriging metamodel is the prediction, so the implementation of

E [Z | Y = y]
with Y = y some training data. The two methods of the literature described in this subsection work at, amongst others, computing a similar formula, which is of the form

E [W (x) | a W (D) b],
i.e. the expectation of the process W conditioned to be bounded on D.

Discretizing the nullity domain In the paper [Da Veiga and Marrel, 2012], this expectation is first approximated by discretizing D in D (where points in D are choosen uniformly within D). The goal becomes to compute the expectation

E [W (x) | a W (D) b]
. This is the expectation of a truncated normal distribution, whose formula is known analytically in its integral form. The paper then suggests to compute the integrals with adapted numerical approximations. Here, as the inequality constraints are equalities, the expectation is directly computable with the usual formula (see proposition 1).

The drawback of this method is the approximation of D in D which does not able to strictly verify the nullity constraint everywhere in D.

Using a finite dimensional GP [Maatouk and Bay, 2017], [START_REF] López-Lopera | Finite-dimensional gaussian approximation with linear inequality constraints[END_REF], and [Bachoc et al., 2020] tackle the computation of the same expectation

E [W (x) | a W (D) b],
but this time the inequality constraint must be verified in the whole input space (D = X W ). The process W (x) is approximated in a finite dimensional way by Ŵ (x). The method is generalized for any dimension of the input space but is here recalled in 1D for the sake of understanding:

Ŵ (x) = m j=0 ξ j h j (x),
where ξ = (ξ 0 , . . . , ξ m ) is a centered Gaussian vector of covariance matrix denoted by Γ = (k(x (i) , x (j) )) i,j , with k the kernel of W . h = (h 0 , . . . , h m ) is a set of basis functions defined by : The first drawback of this method is that it is expensive in high dimension. Some work has been done in [Bachoc et al., 2020] to apply it at a reduced cost in high dimension but only works if there is a small amount of influential inputs. The second drawback is that the constraint is applied to the whole domain X W , and its adaptation to a subset D does not seem straightforward.

h j (x) = 1 - x -x (j) ∆ m 1 x-x (j)

2.2.2

Imposing the constraint a priori with a conditional GP [Gauthier, 2011] directly suggests a candidate for the process Z null on D. This process is presented in definition 6.

Definition 6 Let

∼ Z : Ω × X Z → R be a centered Gaussian process of law: ∼ Z ∼ GP (0, k(x, x )), with k a positive definite kernel. The Gaussian process defined in [Gauthier, 2011] is equal to:

Z(x) = ∼ Z(x) -E ∼ Z(x) | ∼ Z(t), ∀t ∈ D , ∀x ∈ X Z ,
where the expectation part is the orthogonal projection of ∼ Z(x) on the Gaussian space generated by ∼ Z(t), t ∈ D .

Formula of the expectation part for any D Proposition 7 D is supposed to be indexed by a set S: D = {x s , s ∈ S}. Let ν be σ-finite measure on S.

• The expectation part E ∼ Z(x) | ∼ Z(t), ∀t ∈ D is given by: E ∼ Z(x) | ∼ Z(s), ∀t ∈ D = +∞ n=1 φ n (x) S ∼ φ n (s) ∼ Z(x s )dν(s),
with φ n : X → R suh that:

φ n (x) = 1 λ n S k(x, x s ) ∼ φ n (s)dν(s),
and (λ n , ∼ φ n ) solutions of the following eigen problem:

S k(s, u) ∼ φ n (u)dν(u) = λ n ∼ φ n (s), ∀s ∈ S.
The ∼ φ n : S → R are such that:

S ∼ φ n (s) ∼ φ m (s)dν(s) = δ nm , ∀n, m ∈ N\{0},
with δ nm is the Kronecker symbol (δ nn = 1 and δ nm = 0 if n = m).

• The expectation

E ∼ Z(x) | ∼ Z(t), ∀t ∈ D is a centered Gaussian process of covariance kernel: κ(x, t) = +∞ n=1 λ n φ n (x)φ n (t).
Proof See [START_REF] Gauthier | Spectral approach for kernelbased interpolation[END_REF]]. An alternative proof using the Karhunen-Loeve decomposition is detailed in appendix 9.2.7.

Formula for finite D Proposition 8 If D = D is a finite subset of X , then, the expectation part of the process Z coincides with the classical formula of the conditional expectation (see proposition 1):

E ∼ Z(x) | ∼ Z(D) = k(x, D)k(D, D) -1 ∼ Z(D).
It is a centered Gaussian process of covariance kernel given by:

κ(x, t) = k(x, D)k(D, D) -1 k(D, t)
Thus, the process Z coincides with the formula of proposition 2:

Z(x) = ∼ Z(x) -k(x, D)k(D, D) -1 ∼ Z(D) = ∼ Z(x) | ∼ Z(D) = 0 .
Proof The proof of this proposition, given in [Gauthier and Bay, 2012a], is recalled in appendix 9.2.8.

In fact, the process

E ∼ Z | ∼ Z(D) (resp. ∼ Z -E ∼ Z | ∼ Z(D)
), whose formula is given in proposition 7 (resp. definition 6) for any D, is the generalization of the conditional expectation given in proposition 1 (resp. the conditional process

∼ Z | ∼ Z(D) = 0 given in proposition 2)
which is defined for finite D.

Chapter 3

Probabilistic model

This chapter has two objectives. Firstly, the goal is to suggest a sequential model based on Gaussian processes (called seqGPR) to take into account the results of numerical simulations obtained in spaces of increasing dimension. This model, inspired from the autoregressive multifidelity metamodel of [Kennedy and O'Hagan, 2000], involves processes null on a continuum of points. The second goal is to propose numerically tractable kernels for that kind of processes. For that purpose, the work of [Gauthier, 2011] is used and extended and other kernels are defined. Section 3.1 defines the probabilistic model on which is based the seqGPR metamodel. Three candidates for the correction processes involved in the model are detailed in section 3.2. The candidates are compared in terms of path shapes and parameter influence in section 3.3. Section 3.4 gives the likelihood formulae for each candidate and compares their parameter estimation by maximum likelihood.

Model

In this section, a formal probabilistic model of the problem is proposed and the notations that will be used in the rest of the thesis are defined.

General formalism

Let f : [0, 1] d 1 +...+d N → R be an output of an expensive simulation code depending on d 1 + . . . + d N variables. A response surface of f has to be built taking into account that the N -1 first steps of the study (of increasing dimensions) have already been completed. Each of these steps is a focus on the relation between the output and a subset of free variables. The other variables are temporarily fixed but then progressively released in the further steps. This thesis deals with the handling of the last step of the study, denoted by N , where all inputs are free. Let I n denote the index set of the variables that are released at step n ∈ 1, N and d n = card(I n ). For every index set I, let x I denote the associated subvector of x. The following framework, based on Gaussian process regression, is introduced to model the sequential study:

= m + Z 1 of mean m ∈ R and with Z 1 : Ω × [0, 1] d 1 → R a centered Gaussian Process of covariance kernel σ 2 1 ρ 1 . Let y 1 = f 1 (X 1
) represent the vector of observations of the output on DoE X 1 .

• At step 2, a second range of simulations is then launched on a subspace of higher dimension [0, 1]

∪•••∪I N ∈ [0, 1] d 3 +•••+d N . Let f 2 be the corresponding restriction of f on the subspace [0, 1] d 1 +d 2 .
The function f 2 is assumed to be the realization of a Gaussian process

Y 2 : Ω × [0, 1] d 1 +d 2 → R ,
Y 2 (x I 1 , x I 2 ) = Y 1 (x I 1 ) + Z 2 (x I 1 , x I 2 ), ∀(x I 1 , x I 2 ) ∈ [0, 1] d 1 +d 2 , with Z 2 a centered Gaussian process independent from Z 1 , of covariance kernel σ 2 2 ρ 2 such that for all x I 1 in [0, 1] d 1 , Z 2 (x I 1 , xI 2 ) = 0. Let y 2 = f 2 (X 2
) denote the vector of observations for this step.

• In the same way as previous steps, at step n ∈ 3, N , a DoE X n is created in the higher space [0, 1]

d 1 +•••+dn . The variables x I 1 ∪•••∪I n-1 and
x In are free and the others x

I n+1 ∪•••∪I N are set to xI n+1 ∪•••∪I N ∈ [0, 1] d n+1 +•••+d N .
The corresponding restriction of f on this subspace, f n , is evaluated on X n . The values are stored in y n = f n (X n ). The function f n is modeled as the realization of a Gaussian process

Y n : Ω × [0, 1] d 1 +•••+dn → R .
Following the same arguments as in step 2, Y n is defined as (for all (x

I 1 ∪•••I n-1 , x In ) in [0, 1] d 1 +•••+dn ): Y n (x I 1 ∪•••∪I n-1 , x In ) = Y n-1 (x I 1 ∪•••∪I n-1 ) + Z n (x I 1 ∪•••∪I n-1 , x In ), with Z n a centered Gaussian process independent from (Z 1 , • • • , Z n-1 ) of covariance kernel σ 2 n ρ n and such that Z n (x I 1 ∪•••∪I n-1 , xIn ) = 0, ∀x I 1 ∪•••∪I n-1 ∈ [0, 1] d 1 +•••+d n-1 .

Examples

Example in 2D This example will follow the course of the thesis to illustrate the method in a simple case. Let f (x 1 , x 2 ) be a function of two variables to approximate by a metamodel.

A two-step study is carried out:

• At step 1, the input x 2 is fixed to 0.7x 1 +0.2. The subspace of the input space considered at this step is shown in figure below.

Here, this subspace is not indexed by the curvilinear abscissa (a c = x 2 1 + (0.7x 1 + 0.2) 2 ) but simply by

x 1 : {(x 1 , 0.7x 1 + 0.2), x 1 ∈ [0, 1]}. The function to approximate is f 1 (x 1 ) = f (x 1 , 0.7x 1 + 0.2)
. This restriction is modeled by a Gaussian process Y 1 (x 1 ) = m+Z 1 (x 1 ) with m a scalar equal to the mean of Y 1 , and Z 1 a centered Gaussian process of covariance kernel k 1 (x 1 , t 1 ). Thus, Y 1 is a function of x 1 (and not of the curvilinear abscissa a c ). A DoE is generated

X 1 =    x (1) 1 . . . x (n 1 ) 1    containing values of x 1 .
For these points, the values of x 2 are implicitely fixed to 0.7x 1 + 0.2. The values of f 1 at the points of X 1 are known, equal to

y 1 =    f 1 (x (1) 1 ) . . . f 1 (x (n 1 ) 1 )    .
• At step 2, the entire input space [0, 1] 2 is considered. The function to approximate is now

f 2 (x 1 , x 2 ) = f (x 1 , x 2 ). It is modeled by a Gaussian process Y 2 (x 1 , x 2 ) linked to Y 1 (x 1 ) by the formula Y 2 (x 1 , x 2 ) = Y 1 (x 1 ) + Z 2 (x 1 , x 2 )
. This formula looks like the multifidelity metamodel, but the difference is that Y 1 and Y 2 model the same level of fidelity. Thus, they must coincide on the subspace x 2 = 0.7x 1 + 0.2. This coincidence, shown in the figure below, implies that Z 2 (x 1 , 0.7x 1 + 0.2) = 0. Y 1 can be seen as a function of (x 1 , x 2 ) constant in x 2 (see the middle panel). On the left and middle panels, Y 1 and Y 2 coincide on the orange line corresponding to the subspace x 2 = 0.7 1 + 0.2.

The right panel shows Y 1 as a function of x 1 .

Visualization of Y 2 Y 1 as a function of (x 1 , x 2 ) Y 1 as a function of x 1 A DoE is generated X 2 =    x (n 1 +1) 1 x (n 1 +1) 2 . . . . . . x (n 1 +n 2 ) 1 x (n 1 +n 2 ) 2    containing values of (x 1 , x 2 ).
The values of f 2 at the points of X 2 are known, equal to . The output considered is a function of 4 inputs f (x 1 , x 2 , x 3 , x 4 ). The study is composed of 3 steps.

y 2 =    f 2 (x (n 1 +1) 1 , x (n 1 +1) 2 ) . . . f 2 (x (n 1 +n 2 ) 1 , x (n 1 +n 2 ) 2 )    . An
• At step 1, x 1 is released and (x 2 , x 3 , x 4 ) are fixed. (x 2 , x 3 ) are equal to (0.4, 0.5). x 4 is equal to x 1 +x 2 +x 3 4 = x 1 +0.9

4

. So the restriction considered is f 1 (x 1 ) = f x 1 , 0.4, 0.5, x 1 +0.9

4

. f 1 is assumed to be the realization of Y 1 (x 1 ) = m + Z 1 (x 1 ), with Z 1 a centered Gaussian process of covariance kernel σ 2 1 ρ 1 (x 1 , t 1 ). The DoE X 1 is of the form:

X 1 =    x (1) 1 . . . x (n 1 ) 1    .
• At step 2, (x 2 , x 3 ) are released and x 4 is still fixed. x 4 is equal to x 1 +x 2 +x 3

4

. The restriction considered is

f 2 (x 1 , x 2 , x 3 ) = f x 1 , x 2 , x 3 , x 1 +x 2 +x 3 4 . f 2 is assumed to be the realization of Y 2 (x 1 , x 2 , x 3 ) = Y 1 (x 1 ) + Z 2 (x 1 , x 2 , x 3 ), with Z 2 a centered Gaussian process of covariance kernel σ 2 2 ρ 2 ((x 1 , x 2 , x 3 ), (t 1 , t 2 , t 3
)) such that Z 2 (x 1 , 0.4, 0.5) = 0, for all x 1 in [0, 1]. This property of Z 2 enables the following equality Y 2 (x 1 , 0.4, 0.5) = Y 1 (x 1 ), for all x 1 in [0, 1]. The DoE X 2 is of the form:

X 2 =    x (n 1 +1) 1 x (n 1 +1) 2 x (n 1 +1) 3 . . . x (n 1 +n 2 ) 1 x (n 1 +n 2 ) 2 x (n 1 +n 2 ) 3    .
• At step 3, x 4 is released. All 4 inputs are free so the whole output f is considered.

It is assumed to be the realization of

Y 3 (x 1 , x 2 , x 3 , x 4 ) = Y 2 (x 1 , x 2 , x 3 )+Z 3 (x 1 , x 2 , x 3 , x 4 ), with Z 3 a centered Gaussian process of covariance kernel σ 2 3 ρ 3 ((x 1 , x 2 , x 3 , x 4 ), (t 1 , t 2 , t 3 , t 4 )) such that Z 3 (x 1 , x 2 , x 3 , x 1 +x 2 +x 3 4 ) = 0, to enable Y 3 (x 1 , x 2 , x 3 , x 1 +x 2 +x 3 4 ) = Y 2 (x 1 , x 2 , x 3 ), for all (x 1 , x 2 , x 3 ) in [0, 1] 3 .
The DoE X 3 is of the form:

X 3 =    x (n 1 +n 2 +1) 1 x (n 1 +n 2 +1) 2 x (n 1 +n 2 +1) 3 x (n 1 +n 2 +1) 4 . . . x (n 1 +n 2 +n 3 ) 1 x (n 1 +n 2 +n 3 ) 2 x (n 1 +n 2 +n 3 ) 3 x (n 1 +n 2 +n 3 ) 4   

Metamodel seqGPR (Sequential Gaussian process regression)

Finally, the problem is modeled by the following statistical model (for

x I 1 ∪•••∪In in [0, 1] d 1 +•••+dn ):    Y 1 (x I 1 ) = m + Z 1 (x I 1 ), Y n (x I 1 ∪•••∪I n-1 , x In ) = Y n-1 (x I 1 ∪•••∪I n-1 ) + Z n (x I 1 ∪•••∪I n-1 , x In ), ∀n ∈ 2, N , (3.1)
where:

• The processes (Z n ) N n=1 are independent Gaussian processes of law:

Z n ∼ GP(0, σ 2 n ρ n (x I 1 ∪•••∪In , t I 1 ∪•••∪In )), ∀n ∈ 1, N . (3.2)
• The processes (Z n ) N n=2 verify the following property:

Z n (x I 1 ∪•••I n-1 , xIn ) = 0, ∀n ∈ 2, N , ∀x I 1 ∪•••I n-1 ∈ [0, 1] d 1 +•••+d n-1 . (3.3)
The same formulae of prediction as for a classic kriging metamodel apply (see subsection 2.1.1 in chapter 2). For every x ∈ [0, 1] d , the prediction mean ŷ(x) and variance v(x) are defined by:

ŷ(x) = E [Y N (x) | Y 1 (X 1 ) = y 1 , • • • , Y N (X N ) = y N ] , v(x) = V ar (Y N (x) | Y 1 (X 1 ) = y 1 , • • • , Y N (X N ) = y N ) .
Remark The independence hypothesis on the (Z n ) 1 n N is added as in the multifidelity model [Kennedy and O'Hagan, 2000], from which this model is inspired. The dependence is privileged between the (Y n ) 1 n N which really model the output. Furthermore, this hypothesis simplifies calculations for the likelihood and so for the parameter estimation, as shown in section 4.1 in chapter 4.

Candidates for the correction processes

In this section, candidates for the correction processes, that must satisfy the nullity property (3.3), are suggested. More precisely, the goal is to build a process Z : [0, 1] d J +d I × Ω → R null on an infinite continuous set of points :

Z(x J , g(x J )) = 0 ∀x J ∈ [0, 1] d J , (3.4) with g : [0, 1] d J → [0, 1] d I a deterministic function.
Three candidates are suggested for the process Z, all based on a latent Gaussian process ∼ Z ∼ GP (0, σ 2 r((x J , x I ), (t J , t I ))) of covariance kernel σ 2 r. The function g is imposed by industrial concerns, however one can discuss about its regularity. If the goal is to have a Z kernel of the same regularity than the ∼ Z kernel, then a reasonable rule may be that g must at least be as regular as the kernel of ∼ Z.

The first candidate results from a reduction of the latent process and is called the Red process. The second candidate, called the Psi process, involves a multiplicating term denoted by Ψ, null on the zone of nullity. The third candidate is a tractable version of the process defined by [Gauthier, 2011], which has been recalled in subsection 2.2.2 in chapter 2.

Red (Reduced) process

The transformation leading to the Red process consists in subtracting from ∼ Z its value on the subspace {(x J , g(x J )), x J ∈ [0, 1] d J }. Thus, the Red process is equal to:

Z Red (x J , x I ) = ∼ Z(x J , x I ) - ∼ Z(x J , g(x J )).
It is a centered Gaussian process of covariance kernel σ 2 ρ Red with:

ρ Red ((x J , x I ), (t J , t I )) = r ((x J , x I ), (t J , t I )) -r ((x J , x I ), (t J , g(t J )))
-r ((x J , g(x J )), (t J , t I )) + r ((x J , g(x J )), (t J , g(t J )) .

As the process used to transform the latent process is constant in x I , the transformation is global and not just near the nullity zone. The resulting process is entirely disturbed.

Psi process

Another transformation to impose the nullity property while modifying the latent process only near the nullity subspace consists in multiplying the latent process by a function null on the nullity subspace {(x J , g(x J )), x J ∈ [0, 1] d J } and equal to one far from it. The resulting process is called the Psi process:

Z P si (x J , x I ) = Ψ (x I -g(x J )) ∼ Z(x J , x I ).
with Ψ equal to:

Ψ(t) = 1 -exp - d I i=1 t 2 i 2δ 2 i
The parameter δ i (i ∈ 1, d I ) controls the speed at which Ψ goes to 0 in the direction t i .

The smaller δ i is, the steeper is the slope toward 0.

The Psi process is a centered Gaussian process of covariance kernel σ 2 ρ P si with:

ρ P si ((x J , x I ), (t J , t I )) = Ψ (x I -g(x J )) Ψ (t I -g(t J )) r ((x J , x I ), (t J , t I ))
As the Psi function used to transform the latent process is almost constant equal to 1 except near the nullity zone where it quickly goes down to 0, the transformation is local, located around the nullity zone. The resulting process is disturbed near the nullity zone, but the disturbation is important and not smooth.

P (Preconditioned) process

An alternative local and smooth transformation of the latent process to impose the nullity property consists in conditioning the latent process to be null on the nullity subspace D = {(x J , g(x J )), x J ∈ [0, 1] d J }. As this nullity subspace is an infinite continuous set of points, the usual conditioning seen in proposition 2 in chapter 2 cannot be used directly. Instead, a generalization of the conditioning defined in [START_REF] Gauthier | Spectral approach for kernelbased interpolation[END_REF] and detailed in subsection 2.2.2 in chapter 2 is used. The resulting process is called the P process:

Z P (x J , x I ) = ∼ Z(x J , x I ) -E ∼ Z(x J , x I ) | ∼ Z(s J , g(s J )), ∀s J ∈ [0, 1] d J . (3.5)
The term

E ∼ Z(x J , x I ) | ∼ Z(s J , g(s J )), ∀s J ∈ [0, 1] d J is the orthogonal projection of ∼ Z(x J , x I )
in the sub Gaussian Space engendered by the family

∼ Z(D) = ∼ Z(s J , g(s J )) s J ∈[0,1] d J
. Its general expression is given in subsection 2.2.2 in chapter 2 and is recalled below:

E ∼ Z(x) | ∼ Z(D) = +∞ n=1 φ n (x) S ∼ φ n (s) ∼ Z(x s )dν(s), (3.6)
with D = {x s , s ∈ S} the nullity subspace, S its index set, ν a measure on S, and

φ n (x) = 1 λn S σ 2 r(x, x s ) ∼ φ n (s)dν(s). (λ n , ∼ φ n ) n 1 are solutions of the eigen problem : S σ 2 r (x s , x u ) ∼ φ n (u)dν(u) = λ n ∼ φ n (s), ∀s ∈ S, such that S ∼ φ n (s) ∼ φ m (s)dν(s) = δ nm ∀n, m 1,
where δ nm is the Kronecker symbol. It is applied to the case:

   S = [0, 1] d J , ν any measure on [0, 1] d J , x s = (s J , g(s J )).
This kernel cannot be used just as it is in practice, because the solutions of the eigenvalue problem are not explicit in general and the sums are infinite. Two ways of computing this kernel are proposed. The first is based on the discretization of the spectral decomposition.

The second is based on a wise choice of r (the correlation kernel of the latent process ∼ Z) for which an explicit formula is known.

In this paragraph, a first way of computing the kernel is given by an approximation based on the discretization of the spectral decomposition that reduces the functional eigenvalue problem to a finite dimensional one. In that case, the terms from the spectral decomposition vanish and the formula of the resulting approximate kernel only depends on the kernel of the latent process.

Proposition 9 Discretizing the spectral decomposition of the P process Z by a quadrature method (as in the 2D example in subsection 3.2.4) using weights (ω i ) L i=1 , quadrature points (s (i) ) L i=1 , and the associated discretized nullity subspace D = (s (i) , g(s (i) )), i ∈ 1, L , is equivalent to approximating the P process by the process ∼ Z conditioned to be null on the points of D :

Z D = ∼ Z | ∼ Z(D) = 0 .
It is a centered Gaussian process of covariance kernel σ 2 ρ D :

ρ D (x, t) = r(x, t) -r(x, D)r(D, D) -1 r(D, t) ∀x, t ∈ [0, 1] d J +d I .
Proof It is a straightforward application of proposition 8 with D = D, S = S, and ν = L i=1 ω i δ s (i) . An alternative proof is suggested in appendix 9.3.1 in the case of a Monte-Carlo method.

This process does not suit the original expectations as the nullity is not fully respected on the continous set of points. The size of D must be high to approach correctly the full nullity. It is more and more difficult as the dimension of the input space increases, and it leads to very expensive computations (with inversion of huge matrices). A second way of computing this kernel is proposed. It consists in finding forms of the kernel of ∼ Z that enable the kernel of Z P to be tractable.

Proposition 10 Let r J and r I be two stationary correlation kernels. If the covariance kernel of ∼ Z is of the form σ 2 r with:

r((x J , x I ), (t J , t I )) = r J (x J , t J )r I (x I -g(x J ), t I -g(t J )),
then the P process is a centered Gaussian process equal to :

Z P (x J , x I ) = ∼ Z(x J , x I ) -r I (x I -g(x J ), 0) ∼ Z(x J , g(x J )),
whose covariance kernel is σ 2 ρ P with:

ρ P ((x J , x I ), (t J , t I )) = r J (x J , t J ) [r I (x I -g(x J ), t I -g(t J )) -r I (x I -g(x J ), 0)r I (0, t I -g(t J ))] .
Proof See appendix 9.3.2 for the proof of this proposition.

Corollary 1 If g is constant equal to c ∈ [0, 1] d I and if r is a stationary kernel of the form r((x J , x I ), (t J , t I )) = r J (x J , t J )r I (x I , t I ), then the P process is a centered Gaussian process equal to :

Z P (x J , x I ) = ∼ Z(x J , x I ) -r I (x I , c) ∼ Z(x J , g(x J )),
whose covariance kernel is σ 2 ρ P with:

ρ P ((x J , x I ), (t J , t I )) = r J (x J , t J ) [r I (x I , t I ) -r I (x I , c)r I (c, t I )] .
Proof The proof is straightforward, by applying proposition 10 with g(x J ) = c and using the stationarity of r I : r I (x Ic, t Ic) = r I (x I , t I ). The model presented in the corollary (where the variables that are released were previously fixed at a constant value) is illustrated in figure 3.1.

The expectation used to transform the latent process is smooth and equal to zero near the nullity zone. The transformation is local, located around the nullity zone. The resulting process is disturbed near the nullity zone, but the disturbation is smooth. In the following, the P processes are assumed to be built as in proposition 10 or corrollary 1.

Example in 2D

This subsection describes the three candidates in the context of the 2D example defined in subsection 3.1.2. They are candidates for the correction process Z 2 which verifies the nullity property Z 2 (x 1 , g(x 1 )) = 0, where g(x 1 ) = 0.7x 1 + 0.2.

Red process If Z 2 is a Red process, it is defined as:

Z 2 (x 1 , x 2 ) = ∼ Z 2 (x 1 , x 2 ) - ∼ Z 2 (x 1 , 0.7x 1 + 0.2).
This transformation of ∼ Z 2 is linear and therefore, the Gaussian law is preserved. The nullity property is easily verified and the kernel can directly be expressed as a linear transformation of the latent kernel so stays computable. The covariance kernel of Z 2 is equal to σ 2 2 ρ Red 2 with:

ρ Red 2 ((x 1 , x 2 ), (t 1 , t 2 )) = r 2 ((x 1 , x 2 ), (t 1 , t 2 )) + r 2 ((x 1 , 0.7x 1 + 0.2), (t 1 , 0.7t 1 + 0.2)) -r 2 ((x 1 , x 2 ), (t 1 , 0.7t 1 + 0.2)) -r 2 ((x 1 , 0.7x 1 + 0.2), (t 1 , t 2 )).
The different processes involved are shown in figure below: the Red process in left panel, the latent process on middle panel, and its value on the line x 2 = 0.7x 1 + 0.2 (which is a process function of x 1 that can be seen as a process function of (x 1 , x 2 ) constant in x 2 ) on right panel.

Z Red 2 (x 1 , x 2 ) ∼ Z 2 (x 1 , x 2 ) ∼ Z 2 (x 1 , 0.7x 1 + 0.2)
Psi process The Psi process is defined as:

Z 2 (x 1 , x 2 ) = Ψ(x 2 -0.7x 1 -0.2) ∼ Z 2 (x 1 , x 2 ), with Ψ(t) = 1 -exp t 2 2δ 2
for all t in [-1, 1]. The function Ψ is null at 0 and near 1 far from 0, and the parameter δ controls the speed at which Ψ goes to 0. The smaller δ is, the steeper is the slope toward 0, as illustrated on the figure below.

δ = 0.1 δ = 0.3
The covariance kernel of Z 2 is equal to σ 2 2 ρ P si 2 with:

ρ 2 ((x 1 , x 2 ), (t 1 , t 2 )) = Ψ(x 2 -0.7x 1 -0.2)Ψ(t 2 -0.7t 1 -0.2)r 2 ((x 1 , x 2 ), (t 1 , t 2 )).
The different processes involved are shown in figure below: the Psi process in left panel, the latent process on middle panel, and the function Ψ(x 2 -0.7x 1 -0.2) on right panel.

Z P si 2 (x 1 , x 2 ) ∼ Z 2 (x 1 , x 2 ) Ψ(x 2 -0.7x 1 -0.2)
P process The P process is defined as:

Z 2 (x 1 , x 2 ) = ∼ Z 2 (x 1 , x 2 ) -E ∼ Z 2 (x 1 , x 2 ) | ∼ Z 2 (t 1 , 0.7t 1 + 0.2)∀t 1 ∈ [0, 1] ,
where

E ∼ Z 2 (x 1 , x 2 ) | ∼ Z 2 (t 1 , 0.7t 1 + 0.2)∀t 1 ∈ [0, 1] (denoted by E(x 1 , x 2 )
to simplify notations) is itself a generalization of the conditional expectation, defined as the projection of

∼ Z 2 (x 1 , x 2 ) in the subGaussian space generated by ∼ Z 2 (t 1 , 0.7t 1 + 0.2) t 1 ∈[0,1]
. The formula of the expectation, taken from the proposition 7 and recalled in the next paragraph, is here adapted to the case D = {(s, 0.7s + 0.2), s ∈ [0, 1]}, S = [0, 1], and ν any measure on S:

E(x 1 , x 2 ) = +∞ n=1 φ n (x 1 , x 2 ) 1 0 ∼ φ n (s) ∼ Z 2 (s, 0.7s + 0.2)dν(s),
where:

φ n (x 1 , x 2 ) = 1 λ n 1 0 σ 2 2 r 2 ((
) ∼ φ n (u)dν(u) = λ n ∼ φ n (s). ( ∼ φ n ) n 1 is an orthonormal basis of L 2 ([0, 1], ν): 1 0 ∼ φ n (s) ∼ φ m (s)dν(s) = δ nm .
The first way to compute its kernel is based on the discretization of its spectral decomposition. Discretizing the spectral decomposition of the P process means that the integrals are approximated by quadrature methods, i.e.

1 0 g(s)dν(s) ≈ L i=1 ω i g(s (i)
) with (ω i ) L i=1 the weigths and (s (i) ) L i=1 the quadrature points choosen in [0, 1]. For example, in the case of the Monte-Carlo method, ω i = 1

L and S = s (i) L i=1 is choosen uniformly in [0, 1]. Using a quadrature approximation of the integral, the eigenvalue problem becomes:

L i=1 ω i σ 2
2 r 2 ((s, 0.7s + 0.2), (s (i) , 0.7s

(i) + 0.2)) ∼ φ(s (i) ) ≈ λ ∼ φ(s), ∀s ∈ [0, 1].
Then, by considering only the values of the eigenfunction ∼ φ at the quadrature points (s (i) ) L i=1 , the eigenvalue problem is replaced by a finite dimensional one:

σ 2 2 r 2 (D, D)W Φ = γΦ, where D = (s (i) , 0.7s (i) + 0.2) L i=1 is the discretization of the nullity subspace D corresponding to S (discretization of S), W =      ω 1 0 . . . 0 0 . . . . . . . . . . . . . . . . . . 0 0 . . . 0 ω L     
is the diagonal matrix whose diagonal components are the weights of the quadrature method. According to proposition 7, there exist (γ n , Φ n ) L n=1 solutions of this finite dimensional eigenvalue problem such that (Φ n ) L n=1 is an orthonormal basis of R L for the scalar product associated with W :

Φ n W Φ m = δ nm .
The functions (φ n ) L n=1 can be approximated in the same way by:

φ D n (x 1 , x 2 ) = 1 γn σ 2 2 r 2 ((x 1 , x 2 ), D)W Φ n .
The expectation E(x 1 , x 2 ) is approximated by:

E D (x 1 , x 2 ) = L n=1 φ n (D)(x 1 , x 2 ) Φ n W ∼ Z(D) .
The P process Z 2 is approximated by:

Z D 2 (x 1 , x 2 ) = ∼ Z 2 (x 1 , x 2 ) -E D (x 1 , x 2 ).
According to proposition 9, this approximation of the P process is equal to the process ∼ Z 2 conditioned to be null on the subspace D:

Z D 2 (x 1 , x 2 ) = ∼ Z 2 (x 1 , x 2 ) | ∼ Z 2 (D) = 0 .
The second way is to find a covariance kernel of

∼ Z 2 such that the kernel of Z 2 is computable. If the covariance kernel of ∼ Z 2 is of the form: σ 2 2 r 2 ((x 1 , x 2 ), (t 1 , t 2 )) = σ 2 r 2,1 (x 1 , t 1 )r 2,2 (x 2 -0.7x 1 , t 2 -0.7t 1 ),
with r 2,1 and r 2,2 stationary kernels, and if Z 2 is a P process, then according to proposition 10, it is equal to:

Z 2 (x 1 , x 2 ) = ∼ Z 2 (x 1 , x 2 ) -r 2,2 (x 2 -0.7x 1 -0.2, 0) ∼ Z 2 (x 1 , 0.7x 1 -0.2).
It is a centered Gaussian process of covariance kernel σ 2 2 ρ 2 with:

ρ 2 ((x 1 , x 2 ), (t 1 , t 2 )) = r 2,1 (x 1 , t 1 ) [r 2,2 (x 2 -0.7x 1 , t 2 -0.7t 1 ) -r 2,2 (x 2 -0.7x 1 -0.2, 0)r 2,2 (t2 -0.7t 1 -0.2, 0)] .
The different processes involved are shown in figure below: the P process in left panel, the latent process on middle panel, and the expectation

E(x 1 , x 2 ) = r 2,2 (x 2 -0.7x 1 - 0.2, 0) ∼ Z 2 (x 1 , 0.7x 1 + 0.
2) on right panel.

Z P 2 (x 1 , x 2 ) ∼ Z 2 (x 1 , x 2 ) E(x 1 , x 2 )
The example in 4D can be found in appendix 9.1.1.

Qualitative comparison of the processes

In this section, the name Classic process denotes a stationary Gaussian process. The processes Classic, Psi, Red and P (the three later are built using the Classic process as latent process) are compared in terms of path shape and influence of parameters. The processes are defined in the context of the example in 2D from subsection 3.1.2. The nullity property (see equation (3.4)) has to be verified at the line x 2 = 0.7x 1 + 0.2. The figures follow the same color code : the paths of the Classic process are in grey, those of the Psi process are in green, those of the Red process are in red, and those of the P process are in blue.

All the processes are centered. The different parameters at stake are σ 2 the variance parameter, θ = (θ 1 , θ 2 ) the covariance parameters, and δ the additional parameter of the Psi process. The covariance kernel of the Classic process is the Matern 5 2 kernel: ). Each path x → Z(x, ω) was taken at the same value of ω and with the same parameters :

k Classic ((x 1 , x 2 ), (t 1 , t 2 )) = σ 2 2 i=1 (1 + √ 5 |x i -t i | θ i + 5(x i -t i ) 2 3θ 2 i ) exp - √ 5 |x i -t i | θ i .

Shapes of the paths

σ 2 = 1, θ = ( √ 2 4 , √ 2
4 ), δ = 0.1. The orange line is the zone of nullity of the Psi, Red and P processes. It corresponds to the input zone x 2 = 0.7x 1 + 0.2. As the same ω was taken for each process, the shapes have similarities. The Red process seems to accentuate the elevations and depressions of the Classic one while the Psi and P processes keep the same order of smoothness.

A possible interpretation of that phenomenon is that the Classic process (see panel 3.2a), used as latent process for Psi, Red and P, is disturbed locally in the case of the Psi and P processes, whereas it is disturbed globally in the case of the Red process. An illustration of the processes at stake in the construction of Psi, Red and P processes is shown in figure 3.3. ∼ Z(x 1 , 0.7x 1 + 0.2) (see panel 3.3b), which is subtracted from the latent process to build the Red process, is constant in x 2 , and consequently generates a disruption of the latent process ∼ Z(x 1 , x 2 ) in the whole input space [0, 1] 2 . On the contrary, the conditional expectation

E ∼ Z(x 1 , x 2 ) | ∼ Z(t 1 , 0.7t 1 + 0.2), ∀t 1 ∈ [0, 1] (see panel 3.3c)
, which is subtracted from the latent process to build the P process, depends on x 2 , especially near the line x 2 = 0.7x 1 + 0.2, and tends to 0 far from the line. The modification it implies on the latent process is therefore located almost only along the line. Similarly, Ψ(x 2 -0.7x 1 -0.2) (see panel 3.3a), which is multiplied by the latent process to build the Psi process, is equal to 0 on the line and to 1 far from the line. Although the disturbations are both local, they are of different nature: the one of the P process is smoother than the one of Psi. The P process combines the locality of the disturbation of the Psi process and the smoothness of disturbation of the Red process. 3.4 shows paths of the Classic, Psi, Red, and P processes with the same parameters than in figure 3.2 except that σ 2 = 3. Consequently to the increase of σ 2 , all the paths have a bigger range. The influence of σ 2 is the same for all processes. It defines the degree of dispersion of the path. The bigger σ 2 is, the more the path is dispersed. The influence of θ 1 is the same for all processes. It defines the degree of chaos of the path in the x 1 direction. The smaller θ 1 is, the more chaotic the path is in the x 1 direction.

Influence of σ 2

Influence of θ 2

Figure 3.6 shows paths of the Classic, Psi, Red, and P processes with the same parameters than in figure 3.2 except that θ 2 = 0.1. This time, the influence of θ 2 is not the same for all the processes. For the Classic and Psi processes, it defines the degree of chaos of the path in the direction x 2 . For the P process, it defines the degree of chaos of the path in the direction orthogonal to the line x 2 = 0.7x 1 + 0.2. It is due to the fact that the variable associated to θ 2 in the covariance kernel is not

x 2 but x 2 -0.7x 1 -0.2 (see subsection 3.2.
3). For the Red process, θ 2 defines the degree of chaos both in x 2 and x 1 directions. It is due to the fact that both x 2 and 0.7x 1 + 0.2 are associated with θ 2 in the covariance kernel (see subsection 3.2.1).

Influence of δ

Figure 3.7 shows paths of the Psi process with δ = 0.05 and δ = 0.3. δ acts as a smoother of the Psi paths in the direction orthogonal to the line x 2 = 0.7x 1 + 0.2. Increasing δ implies a wider zone centered on the line where the path is flat near 0. In both cases, the Psi process is really disturbing the initial Classic path in the area of action delimited by δ. Indeed, for small δ, it gives a really sharp zone where the path brutally goes to 0. For big δ, it gives a really flat path in the middle, and looking like the initial Classic path only on the sides of the input space.

σ 2 = 1, θ 1 = θ 2 = √ 2
4 , δ = 0.1.

Estimation of the parameters

The current section aims at knowing if the candidates Psi, Red, and P can be used in pratice in the method seqGPR. A quantitative comparison is carried out between the processes Classic, Psi, Red, and P, about their performance in parameter estimation which is done by maximum likelihood. The goal is to find the process parameters from observations of these processes. These observations are generated by simulations. In the real studies, observations of the correction processes are not always available (only if the samples are nested as in subsection 4.1.1 in chapter 4). Following subsection 2.1.2 in chapter 2, the training data is composed of simulations, denoted by z = (z 1 , • • • , z n X ) , of the candidate process Z ∼ GP(0, (σ simu ) 2 ρ ξ simu (x, t)) (where σ simu and ξ simu are the true parameters) on a set of points X =   

x (1) . . .

x (n X )    of size n X .
The notation ξ simu is introduced to caracterize all the covariance parameters except σ simu .The parameters η = (σ 2 , ξ) are estimated from the training data by maximizing the likelihood L(η; z) or equivalently by minimizing the loss function l(ξ; Z) defined by :

l(ξ; Z) = n X log(σ 2 (ξ)) + log |ρ ξ (X, X)| , with σ 2 (ξ) = z ρ ξ (X,X) -1 z n X
the estimation of σ 2 as a function of ξ (see subsection 2.1.2 in chapter 2).

The notations X J , X I , used in the expressions of the likelihood, are explained in this paragraph. X J (resp. X I ) is the submatrix of X taking only the first d J components (resp. the last d I components) of each x (i) . The matrix (X J , g(X J )) is equal to the matrix X where the last d I components of each x (i) have been replaced by g(x

(i) J ). If X =    x (1) J x (1) J . . . x (n X ) J x (n X ) J    ,
then the matrices X J , X I and g(X J ) are defined as

X J =    x (1) J . . . x (n X ) J    , X I =    x (1) I . . . x (n X ) I    and g(X J ) =    g(x (1) J ) . . . g(x (n X ) J )    . Finally, the matrix (X J , g(X J )) is equal to : (X J , g(X J )) =    x (1) J g(x (1) J ) . . . x (n X ) J g(x (n X ) J )    . In the example in 2D from subsection 3.1.2, a DoE X is of the form X =    x (1) 1 x (1) 2 . . . . . . x (n X ) 1 x (n X ) 2    .
The matrices X J , X I , g(X J ), and (X J , g(X J )) are equal to:

X J =    x (1) 1 . . . x (n X ) 1    , X I =    x (1) 2 . . . x (n X) 2    , g(X J ) =    0.7x (1) 1 + 0.2 . . . 0.7x (n X ) 1 + 0.2    , (X J , g(X J )) =    x (1) 1
0.7x

(1)

1 + 0.2 . . . . . . x (n X ) 1 0.7x (n X ) 1 + 0.2    . An example of samples
X and (X J , g(X J )) is given on figure below:

The minimization problem resulting from the maximum likelihood estimation is given for the Psi, Red and P process. The maximum likelihood estimation is then performed for each process on the example in 2D.

Psi Likelihood

In the case of the Psi process, the correlation parameters are ξ = (θ, δ). As a recall, the correlation kernel of the latent process is denoted by r θ . The correlation matrix on the DoE is equal to

ρ θ,δ (X, X) = diag(Ψ δ (X I -g(X J )))r θ (X, X)diag(Ψ δ (X I -g(X J ))),
where diag(Ψ δ (X Ig(X J ))) is the diagonal matrix whose diagonal components are the components of the vector

Ψ δ (X I -g(X J )) = Ψ δ (x (i) I -g(x (i) J )) i∈ 1,n X .
The estimation problem consists in finding (θ, δ) minimizing :

l(θ, δ; z) = n X log σ 2 (θ, δ) + log |r θ (X, X)| + 2 n X i=1 Ψ δ (x (i) I -g(x (i) J )), with σ 2 (θ, δ) = ∼ z δ r θ (X, X) -1 ∼ z δ n X ,
where

∼ z δ = z i Ψ δ (x (i) I -g(x (i) J )) i∈ 1,n X .
Remark If δ is known, the problem is equivalent to minimizing the loss function of the latent process ∼ Z based on its observations ∼ z δ on the sample X.

Red Likelihood

In the case of the Red (Reduced) process, the correlation parameters are ξ = θ. The estimation problem consists in finding θ minimizing

l(θ; z) = n X log σ 2 (θ) + log |ρ θ (X, X)| , with σ 2 (θ) = z ρ θ (X, X) -1 z n X , where ρ θ (X, X) = r θ (X, X) -r θ (X, (X I , g(X J ))) -r θ ((X I , g(X J )) , X) + r θ ((X I , g(X J )) , (X I , g(X J )))).

P Likelihood

In the case of the P (Preconditioned) process, the correlation parameters are equal to ξ = θ. The correlation kernel of the latent process is decomposed as :

r θ ((x J , x I ), (t J , t I )) = r θ J (x J , t J )r θ I (x I -g(x J ), t I -g(t J )), where θ = (θ J , θ I ). The estimation problem consists in finding θ minimizing l(θ; Z) = n X log σ 2 (θ) + log |ρ θ (X, X)| , with σ 2 (θ) = z ρ θ (X, X) -1 z n X , where ρ θ (X, X) = r θ J (X J , X J ) • r θ I (X I -g(X J ), X I -g(X J )) -r θ I (X I -g(X J ), 0) r θ I (0, X I -g(X J )) .
The notation • is used to denote the pointwise product.

Comparison of the estimations on a 2D example

In this subsection, the three previously defined Gaussian processes (Psi, Red and P) are compared in terms of parameter estimation, based on a simulation of themselves (with imposed parameter values) on a training sample. This comparison is done in the context of the example in 2D from subsection 3.1.2. The goal is to recover the values of the parameters used in the simulation, which are mentioned using the subscript simu (θ simu , σ 2 simu , and δ simu ). The Classic process is added to the comparison as a "control" individual. The same definition of the processes as in section 3.3 is taken. In the following example, the processes are defined on [0, 1] 2 , and the Psi, Red and P processes verify the nullity property (3.4) on the line x 2 = 0.7x 1 + 0.2. The processes are defined as follows:

• Classic: the Classic Gaussian process of parameters σ 2 simu , θ simu , with a covariance kernel of type stationary tensor product matern 5 2 whose fomula is recalled in subsection 2.1.1 in chapter 2 and in section 3.3.

• P: the P process built on a latent process, of parameters σ 2 simu and θ simu , derived from the Classic process, whose covariance kernel is defined as:

k(x, t) = σ 2 r θ 1 (x 1 , t 1 )r θ 2 (x 2 -0.7x 1 -0.2, t 2 -0.7t 1 -0.2),
where r θ 1 and r θ 2 are 1D Matern 5 2 kernels. • Psi: the Psi process built on a latent process equal to the Classic process, of parameters σ 2 simu , θ simu , δ simu . • Red: the Red process built on a latent process equal to the Classic process, of parameters σ 2 simu , θ simu .

The following values are taken for the parameters :

σ 2 simu = 1, θ simu = √ 2 4 , √ 2 4
(called thetamin) or 2 √ 2, 2 √ 2 (called thetamax), and

δ simu = √ 2
4 (called deltamin) or √ 2 (called deltamax). Each process is simulated on a DoE in [0, 1] 2 of size 10 or 100 and its parameters are reestimated by maximum likelihood on the simulated data, using the minimization problems defined in subsections 3.4.1, 3.4.2, and 3.4.3. For the Classic process, the training sample is an LHS optimized for the generalized maximin criterion with p = 50 (see subsection 2.1.5 in chapter 2) using the R package DiceDesign. For the other processes, the design is built as described in section 5.1 in chapter 5, such that it is well spread in [0, 1] 2 , it is sufficiently far from the line x 2 = 0.7x 1 + 0.2, and its projection on the x 1 direction is equal to ( i n X -1 ) i∈ 0,n X -1 (with n X = 10 or 100). The boxplots of the parameter estimators are computed other 100 simulations for each training size. The boxplots corresponding to an estimation over the Classic, Psi, Red and P processes are respectively in grey, green, red and blue. A distinction is made for the Psi process between an estimation when δ = Classic process is more robust than the others. This can be due to the fact that this process is stationary, but also that the DoE's are better spread in the input space (because subject to less constraints in their building). Red and P are comparable and Psi (Psi_deltamin or Psi_deltamax) is a bit worse than the others. This result may be explained by the fact that the Psi process has one more parameter to estimate and that can disturb the estimation. σ 2 estimation Figure 3.9 shows boxplots of estimated values of σ 2 for the two sizes of the design. All estimations seem to converge except in the case θ simu = (

√ 2 4 , √ 2
4 ), δ simu = √ 2. Furthermore, the estimation of σ 2 is really bad for a training size of 10 in the case of the Psi process, compared to the other processes.

4 ), δ simu = √ 2, the estimation of δ seems to converge towards the true value δ simu . For this particular case when it does not converge, the estimations are very scattered from 0 to 10. The problem is probably that δ and θ play similar roles of caracteristic lengths and that may make their separate estimation impossible.

Conclusion

This chapter defines the underlying Gaussian processes and their relations in the seqGPR metamodel: a Gaussian process Y 1 modeling the training data at step 1, and correction processes modeling the additional pieces of information brought about by the successive steps. The different processes are assumed independent to simplify calculations. This hypothesis is necessary and not directly implied by their definition. The correction processes must be Gaussian processes null on an infinite continuous set of points.

Three candidates are defined for the correction processes: Red, Psi and P. They result from transformations of a latent process whose kernel is numerically computable. The Red process is the most simple candidate. It is equal to the latent process reduced by its value on the subspace of nullity. This transformation disturbs the value of the latent process in the entire input space. Therefore, a second candidate is suggested, called Psi, which consists in multiplying the latent process by a function null on the subspace of nullity. This transformation of the latent process is local (near the nullity subspace), but not as smooth as the previous one, as the Psi process is equal to the latent process far from the nullity zone, but is forced to go to zero on this zone. Finally, a last candidate is proposed, called P, which is nearer to the Red process. The difference is that the latent process is not reduced directly by its value on the nullity subspace, but by its expectation. The expectation being dependent on all the inputs, the transformation is smoother than for Psi but stays local, and far from the nullity subspace, the P process is equal to the latent process. The problem of this process is that its kernel is not always numerically computable. Two methods are tried to obtain that computability. The first one, consisting in discretizing its spectral decomposition is not retained, as it does not preserve the nullity in the whole nullity subspace. The second method is prefered, which directly gives an analytical formula for a particular choice of the latent process kernel.

The parameter influences on the path shapes of the candidates and the Classic process (which is used as latent process to build them) are compared in the 2D example. The shapes are

(a) Estimation of θ 1 with θ simu = ( √ 2 4 , √ 2
4 ) (b) Estimation of θ 2 with θ simu = ( √ 2 4 , √ 2
4 ) (c) Estimation of θ 1 with θ simu = (2 √ 2, 2 √ 2)(d) Estimation of θ 2 with θ simu = (2 √ 2, 2 √ 2)
θ simu = ( √ 2 4 , √ 2
4 ) (in panels 3.8a and 3.8b), or

θ simu = (2 √ 2, 2 √
2) (in panels 3.8c and 3.8d), along with different sizes of training sample (10 or 100). The variance parameter σ 2 simu is set to 1. In the case of the Psi process, the parameter δ simu is set to

√ 2 4 (legend Psi_deltamin) or √ 2 (legend Psi_deltamax).
In each boxplot, the black horizontal line is the value of θ simu . The estimated value of θ results from the joint estimation of (θ, σ 2 , δ) by maximization of the likelihood.

(a) θ simu = ( √ 2 4 , √ 2
4 ) (b) θ simu = (2 √ 2, 2 √ 2)
Figure 3.9: Estimation of the variance parameter σ 2 . The true value σ 2 simu , represented by the black horizontal line, is set to 1. θ simu is set to

( √ 2 4 , √ 2
4 ) (panel 3.9a) or (2 √ 2, 2 √ 2) (panel 3.9b). δ simu is set to √ 2 4 or √ 2.
For each training size (10 or 100), from left to right are represented the boxplots of the following processes : Classic, Psi (Psi_deltamin when

δ simu = √ 2
4 and Psi_deltamax when δ simu = √ 2), Red and P. The y-axis is plotted on a logarithmic scale. similar, the one of the Red process is more different as the transformation concerns the entire input space. The influence of σ 2 and θ 1 are the same for all processes: σ 2 determines the dispersion of the path, and θ 1 monitors the smoothness in the x 1 direction. θ 2 has a different meaning depending on the process. For Classic and Psi, it influences the smoothness of the path in the x 2 direction, for P it influences the smoothness in the direction orthogonal to the nullity subspace, for Red it influences the smoothness in x 1 and x 2 directions. δ determines the speed at which the Psi process goes from 0 to the value of the latent process.

It seems that the maximum likelihood estimation of the parameters of the Psi process is not robust (especially for the δ parameter), even in a simple case like the one of the 2D example (with only two inputs and a big training sample of size 100). Furthermore, a big dimension implies a lot more parameters to estimate than the other processes. The estimation of the Red and P parameters is of good quality: it converges with the size of the DoE and it is of the same order of magnitude as the estimation of the Classic process parameters. The latter is a little bit more robust because Red and P processes are more complicated as they are not second-order stationary, but they seem viable. From these observations, only the processes Red and P, whose paths are smoother and whose parameter estimation is robust, are kept to build the metamodel seqGPR.

(a) θ simu = ( √ 2 4 , √ 2
4 ),δ simu = √ 2 4 (b) θ simu = ( √ 2 4 , √ 2
4 ),δ simu = √ 2 (c) θ simu = (2 √ 2, 2 √ 2),δ simu = √ 2 4 (d) θ simu = (2 √ 2, 2 √ 2),δ simu = √ 2

Chapter 4 seqGPR methodology

Chapter 3 proposes the methodology seqGPR taking into account results from numerical simulations y 1 , ..., y N , obtained from DoE's X 1 , ..., X N in spaces of increasing dimension

[0, 1] d 1 , ..., [0, 1] d 1 +•••+d N (see subsection 3.1.1 in chapter 3
). More precisely, y 1 , ..., y N are assumed to be the realizations of the processes Y 1 , ..., Y N at the designs X 1 , ..., X N , where (for

x I 1 ∪•••∪In in [0, 1] d 1 +•••+dn ):    Y 1 (x I 1 ) = m + Z 1 (x I 1 ), Y n (x I 1 ∪•••∪I n-1 , x In ) = Y n-1 (x I 1 ∪•••∪I n-1 ) + Z n (x I 1 ∪•••∪I n-1 , x In ), ∀n ∈ 2, N , (4.1)
where:

• The processes (Z n ) N n=1 are independent Gaussian processes of law:

Z n ∼ GP(0, σ 2 n ρ n (x I 1 ∪•••∪In , t I 1 ∪•••∪In )), ∀n ∈ 1, N . (4.2)
• The processes (Z n ) N n=2 verify the following property:

Z n (x I 1 ∪•••∪I n-1 , xIn ) = 0, ∀n ∈ 2, N , ∀x I 1 ∪•••∪I n-1 ∈ [0, 1] d 1 +•••+d n-1 . (4.
3)

The issue of defining the correction processes (Z n ) N n=2 as Gaussian processes verifying the nullity property (4.3) is solved in chapter 3 where two candidates are retained: the Red and P processes. This new chapter deals with the estimation of the parameters of the processes involved in the model (4.1), and the prediction of the model.

The law of Y 1 (see equations (4.1) and(4.2)) depends on the parameters: m (scalar mean parameter), σ 2 1 (scalar variance parameter), and θ 1 (vector of covariance parameters). For all n ∈ 2, N , Z n (see equation ( 4.2)) is either a Red or P process built on the latent process

∼ Z n ∼ GP(0, σ 2 n r n ) (see chapter 3
) of parameters: σ 2 n (scalar variance parameter), θ n (vector of covariance parameters). In order to emphasize the dependence of ρ n (the covariance kernel of Z n , n ∈ 1, N ) on the parameter θ n , this kernel is denoted by ρ θn . The purpose of this chapter is to estimate the parameters η

= (m, σ 2 1 , θ 1 η 1 , σ 2 2 , θ 2 η 2 , • • • , σ 2 n , θ n ηn , • • • , σ 2 N , θ N η N ) based
on the observed data. The maximum likelihood estimator is used. Using the notations

y = (y 1 , • • • , y N ) and Y = (Y 1 (X 1 ) , • • • , Y N (X N ) )
, the following loss function1 has to be minimized:

l(η; y) = log |Cov η (Y , Y )| + (y -E η [Y ]) Cov η (Y , Y ) -1 (y -E η [Y ]).
This optimization problem is complex as η can reach big dimensions.

To optimize this loss function, similarly to what occurs in a mutifidelity context, two cases may be distinguished. When the designs are nested [Le Gratiet and Garnier, 2014], observa-

tions of Y 1 , Y 2 ..., Y N imply observations of Y 1 , Z 2 , ..., Z N .
The likelihood can be decoupled into independent parts to be optimized in small dimension. For each Z n , the estimation of its parameters is reduced to the case treated in section 3.4 in chapter 3. The case with nested designs is tackled in subsection 4.1.1. When the designs are non-nested [Zertuche, 2015], the loss function cannot be decoupled and an EM (Expectation-Maximization, see subsection 2.1.2 in chapter 2) algorithm is used instead. In subsection 4.1.2, the EM algorithm is adapted to the seqGPR metamodel, in order to optimize the joint loss function while taking advantage of the underlying additive model involving independent processes. In section 4.2, the seqGPR metamodel is tested on two analytic and one industrial test cases.

Estimation and prediction

This section shows estimation and prediction methods for the seqGPR metamodel in case of nested or non-nested designs.

Nested designs Definition 7 (Nested designs)

In what follows, the notation X n X n-1 means that the submatrix of X n composed of the columns corresponding to

x I 1 ∪•••∪I n-1 is included in X n-1 .
In this case, the designs X n and X n-1 are said nested.

Proposition 11 If all the designs are nested: X N • • • X 1 (see definition 7), then the loss function can be decoupled:

l(η; y) = l 1 (η 1 ; y 1 ) + N n=2 l n (η n ; z n ), with l 1 (η 1 ; y 1 ) (respectively l n (η n ; z n ), n ∈ 2, N ) the loss function associated with the train- ing data Y 1 (X 1 ) = y 1 (respectively Z n (X n ) = z n ). Proof (z n ) 2 n N is observed because the designs are nested. The loss function can be decou- pled because (Y 1 , Z 2 , • • • , Z N ) are independent.
If all the designs are nested, the parameters η n can be estimated separately by optimizing the loss functions l n . The formulae of the loss functions (l n ) N n=2 are given in subsections 3.4.2 or 3.4.3 depending whether Z n is a Red or P process. The formula of l 1 is given in subsection 2.1.2 in chapter 2.

The same conclusion can be done for the prediction mean and variance:

Proposition 12 If all the designs are nested: X N • • • X 1 (see definition 7), the prediction formulae can be decoupled:

           ŷ(x) = E [Y 1 (x) | Y 1 (X 1 ) = y 1 ] + N n=2 E [Z n (x) | Z n (X n ) = z n ] , v(x) = V ar [Y 1 (x) | Y 1 (X 1 ) = y 1 ] + N n=2 V ar [Z n (x) | Z n (X n ) = z n ] .
Proof Same arguments as for proposition 11.

In fact, the metamodel is the sum of N independent metamodels.

Remark (properties of nested designs)

To optimize the likelihoods, the matrices cov

(Z n (X n ), Z n (X n )) = σ 2 n ρ θn (X n , X n ) (n in 1, N ) must be inverted.
First, for all n in 1, N , the design X n is assumed not having any redundant points such that the associated covariance matrix has no identical rows. Furthermore, the points must be sufficiently far from each other to ensure the good conditioning of the matrix. This condition is sufficient for the first matrix as long as ρ θ 1 is a positive definite kernel. As it is not the case for the (ρ θn ) N n=2 , to ensure that the matrices are not singular, each X n (n in 2, N ) is also supposed to not contain any point belonging to the nullity zone {(x

I 1 ∪•••∪I n-1 , xIn ), x I 1 ∪•••∪I n-1 ∈ [0, 1] d 1 +•••+d n-1
} so there are no rows full of zeros in the matrices. Section 5.1 in chapter 5 describes the building method retained for the samples in this context.

Drawbacks This nesting constraint on the designs implies that the size of X n cannot increase with n. This is problematic as conversely the dimension of the input space increases with n, so more and more information should be brought as n increases. Furthermore, this limits the exploration of the first dimensions as the same values are used in all the designs. That is why an alternative is proposed in the next subsection with designs that are subject to less radical constraints.

Non-nested designs

In this subsection, the designs are said non-nested if for all n in 1, N , the

x I 1 ∪•••∪In part of X n , • • • , X N have no points in common.
If the designs are non-nested, the loss function is not decoupled and therefore is difficult to optimize. An EM (Expectation-Maximization, see subsection 2.1.2 in chapter 2) algorithm is then used to reduce the dimension of the optimization problem. To define this algorithm, the notion of complete data is introduced.

Definition 8 (Complete data) Let ∼ X n (∀1 n N ) denote the union of all designs that concern Z n . ∼ X n is a matrix of d 1 + • • • + d n columns, composed of the concatenation of the parts of X n , • • • , X N corresponding to the input variables x I 1 ∪•••∪In . The complete data is the random vector: T = (Y 1 ( ∼ X 1 ), Z 2 ( ∼ X 2 ), • • • , Z N ( ∼ X N )).
In the case of non-nested designs (n ∈ 2, N ), denoted by z n , were observed, the loss function associated to all those DoE's (equal to twice the negative loglikelihood of the complete data up to a constant), denoted by l c , could be decomposed as

∼ X 1 , • • • , ∼ X N have
l c (η; z 1 , • • • , z n ) = N n=1 l n c (η n ; z n ), where l 1 c (η 1 ; z 1 ) is the loss function associated to the training data Y 1 ( ∼ X 1 ) = z 1 , and l n c (η n ; z n ) (n ∈ 2, N ) is the loss function associated to the training data Z n ( ∼ X n ) = z n .
As some of the complete data is not observed (missing data), an EM algorithm seems adequate to optimize the loss function.

Definition 9 (EM algorithm)

The EM algorithm is defined by • Expectation Instead of the observed data loss function, the expectation of the complete data loss function conditioned by the observed data is considered. This quantity can be decomposed in N terms:

Q(η, η * ) = E η * [l c (η; T ) | Y = y] , = N n=1 Q n (η n ; η * ), with ∀n ∈ 1, N Q n (η n , η * ) = E η * l n c (η n ; Z n ( ∼ X n )) | Y = y .
• Maximization The EM algorithm consists in building the sequence (η

(i) ) i 0 = (η (i) 1 , • • • , η (i)
N ) i 0 such that η (0) is user defined and ∀i 0, η (i+1) is solution of the optimization problem:

min η Q(η, η (i) ). For all n ∈ 1, N , η (i+1) n
is solution of the following optimization problem:

min ηn Q n (η n , η (i) ). (4.4)
See appendix 9.4.1 for the explicit formulae of the Q n . In the usual EM algorithm (see subsection 2.1.2 in chapter 2), the maximization step consists in a maximization problem because the Q criterion is built on the loglikelihood. However, in this case, the Q criterion is built on twice the negative loglikelihood, that is why the optimization problem becomes a minimization one. Finally, η is estimated by a term of the sequence of sufficiently high rank, the choice of the rank being done by imposing a maximum number of iterations and a threshold on the variation of the loss function between iterations.

Remark (properties of non-nested designs)

To optimize Q n , the matrices cov Z n ( ∼ X n ), Z n ( ∼ X n ) = σ 2 n ρ θn ( ∼ X n , ∼ X n ) (n ∈ 1, N ) need to be inverted. First, for all n ∈ 1, N , ∼
X n is assumed not having any redundant points such that the matrices have no identical rows. This condition is sufficient for the first matrix to be invertible as long as ρ θ 1 is a positive-definite kernel. As it is not the case of the ρ θn (n 2), to ensure that the matrices are not singular, each ∼ X n (n ∈ 2, N ) is also supposed not to contain any point in the nullity subspace {(x

I 1 ∪•••∪I n-1 , xIn ), x I 1 ∪•••∪I n-1 ∈ [0, 1] d 1 +•••+d n-1 }
so there are no rows full of zeros in the matrices. Section 5.2 in chapter 5 describes the building method retained for the samples satisfying these constraints.

Example in 2D

This section illustrates the building methods of the seqGPR metamodel in the case of the example in 2D defined in subsection 3.1.2 in chapter 3.

Nested designs

The two samples X 1 and X 2 are assumed nested (denoted by X 2 X 1 , see definition 7 in subsection 4.1.1), i.e. they are of the form:

X 1 =    x (1) 1 . . . x (n 1 ) 1    , X 2 =    x (i 1 ) 1 x (i 1 ) 2 . . . x (in 2 ) 1 x (in 2 ) 2    , {i 1 , • • • , i n 2 } ⊂ {1, • • • , n 1 }.
The values of x 1 contained in X 2 are also contained in X

(i k ) 1 , x (i k ) 2 )
denote a point from X 2 , then:

Y 2 (x (i k ) 1 , x (i k ) 2 ) =y (k) 2 = Y 1 (x (i k ) 1 ) =y (i k ) 1 + Z 2 (x (i k ) 1 , x (i k ) 2 ) =z (k) 2 .
The value of Y 2 (x

(i k ) 1 , x (i k )
2 ) is deduced by taking the kth term of the vectorial equality

Y 2 (X 2 ) = y 2 . The value of Y 1 (x (i k ) 1 ) is deduced by taking the i k th term from the vecto- rial equality Y 1 (X 1 ) = y 1 , because x (i k ) 1 is the x 1 component of the kth point of X 2 but, as X 2 X 1 (see definition 7 in subsection 4.1.1), it is also the x 1 component of the i k th point of X 1 . The value of Z 2 (x (i k ) 1 , x (i k )
2 ) is then deduced by subtraction:

z (k) 2 = y (k) 2 -y (i k ) 1 . Thus, a new training data is known: Y 1 (X 1 ) = y 1 Z 2 (X 2 ) = z 2 ,
with z 2 the vector whose kth component is equal to

z (k)
2 . The likelihood to maximize becomes (with h V (v) denoting the density of random vector V taken at v):

h Y (y) = h Y 1 (X 1 ),Y 2 (X 2 ) (y 1 , y 2 ) = h Y 1 (X 1 ),Z 2 (X 2 ) (y 1 , z 2 ) = h Y 1 (X 1 ) (y 1 )h Z 2 (X 2 ) (z 2 ).
The last equality is explained by the independence of Y 1 and Z 2 . As a consequence, the loss function l(η; y), equal to -2 log(h Y (y)) up to a constant, is decomposed as the sum of the two loss functions associated with Y 1 (X 1 ) and Z 2 (X 2 ):

l(η; y) = l 1 (η 1 ; y 1 ) + l 2 (η 2 ; z 2 ).
These two loss functions can be optimized separately, which leads to two simple optimization problems instead of one complex optimization problem. Thus, the parameter estimation is made easier. The same decoupling property is observed for the prediction:

ŷ(x 1 , x 2 ) = E [Y 1 (x 1 ) | Y 1 (X 1 ) = y 1 ] + E [Z 2 (x 1 , x 2 ) | Z 2 (X 2 ) = z 2 ] , v(x 1 , x 2 ) = V ar (Y 1 (x 1 ) | Y 1 (X 1 ) = y 1 ) + V ar (Z 2 (x 1 , x 2 ) | Z 2 (X 2 ) = z 2 ) .
In order to compute the likelihoods and the predictions, the matrices cov(Y

1 (X 1 ), Y 1 (X 1 )) = σ 2 1 ρ θ 1 (X 1 , X 1 ) and cov(Z 2 (X 2 ), Z 2 (X 2 )) = σ 2 2 ρ θ 2 (X 2 , X 2
) must be invertible. If a positive definite covariance kernel is taken for Y 1 , the fact that X 1 is composed of distinct points sufficiently far from each other is sufficient to ensure the invertibility of the first matrix, as the matrix has distinct rows in this case. Because of the nullity property of Z 2 : Z 2 (x 1 , 0.7x 1 + 0.2) = 0 (for all x 1 in [0, 1]), its covariance kernel is not positive definite, therefore the same constraint on X 2 is not sufficient. An additional constraint must be added, that the points of X 2 are sufficiently far from the nullity subspace

D = {(x 1 , 0.7x 1 + 0.2), x 1 ∈ [0, 1]}, i.e. for all k in 1, n 2 , x (i k ) 2
must be sufficiently far from 0.7x

(i k ) 1 + 0.2.
This second constraint ensures that the covariance matrix has no rows full of zeros.

Non-nested designs The designs X 1 and X 2 are said non-nested if they have no common values of x 1 , i.e. if they are of the form:

X 1 =    x (1) 1 . . . x (n 1 ) 1    , X 2 =    x (n 1 +1) 1 x (n 1 +1) 2 . . . . . . x (n 1 +n 2 ) 1 x (n 1 +n 2 ) 2    , {x (1)
1 , . . . , x

(n 1 ) 1 } ∩ {x (n 1 +1) 1 , . . . , x (n 1 +n 2 ) 1 } = ∅.
An example of such designs is shown in figure below:

The prediction cannot be simplified but the estimation can. To be reduced to the previous case with nested designs, new designs must be defined, that take the role previously played by X 1 and X 2 . These new designs, denoted ∼ X 1 and ∼ X 2 , are defined as:

∼ X 1 =    x (1) 1 . . . x (n 1 +n 2 ) 1    , ∼ X 2 =    x (n 1 +1) 1 x (n 1 +1) 2 . . . . . . x (n 1 +n 2 ) 1 x (n 1 +n 2 ) 2    = X 2 .
∼ X 1 contains all the x 1 values of the matrices X 1 and X 2 .

∼ X 2 contains all the (x 1 , x 2 ) values of X 2 so is equal to X 2 . The building of

∼ X 1 ), Z 2 ( ∼ X 2 )
), which can be decoupled, is more adapted than the loss function associated with (Y 1 (X 1 ), Y 2 (X 2 )), which cannot. The problem is that the value of Y 1 is not known everywhere in ∼ X 1 , so some data is missing to use the complete data loss function. The EM algorithm is then used, in which the complete data loss function is replaced by its expectation conditioned by the training data:

Q(η, η * ) = E η * l 1 (η 1 ; Y 1 ( ∼ X 1 )) + l 2 (η 2 ; Z 2 ( ∼ X 2 )) | Y = y = E η * l 1 (η 1 ; Y 1 ( ∼ X 1 )) | Y = y =Q 1 (η 1 ,η * ) + E η * l 2 (η 2 ; Z 2 ( ∼ X 2 )) | Y = y =Q 2 (η 2 ,η * ) . The EM algorithm consists in computing a sequence (η (i) ) i 0 in which η (i+1) is built from η (i) by minimizing Q(η, η (i) ). The subvectors η (i+1) 1 and η (i+1) 2 are built separately by minimizing respectively Q 1 (η 1 , η (i) ) and Q 2 (η 2 , η (i) ). To optimize Q 1 and Q 2 , the matrices cov Y 1 ( ∼ X 1 ), Y 1 ( ∼ X 1 ) = σ 2 1 ρ θ 1 ( ∼ X 1 , ∼ X 1 ) and cov Z 2 ( ∼ X 2 ), Z 2 ( ∼ X 2 ) = σ 2 2 ρ θ 2 ( ∼ X 2 , ∼ X
2 ) need to be inverted. As for the nested case, ∼ X 1 must have distinct points sufficiently far from each other, and ∼ X 2 must have distinct points sufficiently far from each other and sufficiently far from the nullity subspace x 2 = 0.7x 1 + 0.2. The last constraint means that, for all k in 1, n 2 , x (n 1 +k) 2 must be sufficiently far from 0.7x Application The EM algorithm is tested in the context of the example in 2D from subsection 3.1.2 in chapter 3, where the process

(n 1 +k) 1 + 0.2.
m = 0 σ 2 1 = 1 θ 1 = 0.5 σ 2 2 = 1 θ 21 = 0.5 θ 22 = 0.
Y 2 (x 1 , x 2 ) = Y 1 (x 1 ) + Z 2 (x 1 , x 2 ), with Y 1 (x 1 ) = m + Z 1 (x 1
) and Z 2 (x 1 , 0.7x 1 + 0.2) = 0, is simulated. The mean of the process is equal to m = 1. Z 1 (resp. Z 2 ) has a variance parameter equal to σ 2 1 = 1 (resp. equal to σ 2 2 = 1) and a covariance parameter equal to θ 1 = 0.5 (resp. equal to θ 2 = (0.5, 0.5)). The simulation of the process Y 2 is realised 100 times, each time on a DoE X 1 of size 8 and a DoE X 2 of size 18 (see figure 4.1). X 1 and X 2 are non-nested, X 2 is built following the method described in section 5.2 in chapter 5.

For both Red and P the EM algorithm seems to have converged towards a local optimum of the likelihood as shown for one particular DoE in figure 4.2.

The comparison of the estimations in the case Z 2 is a Red or P process is shown in table 4.1. The estimations are good for the covariance parameters (θ 1 and θ 2 ) and less precise for the mean and variance parameters. Red and P are globally equivalent in terms of parameter estimation.

See appendix 9.1.2 for the example in 4D. Now that the metamodel building method is complete, it must be evaluated.

Test cases

In what follows, three metamodels are compared on different test cases:

• seqGPR: the metamodel introduced in this thesis. The kernel of Y 1 and the ones on which are based (Z n ) N n=2 are stationary tensor product matern 5 2 covariance kernels.

• K_N (N being the number of the last step): a classic kriging metamodel with a stationary tensor product matern 5 2 covariance kernel and a constant mean which is built on the last training sample (X N , y N ).

• K_tot: a classic kriging metamodel similar to K_N, but built on all training samples

(X 1 , y 1 ), • • • , (X N , y N ).
Four

RM SE = ntest i=1 (y (i) test -ŷ(x (i) test )) 2 n test , with (x (i
(i) test )) ntest i=1
the predictions of the metamodel on the test sample. A simple likelihood estimation is used for the two kriging models K_N and K_tot. An estimation by EM (in the non-nested case) or by independent maximization of the likelihoods of Y 1 and the (Z n ) N n=2 (in the nested case) is carried out for the metamodel seqGPR. Two test cases involve an analytic output and one describes an industrial case.

Robustness

The philosophy of the method seqGPR is to explain a maximum of the data by Y 1 , and to correct it thanks to the (Z n ) N n=2 , so all the parameters for Y 1 are estimated individually and on the contrary some parameters of the (Z n ) N n=2 are grouped, all parameters being equal in the same group. Two types of parametrization for the correction processes are defined:

• The first type is called Full. It is the usual one, where all components are estimated individually. The vector θ n , of covariance parameters of Z n , is equal to

θ n = (θ 1 n , • • • , θ d 1 +•••+dn n
), with one component θ i n for each input variable x i . This type of parametrization has the advantage of giving a great flexibility in the fitting to the training data, however it may lack robustness in the prediction on other data.

• The second type is called Robust. Some components are affected the same value and are estimated together while some others are estimated individually. The vector θ n is equal to

θ n = (α n , • • • , α n I 1 ∪•••∪I n-1 , θ 1 n , • • • , θ dn n In ), with α n common to all components of x I 1 ∪•••∪I n-1 ,
and one parameter θ i n for each component of x In . This type is motivated by a search for compromise between bias and variance. Diminishing the number of parameters to estimate automatically decreases the chances of overfitting.

Analytic test case in dimension 4

The output f considered in this test case is a function of 4 inputs x = (x 1 , x 2 , x 3 , x 4 ):

f (x 1 , x 2 , x 3 , x 4 ) = g 1 (x 1 , x 2 ) + g 2 (x 1 , x 2 , x 3 , x 4 ), with:      g 1 (x 1 , x 2 ) = 4 -2.1(4x 1 -2) 2 + (4x 1 -2) 4 3 (4x 1 -2) 2 + (4x 1 -2)(2x 2 -1) + [-4 + 4(2x 2 -1) 2 ] (2x 2 -1) 2 , g 2 (x 1 , x 2 , x 3 , x 4 ) = 4 exp -x -0.3 2 .
g 1 is the six-hump camel back resized in [0, 1] 2 . It is a heckled 2D surface. g 2 is a smoother Gaussian function of 4 inputs. The study is composed of two steps:

• At step 1, computer code evalutations are run at DoE X 1 in dimension 2, such that

f 1 (X 1 ) = y 1 , with f 1 defined by f 1 (x 1 , x 2 ) = f (x 1 , x 2 , x3 , x4 ).
Only the first two variables x 1 and x 2 are free and the other two variables are fixed:

x3 = x 1 +x 2 2 , x4 = 0.2x 1 + 0.7.
• At step 2, new simulations are launched at points X 2 , a design in dimension 4. The last two variables (x 3 , x 4 ) are now released.

The total number of variables is d = 4, the number of steps is N = 2, the index set of variables released at step 1 is I 1 = {1, 2} and the index set of variables released at step 2 is A comparison is made between nested designs (built following section 5.1 in chapter 5) and non-nested ones (built following section 5.2 in chapter 5). Different sizes of design are also compared to see the evolution of the RMSE with the number of training points and see the influence of the size of X 1 relative to the size of X 2 :

I 2 = {3, 4}.
• X 1 bigger than X 2 : 10pts/5pts (X 1 of size 10 and X 2 of size 5), 20pts/10pts

• X 1 smaller than X 2 : 10pts/20pts, 20pts/40pts

• X 1 of same size as X 2 : 10pts/10pts, 15pts/15pts, 20pts/20pts, 30pts/30pts.

The standard deviation of the output on the test set is represented by the black horizontal line. If the RMSE reaches this value, the corresponding metamodel is considered as not predictive.

As regards the influence of the training samples, all metamodel performances improve with the size of the training sample. Results show that including the previous information of (X 1 , y 1 ) is useful, as it greatly improves the performances of the metamodels. Indeed K_2 is by far the worst metamodel and sometimes is even not predictive, as it reaches the black line. The metamodels built on non-nested designs are often singularly more accurate than the metamodels built on nested designs. In the case of non-nested designs, which is the most interesting, no significant difference is shown in the distribution of training points between X 1 and X 2 : the case with 10 points for X 1 and 20 points for X 2 , or with 15 points for X 1 and 15 points for X 2 give similar accuracy. The same conclusion can be drawn from the comparison of the case with 20 points for X 1 and 40 points for X 2 , or with 30 points for X 1 and 30 points for X 2 .

As regards the comparison of the metamodels for the non-nested design case, the robust seqGPR metamodels are better than the full, they are equivalent to or better than K_tot. It seems that the cases with a small or high number of training points are more discriminating than the middle cases. With a small number of points (10pts/5pts), K_tot seems more destabilized than seqGPR. With a high number of points (20pts/40pts), seqGPR is more accurate than K_tot. Finally, seqGPR using Red is better than seqGPR using P.

In the following test cases, only the robust versions of seqGPR, which are more performing and whose parameter estimation is less complex, are compared to K_tot, and only the nonnested designs are used.

Analytic test case in dimension 15

The following test case should be less favorable to seqGPR. The objective function considered is in higher dimension and not decomposed as a sum of functions that respects the order in which the variables are released

f : [-3, 3] 15 → R x → a 1 x + a 2 sin x + a 3 cos x + x M x. (4.5)
The function f was first used in [Oakley and O'Hagan, 2004]. The values of its coefficients a 1 , a 2 , a 3 and M can be found in www.sheffield.ac.uk/st1jeo. The inputs are rescaled in [0, 1] and are rearranged by decreasing order of Sobol index, but for the sake of simplicity, the resulting output is still denoted by f . In practice, the first studies are done on the most influential inputs, chosen according to experts' physical knowledge. Here, as the function is analytical, the choice of the releasing order for the inputs is made by sensitivity analysis, which is directly performed on the function. Following the distribution of the Sobol indices (see subsection 2.1.4 in chapter 2 for their definition) shown on figure 4.4, a study of N = 3 steps is made, adding respectively the groups of variables I 1 = {1, 2, 3}, I 2 = {4, 5, 6, 7, 8, 9}, and I 3 = {10, 11, 12, 13, 14, 15}.

The variables that are fixed are set to 0.5. Three non-nested designs (built following the method suggested in section 5.2 in chapter 5), one for each step, are generated with 5 points per dimension considered at the step: This can be due to the fact that the P version is this time based on a stationary latent process contrarily to the previous test case. Indeed the variables which are fixed at a given step are set to a constant instead of varying in function of the free variables, as it is the case in the previous test case.

X 1 ⊂ [0, 1] 3 of size 15, X 2 ⊂ [0, 1]

Industrial test case

The industrial product which is studied in this section is the fan system (presented in section 1.1 in chapter 1) which is part of a car engine cooling system from Valeo company (see figure • The stagger angles at the five sections:

Stag1, Stag2, Stag3, Stag4, Stag5
• The chord lengths at the five sections: Chord1, Chord2, Chord3, Chord4, Chord5

• The sweeps at sections 2 to 5: Swe2, Swe3, Swe4, Swe5

In the rest of this test case, all variables are adimensionalized in [0, 1] and the output ∆P is adimensionalized in [-1, 1].

Industrial experts at Valeo have carried out a two-step (N = 2) study in the context of this work.

• At step 1, all the sweeps are fixed to constants: Swe2 = 0.517645, Swe3 = 0.82, Swe4 = 1, Swe5 = 1. An OLH (see subsection 2.1.5 in chapter 2 for its definition) of size 126 has been created on the 11 free inputs (I 1 = {1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11}).

• At step 2, all the inputs are free (I 2 = {12, 13, 14, 15}). An OLH of size 299 has been created for the 15 inputs.

The seqGPR metamodel is fitted on X 1 and X 2 where X 1 (resp. X 2 ) is a subset of size 50 extracted from the OLH generated at step 1 (resp. a subset of size 50 extracted from the OLH generated at step 2). The two subsets X 1 and X 2 are optimized for the generalized maximin criterion, they are non-nested. The complementary subset of X 2 is used as a test sample to evaluate the method. The procedure is made 30 times, each time new X 1 and X 2 are extracted. This cross-validation methodology enables to have a robust comparison of the models, and to test them in an extreme case with very few training points. ). The best performances are in bold.

Conclusion

In this chapter, two estimation methods are suggested. The first one is a decoupling of the likelihoods in case the designs are nested. The parameters of the Z i are estimated individually by optimizing the likelihoods of the Z i whose formulae are written in 3.4. The use of nested designs has the advantage of decoupling the predictions too. Their drawbacks are that they limit the size of the designs and the exploration of the input space. The second method is an EM algorithm that uses this decoupling of the likelihood in a sequential way. It uses non-nested designs that have the drawback of not decoupling the likelihood and prediction anymore. However, their size is not limited and neither is the exploration of the input space. This new estimation method, which transforms one complex optimization problem in several easier ones, is tested on a simulated example and the estimations are compared between Red and P processes. The conclusion is that they are equivalent and that the estimation is good for the covariance parameters and less accurate for the variance and mean parameters as it is the case for classic kriging.

The prediction accuracy of different versions of the metamodel seqGPR is compared to a classic kriging metamodel on one simple analytic test case, one more complicated and the industrial test case which motivated the thesis. The first conclusion is that the previous training samples, generated before the current step, bring useful information and improve the prediction of the metamodels. The second conclusion is that the non-nested DoE's give better results than the nested ones. Finally, the method seqGPR is competitive, and especially its Red_robust version (using Red process for the (Z n ) N n=2 with robust parametrization of their kernel).

Chapter 5

Designs of Experiments

This chapter describes building methods for the DoE's X 1 , • • • , X N in the seqGPR metamodel context (see equation (3.1) in chapter 3). As ρ 1 is a usual positive-definite kernel used in kriging, the only source of singularity due to X 1 for the covariance matrices involved in the estimation process is the presence of identical rows caused by redundant points in X 1 . A good value of the generalized maximin criterion (see subsection 2.1.5 in chapter 2) is sufficient to ensure a good space filling property of X 1 in [0, 1] d 1 and the good conditioning of ρ 1 (X 1 , X 1 ) (as the points are far from each other). Therefore, X 1 is taken as an LHS optimized for the generalized maximin criterion. However, the (X n ) N n=2 must verify other properties. The building methods introduced below enable a good space-filling of the samples and the invertibility of the covariance matrices involved in the estimation process (see chapter 4). It is assumed that the training sample sizes n n of X n (n ∈ 1, N ) are imposed, that the designs are built recursively and greedily (first X 1 , then X 2 , • • • , then X N ), and that the building method is identical for all designs from X 2 to X N .

A building method of (X n ) n∈ 2,N must be defined in a nested designs context. These designs are interesting as they enable the decoupling of the loss function for the parameter estimation (see subsection 4.1.1 in chapter 4). However, they are very constrained on the size which cannot increase with the step, and on the input space exploration, which is limited. In the case of nested designs, the covariance matrices that need to be inverted in the likelihoods are the

(cov(Z n (X n ), Z n (X n ))) N n=1 .
A building method must also be suggested to generate non-nested samples for the seqGPR metamodel. Non-nested designs have the advantage of being more flexible: no constraint on the size and more possibility to explore the input space. However the parameter estimation is more complicated as the loss function is not decoupled. The covariance matrices that need to be inverted in the EM algorithm (see subsection 4.1.2 in chapter 4) are the cov(Z n (

∼ X n ), Z n ( ∼ X n )) N n=1
. Nested and non-nested designs are illustrated on figure 5.1 in the case of the example in 2D of subsection 3.1.2 in chapter 3.

Building methods are described for samples X 2 to X N , in the case of nested designs in section 5.1, and in the case of non-nested designs in section 5.2.

Nested designs

This section describes the building method for nested designs. The constraints on X n (n 2), necessary to invert cov(Z n (X n ), Z n (X n )), are enumerated and lead to an optimization problem. An algorithm is then implemented to solve this optimization problem. As a recall, the size of the design X n cannot increase with n: n

N n N -1 • • • n 1 .

Iterative construction procedure

In addition to the space-fillingness and remoteness of the points, X n must be nested in X n-1 (X n X n-1 , see definition 7 in subsection 4.1.1 in chapter 4), that means that it must share the same values for the columns corresponding to the variables x I 1 ∪•••∪I n-1 . Secondly, because of the nullity property (see equation (3.3)) of Z n , a new source of singularity appears which is the presence of rows full of zeros at points where Z n is null almost surely. X n must then have an additional property which is its remoteness from the subspace {(x

I 1 ∪•••∪I n-1 , xIn ), x I 1 ∪•••∪I n-1 ∈ [0, 1] d 1 +•••+d n-1 }.
The following optimization problem is chosen to build X n :

     min n⊂[0,1] d 1 +•••+dn Φ n (X n ), under the constraint: X n X n-1 ,
with Φ n derived from the generalized maximin criterion (see subsection 2.1.5 in chapter 2):

Φ n (X n ) =    x,t∈Xn x =t 1 x -t 2 [0,1] d 1 +•••+dn p    1 p =φ 1 + x∈Xn 1 x In -xIn 2 [0,1] dn p 1 p φ 2 .
φ 1 ensures the space-fillingness and remoteness of the points of X n and φ 2 ensures that X n is far from the subspace where Z n is null. The default value of p is 50 in the R package DiceDesign and this value is imposed in what follows.

Numerical implementation

The algorithm consists in interverting rows from an initial design X (0) by a simulated annealing process. Denoting the sample X n-1 by:

X n-1 =    x (1) I 1 ∪•••∪I n-1 . . . x (n n-1 ) I 1 ∪•••∪I n-1    ,
the initial design is taken equal to:

X (0) =                     x (1) I 1 ∪•••∪I n-1 1 2nn . . . 1 2nn . . . . . . . . . x (k) I 1 ∪•••∪I n-1 1 2nn + k-1 nn . . . 1 2nn + k-1 nn . . . . . . . . . x (nn) I 1 ∪•••∪I n-1 1 -1 2nn . . . 1 -1 2nn x (nn+1) I 1 ∪•••∪I n-1 -1 . . . -1 . . . . . . . . . x (n n-1 ) I 1 ∪•••∪I n-1 -1 . . . -1                    
The leftmost columns correspond to the already free variables at step n -1,

x I 1 ∪•••∪I n-1 .
In the simulated annealing algorithm, all the rows can be interverted from row 1 to row n n-1 for these columns. To keep the nesting property, the components on the same row are constrained to stay together, i.e. only interversions between entire x (i)

I 1 ∪•••∪I n-1 are allowed.
The rightmost columns correspond to the newly released variables at step n, x In . Their values in the design belong to the dicretization { 1 2nn + k-1 nn , k ∈ 1, n n }. In the simulated annealing algorithm, only rows from 1 to n n can be interverted, so that the "-1" stay in the last rows. The "-1" are just added to complete the lines that are not kept. The initial associated candidate sample is :

X (0) c =         x (1) I 1 ∪•••∪I n-1 1 2nn . . . 1 2nn . . . . . . . . . x (k) I 1 ∪•••∪I n-1 1 2nn + k-1 nn . . . 1 2nn + k-1 nn . . . . . . . . . x (nn) I 1 ∪•••∪I n-1 1 -1 2nn . . . 1 -1 2nn .        
The simulated annealing algorithm is shown in algorithm 1 in appendix 9.4.2. It generates a sequence X (l) l 0 from which the corresponding sequence of candidate samples X (l) c l 0 is deduced by removing the rows of X (l) containing components equal to -1. X n is then equal to the last term of the sequence X (l) c l 0

, when the maximum of iterations or the threshold temperature has been reached.

Example in 2D

In the example in 2D from subsection 3.1.2 in chapter 3, in order for ρ 2 (X 2 , X 2 ) to be wellconditioned, X 2 must have distinct points sufficiently far from each other (to avoid identical rows) and from the nullity subset of Z 2 (to avoid rows full of zeros):

D = {(x 1 , 0.7x 1 + 0.2), x 1 ∈ [0, 1]}.
The last constraint means that: for all k in 1, n 2 , x

(i k ) 2
must be far from 0.7x

(i k ) 1 + 0.2.
The following optimization problem is chosen to built X 2 :

     min X 2 ⊂[0,1] 2 Φ 2 (X 2 ), under the constraint: X 2 X 1 ,
with Φ 2 derived from the generalized maximin criterion (see subsection 2.1.5 in chapter 2):

Φ 2 (X 2 ) =    n 2 k=1 n 2 l=1 l =k 1 (x (i k ) 1 -x (i l ) 1 ) 2 + (x (i k ) 2 -x (i l ) 2 ) 2 50    1 50 =φ 1 +   n 2 k=1 1 (x (i k ) 2 -0.7x (i k ) 1 -0.2) 2 50   1 50 =φ 2 .
To illustrate the building algorithm, it is assumed that nested samples X 1 , X 2 , respectively of size 4, and 3, must be generated. At step 1, the training sample is supposed to be equal

to X 1 =     0.2 0.4 0.6 0.8     . The variable x 2 is fixed to 0.7x 1 + 0.2. At step 2, the goal is to generate a sample X 2 nested in X 1 . X 2 contains the values of x 1 , x 2 .
The nesting property of X 2 implies that its size is smaller than or equal to the size of X 1 . For example, it is here imposed to 3. To build X 2 , a matrix is created containing the first column of X 1 and initializing the values of x 2 in the second column as a discretization of [0, 1]:

          0.2 1 6 0.4 3 6 0.6 5 6 0.8 -1          
. The matrix X 2 is deduced by taking the rows where

x 2 = -1. The initial value of X 2 is illustrated in figure below. Φ(X 2 ) = 52.57
The building algorithm consists in permutating rows inside each column until finding the optimal combination for the criterion Φ. The -1 stays at the bottom of the second column. X 2 is then deduced by keeping only the first 3 rows. The permutation is done on all rows in the first column and on the first three rows in the second column. One example of possible permutations is shown in figure below. Inside the first column (resp. the second column), the permutation 4 2 1 3 (resp. 2 3 1 ) is carried out. Φ(X 2 ) = 7.69

The optimal permutations for the criterion Φ are shown in figure below. They are found using algorithm 1 in appendix 9.4.2.

Φ(X 2 ) = 6.21

(a) X 2 nested in X 1 (b) X 3 nested in X 2 Figure 5
.2: Nested designs: X 1 in black, X 2 in red, and X 3 in green. Panel 5.2a is a 2D representation of X 1 and X 2 . The dotted red lines show that X 2 is nested in X 1 (shares same values of x 1 ). Panel 5.2b is a 3D representation of X 1 , X 2 , and X 3 . The green lines show that X 3 is nested in X 2 (shares same values of (x 1 , x 2 )).

The example in 4D is shown in appendix 9.1.3.

Other illustrations

Two examples in dimension 3 are considered, one with three steps and one with two steps.

Three-step example In this first example, the three steps are defined by:

• Step 1: x 1 is free. (x 2 , x 3 ) is fixed equal to (0.7x 1 + 0.2, x 1 +0.7x 1 +0.2 2
).

• Step 2: x 1 and x 2 are free. x 3 is fixed to x 1 +x 2 2 . • Step 3: all variables are free.

Three nested DoE's X 1 , X 2 , X 3 of size 20 are generated. X 1 is a grid in [0, 1]. X 2 and X 3 are built following algorithm 1. They are shown in figure 5.2. The design X 2 (resp. X 3 ) verifies the nesting property X 2 X 1 (resp. X 3 X 2 ), is well spread in [0, 1] 2 (resp. in [0, 1] 3 ), and is far from the subspace (x 2 , x 3 ) = (0.7x 1 + 0.2, x 1 +0.7x 1 +0. ) represented by the black line on panel 5.2a (resp. the subspace

x 3 = x 1 +x 2 2
represented by the black plane in panel 5.2b).

Two-step example

In this second example, the two steps are defined by:

• Step 1: x 1 is free. (x 2 , x 3 ) is fixed equal to (0.7x 1 + 0.2, -0.3x 1 + 0.5).

• Step 2: all variables are free.

Two nested designs X 1 and X 2 respectively of size 15 and 10 are generated, X 1 composed of equispaced points in [0, 1], and X 2 built following algorithm 1. They are shown in figure 5.3.

Non-nested designs

This section deals with the building method of X n (n 2) in a non-nested training designs context. In this case, the designs ∼ X n (n 2) play the role that was embodied by (X n ) N n=1 in the previous section as they are nested. ∼ X n is defined as the concatenations of the parts of X n , ..., X N corresponding to the variables x I 1 ∪•••∪In (see definition 8 in chapter 4). In order to invert the matrices (cov(Z n (

∼ X n ), Z n ( ∼ X n ))) N
n=1 , the designs ∼ X n must have points sufficiently appart from each other (for n 1) and far from the nullity subspaces (for n 2). From this wish, optimization problems are defined for the different submatrices of the designs and solved by a simulated annealing algorithm.

Iterative construction procedure

The design ∼ X n (n ∈ 1, N ) must verify the same constraints than X n in section 5.1:

• It must be space-filling in [0, 1] d 1 +•••+dn and have distinct points sufficiently far from each other.

• It must have points far from the subspace on which Z n is null (for n 2).

These two properties are conditions to the invertibility of the covariance matrices.

The design

X n (n ∈ 2, N ) is involved in ∼ X 1 , • • • , ∼ X n .
In particular, at step n, the following samples X n+1 , • • • , X N do not exist yet and are not taken into account in the samples

∼ X 1 , • • • , ∼ X n . The parts of the samples ∼ X 1 , • • • , ∼
X n taken into account in the building of X n (that are formed only using

X 1 , • • • , X n ) are called ∼ X n 1 , • • • , ∼ X n n .
To ensure the properties that must be verified by these samples, the chosen approach is to build X n by group of variables: first build the columns corresponding to x I 1 (called X n,I 1 ), then build the columns corresponding to x I 2 (called X n,I 2 ), • • • , then to x In (called X n,In ). Each part of X n must solve a particular optimization problem.

• X n,I 1 must solve the following optimization problem min

X n,I 1 ⊂[0,1] d 1 Φ n,1 (X n,I 1 ), with Φ n,1 equal to: Φ n,1 (X n,I 1 ) =      x,t∈ ∼ X n 1 x =t 1 x -t 2 [0,1] d 1 +•••+dn 50      1 50
Φ n,1 is the generalized maximin applied to the sample

∼ X n 1 . • X n,I k (k ∈ 2, n ) must solve the following optimization problem:      min X n,I k ⊂[0,1] d 1 +•••+d k Φ n,k (X n,I k ),
under the constraint:

X n,I 1 ∪•••∪I k X n,I 1 ∪•••∪I k-1 , with Φ n,k equal to: Φ n,k (X n,I k ) =      x,t∈ ∼ X n k x =t 1 x -t 2 [0,1] d 1 +•••+dn 50      1 50 +   x∈X n,I 1 ∪•••∪I k 1 x I k -xI k 2 [0,1] d k 50   1 50
.

The first part is the generalized maximin applied to

∼ X n k , the second part ensures that X n,I 1 ∪•••∪I k is far from the nullity subspace {(x I 1 ∪•••∪I k-1 , xI k ), x I 1 ∪•••∪I k-1 ∈ [0, 1] d 1 +•••+d k-1 }
where Z k is null almost surely.

Numerical implementation

From X 1 , • • • , X n-1 , X n is built block by block.

• If I 1 = {1}, the matrix X n,I 1 is a column whose components are adjusted by hand to have a good value of Φ n,1 . If I 1 is of length greater than 1, the matrix X n,I 1 is created by interverting rows from an initial design X (0) by a simulated annealing process, in order to minimize the criterion Φ n,1 (X n,I 1 ). X (0) is an LHS design of size n n independent from

∼ X n-1 1
, with d 1 columns. The simulated annealing algorithm is described in algorithm 2 in appendix 9.4.2.

• For k ∈ 2, n , X n,I 1 ∪•••∪I k is created by interverting rows from an initial design X (0) by a simulated annealing process, in order to minimize the criterion Φ n,k (X n,I k ). Denoting the sample X n,I 1 ∪•••∪I k-1 by:

X n,I 1 ∪•••∪I k-1 =    x (1) I 1 ∪•••∪I k-1 . . . x (nn) I 1 ∪•••∪I k-1    ,
the initial design is taken equal to:

X (0) =          x (1) I 1 ∪•••∪I k-1 1 2nn . . . 1 2nn . . . . . . . . . x (i) I 1 ∪•••∪I k-1 1 2nn + i-1 nn . . . 1 2nn + i-1 nn . . . . . . . . . x (nn) I 1 ∪•••∪I k-1 1 -1 2nn . . . 1 -1 2nn         
The simulated annealing algorithm is again algorithm 1, as in section 5.1, where

X n,I 1 ∪•••∪I k (resp. X n,I 1 ∪•••∪I k-1 and Φ n,k ) plays the role of X n (resp. X n-1 and Φ n ).
This time both samples are of the same size, all the rows are kept, and there is no need to intervert rows on the left side.

Examples

Example in 2D In the example in 2D from subsection 3.1.2 in chapter 3, X 1 is supposed

equal to X 1 =    x (1) 1
. . . The chosen approach is to build X 2 one column at a time.

• The first column

X 2,1 =    x (n 1 +1) 1 . . . x (n 1 +n 2 ) 1    must solve the following optimization problem min X 2,1 ⊂[0,1] Φ 2,1 (X 2,1 ), with Φ 2,1 equal to: Φ 2,1 (X 2,1 ) =    n 1 +n 2 i=1 n 1 +n 2 j=1 j =i 1 (x (i) 1 -x (j) 1 ) 2 50    1 50
• When the first column is created, the problem is reduced to the one of the previous subsection, of generating a design X 2 nested in X 2,1 . The second column

X 2,2 =    x (n 1 +1) 2 . . . x (n 1 +n 2 ) 2    must solve the following optimization problem:      min X 2,2 ⊂[0,1] Φ 2,2 (X 2,2 ), under the constraint: X 2 X 2,1 ,
with Φ 2,2 equal to:

Φ 2,2 (X 2,2 ) =    n 1 +n 2 i=n 1 +1 n 1 +n 2 j=n 1 +1 j =i 1 (x (i) 1 -x (j) 1 ) 2 + (x (i) 2 -x (j) 2 ) 2 50    1 50 +   n 1 +n 2 i=n 1 +1 1 (x (i) 2 -0.7x (i) 1 -0.2) 2 50   1 50
.

To illustrate the building method, it is assumed that a design

X 1 =            
has been generated at step 1. The variable x 2 is implicitely fixed to 0.7x 1 + 0.2. The goal is now to build X 2 of size 4 with x 2 that can take its values freely in [0, 1].

• The components of the first column X 2,1 are adjusted by hand so that ∼ X 1 has all the required properties. For example, X 2,1 is chosen equal to:

X 2,1 =           0 6 2 6 4 6 6 6           . The corresponding ∼ X 1 is represented below. • Then the second column is built so that X 2 is nested in X 2,1 . It is initialized to X 2 =           0 6 1 8 2 6 3 8 4 6 5 8 6 6 7 8          
. The optimal design for the criterion Φ 2 is obtained for the permutation 3 4 1 2 in the second column, giving the design: Optimal design: Φ 2 (X 2 ) = 5.05

X 2 =          
The example in 4D is shown in appendix 9.1.3.

Example in 3D

In the examples in 2D (see paragraph above) and 4D (see appendix 9.1.3 ), the case I 1 of length greater than 1 is not tackled. So this case is illustrated in the following example in 3D. In this example, the first two variables are released at step 1 (I 1 = {1, 2}), x 3 being fixed to 0.6 and then released at step 2 (I 2 = {3}). Designs X 1 of size 4 and X 2 of size 5 must be created respectively at step 1 and 2.

• At step 1, the DoE X 1 is taken as an LHS optimized for the generalized maximin criterion:

X 1 =    
0.91 0.51 0.11 0.33 0.52 0.20 0.43 0.78

   
• At step 2, the goal is to create the design X 2 of size 5.

-The submatrix X 2,I 1 composed of the first two columns of X 2 is build in first. It is initialized to an LHS: X

             
(the concatenation of X 1 and X 2,I 1 ). The optimal design is obtained for the permutation 2 1 4 3 5 in the first column of X 2,I 1 and for the permutation Optimal design: Φ 2,1 (X 2,I 1 ) = 5.05 -X 2 is then built as a design in dimension 3 nested in X 2,I 1 . The goal is to built its third column. The design is initialized to

X 2 =               0.06 0.
              , Φ 2,2 (X 2 ) = 12.73.
The optimal design for the criterion Φ 2,2 is obtained for the permutation 3 2 4 5 1 in the last column. It is found using algorithm 1 in appendix 9.4.2.

(a) Projections of (black line). X 2 is represented by the red points. It is the sample from step 2 when x 2 is released and

X 1 , X 2 , X 3 on x 1 (b) Projections of X 2 , X 3 on (x 1 , x 2 ) (c) X 1 , X 2 , X 3 samples in 3D
x 3 = x 1 +x 2
2 . X 3 is represented by the green points. It is the sample from step 3 when all inputs are released. Its projections on the plane together with the X 2 points form a space filling design of [0, 1] 2 (see panel 5.4b). Its projections on the line together with X 1 and X 2 points form a space filling design of [0, 1] (see panel 5.5a). X 1 , X 2 , and X 3 are represented in 3D in panel 5.4c

X 2 =               0.06 0.
              , Φ 2,2 (X 2 ) = 11.89.

Other illustrations

Two examples in dimension 3 are considered, one with three steps and one with two steps. Three-step example It is the same three-step example as in section 5.1 but this time X 1 is taken of size 5, X 2 of size 8, and X 3 of size 12, which is not enabled in the previous case as the size of X n cannot increase with n. See figure 5.4. Here the algorithm 2 is not used as the x 1 values of the different samples are just taken from an equispaced discretization of [0, 1].

Two-step example

In this example there are three inputs (x 1 , x 2 , x 3 ).

• At step 1, x 1 and x 2 are free, x 3 is fixed equal to x 1 +x 2 2 .

• At step 2, all inputs are free. (black plane). X 2 is represented by the red points. It is the sample from step 2 when x 3 is released. Its projections on the plane together with the X 1 points form a space filling design of [0, 1] 2 . X 1 and X 2 are represented in the 3D input space on panel 5.5b. Their 2D projections on the plane are shown on panel 5.5a.

X 1 is of size 30 and X 2 of size 20. See figure 5.5. This time, algorithm 2 has been used to attribute the values of (x 1 , x 2 ) to X 2 such that its projection, together with X 1 , forms a space filling design with remote points in [0, 1] 2 .

Conclusion

In this chapter, a building method is suggested to generate nested samples for the seqGPR metamodel. From the wish to have designs with points sufficiently appart from each other and far from the nullity subspaces, an optimization problem is defined and solved by a simulated annealing algorithm. Two illustrations are then shown of the designs resulting from this algorithm: one example with three steps respectively in 1D, 2D and 3D, and one example with two steps respectively in 1D and 3D. Nested designs have the advantage of making the parameter estimation of the seqGPR metamodel easier as the loss function can be decoupled, however they limit the exploration of the input space as their building is very constrained, and their size cannot increase with the number of the step which is problematic with the increasing dimension of the subspace considered.

A building method is also suggested to generate non-nested samples for the seqGPR metamodel. From the wish to have designs with points sufficiently appart from each other and far from the nullity subspaces, optimization problems are defined for the different submatrices of the designs and solved by a simulated annealing algorithm. Two illustrations are then shown of the designs resulting from this algorithm: one example with three steps respectively in 1D, 2D and 3D, and one example with two steps respectively in 2D and 3D. Non-nested designs have the advantage of being more flexible: no constraint on the size and more possibility to explore the input space. However the parameter estimation is more complicated as the loss function is not decoupled. This problem is solved in subsection 4.1.2 in chapter 4 with the use of the EM algorithm.

Chapter 6

Additional contributions

Previous chapters define the seqGPR metamodel (caracterized by the equation (3.1) in chapter 3), build its underlying processes, suggest estimation methods for its parameters, and propose methodologies to generate its DoE's. This chapter introduces new works on the se-qGPR metamodel to try to answer some unsolved issues. Section 6. Here, the original model (3.1) is considered. After having fitted the seqGPR metamodel on the samples, the goal is to add relevant points to the samples to improve its prediction. In particular, a criterion to use in the search for enrichment points is proposed and the enrichment procedure is evaluated on the test cases of chapter 4.

Conditioning on multiple subspaces

This section describes the case where, at each step, several subspaces corresponding to different values of the fixed inputs are considered (instead of one as that was the case until now).

The adapted seqGPR model is detailed and new candidates for the (Z n ) n 2 , which verify a new nullity property, are suggested. The new metamodel is then compared to classic kriging on analytic test cases.

Model

Let f (x I 1 , • • • , x I N )
be the output to approximate by a metamodel. Let

B 2 =      0 b 2 2 . . . b K 2 2      ⊂ [0, 1] d 2 , B 3 =      0 b 2 3 . . . b K 3 3      ⊂ [0, 1] d 3 , • • • , B N =      0 b 2 N . . . b K N N      ⊂ [0, 1] d N ,
be matrices such that B n contains the constant values which constitute the difference between the subspaces at step n -1. The DoE's and subspaces considered are illustrated in figure 6 n+1 . As a consequence, the process Y n modeling the output at step n must also be function of x I n+1 . The problem is modeled by the following statistical model:

   Y 1 (x I 1 , x I 2 ) = m + Z 1 (x I 1 , x I 2 ), Y n (x I 1 ∪•••∪In , x I n+1 ) = Y n-1 (x I 1 ∪•••∪In ) + Z n (x I 1 ∪•••∪In , x I n+1 ), Y N (x I 1 , • • • , x I N ) = Y N -1 (x I 1 , • • • , x I N ) + Z N (x I 1 , • • • , x I N ), . (6.1)
where:

• The processes

(Z n ) 1 n N are centered independent Gaussian processes of covariance kernel (σ 2 n ρ n ) N n=1
• They must verify the following property:

Z n (x I 1 ∪•••I n-1 , xIn + B n , xI n+1 ) = 0, ∀n ∈ 2, N -1 , ∀x I 1 ∪•••I n-1 ∈ [0, 1] d 1 +•••+d n-1 , Z N (x I 1 ∪•••∪I N -1 , xI N + B N ) = 0, ∀x I 1 ∪•••∪I N -1 ∈ [0, 1] d 1 +•••+d N -1 . (6.2)
Estimation and prediction of this metamodel is not different from the methodology described in chapter 4. However, the nullity property (6.2) is more complicated than (3.3), as it is on multiple subspaces and not just one. The definition of a correction process Z n (n 2) verifying it is the object of the next subsection.

Candidate for the correction processes

The goal of this subsection is to build a process Z : [0, 1] d J +d I × Ω → R such that:

Z(x J , g(x J ) + B) = 0 ∀x J ∈ [0, 1] d J , (6.3) with g : [0, 1] d J → [0, 1] d I a deterministic function, and B = {b 1 , • • • , b K } a finite sub- set of [0, 1] d I . Z(x J , g(x J ) + B) is the vector of size K whose component i (i ∈ 1, K ) is equal to Z(x J , g(x J ) + b i ).
The proposed candidate is derived from the P process which is the easiest to generalize to this case (see subsection 3.2.3 in chapter 3). It is built based on a latent process ∼ Z ∼ GP (0, σ 2 r((x J , x I ), (t J , t I ))) of covariance kernel σ 2 r. It is defined as:

Z P (x J , x I ) = ∼ Z(x J , x I ) -E ∼ Z(x J , x I ) | Z(t J , g(t J ) + b), ∀t J ∈ [0, 1] d J , ∀b ∈ B ,
where

E ∼ Z(x J , x I ) | Z(t J , g(t J ) + b) ∀t J ∈ [0, 1] d J , ∀b ∈ B is the projection of ∼ Z(x J , x I ) on
the sub-Gaussian space engendered by the family

{ ∼ Z(t J , g(t J ) + b), t J ∈ [0, 1] d J , b ∈ B}.
Its general expression is given in subsection 2.2.2 in chapter 2 and is recalled below:

E ∼ Z(x) | ∼ Z(D) = +∞ n=1 φ n (x) S ∼ φ n (s) ∼ Z(x s )dν(s), (6.4)
with D = {x s , s ∈ S}, ν a measure on S, and

φ n (x) = 1 λn S σ 2 r(x, x s ) ∼ φ n (s)dν(s). (λ n , ∼ φ n ) n 1 are solutions of the eigen problem: S σ 2 r (x s , x u ) ∼ φ n (u)dν(u) = λ n ∼ φ n (s), ∀s ∈ S, such that S ∼ φ n (s) ∼ φ m (s)dν(s) = δ nm ∀n, m 1,
where δ nm is the Kronecker symbol. It is applied to the case:

         D = {(s J , g(s J ) + s I ), s = (s J , s I ) ∈ S}, S = [0, 1] d J × B = {(s J , s I ), s J ∈ [0, 1] d J , s I ∈ B}, ν = λ ⊗ K i=1 δ b i , x s = (s J , g(s J ) + s I ),
with λ the Lebesgues measure on [0, 1] d J and δ b i the Dirac measure at b i .

Proposition 13 Let r J and r I be two stationary correlation kernels. If r is of the form:

r((x J , x I ), (t J , t I )) = r J (x J , t J )r I (x I -g(x J ), t I -g(t J )),
then the process Z P is a centered Gaussian process equal to:

Z P (x J , x I ) = ∼ Z(x J , x I ) -r I (x I -g(x J ), B)r I (B, B) -1 ∼ Z(x J , g(x J ) + B), ∀(x J , x I ) ∈ [0, 1] d J +d I .
whose covariance kernel is σ 2 ρ P with:

ρ P ((x J , x I ), (t J , t I )) = r J (x J , t J ) [r I (x I -g(x J ), t I -g(t J )) -r I (x I -g(x J ), B)r I (B, B) -1 r I (B, t I -g(t J ))] .
Proof See appendix 9.3.3 for a proof of this proposition.

Example in 2D

The example in 2D from subsection 3.1.1 in chapter 3 is here adapted to the multi-conditioning case. A two-step study is carried out:

• At step 1, two subspaces are considered: x 2 = 0.7x 1 + 0.2 and x 2 = 0.7x 1 .

Here, each subspace is indexed by

x 1 : {(x 1 , 0.7x 1 +0.2), x 1 ∈ [0, 1]} and {(x 1 , 0.7x 1 ), x 1 ∈ [0, 1]}.
The restrictions of f on these two subspaces f (x 1 , 0.7x 1 + 0.2) and f (x 1 , 0.7x 1 ) need to be approximated. These restrictions are modeled by one Gaussian process

Y 1 (x 1 , x 2 ) = m + Z 1 (x 1 , x 2 )
with m a scalar equal to the mean of Y 1 , and Z 1 a centered Gaussian process of covariance kernel k 1 ((x 1 , x 2 ), (t 1 , t 2 )). The difference with the previous model is that Y 1 is also a function of x 2 . This process is such that

f (x 1 , 0.7x 1 + 0.2) (resp. f (x 1 , 0.7x 1 )
) is the realization of its restriction on the first subspace Y 1 (x 1 , 0.7x 1 + 0.2) (resp. on the second subspace Y 1 (x 1 , 0.7x 1 )). Two DoE's are generated:

X 1 1 =    x (1,1) 1 0.7x (1,1) 1 + 0.2 . . . . . . x (1,n 1 ) 1 0.7x (1,n 1 1 ) 1 + 0.2  
 belonging to the first subspace and

X 2 1 =    x (1,n 1 1 +1) 1 0.7x (1,n 1 1 +1) 1 . . . . . . x (1,n 1 1 +n 2 1 ) 1 0.7x (1,n 1 1 +n 2 1 ) 1    belonging to the second subspace. The values of f at the points of X 1 1 are known, equal to y 1 1 =    f 1 (x (1,1) 1 , 0.7x (1,1) 1 + 0.2) . . . f 1 (x (1,n 1 1 ) 1 , 0.7x (1,n 1 1 ) 1 + 0.2)    . Similarly, the values of f at the points of X 2 1 are known, equal to y 2 1 =    f 1 (x (1,n 1 1 +1) 1 , 0.7x (1,n 1 1 +1) 1 ) . . . f 1 (x (1,n 1 1 +n 2 1 ) 1 , 0.7x (1,n 1 1 +n 2 1 ) 1 )    . • At step 2, the entire input space [0, 1] 2 is considered. The function to approximate is now f 2 (x 1 , x 2 ) = f (x 1 , x 2 ). It is modeled by a Gaussian process Y 2 (x 1 , x 2 ) linked to Y 1 (x 1 , x 2 ) by the formula Y 2 (x 1 , x 2 ) = Y 1 (x 1 , x 2 ) + Z 2 (x 1 , x 2 )
. This formula looks like the multifidelity metamodel, but the difference is that Y 1 and Y 2 model the same level of fidelity. Thus, they must coincide on the subspaces x 2 = 0.7x 1 + 0.2 and x 2 = 0.7x 1 . This coincidence, shown in the figures below, implies that Z 2 (x 1 , 0.7x 1 + 0.2) = 0 and Z 2 (x 1 , 0.7x 1 ) = 0. Figures on the top show Y 2 (on the left) and Y 1 (on the right) and their coincidence on the first subspace in orange and on the second subspace in blue. Their restriction on these subspaces are illustrated on the bottom figures (subspace x 2 = 0.7x 1 + 0.2 on the left and x 2 = 0.7x 1 on the right).

Visualization of Y 2 Y 1 as a function of (x 1 , x 2 ) First restriction of Y 1 Second restriction of Y 1 A DoE is generated X 2 =    x (2,1) 1 x (2,1) 2 . . . . . . x (2,n 2 ) 1 x (2,n 2 ) 2    containing values of (x 1 , x 2 )
. The values of f 2 at the points of X 2 are known, equal to

y 2 =    f 2 (x (2,1) 1 , x (2,1) 2
) . . .

f 2 (x (2,n 2 ) 1 , x (2,n 2 ) 2 )    . An example of the DoE's X 1 1 , X 2
1 and X 2 is shown in the figure below:

The samples X 1 1 , X 2 1 , X 2 belong to the same space [0, 1] 2 . To apply the EM algorithm, a new definition of ∼ X 1 and ∼ X 2 is used, as shown below:

The goal is to find a candidate for the process Z 2 (x 1 , x 2 ), modeling the piece of information added at step 2 by releasing x 2 which was fixed to 0.7x 1 + 0.2 or 0.7x 1 at step 1. On the nullity subspaces

D 1 = {(x 1 , 0.7x 1 + 0.2), x 1 ∈ [0, 1]} and D 2 = {(x 1 , 0.7x 1 ), x 1 ∈ [0, 1]}, Z 2
adds no information and must be equal to 0:

Z 2 (x 1 , 0.7x 1 + 0.2) = 0, ∀x 1 ∈ [0, 1], Z 2 (x 1 , 0.7x 1 ) = 0, ∀x 1 ∈ [0, 1].
How to find a Gaussian process Z 2 verifying this nullity property and having a computable covariance kernel ? The idea suggested in what follows is to build Z 2 by a transformation of a latent centered Gaussian process ∼ Z 2 whose kernel σ 2 2 r 2 is known and computable. The transformation must impose the nullity property, keep the law of the process Gaussian, and keep the computability of the kernel. The transformation consists in conditioning the latent process to be null on the reunion of the nullity subspaces D 1 and D 2 . As this nullity subspace is an infinite continuous set of points, the usual conditioning seen in proposition 2 cannot be used directly. Instead, a generalization of the conditioning, defined in [START_REF] Gauthier | Spectral approach for kernelbased interpolation[END_REF], is used. The resulting process, derived from the P process, is defined as:

Z 2 (x 1 , x 2 ) = ∼ Z 2 (x 1 , x 2 ) -E ∼ Z 2 (x 1 , x 2 ) | ∼ Z 2 (t 1 , 0.7t 1 + 0.2), ∼ Z 2 (t 1 , 0.7t 1 )∀t 1 ∈ [0, 1] , where E ∼ Z 2 (x 1 , x 2 ) | ∼ Z 2 (t 1 , 0.7t 1 + 0.2), ∼ Z 2 (t 1 , 0.7t 1 )∀t 1 ∈ [0, 1] (denoted by E(x 1 , x 2 )
to simplify notations) is itself a generalization of the conditional expectation, defined as the projec-

tion of ∼ Z 2 (x 1 , x 2 ) in the subGaussian space generated by ∼ Z 2 (t 1 , 0.7t 1 + 0.2), ∼ Z 2 (t 1 , 0.7t 1 ) t 1 ∈[0,1]
.

The formula of the expectation, taken from the proposition 7 and recalled in the next paragraph, is here adapted to the case

D = D 1 ∪ D 2 = {(s 1 , 0.7s 1 + s 2 ), s = (s 1 , s 2 ) ∈ [0, 1] × {0, 0.2}}, S = [0, 1] × {0, 0.2}
, and ν any measure on S:

E(x 1 , x 2 ) = +∞ n=1 φ n (x 1 , x 2 ) s∈S ∼ φ n (s) ∼ Z 2 (s 1 , 0.7s 1 + s 2 )dν(s),
where:

φ n (x 1 , x 2 ) = 1 λ n s∈S σ 2 2 r 2 ((x 1 , x 2 ), (s 1 , 0.7s 1 + s 2 )) ∼ Z 2 (s 1 , 0.7s 1 + s 2 )dν(s),
and (λ n , ∼ φ n ) +∞ n=1 are the solutions of the eigenvalue problem:

s∈S σ 2 2 r 2 ((s 1 , 0.7s 1 + s 2 ), (u 1 , 0.7u 1 + u 2 )) ∼ φ n (u)dν(u) = λ n ∼ φ n (s). ( ∼ φ n ) n 1 is an orthonormal basis of L 2 ([0, 1], ν): s∈S ∼ φ n (s) ∼ φ m (s)dν(s) = δ nm .
This kernel cannot be used just as it is in practice, because the solutions of the eigenvalue problem are not explicit in general and the sums are infinite. To make this kernel tractable, a wise choice of r 2 (the correlation kernel of the latent process ∼ Z 2 ) is suggested, for which an explicit formula is known. If the covariance kernel of ∼ Z 2 is of the form:

σ 2 2 r 2 ((x 1 , x 2 ), (t 1 , t 2 )) = σ 2 2 r 2,1 (x 1 , t 1 )r 2,2 (x 2 -0.7x 1 , t 2 -0.7t 1 )
, with r 2,1 and r 2,2 stationary kernels, and if Z 2 is a P process, then according to proposition 13, it is equal to:

Z 2 (x 1 , x 2 ) = ∼ Z 2 (x 1 , x 2 ) -r 2,2 (x 2 -0.7x 1 , 0) r 2,2 (x 2 -0.7x 1 , 0.2) r 2,2 (0, 0) r 2,2 (0, 0.2) r 2,2 (0.2, 0) r 2,2 (0.2, 0.2) -1 ∼ Z 2 (x 1 , 0.7x 1 ) ∼ Z 2 (x 1 , 0.7x 1 + 0.2)
.

It is a centered Gaussian process of covariance kernel σ 2 2 ρ 2 with: r 2,2 (0.2,0) r 2,2 (0.2,0 r 2,0.2) ].

ρ 2 ((x 1 , x 2 ), (t 1 , t 2 )) = r 2,1 (x 1 , t 1 )[r 2,2 (x 2 -0.7x 1 , t 2 -0.7t 1 ) -r 2,2 (x 2 -0.7x 1 , 0) r 2,2 (x 2 -0.7x 1 , 0.2) r 2,2 (0, 0) r 2,2 (0, 0.2)
.2) -1 r 2,2 (t 2 -0.7t 1 , 0)

Test cases

The seqGPR metamodel is compared to the classic kriging metamodel (called K_tot) in the two analytic test cases from section 4.2 in chapter 4. Two types of parametrization are proposed for the (Z n ) N n=1 in the seqGPR metamodel:

• Semi-robust: Z 1 with one covariance parameter θ i 1 for each input variable x i , i ∈ I 1 ∪I 2 , for all n in 2, N -1 , Z n with one covariance parameter α n common to all components of x I 1 ∪•••∪I n-1 , and one covariance parameter θ i n for each component of x In∪I n+1 , and Z N with one covariance parameter α N common to all components of x I 1 ∪•••∪I N -1 , and one covariance parameter θ i N for each input x i , i ∈ I N ):

                             θ 1 =   θ 1 1 , • • • , θ d 1 1 I 1 , θ d 1 +1 1 , • • • , θ d 1 +d 2 1 I 2   , θ n =    α n , • • • , α n I 1 ∪•••∪I n-1 , θ 1 n , • • • , θ dn n In , θ dn+1 n , • • • , θ dn+d n+1 n I n+1    , ∀n ∈ 2, N -1 , θ N =   α N , • • • , α N I 1 ∪•••∪I N -1 , θ 1 N , • • • , θ d N N I N   .
• Robust: Z 1 with one covariance parameter θ i 1 for each variable x i , i ∈ I 1 , and one covariance parameter β 1 common to all variables x i , i ∈ I 2 , for all n in 2, N -1 , Z n with one covariance parameter α n common to all components of x I 1 ∪•••∪I n-1 , one covariance parameter θ i n for each component of x In , and one covariance parameter β n common to all components of x I n+1 , and Z N with one covariance parameter α N common to all components of x I 1 ∪•••∪I N -1 , and one covariance parameter for each variable

x i , i ∈ I N .                              θ 1 =   θ 1 1 , • • • , θ d 1 1 I 1 , β 1 , • • • , β 1 I 2   , θ n =   α n , • • • , α n I 1 ∪•••∪I n-1 , θ 1 n , • • • , θ dn n In , β n , • • • , β n I n+1   , ∀n ∈ 2, N -1 , θ N =   α N , • • • , α N I 1 ∪•••∪I N -1 , θ 1 N , • • • , θ d N N I N   .
The classic kriging metamodel, K_tot, is trained on the reunion of the designs (X g 2 (x 1 ,x 2 ,x 3 ,x

            f (x 1 , x 2 , x 3 , x 4 ) = g 1 (x 1 , x 2 ) + g 2 (x 1 , x 2 , x 3 , 4 ) with g 1 (x 1 , x 2 ) = 4 -2.1(4x 1 -2) 2 + (4x 1 -2) 4 3 (4x 1 -2) 2 + (
4 ) = 4 exp -x -0.3 2 .
The study is composed of two steps

• At step 1, three different restrictions are considered    f 1 1 (x 1 , x 2 ) = f (x 1 , x 2 , 0.4757155, 0.7005368), f 2 1 (x 1 , x 2 ) = f (x 1 , x 2 , 0.2092920, 0.2661060), f 3 1 (x 1 , x 2 ) = f (x 1 ,
x 2 , 0.9114971, 0.3639307). The fixed values of (x 3 , x 4 ) are chosen so that they form an LHS optimized for the generalized maximin criterion in [0, 1] 2 . The DoE is of the form: 0.9114971 0.3639307

X 1 =                   x (1) 1 x (
                 
.

The same values of (x 1 , x 2 ) are taken in the subspaces. They form an LHS optimized for the generalized maximin criterion in [0, 1] 2 .

• At step 2, new simulations are launched at points X 2 , a design of size 20 in dimension 4. The last two variables (x 3 , x 4 ) are now released. X 2 is an LHS, independent from X 1 , also optimized for the generalized maximin criterion. The two versions of the seqGPR metamodel have worse performances than K_tot (higher median and wider interquartile interval). No type of parametrization stands out from the other. One explanation for this counter performance of the method is that it is more complex than in the previous case with one subspace at each step. The input spaces of the (Z n ) N n=1 are in higher dimension and the initial philosophy that consists in explaining a maximum of the data thanks to the small dimension is lost.

Test case from subsection 4.2.3 in chapter 4

The output is the same as in subsection 4.2.3 in chapter 4:

f : [-3, 3] 15 → R x → a 1 x + a 2 sin x + a 3 cos x + x M x.
K_tot Semi-robust Robust Median (•10 ). K denotes K_tot, S denotes the semi-robust version of seqGPR, and R denotes the robust version of seqGPR. 45pts-45pts/45pts means that X 1 is of size 90 (with 45 points in the first subspace and 45 points in the second subspace), and that X 2 is of size 45. 45pts-45pts/75pts means that X 1 is of size 90 (with 45 points in the first subspace and 45 points in the second subspace), and that X 2 is of size 75. 90pts-90pts/150pts means that X 1 is of size 180 (with 90 points in the first subspace and 90 points in the second subspace), and that X 2 is of size 150. The best performances are in bold.

The inputs are rescaled in [0, 1] and are rearranged by decreasing order of Sobol index, but for the sake of simplicity, the resulting output is still denoted by f . The study is composed of two steps:

• At step 1, two restrictions are considered:

f 1 (x 1 , x 2 , x 3 , x 4 , x 5 , x 6 , x 7 , x 8 , x 9 ) = f (x 1 , x 2 ,
x 3 , x 4 , x 5 , x 6 , x 7 , x 8 , x 9 , 0, 0, 0, 0, 0, 0),

f 2 (x 1 , x 2 , x 3 , x 4 , x 5 , x 6 , x 7 , x 8 , x 9 ) = f (x 1 , x 2 , x 3 , x 4 , x 5 , x 6 , x 7 , x 8 , x 9 , 1, 1, 1, 1, 1, 1).
A DoE X 1 is generated, with the same values of (x 1 , • • • , x 9 ) taken in each subspace, such that they form an LHS optimized for the generalized maximin criterion.

• At step 2, the last variables (x 10 , • • • , x 15 ) are released. New simulations are launched at points X 2 ⊂ [0, 1] 15 . X 2 is an LHS, independent from X 1 , also optimized for the generalized maximin criterion.

The RMSE median and interquartile range, computed on a Sobol sequence of size 10000 used as a test set, are given for 100 different DoE's (X 1 , y 1 ) and (X 2 , y 2 ) in table 6.2. The performances of the metamodels increase with the size of the training sample, but the comparison between the metamodels is the same for each training sample. Again, the same conclusion can be drawn: the two versions of the seqGPR metamodel have worse performances than K_tot (higher median and wider interquartile interval).

Enrichment

In this section, the non-nested designs are used: for all n in 1, N , the x I 1 ∪•••∪In part of X n , • • • , X N have no points in common (see section 5.2 in chapter 5). An enrichment method for the DoE X N of the seqGPR metamodel is suggested. The proposed optimization problem and the associated enrichment method are detailed. The method is then tested on analytic test cases.

Process of enrichment

The following optimization problem (derived from the one of [START_REF] Gratiet | Multi-fidelity Gaussian process regression for computer experiments[END_REF] described in subsection 2.1.5 in chapter 2) is chosen to seek an interesting point to add to the training sample

               max x∈X v(x) • 1 + n 1 +...+n N i=1 (y i -ŷ-i (x (i) )) 2 v-i (x (i) ) 1 x∈V i , under the constraints: dist(x I 1 , ∼ X 1 ) > u, dist(x In , xIn ) > v n , ∀n ∈ 2, N (6.5)
The thresholds u and (v n ) N n=2 are user defined. The constraints aim at keeping the invertibility property of the

∼ X n N n=1 (so that ρ n ∼ X n , ∼ X n N n=1 are invertible). The constraint dist(x I 1 , ∼ X 1 ) > u also implies that dist(x I 1 ∪•••∪x In , ∼ X n ) is sufficiently big for n 2.
The goal is to add a batch of points of size L (specified by the user) to the design X N , by solving the previously defined optimization problem. The enrichment algorithm is described below. It requires DoE's (X 1 , y 1 ), ..., (X N , y N ), and a seqGPR metamodel trained on these samples.

Compute the LOO-CV errors (y

i -ŷ -i (x (i) )) 2 v-i (x (i) )
of the seqGPR metamodel. These will not be updated during the algorithm.

2. Generate a sobol sequence of size 1000000 in the whole input space [0, 1] d 1 +•••+d N . The batch of enrichment points will be choosen among these points.

3. Keep only the points verifying the constraints.

4. Choose the point optimal for the criterion.

5. Add the point to X N and update the prediction variance v of the seqGPR metamodel, but not the LOO-CV errors.

6. Start again to phase 3, until the size L has been reached.

Example in 2D

In the example in 2D from subsection 3.1.2 in chapter 3, the metamodel seqGPR has been built on the non-nested samples:

X 1 =    x (1) 1 . . . x (n 1 ) 1    , X 2 =    x (n 1 +1) 1 x (n 1 +1) 2 . . . . . . x (n 1 +n 2 ) 1 x (n 1 +n 2 ) 2    ,
with the associated output values y 1 and y 2 . The goal is to add new relevant points to X 2 to improve the metamodel accuracy. An optimization problem is defined which has to be solved by the new point. This optimization problem involves an objective function to maximize defined by [START_REF] Gratiet | Multi-fidelity Gaussian process regression for computer experiments[END_REF] (see subsection 2.1.5 in chapter 2):

τ (x) = v(x)        1 + n 1 i=1 (y i -ŷ-i (x (i) 1 , 0.7x (i) 1 + 0.2)) 2 v-i (x (i) 1 , 0.7x (i) 1 + 0.2) 1 V i (x) =τ 1 (x) + n 1 +n 2 i=n 1 +1 (y i -ŷ-i (x (i) 1 , x (i) 2 )) 2 v-i (x (i) 1 , x (i)
2 )

1 x∈V i =τ 2 (x)       
This enrichment criterion is the product of two terms. The first one is the prediction variance of the seqGPR metamodel, it is a spatial criterion that favours points far from the designs X 1 and X 2 . The second one takes into account the prediction error of the seqGPR metamodel by involving an LOO-CV criterion. τ 1 (resp. τ 2 ) is an LOO-CV criterion applied to the design X 1 (resp. X 2 ). ŷ-i and v-i are the prediction mean and variance if the ith training point is not used in the metamodel fitting, and y iŷ-i (x

(i) 1 , 0.7x (i) 1 + 0.2) (resp. y i -ŷ-i (x (i) 1 , x (i) 2 ))
is the associated prediction error. V i is the Voronoï cell of the ith training point. The Voronoï cells (V i ) n 1 +n 2 i=1 form a partition of the input space, for each i, V i contains the points that are closer to the ith training point than the other training points. The prediction variance is an estimation of the prediction error and the criterion τ acts as a corrected estimation of the prediction error. Indeed, in areas where the prediction error is supposedly underestimated, i.e. in V i such that (y iŷ-i (x

(i) 1 , 0.7x (i) 1 + 0.2)) 2 >> v-i (x (i) 1 , 0.7x (i) 1 + 0.2) (resp. (y i -ŷ-i (x (i) 1 , x (i) 2 )) 2 >> v-i (x (i) 1 , x (i)
2 )), the second factor contributes to increasing the value of τ . Let x = (x 1 , x 2 ) denote the new point. The enriched designs are now denoted by X + 2 ,

∼ X + 1 : ∼ X + 1 =      x (1) 1 . . . x (n 1 +n 2 ) 1 x 1      , X + 2 =      x (n 1 +1) 1 x (n 1 +1) 2 . . . . . . x (n 1 +n 2 ) 1 x (n 1 +n 2 ) 2 x 1 x 2      .
Some regions of the input space are ineligible to contain x in order to maintain the invertibility of ρ 1 (

∼ X + 1 , ∼ X + 1
) and ρ 2 (X + 2 , X + 2 ) (the covariance matrices involved in the EM algorithm, see subsection 4.1.2).

• In order for ρ 1 (

∼ X + 1 , ∼ X + 1
) to be well-conditioned, x 1 cannot already belong to ∼ X 1 . The related constraint for the optimization problem is dist(x 1 , ∼ X 1 ) > u 1 with u 1 > 0 a user-defined scalar and:

dist(x 1 , ∼ X 1 ) = min 1 i n 1 +n 2 x 1 -x (i) 1 .
• In order for ρ 2 (X + 2 , X + 2 ) to be well-conditioned, x = (x 1 , x 2 ) cannot already belong to X 2 . The related constraint for the optimization problem is dist(x, X 2 ) > u 2 with u 2 > 0 a user-defind scalar and:

dist(x, X 2 ) = min n 1 +1 i n 1 +n 2 (x 1 -x (i) 1 ) 2 + (x 2 -x (i) 2 ) 2 .
In fact, the first constraint implies this one, which is not kept.

• In order for ρ 2 (X + 2 , X + 2 ) to be well-conditioned, x must also not belong to the nullity zone of Z 2 , i.e. x 2 must be different from 0.7x 1 + 0.2. The related constraint for the optimization problem is dist(x 2 , 0.7x 1 + 0.2) > v with v > 0 a user-defined scalar and:

dist(x 2 , 0.7x 1 + 0.2) = |x 2 -0.7x 1 -0.2| .
To summarize, the optimization problem contains two constraints:

dist(x 1 , ∼ X 1 ) > u, dist(x 2 , 0.7x 1 + 0.2) > v.
The areas which are not eligible are illustrated in green in the graph below:

The goal may be to add several points at a time. In this case, the enrichment algorithm is described below.

Compute the LOO-CV errors

(y i -ŷ -i (x (i) 1 ,0.7x (i) 1 +0.2)) 2 v-i (x (i) 1 ,0.7x (i) 1 +0.2) (1 i n 1 )
and

(y i -ŷ -i (x (i) 1 ,x (i) 2 )) 2 v-i (x (i) 1 ,x (i) 2 )
(n 1 + 1 i n 1 + n 2 ) of the seqGPR metamodel. These will not be updated during the algorithm.

2. Generate a sobol sequence of size 1000000 in the whole input space [0, 1] 2 . The batch of enrichment points will be choosen among these points.

3. Keep only the points verifying the constraints.

4. Choose the point optimal for the criterion.

5. Add the point to X 2 and update the prediction variance v of the seqGPR metamodel, but not the LOO-CV errors.

6. Start again to phase 3, until the required number of points is reached.

Test cases

The tests are done with the seqGPR metamodel with the Red process and robust kernel (see section 4.2 in chapter 4). On the different test cases, the initial metamodel (I) is compared to the one enriched with 10 points. Three methods are tried to seek the enrichment points:

• C (criterion): the method described in the previous subsection 6.2.1.

• V (var): same method as C but replacing the criterion to optimize by the prediction variance alone v (without the LOO-CV errors).

• R (random): ten points taken randomly among the points of the sobol sequence of size 10000.

The test cases are the same than in section 4.2 in chapter 4. The initial seqGPR model is the one computed during these test cases.

Test case from subsection 4.2.2 in chapter 4 It is the exact same test case with the same step definition. The enrichment method is tested on the cases 10pts/20pts (X 1 of size 10 and X 2 of size 20) and 20pts/40pts (X 1 of size 20 and X 2 of size 40) with non-nested designs. The results are shown on table 6.3. In the case 10ts/20pts, only the R enrichment approach improves the accuracy of the metamodel, the two others disturb it (same median but higher interquartile range than the initial metamodel). However, in the case 20pts/40pts, the enrichments V and C, using the variance or the criterion of subsection 6.2.1 perform better, and especially the method C which is the best in this case. One conclusion is that the method C seems efficient only if the metamodel is sufficiently accurate. The enrichment can be divided in two parts, one with random points added until the metamodel has reached a certain accuracy level, and the following part with points added using the method C. One perspective is then to determine when to switch from one part to the other and when to stop the enrichment.

Test case from subsection 4.2.3 in chapter 4 As a recall, X 1 is of size 15, X 2 of size 45 and X 3 of size 75 and the designs are not nested. The results are shown on table 6.4. This time the median of V and C are equivalent and C has a smaller interquartile range. Therefore, it seems to be the best enrichment strategy, but not by much.

Location of the enrichment points Figure 6.2 (resp. figure 6.3) shows the locations of the enrichment points for the analytic test case in 4D described in subsection 4.2.2 in chapter 4 with initial DoE's X 1 of size 10 and X 2 of size 20 (resp. X 1 of size 20 and X 2 of size 40). Figure 6.4 shows the locations of the enrichment points for the analytic test case in 15D described in subsection 4. the input space. The enrichment points choosen by maximum prediction variance are mostly located on the sides and corners of the input space (and especially in the corners in the 4D analytic test case). Those choosen by the criterion defined in (6.5) are also located on the sides of the input space for most of them but there are more points within the input space.

Conclusion

This chapter tackles the case where multiple subspaces are explored at each step instead of one. An adaptation of the seqGPR metamodel is suggested and a new candidate for Z n (n in 2, N ) is introduced, derived from the P process. The method is compared to a classic kriging metamodel on the same test cases as in section 4.2 in chapter 4, but this time, the seqGPR metamodel is not better than K_tot. This adapted version of the seqGPR metamodel may be too complex, the advantage of Z n (with n small) learning f in small dimension is lost.

This chapter also deals with the issue of enriching the DoE's of the seqGPR metamodel.

A strategy is proposed to add a batch of points verifying some constraints proper to the seqGPR metamodel and optimal for a criterion found in the literature. The method is compared to other enrichment strategies on the test cases of section 4.2 in chapter 4. It is not as convincing as expected. A study of the enrichment points location shows that they are mostly located in the sides of the input space, even if this trend is less pronounced than for the maximum prediction variance enrichment. As an enrichment within the input space is more interesting (see [START_REF] Henner | Sampling strategies for metamodel enrichment and automotive fan optimization[END_REF] in appendix 9.5.1), a new enrichment method can be proposed, either by applying the criteria defined in (6.5) but adding to the constraints a minimal distance to the sides, or by defining a new criteria derived from the one defined in (6.5), taking into account the distance to the sides:

v(x) 1 + n 1 +...+n N i=1 (y i -ŷ-i (x (i) )) 2 v-i (x (i) ) 1 x∈V i 1 + d 1 +...+d N i=1 -x i (x i -1) .
The new factor is maximal when all x components are equal to 0.5 (see figure 6.5). Chapter 7

Conclusions and perspectives

Conclusions

The goal of this thesis is to build a metamodel defined on nested subspaces of increasing dimension, i.e. some input variables are fixed and then released progressively. The work first consists in the definition of a method, called seqGPR (sequential Gaussian process regression). It consists in creating a kriging metamodel with a particular kernel on the reunion of all DoE's. A probabilistic framework is defined in which the kernel (which is unstationary) is the covariance kernel of a Gaussian process (modeling the output at the last step) defined recursively. At a given step, the process modeling the output is equal to the sum of the process modeling the output at the previous step and an independent correction term.

The correction term is a Gaussian process null on a continuous part of its definition space. One issue raised by the probabilistic framework is then to define such a process both verifying the nullity property and with a tractable kernel. Three candidates are proposed : Psi (latent process multiplied a deterministic function null on the concerned subspace), Red (latent process minus its value on the subspace), and P (latent process minus its conditional expectation on the subspace). The P process comes from the literature, and a work is done in the thesis to make it tractable. The Psi process is not retained, as it has the drawback of generating disturbed paths and as it has more parameters to estimate.

Another issue raised by the definition of the seqGPR metamodel is the estimation of its parameters. The choice is set on the maximum likelihood estimation and two simplifications are made : decoupling of the likelihood in the case of nested designs, use of an EM (Expectation-Maximization) algorithm when the designs are non-nested.

The metamodel seqGPR is tested on two analytic test cases and an industrial one. Different types of parametrization are tested and the method is compared to a classic kriging metamodel. Results show that non-nested designs are more performant than nested designs, the method seqGPR is better than the classic kriging metamodel, the best metamodel is the seqGPR with a Red robust kernel.

Methods are detailed to generate nested or non-nested designs in case they are not furnished by the user. These methods enable a good conditioning of the matrices to invert in the parameter estimation and a good space filling property of the designs. 7.2 Perspectives

Multi-conditioning

The first perspective is to adapt the seqGPR metamodel in the case of multiple subspaces at each step. A first suggestion of adaptation is made, involving a generalization of the P process. The tests on analytical test cases show similar accuracy between the adapted seqGPR metamodel and the classic kriging metamodel. Future work will consist in finding alternative ways of defining the seqGPR metamodel in this context.

Enrichment

The second perspective is to enrich the training samples of the seqGPR metamodel to improve its accuracy. A strategy is suggested to enrich the training sample with a batch of points of size defined by the user. This batch of points must verify a certain optimization problem. This enrichment strategy does not give significantly better results than more classical ones on the two analytical test cases it was tested on. Future work will consist in studying alternative enrichment criteria like the one proposed in the conclusion of chapter 6 that involves a factor taking into account the distance to the input space sides. A completely new optimization problem can also be searched, mixing mathematical and physical knowledge.

Categorical variables

The next step is then to adapt the seqGPR metamodel to a test case mixing continuous and categorical variables. Such a test case is already studied in Valeo. As for the fan system (see section 1.1 in chapter 1), the newly studied system is also part of the car engine cooling system (see figure 7.1). Via this system, a cooling fluid circulates in the engine to decrease its temperature. On the left part of figure 7.1, the hot fluid getting out of the engine is represented in red. The fluid itself is then cooled down in the radiator via the fan system which projects air on it. The cold fluid getting out of the radiator is represented in blue.

The middle part of the figure represents the radiator. It is composed of a vertical stacking of, alternately, fluid circulation tubes and air inlets, as shown in the right part.

The studied system is part of an air inlet (see figure 7.2). The air inlet plays the role of a turbulence generator, to favour heat exchange. The air inlet is constituted of a periodic horizontal juxtaposition of an elementary pattern. The middle part of the figure represents one air inlet with a circulation tube. The system studied in this subsection is this elementary pattern, called fin system, whose diagram is represented on the right part of the figure.

α 4 = α 1 +α 3 2 = 3α 1 +α 2 4 , α 5 = α 3 +α 2 2 = α 1 +3α 2 4
(see figure 7.4a). The variables N br, K 1 , α 1 , α 2 are free.

• At step 2, α 3 is released. α 4 and α 5 are still fixed to α 4 = α 1 +α 3 2 and α 5 = α 3 +α 2

2

(see figure 7.4b). The variables N br, K 1 , α 1 , α 2 , α 3 are free.

• At step 3, α 4 is released. α 5 is still fixed to α 5 = α 3 +α 2 2 (see figure 7.4c). The variables N br, K 1 , α 1 , α 2 , α 3 , α 4 are free.

• At step 4, α 5 is released (see figure 7.4d). All variables are free.

Thus one perspective is to adapt the seqGPR metamodel to this test case which has the particularity of involving a categorical variable N br.

Chapter 8

Résumé en Français

Le but de cette thèse est d'approximer par un métamodèle une fonction f (x 1 , • • • , x d ) dont on connait les valeurs en plusieurs plans d'entraînement (X 1 , y 1 ), • • • , (X N , y N ), qui ont été construits au cours d'une étude composée de N étapes. Le plan

X n ⊂ [0, 1] d 1 +•••+dn , généré à l'étape n ∈ 1, N , est
Y 1 (x I 1 ) = m + Z 1 (x I 1 ), Y n (x I 1 , • • • , x I n-1 , x In ) = Y n-1 (x I 1 , • • • , x I n-1 ) + Z n (x I 1 , • • • , x I n-1 , x In ), ∀n ∈ 2, N .
Le processus Z n (n 2) est un processus correctif captant l'information apportée par les variables nouvellement libérées. Il est nul sur la partie de l'espace correspondant à l'étape n -1 afin que le processus Y n coincide avec le processus Y n-1 sur cette zone:

Z n (x I 1 , • • • , x I n-1 , xIn ) = 0.
Le métamodèle seqGPR utilise les formules habituelles de moyenne et variance de prédiction, qui sont les moments du processus conditionnel

[Y N (x) | Y 1 (X 1 ) = y 1 , • • • , Y N (X N ) = y N ]: ŷ(x) = E [Y N (x) | Y 1 (X 1 ) = y 1 , • • • , Y N (X N ) = y N ] , v(x) = V ar (Y N (x) | Y 1 (X 1 ) = y 1 , • • • , Y N (X N ) = y N ) .
Trois questions se posent. Comment construire les processus (Z n ) N n=2 ? Comment construire les plans (X n ) N n=1 ? Comment estimer les paramètres du modèle ?

                     Z P si n (x I 1 ∪•••∪I n-1 , x In ) = Ψ(x In -xIn ) ∼ Z n (x I 1 ∪•••∪I n-1 , x In ), Z Red n (x I 1 ∪•••∪I n-1 , x In ) = ∼ Z n (x I 1 ∪•••∪I n-1 , x In ) - ∼ Z n (x I 1 ∪•••∪I n-1 , xIn ), Z P n (x I 1 ∪•••∪I n-1 , x In ) = ∼ Z n (x I 1 ∪•••∪I n-1 , x In ) -E ∼ Z n (x I 1 ∪•••∪I n-1 , x In )) | ∼ Z n (t I 1 ∪•••∪I n-1 , tIn ), ∀t I 1 ∪•••∪I n-1 ∈ [0, 1] d 1 +•••+d n-1 .
∼ Z n est un processus Gaussien dont le noyau de covariance est calculable directement. Il est appelé processus latent. C'est à partir de lui que sont construits les trois candidats. La fonction Ψ est définie par :

Ψ(t) = 1 -exp - dn i=1 t i 2δ i .
Elle s'annule en 0 et vaut 1 loin de 0. Enfin l'espérance intervenant dans la définition du processus P est la projection orthogonale de

∼ Z n (x I 1 ∪•••∪I n-1 , x In ) sur l'espace Gaussien engendré par les variables ∼ Z n (t I 1 ∪•••∪I n-1 , tIn ) t I 1 ∪•••∪I n-1 ∈[0,1] d 1 +•••+d n-1
. Le processus P a une forme analytique facilement calculable numériquement dans le cas précis où le processus latent ∼ Z n a un noyau de covariance de la forme :

k((x I 1 ∪•••∪I n-1 , x In ), (t I 1 ∪•••∪I n-1 , t In )) = σ 2 r 1 (x I 1 ∪•••∪I n-1 , t I 1 ∪•••∪I n-1 )r 2 (x In -xIn , t In -tIn ).
Dans ce cas, le processus P s'écrit :

Z P n (x I 1 ∪•••∪I n-1 , x In ) = ∼ Z n (x I 1 ∪•••∪I n-1 , x In ) -r 2 (x In -xIn , 0) ∼ Z n (x I 1 ∪•••∪I n-1 , xIn ).
Le processus Psi est finalement abandonné car l'estimation de ses paramètres est plus compliquée que pour les deux autres.

Construction des plans (X n ) N n=1

Deux types de familles de plans sont considérés : les plans imbriqués et les plans non imbriqués.

Les plans imbriqués

Le plan X n (n 2) est dit imbriqué dans X n-1 s'ils sont de la forme :

                       X n-1 =    x (1) I 1 ∪•••∪I n-1 • • • x (n n-1 ) I 1 ∪•••∪I n-1    , X n =    x (i 1 ) I 1 ∪•••∪I n-1 x (i 1 ) In • • • x (in n ) I 1 ∪•••∪I n-1 x (in n ) In    , with {i 1 , • • • , i nn } ⊂ {1, • • • , n n-1 }.
Le block de X n correspondant aux variables x I 1 ∪•••∪I n-1 reprend les lignes de X n-1 . Pour construire le plan X n à partir du plan X n-1 , un algorithme de recuit simulé est utilisé pour intervertir les lignes de la matrice :

                    x (1) I 1 ∪•••∪I n-1 1 2nn . . . 1 2nn . . . . . . . . . x (k) I 1 ∪•••∪I n-1 1 2nn + k-1 nn . . . 1 2nn + k-1 nn . . . . . . . . . x (nn) I 1 ∪•••∪I n-1 1 -1 2nn . . . 1 -1 2nn x (nn+1) I 1 ∪•••∪I n-1 -1 . . . -1 . . . . . . . . . x (n n-1 ) I 1 ∪•••∪I n-1 -1 . . . -1                    
.

Dans la partie gauche, les lignes x (k)

I 1 ∪•••∪I n-1 (k dans 1, n n-1
) doivent être interverties d'un block. Dans la partie droite, les intervertions se font à l'intérieur d'une même colonne. X n est finalement égale à la matrice résultant de ce recuit simulé dont sont enlevées les lignes comportant des -1. Le recuit simulé se fait dans le but de minimiser la fonction objectif suivante :

   x,t∈Xn x =t 1 x -t 2 [0,1] d 1 +•••+dn 50    1 50 =φ 1 +   x∈Xn 1 x In -xIn 2 [0,1] dn 50   1 50 φ 2 ,
où Φ 1 est un critère de maximin généralisé qui fait en sorte que les points soient bien répartis dans [0, 1]

d 1 +•••+dn , et Φ 2 impose que X n soit éloigné de la zone d'annulation de Z n .

Les plans non imbriqués

Dans le cas des plans non imbriqués, X n (n 2) se construit à partir de tous les plans précédents :

X 1 , • • • , X n-1 . Les matrices X 1 , • • • , X n sont notées :                                                                        X 1 =    x (1) I 1 . . . x (n 1 ) I 1    , X 2 =    x (n 1 +1) I 1 x (n 1 +1) I 2 . . . . . . x (n 1 +n 2 ) I 1 x (n 1 +n 2 ) I 2    ,
. . .

X n-1 =    x (n 1 +•••+n n-2+1 ) I 1 • • • x (n 1 +•••+n n-2+1 ) I n-1 . . . . . . . . . x (n 1 +•••+n n-2 +n n-1 ) I 1 • • • x (n 1 +•••+n n-2 +n n-1 ) I n-1    , X n =    x (n 1 +•••+n n-1+1 ) I 1 • • • x (n 1 +•••+n n-1+1 ) I n-1 x (n 1 +•••+n n-1+1 ) In . . . . . . . . . x (n 1 +•••+n n-1 +nn) I 1 • • • x (n 1 +•••+n n-1 +nn) I n-1 x (n 1 +•••+n n-1 +nn) In    .
Le plan X n est construit par block de variables : d'abord les colonnes correspondant à x I 1 (notées X n,I 1 ), auxquelles sont ajoutées celles correspondant à x I 2 (notées X n,I 2 ) pour former X n,I 1 ∪I 2 , • • • , enfin celles correspondant à x In (notées X n,In ), pour former X n tout entier.

• Les colonnes X n,I 1 sont construites en générant un LHS dans [0, 1] d 1 . Cet LHS est ensuite optimisé par recuit simulé pour minimiser le critère de maximin généralisé appliqué à la concaténation des colonnes correspondant à x I 1 des matrices

X 1 à X n :    x (1) I 1 . . . x (n 1 +•••+nn) I 1    . • Les colonnes X n,I k (k ∈ 2, N ) sont construites en générant un plan X n,I 1 ∪•••∪I k (con- tenant les colonnes de X n associées aux variables (x I 1 , • • • , x I k )) imbriqué dans X n,I 1 ∪•••∪I k-1
(voir section précédente), minimisant la fonction objectif suivante :

     x,t∈ ∼ X n k x =t 1 x -t 2 [0,1] d 1 +•••+dn 50      1 50 +   x∈X n,I 1 ∪•••∪I k 1 x I k -xI k 2 [0,1] d k 50   1 50 , avec : ∼ X n k =    x (n 1 +•••+n k-1 +1) I 1 • • • x (n 1 +•••+n k-1 +1) I k . . . . . . . . . x (n 1 +•••+n k-1 +•••+nn) I 1 • • • x (n 1 +•••+n k-1 +•••+nn) I k    .

Estimation des paramètres

La méthode choisie dans cette thèse est la maximisation de la vraisemblance. Ainsi, en notant

(η 1 , • • • , η N ) les paramètres respectifs de (Y 1 , • • • , Y N ), la vraisemblance vaut : L(η 1 , • • • , η N ; y 1 , • • • , y N ) = h Y (y). C'est la densité du vecteur Y = (Y 1 (X 1 ), • • • , Y N (X N )) appliquée aux observations y = (y 1 , • • • , y N ).
Dans le cas où les plans sont imbriqués, les observations de

(Z n (X n )) N n=2 sont connues (et notées (z n ) N n=2
). L'indépendance des Z n conduit au découplage des vraisemblances :

L(η 1 , • • • , η N ; y 1 , • • • , y N ) = L(η 1 ; y 1 ) • N n=2 L(η n ; z n ).
Celles-ci peuvent alors être optimisées séparément ce qui simplifie l'estimation. Dans le cas où les plans ne sont pas imbriqués, la vraisemblance ne peut pas être découplée et un algorithme d'EM (Expectation-Maximisation) est utilisé. L'idée est de se ramener au cas découplé en considérant les ∼ X N n , qui sont bien imbriqués les uns dans les autres, à la place des X n , et les observations de

Y 1 ( ∼ X N 1 ) et des Z n ( ∼ X N n ) N n=2
à la place de y 1 et des (z n ) N n=2 . La différence est que ces observations ne sont plus connues. Les vraisemblances L(η n ; z n ) (respectivement L(η 1 ; y 1 )) sont remplacées par leurs estimations a posteriori :

       Q 1 (η 1 , η * ) = E η * L η 1 ; Y 1 ∼ X N 1 | Y = y , Q n (η 1 , η * ) = E η * L η n ; Z n ∼ X N n | Y = y .
L'algorithme consiste alors a créer une suite η (i) i 0 où, à chaque nouvelle itération i + 1, les paramètres η (i+1) n sont solutions de :

max ηn Q n (η n , η (i)
n ). L'estimation des paramètres, qui est censée être la limite de cette suite, est prise en tronquant la suite à partir d'un certain rang. L'algorithme EM permet de passer d'un seul problème d'optimisation complexe à plusieurs problèmes d'optimisation simples.

Cas test

Le métamodèle seqGPR est testé sur trois cas test : deux analytiques et un provenant d'un code de calcul industriel. Comme il y a beaucoup de paramètres à estimer dans ce métamodèle, une tentative de simplification est faite en imposant que des groupes de paramètres soient de même valeur. Ainsi chaque processus correctif Z n (n 2) est doté d'un vecteur de paramètres de covariance de la forme :

θ n =   α n , • • • , α n I 1 ∪•••∪I n-1 , θ 1 n , • • • , θ dn n In   .
La version de seqGPR avec ce regroupement de paramètres est appelée Robust, tandis que celle avec tous les paramètres dissociés est appelée Full. Sont également comparées les versions avec des processus correctifs de type Red ou de type P. Enfin, le métamodèle seqGPR est comparé à un krigeage classique entrainé sur tous les plans.

Cas test analytique en dimension 4

La fonction à approximer par un métamodèle est une fonction de quatre variables décomposée en la somme d'une fonction de deux variables et d'une fonction de quatre variables f (x 1 , x 2 , x 3 , x 4 ) = g 1 (x 1 , x 2 )+g 2 (x 1 , x 2 , x 3 , x 4 ). Les deux fonctions sont définies de la manière suivante : g 2 (x 1 ,x 2 ,x 3 ,

     g 1 (x 1 , x 2 ) = 4 -2.1(4x 1 -2) 2 + (4x 1 -2) 4 3 (4x 1 -2) 2 + (

Cas test analytique en dimension 15

La nouvelle fonction à approximer est une fonction de 15 variables :

f : [-3, 3] 15 → R x → a 1 x + a 2 sin x + a 3 cos x + x M x,
dont les entrées sont réarrangées par ordre décroissant d'indice de Sobol, et

   Y 1 (x I 1 , x I 2 ) = m + Z 1 (x I 1 , x I 2 ), Y n (x I 1 ∪•••∪In , x I n+1 ) = Y n-1 (x I 1 ∪•••∪In ) + Z n (x I 1 ∪•••∪In , x I n+1 ), ∀n ∈ 2, N -1 , Y N (x I 1 , • • • , x I N ) = Y N -1 (x I 1 , • • • , x I N ) + Z N (x I 1 , • • • , x I N ).
La nouvelle propriété de nullité vérifiée par les (Z n ) N n=2 est :

Z n (x I 1 ∪•••I n-1 , xIn + B n , xI n+1 ) = 0, ∀n ∈ 2, N -1 , ∀x I 1 ∪•••I n-1 ∈ [0, 1] d 1 +•••+d n-1 , Z N (x I 1 ∪•••∪I N -1 , xI N + B N ) = 0, ∀x I 1 ∪•••∪I N -1 ∈ [0, 1] d 1 +•••+d N -1 . avec B n =    b 1 n . . . b Kn n    , pour tout n 2.
Si la méthodologie d'estimation par EM fonctionne encore dans ce nouveau format, il faut reconstruire les (Z n ) N n=2 dont la propriété de nullité est désormais multiple. Le choix est porté sur une généralisation du processus P à plusieurs plans :

                 Z n (x I 1 ∪•••I n-1 , x In , x I n+1 ) = ∼ Z n (x I 1 ∪•••I n-1 , x In , x I n+1 ) -E ∼ Z n (x I 1 ∪•••I n-1 , x In , x I n+1 ) | ∼ Z n (D n ) , ∀n ∈ 2, N -1 , Z N (x I 1 ∪•••I N -1 , x I N ) = ∼ Z N (x I 1 ∪•••I N -1 , x I N ) -E ∼ Z N (x I 1 ∪•••I N -1 , x I N ) | ∼ Z N (D N ) . où D n = {(t I 1 ∪•••I n-1 , tIn + B n , tI n+1 ), t I 1 ∪•••I n-1 ∈ [0, 1] d 1 +•••+d n-1 } est la zone de nullité à l'étape n et D N = {(t I 1 ∪•••I N -1 , tI N + B N ), t I 1 ∪•••I N -1 ∈ [0, 1] d 1 +•••+d N -1 }
           k n ((x I 1 ∪•••∪I n-1 , x In∪I n+1 ), (t I 1 ∪•••∪I n-1 , t In∪I n+1 )) = σ 2 n r 1 n (x I 1 ∪•••∪I n-1 , t I 1 ∪•••∪I n-1 )r 2 n (x In∪I n+1 -xIn∪I n+1 , t In∪I n+1 -tIn∪I n+1 ), ∀n ∈ 2, N -1 , k N ((x I 1 ∪•••∪I N -1 , x I N ), (t I 1 ∪•••∪I N -1 , t I N )) = σ 2 N r 1 N (x I 1 ∪•••∪I N -1 , t I 1 ∪•••∪I N -1 )r 2 N (x I N -xI N , t I N -tI N )
. Dans ce cas les processus P valent : i) . La fonction objectif veille à ce que le point soit choisi dans une zone de forte variance de prédiction v(x), c'est-à-dire loin des points d'entraînement déjà présents, mais également dans une zone où l'erreur de prédiction réelle est potentiellement forte (ici c'est l'erreur de prédiction du point d'entraînement le plus proche qui donne une idée de sa valeur). Les contraintes veillent à ce que les ∼ X N n aient des points suffisamment éloignés les uns des autres, et que le point x soit éloigné de tous les sous-espaces des étapes précédentes, le but ultime étant que les matrices mises en jeu dans l'EM soient inversibles.

               Z n (x I 1 ∪•••∪I n-1 , x In∪I n+1 ) = ∼ Z n (x I 1 ∪•••∪I n-1 , x In∪I n+1 ) -r 2 n (x In∪I n+1 , B n )r 2 n (B n , B n ) ∼ Z n (x I 1 ∪•••∪I n-1 , xIn∪I n+1 + B n ), ∀n ∈ 2, N -1 , Z N (x I 1 ∪•••∪I N -1 , x I N ) = ∼ Z N (x I 1 ∪•••∪I N -1 , x I N ) -r 2 N (x I N , B N )r 2 N (B N , B N ) ∼ Z N (x I 1 ∪•••∪I N -1 , xI N + B N ), avec B n =    b 1 n 0 . . . . . . b Kn n 0    ,
                 max x∈X v(x) 1 + n i=1 (y i -ŷ-i (x (i) )) 2 v-i (x (i) ) 1 x∈V i , sous les contraintes : dist x I 1 ∪•••∪In , ∼ X N n > u n , ∀n ∈ 1, N dist(x In -xIn ) > v n , ∀n ∈ 2, N V i designe la cellule de Voronoï du point d'entraînement x (
Etant donné que l'enrichissement ne se fait pas point par point mais plutôt par paquets de points choisis en même temps, une procédure gloutonne utilisant le problème d'optimisation précédemment défini est mise en place. Le paquet de points, dont la taille est imposée par l'utilisateur, est sélectionné parmi une suite de Sobol qui forme un ensemble de points candidats. La procédure consiste à, de manière séquentielle, retirer les points candidats ne vérifiant pas les contraintes, puis choisir le point candidat optimal pour la fonction objectif, ensuite mettre à jour le critère de prédiction avec ce nouveau point, et enfin recommencer jusqu'à avoir atteint la taille requise. Le paquet ainsi confectionné peut alors enfin être soumis au code de calcul, pour en connaître la valeur de sortie, et ajouté à l'ensemble d'entraînement du métamodèle seqGPR.

La comparaison de cette méthode d'enrichissement avec d'autres méthodes comme le maximum de variance de prédiction ou encore la sélection d'un paquet aléatoire donne des résultats assez différents suivant les cas test. Il semble que globalement cette méthode d'enrichissement ne soit pas très performante tant que le métamodèle n'est pas assez prédictif.

Chapter 9 Appendix

Example in 4D

This section develops the example in 4D, introduced in subsection 3.1.2 in chapter 3, for the different chapters of the thesis.

Illustration of chapter 3

Red process

• If Z 2 is a Red process, it is defined as Z 2 (x 1 , x 2 , x 3 ) = ∼ Z 2 (x 1 , x 2 , x 3 ) - ∼ Z 2 (x 1 , 0.4, 0.5).
Its covariance kernel is equal to σ 2 2 ρ 2 with ρ 2 ((x 1 , x 2 , x 3 ), (t 1 , t 2 , t 3 )) = r 2 ((x 1 , x 2 , x 3 ), (t 1 , t 2 , t 3 )) + r 2 ((x 1 , 0.4, 0.5), (t 1 , 0.4, 0.5)) r 2 ((x 1 , x 2 , x 3 ), (t 1 , 0.4, 0.5)) r 2 ((x 1 , 0.4, 0.5), (t 1 , 2 , t 3 )) .

• If Z 3 is a Red process, it is defined as

Z 3 (x 1 , x 2 , x 3 , x 4 ) = ∼ Z 3 (x 1 , x 2 , x 3 , x 4 ) - ∼ Z 3 x 1 , x 2 , x 3 , x 1 + x 2 + x 3 4 .
Its covariance kernel is equal to σ 2 3 ρ 3 with

ρ 3 ((x 1 , x 2 , x 3 , x 4 ), (t 1 , t 2 , t 3 , t 4 )) = r 3 ((x 1 , x 2 , x 3 , x 4 ), (t 1 , t 2 , t 3 , t 4 )) + r 3 (x 1 , x 2 , x 3 , x 1 +x 2 +x 3 4 ), (t 1 , t 2 , t 3 , t 1 +t 2 ,t 3 4 ) -r 3 (x 1 , x 2 , x 3 , x 4 ), (t 1 , t 2 , t 3 , t 1 +t 2 +t 3 4 ) -r 3 (x 1 , x 2 , x 3 , x 1 +x 2 +x 3 4 ), (t 1 , t 2 , t 3 , t 4 ) Psi process • If Z 2 is a Psi process, it is defined as Z 2 (x 1 , x 2 , x 3 ) = Ψ 2 (x 2 -0.4, x 3 -0.5) ∼ Z 2 (x 1 , x 2 , x 3 ), with for all (t 1 , t 2 ) in [-1, 1] 2 , Ψ 2 (t 1 , t 2 ) = 1 -exp - t 2 1 2δ 2 2,1 - t 2 2 2δ 2 2,2 . Its covariance kernel is equal to σ 2 2 ρ 2 with ρ 2 ((x 1 , x 2 , x 3 ), (t 1 , t 2 , t 3 )) = Ψ 2 (x 2 -0.4, x 3 -0.5) × Ψ 2 (t 2 -0.4, t 3 -0.5) × r 2 ((x 1 , x 2 , x 3 ), (t 1 , t 2 , t 3 )) .
• If Z 3 is a Psi process, it is defined as

Z 3 (x 1 , x 2 , x 3 , x 4 ) = Ψ 3 x 4 - x 1 + x 2 + x 3 4 ∼ Z 3 (x 1 , x 2 , x 3 , x 4 ), with for all t in [0, 1], Ψ 3 (t) = 1 -exp -t 2 2δ 2 3
. Its covariance kernel is equal to σ 2 3 ρ 3 with

ρ 3 ((x 1 , x 2 , x 3 , x 4 ), (t 1 , t 2 , t 3 , t 4 )) = Ψ 3 x 4 -x 1 +x 2 +x 3 4 × Ψ 3 t 4 -t 1 +t 2 +t 3 4 × r 3 ((x 1 , x 2 , x 3 , x 4 ), (t 1 , t 2 , t 3 , t 4 )) .

P process

• If Z 2 is a P process, it is defined as

Z 2 (x 1 , x 2 , x 3 ) = ∼ Z 2 (x 1 , x 2 , x 3 ) -r 2,3 ((x 2 , x 3 ), (0.4, 0.5)) ∼ Z 2 (x 1 , 0.4, 0.5).
where

∼ Z 2 ∼ GP (0, σ 2 2 r 1 (x 1 , t 1 ) r 2,3 ((x 1 , x 2 ), (t 1 , t 2 )
)) with r 1 and r 2,3 stationary correlation kernels. Its covariance kernel is equal to σ 2 2 ρ 2 with

ρ 2 ((x 1 , x 2 , x 3 ), (t 1 , t 2 , t 3 )) = r 1 (x 1 , t 1 ) × [r 2,3 ((x 2 , x 3 ), (t 2 , t 3 )) -r 2,3 ((x 2 , x 3 ), (0.4, 0.5)) r 2,3 ((0.4, 0.5), (t 2 , t 3 ))] .
• If Z 3 is a P process, it is defined as

Z 3 (x 1 , x 2 , x 3 , x 4 ) = ∼ Z 3 (x 1 , x 2 , x 3 , x 4 ) -r 4 x 4 -x 1 +x 2 +x 3 4 , 0 ∼ Z 3 x 1 , x 2 , x 3 , x 1 +x 2 +x 3 4 . Where ∼ Z 3 ∼ GP 0, σ 2 3 r 1,2,3 ((x 1 , x 2 , x 3 ), (t 1 , t 2 , t 3 )) r 4 x 4 -x 1 +x 2 +x 3 4
, t 4 -t 1 +t 2 +t 3 4 with r 1,2,3 and r 4 stationary correlation kernels. Its covariance kernel is equal to σ 2 3 ρ 3 with

ρ 3 ((x 1 , x 2 , x 3 , x 4 ), (t 1 , t 2 , t 3 , t 4 )) = r 1,2,3 ((x 1 , x 2 , x 3 ), (t 1 , t 2 , t 3 )) × r 4 x 4 -x 1 +x 2 +x 3 4 , t 4 -t 1 +t 2 +t 3 4 -r 4 x 4 -x 1 +x 2 +x 3 4 , 0 r 4 0, t 4 -t 1 +t 2 +t 3 4 .

Illustration of chapter 4

Nested designs The designs X 1 , X 2 , and X 3 are nested (X 3 X 2 X 1 ) if they are of the form:

                                           X 1 =    x (1) 1 . . . x (n 1 ) 1    , X 2 =    x (i 1 ) 1 x (i 1 ) 2 x (i 1 ) 3 . . . . . . . . . x (in 2 ) 1 x (in 2 ) 2 x (in 2 ) 3    , with {i 1 , • • • , i n 2 } ⊂ {1, • • • , n 1 }, X 3 =    x (j 1 ) 1 x (j 1 ) 2
x (j 1 ) 3

x (j 1 ) 4

. . . . . . . . . . . .

x (jn 3 ) 1

x (jn 3 ) 2

x (jn 3 ) 3

x

(jn 3 ) 4    , with {j 1 , • • • , j n 3 } ⊂ {i 1 , • • • , i n 2 }.
In order to compute the likelihoods and the predictions, the matrices cov(Y

1 (X 1 ), Y 1 (X 1 )) = σ 2 1 ρ θ 1 (X 1 , X 1 ), cov(Z 2 (X 2 ), Z 2 (X 2 )) = σ 2 2 ρ θ 2 (X 2 , X 2 ), and cov(Z 3 (X 3 ), Z 3 (X 3 )) = σ 2 3 ρ θ 3 (X 3 , X 3
) must be invertible. X 1 must be composed of distinct points sufficiently far from each other to ensure the invertibility of the first matrix. Because of the nullity property of

Z 2 : Z 2 (x 1 , 0.4, 0.5) = 0 for all x 1 in [0, 1] (resp. Z 3 : Z 3 (x 1 , x 2 , x 3 , x 1 +x 2 +x 3 4 ) = 0 for all (x 1 , x 2 , x 3 ) ∈ [0, 1] 3 )
, its covariance kernel is not positive definite, therefore the same constraint on X 2 (resp. X 3 ) is not sufficient. An additional constraint must be added, that the points of X 2 (resp. X 3 ) are sufficiently far from the nullity subspace D 2 = {(x 1 , 0.4, 0.5),

x 1 ∈ [0, 1]} (resp. D 3 = {(x 1 , x 2 , x 3 , x 1 +x 2 +x 3 4 ), (x 1 , x 2 , x 3 ) ∈ [0, 1] 3 }), i.e. for all k in 1, n 2 , (x (i k ) 2 , x (i k )
3 ) must be sufficiently far from (0.4, 0.5) (resp. for all k in 1, n 3 , x (j k ) 4 must be sufficiently far from

x (j k ) 1 +x (j k ) 2 +x (j k ) 3 4
). This second constraint ensures that the covariance matrix has no rows full of zeros.

Non-nested designs The designs X 1 , X 2 and X 3 are non-nested if they are of the form:

                                   X 1 =    x (1) 1 . . . x (n 1 ) 1    , X 2 =    x (n 1 +1) 1 x (n 1 +1) 2 x (n 1 +1) 3 . . . . . . . . . x (n 1 +n 2 ) 1 x (n 1 +n 2 ) 2 x (n 1 +n 2 ) 3    , X 3 =    x (n 1 +n 2 +1) 1 x (n 1 +n 2 +1) 2 x (n 1 +n 2 +1) 3 x (n 1 +n 2 +1) 4 . . . . . . . . . . . . x (n 1 +n 2 +n 3 ) 1 x (n 1 +n 2 +n 3 ) 2 x (n 1 +n 2 +n 3 ) 3 x (n 1 +n 2 +n 3 ) 4    ,
with the following constraints:

                                                x (1) 1
. . .

x (n 1 ) 1    ∩    x (n 1 +1) 1 . . . x (n 1 +n 2 ) 1    = ∅,    x (1) 1
. . .

x (n 1 +n 2 ) 1    ∩    x (n 1 +n 2 +1) 1 . . . x (n 1 +n 2 +n 3 ) 1    = ∅,    x (n 1 +1) 1 x (n 1 +1) 2 x (n 1 +1) 3 . . . . . . . . . x (n 1 +n 2 ) 1 x (n 1 +n 2 ) 2 x (n 1 +n 2 ) 3    ∩    x (n 1 +n 2 +1) 1 x (n 1 +n 2 +1) 2 x (n 1 +n 2 +1) 3 . . . . . . . . . x (n 1 +n 2 +n 3 ) 1 x (n 1 +n 2 +n 3 ) 2 x (n 1 +n 2 +n 3 ) 3    = ∅.
An illustration of the different

∼ X n is shown in figure below. To compute Q 1 , Q 2 and Q 3 , the designs ∼ X 1 , ∼ X 2 ,
D 2 = {(x 1 , 0.4, 0.5), x 1 ∈ [0, 1]} (resp. D 3 = {(x 1 , x 2 , x 3 , x 1 +x 2 +x 3 4 ), (x 1 , x 2 , x 3 ) ∈ [0, 1] 3 }). For all k in 1, n 2 + n 3 , (x (n 1 +k) 2 , x (n 1 +k) 3
) must be far from (0.4, 0.5), and for all k in 1, n 3 , x

Illustration of chapter 5

Nested designs

• The Φ 2 criterion applied to X 2 is equal to:

Φ 2 (X 2 ) =   n 2 k=1 n 2 l=1 l =k 1 x (i k ) -x (i l ) 2 [0,1] 3 50   1 50 + n 2 k=1 1 (x (i k ) 2 -0.4) 2 +(x (i k ) 3 -0.5) 2 50 1 50
.

• The Φ 3 criterion applied to X 3 is equal to:

Φ 3 (X 3 ) =   n 3 k=1 n 3 l=1 l =k 1 x (j k ) -x (j l ) 2 [0,1] 4 50   1 50 +   n 3 k=1 1 (x (j k ) 4 - x (j k ) 1 +x (j k ) 2 +x (j k ) 3 4 ) 2 50   1 50
.

The building algorithm is illustrated below.

• At step 1, a design X 1 =     0.2 0.4 0.6 0.8   
 has been generated. The values of x 2 and x 3 are respectively fixed to 0.4 and 0.5. The values of x 4 are fixed to x 1 +x 2 +x 3 4 .

• At step 2, the goal is to generate a design X 2 of size 3 nested in X 1 . The following matrix is defined

          0.2 1 6 1 6 0.4 3 6 3 6 0.6 5 6 5 6 0.8 -1 -1          
, from which the initialization of X 2 is deduced by keeping the first three rows. The initialization of the design X 2 is illustrated in figure below.

Φ 2 (X 2 ) = 11.98.

An example of candidate for X 2 resulting from permutations of the rows of the initial candidate is illustrated in figure below. It is obtained by performing the permutation 1 4 2 3 in the first column, the permutation 2 1 3 in the second column, and the permutation 2 3 1 in the third column. The optimal candidate for the criteria Φ 2 is shown in figure below. It is found using algorithm 1 in appendix 9.4.2.

Φ 2 (X 2 ) = 4.20.

• At step 3, the goal is to generate a design X 3 of size 3 nested in X 2 . The initialization of the design X 3 is illustrated below:

Φ 3 (X 3 ) = 61.20.
The possible candidates for X 3 result from permutations of rows in the initial candidate. Two kinds of permutations of rows are possible, by block (the blocks are represented by the orange rectangles with mixed dashes) in the first three columns, by component in the 4th column. An example of candidate is shown in figure below. It is built by applying the permutation 2 1 3 for the blocks of the first three columns, and the permutation 1 3 2 in the last column.

Φ 3 (X 3 ) = 9.78.

The optimal candidate for the criterion Φ 3 is shown below. It is found using algorithm 1 in appendix 9.4.2.

Φ 3 (X 3 ) = 4.35.

Non-nested designs

• X 1 is of the form X 1 =    x (1) 1
. . .

x (n 1 ) 1    . • X 2 is of the form X 2 =    x (n 1 +1) 1 x (n 1 +1) 2 x (n 1 +1) 3 . . . . . . . . . x (n 1 +n 2 ) 1 x (n 1 +n 2 ) 2 x (n 1 +n 2 ) 3    .
It must ensure that the design:

∼ X 2 1 =           x (1) 1
. . .

x (n 1 ) 1 x (n 1 +1) 1 . . . x (n 1 +n 2 ) 1          
, is space-filling with points distant from each other in [0, 1] (it is the part of ∼ X 1 concerned at the time, here X 3 is not taken into account). X 2 itself (equal to

∼ X 2
2 ) must be spacefilling with remote points in [0, 1] 3 and verifying that (x

(n 1 +i) 2 , x (n 1 +i) 3 ) (i ∈ 1, n 2 ) is far from (0.4, 0.5). X 2 is built in two phases: -The first column X 2,I 1 =    x (n 1 +1) 1 . . . x (n 1 +n 2 ) 1    (I 1 = {1} is the index set of the variables released at step 1) is built first to ensure the properties of ∼ X 2 1 . X 2,I 1 must minimize: Φ 2,1 (X 2,I 1 ) =    n 1 +n 2 i=1 n 1 +n 2 j=1 j =i 1 (x (i) 1 -x (j) 1 ) 2 50    1 50
which is the generalized maximin criterion applied to

∼ X 2 1 .
-The columns two and three

X 2,I 2 =    x (n 1 +1) 2 x (n 1 +1) 3 . . . . . . x (n 1 +n 2 ) 2 x (n 1 +n 2 ) 3    (I 2 = {2, 3} is the index set of the variables released at step 2) are built together to ensure the properties of ∼ X 2 2 . The goal is to create the design X 2 such that it is nested in X 2,I 1 . X 2,I 2 must minimize: Φ 2,2 (X 2,I 2 ) = n 1 +n 2 i,j=n 1 +1 j =i 1 (x (i) 1 -x (j) 1 ) 2 +(x (i) 2 -x (j) 2 ) 2 +(x (i) 3 -x (j) 3 ) 2 50 1 50 + n 1 +n 2 i=n 1 +1 1 (x (i) 2 -0.4) 2 +(x (i) 3 -0.5) 2 50 1 50
.

The first part is the generalized maximin applied to ∼ X 2 2 , the second part ensures that X 2 is far from the subspace where Z 2 is null.

• X 3 is of the form X 3 =    x (n 1 +n 2 +1) 1 x (n 1 +n 2 +1) 2 x (n 1 +n 2 +1) 3 x (n 1 +n 2 +1) 4 . . . . . . . . . . . . x (n 1 +n 2 +n 3 ) 1 x (n 1 +n 2 +n 3 ) 2 x (n 1 +n 2 +n 3 ) 3 x (n 1 +n 2 +n 3 ) 4    . It must ensure that ∼ X 3 1 = ∼ X 1 is space-filling with points remote from each other in [0, 1]. It must also ensure that ∼ X 3 2 = ∼
X 2 is space-filling with remote points in [0, 1] 3 and verifying that (x

(n 1 +i) 2 , x
(n 1 +i) 3

) (i ∈ 1, n 2 + n 3 ) is far from (0.4, 0.5). X 3 itself (equal to ∼ X 3
3 ) must be space-filling with remote points in [0, 1] 4 and verifying that x

(n 1 +n 2 +i) 4 (i ∈ 1, n 3 ) is far from x (n 1 +n 2 +i) 1 +x (n 1 +n 2 +i) 2 +x (n 1 +n 2 +i) 3

4

. X 3 is built in three phases:

-The first column X 3,I 1 =    x (n 1 +n 2 +1) 1 . . . x (n 1 +n 2 +n 3 ) 1    is built to ensure the properties of ∼ X 1 . X 3,I 1 must minimize: Φ 3,1 (X 3,I 1 ) =    n 1 +n 2 +n 3 i=1 n 1 +n 2 +n 3 j=1 j =i 1 (x (i) 1 -x (j) 1 ) 2 50    1 50
which is the generalized maximin criterion applied to ∼ X 1 .

-The columns two and three X 3,

I 2 =    x (n 1 +n 2 +1) 2 x (n 1 +n 2 +1) 3 . . . . . . x (n 1 +n 2 +n 3 ) 2 x (n 1 +n 2 +n 3 ) 3    are built together to ensure the properties of ∼ X 2 .
The goal is to create the design X 3,I 1 ∪I 2 grouping the first three columns of X 3 such that it is nested in X 3,I 1 . X 3,I 2 must minimize:

Φ 3,2 (X 3,I 2 ) = n 1 +n 2 +n 3 i,j=n 1 +1 j =i 1 (x (i) 1 -x (j) 1 ) 2 +(x (i) 2 -x (j) 2 ) 2 +(x (i) 3 -x (j) 3 ) 2 50 1 50 + n 1 +n 2 +n 3 i=n 1 +1 1 (x (i) 2 -0.4) 2 +(x (i) 3 -0.5) 2 50 1 50
.

The first part is the generalized maximin applied to ∼ X 2 , the second part ensures that ∼ X 2 is far from the subspace where Z 2 is null.

-The last column X 3,I 3 (I 3 = {4} is the index set of the variables released at step 3) is finally built to ensure the properties of ∼ X 3 = X 3 . The goal is to create the design X 3 such that it is nested in X 3,I 1 ∪I 2 . X 3,I 3 must minimize:

Φ 3,3 (X 3,I 3 ) = n 1 +n 2 +n 3 i,j=n 1 +n 2 +1 j =i 1 (x (i) 1 -x (j) 1 ) 2 +(x (i) 2 -x (j) 2 ) 2 +(x (i) 3 -x (j) 3 ) 2 +(x (i) 4 -x (j) 4 ) 50 1 50 +   n 1 +n 2 +n 3 i=n 1 +n 2 +1 1 (x (i) 4 - x (i) 1 +x (i) 2 +x (i) 3 4
)

2 50   1 50
.

The first part is the generalized maximin applied to X 3 , the second part ensures that X 3 is far from the subspace where Z 3 is null.

To illustrate the building algorithm, it is assumed that 3 non-nested designs X 1 , X 2 , X 3 respectively of size 2, 3, and 4, must be created.

• At step 1, X 1 is taken equal to X 1 = 0 1 .

• At step 2, the goal is to create X 2 of size 3.

-The first column X 2,I 1 is adjusted by hand to have a good value of Φ

2,1 . X 2,I 1 is taken equal to X 2,I 1 =       2 8 4 8 6 8       . The resulting ∼ X 2 1 =               0 8 8 8 2 8 4 8 6 8               (the part of ∼ X 1 concerned at step 2) is formed of equispaced points in [0, 1].
-X 2 is created as a design nested in X 2,I 1 . It is initialized to

X 2 =       2
      , Φ 2,2 (X 2 ) = 11.90.
The optimal design for the criterion Φ 2,2 is obtained for the permutation 3 1 2 in the second column and the permutation 2 3 1 in the third column

X 2 =       2 8 5 6 3 6 4 8 1 6 5 6 6 8 3 6 1 6 
     , Φ 2,2 (X 2 ) = 4.33.
It is found using algorithm 1 in appendix 9.4.2.

• At step 3, the goal is to create X 3 of size 4.

-The first column X 3,I 1 is adjusted by hand to have a good value of Φ

3,1 . X 3,I 1 is taken equal to X 3,I 1 =           1 8 3 8 5 8 7 8           . The resulting ∼ X 1 =                               0 8 8 8 2 8 4 8 6 8 1 8 3 8 5 8 7 8                               is formed of equispaced points in [0, 1].
-The submatrix X 3,I 1 ∪I 2 composed of the first three columns of X 3 is created as a design nested in X 3,I 1 . It is initialized to:

X 3,I 1 ∪I 2 =          
          , Φ 3,2 (X 3,2 ) = 10.21.
The optimal design for the criterion Φ 3,2 is obtained for the permutation 2 1 4 in the second column and the permutation 1 4 2 3 in the third column

X 3,I 1 ∪I 2 =           1 8 3
          , Φ 3,2 (X 3,2 ) = 5.58.
It is found using algorithm 1 in appendix 9.4.2. The resulting

∼ X 2 =                       2
                     
has the good properties: space-filling with separate points and far from the nullity zone (x 2 , x 3 ) = (0.4, 0.5).

-X 3 is finally created as a design nested in X 3,I 1 ∪I 2 . It is initialized to

X 3 =          
          , Φ 3,3 (X 2 ) = 33.29.
The optimal design for the criterion Φ 3,3 is obtained for the permutation 2 3 4

in the last column:

X 3 =          
          , Φ 3,3 (X 2 ) = 6.08.
It is found using algorithm 1 in appendix 9.4.2.

Illustration of chapter 6

Multi-conditioning The example in 4D is modified to take into account multiple subspaces. The output considered is still a function of 4 inputs f (x 1 , x 2 , x 3 , x 4 ). The study is composed of 3 steps with I 1 = {1}, I 2 = {2, 3} and I 3 = {4}.

• At step 1, x 1 is released and (x 2 , x 3 , x 4 ) are fixed. Three different restrictions are considered: f (x 1 , 0.4, 0.5, x 1 +0.4+0.5

4

), f (x 1 , 0.1, 0.7, x 1 +0.1+0.7 4), and f (x 1 , 0.8, 0.3, x 1 +0.8+0.3

4

). On the three subspaces considered, the values of (x 2 , x 3 ) are modified by an additive constant:

       xI 2 = (0.4, 0.5), B 2 =   0 0 -0.3 0.2 0.4 -0.2   .
and x 4 is equal to the same function of (x 1 , x 2 , x 3 ): x4 = x 1 +x 2 +x 3

4

. The restrictions can be summarized in the notation f (x I 1 , xI 2 + B 2 , xI 3 ). The three restrictions are modeled by Y 1 (x 1 , x 2 , x 3 ) = m + Z 1 (x 1 , x 2 , x 3 ), such that f (x 1 , 0.4, 0.5, x 1 +0.4+0.5 4) is the realization of Y 1 (x 1 , 0.4, 0.5), f (x 1 , 0.1, 0.7, x 1 +0.1+0.7 4) is the realization of Y 2 (x 1 , 0.1, 0.7), and f (x 1 , 0.8, 0.3, x 1 +0.8+0.3 4) is the realization of Y 3 (x 1 , 0.8, 0.3). This can be summarized by saying that f (x

I 1 , xI 2 + B 2 , xI 3 ) is the realization of Y 1 (x I 1 , xI 2 + B 2 ). Z 1 is a cen- tered Gaussian process of covariance kernel σ 2 1 ρ 1 ((x 1 , x 2 , x 3 ), (t 1 , t 2 , t 3 )). Three DoE's are generated X 1 1 , X 2 1 , X 3 1 ⊂ [0, 1] 3 , of the form: X 1 1 =     x (1,1) 1 0.4 0.5 . . . . . . . . . x (1,n 1 1 ) 1 0.4 0.5     , X 2 1 =     x (1,n 1 1 +1) 1 0.1 0.7 . . . . . . . . . x (1,n 1 1 +n 2 1 ) 1 0.1 0.7     , X 3 1 =     x (1,n 1 1 +n 2 1 +1) 1 0.8 0.3 . . . . . . . . . x (1,n 1 1 +n 2 1 +n 3 1 ) 1 0.8 0.3     ,
with y 1 1 , y 2 1 , y 3 1 the corresponding output values. • At step 2, (x 2 , x 3 ) are released and x 4 is still fixed. Two different restrictions are considered: [START_REF] Qian | Nested latin hypercube designs[END_REF] ). The first subspace has the same x 4 value than at step 1, the value of x 4 on the second subspace is modified by an additive constant:

f (x 1 , x 2 , x 3 , x 1 +x 2 +x 3 4 ) and f (x 1 , x 2 , x 3 , x 1 +x 2 +x 3 4 + 1
   xI 3 = x 1 +x 2 +x 3 4 , B 3 = 0 1 4
.

The restrictions can be summarized in the notation f (x

I 1 , x I 2 , xI 3 + B 3 ). The two re- strictions are modeled by Y 2 (x 1 , x 2 , x 3 , x 4 ) = Y 1 (x 1 , x 2 , x 3 )+Z 2 (x 1 , x 2 , x 3 , x 4 ), such that f (x 1 , x 2 , x 3 , x 1 +x 2 +x 3 4 ) is the realization of Y 2 (x 1 , x 2 , x 3 , x 1 +x 2 +x 3 4
), and [START_REF] Qian | Nested latin hypercube designs[END_REF] ). This can be summarized by saying that f (x

f (x 1 , x 2 , x 3 , x 1 +x 2 +x 3 4 + 1 4 ) is the realization of Y 2 (x 1 , x 2 , x 3 , x 1 +x 2 +x 3 4 + 1
I 1 , x I 2 , xI 3 + B 3 ) is the realization of Y 2 (x I 1 , x I 2 , xI 3 + B 3 ). Z 2 is a centered Gaussian process of covariance kernel σ 2 2 ρ 2 ((x 1 , x 2 , x 3 , x 4 ), (t 1 , t 2 , t 3 , t 4 )) such that:    Z 2 (x 1 , 0.4, 0.5, x 1 +0.4+0.5 4 ) = 0, ∀x 1 ∈ [0, 1], Z 2 (x 1 , 0.1, 0.7, x 1 +0.1+0.7 4 ) = 0, ∀x 1 ∈ [0, 1], Z 2 (x 1 , , 0.8, 0.3, x 1 +0.8+0.3 4 ) = 0, ∀x 1 ∈ [0, 1].
This can be summarized in:

Z 2 (x I 1 , xI 2 + B 2 , xI 3 ) = 0, for all x 1 in [0, 1]. Two DoE's are generated X 1 2 , X 2 2 ⊂ [0, 1] 4 , of the form:                        X 1 2 =     x (2,1) 1 x (2,1) 2 x (2,1) 3 x (2,1) 1 +x (2,1) 2 +x (2,1) 3 4 
   , . . . . . . . . . . . . x (2,n 1 2 ) 1 x (2,n 1 2 ) 2 x (2,n 1 2 ) 3 x (2,n 1 2 ) 1 +x (2,n 1 2 ) 2 +x (2,n 1 2 ) 3 4 X 2 2 =     x (2,n 1 2 +1) 1 x (2,n 1 2 +1) 2 x (2,n 1 2 +1) 3 x (2,n 1 2 +1) 1 +x (2,n 1 2 +1) 2 +x (2,n 1 2 +1) 3 4 + 1 4     , . . . . . . . . . . . . x (2,n 1 2 +n 2 2 ) 1 x (2,n 1 2 +n 2 2 ) 2 x (2,n 1 2 +n 2 2 ) 3 x (2,n 1 2 +n 2 2 ) 1 +x (2,n 1 2 +n 2 2 ) 2 +x (2,n 1 2 +n 2 2 ) 3 4 + 1 4
with y 1 2 , y 2 2 the corresponding output values.

• At step 3, x 4 is released. f is modeled as the realization of Y 3 which is defined by

Y 3 (x 1 , x 2 , x 3 , x 4 ) = Y 2 (x 1 , x 2 , x 3 , x 4 ) + Z 3 (x 1 , x 2 , x 3 , x 4 ). Z 3 is a centered Gaussian pro- cess of covariance kernel σ 2 3 ρ 3 ((x 1 , x 2 , x 3 , x 4 ), (t 1 , t 2 , t 3 , t 4
)) such that:

Z 3 (x 1 , x 2 , x 3 , x 1 +x 2 +x 3 4 ) = 0, ∀(x 1 , x 2 , x 3 ) ∈ [0, 1] 3 , Z 3 (x 1 , x 2 , x 3 , x 1 +x 2 +x 3 4 + 1 4 ) = 0, ∀(x 1 , x 2 , x 3 ) ∈ [0, 1] 3 .
This can be summarized in:

Z 3 (x I 1 , x I 2 , xI 3 + B 3 ) = 0, for all (x I 1 , x I 2 ) in [0, 1] 3 . One DoE X 3 ⊂ [0, 1]
4 is generated with the corresponding output value y 3 :

X 3 =    x (3,1) 1 x (3,1) 2 x (3,1) 3 x (3,1) 4 . . . . . . . . . . . . x (3,n 3 ) 1 x (3,n 3 ) 2 x (3,n 3 ) 3 x (3,n 3 ) 4    . The new definition of ∼ X 1 , ∼ X 2 ,
and

∼ X 3 is shown on figure below.
The goal is now to find candidates for the correction processes.

• At step 2, J = {1}, I = {2, 3, 4}. Z 2 (x 1 , x 2 , x 3 , x 4 ) must be null on:

D 2 = {(s 1 , g 2 (s 1 ) + (s 2 , s 3 , s 4 )), s = (s 1 , s 2 , s 3 , s 4 ) ∈ [0, 1] × B 2 =S 2 }, with        g 2 (s 1 ) = (0.4, 0.5, s 1 +0.4+0.5 4 ), ∀s 1 ∈ [0, 1], B 2 =   0 0 0 -0.3 0.2 -0.3+0.2 4 0.4 -0.2 0.4-0.2 4   .
Z 2 is defined as:

Z 2 (x 1 , x 2 , x 3 , x 4 ) = ∼ Z 2 (x 1 , x 2 , x 3 , x 4 ) -E ∼ Z 2 (x 1 , x 2 , x 3 , x 4 ) | ∼ Z 2 (D 2 ) , with ∼ Z 2 a centered Gaussian process of covariance kernel equal to σ 2 2 r 2,1 (x 1 , t 1 )r 2,2 ((x 2 , x 3 , x 4 ) -g 2 (x 1
), (t 2 , t 3 , t 4 )g 2 (t 1 )) and r 2,1 , r 2,2 stationary correlation kernels. It is equal to:

Z 2 (x 1 , x 2 , x 3 , x 4 ) = ∼ Z 2 (x 1 , x 2 , x 3 , x 4 ) -r 2,2 ((x 2 , x 3 , x 4 ) -g(x 1 ), B 2 ) r 2,2 (B 2 , B 2 ) -1 ∼ Z 2 (x 1 , g(x 1 ) + B 2 ).
• At step 3, J = {1, 2, 3}, I = {4}. Z 3 must be null on:

D 3 = {(s 1 , s 2 , s 3 , g 3 (s 1 , s 2 , s 3 ) + s 4 ), (s 1 , s 2 , s 3 , s 4 ) ∈ [0, 1] 3 × B 3 =S 3 }, with    g 3 (x 1 , x 2 , x 3 ) = x 1 +x 2 +x 3 4 , ∀(x 1 , x 2 , x 3 ) ∈ [0, 1] 3 B 3 = 0 1 4
. Z 3 is defined as:

Z 3 (x 1 , x 2 , x 3 , x 4 ) = ∼ Z 3 (x 1 , x 2 , x 3 , x 4 ) -E ∼ Z 3 (x 1 , x 2 , x 3 , x 4 ) | ∼ Z 3 (D 3 ) , with ∼ Z 3 a centered Gaussian process of covariance kernel equal to σ 2 3 r 3,1 ((x 1 , x 2 , x 3 ), (t 1 , t 2 , t 3 ))r 3,2 (x 4 -g 3 (x 1 , x 2 , x 3 ), t 4 -g 3 (t 1 , t 2 , t 3
)) and r 3,1 , r 3,2 stationary correlation kernels.It is equal to:

Z 3 (x 1 , x 2 , x 3 , x 4 ) = ∼ Z 3 (x 1 , x 2 , x 3 , x 4 ) -r 3,2 (x 4 -g 3 (x 1 , x 2 , x 3 ), B 3 ) r 3,2 (B 3 , B 3 ) -1 ∼ Z 3 (x 1 , x 2 , x 3 , g 3 (x 1 , x 2 , x 3 ) + B 3 )).
Enrichment In the example in 4D from subsection 3.1.1 in chapter 3, the metamodel seqGPR has been built on the non-nested samples:

                                   X 1 =    x (1) 1
. . .

x (n 1 ) 1    , X 2 =    x (n 1 +1) 1 x (n 1 +1) 2 x (n 1 +1) 3 . . . . . . . . . x (n 1 +n 2 ) 1 x (n 1 +n 2 ) 2 x (n 1 +n 2 ) 3    , X 3 =    x (n 1 +n 2 +1) 1 x (n 1 +n 2 +1) 2 x (n 1 +n 2 +1) 3 x (n 1 +n 2 +1) 4 . . . . . . . . . . . . x (n 1 +n 2 +n 3 ) 1 x (n 1 +n 2 +n 3 ) 2 x (n 1 +n 2 +n 3 ) 3 x (n 1 +n 2 +n 3 ) 4    .
with the associated output values y 1 , y 2 , and y 3 . The designs ∼ X 1 and ∼ X 2 are defined as:

                   ∼ X 1 =    x (1) 1 . . . x (n 1 +n 2 +n 3 ) 1    , ∼ X 2 =    x (n 1 +1) 1 x (n 1 +1) 2 x (n 1 +1) 3 . . . . . . . . . x (n 1 +n 2 +n 3 ) 1 x (n 1 +n 2 +n 3 ) 2 x (n 1 +n 2 +n 3 ) 3    .
The enrichment point x = (x 1 , x 2 , x 3 , x 4 ) must maximize the following objective function which is an adjusted prediction variance taking into account the LOO-CV prediction error:

τ (x) = v(x) [1 + τ 1 (x) + τ 2 (x) + τ 3 (x)] 1 V i (x).
τ 1 , τ 2 , τ 3 are LOO-CV criteria respectively applied to X 1 , X 2 , and X 3 .

The enrichment point must verify some constraints:

     dist(x I 1 , ∼ X 1 ) > u, dist(x I 2 , xI 2 ) > v 2 , dist(x I 3 , xI 3 ) > v 3 . • The constraint dist(x I 1 , ∼ X 1 ) > u ensures that ∼ X 1 ,
∼ X 2 and X 3 enriched with x are still composed of points sufficiently distant from each other. This constraint can be rewritten as: min

1 i n 1 +n 2 +n 3 x 1 -x (i) 1
> u.

• The constraint dist(x I 2 , xI 2 ) > v 2 ensures that ∼ X 2 enriched with x is still far from the nullity subspace D 2 = {(x 1 , 0.4, 0.5), x 1 ∈ [0, 1]}. This constraint can be rewritten as:

(x 2 -0.4) 2 + (x 3 -0.5) 2 > v 2 .
• The constraint dist(x I 3 , xI 3 ) > v 3 ensures that X 3 enriched with x is still far from the nullity subspace

D 3 = (x 1 , x 2 , x 3 , x 1 +x 2 +x 3 4 ), (x 1 , x 2 , x 3 ) ∈ [0, 1] 3
. This constraint can be rewritten as:

x 4 - x 1 + x 2 + x 3 4 > v 3 .
The optimization problem is then equal to:

min x∈[0,1] 4 τ (x), min 1 i n 1 +n 2 +n 3 x 1 -x (i) 1 > u, (x 2 -0.4) 2 + (x 3 -0.5) 2 > v 2 , x 4 -x 1 +x 2 +x 3 4 > v 3 .

Proofs state-of-the-art

This section gives the proofs of the propositions of chapter 2 which draws up a state of the art of the tools linked to the thesis. As a recall, all equalities between processes are in L 2 (Ω) (see definition 1):

X = Y ⇔ E (X -Y ) 2 = 0.

Proof of proposition 1

This subsection deals with the proof of proposition 1, which gives an explicit formula of the conditional expectation in the Gaussian case for any covariance matrix Σ 22 . As a reminder,

V = (V 1 , V 2
) is a Gaussian vector of mean vector and covariance matrix given by:

       E [V ] = m 1 m 2 , Cov (V , V ) = Σ 11 Σ 12 Σ 21 Σ 22 ,
where

m 1 = E [V 1 ], m 2 = E [V 2 ], Σ 11 = Cov (V 1 , V 1 ), Σ 12 = Cov (V 1 , V 2 ), Σ 21 = Cov (V 2 , V 1 ), and
Σ 22 = Cov (V 2 , V 2 ). Proof Let V 1 = V (1) 1 , • • • , V (m) 1
and

V 2 = V (1) 2 , • • • , V (n) 2
denote the components of

V 1 and V 2 . E [V 1 | V 2 ] is easier to compute component by component, i.e. E V (i) 1 | V 2 .
• First case:

m 1 = m 2 = 0. The goal is to find α = (α 1 , • • • , α n ) solution of min α E V (i) 1 -α V 2 2
Then the conditional expectation is equal to:

E V (i) 1 | V 2 = α V 2 .
The expectation to minimize can be rewritten as

E V (i) 1 -α V 2 2 = V ar V (i) 1 + α Σ 22 α -2α Cov V 2 , V (i) 1 . (9.1)
As Σ 22 is a covariance matrix, it is positive semidefinite and can be decomposed as:

Σ 22 = P SP (9.2) with P × P = I and S =             
            
where (λ 1 , • • • , λ q ) are the positive eigen values of Σ 22 . Then, injecting the spectral decomposition of Σ 22 (equation

(9.
2)) in the expectation (equation (9.1)) and using P P = I, it becomes:

E α V 2 -V (i) 1 2 = V ar V (i) 1 + α P SP α -2α P × P Cov V 2 , V (i) 1 = V ar V (i) 1 + x Sx -x c, with x = P α and c = 2P Cov V 2 , V (i) 1 . Then, denoting x = (x 1 , • • • , x n ) and c = (c 1 , • • • , c n ) , the goal is to find x minimizing q j=1 λ j x 2 j - n j=1 c j x j
-If q = n (i.e. Σ 22 is positive definite), then x must minimize n j=1 λ j x 2 jc j x j . So each x j must minimize a quadratic polynomial λ j x 2 jc j x j . The solution is

x j = c j 2λ j , or x =    c 1 2λ 1 . . . cn 2λn    = 1 2 S -1 c = S -1 P Cov V 2 , V (i) 1
.

In this case α = P x

= P S -1 P Cov V 2 , V (i) 1 = Σ -1 22 Cov V 2 , V (i) 1
.

-If q < n, then x must minimize q j=1 (λ j x 2 jc j x j ) -n j=q+1 c j x j . In fact, for all j q + 1, c j = 0. Indeed, if there exists j q + 1 such that c j = 0, then by

taking x =            1 0 . . . . . . j -1 0 j c j + V ar(V (i) 1 ) c j j + 1 0 . . . . . . n 0           
(or the corresponding α = P x), the initial expectation to minimize becomes

E α V 2 -V (i) 1 2 = V ar V (i) 1 + x Sx -x c = V ar V (i) 1 + 0 -c 2 j -V ar V (i) 1 = -c 2 j < 0
which is impossible (the expectation of a square in non-negative). This time, x must minimize q j=1 λ j x 2 jc j x j . There is apparently an infinite number of solu-

tions x ∈                              c 1 2λ 1 . . . cq 2λq
x q+1 . . .

x n           , (x q+1 , • • • , x n ) ∈ R n-q                    , or α ∈                    P           c 1 2λ 1 . . . cq 2λq
x q+1 . . .

x n           , (x q+1 , • • • , x n ) ∈ R n-q                   
, but the corresponding random variables are all equal in L 2 (Ω). Indeed:

E c 1 2λ 1 . . . cq 2λq x q+1 . . . x n P V 2 -c 1 2λ 1 . . . cp 2λq t q+1 . . . t n P V 2 2 = E 0 . . . 0 x q+1 . . . x n P V 2 -0 . . . 0 t q+1 . . . t n P V 2 2 = E 0 . . . 0 x q+1 -t q+1 . . . x n -t n P V 2 2 = 0 . . . 0 x q+1 -t q+1 . . . x n -t n P Σ 22 P =S          0 . . . 0 x q+1 -t q+1 . . . x n -t n          = 0
In particular, the simplest solution is:

α = P           c 1 2λ 1 . . . cq 2λq 0 . . . 0           = 1 2 P             1 λ 1 0 • • • 0 0 • • • 0 0 . . . . . . . . . . . . . . . . . . . . . . . . 0 . . . • • • . . . 0 • • • 1 λq 0 • • • 0 0 • • • 0 0 • • • 0 . . . . . . . . . . . . . . . 0 • • • 0 0 • • • 0             c = P S + P Cov V 2 , V (i) 1 = Σ + 22 Cov V 2 , V (i) 1
The conditional expectation is equal to:

E [V 1 | V 2 ] =      E V (1) 1 | V 2 . . . E V (m) 1 | V 2      =      Cov V (1) 1 , V 2 Σ + 22 V 2 . . . Cov V (m) 1 , V 2 Σ + 22 V 2      =      Cov V (1) 1 , V 2 . . . Cov V (m) 1 , V 2      Σ + 22 V 2 = Σ 12 Σ + 22 V 2 • Second case: m 1 = 0, m 2 = 0. V 1 -m 1 and V 2 -m 2 are centered Gaussian vectors.
The previous formula can be used:

E [V 1 -m 1 | V 2 -m 2 ] = Σ 12 Σ + 22 (V 2 -m 2 ) .
As the conditional expectation is linear and the event spaces generated by V 2 and V 2m 2 are equal, it follows that:

E [V 1 | V 2 ] = m 1 + Σ 12 Σ + 22 (V 2 -m 2 ) . The mean of E [V 1 | V 2 ]
is m 1 and its covariance matrix is:

Cov (E [V 1 | V 2 ] , E [V 1 | V 2 ]) = Cov Σ 12 Σ + 22 V 2 , Σ 12 Σ + 22 V 2 = Σ 12 Σ + 22 Σ 22 Σ + 22 =Σ + 22 Σ 21 = Σ 12 Σ + 22 Σ 21 .

Proof of proposition 2

This section deals with the proof of proposition 2, which gives a formula of the conditional vector

[V 1 | V 2 = v],
following the notations of subsection 9.2.1.

Proof • First case: m 1 = m 2 = 0. As for all i in 1, m , E V (i) 1 | V 2 is the orthogonal projection V (i) 1 in span(V 2 ), V 1 can be decomposed as V 1 = E [V 1 | V 2 ] + (V 1 -E [V 1 | V 2 ]) = Σ 12 Σ + 22 V 2 + (V 1 -E [V 1 | V 2 ]) (9.3)
with for all i in 1, m ,

V (i) 1 -E V (i) 1 | V 2 ⊥ span(V 2 ). Then conditioning by V 2 = v in equation (9.3): [V 1 | V 2 = v] = Σ 12 Σ + 22 v + (V 1 -E [V 1 | V 2 ]) . (9.4
)

For all i in 1, m , V (i) 1 -E V (i) 1 | V 2 is unchanged because of its orthogonality to span(V 2 ).
• Second case: m 1 = 0, m 2 = 0. The previous formula can be applied to

V 1 -m 1 and V 2 -m 2 : [V 1 -m 1 | V 2 -m 2 = v -m 2 ] = Σ 12 Σ + 22 (v -m 2 ) + (V 1 -m 1 -E [V 1 -m 1 | V 2 -m 2 ]) , ⇔ [V 1 | V 2 = v] -m 1 = Σ 12 Σ + 22 (v -m 2 ) + (V 1 -m 1 -E [V 1 | V 2 ] + m 1 ) , ⇔ [V 1 | V 2 = v] = m 1 + Σ 12 Σ + 22 (v -m 2 ) + (V 1 -E [V 1 | V 2 ]) , The expectation of [V 1 | V 2 = v] has the same formula as E [V 1 | V 2 ] but V 2 is replaced by v: E [V 1 | V 2 = v] = m 1 + Σ 12 Σ + 22 (v -m 2 ) . The covariance matrix of [V 1 | V 2 = v] is equal to: Cov ([V 1 | V 2 = v] , [V 1 | V 2 = v]) = Cov (V 1 -E [V 1 | V 2 ] , V 1 -E [V 1 | V 2 ]) = Cov (V 1 , V 1 ) + Cov (E [V 1 | V 2 ] , E [V 1 | V 2 ]) -2Cov (V 1 , E [V 1 | V 2 ]) = Σ 11 + Σ 12 Σ + 22 Σ 21 -2Σ 12 Σ + 22 Σ 21 = Σ 11 -Σ 12 Σ + 22 Σ 21 9.2.

Proof of proposition 3

This subsection deals with the proof of proposition 3, which gives the equivalence between all the ways of conditioning a Gaussian vector. As a recall, (U, V , W ) is a Gaussian vector with U a Gaussian variable and V and W Gaussian vectors.

Proof

• Case where all variables are centered.

-First way of conditioning U :

Z 1 = [U | V = v, W = w].
U can be decomposed as

U = E [U | V , W ] + T = α V + β W + T, (9.5)
with T ⊥ span(V , W ). Using the decomposition (9.5), Z 1 is equal to:

Z 1 = α v + β w + T.
-Second way of conditioning U :

X = [U | V = v], Y = [W | V = v] and then Z 2 = [X | Y = w].
Using the decomposition of U in equation (9.5), X is equal to:

X = α v + β Y + T because T ⊥ V . Then Z 2 is equal to Z 2 = α v + β w + [T | Y = w] .
As Y ∈ span(V ), T ⊥ Y , so [T | Y = w] = T . Finally:

Z 2 = α v + β w + T = Z 1 .
• General case: let m U , m V , and m W denote the means of U , V , and W . The previous formulae can be applied using

Û = U -m U , V = V -m V , v = v-m V , Ŵ = W -m W , ŵ = w -m W , X = Û | V = v , Ŷ = Ŵ | V = v , Ẑ1 = Û | V = v, Ŵ = ŵ ,
and Ẑ2 = X | Ŷ = ŵ . Ẑ1 can be rewritten as

Ẑ1 = Û | V = v, Ŵ = ŵ = U -m U | V -m V = v -m V , Ŵ -m W = w -m W = [U | V = v, W = w] -m U = Z 1 -m U .
X can be rewritten as

X = Û | V = v = [U | V = v] -m U = X -m U . Similarly, Ŷ is equal to Ŷ = Y -m W . Finally Ẑ2 is equal to Ẑ2 = X | Ŷ = ŵ = [X | Y = w] -m U = Z 2 -m U .
The study of the previous case implies Ẑ1 = Ẑ2 . As a consequence Z 1 = Z 2 .

9.2.4 Proof of proposition 4 [Friedman et al., 2001] This subsection deals with the proof of proposition 4, which says that the likelihood increases at each iteration of the EM algorithm.

Proof By definition of the conditional density

h Z;η (z) = h Z,Z m ;η (z,z m ) h Z m |Z=z;η (z m ) = h T ;η (t) h Z m |Z=z;η (z m )
Taking the log:

l(η, z) = l 0 (η, t) -log(h Z m |Z=z;η (z m ))
Randomizing the expression:

l(η, Z) = l 0 (η, T ) -log(h Z m |Z=z;η (Z m )1 Z=z )
Taking the expectation conditionally to Z = z with the assumption that Z distribution is parameterized by η(k) :

l(η; z) = E η(k) [l 0 (η; T ) | Z = z] Q(η,η (k) ) -E η(k) log h Z m |Z=z;η (Z m ) | Z = z = Q(η, η(k) ) -E η(k)     log     h Z m | Z = z ∼ Z m ;η (Z m | Z = z ∼ Z m )         = Q(η, η(k) ) -E η(k) log h∼ Z m ;η ( ∼ Z m ) R(η,η (k) ) = Q(η, η(k) ) -R(η, η(k) )
Then:

l(η (k+1) , z) -l(η (k) , z) = Q(η (k+1) , η(k) ) -Q(η (k) ; η(k) ) -R(η (k+1) , η(k) ) -R(η (k) ; η(k) )
By definition of η(k+1) :

Q(η (k+1) , η(k) ) -Q(η (k) ; η(k) ) 0
So it must be shown that:

R = R(η (k+1) , η(k) ) -R(η (k) ; η(k) ) 0 Indeed: R = E η(k) log h∼ Z m ;η (k+1) ( ∼ Z m ) -E η(k) log h∼ Z m ;η (k) ( ∼ Z m ) = E η(k) log h∼ Z m ;η (k+1) ( ∼ Z m ) h∼ Z m ;η (k) ( ∼ Z m ) log          E η(k)          h∼ Z m ;η (k+1) ( ∼ Z m ) h∼ Z m ;η (k) ( ∼ Z m ) g( ∼ Z m )                  
(Jensen inequality applied to a concave function)

log E η(k) g( ∼ Z m ) log g( ∼ z m )h∼ Z m ;η (k) ( ∼ z m )dµ( ∼ z m )
Law of the unconscious statistician

log h∼ Z m ;η (k+1) ( ∼ z m ) h∼ Z m ;η (k) ( ∼ z m ) h∼ Z m ;η (k) ( ∼ z m )dµ( ∼ z m ) log h∼ Z m ;η (k+1) ( ∼ z m )dµ( ∼ z m )
log(1) 0 9.2.5 Proof of proposition 5 [Zertuche, 2015] This subsection deals with the proof of proposition 5 which gives the formulae used in the EM algorithm.

Proof

• The randomized complete data loglikelihood is equal to

l 0 (η; Y 1 (X 1 ), Y 1 (X 2 ), Y c (X 2 )) = log h Y 1 (X 1 ),Y 1 (X 2 ),Yc(X 2 );η (Y 1 (X 1 ), Y 1 (X 2 ), Y c (X 2 )) = log h Y 1 (X 1 ),Y 1 (X 2 );η 1 (Y 1 (X 1 ), Y 1 (X 2 )) + log h Yc(X 2 );ηc (Y c (X 2 ))
• The expectation to maximize is

Q(η, η * ) = E η * [l 0 (η; Y 1 (X 1 ), Y 1 (X 2 ), Y c (X 2 )) | Y 1 (X 1 ) = y 1 , Y 2 (X 2 ) = y 2 ] = E η * log h Y 1 (X 1 ),Y 1 (X 2 );η 1 (Y 1 (X 1 ), Y 1 (X 2 )) | Y 1 (X 1 ) = y 1 , Y 2 (X 2 ) = y 2 =Q 1 (η 1 ,η * ) + E η * log h Yc(X 2 );ηc (Y c (X 2 )) | Y 1 (X 1 ) = y 1 , Y 2 (X 2 ) = y 2 Qc(ηc,η * )
• Q 1 can be rewritten as:

Q 1 (η 1 , η * ) = E η * log h Y 1 (X 1 ),Y 1 (X 2 );η 1 (Y 1 (X 1 ), Y 1 (X 2 )) | Y 1 (X 1 ) = y 1 , Y 2 (X 2 ) = y 2 = E η *     log h Y 1 (X 1 ),Y 1 (X 2 );η 1     y 1 , Y 1 (X 2 ) | Y 1 (X 1 ) = y 1 , Y 2 (X 2 ) = y 2 = ∼ Y 1 (X 2 )         = -n 1 +n 2 2 log 2π -1 2 log |Σ θ 1 | -E η *         (y 1 -m 1 1 n 1 ) ∼ Y 1 (X 2 ) -m 1 1 n 2   Σ -1 θ 1   y 1 -m 1 1 n 1 ∼ Y 1 (X 2 ) -m 1 1 n 2   2       = -n 1 +n 2 2 log 2π -1 2 log |Σ θ 1 | -1 2 tr Σ -1 θ 1 cov η * y 1 -m 1 1 n 1 ∼ Y 1 (X 2 ) -m 1 1 n 2 , y 1 -m 1 1 n 1 ∼ Y 1 (X 2 ) -m 1 1 n 2 =t -1 2       (y 1 -m 1 1 n 1 )       E η * ∼ Y 1 (X 2 ) = ∼ µ η * -m 1 1 n 2             Σ -1 θ 1      y 1 -m n 1 E η * ∼ Y 1 (X 2 ) = ∼ µ η * -m 1 1 n 2     
The last equality is obtained using the expectation of a quadratic form formula. Using the componentwise matrix inversion of Σ θ 1 , the trace term is equal to:

t = tr     Σ -1 θ 1     0 n 1 ,n 1 0 n 1 ,n 2 0 n 2 ,n 1 cov η * ( ∼ Y 1 (X 2 ), ∼ Y 1 (X 2 )) = ∼ Σ η *         = tr (k θ 1 (X 2 , X 2 ) -k θ 1 (X 2 , X 1 )k θ 1 (X 1 , X 1 ) -1 k θ 1 (X 1 , X 2 )) -1 ∼ Σ η * • Q c can be rewritten as Q c (η c , η * ) = E η * log h Yc(X 2 );ηc (Y c (X 2 )) | Y 1 (X 1 ) = y 1 , Y 2 (X 2 ) = y 2 = E η * log h Yc(X 2 );ηc (Y 2 (X 2 ) -g(X 2 ) • Y 1 (X 2 )) | Y 1 (X 1 ) = y 1 , Y 2 (X 2 ) = y = E η * log h Yc(X 2 );ηc y 2 -g(X 2 ) • ∼ Y 1 (X 2 ) = -n 2 2 log 2π -1 2 log |Σ θc | -1 2 E η * y 2 -g(X 2 ) • ∼ Y 1 (X 2 ) -m c 1 n 2 Σ -1 θc y 2 -g(X 2 ) • ∼ Y 1 (X 2 ) -m c 1 n 2 = -n 2 2 log(2π) -1 2 log(|Σ θ d |) -1 2 tr Σ -1 θ d diag(g(X 2 )) ∼ Σ η * diag(g(X 2 )) -1 2 Y 2 -g(X 2 ) • ∼ µ η * -m d 1 n 2 Σ -1 θ d Y 2 -g(X 2 ) • ∼ µ η * -m d 1 n 2 9.
2.6 Proof of proposition 6 [Le Gratiet, 2013a] This subsection deals with the proof of proposition 6 which gives a formula of the sobol index easy to implement by a Monte-Carlo estimation.

Proof

Let X = (X 1 , • • • , X i-1 , X i , X i+1 , • • • , X d ) be a random vector. Let ∼ X = ( ∼ X 1 , • • • , ∼ X i-1 , X i , ∼ X i+1 , • • • , ∼ X d
) be another random vector with ∼ X j independent of same law than X j . The covariance between f (X) and f ( ∼ X) can be rewritten as:

Cov(f (X), f ( ∼ X)) = E f (X)f ( ∼ X) -E [f (X)] E f ( ∼ X) = E f (X)f ( ∼ X) -E [f (X)] 2 , because f (X) L = f ( ∼ X) = E E f (X)f ( ∼ X) | X i -E [E [f (X) | X i ]] 2 = E E [f (X) | X i ] E f ( ∼ X) | X i -E [E [f (X) | X i ]] 2 , because ∼ X j ⊥ X j = E E [f (X) | X i ] 2 -E [E [f (X) | X i ]] 2 = V ar(E [f (X) | X i ]).

Proof of proposition 7

This subsection deals with the proof of proposition 7 which gives a formula of the expectation of a centered Gaussian process ∼ Z of kernel k conditioned on any set of points D indexed by a set S of measure ν. This expectation is defined as the projection on the Gaussian space generated by { ∼ Z(t), t ∈ D} and is denoted by:

P ( ∼ Z)(x) = E ∼ Z(x) | ∼ Z(t), t ∈ D .
The proof given in this appendix is different from the one proposed by [Gauthier, 2011]. It uses the Karhunen-Loeve decomposition.

Proof

• The first step consists in focusing on the restriction of P (

P ( ∼ Z)(x s ) = +∞ n=1 ∼ φ n (s) S ∼ φ n (s) ∼ Z(x s )dν(s). ( ∼ φ n ) +∞
n=1 is an orthonormal basis of L 2 (S, ν) of eigenfunctions of the eigenvalue problem:

S k(x s , x u ) ∼ φ n (u)dν(u) = λ n ∼ φ n (s) with S ∼ φ n (s) ∼ φ m (s)dν(s) = δ nm .
• ∼ φ n can be seen as the restriction on D of a function φ n defined on X Z :

∼ φ n (s) = φ n (x s ). Then, the expectation is searched of the form:

P ( ∼ Z)(x) = +∞ n=1 φ n (x) S ∼ φ n (s) ∼ Z(x s )dν(s), ∀x ∈ X Z .
The goal is to find φ n such that P (

Cov ∼ Z(x) -P ( ∼ Z)(x), P ( ∼ Z)(x) = 0 ⇔ Cov ∼ Z(x), P ( ∼ Z)(x) -Cov P ( ∼ Z)(x), P ( ∼ Z)(x) = 0 (9.6)
The first term is equal to:

Cov ∼ Z(x), P ( ∼ Z)(x) = Cov ∼ Z(x), +∞ n=1 φ n (x) S ∼ φ n (s) ∼ Z(x s )dν(s) = +∞ n=1 φ n (x) S ∼ φ n (s)Cov ∼ Z(x), ∼ Z(x s ) dν(s) = +∞ n=1 φ n (x) S ∼ φ n (s)k(x, x s )dν(s).
the second term is equal to:

Cov P ( ∼ Z)(x), P ( ∼ Z)(x) = Cov +∞ n=1 φ n (x) S ∼ φ n (s) ∼ Z(x s )dν(s), +∞ m=1 φ m (x) S ∼ φ m (u) ∼ Z(x u )dν(u) = +∞ n=1 +∞ m=1 φ n (x)φ m (x) S ∼ φ n (s) S ∼ φ m (u)Cov (Z(x s ), Z(x u )) dν(u)dν(s) = +∞ n=1 +∞ m=1 φ n (x)φ m (x) S ∼ φ n (s) S ∼ φ m (u)k(x s , x u )dν(u) λm ∼ φ m (s) dν(s) = +∞ n=1 +∞ m=1 λ m φ n (x)φ m (x) S ∼ φ n (s) ∼ φ m (s)dν(s) δnm = +∞ n=1 λ n φ n (x) 2 .
Using the expressions obtained above, equation (9.6) becomes:

+∞ n=1 φ n (x) S ∼ φ n (s)k(x, x s )dν(s) -λ n φ n (x) = 0.
This equation is satisfied by defining:

φ n (x) = 1 λ n S k(x, x s ) ∼ φ n (s)dν(s).
With this definition, ∼ φ n is the restriction of φ n on D:

φ n (x s ) = 1 λn S k(x s , x u ) ∼ φ n (u)dν(u) λn ∼ φ n (s) = ∼ φ n (s).
9.2.8 Proof of proposition 8 [Gauthier and Bay, 2012a] This subsection recalls the proof of proposition 8, which establishes, when D = D is finite, the equality between the two formulae of

Z(x) = ∼ Z(x) -E ∼ Z(x) | Z(D) , with
∼ Z a centered Gaussian process defined on X = R d of covariance kernel k, and D = {x s , s ∈ S} a subset of X indexed by S of measure ν. The first formula of this process is described in proposition 2:

Z(x) = ∼ Z(x) | ∼ Z(D) = 0 = ∼ Z(x) -k(x, D)k(D, D) -1 ∼ Z(D).
The second formula is given in proposition 7, the reader can refer to it for the notations:

Z(x) = ∼ Z(x) - L n=1 φ n (x) S ∼ φ n (s) ∼ Z(x s )dν(s).
Proof Let D = x (1) , . • Eigenvalue problem. The eigenvalue problem is equal to:

S k(x t , x s ) ∼ φ(s)dν(s) = λ ∼ φ(t), ∀t ∈ S, ⇔ L j=1 k(x (i) , x (j) )ω j ∼ φ(j) = λ ∼ φ(i), ∀i ∈ {1, . . . , L} , ⇔ k(D, D)W ∼ φ(S) = λ ∼ φ(S), ⇔ T W ∼ φ(S) = λ ∼ φ(S),
where T = k(D, D), λ is an eigen value of the problem, and especially of the matrix T W , and • Eigen values and orthonormal eigen vectors. The endomorphism T W is a selfadoint endomorphism of R d for the scalar product (, ) W :

(T W x, y) W = (T W x) W y = x W T W y = x W (T W y) = (x, T W y) W .
As a consequence (see theorem 3 of [Gourdon, 1994]), there exist eigen vectors ( • Formula of φ n : The function φ n (x) (x ∈ X ) is equal to:

φ n (x) = 1 λn S k(x, x s ) ∼ φ n (s)dν(s) = 1 λn k(x, D)W ∼ φ n (S) = 1 λn ∼ φ n (S) W k(D, x).
The functional vector φ

(x) =    φ 1 (x) . . . φ L (x)    is equal to: φ(x) = Λ -1 P W k(D, x).
• Value of the expectation. The formula of the expectation

E ∼ Z(x) | ∼ Z(D)
given in proposition 7 becomes:

E ∼ Z(x) | ∼ Z(D) = L n=1 φ n (x) S ∼ φ n (s) ∼ Z(x s )dν(s) = φ 1 (x) . . . φ L (x)     S ∼ φ 1 (s) ∼ Z(x s )dν(s) . . . S ∼ φ L (s) ∼ Z(x s )dν(s)     = φ(x)     ∼ φ 1 (S) W ∼ Z(D) . . . ∼ φ L (S) W ∼ Z(D)     = φ(x)     ∼ φ 1 (S) . . . ∼ φ L (S)     W ∼ Z(D) = φ(x) P W ∼ Z(D) = k(x, D)W P Λ -1 P W ∼ Z(D) = k(x, D)W P Λ -1 P W P P -1 ∼ Z(D) = k(x, D)W P Λ -1 P -1 ∼ Z(D) = k(x, D)W (T W ) -1 ∼ Z(D) = k(x, D)W W -1 T -1 ∼ Z(D) = k(x, D)T -1 ∼ Z(D) = k(x, D)k(D, D) -1 ∼ Z(D).
This formula coincides with the formula given in proposition 1 and gives the equality wanted.

Proofs of PhD propositions

This section deals with all the proofs related to the analytical formulae of the P process in the context of the seqGPR metamodel.

Proof of proposition 9 for a Monte-Carlo method

This subsection deals with the proof of proposition 9, which establishes the equivalence between approximating the process Z P (defined in equation (3.5)) by discretizing its spectral decomposition with a Monte-Carlo method and conditioning it on the discretization points.

Proof D = {(s J , g(s J )), s J ∈ [0, 1] d J } is indexed by S = [0, 1] d J of measure ν = λ (Lebesgues measure).
• The eigenvalue problem is equal to:

S k(x s J , x u J ) ∼ φ(u J )du J = λ ∼ φ(s J ), ∀s J ∈ S, ⇔ [0,1] d J σ 2 r((s J , g(s J )), (u J , g(u J ))) ∼ φ(u J )du J = λ ∼ φ(s J ) ∀s J ∈ [0, 1] d J .
It is discretized using a Monte-Carlo method. A sample S = (s

(i) J ) 1 i L is generated uniformly in [0, 1] d J to build D = s (i) J , g(s (i) J ) 1 i L
which is a discretization of the subspace D = {(s J , g(s J )), s J ∈ [0, 1] d J }. The Monte-Carlo approximation of the integral in the eigenvalue problem is:

1 L L i=1 σ 2 r((s J , g(s J )), (s (i) J , g(s (i) J ))) ∼ φ(s (i) J ) ≈ λ ∼ φ(s J ).
Considering only the values of ∼ φ in S, the eigenvalue problem becomes a finite dimensional one:

1 L σ 2 r(D, D) ∼ Φ = γ ∼ Φ. (9.7)
The solutions of (9.7) are noted (γ n , V n ) 1 n L . The V n are taken such that they verify:

V n V m = δ nm .
The discretization of

∼ φ n (noted ∼ Φ n ) must verify the discrete equivalent of [0,1] d J ∼ φ n (s J ) ∼ φ m (s J )ds J = δ nm , which is 1 L ∼ Φ n ∼ Φ m = δ nm .
So the following relation between V n and

∼ Φ n is verified: V n = 1 √ L ∼ Φ n ⇔ ∼ Φ n = √ LV n .
• Approximation of φ n :

As

φ n (x J , x I ) = 1 λn [0,1] d J σ 2 r ((x J , x I ), (s J , g(s J ))) ∼ φ n (s J )ds J ∀(x J , x I ) ∈ [0, 1] d J × [0, 1] d I ,
using the same Monte-Carlo approximation and replacing (λ n , ∼ φ n ) by (γ n , ∼ Φ n ), the following approximation of φ n is obtained:

φ D n (x) = 1 γn 1 L σ 2 r(x, D) ∼ Φ n ∀x ∈ [0, 1] d J +d I .
• Decomposition of the matrix (σ 2 r(D, D))

-1

(γ n , V n ) n 1 are the eigenvalues and (orthonormal) eigenvectors of 1 L σ 2 r(D, D). So ( 1 γn , V n ) n 1 are the eigenvalues and (orthonormal) eigenvectors of L (σ 2 r(D, D)) -1 , and:

L (σ 2 r(D, D)) -1 = L n=1 1 γ n V n V n , = L n=1 1 γ n 1 √ L ∼ Φ n 1 √ L ∼ Φ n , = 1 L L n=1 γ n ∼ Φ n ∼ Φ n . So σ 2 r(D, D) -1 = 1 L 2 L n=1 γ n ∼ Φ n ∼ Φ n .
• Approximation of the process Z P :

The integral [0,1] d J

∼

φ n (t J ) ∼ Z(t J , g(t J ))dt J , involved in the formula of Z P (see (3.5)), is discretized the same way as before. It becomes:

1

L ∼ Φ n ∼ Z(D).
The approximation of the process is:

Z D (x) = ∼ Z(x) - L n=1 φ D n (x) 1 L ∼ Φ n ∼ Z(D) , = ∼ Z(x) - L n=1 1 L 1 γ n σ 2 r(x, D) ∼ Φ n 1 L ∼ Φ n ∼ Z(D) , = ∼ Z(x) -(σ 2 r(x, D)) 1 L 2 L n=1 1 γ n ∼ Φ n ∼ Φ n ∼ Z(D), = ∼ Z(x) -(σ 2 r(x, D)) (σ 2 r(D, D)) -1 ∼ Z(D), = ∼ Z(x) -E[ ∼ Z(x) | ∼ Z(D)], = ∼ Z(x) | ∼ Z(D) = 0 .
The process approximating Z P is the conditioned (on a finite set of points) Gaussian process Z D . It is a centered Gaussian process of covariance kernel σ 2 ρ D with:

ρ D (x, x ) = r(x, x ) -r(x, D)r(D, D) -1 r(D, x ) ∀x, x ∈ [0, 1] d J +d I .

Proof of proposition 10

This subsection deals with the proof of proposition 10 which gives a closed form formula of the process

Z P = ∼ Z -E ∼ Z | ∼ Z(D) (where D = {(s J , g(s J )) xs J , s J ∈ S} is indexed by S = [0, 1] d J of measure ν)
for a particular choice of the kernel of ∼ Z: k((x J , x I ), (t J , t I )) = σ 2 r J (x J , t J )r I (x Ig(x J ), t Ig(t J )) with r J and r I two stationary correlation kernels. In this case, the formula is:

Z P (x J , x I ) = ∼ Z(x J , x I ) -r I (x I -g(x J ), 0) ∼ Z(x J , x I ).

Proof

• The eigenvalue problem is equal to:

S σ 2 r (x s J , x u J ) ∼ φ n (u J )dν(u J ) = λ n ∼ φ n (s J ), ∀s J ∈ S, ⇔ [0,1] d J σ 2 r ((s J , g(s J )), (u J , g(u J ))) ∼ φ n (u J )dν(u J ) = λ n ∼ φ n (s J ), ∀s J ∈ [0, 1] d J , ⇔ [0,1] d J σ 2 r J (s J , u J ) r I (g(s J ) -g(s J ), g(u J ) -g(u J )) ∼ φ n (u J )dν(u J ) = λ n ∼ φ n (s J ), ⇔ [0,1] d J σ 2 r J (s J , u J ) r I (0, 0) =1 ∼ φ n (u J )dν(u J ) = λ n ∼ φ n (s J ), ⇔ [0,1] d J σ 2 r J (s J , u J ) ∼ φ n (u J )dν(u J ) = λ n ∼ φ n (s J ). (9.8)
• Expression of φ n :

φ n can be rewritten as (for

(x J , x I ) in [0, 1] d J × [0, 1] d I ): φ n (x J , x I ) = 1 λn [0,1] d J σ 2 r ((x J , x I ), (s J , g(s J ))) ∼ φ n (s J )dν(s J ), = 1 λn [0,1] d J σ 2 r J (x J , s J ) r I (x I -g(x J ), g(s J ) -g(s J )) ∼ φ n (s J )dν(s J ), = 1 λn [0,1] d J σ 2 r J (x J , s J ) ∼ φ n (s J )dν(s J ) r I (x I -g(x J ), 0)) , = 1 λn λ n ∼ φ n (x J ) r I (x I -g(x J ), 0)) , = ∼ φ n (x J )r I (x I -g(x J ), 0)) .
The second last equality is due to the fact that ∼ φ n is solution of the eigenvalue problem (9.8).

• The P process can be rewritten as (for

(x J , x I ) in [0, 1] d J × [0, 1] d I ): Z P (x J , x I ) = ∼ Z(x J , x I ) - +∞ n=1 φ n (x J , x I ) [0,1] d J ∼ φ n (s J ) ∼ Z(s J , g(s J ))dν(s J ), = ∼ Z(x J , x I ) - +∞ n=1 ∼ φ n (x J )r I (x I -g(x J ), 0)) [0,1] d J ∼ φ n (s J ) ∼ Z(s J , g(s J ))dν(s J ), = ∼ Z(x J , x I ) - +∞ n=1 ∼ φ n (x J ) [0,1] d J ∼ φ n (s J ) ∼ Z(s J , g(s J ))dν(s J ) r I (x I -g(x J ), 0) ,
and, because ∼ Z(x J , g(x J )) belongs to the sub Gaussian space engendered by the { ∼ Z(s J , g(s J )) ∀s J ∈ [0, 1] d J }, its projection in this subspace is equal to itself:

∼ Z(x J , g(x J )) = E[ ∼ Z(x J , g(x J )) | ∼ Z(s J , g(s J ))∀s J ], ∀x J ∈ [0, 1] d J , = +∞ n=1 φ n (x J , g(x J )) ∼ φ n (s J ) ∼ Z(s J , g(s J ))dν(s J ), = +∞ n=1 ∼ φ n (x J ) r I (g(x J ) -g(x J ), 0)) =1 ∼ φ n (s J ) ∼ Z(s J , g(s J ))dν(s J ), = +∞ n=1 ∼ φ n (x J ) ∼ φ n (s J ) ∼ Z(s J , g(s J ))dν(s J ).
The second equality is the formula of the conditional expectation (see (3.6)).

Finally:

Z P (x) = ∼ Z(x) -r I (x I -g(x J ), 0) ∼ Z(x J , g(x J )).

Proof of proposition 13

This subsection deals with the proof of proposition 13, which gives an analytical formula of the P process in the case where the conditioning is done on multiple subspaces.

Proof The notations of subsection 2.2.2 in chapter 2 are adapted to proposition 13:

D = {x s , s ∈ S}, with S = [0, 1] d J × B. ν = λ ⊗ ( n B i=1 δ b i )
is choosen as a measure on S. λ is the Lebesgues measure on [0, 1] d J , b i is the ith component of B, K is the size of B, and δ b i is the Dirac measure at b i .

• The eigenvalue problem can be rewritten as (for (s I , s J ) in S):

S σ 2 r((s J , g(s J ) + s Ig(s J )), (t J , g(t J ) + t Ig(t J )))

∼ φ n (t J , t I )dν(t J , t I ) = λ n ∼ φ n (s J , s I ) ↔ S σ 2 r((s J , s I ), (t J , t I )) ∼ φ n (t J , t I )dν(t J , t I ) = λ n ∼ φ n (s J , s I ) ⇔ [0,1] d J σ 2 r J (s J , t J ) B r I (s I , t I ) ∼ φ n (t J , t I )( K i=1 dδ bi (t I )) dt J = λ n ∼ φ n (s J , s I ), ⇔ [0,1] d J σ 2 r J (s J , t J ) K i=1 r I (s I , b i ) ∼ φ n (t J , b i ) dt J = λ n ∼ φ n (s J , s I ),
This eigenvalue problem has the following matricial form: • φ n (x J , x I ) (n 1, (x J , x I ) ∈ [0, 1] d J +d I ) is equal to:

σ 2 r I (B, B) [0,1] d J r J (s J , t J ) ∼ φ n (t J , B)dt J = λ n ∼ φ n (s J , B) ∀s J ∈ [0, 1] d J , ( 9
φ n (x J , x I ) = 1 λn [0,1] d J ×B σ 2 r((x J , x I ), (t J , t I )) ∼ φ n (t J , t I )dν(t J , t I ), = σ 2 λn [0,1] d J r J (x J , t J ) t I ∈B r I (x I , t I ) ∼ φ n (t J , t I )dν(t I ) dt J = r I (x I , B) σ 2 λn [0,1] d J r J (x J , t J ) ∼ φ n (t J , B)dt J = r I (x I , B)r I (B, B) -1 σ 2 r I (B,B) λn [0,1] d J r J (x J , t J ) ∼ φ n (t J , B)dt J = r I (x I , B)r I (B, B) -1 ∼
φ n (x J , B) (using (9.9)).

• The expectation

E ∼ Z(x J , x I ) | ∼ Z(t J , g(t J ) + t I ) ∀(t J , t I ) ∈ [0, 1] d J × B can be rewritten (for (x J , x I ) in [0, 1] d J +d I ): E ∼ Z(x J , x I ) | ∼ Z(t J , g(t J ) + t I ) ∀(t J , t I ) ∈ [0, 1] d J × B = +∞ n=1 φ n (x J , x I ) [0,1] d J ×B ∼ φ n (t J , t I ) ∼ Z(t J , t I )dµ(t J , t I ) = r I (x I , B)r I (B, B) -1 +∞ n=1 ∼ φ n (x J , B) [0,1] d J ×B ∼ φ n (t J , t I ) ∼ Z(t J , t I )dµ(t J , t I )
and, because ∼ Z(x J , g(x J ) + B) belongs to the sub Gaussian space engendered by the { ∼ Z(t J , g(t J ) + t I ) ∀(t J , t I ) ∈ [0, 1] d J × B}, its projection in this subspace is itself:

∼ Z(x J , g(x J ) + B) = E ∼ Z(x J , g(x J ) + B) | ∼ Z(t J , g(t J ) + t I ) ∀(t J , t I ) ∈ [0, 1] d J × B = +∞ n=1 φ n (x J , B) [0,1] d J ×B ∼ φ n (t J , t I ) ∼ Z(t J , g(t J ) + t I )dµ(t J , t I ) = +∞ n=1 r I (B, B)r I (B, B) -1 I K ∼ φ n (x J , B) [0,1] d J ×B ∼ φ n (t J , t I ) ∼ Z(t J , g(t J ) + t I )dµ(t J , t I ) = +∞ n=1 ∼ φ n (x J , B) [0,1] d J ×B ∼ φ n (t J , t I ) ∼ Z(t J , g(t J ) + t I )dµ(t J , t I ).
I K is the identity matrix of size K × K. The expectation becomes (for (x J , x I ) in [0, 1] d I ):

E ∼ Z(x J , x I ) | ∼ Z(t J , g(t J ) + t I ) ∀(t J , t I ) ∈ [0, 1] d J × B = r I (x I , B)r I (B, B) -1 ∼ Z(x J , g(x J ) + B).
• Finally, the P process is equal to

Z P (x J , x I ) = ∼ Z(x J , x I ) -r I (x I , B)r I (B, B) -1 ∼ Z(x J , g(x J ) + B) ∀(x J , x I ) ∈ [0, 1] d J × [0, 1] d I .

Fomulae and algorithms

Formulae of the EM procedure

This subsection gives the formulae of the Q n involved in the optimization problems (4.4) used to estimate the parameters η n in the EM algorithm. 1∼ X 1 is the unit column vector of size nrows ∼ X 1 . Denoting by X 1 and X 2 two DoE's, 0 X 1 ,X 2 is the null matrix of size nrows(X 1 ) × nrows(X 2 ). and θ (i+1) 1 solution of the following problem

                             min θ 1 n∼ X 1 log σ (i+1) 1 (θ 1 ) 2 + log ρ θ 1 ∼ X 1 , ∼ X 1 , with σ (i+1) 1 (θ 1 ) 2 = T r ρ θ 1 ( ∼ X 1 , ∼ X 1 ) -1 C 1 (η (i) ) +(E 1 (η (i) )-m (i+1) (θ 1 )1∼ X 1 ) ρ θ 1 ( ∼ X 1 , ∼ X 1 ) -1 (E 1 (η (i) )-m (i+1) (θ 1 )1∼ X 1 ) n∼ X 1
, and m (i+1) (θ 1 ) =

1 ∼ X 1 ρ θ 1 ∼ X 1 , ∼ X 1 -1 E 1 (η (i) ) 1 ∼ X 1 ρ θ 1 ∼ X 1 , ∼ X 1 -1 1∼ X 1
, where E n (η (i) ) and C n (η (i) ) (n ∈ 1, N ) are given in equation (9.10).

Algorithms of chapter 5

Algorithm 1 optimizes a nested design of higher dimension. Algorithm 2 optimizes a nonnested design of same dimension. In both algorithms, m opt takes the successive values of the sequence X (l) . For any sample m, m[R, C] denotes the submatrix of m located at rows indexed by the vector R and columns indexed by the vector C. m[R, ] denotes the submatrix located at rows indexed by R and containing all columns. Similarly, m[, C] denotes the submatrix located at columns indexed by C and containing all rows. U (V ) denotes the uniform distribution among V components.

9.5 Papers

ABSTRACT

The strategy aimed at reducing the development time for a fan, which is oriented through the use of large meta-models which can be re-used and enriched along time. Efforts have been brought in the parameterization of the geometry and in the simulation process which provides pressure rise, torque and consequently efficiency.

Numerical Designs of Experiments (DoE) are then conducted in an 11 factor problem to fill the space and to build a first kriging model. This latter assumes that the output is a gaussian field for which parameters are computed by maximum likelihood estimation based on the numerical runs. Uncertainties associated with any predicted values can then be assessed using the variance, and two small validation plans are used to measure statistically the errors.

The variance given by the model is further used to map the areas in the domain which would need additional sampling. Then, two strategies are tested to select the most relevant sampling points, the first one being to reduce the range of the parameter variation, and the other one being to select them according to turbomachine design rules which would have dismiss some factor combinations. These two methods for sequential enrichment of the response are then compared and can even be combined with a trend given on the pressure rise. Answers from the kriging models are then assessed again in terms of statistical errors. These strategies will be described in the proposed paper through model comparison and optimization results. Optimization is therefore perfectly adapted, and the current strategies aimed at reducing the development time for the fan, and be re-used and enriched along time. aimed to prepare a methodology for large dimensions, i.e. starting from 30 and going beyond, possibly up to 60. Accuracy must not be compromised neither by the space dimension (given by the number of parameters), nor by the widest range of possible variation run has a non-neglectable cost which is not compatible with an intense sampling campaign. In the present case, 11 design variables simulations, which is of course simply not possible. Optimization is therefore perfectly adapted, and the current strategies aimed at reducing the , and are then oriented through the use of large meta used and enriched along time. Investigations done on the current 11 parameters DoE are methodology for large dimensions, i.e. starting from 30 and going beyond, Accuracy must not be compromised neither by the space dimension (given by the arameters), nor by the widest range of possible variations. However, each simulation neglectable cost which is not compatible with an intense sampling campaign. In the design variables have been selected and the full factorial plan would yield 2 simulations, which is of course simply not possible. Optimization is therefore perfectly adapted, and the current strategies aimed at reducing the are then oriented through the use of large meta-models which can vestigations done on the current 11 parameters DoE are methodology for large dimensions, i.e. starting from 30 and going beyond, Accuracy must not be compromised neither by the space dimension (given by the . However, each simulation neglectable cost which is not compatible with an intense sampling campaign. In the l plan would yield 2 11 Some strategies are then investigated in order to enrich initial Design of Experiment (DoE) which are classically based on a statistical approach uncorrelated from the studied physics. The question is to find a proper way to select the most relevant points which would improve the accuracy. However, one can object that the accuracy is only of interest for some limited set of parameters, when geometries are conforming to turbomachine laws. Therefore, in the context of a limited simulation budget, should the enrichment be based on a purely statistical approach, or is it worthy to use geometrical rules which exclude any sampling on a priori non interesting areas?

KEYWORDS

OBJECTIVES

The objectives of the current work are to investigate a sequential strategy for fan optimization, in the context of a limited simulation budget, which uses at first a meta-model from a kriging method, and secondly a genetic algorithm for optima research. Such methods have been intensively used and presented for low speed fans, for instance by Bamberger [START_REF] Bamberger | Opt losses and noise reduction[END_REF] or Verstraete [START_REF] Verstraete | automotive applications[END_REF].

Several strategies for the selection of potential additional runs are proposed and compared between them. Performances of these methods are assessed through different criteria measured on each surrogate model, which are given by global or local variances, by discrepancies with numerical results from either a global or a local validation plan, and by comparing the optimized final designs.

Geometrical parameterization of the fans and simulation processes are explained in the next paragraph, followed by some theoretical information on the kriging method which has been used. Sequential plans and methods for comparison are then presented before giving results and commenting on their respective efficiencies in the conclusion.

PARAMETERIZATION AND SIMULATIONS

Fan parameterization

A typical automotive fan is presented in figure 2. Several characteristics are frozen to standard values, and the selected parameters for optimization are the chord lengths and the stagger angles for five different radius. These ten geometrical factors are then completed by the flow rate which is imposed in the numerical experiment. DoE must have independent parameters and ensure that all their combinations are feasible in the ranges of variation, meaning that no effect between them would prevent any geometry to be tested. However, it can lead to the creation of designs which can be said at least unconventional, or at worst opposite to the rule of turbomachine design. One can appreciate in figure 3, as an example a surprising resulting geometry on which the DoE can ask an evaluation of performances. In such cases, the heckled surfaces lead to chaotic flow patterns, yielding poor fan efficiencies and difficult numerical convergences (and consequently "signal" noise during the creation of meta-models). At a first thought, they are not representing the most interesting sampling points.

Numerical modeling

As presented in [3], performance predictions for the DoE are obtained, thanks to RANS simulations with the commercial code CCM+, using polyhedra meshes and an accurate description of the test rig (figure 4). The simulation process takes one hour of meshing (a new mesh is produced for each geometry), and requires between 4 to 8 wall clock time with 196 CPU (depending on the convergence). Mesh parameters are kept constant from one geometry to the other in order to minimize discrepancies. An accuracy assessment is provided in [3], and allow us to determine the mesh size above which variations remain below 1%. About 20 millions polyhedra are used, mainly concentrated in the rotating domain (15 millions). The motion of the fan in the steady state run is modeled through multiple frames of refe equations model k-ω SST from Menter, and a two layer boundary layer on the walls. The layers of cell extruded from the surface mesh yield to an average wall y+ close to 1 on the fan blades. The flow rate is imposed at the inlet plenum of the test rig, whereas an atmospheric condition is imposed in the far field. All factors have been dimensionalized in the range [-1, 1], the set of data with 11 factors at zero being the center of the domain. An Orthogonal Latin Hypercube (OLH) plan has been selected for the initial sampling and the training of the meta-model. It is composed of 127 points that have been selected in the [-1, 1] 11 domain, according to the theory which imposes at first that each point must have a different value for each direction (a direction corresponding to an input variable), and secondly that the columns of the design matrix are orthogonal (one column contains the observed values of one design variable).

Two other Latin Hypercube Samples (LHS) designs (which were optimized for discrepancy) of 25 points each have been established for testing the meta-models trained from the 127 runs (and the points for enrichment if any). The first one is set in [-1, 1] 11 , and the second one in [-0.5, 0.5] 11 . For this later it means that all factors are limited in the range [-0.5, 0.5], i.e. only a central part of the domain is considered Kriging meta-model building [5] The kriging meta-model models the computer code, f, as a realization of a random field {Y(x), x in [-1, 1] 11 }. Each component of x is a particular input variable of the simulator. This random field is supposed to be such that:

𝑌(𝑥) = 𝑚 + 𝑍(𝑥) (1)
where m is a scalar constant, called trend, and Z is a gaussian field with zero mean and a covariance function of the type :

𝑘(𝑥, 𝑥 ) = 𝜎 𝑟 (𝑥, 𝑥 ) (2)
𝑟 is called the correlation function. The one used in this work is a tensor product of the Matern kernel :

𝑟 (𝑥, 𝑥′ ) = ∏ (1 + √ ′ + ′ )exp(- √ ′ ) (3)
The Matern kernel is the most commonly used because it is twice differentiable and it is the most realistic with respect to the actual simulation outputs. Other properties of gaussian kernel or exponential kernel are detailed for instance in [START_REF] Le | Multi-fidelity Gaussian process regression for computer experiments[END_REF].

The three parameters m, σ 2 , and θ are determined by maximizing the likelihood function. θ is a vector of 11 components aimed to take into account of the anisotropy of the model. If we consider the set of training observations y, the likelihood function is:

ℒ(𝑚, 𝜎 , 𝜃) = ℙ(𝑌 = 𝑦|𝑚, 𝜎 , 𝜃) = ( ) exp(- ( ) (
) ) ( 4)

where 𝑅 = 𝑟 𝑥 ( ) , 𝑥 ( ) is the correlation matrix on the design points 𝑥 ( ) ,…,𝑥 ( ) . The matrix of the observations (whose lines are the 𝑥 ( ) ) is noted 𝑋, and is called design matrix. The estimation of 𝑚 is different from the least square estimation. The least square estimation of 𝑚 would be given by the formula:

𝒚 (𝟏) ⋯ 𝒚 (𝒏) 𝒏 = 𝟏 𝒏 𝒚 𝟏 𝒏 𝟏 𝒏
. However, the likelihood estimation of 𝑚 is given by the formula: . The least square estimation is a particular case of the maximum likelihood estimation when the correlation matrix 𝑅 is equal to the identity matrix.

Given that, the prediction is:

𝑦 (𝑥) = 𝐸[𝑌(𝑥)|𝑌(𝑋) = 𝑦] = 𝑚 + 𝑟 (𝑥)𝑅 (𝑦 -𝑚1 ) (5)
where 𝑟(𝑥) = (𝑟 (𝑥, 𝑥 ( ) )) is the vector containing the correlation between Y(x) and 𝑌(𝑥 ( ) ),…,𝑌(𝑥 ( ) ).

A kriging variance is also associated to each point of the input domain. It measures the uncertainty of the prediction at this point. The kriging variance is given by:

𝜎 (𝑥) = 𝜎 -𝑟 (𝑥)𝑅 𝑟(𝑥) (6)
Finally, if the main effects of the design variables 𝑥 are near known functions ℎ (𝑥 ), this information can be added to the model by using a trend equal to a linear combination of the main effects rather than a constant trend. The prediction becomes:

𝑦 (𝑥) = 𝑚 + ∑ 𝛼 ℎ (𝑥 ) + 𝑟 (𝑥)𝑅 (𝑦 -𝐹 𝛼) (7)
where 𝐹 = (1 𝐻), 𝐻 = ℎ 𝑥 ( ) ∈{ ,…, }, ∈{ ,…, }

,and 𝛼 = (𝑚 𝛼 … 𝛼 ) .

The vector 𝛼 is estimated by maximum likelihood. It is computed by the formula : (𝐹 𝑅 𝐹) 𝐹 𝑅 𝑦. The least square estimation would be : (𝐹 𝐹) 𝐹 𝑦. Again, the least square estimation is a particular case of the maximum likelihood estimation when the correlation matrix 𝑅 is equal to the identity matrix.

The goal is to improve the accuracy by adding global information on the "shape" of the metamodel.

SAMPLING METHODS

A previous approach studied by Ribaud [7] consists of adding points in the area of interest, extracting them from the Pareto front and selecting them according to a robustness criterion. This method is relevant for a known optimization problem (i.e. when the process can be guided in a limited area of the solution domain), but has no benefit for the global improvement of the metamodel. One proposal from Zhang [8] consists of, in using a small training sample (for example OLH) and then in adding points. His method proved to be very fast because he could use a cokriging technique that drew additional information from the first and second derivatives. Of course, the great difficulty there is to have access to these values, and it implies either to derive the Navier-Stokes equations [9][10] or to assess the derivative with a differentiation method [START_REF] Zhang | Meta-model based optimization of a large diameter semi-radial conical hub engine cooling fan[END_REF].

Enrichment in a restricted hypercube

The first trial consists of selecting one round introduced in the equation ( 6), the objective being to find the points with the maximum varia which can help to improve the model globally by filling the most uncertain areas (factors equal to 1 or -1) are systematically proposed (figure zone where the variance becomes large. Unfortunately, improvement by selecting the 2 11 of interest since it would help to improve the model in the 11 directions. By limiting the selection in the [-0.5, 0.5] 11 domain, it is observed that the first point is very close to the center (red point in figure 6), and then the other points To compare the effect of adding point either Sobol sequence were generated and used to compare the mean of their kriging variance. the mean of the variance criteri because it is the first added point of the second method. meta-model each time another point has been added at black curve (figure 7) shows the in [-1, 1] 11 whereas the red curve is worthy, and then the rate of improvement is rather equivalent The first method to be evaluated in this article will be the one that adds points of maximum kriging variance in the restricted domain [-0.5, 0.5] 11 .

Adding geometrical constraints

Points which are added must be relevant from a turbomachine point of view, in order to avoid strange geometries like in figure 3. When adding geometrical constraints, the selection process for the additional points simply discards the geometries which are not considered credible, and iterates until a validated point is found. The rules are simply based on the observation that irrelevant geometries are characterized by abrupt variations and large amplitudes of both chord length and stagger angles. It does not make sense to have angles of incidence which vary greatly from one radius to another, and that the load is not distributed smoothly along the blade span. Two rules are therefore applied and must be verified simultaneously, one concerning stagger angles and one concerning chord lengths. The rules are defined to limit variations from one radius to another. The examples of figures 8 and 9 show possible evolutions, when variation for stagger angles are limited to +/-10%, and chord lengths do not deviate from the mean line between bottom and top by more than +/-10%. For the sake of comparison, the variations of the fan of figure 3 are presented on the graph. It is obvious that these former geometries don't follow these rules.

The second method to be evaluated in this article will be the one that adds points of maximum kriging variance among the points that satisfy geometrical constraints.

Adding a linear trend

As introduced by the formula (7), the trend can be defined by the following formula (given in eleven dimensions): 𝑚 + ∑ ∝ ℎ (𝑥 ) . This method can be used by considering the pressure rise variation (ΔP) which is rather linear, and systematically decreasing versus flow rate (Qv). The linear trend will therefore have the following shape: 𝑚 + 𝛼 𝑄. In the article, the constant trend and the linear trend will be tested and compared for the ouput ∆𝑃.

Comparing methods

The two validation plans (in [-1, 1] 11 , and in [-0.5, 0.5] 11 ) are used to assess a criterion based on the Root Mean Square Error (RMSE), according to the formula:

𝑅𝑀𝑆𝐸 = ∑ (𝒚 (𝒊) 𝒎 (𝒙 (𝒊) )) 𝟐 𝑵 𝒊 𝟏 𝑵 (8)
𝑥 ( ) is the i th point of the test sample. 𝑦 ( ) is the true output value associated to 𝑥 ( ) . 𝑚 (𝑥 ( ) ) is the meta-model predicted output value associated to 𝑥 ( ) . 𝑁 is the number of points in the test sample.

Accuracy assessment in the Whole Domain

The objective in this paragraph is to compare the quality of two different strategies of enrichment which use only 10 additional points each. Of course these points are not the same according to the strategy. The two methods will be compared to the so-called "initial" model, for which the training sample is limited to the initial sample of 127 points. In the first method, ten points of maximum variance have been selected in [-0.5, 0.5] 11 and added to the initial training sample (forming a new training sample of 137 points). In the second model, ten points of maximum variance among points satisfying the geometrical rules have been added to the initial training sample (forming another training sample of 137 points). In addition with these sampling strategies, linear and constant trends on the pressure curves have been compared.

In summary, there are 3 models with constant trend for the pressure rise, three others with linear trend for the same output, and 3 models for the torque (only constant trend on torque). Table 1 gives the comparison for initial model and the 2 sampling methods.

It can be observed that the enrichment methods have a low incidence on the RMSE criterion since the mean error remains at 12 Pa for the pressure rise, and at 0.03 N.m for the torque. Adding a trend is more promising since it improves the RMSE by 1 to 2 Pa. . show the set of data for the optimized solutions ( kriging respectively without constant or linear similar, i.e. with the initial DoE that the meta-model is already relevant and a small enrichment, despite being based on a statistical approach with variance, does not improve the result. In addition, the small improvement noticed on the RMSE with the trend does not reall an optimized fan is shown in figure 1 The result is slightly different when using the meta fulfilling some criteria based on design rules. angles and in chord length from a radius to another, indicating that the rules have modified the meta-model around their domains of validity. However, despite being less chaotic, the stagger angles still show a big variation from radius 1 to 2 and from radius 4 to 5. For the chord length, there is still a big change from radius 4 to 5. As a consequence, it must be noticed that the optimized solutions using the meta-model with rules are outside these rules. Errors given by the meta-models are presented in table 3, where the act those given by the simulation. The table shows the results obtained when using the 10 models are then compared on two optimization problems:

RMSE on

Maximize ΔP, T=<1N.m, Q set at a given nominal flow rate Minimize T, ΔP>=250 Pa, Q set at a given nominal flow rate show the set of data for the optimized solutions (maximizing constant or linear trend. In both cases, the first two and the enrichment in the middle. At this point, it model is already relevant and a small enrichment, despite being based on a statistical approach with variance, does not improve the result. In addition, the small improvement noticed on the RMSE with the trend does not really change the optimum proposed. For information, a view of figure 12.

The result is slightly different when using the meta-model which has been enriched with points fulfilling some criteria based on design rules. The set of data shows smaller variations in stagger angles and in chord length from a radius to another, indicating that the rules have modified the model around their domains of validity. However, despite being less chaotic, the stagger on from radius 1 to 2 and from radius 4 to 5. For the chord length, there is still a big change from radius 4 to 5. As a consequence, it must be noticed that the optimized model with rules are outside these rules.

solutions when using ΔP meta-(first optim.)

Figure 11: solutions when using ΔP meta models with linear trend (first optim.) Figure 12: optimized fan models are presented in table 3, where the act those given by the simulation. The table shows the results obtained when using the maximizing ΔP), using trend. In both cases, the first two models are rather and the enrichment in the middle. At this point, it would indicate model is already relevant and a small enrichment, despite being based on a statistical approach with variance, does not improve the result. In addition, the small improvement noticed on or information, a view of model which has been enriched with points ows smaller variations in stagger angles and in chord length from a radius to another, indicating that the rules have modified the model around their domains of validity. However, despite being less chaotic, the stagger on from radius 1 to 2 and from radius 4 to 5. For the chord length, there is still a big change from radius 4 to 5. As a consequence, it must be noticed that the optimized : solutions when using ΔP metatrend (first optim.) models are presented in table 3, where the actual performances are those given by the simulation. The table shows the results obtained when using the linear trend in the meta-model, however the results are quite similar without it as demonstrated by comparing figures 10 and 11. All proposed solutions are within a reasonable range of performance which is at the state of the art for such fans, i.e. close to 50% of static to static efficiency.

The enrichment in [-0.5, 0.5] 11 , despite showing no RMSE improvement, has however brought a noticeable improvement in efficiency (+0.4% and +1.5%) versus the initial DoE. As no improvement is noticed in accuracy between these optimizations from the two meta-model, it would suggest that expecting accuracy is not the point, but rather having the good trend.

The last model which has introduced some rules for the choice of added points is finally the less performing one. Efficiencies are almost 3% below those obtained with the enrichment in the middle. Removing some points where the variance was important because of the geometrical criteria has probably lowered the ability of the kriging process to find good trends in the whole domain. 3: Actual performances for the optima, and errors of meta-model predictions

CONCLUSIONS

Investigations have been conducted in order to define the most relevant strategy for metamodeling with sequential enrichment. It is assumed that the number of added points must be limited, and therefore a smart solution for optimizing the proposed runs must be found. Two enrichment methods have been proposed, one being based on the variance in the center of the domain, and one adding some geometrical rules aimed to discard non conventional designs. In addition, a linear trend on the pressure rise has been included in the models and compared to the usual constant trend.

Accuracy assessment obtained with a RMSE criterion on validation designs has shown that limiting the sampling in the center of the domain seems to be an interesting strategy for scarce additional points, and that adding the trend improves the models. This would be all the more to be confirmed since tests with optimization have shown a small gain on maximum efficiency.

Conversely using geometrical rules for the enrichment does not help and even deteriorate quality of both the meta-model and the optimization. It is still not explained, but presuming solutions is ultimately more counterproductive than letting the model learn from all points, including from those that are not good geometries.

Introduction

In industry, numerical codes are widely used to model physical phenomena involved in manufacturing (see section 5.3 for an example of industrial product). Computer experiments make the search for performance easier and cheaper than physical experiments. However, computer codes are confronted to some limitations. As they model complex physical phenomena, they are often computationally expensive. This problem is solved by the use of metamodels [Forrester et al., 2008], [Sacks et al., 1989]. A metamodel is a simpler statistical model, like radial basis neural network [Cheng and Titterington, 1994], kriging model [START_REF] Santner | The design and analysis of computer experiments[END_REF], [Roustant et al., 2012], support vector regression [Clarke et al., 2005]. This simpler model is fit on a sample of well-chosen runs, also called design of experiments (DoE) [Pronzato and Müller, 2012], [Dupuy et al., 2015]. Metamodels provide fast approximations of the code outputs. Besides, they have a proactive role as they help to select relevant simulations to be run [Jones et al., 1998]. The computer code is then used more efficiently.

Building surrogate models can be difficult when numerical codes involve lots of variables: geometrical parameters, environmental ones etc. Complex engineering systems can be found in [START_REF] Makowski | Global sensitivity analysis for calculating the contribution of genetic parameters to the variance of crop model prediction[END_REF], [START_REF] Lefebvre | A methodological approach for statistical evaluation of aircraft infrared signature[END_REF], [START_REF] Auder | Screening and metamodeling of computer experiments with functional outputs. application to thermal-hydraulic computations[END_REF]. Dimension reduction is a common approach to deal with it. It consists in considering only the influential inputs. The influential inputs are often detected by sensitivity analysis methods ( [Saltelli, 2000], [Sobol, 1993], [Iooss and Prieur, 2019]). However, this approach leads to a vicious circle. Indeed, on one side, the variables selected by sensitivity analysis are relevant only if the metamodel is sufficiently accurate, and on the other side, the dimension reduction is used to increase the metamodel accuracy. An alternative method that is usually chosen in industry is to act step by step. The first studies carried out on the code have focused on a small amount of variables deemed as important based on expert knowledge. The other variables are set to nominal values. A metamodel is created and used only for these variables until the understanding of their influence is sufficient. Then, to obtain better performances, finer studies are proceeded with, in which new variables are added progressively. The classical approach is to build a metamodel on a new independent space-filling design in the wider input space. Yet, this approach may be criticized as the information provided by the previous DoE's and metamodels is disregarded.

In the present work, an original metamodel is introduced, which takes into account the information from all the steps. The chosen approach consists in creating dependent metamodels from one step to another. The metamodels are based on Gaussian process regression [START_REF] Santner | The design and analysis of computer experiments[END_REF]Rasmussen, 2006]. One way of taking into account the different designs could be to use a multifidelity model for which the levels are not defined on the same input space. That is what is done in [Hebbal et al., 2021] with deep Gaussian processes. Nevertheless, this model is really expensive to learn. Furthermore, the multifidelity context is not suitable since the runs obtained at the previous steps have the same accuracy level. Another way could be to define a virtual categorical input, equal to the number of the step, that influences the choice of the input variables to consider. [Pelamatti et al., 2021] defines a Gaussian process whose input space varies dynamically with this virtual categorical input. This modelization does not take into account the fact that the input sets are entwined from one step to another. This particularity can lead to a simpler modelization. The approach in this work, called seqGPR (sequential Gaussian process regression), is based on a recursive statistical model inspired by the autoregressive multifidelity model [Kennedy and O'Hagan, 2000] but with the same accuracy for all the runs and with an input space of increasing dimension. The output of the code at a given step is modeled as the realization of a Gaussian process being the sum of the Gaussian process that models the previous step and a correction term. The correction term stands for the information provided by the new variables.

As regards this probabilistic model, one first difficulty is to build an appropriate correction term to represent the gap between the same numerical model observed on two nested subspaces. Thus, the correction process must be null on the subspace corresponding to the previous step. As this subspace is composed of an infinite continuous set of points, the question is then how to build a Gaussian process null on an infinite continuous set of points with a numerically computable covariance kernel. One idea could be to enjoin the prediction to verify the nullity property. For example, relevant points from the concerned subspace can be added to the training sample. This technique is used in [Da Veiga and Marrel, 2012] in the context of monotonicity, boundedness, convexity constraints. One drawback is the greediness of that method. Besides, the nullity cannot be verified in the whole subspace. Instead of establishing the nullity in retrospect, it can be verified at first glance, as an intrinsic property of the correction term. The correction process could be defined in a finite dimensional way, with adequate basis functions, as it is done in [Maatouk and Bay, 2017], [Lopera, 2019], [Bachoc et al., 2020] to ensure some properties of monotonicity, boundedness, convexity. Yet, this approach is greedy as the number of basis functions increases with the dimension. In this paper, the Gaussian process introduced in [Gauthier and Bay, 2012] which is an extension of the conditional expectation to an infinite continuous set of points is retained. A tractable kernel is sought for this process which is then compared to another candidate verifying the nullity condition.

Once a tractable candidate for the correction term is proposed, another difficulty lies in the estimation of the hyperparameters. One way is to optimize the likelihood. This task can be numerically difficult, since the likelihood involves several sets of hyperparameters for the different processes. One way to reduce the dimension of the optimization space is to propose nested designs as it is done in [Le Gratiet and Garnier, 2014]. In this paper an alternative based on the expectation maximization algorithm [Friedman et al., 2001] is introduced. It allows the reduction of the dimension with limited constraints on the designs.

The formulation of the metamodel seqGPR (sequential Gaussian process regression) is detailed in section 2. Two candidates for the correction term are proposed in section 3. An EM (expectation-maximization) algorithm is suggested in section 4 to estimate the model parameters. Finally, the method seqGPR is tested on two analytic examples and one industrial case in section 5.

Metamodel

Underlying processes Let f : [0, 1] d → R be an output of an expensive simulation code depending on d variables. A response surface of f has to be created taking into account that N -1 previous studies of increasing dimension have already been completed. Each of these steps is a focus on the relation between the output and a subset of free variables. The other variables are temporarily fixed but then progressively released in the further steps. This paper deals with the handling of the last step, denoted by N , where all inputs are free. Let I n denote the index set of the variables that are released at step n ∈ 1, N and d n = card(I n ). For every index set I, let x I denote the associated subvector of x. The following framework, based on Gaussian process regression, is introduced to model the successive studies:

Y 2 : Y 2 (x I1 , x I2 ) = Y 1 (x I1 ) + Z 2 (x I1 , x I2 ), ∀(x I1 , x I2 ) ∈ [0, 1] d1+d2 ,
with Z 2 a centered Gaussian process independent from Z 1 , of covariance kernel σ 2 2 ρ 2 such that for all x I1 in [0, 1] d1 , Z 2 (x I1 , xI2 ) = 0. Let y 2 = f 2 (X 2 ) denote the vector of observations for this step.

• In the same way as previous steps, at step n ∈ 3, N , a DoE X n is created in the higher space

(x I1∪•••∪In-1 , xIn ) = 0, ∀x I1∪•••∪In-1 ∈ [0, 1] d1+•••+dn-1 .
The coincidence of the (Y n ) 1 n N on the subspaces is illustrated on figure 1 for N = 2.

Metamodel seqGPR Finally, the problem is modeled by the following statistical model:

       Y 1 (x I1 ) = m + Z 1 (x I1 ), ∀x I1 ∈ [0, 1] d1 , Y n (x I1∪•••∪In-1 , x In ) = Y n-1 (x I1∪•••∪In-1 ) + Z n (x I1∪•••∪In-1 , x In ), ∀n ∈ 2, N , ∀x I1∪•••∪In ∈ [0, 1] d1+•••+dn , (1) where:
• The processes (Z n ) 1 n N are independent Gaussian processes of law:

Z n ∼ GP(0, σ 2 n ρ n (x I1∪•••∪In , t I1∪•••∪In )) ∀n ∈ 1, N . (2)
The same formulas of prediction as for a classic kriging metamodel apply. For every x ∈ [0, 1] d , the prediction mean ŷ(x) and variance v(x) are defined by

ŷ(x) = E [Y N (x) | Y 1 (X 1 ) = y 1 , • • • , Y N (X N ) = y N ] , v(x) = V ar (Y N (x) | Y 1 (X 1 ) = y 1 , • • • , Y N (X N ) = y N ) .
Issues This model raises two issues. The first difficulty is to build (Z n ) 2 n N such that it verifies the nullity property (cf ( 3)). This issue is tackled in section 3. Secondly, the estimation of the parameters of Y 1 and (Z n ) 2 n N at step N , in the presence of training samples from the different steps, stands ahead as a thorny hurdle to overcome. It is detailed in section 4.

Candidates for the correction process

This section answers the first issue, which is to build a process that is null on an infinite continuous set of points. More precisely, the goal is to build a process Z : [0, 1] d J +d I × Ω → R such that:

Z(x J , g(x J )) = 0 ∀x J ∈ [0, 1] d J , (4)
with g : [0, 1] d J → [0, 1] d I a deterministic function. In this section, two candidates are suggested for the process Z, both based on a latent Gaussian process ∼ Z ∼ GP 0, σ 2 r((x J , x I ), (t J , t I )) of covariance kernel σ 2 r.

Red (reduced) process

A natural idea to obtain the nullity property ( 4) is to subtract from ∼ Z its value on the subspace of [0, 1] d J +d I equal to {(x J , g(x J )) | x J ∈ [0, 1] d J }. The first candidate is:

Z Red (x J , x I ) = ∼ Z(x J , x I ) - ∼ Z(x J , g(x J )).
It is called the Red process. It is a centered Gaussian process of covariance kernel σ 2 ρ Red with: ρ Red ((x J , x I ), (t J , t I )) = r ((x J , x I ), (t J , t I ))r ((x J , x I ), (t J , g(t J )))

-r ((x J , g(x J )), (t J , t I )) + r ((x J , g(x J )), (t J , g(t J )) .

P (preconditioned) process

Another way to obtain the nullity property ( 4) is to use the "conditional expectation" [Gauthier and Bay, 2012].

The second candidate is defined by

Z P (x J , x I ) = ∼ Z(x J , x I ) -E ∼ Z(x J , x I ) | ∼ Z(s J , g(s J )), ∀s J ∈ [0, 1] d J , ∀(x J , x I ) ∈ [0, 1] d J +d I . (5)
It is called the P process. The term E ∼ Z(x J , x I ) | ∼ Z(s J , g(s J )), ∀s J ∈ [0, 1] d J is the orthogonal projection of ∼ Z(x J , x I ) in the sub Gaussian space engendered by the ∼ Z(s J , g(s J ))

s J ∈[0,1] d J
. Its expression is given by [Gauthier and Bay, 2012]:

E ∼ Z(x J , x I ) | ∼ Z(s J , g(s J )), ∀s J ∈ [0, 1] d J = +∞ n=1 φ n (x J , x I ) [0,1] d J ∼ φ n (s J ) ∼ Z(s J , g(s J ))ds J , (6)
with:

φ n (x J , x I ) = 1 λn [0,1] d J σ 2 r((x J , x I ), (s J , g(s J )))

∼ φ n (s J )ds J , and (λ n , ∼ φ n ) n 1 are solutions of the eigen problem:

[0,1] d J σ 2 r ((x J , g(x J )), (s J , g(s J )))

∼ φ n (s J )ds J = λ n ∼ φ n (x J ), ∀x J ∈ [0, 1] d J ,
such that:

[0,1] d J ∼ φ n (s J ) ∼ φ m (s J )ds J = δ nm , ∀n, m 1,
where δ nm is the Kronecker symbol. Z P is a centered Gaussian process of covariance kernel:

σ 2 ρ P ((x J , x I ), (t J , t I )) = σ 2 r((x J , x I ), (t J , t I )) -+∞ n=1

λ n φ n (x J , x I )φ n (t J , t I ),

∀(x J , x I ) ∈ [0, 1] d J × [0, 1] d I , ∀(t J , t I ) ∈ [0, 1] d J × [0, 1] d I .
This kernel cannot be used just as it is in practice, because the solutions of the eigen problem are not explicit in general and the sums are infinite. In this paper, two ways of computing this kernel are proposed. The first is based on the discretization of the spectral decomposition. The second is based on a wise choice of r (the correlation kernel of the latent process ∼ Z) for which an explicit formula is known.

Numerical approximation

In this paragraph, a first way of computing the kernel is given by an approximation based on the discretization of the spectral decomposition that reduces the functional eigen problem to a finite dimensional one. In that case, the terms from the spectral decomposition vanish and the formula of the resulting approximate kernel only depends on the kernel of the latent process.

Proposition 1 Let D = t (i)
J , g(t

(i) J ) 1 i L
be a uniform sample in the subspace {(t J , g(t J )), t J ∈ [0, 1] d J }.

Then, discretizing the spectral decomposition of the P process Z by a Monte-Carlo method using D is equivalent to approximating the P process by the process ∼ Z conditioned to be null on the points of D:

Z D = ∼ Z | ∼ Z(D) = 0 .
Z D is a centered Gaussian process of covariance kernel σ 2 ρ D : ρ D (x, t) = r(x, t)r(x, D)r(D, D) -1 r(D, t) ∀x, t ∈ [0, 1] d J +d I .

See appendix A for the proof of this proposition.

This process does not match the original expectations as the nullity is not fully respected on the continous set of points. The size of D must be high to correctly approach the full nullity. It is more and more difficult as the dimension of the input space increases, and it leads to very expensive computations (with inversion of huge matrices). Therefore, a second way of computing this kernel is proposed. It consists in finding forms of the kernel of ∼ Z that enable the kernel of Z P to be tractable. Analytical expression in the case of a product kernel Proposition 2 Let r J and r I be two stationary correlation kernels. If r is of the form:

r((x J , x I ), (t J , t I )) = r J (x J , t J )r I (x Ig(x J ), t Ig(t J )),

then the P process is a centered Gaussian process equal to:

Z P (x J , x I ) = ∼ Z(x J , x I ) -r I (x I -g(x J ), 0) ∼ Z(x J , g(x J )),
whose covariance kernel is σ 2 ρ P with:

ρ P ((x J , x I ), (t J , t I )) = r J (x J , t J ) [r I (x Ig(x J ), t Ig(t J ))r I (x Ig(x J ), 0)r I (0, t Ig(t J ))] .

See appendix B for the proof of this proposition.

Corollary 1 If g is constant equal to c ∈ [0, 1] d I and if r is a stationary kernel of the form r((x J , x I ), (t J , t I )) = r J (x J , t J )r I (x I , t I ), then the P process is a centered Gaussian process equal to:

Z P (x J , x I ) = ∼ Z(x J , x I ) -r I (x I , c) ∼ Z(x J , c),
whose covariance kernel is σ 2 ρ P with:

ρ P ((x J , x I ), (t J , t I )) = r J (x J , t J ) [r I (x I , t I )r I (x I , c)r I (c, t I )] .

The model presented in the corollary (where the released variables were previously fixed at a constant value) is illustrated in figure 2.

In the following, the P processes are assumed to be built as in proposition 2 or corollary 1.

Interpretation of the candidates

An illustration of the processes at stake in the construction of P and Red processes is shown in figure 3. They are used to build Z(x 1 , x 2 ) such that Z(x 1 , 0) = 0. One main difference between Red and P process is that the initial latent process is disturbed locally in the case of the P process, whereas it is disturbed globally in the case of the Red process. Indeed, ∼ Z(x 1 , 0) (see panel 3a), which is used to build the Red process, is constant in x 2 and consequently generates a disruption of that should be null on the line x 2 = 0. On left panel, the process which is removed from the latent process to create the Red process. On right panel, the process (denoted by E for expectation) which is removed from the latent process to create the P process.

Estimation of the parameters

This section answers the second issue which is to estimate the parameters of the metamodel seqGPR defined in section 2. An EM algorithm is presented to ease the maximization of the likelihood.

Notations At step N , the following training samples (also called observed data) are available: (X 1 , y 1 ), ..., (X N , y N ). The training data can be written as: The law of Y 1 (see equations ( 1) and ( 2)) depends on the parameters: m (scalar mean parameter), σ 2 1 (scalar variance parameter), and θ 1 (vector of covariance parameters). For all n ∈ 2, N , Z n (see equation ( 2)) is either a Red or P process built on the latent process ∼ Z n ∼ GP(0, σ 2 n r n ) (see section 3) of parameters: σ 2 n (scalar variance parameter), θ n (vector of covariance parameters). In order to emphasize the dependence of ρ n (the covariance kernel of Z n , n ∈ 1, N ) on the parameter θ n , this kernel is denoted by ρ θn . The purpose of this section is to estimate the parameters η = (m, σ In what follows, the notation X n ⊂ X n-1 means that the submatrix of X n composed of the columns corresponding to x I1∪•••∪In-1 is included in X n-1 . In this case, the designs X n and X n-1 are said nested. Table 2: Performance of the metamodels for the 100 studies made on the industrial 15D example. The performances are shown in terms of median of RMSE's and interquartile range (q 75%q 25% )

         Y 1 (X 1 ) = y 1 , . . .
• The chord lengths at the five sections: Chord1, Chord2, Chord3, Chord4, Chord5

• The sweeps at sections 2 to 5: Swe2, Swe3, Swe4, Swe5

In the rest of this example, all variables are adimensionalized in [0, 1] and the output ∆P is adimensionalized in [-1, 1].

Industrial experts at Valeo have carried out a two-step (N = 2) study in the context of this work.

• At step 1, all the sweeps are fixed to constants: Swe2 = 0.517645, Swe3 = 0.82, Swe4 = 1, Swe5 = 1.

An OLH of size 126 has been created on the 11 free inputs (I 1 = {1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11}).

• At step 2, all the inputs are free (I 2 = {12, 13, 14, 15}). An OLH of size 299 has been created for the 15 inputs.

In order to obtain robust results, a cross-validation procedure is implemented. Each OLH is decomposed in 30 subsamples of size 50 optimized for the maximin criterion (they can share common points). The 30 complementary subsamples of size 249 from the OLH of step 2 are used as test samples.

Table 2 shows performances of the different metamodels on the test samples. The metamodel seqGPR is better than K_tot. Its version with P is this time slightly better than with Red, again probably because the fixed inputs are constant as in the previous example.

Conclusion

In the framework of Gaussian process regression, the problem of building metamodels in nested spaces of increasing dimension is studied. The variables are first fixed and then released progressively. The suggested method, called seqGPR, consists in creating a kriging metamodel with a non-stationary kernel on the reunion of all training samples. This kernel is the covariance kernel of a process defined recursively. At a given step, the process modeling the output is equal to the sum of the process modeling the output at the previous step and an independent correction term.

The correction term must be a Gaussian process which is null on a part of the input space. Two candidates are proposed: the Red (Reduced) process, and the P (Preconditioned) process. The P process, found in the literature, is intractable. Two ways are tried to make it tractable. The first way is to discretize its spectral decomposition. But this technique leads to a Gaussian process conditioned to be null on some points of the space. This process is never exactly null everywhere in the part we want. To approach this nullity, a lot of points are needed in the conditioning, and that is very computationally expensive. In a second approach, a tractable exact expression is directly sought, in exchange for a modification of the latent process it is built upon.

Then, the issue of the parameter estimation is addressed. Instead of directly optimizing the likelihood, an EM algorithm is implemented, which is adapted to the additive structure composed of independent processes. Finally, the metamodel seqGPR is compared to a classic kriging metamodel, based on a stationary kernel, on two analytic and one industrial examples. Different levels of parameter sparsity for the correction processes are tried. The level of sparsity that seems to always be better than classic kriging is the Robust one with either Red or P processes. The Red version seems to achieve higher performances than the P version most of the time, so the Red_rob version is recommended.

which is

1 L ∼ Φ n ∼ Φ m = δ nm .
So the following relation between V n and ∼ Φ n is verified:

V n = 1 √ L ∼ Φ n ⇔ ∼ Φ n = √ LV n .
• Approximation of φ n : As φ n (x J , x I ) = 1 λn [0,1] d J σ 2 r ((x J , x I ), (s J , g(s J )))

∼ φ n (s J )ds J ∀(x J , x I ) ∈ [0, 1] d J × [0, 1] d I , using the same Monte-Carlo approximation and replacing (λ n , ∼ φ n ) by (γ n , ∼ Φ n ), the following approximation of φ n is obtained:

φ D n (x) = 1 γn 1 L σ 2 r(x, D) ∼ Φ n ∀x ∈ [0, 1] d J +d I .
• Decomposition of the matrix σ 2 r(D, D)

-1

(γ n , V n ) n 1 are the eigenvalues and (orthonormal) eigenvectors of 1 L σ 2 r(D, D). So ( 1 γn , V n ) n 1 are the eigenvalues and (orthonormal) eigenvectors of L σ 2 r(D, D)

-1 , and:

L σ 2 r(D, D) -1 = L n=1 1 γ n V n V n , = L n=1 1 γ n 1 √ L ∼ Φ n 1 √ L ∼ Φ n , = 1 L L n=1 γ n ∼ Φ n ∼ Φ n . So σ 2 r(D, D) -1 = 1 L 2 L n=1 γ n ∼ Φ n ∼ Φ n .
• Approximation of the process Z P :

The integral [0,1] d J ∼ φ n (t J ) ∼ Z(t J , g(t J ))dt J , involved in the formula of Z P (see (5)), is discretized the same way as before. It becomes:

1

L ∼ Φ n ∼ Z(D).
The approximation of the process is:

Z D (x) = ∼ Z(x) - L n=1 φ D n (x) 1 L ∼ Φ n ∼ Z(D) , = ∼ Z(x) - L n=1 1 L 1 γ n σ 2 r(x, D) ∼ Φ n 1 L ∼ Φ n ∼ Z (

B Proof of proposition 2

This section deals with the proof of proposition 2 which gives a closed form formula of the process Z P (see equation ( 5)) for a particular choice of the kernel of ∼ Z.

Proof

• Eigenvalue problem: The eigenvalue problem can be rewritten as:

[0,1] d J σ 2 r ((x J , g(x J )), (s J , g(s J ))) ∼ φ n (s J )ds J = λ n ∼ φ n (x J ), ∀x J ∈ [0, 1] d J , ⇔ [0,1] d J σ 2 r J (x J , s J ) r I (g(x J )g(x J ), g(s J )g(s J ))

∼ φ n (s J )ds J = λ n ∼ φ n (x J ), ⇔ [0,1] d J σ 2 r J (x J , s J ) r I (0, 0)

=1 ∼ φ n (s J )ds J = λ n ∼ φ n (x J ), ⇔ [0,1] d J σ 2 r J (x J , s J ) ∼ φ n (s J )ds J = λ n ∼ φ n (x J ). (10)
• Expression of φ n :

φ n can be rewritten as:

φ n (x J , x I ) = 1 λn [0,1] d J σ 2 r ((x J , x I ), (s J , g(s J )))

∼ φ n (s J )ds J , ∀(x J , x I ) ∈ [0, 1] d J × [0, 1] d I , = 1 λn [0,1] d J σ 2 r J (x J , s J ) r I (x Ig(x J ), g(s J )g(s J ))

∼ φ n (s J )ds J , = 1 λn [0,1] d J σ 2 r J (x J , s J ) ∼ φ n (s J )ds J r I (x Ig(x J ), 0)) ,

= 1 λn λ n ∼ φ n (x J ) r I (x I -g(x J ), 0)) , = ∼ φ n (x J )r I (x I -g(x J ), 0)) .
The second last equality is due to the fact that ∼ φ n is solution of the eigenvalue problem (10).

• The P process can be rewritten as: The second equality is the formula of the conditional expectation (see [START_REF] Le | Multi-fidelity Gaussian process regression for computer experiments[END_REF]).

So

Z P (x) = ∼ Z(x)r I (x Ig(x J ), 0) ∼ Z(x J , g(x J )).

Figure 1 . 1 :

 11 Figure 1.1: On left panel, diagram of the car engine cooling system. On right panel, diagram of fan system.[Valeo, ]

5 Figure 1 . 3 :

 513 Figure 1.3: On the left panel, visualization of the five sections of the blade. On the middle panel, definition of the stagger angle and the chord length at a given section. On the right panel, definition of the sweep at section 5.

∆m 1 with

 1 x (j) = j∆ m (j ∈ 0, m ) a discretization of [0, 1] and ∆ m = 1 m the discretization interval. The expectation E Ŵ (x) | a Ŵ (X ) b is computed by a Monte-Carlo strategy where the paths of Ŵ (x) | a Ŵ (X ) b are generated by rejection sampling. It is made possible as it is reduced to simulating the vector (ξ 0 , • • • , ξ m ) with the constraints a < ξ i < b.

  example of the DoE's X 1 and X 2 is shown in the figure below: Example in 4D This example will follow the course of the thesis to illustrate the general definitions in a particular case, which tries to encompass the problem in its generality: variables fixed at constants or deterministic functions, variables released one by one or several at a time, and a study composed of more than two steps. In this example, N = 3, I 1 = 1, I 2 = {2, 3}, I 3 = 4, (x 2 , x3 ) = (0.4, 0.5), x4 = x 1 +x 2 +x

3

 3

4

 4

1 Figure 3 . 1 :

 131 Figure 3.1: Illustration of the coincidance of Y n and Y n-1 when xIn is constant.

Figure 3 .

 3 Figure 3.2 shows a path of each process : Classic (see panel 3.2a), Psi (see panel 3.2b), Red (see panel 3.2c), and P (see panel 3.2d). Each path x → Z(x, ω) was taken at the same value of ω and with the same parameters : σ 2 = 1, θ = (

Figure

  Figure 3.4 shows paths of the Classic, Psi, Red, and P processes with the same parameters than in figure 3.2 except that σ 2 = 3. Consequently to the increase of σ 2 , all the paths have a bigger range. The influence of σ 2 is the same for all processes. It defines the degree of dispersion of the path. The bigger σ 2 is, the more the path is dispersed.

Figure 3 . 2 :

 32 Figure 3.2: Comparison of Classic, Psi, Red and P paths. Visualization of a path x → Z(x, ω) of each process. The same ω is used for every process. The orange line is the line x 2 = 0.7x 1 + 0.2 on which the Psi, Red and P processes are null. On topleft, panel 3.2a shows a path of the Classic process. On topright, panel 3.2b shows a path of the Psi process. On bottomleft, panel 3.2c shows a path of the Red process. On bottomright, panel 3.2d shows a path of the P process.

Figure 3 . 3 :

 33 Figure 3.3: Processes used to transform the latent process in order to verify the nullity property. The processes on panel 3.3a, 3.3b, and 3.3c are used respectively to build the Psi, Red and P processes.

3. 3 . 3 1 Figure 3 .

 3313 Figure 3.5 shows paths of the Classic, Psi, Red, and P processes with the same parameters than in figure3.2 except that θ 1 = 0.1. The influence of θ 1 is the same for all processes. It defines the degree of chaos of the path in the x 1 direction. The smaller θ 1 is, the more chaotic the path is in the x 1 direction.

Figure 3 . 4 : 2 4

 342 Figure 3.4: Paths of Classic, Psi, Red, and P processes for σ 2 = 3 instead of σ 2 = 1 in figure 3.2. The other parameters are unchanged : θ 1 = θ 2 = √ 2 4 , δ = 0.1.

Figure 3 . 5 : 2 . 2 4

 3522 Figure 3.5: Paths of Classic, Psi, Red, and P processes for θ 1 = 0.1 instead of θ 1 = √ 2 4 in figure 3.2. The other parameters are unchanged : σ 2 = 1, θ 2 = √ 2 4 , δ = 0.1.

Figure 3 . 6 : 2 .Figure 3 . 7 :

 36237 Figure 3.6: Paths of Classic, Psi, Red, and P processes for θ 2 = 0.1 instead of θ 2 = √ 2 4 in figure 3.2. The other parameters are unchanged : σ 2 = 1, θ 1 = √ 2 4 , δ = 0.1.

√ 2 4

 2 (called Psi_deltamin in lightgreen) and δ = √ 2 (called Psi_deltamax in green). θ estimation Figure 3.8 shows boxplots of estimated values of θ 1 and θ 2 for the two sizes of the training sample. All estimations are centered on the true values θ simu and become more accurate when the training size increases. This result seems to indicate that the maximum likelihood estimators converge towards the true values when the sample size goes to +∞. For a training size of 100, all processes have similar performances. For a training size of 10, the

δ

  estimation for Psi Figure 3.10 shows boxplots of estimated values of δ for the different training sizes and values of δ simu and θ simu . In all cases except θ simu = (

Figure 3 . 8 :

 38 Figure 3.8: Estimation, on 100 training samples, of the covariance parameters of the different processes (Classic, Psi, Red, P) : θ 1 (panels 3.8a and 3.8c) and θ 2 (panels 3.8b and 3.8d). Different values of θ are used in the DoE : θ simu = (

Figure 3 . 10 : 2 4

 3102 Figure 3.10: Estimation of δ for the Psi process. The true value δ simu , represented by the black horizontal line, is set to √ 2 4 (panels 3.10a and 3.10c) or √ 2 (panels 3.10b and 3.10d). θ simu is set to ( √ 2 4 , √ 2 4 ) (panels 3.10a and 3.10b) or (2 √ 2, 2 √ 2) (panels 3.10c and 3.10d). σ 2 simu is set to 1. The training sizes are equal to 10 or 100.

  no redundant points and are nested in each other, so they play the role of X 1 , • • • , X N in the case of nested designs. According to proposition 11, if the values of Y 1 on ∼ X 1 , denoted by z 1 , and the values of the Z n on the ∼ X n

  1 . An example of such samples is shown in figure below: If X 1 and X 2 are nested, then the values of Y 1 and Z 2 on X 2 can be deduced. Indeed, let denote by y 1 =

  of y 1 and y 2 , and let (x

∼ X 1 2 ∼X 1

 121 and ∼ X 2 is illustrated in figure below: These new designs are nested: ∼ X (see definition 7 in subsection 4.1.1) and thus, the complete data loss function (i.e. associated to (Y 1 (

Figure 4 . 1 :

 41 Figure 4.1: Example of training samples used for the test of the EM estimation.

Figure 4 . 2 :

 42 Figure 4.2: Convergence of the loss function in the EM algorithm. Panels 4.2c and 4.2d show the values of the loss function starting from iteration 2 of the EM algorithm, whereas panels 4.2a and 4.2b show the values of the loss function starting from iteration 1.

Figure 4 .

 4 Figure 4.3 shows RMSE of the different methods computed on a Sobol sequence of size 10000 used as a test set. The RMSE is computed for 100 different DoE's (X 1 , y 1 ) and (X 2 , y 2 ). The 100 RMSE's are represented by a boxplot for each metamodel. Different sizes of DoE's are tested. A comparison is made between nested designs (built following section 5.1 in chapter 5) and non-nested ones (built following section 5.2 in chapter 5). Different sizes of design are also compared to see the evolution of the RMSE with the number of training points and see the influence of the size of X 1 relative to the size of X 2 :

  (a) small nested designs (b) small non-nested designs (c) big nested designs (d) big non-nested designs

Figure 4 . 3 :

 43 Figure 4.3: RMSE's (see equation 2.2) of the different metamodels on 100 studies. The black line is the standard deviation of the output on the test sample. In panels 4.3a and 4.3b, small training sizes are tested : 10pts/5pts (X 1 of size 10, X 2 of size 5), 10pts/10pts (X 1 of size 10, X 2 of size 10), 10pts/20pts (X 1 pf size 10, X 2 of size 20), and 15pts/15pts (X 1 of size 15, X 2 of size 15). In panels 4.3c and 4.3d, bigger training sizes are tested : 20pts/10pts (X 1 of size 20, X 2 of size 10), 20pts/20pts (X 1 of size 20, X 2 of size 20), 20pts/40pts (X 1 of size 20, X 2 of size 40), and 30pts/30pts (X 1 of size 30, X 2 of size 30). In panels 4.3a and 4.3c, the designs are nested whereas in panels 4.3b and 4.3d, the designs are non-nested.

Figure

  Figure 4.4: Sobol indices of the objective function (4.5).

4 . 4 :

 44 Figure 4.4: Sobol indices of the objective function (4.5).

Figure 4 . 5 :Figure 4 . 6 :

 4546 Figure 4.5: On left panel, diagram of the car engine cooling system. On right panel, diagram of fan system.[Valeo, ]

5 Figure 4 . 7 :

 547 Figure 4.7: On the left panel, visualization of the five sections of the blade. On the middle panel, definition of the stagger angle and the chord length at a given section. On the right panel, definition of the sweep at section 5.

Figure 5 . 1 :

 51 Figure 5.1: Illustration of nested (panel 5.1a) and non-nested (panel 5.1b) designs for the 2D example of subsection 3.1.2 in chapter 3. X 1 (resp. X 2 ) is represented by black points (resp. red points). The nullity subspace x 2 = 0.7x 1 + 0.2 is represented by the black line.

2

 2

2

 2

Figure 5 . 3 :

 53 Figure 5.3: 3D representation of the nested DoE's: X 1 in black and X 2 in red. The red plans show that X 2 is nested in X 1 as it shares the same values of x 1 .

∼X 1

 1 goal is to build X 2 such that:• is space filling in [0, 1], has distinct points far from each other. This property is illustrated below. All the points of ∼ X 1 are represented in [0, 1]. A distinction is made between the points coming from X 1 in black, and those coming from X 2 in red.• ∼ X 2 = X 2 is space-filling in [0, 1] 2 , has distinct points far from each other and far from the nullity subspace D = {(x 1 , 0.7x 1 + 0.2), x 1 ∈ [0, 1]}. This design is illustrated below with red points, their distance to the nullity subspace is shown in red arrow.

.

  It is found using algorithm 1 in appendix 9.4.2. The two designs are illustrated below. Initial design: Φ 2 (X 2 ) = 42.45

  algorithm 2 in appendix 9.4.2. The two designs are shown below. Initial design: Φ 2,1 (X 2,I 1 ) = 16.26

Figure 5 . 4 :

 54 Figure 5.4: X 1 is represented by the black points. It is the sample from step 1 when x 2 = 0.7x 1 + 0.2 and x 3 = x 1 +0.7x 1 +0.22

Figure 5 . 5 :

 55 Figure 5.5: X 1 is represented by the black points. It is the sample from step 1 when x 3 = x 1 +x 2 2

. 1 .Figure 6 . 1 :

 161 Figure 6.1: Illustration of the designs in the subspaces at step n -1 in panel 6.1a, and at step n in panel 6.1b. Here, there are three subspaces at step n -1, so K n = 3, and there are two subspaces at step n, so K n+1 = 2.

Figure 6 . 2 :

 62 Figure 6.2: Location of the enrichment points of the 100 studies on the analytic 4D test case using the seqGPR metamodel based on the training samples X 1 in 2D of size 10 and X 2 in 4D of size 20. The enrichment points in left panel are choosen at random, those on middle panel are choosen using the criterion defined in (6.5), and those on the right panel are choosen using the prediction variance.

Figure 6 . 3 :

 63 Figure 6.3: Location of the enrichment points of the 100 studies on the analytic 4D test case using the seqGPR metamodel based on the training samples X 1 in 2D of size 20 and X 2 in 4D of size 40. The enrichment points in left panel are choosen at random, those on middle panel are choosen using the criterion defined in (6.5), and those on the right panel are choosen using the prediction variance.

Figure 6 . 4 :

 64 Figure 6.4: Location of the enrichment points of the 100 studies on the analytic 15D test case using the seqGPR metamodel based on the training samples X 1 in 3D of size 15, X 2 in 9D of size 45, and X 3 in 15D of size 75. The enrichment points in left panel are choosen at random, those on middle panel are choosen using the criterion defined in(6.5), and those on the right panel are choosen using the prediction variance.

Figure 6 . 5 :

 65 Figure 6.5: Visualization of -x i (x i -1), the term associated to the component x i of the new factor.

Figure 7 . 1 :

 71 Figure 7.1: Diagrams of the car engine cooling system and its radiator.[Valeo, ]

Figure 7 . 2 :Figure 7 . 3 :α 4 +α 5 2 = α 1 +α 2 2 ,Figure 7 . 4 :

 72732274 Figure 7.2: Diagrams of the exchange surface air-fluid and of the elementary pattern.[Valeo, ]

  localisé dans une zone précise de l'espace des entrées. Il contient les valeurs des variables(x I 1 , • • • , x In ) tandis que les variables (x I n+1 , • • • , x I N ) sont fixées aux valeurs (x I n+1 , • • • , xI N ) (cesvaleurs sont imposées, elles peuvent être constantes ou dépendre de façon déterministe des variables libres). Le vecteur I n contient les indices des variables libérées à l'étape n. Pour prendre en compte cette distribution particulière des plans d'expérience, la modélisation suivante est proposée. La fonction f est supposée être la réalisation d'un certain processus Gaussien Y N défini récursivement par:

X 3

 3 and ∼ must have distinct points sufficiently far from each other. ∼ X 2 (respectively ∼ X 3 ) should have points sufficiently far from the nullity subset

Φ 2 (

 2 X 2 ) = 11.95.

  . 0 λ q 0 . . . 0 0 . . . . . . 0 0 . . . 0 . . . . . . . . . . . . 0 . . . . . . 0 0 . . . 0

  They can be considered as functions of s defined on S. The Karhunen-Loeve decomposition of ∼ Z (and so of P ( ∼ Z)) on D is:

  . . , x(L) denote the elements of D. D is indexed by S = {1, . . . , L} of measure ν = L i=1 ω i δ i , with δ i the Dirac measure of integer i and ω i > 0 the weight associated with i. X = R d is provided with the scalar product (x, y) W = x W y with W =

∼φ:

  S → R is an eigen function of the problem and can be seen as an eigen vector of the matrix T W , by denoting it as ∼ φ(S).

  that form an orthonormal basis of X = R d for the scalar product (, ) W . Let denote by (λ n ) L n=1 the associated eigen values of T W , the spectral decomposition of T W is T W = P ΛP -1 where

. 9 )

 9 with r I (B, B) = (r I (b i , b j )) 1 i j K the covariance matrix composed of the r I (b i , b j ), and ∼ φ n (•, B) the column vector whose components are the functions x J → ∼ φ n (x J , b i ) .

INTRODUCTION

  Specifications in automotive applications can cover a wide range of operating points since design, architecture and engine power are very different from one vehicle model to the other. Each new development for a fan involves therefore a new optimization process to reach nominal pressure rise and torque. Typical pressure and efficiency curves versus flow rate are presented in figure1with indications on operating points.

Figure 1 :

 1 Figure 1: pressure and efficiency curves

2

  ance kernel of Z correlation kernel of Z likelihood function of m, σ^2, and θ training point, or i th test point matrix of all training points, also called design matrix, its lines are the x^((i)) tor of output values associated with training points, or with test points correlation matrix of the training points correlation vector between x and the training points usual function applied to a design variable Mean Square of the prediction Error of the metamodel on a test sample umber of test points umber of training points omputer code value riging prediction, also called Kriging mean riging prediction, also called kriging variance Specifications in automotive applications can cover a wide range of operating points since design, architecture and engine power are very different from one vehicle model to the other. Each n involves therefore a new optimization process to reach nominal pressure Typical pressure and efficiency curves versus flow rate are presented in figure 1 with indications on operating points.

Figure 1 :

 1 Figure 1: pressure and efficiency curves with operating points

  lines are the x^((i)) tor of output values associated with training points, or with test points Mean Square of the prediction Error of the metamodel on a test sample Specifications in automotive applications can cover a wide range of operating points since design, architecture and engine power are very different from one vehicle model to the other. Each n involves therefore a new optimization process to reach nominal pressure Typical pressure and efficiency curves versus flow rate are presented in figure 1 with operating points

Figure 2 :

 2 Figure 2: typical automotive fan

Figure 4 :

 4 Figure 4: Simulation domain

Figure 5 .

 5 Figure 5. Added points with the maximal variance in [-1, 1] for ΔP.

Figure 7 .

 7 Figure 7. Comparison of the methods on the mean of variance criterion.

Figure 8 :

 8 Figure 8: Examples satisfying the stagger angle constraints (red) or not satisfying them (blue, geometry from fig.3).

Figure

  Figure satisfying the chord length constraints satisfying geometry from fig.3

Figure 10 :

 10 Figure 10: solutions when using ΔP meta models with constant trend (first optim.)

•

  At step 1, a first series of computer code evaluations is run. The set x I1 of the first d 1 components of x, is free in [0, 1] d1 . Different values are explored. They are stored in DoE X 1 ⊂ [0, 1] d1 . The other components are fixed to preset values xI2∪•••∪I N (entirely determined by x I1 ). Let f 1 denote the corresponding restriction of f on the subspace [0, 1] d1 . The function f 1 is assumed to be the realization of a stationary Gaussian processY 1 = m + Z 1 of mean m ∈ R and with Z 1 : Ω × [0, 1] d1 → R, a centered Gaussian Process of covariance kernel σ 2 1 ρ 1 . Let y 1 = f 1 (X 1) represent the vector of observations of the output on DoE X 1 .• At step 2, a second range of simulations is then launched on a subspace of a higher dimension [0, 1] d1+d2 .The variables x I2 , fixed at step 1, are released. Different values of x I1∪I2 , stored in DoE X 2 , are explored.The other variablesx I3∪•••∪I N are fixed to the new set of values xI3∪•••∪I N ∈ [0, 1] d3+•••+d N . Let f 2 be the corresponding restriction of f on the subspace [0, 1] d1+d2 .The function f 2 is assumed to be the realization of a Gaussian process Y 2 : Ω × [0, 1] d1+d2 → R, which is the sum of the process at step 1 Y 1 , and a correction term Z 2 , which represents the additional information provided by the released variables. As f 1 and f 2 are restrictions of the same function f , Y 1 and Y 2 have to coincide on the subspace defined at step 1. This results in the following definition of

[0, 1 ]

 1 d1+•••+dn . The variables x I1∪•••∪In-1 and x In are free and the others xIn+1∪•••∪I N are set to xIn+1∪•••∪I N ∈ [0, 1] dn+1+•••+d N .The corresponding restriction of f on this subspace, f n , is evaluated on X n . The values are stored in y n = f n (X n ). The function f n is modeled as the realization of a Gaussian process Y n : Ω × [0, 1] d1+•••+dn → R. Following the same arguments as in step 2, Y n is defined as:Y n (x I1∪•••∪In-1 , x In ) = Y n-1 (x I1∪•••∪In-1 ) + Z n (x I1∪•••∪In-1 , x In ), ∀(x I1∪•••In-1 , x In ) ∈ [0, 1]

1 Figure 1 :

 11 Figure 1: Illustration of the model for N = 2. f is the realization of Y 2 (see panels on the left and center). On the cross section of equation x I2 = xI2 , Y 2 = Y 1 (see panels on the left and right)

1 Figure 2 :

 12 Figure 2: Illustration of the model for N = 2 (see figure 1), when xI2 is constant.

∼

  Z(x 1 , x 2 ) (see panel 3b) in the whole input space [0, 1] 2 . On the contrary, the conditional expectationE ∼ Z(x 1 , x 2 ) | ∼ Z(t 1 , 0)∀t 1 ∈ [0, 1] (see panel 3c), usedto build the P process, depends on x 2 , especially near the line x 2 = 0, and tends to 0 far from the line. The modification it implies on the latent process is therefore located almost only along the line.

  (a) Value of latent process on the border x 2 = 0 (b) Initial latent process (c) Conditional expectation of latent process

Figure 3 :

 3 Figure3: On middle panel, a realization of the latent process from which are built the two candidates for Z 1 that should be null on the line x 2 = 0. On left panel, the process which is removed from the latent process to create the Red process. On right panel, the process (denoted by E for expectation) which is removed from the latent process to create the P process.

Y

  N -1 (X N -1 ) = y N-1 , Y N (X N ) = y N ,

  data. The maximum likelihood estimator is used. The following loss function 1 has to be minimized:l(η; y) = log |Cov η (Y, Y)| + (y -E η [Y]) Cov η (Y, Y) -1 (y -E η [Y]),with for any matrix M , M denoting the transpose of M . This optimization problem is complex as η can reach big dimensions.

Figure 5 :Figure 6 :

 56 Figure 5: Boxplots of the metamodel RMSE's on 100 studies, for the analytic example in 4D. In abscissa are the sizes of the training samples.The following sizes are tested from left to right: X 1 of size 10 and X 2 of size 5 (10pts-5pts), X 1 of size 10 and X 2 of size 10 (10pts-10pts), X 1 of size 10 and X 2 of size 20 (10pts-20pts), X 1 of size 20 and X 2 of size 10 (20pts-10pts), X 1 of size 20 and X 2 of size 20 (20pts-20pts), X 1 of size 20 and X 2 of size 40 (20pts-40pts)

  Chord length and stagger angle at a given section (c) Definition of the sweep at section 5

Figure 8 :

 8 Figure 8: On the left panel, visualization of the five sections of the blade. On the middle panel, definition of the stagger angle and the chord length at a given section. On the right panel, definition of the sweep at section 5. K_tot P_rob Red_rob Median (•10 -3 ) 44.585 41.207 42.088 Interquartile range (•10 -3 ) 5.892 4.657 4.609

  ) = 0 .The process approximating Z P is the conditioned (on a finite set of points) Gaussian process Z D . It is a centered Gaussian process of covariance kernel σ 2 ρ D with:ρ D (x, x ) = r(x, x )r(x, D)r(D, D) -1 r(D, x ) ∀x, x ∈ [0, 1] d J +d I .

Z

  P (x J , x I ) = ∼ Z(x J , x I ) -+∞ n=1 φ n (x J , x I ) ∼ φ n (s J ) ∼ Z(s J , g(s J ))ds J , ∀(x J , x I ) ∈ [0, 1] d J +d I , = ∼ Z(x J , x I ) -+∞ n=1 ∼ φ n (x J )r I (x Ig(x J ), 0)) ∼ φ n (s J ) ∼ Z(s J , g(s J ))ds J , = ∼ Z(x J , x I ) -+∞ n=1 ∼ φ n (x J ) ∼ φ n (s J ) ∼ Z(s J , g(s J ))ds J r I (x Ig(x J ), 0)) ,and, because ∼ Z(x J , g(x J )) belongs to the sub Gaussian space engendered by the{ ∼ Z(s J , g(s J )) ∀s J ∈ [0, 1] d J }, its projection in this subspace is equal to itself: ∼ Z(x J , g(x J )) = E[ ∼ Z(x J , g(x J )) | ∼ Z(s J , g(s J ))∀s J ], ∀x J ∈ [0, 1] d J , x J ) r I (g(x J )g(x J ), 0)) J , g(t J ))ds J , = +∞ n=1 ∼ φ n (x J ) ∼ φ n (s J ) ∼Z(s J , g(s J ))ds J .

  Industrial motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.2 Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.3 Problematics and plan of the thesis . . . . . . . . . . . . . . . . . . . . . . .2 State of the art 2.1 Generalities on the kriging metamodel . . . . . . . . . . . . . . . . . . . . . 2.1.1 From Gaussian vectors to Gaussian process regression . . . . . . . . . 2.1.2 Maximum Likelihood estimation and EM (Expectation -Maximization) algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.1.3 Multifidelity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.1.4 Sobol index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.1.5 Design of Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . 2.2 Kriging under constraints . . . . . . . . . . . . . . . . . . . . . . . . . . . . Candidates for the correction processes . . . . . . . . . . . . . . . . . . . . . 3.2.1 Red (Reduced) process . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2.2 Psi process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2.3 P (Preconditioned) process . . . . . . . . . . . . . . . . . . . . . . . . 3.2.4 Example in 2D . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3 Qualitative comparison of the processes . . . . . . . . . . . . . . . . . . . . . 3.3.1 Shapes of the paths . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3.2 Influence of σ 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3.3 Influence of θ 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3.4 Influence of θ 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3.5 Influence of δ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.4 Estimation of the parameters . . . . . . . . . . . . . . . . . . . . . . . . . .

	Contents
	1 Introduction 1.1

2.2.1 Imposing the constraint a posteriori . . . . . . . . . . . . . . . . . . . 2.2.2 Imposing the constraint a priori with a conditional GP . . . . . . . . 3 Probabilistic model 3.1 Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.1.1 General formalism . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.1.2 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.1.3 Metamodel seqGPR (Sequential Gaussian process regression) . . . . . 3.2 3.4.1 Psi Likelihood . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.4.2 Red Likelihood . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.4.3 P Likelihood . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.4.4 Comparison of the estimations on a 2D example . . . . . . . . . . . . 3.5 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

•

  At step 1, a first series of computer code evaluations is run. The set x I 1 of the first d 1 components of x, are free in [0, 1] d 1 . Different values are explored. They are stored in DoE X 1 ⊂ [0, 1] d 1 . The other components are fixed to preset values xI 2 ∪•••∪I N (entirely determined by x I 1 ). Let f 1 denote the corresponding restriction of f on the subspace [0, 1] d 1 . The function f 1 is assumed to be the realization of a stationary Gaussian process Y 1

  d 1 +d 2 . The variables x I 2 , fixed at step 1, are released. Different values of x I 1 ∪I 2 , stored in DoE X 2 , are explored. The other variables x I 3 ∪•••∪I N are fixed to the new set of values xI 3

  which is the sum of the process at step 1 Y 1 , and a correction term Z 2 , which represents the additional information provided by the released variables. As f 1 and f 2 are restrictions of the same function f , Y 1 and Y 2 have to coincide on the subspace defined at step 1. This results in the following definition of Y 2 :

Table 4 .

 4 5

		Red	P	Red	P	Red	P	Red	P	Red	P	Red	P
	Median Interquartile range	0.41 0.42 1.00 1.04 0.50 0.54 0.51 0.53 0.62 0.95 0.20 0.23 1.07 1.22 0.19 0.17 0.22 0.24 -0.18 -0.10 0.31 0.4 0.94 0.79

1: Estimations of the seqGPR parameters in case Z 2 is a P or Red process. Results are shown in terms of median and interquartile range of estimation. The best results are in bold.

  versions of seqGPR are compared. P_full and P_rob (resp. Red_full and Red_rob) use P (resp. Red) processes defined in section 3.2.3 (resp. 3.2.1) in chapter 3, respectively of type Full and Robust for the (Z n ) N n=2 (the types are developed in subsection 4.2.1). The metamodels are compared in terms of RMSE on a test sample. The RMSE formula is recalled below:

  9 of size 45, and X 3 ⊂ [0, 1] 15 of size 75. A Sobol sequence of size 10000 in dimension 15 is used as a test sample. The study is done 100 times, each time with new DoE's.The performances of the metamodels are shown in table 4.2. All versions of seqGPR are better than the classic kriging, with a slight improvement for the Red version in comparison with the P version. Red and P are almost equivalent as opposed to the previous test case.

Table 4 .

 4 2: Performance of the metamodels for the 100 studies made on the analytic 15D test case. The performances are shown in terms of median of RMSE's and interquartile range (q 75%q 25% ). The best performances are in bold.

	K_tot P_rob Red_rob 46.039 37.135 36.903 Interquartile range (•10 -4 ) 7.258 Median (•10 -4 ) 6.660 6.570

Table 4 .

 4

	K_tot P_rob Red_rob 44.585 41.207 42.088 Interquartile range (•10 -3 ) 5.892 Median (•10 -3 ) 4.657 4.609

3

shows performances of the different metamodels on the test samples. The metamodel seqGPR is better than K_tot. Its version with P is this time slightly better than with Red, again probably because the fixed inputs are constant as in the previous test case.

Table 4 .

 4 3: Performance of the metamodels for the 100 studies made on the industrial 15D test case. The performances are shown in terms of median of RMSE's and interquartile range (q 75%q 25%

  1 tackles the case where multiple designs are generated at a given step for different values of the fixed variables. This issue implies a new model than (3.1) as this previous model concerns only the case with one design per step, associated with one value of the fixed variables. The goal of this section is to build this new model and see if it performs as well as the classic kriging model on the test cases of chapter 4. Section 6.2 deals with the enrichment of the training samples of the seqGPR metamodel.

  Test case from subsection 4.2.2 in chapter 4 The output is the same as in subsection 4.2.2 in chapter 4: 

	1 1 , y 1 1 ),
	(X 2 1 , y 2

1

) and (X 2 , y 2 ). The covariance kernels of K_tot, Z 1 , and

∼ Z n N n=2

(the latent processes on which are built (Z n ) N n=2 ) are tensor product Matern 5 2 kernels, defined in subsection 2.1.1 in chapter 2.

Table 6 .

 6 1 shows RMSE median and interquartile range of the different methods computed on a Sobol sequence of size 10000 used as a test set. The results are computed for 100 DoE's.

Table 6 .

 6 1: Performance of the metamodels for the 100 studies made on the analytic 4D test case. The performances are shown in terms of median of RMSE's and interquartile range (q 75%q 25% ). The best performances are in bold.

	51.131 Interquartile range (•10 -1 ) 16.037 -1 )		55.499 23.102	55.529 22.741
		45pts-45pts/45pts	45pts-45pts/75pts	90pts-90pts/150pts
		K	S	R	K	S	R	K	S	R
	Median (•10 -4 ) Interquartile range (•10 -4 )	62.014 62.464 62.527 39.287 39.645 39.634 7.605 7.622 7.620 7.025 6.343 6.342 6.709 6.879 6.939 0.594 0.599 0.599

Table 6 .

 6 2: Performance of the metamodels for the 100 studies made on the analytic 15D test case. The performances are shown in terms of median of RMSE's and interquartile range (q 75%q 25%

Table 6 .

 6 3: Performance of the metamodels for the 100 studies made on the analytic 4D test case. The performances are shown in terms of median of RMSE's and interquartile range (q 75%q 25% ). I denotes the initial metamodel seqGPR, R denotes the metamodel enriched with random points, V denotes the metamodel enriched by maximum variance, and C denotes the metamodel enriched with maximum criterion (defined in subsection 6.2.1). 10pts/20pts means that X 1 is of size 10 and that X 2 is of size 20. 20pts/40pts means that X 1 is of size 20 and that X 2 is of size 40. The best performances are in bold.

		10pts/20pts			20pts/40pts
	I	R	V	C	I	R	V	C
	42.108 31.915 40.079 38.041 7.292 4.586 4.044 3.916 Interquartile range (•10 -2 ) 9.588 Median (•10 -2 ) 5.870 13.331 15.651 2.100 1.360 1.050 0.847

2.3 in chapter 4.

The same conclusion can be drawn from all the figures. The enrichment points choosen at random are unsurprisingly uniformly spread in

  . g 1 est la fonction six-hump Camel redimensionnée dans [0, 1] 2 . C'est une fonction assez chahutée. g 2 est une gaussienne beaucoup plus lisse. L'étude réalisée sur cette fonction est composée de deux étapes : • A l'étape 1, les variables (x 1 , x 2 ) sont libres et les variables (x 3 , x 4 ) sont fixées aux valeurs ( x 1 +x 2 2 , 0.2x 1 + 0.7). • A l'étape 2, toutes les variables sont libres. Des plans de différentes natures sont générés durant ces deux étapes : imbriqués ou nonimbriqués, et de différentes tailles. Plusieurs résultats sont montrés par ce cas test. D'abord, les plans non-imbriqués donnent de meilleurs résultats à taille fixée. Ensuite, la version robuste du métamodèle seqGPR est meilleure que la version full et que le krigeage classique. Enfin, la version Red est plus performante que la version P. De ces conclusions sont tirées deux mesures appliquées dans la suite : l'utilisation de plans non-imbriqués et de la version robuste du métamodel seqGPR.

  redimensionnées dans [0, 1] 15 . Cette fois-ci, une étude de trois étapes est réalisée : libérant les trois premières variables, puis les 6 suivantes, et enfin les 6 dernières. Des plans d'entraînement X 1 , X 2 , X 3 sont générés, respectivement de taille 15, 45, et 75. A chaque étape, les variables fixées sont prises de valeur 0.5. Les mêmes conclusions sont tirées par rapport au cas test précédent : à savoir que le métamodèle seqGPR est meilleur que le krigeage classique, et que la version Red est meilleure que la version P. les autres comme ensemble test. 30 évaluations des métamodèles sont réalisées (avec des X 1 et X 2 différents à chaque fois). Encore une fois, le métamodèle seqGPR a de meilleurs résultats de prédiction que le krigeage classique, cette fois-ci avec un léger avantage pour la version P.Dans cette sous-section, les études changent légèrement de forme. A chaque étape n ∈ 1, N -1 , plusieurs plans sont générés dans différents sous-espaces :X 1 n , • • • , X K n+1 n. Ces K n+1 sous-espaces sont liés par le fait que les variablesx I n+2 , • • • , x I N sont fixées aux mêmes valeurs : xI n+2 , • • • , xI N (qui dépendent des variables x I 1 , • • • , x I n+1 ),et que d'un autre côté, les valeurs des variables x I n+1 dans ces sous-espaces diffèrent d'une constante b i n+1 , valant xI n+1 + b in+1 dans le sous-espace i ∈ 1, K n+1 . La dernière étape N est inchangée, où un unique plan X N est généré dans l'espace tout entier. Cette multiplicité des sous-espaces à étape fixée oblige à une modification de la modélisation dans laquelle les processus des étapes intermédiaires doivent prendre en compte un groupe de variable supplémentaire:

	8.5 Tentatives d'approfondissement de la méthode
	8.5.1 Conditionnement multiple
	8.4.3 Cas test industriel en dimension 15

Le dernier cas test, qui est celui ayant motivé cette thèse, comporte également 15 entrées. Une étude de deux étapes est réalisée : dans la première, les 11 premières variables sont libres et les 4 dernières sont fixées à une constante, dans la deuxième, toutes les variables sont libres. Un plan est réalisé à l'étape 1, de taille 126, et un autre à l'étape 2, de taille 299. L'évaluation des métamodèles est réalisée sous la forme d'une validation croisée. X 1 prend 50 points du plan généré à l'étape 1, tandis que X 2 prend 50 points parmi ceux générés à l'étape 2, laissant

  de sorte que xIn∪I n+1 + B n = xIn + B n , xI n+1 .

L'évaluation de cette adaptation du métamodèle seqGPR sur les cas test précédents ne révèle aucune amélioration par rapport au krigeage classique dans ce cas. Sans doute la complexité trop élevée du modèle n'est cette fois-ci pas compensée par son adaptation au format des données d'entraînement.

8.5.2 Enrichissement des plans

La deuxième piste d'approfondissement est l'enrichissement des plans d'expérience, dans le cadre de plans non-imbriqués. L'enrichissement se fait en sélectionnant un point solution d'un certain problème d'optimisation. Le choix se porte sur le problème d'optimisation suivant :

Table 1 :

 1 Error assessment of the meta-model in the whole domain Accuracy assessment in the Middle of the Domain Table2shows the results obtained with the same models than table 1, but evaluated in the domain [-0.5, 0.5] 11 . Once again, the linear trend provides a rather good improvement of the RMSE of 1 to 3 Pa. Comparing the two sampling methods, it shows this time a noticeable difference: the geometrical rules even deteriorate the meta-model globally. However it does not state if it is relevant for an optimization which is supposed to go only in areas of interest for turbomachine design.

	ΔP	RMSE on T
	With	With linear
	constant trend	trend

Table 2 :

 2 Error assessment of the meta-model in the middle of the domain

	OPTIMIZATION
	The meta-models are then compared
	 Maximize ΔP, T=<1N.m, Q set at a given nominal flow rate
	 Minimize T, ΔP>=250 Pa, Q set at a given nominal flow rate
	Figures 10 and 11

  d1+

  ••+dn , with Z n a centered Gaussian process independent from (Z 1 , • • • , Z n-1 ) of covariance kernel σ 2 n ρ n and such that Z n

  d1+

equal to twice the negative log-likelihood up to a constant

This paper was written in the context of an ANR project called PEPITO which focused on sampling strategies for metamodels applied on industrial issues, and more precisely on the fan system of Valeo. The paper in itself tackles the definition of the geometric parameters, the construction of the input space controlled by geometric constraints, and the enrichment strategy of the training samples for the metamodels.

This submitted paper deals with the main work of this thesis, which is to build a metamodel based on Gaussian process regression, which takes into account different DoE's, defined on nested subspaces of the input space, of increasing dimension.

Remerciements

ACKNOWLEDGMENTS

This work is being supported by the French Research Agency ANR [12], for the project PEPITO ("Plan d'Experience Pour l'Industrie du Transport et l'Optimisation")

Software and Acknowledgments

Implementation have been done in R, where all the kernels and methods of estimation and prediction have been implemented using the package RcppArmadillo.

This work, carried out at Ecole Centrale de Lyon, was financially subsidized by Valeo. The authors wish to show their particular gratitude to Mr. Manuel Henner and Mr. Nicolas Yoan François for their support and fruitful discussion on the industrial point of view. The authors are grateful to the Members of GdRMascotnum for their useful advice, and in particular to Mr. Olivier Roustant and Mr. Xavier Bay for constructive discussions on the subject on the theoretical point of view. The authors would also like to acknowledge Mr. Philippe du Bouays and Ms. Laura Gay for their careful proofreading.

with:

1 , 0.4, 0.5,

1 , 0.4, 0.5,

1 +0.9 4

1 +x

1 +x

Using the expectation of a quadratic form formula to Gaussian vectors:

)

Partial analytical solution The optima m (i+1) and σ (i+1) n

(n ∈ 1, N ) have analytical forms obtained by solving the system formed when the corresponding partial derivatives of the Q n vanish. Finally, at each new iteration i + 1, the goal is to find θ

with l 1 (η 1 ; y 1 ) (respectively l n (η n ; z n ), n ∈ 1, N ) the loss function associated with the training data Y 1 (X 1 ) = y 1 (respectively

Proof (z n ) 1 n N is observed because the designs are nested. The loss function can be decoupled because

If all the designs are nested, the parameters η n can be estimated separately by optimizing the loss functions l n .

Non nested designs

If the designs are not nested, the loss function is not decoupled. An EM (Expectation-Maximization, see [Friedman et al., 2001]) algorithm is then used to reduce the dimension of the optimization problem. To define this algorithm, the notion of complete data is introduced. In the case of not nested designs

According to proposition 3, if the values of Y 1 on ∼ X 1 , denoted by z 1 , and the values of the Z n on the ∼ X n (n ∈ 2, N ), denoted by z n , were observed, the loss function associated to all those training samples, denoted by l c , could be decomposed as

As some of the complete data is not observed (missing data), an EM algorithm seems adequate to optimize the loss function.

Definition 2 (EM algorithm)

The EM algorithm is defined by • Expectation Instead of the observed data loss function, the expectation of the complete data loss function conditioned by the observed data is considered. This quantity can be decomposed in N terms:

• Maximization The EM algorithm consists in building the sequence (η (i) ) i 0 = (η

N ) i 0 such that η (0) is user defined and ∀i 0, η (i+1) is solution of the optimization problem

is solution of the following optimization problem:

See appendix C for the explicit formulas of the Q n . It is known that the sequence (η (i) ) i 0 verifies that (l(η (i) ; y)) i 0 is a decreasing sequence (see appendix D). Finally, η is estimated by a term of the sequence of sufficiently high rank, the choice of the rank being done by imposing a maximum number of iterations and a threshold on the variation of likelihood between iterations.

X n is assumed not having any redundant points such that the matrices have no identical rows. This condition is sufficient for the first matrix to be invertible as long as ρ θ1 is a positive-definite kernel. As it is not the case of the ρ θn (n 2), to ensure that the matrices are not singular, each ∼ X n (n ∈ 2, N ) is also supposed to not contain any point on which Z n is null so there are no rows full of zeros in the matrices.

Examples of application

In what follows, three metamodels are compared on three examples, two analytic functions and one output from an industrial code:

• seqGPR: the metamodel introduced in this paper. The kernel of Y 1 and the ones on which are based (Z n ) N n=2 are stationary tensor product matern 5 2 covariance kernels. • K_N: a classic kriging metamodel with a stationary tensor product matern 5 2 covariance kernel and a constant mean which is built on the last training sample (X N , y N ).

• K_tot: a classic kriging metamodel similar to K_N, but built on all training samples

The philosophy of the method seqGPR is to explain a maximum of the data by Y 1 , and to correct it thanks to the (Z n ) N n=2 , so all the parameters for Y 1 are estimated individually and on the contrary some parameters of the Z n are grouped. Different parameter sparsities for the Z n are tried, to seek balance between fitting and robustness:

• Robust:

common to all components of x I1∪•••∪In-1 , and one parameter θ i n for each component of x In . Four versions of seqGPR are compared. P_full and P_rob (resp. Red_full and Red_rob) use P (resp. Red) processes defined in subsection 3.2 (resp. 3.1), respectively with parameter sparsity of type Full and Robust for the (Z n ) N n=2 . The metamodels are compared in terms of RMSE on a test sample. The training samples are not nested so the EM algorithm is used to estimate the parameters of the 4 versions of seqGPR. A simple likelihood estimation is used for the two kriging models.

Analytic example in dimension 4

The output f considered in this example is a function of 4 inputs

The study is composed of two steps: All metamodel performances improve with the size of the training sample. Results show that including the previous information of (X 1 , y 1 ) is useful, as it greatly improves the performance of the kriging metamodel (K_tot is better than K_2). The robust seqGPR metamodels are better than the full, they are equivalent to or better than K_tot. It seems that the cases with a small or high number of training points are more discriminating than in the middle cases. With a small number of points (10pts-5pts), K_tot seems more destabilized than seqGPR. With a high number of points (20pts-40pts), seqGPR is more accurate than K_tot. Finally, seqGPR using Red are better than seqGPR using P. In the following examples, only the robust versions of seqGPR, which are more performing and whose parameter estimation is less complex, are compared to K_tot.

Analytic example in dimension 15

The following example should be less favorable to seqGPR. The objective function considered is in higher dimension and not decomposed as a sum of functions that respects the order in which the variables are released

The function f was first used in [Oakley and O'Hagan, 2004]. The values of its coefficients a 1 , a 2 , a 3 and M can be found in www.sheffield.ac.uk/st1jeo. The inputs are rescaled in [0, 1] and are rearranged by decreasing order of Sobol index.

K_tot P_rob Red_rob Median (•10 -4 )

46.039 37.135 36.903 Interquartile range (•10 -4 ) 7.258 6.660 6.570

Table 1: Performance of the metamodels for the 100 studies made on the analytic 15D example. The performances are shown in terms of median of RMSE's and interquartile range (q 75%q 25% ).

(a) Car engine cooling system (b) Fan system In practice, the first studies are done on the most influential inputs, chosen according to experts' physical knowledge. Here, as the function is analytical, the choice of the releasing order for the inputs is made by sensitivity analysis, which is directly performed on the function. Following the distribution of the Sobol indices (see figure 6), a study of N = 3 steps is made, adding respectively the groups of variables I 1 = {1, 2, 3}, 5, 6, 7, 8, 9}, and 11, 12, 13, 14, 15}. The variables that are fixed are set to 0.5. Three training samples, one for each step, are generated with 5 points per dimension considered at the step: The performances of the metamodels are shown in table 1. All versions of seqGPR are better than the classic kriging, with a slight improvement for the Red version in comparison with the P version. Red and P are almost equivalent as opposed to the previous example. This can be due to the fact that the P version disturbs less the initial stationary kernel because the variables which are fixed at a given step are set to a constant instead of varying in function of the free variables, as it is the case in the previous example.

Industrial example: fan system in dimension 15

The industrial product which is studied in this section is the fan system which is part of a car engine cooling system from Valeo company (see figure 7). This cooling system is composed of a cold fluid circulating in part in the car engine to regulate its temperature and in part in a radiator where it is itself cooled down. When the car moves, the wind generated by the car speed and reaching the radiator is sufficient to evacuate the heat from the fluid. When the car is motionless, the fan is activated to replace the wind.

The output considered here is the Pressure difference (∆P ) between the upstream and the downstream of the air flow crossing the fan. It is function of 15 input variables. One input is the flowrate (Q). The 14 others are geometric. The fan blade geometries are supposed identical with each other. The geometry is monitored at 5 different sections (see figure 8a). At each section, the stagger angle and the chord length are controlled (see figure 8b). The chord is the line formed by the two borders of the blade at the considered section. The stagger angle is the angle between the chord and the rotation axis. The sweep is manipulated at each section. It is defined as the distance between the line formed by the rotation axis and the right border of the first section and the right border of the considered section (see figure 8c). At section 1, it is always equal to 0 and therefore not kept as an input.

To summarize, the variables are

• The flowrate: Q

• The stagger angles at the five sections:

Appendix A Proof of proposition 1

This section deals with the proof of proposition 1, which gives an approximation of the process Z P (defined in equation ( 5)) by discretizing its spectral decomposition.

The following eigenvalue problem is considered:

It is discretized using a Monte-Carlo method. A sample (s

Discretizing in x J with D, the eigen problem becomes a finite dimensional one:

The solutions of (9) are noted (γ n , V n ) 1 n L . The V n are taken such that they verify:

The discretization of

C Formulas of the EM procedure

This section gives the formulas of the Q n involved in the optimization problems (7) used to estimate the parameters η n in the EM algorithm. 1∼ X1 is the unit column vector of size nrows ∼ X 1 . Denoting by X 1 and X 2 two DoE's, 0 X 1 ,X 2 is the null matrix of size nrows(X 1 ) × nrows(X 2 ).

Using the expectation of a quadratic form formula to Gaussian vectors:

Partial analytical solution The optima m (i+1) and σ (i+1) n

(n ∈ 1, N ) have analytical forms obtained by solving the system formed when the corresponding partial derivatives of the Q n vanish. Finally, at each new iteration i + 1, the goal is to find θ

, where E n (η (i) ) and C n (η (i) ) (n ∈ 1, N ) are given in equation [START_REF] Zhang | Meta-model based optimization of a large diameter semi-radial conical hub engine cooling fan[END_REF].

D Proof of the monotonicity of the sequence from the EM algorithm

. Let y, u, and t be the realizations of respectively Y, U, and T. By definition of the conditional density: The difference between the loss function taken at the updated and previous terms of the EM sequence is the sum of two quantities.

l(η (i+1) )l(η (i) ) = Q(η (i+1) , η (i) ) -Q(η (i) , η (i) ) 0 +2 R(η (i+1) , η (i) ) -R(η (i) , η (i) )

=R .

The first quantity is negative by definition of the EM algorithm. The second quantity can be rewritten as: Indeed the inequality l(η (i+1) )-l(η (i) ) 0 is verified by the sequence η (i) i built following the EM algorithm.