Keywords: Obtained prediction errors. . . . . . . . . . . meta-model, reproducing kernel Hilbert space, ridge group sparse, risk upper bound

Before supervising my thesis, Marie-Luce Taupin was my professor when I was in Master 2. It is actually thanks to her that I met Sylvie Huet and started under their supervision an internship in the unité MaIAGE of INRA in Jouy en Josas. Working with them and along with the rest of the MaIAGE members, inspired me to pursue my studies in this direction, and I am grateful to them for allowing me to do so. I would like to thank my supervisors Sylvie and Marie-Luce for their support, their reactivity and their trust. I have learned a lot from them, both scientifically and in terms of human relations in the world of research.

When I started my internship in MaIAGE, I was really welcomed and I felt as well all along my three first years of thesis. I am grateful that I had the opportunity to discover the members of the unité LaMME in Evry almost in the last year of my thesis. A great thank you to all the memebers of unité MaIAGE and LaMME. I would like to also thank all the PhD students of MaIAGE and LaMME for having so kindly integrated me into their little teams.

Lastly, I would like to thank my husband Rebaz

Chapter 1

Introduction

Cadre de travail

Considérons un phénomène décrit par un modèle m dépendant de d variables d'entrée X = (X 1 , ..., X d ). Ce modèle m de R d vers R peut être complexe, présenter de fortes non-linéarités et des effets d'interaction d'ordre élevé. Dans le cadre classique de l'analyse de sensibilité, le modèle m peut être calculé en un nombre fini de points.

Lorsque les coordonnées de X sont indépendantes, le modèle m peut se décomposer selon la décomposition dite de Hoeffding. Quand la loi des coordonnées de X est connue, la décomposition de Hoeffding de m permet d'effectuer l'analyse de sensibilité, et plus précisément de calculer les indices de Sobol de m [START_REF] Sobol | Global sensitivity indices for nonlinear mathematical models and their Monte Carlo estimates[END_REF], [START_REF] Saltelli | Sensitivity Analysis[END_REF]). Cependant, le calcul de ces indices peut être très difficile, voire impossible, surtout lorsque le nombre de variables d'entrée d est grand [START_REF] Iooss | Revue sur l'analyse de sensibilité globale de modèles numériques[END_REF]).

Une approche récente consiste à approcher m par un méta-modèle additif impliquant les coordonnées de X ainsi que leurs interactions, comme proposée par [START_REF] Durrande | Anova kernels and rkhs of zero mean functions for model-based sensitivity analysis[END_REF]. Ce méta-modèle, noté f * , est la projection orthogonale de m sur un espace de Hilbert à noyau auto-reproduisant (RKHS), noté H. L'espace H est associé à un noyau dite d'ANOVA qui est défini de façon à obtenir l'expression analytique des termes de la décomposition de Hoeffding des fonctions de H. Comme f * est la projection orthogonale de m sur H, chaque terme de sa décomposition est une approximation du terme associé de la décomposition de Hoeffding de m.

Lorsque le nombre de variables d'entrée d est grand, le nombre total de termes dans la décomposition de Hoeffding de f * devient très élevé. Une solution consiste à calculer une approximation sparse ou parcimonieuse de f * en utilisant le critère des moindres carrés pénalisé comme dans le modèle de régression non-paramétrique. Sparse ou parcimonieuse au sens où le nombre de termes non-nuls dans la décomposition de Hoeffding de f * est contrôlé.

Dans cette thèse, deux cadres sont considérés: l'analyse de sensibilité où m(X) est calculable en tout point X, et le modèle de régression où m est inconnu et ne peut donc pas être calculé.

Dans le second cas, pour un X donné, m(X) est observable à une erreur près. Ainsi, on dispose de l'observation Y telle que, Y = m(X) + σε, σ > 0.

(1.1)

Comme dans le cadre classique de l'analyse de sensibilité, l'idée est d'approcher la décomposition de Hoeffding de m par le méta-modèle f * , puis de calculer un groupe de variables sur la sortie du modèle. Les objectifs principaux de l'analyse de sensibilité sont la calibration et la validation des modèles ainsi que l'aide à la prise de décision. Les méthodes et objectifs classiques de l'analyse de sensibilité sont décrits dans les ouvrages suivants: [START_REF] Cacuci | Sensitivity and Uncertainty Analysis, Theory[END_REF], [START_REF] Fang | Design and Modeling for Computer Experiments (Computer Science & Data Analysis[END_REF], [START_REF] Dean | Screening: Methods for Experimentation in Industry, Drug Discovery, and Genetics[END_REF], [START_REF] De Rocquigny | Uncertainty in Industrial Practice: A Guide to Quantitative Uncertainty Management[END_REF], [START_REF] Saltelli | Global sensitivity analysis: the primer[END_REF], [START_REF] Helton | Uncertainty and Sensitivity Analysis for Models of Complex Systems[END_REF], [START_REF] Saltelli | Sensitivity Analysis[END_REF], [START_REF] Faivre | Analyse de sensibilité et exploration de modèles, Collection Savoir-Faire[END_REF], [START_REF] Borgonovo | Sensitivity analysis: A review of recent advances[END_REF]. L'analyse de sensibilité repose sur le calcul et l'analyse des mesures qui évaluent l'effet des variables d'entrée sur la sortie du modèle. Par exemple, l'effet d'une variable d'entrée sur la sortie du modèle peut être évalué par la contribution de cette variable d'entrée sur la variance de la sortie du modèle. Les méthodes d'analyse de sensibilité peuvent être classées en deux groupes principaux: L'analyse de sensibilité locale où il s'agit d'étudier l'impact local des variables d'entrée sur la variable de sortie. Elle consiste à calculer le gradient de la variable de sortie par rapport aux variables d'entrée autour d'une valeur choisie (la valeur moyenne des variables d'entrée par exemple). De nombreuses méthodes ont été développées pour calculer efficacement le gradient, notamment la modélisation Adjointe [START_REF] Cacuci | Sensitivity and Uncertainty Analysis, Theory[END_REF], [START_REF] Cacuci | Sensitivity and Uncertainty Analysis: Applications to Large-Scale Systems[END_REF]) et la Différenciation Automatisée [START_REF] Griewank | Evaluating Derivatives: Principles and Techniques of Algorithmic Differentiation[END_REF]). Les méthodes locales n'explorent pas pleinement l'espace des variables d'entrée, mais étudient l'impact de petites perturbations des variables d'entrée (généralement une variable à la fois) sur la variable de sortie.

L'analyse de sensibilité globale où il s'agit de calculer l'incertitude de la variable de sortie due aux variations des variables d'entrée. Contrairement à l'analyse de sensibilité locale, cette classe de méthodes prend en compte toute la gamme de variation des variables d'entrée. Les méthodes d'analyse de sensibilité globale sont nombreuses, voir par exemple [START_REF] Saltelli | Sensitivity Analysis[END_REF] pour un aperçu et [START_REF] Iooss | A review on global sensitivity analysis methods[END_REF] pour une revue de ces méthodes. Généralement, les méthodes de l'analyse de sensibilité globale qui permettent de calculer les mesures de sensibilité quantitatives les plus utilisées peuvent être présentées en deux groupes:

Les méthodes basées sur la régression sont appropriées lorsque le modèle est linéaire, c'est-à-dire si le coefficient de détermination R 2 est proche de un. Les mesures de sensibilité les plus utilisées dans ce cas sont: les coefficients de régression standardisés, les coefficients de corrélation de Pearson, et les coefficients de corrélation partielle. Dans le cas d'un modèle non-linéaire monotone, ces coefficients sont utilisés pour calculer des mesures de sensibilité, après avoir appliqué une transformation en rangs [START_REF] Saltelli | Sensitivity Analysis[END_REF]). Lorsque le modèle est non-linéaire et non-monotone, ces méthodes ne produisent pas de mesures de sensibilité satisfaisantes [START_REF] Saltelli | About the use of rank transformation in sensitivity analysis of model output[END_REF]).

Les méthodes basées sur la décomposition de la variance s'appliquent aux modèles non-linéaires et non-monotones. Il s'agit alors d'effectuer une décomposition de la variance de la variable de sortie. Plus précisément, la variance de la variable de sortie est décomposée en parties attribuables à chacune des variables d'entrée et à leurs interactions. Les mesures de sensibilité sont exprimées comme le rapport de la variance due à chacun de ces groupes de variables (variables individuelles ou interactions de plusieurs variables) à la variance de la variable de sortie. La décomposition de la variance est pertinente si les variables d'entrée sont indépendantes les unes des autres [START_REF] Saltelli | On the relative importance of input factors in mathematical models[END_REF]). Ces méthodes sont largement utilisées car elles permettent d'explorer complètement l'espace des variables d'entrée, en tenant compte des effets d'interactions des variables d'entrée sur le modèle et de la non-linéarité du modèle.

1.1.2 Analyse de sensibilité globale: méthodes basées sur la décomposition de la variance

Considérons un modéle m dépendant de d variables d'entrée X = (X 1 , ..., X d ) qui sont indépendantes et ont une loi connue P X = P 1 ⊗ ... ⊗ P d sur X = X 1 × ... × X d , un sous-ensemble de R d . Le modèle m de R d vers R est de carré-intégrable sur X , c'est-à-dire m ∈ L 2 (X , P X ).

Dans le cadre classique de l'analyse de sensibilité, où pour chaque valeur de X on peut calculer m(X), on peut utiliser la méthode de [START_REF] Sobol | Sensitivity estimates for nonlinear mathematical models[END_REF] brièvement décrite ci-dessous.

L'indépendance entre les coordonnées de X permet d'écrire le modèle m selon sa décomposition de Hoeffding [START_REF] Sobol | Sensitivity estimates for nonlinear mathematical models[END_REF], van (1.2)

Les termes de cette décomposition sont définis en terme d'espérance conditionnelle:

m 0 = E X (m(X)), m a (X a ) = E X (m(X)|X a ) -m 0 , m a,a (X a , X a ) = E X (m(X)|X a , X a ) -m a (X a ) -m a (X a ) -m 0 ,
et ainsi de suite pour les interactions d'ordre supérieur à deux.

Ces termes sont appelés terme constant, effets principaux, interactions d'ordre deux et d'ordre supérieur.

Soit P l'ensemble de tous les sous-ensembles de {1, ..., d} de dimension 1 à d. Pour un ensemble A on note |A| son cardinal. Pour tout v ∈ P et X ∈ X , soit X v le vecteur de composantes X a , a ∈ v et m v : R |v| → R la fonction associée à X v dans l'équation (1.2). L'équation (1.2) peut alors être exprimée comme suit:

m(X) = m 0 + v∈P m v (X v ).
(1.3) Cette décomposition est unique, tous les termes m v , v ∈ P, sont centrés et orthogonaux par rapport à L 2 (X , P X ), c'est-à-dire, ∀v ∈ P, E X (m v (X v )) = 0,

1.1. Cadre de travail et ∀v, v ∈ P, v = v , E X (m v (X v )m v (X v )) = 0.
Étant donné que les termes de la décomposition (1.3) sont centrés, de carré-intégrable et orthogonaux deux à deux par rapport à la distribution de X, la variance de m(X) se décompose comme suit:

var(m(X)) = v∈P var(m v (X v )).

(1.4)

Pour tout groupe de variables X v , v ∈ P, les indices de Sobol sont définis par:

S v = var(m v (X v ))
var(m(X)) .

Pour chaque v, S v exprime la fraction de la variance de m(X) expliquée par X v .

Pour tout v ∈ P, quand |v| = 1, les S v sont appelés indices du premier ordre ou indices des effets principaux. Quand |v| = 2, c'est-à-dire v = {a, a } et a = a , ils sont appelés indices du second ordre ou indices d'interaction d'ordre deux (entre X a et X a ). Et ainsi de suite pour |v| > 2.

Le nombre total des indices de Sobol à calculer est égal à |P| = 2 d -1, qui augmente exponentiellement avec le nombre de variables d'entrée d. Lorsque d est grand, l'évaluation de tous les indices peut être très coûteuse voire même impossible. Pour cette raison, seuls les indices d'ordre inférieur ou égale à deux sont calculés en pratique. Cependant, les indices du premier et du second ordre ne peuvent pas toujours fournir une information sur la sensibilité du modèle. Afin de fournir une meilleure information sur la sensibilité du modèle, [START_REF] Homma | Importance measures in global sensitivity analysis of nonlinear models[END_REF] ont proposé de calculer les indices du premier ordre et les indices d'ordre total définis comme suit:

Soit P a ⊂ P l'ensemble de tout les sous-ensembles de {1, ..., d} incluant a, alors

S Ta = v∈Pa S v .
Pour tout a ∈ {1, ..., d}, S Ta indique l'effet total de la variable X a . Il exprime la fraction de la variance expliquée par la variable X a seule et toute interaction de X a avec les autres variables. Les indices d'ordre total permettent de classer les variables d'entrée selon la quantité de leur effet sur la variable de sortie. Néanmoins, ils ne fournissent pas d'informations complètes sur la sensibilité du modèle comme le font tous les indices de Sobol. Le calcul des indices de Sobol est généralement effectué par les méthodes de Monte Carlo (voir par exemple: [START_REF] Sobol | Sensitivity estimates for nonlinear mathematical models[END_REF] pour les effets principaux et interactions, et [START_REF] Saltelli | Making best use of model evaluations to compute sensitivity indices[END_REF] pour les effets principaux et indices d'ordre total). Ces méthodes sont très coûteuses, car elles peuvent nécessiter le calcul du modèle plusieurs milliers de fois pour obtenir des estimations précises des indices de Sobol. Ainsi dans le cas où d est grand, m est complexe et où le calcul des variances est numériquement compliqué voire impossible, comme dans le cas où le modèle m est inconnu, les méthodes décrites ci-dessus ne sont pas pertinentes.

Une autre méthode consiste à approcher m par un modèle simplifié, appelé métamodèle, qui est beaucoup plus rapide à évaluer, et à effectuer l'analyse de sensibilité sur celui-ci. Non seulement un méta-modèle permet de calculer à moindre coût des indices de Sobol approchés, mais il fournit de l'information sur la nature des effets des variables d'entrées ou de leurs interactions sur la variable de sortie.

Méta-modélisation

La méta-modélisation consiste à construire une fonction qui est calculable, facile à interpréter et qui a de bonnes qualités de prédiction. Soit {m(X i )} n i=1 les résultats de n évaluations du modèle m basées sur un plan d'expérience {X i } n i=1 . Dans ce contexte, un méta-modèle est une approximation du modèle m construite à partir du plan d'expérience {X i } n i=1 et des sorties {m(X i )} n i=1 . Il existe différentes approches de méta-modélisation, voir [START_REF] Sacks | Design and analysis of computer experiments[END_REF], [START_REF] Friedman | Multivariate adaptive regression splines[END_REF], [START_REF] Breiman | Random forests[END_REF], [START_REF] Friedman | Greedy function approximation: A gradient boosting machine[END_REF], [START_REF] Kennedy | Bayesian calibration of computer models[END_REF], [START_REF] Oakley | Probabilistic sensitivity analysis of complex models: a bayesian approach[END_REF], [START_REF] Storlie | Multiple predictor smoothing methods for sensitivity analysis: Description of techniques[END_REF], [START_REF] Storlie | Implementation and evaluation of nonparametric regression procedures for sensitivity analysis of computationally demanding models[END_REF], [START_REF] Storlie | Surface estimation, variable selection, and the nonparametric oracle property[END_REF] pour différents exemples, et [START_REF] Touzani | Response surface methods based on analysis of variance expansion for sensitivity analysis[END_REF] pour un aperçu.

Dans le cadre de l'analyse de sensibilité globale, on considère un méta-modèle dont la décomposition additive est candidate pour approcher la décomposition de Hoeffding de m. Ce méta-modèle permettra ainsi d'effectuer l'analyse de sensibilité globale de m, en calculant des indices de Sobol, éventuellement d'ordre élevé. Par une fonction qui a la décomposition additive, on entend une fonction f de X ⊂ R d vers R qui est définie comme suit:

f = f 0 + v∈P f v (X v ), E X (f v (X v )) = 0, E X (f v (X v )f v (X v )) = 0, ∀v, v ∈ P, v = v ,
où f 0 est une constante, et les fonctions f v sont supposées appartenir aux espaces fonctionnels.

Parmi les approches de méta-modélisation proposées dans la littérature, la décomposition basée sur les polynômes de Chaos [START_REF] Wiener | The homogeneous chaos[END_REF], [START_REF] Schoutens | Stochastic Processes and Orthogonal Polynomials, Lecture Notes in Statistics[END_REF]) permet d'approcher la décomposition de Hoeffding de m [START_REF] Sudret | Global sensitivity analysis using polynomial chaos expansions[END_REF]).

Le principe de la décomposition selon les polynômes de Chaos est de projeter m sur une base de polynômes orthonormés de la façon suivante [START_REF] Soize | Physical systems with random uncertainties: Chaos representations with arbitrary probability measure[END_REF]):

m(X) = ∞ j=0 h j φ j (X),
(1.5) où {h j } ∞ j=0 sont les coefficients, et {φ j } ∞ j=0 sont des polynômes orthonormés multivariés associés à X qui sont déterminés par la distribution des coordonnées de X. En pratique, la série définie en (1.5) doit être tronquée conduisant à approcher m par: m(X) ≈ vmax j=0 h j φ j (X),

(1.6) où v max doit être déterminé par une méthode numérique. Dans cette approche, les indices de Sobol sont explicitement donnés à partir des carrés des coefficients associés. [START_REF] Blatman | Adaptive sparse polynomial chaos expansion based on least angle regression[END_REF] ont proposé une méthode pour tronquer la série (1.5) et un algorithme basé sur la méthode least-angle regression pour sélectionner les termes pertinents dans le développement.

Dans cette approche, la famille des polynômes orthonormés {φ j } ∞ j=0 est déterminée de manière unique par la distribution des coordonnées de X. Cependant, cette famille ne constitue pas nécessairement la meilleure base fonctionnelle pour bien approcher m.

Une autre approche pour construire des méta-modèles est la modélisation par le processus gaussien (GP) [START_REF] Welch | Screening, predicting, and computer experiments[END_REF], [START_REF] Oakley | Probabilistic sensitivity analysis of complex models: a bayesian approach[END_REF], [START_REF] Kleijnen | Design and Analysis of Simulation Experiments[END_REF][START_REF] Kleijnen | Kriging metamodeling in simulation: A review[END_REF], [START_REF] Marrel | Calculations of sobol indices for the gaussian process metamodel[END_REF], [START_REF] Durrande | Additive covariance kernels for high-dimensional gaussian process modeling[END_REF], Le [START_REF] Gratiet | A bayesian approach for global sensitivity analysis of (multifidelity) computer codes[END_REF]). Le principe est de modéliser la distribution a priori de m(X) par un modèle de GP, noté Z(X), de moyenne m Z (X) et de noyau de covariance k Z (X, X ). Pour effectuer l'analyse de sensibilité, on peut remplacer le vrai modèle m(X) par l'espérence de la loi à posteriori de Z(X), et en déduire les indices de Sobol. La plupart du temps, avec GP, les indices de Sobol sont estimés à l'aide de méthodes de Monte Carlo.

Une revue de la méta-modélisation basée sur les polynômes de Chaos et le GP est présentée dans l'ouvrage de Le [START_REF] Gratiet | Metamodel-Based Sensitivity Analysis: Polynomial Chaos Expansions and Gaussian Processes[END_REF]. [START_REF] Durrande | Anova kernels and rkhs of zero mean functions for model-based sensitivity analysis[END_REF] ont considéré une classe de méthodes d'approximation fonctionnelle similaire au GP et ont obtenu un méta-modèle qui satisfait les propriétés de la décomposition de Hoeffding. Ils ont proposé d'approcher m par des fonctions appartenant à un RKHS H qui est construit comme une somme directe d'espaces RKHS, de sorte que la projection de m sur H est une approximation de la décomposition de Hoeffding de m.

Dans le modèle de régression, lorsque les valeurs de {m(X i )} n i=1 ne peuvent pas être calculées, on peut utiliser les méthodes de projection sur une base fonctionnelle pour estimer un méta-modèle pour m. Ce méta-modèle sera estimé en utilisant des approches d'estimation non-paramétriques à partir des observations {(X i , Y i )} n i=1 , et on déduira de cet estimateur des estimateurs des indices de Sobol de m. Huet and Taupin (2017) ont considéré les mêmes espaces d'approximation fonctionnels que [START_REF] Durrande | Anova kernels and rkhs of zero mean functions for model-based sensitivity analysis[END_REF], et ont proposé un estimateur d'un méta-modèle qui approche la décomposition de Hoeffding de m. Elles ont déduit de ce méta-modèle estimé, des estimateurs pour les indices de Sobol de m. Cette approche est présentée plus en détail par la suite.

Méta-modèles basés sur des espaces à noyaux auto-reproduisants (RKHS)

Cette section commencera tout d'abord par une brève introduction des espaces RKHS. La méthode de [START_REF] Durrande | Anova kernels and rkhs of zero mean functions for model-based sensitivity analysis[END_REF] pour construire le RKHS et la définition de méta-modèle f * qui approche la décomposition de Hoeffding de m, sont décrites respectivement dans les sections 1.1.4.2 et 1.1.4.3.

Introduction aux espaces RKHS

Soit H un espace de Hilbert de fonctions définies sur un ensemble X . L'espace H est un RKHS si pour tout X ∈ X les fonctionnelles,

L X : H → R f → f (X),
sont continues. Le théorème de représentation de Riesz assure l'existence d'un élément unique k X (.) dans H vérifiant la propriété suivante:

∀X ∈ X , ∀f ∈ H, f (X) = L X (f ) = f, k X H .
où ., . H dénote le produit scalaire dans H.

Il en découle que pour tout X, X dans X , et k X (.), k X (.) dans H, on a,

k X (X ) = L X (k X ) = k X , k X H .
(1.7)

Cela permet de définir le noyau auto-reproduisant de H comme suit:

k : X × X → R (X, X ) → k X (X ).
Le noyau auto-reproduisant k(X, X ) satisfait les propriétés suivantes:

-Il est symétrique. En effet, par définition de k(., .) et grâce à la propriété (1.7), on a:

k(X, X ) = k X (X ) = k X , k X H = k X (X) = k(X , X). -Pour tout n ∈ N, {X i } n i=1 ∈ X et {c i } n i=1 ∈ R, on a: n i=1 n j=1 c i c j k(X i , X j ) = n i=1 n j=1 c i k(X i , .), c j k(X j , .) H , = n i=1 c i k(X i , .) 2 H ≥ 0.
Ainsi, k(X, X ) est défini positif. Davantage d'informations sur les espaces RKHS sont indiquées dans des ouvrages standards comme [START_REF] Aronszajn | Theory of reproducing kernels[END_REF], [START_REF] Saitoh | Theory of reproducing kernels and its applications[END_REF] et [START_REF] Berlinet | Reproducing Kernel Hilbert Spaces in Probability and Statistics[END_REF].

Construction du RKHS et décomposition de Hoeffding

L'idée est de construire un RKHS incluant les fonctions qui ont la décomposition additive et qui sont candidates pour approcher la décomposition de Hoeffding de m. Pour cela, on utilise la méthode de [START_REF] Durrande | Anova kernels and rkhs of zero mean functions for model-based sensitivity analysis[END_REF] décrite ci-dessous.

Soit X = X 1 × . . . × X d un sous-ensemble de R d . Pour chaque a ∈ {1, • • • , d}, on choisit un RKHS H a et son noyau associé k a défini sur l'ensemble X a ⊂ R, tels que les deux propriétés suivantes soient satisfaites:

(i) k a : X a × X a → R est P a ⊗ P a mesurable, (ii) E Xa k a (X a , X a ) < ∞.
La propriété (ii) dépend du noyau k a , a = 1, ..., d et de la loi de X a , a = 1, ..., d. Elle est relativement peu restrictive car elle est satisfaite, par exemple, pour tous les noyaux bornés.

Le RKHS H a peut être décomposé en une somme de deux sous-RKHS orthogonaux,

H a = H 0a ⊥ ⊕ H 1a ,
où H 0a est le RKHS des fonctions centrées,

H 0a = f a ∈ H a : E Xa (f a (X a )) = 0 ,
et H 1a est le RKHS des fonctions constantes,

H 1a = f a ∈ H a : f a (X a ) = C .
Le noyau k 0a associé au RKHS H 0a est défini par:

k 0a (X a , X a ) = k a (X a , X a ) - E U ∼Pa (k a (X a , U ))E U ∼Pa (k a (X a , U )) E (U,V )∼Pa⊗Pa k a (U, V ) . (1.8) Soit k v (X v , X v ) = a∈v k 0a (X a , X a )
, le noyau ANOVA k(., .) est défini comme suit:

k(X, X ) = d a=1 1 + k 0a (X a , X a ) = 1 + v∈P k v (X v , X v ).
Pour H v étant le RKHS associé au noyau k v , le RKHS associé au noyau ANOVA est défini par,

H = d a=1 1 ⊥ ⊕ H 0a = 1 + v∈P H v ,
où ⊥ correspond à une orthogonalité pour le produit scalaire sur L 2 . D'après cette construction, toute fonction f ∈ H satisfait la décomposition suivante:

f (X) = f, k(X, .) H = f 0 + v∈P f v (X v ),
qui est la décomposition de Hoeffding de f .

Les propriétés de régularité du RKHS H construit comme décrit ci-dessus, dépendent de l'ensemble des noyaux (k a , a = 1, ..., d). Cette méthode permet de choisir différents espaces d'approximation indépendamment de la distribution des coordonnées de X, en choisissant différents ensembles de noyaux. Alors que, comme indiqué précédemment, dans l'approche de méta-modélisation basée sur le développement en polynômes de Chaos, la famille des polynômes orthonormés {φ j } ∞ j=0 est déterminée de manière unique par la distribution des coordonnées de X. Ici, la distribution des coordonnées de X n'intervient que pour l'orthogonalisation des espaces H v , v ∈ P mais pas dans le choix des RKHS, pourvu que les propriétés (i) et (ii) soient satisfaites. C'est l'un des principaux avantages de cette méthode par rapport à l'approche basée sur le développement en polynômes de Chaos où la régularité de l'approximation n'est gérée que par le choix de v max (voir l'équation (1.6)) et non par celui de la base fonctionnelle [START_REF] Blatman | Adaptive sparse polynomial chaos expansion based on least angle regression[END_REF]).

1.1.4.3 Approximation de la décomposition de Hoeffding de m Soit f * la projection orthogonale de m sur H définie par:

f * = arg min f ∈H m -f 2 2 = arg min f ∈H E X (m(X) -f (X)) 2 .
La fonction f * ∈ H, f * = f * 0 + v∈P f * v est l'approximation de m sur le RKHS H, et sa décomposition de Hoeffding est une approximation de la décomposition de Hoeffding de m. Par conséquent, pour chaque v ∈ P, la fonction f * v approche la fonction m v dans l'équation (1.3).

Le nombre de fonctions f * v est lié au cardinal de P, égal à 2 d -1, qui peut devenir très grand dès que d est grand. Ainsi, l'idée est de calculer un estimateur sparse de f * comme estimateur de m en utilisant des méthodes d'estimation non-paramétriques.

Méthode d'estimation

Considérons le modèle de régression défini dans l'équation (1.1), Y = m(X) + σε, σ > 0.

La fonction inconnue m est approchée par le méta-modèle f * qui est ensuite estimé par un estimateur sparse f . Cet estimateur f , basé sur n observations {(Y i , X i )} n i=1 , minimise un critère pénalisé. La fonction de pénalité prend en compte à la fois la nature non-paramétrique du problème et le nombre éventuellement important de fonctions qui doivent être estimées.

Avant de décrire la méthode pour calculer f , on rappelle quelques méthodes liées à l'estimation dans un modèle de régression non-paramétrique additif.

Certains auteurs approchent m par une fonction qui a une décomposition additive univariée et sparse de la forme,

f (X) = f 0 + a∈S f a (X a ) avec |S| < d,
(1.9) où f 0 est une constante et où pour tout a ∈ S, les f a sont des fonctions supposées régulières. Les fonctions f a sont estimées à partir des observations à l'aide d'un critère pénalisé. [START_REF] Ravikumar | Sparse additive models[END_REF] ont considéré un espace de Hilbert H de fonctions qui ont une forme additive univariée. Leur espace d'approximation fonctionnelle H est construit comme une somme directe des espaces de Hilbert, c'est-à-dire

H = d a=1 H a ,
où pour tout a ∈ {1, ..., d}, H a est le sous-espace de Hilbert de L 2 (X a , P a ) des fonctions univariées f a qui sont centrées et P a mesurables. Afin de favoriser sparsité et régularité, ils ont proposé la méthode SpAM (Sparse Additive Models). Leur méthode est basée sur la minimisation du critère des moindres carrés pénalisé par une fonction de pénalité définie comme suit, [START_REF] Meier | High-dimensional additive modeling[END_REF] ont proposé un estimateur qui est dans l'espace des splines cubiques naturel. Leur méthode est basée sur la minimisation du critère des moindres carrés pénalisé par une fonction de pénalité de la forme,

λ d a=1 (f a (X a )) 2 dX a , λ ∈ R + .
d a=1 λ 1 f a 2 n + λ 2 (f a (X a )) 2 dX a , λ 1 , λ 2 ∈ R + , où f a 2 n = 1 n n i=1 f 2
a (X ai ). Leur fonction de pénalité est composée de deux parties: la première partie favorise la sparsité et la deuxième partie favorise la régularité. [START_REF] Raskutti | Minimax-optimal rates for sparse additive models over kernel classes via convex programming[END_REF] ont considéré plusieurs espaces d'approximation fonctionnelle, y compris les polynômes, les splines et les classes de Sobolev. Leur méthode est basée sur la minimisation du critère des moindres carrés pénalisé par une fonction de pénalité de la forme,

γ f n,1 + µ f H,1 , γ, µ ∈ R + ,
(1.10) où f n,1 = d a=1 f a n et f H,1 = d a=1 f a Ha . Dans leur fonction de pénalité, la première partie favorise la sparsité et la deuxième partie favorise la régularité.

Effectuer l'analyse de sensibilité globale sur un modèle additif univarié conduit à n'obtenir que les indices de Sobol de premier ordre, ce qui ne fournit peut-être pas une bonne information sur la sensibilité du modèle. Les interactions entre les variables qui peuvent affecter la relation entre Y et X sont complètement ignorées dans ce contexte. Afin d'inclure les effets d'interaction, on peut approcher m par une fonction qui a une décomposition additive multivariée et sparse de la forme,

f (X) = f 0 + v∈S f v (X v ) avec |S| < |P|,
qui est une généralisation de la décomposition additive univariée définie dans l'équation (1.9).

Dans le cadre du lissage par splines de type ANOVA [START_REF] Wahba | Spline Models for Observational Data[END_REF], [START_REF] Friedman | Multivariate adaptive regression splines[END_REF], [START_REF] Wahba | Smoothing spline anova for exponential families, with application to the wisconsin epidemiological study of diabetic retinopathy[END_REF]), [START_REF] Lin | Component selection and smoothing in multivariate nonparametric regression[END_REF] ont proposé la méthode COSSO (Component Selection and Smoothing Operator ). Leur méthode est basée sur la minimisation du critère des moindres carrés pénalisé par une fonction de pénalité qui est la combinaison de la norme l 1 avec la norme de Hilbert. L'implémentation de COSSO s'effectue sur les espaces de Sobolev de second ordre. [START_REF] Kandasamy | Additive approximations in high dimensional nonparametric regression via the salsa[END_REF] Huet and Taupin (2017) ont considéré un estimateur d'un méta-modèle qui approche la décomposition de Hoeffding de m définie dans l'équation (1.3). Leur estimateur est la solution de minimisation du critère des moindres carrés pénalisé, où la fonction de pénalité est définie dans l'équation (1.10) et est adaptée au cadre multivarié,

γ f n + µ f H avec f H = v∈P f v Hv , et f n = v∈P f v n .
Leur méthode, appelée ridge group sparse, estime les groupes v qui sont pertinents pour prédire le méta-modèle f * et la relation entre f * v et X v pour chaque groupe v ∈ P. L'estimateur obtenu, appelé l'estimateur RKHS ridge group sparse, est ensuite utilisé pour estimer les indices de Sobol de m. Cette méthode permet d'estimer les indices de Sobol pour tous les groupes dans le support de l'estimateur RKHS ridge group sparse, y compris les interactions d'ordre élevé, un point connu pour être difficile à mettre en pratique.

Dans ce travail, la méthode proposée par Huet and Taupin (2017) est utilisée afin de calculer un estimateur sparse du méta-modèle f * qui permet également de calculer les estimateurs des indices de Sobol de m. Décrivons plus en détail cette méthode et la méthode pour estimer les indices de Sobol de m respectivement dans la section suivante et la section 1.1.5.2.

Procédure ridge group sparse et estimateur associé

Pour tout v ∈ P, soit X v la matrice des variables correspondant au v-ième groupe,

X v = (X vi , i = 1, ..., n, v ∈ P) ∈ R n×|P| .

Cadre de travail 13

Pour tout f ∈ H tel que f = f 0 + v∈P f v , et pour des paramètres de régularisations γ v , µ v , v ∈ P, le critère ridge group sparse est défini comme suit: (1.11) où f v n est la norme empirique L 2 de f v définie en fonction de l'échantillon {X vi } n i=1 comme suit:

L(f ) = 1 n n i=1 Y i -f 0 - v∈P f v (X vi ) 2 + v∈P γ v f v n + v∈P µ v f v Hv ,
f v 2 n = 1 n n i=1 f 2 v (X vi ).
La fonction de pénalité dans le critère L(f ) est la somme de la norme empirique et de la norme de Hilbert, ce qui permet de sélectionner peu de termes dans la décomposition additive de f sur les ensembles v ∈ P. De plus, la norme de Hilbert favorise la régularité du f v , v ∈ P estimé. Définissons l'ensemble des fonctions,

F = f : f = f 0 + v∈P f v , with f v ∈ H v , and f v Hv ≤ r v , r v > 0 .
(1.12) L'estimateur RKHS ridge group sparse de m est défini par:

f = arg min f ∈F L(f ).
(1.13) D'après le representer théorème [START_REF] Kimeldorf | A correspondence between bayesian estimation on stochastic processes and smoothing by splines[END_REF]), le problème de minimisation fonctionnelle non-paramétrique (1.13) est équivalent à un problème de minimisation paramétrique. En effet, la solution du problème de minimisation (1.13) appartenant au RKHS H est écrite comme f = f 0 + v∈P f v , où pour une matrice θ = (θ vi , i = 1, ..., n, v ∈ P) ∈ R n×|P| on a pour tout v ∈ P,

f v (.) = n i=1 θ vi k v (X vi , .).
Soit . la norme euclidienne dans R n , et pour chaque v ∈ P, soit K v la n × n matrice de Gram associée au noyau k v (., .), c'est-à-dire

(K v ) i,i = k v (X vi , X vi ). Soit aussi K 1/2 v la matrice qui satisfait t(K 1/2 v )K 1/2 v = K v , et soit f 0 et θ les solutions de minimisation du critère suivant: C(f 0 , θ) = Y -f 0 I n - v∈P K v θ v 2 + √ n v∈P γ v K v θ v + n v∈P µ v K 1/2 v θ v .
(1.14) Alors l'estimateur f défini dans l'équation (1.13) satisfait,

f (X) = f 0 + v∈P f v (X v ) avec f v (X v ) = n i=1 θ vi k v (X vi , X v ).
(1.15)

Comme le critère C(f 0 , θ) est convexe et séparable, on peut calculer θ en utilisant un algorithme de block coordinate descent [START_REF] Boyd | Distributed optimization and statistical learning via the alternating direction method of multipliers[END_REF][START_REF] Bubeck | Convex optimization: Algorithms and complexity[END_REF]).

Remarque 1.1.1 La contrainte f v Hv ≤ r v n'est pas prise en compte dans le problème de minimisation paramétrique. Cette contrainte est cruciale pour les propriétés théoriques, mais la valeur de r v est inconnue et n'est pas utile en pratique.

Estimation des indices de Sobol de m

La variance de la fonction m est estimée par la variance de l'estimateur f . Comme l'estimateur f appartient au RKHS H, il admet la décomposition de Hoeffding et,

var( f (X)) = v∈P var( f v (X v )), où pour tout v ∈ P, var( f v (X v )) = E X ( f 2 v (X v )) = f v 2 2 .
Afin de réduire le temps de calcul, on peut estimer les variances de

f v (X v ), v ∈ P par leurs variances empiriques. Soit f v. la moyenne empirique de { f v (X vi )} n i=1 , f v. = 1 n n i=1 f (X vi ), alors var( f v (X v )) = 1 n -1 n i=1 ( f v (X vi ) -f v. ) 2 .
Pour les groupes v qui appartiennent au support de f , les estimateurs des indices de Sobol de m sont définis par,

S v = var( f v (X v )) v∈P var( f v (X v ))
, et pour les groupes v qui n'appartiennent pas au support de f , on a S v = 0.

1.2 Résumé du chapitre 3

Objectifs et résultats

Pour un estimateur f d'un modèle m soit R(m, f ) son risque. Le risque R(m, f ) est une mesure qui caractérise la précision de l'estimateur f et qui peut être exprimé en fonction du biais et de la variance de f :

R(m, f ) = (biais( f )) 2 + var( f ).

Résumé du chapitre 3

Ainsi, la qualité de l'estimateur f peut être mesurée par son risque. On considère le risque empirique

L 2 de l'estimateur f , c'est-à-dire lorsque R(m, f ) = m -f 2 n , et le risque quadratique de l'estimateur f , c'est-à-dire lorsque R(m, f ) = m -f 2 2
. On s'intéresse à des propriétés non-asymptotiques de l'estimateur f , au sens où l'on ne suppose pas que le nombre d'observations n tend vers l'infini. Nos résultats sont donc valables pour tout n avec une grande probabilité. On établit en particulier, des majorations du risque R(m, f ) de la forme,

R(m, f ) ≤ C inf f ∈F {R(m, f ) + r n (f )}, (1.16) où C est une constante, et F est l'espace d'approximation.
Soit f la fonction dans F pour laquelle l'infimum du membre de droite de l'inégalité (1.16) est réalisé. Le terme R(m, f ) est le terme de biais qui dépend du choix de l'espace d'approximation. Le terme r n (f ) est le terme de variance qui doit décroître avec n. Il contrôle la vitesse de convergence, c'est-à-dire la vitesse à laquelle le risque de l'estimateur va s'approcher du meilleur possible. Le terme de variance dépend de la régularité des noyaux k v , v ∈ P, du nombre de termes intervenant dans la décomposition de la fonction f sur l'espace d'approximation, du nombre de variables d'entrée d, et du nombre d'observations n.

Dans le modèle de régression gaussienne, c'est-à-dire lorsque ε dans l'équation (1.1) est une variable gaussienne centrée, Huet and Taupin (2017) ont établi les majorations du risque empirique L 2 et du risque quadratique de l'estimateur RKHS ridge group sparse f . Dans ce chapitre, on considère le modèle de régression avec l'erreur ε nongaussienne et non-bornée. Dans ce contexte, l'objectif est d'établir des majorations du risque de l'estimateur RKHS ridge group sparse f de la forme (1.16) avec la même vitesse de convergence que dans le modèle de régression gaussienne. Les majorations du risque empirique L 2 et du risque quadratique de l'estimateur f sont présentées respectivement dans le résultat 1 et le résultat 2.

Présentation du modèle

Considérons le modèle de régression défini dans l'équation (1.1),

Y = m(X) + σε, σ > 0.
Les variables d'entrée X = (X 1 , ..., X d ) sont indépendantes et ont une loi connue P X = d a=1 P a sur X = d a=1 X a , un sous-ensemble compact de R d . La fonction m : R d → R est inconnue, peut-être complexe, et elle est supposée être de carréintégrable sur X .

Soit D l'ensemble des densités,

D = π α : π α (x) = a α exp(-|x| α ), avec (a α ) -1 = R exp(-|x| α )dx, α > 2 .
(1.17)

Dans ce chapitre, on suppose que l'erreur ε est égale à Z/σ α , où Z est une variable aléatoire de densité π α ∈ D et σ 2 α = var(Z).

Les principaux résultats

Commençons par quelques notations, propriétés et hypothèses qui sont nécessaires pour annoncer le résultat 1 et le résultat 2.

Notations

Pour une fonction f ∈ H, soit S f son support,

S f = {v ∈ P : f v = 0}. À chaque noyau k v , v ∈ P on associe l'opérateur intégral T kv de L 2 (X v , P v ) vers L 2 (X v , P v ) défini par: ∀f ∈ L 2 (X v , P v ), T kv (f ) = Xv k v (., t)f (t)dP v (t).
Pour chaque v ∈ P, soit ω v,1 ≥ ω v,2 ≥ ... ≥ 0 les valeurs propres de l'opérateur intégral T kv . Définissons la fonction Q n,v (t) pour un t positif comme suit:

Q n,v (t) = 5 n ≥1 min(t 2 , ω v, ).
Pour une constante ∆ > 0, soit ν n,v défini par:

ν n,v = inf t Q n,v (t) ≤ ∆t 2 .
(1.18)

Pour chaque v ∈ P, ν n,v est la vitesse minimax d'estimation par rapport à la norme L 2 (X , P X ) dans le RKHS H v [START_REF] Mendelson | Geometric parameters of kernel machines[END_REF]).

Remark 1.2.1 la vitesse d'estimation ν n,v , v ∈ P, est liée à la régularité du RKHS via le taux de décroissant des valeurs propres {ω v, } ∞ =1 . Lorsque le RKHS est très régulière, c'est-à-dire lorsque les valeurs propres {ω v, } ∞ =1 tendent rapidement vers 0, la vitesse ν n,v , v ∈ P sera proche de la vitesse paramétrique (voir section 3.3.1 du chapitre 3).

Propriétés

La construction du RKHS décrite à la section 1.1.4.2 assure que les propriétés suivantes sont satisfaites:

P1 Pour tout v ∈ P, les fonctions f v ∈ H v sont centrées et de carré-intégrables, E X (f v (X v )) = 0 et E X (f 2 v (X v )) < ∞. P2 Pour tout v, v ∈ P, v = v , les fonctions f v ∈ H v et f v ∈ H v sont orthogo- nales par rapport à L 2 (X , P X ), E X (f v (X v )f v (X v )) = 0.

Résumé du chapitre 3

Hypothèses

H1 Pour tout v ∈ P, les fonctions f v ∈ H v sont uniformément bornées, ∃R > 0 tel que f v ∞ = sup Xv |f v (X v )| ≤ R.
Cette hypothèse est satisfaite lorsque le noyau k v est borné sur l'ensemble compact X . En effet,

f v ∞ ≤ sup X∈X k v (X v , X v ) f v Hv .
Pour chaque v ∈ P, soit λ n,v défini de la façon suivante:

λ n,v = max ν n,v , d n . (1.19)
Les paramètres de régularisation µ v et γ v intervenant dans le critère (1.11) sont choisis comme suit:

H2 Pour une constante C 1 > 10 + 4∆, ∀v ∈ P, µ v = C 1 λ 2 n,v , γ v = C 1 λ n,v .
H3 Il existe des constantes positives C 2 , C 3 , et 0 < β < 1/α telles que les conditions suivantes sont satisfaites:

∀v ∈ P, nλ 2 n,v ≥ -C 2 log λ n,v , (1.20) et ∀f ∈ F, v∈S f λ 2 n,v ≤ C 3 n 2β-1 . (1.21)
Le chapitre 3 présente les deux résultats suivants.

Résultat 1: la majoration du risque empirique L 2 de l'estimateur RKHS ridge group sparse Considérons le modèle de régression décrit dans la section 1.2.

2 avec σ = 1. Soit {(Y i , X i )} n i=1 un échantillon de taille n de la même loi que (Y, X), et soient {ε i } n i=1
les erreurs aléatoires qui sont indépendantes et identiquement distribuées (i.i.d.) comme ε. Soit aussi l'estimateur RKHS ridge group sparse f défini par (1.13) avec r v = 1 dans (1.12). Sous les hypothèses H1, H2 et H3, il existe une constante positive C et 0 < η < 1 (η tend vers 0 lorsque n augmente) tels que,

m -f 2 n ≤ C inf f ∈F m -f 2 n + v∈S f (µ v + γ 2 v ) , (

R4

Le résultat 1 peut être généralisé au cas où σ = 1 dans l'équation (1.1), et où r v = 1 dans l'équation (1.12), voir la remarque 3.3.5 du chapitre 3 pour une brève démonstration de ce point.

Résultat 2: la majoration du risque quadratique de l'estimateur RKHS ridge group sparse Sous les mêmes hypothèses que pour le résultat 1, on a avec une grande probabilité pour une constante positive C que,

m -f 2 2 ≤ C inf f ∈F m -f 2 n + m -f 2 2 + v∈S f (µ v + γ 2 v ) .
Remark 1.2.2 Le résultat 2 peut être généralisé au cas où σ = 1 dans l'équation (1.1), et où r v = 1 dans l'équation (1.12) (voir la remarque 3.3.6 du chapitre 3 pour plus de détails sur ce point).

Vitesse de convergence

Sous les mêmes hypothèses que pour le résultat 1, on a:

m -f 2 n ≤ C inf f ∈F m -f 2 n + v∈S f ν 2 n,v + d|S f | n .
Cette inégalité met en évidence que la borne supérieure est pertinente lorsque l'infimum est atteint pour les fonctions f qui ont une décomposition sparse dans [START_REF] Raskutti | Minimax-optimal rates for sparse additive models over kernel classes via convex programming[END_REF] ont supposé que la fonction m a une représentation additive univariée et sparse (telle que définie dans l'équation (1.9)) de sorte que chaque fonction univariée se trouve dans un RKHS. Ils ont proposé la procédure ridge group sparse pour calculer l'estimateur de m, et ont étudié les propriétés théoriques de leur estimateur dans le modèle de régression gaussienne. Ils ont fourni les majora-tions du risque empirique L 2 et du risque quadratique et une minoration du risque quadratique de leur estimateur sur des espaces de modèles additifs sparse, y compris les polynômes, les splines et les classes de Sobolev. Huet and Taupin (2017) [START_REF] Pisier | The volume of convex bodies and Banach space geometry[END_REF], [START_REF] Massart | About the constants in talagrand's concentration inequalities for empirical processes[END_REF], [START_REF] Van De Geer | Empirical Processes in M-Estimation, Cambridge Series in Statistical and Probabilistic Mathematics[END_REF], [START_REF] Ledoux | The Concentration of Measure Phenomenon, Mathematical surveys and monographs[END_REF]), ainsi que sur les résultats sur la complexité Rademacher des classes de noyau [START_REF] Mendelson | Geometric parameters of kernel machines[END_REF], [START_REF] Bartlett | Local rademacher complexities[END_REF]). [START_REF] Talagrand | The supremum of some canonical processes[END_REF]. La minoration donnée par [START_REF] Talagrand | The supremum of some canonical processes[END_REF] est spécifique aux densités π α (voir l'équation (1.17)). C'est la raison pour laquelle la classe de densité D est considérée dans ce travail. La minoration de type Sudakov adaptée à notre travail est déduite du théorème 3.1. de Talagrand (1994). Rappelons cette minoration. Soient ε = (ε 1 , ..., ε n ) des variables aléatoires i.i.d. distribuées avec la densité π α ∈ D, et pour une fonction g : R |v| → R, v ∈ P appartenant à une classe de fonctions G, soit V n,ε le processus empirique associé au vecteur aléatoire ε,

Outils techniques pour les preuves

V n,ε (g) = 1 n n i=1 ε i g(X v,i ). (1.23) Alors, pour tout δ > 0, 1 K log N (δ, G, . ) ≤ ( 2nE ε sup g∈G |V n,ε (g)| δ ) 2 1 [2nEε sup g∈G |Vn,ε(g)|,∞) (δ) + ( 2nE ε sup g∈G |V n,ε (g)| δ ) α 1 (0,2nEε sup g∈G |Vn,ε(g)|] (δ), (1.24)
où K est une constante qui ne dépend que de α, N (δ, G, . ) est le nombre de recouvrements de l'espace métrique (G, . ) par des boules de rayon inférieur à δ, et

1 A : A → {0, 1} est la fonction caractéristique de A ⊂ A, 1 A (a) = 1 if a ∈ A, 0 if a / ∈ A.
Concernant l'inégalité de concentration, il est démontré que les fonctions de distribution associées aux densités π α ∈ D appartiennent à une classe de fonctions de distribution définie par [START_REF] Adamczak | Logarithmic sobolev inequalities and concentration of measure for convex functions and polynomial chaoses[END_REF], pour laquelle l'inégalité de log-Sobolev [START_REF] Gross | Logarithmic sobolev inequalities[END_REF]) est satisfaite (voir lemme 3.4.2 du chapitre 3). [START_REF] Shu | A characterization of a class of convex log-sobolev inequalities on the real line[END_REF] ont établi des bornes pour les queues inférieures et supérieures de fonctions convexes de variables aléatoires indépendantes qui satisfont l'inégalité de log-Sobolev. Comme les fonctions de distribution associées aux densités π α ∈ D satisfont l'inégalité de log-Sobolev, l'inégalité de concentration dérivée par [START_REF] Shu | A characterization of a class of convex log-sobolev inequalities on the real line[END_REF] est valable pour elles. L'inégalité de concentration adaptée à notre travail est déduite du corollaire 1.7. de Shu and Strzelecki (2017) (voir corollaire 3.4.2 du chapitre 3).

1.3 Résumé du chapitre 4

Objectifs et résultats

Un package R, appelé RKHSMetaMod, a été développé pour mettre en oeuvre la procédure ridge group sparse décrite dans la section 1.1.5.1. Ce package permet de: calculer les noyaux auto-reproduisants comme décrit dans la section 1.1.4.2, et leurs matrices de Gram associées, mettre en oeuvre la procédure RKHS ridge group sparse et un cas particulier de celle-ci appelé RKHS group lasso (lorsque γ v = 0, v ∈ P dans le critère (1.14)) afin d'estimer les termes f * v dans la décomposition de Hoeffding de f * conduisant à une estimation de la fonction m, choisir les paramètres de régularisation µ v , γ v , v ∈ P dans le critère (1.14) en utilisant une procédure qui permet d'obtenir le meilleur estimateur RKHS ridge group sparse en termes de qualité de prédiction, d'estimer les indices de Sobol de la fonction m comme décrit à la section 1.1.5.2.

Le package RKHSMetaMod fournit une interface entre l'environnement de calcul statistique R et les bibliothèques C++ Eigen et GSL. Afin d'optimiser le temps de calcul et la mémoire de stockage, toutes les fonctions de ce package ont été écrites en utilisant les bibliothéques Eigen et GSL de C++ à l'exception d'une fonction qui est écrite en R. Elles sont ensuite interfacées avec l'environnement R afin de proposer un package facilement exploitable aux utilisateurs de R. Le package RKHSMetaMod est dédié à l'estimation du méta-modèle f * d'un modèle m sur le RKHS H. Les algorithmes d'optimisation convexe utilisés dans ce package sont adaptés pour prendre en compte le problème de la grande dimensionnalité dans ce contexte. Ce package est disponible sur le Comprehensive R Archive Network (CRAN) à https://cran.r-project.org/web/packages/RKHSMetaMod/.

Présentation du modèle

Considérons un phénomène décrit par un modèle m dépendant de d variables d'entrée X = (X 1 , ..., X d ). Ce modèle m de R d vers R peut être un modèle connu qui est calculable en tout point X, ou un modèle de regression comme défini dans l'équation (1.1). Dans le second cas, l'erreur ε est supposée être centrée avec une variance finie. Les coordonnées de X sont indépendantes et ont la loi uniforme sur X = [0, 1] d . C'est-à-dire X ∼ P X = d a=1 P a , où chaque P a , a = 1, ..., d représente la loi uniforme sur l'intervalle [0, 1]. Le modèle m peut être complexe, présenter de fortes non-linéarités et des effets d'interaction d'ordre élevé, et il est supposé être de carréintégrable sur X .

Critère à minimiser

Considérons la forme paramétrique du critère RKHS ridge group sparse défini dans l'équation (1.14), où γ v et µ v , v ∈ P sont choisis comme suit:

Pour chaque v ∈ P, soient γ v et µ v les poids qui sont choisis de manière appropriée. Alors,

γ v = γ × γ v et µ v = µ × µ v , γ, µ ∈ R + .
Remark 1.3.1 Cette formulation simplifie le choix des paramètres de régularisation, car au lieu de sélectionner les paramètres γ v et µ v pour tous les v ∈ P, seuls deux paramètres γ et µ sont sélectionnés. De plus, les poids γ v et µ v , v ∈ P, peuvent être intéressants pour les applications. Par exemple, on peut prendre des poids qui augmentent avec le cardinal de v afin de favoriser les effets avec un ordre d'interaction petit entre les variables.

Pour plus de simplicité, dans le reste de ce chapitre les valeurs de γ v et µ v pour tout v ∈ P sont fixées à 1, et le critère RKHS ridge group sparse est alors exprimé comme suit:

C(f 0 , θ) = Y -f 0 I n - v∈P K v θ v 2 + √ nγ v∈P K v θ v + nµ v∈P K 1/2 v θ v .
En ne considérant que la deuxième partie de la fonction de pénalité dans le critère ci-dessus, c'est-à-dire en fixant γ à zéro, on obtient le critère du RKHS group lasso comme suit,

C g (f 0 , θ) = Y -f 0 I n - v∈P K v θ v 2 + nµ v∈P K 1/2 v θ v .
À une transformation près,

β v = K 1/2
v θ v , le critère C g est exactement un critère de group lasso [START_REF] Yuan | Model selection and estimation in regression with grouped variables[END_REF]).

Il convient de préciser que, dans le package RKHSMetaMod, les solutions de l'algorithme du RKHS group lasso sont utilisées afin d'initialiser les paramètres d'entrées de l'algorithme du RKHS ridge group sparse. En effet, la fonction de pénalité dans le critère de RKHS group lasso C g (f 0 , θ) assure la sparsité de la solution. Ainsi, pour une valeur donnée de µ, en implémentant l'algorithme du RKHS group lasso, on obtient une solution avec peu de termes dans sa décomposition additive.

Le paramètre de régularisation dans l'algorithme du RKHS group lasso sera noté par:

µ g = √ nµ.
(1.25)

Choix des paramètres de régularisation

Lorsque on est confronté à un problème d'optimisation, l'une des étapes essentielles consiste à choisir correctement les paramètres de régularisation. Pour cela, d'abord une grille de valeurs de µ et γ est choisie.

Soit µ max la valeur la plus petite de µ g (voir équation (1.25)), de sorte que la solution à la minimisation du problème de RKHS group lasso pour tout v ∈ P est θ v = 0. On a,

µ max = max v 2 √ n K 1/2 v (Y -Ȳ ) .
Pour configurer la grille de valeurs de µ, il suffit de trouver µ max , puis une grille de valeurs de µ est définie comme suit:

µ l = µ max ( √ n × 2 l )
, l ∈ {1, ..., l max }.

La grille de valeurs de γ est choisie par l'utilisateur.

ErrPred(µ, γ) = 1 n test n test i=1 (Y test i -f (µ,γ) (X test i )) 2 , où pour S f étant le support de l'estimateur f (µ,γ) , f (µ,γ) (X test ) = f 0 + v∈S f n i=1 θ vi k v (X vi , X test v ).
La paire ( µ, γ) avec la plus petite valeur de l'erreur de prédiction est choisie, et l'estimateur f ( µ, γ) est considéré comme le meilleur estimateur de la fonction m, par rapport à l'erreur de prédiction.

Dans le package RKHSMetaMod, les algorithmes pour calculer une suite d'estimateurs f , la valeur de µ max et l'erreur de prédiction sont implémentés respectivement dans les fonctions RKHSMetMod, mu_max et PredErr.

Estimation des indices de Sobol

Les indices de Sobol de la fonction m sont estimés par les indices de Sobol empiriques de l'estimateur f comme décrit dans la section 1.1.5.2,

S v =    var( fv(Xv)) v∈P var( fv(Xv)) pour v ∈ S f , 0 pour v / ∈ S f .
Dans le package RKHSMetaMod, l'algorithme permettant de calculer des indices empiriques de Sobol S v , v ∈ P est implémenté dans la fonction SI_emp.

Algorithmes

k a (X a , X a ) = min(X a , X a ) + 1.
Le RKHS associé au noyau k a est l'ensemble,

H a = f : [0, 1] → R est absolument continu et f (0) = 0, 1 0 f (X a ) 2 dX a < ∞ , avec le produit scalaire f, h Ha = 1 0 f (X a )h (X a )dX a .
Le noyau k 0a associé au noyau brownien est calculé comme suit,

k 0a = min(X a , X a ) + 1 - ( 1 0 (min(X a , U ) + 1)dU )( 1 0 (min(X a , U ) + 1)dU ) ( 1 0 1 0 (min(U, V ) + 1)dU dV ) , = min(X a , X a ) + 1 - 3 4 (1 + X a - X 2 a 2 )(1 + X a - X 2 a 2 ).
Le RKHS associé au noyau k 0a est l'ensemble,

H 0a = f ∈ H a : 1 0 f (X a )dX a = 0 .
Enfin, le RKHS H = 1 + v∈P H v est l'ensemble suivant,

H = f : [0, 1] d → R : f = f 0 + v∈P f v (X v ), avec f v ∈ H v .
Calculer les matrices de Gram K v pour tout v:

Tout d'abord, pour tout a = 1, ...d les matrices de Gram K a associées aux noyaux k 0a sont calculées en utilisant l'équation (1.8),

(K a ) i,i = k 0a (X ai , X ai ).
Ensuite, pour tout v ∈ P, les matrices de Gram K v associées au noyau k v = a∈v k 0a sont calculées comme suit: La valeur de "tol" est fixée par défaut à 1e -8 , mais on peut considérer une valeur plus petite ou plus grande en fonction du noyau choisi et de n.

K v = a∈v K a ,

Algorithmes d'optimisation

RKHS group lasso Afin de résoudre le problème d'optimisation du RKHS group lasso, l'algorithme classique de block coordinate descent est utilisé [START_REF] Boyd | Distributed optimization and statistical learning via the alternating direction method of multipliers[END_REF], [START_REF] Bubeck | Convex optimization: Algorithms and complexity[END_REF]). La minimisation du critère C g (f 0 , θ) se fait à travers chaque groupe v à chaque fois. À chaque étape de l'algorithme, le critère est minimisé en fonction des paramètres du bloc actuel, tandis que les valeurs des paramètres des autres blocs sont fixées à leurs valeurs actuelles. La procédure est répétée jusqu'à convergence.

Dans le package RKHSMetaMod, l'algorithme classique de block coordinate descent pour résoudre le problème d'optimisation du RKHS group lasso est implémenté dans la fonction RKHSgrplasso. Dans la deuxième partie de ce travail, la procédure de calcul de f est mise en oeuvre dans un package R, appelé RKHSMetaMod. Afin d'optimiser le temps de calcul et la mémoire de stockage, toutes les fonctions de ce package ont été écrites en utilisant les bibliothéques GSL et Eigen de C++ à l'exception d'une fonction qui est écrite en R. Elles sont ensuite interfacées avec l'environnement R afin de proposer un package facilement exploitable aux utilisateurs de R. Le package RKHSMetaMod s'applique indifféremment dans le cas où le modèle m est calculable et le cas du modèle de régression. Une étude de simulation est fournie afin de valider la performance des fonctions du package en termes de qualité prédictive de l'estimateur et d'estimation des indices de Sobol. Comme tous les travaux de recherche qui sont menés dans un temps limité, de nombreuses pistes n'ont pas été explorées dans ce travail et il y a plusieurs perspectives à considérer pour une étude plus approfondie. Mentionnons en quelques-unes.

RKHS ridge group sparse

Variables d'entrée non-indépendantes

Dans les deux parties de cette thèse, les variables d'entrée X 1 , ..., X d du modèle m sont supposées indépendantes et leur loi est connue. Sous ces hypothèses, il est possible de construire des espaces d'approximation tels que toute fonction dans ces espaces se décompose selon sa décomposition de Hoeffding. La décomposition est unique et les termes de cette décomposition sont orthogonaux.

Si les variables X 1 , ..., X d ne sont pas indépendantes, il n'y a plus d'orthogonalité entre les termes de la décomposition sur les espaces d'approximation et la décomposition d'une fonction sur ces espaces n'est pas nécessairement unique. Il s'ensuit que la décomposition de la variance donnée à l'équation (1.6) n'est plus valable, ni le calcul des indices de Sobol. Néanmoins, l'approximation du modèle sur un espace fonctionnel selon une décomposition additive peut s'avérer intéressante en pratique, l'estimation du méta-modèle pouvant aider à l'interprétation des effets des variables d'entrée sur la variable de sortie.

Le cas où les variables X 1 , ..., X d ne sont pas indépendantes a été considéré par [START_REF] Koltchinskii | Sparsity in multiple kernel learning[END_REF]. Leur espace d'approximation H est l'espace linéaire engendré (ou linear span (l.s.)) par un dictionnaire d'espaces RKHS H 1 , ..., H N ,

H = l.s. N j=1 H j .
L'espace H est ainsi constitué de toutes les fonctions f qui ont une représentation additive de la forme,

f = N j=1 f j (X), f j ∈ H j , j = 1, ..., N.
(1.26) Concernant la réalisation de l'analyse de sensibilité dans ce cas, comme le calcul des indices de Sobol n'est plus possible, on peut considérer les valeurs de Shapley [START_REF] Shapley | A value for n-person games[END_REF]), voir par exemple [START_REF] Owen | Sobol' indices and shapley value[END_REF], [START_REF] Song | Shapley effects for global sensitivity analysis: Theory and computation[END_REF], [START_REF] Owen | On shapley value for measuring importance of dependent inputs[END_REF], [START_REF] Benoumechiara | Shapley effects for sensitivity analysis with dependent inputs: bootstrap and kriging-based algorithms[END_REF]Elie-Dit-Cosaque (2019), Broto et al. (2019), [START_REF] Iooss | Shapley effects for sensitivity analysis with correlated inputs: comparisons with Sobol' indices, numerical estimation and applications[END_REF].

Généralisation au modèle de régression avec erreur log-concave

Le résultat 1 montre que la majoration du risque dans le cas où les erreurs sont de densité π α ∈ D est la même que celle obtenue dans le cas des erreurs gaussiennes. Cependant, la classe des densités π α est restrictive et il serait intéressant d'obtenir un résultat pour des classes de densité plus grandes, comme les densités log-concaves par exemple.

Comme expliqué à la section 1.2.5, l'une des étapes essentielles de la preuve du résultat repose sur une minoration de type Sudakov de l'espérance du processus empirique, lorsque les variables aléatoires sont supposées non-bornées et nongaussiennes.

Dans le cas gaussien, la minoration de Sudakov s'énonce de la façon suivante [START_REF] Pisier | The volume of convex bodies and Banach space geometry[END_REF]):

Chapter 1. Introduction Soient ε = (ε 1 , ..., ε n ) des variables aléatoires i.i.d. gaussiennes, et pour une fonction g : R |v| → R, v ∈ P appartenant à une classe de fonctions G, soit V n,ε le processus empirique associé au vecteur aléatoire ε défini dans l'équation (1.23). Alors pour tout δ > 0,

1 C log N (δ, G, . ) ≤ nE ε sup g∈G |V n,ε (g)| δ 2 , (1.27) où C est une constante, et N (δ, G, .
) est le nombre de recouvrements de l'espace métrique (G, . ) par des boules de rayon inférieur à δ.

Il reste ensuite à caractériser la complexité de l'espace fonctionnel G pour obtenir une minoration de l'espérance du processus empirique et en déduire le résultat dans la borne de risque.

Dans le cas où les ε = (ε 1 , ..., ε n ) sont de densité π α , le théorème 3.1. de [START_REF] Talagrand | The supremum of some canonical processes[END_REF] établit l'inégalité donnée à l'équation (3.34) d'où on peut déduire la minoration de l'espérance du processus empirique donnée à l'équation (1.24).

Dans le cas de ε non-gaussienne et non-bornée, une minoration de type Sudakov pour les variables aléatoires i.i.d. log-concaves est donnée par [START_REF] Latała | Sudakov-type minoration for log-concave vectors[END_REF]. Une mesure sur R n est log-concave si et seulement si elle a une densité de la forme exp(-φ(x)), où φ : R n → (-∞, ∞] est convexe [START_REF] Borell | Convex measures on locally convex spaces[END_REF]). La minoration de type Sudakov donnée par Latała (2014) est de la forme suivante:

Soient ε = (ε 1 , ..., ε n ) des variables aléatoires i.i.d. log-concaves, alors:

1 K min c 2 δ, log N (2 × max(cδ 1/2 , c 2 δ), G, . ) ≤ nE ε sup g∈G |V n,ε (g)|.
(1.28) où K est une constante universelle, et c = 1/ max(512c , 8) pour c étant une constante universelle.

On n'a pas pu déduire de l'inégalité (1.28) la minoration de type Sudakov adaptée qui conduise à la vitesse de convergence optimal pour l'estimateur RKHS ridge group sparse. Par optimal, on entend la même vitesse de convergence que dans le modèle de régression gaussien (voir Huet and Taupin (2017)). C'est la raison pour laquelle, dans ce travail, les densités π α ∈ D sont considérées. Néanmoins, un travail supplémentaire dans cette direction ainsi qu'une recherche bibliographique mérite d'être effectués.

Chapter 2

Introduction in english

Framework

Consider a phenomenon described by a model m depending on d input variables X = (X 1 , ..., X d ). This model m from R d to R, may be complex including strong nonlinearities and high order interaction effects. In the classical framework of sensitivity analysis, the model m can be calculated in a finite number of points.

When the components of X are independent, the model m can be decomposed as a so-called Hoeffding decomposition. If the law of the components of X is known, this decomposition allows to perform sensitivity analysis, and more precisely to calculate the Sobol indices of m [START_REF] Sobol | Global sensitivity indices for nonlinear mathematical models and their Monte Carlo estimates[END_REF], [START_REF] Saltelli | Sensitivity Analysis[END_REF]). However, the calculation of these indices may be very difficult or even impossible, especially when the number of the input variables d is large [START_REF] Iooss | Revue sur l'analyse de sensibilité globale de modèles numériques[END_REF].

A recent approach is to approximate m by an additive meta-model involving variables X 1 , ..., X d and interactions between them, as proposed by [START_REF] Durrande | Anova kernels and rkhs of zero mean functions for model-based sensitivity analysis[END_REF]. This meta-model, denoted f * , is the orthogonal projection of m on a reproducing kernel Hilbert space (RKHS), denoted H. The space H is associated with a so-called ANOVA kernel which is defined in order to obtain the analytical expression of the terms of the Hoeffding decomposition of the functions of H. As f * is the orthogonal projection of m on H, each term in its decomposition is an approximation of the associated term in the Hoeffding decomposition of m.

When d, the number of the input variables is large, the total number of terms in the Hoeffding decomposition of f * becomes very high. One solution is to calculate a sparse approximation of f * using penalized least-squares criterion as it is done in the non-parametric regression framework.

In this thesis, two frameworks are considered: the classical framework of sensitivity analysis where m(X) is calculable in all points X, and the regression framework where m is unknown and so can not be calculated.

In the second case, for a given X, the value of m(X) with respect to an error term ε is observable. Therefore, we have the observations Y such that, Y = m(X) + σε, σ > 0.

(2.1)

As in the classical framework of sensitivity analysis, the idea is to approximate the Hoeffding decomposition of m by the meta-model f * , and then calculate a sparse estimator of f * using non-parametric estimation approaches. This estimator, denoted f , is the solution of a least-squares minimization problem penalized by a Chapter 2. Introduction in english penalty function that imposes sparsity and smoothness. The construction of the estimator f allows to estimate easily the Sobol indices of m. This thesis consists of a theoretical part and a practical part:

-In the theoretical part, I established the upper bounds of the empirical L 2 risk and the L 2 risk of the estimator f of a regression model as described in Equation (2.1) with error ε that is non-Gaussian and non-bounded. That is, the upper bounds with respect to the L 2 -norm and the empirical L 2 -norm for the distance between the true function m and its estimation f into the RKHS H. This part is presented in Chapter 3.

-In the practical part, I developed an R package, called RKHSMetaMod, for implementing the estimation methods of the meta-model f * of a model m. This package deals both with the case where m is calculable and the case of the regression model. This part is presented in Chapter 4 and Appendix A.

-In Chapter 4, the estimation methods and the algorithms used in the package are described. The performances of the package functions in terms of the predictive quality of the estimator and the estimation of the Sobol indices, are validated by a simulation study.

-In Appendix A, the complete documentation of the package, including detailed explanations of the package functions and the examples of usage of each function of the package, is provided.

The summaries of Chapters 3 and 4 are presented in Sections 2.2 and 2.3, respectively. Before that, several tools that are common to these two Chapters are briefly described. More precisely:

introduction to the sensitivity analysis (see Section 2.1.1),

focus on the variance based methods of global sensitivity analysis (see Section 2.1.2),

introduction to the meta-modelling (see Section 2.1.3),

construction of a meta-model by projection on the reproducing kernel Hilbert spaces (RKHS) (see Section 2.1.4),

the estimation method (see Section 2.1.5).

Introduction to the sensitivity analysis

The sensitivity analysis methods allow to study the relationships between the output and input variables of the model, and measure the effect of each variable or groups of variables on the model output. The underlying goals for sensitivity analysis are model calibration, model validation and assisting with the decision making process. Most of the classical methods and objectives of the sensitivity analysis can be found in [START_REF] Cacuci | Sensitivity and Uncertainty Analysis, Theory[END_REF], [START_REF] Fang | Design and Modeling for Computer Experiments (Computer Science & Data Analysis[END_REF], [START_REF] Dean | Screening: Methods for Experimentation in Industry, Drug Discovery, and Genetics[END_REF]Lewis (2006), de Rocquigny et al. (2008), [START_REF] Saltelli | Global sensitivity analysis: the primer[END_REF], [START_REF] Helton | Uncertainty and Sensitivity Analysis for Models of Complex Systems[END_REF], [START_REF] Saltelli | Sensitivity Analysis[END_REF], [START_REF] Faivre | Analyse de sensibilité et exploration de modèles, Collection Savoir-Faire[END_REF], [START_REF] Borgonovo | Sensitivity analysis: A review of recent advances[END_REF].

The sensitivity analysis procedure implies the computation and analysis of some measures that evaluate the effect of the input variables on the model output. For example, the effect of an input variable on the model output can be evaluated by the amount of variance in the model output caused by that input variable. The sensitivity analysis methods can be classified in two main groups:

Local sensitivity analysis that studies the local impact of the input variables on the output variable. It consists in calculating the gradient of the output variable with respect to the input variables around a chosen value (the mean value of the input variables for example). Numerous methods have been developed to compute the gradient efficiently, including Adjoint modelling [START_REF] Cacuci | Sensitivity and Uncertainty Analysis, Theory[END_REF], [START_REF] Cacuci | Sensitivity and Uncertainty Analysis: Applications to Large-Scale Systems[END_REF]) and Automated Differentiation [START_REF] Griewank | Evaluating Derivatives: Principles and Techniques of Algorithmic Differentiation[END_REF]). Local methods do not fully explore the space of input variables, since they study the impact of small perturbations of input variables (generally one variable at a time) on the output variable.

Global sensitivity analysis calculates the uncertainty of the output variable due to the variations in the input variables or groups of input variables. In contrast to the local sensitivity analysis, this class of methods considers the whole variation range of the input variables. The global sensitivity analysis methods are numerous, see for example [START_REF] Saltelli | Sensitivity Analysis[END_REF] for a good state-of-the-art and [START_REF] Iooss | A review on global sensitivity analysis methods[END_REF] for a review of these methods. Generally, the methods of the global sensitivity analysis which allow to calculate the most used quantitative sensitivity measures can be gathered into two groups:

Regression-based methods are suitable when the model is linear, i.e. if the coefficient of determination R 2 is close to one. The commonly used sensitivity measures in this case are: the standardized regression coefficients, the Pearson correlation coefficients, and partial correlation coefficients. For a non-linear model that is monotonic, these coefficients could be still used to represent the output sensitivities by applying a rank transformation [START_REF] Saltelli | Sensitivity Analysis[END_REF]). When the model is non-linear and non-monotonic these methods fail to produce satisfactory sensitivity measures [START_REF] Saltelli | About the use of rank transformation in sensitivity analysis of model output[END_REF]).

Variance-based methods can be applied to non-linear and non-monotonic models. They consist of decomposing the variance of the model output into parts attributable to each input variable and groups of them (interactions). The sensitivity measures in this case are expressed as the ratio of the variance of each input variable or groups of them over the variance of the model output. The decomposition of variance is meaningful if the input variables are independent from one another [START_REF] Saltelli | On the relative importance of input factors in mathematical models[END_REF]). These methods are widely used as they allow to explore whole variation range of the input variables, accounting for interactions, and non-linear non-monotonic models.

Global sensitivity analysis: variance-based methods

Let us consider a model m depending on d input variables X = (X 1 , ..., X d ) that are independent and distributed with a known law

P X = P 1 ⊗...⊗P d on X = X 1 ×...×X d a subset of R d . The model m from R d to R is square-integrable, i.e. m ∈ L 2 (X , P X ).
In the classical framework of sensitivity analysis, where for each value of X the value of m(X) can be calculated, one may use the method of [START_REF] Sobol | Sensitivity estimates for nonlinear mathematical models[END_REF] to perform sensitivity analysis on m. Let us briefly recall this method.

The independency between the components of X allows to write the model m according to its Hoeffding decomposition [START_REF] Sobol | Sensitivity estimates for nonlinear mathematical models[END_REF], van der Vaart (1998)):

m(X) = m 0 + d a=1 m a (X a ) + a<a m a,a (X a , X a ) + ... + m 1,...,d (X).
(2.

2)

The terms in this decomposition are defined in terms of the conditional expected values:

m 0 = E X (m(X)), m a (X a ) = E X (m(X)|X a ) -m 0 , m a,a (X a , X a ) = E X (m(X)|X a , X a ) -m a (X a ) -m a (X a ) -m 0 ,
and so on for interactions of order higher than two. These terms are known as constant term, main effects, interactions of order two and so on.

Let P be the set of all subsets of {1, ..., d} with dimension 1 to d. For all v ∈ P and X ∈ X , let X v be the vector with components X a , a ∈ v and m v be the function associated with X v in Equation (2.2). Then Equation (2.2) can be expressed as follows:

m(X) = m 0 + v∈P m v (X v ). (2.3)
This decomposition is unique, all the terms m v , v ∈ P are centered, and they are orthogonal with respect to L 2 (X , P X ), i.e.

∀v ∈ P, E X (m v (X v )) = 0, and
∀v, v ∈ P, v = v , E X (m v (X v )m v (X v )) = 0.
The function m as well as all the functions m v in Equation ( 2.3) are squareintegrable. As any two terms of decomposition (2.3) are orthogonal, by squaring (2.3) and integrating it with respect to the distribution of X, a decomposition of the variance of m(X) is obtained as follows:

var(m(X)) = v∈P var(m v (X v )). (2.4)
For any group of variables X v , v ∈ P, the Sobol indices are defined by:

S v = var(m v (X v )) var(m(X)) .
For each v, S v expresses the fraction of variance of m(X) explained by X v . For all v ∈ P, when |v| = 1, the S v are referred to as the first order indices or main effects. When |v| = 2, i.e. v = {a, a } and a = a , they are referred to as the second order indices or the interaction indices of order two (between X a and X a ). And the same holds for |v| > 2.

The total number of the Sobol indices to be calculated is equal to |P| = 2 d -1, which raises exponentially with the number of the input variables d. When d is large, the evaluation of all the indices can be too computationally demanding and even not reachable. For this reason, only the indices of order not higher than two are calculated in practice. However, only first and second order indices may not provide a good information on the model sensitivities. In order to provide a better information on the model sensitivities, [START_REF] Homma | Importance measures in global sensitivity analysis of nonlinear models[END_REF] proposed to calculate the first order and the total indices defined as follows:

Let P a ⊂ P be the set of all the subsets of {1, ..., d} including a, then

S Ta = v∈Pa S v .
For all a ∈ {1, ..., d}, S Ta denotes the total effect of X a . It expresses the fraction of variance of m(X) explained by X a alone and all the interactions of it with the other variables.

The total indices allow to rank the input variables with respect to the amount of their effect on the output variable. However, they do not provide complete information on the model sensitivities as do all the Sobol indices.

The classical computation of the Sobol indices is based on the Monte Carlo methods (see for example: [START_REF] Sobol | Sensitivity estimates for nonlinear mathematical models[END_REF] for the main effect and interaction indices, and [START_REF] Saltelli | Making best use of model evaluations to compute sensitivity indices[END_REF] for the main effect and total indices). These methods are very costly, since they require many thousands of model runs to get precise estimates of the Sobol indices. Thus in the case where d is large, m is complex and the calculation of the variances is numerically complicated or not possible as in the case where the model m is unknown, the methods described above are not applicable.

Another method is to approximate m by a simplified model, called a metamodel, which is much faster to evaluate and to perform sensitivity analysis on it. A meta-model provides additional information than just scalar indices. It provides the approximations of the Sobol indices of m at a lower computational cost, and also a deeper view of the input variables effects on the model output.

Meta-modelling

Meta-modelling consists in building a function which is computationally tractable, easy to interpret and has good prediction qualities. Let {m(X i )} n i=1 be the outputs of n evaluations of the model m based on an experimental design {X i } n i=1 . In this context, a meta-model is an approximation of the model m which is constructed based on the experimental design {X i } n i=1 and the outputs {m(X i )} n i=1 . There exists different approaches of meta-modelling, see [START_REF] Sacks | Design and analysis of computer experiments[END_REF], [START_REF] Friedman | Multivariate adaptive regression splines[END_REF], [START_REF] Breiman | Random forests[END_REF], [START_REF] Friedman | Greedy function approximation: A gradient boosting machine[END_REF], [START_REF] Kennedy | Bayesian calibration of computer models[END_REF], [START_REF] Oakley | Probabilistic sensitivity analysis of complex models: a bayesian approach[END_REF], [START_REF] Storlie | Multiple predictor smoothing methods for sensitivity analysis: Description of techniques[END_REF], [START_REF] Storlie | Implementation and evaluation of nonparametric regression procedures for sensitivity analysis of computationally demanding models[END_REF], [START_REF] Storlie | Surface estimation, variable selection, and the nonparametric oracle property[END_REF] for different examples, and Touzani ( 2011) for an overview.

In the framework of the global sensitivity analysis, one may consider a metamodel that has the additive decomposition, and that is candidate to approximate the Hoeffding decomposition of m. This meta-model allows to perform global sensitivity analysis and calculate the Sobol indices of m, even of high order. By a function that has the additive decomposition, we mean a function f from X ⊂ R d to R that is defined as follows:

f = f 0 + v∈P f v (X v ), E X (f v (X v )) = 0, E X (f v (X v )f v (X v )) = 0, ∀v, v ∈ P, v = v ,
where f 0 is a constant, and the functions f v are supposed to belong to some functionnal spaces.

Among the meta-modelling approaches proposed in the literature, the decomposition based on polynomial Chaos [START_REF] Wiener | The homogeneous chaos[END_REF], [START_REF] Schoutens | Stochastic Processes and Orthogonal Polynomials, Lecture Notes in Statistics[END_REF]) can be used to approximate the Hoeffding decomposition of m [START_REF] Sudret | Global sensitivity analysis using polynomial chaos expansions[END_REF]).

The principle of the polynomial Chaos is to project m onto a basis of orthonormal polynomials. The Chaos representation of m is written as [START_REF] Soize | Physical systems with random uncertainties: Chaos representations with arbitrary probability measure[END_REF]):

m(X) = ∞ j=0 h j φ j (X), (2.5)
where {h j } ∞ j=0 are the coefficients, and {φ j } ∞ j=0 are multivariate orthonormal polynomials associated with X that are determined according to the distribution of the components of X. In practice, expansion (2.5) shall be truncated for computational purposes, and the model m may be approximated by:

m(X) ≈ vmax j=0 h j φ j (X),
where v max is determined using a truncation scheme. In this approach, the Sobol indices are obtained by summing up the squares of the suitable coefficients. [START_REF] Blatman | Adaptive sparse polynomial chaos expansion based on least angle regression[END_REF] proposed a method for truncating the polynomial Chaos expansion and an algorithm based on least-angle regression for selecting the terms in the expansion.

In this method, according to the distribution of the components of X a unique family of orthonormal polynomials {φ j } ∞ j=0 is determined. However, this family may not be necessarily the best functional basis to approximate m well.

Another approach to construct meta-models is given by Gaussian Process (GP) modelling [START_REF] Welch | Screening, predicting, and computer experiments[END_REF], [START_REF] Oakley | Probabilistic sensitivity analysis of complex models: a bayesian approach[END_REF], [START_REF] Kleijnen | Design and Analysis of Simulation Experiments[END_REF][START_REF] Kleijnen | Kriging metamodeling in simulation: A review[END_REF], [START_REF] Marrel | Calculations of sobol indices for the gaussian process metamodel[END_REF], [START_REF] Durrande | Additive covariance kernels for high-dimensional gaussian process modeling[END_REF], [START_REF] Gratiet | A bayesian approach for global sensitivity analysis of (multifidelity) computer codes[END_REF]). The principle is to consider that the prior knowledge about the function m(X), can be modelled by a GP Z(X) with a mean m Z (X) and a covariance kernel k Z (X, X ). To perform sensitivity analysis from a GP model one may replace the true model m(X) with the mean of the conditional GP, and deduce the Sobol indices from it. Most of the time, with GP, the Sobol indices are estimated using Monte Carlo methods.

A review on the meta-modelling based on polynomial Chaos and GP is presented in Le [START_REF] Gratiet | Metamodel-Based Sensitivity Analysis: Polynomial Chaos Expansions and Gaussian Processes[END_REF]. [START_REF] Durrande | Anova kernels and rkhs of zero mean functions for model-based sensitivity analysis[END_REF] considered a class of functional approximation methods similar to the GP regression and obtained a meta-model that satisfies the properties of the Hoeffding decomposition. They proposed to approximate m by functions belonging to a RKHS H which is constructed as a direct sum of Hilbert spaces, such that the projection of m onto H is an approximation of the Hoeffding decomposition of m.

In the regression framework, when the values of {m(X i )} n i=1 can not be calculated, one may use the projection methods on a functional basis to estimate a metamodel of m. This meta-model is estimated based on the observations {(X i , Y i )} n i=1 by using non-parametric estimation approaches. The estimator obtained can be used then to estimate the Sobol indices of m. Huet and Taupin (2017) considered the same approximation functional spaces as [START_REF] Durrande | Anova kernels and rkhs of zero mean functions for model-based sensitivity analysis[END_REF], and proposed an estimator of a meta-model that approximates the Hoeffding decomposition of m. They deduced from this estimated meta-model, estimators for the Sobol indices of m. This approach is presented in more details in the following.

Meta-models based on the reproducing kernel Hilbert spaces (RKHS)

Let us begin this Section with a brief introduction to the RKHS. The method of [START_REF] Durrande | Anova kernels and rkhs of zero mean functions for model-based sensitivity analysis[END_REF] to construct the RKHS, and the definition of the meta-model f * that approximates the Hoeffding decomposition of m are presented in Sections 2.1.4.2 and 2.1.4.3, respectively.

Introduction to the RKHS

Let H be a Hilbert space of real valued functions on a set X . The space H is a RKHS if for all X ∈ X the evaluation functionals

L X :H → R f → f (X),
are continuous. The Riesz representation Theorem ensures the existence of an unique element k X (.) in H verifying the following property:

∀X ∈ X , ∀f ∈ H, f (X) = L X (f ) = f, k X H ,
where ., . H denotes the inner product in H.

It follows that for all X ,X in X , and k X (.), k X (.) in H, we have,

k X (X ) = L X (k X ) = k X , k X H . (2.6)
This allows to define the reproducing kernel of H as follows:

k : X × X → R (X, X ) → k X (X ).
The reproducing kernel k(X, X ) satisfies the following properties:

-It is symmetric. Indeed, by definition of k(., .) and thanks to the property (2.6), we have:

k(X, X ) = k X (X ) = k X , k X H = k X (X) = k(X , X). -For any n ∈ N, {X i } n i=1 ∈ X and {c i } n i=1 ∈ R, we have: n i=1 n j=1 c i c j k(X i , X j ) = n i=1 n j=1 c i k(X i , .), c j k(X j , .) H , = n i=1 c i k(X i , .) 2 H ≥ 0.
Thus, k(X, X ) is positive definite.

For more background on RKHS, we refer to various standard references such as [START_REF] Aronszajn | Theory of reproducing kernels[END_REF], [START_REF] Saitoh | Theory of reproducing kernels and its applications[END_REF][START_REF] Berlinet | Reproducing Kernel Hilbert Spaces in Probability and Statistics[END_REF][START_REF] Berlinet | Reproducing Kernel Hilbert Spaces in Probability and Statistics[END_REF].

RKHS construction and Hoeffding decomposition

The idea is to construct an RKHS including the functions that have the additive decomposition and that are candidate to approximate the Hoeffding decomposition of m. To do so, we use the method of [START_REF] Durrande | Anova kernels and rkhs of zero mean functions for model-based sensitivity analysis[END_REF] that we recall briefly in the following.

Let X = X 1 × . . . × X d be a subset of R d . For each a ∈ {1, • • • , d}, we choose a RKHS H a and its associated kernel k a defined on the set X a ⊂ R such that the two following properties are satisfied:

(i) k a : X a × X a → R is P a ⊗ P a measurable, (ii) E Xa k a (X a , X a ) < ∞.
The property (ii) depends on the kernel k a , a = 1, ..., d and the distribution of X a , a = 1, ..., d. It is not very restrictive since it is satisfied, for example, for any bounded kernel.

The RKHS H a can be decomposed as a sum of two orthogonal sub-RKHS,

H a = H 0a ⊥ ⊕ H 1a ,
where H 0a is the RKHS of zero mean functions,

H 0a = f a ∈ H a : E Xa (f a (X a )) = 0 ,
and H 1a is the RKHS of constant functions,

H 1a = f a ∈ H a : f a (X a ) = C .
The kernel k 0a associated with the RKHS H 0a is defined by:

k 0a (X a , X a ) = k a (X a , X a ) - E U ∼Pa (k a (X a , U ))E U ∼Pa (k a (X a , U )) E (U,V )∼Pa⊗Pa k a (U, V ) . (2.7) Let k v (X v , X v ) = a∈v k 0a (X a , X a ), then the ANOVA kernel k(., .) is defined as follows: k(X, X ) = d a=1 1 + k 0a (X a , X a ) = 1 + v∈P k v (X v , X v ).
For H v being the RKHS associated with the kernel k v , the RKHS associated with the ANOVA kernel is then defined by,

H = d a=1 1 ⊥ ⊕ H 0a = 1 + v∈P H v ,
where ⊥ denotes the L 2 inner product.

According to this construction, any function f ∈ H satisfies the following decomposition:

f (X) = f, k(X, .) H = f 0 + v∈P f v (X v ),
which is the Hoeffding decomposition of f .

The regularity properties of the RKHS H constructed as described above, depend on the set of the kernels (k a , a = 1, ..., d). This method allows to choose different approximation spaces independently of the distribution of the input variables X 1 , ..., X d , by choosing different sets of kernels. While as mentioned earlier, in the meta-modelling approach based on polynomial Chaos expansion, according to the distribution of the input variables X 1 , ..., X d a unique family of orthonormal polynomials {φ j } ∞ j=0 is determined. Here, the distribution of the components of X occurs only for the orthogonalization of the spaces H v , v ∈ P, and not in the choice of the RKHS, under the condition that properties (i) and (ii) are satisfied. This is one of the main advantages of this method compared to the method based on the truncated polynomial Chaos expansion where the smoothness of the approximation is handled only by the choice of the truncation [START_REF] Blatman | Adaptive sparse polynomial chaos expansion based on least angle regression[END_REF]).

f * = arg min f ∈H m -f 2 2 = arg min f ∈H E X (m(X) -f (X)) 2 .
The function

f * = f * 0 + v∈P f * v
is the approximation of m on the RKHS H, and its Hoeffding decomposition is an approximation of the Hoeffding decomposition of m. Therefore, for each v ∈ P the function f * v approximates the function m v in Equation ( 2.3).

The number of functions f * v is related to the cardinal of P, equal to 2 d -1, that may be huge. So, the idea is to calculate a sparse estimator of f * as an estimator of m using non-parametric approaches.

Estimation method

Let us consider the regression model defined in Equation ( 2.1),

Y = m(X) + σε, σ > 0.
The unknown function m is approximated by the meta-model f * which is then estimated by a sparse estimator f . This estimator f , based on n observations

{(Y i , X i )} n i=1
, minimizes a penalized criterion. The penalty function deals both with the non-parametric nature of the problem, and the possibly large number of functions that have to be estimated.

Before describing the method to calculate f , let us recall some methods related to the estimation in a non-parametric additive regression model. Some authors approximate m by a function that has a sparse univariate additive decomposition of the form,

f (X) = f 0 + a∈S f a (X a ) with |S| < d, (2.8)
where f 0 is a constant, and for all a ∈ S the f a are unknown smooth functions fitted from the data. [START_REF] Ravikumar | Sparse additive models[END_REF] considered a Hilbert space H of functions that have univariate additive form. Their functional approximation space H is constructed as a direct sum of Hilbert spaces, i.e.

H = d a=1 H a ,
where for all a ∈ {1, ..., d}, H a is the Hilbert subspace of L 2 (X a , P a ) of P a measurable univariate functions f a with zero mean. In order to control smoothness and to enforce sparsity in the univariate additive decomposition, they proposed the Sparse Additive Models (SpAM) method. Their method is based on the minimization of the least-squares criterion penalized with a penalty function defined as follows, [START_REF] Meier | High-dimensional additive modeling[END_REF] proposed an estimator which lies in the space of natural cubic splines. Their method is based on the minimization of the least-squares criterion penalized with a penalty function of the form,

λ d a=1 (f a (X a )) 2 dX a , λ ∈ R + .
d a=1 λ 1 f a 2 n + λ 2 (f a (X a )) 2 dX a , λ 1 , λ 2 ∈ R + ,
where f a

2 n = 1 n n i=1 f 2 a (X ai ).
Their penalty function consists of two parts: the first part controls the sparsity and the second part controls the smoothness. [START_REF] Raskutti | Minimax-optimal rates for sparse additive models over kernel classes via convex programming[END_REF] considered several functional approximation spaces including polynomials, splines and Sobolev. Their method is based on the minimization of the least-squares criterion penalized with a penalty function of the form,

γ f n,1 + µ f H,1 , γ, µ ∈ R + , (2.9)
where f n,1 = d a=1 f a n , and f H,1 = d a=1 f a Ha . In their penalty function, the first part controls the sparsity and the second part controls the smoothness.

Performing global sensitivity analysis on an univariate additive model leads to obtain only the first order Sobol indices, which may not provide a good information on the model sensitivities. The interactions between variables that may affect the relationship between Y and X are completely ignored in this setting.

In order to include the interaction effects, one may approximate m by a function that has a sparse multivariate additive decomposition of the form,

f (X) = f 0 + v∈S f v (X v ) with |S| < |P|,
which is a generalization of the sparse univariate additive decomposition defined in Equation (2.8).

In the framework of smoothing spline ANOVA [START_REF] Wahba | Spline Models for Observational Data[END_REF], [START_REF] Friedman | Multivariate adaptive regression splines[END_REF], [START_REF] Wahba | Smoothing spline anova for exponential families, with application to the wisconsin epidemiological study of diabetic retinopathy[END_REF]), [START_REF] Lin | Component selection and smoothing in multivariate nonparametric regression[END_REF] proposed the Component Selection and Smoothing Operator (COSSO) method. Their method is based on the minimization of the least-squares criterion penalized with a penalty function that is the combination of the l 1 -norm with the Hilbert norm. The implementation of COSSO is carried out over the second-order Sobolev spaces. [START_REF] Kandasamy | Additive approximations in high dimensional nonparametric regression via the salsa[END_REF] proposed the Shrunk Additive Least Squares Approximation (SLASA) method which is based on the minimization of the least-squares criterion penalized with the sum of squared RKHS norms. Their estimator is a Chapter 2. Introduction in english v max -th order multivariate additive function containing d vmax terms in its expansion. The value of v max is determined using a cross validation procedure. Huet and Taupin (2017) considered an estimator of a meta-model that approximates the Hoeffding decomposition of m defined in Equation ( 2.3). Their estimator is the solution of least-squares minimization penalized by the penalty function defined in Equation (2.9) adapted to the multivariate setting,

γ f n + µ f H with f H = v∈P f v Hv , and f n = v∈P f v n .
Their method, called ridge group sparse, estimates the groups v that are suitable for predicting the meta-model f * , and the relationship between f * v and X v for each group v ∈ P. The obtained estimator, called RKHS ridge group sparse estimator, is used then to estimate the Sobol indices of m. This method makes it possible to estimate the Sobol indices for all groups in the support of the RKHS ridge group sparse estimator, including the interactions of possibly high order, a point known to be difficult in practice.

In order to obtain a sparse estimator of the meta-model f * in this work, we use the method proposed by Huet and Taupin (2017), which allows also to obtain the estimators of the Sobol indices of m. We recall this method and the method to estimate the Sobol indices of m in the next Section and Section 2.1.5.2, respectively.

Ridge group sparse procedure and associated estimator

For all v ∈ P, let X v be the matrix of variables corresponding to the v-th group,

X v = (X vi , i = 1, ..., n, v ∈ P) ∈ R n×|P| .
For any f ∈ H such that f = f 0 + v∈P f v , and for some tuning parameters γ v , µ v , v ∈ P, the ridge group sparse criterion is defined as follows:

L(f ) = 1 n n i=1 Y i -f 0 - v∈P f v (X vi ) 2 + v∈P γ v f v n + v∈P µ v f v Hv , (2.10)
where f v n is the empirical L 2 -norm of f v defined by the sample {X vi } n i=1 as follows:

f v 2 n = 1 n n i=1 f 2 v (X vi ).
The penalty function in the criterion L(f ) is the sum of the Hilbert norm and the empirical norm, which allows to select few terms in the additive decomposition of f over sets v ∈ P. Moreover, the Hilbert norm favours the smoothness of the estimated f v , v ∈ P.

Let us define the set of functions,

F = f : f = f 0 + v∈P f v , with f v ∈ H v , and f v Hv ≤ r v , r v > 0 . (2.11)
Then, the RKHS ridge group sparse estimator of m is defined by,

f = arg min f ∈F L(f ).
(2.12)

According to the Representer Theorem [START_REF] Kimeldorf | A correspondence between bayesian estimation on stochastic processes and smoothing by splines[END_REF]), the nonparametric functional minimization problem (2.12) is equivalent to a parametric minimization problem. Indeed, the solution of the minimization problem (2.12) belonging to the RKHS H is written as f = f 0 + v∈P f v , where for some matrix θ = (θ vi , i = 1, ..., n, v ∈ P) ∈ R n×|P| we have for all v ∈ P,

f v (.) = n i=1 θ vi k v (X vi , .).
Let . be the Euclidean norm in R n , and for each v ∈ P, let K v be the n × n Gram matrix associated with the kernel k v (., .), i.e.

(K v ) i,i = k v (X vi , X vi ). Let also K 1/2 v
be the matrix that satisfies t(K

1/2 v )K 1/2 v = K v ,
and let f 0 and θ be the minimizers of the following penalized least-squares criterion:

C(f 0 , θ) = Y -f 0 I n - v∈P K v θ v 2 + √ n v∈P γ v K v θ v + n v∈P µ v K 1/2 v θ v .
(2.13)

Then, the estimator f defined in Equation (2.12) satisfies,

f (X) = f 0 + v∈P f v (X v ) with f v (X v ) = n i=1 θ vi k v (X vi , X v ). (2.14)
As criterion C(f 0 , θ) is convex and separable, one may calculate θ using a block coordinate descent algorithm [START_REF] Boyd | Distributed optimization and statistical learning via the alternating direction method of multipliers[END_REF][START_REF] Bubeck | Convex optimization: Algorithms and complexity[END_REF]).

Remark 2.1.1 The constraint f v Hv ≤ r v is not taken into account in the parametric minimization problem. This constraint is crucial for theoretical properties but the value of r v is unknown and has no practical usefulness.

Estimation of the Sobol indices of m

The variance of the function m is estimated by the variance of the estimator f . As the estimator f belongs to the RKHS H, it admits the Hoeffding decomposition and,

var( f (X)) = v∈P var( f v (X v )),
where for all v ∈ P,

var( f v (X v )) = E X ( f 2 v (X v )) = f v 2 2 .
In order to reduce the computational cost in practice, one may estimate the variances of f v (X v ), v ∈ P by their empirical variances.

Let

f v. be the empirical mean of { f v (X vi )} n i=1 , f v. = 1 n n i=1 f (X vi ), then var( f v (X v )) = 1 n -1 n i=1 ( f v (X vi ) -f v. ) 2 .
For the groups v that belong to the support of f , the estimators of the Sobol indices of m are defined by,

S v = var( f v (X v )) v∈P var( f v (X v ))
, and for the groups v that do not belong to the support of f , we have S v = 0.

2.2 Summary of Chapter 3

Objectives and results

For an estimator f of a model m let R(m, f ) be its risk. The risk R(m, f ) is a measure that characterizes the precision of the estimator f and that can be expressed as a function of the bias and the variance of f :

R(m, f ) = (bias( f )) 2 + var( f ).
Thus, the quality of the estimator f can be measured by its risk. We consider the empirical L 2 risk of the estimator f , i.e. when R(m, f ) = mf 2 n , and the L 2 risk of the estimator f , i.e. when R(m, f ) = mf 2 2 . We are interested in non-asymptotic properties of the estimator f , in the sense that the number of observations n is not assumed to tend to infinity. So, our results are valid for all n with a high probability. In particular, we establish the upper bounds of the risk R(m, f ) of the form,

R(m, f ) ≤ C inf f ∈F {R(m, f ) + r n (f )}, (2.15)
where C is a constant, and F is the approximation space.

Let f be the function in F such that the infimum of the right hand side of the inequality (2.15) is realized. The term R(m, f ) is the bias term which depends on the choice of the approximation space. The term r n (f ) is the variance term which has to decrease with n. It gives the rate of convergence of the estimator f , i.e. the speed at which the estimator f approaches the true function m. The variance term depends on the regularity of the kernels k v , v ∈ P, the number of terms involved in the decomposition of the function f on the approximation space, the number of input variables d, and the number of observations n.

In the Gaussian regression framework, i.e. when ε in Equation ( 2.1) is distributed as a centered Gaussian random variable, Huet and Taupin (2017) established the upper bounds of the empirical L 2 risk and the L 2 risk of the RKHS ridge group sparse estimator.

In this Chapter, we consider the regression framework with error ε that is non-Gaussian and non-bounded. In this context, the objective is to establish the risk upper bounds of the RKHS ridge group sparse estimator of the form (2.15) with the same rate of convergence as in the Gaussian regression framework. The upper bounds of the empirical L 2 risk and the L 2 risk of the estimator f are presented in Result 1 and Result 2, respectively.

Presentation of the model

Consider the regression model defined in Equation ( 2.1),

Y = m(X) + σε, σ > 0.
The input variables X = (X 1 , ..., X d ) are independent and have a known law P X = d a=1 P a on X = d a=1 X a , a compact subset of R d . The function m : R d → R is unknown, maybe complex, and it is assumed to be square-integrable.

Let D be the set of densities,

D = π α : π α (x) = a α exp(-|x| α ), with (a α ) -1 = R exp(-|x| α )dx, α > 2 .
(2.16)

In this Chapter, we assume that the error term ε is equal to Z/σ α , where Z is a random variable with density π α ∈ D and σ 2 α = var(Z).

Main results

Let us begin with some notations, properties and assumptions that are needed to state Result 1 and Result 2.

Notations

For a function f ∈ H, let S f be its support,

S f = {v ∈ P : f v = 0}. Each kernel k v , v ∈ P is associated with an integral operator T kv from L 2 (X v , P v ) to L 2 (X v , P v ) defined by: ∀f ∈ L 2 (X v , P v ), T kv (f ) = Xv k v (., t)f (t)dP v (t).
For each v ∈ P, let ω v,1 ≥ ω v,2 ≥ ... ≥ 0 be the eigenvalues of the integral operator T kv . Let us define the function Q n,v (t) for some positive t as follows:

Q n,v (t) = 5 n ≥1 min(t 2 , ω v, ).
For some ∆ > 0 let ν n,v be defined by:

ν n,v = inf t Q n,v (t) ≤ ∆t 2 .
(2.17)

For each v ∈ P, ν n,v refers to the minimax optimal rate for L 2 (X , P X )estimation in the RKHS H v [START_REF] Mendelson | Geometric parameters of kernel machines[END_REF]).

Remark 2.2.1 The rate ν n,v , v ∈ P, depends on the regularity of the RKHS via the decreasing rate of the eigenvalues {ω v, } ∞ =1 . When RKHS is of high regularity, i.e. when the eigenvalues {ω v, } ∞ =1 decrease quickly, then the rate ν n,v , v ∈ P will be close to the parametric rate of convergence (see Section 3.3.1 of Chapter 3).

Properties

The RKHS construction as described in Section 2.1.4.2 insures that the following properties are satisfied: P1 For all v ∈ P, the functions f v ∈ H v are centered and are square-integrable,

E X (f v (X v )) = 0 and E X (f 2 v (X v )) < ∞. P2 For all v, v ∈ P, v = v , the functions f v ∈ H v and f v ∈ H v are orthogonal with respect to L 2 (X , P X ), E X (f v (X v )f v (X v )) = 0.
Assumptions A1 For all v ∈ P, the functions f v ∈ H v are uniformly bounded,

∃R > 0 such that f v ∞ = sup Xv |f v (X v )| ≤ R.
This assumption is satisfied as soon as the kernel k v is bounded on the compact set X . Indeed,

f v ∞ ≤ sup X∈X k v (X v , X v ) f v Hv .
For each v ∈ P, let us consider the quantity λ n,v defined by:

λ n,v = max ν n,v , d n .
(2.18)

The regularization parameters µ v and γ v involved in the criterion (2.10) are chosen as follows:

A2 For some constant C 1 > 10 + 4∆,

∀v ∈ P, µ v = C 1 λ 2 n,v , γ v = C 1 λ n,v .
A3 There exists positive constants C 2 , C 3 , and 0 < β < 1/α such that the following conditions are satisfied:

∀v ∈ P, nλ 2 n,v ≥ -C 2 log λ n,v , (2.19)
and

∀f ∈ F, v∈S f λ 2 n,v ≤ C 3 n 2β-1 . (2.20)
Chapter 3 presents the two following results.

Result 1: upper bound of the empirical L 2 risk of the RKHS ridge group sparse estimator

Consider the regression model described in Section 2.2.

2 with σ = 1. Let {(Y i , X i )} n i=1
be a n-sample with the same law as (Y, X), and let {ε i } n i=1 be the random errors that are independent and identically distributed (i.i.d.) like ε. Let also the RKHS ridge group sparse estimator f be defined by (2.12) with r v = 1 in (2.11). Under the assumptions A1, A2, and A3, there exists a positive constant C and 0 < η < 1 (η tends to 0 as n increases) such that,

m -f 2 n ≤ C inf f ∈F m -f 2 n + v∈S f (µ v + γ 2 v ) ,
with probability greater than 1 -η.

Let us comment on this Result 1:

R1 Let f be the function in F such that the infimum of the right hand side of the oracle inequality is realized. The term mf 2 n is the usual bias term. It quantifies both the approximation properties of the RKHS H, and the biasvariance trade-off.

R2 This result is similar to the one obtained in the Gaussian regression model at the cost of the additional Assumption (2.20). This assumption allows to obtain the same rate of convergence for the RKHS ridge group sparse estimator as in the Gaussian regression model (see Huet and Taupin (2017)). However, it implies some restrictions on the regularity of the RKHS H. Indeed, as for all v ∈ P, λ n,v ≥ ν n,v (see Equation (2.18)), it follows that v∈S f ν 2 n,v ≤ C 3 n 2β-1 , which implies some restrictions on the regularity of the RKHS: if β is small, which will be the case if α is large, then the RKHS should be of high regularity.

R3 By Equation (2.18), we also have that for all v ∈ P, λ n,v ≥ d/n. This assumption allows to control the probability of the |P| events (see Equation (3.48) of Chapter 3), where log(|P|) is of order d.

R4

The Result 1 can be generalized to the case where σ = 1 in Equation ( 2.1), and where r v = 1 in Equation (2.11). We refer to Remark 3.3.5 of Chapter 3 for a brief demonstration of this point.

Result 2: upper bound of the L 2 risk of the RKHS ridge group sparse estimator Under the same assumptions as Result 1, we have with high probability for some positive constant C that,

m -f 2 2 ≤ C inf f ∈F m -f 2 n + m -f 2 2 + v∈S f (µ v + γ 2 v ) , Remark 2.2.2
The Result 2 can be generalized to the case where σ = 1 in Equation (2.1), and where r v = 1 in Equation (2.11) (see Remark 3.3.6 of Chapter 3 for more details about this point).

Rate of convergence

Under the same assumptions as Result 1, we have

m -f 2 n ≤ C inf f ∈F m -f 2 n + v∈S f ν 2 n,v + d|S f | n .
This inequality highlights that the upper bound is relevant when the infimum is reached for functions f that have a sparse decomposition in H, i.e. |S f | is small, and when d is small face to n. When d is large, the decomposition of functions in H should be limited to interactions of a limited order, so that the number of elements in the estimated meta-model is of order smaller than d r for some small r, say r = 2 for example. In such a case, the cardinality of P will be smaller than d 2 .

As we mentioned in Remark R3, the assumption λ n,v ≥ d/n is needed to control the value log(|P|), which will be now smaller than 2 log(d). Therefore, the value d in the definition of λ n,v (see Equation (2.18)) as well as the term d|S f |/n in the infimum above will be replaced by 2 log(d) and 2 log(d)|S f |/n, respectively.

Related works

Several authors studied the theoretical properties of estimators similar to the RKHS ridge group sparse estimator. Let us briefly review their framework and their results. [START_REF] Meier | High-dimensional additive modeling[END_REF] considered an estimator similar to the RKHS ridge group sparse estimator. Instead of adding two separate sparsity and smoothness penalties, they combine the two terms into a single sparsity and smoothness penalty. In the fixed design regression framework with error ε that is distributed as a sub-Gaussian random variable, they established the empirical risk upper bounds for the estimation of m onto the set of univariate additive functions. Afterwards, [START_REF] Raskutti | Minimax-optimal rates for sparse additive models over kernel classes via convex programming[END_REF] showed (in Section 3.4. of their paper) that the convergence rate of this estimator is sub-optimal.

Koltchinskii and Yuan (2010) considered a ridge group sparse type estimator defined on a set of additive functions where each term belongs to a RKHS. They do not assume that the input variables X 1 , ..., X d are independent, nor that there is orthogonality between the RKHS spaces. Instead, they introduce some characteristics related to the degree of dependence of the RKHS spaces which insures almost orthogonality between these spaces. Under a global boundedness condition, they established upper bounds on the excess risk of their estimator by assuming that the function m has a sparse representation. A global boundedness condition means that the quantity sup f ∈H sup X∈X |f (X)| is assumed to be bounded independently of dimension d. Their results are valid for a large class of loss functions called losses of quadratic type which satisfy some defined boundedness conditions on the support of the output variable Y . Section 2.1. of their paper provides several examples of the framework for applying their results. Note that, the quadratic loss function in the case where Y is non-bounded does not belong to the class of the losses of quadratic type. The proofs of their results rely on the elementary empirical and Rademacher process methods such as symmetrization and concentration inequalities for Rademacher processes and Bernstein type exponential bounds. [START_REF] Raskutti | Minimax-optimal rates for sparse additive models over kernel classes via convex programming[END_REF] assumed that the function m has a sparse univariate additive representation (as defined in Equation (2.8)) such that each univariate function lies in a RKHS. They proposed the ridge group sparse procedure to calculate the estimator of m, and studied the theoretical properties of their estimator in the Gaussian regression framework. They provided upper bounds for the empirical L 2 and the L 2 risks and a lower bound for the L 2 risk of their estimator over spaces of sparse additive models, including polynomials, splines and Sobolev classes.

Huet and Taupin (2017) studied the theoretical properties of the RKHS ridge group sparse estimator, in the Gaussian regression framework. They derived upper bounds of the empirical L 2 risk and the L 2 risk of the RKHS ridge group sparse estimator, i.e. the upper bounds with respect to the L 2 -norm and the empirical L 2 -norm for the distance between the true function m and its estimation f into the RKHS H. [START_REF] Raskutti | Minimax-optimal rates for sparse additive models over kernel classes via convex programming[END_REF] and Huet and Taupin (2017) do not assume the global boundedness condition. Instead, they consider Assumption A1 where for all v ∈ P, sup Xv |f v (X v )| is bounded. The proofs of their results rely on the probabilistic methods of the empirical Gaussian processes such as concentration inequalities and Sudakov minoration [START_REF] Pisier | The volume of convex bodies and Banach space geometry[END_REF], [START_REF] Massart | About the constants in talagrand's concentration inequalities for empirical processes[END_REF], [START_REF] Van De Geer | Empirical Processes in M-Estimation, Cambridge Series in Statistical and Probabilistic Mathematics[END_REF], [START_REF] Ledoux | The Concentration of Measure Phenomenon, Mathematical surveys and monographs[END_REF]), as well as the results on the Rademacher complexity of kernel classes [START_REF] Mendelson | Geometric parameters of kernel machines[END_REF], [START_REF] Bartlett | Local rademacher complexities[END_REF]).

Technical tools for the proofs

In this work, the upper bounds of the empirical L 2 risk and the L 2 risk of the RKHS ridge group sparse estimator are provided, in the regression framework where the error ε is non-Gaussian and non-bounded, and by considering a penalized least-squares criterion. In this case the conditions assumed in [START_REF] Koltchinskii | Sparsity in multiple kernel learning[END_REF] are not satisfied, and the usual probabilistic methods of the empirical Gaussian processes such as concentration inequalities and Sudakov minoration do not apply. The proofs of our results require different mathematical tools from those used in the past works:

a Sudakov type minoration that for the non-Gaussian and non-bounded random variables, a concentration bound for the lower and upper tails of a convex function of the non-Gaussian and non-bounded random variables.

To the best of our knowledge, in our context of non-Gaussian and non-bounded errors, and with the least-squares criterion, the only Sudakov type minoration which allows to obtain the same rate of convergence for the RKHS ridge group sparse estimator as in the Gaussian regression framework (see Huet and Taupin (2017)), is the one obtained by [START_REF] Talagrand | The supremum of some canonical processes[END_REF]. The minoration obtained by [START_REF] Talagrand | The supremum of some canonical processes[END_REF] is specific to the densities π α (see Equation (2.16)). This is the reason why the class of densities D is considered in this work. The Sudakov type minoration adapted to our work is derived from Theorem 3.1. in [START_REF] Talagrand | The supremum of some canonical processes[END_REF]. Let us recall this minoration.

Let ε = (ε 1 , ..., ε n ) be i.i.d. random variables distributed with density π α ∈ D, and for a function g : R |v| → R, v ∈ P belonging to a class of functions G, let V n,ε be the empirical process associated with the random vector ε,

V n,ε (g) = 1 n n i=1 ε i g(X v,i ). (2.21)
Then, for all δ > 0,

1 K log N (δ, G, . ) ≤ ( 2nE ε sup g∈G |V n,ε (g)| δ ) 2 1 [2nEε sup g∈G |Vn,ε(g)|,∞) (δ) + ( 2nE ε sup g∈G |V n,ε (g)| δ ) α 1 (0,2nEε sup g∈G |Vn,ε(g)|] (δ), (2.22)
where K is a constant that depends on α only, N (δ, G, . ) is the δ-covering number of the metric space (G, . ), and

1 A : A → {0, 1} is the indicator function of A ⊂ A, 1 A (a) = 1 if a ∈ A, 0 if a / ∈ A.
Concerning the concentration bound, it is shown that the distribution functions associated with the densities π α ∈ D belong to a class of distribution functions defined by [START_REF] Adamczak | Logarithmic sobolev inequalities and concentration of measure for convex functions and polynomial chaoses[END_REF], for which the log-Sobolev inequality [START_REF] Gross | Logarithmic sobolev inequalities[END_REF]) is satisfied (see Lemma 3.4.2 in Chapter 3). [START_REF] Shu | A characterization of a class of convex log-sobolev inequalities on the real line[END_REF] provided bounds for the lower and upper tails of convex functions of independent random variables which satisfy the log-Sobolev inequality. As the distribution functions associated with the densities π α ∈ D satisfy the log-Sobolev inequality, the concentration inequality derived by [START_REF] Shu | A characterization of a class of convex log-sobolev inequalities on the real line[END_REF]

Summary of Chapter 4

Objectives and results

An R package, called RKHSMetaMod, is developed to implement the ridge group sparse procedure described in Section 2.1.5.1. This package allows to: calculate reproducing kernels as described in Section 2.1.4.2, and their associated Gram matrices, implement the RKHS ridge group sparse procedure and a special case of it called RKHS group lasso procedure (when γ v = 0, v ∈ P in criterion (2.13)) in order to estimate the terms f * v in the Hoeffding decomposition of f * leading to an estimation of the function m, choose the tuning parameters µ v , γ v , v ∈ P in the criterion (2.13) using a procedure that leads to obtain the best RKHS ridge group sparse estimator in terms of the prediction quality, estimate the Sobol indices of the function m as described in Section 2.1.5.2.

The RKHSMetaMod package provides an interface from R statistical computing environment to the C++ libraries Eigen and GSL. In order to optimize the execution time and the storage memory, except for a function that is written in R, all of the functions of this package are written using the efficient C++ libraries through RcppEigen and RcppGSL packages. These functions are then interfaced in the R environment in order to propose an user friendly package. The RKHSMeta-Mod package is dedicated to the meta-model estimation on a RKHS. The convex optimization algorithms used in this package are adapted to take into account the problem of high dimensionality in this context. This package is available from the Comprehensive R Archive Network (CRAN) at https://cran.r-project.org/ web/packages/RKHSMetaMod/.

Presentation of the model

Consider a phenomenon described by a model m depending on d input variables X = (X 1 , ..., X d ). This model m from R d to R may be a known model that is calculable in all points X, or a regression model as defined in Equation (2.1). In the second case, the error ε is assumed to be centered with a finite variance. The components of X are independent and uniformly distributed on X = [0, 1] d , i.e. X ∼ P X = P 1 × ... × P d , with P a , a = 1, ..., d being the uniform law on the interval [0, 1]. The model m may present high complexity as strong non-linearities and high order interaction effects, and it is assumed to be square-integrable.

Criterion to minimize

Let us consider the parametric form of the RKHS ridge group sparse criterion defined in Equation (2.13), where γ v and µ v , v ∈ P are chosen as follows:

For each v ∈ P, let γ v and µ v be the weights that are chosen suitably. Then,

γ v = γ × γ v and µ v = µ × µ v with γ, µ ∈ R + .
Remark 2.3.1 This formulation simplify the choice of the tuning parameters, since instead of tuning the parameters γ v and µ v for all v ∈ P, only two parameters γ and µ are tuned. Moreover, the weights γ v and µ v , v ∈ P, may be of interest in applications. For example, one can take weights that increase with the cardinal of v in order to favour effects with small interaction order between variables.

For the sake of simplicity, in the rest of this Chapter for all v ∈ P the weights γ v and µ v are assumed to be setted as 1, and the RKHS ridge group sparse criterion is then expressed as follows:

C(f 0 , θ) = Y -f 0 I n - v∈P K v θ v 2 + √ nγ v∈P K v θ v + nµ v∈P K 1/2 v θ v .
By considering only the second part of the penalty function in the criterion above, i.e. set γ = 0, we obtain the RKHS group lasso criterion,

C g (f 0 , θ) = Y -f 0 I n - v∈P K v θ v 2 + nµ v∈P K 1/2 v θ v ,
which is a group lasso criterion [START_REF] Yuan | Model selection and estimation in regression with grouped variables[END_REF]) up to a scale transformation. We would like to mention that, in the RKHSMetaMod package, the solutions of the RKHS group lasso algorithm is used in order to initialize the input parameters of the RKHS ridge group sparse algorithm. Indeed, the penalty function in the RKHS group lasso criterion C g (f 0 , θ) insures the sparsity in the solution. Therefore, for a given value of µ, by implementing the RKHS group lasso algorithm, we obtain a solution with few terms in its additive decomposition.

From now on, the tuning parameter in the RKHS group lasso algorithm will be denoted by:

µ g = √ nµ.
(2.23)

Choice of the tuning parameters

While dealing with an optimization problem, one of the essential steps is to choose appropriately the tuning parameters. To do so, first, a grid of values of the tuning parameters µ and γ is chosen.

Let µ max be the smallest value of µ g (see Equation (2.23)), such that the solution to the minimization of the RKHS group lasso problem for all v ∈ P is θ v = 0. We have,

µ max = max v 2 √ n K 1/2 v (Y -Ȳ ) .
In order to set up the grid of values of µ, one may find µ max , and then a grid of values of µ is defined as follows:

µ l = µ max ( √ n × 2 l ) , l ∈ {1, ..., l max }.
The grid of values of γ is chosen by the user.

next, for the grid of values of µ and γ a sequence of estimators is calculated. Each estimator associated with the pair (µ, γ) in the grid of values of µ and γ, denoted by f (µ,γ) , is the solution of the RKHS ridge group sparse optimization problem or the RKHS group lasso optimization problem if γ = 0.

finally, the obtained estimators f (µ,γ) are evaluated using a testing dataset,

{(Y test i , X test i )} n test i=1 . The prediction error associated with the estimator f (µ,γ) is calculated by, ErrPred(µ, γ) = 1 n test n test i=1 (Y test i -f (µ,γ) (X test i )) 2 ,
where for S f being the support of the estimator f (µ,γ) ,

f (µ,γ) (X test ) = f 0 + v∈S f n i=1 θ vi k v (X vi , X test v ).
The pair ( µ, γ) with the smallest value of the prediction error is chosen, and the estimator f ( µ, γ) is considered as the best estimator of the function m, in terms of the prediction error.

In the RKHSMetaMod package, the algorithm to calculate a sequence of the estimators f , the value of µ max , and the prediction error are implemented as RKHSMetMod, mu_max, and PredErr functions, respectively.

Estimation of the Sobol indices

The Sobol indices of the function m are estimated by the empirical Sobol indices of the estimator f as described in Section 2.1.5.2,

S v =    var( fv(Xv)) v∈P var( fv(Xv)) for v ∈ S f , 0 for v / ∈ S f .
In the RKHSMetaMod package, the algorithm to calculate the empirical Sobol indices S v , v ∈ P is implemented as SI_emp function.

Algorithms

The RKHSMetaMod package implements two optimization algorithms: the RKHS ridge group sparse and the RKHS group lasso. These algorithms rely on the Gram matrices K v , v ∈ P, that have to be positive definite. Therefore, the first and essential step in the RKHSMetaMod package, is to calculate these matrices and insure their positive definiteness. The second step is to calculate the estimator f . In the RKHSMetaMod package two different objectives based on different procedures are considered in order to calculate this estimator:

The estimator with the best prediction quality:

In this case the best estimator is calculated using the procedure as described in Section 2.3.3.1.

The estimator with at most qmax active groups:

The tuning parameter γ is set as zero. A value of µ for which the number of groups in the solution of the RKHS group lasso problem is equal to qmax, is computed. This value will be denoted by µ qmax . Then, the RKHS ridge group sparse algorithm is implemented for a grid of values of γ = 0 and the value µ qmax . This procedure is implemented in the RKHSMetaMod package as RKHSMetMod_qmax function.

Calculation of the Gram matrices

The available kernels in the RKHSMetaMod package are: linear kernel, quadratic kernel, brownian kernel, matern kernel and gaussian kernel. The choice of the kernel that is done by the user, determines the functional approximation space. For a chosen kernel, the algorithm to calculate the Gram matrices K v , v ∈ P in the RKHSMetaMod package is implemented as calc_Kv function, and is based on three essential points:

Modify the chosen kernel:

In order to satisfy the conditions of constructing the RKHS H described in Section 2.1.4.2, these kernels are modified according to Equation (2.7). Let us take the example of the Brownian kernel:

Example 2.3.1 The usual presentation of the brownian kernel is as follows:

k a (X a , X a ) = min(X a , X a ) + 1.
The RKHS associated with the kernel k a is the set,

H a = f : [0, 1] → R is absolutely continuous, and f (0) = 0, 1 0 f (X a ) 2 dX a < ∞ ,
with the inner product

f, h Ha = 1 0 f (X a )h (X a )dX a .
The kernel k 0a associated with the brownian kernel is calculated as follows,

k 0a = min(X a , X a ) + 1 - ( 1 0 (min(X a , U ) + 1)dU )( 1 0 (min(X a , U ) + 1)dU ) ( 1 0 1 0 (min(U, V ) + 1)dU dV ) , = min(X a , X a ) + 1 - 3 4 (1 + X a - X 2 a 2 )(1 + X a - X 2 a 2
).

The RKHS associated with the kernel k 0a is the set,

H 0a = f ∈ H a : 1 0 f (X a )dX a = 0 .
Finally, the RKHS H = 1 + v∈P H v is the following set,

H = f : [0, 1] d → R : f = f 0 + v∈P f v (X v ), with f v ∈ H v .
Calculate the Gram matrices K v for all v:

First, for all a = 1, ...d the Gram matrices K a associated with kernels k 0a are calculated using Equation (2.7),

(K a ) i,i = k 0a (X ai , X ai ).
Then, for all v ∈ P, the Gram matrices K v associated with kernel k v = a∈v k 0a are calculated as follows:

K v = a∈v K a ,
where denotes the Hadamard product.

Insure the positive definiteness of the matrices K v , v ∈ P:

The output of the function calc_Kv is one of the input arguments of the functions associated with the RKHS group lasso and the RKHS ridge group sparse algorithms. As both of these algorithms rely on the positive definiteness of these matrices, it is mandatory to have K v , v ∈ P that are positive definite. For this reason, the calc_Kv function modifies the eigenvalues of the matrices K v , v ∈ P if necessary.

For each group v ∈ P, let λ v,max and λ v,min be respectively the maximum and the minimum eigenvalues associated with the matrix K v , and let "tol" be a positive scalar to be fixed. For each matrix

K v , "if λ v,min < λ v,max × tol", then, λ v,max × tol is added to all eigenvalues of K v .
The value of "tol" is set as 1e -8 by default, but one may consider a smaller or greater value for it depending on the kernel chosen and the value of n.

Optimization algorithms

RKHS group lasso In order to solve the RKHS group lasso optimization problem, the classical block coordinate descent algorithm is used [START_REF] Boyd | Distributed optimization and statistical learning via the alternating direction method of multipliers[END_REF], [START_REF] Bubeck | Convex optimization: Algorithms and complexity[END_REF]). The minimization of criterion C g (f 0 , θ) is done along each group v at a time. At each step of the algorithm, the criterion is minimized as a function of the current block's parameters, while the parameters values for the other blocks are fixed to their current values. The procedure is repeated until convergence.

In the RKHSMetaMod package the classical block coordinate descent algorithm to solve the RKHS group lasso optimization problem is implemented as RKHSgrplasso function.

RKHS ridge group sparse In order to solve the RKHS ridge group sparse optimization problem, an adapted block coordinate descent algorithm is proposed. This algorithm provides two steps:

Step 1 Initialize the input parameters by the solutions of the RKHS group lasso algorithm for each value of the tuning parameter µ, and run the RKHS ridge group sparse algorithm through active support of the RKHS group lasso solutions until it achieves convergence.

This step is provided in order to decrease the execution time.

Step 2 Re-initialize the input parameters with the obtained solutions of Step 1 and implement the RKHS ridge group sparse algorithm through all groups in P until it achieves convergence.

This second step makes it possible to verify that no group is missing in the output of Step 1.

The adapted block coordinate descent algorithm to solve RKHS ridge group sparse optimization problem is implemented in the RKHSMetaMod package, as pen_MetMod function.

Summary and perspectives

The work presented in this thesis focuses on the problem of estimating a metamodel that approximates the Hoeffding decomposition of a complex model, denoted m. The model m depends on d input variables X 1 , ..., X d that are independent and have a known law. The meta-model belongs to a RKHS H, which is constructed in a way such that the additive decomposition of any function f in H is the Hoeffding decomposition of f (Durrande et al. ( 2013)). The estimator of the meta-model, denoted f , minimizes a least-squares criterion penalized by a penalty function which is the sum of the Hilbert norm and the empirical L 2 -norm. This procedure, called RKHS ridge group sparse, allows both to select and estimate the terms in the Hoeffding decomposition of the meta-model, and therefore, to select the Sobol indices that are non-zero and estimate them (Huet and Taupin (2017)).

The first part of this work is dedicated to study the theoretical properties of the estimator f in the regression framework where the error ε that is non-Gaussian and non-bounded. The upper bounds of the empirical L 2 and the L 2 risks of this estimator are provided.

In the second part of this work, the procedure of calculating f is implemented in an R package, called RKHSMetaMod. In order to optimize the execution time and also the storage memory, except for a function that is written in R, all of the functions in this package are written using C++ libraries GSL and Eigen. They are then interfaced with the R environment in order to propose an user friendly package. The RKHSMetaMod package deals both with a calculable model and a regression model. A simulation study is provided in order to validate the performance of the package functions in terms of the predictive quality of the estimator obtained and the estimation of the Sobol indices.

Like all research works that are carried out in a limited period of time, many pistes have not been explored in this work and there are several perspectives to be considered for further study. Let us mention some of them.

Non-independent input variables

In both parts of this thesis, the input variables X 1 , ..., X d are assumed to be independent and their law is known. Under these assumptions, it is possible to construct the approximation spaces such that any function in these spaces is decomposed according to its Hoeffding decomposition. This decomposition is unique and the terms of it are orthogonal.

If the variables X 1 , ..., X d are not independent, there is no longer orthogonality between the terms of the decomposition on the approximation spaces and the decomposition of a function on these spaces is not necessarily unique. It follows that the decomposition of the variance given in Equation (2.4) is no longer valid, nor the calculation of the Sobol indices. However, approximating a model on a functional space by an additive decomposition may be interesting in practice, since the estimation of the meta-model can still help the interpretation of the effects of the input variables on the output variable.

The case where the variables X 1 , ..., X d are not independent has been considered by [START_REF] Koltchinskii | Sparsity in multiple kernel learning[END_REF]. Their approximation space H is the linear span (l.s.) of a large dictionary consisting of N RKHS spaces H 1 , ..., H N ,

H = l.s. N j=1 H j .
The space H consists of all functions f that have an additive representation of the form,

f = N j=1 f j (X), f j ∈ H j , j = 1, ..., N.
(2.24)

Under some assumptions on the degree of dependence of the RKHS spaces H j spaces ensuring almost orthogonality between these spaces, [START_REF] Koltchinskii | Sparsity in multiple kernel learning[END_REF] established upper bounds in the excess risk of a ridge group sparse type estimator. Note that, the approximation spaces considered in this thesis are a special case of the spaces considered by [START_REF] Koltchinskii | Sparsity in multiple kernel learning[END_REF]. However, the context in which their results are established differs from ours in several points. On the one hand, the functions in the space H are assumed to be uniformly bounded. On the other hand, the statistical model is different and in particular the case of the regression model with non-bounded additive error is not considered in their work.

My objective would be then to establish the risk upper bounds of a ridge group sparse estimator on the approximation spaces constructed as proposed by [START_REF] Durrande | Anova kernels and rkhs of zero mean functions for model-based sensitivity analysis[END_REF], in a context where the input variables X 1 , ..., X d are non-independent, for the regression model with non-bounded additive error. One of the essential steps of the proof relies on the upper bounding the L 2 -norm in H by the empirical L 2norm in H (Lemma 3.5.4). The calculation of this upper bound requires the control of the moments of order 4 of the functions in H. In the case where the variables X 1 , ..., X d are independent, this control is obtained since there is orthogonality between the terms in the decomposition of the functions in H. In the contrary case, the assumptions on the degree of dependence of H j spaces formulated by Koltchinskii and [START_REF] Koltchinskii | Sparsity in multiple kernel learning[END_REF] is not sufficient to handle the calculation of moments of order 4. It remains therefore to continue this work to establish the upper bounds of the risk of the RKHS ridge group sparse estimator under assumptions which have to be specified. To our knowledge, this case has not been studied until now.

Concerning the implementation of sensitivity analysis in this case, as the calculation of the Sobol indices is not possible any more, one may consider Shapley values (Shapley (1953)), see for example [START_REF] Owen | Sobol' indices and shapley value[END_REF], [START_REF] Song | Shapley effects for global sensitivity analysis: Theory and computation[END_REF], [START_REF] Owen | On shapley value for measuring importance of dependent inputs[END_REF], Benoumechiara and Elie-Dit-Cosaque (2019), [START_REF] Broto | Sensitivity indices for independent groups of variables[END_REF], [START_REF] Iooss | Shapley effects for sensitivity analysis with correlated inputs: comparisons with Sobol' indices, numerical estimation and applications[END_REF].

Generalization to the regression framework with log-concave error

Result 1 shows that the risk upper bound in the regression setting with errors that are distributed with density π α ∈ D is the same as the one obtained in the regression setting with Gaussian errors. However, the class of densities π α is restrictive, and it would be interesting to obtain a result for larger density classes, such as log-concave densities for example.

As explained in Section 2.2.5, one of the essential steps of the proof of Result 1 is based on a Sudakov type minoration of the expectation of the empirical process associated with the random variables that are assumed to be non-Gaussian and non-bounded.

In the Gaussian case, Sudakov's minoration is stated as follows [START_REF] Pisier | The volume of convex bodies and Banach space geometry[END_REF]): Let ε = (ε 1 , ..., ε n ) be i.i.d. Gaussian random variables, and for a function g : R |v| → R, v ∈ P belonging to a class of functions G, let V n,ε be the empirical process associated with the random vector ε defined in Equation (2.21). Then, for all δ > 0,

1 C log N (δ, G, . ) ≤ nE ε sup g∈G |V n,ε (g)| δ 2 , (2.25)
where C is a constant, and N (δ, G, . ) is the δ-covering number of the metric space (G, . ).

It remains then to characterize the complexity of the functional space G to obtain a minoration of the expectation of the empirical process and to deduce the result in the risk bound.

When ε = (ε 1 , ..., ε n ) are distributed according to density π α , Theorem 3.1. in [START_REF] Talagrand | The supremum of some canonical processes[END_REF] establishes the inequality given in Equation (3.34) from which we could deduce the minoration of the expectation of the empirical process given in Equation (2.22).

When ε is non-Gaussian and non-bounded, a Sudakov type minoration for the independent log-concave random variables is given by [START_REF] Latała | Sudakov-type minoration for log-concave vectors[END_REF]. A measure on R n with the full dimensional support is log-concave if and only if it has a density of the form exp(-φ(x)), where φ : R n → (-∞, ∞] is convex [START_REF] Borell | Convex measures on locally convex spaces[END_REF]). Let us recall the Sudakov type minoration obtained by [START_REF] Latała | Sudakov-type minoration for log-concave vectors[END_REF]:

Let ε = (ε 1 , ..., ε n ) be i.i.d. log-concave random variables, then:

1 K min c 2 δ, log N (2 × max(cδ 1/2 , c 2 δ), G, . ) ≤ nE ε sup g∈G |V n,ε (g)|, (2.26)
where K is a universal constant, and c = 1/ max(512c , 8) for c being a universal constant.

We could not deduce from inequality (2.26) the adapted Sudakov type minoration that leads to obtain the optimal rate of convergence for the RKHS ridge group sparse estimator. By optimal we mean the same rate of convergence as in the Gaussian regression setting (see Huet and Taupin (2017)). This is the reason why, in this work, the densities π α ∈ D are considered. Nevertheless, some additional work in that direction together with bibliography research is a worthwhile direction.

Introduction

Let us consider the following regression model:

Y = m(X) + σε, σ > 0, (3.1)
where the variables X = (X 1 , ..., X d ) are independent with a known law

P X = d a=1 P a on X = d a=1 X a , a compact subset of R d .
The number d of components of X may be large. The model m from R d to R maybe complex, presenting strong non-linearities, and it is assumed to be square-integrable, i.e. m ∈ L 2 (X , P X ).

Let D be the set of densities,

D = π α : π α (x) = a α exp(-|x| α ), with (a α ) -1 = R exp(-|x| α )dx, α > 2 . (3.2)
In this Chapter, we assume that the error term ε is equal to Z/σ α , where Z is a random variable with density π α ∈ D and σ 2 α is its variance, i.e. var(Z) = σ 2 α . in the regression model with non-Gaussian and non-bounded error

Based on n data points {(X i , Y i )} n i=1 , a meta-model that approximates the Hoeffding decomposition of m is estimated. This meta-model belongs to a reproducing kernel Hilbert space (RKHS), which is constructed as a direct sum of Hilbert spaces [START_REF] Durrande | Anova kernels and rkhs of zero mean functions for model-based sensitivity analysis[END_REF]). The estimation of the meta-model is carried out via a penalized least-squares minimization allowing to select the subsets of variables X that contribute to predict the output Y (Huet and Taupin (2017)).

Let us be more precise on the Hoeffding decomposition. Let P be the set of all the subsets of {1, ..., d} with dimension 1 to d, and for all v ∈ P and X ∈ X , let X v be the vector with components X a for all a ∈ v. Let also |A| be the cardinality of a set A and for all v ∈ P, let m v : R |v| → R be a function of X v . Then, the Hoeffding decomposition of m is written as [START_REF] Hoeffding | A class of statistics with asymptotically normal distribution[END_REF], [START_REF] Sobol | Sensitivity estimates for nonlinear mathematical models[END_REF], van der Vaart (1998)),

m(X) = m 0 + v∈P m v (X v ), (3.3)
where m 0 is a constant. This decomposition (3.3) is unique [START_REF] Sobol | Sensitivity estimates for nonlinear mathematical models[END_REF]), all the functions m v are centered, and they are orthogonal with respect to L 2 (X , P X ).

The Hoeffding decomposition of m is approximated by the orthogonal projection of m on a RKHS H which is constructed as a direct sum of Hilbert spaces [START_REF] Durrande | Anova kernels and rkhs of zero mean functions for model-based sensitivity analysis[END_REF]).

Let ., . H be the inner product in H, and let k and k v be the reproducing kernels associated with the RKHS H and the RKHS H v , respectively. The properties of the RKHS H insures that any function f ∈ H, f : X ⊂ R d → R can be written as the following decomposition:

f (X) = f, k(X, .) H = f 0 + v∈P f v (X v ), (3.4)
where f 0 is a constant, and f v : R |v| → R is defined by,

f v (X) = f, k v (X, .) H .
For all v ∈ P, the functions f v (X v ) are centered and for all v, v ∈ P, v = v , the functions f v (X v ) and f v (X v ) are orthogonal with respect to L 2 (X , P X ). Therefore, the decomposition of any function f presented in Equation (3.4) is unique and is its Hoeffding decomposition.

The meta-model f * that approximates the Hoeffding decomposition of m is defined as follows:

f * = arg min f ∈H m -f 2 2 = arg min f ∈H E X (m(X) -f (X)) 2 .
Since the function f * belongs to the RKHS H, its decomposition on H is its Hoeffding decomposition:

f * = f * 0 + v∈P f * v . (3.5)
And for all v ∈ P, the function f * v in Equation (3.5) approximates the function m v in Equation (3.3). Decomposition (3.5) contains |P| terms f * v to be estimated. The cardinality of P is equal to 2 d -1 which may be huge since it raises very quickly by increasing d. In order to deal with this problem, one may estimate f * by a sparse estimator f ∈ H. To this purpose, the estimation of f * is done on the basis of n observations by minimizing an empirical least-squares criterion penalized by the sum of the Hilbert norm and the empirical norm. This procedure, called ridge group sparse, estimates the groups v that are suitable for predicting f * , and the relationship between f * v and X v for each group v (Huet and Taupin (2017)). The estimator so obtained is called theRKHS ridge group sparse estimator.

Several authors studied the theoretical properties of estimators similar to the RKHS ridge group sparse estimator. Let us briefly review their framework and their results. [START_REF] Meier | High-dimensional additive modeling[END_REF] considered an estimator similar to the RKHS ridge group sparse estimator. Instead of adding two separate sparsity and smoothness penalties, they combine these two terms into a single sparsity and smoothness penalty. In the fixed design regression model with error ε that is distributed as a sub-Gaussian random variable, they established upper bounds of the empirical risk for estimating the projection of m onto the set of univariate additive functions. Afterwards, [START_REF] Raskutti | Minimax-optimal rates for sparse additive models over kernel classes via convex programming[END_REF] showed (in Section 3.4. of their paper) that the convergence rate of this estimator is sub-optimal.

Koltchinskii and Yuan (2010) considered a more general RKHS including the functions that have an additive representation over kernel spaces and obtained an estimator based on a ridge group sparse type procedure. Under a global boundedness condition, they established upper bounds on the excess risk assuming that the function m has a sparse representation. A global boundedness condition means that the quantity sup f ∈H sup X∈X |f (X)| is assumed to be bounded independently of dimension d. Their results are valid for a large class of loss functions, and for distributions of the observations Y such that some defined boundedness conditions on the loss functions are satisfied (see Section 2.1. of their paper). In their framework, the input variables X are not assumed to be independent and there is no orthogonality assumption between the kernel spaces. Instead, the authors introduced some characteristics related to the degree of dependence of their kernel spaces which insures almost orthogonality between these spaces. Their method to derive their upper bounds relies on the elementary empirical and Rademacher process methods such as symmetrization and concentration inequalities for Rademacher processes and Bernstein type exponential bounds. [START_REF] Raskutti | Minimax-optimal rates for sparse additive models over kernel classes via convex programming[END_REF] assumed that the function m has a sparse univariate additive representation, i.e. m = a∈S m a (X a ) for m a (X a ) being univariate functions and |S| < d, such that each univariate function m a lies in a RKHS H a . They used the ridge group sparse procedure to calculate the estimator of m, and studied the theoretical properties of their estimator in the Gaussian regression model, i.e. ε in Equation (3.1) is distributed as a centered Gaussian random variable. They pro-in the regression model with non-Gaussian and non-bounded error vided upper bounds for the integrated and the empirical risks and a lower bound for the integrated risk of their estimator over spaces of sparse additive models, including polynomials, splines and Sobolev classes. Huet and Taupin (2017) studied the theoretical properties of the RKHS ridge group sparse estimator in the Gaussian regression model. They derived upper bounds with respect to the L 2 -norm and the empirical L 2 -norm for the distance between the true function m and its estimation f into the RKHS H. [START_REF] Raskutti | Minimax-optimal rates for sparse additive models over kernel classes via convex programming[END_REF] and Huet and Taupin (2017) did not assume the global boundedness condition. Instead, they assumed that each function within the unit ball of the Hilbert space H v is uniformly bounded by a constant. The proof of their results is based on the probabilistic methods of empirical Gaussian process such as concentration inequalities and Sudakov minoration (e.g. [START_REF] Pisier | The volume of convex bodies and Banach space geometry[END_REF], [START_REF] Massart | About the constants in talagrand's concentration inequalities for empirical processes[END_REF], [START_REF] Van De Geer | Empirical Processes in M-Estimation, Cambridge Series in Statistical and Probabilistic Mathematics[END_REF], [START_REF] Ledoux | The Concentration of Measure Phenomenon, Mathematical surveys and monographs[END_REF]), as well as results on the Rademacher complexity of kernel classes [START_REF] Mendelson | Geometric parameters of kernel machines[END_REF], [START_REF] Bartlett | Local rademacher complexities[END_REF]).

In this Chapter, the upper bounds of the empirical L 2 risk and the L 2 risk of the RKHS ridge group sparse estimator are provided, in the regression model (see Equation (3.1)) with non-Gaussian and non-bounded error ε, and by considering a quadratic loss function. In this case the conditions assumed in [START_REF] Koltchinskii | Sparsity in multiple kernel learning[END_REF] are not satisfied, and the empirical Gaussian process methods such as concentration inequalities and Sudakov minoration can not be used.

The proof of our results requires different mathematical tools than those used in the works mentioned above: a Sudakov type minoration that is satisfied for the non-Gaussian and nonbounded random variables, a concentration bound for the lower and upper tails of a convex function of the random variables {ε i } n i=1 that are non-Gaussian and non-bounded.

To the best of our knowledge, in our context of regression model with non-Gaussian and non-bounded error ε, and with quadratic loss function, the only Sudakov type minoration which allows to obtain the same rate of convergence for the RKHS ridge group sparse estimator as in the Gaussian regression model (see Huet and Taupin (2017)), is the one obtained by [START_REF] Talagrand | The supremum of some canonical processes[END_REF]. The minoration obtained by [START_REF] Talagrand | The supremum of some canonical processes[END_REF] is specific to the densities π α ∈ D as defined in Equation (3.2). This is the reason why this class of densities is considered in this work.

Concerning the concentration bound, it can be shown that the distribution functions associated with the densities π α ∈ D belong to a class of distribution functions defined by [START_REF] Adamczak | Logarithmic sobolev inequalities and concentration of measure for convex functions and polynomial chaoses[END_REF], for which the log-Sobolev inequality [START_REF] Gross | Logarithmic sobolev inequalities[END_REF]) is satisfied. [START_REF] Shu | A characterization of a class of convex log-sobolev inequalities on the real line[END_REF] provided bounds for the lower and upper tails of convex functions of independent random variables which satisfy the log-Sobolev inequality. Since the distribution functions associated with the densities π α ∈ D satisfy the log-Sobolev inequality, the concentration inequality derived by [START_REF] Shu | A characterization of a class of convex log-sobolev inequalities on the real line[END_REF] holds for them. This Chapter is organised as follows: The RKHS construction and the procedure for estimating a meta-model are presented in Section 3.2. The theoretical properties of the RKHS ridge group sparse estimator are stated in Theorem 3.3.1 and Corollary 3.3.2. The proof of Theorem 3.3.1 is postponed in Section 3.5. In Section 3.4 the main arguments of the proof of Theorem 3.3.1 and motivation for the choice π α are detailed.

Meta-modelling and the RKHS ridge group sparse estimator

The independency between the input variables X allows to write the function m according to its Hoeffding decomposition presented in Equation (3.3),

m(X) = m 0 + v∈P m v (X v ).
The unknown function m is approximated by its orthogonal projection, denoted f * , on a RKHS, denoted H, that is constructed as a direct sum of Hilbert spaces. The RKHS H is associated with a so-called ANOVA kernel which is defined in order to obtain the analytical expression of the terms of the Hoeffding decomposition of the functions of H. As f * is the orthogonal projection of m on H, each term in its decomposition is an approximation of the associated term in the Hoeffding decomposition of m. The construction of the RKHS H has been proposed by [START_REF] Durrande | Anova kernels and rkhs of zero mean functions for model-based sensitivity analysis[END_REF] that we recall briefly in the following.

RKHS construction

Let X = X 1 × . . . × X d be a subset of R d . For each a ∈ {1, • • • , d}, we choose a RKHS H a , and its associated kernel k a defined on the set X a ⊂ R such that the two following properties are satisfied:

(i) k a : X a × X a → R is P a ⊗ P a measurable, (ii) E Xa k a (X a , X a ) < ∞.
The property (ii) depends on the kernel k a , a = 1, ..., d and the distribution of X a , a = 1, ..., d. It is not very restrictive since it is satisfied, for example, for any bounded kernel.

The RKHS H a can be decomposed as a sum of two orthogonal sub-RKHS,

H a = H 0a ⊥ ⊕ H 1a ,
where H 0a is the RKHS of zero mean functions,

H 0a = f a ∈ H a , E Xa (f a (X a )) = 0
, in the regression model with non-Gaussian and non-bounded error and H 1a is the RKHS of constant functions,

H 1a = f a ∈ H a , f a (X a ) = C .
The kernel k 0a associated with the RKHS H 0a is defined as follows:

k 0a (X a , X a ) = k a (X a , X a ) - E U ∼Pa (k a (X a , U ))E U ∼Pa (k a (X a , U )) E (U,V )∼Pa⊗Pa k a (U, V ) . Let k v (X v , X v ) = a∈v k 0a (X a , X a ), then the ANOVA kernel k is defined by: k(X, X ) = d a=1 1 + k 0a (X a , X a ) = 1 + v∈P k v (X v , X v ).
For H v being the RKHS associated with the kernel k v , the RKHS associated with the ANOVA kernel k is then defined by:

H = d a=1 1 ⊥ ⊕ H 0a = 1 + v∈P H v ,
where ⊥ denotes the L 2 inner product.

According to this construction, any function f ∈ H satisfies the following decomposition,

f (X) = f, k(X, .) H = f 0 + v∈P f v (X v ).
which is the Hoeffding decomposition of f .

For more background on the RKHS spaces see [START_REF] Aronszajn | Theory of reproducing kernels[END_REF], [START_REF] Saitoh | Theory of reproducing kernels and its applications[END_REF], [START_REF] Berlinet | Reproducing Kernel Hilbert Spaces in Probability and Statistics[END_REF].

Approximating the Hoeffding decomposition of m

Let f * ∈ H be defined as follows:

f * = arg min f ∈H m -f 2 2 = arg min f ∈H E X (m(X) -f (X)) 2 .
The function

f * = f * 0 + v∈P f * v ,
is the approximation of m on the RKHS H, and its Hoeffding decomposition is an approximation of the Hoeffding decomposition of m. Therefore, according to Equation (3.3), for all v ∈ P, each function f * v approximates the function m v .

The number of functions f * v is related to the cardinality of P, i.e. 2 d -1, that may be huge. The idea is to calculate a sparse estimator of f * as an estimator of m. To do so, the ridge group sparse procedure as proposed by Huet and Taupin (2017) is used that we recall in the following.

Ridge group sparse procedure and associated estimator

Let n be the number of observations. For all v ∈ P, let X v be the matrix of variables corresponding to the v-th group, i.e.

X v = (X vi , i = 1, ..., n, v ∈ P) ∈ R n×|P| .
For any f ∈ H such that f = f 0 + v∈P f v , and for some tuning parameters γ v , µ v , v ∈ P, the ridge group sparse criterion is defined as follows:

L(f ) = 1 n n i=1 Y i -f 0 - v∈P f v (X vi ) 2 + v∈P γ v f v n + v∈P µ v f v Hv ,
where

f v n is the empirical L 2 -norm of f v defined by the sample {X vi } n i=1 as f v 2 n = 1 n n i=1 f 2 v (X vi ).
The penalty function in the criterion L(f ) is the sum of the Hilbert norm and the empirical norm, which allows to select few terms in the additive decomposition of f over sets v ∈ P. Moreover, the Hilbert norm favours the smoothness of the estimated f v , v ∈ P.

Let us define the set of functions,

F = f : f = f 0 + v∈P f v , with f v ∈ H v , and f v Hv ≤ r v , r v > 0 . (3.6)
Then the RKHS ridge group sparse estimator is defined by,

f = arg min f ∈F L(f ).
(3.7)

Risk upper bounds

In this Section, the upper bounds of the empirical L 2 risk and the L 2 risk of the RKHS ridge group sparse estimator are presented in Theorem 3.3.1 and Corollary 3.3.1, respectively. Before stating these results, let us introduce some notation and assumptions that are needed in the rest of this Chapter. For a function f ∈ H, let S f be its support,

S f = {v ∈ P : f v = 0}. (3.8)
The RKHS construction as described in Section 3.2.1 insures that the following properties are satisfied:

for all v ∈ P, the functions f v ∈ H v are centered and are square-integrable, i.e.

E X (f v (X v )) = 0 and E X (f 2 v (X v )) < ∞
, in the regression model with non-Gaussian and non-bounded error for all v, v ∈ P, v = v , the functions f v ∈ H v and f v ∈ H v are orthogonal with respect to L 2 (X , P X ), i.e.

E X (f v (X v )f v (X v )) = 0.
We assume moreover that, for all v ∈ P, the functions f v ∈ H v are uniformly bounded, i.e.

∃R > 0 such that f v ∞ = sup Xv |f v (X v )| ≤ R. Each kernel k v , v ∈ P is associated with an integral operator T kv from L 2 (X v , P v ) to L 2 (X v , P v ) defined by: ∀f ∈ L 2 (X v , P v ), T kv (f ) = Xv k v (., t)f (t)dP v (t).
For each v ∈ P, let ω v,1 ≥ ω v,2 ≥ ... ≥ 0 be the eigenvalues of the integral operator T kv (see Equation (3.19)). Let us define the function Q n,v (t) for some positive t as follows:

Q n,v (t) = 5 n ≥1 min(t 2 , ω v, ), (3.9)
and for some ∆ > 0 let ν n,v be defined by:

ν n,v = inf t Q n,v (t) ≤ ∆t 2 .
(3.10)

For each v ∈ P, ν n,v refers to the minimax optimal rate for L 2 (X , P X )-estimation in the RKHS H v [START_REF] Mendelson | Geometric parameters of kernel machines[END_REF]).

Remark 3.3.1 The rate ν n,v , v ∈ P, depends on the regularity of the RKHS via the decreasing rate of the eigenvalues {ω v, } ∞ =1 . When RKHS is of high regularity, i.e. when the eigenvalues {ω v, } ∞ =1 decrease quickly, then the rate ν n,v , v ∈ P will be close to the parametric rate of convergence (see Section 3.3.1).

The choice of tuning parameters in the criterion L(f ) is specified in terms of the following quantity:

λ n,v = max ν n,v , d n .
(3.11)

Theorem 3.3.1 Consider the regression model defined at Equation (3.1) with σ = 1. Let {(Y i , X i )} n i=1 be a n-sample with the same law as (Y, X), and let {ε i } n i=1 be the random errors that are independent and identically distributed (i.i.d.) like ε. Let also f be defined by (3.7) with r v = 1 in (3.6), and let the tuning parameters µ v 's and γ v 's be chosen as follows:

For some constant C 1 > 10 + 4∆,

∀v ∈ P, µ v = C 1 λ 2 n,v , γ v = C 1 λ n,v .
(3.12)

If there exists positive constants C 2 , C 3 , and 0 < β < 1/α such that the following assumptions are satisfied:

∀v ∈ P, nλ 2 n,v ≥ -C 2 log λ n,v , (3.13
)

and ∀f ∈ F, v∈S f λ 2 n,v ≤ C 3 n 2β-1 , (3.14)
then, there exists 0 < η < 1 depending on constants {C i } 3 i=1 , β, and n (η tends to 0 as n increases), such that with probability greater than 1 -η, we have for some

constant C, m -f 2 n ≤ C inf f ∈F m -f 2 n + v∈S f (µ v + γ 2 v ) . (3.15)
Let us now comment on the theorem.

Remark 3.3.2 Let f be the function in F such that the infimum of the right hand side of the inequality (3.15) is realized. The term m-f 2 n is the usual bias term. It quantifies both the approximation properties of the RKHS H, and the bias-variance trade-off.

Remark 3.3.3 This result is similar to the one obtained in the Gaussian regression model at the cost of the additional Assumption (3.14). This assumption allows to obtain the same rate of convergence for the RKHS ridge group sparse estimator as in the Gaussian regression model (see Huet and Taupin (2017)). However, it implies some restrictions on the regularity of the RKHS H. Indeed, as for all v ∈ P,

λ n,v ≥ ν n,v (see Equation (3.11)), it follows that v∈S f ν 2 n,v ≤ C 3 n 2β-1
, which implies some restrictions on the regularity of the RKHS: if β is small, which will be the case if α is large, then the RKHS should be of high regularity.

Remark 3.3.4 By Equation (3.11), we also have that for all v ∈ P, λ n,v ≥ d/n. This assumption allows to control the probability of the |P| events (see Equation (3.48)), where log(|P|) is of order d.

Remark 3.3.5 The result in Theorem 3.3.1 can be generalized to the case where σ = 1 in Equation (3.1), and where r v = 1 in (3.6).

Let g be defined as follows:

g = arg min g∈F Y σ -g 2 n + 1 σ v γ v g v n + 1 σ v µ v g v Hv , (3.16) with F = g : g = g 0 + v g v , with g v ∈ H v , and g v Hv ≤ r v σ .
(3.17) in the regression model with non-Gaussian and non-bounded error

We have f = σ g for f being defined by (3.7).

For all u > 0, let H u v be the RKHS associated with the kernel uk

v . If u = r 2 v /σ 2 , then g = arg min g∈F Y σ -g 2 n + 1 σ v γ v g v n + 1 σ 2 v µ v r v g v H u v .
where

F = g : g = g 0 + v g v , with g v ∈ H u v , and g v H u v ≤ 1 .
We apply Theorem 3.3.1 with Y /σ and m/σ in place of Y and m, to g defined as above.

Let

Q u n,v (t) = 5 n ≥1 min(t 2 , uω v, ),
and for ∆ > 0, let

ν u n,v (∆ ) = inf t Q u n,v (t) ≤ ∆ t 2 .
Let also

λ u n,v = max ν u n,v , d n .
For some constant C 1 > 10 + ∆ , take

µ v r v σ 2 = C 1 λ u n,v 2 , γ v σ = C 1 λ u n,v .
Then, for S g being defined as follows

S g = {v ∈ P : g v = 0}, (3.18) we have m σ -g 2 n ≤ C inf g∈F m σ -g 2 n + 1 σ 2 v∈Sg (µ v r v + γ 2 v ) ,
or, multiplying both sides by σ 2 , and taking

u = r 2 v /σ 2 , m -σ g 2 n ≤ C inf g∈F m -σg 2 n + v∈Sg (µ v r v + γ 2 v ) .
Corollary 3.3.1 Under the same assumptions as Theorem 3.3.1, we have with high probability for some constant C that, Then, with similar demonstration as in Remark 3.3.5 we obtain,

m -f 2 2 ≤ C inf f ∈F m -f 2 n + m -f 2 2 + v∈S f (µ v + γ 2 v ) .
m -σ g 2 2 ≤ C inf g∈F m -σg 2 n + m -σg 2 2 + v∈Sg (µ v r v + γ 2 v ) ,
where F and S g are defined in Equations (3.17) and (3.18), respectively.

Rate of convergence

Corollary 3.3.2 Under the same assumptions as Theorem 3.3.1, we have

m -f 2 n ≤ C inf f ∈F m -f 2 n + v∈S f ν 2 n,v + d|S f | n .
This Corollary highlights that the upper bound is relevant when the infimum is reached for functions f that have a sparse decomposition in H, i.e. |S f | is small, and when d is small face to n. When d is large, the decomposition of functions in H should be limited to interactions of a limited order, so that the number of elements in the estimated meta-model is of order smaller than d r for some small r, say r = 2 for example. In such a case, the cardinality of P will be smaller than d 2 . As we mentioned in Remark 3.3.4, the assumption λ n,v ≥ d/n is needed to control the value log(|P|), which will be now smaller than 2 log(d). Therefore, the value d in the definition of λ n,v (see Equation (3.11)) as well as the term d|S f |/n in the infimum above will be replaced by 2 log(d) and 2 log(d)|S f |/n, respectively. Let us discuss the rate of convergence given by v∈S f ν 2 n,v . For the sake of simplicity we consider the case where the variables X 1 , . . . , X d have the same distribution P 1 on X 1 ⊂ R, and where the unidimensional kernels k 0a are all identical, such that k v (X v , X v ) = a∈v k 0 (X a , X a ). The kernel k 0 admits an eigen expansion given by

k 0 (X a , X a ) = a≥1 ω 0, a φ a (X a )φ a (X a ),
where the eigenvalues {ω 0, a } ∞ a=1 are non-negative and ranged in the decreasing order, and where the {φ a } ∞ a=1 are the associated eigenfunctions, orthonormal with respect to L 2 (X 1 , P 1 ). Therefore, the kernel k v admits the following expansion,

k v (X v , X v ) = =( 1 ... |v| ) |v| a=1 ω 0, a ω v, |v| a=1 φ a (X a ) φ v, (Xv) |v| a=1 φ a (X a ) φ v, (X v )
.

(3.19)

Consider the case where the eigenvalues {ω 0, a } ∞ a=1 are decreasing at a rate -2α a for some α > 1/2, i.e. the ω 0, are of order -2α = ( |v| a=1 a ) -2α . It is shown in in the regression model with non-Gaussian and non-bounded error Section 8.3. of Huet and Taupin (2017), that

ν n,v ∝ n -α (2α +1) (log n) γ ,
where the rate ν n,v is defined at Equation (3.10) and

γ ≥ (|v| -1) α (2α -1)
.

For all f ∈ F we have then,

v∈S f ν 2 n,v ∝ |S f |n -2α (2α +1) (log n) 2γ .
Note that in this particular case, the rate of convergence depends on |v| through the logarithmic term (log n) 2γ , and that up to this logarithmic term the rate of convergence has the same order than the usual non-parametric rate for unidimensional functions. It follows that the RKHS space H should be chosen such that the unknown function m is well approximated by sparse functions in H with low order of interactions.

Besides, the rate ν n,v should satisfy assumption (3.14),

v∈S f ν 2 n,v ≤ C 3 n 2β-1 , which holds if α > 1 -2β 4β > α -2 4 . (3.20)
This shows that for the large values of α the assumption (3.14) implies some restrictions on the regularity of the RKHS chosen: If α < 4, then all α greater than 1/2 satisfy Equation (3.20), since (α -2)/4 < 1/2. If α ≥ 4, then we have α > (α -2)/4 > 1/2. As α increases, i.e. β decreases (recall that 0 < β < 1/α), and assumption (3.14) implies that the RKHS chosen should be of high regularity.

Main arguments of the proof of Theorem 3.3.1 and motivation for the choice π α

The proof of Theorem 3.3.1 starts in the same way as the proof of Theorem 2.1. in Huet and Taupin (2017) where they considered the Gaussian regression model. However, it differs in two essential points:

1. Sudakov type minoration, 2. Concentration inequality.

Main arguments of the proof of Theorem 3.3.1 and motivation for the choice π α 73

In the following Section, we give a sketch of the proof of Theorem 3.3.1, we highlight the two points above that differs the proof from the proof in the Gaussian regression model, and we provide a detailed comparison to the related works. In Section 3.4.2 we give a brief introduction to the Sudakov type minoration context, we explain the motivation for choosing densities π α ∈ D defined in Equation (3.2), and we state in Corollary 3.4.1 the appropriate Sudakov minoration used in the proof of Theorem 3.3.1. In Section 3.4.3 we present the concentration inequality context, and we state in Corollary 3.4.2 the appropriate concentration inequality used in the proof of Theorem 3.3.1.

Sketch of the proof

We give here a sketch of the proof of Theorem 3.3.1, and we postpone to Section 3.5 for complete statements. We begin by introducing some notation. We denote by C constants that vary from an equation to the other. For v ∈ P, and for a function φ : R |v| → R, we denote by V n,ε the empirical process defined as,

V n,ε (φ) = 1 n n i=1 ε i φ(X v,i ). (3.21)
For all v ∈ P, let H v be the RKHS associated with the reproducing kernel k v . For any function g v ∈ H v , v ∈ P, and V n,ε being defined in Equation (3.21), we consider two following processes,

W n,2,v (t) = sup |V n,ε (g v )|, g v Hv ≤ 2, g v 2 ≤ t , (3.22) W n,n,v (t) = sup |V n,ε (g v )|, g v Hv ≤ 2, g v n ≤ t . (3.23)
Starting from the definition of f , some simple calculations give that for all f ∈ F,

C m -f 2 n ≤ m -f 2 n + |V n,ε ( f -f )| + v∈S f [γ v f v -f v n + µ v f v -f v Hv ] - v / ∈S f [µ v f v Hv + γ v f v n ], ≤ m -f 2 n + |V n,ε ( f -f )| + v∈S f [γ v f v -f v n + µ v f v -f v Hv ]. If we set g = f -f , then g ∈ H, g = g 0 + v g v , with g v = f v -f v , and for each v, g v Hv ≤ 2.
The main problem is now to control the empirical process V n,ε . For each v, letting λ n,v as in (3.11), we state (see Lemma 3.5.1, page 85) that, with high probability,

|V n,ε (g v )| ≤ Cλ 2 n,v g v Hv + Cλ n,v g v n .
(3.24)

One of the key points in the proof of Lemma 3.5.1 is to find an upper bound for the two following quantities:

|W n,n,v (t) -E ε (W n,n,v (t))|, and |W n,2,v (t) -E ε (W n,2,v (t))|.
(3.25) in the regression model with non-Gaussian and non-bounded error

In the Gaussian regression model, one use the isoperimetric inequality for Gaussian processes in [START_REF] Massart | Concentration Inequalities and Model Selection: Ecole d'Eté de Probabilités de Saint-Flour XXXIII -2003[END_REF].

When dealing with errors that are not distributed as a Gaussian distribution, different tools are needed to obtain the upper bounds for the quantities in Equation (3.25) (see Section 3.4.3 for a complete discussion of this point of the proof). Let us continue the sketch of the proof before coming back to this point.

If for all v, µ v and γ v satisfying Equation (3.12), by using Equation (3.24) we deduce that with high probability,

C m -f 2 n ≤ m -f 2 n + v∈S f [γ v g v n + µ v g v Hv ] + v / ∈S f [γ v f v n + µ v f v Hv ].
Besides, we can express the decomposability property of the penalty as follows (see lemma 3.5.2, page 85): over the set where the empirical process is controlled as stated above, we have with high probability,

v / ∈S f [γ v f v n + µ v f v Hv ] ≤ C v∈S f [γ v g v n + µ v g v Hv ].
Putting the things together, and using that g v Hv ≤ 2, we obtain the following upper bound:

C m -f 2 n ≤ m -f 2 n + v∈S f [µ v + γ v g v n ].
The last important step consists in comparing v∈S f g v n to v∈S f g v n . To do so, we show first (see lemma 3.5.3 page 86) that for all v ∈ P, with high probability,

g v n ≤ 2 g v 2 + γ v .
Using inequality above and that for all positive K, 2ab ≤ (1/K)a 2 + Kb 2 we obtain,

C m -f 2 n ≤ m -f 2 n + v∈S f (µ v + γ 2 v ) + v∈S f g v 2 2 , ≤ m -f 2 n + v∈S f (µ v + γ 2 v ) + v∈P g v 2 2
.

Then we use the orthogonality assumption between the spaces H v ,

v∈P g v 2 2 = v∈P g v 2 2 = g 2 2 ,
which allows us to obtain the following result:

C m -f 2 n ≤ m -f 2 n + v∈S f (µ v + γ 2 v ) + f -f 2 2 .
It remains now to consider different cases according to the rankings of ff 2 2 and ff 2 n to get the result of Theorem 3.3.1.

Main arguments of the proof of Theorem 3.3.1 and motivation for the choice π α 75

If ff 2 ≤ ff n the result is obtained by a simple rearrangement of the terms.

If ff 2 ≥ ff n , under some suitable assumptions it is shown (see Lemma 3.5.4 page 87) that with high probability we have

f -f 2 ≤ √ 2 f -f n .
One of the steps to prove the inequality above is to lower bound the expectation of the supremum of the empirical process, i.e. E ε sup g |V n,ε (g)| by a function of the covering number of the functional class under study, say G. In order to solve this step in the Gaussian regression model one may use the Sudakov minoration in [START_REF] Pisier | The volume of convex bodies and Banach space geometry[END_REF], for which the minoration is obtained thanks to the Slepian's Lemma. The Slepian's Lemma is specific to the Gaussian setting, and it does not hold when dealing with errors that are not distributed as a centered Gaussian distribution.

In the regression model (see Equation (3.1)) with error ε that is distributed with density proportional to π α ∈ D, the proof of the upper bound stated in Theorem 3.3.1, needs two following mathematical tools: The Point 1. is solved using a Sudakov type minoration which is a consequence of the result obtained by [START_REF] Talagrand | The supremum of some canonical processes[END_REF]. More precisely, it can be shown (see Corollary 3.4.1 page 80) that for ε = (ε 1 , ..., ε n ) being i.i.d. random variables distributed with density π α ∈ D (see Equation (3.2)), and for all δ > 0, we have,

1 K log N (δ, G, . ) ≤ ( 2nE ε sup g∈G |V n,ε (g)| δ ) 2 1 [2nEε sup g∈G |Vn,ε(g)|,∞) (δ) + ( 2nE ε sup g∈G |V n,ε (g)| δ ) α 1 (0,2nEε sup g∈G |Vn,ε(g)|] (δ), (3.26)
where K is a constant that depends on α only, . is the Euclidean norm, N (δ, G, . ) is the δ-covering number of the metric space (G, . ), and

1 A : A → {0, 1} is the indicator function of A ⊂ A, i.e. 1 A (a) = 1 if a ∈ A, 0 if a / ∈ A.
The proof of Lemma 3.5.4 proceeds using Equation (3.24) and is concluded under the Hypothesis (3.12) and (3.14).

The Point 2. is solved using a concentration inequality (see Corollary 3.4.2 page 83) which is a consequence of the result obtained by [START_REF] Shu | A characterization of a class of convex log-sobolev inequalities on the real line[END_REF]. The control of the Empirical process is done in their Lemma 1. This lemma is proved using Lemma 8.4 in van [START_REF] Van De Geer | Empirical Processes in M-Estimation, Cambridge Series in Statistical and Probabilistic Mathematics[END_REF], for which the errors should have sub-Gaussian tails, i.e.

max i E exp( ε 2 i C 1 ) ≤ C 2 ,
where C 1 and C 2 are constants.

Afterwards, it was shown by [START_REF] Raskutti | Minimax-optimal rates for sparse additive models over kernel classes via convex programming[END_REF] (see Section 3.4. of their paper) that the convergence rate of this estimator is sub-optimal.

Koltchinskii and Yuan (2010) considered a large class of loss functions, called loss functions of quadratic type, which satisfies the boundedness conditions. More precisely, for l being a loss function, they assume that l(Y, .) is uniformly bounded from above by a numerical constant. So for a given distribution of the observations Y , there may exists a loss function that belongs to the class of the loss functions of quadratic type (see Section 2.1. of their paper for some examples).

They consider the input variables X that may be not independent, and they do not assume that there is orthogonality between their RKHS, therefore

v f v 2 = v f v 2 .
Instead, in their Section 2.2., they introduce some geometric characteristics related to the degree of dependence of their RKHS, which insures almost orthogonality between these spaces.

The control of the empirical process is done in their Lemma 9. This lemma is proved under the global boundedness condition and the assumptions of the loss functions of quadratic type. We consider the quadratic loss function to obtain an estimator of the function m in the regression model defined in Equation (3.1), with error ε that is non-bounded. This case is not included in the class of the loss functions of quadratic type. We do not impose the global boundedness condition. Instead, we assume that for all v ∈ P the functions f v are uniformly bounded. More precisely, the quantity sup X∈X |f v (X)| is bounded from above by a constant. This assumption is easily satisfied as soon as the kernel k v is bounded on the compact set X ,

sup X∈X |f v (X)| ≤ sup X∈X k v (X v , X v ) f v Hv .
For a detailed discussion on this subject, we refer to the paper by [START_REF] Raskutti | Minimax-optimal rates for sparse additive models over kernel classes via convex programming[END_REF].

In the Gaussian regression model, -Huet and Taupin (2017) assumed that the unknown function m admits a Hoeffding decomposition involving the main effects and interactions. They obtained a RKHS ridge group sparse estimator of a meta-model that approximates the Hoeffding decomposition of m. They established upper bounds on the risk in the L 2 -norm and the empirical L 2 -norm. [START_REF] Raskutti | Minimax-optimal rates for sparse additive models over kernel classes via convex programming[END_REF] and Huet and Taupin (2017) do not assume global boundedness condition. Instead, they assume that for all v ∈ P the functions f v are uniformly bounded. The proof of their results relies on the empirical Gaussian process methods such as Sudakov minoration [START_REF] Pisier | The volume of convex bodies and Banach space geometry[END_REF] and concentration inequalities for Gaussian processes.

As we are not in the Gaussian regression model, these methods could not be used in our work. We require new tools that we describe in details in the two next Sections.

Sudakov minoration

In the following Section, we recall the definition of the covering numbers, the statement of the classical Sudakov minoration, which is specific to the Gaussian process, and the generalized Sudakov minoration known also as the Sudakov minoration principal, which could be applied to some other processes. In Section 3.4.2.2 we state the appropriate Sudakov type minoration to the process associated with the random variables that are distributed with density π α ∈ D (see Equation (3.2)) in Corollary 3.4.1. in the regression model with non-Gaussian and non-bounded error

Introduction

Let T be a set of square-integrable functions, i.e. T ⊂ L 2 , and . be the Euclidean norm. For any δ > 0, we denote by C(δ, T, . ) the δ-covering set of the metric space (T, . ):

C(δ, T, . ) = f 1 , ..., f N : ∀f ∈ T, ∃k ∈ {1, ..., N } such that f -f k ≤ δ .
The δ-covering number of (T, . ), denoted N (δ, T, . ), is the cardinal of the smallest covering set. A proper covering restricts the covering to use only elements in the set T . It can be shown that the covering numbers and the proper covering numbers are related by the following inequality:

N (δ, T, . ) ≤ N proper (δ, T, . ) ≤ N ( δ 2 , T, . ). (3.27)
Consider a random variable Z such that E(Z 2 ) < ∞, and consider an i.i.d. sequence

{Z i } n i=1 distributed like Z. To each t = (t 1 , ..., t n ) of T ⊂ L 2 one can associate the process V t = n i=1 Z i t i , t ∈ T .
In order to link the covering number on a class T , i.e. N (δ, T, . ), to the expectation of the supremum of the process V t = n i=1 Z i t i in the Gaussian setting, the classical Sudakov minoration could be used [START_REF] Pisier | The volume of convex bodies and Banach space geometry[END_REF]):

1 K log N (δ, T, . ) ≤ nE Z sup t∈T n i=1 Z i t i δ 2 .
(3.28)

When dealing with the processes V t = n i=1 Z i t i , t ∈ T associated with the random variables {Z i } n i=1 that are not Gaussian, a generalized Sudakov minoration, known also as the Sudakov minoration principal, could be used to lower bound the value E Z sup t∈T n i=1 Z i t i . Let us recall this inequality. Definition 3.4.1 (Definition 1.1. in Latała ( 2014)) Let Z = (Z 1 , ..., Z n ) be a random vector in R n . We say that Z satisfies the L p -Sudakov minoration principle with a constant K > 0, SM P p (K ), if for any set T ⊂ R n with |T | > exp(p) such that

E Z t,s∈T | n i=1 (t i -s i )Z i | p 1/p := n i=1 (t i -s i )Z i p ≥ δ, ∀s, t ∈ T, s = t, (3.29)
we have

K δ ≤ E Z sup t,s∈T n i=1 (s i -t i )Z i .
A random vector Z satisfies the Sudakov minoration principle with a constant K , SM P (K ), if it satisfies SM P p (K ) for any p ≥ 1.

If {Z i } n i=1 are independent symmetric ±1 random variables or equivalently if the vector Z = (Z 1 , ..., Z n ) is uniformly distributed on the cube [-1, 1] the Sudakov minoration principal with universal K was proven by [START_REF] Talagrand | Regularity of infinitely divisible processes[END_REF].

Main arguments of the proof of Theorem 3.3.1 and motivation for the choice π α 79

Latała (2014) proved the Sudakov minoration principal for the independent logconcave random variables. A measure on R n with the full dimensional support is log-concave if and only if it has a density of the form exp(-φ(x)), where φ : R n → (-∞, ∞] is convex [START_REF] Borell | Convex measures on locally convex spaces[END_REF]). In the dependent setting the Sudakov minoration principal for the log-concave random variables was proven by [START_REF] Bednorz | Some remarks on the sudakov minoration[END_REF].

As we are in the independent setting and the densities π α ∈ D (see Equation (3.2)) are log-concave, the Sudakov minoration obtained by [START_REF] Latała | Sudakov-type minoration for log-concave vectors[END_REF] holds in our context. However, we could not deduce from the result obtained by Latała (2014) the adapted Sudakov type minoration that leads to obtain the optimal rate of convergence for our estimator. By optimal we mean the same rate of convergence as in the Gaussian regression setting (see Huet and Taupin (2017)). This is the reason why we restricted ourselves to the densities π α ∈ D for which there exists a result given by [START_REF] Talagrand | The supremum of some canonical processes[END_REF].

In the next Section we provide in Corollary 3.4.1 the appropriate Sudakov type minoration for the random variables that are distributed with density π α ∈ D. This Corollary is a consequence of the result obtained by [START_REF] Talagrand | The supremum of some canonical processes[END_REF].

Sudakov minoration for density π α

In this Section we state in Corollary 3.4.1 the Sudakov minoration appropriate for the random variables that are distributed with density π α ∈ D (see Equation (3.2)). This Corollary is a consequence of the Sudakov minoration stated in Theorem 3.1. in [START_REF] Talagrand | The supremum of some canonical processes[END_REF]. We start by introducing some notation that we need in the rest of this Section.

Let us denote by α the conjugate exponent of α, i.e. 1/α + 1/α = 1. So, for all α > 2 we have 1 < α < 2.

We consider the sets B α and U α(u), u ≥ 0 defined as follows:

B α = x ∈ R n : n k=1 |x k | α ≤ 1 , (3.30)
and

U α(u) = x ∈ R n : n i=1 η α(x i ) ≤ u, u ≥ 0 , (3.31) where η α(x i ) = x 2 i 1 [-1,1] (x i ) + |x i | α1 (-∞,-1]∩[1,∞) (x i ).
For T ⊂ L 2 and u ≥ 0, let D(T, U α(u)) be a covering set of translates of T by U α(u):

D(T, U α(u)) = f 1 , ..., f N : ∀f ∈ T, ∃k ∈ {1, ..., N } such that f -f k ∈ U α(u) , = f 1 , ..., f N : ∀f ∈ T, ∃k ∈ {1, ..., N } such that N i=1 η α(f i -f k i ) ≤ u .

in the regression model with non-Gaussian and non-bounded error

We denote by N (T, U α(u)) the minimum number of translates of U α(u) by elements of T needed to cover T .

Lemma 3.4.1 For all α ≤ 2 and u ≥ 0, it is shown that [START_REF] Talagrand | The supremum of some canonical processes[END_REF]):

U α(u) ⊂ (u 1/2 B 2 + u 1/ αB α).

(3.32)

Remark 3.4.1 If α ≤ 2 and u ≥ 0, then

U α(u) ⊂ 2 × max(u 1/2 , u 1/ α)B 2 .
The proof of Remark 3.4.1 is given in Section 3.A.1 page 108.

Theorem 3.4.1 (Theorem 3.1. in [START_REF] Talagrand | The supremum of some canonical processes[END_REF]) Let Z = (Z 1 , ..., Z n ) be i.i.d. random variables distributed with density π α ∈ D defined in Equation (3.2), U α(u), u ≥ 0 be defined by (3.31) and T ⊂ L 2 . Set

M = E Z sup t∈T n i=1 t i Z i , (3.33)
then it is shown that:

N (T, U α(M )) ≤ exp(KM ), (3.34)
where K is a constant that depends on α only.

Remark 3.4.2 According to Theorem 3.4.1 and Remark 3.4.1 for all u ≥ 0 we have,

N (2 × max(u 1/2 , u 1/ α), T, . ) ≤ N (T, U α(u)) ≤ exp(Ku). (3.35)
To be more precise, since 1 < α < 2 we have (i) For u ≤ 1, u 1/ α ≤ u 1/2 and N (2u 1/2 , T, . ) ≤ exp(Ku).

(ii) For u ≥ 1, u 1/ α ≥ u 1/2 and N (2u 1/ α, T, . ) ≤ exp(Ku).

Corollary 3.4.1 Under the same assumptions as for Theorem 3.4.1 we have for all δ > 0,

1 K log N (δ, T, . ) ≤ ( 2M δ ) α 1 (0,2M ] (δ) + ( 2M δ ) 2 1 [2M,∞) (δ),
which is exactly Equation (3.26) with M defined in Equation (3.33).

The proof of Corollary 3.4.1 is given in Section 3.A.2 page 108.

Concentration inequality

We start this Section with a small introduction on the concentration inequalities context in Section 3.4.3.1, and we detail the concentration inequality used in our work in Section 3.4.3.2.

Introduction

Let Z = (Z 1 , ..., Z n ) be a random vector in R n , and the function φ from R n to R be convex and 1-Lipschitz with respect to the Euclidean norm on R n , i.e.

φ(Z) -φ(Z ) ≤ Z -Z , Z, Z ∈ R n .
We are interested in the concentration inequalities of order two that provide bounds on how φ(Z) deviates from its expected value. More precisely, for P being the probability measure on R n , and for all u ≥ 0,

P |φ(Z) -E(φ(Z))| ≥ u ≤ C 1 exp - u 2 C 2 , (3.36)
where C 1 , and C 2 are constants.

It was shown by [START_REF] Ledoux | Probability in Banach Spaces: isoperimetry and processes[END_REF] that, if Z = (Z 1 , ..., Z n ) is a centered Gaussian random vector in R n , then:

P |φ(Z) -E(φ(Z))| ≥ u ≤ 4 exp - u 2 2 .
This result could be proved using an inequality established by geometric arguments and an induction on the number of coordinates.

After that, an alternative approach to some of Talagrand's inequalities was proposed by [START_REF] Ledoux | On talagrand's deviation inequalities for product measures[END_REF] based on the log-Sobolev inequalities. He showed that if the probability measure P on [0, 1] n satisfies the log-Sobolev inequality then it satisfies the concentration inequalities of the form (3.36), i.e. the log-Sobolev inequality implies the deviation inequality.

We say that the probability measure P satisfies the log-Sobolev inequality for a class of functions Ψ with loss function R : R n → [0, +∞), if for every ψ ∈ Ψ we have,

Ent(exp(ψ)) ≤ CE(R(∇ψ) exp(ψ)),
where ∇ψ is the usual gradient of ψ, and Ent(exp(ψ)) is the usual entropy of exp(ψ), i.e. Ent(exp(ψ

)) = E(ψ exp(ψ)) -E(exp(ψ)) log(E(exp(ψ))).
This inequality was first introduced by [START_REF] Gross | Logarithmic sobolev inequalities[END_REF] with R(x) = x 2 , x ∈ R n and Ψ being the class of C 1 functions. A lot of work has been done with different loss and class of functions, see for example [START_REF] Bobkov | Poincaré's inequalities and talagrand's concentration phenomenon for the exponential distribution[END_REF][START_REF] Gentil | Modified logarithmic Sobolev inequalities and transportation inequalities[END_REF][START_REF] Kleijnen | Design and Analysis of Simulation Experiments[END_REF].

In the rest of this Chapter, we assume that Ψ is the class of convex functions, and we consider only the quadratic loss R(x) = x 2 , x ∈ R n . Therefore, the probability measure P satisfies the convex log-Sobolev inequality if,

E(ψ exp(ψ)) -E(exp(ψ)) log(E(exp(ψ))) ≤ CE( ∇ψ 2 exp(ψ)).
(3.37) in the regression model with non-Gaussian and non-bounded error [START_REF] Adamczak | Logarithmic sobolev inequalities and concentration of measure for convex functions and polynomial chaoses[END_REF] found a sufficient condition for a class of probability distributions, denoted M(m, ρ 2 ) with m > 0 and ρ ≥ 0, on the real line, to satisfy the convex log-Sobolev inequality. He deduced then the following concentration inequality which is satisfied for all probability distributions belonging to M(m, ρ 2 ):

P φ(Z) -E(φ(Z)) ≥ u ≤ exp - u 2 4C(m, ρ 2 ) . (3.38)
We show in Lemma 3.4.2 that the probability distributions associated with the densities π α ∈ D defined in Equation (3.2) belong to M(m, ρ 2 ), and so they satisfy the convex log-Sobolev inequality. As a consequence the concentration inequality (3.38) holds for them.

Recall that (see Section 3.4.1 page 73) we need concentration bounds for the lower and upper tails of φ(Z), while the concentration inequality (3.38) does not contain these two sides. [START_REF] Shu | A characterization of a class of convex log-sobolev inequalities on the real line[END_REF] gave a sufficient and necessary condition for a probability measure on the real line to satisfy the convex log-Sobolev inequality. They obtained concentration bounds for the lower and upper tails of convex functions of independent random variables which satisfy the convex log-Sobolev inequality.

The result obtained by [START_REF] Shu | A characterization of a class of convex log-sobolev inequalities on the real line[END_REF] allows us to state in Corollary 3.4.2 the appropriate concentration inequality for the probability distributions associated with the densities π α ∈ D.

Concentration inequality for density π α

In this Section we give the definition of the class of probability distributions M(m, ρ 2 ) and some of its properties. We show in Lemma 3.4.2 that the probability distributions associated with the densities π α ∈ D (see Equation (3.2)) belong to M(m, ρ 2 ), and so they satisfy the convex log-Sobolov inequality (3.37). Finally, we state in Corollary 3.4.2 the appropriate concentration inequality for our work which is a consequence of the concentration inequality stated in Corollary 1.7. of the paper by [START_REF] Shu | A characterization of a class of convex log-sobolev inequalities on the real line[END_REF]. 2005)) The absolutely continuous distributions Π that satisfy for t ≥ m, (3.39) belong to M(m, ρ 2 ). In particular, if Π has density of the form exp(-V (x)) with dV (x)/dx ≥ x/ρ 2 and dV (-x)/dx ≤ -x/ρ 2 then Π ∈ M(1, ρ 2 ).

d dt log Π([t, ∞)) ≤ - t ρ 2 and d dt log Π((-∞, -t]) ≤ - t ρ 2 .
It is shown by [START_REF] Adamczak | Logarithmic sobolev inequalities and concentration of measure for convex functions and polynomial chaoses[END_REF] that the probability distributions belonging to M(m, ρ 2 ) satisfy the convex log-Sobolov inequality (3.37). Let us denote by Π α the probability distribution associated with the density π α ∈ D defined in Equation (3.2). In the following Lemma we will show that Π α satisfies the convex log-Sobolev inequality (3.37).

Lemma 3.4.2 There exists some m such that Π α ∈ M(m, ρ 2 ), and therefore Π α satisfies the convex log-Sobolev inequality (3.37).

The proof of Lemma 3.4.2 is given in Section 3.B.1 page 109.

As Π α ∈ M(m, ρ 2 ) and they satisfy the convex log-Sobolev inequality (3.37), so the concentration bound (3.38) holds for them. Recall that (see Section 3.4.1 page 73), we need a concentration bound for the both upper and lower tails of a convex function of the random variables that are distributed as Π α . Therefore, the concentration bound (3.38) is not sufficient for our work. We state in Corollary 3.4.2 the appropriate concentration inequality for our work which is a consequence of the concentration inequality obtained by [START_REF] Shu | A characterization of a class of convex log-sobolev inequalities on the real line[END_REF]. This result holds under a supplementary condition that we will state in the following Remark.

Remark 3.4.3 Let Z be a random variable distributed as Π α , then for every s > 0 the quantity E(exp(s|Z|)) exists and is finite.

The proof of Remark 3.4.3 is given in Section 3.B.2 page 110.

Note that, if α < 2 then E(exp(s|Z|)) ≮ ∞.

Corollary 3.4.2 Let Z = (Z 1 , ..., Z n ) be i.i.d. random variables distributed as Π α .

Then there exists A, B < ∞ (depending only on C in the log-Sobolev inequality (3.37)), such that for any convex (or concave) function φ : R n → R which is 1-Lipschitz (with respect to the Euclidean norm on R n ) we have:

P |φ(Z) -E(φ(Z))| ≥ u ≤ 2B exp - u 2 8A , u ≥ 0. (3.40)
Corollary 3.4.2 is a consequence of the concentration inequality shown by [START_REF] Shu | A characterization of a class of convex log-sobolev inequalities on the real line[END_REF]:

P |φ(Z) -M (φ(Z))| ≥ u ≤ B exp - u 2 A , u ≥ 0, (3.41)
where M is the median of φ(Z). in the regression model with non-Gaussian and non-bounded error

The proof of Corollary 3.4.2 is given in Section 3.B.3 page 111 and is based on the fact that the concentration inequalities around the mean and the median are equivalent up to a numerical constant [START_REF] Milman | Asymptotic Theory of Finite Dimensional Normed Spaces[END_REF]).

Proof of Theorem 3.3.1

The proof is based on four main lemmas proved in Section 3.5.2. In Section 3.5.1 other lemmas used all along the proof are stated.

Let us first establish inequalities that will be used in the following. Let f ∈ H and v ∈ S f (see (3.8)).

Using that for any v ∈ S f , and any norm

• in H v , f v -f v ≤ f v -f v and that for any v / ∈ S f , f v = 0, we get, v∈P µ v f v Hv - v∈P µ v f v Hv ≤ v∈S f µ v f v -f v Hv - v / ∈S f µ v f v Hv , (3.42)
and,

v∈P γ v f v n - v∈P γ v f v n ≤ v∈S f γ v f v -f v n - v / ∈S f γ v f v n . (3.43)
Combining (3.42), and (3.43), to the fact that for any function

f ∈ H, L( f ) ≤ L(f ), we obtain, m -f 2 n ≤ m -f 2 n + B, with B = 2V n,ε f -f + v∈S f [µ v f v -f v Hv + γ v f v -f v n ] - v / ∈S f [µ v f v Hv + γ v f v n ]. (3.44) If m -f 2 n ≥ B, we immediately get the result since in that case m -f 2 n ≤ 2 m -f 2 n ≤ 2 m -f 2 n + v∈S f µ v + v∈S f γ 2 v . If m -f 2 n < B, we get that f -m 2 n ≤2B (3.45) ≤4|V n,ε f -f | + 2 v∈S f [µ v f v -f v Hv + γ v f v -f v n ]. (3.46)
The control of the empirical process |V n,ε ff | is given by the following lemma (proved in Section 3.5.2.1, page 89).

Lemma 3.5.1 Let V n,ε be defined in (3.21). For any f in F, we consider the event T defined as (3.47) where λ n,v is defined in Equation (3.11) and where κ = 10 + 4∆. Then, for some positive constants c 1 , c 2 ,

T = ∀f ∈ F, ∀v ∈ P, |V n,ε f v -f v | ≤ κλ 2 n,v f v -f v Hv + κλ n,v f v -f v n ,
P X,ε (T ) ≥ 1 -c 1 v∈P exp(-nc 2 λ 2 n,v ). (3.48) Conditioning on T , Inequality (3.46) becomes f -m 2 n ≤4κ v∈P [λ 2 n,v f v -f v Hv + λ n,v f v -f v n ]+ 2 v∈S f [µ v f v -f v Hv + γ v f v -f v n ],
which may be decomposed as follows

f -m 2 n ≤ v∈S f [4κλ 2 n,v + 2µ v ] f v -f v Hv + v∈S f [4κλ n,v + 2γ v ] f v -f v n + 4 v / ∈S f κλ 2 n,v f v -f v Hv + 4 v / ∈S f κλ n,v f v -f v n .
If we choose C 1 ≥ κ in Theorem 3.3.1, then κλ 2 n,v ≤ µ v and κλ n,v ≤ γ v and the previous inequality becomes

f -m 2 n ≤6 v∈S f [µ v f v -f v Hv + γ v f v -f v n ]+ 4 v / ∈S f [µ v f v Hv + γ v f v n ]. (3.49)
Next we use the decomposability property of the penalty expressed in the following lemma (proved in Section 3.5.2.2 page 92).

Lemma 3.5.2 For any f ∈ F, under the assumptions of Theorem 3.3.1, conditionally on T (see (3.47)), we have:

v / ∈S f µ v f v Hv + v / ∈S f γ v f v n ≤ 3 v∈S f µ v f v -f v Hv + 3 v∈S f γ v f v -f v n . (3.50)
Hence, by combining (3.49) and Lemma 3.5.2 we obtain

f -m 2 n ≤ 18 v∈S f µ v f v -f v Hv + γ v f v -f v n .

in the regression model with non-Gaussian and non-bounded error

For each v, f v -f v Hv ≤ 2 (because the functions f v et f v belong to the class F, see (3.6)), and consequently, for some constant C,

f -m 2 n ≤ C v∈S f µ v + v∈S f γ v f v -f v n . (3.51)
To finish the proof it remains to compare the two quantities

v∈S f f v -f v 2 n and v∈S f f v -f v 2 n .
For that purpose we show that

v∈S f f v -f v n is less than v∈S f f v -f v 2
2 plus an additive term coming from concentration results (see the Lemma given below). Next, thanks to the orthogonality of the spaces H v with respect to L 2 (P X , X ),

v∈S f f v -f v 2 2 = v∈S f f v -f v 2 2 .
To conclude, it remains to consider several cases, according to the rankings of

v∈S f f v -f v 2 2
and

v∈S f f v -f v 2
n . This is the subject of the following lemma whose proof is given in Section 3.5.2.3, page 93.

Lemma 3.5.3 For f ∈ H, let A be the event

A = ∀f ∈ F, ∀v ∈ P, f v -f v n ≤ 2 f v -f v 2 + γ v .
(3.52)

Then, for some positive constant c 2 ,

P X,ε (A) ≥ 1 - v∈P exp(-nc 2 γ 2 v ).
On the set A, Inequality (3.51) provides that, for all

K > 0 1 C f -m 2 n ≤ v∈S f [µ v + 2γ v f v -f v 2 + γ 2 v ], ≤ v∈S f [µ v + (1 + K)γ 2 v + 1 K f v -f v 2 2 ], (3.53) ≤ v∈S f [µ v + (1 + K)γ 2 v ] + 1 K v∈P f v -f v 2 2 , ≤ v∈S f [µ v + (1 + K)γ 2 v ] + 1 K v∈P f v -f v 2 2 .
(3.54) Inequality (3.53) uses the inequality 2ab ≤ 1 K a 2 + Kb 2 for all positive K, and Inequality (3.54) uses the orthogonality with respect to L 2 (P X ).

In the following we have to consider several cases, according to the rankings of

v∈P f v -f v 2 and v∈P f v -f v n .
More precisely, we consider two following cases:

Case 1: If v∈P f v -f v 2 ≤ v∈P f v -f v n . Case 2: If v∈P f v -f v 2 ≥ v∈P f v -f v n .

Proof of Theorem 3.3.1 87

Case 1: From (3.54), for any f ∈ H, we get

1 C f -m 2 n ≤ v∈S f [µ v + (1 + K)γ 2 v ] + 1 K f -f 2 n .
Hence, using that for all K > 0,

f -f 2 n ≤ (1 + K ) f -m 2 n + (1 + 1 K ) f -m 2 n , (3.55)
we obtain for a suitable choice of K , say 1 + K < K/C, that, for some positive constant C ,

f -m 2 n ≤ C f -m 2 n + v∈S f µ v + v∈S f γ 2 v .
This shows the result in Case 1.

Case 2: This case is solved by applying the following Lemma (proved in Section 3.5.2.4, page 93), which states that with high probability, f -

f 2 ≤ √ 2 f -f n . Lemma 3.5.4 Let f = v f v ∈ F with support S f , λ n,v
be defined by (3.11), and let G(f ) be the class of functions written as g = v∈P g v , such that

g v Hv ≤ 2 satisfying for all f ∈ F C1 v∈P µ v g v Hv + v∈P γ v g v n ≤ 4 v∈S f µ v g v Hv + 4 v∈S f γ v g v n C2 v∈S f γ v g v n ≤ 2 v∈S f γ v g v 2 + v∈S f γ 2 v C3 g n ≤ g 2
Then the event

g 2 n ≥ g 2 2 2 ,
have probability greater than 1 -c 1 exp(-nc 3 v∈S f λ 2 n,v ) for some constants c 1 and c 3 .

If f is such that |S f | = 0, then Condition C1 is not satisfied except if g v = 0
for all v ∈ P. Because we will apply Lemma 3.5.4 to

g v = f v -f v , this event has probability 0. If f is such that |S f | ≥ 1, then Condition C1 is satisfied: from Equation (3.50) in Lemma 3.5.2 we have, v / ∈S f µ v f v Hv + v∈S f µ v f v -f v Hv + v / ∈S f γ v f v n + v∈S f γ v f v -f v n ≤ 3 v∈S f µ v f v -f v Hv + v∈S f µ v f v -f v Hv + 3 v∈S f γ v f v -f v n + v∈S f γ v f v -f v n , ⇔ v∈P µ v f v -f v Hv + v∈P γ v f v -f v n ≤ 4 v∈S f µ v f v -f v Hv + 4 v∈S f γ v f v -f v n
. in the regression model with non-Gaussian and non-bounded error

Moreover, Assumption nλ 2 n,v ≥ -C 2 log(λ n,v ) implies that λ n,v = K n,v / √ n with K n,v → ∞.
Then,

exp(-nc 3 v∈S f λ 2 n,v ) ≤ exp(-c 3 |S f | min v∈P K 2 n,v ),
and the event

C = ∀f ∈ F, such that g = v∈P ( f v -f v ) ∈ G(f ), and g 2 n ≥ g 2 2 2
(3.56) has probability greater than 1 -η/3 for some 0 < η < 1.

Conditioning on the events T and A (defined by (3.47) and (3.52)), v∈P ( f vf v ) belongs to the set G(f ). According to (3.54), we conclude in the same way as in the first case.

Finally, it remains to quantify P X,ε (T ∩ A ∩ C). Following Lemma 3.5.1, and Lemma 3.5.3, T , respectively A, has probability greater than 1-c 1 v∈P exp(-nc 2 λ 2 n,v ), respectively 1-v∈P exp(-nγ 2 v ). Each of these probabilities is greater than 1-η/3 thanks to the assumption nλ 2 n,v ≥ -C 2 log λ n,v . 2

Intermediate Lemmas

Lemma 3.5.5 If E X,ε denotes the expectation with respect to the distribution of (X, ε), we have for all t > 0,

E X,ε W n,2,v (t) ≤ Q n,v (t).
Its proof is given in Section 3.5.3.1 page 101.

Lemma 3.5.6 Let b > 0 and let G(t) be the following class of functions:

G(t) = g v ∈ H v , g v Hv ≤ 2, g v 2 ≤ t, g v ∞ ≤ b . (3.57)
Let Ω v,t be the event defined as

Ω v,t = sup gv∈G(t) {| g v 2 -g v n |} ≤ bt 2 .
(3.58)

Then for any t ≥ ν n,v , the event Ω v,t has probability greater than 1 -exp(-c 2 nt 2 ), for some positive constant c 2 .

Its proof is given in Section 3.5.3.2, page 101.

Lemma 3.5.7 For any function

g v ∈ H v satisfying g v Hv ≤ 2, g v ∞ ≤ b and g v 2 ≥ t, for all t ≥ ν n,v and b ≥ 1, the event (1 - b 2 ) g v 2 ≤ g v n ≤ (1 + b 2 ) g v 2
has probabilty greater than 1 -exp(-c 2 nt 2 ) for some positive constant c 2 .

Proof of Theorem 3.3.1 89

Its proof is given in Section 3.5.3.3, page 103.

Lemma 3.5.8 If E ε denotes the expectation with respect to the distribution of ε, we have

P X,ε |W n,n,v (t) -E ε W n,n,v (t) | ≥ δt ≤ 2B exp(- nδ 2 8A
).

(3.59)

Its proof is given in Section 3.5.3.4, page 103.

Lemma 3.5.9 Conditionally on the space Ω v,t defined by (3.58), we have the following inequalities:

P X,ε |W n,2,v (t) -E ε W n,2,v (t) | ≥ δt ≤ 2B exp(- nδ 2 32A
), (3.60)

P X E ε W n,2,v (t) -E X,ε W n,2,v (t) ≥ x ≤ exp(- nx 2 Q n,v (t)
).

(3.61)

Its proof is given in Section 3.5.3.5, page 104.

Lemma 3.5.10 Let λ n,v be defined at Equation (3.11), ∆ at Equation (3.10) and κ = 10 + 4∆. Conditionally on the space Ω v,λn,v defined at Equation (3.58), for some positive constants c 1 , c 2 , with probability greater than

1 -c 1 exp(-c 2 nλ 2 n,v ), we have W n,n,v (λ n,v ) ≤ κλ 2 n,v and E ε W n,n,v (λ n,v ) ≤ κλ 2 n,v . (3.62)
Its proof is given in Section 3.5.3.6, page 106.

3.5.2 Proof of lemma 3.5.1 to 3.5.4

3.5.2.1 Proof of lemma 3.5.1

For f ∈ F and v ∈ P, let

g v = f v -f v . Note that g v Hv ≤ 2. Let us show that |V n,ε (g v )| ≤ κ λ 2 n,v g v Hv + λ n,v g v n .
(3.63)

We start by writing that

|V n,ε (g v )| = g v Hv V n,ε g v g v Hv ≤ g v Hv W n,n,v g v n g v Hv .
(3.64)

Consider the two following cases:

Case A:

g v n ≤ λ n,v g v Hv ,
Case B:

g v n > λ n,v g v Hv .

in the regression model with non-Gaussian and non-bounded error

Case A: Since g v n ≤ λ n,v g v Hv , we have

W n,n,v g v n g v Hv ≤ W n,n,v (λ n,v ).
We then apply Lemma 3.5.10, page 89, and conclude that (3.63) holds in Case A for each v ∈ P since, with high probability

|V n,ε (g v )| ≤ κλ 2 n,v g v Hv ≤ κλ 2 n,v g v Hv + κλ n,v g v n .
(3.65)

Case B: Consider now the case g v n > λ n,v g v Hv and let us show that for any v ∈ P,

W n,n,v ( g v n g v Hv ) ≤ κλ n,v g v n .
Let r v be a deterministic number such that r v > λ n,v . Our first step relies on the study of the process W n,n,v (r v ), for r v > λ n,v . In that case we state two results:

R1 For any deterministic r v ≥ λ n,v , with probability greater than 1-c 1 exp(-c 2 nλ 2 n,v ), W n,n,v (r v ) ≤ κr v λ n,v .
(3.66)

R2 Inequality (3.66) continues to hold for random r v of the form

r v = g v n g v Hv .
Combining these two points implies that, with probability greater than 1-c 1 exp(-c 2 nλ 2 n,v ),

g v Hv W n,n,v g v n g v Hv ≤ κ g v n λ n,v .
Consequently, in Case B, according to (3.64), for each v, Inequality (3.63) holds because

|V n,ε (g v )| ≤ κ g v n λ n,v ≤ κλ 2 n,v g v Hv + κλ n,v g v n .
This ends up the proof of Lemma 3.5.1.

Proof of R1 From Lemma 3.5.8, page 89 with t = r v and δ = λ n,v , we get that with probability greater than 1 -2B exp(-nλ 2 n,v /8A),

W n,n,v (r v ) ≤ E ε (W n,n,v (r v )) + r v λ n,v (3.67)
Next we prove that for some positive r v , with probability greater than 1-exp(-ncλ 2 n,v ), we have

E ε (W n,n,v (r v )) ≤ κr v λ n,v .
(3.68)

Let ν n,v be defined as the smallest solution of E ε (W n,n,v (t)) ≤ κt 2 . For W n,n,v , defined by (3.23), we write

E ε (W n,n,v (r v )) = r v ν n,v E ε sup |V n,ε (g v )|, g v Hv ≤ 2( ν n,v r v ), g v n ≤ ν n,v .
Besides, Lemma 3.5.10 stated that on the event

Ω v,λn,v , E ε (W n,n,v (λ n,v )) ≤ κλ 2 n,v .
It follows from the definition of ν n,v , and Lemma 3.5.6, that ν n,v ≤ λ n,v for all v ∈ P with probability greater than 1 -exp(-nc 2 v∈P λ 2 n,v ). Consequently, for any deterministic r v such that r v ≥ λ n,v , we have

ν n,v ≤ λ n,v ≤ r v ⇔ ν n,v r v ≤ 1,
and so,

E ε (W n,n,v (r v )) = r v ν n,v E ε sup |V n,ε (g v )|, g v Hv ≤ 2, g v n ≤ ν n,v , ≤ r v ν n,v E ε (W n,n,v ( ν n,v )) ≤ r v ν n,v κ ν 2 n,v = κr v ν n,v ≤ κr v λ n,v .
Proof of R2 Let us prove R2 by using a peeling-type argument. Our aim is to prove that (3.66) holds for any r v of the form

r v = g v n g v Hv .
Since g v ∞ / g v Hv ≤ 1, we have g v n / g v Hv ≤ 1. We thus restrict ourselves to

r v satisfying r v = g v n / g v Hv with g v n / g v Hv ∈ (λ n,v , 1].
We start by splitting the interval (λ n,v , 1] into M disjoint intervals such that

(λ n,v , 1] = ∪ M k=1 (2 k-1 λ n,v , 2 k λ n,v ],
for some M that will be chosen later. Consider the event D c defined as follows:

D c = ∃v ∈ P and ∃g v , such that |V n,ε (g v )| ≥ κλ n,v g v n , with g v n g v Hv ∈ (λ n,v , 1] .
We prove that, for some positive constants c 1 , c 2 ,

P (D c ) ≤ c 1 exp(-c 2 nλ 2 n,v ). For g v ∈ D c , let k be the integer in {1, • • • , M }, such that 2 k-1 λ n,v ≤ g v n g v Hv ≤ 2 k λ n,v .
This k satisfies

g v Hv W n,n,v 2 k λ n,v ≥ g v Hv W n,n,v g v n g v Hv ≥ |V n,ε (g v )| ≥ κλ n,v g v n .
in the regression model with non-Gaussian and non-bounded error Therefore, we get

W n,n,v (2 k λ n,v ) ≥ κλ n,v g v n g v Hv ≥ κλ 2 n,v 2 k-1 ≥ κ λ n,v 2 2 k λ n,v .
By taking r v = 2 k λ n,v in (3.66), we have

P W n,n,v (2 k λ n,v ) ≥ κ λ n,v 2 2 k λ n,v ≤ c 1 exp(-c 2 nλ 2 n,v ).
Now let us write D c as follows:

D c = M k=1 ∃v and ∃ g v such that |V n,ε (g v )| ≥ κλ n,v g v n , with g v n g v H ∈ (2 k-1 λ n,v , 2 k λ n,v ] .
The set D c has probability smaller than

c 1 M exp(-c 2 nλ 2 n,v ). If we choose M such that log M ≤ (c 2 /2)nλ 2 n,v
, then the probability of the set T is greater than

1 - v∈P c 1 exp(- c 2 2 nλ 2 n,v ).
It follows that R2 is proved which ends up the proof of Lemma 3.5.1.

1 2 f -m 2 n ≤2|V n,ε ( f -f )| + v∈S f [µ v f v -f v Hv + γ v f v -f v n ]- v / ∈S f [µ v f v Hv + γ v f v n ].
On the event T defined in (3.47) we have

1 2 f -m 2 n ≤2κ v∈P λ 2 n,v f v -f v Hv + 2κ v∈P λ n,v f v -f v n + v∈S f [µ v f v -f v Hv + γ v f v -f v n ] - v / ∈S f [µ v f v Hv + γ v f v n ].
Rearranging the terms we obtain that

1 2 f -m 2 n ≤ v∈S f (2κλ 2 n,v + µ v ) f v -f v Hv + v∈S f (2κλ n,v + γ v ) f v -f v n + v / ∈S f (2κλ 2 n,v -µ v ) f v Hv + v / ∈S f (2κλ n,v -γ v ) f v n .

Proof of Theorem 3.3.1 93

Now, thanks to Assumption (3.12) with C 1 ≥ κ we have κλ 2 n,v ≤ µ v and 2κλ n,v ≤ γ v and Lemma 3.5.2 is shown since

0 ≤ 1 2 f -m 2 n ≤3 v∈S f µ v f v -f v Hv + 3 v∈S f f v -f v n - v / ∈S f µ v f v Hv - v / ∈S f γ v f v n . 2 3.5.2.3 Proof of lemma 3.5.3
Let us consider the following two cases:

f v -f v 2 ≤ γ v .
We apply Lemma 3.5.6 (page 88) to the function

g v = f v -f v . It satisfies g v ∈ G(γ v ) with b = 2 (recall that • ∞ ≤ • Hv ). Moreover, γ v ≥ C 1 λ n,v ≥ C 1 ν nv ≥ ν n,v as soon as C 1 ≥ 1.
It follows that, for some positive c 2 , with probability greater than 1-exp(-nc 2 γ 2 v ),

f v -f v n ≤ f v -f v 2 + γ v . f v -f v 2 ≥ γ v .
We apply Lemma 3.5.7 (page 88) to the function

g v = f v -f v with b = 2.
It follows that, for some positive c 2 , with probability greater than

1 -exp(-nc 2 γ 2 v ), f v -f v n ≤ 2 f v -f v 2 .
2 3.5.2.4 Proof of lemma 3.5.4

Throughout the proof, we make use of the quantity d n defined as follows: For β < 1/α and some constant η ,

d 2 n ≥ η n αβ-1 . (3.69)
Let G(f ) and G (f ) be the following sets:

G(f ) = g = v∈P g v , satisfying g v Hv ≤ 2, and Conditions C1, C2, C3 , G (f ) = g ∈ G(f ), such that g 2 = d n .
In order to prove this lemma we consider two cases: if

v∈P f v -f v 2 ≥ d n , and if v∈P f v -f v 2 ≤ d n . First, we suppose that v∈P f v -f v 2 ≥ d n ,
and we consider the two events B and B defined as follows:

B = ∀h ∈ G, h 2 n ≥ h 2 2 2
, and h 2 ≥ d n , in the regression model with non-Gaussian and non-bounded error and

B = ∀h ∈ G , h 2 n ≥ d 2 n 2 . (3.70) If h ∈ B , then h ∈ G, h 2 = d n and h 2 n ≥ d 2 n /2. It follows that h 2 n ≥ h 2 2 /2 and h 2 ≥ d n .
We just showed that the event B is included into the event B. So, this case is proved if the event B holds with high probability. Consider

Z n (G ) = sup g∈G d 2 n -g 2 n .
We show that the event

Z n (G ) ≤ d 2 n /2 has probability greater than 1-c 1 exp(-nc 3 d 2 n ). Consider a d n /8-covering of (G , • n ). So that, for all g in G there exists g k such that g -g k n ≤ d n 8 .
The associated proper covering number is:

N pr = N pr ( d n 8 , G , • n ). (3.71)
Now, for all g ∈ G , we write:

d 2 n -g 2 n = T 1 + T 2 , (3.72) with T 1 = g k 2 n -g 2 n and T 2 = d 2 n -g k 2 n .
The proof is splitted into four steps:

Step 1 The first step consists in showing that

T 1 = g k 2 n -g 2 n ≤ d 2 n 4 . (3.73)
Step 2 The second step consists in proving that, for N pr given at Equation (3.71) and for some constant C,

P X max k∈{1,••• ,Npr} [d 2 n -g k 2 n ] ≥ d 2 n 4 ≤ exp log N pr -Cnd 2 n .
Step 3 The third step concerns the control of N pr . Let σ 2 α be the variance of a random variable distributed with density π α ∈ D (see Equation (3.2)), then for some

K > 0, 1 K log N pr ≤ 32σ α √ n(E ε sup g∈G |V n,ε (g)|)/d n α 1 (0,32σα √ nEε sup g∈G |Vn,ε(g)|] (d n )+ 1 [32σα √ nEε sup g∈G |Vn,ε(g)|,∞) (d n ).
Step 4 The last step consists in bounding from above the Gaussian complexity. For some κ > 0

E ε sup g∈G v∈P |V n,ε (g v )| ≤ 4κ C 1 v∈S f (2µ v + γ 2 v ) + 2( v∈S f γ 2 v ) 1 2 d n ,
Let us conclude the proof of the lemma before proving these four steps. Putting together Steps 3 and 4 we have:

If d n ∈ [32σ α √ nE ε sup g∈G |V n,ε (g)|, ∞), then 1 K log N pr d n 8 , G , . n ≤ 1.

Thanks to

Step 2,

P X T 2 ≥ d 2 n 4 ≤ P X max k∈{1,••• ,Npr} [d 2 n -g k 2 n ] ≥ d 2 n 4 ≤ K exp -Cnd 2 n ,
and, therefore

P X Z n (G ) ≤ d 2 n 2 ≤ K exp -Cnd 2 n . (3.74) If d n ∈ (0, 32σ α √ nE ε sup g∈G |V n,ε (g)|], then 1 K log N pr d n 8 , G , . n ≤ (32σ α ) α n α 2 E ε sup g∈G |V n,ε (g)| d n α , ≤ (32σ α ) α n α 2 4κ C 1 d n ( v∈S f (2µ v + γ 2 v ) + 2( v∈S f γ 2 v ) 1 2 d n ) α , ≤ 128κσ α C 1 α n α 2 v∈S f (2µ v + γ 2 v ) d n + 2( v∈S f γ 2 v ) 1 2 α .
We have to show that log N pr -Cnd 2 n ≤ -c 3 nd 2 n or equivalently that log N pr ≤ Cnd 2 n , where

C = C -c 3 . Let A = K(128κσ α /C 1 ) α . We have, log N pr ≤ Cnd 2 n ⇔ An α 2 v∈S f (2µ v + γ 2 v ) d n + 2( v∈S f γ 2 v ) 1 2 α ≤ Cnd 2 n , ⇔ v∈S f (2µ v + γ 2 v ) d n + 2( v∈S f γ 2 v ) 1 2 α ≤ C A n 1-α 2 d 2 n , ⇔ v∈S f (2µ v + γ 2 v ) d n + 2( v∈S f γ 2 v ) 1 2 ≤ ( C A ) 1 α n 1 α -1 2 d 2 α
n .

Because

γ v = C 1 λ n,v and µ v = C 1 λ 2 n,v , log N pr ≤ Cnd 2 n ⇔ C 1 (2 + C 1 ) v∈S f λ 2 n,v d n + 2C 1 ( v∈S f λ 2 n,v ) 1 2 ≤ ( C A ) 1 α n 1 α -1 2 d 2 α
n .

Considering the first term in the left hand side, let

B = 1 2 × 1 C 1 (2 + C 1 ) ( C A ) 1 α , 96
Chapter 3. Risk upper bounds for RKHS ridge group sparse estimator in the regression model with non-Gaussian and non-bounded error

then v∈S f λ 2 n,v d n ≤ Bn 1 α -1 2 d 2 α n ⇔ d 2 n ≥ B -2α 2+α v∈S f λ 2 n,v 2α 2+α n α-2 α+2 .
As

v∈S f λ 2 n,v ≤ C 3 n 2β-1 (see Equation (3.14)), we get B -2α 2+α v∈S f λ 2 n,v 2α 2+α n α-2 α+2 ≤ ( B C 3 ) -2α 2+α n 4αβ 2+α -1 .
Therefore, the inequality

C 1 (2 + C 1 ) v∈S f λ 2 n,v d n ≤ 1 2 ( C A ) 1 α n 1 α -1 2 d 2 α n ,
will be satisfied if

d 2 n ≥ ( C 3 B ) 2α α+2 n 4αβ α+2 -1 .
For the second term, let

B = 1 2 × 1 2C 1 ( C A ) 1 α , then v∈S f λ 2 n,v 1 2 ≤ B n 1 α -1 2 d 2 α n ⇔ d 2 n ≥ B -α v∈S f λ 2 n,v α 2 n α-2 2 .
As

v∈S f λ 2 n,v ≤ C 3 n 2β-1 (see Equation (3.14)), then B -α v∈S f λ 2 n,v α 2 n α-2 2 ≤ C 3 B 2 α 2 n αβ-1 .
Therefore the inequality

2C 1 v∈S f λ 2 n,v 1 2 ≤ 1 2 C A 1 α n 1 α -1 2 d 2 α n ,
will be satisfied if

d 2 n ≥ C 3 B 2 α 2 n αβ-1 .
As α > 2, 4αβ/(α + 2) < αβ. Therefore, there exists a constant η , take for example

η = max ( C 3 B 2 ) α 2 , ( C 3 B ) 2α α+2 , such that if d 2 n ≥ η n αβ-1 , then log N pr ≤ Cnd 2 n ,

and

Step 2 states that

P X T 2 ≥ d 2 n 4 ≤ P X max k∈{1,••• ,Npr} [d 2 n -g k 2 n ] ≥ d 2 n 4 ≤ exp -c 3 nd 2 n .
3.5. Proof of Theorem 3.3.1 97 Now, we have

P X Z n (G ) ≤ d 2 n 2 = P X max g 1 ,••• ,g N [d 2 n -g k 2 n ] ≥ d 2 n 4 ≤ exp -c 3 nd 2 n . (3.75)
Finally, we obtain for c 1 = max(K, 1) and c 3 ≤ C (see Equations (3.74) and (3.75)):

P X Z n (G ) ≤ d 2 n 2 ≤ c 1 exp -c 3 nd 2 n .
Moreover, for n large enough, we have v∈S f λ 2 n,v ≤ d 2 n ≤ 1 (see Equations (3.14) and (3.69)), and

1 -c 1 exp -c 3 nd 2 n ≥ 1 -c 1 exp -c 3 n v∈S f λ 2 n,v .
Therefore,

P X Z n (G ) ≤ d 2 n 2 ≤ c 1 exp -c 3 n v∈S f λ 2 n,v .
Before proving the Steps 1 to 4 let us solve the second case: if v∈P f v -f v 2 ≤ d n then we consider the event B defined as follows:

B = ∀h ∈ G, h 2 n ≥ h 2 2 2
, and h 2 ≤ d n .

We have that the event B defined in Equation (3.70) is included in B and the same proof as in the first case applies. It is easy to see that,

Proofs of

T 1 = g k 2 n -g 2 n = 1 n n i=1 [(g k (X i )) 2 -(g(X i )) 2 ] = 1 n n i=1 [g k (X i ) -g(X i )][g k (X i ) + g(X i )] ≤ g k -g n 1 n n i=1 [g k (X i ) + g(X i )] 2 1 2
where in the inequality above we used Cauchy Schwarz inequality. Using the inequality (a + b) 2 ≤ 2a 2 + 2b 2 , g ∈ G , and the property that g satisfies Condition C3, we get

1 n n i=1 [g k (X i ) + g(X i )] 2 ≤ 2 g k 2 n + 2 g 2 n ≤ 4d 2 n .
in the regression model with non-Gaussian and non-bounded error Besides, the covering set is constructed such that g kg n ≤ d n /8. It follows that

Step 1 is proved.

Proof of Step 2:

We prove that for some constant C,

P X T 2 ≥ d 2 n 4 ≤ P X max 1≤k≤Npr [d 2 n -g k 2 n ] ≥ d 2 n 4 ≤ exp log N pr -Cnd 2 n .
As

g k ∈ G , d n = g k 2 .
Then

max 1≤k≤Npr [d 2 n -g k 2 n ] = max 1≤k≤Npr [ g k 2 2 -g k 2 n ].
Applying Theorem 3.5. in [START_REF] Chung | Concentration inequalities and martingale inequalities: a survey[END_REF] with

X = i (g k (X i )) 2
, for all positive λ we have:

P X n i=1 [g k (X i )) 2 ≤ nE(g k (X i )] 2 -λ ≤ exp - λ 2 2nE(g k (X)) 4 , or equivalently, P X g k 2 2 -g k 2 n ≥ λ n ≤ exp - λ 2 2nE(g k (X)) 4 .
Taking λ = nd 2 n /4 and using that g k 2 2 = d 2 n we get

P X d 2 n -g k 2 n ≥ d 2 n 4 ≤ exp - nd 4 n 32E(g k (X)) 4 . It follows that P X max 1≤k≤Npr [d 2 n -g k 2 n ] ≥ d 4 n 4 ≤ Npr k=1 exp - nd 4 n 32E(g k (X)) 4 ≤ exp log N pr - nd 4 n 32 max k E(g k (X)) 4 . (3.76) Moreover, g ∈ H, so g = v∈P g v ,
where the functions g v are centered and orthogonal in L 2 (P X ). Therefore E(g(X)) 4 is the sum of the following terms:

A 1 = v∈P E X g 4 v (X v ), A 2 = 4 2 v =v E X g 2 v (X v )g 2 v (X v ), A 3 = 4 3 v 1 =v 2 =v 3 E X g 2 v 1 (X v 1 )g v 2 (X v 2 )g v 3 (X v 3 ), A 4 = 4 3 v 1 =v 2 E X g 3 v 1 (X v 1 )g v 2 (X v 2 ), A 5 = 4 1 v 1 =v 2 =v 3 =v 4 E X g v 1 (X v 1 )g v 2 (X v 2 )g v 3 (X v 3 )g v 4 (X v 4 ).
Using the Cauchy Schwartz inequality and the fact that g v ∞ ≤ g v Hv ≤ 2, and andA 4 to d 3 n . For example,

g 2 = d n (because g ∈ G ), we get that A 1 is proportional to d 2 n , A 2 , A 3 , A 5 to d 4 n ,
A 1 = v∈P E X g 4 v (X v ) ≤ g 2 ∞ v∈P g v 2 2 = g 2 ∞ v∈P g v 2 2 ≤ 4d 2 n .
After calculation of the terms A i , since d 2 n is assumed to be smaller than one, we get that:

max k E X (g k (X)) 4 ≤ Cd 2 n (1 + O(d 2 n )). (3.77)
Step 2 is proved by combining (3.76) and (3.77).

We now focus on Step 3 and Step 4:

Proof of
Step 3: Let N pr be defined at Equation (3.71). We prove that

1 K log N pr d n 8 , G , . n ≤ 32σ α √ n(E ε sup g∈G |V n,ε (g)|)/d n α 1 (0,32σα √ nEε sup g∈G |Vn,ε(g)|] (d n )+ 1 [32σα √ nEε sup g∈G |Vn,ε(g)|,∞) (d n ).
We start from Equation (3.27) and write that:

log N pr d n 8 , G , • n ≤ log N d n 16 , G , • n .
Next, we use Corollary 3.4.1: Let Z = (Z 1 , ..., Z n ) be i.i.d. random variables distributed with density π α ∈ D defined in Equation (3.2) with var

(Z i ) = σ α . Set T = G , δ = √ nd n /16 and M = n × E Z sup g∈G |V n,Z (g)|, then for all α ≥ 2 we have, log N d n 16 , G , . n = log N √ nd n 16 , G , . , ≤K 32nE Z sup g∈G |V n,Z (g)| √ nd n α 1 (0,2n×E Z sup g∈G |V n,Z (g)|] ( √ nd n 16 )+ K 32nE Z sup g∈G |V n,Z (g)| √ nd n 2 1 [2n×E Z sup g∈G |V n,Z (g)|,∞) ( √ nd n 16 ),
or equivalently,

log N d n 16 , G , . n ≤K 32nE Z sup g∈G |V n,Z (g)| √ nd n α 1 (0,32 √ nE Z sup g∈G |V n,Z (g)|] (d n )+ K 32nE Z sup g∈G |V n,Z (g)| √ nd n 2 1 [32 √ nE Z sup g∈G |V n,Z (g)|,∞) (d n ). Take ε i = Z i /σ α = h(Z i ) for i = 1, ..., n, then var(ε i ) = 1 and, E ε (ε i ) = E ε (h(Z i )) = h(Z i )π α (Z i )dZ i = 1 σ α Z i π α (Z i )dZ i = 1 σ α E Z (Z i )
. in the regression model with non-Gaussian and non-bounded error

Therefore, E Z sup g∈G |V n,Z (g)| = σ α E ε sup g∈G |V n,ε (g)| and, log N d n 16 , G , . n ≤K 32nσ α E ε sup g∈G |V n,ε (g)| √ nd n α 1 (0,32σα √ nEε sup g∈G |Vn,ε(g)|] (d n )+ K 32nσ α E ε sup g∈G |V n,ε (g)| √ nd n 2 1 [32σα √ nEε sup g∈G |Vn,ε(g)|,∞) (d n ), ≤K(32σ α ) α n α 2 E ε sup g∈G |V n,ε (g)| d n α 1 (0,32σα √ nEε sup g∈G |Vn,ε(g)|] (d n )+ K 32σ α √ nE ε sup g∈G |V n,ε (g)| d n 2 1 [32σα √ nEε sup g∈G |Vn,ε(g)|,∞) (d n ), ≤K(32σ α ) α n α 2 E ε sup g∈G |V n,ε (g)| d n α 1 (0,32σα √ nEε sup g∈G |Vn,ε(g)|] (d n )+ K1 [32σα √ nEε sup g∈G |Vn,ε(g)|,∞) (d n ).

Proof of Step 4:

This Step consists in bounding from above the quantity E ε sup g∈G |V n,ε (g)|. According to Inequality (3.63) we have,

v∈P |V n,ε (g v )| ≤ κ v∈P λ 2 n,v g v Hv + v∈P λ n,v g v n ,
with λ n,v defined by Equation (3.11) satisfying Equation (3.12) for all v ∈ P. It follows

sup g∈G v∈P |V n,ε (g v )| ≤ κ sup g∈G v∈P λ 2 n,v g v Hv + v∈P λ n,v g v n , ≤ κ C 1 sup g∈G v∈P µ v g v Hv + v∈P γ v g v n .
Thanks to Condition C1 and using g v Hv ≤ 2 we obtain then:

sup g∈G v∈P |V n,ε (g v )| ≤ 4κ C 1 sup g∈G v∈S f µ v g v Hv + sup g∈G v∈S f γ v g v n , ≤ 4κ C 1 2 v∈S f µ v + sup g∈G v∈S f γ v g v n .
Now, according to Condition C2, we get

sup g∈G v∈P |V n,ε (g v )| ≤ 4κ C 1 2 v∈S f µ v + 2 sup g∈G v∈S f γ v g v 2 + v∈S f γ 2 v , ≤ 4κ C 1 v∈S f (2µ v + γ 2 v ) + 2 sup g∈G ( v∈S f γ 2 v ) 1/2 ( v∈S f g v 2 2 ) 1/2 , ≤ 4κ C 1 v∈S f (2µ v + γ 2 v ) + 2( v∈S f γ 2 v ) 1/2 d n ,
where in the second inequality we used Cauchy Schwarz inequality and the third inequality coming from the fact that for all g ∈ G , The kernel k v is written as :

g 2 2 = d 2 n ≥ v∈S f g v 2
k v (X v , X v ) = ≥1 ω v, φ v, (X v )φ v, (X v )
where {φ v, } ∞ =1 is an orthonormal basis of L 2 (P v ) with P v = a∈v P a . Let us consider the class of functions K(t) defined as

K(t) = {g v ∈ H v , g v Hv ≤ 2, g v 2 ≤ t} .

It comes that

g v = a φ v, , with g v 2 Hv = a 2 ω v, ≤ 4, and g v 2 2 = a 2 ≤ t 2
In the following, we set µ v, (t) = min t 2 , ω v, . Hence

a 2 µ v, (t) ≤ 1 t 2 a 2 + a 2 ω v, = 1 t 2 g v 2 2 + g v 2 Hv ≤ 5, (3.78)
as soon as g v ∈ K(t). Now, let us prove the lemma:

E X,ε W n,2,v (t) = E X,ε sup g∈K(t) | 1 n n i=1 ε i a φ v, (X vi )|, = E X,ε sup g∈K(t) | 1 n a µ v, (t) n i=1 ε i µ v, (t)φ v, (X vi )|, ≤ √ 5 E X,ε 1 n n i=1 ε i µ v, (t)φ v, (X vi ) 2 .
The last inequality follows from the Cauchy-Schwartz inequality and Inequality (3.78). Now, simple calculation leads to

E X,ε W n,2,v (t) ≤ √ 5 1 n µ v, (t). 2 3.5.3.2 Proof of Lemma 3.5.6 Using that | √ a - √ b| ≤ |a -b|, we get | g v 2 -g v n | ≤ g v 2 2 -g v 2 n
. in the regression model with non-Gaussian and non-bounded error Hence

g v ∞ ≤ b, | g v 2 -g v n | ≥ bt 2 ⊂ g v 2 2 -g v 2 n ≥ b 2 t 2 4 .
The centered process

g v 2 2 -g v 2 n = | 1 n n i=1 g 2 v (X v,i ) -E(g 2 v (X v ))|,
satisfies a concentration inequality given, for example, by Theorem 2.1 in [START_REF] Bartlett | Local rademacher complexities[END_REF] : if C is a class of functions f such that f ∞ ≤ B and Ef (X) = 0, and if there exists γ > 0 such that for every f ∈ C, Varf (X) ≤ γ 2 . Then for every x > 0, with probability at least 1 -e -x ,

sup f ∈C 1 n | n j=1 f (X j )| ≤ inf α>0 2(1 + α)E(sup f ∈C 1 n | n j=1 f (X j )|) + 2x n γ + B 1 3 + 1 α x n .
(3.79) For any t > 0, for G(t) defined by (3.57), let us consider the class of functions C(t) defined as follows

C(t) = f such that f = g 2 v -E(g 2 v ), with g v ∈ G(t) . Note that if f ∈ C(t), E X f (X v ) = 0 and f ∞ ≤ b 2 .
We have to study

γ 2 (t) = sup gv∈G(t) E X g 2 v (X) -g v 2 2 ) 2 and Γ(t) = E X sup gv∈G(t) g v 2 n -g v 2 2
.

It is easy to see that

γ 2 (t) ≤ b 2 sup gv∈G(t) E X (g v (X) + g v 2 ) 2 ≤ 4b 2 t 2 .
Let ζ i be i.i.d. Rademacher random variables and let E X,ζ denotes the expectation with respect to the law of (X, ζ). By a symmetrization argument,

Γ(t) ≤ 2E X,ζ sup gv∈G(t) | 1 n n i=1 ζ i g 2 v (X i )|.
Since g v ∞ ≤ b, applying the contraction principal (see [START_REF] Ledoux | Probability in Banach Spaces: isoperimetry and processes[END_REF]) we get that, for Q n,v (t) defined by (3.9),

E X,ζ sup gv∈G(t) | 1 n n i=1 ζ i g 2 v (X i )| ≤ 4bE X,ζ sup gv∈G(t) | 1 n n i=1 ζ i g v (X i )| ≤ 4bQ n,v (t).
The last inequality was proved by [START_REF] Mendelson | Geometric parameters of kernel machines[END_REF], Theorem 41 (see the proof of Lemma 3.5.5). Now, thanks to (3.79) we get that for all x > 0, with probability greater than 1 -e -x sup gv∈G(t)

| g v 2 n -g v 2 2 | ≤ inf α>0 16(1 + α)bQ n,v (t) + 2x n 2bt + b 2 1 3 + 1 α x n .
Taking x = c 2 nt 2 , t ≥ ν n , we have that with probability greater than 1 -e -c 2 nt 2 sup gv∈G(t)

g v 2 n -g v 2 2 ≤ inf α>0 t 2 16(1 + α)b∆ + √ 2c 2 4b + b 2 1 3 + 1 α c 2 .
The infimum of the right hand side is reached in α = c 2 b/16∆, and equals

b 2 c 2 3 + 8 ∆c 2 b 3/2 + 4(4∆ + √ 2c 2 )b.
The constants ∆ and c 2 should satisfy that this infimum is strictly smaller than b 2 /4. For example, if 16∆ < b/8, it remains to choose c 2 small enough such that

b c 2 3 + √ 2c 2 2 + 4 √ 2c 2 < b 8 . 2 3.5.3.3 Proof of Lemma 3.5.7
Let t > ν n,v and h be defined as

h = tg v g v 2 .
If g v satisfies the assumptions of the lemma, then h satisfies h 2 = t, h H ≤ 2 and h ∞ ≤ b. Applying Lemma 3.5.6 (page 88) to the function h, we obtain that for all t ≥ ν n,v , with probability greater than 1 -exp(-c 2 nt 2 ), we have

|t -h n | ≤ bt 2 for all h ∈ G(t).
This concludes the proof of the lemma. 2 3.5.3.4 Proof of Lemma 3.5.8

We apply Corollary 3.4.2 to

φ(ε 1 , . . . , ε n ) = √ n t W n,n,v (t).
Using Cauchy-Schwarz Inequality and the fact that

g v n ≤ t, |φ(ε) -φ(ε )| ≤ √ n t sup gv n≤t g v n ε -ε n ≤ √ n t t ε -ε n , leading to φ L = 1. So, P X,ε | √ n t W n,n,v (t) - √ n t E ε W n,n,v (t)| ≥ u ≤ 2B exp - u 2 8A ,
and Lemma 3.5.8 is proved by taking δ = u/ √ n.

104

Chapter 3. Risk upper bounds for RKHS ridge group sparse estimator in the regression model with non-Gaussian and non-bounded error 3.5.3.5 Proof of Lemma 3.5.9

We start with the proof of (3.60) in Lemma 3.5.9 by applying once again Corollary 3.4.2, to the function

φ(ε) = φ(ε 1 , . . . , ε n ) = √ n 2t W n,2,v (t).
On the event Ω v,t defined by (3.58), we have

g v n ≤ bt 2 + g v 2 .
Besides if

g v Hv ≤ 2, then g v ∞ ≤ 2. Therefore applying Lemma 3.5.6 with b = 2, we get that if g v 2 ≤ t, |φ(ε) -φ(ε )| ≤ √ n 2t sup gv n≤2t g v n ε -ε n ≤ √ n 2t 2t ε -ε n , leading to φ L = 1. So, P X,ε | √ n 2t W n,2,v (t) - √ n 2t E ε (W n,2,v (t))| ≥ u ∩ Ω c v,t ≤ 2B exp - u 2 8A ,
and inequality (3.60) in Lemma 3.5.9 is proved by taking δ = 2u/ √ n.

We now come to the proof of the inequality (3.61) in Lemma 3.5.9 using a Poissonian inequality for self-bounded processes (see [START_REF] Boucheron | A sharp concentration inequality with applications[END_REF]) and Theorem 5.6, p 158 in [START_REF] Massart | Concentration Inequalities and Model Selection: Ecole d'Eté de Probabilités de Saint-Flour XXXIII -2003[END_REF]). Let us recall it in the particular case we are interested in:

Theorem 3.5.1 Let X 1 , • • • , X n be n i.i.d. random variables. For i ∈ {1, • • • , n} let X (-i) = (X 1 , . . . , X i-1 , X i+1 , . . . , X n ).
Let h be a non-negative and bounded measurable function of

X = (X 1 , • • • , X n ).
Assume that for all i ∈ {1, • • • , n}, there exists a measurable function

h i of X (-i) such that 0 < h -h i ≤ 1, and n i=1 (h -h i ) ≤ h.
Then, for all x > 0, we have

P h ≥ E(h) + x ≤ exp - x 2 2E(h)
.

We apply this result to h defined as

h = h(X 1 , • • • , X n ) = nE ε W n,2,v (t) = nE ε sup |V n,ε (g v )|, g v 2 ≤ t, g v Hv ≤ 2 .
The variable h is positive, and because the distribution of (ε 1 , . . . , ε n ) is symmetric, we have that

h = E ε sup nV n,ε (g v ), g v 2 ≤ t, g v Hv ≤ 2 .
Let τ be the function in H v such that h = E ε nV n,ε (τ ) (note that τ depends on (X 1 , . . . , X n ) and on (ε 1 , . . . , ε n )), and let

h i = E ε sup gv j =i ε j g v (X j ).
We show that h and h i satisfy the assumptions of Theorem 3.5.1:

h -h i = E ε ε i τ (X i ) + j =i ε j τ (X j ) -sup gv j =i ε j g v (X j ) , ≤ E ε ε i τ (X i ) , ≤ E ε |ε i | sup x∈X |τ (X)| , ≤ 2E ε |ε i | ,
where the last inequality comes from the fact that

sup x∈X |τ (X)| ≤ τ Hv ≤ 2. Let Z = (Z 1 , ..., Z n ) be i.i.d. random variables distributed with density π α ∈ D defined in Equation (3.2) with var(Z i ) = σ α . Take ε i = Z i /σ α for i = 1, ..., n, then var(ε i ) = 1 and E ε (|ε i |) = E Z (|Z i |)/σ α . We have: E Z (|Z i |) = R |Z i |a α exp -|Z i | α dZ i . Take |Z i | = u 1/α and dZ i = 1 α u 1 α -1 du if Z i ≥ 0, -1 α u 1 α -1 du if Z i ≤ 0.
Therefore,

E Z (|Z i |) = +∞ 0 a α u 1 α exp(-u) 1 α u 1 α -1 du - 0 +∞ a α u 1 α exp(-u) 1 α u 1 α -1 du, = 2 +∞ 0 a α u 1 α exp(-u) 1 α u 1 α -1 du, = a α +∞ 0 2 α u 2 α -1 exp(-u)du, = a α 2 α Γ( 2 α ) = a α Γ(1 + 2 α ),
where Γ(.) is the gamma function.

It follows that,

h -h i ≤ 2a α σ α Γ(1 + 2 α ). Moreover, h -h i ≥ 0 since h = E ε sup gv n j=1 ε j g v (X j ) = E ε E ε i sup gv n j=1 ε j g v (X j ) ≥ E ε sup gv E ε i n j=1 ε j g v (X j ) = h i .

in the regression model with non-Gaussian and non-bounded error

Finally we have:

i (h-h i ) = n i=1 E ε ε i τ (X i )+ n j =i ε j τ (X j )-sup g-v n j =i ε j g v (X j ) ≤ n i=1 E ε ε i τ (X i ) = h.
Therefore, following Theorem 3.5.1, we get that for all postive u

P X,ε E ε W n,2,v (t) -E X,ε W n,2,v (t) ≤ u n ≤ exp - u 2 E X,ε W n,2,v (t)
.

As E X,ε W n,2,v (t) ≤ Q n,v (t)
, see Lemma 3.5.5 page 88, we get the expected result since for all positive x From Lemma 3.5.8, page 89 with t = λ n,v = δ, with probability greater than

P X E ε W n,2,v (t) ≥ E X,ε W n,2,v (t) + x ≤ exp - nx 2 Q n,v (t) .
1 -2B exp(-nλ 2 n,v /8A
), we get that:

E ε (W n,n,v (λ n,v )) ≤ λ 2 n,v + W n,n,v (λ n,v ) (3.80) The next step consists in comparing W n,n,v (λ n,v ) and W n,2,v (2λ n,v ). Recall that λ n,v ≥ ν n,v , see (3.11). Let g v such that g v n ≤ λ n,v .
When g v 2 ≤ λ n,v , according to Lemma 3.5.6 (page 88), taking b = 2 , since since g v n ≤ λ n,v , we get that with probability greater than 1exp(-c 2 nλ 2 n,v ),

g v n -λ n,v ≤ g v 2 ≤ g v n + λ n,v ≤ 2λ n,v .
When g v 2 ≥ t, we apply Lemma 3.5.7 (page 88) with b = 2. For any function

g v such that g v ∞ ≤ 2, and g v 2 ≥ λ n,v , we have g v 2 ≤ 2 g v n ≤ 2λ n,v .
This implies that, with probability greater than 1 -exp(-c 2 nλ 2 n,v ) we have

W n,n,v (λ n,v ) ≤ W n,2,v (2λ n,v ).
We now study the process W n,2,v (λ n,v ). By applying (3.60) in Lemma 3.5.9, page 89, with δ = t = λ n,v we get that with probability greater than

1-2B exp(-nλ 2 n,v /32A) W n,2,v (λ n,v ) ≤ λ 2 n,v + E ε (W n,2,v (λ n,v )). It follows that E ε W n,n,v (λ n,v ) ≤ λ 2 n,v + W n,n,v (λ n,v )), ≤ λ 2 n,v + W n,2,v (2λ n,v )), ≤ 5λ 2 n,v + E ε (W n,2,v (2λ n,v )).

Proof of Corollary 3.3.1 107

Next, we apply (3.61) in Lemma 3.5.9, with t = 2λ n,v and x = 4λ 2 n,v . We get that

E ε W n,2,v (2λ n,v ) ≤ 4λ 2 n,v + E X,ε (W n,2,v (2λ n,v
)), with probability greater than

1 -2 exp(-16 nλ 4 n,v Q n,v (2λ n,v ) ) ≥ 1 -2 exp(- 4nλ 2 n,v ∆ ).
The last inequality comes from the definition of ν n,v , see (3.10), and from the fact that λ n,v ≥ ν n,v , see (3.11).

Putting everything together, we get that with probability greater than 1 -

c 1 exp(-c 2 nλ 2 n,v ) for some positive constants c 1 , c 2 , E ε W n,n,v (λ n,v ) ≤ 9λ 2 n,v + E X,ε (W n,2,v (2λ n,v )), ≤ 9λ 2 n,v + Q n,v (2λ n,v
), thanks to Lemma 3.5.5, page 88, ≤ 9λ 2 n,v + 4∆λ 2 n,v . Applying once again Lemma 3.5.8, page 89, we get that

W n,n,v (λ n,v ) ≤ E ε W n,n,v (λ n,v ) + λ 2 n,v ≤ 10 + 4∆ λ 2 n,v .
This ends the proof of the lemma by taking κ = 10 + 4∆. 2

3.6 Proof of Corollary 3.3.1

According to Theorem 3.3.1 we have with high probability,

f -m 2 n ≤ C inf f ∈F m -f 2 n + v∈S f (µ v + γ 2 v ) . (3.81)
Besides, for all K > 0,

f -m 2 2 ≤ (1 + K) f -f 2 2 + (1 + 1 K ) m -f 2 2 .
(3.82)

We consider once again two cases defined in page 87.

Case 1:

f -f 2 ≤ f -f n , In this case Equation (3.82) gives, f -m 2 2 ≤ (1 + K) f -f 2 n + (1 + 1 K ) m -f 2 2 .
Then, using Equations (3.55) and (3.81) we obtain the result. Case 2: ff 2 ≥ ff n , Apply Lemma 3.5.4 (page 87) and conclude that conditioning on the events T and A, defined by (3.47) and (3.52), then ff belongs to G(f ) defined in Lemma 3.5.4. Now, conditioning on the event C we get the result as in Case 1 since, From Lemma 3.4.1 we have U α(u) ⊂ (u 1/2 B 2 + u 1/ αB α). It suffices to show that

f -f 2 ≤ √ 2 f -f n .
(u 1/2 B 2 + u 1/ αB α) ⊂ 2 × max(u 1/2 , u 1/ α)B 2 . Consider x ∈ u 1/2 B 2 + u 1/ αB α, x = y + z with y ∈ u 1/2 B 2 , means n i=1 y 2 i ≤ u, and z ∈ u 1/ αB α, means n i=1 z α i ≤ u. Moreover, we know that x ≤ y + z which leads to x ≤ u 1/2 + u 1/ α and x 2 ≤ 2(u + u 2/ α) ≤ 4 × max(u, u 2/ α). 2 3.A.2 Proof of Corollary 3.4.1 From Equation (3.35) we have N (2 × max(M 1/2 , M 1/ α), T, . ) ≤ exp(KM ).
Using this on sT for s > 0 we have sM = E Z sup t ∈sT n i=1 t i Z i and,

N (2 × max((sM ) 1/2 , (sM ) 1/ α), sT, . ) ≤ exp(KsM ).
Moreover,

N (2 × max((sM ) 1/2 , (sM ) 1/ α), sT, . ) = N ( 2 s × max((sM ) 1/2 , (sM ) 1/ α), T, . ),
since for all t 1 , t 2 ∈ T and some constant C, st 1 -st 2 ≤ C is equivalent to t 1 -t 2 ≤ C/s. We obtain then,

N ( 2 s × max((sM ) 1/2 , (sM ) 1/ α), T, . ) ≤ exp(KsM ).
As in Remark 3.4.2 for u = sM we consider two following cases (recall that 1 < α <

2):

(i) If sM ≤ 1 we have (sM ) 1/ α ≤ (sM ) 1/2 and so, N (2( M s ) 1/2 , T, . ) ≤ exp(KsM ).
Take δ = 2(M/s) 1/2 and thus s = 4M/δ 2 . Moreover, sM ≤ 1 (i.e. (4M/δ 2 ) × M ≤ 1) and so δ ≥ 2M . Finally, we obtain in this case:

∀δ ≥ 2M, log N (δ, T, . ) ≤ K( 2M δ ) 2 .
(ii) If sM ≥ 1 we have (sM ) 1/2 ≤ (sM ) 1/ α and so, N ( 2 s (sM ) 1/ α, T, . ) ≤ exp(KsM ).

3.B. Proofs of Section 3.4.3 109

Take δ = (2/s)(sM ) 1/ α and thus s = (2/δ) α/( α-1) M 1/( α-1) . Moreover, sM ≥ 1 (i.e. (2M/δ) α/( α-1) ≥ 1) and so 0 < δ ≤ 2M . Finally, we obtain in this case:

∀0 < δ ≤ 2M, log N (δ, T, . ) ≤ K( 2M δ ) α/( α-1) = K( 2M δ ) α . 2
3.B Proofs of Section 3.4.3

3.B.1 Proof of Lemma 3.4.2
In order to prove this Lemma it suffices to show that Π α ∈ M(m, ρ 2 ) for some m.

To do so, we use Example 3.4.1.

First show d log Π α ([t, ∞))/dt ≤ -t/ρ 2 :
We know that

d dt log Π α ([t, ∞)) = d dt log(1 -Π α ((-∞, t])) = - π α (t) 1 -Π α ((-∞, t]) = - π α (t) Π α ([t, ∞))
.

For all t > 0 we have,

Π α ([t, ∞)) = ∞ t a α exp(-|x| α )dx = ∞ t a α exp(-x α )dx.
Take x = u 1/α , so dx = (1/α)u (1/α)-1 du, and

Π α ([t, ∞)) = ∞ t α a α α u (1/α)-1 exp(-u)du, = a α α Γ( 1 α , t α ),
where

Γ( 1 α , t α ) is incomplete gamma function. Moreover, for s ∈ R as x → ∞, Γ(s, x) x s-1 exp(-x) → 1.
Therefore,

Π α ([t, ∞)) = a α α t 1-α exp(-t α ).
Since t > 0 so π α (t) = a α exp(-t α ), and

d dt log Π α ([t, ∞)) = - αa α exp(-t α ) a α t 1-α exp(-t α ) = -αt α-1 .
The inequality -αt α-1 ≤ -t/ρ 2 (i.e. t α-2 ≥ 1/αρ 2 ) holds for all α > 2 and t ≥ (1/αρ 2 ) 1/(α-2) . Second show d log Π α ((-∞, -t])/dt ≤ -t/ρ 2 : in the regression model with non-Gaussian and non-bounded error

The probability distribution Π α is symmetric, therefore Π α ((-∞, -t]) = Π α ([t, ∞)), and 2) . Take m = (1/αρ 2 ) 1/(α-2) , then for x ≥ m, Π α verifies the Equations (3.39). That is

d dt log Π α ((-∞, -t]) = d dt log Π α ([t, ∞)) = -αt α-1 , which is smaller than -t/ρ 2 if α > 2 and t ≥ (1/αρ 2 ) 1/(α-
Π α ∈ M((1/αρ 2 ) 1/(α-2) , ρ 2 ).
E(exp(s|Z|)) = 2a α √ π exp s 2 4 . (3.83) If α > 2 we have, E(exp(s|Z|)) = +∞ -∞ exp(s|z|)a α exp(-|z| α )dz = 2a α S,
where

S = +∞ 0 exp(sz -z α )dz = 1 0 exp(sz -z α )dz S 1 + +∞ 1 exp(sz -z α )dz S 2
.

For z ∈ [0, 1] we have exp(-z α ) ≤ 1 and so

S 1 ≤ 1 0 exp(sz)dz = exp(s) -1 s .
For z ≥ 1 we have exp(z 2 -z α ) < 1 and so

S 2 = +∞ 1 exp(sz -z 2 + z 2 -z α )dz < +∞ 1 exp(sz -z 2 )dz < √ π exp s 2 4 ,
where the last inequality is obtained using Equation (3.83). Finally, we obtain

S < exp(s) -1 s + √ π exp s 2 4 ,
and therefore

E(exp(s|Z|)) < 2a α exp(s) -1 s + √ π exp s 2 4 .

3.B.3 Proof of Corollary 3.4.2

We suppose that the inequality (3.41) holds and we want to find an upper bound for P |φ(Z) -E(φ(Z))| ≥ u . Using the Markov's inequality we have,

P |φ(Z) -E(φ(Z))| > u = P exp(λ|φ(Z) -E(φ(Z))| 2 ) > exp(λu 2 ) , ≤ exp(-λu 2 )E exp(λ|φ(Z) -E(φ(Z))| 2 ) . (3.84)
To demonstrate the result of the Theorem, it suffices to find an upper bound for the following quantity

E exp(λ|φ(Z) -E(φ(Z))| 2 ) .
Let Z 1 and Z 2 be two independent random variables distributed with the same law, then for all u > 0 we have:

P |Z 1 -Z 2 | > u ≤ P |Z 1 -M (φ(Z 1 ))| > u 2 +P |Z 2 -M (φ(Z 2 ))| > u 2 . (3.85)
Furthermore, for all convex function ψ we have:

E ψ(Z 1 -E(Z 1 )) = ψ (z 1 -z 2 )dP (z 2 ) dP (z 1 ).
Applying the Jensen's inequality we obtain then,

E ψ(Z 1 -E(Z 1 )) ≤ ψ(z 1 -z 2 )dP (z 2 ) dP (z 1 ), ≤ E ψ(Z 1 -Z 2 ) . (3.86)
Set ψ(t) = exp(λt 2 ) for λ > 0, Z 1 = φ(Z) and Z 2 = φ(Z ). Since ψ(t) is convex, then Equation (3.86) gives:

E exp(λ|φ(Z) -E(φ(Z))| 2 ) ≤ E exp(λ(φ(Z) -φ(Z )) 2 ) .
(3.87)

For all non-negative random variables Z we have E(Z) = [0,∞) P (Z ≥ z)dz. So, we obtain from Equation (3.87):

E exp(λ|φ(Z) -E(φ(Z))| 2 ) ≤ ∞ 0 P exp(λ(φ(Z) -φ(Z )) 2 ) > t dt.
Using Equation (3.85) and simple calculations leads to:

E exp(λ|φ(Z) -E(φ(Z))| 2 ) ≤ ∞ 1 P |φ(Z) -φ(Z )| > log(t) λ dt, ≤ 2 ∞ 1 P |φ(Z) -M (φ(Z))| > 1 2 log(t) λ dt.
(3.88)

Chapter 3. Risk upper bounds for RKHS ridge group sparse estimator in the regression model with non-Gaussian and non-bounded error

In this step we can use the result in Equation (3.41), from which we obtain: (3.89) and, therefore,

E exp(λ|φ(Z) -E(φ(Z))| 2 ) ≤ 2B ∞ 1 exp - log(t) 4λA dt,
E exp(λ|φ(Z) -E(φ(Z))| 2 ) ≤ 8λAB 1 -4λA , ∀λ < 1 4A . (3.90)
The proof is complete by taking λ = 1/8A.

Chapter 4

Estimate the Hoeffding decomposition of a complex model by solving RKHS ridge group sparse optimization problem

Abstract

We propose an R package, called RKHSMetaMod, that implements a procedure for estimating a meta-model of a complex model m. The meta-model approximates the Hoeffding decomposition of m and allows to perform sensitivity analysis on it. It belongs to a reproducing kernel Hilbert space that is constructed as a direct sum of Hilbert spaces. The estimator of the meta-model is the solution of a penalized least-squares minimization with the sum of the Hilbert norm and the empirical L 2norm. This procedure, called RKHS ridge group sparse, allows both to select and estimate the terms in the Hoeffding decomposition, and therefore, to select and estimate the Sobol indices that are non-zero. This package provides an interface from R statistical computing environment to the C++ libraries Eigen and GSL.

In order to speed up the execution time and optimize the storage memory, except for a function that is written in R, all of the functions of RKHSMetaMod package are written using the efficient C++ libraries through RcppEigen and RcppGSL packages. These functions are then interfaced in the R environment in order to propose an user friendly package.

Keywords: meta-model, Hoeffding decomposition, ridge group sparse, reproducing kernel Hilbert space, Sobol indices.

Introduction

Let us consider a phenomenon described by a model m depending on d input variables X = (X 1 , ..., X d ). This model m from R d to R may be a known model that can be calculated in all points of X, or it may be an unknown regression model defined as follows:

Y = m(X) + σε, σ > 0, (4.1)
where the error ε is assumed to be centered with a finite variance, i.e. E(ε) = 0 and var(ε) < ∞. by solving RKHS ridge group sparse optimization problem

The components of X are independent and have a known law P X = d a=1 P Xa on X , a subset of R d . The number d of components of X may be large. The model m may present high complexity as strong non-linearities and high order interaction effects, and it is assumed to be square-integrable, i.e. m ∈ L 2 (X , P X ).

Based on n data points {(X i , Y i )} n i=1 , a meta-model that approximates the Hoeffding decomposition of m is estimated. This meta-model belongs to a reproducing kernel Hilbert space (RKHS), which is constructed as a direct sum of Hilbert spaces leading to an additive decomposition including the variables and interactions between them [START_REF] Durrande | Anova kernels and rkhs of zero mean functions for model-based sensitivity analysis[END_REF]). The estimator of the meta-model is calculated by minimizing a least-squares criterion penalized by the sum of two penalty terms: the Hilbert norm and the empirical norm (Huet and Taupin (2017)). This procedure allows to select the subsets of variables X that contribute to predict Y . The estimated meta-model is used to perform sensitivity analysis, and so allows to determine the influence of each variable and groups of them on the output variable Y .

In the classical framework of sensitivity analysis m(X) is calculable in all points of X. In this framework, one may use the method of [START_REF] Sobol | Sensitivity estimates for nonlinear mathematical models[END_REF] for variance-based methods of global sensitivity analysis in order to perform sensitivity analysis on m. Let us briefly recall this method.

Let P be the set of all subsets of {1, ..., d} with dimension 1 to d. For all X ∈ X and v ∈ P, let X v be the vector with components X a for all a ∈ v. For a set A let |A| be its cardinality, and for all v ∈ P, let m v : R |v| → R be a function of X v .

The independency between the components of X allows to write the function m according to its Hoeffding decomposition [START_REF] Sobol | Sensitivity estimates for nonlinear mathematical models[END_REF], van der Vaart (1998)):

m(X) = m 0 + v∈P m v (X v ), (4.2)
where m 0 is known as constant term, when |v| = 1 the functions m v are known as main effects, when |v| = 2, i.e. v = {a, a } and a = a , the functions m v are known as second order interactions, and so on. This decomposition (4.2) is unique, all the terms m v , v ∈ P are centered, and they are orthogonal with respect to L 2 (X , P X ). The function m as well as all the functions m v in Equation (4.2) are square-integrable. As any two terms of decomposition (4.2) are orthogonal, by squaring (4.2) and integrating it with respect to the distribution of X, a decomposition of the variance of m(X) is obtained as follows:

var(m(X)) = v∈P var(m v (X v )).
(4.3)

For any group of variables X v , v ∈ P, the Sobol indices are defined by:

S v = var(m v (X v )) var(m(X)) . (4.4)
For each v, S v expresses the fraction of variance of m(X) explained by X v .

For all v ∈ P, when |v| = 1, the S v 's are referred to as the first order indices. When |v| = 2, i.e. v = {a, a } and a = a , they are referred to as the second order indices or the interaction indices of order two (between X a and X a ). And the same holds for |v| > 2.

The total number of the Sobol indices to be calculated is equal to |P| = 2 d -1, which raises exponentially with the number d of the input variables. When d is large, the evaluation of all the indices can be too computationally demanding and even not reachable. For this reason, only the indices of order not higher than two are calculated in practice. However, only first and second order indices may not provide a good information on the model sensitivities. In order to provide a better information on the model sensitivities, [START_REF] Homma | Importance measures in global sensitivity analysis of nonlinear models[END_REF] proposed to calculate the first order and the total indices defined as follows:

Let P a ⊂ P be the set of all the subsets of {1, ..., d} including a, then

S Ta = v∈Pa S v .
For all a ∈ {1, ..., d}, S Ta denotes the total effect of X a . It expresses the fraction of variance of m(X) explained by X a alone and all the interactions of it with the other variables.

The total indices allow to rank the input variables with respect to the amount of their effect on the output variable. However, they do not provide complete information on the model sensitivities as do all the Sobol indices.

The classical computation of the Sobol indices is based on the Monte Carlo methods (see for example: [START_REF] Sobol | Sensitivity estimates for nonlinear mathematical models[END_REF] for the main effect and interaction indices, and [START_REF] Saltelli | Making best use of model evaluations to compute sensitivity indices[END_REF] for the main effect and total indices). For models that are expensive to evaluate, the Monte Carlo methods lead to high computational burden. Moreover, in the case where d is large, m is complex and the calculation of the variances (see Equation (4.3)) is numerically complicated or not possible, as in the case where the model m is unknown, the methods described above are not applicable.

Another method is to approximate m by a simplified model, called a metamodel, which is much faster to evaluate and to perform sensitivity analysis on it. A meta-model provides additional information than just scalar indices. It provides the approximations of the Sobol indices of m at a lower computational cost, and also a deeper view of the input variable's effects on the model output.

Among the meta-modelling methods proposed in the literature, the expansion based on polynomial Chaos [START_REF] Wiener | The homogeneous chaos[END_REF], [START_REF] Schoutens | Stochastic Processes and Orthogonal Polynomials, Lecture Notes in Statistics[END_REF]) can be used to approximate the Hoeffding decomposition of m [START_REF] Sudret | Global sensitivity analysis using polynomial chaos expansions[END_REF]).

The principle of the polynomial Chaos is to project m onto a basis of orthonormal polynomials. The polynomial Chaos expansion of m is written as [START_REF] Soize | Physical systems with random uncertainties: Chaos representations with arbitrary probability measure[END_REF]):

m(X) = ∞ j=0 h j φ j (X),
(4.5) by solving RKHS ridge group sparse optimization problem where {h j } ∞ j=0 are the coefficients, and {φ j } ∞ j=0 are multivariate orthonormal polynomials associated with X that are determined according to the distribution of the components of X. In practice, expansion (4.5) shall be truncated for computational purposes, and the model m may be approximated by:

m(X) ≈ vmax j=0 h j φ j (X),
where v max is determined using a truncation scheme. In this approach, the Sobol indices are obtained by summing up the squares of the suitable coefficients. [START_REF] Blatman | Adaptive sparse polynomial chaos expansion based on least angle regression[END_REF] proposed a method for truncating the polynomial Chaos expansion and an algorithm based on least angle regression for selecting the terms in the expansion.

In this method, according to the distribution of the components of X a unique family of orthonormal polynomials {φ j } ∞ j=0 is determined. However, this family may not be necessarily the best functional basis to approximate m well.

Another method to construct meta-models is the Gaussian Process (GP) modelling [START_REF] Welch | Screening, predicting, and computer experiments[END_REF], [START_REF] Oakley | Probabilistic sensitivity analysis of complex models: a bayesian approach[END_REF], [START_REF] Kleijnen | Design and Analysis of Simulation Experiments[END_REF][START_REF] Kleijnen | Kriging metamodeling in simulation: A review[END_REF], [START_REF] Marrel | Calculations of sobol indices for the gaussian process metamodel[END_REF], [START_REF] Durrande | Additive covariance kernels for high-dimensional gaussian process modeling[END_REF], [START_REF] Gratiet | A bayesian approach for global sensitivity analysis of (multifidelity) computer codes[END_REF]). The principle is to consider that the prior knowledge about the function m(X), can be modelled by a GP Z(X) with a mean m Z (X) and a covariance kernel k Z (X, X ). To perform sensitivity analysis from a GP model one may replace the true model m(X) with the mean of the conditional GP, and deduce the Sobol indices from it.

A review on the meta-modelling based on polynomial Chaos and GP is presented in Le [START_REF] Gratiet | Metamodel-Based Sensitivity Analysis: Polynomial Chaos Expansions and Gaussian Processes[END_REF]. [START_REF] Durrande | Anova kernels and rkhs of zero mean functions for model-based sensitivity analysis[END_REF] considered a class of functional approximation methods similar to the GP and obtained a meta-model that satisfies the properties of the Hoeffding decomposition. They proposed to approximate m by functions belonging to a RKHS H which is constructed as a direct sum of Hilbert spaces such that the projection of m onto H, denoted f * , is an approximation of the Hoeffding decomposition of m.

The function f * is defined as the minimizer over the functions f ∈ H of the following criterion,

E X (m(X) -f (X)) 2 .
Let ., . H be the inner product in H, let also k and k v be the reproducing kernels associated with the RKHS H and the RKHS H v , respectively. The properties of the RKHS H insures that any function f ∈ H, f : X ⊂ R d → R is written as the following decomposition:

f (X) = f, k(X, .) H = f 0 + v∈P f v (X v ), (4.6)
where f 0 is a constant, and f v : R |v| → R is defined by,

f v (X) = f, k v (X, .) H .
For all v ∈ P, the functions f v (X v ) are centered and for all v = v , the functions f v (X v ) and f v (X v ) are orthogonal with respect to L 2 (X , P X ). So the decomposition of the function f presented in Equation (4.6) is its Hoeffding decomposition.

As the function f * belongs to the RKHS H, it is decomposed as its Hoeffding decomposition:

f * = f * 0 + v∈P f * v , (4.7)
and each function f * v approximates the function m v in Equation (4.2). In the decomposition (4.7), we have |P| terms f * v to be estimated. The cardinality of P is equal to 2 d -1 which may be huge since it raises very quickly by increasing d. In order to deal with this problem, in the regression framework, one may estimate f * by a sparse meta-model f ∈ H. To this purpose, the estimation of f * is done on the basis of n observations by minimizing a least-squares criterion suitably penalized in order to deal both with the non-parametric nature of the problem, and with the possibly large number of functions that have to be estimated.

Note that, in the classical framework of sensitivity analysis, where m(X) is calculable in all points X, one may calculate a sparse approximation of f * using leastsquares penalized criterion as it is done in the non-parametric regression framework.

In order to obtain a sparse solution of a minimization problem, the penalty function should enforce the sparsity. There exists various ways of enforcing sparsity for a minimization (maximization) problem, see for example [START_REF] Hastie | Statistical Learning with Sparsity: The Lasso and Generalizations[END_REF] for a review. Some methods, such as the Sparse Additive Models (SpAM) procedure [START_REF] Ravikumar | Sparse additive models[END_REF], [START_REF] Liu | Nonparametric regression and classification with joint sparsity constraints[END_REF]) are based on a combination of the l 1 -norm with the empirical L 2 -norm,

f n,1 = d a=1 f a n , where f a 2 n = 1 n n i=1 f 2 a (X ai ),
is the squared empirical L 2 -norm of the univariate function f a . The Component Selection and Smoothing Operator (COSSO) method developed by [START_REF] Lin | Component selection and smoothing in multivariate nonparametric regression[END_REF] enforces sparsity using a combination of the l 1 -norm with the Hilbert norm,

f H,1 = d a=1 f a Ha .
Instead of focusing on only one penalty term, one may consider a more general family of estimators, called doubly penalized estimator, that is obtained by minimizing a criterion penalized by the sum of two penalty terms. [START_REF] Raskutti | Lower bounds on minimax rates for nonparametric regression with additive sparsity and smoothness[END_REF][START_REF] Raskutti | Minimax-optimal rates for sparse additive models over kernel classes via convex programming[END_REF] proposed a doubly penalized estimator which is the solution of the minimization of a least-squares criterion penalized by the sum of a sparsity penalty term and a by solving RKHS ridge group sparse optimization problem combination of the l 1 -norm with the Hilbert norm,

γ f n,1 + µ f H,1 , (4.8)
where γ, µ ∈ R are the tuning parameters that should be suitably chosen. [START_REF] Meier | High-dimensional additive modeling[END_REF] proposed a related family of estimators, based on the penalization with the empirical L 2 -norm. Their penalty function is the sum of the sparsity penalty term, f n,1 , and a smoothness penalty term. Huet and Taupin (2017) considered the same approximation functional spaces as [START_REF] Durrande | Anova kernels and rkhs of zero mean functions for model-based sensitivity analysis[END_REF], and obtained a doubly penalized estimator of a metamodel which approximates the Hoeffding decomposition of m. Their estimator is the solution of least-squares minimization penalized by the penalty function defined in Equation (4.8) adapted to the multivariate setting,

γ f n + µ f H , (4.9)
with

f n = v∈P f v n , and f H = v∈P f v Hv .
This procedure, called RKHS ridge group sparse, estimates the groups v that are suitable for predicting f * , and the relationship between f * v and X v for each group. The obtained estimator, called RKHS meta-model, is used then to estimate the Sobol indices of m. This approach makes it possible to estimate the Sobol indices for all groups in the support of the RKHS meta-model, including the interactions of possibly high order, a point known to be difficult in practice.

In this Chapter, an R package, called RKHSMetaMod, that implements the RKHS ridge group sparse procedure is proposed. This package deals with the input variables X = (X 1 , ..., X d ) that are independent and uniformly distributed on X = [0, 1] d , i.e. X ∼ P X = P 1 × ... × P d , with P a , a = 1, ..., d being the uniform law on the interval [0, 1]. It allows to:

(1) calculate reproducing kernels and their associated Gram matrices (see Section 4.3.1),

(2) implement the RKHS ridge group sparse procedure and a special case of it called the RKHS group lasso procedure, i.e. when γ = 0 in the penalty function (4.9), in order to estimate the terms f * v in the Hoeffding decomposition of f * leading to an estimation of the function m (see Section 4.3.2),

(3) choose the tuning parameters µ and γ (see Equation (4.9)), using a procedure that leads to obtain the best RKHS meta-model in terms of the prediction quality, (4) estimate the Sobol indices of the function m (see Section 4.2.4). by solving RKHS ridge group sparse optimization problem of f over sets v ∈ P. Moreover, the Hilbert norm favours the smoothness of the estimated f v , v ∈ P. Let us define the set of functions,

F = f : f = f 0 + v∈P f v , with f v ∈ H v , and f v Hv ≤ r v , r v > 0 .
Then the RKHS meta-model is defined by,

f = arg min f ∈F L(f ). (4.11)
According to the Representer Theorem [START_REF] Kimeldorf | A correspondence between bayesian estimation on stochastic processes and smoothing by splines[END_REF]) the nonparametric functional minimization problem described above is equivalent to a parametric minimization problem. Indeed, the solution of the minimization problem (4.11) belonging to the RKHS H is written as f = f 0 + v∈P f v , where for some matrix θ = (θ vi , i = 1, ..., n, v ∈ P) ∈ R n×|P| we have for all v ∈ P,

f v (.) = n i=1 θ vi k v (X vi , .).
Let . be the Euclidean norm in R n , and for each v ∈ P, let K v be the n × n Gram matrix associated with the kernel k v (., .), i.e.

(K v ) i,i = k v (X vi , X vi ). Let also K 1/2 v
be the matrix that satisfies t(K

1/2 v )K 1/2 v
= K v , and let f 0 and θ be the minimizers of the following penalized least-squares criterion:

C(f 0 , θ) = Y -f 0 I n - v∈P K v θ v 2 + √ n v∈P γ v K v θ v + n v∈P µ v K 1/2 v θ v .
Then the estimator f defined in Equation (4.11) satisfies,

f (X) = f 0 + v∈P f v (X v ) with f v (X v ) = n i=1 θ vi k v (X vi , X v ).
Remark 4.2.1 The constraint f v Hv ≤ r v is not taken into account in the parametric minimization problem. This constraint is crucial for theoretical properties but the value of r v is unknown and has no practical usefulness.

For each v ∈ P, let γ v and µ v be the weights that are chosen suitably. We define,

γ v = γ × γ v and µ v = µ × µ v with γ, µ ∈ R + .
Remark 4.2.2 This formulation simplify the choice of the tuning parameters, since instead of tuning the parameters γ v and µ v for all v ∈ P, only two parameters γ and µ are tuned. Moreover, the weights γ v and µ v , v ∈ P, may be of interest in applications. For example, one can take weights that increase with the cardinal of v in order to favour effects with small interaction order between variables.

For the sake of simplicity, in the rest of this Chapter for all v ∈ P the weights γ v and µ v are assumed to be setted as 1, and the RKHS ridge group sparse criterion is then expressed as follows:

C(f 0 , θ) = Y -f 0 I n - v∈P K v θ v 2 + √ nγ v∈P K v θ v + nµ v∈P K 1/2 v θ v . (4.12)
By considering only the second part of the penalty function in the RKHS ridge group sparse criterion (4.12), i.e. by setting γ = 0, the RKHS group lasso criterion is obtained as follows:

C g (f 0 , θ) = Y -f 0 I n - v∈P K v θ v 2 + nµ v∈P K 1/2 v θ v , (4.13)
which is a group lasso criterion [START_REF] Yuan | Model selection and estimation in regression with grouped variables[END_REF]) up to a scale transformation.

In the RKHSMetaMod package, the RKHS ridge group sparse algorithm is initialized using the solutions obtained by solving the RKHS group lasso algorithm. Indeed, the penalty function in the RKHS group lasso criterion (4.13) insures the sparsity in the solution. Therefore, for a given value of µ, by implementing the RKHS group lasso algorithm (see Section 4.3.2.1), a RKHS meta-model with few terms in its additive decomposition is obtained. The support and the coefficients of a RKHS meta-model which is obtained by implementing RKHS group lasso algorithm will be denoted by S f Group Lasso and θ Group Lasso , respectively.

From now on the tuning parameter in the RKHS group lasso criterion will be denoted by:

µ g = √ nµ. (4.14)

RKHS construction

We consider a RKHS H that is constructed as a direct sum of Hilbert spaces, and that is associated with a so-called ANOVA kernel. The ANOVA kernel is defined in order to obtain the analytical expression of the terms of the Hoeffding decomposition of the functions of H. Therefore, any function f in H is a candidate to approximate the Hoeffding decomposition of m. The construction of the RKHS H has been proposed by [START_REF] Durrande | Anova kernels and rkhs of zero mean functions for model-based sensitivity analysis[END_REF] that we recall briefly in the following. Let X = X 1 × . . . × X d be a subset of R d . For each a ∈ {1, • • • , d}, we choose a RKHS H a and its associated kernel k a defined on the set X a ⊂ R such that the two following properties are satisfied:

(i) k a : X a × X a → R is P a ⊗ P a measurable, (ii) E Xa k a (X a , X a ) < ∞.
The property (ii) depends on the kernel k a , a = 1, ..., d and the distribution of X a , a = 1, ..., d. It is not very restrictive since it is satisfied, for example, for any bounded kernel. by solving RKHS ridge group sparse optimization problem

The RKHS H a can be decomposed as a sum of two orthogonal sub-RKHS,

H a = H 0a ⊥ ⊕ H 1a ,
where H 0a is the RKHS of zero mean functions,

H 0a = f a ∈ H a : E Xa (f a (X a )) = 0 ,
and H 1a is the RKHS of constant functions,

H 1a = f a ∈ H a : f a (X a ) = C .
The kernel k 0a associated with the RKHS H 0a is defined by:

k 0a (X a , X a ) = k a (X a , X a ) - E U ∼Pa (k a (X a , U ))E U ∼Pa (k a (X a , U )) E (U,V )∼Pa⊗Pa k a (U, V ) . (4.15) Let k v (X v , X v ) = a∈v k 0a (X a , X a )
, then the ANOVA kernel k(., .) is defined as follows:

k(X, X ) = d a=1 1 + k 0a (X a , X a ) = 1 + v∈P k v (X v , X v ).
For H v being the RKHS associated with the kernel k v , the RKHS associated with the ANOVA kernel is then defined by,

H = d a=1 1 ⊥ ⊕ H 0a = 1 + v∈P H v .
where ⊥ denotes the L 2 inner product.

According to this construction, any function f ∈ H satisfies decomposition (4.6),

f (X) = f, k(X, .) H = f 0 + v∈P f v (X v ),
which is the Hoeffding decomposition of f . The regularity properties of the RKHS H constructed as described above, depend on the set of the kernels (k a , a = 1, ..., d). This method allows to choose different approximation spaces independently of the distribution of the input variables X 1 , ..., X d , by choosing different sets of kernels. While as mentioned earlier, in the meta-modelling approach based on polynomial Chaos expansion, according to the distribution of the input variables X 1 , ..., X d a unique family of orthonormal polynomials {φ j } ∞ j=0 is determined. Here, the distribution of the components of X occurs only for the orthogonalization of the spaces H v , v ∈ P, and not in the choice of the RKHS, under the condition that properties (i) and (ii) are satisfied. This is one of the main advantages of this method compared to the method based on the truncated polynomial Chaos expansion where the smoothness of the approximation is handled only by the choice of the truncation [START_REF] Blatman | Adaptive sparse polynomial chaos expansion based on least angle regression[END_REF]).

Estimation of the Sobol indices

The variance of the function m is estimated by the variance of the estimator f . As the estimator f belongs to the RKHS H, it admits the Hoeffding decomposition and,

var( f (X)) = v∈P var( f v (X v )),
where for all v ∈ P,

var( f v (X v )) = E X ( f 2 v (X v )) = f v 2 2 .
In order to reduce the computational cost in practice, one may estimate the variances of f v (X v ), v ∈ P by their empirical variances.

Let f v. be the empirical mean of f v (X vi ), i = 1, ..., n, then var(

f v (X v )) = 1 n -1 n i=1 ( f v (X vi ) -f v. ) 2 .
For the groups v that belong to the support of f , the estimators of the Sobol indices of m are defined by,

S v = var( f v (X v )) v∈P var( f v (X v ))
, and for the groups v that do not belong to the support of f , we have S v = 0.

In the RKHSMetaMod package, the algorithm to calculate the empirical Sobol indices S v , v ∈ P is implemented as SI_emp function. This function is described in Section 4. 4.2 and illustrated in Examples 4.5.1,4.5.3,4.5.4.

Algorithms

The RKHSMetaMod package implements two optimization algorithms: the RKHS ridge group sparse (see Algorithm 2), and the RKHS group lasso (see Algorithm 1). These algorithms rely on the Gram matrices K v , v ∈ P, that have to be positive definite. Therefore, the first and essential step in this package, is to calculate these matrices and insure their positive definiteness. This step is detailed in an algorithm that is described in Section 4.3.1.

The second step is to estimate the RKHS meta-model. In the RKHSMetaMod package, two different objectives based on different procedures are considered in order to calculate this estimator:

1. The RKHS meta-model with the best prediction quality:

A sequence of values of the tuning parameters (µ, γ) is considered, and the RKHS meta-models associated with each pair of values of (µ, γ) are calculated. For γ = 0, the RKHS meta-model is obtained by solving the RKHS group lasso optimization problem, while for γ = 0 the RKHS ridge group sparse optimization problem is solved to calculate the RKHS meta-model. The obtained meta-models are evaluated by considering a new dataset. The RKHS meta-model with minimum value of prediction error is chosen as the best estimator (see Section 4.2.3).

2. The RKHS meta-model with at most qmax active groups:

The tuning parameter γ is set as zero. A value of µ for which the number of groups in the solution of the RKHS group lasso optimization problem is equal to qmax, is computed. This value will be denoted by µ qmax . Then, the RKHS ridge group sparse algorithm is implemented for a grid of values of γ = 0 and µ qmax . This algorithm is described in Section 4.3.2.3.

Calculation of the Gram matrices

The available kernels in the RKHSMetaMod package are: linear kernel, quadratic kernel, brownian kernel, matern kernel and gaussian kernel. The usual presentation of these kernels is given in Table 4.1. The choice of the kernel that is done by

Kernel type Mathematics formula for u ∈ R n , v ∈ R RKHSMetaMod name Linear k a (u, v) = u T v + 1 "linear" Quadratic k a (u, v) = (u T v + 1) 2 "quad" Brownian k a (u, v) = min(u, v) + 1 "brownian" Matern k a (u, v) = (1 + 2|u -v|) exp(-2|u -v|) "matern" Gaussian k a (u, v) = exp(-2 u -v 2 )
"gaussian" Table 4.1: List of the reproducing kernels used to construct the RKHS H.

the user, determines the functional approximation space. For a chosen kernel, the algorithm to calculate the Gram matrices K v , v ∈ P in the RKHSMetaMod package, is implemented as calc_Kv function. This algorithm is based on three essential points:

(1) Modify the chosen kernel:

In order to satisfy the conditions of constructing the RKHS H described in Section 4.2.2, these kernels are modified according to Equation (4.15). Let us take the example of the Brownian kernel:

Example 4.3.1 The RKHS associated with the brownian kernel k a (X a , X a ) = min(X a , X a ) + 1 is well known to be the set,

H a = f : [0, 1] → R is absolutely continuous, and f (0) = 0, 1 0 f (X a ) 2 dX a < ∞ , with the inner product f, h Ha = 1 0 f (X a )h (X a )dX a .

by solving RKHS ridge group sparse optimization problem

The kernel k 0a associated with the brownian kernel is calculated as follows,

k 0a = min(X a , X a ) + 1 - ( 1 0 (min(X a , U ) + 1)dU )( 1 0 (min(X a , U ) + 1)dU ) ( 1 0 1 0 (min(U, V ) + 1)dU dV ) , = min(X a , X a ) + 1 - 3 4 (1 + X a - X 2 a 2 )(1 + X a - X 2 a 2
).

The RKHS associated with the kernel k 0a is the set,

H 0a = f ∈ H a : 1 0 f (X a )dX a = 0 .
Finally, the RKHS H = 1 + v∈P H v is the following set,

H = f : [0, 1] d → R : f = f 0 + v∈P f v (X v ), with f v ∈ H v .
Remark 4.3.1 In this package, the input variables X = (X 1 , ..., X d ) that are uniformly distributed on [0, 1] d are considered. In order to consider the input variables that are not distributed uniformly, it suffices to modify a part of the function calc_Kv related to the calculation of kernels k 0a , a = 1, ..., d. For example, for X = (X 1 , ..., X d ) being distributed with law P X = d a=1 P a on X = d a=1 X a ⊂ R d , the kernel k 0a associated with the brownian kernel is calculated as follows, k 0a = min(X a , X a ) + 1 -( Xa (min(X a , U ) + 1)dP a )( Xa (min(X a , U ) + 1)dP a )

( Xa Xa (min(U, V ) + 1)dP a dP a ) .

The other parts of function calc_Kv remain unchanged.

(2) Calculate the Gram matrices K v for all v:

First, for all a = 1, ...d the Gram matrices K a associated with kernels k 0a are calculated using Equation (4.15),

(K a ) i,i = k 0a (X ai , X ai ).
Then, for all v ∈ P, the Gram matrices K v associated with kernel k v = a∈v k 0a are calculated as follows:

K v = a∈v K a ,
where denotes the Hadamard product.

(3) Insure the positive definiteness of the matrices K v :

The output of the function calc_Kv is one of the input arguments of the functions associated with the RKHS group lasso and the RKHS ridge group sparse algorithms. As both of these algorithms rely on the positive definiteness of these matrices, it is mandatory to have K v , v ∈ P that are positive definite. The options, "correction" and "tol", are provided by the function calc_Kv in order to insure the positive definiteness of the matrices K v , v ∈ P. Let us briefly explain this part of the algorithm:

For each group v ∈ P, let λ v,i , i = 1, ..., n be the eigenvalues associated with the matrix K v . Set λ v,max = max i λ v,i and λ v,min = min i λ v,i . For some fixed value of tolerance "tol", and for each matrix

K v , "if λ v,min < λ v,max × tol", then the "correction" to K v is done. That is, "The eigenvalues of K v are replaced by λ v,i + epsilon",
where "epsilon" is equal to λ v,max ×"tol".

The value of "tol" is set as 1e -8 by default, but one may consider a smaller or greater value for it depending on the kernel chosen and the value of n.

The function calc_Kv is described in Section 4.4.2 and illustrated in Example 4.5.3.

Optimization algorithms

The RKHS meta-model is the solution of one of the optimization problems: the minimization of the RKHS group lasso criterion presented in Equation (4.13) (if γ = 0), or the minimization of the RKHS ridge group sparse criterion presented in Equation (4.12) (if γ = 0). In the following the algorithms to solve these optimization problems are presented.

RKHS group lasso

A popular technique for doing group wise variable selection is group lasso. With this procedure, depending on the value of the tuning parameter µ, an entire group of predictors may drop out of the model. An efficient algorithm for solving group lasso problem is the classical block coordinate descent algorithm [START_REF] Boyd | Distributed optimization and statistical learning via the alternating direction method of multipliers[END_REF], [START_REF] Bubeck | Convex optimization: Algorithms and complexity[END_REF]). Following the idea of Fu (1998), [START_REF] Yuan | Model selection and estimation in regression with grouped variables[END_REF] implemented a block wise descent algorithm for the group lasso penalized least-squares, under the condition that the model matrices in each group are orthonormal. A block coordinate (gradient) descent algorithm for solving the group lasso penalized logistic regression is then developed by [START_REF] Meier | The group lasso for logistic regression[END_REF]. This algorithm is implemented in the grplasso R package available from CRAN at https://cran.r-project.org/ web/packages/grplasso/. [START_REF] Yang | A fast unified algorithm for solving group-lasso penalize learning problems[END_REF] proposed an unified algorithm, named groupwise majorization descent, for solving the general group lasso learning problems by assuming that the loss function satisfies a quadratic majorization condition. The implementation of their work is done in the gglasso R package available at https://cran.r-project.org/web/packages/gglasso/ from CRAN. by solving RKHS ridge group sparse optimization problem

In order to solve the RKHS group lasso optimization problem, the classical block coordinate descent algorithm is used. The minimization of criterion C g (f 0 , θ) (see Equation (4.13)) is done along each group v at a time. At each step of the algorithm, the criterion C g (f 0 , θ) is minimized as a function of the current block's parameters, while the parameters values for the other blocks are fixed to their current values. The procedure is repeated until convergence.

This procedure leads to Algorithm 1 (see Section 4.A for more details on this procedure).

In the RKHSMetaMod package the Algorithm 1 is implemented as RKHSgrplasso function. This function is described in Section 4.4.2 and illustrated in Example 4.5.3.

Algorithm 1 RKHS group lasso algorithm:

1: Set θ 0 = [0] |P|×n 2: repeat 3: Calculate f 0 = arg min f 0 C g (f 0 , θ) 4: for v ∈ P do 5: Calculate R v = Y -f 0 -v =w K w θ w 6: if 2 √ n K 1/2 v R v ≤ µ g then 7: θ v ← 0 8: else 9: θ v ← arg min θv C g (f 0 , θ) 10: end if 11:
end for 12: until convergence

RKHS ridge group sparse

In order to solve the RKHS ridge group sparse optimization problem, an adapted block coordinate descent algorithm is proposed. This algorithm provides two steps:

Step 1 Initialize the input parameters by the solutions of the RKHS group lasso algorithm for each value of the tuning parameter µ, and implement the RKHS ridge group sparse algorithm through active support of the RKHS group lasso solutions until it achieves convergence.

This step is provided in order to decrease the execution time. In fact, instead of implementing the RKHS ridge group sparse algorithm over the set of all groups P, it is implemented only over the active support obtained by the RKHS group lasso algorithm, S f Group Lasso .

Step 2 Re-initialize the input parameters with the obtained solutions of Step 1 and implement the RKHS ridge group sparse algorithm through all groups in P until it achieves convergence.

This second step makes it possible to verify that no group is missing in the output of Step 1.

This procedure leads to Algorithm 2 (see Section 4.A for more details on this procedure).

In the RKHSMetaMod package the Algorithm 2 is implemented as pen_MetMod function. This function is described in Section 4.4.2 and illustrated in Example 4.5.3.

Algorithm 2 RKHS ridge group sparse algorithm:

1:

Step 1: 2: Set θ 0 = θ Group Lasso and P = S f Group Lasso 3: repeat 4:

Calculate f 0 = arg min f 0 C(f 0 , θ) 5: for v ∈ P do 6: Calculate R v = Y -f 0 -v =w K w θ w 7: Solve J * = arg min tv∈R n {J( t v ), such that K -1/2 v t v ≤ 1} 8: if J * ≤ γ then 9: θ v ← 0 10: else 11: θ v ← arg min θv C(f 0 , θ) 12:
end if 13:

end for 14: until convergence 15: Step 2: 16: Implement the same procedure as Step 1 with θ 0 = θ old , P = P θ old is the estimation of θ in Step 1.

RKHS meta-model with qmax active groups

By considering some prior information about the data, one may be interested in an RKHS meta-model f with the number of active groups not greater than some "qmax". In order to obtain the estimator f with at most "qmax" active groups, the following procedure is provided in the RKHSMetaMod package: First, the tuning parameter γ is set as zero and a value of µ for which the solution of the RKHS group lasso algorithm, Algorithm 1, contains exactly qmax active groups is computed. This value is denoted by µ qmax .

Then, the RKHS ridge group sparse algorithm, Algorithm 2, is implemented by setting the tuning parameter µ equals to µ qmax , and a grid of values of the tuning parameter γ > 0.

Algorithm 3 Algorithm to estimate RKHS meta-model with at most qmax active groups:

1: Calculate µ max = max v 2 √ n K 1/2 v (Y -Y ) 2:
Set µ 1 = µ max and µ 2 = µmax rat "rat" is setted by user. Implement RKHS group lasso algorithm, Algorithm 1, with µ i = µ 1 +µ 2 2 5:

Set q = | S f Group Lasso | 6: if q > qmax then 7: Set µ 1 = µ 1 and µ 2 = µ i 8: else 9: Set µ 1 = µ i and µ 2 = µ 2 10:
end if 11: until q = qmax or i >Num "Num" is setted by user.

12: Implement RKHS ridge group sparse algorithm, Algorithm 2, with (µ = µ qmax , γ > 0)

This procedure leads to Algorithm 3. This algorithm is implemented in the RKHSMeta-Mod package, as function RKHSMetMod_qmax. This function is described in Section 4.4.1 and illustrated in Example 4.5.2.

Remark 4.3.2 As both terms in the penalty function of criterion (4.12) enforce sparsity to the solution, the estimator obtained by solving the RKHS ridge group sparse associated with the pair of the tuning parameters (µ qmax , γ > 0) may contain a smaller number of groups than the solution of the RKHS group lasso optimization problem (i.e. the RKHS ridge group sparse with (µ qmax , γ = 0)). And therefore, the estimated RKHS meta-model contains at most "qmax" active groups.

Overview of the RKHSMetaMod functions

In the R environment, one can install and load the RKHSMetaMod package by using the following commands:

R> install.packages("RKHSMetaMod") R> library("RKHSMetaMod")

The optimization problems in this package are solved using block coordinate descent algorithm which requires various computational algorithms including generalized Newton, Broyden and Hybrid methods. In order to gain the efficiency in terms of the calculation time and be able to deal with high dimensional problems, the computationally efficient tools of C++ packages Eigen ( rat: positive scalar, to restrict the minimum value of µ considered in Algorithm 3,

µ min = µ max ( √ n × rat) ,
where the value of µ max is given by Equation (4.16) and is calculated inside the program.

-Num: integer, to restrict the number of different values of the tuning parameter µ to be evaluated in the RKHS group lasso algorithm until it achieves µ qmax . For example, if Num equals to 1 the program is implemented for three different by solving RKHS ridge group sparse optimization problem mus: vector of all values of µ i in Algorithm 3.

qs: vector with the same length as mus. Each element of the vector shows the number of active groups in the RKHS meta-model obtained by solving RKHS group lasso problem for an element in mus.

MetaModel: list with the same length as the vector gamma. Each component of the list is a list of three components "mu", "gamma" and "Meta-Model":

mu: value of µ qmax .

gamma: element of the input vector gamma associated with the estimated "Meta-Model".

-Meta-Model: an RKHS ridge group sparse or RKHS group lasso object associated with the tuning parameters mu and gamma (see Table 4.3).

Companion functions

calc_Kv function: For a given value of Dmax and a chosen kernel (see Table 4.1), this function calculates the Gram matrices K v , v ∈ P Dmax , and returns their associated eigenvalues and eigenvectors. This function has, four mandatory input arguments:

-Y , X, kernel, Dmax (see Table 4.2). three facultative input arguments:

correction: logical, set as TRUE to make correction to the matrices K v (see Section 4.3.1). It is set as TRUE by default.

verbose: logical, set as TRUE to print: the group for which the correction is done. It is set as TRUE by default.

tol: scalar to be chosen small, set as 1e -8 by default.

The calc_Kv function returns a list of two components "kv" and "names.Grp": kv: list of vMax components, each component is a list of, -Evalues: vector of eigenvalues.

-Q: matrix of eigenvectors.

names.Grp: vector of group names of size vMax.

RKHSgrplasso function: For a given value of the tuning parameter µ g , this function fits the solution to the RKHS group lasso optimization problem by implementing Algorithm 1. This function has, three mandatory input arguments: 4.2).

-Kv: list of the eigenvalues and the eigenvectors of the positive definite Gram matrices K v for v = 1, ...,vMax and their associated group names (output of the function calc_Kv).

mu: positive scalar indicates the value of the tuning parameter µ g defined in Equation (4.14).

two facultative input arguments:

-maxIter: integer, to set the maximum number of loops through all groups.

It is set as 1000 by default.

verbose: logical, set as TRUE to print: the number of current iteration, active groups and convergence criterion. It is set as FALSE by default.

This function returns an RKHS group lasso object associated with the tuning parameter µ g . Its output is a list of 13 components:

intercept, teta, fit.v, fitted, Norm.H, supp, Nsupp, SCR, crit, MaxIter, convergence, RelDiffCrit, RelDiffPar (see Table 4.3).

mu_max function: This function calculates the value µ max defined in Equation (4.16). It has two mandatory input arguments: the response vector Y , and the list matZ of the eigenvalues and eigenvectors of the positive definite Gram matrices K v for v = 1, ...,vMax. This function returns the µ max value.

pen_MetMod function: This function produces a sequence of the RKHS metamodels associated with a given grid of values of the tuning parameters µ, γ. Each RKHS meta-model in the sequence is the solution to the RKHS ridge group sparse optimization problem (obtained by implementing Algorithm 2) associated with a pair of values of (µ, γ) in the grid of values of µ, γ. This function has, seven mandatory input arguments:

-Y (see Table 4.2).

gamma: vector of positive scalars. Values of the penalty parameter γ in decreasing order.

-Kv: list of the eigenvalues and the eigenvectors of the positive definite Gram matrices K v for v = 1, ...,vMax and their associated group names (output of the function calc_Kv).

mu: vector of positive scalars. Values of the tuning parameter µ in decreasing order.

resg: list of the RKHS group lasso objects associated with the components of "mu", used as initial parameters at Step 1. by solving RKHS ridge group sparse optimization problem -gama_v and mu_v: vector of vMax positive scalars. These two inputs are optional, they are provided to associate the weights to the two penalty terms in the RKHS ridge group sparse criterion (4.12). They set to scalar 0, to consider no weights, i.e. all weights equal to 1.

three facultative input arguments:

-maxIter: integer, to set the maximum number of loops through initial active groups at Step 1 and maximum number of loops through all groups at Step 2. It is set as 1000 by default.

verbose: logical, set as TRUE to print: for each pair of the tuning parameters (µ, γ): the number of current iteration, active groups and convergence criterion. It is set as FALSE by default.

-calcStwo: logical, set as TRUE to execute Step 2. It is set as FALSE by default.

The function pen_MetMod returns a list of l components, with l equals to the number of pairs of the tuning parameters (µ, γ). Each component of the list is a list of three components "mu", "gamma" and "Meta-Model": mu: positive scalar, an element of the input vector "mu" associated with the estimated "Meta-Model".

gamma: positive scalar, an element of the input vector "gamma" associated with the estimated "Meta-Model".

Meta-Model: an RKHS ridge group sparse object associated with the tuning parameters mu and gamma (see Table 4.3).

PredErr function: By considering a testing dataset, this function calculates the prediction errors for the obtained RKHS meta-models. This function has eight mandatory input arguments:

-X, gamma, kernel, Dmax (see Table 4.2).

-XT : matrix of observations of the testing dataset with n test rows and d columns.

-Y T : vector of response observations of the testing dataset of size n test .

mu: vector of positive scalars. Values of the tuning parameter µ in decreasing order.

res: list of the estimated RKHS meta-models for the learning dataset associated with the tuning parameters (µ, γ) (it could be the output of one of the functions RKHSMetMod, RKHSMetMod_qmax or pen_MetMod).

RKHSMetaMod through examples 137

Note that, the same kernel and Dmax have to be chosen as the ones used for the learning dataset.

The function PredErr returns a matrix of the prediction errors. Each element of the matrix corresponds to the prediction error of one RKHS meta-model in "res".

SI_emp function: For each RKHS meta-model f , this function calculates the empirical Sobol indices for all groups that are active in the support of f . This function has two input arguments:

res: list of the estimated meta-models using RKHS ridge group sparse or RKHS group lasso algorithms (it could be the output of one of the functions RKHSMetMod, RKHSMetMod_qmax or pen_MetMod).

-ErrPred: matrix or NULL. If matrix, each element of the matrix corresponds to the prediction error of an RKHS meta-model in "res" (output of the function PredErr). Set as NULL by default.

The empirical Sobol indices are then calculated for each RKHS meta-model in "res", and a list of vectors of the Sobol indices is returned. If the argument "ErrPred" is the matrix of the prediction errors, the vector of empirical Sobol indices is returned for the best RKHS meta-model in the "res".

RKHSMetaMod through examples

Let us consider the g-function of Sobol [START_REF] Saltelli | Sensitivity Analysis[END_REF]) in the Gaussian regression framework, i.e. Y = m(X) + σε, σ > 0,

where the error term ε is a centered Gaussian random variable, and where the function m is the g-function of Sobol defined over [0, 1] d by,

m(X) = d a=1 |4x a -2| + c a 1 + c a , c a > 0.
(4.17)

The Sobol indices of the g-function can be expressed analytically:

∀v ∈ P, S v = 1 D a∈v D a , D a = 1 3(1 + c a ) 2 , D = d a=1 (D a + 1) -1.
Set c 1 = 0.2, c 2 = 0.6, c 3 = 0.8 and (c a ) a>3 = 100. With these values of coefficients c a , the variables X 1 , X 2 and X 3 explain 99.99% of the variance of the function m(X) [START_REF] Durrande | Anova kernels and rkhs of zero mean functions for model-based sensitivity analysis[END_REF]). The values of S v , v ∈ P, when d = 5 and d = 10 are displayed in Tables 4.6 and 4.7, respectively.

In this Section, four examples are presented. In all examples the value of Dmax is set as three. Example 4.5.1 illustrates the use of the RKHSMetMod function by considering three different kernels, "matern", "brownian", and "gaussian" (see Table by solving RKHS ridge group sparse optimization problem "the value of mu is: 2.083 and the value of gamma is: 0" "the active groups are: " "v1." "v2." "v3." Let us comment the outputs of the function RKHSMetMod_qmax: for γ = 0 the value "mu" corresponds to the value of µ qmax=3 which is equal to 0.093, while for γ = 0 the value "mu" corresponds to the value of µ g defined in Equation (4.14),

µ g = √ n × 0.093 = 2.083.
For each pair of the tuning parameters (µ qmax , γ i ), i = 1, ..., 5, the estimated RKHS meta-model contains three groups. The groups associated with X 1 , X 2 , and X 3 are "v1.", "v2.", and "v3.", that are active in the estimators obtained, as expected.

Example 4.5.3 A time saving trick to obtain the "optimal" tuning parameters when dealing with larger datasets:

A dataset of n points maximinLHS over [0, 1] d with n = 1000 and d = 10 is generated. Firstly, the eigenvalues and eigenvectors of the positive definite matrices K v , and the value of µ max is computed using functions calc_Kv and mu_max, respectively:

R> kernel <-"matern" R> Dmax <-3 R> Kv <-calc_Kv(X,kernel,Dmax,TRUE,TRUE) R> mumax <-mu_max(Y,Kv$kv) Then, the two following steps are considered:

1. Set γ = 0 and, µ (1:9) = µ max ( √ n × 2 (2:10) )

.

Calculate the RKHS meta-models associated with the values of µ g = µ × √ n by using the function RKHSgrplasso. Gather the obtained RKHS meta-models in a list, "res_g". While this job could be done with the function RKHSMetMod by setting γ = 0, in this example we use the function RKHSgrplasso in order to avoid the re-calculation of K v 's at the next step. Thereafter, for each estimator in the res_g the prediction error is calculated by considering a new dataset and using the function PredErr. The value of µ with the smallest error of prediction in this step is denoted by µ i .

Let us implement this step:

For a grid of values of µ g , a sequence of the RKHS meta-models are calculated and gathered in the "res_g" list:

R> frc <-c (4,8,16,32,64,128,256,512,1024) R> mu_g <-mumax/frc R> res_g <-list();resg <-list() R> for(i in 1:length(mu_g)){ resg[ [i]] <-RKHSgrplasso(Y,Kv,mu_g [i],1000,FALSE) res_g[ [i]] <-list("mu_g"=mu_g,"gamma"=0,"MetaModel"=resg[ [i] Two datasets of n points maximinLHS over [0, 1] d with n ∈ {2000, 5000} and d = 10 are generated. In order to obtain one RKHS meta-model associated with one pair of the tuning parameters (µ, γ), the number of coefficients to be estimated is equal to n×vMax= n × 175. Table 4.11 gives the execution time for different functions used throughout the Examples 4.5.1-4.5.4. As we can see, the execution time increases fastly as n increases. In Figure 4.1 the plot of the logarithm of the time versus the logarithm of n is displayed for the functions calc_Kv, mu_max, RKHSgrplasso and pen_MetMod. It appears that, the algorithms of these functions are of polynomial time O(n α ) with α 3 for the functions calc_Kv and mu_max, and α 2 for the functions RKHSgrplasso and pen_MetMod.

Taking into account the results obtained for the prediction error and the values of ( µ, γ) in Example 4.5.3, in this example only two values of the tuning parameter µ and one value of the tuning parameter γ are considered:

µ = ( µ max ( √ n × 2 7 ) , µ max ( √ n × 2 8 )
) and γ = 0.01. by solving RKHS ridge group sparse optimization problem kernels allows to consider different approximation spaces and choose the one that gives the best result.

For the large values of n and d the calculation and storage of the eigenvalues and the eigenvectors of all Gram matrices K v , v ∈ P requires a lot of time and a very large amount of memory. In order to optimize the execution time and also the storage memory, except for a function that is written in R, all of the functions of RKHSMetaMod package are written using the efficient C++ libraries through RcppEigen and RcppGSL packages. These functions are then interfaced with the R environment in order to propose an user friendly package.

The performance of the package functions in terms of the predictive quality of the estimator and the estimation of the Sobol indices, is validated by a simulation study (see Examples 4.5.1-4.5.4).

The strategy of choosing the tuning parameters in this package is based on the minimization of the prediction error of the estimated meta-model, the prediction error being estimated using a testing dataset. The best estimator is selected in terms of the prediction quality, and the Sobol indices are deduced from it. If one is specially interested in the estimation of the Sobol indices, an alternative to our approach could be to calculate the tuning parameters which minimize the prediction error of the Sobol indices. by solving RKHS ridge group sparse optimization problem Finally, we get

∂y(ρ) ∂ρ = 2 ( In ρ + k -1 v ) -3/2 k -1/2 v R v ρ 2 ( In ρ + k -1 v ) -1 k -1/2 v R v > 0.
So y(ρ) is an increasing function of ρ, and the proof is complete. 2 In order to calculate ρ and so θ v = θ(ρ) we use Algorithm 4 which is a part of the RKHS group lasso Algorithm 1 when θ v = 0.

Algorithm 4 Algorithm to find ρ as well as θ v 1: if θ old = 0 then θ old is θ v computed in the previous step of the RKHS group lasso algorithm.

2:

Set ρ ← 1 and calculate y(ρ)

3:

if y(ρ) > 0 then 4:

Find ρ that minimizes y(ρ) on the interval [0, 1]

4.A.2 RKHS ridge group sparse algorithm

We consider the minimization of the RKHS ridge group sparse criterion:

C(f 0 , θ) = Y -f 0 I n - v∈P K v θ v 2 + √ nγ v∈P K v θ v + nµ v∈P K 1/2 v θ v .
The constant term f 0 is estimated as in the RKHS group lasso algorithm. In order to calculate θ = arg min θ∈R n×|P| C(f 0 , θ), we use once again the block coordinate descent algorithm group v by group v. In the following, we fix a group v, and we find the minimizer of C(f 0 , θ) with respect to θ v for given values of f 0 and θ w , w = v. We aim at minimizing with respect to θ v ,

C v (f 0 , θ v ) = R v -K v θ v 2 + √ nγ K v θ v + nµ K 1/2 v θ v
, where R v is defined by (4.19).

Let ∂C v be the sub-differential of C v (f 0 , θ v ) with respect to θ v ,

∂C v = {-2K v (R v -K v θ v ) + √ nγs v + nµt v : s v ∈ ∂ K v θ v , t v ∈ ∂ K 1/2 v θ v },
According to the first order optimality condition (see Preliminary 4.A.2), we know that there exists

s v ∈ ∂ K v θ v and t v ∈ ∂ K 1/2 v θ v such that, -2K v (R v -K v θ v ) + √ nγ s v + nµ t v = 0. (4.23)
The sub-differential definition (see Preliminary 4.A.1) gives,

{∂ K 1/2 v θ v = { K v θ v K 1/2 v θ v }, ∂ K v θ v = { K 2 v θ v K v θ v }} if θ v = 0,
and,

{∂ K 1/2 v θ v = { t v ∈ R n , K -1/2 v t v ≤ 1}, ∂ K v θ v = { s v ∈ R n , K -1 v s v ≤ 1}} if θ v = 0.
Let θ v be the minimizer of the C v (f 0 , θ v ). Using the sub-differential equations above, the estimator θ v , v ∈ P is obtained following two cases below:

Case 1. If θ v = 0 then there exists s v ∈ R n such that K -1 v s v ≤ 1 and it fulfils Equation (4.23): and, J * = arg min

2K v R v -nµ t v = √ nγ s v , with t v ∈ R n , K -1/2 v t v ≤ 1. Set J( t v ) = 2R v -nµK -1 v t v ,
tv∈R n {J( t v ), such that K -1/2 v t v ≤ 1}.
Then the solution to Equation (4.23) is zero if and only if J * ≤ γ. Case 2. If θ v = 0 then we have

s v = K 2 v θ v / K v θ v , and t v = K v θ v / K 1/2
v θ v fulfilling Equation (4.23):

2K v (R v -K v θ v ) = √ nγ K 2 v θ v K v θ v 2 + nµ K v θ v K 1/2 v θ v ,
that is,

θ v = (K v + √ nγ 2 K v θ v K v + nµ 2 K 1/2 v θ v I n ) -1 R v if θ v = 0.
In this case the calculation of θ v needs a numerical algorithm which is explained in Huet and Taupin (2017).

Contents

"nearest positive definite" matrices.

Usage calc_Kv(X, kernel, Dmax, correction, verbose, tol)

Arguments

X

Matrix of observations with n rows and d columns.

kernel Character, the type of the reproducing kernel: matern (matern kernel), brownian (brownian kernel), gaussian (gaussian kernel), linear (linear kernel), quad (quadratic kernel).

Dmax

Integer, between 1 and d, indicates the order of interactions considered in the meta-model: Dmax= 1 is used to consider only the main effects, Dmax= 2 to include the main effects and the interactions of order 2, . . .. correction Logical, if TRUE, the program makes the correction to the matrices K v that are not positive definite (see details). Set as TRUE by default.

verbose Logical, if TRUE, the group v for which the correction is done is printed. Set as TRUE by default.

tol Scalar, used if correction is TRUE. For each matrix K v if λ min < λ max ×tol, then the correction to K v is done (see details). Set as 1e -8 by default. Details Let λ v,i , i = 1, ..., n be the eigenvalues associated with matrix K v . Set λ max = max i λ v,i and λ min = min i λ v,i . The eigenvalues of K v that is not positive definite are replaced by λ v,i +epsilon, with espilon= λ max ×tol. The value of tol depends on the type of the kernel and it is chosen small.

Value

List of two components "names.Grp" and "kv":

names.Grp

Vector of size vMax, indicates the name of groups included in the meta-model.

kv

List of vMax components with the same names as the vector names.Grp. Each element of the list is a list of two components "Evalues" and "Q":

Evalues

Vector of size n, eigenvalues of each Gram matrix K v .

Q

Matrix with n rows and n columns, eigenvectors of each Gram matrix K v . Note: Note.

Author(s) Halaleh Kamari

RKHSMetaMod

Examples d <-3 n <-50 library(lhs) X <-maximinLHS(n, d) c <-c(0.2,0.6,0.8) F <-1;for (a in 1:d) F <-F*(abs(4*X[,a]-2)+c[a])/(1+c[a]) epsilon <-rnorm(n,0,1);sigma <-0.2 Y <-F + sigma*epsilon Dmax <-3 kernel <-"matern" Kv <-calc_Kv(X, kernel, Dmax) names <-Kv$names.Grp Eigen.val1 <-Kv$kv$v1.$Evalues Eigen.vec1 <-Kv$kv$v1.$Q A.2 grplasso_q function grplasso_q Function to fit a solution with q active groups of the RKHS group lasso optimization problem.

Description

This function determines the value µ g (q), for which the number of active groups in the solution of the RKHS group lasso problem is equal to q, and returns the RKHS meta-model associated with µ g (q). Usage grplasso_q(Y, Kv, q, rat, Num) Arguments 160

Contents

Y

Vector of response observations of size n.

Kv

List of eigenvalues and eigenvectors of positive definite Gram matrices K v and their associated group names. It should have the same format as the output of the function calc_Kv (see details).

q Integer, the number of active groups in the obtained solution.

rat Positive scalar, used to restrict the minimum value of µ g , to be evaluted in the RKHS group lasso algorithm, µ min = µ max /rat. The value µ max is calculated inside the program, see function mu_max.

Num

Integer, used to restrict the number of different values of the penalty parameter µ g to be evaluated in the RKHS group lasso algorithm, until it achieves µ g (q): for Num= 1 the program is done for 3 values of µ g , µ 1 = (µ min + µ max )/2, µ 2 = (µ min + µ 1 )/2 or µ 2 = (µ 1 + µ max )/2 depending on the value of q associated with µ 1 , µ 3 = µ min . Details Input Kv should contain the eigenvalues and eigenvectors of positive definite Gram matrices K v . It is necessary to set input "correction" in the function calc_Kv equal to "TRUE".

Value

List of 4 components: "mus", "qs", "mu", "res":

mus

Vector, values of the evaluated penalty parameters µ g in the RKHS group lasso algorithm until it achieves µ g (q). qs Vector, number of active groups associated with each value of µ g in mus.

mu Scalar, value of µ g (q). res An RKHS group lasso object:

intercept Scalar, estimated value of intercept.

teta Matrix with vMax rows and n columns. Each row of the matrix is the estimated vector θ v for v = 1, ...,vMax.

fit.v

Matrix with n rows and vMax columns. Each row of the matrix is the estimated value of

f v = K v θ v .
fitted Vector of size n, indicates the estimator of m.

Norm.H

Vector of size vMax, estimated values of the penalty norm.

supp Vector of active groups.

Nsupp

Vector of the names of the active groups.

SCR

Scalar, equals to

Y -f 0 -v K v θ v 2 .
crit Scalar, indicates the value of the penalized criterion.

MaxIter

Integer, number of iterations until convergence is reached.

convergence TRUE or FALSE. Indicates whether the algorithm has converged or not.

RelDiffCrit

Scalar, value of the first convergence criterion at the last iteration,

Details

For more details about the maximal value of the penalty parameter in the ordinary group lasso algorithm see [START_REF] Meier | The group lasso for logistic regression[END_REF].

Value

An object of type numeric is returned.

Description

This function produces a sequence of the RKHS meta-models associated with a given grid of values of the tuning parameters µ, γ. Each RKHS meta-model in the sequence is the solution to the RKHS ridge group sparse optimization problem associated with a pair of values of (µ, γ) in the grid of values of µ, γ.

Usage pen_MetMod (Y, Kv, gamma, mu, resg, gama_v, mu_v, maxIter, verbose, calcStwo)

Arguments

Y

Vector of response observations of size n.

Kv

List, includes the eigenvalues and eigenvectors of the positive definite Gram matrices K v , v = 1, ..

Value

List of l components, with l equals to the number of pairs of the penalty parameters (µ, γ). Each component of the list is a list of 3 components "mu", "gamma" and "Meta-Model": mu

Positive scalar, an element of the input vector mu associated with the estimated Meta-Model.

gamma Positive scalar, an element of the input vector gamma associated with the estimated Meta-Model.

Meta-Model

Estimated meta-model associated with penalty parameters mu and gamma. List of 16 components:

intercept Scalar, estimated value of intercept.

teta Matrix with vMax rows and n columns. Each row of the matrix is the estimated vector θ v for v = 1, ...,vMax.

fit.v

Matrix with n rows and vMax columns. Each row of the matrix is the estimated value of

f v = K v θ v .
fitted Vector of size n, indicates the estimator of m.

Norm.n

Vector of size vMax, estimated values for the ridge penalty norm.

Norm.H

Vector of size vMax, estimated values for the group sparse penalty norm.

supp Vector of active groups.

Nsupp

Vector of the names of the active groups. For more details about the ordinary group lasso algorithm see [START_REF] Meier | The group lasso for logistic regression[END_REF].

SCR

Value

Estimated RKHS meta-model, list with 13 components: intercept Scalar, estimated value of intercept.

teta Matrix with vMax rows and n columns. Each row of the matrix is the estimated vector θ v for v = 1, ...,vMax.

fit.v

Matrix with n rows and vMax columns. Each row of the matrix is the estimated value of

f v = K v θ v .
fitted Vector of size n, indicates the estimator of m.

Norm.H

Vector of size vMax, estimated values of the penalty norm.

supp Vector of active groups.

Nsupp

Vector of the names of the active groups.

SCR

Scalar equals to

Y -f 0 -v K v θ v 2 .
crit Scalar indicates the value of the penalized criterion.

MaxIter

Integer, number of iterations until convergence is reached.

convergence TRUE or FALSE. Indicates whether the algorithm has converged or not.

RelDiffCrit

Scalar, value of the first convergence criterion at the last iteration,

See Also calc_Kv

Examples d <-3 n <-50 library(lhs) X <-maximinLHS(n, d) c <-c(0.2,0.6,0.8) F <-1;for (a in 1:d) F <-F*(abs(4*X[,a]-2)+c[a])/(1+c[a]) epsilon <-rnorm(n,0,1);sigma <-0.2 Y <-F + sigma*epsilon Dmax <-3 kernel <-"matern" Kv <-calc_Kv(X, kernel, Dmax, TRUE, TRUE) matZ <-Kv$kv mumax <-mu_max(Y, matZ) mug <-mumax/10 gr <-RKHSgrplasso(Y,Kv, mug , 1000, FALSE) gr$Nsupp

A.7 RKHSMetMod function

RKHSMetMod

Function to produce a sequence of the RKHS meta-models that are the solutions of the RKHS ridge group sparse or the RKHS group lasso optimization problems.

Description

For a given value of Dmax and a chosen reproducing kernel, this function calculates the Gram matrices K v , v ∈ P Dmax , and produces a sequence of estimators f associated with a given grid of values of tuning parameters µ, γ, i.e. the solutions to the RKHS ridge group sparse (if γ = 0) or the RKHS group lasso problem (if γ = 0).

Usage

RKHSMetMod(Y, X, kernel, Dmax, gamma, frc, verbose) Arguments frc <-c(10) gamma=c(.5,.01,.001) res <-RKHSMetMod(Y,X,kernel,Dmax,gamma,frc,FALSE) mu <-vector() l <-length(gamma) for(i in 1:length(frc))mu [i]=res[[(i-1)*l+1]]$mu error <-PredErr(X,XT, YT,mu,gamma, res, kernel,Dmax) SI.minErr <-SI_emp(res, error) SI <-SI_emp(res, NULL) Titre: Qualité prédictive des méta-modèles construits sur des espaces de Hilbert à noyau auto-reproduisant et analyse de sensibilité des modèles complexes Il appartient à un espace de Hilbert à noyau auto-reproduisant qui est construit comme une somme directe d'espaces de Hilbert [START_REF] Durrande | Anova kernels and rkhs of zero mean functions for model-based sensitivity analysis[END_REF]). L'estimateur du f * , noté f , est calculé en minimisant un critère des moindres carrés pénalisé par la somme de la norme de Hilbert et de la norme empirique L 2 (Huet and Taupin (2017)). Ce travail se compose d'une partie théorique et d'une partie pratique. Dans la partie théorique, j'ai établi les majorations du risque empirique L 2 et du risque quadratique de l'estimateur f d'un modèle de régression où l'erreur est non-gaussienne et non-bornée. Dans la partie pratique, j'ai développé un package R appelé RKHSMetaMod, pour la mise en oeuvre des méthodes d'estimation du méta-modèle f * . Afin d'optimiser le temps de calcul et la mémoire de stockage, toutes les fonctions de ce package ont été écrites en utilisant les bibliothéques GSL et Eigen de C++ à l'exception d'une fonction qui est écrite en R. Elles sont ensuite interfacées avec l'environnement R afin de proposer un package facilement exploitable aux utilisateurs. La performance des fonctions du package en termes de qualité prédictive de l'estimateur et de l'estimation des indices de Sobol, est validée par une étude de simulation.

Title: Predictive quality of meta-models constructed on the reproducing kernel Hilbert spaces and sensitivity analysis of complex models Keywords: meta-model, reproducing kernel Hilbert spaces, Sobol indices, non-parametric regression, penalized least-squares criterion, risk upper bound Abstract: In this work, the problem of estimating a meta-model of a complex model, denoted m, is considered. The model m depends on d input variables that are independent and have a known law. The meta-model, denoted f * , approximates the Hoeffding decomposition of m, and allows to estimate its Sobol indices. It belongs to a reproducing kernel Hilbert space (RKHS) which is constructed as a direct sum of Hilbert spaces [START_REF] Durrande | Anova kernels and rkhs of zero mean functions for model-based sensitivity analysis[END_REF]). The estimator of f * , denoted f , is calculated by minimizing a least-squares criterion penalized by the sum of the Hilbert norm and the empirical L 2norm (Huet and Taupin (2017)). This work consists of a theoretical part and a practical part. In the theoretical part, I established upper bounds of the empirical L 2 risk and the L 2 risk of the estimator f in the regression framework with non-Gaussian and non-bounded error term. In the practical part, I developed an R package, called RKHSMetaMod, that implements the estimation methods of the meta-model f * . In order to optimize the execution time and the storage memory, except for a function that is written in R, all of the functions of this package are written using C++ libraries GSL and Eigen. These functions are then interfaced with the R environment in order to propose an user friendly package. The performance of the package functions in terms of the predictive quality of the estimator and the estimation of the Sobol indices, is validated by a simulation study.

Université Paris-Saclay Espace Technologique / Immeuble Discovery Route de l'Orme aux Merisiers RD 128 / 91190 Saint-Aubin, France

  der Vaart (1998)): m(X) = m 0 + d a=1 m a (X a ) + a<a m a,a (X a , X a ) + ... + m 1,...,d (X).

Chapter 2 .

 2 Introduction in english 2.1.4.3 Approximating the Hoeffding decomposition of m Let f * ∈ H be the orthogonal projection of m on H defined by:

  The result in Corollary 3.3.1 can be generalized to the case where σ = 1 in Equation (3.1), and where r v = 1 in (3.6). It suffices to apply Corollary 3.3.1 with Y /σ and m/σ in place of Y and m, to g as defined in Equation (3.16).

  Point 1. a Sudakov type minoration to link the covering number on a class G to the expectation of the supremum of the empirical process over this class G, E ε sup g∈G |V n,ε (g)|, and conclude Lemma 3.5.4, Point 2. a concentration inequality to bound the quantities defined in Equation (3.25) which leads to bound the empirical process V n,ε and conclude Lemma 3.5.1.

2 2 n

 22 The appropriate results to solve Point 1. and Point 2. are stated in Corollary 3.4.1 in Section 3.4.2.2 and Corollary 3.4.2 in Section 3.4.3.2, respectively. in the regression model with non-Gaussian and non-bounded error 3.4.1.1 Comparison with related works Meier et al. (2009) considered a least-squares criterion penalized by a penalty function similar to the one we consider in our work. Their estimator of the unknown function m has an univariate additive decomposition, i.e. decomposition (3.3) limited to the main effects. They used a compatibility condition to compare the sum of the empirical L 2norm of the univariate functions to the empirical L 2 -norm of the sum of the univariate functions. More precisely, Let S * = {a ∈ {1, ..., d}, f a n = 0}, then for C(f a ) being a term depending on the functions f a , a ∈ S * , + C(f a ).

-

  [START_REF] Raskutti | Minimax-optimal rates for sparse additive models over kernel classes via convex programming[END_REF] assumed that the unknown function m has a sparse univariate decomposition, where each component in its decomposition lies in a RKHS. They obtained an estimator for m, based on a ridge group sparse type procedure. They established upper and lower bounds on the risk in the L 2 -norm and upper bound on the risk in the empirical L 2 -norm.

  Definition 3.4.2 (Definition 4 in[START_REF] Adamczak | Logarithmic sobolev inequalities and concentration of measure for convex functions and polynomial chaoses[END_REF]) For m > 0 and ρ ≥ 0 let M(m, ρ 2 ) denote the class of probability distributions Π on R for whichυ + (A) ≤ ρ 2 Π(A), for all sets A of the form A = [x, ∞), x ≥ m and υ -(A) ≤ ρ 2 Π(A),for all sets A of the form A = (-∞, -x], x ≥ m, where υ + is the measure on [m, ∞) with density xΠ([x, ∞)) and υ -is the measure on (-∞, -m] with density -xΠ((-∞, x]).

3. 4 .

 4 Main arguments of the proof of Theorem 3.3.1 and motivation for the choice π α 83 Example 3.4.1 (Example page 5 in Adamczak (

  Starting from (3.45) with B defined by Equation (3.44), we write

Steps 1 to 4

 4 The proofs of Step 1 and Step 2 are strictly the same as in the Gaussian case. More precisely Proof of Step 1:

2 3

 2 .5.3.6 Proof of Lemma 3.5.10

108Chapter 3 .

 3 Risk upper bounds for RKHS ridge group sparse estimator in the regression model with non-Gaussian and non-bounded error Appendix 3.A Proofs of Section 3.4.2 3.A.1 Proof of Remark 3.4.1

2 3

 2 .B.2 Proof of Remark 3.4.3 If α = 2, according to the Laplace transform of the Gaussian function we have

  Find ρ that minimizes y(ρ) on the interval [ρ/10, ρ] Find ρ that minimizes y(ρ) on the interval [ρ, ρ × 10] Set ρ ← ρ × 10 and calculate y(ρ) Find ρ that minimizes y(ρ) on the interval [ρ/10, ρ] 23:end if 24: end if 25: calculate θ v = θ( ρ)

2 .Y

 2 crit lastIter -crit lastIter-1 crit lastIter-1 . RelDiffPar Scalar, value of the second convergence criterion at the last iteration, θ lastIter -θ lastIter-1 θ lastIter-1 Contents Vector of response observations of size n. matZ List of vMax components. Each component includes the eigenvalues and eigenvectors of the positive definite Gram matrices K v , v = 1, ...,vMax. It should have the same format as the output "kv" of the function calc_Kv.

  .Van de Geer, S. and Buhlmann, P. (2008) The group lasso for logistic regression,Eidgenossische Technische Hochschule, Zurich, Switzerland. J. R. Statist. Soc. B (2008) 70, Part 1, pp. 53-71. for (a in 1:d) F <-F*(abs(4*X[,a]-2)+c[a])/(1+c[a]) epsilon <-rnorm(n,0,1);sigma <-0.2 Y <-F + sigma*epsilon Dmax <-3 kernel <-"matern" Kv <-calc_Kv(X, kernel, Dmax, TRUE,TRUE) matZ <-Kv$kv mumax <-mu_max(Y, matZ) mumax A.4 pen_MetMod function pen_MetMod Function to fit a solution of the RKHS ridge group sparse optimization problem.

  .Van de Geer, S. and Buhlmann, P. (2008) The group lasso for logistic regression,Eidgenossische Technische Hochschule, Zurich, Switzerland. J. R.Statist. Soc. B (2008) 70, Part 1, pp. 53-71.

  Mots clés: méta-modèle, des espaces de Hilbert à noyau auto-reproduisant, indice de Sobol, régression non-paramétrique, critère des moindres carrés pénalisé, majoration du risque Résumé: Ce travail porte sur le problème de l'estimation d'un méta-modèle d'un modèle complexe, noté m. Le modèle m dépend de d variables d'entrée qui sont indépendantes et ont une loi connue. Le méta-modèle, noté f * , approche la décomposition de Hoeffding de m et permet d'estimer ses indices de Sobol.

  Chapter 1. Introduction R1 Soit f la fonction dans F pour laquelle l'infimum du membre de droite de l'inégalité (1.22) est réalisé. Le terme mf 2 n est le terme de biais habituel. Il quantifie à la fois les propriétés d'approximation du RKHS H et le compromis biais-variance. R2 Ce résultat est similaire à celui obtenu dans le cas où ε est gaussienne mais avec l'hypothèse supplémentaire (1.21). Cette hypothèse permet d'obtenir la même vitesse de convergence pour l'estimateur RKHS ridge group sparse que dans le cas où ε est gaussienne (voir Huet and Taupin (2017)). Cependant, elle implique certaines restrictions sur la régularité du RKHS H. En effet, comme pour tout v ∈ P, λ n,v ≥ ν n,v (voir l'équation (1.19)), il s'ensuit que v∈S f ν 2 n,v ≤ C 3 n 2β-1 , ce qui implique certaines restrictions sur la régularité du RKHS: si β est petit, ce qui sera le cas si α est grand, alors le RKHS devra être très régulier. R3 Par l'équation (1.19), on a aussi que pour tout v ∈ P, λ n,v ≥ d/n. Cette hypothèse permet de contrôler la probabilité de |P| événements (voir l'équation (3.48) du chapitre 3), où log(|P|) est d'ordre d.

1.22)

avec une probabilité supérieure à 1 -η.

Commentons ce résultat 1:

  la décomposition des fonctions dans H doit être limitée aux interactions d'un ordre limité, de sorte que le nombre d'éléments dans le méta-modèle estimé soit d'un ordre inférieur à d r pour un petit r, disons r = 2 par exemple. Dans ce cas, le cardinal de P sera donc inférieur à d 2 . Comme indiqué dans la remarque R3, l'hypothèse λ n,v ≥ d/n est nécessaire pour contrôler la valeur log(|P|), qui sera désormais inférieur à 2 log(d). Par conséquent, la valeur d dans la définition de λ n,v (voir l'équation (1.19)) ainsi que le terme d|S f |/n dans l'infimum ci-dessus seront remplacés par 2 log(d) et 2 log(d)|S f |/n, respectivement.

	1.2.4 Travaux antérieurs

H, c'est-à-dire |S f | est petit, et lorsque d est petit devant n. Lorsque d est grand, 1.2. Résumé du chapitre 3 Plusieurs auteurs ont étudié les propriétés théoriques d'estimateurs similaires à l'estimateur RKHS ridge group sparse. Rappelons brièvement leur cadre de travail et leurs résultats. Meier et al. (2009) ont considéré un estimateur similaire à l'estimateur RKHS ridge group sparse. Au lieu d'ajouter deux pénalités distinctes de sparsité et de régularité, ils combinent les deux termes en une seule pénalité de sparsité et de régularité. Ils considèrent un modèle de régression où les variables X 1 , ..., X d sont contrôlées (non aléatoires) et où l'erreur ε est de distribution sous-gaussienne. Ils ont établi des majorations du risque empirique pour l'estimation de m sur l'ensemble des fonctions additives univariées. Par la suite, Raskutti et al. (2012) ont montré (dans la section 3.4. de leur article) que la vitesse de convergence de cet estimateur est sous-optimale. Koltchinskii and Yuan (2010) ont considéré un estimateur de type ridge group sparse défini sur un ensemble de fonctions additives dont chaque terme appartient à un espace RKHS. Ils ne supposent pas que les variables d'entrées X 1 , ..., X d sont indépendantes, ni l'orthogonalité entre les espaces RKHS. Par contre, ils introduisent des hypothèses liées au degré de dépendance des RKHS, ce qui assurant ainsi une quasi orthogonalité entre ces espaces. Sous l'hypothèse où sup f ∈H sup X∈X |f (X)| est borné indépendamment de la dimension d , ils ont établi des majorations de l'excès du risque en supposant que la fonction m a une représentation sparse. Leurs résultats sont valables pour une grande classe de fonctions de perte, appelée pertes de type quadratique qui doivent satisfaire des conditions de bornitude sur le support de la variable de sortie Y . La section 2.1. de leur article donne plusieurs exemples du cadre d'application de leurs résultats. Il convient de noter que la fonction de perte quadratique dans le cas où Y n'est pas bornée ne satisfait pas les conditions dans Koltchinskii and Yuan (2010). Les preuves de leurs résultats reposent sur les résultats établis pour la symétrisation et les inégalités de concentration pour les processus de Rademacher et sur les bornes exponentielles de type Bernstein.

  ont étudié les propriétés théoriques de l'estimateur RKHS ridge group sparse, dans le modèle de régression gaussienne. Elles ont établi les majorations du risque empirique L 2 et du risque quadratique de l'estimateur RKHS ridge group sparse, c'est-à-dire des bornes supérieures par rapport à la norme L 2 et à la norme empirique L 2 pour la distance entre la fonction m et son estimation dans le RKHS H. Raskutti et al. (2012) et Huet and Taupin (2017) ne supposent pas que la quantité sup f ∈H sup X∈X |f (X)| est borné. Par contre, ils considèrent l'hypothèse H1 où pour tout v ∈ P, sup Xv |f v (X v )| est borné. Les preuves de leurs résultats reposent sur les méthodes probabilistes des processus empiriques gaussiens telles que les inégalités de concentration et la minoration de Sudakov

  ensuite, pour la grille de valeurs de µ et γ, une suite d'estimateurs est calculée.

	Chaque estimateur associé à la paire (µ, γ) dans la grille de valeurs de µ et
	γ, noté par f (µ,γ) , est la solution du problème d'optimisation de RKHS ridge group sparse ou du problème d'optimisation de RKHS group lasso si γ = 0.
	enfin, les estimateurs f (µ,γ) sont évalués à l'aide d'un ensemble de données de test, {(Y test i , X test i )} n test i=1 .
	L'erreur de prédiction associée à l'estimateur f (µ,γ) est calculée par,

La présentation habituelle du noyau brownien est la suivante:

  Le paramètre de régularisation γ est fixé à zéro. Une valeur de µ pour laquelle le nombre de groupes dans la solution du problème du RKHS group lasso est égal à qmax est calculée. Cette valeur sera notée par µ qmax . Ensuite, l'algorithme du RKHS ridge group sparse est implémenté pour une grille de valeurs de γ = 0 et la valeur de µ qmax .

	L'estimateur ayant la meilleure qualité de prédiction:
	Dans ce cas, le meilleur estimateur est calculé en utilisant la procédure décrite
	à la section 1.3.3.1.
	L'estimateur avec pour maximum qmax groupes actifs:
	Cette procédure est implémentée dans le package RKHSMetaMod dans la
	fonction RKHSMetMod_qmax.
	1.3.4.1 Calcul des matrices de Gram
	Les noyaux disponibles dans le package RKHSMetaMod sont: noyau linéaire, noyau
	quadratique, noyau brownien, noyau matérn et noyau gaussien. Le choix du noyau,
	par l'utilisateur, détermine l'espace d'approximation fonctionnelle. Pour un noyau
	choisi, l'algorithme de calcul des matrices de Gram K v , v ∈ P dans le package
	RKHSMetaMod est implémenté dans la fonction calc_Kv, et est basé sur trois
	points essentiels:
	Modifier le noyau choisi:
	Afin de satisfaire les conditions de construction du RKHS H décrites dans
	la section 1.1.4.2, ces noyaux sont modifiés selon l'équation (1.8). Ci-dessous
	l'exemple du noyau brownien.
	Exemple 1.3.1
	Le package RKHSMetaMod met en oeuvre deux algorithmes d'optimisation: le
	RKHS ridge group sparse et le RKHS group lasso. Ces algorithmes reposent sur
	les matrices de Gram K v , v ∈ P qui doivent être définies positives. Ainsi, la pre-
	mière étape essentielle du package RKHSMetaMod consiste à calculer ces matrices
	et à s'assurer qu'elles sont définies positives.
	La deuxième étape consiste à calculer l'estimateur f . Dans le package RKHSMeta-
	Mod, deux objectifs différents basés sur des procédures différentes sont considérés
	afin de calculer cet estimateur:

  Afin de résoudre le problème d'optimisation RKHS ridge group sparse, un algorithme adapté de block coordinate descent est proposé. Cet algorithme fournit deux étapes:

	Chapter 1. Introduction
	bornée. Les majorations du risque empirique L 2 et du risque quadratique de cet
	estimateur sont fournies.
	Étape 1 Initialiser les paramètres d'entrées par les solutions de l'algorithme RKHS
	group lasso pour chaque valeur du paramètre de régularisation µ et exécuter
	l'algorithme RKHS ridge group sparse via le support actif des solutions RKHS
	group lasso jusqu'à ce qu'il atteigne la convergence.
	Cette étape est prévue afin de diminuer le temps de calcul.
	Étape 2 Réinitialiser les paramètres d'entrées avec les solutions obtenues à l'Étape 1 et
	implémenter l'algorithme RKHS ridge group sparse à travers tous les groupes
	de P jusqu'à ce qu'il atteigne la convergence.
	Cette deuxième étape permet de vérifier qu'aucun groupe ne manque dans la
	sortie de l'Étape 1.
	L'algorithme adapté de block coordinate descent pour résoudre le problème d'optimisation
	RKHS ridge group sparse est implémenté dans le package RKHSMetaMod, dans la
	fonction pen_MetMod.
	1.4 Résumé et perspectives
	Les travaux présentés dans cette thèse portent sur le problème de l'estimation d'un
	méta-modèle qui approche la décomposition de Hoeffding d'un modèle complexe,
	noté m. Le modèle m dépend de d variables d'entrée X 1 , ..., X d qui sont indépen-
	dantes et ont une loi connue. Le méta-modèle appartient à un RKHS H, qui est
	construit de telle manière que la décomposition additive de toute fonction f dans
	H est la décomposition de Hoeffding de f (Durrande et al. (2013)). L'estimateur
	du méta-modèle, noté f , minimise un critère des moindres carrés pénalisé par une
	fonction de pénalité qui est la somme de la norme de Hilbert et de la norme em-
	pirique L 2 . Cette procédure, appelée RKHS ridge group sparse, permet à la fois de
	sélectionner et d'estimer les termes importants de la décomposition de Hoeffding du
	méta-modèle, et donc de sélectionner les indices de Sobol non-nuls et de les estimer
	(Huet and Taupin (2017)).
	La première partie de ce travail est dédiée à l'étude des propriétés théoriques de
	l'estimateur f d'un modèle de régression où l'erreur ε est non-gaussienne et non-

  [START_REF] Guennebaud | Eigen v[END_REF]) and GSL[START_REF] Galassi | Gnu scientific library reference manual[END_REF]) via RcppEigen[START_REF] Bates | Fast and elegant numerical linear algebra using the RcppEigen package[END_REF]) and RcppGSL[START_REF] Eddelbuettel | RcppGSL: 'Rcpp' Integration for 'GNU GSL' Vectors and Matrices[END_REF]) packages, are used in the RKHSMetaMod package. For different examples of usage of RcppEigen and RcppGSl functions see the by solving RKHS ridge group sparse optimization problem and d, indicates the maximum order of interactions considered in the RKHS meta-model: Dmax= 1 is used to consider only the main effects, Dmax= 2 to include the main effects and the second-order interactions, and so on. gamma Vector of non-negative scalars, values of the tuning parameter γ in decreasing order. If γ = 0 the function solves the RKHS group lasso optimization problem and for γ > 0 it solves the RKHS ridge group sparse optimization problem.

	Input parameter Description
	Y	Vector of the response observations of size n.
		Matrix of the input observations with n rows and d columns.
	X	Rows correspond to the observations and columns corre-
		spond to the variables.
	kernel	Character, indicates the type of the kernel (see Table 4.1) chosen to construct the RKHS H.
		Integer, between 1
	Dmax

frc

Vector of positive scalars. Each element of the vector sets a value to the tuning parameter µ: µ = µ max /( √ n × frc). The value µ max (see Equation (4.16)) is calculated inside the program. verbose Logical. Set as TRUE to print: the group v for which the correction of the Gram matrix K v is done (see Section 4.3.1), and for each pair of the tuning parameters (µ, γ): the number of current iteration, active groups and convergence criterion. It is set as FALSE by default. Table 4.2: List of the input arguments of the RKHSMetMod function.

  Table 4.9: Example 4.5.3: Obtained prediction errors in step 2. For each v, let S v be the true value of the Sobol indices displayed in Table 4.7 and S v be the estimated empirical Sobol indices. Then In Table 4.10 the estimated empirical Sobol indices, their sum, and the value of RE are displayed. Table 4.10: Example 4.5.3: The estimated empirical Sobol indices ×100 greater than 10 -2 . The last two columns show v S v and RE, respectively.The RE for each group v is smaller than 1.64%, so the estimated Sobol indices in this example are very close to the true values of the Sobol indices displayed in the Table4.7. In this example the significant values of the Sobol indices for interactions of order two are obtained.

			µ		0.081/	√ n 0.041/	√ n 0.020/	√ n
			γ = 0.2	0.153		0.131	0.119
			γ = 0.1	0.098		0.079	0.072
			γ = 0.01	0.065		0.054	0.053
			γ = 0.005 0.064		0.054	0.054
					RE =	v	| S v -S v | S v	.	(4.18)
	v	{1}	{2}	{3}	{1, 2} {1, 3} {2, 3} {1, 2, 3} sum	RE
	S v 42.91 25.50 20.81 4.40		3.84	2.13	0.00	99.60 1.64
	Example 4.5.4 Dealing with larger datasets:
							])

  .,vMax and their associated group names. It should have the same format as the output of the function calc_Kv (see details). gamma Vector of positive scalars. Values of the penalty parameter γ in decreasing order. mu Vector of positive scalars. Values of the penalty parameter µ in decreasing order. resg List of initial parameters, includes the RKHSgrplasso objects for each value of the penalty parameter µ. gama_v Scalar zero or vector of vMax positive scalars, considered as weights for the ridge penalty. Set to zero, to consider no weights, i.e. all weights equal to 1. mu_v Scalar zero or a vector with vMax scalars, considered as weigths of sparse group penalty. Set to zero, to consider no weights, i.e. all weights equal to 1. maxIter Integer, shows the maximum number of loops through initial active groups at the first step and maximum number of loops through all groups at the second step. Set as 1000 by default. verbose Logical, if TRUE, for each pair of penalty parameters (µ, γ) it prints: the number of current iteration, active groups and convergence criteria. Set as FALSE by default. calcStwo Logical, if TRUE, the program does a second step after convergence: the algorithm is done over all groups by taking the estimated parameters at the first step as initial values. Set as FALSE by default. Details Input Kv should contain the eigenvalues and eigenvectors of positive definite Gram Contents matrices K v . It is necessary to set input "correction" in the function calc_Kv equal to "TRUE".

  Scalar equals to Yf 0v K v θ v 2 . crit Scalar indicates the value of the penalized criterion. Vector of size vMax, coefficients of the group sparse penalty norm, nµ×mu_v. iter List of three components if calcStwo=TRUE (two components if calcStwo=FALSE): maxIter, number of iterations until convergence is reached at first step and the number of iterations until convergence is reached at second step (maxIter, and the number of iterations until convergence is reached at first step). Kv List, includes the eigenvalues and eigenvectors of the positive definite Gram matrices K v , v = 1, ...,vMax and their associated group names. It should have the same format as the output of the function calc_Kv (see details). mu Positive scalar, value of the penalty parameter µ g in the RKHS group lasso problem. maxIter Integer, shows the maximum number of loops through all groups. Set as 1000 by default. verbose Logical, if TRUE, prints: the number of current iteration, active groups and convergence criteria. Set as FALSE by default. Details Input Kv should contain the eigenvalues and eigenvectors of positive definite Gram matrices K v . It is necessary to set input "correction" in the function calc_Kv equal to "TRUE".

			Contents
	Y	Vector of response observations of size n.
	gamma.v		Vector of size vMax, coefficients of the ridge penalty norm, √ nγ×gama_v.
	mu.v
	convergence	TRUE or FALSE. Indicates whether the algorithm has converged or not.
			List of two components if calcStwo=TRUE (one component if
	RelDiffCrit	calcStwo=FALSE): value of convergence criterion at the last it-eration of each step, θ lastIter -θ lastIter-1 2 . θ lastIter-1 List of two components if calcStwo=TRUE (one component if
	RelDiffPar	calcStwo=FALSE): value of convergence criterion at the last it-
			eration,
	Note:

crit lastIter -crit lastIter-1 crit lastIter-1 of each step.

To the best of our knowledge, there is no other package available that implements the RKHS ridge group sparse procedure. The RKHSMetaMod package is dedicated to the meta-model estimation on the RKHS H. The convex optimization algorithms used in this package are adapted to take into account the problem of high dimensionality in this context. This package is available from the Comprehensive R Archive Network (CRAN) at https://cran.r-project.org/web/packages/ RKHSMetaMod/.

The organization of this Chapter is as follows: In Section 4.2, the estimation method is described. In Section 4.3, the algorithms used in the RKHSMetaMod package to obtain the RKHS meta-model are detailed. In Section 4.4, an overview of the package functions as well as a brief documentation of them are given. In Section 4.5, a simulation study to validate the performances of the RKHSMetaMod package functions is given.

Estimation method

In Section 4.2.1, the RKHS ridge group sparse and the RKHS group lasso procedures are presented. In Section 4.2.2, the method of [START_REF] Durrande | Anova kernels and rkhs of zero mean functions for model-based sensitivity analysis[END_REF] to construct the RKHS H is recalled. The strategy of choosing the tuning parameters in the RKHS ridge group sparse algorithm is detailed in Section 4. 2.3,and in Section 4.2.4, the calculation of the empirical Sobol indices of the RKHS meta-model is described.

RKHS ridge group sparse and RKHS group lasso procedures

Let denote by n, the number of observations. The dataset consists of a vector of n observations Y = (Y 1 , ..., Y n ), and a n × d matrix of features X with components, (X ai , i = 1, ..., n, a = 1, ..., d) ∈ R n×d .

For some tuning parameters γ v , µ v , v ∈ P, the RKHS ridge group sparse criterion is defined by,

where X v represents the matrix of variables corresponding to the v-th group,

and where f v n is the empirical L 2 -norm of f v defined by the sample {X vi } n i=1 as,

The penalty function in the criterion (4.10) is the sum of the Hilbert norm and the empirical norm, which allows to select few terms in the additive decomposition

Choice of the tuning parameters

While dealing with an optimization problem, one of the essential steps is to choose appropriately the tuning parameters. To do so, first, a grid of values of the tuning parameters µ and γ is chosen.

Let µ max be the smallest value of µ g (see Equation (4.14)), such that the solution to the minimization of the RKHS group lasso problem for all v ∈ P is θ v = 0. We have,

In order to set up the grid of values of µ, one may find µ max , and then a grid of values of µ is defined as follows:

, l ∈ {1, ..., l max }.

The grid of values of γ is chosen by the user.

next, for the grid of values of µ and γ a sequence of estimators is calculated. Each estimator associated with the pair (µ, γ) in the grid of values of µ and γ, denoted by f (µ,γ) , is the solution of the RKHS ridge group sparse optimization problem or the RKHS group lasso optimization problem if γ = 0.

finally, the obtained estimators f (µ,γ) are evaluated using a testing dataset,

The prediction error associated with the estimator f (µ,γ) is calculated by,

where for S f being the support of the estimator f (µ,γ) ,

The pair ( µ, γ) with the smallest value of the prediction error is chosen, and the estimator f ( µ, γ) is considered as the best estimator of the function m, in terms of the prediction error.

In the RKHSMetaMod package, the algorithm to calculate a sequence of the RKHS meta-models, the value of µ max , and the prediction error are implemented as RKHSMetMod, mu_max, and PredErr functions, respectively. These functions are described in Section 4.4,and illustrated in Example 4.5.1,Example 4.5.3,and Examples 4.5.1,4.5.3,4.5.4,respectively. work by [START_REF] Eddelbuettel | Seamless R and C++ Integration with Rcpp[END_REF].

The complete documentation of RKHSMetaMod package is available at https: //cran.r-project.org/web/packages/RKHSMetaMod/RKHSMetaMod.pdf. Here, a brief documentation of some of its main and companion functions is presented in Sections 4.4.1 and 4.4.2, respectively.

Main RKHSMetaMod functions

Let us begin by introducing some notations. For a given Dmax∈ N, let P Dmax be the set of parts of {1, ..., d} with dimensions 1 to Dmax. The cardinal of P Dmax is denoted by vMax,

RKHSMetMod function: For a given value of Dmax and a chosen kernel (see Table 4.1), this function calculates the Gram matrices K v , v ∈ P Dmax , and produces a sequence of estimators f associated with a given grid of values of tuning parameters µ, γ, i.e. the solutions to the RKHS ridge group sparse (if γ = 0) or the RKHS group lasso problem (if γ = 0). Table 4.2 gives a summary of all input arguments of the RKHSMetMod function and default values for non-mandatory arguments.

The RKHSMetMod function returns a list of l components, with l equals to the number of pairs of the tuning parameters (µ, γ), i.e. l = |gamma| × |frc|. Each component of the list is a list of three components "mu", "gamma" and "Meta-Model": mu: value of the tuning parameter µ if γ > 0, or µ g = √ n × µ if γ = 0.

gamma: value of the tuning parameter γ.

Meta-Model: an RKHS ridge group sparse or RKHS group lasso object associated with the tuning parameters mu and gamma. Table 4.3 gives a summary of all arguments of the output "Meta-Model" of RKHSMetMod function.

RKHSMetMod_qmax function: For a given value of Dmax and a chosen kernel (see Table 4.1), this function calculates the Gram matrices K v , v ∈ P Dmax , determines µ, denoted µ qmax , for which the number of active groups in the RKHS group lasso solution is equal to qmax, and produces a sequence of estimators f associated with the tuning parameter µ qmax and a grid of values of the tuning parameter γ. All the estimators f produced by this function have at most qmax active groups in their support. This function has the following input arguments:

-Y , X, kernel, Dmax, gamma, verbose (see Table 4.2).

qmax: integer, the maximum number of active groups in the obtained solution.

Output parameter Description intercept

Scalar, estimated value of intercept.

teta Matrix with vMax rows and n columns. Each row of the matrix is the estimated vector θ v for v = 1, ...,vMax.

fit.v

Matrix with n rows and vMax columns. Each row of the matrix is the estimated value of

Vector of size n, indicates the estimator of m.

Norm.n

Vector of size vMax, estimated values for the empirical L values of µ ∈ [µ min , µ max ):

where | S f (µ 1 ) Group Lasso | is the number of active groups in the solution of the RKHS group lasso problem, Algorithm 1, associated with µ 1 .

If Num> 1, the path to cover the interval [µ min , µ max ) is detailed in Algorithm 3.

The RKHSMetMod_qmax function returns a list of three components "mus", "qs", and "MetaModel": In each example, two independent datasets are generated: (X, Y ) to estimate the meta-models, and (XT, Y T ) to estimate the prediction errors. The design matrices X and XT are the Latin Hypercube Samples of the input variables that are generated using maximinLHS function of the package lhs available at https: //CRAN.R-project.org/package=lhs: R> library(lhs) R> X <-maximinLHS(n,d) R> XT <-maximinLHS(n,d) The response variables Y and Y T are calculated as Y = m(X) + σε and Y T = m(XT ) + σε T , where σ = 0.2, and ε, ε T are distributed independently according to the centered Gaussian distribution with variance equals to one: 2,0.6,0.8,100,100,100,100,100,100,100)[1:d] R> sigma <-0.2 R> g=1;for (i in 1:d) g=g*(abs(4*X[,i]-2)+a [i])/(1+a [i]) R> epsilon <-rnorm(n,0,1) R> Y <-g + sigma*epsilon R> gT=1;for (i in 1:d) gT=gT*(abs(4*XT[,i]-2)+a [i])/(1+a [i]) R> epsilonT <-rnorm(n,0,1) R> YT <-gT + sigma*epsilonT Example 4.5.1 RKHS meta-model estimation using RKHSMetMod function:

In this example, three datasets of n points maximinLHS over [0, 1] d are generated with n ∈ {50, 100, 200} and d = 5, and a grid of five values for each of the tuning parameters µ and γ is considered as follows:

, γ (1:5) = (0.2, 0.1, 0.01, 0.005, 0).

For each dataset, the experiment is repeated N r = 50 times. At each repetition, the RKHS meta-models associated with the pair of the tuning parameters (µ, γ) are estimated using the RKHSMetMod function:

R> kernel <-"matern" # kernel <-"brownian" # kernel <-"gaussian" R> Dmax <-3 R> gamma <-c(0. ), i = 1, ..., N , with N = 1000. The global prediction error is calculated as follows:

Let f r (.) be the best RKHS meta-model obtained in the repetition r, r = 1, ..., N r , then

The values of GP E obtained for different kernels and values of n are given in andd = 5. Each line of the table, from up to down, gives the value of GPE obtained for each dataset associated with the "matern", "brownian" and "gaussian" kernels, respectively. GP E are obtained when using the "matern" kernel.

In order to sum up the behaviour of the procedure for estimating the Sobol indices, the mean square error (MSE) is estimated as follows: Let

where for each group v, S v denotes the true values of the Sobol indices, and for S v,r being the empirical Sobol indices of the best RKHS meta-model in repetition r, S v,. denotes the mean of the empirical Sobol indices of the best RKHS meta-models through all repetitions:

Then, appears that the estimated Sobol indices are close to the true ones, nevertheless they are over estimated for the main effects, i.e. groups v ∈ {{1}, {2}, {3}}, and under estimated for the interactions of order two and three, i.e. groups v ∈ {{1, 2}, {1, 3}, {2, 3}, {1, 2, 3}}.

Note that, the strategy of choosing the tuning parameters is based on the minimization of the prediction error of the estimated meta-model, which may not minimize the error of estimating the Sobol indices.

Taking into account the results obtained for this Example 4.5.1, the calculations in the rest of the examples is done using only the "matern" kernel. model. So, one may be interested in estimating the function m(X) (see Equation (4.17)) by a meta-model that includes at most three active groups (the main effects only). In order to calculate the RKHS meta-models that contain at most three active groups the RKHSMetMod_qmax function is used with, "gamma"= (0.2, 0.1, 0.01, 0.005, 0), "rat"= 100: the minimum value of µ considered in the algorithm is then

,

"Num"= 10: the maximum number of values of µ ∈ [µ min , µ max ) to be evaluated is equal to twelve (see Algorithm 3).

R> kernel <-"matern" R> Dmax <-3 R> gamma <-c(0.2,0.1,0.01,0.005,0) R> qmax <-3;Num <-10;rat <-100 R> res <-RKHSMetMod_qmax(Y,X,kernel,Dmax,gamma,qmax,Num,rat,FALSE)

The RKHS meta-models are estimated for the obtained value of µ qmax and different values of the tuning parameter γ:

R> for(i in 1:length(gamma)){ print(paste("In meta model ",i)) print(paste("the value of mu is: ",res$MetaModel[ [i]]$mu, "and the value of gamma is: ",res$MetaModel[ [i]]$gamma)) print("the active groups are: ") print(res$MetaModel[ [i]]$'Meta-Model'$Nsupp) } "In meta model 1" "the value of mu is: 0.093 and the value of gamma is: 0.2" "the active groups are: " "v1." "v2." "v3." "In meta model 2" "the value of mu is: 0.093 and the value of gamma is: 0.1" "the active groups are: " "v1." "v2." "v3." "In meta model 3" "the value of mu is: 0.093 and the value of gamma is: 0.01" "the active groups are: " "v1." "v2." "v3." "In meta model 4" "the value of mu is: 0.093 and the value of gamma is: 0.005" "the active groups are: " "v1." "v2." "v3." "In meta model 5" } Output res_g contains nine RKHS meta-models and they are evaluated using a testing dataset: Let us go back to the implementation of the example and apply this step 2:

The grid of values of µ in this step is,

The RKHS meta-models associated with this grid of values of µ are gathered in a new list "resgnew". Set γ (1:4) = (0.2, 0.1, 0.01, 0.005), and calculate the RKHS meta-models for this new grid of values of (µ, γ) using pen_MetMod function:

R> res <-pen_MetMod(Y,Kv,gamma,mu,resgnew,0,0)

The output "res" is a list of twelve RKHS meta-models. These meta-models are evaluated using a new dataset, and their prediction errors are displayed in Table 4.9.

The minimum prediction error is associated with the pair (0.020/ √ n, 0.01),

and the best RKHS meta-model is then f (0.020/ √ n,0.01) . The performances of this procedure for estimating the Sobol indices is evaluated using the relative error (RE) defined as follows: The RKHS meta-models associated with the pair of values (µ i , γ), i = 1, 2 are estimated using the RKHSMetMod function:

Chapter 4. Estimate the Hoeffding decomposition of a complex model by solving RKHS ridge group sparse optimization problem R> kernel <-"matern" R> Dmax <-3 R> gamma <-c(0.01) R> frc <-c(128,256) R> res <-RKHSMetMod(Y,X,kernel,Dmax,gamma,frc,FALSE)

The prediction error and the empirical Sobol indices are then calculated for the obtained meta-models using the functions PredErr and SI_emp:

) and γ = 0.01.

estimated for n equals to 5000, so as expected, the estimation of the Sobol indices is better for larger values of n.

In Figure 4.2 the result of the prediction quality and the Sobol indices for dataset with n equals to 5000, d equals to 10, and (µ 2 , γ) are displayed. The line y = x in red crosses the cloud of points as long as the values of the g-function are smaller than three. When the values of the g-function are greater than three, the estimator f tends to under estimate the g-function.

Summary and discussion

An R package, called RKHSMetaMod, that estimates a meta-model of a complex model m, is proposed. This meta-model belongs to a reproducing kernel Hilbert space constructed as a direct sum of Hilbert spaces [START_REF] Durrande | Anova kernels and rkhs of zero mean functions for model-based sensitivity analysis[END_REF]). The estimation of the meta-model is carried out via a penalized least squares minimization allowing both to select and estimate the terms in the Hoeffding decomposition, and therefore, to select the Sobol indices that are non-zero and estimate them (Huet and Taupin (2017)). This procedure makes it possible to estimate Sobol indices of high order, a point known to be difficult in practice.

Using the convex optimization tools, RKHSMetaMod package implements two optimization algorithms: the minimization of the RKHS ridge group sparse criterion (4.12) and the RKHS group lasso criterion (4.13). Both of these algorithms rely on the Gram matrices K v , v ∈ P and their positive definiteness.

Currently, the package considers only uniformly distributed input variables. If one is interested by another distribution of the input variables, it suffices to modify the calculation of the kernels k 0a , a = 1, ..., d (see Equation (4.15)) in the function calc_Kv of this package (see Remark 4.3.1).

The available kernels in the RKHSMetaMod package are: linear kernel, quadratic kernel, brownian kernel, matern kernel and gaussian kernel (see Table 4.1). Regarding to the problem under study, one may consider another kernel and add it easily to the list of the kernels in the calc_Kv function. Indeed, the choice of different Appendix 4.A More technical details Preliminary 4.A.1 For F (x) = Ax , where A is a symmetric matrix that not depends on x, the sub-differential of F at point x, denoted by ∂F (x), is defined as follows:

Preliminary 4.A.2 Let F : R n → R be a convex function. we have the following first order optimality condition:

x ∈ arg min

This follows from the fact that F (y) ≥ F ( x) + 0, yx for all y ∈ R n in both cases [START_REF] Giraud | Introduction to High-Dimensional Statistics, Chapman & Hall/CRC Monographs on Statistics & Applied Probability[END_REF]).

4.A.1 RKHS group lasso algorithm

We consider the minimization of the RKHS group lasso criterion given by,

We begin with the constant term f 0 . The ordinary first derivative of the function C g (f 0 , θ) at f 0 is equal to:

and therefore,

where

Since C g (f 0 , θ) is convex and separable, we use a block coordinate descent algorithm, group v by group v. In the following, we fix a group v, and we find the minimizer of C g (f 0 , θ) with respect to θ v for given values of f 0 and θ w , w = v. Set

Chapter 4. Estimate the Hoeffding decomposition of a complex model by solving RKHS ridge group sparse optimization problem

where

We aim to minimize C g,v (f 0 , θ v ) with respect to θ v . Let ∂C g,v be the sub-differential of C g,v (f 0 , θ v ) with respect to θ v :

The first order optimality condition (see Preliminary (4.A.2)) ensures the existence of

Using the sub-differential definition (see Preliminary 4.A.1) we obtain,

and,

Let θ v be the minimizer of C g,v . The sub-differential equations above give the two following cases: Case 1. If θ v = 0 then there exists t v ∈ R n such that K -1/2 v t v ≤ 1 and it fulfils Equation (4.20):

So, the necessary and sufficient condition for which the solution θ v = 0 is the optimal one is:

v θ v and it fulfils Equation (4.20):

We obtain then,

Since θ v appears in both sides of the Equation (4.21), a numerical procedure is needed:

There exists a non-zero solution to Equation (4.21) if and only if there exists ρ > 0 such that

Then θ v = θ(ρ).

4.A. More technical details 151

Proof If there exists a non-zero solution to Equation (4.21), then K

and, for such ρ Equation (4.22) is satisfied. Conversely, if there exists ρ > 0 such that Equation (4.22) is satisfied, then

.

Therefore,

then y(ρ) = 0 has a unique solution, denoted ρ, which leads to calculate θ( ρ).

Proof For ρ = 0 we have y(0) = -√ nµ g < 0, since µ g > 0; and for ρ → +∞ we have y(ρ) > 0, since K

Moreover, we have

and,

Appendix A

Package 'RKHSMetaMod'

RKHSMetaMod-package

Produces a sequence of meta-models that are the solutions of the RKHS ridge group sparse or the RKHS group lasso optimization problems.

Description

Estimates a meta-model that approximates the Hoeffding decomposition of a complex model m by solving the ridge group sparse (or group lasso) optimization prob-Contents lem based on a Reproducing Kernel Hilbert Space (RKHS). The model m depends on d input variables X = (X 1 , ..., X d ) that are independent and uniformly distributed on [0, 1] d . This model m from R d to R may be a known model that can be calculated in all points of X, or it may be an unknown regression model defined as follows:

where the error ε is assumed to be centered with a finite variance, i.e. E(ε) = 0 and var(ε) < ∞.

Let P be the set of parts of {1, ..., d} with dimension 1 to d. The RKHS ridge group sparse criterion is defined by:

where for all v ∈ P, γ v and µ v are the vector of weights that should be chosen suitably, and K v are the Gram matrices associated with a chosen reproducing kernel.

The RKHS group lasso criterion is obtained by setting γ = 0 in the RKHS ridge group sparse criterion above. The RKHS group lasso penalty parameter is denoted by µ g = √ nµ.

For each pair of the penalty parameters (µ, γ) in the RKHS ridge group sparse criterion, one estimator, called RKHS meta-model, is calculated. For a given value Dmax∈ N, the RKHS meta-model f has an additive representation including the variables and interactions between them of order maximum equal to Dmax:

where f 0 is a constant, and P Dmax is the set of parts of {1, ..., d} with dimensions 1 to Dmax and cardinality equal to vMax:

For a given grid of values of the tuning parameters (µ, γ) a sequence of the RKHS meta-models are produced by minimizing the RKHS ridge group sparse criterion (if γ = 0) or the RKHS group lasso criterion (if γ = 0). These meta-models are evaluated using a testing dataset. That is, the prediction error is calculated for each RKHS meta-model, and the one with the minimum prediction error is the best estimator for the true model m. This package provides a function that estimates the empirical Sobol indices of the obtained RKHS meta-models. The estimators of the Sobol indices of m are deduced from the best RKHS meta-model.

See Also RKHSMetaMod

Examples d <-3 n <-50 library(lhs) X <-maximinLHS(n, d) c <-c(0.2,0.6,0.8) F <-1;for (a in 1:d) F <-F*(abs(4*X[,a]-2)+c[a])/(1+c[a]) epsilon <-rnorm(n,0,1);sigma <-0.2 Y <-F + sigma*epsilon Dmax <-3 kernel <-"matern" frc <-c(10,100) gamma <-c(.5,.01,.001,0) result <-RKHSMetMod(Y,X,kernel,Dmax,gamma,frc,FALSE)

A.1 calc_Kv function calc_Kv

Function to calculate the eigenvalues and eigenvectors of the Gram matrices K v , v ∈ P Dmax .

Description

For a given value of Dmax this function calculates the Gram matrices K v for v ∈ P Dmax , and returns their associated eigenvalues and eigenvectors. The calculated Gram matrices may be not positive definite. The option "correction" of this function allows to replace the matrices K v that are not positive definite by their Note: Note.

Author(s) Halaleh Kamari

References.

See Also calc_Kv, mu_max

) epsilon <-rnorm(n,0,1);sigma <-0.2 Y <-F + sigma*epsilon Dmax <-3 kernel <-"matern" Kv <-calc_Kv(X, kernel, Dmax, TRUE, TRUE) result <-grplasso_q(Y,Kv,5,100 ,Num=10) result$mu result$res$Nsupp

A.3 mu_max function mu_max

Function to find the maximal value of the penalty parameter in the RKHS group lasso optimization problem.

Description

Calculates the value of the penalty parameter in the RKHS group lasso optimization problem when the first penalized parameter group enters the model.

Usage mu_max(Y, matZ)

Arguments

For more details about the algorithm see Huet and Taupin (2017).

Author(s) Halaleh Kamari

Contents

A.5 PredErr function

PredErr Function to calculate the prediction error.

Description

Computes the prediction error by considering a testing dataset. res List, includes a squence of estimated meta-models for the learning dataset, using RKHS ridge group sparse or RKHS group lasso algorithm, associated with the penalty parameters mu and gamma. It should have the same format as the output of one of the functions: pen_MetMod, RKHSMetMod or RKHSMetMod_qmax.

Usage

kernel

Character, shows the type of the reproducing kernel: matern, brownian, gaussian, linear, quad. The same kernel should be chosen as the one used for the learning dataset. See function calc_Kv.

Dmax

Integer between 1 and d. The same Dmax should be chosen as the one used for learning dataset. See function calc_Kv. Details Details.

Value

Matrix of the prediction errors is returned. Each element of the matrix is the obtained prediction error associated with one RKHS meta-model in "res".

Note:

Note. ) sigma <-0.2 epsilon <-rnorm(n,0,1);Y <-F + sigma*epsilon epsilonT <-rnorm(nT,0,1);YT <-FT + sigma*epsilonT Dmax <-3 kernel <-"matern" frc <-c(10,100) gamma=c(.5,.01,.001) res <-RKHSMetMod(Y,X,kernel,Dmax,gamma,frc,FALSE) mu <-vector() l <-length(gamma) for(i in 1:length(frc))mu [i]=res[[(i-1)*l+1]]$mu error <-PredErr(X,XT, YT,mu,gamma, res, kernel,Dmax) error

Author(s) Halaleh Kamari

A.6 RKHSgrplasso function

RKHSgrplasso

Function to fit a solution of an RKHS group lasso optimization problem.

Description

For a given value of the tuning parameter µ g , this function fits the solution to the RKHS group lasso optimization problem.

Value

List of l components, with l equals to the number of pairs of the penalty parameters (µ, γ). Each component of the list is a list of 3 components "mu", "gamma" and "Meta-Model": mu

Positive scalar, penalty parameter µ associated with the estimated Meta-Model.

gamma Positive scalar, an element of the input vector gamma associated with the estimated Meta-Model.

Meta-Model

An RKHS ridge group sparse or RKHS group lasso object associated with the penalty parameters mu and gamma:

intercept Scalar, estimated value of intercept.

teta Matrix with vMax rows and n columns. Each row of the matrix is the estimated vector θ v for v = 1, ...,vMax.

fit.v

Matrix with n rows and vMax columns. Each row of the matrix is the estimated value of

fitted Vector of size n, indicates the estimator of m.

Norm.n

Vector of size vMax, estimated values for the ridge penalty norm.

Norm.H

Vector of size vMax, estimated values for the group sparse penalty norm.

supp Vector of active groups.

Nsupp

Vector of the names of the active groups.

crit Scalar indicates the value of the penalized criterion.

gamma.v

Vector of size vMax, coefficients of the ridge penalty norm, √ nγ×gama_v.

mu.v

Vector of size vMax, coefficients of the group sparse penalty norm, nµ×mu_v.

iter List of two components: maxIter, and the number of iterations until the convergence is achieved.

convergence TRUE or FALSE. Indicates whether the algorithm has converged or not.

RelDiffCrit

Scalar, value of the first convergence criterion at the last iteration,

RelDiffPar Scalar, value of the second convergence criterion at the last iteration, crit lastIter -crit lastIter-1 crit lastIter-1

.

Note: Note

Author(s) Halaleh Kamari

Contents

Examples d <-3 n <-50 library(lhs) X <-maximinLHS(n, d) c <-c(0.2,0.6,0.8) F <-1;for (a in 1:d) F <-F*(abs(4*X[,a]-2)+c[a])/(1+c[a]) epsilon <-rnorm(n,0,1);sigma <-0.2 Y <-F + sigma*epsilon Dmax <-3 kernel <-"matern" frc <-c(10,100) gamma <-c(.5,.01,.001,0) result <-RKHSMetMod(Y,X,kernel,Dmax,gamma,frc,FALSE) l <-length(result) for(i in 1:l)print(result[ [i]]$mu) for(i in 1:l)print(result[ [i]]$gamma) for(i in 1:l)print(result[ [i]]$'Meta-Model'$Nsupp)

A.8 RKHSMetMod_qmax function

RKHSMetMod_qmax

Function to produce a sequence of the RKHS meta-models, with at most qmax active groups in their support. These meta-models are the solutions of the RKHS ridge group sparse or the RKHS group lasso optimization problems.

Description

For a given value of Dmax and a chosen reproducing kernel, this function calculates the Gram matrices K v , v ∈ P Dmax , determines µ, denoted µ qmax , for which the number of active groups in the RKHS group lasso solution is equal to qmax, and produces a sequence of the RKHS meta-models associated with the tuning parameter µ qmax and a grid of values of the tuning parameter γ. All the RKHS meta-models produced by this function have at most qmax active groups in their support.

Usage

Value

List of three components "mus", "qs", and "MetaModel":

Contents mus

Vector, values of the evaluated penalty parameters µ in the RKHS group lasso algorithm until it achieves µ(qmax).

qs Vector, number of active groups associated with each element in mus.

MetaModel

List with the same length as the vector gamma. Each component of the list is a list of 3 components "mu", "gamma" and "Meta-Model": mu Scalar, the value µ(qmax).

gamma Positive scalar, element of the input vector gamma associated with the estimated Meta-Model.

Meta-Model

An RKHS ridge group sparse or RKHS group lasso object associated with the penalty parameters mu and gamma:

intercept Scalar, estimated value of intercept.

teta Matrix with vMax rows and n columns. Each row of the matrix is the estimated vector θ v for v = 1, ...,vMax.

fit.v

Matrix with n rows and vMax columns. Each row of the matrix is the estimated value of

fitted Vector of size n, indicates the estimator of m.

Norm.n

Vector of size vMax, estimated values for the ridge penalty norm.

Norm.H

Vector of size vMax, estimated values for the group sparse penalty norm.

supp Vector of active groups.

Nsupp

Vector of the names of the active groups.

SCR

Scalar equals to

crit Scalar indicates the value of the penalized criterion.

gamma.v

Vector of size vMax, coefficients of the ridge penalty norm, √ nγ×gama_v.

mu.v

Vector of size vMax, coefficients of the group sparse penalty norm, nµ×mu_v.

iter List of two components: maxIter, and the number of iterations until the convergence is achieved.

convergence TRUE or FALSE. Indicates whether the algorithm has converged or not.

RelDiffCrit

Scalar, value of the first convergence criterion at the last iteration,

RelDiffPar Scalar, value of the second convergence criterion at the last iteration, crit lastIter -crit lastIter-1 crit lastIter-1

.

Note: For the case γ = 0 the outputs "mu"= µ g and "Meta-Model" is the same as the one returned by the function RKHSgrplasso.

Author(s) Halaleh Kamari

See Also mu_max, RKHSgrplasso, pen_MetMod, grplasso_q Examples d <-3 n <-50 library(lhs) X <-maximinLHS(n, d) c <-c(0.2,0.6,0.8) F <-1;for (a in 1:d) F <-F*(abs(4*X[,a]-2)+c[a])/(1+c[a]) epsilon <-rnorm(n,0,1);sigma <-0.2 Y <-F + sigma*epsilon Dmax <-3 kernel <-"matern" gamma <-c(.5,.01,.001,0) Num <-10 rat <-100 qmax <-4 result <-RKHSMetMod_qmax(Y, X, kernel, Dmax, gamma, qmax, rat, Num,FALSE) names(result) result$mus result$qs l <-length(gamma) for(i in 1:l)print(result$MetaModel[ [i]]$mu) for(i in 1:l)print(result$MetaModel[ [i]]$gamma) for(i in 1:l)print(result$MetaModel[ [i]]$'Meta-Model'$Nsupp)

A.9 SI_emp function SI_emp Function to calculate the empirical Sobol indices.

Description

For each RKHS meta-model, this function calculates the empirical Sobol indices for all groups that are active in its support.

Usage

Value

If input ErrPred ="NULL", Vector of the empirical Sobol incdices for the metamodel with the minimum Prediction error is returned. If ErrPred="NULL", a list of the vectors is returned. Each vector is the obtained Sobol indices associated with one meta-model in "res".

Note:

Note.

Author(s) Halaleh Kamari

References.

See Also PredErr, pen_MetMod, RKHSMetMod, RKHSMetMod_qmax Examples d <-3 n <-50;nT <-50 library(lhs) X <-maximinLHS(n, d);XT <-maximinLHS(nT, d) c <-c(0.2,0.6,0.8) F <-1;for (a in 1:d) F <-F*(abs(4*X[,a]-2)+c[a])/(1+c[a]) FT <-1;for (a in 1:d) FT <-FT*(abs(4*XT[,a]-2)+c[a])/(1+c[a]) sigma <-0.2 epsilon <-rnorm(n,0,1);Y <-F + sigma*epsilon epsilonT <-rnorm(nT,0,1);YT <-FT + sigma*epsilonT Dmax <-3 kernel <-"matern"