Keywords: errors-in-variables, high dimensional estimation, Lasso, Slope, convex optimization, semiparametric model, inverse problems, nonparametric model, TMLE, variable importance measure, marginal structural model, statistical inference

Nə ̀ Məmtə

Tunù : Nə tə ndom nǔm jûncob ncàŋ Mbàtə ǹjòŋ nshʉαtə nǔm nə ̀ tαmtə nkǎnkὰn mbὰ kə -kαň ntʉ̂ mfì' ndʉb.

Màd zə ̀ ngαmnkʉ̌n cwɛ̌d nkʉ̂ndə cə̂n ngǔ'gham nì lα kwì'tə̌ zə bə ǹ nə ta nə ̀ diαǧ cûnkʉ ̀n yi ntʉ̂ nzə̌nzə̌ ngαmfà' mbα̂ nə cwìntə̌ ngòkɛd, Nə ̀ dʉ'ca'a, Nə ̀ nαb yaŋmbà', ngʉ̂ tsə

under the then common low dimensional framework, where the number of covariates is smaller than the size of the dataset, had to be extended. The literature now abounds with significant contributions in high dimensional settings. Following this path, the current thesis touches on the concept of variable importance that is, a methodology used to assess the significance of a variable. It is a focal point in today's era of big data. As an example, it is often used for prediction models in high dimensional settings to select the main predictors.

Our contributions can be divided in three parts.

In the first part of the thesis, we rely on semiparametric models for our analysis. We introduce a multivariate variable importance measure, defined as a sound statistical parameter, which is complemented by user defined marginal structural models. It allows one to quantify the significance of an exposure on a response while taking into account all other covariates. The parameter is studied through the Targeted Minimum Loss Estimation (TMLE) methodology. We perform its full theoretical analysis. We are able to establish consistency and asymptotic results which provide as a consequence p-values for hypothesis testing of the parameter of interest. A numerical analysis is conducted to illustrate theoretical results. It is achieved by extending the implementation of the TMLE.NPVI package [20] such that it is able to cope with multivariate parameter.

Résumé

Titre: Mesures de l'importance de variable au travers de modèles semi-paramétriques et en grande dimension avec ou sans erreurs sur les variables Les progrès technologiques de ces dernières décennies ont considérablement accru nos capacités à collecter et sauvegarder une quantité importante d'information. Ce faisant, notre marge de manoeuvre pour l'exploitation de ces données, a été amplifiée. Les répercussions, sur de nombreux domaines scientifiques, ont été fulgurantes. En analyse statistique par exemple, de nombreux résultats obtenus sous le canevas habituel d'étude en petite dimension, qui consistait à considérer le nombre de variables explicatives inférieur à la taille de l'échantillon, ont dû être étendus. La littérature scientifique abonde maintenant de nombreux résultats qui ont été mis en exergue, en prenant en compte cette nouvelle realité qu'est la présence des données en grande dimension. Nos travaux s'inscrivent dans cette droite lignée. En effet, cette thèse aborde le concept d'importance de variables, c'està-dire un canevas permettant de déterminer la portée d'une variable. Il s'agit là d'un point crucial dans cette nouvelle ère de données de grande taille. À titre d'exemple, ce concept est largement utilisé dans des modèles de prédiction afin d'améliorer le choix des variables explicatives. Nos contributions peuvent être divisées en trois parties.

Dans la première partie, nous introduisons une mesure multivariée dénommée mesure de l'importance de variable, définit en tant que paramètre statistique, assujettie à des modèles de structures marginaux. Nous nous sommes appuyés sur des modèles semi-paramétriques pour son analyse. Cette mesure permet notamment de quantifier la pertinence d'une variable explicative sur une réponse, en prenant en compte le reste des variables du problème.

Le paramètre d'intérêt est étudié grâce à la méthode du TMLE (Tartgeted Minimum Loss Estimation). Nous effectuons son analyse théorique complète et sommes ainsi en mesure d'établir la consistance de notre estimateur, ainsi que sa convergence asymptotique. Ce dernier résultat nous permet donc de déduire les intervalles de confiance liés à l'estimateur.

Nous effectuons également une analyse numérique afin d'illustrer nos résultats théoriques.

A cet effet, nous avons étendu l'implémentation du package TMLE.NPVI [20], de telle sorte qu'il puisse traiter des cas où le paramètre d'intérêt est multivarié.

Dans la seconde partie de cette thèse, nous introduisons une mesure de l'importance de variable définie au travers d'un modèle de régression non-paramétrique en grande dimension. Cette mesure provient en partie de celle introduite dans la première partie, sans la contrainte supplémentaire que l'utilisateur doive fournir un modèle de structure marginal.

Au-delà, nous considérons également le cas où les données de notre échantillon sont polluées. En s'appuyant sur une décomposition finie sur une base orthonormée du type base de Fourier ou Splines par exemple, et en utilisant la méthode dite du Lasso, nous établissons les vitesses de convergence de notre estimateur. Nous mettons aussi en exergue l'impact des erreurs de mesure, dans notre design, sur ces vitesses de convergence. Au-delà, nous proposons également une étude numérique basée sur des données synthétiques et une application, s'appuyant sur des données financières réelles.

Dans la troisième et dernière partie, nous considérons une mesure d'importance de variable définie grâce à un modèle de régression linéaire soumis à des erreurs de mesure sur son échantillon. Ce modèle de régression trouve son origine dans la partie précédente. L'estimation de notre paramètre d'intérêt s'effectue au travers d'un problème d'optimisation convexe, obtenu en projetant la covariance empirique du design sur l'ensemble de matrices définies positives, et en utilisant la pénalisation Slope. Nous effectuons ainsi une analyse théorique et numérique complète. Au-delà, nous établissons les conditions suffisantes, assez restrictives concernant les erreurs, à respecter afin d'atteindre des vitesses optimales de convergence de notre paramètre d'intérêt, tout en mettant l'accent sur l'impact de la pollution de notre échantillon sur ces vitesses.

Mots Clés: erreurs de mesure, estimation en grande dimension, inférence statistique, Lasso, Slope, optimisation convexe, modèle semi-paramétrique, modèle non-paramétrique, mesure de l'importance de variable, modèle de structure marginal, problèmes inverses, TMLE

Remerciements.

If I have seen further, it is by standing on the shoulders of giants.

Isaac Newton

Je souhaite exprimer ma profonde gratitude à mes deux directeurs de thèse : Cristina et Antoine. Il est certain que sans vous et votre support inconditionnel, je ne me trouverais point en train d'écrire ces quelques mots, qui marquent la fin de ce long et tortueux parcours scientifique, mais ô combien formateur. Ne vous inquiétez pas car je sais, qu'en réalité, il ne fait que commencer. Ne dit-on pas au pays de Toussaint Louverture : "Beyond the mountains, more mountains". Cette thèse n'a pas été conventionnelle mais grâce à votre soutien, elle m'a permis de grandir et d'affermir non seulement mes connaissances scientifiques, mais aussi humaines. Cristina, je te dis un grand merci pour tous tes conseils, ta rigueur et cette attention méticuleuse aux détails. Tout ceci a été contagieux mais je sais que j'ai encore du travail à fournir pour atteindre un niveau similaire au tien. Et bien sûr, merci pour tous les accueils chaleureux sur le "platal " et aussi pour m'avoir redonné plaisir à travailler sur un tableau ! Antoine, un grand merci à toi également pour les conseils et échanges très instructifs à Nanterre et au Panthéon. La distance physique n'a pas aidé mais tu as su rester à mes côtés jusqu'au bout. Je vous dois énormément à tous les deux ! Je suis honoré que Mme Gabriela Ciuperca et M. Eric Gautier aient accepté de rapporter ma thèse. Vos remarques et suggestions ont permis d'améliorer ce manuscrit. Je remercie Mme Céline Duval et M. Viet Chi Tran pour leur participation à mon Jury.

"Ne jamais t'endormir sur tes lauriers" ! C'est avec cette citation, proférée si souvent au détour de nos nombreuses et précieuses discussions, que mes parents n'ont eu de cesse d'essayer de me faire prendre conscience de la profondeur des concepts d'ardeur au travail et de résilience. Sans ces derniers, je n'aurai pas pu parachever cette thèse. C'est donc tout naturellement que je vous remercie pour ces valeurs qui raisonnent plus que jamais en moi. Je vous dis merci du fond du coeur pour tout, mais surtout pour votre amour inconditionnel.

Linda, lors de ma cérémonie de graduation de Master, tu m'avais susurré l'idée de faire un doctorat. Tu ignorais sans doute à ce moment-là que j'allais prendre ce conseil pour ordre et m'exécuter. À toi, je dis merci. Rita et Winnie, vous qui m'avez encouragé et supporté tout au long de ce voyage quasi initiatique, je vous dis également merci.

Hulda, comme tu le constates, il m'aura fallu moins de 20 ans ! Enfin presque ! Merci à toi ma très chère pour ton aide, les fous rires, ta tendresse, le soutien moral incommensurable après mes séjours presque toujours difficiles sur Paris. Un grand merci pour tes encouragements, ton amour et les sacrifices consentis pour me permettre de mener à bien cette thèse.

Je tiens aussi à remercier mon oncle Jean Marie pour son soutien et ses conseils. Merci d'avoir été à mes côtés depuis mon arrivée sur Paris. Je n'aurais sans doute jamais pu effectuer ce parcours académique sans toi. Une pensée toute particulière pour mon oncle et grand père Tchakam Flaubert, qui est toujours dans mes louanges. Un grand merci à mes oncles Roussel, Daniel et Hughes. J'ai aussi une pensée très chaleureuse pour ma tante Véronique : merci pour ton hospitalité légendaire et ton affection.

Pour finir, j'adresse également des remerciements chaleureux à tous ceux qui ont de près, comme de loin, contribué à ce travail : Olivia, Nelsone, Loic et Yannick. Un grand merci tout particulier à la famille Nana, pour m'avoir accueilli à bras ouvert et ainsi facilité, à bien des égards, mes dernières années de thèse. Je ne saurais terminer ce document sans mentionner mes amis de longue date : Landry, Alexandre, Ronald et Vincent. Nos conversations et échanges, surtout durant ces dernières années, m'ont permis d'affronter l'adversité avec force et conviction. J'oublie certainement de nombreuses personnes et je m'en excuse auprès de vous par avance. Sachez que c'est l'esprit Diopien, qui m'anime de plus en plus, qui me pousse à cette concision. Je suis convaincu que vous ne m'en tiendrez pas trop rigueur.

À mon père, à ma mère et mes ancêtres.

Modeste expression de mon infini reconnaissance et de ma piété filiale.

Résumé substantiel

Cette thèse aborde principalement le concept de mesure de l'importance de variable. Il s'agit d'un canevas qui permet de déterminer quantitativement, et parfois qualitativement, la portée d'une variable. C'est une notion primordiale dans de nombreux domaines scientifiques, notamment les problèmes statistiques comportant des jeux de données de grande taille.

Notre étude de mesure d'importance de variable s'effectue sous deux principaux registres. Premièrement, on définit et étudie une mesure d'importance de variable de dimension finie et définit sur un espace non paramétrique. La mesure est ainsi qualifiée de non paramétrique en ceci qu'elle est définie comme une valeur ψ f (P 0 ) d'une loi inconnue P 0 , provenant d'une fonctionnelle ψ f . Cette fonctionnelle est définie sur un ensemble non paramétrique de loi M qui est soumise à des contraintes peu restrictives. Dans ce cadre, f est une fonction fournie par l'utilisateur. Dans le Chapitre 3, nous développons et analysons un estimateur basé sur la méthode dénommée targeted minimum loss estimation. Sous des hypothèses bénignes, nous pouvons construire des intervalles de confiance asymptotique de notre estimateur, associés à un niveau donné. Deuxièmement, en s'inspirant de la définition de ψ f , nous introduisons une seconde fonctionnelle ψ pen , de dimension finie. Elle est définie sur l'ensemble M ′ ⊂ M, caractérisé sous M par la présence en son sein d'un modèle de régression. Contrairement à ψ f , ψ pen ne requiert point une fonction définie par l'utilisateur. En supposant que P 0 soit un élément de M ′ , ψ pen (P 0 ) est bien défini et ce vecteur peut être interprété comme une autre mesure d'importance de variable similaire à ψ f (P 0 ) et identique à ce dernier pour un choix précis de f . Dans les chapitres 3 et 4, nous développons et analysons deux estimateurs de ψ pen (P 0 ), basés respectivement sur les méthodes de Lasso et Slope. Sous des hypothèses propres à ce registre, nous établissons des vitesses de convergence. Des analyses numériques sont effectuées afin d'illustrer nos résultats théoriques.

La mesure d'importance de variable ψ f sous M D'importants progrès ont été réalisés dans l'article [START_REF] Van Der Laan | Statistical inference for variable importance[END_REF] dans le domaine de mesure d'importance de variable pour les problèmes statistiques. L'auteur y a proposé de définir cette mesure comme un paramètre statistique. Il pouvait ainsi être étudié au travers d'estimateurs adéquats. On pouvait donc en déduire des propriétés asymptotiques pour les estimateurs, facilitant ainsi la construction d'intervalles de confiance pour un niveau donné. Cette approche a inspiré de nombreux articles ultérieurs, dont le plus important pour nous fût [START_REF] Chambaz | Estimation of a non-parametric variable importance measure of a continuous exposure[END_REF], sur lequel notre travail se repose.

Considérons le problème statistique s'appuyant sur la structure de données O = (X, W, Y ), provenant d'une unité expérimentale d'intérêt W ∈ W ⊂ R d , représentant des vecteurs de variables explicatives, X ∈ R (une exposition continue) une variable réelle de cause affectant Y ∈ R (une réponse continue), une valeur réelle de variables d'effets.

On nomme par P 0 la vraie loi générant la structure de données O. On suppose que l'exposition contient un niveau de référence x 0 . En d'autres termes, il existe 0 < c < 1 2 tel que P 0 (X ≠ x 0 W ) ∈ [c, 1c] P 0 -presque surement.

Notre objectif est de quantifier la relation qui existe entre X et Y , tout en prenant en compte W . Il est primordial de considérer W pour établir cette relation car nous ne pouvons pas tout simplement ignorer son impact. Par analogie avec [START_REF] Chambaz | Estimation of a non-parametric variable importance measure of a continuous exposure[END_REF], nous introduisons une mesure d'importance de variable ψ f (P 0 ) caractérisée par ψ f (P ) = arg min

β∈R d E P Y -E P (Y X = x 0 , W ) -(X -x 0 )f β (W ) 2 , (1)
Y = E P 0 (Y X = x 0 , W ) -(X -x 0 )f β (W ) + ,
où est un bruit inconnu. En d'autres termes, ψ f (P 0 ) est universellement bien défini.

Le paramètre d'intérêt

Afin d'étudier le paramètre d'intérêt, nous utilisons une méthode d'estimation semiparamétrique dénommée Targeted Minimum Likelihood Estimation (TMLE ) par analogie avec [START_REF] Chambaz | Estimation of a non-parametric variable importance measure of a continuous exposure[END_REF].

Considérons (1) et supposons sans perte de généralité que x 0 = 0. Nous dénommons ḟβ = ∂f β ∂β = f CT , le gradient de f β et L 2 0 (P ) l'ensemble des fonctions mesurables g tel que E P [g 2 ] < ∞ et E P [g] = 0. Nous dénommons aussi θ(P )(X, W ) = E P (Y X, W ), g(P )(W ) = P (X ≠ 0 W ),

µ(P )(W ) = E P (X ḟβ (W ) W ) et Σ(P ) = E P [X 2 ḟβ (W ) ⊺ ḟβ (W )],
quelques propriétés importantes de la loi P . Si nous supposons que

• toutes les propriétés de P sont bien définies,

• Σ est inversible,

• il existe un c ∈]0, 1 2[ tel que g(P )(W ) ∈]c, 1c[ P -presque surement, alors, la solution de (1) est unique et est donnée par ψ f (P ) = Σ(P ) -1 E P X ḟβ (W ) (θ(P )(X, W )θ(P )(0, W )) .

La procédure d'inférence statistique

La procédure d'inférence statistique est au coeur de notre analyse numérique. Elle s'appuie sur n indépendantes copies O (i) , avec i ∈ {1, . . . , n} de la structure de données observées O. Comme mentionné plus haut, l'estimation de ψ f (P 0 ) est basée sur la méthode dite du TMLE. Elle peut être divisée en deux principales étapes. Dans la première étape, l'estimateur initial ψ f (P 0 n ) est évalué, avec P 0 n construit comme un élément de M s'appuyant sur les données O (1) , . . . , O (n) .

Nous remarquons que cet estimateur initial peut être biaisé. Ceci ne constitue pas une difficulté car ce biais, si il existe, sera corrigé au travers des mises à jour successives. On remarque également qu'il n'est pas nécessaire d'estimer toute la loi P 0 , mais seulement certaines de ses propriétés, à savoir : µ(P 0 ), g(P 0 ), Σ(P 0 ) et θ(P 0 ).

Dans la seconde étape, on construit k mises à jour P 1 n , . . . , P k n , qui nous permettent d'établir la séquence d'estimateurs {ψ j n = ψ f (P j n )} j=1,...,k . Les mises à jour P k n sont construites à partir de la fonction d'influence éfficace liée à ψ f . La procédure s'arrête lorsque la suite d'estimateurs converge. Dans le chapitre 2, nous sommes donc en mesure d'établir un théorème de convergence de notre estimateur. Il prouve ainsi qu'il est convergent et asymptotiquement normal.

Au-delà de ψ f au travers un sous ensemble de M Considérons la structure de données O = (X, W, Y ) d'une unité expérimentale d'intérêt où X ∈ R est une exposition, Y ∈ R est une réponse et W ∈ W ⊂ R d représente des variables explicatives. Cette structure de données est extraite de la vraie loi

P ∈ M ′ , où M ′ ⊂ M et M ′ est caractérisé par (η ⋆ P , θ ⋆ P ) ∈ R m × R k tel que Y = Φ⋆ m (W )η ⋆ P + X ⋅ Φ⋆ k (W )θ ⋆ P + ε, où Φ⋆ i = {Φ ⋆ 1 , . . . , Φ ⋆ i } est
E P (Y -E P (Y X = x 0 , W ) -(X -x 0 ) ⋅ f (W )) 2 .
(

) 2
Nous remarquons que le paramètre d'intérêt est maintenant une fonction. Ce faisant, nous introduisons une méthode plus flexible permettant de caractériser les interactions entre les éléments de W . Ceci nous permettra ainsi d'avoir une lecture plus complète de notre mesure d'importance de variable.

La fonction d'intérêt

Considérons Φ = (Φ 1 , . . . , Φ p , . . .) = {Φ j } ∞ j=1 une base orthonormée (b.o.n.) de H, un espace d'Hilbert. Sachant que P ∈ M ′ , on sait qu'il existe (η P , θ P ) ∈ R m × R k tel que, sous P , Y = Φm (W )η P + X ⋅ Φk (W )θ P + ε, (3)
où ε est une variable aléatoire centrée gaussienne, ayant pour variance σ 2 , indépendante de (X, W ), et Φj = {Φ 1 , . . . , Φ j } représente un vecteur dont les éléments sont les j premières composantes de la b.o.n. H. En plus, g(P

)(W ) = E P [Y X = 0, W ] = Φm (W )η P et, si on choisit f (W ) = Φk (W )θ P , alors ψ f (P ) = θ P .
Par analogie avec (1), Φk (W ) ⋅ θ P est notre nouvelle mesure d'importance de variable.

Nous n'exploitons par la forme de ψ pen (P ) définit sous (2), mais plutôt nous nous appuyons sur le modèle de régression (3) pour son analyse.

On réécrit (3) au travers de

Y = X ⋅ β + , (4)
où β = ⎛ ⎝ η θ ⎞ ⎠ ∈ R p avec p = m + k et X correspondant à X = ⎡ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎣ Φ 1 (W 1 ) . . . Φ m (W 1 ) X 1 ⋅ Φ 1 (W 1 ) . . . X 1 ⋅ Φ k (W 1 ) ⋮ ⋮ ⋮ ⋮ Φ 1 (W n ) . . . Φ m (W n ) X n ⋅ Φ 1 (W n ) . . . X n ⋅ Φ k (W n ) ⎤ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎦ . ( 5
)
On assume n ≪ p. Nous faisons ainsi face à un problème de grande taille. Les vecteurs θ et η sont supposés creux avec des coefficients respectifs donnés par s θ et s η . Par conséquent,

β est aussi creux (c'est-à-dire, ∑ p i=1 I(β j ≠ 0) = s ≪ p) avec pour coefficient s = s η + s θ .

Estimateurs pénalisés

Dans le chapitre 3, nous optons pour la méthode du Lasso afin de trouver un estimateur à notre paramètre d'intérêt. Le problème peut donc s'écrire

β = ⎛ ⎝ η θ ⎞ ⎠ = arg min β∈R p 1 2n n i=1 Y i -(Xβ) i 2 + λ β 1 , (6)
où β 1 = ∑ p i=1 β i et λ > 0 est un paramètre de réglage.

Au travers d'une condition de compatibilité (voire [START_REF] Sorensen | Measurement error in lasso : Impact and likelihood bias correction[END_REF][START_REF] Van De Geer | The deterministic lasso[END_REF]), on arrive à établir les vitesses de convergence ainsi que la consistance de notre estimateur β, et par conséquent

η et θ.
Il est important de se rappeler que notre nouvelle mesure d'importance de variable est donnée par Φk (W ) ⋅ θ P . Ce faisant, il peut être avantageux à bien des égards de considérer un problème d'optimisation dont la pénalité s'appuie uniquement sur θ. Ceci nous permettra ainsi d'avoir une vision plus appropriée de l'estimateur de la fonction d'intérêt. Avec ce cadre, m est supposé largement inférieur à n, mais l'on conserve n ≪ k.

On dénomme par Ψ m , la matrice donnée par les m premières fonctions de la base

{Φ} ∞ j=1 au points W 1 , . . . , W n . Elle correspond à Ψ m = ⎡ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎣ Φ 1 (W 1 ) . . . Φ m (W 1 ) ⋮ ⋮ Φ 1 (W n ) . . . Φ m (W n ) ⎤ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎦ Notre nouveau problème d'optimisation peut donc s'écrire ⎛ ⎝ η θ ⎞ ⎠ = arg min η∈R m ,θ∈R k 1 2n n i=1 Y i -(Ψ i m ⋅ η + X i ⋅ (Ψ i k ⋅ θ)) 2 + λ θ 1 . (7)
Grâce à une condition de compatibilité, nous arrivons à établir les vitesses de convergence de nos estimateurs η et θ, de même que leur consistance. Ceci améliore les vitesses d'estimation de θ, lié au modèle [START_REF] Belloni | An { 1 , 2 , ∞ }-regularization approach to high-dimensional errors-in-variables models[END_REF].

Estimateurs pénalisés avec erreurs sur les variables

Nous savons que dans de nombreux cas de la vie réelle, les données observées sont en général polluées. Ceci a été pris en considération dans la seconde phase de notre analyse.

Précisément, nous assumons que le modèle (4) est soumis à des erreurs additives sur les variables correspondantes à Z = X + ν, où les éléments de ν sont extraits d'une distribution centrée gaussienne de variance µ 2 . Nous introduisons ainsi le nouveau modèle caractérisé par

⎧ ⎪ ⎪ ⎪ ⎨ ⎪ ⎪ ⎪ ⎩ Y = X ⋅ β + Z = X + K , (8)
où β = ⎛ ⎝ η θ ⎞ ⎠ , et K est donné par K = ⎡ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎣ 0 . . . 0 ν 1 ⋅ Φ 1 (W 1 ) . . . ν 1 ⋅ Φ k (W 1 ) ⋮ ⋮ ⋮ ⋮ 0 . . . 0 ν n ⋅ Φ 1 (W n ) . . . ν n ⋅ Φ k (W n ) ⎤ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎦ .
La présence d'erreurs sur les variables a naturellement un impact sur la formulation du problème d'optimisation. Il s'écrit dorénavant sous la forme

β ∈ arg min β∈R p 1 2 β ⊺ Γβ -γ T β + λ β 1 tel que β 1 ≤ c 0 √ s, où Γ = 1 n X⊺ X -µ 2 ζ ⊺ ζ, γ = 1 n X⊺ Y , X = [Ψ m , D Z Ψ k ], ξ = (0, . . . , 0, 1, . . . , 1
θ 1 ≤c 0 √ s θ 1 2 θ ⊺ Γθ -γ ⊺ θ + λ θ 1 .
En s'appuyant de nouveau sur la condition Restricted Eigenvalues, mentionnée ci-dessus, nous établissons également la consistance et les vitesses de convergence de nos estimateurs sous certaines conditions.

Estimateurs Slope avec erreurs sur les variables

Dans le chapitre 4, nous généralisons le modèle de régression (8) et introduisons le modèle

⎧ ⎪ ⎪ ⎪ ⎨ ⎪ ⎪ ⎪ ⎩ Y = X ⋅ β ⋆ + W = X + U (9) où Y = (Y 1 , . . . , Y n ) ⊺ est
2 n Y ⊺ W β + β ⋆ , (10)
où Σ ∈ arg min

M ∈S ≥0 Σ -M 2 avec Σ = 1 n W ⊺ W -C U ,
et l'ensemble S ≥0 est un ensemble de matrices symétriques définies positives découlant d'une projection au travers de la norme de Frobenius. La norme Slope est donnée par

β ⋆ = ∑ p i=1 λ i β (i)
, où λ 1 ≥ . . . ≥ λ p > 0 sont des paramètres de réglage. En utilisant la projection Σ au lieu de Σ, nous assurons la convexité du critère [START_REF] Birkner | Data adaptive pathway testing[END_REF].

Chapter 1

Introduction

During the last couple of decades, we have witnessed an unprecedented development in technology. It has ultimately allowed us to collect a tremendous amount of data. Commonly called the era of big data, its impact looms large, touching a wide variety of scientific fields such as economics, biology and medicine. Its repercussions on our daily life are becoming more and more visible since it is affecting a large portion of the "real" economy.

As examples, we can mention the health-care and automotive industries. The potential of this era has not yet been fully harnessed. One of the reasons, for this state of affairs, is that it comes with a lot of challenges. As far as statisticians are concerned, one of these challenges is the curse of dimensionality. It has prompted an extensive amount of research. The outcomes of these studies, in many cases, led to the extension of results that were proven true in low dimension settings. Furthermore, the high dimensionality of available data has also brought to light the need to carefully choose adequate covariates, when defining a model. As such, several tools have been introduced among which the concept of variable importance measure. It is at the core of this thesis.

Overview of our contributions. Our study of variable importance measure is achieved under two main frameworks. On the one hand, we define and study a new nonparametric, finite-dimensional, covariate-adjusted variable importance measure. It is nonparametric in the sense that it is defined as the value ψ f (P 0 ) at the law P 0 of the data of a functional ψ f , defined on a nonparametric set M of laws subject to mild constraints. Here, f is a user-supplied function. In Chapter 2, we develop and analyze an estimator based on the targeted minimum loss estimation methodology. Under mild assumptions, the estimator lends itself to the construction of confidence regions of given asymptotic confidence level and to the derivation of asymptotic p-values for related hypothesis testing.

On the other hand, drawing inspiration from the definition of ψ f , we introduce a second high-dimensional functional ψ pen , defined on a set M ′ ⊂ M characterized within M by a regression model. As opposed to ψ f , ψ pen does not rely on a user-supplied function.

Assuming that P 0 falls in M ′ , ψ pen (P 0 ) is well defined and a subvector thereof can be interpreted as another variable importance measure akin to ψ f (P 0 ), and equal to it for a particular choice of f . In Chapters 3 and 4, we develop and study two estimators of ψ pen (P 0 ), based respectively on the Lasso and the Slope methodologies. Under assumptions that are typical of this kind of framework, we derive some convergence rates. Simulation studies illustrate the theoretical results.

Variable importance measure

The analysis of variable importance has been simultaneously developed in several fields such as engineering, economics, biology, statistics and machine learning, to only mention those.

The era of big data has made these techniques of paramount importance. The authors in [START_REF] Wei | Variable importance analysis: a comprehensive review[END_REF] provide a rather exhaustive list of variable importance measures (VIM), covering a broad spectrum of disciplines such as astronautic engineering, chemistry and environmental science. They argue that these measures could be regrouped in seven main categories: Difference-based VIMs (i.e.: Derivatives-based methods [START_REF] Sobol | A new derivative based importance criterion for groups of variables and its link with the global sensitivity indices[END_REF]), Parametric regression techniques (i.e.: correlation coefficient), Nonparametric regression techniques (i.e.: Generalized Additive Model), Hypothesis test techniques (i.e.: entropy based measure [START_REF] Tang | Entropy-based importance measure for uncertainty model inputs[END_REF]), Variance based VIMs [START_REF] Homma | Importance measures in global sensitivity analysis of nonlinear models[END_REF], Moment-independent VIMs [START_REF] Borgonovo | A new uncertainty importance measure[END_REF] and Graphs VIMs [START_REF] Bolado-Lavin | Contribution to the sample mean plot for graphical and numerical sensitivity analysis[END_REF].

There is not an unified definition for the notion of variable importance measure. However, within the predictive model world, it can be viewed as a scale which helps to determine the dependence of an outcome of a regression model upon a single or a set of input variables.

We can cite as an example Random Forest [START_REF] Breiman | Random forests[END_REF], which is a well-known machine learning algorithm for regression and classification. It relies on two main measures to evaluate a variable importance. The first one, called Mean Decrease Impurity importance (MDI) or Mean Decrease Gini, is based on the aggregation of the Gini Impurity across all nodes of the decision tree, which helps to determine how to split the data at each node. The second one, called Permutation Importance, is also based on the aggregation of a measure of accuracy of the predictor, computed at each node of the tree. Random Forest and similar decision tree techniques have a lot of benefits. They are well design for high dimensional dataset. The construction of decision trees requires, at each node, only a subset of the entire dataset. Furthermore, Random Forest is well-known to still be robust when the dataset contains outliers.

Similar positive features can be presented regarding other VIMs of the same family. However, it is also important to emphasize that most of them do have a few drawbacks. We know as an example that decision tree methods sometimes produce optimal predictors which have very few variables, given the initial dataset (see [START_REF] Birkner | Data adaptive pathway testing[END_REF]). Random Forest is known for over-fitting learning datasets that are particularly noisy. Furthermore, most VIMs techniques are known not to allow users to infer p-values, for hypothesis testing, from the quantified variable importance. Some of these methods have been questioned by researchers because they do not faithfully represent the true importance of a variable (see [54]).

1.2

The variable importance measure ψ f over model M Significant breakthroughs were achieved in the article [START_REF] Van Der Laan | Statistical inference for variable importance[END_REF] within the field of variable importance measure for statistical problems. The author proposed to define the latter measure as a sound statistical parameter. It could then be studied through adequate estimators. One could therefore infer asymptotic properties for these estimators, hence easing the derivation of p-values for hypothesis testing. This work inspired several subsequent articles, with the most important for us being [START_REF] Chambaz | Estimation of a non-parametric variable importance measure of a continuous exposure[END_REF], on which our work is based.

Let us consider a statistical problem with a data structure O = (X, W, Y ) of an experimental unit of interest W ∈ W ⊂ R d , representing the vectors of covariates, X ∈ R (a continuous exposure) a real valued variable of cause affecting Y ∈ R (a continuous response), a real valued variable of effect. We denote by P 0 the true data-generating distribution of the data structure O. The exposure is assumed to feature a reference level x 0 . In words, there exists 0 < c < 1 2 such that P 0 (X ≠ x 0 W ) ∈ [c, 1c] P 0 -almost surely.

Our objective is to quantify the relationship, if any, that exists between X and Y , while taking into account the covariates W . Taking the covariates in consideration is critical for us since we cannot rule out their impact on the relationship between the exposure and the response. By analogy with [START_REF] Chambaz | Estimation of a non-parametric variable importance measure of a continuous exposure[END_REF], we introduce a variable importance measure ψ f (P 0 ) characterized by

ψ f (P ) = arg min β∈R d E P Y -E P (Y X = x 0 , W ) -(X -x 0 )f β (W ) 2 , (1.1)
where

f β (W ) = β ⊺ ⋅ f CT (W ) with f CT ∶ W → R d is a user-supplied function. The as-
sumption here is that one could predefine the type of relationship that exists between the covariates. As an example, with d = 2, we could have

f CT (W ) = (W 2 1 , W 2 2 ), where W = (W 1 , W 2 ). As such, we can infer that (X -x 0 )f ψ f (P 0 ) (W ) is the best approximation of the form (X -x 0 )f β (W ) of Y -E P (Y X = x 0 , W ).
It is important to note here that ψ f (P 0 ) is multivariate since its size corresponds to the number of covariates. This feature is crucial since it allows us to have a variable importance measure which captures the role played by each covariate in the relationship between the exposure and the response.

We emphasize here that we do not assume that there exists β such that

Y = E P 0 (Y X = x 0 , W ) + (X -x 0 )f β (W ) + ,
where is an unknown noise. In words, ψ f (P 0 ) is universally defined.

The parameter of interest

In order to study the parameter of interest, we use a semiparametric estimation methodology called targeted minimum loss estimation (TMLE ), by analogy with [START_REF] Chambaz | Estimation of a non-parametric variable importance measure of a continuous exposure[END_REF]. We need to identify and study its key properties in order to deploy the semiparametric model theory.

Influence Curve. a semiparametric model is a statistical model in which parameters of interest are both an Euclidean vector and an infinite-dimensional parameter (also called nonparametric component). When dealing with these models, one is usually interested in estimating the Euclidean vector.

To solve this problem, the theory of asymptotic efficiency, as developed for parametric models, was extended. As such, the notion of semiparametric efficiency bounds was introduced in [START_REF] Stein | Efficient nonparametric testing and estimation[END_REF] and further developed in [3,[START_REF] Bickel | Efficient and adaptive estimation for semiparametric models[END_REF][START_REF] Koshevnik | On a nonparametric analogue of the information matrix[END_REF][START_REF] Pfanzagl | Contributions to a general asymptotic statistical theory[END_REF].

Keeping the above in mind, we can infer that the influence function plays a similar role as the normalized score function in parametric models (see [START_REF] Van Der | Asymptotic Statistics[END_REF]). However, the influence function is not unique. A corresponding function can be found through a projection on the closure of a linear span of the tangent space (see [START_REF] Van Der | Asymptotic Statistics[END_REF]). This function, result of a projection, is unique and is called the efficient influence curve. Through [START_REF] Bickel | Efficient and adaptive estimation for semiparametric models[END_REF], we can establish that by knowing the influence function (or even better the efficient influence function) of an estimator, we can deduce its asymptotic distribution. Now, considering (1.1), we assume without loss of generality that x 0 = 0. Let us denote by ḟβ = ∂f β ∂β = f CT , the gradient of f β and L 2 0 (P ) the set of measurable functions g such that E P [g 2 ] < ∞ and E P [g] = 0. We also denote by

θ(P )(X, W ) = E P (Y X, W ), g(P )(W ) = P (X ≠ 0 W ), µ(P )(W ) = E P (X ḟβ (W ) W ) and Σ(P ) = E P [X 2 ḟβ (W ) ḟβ (W ) ⊺ ],
some relevant features of the distribution P . If we assume that • all features of P are well-defined,

• Σ is invertible,

• there exists a c ∈]0, 1 2[ such that g(P )(W ) ∈]c, 1c[ P -almost surely, then the solution to (1.1) is unique and is given by

ψ f (P ) = Σ(P ) -1 E P X ḟβ (W ) (θ(P )(X, W ) -θ(P )(0, W )) .
Furthermore, we can also infer that the functional ψ f is path-wise differentiable at P . Let us consider a bounded function s ∈ L 2 0 (P ) and ∈ R d such that s ∞ < ∞ and ∞ < s -1 ∞ . We can characterize a distribution P ∈ M such that

dP (O) = 1 + ⊺ s(O) dP (O), (1.2)
If P verifies the conditions defined above, then ψ f (P ) is differentiable at = 0 and its derivative is given by

lim →0 ψ f (P ) -ψ f (P ) = E P s(O) ⊺ D ⋆ (P )(O) ,
where D ⋆ (P ) is the efficient influence curve, given by

D ⋆ (P ) = Σ(P ) -1 θ(P )(X, W ) -θ(P )(0, W ) -Xf β (W ) X ḟ (W ) + Σ(P ) -1 (Y -θ(P )(X, W )) X ḟ (W ) - 1 X=0 g(P )(0 W ) µ(P )(W ) .
We note here that this influence curve enjoys a key feature. It is double-robust (see [START_REF] Chambaz | Estimation of a non-parametric variable importance measure of a continuous exposure[END_REF]) in the sense that for any (P, P ′ ) ∈ M 2 , if either (µ(P ′ ) = µ(P ) and g(P ′ ) = g(P )) or θ(P ′ )(0, ⋅) = θ(P )(0, ⋅) holds then P D ⋆ (P ′ ) = 0 implies that ψ f (P ′ ) = ψ f (P ). This property is of importance because it plays a crucial role in proving the consistency and asymptotic property of the estimator of ψ f (P 0 ).

Statistical inference procedure

Having laid the groundwork of the underlying theory of the estimation methodology, we turn our attention to the statistical inference procedure. It is at the heart of the numerical analysis. The procedure is based on n independent copies O (i) , with i ∈ {1, . . . , n} of the observed data structure O. As mentioned above, the estimation of ψ f (P 0 ) is based on the TMLE methodology which can be divided in two main steps. In the first step, an initial estimate ψ f (P 0 n ) is evaluated, where P 0 n ∈ M is built as an element of M based on O (1) , . . . , O (n) .

It is important to note that this initial estimate can be biased. It does not constitute a drawback for the rest of the procedure, since this bias, if it exists, is corrected with subsequent updates. We also emphasize that it is not required to estimate the whole distribution P 0 for our objective. In fact, only a few features of P 0 must be estimated, namely : µ(P 0 ), g(P 0 ), Σ(P 0 ) and θ(P 0 ).

In the second step, using (1.2), we then construct k successive updates P 1 n , . . . , P k n which allow us to build a sequence of estimates {ψ j n = ψ f (P j n )} j=1,...,k . For k large enough, the procedure converges. In Chapter 2 (see 2.1), we then derive the following result: Theorem 1.1. Let us denote ψ 0 = ψ f (P 0 ). Suppose that performing k n iterations of the updating procedure guarantees that

P n D ⋆ (P kn n ) ∞ = o P ( 1 √ n ).
Suppose moreover that there exists a function f 1 with P 0 f 1 = 0 such that P 0 (D ⋆ (P kn n )f 1 )(D ⋆ (P kn n )f 1 ) ⊺ ∞ = o P (1), and that ψ f (P kn n )ψ f (P 0 ) -

P 0 D ⋆ (P kn n ) ∞ = o P ( 1 √ n ).
In addition, suppose that S n estimates consistently

E P 0 [f 1 (O)f 1 (O) ⊺ ]. Then, ψ ⋆ n = ψ f (P kn n ) satisfies √ n(ψ ⋆ -ψ 0 ) = (P n -P 0 )f 1 +o P (1), hence √ nS -1 2 n (ψ ⋆ n -ψ 0 )
converges in law to the d-multivariate Gaussian law with zero mean and identity covariance matrix.

This result proves the consistency of the TMLE estimator and its asymptotic normality. It therefore allows one to construct confidence regions given a confidence level. Furthermore, one can derive the parameter of interest p-values for hypothesis testing.

Y = Φ⋆ m (W )η ⋆ P + X ⋅ Φ⋆ k (W )θ ⋆ P + ε,
where Φ⋆ j = {Φ ⋆ 1 , . . . , Φ ⋆ j } is a vector containing the first j elements of an orthonormal basis (o.n.b) of a Hilbert space. We recall that (1.1) is given by

ψ f (P ) = arg min β∈R p E P Y -E P (Y X = x 0 , W ) -(X -x 0 ) ⋅ f β (W ) 2 ,
where f β (W ) = β ⋅f CT (W ), with f CT a function provided by the user. We chose to have f β linear in β for simplicity. It allowed a clearer path towards strong convergence rates results.

Furthermore, we emphasize here that the function f CT plays the role of representing the interactions that exist between the covariates. Strictly speaking, it is an unknown feature of our problem which is, in most cases, a source of considerable complexity. In Chapter 3, we decide to analyze the problem through different lenses. We consider the same variable importance measure without requesting that f β be specified by the user.

Our objective remains to quantify the relationship that exists between the response Y and the exposure X (having a reference level x 0 ), while taking into account the covariates W . Let us denote by F, the nonparametric set of all continuous functions on R d . We introduce, by analogy with (1.1), a statistical parameter ψ pen (P ) characterized by

ψ pen (P ) = arg min f ∈F E P (Y -E P (Y X = x 0 , W ) -(X -x 0 ) ⋅ f (W )) 2 . (1.3)
It is important to note here that the parameter of interest is now a function. As such, we are aiming at introducing a more flexible way of characterizing the interactions between the covariates. It allows us to obtain a complete picture of the variable importance measure with respect to the covariates in the dataset. From now on, we assume x 0 = 0, without loss of generality.

The function of interest

Nonparametric regression models have been extensively studied in the literature. Several groups of techniques used to solve these problems have emerged through the years. We can mention as examples the Kernel smoothing method [START_REF] Tsybakov | An Introduction to Nonparametrics Estimation[END_REF], k-nearest-neighbors (see [1,[START_REF] Gyorfi | A distribution free theory of nonparametric regression[END_REF] with few variations such as the weighted k-nearest-neighbors regression [START_REF] Samworth | Optimal weighted nearest neighbour classifiers[END_REF]), Wavelets (see [START_REF] Mallat | A wavelet tour of signal processing[END_REF]), Smoothing spline (see [START_REF] Schoenberg | Spline functions and the problem of graduation[END_REF][START_REF] Tsybakov | An Introduction to Nonparametrics Estimation[END_REF]) and nonparametric least squares. We rely upon this last technique for what follows.

Let us consider Φ = (Φ 1 , . . . , Φ p , . . .) = {Φ j } ∞ j=1 a complete orthonormal base of H, a Hilbert space. Given that P ∈ M ′ , we know that there exists (η P , θ P ) ∈ R m × R k such that, under P ,

Y = Φm (W )η ⊺ P + X ⋅ Φk (W )θ P + ε, (1.4)
where ε is a centered Gaussian random variable with variance σ 2 that is independent of (X, W ), and Φj = {Φ 1 , . . . , Φ j } represents a vector whose elements are the first j components of an o.n.b. of H. Furthermore, g(P )(W ) = E P [Y X = 0, W ] = Φm (W )η P and, if one chooses f (W ) = Φk (W )θ P , then ψ f (P ) = θ P . We emphasize here that by analogy to (1.1), Φk (W ) ⋅ θ P is our new variable importance measure. Furthermore, we note that the whole base {Φ j } ∞ j=1 is not used in our decomposition. As such, there are potentially residual terms which one would want to take into consideration. We assume here that they are null under some smoothness conditions.

It is important to note that in what follows, we do not exploit the form of ψ pen (P ) given by (1.3), but rather rely on the regression model (1.4) for its analysis. The indices {m, k} are assumed known and increase with n, but are function of P . The latter dependence is not problematic in this study since we only consider a single data generating distribution P .

We can now rewrite (1.4) as

Y = X ⋅ β + , (1.5)
where

β = ⎛ ⎝ η θ ⎞ ⎠ ∈ R p with p = m + k and X is given by X = ⎡ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎣ Φ 1 (W 1 ) . . . Φ m (W 1 ) X 1 ⋅ Φ 1 (W 1 ) . . . X 1 ⋅ Φ k (W 1 ) ⋮ ⋮ ⋮ ⋮ Φ 1 (W n ) . . . Φ m (W n ) X n ⋅ Φ 1 (W n ) . . . X n ⋅ Φ k (W n ) ⎤ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎦ . (1.6)
We assume n ≪ p. As such, our problem is in high dimension. Giving this setting, the vectors θ and η are assumed sparse with respective coefficients given by s θ and s η . As a result, β is also sparse (in essence: In high dimensional settings, we know that the matrix X ⊺ X cannot be positive definite.

∑ p i=1 I(β j ≠ 0) = s ≪ p)
As such, (1.7) does not admit a unique solution.

One way of solving this issue in the literature has been to target a set of potential solutions β ⋆ which are sparse. This sparsity assumption implies that we do not want to have values of β ⋆ that are large, even if they lead to better fit. It is then common to adjust the problem (1.7) by introducing a penalty term such that it is now given by

βLASSO = arg min β∈R p Y -Xβ 2 + pen(β), (1.8)
where pen(⋅) is a real-valued function of β. Several alternatives have been considered for the choice of pen(⋅). The most natural one appeared to be pen(β) = λ ⋅ β 0 , where λ > 0 is a tuning parameter and ⋅ 0 represents the total number of non-zero elements. However, with this choice of penalization problem, the resulting problem is impossible to compute in polynomial time (it is N P -hard). The Lasso estimator, introduced in [START_REF] Tibshirani | Regression shrinkage and selection via the lasso[END_REF], has opted for a convex relaxation of the penalty term with pen(β) = λ ⋅ β 1 , where ⋅ 1 is the l 1 -norm.

The authors in [START_REF] Bellec | Slope meets lasso: improved oracle bounds and optimality[END_REF] have shown that there exists a λ ⋆ such that the Lasso estimator reaches the optimal minimax rates of prediction and estimation, under some conditions on the design matrix X.

We note that λ ⋆ remains a function of the sparsity coefficient s of β, which is unknown in practice. A solution that has been explored to alleviate this difficulty is to consider the Slope estimator (see [START_REF] Malgorzata | Slope -adaptative variable selection via convex optimization[END_REF]). It relies on a penalty term given by pen(β) = ∑ p j=1 λ j β (j) , where the tuning parameters verifies λ 1 ≥ ⋯ ≥ λ p > 0 and β (j) is the j th largest component of β . With this form, the objective is to penalize more the larger components of β, compare to the smaller ones. The authors in [START_REF] Bellec | Slope meets lasso: improved oracle bounds and optimality[END_REF] have shown that there exist tuning parameters {λ ⋆ i } i=1,...,p such that the Slope estimator attains the optimal minimax rates of estimation and prediction. 1.3.2 The optimization problems and corresponding convergence rates using Lasso

In Chapter 3, we opt for the Lasso methodology to find the estimator of the parameter of interest. As mentioned above, it appears natural to use the penalty function pen(β) = λ β 1 for our optimization problem. However, we recall that Φk (W ) ⋅ θ P is now our new variable importance measure. One can then prefer to use the penalty function pen(θ) = λ ⋅ θ 1 . By only penalizing θ P , our variable of interest, it might be possible to infer a better estimator. Both cases are explored below.

Penalty function based on β.

The optimization problem is given by

β = ⎛ ⎝ η θ ⎞ ⎠ = arg min β∈R p 1 2n n i=1 (Y i -(Xβ) i ) 2 + λ β 1 (1.9)
where β 1 = ∑ p i=1 β i and λ > 0 is a tuning parameter. We rely on the well-known compatibility condition (see [START_REF] Sorensen | Measurement error in lasso : Impact and likelihood bias correction[END_REF][START_REF] Van De Geer | The deterministic lasso[END_REF]). It stipulates that, on an index set S 0 having s 0 elements, if for some φ 0 > 0 and all γ ∈ R p such that γ S c 0 1 ≤ 3 γ S 0 1 , it holds that

γ S 0 2 1 ≤ s 0 Xγ 2 2 nφ 2 0 .
It plays an important role in establishing the convergence rate of the estimator β. In Chapter 3 (see 3.1), we then derive the following result:

Proposition 1.2. Assume that the variable X takes values in a compact space. Furthermore, assume that the compatibility condition holds. Then, with probability larger than

1 -2 exp(-t 2 2 ) and λ = 2σ X ∞ t 2 +2 log(p) n
with arbitrary t > 0, the estimator β of (1.9)

verifies 1 2n X β -β 2 2 + λ β -β 1 ≤ 4σ 2 X 2 ∞ 8s φ 0 . t 2 + 2 log(p) n .
As a consequence, with the same probability,

β -β 1 ≤ 2σ X ∞ ⋅ 8s φ 2 0 ⋅ t 2 + 2 log(p) n .
We can then infer similar results for the estimator η, and ultimately, θ the estimator of our parameter of interest.

Penalty function based on θ.

This framework relates to a partially linear model with high-dimensional linear component, i.e. m is assumed smaller than n, but we keep n ≪ k. We put more emphasis on the estimation of θ as it is a key component of our parameter of interest. Let us denote by Ψ m , the matrix given by the first m functions of the base {Φ} ∞ j=1 at points W 1 , . . . , W n , that is,

Ψ m = ⎡ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎣ Φ 1 (W 1 ) . . . Φ m (W 1 ) ⋮ ⋮ Φ 1 (W n ) . . . Φ m (W n ) ⎤ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎦ and by D X the diagonal matrix with elements {X 1 , . . . , X n }. Thus, X = [Ψ m , D X Ψ k ].
The new optimization problem can now be written as

⎛ ⎝ η θ ⎞ ⎠ = arg min η∈R m ,θ∈R k 1 2n n i=1 Y i -(Ψ i m ⋅ η + X i ⋅ (Ψ i k ⋅ θ)) 2 + λ θ 1 .
(1.10)

We solve this problem in two steps. In the first step, we solve for η which minimizes (1.10) for any θ.

It corresponds to a classic least square estimator whose solution is given by

η(θ) = 1 n Ψ ⊺ m ⋅ (Y -(D X Ψ k )) , where Y = {Y i } i=1,...,n .
In the second step, we replace the solution of the previous solution in (1.10), and then obtain a resulting classic l 1 optimization problem solely based on θ and given by θ = arg min

θ∈R k 1 n V ⋅ Y -Ψk ⋅ θ 2 2 + λ θ 1 , (1.11)
where λ > 0 is a tuning parameter, V = I n×n -1 n Ψ m ⋅ Ψ ⊺ m and Ψk = D X Ψ k . We are then able to deduce convergence rates of our estimator. In Chapter 3 (see 3.2), we then derive the following result: Proposition 1.3. Let us assume that the compatibility condition holds. Thus with probability larger than 1 -2 exp(-t 2

2 ) and for λ = 2σ X ∞

t 2 +2 log(k) n , the estimator θ is such that 1 2n V ⋅ Ψk ⋅ θ -θ 2 2 + λ θ -θ 1 ≤ 4σ 2 X 2 ∞ ⋅ 8s θ φ 2 0 ⋅ t 2 + 2 log(k) n
for some arbitrary t > 0. Hence,

θ -θ 1 ≤ 2σ X ∞ 8s θ φ 2 0 ⋅ t 2 + 2 log(k) n .
This result shows an improvement in the estimation of θ. However, we can notice that the estimation rate of η deteriorates with respect to the case where θ and η are estimated at the same time.

Adding errors-in-variables and their impact on convergence rates using Lasso

We know that in most real life examples, the dataset used for statistical analysis is polluted. We have taken the latter fact in consideration in this second phase of our analysis.

Specifically, we assume that the model (1.5) is subject to additive measurement errors corresponding to Z = X + ν, where elements of ν are drawn from a centered Gaussian distribution with variance µ 2 . The variables and ν are assumed independent. Thus, we introduce the model given by

⎧ ⎪ ⎪ ⎪ ⎨ ⎪ ⎪ ⎪ ⎩ Y = X ⋅ β + Z = X + K , (1.12)
where

β = ⎛ ⎝ η θ ⎞ ⎠
, and K is given by

K = ⎡ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎣ 0 . . . 0 ν 1 ⋅ Φ 1 (W 1 ) . . . ν 1 ⋅ Φ k (W 1 ) ⋮ ⋮ ⋮ ⋮ 0 . . . 0 ν n ⋅ Φ 1 (W n ) . . . ν n ⋅ Φ k (W n ) ⎤ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎦ .
We further consider that there exists L > 0 such that

Ψ k (W )θ 2 2 < L, P -almost surely.
This assumption plays a key role in deriving the convergence rates of the estimators.

Similarly to Section 1.3.2, two types of penalty functions are considered.

Penalty function based on β.

We note that the presence of measurement errors surely has an impact on the optimization problem as developed in (1.9). In fact,

E[ Y -(Ψ m η + (D Z Ψ k )θ) 2 2 ] = E[ Y -(Ψ m η + (D X Ψ k )θ) 2 2 ] + E[ (D ν Ψ k )θ 2 2 ] -2E[ ⊺ (D ν Ψ k )θ] = E[ Y -(Ψ m η + (D X Ψ k )θ) 2 2 ] + nµ 2 θ 2 2 .
From the above, it appears that the regularization requires a correction by a factor of

nµ 2 θ 2 2 .
As such, the estimators η and θ are then defined by

⎛ ⎝ η θ ⎞ ⎠ = arg min η∈R m ,θ∈R k L(η, θ) with L(η, θ) = 1 2n Y -(Ψ m ⋅ η + (D Z Ψ k ) ⋅ θ)) 2 2 - 1 2 µ 2 θ 2 2 + λ 1 η 1 + λ 2 θ 1 ,
where λ 1 ≥ 0 and λ 2 ≥ 0.

We denote by X, the equivalent of X in (1.5) with X replaced by its counterpart Z. The matrix

X corresponds to X = [Ψ m , D Z Ψ k ],
where D Z is a diagonal matrix with diagonal elements Z 1 , . . . , Z n . We further consider the vector ξ = (0, . . . , 0, 1, . . . , 1), of size p, where the first m elements are equal to 0 and the last k are all equal to 1. Decomposing the loss function, we obtain:

L(η, θ) = 1 2 1 n Ψ m η + (D Z Ψ k )θ 2 2 -µ 2 θ 2 2 - 1 n ⟨Y, Ψ m η + (D Z Ψ k )θ⟩ + 1 2n Y 2 2 + λ 1 η 1 + λ 2 θ 1 = 1 2 1 n Xβ 2 2 -µ 2 ζβ 2 2 - 1 n ⟨Y, Xβ⟩ + λ β 1 + 1 2n Y 2 2 with λ = ⎛ ⎝ λ 1 λ 2 ⎞ ⎠ , where β = ⎛ ⎝ η θ ⎞ ⎠ and ζ = diag(ξ). Thus L(η, θ) = 1 2 β ⊺ Γβ -γ T β + λ β 1 + 1 2n Y 2 2 , with Γ = 1 n X⊺ X -µ 2 ζ ⊺ ζ and γ = 1 n X⊺ Y .
The optimization problem becomes then

β = arg min β∈R p 1 2 β ⊺ Γβ -γ T β + λ β 1 .
For µ ≠ 0, the matrix Γ is not positive definite. As such, we are dealing with a non convex quadratic problem. Furthermore, if Γ has negative eigenvalues, the problem is unbounded.

Thus, we need to add constraints to the optimization problem (see 3.3.1). Therefore, the problem is now given by β ∈ arg min

β∈R p 1 2 β ⊺ Γβ -γ T β + λ β 1 , such that β 1 ≤ c 0 √ s, (1.13)
where c 0 > 0 is large enough.

Furthermore, we use a well-known Restricted Eigenvalues condition (see [START_REF] Bickel | Simultaneous analysis of lasso and dantzig selector[END_REF]) on the matrix Γ. It stipulates that for some integer s, such that 1 ≤ s ≤ p, t 0 > 0, a positive number c 0 , the condition holds for the index set J 0 ∈ {1, . . . , p} and for all δ ∈ R ⋆ such that

J 0 ≤ s and δ J c 0 1 ≤ c 0 δ J 0 1 , if we have δ ⊺ Γδ δ J 0 2 2 ≥ t 0 . (1.14)
The above condition is crucial since it allows us to control the rank deficiency problem related to the definition of Γ. In Chapter 3 (see 3.3), we then derive the following result:

Theorem 1.4. We assume that the variable X takes values in a compact space. Furthermore, we also assume that the Restricted Eigenvalues condition holds. Then, there exist

t > 0, t 0 > 0 and λ 0 = X ∞ (σ + µ √ L) + µ √ L 2(1+t) log(p) n τ 0 , with τ 0 = max 2µ 2 √ L √ n √ t, 4µ 2 L n t + (σ 2 + µ 2 ) ⋅ max t n , 2δ t n
such that for 2λ 0 ≤ λ, the estimators η and θ verify

β -β 1 = θ -θ 1 + η -η 1 ≤ 12λ ⋅ s t 0
with probability greater than 1 -5 exp(-t 2 ).

Penalty function based on θ.

As in Section 1.3.2, the resolution of the optimization problem is done in a two-step process.

The index m is assumed smaller than n, but we keep n ≪ k. We put more emphasis on the estimation of θ as it is a key component of our parameter of interest. In the first step, we solve for η since it corresponds to a classic least square estimator. In the second step, the optimization problem corresponding to the estimator θ is adjusted due to the presence of measurement errors. It is tentatively given by

θ = arg min θ∈R k 1 2 θ ⊺ Γθ -γ ⊺ θ + λ θ 1 , (1.15)
where

Γ = 1 n (V Ψk ) ⊺ (V Ψk ) -1 n µ 2 (V Ψ k ) ⊺ (V Ψ k ) and γ = 1 n (V Ψk ) ⊺ Y .
Again, Γ is singaular, we have to further constrain the problem. Hence, following [START_REF] Loh | High-dimensional regression with noisy and missing data : provable guarantees with nonconvexity[END_REF],

the optimization problem (1.15) is replaced by

θ ∈ arg min θ 1 ≤c 0 √ s θ 1 2 θ ⊺ Γθ -γ ⊺ θ + λ θ 1
where c 0 > 0 is a large enough constant. We can then derive the convergence rate for our estimator (see 3.4).

Theorem 1.5. We assume that the variable X takes values in a compact space. Furthermore, we assume that the Restricted Eigenvalues condition holds.

Then, there exists t > 0, t 0 > 0 and

λ 0 = X ∞ σ + µ √ L 2(1+t) log(k) n τ 0 , with τ 0 = max 2µ 2 √ L √ n √ t, 4µ 2 L n t + (σ 2 + µ 2 ) ⋅ max t n , 2δ t n such that for λ 0 ≤ 2λ, the estimator θ verifies θ -θ 1 ≤ 12λ s θ t 0
with probability greater than 1 -5 exp(-t 2 ).

Convergence rates using Slope

We now generalize the regression model (1.12) and consider the model

⎧ ⎪ ⎪ ⎪ ⎨ ⎪ ⎪ ⎪ ⎩ Y = X ⋅ β ⋆ + W = X + U (1.16)
where Y = (Y 1 , . . . , Y n ) ⊺ is the vector of responses, X = Xij 1≤i≤n,1≤j≤p is the n × p design matrix of covariates and = ( 1 , . . . , n ) ⊺ is a Gaussian vector having centered independent and identically distributed elements, with variance

σ 2 . Furthermore, W = [W ij ] 1≤i≤n,1≤j≤p
is a n × p noisy design matrix, polluted by U ∈ R n×p . We assume here that the rows of the matrix U are drawn independently from a centered Gaussian multivariate distribution with a p × p covariance matrix denoted by C U .

Our objective is to estimate the parameter of interest β ⋆ . It is assumed s-sparse, with 0 < s ≪ p. We recall that the dataset is in high dimension and the sample size n is much smaller than p. The variables and U are assumed independent. The Slope estimator β of β ⋆ is tentatively defined through the following optimization problem

β ∈ arg min β∈R p 1 √ n Y -X ⋅ β 2 2 + β ⋆ , (1.17)
with

β ⋆ = ∑ p i=1 λ i β (i)
, known as the Slope norm, where λ 1 ≥ . . . ≥ λ p > 0 are tuning parameters. The presence of errors-in-variables requires a correction of the optimization problem (1.17).

As such, instead of (1.17) we tentatively consider

β ∈ arg min β∈R p 1 n Y -W β 2 2 -β ⊺ C U β + β ⋆ ∈ arg min β∈R p β ⊺ ( 1 n W ⊺ W -C U )β - 2 n Y ⊺ W β + 1 n Y ⊺ Y + β ⋆ ∈ arg min β∈R p β ⊺ Σβ - 2 n Y ⊺ W β + β ⋆
The problem above is not convex because the matrix Σ could have negative eigenvalues.

Thus, we propose to project Σ on the set S ≥0 of symmetric positive semi-definite matrices using the Frobenius norm. Therefore, we introduce Σ defined by

Σ ∈ arg min M ∈S ≥0 Σ -M F , which is equivalent to Σ ∈ arg min M ∈S ≥0 Σ -M 2 ,
and finally consider the convex optimization problem given by

β ∈ arg min β∈R p β ⊺ Σβ - 2 n Y ⊺ W β + β ⋆ . (1.18)
In order to derive the bounds on the convergence rates of this estimator, we need to rely on the well-known Weighted Restricted Eigenvalues condition, introduced in [4] and applied to the design X. It was a key factor in deriving our convergence rates.

The condition states that the design matrix X satisfied the W RE(s, c 0 ) condition if max j=1,...,p

Xe j n ≤ 1, and

κ = min δ∈C W RE(s,c 0 ) Xδ 2 n δ 2 > 0 (1.19)
where

C W RE(s,c 0 ) = {δ ∈ R p ∶ δ ⋆ ≤ c 0 δ ⋆ ∑ p j=1 λ 2 j } is a cone in R p .
We show that under mild conditions and if X verifies the W RE(s, c 0 ) condition, then there exists a

c 0 ∈ R such that κ 2 = min δ∈C W RE(s,c 0 ) δ ⊺ Σδ δ 2 2 > 0 (1.20) with probability at least 1 -4 n -2 exp(-t 2
2 ), where t > 0. The convergence rate for our estimator β (see 4.1) is then given by Theorem 1.6. Let s ∈ {1, . . . , p} and assume that the WRE condition holds. We choose the following tuning parameters

λ j = ⎧ ⎪ ⎪ ⎪ ⎪ ⎨ ⎪ ⎪ ⎪ ⎪ ⎩ γ ′ log(2p j) n if ∑ s j=1 log( 2p j ) ≥ R 2 n,p Rn,p √ s if ∑ s j=1 log( 2p j ) < R 2 n,p
, and assume that δ 0 < e2 -1 2 and

γ ′ ≥ 2σ(4 + √ 2).
Then, with probability at least 1 -8 n -4p 1-2 log(2 δ 0 )δ 0 , we have

β -β ⋆ 2 ≤ max C 1 s n log( 2ep s ), C 2 log 2 (1 δ 0 ) sn log(2ep s) , C 3 R n,p where t = 2 log(2 δ 0 ), C 1 = (3+4σ)γ ′ κ , C 2 = 4(4 + √ 2) 2 and C 3 = 28 κ . Furthermore, R n,p (C U , β ⋆ 2 , t) = R n,p = t 2 C U 2 n + β ⋆ 2 ⎛ ⎝ 2A n,p (C U ) + 3tp 2 log(p) n max 1≤j≤p C jj U ⎞ ⎠
where

A n,p (C U ) = c ⋅ max T r(C U ) log(pn) n , T r(C U ) C U 2 log(pn) n .
. We note that the convexity of the problem (1.18) allows to minimize overall possible values of β without restrictions. However, the rates are driven by the estimation rates of the large covariance matrix of the design in the convolution model. Under very restrictive conditions on the noise, we recover the optimal rates for estimating β ⋆ .

1.4 Computational contributions and simulation studies

Computational contributions

Our contributions are two-fold. On the one hand, we extended the TMLE package [20] so that it could cope with the estimation of the multivariate variable importance measure, that we developed in Section 1.2. The extension allows the user to derive confidence regions.

On the other hand, we have relied on a bespoke version of coordinate descent to evaluate the estimators of ψ pen . In order to conduct our numerical studies, we could have chosen a well-known algorithm such as Lars( [START_REF] Efron | Least angle regression[END_REF]). However, a majority of the problems we were faced with assumed that our datasets were subject to measurement errors. This feature is not always dealt with adequately in most of the existing solving packages. So we decided to develop our own version of the coordinate descent algorithm, known to be very efficient. By fully implementing our procedures, we could then control our setup and easily include the intricacies of our problems. Furthermore, this choice allowed us to extend our procedures initially written in R, to C++, in order to improve the overall speed.

Coordinate descent. The intuition behind Coordinate descent algorithm is quite straightforward: apply Newton Raphson algorithm consecutively and sequentially to each element of a parameter of interest of the problem until convergence. It was first used to resolve Lasso like problems in [START_REF] Fu | Penalized regressions: The bridge versus the lasso[END_REF], where it was called Modified Newton Raphson. The author analyzed the structures of bridge estimators and developed an overall approach to solve bridge regression (penalized regression with penalty function given by ∑ β j γ and γ ≥ 1).

We can also mention [START_REF] Daubechies | An iterative thresholding algorithm for linear inverse problems with a sparsity constraint[END_REF] where the authors studied a linear inverse problem assumed to have a sparse expansion. The corresponding quadratic regression problem was adjusted through weighted l α penalties, where 1 ≤ α ≤ 2. However, it is truly the articles [START_REF] Friedman | pathwise coordinate optimization[END_REF][START_REF] Tong | Coordinate descent algorithm for lasso penalized regression[END_REF] which have been a catalyst for a greater interest by the statistician community in the methodology. Since then, it has been utilized in different forms and for a large variety of domain: Block coordinate descent( [START_REF] Liu | Blockwise coordinate descent procedures for multi-task lasso, with applications to neural semantic basis discovery[END_REF]), Cyclic coordinate descent ( [START_REF] Liu | Blockwise coordinate descent procedures for multi-task lasso, with applications to neural semantic basis discovery[END_REF]) and Stochastic coordinate descent ( [START_REF] Chang | Coordinate descent method for large-scale l2-loss linear support vector machines[END_REF]) are some of the known variations.

We recall here the regression model (1.16) is given by

⎧ ⎪ ⎪ ⎪ ⎨ ⎪ ⎪ ⎪ ⎩ Y = X ⋅ β ⋆ + W = X + U
where X ∈ R n×p is a design matrix, is a centered Gaussian vector, Y ∈ R n and β ∈ R p is our parameter of interest. Furthermore, U is a n×p matrix whose elements are drawn from a p-multivariate centered Gaussian distribution of covariance C U ∈ R p . The model is set in a high dimensional framework with n ≪ p. The Lasso optimization problem corresponding to the regression model above is defined by

β = arg min β∈R p 1 n Y -X ⋅ β 2 2 + λ β 1 , (1.21)
where λ > 0. In order to account for the pollution of the dataset, (1.21) has to be adjusted.

Instead of constructing β given by (1.21), we aim for

β = arg min β∈R p 1 n Y -W ⋅ β 2 2 -β ⊺ ⋅ C U ⋅ β + λ β 1 = arg min β∈R p f (β) + ḡ(β) , (1.22)
where

ḡ(β) = λ β 1 and f (β) = 1 n Y -W ⋅ β 2 2 -β ⊺ ⋅ C U ⋅ β.
In order to solve numerically for β, using coordinate descent algorithm, one has to find the gradient of the functions f and ḡ. The gradient associated with the function f can be derived easily. However, the function ḡ is not differentiable for all β. To solve this problem, the methodology relies on a sub-gradient theory to compute ∇ḡ. We have

∂ i ḡ(β) = λ ⋅ ⎧ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎨ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎩ 1 if β i > 0 -1 if β i < 0 [-1, 1] if β i = 0
The coordinate descent algorithm can then be described as follows: Let us denote by

∇ i h(β) = ∇ i f (β) + ∇ i ḡ(β) the i th component of the gradient ∇h(β) = ∇ f (β) + ∇ḡ(β).
At each point of the iterative process, we update the i th k element of our estimate β k ; β k i k while fixing all the remaining elements. As such, (1.22) becomes a unidimensional minimization problem. We can then rely on the gradient ∇h(β k ) to produce an update of β k i k . The procedure can be briefly resumed in the following pseudo code.

Algorithm 1.1 Coordinate descent for (1.22) Set k ← 0 and choose β 0 ∈ R p . While Termination test is not satisfied do Choose index i k ∈ {1, 2, . . . , p}. β k+1 ← β k -α k [∇h(β k )] i k e i k for some α k > 0. k ← k + 1.

End While

The resolution of a Slope based optimization problem follows a similar pattern.

Simulation studies

We provide in this thesis two main approaches to estimate the variable importance measures ψ f and ψ pen that we introduce. Naturally, one would like to compare them. This is easy when f , the user-supplied function, satisfies f = Φk in which case Φk (W ) ⋅ ψ f (P 0 ) = ψ pen (P 0 ).

Let us denote by ψf and ψpen the estimators of ψ f and ψ pen in this special estimation.

One expects by design that the prediction risk linked to ψpen will be smaller than the one linked to ψf . On the contrary, the confidence regions built from ψpen will surely be less precise than the ones built from ψf .

Comparison of Prediction and Estimation Risks -No measurement errors

Given the above, we now introduce a framework, relying on the model (1.4), which will be the base case for our analysis. We assume that the function g is given by g(W ) = W and the function f , is given by f (W ) = 20 ⋅ cos(2πW ) + 5 ⋅ sin(2π2W ). We also assume that the variables W and X are drawn from a [0, 1]-uniform distribution. We then force randomly 15% of values of X to be equal to 0, so as to be in line with the constraints introduced in [START_REF] Chanang Tondji | Inference of a non-parametric covariate-adjusted variable importance measure of a continuous exposure[END_REF]. The variable Y is generated through (1.4). We use a Fourier basis for our decomposition. We generate data samples O = (X, W, Y ) of size n = {100, 200, 300, 400, 500, 600}.

The Prediction Risk calculation is done through a two-step cross-validation process.

We partition the sample O in l distinct subsets O subset , of equal size. In the firs step, we set aside one of the O subset {i} , which is used as the test set, and use the l-1 remaining subsets as training sets. For each test set, we compute the estimator θ{-i} . In the second step, we use the O subset {i} to determine how close the estimator θ{-i} is to the real value. These two steps are repeated successively l times. For each iteration calculated the difference between the estimator and the true value function using the test set. The empirical average of these l distances give us the desired Prediction and Estimation Risks.

We see from figures (1.1b) and (1.1a) that the TMLE estimators perform relatively poorly compared to Lasso based ones for very small sample size. However, as data increases, the performance, of both methods, tends to converge.

Comparison of Prediction and Estimation Risks -With measurement errors

We rely here on the model (1.12) in order to build the framework of our analysis. We assume that W is drawn from a [0, 1]-uniform distribution. The elements of Z are given by Z = X + ν where X is drawn from a [0, 1]-uniform distribution, and ν is a vector coming from a centered Gaussian distribution with standard deviation µ = 20%. As before, we force 15% of the elements of Z to be equal to 0. he variable Y is generated through (1.4).

We use a Fourier basis for our decomposition. We generate data samples O = (Z, W, Y ) of q q q q q q 100 200 300 400 500 600 size n = {100, 200, 300, 400, 500, 600}. We rely on the procedure described in the previous paragraph to infer our Prediction and Estimation Risks for this case as well.

We see from figures (1.2b) and (1.2a) that the TMLE performs poorly for small sample size and further struggles to improve even when we increase the sample size. As expected, it is biased for cases when data sample is polluted.

q q q q q q 100 200 300 400 500 600 Inference of a non-parametric covariate-adjusted variable importance measure of a continuous exposure This chapter is based on the manuscrit [START_REF] Chanang Tondji | Inference of a non-parametric covariate-adjusted variable importance measure of a continuous exposure[END_REF].

Abstract

We consider a setting where a real-valued variable of cause X affects a real-valued variable of effect Y in the presence of a context variable W . We aim at quantifying the impact of (X, W ) on Y , while making only realistic assumptions on the true data generating distribution of (W, X, Y ). To do so, we introduce a non-parametric, context-adjusted variable importance measure, whose definition relies on a user-supplied marginal structural model. It generalizes the variable importance measure introduced by [START_REF] Chambaz | Estimation of a non-parametric variable importance measure of a continuous exposure[END_REF]. We show how to infer it by targeted minimum loss estimation (TMLE), conduct a simulation study and present an illustration of its use.

Introduction

The statistical literature offers different ways to assess the importance of a variable. One way essentially consists in comparing the prediction performances of two models. The bigger model uses all the prediction variables whereas the smaller one, a subset of the bigger model, also relies on all prediction variables except the variable of interest. More algorithmic in nature, another way consists in assessing the importance of a variable by counting "the number of times" the variable of interest is used in the process of making predictions. This is the case for instance in random forests [START_REF] Breiman | Random forests[END_REF]. To the best of our knowledge, neither approach lends itself well to the construction of confidence regions.

Following the seminal article [START_REF] Van Der Laan | Statistical inference for variable importance[END_REF], a third way consists in defining the variable importance measure as a sound statistical parameter and developing tailored estimators which, by the central limit theorem, can be used to construct confidence regions. An example of such a variable importance measure is introduced and studied by [START_REF] Chambaz | Estimation of a non-parametric variable importance measure of a continuous exposure[END_REF], upon which this manuscript builds.

β ↦ E P 0 [Y -E P 0 (Y X = x 0 , W ) -msm β (X, W )] 2 ,
where x 0 is a reference value for X for which there exists 0 < c < 1 2 such that P 0 (X ≠

x 0 W ) ∈ [c, 1-c] P 0 -almost surely.
For instance, choosing a marginal structural model with B = R and msm β given by msm β (X, W ) = β(Xx 0 ) yields the non-parametric variable importance measure studied in [START_REF] Chambaz | tmle.npvi: targeted, integrative search of associations between DNA copy number and gene expression, accounting for DNA methylation[END_REF][START_REF] Chambaz | Estimation of a non-parametric variable importance measure of a continuous exposure[END_REF].

For technical reasons, we focus on marginal structural models of the form

{(X, W ) ↦ (X -x 0 )f β (W ) ∶ β ∈ B}, (2.1)
where f β is linear in β. The statistical parameter of interest is formally defined as

ψ 0 = arg min β∈B E P 0 [Y -E P 0 (Y X = x 0 , W ) -(X -x 0 )f β (W )] 2 , (2.2)
assuming that the minimizer exists and is unique. We interpret (Xx 0 )f ψ 0 (W ) as the best approximation of the form (X -

x 0 )f β (W ) to (E P 0 (Y X, W ) -E P 0 (Y X = x 0 , W )).
It quantifies the influence of X on Y , using x 0 as a reference value, while accounting for the covariates W on a linear scale.

Relevant literature. Our main source of inspiration is [START_REF] Chambaz | tmle.npvi: targeted, integrative search of associations between DNA copy number and gene expression, accounting for DNA methylation[END_REF][START_REF] Chambaz | Estimation of a non-parametric variable importance measure of a continuous exposure[END_REF]. These articles were motivated by an application to the analysis of the effect of DNA copy number variations (the exposure X) on gene expression (a response Y ) accounting for DNA methylation (the covariates W ). Their parameter of interest corresponds to the marginal structural model

{(X, W ) ↦ β(X -x 0 ) ∶ β ∈ R}. (2.3)
Contrary to [2,[START_REF] Louhimo | CNAmet: an R package for integrating copy number, methylation and expression data[END_REF][START_REF] Pollack | Microarray analysis reveals a major direct role of DNA copy number alteration in the transcriptional program of human breast tumors[END_REF][START_REF] Sun | Integrated analysis of gene expression, CpG island methylation, and gene copy number in breast cancer cells by deep sequencing[END_REF][START_REF] Van Wieringen | Non parametric testing for DNA copy number induced differential mRNA gene expression[END_REF], [START_REF] Chambaz | tmle.npvi: targeted, integrative search of associations between DNA copy number and gene expression, accounting for DNA methylation[END_REF][START_REF] Chambaz | Estimation of a non-parametric variable importance measure of a continuous exposure[END_REF] do not assume that the real-valued variable of cause X is discrete (or do not discretize it), but rather exploit the fact that X has a reference value x 0 (x 0 = 2 for DNA copy variations) and features a continuum of other values. Moreover, contrary to [START_REF] Robins | Estimating exposue effects by modelling the expectation of exposure conditional on confounders[END_REF][START_REF] Robins | Comment on inference for semiparametric models : some questions and an answer by[END_REF], [START_REF] Chambaz | tmle.npvi: targeted, integrative search of associations between DNA copy number and gene expression, accounting for DNA methylation[END_REF][START_REF] Chambaz | Estimation of a non-parametric variable importance measure of a continuous exposure[END_REF] do not assume a semi-parametric model but rather exploit one. We do too exploit the marginal structural model (2.1) and do not

assume that E P 0 (Y X, W ) -E P 0 (Y X = x 0 , W ) belongs to it.
Our parameter of interest ψ 0 (2.2) points to an element of (2.1) such that E P 0 (Y X, W )-

E P 0 (Y X = x 0 , W ) is best approximated by (X -x 0 )f ψ 0 (W ).
In words, W is not averaged out completely like in [START_REF] Chambaz | tmle.npvi: targeted, integrative search of associations between DNA copy number and gene expression, accounting for DNA methylation[END_REF][START_REF] Chambaz | Estimation of a non-parametric variable importance measure of a continuous exposure[END_REF]. Instead, the effect of X on Y is quantified as (Xx 0 ) times a function of the linear expression f ψ 0 (W ) of W . Hence, contrary to [START_REF] Chambaz | tmle.npvi: targeted, integrative search of associations between DNA copy number and gene expression, accounting for DNA methylation[END_REF][START_REF] Chambaz | Estimation of a non-parametric variable importance measure of a continuous exposure[END_REF], we are able to capture the role played by each component of W in the relationship between X and Y .

We will view ψ 0 as the value of a functional Ψ at P 0 , where Ψ maps the statistical model describing the experiment to B. Since Ψ is path-wise differentiable, we can carry out the inference of ψ 0 by targeted minimum loss estimation (TMLE) [START_REF] Chambaz | tmle.npvi: targeted, integrative search of associations between DNA copy number and gene expression, accounting for DNA methylation[END_REF][START_REF] Chambaz | Estimation of a non-parametric variable importance measure of a continuous exposure[END_REF][START_REF] Van Der Laan | Targeted maximum likelihood learning[END_REF][START_REF] Van Der Laan | Targeted Learning : Causal Inference for Observational and Experimental Data[END_REF]. The TMLE template is a stepwise procedure. Its instantiation when the negative log-likelihood is chosen as loss function can be described as follows. Set k = 0. The first step consists in defining the parameter of interest as the value of a smooth functional Ψ at P 0 . The second step consists in using machine learning to estimate some relevant features of P 0 which are needed to derive an initial substitution estimator ψ k n = Ψ(P k n ) of ψ 0 and the derivative D * (P k n ) of the functional Ψ at P k n . This initial estimator is typically biased, or does not satisfy a central limit theorem. The third step consists in defining a parametric model through P k n whose score at P k n spans D * (P k n ). The fourth step is a maximization of the log-likelihood of this parametric model, which yields an update P k+1 n of P k n , hence and updates

ψ k+1 n = Ψ(P k+1 n ), D * (P k+1 n ) of ψ k n = Ψ(P k n ), D * (P k n ).
The third and fourth steps are iterated (in k) till convergence (specifically, in our case, till ´D *

(P k n )dP n = o P (1 √ n),
where P n is the empirical measure). The TMLE estimator is the last of the updated substitution estimators, denoted Ψ(P ⋆ n ). Organization. Section 2.2 defines and studies the functional Ψ associated to ψ 0 (2.2). Section 2.3 describes the inference procedure tailored to the construction of confidence regions of a given asymptotic level for ψ 0 . Section 4.4 presents the results of a simulation study. Section 2.5 gives an illustration based on real data on climate change. Relevant materials and proofs are gathered in the appendix.

Studying the parameter of interest

Without loss of generality, we assume from now on that x 0 = 0. Differentiability and robustness. We denote ḟ = ∂ ∂β f β the gradient of f β which, by choice, does not depend on β. Denote d the dimension of the space where ḟ (W ) lives. For every possible data-generating distribution P of O, we denote θ(P )(X, W ) = E P (Y X, W ) and g(P )(W ) = P (X ≠ 0 W ). Assume that P is chosen such that

1. µ(P )(W ) = E P (X ḟ (W ) W ) and Σ(P ) = E P [X 2 ḟ (W ) ⊺ ḟ (W )] are well-defined fea- tures of P ; 2. Σ(P ) is invertible; 3. there exists c ∈]0, 1 2[ such that g(P )(W ) ∈ [c, 1 -c] P -almost surely.
Condition 3 is called a "positivity assumption". It guarantees that, in every stratum of W , the conditional probabilities of X = 0 and X ≠ 0 are larger than c > 0. Conditions 1 and 2 on the joint distribution of (X, ḟ (W )) are technical but standard. They are met if X ḟ (W ) is a bounded random variable and if there is no deterministic linear combination of the components of X ḟ (W ) which equals 0 P -almost surely. For such a P , the equation

Ψ(P ) = arg min β∈B E P θ(P )(X, W ) -θ(P )(0, W ) -Xf β (W ) 2 (2.4)
uniquely characterizes a parameter of P such that Ψ(P 0 ) = ψ 0 , if P 0 meets the constraints, which we assume from now on to be true. It occurs that Ψ(P ) rewrites as

Ψ(P ) = Σ(P ) -1 [E P (X ḟ (W )(θ(P )(X, W ) -θ(P )(0, W )))].
(2.5)

The functional Ψ is path-wise differentiable at P , with an efficient influence curve given by D ⋆ (P ) = D ⋆ 1 (P ) + D ⋆ 2 (P ) where D ⋆ 1 (P ) and D ⋆ 2 (P ) are two L 2 0 (P )-orthogonal components characterized by

D ⋆ 1 (P )(O) = Σ(P ) -1 (θ(P )(X, W ) -θ(P )(0, W ) -Xf β (W )) X ḟ (W ), and
D ⋆ 2 (P )(O) = Σ(P ) -1 (Y -θ(P )(X, W )) X ḟ (W ) - 1 {X=0} g(P )(0 W ) µ(P )(W ) .
This means that, for any bounded s ∈ L 2 0 (P ) taking values in

R d and ε ∈ R d with ε ∞ < s -1 ∞ , if we characterize a data-generating distribution P ε of O by setting dP ε dP (O) = 1 + ε ⊺ s(O),
then for ε small enough, P ε meets Conditions 1, 2, 3, hence Ψ(P ε ) is well-defined, and moreover ε ↦ Ψ(P ε ) is differentiable at ε = 0 with a derivative given by

lim ε→0 Ψ(P ε ) -Ψ(P ) ε = E P [s(O) ⊺ D ⋆ (P )(O)].
The efficient influence curve D ⋆ enjoys a remarkable property: if P, P ′ are two datagenerating distributions of O satisfying Conditions 1, 2, 3 and such that E P (D ⋆ (P ′ )(O)) = 0, then Ψ(P ) = Ψ(P ′ ) whenever θ(P ′ )(0, ⋅) = θ(P )(0, ⋅) or (µ(P ) = µ(P ′ ) and g(P ′ ) = g(P )). The validity of all the statements made in this section can be checked by adapting, mutatis mutandis, the proofs in [START_REF] Chambaz | Estimation of a non-parametric variable importance measure of a continuous exposure[END_REF].

Although not straightforward, the exercise of adapting the prior proofs was direct enough for us to decide to omit them in this document.

It is common sometimes to give a causal interpretation to a variable importance measure. It is possible to give such an interpretation in our case.

Causal interpretation. The causal interpretation partly relies on untestable assumptions. Assume, in this section only, that there exists a collection (Y x ) x∈R of random variables such that (i) (Y x ) x∈R ⊥ X W (randomization assumption), and (ii) Y = Y X (consistency assumption). The above holds for instance in the following structural equation model: there exists three deterministic functions f W , f X , f Y and three independent random

variables U W , U X , U Y such that W = f W (U W ), X = f X (W, U X ) and Y = f Y (W, X, U Y ).
In addition, assume that the conditional laws of X given W are all dominated by a common measure µ. Then, there exists a collection of conditional densities φ(⋅ W ) of X given W , all with respect to µ.

Let us denote by P the law of the full data (W, X,

(Y x ) x∈R ). It holds that E P (Y X = x, W ) = E P (Y x X = x, W ) = E P (Y x W )
, by independence of Y x and X. Furthermore, for each β ∈ B,

E P {(E P (Y X, W ) -E P (Y X = 0, W ) -Xf β (W )) 2 } = ˆEP (E P (Y x -Y 0 -xf β (W ) W )) 2 φ(x W ) µ(dx). (2.6)
proof of (2.6). The following series of equalities proves (2.6), where the third one is a consequence of Fubini's theorem:

E P {(E P (Y X, W ) -E P (Y X = 0, W ) -Xf β (W )) 2 } =E P E P (E P (Y X, W ) -E P (Y X = 0, W ) -Xf β (W )) 2 W =E P ˆ(E P (Y x W ) -E P (Y 0 W ) -xf β (W )) 2 φ(x W )µ(dx) = ˆEP (E P (Y x W ) -E P (Y 0 W ) -xf β (W )) 2 φ(x W ) µ(dx) = ˆEP (E P (Y x -Y 0 -xf β (W ) W )) 2 φ(x W ) µ(dx).
We recall here that, as aforementioned, we interpret (Xx 0 )f ψ 0 (W ) as the best approximation of the form (X -

x 0 )f β (W ) to (E P 0 (Y X, W ) -E P 0 (Y X = x 0 , W )). Thus,
given the above, Ψ(P ) can be interpreted as the coefficient associated with the regression of Y x on Y 0 + f β (x, W ) based on a weighted L 2 -loss function.

Inference

Inference is based of n independent random variables O (i) (i = 1, . . . , n) drawn from P 0 .

We infer ψ 0 = Ψ(P 0 ) by TMLE.

Initialization. The initialization consists in estimating the following features of P 0 : marginal distribution of W , µ(P 0 ), g(P 0 ), θ(P 0 ), Σ(P 0 ) and, for each of them, a companion feature required to update them at the next step [see 22, Lemma 1]. We denote P 0 n a datagenerating distribution chosen such that (i) each estimator η n of a feature η(P 0 ) among the above features of interest can be rewritten η n = η(P 0 n ), and (ii) we can sample (W, X) from P 0 n . As soon as we have built estimators of the marginal distribution of W , µ(P 0 ), g(P 0 ), θ(P 0 ) and Σ(P 0 ), we can also estimate ψ 0 and D ⋆ (P 0 ). This initial estimator of ψ 0 can be biased. The evaluation of ψ 0 is performed by Monte-Carlo simulation: we simulate B independent random variables (W (0,b) , X (0.b) ) from the marginal joint distribution of (W, X) under P 0 n , then compute

ψ 0 n = B -1 B b=1 Σ(P 0 n ) -1 X (0,b) ḟ (W (0,b) ) θ(P 0 n )(X (0,b) , W (0,b) ) -θ(P 0 n )(0, W (0,b) .
The construction of the marginal distribution of X given (W, X) ≠ 0 under P 0 n , has been subject to some modifications relatively to [START_REF] Chambaz | Estimation of a non-parametric variable importance measure of a continuous exposure[END_REF]. In fact, the conditional distribution can be any distribution whose conditional mean is deduced from µ(P 0 n ) by

E P 0 n (X ḟ (W ) X ≠ 0, W ) = µ(P 0 n )(W ) 1 -g(P 0 n )(0 W ) , (2.7)
and such that the variable Σ verifies :

E P 0 n (1 -g(P 0 n )(0 W ))E P 0 n (X 2 ḟ (W ) ⊺ ḟ (W )) = Σ(P 0 n ). (2.8)
Iterative updating. Say we have built (k -1) updates P 1 n , . . . , P k-1 n of P 0 n . The k th update goes as follows. Set 0 < ρ < 1 a constant close to 1, for instance ρ = 0.99 and, for

each ε ∈ R d , ε ∞ ≤ ρ D ⋆ (P k-1 n ) ∞ , introduce P k-1 n (ε) given by dP k n (ε) dP k-1 n (O) = 1 + ε ⊺ D ⋆ (P k-1 n )(O)
where

D ⋆ (P k-1 n )(O)
is the current estimator of the efficient influence curve. This defines a d-dimensional parametric model through P k-1 n fluctuating it in the direction of D ⋆ (P k-1 n ). We let ε k-1 n be the maximum likelihood estimator of ε in this model and characterize the k th update as

P k n = P k-1 n (ε k-1 n ).
This yields updated estimators of the features of interest in the spirit of [22, Lemma 1], since it holds that

θ(P ) = θ(P )(X, W ) + E P (Y s(O) X, W ) 1 + E P (s(O) X, W ) , µ(P )(W ) = µ(P )(W ) + E P (Xs(O) W ) 1 + E P (s(O) X, W ) , g(P )(0 W ) = g(P )(0 W ) + E P (1{X = 0}s(O) W ) 1 + E P (s(O) X, W ) , Σ(P ) = Σ(P ) + E P [D ⋆ (O)X 2 ḟ (W ) ḟ T (W )].
The k th update of ψ 0 n is obtained by simulating B independent random variables (W (k,b) , X (k,b) ) from the marginal joint distribution of (W, X) under P k n then computing

ψ k n = B -1 B b=1 Σ(P k n ) -1 X (k,b) ḟ (W (k,b) ) θ(P k n )(X (k,b) , W (k,b) ) -θ(P k n )(0, W (k,b) . (2.9)
Theorem 2.1 (Central limit theorem). Suppose that performing k n iterations of the updating procedure guarantees that P n D ⋆ (P kn n ) = o P (1 √ n). Suppose moreover that there exists a function f 1 with P 0 f 1 = 0 such that P 0 (D ⋆ (P kn n )f 1 )(D ⋆ (P kn n )f 1 ) ⊺ = o P (1), and that

Ψ(P kn n ) -ψ 0 -P 0 D ⋆ (P kn n ) = o P (1 √ n).
In addition, suppose that S n estimates consistently

E P 0 [f 1 (O)f 1 (O) ⊺ ]. Then ψ * n = Ψ(P kn n ) satisfies √ n(ψ ⋆ n -ψ 0 ) = (P n -P 0 )f 1 + o P (1), hence √ nS -1 2 n (ψ ⋆ n -ψ 0 )
converges in law to the d-multivariate Gaussian law with zero mean and identity covariance matrix. We refer the reader to [22, appendix] for the proof of a similar result.

Simulation study

Simulation scheme. We essentially reproduce the simulation framework that has been substantially developed in [START_REF] Chambaz | Estimation of a non-parametric variable importance measure of a continuous exposure[END_REF].

Let O 1 = (O W 1 , O X 1 , O Y 1 ), O 2 = (O W 2 , O X 2 , O Y 2 ) and O 3 = (O W 3 , O X 3 , O Y
3 ) be the same real data structures as in [START_REF] Chambaz | Estimation of a non-parametric variable importance measure of a continuous exposure[END_REF]Section 6.4]. Let p = (p 1 , p 2 , p 3 ) be such that p 1 , p 2 , p 3 ≥ 0 and p 1 + p 2 + p 3 = 1 and w = (w 1 , w 2 , w 3 ) be a vector of positive numbers. Let λ 0 ∶ [0, 1] → [0, 1] be a non-increasing mapping, σ 2 be a positive number and Σ 1 , Σ 3 be two 2 × 2 covariance matrices.

The sampling of a generic data structure O = (W, X, Y ) from the synthetic datagenerating distribution P s unfolds as follows. We first draw a latent class assignment U from the multinomial distribution with parameter (1, p). Conditionally on U , the first component W 1 of W is

W 1 = expit(logit(O W U ) + w U Z))
where Z is a standard normal random variable independent of U . The (d -1) remaining components of W are drawn from the Gaussian distribution with mean zero and identity covariance matrix. Finally, (X, Y ) is drawn conditionally on (U, W ):

• if U = 2, then (X, Y ) = (0, O Y 2 + λ 0 (W 1 ) + σ 2 Z ′ )
, where Z ′ is a standard normal random variable independent of (U, W, Z).

• if U ≠ 2, then (X, Y ) is drawn conditionally on (U, W ) from the bivariate Gaussian distribution with mean (O X U -O X 2 , O Y U ) and covariance matrix Σ U .
Implementation. We have substantially adapted the package [20]. The main changes concern:

• the characterization of f β in (2.1), which takes the form of a R formula;

• the adaptation of the fitting procedures of the features g, mu and sigma, and the storage of the fitted objects;

• the computation of the confidence regions.

Results of the simulation study. We considered two choices of marginal structural models (2.1): one is based on

f β (W ) = β 1 W 1 + β 2 W 2 , the other based on f β (W ) = β 1 W 1 + β 2 W 2 1 (β ∈ R 2
). The evaluation of the true value of ψ 0 for each choice of marginal structural model was performed by Monte-Carlo based on (2.5). We report the values in the second row of Table 2.1. For each choice, independently, we repeated independently B = 1000 times the simulation of a data set of sample size n = 1000 and the simulation of another data set of sample size n = 2000. We applied the TMLE procedure described in Section 2.3 to each data set, with the same choice of the fine-tune parameters as in [START_REF] Chambaz | Estimation of a non-parametric variable importance measure of a continuous exposure[END_REF] and with the option flavor="learning".

The results are summarized in Table 2.1. The empirical coverage is satisfying.

MSM f β (W 1 , W 2 ) = β 1 W 1 + β 2 W 2 f β (W 1 , W 2 ) = β 1 W 1 + β 2 W 2 1 n ψ 0,1 = 0.56 ψ 0,2 = 0 ψ 0,1 = 1.53 ψ 0,2 = -

Illustration

It is commonly agreed today that human activities have significant impact on climate.

Among others institutions, IPCC (Intergovernmental Panel on Climate Change) has been conducting exhaustive studies on the topic for decades. The effect of CO 2 emissions on climate change is now much better understood [START_REF] Ramanathan | Global and regional climate changes due to black carbon[END_REF][START_REF] Solomon | Irreversible climate change due to carbon dioxide emissions[END_REF]. However, one of the major remaining challenges is to understand which factors drive climate change. Our parameter (2.4) can prove useful in this regard.

We exploit a publicly available data set of the World Bank1 . We extract from it our

data set. It consists of n = 126 observed data-structures O 1 , . . . , O i = (W i , X i , Y i ), . . . , O n
where, for the ith country,

• W i gathers its under-five mortality rate, population growth, urban population growth, x 0 = 0 exactly all the X i smaller than 0.99, which is the 25%-quantile of the empirical distribution of X.

We assume that O 1 , . . . , O n are independently drawn from a common distribution P 0 .

We infer ψ 0 = Ψ(P 0 ) given by (2.5) for the marginal structural model {(X,

W ) ↦ Xf β (W ) ∶ β ∈ R 6 } with f β (W ) = β ⊺ W .
Using the asymptotic normality of the TMLE ψ * n , we carry out Student tests of "ψ 0,k = 0" against "ψ 0,k ≠ 0" for k = 1, . . . , 6. We reject the null for its alternative at level 5% only for k = 2, 3, i.e., for population growth and urban population growth, with p-values respectively equal to 3.69 × 10 -10 and 2.01 × 10 -8 . The corresponding estimates are ψ * n,2 = 9.30 ± 1.33 and ψ * n,3 = -8.34 ± 1.35, see also the RHS plot in Figure 2.1. In other words, we estimate

f ψ 0 (W ) with f ψ * n (W ) ≈ 9.30 × W 2 -8.34 × W 3 .
The results above teach us that only population growth and urban population growth seem to be playing key roles in the relationship between climate change and CO 2 emissions per capita.

Remark. We have carried out the same study with (W, X) corresponding to the years 1990 to 1997. The results of inference and subsequent conclusions were very similar to those presented here (results not shown).

Conclusion.

We have generalized the variable importance measure introduced in [START_REF] Chambaz | Estimation of a non-parametric variable importance measure of a continuous exposure[END_REF], through the introduction of a more general and flexible marginal structural model, where each covariate W is individually taken into account. Similarly to [START_REF] Chambaz | Estimation of a non-parametric variable importance measure of a continuous exposure[END_REF], we proved that its TMLE estimator is consistent and asymptotically convergent, under mild conditions (as mentioned before, although not straightforward, the exercise of adapting the proofs from [START_REF] Chambaz | Estimation of a non-parametric variable importance measure of a continuous exposure[END_REF] was direct enough for us to decide to omit them in this document.). Furthermore, we have extended the package [20] so that it can handle the new parameter of interest.

Chapter 3

Linear regression model with functional coefficients and errors-in-variables

This chapter is based on the manuscript [START_REF] Chanang Tondji | Linear regression model with functional coefficients and errorsin-variables[END_REF].

Abstract

We study an univariate linear regression model Y = g(W ) + X ⋅ f (W ) + , whose coefficients g and f are real-valued functions of (possibly multivariate) covariates W . We assume that g and f admit a finite dimensional expansion on some orthonormal basis in the Hilbert L 2 (W), where W ⊂ R d is the space of outcomes of the covariates. These dimensions can be large for both g and f , or just for f . We estimate g and f by a Lasso like procedure. We also study the same estimation problem in the presence of errors-in-variables, that is, instead of X we observe Z = X + ν where ν is assumed centered, with known variance and independent of X. We evaluate the behaviour of the procedure on synthetic data and compare our estimator of f with the TMLE in a semi-parametric model for f . Finally, our methodology is applied to a financial dataset in order to search for a (relatively) small portfolio to replicate the predict (daily return) of a given financial index.

Introduction

Statisticians are often confronted with the problem of selecting a subset within a large sample of variables in order to build a predictor of an outcome of interest. The selection is important because it helps choose relevant explanatory variables, necessary to define a consistent and unbiased predictor.

When dealing with high dimensional data, the selection problem becomes more acute.

Consider the case where we use all variables in the dataset to build a regression model. The resulting predictor will likely be poor. In fact, it will yield too many main regressors. One can attempt to change the functional form of the model in order to improve the performance of the predictor, but the selection of a smaller set will always be preferable. As such, many solutions have been developed in the literature to help quantify the importance of a variable in a given regression model or while attempting to define one. They can essentially be divided in two groups. The first group, generally, builds successive predictors with and without a variable (or a set of variables) of interest, respectively, which we wish to measure the importance. The difference in accuracy between both predictors will provide the measure of importance. A general, but detailed, overview of these methods is provided in [START_REF] Hocking | The analysis and selection of variables in linear[END_REF]. The second group consists of ensemble learning methods which, for some of them, relies on decision trees learning in order to define the best predictor. An early example of such algorithm is called bagging and was introduced in [START_REF] Breiman | Bagging predictors[END_REF]. A detailed comparison, of some of these methods (i.e.: bagging, boosting and randomization), is proposed in [START_REF] Dietterich | An experimental comparison of three methods for constructing ensembles of decision trees: Bagging, boosting and randomization[END_REF].

A similar method called Random Forest, which is based on a random bootstrapping of the main learning set was also introduced in [START_REF] Breiman | Random forests -random features[END_REF]. For this group of methods, two main measures are commonly used to quantify the variable importance. The first one is known as

Mean Decrease Impurity importance (MDI) or Mean Decrease Gini. It is based on the Gini impurity which is computed at each node of the decision tree to help determine how to split the data within the node into smaller datasets. Once aggregated over the entire decision tree, it produces the MDI value for each covariate. The second measure is called Mean Decrease Accuracy (MDA), also known as Mean Permutation Importance. Similarly to the first measure, at each node of the decision tree, the accuracy of the predictor is evaluated by randomly permutating a variable in the out-of-bag sample. An aggregation over the entire decision tree provides the desired value. We note however that these solutions have a few drawbacks. Tree decision methods sometimes produce optimal predictors which have very few variables, given the initial dataset (see [START_REF] Birkner | Data adaptive pathway testing[END_REF]). This is far from being a desirable feature. Random Forest is known for over-fitting learning datasets that are particularly noisy. Moreover, both of these groups do not allow the calculation of a confidence interval related to the variable importance measure.

A new type of measure was introduced in [START_REF] Van Der Laan | Statistical inference for variable importance[END_REF] to solve some of these issues. It relies on a model which establishes a relation between an outcome Y and a set of covariates W . It is defined as a real-valued parameter ϕ vdL (P ), predictor of E P (Y A, W ) and given by

ϕ vdL (P )(a) = E P ⋆ (E P (Y A = a, W ) -E P (Y A = 0, W )), (3.1)
where (W, Y ) are elements of an unknown distribution P , P ⋆ is a known function of P and A is considered to be either a subset or a function of W . This parameter is interesting because it provides a predictor which measures directly the variable importance of a covariate, and to which we can associate a p-value. A similar variable importance measure was introduced in [START_REF] Chambaz | Estimation of a non-parametric variable importance measure of a continuous exposure[END_REF], and extended to a multivariate setup in the manuscript [START_REF] Chanang Tondji | Inference of a non-parametric covariate-adjusted variable importance measure of a continuous exposure[END_REF].

Let us consider the following statistical problem. We observe the data structure O = (W, X, Y ) of an unknown distribution P 0 , containing n i.i.d elements, where W ∈ R d represents a vector of covariates, with d ≥ 1, X ∈ R represents an exposure and Y ∈ R is a response. The exposure has a reference level x 0 which is considered to be given by x 0 = 0, without loss of generality. We want to study the relationship that exists between the response Y and the exposure X, when taking into account the covariates W . Their inclusion makes sense specifically when it is not possible to rule out completely their influence in the relationship. In [START_REF] Chambaz | Estimation of a non-parametric variable importance measure of a continuous exposure[END_REF], the parameter ϕ CN vdL is given by

ϕ CN vdL (P ) = arg min b∈R E P (Y -E P (Y X = 0, W ) -X ⋅ b) 2 .
Hence, ϕ CN vdL (P ) can be seen as the coefficient of the best linear approximation of Y -

E P (Y X = 0, W ).
It is important to note here that ϕ CN vdL is a real number. As such, it only provides an aggregated view of the variable importance measure of all covariates in the model through a single real value. One might want to have a granular view of that measure.

To be specific, it might be more informative to have a variable importance measure whose elements are uniquely linked to each covariate of the problem. This gap was addressed, in [START_REF] Chanang Tondji | Inference of a non-parametric covariate-adjusted variable importance measure of a continuous exposure[END_REF], where we considered a function f given by f

(B, W ) = B ⊺ ⋅ f CT (W ), with f CT (W ) ∶ R d → R d supposed
B E P (Y -E P (Y X = 0, W ) -X ⋅ f (B, W )) 2 . (3.2)
We note that ϕ CT (P ) takes into account each covariate present in the model, when describing the relationship that exists between the exposure X and the response Y . It is important to note that the parameters ϕ CN vdL and ϕ CT were both studied through a statistical inference method called TMLE (Targeted Minimum Likelihood Estimator). A brief overview of the analysis of the parameter ϕ CT is presented in Section 3.4.2.

As aforementioned, we are interested in this article in the estimation of the function f as a mean to derive a more refined expression of the variable importance measure. By assuming f CT known in [START_REF] Chanang Tondji | Inference of a non-parametric covariate-adjusted variable importance measure of a continuous exposure[END_REF], the author was constraining his framework to a very specific form in order to define the relation that existed between the covariates. As an example, considering the case where the model had only two covariates, the user may choose to define

f CT through f (W, β) = β ⊺ ⋅ f CT (W ) = β 1 W 2 1 + β 2 W 2 2
, with W = (W 1 , W 2 ) and β = (β 1 , β 2 ). In fact, for the parameter ϕ CT to be well defined, due to the definition space of f CT , it had to be of length d. Now, by considering f unknown, we are deriving a flexible relationship between the covariates and as a consequence lifting the constraint on the size of ϕ CT . It then allows us to obtain a complete picture of the measure of variable importance linked to the covariates of the model, since we are now able, as an example, to potentially take into account a large variety of interaction terms among the covariates.

We recall that f ∶ R d → R is now the non-parametric function of interest. Keeping in mind the framework developed in [START_REF] Chanang Tondji | Inference of a non-parametric covariate-adjusted variable importance measure of a continuous exposure[END_REF], the function of interest is an extended, unknown and nonparametric form of the parametric form f (ϕ CT , W ), where f was supposed known.

Hence, it can be viewed as a real-valued variable importance measure which takes into account the impact of the covariates W . Its estimator is computed through a penalized optimization problem. Furthermore, in order to derive the estimator's convergence rate, we assume the existence of a model which defines the relationship that exists between the exposure X, the response Y and the covariates W .

Consider the model

Y = g(X, W ) + (3.3)
where is a centered Gaussian variable of variance σ 2 , independent of X and W , and g(X, W ) = g(0, W ) + f 1 (X, W ), with g(0, W ) = E P (Y X = 0, W ) and f 1 an unknown function. We further simplify the model (3.3) by analogy to (3.2). We assume that g is linear in X such that

g(X, W ) = g(0, W ) + X ⋅ f (W ). (3.4)
We may see f as the partial derivative of g with respect to X at point 0. For the rest of this article, we denote g(W ) = g(0, W ), through a slight abuse of notation, with f, g ∶ R d → R.

Let us consider Φ = (Φ 1 , ⋯, Φ p , ⋯) = {Φ j } ∞ j=1 be an orthonormal base of the Hilbert space L 2 (R d ). We define the subspace S(s, p) ⊂ H, with {s, p} ∈ N. A function h ∈ S(s, p), if the function verifies the following properties: i) can be written as h(W ) = From now on, we assume that there exists {s m , m} ∈ N and {s k , k} ∈ N such that g ∈ S(s m , m) and f ∈ S(s k , k). Hence,

g(X i , W i ) = m j=1 Φ j (W i ) ⋅ η j + X i k j=1 Φ j (W i ) ⋅ θ j , with η ∈ R m and θ ∈ R k . (3.5)
It is important to note here that, by analogy to (3.2), Φ(W ) ⋅ θ is the new variable importance measure. We note here that we could have assumed that f and g relied, for their definition, on the full list of elements of the basis Φ. However, following this hypothesis, we would have g(

W i ) = ∑ m j=1 Φ j (W i ) ⋅ η j + r gm (W i ) and f (W i ) = ∑ k j=1 Φ j (W i ) ⋅ θ j + r f k (W i )
, where the functions r f k and r gm could be considered as residuals, that are small enough under smoothness assumptions. Therefore, it would be necessary to first include the impact of these residuals on consistency and convergence rate when estimating f and g. Secondly, we would also have to derive their appropriate characteristics in order to improve the previous estimation measures. Given (3.5), the linear model (3.3) becomes

Y = X ⋅ β + (3.6)
where

β = ⎛ ⎝ η θ ⎞ ⎠ ∈ R p with p = m + k. Moreover, X is given by X = ⎡ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎣ Φ 1 (W 1 ) . . . Φ m (W 1 ) X 1 ⋅ Φ 1 (W 1 ) . . . X 1 ⋅ Φ k (W 1 ) ⋮ ⋮ ⋮ ⋮ Φ 1 (W n ) . . . Φ m (W n ) X n ⋅ Φ 1 (W n ) . . . X n ⋅ Φ k (W n ) ⎤ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎦ . (3.7)
Furthermore, we denote by Ψ m , the matrix given by the first m functions at points

W 1 , ⋯, W n Ψ m = ⎡ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎣ Φ 1 (W 1 ) . . . Φ m (W 1 ) ⋮ ⋮ Φ 1 (W n ) . . . Φ m (W n ) ⎤ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎦
and by D X the diagonal matrix with diagonal elements X 1 , ⋯, X n . Thus,

X = [Ψ m , D X Ψ k ].
A Lasso like methodology is developed and used to derive a consistent estimator β of β, an ultimately the estimator θ of θ, our main variable of interest. From now on, we consider that there exists δ > 0, a fixed constant, such that:

Φ j (W ) ≤ δ, for all W in the data structure O and j ∈ {1, . . . , p}.

Measurement errors occur in most experiments and are integral part of any scientific process. As such, the aforementioned elements (X, W, Y ) can rarely be collected without being subject to errors. We consider the case of additive error on the random variable X i , i.e. we observe Z i = X i + ν i with i = 1, ⋯, n. As such, further generalizing (3.3), we preserve the statistical framework introduced above, except from the sample's elements. Hence, the dataset is given by Õ = (W, Z, Y ) of a distribution P , such that

⎧ ⎪ ⎪ ⎪ ⎨ ⎪ ⎪ ⎪ ⎩ Y = g(X, W ) + Z = X + ν (3.9)
where ν is a centered Gaussian distribution of variance µ 2 . The variables ν and are considered independent. Our goal is to see how the measurement errors affect the estimation of g and f , as well as the prediction risk of our model. From now on, we assume, without loss of generality that, for all j, ∑ n i=1 Φ j (W i ) 2 = n.

Review of Literature

Looking at (3.4), one can realize that f is the partial derivative of g(X, W ) in X, at X = 0. Hence, estimating the function f , corresponds in this setting at finding the best approximate of the derivative of an unknown function. This problem has been studied in the literature (see [START_REF] Brabanter | Nonparametric derivatives estimation[END_REF][START_REF] Hall | Estimation of functional derivatives[END_REF][START_REF] Rondonotti | Sizer for time series : a new approach to the analysis of trends[END_REF][START_REF] Zhou | On derivative estimation in spline regression[END_REF]). However, we did not pursue this path. Given the model formulation in (3.6), we opted for a Lasso based approach.

The Least Absolute Shrinkage and Selection Operator(Lasso) was introduced in [START_REF] Tibshirani | Regression shrinkage and selection via the lasso[END_REF]. It is well known to be attractive for large and sparse set of high dimensional data which is a key characteristic of interest for the models developed in this article. The Lasso has been extensively studied in the literature. The book [START_REF] Hastie | Statistical Learning with Sparsity -The Lasso and generalizations[END_REF] provides a good overview of its different properties in both low and high dimensional settings. Some of them are worth a reminder.

In fact, within the space of linear models, [START_REF] Meinshausen | High-dimensional graphs and variable selection with the lasso[END_REF] showed that under an appropriate set of conditions, Lasso estimator is consistent. We can also cite [START_REF] Zhao | On model selection consistency of lasso[END_REF] who proved the consistency of the model selection performed by the Lasso. Finally, [START_REF] Huang | Adaptative lasso for sparse high dimensional regression[END_REF] generalized, to high dimensional settings, the oracle property of estimator based on adaptative Lasso, introduced in [START_REF] Zou | The adaptative lasso and its oracle properties[END_REF].

The latter is a L 1 weighted optimization problem, relying on a pre-computed starting point. If we suppose an initial estimator β 0 = w n j with j = 1, . . . , p n , then the least square optimization problem is given by Y -Xβ 2 2 -2λ ∑ n i=1 w n j β j where X is a matrix. The goal of the weight is to reduce the estimation bias and improve the variable selection accuracy.

Under an adaptative irrepresentable condition, [START_REF] Zou | The adaptative lasso and its oracle properties[END_REF] proved that the estimator satisfies an oracle property. Some of the properties mentioned above, are covered in this document through the analysis of (3.6), which corresponds to a linear model case. We derived the error bounds of β and then deduced the ones of the variables η and θ.

Errors-in-variables models have been studied in the literature. Let us cite the semiparametric model Y = f θ 0 (X) + ζ and Z = X + , developed in [START_REF] Butucea | New m-estimators in semi-parametric regression with errors in variables[END_REF], where the efforts were directed at attaining parametric rates for estimating the finite dimensional θ 0 . We further note that high-dimensional linear models have been studied in the presence of measurement errors. We can mention [START_REF] Rosenbaum | Sparse recovery under matrix uncertainty[END_REF], who analyzed a linear regression model given by Y = R c ⋅ θ + and Z = R c + U , where the number of covariates is much larger than the sample size and the matrix R c is polluted by the matrix U . They proved that the regular Lasso estimator, as well as the Dantzig selector produced unstable results. They then introduced a Matrix Uncertainty selector which, under certain eigenvalues restrictions applied on R c , is stable and capable of reproducing a sparse pattern of θ. Furthermore, [START_REF] Sorensen | Measurement error in lasso : Impact and likelihood bias correction[END_REF] studied a linear model using the Lasso procedure knowing that the covariates are subject to measurement errors and for p ≫ n. They proved that using a correction method and under certain conditions, the corrected Lasso produced sign consistent covariate selection.

A similar correction method is used in this document, when dealing with noisy data. [START_REF] Datta | Cocolasso for high-dimensional error-in-variables regression[END_REF] completed the study of a similar setting, by introducing a convex corrected Lasso. They established error bounds of the corresponding estimator and showed its asymptotic sign consistent selection property. Finally, we can also mention [START_REF] Loh | High-dimensional regression with noisy and missing data : provable guarantees with nonconvexity[END_REF], whose work is based on the same setting, using both multiplicative and additive type errors, through a non convex optimization problem. They established non asymptotic error bounds for the estimator and showed that a gradient descent based algorithm makes the estimator converge in polynomial time, in a neighborhood of the set of all global optimizer. We note here that for most, if not all of the aforementioned articles, measurement errors are applied on the design matrix R c . In our model (3.9), the noise is affecting the variable multiplicative of the function f . To the best of our knowledge, we have not seen an article treating of measurement errors while considering this type of noise.

Looking closely at (3.5), we realize that it provides us with a very specific model which might find its roots in the family of partial linear models. The latter are characterized by regression of the type Y = X ⋅ β + v(Z) + where β is the variable of interest, X a matrix and the function v is unknown. These models have been explored in the literature, usually through the use of polynomial splines, for estimating the non-parametric part v.The authors in [START_REF] Liu | Estimation and variable selection for semiparametric additive partial linear models[END_REF] used such a method and were able to derive the asymptotic normality of the estimators. Looking also at (3.3) and (3.4), we also realize that it is a regression model which shares some similarities with problems based on the estimation of linear functionals. We can cite for example [START_REF] Butucea | Adaptive estimation of linear functionals in the convolution model and applications[END_REF]. In the latter article, the authors considered the regression model Z i = X i + , where (X i ) i∈N and ( i ) i∈N are independent sequences of real valued random variables. They were interested in the estimation of linear functionals linked to the unknown density function of X i , studying specifically the rate of convergence of its quadratic risk. As previously mentioned, instead of using a spline methodology, we relied on a decomposition in a Hilbert space to provide a desirable form to our unknown functions and hence derived their estimators. We note also that the partial linear models have been studied in the literature when subject to measurement errors such as [START_REF] Liang | Variable selection for partially linear models with measurement errors[END_REF]. They used a non convex corrected penalized least square, as well as a penalized quantile regression to calculate its estimators. They established their convergence rate and asymptotic normality property. A lot of similarities can be drawn between our model and the partial linear model but as mentioned before, they are not identical. The studies that have been achieved so far, when the model is subject to measurement errors, usually assumed that the linear segment of the partial linear model has a fixed dimension. In our case, we do not. In fact, we assume that m can be as large as possible hence adding another layer of complexity and allowing to derive better estimator of the unknown function g. Furthermore, in contrary to both articles mentioned above, we used a Lasso type methodology to find our estimators.

The rest of this article is organized as follows. Section 3.2 establishes convergence rates of the estimator of β under two different penalty functions used to define problem (3.6). Section 3.3 establishes convergence rate of the estimator of β based on the errorsin-variables model (3.9). Two different penalty functions are also considered to define the optimization problem from their model. Section 3.4 provides a simulation study which exhibits the aforementioned characteristics of our estimators for both models (3.6) and (3.9).

We also compare these results to the ones computed through the TMLE estimator. In Section 3.5, we present an application of our methodology in finance by finding an optimal portfolio which replicates the daily return of the S&P 500 index. All proofs are developed in the Appendix 3.6.

High dimensional regression model

Let us consider the model (3.6). The variable is a vector of size n, whose elements are drawn from a centered Gaussian distribution of variance σ 2 .

Convergence rate for globally penalized Lasso

The penalized optimization problem derived from (3.6) is given by

β = ⎛ ⎝ η θ ⎞ ⎠ = arg min β∈R p 1 2n n i=1 (Y i -(Xβ) i ) 2 + λ β 1 , (3.10)
where λ > 0. In order to bound the estimation error of the variables of interest, a compatibility condition is needed. It was first coined in [START_REF] Van De Geer | The deterministic lasso[END_REF] for the resolution of a Lasso based optimization problem. As we see below, it is an additional constraint, not always verifiable, applied to the matrix X such as to derive fast convergence rate of the estimator. It is a well known condition, which has been extensively used in the literature (see [START_REF] Sorensen | Measurement error in lasso : Impact and likelihood bias correction[END_REF]).

Definition 1. Let us consider the n × p matrix X. The compatibility condition holds for the index set S 0 , having s 0 elements, if for some φ 0 > 0 and all γ ∈ R p such that with arbitrary t > 0, the estimator β of (3.10) verifies

γ S c 0 1 ≤ 3 γ S 0 1 , it holds that γ S 0 2 1 ≤ s 0 Xγ 2 2 nφ 2 0 . ( 3
1 2n X( β -β) 2 2 + λ β -β 1 ≤ 4σ 2 X 2 ∞ 8s φ 2 0 ⋅ t 2 + 2 log(p) n .
As a consequence, with the same probability,

β -β 1 ≤ 2σ X ∞ ⋅ 8s φ 2 0 ⋅ t 2 + 2 log(p) n .
Similar results could be inferred directly on the distinct estimators η and θ through a unique equation. The demonstration of the proposition has been inserted in the Appendix, for reader convenience. The convergence rate of the estimator is then of the order of

s log(p) n .

Convergence rate for partially penalized Lasso

We study here the model (3.6) by only penalizing the variable θ. As mentioned before, f (W ) can be viewed as the partial derivative in X of the function g(X, W ). As such, it is natural to assume that we need more information (hence, more coefficients) in order to fully capture its characteristics. The corresponding penalization problem is then given by

⎛ ⎝ η θ ⎞ ⎠ = arg min η,θ 1 2n n i=1 Y i -(Ψ i m ⋅ η + X i (Ψ i k ⋅ θ)) 2 + λ θ 1 . (3.12)
It can be solved in two steps. We first find the estimator η which minimizes (3.12), for any θ. It corresponds to a classical least square estimator. We assume here that m ≤ n and that the matrix Ψ ⊺ m ⋅ Ψ m = n ⋅ I m×m , with I being the identity matrix. Hence,

η(θ) = 1 n Ψ ⊺ m ⋅ (Y -(D X Ψ k ) ⋅ θ) . (3.13)
The penalization problem (3.12) can then be rewritten using (3.13) as

θ = arg min θ 1 n V ⋅ Y -Ψk ⋅ θ 2 2 + λ θ 1 , (3.14)
where

V = I n×n -1 n Ψ m ⋅ Ψ ⊺ m and Ψk = D X Ψ k .
Note that V is an orthogonal projection (V 2 = V , with V symmetric) and that rank(V ) = nm. Thus, the data V ⋅ Y lives in the image space of V . Proposition 3.2. Let us denote by s θ the sparsity factor of the variable θ. Let us assume that the compatibility condition (3.11) holds. Thus, with probability larger than

1 -2 exp(-t 2 2 ) and for λ = 2σ X ∞ t 2 +2 log(k) n
, the estimator θ is such that

1 2n V Ψk ⋅ ( θ -θ) 2 2 + λ θ -θ 1 ≤ 4σ 2 X 2 ∞ ⋅ 8s θ φ 2 0 ⋅ t 2 + 2 log(k) n
for some arbitrary t > 0. Hence,

θ -θ 1 ≤ 2σ X ∞ 8s θ φ 2 0 t 2 + 2 log(k) n .
Proposition 3.2 gives the expected rate related to the sparsity s θ . Furthermore, we note that the rate depends on s θ log(k)

n . However, the estimator of η suffers from the estimation of θ which acts here as a nuisance parameter.

E η( θ) -η 2 2 = E 1 n Ψ ⊺ m (Y -Ψk θ) -η 2 2 = E 1 n Ψ ⊺ m (Ψ m η + Ψk θ + -Ψk θ) -η 2 2 = E 1 n Ψ ⊺ m + 1 n Ψ ⊺ m Ψk (θ -θ) 2 2 ≤ 2 n 2 σ 2 T r(Ψ m Ψ ⊺ m ) + 2 n 2 E Ψ ⊺ m Ψk (θ -θ) 2 2
where T r is the Trace of the matrix. On one hand, we have 1 n T r(Ψ m Ψ ⊺ m ) = m and on the other hand,

2 n 2 E Ψ ⊺ m Ψk (θ -θ) 2 2 ≤ 2 n 2 E[(θ -θ) ⊺ Ψ⊺ k Ψ m Ψ ⊺ m Ψm (θ -θ)] ≤ 2 n E Ψk (θ -θ) 2 2 ≤ 2 n X 2 ∞ E Ψ k (θ -θ) 2 2 ≤ 2 X 2 ∞ E θ -θ 2 2 ≤ 2 X 2 ∞ E θ -θ 2 1
since for all y, y 2 ≤ y 1 .

However,

E θ -θ 2 1 = ˆ∞ 0 P θ -θ 2 1 ≥ u du ≤ 2 ˆ∞ 0 exp - 1 2 nu 4σ 2 X 2 ∞ φ 2 0 8s θ -2 log(k) du ≤ 2 8σ 2 X 2 ∞ n ( 8s θ φ 2 0 ) 2 k.
Finally, we can conclude that

E η( θ) -η 2 2 ≤ 2σ 2 m n + 4k ⋅ 8σ 2 X 4 ∞ n 8s θ φ 2 0 2 .
Discussion. We emphasize here that in both sections above, β is an aggregate of two signals : η and θ. In Section 3.2.1, the estimators of these variables of interest were computed simultaneously. Hence, the sparsity inherent in the Lasso like procedure is present by design in both variables. However, in Section 3.2.2, the objective was to estimate both signals where one is sparse, θ. The convergence rate of the estimator θ is similar in both cases. However, the convergence rate of η is negatively impacted when η is function of θ.

High dimensional regression model with errors-in-variables

All proofs are developed in Section 3.6. Let us consider the model (3.9) with g(X, W ) in (3.4) and such that g ∈ S(s m , m) and f ∈ S(s k , k). It can be written as

⎧ ⎪ ⎪ ⎪ ⎨ ⎪ ⎪ ⎪ ⎩ Y = g(W ) + Xf (W ) + Z = X + ν (3.15)
where the variables ν ⊥ X, ⊥ ν and W ⊥ ν. Moreover, elements of ν are drawn from a centered Gaussian distribution with variance µ 2 . However, similar results can be obtained for more general sub-exponential distributions. The above set of equations can be rewritten as:

⎧ ⎪ ⎪ ⎪ ⎨ ⎪ ⎪ ⎪ ⎩ Y = X ⋅ β + Z = X + K (3.16)
where

β = ⎛ ⎝ η θ ⎞ ⎠
, X is the matrix introduced in (3.6) and K is given by

K = ⎡ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎣ 0 . . . 0 ν 1 ⋅ Φ 1 (W 1 ) . . . ν 1 ⋅ Φ k (W 1 ) ⋮ ⋮ ⋮ ⋮ 0 . . . 0 ν n ⋅ Φ 1 (W n ) . . . ν n ⋅ Φ k (W n ) ⎤ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎦ .
Under a certain set of assumptions, [START_REF] Candes | The dantzig selector: Statistical estimation when p is much larger than n[END_REF] established the consistency, as well as the convergence rate of the estimator of our variable of interest. The above methodology seems to be the most natural way to resolve (3.9). However, we use the specificities of our model to provide a Lasso type estimator of

β = ⎛ ⎝ η θ ⎞ ⎠
. From now on, we consider that there exists L > 0, such that :

Ψ k (W )θ 2 2 <
L, for all W of our data structure O.

(3.17)

Convergence rate for globally penalized corrected Lasso

One could assume that even with the presence of measurement errors, the optimization problem could be written exactly as in (3.10), where the variable X is to be replaced by Z. Nonetheless, the regularization problem needs to be adjusted. In fact,

E[ Y -(Ψ m η + (D Z Ψ k )θ) 2 2 ] = E[ Y -(Ψ m η + (D X Ψ k )θ) 2 2 ] + E[ (D ν Ψ k )θ 2 2 ] -2E[ ⊺ (D ν Ψ k )θ] = E[ Y -(Ψ m η + (D X Ψ k )θ) 2 2 ] + nµ 2 E[ θ 2 2 ].
From the above, it appears that the regularization needs to be corrected by the factor

nµ 2 θ 2 2 .
As such, the estimators η and β are then defined by

⎛ ⎝ η θ ⎞ ⎠ = arg min η,θ L(η, θ) (3.18) with L(η, θ) = 1 2n Y -(Ψ m ⋅ η + (D Z Ψ k ) ⋅ θ)) 2 2 - 1 2 µ 2 θ 2 2 + λ 1 η 1 + λ 2 θ 1 ,
where λ 1 ≥ 0 and λ 2 ≥ 0. We denote by X, the equivalent of X in (3.7) where the variable X has been replaced by its counterpart Z. We also consider the vector ξ = (0, ⋯, 0, 1, ⋯, 1), of size p, where the first m elements are equal to 0 and the last k are all equal to 1.

Decomposing the loss function, we obtain:

L(η, θ) = 1 2 1 n Ψ m η + (D Z Ψ k )θ 2 2 -µ 2 θ 2 2 - 1 n ⟨Y, Ψ m η + (D Z Ψ k )θ⟩ + 1 2n Y 2 2 + λ 1 η 1 + λ 2 θ 1 = 1 2 1 n Xβ 2 2 -µ 2 ζβ 2 2 - 1 n ⟨Y, Xβ⟩ + λ β 1 + 1 2n Y 2 2 with λ = ⎛ ⎝ λ 1 λ 2 ⎞ ⎠ where β = ⎛ ⎝ η θ ⎞ ⎠ and ζ = diag(ξ). Thus L(η, θ) = 1 2 β ⊺ Γβ -γ T β + λ β 1 + 1 2n Y 2 2 where Γ = 1 n X⊺ X -µ 2 ζ ⊺ ζ and γ = 1 n X⊺ Y .
The optimization problem becomes then

β = arg min β∈R d 1 2 β ⊺ Γβ -γ T β + λ β 1 .
For µ ≠ 0, the matrix Γ is certainly not positive definite. As such, we are dealing with a non convex quadratic problem. Furthermore, if Γ has negative eigenvalues, the problem is unbounded. Thus, we have to add additional constraints to the optimization problem.

Hence, β ∈ arg min

β∈R d 1 2 β ⊺ Γβ -γ T β + λ β 1 , such that β 1 ≤ c 0 √ s β (3.19)
where c 0 > 0 is large enough, and s β is the sparsity of β. We note here that s β = s θ + s η , with s θ and s θ the sparsity attached to the variables θ and η. Furthermore, we define an assumption that is used in order to bound the estimation error. It is known as the restricted eigenvalues assumption and is discussed in [START_REF] Bickel | Simultaneous analysis of lasso and dantzig selector[END_REF].

Definition 2. For some integer s, such that 1 ≤ s ≤ p, t 0 > 0, positive number c 0 , the Restricted Eigenvalues (RE) condition holds for the index set J 0 ∈ {1, ⋯, p} and for all δ ∈ R p ⋆ such that J 0 ≤ s and δ J c 0 1 ≤ c 0 δ J 0 1 if we have:

δ ⊺ Γδ δ J 0 2 2 ≥ t 0 . (3.20)
The integer s here plays the role of an upper bound of the sparsity of the vector δ.

It is important to note that in high dimensional settings, for the least square loss function to be strongly convex, it requires the eigenvalues of the core matrix to be bounded away from 0. In our case, the matrix Γ can be rank deficient. As such, in order to resolve the optimization problem, we need to impose a strong convexity condition. We emphasize here the similarity that exists between the Restricted Eigenvalues (3.20) and the compatibility (3.11) conditions. The RE condition is actually stronger since its implies the compatibility's one. In fact,

δ 2 2 ≥ δ J 2 2 ≥ 1 s δ J 2 1 .
Theorem 3.3. (Prediction and Estimation Risk) We assume that the variable X takes values in a compact space. Furthermore, we also assume that the Restricted Eigenvalues condition (3.20) holds. Then, given (3.17), there exists t > 0, t 0 > 0 and λ 0 =

X ∞ (σ + µ √ L) + µ √ L 2(1+t) log(p) n + τ 0 , with τ 0 = max 2µ 2 √ L √ n √ t, 4µ 2 L n t + (σ 2 + µ 2 ) max t n , 2δ t n ,
where the constants δ and L are defined respectively in (3.8) and (3.17), and such that with 2λ 0 ≤ λ, the estimators η and θ verify

β -β 1 = θ -θ 1 + η -η 1 ≤ 12λ ⋅ s β t 0
with probability greater than 1 -5 exp(-t 2 ).

Convergence rate for partially penalized corrected Lasso

The optimization problem can not be the same as (3.14) with the presence of measurement errors. In fact,

E[ V (Y -(D Z Ψ k )θ) 2 2 ] = E[ V (Y -(D X Ψ k )θ) 2 2 ] + E[ V (D ν Ψ k )θ 2 2 ] -2E[ ⊺ V ⊺ (D ν Ψ k )θ] = E[ V (Y -(D X Ψ k )θ) 2 2 ] + µ 2 E[ V Ψ k θ 2 2 ].
We recall that

V = I n×n -1 n Ψ m ⋅ Ψ ⊺ m .
As such, the initial optimization problem needs to be adjusted. The estimator is then given by

θ = arg min θ 1 2n V ⋅ (Y -(D Z Ψ k ) ⋅ θ) 2 2 - 1 2n µ 2 V ⋅ Ψ k ⋅ θ 2 2 + λ θ 1 , (3.21)
where λ > 0. We introduce a loss function L(θ) defined by θ = arg min θ L(θ). Decomposing the loss function gives us

L(θ) = 1 2n V ⋅ (Y -D Z Ψ k ⋅ θ) 2 2 - 1 2n µ 2 V ⋅ Ψ k ⋅ θ 2 2 + λ θ 1 = 1 2 θ ⊺ 1 n (V D Z Ψ k ) ⊺ (V D Z Ψ k ) - 1 n µ 2 (V Ψ k ) ⊺ (V Ψ k ) θ - 1 n (V Y ) T (V D Z Ψ k )θ + λ θ 1 + V Y 2 2 .
Since V Y 2 2 is not a function of θ, we have

θ = arg min θ 1 2 θ ⊺ Γθ -γ ⊺ θ + λ θ 1 ,
where

Γ = 1 n (V Ψk ) ⊺ (V Ψk ) -1 n µ 2 (V Ψ k ) ⊺ (V Ψ k ) and γ = 1 n (V Ψk ) ⊺ Y .
The matrix Γ is degenerate in the sense that it is not positive definite. As such, the eigenvalues of the optimization problem can be negative, thus the problem is unbounded and as a consequence we need to add more constraints to the estimator. Hence following [START_REF] Loh | High-dimensional regression with noisy and missing data : provable guarantees with nonconvexity[END_REF], the optimization problem is replaced by: θ ∈ arg min

θ 1 ≤c 0 √ s θ 1 2 θ ⊺ Γθ -γ ⊺ θ + λ θ 1 , with c 0 > 0. (3.22)
Theorem 3.4. (Prediction and Estimation Risk) We assume that the variable X takes values in a compact space. Furthermore, let us assume that the Restricted Eigenvalues condition (3.20) holds. Then, given (3.17), there exists t > 0, t 0 > 0 and λ 0 =

X ∞ σ + µ √ L 2(1+t) log(k) n + τ 0 , with τ 0 = max 2µ 2 √ L √ n √ t, 4µ 2 L n t + (σ 2 + µ 2 ) max t n , 2δ t n ,
where the constants δ and L are defined respectively in (3.8) and (3.17), and such that with

λ ≥ λ 0 2 , the estimator θ verifies θ -θ 1 ≤ 12λ s θ t 0 ,
with probability greater than 1 -5 exp(-t 2 ).

Simulation study

In this section, we show the accuracy of our estimation methodology, using practical cases.

Two main risk metrics are used: Prediction Risk and Estimation Risk.

Evaluation of estimates

Let us consider a sample data containing the observations

O i = (Y i , X i , W i ) of P ⋆ , a known distribution. We assume that W ∈ [0, 1] d , X ∈ [0, 1] and Y ∈ R, with d ≥ 1.
We know that the observations X i , Y i and W i are linked through (3.3), (3.4) and also (3.6). For what follows, the variables W and X are generated from the [0, 1]-uniform distribution. We consider a single set of basis functions, for the decomposition of the functions f and g : the Fourier Basis. The simulation described here can be adapted with other basis functions such as the Spline Basis.

To evaluate the accuracy of the estimator value of a sample of size n is given by

f (W ) = Ψ k (W )⋅ θ,
P R = 1 l l i=1 1 O {i} f (W O subset {i} ) -fn-i (W O subset {i} ) 2 2 , (3.23)
where O {i} is the number of elements of the subset O subset {i} , and fn-i

(W ) = Ψ k (W ) ⋅ θi .
For a single sample size, we computed the Prediction Risk u(≥ 20) times. It allowed us to evaluate the confidence interval associated with the measure. It is important to note that for a convergent estimation procedure, the PR should converge to 0 as n increases.

We show below results for various cases : univariate (W ∈ R) , bivariate (W ∈ R 2 ) and lastly an univariate case where X is measured with errors. Furthermore, for each estimator f , corresponding to a given sample size, we compared it to its corresponding true function f . To achieve this, we constructed a grid linked to the space on which lies the variable W .

We then compared the ordinate difference between the estimator and the true function.

The empirical square difference gave us a measure of our Risk Estimation. In brief, if we consider a grid κ and an estimator fj , the Estimation Risk of our function f is given by

ER = 1 V ol(κ) κ⊂[0,1] (f (W j ) -fj (W i )) 2 , (3.24)
where V ol(κ) represents the volume of our equidistant grid (i.e.: , if κ = [0, 1] ⊆ R and we consider a grid step given by δ = 0.01, then V ol(κ) = 100.

Similarly, if κ = [0, 1] 2 ⊆ R 2 and
we consider a grid step given by δ = 0.01, then V ol(κ) = 100 2 ).

Beyond these risk measures, we also introduced a breaking point concept. It consists of lowering iteratively as much as possible one of the coefficient associated with the definition of the true function f and identifying the level at which the estimator is not able to recover it, for a given sample size. This is an important exercise as it allows us to test the quality of fit for extreme values. We now present in details three examples, to put in perspective the notions we just described.

Univariate covariate W case (d = 1)

Let us consider the functions f

(W i ) = 20⋅cos(2π ⋅5W i )+10⋅sin(2π ⋅4W i )+ √ 5 sin(2π ⋅8W i )- 7 sin(2π ⋅ 11W i ) and g(W i ) = W i .
The variables W and X are drawn from a [0, 1]-uniform distribution. We generated data samples of size n = {50, 100, 150, 200, 250, 300} and used each sample to compute the estimator of f . As shown in figure 3.1a, we computed the PR corresponding to each sample size, with u = 30. Furthermore, we used a Fourier basis of size p = 450(m = 50; k = 400). We note here that we chose a greater value for k compared with m because we wanted to put more emphasis on the estimation of the function f .

One can see that for a very small sample size, the methodology provides less accurate estimators. However, it improves rapidly as the sample size reaches n = 100.

We also show in figure 3.1b that the Estimation Risk changes with increasing sample size. As before, it converges towards 0, as expected. The Estimation Risk was computed using a grid with an incremental step δ = 0.05. It is important to note that even for small sample size (i.e: n = 50), where the results of both Estimation Risk and Prediction Risk appear relatively off compared to the other samples, the resulting estimator is still of good enough quality. We show in figure 3.2 the different estimators of f per sample size.

Remark: We tried our methodology with different variations of the function g. The results were very convincing, but are not shown here.

To further study the stability of the simulation, we have analyzed its behaviour through the aforementioned breaking point concept. In fact, we iteratively reduced the coefficient associated with the component "sin(2π ⋅ 8W i )", called B, of the function f . We wanted to see the lowest coefficient for which the methodology was still able capture its impact on the response variable Y . We ran the simulation method for three different sample sizes.

For each one, we iteratively increased the value of B, starting at 0. For each value of the coefficient B, we ran the simulation 100 times and counted the number of estimators in which the coefficient associated with the breaking point component was null.

What transpires from Table 3.1 is that, the methodology struggles to recover the effect played by the component B when its value is very small. This is magnified when the sample size is small as well. However, the greater the sample size the better the methodology is able to capture the component's effect, even for very small value. where the component associated with B is not part of the estimatorf

(W i ) = 20 ⋅ cos(2π ⋅ 5W i ) + 10 ⋅ sin(2π ⋅ 4W i ) + B ⋅ sin(2π ⋅ 8W i ) -7 sin(2π ⋅ 11W i )
significantly high value of the coefficient, the methodology performs well, even for small sample size.

Bivariate covariates W case (d = 2)

Let us consider a function

f ∶ R 2 → R such that f (W i ) = f (W 1 i , W 2 i ) = 10 ⋅ cos(2π ⋅ 2W 1 i ) ⋅ cos(2π ⋅ W 2 i ) + 4 ⋅ sin(2π ⋅ W 1 i ) ⋅ cos(2π ⋅ W 2 i ), where W i ∈ [0, 1] 2 .
We assume the function g to be defined by g

(W i ) = g(W 1 i , W 2 i ) = W 1 i + W 2 i .
The elements of the variables W and X drawn from the [0, 1]-uniform distribution as before. We computed the risk prediction values for several sample sizes n = {50, 100, 150, 200, 250, 300} and with u = 30. As one can see in figure 3.3a and 3.3b, both Estimation Risk and Prediction Risk measures decrease as the sample size increases. As before, the first sample size doesn't perform well compared to the others. The estimation risk measure was computed using 2-dimension grid with an increment given by δ = 0.05.

Univariate covariate W (d=1) with errors-in-variables

We put in perspective in this case the behaviour of the methodology assuming that the data is subject to measurement errors. As such, let us consider the function f given by

f (W i ) = 20⋅cos(2π⋅5W i )+10⋅sin(2π⋅4W i )+ √ 5 sin(2π⋅8W i )-7 sin(2π⋅11W i ) and g(W i ) = W i .
The sample data is based on (Y, Z, W ) where Z = X + ν, with ν is a centered normal distribution of standard deviation µ. The variable W is drawn from the [0, 1]-uniform distribution, as well as the variable X. The latter, combined with ν allows us to derive the values of Z. We used several sample sizes which were n = {50, 100, 150, 200, 250, 300}

and u = 30. We also used three different values of µ = {0.033, 0.066, 0.099}. This allowed us to test the accuracy of the methodology with increasing noise size. We implemented a local version of the Coordinate Descent algorithm (see [START_REF] Friedman | pathwise coordinate optimization[END_REF]), in order to solve the optimization problem and find the estimators. The implementation was completed in C++, and linked with the software R through packages Rcpp and RcppArmadillo.

We significantly increased the computation time in comparison with a similar algorithm fully set up in R.

For all different values of µ, we can see from figures 3.4b, 3.5b, 3.7b, 3.4a, 3.5a and 3.7a that the methodology performs well. However, we note that the magnitude of our risk estimation level, as well as the prediction risk level, are greater than the ones shown in section 3.4.1. Furthermore, we also see that the algorithm needs more information, said differently more data points in the sample, to reach levels seen in section 3.4.1. As before, we can see in figures 3.6 and 3.8 that the estimators of f , for all sample sizes, are of good quality.

Comparison with the TMLE estimator ϕ CT

The TMLE (Targeted Minimum Likelihood Estimation) has been used, among others, in the literature to find etimators of the variable importance measure. The latter was introduced in [START_REF] Van Der Laan | Statistical inference for variable importance[END_REF] and further generalized in [START_REF] Chambaz | Estimation of a non-parametric variable importance measure of a continuous exposure[END_REF] by assuming the covariates to be continuous.

It was further extended in the manuscript [START_REF] Chanang Tondji | Inference of a non-parametric covariate-adjusted variable importance measure of a continuous exposure[END_REF] by adjusting the definition of the parameter of interest such that it becomes multivariate. We recall here the definition of the variable q q q q q q 50 100 150 ϕ CT (3.2) which is given by ϕ CT (P ) = arg min

B∈R d E P (Y -E P (Y X = 0, W ) -X ⋅ f (B, W )) 2 ,
where f is assumed known. This definition introduces a variable importance measure whose size is equal to d, the number of covariates since ϕ CT (P ) ∈ R d . Hence, it provides a granular view of the variable importance such that each element of the parameter of interest ϕ CT (P ) can be linked to a covariate.

The TMLE procedure is an iterative method which relies on the computation of the efficient influence curve of the parameter of interest, in order to derive its consecutive estimators. The iterative procedure can be described in few steps. First, we define the parameter of interest as a smooth functional Ψ at P 0 (the true and unknown data-generating distribution). Second, using machine learning techniques, we compute the initial estimate, based on an empirical law P 0 n , which can be biased. Third, we define a submodel based on P 0 n such that its score relies on the gradient of the functional Ψ. Fourth, we maximize the log-likelihood of the submodel such that we can build an update estimate P k n . In the final phase, the last two steps are iterated until the procedure converges to an updated P ⋆ n . The final estimator is then given by ϕ ⋆ CT = Ψ(P ⋆ n ). It is shown, in [START_REF] Chanang Tondji | Inference of a non-parametric covariate-adjusted variable importance measure of a continuous exposure[END_REF], that the estimator ϕ ⋆ CT is consistent and asymptotically normal under mild conditions.

We would like in this section to evaluate the performance of a TMLE based estimator against the ones from the methodologies introduced in this document. We consider consecutively the model (3.3) and (3.9), while assuming that our function g is given by g(W ) = W . Furthermore, the function f , is given by f (W ) = 20cos(2πW ) + 5sin(2π2W ).

We also ensure that (i) P (X = 0) > δ with δ > 0, in order to respect a constraint introduced in [START_REF] Chambaz | Estimation of a non-parametric variable importance measure of a continuous exposure[END_REF] and preserved in [START_REF] Chanang Tondji | Inference of a non-parametric covariate-adjusted variable importance measure of a continuous exposure[END_REF]. We note that such a direct comparison is possible because we chose a function f which is a linear combination of elements of a Fourier basis. As such, the TMLE estimator is then computing the coefficients associated with the function.

Through our tests, we have realized that the TMLE doesn't perform well in high dimensional settings and even more so when the parameter of interest is sparse. As such, we have restricted ourself to a compact case, which should ease the performance evaluation. We assume that the number of elements in our variable of interest is much lower than our sample size (p ≪ n). We also use metrics previously introduced for our evaluation : Prediction Risk and Estimation Risk. We generate data samples of size n = {100, 200, 300, 400, 500, 600}

and decompose the function f and g using a Fourier basis such that p = 10(m = 5, k = 5).

We note here that each data sample is used for both methods. Furthermore, similarly to the previous section, we repeat our procedure u = 20 times for each data sample size, in order to find error bounds for our metrics. q q q q q q 100 200 300 400 500 600

Model without measurement errors

Let us consider the data sample named O = (X, W, Y ). The variable W and X are drawn from a [0, 1]-uniform distribution. We then force randomly 15% of values of X to be equal to 0, to be in line with condition (i). The variable Y is generated through the aforementioned functions f and g.

We see from figures (3.9b) and (3.9a) that the TMLE estimators perform relatively poorly compared to our estimators for very small sample size. However, as data increases, the performance of both methods tends to converge.

Performance evaluation for model with measurement errors

Let us consider the data sample named O err = (Z, W, Y ), where W is drawn from a [0, 1]uniform distribution. The variable Z is given by Z = X + ν where X is drawn from a [0, 1]uniform distribution, and ν is a centered Gaussian distribution with standard deviation µ = 20%. As before, we force 15% of the elements of Z to be equal to 0. The resulting variable Y will be generated through the function f and g.

We see from figures (3.10b) and (3.10a) that the TMLE performs poorly for small sample size and further struggle to improve even when we increase the sample. Clearly, it is biased for cases when measurement errors are added to the covariates. To the best of our knowledge, there are no bias corrections in the literature of TMLE for errors-in-variables q q q q q q 100 200 300 400 500 600 Discussion: The main objective in [START_REF] Chambaz | Estimation of a non-parametric variable importance measure of a continuous exposure[END_REF] is to find estimators of the variable of interest

Ψ(P ) (3.
2), using the TMLE methodology. The model developed in this document put more emphasis on finding estimator of the function f , defining the relationship that exists between the covariates W and the response variable Y . It was supposed known in [START_REF] Chambaz | Estimation of a non-parametric variable importance measure of a continuous exposure[END_REF][START_REF] Chanang Tondji | Inference of a non-parametric covariate-adjusted variable importance measure of a continuous exposure[END_REF].

Assuming a limited amount of information on the characteristics of the function f , finding a good estimator of f , can lead to a good estimator of Ψ(P ). However, for an unknown function f , the estimator of Ψ(P ) can be derived through the metric Prediction Risk. As such, we made sure to include its characteristics in the simulation cases explored above.

Application to a real data set

It is very common in financial markets for an investor to build portfolios which attempt to track the return of an existing financial index. As such, the investor has to find a selection of securities which, bundle together, recover to a certain degree of accuracy the return and risk profile of the target index. This exercise doesn't have to be a full replication of the index in question. It should not, in most cases. In fact, a full replication, which consists of buying exactly the same amount of securities which are existing in the index, is impractical and not cost effective.

Let us consider a portfolio containing p securities, where the return of the i th security at time t is denoted by r {i,t} =

P t+1 i P t i -1
, where P t i (respectively P t+1 i ) represents the closing price of the i th security at time t (respectively t+1). Furthermore, let us consider a universe of n timestamps {t 1 , ⋯, t n } where one is able to construct a n × p matrix of return, called R n p , of all the securities in the portfolio. We denote by w i , with i ∈ {1, ⋯, p}, the amount of capital invested in the i th security and by r y , the 1 × n vector of return of the target index. Thus, at time t, r y,t = P y t+1 P y t -1, where P y t (respectively P y t+1 ) is the price of the index at time t (respectively t + 1). The optimal replicating portfolio is given by

ŵ = arg min w r y -R n p ⋅ w 2 2 ,
where w = (w 1 , ⋯, w p ). It is a linear regression problem whose solution is known to be unstable. However, it can be improved by adding a L 1 regularization to the problem.

Indeed, it has several benefits among which the sparsity of the solution. When trying to replicate an index, investors are most often looking for portfolio with a low amount of securities. The reduction of all costs is key since they impact the overall return of the strategy.

As such, the replication problem is rewritten as

ŵ = arg min w r y -R n p ⋅ w 2 2 + λ w 1 (3.25)
such that λ > 0 and ∑ p i=1 w i = 1. It can be further extended by taking into account errorsin-variables in the model.

An investor trying to replicate the return profile of an index needs to take into consideration not only the open and close prices of the constituents and the index, which are in most cases publicly known, but few other factors which could impact its net return. We can mention as an example the management fees, which are periodic payments paid to the fund investment advisor. We can also mention the transaction costs which are expenses incurred when buying or selling some constituents when attempting to rebalance a portfolio.

Another point to consider is also the liquidity cost which is linked to the daily volatility of the financial instruments in the market. This price volatility can be substantial during periods of considerable market stress. As such, it is also important for investors to have access to live and historical intraday prices which sometimes come at a hefty cost. Some of these historical data are in some instances polluted. To incorporate the aforementioned elements in the regression model guiding the relationship between the return of an index and its constituents, one can assume that the model is subject to measurement errors. The article [START_REF] Rosenbaum | Sparse recovery under matrix uncertainty[END_REF] used similar considerations.

As such, we observe a matrix Z n p = R n p + Σ, where Σ is considered to be a multivariate Gaussian distribution. The replication problem can then be reformulated as

⎧ ⎪ ⎪ ⎪ ⎨ ⎪ ⎪ ⎪ ⎩ ŵ = arg min w r y -R n p ⋅ w 2 2 + λ w 1 Z n p = R n p + Σ. (3.26)
Let us consider that there exists a function g, element of a Hilbert space H, such that r y = g(r) + , where is a white noise and r = {r i } i=1,⋯,p is the vector of return of all the constituents in the initial portfolio. The estimation of the function g, leads to the precise definition of the constituents we want to keep in the replicating portfolio, as well as their corresponding weight. We consider Φ = (Φ 1 , ⋯, Φ p , ⋯) = {Φ j } ∞ j=1 , a complete Legendre orthonormal base in H. Hence, following (3.5), there exists m > 0 and β ∈ R m such that

g(w i ) = ∑ m j=0 Φ j (w i ) ⋅ β j .
Based on the matrix form developed in (3.6), we can rewrite (3.25) as

β = arg min β r y -Rn m ⋅ β 2 2 + λ β 1 (3.27)
where Rn m is a m × n matrix which correspond the expansion of R n p in the base Φ. We note that m ≫ n. Similarly as above, (3.26) can then be rewritten as

⎧ ⎪ ⎪ ⎪ ⎨ ⎪ ⎪ ⎪ ⎩ β = arg min β y -Rn m ⋅ β 2 2 + λ β 1 Z = Rn m + Σ. (3.28)
The index of interest is called Standard & Poor 's 500 (also known as S&P 500). It is an American stock market index based on the capitalizations of 500 large companies having common stock listed on the New York Stock Exchange or the NASDAQ stock exchange.

It is an extremely liquid equity index and certainly one of the most followed. Because of its broad constituency, it is considered as one of the best representation of the U.S.

stock market. In order to categorize the majorly traded public companies, the term GICS was coined in 1999 by MSCI and Standard & Poor's for use in the financial community.

GICS, which stands for Global Industry Classification Standard, consists of 11 sectors :

Energy, Materials, Industrials, Consumer Discretionary, Consumer Staples, Health Care, Financials, Information Technology, Telecommunication Services, Utilities and Real Estate. The companies in the S&P 500 covered all 11 GICS Sectors. This point will be of further importance later on. The data used to perform the analysis was retrieved from the yahoo website (https://finance.yahoo.com/quote/DATA/ ). The list of constituents of the S&P 500 is also public. The dataset extracted only contained 375 of the full 500 that could be found in the aforementioned constituent's list. We now illustrate the notions developed above through three cases : synthetic target index, S&P 500 index with direct observations and with errors-in-variables, respectively.

Case 1: We start here by constructing synthetic target indices, that we aim to track.

These synthetic indices are based on securities which are within the dataset we extracted.

We constructed a total of two synthetic indices. Each target index closing price S t is the sum of its constituents closing prices. Using all S t , we can infer the daily return of the synthetic target portfolio.

To produce replicating portfolios for both synthetic indices (see tables 3.2 and 3.3), we used the 375 constituents of the initial dataset, amongst them were the ones used to build the synthetic targets. We chose to only use two degree of freedom for the Legendre Basis decomposition of g. Two main reasons motivated this decision. First of all, it allows us to have a matrix Rn m , which not only contains the return linked to each constituent but also to their cross product of size 2. We could then measure their impact, if any. Secondly, prior analysis taking into account higher degree of freedom showed us that is was not relevant for the study to go beyond that level. The above cross products allow one to quantify the role played, if any, by the interdependence between a pair of stocks. It is a feature mostly known in the literature as the interaction term, and has been used in econometrics (see [START_REF] Ozer-Balli | Interaction effects in econometrics[END_REF]) and quantitative finance (see [START_REF] Fricke | Capital investment and stock returns: An alternative test of investment frictions[END_REF]).

We chose windows of one-year length for this study. The amount of data points available n, was then to remain well below the number of basis m. We extracted daily closing prices for our synthetic index and all the constituents of our portfolio and then deduced the daily discrete return rate for each one of them. Using Legendre basis, we built the matrix Rn m , with m = 375, n ≈ 250 (for each portfolio) and computed the measure associated with each element of our replicating portfolio. We emphasize here that the underlying Lasso methodology used also performs a variable selection. Hence, not only do we know what are the constituents to include in our replicating portfolio, but also the capital to associate to each of them. Furthermore, it outlined any interdependency between constituents, if any.

The windows considered were: 2008 and 2014. The first one corresponds to the start of the financial crisis in which we experienced a lot of volatility in the financial markets. The second is considered as a post crisis period, where volatility was low. Those two different windows are important because they allow us to apply the method in two different regimes and hence help assess the quality of the estimation methodology.

For the period 2014, we obtained a replicating portfolio whose constituents are displayed in Table 3.5. It contains all the securities available in the synthetic portfolio (see Table 3.2).

An intercept was produced with a β value of 0.00133, negligible then. For the period 2008, we obtained a replicating portfolio whose elements are in Table 3.4. The match, again, in this case is perfect as we retrieved the exact number of constituents which were in the synthetic portfolio (see Table 3.3). An intercept was produced with a β value of -0.00278, negligible as for the previous case. From the full dataset, we were able to identify the key drivers of our synthetic indices. Furthermore, we see that even in high volatility period, we are still very accurate in our reconstruction. Having constructed these portfolios, we were then interested in measuring their stability. Given the scaling profile of these replicating portfolios (β values corresponding to each constituent) provided by the decomposition, we wanted to know how the return profile performs using out of sample data relatively close to the time frame of the analysis windows.

Let us denote by {t n+1 , ⋯, t m } the out of sample time-stamps we want to use to gauge the portfolio performance. Let us denote by β i , with i ∈ {1, ⋯, p} where p represents the number of drivers infer from our methodology. If we denote by ỹ the vector containing the prediction values of our replicating portfolio, such that ỹj =

P ỹ j P ỹ j-1
-1, where P ỹ j = ∑ p i=1 β i ⋅P j i , j ∈ {t n+1 , ⋯, t m } and P j i is the closing price of the i th key driver at time j. We can see from the pictures 3.11 and 3.12 that the replicating portfolios performed well.

Case 3: The target index remains the S&P 500. We would like to replicate it within the framework defined in (3.26). We applied the same standard deviation µ to all constituents in the initial portfolio, hence defining the variable Σ. Like in prior cases, we have considered two periods for this analysis : the year 2008, at the heart of the financial crisis, characterized by a high volatility and the year 2014, which is more of a low volatility period. For each period, we used the following values for µ ∈ {0.001; 0.003; 0.005; 0.007; 0.01} which correspond to cases where the matrix of return, that we observed, is more and more polluted.

Our results (see Tables 3.7 and 3.8) show us that the greater the value of µ, which implies a heavily polluted matrix of return, the more difficult it is for the methodology to construct a well-behaved replicating portfolio. However, we can still note that even with large amount of noise in the data, the methodology constructs a portfolio whose constituents are adequately distributed among GICS sector. Hence, even through the noise, the methodology is able to recognize the role played by each sector in defining the overall behavior of the target index.

Appendix

Proof of Proposition 3.1: Given (3.10), we get

1 2n Y -X β 2 2 + λ β 1 ≤ 1 2n Y -Xβ 2 2 + λ β 1
for all β ∈ R p . This can be rewritten as

1 2n X( β -β) 2 2 + λ β 1 ≤ 1 n ⊺ X( β -β) + λ β 1 .
We write, through Hölder inequality,

1 n ⊺ X( β -β) ≤ 1 n ⊺ X ∞ β -β 1
and consider the event

A = { 1 n ⊺ X ∞ ≤ λ 0 }. On A, we get 1 2n X( β -β) 2 2 ≤ λ 0 β -β 1 + λ β 1 -λ β 1 .
We denote by S = {j ∶ β j ≠ 0} and by N = {j ∶ β j = 0} and we split the vector ββ and β according to the sets S and N . Note also that β 1 = β S 1 , thus

1 2n X( β -β) 2 2 ≤ λ 0 ( β -β) S 1 + λ 0 ( β -β) N 1 + λ β S 1 -λ βS 1 -λ βN 1 .
We remark that βS -

β S 1 ≥ β S 1 -βS 1 , which implies that -λ βS 1 ≤ -λ β S 1 + λ ( β - β) S 1 . Hence, 1 2n X( β -β) 2 2 ≤ (λ 0 + λ) ( β -β) 1 + λ 0 ( β -β) N 1 -λ βN 1 ≤ (λ 0 + λ) ( β -β) S 1 + (λ 0 -λ) βN 1 ≤ 3λ 2 ( β -β) S 1 - λ 2 βN 1 for λ 0 ≤ λ 2 .
In particular, we obtained here that ( ββ)

N 1 = βN 1 ≤ 3 ( β -β) 1 .
We used here the compatibility condition (3.11) and apply it to ββ and get

( β -β) S 1 ≤ √ s φ 0 √ n X( β -β) 2 . We then have 1 2n X( β -β) 2 2 + λ 2 β -β 1 ≤ 3λ 2 ( β -β) S 1 - λ 2 βN 1 + λ 2 ( β -β) S 1 + λ 2 βN -β N 1 ≤ 2λ ( β -β) S 1 ≤ 2λ √ s φ 0 √ n X( β -β) 2 .
We use the inequality 2ab ≤ a 2 4 + 4b 2 to get

1 2n X( β -β) 2 2 + λ 2 β -β 1 ≤ 1 4n X( β -β) 2 2 + 4λ 2 s φ 2 0 and this gives 1 2n X( β -β) 2 2 + λ β -β 1 ≤ 8λ 2 s φ 2 0 . For λ = 2σ X ∞ t 2 +2 log(p) n
, we can then conclude that

1 2n X( β -β) 2 2 + λ β -β 1 ≤ 4σ 2 X 2 ∞ 8s φ 2 0 t 2 + 2 log(p) n and β -β 1 ≤ 2σ X ∞ ⋅ 8s φ 2 0 ⋅ t 2 + 2 log(p) n .
We note that the choice of λ is such that λ ≥ 2λ 0 and that A holds with high probability

P (A c ) = P (max 1≤j≤p 1 n ⊺ X j > λ 0 ) ≤ 2 exp(- t 2
2 )

where

λ 0 = 2σ X ∞ t 2 +2 log(p) n with t > 0.
The above inequality relies on what follows. It is used at several occasions in the demonstrations below.

We note here that if we assume Z ∼ N (0, 1). Hence, for all t > 0

P (Z > t) = 1 2 √ π ˆ∞ t exp(- x 2 2 )dx ≤ 1 2 √ π ˆ∞ t x t exp(- x 2 2 )dx = 1 2t √ π exp(- t 2 2
).

Proof of Proposition 3.2:

We repeat here classical arguments for proving convergence rates for (3.14). By definition, for all θ ∈ R k , we have

1 2n V ⋅ (Y -Ψk θ) 2 2 + λ θ 1 ≤ 1 2n V ⋅ (Y -Ψk θ) 2 2 + λ θ 1 ,
which can be rearranged into

1 2n ( V Ψk θ 2 2 -V Ψk θ 2 2 ) ≤ 2 2n ⟨V Y, V Ψk ( θ -θ)⟩ + λ θ 1 -λ θ 1
giving also

1 2n V Ψk ( θ -θ) 2 2 ≤ 2 2n ⟨V (Y -Ψk θ), V Ψk ( θ -θ)⟩ + λ θ 1 -λ θ 1
We know that V (Y -Ψk θ) = V (Ψ m η + ) = V . Indeed, V Ψ m = 0. Therefore,

1 2n V Ψk ( θ -θ) 2 2 ≤ 2 2n ⊺ V Ψk ( θ -θ) + λ θ 1 -λ θ 1 ≤ 1 n ⊺ V Ψk ∞ θ -θ 1 + λ θ 1 -λ θ 1 through Hölder inequality.
Let us assume that 1 n ⊺ V Ψk ∞ ≤ λ 0 on some event A. However, we split the coordinates of θ in S θ = {j ∶ θ j ≠ 0} and N θ = {j ∶ θ j = 0}. Thus,

1 2n V Ψk ( θ -θ) 2 2 ≤ λ 0 θS -θ S 1 + λ 0 θN 1 + λ θ S 1 -λ θS 1 -λ θN 1
Using θSθ S 1 ≥ θ S 1 -θS 1 , we then get

1 2n V Ψk ( θ -θ) 2 2 ≤ (λ 0 + λ) θS -θ S 1 + (λ 0 -λ) θN 1 3λ 2 θS -θ S 1 - λ 2 θN 1 for λ 0 ≤ λ 2 .
Given the compatibility condition (3.11), we know that there exists φ 0 > 0 and s θ the sparsity of θ such that for all θ in the cone θ N 1 ≤ 3 θ S 1 , we have

θ S 2 1 ≤ s θ φ 2 0 V Ψk θ 2 2 n .
We can then say that

1 2n V Ψk ( θ -θ) 2 2 + λ 2 θ -θ 1 ≤ 2λ θS -θ S 1 ≤ 2λ √ s θ φ 0 √ n V Ψk ( θ -θ) 2 .
We use the inequality 2ab ≤ a 2 4 + 4b 2 to infer that

1 2n V Ψk ( θ -θ) 2 2 + λ 2 θ -θ 1 ≤ 1 4n V Ψk ( θ -θ) 2 2 + 4λ 2 s θ φ 2 0 thus, 1 2n V Ψk ( θ -θ) 2 2 + λ θ -θ 1 ≤ λ 2 8s θ φ 2 0 . For λ = 2σ X ∞ t 2 +2 log(k) n
, we can then conclude that

1 2n V Ψk ( θ -θ) 2 2 + λ θ -θ 1 ≤ 4σ 2 X 2 ∞ 8s θ φ 2 0 ⋅ t 2 + 2 log(k) n .
Furthermore,

θ -θ 1 ≤ 2σ X ∞ 8s θ φ 2 0 ⋅ t 2 + 2 log(k) n .
This choice of λ ≥ 2λ 0 is such that P (

1 n ⊺ V Ψk ∞ > λ 0 ) ≤ 2 exp(-t 2
2 ) for some t > 0. We note that

⊺ V Ψk ∞ = ( ⊺ V Ψk ) j = max j=1,...,k n i=1 i [V Ψk ] ij . we have ∑ n i=1 i [V Ψk ] ij ∼ N (0, σ 2 v 2 j ) and v 2 j = n i=1 [V Ψk ] 2 ij = n i=1 X 2 i [V Ψ k ] 2 ij ≤ max i X i 2 (V Ψ k Ψ ⊺ k V ⊺ ) ij ≤ X 2 ∞ (Ψ ⊺ k V 2 Ψ k ) jj ≤ X 2 ∞ (Ψ ⊺ k V Ψ k ) jj since V 2 = V , and 0 ≤ V ≤ I (the identity matrix) ≤ X 2 ∞ (Ψ ⊺ k Ψ k ) jj = n X 2 ∞ .
Hence, we have

⊺ V Ψk ∞ ≤ σ X ∞ 2n(1 + t) log(k) with probability larger than 1 - exp(-t 2 2 ).
Proof of Theorem 3.3:

We denote by L = 1 2 β ⊺ Γβγ T β + λ β 1 the loss function to be minimized. We denote by β the true value of our variable of interest. Hence, we know that β is feasible but the estimator β is optimal for (3.19). Hence, L( β) ≤ L(β). Thus,

β⊺ Γ β -β ⊺ Γβ ≤ 2⟨γ, β -β⟩ + 2λ( β 1 -β 1 ). (3.29)
Let us denote define the variable ∆ = ββ. We have

∆ ⊺ Γ∆ = ∆ ⊺ Γ β -∆ ⊺ Γβ = β⊺ Γ β -β ⊺ Γ β + β⊺ Γβ -β ⊺ Γβ -2∆ ⊺ Γβ.
Then,

∆ ⊺ Γ∆ + 2∆ ⊺ Γβ = β⊺ Γ β -β ⊺ Γβ.
Thus (3.29) can be rearranged into,

∆ ⊺ Γ∆ ≤ 2⟨∆, γ -Γβ⟩ + 2λ( β 1 -β 1 ). (3.30)
Let us now turn our attention to the term ⟨∆, γ -Γβ⟩. Using Hölder inequality, it can be written,

⟨∆, γ -Γβ⟩ ≤ γ -Γβ ∞ ∆ 1 .
Furthermore, we know that

γ -Γβ ∞ = 1 n X⊺ Y -X⊺ Xβ + nµ 2 ζβ ∞ , since ζ 2 = ζ ⊺ ζ = ζ. (3.31)
We note that,

X⊺ X = X⊺ X + X⊺ K = X ⊺ X + X ⊺ K + (X ⊺ K) ⊺ + K ⊺ K, moreover, U 1 = (X ⊺ K) ⊺ = [0 n×m , D ν Ψ k ] ⊺ [Ψ m , D X Ψ k ] = ⎡ ⎢ ⎢ ⎢ ⎢ ⎣ 0 m×m 0 m×k (D ν Ψ k ) ⊺ Ψ m (D ν Ψ k ) ⊺ D X Ψ k ⎤ ⎥ ⎥ ⎥ ⎥ ⎦
and

U 2 = K ⊺ K = [0 n×m , D ν Ψ k ] ⊺ [0 n×m , D ν Ψ k ] = ⎡ ⎢ ⎢ ⎢ ⎢ ⎣ 0 m×m 0 m×k 0 k×k (D ν Ψ k ) ⊺ D ν Ψ k ⎤ ⎥ ⎥ ⎥ ⎥ ⎦ . Thus, X⊺ X = X ⊺ X + U 1 + U ⊺ 1 + U 2 .
Knowing that Y = Xβ + and the previous expression of X⊺ X, we can then write (3.31)

γ -Γβ ∞ = 1 n X ⊺ + K ⊺ -U ⊺ 1 β -U ⊺ 2 β + nµ 2 ζβ ∞ ≤ 1 n (X ⊺ + K ⊺ ) ∞ + 1 n U ⊺ 1 β ∞ + 1 n (Ψ ⊺ k D ν 2 Ψ k -Ψ ⊺ k D µ 2 Ψ k )θ ∞ ≤ 1 n X ⊺ ∞ + 1 n K ⊺ ∞ + 1 n U ⊺ 1 β ∞ + 1 n Ψ ⊺ k D ν 2 -µ 2 Ψ k θ ∞ .
In order to find an upper bound of γ -Γβ ∞ , we look individually at each of the four terms of the inequality above and find their corresponding upper bound.

(i) We know that X ⊺ ∞ = max j=1,...,p ∑ n i=1 i X i Φ j (W i ) ∼ N (0, σ 2 d 2 j ). The variable d j can be bounded as follows :

d 2 j = ∑ n i=1 X i Φ 2 j ≤ n X 2 ∞ .
Hence, we can conclude that,

1 n X ⊺ ∞ ≤ σ X ∞ 2(1+t) log(p) n
with probability greater than 1exp(-t 2 2 ). (ii) We have

K ⊺ ∞ = max j=1,...,k n i=1 K m+j i ≤ max j=1,...,k n i=1 i ν i Φ j (W i ) ≤ max j=1,...,k 1 4 n i=1 ̃ 2 i Φ j (W i ) + max j=1,...,k 1 4 n i=1 ν2 i Φ j (W i ) (3.32) with νi = i -ν i ∼ N (0, σ 2 + µ 2 ) , ̃ i = i + ν i ∼ N (0, σ 2 + µ 2
) and using the equality xy =

1 4 {(x + y) 2 -(x -y) 2 }. Let us introduce the variable N j = 1 4n ∑ n i=1 ̃ 2 i Φ j (W i ) = 1 4n ∑ n i=1 ̃ 2 i B ij with B ij = Φ j (W i ) .
Let us found from above the Laplace transform of N j :

E exp λ 4n n i=1 ̃ 2 i B ij = n i=1 E exp λ 4n ̃ 2 i B ij = n i=1 E exp λα 2 4n B ij ⋅ ̃ 2 i α 2 with α 2 = σ 2 + µ 2 ≤ n i=1 exp(2( λα 2 4n B ij ) 2 ) ≤ exp 1 2 λ 2 α 4 4n 2 n i=1 B 2 ij with λ subject to λ ≤ n α 2 B ij ∀i = 1, . . . , n ≤ exp 1 2 λ 2 α 4 4n with λ ≤ n α 2 δ
with the constant δ > 0 which satisfies φ j (W i ) ≤ δ for all i and j. Note that such a constant exists by construction. We know that if Z is a sub-exponential variable with non-negative parameters (κ, b), then

P Z -E[Z] ≥ max{κ √ u, bu} ≤ exp - u 2
, for all u > 0.

(3.33)

Given the expression of the Laplace transform, we obtained that N j is a sub-exponential random variable with parameter κ = α 2

√

n and b = α 2 δ n . Hence, for all j,

N j ≤ max α 2 2 t n , α 2 δ t n ,
for t > 0, with probability greater than 1exp(-t 2 ). The same analysis provides the same bounds for the second term in (3.32). We then have,

1 n K ⊺ ∞ ≤ (σ 2 + µ 2 ) max t n , 2δ t n
with probability larger than 1 -2 exp(-t 2 ).

(iii) Let us denote by

T j = ∑ n i=1 ν i Φ j (W i )[Ψ k θ] i and S j = ∑ n i=1 ν i X i Φ j (W i )[Ψ k θ] i , such that U ⊺ 1 β ∞ = max{max j=1,.
..,p T j , max j=1,...,p S j }. We know that for all j, T j ∼ N (0, µ 2 v 2 j ). The variance v 2 j can be bounded as follows :

v 2 j = n i=1 (Φ j (W i ) 2 [Ψ k θ] i ) 2 = Φ ⊺ j Ψ k θθ ⊺ Ψ ⊺ k Φ j ≤ T r(Ψ k θθ ⊺ Ψ ⊺ k )T r(Φ j Φ ⊺ j ) ≤ nL since k l=1 (Φ l (W i )θ l ) 2 ≤ L by hypothesis.
Similarly, we know that for all j, S j ∼ N (0, µ 2 v 2 j ). As before, we show that

v 2 j ≤ X 2 ∞ max i [Φ j (W i )(Ψ k θ) i ] 2 = X 2 ∞ nL.
As such, we can write that (iii),

1 n U ⊺ 1 β ∞ ≤ µ(1 + X ∞ ) 2(1+t)L log(p) n
with probability greater than 1exp(-t 2 2 ). (iv) Let us denote by M j the j th element of the vector

1 n [Ψ ⊺ k D ν 2 -µ 2 Ψ k θ] such that M j = 1 n [Φ j D ν 2 -µ 2 (Ψ k θ)] j = 1 n ∑ n i=1 (ν 2 i -µ 2 )V ij where V ij = ∑ k l=1 Φ l (W i )θ l Φ j (W i ).
Applying the Laplace transform to the variable M j , we obtain

E[exp( λ n n i=1 (ν 2 i -µ 2 )V ij )] = n i=1 E[exp( λ n (ν 2 i -µ 2 )V ij )] = n i=1 E[exp( λµ 2 n ( ν 2 i µ 2 -1)V ij )] with ν 2 i µ 2 ∼ χ 2 1 = n i=1 exp( λµ 2 n V ij ) 1 -2V ij λµ 2 n ≤ n i=1 exp( 1 2 4λ 2 µ 4 n 2 V 2 ij ) with λ µ 2 n V ij < 1 4 , ∀i = 1, . . . , n since, max i V ij ≤ max i φ j (W i ) k l=1 φ l (W i )θ l ≤ δ Ψ k (W )θ 2 2 ≤ δL, and
1 n n i=1 V 2 ij ≤ L by hypothesis .
We have,

E[exp( λ n n i=1 (ν 2 i -µ 2 )V ij )] ≤ exp( 1 2 4λ 2 µ 4 n L) for λ ≤ n 2µ 2 δL
Given the expression of the Laplace Transform, we know that we are dealing with a subexponential random variable with parameter κ = 2µ 2 √ L √ n and b = 4µ 2 δL n . Hence, from (3.33), we can conclude that, for all j, M j ≤ max{ 2µ 2 √ L √ n √ t, 4µ 2 L n t} with probability greater than 1exp(-t 2 ).

From the results (i), (ii), (iii) and (iv), we can then conclude that, with probability greater than 1 -5 exp(-t 2 ), with t large enough (we note here that P (A ∩ B) ≥ P (A) + P (B) -1). Hence, there exists λ 0 ∈ R such that γ -Γβ ∞ ≤ λ 0 , with

λ 0 = σ X ∞ + µ √ L(1 + X ∞ ) 2(1 + t) log(p) n + max 2µ 2 √ L √ n √ t, 4µ 2 δL n t +(σ 2 + µ 2 ) ⋅ max t n , 2δ t n .
Thus (3.30) can be rewritten as,

∆ ⊺ Γ∆ ≤ 2λ 0 ∆ 1 + 2λ( β 1 -β 1 ).
We know that S = {j ∶ β j ≠ 0} and by N = {j ∶ β j = 0}. Hence,

λ( β -β 1 ) ≤ λ β S -λ βS 1 -λ βN 1 since β N = 0 ≤ λ βS -β S 1 -λ βN 1 therefore, ∆ ⊺ Γ∆ ≤ 2(λ 0 + λ) βS -β S 1 + 2(λ 0 -λ) βN -β N 1 ≤ 3λ βS -β S 1 -λ βN -β N 1 , assuming λ 0 ≤ λ 2 ≤ 3λ βS -β S 1
From the above, we can infer that ∆ verifies the cone property with c 0 = 3. Given the Restricted Eigenvalues condition (3.20), we can write that ∆

⊺ Γ∆ ≥ t 0 ∆ S 2 2 . Since, ∆ S 2 1 ≤ s β ∆ S 2 2 , then we have ∆ ⊺ Γ∆ ≥ t 0 s β ∆ S 2
1 . Then, from the above, we have:

t 0 s β ∆ S 2 1 ≤ 3λ ∆ S 1 , then ∆ S 1 ≤ 3λ s β t 0 . Furthermore, we know that ∆ N 1 ≤ c 0 ∆ S 1 , thus ∆ 1 ≤ (c 0 + 1) ∆ S 1 . We can then conclude that β -β 1 ≤ λ ⋅ s β ⋅ 12 t 0
with probability larger than 1 -5 exp(-t 2 ), and t large enough.

Proof of Theorem 3.4:

Let us denote by θ the true value of our parameter of interest. We know that θ is feasible and θ is the optimal parameter for (3.22). Hence, L( θ) ≤ L(θ). Thus,

θ⊺ Γ θ -θ ⊺ Γθ ≤ 2⟨γ, θ -θ⟩ + 2λ( θ -θ 1 ).
It can be rearranged into,

∆ ⊺ Γ∆ ≤ 2⟨∆, γ -Γθ⟩ + 2λ( θ -θ 1 )
, similarly to (3.30) and with ∆ = θθ. (3.34)

Through Hölder inequality, we can write,

⟨∆, γ -Γβ⟩ ≤ γ -Γβ ∞ ∆ 1 . Furthermore, knowing that V = V Y -V D X Ψ k θ we have, γ -Γθ ∞ = 1 n (V D Z Ψ k ) ⊺ Y -(V D Z Ψ k ) ⊺ (V D Z Ψ k )θ + µ 2 (V Ψ k ) ⊺ (V Ψ k )θ ∞ = 1 n (D Z Ψ k ) ⊺ V -(D Z Ψ k ) ⊺ V D ν Ψ k θ + µ 2 Ψ ⊺ k V Ψ k ∞ = 1 n Ψ ⊺ k D Z V -Ψ ⊺ k V D X D ν Ψ k θ + Ψ ⊺ k D ν 2 -µ 2 V Ψ k ∞ ≤ 1 n Ψ ⊺ k D Z V ∞ + 1 n Ψ ⊺ k V D X D ν Ψ k θ ∞ + 1 n Ψ ⊺ k D ν 2 -µ 2 V Ψ k ∞ ≤ 1 n Ψ ⊺ k D X V ∞ + 1 n Ψ ⊺ k D ν V ∞ + 1 n Ψ ⊺ k V D X D ν Ψ k θ ∞ + 1 n Ψ ⊺ k D ν 2 -µ 2 V Ψ k ∞
In order to find an upper bound of γ -Γθ ∞ , we look individually at each of the four terms of the inequality above and find their corresponding upper bound.

(i) We know that Ψ ⊺ k D X V ∞ ≤ max j=1,...,k ∑ n i=1 i X i Φ j (W i ) ∼ N (0, σ 2 d 2 j
). The variable d j can be bounded as follows :

d 2 j = ∑ n i=1 X 2 i (Φ ⊺ k V ) 2 ij ≤ n X 2 ∞ since V 2 = V and 0 ≤ V ≤ I (Identity matrix). Hence, we can conclude that 1 n Ψ ⊺ k D X V ∞ ≤ σ X ∞ 2(1+t) log(k) n
with probability greater than 1exp(-t 2 2 ). (ii) We also have,

Ψ ⊺ k D ν V ∞ ≤ max j=1,...,k n i=1 i ν i Φ j (W i ) ≤ max j=1,...,k n i=1 i ν i Φ j (W i ) ≤ max j=1,...,k 1 4 n i=1 ̃ 2 i Φ j (W i ) + max j=1,...,k 1 4 n i=1 ν2 i Φ j (W i ) (3.35) with νi = i -ν i ∼ N (0, σ 2 + µ 2 ) , ̃ i = i + ν i ∼ N (0, σ 2 + µ 2
) and using the equality xy =

1 4 {(x + y) 2 -(x -y) 2 }. Let us introduce the variable N j = 1 4n ∑ n i=1 ̃ 2 i Φ j (W i ) = 1 2n ∑ n i=1 ̃ 2 i B ij with B ij = Φ j (W i ) . Let us found from above the Laplace transform of N j : E exp λ 4n n i=1 ̃ 2 i B ij = n i=1 E exp λ 4n ̃ 2 i B ij = n i=1 E exp λα 2 4n B ij ⋅ ̃ 2 i α 2 with α 2 = σ 2 + µ 2 ≤ n i=1 exp(2( λα 2 4n )) 2 ≤ exp 1 2 λ 2 α 4 4n 2 n i=1 B 2 ij with λ subject to λ ≤ n α 2 B ij , ∀i = 1, . . . , n ≤ exp 1 2 λ 2 α 4 4n with λ ≤ n α 2 δ
with the variable δ > 0 an assumption of the theorem which satisfies φ j (W i ) ≤ δ for all i and j. Note that such a constant exists by construction. Given the expression of the Laplace transform, we obtained N j is a sub-exponential variable with parameter κ = α 2 2 √ n and b = α 2 δ n . Hence, given (3.33), for all j, N j ≤ max α 2 2 t n , α 2 δ t n for t > 0, with probability greater than 1exp(-t 2 ). The same analysis provides the same bounds for the second term in (3.35). We then have

Ψ ⊺ k D ν V ∞ ≤ (σ 2 + µ 2 ) ⋅ max t n , 2δ t n with probability greater than 1 -2 exp(-t 2 ). (iii) We also know that Ψ ⊺ k V D X D ν Ψ k θ ∞ = max j=1,...,k ∑ n i=1 ν i X i Φ j (W i )[V Ψ k θ] i = max j=1,.
..,k T j . We can infer that T j ∼ N (0, µ 2 v 2 j ). The variable v j can be bounded as follows

v 2 j = n i=1 X 2 i Φ j (W i ) 2 [V Ψ k θ] 2 i ≤ X 2 ∞ max i [Φ j (W i )(V ψ k θ)] 2 ≤ X 2 ∞ (θ ⊺ Ψ ⊺ k V ⊺ Φ j V Ψ k θ) jj ≤ X 2 ∞ nL where ( k l=1 Φ l (W l )θ l ) 2 ≤
L by hypothesis and 0 ≤ V ≤ I(Identity matrix).

Hence, we can conclude that,

1 n Ψ k V D X D ν Ψ k θ ∞ ≤ µ √ L X ∞ 2(1+t) log(k) n
with probability greater than 1exp(-t 2 2 ).

(iv) Let us denote by M j the j th element of the vector

1 n [Ψ ⊺ k D ν 2 -µ 2 V Ψ k θ] such that M j = 1 n [Φ j D ν 2 -µ 2 (V Ψ k θ)] j = 1 n ∑ n i=1 (ν 2 i -µ 2 )G ij where G ij = (V Ψ k θ) i Φ j (W i ).
Applying the Laplace transform to the equation above, we obtain

E[exp( λ n n i=1 (ν 2 i -µ 2 )G ij )] = n i=1 E[exp( λ n (ν 2 i -µ 2 )G ij )] = n i=1 E[exp( λµ 2 n ( ν 2 i µ 2 -1)G ij )] with ν 2 i µ 2 ∼ χ 2 1 = n i=1 exp( λµ 2 n G ij ) 1 -2G ij λµ 2 n ≤ n i=1 exp( 1 2 4λ 2 µ 4 n 2 G 2 ij ) with λµ 2 n G ij < 1 4 , ∀i = 1, . . . , n since, max i G ij ≤ δ V Ψ k (W )θ 2 2 ≤ δL and 1 n n i=1 G 2 ij ≤ L by hypothesis hence, E[exp( λ n n i=1 (ν 2 i -µ 2 )G ij )] ≤ exp( 1 2 4λ 2 µ 4 n L) since 1 n n i=1 (G ij ) 2 ≤ L.
Given the expression of the Laplace transform, we know that we are faced with a subexponential random variable with parameter κ = 2µ 2 √ L √ n and b = 4µ 2 δL n . Hence, from (3.33), we can conclude that, for all j, M j ≤ max 2µ 2 √ L √ n √ t, 4µ 2 L n t with probability greater than 1exp(-t 2 ). From the results (i), (ii), (iii) and (iv), we can conclude that with probability greater than 1 -5 exp(-t 2 ), with t large enough (we note here that P (A ∩ B) ≥ P (A) + P (B) -1). Hence, there exists λ 0 ∈ R such that γ -Γθ ∞ ≤ λ 0 , with

λ 0 = X ∞ (µ √ L + σ) 2(1 + t) log(k) n + max 2µ 2 √ L √ n √ t, 4µ 2 L n t + (σ 2 + µ 2 ) max t n , 2δ t n .
Thus (3.34) can be rewritten as,

∆ ⊺ Γ∆ ≤ 2λ 0 ∆ 1 + 2λ( θ -θ 1 ).
We know that S = {j ∶ θ j ≠ 0} and by N = {j ∶ θ j = 0}. Hence,

λ( θ -θ 1 ) ≤ λ θ S -λ θS 1 -λ θS 1 since θ N = 0 ≤ λ θS -θ S 1 -λ θN 1 therefore, ∆ ⊺ Γ∆ ≤ 2(λ 0 + λ) θS -θ S 1 + 2(λ 0 -λ) θN -θ N 1 ≤ 3λ θS -θ S 1 -λ θN -θ N 1 , assuming λ 0 ≤ λ 2 ≤ 3λ θS -θ S 1

Introduction

We consider the errors-in-variables linear regression model where we observe Y and W satisfying

⎧ ⎪ ⎪ ⎪ ⎨ ⎪ ⎪ ⎪ ⎩ Y = Xβ ⋆ + W = X + U (4.1)
where Y = (Y 1 , ⋯, Y n ) ⊺ is the vector of responses, X = [X ij ] 1≤i≤n,1≤j≤p is the n × p design matrix of covariates and = ( 1 , ⋯, n ) ⊺ is a Gaussian vector, having centered independent and identically distributed elements with variance σ 2 . Furthermore,

W = [W ij ] 1≤i≤n,1≤j≤p
is a n × p matrix of noisy covariates X, that is X being polluted by additive measurement errors U ∈ R n×p . We will assume that the rows of U are drawn independently from a centered Gaussian multivariate distribution such that its p×p covariance matrix is denoted by C U .

We want to estimate the unknown p-dimensional parameter β ⋆ . It is assumed s-sparse, hence β ⋆ 0 ≤ s. The model is considered to be high dimensional which usually implies that the number of covariates p is assumed (much) larger than the number of elements n of the dataset (p > n). The variables and U will be considered independent.

We call the direct model the linear regression model where Y and X are observed. The Slope estimator β of β ⋆ is defined through the following optimization problem

β ∈ arg min β∈R p 1 √ n Y -Xβ 2 2 + β ⋆ (4.2)
with β ⋆ = ∑ p i=1 λ i β (i) , known as the Slope norm, where λ 1 ≥ ⋯ ≥ λ p > 0 are tuning parameters.

In this paper, we introduce a Slope based estimator for errors-in-variables regression model (4.1) and study its properties.

There is a vast literature on high dimensional regression methods. Most popular and immensely studied are the Lasso estimator (Least Absolute Shrinkage and selection operator ), which was introduced in [START_REF] Tibshirani | Regression shrinkage and selection via the lasso[END_REF] and the Dantzig selector [START_REF] Candes | The dantzig selector: Statistical estimation when p is much larger than n[END_REF]. Its full theoretical study was developed through multiple subsequent articles and several books such as [START_REF] Hastie | The Lasso and generalizations[END_REF] and [START_REF] Giraud | Introduction to high-dimensional statistics[END_REF] that provide an exhaustive list of its properties. Recently, it was noted that the Lasso attains a suboptimal rate that can be improved by the Slope Sorted L-One Penalized Estimation, see [START_REF] Su | SLOPE is adaptive to unknown sparsity and asymptotically minimax[END_REF] and [START_REF] Malgorzata | Slope -adaptative variable selection via convex optimization[END_REF]. The authors introduced a convex optimization problem with a different penalty whose algorithm solution has a complexity similar to most common 1 penalization procedures. They provided a path for solving the question of variable selection through a non-orthogonal design. The properties, namely the convergence rate, of both aforementioned methods were improved to s log(p s) n instead of s log(p) n for the Lasso. In [START_REF] Bellec | Slope meets lasso: improved oracle bounds and optimality[END_REF], the authors were able to prove that both Lasso (with known or estimated sparsity s) and Slope estimators achieve the minimax prediction and l 2 estimation rate of s log(p s) n. The minimax optimal bounds were also obtained for an l q estimation error, with 1 ≤ q ≤ 2.

Many derivations of the Lasso were thoroughly studied. Let us mention the Square Root Lasso, which was introduced in [START_REF] Belloni | Pivotal estimation via square-root lasso in nonparametric regression[END_REF]. The authors showed that by minimizing the squared root of the least-squares criterion penalized with the usual 1 norm, the Square Root Lasso estimator did not rely on the noise of the model, and was able to achieve minimax optimal rate of σ s log(p) n, where σ is the variance of the noise of the model.

Based on some of the results developed in [START_REF] Bellec | Slope meets lasso: improved oracle bounds and optimality[END_REF], the author in [START_REF] Derumigny | Improved bounds for square-root lasso and square-root slope[END_REF] improved the estimation rate of the Square Root Lasso and introduced the Square Root Slope estimator. He showed that both estimators can achieve optimal minimax rate of s log(p s) n. The author was able to preserve the adaptivity to σ.

In many examples, measurement errors are inevitable. Errors-in-variables (EIV) regression models (4.1) have been largely considered in the literature, mostly in parametric or semi-parametric setups. More recently, in high dimensional EIV linear regression models, [START_REF] Rosenbaum | Sparse recovery under matrix uncertainty[END_REF] have proved that the regular Lasso or Dantzig estimator is unstable. They then introduced matrix uncertainty selectors and their improvements in [START_REF] Belloni | An { 1 , 2 , ∞ }-regularization approach to high-dimensional errors-in-variables models[END_REF], which are stable, consistent and capable of reproducing a sparsity pattern common to most high dimensional settings, see also [START_REF] Belloni | Linear and conic programming estimators in high dimensional errors-in-variables models[END_REF]. We can also mention the authors in [START_REF] Sorensen | Measurement error in lasso : Impact and likelihood bias correction[END_REF] who introduced a corrected Lasso estimator and were able to prove its sign consistency, among other properties. Similarly, the authors in [START_REF] Datta | Cocolasso for high-dimensional error-in-variables regression[END_REF] introduced a corrected convex optimization problem to tackle the errors-in-variables model. They developed a convex corrected Lasso estimator and showed that it was sign consistent. Lastly, the authors in [START_REF] Loh | High-dimensional regression with noisy and missing data : provable guarantees with nonconvexity[END_REF] opted for a non-convex optimization problem from which they derived the estimator's error bounds and showed that a gradient descent based algorithm allowed the estimator to converge in polynomial time.

In this paper, we consider a corrected least-square criterion that we project on the space of convex quadratic functions. We introduce a Slope penalized procedure for estimating the errors-in-variables linear regression model (4.1) and study its properties. We describe sufficient conditions on the parameters of the model in order to attain the rates known optimal in the direct case. As expected, these conditions are quite stringent, however we also give upper bounds on the slower rates that are attained in the opposite case. As discussed at the beginning of Section 4.6.2, these results can be extended to sub-Gaussian random rows under an additional assumption, following [START_REF] Bunea | On the sample covariance matrix estimator of reduced effective rank population matrices, with applications to fpca[END_REF].

Notation and organization of the article

For a given vector v ∈ R p , we use the notation v q for the l q norm, with 1 ≤ q ≤ ∞, and v 0 for the number of non-zero coordinates of v. For any set of coordinates S ⊂ {1, ⋯, p}, Lemma 1. The matrix Σ defined in (4.4) is such that

Σ ∈ arg min M ∈S ⪰0 Σ -M 2 . (4.5)
The proof is in Section 4.6.3.

Let us see that Σ is easily constructed from Σ. We note here that Σ is a p×p symmetric matrix and thus, it allows a spectral decomposition. Let us denote by D a p × p diagonal matrix whose elements are the eigenvalues (η j ) j=1,⋯,p of the matrix Σ. We also denote by V a p × p orthogonal matrix of eigenvectors of Σ. As such, we can write Σ = V ⊺ DV . From the equation (4.4), we can then deduce that

Σ = V ⊺ D + V, (4.6)
with D + the diagonal matrix whose elements are the positive parts (η j ) + of ηj , j = 1, ..., p.

We define the estimator β of β * by

β ∈ arg min β∈R p β ⊺ Σβ - 2 n Y ⊺ W β + β ⋆ (4.7)
where

β ⋆ = ∑ p i=1 λ i β (i) , with λ 1 ≥ ⋯ ≥ λ p > 0.
The non-convex character of the optimization problem (4.3) has been addressed through different means in the literature. For example, the authors in [START_REF] Loh | High-dimensional regression with noisy and missing data : provable guarantees with nonconvexity[END_REF] introduced a constraint on the space of possible solutions through the equation

β 1 ≤ b 0 √ s where b 0 is a suit- able constant which must verify b 0 ≥ β ⋆ 2 .
This assumption is restrictive and relies on a constant which can not be necessarily implied from the dataset, since β ⋆ is an unknown parameter. We can also mention the authors in [START_REF] Datta | Cocolasso for high-dimensional error-in-variables regression[END_REF] which have relied on a positive semidefinite projection of the matrix Σ in order to derive a convex optimization problem. However, the projection is an element-wise one which was then complemented by elementwise concentration inequalities in order to derive desirable error bounds. These inequalities help define suitable convergence rates but yet again it is not obvious under which global assumptions they are satisfied. In this paper, we consider a matrix projection of Σ in Frobenius norm on the space S ⪰0 of positive and semi-definite matrices. Using this convex criterion, we derive sufficient conditions for optimal convergence rate of our estimator, as well as quantify the loss in the rate when these conditions are not satisfied.

Sufficient conditions for optimal convergence rate

In this section, we derive sufficient conditions to attain the optimal rate for the Slope estimator β. In most cases, the rates attained by our procedure are much slower due to the high-dimensional noise that interferes with the estimation. We will illustrate a few examples where the optimal rates from the direct problem can still be attained. We recall the Weighted Restricted Eigenvalues (WRE) condition introduced in [START_REF] Bellec | Slope meets lasso: improved oracle bounds and optimality[END_REF].

Definition We say that the design matrix X satisfies the W RE(s, c 0 ) condition if where

C W RE(s,c 0 ) = {δ ∈ R p ∶ δ ⋆ ≤ c 0 δ 2 ∑ s j=1 λ 2 j } is a cone in R p .
Note that the W RE(s, c 0 ) condition is written with respect to X which is unknown. It is possible as we will see below to deduce an analogous condition with respect to δ ⊺ Σδ with high probability. Indeed, δ ⊺ Σδ is the natural quadratic risk associated to our optimization problem (4.7).

Let us introduce the following notation

R n,p (C U , β ⋆ 2 , t) = t 2 C U 2 n + β ⋆ 2 ⎛ ⎝ 2A n,p (C U ) + 16tp 1 n C U 2 ⎞ ⎠ (4.9) A n,p (C U ) = c ⋅ max ⎧ ⎪ ⎪ ⎨ ⎪ ⎪ ⎩ T r(C U ) log(pn) n , T r(C U ) C U 2 log(pn) n ⎫ ⎪ ⎪ ⎬ ⎪ ⎪ ⎭ , (4.10)
with t > 1 and c > 0.

Assumption. We assume that

A n,p (C U ) + 2t p n C U 2 ≤ κ 2 , (4.11)
when n, p, s are large enough and for t > 0 large enough. In the rest of the article, A n,p,C U will be used interchangeably with A. with probability at least 1 -4 n -2 exp(-t 2 2 ).

The proof is in Section 4.6.3.

The next theorem gives upper bounds for the L 2 -risk. These bounds allow to deduce sufficient conditions on the parameters of the noise distribution in order to attain the known optimal rates for estimating the sparse parameter β. They also quantify the loss in the rate when these conditions are not verified.

Theorem 4.1. Let s ∈ {1, ⋯, p} and assume that the unknown design X satisfies the W RE(s, c 0 ) condition (4.12) and that the assumption (4.11) holds. We choose the following tuning parameters The proof is in Section 4.6.3.

λ j = v ⋅ max ⎧ ⎪ ⎪ ⎨ ⎪ ⎪ ⎩ 1 n ,
As expected, the presence of measurement errors affects the overall convergence rates in most cases. If the covariance matrix has small enough rank and spectral norm, it is possible to attain the rates of convergence known optimal in noiseless case. Let us illustrate with the following example sufficient conditions for optimal rates of convergence. Suppose the noise W has a diagonal covariance matrix C U = v ⋅ diag(1, ..., 1, 0, ...0) where the rank is r < p. We get A n,p (C U ) = c ⋅ v r ⋅ log(pn) n that needs to be bounded. Moreover, For example, v = p -2-and any r ∈ {1, ..., p} check the constraints, that is a full rank covariance matrix C U = p -2-I p or p -2-diag(1, ..., 1, 0, ...0).

R

Simulation Analysis

We highlight below experiments that were conducted to illustrate the optimization methodology, on a predefined dataset. The implementation is based on the coordinate descent algorithm. The exact same methodology was used to imply the Prediction Risk with our estimate given by X ⋅ β. As such the Prediction Risk (PR) value of a sample size n was given by

P R = 1 l l i=1 1 O {i} X ⋅ β ⋆ -X ⋅ β 2 2 .
Similarly, for a single sample size, we computed the Prediction Risk u times. It allowed us to imply the confidence interval of the measure. It is important to note that for a convergent estimation procedure, the values P R and ER should converge to 0, as n increases, for a given p.

Results

We show below two set of results corresponding to the evolution of the key metrics: Estimation Risk, Prediction Risk. We also show for completeness at each step the value taken by key noise parameters such as T r(C u ) and C U 2 . In each of these two cases, the value p is fixed while n increases. It is important to note that the synthetic data construction methodology follows the step described in section 4.4.1. For both cases, u = 20.

First case. p = 500, σ = 1, µ = .05 and n = {150, 200, 250, 300, 350, 400}.

As the value of n increases, we realize that the Prediction Risk and Estimation Risk converges steadily towards 0. Furthermore, T r(C U ) and C U 2 are well below critical levels.

Second case. p = 500, σ = 1, µ = .1 and n = {150, 200, 250, 300, 350, 400}. Similarly as the first case, the overall conclusion is the same. However, we see a deterioration of the estimation risk, even though the values themselves remain very small.

Remark. As the value of µ further increases, the deterioration of the overall estimation metrics become more and more flagrant. The results are not shown here.

Conclusion

We have established sufficient conditions under which we can reach optimal rates for our Slope based estimators, given a polluted design matrix. The impact on rates is also outlined when these conditions are not fulfilled. Our approach is novel since it relies on a

Frobenius based projection in order to derive a convex optimization problem. The study was completed by a simulation study which illustrated the accuracy and correctness of our optimization methodology, with an implementation based on the coordinate descent algorithm.

Proofs

Preliminary results

The following result is demonstrated in [36, p. 112]. It has been used as a lemma in [START_REF] Derumigny | Improved bounds for square-root lasso and square-root slope[END_REF] and is reproduced here for the reader's convenience. Proof of Lemma 6. The proof of the lemma is a direct consequence of the application of Cauchy-Schwarz inequality and the Theorem 2.1 in [START_REF] Bunea | On the sample covariance matrix estimator of reduced effective rank population matrices, with applications to fpca[END_REF]. Hence,

v ⊺ ( 1 n U ⊺ U -C U )w ≤ v 2 ⋅ w 2 ⋅ 1 n U ⊺ U -C U 2 ≤ v 2 ⋅ w 2 ⋅ c ⋅ max ⎧ ⎪ ⎪ ⎨ ⎪ ⎪ ⎩ T r(C U ) log(pn) n , T r(C U ) C U 2 log(pn) n ⎫ ⎪ ⎪ ⎬ ⎪ ⎪ ⎭
with probability at least 1 -4 n .

Lemma 7. For all t > 1 and β ∈ R p , we will have

1 n ⊺ U β ≤ σ √ n β 2 C U 1 2 2 t,
with probability larger than 1 -2e -t 2 .

Proof of Lemma 7. The scalar product 1 n ⊺ U β can be rewritten as:

1 n ⊺ U β = 1 n n i=1 i U i ⋅ β, where U i⋅ ⋅ β = p j=1 U ij β j .
The Laplace transform of the above random variable writes, for all t ∈ R and β ∈ R p

E exp t n ⊺ U β = E exp n i=1 t n i U i,⋅ ⋅ β = E n i=1 E exp t n i (U i,⋅ β) U i,⋅ ≤ E n i=1 exp t 2 2n 2 σ 2 (U i,⋅ β) 2 = E exp t 2 2n σ 2 (U 1,⋅ β) 2 ,
where we used that i are Gaussian i.i.d. and that the rows U i,⋅ are independent. Now, since U 1,⋅ is a C U 2 -subGaussian vector, then (U 1,⋅ ⋅ β) 2 is subexponential and that implies

E exp t 2 σ 2 2n (U 1,⋅ β) 2 ≤ E exp t 2 σ 2 2n ⋅ β 2 2 C U 2 ( U 1,⋅ β √ β ⊺ C U β ) 2 = E exp(λZ 2 ),
where Z is a standard Gaussian random variable and λ = By the Bernstein inequality,

1 n β ⊺ U ⊺ Xu ≤ t ⋅ u 2 β 2 p n ⋅ C U 2 ,
with probability larger than 1 -2e -t 2 2 .

Lemma 9. For all t > 1, c > 0 and β, u ∈ R p , we will have

u ⊺ Σ -Σ β ≤ β 2 ⋅ u 2 ⋅ ⎧ ⎪ ⎪ ⎨ ⎪ ⎪ ⎩ c ⋅ max ⎧ ⎪ ⎪ ⎨ ⎪ ⎪ ⎩ T r(C U ) log(pn) n , T r(C U ) C U 2 log(pn) n ⎫ ⎪ ⎪ ⎬ ⎪ ⎪ ⎭ + 1 n (X ⊺ U + U ⊺ X) 2 ,
with probability 1 -4 n . Moreover,

1 n (X ⊺ U + U ⊺ X) 2 ≤ 3tp 2 log(p) n max 1≤j≤p C jj U
with probability larger than 1 -4 exp(-(t 2 -1) log(p)), and

1 n (X ⊺ U + U ⊺ X) 2 ≤ 16tp 1 n C U 2
with probability larger than 1 -2 ⋅ exp(-2p(t 2 -1)).

On the other hand, Secondly, a slightly different bound can be obtained for 1 n (X ⊺ U + U ⊺ X) 2 using the definition of the norm and the metric entropy of the sphere, as follows. Denote by W = 1 n (X ⊺ U + U ⊺ X) and recall that

1 n X ⊺ U 1 = max 1≤j≤p p i=1
W 2 = max v∶ v 2 ≤1 v ⊺ W v .
For any point v, there exists a point v k belonging to an 1/2-net {v 1 , ..., v N } such that vv k 2 ≤ 1 2. It is known that N ≤ 5 p , we use log(N ) ≤ 2p. We write for any v such that By the definition given in (4.4), we know that δ ⊺ Σ -Σ δ ≥ 0, hence using Lemma 6, Lemma 8 and the assumption (4.11) we have The function f is convex, hence we will have

δ ⊺ Σδ ≥ 1 n Xδ 2 2 - 2 n δ ⊺ X ⊺ U δ -δ 2 2 1 n U ⊺ U -C u 2 ≥ κ δ 2 2 -2t δ 2 2 p n C U 2 -δ 2 2 ⋅ A n,p ≥ δ 2 2 κ -2t p n C U 2 -A n,p ≥ δ 2
f ( β) -f (β ⋆ ) ≥ K(β ⋆ ) ⋅ ( β -β ⋆ )
that can also be written as

β⊺ Σ β -2γ ⊺ β -β ⋆⊺ Σβ ⋆ -2γ ⊺ β ⋆ ≥ ⟨K(β ⋆ ), δ⟩ with δ = β -β ⋆ . (4.23)
The estimator β is solution of the optimization problem (4.7):

β⊺ Σ β -2γ ⊺ β + β ⋆ ≤ β ⋆⊺ Σβ ⋆ -2γ ⊺ β ⋆ + β ⋆ ⋆ , then β⊺ Σ β -2γ ⊺ β -β ⋆⊺ Σβ ⋆ -2γ ⊺ β ⋆ ≤ β ⋆ ⋆ -β ⋆ .
We follow here the lines of proof in [START_REF] Derumigny | Improved bounds for square-root lasso and square-root slope[END_REF]. We recall that the sorted l 1 norm can be written as

v ⋆ = max p j=1 λ j v φ(j)
where the maximum is taken over all permutations φ = (φ(1), ⋯, φ(p)) of {1, ⋯, p}. (4.28)

We will then have, with probability at least 1 -

  où f β (W ) = β ⊺ ⋅ f CT (W ) avec f CT ∶ W → R d unefonction fournie par l'utilisateur. Nous remarquons ici que ψ f (P 0 ) est multivarié et sa taille correspond au nombre d'éléments de W . Cette dernière characteristique de ψ f (P 0 ) est cruciale car elle permet d'obtenir une mesure d'importance de variable qui capture le rôle que joue chaque élément de W dans la relation qui existe entre X et Y . Nous ne faisons pas l'hypothèse qu'il existe un β tel que

  un vecteur contenant les i premières composantes d'un espace d'Hilbert. La mesure d'importance de variable ψ f (P ) (1) s'appuie sur une fonction f β qui est fournie par l'utilisateur, pour des raisons de simplicité. Cette fonction est par nature une propriété inconnue de notre problème statistique et ce faisant une source de complexité. Dans le chapitre 3, nous décidons d'aborder le problème sous un autre angle. Nous utilisons la même mesure d'importance de variable sans toutefois demander à ce que la fonction f β soit spécifiée par l'utilisateur. Notre objectif demeure de quantifier la relation qui existe entre la réponse Y et l'exposition X, tout en prenant en compte W . On dénomme par F, l'ensemble des fonctions non paramétriques et continues sur R d . On introduit par analogie avec (1), un paramètre statistique ψ pen (P ) caractérisé par ψ pen (P ) = arg min f ∈F

1. 3

 3 Going beyond ψ f within a submodel of M Let us consider the data structure O = (X, W, Y ) of an experimental unit of interest, where X ∈ R is an exposure, Y ∈ R is a response and W ∈ W ⊂ R d represents our covariates, assumed drawn from the true data generating distribution P 0 ∈ M ′ , where M ′ ⊂ M and M ′ is characterized by (η ⋆ P , θ ⋆ P ) ∈ R m × R k with potentially high dimensions m and k, such that

Figure 1 . 1 -

 11 Figure 1.1 -Model without measurement errors

Figure 1 . 2 -

 12 Figure 1.2 -Model with measurement errors

  The setting. Consider the situation where a real-valued variable of cause, X ∈ R (a continuous exposure), affects a [0, 1]-valued variable of effect, Y (a continuous response), in the presence of a variable of context W ∈ W (covariates). The objective is to assess to what extent (X, W ) influences Y while making only realistic assumptions on the unknown distribution P 0 of O = (W, X, Y ). This requires both the definition of a tailored statistical parameter and the elaboration of a semi-parametric inferential procedure to construct confidence regions of a given asymptotic level based on independent copies of O drawn from P 0 .Marginal structural models are very useful tools in this regard. The notion of marginal structural model was introduced in[START_REF] Robins | Marginal structural models and causal inference in epidemiology[END_REF]. Widely used in the epidemiology literature, they are parametric classes of regression functions. Let {msm β ∶ β ∈ B} be such a class of functions mapping R × W to R. It yields a parameter, defined as a minimizer in β of

CO 2 Figure 2 . 1 -

 221 Figure 2.1 -Left: Histogram of the variable X. Right: Confidence region of asymptotic level 95% for parameter (ψ 0,2 , ψ 0,3 ).

  known and B ∈ R d such that the d-dimensional parameter of interest ϕ CT , was given by ϕ CT (P ) = arg min

Φ

  j (W )b j where b ∈ R p ii) the support of b is of size smaller or equal to s( {b j ; b j ≠ 0} ≤ s).

  we computed its Prediction Risk value, for several sample sizes through a two-step cross-validation process. Let us denote by n, the data sample size. We partition the sample in l distinct subsets O subset , of equal size. First step: We set aside one of the O subset {i} , which are used as the test set, and we use the l -1 remaining subsets as training sets. Using the latter, we then compute the estimator θ{-i} . Second Step: we use the O subset {i} (test set -left out from the initial estimation), to determine how close the estimator θ{-i} is to its real value. These two steps are repeated successively l times. For each iteration, we calculated the difference between the estimator and the true value function using the test set. The empirical average of these l distances give us the desired Prediction Risk measure. In brief, the Prediction Risk (P R)

  Evolution of the Prediction Risk.

  Evolution of the Estimation Risk.

Figure 3 . 1 -Figure 3 . 2 -

 3132 Figure 3.1 -Univariate covariate case with function f from Section 3.4.1 -Confidence Intervals were computed through u = 30 replications for each sample size

Figure 3 . 3 -

 33 Figure 3.3 -Bivariate covariate case with function f from Section 3.4.1 -Confidence Intervals were computed through u = 30 replications for each sample size

Figure 3 . 4 -

 34 Figure 3.4 -Univariate covariate case with errors-in-variables and function f from Section 3.4.1 -Confidence Intervals were computed through u = 30 replications for various sample sizes -noise parameter given by µ = 3.33%

Figure 3 . 5 -Figure 3 . 6 - 6

 35366 Figure 3.5 -Univariate covariate case with errors-in-variable and function f from Section 3.4.1 -Confidence Intervals are computed through u = 30 replications for various sample sizes -noise parameter given by µ = 6.66%

Figure 3 . 7 -Figure 3 . 8 -

 3738 Figure 3.7 -Univariate covariate case with errors-in-variables -and function f from Section 3.4.1 -Confidence Intervals were computed through u = 30 replications for each sample size -noise parameter given by µ = 9.99%

Figure 3 . 9 -

 39 Figure 3.9 -Model without measurement errors using both TMLE and Lasso methodologies for function f

Figure 3 . 10 -

 310 Figure 3.10 -Model with measurement errors using both TMLE and Lasso methodologies for the function f -noise parameter given by µ = 20%

Lemma 2 .

 2 Under the assumption (4.11) and if X verifies the WRE(s, c 0 ) condition then

Figure 4 . 1 -

 41 Figure 4.1 -Evolution of the Estimation Risk for different value of n -First case.

Figure 4 . 2 -Figure 4 . 3 -

 4243 Figure 4.2 -Evolution of the Prediction Risk for different value of n -First case.

Figure 4 . 4 -

 44 Figure 4.4 -Levels taken by T r(C U ) at each step of the iteration process -First case. -First case.

Figure 4 . 5 -

 45 Figure 4.5 -Evolution of the Prediction Risk for different value of n -Second case.

Figure 4 . 6 -

 46 Figure 4.6 -Levels taken by C U 2 at each step of the iteration process -Second case.

Figure 4 . 7 -

 47 Figure 4.7 -Levels taken by T r(C U ) at each step of the iteration process -Second case.

Lemma 3 .Lemma 5 ([ 4 ,

 354 With probability at least 1 -(1 + e 2 )e -n 24 , we have following results are cited for reader's convenience. In the following Lemmas 4 and 5, we useλ j = γ log(2p j) n , for γ ≥ 4(4 + √ 2) > 0 large enough.Lemma 4 ([30]). We have,γ (s n) log(2p s) ≤ s j=1 λ 2 j ≤ γ (s n) log(2ep s). Theorem 4.1]). Let 0 < δ 0 < 1 and let X ∈ R n×p be a matrix such that max j=1,⋯,p Xe j n ≤ 1.

2 κ 2 with probability at least 1 - 4 n

 224 -2 exp(-t 2 2 ). Proof of Theorem 4.1. Let us consider the function f ∶ β → β ⊺ Σβ -2γ ⊺ β, where γ ∶= 1 n W ⊺ Y , and denote by K(β) ∶= ∇f (β) = Σβ -1 n W ⊺ Y. (4.22)

Let us choose the permutation ψ such that β * ⋆ = s j=1 λλλ 2 j 2 ≤ 2 s j=1 λ 2 j

 j=1222j=12 j β * ψ(j) and δ ψ(s+1) ≥ δ ψ(s+s) ≥ ⋯ ≥ δ ψ(p) . j β ⋆ ψ(j) -βψ(j) -p j=s+1 λ j β ⋆ ψ(j) -βψ(j) . (4.24)Hence, given the permutation mentioned above, we can then write j δ (j) -⟨K(β ⋆ ), δ⟩ .(4.26)From now on, the calculations are specific to our model and estimator. Given Lemma[START_REF] Birkner | Data adaptive pathway testing[END_REF], we can conclude thatp j=s+1 λ j δ (j) ≤ s j=1 λ j δ (j) + 1 n ⊺ Xδ + δ 2 R n,p (C U , β ⋆ , t),then, by Lemma 5,δ ⋆ ≤ 2 s j=1 λ j δ (j) + max{H(δ), G(δ)} + δ 2 R n,p (C U , β ⋆ , t).(4.27)From now on, we keep the notation λ j for the case where R n,p = O(1) √ n and use λj otherwise (see Case III below).Case I. max {H(δ) + δ 2 ⋅ R n,p , G(δ)} ≤ H(δ) + σ δ 2 ∑ s j=1 λFrom (4.27) and (4.14), we infer thatδ ⋆ ≤ 2 s j=1 λ j δ (j) + 1 n ⊺ Xδ + R n,p δ j δ (j) + H(δ) + 2σ δ 2We can then conclude that δ ∈ C W RE (c 0 ) with c 0 = 4(1 + σ).By the definition of β as the solution of the optimization problem (4.7), there exists w such that ∇f (β) + w = 0.

  that δ ⊺ Σδ = ( ββ ⋆ ) ⊺ Σ( ββ ⋆ ) = ⟨ Σ β -1 n Y ⊺ W, δ⟩ -⟨ Σβ ⋆ , δ⟩ + ⟨ 1 n Y ⊺ W, δ⟩ = ⟨K( β), δ⟩ -⟨K(β ⋆ ), δ⟩ = -⟨w, δ⟩ + ⟨K(β ⋆ ), δ⟩ ≤ δ ⋆ + ⟨K(β ⋆ ), δ⟩ , since w dual ≤ 1.

  with a sparsity coefficient denoted by s = s η + s θ .

	Nonparametric Least square. Let us consider the statistical problem (1.5). A non-
	parametric least square estimate of β is given by
	arg min β∈R p Y -Xβ 2 .	(1.7)

Table 2 .

 2 1 -Summary of the results of the simulation study. The values of the true parameter are reported in the second row. The third and fourth row give the empirical coverage of the regions of confidence for each coordinate and each sample size n. MSM stands for "marginal structural model".

	1.42

  Assume that the variable X takes values in a compact space. Furthermore, let us assume that the compatibility condition(3.11) holds. Then, with probability larger than 1 -2 exp( -t 22 ) and λ = 2σ X ∞

	t 2 +2 log(p)
	n

.11) Proposition 3.1.

Table 3 . 1 -

 31 Breaking point analysis -fraction of simulation runs (out of 100 replications)

	On the contrary, for

Table 3 .

 3 These synthetic indices help demonstrate the correctness of the method. Thus we can conclude through simple yet convincing examples We turn our attention to the S&P 500 index, which now represents our target index. Similarly to the prior case, two windows were considered: 2008 and 2014. The reasons for this choice are the same as previously discussed.For the first period, results show a replicating portfolio containing 80 constituents (the full list is not shown here). All these elements can be grouped in 11 distinct GICS Sectors.They represent the exact number of unique GICS sectors in the original S&P 500 list of constituents. As such, the replicating portfolio does cover the full spectrum of market segments represented by the index. For the second period 2014, the replicating portfolio 5 -Replicating Portfolio 1 -with their respective importance measure -year 2014 has 139 constituents. As for the prior period, it covers as well all 11 sectors present in the index list of constituents. We represent in Table3.6 the number (in percentage) of securities linked to each GICS Sector. Thus, we show the distribution of our replicating portfolios.

	Ticker Symbol	Security		GICS Sector
	CSCO	Cisco Systems	Information Technology
	HRS	Harris Corporation Information Technology
	XRX	Xerox Corporation Information Technology
	Table 3.2 -Synthetic Index 1 -based on securities from the same GICS Sector
	Ticker Symbol	Security		GICS Sector
	GPS	Gap Inc.		Consumer Discretionary
	HST	Host Hotels & Resorts.	Real Estate
	ORLY	O Ŕeilly Automative	Consumer Discretionary
	Table 3.3 -Synthetic Index 2 -based on securities from different GICS Sector
	that on a predefined framework, we can accurately identify the key drivers of a target
	index.
	Case 2: Ticker Symbol	Security		GICS Sector	β value
	GPS	Gap Inc.		Consumer Discretionary 0.04483074
	HST	Host Hotels & Resorts	Real Estate	0.01420947
	ORLY	O Ŕeilly Automative Consumer Discretionary 0.1347531
	Table 3.4 -Replicating Portfolio 2 -with their respective importance measure -year 2008

Table 3 .

 3 GICS Sector Name S&P 500 Rep. Port. -2008 Rep. Port. -2014 6 -percentage of each sector present in the S&P 500 and the replicating portfolios for the period 2008 and 2014.

	Industrials	13.09 %	12.658%	14.49%
	Health Care	12.3 %	13.92%	13.7%
	Information Technology	13.88%	16.455%	14.49%
	Consumer Discretionary	16.66%	12.65%	13.76%
	Utilities	5.55%	5.06%	4.34%
	Financials	13.49%	15.18%	14.49%
	Materials	4.96%	6.32%	6.52%
	Real Estate	6.54%	5.06%	2.17%
	Consumer Staples	6.74%	3.79%	8.69%
	Energy	6.34%	7.59%	5.79%
	Telecommunication Services	0.59%	1.26%	1.44%
	Number of Securities in Portfolio	500	80	139

Table 3 .

 3 [START_REF] Belloni | Linear and conic programming estimators in high dimensional errors-in-variables models[END_REF] -representation of each sector in the replicating portfolio with respect to the starting list, for each value of µ, on the period 2008

	µ	0.001	0.003	0.005	0.007	0.01
	Industrials	15.85% 12.5% 15.68% 21.73% 20.58%
	Health Care	13.41% 17.85% 15.68% 13.04% 8.82%
	Information Technology	13.41% 16.07% 25.49% 15.21% 11.76%
	Consumer Discretionary	10.97% 5.35%	7.84% 10.86% 11.76%
	Utilities	2.43%	3.57%	1.96%	2.17%	2.94%
	Financials	20.73% 19.64% 23.52% 23.91% 23.52%
	Materials	4.87%	8.92%	3.92%	6.52%	8.82%
	Real Estate	2.43%	1.78%	0%	0%	0%
	Consumer Staples	7.31%	7.14%	0%	0%	2.94%
	Energy	7.31%	7.14%	5.88%	2.17%	8.82%
	Telecommunication Services	1.22%	0%	0%	4.34%	0%
	Number of Securities in Portfolio	83	57	52	47	35

Table 3 .

 3 8 -representation of each sector in the replicating portfolio with respect to the starting list, for each value of µ, on the period 2014

  R n,p (C U , B, t) where t = 2 log(2 δ 0 ), B is an upper bound of β * 2 and we assume that δ 0 < 2e -1 2 and that Then, with probability at least 1 -8 n -4 exp(-2p(2 log(2 δ 0 ) -1))δ 0 , we have

			⎫ ⎪ ⎪ ⎬ ⎪ ⎪ ⎭	⋅ log(2p j),	(4.13)
	v ≥ 2σ(4 +	√	2).		(4.14)
	β -β ⋆ where C 1 = 2 ≤ max (3+4σ)γ ⎧ ⎪ ⎪ ⎪ ⎨ ⎪ ⎪ ⎪ ⎩ C 1 κ , C 2 = 4(4 + s n log( 2ep s √ 2) 2 and C 3 = 28 ), C 2 log 2 (1 δ 0 ) sn log(2ep s) κ .	, C 3 R n,p (C U , B, t) s log(	2ep s (4.15) ) ⎫ ⎪ ⎪ ⎪ ⎬ , ⎪ ⎪ ⎪ ⎭

  t 2 σ 2 2n ⋅ β 2 2 C U 2 . Now, E exp(λZ 2 ) = e λ E exp(λ(Z 2 -1)) ≤ Lemma 8. For all t ∈ R, and β, u ∈ R p , we will have1 n β ⊺ U ⊺ Xu ≤ t ⋅ u 2 β 2 p n ⋅ C U 2 with probability 1 -2e -t 2 2Proof of Lemma 8. Let us bound from above the Laplace transform, (X k⋅ u) 2 ⋅ β ⊺ C U β)

		E exp(	t n	β ⊺ U ⊺ Xu)
			= E exp		t n	n k=1	β ⊺ U ⊺ k⋅ X k⋅ u
			=	n k=1	E exp	t n	X k⋅ u ⋅ U k⋅ β
			= 2n 2 ≤ exp n k=1 t 2 exp( ⎛ ⎝ t 2 2n 2 n k=1	p j=1	X 2 kj u 2 2 β 2 2 C U 2	⎞ ⎠
			≤ exp	t 2 p 2n	u 2 2 β 2 2 C U 2	, since	1 n	k=1 n	X 2 kj ≤ 1.
								√	1 1 -2λ	≤ e λ ,
	for λ < 1 2. In conclusion,
	E exp	t n	⊺ U β ≤ exp	t 2 2n	σ 2 β 2 2 C U 2 , for all	t 2 n	σ 2 β 2 2 C U 2 ≤ 1
	which means that 1 n	⊺ U β is ( σ 2 n β 2 2 C U 2 , σ √ n β 2 C U	1 2 2 )-subexponential, and we get
	the result.

  with probability larger than 1 -4 exp(-(t 2 -1) log(p)). This result and Lemma 6 conclude the proof.

						1 n	(X ⊺ U ) ij = max 1≤j≤p	p i=1	n k=1	1 n	X ki U kj =∶ max 1≤j≤p	W j .
	The Laplace transform of W j
			E [exp(tW j )] = E exp(	t n	p i=1	n k=1	X ki U kj )
			=	n k=1	E exp(	t n	p i=1	X ki U kj )
			≤	n k=1	exp		t 2 2n 2 (	i=1 p	X ki ) 2 C jj U
			≤ exp	t 2 2n 2	k=1 n (	i=1 p	X ki ) 2 C jj U
			≤ exp	t 2 p 2n	C jj U	since	1 n	i=1 n	X 2 kj ≤ 1.
	Thus,
	W j ≤ t	p √ n	C jj U , with probability larger than 1 -2 exp(-	t 2 2	)
	and, by a union bound,
				1 n	X ⊺ U 1 ≤ tp 2	log(p) n	max 1≤j≤p	C jj U ,	(4.20)
	with probability larger than 1 -2 exp(-(t 2 -1) log(p)). Putting together (4.18), (4.19) and
	(4.20) we get that
	1 n	(X ⊺ U + U ⊺ X) 2 ≤ t	2 log(p) n	( 3T r(C U ) + p max 1≤j≤p	C jj U )
						≤ 3tp	2 log(p) n	max 1≤j≤p	C jj U ,

  8 n -4 exp(-t 22 ) -4 exp(-2p(t 2 -1)) -δ 0 2 ,δ ⊺ Σδ ≤ δ ⋆ + ⟨K(β ⋆ ), δ⟩ ≤ δ ⋆ + 1 n ⊺ Xδ + R n,p δ 2 ≤ δ ⋆ + H(δ) + 2σ δ 2 Case II. max H(δ) + δ 2 ⋅ R n,p , H(δ) + σ δ 2 ∑ s j=1 λ 2 j ≤ G(δ)From the assumption of this case, we infer that when n, s, p are large enough. Hence, we deduce that for such large enough values of n, s, pCase III. max G(δ), H(δ) + σ δ 2 ∑ s j=1 λ 2 j ≤ H(δ) + δ 2 ⋅ R n,p Given (4.27), we deduce that δ ⋆ ≤ 2 s j=1 λj δ (j) + H(δ) + 2R n,p ⋅ δ 2As already seen in the previous two cases, we write that with probability at least 1 -8 n -4 exp(-t 2 2 ) -4 exp(-2p(t 2 -1)) -δ 0 2 δ ⊺ Σδ ≤ δ ⋆ + H(δ) + 2R n,p δ 2 ≤ 3 2 δ ⋆ + 2 δ 2 Rn,p s log(2pe s) ≤ 14R n,p δ 2 s log(2pe s),

					(4 + 4 s n log( 2p √ 2) s )	log(1 δ 0 ) n	⎛ ⎝ A n,p + 2t	T r(C U ) n	⎞ ⎠	≤	1 2	.
	Together with (4.31) this gives that
													1 2	Xδ 2 n ≤ δ ⊺ Σδ,
	and using (4.30) we get
											Xδ n ≤ 4γ	log(1 δ 0 ) n	.
	Furthermore, this leads to
											δ ⋆ ≤ 4(γ) 2 log(1 δ 0 ) n
	and to				δ 2 ≤	(4 + ∑ s √ j=1 λ 2 2)σ j	4γ	log(1 δ 0 ) n	≤	4(4 + sn log(2p s) √ 2)σ	s j=1 log(1 δ 0 ).	λ 2 j .
	Let us recall at this point that H(δ) = (4 +	√	2) ∑	p j=1 δ (j) σ	log(2p j) n	≤ δ * 2. This gives
					δ ⊺ Σδ ≤	3 2	δ ⋆ + 2σ δ 2	s j=1	λ 2 j ≤ (	3 2	+ 2σ) δ 2	s j=1	λ 2 j
	≤ Given the W RE condition (4.12) 1 2 (3 + 4σ) δ 2 ≤ 2 δ 2 s j=1 λ2 j + 1 2 δ * + 2R n,p δ 2 ≤ s j=1 λ 2 j .	1 2	δ * + 4 δ 2	j=1 s	λ2 j	(4.29)
	giving and using Lemma 4, we have δ ⋆ ≤ 8 δ 2			s	δ ⊺ Σδ ≥ λ2	κ 2	δ 2 2
												j=1
								κ 2	δ 2 ≤		1 2	(3 + 4σ)	s j=1	λ 2 j
											≤		1 2	(3 + 4σ)γ	s n	log(	2ep s	)
	hence,
									δ 2 ≤	(3 + 4σ)γ κ	s n	log(	2ep s	).
	hence,
										δ 2 ≤	28 κ	R n,p s log(2pe s).
	(4 +	√	2)	p j=1	δ (j) σ		log(2p j) n	+ σ δ 2	s j=1	λ 2 j ≤ σ(4 +	√ 2)	log(1 δ 0 ) n	Xδ n

j ≤ 8 δ 2 R n,p s log(2pe s).

http://data.worldbank.org/data-catalog/climate-change

mo'o. ' yi lα bwə. Mfì' bə α̂ mbə ngαmsiaŋtə ̀ à à' kə mbə fà' nkə ntəlαg nǔ Ntotə̂ sə̂ lɛ̌n mfì' nə ̀ ma'a. Caŋ yə̂n lɛ̌n mfì' nə ̀ mà' lα bə α̂ nə ̀ totə nkò' à bə̂ nə tɛ̌d tà' Nkʉ̌nù (zə ̀ bǒ nkə ntsiâŋ mǒngə Mbàtənjòŋ lα. Cʉ̌ mǎd zə ̀ bə̂ bǒ kî yə̂n nkò' lα li Y = f(X), dʉ̌' zə ̀ f bə̂ tà' Nə fə zə ̀ ǎ cwɛ̌d ju' α̂ Tûmnù bò kû'ni nə ̀ bə kə ̀ lɛ ǹ nα (yə̂n ngə laŋ nα bǒ tsiâŋ lɛ̌n kαn), kə ̀ nə bə kə tsə bə̂ bǒ fâ' yi lα. nǔm màd nzwə nə ̀ kǎ Ntotə Mbàtə ǹjòŋ, mə bwɔ mə bwɔ ngə lâŋ bǒ cwɛ̌d mfi'tə i α̂ tʉ̂ laŋfà' ndʉb lα. Bǎg zi'tə màd nə ̀ fa' bô nkὰn ǎ nsi lα.

Cʉαtə bwə ntswî màd ǎ nkə nzîn nα nə ̀ lò' ndâ'tə njàŋ bǐn nə tαmtə nǔm bǐn mfǎ'mbu.

From the above, we can infer that ∆ verifies the cone property with c 0 = 3. Hence, given the Restricted Eigenvalues condition (3.20), we can write that ∆ ⊺ Γ∆ ≥ t 0 ∆ S 2 2 . Since,

1 . Then, from the above, we have,

Furthermore, we know that ∆ N 1 ≤ c 0 ∆ S 1 , thus ∆ 1 ≤ (c 0 + 1) ∆ S 1 . We can then conclude that θθ 1 ≤ λ ⋅ s θ ⋅ 12 t 0 with probability larger than 1 -5 exp(-t 2 ), and t large enough.

Chapter 4

Slope for high dimensional linear models with measurement errors

This chapter is based on the manuscript [START_REF] Chanang Tondji | Slope for high dimensional linear models with measurement errors[END_REF].

Abstract

We study the linear regression model Y = X ⋅ β ⋆ + , while considering that the n × p design matrix X is subject to an additive noise given by W = X + U , where n is the number of elements of our dataset, p is the number of covariates and β ⋆ is our true parameter of interest. The response Y ∈ R n and the matrix of covariates W ∈ R n×p are observed. The n × p matrix U is drawn from a Gaussian distribution with a known covariance matrix C U . We consider a Slope based optimization procedure to estimate our parameter of interest. A correction of the least-squares criterion is inevitable to take into account measurement errors. The corrected risk is no longer convex and we project it on the space of convex quadratic functions. We give sufficient conditions on the errors' distribution in order to attain the optimal rates and quantify the loss in the rate otherwise.

we denote by v S , the vector (v i 1{i ∈ S}) i=1,⋯,p . Furthermore, we denote by v (j) , the j-th largest component of v. As an example, v (k) is the k-th largest component of the vector v , whose components are the absolute value of the components of v. We will use the notation ⟨⋅, ⋅⟩ for the inner product with respect to the Euclidean norm and (e j ) j=1,⋯,p for the canonical basis in R p . We will also use the notation (a) + = max{a, 0}, a ∨ b = max{a, b} and a ∧ b = min{a, b} for all a, b ∈ R. Similarly, for all v ∈ R p and A ∈ R p×p , we will have v + = (max{v i , 0}) i=1,⋯,p . The transpose of a matrix X is noted X ⊺ . If we consider a random variable z, the symbol E[z] denotes its expected value. Let us denote by M a p × p matrix. We will consider that

is the spectral or operator norm. Furthermore, M 1 (respectively M ∞ ) is the l 1 induced norm (respectively the l ∞ induced norm). In general, an l q induced norm is given by

The article will be presented as follows. Section 4.2 presents the adjustments performed on the optimization problem (4.2) in order to construct our estimator β of β ⋆ . Section 4.3 gives upper bounds of the estimation risk of β. A numerical analysis of the estimation procedure is provided in section 4.4. All proofs are developed in Section 4.6.

Estimation procedure

Penalized methods are based on unbiased estimators of E 1 n Y -Xβ 2 2 . In order to take into account the errors-in-variables, a correction for the errors is necessary and we consider

C U is the covariance matrix of the p-dimensional vector U .

The empirical version given by

has the major inconvenient of being a quadratic form which is not necessarily convex.

Indeed, the symmetric matrix

may have negative eigenvalues. Therefore, we project Σ on the set S ⪰0 of symmetric positive semi-definite matrices in Frobenius norm. We denote by

The following shows that Σ defined in (4.4) also minimizes the operator norm over S ⪰0 .

Construction of datasets

The n × p elements of the matrix X are drawn from i.i.d [0, 1]-uniform distribution. The elements of are drawn from a centered Gaussian distribution with standard deviation given by σ. The matrix C U is built using a centered Gaussian distribution, with standard deviation µ (we assumed that it exists a c > 0 such that µ = c⋅σ) such that C U is symmetric.

Two cases are considered in order to ensure that C U is symmetric. In the first one, we only draw the diagonal term from the aforementioned Gaussian distribution and we further set all of the non-diagonal term to be equal to 0. In the second case, we draw all the p × p elements of a matrix M from the same Gaussian distribution. The matrix C U is implied from M using the formula

The elements of the parameter of interest β ⋆ are drawn from a [0, 1]-uniform distribution such that β ⋆ 0 = s. Hence, for known n, p, σ, s and c, we can infer using the above methodology a predefined dataset. In fact, having drawn the parameters X, β ⋆ , C U , we then draw the elements of the n × p matrix of U from a centered multivariate Gaussian distribution with matrix of covariance given by C U . The elements of W are given by W = X + U . Similarly, the elements of Y , will be given by Y = X ⋅ β ⋆ + .

Convergence metrics

Let us now considered a predefined dataset given by O = (W, Y ) of size n which is derived using the methodology described above. We then compute an estimator of our parameter of interest β, defined by the optimization problem (4.7). To assess the optimization methodology of our estimator, we compute its Estimate Risk value, for several sample sizes n, for a given p, through a two-step cross-validation process.

In fact, we partitioned the sample in l distinct subsets O subset , of equal size. In the first step, we set aside one of the O subset {i} , which will be used as the test set, and we use the l -1 remaining subsets as training sets. Using the latter, we then compute the estimator β. In the second step, we use the subset O subset {i} (test set left out from the initial estimation), to determine how close the estimator β is to its real value. These two steps are repeated successively l times. For each iteration, we calculate the difference between the estimator and the true oracle value, using the test set. The empirical average of these l distances give us the desired Estimation Risk measure. In brief, the Estimation Risk (ER) value of a sample of size n is given by

where O {i} is the number of elements of the subset O s {i} ubset. For a single sample size n, we computed the Estimation Risk u(>= 20) times. It allowed us to imply the confidence interval associated with the measure.

For any u = (u 1 , ⋯, u p ) ∈ R p , we define :

If ∼ N (0, σ 2 I n×n ), then the random event

is of probability at least 1δ 0 2. Moreover, by the Cauchy-Schwarz inequality, we have

Auxiliary results

We say that a real valued random variable V is ν 2 -subGaussian if E[e tV ] ≤ exp(t 2 ν 2 2), for all t. We denote by ⋅ ψ 2 the sub-Gaussian norm defined by

. Following [START_REF] Bunea | On the sample covariance matrix estimator of reduced effective rank population matrices, with applications to fpca[END_REF], we say that a random vector V of R p is subGaussian if V ⊺ u is subGaussian for all p-dimensional vector u and define the subGaussian norm using the Orlicz norm

We also make the additional assumption that there exists a constant c 0 > 0 such that

, with probability larger than 1 -2e -u 2 , for all u > 0.

Lemma 6. Let us consider two arbitrary vectors v, w ∈ R p . If U has independent rows, identically distributed as a subGaussian vector satisfying (4.16) we have, with probability at least 1 -4 n :

where c > 0 depends only on c 0 . Note that, if the rows {U i⋅ } i=1,...,n are Gaussian, then c 0 = 1.

Proofs

Proof of Lemma 9. We know that through Cauchy-Schwarz inequality we can write

We also know that Σ is the positive semi-definite projection of Σ. As such, for all Σ 0 ∈ S ⪰0 , we will have Σ-

We give 2 different bounds on this last quantity. Firstly, the squared matrix 1 n (X ⊺ U + U ⊺ X) is symmetric. Therefore, its spectral norm is the smallest of all matrix induced norms. Hence, we can write

On the one hand, we have

Let us bound from above the Laplace transform on V j . We will have

We can then deduce that

and then, for t > 1,

Lemma 10. Let us recall the definition R n,p in (4.9). We have

with probability larger than 1 -8 n -4 exp(-t 2 2 ) -4 exp(-2p(t 2 -1)).

Proof of Lemma 10. We know that

Given Lemma 9 and the definition (4.10) of A n,p , we have

with probability at least 1 -4 n -4 exp(-2p(t 2 -1). Furthermore, from Lemma 6, we have

with probability at least 1 -4 n . Moreover, from Lemma 7 we will have

with probability at least 1 -2 exp(-t 2 2 ). Finally, from Lemma 8 we have

with probability at least 1 -2 exp(-t 2 2 ). Given the results (i) -(iv), we can conclude that

with probability larger than 1 -8 n -4 exp(-t 2 2 ) -4 exp(-2p(t 2 -1)).

Additional Proofs

Proof of Lemma 1. By definition of Σ, for any symmetric, positive definite matrix M , we know that

The matrix Σ can be rewritten as Σ = Σ p i=1 ηi + v i v ⊺ i where η i is the i-th eigenvalue of the matrix Σ and (v i ) i=1,⋯,p are its eigenvectors. Similarly, Σ = Σ p i=1 ηi v i v ⊺ i . For convenience, let us assume that η1 ≥ η2 ≥ ⋯ ≥ ηp . Furthermore, we will also assume that ηp < 0, otherwise Σ = Σ and the result is trivial. We would like to show that Σ is also a solution of arg min

Let Σ be any positive semidefinite matrix, we will have

Proof of Lemma 2. Let us consider δ ⊺ Σδ. We write that

We can imply from the above that both We also know that, using Lemma 6 and Lemma 8