Improved bounds for Square-root Lasso

Keywords: Sparse linear regression, minimax rates, high-dimensional statistics, adaptivity, square-root estimators root Slope Median-of-means, robustness, adaptivity, minimax optimal rates. Based on [37]: Derumigny, A., Robust-to-outliers simultaneous inference and noise level estimation using Conditional copula, simplifying assumption, bootstrap. Based on [38]: Derumigny, A., & Fermanian, Conditional dependence measures, kernel smoothing, conditional Kendall's tau. Based on [40]: Derumigny, A., & Fermanian

substantiel de la thèse en franc ¸ais Dans cette thèse, nous étudions les propriétés de plusieurs modèles statistiques. Dans la majorité des cas, il s'agira de modèles de dépendance, c'est-à-dire de modèles statistiques qui spécifient les interactions entre différentes variables aléatoires.

Le premier modèle étudié est le modèle de régression linéaire en grande dimension. Classiquement, on observe n réplications indépendantes et identiquement distribuées d'une variable aléatoire réelle Y , que l'on cherche à expliquer par p variables explicatives X 1 , . . . , X p . On suppose qu'il existe un vecteur β * ∈ R p tel que Y = X T β * +ε, où X = (X 1 , . . . X p ) et ε est une variable aléatoire indépendante de X. On s'intéresse plus particulièrement au cas dit "sparse", où le vecteur β * est composé d'un grand nombre de composante nulles. Autrement dit, on suppose que la sparsité s est faible, où s := Card{i :

Dans ce cadre, il est habituel d'utiliser des estimateurs de type moindres-carrés pénalisés, qui peuvent permettent d'atteindre des vitesses optimales au sens minimax (cf. [12]). Néanmoins, ces estimateurs nécessitent la connaissance a priori de l'écart-type σ * du bruit ε. Dans le Chapitre 2, nous prouvons que deux estimateurs, le Square-root Lasso et le Square-root Slope permettent d'atteindre les vitesses optimales à distance finie sans nécessiter la connaissance de σ * . En outre, le Squareroot Slope ne nécessite pas non plus la connaissance de la sparsité s. Nous détaillons également des algorithmes permettant le calcul du Square-root Slope. Dans le Chapitre 3, nous proposons des modifications de ces estimateurs, qui leur permettent d'être davantage robustes. En effet, les estimateurs traditionnels à base de moindres carrés sont, tout comme la moyenne empirique, très sensibles aux valeurs aberrantes, et la présence d'une seule valeur aberrante peut considérablement réduire les performances d'un estimateur (cf. [91]). Dans cette optique, nous proposons une version dite MOM (pour médiane-de-moyennes) des estimateurs adaptatifs du précédent chapitre. Ces estimateurs MOM adaptatifs peuvent encore atteindre les vitesses optimales en même temps qu'estimer l'écart-type du bruit σ * . Nous proposons également un algorithme permettant de calculer ces estimateurs.

La seconde partie de cette thèse est consacrée aux modèles de copules conditionnelles. Ces modèles permettent de mettre en valeur la fac ¸on dont la dépendance entre les différentes composantes d'un vecteur X varie en fonction d'une variable explicative Z. Formellement, par le théorème de Sklar, on peut décomposer la fonction de répartition conditionnelle de X sachant Z = z de la fac ¸on suivante

la fonction de répartition conditionnelle jointe de X sachant Z, pour i ∈ {1, . . . , n}, F Xi|Z est la fonction de répartition conditionnelle de X i sachant Z et C X|Z est la copule conditionnelle de X Contents 10 sachant Z. L'idée principale d'une telle décomposition est de séparer l'influence de Z sur les variables marginales X i d'une part ; et l'influence de Z sur la dépendance entre les composantes de Z, d'autre part. Cette dernière est représentée par la copule conditionnelle de X sachant Z. Autrement dit, dans le cas général, pour chaque valeur z de la variable conditionnante Z, il existe une copule conditionnelle C X|Z ( • |Z = z) décrivant la dépendance entre les différentes composantes de X conditionnellement à l'évènement Z = z. Certains auteurs disent qu'un tel niveau de complexité n'est pas nécessaire, et affirment parfois qu'une modélisation fixe de la copule conditionnelle est préférable. Cela permettrait d'estimer une copule fixe, plutôt qu'une famille (infinie) de copules indexée par le paramètre z. D'autres disent qu'un tel modèle est peu susceptible d'apparaître en pratique, et ne saurait être proche de la réalité en général. Dans le chapitre 4, on développe des tests de cette hypothèse, appelée "hypothèse simplificatrice". Formellement, cette hypothèse peut s'écrire "z → C X|Z ( • |Z = z) est une fonction constante", c'està-dire que la fonction C X|Z ( • |Z = z) ne dépend pas du choix de z. Une première idée naturelle est de construire un test de H 0 basé sur une comparaison entre la copule conditionnelle C I|J estimée avec et sans l'hypothèse simplificatrice. Ces estimateurs seront appelés respectivement Ĉs,I|J et ĈI|J . Ainsi, en introduisant une certaine distance D entre copules conditionnelles, un test peut être basé sur la statistique D( ĈI|J , Ĉs,I|J ). Nous proposons des procédures de ré-échantillonage pour évaluer la loi limite de telles statistiques de test et prouvons la validité d'un schéma de ré-échantillonage semiparamétrique spécifique. Si l'hypothèse simplificatrice est rejetée, il faut modéliser la dynamique de la dépendance en fonction de z. Pour ce faire, nous utilisons la version conditionnelle d'un indicateur de dépendance usuel, le tau de Kendall. Pour un entier p fixé, et pour chaque z ∈ R p , le tau de Kendall conditionnel d'un vecteur

bivarié X := (X 1 , X 2 ) sachant un vecteur de covariables Z = z est défini par

) sont deux copies indépendantes de (X, Z).

On peut noter que le tau de Kendall conditionnel appartient toujours à l'intervalle

Contrairement aux corrélations, cette mesure a l'avantage d'être toujours bien définie, même si l'un des X k , k = 1, 2, n'admet pas de moments d'ordre 2. C'est le cas lorsqu'il suit une loi de Cauchy par exemple. Dans le chapitre 5, nous proposons des estimateurs du tau de Kendall conditionnel utilisant des techniques d'estimation à noyau. Nous prouvons leurs propriétés asymptotiques et à distance finie, sous des conditions faibles. Puis, dans le chapitre 6, nous proposons un modèle de type régression pour le tau de Kendall conditionnel, de la forme Λ(τ 1,2|z=z ) = ψ(z) T β * , où β * est un paramètre à estimer, Λ est une transformation connue et ψ est un dictionnaire de fonctions. Un tel modèle peut être utile également pour donner une estimation directe d'effets marginaux, du type ∂τ 1,2|z=z /∂z 1 , qui quantifient l'influence locale d'une des variables sur la dépendance entre X 1 et X 2 . Dans le chapitre 7, nous montrons les liens existants entre l'estimation du tau de Kendall conditionnel et les problèmes de classification. Ainsi, soient W

). Nous pouvons alors Contents 11 remarquer que la prédiction de la concordance/discordance parmi les paires d'observations (X 1 , X 2 ) sachant Z peut être vue comme un problème de classification de ces paires. Si un modèle est capable d'estimer la probabilité conditionnelle d'observer des paires concordantes d'observations, alors il est capable d'estimer le tau de Kendall conditionnel. De telles probabilités conditionnelles sont justement les sorties données par la plupart des algorithmes de classification usuels. Ainsi, la plupart des classifieurs peuvent être utilisés ici (par exemple les classifieurs linéaires, les arbres de décision, les forêts aléatoires, les réseaux de neurones, et ainsi de suite), mais appliqués ici à des paires d'observations. Dans la troisième partie de cette thèse, nous abordons deux autres thèmes liés à l'inférence des modèles statistiques. Dans le chapitre 8, nous montrons comment les techniques développées dans le chapitre 6 peuvent se généraliser au cas de n'importe quelle U-statistique conditionnelle. Finalement, dans le chapitre 9, nous étudions la construction d'intervalles de confiance pour des ratios de moyennes. Nous montrons que les estimateurs asymptotiques classiques construits à partir du théorème central limite et de la delta-méthode peuvent ne pas marcher dans certains régimes. Nous proposons des indicateurs fondés sur des théorèmes d'impossibilité pour détecter de tels cas, et proposons d'autres intervalles de confiance, uniformément valides sur des classes de distributions. Contents In [91], Lecué and Lerasle propose to replace the empirical mean by an empirical median-of-mean (MOM) and use a minimaxisation trick as follows. 1. Given a dataset D of size n, we can divide it into K blocks D 1 , . . . D K of size n/K (assumed to be an integer) corresponding to a partition {1, . . . , n} = B 1 • • • B K ;

Introduction

In the era of big data, statistics and data science are increasingly useful and necessary to analyze the huge volume of information that is available. Of primary importance are the tasks of estimation and prediction: estimation because we want to infer some parameter or functional that determine the unknown data generating process, and prediction because we want to learn from some datasets to deduce some information about the future.

Often, several random variables X 1 , . . . , X d are available and interact with each other. Therefore it is obvious that inference from the joint law of X := (X 1 , . . . , X d ) should take into account the potential dependence between the different components of X.

Indeed, in applications, many datasets are fundamentally multivariate. Let us describe shortly a few examples: in finance, we may want to model the joint distribution of the returns of several assets ; in hydrology, it is important to model the joint distribution of the characteristic of several rivers ; in social sciences, usually several variables are available, in the individual level (income, wealth, age and so on) as well as in the aggregated level (GDP, unemployment rate, average life expectancy and so on) ; in biostatistics, interactions between different genes or different characteristics of the patients may be of interest.

In this thesis, we study two main dependence modeling frameworks : the high-dimensional sparse linear regression, studied in Chapters 2 and 3, and the conditional dependence framework, studied in Chapters 4, 5, 6 and 7. Other related topics are discussed in the last part : estimation of conditional U-statistics in Chapter 8, and confidence intervals for ratios of means in Chapter 9. 2. Improved bounds for Square-root Lasso and Square-root Slope 3. Robust-to-outliers simultaneous inference and noise level estimation 4. About tests of the simplifying assumption 5. About kernel-based estimation of the conditional Kendall's tau 6. About Kendall's regression 7. A classification point-of-view of the conditional Kendall's tau 8. Estimation of a regular conditional functional 9. Confidence intervals for ratios of means

Least-squares estimators and penalization

One of the oldest well-known dependence model is the linear regression. In such cases, the statistician tries to explain specifically one of the variables, called for example Y , using the other variables X 1 , . . . , X p . As usual, we assume that there is a true parameter β * ∈ R p such that Y = X T β * + ε, where X := (X 1 , . . . , X p ) is the vector of explanatory variables and ε is a random variable representing the uncertainty. Indeed, in the general case, there is no reason why there would be enough information in X to completely explain the variable Y . We observe n > 0 i.i.d. replications of (X, Y ), denoted by (X 1 , Y 1 ), . . . , (X n , Y n ), and we define the design matrix as X := (X 1 , . . . , X n ) T and Y := (Y 1 , . . . , Y n ) T .

As our goal is to infer the unknown parameter β * , we can invoke the usual least-squares estimator βLS := arg min β∈R p ||Y -Xβ|| 2 n , where ||u|| 2 n := (1/n) n i=1 u 2 i for a vector u ∈ R n .

If p > n, the estimator βLS may not be relevant. Often, it is no longer unique, and may lead to a perfect reconstruction of the signal Y as an artifact of lack of observations due to the small size n. In this case, additional assumptions have to be made about the model to make the task of inference easier.

The most usual assumption is the so-called sparsity of the vector β * : a vector β * is said to be s-sparse for a given integer s > 0 if the number of nonzero coefficients of β * , denoted by |β * | 0 ≤ s. In other words, we assume that Y is not affected by all the variables X 1 , . . . X p , but only by s of them. Obviously, the statistician does not have knowledge of the precise set of variables that are relevant. For example, in biostatistics, the statistician may assume that only a few genes are relevant without knowing which ones.

As a consequence of this sparsity assumption, we do not want to have β "too big", in the sense that large values of β should be seen as less appropriate, even if they seem to lead to a better fit. Therefore, we introduce penalized least-squares estimators of β * , of the form βpen := arg min

β∈R p ||Y -Xβ|| 2 n + pen(β),
where pen(•) is a real-valued function of β ∈ R p . Choice of the penalization function is not an easy task, and different alternatives are possible. The penalization pen(β) := λ × |β| 0 with a tuning parameter λ seems to be a natural choice, but this estimator is NP-hard to compute, meaning that it is impossible to compute in polynomial time.

The Lasso estimator [START_REF] Tibshirani | Regression shrinkage and selection via the lasso[END_REF] uses a convex relaxation of this penalization, with pen(β)

:= λ × |β| 1 ,
where | • | 1 is the l 1 -norm, i.e. the sum of the absolute values of the coefficients. Bellec, Lecué and

Tsybakov [START_REF] Bellec | Slope meets lasso: improved oracle bounds and optimality[END_REF] have shown that it is possible to choose a tuning parameter λ Lasso, opt. such that the Lasso estimator with this choice of λ attains the optimal minimax rates of prediction and estimation in this sparse framework, under an assumption on the design matrix X.

Nevertheless, the tuning parameter λ Lasso, opt. depends on the sparsity s, i.e. the number of non-zero coefficients of β, that is unknown in practice. Bellec, Lecué and Tsybakov [START_REF] Bellec | Slope meets lasso: improved oracle bounds and optimality[END_REF] provide two possibilities to solve this issue. The first solution is to use a Lepski-type procedure to aggregate all the Lasso estimators for every choice of s. As a result, the aggregated estimator is adaptive to s. The second solution consists in using a different estimator, the Slope.

The Slope estimator [START_REF] Bogdan | Slope -adaptive variable selection via convex optimization[END_REF] is another penalized least-squares estimator, whose penalty is defined by

|β| * := p j=1 λ j |β| (j)
, with tuning parameters λ 1 ≥ • • • ≥ λ p > 0, defining |β| (j) as the j-th largest component of |β|. The intuition behind the Slope estimator is simple: larger components of β should be more penalized than smaller ones. Bellec, Lecué and Tsybakov [START_REF] Bellec | Slope meets lasso: improved oracle bounds and optimality[END_REF] have shown that there exists optimal tuning parameters λ Slope, opt. 1 , . . . , λ Slope, opt. p such that the Slope estimator also attains the optimal minimax rates of estimation and prediction.

Adaptivity to σ using two square-root estimators

Both the Lasso and the Slope estimators attains optimal rates, but the chosen tuning parameter depends on σ, the standard deviation of the noise ε. This may cause a problem since there is no reason why σ should be known in practice, making both estimators impossible to compute due to a lack of knowledge about σ. Looking closer at both estimators, it appears that the "optimal" tuning parameters are both proportional to σ. We can therefore rewrite those estimators as βpen := arg min β∈R p ||Y -Xβ|| 2 n + σ × pen(β), where pen(β) is a "standardized" version of the penalty.

(Y i -X T i β) 2 .
The name "square-root" follows from the first part of the objective function ||Y -Xβ|| n , which is the square-root of the usual least-squares criteria. A first member of this family is the square-root Lasso estimator, defined as the square-root estimator using the | • | 1 penalization. It was first introduced by Sun and Zhang [START_REF] Sun | Scaled sparse linear regression[END_REF] and Belloni et al. [START_REF] Belloni | Square-root lasso: pivotal recovery of sparse signals via conic programming[END_REF] under the name "scaled Lasso". In a similar way, we define the square-root Slope using the penalization |β| * .

In Chapter 2, we show that the square-root Lasso estimator βSQL attains the minimax rates of estimation and prediction under Gaussian noise and a restricted eigenvalue condition about the design matrix X. Indeed, there exists universal constants C 1 , C 2 > 0, such that for every n large enough, with IP β * -probability at least 1 -(s/p) s -(1 + e 2 )e -n/24 , the estimator βSQL with the tuning parameter chosen as λ SQL, opt. := γ (1/n) log(2p/s) and γ ≥ 16 + 4 √ 2 satisfies

||X( βSQL -β * )|| n ≤ C 1 κ 2 σ s n log p s , ∀q ∈ [1, 2], | βSQL -β * | q ≤ C 2 κ 2 σs 1/q 1 n log 2p s ,
where |•| q is the l q -norm and κ is a constant that only depend on the design X. This estimator is adaptive to σ in the sense that λ SQL, opt. does not depend on σ anymore, as opposed to the tuning parameters of the usual Lasso and Slope.

Nevertheless, the square-root Lasso estimator is not adaptive to s, in the sense that knowledge of the true sparsity s is necessary to achieve this bound. Using a Lepski-type adaptation of this estimator, we prove that an aggregated estimator of the square-root Lasso achieves optimality in the prediction norm || • || n or in an estimation norm | • | q , for a fixed q ∈ [1,2] without any knowledge on s. In fact, we show a more general result: any family of estimators of β depending on s admits an adaptive to s aggregated version that achieve the same rate as the oracle estimator with the true s. Computation of this aggregated version is detailed in Algorithm 2.

Combining the ideas of the Slope and the square-root estimators, the Square-root Slope should be adaptive to both s and σ. We explain how any algorithm to compute the Slope estimator can be adapted to compute the Square-root Slope. Then, we show that there is a universal choice of tuning parameters λ SQS, opt. j := γ (1/n) log(2p/j) independent of s and σ, and γ can be any real greater or equal to

16 + 4 √ 2.
With such a choice of tuning parameters and under an assumption on the design matrix X, there exists universal constants C 1 , C 2 > 0, such that for every n large enough, with IP β * -probability at least 1 -(s/p) s -(1 + e 2 )e -n/24 , the estimator βSQS satisfies

||X( βSQL -β * )|| n ≤ C 1 κ 2 σ s n log p s , | βSQL -β * | 2 ≤ C 2 κ 2 σs 1/2 1 n log 2p s ,
where κ is a constant that only depends on the design X. This shows that the Square-root Slope attains the minimax optimal rates of prediction and estimation in the l 2 norm.

Robustness to outliers using the Median-of-Means approach

Robustness is a natural idea in statistics: it seems interesting to ask what would happen to the estimator if the model is wrong, or if some observations have been "contaminated". Sometimes, the estimators are not robust, in the sense that only one outlier is enough to destroy the whole performance of an estimator.

Following [START_REF] Donoho | The notion of breakdown point[END_REF], we define the breakdown point of an estimator T by : * (T, D I ) = min In other words, it is the minimum number of points that we need to add to the dataset D I of informative data such that the estimator T can lie arbitrary far away from its previous value T (D I ). The quantity * (T, D I ) can be therefore seen as a measure of the robustness of the estimators (see [START_REF] Lecué | Robust machine learning by median-of-means: theory and practice[END_REF] for a review of such measures of robustness and new robust estimators).

Most of the times, the breakdown point of an estimator does not directly depends on the dataset D I , but it is often only a function of its size n = |D I |. For example, the breakdown point of the empirical mean is 1/(n + 1), meaning that one needs only to add a single outlier to any dataset to make the empirical mean arbitrary high. The median is another location estimator, with a better breakpoint of 1/2. Indeed, we need to double the size of the any dataset in order to be able to arbitrary modify its median.

The same problem happens with the Lasso and other least-squares estimators : a single outlier added to any dataset can reduce strongly the performance of the Lasso estimator, as proved in [START_REF] Lecué | Robust machine learning by median-of-means: theory and practice[END_REF].

This problem affects in fact all the previous estimators since they are based on the minimization of the 2 , which is a kind of empirical mean.

empirical risk ||Y -Xβ|| n = 1 n n i=1 (Y i -X T i β)
2. On each block B k , we compute the criteria of β against β

P B k (l β -l β ) := 1 |B k | i∈B k (Y i -X T i β) 2 -(Y i -X T i β) 2 ,
for every β, β ∈ R p ;

3. The global MOM criteria of β against β is defined by M OM K (l β -l β ) := M edian P B k (l β -l β ), k = 1, . . . , K ;

4. Finally, the MOM-K estimator of β is defined by βMOM-K := arg min

β∈R p max β∈R p M OM K l β -l β + λ pen(β) -pen( β) , (1.2)
where λ is a tuning parameter and pen(•) is a penalty function, such as the Lasso or the Slope penalty.

They show that the resulting estimator, called the MOM-Lasso (resp. MOM-Slope) attains the optimal minimax rate of convergence while having a breakdown point of (K/2)/(n + K/2). Indeed, both estimators are robust to up to K/2 outliers, because such a small number of outliers could not affect the median in step 3 above. Lecué and Lerasle also propose similar estimators for a more general framework where Y i = f (X i ) + ε, with f belonging to a space of functions F.

Nevertheless, their estimators are not adaptive to the moments of ε, meaning that, to attain the optimal rates, the tuning parameter λ has to be chosen in a way that depends on the standard deviation of ε, which is unknown in practice. We propose, in Chapter 3, an adaptive version of the MOM-Lasso and MOM-Slope. Moreover, it allows a simultaneous inference of the noise level, i.e. the standard deviation σ of ζ. With the Lasso penalty, this joint estimator is defined by ( β, σ) Adaptive-M OM -Lasso := arg min

β∈R d , σ>σmin max β∈R d , χ>σmin M OM K l β σ + σ - l β χ -χ + µ |β| 1 -| β| 1 , (1.3)
where

M OM K l β σ + σ - l β χ -χ := M edian P B k l β σ + σ - l β χ -χ , k = 1, . . . , K
and, for all k = 1, . . . , K,

P B k l β σ + σ - l β χ -χ := 1 |B k | i∈B k (Y i -X T i β) 2 σ - (Y i -X T i β) 2 χ .
We prove that this adaptive version is still robust to up to K/2 outliers and still attains the optimal rates of convergence in a non-asymptotic way. Moreover, the optimal choice of the tuning parameter µ that allows to attain these rates does not depend on σ anymore. This means our estimators is also adaptive to σ. We propose also a generalization of this estimator to the framework where Y i = f (X i ) + ε, and prove a bound on the errors of the estimator ( f , σ) where f is an estimator of f and σ is an estimator of the standard deviation of ε.

Copulas and conditional dependence modeling 1.2.1 Distributions with given margins

In the previous section, we studied different ways of estimating adaptively the conditional mean of a variable Y given other variables X 1 , . . . , X p . Nevertheless, more general models are needed if one wants to estimate a multivariate law without any specific separation between explained or explanatory variables. More precisely, in our framework, the statistician is given n i.i.d. replications X 1 , . . . , X n of a random vector X in R d , and our goal is to estimate the law of X. It can be convenient to model use a parametric model for the law of X, so that the values of the estimated parameters can be easily interpreted in application. Often, it can be challenging to find a good parametric model for the data, especially in a multivariate framework.

We will distinguish two kind of parameters: on the first side, marginal parameters, i.e. parameters that only influence the univariate margins X 1 , . . . , X d ; and on the other side, "pure" dependence parameters.

For example, assume that d = 2 and that law of X is bivariate Gaussian with means µ 1 , µ 2 , standard deviation σ 1 , σ 2 and correlation ρ. Then (µ 1 , µ 2 , σ 1 , σ 2 ) is a vector of marginal parameters and ρ is a pure dependence parameter, meaning that the distributions of X 1 and X 2 do not change with ρ. On the contrary, the covariance Cov 1,2 := σ 1 σ 2 ρ is a somehow mixed parameter that we would like to avoid, since it contained some information about the margins and about the dependence at the same time.

One fruitful idea for inference is a generalization of this idea. This is the fundamental concept of copula modeling, giving general and flexible estimation techniques such as inference from margins (see Algorithm 1), where the marginals are estimated first and the dependence is modelled in a second step.

The concept of copula itself comes from Sklar [START_REF] Sklar | Fonctions de repartition an dimensions et leurs marges[END_REF] in 1959 (see Sklar [START_REF] Sklar | Random variables, distribution functions, and copulas: a personal look backward and forward[END_REF] and Scheizer [START_REF] Schweizer | Thirty years of copulas[END_REF] for historical references). Probabilists were interested in properties of several classes of distributions, in particular distributions with given margins. For example, if we have d continuous distributions F 1 , . . . , F d on R, how can we construct d-dimensional distributions F 1:d whose margins are F 1 , . . . , F d ?

Theorem 1.1 [START_REF] Sklar | Fonctions de repartition an dimensions et leurs marges[END_REF]. Let d ≥ 2 be an integer.

1. Let F 1:d be a distribution function on R d with continuous margins F 1 , . . . , F d , and X ∼ F 1:d . Then there exists a distribution C on [0, 1] d with uniform margins, named the copula of X 1 , . . . , X d such that the following equation holds

∀x = (x 1 , . . . , x d ) ∈ R d , F 1:d (x) = C F 1 (x 1 ), . . . , F d (x d ) ,
and C is given by

∀u = (u 1 , . . . , u d ) ∈ [0, 1] d , C(u 1 , . . . , u d ) = F 1:d F - 1 (u 1 ), . . . , F - d (u d ) ,
where F - i is the inverse function of F i , for i = 1, . . . , d. 3. Moreover, C is the joint distribution of U = (U 1 , . . . , U d ) where U i := F i (X i ) for i = 1, . . . , d when X = (X 1 , . . . , X d ) ∼ F 1:d .

Conversely, if

Therefore, we have a bijection between the joint cdf F 1:d and the decomposition (F 1 , . . . , F d , C). This allows to separate on the one hand the marginal distributions F 1 , . . . F d , that can be estimated separately, with possibly different models, and on the other hand the copula C, which summarizes the whole dependence between the components of X. The copula C can be understood as a standardization of the law of X where all the information about the margins has been removed. Indeed, for every j ∈ {1, . . . , d}, U j := F j (X j ) follows a uniform distribution on [0, 1] and this is true as long as the marginal distributions F i are continuous, that will be assumed everywhere in the following.

X 1

. . .

X d U 1 U d . . . F 1 F d C F

Inference of copulas models

In practice, the univariate margins F 1 , . . . F d are unknown and need to be estimated in a first, preliminary step. Let F1 , . . . , Fd be their respective estimators. They can be of any type (parametric, semi-parametric or non-parametric), but a simple choice consists in using the empirical marginal cumulative distribution function, i.e. Fj (t) := (1/n) n i=1 1{X i,j ≤ t}.

From these estimators, we compute the pseudos-observations Ûi,j := Fj (X i,j ) for every 1 ≤ i ≤ n, 1 ≤ j ≤ d. Indeed, the statistician cannot access the true values U i,j := F j (X i,j ) since they are not observe and depend on the unknown cdfs F 1 , . . . F d . Strictly speaking, for every i ∈ {1, . . . , n}, the random vector (U i,1 , . . . , U i,d ) follows the copula C, but the pseudo-observation Ûi := ( Ûi,1 , . . . , Ûi,d )

will not follow C. As n → ∞, the estimated marginals cdfs F1 , . . . , Fd should converge to the true univariate cdfs, so it is still relevant to use the pseudos-observations ( Ûi,j ) i,j to estimate C.

The equivalent of the empirical cdf in this setting is the empirical copula, defined by

∀u := (u 1 , . . . , u d ) ∈ [0, 1] d , Ĉn (u) := 1 n n i=1 1{ Ûi ≤ u} = 1 n n i=1 1{ Ûi,1 ≤ u 1 , . . . , Ûi,d ≤ u d }. (1.4)
Note that the empirical copula Ĉn is not a copula since its margins are not continuous, and therefore not uniform on [0, 1]. The empirical copula has been well-studied and converges to the true copula under mild assumptions, see Fermanian et al. [START_REF] Fermanian | Weak convergence of empirical copula processes[END_REF] and references therein.

Several parametric models of copulas are also available. One of the simplest copula is the Gaussian copula with correlation matrix Σ, defined as the copula of the Gaussian distribution N (0, Σ). One can similarly construct the Student copula as the copula of the multivariate Student distribution with correlation matrix Σ and degrees of freedom ν.

Another important class of copulas is the class of Archimedean copulas. A copula C of dimension 2 is called Archimedean if there exists a function ϕ convex, continuous, strictly decreasing from [0, 1] to Chapter 1. Introduction 20

[0, +∞[ such that ϕ(1) = 0 and ∀(u, v) ∈ [0, 1] 2 , C(u, v) = ϕ (-1) ϕ(u) + ϕ(v) . Archimedean copulas also exist in dimensions strictly larger than 2, under stronger assumptions on the function ϕ. In practice, we can fix a family of generator (ϕ θ ) θ∈Θ for a low dimension parameter space Θ, and a parameter θ that can be easily estimated. Several examples of such families are given in [START_REF] Nelsen | An introduction to copulas[END_REF][Table 4.1].

More generally, assume that we have a finite-dimensional parameter space Θ ⊂ R p , and a parametric family of copulas C := {C θ , θ ∈ Θ}, with densities c θ . As before, we assume that we are given n i.i.d.

replications X 1 , . . . , X n of a random vector X ∈ R d , whose copula C θ * belongs to the parametric family C, for a true unknown parameter θ * ∈ Θ. One of the main techniques for estimating θ * is the so-called Canonical Maximum Likelihood Estimation, which simply the pseudo-maximum likelihood estimation of the model using the pseudos-observations Ûi,j := Fj (X i,j ), i = 1, . . . , n, j = 1, . . . , p where Fj is the estimated univariate cdf of X j . Usually Fj is chosen as the empirical cdf of X j , that allow for any marginal distributions. This gives an estimator θ defined by θ := arg max θ∈Θ n i=1 log c θ Ûi,1 , . . . Ûi,n .

(1.5) Such an estimator is convergent and asymptotically normal under usual regularity conditions, see Tsukahara [START_REF] Tsukahara | Semiparametric estimation in copula models[END_REF] for details and a proof.

Algorithm 1: Algorithm for the inference from margins Input: n i.i.d. observations X 1 , . . . , X n of dimension d for j ← 1 to d do estimate the marginal law F j by the empirical cdf Fj or using a parametric model ;

for every i = 1, . . . , n, compute the pseudos-observation Ûi,j := Fj (X i,j ) ; end estimate the copula Ĉ using the pseudos-observations Û1:n,1:d

• either by the empirical copula, defined in Equation (1.4)

• or by Canonical Maximum Likelihood Estimation Ĉn := C θ where θ is defined by Equation (1.5) ;

Output: d marginals F1 , . . . Fd and a copula Ĉ.

Conditional copulas and the simplifying assumption

We study now a related framework, where the statistician observe i.i.d. replications of a vector X = (X I , X J ) where X I ∈ R p is a vector of conditioned variables and X I ∈ R d-p is a vector of conditioning variables, in the sense that we want to model the law of X I given X J . In the previous sections, we have separated marginal and dependence parameters of a given distribution F . Similarly, we would like to separate

• "conditional marginal parameters", i.e. parameters linked to the conditional marginals cdfs F j|J := F Xj |J of X j given J for j = 1, . . . , p;

• "conditional dependence parameters", i.e. parameters linked to the conditional copula C I|J of X I given X J .

Chapter 1. Introduction

This conditional copula exists by the conditional version of Sklar's Theorem (Theorem 1.1), by which we can decompose the conditional multivariate cdf F I|J as follows ∀x I ∈ R p , ∀x J , F I|J (x I |X J = x J ) = C I|J F 1|J (x 1 |X J = x J ), . . . , F p|J (x p |X J = x J ) X J = x J . (1.6)

These conditional copulas have been introduced by Patton [START_REF] Patton | Estimation of multivariate models for time series of possibly different lengths[END_REF][START_REF] Patton | Modelling asymmetric exchange rate dependence[END_REF] and in a more general context by Fermanian and Wegkamp [START_REF] Fermanian | Time-dependent copulas[END_REF]. For example, in a time series context, we may have a sequence of random vectors (X t ) t indexed by the time t ∈ Z. To predict one observation using the previous one in a Markov-chain like model, we would need to estimate the conditional law of X t+1 given X t . This is close to the previous framework, with the formal choice X I := X t+1 and X J := X t . In this case, the conditional copula of X t+1 given X t can be understood as the prediction of the dependence between the different components of X t+1 given X t .

Conditional copulas also naturally appear in the so-called vine framework, see [START_REF] Joe | Families of m-variate distributions with given margins and m (m-1)/2 bivariate dependence parameters[END_REF][START_REF] Joe | Dependence modeling: vine copula handbook[END_REF][START_REF] Bedford | Vines: A new graphical model for dependent random variables[END_REF]. Let us detail this idea. Using Bayes' theorem, one can show that any (unconditional) copula of dimension d can be decomposed using d(d -1)/2 bivariate conditional copula. By this term, we mean conditional copulas where the conditioned vector X I is of dimension 2, while the conditioning vector X J has a dimension between 0 and d-2. This decomposition, also called pair-copula construction [1], allows a a very flexible way of constructing any multivariate copula.

Getting back in the classical framework, the conditional copula of an explained random vector X I given an explanatory vector X I can be used to explain how the dependence among the components of X can change with the values of the conditioning variable. Indeed, in the general case, the conditional copula of X I given X J = x J does depend on the conditioning variable x J . Sometimes, to make the inference easier, people assume that the conditional copula is constant with respect to the conditioning variable x J . This is called the "Simplifying Assumption" for a given conditional copula model and may or may not be satisfied in practice, i.e. with a given data-generating process. A visual representation of the simplifying assumption when d = 2 is given on In Chapter 4, we provide some tests of this Simplifying Assumption H 0 . Formally, it is defined as follows:

H 0 : the function x J → C I|J (•|X J = x J ) is constant. (1.7)
Let us define C s,I|J be the cdf of the vector Z I|J := (F 1|J (X 1 |X J ), . . . , F p|J (X p |X J ) , called the simplified, or partial copula. We prove in Proposition 4.4 that H 0 is equivalent to the independence be-

tween Z I|J and X J , and that if the conditional copula is constant with respect to the conditioning variable, then it is constant to the copula C s,I|J . Note that the simplifying assumption does not imply that C I|J (u I |X J = x J ) = C I (u J ) where C I is the usual (unconditional) copula of X I (see Remark 4.1). This means that the simplified copula C s,I|J has no reason to be equal to the usual copula C I .

Nevertheless, we can have an estimate of Ĉs,I|J by averaging kernel-based estimators of the conditional copula ĈI|J , or as the empirical cdf or empirical copula of Z I|J . This allows us to adapt any test statistics for the constancy of a function to the test of our simplifying assumption. for some weight function of bounded variation w, that could be chosen as random. Other tests statistics can be constructed using the fact that H 0 is equivalent to the independence between Z I|J and X J , recycling usual independence test statistics.

Similar test statistics can be constructed under parametric assumptions. Let C := {C θ , θ ∈ Θ} be a family of copulas and assume that for every x J ∈ R p-d , there exists a conditional parameter θ(x J ) ∈ Θ

such that C I|J (•|X J = x J ) = C θ(x J )
. Then H 0 is equivalent to the existence of a constant parameter θ 0 such that ∀x J , θ(x J ) = θ 0 . Test statistics can in this case be constructed as || θ(•) -θ0 ||, using weighted versions of the CMLE estimators defined in Equation 1.5 and a norm || • || on the space of functions R d-p → Θ.

In both cases, asymptotic distributions of these tests statistics are complicated to obtain and may not pivotal, in the sense that they may depend on unknown parameters. Therefore, we propose to use the bootstrap to estimate quantiles of the asymptotic distribution of a given test statistic T . We propose different bootstrap schemes in general nonparametric framework, as well as under parametric assumptions.

The simplifying assumption that we have studied before concerns the constancy of conditional copulas with respect to pointwise conditioning events, i.e. conditioning events of the form X J = x J , for some fixed value of x J . One can wonder what would happen for different conditioning events, for example if we would conditioned by an event of the form x J ∈ A J for a borelian subset A J ⊂ R d-p . Similarly as in Theorem 1.1 and Equation 1.6, we have the decomposition

F I|J (X I ≤ x I |X J ∈ A J ) = C A J I|J F 1|J (x 1 |X J ∈ A J ), . . . , F p|J (x p |X J ∈ A J ) | X J ∈ A J ,
for every point x I ∈ R p and every subset A J ∈ A J , where A J is the set of all Borel subsets of R d-p .

This defines implicitly a conditional copula C A J I|J that depends of the conditioning set A J in the general case. So, it is tempting to replace H 0 by H 0 : C A J I|J (u I |X J ∈ A J ) does not depend on A J ∈ A J , for any u I .

Surprisingly and counter-intuitively, C A J I|J still depends on A J even under the simplifying assumption H 0 ! This is due to the non-linear transformation in Sklar's theorem. In fact, we prove that H0 is equivalent to a test of independence between X I and X J , which is much stronger than the simplifying assumption H 0 . Indeed, H 0 means that X J has no influence on the conditional dependence between the components of X I ; if X I and X J were independent, X J would have no influence on the conditional distribution of X J , meaning no influence on its conditional margins nor on its conditional dependence.

Nevertheless, one can weaken the latter assumption, and restrict oneself to a finite family A J of subsets with positive probabilities. For such a family, we could test the assumption

H 0 : A J → C A J I|J ( • |X J ∈ A J ) is constant over A J .
We propose similar test statistics of H 0 in both non-parametric and parametric frameworks, as well as adapted bootstrap procedures. We prove the validity of a particular bootstrap resampling schemes.

Finally, we illustrate the relevance of all these test statistics and related methods on simulated data.

Kendall's tau: a measure of dependence, and its conditional version

Since the copula is a cdf, it lives in an infinite-dimensional space and can be hard to represent, store (in the memory of a computer) or interpret in applications. Therefore, it may be useful to model the dependence by a number, rather than by a function. One can invoke the usual (Pearson's) coefficient of correlation, but it is not always defined. More precisely, the correlation coefficient does not exists when one of the marginal distribution does not belongs to L 2 , for example, if it is a Cauchy distribution.

Moreover, the correlation coefficient is no invariant with respect to increasing transformations of the margins, such as a logarithmic transformation.

Several margin-free measures of dependence have been proposed, one of the best-known among them is Kendall's tau [START_REF] Kendall | A new measure of rank correlation[END_REF]. For a bivariate random vector X = (X 1 , X 2 ), it is defined as

τ 1,2 := IP (X 2,1 -X 1,1 )(X 2,2 -X 1,2 ) > 0 -IP (X 2,1 -X 1,1 )(X 2,2 -X 1,2 ) < 0 , (1.8)
where X 1 := (X 1,1 , X 1,2 ) and X 2 := (X 2,1 , X 2,2 ) are two i.i.d. replications of X. This can be interpreted as the probability of observing a concordant pair ("the two variables move in the same direction") minus the probability of observing a discordant pair ("the two variables move in opposite direction"). Note that Kendall's tau is always defined for any distribution on R 2 without any moment assumption and lies in the interval [-1, 1]. It is invariant by increasing transformations of the marginal distribution and therefore only depends on the copula of X. The link between the Kendall's tau of a given distribution and its copula is in fact explicit and given by τ 1,2 = 4 [0,1] 2 C(u, v)dC(u, v) -1. Further properties of Kendall's tau and related dependence measures are detailed in [START_REF] Nelsen | An introduction to copulas[END_REF].

Kendall's tau can be estimated easily by the empirical proportion of concordant pairs minus the empirical proportion of discordant pairs, giving an estimator τ . For most bivariate families of copulas C = {C θ , θ ∈ Θ ⊂ R}, there exists a bijection Ψ between the Kendall's tau and the parameter θ, such that τ = Ψ(θ). Then a natural estimator for θ is given by the technique called "Inversion of Kendall's tau", that is θ := Ψ (-1) (τ ), where Ψ (-1) denotes the inverse of Ψ.

In a bivariate framework, the inference procedure for the law of X can be therefore divided in four independent steps:

1. Estimation of the first marginal distribution F 1 ;

2. Estimation of the second marginal distribution F 2 ;

3. Choice / selection of the family of copula C, which determines the shape of the dependence ;

4. Estimation of the Kendall's tau τ and the corresponding parameter θ, which determines the strength of the dependence.

By using Sklar's theorem, we can construct an estimator of the cdf of X as FX (x) := C Ψ (-1) τ F1 (x 1 ) , F2|Z (x 2 ) , where x = (x 1 , x 2 ) ∈ R 2 .

Assume now that we observe n > 0 i.i.d replications (X 1 , Z 1 ), . . . , (X n , Z n ) of a vector (X, Z) where X is a bivariate random vector and Z is a vector of explanatory variables. The conditional law of X given Z can be decomposed using the conditional version of Sklar's Theorem, as in Equation 1.6, using the conditional marginals F 1|Z and F 2|Z and the conditional copula C X|Z . Assume that the two conditional margins F 1|Z and F 2|Z have already been estimated parametrically or non-parametrically (for example, using kernel smoothing techniques). If the simplifying assumption holds, the conditional copula C X|Z is constant with respect to Z and usual (unconditional) copula models can be used to estimate it.

On the contrary, if the simplifying assumption does not hold, then the statistician has to estimate the conditional copula C X|Z . This corresponds to the estimation of a model of the thick arrow in Figure 1.4.

It is always possible to use nonparametric estimators of a conditional copula, for example by kernelsmoothing, but they are very hard to visualize -and a fortiori to interpret. Indeed, even when the dimension of Z is one, the graph of a conditional copula is an hyper-surface in dimension 4.

A simple idea is the use of the conditional Kendall's tau. Remember that Kendall's tau for a copula C is τ (C) = 4 CdC -1. We can therefore extend this definition in a straightforward way as

τ 1,2|Z=z := τ (C X|Z=z ) = 4 C X|Z=z dC X|Z=z -1 = IP (X 2,1 -X 1,1 )(X 2,2 -X 1,2 ) > 0 Z 1 = Z 2 = z -IP (X 2,1 -X 1,1 )(X 2,2 -X 1,2 ) < 0 Z 1 = Z 2 = z , (1.9)
where (X 1 , Z 1 ) = (X 1,1 , X 1,2 , Z 1,1 , . . . , Z 1,p ) and (X 2 , Z 2 ) = (X 2,1 , X 2,2 , Z 2,1 , . . . , Z 2,p ) are two independent versions of (X, Z). This can be interpreted as the probability of observing a concordant pair conditionally to Z = z minus the probability of observing a discordant pair conditionally to Z = z. We note that the conditioning event Z 1 = Z 2 = z in Equation 1.9 is unusual, but it is necessary. Indeed, conditional probabilities of concordance/discordance are only relevant when the two observations follow the same distribution. In this case, it corresponds to the constraint

Z 1 = Z 2 .
In a parametric framework, one can use the conditional Kendall's tau to estimate the conditional parameter of a conditional copula. Indeed, let C 1,2|Z (•|Z = z) be the conditional copula between X 1 and X 2 given Z = z, C := {C θ , θ ∈ Θ} be a family of copulas and assume that for every z, there exists a conditional parameter θ(z) ∈ Θ such that C 1,2|Z (•|Z = z) = C θ(z) . Assume that there is a bijection Ψ such that τ (C θ ) = Ψ(θ), where τ (C θ ) denotes the Kendall's tau of the copula C θ . Then if we are given an estimator τ1,2|Z=z of the conditional Kendall's tau between X 1 and X 2 given Z = z, we can estimate the conditional copula between X 1 and X 2 given Z = z by

Ĉ1,2|Z ( • | Z = z) := C Ψ (-1) τ1,2|Z=z ,
where Ψ (-1) denotes the inverse function of Ψ. This allows one to separate the inference procedure for the joint law of (X, Z) in five independent steps:

1. Estimation of the first conditional marginal distribution F 1|Z which determines the influence of the covariate Z on X 1 ;

2. Estimation of the first conditional marginal distribution F 2|Z which determines the influence of the covariate Z on X 2 ;

3. Choice / selection of the family of conditional copula C, which determines the shape of the conditional dependence ;

4. Estimation of the conditional Kendall's tau τ 1,2|Z=• and the corresponding parameter θ(z), which determines the strength of the conditional dependence ;

5. Estimation of the law of the conditioning variable Z.

As a consequence, an estimator of the joint cdf F X,Z can be constructed using these five pieces, by FX,Z (x, z) := FZ (z) × C Ψ (-1) τ1,2|Z=z F1|Z (x 1 |Z = z) , F2|Z (x 2 |Z = z) , where x = (x 1 , x 2 ) ∈ R 2 and z ∈ R p .

Estimation of the conditional Kendall's tau

To estimate the conditional Kendall's tau, we will follow three different approaches. In Chapter 5, we show how kernel-based estimators can be used to estimate the conditional Kendall's tau. In Chapter 6, we propose to estimate the conditional Kendall's tau by a regression-type model. Finally, in Chapter 7, we show how classification-based methods can be used for the estimation of the conditional Kendall's tau.

The conditional Kendall's tau can be rewritten as τ 1,2|Z=z = IE[g(X 1 , X 2 )|Z 1 = Z 2 = z] for a wellchosen choice of g and two i.i.d. replications (X 1 , Z 1 ), (X 2 , Z 2 ) of a random vector (X, Z) ∈ R 2+p , where dim(Z) = p > 0. Therefore, in Chapter 5, we propose a corresponding kernel-based estimator of the conditional Kendall's tau. It is given by τ1,2|Z=z := n i=1 n j=1 w i,n (z)w j,n (z)g(X i , X j ), where w i,n (z) = K h (Z i -z), K is a kernel on R p and h > 0 is the so-called bandwidth.

Our first result is a concentration inequality for the estimated conditional Kendall's tau: we show that for a given z, t > 0 small enough and t > 0, with probability larger than 1 -exp -nh p t 2 /(C 1 + C 2 t) -exp -nh 2p t 2 /(C 3 + C 4 t ) , we have τ1,2|Z=z -τ 1,2|Z=z < φ(t, t , h), for some explicit constants C 1 , . . . , C 4 and an explicit function φ, under some regularity conditions. Then, we show its consistency, uniform consistency and asymptotic normality under different regularity assumptions on the data-generating process. This kernel-based can be nevertheless costly to compute when we want to estimate the whole curve z → τ 1,2|Z=z . We may also want to have a parametric form for the conditional Kendall's tau that would be easier to interpret in applications. Therefore, in Chapter 6, we propose the following regressiontype model. For a given link function Λ : [-1, 1] → R, and a given basis (ψ i ), we can always do the decomposition

Λ τ 1,2|Z=z = i ψ i (z)β * i = ψ(z) T β * , (1.10)
where the β * i are real coefficients, assuming that only a finite number of functions is necessary to reconstruct this transformation of the conditional Kendall's tau. It seems difficult to have intuition about what kind of functions should be incorporated in the family ψ i . As a consequence, we advise to include different transformations, such as polynomials, trigonometric or indicator functions, keeping in mind that only a few of them might be relevant to the analysis. This means that the vector β * is sparse, in the sense that most of its coefficients may be equal to 0. As in the previous section, we would like to use a penalized regression to estimate Model 1.10. A direct estimation of it seems difficult since we do not observe realizations of the conditional Kendall's tau itself. Therefore, we will use a two-step based procedure :

1. Fixing some design points z 1 , . . . , z n with n > 0, we estimate the conditional Kendall's tau using a kernel-based estimator at each of those points, giving estimators τ1,2|Z=z i for i = 1, . . . , n .

2. Then we estimate β * by the following penalized least-squares estimator βP LS := arg min

β∈R p 1 n n i=1 Λ(τ 1,2|Z=z i ) -ψ(z i ) T β 2 + λ|β| 1 , (1.11)
where | • | 1 is the l 1 norm and λ is a tuning parameter.

Finally, for any point z, the conditional Kendall's tau itself can be estimated by Λ -1 z T βP LS ). As a byproduct, the coefficient βP LS can also be used to estimate marginal effects of the form ∂τ 1,2|Z=z /∂z j for j = 1, . . . , p. This can be useful to have a quantity that measures how the conditional dependence between X 1 and X 2 change with the variable Z j , for example.

We prove a concentration bound for βP LS with explicit constants that holds with high probability under regularity assumptions on the data-generating process and on the design matrix. Then we prove the consistency and the asymptotic normality of βP LS , when n → ∞ and n is fixed. In this situation, our estimator βP LS does not fulfill the oracle property in the sense that it fails to estimate the true set of relevant variables with probability tending to 1.

We show that a related adaptive procedure can recover the true set of relevant variable. It is defined with the same l 1 -penalized criteria, but it uses a random tuning parameter λ = µ/| β| δ , where β is a consistent estimator of β * and µ = µ n,n is a deterministic sequence satisfying some rate assumption.

Getting back to the previous estimator β, we show that, under some regularity assumptions, it is consistent in the framework where both n and n tend to the infinity. We also compute its asymptotic distribution, and show that the rate significantly improved in this double-asymptotic framework: we obtain βP LS n,n -β * (nn h p ) -1/2 , that can be compared with the equivalent βP LS n,n -β * (nh p ) -1/2 obtained when n is fixed.

In Chapter 7, we show how the problem of estimating the conditional Kendall's tau can be rewritten as a classification task. Indeed, remember that it is defined as

τ 1,2|Z=z = IP (X 2,1 -X 1,1 )(X 2,2 -X 1,2 ) > 0 Z 1 = Z 2 = z -IP (X 2,1 -X 1,1 )(X 2,2 -X 1,2 ) < 0 Z 1 = Z 2 = z .
We now introduce the variable W defined by

W = 1 (X 2,1 -X 1,1 )(X 2,2 -X 1,2 ) > 0 -1 (X 2,1 -X 1,1 )(X 2,2 -X 1,2 ) < 0 .
As the distribution of X is continuous, W belongs to the set {-1, 1} almost surely, where W = 1 corresponds to the observation of a concordant pair and W = -1 to a discordant pair. Using this new variable W , we can rewrite the conditional Kendall's tau as

τ 1,2|Z=z = IP W = 1 Z 1 = Z 2 = z -IP W = -1 Z 1 = Z 2 = z = 2IP W = 1 Z 1 = Z 2 = z -1 = 2p(z) -1, (1.12)
where p(z) := IP W = 1 Z 1 = Z 2 = z . Therefore, estimating the conditional probability of the variable W being equal to 1 or -1 is equivalent to estimating the conditional Kendall's tau. Actually, the prediction of concordance/discordance among pairs of observations (X 1 , X 2 ) given Z can be seen as a classification task of such pairs. If a model is able to evaluate the conditional probability of observing concordant pairs of observations, then it is able to evaluate conditional Kendall's tau, and the former quantity is one of the outputs of most classification techniques.

For an i.i.d. sample D := (X i , Z i ) i=1,...,n , we define W (i,j) as

W (i,j) := 2 × 1{(X j,1 -X i,1 )(X j,2 -X i,2 ) > 0} -1 =    1 if (i, j) is a concordant pair, -1 if (i, j) is a discordant pair, (1.13)
for every 1 ≤ i, j ≤ n, i = j. A classification technique will allocate a given couple (i, j) into one of the two categories {1, -1} (or "concordant versus discordant", equivalently), with a certain probability, given the value of the common covariate Z.

We first consider a parametric approach, assuming a single-index model of the form τ 1,2|Z=z = g ψ(z) T β * , similarly as in Model 1.10, where g = Λ (-1) . We propose to use maximum-likelihood-type methods to estimate the parameter β * . For one observation (W (i,j) , Z i , Z j ), given Z i = Z j = z, its log-likelihood is

β (W (i,j) , z) := 1 + W (i,j) 2 log IP β W (i,j) = 1 Z i = Z j = z + 1 -W (i,j) 2 log IP β W (i,j) = -1 Z i = Z j = z .
In practice, when the underlying law of Z is continuous, there is virtually no couple for which Z i = Z j .

Therefore, we will consider a localized "approximated" log-likelihood, based on (W (i,j) , Z i , Z j ) for all pairs (i, j), i = j. It will be defined as the double sum

L n (β) := 1 n(n -1) i,j;i =j K h (Z i -Z j ) β (W (i,j) , Zi,j ),
where K is a kernel on R p , h a bandwidth, and Zi,j a point belonging to a neighborhood of Z i or Z j , for example Z i or Z j themselves, or (Z i + Z j )/2. Finally, we define a penalized approximate maximum likelihood estimator of β * by βP AM L := arg max

β∈R p L n (β) -λ n |β| 1 . (1.14)
Note that, contrary to the two-step estimators considered previously in Equation (1.11), βP AM L is not a solution of a convex program in general, and therefore may difficult to compute in practice. Nevertheless, it does not need any choice of design points as previously. If g is chosen as the equivalent of the probit or logit link function, then we prove that the optimization program (1.14) is in fact a convex program, that can be easily solved in polynomial time using a classical software for solving (penalized) weighted probit or logit regressions.

More generally, if we are given a dataset D := (X i , Z i ) i=1,...,n , we can always construct the dataset of pairs, defined as D := (W k , Zk , V k ) 1≤k≤n(n-1)/2 , where each pair (i, j) with i < j is indexed by an integer k. Each observation in this dataset is made up of three components. The first is W k := W i,j , i.e. the concordance/discordance of the pair (i, j) ; this is the variable that we want to predict. The explanatory variable of the pair (i, j) is defined by Zk := Zi,j . Finally we construct a weight variable

V k := V i,j = K h (Z i -Z j )
, which measures how close Z i and Z j are, and therefore, it measures how relevant for the estimation of the conditional Kendall's tau the pair (i, j) is. Indeed, remember that in the definition of the conditional Kendall's tau τ 1,2|Z=z , the conditioning event is of the form Z 1 = Z 2 = z.

Assume we are given a classification algorithm that takes in input a dataset with an explained variable, a vector of explanatory variables and a weight for each observation. We remark that it corresponds exactly to the caracteristic of the dataset of pairs D constructed above. Therefore, we can directly apply any classification algorithm on the dataset D. As a result we will obtained an estimated function p(z) of the probability of observing a concordant pair given the covariate z. We can plug this estimate to obtain an estimator of the conditional Kendall's tau by τ1,2|Z=z = 2p(z) -1, using Equation (1.10).

Nearly all classification algorithms can be adapted following this principled, and we detail the case of the probit logit, decision trees, random forest, nearest neighbors and neural networks. One can remark that observations in the dataset of pairs D are not independent. For example, the pair (1, 2) and (1, 3) are not independent since they share the same first observation (X 1 , Z 1 ). Nevertheless, we show that this lack of independence is not too much harmful since most couple of pairs are independent in fact. The performance of all these estimators and their sensitivities to each component of the model is assessed in a simulation study. Finally, we apply all these estimators to a dataset of European stock indices, and compare the conditional dependence between them.

Other topics in inference 1.3.1 Estimation of a regular conditional functional by conditional U-statistic regression

Remember that Kendall's tau, defined in Equation (1.8), can be also rewritten as

τ 1,2 := 1 (x 2,1 -x 1,1 )(x 2,2 -x 1,2 ) > 0 -1 (x 2,1 -x 1,1 )(x 2,2 -x 1,2 ) < 0 dIP X (x 1,1 , x 2,1 )dIP X (x 1,1 , x 2,1 ).
This means Kendall's tau is what we will call a regular function θ(IP X ) of the law of X. Similarly, the conditional Kendall's tau τ 1,2|Z=z can be rewritten as θ(IP X|Z=z ) with the same conditional θ(•). Many results detailed in Chapters 5 and 6 are not specific to the case of the (conditional) Kendall's tau, but can be generalized to any regular (conditional) functional. Such generalizations are studied in Chapter 8, with corresponding theoretical results.

Our framework will be the following. We observe n i.i.d. replications

(X i , Z i ) ∼ (X, Z), i = 1, . . . , n.
The random variable X belongs to a measurable space (X , A) while Z ∈ R p . We will denote the joint law of (X, Z) by IP X,Z and the conditional law of X|Z by IP X|Z . A regular conditional functional is defined as a functional of the form

θ(z 1 , . . . , z k ) = θ IP X|Z=z1 , . . . , IP X|Z=z k = g * (x 1 , . . . , x k )dIP X|Z=z1 (x 1 ) • • • dIP X|Z=z k (x k ),
for a fixed function g * : X k → R, where k > 0. For example, in the case of the conditional Kendall's tau,

X = R 2 , k = 2, and g * (x 1 , x 2 ) is 1 if the pair (x 1 , x 2 ) is concordant and -1 if it is discordant.
Our goal is to estimate the function (z 1 , . . . z k ) → θ(z 1 , . . . , z k ). The first method consists in using a kernel estimator θ of θ. Asymptotic properties of such an estimator were proved by Stute [START_REF] Stute | Conditional U-statistics[END_REF]. Under some regularity assumptions, we prove that, with probability greater than 1 -

2 exp -[n/k]t 2 h kp /(C 1 + C 2 t) -2 exp -[n/k]t 2 h kp /(C 6 + C 7 t ) , we have θ(z 1 , . . . , z k ) -θ(z 1 , . . . , z k ) ≤ 1 + C 3 h α + C 4 t × C 5 h k+α + t , for some explicit constants C 1 , . . . , C 7 .
Then we propose a regression-type model generalizing the parametric model that we proposed for the conditional Kendall's tau in Equation (1.10). It is defined as

∀(z 1 , . . . , z k ) ∈ Z k , Λ θ(z 1 , . . . , z k ) = ψ(z 1 , . . . , z k ) T β * , (1.15)
for a given transformation Λ : Range(g * ) → R, a basis ψ of size r > 0, and a true unknown parameter β * to be estimated. This model cannot be directly estimated, as both sides of the equation are unobserved.

Nevertheless, given a finite collection of points z 1 , . . . , z n ∈ Z n and a collection I k,n of injective functions σ : {1, . . . , k} → {1, . . . , n }, we can use the kernel estimate θ(z σ(1) , . . . , z σ(k) ) for each choice of σ ∈ I k,n . Then, the estimator β can be defined in a second step as the minimizer of the following

l 1 -penalized criteria β(g * ) := arg min β∈R r   (n -k)! n ! σ∈I k,n Λ θ z σ(1) , . . . , z σ(k) -ψ z σ(1) , . . . , z σ(k) T β 2 + λ|β| 1   ,
where λ is a tuning parameter. We prove a nonasymptotic bound on β(g * ) that holds with high probability.

Then we consider two different asymptotic regimes : n → ∞ with a fixed n and (n, n ) jointly tends to the infinity. In both regimes, we show that, under suitable regularity conditions, the estimator β(g * ) is consistent and we derive its asymptotic law.

About confidence intervals for ratios of means

In Section 1.3.1, we have given non-asymptotic bounds for our kernel estimator, which is defined as a ratio of two sums. In fact, the same techniques can be used to obtain concentration inequalities for ratios of means. These inequalities are of the form,

X n /Y n -IE[X]/IE[Y ]| ≤ Ψ(α)
, with probability at least 1 -α, where Ψ is a function that depends on the regularity assumptions made on the distribution of X and Y , X n := n -1 n i=1 X i,n , and we observe n i.i.d. replications

(X i,n , Y i,n ) ∼ (X, Y ).
In Chapter 9, we prove two versions of these concentrations inequalities, when X and Y both admits second moments, or when the supports of X and Y are bounded. We give explicit expressions for the corresponding functions Ψ, which are of the form

(C + o(1))/ √ nα (respectively (C + o(1)) ln(1/α)/n).
This allows us to construct confidence intervals for ratios of means, that are valid for any fixed n ∈ N .

The most popular method for constructing confidence intervals is based on an application of the delta method. In practice, statisticians tend to examine thinner and thinner effects as more and more data are available. This forms a sequence-of-models framework under which we prove a generalization of the delta method for ratios of means. Finally, we prove a lower bound and an impossibility result related to the nonasymptotic inference of ratios of means. More precisely, we show that there exists a minimum size for any nonasymptotic confidence interval and a minimum level under which no "reasonable test" can be defined.

Introduction

In a recent paper by Bellec, Lecué and Tsybakov [START_REF] Bellec | Slope meets lasso: improved oracle bounds and optimality[END_REF], it is shown that there exist high-dimensional statistical methods realizable in polynomial time that achieve the minimax optimal rate (s/n) log (p/s) in the context of sparse linear regression. Here, n is the sample size, p is the dimension and s is the sparsity parameter. The result is achieved by the Lasso and Slope estimators, and the Slope estimator is adaptive to the unknown sparsity s. Bounds for more general estimators are proved by Bellec, Lecué

and Tsybakov [START_REF] Bellec | Bounds on the prediction error of penalized least squares estimators with convex penalty[END_REF][START_REF] Bellec | Towards the study of least squares estimators with convex penalty[END_REF]. These articles also establish bounds in deviation that hold for any confidence level and for the risk in expectation. However, the estimators considered in [START_REF] Bellec | Slope meets lasso: improved oracle bounds and optimality[END_REF][START_REF] Bellec | Bounds on the prediction error of penalized least squares estimators with convex penalty[END_REF][START_REF] Bellec | Towards the study of least squares estimators with convex penalty[END_REF] require the knowledge of the noise variance σ 2 . To our knowledge, no polynomial-time methods, which would be at the same time optimal in a minimax sense and adaptive both to σ and s are available in the literature.

Estimators similar to the Lasso, but adaptive to σ are the Square-Root Lasso and the related Scaled Lasso, introduced by Sun and Zhang [START_REF] Sun | Scaled sparse linear regression[END_REF] and Belloni, Chernozhukov and Wang [START_REF] Belloni | Square-root lasso: pivotal recovery of sparse signals via conic programming[END_REF]. It has been shown to achieve the rate (s/n) log(p) in deviation with the value of the tuning parameter depending on the confidence level. A variant of this estimator is the Heteroscedastic Square-Root Lasso, which is studied in more general nonparametric and semiparametric setups by Belloni, Chernozhukov and Wang [START_REF] Belloni | Pivotal estimation via square-root lasso in nonparametric regression[END_REF], but it also achieves the rate (s/n) log(p) and depends on the confidence level. We refer to the book by Giraud [START_REF] Giraud | Introduction to high-dimensional statistics[END_REF] for the link between the Lasso and the Square-Root Lasso and a short proof of oracle inequalities for the Square-root Lasso. In summary, there are two points to improve for the Square-root Lasso method:

(i) The available results on oracle inequalities are valid only for the estimators depending on the confidence level. Thus, one cannot have an oracle inequality for one given estimator at any confidence level except the one that was used to design it.

(ii) The obtained rate is (s/n) log(p) which is greater than the minimax rate (s/n) log(p/s).

The Slope, which is an acronym for Sorted L-One Penalized Estimation, is an estimator introduced by Bogdan et al. [START_REF] Bogdan | Slope -adaptive variable selection via convex optimization[END_REF], that is close to the Lasso, but uses the sorted l 1 norm instead of the standard l 1 norm for penalization. Su and Candès [START_REF] Su | Slope is adaptive to unknown sparsity and asymptotically minimax[END_REF] proved that, as opposed to the Lasso, the Slope estimator is asymptotically minimax, in the sense that it attains the rate (s/n) log(p/s) for two isotropic designs, that is either for X deterministic with 1 n X T X = I p×p or when X is a matrix with i.i.d. standard normal entries. Moreover, their result has not only the optimal minimax rate, but also the exact optimal constant. General isotropic random designs are explored by Lecué and Mendelson [START_REF] Lecué | Regularization and the small-ball method i: sparse recovery[END_REF]. For non-isotropic random designs and deterministic designs under conditions close to the Restricted Eigenvalue, the behavior of the Slope estimator is studied in [START_REF] Bellec | Slope meets lasso: improved oracle bounds and optimality[END_REF]. The Slope estimator is adaptive only to s, and requires knowledge of σ, which is not available in practice. In order to have an estimator which is adaptive both to s and σ, we will use the Square-Root Slope, introduced by Stucky and van de Geer [START_REF] Stucky | Sharp oracle inequalities for square root regularization[END_REF]. They give oracle inequalities for a large group of square-root estimators, including the new Square-Root Slope, but still following the scheme where (i) and (ii) cannot be avoided. The square-root estimators are also members of a more general family of penalized estimators defined by Owen [109, equations ( 8)-( 9)] ; using their notation, these estimators correspond to the case where H M is the squared loss and B M is a norm (either the l 1 norm or the slope norm).

The paper is organized as follows. In Section 2.2, we provide the main definitions and notation. In Section 2.3, we show that the Square-Root Lasso is minimax optimal if s is known while being adaptive to σ under a mild condition on the design matrix (SRE). In Section 2.4, we show that any sequence of estimators can be made adaptive to the sparsity parameter s, while keeping the same rate up to some constant, with a computational cost increased by a factor of log(s * ) where s * is an upper bound on the sparsity parameter s. As an application, the Square-root Lasso modified by this procedure is still optimal while being now adaptive to s (in addition of being already adaptive to σ). In Section 2.5, we show how to adapt any algorithm for computing the Slope estimator to the case of the Square-root Slope estimator.

In Section 2.6, we study the Square-Root Slope estimator, and show that it is minimax optimal and adaptive both to s and σ, under a slightly stronger condition (WRE). The (SRE) and (WRE) conditions have already been studied by Bellec, Lecué and Tsybakov [START_REF] Bellec | Slope meets lasso: improved oracle bounds and optimality[END_REF] and hold with high probability for a large class of random matrices. Moreover, the inequalities we obtain for each estimator are valid for a wide range of confidence levels. Proofs are given in Section 2.7.

The framework

We use the notation | • | q for the l q norm, with 1 ≤ q ≤ ∞, and | • | 0 for the number of non-zero coordinates of a given vector. For any v ∈ R p , and any set of coordinates J, we denote by v J the vector (v j 1{i ∈ J}) i=1,...,p , where 1 is the indicator function. We also define the empirical norm of

a vector u = (u 1 , . . . , u n ) as ||u|| 2 n := 1 n n i=1 u 2 i .
For a vector v ∈ R p , we denote by v (j) the j-th largest component of v. As a particular case, |v| (j) is the j-th largest component of the vector |v| whose components are the absolute values of the components of v. We use the notation •, • for the inner product with respect to the Euclidean norm and (e j ) j=1,...,p for the canonical basis in R p .

Let Y ∈ R n be the vector of observations and let X ∈ R n×p be the design matrix. We assume that the true model is the following

Y = Xβ * + ε. (2.1)
Here β * ∈ R p is the unknown true parameter. We assume that ε is the random noise, with values in R n , distributed as N (0, σ 2 I n×n ), where I n×n is the identity matrix. We denote by IP β * the probability distribution of Y satisfying (2.1). In what follows, we define the set B 0 (s

) := {β * ∈ R p : |β * | 0 ≤ s}.
In the high-dimensional framework, we have typically in mind the case where s is small, p is large and possibly p n.

We define two square-root type estimators of β * : the Square-Root Lasso βSQL and the Square-Root Slope βSQS by the following relations βSQL ∈ arg min

β∈R p 1 √ n |Y -Xβ| 2 + λ|β| 1 , (2.2)
βSQS ∈ arg min

β∈R p 1 √ n |Y -Xβ| 2 + |β| * , (2.3)
where λ > 0 is a tuning parameter to be chosen, and the sorted

l 1 norm, | • | * , is defined for all u ∈ R p by |u| * = p j=1 λ j |u| (j) , with tuning parameters λ 1 ≥ • • • ≥ λ p > 0.

Optimal rates for the Square-Root Lasso

In this section, we derive oracle inequalities with optimal rate for the Square-Root Lasso estimator.

We will use the Strong Restricted Eigenvalue (SRE) condition, introduced in [START_REF] Bellec | Slope meets lasso: improved oracle bounds and optimality[END_REF]. For c 0 > 0 and s ∈ {1, . . . , p}, it is defined as follows, SRE(s, c 0 ) condition : The design matrix X satisfies max j=1,...,p

||Xe j || n ≤ 1 and κ(s) := min δ∈C SRE (s,c0):δ =0 ||Xδ|| n |δ| 2 > 0, (2.4)
where

C SRE (s, c 0 ) := {δ ∈ R p : |δ| 1 ≤ (1 + c 0 ) √ s|δ| 2 } is a cone in R p .
The condition max j=1,...,p ||Xe j || n ≤ 1 is standard and corresponds to a normalization. It is shown in [START_REF] Bellec | Slope meets lasso: improved oracle bounds and optimality[END_REF]Proposition 8.1] that the SRE condition is equivalent to the Restricted Eigenvalue (RE) condition of [START_REF] Bickel | Simultaneous analysis of Lasso and Dantzig selector[END_REF] if that is considered in conjunction with such a normalization. By the same proposition, the RE condition is also equivalent to the s-sparse eigenvalue condition, which is satisfied with high probability for a large class of random matrices. It is the case, if for instance, n ≥ Cs log(ep/s) and the rows of X satisfies the small ball condition, which is very mild, see, e.g. [START_REF] Bellec | Slope meets lasso: improved oracle bounds and optimality[END_REF].

Note that the minimum in (2.4) is the same as the minimum of the function δ → ||Xδ|| n on the set

C SRE (s, c 0 ) ∩ {δ ∈ R p : |δ| 2 = 1}
, which is a continuous function on a compact of R p , therefore this minimum is attained. When there is no ambiguity over the choice of s, we will just write κ instead of κ(s). Then, for every δ 0 ≥ exp(-n/4γ 2 ) and every 24 , we have

β * ∈ R p such that |β * | 0 ≤ s, with IP β * -probability at least 1 -δ 0 -(1 + e 2 )e -n/
||X( βSQL -β * )|| n ≤ σ max C 1 κ 2 s n log p s , C 2 log(1/δ 0 ) n , (2.7)
| βSQL -β * | q ≤ σ max   C 3 κ 2 s 1/q 1 n log 2p s , C 4 s 1/q-1 log 2 (1/δ 0 ) n log(2p/s)   , (2.8)
where 1 ≤ q ≤ 2, and

C 1 > 0, C 2 > 0, C 3 > 0, C 4 >
||X( βSQL -β * )|| n ≤ C 2 κ 2 σ s n log p s , | βSQL -β * | q ≤ C 4 κ 2 σs 1/q 1 n log 2p s ,
where 1 ≤ q ≤ 2.

Theorem 2.1 and Corollary 2.2 give bounds that hold with high probability for both the prediction error and the estimation error in the l q norm, for every q in [1,2]. Note that the bounds are best when the tuning parameter is chosen as small as possible, i.e. with γ = 16 + 4 √ 2. As shown in Section 7 of Bellec, Lecué and Tsybakov [START_REF] Bellec | Slope meets lasso: improved oracle bounds and optimality[END_REF], the rates of estimation obtained in the latter corollary are optimal in a minimax sense on the set B 0 (s) := {β * ∈ R p : |β * | 0 ≤ s}. We obtain the same rate of convergence as [START_REF] Bellec | Slope meets lasso: improved oracle bounds and optimality[END_REF] (see the paragraph after Corollary 4.3 in [START_REF] Bellec | Slope meets lasso: improved oracle bounds and optimality[END_REF]) up to some multiplicative constant.

The rate is also the same as in Su and Candès [START_REF] Su | Slope is adaptive to unknown sparsity and asymptotically minimax[END_REF], but the framework is quite different: we obtain a non-asymptotic bound in probability whereas they consider asymptotic bounds in expectation (cf. Theorem 1.1 in [START_REF] Su | Slope is adaptive to unknown sparsity and asymptotically minimax[END_REF]) and in probability (Theorem 1.2) but without giving an explicit expression of the probability that their bound is valid. Our result is non-asymptotic and valid when general enough conditions on X are satisfied whereas the result in [START_REF] Su | Slope is adaptive to unknown sparsity and asymptotically minimax[END_REF] is asymptotic as n → ∞, and valid for two isotropic designs, that is either for X deterministic with 1 n X T X = I p×p or when X is a matrix with i.i.d. standard normal entries.

Similarly to [START_REF] Bellec | Slope meets lasso: improved oracle bounds and optimality[END_REF], for each tuning parameter γ, there is a wide range of levels of confidence δ 0 under which the bounds of Theorem 2.1 are valid. However, [START_REF] Bellec | Slope meets lasso: improved oracle bounds and optimality[END_REF] allows for an arbitrary small confidence level while in our case, there is a lower bound on the size of the confidence level under which the rate is obtained. Note that this bound can be made arbitrary small by choosing a sample size n large enough.

Note that the possible values chosen for the tuning parameter λ are independent of the underlying standard deviation σ, which is unknown in practice. This gives an advantage for the Square-Root Lasso over other methods such as the ordinary Lasso. Nevertheless, this estimator is not adaptive to the sparsity s, so that we need to know that |β * | 0 ≤ s in order to be able to apply this result. In the following section, we suggest a procedure to make the Square-root Lasso adaptive to s while keeping its optimality and adaptivity to σ.

Adaptation to sparsity by a Lepski-type procedure

Let s * be an integer in {2, . . . , p/e}. We want to show that the Square-Root Lasso can also achieve the minimax optimal bound, adaptively to the sparsity s on the interval [1, s * ] (in addition of being already adaptive to σ). Following [START_REF] Bellec | Slope meets lasso: improved oracle bounds and optimality[END_REF], we will use aggregation of at most log 2 (s * ) Square-Root Lasso estimators with different tuning parameters to construct an adaptive estimator β of β and at the same time an estimator s of the sparsity s.

In the following, we use the notation

κ * := κ(2s * ).
Note that κ * = min s=1,...,2s * κ(s). Indeed, the function κ(•) is decreasing, because the minimization (2.4) is done on spaces that are growing with s, in the sense of the inclusion. We will assume that the condition SRE(2s * , 5/3) holds and that (2s * /n) log 2p/(2s * ) ≤ 9κ

IP β * ||X( βSQL (s,γ) -β * )|| n ≤ C 2 (γ) κ 2 * σ s n log p s ≥ 1 - s p s -(1 + e 2 )e -n/
IP β * ||X( βSQL (2s,γ) -β * )|| n ≤ C 2 (γ) κ 2 * σ 2s n log p 2s ≥ 1 - 2s p 2s -(1 + e 2 )e -n/24 . (2.10) Remark that λ (s,γ) = γ 1 n log 2p s = γ 1 n log 2p s -log(2) n = λ (2s,γ) for some γ > γ.
IP β * ||X( βSQL (s,γ) -β * )|| n ≤ C 2 (γ) κ 2 * σ 2s n log p 2s ≥ 1 - 2s p 2s -(1 + e 2 )e -n/24 .
(2.11)

Note that equations (2.9) and (2.11) are the same as equations (5.2) and (5.4) in Bellec, Lecué and Tsybakov [START_REF] Bellec | Slope meets lasso: improved oracle bounds and optimality[END_REF], taking

C 0 := max C 2 (γ), C 2 (γ) /κ
IP β * |X( βSQL (s,γ) -β * )| q ≤ C 4 (γ) κ 2 * σs 1/q s n log 2p s ≥ 1 - s p s -(1 + e 2 )e -n/24 , (2.12
)

sup β * ∈B0(2s) IP β * | βSQL (s,γ) -β * | q ≤ C 4 (γ) κ 2 * σs 1/q 2s n log 2p 2s ≥ 1 - 2s p 2s -(1 + e 2 )e -n/24 . (2.13)
We describe now an algorithm to compute this adaptive estimator. The idea is to use an estimator s of s which can be written as s := 2 m for some positive data-dependent integer m. We will use the notation M := max{m ∈ N : 2 m ≤ s * }, so that the number of estimators we consider in the aggregation is M .

The suggested procedure is detailed in Algorithm 2 below, with the distance d(β, β

) = ||X(β -β )|| n or d(β, β ) = |β -β | q for q ∈ [1, 2].
It can be used for any family of estimators ( β(s) ) s=1,...s * , and chooses the best one in terms of the distance d(•, •), resulting in an aggregated estimator β. Note that the weight function w(•) used in the algorithm cannot depend on σ as in [START_REF] Bellec | Slope meets lasso: improved oracle bounds and optimality[END_REF], i.e. to have the form

w(b) = C 0 σ (b/n) log(p/b) (respectively w(b) = C 0 σb 1/q (1/n) log(p/b)
), because we are looking for a procedure adaptive to σ. Therefore, we will remove σ from w and use an estimate σ.

Algorithm 2: Algorithm for adaptivity. Then, there exists a constant C 5 , depending on C 0 , C , C , C 2 , κ and α such that, for all β * ∈ B 0 (s), the aggregated estimator β satisfies:

Input: a distance d(•, •) on R p Input: a function w(•) : [1, s * ] → R + satisfying Assumption 2.4.1 Input: a family of estimators β(s) s=1,...,s * M ← log 2 (s * ) ; for m ← 1 to M + 1 do compute the estimator β(2 m ) ; end compute σ ← ||Y -X β(2 M +1 ) || n ; compute the set S 1 ← m ∈ {1, . . . , M } : d β(2 k-1 ) , β(2 k ) ≤ 4σC 0 w(2 k ), for all k ≥ m ; if S 1 = ∅ then m ← min S 1 else m ← M ; Output: s ← 2 m Output: β ← β(s)
IP β * σ/2 ≤ σ ≤ ασ ≤ u n,p,M , with a constant α > 0, σ := ||Y -X β(2
IP β * d( β(s) , β * ) ≤ C 0 σw(s) ≥ 1 - s p s -u n , (2.14
IP β * d( β, β * ) ≤ C 5 • σw(s) ≥ 1 -3(log 2 (s * ) + 1) 2 2s p 2s + u n -u n,p,M .
Furthermore,

IP β * s ≤ s ≥ 1 -2(log 2 (s * ) + 1) 2 2s p 2s + u n -u n,p,M .
This theorem is proved in Section 2.7.3.1. In particular, it implies that when β(s) = βSQL (s,γ) , the aggregated estimator β has the same rate on B 0 (s) as the estimators with known s. We detail it below.

w(b) = (b/n) log(p/b) and w(b) = b 1/q (1/n) log(2p/b), for q ∈ [1, 2].
IP β * ||X( β -β * )|| n ≤ C 5 • σ s n log p s ≥ 1 -3(log 2 (s * ) + 1) 2 2s p 2s + u n -u n,p,M , and
IP β * s ≤ s ≥ 1 -2(log 2 (s * ) + 1) 2 2s p 2s + u n -u n,p,M ,
where u n = (1 + e 2 )e -n/24 , u n,p,M = (2 M +1 /p)

(b) = b 1/q (1/n) log(2p/b), for q ∈ [1; 2],
we have that, for all β * ∈ B 0 (s), the aggregated estimator β satisfies

IP β * | β -β * | q ≤ C 5 • σs 1/q 1 n log p s ≥ 1 -3(log 2 (s * ) + 1) 2 2s p 2s + u n -u n,p,M , and
IP β * s ≤ s ≥ 1 -2(log 2 (s * ) + 1) 2 2s p 2s + u n -u n,p,M ,
where 24 , and C 5 is a constant depending only on γ and κ * .

u n = (1 + e 2 )e -n/24 , u n,p,M = (2 M +1 /p) 2 M +1 -(1 + e 2 )e -n/
Thus, we have shown that the suggested aggregated procedure based on the Square-root Lasso is adaptive to s while still being adaptive to σ and minimax optimal. Note that the computational cost is multiplied by O(log(s * )).

Algorithms for computing the Square-root Slope

In this part, our goal is to provide algorithms for computing the square-root Slope estimator. A natural idea is revisiting the algorithms used for the square-root Lasso and for the Slope, then adapting or combining them.

Belloni, Chernozhukov and Wang [START_REF] Belloni | Square-root lasso: pivotal recovery of sparse signals via conic programming[END_REF]Section 4] have proposed to compute the Square-root Lasso estimator by reducing its definition to an equivalent problem, which can be solved by interior-point or first-order methods. The equivalent formulation as the Scaled Lasso, introduced by Sun and Zhang [START_REF] Sun | Scaled sparse linear regression[END_REF] allows one to view it as a joint minimization in (β, σ). Sun and Zhang [START_REF] Sun | Scaled sparse linear regression[END_REF] propose an iterative algorithm which alternates estimation of β using the ordinary Lasso and estimation of σ.

Zeng and Figueiredo [START_REF] Zeng | The ordered weighted 1 norm: Atomic formulation, projections, and algorithms[END_REF] studied several algorithms related to estimation of the regression with the ordered weighted l 1 -norm, which is the Slope penalization. Bogdan et al. [START_REF] Bogdan | Slope -adaptive variable selection via convex optimization[END_REF] provide an algorithm for computing the Slope estimator using a proximal gradient.

As in the case of the Square-root Lasso, we still have for any β,

||Y -Xβ|| n = min σ>0 σ + ||Y -Xβ|| 2 n σ , (2.16)
where the minimum is attained for σ = ||Y -Xβ|| n . As a consequence, βSQS ∈ arg min

Optimal rates for the Square-Root Slope

In this part, we will use another condition, the Weighted Restricted Eigenvalue condition, introduced in [START_REF] Bellec | Slope meets lasso: improved oracle bounds and optimality[END_REF]. For c 0 > 0 and s ∈ {1, . . . , p}, it is defined as follows, W RE(s, c 0 ) condition : The design matrix X satisfies max j=1,...,p

||Xe j || n ≤ 1 and κ := min δ∈C W RE (s,c0):δ =0 ||Xδ|| n |δ| 2 > 0, (2.17)
where

C W RE (s, c 0 ) :=    δ ∈ R p : |δ| * ≤ (1 + c 0 )|δ| 2 s j=1 λ 2 j    is a cone in R p .
To obtain the following result, we assume that the Weighted Restricted Eigenvalue condition holds.

This condition is shown to be only slightly more constraining than the usual Restricted Eigenvalue condition of [START_REF] Bickel | Simultaneous analysis of Lasso and Dantzig selector[END_REF], but is nevertheless satisfied with high probability for a large class of random matrices, see Bellec, Lecué and Tsybakov [START_REF] Bellec | Slope meets lasso: improved oracle bounds and optimality[END_REF] for a discussion. Note that, in a similar way as in definition (2.4), the minimum is attained. Indeed, κ is equal to the minimum of the function δ → ||Xδ|| n on the set

C W RE (s, c 0 ) ∩ {δ ∈ R p : |δ| 2 = 1},
λ j = γ log(2p/j) n , for j = 1, . . . , p, (2.18)
and assume that

γ ≥ 16 + 4 √ 2 and s n log 2ep s ≤ κ 2 256γ 2 .
(2.19)

Then, for every δ 0 ≥ exp(-n/4γ 2 ) and every β * ∈ R p such that |β * | 0 ≤ s, with IP β * -probability at least 24 , we have

1 -δ 0 -(1 + e 2 )e -n/
||X( βSQS -β * )|| n ≤ σ max C 1 κ s n log p s , C 2 log(1/δ 0 ) n , (2.20)
| βSQS -β * | * ≤ σ max C 1 κ 2 s n log p s , C 2 log(1/δ 0 ) n , (2.21)
| βSQS -β * | 2 ≤ σ max C 1 κ 2 s n log p s , C 2 log 2 (1/δ 0 ) sn log(p/s) , (2.22)
for constants C 1 > 0 and C 2 > 0 depending only on γ .

The values of the constants C 1 and C 2 can be found in the proof, in Subsection 2.7.4. Note that the bounds are best when the tuning parameters is chosen as small as possible, i.e. using the choice

γ = 16 + 4 √ 2.
Using the fact that κ ≤ 1 and choosing δ 0 = (s/p) s , we get the following corollary.

Corollary 2.9. Under the assumptions of Theorem 2.8, with IP β * -probability at least 1 -(s/p) s -(1 + e 2 )e -n/24 , we have

||X( βSQS -β * )|| n ≤ C 1 κ σ s n log p s , | βSQS -β * | * ≤ C 1 κ 2 σ s n log p s , | βSQS -β * | 2 ≤ C 1 κ 2 σ s n log p s ,
These results show that the Square-Root Slope estimator, with a given choice of parameters, attains the optimal rate of convergence in the prediction norm || • || n and in the estimation norm | • | 2 . We also provide a bound on the sorted l 1 norm | • | * of the estimation error. One can note that the choice of λ i that allows us to obtain optimal bounds does not depend on the level of confidence δ 0 , but only influence the size of the range of valid δ 0 . This improves upon the oracle result of Stucky and van de Geer [START_REF] Stucky | Sharp oracle inequalities for square root regularization[END_REF], in which the parameter does depend on the level of confidence and the rate does not scale in the optimal way, i.e., as (s/n) log(p/s). Moreover, we can see that our estimator is independent of the underlying standard deviation σ and of the sparsity s, even if the rates depend on them. Note that, up to some multiplicative constant, we obtain the same rates as for the Slope in Bellec, Lecué and Tsybakov [START_REF] Bellec | Slope meets lasso: improved oracle bounds and optimality[END_REF].

In Su and Candès [START_REF] Su | Slope is adaptive to unknown sparsity and asymptotically minimax[END_REF], the Slope estimator is proved to attain the sharp constant in the asymptotic framework where σ is known and for specific X ; whereas here we obtain only the minimax rates, but in a non-asymptotic framework, and under general assumptions on the design matrix X.

For this estimator, we did not provide a bound for the l 1 norm, for the same reasons as in [START_REF] Bellec | Slope meets lasso: improved oracle bounds and optimality[END_REF].

Indeed, the coefficients λ j of the components of β are different in the sorted norm. As a consequence, we do not provide inequalities for l q norms when q < 2, that are obtained by interpolation between the l 1 and l 2 norms.

Proofs

Preliminary lemmas

Let β * ∈ R p , S ⊂ {1, . . . , p} with cardinality s and denote by S C the complement of S. For i ∈ {1, . . . , p}, let β * i be the i-th component of β * and assume that for every i ∈ S C , β * i = 0.

Lemma 2.10. We have

( βSQL -β * ) S C 1 ≤ ( βSQL -β * ) S 1 + 1 λ √ n|ε| 2 X T ε , βSQL -β * .
The proof follows from the arguments in Giraud [67, pages 110-111], and it is therefore omitted.

Lemma 2.11. Let u ∈ R p be defined by u := βSQS -β * . We have

p j=s+1 λ j |u| (j) ≤ s j=1 λ j |u| (j) + 1 √ n|ε| 2 X T ε , u .
Proof : We combine the arguments from Giraud [67, pages 110-111], and from the proof of Lemma A.1 in [START_REF] Bellec | Slope meets lasso: improved oracle bounds and optimality[END_REF]. First, we remark that the sorted l 1 norm can be written as follows, for any v ∈ R p ,

|v| * = max φ p j=1 λ j v φ(j) ,
where the maximum is taken over all permutations φ = (φ(1), . . . , φ(p)) of {1, . . . , p}.

By definition, βSQS is a minimizer of (2.3), so we have

|Y -X βSQS | 2 -|Y -Xβ * | 2 ≤ √ n |β * | * -| βSQS | * .
Let φ be any permutation of {1, . . . , p} such that

|β * | * = s j=1 λ j |β * φ(j) | and |u φ(s+1) | ≥ |u φ(s+2) | ≥ • • • ≥ |u φ(p) |. (2.23)
We have

|β * | * -| βSQS | * ≤ s j=1 λ j β * φ(j) -βSQS φ(j) - p j=s+1 λ j βSQS φ(j) ≤ s j=1 λ j u φ(j) - p j=s+1 λ j βSQS φ(j) = s j=1 λ j u φ(j) - p j=s+1 λ j u φ(j) .
Since the sequence λ j is non-increasing, we have

s j=1 λ j |u φ(j) | ≤ s j=1 λ j |u| (j)
. The permutation φ satisfies (2.23), therefore,

p j=s+1 λ j |u| (j) ≤ p j=s+1 λ j |u φ(j) |. From the previous inequalities, we get that |Y -X βSQS | 2 -|Y -Xβ * | 2 ≤ √ n   s j=1 λ j |u| (j) - p j=s+1 λ j |u| (j)   . (2.24)
By convexity of the mapping β → ||Y -Xβ|| 2 , we have

|Y -X βSQS | 2 -|Y -Xβ * | 2 ≥ - X T ε |ε| 2 , βSQS -β * = - 1 |ε| 2 X T ε , βSQS -β * . (2.25)
Combining (2.24) and (2.25), we get

- 1 |ε| 2 X T ε , βSQS -β * ≤ √ n   s j=1 λ j |u| (j) - p j=s+1 λ j |u| (j)   ,
which concludes the proof.

Lemma 2.12. We have

|X( βSQL -β * )| 2 2 ≤ X T ε , βSQL -β * + λ √ n|Y -X βSQL | 2 | βSQL -β * | 1 .
Lemma 2.13. We have

|X( βSQS -β * )| 2 2 ≤ X T ε , βSQS -β * + √ n|Y -X βSQS | 2 | βSQS -β * | * .
Proof : We will give a general proof of Lemmas 2.12 and 2.13 in the case of an estimator defined by

β := arg min β∈R p 1 √ n |Y -Xβ| 2 + ||β|| , (2.26)
where || • || is a norm on R p . Lemmas 2.12 and 2.13 are obtained as special cases corresponding to

|| • || = λ| • | 1 and || • || = | • | * . Denote by || • || dual the norm dual to || • ||.
Since β is optimal, we know that X T (Y -X β)/( √ n|Y -X β| 2 ) belongs to the subdifferential of the function || • || evaluated at β. Thus, there exists v ∈ R p such that ||v|| dual ≤ 1 and

X T (Y -X β) √ n|Y -X β| 2 + v = 0.
Thus, we have

|X( β -β * )| 2 2 = X T ε , β -β * + √ n|Y -X β * | 2 v , β -β * .
The conclusion results from the inequality

v , β -β * ≤ ||v|| dual || β -β * || ≤ || β -β * ||.
Lemma 2.14. We have

γ (s/n) log(2p/s) ≤ s j=1 λ 2 j ≤ γ (s/n) log(2ep/s).
Proof : From Stirling's formula, we deduce that s log(s/e) ≤ log(s!) ≤ s log(s). Therefore

s log(2p/s) ≤ s j=1 log(2p/j) = log(2p) -log(s!) ≤ s log(2ep/s).
The conclusion follows from the definition of the λ j in (2.18).

The following simple property is proved in Giraud [67, page 112]. For convenience, it is stated here as a lemma. For any u = (u 1 , . . . u p ) in R p , we define :

G(u) := (4 + √ 2)σ log(1/δ 0 ) n ||Xu|| n , H(u) := (4 + √ 2) p j=1 |u| (j) σ log(2p/j) n , F (u) := (4 + √ 2)σ log(2p/s) n   √ s|u| 2 + p j=s+1 |u| (j)   .
If ε ∼ N (0, σ 2 I n×n ), then the random event

1 n ε T Xu ≤ max H(u), G(u) , ∀u ∈ R p , is of probability at least 1 -δ 0 /2.
Moreover, by the Cauchy-Schwarz inequality, we have H(u) ≤ F (u), for all u in R p .

Proof of Theorem 2.1

Lemma 2.16 allows one to control the random variable ε T Xu that appears in Lemmas 2.10 and 2.12 with u := βSQL -β * . Our calculations will take place on an event of probability at least 1 -δ 0 -(1 + e 2 )e -n/24 , where both Lemmas 2.15 and 2.16 can be used. Applying Lemma 2.16, we will distinguish between the two cases : G(u) ≤ F (u) and F (u) < G(u).

First case : G(u) ≤ F (u).

Then we have

(4 + √ 2) log(1/δ 0 ) n ||Xu|| n ≤ (4 + √ 2) log(2p/s) n   √ s|u| 2 + p j=s+1 |u| (j)   .
We will show first that u is in the SRE cone, so that we can use the SRE assumption. From Lemma 2.10, we have

|u S C | 1 ≤ |u S | 1 + 1 λ √ n|ε| 2 X T ε , βSQL -β * ≤ |u S | 1 + 1 √ nλ|ε| 2 nσ(4 + √ 2) log(2p/s) n   √ s|u| 2 + p j=s+1 |u| (j)   ≤ |u S | 1 + 1 4 √ s|u| 2 + |u S C | 1 ,
where in the last inequality, we have used Lemma 2.15 and assumption (2.6). We deduce that

3 4 |u| 1 ≤ 7 4 |u S | 1 + 1 4 √ s|u| 2 ≤ 7 4 √ s|u| 2 + 1 4 √ s|u| 2 = 2 √ s|u| 2 .
Therefore, we have

|u| 1 ≤ 8 3 √ s|u| 2 , (2.27)
and thus, the following inequality holds

|u| 1 ≤ (1 + c 0 ) √ s|u| 2
, with c 0 = 5/3, allowing us to use the SRE(s, 5/3) assumption.

From Lemmas 2.12 and 2.16, and using that, in view of the SRE(s, 5/3) condition, ||Xu|| n ≥ κ|u| 2 , we deduce that

||Xu|| 2 n ≤ (4 + √ 2)σ log(2p/s) n   √ s|u| 2 + p j=s+1 |u| (j)   + |ε| 2 √ n + ||Xu|| n 8 3 λ √ s|u| 2 ≤ (4 + √ 2) 11 3 σ s log(2p/s) n ||Xu|| n κ + (2σ + ||Xu|| n ) 8 3 λ √ s ||Xu|| n κ .
Thus,

||Xu|| n ≤ (4 + √ 2) 11 3 σ s log(2p/s) n 1 κ + (2σ + ||Xu|| n ) 8 3 λ √ s 1 κ .
Under assumptions (2.5) and (2.6), we have

8λ √ s 3κ = 8γ 3κ s n log 2p s ≤ 1 2 .
Thus, we have

||Xu|| n ≤ 2 44 + 11 √ 2 3κ σ s n log 2p s + 16σλ √ s 3κ ≤ 88 + 22 √ 2 + 32γ 3κ σ s n log 2p s .
(2.28)

We have proved in (2.27) that

|u| 1 ≤ (1 + c 0 ) √ s|u| 2 , with c 0 = 5/3, so we get that |u| 2 ≤ ||Xu|| n /κ.
Therefore, we can deduce the following inequalities

|u| 2 ≤ 88 + 22 √ 2 + 32γ 3κ 2 σ s n log 2p s , (2.29
)

|u| 1 ≤ 704 + 176 √ 2 + 256γ 9κ 2 σs 1 n log 2p s .
(2.30)

Second case : F (u) ≤ G(u).
Then we have

(4 + √ 2) log(2p/s) n   √ s|u| 2 + p j=s+1 |u| (j)   ≤ (4 + √ 2) log(1/δ 0 ) n ||Xu|| n .
Thus

|u| 1 ≤ √ s|u| 2 + p j=s+1 |u| (j) ≤ log(1/δ 0 ) log(2p/s) ||Xu|| n .
From Lemmas 2.12 and 2.16, we find

||Xu|| 2 n ≤ (4 + √ 2)σ log(1/δ 0 ) n ||Xu|| n + λ |ε| 2 √ n + ||Xu|| n |u| 1 ≤ (4 + √ 2)σ log(1/δ 0 ) n ||Xu|| n + λ (2σ + ||Xu|| n ) log(1/δ 0 ) log(2p/s) ||Xu|| n .
Thus,

||Xu|| n ≤ (4 + √ 2)σ log(1/δ 0 ) n + λ (2σ + ||Xu|| n ) log(1/δ 0 ) log(2p/s) .
We have chosen λ = γ 1 n log 2p s , therefore we have

||Xu|| n ≤ σ log(1/δ 0 ) n (4 + √ 2 + 2γ) + ||Xu|| n γ log(1/δ 0 ) n .
By assumption, exp(-n/4γ 2 ) ≤ δ 0 , thus we have

||Xu|| n ≤ σ log(1/δ 0 ) n (8 + 2 √ 2 + 4γ). (2.31)
As a consequence, we have

|u| 1 ≤ log(1/δ 0 ) log(2p/s) ||Xu|| n ≤ σ log 2 (1/δ 0 ) n log(2p/s) (8 + 2 √ 2 + 4γ). (2.32)
We have also

√ s|u| 2 ≤ log(1/δ0) log(2p/s) ||Xu|| n , thus |u| 2 ≤ σ log 2 (1/δ 0 ) sn log(2p/s) (8 + 2 √ 2 + 4γ). (2.33)
As a conclusion, we can prove the result (2.7) by combining the inequalities (2.28) and (2.31). The general bound for |u| q , with 1 ≤ q ≤ 2 is a consequence of the norm interpolation inequality

|u| q ≤ |u| 2/q-1 1 |u| 2-2/q 2
which proves (2.8).

Proofs of the adaptive procedure

Proof of Theorem 2.3

We choose s ∈ [1, s * ] and assume that β * ∈ B 0 (s). Define IP := IP β * and m 0 := log 2 (s) + 1.

For any a > 0, we have

IP d( β, β * ) ≥ a ≤ IP d( β, β * ) ≥ a, m ≤ m 0 + IP( m ≥ m 0 + 1). (2.34)
On the event { m ≤ m 0 }, we have the decomposition

d( β, β * ) ≤ m0 k= m+1 d β(2 k-1 ) , β(2 k ) + d β(2 m 0 ) , β * . (2.35)
Using Assumption 2.4.1, we get that,

m0 k= m+1 d β(2 k-1 ) , β(2 k ) ≤ m0 k= m+1 4σC 0 w(2 k ) (2.36) ≤ 4σC 0 C w(2 m0 ) ≤ 4σC 0 C C w(s). (2.37)
We have 2 m0 ≤ 2s, therefore applying Assumption (2.15), we have with IP β * -probability at least 1 -

(2s/p) 2s -u n , d( β(2 m 0 ) , β * ) ≤ C 2 (γ) κ 2 σw(2s) ≤ C 2 (γ)C κ 2 σw(s). ( 2
1 -(2s/p) 2s -u n -u n,p,M , d( β, β * ) ≤ 4σC 0 C C α + C 2 (γ)C κ 2 σw(s). (2.39)
We now bound the probability IP( m ≥ m 0 + 1).

IP( m ≥ m 0 + 1) ≤ M m=m0+1 IP( m = m 0 + 1) ≤ M m=m0+1 M k=m IP d β(2 k-1 ) , β(2 k ) > 4σC 0 w(2 k ) ≤ M m=m0+1 M k=m IP d β(2 k-1 ) , β * > 2σC 0 w(2 k ) + IP d β(2 k ) , β * > 2σC 0 w(2 k ) ≤ 2 M m=m0+1 M k=m-1 IP d β(2 k-1 ) , β * > 2σC 0 w(2 k ) ≤ 2 M m=m0+1 M k=m-1 IP d β(2 k-1 ) , β * > 2σC 0 w(2 k ), σ ≥ σ 2 + IP σ < σ 2 .
Combining the previous equation with Assumption 2.4.2, and then with Assumption (2.15), we get

IP( m ≥ m 0 + 1) ≤ 2 M m=m0+1 M k=m-1 IP d β(2 k-1 ) , β * > σC 0 w(2 k ) -u n,p,M ≤ 2M 2 2s p 2s + u n -u n,p,M ≤ 2(log 2 (s * ) + 1) 2 2s p 2s + u n -u n,p,M .
As a consequence, we deduce the bound on s. Combining the last equation with equations (2.34) and

(2.39), we finally get that

IP d( β, β * ) ≥ 4σC 0 C C α + C 2 (γ)C κ 2 σw(s) ≤ 3(log 2 (s * ) + 1) 2 2s p 2s + u n -2u n,p,M .

Proof of Lemma 2.4

Now, we consider the general case of the function w(b) = b 1/q (1/n) log(ap/b), with q a fixed number of the interval [1,2]. The first case will correspond to a = 1 and q = 2 and the second case will correspond to a = 2 with any choice of q.

We want to that the first part of Assumption 2.4.1 is satisfied, i.e., w is increasing on the interval

[1, s * ]. Let b ∈ [1, s * ]. We have w (b) = 1 q b (1/q)-1 1 n log ap b + b (1/q) -1 nb 2 1 n log ap b = b (1/q)-1 n -1/2 (2/q) log ap b -1 2 log ap b ,
which is positive when (2/q) log ap b -1 ≥ 0, that is, when b ≤ ape -q/2 . We have b ≤ s * ≤ p/e = ape -q/2 when a = 1 and q = 2. When a = 2 and q ∈ [1, 2], p/e ≤ 2pe -1 ≤ ape -q/2 . In the two cases we consider, we have proved that w (•) ≥ 0 on the interval [1, s * ], thus the function w is increasing on this interval. This proves that the first part of Assumption 2.4.1 is satisfied.

Let m be an integer in the interval [1, M ].

m k=1 w(2 k ) = m k=1 2 k/q 1 n log ap 2 k = m-1 k=0 2 (m-k)/q 1 n log ap 2 m-k = 2 m/q √ n m-1 k=0 1 2 k/q log ap 2 m + k log(2) ≤ 2 m/q √ n m-1 k=0 1 2 k/q log ap 2 m + m-1 k=0 √ k 2 k/q log(2) ≤ 2 m/q √ n log ap 2 m 1 1 -2 -1/q + m-1 k=0 4 2 k/2q log(2) ≤ 2 m/q 1 n log ap 2 m 1 1 -2 -1/q + 4 log(2) 1 -2 -1/(2q) ,
which proves that the second part is satisfied.

Let b be an integer of [1, s * ]. We have w(2b) = (2b) 1/q (1/n) log(2p/(2b)) ≤ 2 1/q w(b), which proves that the third part is satisfied.

Proof of Lemma 2.5

We have β * ∈ B 0 (s) ⊂ B 0 (2 M +1 ), therefore, we can apply Corollary 2.2 and Lemma 2.15, we have with

IP β * -probability at least 1 -(2 M +1 /p) 2 M +1 -(1 + e 2 )e -n/24 , σ ≤ ||ε|| n + X( β(2 M +1 ) -β * ) n ≤ 2σ + C 2 (γ) κ 2 * σ 2 M +1 n log p 2 M +1 ≤ σ 2 + C 2 (γ) κ 2 * 2s n log 2p s ≤ σ 2 + 3 √ 2C 2 (γ) 16κ * γ , and σ ≥ ||ε|| n -X( β(2 M +1 ) -β * ) n ≥ σ √ 2 - C 2 (γ) κ 2 * σ 2 M +1 n log p 2 M +1 ≥ σ 1 √ 2 - √ 2C 2 (γ) κ 2 * s n log 2p s ≥ σ 1 √ 2 - √ 2C 2 (γ) κ 2 * 2s * n log p s * ≥ σ   1 √ 2 - 1 √ 2 - 1 2 2   ≥ σ 2 .

Proof of Theorem 2.8

We act as in Section 2.7.2, with suitable modifications. We place ourselves in the event where both Lemmas 2.15 and 2.16 are valid, and set now u := βSQS -β * . Applying Lemma 2.16, we will distinguish between the two cases :

G(u) ≤ H(u) + σ|u| 2 s j=1 λ 2 j and H(u) + σ|u| 2 s j=1 λ 2 j < G(u). First case : G(u) ≤ H(u) + σ|u| 2 s j=1 λ 2 j .
Applying Lemma 2.11, Lemma 2.16 and then Lemma 2.15, we have

|u| * = p j=1 λ j |u| (j) ≤ 2 s j=1 λ j |u| (j) + 1 √ n|ε| 2 X T ε , βSQS -β * ≤ 2 s j=1 λ 2 j |u| 2 + n √ n|ε| 2   (4 + √ 2) σ γ |u| * + σ|u| 2 s j=1 λ 2 j   ≤ 4 s j=1 λ 2 j |u| 2 + 8 + 2 √ 2 γ |u| * ,
and we get

|u| * ≤ 4|u| 2 1 - 8 + 2 √ 2 γ s j=1 λ 2 j ,
Using assumption (2.

||Xu|| 2 n ≤ 1 n X T ε , u + 1 √ n |Y -X β| 2 |u| * ≤ (4 + √ 2) σ γ |u| * + σ|u| 2 s j=1 λ 2 j + (2σ + ||Xu|| n )|u| * ≤ (32 + 8 √ 2) σ γ + 17σ + 8||Xu|| n |u| 2 s j=1 λ 2 j ≤ (32 + 8 √ 2) σ γ + 17σ + 8||Xu|| n ||Xu|| n κ γ (s/n) log(2ep/s).
Thus, Second case :

||Xu|| n ≤ σ κ s n log 2ep s 32 + 8 √ 2 + 17γ
H(u) + σ|u| 2 s j=1 λ 2 j ≤ G(u).
Then we have

(4 + √ 2) σ γ |u| * + σ|u| 2 s j=1 λ 2 j ≤ (4 + √ 2)σ log(1/δ 0 ) n ||Xu|| n .
Therefore we have

|u| * ≤ γ log(1/δ 0 ) n ||Xu|| n , and |u| 2 s j=1 λ 2 j ≤ (4 + √ 2) log(1/δ 0 ) n ||Xu|| n . (2.43)
Invoking Lemmas 2.13 and 2.16, and using (2.43), we get

||Xu|| 2 n ≤ (4 + √ 2)σ log(1/δ 0 ) n ||Xu|| n + σ|u| 2 s j=1 λ 2 j + (2σ + ||Xu|| n )|u| * ≤ (4 + √ 2)σ log(1/δ 0 ) n ||Xu|| n + σ(4 + √ 2) log(1/δ 0 ) n ||Xu|| n + (2σ + ||Xu|| n )γ log(1/δ 0 ) n ||Xu|| n .
which yields

||Xu|| n ≤ (8 + 2 √ 2 + 2γ )σ log(1/δ 0 ) n + ||Xu|| n γ log(1/δ 0 ) n ,
We have chosen exp(-n/4γ 2 ) ≤ δ 0 , which implies that

||Xu|| n ≤ (16 + 4 √ 2 + 4γ )σ log(1/δ 0 ) n . (2.44)
We can deduce from (2.43) that

|u| * ≤ (16 + 4 √ 2 + 4γ )σγ log(1/δ 0 ) n , (2.45)
and combining the second part of (2.43) with Lemma 2.14, we get

|u| 2 γ s n log p s ≤ (4 + √ 2) log(1/δ 0 ) n ||Xu|| n ≤ (4 + √ 2)(16 + 4 √ 2 + 4γ )σ log(1/δ 0 ) n .
Finally, we get that

|u| 2 ≤ (4 + √ 2)(16 + 4 √ 2 + 4γ ) γ σ log 2 (1/δ 0 ) sn log(p/s) .
(2.46)

Introduction

One of the most simple problem in statistics is the estimation of an univariate mean. Of course, it is always possible to use the empirical mean, but it may not be an ideal choice. Indeed, the empirical mean is not robust, meaning that only one outlier in the dataset can push the mean towards infinity.

One of the procedures to guarantee optimality even in the presence of outliers is the Median-of-Means (MOM) framework [START_REF] Devroye | Sub-gaussian mean estimators[END_REF]. Its principle is very simple: divide the sample in K blocks, compute the empirical mean on each block, and return the median of these means. Note that this procedure is naturally robust to the presence of up to K/2 outliers. In the multivariate case, it is more difficult to construct robust and optimal estimators, see [START_REF] Lugosi | Sub-gaussian estimators of the mean of a random vector[END_REF][START_REF] Hopkins | Sub-gaussian mean estimation in polynomial time[END_REF].

Chapter 3. Robust-to-outliers simultaneous inference and noise level estimation using a MOM approach 56

Recently MOM procedures have received a great amount of attention for estimating conditional means, see [START_REF] Lecué | Learning from mom's principles: Le cam's approach[END_REF][START_REF] Lecué | Robust machine learning by median-of-means: theory and practice[END_REF][START_REF] Minsker | Uniform bounds for robust mean estimators[END_REF]. Furthermore, Median-of-Means (MOM) type approaches have been developed for a lot of frameworks such as classification [START_REF] Lecué | Robust classification via mom minimization[END_REF], empirical risk minimization [START_REF] Chinot | Statistical learning with lipschitz and convex loss functions[END_REF], sub-sampling and hyper-parameter tuning [START_REF] Kwon | Median of means principle as a divide-and-conquer procedure for robustness, sub-sampling and hyper-parameters tuning[END_REF], and reproducing kernel Hilbert space embedding [START_REF] Lerasle | Monk-outlier-robust mean embedding estimation by median-of-means[END_REF].

In the recent paper [START_REF] Lecué | Robust machine learning by median-of-means: theory and practice[END_REF], it was proved that minimaximization of a MOM-type criteria can be used to construct robust minimax optimal estimators in the linear high-dimensional framework. Nevertheless, these estimators require the knowledge of the standard deviation of the noise, which is unknown in practice. It is proposed in [START_REF] Lecué | Robust machine learning by median-of-means: theory and practice[END_REF] to use the square-root Lasso or the square-root Slope to estimate the noise level at a first step. If there is even one outlier in the data, this noise level estimator will not be robust which makes the whole procedure not robust as a consequence.

In Section 3.2, we propose a joint estimator for the parameter and the noise level in the high dimensional framework under sparsity. We propose an algorithm to compute it using penalized optimization techniques. We show that, under some conditions, this estimator can achieve optimal rates of estimation. This is proved by an application of Section 3.3 where we give a more general result of the estimation of a conditional mean in a given function class F under some technical assumptions. The rest of the chapter is devoted to the proofs of the main results. d+1) , where the observations are divided into the two following groups:

Results in the high-dimensional linear regression framework

Let X ∈ R d and Y ∈ R, with d ≥ 1 be two random variables such that Y = X T β * + ζ, , (3.1
(X i , Y i ) i=1,...,n ∈ R n×(
• the informative group I ⊂ {1, . . . , n}, where the pairs (X i , Y i ), i ∈ I are independent and follow the same distribution as (X, Y ) ;

• the outlier group O ⊂ {1, . . . , n} : for every i ∈ O, (X i , Y i ) are outliers, meaning that they may not follow Model (3.1) ; they may be deterministic and even adversarial in the sense that they may depend on the informative data (X i , Y i ) i∈I defined above and on the choice of the estimator.

Obviously, we have I ∪O = {1, . . . , n} and I ∩O = ∅ meaning that, for each pair (X i , Y i ), the corresponding index i ∈ {1, . . . , n} belongs to one of the two groups. Of course, it is unknown to the statistician whether i belongs to I or O. Otherwise, we would just remove the outlier group. To construct a robust estimator, we will use the Median-of-Means framework. Our estimator is defined as follows:

1. Given a dataset D = {(X i , Y i ), i = 1, . . . , n} of size n > 0, we can divide it into K blocks D 1 , . . . D K of size n/K (assumed to be an integer) corresponding to a partition {1, . . . , n}

= B 1 • • • B K ,
where K is a tuning parameter to be chosen later ;

Chapter 3. Robust-to-outliers simultaneous inference and noise level estimation using a MOM approach 57

2. For each k = 1, . . . , K, the criteria of (β, σ) against ( β, χ) on the block B k is defined by

P B k l β σ + σ - l β χ -χ := 1 |B k | i∈B k (Y i -X T i β) 2 σ + σ - (Y i -X T i β) 2 χ -χ , for every β, σ, β, χ ∈ R p × R + × R p × R + , where |B k | denotes the cardinal of B k ;
3. The global MOM criteria of (β, σ) against ( β, χ) is defined by

M OM K l β σ + σ - l β χ -χ := M edian P B k l β σ + σ - l β χ -χ , k = 1, . . . , K ;
4. Finally, the MOM-K estimator of (β * , σ * ) is defined by

( β, σ) := arg min β∈R d , σ>σmin max β∈R d , χ>σmin M OM K l β σ + σ - l β χ -χ + µ |β| 1 -| β| 1 , (3.2)
where µ is a tuning parameter to be chosen. As the minimization and maximization steps are similar, we detail only the former. We separate the joint minimization step in (β, σ) in two steps, one in β, and one in σ. When σ and ( β, χ) are fixed, the partial minimization in β of the criterion (3.2) is locally equivalent to the minimization of the program

min β i∈B k (Y i -X T i β) 2 + µσ|β| 1
. This is the classical Lasso program on the dataset D k , with a tuning parameter defined as λ := µσ. Therefore, any algorithm to compute the usual Lasso estimator (or even one step of the optimization program of the Lasso) may be used to update β. In Algorithm 4 and as an example, we choose to do a subgradient step. It is equal to

2X T k (Y k -X k β (t-1) ) -σ (t-1) sign(β (t-1) ),
where σ (t-1) is the estimated standard deviation at the previous step, sign is the component-wide sign, with the convention sign(0) = 0, Y k is the vector (Y i ) i∈B k and X k is the matrix whose lines are the vectors X T i , for i ∈ B k . Note that other choices of updates for β are possible, such as the use of proximal gradient descent, alternating direction method of multipliers (ADMM), and cyclic coordinate descent, as for non-adaptive minmax-MOM estimators (see respectively Algorithms 2, 3 and 4 in [START_REF] Lecué | Robust machine learning by median-of-means: theory and practice[END_REF]).

When β and ( β, χ) are fixed, the partial minimization in σ of the criterion (3.2) is locally equivalent to the minimization of the program min σ a/σ + σ, where a

= (1/|B k |) i∈B k (Y i -X T i β) 2 .
This optimization program admits a closed-form solution σ = √ a, which is the update step of σ that we will use. If we observe σ < σ min , then we update σ := σ min , but this should not occur too much often in practice.

Indeed, if a given σ is very small, it means that for one of the blocks, there exists a parameter β for which 2 . This would mean that at least for one of the blocks, the noise has a very small intensity, and should rather invite the applied statistician to set a lower σ min . There exists other possibilities of updating σ. For instance, if σ is too much unstable, then we could use instead the update step σ (t) ← (σ (t-1) + √ a)/2 or other autoregressive filters. This would slow the convergence of σ, but would introduce more stability.

a = (1/|B k |) i∈B k (Y i -X T i β)
Chapter 3. Robust-to-outliers simultaneous inference and noise level estimation using a MOM approach 58

The maximization in ( β, χ) follows exactly the same procedure as the minimization in (β, σ): we do a first update step on β and a second update step of the parameter σ. Following the conclusions of [START_REF] Lecué | Robust machine learning by median-of-means: theory and practice[END_REF],

we use a random choice of blocks at each step. Such a choice is useful in the sense that it should help to avoid saddle-points. Indeed, when the blocks are fixed, only the central block is used in the optimization step. This could mean that other blocks don't influence the optimization process. By randomizing the blocks, we are sure to change at each step which dataset is used. This allows to "explore" better the space of parameters, and improves strongly the performance of minmax-MOM algorithms.

Algorithm 4: An alternating sub-gradient algorithm to compute the adaptive minmax MOM-LASSO estimator of (β * , σ * ) using random blocks Input: a dataset D = (X i , Y i ) i=1,...,n ;

Input: tuning parameters µ, σ min > 0, K ∈ {1, . . . , n}, two step size sequences (η t ) t∈N and (η t ) t∈N

Input: initial point (β (0) , σ (0) , β(0) , χ (0) ) ∈ R d × R + × R d × R + ,
M OM K l β (t-1) σ (t-1) + σ (t-1) - l β(t-1)
χ (t-1) -χ (t-1) = P B k l β (t-1) σ (t-1) + σ (t-1) - l β(t-1)
χ (t-1) -χ (t-1) ; Update β (t) ← β (t-1) + 2η t X T k (Y k -X k β (t-1) ) -µσ (t-1) η t sign(β (t-1) ) ; Update σ (t) ← |Y k -X k β (t) | 2 /|B k | 1/2 ;
/* Update of β and χ */ Partition {1, . . . , n} into K blocks B 1 , . . . B K at random ; Find k ∈ {1, . . . , K} such that

M OM K l β (t) σ (t) + σ (t) - l β(t-1) χ (t-1) -χ (t-1) = P B k l β (t) σ (t) + σ (t) - l β(t-1) χ (t-1) -χ (t-1) ; Update β(t) ← β(t-1) + 2η t X T k (Y k -X k β(t-1) ) -µχ (t-1) ηt sign( β(t-1) ) ; Update χ (t) ← |Y k -X k β(t) | 2 /|B k | 1/2 ; until |β (t-1) -β (t) | 2 < 0 , | β(t-1) -β(t) | 2 < 0 , |σ (t-1) -σ (t) | < 0 and |χ (t-1) -χ (t) | < 0 ; Output: a joint estimator β, σ := β (t) , σ (t) ;
To prove statistical properties of ( β, σ), we will need the following set of assumptions.

X ) = |f (x)| p dP X (x)
1/p , P X denotes the law of the random vector X, and X T denotes the transpose of the random vector X.

Chapter 3. Robust-to-outliers simultaneous inference and noise level estimation using a MOM approach 59

The first assumption affirms that the number of informative data should be at least half of the sample size. Conversely, if more than half of the data have been corrupted, than there is little hope of recovering information about the informative data. Second, the outlier should not be too much numerous compared to the number of observations times the minimax optimal rates. This is similar to the assumption needed in [START_REF] Lecué | Robust machine learning by median-of-means: theory and practice[END_REF]. The second assumption is necessary to apply [START_REF] Mendelson | On multiplier processes under weak moment assumptions[END_REF]Theorem 1.6] to bound Gaussian mean widths (that are related to the Rademacher complexities that we will define in Section 3.3) by local Gaussian mean widths. These local Gaussian mean widths will be bounded by the minimax rates using [START_REF] Lecué | Regularization and the small-ball method i: sparse recovery[END_REF]Lemma 5.3]. These two steps will ensure that our estimator of β * attains the optimal minimax rates. The third assumption is a special case of small ball property, which is often needed to establish bounds in learning without using concentration inequalities, see [START_REF] Mendelson | Learning without concentration[END_REF][START_REF] Koltchinskii | Bounding the smallest singular value of a random matrix without concentration[END_REF]. Finally, the last inequality is satisfied when ζ and X are independent. We relax this hypothesis of independence, but still assume that the dependence between ζ and X is not too strong.

The following theorem is proved in Section 3.7.2, and show that the estimator ( β, σ) attains the minimax optimal rate adaptively in σ.

µ := c 1 1 n log ed s ,
we have

IP ∀p ∈ [1, 2], | β -β * | p ≤ c 2 ||ζ|| L 4 s 1/p 1 n log ed s and |σ -σ * | ≤ σ * (K/n) 1/2 (κ * -1) 1/2 /10 ≥ 1 -c 3 exp(-K/c 4 ),
for every integer K such that c 5 max |O|, ||ζ|| L 4 s log(ed/s) ≤ K ≤ c 6 n.

Note that these constants are fixed but might be difficult to compute in practice. To obtain precise values for these constants, for instance c 1 , one would need to quantify the constants c 2 in [103, Theorem 1.6] for q 0 = 4 and L = 2 and C in [93, Lemma 5.3], using their notation.

A general framework

In this section, we generalize the results of the previous section. We now assume that the explanatory variable X has values in a measurable space X , and we denote by F a class of measurable functions from X to R included in the space L 2 (P X ), where P X is the law of X. This means that for every f ∈ F , ||f (X)|| 2 L2 := f (x) 2 dP X (x) is supposed to be finite. The random variable Y is still with values in R, and we assume that

Y = f * (X) + ζ, (3.3)
for a given unknown function f * in the class F . The assumptions on ζ are the same as in the previous section: it has finite moments at least of the order 4, and an unknown square deviation σ * greater than a lower bound σ min > 0. This lower bound will be needed in the proofs, see Section 3.6.9.1.

Chapter 3. Robust-to-outliers simultaneous inference and noise level estimation using a MOM approach 60

Note that, if X = R d for a fixed d > 0 and F is the class of sparse linear functionals, i.e. if for a given

s ≤ d, we have F = {f β := (x ∈ R d → β T x ∈ R), β ∈ R d , |β| 0 ≤ s},
we recognize indeed the framework of the previous section.

Finally, our goal is to estimate (f * , σ * ) given a dataset of random pairs D = (X i , Y i ), i = 1, . . . , n,

where the observations are split in two groups : the informative group I, for which (X i , Y i ) are independent and distributed as (X, Y ) ; and the outlier group O, made up of potentially adversarial variables.

Our estimators is a generalization of (3.2), defined by :

1. Given a dataset D = {(X i , Y i ), i = 1, . . . , n} of size n > 0, we can divide it into K blocks D 1 , . . . D K of size n/K (assumed to be an integer) corresponding to a partition {1, . . . , n}

= B 1 • • • B K ;
2. For each k = 1, . . . , K, the criteria of (f, σ) against ( f , χ) on the block B k is defined by

P B k l f σ + σ - l f χ -χ := 1 |B k | i∈B k (Y i -f (X i )) 2 σ + σ - (Y i -f (X i )) 2 χ -χ , for every f, σ, f , χ ∈ F × R + × F × R + , where |B k | denotes the cardinal of B k ;
3. The global MOM criteria of (f, σ) against ( f , χ) is defined by

M OM K l f σ + σ - l f χ -χ := M edian P B k l f σ + σ - l f χ -χ , k = 1, . . . , K ;
4. Finally, the MOM-K estimator of (β * , σ * ) is defined by

( f , σ) := arg min f ∈F, σ>σmin max g∈F, χ>σmin M OM K l f σ + σ - l g χ -χ + µ(||f || -||g||) (3.4) = arg min f ∈F, σ>σmin max g∈F, χ>σmin T K,µ (g, χ, f, σ) = arg min f ∈F, σ>σmin C K,µ (f, σ),
where µ is a tuning parameter to be chosen, || • || is a norm on the space of functions generated by

F , T K,µ (g, χ, f, σ) := M OM K l f σ + σ - l g χ -χ + µ(||f || -||g||)
and C K,µ (f, σ) := max g∈F, χ>σmin T K,µ (g, χ, f, σ).

Note that this estimator ( f , σ) depends only on two tuning parameters K and µ. To prove its properties, we will need the following assumption, which is a generalization of Assumption 3.2.1.

Assumption 3.3.1.

There exists θ 0 , θ m such that, for all i ∈ I and f ∈ F

1. V ar ζ(f -f * )(X) ≤ θ 2 m ||f -f * || 2 L2 , 2. ||f -f * || L2 ≤ θ 0 ||f -f * || L1 .
The first part of this assumption specifies that the dependence between the noise ζ and transformations f -f * of X is not too strong compared to the norm of these transformations. It is satisfied as a special case when ζ and X are independent. The second assumption is related to the small ball assumption and controls the link between the L 1 and the L 2 norms of f -f * . 2. For any ρ > 0, we set

H ρ := {f ∈ F : ||f -f * || = ρ and ||f -f * || L2 ≤ r(ρ)}, Γ f * := f ∈f * +(ρ/20)B ∂|| • || f , ∆(ρ) := inf f ∈Hρ sup z * ∈Γ f * (ρ K ) z * (f -f * ),
3. We will use the so-called sparsity inequality ∆(ρ) ≥ 4ρ/5, and we define ρ * as the smallest ρ > 0 satisfying it.

The risk of a function

f ∈ F is defined by R(f ) := IE (Y -f (X)) 2 1/2 .
Definition 3.3. Let i be independent random variables uniformly distributed on {-1, 1}, and independent from the dataset D. For all f ∈ F , r > 0 and ρ > 0, we denote B reg (f, ρ, r)

:= {g ∈ F : ||g -f || L2 ≤ r, ||g -f || ≤ ρ}.
For every γ Q , γ M > 0, we define two so-called Rademacher complexities by

r Q (ρ, γ Q ) = inf r > 0 : ∀J ⊂ I, |J| ≥ N 2 , IE sup f ∈Breg(f * ,ρ,r) i∈J i (f -f * )(X i ) ≤ γ Q |J|r , r M (ρ, γ M ) = inf r > 0 : ∀J ⊂ I, |J| ≥ N 2 , IE sup
and let r = r( • , γ M , γ Q ) be a continuous non-decreasing function R + → R + depending on γ Q , γ M such that for every ρ > 0, r(ρ, γ Q , γ M ) > r Q (ρ, γ Q ) and r(ρ, γ Q , γ M ) > r M (ρ, γ M ).
Theorem 3.4. Let Assumption 3.3.1 hold and assume that

n ≥ K(κ * -1) 1/2 /400, (3.5)
r 2 (2ρ K ) ≤ min 2γ 1 σ * 2 , 2cθ 2 0 σ * 2 , (3.6)
(2 + γ1 )γ 1 σ * ≤ c 40µ 0 cθ 0 r 2 (2ρ K ), (3.7)
where γ1 := (K/n) 1/2 (κ * -1) 1/2 /10, with the choice γ Q = 1/(720θ 0 ), and γ M = /360. Let us define K * as the smallest integer satisfying K * ≥ n r 2 (ρ * )/(284θ 2 m ) and := 1/(cθ 2 0 ) for a constant c > 0. Let K be an integer in max(K * , 8|O|) , n/(96(θ 0 θ m ) 2 ) . We define implicitly ρ K as the solution of Definition 5]. Let c , µ 0 > 0 such that

r 2 (ρ K ) = (384θ 2 m / 2 ) • (K/n), where r(•) is defined in [91,
µ 0 ≤ 64σ * (1 + γ1 )/5, (3.8)
µ 0 ≤ 4c σ min /5, (3.9)

µ 0 ≥ 16c c -32 σ * 1 + γ1 /2 (3.10) σ * ≤ 1 + (17c /20µ 0 ) σ min (3.11) c ≥ 32σ * (1 -γ1 ) 2 σ * -r(ρ K ) + 2 σ * -σ * γ1 + 4c µ 0 (3.12) c ≥ 10µ 0 σ min - 5µ 0 σ * . (3.13)
Chapter 3. Robust-to-outliers simultaneous inference and noise level estimation using a MOM approach 62

Choosing the regularization parameter as µ := (c /µ 0 )r 2 (ρ K )/ρ K , we have

|| f -f * || ≤ 2ρ K , (3.14) || f -f * || L2 ≤ r(2ρ K ), (3.15) |σ -σ * | ≤ σ * γ1 , (3.16
)

R( f ) ≤ R(f * ) + 2 1 + + γ1 r 2 (2ρ K ) + σ * (1 + γ1 )(σ * γ1 + 2µρ K ) + (σ * + r(2ρ K )) 2 1 + γ1 σ * γ1 + 2 σ * -r(κρ K ) + c κ µ 0 r 2 (ρ K ) . (3.17)
with probability 1 -c 1 exp(-K/c 2 ), where c 1 and c 2 are universal constants.

This theorem allows us to bound in a general framework statistical errors on an event that holds with high probability. Indeed, on this event of probability 1 -c 1 exp(-K/c 2 ), we control the distance of the estimators f to its true value f * in the regularization norm || • || and in the || • || L2 norm. We also control the distance between the estimated standard deviation σ and the true standard deviation σ * . Finally, we control the risk of f , meaning that it is smaller than the risk associated with the true function f * , plus a residual term.

Note that, even if many parameters appear in the statement of Theorem 3.4 above, the estimator still depends only on two parameters, that are µ and K. If µ can be chosen as in the statement of this theorem, as well as other parameters, the theorem apply, and gives bounds on our estimators with high probability. If the condition of Theorem 3.4 could not be satisfied with any choice of c , c, µ 0 for a given µ, then the joint estimator ( f , σ) can still be computed, but without any theoretical guarantee on its performance.

Technical lemmas

The following lemma is adapted from from Lecué and Lerasle [START_REF] Lecué | Robust machine learning by median-of-means: theory and practice[END_REF]Equations (16), [START_REF] Bickel | Uniform convergence of probability measures on classes of functions[END_REF], and [START_REF] Bickel | Simultaneous analysis of Lasso and Dantzig selector[END_REF]].

Lemma 3.5.

There exists an event Ω 1 (K) of probability bigger than 1 -4 exp(-K/4320) such that, for all ρ ∈ {κρ K : κ ∈ {1, 2}}, and all f ∈ F such that ||f -f * || ≤ ρ, we have

1. If ||f -f * || L2 ≥ r Q (ρ, γ Q ), then Q 1/4,K (f -f * ) 2 ≥ Q 1/8,K (f -f * ) 2 ≥ 1 (4θ0) 2 ||f -f * || 2 L2 , 2. Q 3/4,K 2ζ(f -f * ) ≤ Q 7/8,K 2ζ(f -f * ) ≤ α 1 3. P [-2ζ(f -f * )] ≤ min Q 1/8,K [-2ζ(f -f * )] , Q 1/4,K [-2ζ(f -f * )] + α 1
where

α 1 := 2 • max r 2 M (ρ, γ M ) , 720θ 2 m 2 K n , ||f -f * || L2 and := 1/(cθ 2 0 ).
Proof of Lemma 3.5: The results are proved by following the same steps as in the proof of Equations ( 16), [START_REF] Bickel | Uniform convergence of probability measures on classes of functions[END_REF], and [START_REF] Bickel | Simultaneous analysis of Lasso and Dantzig selector[END_REF] in Lecué and Lerasle [START_REF] Lecué | Robust machine learning by median-of-means: theory and practice[END_REF], choosing η = 1/8, γ = 15/16, α = x = 1/45, γ Q = 1/(720θ 0 ), and γ M = /360.

The following lemma is proved in [91, Lemma 4]. We reproduce it here for convenience.

Chapter 3. Robust-to-outliers simultaneous inference and noise level estimation using a MOM approach 63

Lemma 3.6. Let ρ ≥ 0, Γ f * (ρ) := f ∈f * +(ρ/20)B ∂|| • || f . For all g ∈ F , ||f * || -||g|| ≤ ρ 10 -sup z * ∈Γ f * (ρ) z * (g -f * ). Lemma 3.7. Let ρ > 0, for all f ∈ F such that ||f -f * || ≤ ρ and |σ -σ * | ≤ α σ , we have P [-2ζ(f -f * )] ≤ (σ * + α σ )T K,λ (f * , σ * , f , σ) + α 1 + (σ * 2 + γ 1 ) α σ σ * 2 + (σ * + α σ )(α σ + µρ).
where

α 1 := 2 • max r 2 M (ρ, γ M ) , 720θ 2 m 2 K n , ||f -f * || L2 .
Proof: We apply part 3 of Lemma 3.5 and get

P [-2ζ(f -f * )] ≤ Q 1/4,K [-2ζ(f -f * )] + α 1 ≤ Q 1/4,K [(f -f * ) 2 -2ζ(f -f * )] + α 1 ≤ σQ 1/4,K 1 σ (l f -l f * ) + α 1 ≤ σQ 1/4,K l f σ + (σ -σ * ) - l f * σ * + l f * 1 σ * - 1 σ + α 1 + σα σ ≤ σQ 1/2,K l f σ + (σ -σ * ) - l f * σ * -Q 1/4,K -l f * 1 σ * - 1 σ + σµ(||f || -||f * ||) + α 1 + σ(α σ + µρ) ≤ σT K,λ (f * , σ * , f , σ) -Q 1/4,K [-ζ 2 ] 1 σ * - 1 σ + α 1 + σ(α σ + µρ).
We apply now Lemma 3.9 and the mean value theorem to the function x → 1/x, and we get

P [-2ζ(f -f * )] ≤ σT K,λ (f * , σ * , f , σ) + α 1 + (σ * 2 + γ 1 ) α σ σ * 2 + σ(α σ + µρ) ≤ (σ * + α σ )T K,λ (f * , σ * , f , σ) + α 1 + (σ * 2 + γ 1 ) α σ σ * 2 + (σ * + α σ )(α σ + µρ).
In the following, we will use the notation K := {k ∈ {1, . . . , K} : B k ⊂ I}.

Lemma 3.8. Let γ, γ 1 , η, x > 0 such that γ(1 -KV ar[Z]/(nγ 2 1 ) -x) ≥ 1 -η. Let K ∈ [|O|/(1 -γ), n].
Let Z be a real-valued random variable. There exists an event Ω(Z, K) with probability greater than

1 -exp(-γx 2 K/2) such that, on the event Ω(Z, K) {k ∈ [K] : |P B k (Z) -IEZ| ≤ γ 1 } ≥ K(1 -η). Proof : We have |{k ∈ [K] : |P B k (Z) -IEZ| ≤ γ 1 }| ≥ k∈K 1{|P B k (Z) -IEZ| ≤ γ 1 } = |K| - k∈K IP{|P B k (Z) -IEZ| ≥ γ 1 } - k∈K 1{|P B k (Z) -IEZ| ≥ γ 1 } -IP{|P B k (Z) -IEZ| ≥ γ 1 } .
We bound the first term using Chebychev's inequality

k∈K IP{|P B k (Z) -IEZ| ≥ γ 1 } ≤ |K| V ar[P B k (Z) -IEZ] γ 2 1 = |K| V ar[Z] |B k |γ 2 1 = |K| KV ar[Z] nγ 2 1 .
Chapter 3. Robust-to-outliers simultaneous inference and noise level estimation using a MOM approach 64

We bound the other term using Hoeffding's inequality

k∈K 1{|P B k (Z) -IEZ| ≥ γ 1 } -IP{|P B k (Z) -IEZ| ≥ γ 1 } ≤ x|K|,
on an event Ω(Z, K) of probability greater than 1 -exp(-x 2 |K|/2). Combining the previous inequalities, we get that on Ω(Z, K),

|{k ∈ K : |P B k (Z) -IEZ| ≤ γ 1 }| ≥ |K| 1 - KV ar[Z] nγ 2 1 -x ≥ Kγ 1 - KV ar[Z] nγ 2 1 -x .
Lemma 3.9 (Bounding the quantiles of ζ 2 ). Assume that K ≤ nα 3 . On an event Ω 2 (K) of probability greater than 1 -exp(-5K/13824), we have

Q 1/4,K [ζ 2 ] ≤ σ * 2 + γ 1 and -σ * 2 -γ 1 ≤ Q 1/4,K [-ζ 2 ] ≤ -σ * 2 + γ 1 , where γ 1 = α 1/2 3 σ * 2 (κ * -1) 1/2 /5.
The same inequalities are also valid with 1/4 replaced by 1/8, on the same event. As a consequence, this is valid for the choice α 3 := K/n.

Proof of Lemma 3.9: We apply Lemma 3.8 with

Z := ζ 2 , V ar[Z] = IE[ζ 4 ] -IE[ζ 2 ] 2 = σ * 4 (κ * -1), η = 1/8, γ = 15/16, x = 1/36, γ 1 = 6 α 3 V ar[Z] 1/2 , so that γ(1 -KV ar[Z]/(nγ 2 1 ) -x) ≥ 1 -η with probability 1 -exp(-γx 2 K/2) = 1 -exp(-5K/13824). Therefore, on the same event, all Q 1/8,K [ζ 2 ], Q 1/4,K [ζ 2 ], Q 3/4,K [ζ 2 ] and Q 7/8,K [ζ 2 ] belongs to the interval IEζ 2 -γ 1 , IEζ 2 + γ 1 .
We get also that

Q 1/8,K [-ζ 2 ], Q 1/4,K [-ζ 2 ] ∈ -IEζ 2 + γ 1 , -IEζ 2 + γ 1 .
Lemma 3.10. Let κ ∈ {1, 2} and α 2,κ = (K/n) 1/2 σ * (κ * -1) 1/2 /10. Under the assumptions of Theorem 3.4, we have

1. γ 1 ≤ 2σ * α 2,κ + α 2 2,κ , therefore σ * 2 + γ 1 ≤ σ * + α 2,κ ; 2. 2 -(4θ0) -2 σ * +α2,κ + c µ0 ≤ 0 ; 3. α 2,κ 2 ≤ γ 1 + 2α 2,κ σ * -r 2 (κρ K ), therefore σ * -α 2,κ ≤ σ * 2 + γ 1 -r 2 (κρ K ) ; 4. 2 r 2 (ρ K )-γ1 σ * 2 ≤ 4 + γ1 σ * 2 ; 5. 2 -(4θ0) -2 σ * +α2,1 + 11c 10µ0 ≤ 2 σ * -r(ρ K ) + c µ0
where

γ 1 := (K/n) 1/2 σ * 2 (κ * -1) 1/2 /5
Proof of Lemma 3.10: 1. By construction, we have γ 1 = 2σ * α 2,κ . Therefore, the claimed inequality is satisfied.

2. Because we have assumed that Equation (3.10) holds, we have

µ 0 ≥ 16c c -32 σ * 1 + (K/n) 1/2 (κ * -1) 1/2 /10 .
This is equivalent to

µ 0 2 c - 1 16 + c c σ * 1 + (K/n) 1/2 (κ * -1) 1/2 /10 ≤ 0,
which can be rewritten as 5. We have

2/(cθ 2 0 ) -1/(4θ 0 ) 2 σ * 1 + (K/n) 1/2 (κ * -1)
2 -(4θ 0 ) -2 σ * + α 2,1 + 11c 10µ 0 ≤ c 10µ 0 ≤ 2 σ * -r(ρ K ) + c µ 0 ,
where the first inequality is a consequence of part 4 of this lemma.

3.5 Control of the supremum of T K,µ (g, χ, f * , σ * ) on each F (κ) i

Preliminaries

In this section, we will assume to be on the event Ω(K) := Ω 1 (K) ∩ Ω 2 (K), where Ω 1 (K) is defined in Lemma 3.5 and Ω 2 (K) is defined in Lemma 3.9. For κ ∈ {1, 2}, and any fixed α 2,κ > 0, let us define

F (κ) 1 := {(g, χ) ∈ F × R * + : ||g -f * || ≤ κρ K , ||g -f * || L2 ≤ r(κρ K ) and |σ * -χ| ≤ α 2,κ } F (κ) 2 := {(g, χ) ∈ F × R * + : ||g -f * || ≤ κρ K , ||g -f * || L2 > r(κρ K ) and |σ * -χ| ≤ α 2,κ } F (κ) 3 := {(g, χ) ∈ F × R * + : ||g -f * || > κρ K and |σ * -χ| ≤ α 2,κ } F (κ) 4 := {(g, χ) ∈ F × R * + : ||g -f * || ≤ κρ K , ||g -f * || L2 ≤ r(κρ K ) and χ > σ * + α 2,κ } F (κ) 5 := {(g, χ) ∈ F × R * + : ||g -f * || ≤ κρ K , ||g -f * || L2 > r(κρ K ) and χ > σ * + α 2,κ } F (κ) 6 := {(g, χ) ∈ F × R * + : ||g -f * || > κρ K and χ > σ * + α 2,κ } F (κ) 7 := {(g, χ) ∈ F × R * + : ||g -f * || ≤ κρ K , ||g -f * || L2 ≤ r(κρ K ) and χ < σ * -α 2,κ } F (κ) 8 := {(g, χ) ∈ F × R * + : ||g -f * || ≤ κρ K , ||g -f * || L2 > r(κρ K ) and χ < σ * -α 2,κ } F (κ) 9 := {(g, χ) ∈ F × R * + : ||g -f * || > κρ K and χ < σ * -α 2,κ }.
Lemma 3.11. On the event Ω(K), it holds for all κ ∈ {1, 2} that sup

(g,χ)∈F (κ) 1 T K,µ (g, χ, f * , σ * ) ≤ 2 + γ 1 σ * 2 α 2,κ + 2 σ * -r(κρ K ) + c κ µ 0 r 2 (ρ K ) := B 1,κ sup (g,χ)∈F (κ) 2 T K,µ (g, χ, f * , σ * ) ≤ 2 + γ 1 σ * 2 α 2,κ + 2 -(4θ 0 ) -2 σ * -α 2,κ + c κ µ 0 r 2 (ρ K ) := B 2,κ sup (g,χ)∈F (κ) 3 T K,µ (g, χ, f * , σ * ) ≤ max 2 + γ 1 σ * 2 α 2,κ + κ 2 σ * -α 2,κ - 7c 10µ 0 r 2 (ρ K ) , 2 + γ 1 σ * 2 α 2,κ + κ 2 -(4θ 0 ) -2 σ * + α 2,κ + 11c 10µ 0 r 2 (ρ K ) := B 3,κ sup (g,χ)∈F (κ) 4 T K,µ (g, χ, f * , σ * ) ≤ γ 1 σ * 2 α 2,κ + 2 σ * + α 2,κ + c κ µ 0 r 2 (ρ K ) := B 4,κ sup (g,χ)∈F (κ) 5 T K,µ (g, χ, f * , σ * ) ≤ γ 1 σ * 2 α 2,κ + c κ µ 0 r 2 (ρ K ) := B 5,κ sup (g,χ)∈F (κ) 6 T K,µ (g, χ, f * , σ * ) ≤ max γ 1 σ * 2 α 2κ + κ 2 σ * + α 2,κ - 7c 10µ 0 r 2 (ρ K ) ,
Chapter 3. Robust-to-outliers simultaneous inference and noise level estimation using a MOM approach 66

γ 1 σ * 2 -1 α 2,κ + κ 2 -(4θ 0 ) -2 σ + α 2,κ + 11c 10µ 0 r 2 (ρ K ) := B 6,κ sup (g,χ)∈F (κ) 7 T K,µ (g, χ, f * , σ * ) ≤ 2 r 2 (κρ K ) -γ 1 σ * 2 -2 α 2,κ + 2 σ * + c κ µ 0 r 2 (ρ K ) := B 7,κ sup (g,χ)∈F (κ) 8 T K,µ (g, χ, f * , σ * ) ≤ 2 + γ 1 σ * 2 α 2,κ + (2 -(4θ 0 ) -2 ) σ * -α 2,κ + c κ µ 0 r 2 (ρ K ) := B 8,κ sup (g,χ)∈F (κ) 9 T K,µ (g, χ, f * , σ * ) ≤ max γ 1 σ * 2 α 2,κ + 2 σ min - 7c 10µ 0 κr 2 (ρ K ) , 2 + γ 1 σ * 2 α 2,κ + κ 2 -(4θ 0 ) -2 σ -α 2,κ + 11c 10µ 0 r 2 (ρ K ) := B 9,κ .
Each bound is respectively proved in each of the subsections of Section 3.6. The following lemma gives a comparison between all these bounds. Lemma 3.12. We have B 1,1 = max i=1,...,9 B i,1 and -B 1,1 > max i=2,...,9 B i,2 .

Proof of the first assertion of Lemma 3.12

In this section, we show that B 1,1 is bigger than the other B i,1 , i = 2, . . . , 9.

Case i = 2: using α 2,1 ≤ r(1 × ρ K ), we get 2 -(4θ 0 ) 2 σ * -α 2,1 ≤ 2 σ * -α 2,1 ≤ 2 σ * -r(1 × ρ K ) , therefore B 2,1 ≤ B 1,1 . Case i = 3: 2 σ * -α 2,1 - 7c 10µ 0 ≤ 2 σ * -r(1 × ρ K ) - 7c 10µ 0 ≤ 2 σ * -r(1 × ρ K ) + c × 1 µ 0 ,
and, applying part 5 of Lemma 3.10,

2 -(4θ0) -2 σ * +α2,1 + 11c 10µ0 ≤ 2 σ * -r(1×ρ K ) + c µ0 , therefore B 3,1 ≤ B 1,1 .
Cases i = 4, 5, 8: Each of the two terms in the definition of B i,1 is smaller than the corresponding term in the definition of B 1,1 , therefore B i,1 ≤ B 1,1 .

Case i = 6: Same as for i = 3.

Case i = 7: The result in this case follows because

2+ γ1 σ * 2 ≥ 2 r 2 (ρ K )-γ1 σ * 2
-2, by part 4 of Lemma 3.10.

Case i = 9: B 9,1 ≤ B 1,1 if 2 σ min - 7c 10µ 0 ≤ 2 σ * -r(1 × ρ K ) + c × 1 µ 0 ,
which is equivalent to

1 σ min ≤ 1 σ * -r(ρ K ) + 17c 20µ 0 ,
i.e. σ * -r(κρ K ) ≤ 1 + (17c /20µ 0 ) σ min , which is true as we have assumed Equation (3.11).

Chapter 3. Robust-to-outliers simultaneous inference and noise level estimation using a MOM approach 67

Proof of the second assertion of Lemma 3.12

In this section, we show that -B 1,1 is bigger than B i,2 , for all i = 2, . . . , 9. We only give a sketch of the proof here since it results from elementary computations. Because of Equation (3.7) we can control all the terms of the form α 2,κ by terms of the form r 2 (ρ K ). Therefore, we only compare such terms. All inequalities are similar and consequences of Equation (3.12), using the definition = 1/(cθ 2 0 ). For the term i = 9, the inequality is a consequence of Equation (3.13).

Proof of Lemma 3.11

Let us remark that

T K,µ (f, σ, f * , σ * ) := M OM K l f * σ * + σ * - l f σ -σ + µ(||f * || -||f ||) = M OM K l f * 1 σ * - 1 σ + (σ * -σ) + 1 σ (l f * -l f ) + µ(||f * || -||f ||) = M OM K l f * 1 σ * - 1 σ + (σ * -σ) + 1 σ 2ζ(f -f * ) -(f -f * ) 2 + µ(||f * || -||f ||).
This decomposition will be a key component of the proofs below.

Bound on

F (κ) 1 Let g ∈ F (κ)
1 . Using the inequality (g -f * ) 2 ≥ 0 and the triangular inequality, we get

T K,µ (g, χ, f * , σ * ) = M OM K l f * 1 σ * - 1 χ + (σ * -χ) + 1 χ 2ζ(g -f * ) -(g -f * ) 2 + µ(||f * || -||g||) ≤ α 2,κ + M OM K l f * 1 σ * - 1 χ + 2 χ ζ(g -f * ) + µ||f * -f || ≤ α 2,κ + 2 χ Q 3/4,K ζ(g -f * ) -Q 1/4,K [ζ 2 ] 1 χ - 1 σ * + µκρ K ≤ α 2,κ + 1 σ * -r(κρ K ) Q 3/4,K 2ζ(g -f * ) + (σ * 2 + γ 1 ) α 2,κ σ * 2 + µκρ K .
where in the last line, we used Lemma 3.9. By the part 2 of Lemma 3.5, using the fact that α 1 ≤ 2 r 2 (κρ K ), and plugging our choice of µ, we get

T K,µ (g, χ, f * , σ * ) ≤ 1 + σ * 2 + γ 1 σ * 2 α 2,κ + 2 σ * -r(κρ K ) + c κ µ 0 r 2 (ρ K ).

Bound on

F (κ) 2 Let (g, χ) ∈ F (κ)
2 . We have

T K,µ (g, χ, f * , σ * ) = M OM K l f * 1 σ * - 1 χ + (σ * -χ) + 1 χ 2ζ(g -f * ) -(g -f * ) 2 + µ(||f * || -||g||) ≤ (σ * -χ) + Q 3/4,K l f * 1 σ * - 1 χ + 1 χ 2ζ(g -f * ) - 1 χ Q 1/4 [(g -f * ) 2 ] + µ(||f * -g||) ≤ α 2,κ + Q 3/4,K l f * 1 σ * - 1 χ + 1 χ 2ζ(g -f * ) - 1 χ Q 1/4 [(g -f * ) 2 ] + µκρ K
Chapter 3. Robust-to-outliers simultaneous inference and noise level estimation using a MOM approach 68

≤ α 2,κ + 1 χ Q 7/8,K 2ζ(g -f * ) -Q 1/8 [-ζ 2 ] 1 σ * - 1 χ - 1 χ Q 1/4 [(g -f * ) 2 ] + µκρ K .
On the right hand-side of the last equation, we will bound the second term using part 1 of Lemma 3.5, the third term using Lemma 3.9 and the fourth term using part 2 of Lemma 3.5. Finally, we replace µ and ρ K by their values so that we get

T K,µ (g, χ, f * , σ * ) ≤ α 2,κ + α 1 -||f * -g|| 2 L 2 P (4θ 0 ) -2 χ + α 2,κ (σ * 2 + γ 1 ) σ * 2 + c κ µ 0 r 2 (ρ K ).
We have 2 < (4θ 0 ) -2 , therefore α 1 -||f * -g|| 2

L 2 P (4θ 0 ) -2 ≤ (2 -(4θ 0 ) -2 )r 2 (ρ K ) and we can deduce that T K,µ (g, χ, f * , σ * ) ≤ 1 + σ * 2 + γ 1 σ * 2 α 2,κ + 2 -(4θ 0 ) -2 σ * -α 2,κ + c κ µ 0 r 2 (ρ K ).

Bound on

F (κ) 3 Let (g, χ) ∈ F (κ)
3 . We have

T K,µ (g, χ, f * , σ * ) = M OM K l f * 1 σ * - 1 χ + (σ * -χ) + 1 χ 2ζ(g -f * ) -(g -f * ) 2 + µ(||f * || -||g||) ≤ (σ * -χ) + M OM K l f * 1 σ * - 1 χ + 1 χ 2ζ(g -f * ) -(g -f * ) 2 -µ sup z * ∈Γ f * (ρ K ) z * (g -f * ) + µρ K 10 , (3.18)
where the last line results from the application of Lemma 3.6 with ρ = ρ K . We follow now the proof of Lemma 5 in [START_REF] Lecué | Robust machine learning by median-of-means: theory and practice[END_REF]. Let us define f := f * + ρ K (g -f * )/||g -f * ||. By convexity of F , we get f ∈ F . Let

Υ := ||g -f * ||/ρ K . Noting that ||f -f * || = ρ K , and g -f * = Υ(f -f * ), we have M OM K l f * 1 σ * - 1 χ + 1 χ 2ζ(g -f * ) -(g -f * ) 2 -µ sup z * ∈Γ f * (ρ K ) z * (g -f * ) = M OM K l f * 1 σ * - 1 χ + 1 χ 2Υζ(f -f * ) -Υ 2 (f -f * ) 2 -µΥ sup z * ∈Γ f * (ρ K ) z * (f -f * ) ≤ M OM K l f * 1 σ * - 1 χ + 1 χ 2Υζ(f -f * ) -Υ(f -f * ) 2 -µΥ sup z * ∈Γ f * (ρ K ) z * (f -f * ),
where in the last line, we used the fact that Υ ≥ 1. Therefore, combining the previous equation with Equation (3.18), we get

T K,µ (g, χ, f * , σ * ) ≤ (σ * -χ) + M OM K l f * 1 σ * - 1 χ + Υ χ 2ζ(f -f * ) -(f -f * ) 2 -µΥ sup z * ∈Γ f * (ρ K ) z * (f -f * ) + µκρ K 10 , (3.19
)

3.6.3.1 Case ||f -f * || L 2 P ≤ r(ρ K )
Remembering that ||f -f * || = ρ K , we can deduce that f ∈ H ρ K Using the definition of K * , the fact that K ≥ K * , we get that ρ K ≥ ρ, and therefore ρ K follows the sparsity equation, from which we derive

sup z * ∈Γ f * (ρ K ) z * (f -f * ) ≥ ∆(ρ K ) ≥ 4ρ K /5.
Using our choice of µ, we get

-µ sup z * ∈Γ f * (ρ K ) z * (f -f * ) ≤ - 4c 5µ 0 r 2 (ρ K ).
Chapter 3. Robust-to-outliers simultaneous inference and noise level estimation using a MOM approach 69

Combining this with Equation (3.19) and the fact that (f -f * ) 2 ≥ 0, we get that

T K,µ (g, χ, f * , σ * ) ≤ (σ * -χ) + M OM K l f * 1 σ * - 1 χ + Υ χ 2ζ(f -f * ) -Υ 4c 5µ 0 r 2 (ρ K ) + c κ 10µ 0 r 2 (ρ K ) ≤ α 2,κ + Υ χ Q 3/4,K 2ζ(f -f * ) - 1 σ * - 1 χ Q 1/4,K -ζ 2 -Υ 4c 5µ 0 r 2 (ρ K ) + c κ 10µ 0 r 2 (ρ K ) ≤ 1 + σ * 2 + γ 1 σ * 2 α 2,κ + Υ 1 χ Q 3/4,K 2ζ(f -f * ) - 4c 5µ 0 r 2 (ρ K ) + c κ 10µ 0 r 2 (ρ K ).
Applying part 2 of Lemma 3.5, we get

Q 3/4,K 2ζ(f -f * ) ≤ α 1 ≤ 2 r 2 (ρ K ). Therefore, T K,µ (g, χ, f * , σ * ) ≤ 2 + γ 1 σ * 2 α 2,κ + Υ 2 σ * -α 2,κ - 4c 5µ 0 r 2 (ρ K ) + c κ 10µ 0 r 2 (ρ K ).
Using the inequalities µ 0 ≤ 4c (σ * -α 2,κ )/5, and Υ > κ , we finally get

T K,µ (g, χ, f * , σ * ) ≤ 2 + γ 1 σ * 2 α 2,κ + κ 2 σ * -α 2,κ - 7c 10µ 0 r 2 (ρ K ). 3.6.3.2 Case ||f -f * || L 2 P > r(ρ K )
We have ||f -f * || = ρ K , therefore, it follows from parts 1 and 2 of Lemma 3.5,

M OM K l f * 1 σ * - 1 χ + Υ χ 2ζ(f -f * ) -(f -f * ) 2 ≤ Υ χ Q 7/8,K 2ζ(f -f * ) -Q 1/4,K (f -f * ) 2 - 1 σ * - 1 χ Q 1/8,K -ζ 2 ≤ Υ χ α 1 -4θ -2 0 ||f -f * || 2 L2 + σ * 2 + γ 1 σ * 2 α 2,κ ≤ Υ χ 2 -(4θ 0 ) -2 ||f -f * || 2 L2 + σ * 2 + γ 1 σ * 2 α 2,κ ≤ κ σ * + α 2,κ 2 -(4θ 0 ) -2 r 2 (ρ K ) + σ * 2 + γ 1 σ * 2 α 2,κ , because 2 -(4θ 0 ) -2 ≤ 0.
T K,µ (g, χ, f * , σ * ) ≤ 2 + γ 1 σ * 2 α 2,κ + Υ 2 -(4θ 0 ) -2 σ * + α 2,κ r 2 (ρ K ) + µρ K + µκρ K 10 ≤ 2 + γ 1 σ * 2 α 2,κ + Υ 2 -(4θ 0 ) -2 σ * + α 2,κ + 16 µ 0 r 2 (ρ K ) + c κ 10µ 0 r 2 (ρ K ) ≤ 2 + γ 1 σ * 2 α 2,κ + κ 2 -(4θ 0 ) -2 σ * + α 2,κ + 11c 10µ 0 r 2 (ρ K ).

Bound on

F (κ) 4 Let (g, χ) ∈ F (κ)
4 . Recall that, in this case, χ > σ * + α 2,κ . We have

T K,µ (g, χ, f * , σ * ) = M OM K l f * 1 σ * - 1 χ + (σ * -χ) + 1 χ 2ζ(g -f * ) -(g -f * ) 2 + µ(||f * || -||g||) ≤ (σ * -χ) + M OM K l f * σ * + 1 χ 2ζ(g -f * ) + µ||f * -g|| ≤ (σ * -χ) + 1 χ Q 3/4,K 2ζ(g -f * ) + 1 σ * - 1 χ Q 1/4,K [-ζ 2 ] + µκρ K .
Chapter 3. Robust-to-outliers simultaneous inference and noise level estimation using a MOM approach 70

We apply part 2 of Lemma 3.5 and Lemma 3.9, and, using our choice of µ and ρ K , we deduce that

T K,µ (g, χ, f * , σ * ) ≤ (σ * -χ) + 1 σ * - 1 χ σ * 2 + γ 1 + 2 σ * + α 2,κ + c κ µ 0 r 2 (ρ K ).
The function χ → -χ -a/χ is decreasing for χ ≥ √ a. Applying the first part of Lemma 3.10, we have

σ * 2 + γ 1 ≤ σ * + α 2,κ .
Therefore, the maximum of the former function is attained for χ = σ * + α 2,κ , which yields

T K,µ (g, χ, f * , σ * ) ≤ -α 2,κ + 1 σ * - 1 σ * + α 2,κ σ * 2 + γ 1 + 1 σ * + α 2,κ + c κ µ 0 r 2 (ρ K ) ≤ σ * 2 + γ 1 σ * 2 -1 α 2,κ + 2 σ * + α 2,κ + c κ µ 0 r 2 (ρ K ).

Bound on

F (κ) 5 Let (g, χ) ∈ F (κ)
5 . Recall that, in this case, χ > σ * + α 2,κ . We have

T K,µ (g, χ, f * , σ * ) = M OM K l f * 1 σ * - 1 χ + (σ * -χ) + 1 χ 2ζ(g -f * ) -(g -f * ) 2 + µ(||f * || -||g||) ≤ (σ * -χ) + 1 χ Q 7/8,K 2ζ(g -f * ) -Q 1/4,K (g -f * ) 2 - 1 σ * - 1 χ Q 1/8,K -ζ 2 + µκρ K .
We bound Q 7/8,K 2ζ(g -f * ) using part 2 of Lemma 3.5 ; Q 1/4,K (g -f * ) 2 using part 1 of Lemma 3.5

; and Q 1/8,K -ζ 2 by Lemma 3.9.

T K,µ (g, χ, f * , σ * ) ≤ (σ * -χ) + α 1 -||f * -g|| 2 L 2 P (4θ 0 ) -2 χ + 1 σ * - 1 χ σ * 2 + γ 1 + µκρ K .
We have 2 < (4θ 0 ) -2 , therefore α 1 -||f * -g|| 2 L 2 P (4θ 0 ) -2 ≤ 0 and, plugging in our choice of µ and ρ K , we can deduce that

T K,µ (g, χ, f * , σ * ) ≤ (σ * -χ) + 1 σ * - 1 χ σ * 2 + γ 1 + c κ µ 0 r 2 (ρ K )
The function χ → -χ -a/χ is decreasing for χ ≥ √ a. Applying the first part of Lemma 3.10, we have

σ * 2 + γ 1 ≤ σ * + α 2,κ .
Therefore, the maximum of the former function is attained for χ = σ * + α 2,κ , which yields

T K,µ (g, χ, f * , σ * ) ≤ -α 2,κ + 1 σ * - 1 σ * + α 2,κ σ * 2 + γ 1 + c κ µ 0 r 2 (ρ K ) ≤ σ * 2 + γ 1 σ * 2 -1 α 2,κ + c κ µ 0 r 2 (ρ K ).

Bound on F (κ) 6

Let (g, χ) ∈ F (κ) [START_REF] Almeida | Efficient bayesian inference for stochastic time-varying copula models[END_REF] . Recall that, in this case, χ > σ * + α 2,κ . Following the beginning of the proof in Section 3.6.3, we have as in Equation (3.19)

T K,µ (g, χ, f * , σ * ) ≤ (σ * -χ) + M OM K l f * 1 σ * - 1 χ + Υ χ 2ζ(f -f * ) -(f -f * ) 2 -µΥ sup z * ∈Γ f * (ρ K ) z * (f -f * ) + µκρ K 10 , (3.20)
Chapter 3. Robust-to-outliers simultaneous inference and noise level estimation using a MOM approach 71

3.6.6.1 Case ||f -f * || L 2 P ≤ r(ρ K )
Remembering that ||f -f * || = ρ K , we can deduce that f ∈ H ρ K Using the definition of K * , the fact that K ≥ K * , we get that ρ K ≥ ρ, and therefore ρ K follows the sparsity equation, from which we derive

sup z * ∈Γ f * (ρ K ) z * (f -f * ) ≥ ∆(ρ K ) ≥ 4ρ K /5.
Using our choice of µ, we get

-µ sup z * ∈Γ f * (ρ K ) z * (f -f * ) ≤ - 4c 5µ 0 r 2 (ρ K ).
Combining this with Equation (3.20) and the fact that (f -f * ) 2 ≥ 0, we get that

T K,µ (g, χ, f * , σ * ) ≤ (σ * -χ) + M OM K l f * 1 σ * - 1 χ + Υ χ 2ζ(f -f * ) -Υ 4c 5µ 0 r 2 (ρ K ) + c κ 10µ 0 r 2 (ρ K ) ≤ (σ * -χ) + Υ χ Q 3/4,K 2ζ(f -f * ) - 1 σ * - 1 χ Q 1/4,K -ζ 2 -Υ 4c 5µ 0 r 2 (ρ K ) + c κ 10µ 0 r 2 (ρ K ) ≤ (σ * -χ) + 1 σ * - 1 χ (σ * 2 + γ 1 ) + Υ 1 χ Q 3/4,K 2ζ(f -f * ) - 4c 5µ 0 r 2 (ρ K ) + c κ 10µ 0 r 2 (ρ K ).
Applying part 2 of Lemma 3.5, we get

Q 3/4,K 2ζ(f -f * ) ≤ α 1 ≤ 2 r 2 (ρ K ). Therefore, T K,µ (g, χ, f * , σ * ) ≤ (σ * -χ) + 1 σ * - 1 χ (σ * 2 + γ 1 ) + Υ 2 σ * + α 2,κ - 4c 5µ 0 r 2 (ρ K ) + c κ 10µ 0 r 2 (ρ K ).
Using the inequalities µ 0 ≤ 4c (σ * + α 2,κ )/5, and Υ > κ, we finally get

T K,µ (g, χ, f * , σ * ) ≤ (σ * -χ) + 1 σ * - 1 χ (σ * 2 + γ 1 ) + κ 2 σ * + α 2,κ - 7c 10µ 0 r 2 (ρ K ). (3.21)
The function χ → -χ -a/χ is decreasing for χ ≥ √ a. Applying the first part of Lemma 3.10, we have

σ * 2 + γ 1 ≤ σ * + α 2,κ .
Therefore, the maximum of the former function is attained for χ = σ * + α 2,κ , which, combined with Equation (3.21), yields

T K,µ (g, χ, f * , σ * ) ≤ -α 2κ + α 2,κ σ * 2 (σ * 2 + γ 1 ) + κ 2 σ * + α 2,κ - 7c 10µ 0 r 2 (ρ K ) ≤ γ 1 σ * 2 α 2κ + κ 2 σ * + α 2,κ - 7c 10µ 0 r 2 (ρ K ). 3.6.6.2 Case ||f -f * || L 2 P > r(ρ K )
We have ||f -f * || = ρ K , therefore, it follows from parts 1 and 2 of Lemma 3.5,

M OM K l f * 1 σ * - 1 χ + Υ χ 2ζ(f -f * ) -(f -f * ) 2 ≤ Υ χ Q 7/8,K 2ζ(f -f * ) -Q 1/4,K (f -f * ) 2 - 1 σ * - 1 χ Q 1/8,K -ζ 2 ≤ Υ χ α 1 -(4θ 0 ) -2 ||f -f * || 2 L2 + 1 σ * - 1 χ (σ * 2 + γ 1 ). ≤ Υ χ 2 -(4θ 0 ) -2 ||f -f * || 2 L2 + 1 σ * - 1 χ (σ * 2 + γ 1 ).
Plugging this back in Equation 3.20, we get

T K,µ (g, χ, f * , σ * ) ≤ (σ * -χ) + Υ χ 2 -(4θ 0 ) -2 ||f -f * || 2 L2 + 1 σ * - 1 χ (σ * 2 + γ 1 ) -µΥ sup z * ∈Γ f * (ρ K ) z * (f -f * ) + µκρ K 10 .
As 2 -(4θ 0 ) -2 ≤ 0, the function χ → -χ -a/χ is decreasing for χ ≥ √ a, and σ * + α 2,κ ≥ √ a, where here, a = σ * 2 + γ 1 + (4θ 0 ) -2 -2 , we get

T K,µ (g, χ, f * , σ * ) ≤ -α 2,κ + Υ σ + α 2,κ 2 -(4θ 0 ) -2 ||f -f * || 2 L2 + σ * 2 + γ 1 σ * 2 α 2,κ -µΥ sup z * ∈Γ f * (ρ K ) z * (f -f * ) + µκρ K 10 ≤ γ 1 σ * 2 α 2,κ + Υ 2 -(4θ 0 ) -2 r 2 (ρ K ) σ * + α 2,κ + µρ K + µκρ K 10 ≤ γ 1 σ * 2 α 2,κ + Υ 2 -(4θ 0 ) -2 σ * + α 2,κ + c µ 0 r 2 (ρ K ) + c κ r 2 (ρ K ) 10µ 0 .
Using Υ ≥ κ ≥ 0, and 2 -(4θ0) -2 σ * +α2,κ + c µ0 ≤ 0, by the second part of Lemma 3.10, we get

T K,µ (g, χ, f * , σ * ) ≤ γ 1 σ * 2 α 2,κ + κ 2 -(4θ 0 ) -2 σ * + α 2,κ + c µ 0 r 2 (ρ K ) + c κ r 2 (ρ K ) 10µ 0 ≤ γ 1 σ * 2 α 2,κ + κ 2 -(4θ 0 ) -2 σ * + α 2,κ + 11c 10µ 0 r 2 (ρ K ).

Bound on

F (κ) 7 Let (g, χ) ∈ F (κ) 7 .
Recall that, in this case, χ < σ * -α 2,κ . We have

T K,µ (g, χ, f * , σ * ) = M OM K l f * 1 σ * - 1 χ + (σ * -χ) + 1 χ 2ζ(g -f * ) -(g -f * ) 2 + µ(||f * || -||g||) ≤ σ * -χ + M OM K -l f * 1 χ - 1 σ * + 1 χ 2ζ(g -f * ) + µ(||f * -g||) ≤ σ * -χ + 1 χ Q 3/4,K 2ζ(g -f * ) + 1 χ - 1 σ * Q 1/4,K -ζ 2 + µκρ K .
We apply part 2 of Lemma 3.5, using the fact that α 1 ≤ 2 r 2 (ρ K ), and Lemma 3.9 to get

T K,µ (g, χ, f * , σ * ) ≤ σ * -χ + 2 r 2 (ρ K ) χ + 1 σ * - 1 χ (σ * 2 + γ 1 ) + µκρ K ≤ σ * -χ + σ * 2 + γ 1 σ * + 2 r 2 (κρ K ) -σ * 2 -γ 1 χ + µκρ K . We have 2 r 2 (ρ K ) < σ * 2 + γ 1 , so that r 2 (κρ K ) -σ * 2 -γ 1 < 0. The function χ → -χ -a/χ is increasing for 0 < χ ≤ √ a.
Applying the third part of Lemma 3.10, we have

σ * -α 2,κ ≤ σ * 2 + γ 1 -r 2 (κρ K ).
Therefore, the highest value is attained when χ = σ * -α 2,κ , and we get

T K,µ (g, χ, f * , σ * ) ≤ -α 2,κ + σ * 2 + γ 1 σ * + 2 r 2 (κρ K ) -σ * 2 -γ 1 σ * -α 2,κ + µκρ K = -α 2,κ + σ * 2 + γ 1 σ * + 2 r 2 (κρ K ) -σ * 2 -γ 1 σ * - 2 r 2 (κρ K ) -σ * 2 -γ 1 σ * + 2 r 2 (κρ K ) -σ * 2 -γ 1 σ * -α 2,κ + µκρ K ≤ -α 2,κ + 2 r 2 (κρ K ) σ * + α 2,κ 2 r 2 (κρ K ) -σ * 2 -γ 1 σ * 2 + µκρ K ,
Replacing µ and ρ K by their values, we get

T K,µ (g, χ, f * , σ * ) ≤ 2 r 2 (κρ K ) -σ * 2 -γ 1 σ * 2 -1 α 2,κ + 2 σ * + c κ µ 0 r 2 (ρ K ),
Chapter 3. Robust-to-outliers simultaneous inference and noise level estimation using a MOM approach 73

Bound on F (κ) 8

Let (g, χ) ∈ F (κ) 8 . Recall that, in this case, χ < σ * -α 2,κ . We have

T K,µ (g, χ, f * , σ * ) = M OM K l f * 1 σ * - 1 χ + (σ * -χ) + 1 χ 2ζ(g -f * ) -(g -f * ) 2 + µ(||f * || -||g||) ≤ (σ * -χ) + 1 χ Q 7/8,K 2ζ(g -f * ) - 1 χ Q 1/4,K (g -f * ) 2 + 1 χ - 1 σ * Q 1/8,K [-ζ 2 ] + µκρ K .
We bound the second term using part 2 of Lemma 3.5, the third using part 1 of Lemma 3.5 and the fourth using Lemma 3.9. Finally, we replace in the last term µ and ρ K by their values, so that we have

T K,µ (g, χ, f * , σ * ) ≤ (σ * -χ) + α 1 -||f * -g|| 2 L 2 P (4θ 0 ) -2 χ + 1 χ - 1 σ * -σ * 2 -γ 1 + c κ µ 0 r 2 (ρ K ) We have 2 < (4θ 0 ) -2 , therefore α 1 -||f * -g|| 2 L 2 P (4θ 0 ) -2 ≤ (2 -(4θ 0 ) -2 )r 2 (ρ K ) and we can deduce that T K,µ (g, χ, f * , σ * ) ≤ (σ * -χ) + 1 σ * - 1 χ σ * 2 + γ 1 + (2 -(4θ 0 ) -2 ) σ * -α 2,κ + c κ µ 0 r 2 (ρ K ).
The function χ → -χ -a/χ is increasing for 0 < χ < √ a. We have σ * -α 2,κ < σ * 2 + γ 1 . Therefore, the highest value is attained when χ = σ * -α 2,κ , and we get

T K,µ (g, χ, f * , σ * ) ≤ 1 + σ * 2 + γ 1 σ * 2 α 2,κ + (2 -(4θ 0 ) -2 ) σ * -α 2,κ + c κ µ 0 r 2 (ρ K ).
3.6.9 Bound on F (κ) 9

Let (g, χ) ∈ F (κ) [START_REF] Bahadur | The nonexistence of certain statistical procedures in nonparametric problems[END_REF] . Recall that in this case, χ < σ * -α 2,κ Following the beginning of the proof in Section 3.6.3, we have as in Equation 3.19

T K,µ (g, χ, f * , σ * ) ≤ (σ * -χ) + M OM K l f * 1 σ * - 1 χ + Υ χ 2ζ(f -f * ) -(f -f * ) 2 -µΥ sup z * ∈Γ f * (ρ K ) z * (f -f * ) + µκρ K 10 . (3.22) 3.6.9.1 Case ||f -f * || L 2 P ≤ r(ρ K ) Remembering that ||f -f * || = ρ K , we can deduce that f ∈ H ρ K .
Using the definition of K * , the fact that K ≥ K * , we get that ρ K ≥ ρ, and therefore ρ K follows the sparsity equation, from which we derive

sup z * ∈Γ f * (ρ K ) z * (f -f * ) ≥ ∆(ρ K ) ≥ 4ρ K /5.
Using our choice of µ, we get

-µ sup z * ∈Γ f * (ρ K ) z * (f -f * ) ≤ - 4c 5µ 0 r 2 (ρ K ).
Combining this with Equation (3.20) and the fact that (f -f * ) 2 ≥ 0, we get that

T K,µ (g, χ, f * , σ * ) ≤ (σ * -χ) + M OM K l f * 1 σ * - 1 χ + Υ χ 2ζ(f -f * ) -Υ 4c 5µ 0 r 2 (ρ K ) + c κ 10µ 0 r 2 (ρ K ) ≤ (σ * -χ) + Υ χ Q 3/4,K 2ζ(f -f * ) - 1 σ * - 1 χ Q 1/4,K -ζ 2 -Υ 4c 5µ 0 r 2 (ρ K ) + c κ 10µ 0 r 2 (ρ K ) ≤ (σ * -χ) + 1 σ * - 1 χ (σ * 2 + γ 1 ) + Υ 1 χ Q 3/4,K 2ζ(f -f * ) - 4c 5µ 0 r 2 (ρ K ) + c κ 10µ 0 r 2 (ρ K ).
Chapter 3. Robust-to-outliers simultaneous inference and noise level estimation using a MOM approach 74

Applying part 2 of Lemma 3.5, we get

Q 3/4,K 2ζ(f -f * ) ≤ α 1 ≤ 2 r 2 (ρ K ). Therefore, T K,µ (g, χ, f * , σ * ) ≤ (σ * -χ) + 1 σ * - 1 χ (σ * 2 + γ 1 ) + Υ 2 χ - 4c 5µ 0 r 2 (ρ K ) + c κ 10µ 0 r 2 (ρ K ) ≤ (σ * -χ) + σ * 2 + γ 1 σ * + 1 χ -σ * 2 -γ 1 + 2Υ r 2 (ρ K ) -Υ 4c 5µ 0 r 2 (ρ K ) + c κ 10µ 0 r 2 (ρ K )
The minimization is done on χ > σ min , therefore we have χ -1 ≤ σ -1 min , and we get

T K,µ (g, χ, f * , σ * ) ≤ (σ * -χ) + σ * 2 + γ 1 σ * + -σ * 2 -γ 1 χ + 2Υ r 2 (ρ K ) σ min -Υ 4c 5µ 0 r 2 (ρ K ) + c κ 10µ 0 r 2 (ρ K ) ≤ (σ * -χ) + σ * 2 + γ 1 σ * + -σ * 2 -γ 1 χ + 2 σ min - 4c 5µ 0 Υr 2 (ρ K ) + c κ 10µ 0 r 2 (ρ K )
Using the inequalities µ 0 ≤ 4c σ min /5, and Υ > κ, we get

T K,µ (g, χ, f * , σ * ) ≤ (σ * -χ) + σ * 2 + γ 1 σ * + -σ * 2 -γ 1 χ + 2 σ min - 4c 5µ 0 κr 2 (ρ K ) + c κ 10µ 0 r 2 (ρ K ) ≤ (σ * -χ) + σ * 2 + γ 1 σ * + -σ * 2 -γ 1 χ + 2 σ min - 7c 10µ 0 κr 2 (ρ K )
The function χ → -χ -a/χ is increasing for χ ≤ √ a, and σ * -α 2,κ ≤ √ a, where here a = σ * 2 + γ 1 .

Therefore, the maximum of the former function is attained for χ = σ * -α 2,κ , which yields

T K,µ (g, χ, f * , σ * ) ≤ α 2,κ + (σ * 2 + γ 1 ) 1 σ * - 1 σ * -α 2,κ + 2 σ min - 7c 10µ 0 κr 2 (ρ K ) ≤ σ * 2 + γ 1 σ * 2 -1 α 2,κ + 2 σ min - 7c 10µ 0 κr 2 (ρ K ).
3.6.9.2 Case ||f -

f * || L 2 P > r(ρ K )
We have ||f -f * || = ρ K , therefore, it follows from parts 1 and 2 of Lemma 3.5,

M OM K l f * 1 σ * - 1 χ + Υ χ 2ζ(f -f * ) -(f -f * ) 2 ≤ Υ χ Q 7/8,K 2ζ(f -f * ) -Q 1/4,K (f -f * ) 2 - 1 σ * - 1 χ Q 1/8,K -ζ 2 ≤ Υ χ α 1 -(4θ 0 ) -2 ||f -f * || 2 L2 + 1 σ * - 1 χ (σ * 2 + γ 1 ). ≤ Υ χ 2 -(4θ 0 ) -2 ||f -f * || 2 L2 + 1 σ * - 1 χ (σ * 2 + γ 1 ).
Plugging this back in Equation 3.22, we get

T K,µ (g, χ, f * , σ * ) ≤ (σ * -χ) + Υ χ 2 -(4θ 0 ) -2 ||f -f * || 2 L2 + 1 σ * - 1 χ (σ * 2 + γ 1 ) -µΥ sup z * ∈Γ f * (ρ K ) z * (f -f * ) + µκρ K 10 .
As 2 -(4θ 0 ) -2 ≤ 0, the function χ → -χ -a/χ is increasing for χ ≤ √ a, and σ * -α 2,κ ≤ √ a, where here, a = σ * 2 + γ 1 -2 + (4θ 0 ) -2 , we get

T K,µ (g, χ, f * , σ * ) ≤ α 2,κ + Υ σ -α 2,κ 2 -(4θ 0 ) -2 ||f -f * || 2 L2 + σ * 2 + γ 1 σ * 2 α 2,κ -µΥ sup z * ∈Γ f * (ρ K ) z * (f -f * ) + µκρ K 10 ≤ 2 + γ 1 σ * 2 α 2,κ + Υ 2 -(4θ 0 ) -2 r 2 (ρ K ) σ -α 2,κ + µρ K + µκρ K
Chapter 3. Robust-to-outliers simultaneous inference and noise level estimation using a MOM approach 75

≤ 2 + γ 1 σ * 2 α 2,κ + Υ 2 -(4θ 0 ) -2 σ -α 2,κ + c µ 0 r 2 (ρ K ) + c κ r 2 (ρ K ) 10µ 0 ≤ 2 + γ 1 σ * 2 α 2,κ + κ 2 -(4θ 0 ) -2 σ -α 2,κ + 11c 10µ 0 r 2 (ρ K ).

Proofs of main results

Proof of Theorem 3.4

We begin the proof by applying Lemmas 3.5 and 3.9, and on the rest of the proof, we will reason on the set Ω(K) := Ω 1 (K) ∩ Ω 2 (K). By Lemmas 3.5 and 3.9, IP Ω(K) ≥ 1 -4 exp(-K/4320)exp(-5K/13824) ≥ 1 -5 exp(-K/4320). Applying the definition of the estimators in Equation (3.4), we have on Ω(K)

C K,µ ( f , σ) ≤ C K,µ (f * , σ * ) = sup g∈F, χ>σmin T K,µ (g, χ, f * , σ * ) ≤ max i∈[5] sup (g,χ)∈F (1) i T K,µ (g, χ, f * , σ * ) ≤ max i∈[5] B i,1 ≤ B 1,1 ,
where the last inequality follows by combining Lemma 3.11 (for κ = 1) and the first part of Lemma 3.12.

The proof is completed by application of the following lemma.

R( f ) ≤ R(f * ) + 1 + 2 + σ * 2 + γ 1 σ * 2 r 2 (2ρ K ) + (σ * + r(2ρ K ))B 1,1 + (σ * + α 2,2 )(α 2,2 + 2µρ K ). Proof: For any x ∈ R K , Q 1/2 (x) ≥ -Q 1/2 (-x). As a consequence, B 1,1 ≥ C K,µ ( f , σ) = sup g∈F, χ>σmin T K,µ (g, χ, f , σ) ≥ T K,µ (f * , σ * , f , σ) = -T K,µ ( f , σ, f * , σ * ).
We deduce that on

Ω(K), ( f , σ) ∈ {(g, χ) ∈ F × R * + : T K,µ (g, χ, f * , σ * ) ≥ -B 1,1 }.
Applying the second part of Lemma 3.12, we have -B 1,1 > sup i=2,...9 B i,2 and, combining this with Lemma 3.11 (for κ = 2),

we get that ( f , σ) ∈ F (2) 1 . By definition of F (2) 1 , we have || f -f * || ≤ 2ρ K , || f -f * || L2 ≤ r(2ρ K ), and |σ -σ * | ≤ α 2,2 , as claimed.
Finally, we prove the control on the excess risk. We apply Lemma 3.7 with ρ := 2ρ K and α

σ := α 2,2 R( f ) -R(f * ) = || f -f * || 2 L2 + P [-2ζ(f -f * )] ≤ r 2 (2ρ K ) + (σ * + r(2ρ K ))T K,λ (f * , σ * , f , σ) + 2 • max r 2 M (ρ, γ M ) , 384θ 2 m 2 K n , ||f -f * || L2 + (σ * 2 + γ 1 ) r(2ρ K ) σ * 2 + (σ * + α 2,2 )(α 2,2 + 2µρ K ).
We bound T K,λ (f * , σ * , f , σ) by C K,λ ( f , σ), and r 2 M (ρ, γ M ) ,

384θ 2 m 2 K n , ||f -f * || L2 by r 2 (2ρ K ). R( f ) -R(f * ) ≤ 1 + 2 + σ * 2 + γ 1 σ * 2 r 2 (2ρ K ) + (σ * + r(2ρ K ))C K,λ ( f , σ) + (σ * + α 2,2 )(α 2,2 + 2µρ K )
Chapter 3. Robust-to-outliers simultaneous inference and noise level estimation using a MOM approach 76

≤ 1 + 2 + σ * 2 + γ 1 σ * 2 r 2 (2ρ K ) + (σ * + r(2ρ K ))B 1,1 + (σ * + α 2,2 )(α 2,2 + 2µρ K ).

Proof of Theorem 3.1

We will apply Theorem 3.4. Following [91, pages 39-40] and with q 0 = 4 (in their notation), it is possible to choose the function r(•) and ρ such that

ρ * = c L,1 ||ζ|| L 4 s 1 n log ed s and r 2 (ρ * ) = c L,2 ||ζ|| 2 L 4 s n log ed s ,
where c L,1 , c L,2 are constants depending on C 1 . Therefore, we can see that, with c 1 := c c L,2 /c L,1 and := 1/(833θ 2 0 ), our choice of µ satisfies

µ := c 1 1 n log ed s = c µ 0 r 2 (ρ * ) ρ * ,
where

µ 0 := ||ζ|| L 4 . Note that µ 0 = IE[ζ 4 ] 1/4 = κ * 1/4 σ * .
Therefore, conditions in Theorem 3.4 are satisfied, which give the bounds

| β -β * | 1 ≤ 2ρ * ∼ c L,1 ||ζ|| L 4 s 1 n log ed s , | β -β * | 2 ≤ 2r 2 (ρ * ) ∼ c L,2 ||ζ|| 2 L 4 s n log ed s .
The results follows by application of the norm interpolation inequality.

| β -β * | p ≤ | β -β * | -1+2/p 1 | β -β * | 2-2/p 2 ||ζ|| L 4 s 1/p 1 n log ed s .

Introduction

In statistical modelling and applied science more generally, it is very common to distinguish two subsets of variables: a random vector of interest (also called explained/exogenous variables) and a vector of covariates (explanatory/endogenous variables). The objective is to predict the law of the former vector given the latter vector belongs to some subset, possibly a singleton. This basic idea constitutes the first step towards forecasting some important statistical sub-products as conditional means, quantiles, volatilities, etc. Formally, consider a d-dimensional random vector X. We are faced with two random sub-vectors X I and X J , s.t. X = (X I , X J ), I ∪ J = {1, . . . , d}, I ∩ J = ∅, and our models of interest specify the conditional law of X I knowing X J = x J or knowing X J ∈ A J for some subset A J ⊂ R |J| .

We use the standard notation for vectors: for any set of indices I, x I means the |I|-dimensional vector whose arguments are the x k , k ∈ I. For convenience and without a loss of generality, we will set I = {1, . . . , p} and J = {p + 1, . . . , d}.

Besides, the problem of dependence among the components of d-dimensional random vectors has been extensively studied in the academic literature and among practitioners in a lot of different fields.

The raise of copulas for more than twenty years illustrates the need of flexible and realistic multivariate Chapter 4. About tests of the "simplifying" assumption for conditional copulas 80 models and tools. When covariates are present and with our notation, the challenge is to study the dependence among the components of X I given X J . Logically, the concept of conditional copulas has emerged. First introduced for pointwise (atomic) conditioning events by Patton ([111,[START_REF] Patton | Modelling asymmetric exchange rate dependence[END_REF]), the definition has been generalized in [START_REF] Fermanian | Time-dependent copulas[END_REF] for arbitrary measurable conditioning subsets. In this paper, we rely on the following definition: for any borel subset

A J ⊂ R d-p , a conditional copula of X I given (X J ∈ A J ) is denoted by C A J I|J (•|X J ∈ A J )
. This is the cdf of the random vector (F 1|J (X 1 |X J ∈ A J ), . . . , F p|J (X p |X J ∈ A J )) given (X J ∈ A J ). Here, F k|J (•|X J ∈ A J ) denotes the conditional law of X k knowing X J ∈ A J , k = 1, . . . , p. The latter conditional distributions will be assumed continuous in this paper, implying the existence and uniqueness of C A J I|J (Sklar's theorem). In other words, for any

x I ∈ R p , IP (X I ≤ x I |X J ∈ A J ) = C A J I|J F 1|J (x 1 |X J ∈ A J ), . . . , F p|J (x p |X J ∈ A J ) X J ∈ A J .
Note that the influence of A J on C A J I|J is twofold: when A J changes, the conditioning event (X J ∈ A J ) changes, but the conditioned random vector

(F 1|J (X 1 |X J ∈ A J ), . . . , F p|J (X p |X J ∈ A J )) changes too.
In particular, when the conditioning events are reduced to singletons, we get that the conditional copula of X I knowing

X J = x J is a cdf C I|J (•|X J = x J ) on [0, 1] p s.t., for every x I ∈ R p , IP (X I ≤ x I |X J = x J ) = C I|J F 1|J (x 1 |X J = x J ), . . . , F p|J (x p |X J = x J ) | X J = x J .
With generalized inverse functions, an equivalent definition of a conditional copula is as follows:

C I|J (u I | X J = x J ) = F I|J F - 1|J (u 1 |X J = x J ), . . . , F - p|J (u p |X J = x J )|X J = x J ,
for every u I and x J , setting

F I|J (x I |X J = x J ) := IP (X I ≤ x I |X J = x J ).
Most often, the dependence of C I|J (•|X J = x J ) w.r.t. to x J is a source of significant complexities, in terms of model specification and inference. Therefore, most authors assume that the following "simplifying assumption" is fulfilled.

Simplifying assumption (H 0 ): the conditional copula C I|J (•|X J = x J ) does not depend on x J , i.e., for every u I ∈ [0, 1] p , the function

x J ∈ R d-p → C I|J (u I |X J = x J ) is a constant function (that depends on u I ).
Under the simplifying assumption, we will set C I|J (u I |X J = x J ) =: C s,I|J (u I ). The latter identity means that the dependence on X J across the components of X I is passing only through their conditional margins. Note that C s,I|J is different from the usual copula of

X I : C I (•) is always the cdf of the vector (F 1 (X 1 ), . . . , F p (X p )) whereas, under H 0 , C s,I|J is the cdf of the vector Z I|J := (F 1|J (X 1 |X J ), . . . , F p|J (X p |X J )) (see Proposition 4.4 below).
Note that the latter copula is identical to the partial copula introduced by Bergsma [START_REF] Bergsma | Nonparametric testing of conditional independence by means of the partial copula[END_REF], and recently studied in [START_REF] Gijbels | Partial and average copulas and association measures[END_REF][START_REF] Spanhel | The partial vine copula: A dependence measure and approximation based on the simplifying assumption[END_REF] in particular. Such a partial copula is always be defined (whether H 0 is satisfied or not) as the cdf of Z I|J . Note that it is

equal to R d-p C I|J (u I |X J = x J )dP J (x J ).
Remark 4.1. The simplifying assumption H 0 does not imply that C s,I|J (•) is C I (•), the usual copula of X I . This can be checked with a simple example: let X = (X 1 , X 2 , X 3 ) be a trivariate random vector s.t., given X 3 , X 1 ∼ N (X 3 , 1) and X 2 ∼ N (X 3 , 1). Moreover, X 1 and X 2 are independent given X 3 . The latter variable may be N (0, 1), to fix the ideas. Obviously, with our notation, I = {1, 2}, J = {3}, d = 3

and p = 2. Therefore, for any couple (u 1 , u 2 ) ∈ [0, 1] 2 and any real number

x 3 , C 1,2|3 (u 1 , u 2 |x 3 ) = u 1 u 2
and does not depend on x 3 . Assumption H 0 is then satisfied. But the copula of (X 1 , X 2 ) is not the independence copula, simply because X 1 and X 2 are not independent.

Chapter 4. About tests of the "simplifying" assumption for conditional copulas 81 Basically, it is far from obvious to specify and estimate relevant conditional copula models in practice, especially when the conditioning and/or conditioned variables are numerous. The simplifying assumption is particularly relevant with vine models (see [1], among others). Indeed, to build vines from a d-dimensional random vector X, it is necessary to consider sequences of conditional bivariate copulas C I|J , where I = {i 1 , i 2 } is a couple of indices in {1, . . . , d}, J ⊂ {1, . . . , d}, I ∩ J = ∅, and (i 1 , i 2 |J) is a node of the vine. In other words, a bivariate conditional copula is needed at every node of any vine, and the sizes of the conditioning subsets of variables are increasing along the vine. Without additional assumptions, the modelling task becomes rapidly very cumbersome (inference and estimation by maximum likelihood). Therefore, most authors adopt the simplifying assumption H 0 at every node of the vine. Note that the curse of dimensionality still apparently remains because conditional marginal cdfs

F k|J (•|X J )
are invoked with different subsets J of increasing sizes. But this curse can be avoided by calling recursively the non-parametric copulas that have been estimated before (see [START_REF] Nagler | Evading the curse of dimensionality in nonparametric density estimation with simplified vine copulas[END_REF]).

Nonetheless, the simplifying assumption has appeared to be rather restrictive, even if it may be seen as acceptable for practical reasons and in particular situations. The debate between pro and cons of the simplifying assumption is still largely open, particularly when it is called in some vine models. On one side, [START_REF] Hobaek Haff | On the simplified pair-copula construction-simply useful or too simplistic?[END_REF] affirms that this simplifying assumption is not only required for fast, flexible, and robust inference, but that it provides "a rather good approximation, even when the simplifying assumption is far from being fulfilled by the actual model". On the other side, [START_REF] Acar | Beyond simplified pair-copula constructions[END_REF] maintain that "this view is too optimistic".

They propose a visual test of H 0 when d = 3 and in a parametric framework. Their technique was based on local linear approximations and sequential likelihood maximizations. They illustrate the limitations of H 0 by simulation and through real datasets. They note that "an uncritical use of the simplifying assumption may be misleading". Nonetheless, they do not provide formal test procedures. Beside, [4] have proposed a formal likelihood test of the simplifying assumption but when the conditional marginal distributions are known, a rather restrictive situation. Some authors have exhibited classes of parametric distributions for which H 0 is satisfied: see [START_REF] Hobaek Haff | On the simplified pair-copula construction-simply useful or too simplistic?[END_REF], significantly extended by [START_REF] Stoeber | Simplified pair copula constructions-limitations and extensions[END_REF]. Nonetheless, such families are rather strongly constrained. Therefore, these two papers propose to approximate some conditional copula models by others for which the simplifying assumption is true. This idea has been developed in [START_REF] Spanhel | The partial vine copula: A dependence measure and approximation based on the simplifying assumption[END_REF] in a vine framework, because they recognize that "it is very unlikely that the unknown data generating process satisfies the simplifying assumption in a strict mathematical sense."

Therefore, there is a need for formal universal tests of the simplifying assumption. It is likely that the latter assumption is acceptable in some circumstances, whereas it is too rough in others. This means, for given subsets of indices I and J, we would like to test

H 0 : C I|J (•|X J = x J ) does not depend on x J ,
against that opposite assumption. Hereafter, we will propose several test statistics of H 0 , possibly assuming that the conditional copula belongs to some parametric family.

Note that several papers have already proposed estimators of conditional copula. [START_REF] Veraverbeke | Estimation of a conditional copula and association measures[END_REF], [START_REF] Gijbels | Conditional copulas, association measures and their applications[END_REF] and [START_REF] Fermanian | Time-dependent copulas[END_REF] have studied some nonparametric kernel based estimators. [START_REF] Craiu | In mixed company: Bayesian inference for bivariate conditional copula models with discrete and continuous outcomes[END_REF], [START_REF] Sabeti | Additive models for conditional copulas[END_REF] studied bayesian additive models of conditional copulas. Recently, [START_REF] Schellhase | Estimating non-simplified vine copulas using penalized splines[END_REF] invoke B-splines to manage vectors of conditioning variables. In a semiparametric framework, i.e. assuming an underlying parametric family of conditional copulas, numerous models and estimators have been proposed, notably [3], [2], [START_REF] Fermanian | Single-index copulas[END_REF] (single-index type models), [START_REF] Vatter | Generalized additive models for conditional dependence structures[END_REF] (additive models), among others. But only a few of these papers have a focus on testing the simplifying assumption H 0 specifically, although convergence of the proposed estimators are necessary to lead such a task in theory. Actually, some tests of H 0 is invoked "in passing" in these papers as potential applications, but without a general approach and/or without some guidelines to evaluate pvalues in practice. As exceptions, in very recent papers, [START_REF] Gijbels | Nonparametric testing for no covariate effects in conditional copulas[END_REF] have tackled the simplifying assumption directly through comparisons between conditional and unconditional Kendall's tau. Moreover, [START_REF] Kurz | Testing the simplifying assumption in high-dimensional vine copulas[END_REF] have proposed tests of the latter assumption for vine models.

Example 4.2. To illustrate the problem, let us consider a simple example of H 0 in dimension 3. Assume that p = 2 and d = 3. For simplicity, let us assume that (X 1 , X 2 ) follows a Gaussian distribution conditionally on X 3 , that is :

X 1 X 2 X 3 = x 3 ∼ N µ 1 (x 3 ) µ 2 (x 3 ) , σ 2 1 (x 3 ) ρ(x 3 )σ 1 (x 3 )σ 2 (x 3 ) ρ(x 3 )σ 1 (x 3 )σ 2 (x 3 ) σ 2 2 (x 3 ) . (4.1)
Obviously, α(•) := (µ 1 , µ 2 , σ 1 , σ 2 )(•)
is a parameter that only affects the conditional margins. Moreover, the conditional copula of (X 1 , X 2 ) given X 3 = x 3 is gaussian with the parameter ρ(x 3 ). Six possible cases can then be distinguished:

a. All variables are mutually independent. b. (X 1 , X 2 ) is independent of X 3 , but X 1 and X 2 are not independent.
c. X 1 and X 2 are both marginally independent of X 3 , but the conditional copula of X 1 and X 2 depends on X 3 .

d. X 1 (or X 2 ) and X 3 are not independent but X 1 and X 2 are independent conditionally given X 3 .

e. X 1 (or X 2 ) and X 3 are not independent but the conditional copula of X 1 and X 2 is independent of

X 3 .
f. X 1 (or X 2 ) and X 3 are not independent and the conditional copula of X 1 and X 2 is dependent of

X 3 .
These six cases are summarized in the following table:

ρ(•) = 0 ρ(•) = ρ 0 ρ(•) is not constant α(•) = α 0 a b c α(•) is not constant d e f
In the conditional Gaussian model (4.1), the simplifying assumption H 0 consists in assuming that we live in one of the cases {a, b, d, e}, whereas the alternative cases are c and f . In this model, the conditional copula is entirely determined by the conditional correlation. Note that, in some other models, the conditional correlation can vary only because of the conditional margins, while the conditioning copula stay constant: see [START_REF] Spanhel | The partial vine copula: A dependence measure and approximation based on the simplifying assumption[END_REF].

Note that, in general, there is no reason why the conditional margins would be constant in the conditioning variable (and in most applications, they are not). Nevertheless, if we knew the marginal cdfs'

were constant with respect to the conditioning variable, then the test of H 0 (i.e. b against c) would become a classical test of independence between X I and X J .

Testing H 0 is closely linked to the m-sample copula problem, for which we have m different and independent samples of a p-dimensional variable X I = (X 1 , . . . , X p ). In each sample k, the observations are i.i.d., with their own marginal laws and their own copula C I,k . The m-sample copula problem consists on testing whether the m latter copulas C I,k are equal. Note that we could merge all samples into a single one, and create discrete variables Y i that are equal to k when i lies in the sample k. Therefore, the m-sample copula problem is formally equivalent to testing H 0 with the conditioning variable X J := Y .

Conversely, assume we have defined a partition {A 1,J , . . . , A m,J } of R d-p composed of borelian subsets such that IP(X J ∈ A k,J ) > 0 for all k = 1, . . . , m, and we want to test

H 0 : k ∈ {1, . . . , m} → C A k,J I|J ( • |X J ∈ A k,J ) does not depend on k.
Then, divide the sample in m different sub-samples, where any sub-sample k contains the observations for which the conditioning variable belongs to A k,J . Then, H 0 is equivalent to a m-sample copula problem. Note that H 0 looks like a "consequence" of H 0 when it is not the case in general (see Section 4.3.1), for continuous X J variables.

Nonetheless, H 0 conveys the same intuition as H 0 . Since it can be led more easily in practice (no smoothing is required), some researchers could prefer the former assumption than the latter. That is why it will be discussed hereafter. Note that the 2-sample copula problem has already been addressed by [START_REF] Rémillard | Testing for equality between two copulas[END_REF], and the m-sample by [START_REF] Bouzebda | K-sample problem using strong approximations of empirical copula processes[END_REF]. However, both paper are designed only in a nonparametric framework, and these authors have not noticed the connection with the simplifying assumption.

The goal of the paper is threefold: first, to write a "state-of-the art" of the simplifying assumption problem; second to propose some "reasonable" test statistics of the simplifying assumption in different contexts; third, to introduce a new approach of the latter problem, through "box-related" zero assumptions and some associated test statistics. Since it is impossible to state the theoretical properties of all these test statistics, we will rely on "ad-hoc arguments" to convince the reader they are relevant, without trying to establish specific results. Globally, this paper can be considered also as a work program around the simplifying assumption H 0 for the next years.

In Section 4.

Tests of the simplifying assumption

"Brute-force" tests of the simplifying assumption

A first natural idea is to build a test of H 0 based on a comparison between some estimates of the conditional copula C I|J with and without the simplifying assumption, for different conditioning events. Such estimates will be called ĈI|J and Ĉs,I|J respectively. Then, introducing some distance D between conditional distributions, a test can be based on the statistics D( ĈI|J , Ĉs,I|J ). Following most authors, we immediately think of Kolmogorov-Smirnov-type statistics

T 0 KS,n := ĈI|J -Ĉs,I|J ∞ = sup u I ∈[0,1] p sup x J ∈R d-p | ĈI|J (u I |x J ) -Ĉs,I|J (u I )|, (4.2)
or Cramer von-Mises-type test statistics

T 0 CvM,n := ĈI|J (u I |x J ) -Ĉs,I|J (u I ) 2 w(du I , dx J ), (4.3)
for some weight function of bounded variation w, that could be chosen as random (see below).

To evaluate ĈI|J , we propose to invoke the nonparametric estimator of conditional copulas proposed by [START_REF] Fermanian | Time-dependent copulas[END_REF]. Alternative kernel-based estimators of conditional copulas can be found in [START_REF] Gijbels | Conditional copulas, association measures and their applications[END_REF], for instance.

Let us start with an iid d-dimensional sample (X i ) i=1,...,n . Let Fk be the marginal empirical distribution function of X k , based on the sample (X 1,k , . . . , X n,k ), for any k = 1, . . . , d. Our estimator of C I|J will be defined as

ĈI|J (u I |X J = x J ) := FI|J F - 1|J (u 1 |X J = x J ), . . . , F - p|J (u p |X J = x J )|X J = x J , FI|J (x I |X J = x J ) := 1 n n i=1 K n (X i,J , x J )1(X i,I ≤ x I ), (4.4)
where

K n (X i,J , x J ) := K h Fp+1 (X i,p+1 ) -Fp+1 (x p+1 ), . . . , Fd (X i,d ) -Fd (x d ) , K h (x J ) := h -(d-p) K (x p+1 /h, . . . , x d /h) ,
and K is a (d -p)-dimensional kernel. Obviously, for k ∈ I, we have introduced some estimates of the marginal conditional cdfs' similarly:

Fk|J (x|X J = x J ) := n i=1 K n (X i,J , x J )1(X i,I ≤ x I ) n j=1 K n (X j,J , x J ) • (4.5)
Obviously, h = h(n) is the term of a usual bandwidth sequence, where h(n) → 0 when n tends to the infinity. Since FI|J is a nearest-neighbors estimator, it does not necessitate a fine-tuning of local bandwidths (except for those values x J s.t. F J (x J ) is close to one or zero), contrary to more usual Nadaraya-Watson techniques. In other terms, a single convenient choice of h would provide "satisfying" estimates of ĈI|J (x I |X J = x J ) for most values of x. For practical reasons, it is important that Fk|J (x k |x J ) belongs to [0, 1] and that Fk|J (•|x J ) is a true distribution. This is the reason why we use a normalized version for the estimator of the conditional marginal cdfs.

To calculate the latter statistics (4.2) and (4.3), it is necessary to provide an estimate of the underlying conditional copula under H 0 . This could be done naively by particularizing a point x * J ∈ R d-p and by setting

Ĉ(1) s,I|J (•) := ĈI|J (•|X J = x * J ). Since the choice of x * J is too arbitrary, an alternative could be to set Ĉ(2) s,I|J (•) := ĈI|J (•|X J = x J ) w(dx J ),
for some function w that is of bounded variation, and w(dx J ) = 1. Unfortunately, the latter choice induce (d -p)-dimensional integration procedures, that becomes a numerical problem rapidly when d -p is larger than three.

Therefore, let us randomize the "weight" functions w, to avoid multiple integrations. For instance, choose the empirical distribution of X J as w, providing

Ĉ(3) s,I|J (•) := ĈI|J (•|X J = x J ) FJ (dx J ) = 1 n n i=1 ĈI|J (•|X J = X i,J ). (4.6)
An even simpler estimate of C s,I|J , the conditional copula of X I given X J under the simplifying assumption, can be obtained by noting that, under H 0 , C s,I|J is the joint law of

Z I|J := (F 1 (X 1 |X J ), . . . , F p (X p |X J ))
(see Property 4.4 below). Therefore, it is tempting to estimate C s,I|J (u I ) by

Ĉ(4) s,I|J (u I ) := 1 n n i=1 1( F1|J (X i,1 |X i,J ) ≤ u 1 , . . . , Fp|J (X i,p |X i,J ) ≤ u p , (4.7
)

when u I ∈ [0, 1] p , for some consistent estimates Fk|J (x k |x J ) of F k|J (x k |x J ).
A similar estimator has been promoted and studied in [START_REF] Gijbels | Estimation of a copula when a covariate affects only marginal distributions[END_REF] or in [START_REF] Portier | On the weak convergence of the empirical conditional copula under a simplifying assumption[END_REF], but they have considered the empirical copula associated to the pseudo sample (( F1 (X i1 |X iJ ), . . . , Fp (X ip |X iJ ))) i=1,...,n instead of its empirical cdf. It will be called Ĉ( 5) s,I|J . Hereafter, we will denote Ĉs,I|J one of the "averaged" estimators Ĉ(k) s,I|J , k > 1 and we can forget the naive pointwise estimator Ĉ(1) s,I|J . Therefore, under some conditions of regularity, we guess that our estimators Ĉs,I|J (u I ) of the conditional copula under H 0 will be √ n-consistent and asymptotically normal. It has been proved for C

s,I|J in [START_REF] Gijbels | Estimation of a copula when a covariate affects only marginal distributions[END_REF] or in [START_REF] Portier | On the weak convergence of the empirical conditional copula under a simplifying assumption[END_REF], as a byproduct of the weak convergence of the associated process.

Under H 0 , we would like that the previous test statistics T 0 KS,n or T 0 CvM,n are convergent. Typically, such a property is given as a sub-product by the weak convergence of a relevant empirical process, here

(u I , x J ) ∈ [0, 1] p × R d-p → nh d-p n ( ĈI|J -C I|J )(u I |x J ).
Unfortunately, this will not be the case in general seing the previous process as a function indexed by x J , at least for wide ranges of bandwidths.

Due to the difficulty of checking the tightness of the process indexed by x J , some alternative techniques may be required as Gaussian approximations (see [START_REF] Chernozhukov | Gaussian approximation of suprema of empirical processes[END_REF], e.g.). Nonetheless, they would lead us far beyond the scope of this paper. Therefore, we simply propose to slightly modify the latter test statistics, to manage only a fixed set of arguments x J . For instance, in the case of the Kolmogorov-Smirnov-type test, consider a simple grid χ J := {x 1,J , . . . , x m,J }, and the modified test statistics

T 0,m KS,n := sup u I ∈[0,1] p sup x J ∈χ J | ĈI|J (u I |x J ) -Ĉs,I|J (u I )|.
In the case of the Cramer von-Mises-type test, we can approximate any integral by finite sums, possibly after a change of variable to manage a compactly supported integrand. Actually, this is how they are calculated in practice! For instance, invoking Gaussian quadratures, the modified statistics would be

T 0,m CvM,n := m j=1 ω j ĈI|J (u j,I |x j,J ) -Ĉs,I|J (u j,I ) 2 , (4.8)
for some conveniently chosen constants ω j , j = 1, . . . , m. Note that the numerical evaluation of ĈI|J is relatively costly. Since quadrature techniques require a lot less points m than "brute-force" equally spaced grids (in dimension d, here), they have to be preferred most often.

Therefore, at least for such modified test statistics, we can insure the tests are convergent. Indeed, under some conditions of regularity, it can be proved that ĈI|J (u I |X J = x J ) is consistent and asymptotically normal, for every choice of u I and x J (see [START_REF] Fermanian | Time-dependent copulas[END_REF]). And a relatively straightforward extension of their Corollary 1 would provide that, under H 0 and for all U := (u I,1 , . . . , u I,q ) ∈ [0, 1] p(q+r) and

X := (x J,1 , . . . , x J,q ) ∈ R (d-p)q , nh d-p n ( ĈI|J -C s,I|J )(u I,1 |X J = x J,1 ), . . . , nh d-p n ( ĈI|J -C s,I|J )(u I,q |X J = x J,q ), √ n( Ĉs,I|J -C s,I|J )(u I,q+1 ), . . . , √ n( Ĉs,I|J -C s,I|J )(u I,q+r ) ,
converges in law towards a Gaussian random vector. As a consequence, nh d-p n T 0,m KS,n and nh d-p n T 0,m CvM,n tends to a complex but not degenerate law under the H 0 .

Remark 4.3. Other test statistics of H 0 can be obtained by comparing directly the functions ĈI|J (•|X J =

x J ), for different values of x J . For instance, let us define

T 0 KS,n := sup x J ,x J ∈R d-p ĈI|J (•|x J ) -ĈI|J (•|x J ) ∞ = sup x J ,x J ∈R d-p sup u I ∈[0,1] p | ĈI|J (u I |x J ) -ĈI|J (u I |x J )|, (4.9
)

or T 0 CvM,n := ĈI|J (u I |x J ) -ĈI|J (u I |x J ) 2 w(du I , dx J , dx J ), (4.10)
for some function of bounded variation w. As above, modified versions of these statistics can be obtained considering fixed x J -grids. Since these statistics involve higher dimensional integrals/sums than previously, they will not be studied more in depth.

The L 2 -type statistics T 0 CvM,n and T 0 CvM,n involve at least d summations or integrals, which can become numerically expensive when the dimension of X is "large". Nonetheless, we are free to set convenient weight functions. To reduce the computational cost, several versions of T 0 CvM,n are particularly well-suited, by choosing conveniently the functions w. For instance, consider

T (1) CvM,n := ĈI|J (u I |x J ) -Ĉs,I|J (u I ) 2 ĈI (du I ) FJ (dx J ),
where FJ and ĈI denote the empirical cdf of (X i,J ) and the empirical copula of (X i,I ) respectively.

Therefore, T

CvM,n simply becomes

T (1) CvM,n = 1 n 2 n j=1 n i=1 ĈI|J ( Ûi,I |X J = X j,J ) -Ĉs,I|J ( Ûi,I ) 2 , (4.11)
where Ûi,I = ( F1 (X i,1 ), . . . , Fp (X i,p )), i = 1, . . . , n. Similarly, we can choose

T (1) CvM,n := ĈI|J (u I |x J ) -ĈI|J (u I |x J ) 2 ĈI (du I ) FJ (dx J ) FJ (dx J ) = 1 n 3 n j=1 n j =1 n i=1 ĈI|J ( Ûi,I |X J = X j,J ) -ĈI|J ( Ûi,I |X J = X j ,J ) 2 .
To deal with a single summations only, it is even possible to propose to set

T (2) CvM,n := ĈI|J ( F1|J (x 1 |x J ), . . . , Fp|J (x p |x J )|x J ) -Ĉs,I|J ( F1|J (x 1 |x J ), . . . , Fp|J (x p |x J )) 2 F (dx I , dx J ),
where F denotes the empirical cdf of X. This means

T (2) CvM,n = 1 n n i=1 ĈI|J F1|J (X i,1 |X i,J ), . . . , Fp|J (X i,p |X i,J )|X J = X i,J -Ĉs,I|J F1|J (X i,1 |X i,J ), . . . , Fp|J (X i,p |X i,J ) 2 .
We have introduced some tests based on comparisons between empirical cdfs'. Obviously, the same idea could be applied to associated densities, as in [START_REF] Fermanian | Goodness-of-fit tests for copulas[END_REF] for instance, or even to other functions of the underlying distributions.

Since the previous test statistics are complicated functionals of some "semi-smoothed" empirical process, it is very challenging to evaluate their asymptotic laws under H 0 analytically. In every case, these limiting laws will not be distribution free, and their calculation would be very tedious. Therefore, as usual with copulas, it is necessary to evaluate the limiting distributions of such tests statistics by a convenient bootstrap procedure (parametric or nonparametric). These bootstrap techniques will be presented in Section 4.2.4.

Tests based on the independence property

Actually, testing H 0 is equivalent to a test of the independence between the random vectors X J and

Z I|J := (F 1 (X 1 |X J ), . . . , F p (X p |X J ))
strictly speaking, as proved in the following proposition.

Proposition 4.4. The vectors Z I|J and X J are independent iff C I|J (u I |X J = x J ) does not depend on

x J for every vectors u I and x J . In this case, the cdf of Z I|J is C s,I|J .

Proof: For any vectors u I ∈ [0, 1] p and any subset

A J ⊂ R d-p , IP(Z I|J ≤ u I , X J ∈ A J ) = IE 1((X J ∈ A J )IP(Z I|J ≤ u I |X J ) = 1((x J ∈ A J )IP(Z I|J ≤ u I |X J = x J ) dIP X J (x J ) = A J IP(F k (X k |X J = x J ) ≤ u k , ∀k ∈ I|X J = x J ) dIP X J (x J ) = A J C I|J (u I |X J = x J ) dIP X J (x J ).
If Z I|J and X J are independent, then

IP(Z I|J ≤ u I )IP(X J ∈ A J ) = 1((x J ∈ A J )C I|J (u I |X J = x J ) dIP X J (x J ),
for every u I and A J . This implies IP(Z I|J ≤ u I ) = C I|J (u I |X J = x J ) for every u I ∈ [0, 1] p and every x J in the support of X J . This means that C I|J (u I |X J = x J ) does not depend on x J , because Z I|J does not depend on any x J by definition.

Reciprocally, under H 0 , C s,I|J is the cdf of Z I|J . Indeed,

IP(Z I|J ≤ u I ) = IP (F k (X k |X J ) ≤ u k , ∀k ∈ I) = IP (F k (X k |X J = x J ) ≤ u k , ∀k ∈ I| X J = x J ) dIP X J (x J ) = C I|J (u I |X J = x J ) dIP X J (x J ) = C s,I|J (u I ) dIP X J (x J ) = C s,I|J (u I ).
Moreover, due to Sklar's Theorem, we have

IP(Z I|J ≤ u I , X J ∈ A J ) = 1(x J ∈ A J )C I|J (u I |X J = x J ) dIP X J (x J ) = 1(x J ∈ A J )C s,I|J (u I ) dIP X J (x J ) = IP(Z I|J ≤ u I )IP(X J ∈ A J ),
implying the independence between Z I|J and X J .

Chapter 4. About tests of the "simplifying" assumption for conditional copulas 88

Then, testing H 0 is formally equivalent to testing

H * 0 : Z I|J = (F 1 (X 1 |X J ), . . . , F p (X p |X J )
) and X J are independent.

Since the conditional marginal cdfs' are not observable, keep in mind that we have to work with pseudo-observations in practice, i.e. vectors of observations that are not independent. In other words, our tests of independence should be based on pseudo-samples

F1|J (X i,1 |X i,J ), . . . , Fp|J (X i,p |X i,J ) i=1,...,n := ( Ẑi,I|J ) i=1,...,n , (4.12)
for some consistent estimate Fk|J (•|X J ), k ∈ I of the conditional cdfs', for example as defined in Equation (4.5). The chance of getting distribution-free asymptotic statistics will be very tiny, and we will have to rely on some bootstrap techniques again. To summarize, we should be able to apply some usual tests of independence, but replacing iid observations with (dependent) pseudo-observations.

Most of the tests of H * 0 rely on the joint law of (Z I|J , X J ), that may be evaluated empirically as

G I,J (x I , x J ) := IP(Z I|J ≤ x I , X J ≤ x J ) ĜI,J (x) := n -1 n i=1 1( Ẑi,I|J ≤ x I , X i,J ≤ x J ).
Now, let us propose some classical strategies to build independence tests.

• Chi-square-type tests of independence: Let B 1 , . . . , B N (resp. A 1 , . . . , A m ) some disjoint subsets in R p (resp. R d-p ). I χ,n = n N k=1 m l=1 ĜI,J (B k × A l ) -ĜI,J (B k × R d-p ) ĜI,J (R p × A l ) 2 ĜI,J (B k × R d-p ) ĜI,J (R p × A l ) • (4.13)
• Distance between distributions:

I KS,n = sup x∈R d | ĜI,J (x) -ĜI,J (x I , ∞ d-p ) ĜI,J (∞ p , x J )|, or (4.14)
I 2,n = ĜI,J (x) -ĜI,J (x I , ∞ d-p ) ĜI,J (∞ p , x J ) 2 ω(x) dx, (4.15)
for some (possibly random) weight function ω. Particularly, we can propose the single sum

I CvM,n = ĜI,J (x) -ĜI,J (x I , ∞ d-p ) ĜI,J (∞ p , x J ) 2 ĜI,J (dx) = 1 n n i=1 ĜI,J ( Ẑi,I|J , X i,J ) -ĜI,J ( Ẑi,I|J , ∞ d-p ) ĜI,J (∞ p , X i,J ) 2 . (4.16)
• Tests of independence based on comparisons of copulas: let CI,J and ĈJ be the empirical copulas based on the pseudo-sample ( Ẑi,I|J , X i,J ) i=1,...,n , and (X i,J ) i=1,...,n respectively. Set

ȊKS,n = sup u∈[0,1] d | CI,J (u) - Ĉ(k) s,I|J (u I ) ĈJ (u J )|, k = 1, . . . , 5, or Ȋ2,n = u∈[0,1] d CI,J (u) - Ĉ(k) s,I|J (u I ) ĈJ (u J ) 2 ω(u) du,
and in particular

ȊCvM,n = u∈[0,1] d CI,J (u) - Ĉ(k) s,I|J (u I ) ĈJ (u J ) 2 CI,J (du).
The underlying ideas of the test statistics ȊKS,n and ȊCvM,n are similar to those that have been proposed by Deheuvels ([33], [START_REF] Deheuvels | A kolmogorov-smirnov type test for independence and multivariate samples[END_REF]) in the case of unconditional copulas. Nonetheless, in our case, we have to calculate pseudo-samples of the pseudo-observations ( Ẑi,I|J ) and (X i,J ), instead of a usual pseudo-sample of (X i ).

Note that the latter techniques require the evaluation of some conditional distributions, for instance by kernel smoothing. Therefore, the level of numerical complexity of these test statistics of H * 0 is comparable with those we have proposed before to test H 0 directly.

Parametric tests of the simplifying assumption

In practice, modelers often assume a priori that the underlying copulas belong to some specified para-

metric family C := {C θ , θ ∈ Θ ⊂ R m }.
Let us adapt our tests under this parametric assumption. Apparently, we would like to test

Ȟ0 : C I|J (•|X J ) = C θ (•), for some θ ∈ Θ and almost every X J .
Actually, Ȟ0 requires two different things: the fact that the conditional copula is a constant copula w.r.t. its conditioning events (test of H 0 ) and, additionally, that the right copula belongs to C (classical composite Goodness-of-Fit test). Under this point of view, we would have to adapt "omnibus" specification tests to manage conditional copulas and pseudo observations. For instance, and among of alternatives, we could consider an amended version of Andrews's ( [START_REF] Andrews | A conditional Kolmogorov test[END_REF]) specification test

CK n := 1 √ n max j≤n | n i=1 1( Ẑi,I|J ≤ Ẑj,I|J ) -C θ0 ( Ẑj,I|J ) 1(X i,J ≤ X j,J )|,
recalling the notation in (4.12). For other ideas of the same type, see [START_REF] Zheng | A consistent test of conditional parametric distributions[END_REF] and the references therein.

The latter global approach is probably too demanding. Here, we prefer to isolate the initial problem that was related to the simplifying assumption only. Therefore, let us assume that, for every x J , there

exists a parameter θ(x J ) such that C I|J (•|x J ) = C θ(x J ) (•).
To simplify, we assume the function θ(•) is continuous. Our problem is then reduced to testing the constancy of θ, i.e.

H c 0 : the function x J → θ(x J ) is a constant, called θ 0 .
For every x J , assume we estimate θ(x J ) consistently. For instance, this can be done by modifying the standard semiparametric Canonical Maximum Likelihood methodology ( [START_REF] Genest | A semiparametric estimation procedure of dependence parameters in multivariate families of distributions[END_REF][START_REF] Tsukahara | Semiparametric estimation in copula models[END_REF]): set

θ(x J ) := arg max θ∈Θ n i=1 log c θ F1|J (X i,1 |X J = X i,J ), . . . , Fp|J (X i,p |X J = X i,J ) • K n (X i,J , x J ), through usual kernel smoothing in R d-p , where c θ (u) := ∂ p C θ (u)/∂u 1 • • • ∂u p for θ ∈ Θ and u ∈ [0, 1] p .
Alternatively, we could consider

θ(x J ) := arg max θ∈Θ n i=1 log c θ F1|J (X i,1 |X J = x J ), . . . , Fp|J (X i,p |X J = x J ) • K n (X i,J , x J ),
instead of θ(x J ). See [2] concerning the theoretical properties of θ(x J ) and some choice of conditional cdfs'. Those of θ(x J ) remain to be stated precisely, to the best of our knowledge. But there is no doubt both methodologies provide consistent estimators, even jointly, under some conditions of regularity.

Chapter 4. About tests of the "simplifying" assumption for conditional copulas 90 Under H c 0 , the natural "unconditional" copula parameter θ 0 of the copula of the Z I|J will be estimated by

θ0 := arg max θ∈Θ n i=1 log c θ F1|J (X i,1 |X i,J ), . . . , Fp|J (X i,p |X i,J ) . (4.17)
Surprisingly, the theoretical properties of the latter estimator do not seem to have been established in the literature explicitly. Nonetheless, the latter M-estimator is a particular case of those considered in [START_REF] Fermanian | Single-index copulas[END_REF] in the framework of single-index models when the link function is a known function (that does not depend on the index). Therefore, by adapting their assumption in the current framework, we easily obtain that θ0 is consistent and asymptotically normal if c θ is sufficiently regular, for convenient choices of bandwidths and kernels. Now, there are some challengers to test H c 0 :

• Tests based on the comparison between θ(•) and θ0 :

T c ∞ := sup x J ∈R d-p θ(x J ) -θ0 , or T c 2 := θ(x J ) -θ0 2 ω(x J ) dx J , (4.18)
for some weight function ω.

• Tests based on the comparison between C θ(•) and C θ0 :

T c dist := dist C θ(x J ) , C θ0 ω(x J ) dx J , (4.19)
for some distance dist(•, •) between cdfs'.

• Tests based on the comparison between copula densities (when they exist):

T c dens := c θ(x J ) (u I ) -c θ0 (u I ) 2 ω(u I , x J ) du I dx J . (4.20)
Remark 4.5. It might be difficult to compute some of these integrals numerically, because of unbounded supports. One solution is to to make change of variables. For example,

T c 2 = θ(F - J (u J )) -θ0 2 ω(F - J (u J )) du J f J (F - J (u J ))
• Therefore, the choice ω = f J allows us to simplify the latter statistics to θ(F - J (u J )) -θ0 2 du J , which is rather easy to evaluate. We used this trick in the numerical section below.

Bootstrap techniques for tests of H 0

It is necessary to evaluate the limiting laws of the latter test statistics under the null. As a matter of fact, we generally cannot exhibit explicit -and distribution-free a fortiori -expressions for these limiting laws.

The common technique is provided by bootstrap resampling schemes.

More precisely, let us consider a general statistics T , built from the initial sample S := (X 1 , . . . , X n ).

The main idea of the bootstrap is to construct N new samples S * := (X * 1 , . . . , X * n ) following a given resampling scheme given S. Then, for each bootstrap sample S * , we will evaluate a bootstrapped test statistics T * , and the empirical law of all these N statistics is used as an approximation of the limiting law of the initial statistic T .

Some resampling schemes

The first natural idea is to invoke Efron's usual "nonparametric bootstrap", where we draw independently with replacement X * i for i = 1, . . . , n among the initial sample S = (X 1 , . . . , X n ). This provides a bootstrap sample S * := (X * 1 , . . . , X * n ).

The nonparametric bootstrap is an "omnibus" procedure whose theoretical properties are well-known but that may not be particularly adapted to the problem at hand. Therefore, we will propose alternative sampling schemes that should be of interest, even if we do not state their validity on the theoretical basis.

Such a task is left for further researches.

An natural idea would be to use some properties of X under H 0 , in particular the characterization given in Proposition 4.4: under H 0 , we known that Z i,I|J and X i,J are independent. This will be only relevant for the tests of Subsection 4.2.2, and for a few tests of Subsection 4.2.1, where such statistics are based on the pseudo-sample ( Ẑi,I|J , X i,J ) i=1,...,n . Therefore, we propose the following so-called "pseudo-independent bootstrap" scheme:

Repeat, for i = 1 to n, 1. draw X * i,J among (X j,J ) j=1,...,n ; 2. draw Ẑ * i,I|J independently, among the observations Ẑj,I|J , j = 1, . . . , n.
This provides a bootstrap sample

S * := ( Ẑ * 1,I|J , X * 1,J ), . . . , ( Ẑ * n,I|J , X * n,J ) .
Note that we could invoke the same idea, but with a usual nonparametric bootstrap perspective: draw with replacement a n-sample among the pseudo-observations ( Ẑi,I|J , X i,J ) i=1,...,n for each bootstrap sample. This can be called a "pseudo-nonparametric bootstrap" scheme.

Moreover, note that we cannot draw independently X * i,J among (X j,J ) j=1,...,n , and beside X * i,I

among (X j,I ) j=1,...,n independently. Indeed, H 0 does not imply the independence between X I and X J . At the opposite, it makes sense to build a "conditional bootstrap" as follows:

Repeat, for i = 1 to n, 1. draw X * i,J among (X j,J ) j=1,...,n ; 2. draw X * i,I independently, along the estimated conditional law of X I given X J = X * i,J . This can be down by drawing a realization along the law FI|J (•|X J = X * i,J ), for instance (see (4.4)). This can be done easily because the latter law is purely discrete, with unequal weights that depend on X * i,J and S.

This provides a bootstrap sample S

* := ( X * 1,I , X * 1,J ), . . . , ( X * n,I , X * n,J ) .
Remark 4.6. Note that the latter way of resampling is not far from the usual nonparametric bootstrap.

Indeed, when the bandwidths tend to zero, once x * J = X i,J is drawn, the procedure above will select the other components of X i (or close values), i.e. the probability that x * I = X i,I is "high".

In the parametric framework, we might also want to use an appropriate resampling scheme. As a matter of fact, all the previous resampling schemes can be used, as in the nonparametric framework, but we would not take advantage of the parametric hypothesis, i.e. the fact that all conditional copulas belong to a known family. We have also to keep in mind that even if the conditional copula has a Chapter 4. About tests of the "simplifying" assumption for conditional copulas 92 parametric form, the global model is not fully parametric, because we have not provided a parametric model neither for the conditional marginal cdfs F k|J , k = 1, . . . , p, nor for the cdf of X J .

Therefore, we can invoke the null hypothesis H c 0 and approximate the real copula C θ0 of Z I|J by C θ0 . This leads us to define the following "parametric independent bootstrap":

Repeat, for i = 1 to n, 1. draw X * i,J among (X j,J ) j=1,...,n ; 2. sample Z * i,I|J, θ0
from the copula with parameter θ0 independently.

This provides a bootstrap sample

S * := (Z * 1,I|J, θ0 , X * 1,J ), . . . , (Z * n,I|J, θ0
, X * n,J ) .

Remark 4.7. At first sight, this might seem like a strange mixing of parametric and nonparametric bootstrap. If |J| = 1, we can nonetheless do a "full parametric bootstrap", by observing that all estimators of our previous test statistics do not depend on X J , but on realizations of FJ (X J ) (see Equations (4.4) and (4.5)). Since the law of latter variable is close to a uniform distribution, it is tempting to sample

V * i,J ∼ U [0,1
] at the first stage, i = 1, . . . , n, and then to replace FJ (X i,J ) with V * i,J to get an alternative bootstrap sample.

Without using H c 0 , we could define the "parametric conditional bootstrap" as:

Repeat, for i = 1 to n,

• draw X * i,J among (X j,J ) j=1,...,n ;

• sample Z * i,I|J,θ * i from the copula with parameter θ(X * i,J ).

This provides a bootstrap sample S

* := (Z * 1,I|J,θ * i , X * 1,J ), . . . , (Z * n,I|J,θ * i , X * n,J ) .
Note that, in several resampling schemes, we should be able to keep the same X J as in the original sample, and simulate only Z * i,I|J in step 2, as in [START_REF] Andrews | A conditional Kolmogorov test[END_REF], pages 10-11. Such an idea has been proposed by [START_REF] Omelka | Bootstrapping the conditional copula[END_REF], in a slightly different framework and univariate conditioning variables. They proved that such a bootstrap scheme "works", after a fine-tuning of different smoothing parameters: see their Theorem 1.

Bootstrapped test statistics

The problem is now to evaluate the law of a given test statistic, say T , under H 0 by the some bootstrap techniques. We recall the main technique in the case of the classical nonparametric bootstrap. We conjecture that the idea is still theoretically sound under the other resampling schemes that have been proposed in Subsection 4.2.4.1.

The principle for the nonparametric bootstrap is based on the weak convergence of the underlying empirical process. Formally, if

S := {X 1 , . . . , X n } in an iid sample in R d , X ∼ F and if F n denotes its empirical distribution, it is well-known that √ n (F n -F ) tends weakly in ∞ towards a d-dimensional
Brownian bridge B F . And the nonparametric bootstrap works in the sense that √ n (F * n -F n ) converges weakly towards a process B F , an independent version of B F , given the initial sample S.

Due to the Delta Method, for every Hadamard

-differentiable functional χ from ∞ (R d ) to R, there exists a random variable H χ s.t. √ n (χ(F n ) -χ(F )) ⇒ H χ .
Assume a test statistics T n of H 0 can be written as a sufficiently regular functional of the underlying empirical process as

T n := ψ √ n (χ s (F n ) -χ(F n )) ,
Chapter 4. About tests of the "simplifying" assumption for conditional copulas 93 where χ s (F ) = χ(F ) under the null assumption. Then, under H 0 , we can rewrite this expression as

T n := ψ √ n (χ s (F n ) -χ s (F ) + χ(F ) -χ(F n )) . (4.21)
Given any bootstrap sample S * and the associated empirical distribution F * n , the usual bootstrap equivalent of T n is

T * n := ψ √ n (χ s (F * n ) -χ s (F n ) + χ(F n ) -χ(F * n )) ,
from Equation (4.21). See [START_REF] Van Der | Weak convergence[END_REF], Section 3.9, for details and mathematically sound statements.

Applying these ideas, we can guess the bootstrapped statistics corresponding to the tests statistics of H 0 , at least when the usual nonparametric bootstrap is invoked.

Let us illustrate the idea with

T 0 KM,n . Note that ĈI|J (•|X J = •) = χ KM (F n )(•) and Ĉs,I|J = χ s,KM (F n ) for some smoothed functional χ KM and χ s,KM . Under H 0 , χ KM = χ s,KM and T 0 KS,n := χ KM (F n ) - χ KM (F ) -χ s,KM (F n ) + χ s,KM (F ) ∞ . Therefore, its bootstrapped version is T 0, * KS,n := χ KM (F * n ) -χ KM (F n ) -χ s,KM (F * n ) + χ s,KM (F n ) ∞ = Ĉ * I|J -ĈI|J -Ĉ * s,I|J + Ĉs,I|J ∞ .
Obviously, the functions Ĉ * I|J and Ĉ * s,I|J have been calculated as ĈI|J and Ĉs,I|J respectively, but replacing S by S * . Similarly, the bootstrapped versions of some Cramer von-Mises-type test statistics are

T 0, * CvM,n := Ĉ * I|J (u I |x J ) -ĈI|J (u I |x J ) -Ĉ * s,I|J (u I ) + Ĉs,I|J (u I ) 2 w(du I , dx J ).
When playing with the weight functions w, it is possible to keep the same weights for the bootstrapped versions, or to replace them with some functionals of F * n . For instance, asymptotically, it is equivalent to consider

T (1), * CvM,n := Ĉ * I|J (u I |x J ) -ĈI|J (u I |x J ) -Ĉ * s,I|J (u I ) + Ĉs,I|J (u I ) 2 Ĉn (du I ) FJ (dx J ), or T (1), * CvM,n := Ĉ * I|J (u I |x J ) -ĈI|J (u I |x J ) -Ĉ * s,I|J (u I ) + Ĉs,I|J (u I ) 2 Ĉ * n (du I ) F * J (dx J ).
Similarly, the limiting law of

T (2), * CvM,n := Ĉ * I|J ( F * n,1 (x 1 |x J ), . . . , F * n,p (x p |x J )|x J ) -ĈI|J ( F * n,1 (x 1 |x J ), . . . , F * n,p (x p |x J )|x J ) -Ĉ * s,I|J ( F * n,1 (x 1 |x J ), . . . , F * n,p (x p |x J )) + Ĉs,I|J ( F * n,1 (x 1 |x J ), . . . , F * n,p (x p |x J )) 2 H n (dx I , dx J ), given F n is unchanged replacing H n by H * n .
The same ideas apply concerning the tests of Subsection 4.2.2, but they require some modifications.

Let H be some cdf on R d . Denote by H I and H J the associated cdf on the first p and d -p components respectively. Denote by Ĥ, ĤI and ĤJ their empirical counterparts. Under H 0 , and for any measurable subsets B I and A J , H(B I × A J ) = H(B I )H(A J ). Our tests will be based on the difference

Ĥ(B I × A J ) -ĤI (B I ) ĤJ (A J ) = ( Ĥ -H)(B I × A J ) -( ĤI -H I )(B I ) ĤJ (A J ) -( ĤJ -H J )(A J )H I (B I ).
Therefore, a bootstrapped approximation of the latter quantity will be

( Ĥ * -Ĥ)(B I × A J ) -( Ĥ * I -ĤI )(B I ) Ĥ * J (A J ) -( Ĥ * J -ĤJ )(A J ) ĤI (B I ).
To be specific, the bootstrapped versions of our tests are specified as below.

Chapter 4. About tests of the "simplifying" assumption for conditional copulas 94

• Chi-square-type test of independence:

I * χ,n := n N k=1 m l=1 1 Ĝ * I,J (B k × R d-p ) Ĝ * I,J (R p × A l ) ( Ĝ * I,J -ĜI,J )(B k × A l ) -Ĝ * I,J (B k × R d-p ) Ĝ * I,J (R p × A l ) + ĜI,J (B k × R d-p ) ĜI,J (R p × A l ) 2 .
• Distance between distributions:

I * KS,n = sup x∈R d |( Ĝ * I,J -ĜI,J )(x) -Ĝ * I,J (x I , ∞ d-p ) Ĝ * I,J (∞ p , x J ) + ĜI,J (x I , ∞ d-p ) ĜI,J (∞ p , x J )| I * 2,n = ( Ĝ * I,J -ĜI,J )(x)-Ĝ * I,J (x I , ∞ d-p ) Ĝ * I,J (∞ p , x J )+ ĜI,J (x I , ∞ d-p ) ĜI,J (∞ p , x J ) 2 ω(x) dx,
and I * CvM,n is obtained replacing ω(x) dx by Ĝ * I,J (dx) (or even ĜI,J (dx)).

• A test of independence based on the independence copula: Let C * I,J , C * I|J and Ĉ * J be the empirical copulas based on a bootstrapped version of the pseudo-sample ( Ẑi,I|J , X i,J ) i=1,...,n , ( Ẑi,I|J ) i=1,...,n and (X i,J ) i=1,...,n respectively. This version can be obtained by nonparametric bootstrap, as usual, providing new vectors Ẑ * i,I|J at every draw. The associated bootstrapped statistics are

Ȋ * KS,n = sup u∈[0,1] d |( C * I,J -CI,J )(u) -C * I|J (u I ) Ĉ * J (u J ) + CI|J (u I ) ĈJ (u J )|, Ȋ * 2,n = u∈[0,1] d ( C * I,J -CI,J )(u) -C * I|J (u I ) Ĉ * J (u J ) + CI|J (u I ) ĈJ (u J ) 2 ω(u) du, Ȋ * CvM,n = u∈[0,1] d ( C * I,J -CI,J )(u) -C * I|J (u I ) Ĉ * J (u J ) + CI|J (u I ) ĈJ (u J ) 2 C * I,J (du).
In the case of the parametric statistics, the situation is pretty much the same, as long as we invoke the nonparametric bootstrap. For instance, the bootstrapped versions of some previous test statistics are

(T c 2 ) * := θ * (x J ) -θ(x J ) -θ * 0 + θ0 2 ω(x J ) dx J , or (T c dens ) * := c θ * (x J ) (u I ) -c θ(x J ) (u I ) -c θ * 0 (u I ) + c θ0 (u I ) 2 ω(u I , x J )du I dx J .
in the case of the nonparametric bootstrap. We conjecture that the previous techniques can be applied with the other resampling schemes that have been proposed in Subsection 4.2.4.1. Nonetheless, a complete theoretical study of all these alternative schemes and the statement of the validity of their associated bootstrapped statistics is beyond the scope of this paper.

Remark 4.8. For the "parametric independent" bootstrap scheme, we have observed that the test powers are a lot better by considering

(T c 2 ) * * := θ * (x J ) -θ * 0 2 ω(x J ) dx J , or (T c dens ) * * := c θ * (x J ) (u I ) -c θ * 0 (u I ) 2 ω(u I , x J )du I dx J ,
instead. The relevance of such statistics may be theoretically justified in the slightly different context of "box-type" tests in the next Section (see Theorem 4.14). Since our present case is close to the situation of "many small boxes", it is not surprising that we observe similar features. Note that, contrary to the nonparametric bootstrap or the "parametric conditional" bootstrap, the "parametric independent" bootstrap scheme uses H 0 . More generally, and following the same idea, we found that using the statistic

T * * := ψ ( √ n (χ s (F * n ) -χ(F * n )))
for the pseudo-independent bootstrap yields much better performance than T * . In our simulations, we will therefore use T * * as the bootstrap test statistic (see Figures 4.1 and 4.2). Remark 4.9. In a vine model, every node is associated with a bivariate conditional copula, and it is desirable that they satisfy H 0 . Unfortunately, the arguments of such copulas are defined through conditional distributions F i (X i |X K ) for some subsets K ⊂ {1, . . . , d}. Therefore, we do not observe realizations of such arguments, except for the first level. In practice, they have to be replaced with pseudo-observations in our previous test statistics. Their calculation involves the bivariate conditional copulas that are associated with the previous nodes in a recursive way. The theoretical analysis of the associated bootstrap schemes is challenging and falls beyond the scope of the current work.

Tests with "boxes"

The link with the simplifying assumption

As we have seen in Remark 4.1, we do not have C s,I|J = C I in general. This is the hint there are some subtle relations between conditional copulas when the conditioning event is pointwise or when it is a measurable subset. Actually, to test H 0 in Section 4.2, we have relied on kernel estimates and smoothing parameters, at least to evaluate conditional marginal distributions empirically. To avoid the curse of dimension (when d -p is "large" i.e. larger than three in practice), it is tempting to replace the pointwise conditioning events X J = x J with X J ∈ A J for some borelian subsets A J ⊂ R d-p , IP(X J ∈ A J ) > 0. As a shorthand notation, we shall write A J the set of all such A J . We call them "boxes" because choosing d -p-dimensional rectangles (i.e. intersections of half-spaces separated by orthogonal hyperplans) is natural, but our definitions are still valid for arbitrary borelian subsets in R d-p . Technically speaking, we will assume that the functions x J → 1(x J ∈ A J ) are Donsker, to apply uniform CLTs' without any hurdle. Actually, working with X J -"boxes" instead of pointwise will simplify a lot the picture. Indeed, the evaluation of conditional cdfs' given X J ∈ A J does not require kernel smoothing, bandwidth choices, or other techniques of curve estimation that deteriorate the optimal rates of convergence.

Note that, by definition of the conditional copula of X I given (X J ∈ A J ), we have

IP(X I ≤ x I |X J ∈ A J ) = C A J I|J (IP(X 1 ≤ x 1 |X J ∈ A J ), . . . , IP(X p ≤ x p |X J ∈ A J )|X J ∈ A J ) ,
for every point x I ∈ R p and every subset A J in A J . So, it is tempting to replace H 0 by

H 0 : C A J I|J (u I |X J ∈ A J ) does not depend on A J ∈ A J , for any u I .
For any x J , consider a sequence of boxes (A

(n) J (x J )) s.t. ∩ n A (n) J (x J ) = {x J }. If the law of X is sufficiently regular, then lim n C A (n) J I|J (u I |X J ∈ A (n) J ) = C I|J (u I |X J = x J )
for any u I . Therefore, H0 implies H 0 . This is stated formally in the next proposition.

Proposition 4.10. Assume that the function

h : R d → [0, 1], defined by h(y) := IP(X I ≤ y I |X J = y J ) is continuous everywhere. Let x J ∈ R d-p such that F i|J (•|x J ) is strictly increasing for every i = 1, . . . , d.

Then, for any sequence of boxes (A

(n) J (x J )) such that ∩ n A (n) J (x J ) = {x J }, we have lim n C A (n) J (x J ) I|J (u I |X J ∈ A (n) J (x J )) = C I|J (u I |X J = x J ), for every u I ∈ [0, 1] p .
Proof: Consider a particular u I ∈ [0, 1] p . If one component of u I is zero, the result is obviously satisfied. If one component of u I is one, this component does not play any role. Therefore, we can restrict ourselves on u I ∈ (0, 1) p . By continuity, there exists

x I ∈ R p s.t. u i = F i|J (x i |x J ) for every i = 1, . . . , p. Let the sequences (x (n) i ) such that u i = F i|J (x (n) i |X J ∈ A (n)
J ) for every n and every i = 1, . . . , p. First, let us show that x (n) i → x i when n tends to the infinity. Indeed, by the definition of conditional probabilities ( [START_REF] Shiryaev | Probability[END_REF], p.220), we have

u i = IP(X i ≤ x (n) i |X J ∈ A (n) J ) = 1 IP(X J ∈ A (n) J ) {y J ∈A (n) J } IP(X i ≤ x (n) i |X J = y J ) dIP X J (y J ),
and

u i = IP(X i ≤ x i |X J = x J ) = 1 IP(X J ∈ A (n) J ) {y J ∈A (n) J } IP(X i ≤ x (n) i |X J = x J ) dIP X J (y J ) + IP(X i ≤ x i |X J = x J ) -IP(X i ≤ x (n) i |X J = x J ).
By substracting the two latter identities, we deduce

1 IP(X J ∈ A (n) J ) {y J ∈A (n) J } IP(X i ≤ x (n) i |X J = y J ) -IP(X i ≤ x (n) i |X J = x J ) dIP X J (y J ) = IP(X i ≤ x i |X J = x J ) -IP(X i ≤ x (n) i |X J = x J ). (4.22)
But, by assumption, F i|J (t|y J ) tends towards F i|J (t|x J ) when y J tends to x J , for any t (pointwise convergence). Actually, the latter convergence is uniform on R:

F i|J (•|y J ) -F i|J (•|x J ) ∞ tends to
zero when y J → x J . This is a straightforward consequence of Pólya's Theorem (also called second Dini's Theorem in the literature): see Subsection (A.1) in [START_REF] Bickel | Uniform convergence of probability measures on classes of functions[END_REF] for instance. From (4.22), we deduce that

IP(X i ≤ x (n) i |X J = x J ) → IP(X i ≤ x i |X J = x J ). By the continuity of F i|J (•|x J ), we get x (n) i → x i , for any i = 1, . . . , p.
Second, let us come back to conditional copulas: setting x

(n) I := (x (n) 1 , . . . , x (n)
p ), we have

C A (n) J I|J (u I |A (n) J ) -C I|J (u I |x J ) = C A (n) J I|J (F 1|J (x (n) 1 |A (n) J ), . . . , F p|J (x (n) p |A (n) J )|A (n) J ) -C I|J (F 1|J (x 1 |x J ), . . . , F p|J (x p |x J )|x J ) = F I|J (x (n) I |A (n) J ) -F I|J (x I |x J ) = 1 IP(X J ∈ A (n) J ) {y J ∈A (n) J } IP(X I ≤ x (n) I |X J = y J ) -IP(X I ≤ x I |X J = x J ) dIP X J (y J ).

Since x

(n) I tends to x I when n → ∞ and invoking the continuity of h at (x I , x J ), we get C

A (n) J I|J (u I |A (n) J ) → C I|J (u I |x J ) when n → ∞.
Unfortunately, the opposite is false. Counter-intuitively, H0 does not lead to a consistent test of the simplifying assumption. Indeed, under H 0 , we can see that

C A J I|J (u I |X J ∈ A J ) depends on A J in general, even if C I|J (u I |X J = x J ) does not depend on x J !
Chapter 4. About tests of the "simplifying" assumption for conditional copulas 97 This is due to the nonlinear transform between conditional (univariate and multivariate) distributions and conditional copulas. In other words, for a usual d-dimensional cdf H, we have

H(x I |X J ∈ A J ) = 1 IP(A J ) A J H(x I |X J = x J ) dIP X J (x J ), (4.23)
for every measurable subset A J ∈ A J and x I ∈ R p . At the opposite and in general, for conditional copulas,

C A J I|J (u I |X J ∈ A J ) = 1 IP(A J ) A J C I|J (u I |X J = x J ) dIP X J (x J ), (4.24)
for u I ∈ [0, 1] p . And even if we assume H 0 , we have in general,

C A J I|J (u I |X J ∈ A J ) = 1 IP(A J ) A J C s,I|J (u I ) dIP X J (x J ) = C s,I|J (u I ). (4.25)
As a particular case, taking

A J = R d-p , this means again that C I (u I ) = C s,I|J (u I ).
Let us check this rather surprising feature with the example of Remark 4.1 for another subset A J .

Recall that H 0 is true and that

C s,1,2|3 (u, v) = uv for every u, v ∈ [0, 1]. Consider the subset (X 3 ≤ a),
for any real number a. The probability of this event is Φ(a). Now, let us verify that

uv = H(F - 1|3 (u|X 3 ≤ a), F - 2|3 (v|X 3 ≤ a)|X 3 ≤ a),
for some u, v in (0, 1). Clearly, for every real number x k , we have

IP(X k ≤ x k |X 3 ≤ a) = 1 Φ(a) a -∞ Φ(x k -z)φ(z) dz, k = 1, 2, and IP(X 1 ≤ x 1 , X 2 ≤ x 2 |X 3 ≤ a) = 1 Φ(a) a -∞ Φ(x 1 -z)Φ(x 2 -z)φ(z) dz.
In particular, IP(X k ≤ 0|X 3 ≤ a) = (1 + Φ(-a))/2. Therefore, set u * = v * = (1 + Φ(-a))/2 and we get

H(F - 1|3 (u * |X 3 ≤ a), F - 2|3 (v * |X 3 ≤ a)|X 3 ≤ a) = H(0, 0|X 3 ≤ a) = 1 3 1 + Φ(-a) + Φ 2 (-a) = u * v * . In this example, C s,1,2|3 (•) = C ]-∞,a] 1,2|3 (•|X 3 ≤ a), for every a, even if H 0 is satisfied.
Nonetheless, getting back to the general case, we can easily provide an equivalent of Equation (4.23) for general conditional copulas, i.e. without assuming H 0 .

Proposition 4.11. For all u I ∈ [0, 1] p and all A J ∈ A J ,

C A J I|J (u I |X J ∈ A J ) = 1 IP(A J ) A J ψ(u I , x J , A J )dIP X J (x J ), with ψ(u I , x J , A J ) = C I|J F 1|J F - 1|J (u 1 |X J ∈ A J ) X J = x J , . . . , F p|J F - p|J (u p |X J ∈ A J ) X J = x J X J = x J .
Proof: From (4.23), we get :

H(x I |X J ∈ A J ) = 1 IP(A J ) A J H(x I |X J = x J ) dIP X J (x J ) = 1 IP(A J ) A J C I|J F 1|J (x 1 |X J = x J ), . . . , F p|J (x p |X J = x J ) X J = x J dIP X J (x J ).
We can conclude by using the following definition of the conditional copula

C A J I|J (u I |X J ∈ A J ) = H(F - 1|J (u 1 |X J ∈ A J ), . . . , F - p|J (u p |X J ∈ A J )|X J ∈ A J ).
Now, we understand why (4.24) (and (4.25) under H 0 ) are not identities: the conditional copulas, given the subset A J , still depend on the conditional margins of X I given X J pointwise in general.

Note that, if X i is independent of X J for every i = 1, . . . , p, then, for any such i,

F i|J F - i|J (u i |X J ∈ A J ) X J = x J = F i F - i (u i ) = u i ,
and we can revisit the identity of Proposition 4.11: under H 0 , we have

C A J I|J (u I |X J ∈ A J ) = 1 IP(A J ) A J C I|J (u I |X J = x J ) dIP X J (x J ) = 1 IP(A J ) A J C s,I|J (u I ) dIP X J (x J ) = C s,I|J (u I ).
This means H 0 and H0 are equivalent. We consider such circumstances as very peculiar and do not have to be confused with a test of H 0 . Therefore, we advise to lead a preliminary test of independence between X I and X J (or at least between X i and X J for any i = 1, . . . , p) before trying to test H 0 itself. Now, let us revisit the characterisation of H 0 in terms of the independence property, as in Subsection 4.2.2. The latter analysis is confirmed by the equivalent of Proposition 4.4 in the case of conditioning subsets A J . Now, the relevant random vector would be

Z I|A J := F 1|J (X 1 |X J ∈ A J ), . . . , F p|J (X p |X J ∈ A J ) ,
that has straightforward empirical counterparts. Then, it is tempting to test

H * 0 : Z I|A J and (X J ∈ A J ) are independent for every borelian subset A J ⊂ R d-p .
Nonetheless, it can be proved easily that this is not a test of H 0 , unfortunately.

Proposition 4.12. Z I|A J and (X J ∈ A J ) are independent for every measurable subset A J ⊂ R d-p iff X I and X J are independent.

Proof: For any measurable subset A J and any u I ∈ [0, 1] p , under H * 0 , we have

IP Z I|A J ≤ u I , X J ∈ A J = IP Z I|A J ≤ u I IP(X J ∈ A J ).
Consider x I ∈ R p . Due to the continuity of the conditional cdfs', there exists

u k s.t. F k (x k |X J ∈ A J ) = u k , k = 1, . . . , p. Then, using the invertibility of x → F k (x|X J ∈ A J ), we get IP Z I|A J ≤ u I , X J ∈ A J = IP (X I ≤ x I , X J ∈ A J )
. This implies that H * 0 is equivalent to the following property: for every x I ∈ R p and A J ,

IP (X I ≤ x I , X J ∈ A J ) = IP (X I ≤ x I ) IP (X J ∈ A J ) .
The previous result shows that a test of H * 0 is a test of independence between X I and X J . When the latter assumption is satisfied, H0 and then H 0 are true too, but the opposite is false.

Previously, we have exhibited a simple trivariate model where H 0 is satisfied when X I and X J are not independent. Then, we see that it is not reasonable to test whether the mapping A J → C A J I|J (•|X J ∈ A J ) is constant over A J , the set of all A J such that IP X J (A J ) > 0, with the idea of testing H 0 .

Nonetheless, one can weaken the latter assumption, and restrict oneself to a finite family A J of subsets with positive probabilities. For such a family, we could test the assumption

H 0 : A J → C A J I|J ( • |X J ∈ A J ) is constant over A J .
To fix the ideas and w.l.o.g., we will consider a given family of disjoint subsets

A J = {A 1,J , . . . , A m,J } in R d-p hereafter.
Note the following consequence of Proposition 4.11.

Proposition 4.13. Assume that, for all A J ∈ A J and for all i ∈ I,

F i|J (x|X J = x J ) = F i|J (x|X J ∈ A J ), ∀x J ∈ A J , x ∈ R. (4.26)
Then, H 0 implies H 0 .

Obviously, if the family A J is too big, then (4.26) will be too demanding: H 0 will be close to a test of independence between X I and X J , and no longer a test of H 0 . Moreover, the chosen subsets in the family A J do not need to be disjoint, even if this would be a natural choice. As a special case, if R d-p ∈ A J , the previous condition is equivalent to the independence between X i and X J for every i ∈ I.

Note that (4.26) does not imply that the vector of explanatory variables X J should be discretized.

Indeed, the full model requires the specification of the underlying conditional copula too, independently of the conditional margins and arbitrarily. For instance, we can choose a Gaussian conditional copula whose parameter is a continuous function of X J , even if (4.26) is fulfilled. And the law of X I given X J will depend on the current value of X J .

A test of H 0 may be relevant in a lot of situations, beside technical arguments as the absence of smoothing. First, the case of discrete (or discretized) explanatory variables X J is frequent. When X J is discrete and takes a value among {x 1,J , . . . , x m,J }, set A k,J = {x k,J }, k = 1, . . . , m. Then, there is identity between testing H 0 and H 0 , with A J = {A 1,J , . . . , A m,J }. Second, the level of precision and sharpness of a copula model is often lower than the models for (conditional) margins. To illustrate this idea, a lot of complex and subtle models to explain the dynamics of asset volatilities are available when the dynamics of cross-assets dependencies are often a lot more basic and without clear-cut empirical findings. Therefore, it makes sense to simplify conditional copula models compared to conditional marginal models. This can be done by considering only a few possible conditional copulas, associated to some events (X J ∈ A k,J ), k = 1, . . . , m. For example, Jondeau and Rockinger [START_REF] Jondeau | The copula-garch model of conditional dependencies: An international stock market application[END_REF] (the first paper that introduced conditional dependence structures, beside Patton [START_REF] Patton | Modelling asymmetric exchange rate dependence[END_REF]) proposed a Gaussian copula parameter that take a finite of values randomly, based on the realizations of some past asset returns.

Third, similar situation occur with most Markov-switching copula models, where a finite set of copulas is managed. In such models, the (unobservable, in general) underlying state of the economy determines the index of the box: see [START_REF] Chollete | Modeling international financial returns with a multivariate regime-switching copula[END_REF], [START_REF] Wang | A revisit to the dependence structure between the stock and foreign exchange markets: A dependence-switching copula approach[END_REF], [START_REF] St | Regime switches in the dependence structure of multidimensional financial data[END_REF], [START_REF] Fink | Regime switching vine copula models for global equity and volatility indices[END_REF], among others.

Therefore, testing H 0 is of interest per se. Even if this is not equivalent to H 0 (i.e. the simplifying assumption) formally, the underlying intuitions are close. And, particularly when the components of the conditioning variable X J are numerous, it can make sense to restrict the information set of the underlying conditional copula to a fixed number of conveniently chosen subsets A J . And the constancy of the underlying copula when X J belongs to such subsets is valuable in a lot of practical situations.

Therefore, in the next subsections, we study some specific tests of H 0 itself.

Non-parametric tests with "boxes"

To specify such tests, we need first to estimate the conditional marginal cdfs', for instance by

Fk|J (x|X J ∈ A J ) := n i=1 1(X i,k ≤ x, X i,J ∈ A J ) n i=1 1(X i,J ∈ A J )
, for every real x and k = 1, . . . , p. Similarly the joint law of X I given (X J ∈ A J ) may be estimated by

FI|J (x I |X J ∈ A J ) := n i=1 1(X i,I ≤ x I , X i,J ∈ A J ) n i=1 1(X i,J ∈ A J )
•

The conditional copula given (X J ∈ A J ) will be estimated by

ĈA J I|J (u I |X J ∈ A J ) := FI|J ( F - 1|J (u 1 |X J ∈ A J ), . . . , F - p|J (u p |X J ∈ A J )|X J ∈ A J ).
Therefore, it is easy to imagine tests of H 0 , for instance

T KS,n := sup u I ∈[0,1] d sup k,l=1,...,m | ĈA k,J I|J (u I |X J ∈ A k,J ) - ĈA l,J I|J (u I |X J ∈ A l,J )|, (4.27)
T CvM,n := m k,l=1 ĈA k,J I|J (u I |X J ∈ A k,J ) - ĈA l,J I|J (u I |X J ∈ A l,J ) 2 w(du I ), (4.28)
for some nonnegative weight functions w, or even

T dist,n := m k,l=1 dist ĈA k,J I|J (•|X J ∈ A k,J ), ĈA l,J I|J (•|X J ∈ A l,J ) , (4.29)
where dist(•, •) denotes a distance between cdfs' on [0, 1] p . More generally, define the matrix

M (A J ) := 1(k = l) dist ĈA k,J I|J (•|X J ∈ A k,J ), ĈA l,J I|J (•|X J ∈ A l,J ) 1≤k,l≤m
, and any statistic of the form || M (A J )|| can be used as a test statistics of H 0 , when || • || is a norm on the set of m × m-matrices. Obviously, it is easy to introduce similar statistics based on copula densities instead of cdfs'.

Parametric test statistics with "boxes"

When we work with subsets A J ∈ R d-p instead of pointwise conditioning events (X J = x J ), we can adapt all the previous parametric test statistics of Subsection 4.2.3. Nonetheless, the framework will be slightly modified.

Let us assume that, for every

A J ∈ A J , C A J I|J (•|X J ∈ A J ) belongs to the same parametric copula family C = {C θ , θ ∈ Θ}. In other words, C A J I|J (•|X J ∈ A J ) = C θ(A J ) (•)
for every A J ∈ A J . Therefore, we could test the constancy of the mapping A J → θ(A J ), i.e. to test

H c 0 : the function k ∈ {1, . . . , m} → θ(A k,J ) is a constant called θ b 0 .
Clearly, for every A J ∈ A J , we can estimate θ(A J ) by

θ(A J ) := arg max θ∈Θ n i=1 log c θ F1|J (X i,1 |X i,J ∈ A J ), . . . , Fp|J (X i,p |X i,J ∈ A J ) 1(X i,J ∈ A J ).
It can be proved that the estimate θ(A J ) is consistent and asymptotically normal, by revisiting the proof of Theorem 1 in [START_REF] Tsukahara | Semiparametric estimation in copula models[END_REF]. Here, the single difference w.r.t. the latter paper is induced by the random sample size, modifying the limiting distributions. The proof is left to the reader.

Under the zero assumption H c 0 , the parameter of the copula of (F 1 (X

1 |X J ∈ A k,J ), . . . , F p (X p |X J ∈ A k,J )) given (X J ∈ A k,J
) is the same for any k = 1, . . . , m. It will be denoted by θ b 0 , and we can still estimate it by the semi-parametric procedure

θb 0 := arg max θ∈Θ m k=1 n i=1 log c θ F1|J (X i,1 |X i,J ∈ A k,J ), . . . , Fp|J (X i,p |X i,J ∈ A k,J ) 1(X i,J ∈ A k,J ).
Obviously, under some conditions of regularity and under H c 0 , it can be proved that θb 0 is consistent and asymptotically normal, by adapting the results of [START_REF] Tsukahara | Semiparametric estimation in copula models[END_REF].

For convenience, let us define the "box index" function k(x J ) := m k=1 k1{x J ∈ A k,J }, for any x J ∈ R d-p . In other words, k is the index of the box A k,J that contains x J . It equals zero, when no box in A J contains x J . Let us introduce the r.v. Y i := k(X i,J ), that stores only all the needed information concerning the conditioning with respect to the variables X i,J . We can then define the empirical pseudoobservations as

Z i,I|Y := m k=1 F 1|J (X i,1 |X J ∈ A k,J ), . . . , F p|J (X i,p |X J ∈ A k,J ) 1{X i,J ∈ A k,J } = F 1|J (X i,1 |X J ∈ A k(X i,J ),J ), . . . , F p|J (X i,p |X J ∈ A k(X i,J ),J ) = F 1|Y (X i,1 |Y i ), . . . , F p|Y (X i,p |Y i ) ,
for any i = 1, . . . , n. Since we do not observe the conditional marginal cdfs', we define the observed pseudo-observations that we calculate in practice: for i = 1, . . . , n,

Ẑi,I|Y := F1|J (X i,1 |X J ∈ A Yi,J ), . . . , Fp|J (X i,p |X J ∈ A Yi,J ) .
Note that we can then rewrite the previous estimators as

θ(A k,J ) = arg max θ∈Θ n i=1 log c θ Ẑi,I|Y 1(Y i = k), and θb 0 = arg max θ∈Θ n i=1 log c θ Ẑi,I|Y .
Now, let us revisit some of the previously proposed test statistics in the case of "boxes".

• Tests based on the comparison between θ(•) and θ0 :

T c ∞ := √ n max k=1,...,m θ(A k,J ) -θ0 , T c 2 := n m k=1 θ(A k,J ) -θ0 2 ω k , (4.30)
for some weights ω k .

• Tests based on the comparison between C θ(•) and C θ0 :

T c dist := m k=1 dist(C θ(A k ) , C θ0 )ω k , (4.31)
and others.

Bootstrap techniques for tests with boxes

In the same way as in the previous section, we will need bootstrap schemes to evaluate the limiting laws of the test statistics of H 0 or H c 0 under the null. All the nonparametric resampling schemes of Subsection 4.2.4.1 (in particular Efron's usual bootstrap) can be used in this framework, replacing the conditional pseudo-observations Ẑi,I|J by Ẑi,I|Y , i = 1, . . . , n. The parametric resampling schemes of Subsection 4.2.4.1 can also be applied to the framework of "boxes", replacing θ0 by θb 0 and θ(x J ) by θ(A J ). In the parametric case, the bootstrapped estimates are denoted by θ * 0 and θ * (A J ). They are the equivalents of θb 0 and θn (A J ), replacing ( Ẑi,I|J , Y i ) by (Z * i , Y * i ). The bootstrapped statistics will also be changed accordingly. Writing them explicitly is a rather straightforward exercise and we do not provide the details, contrary to Subsection 4.2.4. For example, the bootstrapped statistics corresponding to (4.30) is

T c 2 * := n m k=1 θ * (A k,J ) -θ(A k,J ) -θ * 0 + θb 0 2 ω k ,
where θ * 0 is the result of the program arg max θ n i=1 log c θ Ẑ * i,I|Y , in the case of Efron's nonparametric bootstrap.

As we noticed in Remark 4.8, some changes are required when dealing with the "parametric independent" bootstrap. Indeed, under the alternative, we observe θ * (A k,J )-θ * 0 ≈ 0, because we have precisely generated a bootstrap sample under H c 0 . As a consequence, the law of T c 2 * would be close to the law of T c 2 but under the alternative, providing very small powers. Therefore, convenient bootstrapped test statistics of H 0 under the "parametric independent" scheme will be of the type

T c 2 * * := n m k=1 θ * (A k,J ) -θ * 0 2 ω k .
Such a result is justified theoretically by the following theorem. Theorem 4.14. Assume that H c 0 is satisfied, and that we apply the parametric independent bootstrap. Set

Θn, 0 := √ n θ0 -θ 0 , Θ n,k := √ n θ(A k,J ) -θ 0 , k = 1, . . . , m, Θ * n,0 := √ n θ * 0 -θ 0 , and Θ * n,k := √ n θ * (A k,J ) -θ 0 , k = 1, . . . , m.
Then there exists two independent and identically distributed random vectors Θ 0 , . . . , Θ m and Θ ⊥ 0 , . . . , Θ ⊥ m , and a real number a 0 such that

Θ n,0 , . . . , Θ n,m , Θ * n,0 , . . . , Θ * n,m =⇒ Θ 0 , . . . , Θ m , Θ ⊥ 0 + a 0 Θ 0 , . . . , Θ ⊥ m + a 0 Θ 0 .
The proof of this theorem has been postponed in Appendix 4.7.

As a consequence of the latter result, applying the parametric independent bootstrap procedures for some test statistics based on comparisons between θ0 and the θ(A k,J ) is valid. For instance, T c 2 and T c 2 * * will converge jointly in distribution to a pair of independent and identically distributed variables. Indeed, we have

T c 2 , T c 2 * * = n m k=1 θb n,0 -θn (A k,J ) 2 ω k , n m k=1 θ * n,0 -θ * n (A k,J ) 2 ω k = n m k=1 θb n,0 -θ 0 + θ 0 -θn (A k,J ) 2 ω k , n m k=1 θ * n,0 -θ 0 + θ 0 -θ * n (A k,J ) 2 ω k =⇒ m k=1 Θ 0 -Θ k 2 ω k , m k=1 Θ ⊥ 0 + a 0 Θ 0 -Θ ⊥ k -a 0 Θ 0 2 ω k .
The

Numerical applications

Now, we would like to evaluate the empirical performances of some of the previous tests by simulation.

Such an exercise has been led by [START_REF] Genest | Goodness-of-fit tests for copulas: A review and a power study[END_REF] or [START_REF] Berg | Copula goodness-of-fit testing: an overview and power comparison[END_REF] extensively in the case of goodness-of-fit test for unconditional copulas. Our goal is not to replicate such experiments in the case of conditional copulas and for tests of the simplifying assumption. Indeed, we have proposed dozens of test statistics and numerous bootstrap schemes. Moreover, testing the simplifying assumption through H 0 or some "box-type"

problems through H 0 doubles the scale of the task. Finally, in the former case, we depend on smoothing parameters that induce additional degrees of freedom for the fine tuning of the experiments (note that [START_REF] Genest | Goodness-of-fit tests for copulas: A review and a power study[END_REF] and [START_REF] Berg | Copula goodness-of-fit testing: an overview and power comparison[END_REF] have renounced to consider tests that require additional smoothing parameters, as the pivotal test statistics proposed in [START_REF] Fermanian | Goodness-of-fit tests for copulas[END_REF]. In our opinion, an exhaustive simulation experiment should be the topic of (at least) one additional paper. Here, we will restrict ourselves to some partial numerical elements. They should convince readers that the methods and techniques we have discussed previously provide fairly good results and can be implemented in practice safely.

Hereafter, we consider bivariate conditional copulas and a single conditioning variable, i. In terms of model specification, the margins of X = (X 1 , X 2 , X 3 ) will depend on X 3 as

X 1 ∼ N (X 3 , 1), X 2 ∼ N (X 3 , 1
) and X 3 ∼ N (0, 1).

We have studied the following conditional copula families: given

X 3 = x,
• the Gaussian copula model, with a correlation parameter θ(x),

• the Student copula model, with 4 degrees of freedom and a correlation parameter θ(x),

• the Clayton copula model, with a parameter θ(x),

• the Gumbel copula model, with a parameter θ(x),

• the Frank copula model, with a parameter θ(x).

In every case, we calibrate θ(x) such that the conditional Kendall's tau τ (x) satisfies τ (x) = Φ(x)τ max , for some constant τ max ∈ (0, 1). By default, τ max is equal to one. In this case, the random Kendall's tau are uniformly distributed on [0, 1].

Test of H 0 : we calculate the percentage of rejections of H 0 , when the sample is drawn under the true law (level analysis) or when it is drawn under the same parametric copula family, but with varying Chapter 4. About tests of the "simplifying" assumption for conditional copulas 104 parameters (power analysis). For example, when the true law is a Gaussian copula with a constant parameter ρ corresponding to τ = 1/2, we draw samples under the alternative through a bivariate Gaussian copula whose random parameters are given by τ (X 3 ) = Φ(X 3 ). The chosen test statistics are T 0 CvM , T 0 CvM (nonparametrics test of H 0 ), I χ,n and I 2,n (nonparametric tests of H 0 based on the independence property) and T c 2 (a parametric test of H c 0 ). To compute these statistics, we use the estimator of the partial copula defined in Equation (4.6).

Test of H 0 : in the case of the test with boxes, the data-generating process will be

X 1 ∼ N (γ(X 3 ), 1), X 2 ∼ N (γ(X 3 ), 1) and X 3 ∼ N (0, 1),
where γ(x) = Φ -1 ( mΦ(X 3 ) /m), so that the boxes are all of equal probability. As m → ∞, we recover the continuous model for which γ(x) = x.

In the same way, we calibrate the parameter θ(x) of the conditional copulas such that the conditional Kendall's tau satisfies τ (X 3 ) = mΦ(X 3 ) /m. The choice of "the best" boxes A 1,J , . . . , A m,J is not an easy task. This problem happens frequently in statistics (think of Pearson's chi-square test of independence, for instance), and there is no universal answer. Nonetheless, in some applications, intuition can be fuelled by the context. For example, in finance, it makes sense to test whether past positive returns induce different conditional dependencies between current returns than past negative returns. And, as a general "by default" rule, we can divide the space of X J into several boxes of equal (empirical) probabilities. This trick is particularly relevant when the conditioning variable is univariate. Therefore, in our example, we have chosen m = 5 boxes of equal empirical probability for X 3 , with equal weights.

We have only evaluated T c 2 for testing H c 0 . In the following tables, for the parametric tests,

• "bootNP" means the usual nonparametric bootstrap ;

• "bootPI" means the parametric independent bootstrap (where Z I|J is drawn under C θ0 and X J under the usual nonparametric bootstrap);

• "bootPC" means the parametric conditional bootstrap (nonparametric bootstrap for X J , and X I is sampled from the estimated conditional copula C θ(X * J ) ); • "bootPseudoInd" means the pseudo-independent bootstrap (nonparametric bootstrap for X J , and draw Ẑ * I|J independently, among the pseudo-observations Ẑj,I|J );

• "bootCond" means the conditional bootstrap (nonparametric bootstrap for X J , and X I is sampled from the estimated conditional law of X I given X * J ).

Concerning tests of H 0 , the results are relatively satisfying. For the nonparametric tests and those based on the independence property (Tables 4.1 and 4.2) the rejection rates are large when τ max = 1, and the theoretical levels (5%) are underestimated (a not problematic feature in practice). This is still the case for tests of the simplifying assumption under a parametric copula model through T c 2 : see Tables 4 We have not tried to exhibit an "asymptotically optimal" bandwidth selector for our particular testing problem. This could be the task for further research. We have preferred a basic ad-hoc procedure. In our test statistics, we smooth w.r.t. F 3 (X 3 ) (or its estimate, to be specific), whose law is uniform on (0, 1). A reasonable bandwidth h is given by the so-called rule-of-thumb in kernel density estimation, i.e. h * = σ(F 3 (X 3 ))/n 1/5 = 1/( √ 12n 1/5 ) = 0.083. Such a choice has provided reasonable results. The typical influence of the bandwidth choice on the test results is illustrated in Figure 4.7. In general, the latter h * belongs to reasonably wide intervals of "convenient" bandwidth values, so that the performances of our considered tests are not very sensitive to the bandwidth choice.

To avoid boundary problems, we have slightly modified the test statistics: we remove the observa-

tions i such that F 3 (X i,3 ) ≤ h or F 3 (X i,3 ) ≥ 1 -h.

Conclusion

We have provided an overview of the simplifying assumption problem, under a statistical point of view.

In the context of nonparametric or parametric conditional copula models (with unknown conditional marginal distributions), numerous testing procedures have been proposed. We have developed the theory towards a slightly different but related approach, where "box-type" conditioning events replace pointwise ones. This opens a new field for research that is interesting per se. Several new bootstrap procedures have been detailed, to evaluate p-values under the zero assumption in both cases. In particular, we have proved the validity of one of them (the "parametric independent" bootstrap scheme under

H 0 ).
Clearly, there remains a lot of work. We have opened the Pandora box rather than provided definitive answers. Open questions are still numerous: precise theoretical convergence results of our test statistics (and others!), validity of these new bootstrap schemes, bandwidth choices, empirical performances,... All these dimensions would require further research. We have made a contribution to the landscape of problems related to the simplifying assumption, and proposed a working program for the whole copula community. X = (X I , X J ) random vector of size d I, J {1, . . . , p} and {p + 1, . . . , d}

S = (X 1,1:d , . . . , X n,1:d ) initial sample of n i.i.d. observations A J measurable subset in R d-p A J collection of all measurable subsets of R d-p
such that X J is in each set with positive probability They have the same law as the previously called vectors Z i,I|A J or Z i,I|Y under the zero assumption

A J = {A 1,J , . . . , A m,J } partition of R d-p into m sets such that X J is in each set with positive probability Y box index, i.e. Y is the k such that X J ∈ A k,J Ûi,k i-th pseudo-observation of the k-th variable Z I|J conditional observation of X I given X J Z I|Y conditional
Z I|Y given X J ∈ A J F i (•) marginal cdf of X i , i = 1, . . . , d F i|J (•|X J ∈ A J ) conditional marginal cdf of X i given X J ∈ A J , i = 1, . . . , p F i|J (•|X J = x J ) conditional marginal cdf of X i given X J = x J , i = 1, . . . , p F I|J (•|X J ∈ A J ) conditional joint cdf of X I given X J ∈ A J F I|J (•|X J = x J ) conditional joint cdf of X I given X J = x J G I,J (•) joint cdf of (Z I|J , X J ) C A J I|J (•|X J ∈ A J ) conditional copula of X I given X J ∈ A J C I|J (•|X J = x J ) conditional copula of X I given X J = x J C s,I|J (•) partial copula of X I given X J
H c 0 . Let (X i,J ) i=1,...,n be a sequence of i.i.d random vectors in R d-p , X i,J ∼ F J . Let (Z * i ) i=1,.
..,n be an independent sequence of i.i.d random vectors in [0, 1] p , where Z * i ∼ C θ0 exactly as Z i . The three samples (Z i ), (X i,J ) and (Z * i ) are mutually independent. Let (X * i,J ) i=1,...,n be a sequence of i.i.d random vectors in R d-p , which are drawn from F n,J , the empirical cdf of X 1,J , . . . X n,J , and independently of both

(Z i ) and (Z * i ).
In the following, we shall use the notation f ⊗ g := (x, y) → f (x)g(y) when f , g are two real functions, possibly from different spaces. Set l(θ, •) := log c θ (•). We will need some conditions of regularity.

Assumption (R): (θ, u I ) → l(θ, u I ) is three times differentiable with respect to θ, for every u I ∈ (0, 1) p .

Moreover, for every > 0,

IE sup θ| θ-θ0 ≤ sup {z| z-Zi ≤ Ẑi,I|Y -Zi } || ∂ 3 l ∂θ 3 (θ, z) || < +∞.
The latter technical assumption can be weakened through some trimming techniques, as in [START_REF] Fermanian | Single-index copulas[END_REF].

Since this would require to change the definitions of the parametric estimators, we do not try to improve towards this direction. We will set .

c θ := ∂c θ /∂θ and ..

c θ := ∂ 2 c θ /∂θ 2 .
We associate to every X i,J (resp.

X * i,J ) its corresponding index Y i (resp. Y * i ) s.t. X i,J ∈ A Yi (resp. X * i,J ∈ A Y * i ).
For convenience, we assume that (A k ) k=1,...,m is a partition of R d-p . Otherwise, we have to restrict our sample to the observations for which X i,J belongs to some "box" A k , k = 1, . . . , m.

Therefore, denote by C n , C * n , P n,Y and P * n,Y the empirical laws of (Z i ), (Z * i ), (Y i ) and (Y * i ) respectively. The joint law of (Z 1 , Y 1 ) (resp. (Z 1 , X 1,J )) will be denoted by

G := C θ0 ⊗ P Y (resp. G := C θ0 ⊗ F J ), with P Y (k) = IP(Y = k), k = 1, . . . , m. Denote by G n (resp. G n ) the empirical law of (Z i , Y i ) i=1,...,n (resp. (Z i , X i,J ) i=1,...,n ) Moreover, G *
n and G * n will be the empirical distributions of (Z * i , Y * i ) i=1,...,n and (Z * i , X * i,J ) i=1,...,n respectively. Let P n be the joint probability distribution of

Z i , Y i , Z * i , Y * i i=1,...,n ∈ [0, 1] p × {1, . . . , m} ⊗2n .
The following proposition is key. It will be proved in Subsection 4.7.3.

Proposition 4.16. Consider the empirical process defined on

[0, 1] p × R d-p by G n (z, x J ) := √ n(G n -G)(z, x J ) := 1 √ n n i=1 {1 (Z i , X i,J ) ≤ (z, x J ) -C θ0 (z)F J (x J )},
and the corresponding bootstrapped empirical process

G * n (z, x J ) := 1 √ n n i=1 1 (Z * i , X * i,J ) ≤ (z, x J ) -C θ0 (z) 1 √ n n i=1 1(X i,J ≤ x J ), or, equivalently, G * n = √ n(G * n -C θ0 ⊗ F n,J ).
Then there exist two independent and identically distributed Gaussian processes A G and

A ⊥ G such that G n , G * n converges to (A G , A ⊥ G ) weakly in ∞ [0, 1] p × R d-p 2 .
As a Corollary, we deduce the same results when the discrete variables Y i replace the variables X i,J .

Proposition 4.17. Under the assumptions of Proposition 4.16, let the empirical process defined on

[0, 1] p × {1, . . . , m} by G n (z, k) := √ n(G n -G)(z, k) := 1 √ n n i=1 {1 Z i ≤ z, Y i = k -C θ0 (z)P Y (k)},
and its bootstrapped empirical process

G * n (z, k) := 1 √ n n i=1 1 Z * i ≤ z, Y * i = k -C θ0 (z) 1 √ n n i=1 1(Y i = k), or equivalently G * n = √ n(G * n -C θ0 ⊗ P n,Y ), P n,Y (k)
being the empirical proportion of S n -observations into A k . Then, there exist two independent and identically distributed processes

A G and A ⊥ G such that G n , G * n converges to (A G , A ⊥ G ) weakly in ∞ [0, 1] p × {1, . . . , m} 2 .
Remark 4.18. The covariance function of A G (or A ⊥ ) is given by

IE[A G (z, y)A G (z , y )] = lim n IE[G n (z, y)G n (z , y )] = 1(y = y )IP(Y = y)C θ0 (z ∧ z ) -IP(Y = y)IP(Y = y )C θ0 (z)C θ0 (z ).
As a "toolbox", we will need the following lemma. We will assume they lie in the interior of Θ. Set Θ n,0 := √ n θb 0 -θ 0 , and, for k

= 1, . . . , m, Θ n,k := √ n θ(A k ) -θ 0 . Moreover, for any distribution H on [0, 1] p × {1, . . . , m}, set ψ k,1 (H) := ∂l ∂θ   θ 0 , 1{z 1 q ≤ z 2 q , y 1 = y 2 }dH(z 1 , y 1 ) 1{y 1 = k}dH(z 1 , y 1 ) q=1,...,p   1{y 2 = k} dH(z 2 , y 2 ), ψ k,2 (H) := ∂ 2 l ∂θ 2   θ 0 , 1{z 1 q ≤ z 2 q , y 1 = y 2 }dH(z 1 , y 1 ) 1{y 1 = k} dH(z 1 , y 1 ) q=1,...,p   1{y 2 = k} dH(z 2 , y 2 ). (i) For k = 1, . . . , m, Θ n,k = - √ nψ k,1 (G n ) ψ k,2 (G n ) + o P (1).
(ii) For every discrete law P Y with values in {1, . . . , m}, the corresponding distribution

G := C θ0 ⊗ P Y satisfies ψ k,1 ( G) = 0. (iii) ψ 1 := (ψ 1,1 , . . . , ψ m,1
) is Hadamard-differentiable at every cdf H, and its differential is given by .

ψ k,1 (H)(h) = ∂l ∂θ   θ 0 , 1{z 1 q ≤ z 2 q , y 1 = y 2 }dH(z 1 , y 1 ) 1{y 1 = k}dH(z 1 , y 1 ) q=1,...,p   1{y 2 = k} dh(z 2 , y 2 ) + p j=1 ∂ 2 l ∂θ ∂z j   θ 0 , 1{z 1 q ≤ z 2 q , y 1 = y 2 }dH(z 1 , y 1 ) 1{y 1 = k}dH(z 1 , y 1 ) q=1,...,p   1{y 2 = k} • 1{z 1 j ≤ z 2 j , y 1 = y 2 }dh(z 1 , y 1 ) 1{y 1 = k}dH(z 1 , y 1 ) - 1{z 1 j ≤ z 2 j , y 1 = y 2 }dH(z 1 , y 1 ) 1{y 1 = k}dh(z 1 , y 1 ) 1{y 1 = y 2 }dH(z 1 , y 1 ) 2 dH(z 2 , y 2 ) (iv) Θ n,0 = - m k=1 √ n ψ k,1 (G n ) m k=1 ψ k,2 (G n ) + o P (1) = m k=1 ψ k,2 (G n )Θ n,k m k=1 ψ k,2 (G n ) + o P (1)
Proof : Note that Ẑi,I|J is an explicit measurable function of the sample (Z i,I|J ) i=1,...,n . Indeed, for any i = 1, . . . , n and q = 1, . . . , p, Ẑi,q|Y := Fn,q (X i,q |X J ∈ A Yi,J )

:= n j=1 1{X j,q ≤ X i,q , X j,J ∈ A Yi,J } n j=1 1{X j,J ∈ A k(X i,J ),J } = n j=1 1{F q (X j,q |X J ∈ A Yj ,J ) ≤ F q (X i,q |X J ∈ A Yj ,J ), Y j = Y i } n j=1 1{Y j = Y i } = n j=1 1{F q (X j,q |X J ∈ A Yj ,J ) ≤ F q (X i,q |X J ∈ A Yi,J ), Y j = Y i } n j=1 1{Y j = Y i } = n j=1 1{Z j,q|Y ≤ Z i,q|Y , Y j = Y i } n j=1 1{Y j = Y i } . (4.32)
We deduce that θb 0 and θ(A k ) are measurable functions of the unobservable random variables Z i,I|Y and Y i , for i = 1, . . . , n.

(i). Let k ∈ {1, . . . , m}. Applying successively the first order condition for the estimator θ(A k ) and some Taylor series expansions, we have

0 = 1 n n i=1 ∂l ∂θ ( θ(A k ), Ẑi,I|J )1{Y i = k} = B 1,k n -B 2,k n θ(A k,J ) -θ 0 + o P θ(A k,J ) -θ 0 , with B 1,k n := 1 n n i=1 ∂l ∂θ θ 0 , Ẑi,I|J 1{Y i = k} and B 2,k n := - 1 n n i=1 ∂ 2 l ∂θ 2 θ 0 , Ẑi,I|J 1{Y i = k}, implying Θ n,k := √ n θ(A k,J ) -θ 0 = √ nB 1,k n B 2,k n + o P Θ n,k .
Now, invoking (4.32), let us compute the numerator of this expression:

B 1,k n = 1 n n i=1 ∂l ∂θ   θ 0 , n j=1 1{Z j,q ≤ Z i,q , Y q = k} n j=1 1{Y j = k} q=1,...,p   1{Y i = k} = ∂l ∂θ   θ 0 , 1{z 1 q ≤ z 2 q , y 1 = k}dG n (z 1 , y 1 ) 1{y 1 = k}dG n (z 1 , y 1 ) q=1,...,p   1{y 2 = k}dG n (z 2 , y 2 ) = ψ k,1 (G n ).
Chapter 4. About tests of the "simplifying" assumption for conditional copulas 117

In the same way, the denominator can be rewritten as

B 2,k n = - ∂ 2 l ∂θ 2   θ 0 , 1{z 1 q ≤ z 2 q , y 1 = k}dG n (z 1 , y 1 ) 1{y 1 = k}dG n (z 1 , y 1 ) q=1,...,p   1{y 2 = k}dG n (z 2 , y 2 ) = -ψ k,2 (G n ).
(ii). We now prove the second part of the lemma. Since G = C θ0 ⊗ F Y , we get

ψ k,1 ( G) := ∂l ∂θ   θ 0 , 1{z 1 q ≤ z 2 q , y 1 = k}d G(z 1 , y 1 ) 1{y 1 = k}d G(z 1 , y 1 ) q=1,...,p   1{y 2 = k}d G(z 2 , y 2 ) = ∂l ∂θ   θ 0 , IP{Z 1 q ≤ z 2 q , Y 1 = k} IP{Y 1 = k} q=1,...,p   1{y 2 = k}d G(z 2 , y 2 ) = ∂l ∂θ θ 0 , IP{Z 1 q ≤ z 2 q } q=1,...,p dC θ0 (z 2 ) 1{y 2 = k}dF Y (y 2 ) = IP{Y = k} ∂l ∂θ θ 0 , z 2 dC θ0 (z 2 ) = 0.
(iii). We remark that the law G appears three times in ψ k,1 : two times in the log-density l and one time at the end of the main integral. By separating the effect of a change from H to H + h in the main integral only (first term of the differential) and the effect of a change in l, and using the standard rule of differential calculus (l is differentiable), we obtain the second part of the given result.

(iv). As in the proof of (i), we apply successively the first order condition for θb n,0 and some Taylor series expansion to get

0 = 1 n n i=1 ∂l ∂θ ( θb 0 , Ẑi,I|Y ) = B 1 n -θb 0 -θ 0 B 2 n + o P θb 0 -θ 0 , with B 1 n := 1 n n i=1 ∂l ∂θ θ 0 , Ẑi,I|Y = m k=1 B 1,k n and B 2 n := - 1 n n i=1 ∂ 2 l ∂θ 2 θ 0 , Ẑi,I|Y = m k=1 B 2,k n .
We deduce

Θ n,0 := √ n θb 0 -θ 0 = √ nB 1 n B 2 n + o P Θ n,0 = √ n m k=1 B 1,k n m k=1 B 2,k n + o P Θ n,0 = √ n m k=1 ψ k,1 (G n ) m k=1 -ψ k,2 (G n ) + o P Θ n,0 = m k=1 ψ k,2 (G n )Θ n,k m k=1 ψ k,2 (G n ) + o P Θ n,0 .
Lemma 4.20. Let n be defined by

n := n i=1 log c θb 0 (Z * i ) c θ0 (Z * i )
.

If there exists a random vector Θ 0 such that Θ n,0 =⇒ Θ 0 under P n , then we have

n = Θ T 0 W ⊥ - 1 2 Θ T 0 I 0 Θ 0 + o P (1) ,
where W ⊥ ∼ N (0, I 0 ) is independent of the sample Z i,I|Y , Y i i=1,...,n and I 0 is the Fisher information matrix

I 0 := IE C θ 0 . c T θ0 (Z) . c θ0 (Z) c 2 θ0 (Z)
.

Chapter 4. About tests of the "simplifying" assumption for conditional copulas 118

Proof : By a Taylor expansion, we obtain

n = n i=1 {l θb 0 , Z * i -l (θ 0 , Z * i )} = θb 0 -θ 0 T n i=1 ∂l ∂θ (θ 0 , Z * i ) + 1 2 θb 0 -θ 0 T n i=1 ∂ 2 l ∂θ 2 (θ 0 , Z * i ) θb 0 -θ 0 + R n = Θ T n,0 1 √ n n i=1 ∂l ∂θ (θ 0 , Z * i ) + 1 2 Θ T n,0 1 n n i=1 ∂ 2 l ∂θ 2 (θ 0 , Z * i ) Θ n,0 + R n .
First, we have

R n ≤ Cst|| θb 0 -θ 0 || 3 sup θ| θ-θ0 ≤ θb 0 -θ0 || n i=1 ∂ 3 l ∂θ 3 (θ, Z * i ) || ≤ Cst||Θ n,0 || 3 • sup θ| θ-θ0 ≤ θb 0 -θ0 || 1 n n i=1 ∂ 3 l ∂θ 3 (θ, Z * i ) || • 1 √ n = O P 1 √ n ,
by Assumption (R). By the usual CLT, we know that 1

√ n n i=1 ∂l/∂θ (θ 0 , Z * i ) -→ W ⊥ . W ⊥ is independent of Z i,I|Y , Y i i=1,.
..,n as a limit of a sequence of variables that have the same property. Using the law of large numbers, we have also

1 n n i=1 ∂ 2 l ∂θ 2 (θ 0 , Z * i ) = 1 n n i=1 .. c θ c θ (Z * i ) - . c T θ . c θ c 2 θ (Z * i ) =⇒ 0 -I 0

Proof of Theorem 4.14

We first reason under P n as in Theorem 1 in [START_REF] Genest | Validity of the parametric bootstrap for goodness-of-fit testing in semiparametric models[END_REF]. By Proposition 4.17, under P n , there exist two independent and identically distributed processes

A G and A ⊥ G such that √ n G n -C θ0 ⊗ P Y , G * n -C θ0 ⊗ P n,Y =⇒ (A G , A ⊥ G ), weakly in ∞ [0, 1] p × {1, . . . , m} 2
. By (iii) of Lemma 4.19, ψ 1 is Hadamard-differentiable and so, using the functional Delta-method, we deduce

√ n ψ 1 (G n ) -ψ 1 (C θ0 ⊗ P Y ) , ψ 1 (G * n ) -ψ 1 (C θ0 ⊗ P n,Y ) =⇒ . ψ 1 (G)(A G ) , . ψ 1 (G)(A ⊥ G ) . By (ii) of Lemma 4.19, ψ 1 (C θ0 ⊗ P Y ) = ψ 1 (C θ0 ⊗ P n,Y ) = 0, implying √ n ψ 1,1 (G n ), . . . , ψ m,1 (G n ) , ψ 1,1 (G * n ), . . . , ψ m,1 (G * n ) =⇒ . ψ 1,1 (G)(A G ), . . . , . ψ m,1 (G)(A G ) , . ψ 1,1 (G)(A ⊥ G ), . . . , . ψ m,1 (G)(A ⊥ G ) .
By Slutsky's theorem, we have

√ n ψ 1,1 (G n ) ψ 1,2 (G n ) , . . . , ψ m,1 (G n ) ψ m,2 (G n ) , ψ 1,1 (G * n ) ψ 1,2 (G * n ) , . . . , ψ m,1 (G * n ) ψ m,2 (G * n ) =⇒ . ψ 1,1 (G)(A G ) ψ 1,2 (G) , . . . , . ψ m,1 (G)(A G ) ψ m,2 (G) , . ψ 1,1 (G)(A ⊥ G ) ψ 1,2 (G) , . . . , . ψ m,1 (G)(A ⊥ G ) ψ m,2 (G)
.

By (i) of Lemma 4.19, the latter convergence result implies

Θ n,1 , . . . , Θ n,m , Θ * n,1 , . . . , Θ * n,m =⇒ . ψ 1,1 (G)(A G ) -ψ 1,2 (G) , . . . , . ψ m,1 (G)(A G ) -ψ m,2 (G) , . ψ 1,1 (G)(A ⊥ G ) -ψ 1,2 (G) , . . . , . ψ m,1 (G)(A ⊥ G ) -ψ m,2 (G) =: Θ 1 , . . . , Θ m , Θ ⊥ 1 , . . . , Θ ⊥ m .
Moreover, Θ 1 , . . . , Θ m and Θ ⊥ 1 , . . . , Θ ⊥ m are independent and identically distributed under P n , by construction. Because of (iv) of Lemma 4.19, Θ n,0 can asymptotically be seen as a mean of the Θ n,k and this provides

Θ n,0 = m k=1 ψ k,2 (G n )Θ n,k m k=1 ψ k,2 (G n ) =⇒ m k=1 ψ k,2 (G)Θ k m k=1 ψ k,2 (G) =: Θ 0 .
Therefore, by the continuous mapping theorem, we deduce

Θ n,0 , . . . , Θ n,m , Θ * n,0 , . . . , Θ * n,m =⇒ Θ 0 , . . . , Θ m , Θ ⊥ 0 , . . . , Θ ⊥ m ,
and we still have that Θ 0 , . . . , Θ m and Θ ⊥ 0 , . . . , Θ ⊥ m are independent and identically distributed under P n . Now, we will work under P * n the probability measure over [0, 1] p × {1, . . . , m} ⊗2n whose density with respect to P n is

dP * n dP n (z 1 , y 1 , . . . , z n , y n , z * 1 , y * 1 , . . . , z * n , y * n ) = n i=1 c θb 0 (z * i ) c θ0 (z * i )
, where θb 0 is the estimator of θ 0 when applied to the "sample" (z 1 , y 1 , . . . , z n , y n ). We remark that

dP * n dP n (Z 1 , Y 1 , . . . , Z n , Y n , Z * 1 , Y * 1 , . . . , Z * n , Y * n ) = exp( n ).
Since we have shown that Θ n,0 =⇒ Θ 0 under P n , use Lemma 4.20 and obtain

n = Θ T 0 W ⊥ - 1 2 Θ T 0 I 0 Θ 0 + o P (1)
.

Therefore, under P n , we have

dP * n dP n , Θ n,0 , . . . , Θ n,m , Θ * n,0 , . . . , Θ * n,m =⇒ ζ, Θ 0 , . . . , Θ m , Θ ⊥ 0 , . . . , Θ ⊥ m ,
where

ζ := exp Θ T 0 W ⊥ -Θ T 0 I 0 Θ 0 /2 . Note that IE[ζ] = IE[IE[ζ|Θ 0 ]] = 1
because Θ 0 and W ⊥ are independent, and W ⊥ ∼ N (0, I 0 ). This corresponds to condition (iii) of Theorem 3.10.5 of [START_REF] Van Der | Weak convergence[END_REF], and we deduce P * n is contiguous with respect to P n . We can then apply Le Cam's Third Lemma (Theorem 3.10.7 of [START_REF] Van Der | Weak convergence[END_REF]. We get that, under P * n ,

Θ n,0 , . . . , Θ n,m , Θ * n,0 , . . . , Θ * n,m =⇒ Θ0 , . . . , Θm , Θ * 0 , . . . , Θ * m ,
where

IE[χ( Θ0:m , Θ * 0:m )] = IE[ζχ(Θ 0:m , Θ ⊥ 0:m )]
for any simple function χ. Choose w 1 and w 2 ∈ R m+1 and set Σ := V ar Θ 0:m . Then, we have

IE[exp(iw T 1 Θ0:m + iw T 2 Θ * 0:m )] = IE[ζ exp(iw T 1 Θ 0:m + iw T 2 Θ ⊥ 0:m )] = IE[exp(Θ T 0 W ⊥ -Θ T 0 I 0 Θ 0 /2 + iw T 1 Θ 0:m + iw T 2 Θ ⊥ 0:m )] = IE exp(iw T 1 Θ 0:m -Θ T 0 I 0 Θ 0 /2) IE[exp(Θ T 0 W ⊥ + iw T 2 Θ ⊥ 0:m ) | Θ 0:m ] = IE exp(iw T 1 Θ 0:m -Θ T 0 I 0 Θ 0 /2) exp 1 2 -w T 2 Σw 2 + Θ T 0 I 0 Θ 0 + 2iw 2 IE[Θ ⊥ 0:m T W ⊥ ]Θ 0 = IE exp iw T 1 Θ 0:m -w T 2 Σw 2 /2 + iw 2 IE[Θ ⊥ 0:m T W ⊥ ]Θ 0 = IE exp iw T 1 Θ 0:m + iw 2 Θ ⊥ 0:m + iw 2 IE[Θ ⊥ 0:m T W ⊥ ]Θ 0 .
Chapter 4. About tests of the "simplifying" assumption for conditional copulas 120 Therefore, we have proven the following equality:

Θ0 , . . . , Θm , Θ * 0 , . . . , Θ * m law = Θ 0 , . . . , Θ m , Θ ⊥ 0 + a 0 Θ 0 , . . . , Θ ⊥ m + a m Θ 0 ,
where

a k = IE[Θ ⊥ k T W ⊥ ].
To finish the proof, it remains to show that a k = a 0 for all k ∈ {1, . . . , m}, i.e.

IE[Θ ⊥ 0 T W ⊥ ] = IE[Θ ⊥ k T W ⊥ ].
First, we know from the proof of Lemma 4.19 that Θ k,n = -.

ψ k,1 (G)(A G )/ψ k,2 (G) + o P (1), k = 1, . . . , m and Θ 0,n = - . ψ 0,1 (G)(A G )/ψ 0,2 (G) + o P (1)
, where

ψ 0,1 (G) := ∂l ∂θ   θ 0 , 1{z 1 q ≤ z 2 q , y 1 = y 2 }dG(z 1 , y 1 )
1{y 1 = y 2 }dG(z 1 , y 1 ) q=1,...,p   dG(z 2 , y 2 ), and
ψ 0,2 (G) := ∂ 2 l ∂θ 2   θ 0 , 1{z 1 q ≤ z 2 q , y 1 = y 2 }dG(z 1 , y 1 )
1{y 1 = y 2 }dG(z 1 , y 1 ) q=1,...,p   dG(z 2 , y 2 ).
This implies

Θ k = - . ψ k,1 (G)(A G )/ψ k,2 (G), k = 0, . . . , m.
Actually, the reasoning is exactly the same when dealing with Θ * k,n and Θ ⊥ k , k = 0, . . . , m, replacing A G by A ⊥ G . We get

Θ ⊥ k = - . ψ k,1 (G)(A ⊥ G ) ψ k,2 (G)
, and

Θ ⊥ 0 = - . ψ 0,1 (G)(A ⊥ G ) ψ 0,2 (G) • Second, note that, when k = 1, . . . , m, ψ k,2 (G) := ∂ 2 l ∂θ 2   θ 0 , 1{z 1
q ≤ z 2 q , y 1 = k}dG(z 1 , y 1 )

1{y 1 = k} dG(z 1 , y 1 ) q=1,...,p   1{y 2 = k} dG(z 2 , y 2 ) = IP(Y = k) ∂ 2 l ∂θ 2 θ 0 , 1{z 1 q ≤ z 2 q }dC θ0 (z 1 ) q=1,...,p dC θ0 (z 2 ) = IP(Y = k) ∂ 2 l ∂θ 2 (θ 0 , z) dC θ0 (z 2 ) = IP(Y = k)ψ 0,2 (G).
Third, let us calculate .

ψ k,1 (G)(h), k = 0, 1, . . . , m. From Lemma 4.19, we have . ψ k,1 (G)(h) = ∂l ∂θ θ 0 , z 2 1{y 2 = k} dh(z 2 , y 2 ) + p j=1 ∂ 2 l ∂θ ∂z j θ 0 , z 2 1{y 2 = k} • 1{z 1 j ≤ z 2 j , y 1 = y 2 }dh(z 1 , y 1 ) 1{y 1 = y 2 }dG(z 1 , y 1 ) - 1{z 1 j ≤ z 2 j , y 1 = y 2 }dG(z 1 , y 1 ) 1{y 1 = y 2 }dh(z 1 , y 1 ) 1{y 1 = y 2 }dG(z 1 , y 1 ) 2 dG(z 2 , y 2 ).
for k = 1, . . . , m. Since G = C θ0 ⊗ F Y , we can simplify the latter equalities:

.

ψ k,1 (G)(h) = ∂l ∂θ θ 0 , z 2 1{y 2 = k} dh(z 2 , y 2 ) + IP(Y = k) p j=1 ∂ 2 l ∂θ ∂z j θ 0 , z 2 • [1{z 1 j ≤ z 2 j , y 1 = y 2 } -z 2 j 1{y 1 = y 2 }]dh(z 1 , y 1 ) 1{y 1 = y 2 }dG(z 1 , y 1 ) dC θ0 (z 2 ).
Chapter 4. About tests of the "simplifying" assumption for conditional copulas 121 Since .

ψ 0,1 (G)(h) = m k=1 . ψ k,1 (G)(h), we have . ψ 0,1 (G)(h) = ∂l ∂θ θ 0 , z 2 dh(z 2 , y 2 ) + p j=1 ∂ 2 l ∂θ ∂z j θ 0 , z 2 • [1{z 1 j ≤ z 2 j , y 1 = y 2 } -z 2 j 1{y 1 = y 2 }]dh(z 1 , y 1 ) 1{y 1 = y 2 }dG(z 1 , y 1 ) dC θ0 (z 2 )dF Y (y 2 ).
Then, we can rewrite .

ψ k,1 (G)(h) = M 1 (h, k) + IP(Y = k)M 3 (h, k) and . ψ 0,1 (G)(h) = M 2 (h) + m k =1 IP(Y = k )M 3 (h, k ), where M 1 (h, k) := ∂l ∂θ θ 0 , z 2 1{y 2 = k} dh(z 2 , y 2 ), M 2 (h) := ∂l ∂θ θ 0 , z 2 dh(z 2 , y 2 ), M 3 (h, k) := p j=1 ∂ 2 l ∂θ ∂z j θ 0 , z 2 [1{z 1 j ≤ z 2 j , y 1 = k} -z 2 j 1{y 1 = k}]dh(z 1 , y 1 ) 1{y 1 = k}dG(z 1 , y 1 ) dC θ0 (z 2 ).
Substituting h by A ⊥ G , we get

Θ ⊥ k = - . ψ k,1 (G)(A ⊥ G ) ψ k,2 (G) = - M 1 (A ⊥ G , k) IP(Y = k)ψ 0,2 (G) - M 3 (A ⊥ G , k) ψ 0,2 (G)
,

Θ ⊥ 0 = - . ψ 0,1 (G)(A ⊥ G ) ψ 0,2 (G) = - M 2 (A ⊥ G ) ψ 0,2 (G) - m k =1 IP(Y = k )M 3 (A ⊥ G , k ) ψ 0,2 (G) • Fourth, since W ⊥ is the weak limit of n i=1 ∂l ∂θ (θ 0 , Z * i ) / √ n under P n , this implies W ⊥ = . ψ 3 (G)(A ⊥ G ), with ψ 3 (G) =
∂l ∂θ (θ 0 , z) dG(z, y), and .

ψ 3 (G)(h) = ∂l ∂θ (θ 0 , z) dh(z, y).
Finally, by (i) of the following Lemma 4.21, we have

IE[M 1 (A ⊥ G , k) T W ⊥ ] = IP(Y = k)IE[M 2 (A ⊥ G ) T W ⊥ ], By (ii) of the latter lemma, we have IE[M 3 (A ⊥ G , k) T W ⊥ ] = IE[M 3 (A ⊥ G , k ) T W ⊥ ] for all k and k . Finally, we obtain IE[Θ ⊥ 0 T W ⊥ ] = IE[Θ ⊥ k T W ⊥ ], which finishes the proof. Lemma 4.21. Assume that H c 0 is satisfied. Then, (i) For k = 1, . . . , m, IE ∂l ∂θ T (θ 0 , z) 1{y = k} dA ⊥ G (z, y) ∂l ∂θ (θ 0 , z ) dA ⊥ G (z , y ) = IP(Y = k) IE ∂l ∂θ T (θ 0 , z) dA ⊥ G (z, y) ∂l ∂θ (θ 0 , z ) dA ⊥ G (z , y ) .
(ii) The expectations

IE ∂ 2 l ∂θ T ∂z j θ 0 , z 2 [1{z 1 j ≤ z 2 j , y 1 = k} -z 2 j 1{y 1 = k}]dA ⊥ G (z 1 , y 1 )
1{y 1 = k}dG(z 1 , y 1 )

dC θ0 (z 2 ) • ∂l ∂θ θ 0 , z 3 dA ⊥ G (z 3 , y 3 )
do not depend on k = 1, . . . , m.

Chapter 4. About tests of the "simplifying" assumption for conditional copulas 122

Proof : (i) By simple calculations, we obtain

IE ∂l ∂θ T (θ 0 , z) 1{y = k} dA ⊥ G (z, y) ∂l ∂θ (θ 0 , z ) dA ⊥ G (z , y ) = ∂l ∂θ T (θ 0 , z) 1{y = k} ∂l ∂θ (θ 0 , z ) d z,y, z ,y IE A ⊥ G (z, y)A ⊥ G (z , y ) = ∂l ∂θ T (θ 0 , z) 1{y = k} ∂l ∂θ (θ 0 , z ) {δ y =y dIP(y)[dC θ0 (z)δ z =z + dC θ0 (z )δ z=z ] -dC θ0 (z)dC θ0 (z )dIP(y)dIP(y )} = 2IP(Y = k) ∂l ∂θ T (θ 0 , z) ∂l ∂θ (θ 0 , z) dC θ0 (z) -IP(Y = k) ∂l ∂θ T (θ 0 , z) dC θ0 (z) • ∂l ∂θ (θ 0 , z) dC θ0 (z).
By summing up the latter identities w.r.t. k = 1, . . . , m, we prove (i).

(ii) For convenience, let us write φ 2 (z) := ∂ 2 l/(∂θ T ∂z j ) (θ 0 , z) and φ 3 (z) := ∂l(θ 0 , z)/∂θ T . We get the result if we prove that

A 1,k := IE φ 2 (z 2 ) 1{z 1 j ≤ z 2 j }1{y 1 = k}dA ⊥ G (z 1 , y 1 ) 1{y 1 = k}dG(z 1 , y 1 )
dC θ0 (z 2 ) φ 3 (z 3 ) dA ⊥ G (z 3 , y 3 ) and A 2,k := IE φ 2 (z 2 ) z 2 j 1{y 1 = k}dA ⊥ G (z 1 , y 1 ) 1{y 1 = k}dG(z 1 , y 1 )
dC θ0 (z 2 ) φ 3 (z 3 ) dA ⊥ G (z 3 , y 3 )
do not depend on k. We will do the task for A 1,k , k = 1, . . . , m, and the calculations will be similar for

A 2,k . Note that A 1,k = 1 IP(Y = k) φ 2 (z 2 )1{z 1 j ≤ z 2 j }1{y 1 = k}φ 3 (z 3 ) dC θ0 (z 2 )d z 1 ,y 1 ,z 3 ,y 3 IE A ⊥ G (z 1 , y 1 )A ⊥ G (z 3 , y 3 ) = 1 IP(Y = k) φ 2 (z 2 )1{z 1 j ≤ z 2 j }1{y 1 = k}φ 3 (z 3 ) dC θ0 (z 2 )
δ y 3 =y 1 dIP(y 1 )[dC θ0 (z 1 )δ z 3 =z 1 + dC θ0 (z 3 )δ z 1 =z 3 ] -C θ0 (z 1 )dC θ0 (z 3 )dIP(y 1 )dIP(y 3 ) .

We deduce that

A 1,k = 2 φ 2 (z 2 )1{z 1 j ≤ z 2 j }φ 3 (z 1 ) dC θ0 (z 1 )dC θ0 (z 2 ) -φ 2 (z 2 )1{z 1 j ≤ z 2 j }φ 3 (z 3 ) dC θ0 (z 1 )dC θ0 (z 2 )dC θ0 (z 3 ),
that does not depend on k.

Proof of Proposition 4.16

As usual with the nonparametric bootstrap, we rewrite the bootstrapped empirical process by counting the number of times every observation of the initial sample is drawn:

dG * n = 1 n n i=1 M n,i δ (Z * i ,X i,J ) ,
where M n,i denotes the number of times (X * i,J ) has been redrawn in a n-size bootstrap resampling with replacement. It is well-known that M n := (M n,1 , . . . , M n,n ) follows a multinomial distribution M(n, n -1 , . . . , n -1 ): its mean is n and the associated probabilities are 1/n, . . . , 1/n. In other words,

G * n (z, x J ) = 1 √ n n i=1 M n,i {1((Z * i , X i,J ) ≤ (z, x J )) -C θ0 (z)F n,J (x J )}.
Chapter 4. About tests of the "simplifying" assumption for conditional copulas 123

We can remove the dependence between the random components M n,i , i = 1, . . . , n by a "Poissonization" procedure. We mimic van der Vaart and Wellner [START_REF] Van Der | Weak convergence[END_REF]

G * n (z, x J ) := 1 √ n n i=1 M Nn,i {1 (Z * i ,X i,J )≤(z,x J ) -C θ0 (z)F n,J (x J )}.
Actually, the distance between G * n and G * n is negligible. Indeed, for every (z, x J ),

∆ n (z, x J ) := ( G * n -G * n )(z, x J ) = 1 √ n n i=1 (M Nn,i -M n,i ){1 (Z * i ,X i,J )≤(z,x J ) -C θ0 (z)F n,J (x J )}
is centered. Moreover, by independence between the observations and by the resampling scheme, we have

IE[ ∆ n 2 ∞ ] = IE[sup z,x J ∆ 2 n (z, x J )] ≤ 1 n 2 n i,j=1 IE[|(M Nn,i -M n,i )(M Nn,j -M n,j )|] ≤ 1 n 2 n i,j=1 IE[(M Nn,i -M n,i ) 2 ] 1/2 IE[(M Nn,j -M n,j ) 2 ] 1/2 ≤ IE[(M Nn,1 -M n,1 ) 2 ],
because the sequence

(M Nn,i -M n,i ) i=1,...,n is exchangeable. Given N n = k, the i-th variable |M Nn,i - M n,i | is binomial with the parameters (|k -n|, 1/n), i.e. P (|M k,i -M n,i | = l) = C l |k-n| 1 n l 1 - 1 n |k-n|-l
, l = 0, . . . , |k -n|.

Therefore, we obtain

IE[(M Nn,i -M n,i ) 2 ] = ∞ k=0 exp(-n) n k k! |k -n| n (1 - 1 n ) + |k -n| n 2 .
Simple calculations provide

∞ k=0 exp(-n) n k k! |k -n| n = 2n n n! exp(-n) ∼ 2 πn 1/2
, by Stirling's formula, and

∞ k=0 exp(-n) n k k! k -n n 2 = exp(-n) n 2 ∞ k=0 n k k! (k(k -1) + k(1 -2n) + n 2 ) = 1 n • We deduce IE[(M Nn,i -M n,i ) 2 ] = O(n -1/2
) and IP ( ∆ n ∞ > ) -→ 0, when n tends to the infinity, given almost all sequences S n := (Z i , X i,J ) i=1,...,n . This means that we can safely replace G * n by G * n , and the theorem follows if we prove the weak convergence of (G n , G * n ).

Note that we can rewrite

G * n (z, x J ) = 1 √ n n i=1 (M Nn,i -1) {1(Z * i , X i,J ) ≤ (z, x J ) -C θ0 (z)F J (x J )} + 1 √ n n i=1 {1(Z * i , X i,J ) ≤ (z, x J ) -C θ0 (z)F J (x J )} -C θ0 (z) √ n (F n,J -F J ) (x J ) + 1 - 1 n n i=1 M Nn,i C θ0 (z) √ n (F n,J -F J ) (x J ) := G * n,1 (z, x J ) + G * n,2 (z, x J ) -G n,3 (z, x J ) + R n (z, x J ).
Obviously, the last remaining term is o P (1) uniformly w.r.t. (z, x J ), and it can be forgotten. Moreover, since the variables (M Nn,i -1) i=1,...,n are i.i.d., centered with variance one and independent of the data, we can invoke some multiplier bootstrap results. Consider we live in the space

W := [0, 1] p ×[0, 1] p ×R d-p
that is related to our observations W i := (Z i , Z * i , X i,J ), i = 1, . . . , n. The true distribution of W i under the null is P W , whose cdf is C θ0 ⊗ C θ0 ⊗ F J . Applying Corollary 2.9.3. in [START_REF] Van Der | Weak convergence[END_REF], the sequence of processes

(W n , W * n ) := n -1/2 n i=1 (δ Wi -P W ), n -1/2 n i=1 (M Nn,i -1)(δ Wi -P W )
converges weakly in ∞ (F) × ∞ (F) to a vector of independent Gaussian processes, where F denotes any Donsker class of measurable functions from W to R.

Now, let us consider the class F of functions

f z0,z 0 ,x J,0 : (z, z , x J ) → 1(z ≤ z 0 , z ≤ z 0 , x J ≤ x J,0 ), for any triplet (z 0 , z 0 , x J,0 ) in [0, 1] p × [0, 1] p × R d-p . Note that F is Donsker, that G * n,1 (z, x J ) = W * n f 1,z,x J , G * n,2 (z, x J ) = W n f 1,z,x J and that G * n,3 (z, x J ) = C θ0 (z)W n f 1,1,x J .
By the permanence of the Donsker property (see Section 2.10 in [START_REF] Van Der | Weak convergence[END_REF], and the continuity of C θ0 , the process

G * n converges in ∞ ([0, 1] p × R d-p ) to a gaussian process A ⊥ . Obviously, G n tends in distribution in ∞ ([0, 1] p × R d-p
) to a Gaussian process A, whose covariance function is given by

IE G n (z, x J )G n (z , x J ) = C θ0 (z ∧ z )F J (x J ∧ x J ) -C θ0 (z)F J (x J )C θ0 (z )F J (x J ),
for every z, z , x J , x J ). By some standard calculations, we check that

IE[ G * n (z, x J ) G * n (z , x J )] = IE[G n (z, x J )G n (z , x J )
for every couples (z, x J ) and (z , x J ), implying that A and A ⊥ have the same covariance functions. Moreover, the two limiting processes A and A ⊥ are uncorrelated because

IE[G n (z, x J ) G * n (z , x J )] = IE[G n (z, x J )IE[ G * n (z , x J )|S n ]] = 0,
for every couples (z, x J ) and (z , x J ). Therefore, the A and A ⊥ are two independent versions of the same Gaussian process.

Remark 4.22. If there were no resampling of the observations X i,J at the first level, this would no longer be true. Indeed, the corresponding bootstrapped process would be given by

G * * n (z, x J ) := 1 √ n n i=1 1(X i,J ≤ x J ) {1(Z * i ≤ z) -C θ0 (z)} , implying IE G * * n (z, x J )G * * n (z , x J ) = F J (x J ∧ x J )[C θ0 (z ∧ z ) -C θ0 (z)C θ0 (z )], that is different of IE G n (z, x J )G n (z , x J ) .
To conclude, we apply Corollary 1.4.5. in [START_REF] Van Der | Weak convergence[END_REF]: for every bounded nonnegative Lipschitz function h and h,

IE[h(G n ) h( G * n )] -IE[h(A) h(A ⊥ )] = IE[h(G n ) IE[ h( G * n )|S n ] -IE[ h(A ⊥ )] ] + IE[ h(G n ) -IE[h(A)] ]IE[ h(A ⊥ )].
The first (resp. second) term tends to zero by the weak convergence of G * n (resp. G n ). This concludes the proof.

Introduction

In the field of dependence modeling, it is common to work with dependence measures. Contrary to usual linear correlations, most of them have the advantage of being defined without any condition on moments, and of being invariant to changes in the underlying marginal distributions. Such summaries of information are very popular and can be explicitly written as functionals of the underlying copulas: Kendall's tau, Spearman's rho, Blomqvist's coefficient... See Nelsen [START_REF] Nelsen | An introduction to copulas[END_REF] for an introduction. In particular, for more than a century (Spearman (1904), [START_REF] Kendall | A new measure of rank correlation[END_REF]), Kendall's tau has become a popular dependence measure in [-1, 1]. It quantifies the positive or negative dependence between two random variables X 1 and X 2 . Denoting by C 1,2 the unique underlying copula of (X 1 , X 2 ) that are assumed to be Chapter 5. About kernel-based estimation of conditional Kendall's tau: finite-distance bounds and asymptotic behavior 126 continuous, their Kendall's tau can be directly defined as

τ 1,2 := 4 [0,1] 2 C 1,2 (u 1 , u 2 ) C 1,2 (du 1 , du 2 ) -1 (5.1) = IP (X 1,1 -X 2,1 )(X 1,2 -X 2,2 ) > 0 -IP (X 1,1 -X 2,1 )(X 1,2 -X 2,2 ) < 0 ,
where (X i,1 , X i,2 ) i=1,2 are two independent versions of X := (X 1 , X 2 ). This measure is then interpreted as the probability of observing a concordant pair minus the probability of observing a discordant pair.

See [START_REF]Ordinal measures of association[END_REF] for an historical perspective on Kendall's tau. Its inference is discussed in many textbooks (see [START_REF] Hollander | Nonparametric Statistical Methods[END_REF] or [START_REF] Lehmann | Nonparametrics: Statistical Methods Based on Ranks[END_REF], e.g.). Its links with copulas and other dependence measures can be found in [START_REF] Nelsen | An introduction to copulas[END_REF] or [START_REF] Joe | Multivariate models and multivariate dependence concepts[END_REF].

Similar dependence measure can be introduced in a conditional setup, when a p-dimensional covariate Z is available. When thousands of papers refer to Kendall's tau, only a few of them have considered conditional Kendall's tau (as defined below) until now. The goal is now to model the dependence between the two components X 1 and X 2 , given the vector of covariates Z. Logically, we can invoke the conditional copula C 1,2|Z=z of (X 1 , X 2 ) given Z = z for any point z ∈ R p (see Patton [START_REF] Patton | Estimation of multivariate models for time series of possibly different lengths[END_REF][START_REF] Patton | Modelling asymmetric exchange rate dependence[END_REF]), and the corresponding conditional Kendall's tau would be simply defined as

τ 1,2|Z=z := 4 [0,1] 2 C 1,2|Z=z (u 1 , u 2 ) C 1,2|Z=z (du 1 , du 2 ) -1 = IP (X 1,1 -X 2,1 )(X 1,2 -X 2,2 ) > 0 Z 1 = Z 2 = z -IP (X 1,1 -X 2,1 )(X 1,2 -X 2,2 ) < 0 Z 1 = Z 2 = z ,
where (X i,1 , X i,2 , Z i ) i=1,2 are two independent versions of (X 1 , X 2 , Z). As above, this is the probability of observing a concordant pair minus the probability of observing a discordant pair, conditionally on Z 1 and Z 2 being both equal to z. Note that, as conditional copulas themselves, conditional Kendall's taus are invariant w.r.t. increasing transformations of the conditional margins X 1 and X 2 , given Z. Of course, if Z is independent of (X 1 , X 2 ) then, for every z ∈ R p , the conditional Kendall's tau τ 1,2|Z=z is equal to the (unconditional) Kendall's tau τ 1,2 .

Conditional Kendall's tau, and more generally conditional dependence measures, are of interest per se because they allow to summarize the evolution of the dependence between X 1 and X 2 , when the covariate Z is changing. Surprisingly, their nonparametric estimates have been introduced in the literature only a few years ago ( [START_REF] Gijbels | Conditional copulas, association measures and their applications[END_REF], [START_REF] Veraverbeke | Estimation of a conditional copula and association measures[END_REF], [START_REF] Fermanian | Time-dependent copulas[END_REF]) and their properties have not yet been fully studied in depth.

Indeed, until now and to the best of our knowledge, the theoretical properties of nonparametric conditional Kendall's tau estimates have been obtained "in passing" in the literature, as a sub-product of the weak-convergence of conditional copula processes ( [START_REF] Veraverbeke | Estimation of a conditional copula and association measures[END_REF]) or as intermediate quantities that will be "plugged-in" ( [START_REF] Fermanian | Single-index copulas[END_REF]). Therefore, such properties have been stated under too demanding assumptions.

In particular, some assumptions were related to the estimation of conditional margins, while this is not required (Kendall's tau are based on ranks). In this paper, we will directly study nonparametric estimates τ1,2|z without relying on the theory/inference of copulas. Therefore, we will state their main usual properties of statistical estimates: exponential bounds in probability, consistency, asymptotic normality.

Our τ 1,2|Z=z has not to be confused with so-called "conditional Kendall's tau" in the case of truncated data ( [START_REF] Tsai | Testing the assumption of independence of truncation time and failure time[END_REF], [START_REF] Martin | Testing quasi-independence of failure and truncation times via conditional kendall's tau[END_REF]), in the case of semi-competing risk models ( [START_REF] Lakhal | Estimating survival and association in a semicompeting risks model[END_REF], [START_REF] Hsieh | Nonparametric estimation and test of conditional kendall's tau under semi-competing risks data and truncated data[END_REF]), or for other partial information schemes ( [START_REF] Chaieb | Estimating survival under a dependent truncation[END_REF], [START_REF] Kim | Estimation of conditional kendall's tau for bivariate interval censored data[END_REF], among others). Indeed, particularly in biostatistics or reliability, the inference of dependence models under truncation/censoring can be led by considering some types of conditional Kendall's tau, given some algebraic relationships among the underlying random variables. This would Chapter 5. About kernel-based estimation of conditional Kendall's tau: finite-distance bounds and asymptotic behavior 127 induce conditioning by subsets. At the opposite, we will consider point-wise conditioning events only in this paper, under a nonparametric point-of-view. Nonetheless, such point-wise events can be found in the literature, in parametric or semi-parametric frameworks, as for the identifiability of frailty distributions in bivariate proportional models ( [START_REF] Oakes | Bivariate survival models induced by frailties[END_REF], [START_REF] Manatunga | A measure of association for bivariate frailty distributions[END_REF]). Other related papers are [START_REF] Asimit | Tail dependence measure for examining financial extreme co-movements[END_REF] or [START_REF] Liu | Interval estimation for a measure of tail dependence[END_REF], that are dealing with extreme co-movements (bivariate extreme-value theory). There, the the tail conditioning events of Kendall's tau have probabilities that go to zero with the sample size.

In Section 5.2, different kernel-based estimators of the conditional Kendall's tau are proposed. In Section 5.3, the theoretical properties of the latter estimators are proved, first with finite-distance bounds and then under an asymptotic point-of-view. A short simulation study is provided in Section 5.4. Proofs are postponed into the appendix.

Definition of several kernel-based estimators of τ 1,2|z

Let (X i,1 , X i,2 , Z i ), i = 1, . . . , n be an i.i.d. sample distributed as (X 1 , X 2 , Z), and n ≥ 2. Assuming continuous underlying distributions, there are several equivalent ways of defining the conditional Kendall's tau:

τ 1,2|Z=z = 4 IP X 1,1 > X 2,1 , X 1,2 > X 2,2 Z 1 = Z 2 = z -1 = 1 -4 IP X 1,1 > X 2,1 , X 1,2 < X 2,2 Z 1 = Z 2 = z = IP (X 1,1 -X 2,1 )(X 1,2 -X 2,2 ) > 0 Z 1 = Z 2 = z -IP (X 1,1 -X 2,1 )(X 1,2 -X 2,2 ) < 0 Z 1 = Z 2 = z .
Motivated by each of the latter expressions, we introduce several kernel-based estimators of τ 1,2|Z=z :

τ (1) 1,2|Z=z := 4 n i=1 n j=1 w i,n (z)w j,n (z)1 X i,1 < X j,1 , X i,2 < X j,2 -1, τ (2) 1,2|Z=z := n i=1 n j=1 w i,n (z)w j,n (z) 1 (X i,1 -X j,1 ).(X i,2 -X j,2 ) > 0 -1 (X i,1 -X j,1 ).(X i,2 -X j,2 ) < 0 , τ (3) 1,2|Z=z := 1 -4 n i=1 n j=1 w i,n (z)w j,n (z)1 X i,1 < X j,1 , X i,2 > X j,2 ,
where 1 denotes the indicator function, w i,n is a sequence of weights given by

w i,n (z) = K h (Z i -z) n j=1 K h (Z j -z) , (5.2)
with K h (•) := h -p K(•/h) for some kernel K on R p , and h = h(n) denotes a usual bandwidth sequence that tends to zero when n → ∞. In this paper, we have chosen usual Nadaraya-Watson weights.

Obviously, there are alternatives (local linear, Priestley-Chao, Gasser-M üller, etc., weight), that would lead to different theoretical results.

The estimators

τ (1) 1,2|Z=z , τ (2)
1,2|Z=z and τ (3) 1,2|Z=z look similar, but they are nevertheless different, as shown in Proposition 5.1. These differences are due to the fact that all the τ (k) 1,2|Z=z , k = 1, 2, 3 are affine transformations of a double-indexed sum, on every pair (i, j), including the diagonal terms where i = j.

The treatment of these diagonal terms is different for each of the three estimators defined above. Indeed, Chapter 5. About kernel-based estimation of conditional Kendall's tau: finite-distance bounds and asymptotic behavior 128

setting s n := n i=1 w 2 i,n (z), it can be easily proved that τ (1) 1,2|Z=z takes values in the interval [-1 , 1 -2s n ], τ (2) 1,2|Z=z in [-1 + s n , 1 -s n ], and τ (3) 1,2|Z=z in [-1 + 2s n , 1].
Moreover, there exists a direct relationship between these estimators, given by the following proposition. Proposition 5.1. Almost surely, τ

,2|Z=z + s n = τ (2) 1,2|Z=z = τ (3) 1,2|Z=z -s n , where s n := n i=1 w 2 i,n (z). (1) 1
This proposition is proved in Section 5.5.1. As a consequence, we can rescale easily the previous estimators so that the new estimator will take values in the whole interval [-1, 1]. This would yield

τ1,2|Z=z := τ (1) 1,2|Z=z 1 -s n + s n 1 -s n = τ (2) 1,2|Z=z 1 -s n = τ (3) 1,2|Z=z 1 -s n - s n 1 -s n .
Note that none of the latter estimators depends on any estimation of conditional marginal distributions. In other words, we only have to choose conveniently the weights w i,n to obtain an estimator of the conditional Kendall's tau. This is coherent with the fact that conditional Kendall's taus are invariant with respect to conditional marginal distributions. Moreover, note that, in the definition of our estimators, the inequalities are strict (there are no terms corresponding to the cases i = j). This is inline with the definition of (conditional) Kendall's tau itself through concordant/discordant pairs of observations.

The definition of τ (1) 1,2|Z=z can be motivated as follows. For j = 1, 2, let Fj|Z (•|Z = z) be an estimator of the conditional cdf of X j given Z = z. Then, a usual estimator of the conditional copula of X 1 and

X 2 given Z = z is Ĉ1,2|Z (u 1 , u 2 |Z = z) := n i=1 w i,n (z)1 F1|Z (X i,1 |Z = z) ≤ u 1 , F2|Z (X i,2 |Z = z) ≤ u 2 .
See [START_REF] Veraverbeke | Estimation of a conditional copula and association measures[END_REF] or [START_REF] Fermanian | Time-dependent copulas[END_REF], e.g. The latter estimator of the conditional copula can be plugged into (5.1) to define an estimator of the conditional Kendall's tau itself:

τ1,2|Z=z := 4 Ĉ1,2|Z (u 1 , u 2 |Z = z) Ĉ1,2|Z (du 1 , du 2 |Z = z) -1 (5.3) = 4 n j=1 w j,n (z) Ĉ1,2|Z F1|Z (X j,1 |Z = z), F2|Z (X j,2 |Z = z) Z = z -1.
Since the functions Fj|Z (•|Z = z) are non-decreasing, this reduces to

τ1,2|Z=z = 4 n i=1 n j=1 w i,n (z)w j,n (z)1 X i,1 ≤ X j,1 , X i,2 ≤ X j,2 -1 = 4 n i=1 n j=1 w i,n (z)w j,n (z)1 X i,1 < X j,1 , X i,2 < X j,2 -1 + o P (1) = τ (1) 1,2|Z=z + o P (1).
Veraverbeke et al. [START_REF] Veraverbeke | Estimation of a conditional copula and association measures[END_REF], Subsection 3.2, introduced their estimator of τ 1,2|z by (5.3). By the functional Delta-Method, they deduced its asymptotic normality as a sub-product of the weak convergence of the

process √ nh Ĉ1,2|Z (•, •|z) -C 1,2|Z (•, •|z) when Z is univariate.
In our case, we will obtain more and stronger theoretical properties of τ (1) 1,2|Z=z under weaker conditions by a more direct analysis based on ranks. In particular, we will not require any regularity condition on the conditional marginal distributions, contrary to [START_REF] Veraverbeke | Estimation of a conditional copula and association measures[END_REF]. Indeed, in the latter paper, it is required that F j|Z (•|Z = z) has to be two times continuously differentiable (assumption R3) and its inverse has to be continuous (assumption R1). This is not satisfied for some simple univariate cdf as 4), for instance. Note that We could similarly justify τ (3) 1,2|Z=z in a similar way by considering conditional survival copulas.

F j (t) = t1(t ∈ [0, 1])/2+1(t ∈ (1, 2])/2+t1(t ∈ (2, 4])/4+1(t >
Chapter 5. About kernel-based estimation of conditional Kendall's tau: finite-distance bounds and asymptotic behavior 129

Let us define g 1 , g 2 , g 3 by

g 1 (X i , X j ) := 41 X i,1 < X j,1 , X i,2 < X j,2 -1, g 2 (X i , X j ) := 1 (X i,1 -X j,1 ) × (X i,2 -X j,2 ) > 0 -1 (X i,1 -X j,1 ) × (X i,2 -X j,2 ) < 0 , g 3 (X i , X j ) := 1 -41 X i,1 < X j,1 , X i,2 > X j,2 ,
where for i = 1, . . . , n, we set X i := (X i,1 , X i,2 ). Clearly,

τ (k) 1,2|z is a smoothed estimator of IE[g k (X 1 , X 2 )|Z 1 = Z 2 = z], k = 1,
τ (h, k) -(i,j), 1,2|Z=z the estimator τ (k)
1,2|Z=z that is made with the smoothing parameter h and our dataset where the i-th and j-th observations have been removed. As a consequence, the random function

τ (h, k) -(i,j), 1,2|Z=• is independent of (X i , Z i ), (X j , Z j ) .
As usual with kernel methods, the bandwidth ĥ has to be chosen. It would be tempting to propose h as the minimizer of the cross-validation criteria

CV DM (h) := 2 n(n -1) n i,j=1 g k (X i , X j ) - τ (h, k) -(i,j), 1,2|Z=(Zi+Zj )/2 2 K h (Z i -Z j ),
for k = 1, 2, 3 or for τ1,2|Z=• . The latter criterion would be a "naively localized" version of the usual cross-validation method. Unfortunately, we observe that the function h → CV DM (h) is most often decreasing in the range of realistic bandwidth values. If we remove the weight K h (Z i -Z j ), then there is no reason why g k (X i , X j ) should be equal on average to τ (k) -(i,j), 1,2|Z=(Zi+Zj )/2 , and we are not interested in the prediction of concordance/discordance pairs for which the Z i and Z j are far apart.

Therefore, a modification of this criteria is necessary. We propose to separate the choice of h for the terms g k (X i , X j )-τ (h, k) -(i,j), 1,2|Z=(Zi+Zj )/2 and the selection of the "convenient pairs" of observations (i, j). This leads to the new criterion

CV h(h) := 2 n(n -1) n i,j=1 g k (X i , X j ) - τ (h, k) -(i,j), 1,2|Z=(Zi+Zj )/2 2 Kh (Z i -Z j ), (5.4)
with a potentially different kernel K and a new fixed tuning parameter h. Even if more complex procedures are possible, we suggest to simply choose K(z) := 1{|z| ∞ ≤ 1} and to calibrate h so that only a fraction of the pairs (i, j) have non-zero weight. In practice, set h as the empirical quantile of

{|Z i -Z j | ∞ : 1 ≤ i < j = n} of order 2N pairs /(n(n -1))
, where N pairs is the number of pairs we want to keep.

Note that such dependence measures are of interest for the purpose of estimating (conditional or unconditional) copula models too. Indeed, several popular parametric families of copulas have a simple one-to-one mapping between their parameter and the associated Kendall's tau (or Spearman's rho):

Gaussian, Student with a fixed degree of freedom, Clayton, Gumbel and Frank copulas, etc. Then, assume for instance that the conditional copula C 1,2|Z=z belongs is a Gaussian copula with a parameter ρ(z). Then, by estimating its conditional Kendall's tau τ 1,2|Z=z , we get an estimate of the corresponding parameter ρ(z), and finally of the conditional copula itself. See [START_REF] Sabeti | Additive models for conditional copulas[END_REF], e.g.

Theoretical results

Finite distance bounds

Hereafter, we will consider the behavior of conditional Kendall's tau estimates given Z = z belongs to some fixed open subset Z in R p . For the moment, let us state an instrumental result that is of interest per Chapter 5. About kernel-based estimation of conditional Kendall's tau: finite-distance bounds and asymptotic behavior 130 se. Let fZ (z) := n -1 n j=1 K h (Z j -z) be the usual kernel estimator of the density f Z of the conditioning variable Z. Note that the estimators τ (k) 1,2|Z=z , k = 1, . . . , 3 are well-behaved only whenever fZ (z) > 0. Denote the joint density of (X, Z) by f X,Z . In our study, we need some usual conditions of regularity. Assumption 5.3.1. The kernel K is bounded, and set K ∞ =: C K . It is symmetrical and satisfies

K = 1, |K| < ∞.
This kernel is of order α for some integer α > 1: for all j = 1, . . . , α -1 and every indices i 1 , . . . , i j in {1, . . . , p}, K(u)u i1 . . . u ij du = 0. Moreover, IE[K h (Z -z)] > 0 for every z ∈ Z and

h > 0. Set K(•) := K 2 (•)/ K 2 and K ∞ =: C K .
Assumption 5.3.2. f Z is α-times continuously differentiable on Z and there exists a constant C K,α > 0

s.t., for all z ∈ Z, |K|(u) p i1,...,iα=1 |u i1 . . . u iα | sup t∈[0,1] ∂ α f Z ∂z i1 . . . ∂z iα (z + thu) du ≤ C K,α .
Moreover, C K,2 denotes a similar constant replacing K by K and α by two.

Assumption 5.3.3.

There exist two positive constants f Z,min and f Z,max such that, for every z ∈ Z,

f Z,min ≤ f Z (z) ≤ f Z,max .
Proposition 5.2. Under Assumptions 5.3.1-5.3.3 and if C K,α h α /α! < f Z,min , for any z ∈ Z, the estimator fZ (z) is strictly positive with a probability larger than

1 -2 exp -nh p f Z,min -C K,α h α /α! 2 / 2f Z,max K 2 + (2/3)C K (f Z,min -C K,α h α /α!) .
The latter proposition is proved in Section 5.5.2. It guarantees that our estimators τ (k) 1,2|z , k = 1, . . . , 3, are well-behaved with a probability close to one.

The next regularity assumption is necessary to explicitly control the bias of τ1,2|Z=z . Assumption 5.3.4. For every x ∈ R 2 , z → f X,Z (x, z) is differentiable on Z almost everywhere up to the order α. For every 0 ≤ k ≤ α and every 1 ≤ i 1 , . . . , i α ≤ p, let

H k, ι (u, v, x 1 , x 2 , z) := sup t∈[0,1] ∂ k f X,Z ∂z i1 . . . ∂z i k x 1 , z + thu ∂ α-k f X,Z ∂z i k+1 . . . ∂z iα x 2 , z + thv , denoting ι = (i 1 , . . . , i α ). Assume that H k, ι (u, v, x 1 , x 2 , z
) is integrable and there exists a finite constant C XZ,α > 0 such that, for every z ∈ Z and every h < 1,

|K|(u)|K|(v) α k=0 α k p i1,...,iα=1 H k, ι (u, v, x 1 , x 2 , z)|u i1 . . . u i k v i k+1 . . . v iα | du dv dx 1 dx 2 is less than C XZ,α .
The next three propositions state pointwise and inform exponential inequalities for the estimators

τ (k) 1,2|Z=z , when k = 1,
t > 0 such that C K,α h α /α! + t ≤ f Z,min /2 and every t > 0, if C K,2 h 2 < f z (z), we have IP |τ (k) 1,2|Z=z -τ 1,2|Z=z | > c k f 2 z (z) C XZ,α h α α! + 3f z (z) K 2 2nh p + t × 1 + 16f 2 Z (z) f 3 Z,min C K,α h α α! + t ≤ 2 exp - nh p t 2 2f Z,max K 2 + (2/3)C K t + 2 exp - (n -1)h 2p t 2 4f 2 Z,max ( K 2 ) 2 + (8/3)C 2 K t + 2 exp - nh p (f z (z) -C K,2 h 2 ) 2 8f Z,max K2 + 4C K (f z (z) -C K,2 h 2 )/3 ,
for any z ∈ Z and every k = 1, 2, 3.

Chapter 5. About kernel-based estimation of conditional Kendall's tau: finite-distance bounds and asymptotic behavior 131

Alternatively, we could obtain a better rate of approximation, at the price of managing unknown (universal) constants instead of explicit constants.

IP |τ (k) 1,2|Z=z -τ 1,2|Z=z | > c k f 2 z (z) C XZ,α h α α! + 3f z (z) K 2 2nh p + t × 1 + 16f 2 Z (z) f 3 Z,min C K,α h α α! + t ≤ 2 exp - nh p t 2 2f Z,max K 2 + (2/3)C K t + 2 exp - nh p (f z (z) -C K,2 h 2 ) 2 8f Z,max K2 + 4C K (f z (z) -C K,2 h 2 )/3 + 2 exp nh p t 2 32 K 2 ( |K|) 2 f 3 Z,max + 8C K |K|f Z,max t/3 + C 2 exp - α 2 nh p t 8f Z,max ( K 2 )
, As a corollary, the two latter result yield the weak consistency of τ (k) 1,2|Z=z for every z ∈ Z, when nh 2p → ∞ (choose the constants t and t ∼ h p sufficiently small, in Proposition 5.4, e.g.).

for any z ∈ Z and every k = 1, 2, 3, if C K,2 h 2 < f Z (z)
It is possible to obtain uniform bounds, by slightly strengthening our assumptions. Note that this next result will be true if n is sufficiently large, when Proposition 5.4 was true for every n. Assumption 5.3.5. The kernel K is Lipschitz on (Z, • ∞ ), with a constant λ K and Z is a subset of an hypercube in R p whose volume is denoted by V. Moreover, K and K 2 are regular in the sense of [START_REF] Giné | Rates of strong uniform consistency for multivariate kernel density estimators[END_REF] or [START_REF] Einmahl | Uniform in bandwidth consistency of kernel-type function estimators[END_REF]. Proposition 5.6 (Uniform exponential bound). Under the assumptions 5.3.1-5.3.5, there exist some constants L K and C K (resp. L K and C K ) that depend only on the VC characteristics of K (resp. K),

s.t., for every µ ∈ (0, 1) such that µf z,min < C XZ,α h α /α! + b K K 2 f Z,max /C K , if f Z,max < CXZ,2 h 2 /2 + b K K2 f Z,max /C K , IP sup z∈Z |τ (k) 1,2|Z=z -τ 1,2|Z=z | > c k f 2 z,min (1 -µ) 2 C XZ,α h α α! + 3f z,max K 2 2nh p + t ≤ L K exp -C f,K nh p µf z,min - C XZ,α h α α! 2 + C 2 D exp - α 2 nth p 8f Z,max ( K 2 ) + L K exp -C f, K nh p (f z,max -CXZ,2 h 2 ) 2 /4 + 2 exp - A 2 nh p t 2 C -4 K 16 2 A 2 1 K 2 f 3 z,max ( |K|) 2 + 2 exp(- A 2 nh p t 16C 2 K A 1
),

for n sufficiently large, k = 1, 2, 3, and for every t > 0 s.t.

t ≤ 2h p ( K 2 ) 3 f 3 Z,max /C 4 K , -16A 1 C 2 K A g K 2 f 3 z,max ( |K|) 2 n 1/2 h p/2 ln(h p K 2 f 3 z,max ( |K|) 2 ) < t, and
nh p t ≥ K 2 f z,max M 2 (p + β) 3/2 log 4C 2 K h p f z,max K 2 , β = max 0, log D log n , D := V 4C K λ K h p ,
for some universal constants C 2 , α 2 , M 2 , A 1 , A 2 and a constant A g that depends on K and f z,max .

Chapter 5. About kernel-based estimation of conditional Kendall's tau: finite-distance bounds and asymptotic behavior 132

We have denoted

C f,K := log(1 + b K /(4L K ))/(L K b K f z,max K 2 ), for an arbitrarily chosen positive constant b K ≥ C K . Similarly, C f, K := log(1 + b K /(4L K ))/(L K b K f z,max K2 ), b K ≥ C K .

Asymptotic behavior

The previous exponential inequalities are not optimal to prove asymptotic results. Indeed, they directly or indirectly rely on upper bounds, as in Hoeffding or Bernstein-type inequalities. In the case of kernel estimates, this implies the necessary condition nh 2p → ∞, at least. By a direct approach, it is possible to state the consistency of

τ (k) 1,2|Z=z , k = 1, 2,
Z,min s.t. f Z,min ≤ f Z (z) for any z ∈ Z. Then sup z∈Z τ (k) 1,2|Z=z -τ 1,2|Z=z → 0 almost surely, when n → ∞ for any k = 1, 2, 3.
This property is proved in Section 5.5.7. To derive the asymptotic law of this estimator, we will

(nh p n ) 1/2 τ1,2|Z=z i -τ 1,2|Z=z i i=1,...,n D -→ N (0, H (k) ), k = 1, 2, 3,
where τ1,2|Z=z denotes any of the estimators

τ (k) 1,2|Z=z , k = 1, 2, 3 or τ1,2|Z=z
, and H is the n × n diagonal real matrix defined by

[H (k) ] i,j = 4 K 2 1 {i=j} f Z (z i ) IE[g k (X 1 , X)g k (X 2 , X)|Z = Z 1 = Z 2 = z i ] -τ 2 1,2|Z=z i ,
for every 1 ≤ i, j ≤ n , and (X, Z), (X 1 , Z 1 ), (X 2 , Z 2 ) are independent versions.

This proposition is proved in Section 5.5.8.

Remark 5.10. The latter results will provide some simple tests of the constancy of the function z → τ 1,2|z , and then of the constancy of the associated conditional copula itself. This would test the famous "simplifying assumption" ("H 0 : C 1,2|Z=z does not depend on the choice of z"), a key assumption for vine modeling in particular: see [START_REF] Acar | Beyond simplified pair-copula constructions[END_REF] or [START_REF] Hobaek Haff | On the simplified pair-copula construction-simply useful or too simplistic?[END_REF] for a discussion, [START_REF] Derumigny | About tests of the "simplifying" assumption for conditional copulas[END_REF] for a review and a presentation of formal tests for this hypothesis.

Chapter 5. About kernel-based estimation of conditional Kendall's tau: finite-distance bounds and asymptotic behavior 133

Simulation study

In this simulation study, we draw i.i.d. random samples (X i,1 , X i,2 , Z i ), i = 1, . . . , n, with univariate explanatory variables (p = 1). We consider two settings, that correspond to bounded and/or unbounded explanatory variables respectively:

1. Z =]0, 1[ and the law of Z is uniform on ]0, 1[. Conditionally on Z = z, X 1 |Z = z and X 2 |Z = z both follow a Gaussian distribution N (z, 1). Their associated conditional copula is Gaussian and their conditional Kendall's tau is given by τ 1,2|Z=z = 2z -1.

2. Z = R and the law of Z is N (0, 1). Conditionally on Z = z, X 1 |Z = z and X 2 |Z = z both follow a Gaussian distribution N (Φ(z), 1), where Φ(•) is the cdf of the Z. Their associated conditional copula is Gaussian and their conditional Kendall's tau is given by τ 1,2|Z=z = 2Φ(z) -1.

These simple frameworks allow us to compare the numerical properties of our different estimators in different parts of the space, in particular when Z is close to zero or one, i.e. when the conditional Kendall's tau is close to -1 or to 1. We compute the different estimators τ (k) 1,2|Z=z for k = 1, 2, 3, and the symmetrically rescaled version τ1,2|z . The bandwidth h is chosen as proportional to the usual "ruleof-thumb" for kernel density estimation, i.e. h = α h σ(Z)n -1/5 with α h ∈ {0.5, 0.75, 1, 1.5, 2} and n ∈ {100, 500, 1000, 2000}. For each setting, we consider three local measures of goodness-of-fit: for a given z and for any Kendall's tau estimate (say τ1,2|Z=z ), let

• the (local) bias: Bias(z) := IE[τ 1,2|Z=z ] -τ 1,2|Z=z , • the (local) standard deviation: Sd(z) := IE τ1,2|Z=z -IE[τ 1,2|Z=z ] 2 1/2 ,
• the (local) mean square-error: M SE(z) := IE τ1,2|Z=z -τ 1,2|Z=z 2 .

We also consider their integrated version w.r.t the usual Lebesgue measure on the whole support of z, respectively denoted by IBias, ISd and IM SE. Some results concerning these integrated measures are given in Table 5.1 (resp. Table 5.2) for Setting 1 (resp. Setting 2), and for different choices of α h and n. For the sake of effective calculations of these measures, all the theoretical previous expectations are replaced by their empirical counterparts based on 500 simulations.

For every n, the best results seem to be obtained with α h = 1.5 and the fourth (rescaled) estimator, particularly in terms of bias. This is not so surprising, because the estimators τ (k) , k = 1, 2, 3, do not have the right support at a finite distance. Note that this comparative advantage of τ in terms of bias decreases with n, as expected. In terms of integrated variance, all the considered estimators behave more or less similarly, particularly when n ≥ 500.

To illustrate our results for Setting 1 (resp. Setting 2), the functions z → Bias(z), Sd(z) and M SE(z)

have been plotted on Figures 5.1 1,2|z ) is a lot larger. Since the squared biases are here significantly larger than the variances in the tails, τ1,2|z provides the best estimator globally considering "both sides" together. But even in the center of Z's distribution, the latter estimator behaves very well.

In Setting 2 where there is no boundary problem, we also try to estimate the conditional Kendall's tau using our cross-validation criterion (5.4), with N pairs = 1000. More precisely, denoting by h CV the minimizer of the cross-validation criterion, we try different choices h = α h ×h CV with α h ∈ {0.5, 0.75, 1, 1.5, 2}.

The results in terms of integrated bias, standard deviation and MSE are given in Table 5.3. We do not find any substantial improvements compared to the previous Table 5. 2, where the bandwidth was chosen "roughly". In Table 5.4, we compare the average h CV with the previous choice of h. The expectation of h CV is always higher than the "rule-of-thumb" h ref , but the difference between both decreases when the sample size n increases. The standard deviation of h CV is quite high for low values of n, but decreases as a function of n. This may be seen as quite surprising given the fact that the number of pairs N pairs used in the computation of the criterion stays constant. Nevertheless, when the sample size increases, the selected pairs are better in the sense that the differences |Z i -Z j | can become smaller as more replications of Z i are available.

Proofs

For convenience, we recall Berk's (1970) inequality (see Theorem A in Serfling [123, p.201]). Note that, if m = 1, this reduces to Bernstein's inequality.

IP n m -1 c g(X i1 , . . . , X im ) -θ ≥ t ≤ exp - [n/m]t 2 2σ 2 + (2/3)(b -θ)t ,
where c denotes summation over all subgroups of m distinct integers (i 1 , . . . , i m ) of {1, . . . n}.

Proof of Proposition 5.1

Since there are no ties a.s.,

1 + τ (1) 1,2|Z=z = 4 n i=1 n j=1 w i,n (z)w j,n (z) 1 X i,1 < X j,1 -1 X i,1 < X j,1 , X i,2 > X j,2 = 4 n i=1 n j=1 w i,n (z)w j,n (z)1 X i,1 < X j,1 + τ (3) 1,2|Z=z -1. But 1 = n i=1 n j=1 w i,n (z)w j,n (z) = n i=1 n j=1 w i,n (z)w j,n (z) 1 X i,1 ≤ X j,1 + 1 X i,1 > X j,1 = 2 n i=1 n j=1 w i,n (z)w j,n (z)1 X i,1 < X j,1 + n i=1 w 2 i,n , implying that 1 + τ (1) 1,2|Z=z = 4 1 -s n 2 + τ (
τ (2) 1,2|Z=z = n i=1 n j=1 w i,n (z)w j,n (z) 1 X i,1 > X j,1 , X i,2 > X j,2 + 1 X i,1 < X j,1 , X i,2 < X j,2 -1 X i,1 > X j,1 , X i,2 < X j,2 -1 X i,1 < X j,1 , X i,2 > X j,2 = 2 n i=1 n j=1 w i,n (z)w j,n (z) 1 X i,1 > X j,1 , X i,2 > X j,2 -1 X i,1 > X j,1 , X i,2 < X j,2 = 1 2 τ (1) 1,2|Z=z + 1 + 1 2 τ (3) 1,2|Z=z -1 = τ (1) 1,2|Z=z + τ (3) 1,2|Z=z 2 = τ (1) 1,2|Z=z + s n = τ (3) 1,2|Z=z -s n .

Proof of

IP fZ (z) -f Z (z) ≥ C K,α h α α! + t ≤ 2 exp - nh p t 2 2f Z,max K 2 + (2/3)C K t .
This Lemma is proved below. If, for some > 0, we have

C K,α h α /α!+t ≤ f Z,min -, then f (z) ≥ > 0
with a probability larger than 1 -2 exp -nh p t 2 /(2f Z,max K 2 + (2/3)C K t) . So, we should choose the largest t as possible, which yields Proposition 5.2.

It remains to prove Lemma 5.12. Use the usual decomposition between a stochastic component and a bias:

fZ (z) -f Z (z) = fZ (z) -IE[ fZ (z)] + IE[ fZ (z)] -f Z (z)
. We first bound the bias from above.

IE[ fZ (z)] -f Z (z) = R p K(u) f Z z + hu -f Z (z) du. Set φ z,u (t) := f Z z + thu for t ∈ [0, 1]
. This function has at least the same regularity as f Z , so it is α-differentiable. By a Taylor-Lagrange expansion, we get

R p K(u) f Z z + hu -f Z (z) du = R p K(u) α-1 i=1 1 i! φ (i) z,u (0) + 1 α! φ (α) z,u (t z,u ) du,
for some real number t z,u ∈ (0, 1). By Assumption 5.3.1 and for every i < α, R p K(u)φ

(i) z,u (0) du = 0. Therefore, IE[ fZ (z)] -f Z (z) = R p K(u) 1 α! φ (α) z,u (t z,u )du = 1 α! R p K(u) p i1,...,iα=1 h α u i1 . . . u iα ∂ α f Z ∂z i1 . . . ∂z iα z + t z,u hu du ≤ C K,α α! h α .
Second, the stochastic component may be written as

fZ (z) -IE[ fZ (z)] = n -1 n i=1 K h (Z i -z) -IE n -1 n i=1 K h (Z i -z) = n -1 n i=1 g z (Z i ) -IE[g(Z i )] ,
where g(Z i ) := K h (Z i -z). Apply Lemma 5.11 with m = 1 and the latter g(Z i ). Here, we have

b = -a = h -p C K , θ = IE[g(Z 1 )] ≥ 0 and V ar[g(Z 1 )] ≤ h -p f Z,max K 2 ,

and we get

IP 1 n n i=1 K h (Z i -z) -IE[K h (Z i -z)] ≥ t ≤ 2 exp - nt 2 2h -p f Z,max K 2 + (2/3)h -p C K t .
Chapter 5. About kernel-based estimation of conditional Kendall's tau: finite-distance bounds and asymptotic behavior 141

Proof of Proposition 5.3

We show the result for k = 1. The two other cases can be proven in the same way.

Consider the decomposition

τ1,2|Z=z -τ 1,2|Z=z = 4 1≤i,j≤n w i,n (z)w j,n (z)1 X i < X j -4IP X 1 < X 2 Z 1 = Z 2 = z = 4 n 2 f 2 Z (z) 1≤i,j≤n K h (Z i -z)K h (Z j -z) 1 X i < X j -IP X 1 < X 2 Z 1 = Z 2 = z =: 4 f 2 Z (z) 1≤i,j≤n S i,j (z).
Therefore, for any positive numbers x and λ(z), we have

IP(|τ 1,2|Z=z -τ 1,2|Z=z | > x) ≤ IP 1 f 2 Z (z) > 1 + λ(z) f 2 Z (z) + IP 4(1 + λ(z)) f 2 Z (z) × | 1≤i,j≤n S i,j (z)| > x ≤ IP | 1 f 2 Z (z) - 1 f 2 Z (z) | > λ(z) f 2 Z (z) + IP 4(1 + λ(z)) f 2 Z (z) × | 1≤i,j≤n S i,j (z)| > x . For any t s.t. C K,α h α /α! + t < f Z,min /2, set λ(z) = 16f 2 z (z) f 3 Z,min C K,α h α α! + t .
Then, this yields

IP |τ 1,2|Z=z -τ 1,2|Z=z | > x ≤ IP | 1 f 2 Z (z) - 1 f 2 Z (z) | > 16 f 3 Z,min C K,α h α α! + t + IP | 1≤i,j≤n S i,j (z)| > f 2 z (z)x 4(1 + λ(z))
.

By setting

x = 4 f 2 z (z) C XZ,α h α α! + 3f z (z) K 2 2nh p + t 1 + 16f 2 Z (z) f 3 Z,min C K,α h α α! + t ,
and applying the next two lemmas 5.13 and 5.14, we get the result.

Lemma 5.13. Under Assumptions 5.3.1-5.3.3 and if

C K,α h α /α! + t < f Z,min /2 for some t > 0, IP | 1 f 2 Z (z) - 1 f 2 Z (z) | > 16 f 3 Z,min C K,α h α α! + t ≤ 2 exp - nh p t 2 2f Z,max K 2 + (2/3)C K t ,
and fZ (z) is strictly positive on these events.

Proof : Applying the mean value inequality to the function x → 1/x 2 , we get the inequality

1/ f 2 Z (z) - 1/f 2 Z (z) ≤ 2 fZ (z) -f Z (z) /f * Z 3
, where f * Z lies between fZ (z) and f Z (z). Denote by E the event E := | fZ (z) -f Z (z)| ≤ C K,α h α /α! + t . By Lemma 5.12, we obtain

IP(E) ≥ 1 -2 exp - nh p t 2 2f Z,max K 2 + (2/3)C K t . (5.5)
Therefore, on this event E, fZ (z) -f Z (z) ≤ f Z,min /2, so that f Z,min /2 ≤ fZ (z). We have also f Z,min /2 ≤ f Z (z) and then f Z,min /2 ≤ f * Z . Combining the previous inequalities, we finally get

1 f 2 Z (z) - 1 f 2 Z (z) ≤ 16 f 3 Z,min fZ (z) -f Z (z) ≤ 16 f 3 Z,min C K,α h α α! + t ,
Chapter 5. About kernel-based estimation of conditional Kendall's tau: finite-distance bounds and asymptotic behavior 142 on E. But since

IP | 1 f 2 Z (z) - 1 f 2 Z (z) | > 16 f 3 Z,min C K,α h α α! + t ≤ IP(E c ),
we deduce the result.

S i,j (z) > C XZ,α h α α! + 3f z (z) K 2 2nh p + t ≤ 2 exp - (n -1)h 2p t 2 4f 2 Z,max ( K 2 ) 2 + (8/3)C 2 K t + 2 exp - nh p (f z (z) -C K,2 h 2 ) 2 8f Z,max K2 + 4C K (f z (z) -C K,2 h 2 )/3 . Proof : Note that 1≤i,j≤n S i,j (z) = 1≤i =j≤n S i,j (z)-IE[S i,j (z)] +n(n-1)IE[S 1,2 (z)]+ n i=1 S i,i (z). The "diagonal term" n i=1 S i,i (z) = -IP X 1 < X 2 Z 1 = Z 2 = z n i=1 K 2 h (Z i -z)
/n 2 is negative and negligible. It will be denoted by -∆ n (z) < 0. Note that K(•) := K 2 (•)/ K 2 is a two-order kernel. Then, fz (z) := n i=1 Kh (Z i -z)/n is a consistent estimator of f Z (z). Therefore, due to Lemma 5.12 and with obvious notations, we have for every ε > 0

IP fZ (z) -f Z (z) ≥ C K,2 h 2 2 + ε ≤ 2 exp - nh p ε 2 2f Z,max K2 + (2/3)C K ε .
This implies

IP | K 2 n 2 h p n i=1 Kh (Z i -z) - f Z (z) K 2 nh p | ≥ K 2 nh p C K,2 h 2 2 + ε ≤ 2 exp - nh p ε 2 2f Z,max K2 + (2/3)C K ε .
By choosing ε s.t. C K,2 h 2 /2 + ε = f z (z)/2, ∆ n will be smaller than 3f z (z) K 2 /(2nh p ) with a probability that is larger than

1 -2 exp - nh p ε 2 2f Z,max K2 + (2/3)C K ε . (5.6)
Now, let us deal with the main term, that is decomposed as a stochastic component and a bias component. First, let us deal with the bias. Simple calculations provide, if i = j,

IE[S i,j (z)] = n -2 IE K h (Z i -z)K h (Z j -z) 1 X i < X j -IP X i < X j Z i = Z j = z = n -2 R 2p+2 K h (z 1 -z)K h (z 2 -z) 1 x 1 < x 2 -IP X i < X j Z i = Z j = z × f X,Z (x 1 , z 1 ) f X,Z (x 2 , z 2 ) dx 1 dz 1 dx 2 dz 2 = n -2 R 2p+2 K(u)K(v) 1 x 1 < x 2 -IP X i < X j Z i = Z j = z × f X,Z x 1 , z + hu f X,Z x 2 , z + hv -f X,Z (x 1 , z) f X,Z (x 2 , z) dx 1 du dx 2 dv,
because, for every z,

0 = R 4 1 x 1 < x 2 -IP X 1 < X 2 Z 1 = Z 2 = z f X,Z (x 1 , z)f X,Z (x 2 , z) dx 1 dx 2 .
Apply the Taylor-Lagrange formula to the function

φ x1,x2,u,v (t) := f X,Z x 1 , z + thu f X,Z x 2 , z + thv .
With obvious notation, this yields

IE[S i,j (z)] = n -2 K(u)K(v) 1 x 1 < x 2 -IP X i < X j Z i = Z j = z
Chapter 5. About kernel-based estimation of conditional Kendall's tau: finite-distance bounds and asymptotic behavior 143

× α-1 k=1 1 k! φ (k) x1,x2,u,v (0) + 1 α! φ (α) x1,x2,u,v (t x1,x2,u,v ) dx 1 du dx 2 dv = K(u)K(v) n 2 α! 1 x 1 < x 2 -IP X i < X j Z i = Z j = z φ (α) x1,x2,u,v (t x1,x2,u,v )dx 1 du dx 2 dv. Since φ (α) x1,x2,u,v (t) is equal to α k=0 α k p i1,...,iα=1 h α u i1 . . . u i k v i k+1 . . . v iα ∂ k f X,Z ∂z i1 . . . ∂z i k x 1 , z + thu ∂ α-k f X,Z ∂z i k+1 . . . ∂z iα
x 2 , z + thv , using Assumption 5.3.4, we get

IE[S 1,2 (z)] ≤ C XZ,α h α /(n 2 α!).
(5.7)

Second, the stochastic component will be bounded from above. Indeed, 1≤i =j≤n

(S i,j (z) -IE[S i,j (z)]) = 1 n 2 1≤i =j≤n g z (X i , Z i ) , (X j , Z j ) ,
with the function g z defined by

g z (X i , Z i ), (X j , Z j ) := K h (Z i -z)K h (Z j -z) 1 X i < X j -IP X i < X j Z i = Z j = z -IE K h (Z i -z)K h (Z j -z) 1 X i < X j -IP X i < X j Z i = Z j = z .
The symmetrized version of g is gi,j = g z (X i , Z i ) , (X j , Z j ) + g z (X j , Z j ) , (X i , Z i ) /2. We can now apply Lemma 5.11 to the sum of the gi,j . With its notation, θ = IE gi,j = 0. Moreover, V ar g z (X i , Z i ), (X j , Z j )

≤ K 2 h (z 1 -z)K 2 h (z 2 -z) 1 x 1 < x 2 -IP X i < X j Z i = Z j = z 2 × f X,Z (x 1 , z 1 )f X,Z (x 2 , z 2 ) dx 1 dx 2 dz 1 dz 2 ≤ K 2 (t 1 )K 2 (t 2 ) h 2p f X,Z (x 1 , z -ht 1 )f X,Z (x 2 , z -ht 2 ) dx 1 dx 2 dt 1 dt 2 ≤ h -2p f 2 Z,max K 2 2 ,
and the same upper bound applies for gi,j (invoke Cauchy-Schwarz inequality). Here, we choose b =

-a = 2C 2 K h -2p . This yields IP 2 n(n -1) 1≤i<j≤n gi,j > t ≤ exp - [n/2]t 2 2h -2p f 2 Z,max ( K 2 ) 2 + (4/3)C 2 K h -2p t (5.8)
Then, for every t > 0, we obtain

IP | 1≤i =j≤n S i,j (z) -IE[S i,j (z)] | ≥ t ≤ IP 1 n 2 | 1≤i =j≤n g z (X i , Z i ) , (X j , Z j )| ≥ t ≤ IP (n -1) n × 2 n(n -1) | 1≤i<j≤n gi,j | ≥ t ≤ 2 exp - [n/2]t 2 2h -2p f 2 Z,max ( K 2 ) 2 + (4/3)C 2 K h -2p t .
The latter inequality, (5.6) and (5.7) yield the result.

Chapter 5. About kernel-based estimation of conditional Kendall's tau: finite-distance bounds and asymptotic behavior 144

Proof of Proposition 5.4

Alternatively, we can apply Theorem 1 in Major [START_REF] Major | An estimate on the supremum of a nice class of stochastic integrals and U-statistics[END_REF] instead of the Bernstein-type inequality that has been used in the proof of Proposition 5.3. With the notations of this proof, this will yield the following lemma, that straightforwardly implies the result.

Lemma 5.15. Under Assumptions 5.3.1-5.3.4 and when t ≤ 2h p ( K 2 )

3 f 3 Z,max /C 4 K , 6h p f Z,max |K| 2 < K 2 and C K,2 h 2 < f z (z), we have IP 1≤i,j≤n S i,j (z) > C XZ,α h α α! + 3f z (z) K 2 2nh p + t ≤ C 2 exp - α 2 nh p t 8f Z,max ( K 2 ) + 2 exp - nh p (f z (z) -C K,2 h 2 ) 2 8f Z,max K2 + 4C K (f z (z) -C K,2 h 2 )/3 + 2 exp nh p t 2 32 K 2 ( |K|) 2 f 3 Z,max + 8C K |K|f Z,max t/3
for some universal positive constants C 2 and α 2 .

Proof : We lead exactly the same reasoning and the same notations as in Lemma 5.14, until (5.8).

Now, with the same notations, introduce g i := IE[g i,j |X i , Z i ] and consider ξ i,j := gi,j -g i -g j . Then,

ξ i,j is a degenerate (symmetrical) U-statistics because IE[ξ i,j |X i , Z i ] = IE[ξ i,j |X j , Z j ] = 0, when i = j.
Actually ξ i,j =: ξ z (X i , Z i , X j , Z j ) for some function ξ z and set

z : (x 1 , z 1 , x 2 , z 2 ) → h 2p 4C 2 K ξ z (x 1 , z 1 ) , (x 2 , z 2 ) , (5.9)
for a fixed z and a fixed h. This yields z ∞ ≤ 1 and, by usual changes of variables, we obtain

2 z (x 1 , z 1 , x 2 , z 2 ) f X,Z (x 1 , z 1 )f X,Z (x 2 , z 2 ) dx 1 dx 2 dz 1 dz 2 ≤ 3h 2p ( K 2 f z,max ) 2 (4C 2 K ) 2 + 6h 3p K 2 f z,max ( |K|f z,max ) 2 (4C 2 K ) 2 ≤ σ 2 , with σ := h p C σ , C σ := K 2 f z,max 2C 2 K , (5.10) because 6h p K 2 f z,max ( |K|f z,max ) 2 ≤ ( K 2 f z,max ) 2 .
With the notations of [START_REF] Major | An estimate on the supremum of a nice class of stochastic integrals and U-statistics[END_REF], this implies D = 1, m = 1 and L is arbitrarily small. Therefore, Theorem 2 in [START_REF] Major | An estimate on the supremum of a nice class of stochastic integrals and U-statistics[END_REF] yields

IP 1 2n | i =j z (X i , Z i , X j , Z j )| > x ≤ C 2 exp - α 2 x σ , (5.11)
for some universal constants C 2 and α 2 when x ≤ nσ 3 . By setting t/2 = 4C 2 K x/(nh 2p ) and applying Lemma 5.11, this provides

IP | 1≤i =j≤n S i,j (z) -IE[S i,j (z)] | ≥ t ≤ IP 1 n 2 | 1≤i =j≤n ξ ij | ≥ t/2 + IP | 1 n n i=1 g i | ≥ t/4 ≤ C 2 exp - α 2 nth p 8f Z,max ( K 2 ) + 2 exp nh p t 2 32 K 2 ( |K|) 2 f 3 Z,max + 8/3C K |K|f Z,max t when t ≤ 2h p ( K 2 ) 3 f 3 Z,max /C 4 K .
The latter inequality, (5.6) and (5.7) conclude the proof.

Chapter 5. About kernel-based estimation of conditional Kendall's tau: finite-distance bounds and asymptotic behavior 145

Proof of Proposition 5.6

For k = 1, we follow the paths of the proof of Proposition 5.4. Since τ1,2|Z=z -τ 1,2|Z=z = 4 1≤i,j≤n S i,j (z)/ f 2 Z (z), we prove the result if we bound from above 1/ f 2 Z (z) and 1≤i,j≤n S i,j (z) uniformly w.r.t. z ∈ Z. To be specific, for any positive constant µ

< 1, if | fZ (z) -f Z (z)| ≤ µf z,min , then 1/ f 2 z,max (z) ≤ f -2 z,min (1 -µ) -2 . We deduce IP( sup z∈Z |τ 1,2|Z=z -τ 1,2|Z=z | > x) ≤ IP fZ -f Z ∞ > µf z,min + IP( 4 f 2 Z,min (1 -µ) 2 sup z∈Z | 1≤i,j≤n S i,j (z)| > x).
First invoke the uniform exponential inequality, as stated in [START_REF] Rinaldo | Generalized density clustering[END_REF], Proposition 9: for every

ε < b K K 2 f Z,max /C K , IP fZ -f Z ∞ > ε + C XZ,α h α α! ≤ IP fZ -IE[ fZ ] ∞ > ε ≤ L K exp -C f,K nh p ε 2 , (5.12)
for n sufficiently large. Then, apply Lemma 5.16, by setting (x, ε) so that

x = 4 f 2 z,min (1 -µ) 2 C XZ,α h α α! + 3f z,max K 2 2nh p + t and ε + C XZ,α h α α! = µf z,min .
Lemma 5.16. Under the assumptions of Proposition 5.6, we have

IP sup z∈Z 1≤i,j≤n S i,j (z) > C XZ,α h α α! + 3f z,max K 2 2nh p + t ≤ C 2 D exp - α 2 nth p 8f Z,max ( K 2 ) + L K exp -C f, K nh p (f z,max -CXZ,2 h 2 ) 2 /4 + 2 exp - A 2 nh p t 2 C -4 K 16 2 A 2 1 K 2 f 3 z,max ( |K|) 2 + 2 exp(- A 2 nh p t 16C 2 K A 1
),

when t ≤ 2h p ( K 2 ) 3 f 3 Z,max /C 4 K , -16A 1 C 2 K A g K 2 f 3 z,max ( |K|) 2 n 1/2 h p/2 ln(h p K 2 f 3 z,max ( |K|) 2 ) < t, and
nh p t ≥ K 2 f z,max M 2 (p + β) 3/2 log 4C 2 K h p f z,max K 2 , β = max 0, log D log n , D := V 4C K λ K h p ,
for some universal constants C 2 , α 2 , M 2 , A 1 , A 2 and a constant A g that depends on K and f z,max .

Proof : We will use the arguments and notations of the proof of Lemmas 5.14 and 5.15. We still invoke the decomposition 1≤i,j≤n S i,j (z) = 1≤i =j≤n S i,j (z)-IE[S i,j (z)] +n(n-1)IE[S 1,2 (z)]+ n i=1 S i,i (z). First let us find a uniform bound for the "diagonal term" ∆ n (z) = n i=1 S i,i (z) = K 2 fz (z)/(nh p ). As in (5.12), for every

ε < b K K2 f Z,max /C K , IP fZ -f Z ∞ > ε + CXZ,2 h 2 2 ≤ L K exp -C f, K nh p ε 2 ,
for n sufficiently large. This implies

IP sup z∈Z | K 2 n 2 h p n i=1 Kh (Z i -z) - f Z (z) K 2 nh p | ≥ K 2 nh p ε + CXZ,2 h 2 2 ≤ L K exp -C f, K nh p ε 2 .
Chapter with a probability that is larger than

1 -L K exp -C f, K nh p ε 2 . (5.13)
Moreover, it is easy to see that

sup z∈Z IE[S 1,2 (z)] ≤ C XZ,α h α /(n 2 α!). (5.14)
With the same notations as in the proof of Lemma 5.15, the stochastic component will be driven by 1≤i =j≤n

(S i,j (z) -IE[S i,j (z)]) = 1 n 2 1≤i =j≤n g z (X i , Z i ) , (X j , Z j ) = 1 n 2 1≤i =j≤n gi,j = 1 n 2 1≤i =j≤n ξ i,j + 2(n -1) n 2 n i=1 g i .
Now apply Theorem 1 in [START_REF] Major | An estimate on the supremum of a nice class of stochastic integrals and U-statistics[END_REF], by recalling (5.9) and considering the family F := z , z ∈ Z , for a fixed bandwidth h. The constant σ has the same value as in (5.10). It is easy to check that the latter class of functions is L 2 dense (see [START_REF] Major | An estimate on the supremum of a nice class of stochastic integrals and U-statistics[END_REF]). Set ε ∈ (0, 1). Since K is λ K -Lipschitz, every function z ∈ F can be approximated in L 2 by a function zj ∈ F, for some j ∈ {1, . . . , m}

s.t. | z -zi | 2 dν ≤ ε 2 ,
for any probability measure ν.

Indeed, | z -zi | 2 dν ≤ 64λ 2 K z -z j 2 ∞ C 2 K h -2 that is less than ε 2 , if we cover Z by a grid of m points (z j ) in Z s.t. z -z j ∞ ≤ εh/(8C K λ K ) := εδ. This can be done with m ≤ ε -p p k=1 (b k -a k )/δ = ε -p Vδ -p points.
Then, with the notations of [START_REF] Major | An estimate on the supremum of a nice class of stochastic integrals and U-statistics[END_REF], L = p and D = V(8C K λ K /h) p . As above, this yields

IP sup z∈Z 1 n 2 | 1≤i =j≤n ξ Z (X i , Z i , X j , Z j ) , (X j , Z j ) | > t ≤ C 2 D exp - α 2 nh p t 8( K 2 )f Z,max , (5.15)
when t ≤ 2h p ( K 2 ) 3 f 3 Z,max /C 4 K .
It remains to bound IP(sup z∈Z |n -1 n i=1 g i | > t/4). Consider the family of functions

F := {(x 1 , z 1 ) ∈ R × Z → h p 4C 2 K IE[g z (x 1 , z 1 , X, Z)], z ∈ Z}.
This family of functions is bounded is one and its variance is less than σ

2 := h p K 2 f 3 z,max |K| 2 .
Apply Propositions 9 and 10 in [START_REF] Fermanian | Single-index copulas[END_REF] that is coming from [START_REF] Einmahl | Uniform in bandwidth consistency of kernel-type function estimators[END_REF]: for some universal constants A 1 and A 2 , some constant A g that depends on K and f z,max (see Proposition 1 in [START_REF] Einmahl | Uniform in bandwidth consistency of kernel-type function estimators[END_REF]) and for every x > 0, IP sup

z∈Z h p 4C 2 K | n i=1 IE[g z (X i , Z i , X, Z)|X i , Z i ]| > A 1 x + A g n 1/2 σ ln(1/σ) ≤ 2 exp - A 2 x 2 nσ 2 + exp(-A 2 x) , or IP sup z∈Z 1 n | n i=1 g i | > 4A 1 C 2 K x - A g σ n 1/2 h p ln(σ) ≤ 2 exp - A 2 nh 2p x 2 σ 2 + 2 exp(-A 2 nh p x).
For any positive t s.t.

4A 1 C 2 K
(n -1)A g σ n 3/2 h p ln(1/σ) < t/8, note that we can find a real x > th p /(16C 2 K A 1 ). Then, we have

IP sup z∈Z (n -1) n 2 | n i=1 g i | > t 4 ≤ 2 exp - A 2 nh p t 2 C -4 K 16 2 A 2 1 K 2 f 3 z,max ( |K|) 2 + 2 exp(- A 2 nh p t 16C 2 K A 1
).

(5.16)

Chapter 5. About kernel-based estimation of conditional Kendall's tau: finite-distance bounds and asymptotic behavior 147

Therefore, for such t, we obtain from (5.16) and (5.15) that

IP sup z∈Z | 1≤i =j≤n S i,j (z) -IE[S i,j (z)] | ≥ t ≤ C 2 D exp - α 2 nh p t 8( K 2 )f Z,max + 2 exp - A 2 nh p t 2 C -4 K 15 2 A 2 1 K 2 f 3 z,max ( |K|) 2 + 2 exp(- A 2 nh p t 15C 2 K A 1
).

for sufficiently large integers n. The latter inequality, (5.13) and (5.14) yield the exponential upper bound.

Proof of Proposition 5.7

Let us note that τ 1,2|Z=z = IE g k (X 1 , X 2 ) Z 1 = z, Z 2 = z for every k = 1, 2, 3, and that our estimators with the weights (5.2) can be rewritten as

τ (k) 1,2|Z=z := U n (g k ) / {U n (1) + n } where U n (g) := 1 n(n -1)IE[K h (z -Z)] 2 1≤i =j≤n g(X i , X j )K h (z -Z i )K h (z -Z j ) =: 1 n(n -1) 1≤i =j≤n g i,j ,
for any measurable bounded function g, with the residual diagonal term n :=

n i=1 K 2 h (z -Z i )/{n(n - 1)IE[K h (z -Z)] 2 }
. By Bochner's lemma (see Bosq and Lecoutre [START_REF] Bosq | Théorie de l'estimation fonctionnelle[END_REF]), n is O P ((nh p ) -1 ), and it will be negligible compared to U n (1). Since the reasoning will be exactly the same for every estimator τ (k) 1,2|z , i.e. for every function g k , k = 1, 2, 3, we omit the sub-index k. Then, the functions g k will be simply denoted by g.

The expectation of our U-statistics is

IE U n (g) := IE g(X 1 , X 2 )K h (z -Z 1 )K h (z -Z 2 ) /IE[K h (z -Z)] 2 = g(x 1 , x 2 )K(t 1 )K(t 2 )f X,Z (x 1 , z + ht 1 )f X,Z (x 2 , z + ht 2 )dx 1 dx 2 dt 1 dt 2 /IE[K h (z -Z)] 2 → 1 f 2 Z (z) g(x 1 , x 2 )f X,Z (x 1 , z)f X,Z (x 2 , z)dx 1 dx 2 = IE g(X 1 , X 2 ) Z 1 = z, Z 2 = z , applying Bochner's lemma to z → g(x 1 , x 2 )f X|Z=z (x 1 )f X|Z=z (x 2 ) dx 1 dx 2 = τ 1,2|Z=z
, that is a continuous function by assumption. Set θ n := IE[U n (g)], g * (x 1 , x 2 ) := (g(x 1 , x 2 ) + g(x 2 , x 1 ))/2 and g * i,j = (g i,j + g j,i )/2 for every (i, j), i = j. Note that U n (g) = U n (g * ). Since g * is symmetrical, the Hájek projection Ûn (g * ) of U n (g * ) satisfies

Ûn (g * ) := 2 n n j=1 IE[g * 0,j |X j , Z j ] -θ n .
Note that IE[ Ûn (g * )] = θ n = τ 1,2|Z=z +o P (1). Since V ar( Ûn (g * ) = 4V ar(IE[g * 0,j |X j , Z j ])/n = O((nh p ) -1 ), then Ûn (g * ) = θ n + o P (1) = τ 1,2|Z=z + o P (1).

Moreover, using the notation g

i,j := g * i,j -IE[g * i,j |X j , Z j ] -IE[g * i,j |X i , Z i ] + θ n for 1 ≤ i = j ≤ n, we have U n (g * ) -Ûn (g * ) = 1≤i =j≤n g i,j /n(n -1)
. By usual U-statistics calculations, it can be easily checked that

V ar U n (g * ) -Ûn (g * ) = 1 n 2 (n -1) 2 1≤i1 =j1≤n 1≤i2 =j2≤n IE[g i1,j1 g i2,j2 ] = O 1 n 2 h 2p .
Indeed, when all indices (i 1 , i 2 , j 1 , j 2 ) are different, or when there is a single identity among them,

IE[g i1,j1 g i2,j2
] is zero. The first nonzero terms arise when there are two identities among the indices, Chapter 5. About kernel-based estimation of conditional Kendall's tau: finite-distance bounds and asymptotic behavior 148

i.e. i 1 = i 2 and j 1 = j 2 (or i 1 = j 2 and j 1 = i 2 ). In the latter case, we get an upper bound as O((nh p ) -2 ) when f Z is continuous at z, by usual changes of variable techniques and Bochner's Lemma. Then, U n (g * ) = Ûn (g * ) + o P (1) = τ 1,2|Z=z + o P (1). Note that U n (1) + n tends to one in probability (Bochner's lemma). As a consequence, τ1,2|Z=z = U n (g * ) / (U n (1) + n ) tends to τ 1,2|Z=z /1 by the continuous mapping theorem.

Proof of Proposition 5.8

Let us note that

τ 1,2|Z=z = IE g k (X 1 , X 2 ) Z 1 = z, Z 2 = z = g k (x 1 , x 2 )f X|Z=z (x 1 )f X|Z=z (x 2 )dx 1 dx 2 = φ k (z)/f 2 Z (z)
where

φ k (z) := g k (x 1 , x 2 )f X,Z (x 1 , z)f X,Z (x 2 , z)dx 1 dx 2 .
We can also write

τ (k) 1,2|Z=z = φk (z)/ f 2 Z (z), where φk (z) := n -2 n i,j=1 K h (Z i -z)K h (Z j -z)g k (X i , X j ) and fZ (z) := n -1 n i=1 K h (Z i -z).
Therefore, we have

τ (k) 1,2|Z=z -τ 1,2|Z=z = φk (z) -φ k (z) f 2 Z (z) -τ 1,2|Z=z fZ (z) -f Z (z) f 2 Z (z) × fZ (z) + f Z (z) .
By usual uniform consistency results (see for example Bosq and Lecoutre [START_REF] Bosq | Théorie de l'estimation fonctionnelle[END_REF]), sup z∈Z fZ (z)-f Z (z) → 0 almost surely, as n → ∞. We deduce that

min z∈Z f 2 Z (z) ≥ f 2 Z,min /2, and max z∈Z | fZ (z) + f Z (z)| ≤ 2 max z∈Z f Z (z) a.s.
This means it is sufficient to prove the uniform strong consistency of φk on Z, to obtain that sup z∈Z τ (k) 1,2|Z=zτ

φk (t l ) -IE[ φk (t l )] + A n ,
where A n := sup l=1,...ln sup z∈B(t l ,ρn) φk (z) -φk (t l ) -(IE[ φk (z)] -IE[ φk (t l )]) . For any index l ∈ {1, . . . , l n } and any z ∈ B(t l , ρ n ), a first-order expansion yields

φk (z) -φk (t l ) -(IE[ φk (z)] -IE[ φk (t l )]) = 1 n(n -1) 1≤i =j≤n g k (X i , X j )K h (z -Z i )K h (z -Z j ) - 1 n(n -1) 1≤i =j≤n g k (X i , X j )K h (t l -Z i )K h (t l -Z j ) -IE g k (X 1 , X 2 )K h (z -Z 1 )K h (z -Z 2 ) -IE g k (X i , X j )K h (t l -Z i )K h (t l -Z j ) ≤ C Lip,K h -2p-1 |z -t l | IE |g k (X 1 , X 2 )| + 1 n(n -1) 1≤i =j≤n |g k (X i , X j )| = O(h -2p-1 ρ n ) = o(1),
for some constant C Lip,K and by choosing ρ n = o(h 2p+1 n ). Actually, we can cover Z in such a way that

l n = O(h -p(2p+1) n
). This is always possible because Z is a bounded set in R p . Note that the previous upper bound is uniform w.r.t. l and z ∈ B(t l , ρ n ), proving that A n = o(1) everywhere.

Chapter 5. About kernel-based estimation of conditional Kendall's tau: finite-distance bounds and asymptotic behavior 149

Now, for every l = 1, . . . , l n , apply Equation (5.8) for every z = t l . For any t > 0, this provides

IP 1 n(n -1) i =j g (l) (X i , Z i ), (X j , Z j ) -IE g (l) (X 1 , Z 1 ), (X 2 , Z 2 ) > t ≤ exp - C 0 nh 2p n t 2 C 1 + C 2 t ,
for some positive constants C 0 , C 1 , C 2 , by setting

g (l) (X i , Z i ), (X j , Z j ) := g k (X i , X j )K h (t l -Z i )K h (t l -Z j ).
Therefore, we deduce

IP sup l=1,...ln φk (t l ) -IE[ φk (t l )] ≥ t ≤ C 4 h -p(2p+1) n exp - C 0 nh 2p n t 2 C 1 + C 2 t ,
for some constant C 4 . Finally, applying Borel-Cantelli lemma, sup z∈Z φk (z) -IE[ φk (z)] tends to zero a.s., proving the result.

Proof of Proposition 5.9

By Markov's inequality, n i=1 w 2 i,n (z) = O P ((nh p ) -1 ) for any z, that tends to zero. Then, by Slutsky's theorem, we get an asymptotic equivalence between the limiting laws of any τ (k) 1,2|z , k = 1, 2, 3, and of their linearly transformed versions τ1,2|z . Thus, we will prove the asymptotic normality of τ (k) 1,2|z for some index k = 1, 2, 3, simply denoted by τ1,2|z . Let g * (x 1 , x 2 ) := (g k (x 1 , x 2 ) + g k (x 2 , x 1 ))/2 for some index k = 1, 2, 3 (that will be implicit in the proof). We now study the joint behavior of (τ 1,2|Z=z i -τ 1,2|Z=z i ) i=1,...,n . We will extend Stute [START_REF] Stute | Conditional U-statistics[END_REF]'s approach, in the case of multivariate conditioning variable z and studying the joint distribution of Ustatistics at several conditioning points. As in the proof of Proposition 5.7, the estimator with the weights given by (5.2) can be rewritten as τ1,2|Z=z i := U n,i (g * ) / (U n,i (1) + n,i ), where

U n,i (g) := 1 n(n -1)IE[K h (z i -Z)] 2 n j1,j2=1,j1 =j2 g(X j1 , X j2 )K h (z i -Z j1 )K h (z i -Z j2 ),
for any bounded measurable function g : R 4 → R. Moreover, sup i=1,...,n | n,i | = O P (n -1 h -p ). By a limited expansion of f X,Z w.r.t. its second argument, and under Assumption 5.3.4, we easily check that

IE U n,i (g) = τ 1,2|Z=z i + r n,i , where |r n,i | ≤ C 0 h α n /f 2 Z (z i ), for some constant C 0 that is independent of i.
Now, we prove the joint asymptotic normality of U n,i (g) i=1,...,n . The Hájek projection Ûn,i (g) of U n,i (g) satisfies Ûn,i (g) := 2 n j=1 g n,i X j , Z j /n -θ n , where θ n := IE U n,i (g) and

g n,i (x, z) := K h (z i -z)IE g(X, x)K h (z i -Z) / IE[K h (z i -Z)] 2 .
Lemma 5.17. Under the assumptions of Proposition 5.9, for any measurable bounded function g,

(nh p ) 1/2 Ûn,i (g) -IE U n,i (g) i=1,...,n D -→ N (0, M ∞ (g)), as n → ∞,
where, for

1 ≤ i, j ≤ n , [M ∞ (g)] i,j := 4 K 2 1 {z i =z j } f Z (z i ) g x 1 , x)g x 2 , x)f X|Z=z i (x)f X|Z=z i (x 1 )f X|Z=z i (x 2 )dx dx 1 dx 2 . D -→ N (0, H), as n → ∞, setting H := J ρ ( τ , e) M ∞ (g * ) M ∞ (g * , 1) M ∞ (g * , 1) M ∞ (1) J ρ ( τ , e) T ,
where τ = τ 1,2|Z=z i i=1,...,n and e is the vector of size n whose all components are equal to 1. Thus, we have J ρ ( τ , e) = Id n , -Diag( τ ) , denoting by Id n the identity matrix of size n and by Diag( τ ) the diagonal matrix of size n whose diagonal elements are the τ 1,2|z i , for i = 1, . . . , n . To be specific, we get

H = M ∞ (g * ) -Diag( τ )M ∞ (g * , 1) -M ∞ (g * , 1)Diag( τ ) + Diag( τ )M ∞ (1)Diag( τ ).
For i, j in {1, . . . , n } and using the symmetry of the function g * , we obtain

[M ∞ (g * )] i,j = 4 K 2 1 {z i =z j } f Z (z i ) IE[g * (X 1 , X)g * (X 2 , X)|Z = Z 1 = Z 2 = z i ], [Diag( τ )M ∞ (g * , 1)] i,j = τ 1,2|Z=z i 4 K 2 1 {z i =z j } f Z (z i ) IE[g * (X 1 , X)|Z = Z 1 = z i ] = 4 K 2 1 {z i =z j } f Z (z i ) τ 2 1,2|Z=z i = [M ∞ (g * , 1)Diag( τ )] i,j = [Diag( τ )M ∞ (1)Diag( τ )] i,j .
Chapter 5. About kernel-based estimation of conditional Kendall's tau: finite-distance bounds and asymptotic behavior 151

As a consequence, we obtain

[H] i,j = 4 K 2 1 {z i =z j } f Z (z i ) IE[g * (X 1 , X)g * (X 2 , X)|Z = Z 1 = Z 2 = z i ] -τ 2
1,2|Z=z i .

Proof of Lemma 5.17

Let us first evaluate the variance-covariance matrix M n,n := [Cov( Ûn,i , Ûn,j )] 1≤i,j≤n . Note that IE g n,i (X j , Z j ) = IE Ûn,i = IE U n,i (g) , and that

(nh p ) 1/2 Ûn,i -IE[U n,i (g)] i=1,...,n = 2h p/2 n 1/2 n j=1 g n,i (X j , Z j ) -IE[U n,i (g)] i=1,...,n ,
that is a sum of independent vectors. Thus, Cov( Ûn,i , Ûn,j ) = 4n -1 Cov g n,i X, Z , g n,j X, Z , for every i, j in {1, . . . , n }, and

IE g n,i (X, Z)g n,j (X, Z) = K h (z i -z)K h (z j -z) IE g(X, x)K h (z i -Z) IE g(X, x)K h (z j -Z) IE[K h (z i -Z)] 2 IE[K h (z j -Z)] 2 f X,Z (x, z)dx dz ∼ 1 h p f 2 Z (z i )f 2 Z (z j ) g x 1 , x)g x 2 , x)K h (z i -z)K h (z j -z)K h (z i -w 1 )K h (z j -w 2 ) × f X,Z (x, z)f X,Z (x 1 , w 1 )f X,Z (x 2 , w 2 )dx dz dx 1 dw 1 dx 2 dw 2 ∼ 1 h p f 2 Z (z i )f 2 Z (z j ) g x 1 , x)g x 2 , x)K(u 1 )K(u 2 )K(u)K( z j -z i h + u)f X,Z (x, z i -hu) × f X,Z (x 1 , z i -hu 1 )f X,Z (x 2 , z j -hu 2 )dx du dx 1 du 1 dx 2 du 2 .
If i = j and K is compactly supported, the latter term is zero when n is sufficiently large, and

Cov( Ûn,i , Ûn,j ) = -4n -1 IE[U n,i ]IE[U n,j ] ∼ -4n -1 τ 1,2|Z=z i τ 1,2|Z=z j = o (nh p ) -1 .
Otherwise, i = j and, as IE g n,i X 1 , Z 1 = O(1), we have

V ar g n,i (X, Z) 2 ∼ 1 h p f 4 Z (z i ) g x 1 , x)g x 2 , x)K(u 1 )K(u 2 )K 2 (u)f X,Z (x, z i -hu) × f X,Z (x 1 , z i -hu 1 )f X,Z (x 2 , z i -hu 2 ) dx du dx 1 du 1 dx 2 du 2 ∼ K 2 h p f Z (z i ) g x 1 , x)g x 2 , x)f X|Z=z i (x)f X|Z=z i (x 1 )f X|Z=z i (x 2 ) dx dx 1 dx 2 ,
by Bochner's lemma. We have proved that, for every i, j ∈ {1, . . . , n },

nh p [M n,n ] i,j → 4 K 2 1 {z i =z j } f Z (z i ) g x 1 , x)g x 2 , x)f X|Z=z i (x)f X|Z=z i (x 1 )f X|Z=z i (x 2 ) dx dx 1 dx 2 ,
as n → ∞. Therefore, nh p M n,n tends to M ∞ .

We now verify Lyapunov's condition with third-order moments, so that the usual multivariate central limit theorem would apply. It is then sufficient to show that

h p/2 n 1/2 3 n j=1 IE g n,i (X j , Z j ) -IE[U n,i (g)] 3 = o(1).
(5.17)

For any j = 1, . . . , n, we have

IE g n,i (X j , Z j ) -IE[U n,i (g)] 3 ∼ 1 f 2 Z (z i ) g(x 1 , x)K h (z i -z 1 )K h (z i -z)f X,Z (x 1 , z 1 )dx 1 dz 1 -IE U n,i (g) 3 f X,Z (x, z)dx dz.
Chapter 5. About kernel-based estimation of conditional Kendall's tau: finite-distance bounds and asymptotic behavior 152

By the change of variable z 1 = z i -ht 1 and z = z i -ht, we get

IE g n,i (X j , Z j ) -IE[U n,i (g)] 3 ∼ h -2p 1 f 2 Z (z i ) g(x 1 , x)K(t 1 )K(t)f X,Z (x 1 , z i -ht 1 )dx 1 dt 1 -h p IE U n,i (g) 3 f X,Z (x, z i -ht)dx dt = O(h -2p ),
because of Bochner's lemma, under our assumptions. Therefore, we have obtained

h p/2 n 1/2

Introduction

In dependence modeling, it is common to work with scalar dependence measures which are marginfree. They can be used to quantify the positive or negative relationship between two random variables X 1 and X 2 . One of the most popular of them is Kendall's tau, a dependence measure defined by

τ 1,2 := IP (X 1,1 -X 2,1 )(X 1,2 -X 2,2 ) > 0 -IP (X 1,1 -X 2,1 )(X 1,2 -X 2,2 ) < 0 ,
where (X i,1 , X i,2 ), i = 1, 2 are i.i.d. copies of (X 1 , X 2 ), see [START_REF] Nelsen | An introduction to copulas[END_REF]. When a covariate Z is available, it is natural to work with the conditional version of this, i.e. the conditional Kendall's tau. It is defined as

τ 1,2|Z=z := IP (X 1,1 -X 2,1 )(X 1,2 -X 2,2 ) > 0 Z 1 = Z 2 = z -IP (X 1,1 -X 2,1 )(X 1,2 -X 2,2 ) < 0 Z 1 = Z 2 = z ,
where (X i,1 , X i,2 , Z i ), i = 1, 2 are i.i.d. copies of (X 1 , X 2 , Z). In such a model, the goal is to study to what extent a p-dimensional covariate z can affect the dependence between the two variables of interest X 1 and X 2 .

Most often, it is difficult to have a clear intuition about the functional link between some measure of dependence and the underlying explanatory variables. Sometimes, it is even unclear whether the Chapter 6. About Kendall's regression 154 covariates have an influence on the dependence between the variables of interest. This is the socalled "simplifying assumption", well-known in the world of copula modeling (see [START_REF] Derumigny | About tests of the "simplifying" assumption for conditional copulas[END_REF] and the references therein). This issue is particularly crucial with pair-copula constructions, as pointed out in [START_REF] Hobaek Haff | On the simplified pair-copula construction-simply useful or too simplistic?[END_REF], [START_REF] Acar | Beyond simplified pair-copula constructions[END_REF], [START_REF] Kurz | Testing the simplifying assumption in high-dimensional vine copulas[END_REF], among others. In our case, we will evaluate an explicit and flexible link between some dependence measure, the Kendall's tau, and the vector of covariates. As a sub-product of our model, we will be able to provide a test of the "simplifying assumption". Given a dataset (X i,1 , X i,2 , Z i ), i = 1, . . . , n, we will focus on the function z → τ 1,2|Z=z for z ∈ Z, where Z denotes a compact subset of R p . This Z represents a set of "reasonable" values for z, so that the density f Z is bounded from below on Z. In order to simplify notation, the reference to the conditioning event Z ∈ Z will be omitted. A first natural choice would be to invoke a nonparametric estimator of τ 1,2|Z=z as in [START_REF] Gijbels | Conditional copulas, association measures and their applications[END_REF], [START_REF] Veraverbeke | Estimation of a conditional copula and association measures[END_REF] and [START_REF] Derumigny | About kernel-based estimation of the conditional Kendall's tau: finite-distance bounds and asymptotic behavior[END_REF]. Here, we prefer to obtain parameters that can be interpreted and that would sum up the information about the conditional Kendall's tau. Moreover, kernel-based estimation can be very costly under a computational point of view: for m values of z, the prediction of all these conditional Kendall's taus has a total cost of O(mn 2 ), that can be large if a large number m is required.

Other estimators of the conditional Kendall's tau, based on classification methods, are proposed in [START_REF] Derumigny | A classification point-of-view about conditional Kendall's tau[END_REF].

In this paper, our idea is to decompose the function z → τ 1,2|Z=z on some functional basis (ψ i ) i≥1 , as any element of a space of functions from Z to R. First note that a Kendall's tau takes its values in the interval [-1, 1], and not on the whole real line. Nevertheless, for some known increasing and continuously differentiable function Λ : [-1, 1] → R, the function z → Λ τ 1,2|Z=z takes values on up to the whole real line potentially, and it can be decomposed on any basis (ψ i ) i≥1 . Typical transforms are Λ(τ ) = log 1+τ 1-τ (the Fisher transform) or Λ(τ ) = log(-log((1 -τ )/2)). We will assume that only a finite number of elements are necessary to represent this function. This means that we have

Λ τ 1,2|Z=z = p i=1 ψ i (z)β * i = ψ(z) T β * , (6.1)
for all z ∈ Z, with p > 0 and a "true" unknown parameter β * ∈ R p . The function ψ(•) := ψ 1 (•), . . . , ψ p (•) T from R p to R p is known and corresponds to deterministic transformations of the covariates z. In practice, it is not easy to have intuition about which kind of basis to use, especially in our framework of conditional dependence measurement. Therefore, the most simple solution is the use of a lot of different functions : polynomials, exponentials, sinuses and cosinuses, indicator functions, etc... They allow to take into account potential non-linearities and even discontinuities of conditional Kendall's taus with respect to z. For the sake of identifiability, we only require their linear independence, as seen in the following proposition (whose straightforward proof is omitted).

IP Z ψ(Z) T t = 0 = 1 implies t = 0.
With such a large choice among flexible classes of functions, it is unlikely we will be able to guess the right ones ex ante. Therefore, it will be necessary to consider a large number of functions ψ i under a sparsity constraint: the cardinality of S, the set of non-zero components of β * , is less than some s ∈ {1, . . . , p }. It is denoted by |S| = |β * | 0 , where | • | 0 yields the number of non-zero components of any vector in R p . Note that, in this framework, p can be moderately large, for example 10 or 30 while the original dimension p is small, for example p = 1 or 2. This corresponds to the decomposition of a function, defined on a small-dimension domain, in a mildly large basis.

Once an estimator β of β * has been computed, the prediction of all the conditional Kendall's tau's for m values of z, which is just the computation of Λ (-1) ψ(z) T β can be done in O(ms), that is much faster than what was previously required with a kernel-based estimator for large m, as soon as s ≤ n 2 (see Section 6.4.1 for a discussion).

Estimating Model (6.1) not only provides an estimator of the conditional Kendall's tau τ 1,2|Z=z , but also easily provides estimators of the marginal effects of z as by-product. For example, given z ∈ Z, the marginal effect of z 1 , i.e. ∂τ 1,2|Z=z (z)/∂z 1 , can be directly estimated by

∂ z1 ψ(z) T β • Λ (-1) ψ(z) T β ,
assuming that ψ and Λ (-1) are differentiable respectively at z and ψ(z) T β. Such sensitivities can be useful in many applications.

A desirable empirical feature of Model (6.1) would be the possibility of obtaining very high/low levels of dependence between X 1 and X 2 , for some Z values, i.e. Λ (-1) (ψ(z) T β * ) should be close (or even equal) to 1 or -1 for some z. This can be the case even if Z is compact, that is here required for theoretical reasons. Indeed, the image of {τ 1,2|z |z ∈ Z} = [τ min , τ max ] through Λ is an interval

[Λ min , Λ max ]. If ψ(z) T β * ≥ Λ max (resp. ψ(z) T β * ≤ Λ min )
, then simply set τ 1,2|Z=z = τ max or even one (resp. τ 1,2|Z=z = τ min or even (-1)).

Contrary to more usual models, the "explained variable" -the conditional Kendall's tau τ 1,2|Z=z -is not observed in (6.1). Therefore, a direct estimation of the parameter β * (for example, by the ordinary least squares, or by the Lasso) is unfeasible. In other words, even if the function z → Λ τ 1,2|Z=z is deterministic, finding the best β in Model (6.1) is far from being just a numerical analysis problem since the function to be decomposed is unknown. Nevertheless, we will replace τ 1,2|Z=z by a nonparametric estimate τ1,2|Z=z , and use it as an approximation of the explained variable. More precisely, we fix a finite collection of points z 1 , . . . , z n ∈ Z n and we estimate τ1,2|Z=z for each of these points. Then, β is estimated as the minimizer of the l 1 -penalized criteria

β := arg min β∈R p 1 n n i=1 Λ(τ 1,2|Z=z i ) -ψ(z i ) T β 2 + λ|β| 1 , (6.2)
where λ is a positive tuning parameter (that may depend on n and n ), and | • | q denotes the l q norm, for 1 ≤ q ≤ ∞. This procedure is summed up in the following Algorithm 5. Note that even if we study the general case with any λ ≥ 0, the properties of the unpenalized estimator can be derived by choosing the particular case λ = 0.

Algorithm 5: Two-step estimation of β

Input: A dataset (X i,1 , X i,2 , Z i ), i = 1, . . .

, n

Input: A finite collection of points z 1 , . . . , z n ∈ Z n for j ← 1 to n do Compute the estimator τ1,2|Z=z j using the sample (X i,1 , X i,2 , Z i ), i = 1, . . . , n ; end Compute the minimizer β of (6.2) using the τ1,2|Z=z j , j = 1, . . . , n , estimated in the above step ;

Output: An estimator β.

Several nonparametric estimators of τ1,2|Z=z j can potentially be used. We refer to [START_REF] Derumigny | About kernel-based estimation of the conditional Kendall's tau: finite-distance bounds and asymptotic behavior[END_REF] for a detailed analysis of their statistical properties. They are of the form

τ1,2|Z=z := n i=1 n j=1 w i,n (z)w j,n (z)g * (X i , X j ), (6.3)
where g * is a bounded function, X i := (X i,1 , X i,2 ) for i = 1, . . . , n and w i,n (z) := K h (Z i -z)/ n j=1 K h (Z jz), h = h(n) > 0 denoting the bandwidth sequence. In the same way, the conditional Kendall's tau can be rewritten as τ 1,2|Z=z = IE[g * (X 1 , X 2 )|Z 1 = Z 2 = z] for the same choices of g * . Possible choices of g * are given in Section 6.9.

In Section 6.2, we state non-asymptotic results for the our estimator β that hold with high probability.

In Section 6.3, its asymptotic properties are stated. In particular, we will study the cases when n is fixed and n → ∞, and when both indices tend to the infinity. We also give some oracle properties and suggest a related adaptive estimator. Sections 6.4 and 6.5 illustrate respectively the numerical performances of β on simulated and real data. All proofs and two supplementary figures have been postponed at the end of the chapter. Remark 6.2. At first sight, in Model (6.1), there seems to be no noise perturbing the variable of interest.

In fact, this is a simple consequence of our formulation of the model. In the same way, a classical linear

model Y = X T β * + ε can be rewritten as IE[Y |X = x] = x T β * without any explicit noise. By definition, IE[Y |X = x]
is a deterministic function of a given x. In our case, Λ τ 1,2|Z=z is a deterministic function of the variable z. This means that we cannot formally write a model with noise, such as Λ τ 1,2|Z=z = ψ(z) T β * + ε where ε is independent of the choice of z. Indeed, the left-hand side of the latter equality is a z-mesurable quantity, unless ε is constant almost surely. Remark 6.3. Note that the conditioning event of Model (6.1) is unusual: usual regression models consider IE[g(X)|Z = z] as a function of the conditioning variable z. Here, the probabilities of concordant/discordant pairs are made conditionally on Z 1 = Z 2 = z. This unusual conditioning event will necessitate some peculiar theoretical treatments. Remark 6.4. Instead of a fixed design setting (z i ) i=1,...,n in the optimization program, it would be possible to consider a random design: simply draw n realizations of Z, independently of the n-sample that has been used for the estimation of the conditional Kendall's taus. The differences between fixed and random designs are mainly a matter of presentation and the reader could easily rewrite our results in a random design setting. We have preferred the former one to study the finite distance properties and asymptotics when n is fixed (Section 6.3.1). When n and n will tend to the infinity (Section 6.3.3), both designs are encompassed de facto because we will assume the weak convergence of the empirical distribution associated to the sample (z i ) i=1,...,n , when n → ∞.

Finite-distance bounds on β

Our first goal is to prove finite-distance bounds in probability for the estimator β. Let Z be the matrix of size n × p whose lines are ψ(z i ) T , i = 1, . . . , n , and let Y ∈ R n be the column vector whose components are Y i = Λ(τ 1,2|Z=z i ), i = 1, . . . , n . For a vector v ∈ R p , denote by ||v|| n := |v| 2 / √ n its empirical norm. We can then rewrite the criterion (6.2) as β := arg min

β∈R p ||Y -Z β|| 2 n + λ|β| 1
, where Y and Z may be considered as "observed", so that the practical problem is reduced to a standard Lasso estimation procedure. Define some "residuals" by ξ i,n := Λ(τ 1,2|Z=z i ) -ψ(z i ) T β * = Λ(τ 1,2|Z=z i ) -Λ(τ 1,2|Z=z i ), for i = 1, . . . , n . Note that these ξ i,n are not "true residuals" in the sense that they do not depend on the estimator β, but on the true parameter β * . We also emphasized the dependence on n in the notation ξ i,n , which is a consequence of the estimated conditional Kendall's tau.

To get non-asymptotic bounds on β, assume the Restricted Eigenvalue (RE) condition, introduced by [START_REF] Bickel | Simultaneous analysis of Lasso and Dantzig selector[END_REF]. For c 0 > 0 and s ∈ {1, . . . , p}, assume RE(s, c 0 ) condition : The design matrix Z satisfies κ(s, c 0 ) := min

J0 ⊂ {1, . . . , p } Card(J0) ≤ s min δ = 0 |δ J C 0 |1 ≤ c0|δJ 0 |1 |Z δ| 2 √ n |δ| 2 > 0.
Note that this condition is very mild, and is satisfied with a high probability for a large class of random matrices: see [12, Section 8.1] for references and a discussion. Theorem 6.5 (Fixed design case). Suppose that Assumptions 6.9.1-6.9.4 and 6.2.1 hold and that the design matrix Z satisfies the RE(s, 3) condition. Choose the tuning parameter as λ = γt, with γ ≥ 4 and t > 0, and assume that we choose h small enough such that

h α ≤ min f Z,min α! 4 C K,α , f 4 Z,min α! t 8 C ψ C Λ (f 2 Z,min + 8f 2 Z,max )C XZ,α . (6.4)
Then, we have

IP ||Z ( β -β * )|| n ≤ 4(γ + 1) t√ s κ(s, 3) and | β -β * | q ≤ 4 2/q (γ + 1) ts 1/q κ 2 (s, 3) , for every 1 ≤ q ≤ 2 ≥ 1 -2n exp -nh p C 1 -2n exp - (n -1)h 2p t 2 C 2 + C 3 t -2 exp - nh p (f Z,max -C K,2 h 2 ) 2 C 4 + 4C K (f Z,min -C K,2 h 2 )/3 , (6.5)
where

C 1 := f 2 Z,min / 32f Z,max K 2 +(8/3)C K f Z,min , C 2 := {16C ψ C Λ (f 2 Z,min +8f 2 Z,max )f Z,max K 2 } 2 /f 8 Z,min , C 3 := (64/3)C ψ C Λ C 2 K (f 2 Z,min + 8f 2 Z,max )/f 4 Z,min , C 4 := 8f Z,max K2
, and t := t + 3 K 2 /(nh p f Z,min ).

This theorem, proved in Section 6.6.2, yields some bounds that hold in probability for the prediction error ||Z ( β -β * )|| n and for the estimation error | β -β * | q , 1 ≤ q ≤ 2, under the specification (6.1). Note that the influence of n and p is hidden through the Restricted Eigenvalue number κ(s, 3). The result depends on three parameters γ, t and h. Apparently, the choice of γ seems to be easy, as a larger γ deteriorates the upper bounds. Nonetheless, it is a bit misleading because β implicitly depends on λ and then on γ (for a fixed t). Nonetheless, choosing γ = 4 is a reasonable "by default" choice. Moreover, a lower t provides a smaller upper bound, but at the same time the probability of this event is lowered.

This induces a trade-off between the probability of the desired event and the size of the bound, as we want the smallest possible bound with the highest probability. Moreover, we cannot choose a too small t, because of the lower bound (6.4): t is limited by a value proportional to h α . The latter h cannot be chosen as too small, otherwise the probability in Equation (6.5) will decrease. To be short: low values of h and t yield a sharper upper bound with a lower probability, and the opposite. Therefore, a trade-off has to be found, depending of the kind of result we are interested in.

Clearly, we would like to exhibit the sharpest upper bounds in (6.5), with the "highest probabilities".

Let us look for parameters of the form t ∝ n -a and h ∝ n -b , with a, b > 0. The assumptions of Theorem 6.5 imply bα ≥ a (to satisfy (6.4)) and 1 -2a -2pb > 0 (so that the right-hand side of (6.5) tends to 1 as n → ∞, i.e. nh p → ∞ and nt 2 h 2p → ∞). For fixed α and p, what are the "optimal" choices a and b under the constraints bα ≥ a and 1 -2a -2pb > 0 ? The latter domain is the interior of a triangle in the plane (a, b) ∈ R 2 + , whose vertices are O := (0, 0), A := (0, 1/(2p)) and B := (α/(2p+2α), 1/(2p+2α)), plus the segment ]0, B[. All points in such a domain would provide admissible couples (a, b) and then admissible tuning parameters (t, h). In particular, choosing the neighborhood of B, i.e. a = α(1 -)/(2p + 2α) and b = 1/(2p + 2α) for some (small) > 0, will be nice because the upper bounds will be minimized. Corollary 6.6. For 0 < < 1, choosing the parameters λ = 4t, t = (n -1) -α(1-)/(2α+2p) and

h = c h (n -1) -1/(2α+2p) , c h := f 4 Z,min α! 2 C ψ C Λ (f 2 Z,min + 16f 2 Z,max )C XZ,α 1/α
, we have, if n is sufficiently large so that (6.4) is satisfied,

IP ||Z ( β -β * )|| n ≤ 20 √ s κ(s, 3)(n -1) α(1-)/(2α+2p) and | β -β * | q ≤ 5.4 2/q s 1/q κ 2 (s, 3)(n -1) α(1-)/(2α+2p) , for every 1 ≤ q ≤ 2 ≥ 1 -2n exp -C 1 c p h (n -1) (2α+p)/(2α+2p) -2n exp - c 2p h (n -1) 2α /(2p+2α) C 2 + C 3 (n -1) -α(1-)/(2α+2p) -2 exp - c p h (n -1) (2α+p)/(2α+2p) (f Z,max -C K,2 h 2 ) 2 C 4 + 4C K (f Z,min -C K,2 h 2 )/3 .

Asymptotic behavior of β

Asymptotic properties of β when n → ∞ and for fixed n

In this part, n is still supposed to be fixed and we state the consistency and the asymptotic normality of β as n → ∞. As above, we adopt a fixed design: the z i are arbitrarily fixed or, equivalently, our reasonings are made conditionally on the second sample.

For n, n > 0, denote by βn,n the estimator (6.2) with h = h n and λ = λ n,n . The following lemma, proved in Section 6.7.1, provides another representation of this estimator βn,n that will be useful hereafter.

Lemma 6.7. We have βn,n = arg min

β∈R p G n,n (β), where G n,n (β) := 2 n n i=1 ξ i,n ψ(z i ) T (β * -β) + 1 n n i=1 ψ(z i ) T (β * -β) 2 + λ n,n |β| 1 . (6.6)
We will invoke a convexity argument: "Let g n and g ∞ be random convex functions taking minimum values at x n and x ∞ , respectively. If all finite dimensional distributions of g n converge weakly to those of g ∞ and x ∞ is the unique minimum point of g ∞ with probability one, then x n converges weakly to x ∞ "

(see [START_REF] Kato | Asymptotics for argmin processes: Convexity arguments[END_REF], e.g).

Theorem 6.8 (Consistency of β). Under the assumptions of Lemma 6.23, if n is fixed and λ = λ n,n → λ 0 , then, given z 1 , . . . , z n and as n tends to the infinity, βn,n

P -→ β * * := inf β G ∞,n (β), where G ∞,n (β) := n i=1 ψ(z i ) T (β * -β) 2 /n + λ 0 |β| 1 .
In particular, if λ 0 = 0 and < ψ(z 1 ), . . . , ψ(z n ) > = R p , then βn,n

P -→ β * .
Chapter 6. About Kendall's regression 159 Proof : By Lemma 6.23, the first term in the r.h.s. of (6.6) converges to 0 as n → ∞. The third term in the r.h.s. of (6.6) converges to λ 0 |β| 1 by assumption. We have just proven that G n,n → G ∞,n pointwise as n → ∞. We can now apply the convexity argument, because G n,n and G ∞,n are convex functions.

As a consequence, arg min β G n,n (β) → arg min β G ∞,n (β) in law. Since we have adopted a fixed design setting, β * * is non random, given (Z 1 , . . . , Z n ). The convergence in law towards a deterministic quantity implies convergence in probability, which concludes the proof. Moreover, when λ 0 = 0, β * is the minimum of G ∞,n because the vectors ψ(z i ), i = 1, . . . , p generate the space R p . Therefore, this implies the consistency of βn,n .

To evaluate the limiting behavior of βn,n , we need the joint asymptotic normality of (ξ 1,n , . . . , ξ n ,n ),

when n → ∞ and given z 1 , . . . , z n . By applying the Delta-method to the function Λ( • ) component-wise, this is given by the following corollary of Lemma 6.24.

Corollary 6.9. Under the assumptions of Lemma 6.24,

(nh p n ) 1/2 [ξ 1,n , . . . , ξ n ,n ]
T tends in law towards a random vector N 0, H given (z 1 , . . . , z n ), where H is a n × n real matrix defined, for every integers

1 ≤ i, j ≤ n , by [ H] i,j := 4 K 2 1 {z i =z j } f Z (z i ) Λ τ 1,2|Z=z i 2 × IE[g(X 1 , X)g(X 2 , X)|Z = Z 1 = Z 2 = z i ] -τ 2 1,2|Z=z i ,
where g is the symmetrized version g(x 1 , x 2 ) := (g * (x 1 , x 2 ) + g * (x 2 , x 1 ))/2. Theorem 6.10 (Asymptotic law of the estimator). Under the assumptions of Lemma 6.24, and if λ n,n (nh p n,n ) 1/2 tends to when n → ∞, we have

(nh p n,n ) 1/2 ( βn,n -β * ) D -→ u * := arg min u∈R p F ∞,n (u), given z 1 , . . . , z n , where F ∞,n (u) := 2 n n i=1 p j=1 W i ψ j (z i )u j + 1 n n i=1 ψ(z i ) T u 2 + p i=1 |u i |1 {β * i =0} + u i sign(β * i )1 {β * i =0} , with W = (W 1 , . . . , W n ) ∼ N 0, H .
This theorem is proved in Section 6.7.2. When = 0, we can say more about the limiting law in general. Indeed, in such a case, u * = arg min u∈R p F ∞,n (u) is the solution of the first order conditions ∇F ∞,n (u) = 0, that are written as

n i=1 W i ψ(z i ) + n i=1 ψ(z i )ψ(z i ) T u = 0. Therefore, u * = - n i=1 ψ(z i )ψ(z i ) T -1 n i=1 W i ψ(z i ), when Σ n := n i=1 ψ(z i )ψ(z i ) T is invertible.
Then, the limiting law of (nh p n,n ) 1/2 ( βn,n -β * ) is Gaussian, and its asymptotic covariance is

V as := Σ -1 n n i,j=1 [ H] i,j ψ(z i )ψ(z j ) T Σ -1 n .
The previous results on the asymptotic normality of βn,n -β * can be used to test H 0 : β * = 0 against the opposite. As said in the introduction, this would constitute a test of the "simplifying assumption", i.e.

the fact that the conditional copula of (X 1 , X 2 ) given Z does not depend on this covariate. Some tests of significance of β * would be significantly simpler than most of the tests of the simplifying assumption that have been proposed in the literature until now. Indeed, the latter ones have been built on nonparametric estimates of conditional copulas and, as sub-products of the weak convergence of the associated processes, the test statistics behaviors are obtained. Therefore, such statistics depend on a preliminary non-parametric estimation of conditional marginal distributions (see [START_REF] Veraverbeke | Estimation of a conditional copula and association measures[END_REF], [START_REF] Derumigny | About tests of the "simplifying" assumption for conditional copulas[END_REF], e.g.), a source of complexities and statistical noise. At the opposite, some tests of H 0 based on βn,n do not require this

F∞,n (u S ) := 2 n n i=1 j∈S W i ψ j (z i )u j + 1 n n i=1 j∈S ψ j (z i )u j 2 + i∈S u i |β * i | δ sign(β * i ),
with W = (W 1 , . . . , W n ) ∼ N 0, H . Moreover, when = 0, the oracle property is fulfilled: IP (S n = S) → n 1.

Asymptotic properties of β when n and n jointly tend to +∞

Now, we consider a framework in which both n and n are going to the infinity, while the dimensions p and p stay fixed. To be specific, n and n will not be allowed to independently go to the infinity. In particular, for a given n, the other size n (n) (simply denoted as n ) will be constrained, as detailed in the assumptions below. In this section, we still work conditionally on z 1 , . . . , z n , . . .. The latter vectors are considered as "fixed", inducing a deterministic sequence. Alternatively, we could consider randomly drawn z i from a given law. The latter case can easily been stated from the results below but its specific statement is left to the reader. Theorem 6.13 (Consistency of βn,n , jointly in (n, n )). Assume that Assumptions 6.9.1-6.9.4 and 6.2.1 are satisfied. Assume that n i=1 ψ(z i )ψ(z i ) T /n converges to a matrix M ψ,z , as n → ∞. Assume that λ n,n → λ 0 and n exp(-Anh 2p ) → 0 for every A > 0, when (n, n ) → ∞. Then βn,n

P -→ arg min β∈R p G ∞,∞ (β), as (n, n ) → ∞, where G ∞,∞ (β) := (β * -β)M ψ,z (β * -β) T + λ 0 |β| 1 . Moreover, if λ 0 = 0 and M ψ,z is
invertible, then βn,n is consistent and tends to the true value β * .

Proof of this theorem is provided in Section 6.7.5. Note that, since the sequence (z i ) is deterministic, we just assume the usual convergence of

n i=1 ψ(z i )ψ(z i ) T /n in R p 2
. Moreover, if the "second subset" (z i ) i=1,...,n were a random sample (drawn along the law IP Z ), the latter convergence would be understood "in probability". And if IP Z satisfies the identifiability condition (Proposition 6.1), then M ψ,z would be invertible and βn,n → β * in probability. Now, we want to go one step further and derive the asymptotic law of the estimator βn,n .

Assumption 6.3.1. (i) The support of the kernel K(•) is included into [-1, 1] p .
Moreover, for all n, n and every (i, j) ∈ {1, . . . , n } 2 , i = j, we have

|z i -z j | ∞ > 2h n,n . (ii) (a) n (nh p+4α n,n + h 2α n,n + (nh p n,n ) -1 ) → 0, (b) λ n,n (n n h p n,n ) 1/2 → 0, (c) n h p+α n,n / ln n → ∞. (iii) The distribution IP z ,n := n i=1 δ z i /n weakly converges as n → ∞, to a distribution IP z ,∞ on R p , with a density f z ,∞ with respect to the p-dimensional Lebesgue measure. (iv) The matrix V 1 := ψ(z )ψ(z ) T f z ,∞ (z )dz is non-singular. (v) Λ(•) is two times continuously differentiable. Let T be the range of z → τ 1,2|Z=z , from Z towards [-1, 1]. On an open neighborhood of T , the second derivative of Λ(•) is bounded by a constant C Λ .
Part (i) of the latter assumption forbids the design points (z i ) i≥1 from being too close to each other and too fast, with respect to the rate of convergence (h n,n ) to 0. This can be guaranteed by choosing an appropriate design. For example, if p = 1 and Z = [0, 1], choose the dyadic sequence 1/2, 1/4, 3/4, 1/8, 3/8, 5/8, 7/8, . . . Part (ii) can be ensured by first choosing a slowly growing sequence n (n), and then by choosing h that would tend to 0 fast enough. Note that a compromise has to be found concerning these two rates. Chapter 6. About Kendall's regression 162

The sequence λ n,n should be chosen at last, so that (b) is satisfied. Interestingly, it is always possible to choose the asymptotically optimal bandwidth, i.e. h ∝ n -1/(2α+p) . In this case, we can set n = n a , with any a ∈]0, 2α/(2α + p)[ and the constraints are satisfied.

The design points z i are deterministic, similarly to all results in the present paper. For a given n , we can invoke the non-random measure IP z ,n := n -1 n i=1 δ z i . Equivalently, all results can be seen as given conditionally on the sample (z i ) i≥1 . In (iii), we impose the weak convergence of IP z ,n to a measure with density w.r.t. the Lebesgue measure. Intuitively, this means we do not want to observe some design points that would be repeated infinitely often (this would result in a Dirac component in

IP z ,∞
). An optimal choice of the density f z ,∞ is not an easy task. Indeed, even if we knew exactly the true density f Z , there is no obvious reasons why we should select the z i along f Z (at least in the limit). If we want a small asymptotic variance Ṽas (see below), the distribution of the design should concentrate the z i in the regions where Λ τ 1,2|Z=z

2 is small and where ψ(z )ψ(z ) T is big.

Part (iv) of the assumption is usual, and ensure that the design is somehow "asymptotically full rank".

This matrix V 1 will also appear in the asymptotic variance of βn,n .

Part (v) allow us to control a remainder term in a Taylor expansion of Λ. Notice that this technical assumption was not necessary in the previous section, where we used the Delta-method on the vector

(τ 1,2|Z=z i -τ 1,2|Z=z i ) i=1,...,n .
But when the number of terms n tends to infinity, we have to invoke second derivatives to control remainder terms.

The proof of the next theorem is provided in Section 6.8. Theorem 6.14 (Asymptotic law of βn,n , jointly in (n, n )). Under Assumptions 6.3.1 and 6.9.1-6.9.4, we have

(nn h p n,n ) 1/2 ( βn,n -β * ) D -→ N (0, Ṽas ),
where

Ṽas := V -1 1 V 2 V -1 1 , V 1
is the matrix defined in Assumption 6.3.1(iv), and

V 2 := K 2 (g(x 1 , x 3 )g(x 2 , x 3 ) -τ 1,2|z 1 =z 2 =z )Λ τ 1,2|Z=z 2 ψ(z)ψ(z) T , × f X|Z (x 1 |Z = z)f X|Z (x 2 |Z = z)f X|Z (x 3 |Z = z) f z ,∞ (z) f Z (z) dx 1 dx 2 dx 3 dz .

Simulations

Numerical complexity

Let us take a short numerical application to compare the complexity of our new estimator with the kernelbased ones. Assume that the size of our dataset is n = 1.000, with a fixed small p, and p = 100. We want to estimate the conditional Kendall's tau on m = 10.000 given points z 1 , . . . , z m . Using simple kernelbased estimation, the total number of operations is of the order of n 2 × m = 1.000 2 × 10.000 = 10 10 . On the contrary, using our new parametric estimators, the cost can be decomposed in the following way:

1. We choose the design points z 1 , . . . , z n (say, equi-spaced) with n = 100.

2. We estimate the kernel-based estimator on these n points (cost: n 2 × n = 1.000 2 × 100 = 10 8 ).

3. We run the Lasso optimization, which is a convex program, so its computation time is linear in n and p (cost: n × p = 100 × 100 = 10 4 ).

Chapter 6. About Kendall's regression 163 4. Finally, for each z i , we compute the prediction Λ (-1) βT z i , and let us assume that s = 50 (cost:

m × s = 10.000 × 50 = 5 × 10 5 ).

Summing up, the computational cost of this realistic experiment is around 10 8 , which is 100 times faster than the kernel-based estimator. Moreover, each new point z m+1 will result in a marginal supplementary cost of 50 operations, compared with a marginal cost of n 2 = 1.000 2 = 10 6 for the kernel-based estimator.

Such a huge difference is due to the fact that we have transformed what was previously available as Ustatistic of order 2 with a O(n 2 ) computational cost for each prediction, into a linear parametric model with s non-zero parameters, giving a cost of O(s) operations for each prediction.

Choice of tuning parameters and estimation of the components of β

Now, we evaluate the numerical performance of our estimates through a simulation study. In this subsection, we have chosen n = 3000, n = 100 and p = 1. The univariate covariate Z follows a uniform distribution between 0 and 1. The marginals X 1 |Z = z and X 2 |Z = z follow some Gaussian distributions N (z, 1). The conditional copula of (X 1 , X 2 )|Z = z belongs to the Gaussian copula family. Therefore, it will be parameterized by its (conditional) Kendall's tau τ 1,2|Z=z , and is denoted by C τ 1,2|Z=z . Obviously, τ 1,2|Z=z is given by Model (6.1). The dependence between X 1 and X 2 , given Z = z, is specified by

τ 1,2|Z=z := 3z(1 -z) = 3/4 -(3/4)(z -1/2) 2 .
We will choose Λ as the identity function and the z i as a uniform grid on [0.01, 0.99]. The values 0 and 1 for the z i are excluded to avoid boundaries numerical problems. As for regressors, we will consider

p = 12 functions of Z, namely ψ 1 (z) = 1, ψ i+1 (z) = 2 -i (z -0.5) i for i = 1, . . . , 5, ψ 5+2i (z) = cos(2iπz)
and ψ 6+2i (z) = sin(2iπz) for i = 1, 2, ψ 11 (z) = 1{z ≤ 0.4}, ψ 12 (z) = 1{z ≤ 0.6}. They cover a mix of polynomial, trigonometric and step-functions. Then, the true parameter is β * = (3/4, 0, -3/4, 0 9 ), where 0 9 is the null vector of size 9.

Our reference value of the tuning parameter h is given by the usual rule-of-thumb, i.e. h = σ(Z)n -1/5 , where σ is the estimated standard deviation of Z. Data-driven choices of the bandwidth h of the first estimator are presented in [START_REF] Derumigny | About kernel-based estimation of the conditional Kendall's tau: finite-distance bounds and asymptotic behavior[END_REF]. Moreover, we designed a cross validation procedure (see Algorithm 6) whose output is a data-driven choice for the tuning parameter λcv . Finally, we perform the convex optimization of the Lasso criterion using the R package glmnet by [START_REF] Friedman | glmnet: Lasso and elastic-net regularized generalized linear models[END_REF].

Algorithm 6: Cross-validation algorithm for choosing λ.

Divide the dataset D = (X i,1 , X i,2 , Z i ) i=1,...,n into N disjoint blocks D 1 , . . . , D N ; foreach λ do for k ← 1 to N do
Estimate the conditional Kendall's taus τ (k) 1,2|Z=z i i=1,...,n on the dataset D k ; Estimate β(-k) by Equation (6.2) on the dataset D\D k using the tuning parameter λ ;

Compute Err k (λ) := i=1,...,n τ (k) 1,2|Z=z i -ψ(z i ) T β(-k) 2 ;

end end

Return λcv := arg min λ k Err k (λ).

In our simulations, we observed that the estimation of β is not very satisfying if the family of function ψ i is far too large. Indeed, our model will "learn the noise" produced by the kernel estimation, and there Table 6.1: Estimated bias, standard deviation and probability of being non-null for each estimated component of β (h = 0.25 σ(Z)n -1/5 and λ = 2 λcv ).

will be "overfitting" in the sense that the function Λ (-1) ψ(•) T β will be very close to τ1,2|Z=• , but not to the target τ 1,2|Z=• . Therefore, we have to find a compromise between misspecification (to choose a family of ψ i that is not rich enough), and over-fitting (to choose a family of ψ i that is too rich).

We have led 100 simulations for couples of tuning parameters (λ, h), where λ ∝ λcv , and h ∝ σ(Z)n -1/5 . The results in term of empirical bias and standard deviation of β are displayed in Figure 6.1. Empirically, we find the smallest h tend to perform better than the largest ones. The influence of the tuning parameter λ (around reasonable values) is less clear. Finally, we selected h = 0.25σ(Z)n -1/5 and λ = 2 λcv . With the latter choice, the coefficient by coefficient results are provided in Table 6.1. The empirical results are relatively satisfying, despite a small amount of over-fitting. In particular, the estimation procedure is able to identify the non-zero coefficients almost systematically. To give a complete picture, for one particular simulated sample, we show the results of the estimation procedure, as displayed in Figures 6.4 and 6.5 in Section 6.10.

Comparison between parametric and nonparametric estimators of the conditional Kendall's tau

We will now compare our estimator of the conditional Kendall's tau, i.e. z → Λ (-1) ψ(z) T β with the kernel-based estimator, i.e. the first-step estimator. For this, we will consider six different settings:

1. as previously, a Gaussian copula parameterized by its conditional Kendall's tau, given by τ 1,2|Z=z := 3z(1 -z) = 3/4 -(3/4)(z -1/2) 2 (well-specified model) ;

2. a badly-specified model, with a Frank copula whose parameter is given by θ(z) = tan(πz/2). Note that the parameter θ of the Frank family belongs to R\{0} and that its Kendall's tau is not written in terms of standard functions of its parameter θ, see [106, p.171] ;

3. an intermediate model with a Frank copula calibrated to have the same conditional Kendall's tau as in the first setting ;

4. another intermediate model with a Gaussian copula calibrated to have the same conditional Kendall's tau as in the second setting ;

5. a Gaussian copula with a conditional Kendall's tau constant equal to 0.5 ;

6. a Frank copula with a conditional Kendall's tau constant equal to 0.5.

This setting will allows to see the effect of good/bad specifications and of changes in terms of copula families. In Table 6.2, for each setting, we provide five numerical measures of performance of a given estimator:

• the integrated bias:

IBias := z IE[τ 1,2|Z=z ] -τ 1,2|Z=z dz ;
• the integrated variance:

IV ar := z IE τ1,2|Z=z -IE[τ 1,2|Z=z ] 2 dz ;
• the integrated standard deviation:

ISd := z IE τ1,2|Z=z -IE[τ 1,2|Z=z ] 2 1/2
dz ;

• the integrated mean square-error: IM SE := z IE τ1,2|Z=z -τ 1,2|Z=z 2 dz ;

• the CPU time used for the computation.

Note that integrals have been approximately computed using a discrete grid {0.0005 × i, i = 0, . . . , 2000}.

Globally, in terms of IMSE, the parametric estimator of τ 1,2|z is doing a better work than a kernel estimator almost systematically (with the single exception of setting 3) and not only in terms of computation time. Surprisingly, even under mis-specification, this conclusion applies whatever the sample size. The differences are particularly striking when the conditional Kendall's tau is a constant function (i.e. under the simplifying assumption).

Comparison with the tests of the simplifying assumption

Now, under the six previous settings, we compare the test of the simplifying assumption H 0 developed in Section 6.3.1 with some of the bootstrapped-based tests of the latter assumption that has been introduced in [START_REF] Derumigny | About tests of the "simplifying" assumption for conditional copulas[END_REF]. In particular, they propose a nonparametric test, using the statistic T 0 CvM defined by

T 0 CvM := [0,1] 3 Ĉ1,2|Z= F -1 Z (u3) (u 1 , u 2 ) -Ĉs,1,2|Z (u 1 , u 2 ) 2 du 1 du 2 du 3 ,
where Ĉ1,2|Z=z is a kernel-based nonparametric estimator of the conditional copula of (X 1 , X 2 )|Z = z and Ĉs,1,2|Z (u 1 , u 2 ) := n -1 n i=1 Ĉ1,2|Z=Zi (u 1 , u 2 ). We will also invoke their parametric test statistic where θ(z) estimates the parameter of the Gaussian (resp. Frank) copula given Z = z, assuming we know the right family of conditional copula, and θ consistently estimates the parameter of the corresponding simplified copula (under the null). Moreover, F -1 Z denotes the empirical quantile function that is associated to the Z-sample. The latter test statistics depends on an a priori chosen parametric copula family. To evaluate the risk of mis-specification, we also include in our table the parametric test T c 2 assuming that the data come from a Clayton copula, whereas the true copula is Gaussian or Frank. For these three tests, p-values are computed by the usual nonparametric bootstrap, with 100 resampling: see Table 6

T c 2 := 1 0 θ F -1 Z (u) -θ 2 du, Chapter

Dimension 2 and choice of ψ

In this section, we will fix the sample size n = 3000 and the dimension p = 2. The random vector Z will follow a uniform distribution on [0, 1] 2 , X 1 |Z = z ∼ N (0, z 1 ), X 2 |Z = z ∼ N (0, z 1 ). Given Z = z, the conditional copula of X 1 and X 2 is Gaussian. We consider three different choices for the functional form of its conditional Kendall's tau :

Setting 1. τ 1,2|Z=z = (3/4) × (z 1 -z 2 ) ; Setting 2. τ 1,2|Z=z = (4/8) × cos(2πz 1 ) + (2/8) × sin(2πz 2 ) ; Setting 3. τ 1,2|Z=z = (3/4) × tanh(z 1 /z 2 ),
where z = (z 1 , z 2 ). We try different choices of dictionaries ψ. For convenience, define p 0 (x) := 1, p i (x) := 2 -i (x -0.5) i , trig 0 (x) := 1, and trig i (x) := cos(2iπx), sin(2iπx) , for x ∈ R and i ∈ N * . We will use the notation (g 1 , g 2 ) ⊗ (g 3 , g 4 ) := (g 1 g 3 , g 1 g 4 , g 2 g 3 , g 2 g 4 ). We are interested in the following functions ψ, that are defined for every z ∈ R p by

ψ (1) (z) := 1, p i (z 1 ) i=1,...,5 , p i (z 2 ) i=1,...,5 = p i (z 1 ) × p j (z 2 ) min(i,j)=0, max(i,j)≤5 ∈ R 11 , ψ (2) (z) := p i (z 1 ) × p j (z 2 ) min(i,j)≤1, max(i,j)≤5 ∈ R 20 , ψ (3) (z) := p i (z 1 ) × p j (z 2 ) min(i,j)≤2, max(i,j)≤5 ∈ R 27 , ψ (4) (z) := p i (z 1 ) × p j (z 2 ) max(i,j)≤5 ∈ R 36 , ψ (5) (z) := 1, trig i (z 1 ) i=1,...,5 , trig i (z 2 ) i=1,...,5 ∈ R 21 , ψ (6) (z) := trig i (z 1 ) ⊗ trig j (z 2 )
min(i,j)≤1, max(i,j)≤5

∈ R 57 , ψ (7) (z) := trig i (z 1 ) ⊗ trig j (z 2 )

min(i,j)≤2, max(i,j)≤5

∈ R 85 , ψ (8) (z) := trig i (z 1 ) ⊗ trig j (z 2 ) 9) (z) := ψ (1) (z), ψ (5) (z) ∈ R 31 , ψ (10) (z) := ψ (2) (z), ψ (6) (z) ∈ R 76 , ψ (11) (z) := ψ (3) (z), ψ (7) (z) ∈ R 137 , ψ (12) (z) := ψ (4) (z), ψ (8)

max(i,j)≤5 ∈ R 121 , ψ (
(z) ∈ R 156 ,
where in the last 4 dictionaries, we count the function constant to 1 only once. We choose n = 400 and the design points z i are chosen as an equispaced grid on [0.1, 0.9] 2 . We consider similar measures of performance for our estimators as in Section 6.4.3. The only difference is that the integration in z is now done on the unit square [0, 1] 2 . In practice, integrals are discretized, and estimated by a sum over the points {(0.01 × i, 0.01 × j), 0 ≤ i, j ≤ 100}. Results are displayed in the following Table 6.4.

We note that the size of the family ψ seems to have a tiny influence on the computation time, which lies always between 6 and 8 seconds. In all settings, polynomial families (ψ (1) to ψ (4) ) give the best IM SE, even when the true function is trigonometric (Setting 2) or under misspecification (Setting 3).

Nevertheless, using trigonometric functions can help to reduce the integrated biais and standard deviation. Indeed, in Setting 2, trigonometric families (ψ (5) to ψ (8) ) do a fair job according to these two measures of performance. Similarly, in Setting 3, mixed families (ψ (9) to ψ (12) ) achieve an acceptable performance. In Settings 1 and 2, they often yield improvement other a msispecified family, especially in terms of integrated standard deviation.

Comparisons between three indicators IM SE, IBias and ISd may be surprising at first sight, but there is no direct link between their values. Indeed, for every point z, M SE(z) = Bias(z) 2 + Sd(z) that minimize both Ibias and ISd still may not minimize IM SE, and conversely. This is due to the non-linearity of the square function, combined with the integration.

Real data application

Now, we apply the model given by (6.1) to a real dataset. From the website of the World Factbook of the Central Intelligence Agency, we have collected data of male and female life expectancy and GDP per capita for n = 206 countries in the world. We seek to analyze the dependence between male and female life expectancies conditionally on the GDP per capita, i.e. given the explanatory variable Z = log 10 (GDP/capita). This dataset and these variables are similar as those in the first example studied in [START_REF] Gijbels | Conditional copulas, association measures and their applications[END_REF].

We use n = 100, h = 2σ(Z)n -1/5 and the same family of functions ψ i as in Section 6.4.2 above (once composed with a linear transform to be defined on [min(Z), max(Z)]). The results are displayed in Figure 6.2. As expected, the levels of conditional dependence between male and female expectancies are strong overall. Many poor countries suffer from epidemics, malnutrition or even wars. In such cases, life expectancies of both genders are exposed to the same "exogenous" factors, inducing high Kendall's taus. Logically, we observe a monotonic decrease of such Kendall's taus when Z is larger, up to Z 4.5, as already noticed by [START_REF] Gijbels | Conditional copulas, association measures and their applications[END_REF]. Indeed, when countries become richer, more developed and safe, men and women less and less depend on their environment (and on its risks of death, potentially). Nonetheless, when Z become even larger (the richest countries in the world), conditional dependencies between male and female life expectancies interestingly increase again, because men and women behave similarly in terms of way of life. In particular, they can benefit from the same levels of security and health and are exposed to the same lethal risks.

Proofs of finite-distance results for β

In this section, we will use the notation u := β -β * and ξ = [ξ i,n ] i=1,...,n , ξ i,n = Y i -(Z β) i .

Technical lemmas

Lemma 6.15. We have

||Z u|| 2 n ≤ λ|u| 1 + 1 n ξ , Z u .
Proof : As β is optimal, through the Karush-Kuhn-Tucker conditions, we have

(1/n )Z T (Y -Z β) ∈ ∂ λ| β| 1 , where ∂ λ| β| 1 is the subdifferential of the norm λ| • | 1 evaluated at β. The dual norm of | • | 1 is | • | ∞ , so there exists v such that |v| ∞ ≤ 1 and (1/n )Z T (Y -Z β) + λv = 0. We deduce successively Z T Z (β * -β)/n + Z T ξ/n + λv = 0, 1 n |Z (β * -β)| 2 2 + 1 n (β * -β) T Z T ξ + λ(β * -β) T v = 0, and finally
||Z (β * -β)|| 2 n ≤ 1 n Z ( β -β * ) , ξ + λ|β * -β| 1 . Lemma 6.16. We have |u S C | 1 ≤ |u S | 1 + 2 λn ξ , Z u .
Proof : By definition, β is a minimizer of ||Y -Z β|| 2 n + λ|β| 1 . Therefore, we have

||Y -Z β|| 2 n + λ| β| 1 ≤ ||Y -Z β * || 2 n + λ|β * | 1 .
After some algebra, we derive

||Y -Z β|| 2 n -||Y -Z β * || 2 n ≤ λ |(β * -β) S | 1 -|( β -β * ) S C | 1 . Moreover, the mapping β → ||Y -Z β|| 2
n is convex and its gradient at Combining the two previous equations, we get

β * is -2 Z T (Y -Z β * )/n = -2 Z T ξ/n . So, we obtain ||Y -Z β|| 2 n -||Y -Z β * || 2 n ≥ -2 n Z T ξ , β -β * .
(-2) Z T ξ, β -β * /n ≤ λ |(β * -β) S | 1 -|( β -β * ) S C | 1 .
Lemma 6.17. Assume that max j=1,...,p 1 n n i=1 Z i,j ξ i,n ≤ t, for some t > 0, that the assumption RE(s, 3) is satisfied, and that the tuning parameter is given by λ = γt, with γ ≥ 4.

Then, ||Z ( β -β * )|| n ≤ 4(γ + 1)t √ s κ(s, 3)
and | β -β * | q ≤ 4 2/q (γ + 1)ts 1/q κ 2 (s, 3) , for every 1 ≤ q ≤ 2.

Proof : Under the first assumption, we have the upper bound

1 n Z T ξ , u ≤ |u| 1 max j=1,...,p 1 n n i=1 Z i,j ξ i,n ≤ |u| 1 t.
We first show that u belongs to the cone δ ∈ R p : |δ S C | 1 ≤ 3|δ S | 1 , Card(S) ≤ s , so that we will be able to use the RE(s, 3) assumption with J 0 = S. From Lemma 6.16,

|u S C | 1 ≤ |u S | 1 + 2t|u| 1 /λ. With our choice of λ, we deduce |u S C | 1 ≤ |u S | 1 + 2|u| 1 /γ. Using the decomposition |u| 1 = |u S C | 1 + |u S | 1 , we get |u S C | 1 ≤ |u S | 1 (γ + 2)/(γ -2) ≤ 3|u S | 1 .
As a consequence, we have

|u| 1 = |u S C | 1 + |u S | 1 ≤ 4|u S | 1 ≤ 4 √ s|u| 2 ≤ 4 √ s ||Z u|| n /κ(s, 3).
By Lemma 6.15, Now, we compute a general bound for |u| q , with 1 ≤ q ≤ 2, using the H ölder norm interpolation inequality:

||Z u|| 2 n ≤ λ|u| 1 + 1 n ξ , Z u ≤ λ|u| 1 + |u| 1 t ≤ |u| 1 (γ + 1)t ≤ 4 √ s κ(s, 3) ||Z u|| n (γ +
|u| q ≤ |u| 2/q-1 1 |u| 2-2/q 2
≤ 4 2/q (γ + 1)ts 1/q κ 2 (s, 3) •

Proof of Theorem 6.5

Using Lemma 6.21, for every t 1 , t 2 > 0 such that C K,α h α /α!+t 1 ≤ f Z,min /2, with probability greater than

1-2n exp -nh p t 2 1 / 2f Z,max K 2 +(2/3)C K t 1 ) -2n exp -(n-1)h 2p t 2 2 f 4 Z,min / 4f 2 Z,max ( K 2 ) 2 + (8/3)C 2 K f 2 Z,min t 2 -2 exp -nh p (f z (z) -C K,2 h 2 ) 2 / 8f Z,max K2 + 4C K (f z (z) -C K,2 h 2 )/3 , we have max j=1,...,p 1 n n i=1 Z i,j ξ i,n ≤ C ψ max i=1,...,n ξ i,n ≤ C ψ C Λ max i=1,...,n τ1,2|Z=z i -τ 1,2|Z=z i ≤ 4C ψ C Λ 1 + 16f 2 Z,max f 3 Z,min C K,α h α α! + t 1 C XZ,α h α f 2 Z,min α! + t 2 .
We choose t 1 := f Z,min /4 so that, because of Condition (6.4), we get

C K,α h α /α! + t 1 ≤ f Z,min /2. Now we choose t 2 := tf 2 Z,min /{8C ψ C Λ (f 2 Z,min + 8f 2 Z,max )}. By Condition (6.4), C XZ,α h α /(f 2 Z,min α!) ≤ t 2
, so that we have

4C ψ C Λ 1 + 8f 2 Z,max f 2 Z,min × C XZ,α h α f 2 Z,min α! + t 2 ≤ 8t 2 C ψ C Λ 1 + 8f 2 Z,max f 2 Z,min ≤ t.
As a consequence, we obtain that IP max j=1,...,p

1 n n i=1 Z i,j ξ i,n > t + 3 K 2 nh p f Z,min ≤ 2n exp - nh p f 2 Z,min 32f Z,max K 2 + (8/3)C K f Z,min + 2n exp - (n -1)h 2p t 2 C 2 + C 3 t + 2 exp - nh p (f Z,max -C K,2 h 2 ) 2 8f Z,max K2 + 4C K (f Z,min -C K,2 h 2 )/3 ,
and we can apply Lemma 6.17 to get the claimed result.

6.7 Proofs of asymptotic results for βn,n 6.7.1 Proof of Lemma 6.7

Using the definition (6.2) of βn,n , we get βn,n := arg min

β∈R p 1 n n i=1 Λ(τ 1,2|Z=z i ) -ψ(z i ) T β 2 + λ n,n |β| 1 = arg min β∈R p 1 n n i=1 ξ i,n + ψ(z i ) T β * -ψ(z i ) T β 2 + λ n,n |β| 1 = arg min β∈R p 1 n n i=1 ξ 2 i,n + 2 n n i=1 ξ i,n ψ(z i ) T (β * -β) + 1 n n i=1 ψ(z i ) T (β * -β) 2 + λ n,n |β| 1 = arg min β∈R p 2 n n i=1 ξ i,n ψ(z i ) T (β * -β) + 1 n n i=1 ψ(z i ) T (β * -β) 2 + λ n,n |β| 1 .

Proof of Theorem 6.10

Let us define r n,n := (nh p n,n ) 1/2 , u := r n,n (β -β * ) and ûn,n := r n,n ( βn,n -β * ), so that βn,n = β * + ûn,n /r n,n . By Lemma 6.7, βn,n = arg min β∈R p G n,n (β). We have therefore ûn,n = arg min

u∈R p -2 n n i=1 ξ i,n ψ(z i ) T u r n,n + 1 n n i=1 ψ(z i ) T u r n,n 2 + λ n,n β * + u r n,n 1 , or ûn,n = arg min u∈R p F n,n (u), where, for every u ∈ R p , F n,n (u) := -2r n,n n n i=1 ξ i,n ψ(z i ) T u + 1 n n i=1 ψ(z i ) T u 2 + λ n,n r 2 n,n β * + u r n,n 1 -β * 1 .
Note that, by Corollary 6.9, we have

2r n,n n n i=1 ξ i,n ψ(z i ) T u = 2 n n i=1 p j=1 r n,n ξ i,n ψ j (z i )u j D -→ 2 n n i=1 p j=1 W i ψ j (z i )u j .
We also have, for any (fixed) u and when n is large enough,

β * + u r n,n 1 -β * 1 = p i=1 |u i | r n,n 1 {β * i =0} + u i r n,n sign(β * i )1 {β * i =0} . Therefore λ n,n r 2 n,n β * + u/r n,n 1 -β * 1 → p i=1 |u i |1 {β * i =0} + u i sign(β * i )1 {β * i =0} .
We have shown that F n,n (u) D -→ F ∞,n (u). Those functions are convex, hence the conclusion follows from the convexity argument.

Proof of Proposition 6.11

The proof closely follows Proposition 1 in [START_REF] Zou | The adaptive lasso and its oracle properties[END_REF]. It starts by noting that IP (S n = S) ≤ IP βj = 0, ∀j ∈ S .

Because of the weak limit of β (Theorem 6.10 and the notation therein), this implies

lim sup n IP βj = 0, ∀j ∈ S ≤ IP u * j = 0, ∀j ∈ S .
If = 0, then u * is asymptotically normal, and the latter probability is zero. Otherwise, = 0 and define the Gaussian random vector

W ψ := 2 n i=1 W i ψ(z i )/n . The KKT conditions applied to F ∞,n provide W ψ + 2 n n i=1 ψ(z i )ψ(z i ) T u * + v * = 0,
for some vector v * ∈ R p whose components v * j are less than one in absolute value when j ∈ S, and v * j = sign(β * j ) when j ∈ S. If u * j = 0 for all j ∈ S, we deduce

( W ψ ) S + 2 n n i=1 ψ(z i )ψ(z i ) T S,S u * S + sign(β * S ) = 0, and (6.7)
( W ψ ) S c + 2 n n i=1 ψ(z i )ψ(z i ) T S c ,S u * S ≤ , (6.8)
componentwise and with obvious notation. Combining the two latter equations provides

( W ψ ) S c - n i=1 ψ(z i )ψ(z i ) T S c ,S n i=1 ψ(z i )ψ(z i ) T -1 S,S W ψ ) S + sign(β * S ) ≤ , (6.9)
componentwise. Since the latter event is of probability strictly lower than one, this is still the case for the event u * j = 0, ∀j ∈ S .

Proof of Theorem 6.12

The beginning of the proof is similar to the proof of Theorem 6.10. With obvious notation, ǔn,n = arg min u∈R p Fn,n (u), where for every u ∈ R p ,

Fn,n (u) := -2r n,n n n i=1 ξ i,n ψ(z i ) T u + 1 n n i=1 ψ(z i ) T u 2 + µ n,n r 2 n,n p i=1 1 | βi | δ |β * i + u i r n,n | -|β * i | . If β * i = 0, then µ n,n r 2 n,n | βi | δ |β * i + u i r n,n | -|β * i | = µ n,n r n,n | βi | δ u i sign(β * i ) = |β * i | δ u i sign(β * i ) + o P (1). If β * i = 0, then µ n,n r 2 n,n | βi | δ |β * i + u i r n,n | -|β * i | = µ n,n r n,n ν δ n |ν n βi | δ |u i |.
By assumption ν n βi = O p (1), and the latter term tends to the infinity in probability iff u i = 0. As a consequence, if there exists some i ∈ S s.t. u i = 0, then Fn,n (u) tends to the infinity. Otherwise, u i = 0 when i ∈ S and Fn,n (u) → F∞,n (u S ). Since F∞,n is convex, we deduce [START_REF] Kato | Asymptotics for argmin processes: Convexity arguments[END_REF] that ǔS → u * S , and ǔS c → 0 S c , proving the asymptotic normality of βn,n ,S . Now, let us prove the oracle property. If j ∈ S, then βj tends to β j in probability and IP(j ∈ S n ) → 1.

It suffices to show that IP(j ∈ S n ) → 0 when j ∈ S. If j ∈ S and j ∈ S n , the KKT conditions on Fn,n provide

-2r n,n n n i=1 ξ i,n ψ j (z i ) + 2 n n i=1 ψ j (z i )ψ(z i ) T ǔn,n = - µ n,n r n,n ν δ n |ν n βj | δ sign(ǔ j )•
Due to the asymptotic normality of β (that implies the one of ǔn,n ), the left hand side of the previous equation is asymptotically normal, when = 0. On the other side, the r.h.s. tends to the infinity in probability because ν n βj = O P (1). Therefore, the probability of the latter event tends to zero when n → ∞.

Proof of Theorem 6.13

By Lemma 6.7, we have βn,n = arg min

β∈R p G n,n (β), where G n,n (β) := 2 n n i=1 ξ i,n ψ(z i ) T (β * -β) + 1 n n i=1 ψ(z i ) T (β * -β) 2 + λ n,n |β| 1 . Define also G ∞,n (β) := n i=1 ψ(z i ) T (β * -β) 2 /n + λ 0 |β| 1 . We have G n,n (β) -G ∞,n (β) ≤ 2 n n i=1 ξ i,n ψ(z i ) T (β * -β) + |λ n,n -λ 0 | × |β| 1 .
By assumption, the second term on the r.h.s. converges to 0. We now show that the first term on the r.h.s. is negligible. Indeed, for every > 0,

IP 1 n n i=1 ξ i,n ψ(z i ) > ≤ IP C Λ n n i=1 |τ z i -τ z i | × ψ(z i ) > ≤ n i=1 IP |τ z i -τ z i | > Cst ,
where Cst is the constant ( C Λ × C ψ ) -1 . Apply Lemma 6.21 with the t = f Z,min /4 and t / is a sufficiently small constant. When n is sufficiently large, we get

IP |τ 1,2|Z=z -τ 1,2|Z=z | > Cst ≤ 4 exp -nh 2p Cst ,
for some constant Cst > 0. Thus,

n i=1 ξ i,n ψ(z i )/n = o IP (1), and G n,n (β) = G ∞,n (β) + o IP (1) for every β. Since n i=1 ψ(z i )ψ(z i ) T /n tends towards a matrix M ψ,z , deduce that G ∞,n (β) tends to G ∞,∞ (β)
when n → ∞. Therefore, for all β ∈ R p , G n,n (β) weakly tends to G ∞,∞ (β). By the convexity argument, we deduce that arg min β G n,n (β) weakly converges to arg min β G ∞,∞ (β). Since the latter minimizer is non random, the same convergence is true in probability.

Proof of Theorem 6.14

We start as in the proof of Theorem 6.10. Define rn,n := (nn h p n,n ) 1/2 , u := rn,n (β -β * ) and ûn,n := rn,n ( βn,n -β * ), so that βn,n = β * + ûn,n /r n,n . We define for every u ∈ R p ,

F n,n (u) := -2r n,n n n i=1 ξ i,n ψ(z i ) T u + 1 n n i=1 ψ(z i ) T u 2 + λ n,n r2 n,n β * + u rn,n 1 -β * 1 , (6.10)
and we obtain ûn,n = arg min u∈R p F n,n (u).

Lemma 6.18. Under the same assumptions as in Theorem 6.14,

T 1 := (r n,n /n ) n i=1 ξ i,n ψ(z i ) tends in law towards a Gaussian random vector N (0, V 2 ).
This lemma is proved in Section 6.8.1. It will help to control the first term of Equation (6.10), which is

simply -2T T 1 u.
Concerning the second term of Equation (6.10), using Assumption 6.3.1(iii), we have for every u ∈ .11) This has to be read as a convergence of a sequence of real numbers indexed by u, because the design points z i are deterministic. We also have, for any u ∈ R p and when n is large enough,

R p 1 n n i=1 ψ(z i ) T u 2 → ψ(z ) T u 2 f z ,∞ dz . ( 6
β * + u rn,n 1 -β * 1 = p i=1 |u i | rn,n 1 {β * i =0} + u i rn,n sign(β * i )1 {β * i =0} .
Therefore, by Assumption 6.3.1(ii)(b), for every u ∈ R p ,

λ n,n r2 n,n β * + u rn,n 1 -β * 1 → 0, (6.12)
when (n, n ) tends to the infinity. Combining Lemma 6.18 and Equations (6.10-6.12), and defining the function

F ∞,∞ by F ∞,∞ (u) := 2 WT u + ψ(z ) T u 2 f z ,∞ (z )dz , u ∈ R p ,
where W ∼ N (0, V 2 ), we obtain that every finite-dimensional margin of F n,n converges weakly to the corresponding margin of F ∞,∞ . Now, applying the convexity lemma, we get

ûn,n D -→ u ∞,∞ , where u ∞,∞ := arg min u∈R p F ∞,∞ (u). Since F ∞,∞ (u) is a continuously differentiable convex function, we apply the first-order condition ∇F ∞,∞ (u) = 0, which yields 2 W + 2 ψ(z )ψ(z ) T u ∞,∞ f z ,∞ (z )dz = 0. As a consequence u ∞,∞ = -V -1 1 W ∼
N (0, Ṽas ), using Assumption 6.3.1(iv). We finally obtain rn,n βn,n -β * D -→ N 0, Ṽas , as claimed.

Proof of Lemma 6.18 : convergence of T 1

Using a Taylor expansion, we have

T 1 := rn,n n n i=1 ξ i,n ψ(z i ) = rn,n n n i=1 Λ τ1,2|Z=z i -Λ τ 1,2|Z=z i ψ(z i ) = T 2 + T 3 ,
Chapter 6. About Kendall's regression 176 where the main term is

T 2 := rn,n n n i=1 Λ τ 1,2|Z=z i τ1,2|Z=z i -τ 1,2|Z=z i ψ(z i ),
and the remainder is

T 3 := rn,n n n i=1 α 3,i τ1,2|Z=z i -τ 1,2|Z=z i 2 ψ(z i ), with ∀i = 1, . . . , n , |α 3,i | ≤ C Λ /2, by Assumption 6.3.1(v).
Using the definition (6.3) of τ1,2|Z=z , the definition of the weights w i,n (z) and the notation ψ(z) := Λ τ 1,2|Z=z ψ(z), we rewrite T 2 =: T 4 + T 5 , where

T 4 := rn,n n n 2 n i=1 n j1=1 n j2=1 K h (z i -Z j1 )K h (z i -Z j2 ) f 2 Z (z i ) × g * (X j1 , X j2 ) -IE g * (X 1 , X 2 )|Z 1 = Z 2 = z i ψ(z i ), (6.13)
T 5 := rn,n n n 2 n i=1 n j1=1 n j2=1 K h (z i -Z j1 )K h (z i -Z j2 ) 1 fZ (z i ) 2 - 1 f Z (z i ) 2 × g * (X j1 , X j2 ) -IE g * (X 1 , X 2 )|Z 1 = Z 2 = z i ψ(z i ). (6.14)
Note that we can put together the terms (j 1 , j 2 ) and (j 2 , j 1 ). This corresponds to the substitution of g * by its symmetrized version g. In the following, we will therefore assume that g * has been symmetrized

without loss of generality. The random variable T 4 can be seen (see Equation (6.13)) as a sum of (indexed by i) U-statistics of order 2. Its Hájek projection will yield the asymptotically normal dominant term of T 2 .

To lighten notation, we denote

τ i := τ 1,2|Z1=Z2=z i , f (•, •) = f X,Z (•, •) and g i,j1,j2 := g * (X j1 , X j2 ) -IE g * (X 1 , X 2 )|z i = g * (X j1 , X j2 ) -τ i .
Implicitly, all the expectations we will consider are expectations conditionally on the sequence of z i ,

i ≥ 1.
First note that, by usual α-order limited expansions, we have

IE[T 4 ] = rn,n n n 2 n i=1 n(n -1) K h (z i -z 1 )K h (z i -z 2 ) f 2 Z (z i ) g * (x 1 , x 2 ) -τ i × ψ(z i )f (x 1 , z 1 )f (x 2 , z 2 ) dx 1 dx 2 dz 1 dz 2 - rn,n n n n i=1 τ i ψ(z i ) K 2 h (z i -z) f 2 Z (z i ) f (x, z)dx dz = (n -1)r n,n n n n i=1 K(t 1 )K(t 2 ) f 2 Z (z i ) g * (x 1 , x 2 ) -τ i × ψ(z i )f (x 1 , z i -ht 1 )f (x 2 , z i -ht 1 ) dx 1 dx 2 dt 1 dt 2 - rn,n n nh p n i=1 τ i ψ(z i ) K 2 (t) f 2 Z (z i ) f X,Z (x, z i -ht)dx dt = (n -1)r n,n h 2α n n n i=1 K(t 1 )K(t 2 ) f 2 Z (z i ) g * (x 1 , x 2 ) -τ i × ψ(z i )d (α) Z f (x 1 , z * i ) • t (α) 1 d (α) Z f (x 2 , z * i ) • t (α) 2 dx 1 dx 2 dt 1 dt 2 - rn,n n nh p n i=1 τ i K 2 ψ(z i ) f 2 Z (z i ) f (x, z * i ) dx = O rn,n h 2α + rn,n /(nh p ) = O √ nn h p+4α + n /(nh p ) = o(1),
under Assumption 6.3.1 (ii). Above, we have denoted by z * i some vectors in R p s.t. z i -Z * i ∞ < 1. They depend on z i , x 1 , x 2 or x, respectively. Moreover, set

T 4 -IE[T 4 ] = rn,n n n 2 n i=1 n j1,j2=1
ζ i,j1,j2 , (6.15)

ζ i,j1,j2 = K h (z i -Z j1 )K h (z i -Z j2 )g i,j1,j2 -IE K h (z i -Z j1 )K h (z i -Z j2 )g i,j1,j2 ψ(z i ) f 2 Z (z i ) • Note that V ar(T 4 ) = IE[T 4 T T 4 ] + o(1)
and

IE[T 4 T T 4 ] = r2 n,n (n ) 2 n 4 n i1,i2=1 n j1,j2=1 n j3,j4=1 IE[ζ i1,j1,j2 ζ T i2,j3,j4 ]. By independence, IE[ζ i,j1,j2 ζ T i,j3,j4 ] = 0 when {j 1 , j 2 } ∩ {j 3 , j 4 } = ∅.
Otherwise, assume that j 1 = j 3 = j and there are no other identities among the four indices (j 1 , j 2 , j 3 , j 4 ). Set

ζ i := IE K h (z i -Z 1 )K h (z i -Z 2 )g i,1,2 ψ(z i ) f 2 Z (z i ) . (6.16)
Then,

IE[ζ i1,j,j2 ζ T i2,j,j4 ] = ζ i1,j,j2,i2,j,j4 -ζ i1 ζ T i2 ,
where

ζ i1,j,j2,i2,j,j4 := IE K h (z i1 -Z j )K h (z i1 -Z j2 )K h (z i2 -Z j )K h (z i2 -Z j4 )g i1,j,j2 g T i2,j,j4 × ψ(z i1 )ψ(z i2 ) T f 2 Z (z i1 )f 2 Z (z i2 ) = ψ(z i1 )ψ(z i2 ) T h p f 2 Z (z i1 )f 2 Z (z i2 ) K(t 1 )K(t 2 )K z i2 -z i1 h + t 1 K(t 4 ) g * (x 1 , x 2 ) -τ i1 × g * (x 1 , x 4 ) -τ i2 f (x 1 , z i1 -ht 1 )f (x 2 , z i1 -ht 2 )f (x 4 , z i4 -ht 4 ) dx 1 dx 2 dx 4 dt 1 dt 2 dt 4 .
By assumption, ζ i1,j,j2,i2,j,j4 is zero when

i 1 = i 2 . Otherwise, when i 1 = i 2 = i, ζ i,j,j2,i,j,j4 ψ(z i )ψ(z i ) T h p f Z (z i ) K 2 g * (x 1 , x 2 ) -τ i g * (x 1 , x 4 ) -τ i × f X|Z (x 1 |z i )f X|Z (x 2 |z i )f X|Z (x 4 |z i ) dx 1 dx 2 dx 4 := C i,1,2,4 /h p .
It is easy to check that the terms with other identities among the four indices j k , as ζ i,j,j2,i,j,j2 or ζ i,j,j2,i,j,j will induce negligible remainder terms. Therefore, we get

r2 n,n (n ) 2 n 4 n i1,i2=1 n j,j2,j4=1 ζ i1,j,j2,i2,j,j4 1 n n i=1 C i,1,2,4 .
Concerning the terms induced by the product of two ζ i , note that, by limited expansions,

ζ i = ψ(z i ) f 2 Z (z i ) K h (z i -z 1 )K h (z i -z 2 ) g * (x 1 , x 2 ) -τ i f (x 1 , z 1 )f (x 2 , z 2 ) dx 1 dz 1 dx 2 dz 2 = ψ(z i ) f 2 Z (z i ) K(t 1 )K(t 2 ) g * (x 1 , x 2 ) -τ i f (x 1 , z i -ht 1 )f (x 2 , z i -ht 2 ) dx 1 dt 1 dx 2 dt 2 = h 2α ψ(z i ) f 2 Z (z i ) K(t 1 )K(t 2 ) g * (x 1 , x 2 ) -τ i d (α) Z f (x 1 , z * i ) • t (α) 1 d (α) Z f (x 2 , z * i ) • t (α) 2 dx 1 dt 1 dx 2 dt 2 ,
with the same notation as above. As a consequence,

sup i ζ i = O(h 2α ) and r2 n,n (n ) 2 n 4 n i1,i2=1 n j,j2,j4=1 ζ i1 ζ i2 r2 n,n n 1 n n i=1 ζ i,1,2 2 = O h 4α r2 n,n n = O(n h 4α+p ) = o(1)
.

Therefore, we obtain

r2 n,n (n ) 2 n 4 n i1,i2=1 n j,j2,j4=1 IE[ζ i1,j,j2 ζ T i2,j,j4 ] 1 n n i=1 C i,1,2,4 .
To calculate IE[T 4 T T 4 ], there are three other similar terms, that respectively correspond to the cases j 1 = j 4 , j 2 = j 3 or j 2 = j 4 . Therefore, we deduce

V ar(T 4 ) IE[T 4 T T 4 ] 4 n n i=1 C i,1,2,4 4 K 2 ψ(z)ψ(z) T f Z (z) g * (x 1 , x 2 ) -τ 1,2|Z1=Z2=z g * (x 1 , x 4 ) -τ 1,2|Z1=Z2=z × f X|Z (x 1 |z)f X|Z (x 2 |z)f X|Z (x 4 |z)f z ,∞ (z) dx 1 dx 2 dx 4 dz,
that is equal to the so-called variance-covariance matrix V 2 . Now assume that

T 4 -IE[T 4 ] is asymptotically normal, i.e. T 4 -IE[T 4 ] D -→ N (0, V 2 )
. This result will be proved in Subsection 6.8.2.

Let us decompose the term T 5 , as defined in Equation (6.14). For every i = 1, . . . , n , a usual Taylor expansion yields

1 f 2 Z (z i ) - 1 f 2 Z (z i ) = 1 f 2 Z (z i ) 1 1 + fZ (z i ) -f Z (z i ) f Z (z i ) 2 -1 = -2 fZ (z i ) -f Z (z i ) f 3 Z (z i ) + T 7,i ,
where

T 7,i = 3 f 2 Z (z i ) (1 + α 7,i ) -4 fZ (z i ) -f Z (z i ) f Z (z i ) 2 , for some |α 7,i | ≤ fZ (z i ) -f Z (z i ) f Z (z i ) .
Therefore, we obtain the decomposition T 5 = -2 T 6 + T 7 , where

T 6 := rn,n n n 2 n i=1 n j1=1 n j2=1 K h (z i -Z j1 )K h (z i -Z j2 ) fZ (z i ) -f Z (z i ) f 3 Z (z i ) × g * (X j1 , X j2 ) -IE g * (X 1 , X 2 )|Z 1 = Z 2 = z i ψ(z i ), T 7 := rn,n n n 2 n i=1 n j1=1 n j2=1 K h (z i -Z j1 )K h (z i -Z j2 )T 7,i × g * (X j1 , X j2 ) -IE g * (X 1 , X 2 )|Z 1 = Z 2 = z i ψ(z i ).
Chapter 6. About Kendall's regression 179 Summing up all the previous equations, we get

T 1 = T 4 -IE[T 4 ] -2 T 6 + T 7 + T 3 + o(1). (6.17)
Afterwards, we will prove that all the remainders terms T 6 , T 7 and T 3 are negligible, i.e. they tend to zero in probability. These results are respectively proved in Subsections 6.8.3, 6.8.4 and 6.8.5. Combining all these elements with the asymptotic normality of T 4 (proved in Subsection 6.8.2), we get

T 1 D -→ N (0, V 2 ),
as claimed.

Proof of the asymptotic normality of T 4

We will lead the usual Hájek projection of T 4 . To weaken notation, denote

IE[ζ i,j1,j2 |X j1 , Z j1 ] := IE[ζ i,j1,j2 |j 1 ].
Then, recalling (6.15), we can write

T 4 -IE[T 4 ] = T 4,1 + T 4,2 + T 4,3 , with T 4,1 := 2r n,n n n 2 n i=1 n j1,j2=1 1(j 1 = j 2 )IE[ζ i,j1,j2 |j 1 ], T 4,2 := 2r n,n n n 2 n i=1 n j=1 IE[ζ i,j,j |j],
and

T 4,3 := rn,n n n 2 n i=1 n j1,j2=1 ζ i,j1,j2 -IE[ζ i,j1,j2 |j 1 ] -IE[ζ i,j1,j2 |j 2 ] .
We will prove that T 4,2 and T 4,3 are o P (1). Therefore, the asymptotic normality of T 4 reduces to the one of T 4,1 .

Note that nT 4,1 /2(n -1) = n j=1 β j,n,n , where

β j,n,n := rn,n n n n i=1 IE[ζ i,j,0 |j], j = 1, . . . , n,
by formally considering a random vector Z 0 that is independent of the other Z j , j ≥ 1. Therefore, we get a triangular array of random vectors (β j,n,n ) j=1,...,n , s.t., for a fixed n, the variables β j,n,n are mutually independent given the vectors z i , i ≥ 1. Let us check Lyapunov's sufficient condition, that will imply the asymptotic normality of T 4,1 . In other words, it is sufficient to prove that

n j=1 β j,n,n 3
∞ -→ 0, (6.18)
when n and n tend to the infinity. Recalling (6.16), we can rewrite

β j,n,n = rn,n n n n i=1 K h (z i -Z j ) ψ(z i ) f 2 Z (z i ) K h (z i -z) g * (x, X j ) -τ i f (x, z) dx dz -ζ i := rn,n n n n i=1 γ i,j ,
where sup i ζ i := O(h 2α ). Note that

β j,n,n 3
∞ ≤ p 3 r3 n,n (n ) 3 n 3 n i1,i2,i3=1 γ i1,j ∞ γ i2,j ∞ γ i3,j ∞ .
The terms that that involve some products by the means ζ i k , k = 1, 2, 3, are negligible and they may be forgotten here. For some constants Cst, this provides

n j=1 IE β j,n,n 3 ∞ ≤ Cst r3 n,n (n ) 3 n 3 n j=1 n i1,i2,i3=1 ψ ∞ (z i1 ) ψ ∞ (z i2 ) ψ ∞ (z i3 ) f 2 Z (z i1 )f 2 Z (z i2 )f 2 Z (z i3 ) × IE K h (z i1 -Z j ) K h (z i1 -z 1 ) g * (x 1 , X j ) -τ i1 f (x 1 , z 1 ) dx 1 dz 1 × K h (z i2 -Z j ) K h (z i2 -z 2 ) g * (x 2 , X j ) -τ i2 f (x 2 , z 2 ) dx 2 dz 2 × K h (z i3 -Z j ) K h (z i3 -z 3 ) g * (x 3 , X j ) -τ i3 f (x 3 , z 3 ) dx 3 dz 3 .
By some now usual changes of variables, the latter expectations are zero when one of the three indices i 1 , i 2 and i 3 is different from the others. Thus, the non-zero expectations are obtained when i 1 = i 2 = i 3 .

In the latter case, we get When i 1 = i 2 , some usual changes of variables yield

n j=1 β j,n,n 3 ∞ ≤ Cst r3 n,n (n ) 3 n 2 n i=1 ψ 3 ∞ (z i ) f 6 Z (z i ) × |K| 3 h (z i -z) K h (z i -z 1 ) g * (x 1 , x) -τ i f (x 1 , z 1 ) dx 1 dz 1 3 f (x, z) dx dz ≤ Cst r3 n,n (n ) 3 n 2 h 2p n i=1 ψ 3 ∞ (z i ) f 6 Z (z i ) |K| 3 (t) K(t 1 ) g * (x 1 , x) -τ i f (x 1 , z i -ht 1 ) dx 1 dt 1 3 × f (x, z i -ht) dx dt = O r3 n,n (n ) 2 n 2 h 2p = O 1 (nn h p ) 1/2 = o(1).

Concerning the remainder terms

IE IE[ζ i1,j,j |j]IE[ζ T i2,j,j |j] = ψ(z i1 )ψ(z i2 ) T f 2 Z (z i1 )f 2 Z (z i2 ) τ i1 τ i2 × IE K 2 h (z i1 -Z j )K 2 h (z i2 -Z j ) -IE K 2 h (z i1 -Z j ) IE K 2 h (z i2 -Z j ) = O(h -2p ),
uniformly w.r.t. i. By a similar reasoning, we can prove that

sup i IE IE[ζ i,j,j |j]IE[ζ T i,j,j |j] = O(h -3p ).
Therefore,

IE[T 4,2 T T 4,2 ] = O r2 n,n (n ) 2 n 4 (n ) 2 nh -2p + n nh -3p = O n n 2 h p + 1 n 2 h 2p = o(1).
Concerning T 4,3 , this remainder term is centered and

IE[T 4,3 T T 4,3 ] = r2 n,n (n ) 2 n 4 n i1,i2=1 n j1,j2=1 n j3,j4=1 IE {ζ i1,j1,j2 -IE[ζ i1,j1,j2 |j 1 ] -IE[ζ i1,j1,j2 |j 2 ]} × {ζ i2,j3,j4 -IE[ζ i2,j3,j4 |j 3 ] -IE[ζ i2,j3,j4 |j 4 ]} T . (6.19)
The expectations on the latter r.h.s. are zero when {j 1 , j 2 } ∩ {j 3 , j 4 } = ∅ due to independence and the fact that the terms ζ i,j,j are centered. Otherwise, there is at least an identity among the indices j k , k = 1, . . . , 4. For instance, assume j 1 = j 3 = j and j = j 2 = j 4 . Then,

IE {ζ i1,j,j2 -IE[ζ i1,j,j2 |j] -IE[ζ i1,j,j2 |j 2 ]}{ζ i2,j,j4 -IE[ζ i2,j,j4 |j] -IE[ζ i2,j,j4 |j 4 ]} T = IE {ζ i1,j,j2 -IE[ζ i1,j,j2 |j]}{ζ i2,j,j4 -IE[ζ i2,j,j4 |j]} T = IE IE {ζ i1,j,j2 -IE[ζ i1,j,j2 |j]}{ζ i2,j,j4 -IE[ζ i2,j,j4 |j]} T j = IE IE ζ i1,j,j2 ζ T i2,j,j4 |j -IE IE ζ i1,j,j2 |j IE ζ T i2,j,j4 |j = 0.
Due to the symmetry of the latter cross-products, all cases of a single identity among the j k , k = 1, . . . , 4, yield the same result. Therefore, we need (at least) two identities among them to obtain non zero covariances in the calculation of IE[T 4,3 T T 4,3 ]. Thus, let us assume that j 1 = j 3 and j 2 = j 4 . Then, the corresponding terms in (6. [START_REF] Bickel | Simultaneous analysis of Lasso and Dantzig selector[END_REF]) is

r2 n,n (n ) 2 n 4 n i1,i2=1 n j1,j2=1 IE {ζ i1,j1,j2 -IE[ζ i1,j1,j2 |j 1 ] -IE[ζ i1,j1,j2 |j 2 ]}{ζ i2,j1,j2 -IE[ζ i2,j1,j2 |j 1 ] -IE[ζ i2,j1,j2 |j 2 ]} T = r2 n,n (n ) 2 n 4 n i1,i2=1 n j1,j2=1 IE ζ i1,j1,j2 ζ T i2,j1,j2 -2IE IE[ζ i1,j1,j2 |j 1 ]IE[ζ i2,j1,j2 |j 1 ] T =: v 4,3,1 -v 4,3,2 .
By now usual techniques, we get

v 4,3,1 = r2 n,n (n ) 2 n 4 n i1,i2=1 n j1,j2=1 IE ζ i1,j1,j2 ζ T i2,j1,j2 r2 n,n (n ) 2 n 4 n i=1 n j1,j2=1 IE ζ i,j1,j2 ζ T i,j1,j2 r2 n,n (n ) 2 n 2 n i=1 ψ(z i )ψ T (z i ) f 4 Z (z i ) K 2 h (z i -z 1 )K 2 h (z i -z 2 ) g * (x 1 , x 2 ) -τ i 2 f (x 1 , z 1 ) × f (x 2 , z 2 ) dx 1 dz 1 dx 2 dz 2 = O r2 n,n n n 2 h 2p = O 1 nh p = o(1).
Moreover,

v 4,3,2 = r2 n,n (n ) 2 n 4 n i1,i2=1 n j1,j2=1 IE IE[ζ i1,j1,j2 |j 1 ]IE[ζ T i2,j1,j2 |j 1 ] r2 n,n (n ) 2 n 4 n i=1 n j1,j2=1 IE IE[ζ i,j1,j2 |j 1 ]IE[ζ T i,j1,j2 |j 1 ] r2 n,n (n ) 2 n 2 n i=1 ψ(z i )ψ T (z i ) f 4 Z (z i ) K 2 h (z i -z 1 )K h (z i -z 2 )K h (z i -z 3 ) g * (x 1 , x 2 ) -τ i × g * (x 1 , x 3 ) -τ i f (x 1 , z 1 )f (x 2 , z 2 )f (x 3 , z 3 ) dx 1 dz 1 dx 2 dz 2 dx 3 dz 3 = O r2 n,n n n 2 h p = O 1 n = o(1).
Another case of two identities occurs when j 1 = j 4 and j 2 = j 3 , but it can be dealt similarly. Then, we have proved that IE[T 4,3 T T 4,3 ] = o(1) and T 4,3 = o P (1).

Convergence of T 6 to 0

Replacing fZ in the definition of T 6 above by the normalized sum of the kernels, we get

T 6 = rn,n n n 2 n i=1 n j1=1 n j2=1 K h (z i -Z j1 )K h (z i -Z j2 ) f 3 Z (z i ) IE[ fZ (z i )] -f Z (z i ) × g * (X j1 , X j2 ) -IE g * (X 1 , X 2 )|Z 1 = Z 2 = z i ψ(z i ) + rn,n n n 3 n i=1 n j1=1 n j2=1 n j3=1 K h (z i -Z j1 )K h (z i -Z j2 ) f 3 Z (z i ) K h (z i -Z j3 ) -IE[ fZ (z i )] × g * (X j1 , X j2 ) -IE g * (X 1 , X 2 )|Z 1 = Z 2 = z i ψ(z i ) =: T 6,1 + T 6,2 .
The first term T 6,1 is a bias term. By Assumptions 6.9.1-6.9.2,

sup i=1,...,n IE[ fZ (z i )] -f Z (z i ) = O(h α ).
The sum of the diagonal terms in T 6,1 is

- rn,n n n 2 n i=1 n j=1 K 2 h (z i -Z j ) f 3 Z (z i ) IE[ fZ (z i )] -f Z (z i ) IE g * (X 1 , X 2 )|Z 1 = Z 2 = z i ψ(z i ), that is O IP rn,n h α /(nh p ) .
The sum of the extra-diagonal terms in T 6,1 is the r.v.

T 6,1 := rn,n n n 2 n i=1 1≤j1 =j2≤n K h (z i -Z j1 )K h (z i -Z j2 ) f 3 Z (z i ) IE[ fZ (z i )] -f Z (z i ) × g * (X j1 , X j2 ) -IE g * (X 1 , X 2 )|Z 1 = Z 2 = z i ψ(z i ). Note that z → f Z (z) and (z 1 , z 2 ) → IE g * (X 1 , X 2 )|Z 1 = z 1 , Z 2 = z 2 are α-times continuously differen-
tiable on Z and Z 2 respectively, because of Assumptions 6.9.2 and 6.9.4. By α-order Taylor expansions of such terms, they yield some factors h α . It is easy to check that the expectation of (T 6,1 ) 2 is of order r2 n,n h 2α /(n 2 h 2p ). Therefore,

T 6,1 = O IP rn,n h α nh p = O IP (n ) 1/2 h α √ nh p = o IP (1).
Concerning T 6,2 , we can assume that the indices j 1 , j 2 and j 3 are pairwise distinct. Indeed, the cases of one or two identities among such indices can be easily dealt. They yield an upper bound that is O IP (r n,n h α /(nh p )) as above, and they are negligible. Once we remove such terms from the triple sums (indexed by (j 1 , j 2 , j 3 )) defining T 6,2 , we get the centered r.v. T 6,2 . Let us calculate the second moment of T 6,2 .

IE T 2 6,2 := nn h p n 2 n 6 n i1=1 n i2=1 1≤j1 =j2 =j3≤n 1≤j4 =j5 =j6≤n IE K h (z i1 -Z j1 )K h (z i1 -Z j2 ) f 3 Z (z i1 ) × K h (z i1 -Z j3 ) -IE[ fZ (z i1 )] g * (X j1 , X j2 ) -IE g * (X 1 , X 2 )|Z 1 = Z 2 = z i1 ψ(z i1 ) × K h (z i2 -Z j4 )K h (z i2 -Z j5 ) f 3 Z (z i2 ) K h (z i2 -Z j6 ) -IE[ fZ (z i2 )] × g * (X j4 , X j5 ) -IE g * (X 1 , X 2 )|Z 1 = Z 2 = z i2 ψ(z i2 ) T =: nn h p n 2 n 6 n i1,i2=1 1≤j1 =j2 =j3≤n 1≤j4 =j5 =j6≤n E i1,i2,j1-j6 .
Chapter 6. About Kendall's regression 183 When all the indices of the latter sums are different, the latter expectation is zero. Non zero terms above are obtained only when j 3 and j 6 are equal to some other indices. In the case j 3 = j 6 and no other identity among the indices, we obtain two extra factors h α through α-order limited expansions of

(z 1 , z 2 ) → IE g * (X 1 , X 2 )|Z 1 = z 1 , Z 2 = z 2 .
This yields an order O(nn h p+2α /(nh p )). When j 3 and j 6 are equal to two different indices (j 3 = j 4 and j 6 = j 2 , e.g.), we lose another factor h p but we still benefit from the two latter factors h α . This yields an upper bound O(nn h p+2α /(n 2 h 2p )) = o(1). The other situations can be managed similarly. We get

IE T 2 6,2 = O nn h p+2α nh p = o(1).
Globally, we obtain T 6 → 0 in probability under Assumptions 6.3.1(ii)(a).

Convergence of

T 7 to 0 Since sup i=1,...,n | fZ (z i ) -f Z (z i )| = o IP (1), note that sup i=1,...,n |T 7,i | ≤ 6 f 4 Z,min sup i=1,...,n | fZ (z i ) -f Z (z i )| 2 ,
with a probability arbitrarily close to one. Apply Lemma 6.19 with a fixed t > 0 and z = z i for each

i = 1, . . . , n IP sup i=1,...,n |T 7,i | ≥ 6 f 4 Z,min C K,α h α α! + t 2 ≤ 2n exp - nh p t 2 2f Z,max K 2 + (2/3)C K t . Set t ∝ h α/2 . Deduce sup i=1,...,n |T 7,i | = O IP (h α ) since nh p+α / ln n → ∞ by assumption. Then, |T 7 | ≤ rn,n n n 2 sup i |T 7,i | n i=1 |ψ(z i )| × n j1=1 n j2=1 |K| h (z i -Z j1 )|K| h (z i -Z j2 ) g * (X j1 , X j2 ) -IE g * (X 1 , X 2 )|Z 1 = Z 2 = z i .
The expectation of the double sum is O(h α ), by an α-order limited expansion of (z

1 , z 2 ) → IE g * (X 1 , X 2 )|Z 1 = z 1 , Z 2 = z 2 .
Then, by Markov's inequality, we deduce

T 7 = O IP (r n,n sup i |T 7,i |h α ) = O IP (r n,n h 2α ) = O IP (n nh p+4α ) 1/2 ,
and then T 7 = o IP (1) due to Assumption 6.3.1(ii)(a).

Convergence of T 3 to 0

For every > 0, by Markov's inequality,

IP(|T 3 | > ) ≤ C Λ rn,n 2n n i=1 IE τ1,2|Z=z i -τ 1,2|Z=z i 2 ψ(z i ).
An approximated calculation of IE τ1,2|Z=z i -τ 1,2|Z=z i 2 can be obtained following the steps of the proof of Lemma 6.24. Indeed, it can be easily seen that the order of magnitude of the latter expectation is the same as the variance of U n,i (g * ), and then of its Hájek projection Ûn,i (g). Since the latter variance is O((nh p ) -1 ), we get

IP(|T 3 | > ) ≤ B rn,n nh p , for some constant B.
Since n /(nh p ) → 0, we get T 3 = o IP (1), as claimed.

Technical results concerning the first-step estimator

Three possible choices for g * are given in [START_REF] Derumigny | About kernel-based estimation of the conditional Kendall's tau: finite-distance bounds and asymptotic behavior[END_REF]

g 1 (X i , X j ) := 4 • 1 X i,1 < X j,1 , X i,2 < X j,2 -1, g 2 (X i , X j ) := 1 (X i,1 -X j,1 ).(X i,2 -X j,2 ) > 0 -1 (X i,1 -X j,1 ).(X i,2 -X j,2 ) < 0 , g 3 (X i , X j ) := 1 -4 • 1 X i,1 < X j,1 , X i,2 > X j,2 ,
where 1 is the indicator function. In the following, we assume that we have chosen g * as one of the g k for a fixed k ∈ {1, 2, 3}. Assumption 6.9.1. The kernel K is bounded, and set K ∞ =: C K . It is symmetrical and satisfies

K = 1, |K| < ∞.
This kernel is of order α for some integer α > 1: for all j = 1, . . . , α -1 and every indices i 1 , . . . , i j in {1, . . . , p}, R p K(u)u i1 . . . u ij du = 0,. Assumption 6.9.2. f Z is α-times continuously differentiable and there exists a constant C K,α > 0 s.t., for all z ∈ Z,

|K|(u) p i1,...,iα=1 |u i1 . . . u iα | sup t∈[0,1] ∂ α f Z ∂z i1 . . . ∂z iα (z + tu) du ≤ C K,α .
Assumption 6.9.3. There exist two positive constants f Z,min and f Z,max such that, for every z ∈ Z,

f Z,min ≤ f Z (z) ≤ f Z,max .
Lemma 6.19. Under Assumptions 6.9.1, 6.9.2 and 6.9.3, we have for any t > 0,

IP fZ (z) -f Z (z) ≥ C K,α h α α! + t ≤ 2 exp - nh p t 2 2f Z,max K 2 + (2/3)C K t .
Lemma 6.20. Under Assumptions 6.9.1-6.9.3 and if C K,α h α /α! < f Z,min , the estimator fZ (z) is strictly positive with a probability larger than

1 -2 exp -nh p f Z,min -C K,α h α /α! 2 / 2f Z,max K 2 + (2/3)C K (f Z,min -C K,α h α /α!) .
Assumption 6.9.4. For every x ∈ R 2 , z → f X,Z (x, z) is differentiable almost everywhere up to the order α, z ∈ Z. For every 0 ≤ k ≤ α and every

1 ≤ i 1 , . . . , i α ≤ p, let H k, ι (u, v, x 1 , x 2 , z) := sup t∈[0,1] ∂ k f X,Z ∂z i1 . . . ∂z i k x 1 , z + tu ∂ α-k f X,Z ∂z i k+1 . . . ∂z iα x 2 , z + tv , denoting ι = (i 1 , . . . , i α ). Assume that H k, ι (u, v, x 1 , x 2 , z
) is integrable and there exists a finite constant

C XZ,α > 0, such that, for every z ∈ Z, |K|(u)|K|(v) α k=0 α k p i1,...,iα=1 H k, ι (u, v, x 1 , x 2 , z)|u i1 . . . u i k v i k+1 . . . v iα | du dv dx 1 dx 2 is less than C XZ,α .
Lemma 6.21 (Exponential bound for the estimated conditional Kendall's tau). Under Assumptions 6.9.1-6.9.4, for every t > 0 such that C K,α h α /α! + t ≤ f Z,min /2 and every t > 0, we have

IP |τ (k) 1,2|Z=z -τ 1,2|Z=z | > c k f 2 z (z) C XZ,α h α α! + 3f z (z) K 2 2nh p + t × 1 + 16f 2 Z (z) f 3 Z,min C K,α h α α! + t ≤ 2 exp - nh p t 2 2f Z,max K 2 + (2/3)C K t + 2 exp - (n -1)h 2p t 2 4f 2 Z,max ( K 2 ) 2 + (8/3)C 2 K t + 2 exp - nh p (f z (z) -C K,2 h 2 ) 2 8f Z,max K2 + 4C K (f z (z) -C K,2 h 2 )/3 ,
with c 1 := c 3 := 4 and c 2 := 2.

Remark 6.22. In Lemma 6.20 and 6.21, f Z,min can be replaced by f Z (z). Moreover, when the support of

K is included in [-c, c] for some c > 0, f Z,max can be replaced by sup z∈V (z, ) f Z (z), denoting by V(z, )
a closed ball of center z and any radius > 0, when n c < . Lemma 6.23 (Consistency). Under Assumption 6.9.1, if nh p n → ∞, lim K(t)|t| p = 0 when |t| → ∞, f Z and z → τ 1,2|Z=z are continuous on Z, then τ1,2|Z=z tends to τ 1,2|Z=z in probability, when n → ∞.

To derive the asymptotic law of this estimator, we will assume: Assumption 6.9.5.

(i) nh p n → ∞ and nh p+2α n → 0; (ii) K( • ) is compactly supported.
Lemma 6.24 (Asymptotic normality). Assume 6.9.1, 6.9.4, 6.9.5, that the z i are distinct and that f Z and z → f X,Z (x, z) are continuous on Z, for every x.

Then,

(nh p n ) 1/2 τ1,2|Z=z i -τ 1,2|Z=z i i=1,...,n D -→ N (0, H) as n → ∞, where H is a n × n real matrix defined by [H] i,j = 4 K 2 1 {z i =z j } f Z (z i ) IE[g(X 1 , X)g(X 2 , X)|Z = Z 1 = Z 2 = z i ] -τ 2 1,2|Z=z i ,
for every 1 ≤ i, j ≤ n , and (X, Z), (X 1 , Z 1 ), (X 2 , Z 2 ) are independent copies, where g is the symmetrized version g(x 1 , x 2 ) := g * (x 1 , x 2 ) + g * (x 2 , x 1 ))/2. τ1,2|Z=z (red curve), and prediction Λ (-1) ψ(z) T β (blue curve) as a function of z. For the blue curve, the regularization parameter is 2 λcv 0.034 where λcv is selected by Algorithm 6.

Estimation results for a particular sample

Chapter 7

A classification point-of-view on conditional Kendall's tau

Abstract

We show how the problem of estimating conditional Kendall's tau can be rewritten as a classification task. Conditional Kendall's tau is a conditional dependence parameter that is a characteristic of a given pair of random variables. The goal is to predict whether the pair is concordant (value of 1) or discordant (value of -1) conditionally on some covariates. We prove the consistency and the asymptotic normality of a family of penalized approximate maximum likelihood estimators, including the equivalent of the logit and probit regressions in our framework. Then, we detail specific algorithms adapting usual machine learning techniques, including nearest neighbors, decision trees, random forests and neural networks, to the setting of the estimation of conditional Kendall's tau. Finite sample properties of these estimators and their sensitivities to each component of the data-generating process are assessed in a simulation study. Finally, we apply all these estimators to a dataset of European stock indices.

Keywords

Introduction

Beside linear correlations, most dependence measures between two random variables are functions of the underlying copula only: Spearman's rho, Kendall's tau, Blomqvist's beta, Gini's measure of association, etc. As a consequence, they are independent of the corresponding margins. This is seen as a positive point. See Joe [START_REF] Joe | Dependence Modeling with copulas[END_REF], Nelsen [START_REF] Nelsen | An introduction to copulas[END_REF], for instance, and, as a reminder, some basic definitions in 7.7. Such measures are well-known and widely used by practitioners. When some covariates are available, natural extensions of these tools can be defined, providing so-called "conditional" measures of dependence. In theory, it is sufficient to replace copulas by conditional copulas to obtain the "conditional version" of any dependence measure. Surprisingly, this simple and fruitful idea has not yet been Chapter 7. A classification point-of-view on conditional Kendall's tau 188 widely used in the literature. Nonetheless, in a series of papers, Gijbels et al. [START_REF] Gijbels | Conditional copulas, association measures and their applications[END_REF][START_REF] Gijbels | Estimation of a conditional copula and association measures[END_REF][START_REF] Gijbels | Multivariate and functional covariates and conditional copulas[END_REF][START_REF] Gijbels | Partial and average copulas and association measures[END_REF] have popularized this approach, with a focus on conditional Kendall's tau and Spearman's rho. Note that conditional dependence measures have been invoked in different frameworks, often without any explicit link with conditional copulas: truncated data (Tsai [START_REF] Tsai | Testing the assumption of independence of truncation time and failure time[END_REF], e.g.), multivariate dynamic models (Jondeau and Rockinger [START_REF] Jondeau | The copula-garch model of conditional dependencies: An international stock market application[END_REF], Almeida and Czado [START_REF] Almeida | Efficient bayesian inference for stochastic time-varying copula models[END_REF], among others), vine structures (So and Yeung [START_REF] So | Vine-copula garch model with dynamic conditional dependence[END_REF]), etc. Now, let us introduce our key dependence measure: for each z ∈ R p , the conditional Kendall's tau of a bivariate random vector X := (X 1 , X 2 ) given some covariates Z = z may be defined as

τ 1,2|Z=z = IP (X 2,1 -X 1,1 )(X 2,2 -X 1,2 ) > 0 Z 1 = Z 2 = z -IP (X 2,1 -X 1,1 )(X 2,2 -X 1,2 ) < 0 Z 1 = Z 2 = z ,
where

X 1 = (X 1,1 , X 1,2 ) and X 2 = (X 2,1 , X 2,2
) are two independent versions of X. To simplify, we will assume that the law of X given Z = z is continuous w.r.t. the Lebesgue measure, for every z. This implies

τ 1,2|Z=z = 2 IP (X 2,1 -X 1,1 )(X 2,2 -X 1,2 ) > 0 Z 1 = Z 2 = z -1.
A conditional Kendall's tau belongs to the interval [-1, 1] and reflects a positive (τ 1,2|Z=z > 0) or negative (τ 1,2|Z=z < 0) dependence between X 1 and X 2 , given Z = z. Unlike correlations, this measure has the advantage of being always defined, even if some X k , k = 1, 2, has no finite second moments (when it follows a Cauchy distribution, for example).

Some estimators of conditional Kendall's tau have already been proposed in the literature, either as a by-product of the estimation of conditional copulas -see Gijbels et al. [START_REF] Gijbels | Conditional copulas, association measures and their applications[END_REF] and Fermanian and Lopez [START_REF] Fermanian | Single-index copulas[END_REF] -or directly, as in Derumigny and Fermanian [START_REF] Derumigny | About Kendall's regression[END_REF][START_REF] Derumigny | About kernel-based estimation of the conditional Kendall's tau: finite-distance bounds and asymptotic behavior[END_REF]. Nonetheless, to the best of our knowledge,

nobody has yet noticed the relationship between conditional Kendall's tau and classification methods.

Let us explain this simple idea. Denote

W := 2 × 1{(X 2,1 -X 1,1 )(X 2,2 -X 1,2 ) > 0} -1 and
IP (X 2,1 -X 1,1 )(X 2,2 -X 1,2 ) > 0 Z 1 = Z 2 = z = IP W = 1 Z 1 = Z 2 = z =: p(z).
Actually, the prediction of concordance/discordance among pairs of observations (X 1 , X 2 ) given Z can be seen as a classification task of such pairs. If a model is able to evaluate the conditional probability of observing concordant pairs of observations, then it is able to evaluate conditional Kendall's tau, and the former quantity is one of the outputs of most classification techniques. Therefore, most classifiers can potentially be invoked (for example linear classifiers, decision trees, random forests, neural networks and so on [START_REF] Friedman | The elements of statistical learning[END_REF]), but applied here to pairs of observations. Indeed, for every 1 ≤ i, j ≤ n, i = j, define W (i,j) as

W (i,j) := 2 × 1{(X j,1 -X i,1 )(X j,2 -X i,2 ) > 0} -1 =    1 if (i, j) is a concordant pair, -1 if (i, j) is a discordant pair. (7.1)
A classification technique will allocate a given couple (i, j) into one of the two categories {1, -1} (or "concordant versus discordant", equivalently), with a certain probability, given the value of the common covariate Z.

Section 7.2 introduces a general regression-type approach for the estimation of conditional Kendall's tau. Some asymptotic results of consistency and asymptotic normality are stated. In Section 7.3, we explain how some machine learning techniques can be adapted to deal with our particular framework, and we detail the corresponding algorithms. A small simulation study compares the small-sample properties of all these algorithms in Section 7.4. In Section 7.5, these techniques are applied to European stock market data. We evaluate to what extent the dependence between pairs of European stock indices may evolve with respect to different covariates. All proofs have been postponed into appendices.

Regression-type approach

Typically, a regression-type model based on conditional Kendall's tau may be written as

τ 1,2|Z=z = g 0 (z, β * ), ∀z ∈ Z ⊂ R p , (7.2)
for some finite dimensional parameter β * ∈ R p and some function g 0 . As a particular case, a singleindex approach would be

τ 1,2|Z=z = g(ψ(z) T β * ), ∀z ∈ Z, (7.3)
where ψ : R p → R p is known, and g may be known (parametric model) or not (semi-parametric model),

as in [START_REF] Derumigny | About Kendall's regression[END_REF]. In this section, we propose an inference procedure of β * under (7.3) when the link function g is analytically known. This procedure will be based on the signs of pairs only, and not on the specific values of the vectors X i . Then, since inference will be based on the observations of W ∈ {1, -1}, our model belongs to the family of limited-dependent variable methods. One difficulty will arise from the pointwise conditioning events Z i = Z j = z, that will necessitate localization techniques. Actually, we will consider couples of observations X i and X j for which the associate covariates are close to a given value z. Indeed, the relationship (7.3) does not define the dependence levels between every couple (X i , Z i ) and (X j , Z j ), i = j, but only between those that share the same covariate. If the variables Z were discrete, we would consider a subset of couples such that Z i = Z j . In our case of continuous variables Z (see below), the latter event does not occur almost surely, and some smoothing/localization techniques have to be invoked.

Let K be a p-dimensional kernel and (h n ) be a bandwidth sequence. The bandwidth will simply be denoted by h and we set K h (z) = K(z/h)/h p . The log-likelihood associated to the observation

(W (i,j) , Z i , Z j ) given Z i = Z j = z is β (W (i,j) , z) := 1 + W (i,j) 2 log IP β W (i,j) = 1 Z i = Z j = z + 1 -W (i,j) 2 log IP β W (i,j) = -1 Z i = Z j = z .
In practice, when the underlying law of Z is continuous, there is virtually no couple for which Z i = Z j .

Therefore, we will consider a localized "approximated" log-likelihood, based on (W (i,j) , Z i , Z j ) for all pairs (i, j), i = j. It will be defined as the double sum

L n (β) := 1 n(n -1) i,j;i =j K h (Z i -Z j ) β (W (i,j) , Zi,j ),
for any choice of Zi,j that belongs to a neighborhood of Z i or Z j . We will assume that K is a compactly supported p-dimensional kernel of order m ≥ 2.

The most obvious choices would be to select Zi,j among {Z i , Z j , (Z i + Z j )/2}. Here, we propose

L n (β) := 1 n(n -1) i,j;i =j K h (Z i -Z j ) β (W (i,j) , Z i ) = 1 n(n -1) i,j;i =j K h (Z i -Z j ) 1 + W (i,j) 2 log 1 2 + 1 2 g(ψ(Z i ) T β) + 1 -W (i,j) 2 log 1 2 - 1 2 g(ψ(Z i ) T β) , under (7.3)
. We can therefore derive an estimator of β * based on the maximization of the latter function, with a 1 penalty (Lasso-type estimator), as

β := arg max β∈R p L n (β) -λ n |β| 1 , (7.4)
where λ n (also simply denoted as λ) is a tuning parameter to be chosen. Note that L n (β) is not really a likelihood function since the observations (W (i,j) , Z i , Z j ) for every couple (i, j), i = j, are not mutually independent.

If K ≥ 0, the objective function is a concave function of β if it satisfies

δg (t)(1 + δg)(t) ≤ (g ) 2 (t), ∀t, (7.5)
for δ ∈ {1, -1}. When β → L n (β) is concave, the penalized criterion above is concave too and the calculation of β can be led in practical terms through convex optimization routines, even with a large number of regressors (p 1). Since this will be our framework, we will show that (7.5) holds for some usual classification techniques. When it is not the case, we have to rely on other optimization schemes and to avoid considering too many regressors.

Moreover, note that, when g is odd (i.e. g(-t) = -g(t)), the estimator simply becomes

β := arg max β∈R p 1 n(n -1) i,j;i =j K h (Z i -Z j ) log 1 2 + 1 2 g(W (i,j) ψ(Z i ) T β) -λ|β| 1 . (7.6)
The implementation of an algorithm to solve problem (7.4) or its simplified version (7.6) may seem difficult due to the non-differentiability of the l 1 norm. Nevertheless, as in the case of the ordinary Lasso, it can be solved in a very efficient way using the Alternative Direction Method of Multipliers (ADMM)

for general l 1 minimization, following [24, Section 6.1]. More precisely, assume L n (β) is a concave and differentiable function of β (this is the case in both Examples 7.1 and 7.2). Then the optimization task (7.4) can be rewritten as finding the solution

(x, z) ∈ R 2p of    minimize f (x) + g(z) subject to x -z = 0, (7.7)
where f (x) := -L n (x) and g(z) := λ n |z| 1 . The solution is given by iterating the following algorithm, denoting by u ∈ R p the dual variable of the problem (7.7) and by ρ > 0 the step size (similarly to the usual gradient descent algorithm),

x k+1 := arg min x f (x) + (ρ/2)||x -z k + u k || 2 2 , z k+1 := S λn/ρ (x k+1 + u k ), u k+1 := u k + x k+1 -z k+1 ,
where for any κ > 0, S κ is the element-wise soft thresholding operator, i.e. for each component S κ (a) :=

(1-κ/|a|) + ×a, for a = 0, and S κ (0) := 0. Note that we have reduced the non-differentiable problem (7.4) into a sequence of differentiable optimization steps for x, and the computation of the proximal operator S κ for the z-updates. We refer to [START_REF] Parikh | Proximal algorithms[END_REF] for a detailed presentation about proximal operators and their use in optimization. ADMM can also be adapted for large-scale data, using standard libraries and frameworks for parallel computing such as MPI, MapReduce and Hadoop, see [START_REF] Boyd | Distributed optimization and statistical learning via the alternating direction method of multipliers[END_REF] for more details about the implementation of such methods.

Example 7.1 (Logit). If we choose the Fisher transform g(t) = (e t -1)/(e t + 1), then g is odd and the optimization program becomes

β := arg max β∈R p 1 n(n -1) i,j;i =j K h (Z i -Z j ) log logit(W (i,j) ψ(Z i ) T β) -λ|β| 1 ,
Chapter 7. A classification point-of-view on conditional Kendall's tau 191 where the so-called logit link function is defined by logit(x) = e x /(1 + e x ). Therefore β can be seen as the maximizer of the log-likelihood of a weighted logistic regression with independent realizations of an explained variable W (i,j) , given some explanatory variables Z i . On a practical side, when K ≥ 0, the β-criterion is concave. This allows to use the existing software and optimization routines of logistic regression without many changes.

Example 7.2 (Probit). Similarly, choosing g(t) = 2Φ(t) -1, where Φ denotes the cdf of the standard normal distribution, yieds the equivalent of a (weighted) probit regression. Indeed, this function g is odd, (7.6) applies in this case and our criterion in (7.4) is concave wrt β.

Let us assume that a family of models or some statistical procedure allow the calculation of the functional link g( ψ(z) T β) and then p(z), for any z, ∈ {-1, 1} and any given value β: logit, probit, regression trees, neural networks, etc. Then, we can estimate the "true" parameter β * by β, as given by (7.4), in practical terms. Now, we state the asymptotic properties of β, under the assumption that β → L n (β) is concave. To this goal, we introduce some notation.

For any x and y ∈ R p , denote

p(x, y) := IP β * ((X 2,1 -X 1,1 )(X 2,2 -X 1,2 ) > 0|Z 1 = x, Z 2 = y).
The latter expectations are calculated when the underlying parameter is assumed to be the true value β * . Note that p(x) := p(x, x) and 2p(z) -1 = τ 1,2|Z=z . Moreover, for any x, y and z ∈ R p , set p(x, y, z)

:= IP β * ((X 2,1 -X 1,1 )(X 2,2 -X 1,2 ) > 0, (X 3,1 -X 1,1 )(X 3,2 -X 1,2 ) > 0|Z 1 = x, Z 2 = y, Z 3 = z).
This is the conditional probability that X 1 , X 2 and X 3 are concordant, given their respective covariates.

Denote, for any β ∈ R p , φ(x, y, β) := p(x, y) log (q(x, β)) + (1 -p(x, y)) log (1 -q(x, β)) , q(x, β) := 1/2 + g(ψ(x) T β)/2.

Note that q(z, β * ) = p(z). Finally, for any real function f and ε > 0, denote by

f ε the function x → sup t, |x-t|<ε |f (t)|.
Regularity assumption R0: The density f Z of Z is assumed to be m-times continuously differentiable.

Moreover, the functions φ(x, •, β) and q(•, β) are continuous for any x ∈ Z and any β ∈ R p . To simplify, (x, y, z) → p(x, y, z) will be continuous on Z 3 . Theorem 7.3. Under R0, (7.14) and (7.15)

in 7.8, if λ n → λ ∞ and n 2 h p → ∞ when n → ∞, if the
true model is given by (7.3) and β → L n (β) is concave, then the solution β of (7.4) tends in probability

towards β * * := arg max β L ∞ (β) -λ ∞ |β| 1 , where L ∞ (β) := φ(z, z, β)f 2 Z (z) dz.
In particular, when λ ∞ = 0, the estimator β tends to arg max β L ∞ (β) = β * , because φ(z, z, β) is the expected log-likelihood associated to W (1,2) given Z 1 = Z 2 = z. Thus, for every z, the latter quantity is maximal when β = β * .

Theorem 7.4. Under the conditions of Theorem 7.3 and some additional conditions of regularity in 7.9

(notably (7.16), (7.17), (7.18) and (7.19)

), if n 1/2 λ n → µ and nh p → ∞ when n → ∞, then n 1/2 ( β -β * )
weakly tends to

u * = arg min u∈R p W(β * )u + 1 2 u T H(β * )u -µ k;β * k =0 |u k | -µ k;β * k =0 sign(β * k )u k , where W(β * ) ∼ N (0 p , Σ β * ), Σ β * = ∂ β φ(z, z, β * )∂ β φ(z, z, β * ) T f 3 Z (z) dz and H(β * ) = ∂ 2 β,β T φ(z, z, β * )f 2 Z (z) dz.
Remark 7.5. All the previous results and those of the next sections are based on the kernel-weighted log-likelihood criterion L n (β), and then on the choice of the bandwidth h. We have not tried to find an "optimal" smoothing parameter h. This task is outside the scope of this paper and is left for further research. Instead, we have preferred to rely on the usual rule-of-thumb (Scott [START_REF] Scott | Multivariate Density Estimation: Theory, Practice, and Visualization[END_REF]), even if, strictly speaking, it is relevant only for kernel estimators of densities. Nonetheless, we have not empirically found an "excessive sensitivity" of our simulation results w.r.t. h.

Classification algorithms and conditional Kendall's tau

In the latter section, we have studied a localized likelihood procedure to estimate β * under (7.3), when we can explicitly write (and code) the link function g. This may be seen as a restrictive approach, because it is far from obvious to guess the right functional form of g. To improve the level of flexibility of our conditional Kendall's tau model, we recall the estimation of τ 1,2|z is similar to the evaluation of

IP W (1,2) = 1 Z 1 = Z 2 = z , i.
e. the probability p(z) of classifying the couple (1, 2) into one of two categories (concordant or discordant), given a common value z of their covariates. Formally, the answer of such a question can be directly yielded by some classification algorithms. This is the topic of this section. Therefore, instead of estimating an assumed parametric model by penalization, as in (7.4), a classification algorithm will "automatically" evaluate p(z) by p(z). An estimator of the conditional Kendall's tau will simply be τ1,2|Z=z := 2p(z) -1.

Now, we show how different classification algorithms can be used and adapted to the estimation of τ 1,2|Z=z in practice. The first step is to transform the dataset D = (X i,1 , X i,2 , Z i ) i=1,...,n ∈ R 2+p , called the initial dataset, into an object D, that will be called the dataset of pairs (see Algorithm 7). Each element of this dataset of pairs is indexed by an integer k ∈ {1, . . . , n(n -1)/2}, which corresponds to any (unordered) pair (i, j), i = j, of observations in the initial dataset.

For any pair of observations, we compute the associated covariate Zk which is just the average of the two covariates Z i and Z j (contrary to Section 7.2 where we have chosen Z i ). Note that we want Z i and Z j to be close to each other, so that the pair (i, j) is relevant. This means that a weight variable V k is defined for any pair. It is related to the proximity between Z i and Z j . Obviously, if V k = 0 then the corresponding pair is not kept, finally. This selection induces also a computational benefit, by reducing the size of the dataset and the computation time. For example, suppose that n = 4000. Then, up to around 8 × 10 6 possible pairs can be constructed but only a small group of them (around 10 4 or 10 5 pairs, typically) will be relevant. The others are pairs for which the covariates are considered too far apart. Note that, in order to increase the proportion of k such that the weight V k is zero, it is sufficient to use compactly supported kernels. For instance, for any arbitrary p-dimensional kernel K, we can consider K(z) := γK(z)1{ z ∞ ≤ 1}, with some normalizing constant γ so that K = 1.

Algorithm 10: Random forests adapted for the estimation of the conditional Kendall's tau

Input: Initial dataset D = (X i,1 , X i,2 , Z i ) i=1,...,n ∈ R 2+p n ;
for j ← 1 to N tree do Sample a set S j ⊂ {1, . . . , n} without replacement ;

D j ← (X i,1 , X i,2 , Z i ) i∈S j ;
Compute the dataset of pairs Dj = (W k , Zk , V k ) k∈Kj using Algorithm 7 on D j , providing K j ;

Sample a set S j ⊂ {1, . . . , p } without replacement ;

Estimate a classification tree T j on the dataset (W k , (ψ l ( Zk )) l∈S j , V k ) k∈Kj ; end Output: An estimator τ1,2|Z=• := N -1 tree Ntree j=1 T j (•).

Nearest neighbors

The nearest neighbors also provide a very popular classification algorithm and can be directly used on the dataset D (see Algorithm 11). Here, we no longer assume (7.3) or even (7.2), and we live in a nonparametric framework. A pretty difficult problem is to choose a convenient number of nearest neighbors. As usual in nonparametric statistics, we must find a compromise between variance (tendancy to undersmooth, i.e. to choose a too small N ) and bias (tendancy to oversmooth, i.e. to choose a too big N ). Moreover, in our case, with n(n -1)/2 possible pairs, choosing a right value for N can be challenging. Indeed, in the usual (i.i.d.) nearest neighbor framework, the asymptotically optimal N is a power of the sample size. Here, this is different because there are three potential sample sizes: n, if we consider there are fundamentally n sources of randomness, n(n -1)/2 by considering that the new sample has a cardinality equal to the number of pairs, or even |K| that is random and depends on h.

Thus, our problem is to choose a "relevant formula" for N based on the "convenient" sample size.

Algorithm 11: Estimation of the conditional Kendall's tau τ 1,2|Z=z using nearest neighbors.

Input: A dataset of pairs D := (W k , Zk , V k ) k∈K Input: A point z ∈ Z, a number N of nearest neighbors and a distance d on R p ; K z ← arg min E⊂K,|E|=N k∈E d ψ(z), ψ( Zk ) ; Output: An estimator τ (N ) 1,2|Z=z := k∈Kz V k W k / k∈Kz V k .
In applications, one might not be interested in the value of the conditional Kendall's tau at only one point, but also in the whole function z → τ 1,2|Z=z . The goodness of this estimation is linked to the underlying density f Z of Z: the estimation can be made more precise in regions where f Z is high, allowing to use a higher number of neighbors with close covariates. At the opposite, in some regions where f Z is low, a smaller N should be used. Note that, in general, f Z is unknown and its estimation may be difficult as well, due to the curse of dimensionality. Therefore, it is highly desirable to build a local number of neighbors N (z). Such a local choice N (z) will help to avoid both under-and over-smoothing in all parts of the space Z.

Cross-validation techniques are widely use for the choice of tuning parameters, but might not be here the best solution as one would like to find a local choice of N . This problem has similarities with classical non-parametric regression. We propose to use a procedure inspired by Lepski's method for choosing the bandwidth [START_REF] Lepski | Optimal pointwise adaptive methods in nonparametric estimation[END_REF], once adapted to our setting. Lepski's method is built on a simple principle: when two non-parametric estimators are close, the best is the smoothest. When two non-parametric estimators are far apart, the best is the least smooth. Let (Z i ) i∈I be a partition of Z. The goal will be to choose the best estimator on each Z i , which corresponds to the choice of a local number of nearest neighbors N i . This procedure is called "local" since the diameters of the Z i will be small. For example, if p = 1 and Z is a bounded interval then the Z i can be chosen as small intervals. We denote by N ⊂ N the finite set of possible numbers of neighbors. Following Lepski's approach, we choose N as a geometric progression, i.e. N = { a 1 × a i 2 , i = 1, . . . , i max } for some constants a 1 , a 2 > 0, where x denotes the integer part of a real x.

To measure how far the estimators are from each other, we introduce a distance d i , i ∈ I. As our estimators of conditional Kendall's tau are bounded (between -1 and 1) and measurable, several choices are possible. In applications, we will use

d i (f, g) = 1 j max jmax j=1 (f (z i,j ) -g(z i,j ))/M 2 1/2 , z i,1 , . . . , z i,jmax ∈ Z i , (7.8)
where M is a normalization factor independent of i and the subsets z i,1 , . . . , z i,jmax are arbitrarily chosen in Z i , i = 1, . . . , i max . We will use M = (max -min){τ

(N )
1,2|Z=z , N ∈ N , z ∈ Z}. Indeed, in the classical nonparametric regression model Y = f (X)+ε, with an unknown function f , M should be replaced by the standard deviation of the noise ε. In our case, we can define a (pseudo-)noise ξ z,N := τ (N ) 1,2|Z=z -τ 1,2|Z=z , but it is unknown in practice and its distribution is complicated. Therefore M serves as a proxy of the amplitude of the variations in the estimated conditional Kendall's tau. This normalization by M ensures a kind of adaptativity of the estimation.

Algorithm 12: Lepski's method for a local choice of the number of nearest neighbors, and the corresponding estimator of the conditional Kendall's tau.

Input: A set N ⊂ N of possible numbers of nearest neighbors and the corresponding estimates

τ (N )
1,2|Z=• given by Algorithm 11, for all N ∈ N ; Input: A partition (Z i ) i∈I of Z and a distance d i on a space of bounded measurable real functions defined on Z i , for every i ∈ I;

foreach i ∈ I do S i ← N ∈ N : d i τ (N ) 1,2|Z=• , τ (N ) 1,2|Z=• ≤ A (1/N ) log(max(N )/N ), ∀N ∈ N ∩ [1, N ] ; (7.9)
N i ← max(S i ); end Output: An estimator z → τ1,2|Z=z := i∈I 1{z ∈ Z i }τ (Ni) 1,2|Z=z .
We have observed that the sensitivity to N is not too large, if it is chosen in a reasonable way, for example between 5 or 10 possibilities. When Z is univariate, a simple partition (Z i ) i∈I can be given by the deciles of Z. We choose A = 1 for simplicity since we believe there is no procedure for choosing it. A statistician who would like to play with the smoothness of the result is free to adjust the constant A, using an expert knowledge of the situation. Finally, the z i,j can be chosen as quantiles of Z, or as a regular grid on each Z i .

Neural networks

Nowadays, neural networks have become very popular with a wide range of applications. In classification problems, a neural network can be seen as an estimator that depends on some parameters, but in a very flexible and complex way. For every input z, it yields the probability of belonging to any class. In our framework, we will train a network on the dataset of pairs D. It is well-known that most neural networks do not induce convex programs, and the outputs therefore depend on some initial parameter values.

One strategy is to independently train networks with different starting parameter values, that may be randomly chosen, for example.

This method of using independent estimators (conditionally on the initial sample D) and then aggregating them is related to the random forest approach of the previous section and the discussion therein.

Therefore, the same techniques are relevant and we have noticed an improvement in terms of performance by using an adapted version of Algorithm 10. More precisely, we fix a number of neural networks.

For each neural network, we sample without replacement a part of the initial dataset from which the corresponding dataset of pairs is constructed and used as a training set. In order to improve stability, we aggregate the predictions of the different neural networks by using their median as the final predicted Kendall's tau. There is a trade-off between computation time and accuracy: a larger number of networks should improve the accuracy while taking obviously a longer time to be trained. The precise choice of the best architecture of the network is a complicated task, which is left for future research. As we are looking for functions z → τ 1,2|Z=z which are smooth almost everywhere and easy to interpret in applications, we choose a simple architecture with N nnet = 10 neural networks, each having a single hidden layer of 3 neurons. Besides, bigger networks seem to deteriorate the performance of this estimator, see Section 7.4.6.

Algorithm 13: Neural networks with median bagging, adapted for the estimation of the conditional Kendall's tau

Input: Initial dataset D = (X i,1 , X i,2 , Z i ) i=1,...,n ∈ R 2+p n ; for j ← 1 to N nnet do
Sample a set S j ⊂ {1, . . . , n} without replacement ;

D j ← (X i,1 , X i,2 , Z i ) i∈S j ;
Compute the dataset of pairs Dj = (W k , Zk , V k ) k∈Kj using Algorithm 7 on D j , providing K j ;

Estimate a neural net N j on the dataset

(W k , ψ( Zk ), V k ) k∈Kj ; end Output: An estimator τ1,2|Z=• := M edian{N j (•), j = 1, . . . , N nnet }.

Lack of independence and its influence on the proposed algorithms

The machine learning methods that are discussed in this section were all designed for i.i.d. data. But it is easy to see that some observations in the dataset of pairs D will not be independent. Indeed, assume that the observations in the original dataset (X i,1 , X i,2 , Z i ) i=1,...,n are i.i.d., to simplify. The pair

(i = 1, j = 2)
and the pair (i = 1, j = 3) both involve the first observation (X 1,1 , X 1,2 , Z 1 ), and therefore are not independent. This is a theoretical problem, but numerical results in Section 7.4 show that this does not often seem to be a problem in practice.

As far as the logit and probit are concerned, it was proved in the previous Section 7.2 that they are related to a family of estimators that can use D "as is". They yield consistent and asymptotically normal estimates, nonetheless, if the specification is correct. It is likely that the other methods presented here enjoy similar properties and are also largely unaffected by dependence between pairs. Note that, if all observations in D are identically distributed, then the observations in D are identically distributed as well.

This is favourable to our methods.

Concerning the dependence inside D, we will show that it is not too strong. For example, the pairs (1, 2) and (1, 3) are not independent, but the pairs (1, 2) and (3, 4) are indeed independent. This means that there is still "a large proportion of" independence left in D. Formally, if two distinct pairs are randomly chosen in D, the probability that they are really independent is high. Indeed, there are N tot := n(n -1)(n(n -1) -2)/8 couples of distinct pairs. Beside, the number N ind of couples of pairs which are independent is N ind := n(n -1)(n -2)(n -3)/8. The factor 1/8 appears in both N tot and N ind since we can always switch the two observations in the first pair, in the second pair, and switch the two pairs (every 4-tuple is counted

2 3 = 8 times). It is easy to see that N ind /N tot = 1 -O(1/n) as n → ∞.
This means that the pairs are "almost all" independent from each other, as n → ∞. In other words, the dependence between two pairs become negligible with averages. That is the reason why the machine learning methods used will perform well if the original dataset D is large enough. If the original dataset D is not i.i.d., for example as observations of a time series, we conjecture that such methods will work in a similar way as long as dependence is not too strong, for example if the data-generating process satisfies some usual assumptions, see Remark

Whenever bootstrap, subsetting, resampling, or cross-validation is led on these classification-based estimators, we advise to perform them on the original dataset D rather than on the dataset of pairs D, as we did in Sections 7.3.1, 7.3.2 and 7.3.4. This seems to yield a good improvement in performance. An example is given by the difference between Algorithms 9 and 10. This can be simply summed up as "do the resampling on the original dataset D, not on the transformed dataset D". Nevertheless, a complete study and justification of this general principle is beyond the scope of this paper and is left for future work.

Simulation study

In this section, we have studied the relative performances of our estimators by simulation. For a given model and a given method of estimation, we sample 100 different experiments, and estimate the model for each sample. We fix the sample size as n = 3000. We remark that, for a given dimension p > 0 of Z and a given support Z of Z, we have different "blocks" of the model which can be chosen in an independent way:

(i) the law IP Z of Z, (ii) the function z ∈ Z → τ 1,2|Z=z , (iii)
the (conditional) copula family (C τ ) τ ∈(0,1) or (C τ ) τ ∈(-1,1) of (X 1 , X 2 )|Z = z, indexed by its conditional Kendall's tau -for example the Gaussian, Student, Clayton, Gumbel, etc, copula families.

Such a family can also depend on Z: for example, think of a Student copula with varying degrees of freedom -, (iv) the conditional margins X 1 |Z and X 2 |Z, (v) the choice of the functions ψ i , for i = 1, . . . , p , (vi) the choice of the estimator τ1,2|Z=• .

Our so-called "reference setting" will be defined as p = 1, Z = [0, 1] and (i)

IP Z = U [0;1] ; (ii) τ 1,2|Z=z = 3z(1 -z) ; (iii) (C τ ) τ ∈(0,1)
is the Gaussian Copula family ; (iv) IP X1|Z=z = IP X2|Z=z = N (z, 1).

For each tested model, the performance of the estimator will be evaluated by the mean integrated 2

error. With obvious notation, it will be estimated as

Err := IE Z (τ 1,2|Z=z -τ 1,2|Z=z ) 2 dz ≈ 1 N simu N points Nsimu i=1 Npoints j=1 (τ (i) 1,2|Z=z (j) -τ 1,2|Z=z (j) ) 2 , (7.10)
where N simu , N points are positive integers, z (1) , . . . , z (Npoints) are fixed points in Z, and

τ (i) 1,2|Z=z (j)
is the estimated conditional Kendall's tau at point z (j) trained on data from the i-th simulation. We choose N simu := 100 experiments, and in this reference setting, the integral is discretized with N points := 100 equispaced points on the segment [0.01, 0.99], to avoid numerical problems at the boundaries.

In the following simulations, "logit" and "probit" refer to Algorithm 8. "Tree" refers to the application of the method tree() of package tree by Ripley [START_REF] Ripley | Tree: classification and regression trees[END_REF] on the dataset D produced by Algorithm 7.

"Random forests" refers to Algorithm 10. "Nearest neighbors" refers to the adapted version using Algorithm 11, once aggregated using Algorithm 12. Finally "Neural networks" refers to Algorithm 13. Such specifications are now part of our "reference setting".

Choice of the functions {ψ

i }, i = 1, . . . , p .
We consider six different choices of ψ, that are 1. No transformation, i.e. ψ

(1)

1 (z) = z.

Polynomials of degree lower than 4: ψ

(2) i (z) = 2 -i+1 (z -0.5) i-1 for i = 1, . . . , 5.

Polynomials of degree lower than 10:

ψ (3) i (z) = 2 -i+1 (z -0.5) i-1 for i = 1, . . . , 11.

Fourier basis of order 2 with an intercept:

ψ (4) 1 (z) = 1, ψ (4)
2i (z) = cos(2πiz) and ψ (4) 2i+1 (z) = sin(2πiz) for i = 1, 2.

Fourier basis of order 5 with an intercept:

ψ (5) 1 (z) = 1, ψ (5)
2i (z) = cos(2πiz) and ψ [START_REF] Acar | Beyond simplified pair-copula constructions[END_REF] 2i+1 (z) = sin(2πiz) for i = 1, . . . , 5. (2) and ψ (4) , which will be denoted by ψ (6) .

Concatenation of ψ

For each of the choices of ψ above, and each estimator, we compute the criterion (7.10). The results are displayed in the following Table 7.1.

With the choice of ψ (6) , logit and probit methods provide the best results. This good performance deteriorates with other choices of ψ, especially when the model is misspecified. Neural networks provide the best results with ψ (1) , and their performance declines when further transformations of z are introduced in ψ. Nearest neighbors have nearly the best behavior with ψ (1) , and it does not seem that other transformations can significantly increase its performance. On the contrary, for trees and random forests, it seems that bigger families ψ can yield improvements over ψ (1) .

From now on, we will choose ψ (6) for the methods logit, probit, tree and random forests. Indeed, for these methods, this choice of ψ yields nearly the lowest error criterion and presents the advantages Chapter 7. A classification point-of-view on conditional Kendall of proposing various shapes, which will help to combine the performances of both polynomials and oscillating functions. On the contrary, for the methods nearest neighbors and neural networks, we choose ψ (1) as adding new functions does not seem to increase the performance of both of these methods. Figure 7.1 displays a comparison of the different methods on a typical simulated sample.

Comparing different copulas families

Now, we keep the reference setting and we change only its part (iii), i.e. the functional form of the conditional copula. The results are displayed in Table 7.3. We observe that such choice of a parametric copula families has nearly no effect on the performance of the estimators. Nonetheless, with the Student copula (either with fixed or variable degrees of freedom), most estimators have slightly worse performances than with other copulas. This can be explained by the fact that this copula allows asymptotic dependence, i.e. a strong tail association.

Copula family

Comparing different conditional margins

In this subsection, we still start from the reference setting and we change only its part (iv), i.e. the functional form of the conditional margins (X 1 |Z) and (X 2 |Z). We consider the following alternatives:

1. IP X1|Z=z = IP X2|Z=z = N (z, 1) (as in the reference case).

2. IP X1|Z=z = N (cos(10πz), 1) ; IP X2|Z=z = N (z, 1). The idea is to make X 1 oscillate fast enough so that the algorithms will have difficulties to localize concordant and discordant pairs ; In a similar way as in the previous section, the results of these experiments, as displayed in Table 7.4 show that changes in terms of conditional marginal distributions generally have a mild impact on the overall performance of the estimators. Moreover, such changes have no effect on the ranking between estimators: the logit and probit methods are always the best, followed by the neural networks, the nearest neighbors, and the random forests are behind (in this order). The estimator Tree shows the lowest performance, but note that it also has the lowest computation time.

Comparing different forms for the conditional Kendall's tau

In this part, we keep the reference setting, but we change only its part (ii), i.e. the functional form of the conditional Kendall's tau itself. We consider the following choices:

1. f 1 (z) := 0.9 -0.8 1{z ≥ 0.5},

2. f 2 (z) := 3z(1 -z),
3. f 3 (z) := 0.5 + 0.4 sin(4πz),

4. f 4 (z) := 0.1 + 1.6z1{z < 0.5} + 1.6(z -0.5)1{u ≥ 0.5}.
The results are presented in Table 7.5. If the estimated model is close to be well-specified, the best methods are parametric, i.e. the logit and probit regressions. In all the other cases, neural networks seem to perform very well. There appears a compromise between a minimization of the error and a minimization of the computation time. We refer to Table 7 construct the dataset of pairs Dn using Algorithm 7. Use the estimators on the training set Dn .

2. Independent situation: fix n = 3000, simulate n(n -1) 9, 000, 000 independent copies D n(n-1) := (X i,1 , X i,2 , Z i ) i=1,...,n(n-1) . Create the dataset of consecutive pairs D n(n-1) on this sample using Algorithm 14. This means that we use only consecutive pairs, i.e. (1,2), (3,4), [START_REF] Acar | Beyond simplified pair-copula constructions[END_REF][START_REF] Almeida | Efficient bayesian inference for stochastic time-varying copula models[END_REF], and so on.

Use the estimators on the training set D n(n-1) .

Algorithm 14: Algorithm for creating the dataset of consecutive pairs from the initial dataset.

Input: Initial dataset D = (X i,1 , X i,2 , Z i ) i=1,...,n ∈ R 2+p n ; for k ← 1 to n /2 do i, j ← 2k -1, 2k ; Zk ← (Z i + Z j )/2 ; W k ← W (i,j)
as defined in Equation (7.1) ;

V k ← K h (Z i -Z j ) ; end Define K := {k : V k > 0} ; Output: A dataset of pairs D := (W k , Zk , V k ) k∈K ∈ {-1, 1} × R p × R + n /2 .
Note that, by construction, the cardinalities of D n(n-1) and Dn are the same, i.e. both have exactly n(n-1)/2 pairs. This is the reason why we chose to simulate n(n-1) points in the independent situation, so that these two numbers of pairs can match. Note that the elements in D are independent from each other by construction while some elements in D may not be independent from each other in general. We can now compare the performances of the estimators trained on D n(n-1) and on Dn using the criterion (7.10). Some results are given in Table 7.9. Note that the simulation of the each (X i,1 , X i,2 , Z i ) is still made under the previous one-dimensional "reference setting". 7.9: Error criterion (7.10) multiplied by 1000 for each method and each situation. "Independent" means the independent situation with D n(n-1) , and "Not independent" means the reference situation with Dn .

Logit

As expected, all estimators show a better performance in the independent situation. Nonetheless, the independent situation has been simulated using n(n -1) 9, 000, 000 points whereas the reference situation uses only n = 3, 000 points. Even if the numbers of pairs in both experiments are the same, the sample size of the dataset was much larger in the independent situation. This means that there is more information available, and explains also why the independent situation has a better performance: it just uses more data. Such a huge sample may not be available in practice though.

Nevertheless, the original procedure costs O(n 2 ), which can be large for very large values of n. In this case, it is always to possible to restrict oneself to consecutive pairs, with a cost of only O(n). Such a procedure is possible if the dataset is very large and Algorithm 14 can be seen as an alternative to Algorithm 7 where only consecutive pairs are used. This would lower the computation cost at the expense of precision.

Applications to financial data

In this section, we study the changes of the conditional dependence between the daily returns of MSCI stock indices during two periods: the European debt crisis (from 18 March 2009 to 26 August 2012) and the after-crisis period (26 August 2012 to 2 March 2018). We will consider the couples (Germany, France), (Germany, Denmark), (Germany, Greece), respectively denoted by (X 1 , X 2 ), (X 1 , X 3 ), (X 1 , X 4 ).

We will separately consider two choices of conditioning variables Z:

• a proxy variable for the intraday volatility σ := (High -Low)/Close, where High denotes the maximum daily value of the Eurostoxx index, Low denotes its minimum and Close is the index value at the end of the corresponding trading day.

• a proxy of so-called "implied volatility moves" ∆σ I . It will record the daily variations of the EuroStoxx 50 Volatility Index, whose quotes are available at https://www.stoxx.com/index-details?symbol= V2TX:

∆σ I i := V 2T X(i) -V 2T X(i -1)
for each trading day i. The EuroStoxx 50 Volatility Index V 2T X measures the levels of future volatility, as anticipated by the market through option prices.

Note that, for a given couple, the levels of the estimated conditional Kendall's tau are different (in general) for different conditioning variables. Indeed, the unconditional Kendall's tau τ 1,2 , the average conditional Kendall's tau with respect to σ, which is IE σ τ 1,2|σ and the average conditional Kendall's tau with respect to ∆σ I , which is IE ∆σ I τ 1,2|∆σ I have no reason to be equal.

Both conditioning variables σ and ∆σ I are of dimension 1. For each method and each conditioning variable, we will use the "best" choice of ψ as determined from the simulations in Section 7.4.1, that is ψ (6) for the methods logit, probit, tree and random forests and ψ (1) for the methods Nearest neighbors and neural networks. On the following figures, the matching between colors and corresponding estimators still follows Table 7.2. Remark 7.6. It is well-known that sequences of asset returns are not i.i.d. In particular, their volatilities are time-dependent, as in GARCH-type or stochastic volatility models. Moreover, the tail behavior of their distributions is significantly varying, due to some periods of market stress. Several families of models (switching regime models, jumps, etc) have tried to capture such stylized facts. We conjecture that such temporal dependencies will not affect our results too much. Indeed, dependence will be mitigated by considering all possible couples of random vectors, independently of their dates. It is easy to go one step beyond, for instance by keeping only the couples of returns indexed by i and j when |i -j| > m, for some "reasonably chosen" threshold m (m = 20, e.g.). In every case, it is highly likely that our inference procedures are still consistent and asymptotically normal, for most types of dependence between successive observations (mixing processes, weak dependence, m-dependence, mixingales, etc), even if the asymptotic variances are different from ours.

Conditional dependence with respect to the Eurostoxx's volatility proxy σ

We will first consider the conditioning events given by σ, the proxy variable for the market intraday volatil- On most figures, the estimated conditional Kendall's tau seems to exhibit some kind of concavity.

The behavior of these functions can be roughly broken down into two main regimes:

1. The "moderate" volatility regime (also called the "normal regime") in the sense that the volatility stay mild, say in the lower half of its range. In this normal regime, conditional Kendall's tau is an increasing function of volatility. This is coherent with most empirical research where it is shown that dependence increases with volatility.

2. The high volatility regime: this is a "stressed regime" where σ lies in the upper half of its range. In this less frequent regime, the influence of the European volatility σ on the conditional Kendall's tau appears to be less clear: the estimators become more "fluctuating" and more different from each other, as a consequence of the small number of observations in most stressed regimes.

During the European debt crisis (see Figures 7.2,7.4 and 7.6), the three couples seem to exhibit the same shape of conditional dependence with respect to σ, even if their average levels are different. These similarities can be a little bit surprising considering that the economic situations of the corresponding countries are different. It can be conjectured that the heterogeneity in the "mean" levels of conditional dependence is sufficient to reflect this diversity of situations. In this perspective, the increasing pattern of conditional dependence w.r.t. the "volatility" would be a pure characteristic of that period, regardless of the chosen pair of European countries. Indeed, we have observed this pattern for most couples of European countries in the Eurozone. An explanation might be that investors were focusing on the same international news, for example, about the future of the Eurozone, and, therefore, they were reacting in a similar way, irrespective of the country.

For each couple of countries, conditional Kendall's tau is nearly always lower during the After-crisis period than during the European debt crisis. Apparently, in the After-crisis period, factors and events that are specific to each country attract more attention from investors than during the crisis, which results in lower dependence. In this context, the shapes of conditional dependence are no longer similar for different couples. In particular, the conditional Kendall's tau between German and French returns show a significant increase during the low volatility regime and a decrease during the high volatility regime: see Figure 7.3. The conditional dependence between the German and the Danish returns is also increasing during the low volatility regime, but in the high volatility, their conditional Kendall's tau seems to be rather constant, even increasing according to the nearest neighbors and the neural networks estimators.

Concerning Figure 7.7, we do not seem any clear tendency. It is likely that σ has almost no impact on the conditional dependence between the German and Greek stock index returns.

Conditional dependence with respect to the variations ∆σ I of the Eurostoxx's implied volatility index

The implied volatility is computed using option prices. In this sense, this financial quantity reflects investors' anticipation of future uncertainty. When important events happen, investors most often update their anticipations, which results in a change of implied volatilities. This change, denoted by ∆σ I may be Chapter For each couple, the levels of the conditional Kendall's tau are higher during the European debt crisis than during the after-crisis period. This is coherent with our conclusions in the previous subsection. But here, conditional Kendall's taus look like concave functions of ∆σ I during the crisis, while they exhibit "double bumps" features after the crisis. During the crisis, when ∆σ I is small in absolute value, implied volatilities do not change much and the dependence is in general higher than during big changes of the market implied volatility, i.e. when |∆σ I | is high (see Figures 7.10 and 7.12).

One exception is the couple (France, Germany), for which the conditional Kendall's tau is roughly a decreasing function of ∆σ I during the crisis. France and Germany are close countries and have strong economic relationships, but Germany is seen as a country in the "center of Europe" while France share a lot of similarities with countries of the periphery (in the South of Europe). Indeed, during the crisis, when implied volatility decreases (corresponding to a negative value of ∆σ I ), the dependence is higher, which can be interpreted as investors seeing the two countries as close. On the contrary, when the market implied volatility increases, there are concerns in the market about the robustness of Eurozone and investors raise doubts about southern European countries -including France -which tend to decrease the conditional Kendall's tau between French and German returns.

After the crisis, the couples (Germany, France), and (Germany, Denmark) revert to a more usual shape of conditional dependence: when volatility does not change much, conditional Kendall's tau is low ; when volatility changes much, conditional Kendall's tau is higher, reflecting more stressed situations.

In this period, an exception is the couple (Germany, Greece), whose conditional Kendall's tau has a particular shape, that looks like the one of the couple (Germany, France) during the crisis. This is coherent with the fact that, in stressed situations, when volatility increases, investors sometimes remembers that Greece still has a fragile economy, which results in a lower conditional Kendall's tau. But three estimators suggest that, when volatility increases very much, conditional Kendall's tau between Germany and Greece increases again, following the classical tendencies that we had already observed.

Conclusion

In a parametric setting, we have proposed a localized log-likelihood method to estimate conditional Kendall's tau. When the link function is analytically tractable and explicit, it is then possible to code and optimize the full penalized criterion. The consistency and the asymptotic normality of such estimators have been stated. In particular, this is the case for logit or probit-type link functions. We noticed that evaluating a Kendall's tau is equivalent to evaluating a probability of being classified as a concordant pair. Therefore, most classification procedures can be adapted to directly estimate conditional Kendall's tau. Classification trees, random forests, nearest neighbors and neural networks have been discussed.

They generally provide more flexible parametric models than previously.

We note that multiple trade-offs arise when choosing one of these methods, as displayed in Table 7.10. Depending on the requirements of the situation, statisticians can choose some algorithms that best match their needs. To summarize, trees and random forests methods are the fastest ones, but exhibit the lowest performances. Parametric methods such as the logit and probit may perform very well under some "simple" functional forms of g and ψ, but they deteriorate quickly when the true underlying lihood). See Cherubini and Luciano [START_REF] Cherubini | Copula methods in finance[END_REF] for details. Nonparametric methods have been developed too, since the seminal papers of Deheuvels [START_REF] Deheuvels | La fonction de dependence empirique et ses proprietes, un test non parametrique d'independance[END_REF][START_REF] Deheuvels | A kolmogorov-smirnov type test for independence and multivariate samples[END_REF] about empirical copula processes.

Second, conditional copulas have been formally introduced by Patton [START_REF] Patton | Modelling asymmetric exchange rate dependence[END_REF][START_REF] Patton | Estimation of multivariate models for time series of possibly different lengths[END_REF] [START_REF] Fermanian | Time-dependent copulas[END_REF] for extensions of the latter concepts.

Proof of Theorem 7.3

Simple calculations provide: if i = j and under (7.3),

IE[L n (β)] = IE K h (Z i -Z j ) β (W (i,j) , Z i ) = IE K h (Z i -Z j )IE[ β (W (i,j) , Z i )|Z i , Z j ] = IE K h (Z i -Z j ) p(Z i , Z j ) log 1 2 + 1 2 g(ψ(Z i ) T β) + (1 -p(Z i , Z j )) log 1 2 - 1 2 g(ψ(Z i ) T β) = IE [K h (Z i -Z j )φ(Z i , Z j , β)] = IE K(t)φ(Z i , Z i -ht, β)f Z (Z i -ht) dt , that tends to IE [φ(Z i , Z i , β)f Z (Z i )] = L ∞ (β) when n → ∞, if φ(z, •, β)f Z (•) ε (z)f z (z) dz < ∞,
for some ε > 0 (invoke the dominated convergence Theorem and the compact support of K).

Now, let us prove that, for any β, L n (β) tends towards L ∞ (β) in probability, when n → ∞. It is sufficient to prove that the variance of L n (β) tends to zero.

IE L n (β) -IE[L n (β)] 2 = 1 n 2 (n -1) 2 i1,j1;i1 =j1 i2,j2;i2 =j2 IE K h (Z i1 -Z j1 )K h (Z i2 -Z j2 ) β (W (i1,j1) , Z i1 ) β (W (i2,j2) , Z i2 ) -IE[L n (β)] 2 =: 1 n 2 (n -1) 2 i1,j1;i1 =j1 i2,j2;i2 =j2 v i1,j1,i2,j2 ,
with obvious notation. Obviously, v i1,j1,i2,j2 is zero when i 1 and j 1 are not equal to i 2 nor j 2 . At the opposite, in the case of equalities between some of these four indices, we get non-zero terms.

To be specific, when i 1 = i 2 = i, and j 1 = j 2 , we have

v i,j1,i,j2 = IE K h (Z i -Z j1 )K h (Z i -Z j2 ) β (W (i,j1) , Z i ) β (W (i,j2) , Z i ) -IE[L n (β)] 2 = IE K(x)K(y)A(Z i , Z i -hx, Z i -hy)f Z (Z i -hx)f Z (Z i -hy) dx dy -IE[L n (β)] 2 ,
by setting

A(x, y, z) := IE β (W (i,j1) , Z i ) β (W (i,j2) , Z i )|Z i = x, Z j1 = y, Z j2 = z
= p(x, y, z) log 2 q(x, β) + (p(x, y) -p(x, y, z)) log q(x, β) log(1 -q(x, β))

+ (p(x, z) -p(x, y, z)) log q(x, β) log(1 -q(x, β)) + (1 -p(x, y) -p(x, z) + p(x, y, z)) log 2 (1 -q(x, β)).
by setting

E(x, y) := IE 2 β (W (i,j) , Z i )|Z i = x, Z j = y = p(x, y) log 2 q(x, β) + (1 -p(x, y)) log 2 (1 -q(x, β)). If E(z, •) ε (z)f Z,ε (z)f Z (z) dz < ∞, then h p v i,j,i,j tends to a constant when n → ∞.
Finally, if i 1 = j 2 and j 1 = i 2 , we get

v i1,j1,i1,j1 = IE K h (Z i1 -Z j1 ) 2 β (W (i1,j1) , Z i1 ) β (W (j1,i1) , Z j1 ) -IE[L n (β)] 2 = h -p IE K(x) 2 F (Z i , Z i -hx)f Z (Z i -hx) dx -IE[L n (β)] 2 ,
by setting

F (x, y) := IE β (W (i1,j1) , Z i1 ) β (W (j1,i1) , Z j1 )|Z i1 = x, Z j1 = y
= p(x, y) log q(x, β) log q(y, β) + (1 -p(x, y)) log(1 -q(x, β)) log(1 -q(y, β)).

If F (z, •) ε (z)f Z,ε (z)f Z (z) dz < ∞, then h p v i1,j1
,j1,i1 tends to a constant when n → ∞.

Summarizing the previous terms, we have obtained V ar(L n (β)) = O(n -1 + n -2 h -p ), that tends to zero pointwise, when n 2 h p → ∞. We deduce

L n (β) -L ∞ (β) = L n (β) -IE[L n (β)] + IE[L n (β)] -L ∞ (β) = o P (1). Since L n (•)
and L ∞ (•) are concave, invoking the convexity lemma of Geyer [START_REF] Geyer | On the asymptotics of convex stochastic optimization[END_REF] (see Knight and Fu [START_REF] Knight | Asymptotics for lasso-type estimators[END_REF], alternatively), the maximizer β of L n tends in probability towards the maximizer of L ∞ .

We summarize the latter technical assumptions that are sufficient to obtain the consistency of β: for some ε > 0,

φ(z, •, β)f Z (•) ε (z)f z (z) dz < ∞, (7.13)
A(z, •, •) ε (z, z) + B(z, •, •) ε (z, z) + C(•, z, •) ε (z, z) + D(•, z, •) ε (z, z) f 2 Z,ε (z)f Z (z) dz < ∞, (7.14) φ(z, •, β) ε (z) + E(z, •) ε (z) + F (z, •) ε (z) f Z,ε (z)f Z (z) dz < ∞. (7.15)
7.9 Proof of Theorem 7.4

Set u := √ n(β -β * ) and û := √ n( β -β * ). Obviously, û = arg max u∈R p L n (β * + n -1/2 u) -λ n |β * + n -1/2 u| 1 , = arg max u∈R p nL n (β * + n -1/2 u) -nL n (β * ) -nλ n |β * + n -1/2 u| 1 -|β * | 1 .
Note that

nλ n |β * + n -1/2 u| 1 -n|β * | 1 = n 1/2 λ n k;β * k =0 |u k | + n 1/2 λ n k;β * k =0 sign(β * k )u k -→ µ k;β * k =0 |u k | + µ k;β * k =0 sign(β * k )u k ,
when n → ∞. Moreover, 3) , for some (random) β s.t. |β * -β| < |β * -β|. We will successively prove that (i) n 1/2 Ln (β * ) weakly tends to a Gaussian random vector W, W ∼ N (0 p , Σ β * );

nL n (β * + n -1/2 u) -nL n (β * ) = n 1/2 Ln (β * ).u + 1 2 u T Ln (β)u + 1 6 √ n ... Ln(β).u (
(ii) Ln (β * ) tends in probability towards a constant matrix H(β * );

(iii) ... Ln(β) is O P (1).

Then, û = arg max u L n (u), where L n (u) weakly tends to

L ∞ (u) := W.u + 1 2 u T H(β * )u -µ k;β * k =0 |u k | -µ k;β * k =0 sign(β * k )u k ,
that is concave. The result will follow, applying Theorem 1 in Kato [START_REF] Kato | Asymptotics for argmin processes: Convexity arguments[END_REF].

First, let us prove (i), i.e. the asymptotic normality of n 1/2 Ln (β) for any given parameter β. Consider the centered criterion

M n (β) := Ln (β) -IE[ Ln (β)] = 1 n(n -1) i,j;i =j ij (β),
where ij := K h (Z i -Z j )∂ β β (W (i,j) , Z i ) -IE[ Ln (β)].
We symmetrize the localized likelihood:

M n (β) = 1 2n(n -1) i,j;i =j M ij (β), where M ij (β) (or simply M ij ) is ij (β) + ji (β). Note that M ij = M ji and that IE[M ij ] = 0.
By the dominated convergence theorem and a change of variable, we easily check that

IE[ Ln (β)] = ∂ β L ∞ (β) + o(1) if, for some ε > 0, we have ∂ β φ(z, •, β)f Z (•) ε (z)f z (z) dz < ∞. Moreover, by simple calculations, we get, if i = j, IE[M n |Z i ] = 1 2n IE[M ij + M ji |Z i ] = 1 n IE[M ij |Z i ] = 1 n {K h (Z i -z)∂ β φ(Z i , z, β) + K h (z -Z i )∂ β φ(z, Z i , β)}f Z (z) dz - 2 n IE[ Ln (β)] = 1 n K(t){∂ β φ(Z i , Z i -ht, β)f Z (Z i -ht) + ∂ β φ(Z i + ht, Z i , β)f Z (Z i + ht)} dt - 2 n IE[ Ln (β)] = 2 n ∂ β φ(Z i , Z i , β)f Z (Z i ) - 2 n L∞ (β) + o(n -1 ) + r n,i ,
where, by a m-order limited expansion, we obtain

r n,i ≤ Cst.h m |K| nm! (f Z (•)∂ β φ(Z i , •, β)) (m) + (f Z (•)∂ β φ(•, Z i , β)) (m) ε (Z i ),
for any norm • on R p and a positive constant Cst. We deduce that n 1/2 n i=1 IE[M n |Z i ] is asymptotically normal by invoking the usual CLT, under condition (7.17) below. To be specific, the Hájek projection

of M n is √ n 2 n i=1 IE[M n |Z i ] = 1 √ n n i=1 {∂ β φ(Z i , Z i , β)f Z (Z i ) -L∞ (β)} + o P (1) N (0, Σ β ), with Σ β := ∂ β φ(z, z, β)∂ β φ(z, z, β) T f 3 Z (z) dz -L∞ (β) L∞ (β) T .
Note that that L∞ (β * ) = 0. Now consider the "remainder term"

∆ n := M n (β)- n i=1 IE[M n |Z i ]/2. Since IE[M n |Z i ] = n -1 IE[M ij |Z i ], we deduce ∆ n = M n (β) - 1 2 n i=1 IE[M n |Z i ] = 1 2n(n -1) i,j;i =j {M ij -IE[M ij |Z i ]}.
It is relatively easy to prove that ∆ n is negligible, i.e. ∆ n = o P (n -1/2 ). Indeed, let us prove that the variance of n 1/2 ∆ n tends to zero with n:

V ar(n 1/2 ∆ n ) = nIE[∆ n ∆ T n ] = 1 4n(n -1) 2 i1,j1;i1 =j1 i2,j2;i2 =j2 δ(i 1 , i 2 , j 1 , j 2 ), δ(i 1 , i 2 , j 1 , j 2 ) = IE {M i1j1 -IE[M i1j1 |Z i1 ]} × {M i2j2 -IE[M i2j2 |Z i2 ]} T .
If there is no identity among the indices (i 1 , j 1 , i 2 , j 2 ), with i 1 = j 1 and i 2 = j 2 , then δ(i 1 , i 2 , j 1 , j 2 ) is zero.

Moreover, this is still the case when there is only a single identity. For instance, assume i 1 = i 2 = i and

j 1 = j 2 . Then, δ(i, i, j 1 , j 2 ) = IE {M ij1 -IE[M ij1 |Z i ]} × {M ij2 -IE[M ij2 |Z i ]} T = IE {M ij1 -IE[M ij1 |Z i ]} × IE[{M ij2 -IE[M ij2 |Z i ]} T |Z i , Z j1 ] = IE [{M ij1 -IE[M ij1 |Z i ]} × 0] = 0.
The other terms for which a single identity between the indices can be managed similarly.

At the opposite, non-zero terms appear when i 1 = i 2 = i and j 1 = j 2 = j. In this case, we obtain

δ(i, i, j, j) = IE {M ij -IE[M ij |Z i ]}{M ij -IE[M ij |Z i ]} T = IE[M ij M T ij ] -IE[IE[M ij |Z i ]IE[M ij |Z i ] T ].
By a usual change of variable and by symmetry, we get

IE[M ij M T ij ] = 2IE { ij (β) ij (β) T + ij (β) ji (β) T } = 2IE K 2 h (Z i -Z j )∂ β β (W (i,j) , Z i )∂ β β (W (i,j) , Z i ) T + K h (Z i -Z j )K h (Z j -Z i )∂ β β (W (i,j) , Z i )∂ β β (W (j,i) , Z j ) T + O(1) = 2h -p IE K 2 (x)H 1 (Z i , Z i -hx) + K(x)K(-x)H 2 (Z i , Z i -hx) f Z (Z i -hx) dx + O(1) = 2h -p K 2 (x)H 1 (z, z -hx) + K(x)K(-x)H 2 (z, z -hx) f Z (z -hx)f Z (z) dx dz + O(1),
by setting

H 1 (x, y) = IE ∂ β β (W (i,j) , Z i )∂ β β (W (i,j) , Z i ) T |Z i = x, Z j = y = p(x, y) ψ(x)ψ(x) T g (ψ(x) T β) 2 (1 + g(ψ(x) T β)) 2 + (1 -p(x, y)) ψ(x)ψ(x) T g (ψ(x) T β) 2 (1 -g(ψ(x) T β)) 2
, and

H 2 (x, y) = IE ∂ β β (W (i,j) , Z i )∂ β β (W (j,i) , Z j ) T |Z i = x, Z j = y = p(x, y) ψ(x)ψ(y) T g (ψ(x) T β)g (ψ(y) T β) (1 + g(ψ(x) T β))(1 + g(ψ(y) T β)) + (1 -p(x, y)) ψ(x)ψ(y) T g (ψ(x) T β)g (ψ(y) T β) (1 -g(ψ(x) T β))(1 + g(ψ(y) T β)) • Therefore, IE[M ij M T ij ] is O(h -p ), if ( H 1 + H 2 ) ε (z, •)f Z (•) ε (z)f Z (z) dz < ∞.
The last possible case providing non-zero δ(i 1 , i 2 , j 1 , j 2 ) is i 1 = j 2 = i and j 1 = i 2 = j. Then, we obtain

δ(i, j, j, i) = IE {M ij -IE[M ij |Z i ]} × {M ji -IE[M ji |Z j ]} T = δ(i, i, j, j),
due to the symmetry of M ij . Thus, we have proved that V ar(n

1/2 ∆ n ) = O(n -1 h -p )) = o(1), which implies n 1/2 M n (β) = n 1/2 2 n i=1 IE[M n |Z i ] + o P (1).
We deduce that n 1/2 M n (β) weakly tends towards the Gaussian random vector N (0 p , Σ β ) for any β.

When β = β * , ∂ β L ∞ (β * ) = 0, and this yields (i).

Second, let us deal with (ii) above. It is easy to prove that Ln (β * ) tends to H(β * ) in probability, when n tends to the infinity. Indeed, the arguments are exactly the same as in 7.8, where we have proved that

L n (β * )
IE[| ∂ 3 ∂β a ∂β b ∂β c L n (β)|] ≤ Cst × IE |K| h (Z i -Z j ) p(Z i , Z j )H(1, Z i , β, a, b, c, ) + (1 -p(Z i , Z j ))H(-1, Z i , β, a, b, c) ,
where, for every δ ∈ {1, -1},

H(δ, Z i , β, a, b, c) = |g (ψ(Z i ) T β)| 3 |1 + δg(ψ(Z i ) T β)| 3 + |g g (ψ(Z i ) T β)| |1 + δg(ψ(Z i ) T β)| 2 + |g (ψ(Z i ) T β)| |1 + δg(ψ(Z i ) T β)| |ψ(Z i ) a ψ(Z i ) b ψ(Z i ) c |.
and Cst denotes a real constant that depend on g and (a, b, c) only. Therefore, it is sufficient to assume that

|K|(t) p( z -ht)H(1, z, β, a, b, c) + (1 -p(z, z -ht))H(-1, z, β, a, b, c) f Z (z)f Z (z -ht) dt dz < ∞.
This is guaranteed by Assumption (7.19) below. Then, under the latter assumption, (iii) is stated and this finishes the proof.

For convenience, let us gather the main technical assumptions that have been requested to prove Theorem 7.4: for some ε > 0,

∂ β φ(z, •, β)f Z (•) ε (z)f z (z) dz < ∞. (7.16
)

IE (f Z (•)∂ β φ(Z i , •, β)) (m) + (f Z (•)∂ β φ(•, Z i , β)) (m) ε (Z i ) < ∞. (7.17)
( H 1 + H 2 ) ε (z, •)f Z (•) ε (z)f Z (z) dz < ∞. (7.18)
For every indices (a, b, c) ∈ {1, . . . , p } and for V(β * ), some (small) neighborhood around β * ,

sup β∈V (β * ) (p(z, •)f Z (•)) ε (z)H(1, z, β, a, b, c) + ((1 -p)(z, •)f Z (•)) ε (z)H(-1, z, β, a, b, c, ) f Z (z) dz < ∞. ( 7
ψ(z) 3 |1 + δg(ψ(z) T β)| -3 f Z,ε (z)f Z (z) dz < ∞.
for a fixed k > 0, a function g : X k → R and X 1 , . . . X k i.i.d.

∼ IP X . Following Hoeffding [START_REF] Hoeffding | A class of statistics with asymptotically normal distribution[END_REF], a natural estimator of θ(IP X ) is the U-statistic θ(IP X ), defined by

θ(IP X ) := σ∈I k,n g X σ(1) , . . . , X σ(k) ,
where I k,n is the set of injective functions from {1, . . . , k} to {1, . . . , n}. For an introduction to the theory of U-statistics, we refer to Koroljuk and Borovskich [START_REF] Korolyuk | Theory of U -statistics[END_REF] and Serfling [START_REF] Serfling | Approximation theorems of mathematical statistics[END_REF]Chapter 5].

In our framework, we assume that we do not only observe X, but we observe in fact (X, Z) where Z is a p-dimensional covariate. We are now interested in functionals of the conditional law IP X|Z . For each z 1 , . . . , z k ∈ Z, where Z is a compact subset of R p , we can define such a functional θ z1,...z k by

θ z1,...z k (IP X|Z=• ) := θ(IP X|Z=z1 , . . . , IP X|Z=z k ) = IE k i=1 IP X|Z=z i g(X 1 , . . . , X k ) = IE g(X 1 , . . . , X k ) Z i = z i , ∀i = 1, . . . , k = g(x 1 , . . . , x k )dIP X|Z=z1 (x 1 ) • • • dIP X|Z=z k (x k ).
This can be seen as a generalization of θ(IP X ) to the conditional case. Indeed, when X and Z are independent, the new functional θ z1,...z k (IP X|Z=• ) is equal to the unconditional functional θ(IP X ). For convenience, we will use the notation θ(z 1 , . . . , z k ) := θ z1,...z k (IP X|Z=• ), treating the law of (X, Z) as fixed (but unknown). Stute [START_REF] Stute | Conditional U-statistics[END_REF] defined a kernel-based estimator θ(z 1 , . . . , z k ) of the conditional functional

θ(z 1 , . . . , z k ) by θ(z 1 , . . . , z k ) := σ∈I k,n K h Z σ(1) -z 1 • • • K h Z σ(k) -z k g X σ(1) , . . . , X σ(k) σ∈I k,n K h Z σ(1) -z 1 • • • K h Z σ(k) -z k , (8.1)
where h > 0 is the bandwidth, K(•) a kernel on R p , K h (•) := h -p K(•/h), and (X i , Z i )

i.i.d.
∼ IP X,Z .

Stute [START_REF] Stute | Conditional U-statistics[END_REF] proved the asymptotic normality of θ(z 1 , . . . , z k ) and its weak and strong consistency. Dony

and Mason [START_REF] Dony | Uniform in bandwidth consistency of conditional U-statistics[END_REF] derived its uniform in bandwidth consistency under VC-type conditions over a class of possible functions g.

Nevertheless, the estimator (8.1) has several weaknesses. First, interpretation of the whole hypersurface (z 1 , . . . , z k ) → θ(z 1 , . . . , z k ) can be difficult. Indeed, the latter is a curve of dimension 1 + p × k, which is rather challenging to visualize even for small values of p and k. Second, for each new tuple (z 1 , . . . , z k ), computation of θ(z 1 , . . . , z k ) has a cost of O(n k ). This means that if we want to estimate θ(z

(i) 1 , . . . , z (i)
k ) for every i = 1, . . . , N , where z

1 , . . . , z

k , . . . , z

(N ) 1 , . . . , z (N ) k ∈ Z k×N , then the total cost is O(N n k ).
Third, it is well-known that kernel estimators are not very smooth, in the sense that they usually present many spurious local minima and maxima, that can be a problem in applications.

Therefore, we may want to have estimators which are more monotone with respect to the conditioning variables z 1 , . . . z k , and have a simple functional form.

Another idea is to decompose the function (z 1 , . . . , z k ) → θ(z 1 , . . . , z k ) in a basis (ψ i ) i≥0 , generalizing the work of Derumigny and Fermanian [START_REF] Derumigny | About Kendall's regression[END_REF]. This may not be always easy if the range of the function

θ(•, • • • , •) is a strict subset of R.
In that case, it is always possible to use a "link function" Λ, strictly increasing and continuously differentiable such that the range Λ • θ(•, • • • , •) is exactly R. Whatever the choice of Λ (including the identity function), we can decompose the latter function in any basis (ψ i ) i≥0 .

If only a finite number r > 0 of elements of this basis are necessary to represent the whole function

Λ • θ(•, • • • , •) over Z k
, then we have the following parametric model

∀(z 1 , . . . , z k ) ∈ Z k , Λ θ(z 1 , . . . , z k ) = ψ(z 1 , . . . , z k ) T β * , (8.2)
Chapter 8. Estimation of a regular conditional functional by conditional U-statistic regression 223 where β * ∈ R r is the true parameter and ψ(•) := ψ 1 (•), . . . , ψ r (•) T ∈ R r . In most applications, finding an appropriate basis ψ is not easy ; this will depend of the choice of the (conditional) functional θ.

Therefore, the most simple solution consists in choosing a concatenation of several well-known basis such as polynomials, exponentials, sinuses and cosinuses, indicator functions, etc... They allow to take into account potential non-linearities and even discontinuities of the function

Λ • θ(•, • • • , •).
ψ(Z 1 , . . . , Z n ) T t = 0 = 1 =⇒ t = 0.
With such a choice of a wide and flexible class of functions, it is likely that not all these functions are relevant. This is what is know as sparsity, i.e. the number of non-zero coefficients of β * , denoted by |S| = |β * | 0 ≤ s, for some s ∈ {1, . . . , r}, where | • | 0 is the number of non-zero components of a vector of R r and S is the set of non-zero components of β * . Note that in this framework, r can be moderately large, for example 30 or 50 while the original dimension p is small, for example p = 1 or 2. This corresponds to the decomposition of a function, defined on a small-dimension domain, in a mildly large basis. Contrary to more usual models, the explained variable Λ θ(z 1 , . . . , z k ) , is not observed in Model (8.2).

Therefore, a direct estimation of the parameter β * (for example, by the ordinary least squares, or by the Lasso) is unfeasible. In other words, even if the function (z 1 , . . . , z k ) → Λ θ(z 1 , . . . , z k ) is deterministic (by definition of conditional probabilities), finding the best β in Model (8.2) is far from being a numerical analysis problem since the function to be decomposed is unknown. Nevertheless, we will replace Λ θ(z 1 , . . . , z k ) by the nonparametric estimate Λ θ(z 1 , . . . , z k ) , and use it as an approximation of the explained variable.

More precisely, we fix a finite collection of points z 1 , . . . , z n ∈ Z n and a collection I k,n of injective functions σ : {1, . . . , k} → {1, . . . , n }. Note that we are not forced to include all the injective functions in I k,n , reducing its number of elements. This will allow us to decrease the computational cost of the procedure. For every σ ∈ I k,n , we estimate θ(z σ(1) , . . . , z σ(k) ). Finally, the estimator β is defined as the minimizer of the following l 1 -penalized criteria

β := arg min β∈R r   (n -k)! n ! σ∈I k,n Λ θ z σ(1) , . . . , z σ(k) -ψ z σ(1) , . . . , z σ(k) T β 2 + λ|β| 1   , (8.3)
where λ is a positive tuning parameter (that may depend on n and n ), and | • | q denotes the l q norm, for 1 ≤ q ≤ ∞. This procedure is summed up in the following Algorithm 15. Note that even if we study the general case with any λ ≥ 0, corresponding properties of the unpenalized estimator can be derived by choosing the particular case λ = 0. Assumption 8.2.6. For every x ∈ X , z → f X,Z (x, z) is differentiable almost everywhere up to the order α. Moreover, there exists a finite constant C g,f,α > 0, such that, for every positive integers m 1 , . . . , m k such that k i=1 m i = α, for every 0 ≤ j 1 , . . . , j mi ≤ p,

k i=1 g x 1 , . . . , x k -IE g(X 1 , . . . , X k ) Z i = z i , ∀i = 1, . . . , k • ∂ mi f X,Z ∂z j1 • • • ∂z jm i x i , z i + u i - ∂ mi f X,Z ∂z j1 • • • ∂z jm i x i , z i dµ(x 1 ) • • • dµ(x k ) ≤ C g,f,α k i=1 u i ∞ ,
for every choices of x 1 , . . . , x k ∈ X and z 1 , . . . , z k ∈ Z, u 1 , . . . , u k ∈ R p such that z i + u i ∈ Z. There exists a constant C K,α such that

m1+ ••• +m k =α n m 1:k k i=1 K(u i ) p j1,...,jm i =1 u i,j1 . . . u i,jm i k i=1 u i ∞ du 1 • • • du k ≤ C K,α .
An easy situation is the case when g is bounded, i.e. when the following assumption hold.

Assumption 8.2.7.

There exists a constant C g such that ||g|| ∞ ≤ C g < +∞.

When g is not bounded, a weaker result can still be proved under a "conditional Bernstein" assumption. This assumption will help us to control the tail behavior of g so that exponential concentration bounds are available.

Assumption 8.2.8 (conditional Bernstein assumption).

There exists a positive function B g such that, for all l ≥ 1 and z 1 , . . . ,

z k ∈ R kp , IE g(X 1 , . . . , X k ) l Z 1 = z 1 , . . . , Z k = z k ≤ B g (z 1 , . . . , z k ) l l!,
such that B g (Z 1 , . . . , Z k ) ≤ Bg almost surely, for some finite positive constant Bg .

As a shortcut notation, we will define also B g,z := B g (z 1 , . . . , z k ).

IP θ(z 1 , . . . , z k ) -θ(z 1 , . . . , z k ) < 1 + C 3 h α + C 4 t × C 5 h k+α + t ≥ 1 -2 exp - [n/k]t 2 h kp C 1 + C 2 t -2 exp - [n/k]t 2 h kp C 6 + C 7 t ,
where

C 3 := 4f k Z,max f -2k Z,min C K,α /α!, C 4 := 4f k Z,max f -2k Z,min and C 5 := C g,f,α C K,α f -k Z,min /α!.
If Assumption 8.2.7 is satisfied, the result holds with the following values:

C 6 := 2C 2 g f k Z,max f -2k Z,min ||K|| 2k 2 , C 7 := (8/3)C k K C k g f -k Z,
:= 128 B g,z + Bg 2 C 2k-1 K f -2k Z,min , C7 := 2 B g,z + Bg C k K f -k Z,min .

Asymptotic results for θ

The estimator θ(z 1 , . . . , z k ) has been first studied by [START_REF] Stute | Conditional U-statistics[END_REF] [START_REF] Stute | Conditional U-statistics[END_REF]. He proved the consistency and the asymptotic normality of θ(z 1 , . . . , z k ). We recall his results as they will be used in Section 8.3.

Theoretical properties of the estimator β

Let us define the matrix Z of dimension |I k,n |×r by [Z ] i,j := ψ j z σi(1) , . . . , z σi(k) , where

1 ≤ i ≤ |I k,n |, 1 ≤ j ≤ r

Non-asymptotic bounds on θ

We will also use the Restricted Eigenvalue (RE) condition, introduced by Bickel, Ritov and Tsybakov [START_REF] Bickel | Simultaneous analysis of Lasso and Dantzig selector[END_REF].

For c 0 > 0 and s ∈ {1, . . . , p}, it is defined as follows, RE(s, c 0 ) condition : The design matrix Z satisfies κ(s, c 0 ) := min J 0 ⊂ {1, . . . , r}

|J 0 | ≤ s min δ = 0 |δ J C 0 | 1 ≤ c 0 |δ J0 | 1 ||Z δ|| |δ| 2 > 0.
Note that this condition is very mild, and is satisfied with a high probability for a large class of random matrices: see Bellec et al. [START_REF] Bellec | Slope meets lasso: improved oracle bounds and optimality[END_REF]Section 8.1] for references and a discussion. We will also need the following regularity assumption on the function Λ(•). and | β -β * | q ≤ 4 2/q (γ + 1)ts 1/q κ 2 (s, 3) , for every

1 ≤ q ≤ 2 ≥ 1 -2 σ∈I k,n exp - [n/k]f 2 Z,min h kp 16C 1 + 4C 2 f Z,min + exp - [n/k]t
F ∞,n (u) := 2(n -k)! n ! σ∈I k,n r j=1 W σ ψ j z σ(1) , . . . , z σ(k) u j + (n -k)! n ! σ∈I k,n ψ z σ(1) , . . . , z σ(k) T u 2 + r i=1 |u i |1 {β * i =0} + u i sign(β * i )1 {β * i =0} , with W = (W σ ) σ∈I k,n ∼ N 0, H where [ H] σ,ς := k j,l=1 1 z σ(j) =z ς(l) ||K|| 2 2 f Z z σ(j) Λ θ z σ(1) , . . . , z σ(k) Λ θ z ς(1) , . . . , z ς(k)
• θj,l z σ(1) , . . . , z σ(k) , z ς(1) , . . . , z ς(k) -θ z σ(1) , . . . , z σ(k) θ z ς(1) , . . . , z ς(k) ,

and θj,l is as defined in Equation (8.6).

Moreover, lim sup n→∞ IP (S n = S) = c < 1, where S n := {j : βj = 0} and S := {j : β j = 0}.

A usual way of obtaining the oracle property is to modify our estimator in an "adaptive" way. Following

Zou [START_REF] Zou | The adaptive lasso and its oracle properties[END_REF], consider a preliminary "rough" estimator of β * , denoted by βn , or more simply β. Moreover

F∞,n (u S ) := 2(n -k)! n ! σ∈I k,n j∈S W σ ψ j (z i )u j + (n -k)! n ! σ∈I k,n j∈S ψ j (z i )u j 2 + i∈S u i |β * i | δ sign(β * i ), with W = (W σ ) σ∈I k,n ∼ N 0, H .
Moreover, when = 0, the oracle property is fulfilled: IP (S n = S) → 1 as n → ∞.

Asymptotic properties of β jointly in (n, n )

Now, we consider the framework in which both n and n are going to infinity, while the dimensions p and r stay fixed. We now provide a consistency result for βn,n .

Theorem 8. [START_REF] Bellec | Bounds on the prediction error of penalized least squares estimators with convex penalty[END_REF]

-→ arg min β∈R r G ∞,∞ (β), as (n, n ) → ∞, where G ∞,∞ (β) := (β * -β)M ψ,z (β * -β) T + λ 0 |β| 1 .
Moreover, if λ 0 = 0 and M ψ,z is invertible, then βn,n is consistent and tends to the true value β * .

Note that, since the sequence (z i ) is deterministic, we only assume the convergence of the sequence of deterministic matrices σ∈I k,n ψ z σ(1) , . . . , z σ(k) ψ z σ(1) , . . . , z σ(k) T /n in R r 2 . Moreover, if the "second subset" (z i ) i=1,...,n were a random sample (drawn along the law IP Z ), the latter convergence would be understood "in probability". And if IP Z satisfies the identifiability condition (Proposition 8.1), then M ψ,z would be invertible and βn,n → β * in probability. Now, we want to go one step further and derive the asymptotic law of the estimator βn,n . For this, we will need the following assumption. and every (i, j) ∈ {1, . . . , n } 2 , i = j, we have

|z i -z j | ∞ > 2h n,n . (ii) (a) n (nh p+4α n,n + h 2α n,n + h p n,n + (nh p n,n ) -1 ) → 0, (b) λ n,n (n n h p n,n ) 1/2 → 0, (c) n n h p n,n → ∞ and n h p+2α- n,n / ln n → ∞ for some ∈ [0, 2α[. (iii) The distribution IP z ,n := |I k,n | -1
σ∈I k,n δ (z σ(1) ,...,z σ(k) ) weakly converges as n → ∞, to a distribution IP z ,k,∞ on R kp . There exists a distribution IP z ,∞ on R kp , with a density f z ,∞ with respect to the p-dimensional Lebesgue measure such that IP z ,k,∞ = IP ⊗k z ,∞ .

(iv (vi) some integrals exists and are finite. We will especially need the folowing ones.

) The matrix V 1 := ψ(z 1 , . . . z k )ψ(z 1 , . . . z k ) T f z ,∞ (z 1 ) • • • f z ,∞ (z k )dz 1 • • • dz k is non-singular.
Ṽ1 := θ z 1 , . . . , z k Λ θ z 1 , . . . , z k ψ z 1 , . . . , z k f z ,∞ (z 1 ) • • • f z ,∞ (z k ) dz 1 • • • dz k ; V 2 := ||K|| 2 2 f Z (z 1 ) g x 1 ,
(n × n × h p n,n ) 1/2 ( βn,n -β * ) D -→ N (0, Ṽas ),
where

Ṽas := V -1 1 V 2 V -1 1 , V 1
is the matrix defined in Assumption 8.3.2(iv), and V 2 in Assumption 8. 3.2(v).

This theorem is proved in Section 8.7.

Applications and examples

Following Example 4.4 in Stute [START_REF] Stute | Conditional U-statistics[END_REF], we consider the function g(x 1 , x 2 ) := 1{x 1 ≤ x 2 }, with k = 2. In

this case θ(z 1 , z 2 ) = IP(X 1 ≤ X 2 |Z 1 = z 1 , Z 2 = z 2 )
. The parameter θ(z 1 , z 2 ) quantifies the probability that the quantity of interest X be smaller if we knew that Z = z 1 than if we knew that Z = z 2 .

To illustrate our methods, we choose a simple example, with the Epanechnikov kernel, defined by

K(x) := (3/4)(1 -u 2 )1|u| ≤ 1.
It is a kernel of order α = 2, with K 2 = 3/5. Assumption 8.2.1 is then satisfied with

C K := 3/4. Fix p = 1, Z = [-1, 1], X = R, f Z (z) = φ(z)1{|z| ≤ 1}/(1 -2Φ(-1))
, where Φ and φ are respectively the cdf and the density of the standard Gaussian distribution and X|Z = z ∼ N (z, 1), for every z ∈ Z. For any z 1 , z 2 ∈ Z, Therefore, the estimator θ(z 1 , z 2 ) exists with probability greater than 1 -2 exp -(n -1)h 2 0.35 -0.1h 2 2 0.52 + 1.5 × 0.35 -0.1h 2 , Note that this probability is greater than 0.99 as soon as n ≥ 3 0.52 + 1.5 × (0.35 -0.1h 2 ) / h 2 (0.35 -0.1h 2 ) 2 . For example, with h = 0.2, it means that the estimator θ(z 1 , z 2 ) exists with a probability greater than 99% as soon as n is greater than 651.

K u 1 • • • K u k m1+ ••• +m k =α α m 1:k • k i=1 p j1,...,jm i =1 u i,j1 . . . u i,jm i sup t∈[0,1] ∂ mi f Z ∂z j1 • • • ∂z jm i z i + tu i du 1 • • • du k = K(u 1 )K(u 2 ) m1+m2=2 2 m 1 !m 2 ! • k i=1 u mi i sup t∈[0,1] ∂ mi f Z ∂z mi z i + tu i du 1 du 2 ≤ K(u 1 )K(u 2 ) • u 2 2 sup z∈Z |f Z (z)| sup z∈Z ∂ 2 f Z ∂z 2 (z) + 1 2 u 1 u 2 sup z∈Z ∂f Z ∂z (z) sup z∈Z ∂f Z ∂z (z) + u 2 1 sup z∈Z ∂ 2 f Z ∂z 2 (z) sup t∈[0,1] |f Z (z)| du 1 du 2 ≤ 2 × 0.59 2 × K(u)u 2 du + 1 2 × 0.
:= (4/3)C k K = (4/3) × (3/4) 2 = 3/4. Therefore, for any n ≥ 0, h, t > 0, z 1 , z 2 ∈ Z, we have IP N 2 (z 1 , z 2 ) -f Z (z 1 )f Z (z 2 ) ≤ 0.1h α + t ≥ 1 -2 exp - [n/2]t 2 0.26h 2 + 0.
We list below other possible examples of applications. Conditional moments constitute also a natural class of U-statistics. They include the conditional variance (p

X = 1, k = 2, g(X 1 , X 2 ) = X 2 1 -X 1 • X 2 ) and the conditional covariance (p X = 2, k = 2, g(X 1 , X 2 ) := X 1,1 • X 2,1 -X 1,1 • X 2,2
). The conditional variance gives information about the volatility of X given the variable Z. Conditional covariances can be used to describe how the dependence moves as a function of the conditioning variables Z. Higher-order conditional moments (skewness, kurtosis, and so on) can also be estimated by higher-order conditional U-statistics, and they described respectively how the asymmetry and the behavior of the tails of X change as function of Z.

Gini's mean difference, an indicator of dispersion, can also be used in this framework. Formally, it is defined as the U-statistic with p X = 1, k = 2 and g(X 1 , X 2 ) := |X 1 -X 2 |. Its conditional version describes how two variables are far away, on average, given their conditioning variables Z. for example, X could be the income of an individual, Z could be the position of its home, and θ(z 1 , z 2 ) represent the average inequality between the income of two persons, one at point z 1 and the other at point z 2 .

Other conditional dependence measures can also be written as conditional U-statistics, see e.g. Example 1.1.7 of Koroljuk and Borovskisch [START_REF] Korolyuk | Theory of U -statistics[END_REF]. They show how a U-statistic of order k = 5 can be used to estimated the dependence parameter

θ = F 1,2 (x, y) -F 1,2 (x, ∞)F 1,2 (∞, y) dF 1,2 (x, y).
In our framework, we could consider a conditional version, given by

θ(z 1 , z 2 ) = F 1,2|Z=z (x, y) -F 1,2|Z=z (x, ∞)F 1,2|Z=z (∞, y) dF 1,2|Z=z (x, y),
where X is of dimension p X = 2. Finally, the conditional Kendall's tau is also included in our framework,

with p X = 2, k = 2, g(X 1 , X 2 ) := 4 • 1 {X1<X2} -1.

Notations

In the proofs, we will use the following shortcut notations: x 1:k denotes the k-tuple (x 1 , . . . , x k ). Similarly, for a function σ, σ(1 : k) denotes the tuple (σ(1), . . . , σ(k)), and X σ(1:k) is the k-tuple (X σ(1) , . . . , X σ(k) ).

For any variable Y and any collection of given points (z 1 , . . . z k ), the conditional expectation

IE[Y |Z 1:k = z 1:k ] denotes IE[Y |Z 1 = z 1 , . . . , Z k = z k ]. We denote by φ(z 1:k )dz 1:k the integral φ(z 1 , . . . , z k )dz 1 • • • dz k
for any integrable function φ : R k×p → R, and by g(

x 1:k )dµ ⊗k (x 1:k ) the integral g(z 1 , . . . , z k )dµ(x 1 ) • • • dµ(x k )
for any µ-integrable function g : X k → R.

Finite distance proofs for θ and β

For convenience, we recall Berk's (1970) inequality (see Theorem A in Serfling [123, p.201]). Note that, if m = 1, this reduces to Bernstein's inequality.

any t > 0, IP    n k -1 σ∈I ↑ k,n g X σ(1:k) -θ ≥ t    ≤ exp - [n/k]t 2 2σ 2 + (2/3)(b -θ)t ,
where I k,n is the set of bijective functions from {1, . . . , k} to {1, . . . , n} and I ↑ k,n is the subset of I k,n made of increasing functions.

Note that g does not need to be symmetric for this bound to hold. Indeed, if g is not symmetric, we can nonetheless apply this lemma to the symmetrized version g defined as g(x 1:k ) := (k!) -1 σ∈I k,k g(x σ(1:k) ), and we get the result.

Proof of Lemma 8.3

We decompose the quantity to bound into a stochastic part and a bias as follows:

N k (z 1:k ) - k i=1 f Z (z i ) = N k (z 1:k ) -IE[N k (z 1:k )] + IE[N k (z 1:k )] - k i=1 f Z (z i ) .
IE N k (z 1:k ) - k i=1 f Z (z i ) = IE n k -1 σ∈I k,n k i=1 K h Z σ(i) -z i - k i=1 f Z (z i ) = k i=1 f Z (z i + hu i ) - k i=1 f Z (z i ) k i=1 K(u i )du i = φ z,u (1) -φ z,u (0) k i=1 K(u i )du i , where φ z,u (t) := k j=1 f Z z i + thu j for t ∈ [-1, 1]
. Note that this function has at least the same regularity as f Z , so it is α-differentiable, and by a Taylor-Lagrange expansion, we get

IE[N k (z 1:k )] - k i=1 f Z (z i ) = R kp α-1 i=1 1 i! φ (i) z,u (0) + 1 α! φ (α) z,u (t z,u ) k i=1 K(u i )du i .
For l > 0, we have

φ (l) z,u (0) = m1+ ••• +m k =l α m 1:k k i=1 ∂ mi f Z z i + htu i ∂t mi (0) = m1+ ••• +m k =l α m 1:k k i=1 p j1,...,jm i =1 h mi u i,j1 . . . u i,jm i ∂ mi f Z ∂z j1 • • • ∂z jm i z i + t z,u hu i , where α m 1:k := α!/ k i=1 (m i !) is the multinomial coefficient. Using Assumption 8.2.1, for every i = 1, . . . , α -1, we get K(u 1 ) • • • K(u k )φ (i) z,u (0)du 1 • • • du k = 0.
Therefore, only the last term remains and we have

IE[N k (z 1:k )] - k i=1 f Z (z i ) = 1 α! φ (α) z,u (t z,u ) k i=1 K(u i )du i ≤ C K,α α! h α , using Assumption 8.2.2.
Second, we bound the stochastic part. We have

N k (z 1:k ) -IE[N k (z 1:k )] = k!(n -k)! n! σ∈I ↑ k,n k i=1 K h Z σ(i) -z i - k i=1 IE[K h Z i -z i ].
Then, we can apply Lemma 8.15 to the function g defined by g(z 1 , . . . , zk ) :

= k i=1 K h zi -z i . Here, we have b = -a = h -kp C k K , and
V ar g(Z 1 , . . . , Z k ) 2 ≤ IE g(Z 1 , . . . , Z k ) 2 = k i=1 IE K h Z i -z i 2 ≤ h -kp f k Z,max ||K|| 2k 2 .
Finally, we get

IP n k -1 N k (z 1:k ) -IE[N k (z 1:k )] ≥ t ≤ exp - [n/k]t 2 2h -kp f k Z,max ||K|| 2k 2 + (4/3)h -kp C k K t ,

Proof of Proposition 8.5

We have the following decomposition

| θ(z 1:k ) -θ(z 1:k )| = N k (z 1:k ) -1 (n -k)! n! σ∈I k,n k i=1 K h Z σ(i) -z i g(X σ(1:k) ) -IE g(X 1:k ) Z 1:k = z 1:k = k i=1 f Z (z i ) N k (z 1 , . . . , z k ) • (n -k)! n! σ∈I k,n k i=1 K h Z σ(i) -z i f Z (z i ) g(X σ(1:k) ) -IE g(X 1:k ) Z 1:k = z 1:k =: k i=1 f Z (z i ) N k (z 1 , . . . , z k ) • σ∈I k,n S σ .
The conclusion will follow from the next three lemmas, where we will bound separately

k i=1 f Z /N k ,
t > 0, C K,α h α /α! + t < f k Z,min /2, we have IP 1 N k (z 1:k ) - 1 k i=1 f Z (z i ) ≤ 4 f 2k Z,min C K,α h α α! + t ≥ 1 -2 exp - [n/k]t 2 2h -kp f k Z,max ||K|| 2k 2 + (4/3)h -kp C k K t ,
and on the same event, N k (z 1:k ) is strictly positive and

k i=1 f Z (z i ) N k (z 1:k ) ≤ 1 + 4f k Z,max f 2k Z,min C K,α h α α! + t .
Proof : Using the mean value inequality for the function x → 1/x, we get

1 N k (z 1:k ) - 1 k i=1 f Z (z i ) ≤ 1 N 2 * N k (z 1:k ) - k i=1 f Z (z i ) ,
where N * lies between N k (z 1:k ) and k i=1 f Z (z i ). By Lemma 8.3, we get

IP N k (z 1:k ) - k i=1 f Z (z i ) ≤ C K,α α! h α + t ≥ 1 -2 exp - [n/k]t 2 2h -kp f k Z,max ||K|| 2k 2 + (4/3)h -kp C k K t . On this event, N k (z 1:k ) - k i=1 f Z (z i ) ≤ (1/2) k i=1 f Z (z i ) by assumption, so that f k Z,min /2 ≤ N k (z 1:k ). We have also f k Z,min /2 ≤ k i=1 f Z (z i ). Thus, we have f k Z,min /2 ≤ N * .
Combining the previous inequalities, we finally get

1 N k (z 1:k ) - 1 k i=1 f Z (z i ) ≤ 1 N 2 * N k (z 1:k ) - k i=1 f Z (z i ) ≤ 4 f 2k Z,min C K,α h α α! + t .
[S σ ] ≤ C g,f,α C K,α h kα /(f k Z,min α!).
Proof : We remark that

0 = g(x 1:k ) -IE g(X 1:k ) Z 1:k = z 1:k f X|Z=z1 (x 1 ) • • • f X|Z=z k (x k )dµ ⊗k (x 1:k ) = g(x 1:k ) -IE g(X 1:k ) Z 1:k = z 1:k f X,Z (x 1 , z 1 ) • • • f X,Z (x k , z k ) k i=1 f Z (z i ) dµ ⊗k (x 1:k ). (8.10)
We have

IE[S σ ] = IE K h (Z σ(1) -z 1 ) • • • K h (Z σ(k) -z k ) k i=1 f Z (z i ) g X σ(1) , . . . , X σ(k) -IE g(X 1:k ) Z 1:k = z 1:k = g(x 1:k ) -IE g(X 1:k ) Z 1:k = z 1:k k i=1 K(u i ) f Z (z i ) f X,Z (x i , z i + hu i ) dµ(x i )du i = g(x 1:k ) -IE g(X 1:k ) Z 1:k = z 1:k k i=1 f X,Z x i , z i + hu i - k i=1 f X,Z x i , z i k i=1 K(u i ) f Z (z i ) dµ(x i )du i .
We apply now the Taylor-Lagrange formula to the function

φ x 1:k ,u 1:k (t) := k i=1 f X,Z x i , z i + hu i ,
and get

IE[S σ ] = g(x 1:k ) -IE g(X 1:k ) Z 1:k = z 1:k φ x 1:k ,u 1:k (t)(1) -φ x 1:k ,u 1:k (t)(0) k i=1 K(u i ) f Z (z i ) dµ(x i )du i = g(x 1:k ) -IE g(X 1:k ) Z 1:k = z 1:k • α-1 j=1 1 j! φ x 1:k ,u 1:k (t) (j) (0) + 1 α! φ x 1:k ,u 1:k (t) (α) (t x,u ) k i=1 K(u i ) f Z (z i ) dµ(x i )du i = g(x 1:k ) -IE g(X 1:k ) Z 1:k = z 1:k • 1 α! φ x 1:k ,u 1:k (t) (α) (t x,u ) k i=1 K(u i ) f Z (z i ) dµ(x i )du i = g(x 1:k ) -IE g(X 1:k ) Z 1:k = z 1:k • 1 α! φ x 1:k ,u 1:k (t) (α) (t x,u ) -φ x 1:k ,u 1:k (t) (α) (0) k i=1 K(u i ) f Z (z i ) dµ(x i )du i .
For every real t, we have

φ (α) (t) = m1+ ••• +m k =α n m 1:k k i=1 ∂ mi f X,Z x i , z i + htu i ∂t mi = m1+ ••• +m k =α n m 1:k k i=1 p j1,...,jm i =1 h mi u i,j1 . . . u i,jm i ∂ mi f X,Z ∂z j1 • • • ∂z jm i x i , z i + htu i = h α m1+ ••• +m k =α n m 1:k k i=1 p j1,...,jm i =1 u i,j1 . . . u i,jm i ∂ mi f X,Z ∂z j1 • • • ∂z jm i x i , z i + htu i . (8.11)
Therefore, we get

IE[S σ ] = m1+ ••• +m k =α n m 1:k k i=1 K(u i ) k i=1 f Z (z i ) p j1,...,jm i =1 u i,j1 . . . u i,jm i • g(x 1:k ) -IE g(X 1:k ) Z 1:k = z 1:k • ∂ mi f X,Z ∂z j1 • • • ∂z jm i x i , z i + htu i - ∂ mi f X,Z ∂z j1 • • • ∂z jm i x i , z i dµ(x 1 )du 1 • • • dµ(x k )du k ,
and, using Assumption 8.2.6, this yields

IE[S σ ] ≤ C g,f,α C K,α h α+k f k Z,min α! .
Now we bound the stochastic component. We have the following equality

σ∈I k,n S σ -IE[S σ ] = (n -k)! n! σ∈I k,n g (X σ(1) , Z σ(1) ) , . . . , (X σ(k) , Z σ(k) )
with the function g defined by

g (X 1 , Z 1 ) , . . . , (X k , Z k ) = K h Z 1 -z 1 • • • K h Z k -z k k i=1 f Z (z i ) g(X 1:k ) -IE g(X 1:k ) Z 1:k = z 1:k -IE K h Z 1 -z 1 • • • K h Z k -z k k i=1 f Z (z i ) g(X 1:k ) -IE g(X 1:k ) Z 1:k = z 1:k By construction, IE g (X 1 , Z 1 ) , . . . , (X k , Z k ) = 0. If g is
≤ 4C k K h -kp C k g /f k Z,min . Moreover, we have V ar g (X 1 , Z 1 ) , . . . , (X k , Z k ) ≤ IE K 2 h Z 1 -z 1 • • • K 2 h Z k -z k k i=1 f 2 Z (z i ) g 2 (X 1 , . . . , X k ) ≤ C 2 g f k Z,max f -2k Z,min h -kp ||K|| 2k 2 .
Therefore, we can apply Lemma 8.15, and we get

IP σ∈I k,n S σ -IE[S σ ] > t ≤ 2 exp - [n/k]t 2 2C 2 g f k Z,max f -2k Z,min h -kp ||K|| 2k 2 + (8/3)C k K h -kp C k g f -k Z,
IP   σ∈I k,n S σ -IE[S σ ] > t   ≤ exp - t 2 f 2k Z,min h kp [n/k] 128 B g,z + Bg 2 C 2k-1 K + 2t B g,z + Bg C k K f k Z,min .
Proof: Using the same decomposition for U-statistics as in Hoeffding [START_REF] Hoeffding | Probability inequalities for sums of bounded random variables[END_REF], we obtain

σ∈I k,n S σ -IE[S σ ] = 1 n! σ∈In,n 1 [n/k] [n/k] i=1 V n,i,σ , where V n,i,σ := g X σ(1+(i-1)k) , Z σ(2+(i-1)k) , . . . , X σ(ik) , Z σ(jk) .
For any λ > 0, we have

IP   σ∈I k,n S σ -IE[S σ ] > t   ≤ e -λt IE   exp   λ σ∈I k,n S σ -IE[S σ ]     ≤ e -λt IE   exp   λ 1 n! σ∈In,n 1 [n/k] [n/k] i=1 V n,i,σ     Chapter 8. Estimation of a regular conditional functional by conditional U-statistic regression 238 ≤ e -λt 1 n! σ∈In,n IE   exp   λ 1 [n/k] [n/k] i=1 V n,i,σ     ≤ e -λt 1 n! σ∈In,n [n/k] i=1 IE exp λ 1 [n/k] V n,i,σ ≤ e -λt sup σ∈In,n, i=1,...,[n/k] IE exp λ[n/k] -1 V n,i,σ [n/k] . (8.12) Let l ≥ 2. Using the inequality (a + b + c + d) l ≤ 4 l (a l + b l + c l + d l ), we get IE |V n,i,σ | l = IE |V n,1,σ | l ≤ 4 l IE |g(X σ(1) , . . . , X σ(k) )| l k i=1 |K h | l Z σ(i) -z i f l Z (z i ) + 4 l IE IE g(X 1:k ) Z 1:k = z 1:k l k i=1 |K h | l Z σ(i) -z i f l Z (z i ) + 4 l IE g(X σ(1) , . . . , X σ(k) ) k i=1 K h Z σ(i) -z i f l Z (z i ) l + 4 l IE IE g(X 1:k ) Z 1:k = z 1:k k i=1 K h Z σ(i) -z i f l Z (z i ) l
Using Jensen's inequality for the function x → |x| p with the second, third and fourth terms, and the law of iterated expectations for the first and the third terms, we get

IE |V n,i,σ | l ≤ 4 l • 2 IE IE |g(X σ(1) , . . . , X σ(k) )| l Z σ(1) , . . . , Z σ(k) k i=1 |K h | l Z σ(i) -z i f l Z (z i ) + 4 l • 2 IE IE g(X 1:k ) l Z i = z i , ∀i = 1, . . . , k k i=1 |K h | l Z σ(i) -z i f l Z (z i ) ≤ 4 l • 2 IE B l g (Z 1 , . . . , Z k ) + B l g (z 1 , . . . , z k ) l l! k i=1 |K h | l Z σ(i) -z i f l Z (z i ) ≤ 4 l • 2 Bl g + B l g (z 1 , . . . , z k ) l!(h -kp C k K f -k Z,min ) l-1 f -k Z,min ≤ 2 4 Bg + B g,z h -kp C k K f -k Z,min l l! h kp C -1 K ,
where B g,z := B g (z 1 , . . . , z k ). Remarking that IE[V n,i,σ ] = 0 by construction of g, we obtain

IE exp λ[n/k] -1 V n,i,σ = 1 + ∞ l=2 IE (λ[n/k] -1 V n,i,σ ) l l! ≤ 1 + 2C -1 K h kp ∞ l=2 (4λ[n/k] -1 B g,z + Bg h -kp C k K f -k Z,min ) l ≤ 1 + 2C -1 K h kp • 4λ[n/k] -1 B g,z + Bg h -kp C k K f -k Z,min 2 1 -4λ[n/k] -1 B g,z + Bg h -kp C k K f -k Z,min ≤ exp 32λ 2 [n/k] -2 B g,z + Bg 2 h -kp C 2k-1 K f -2k Z,min 1 -4λ[n/k] -1 B g,z + Bg h -kp C k K f -k Z,min
, where the last statement follows from the inequality 1 + x ≤ exp(x). Combining the latter bound with Equation (8.12), we get

IP   σ∈I k,n S σ -IE[S σ ] > t   ≤ exp -λt + 32λ 2 B g,z + Bg 2 C 2k-1 K f 2k Z,min h kp [n/k] -4λ B g,z + Bg C k K f k Z,min . (8.13)
Remarking that the right-hand side term inside the exponential is of the form -λt + aλ 2 b-cλ , we choose the value

λ * = tb 2a + tc = tf 2k Z,min h kp [n/k] 64 B g,z + Bg 2 C 2k-1 K + t B g,z + Bg C k K f k Z,min (8.14)
such that -λ * t +

aλ 2 * b-cλ * = -t 2 b 4a+2ct = -t 2 λ * .
Therefore, the right-hand side term of Equation (8.13) can be simplified, and combining this with Equation (8.14), we obtain

IP   σ∈I k,n S σ -IE[S σ ] > t   ≤ exp - t 2 f 2k Z,min h kp [n/k] 128 B g,z + Bg 2 C 2k-1 K + 2t B g,z + Bg C k K f k Z,min .

Proof of Theorem 8.8

By Proposition 8.5, for every t 1 , t 2 > 0 such that C K,α h α /α! + t < f Z,min /2, we have

IP | θ(z 1 , . . . , z k ) -θ(z 1 , . . . , z k )| < 1 + C 3 h α + C 4 t 1 × C 5 h k+α + t 2 ≥ 1 -2 exp - [n/k]t 2 1 h kp C 1 + C 2 t 1 -2 exp - [n/k]t 2 2 h kp C 6 + C 7 t 2 ,
We apply this proposition to every k-tuple z σ(1) , . . . , z σ(k) where σ ∈ I k,n . Combining it with Assumption 8.3.1, we get

IP sup i |ξ i,n | < C Λ 1 + C 3 h α + C 4 t 1 × C 5 h k+α + t 2 ≥ 1 -2 |I k,n | i=1 exp - [n/k]t 2 1 h kp C 1 + C 2 t 1 + exp - [n/k]t 2 2 h kp C 6 + C 7 t 2 ,
Choosing t 1 := f Z,min /4 and using the bound (8.7) on h, we get

IP sup i |ξ i,n | < C Λ 1 + C 3 f Z,min α! 4C K,α + C 4 f Z,min 4 × C 5 h k+α + t 2 ≥ 1 -2 |I k,n | i=1 exp - [n/k]f 2 Z,min h kp 16C 1 + 4C 2 f Z,min + exp - [n/k]t 2 2 h kp C 6 + C 7 t 2 . Choosing t 2 = t/(2C 8 ) = t/ 2C ψ C Λ 1 + C 3 f Z,min α! 4C K,α + C 4 f Z,min 4
, and using the bound (8.7) on h α , we get

IP sup i |ξ i,n | < t/C ψ ≥ 1 -2 |I k,n | i=1 exp - [n/k]f 2 Z,min h kp 16C 1 + 4C 2 f Z,min + exp - [n/k]t 2 h kp 4C 2 8 C 6 + 2C 8 C 7 t
.

On the same event, we have max j=1,...,p Lemma 25]). Assume that max j=1,...,p 1 n n i=1 Z i,j ξ i,n ≤ t, for some t > 0, that the assumption RE(s, 3) is satisfied, and that the tuning parameter is given by λ = γt, with γ ≥ 4.

Then, ||Z ( β -β * )|| ≤ 4(γ + 1)t √ s κ(s, 3)
and | β -β * | q ≤ 4 2/q (γ + 1)ts 1/q κ 2 (s, 3) , for every 1 ≤ q ≤ 2.

Proof of Theorem 8.14

Define rn,n := (n × n × h p n,n ) 1/2 , u := rn,n (β -β * ) and ûn,n := rn,n ( βn,n -β * ), so that βn,n = β * + ûn,n /r n,n . We define for every u ∈ R p ,

F n,n (u) := -2r n,n |I k,n | σ∈I k,n ξ σ,n ψ z σ(1) , . . . , z σ(k) T u + 1 |I k,n | σ∈I k,n ψ z σ(1) , . . . , z σ(k) T u 2 + λ n,n r2 n,n β * + u rn,n 1 -β * 1 , (8.15)
and we obtain ûn,n = arg min u∈R p F n,n (u) applying Lemma 8.9.

Lemma 8.20. Under the same assumptions as in Theorem 8.14,

T 1 := rn,n |I k,n | σ∈I k,n ξ σ,n ψ z σ(1) , . . . , z σ(k) D -→ N (0, V 2 ).
This lemma is proved in Section 8.

∈ R p 1 |I k,n | σ∈I k,n ψ z σ(1) , . . . , z σ(k) T u 2 → ψ(z 1 , . . . , z k ) T u 2 f z ,∞ (z 1 ) • • • f z ,∞ (z k ) dz 1 • • • dz k . (8.16)
This has to be read as a convergence of a sequence of real numbers indexed by u, because the design points z i are deterministic. We also have, for any u ∈ R p and when n is large enough,

β * + u rn,n 1 -β * 1 = p i=1 |u i | rn,n 1 {β * i =0} + u i rn,n sign(β * i )1 {β * i =0} .
Therefore, by Assumption 8.3.2(ii)(b), for every u ∈ R p ,

λ n,n r2 n,n β * + u rn,n 1 -β * 1 → 0, (8.17)
when (n, n ) tends to the infinity. Combining Lemma 8.20 and Equations (8.15-8.17), and defining the function

F ∞,∞ by F ∞,∞ (u) := 2 WT u + ψ(z 1 , . . . , z k ) T u 2 f z ,∞ (z 1 ) • • • f z ,∞ (z k ) dz 1 • • • dz k ,
where u ∈ R r and W ∼ N (0, V 2 ), we obtain that every finite-dimensional margin of F n,n weakly converges to the corresponding margin of F ∞,∞ . Now, applying the convexity lemma, we get

ûn,n D -→ u ∞,∞ , where u ∞,∞ := arg min u∈R r F ∞,∞ (u).
Since F ∞,∞ (u) is a continuously differentiable convex function, apply the first-order condition ∇F ∞,∞ (u) = 0, which yields

2 W + 2 ψ(z 1 , . . . , z k )ψ(z 1 , . . . , z k ) T u ∞,∞ f z ,∞ (z 1 ) • • • f z ,∞ (z k ) dz 1 • • • dz k = 0. As a consequence u ∞,∞ = -V -1 1
W ∼ N (0, Ṽas ), using Assumption 8.3.2(iv). We finally obtain rn,n βn,n -β * D -→ N 0, Ṽas , as claimed.

Proof of Lemma 8.20

Using a Taylor expansion yields

T 1 := rn,n |I k,n | σ∈I k,n ξ σ,n ψ z σ(1) , . . . , z σ(k) = rn,n |I k,n | σ∈I k,n Λ θ z σ(1) , . . . , z σ(k) -Λ θ z σ(1) , . . . , z σ(k) ψ z σ(1) , . . . , z σ(k) = T 2 + T 3 ,
where the main term is

T 2 := rn,n |I k,n | σ∈I k,n Λ θ z σ(1) , . . . , z σ(k) θ z σ(1) , . . . , z σ(k) -θ z σ(1) , . . . , z σ(k) ψ z σ(1) , . . . , z σ(k) ,
and the remainder is

T 3 := rn,n |I k,n | σ∈I k,n α 3,σ • θ z σ(1) , . . . , z σ(k) -θ z σ(1) , . . . , z σ(k) 2 ψ z σ(1) , . . . , z σ(k) , with ∀σ ∈ I k,n , |α 3,σ | ≤ C Λ /2, by Assumption 8.3.2(v).
Let us define ψ σ := Λ θ z σ(1) , . . . , z σ(k) ψ z σ(1) , . . . , z σ(k) , for every σ ∈ I k,n . Using the definition (8.1), we rewrite T 2 := T 4 + T 5 where

T 4 := rn,n |I k,n | • |I k,n | σ∈I k,n ς∈I k,n k i=1 K h Z ς(i) -z σ(i) k i=1 f Z (z σ(i) ) g X ς(1) , . . . , X ς(k) -θ z σ(1) , . . . , z σ(k) ψ σ , T 5 := rn,n |I k,n | • |I k,n | σ∈I k,n ς∈I k,n k i=1 K h Z ς(i) -z σ(i) g X ς(1) , . . . , X ς(k) -θ z σ(1) , . . . , z σ(k) × 1 N k (z σ(1) , . . . , z σ(k) ) - 1 k i=1 f Z (z σ(i) ) ψ σ .
To lighten the notations, we will define K σ,ς :=

k i=1 K h Z ς(i) -z σ(i) , g ς := g X ς(1) , . . . , X ς(k) , θ σ := θ z σ(1) , . . . , z σ(k) , f Z ,σ := k i=1 f Z (z σ(i) )
, and N σ := N k (z σ(1) , . . . , z σ(k) ), for every σ ∈ I k,n and ς ∈ I k,n , so that

T 4 := rn,n |I k,n | • |I k,n | σ∈I k,n ς∈I k,n K σ,ς f Z ,σ g ς -θ σ ψ σ , (8.18)
T 5 := rn,n |I k,n | • |I k,n | σ∈I k,n ς∈I k,n K σ,ς g ς -θ σ 1 N σ - 1 f Z ,σ ψ σ . (8.19)
Using α-order limited expansions, we get

IE[T 4 ] = rn,n |I k,n | σ∈I k,n k i=1 K h z i -z σ(i) f Z ,σ g x 1:k -θ σ k i=1 f X,Z (x i , z i )dµ ⊗k (x 1:k )dz 1:k (8.20) = rn,n |I k,n | σ∈I k,n k i=1 K t i f Z ,σ g x 1:k -θ σ k i=1 f X,Z (x i , z σ(i) + ht i )dµ ⊗k (x 1:k )dt 1:k = rn,n h kα |I k,n | σ∈I k,n k i=1 K t i f Z ,σ g x 1:k -θ σ k i=1 d (α) Z f X,Z (x i , z * σ(i) )dµ ⊗k (x 1:k )dt 1:k = O rn,n h kα = O (n × n × h p+2kα n,n ) 1/2 = o(1),
where above, z * i denote some vectors in R p such that |z i -z * i | ∞ ≤ 1, depending on z i and x i . We can therefore use the centered version of T 4 , defined as

T 4 -IE[T 4 ] = rn,n |I k,n | • |I k,n | σ∈I k,n ς∈I k,n g σ,ς , g σ,ς := ψ σ f Z ,σ K σ,ς g ς -θ σ -IE K σ,ς g ς -θ σ .
Computation of the limit of the variance matrix V ar[T 4 ].

We have V ar

[T 4 ] = IE[T 4 T T 4 ] + o(1). V ar[T 4 ] = r2 n,n |I k,n | 2 • |I k,n | 2 σ,σ∈I k,n ς,ς∈I k,n IE[g σ,ς g T σ,ς ] + o(1).
By independence, IE[g σ,ς g T σ,ς ] = 0 as soon as ς ∩ ς = ∅, where we identify a permutation ς and its image ς({1, . . . , k}). Therefore, we get

V ar[T 4 ] nn h p n,n |I k,n | 2 • |I k,n | 2 σ,σ∈I k,n ς,ς∈I k,n ς∩ς =∅ IE g σ,ς g T σ,ς = nn h p n,n |I k,n | 2 • |I k,n | 2 σ,σ∈I k,n ς,ς∈I k,n ς∩ς =∅ g σ,ς,σ,ς -gσ gT σ ,
where gσ :

= ψ σ IE K σ,ς g ς -θ σ /f Z ,σ and g σ,ς,σ,ς := ψ σ ψ T σ f Z ,σ f Z ,σ IE K σ,ς K σ,ς g ς -θ σ g ς -θ σ .
Assume now that ς ∩ ς is of cardinality 1, i.e. there exists only one couple (j, j) ∈ {1, . . . , k} 2 such that ς(j) = ς(j). Then,

g σ,ς,σ,ς = ψ σ ψ T σ f Z ,σ f Z ,σ g(X 1:k ) -θ σ g(x k+1 , . . . , x k+j-1 , x j , x k+j+1 , . . . , x 2k ) -θ σ • k i=1 K h z i -z σ(i) f X,Z (x i , z i )dµ(x i )dz i • K h z j -z σ(j) • k i=1, i =j K h z k+i -z σ(i) f X,Z (x k+i , z k+i )dµ(x k+i )dz k+i = ψ σ ψ T σ f Z (z j ) g(X 1:k ) -θ σ g(x k+1 , . . . , x k+j-1 , x j , x k+j+1 , . . . , x 2k ) -θ σ • k i=1 K(t i ) f X,Z (x i , z σ(i) + ht i ) f Z (z σ(i) ) dµ(x i )dt i • h -p K t i + z σ(j) -z σ(j) h • k i=1, i =j K(t k+i ) f X,Z (x k+i , z σ(i) + ht k+i ) f Z (z k+i ) dµ(x k+i )dt k+i ψ σ ψ T σ f Z (z j ) g(X 1:k ) -θ σ g(x k+1 , . . . , x k+j-1 , x j , x k+j+1 , . . . , x 2k ) -θ σ • k i=1 K(t i ) f X,Z (x i , z σ(i) ) f Z (z i ) dµ(x i )dt i • h -p K t i + z σ(j) -z σ(j) h • k i=1, i =j K(t k+i ) f X,Z (x k+i , z σ(i) ) f Z (z σ(i) ) dµ(x k+i )dt k+i .
By assumption, this is zero unless σ(j) = σ(j). In this case, it can be simplified, giving

g σ,ς,σ,ς ψ σ ψ T σ f Z (z j )h p K 2 g(x 1:k ) -θ σ g(x k:2k,j→j ) -θ σ • k i=1 f X|Z=z σ(i) (x k )dµ(x i ) k i=1, i =j f X|Z=z σ(i) (x k+i )dµ(x k+i ) =: h -p g σ,σ,j,j ,
where x k:2k,j→j := (x k+1 , . . . , x k+j-1 , x j , x k+j+1 , . . . , x 2k ).

Note that, if ς ∩ ς is of cardinality strictly greater than 1, some supplementary powers of h -p arise thanks to the repeated kernels in ς and ς. As a consequence, they are of lower order and therefore negligible. Using α-order expansions as in Equation (8.20), we get sup σ |g σ | = O(h kα ). Thus,

V ar[T 4 ] O nn h p+2kα n,n + nn h p n,n |I k,n | 2 • |I k,n | 2 ς∈I k,n k j,j=1 ς∈I k,n ς(j)=ς(j), |ς∩ς|=1 σ,σ∈I k,n , σ(j)=σ(j) h -p g σ,σ,j,j n |I k,n | 2 k j,j=1 σ,σ∈I k,n , σ(j)=σ(j) g σ,σ,j,j → k j,j=1 g j,j,∞ = V 2 , where g j,j,∞ := Λ θ z 1:k Λ θ z k:2k,j→j ψ z 1:k ψ T z k:2k,j→j K 2 f Z (z j ) g(x 1:k ) -θ(z 1:k ) • g(x k:2k,j→j ) -θ(z k:2k,j→j ) 2k i=1,i =k+j f X|Z=z i (x i )f Z ,∞ (z i )dµ(x i )dz i .
In Section 8.7.2, we will prove that T 4 is asymptotically Gaussian ; therefore, its asymptotic variance will be given by V 2 . Now, decompose the term T 5 , defined in Equation (8.19), using a Taylor expansion of the function

x → 1/(1 + x) at 0. 1 N σ - 1 f Z ,σ = 1 f Z ,σ 1 1 + Nσ-f Z ,σ f Z ,σ -1 = - N σ -f Z ,σ f 2 Z ,σ + T 7,σ , where
T 7,σ := 1 f Z ,σ (1 + α 7,σ ) -3 N σ -f Z ,σ f Z ,σ 2 , with |α 7,σ | ≤ N σ -f Z ,σ f Z ,σ .
We have therefore the decomposition T 5 = -T 6 + T 7 , where

T 6 := rn,n |I k,n | • |I k,n | σ∈I k,n ς∈I k,n K σ,ς g ς -θ σ N σ -f Z ,σ f 2 Z ,σ ψ σ , (8.21)
T 7 := rn,n |I k,n | • |I k,n | σ∈I k,n ς∈I k,n K σ,ς g ς -θ σ T 7,σ ψ σ . (8.22)
Summing up all the previous equation, we get Using the Hájek projection of T 4 , we define

T 1 = (T 4 -IE[T 4 ]) -T 6 + T 7 + T 3 + o(1
T 4 -IE[T 4 ] = T 4,1 + T 4,2 , where T 4,1 := rn,n |I k,n | • |I k,n | σ∈I k,n ς∈I k,n k i=1 IE[g σ,ς |ς(i)], T 4,2 := rn,n |I k,n | • |I k,n | σ∈I k,n ς∈I k,n g σ,ς - i=1,...,k IE[g σ,ς |ς(i)] ,
denoting by |i the conditioning with respect to (X i , Z i ), for i ∈ {1, . . . , n}. We will show that T 4,1 is asymptotically normal, and that T 4,2 = o(1).

Using the fact that the (X i , Z i ) i are i.i.d., and denoting by Id the injective function i → i, we have

T 4,1 = kr n,n n|I k,n | σ∈I k,n n i=1 IE ψ σ f Z ,σ K σ,Id g Id -θ σ -g σ i kr n,n n|I k,n | σ∈I k,n n i=1 IE ψ σ f Z ,σ K σ,Id g Id -θ σ i =: n i=1 α 4
IE |α 4,i,n | 3 ∞ = n IE |α 4,1,n | 3 ∞ = k 3 nr 3 n,n n 3 |I k,n | 3 σ,ν,ϑ∈I k,n ψ σ ⊗ ψ ν ⊗ ψ ϑ f Z ,σ f Z ,ν f Z ,ϑ IE IE K σ,Id g Id -θ σ 1 IE K ν,Id g Id -θ ν ) 1 IE K ϑ,Id g Id -θ ϑ ) 1 = k 3 r3 n,n n 2 |I k,n | 3 σ,ν,ϑ∈I k,n ψ σ ⊗ ψ ν ⊗ ψ ϑ f Z (z ν(1) )f Z (z ϑ(1) ) K h z 1 -z σ(1) K h z 1 -z ν(1) K h z 1 -z ϑ(1) • k i=2 K h z i -z σ(i) K h z k+i -z ν(i) K h z 2k+i -z ϑ(i) • g(x 1:k ) -θ σ ) g(x 1 , x (k+2):(2k) ) -θ ν ) g(x 1 , x (2k+2):(3k) ) -θ ϑ ) • k i=1 f X,Z (x i , z i ) f Z (z σ(i) ) dµ(x i )dz i k i=2 f X,Z (x k+i , z k+i ) f Z (z ν(i) ) dµ(x k+i )dz k+i k i=2 f X,Z (x 2k+i , z 2k+i ) f Z (z ϑ(i) ) dµ(x 2k+i )dz 2k+i k 3 r3 n,n n 2 |I k,n | 3 σ,ν,ϑ∈I k,n ψ σ ⊗ ψ ν ⊗ ψ ϑ f Z (z ν(1) )f Z (z ϑ(1) ) h -2p K(t 1 )K t 1 + z σ(1) -z ν(1) h K t 1 + z σ(1) -z ϑ(1) h • k i=2 K h t i K h t k+i K h t 2k+i g(x 1:k ) -θ σ ) g(x 1 , x (k+2):(2k) ) -θ ν ) g(x 1 , x (2k+2):(3k) ) -θ ϑ ) • k i=1 f X|Z=z σ(i) (x i )dµ(x i )dz i k i=2 f X|Z=z ν(i) (x k+i )dµ(x k+i )dz k+i k i=2 f X|Z=z ϑ(i) (x 2k+i , t 2k+i )dµ(x 2k+i )dt 2k+i ,
Chapter 8. Estimation of a regular conditional functional by conditional U-statistic regression 245 where in the last equivalent, we use a change of variable from the z i to the t i , and then the continuity of the density f X,Z with respect to z, because h = o(1).

Because of our assumptions, the terms of the sum for which σ(1) = 1 or ν(1) = 1 are zero. Therefore, we get

n i=1 IE |α 4,i,n | 3 ∞ = r3 n,n h -2p n 2 |I k,n | 3 σ,ν,ϑ∈I k,n ,σ(1)=ν(1)=1 O(1) = O (nn h p ) 3/2 n 2 n 2 h 2p = O 1 (nn h p ) 1/2 = o(1).
We prove now that T 4,2 = o(1). Note first that, by construction, IE[T 4,2 ] = 0. Computing its variance, we get

IE T 4,2 T T 4,2 = IE r2 n,n |I k,n | 2 • |I k,n | 2 σ,σ∈I k,n ς,ς∈I k,n g σ,ς - i=1,...,k IE g σ,ς ς(i) g σ,ς - i=1,...,k IE g σ,ς ς(i) T =: r2 n,n |I k,n | 2 • |I k,n | 2 σ,σ∈I k,n ς,ς∈I k,n IE gσ,σ,ς,ς . (8.23)
Because of IE[g σ,ς ] = 0 and by independence, the terms in the latter sum for which ς ∩ ς = ∅ are zero.

Otherwise, there exists j 1 , j 2 ∈ {1, . . . , k} such that ς(j 1 ) = ς(j 2 ). If ς ∩ ς is of cardinal 1, meaning that there is no other identities between elements of ς and ς, then we will show that the corresponding term is zero as well. We place ourselves in this case, assuming that |ς ∩ ς| = 1, and we get

IE gσ,σ,ς,ς = IE g σ,ς - i=1,...,k IE g σ,ς ς(i) g T σ,ς - i=1,...,k IE g T σ,ς ς(i) = IE g σ,ς -IE g σ,ς ς(j 1 ) g T σ,ς -IE g T σ,ς ς(j 2 ) = IE IE g σ,ς -IE g σ,ς ς(j 1 ) g T σ,ς -IE g T σ,ς ς(j 1 ) ς(j 1 ) = IE IE g σ,ς g T σ,ς ς(j 1 ) -IE IE g σ,ς ς(j 1 ) IE g T σ,ς ς(j 1 ) = 0.
Therefore, non-zero terms in Equation (8.23) correspond to the case where there exists j 3 = j 1 , j 4 = j 1 such that ς(j 3 ) = ς(j 4 ). It is equivalent to |ς ∩ ς| ≥ 2. We will ignore higher-order terms, i.e. the ones for which |ς ∩ ς| > 2, as they yield higher powers of h p and are therefore negligible. Finally, Equation (8.23) becomes

IE T 4,2 T T 4,2 r2 n,n |I k,n | 2 • |I k,n | 2 σ,σ∈I k,n ς,ς∈I k,n |ς∩ς|=2 IE g σ,ς g T σ,ς -2kIE IE g σ,ς ς(i) IE g T σ,ς ς(i) .
As before, using change of variables and limited expansions, we can prove that

r2 n,n |I k,n | 2 • |I k,n | 2 σ,σ∈I k,n ς,ς∈I k,n |ς∩ς|=2 IE g σ,ς g T σ,ς = o(1),
and similarly for the other term.

Convergence of T 6 to 0

Using Equation (8.21), we have T 6 = T 6,1 + T 6,2 , where

T 6,1 := rn,n |I k,n | • |I k,n | σ∈I k,n ς∈I k,n K σ,ς g ς -θ σ N σ -IE[N σ ] f 2 Z ,σ ψ σ , (8.24)
T 6,2 := rn,n |I k,n | • |I k,n | σ∈I k,n ς∈I k,n K σ,ς g ς -θ σ IE[N σ ] -f Z ,σ f 2 Z ,σ ψ σ . (8.25)
We first prove that T 6,1 = o(1). Using Equation (8.5), we have

T 6,1 = rn,n |I k,n | • |I k,n | σ∈I k,n ς∈I k,n 1 f 2 Z ,σ K σ,ς g ς -θ σ N k (z σ(1:k) ) -IE[N k (z σ(1:k) )] ψ σ = rn,n |I k,n | • |I k,n | σ∈I k,n ς∈I k,n 1 f 2 Z ,σ K σ,ς g ς -θ σ ν∈I k,n k i=1 K h Z ν(i) -z σ(i) -IE k i=1 K h Z ν(i) -z σ(i) ψ σ = rn,n |I k,n | • |I k,n | σ∈I k,n ς,ν∈I k,n 1 f 2 Z ,σ K σ,ς g ς -θ σ K σ,ν -IE K σ,ν ψ σ .
The terms for which |ς ∩ ν| ≥ 1 induce some powers of (nh p ) -1 , and are therefore negligible. We remove them to obtain an equivalent random vector T 6,1 , which is centered. Therefore it is sufficient to show that its second moment tends to 0.

IE T 6,1 T T 6,1 = r2 n,n |I k,n | 2 • |I k,n | 2 σ,σ∈I k,n ς,ν∈I k,n ς∩ν=∅ ς,ν∈I k,n ς∩ν=∅ ψ σ f 2 Z ,σ ψ T σ f 2 Z ,σ g σ,σ,ς,ς,ν,ν , g σ,σ,ς,ς,ν,ν := IE K σ,ς g ς -θ σ K σ,ν -IE K σ,ν K σ,ς g ς -θ σ K σ,ν -IE K σ,ν .
The term g σ,σ,ς,ς,ν,ν is 0 in two cases : if ν ∩ (ς ∪ ς ∪ ν) or if ν ∩ (ς ∪ ς ∪ ν). This condition can be written as

∅ = ν ∩ (ς ∪ ν) ∪ ν ∩ (ς ∪ ν) = (ν ∪ ν) ∩ (ς ∪ ν) ∩ (ς ∪ ν).
We deduce that non-zero terms arise only when there exists j 1 , j 2 ∈ {1, . . . , k} such that: ν(j 1 ) = ν(j 2 )

or ν(j 1 ) = ς(j 2 ) or ν(j 1 ) = ς(j 2 ). Therefore, we can write IE T 6,1 T T 6,1 = T 6,1,1 + T 6,1,2 + T 6,1,3 , where

T 6,1,1 = r2 n,n |I k,n | 2 • |I k,n | 2 k j1,j2=1 σ,σ∈I k,n ς,ν∈I k,n ς∩ν=∅ ς,ν∈I k,n ς∩ν=∅,ν(j2)=ν(j1) ψ σ f 2 Z ,σ ψ T σ f 2 Z ,σ g σ,σ,ς,ς,ν,ν , T 6,1,2 = r2 n,n |I k,n | 2 • |I k,n | 2 k j1,j2=1 σ,σ∈I k,n ς,ν∈I k,n ς∩ν=∅ ς,ν∈I k,n ς∩ν=∅,ς(j2)=ν(j1) ψ σ f 2 Z ,σ ψ T σ f 2 Z ,σ g σ,σ,ς,ς,ν,ν , T 6,1,3 = r2 n,n |I k,n | 2 • |I k,n | 2 k j1,j2=1 σ,σ∈I k,n ς,ν∈I k,n ς∩ν=∅ ς,ν∈I k,n ς∩ν=∅,ν(j1)=ς(j2) ψ σ f 2 Z ,σ ψ T σ f 2 Z ,σ g σ,σ,ς,ς,ν,ν ,
We will prove that T 6,1,1 = o(1). The two other terms can be treated in a similar way. Because of our assumptions, the terms for which σ(j 1 ) = σ(j 2 ) are zero. This divides the number of possible terms by n . By using limited expansions as in Equation (8.20), we get that g σ,σ,ς,ς,ν,ν = O(h kα-p ). Therefore, we have T

6,1,1 = O nn h p nn h kα-p = O(h kα ) = o(1).
Chapter 8. Estimation of a regular conditional functional by conditional U-statistic regression 247 Concerning T 6,2 , its variance matrix is given by

V ar T 6,2 = r2 n,n |I k,n | 2 • |I k,n | 2 σ,σ∈I k,n ς,ς∈I k,n IE[N σ ] -f Z ,σ f 2 Z ,σ IE[N σ ] -f Z ,σ f 2 Z ,σ ψ σ ψ σ g σ,σ,ς,ς , g σ,σ,ς,ς := IE K σ,ς K σ,ς g ς -θ σ g ς -θ σ -IE K σ,ς g ς -θ σ IE K σ,ς g ς -θ σ .
Note that g σ,σ,ς,ς = 0 when ς ∩ ς = ∅. This divides the number of terms in the sum above by n, and imposes that σ ∩ ς = 0, which divides the number of terms in the sum above by another n . Finally, limited expansions gives a bound of h kα-p . Summing up all these elements, we obtain V ar T

T 7 = rn,n |I k,n | • |I k,n | σ∈I k,n ς∈I k,n K σ,ς g ς -θ σ T 7,σ ψ σ , T 7,σ := 1 f Z ,σ (1 + α 7,σ ) -3 N σ -f Z ,σ f Z ,σ 2 , with |α 7,σ | ≤ N σ -f Z ,σ f Z ,σ . By Lemma 8.3 applied to z 1 = z σ(1) , . . . , z n = z σ(n ) , for σ ∈ I k,n , we get IP sup σ∈I k,n N σ -f Z ,σ ≤ C K,α α h α + t ≥ 1 -2 exp - [n/k]t 2 h -kp C 1 + h -kp C 2 t ,
for any t > 0. Therefore, sup σ∈I k,n |T 7,σ | = O IP (h 2α ) by choosing t = h α/k . Then,

|T 7 | ≤ sup σ∈I k,n |T 7,σ | rn,n |I k,n | • |I k,n | σ∈I k,n ς∈I k,n |K σ,ς | • g ς -θ σ • |ψ σ |.
The expectation of the double sum is O(h α ), by α-order limited expansions. By Markov's inequality, we deduce

T 7 = O IP rn,n sup σ∈I k,n |T 7,σ |h α ) = O IP (r n,n h 3α ) = O IP (nn h p+3α ) 1/2 , therefore T 7 = o IP (1).

Convergence of T 3 to 0

We have

T 3 := rn,n |I k,n | σ∈I k,n α 3,σ • θ z σ(1) , . . . , z σ(k) -θ z σ(1) , . . . , z σ(k) 2 ψ z σ(1) , . . . , z σ(k) , with ∀σ ∈ I k,n , |α 3,σ | ≤ C Λ /2. Therefore T 3 rn,n |I k,n | σ∈I k,n θ z σ(1) , . . . , z σ(k) -θ z σ(1) , . . . , z σ(k) 2 rn,n |I k,n | 1 |I k,n | ς∈I k,n K σ,ς f Z ,σ g ς -θ σ + K σ,ς g ς -θ σ 1 N σ - 1 f Z ,σ 2 = T 8 + T 9 + T 10 ,
Chapter 8. Estimation of a regular conditional functional by conditional U-statistic regression 248 where

T 8 := rn,n |I k,n | • |I k,n | 2 σ∈I k,n ς,ς∈I k,n K σ,ς K σ,ς f 2 Z ,σ g ς -θ σ g ς -θ σ , T 9 := rn,n |I k,n | • |I k,n | 2 σ∈I k,n ς,ς∈I k,n K σ,ς K σ,ς f Z ,σ g ς -θ σ g ς -θ σ 1 N σ - 1 f Z ,σ , T 10 := rn,n |I k,n | • |I k,n | 2 σ∈I k,n ς,ς∈I k,n K σ,ς K σ,ς g ς -θ σ g ς -θ σ 1 N σ - 1 f Z ,σ 2 .
We show that T 8 = o(1). The two other terms can be treated in a similar way.

IE |T 8 | = IE rn,n |I k,n | • |I k,n | 2 σ∈I k,n ς,ς∈I k,n |K σ,ς K σ,ς | f 2 Z ,σ g ς -θ σ • g ς -θ σ = rn,n |I k,n | • |I k,n | 2 σ∈I k,n ς,ς∈I k,n k i=1 K h z ς(i) -z σ(i) K h z ς(i) -z σ(i) f 2 Z ,σ • g x ς(1:k) -θ σ g x ς(1:k) -θ σ i ∈ ς(1:k) ∪ ς(1:k) f X,Z (x i , z i )dµ(x i )dz i .
Note that terms for which ς = ς ∈ I k,n are zero, because the z i are distinct and because of our Assumption 8.3.2(i). Therefore, we get

IE |T 8 | = rn,n |I k,n | • |I k,n | 2 σ∈I k,n ς∈I k,n k i=1 K h z ς(i) -z σ(i) 2 f 2 Z ,σ g x ς(1:k) -θ σ 2 i ∈ ς(1:k) f X,Z (x i , z i )dµ(x i )dz i = rn,n |I k,n | • |I k,n | σ∈I k,n k i=1 K h z i -z σ(i) 2 f 2 Z ,σ g x ς(1:k) -θ σ 2 k i=1 f X,Z (x i , z i )dµ(x i )dz i = rn,n h -kp |I k,n | • |I k,n | σ∈I k,n k i=1 K t i 2 f 2 Z ,σ g x ς(1:k) -θ σ 2 k i=1 f X,Z (x i , z σ(i) + ht i )dµ(x i )dz i = O rn,n h -kp |I k,n | = O n × n × h (1-k)p |I k,n | 2 1/2 = o(1).

Chapter 9

Confidence intervals for ratios of means: limitations of the delta method and honest confidence intervals

Abstract

In econometrics, many parameters of interest can be written as a ratio of two expectations. The main method to construct a confidence interval for such a parameter consists in an application of the delta method. Nevertheless, the delta method is an asymptotic procedure, and the obtained intervals may not be relevant if the sample size is small, or if we consider a sequence of models in which statisticians or econometricians study thinner and thinner effects as they have access to more and more data. We prove a generalization of the delta method for ratios of expectations in this "sequence of models" framework. Our paper complements it with a partial impossibility result: nonasymptotic confidence intervals can be built for ratios of expectations, but not at every level.

Based on this, we propose an easy-to-compute rule-of-thumb index to appraise the reliability of asymptotic confidence intervals based on the delta method. Some simulations and applications to real data illustrate the practical usefulness of our rule of thumb and how our nonasymptotic confidence intervals compare to the asymptotic ones.

Keywords: Delta-method, confidence regions, uniformly valid inference, sequence of models.

Based on [START_REF] Derumigny | On the construction of confidence intervals for ratios of expectations[END_REF] : Derumigny, A., Girard, L., & Guyonvarch Y., On the construction of confidence intervals for ratios of expectations. Arxiv preprint, arXiv:1904.07111, 2019.

Introduction

In applied econometrics, the prevalent method for constructing confidence intervals (CIs) is asymptotic:

the theoretical guarantees for most CIs used in practice hold only when the number of observations tends to infinity. For a large class of parameters, the construction of asymptotic CIs also relies on the delta method. In this paper, we focus on parameters that can be expressed as ratios of expectations for which the delta method is a standard procedure to conduct inference. The objective is twofold: provide tools to detect cases in which the delta method may behave poorly and develop inference procedures Chapter 9. Confidence intervals for ratios of means: limitations of the delta method and honest confidence intervals 250 that remain valid even in such cases.

Many popular estimands in economics take the form of ratios of expectations. A broad class of such examples are estimands that correspond to conditional expectations, since any conditional expectation with a discrete conditioning variable or a conditioning event can be written as a ratio of unconditional expectations. For instance, assume that we observe an i. As for any parameter, one major goal in practice is to construct confidence intervals that are reliable in finite samples. Nevertheless, CIs based on asymptotic justification are widely used in practice. In that respect, it is crucial to know how such asymptotic CIs perform when the sample size n is finite. For ratios of expectations, we document this issue on simulations (see details in Section 9.3.1). One of our findings is that the coverage of asymptotic CIs based on the delta method happens to be far below the nominal level, even for large sample sizes, when the expectation in the denominator is close to 0. For some scenarios, asymptotic CIs require above 100,000 observations to get reasonably close to their nominal level. Denominators close to 0 are not unusual in practice. Coming back to the treatment/wage example of the previous paragraph, a small denominator would correspond to a binary treatment with a low participation rate.

Besides, it is sometimes of interest to consider sequences of distributions indexed by the sample size as it can rationalize the practice of applied social researchers. The heuristic idea is that researchers can consider narrower effects as the data gets richer. This is similar to some frameworks that have been developed for weak instrumental variables. For instance, a researcher may look at the average value of a variable A of interest in a subgroup of the data. A subgroup could be defined as the intersection of, say, time, geographical area, gender, age, income brackets and so on. As the number of observations n grows, it is possible to consider subgroups g n that become thinner and thinner (intersection of more and more variables, for examples). This practice could be modelled as estimating

θ n := E [A | G n = 1] = E [AG n ] /P (G n = 1)
where G n is a binary variable that is equal to 1 if an individual belongs to the subgroup g n and lim n→+∞ P (G n = 1) = 0. In such a setting, it is unclear, even asymptotically, what the properties of CIs based on the delta method are. We show that usual CI can fail, and the limiting law of θ n -θn may not be Gaussian any more, denoting by θn the empirical ratio. In some cases, the difference θ n -θn may actually have a Cauchy limit, for example. Complementing these asymptotic properties, we show on simulations that when the denominator in the ratio goes to zero, those CIs can behave very badly in finite samples too.

The guiding theme of this paper is therefore to develop valid inference procedures in those settings and a practical and easy-to-compute index assessing the reliability of asymptotic CIs for a given dataset with a finite sample size n.

This goal connects to a broad existing literature. In a nutshell, there exist old-established concentration results for expectations, namely upper bounds on the probability that an empirical mean departs from its expectation more than a given threshold, that enable to construct confidence intervals valid for any sample size and for large classes of probability distributions (see in particular [START_REF] Boucheron | Concentration inequalities: A nonasymptotic theory of independence[END_REF]). To our knowledge, there is no such result for ratios of expectations. One of the contributions of this paper is to provide similar concentration results for ratios of expectations, which yield nonasymptotic confidence intervals Chapter 9. Confidence intervals for ratios of means: limitations of the delta method and honest confidence intervals 251 that are valid for classes of distributions that satisfy suitable moment bounds or support conditions. We consider distributions within a class characterized by a lower bound on the first moment for the denominator variable, and an upper bound on the second moment for both numerator and denominator variables 1 .

In addition, our results highlight that there exists a critical confidence level, above which it is not possible to construct nonasymptotic CIs, uniformly valid on such classes, and that are almost surely bounded under every distribution of those classes. More precisely, we exhibit explicit upper and lower bounds on this critical confidence level: the former is a threshold above which we show it is impossible to construct such CIs; the latter is a threshold below which we show how to construct them.

These ideas closely relate to some impossibility results as regards the construction of confidence intervals. A large share of the research effort has concentrated on the problem of constructing confidence intervals for the expectation of a distribution. In an early contribution, [START_REF] Bahadur | The nonexistence of certain statistical procedures in nonparametric problems[END_REF] show that, when P is the set of all distributions on the real line with finite expectation, θ(P ) is the expectation with respect to P and Θ = R, a confidence interval built from an i.i.d. sample of n ∈ N * observations that has coverage 1 -α must contain any real number with probability at least 1 -α for every P ∈ P. Broadly speaking, any confidence interval must have infinite length with positive probability for every P ∈ P to ensure a coverage of 1 -α > 0.

Stronger results can be derived when one further restricts P or Θ. When P is taken to be the set of all distributions on the real line with variance uniformly bounded by a finite constant (henceforth called the BC-case), it is possible to show (using the Bienaymé-Chebyshev inequality) that for every n ∈ N * and every α ∈ (0, 1), there exists a confidence interval that is almost surely bounded under every P ∈ P and has coverage 1 -α. In this case, the obtained CIs have the advantage that their length shrinks to 0 at the optimal rate 1/ √ n. But on the downside, they are rarely of size 1 -α, even asymptotically, except for some extreme distributions. This means that in practice, they are very conservative.

A strand of the literature has also investigated more complex problems in which θ(P ) is not restricted to being an expectation. For very general parameters, [START_REF] Dufour | Some impossibility theorems in econometrics with applications to structural and dynamic models[END_REF] derives a generalization of [START_REF] Bahadur | The nonexistence of certain statistical procedures in nonparametric problems[END_REF]. An implication of the results in [START_REF] Dufour | Some impossibility theorems in econometrics with applications to structural and dynamic models[END_REF] is the existence of an impossibility theorem for ratios of expectations. Let P be a distribution on R 2 with marginals P X and P Y . If θ(P ) = E P X [X] /E P Y [Y ], then for every α ∈ (0, 1), it is impossible to build nontrivial CIs of coverage 1 -α when P is the set of all distributions on R 2 with finite second moments and Θ

= {θ = E P X [X] /E P Y [Y ] : (E P X [X] , E P Y [Y ]) ∈ R × R * }.
As will be explained below, this impossibility result breaks down as soon as P is chosen such that |E P Y [Y ]| is bounded away from 0 uniformly over P. Interestingly, the impossibility breaks down only partly in the sense that there remains an upper bound on confidence levels (that depends on n) above which it is impossible to build nontrivial CIs.

Other results for parameters built as differentiable transformations of one or more expectations are given in [START_REF] Romano | Finite sample nonparametric inference and large sample efficiency[END_REF] and [START_REF] Pinelis | Optimal-order bounds on the rate of convergence to normality in the multivariate delta method[END_REF]: in particular, the latter gives conditions on P and θ : P → θ(P ) under which CIs constructed with the delta method are asymptotically of size 1 -α uniformly over P.

An interesting consequence of our results is that, even if we assume a known positive lower bound on the expectation in the denominator, the limitation on the coverage of our nonasymptotic CIs remains.

That point complements [START_REF] Dufour | Some impossibility theorems in econometrics with applications to structural and dynamic models[END_REF] and can be interpreted as a partial impossibility results: for a given sample size n, interesting CIs can be built but not at every confidence level. By contrast, provided the expectation 1 We refer to this setting as the "Bienaymé-Chebyshev" (BC) case. In Appendix 9.9, we present similar results for distributions whose supports are bounded ("Hoeffding" case).

Chapter 9. Confidence intervals for ratios of means: limitations of the delta method and honest confidence intervals 252 in the denominator is not null, the delta method gives CIs at every confidence level, but their coverage is only asymptotic.

That discrepancy may cast some doubts on the validity of asymptotic CIs. Hence, we suggest a ruleof-thumb index to assess the reliability of the delta method for ratios of expectations. The heuristic idea is simply, for a given sample, to compute the upper bound on the attainable level of our CIs2 : for a level higher than that bound, asymptotic CIs based on the delta method might not reach the nominal level and could therefore be suspect. We illustrate the empirical usefulness of that rule on various simulations.

The rest of the paper is organized as follows. Section 9.2 details our framework and assumptions.

In Section 9.3, we illustrate the weaknesses of the CIs based on the delta method with a denominator "close to 0" on simulations and detail the asymptotic behavior of the delta-method in such a framework.

Section 9.4 is devoted to the construction of nonasymptotic confidence intervals and presents a lower bound on the critical confidence level. In Section 9.5, we derive an upper bound on the critical confidence level and a lower bound on the length of nonasymptotic CIs. This section also includes the description of a practical index to gauge the soundness of asymptotic CIs from the delta method. In Section 9.6, some simulations and applications to a real dataset are presented to illustrate our methods. Section 9.7 concludes. The proofs of all results are postponed to Appendix 9.8. Additional results under an alternative set of assumptions (Hoeffding case) are also detailed in Appendix 9.9. Finally, Appendix 9.10 shows supplementary simulations.

Our framework

Throughout the paper, for any random variable U and n i.i.d. replications (U 1,n , . . . , U n,n ), we denote by U n the empirical mean of U , that is n -1 n i=1 U i,n . We present the two main assumptions on the data generating process that we maintain throughout the article. Assumption 9.2.1. For every n ∈ N * , we observe a sample (X i,n , Y i,n ) i=1,...,n i.i.d.

∼ P X,Y,n , where P X,Y,n is a given distribution on R 2 . The real random variables X 1,n and Y 1,n are such that E

X 2 1,n +E Y 2 1,n < +∞, P (X 1,n = 0) < 1 and E [Y 1,n ] ≥ l Y,n
, where l Y,n > 0 is known.

Note that in practice, the value of l Y,n may not be available for the statistician. This is the reason why, in Section 9.5.2, we propose practical methods that do not need the knowledge of l Y,n . It is worth noting that n indexes both the distribution P X,Y,n of the observations in this model, and the number of observations n. This encompasses the standard i.i.d. setup if the distribution of the observations does not change with n: for every n ∈ N * , P X,Y,n = P X,Y for some given distribution P X,Y .

The assumption P (X 1,n = 0) < 1 simply rules out the possibility of having a numerator almost surely equal to 0, which is a very mild restriction in practice. The last part of the assumption is more interesting. As we assume the existence of a finite expectation, we can consider E [Y 1,n ] ≥ 0 without loss of generality 3 . In order to have properly defined ratios of interest, we need to assume away a null denominator, namely suppose that for every n

∈ N * , E [Y 1,n ] > 0.
The assumption E [Y 1,n ] ≥ l Y,n > 0 for every n ∈ N * is stronger but, necessary to derive nonasymptotic CIs with maintained coverage and that are not trivial. Otherwise, if l Y,n = 0, the impossibility theorem of [START_REF] Dufour | Some impossibility theorems in econometrics with applications to structural and dynamic models[END_REF] applies and prevents from constructing nontrivial CIs no matter the confidence level. Our assumption allows l Y,n to decrease to 0 though, which enables us to handle cases where the delta method may fail.

Chapter 9. Confidence intervals for ratios of means: limitations of the delta method and honest confidence intervals 253

In a way, given the results of [START_REF] Dufour | Some impossibility theorems in econometrics with applications to structural and dynamic models[END_REF], Assumption 9.2.1 can be seen as close to the minimal hypothesis that allows for the possibility of nontrivial confidence intervals for a ratio of expectations in a nonasymptotic framework.

(P X,Y,n ) = E [X 1,n ] /E [Y 1,n ]
with values in Θ = R. We will study confidence intervals C n that are functions of the n bivariate observations

(X 1,n , Y 1,n ), . . . , (X n,n , Y n,n ).
In practice, it is possible that Y n = 0 for a given sample, and it may even happen with a strictly positive probability for some non-continuous distributions of Y . This means that θn := X n /Y n does not exist for such samples. In such a case, it is difficult to construct a meaningful confidence interval.

Different conventions are possible:

• We could choose to define C n = R in this case. This means that the true parameter θ belongs to C n , by definition. We believe that such a case would artificially improve the coverage of C n , as it means that, the higher P(Y n = 0), the better our interval would be in term of coverage.

• We could choose C n = ∅. This means that the hypothesis θ = θ 0 would be rejected for each θ 0 ∈ R, using the duality between tests and confidence intervals. This also something that we would like to avoid, as it may not seem reasonable to reject every choice of θ 0 for the only reason that it cannot be estimated with the current sample.

• Other choices are possible, for example C n = {0}, but they do not seem reasonable either, as there is no reason to select only 0 in our confidence interval, especially if X n = 0.

For these reasons, we choose to let C n undefined for such samples, in the same way as the ratios x/0 are undefined for any real x. In practice, it means that nothing can be said: for any value x ∈ R, it is undefined whether x belongs or not to C n (in the same way as it is undefined whether 1/0 is smaller or higher than 2). Of course, when meeting such a case, the applied statistician could change the sample, for example by collecting more data. One could also consider sub-samples (possibly several and combine them in some way) of data for which the empirical mean of the denominator differs from 0.

Nevertheless, the construction of satisfactory estimators in this case lies beyond the scope of this paper.

Limitations of the delta method

In general, the coverage of asymptotic CIs with nominal level 1-α is uncontrolled for finite samples: for a sample of size n, the coverage of asymptotic CIs may be well below 1 -α. Intuitively, this phenomenon Chapter 9. Confidence intervals for ratios of means: limitations of the delta method and honest confidence intervals 254 should be driven by "problematic" distributions in P in the following sense: when P is close to the boundary of P, the probability c(n, P ) := P P ⊗n (C n θ(P )) may be much smaller than 1 -α.

Let us focus on our case of interest where θ(P ) is a ratio of expectations. For this problem, it is possible to build valid asymptotic CIs based on the delta method. We recall this fact in Section 9.3.1 and illustrate on simulations that when the expectation in the numerator is close to zero, c(n, P ) can be well below the nominal level of the CIs and it may require a very large number of observations to make c(n, P ) reasonably close to the nominal level. In Section 9.3.2, we investigate a more serious issue: in the sequence-of-model framework presented in Section 9.2, we let the expectation in the denominator not only be close to zero, but converge to zero as n increases. We show on simulations that depending on the speed at which the denominator goes to zero, c(n, P ) can either converge to the nominal level (more or less fast) or even not converge at all to the nominal level. This sheds light on a partial failure of the delta method when the denominator goes to zero that we derive formally in Section 9.3.3.

Asymptotic approximation takes time to hold

We first recall how to derive asymptotic confidence intervals based on the delta method in the case where, for every n ∈ N * , P X,Y,n is identical, hence denoted P X,Y . In this case, we simply denote

E[X] = E [X 1,n ] and E[Y ] = E [Y 1,n ]
. Under Assumption 9.2.1, combining the multivariate central limit theorem and the delta method yields

√ n X n Y n - E[X] E[Y ] d -----→ n→+∞ N (0, ∇φ Σ∇φ) , (9.1)
where ∇φ Σ∇φ = V To assess the quality of this CI, we compute c(n, P ) using simulations for different sample sizes n and distributions P and compare it to the nominal level. By definition, c(n, P ) forms an upper bound of the coverage rate. We therefore denote c(n, P ) the maximal coverage, in the sense that the true coverage of our CIs may be lower, but it could not be higher than c n,P . We focus on CIs with a 95% nominal level. converge to the latter. Our simulations even suggest that the maximal coverage tends to 0 for b > 1/2.

For b < 1/2, the maximal coverage of the delta method seems to tend to 1-α. Yet, in line with Figure 9.1, the validity of the asymptotic approximation requires very large sample sizes.

At this stage, Figure 9.2 present some evidence that the standard asymptotic CIs based on the delta method need to be adapted for sequences of models and that the speed of decrease toward 0 of the expectation E [Y 1,n ] matters. The next subsection detail formal results to adapt the delta method for ratios of expectations in this setup.

Extension of the delta method for ratios of expectations in the sequenceof-models framework

We are interested in the asymptotic distribution, as n tends to infinity, of the real random variable

. If n 1/2 E [Y 1,n ] → 0, then S n is equivalent in distribution to: √ n X n -E [X 1,n ] E [Y 1,n ] - √ n Y n -E [Y 1,n ] E [X 1,n ] E [Y 1,n ] 2
, as n → ∞.

2.

If n 1/2 E [Y 1,n ] → +∞, then S n is equivalent in distribution to:

√ n √ n(X n -E [X 1,n ]) √ n(Y n -E [Y 1,n ]) - E [X 1,n ] E [Y 1,n ]
, as n → ∞.

If there exists a finite constant

C = 0 such that √ nE [Y 1,n ] → C as n → ∞, then S n is equivalent in distribution to: nE [X 1,n ] 1 C + √ n Y n -E [Y 1,n ] - 1 C + n X n -E [X 1,n ] C + √ n Y n -E [Y 1,n ]
, as n → ∞.

Theorem 9.1 can thus be interpreted as a generalization of the result given by the CLT and the delta method for ratios of expectations. The sequence-of-models framework allows the expectation in the denominator to tend to 0. Chapter 9. Confidence intervals for ratios of means: limitations of the delta method and honest confidence intervals 258

> 1/2 n 1/2 W 1 /W 2 n 1/2 W 1 /W 2 -C 1 /C 2 -n b-a+1/2 C 1 /C 2 b = 1/2 n 1-a C 1 /(C 2 + W 2 ) -C 1 /C 2 n 1/2 C 1 /(C 2 + W 2 ) -C 1 /C 2 + W 1 /(C 2 + W 2 ) n 1/2 W 1 /(C 2 + W 2 ) b < 1/2 n 2b-a C 1 W 2 /C 2 2 n b (W 1 /C 1 -C 1 W 2 /C 2 2 ) n b W 1 /C 1
Cauchy distributions with parameters that depend on the data generating process. Theoretically, quantiles of the limiting distribution could be computed and thus asymptotic confidence intervals, as in the Normal case. Overall, the results of the table highlight that the method to conduct inference will depend on the speed of convergence of both numerators and denominators. In comparison, the nonasymptotic confidence intervals shown in Section 9.4 provide a unique way of conducting inference, once a class of distributions is defined.

Construction of nonasymptotic confidence intervals

To construct nonasymptotic confidence intervals for ratios of expectations, we rely on the possibility to ensure that, with large probability, (i) X n is close to E [X ). This limitation does not arise when one builds a nonasymptotic CI for an expectation. In that sense, we can say that building a nonasymptotic CI for a ratio of expectations is more demanding. Intuitively, the extra difficulty of the latter task comes from the need to ensure (ii). To stress that point, we show in the next subsection that when Y n is bounded away from 0 and positive almost surely, we can build a nonasymptotic CI for a ratio at every confidence level.

An easy case: the support of Y is well-separated from 0

We present a simple framework in which it is possible to build nonasymptotic CIs, valid for every n ∈ N * , and with coverage 1 -α for every α ∈ (0, 1). To do so, we restrict further the set P of admissible distributions with the following assumption. n is bounded from above. This assumption obviously rules out binary {0, 1} random variables in the denominator of the ratio, which can be quite restrictive in practice. Under this assumption, the following theorem gives a concentration inequality for our ratio of expectations. It is proved in Section 9.8.2.

P X n Y n - E [X 1,n ] E [Y 1,n ] > ε + √ u X,n ε a Y,n l Y,n + ε l Y,n ≤ u X,n nε 2 + u Y,n -l 2 Y,n nε 2 .
As a consequence, P E

[X 1,n ] /E [Y 1,n ] ∈ [X n /Y n ± t] ≥ 1 -α, with the choice t := 1 l Y,n u X,n + u Y,n -l 2 Y,n nα   1 + 1 a Y,n    u X,n + u Y,n -l 2 Y,n nα + √ u X,n      ,
for every α ∈ (0, 1).

The theorem shows that it is possible to construct nonasymptotic CIs for ratios of expectations, with guaranteed coverage at every confidence level, that are almost surely bounded under every distribution in P. In Section 9.4.2, we give an analogous result that only requires Assumptions 9.2.1 and 9.2.2 to hold, so that it encompasses the case of {0, 1}-valued denominators. However, the cost to pay will be an upper bound on the achievable coverage of the confidence intervals.

Chapter 9. Confidence intervals for ratios of means: limitations of the delta method and honest confidence intervals 259 . 4This theorem is proved in Section 9.8.3. It states that when l Y,n > 0, it is possible to build valid nonasymptotic CIs with finite length up to the confidence level 1 -α n . This is a more positive result than [START_REF] Dufour | Some impossibility theorems in econometrics with applications to structural and dynamic models[END_REF] which states that it is not possible to build nontrivial nonasymptotic CIs when l Y,n is taken equal to 0, no matter the confidence level. On the other hand, it is more negative than Theorem 9.2. Note that Theorem 9.3 is not an impossibility theorem since it only claims that considering confidence levels smaller than 1 -α n is sufficient to build nontrivial CIs under Assumptions 9.2.1 and 9.2.2. The remaining question is to find out whether it is necessary to focus on confidence levels that do not exceed a certain threshold under Assumptions 9.2.1 and 9.2.2. We answer this in Section 9.5.1. Theorem 9.3 has two other interesting consequences: for every confidence level up to 1 -α n , a nonasymptotic CI of the form X n /Y n ± t with t > t has coverage 1 -α but is conservative. Moreover, if the DGP does not depend on n (i.e in the standard i.i.d. setup), for every fixed α, the length of the confidence interval shrinks at the optimal rate 1/ √ n. Note that the coefficient 2 in the definition of α n defined above can be reduced to any number w > 1, at the expense of increasing the length of the confidence interval. In fact, the latter tends to infinity when w → 1. This is due to the fact we equalize both terms in the bound of the probability above, and more subtle choices are possible indeed.

Nonasymptotic CIs: impossibility results and practical guidelines

In this section, we prove two impossibility results: a necessary lower bound on the length of nonasymptotic CIs and a maximum confidence level above which it is impossible to build nontrivial nonasymptotic CIs.

Chapter 9. Confidence intervals for ratios of means: limitations of the delta method and honest confidence intervals 260

An upper bound on testable confidence levels

Proposition 9.4. For every n ∈ N * , and every α ∈ (0, α n ) , where α n := 1-l 2 Y,n /u Y,n n , if l 2 Y,n /u Y,n < 1, there is no finite t > 0 such that X n /Y n ± t has coverage 1 -α over P, where P is the class of all distributions satisfying Assumptions 9.2.1 and 9.2.2 for fixed l Y,n , u X,n and u Y,n . This theorem asserts that confidence intervals of the form X n /Y n ± t with coverage higher than 1-α n under Assumptions 9.2.1 and 9.2.2 are not defined (or are of infinite length) with positive probability for at least one distribution in P. This is due to the fact that α n is the lower bound on P(Y n = 0) over all distributions in P.

Remark that when u Y,n /l 2 Y,n = 1, there is no impossibility result anymore: assume that u Y,n /l 2 Y,n = 1 and let Q be a distribution on R 2 that satisfies Assumptions 9.2.1 and 9. Y,n = 1, it is possible to build nontrivial nonasymptotic CIs at every confidence level. Indeed, in that case, we are in fact only estimating a simple mean, and therefore there is no constraint on α. Proposition 9.4 is in fact a corollary of the more general Theorem 9.5 that states that it is impossible to construct confidence intervals that contain X n /Y n almost surely and are almost surely bounded over P with coverage greater than 1 -α n . It is proven in Section 9.8.5. Combining Theorems 9.3 and 9.5, we conclude that there exists some critical level 1-α c n that belongs to the interval [1 -α n , 1 -α n ] such that it is impossible to build nontrivial nonasymptotic CIs if and only if their nominal level is above 1 -α c n . Finally, it is worth remarking that with a sample of size n, CIs based on the delta method with a nominal level 1 -α > 1 -α c n cannot have coverage 1 -α.

Practical methods and plug-in estimators

Nonasymptotic confidence intervals based on Theorem 9.3 require Assumptions 9.2.1 and 9.2.2. In particular, they require the knowledge of the constants l Y,n , u X,n and u Y,n that determine the class of distributions we consider. In practice, we need to state values for l Y,n , u X,n and u Y,n to build our nonasymptotic CIs and to compute α n (equivalently n α )5 or α n . Note that constructing nontrivial and nonasymptotic CIs that overcome the limitations of having to choose some a priori class of distributions is not possible. Indeed, we would get back to [START_REF] Bahadur | The nonexistence of certain statistical procedures in nonparametric problems[END_REF] and [START_REF] Dufour | Some impossibility theorems in econometrics with applications to structural and dynamic models[END_REF] type impossibility results.

How to choose l Y,n , u X,n and u Y,n depends on the specific application. In some cases, stating values can be sensible if researchers do have (some) control or expert knowledge of the variables. Resuming an example started in Section 9.1, if the variable in the denominator happens to be an indicator of participating to some treatment and in the setting of a Randomized Controlled Trial, researchers can have intuitions about reasonable values for the upper bound u Y,n of the probability of being treated, for instance.

Without prior information, a first approximation would be to replace the unknown constants with their empirical counterparts. Under i.i.d. sampling, the latter are consistent estimates of the former; though Chapter 9. Confidence intervals for ratios of means: limitations of the delta method and honest confidence intervals 261 it ruins the exact nonasymptotic approach. On the other hand, it enables us to construct our CIs and the quantity n α , which can be useful in practice as a rule of thumb. We stick to that principle in all our numerical applications (Section 9.6).

For a given level 1 -α and a class of distributions satisfying Assumptions 9.2.1 and 9.2.2 for some values l Y,n , u X,n and u Y,n , n α is the minimal sample size required to construct our nonasymptotic CIs.

For a sample size n < n α , the data is not rich enough to construct nonasymptotic CIs of Theorem 9.3 at this level. Heuristically, the comparison of n α and n can be used as a rule of thumb to quickly assess whether the nominal level of asymptotic CIs based on the delta method holds 6 . Several simulations tend to confirm the practical interest of that rule of thumb as the obtained n α turns out to be empirically very close to the sample size above which the coverage of asymptotic CIs converges to the nominal level 1 -α (see Section 9.6.1).

A lower bound on the length of nonasymptotic confidence intervals

The following theorem is an extension of [START_REF] Catoni | Challenging the empirical mean and empirical variance: a deviation study[END_REF]

P X n Y n - E [X 1,n ] E [Y 1,n ] > ξ v n 3nα > α,

Numerical applications 9.6.1 Simulations

This section presents simulations that support the use of n α , or equivalently α n , as a rule of thumb to give insight into the reliability of the asymptotic confidence intervals from the delta method. The simulations resume the setting of Figure 9.1.

In Figure 9.4, a nominal level 1 -α is fixed and we show the coverage of asymptotic CIs as a function of the sample size n, as well as n α derived in Theorem 9.3. It happens that the coverage converges toward its nominal level for sample sizes around n α , which supports n α as a rule of thumb of interest in practice 7 . 6 Equivalently, we could compare αn and α. As a rule of thumb, αn can be seen as the lowest α (hence the highest nominal level 1 -α) for which the asymptotic CIs based on the delta method are reliable given the sample size n. 7 This fact holds across various specifications (see additional simulations in Appendix 9.10).

Chapter 9. Confidence intervals for ratios of means: limitations of the delta method and honest confidence intervals 262 The rule of thumb based on α n suggests that at nominal levels higher than 1-α n (and for the sample size at hand) asymptotic CIs show some undercoverage. In the specification of Figure 9.5, α n turns out to fall close to the lowest α (hence highest 1 -α) for which the coverage of asymptotic CIs attains their nominal level 9 .

All in all, Figures 9.4

Application to real data

We illustrate the use of our confidence intervals with two applications on real data using French Labor Survey between 2010 and 2017. 10 Both applications resume our canonical example of conditional expectations and use n = 204, 246 observations. First, we compute the expected wage conditional on belonging to top wage brackets. Let W be a real random variable that denotes the wage of an employee. For a given top wage W 0 , the parameter of in-

terest is E [W | W ≥ W 0 ]
, with W 0 = q 1-τ (W ) the quantile at order 1-τ of W . It can be written as a ratio with the variable in the numerator X = W 1{W ≥ W 0 } and in the denominator Y = 1{W ≥ W 0 }. In what follows, we focus on the comparison between our nonasymptotic confidence intervals and the asymptotic CIs from the delta method. As explained in Section 9.5.2, we compute our confidence intervals using plug-in estimators to delimit the class of distributions we consider. 9.6 compares the confidence intervals constructed using Theorem 9.3 with the confidence intervals constructed using the delta method. As expected, our nonasymptotic CIs are broader than 8 The coverage shown is in fact an estimate obtained with Monte-Carlo simulations, namely the mean over a large number of repetitions, as usual in MC simulations. 9 Again, this fact appears to hold for various specifications (cf. Appendix 9.10). 10 The complete references of the databases are as follows: " Nevertheless, we do not have formal results to compare those lengths. At this stage, we can simply say that it is in some sense reassuring that the length of the asymptotic CIs increases relatively more than the length of the minimal one as we focus on thinner top percentage of the distribution.

Using the same data, our second application is an estimation of the proportion of women within the top brackets of the distribution of income. If G is an indicator variable equal to 1 for women and 0 for men, the conditional expectation of interest can be defined as E [G | W ≥ q 1-τ (W )]. As in the first application, Figure 9.9 compares the different confidence intervals.

Conclusion

We provide an overview of the problem of constructing confidence intervals for a ratio of means in asymptotic and nonasymptotic frameworks. Using the delta method, asymptotic confidence intervals can be constructed but they have no coverage guarantee for a finite sample size. Nonasymptotic confidence intervals are proposed, but they depend on unknown parameters, which are functions of the data-generating process. To overcome such difficulties, we have proposed plug-in estimators based on a rule of thumb that allows the statistician to quantify whether the coverage of the delta-method should Chapter 9. Confidence intervals for ratios of means: limitations of the delta method and honest confidence intervals 265 Then the sequence of random variables A n := X n /Y n -θ X,n /θ Y,n satisfies : -→ ( hY,∞ , hX,∞ , W∞ ), where the convergence is to be seen as of a sequence of random vectors defined on ( Ω, Ũ, P). Let us define

2200
1. If n -1/2 = o(θ Y,n ), then A n is equivalent to n -1/
Ãn := θ X,n + hX,n / √ n θ Y,n + hY,n / √ n - θ X,n θ Y,n d = θ X,n + h X,n / √ n θ Y,n + h Y,n / √ n - θ X,n θ Y,n = X n Y n - θ X,n θ Y,n = A n .
Moreover, we have Wn = 1{θ Y,n + hY,n / √ n = 0} and W∞ = 0 almost surely. We can define Ω * = {ω ∈ Ω : Wn (ω) → 0 and ∃N > 0, ∀n ≥ N, hY,n (ω) = 0}.

By the almost sure convergence of ( hY,n , Wn ), we get P( Ω * ) = 1, and for every ω ∈ Ω * , Wn (ω) = 0 and hY,n (ω) = 0 for every n large enough. This means that for every given ω ∈ Ω * , and for every n large enough, Ãn is well-defined. In the rest of the proof, we will fix such a ω ∈ Ω * , so that all random variables may be considered as deterministic. By the almost sure representation theorem, this means that the equivalents and limits that will be obtained will still be valid in law in the original spaces Ω n .

First case:

We have We factorize by θ X,n in the latter expression, which completes the proof.

Ãn = X n Y n - θ X,
X n Y n - E [X 1,n ] E [Y 1,n ] ≤ |X n | • 1 Y n - 1 E [Y 1,n ] + 1 E [Y 1,n ] X n -E [X 1,n ] ≤ X n -E [X 1,n ] + √ u X,n Y n -E [Y 1,n ] a Y,n l Y,n + X n -E [X 1,n ] l Y,n .
Consequently, the event considered in Theorem 9.2 is included in the event:

X n -E [X 1,n ] + √ u X,n Y n -E [Y 1,n ] a Y,n l Y,n + X n -E [X 1,n ] l Y,n > ε + √ u X,
P X n -E [X 1,n ] + √ u X,n Y n -E [Y 1,n ] a Y,n l Y,n + X n -E [X 1,n ] l Y,n > ε + √ u X,n ε a Y,n l Y,n + ε l Y,n ≤ P X n -E [X 1,n ] > ε ∪ Y n -E [Y 1,n ] > ε ≤ P X n -E [X 1,n ] > ε + P Y n -E [Y 1,n ] > ε ,
where we use the union bound for the last inequality. The first conclusion follows from using twice Bienaymé-Chebyshev's inequality applied to the variables X n and Y n and the fact that under Assumptions 9.2.1 and 9.2.2 and Jensen's inequality, V [X 1,n ] ≤ u X,n and V [Y 1,n ] ≤ u Y,n -l 2 Y,n . The second conclusion follows from solving (u X,n + u Y,n -l 2 Y,n )/(nε 2 ) = α.

Chapter 9. Confidence intervals for ratios of means: limitations of the delta method and honest confidence intervals 268

Proof of Theorem 9.3

We start by introducing and proving an intermediate lemma that is also used to prove Theorem 9.12. For a random variable U , ε > 0, and ε ∈ (0, 1) we define the following events:

A U ε := U n -E[U] ≤ ε , and ÃU ε := U n -E[U] ≤ ε E[U] .
Lemma 9.8. Assume that Assumption 9.2.1 holds. Then for every n ∈ N * , ε > 0 and ε ∈ (0, 1), we have

P X n Y n - E [X 1,n ] E [Y 1,n ] > (|E [X 1,n ] | + ε) ε (1 -ε) 2 + ε 1 |E [Y 1,n ] | ≤ 1 -P A X1,n ε + 1 -P ÃY1,n ε .
We use this lemma with different concentration inequalities, giving different speeds (namely Bienaymé-Chebychev or Hoeffding inequality).

Proof of Lemma 9.8: We fix arbitrary ε > 0 and ε ∈ (0, 1). Without loss of generality, we can assume that By the triangle inequality, we get

E [Y 1,n ] > 0 and E [X 1,n ] ≥ 0.
X n Y n - E [X 1,n ] E [Y 1,n ] ≤ |X n | • 1 Y n - 1 E [Y 1,n ] + 1 E [Y 1,n ] X n -E [X 1,n ] .
We now bound the first term using the mean value theorem applied to the function f (x) := 1/(x+E [Y 1,n ])

1 Y n - 1 E [Y 1,n ] = f (Y n -E [Y 1,n ]) -f (0) ≤ |Y n -E [Y 1,n ] | (1 -ε) 2 E [Y 1,n ] 2 ≤ εE [Y 1,n ] (1 -ε) 2 E [Y 1,n ] 2 ,
where the first inequality uses that, on the event ÃY1,n

n -E [Y 1,n ] is (1 -ε)|E [Y 1,n ] |. Therefore, on A X1,n ε ∩ ÃY1,n ε , X n Y n - E [X 1,n ] E [Y 1,n ] ≤ |X n | • εE [Y 1,n ] (1 -ε) 2 E [Y 1,n ] 2 + ε E [Y 1,n ] ≤ |E [X 1,n ] | + |X n -E [X 1,n ] | ε (1 -ε) 2 E [Y 1,n ] + ε E [Y 1,n ] ≤ |E [X 1,n ] | + ε ε (1 -ε) 2 E [Y 1,n ] + ε E [Y 1,n ]
,

where we use the triangle inequality to get the second line. It is indeed the complement of the event considered in the statement of Lemma 9.8.

Proof of Theorem 9.3:

We fix arbitrary n ∈ N * , ε > 0 and ε ∈ (0, 1). By Lemma 9.8, we have

P X n Y n - E [X 1,n ] E [Y 1,n ] > (|E [X 1,n ] | + ε) ε (1 -ε) 2 + ε 1 |E [Y 1,n ] | ≤ 1 -P X n -E [X 1,n ] ≤ ε + 1 -P Y n -E [Y 1,n ] ≤ ε E [Y
P X n Y n - E [X 1,n ] E [Y 1,n ] > √ u X,n + ε ε (1 -ε) 2 + ε 1 l Y,n ≤ 1 -P X n -E [X 1,n ] ≤ ε + 1 -P Y n -E [Y 1,n ] ≤ ε E [Y 1,n ] .
Using Bienaymé-Chebyshev's inequality twice gives the bounds

1 -P X n -E [X 1,n ] ≤ ε ≤ V [X 1,n ] nε 2 1 -P Y n -E [Y 1,n ] ≤ ε E [Y 1,n ] ≤ V [Y 1,n ] nε 2 (E [Y 1,n ]) 2 .
For the numerator, V [X

as claimed.

For the second part of Theorem 9.3, for a fixed α, we equalize each of the two terms in (9.3) to α/2 and solve for ε and ε, which yields: .

ε 2 = 2u X
The bound α n comes from the fact that ε needs to be smaller than 1.

Proof of Theorem 9.6

To prove Theorem 9.6, we need the following lemma.

Lemma 9.9. For every integer n ≥ 7 and every x ∈ (0, 1), x (1 -x/n) n-1 ≥ x/3.

Proof of Lemma 9.9: we write the lemma to be applied directly in the demonstration of Theorem 9.6, but the inequality is equivalent to (1 -x/n) n-1 ≥ 1/3. Under our assumptions on n and x, ln (1 -x/n) is well-defined. Using Taylor-Lagrange formula on the function [0, x] t → ln (1 -t/n) yields:

1 - x n n-1 = exp (n -1) ln 1 - x n = exp -(n -1) x n + 1 2 (1 -τ x/n) 2 x 2 n 2
for some τ ∈ (0, 1). Using the fact that n-1 n ≤ 1, x ≤ 1 and . This bound is actually valid for every x ∈ (0, 1) and every n ∈ N * . The computation of exp -1 +

1 2n(1-n -1 ) 2
shows that the latter is larger than 1/4 whenever n ≥ 3 and larger than 1/3 whenever n ≥ 7.

Chapter 9. Confidence intervals for ratios of means: limitations of the delta method and honest confidence intervals 270

Proof of Theorem 9.6: we start using arguments developed in the proof of [START_REF] Catoni | Challenging the empirical mean and empirical variance: a deviation study[END_REF][Proposition 6.2]. We detail those for the sake of clarity. For every n ∈ N * and η > √ u X,n /n, let us define the following distribution on R, which will be used for the variable in the numerator 11 :

P n,u X,n ,η := u X,n 2n 2 η 2 δ {-nη} + 1 -u X,n n 2 η 2 δ {0} + u X,n 2n 2 η 2 δ {nη} .

This distribution is symmetric, centered and has variance u X,n . As shown in [START_REF] Catoni | Challenging the empirical mean and empirical variance: a deviation study[END_REF], every i.i.d. sample (X i,n ) n i=1 drawn from P n,u X,n ,η satisfies

P X n ≤ -η = P X n ≥ η ≥ P X n = η ≥ n i=1 P (X i,n = nη, X j,n = 0, ∀j = i) = u X,n 2nη 2 1 - u X,n η 2 n 2 n-1
.

Note further that for every integer n ≥ 2, the inequality P X n ≥ η ≥ P X n = η becomes strict and for every ξ ∈ (0, 1) X n ≥ η ⊆ X n > ξη . As a result, if (X i,n ) n i=1 i.i.d.

∼ P n,u X,n ,η , for every η > 0, we have

P X n > ξη > u X,n nη 2 1 - u X,n η 2 n 2 n-1 . (9.4)
The following steps do not show up in [START_REF] Catoni | Challenging the empirical mean and empirical variance: a deviation study[END_REF] since they are specific to controlling ratios of expectations and sample averages. For every n ∈ N * , let us define the following distribution on R, which will be used for the variable in the denominator Y,n almost surely. This implies that for every η > 0 and ξ ∈ (0, 1), the following holds

X n > l Y,n + u Y,n -l 2 Y,n ξη ⊆ X n Y n > ξη .
For fixed n ≥ 7 and α ∈ 0, 1 ∧ n/ l Y,n + u Y,n -l 2 Y,n

2

, we choose η = η(α) = v n /3nα.

Combining the above inclusion with (9.4), and Lemma 9.9 (with the choice x = 3α), we conclude that there exists a distribution on R 2 , namely P n , that fulfills Assumptions 9.2.1 and 9.2.2 such that P

X n Y n - E [X 1,n ] E [Y 1,n ] > ξ v n 3nα > α,
which completes the proof. Using the assumption that I n is undefined whenever X n /Y n is undefined (which is equivalent to Y n = 0), we deduce that P I n undef ined ≥ α n . 11 The notation δ denotes the Dirac distribution. for every α ∈ (0, 1).

The theorem shows that it is possible to construct nonasymptotic CIs for ratios of expectations at every confidence level that are almost surely bounded. However, it requires the additional Assumption 9.9.2, that in particular does not allow for binary {0, 1} random variables in the denominator which may limit its applicability for various applications. In Section 9.9.2, we give an analogous result that only requires Assumptions 9.2.1 and 9.9.1 to hold, so that it encompasses the case of {0, 1}-valued denominators. However, the cost to pay will be an upper bound on the achievable coverage of the confidence intervals.

Concentration inequality in the general case

We seek to build nontrivial nonasymptotic CIs under Assumptions 9.2.1 and 9.9.1 only. Under Assumption 9.2.1, E [Y 1,n ] = 0, so that there is no issue in considering the fraction E [X 1,n ] /E [Y 1,n ]. However, without Assumption 9.9.2, Y n = 0 has positive probability in general so that X n /Y n is well-defined with probability less than one and undefined else. Note that when P Y,n is continuous wrt to Lebesgue's measure, there is no issue in defining X n /Y n anymore since the event Y n = 0 has probability zero. This is not an easier case to establish concentration inequalities though, since without more restrictions,

Y n can still be arbitrarily close to zero with positive probability.

Theorem 9.12. Assume that Assumptions 9.2.1 and 9.9.1 hold. For every n ∈ N * , ε > 0, ε ∈ (0, 1), we have

P X n Y n - E [X 1,n ] E [Y 1,n ] > (|a X,n | ∨ |b X,n | + ε)ε (1 -ε) 2 + ε 1 l Y,n
≤ 2 exp(-nε 2 γ (X 1,n )) + 2 exp(-nε 2 γ (Y 1,n )),

where γ (X 1,n ) = 2/(b X,n -a X,n ) 2 and γ (Y for every α > α n,H := 4e -nγ(Y1,n) . 12This theorem is proven in Section 9.9.4. It states that when l Y,n > 0, it is possible to build valid nonasymptotic CIs with finite length up to the confidence level 1 -α n,H . This is a more positive result than [START_REF] Dufour | Some impossibility theorems in econometrics with applications to structural and dynamic models[END_REF] which claims that it is not possible to build nontrivial nonasymptotic CIs when l Y,n is taken equal to 0, no matter the confidence level. On the other hand, it is more negative than Theorem 9.11.

Note that Theorem 9.12 is not an impossibility theorem since it only claims that considering confidence levels smaller than 1 -α n,H is sufficient to build nontrivial CIs under Assumptions 9.2.1 and 9.9.1. The remaining question is to find out whether it is necessary to focus on confidence levels that do not exceed a certain threshold under Assumptions 9.2.1 and 9.9.1. We answer this in Section 9.9.3.

Theorem 9.12 has two other interesting consequences: for every confidence level up to 1 -α n,H , a nonasymptotic CI of the form X n /Y n ± t with t > t has good coverage but is too conservative. What is more, if the DGP does not depend on n (i.e in the standard i.i.d. setup), for every fixed α > α n,H , the length of the confidence interval shrinks at the optimal rate 1/ √ n. Abstract: This thesis can be divided into three parts.

In the first part, we study adaptivity to the noise level in the high-dimensional linear regression framework.

We prove that two square-root estimators attains the minimax rates of estimation and prediction. We show that a corresponding median-of-means version can still attains the same optimal rates while being robust to outliers in the data.

The second part is devoted to the analysis of several conditional dependence models. We propose some tests of the simplifying assumption that a conditional copula is constant with respect to its conditioning event, and prove the consistency of a semipara-metric bootstrap scheme. If the conditional copula is not constant with respect to the conditional event, then it can be modelled using the corresponding Kendall's tau. We study the estimation of this conditional dependence parameter using 3 different approaches :

kernel techniques, regression-type models and classification algorithms.

The last part regroups two different topics in inference. We review and propose estimators for regular conditional functionals using U-statistics. Finally, we study the construction and the theoretical properties of confidence intervals for ratios of means under different sets of assumptions and paradigms.

Université Paris-Saclay

Espace Technologique / Immeuble Discovery Route de l'Orme aux Merisiers RD 128 / 91190 Saint-Aubin, France

3

  Robust-to-outliers simultaneous inference and noise level estimation using a MOM approach 3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2 Results in the high-dimensional linear regression framework . . . . . . . . . . . . . . . . 3.3 A general framework . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.4 Technical lemmas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.5 Control of the supremum of T K,µ (g, χ, f * , σ * ) on each F (κ) i . . . . . . . . . . . . . . . . . 3.5.1 Preliminaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.5.2 Proof of the first assertion of Lemma 3.12 . . . . . . . . . . . . . . . . . . . . . . . 3.5.3 Proof of the second assertion of Lemma 3.12 . . . . . . . . . . . . . . . . . . . . . 3.6 Proof of Lemma 3.11 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.6.1 Bound on F (κ) 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3.6.2 Bound on F (κ) 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.6.3 Bound on F (κ) 3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.6.3.1 Case ||f -f * || L 2 P ≤ r(ρ K ) . . . . . . . . . . . . . . . . . . . . . . . . . . 3.6.3.2 Case ||f -f * || L 2 P > r(ρ K ) . . . . . . . . . . . . . . . . . . . . . . . . . . 3.6.4 Bound on F (κ) 4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.6.5 Bound on F (κ) 5 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.6.6 Bound on F (κ) 6 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.6.6.1 Case ||f -f * || L 2 P ≤ r(ρ K ) . . . . . . . . . . . . . . . . . . . . . . . . . . 3.6.6.2 Case ||f -f * || L 2 P > r(ρ K ) . . . . . . . . . . . . . . . . . . . . . . . . . . 3.6.7 Bound on F(κ) 7 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.6.8 Bound on F (κ) 8 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Figure 1 . 1 :

 11 Figure 1.1: Links between the different chapters

  If we knew the true β * , we could replace σ by an oracle estimator σoracle := ||Y -Xβ * || n . It is also possible to replace σ in the minimization program by ||Y -Xβ|| n , which is an estimator of σ depending on β. The corresponding minimization problem would be ||Y -Xβ|| 2 n + ||Y -Xβ|| n × pen(β). Simplifying this expression, we define a family of square-root penalized estimators by βsqrt, pen := arg min β∈R p ||Y -Xβ|| n + pen(β), where ||Y -Xβ|| n := 1 n n i=1

D

  O :|D O |=m T (D I ∪ D O ) -T (D I ) = +∞ , (1.1) for an (uncontaminated) dataset D I of size |D I | = n > 0. The breakdown point * (T, D I ) is the minimum proportion of outliers needed to "break the estimator" with the datset D I .

F 1 ,

 1 . . . , F d are continuous distributions on R, and C is a copula (i.e. a continuous distribution on [0, 1] d with uniform margins), then F 1:d defined by ∀x = (x 1 , . . . , x d ) ∈ R d , F 1:d (x) := C F 1 (x 1 ), . . . , F d (x d ) , is a joint distribution on R d whose margins are respectively distributed as F 1 , . . . , F d and whose copula is C.

Figure 1 . 2 :

 12 Figure 1.2: Decomposition of a vector X = (X 1 , . . . , X d ) with multivariate cumulative distribution function F , margins F 1 , . . . F d and copula C using Sklar's Theorem (Theorem 1.1).

Figure 1 . 3 . 1 X 2 Figure 1 . 3 : 1 X 2 Figure 1 . 4 :

 1312131214 Figure 1.3: The "simplifying assumption": Z has an influence on the conditional margins X 1 and X 2 , but not on the conditional dependence between them.

  For any norm || • || on the space generated by the class of all conditional copulas, the test statistics || ĈI|J -Ĉs,I|J || can be used to test H 0 . For example, we can think of Kolmogorov-Smirnov-type statistics T 0 KS,n := ĈI|J -Ĉs,I|J ∞ = sup u I ∈[0,1] p sup x J ∈R d-p ĈI|J (u I |x J ) -Ĉs,I|J (u I ) , or Cramer von-Mises-type test statistics T 0 CvM,n := ĈI|J (u I |x J ) -Ĉs,I|J (u I ) 2 w(du I , dx J ),

) and sup β *

 β ∈B0(2s)IP β * d( β(s) , β * ) ≤ C 0 σw(2s) ≥ 1 -2s p 2s -u n ,(2.15) for a constant C 0 > 0, a function w(•) : [1, s * ] → R + satisfying Assumption 2.4.1, and u n > 0.

Lemma 3 . 13 .

 313 Assume that C K,µ ( f , σ) ≤ B 1,1 . Then Equations (3.14)-(3.16) hold and

  e. p = 2 and d = 3. The sample sizes will be n = 500, except if it is differently specified. Concerning the bootstrap, we will resample N = 200 times to calculate approximated p-values. Each experiment has been repeated 500 times to calculate the percentages of rejection. The computations have been made on a standard laptop, and, for the non-parametric bootstrap, they took an average time of 14.1 seconds for I χ,n ; 26.9s for T 0,m CvM,n , 103s for I 2,n , 265s for T c 2 and 0.922s for T c 2 .

Figure 4 . 1 :Figure 4 . 2 :Figure 4 . 3 :Figure 4 . 4 :

 41424344 Figure 4.1: Rejection percentages for the statistics I χ (4.13) as a function of τ max : we use the gaussian copula, with a conditional parameter θ(x) calibrated such that the conditional Kendall's tau τ (x) satisfies τ (x) = τ max • Φ(x). Solid line: bootNP. Dashed line : bootPseudoInd. Dotted line : bootCond.

Figure 4 . 5 :Figure 4 . 6 :

 4546 Figure 4.5: QQ-plot of a sample of the test statistic T c 2 and a sample of the bootstrap test statistic (T c 2 ) * using the non-parametric bootstrap for the gaussian copula, with different sample sizes and under H c 0

Figure 4 . 7 :

 47 Figure 4.7: Rejection percentage for the statistic T 0,m CvM,n (4.8) with m = 20 as a function of h. The red (resp. black) line corresponds to the alternative (resp. zero) assumption.

  observation of X I given the box index Y C copula family indexed by the elements of a set Θ C θ copula of the family C with the parameter θ ∈ Θ c θ density of the copula C θ θ 0 unconditional parameter of the copula of Z I|J θ(x J ) parameter of the conditional copula of Z I|J given X J = x J θ b 0 unconditional parameter of the copula of Z I|Y θ(A J ) conditional parameter of the copula of

Lemma 4 . 19 .

 419 Let θb 0 and θ(A k ) be the estimators based on the pseudo-sample ( Ẑi,I|Y , Y i ) i=1,...,n (and then on the sample (Z i , Y i ) i=1,...,n ) as θb 0 := arg max θ∈Θ n i=1 log c θ ( Ẑi,I|Y ), and θ(A k ) := arg max θ∈Θ n i=1 log c θ ( Ẑi,I|Y ).1(Y i = k), k = 1, . . . , m.

2 , 3 .

 23 The choice of the bandwidth h can be done in a data-driven way following the general conditional U-statistics framework detailed in Dony and Mason[START_REF] Dony | Uniform in bandwidth consistency of conditional U-statistics[END_REF] Section 2]. Indeed, for any k ∈ {1, 2, 3} and z ∈ Z, denote by

2 , 3 . 3 := 4 and c 2 := 2 . 5 . 3 (

 2332253 They are proved in Section 5.5.3. We will denote c 1 := c Proposition Exponential bound with explicit constants). Under Assumptions 5.3.1-5.3.4, for every

6 .Proposition 5 . 9 (

 659 (i) nh p n → ∞ and nh p+2α n → 0; (ii) K( • ) is compactly supported. Joint asymptotic normality at different points). Let z 1 , . . . , z n be fixed points in a set Z ⊂ R p . Assume 5.3.1, 5.3.4, 5.3.6, that the z i are distinct and that f Z and z → f X,Z (x, z) are continuous on Z, for every x. Then, as n → ∞,

  -5.2 (resp., both with our empirically optimal choice α h = 1.5. We can note that, considering the bias, the estimator τ behaves similarly as τ (1) when the true τ is close to -1, and similarly as τ (3) when the true Kendall's tau is close to 1. But globally, the best pointwise estimator is clearly obtained with the rescaled version τ1,2|Z=• , after a quick inspection of MSE levels, and even if the differences between our four estimators weaken for large sample sizes. The comparative advantage of τ1,2|z more clearly appears with Setting 2 than with Setting 1. Indeed, in the former case, the support of Z's distribution is the whole line. Then fZ does not suffer any more from the boundary bias phenomenon, contrary to what happened with Setting 1. As a consequence, the biases induced by the definitions ofτ (k) 1,2|z , k = 1,3, appear more strinkingly in Figure 5.3, for instance: when z Chapter 5. About kernel-based estimation of conditional Kendall's tau: finite-distance bounds and asymptotic behavior 134 is close to (-1) (resp. 1), the biases of τ (1) 1,2|z (resp. τ (3) 1,2|z ) and τ1,2|z are close, when the bias τ (3) 1,2|z (resp. τ (1)

Lemma 5 . 11 .

 511 Let m, n > 0, X 1 , . . . , X n i.i.d. random vectors with values in a measurable space X and g : X m → [a, b] be a symmetric real bounded function. Set θ := IE[g(X 1 , . . . , X m )] and σ 2 := V ar[g(X 1 , . . . , X m )]. Then, for any t > 0 and n ≥ m,

Figure 5 . 1 :Figure 5 . 2 :Chapter 5 .

 51525 Figure 5.1: Local bias, standard deviation and MSE for the estimators τ (1) (red) , τ (2) (blue), τ (3) (green), τ (orange), with n = 100 and α h = 1.5 in Setting 1. The dotted line on the first figure is the reference at 0.

Figure 5 . 3 :

 53 Figure 5.3: Local bias, standard deviation and MSE for the estimators τ (1) (red) , τ (2) (blue), τ (3) (green), τ (orange), with n = 100 and α h = 1.5 in Setting 2. The dotted line on the first figure is the reference at 0.

Figure 5 . 4 :Chapter 5 .

 545 Figure 5.4: Local bias, standard deviation and MSE for the estimators τ (1) (red) , τ (2) (blue), τ (3) (green), τ (orange), with n = 500 and α h = 1.5 in Setting 2. The dotted line on the first figure is the reference at 0.

  tends to zero a.s. Note that, by Bochner's Lemma, sup z∈Z IE[ φk (z)] -φ k (z) → 0. Then, it remains to show that sup z∈Z φk (z) -IE[ φk (z)] → 0 almost surely. Let ρ n > 0 be such that we cover Z by the union of l n open balls B(t l , ρ n ), where t 1 , . . . , t ln ∈ R p and l n ∈ N * . Then sup z∈Z φk (z) -IE[ φk (z)] ≤ sup l=1,...ln

Assumption 6 . 2 . 1 .

 621 The function z → ψ(z) are bounded on Z by a constant C ψ . Moreover, Λ(•) is continuously differentiable. Let T be the range of z → τ 1,2|Z=z , from Z towards [-1, 1]. On an open neighborhood of T , the derivative of Λ(•) is bounded by a constant C Λ .

Figure 6 . 2 :

 62 Figure 6.2: Estimated conditional Kendall's tau τ1,2|Z=z (red curve), and prediction Λ (-1) ψ(z) T β (blue curve) as a function of z for the application on real data, where the estimated non-zero coefficients are β1 = 0.78, β7 = -0.043, β8 = 0.069 and β11 = 0.020.

Figure 6 . 3 :

 63 Figure 6.3: Evolution of the estimated non-zero coefficients as a function of the regularization parameter λ for the application on real data. All the other non-displayed ψ i coefficients are zero.

  T 4,2 and T 4,3 , note that IE[T 4,2 ] = IE[T 4,3 ] = 0. Moreover, since IE[ζ i,j,j |j] is centered, ζ i1,j,j |j]IE[ζ T i2,j,j |j] .

Figure 6 . 4 :

 64 Figure 6.4: Evolution of the estimated non-zero coefficients as a function of the regularization parameter λ. The non-zero coefficients are β 1 = 3/4 and β 3 = 3/4. Note that the coefficients β2 , β5 and β9 coefficients are always zero (and are not displayed).

Figure 6 . 5 :

 65 Figure 6.5: True conditional Kendall's tau τ 1,2|Z=z (black curve), estimated conditional Kendall's tau

:

  Conditional Kendall's tau, conditional dependence measure, machine learning, classification task, stock indices. Based on [41]: Derumigny, A., & Fermanian, J. D., A classification point-of-view about conditional Kendall's tau. Computational Statistics & Data Analysis, 135, 70-94, 2019.

3 .Figure 7 . 1 :

 371 Figure 7.1: An example of the estimation of the conditional Kendall's tau using different estimation methods (see Table7.2 below). The black dash-dotted curve is the true conditional Kendall's tau that

Remark 8 . 2 .

 82 At first sight, in Model(8.2), there seem to be no noise perturbing the variable of interest.In fact, it can be seen as a simple consequence of our formulation of the model. In the same way, theclassical linear model Y = X T β * + ε can be rewritten as IE[Y |X = x] = x T β * without any explicit noise. By definition, IE[Y |X = x]is a deterministic function of a given x. In our case, the corresponding fact is: Λ θ(z 1 , . . . , z k ) is a deterministic function of the variables (z 1 , . . . , z k ). This means that we cannot write formally a model with noise, such as Λ θ(z 1 , . . . , z k ) = ψ(z 1 , . . . , z k ) T β * + ε where ε is independent of the choice of (z 1 , . . . , z k ) since the left-hand side of the latter equality is a (z 1 , . . . , z k )-mesurable quantity, unless ε is constant almost surely.

  and σ i is the i-th element of I k,n . The chosen order of I k,n is arbitrary and has no impact in practice. In the same way, we define the vector Y of dimension |I k,n | defined by Y i := Λ θ z σi(1) , . . . , z σi(k) , such that the criterion (8.3) is in the standard Lasso form β := arg min β∈R r ||Y -Z β|| 2 + λ|β| 1 , where for any vector v of size |I k,n |, its scaled norm is defined by ||v|| := |v| 2 / |I k,n |. Following [39], we define ξ i,n , for 1 ≤ i ≤ |I k,n |, by ξ i,n = ξ σi,n := Λ θ z σi(1) , . . . , z σi(k) -ψ z σi(1) , . . . , z σi(k) T β * = Λ θ z σi(1) , . . . , z σi(k) -Λ θ z σi(1) , . . . , z σi(k) .

νTheorem 8 . 12 (

 812 n ( βn -β * ) is assumed to be asymptotically normal, for some deterministic sequence (ν n ) that tends to the infinity. Now, let us consider the same optimization program as in (8.3) but with a random tuning parameter given by λ n,n := λn,n /| βn | δ , for some constant δ > 0 and some positive deterministic sequence ( λn,n ). The corresponding adaptive estimator (solution of the modified Equation (8.3)) will be denoted by βn,n , or simply β. Hereafter, we still set S n = {j : βj = 0}. Asymptotic law of the adaptive estimator of β). Under Assumption 8.2.10, if the following convergence hold λn,n (nh p n,n ) 1/2 → ≥ 0 and λn,n (nh p n,n ) 1/2 ν δ n → ∞ when n → ∞, we have (nh p n,n ) 1/2 ( βn,n -β * ) S D -→ u * * S := arg min u S ∈R s F∞,n (u S ), where

Assumption 8 . 3 . 2 .

 832 (i) The support of the kernel K(•) is included into [-1, 1] p . Moreover, for all n, n

  (v) Λ(•) is two times continuously differentiable. Let T be the range of θ, from Z k towards R. On an open neighborhood of T , the second derivative of Λ(•) is bounded by a constant C Λ .

  75h 2 t , Assumption 8.2.4 is satisfied with f Z,min = φ(1)/(1 -2Φ(-1)) > 0.35, so that we can apply Corollary 8.4.

Lemma 8 . 15 .

 815 Let k > 0, n ≥ k, X 1 , . . . , X n i.i.d. random vectors with values in a measurable space X and g : X k → [a, b] be a real bounded function. Set θ := IE[g(X 1:k )] and σ 2 := V ar[g(X 1:k )]. Then, for

Chapter 8 .

 8 Estimation of a regular conditional functional by conditional U-statistic regression 234 We first bound the bias.

  i.d. sample of individuals indexed by i ∈ {1, . . . , n} with W i the wage of an individual and D i an indicator equal to 1 whenever individual i belongs to some treatment group, say a training program; 0 otherwise. Suppose you are interested in the average wage of participants in the program. We have E [W | D = 1] = E [W D] /E [D] as D is binary, which is indeed a ratio of expectations.

  [X] /E[Y ] 2 + E[X] 2 V[Y ]/E[Y ] 4 -2Cov [X, Y ] E[X]/E[Y ]3 and φ is the function (x, y) → x/y. As with expectations, we use the shortcutsV [X] = V [X 1,n ] and Cov [X, Y ] = Cov [X 1,n , Y 1,n ].Based on (9.1), we can construct an asymptotic CI for the parameter of interest E[X]/E[Y ] using a consistent estimate of the asymptotic variance ∇φ Σ∇φ and Slutsky's lemma.

  More specifically, for different sample sizes n and values of E[Y ], we draw 5000 independent samples of size n(X i , Y i ) n i=1 i.i.d. ∼ N (1, 1)⊗N (E[Y ], 1). We compute the maximal coverage for CIs based on the delta method for every pair (n, E[Y ]) using the 5000 replications. E[Y ] ranges from 0.01 (the denominator is close to zero) to 0.75 (the denominator is far from zero).

Figure 9 .

 9 1 sums up the results. For every n, it turns out that the closer E[Y ] to 0, the smaller the maximal coverage of the delta method. When E[Y ] = 0.01, the maximal coverage gets close to the nominal level for n around 300,000.

9. 3 . 2

 32 Asymptotic results may not hold for sequences of modelsUnlike the result displayed in (9.1), it is unclear how the quantity √ n X n /Y n -E[X]/E[Y ] behaves asymptotically when we consider sequences of statistical models with the expectation in the denominator tending to 0 as n increases. For a given specification, Figure 9.2 shows the maximal coverage of asymptotic CIs based on the delta method when E [Y 1,n ] = Cn -b where C is set to 0.025 and b varies. For a speed b ≥ 1/2, 1/2 being the usual speed of the CLT, the maximal coverage of asymptotic CIs based on (9.1) is incorrect in the sense that it is far lower than the nominal level 1 -α and it does not Chapter 9. Confidence intervals for ratios of means: limitations of the delta method and honest confidence

Figure 9 . 1 :

 91 Figure 9.1: Maximal coverage of asymptotic CIs based on the delta method. Specification: ∀n ∈ N * , PX,Y,n = N (1, 1) ⊗ N (E[Y ], 1). The nominal pointwise asymptotic level is set to 0.95. For a pair (E[Y ], n), the coverage is obtained as the mean over 5,000 repetitions.

Theorem 9 . 1 .Chapter 9 .Figure 9 . 2 :

 91992 Figure 9.2: Coverage of asymptotic CIs based on the delta method. Specification: ∀n ∈ N * , PX,Y,n = N (1, 1) ⊗ N (Cn -b , 1), with C = 0.025. The nominal pointwise asymptotic level is set to 0.95. For a pair (E[Y ], n), the coverage is obtained as the mean over 5,000 repetitions.

1

 1

Figure 9 .Figure 9 . 3 : 2 .

 9932 Figure 9.3: Separation between the different asymptotic regimes as a function of (a, b), where E [X 1,n ] = C 1 /n a and E [Y 1,n ] = C 2 /n b , (a, b) ∈ R * + 2 .

  ) follow the distribution N (0, V ), where V = lim V [(X 1,n , Y 1,n )].

Assumption 9 . 4 . 1 .

 941 There exists a Y,n > 0 such that Y 1,n ≥ a Y,n almost surely. Under Assumption 9.4.1, for every n ∈ N * , Y n ≥ a Y,n > 0 almost surely under every distribution in P and Y -1

Theorem 9 . 2 .

 92 Under Assumptions 9.2.1, 9.2.2 and 9.4.1, we have for every n ∈ N * and ε ∈ R * + ,

2 . 2 .

 22 Let (X i,n , Y i,n ) n i=1 i.i.d. ∼ Q. We have that V [Y 1,n ] = 0, which implies that Y 1,n = E [Y1,n ] almost surely. Assumption 9.2.1 further ensures that Y 1,n = 0 almost surely. Consequently, the results of Section 9.4.1 apply and allow us to conclude that under Assumptions 9.2.1, 9.2.2 and u Y,n /l 2

Theorem 9 . 5 .

 95 Let n ∈ N * , and a random set I n that contains X n /Y n almost surely whenever it is defined and is undefined if Y n = 0. Then sup Pn∈P P I n undef ined ≥ α n .

2 Y,n 2 . 2 Y,n 2 , 2 n

 22222 where v n := u X,n / l Y,n + u Y,n -l With this theorem, we can claim that for every α ∈ 0, 1 ∧ n/ l Y,n + u Y,n -l confidence intervals of the form X n /Y n ± t cannot have uniform coverage 1 -α under Assumptions 9.2.1 and 9.2.2 if they are shorter than v n /(3nα). By a careful inspection of the proof (see Lemma 9.9), we can in fact replace the value 3 in the theorem by any number strictly larger than e = exp(1), at the price of assuming n ≥ n 0 for n 0 large enough. It is interesting to note that the distributions Q that are built in the proof of the theorem are extremal in P in the sense that they satisfy E X 2 n = u X,n , E [Y 1,n ] = l Y,n and E Y = u Y,n .

Figure 9 . 4 :

 94 Figure 9.4: Maximal coverage of asymptotic CIs based on the delta method and rule-of-thumb nα.Specification: ∀n ∈ N * , PX,Y,n = N2 (bivariate Gaussian) with E[X] = 0.5, E[Y ] = 0.1, V[X] = 1, V[Y ] = 2, Corr(X, Y ) = 0.5. The nominal pointwise asymptotic level is set to 0.90. For a sample size n, the coverage is obtained as the mean over 5,000 repetitions. The dashed vertical line shows nα := 2 uY,n -lY,n 2 / αl 2 Y,n , setting here α = 0.1, lY,n = E[Y ], uY,n = E[Y ] 2 + V[Y ].

Figure 9 . 5 :

 95 Figure 9.5: Maximal coverage of asymptotic CIs based on the delta method and rule-of-thumb αn.Specification: sample sizen = 1, 000, PX,Y,n = N2 (bivariate Gaussian) with E[X] = 0.5, E[Y ] = 0.25, V[X] = 2, V[Y ] = 1, Corr(X, Y ) = 0.5.For each nominal level 1 -α in the x-axis, we draw 10, 000 samples, compute the asymptotic CIs and see whether it covers or not the ratio of interest; we report the mean over the 10, 000 repetitions in the y-axis. The solid line is the first bisector y = x. The dashed vertical line shows αn := 2 uY,n -lY,n 2 / nlY,n 2 , setting here lY,n = E[Y ], uY,n = E[Y ] 2 + V[Y ].

Chapter 9 .

 9 Confidence intervals for ratios of means: limitations of the delta method and honest confidence intervals 263In Figure9.5, a sample size is fixed and we show the coverage 8 of asymptotic CIs for different nominal levels, as well as α n . It is the converse of Figure9.4.

  and 9.5 and additional simulations (see Appendix 9.10) advocate the use of the n α derived in Theorem 9.3 (or conversely α n ) as a rule of thumb to appraise the reliability of asymptotic CIs for ratios of expectations.

Figure 9 . 6 :

 96 Figure 9.6: Expected wage conditional on belonging to top % as a function of τ .

Figure

  Figure 9.6 compares the confidence intervals constructed using Theorem 9.3 with the confidence

Figure 9 . 7 :

 97 Figure 9.7: Expected wage conditional on belonging to top % as a function of τ (zoom).

Figures 9 .

 9 Figures 9.7 and 9.8 zoom in subintervals of the previous graph in order to better compare the asymptotic confidence intervals from the delta method and the nonasymptotic CIs based on half-length t * from Theorem 9.6. When focusing on the top of the distribution, the asymptotic CIs happen to be broader than the minimal one (Figure9.7). On the contrary, with an expectation in the denominator further from 0, the length of the asymptotic CIs is lower than the minimal one (Figure9.8).

Figure 9 . 9 : 1 Theorem 9 . 7 .

 99197 Figure 9.9: Proportion of women within top % wage bracket

  Let us define W n := 1{θ Y,n + h Y,n / √ n = 0} and remark that W n = 1 whenever Y n = 0. By assumption P(Y n = 0) → 0, therefore W n d -----→ n→+∞ δ 0 . Moreover, the positive-definiteness of V and the boundedness of E[|X 1,n | 3 ] and E[|Y 1,n | 3 ] ensure that the Cramer-Wold device applies and(h Y,n , h X,n ) d -----→ n→+∞ N (0, V ). By Slutsky's Lemma, we also get(h Y,n , h X,n , W n ) d -----→ n→+∞ N (0, V ) ⊗ δ 0 .We can therefore apply the almost sure representation theorem, see[START_REF] Van Der | Asymptotic statistics[END_REF] Theorem 2.19]. It means that there exists a probability space ( Ω, Ũ, P), a sequence of random vectors ( hY,n , hX,n , Wn ) such that for every n ≥ 1, ( hY,n , hX,n , Wn ) d = (h Y,n , h X,n , W n ), and a random vector ( hY,∞ , hX,∞ , W∞ ) following the distribution N (0, V ) ⊗ δ 0 such that ( hY,n , hX,n , Wn ) a.s.

9. 8 . 2 2

 822 Proof of Theorem 9.We fix arbitrary n ∈ N * and ε ∈ R * + . By the triangle inequality first, then using the bound|X n | ≤ |X n -E [X 1,n ] | + |E [X1,n ] | and Assumptions 9.2.1 to 9.4.1 in the second inequality, we get:

First- 1 . 1 -P

 11 , using the union bound, note that the event A X1,n ε ∩ ÃY1,n ε holds with a probability bigger thanP A X1,n ε + P ÃY1,nε Hence, its complement is of probability lower than1 -P A X1,n ε + ÃY1,n ε .Second, we show that the event considered in Lemma 9.8 is included in the complement of A the proof. To do so, we reason by contraposition and do the following computations on the event A

ε

  , a lower bound on |x + E [Y 1,n ] | with x varying between 0 and Y

1 2( 1 -

 11 τ x/n) 2 ≤ 1 2(1-n -1 )2 , we get that under our assumptions 1 -x n n-1 ≥ exp -1 + 1 2n(1-n -1 ) 2

P-l 2 Y

 2 n,l Y,n ,u Y,n ,n } . Let (X i,n , Y i,n ) n i=1 i.i.d. ∼ P n := P n,u X,n ,η ⊗ P n,l Y,n ,u Y,n . Observe that E [Y 1,n ] = l Y,n and E Y 2 1,n = u Y,n . Furthermore, Y n ≤ l Y,n + u Y,n -l 2

9. 8 . 5 5 By 2 Y,n - 1 ,

 85521 Proof of Theorem 9.Lemma 9.10, for every ξ < 1 ∧ u Y,n /l there exists a distribution P n, ξ such that P Y n = 0 ≥ αn (ξ). Taking the supremum over ξ, we deduce that sup Pn∈P P Y n = 0 ≥ sup ξ αn (ξ) = α n .

Chapter 9 . 271 Lemma 9 . 10 . 2 Y,n > 1 ., 1 .

 9271910211 Confidence intervals for ratios of means: limitations of the delta method and honest confidence intervals For each ξ in the interval 0, 1 ∧ u Y,n /l 2 Y,n -1 , there exists a distribution P n, ξ ∈ P such that P Y n = 0 ≥ αn , where αn := 1 -(1 + ξ)l 2 Y,n /u Y,n n .Note that the interval 0, 1 ∧ u Y,n /l 2 Y,n -1 is not empty since we have assumed u Y,n /l Proof of Lemma 9.10: We consider the following distribution on RP n,l Y,n ,u Y,n ,c,ξ := c nwhere c ∈ (0, n) is some constant to be chosen later,y c-:= l Y,n (1 -√ ξ)/(1 -(c/n) 1/n ) and y c+ := l Y,n (1 + √ ξ)/(1 -(c/n) 1/n ). Let Y 1,n ∼ P n,l Y,n ,u Y,n ,c,ξn . Observe that E [Y 1,n ] = l Y,n and E Y 2 1,n = l 2 Y,n (1 + ξ n )/ 1 -(c/n) 1/n . With the choice c = c n := n 1we have E Y 2 1,n = u Y,n . Note that c n is strictly positive, because 1-l 2 Y,n u Y,n (1 + ξ n ) > 0. This is equivalent to u Y,n /l 2 Y,n > 1 + ξ n ,which is true by assumption.Consider now the following product measure on R 2 defined byP n := δ { √ u X,n } ⊗ P n,l Y,n ,u Y,n ,cn,ξ . Let (X i,n , Y i,n ) n i=1 i.i.d. ∼ P n . These random vectors satisfy E X 2 1,n = u X,n , E [Y 1,n ] = l Y,n and E Y 2 1,n = u Y,n. The next step is to build a lower bound on the event {Y n = 0}.The assumption that(X i,n , Y i,n ) n i=1 i.i.d.∼ P n and the construction of P n,l Y,n ,u Y,n ,cn,ξ imply thatP Y n = 0 = For every n ∈ N * , there exist finite constants a X,n , b X,n , a Y,n , b Y,n such that X 1,n (respectively Y 1,n ) lies P X,Y,n -almost surely in the interval [a X,n , b X,n ] (resp. [a Y,n , b Y,n ]).The support of X 1,n and Y 1,n is allowed to change with n, even though in many examples of interest, the former can be chosen independent from n. Assumptions 9.2.1 and 9.9.1 together correspond to the Hoeffding case because under these two assumptions, we can use the Hoeffding inequality to build nonasymptotic CIs.

9. 9 . 1 Concentration inequality in the easy case Assumption 9 . 9 . 2 .Theorem 9 . 11 .

 91992911 For every n ∈ N * , the lower bound a Y,n is strictly positive. Let u X,n := (b X,n -a X,n ) 2 and u Y,n := (b

Titre:

  Contributions à l'analyse statistique des modèles de dépendance en grande dimension Mots clés: Copules conditionnelles, statistique en grande dimension, distribution conditionnelle Résumé: Cette thèse peut être divisée en trois parties. Dans la première partie, nous étudions des méthodes d'adaptation au niveau de bruit dans le modèle de régression linéaire en grande dimension. Nous prouvons que deux estimateurs à racine carrée, peuvent atteindre les vitesses minimax d'estimation et de prédiction. Nous montrons qu'une version similaire construite à partir de médianes de moyennes, peut encore atteindre les mêmes vitesses optimales en plus d'être robuste vis-à-vis de l'éventuelle présence de données aberrantes. La seconde partie est consacrée à l'analyse de plusieurs modèles de dépendance conditionnelle. Nous proposons plusieurs tests de l'hypothèse simplificatrice qu'une copule conditionnelle est constante vis-à-vis de son évènement conditionnant, et nous prouvons la consistance d'une technique de ré-échantillonage semi-paramétrique. Si la copule conditionnelle n'est pas constante par rapport à sa variable conditionnante, alors elle peut être modélisée via son tau de Kendall conditionnel. Nous étudions donc l'estimation de ce paramètre de dépendance conditionnelle sous 3 approches différentes : les techniques à noyaux, les modèles de type régression et les algorithmes de classification. La dernière partie regroupe deux contributions dans le domaine de l'inférence. Nous comparons et proposons différents estimateurs de fonctionnelles conditionnelles régulières en utilisant des U-statistiques. Finalement, nous étudions la construction et les propriétés théoriques d'intervalles de confiance pour des ratios de moyennes sous différents choix d'hypothèses et de paradigmes. Title: Some statistical results in high-dimensional dependence modeling Keywords: Conditional copulas, high-dimensional statistics, conditional distribution

  2 are respectively increasing (by Lemma 2.4) and decreasing, so this ensures that the second part of condition (2.6) is satisfied for any s = 1, . . . , 2s * .We can reformulate Corollary 2.2 as follows: for any s = 1, . . . , 2s * and any γ ≥ 16 + 4

	√	2
	sup
	β * ∈B0(s)

* /(256γ 2 ). The functions b → (b/n) log(2p/b) and κ(•)

  2 * , except that we have a supplementary term -(1 + e 2 )e -n/24 . Similarly, we deduce from Corollary 2.2 that

	sup
	β * ∈B0(s)

  2 M +1 -(1 + e 2 )e -n/24 .

	Combining Equations (2.9), (2.11) with Theorem 2.3 and Lemmas 2.4 and 2.5, we obtain the following
	results for the case of the Square-root Lasso.

Corollary 2.6. Under the same assumptions as in Lemma 2.5, using Algorithm 2, with ( β(s) ) s=1,...,s * = ( βSQL (s,γ) ) s=1,...,s * , the distance d(β, β ) = ||X(β -β )|| n , and the weight w(b) = (b/n) log(p/b), we have that, for all β * ∈ B 0 (s), the aggregated estimator β satisfies

  and stopping criteria 0 > 0

	Initialize t ← 0 ;
	repeat
	Update t ← t + 1 ;
	/* Update of β and σ	*/
	Partition {1, . . . , n} into K blocks B 1 , . . . B K at random ;
	Find k ∈ {1, . . . , K} such that

  X is isotropic and ∀t ∈ R d , 1 ≤ p ≤ C 2 log(ed), 1 ≤ j ≤ d, ||X T e j || Lp ≤ C 3 √ p ||X T e j || L2 , 3. ∀t ∈ R d , ||X T t|| L2 ≤ θ 0 ||X T t|| L1 , and V ar ζX T t ≤ θ m ||X T t|| L2 ,

	Assumption 3.2.1. Denote by (e j ) j=1,...d the canonical basis of R d . We assume that there exist some
	finite constants C 1 , C 2 , C 3 , θ 0 , θ m such that
	1. |I| ≥ n/2 and |O| ≤ C 1 s log(ed/s),
	2.

where for p > 0 and for any function f : R p → R, ||f (X)|| Lp is defined, if it exists, as ||f (X)|| Lp(P

Theorem 3.1. Assume

  that β * is s-sparse for a given s ≤ d, and that n s log(ed/s).

	Under Assump-
	tion 3.2.1 and if there exist c, c such that Conditions (3.5)-(3.13) are satisfied for µ 0 = κ * 1/4 σ * , there
	exist constants c 1 , c 2 , c 3 , c 4 , c 5 , c 6 > 0 independent of s, d, n, σ * , κ such that, choosing the regularization
	parameter as

Definition 3.2. Let

  The subdifferential of || • || at any f ∈ F is denoted by (∂ || • ||) f := {z * ∈ E * : ||f + h|| ≥ ||f || + z * (h), ∀h ∈ E}, where (E * , || • || * ) is the dual normed space of (E, || • ||).

	Chapter 3. Robust-to-outliers simultaneous inference and noise level estimation using a MOM
	approach	61
	1.

F be a class of functions X → R. Let E be the vector space generated by F and || • || a norm on E, which will be used for regularization.

  (κρ K ) ≤ 2α 2,κ σ * = γ 1 . Combining the two equations, we get α 2,κ 2 + r 2 (κρ K ) ≤ 2α 2,κ σ * + γ 1 .4. Using Equation (3.6), we get r 2 (ρ K ) ≤ 2cθ 2 σ * 2 . Therefore 2 r 2 (ρ K )/σ * 2 ≤ 4, which implies the claimed result.

	Chapter 3. Robust-to-outliers simultaneous inference and noise level estimation using a MOM
	approach				65
	3. By Equation (3.5), we derive that α 2,κ ≤ 2σ * , which means that α 2,κ	2 ≤ 2α 2,κ σ * . Using Equa-
	tion (3.6), we get r 2
	1/2 /10	+	c /(cθ 0 ) µ 0	≤ 0.

  Plugging this back in Equation (3.19), we get

  same reasoning applies with T We have to stress that the first-level bootstrap, i.e. resampling among the conditioning variables X i,J , i = 1, . . . , n is surely necessary to obtain the latter result. Indeed, it can be seen that the key proposition 4.16 is no longer true otherwise, because the limiting covariance functions of the two corresponding processes G n and G * n will not be the same: see remark 4.22 below.

c ∞ and T c dist , for sufficiently regular copula families. Remark 4.15.

  .3 and 4.4. The three bootstrap schemes provide similar numerical results. Remind that the bootstrapped statistics is (T c 2 ) Chapter 4. About tests of the "simplifying" assumption for conditional copulas 105 the two distributions begin to match each other at n = 5000 whereas n = 500 is enough for the parametric independent bootstrap. We have tested the influence of τ max : the smaller is this parameter, the smaller is the percentage of rejections under the alternative, because the simulated model tends to induce lower dependencies of copula parameters w.r.t. X 3 : see Figures 4.1, 4.2, 4.3, and 4.4. Note that, on each of these figures, the point at the left corresponds to a conditional Kendall's tau which is constant, and equal to 0 (because τ max = 0) whereas the rejection percentages in Tables 4.1 and 4.3 correspond to a conditional Kendall'stau constant, and equal to 0.5. As the two data-generating process are not the same, the rejection percentages can differ even if both are under the null hypothesis. Nevertheless, in every case, our empirical sizes converge to 0.05 as the sample size increases. When n = 5000, we found that the percentage of rejections are between 4% and 6%.

* * with bootPI (Remark 4.8). Tests of H 0 under a parametric framework and through T c 2 confirm such observations. To evaluate the accuracy of the bootstrap approximations asymptotically, we have compared the empirical distribution of some test statistics and their bootstrap versions under the null hypothesis for two bootstrap schemes (see Figures 4.5 and 4.6). For the nonparametric bootstrap,

Table 4 .

 4

	Family	T 0 CvM,n (4.8)	T 0 CvM,n (4.10) I χ,n (4.13) I 2,n (4.15)
	Gaussian	0	0	0	0
	Student	0	0	0	0
	Clayton	0	0	1	0
	Gumbel	1	1	0	1
	Frank	0	0	0	0
	Family	T 0 CvM,n (4.8)	T 0 CvM,n (4.10) I χ,n (4.13) I 2,n (4.15)
	Gaussian	98	100	100	93
	Student	100	99	98	90
	Clayton	99	99	99	98
	Gumbel	99	98	100	95
	Frank	98	100	98	50
	Table 4.2: Rejection percentages under the alternative (nonparametric tests, nonparametric bootstrap
	bootNP).

This corresponds to changing the integrals (resp. max) on [0, 1] to integrals (resp. max) on [h, 1 -h]. 1: Rejection percentages under the null (nonparametric tests, nonparametric bootstrap bootNP).

Table 4 .

 4 3: Rejection percentages under the null (parametric tests).

	Family	T c 2 (4.18) bootPI bootPC bootNP bootPI bootPC bootNP T c 2 (4.30)
	Gaussian	100	100	100	100	100	100
	Student	100	100	100	100	100	100
	Clayton	100	62	98	100	98	100
	Gumbel	100	100	34	100	99	76
	Frank	100	100	100	100	100	100

Table 4 .

 4

4: Rejection percentages under the alternative (parametric tests).

Table 4 .

 4

	5: Table of notation

I KS,n

(4.14)

test statistic based on the distance between the joint empirical cdf of ( ẐI|J , X J ) and the product of their empirical cdf, using the L ∞ norm (resp. I 2,n (4.15)) (resp. using the L 2 norm) (resp. I CvM,n (4.16)) (resp. using the L 2 norm, weighted by the joint empirical cdf as weight) T c ∞ (4.18) test statistic based on the L ∞ distance between the parameter of the conditional copula and the constant parameter of the partial copula (resp. T c 2 (4.18)) (resp. L 2 distance) (resp. T c dist (4.19)) (resp. using some distance between the estimated copulas) (resp. T c dens (4.20)) (resp. using the L 2 distance between the estimated copula densities) T dist,n (4.29) brute-force test statistic of H 0 constructed with the distance dist(•, •) between all pairs of conditional copulas with Borelian subsets (resp. T KS,n (4.27)) (resp. with the L ∞ distance) (resp. T CvM,n (4.28)) (resp. with the L 2 distance)

Table 4 .

 4

6: Table of main test statistics Chapter 4. About tests of the "simplifying" assumption for conditional copulas 114 4.

7 Proof of Theorem 4.14 4.7.1 Preliminaires Let

  (Z i ) i=1,...,n be a sequence of i.i.d random vectors in [0, 1] p , Z i being drawn from the true cdf C θ0 .

  , p.346: instead of drawing n times the initial observations, this is done N n times, where N n follows a Poisson distribution with mean n and N n is independent of the initial sample. Then, the n variables M Nn,1 , . . . , M Nn,n are i.i.d. Poisson random variables with mean one. And we can build the new process as

  Proposition 5.4 (Alternative exponential bound without explicit constants). Under Assumptions 5.3.1-5.3.4, for every t > 0 such that C K,α h α /α!+t ≤ f Z,min /2 and every t > 0 s.t. t ≤ 2h p ( K 2 ) 3 f 3 Z,max /C 4 K , there exist some universal constants C 2 and α 2 s.t.

  and 6h p |K| 2 f z,max < K 2 .

Remark 5.5. In

Propositions 5.2, 5.3 and 5.4

, when the support of K is included in [-c, c] p for some c > 0, f Z,max can be replaced by a local bound sup z∈V (z, ) f Z (z), denoting by V(z, ) a closed ball of center z and any radius > 0, when h c < . Moreover, if it is not guaranteed that IE[K h (Z-z)] is positive, the results above apply, replacing 2/3 by 4/3 in the denominators of inequality in

Proposition 5.3.

  3, and then of τ1,2|Z=z , under the weaker condition nh p → ∞. (Consistency).Under Assumption 5.3.1, if nh p n → ∞, lim K(t)|t| p = 0 when |t| → ∞, f Z and z → τ 1,2|Z=z are continuous on Z, then τ (k) 1,2|Z=z tends to τ 1,2|Z=z in probability, when n → ∞ for any k = 1, 2, 3.This property is proved in Section 5.5.6. Moreover, Proposition 5.6 does not allow to state the strong uniform consistency of τ (k) 1,2|Z=z because the threshold t has to be of order h p at most. Here again, a direct approach is possible. (Uniform consistency). Under Assumption 5.3.1, assume that nh 2p n / log n → ∞, lim K(t)|t| p = 0 when |t| → ∞, K is Lipschitz, f Z and z → τ 1,2|Z=z are continuous on a bounded set Z, and there exists a lower bound f

	Proposition 5.7 Proposition 5.8

Table 5 .

 5 1: Results of the simulation in Setting 1. All values have been multiplied by 1000. Bold values indicate optimal choices for the chosen measure of performance. Chapter 5. About kernel-based estimation of conditional Kendall's tau: finite-distance bounds and asymptotic behavior 136

	3) 1,2|Z=z -1,

Table 5 .

 5 2: Results of the simulation in Setting 2. All values have been multiplied by 1000. Bold values indicate optimal choices for the chosen measure of performance. Chapter 5. About kernel-based estimation of conditional Kendall's tau: finite-distance bounds and asymptotic behavior 138

		1,2|Z=z	-207 227	180	-54.1 83.9 16.9	-29.6	55.3 5.81	-16.9 38.9 2.49
		τ (2) 1,2|Z=z	1.15	207	97	0.845 80.5 10.8	0.557	54.4 4.35 0.145 38.6 2.04
		τ (3) 1,2|Z=z	210	228	181	55.7 83.2 16.4	30.7	55.4	5.9	17.2 38.9	2.5
		τ (4) 1,2|Z=z	1.4	225	51.9 0.987 81.4 6.86	0.456	55	3.22 0.175 38.9 1.66
	α h = 0.75	τ (1) 1,2|Z=z τ (2) 1,2|Z=z τ (3) 1,2|Z=z	-144 175 -2.33 163 140 176	98.6 56.2 99.2	-33.3 60.6 1.73 59.4 5.56 -0.0619 41.7 2.51 0.665 30.4 1.24 7.5 -19.8 41.9 3.12 -10.6 30.5 1.42 36.8 60.7 7.73 19.7 42.1 3.12 11.9 30.5 1.45
		τ (4) 1,2|Z=z	-3.15 170	30.3	1.69 60.2 3.85	-0.093 42.1 1.95 0.645 30.5 1.05
		τ (1) 1,2|Z=z	-99.8 143	57.7	-24.9 50.9 5.06	-13.5	36.6 2.28	-6.92 26.6 1.09
	α h = 1	τ (2) 1,2|Z=z τ (3) 1,2|Z=z	1.17 102	132 139	34.6 0.903 50.4 4.02 54.4 26.7 51 5.13	1.16 15.8	36.5 1.97 36.6 2.33	1.46 26.6 0.994 9.83 26.6 1.11
		τ (4) 1,2|Z=z	2.51	138	20.1 0.897 50.9 2.89	1.16	36.7 1.56	1.48 26.7 0.847
		τ (1) 1,2|Z=z	-59.1 104	28.1	-14.7 42.3 3.87	-7.56	29.7 1.86	-4.17 21.8 0.932
	α h = 1.5	τ (2) 1,2|Z=z τ (3) 1,2|Z=z	4.34 99.7 21.4 67.8 103 29.6	2.05 42.1 3.48 18.8 42.3 3.96	2.07 11.7	29.6 1.75 29.6 1.92	1.35 21.8 0.899 6.87 21.8 0.957
		τ (4) 1,2|Z=z	3.34	103	13.4	2.08 42.5	2.6	2.08	29.7 1.39	1.35 21.8 0.755
		τ (1) 1,2|Z=z	-37.2 88.2 23.9	-9.57 38.2	4.6	-3.75	26.2 2.34	-1.09 19.8	1.32
	α h = 2	τ (2) 1,2|Z=z τ (3) 1,2|Z=z	8.17 85.9 53.5 87.4 25.3 21.2	2.69 14.9 38.1 4.74 38 4.45	3.32 10.4	26.1 26.2 2.41 2.3	2.99 7.08	19.8 19.8	1.32 1.36
		τ (4) 1,2|Z=z	8.47 88.5	15	2.69 38.4 3.59	3.33	26.3 1.93	3	19.9 1.15

Table 5 . 3

 53

		1,2|Z=z	-111	154	66.2	-36.9 66.8 9.01	-22.4 48.2 4.06 -12.9 36.1 2.04
		τ (2) 1,2|Z=z	0.0488 137	36.3 0.236 64.2 6.45 0.546 46.8 3.14	1.29 35.7 1.78
		τ (3) 1,2|Z=z	111	151	60.6	37.4 66.3 8.88	23.5 47.2 4.07	15.5 36.2 2.18
		τ (4) 1,2|Z=z	1.38	132	18.3	0.27 64.5 4.49	0.61 46.8 2.36	1.29 35.6 1.49
	α h = 0.75	τ (1) 1,2|Z=z τ (2) 1,2|Z=z τ (3) 1,2|Z=z	-67.4 4.32 76.1	117 108 119	35.7 23.5 0.809 50.7 4.21 -23.3 52.1 5.27 35.4 24.9 51.6 5.12	-13.9 37.8 1.03 37.2 2.07 2.4 16 37.6 2.49	-7.6 1.78 28.8 1.21 29 1.3 11.2 29.1 1.39
		τ (4) 1,2|Z=z	4.98	106	13.3	0.86 51.6 3.13	1.03 37.5 1.63	1.81 28.9 1.02
		τ (1) 1,2|Z=z	-43	101	28	-15.8 45.7 4.44	-9.51 33.1 2.04 -4.68 25.1 1.07
	α h = 1	τ (2) 1,2|Z=z τ (3) 1,2|Z=z	7.87 58.8	93.1 22.4 97.6 27.2	2.01 44.8 3.91 19.8 45.3 4.41	1.57 32.7 1.87 12.7 32.9 2.1	2.29 24.9 1.03 9.27 25.1 1.14
		τ (4) 1,2|Z=z	8.51	98	15.7	2.05	46	3.01	1.57 33.1	1.5	2.33 25.1 0.871
		τ (1) 1,2|Z=z	-16.1	95.6 41.7	-6.36	43	6.35	-4.04 30.6 2.87 -1.11 22.1 1.34
	α h = 1.5	τ (2) 1,2|Z=z τ (3) 1,2|Z=z	14.9 46	92.6 40.4 92.8 42.2	5.08 42.6 16.5 42.6 6.45 6.2	3.17 30.4 2.83 10.4 30.4 2.94	3.47 8.06 22.1 22	1.34 1.4
		τ (4) 1,2|Z=z	15.6	100	35.2	5.11	44	5.31	3.17	31	2.45	3.5	22.4 1.17
					n		100	500	1000 2000
					IE[h CV ]	0.77 0.43	0.34	0.27
					Sd[h CV ]	0.17 0.091 0.060 0.057
					h ref = n -1/5 0.40 0.29	0.25	0.22

: Results of the simulation in Setting 2 using h = α h × h CV where h CV has been chosen by cross-validation. All values have been multiplied by 1000. Bold values indicate optimal choices for the chosen measure of performance.

Table 5 .

 5

	Chapter 5. About kernel-based estimation of conditional Kendall's tau: finite-distance bounds and
	asymptotic behavior	140
	and then	τ (1) 1,2|Z=z =	τ (3) 1,2|Z=z -2s n . Moreover,

4: Expectation and standard deviation of the bandwidth selected by cross-validation as a function of the sample size n, and comparison with bandwidth h ref chosen by the rule-of-thumb.

Proposition 5.2 Lemma

5.12. Under Assumptions 5.3.1, 5.3.2 and 5.3.3, we have for any t > 0,

  5. About kernel-based estimation of conditional Kendall's tau: finite-distance bounds and asymptotic behavior 146 Choose ε s.t. CXZ,2 h 2 /2 + ε = f z,max /2. Then, sup z |∆ n (z)| will be smaller than 3f z,max K 2 /(2nh p )

Table 6 .

 6

	6. About Kendall's regression

2: Comparison of the performance between the two estimators. Integrated measures have been multiplied by 10 3 , for readability.

Table 6 .

 6 .3. Globally, the test based on W n performs very well under all settings, compared to the alternative nonparametric test. It is only beaten by T c 2 that is obtained by choosing the right copula family, a not very realistic situation. When it is not the case, W n does a better work.

			Not under H 0		Under H 0
		1	2	3	4	5	6
	W n	88.7 99.8 87.3 100 12 12.1
	T 0 CvM	59.5	52	64.7 37.5	0	0
	T c 2	100 100 100 100 0.2 2.6
	T c 2 (Clayton)	68	13	100 100 1.8 1.8

3: Comparison of the performance between different tests of the simplifying assumption under the six settings of Section 6.4.3, with n = 500.

Table 6 . 4

 64

			Setting 1			Setting 2			Setting 3
		IBias	ISd IMSE Time	IBias	ISd IMSE Time IBias ISd IMSE Time
	ψ (1)	0.577	19.4 1.44	6.82 -0.632	24	1.4	6.75 -7.71	17	6.79	6.67
	ψ (2)	0.309	18.9 1.43	6.77 -0.166 23.7 1.35	6.66 -7.57 16.9	6.8	6.66
	ψ (3)	0.728	19.9 1.63	6.77	-0.36	27.1	1.9	6.67 -7.63 23.7 3.45	7.06
	ψ (4)	0.513	18.9 1.81	6.77 -0.245 26.5 2.22	6.68 -7.29	25	2.06	7.52
	ψ (5)	1.5	25.7 15.7	6.77 0.0616	15	2.67	6.66 -8.38 21.6 14.9	7.51
	ψ (6)	1.64	26	15.7	6.79	0.269	15	2.61	6.66 -8.23 21.9 14.9	7.52
	ψ (7)	0.311	26.1	17	6.79 0.0167	15	3.14	6.69 -7.33 23.1 15.1	7.26
	ψ (8)	1.2	26	17.3	6.88 -0.113 14.6 3.15	6.7	-7.6 22.9 15.3	7.2
	ψ (9)	0.596	17.7 2.05	6.79	0.492 15.8 2.72	6.67 -7.93 16.3 7.04	7.19
	ψ (10) -0.0921	18	2.08	6.77 -0.493 16.6 2.75	6.66 -7.65 16.7 6.94	7.19
	ψ (11)	0.529	17.3 2.57	6.83 -0.165 15.8 3.08	6.7	-6.87	23	4.76	7.21
	ψ (12)	0.5	16.9 2.64	6.92 -0.078 16.4 3.24	6.76 -7.07 25.5 4.43	7.54

2

, while IM SE = M SE(z)dz, IBias = Bias(z)dz and ISd = Sd(z)dz. Therefore, a procedure : Comparison of the estimation using different ψ families. All integrated measures have been multiplied by 1000. Computation time is given in seconds.

Table 7 .

 7 1: Error criterion (7.10) for each choice of ψ and each method, multiplied by 1000.

	's tau	200

Table 7 .

 7 3: Error criterion (7.10) for each copula family and each method, multiplied by 1000.

		Logit	Probit	Tree	Random	Nearest	Neural
					forests	neighbors	network
	Gaussian	0.456	0.434	4.57	3.15	2.26	0.561
	Student 4 df	0.549	0.515	4.54	3.28	2.87	0.753
	Student (2 + 1/z) df 0.531	0.518	4.66	3.23	2.82	0.805
	Clayton	0.498	0.472	4.52	3.36	2.67	0.742
	Gumbel	0.45	0.431	4.56	3.23	2.66	0.775
	Frank	0.448	0.42	4.5	3.28	2.13	0.615

Table 7

 7

	Method	Logit	Probit	Tree	Random	Nearest	Neural
					forests	neighbors	network
	Algorithm Algorithm 8 Algorithm 8 tree() of [117] Algorithm 10 Algorithms 11-12 Algorithm 13
	Color	orange	red	blue	green	purple	black

.2 below). The black dash-dotted curve is the true conditional Kendall's tau that has been used in the simulation experiment.

Table 7 .

 7 2: Summary of available estimation methods for the estimation of the conditional Kendall's tau and corresponding algorithm and curve color.For each estimator, we state in the second line of the above table the algorithm used to compute it, and in the third line the color of the corresponding curve on Figures 7.1 to 7.13. For example, the estimator "probit" is computed using Algorithm 8 and corresponds to the red curves. IP X1|Z=z = N (0, z 2 ) ; IP X2|Z=z = U [0,|z|] . Then, we will see how estimation is affected by changes in the conditional variance of (X 1 , X 2 ) given Z = z.

	Chapter 7. A classification point-of-view on conditional Kendall's tau	202
	4. Setting	Logit	Probit	Tree	Random forests	Nearest neighbors	Neural network
	1.	0.456	0.434	4.57	3.15	2.26	0.561
	2.	0.809	0.818	4.65	3.72	2.65	0.838
	3.	1.15	1.12	5.29	4.21	3.57	1.32
	4.	0.493	0.471	4.43	3.44	2.54	0.662

Table 7 .

 7 4: Error criterion (7.10) for each choice of conditional margins and each method, multiplied by 1000.

Table 7 .

 7 .8 for a quantitative comparison of the performance of such methods in terms of computation time as a function of the sample size n.

	Setting	Logit	Probit	Tree	Random forests	Nearest neighbors	Neural network
	f 1	11.2	11.6	4.12	4.03	3.89	1.48
	f 2	0.456	0.434	4.57	3.15	2.26	0.561
	f 3	3.77	3.22	5.95	4.76	2.35	2.17
	f 4	12.8	12.8	16.8	10	3.71	1.97

5: Error criterion (7.10) for different Kendall's tau models and each estimation method, multiplied by 1000.

  ity. The results are displayed on Figures 7.2 to 7.7. Intuitively, dependence should tend to increase with , X 4 ) given σ during the After-crisis period Chapter 7. A classification point-of-view on conditional Kendall's tau 208 market volatility: when "bad news" are announced, they are source of stress for most dealers, especially inside the Eurozone that brings together economically connected countries. This phenomenon should be particularly sensitive during the European debt crisis, because a lot of such "bad news" were related to the Eurozone itself (economic/financial news of public debts in several European countries). Let us see whether this is the case.

	Chapter 7. A classification point-of-view on conditional Kendall's tau			207
		0.90					0.90
		0.85					0.85
	Conditional Kendall's tau	0.70 0.75 0.80				Conditional Kendall's tau	0.70 0.75 0.80
		0.65					0.65
		0.60					0.60
		0.01	0.02	0.03	0.04		0.005	0.010	0.015	0.020	0.025	0.030
			Z						Z
	Figure 7.2: Conditional Kendall's tau between	Figure 7.3: Conditional Kendall's tau between
	(X 1 , X 2 ) given σ during the European debt crisis	(X 1 , X 2 ) given σ during the After-crisis period
		0.7					0.7
	Conditional Kendall's tau	0.4 0.5 0.6				Conditional Kendall's tau	0.4 0.5 0.6
		0.3					0.3
		0.01	0.02	0.03	0.04		0.005	0.010	0.015	0.020	0.025	0.030
			Z						Z
	Figure 7.4: Conditional Kendall's tau between	Figure 7.5: Conditional Kendall's tau between
	(X 1 , X 3 ) given σ during the European debt crisis	(X 1 , X 3 ) given σ during the After-crisis period
		0.5					0.5
	Conditional Kendall's tau	0.2 0.3 0.4				Conditional Kendall's tau	0.2 0.3 0.4
		0.1					0.1
		0.01	0.02	0.03	0.04		0.005	0.010	0.015	0.020	0.025	0.030
			Z						Z
	Figure 7.6: Conditional Kendall's tau between	Figure 7.7: Conditional Kendall's tau between
	(X 1 , X 4 ) given σ during the European debt crisis	(X 1

  , X 4 ) given ∆σ I during the After-crisis period linked to variations of the conditional dependence between stock returns of different countries. Figures 7.8 to 7.13 illustrate the variations of the conditional Kendall's tau between couples of stock returns with respect to the conditioning variable ∆σ I during the two periods we study.

			7. A classification point-of-view on conditional Kendall's tau	209
		0.80						0.80
	Conditional Kendall's tau	0.65 0.70 0.75					Conditional Kendall's tau	0.65 0.70 0.75
		-4	-2	0	2	4
								-3	-2	-1	0	1	2	3
				Z
								Z
	Figure 7.8: Conditional Kendall's tau between (X 1 , X 2 ) given ∆σ I during the European debt cri-sis	Figure 7.9: Conditional Kendall's tau between (X 1 , X 2 ) given ∆σ I during the After-crisis period
		0.50						0.50
	Conditional Kendall's tau	0.35 0.40 0.45					Conditional Kendall's tau	0.35 0.40 0.45
		0.30						0.30
		-4	-2	0	2	4
								-3	-2	-1	0	1	2	3
				Z
								Z
	Figure 7.10: Conditional Kendall's tau between (X 1 , X 3 ) given ∆σ I during the European debt cri-sis	Figure 7.11: Conditional Kendall's tau between (X 1 , X 3 ) given ∆σ I during the After-crisis period
	Conditional Kendall's tau	0.1 0.2 0.3					Conditional Kendall's tau	0.3 0.1 0.2
		0.0						0.0
		-4	-2	0	2	4
								-3	-2	-1	0	1	2	3
				Z
								Z
	Figure 7.12: Conditional Kendall's tau between (X 1 , X 4 ) given ∆σ I during the European debt cri-sis	Figure 7.13: Conditional Kendall's tau between (X 1

  . They are rather straightforward extensions of the latter concepts, when dealing with conditional distributions. Formally, for a given sigma-algebra F, let H(•|F) (resp. F k (•|F)) be the conditional distribution of X (resp. X k , k = 1, . . . , d) given F. The "conditional version of" Sklar's theorem now states that there exists a random copula C(•|F) s.t.H(x 1 , . . . , x d |F) = C F 1 (x 1 |F), . . . ,F d (x d |F)|F , a.e. (7.12) for every (x 1 , . . . , x d ) ∈ R d . If the law of H(•|F) is continuous, the latter C(•|F) is unique, and it is called the conditional copula associated to H(•|F), given F. Inversely, given F, a conditional copula C(•|F) and some univariate cdfs' F k (•|F), k = 1, . . . , d, Equation (7.12) defines a d-dimensional conditional cdf H(•|F). See Fermanian and Wegkamp

  is convergent in probability. We only have to replace β (•, •) by its second derivatives w.r.t. β. To save space, the specific derivations of such conditions of regularity are left to the reader: simply replace the functions A, B,...,F of 7.8 by their second derivatives w.r.t. β, taken at β = β * , and rewrite (7.14) ] is bounded from above, uniformly w.r.t. β in a small neighborhood of β * . By derivation, we get, for every indices a, b, c in {1, . . . , p },

	and (7.15).
	Third, to prove (iii), it is sufficient to state that IE[	... Ln(β)

  .[START_REF] Bickel | Simultaneous analysis of Lasso and Dantzig selector[END_REF]

	Remark 7.7. Note that ||p(•, •)|| ∞ ≤ 1. If g and its derivatives are bounded, Condition (7.19) is satisfied
	if
	sup	sup
	β∈V (β * )	δ∈{-1,1}

  The parameter β * is identifiable in Model (8.2) if and only if the functions (ψ 1 (•), . . . , ψ r (•)) are linearly independent IP ⊗n Z -almost everywhere in the sense that, for all vectors t = (t 1 , . . . , t r ) ∈ R r , IP ⊗n

	The only
	condition is their linear independence, as seen in the following proposition (whose straightforward proof
	is omitted).
	Proposition 8.1. Z

  (Exponential bound for the estimator θ(z 1 , . . . , z k ), with fixed z 1 , . . . z k ∈ Z k ). such that C K,α h α /α! + t < f Z,min /2, we have

	The following proposition is proved
	in Section 8.6.2.
	Proposition 8.5 Assume
	either Assumption 8.2.7 or the weaker Assumption 8.2.8. Under Assumptions 8.2.1-8.2.6, for every
	t, t > 0

  min ; in the case of Assumption 8.2.8, the result holds with the following alternative values: C6

Assumption 8.3.1.

  The function z → ψ(z) are bounded on Z by a constant C ψ . Moreover, Λ(•) is continuously differentiable. Let T be the range of θ, from Z k towards R. On an open neighborhood of T , the derivative of Λ(•) is bounded by a constant C Λ .

	The following theorem is proved in Section 8.6.3.
	Theorem 8.8. Assume either Assumption 8.2.7 or the weaker Assumption 8.2.8. Suppose that Assump-
	tions 8.2.1-8.2.6 and 8.3.1 hold and that the design matrix Z satisfies the RE(s, 3) condition. Choose
	the tuning parameter as λ = γt, with γ ≥ 4 and t > 0, and assume that we choose h small enough such
	that
		h ≤ min	f Z,min α! 4 C K,α	1/α	,	t 2C 5 C 8	1/(k+α)	,	(8.7)
	where C 8 := C ψ C Λ 1 + C 4 f Z,min /2 . Then, we have
	IP ||Z ( β -β * )|| ≤	4(γ + 1)t κ(s, 3) √	s

  Theorem 8.11 (Asymptotic law of the estimator). Under Assumption 8.2.10, and if λ n,n (nh p n,n ) 1/2 tends to when n → ∞, we have (nh p n,n ) 1/2 ( βn,n -β * ) := arg min u∈R r F ∞,n (u), given z 1 , . . . , z n , where

	2 h kp 8 C 6,σ + 2C 8 C 7,σ t 4C 2	.	(8.8)

D

-→ u *

  (Consistency of βn,n , jointly in (n, n )). Assume that Assumptions 8.2.1-8.3.1 are satisfied. Assume that σ∈I k,n ψ z σ(1) , . . . , z σ(k) ψ z σ(1) , . . . , z σ(k)

T

/n converges to a matrix M ψ,z , as n → ∞. Assume that λ n,n → λ 0 and n exp(-Anh kp ) → 0 for every A > 0, when (n, n ) → ∞. Then βn,n P

  x 2 , . . . , x k g x 1 , y 2 , . . . , y k Λ 2 θ(z 1 , . . . , z k ) ψ z 1 , . . . , z k ψ z 1 , . . . , z k X|Z=z 1 (x 1 ) dµ(x 1 )dµ(z 1 ) X|Z=z i (y i )f X|Z=z i (x i )f z ,∞ (z i ) dµ(x i )dµ(y i )dz i .

	k
	i=2 f Theorem 8.14 (Asymptotic law of βn,n , jointly in (n, n )). Under Assumptions 8.2.1-8.2.5 and under
	Assumption 8.3.2, we have

T

× f

  Therefore, we can apply Lemma 8.3. We compute the constantsC 1 := 2f k Z,max ||K|| 2k 2 = 2 × 0.59 2 × (3/5) 2 ≤ 0.26 and C 2

		36 2	|u|K(u)du	2 ≤ 0.2.
	1/	Assumption 8.2.2 is then satisfied with C K,α = 0.2. Assumption 8.2.3 is easily satisfied with f Z,max = √ 2π(1 -2Φ(-1)) ≤ 0.59.

  the bias term σ∈I k,n IE[S σ ] and the stochastic component σ∈I k,n S σ -IE[S σ ] .

	Lemma 8.16 (Bound for	k i=1 f Z (z i )/N k ). Under Assumptions 8.2.1, 8.2.2, 8.2.3, and 8.2.4 and if for
	some

  Now, we provide a bound on the bias.

	Lemma 8.17. Under Assumptions 8.2.1 and 8.2.6, we have IE

  bounded, we can derive an immediate bound for this stochastic component. Indeed, we would have ||g|| ∞

  7.1. It will help to control the first term of Equation (8.15), which is Concerning the second term of Equation (8.15), using Assumption 8.3.2(iii), we have for every u

	simply -2T T 1 u.

  Chapter 8. Estimation of a regular conditional functional by conditional U-statistic regression 244 Afterwards, we will prove that all the remainders terms T 6 , T 7 and T 3 are negligible, i.e. they tend to zero in probability. These results are respectively proved in Subsections 8.7.3, 8.7.4 and 8.7.5. Combining all these elements with the asymptotic normality of T 4 (proved in Subsection 8.7.2), we get T 1

	D -→ N (0, V 2 ),
	as claimed.
	8.7.2 Proof of the asymptotic normality of T 4

).

  h kα-p ) = O(h kα ) = o(1). Similarly, we get IE T 6,2 = o(1) by a Taylor expansion.

		6,2 =
	O(	r2 n,n
	8.7.4 Convergence of T 7 to 0
	We recall Equation (8.22):

nn

  Assumption 9.2.2.There exist finite constants u X,n , u Y,n such that the second moments of X 1,n andY 1,n are bounded, i.e. E X 2 1,n ≤ u X,n , and E Y 2 1,n ≤ u Y,n .This framework, where Assumptions 9.2.1 and 9.2.2 hold, will be denoted as the BC case since it is possible under this assumption to construct CIs using the Bienaymé-Chebyshev inequality. In Appendix 9.9, we present an adapted version of all our results under the assumption that X 1,n and Y 1,n have a bounded support instead of Assumption 9.2.2. This corresponds to what we will call the "Hoeffding case" because under such assumptions, we can use the Hoeffding inequality to build nonasymptotic confidence intervals with exponential speed of convergence.Our objective is to make inference on the ratio of expectations E [X 1,n ] /E [Y 1,n ] by constructing confidence intervals. To sum up, Assumptions 9.2.1 and 9.2.2 define a set P of distributions for some known constants l Y,n , u X,n and u Y,n . For a distribution P X,Y,n in P, the parameter of interest is θ

Table 9 .

 9 1: Limiting law of S n := √ n X n /Y n -E [X 1,n ] /E [Y 1,n ] in the nine different regimes. The variables (W 1 , W 2

  1,n ] and (ii) Y n is both close to E [Y 1,n ] and bounded away from 0. Under Assumptions 9.2.1 and 9.2.2, the Bienaymé-Chebyshev inequality can be applied to obtain (i) and (ii). Without extra assumptions, we are only able to build nonasymptotic CIs at confidence levels that are not too close to 1 (see Section 9.4.2

9.4.2 Nonasymptotic confidence intervals with no assumption on the support of PY

  We seek to build nontrivial nonasymptotic CIs under Assumptions 9.2.1 and 9.2.2 only. Under Assumption 9.2.1, E [Y 1,n ] = 0, so that there is no issue in considering the fraction E [X 1,n ] /E [Y 1,n ]. However, without Assumption 9.4.1, Y n = 0 has positive probability in general so that X n /Y n is well-defined with probability less than one. Note that when P Y,n is continuous wrt to Lebesgue's measure, there is no issue in defining X n /Y n anymore since the event Y n = 0 has probability zero. This is not an easier case from a theoretical point-of-view though, since without more restrictions, Y n can still be arbitrarily close to zero with positive probability. Assume that Assumptions 9.2.1 and 9.2.2 hold. For every n ∈ N * , ε > 0, ε ∈ (0, 1), we As a consequence,P X n /Y n -E [X 1,n ] /E [Y 1,n ] > t ≤ α, with the choice √ u X,n + 2u X,n /nα 2(u Y,n -l 2 Y,n )/nαl 2

	Theorem 9.3. have
	P		X n Y n	-	E [X 1,n ] E [Y 1,n ]	>	(	√ u X,n + ε)ε (1 -ε) 2	+ ε	1 l Y,n	≤	u X,n nε 2 +	u Y,n -l 2 Y,n Y,n nε 2 l 2	.
														
	t =	1 l Y,n	 		1 -2(u Y,n -l 2 Y,n )/nαl 2 Y,n	2	Y,n	+	2u X,n nα	  ,
	for every α > α n :=	2(u Y,n -l 2 Y,n ) nl 2 Y,n

  [Proposition 6.2] to ratios. It is proved in Section 9.8.4. For every integer n ≥ 7, every α ∈ 0, 1 ∧ n/ l Y,n + u Y,n -l 2 there exists a distribution Q on R 2 that satisfies Assumptions 9.2.1 and 9.2.2 such that for(X i,n , Y i,n )

		2
	Theorem 9.6. Y,n	, and every
	ξ < 1 n	i.i.d
	i=1

∼ Q, we have

  9. Confidence intervals for ratios of means: limitations of the delta method and honest confidence intervals 264 the ones from the delta method. Both CIs get narrower as the expectation in the denominator moves away from 0. Figure 9.6 also displays a plug-in estimate of nonasymptotic CIs based on the minimal half-length t * presented in Theorem 9.6. As a reminder, no nonasymptotic CI of the form X n /Y n ± t can have good coverage if t < t * . On Figure 9.6, we cannot distinguish CIs based on the delta method from nonasymptotic CIs with minimal half-length (see details below in Figures 9.7 and 9.8).

	Expected wage conditional on belonging to top %	3500 4000 4500 5000				estimate asymp. CI by DM nonasymp. CI by BC nonasymp. minimal CI
		11%	12%	13%	14%	15%
				Top %

Enquête Emploi en continu (version FPR)" -year, INSEE, ADISP, for year going through 2010 to 2017.

Chapter

  Figure 9.8: Expected wage conditional on belonging to top % (zoom)

		2450
	Expected wage conditional on belonging to top %	2250 2300 2350 2400						estimate asymp. CI by DM nonasymp. CI by BC nonasymp. minimal CI
		61%	62%		63%	64%		65%
					Top %
	Proportion of women within top % wage bracket	0.0 0.2 0.4						estimate asymp. CI by DM nonasymp. CI by BC nonasymp. minimal CI
		5%	20%	35%	50%	65%	80%	95%
					Top %

  2 h X,n /θ Y,n -h Y,n θ X,n /θ 2 Y,n , as n → ∞. 2. If θ Y,n = o(n -1/2 ), then A n is equivalent to h X,n /h Y,n -θ X,n /θ Y,n .3. If there exists a finite constantC = 0 such that √ nθ Y,n → C as n → ∞, then A n is equivalent to √ nθ X,n 1/(C + h Y,n ) -1/C + h X,n /(C + h Y,n ).

  Chapter 9. Confidence intervals for ratios of means: limitations of the delta method and honest confidence intervals 267 as claimed.and the result follows from the fact that √ n/ hY,n is negligible compared to 1/θ Y,n .

	Second case: We have
	Ãn = =	θ X,n + hX,n / √ √ n n θ Y,n + hY,n / √ nθ X,n + hX,n hY,n --θ X,n θ X,n θ Y,n θ Y,n + o =	θ X,n + hX,n / √ hY,n + o(1) / √ n n √ nθ X,n + 1	-	θ X,n θ Y,n
	∼ θ X,n	√ hY,n n	-	1 θ Y,n	+	hX,n hY,n	,
	Third case: We have
	Ãn ∼	θ X,n + hX,n / √ C/ √ n + hY,n / √ n n	-	θ X,n C/ √ n	=	√	nθ X,n + hX,n C + h Y,n	-	√	nθ X,n C	.
			=	n θ Y,n θ X,n + hX,n / = √ θ X,n + hX,n / √ √ n n θ Y,n + hY,n / n θ Y,n 1 -hY,n √ nθ Y,n + O ( θ X,n -θ Y,n √ nθ Y,n ) -2 -	θ X,n θ Y,n
			∼	-θ X,n hY,n √ nθ 2 Y,n	+	hX,n √ nθ Y,n	,

  If both |X n -E [X 1,n ] | and |Y n -E [Y 1,n ]| are inferior or equal to ε, event (9.2) cannot happen. By contraposition, we obtain:

	n ε a Y,n l Y,n	+	ε l Y,n	.	(9.2)

  1,n ] . Chapter 9. Confidence intervals for ratios of means: limitations of the delta method and honest confidence intervals 269 Using Jensen's inequality and Assumption 9.2.2, we have |E [X 1,n ] | ≤ (u X,n ) 1/2 , and Assumption 9.2.1 entails 1/|E [Y 1,n ] | ≤ 1/l Y,n . Consequently, we get

  1,n ] = E X 2 1,n -(E [X 1,n ]) 2 ≤ E X 2 1,n ≤ u X,n using Assumption 9.2.2. For the denominator, Assumption 9.2.1 immediately entails that 1/l 2 Y,n is an upper bound on 1/ (E [Y 1,n ])where the second inequality uses Assumption 9.2.2.Combining the two bounds yields the following upper bound on the probability considered in Theo-

		2 ≤	E Y 2 1,n -l Y,n nε 2 l Y,n 2	2	≤	2 2 , u Y,n -l Y,n nε 2 l Y,n
	rem 9.3	u X,n nε 2 +	2 2 , u Y,n -l Y,n nε 2 l Y,n	(

2 and l 2 Y,n a lower bound on (E [Y 1,n ]) 2 . Therefore V [Y 1,n ] nε 2 (E [Y 1,n ])

  Y,n -a Y,n ) 2 . Under Assumptions 9.2.1, 9.9.1 and 9.9.2, we have for every n ∈ N * and ε ∈ R * Chapter 9. Confidence intervals for ratios of means: limitations of the delta method and honest confidence intervals 272As a consequence,P X n /Y n -E [X 1,n ] /E [Y 1,n ] > t ≤ α, with the choice (u X,n ∨ u Y,n ) ln (4/α) 2n 1 + 1 a Y,n |a X,n | ∨ |b X,n | + (u X,n ∨ u Y,n ) ln (4/α) 2n ,

	t :=	1 l Y,n
							+
		P	X n Y n	-	E [X 1,n ] E [Y 1,n ]	>	ε l Y,n	1 +	1 a Y,n

(|a X,n | ∨ |b X,n | + ε) ≤ 4 exp -2nε 2 u X,n ∨ u Y,n

.

  1,n ) = 2l 2 Y,n /(b Y,n -a Y,n ) 2 . As a consequence, P X n /Y n -E [X 1,n ] /E [Y 1,n ] > t ≤ α, with the choice t := ln(4/α) nγ (X 1,n ) ∧ γ (Y 1,n ) |a X,n | ∨ |b X,n | + ln(4/α)/nγ (X 1,n ) (1 -ln(4/α)/nγ (Y 1,n )) 2

	+ 1	1 l Y,n	,

f ∈Breg(f * ,ρ,r) i∈J i ξ i (f -f * )(X i ) ≤ γ M |J|r 2 ,

Equivalently, for a stated nominal level for the confidence interval, we can compute the minimum required sample size.

Otherwise, we simply replace Y i,n by its opposite -Y i,n .

Equivalently, it means that for a given level α, the above choice of t is valid for every integer n > nα:= 2(u Y,n -l 2 Y,n )/αl 2 Y,n .

Actually, the computation of αn and nα only require knowledge of l Y,n and u Y,n .

Equivalently, it means that for a given level α, the choice of t is valid for every integer n > nα := ln(4/α)/γ (Y 1,n ).

Remerciements

Acknowledgement

This work is supported by the Labex Ecodec under the grant ANR-11-LABEX-0047 from the French Agence Nationale de la Recherche. The author thanks Professor Alexandre Tsybakov for helpful comments and discussions. The author acknowledge the Associate Editor and two anonymous reviewers for their comments which lead to significant improvements of this paper.

Chapter 2. Improved bounds for Square-root Lasso and Square-root Slope Acknowledgements: The authors acknowledge the two anonymous reviewers for their numerous and very useful comments and suggestions.

Acknowledgments

This work is supported by the Labex Ecodec under the grant ANR-11-LABEX-0047 from the French Agence Nationale de la Recherche. The first author thanks Solt Kovács for inspiring ideas and a discussion which lead to this article. Chapter 7. A classification point-of-view on conditional Kendall's tau Remerciements Je voudrais adresser tout d'abord mes remerciements à mon directeur de thèse, Jean-David Fermanian. Je me rappelle encore du début, quand on s'est rencontré alors que je n'étais qu'un étudiant en début de deuxième année de l'ENSAE. À l'époque, le projet de Statistique appliquée m'apparaissait comme le cours le plus intéressant de l'année, et c'est grâce à ce projet que je me suis engagé dans la recherche. Je te remercie beaucoup pour tous les conseils et toutes les discussions que nous avons eues depuis, et j'ai appris beaucoup grâce à toi. Je voudrais ensuite remercier Alexandre Tsybakov d'avoir accepté de co-superviser ma thèse. Merci beaucoup de m'avoir accepté au sein du laboratoire de Statistique du CREST, et pour ton aide tout au long de cette thèse. Je remercie aussi Jean-Michel Zakoïan pour son accueil au laboratoire de Finance-Assurance. Je suis très reconnaissant pour tous les échanges que j'ai pu avoir avec les professeurs du CREST, en particulier Pierre Alquier, Guillaume Lecué, Cristina Butucea, Arnak Dalalyan, Nicolas Chopin, Marco Cuturi et Christian Francq, et pour tous ces moments passés avec les autres doctorants, en particulier mes co-auteurs Yannick et Lucas, ainsi que Gautier, Lionel, Badr, Geoffrey, Phillipe, Boris, Clara, Alexander, Vincent, Edwin, Anna, Nicolas, Léna, Mohammed, Amir, Avo, Gabriel, Ophélie et Sébastien. Enfin, je souhaite remercier beaucoup tous les membres du jury: Ivan Kojadinovic et Marten Wegkamp pour leurs rapports très positifs sur cette thèse, la présidente du jury, Anne-Laure Fougères ainsi que les examinateurs Dominique Picard et Matthieu Lerasle.

Je remercie mes amis Julie-Anne, James, Fabien, Bhawna, Guillaume, Ali, Yannis, Alexis, Sylvain et David pour leur soutien durant cette longue période. Je remercie ma famille et en particulier mes parents, qui m'ont offert le meilleur cadre de travail possible pour ma thèse, ainsi que mon frère Nicolas et ma soeur Héloïse. Finalement, j'ai une pensée spéciale pour Anh Thư et Mỹ Phương, qui ont toujours été là pour me soutenir durant ces dernières années. Chapter 9. Confidence intervals for ratios of means: limitations of the delta method and honest confidence intervals

asymptotic behavior 150

This lemma is proved in Section 5.5.9. Similarly as in the proof of Lemma 2.2 in Stute [START_REF] Stute | Conditional U-statistics[END_REF], for every i = 1, . . . , n and every bounded symmetrical measurable function g, we have (nh p ) 1/2 V ar Ûn,i (g) -U n,i (g) = o(1), which implies (nh p ) 1/2 U n,i (g) -IE U n,i (g) i=1,...,n D -→ N (0, M ∞ (g)), as n → ∞.

Considering two measurable and bounded functions g 1 and g 2 , we have U n,i (c 1 g 1 +c 2 g 2 ) = c 1 U n,i (g 1 )+ c 2 U n,i (g 2 ) for every real numbers c 1 , c 2 . By the Cramér-Wold device, we easily state that

as n → ∞, where

Set τ1,2|Z=z i := U n,i (g * ) / U n,i (1). Since (nh p n ) 1/2 τ1,2|Z=z i -τ1,2|Z=z i = O P (nh p n ) 1/2 n,i is o P (1), it is sufficient to establish the asymptotic law of (nh p n ) 1/2 τ1,2|Z=z i -τ 1,2|Z=z i . Since IE[U n,i (1)] = 1 + o((nh p ) -1/2 ) and IE[U n,i (g * )] = τ 1,2|Z=z i + o((nh p n ) -1/2 ), we get

, as n → ∞.

Now apply the Delta-method with the function ρ(x, y) := x/y where x and y are real-valued vectors of size n and the division has to be understood component-wise. The Jacobian of ρ is given by the n × 2n matrix J ρ (x, y) = Diag y -1 1 , . . . y -1 n , Diag -

where, for any vector v of size n , Diag(v) is the diagonal matrix whose diagonal elements are the v i , with i = 1, . . . , n . We deduce (nh p ) 1/2 τ1,2|Z=z i -τ 1,2|Z=z i i=1,...,n stage, at the cost of a (probably small) loss of power. For instance, in the case of = 0, we propose the Wald-type test statistics

Ĥi,j ψ(z i )ψ(z j ) T Σ -1 n .

Ĥi,j :=

where fZ (Z) and G n (z) denote consistent estimators of f Z (z) and IE[g(X 1 , X)g(X 2 , X)|Z = Z 1 = Z 2 = z] respectively. Under H 0 , W n tends to a chi-square distribution with n degrees of freedom. For instance, with the notation of Section 6.1, we propose G n (z) = n i,j,k=1,i =j =k w i,n (z)w j,n (z)w k,n (z)g(X i , X k )g(X j , X k ).

Note that if there is an intercept, i.e. if one of the functions in ψ (say, ψ 1 ) is constant to 1, it should be removed in the statistics above. The corresponding coefficients of β should be removed as well. Indeed, in this case the simplifying assumption does not correspond to β * = 0, but rather to β * -1 = 0 where β * -i denotes the vector β * where the i-th coefficient has been removed.

Oracle property and a related adaptive procedure

Let remember that S := {j : β * j = 0} and assume that |S| = s < p so that the true model depends on a subset of predictors. In the same spirit as [START_REF] Fan | Variable selection via nonconcave penalized likelihood and its oracle properties[END_REF], we say that an estimator β satisfies the oracle property if

• v n ( βS -β * S ) converges in law towards a continuous random vector, for some conveniently chosen rate of convergence (v n ), and

• we identify the nonzero components of the true parameter β * with probability one when the sample size n is large, i.e. the probability of the event {j : βj = 0} = S tends to one.

As above, let us fix n and n will tend to the infinity. Then, denote {j : βj = 0} by S n , that will implicitly depend on n . It is well-known that the usual Lasso estimator does not fulfill the oracle property, see [START_REF] Zou | The adaptive lasso and its oracle properties[END_REF]. Here, this is still the case. The following proposition is proved in Section 6.7.3. Proposition 6.11. Under the assumptions of Theorem 6.10, lim sup n IP (S n = S) = c < 1.

A usual way of obtaining the oracle property is to modify our estimator in an "adaptive" way. Following [START_REF] Zou | The adaptive lasso and its oracle properties[END_REF], consider a preliminary "rough" estimator of β * , denoted by βn , or more simply β. Moreover ν n ( βn -β * ) is assumed to be asymptotically normal, for some deterministic sequence (ν n ) that tends to the infinity. Now, let us consider the same optimization program as in (6.2) but with a random tuning parameter given by λ n,n := µ n,n /| βn | δ , for some constant δ > 0 and some positive deterministic sequence (µ n,n ). The corresponding adaptive estimator (solution of the modified Equation (6.2)) will be denoted by βn,n , or simply β. Hereafter, we still set S n = {j : βj = 0}. The following theorem is proved in Section 6.7.4. Theorem 6.12 (Asymptotic law of the adaptive estimator of β). Under the assumptions of Lemma 6.24, if µ n,n (nh p n,n ) 1/2 → ≥ 0 and µ n,n (nh p n,n ) 1/2 ν δ n → ∞ when n → ∞, we have Input: Initial dataset D = (X i,1 , X i,2 , Z i ) i=1,...,n ∈ R 2+p n ; k ← 0 ;

for i ← 1 to (n -1) do for j ← (i + 1) to n do Zk ← (Z i + Z j )/2 ; W k ← W (i,j) as defined in Equation (7.1) ;

The case of probit and logit classifiers

With the new dataset D, we can virtually apply any classification method to predict the concordance value W k of the pair k, given the covariate Zk and the weight V k . The logit and probit models yield some of the oldest and easiest methods in classification. They have straightforward adapted versions in our case: see Algorithm 8. These weighted penalized GLM procedures are estimated using the R package ordinalNet [START_REF] Wurm | Regularized ordinal regression and the ordinalNet R package[END_REF]. Note that we are still estimating τ 1,2|Z=z under the parametric model given by (7.3). The tuning parameter λ can be chosen using a generalization of Algorithm 2 in Derumigny and

Fermanian [START_REF] Derumigny | About Kendall's regression[END_REF]. The chosen λ is the one which minimizes the cross-validation criterion,

where d(• ; •) is a distance on a space of bounded functions of z, for example the distance generated by the L 2 norm, τ (k) 1,2|Z=• is an estimator of Kendall's tau using the dataset D k , β(λ,-k) is estimated on the dataset D\D k using the tuning parameter λ, and the initial dataset D has been separated at random in N subsets D 1 , . . . , D N of equal size.

Algorithm 8: Estimation of the conditional Kendall's tau τ 1,2|Z=z using a logit (resp. probit) regression.

function ψ and a penalty level λ;

Compute the usual weighted penalized logit (resp. probit) estimator β on the dataset (W k , ψ( Zk ), V k ) k∈K with a tuning parameter λ ;

Output: An estimator τ1,2|Z=z := (e ψ(z) T β -1)/(e ψ(z) T β + 1) (resp. τ1,2|Z=z := 2Φ(ψ(z) T β) -1).

Decision trees and random forests

Now, let us discuss how partition-based methods can be used for the estimation of the conditional Kendall's tau. Strictly speaking, such techniques are parametric: the relationship (7.2) implicitly applies, Chapter 7. A classification point-of-view on conditional Kendall's tau 194 but for some complex untractable function g. And the parameter β * is related to some covariate thresholds, typically. Nonetheless, a classical decision tree can be directly trained on the weighted dataset D.

We use the R package tree by Ripley [START_REF] Ripley | Tree: classification and regression trees[END_REF], following Breiman et al. [START_REF] Breiman | Classification and Regression Trees[END_REF]. Therefore, the application of decision trees to our framework is straightforward, and does not require any special adaptation, contrary to random forests. And the tree procedure allows the calculation of the probability of observing a concordant pair, given any common value of Z.

In a classical classification setting, random forests are techniques of aggregation of decision trees that are built on a subset of samples and subsets of variables. More precisely, a typical random forest algorithm is the following: sample 80% of the rows of the dataset (without replacement), and 80% of the explanatory variables; estimate a tree on this, and repeat this procedure a certain number of times, with different sub-samples every time. In our framework, it is not clear at which level subsampling should take place.

The easiest solution would be to directly plug-in the dataset of pairs D into a classical random forest algorithm, but it does not obviously lead to the best solution. For comparison, we detail this solution in Algorithm 9. We propose now an improvement on Algorithm 9. Indeed, noting that aggregation of trees is useless if all trees are identical, it seems that the more variability in the input of the trees, the better.

Following this idea, we have noticed that the observations in the dataset of pairs are not independent.

Influence of this lack of independence is discussed in a general setting in Section 7.3.5. For example, the pair (1, 2) is usually not independent of the pair (1, 3), because they both share the first observation

. Therefore, to increase the diversity of inputs in the different trees, we suggest to lead a first sampling S j on the initial dataset, and then to build a dataset of pairs on the sampled observations Algorithm 10). As a matter of fact, if for example the first observation does not belong to the sample S j , then the dataset D j and the estimated tree T j become both independent of this first observation (X 1,1 , X 1,2 , Z 1 ). This independence property makes the trees less dependent, and significantly improves the performance in our results compared to the original Algorithm 9.

Algorithm 9: Random forests un-adapted for the estimation of the conditional Kendall's tau

Compute the dataset of pairs D using Algorithm 7 on D ;

Higher dimensional settings

In the previous sections, we had chosen a univariate vector Z, i.e. p = 1. Since this may sound a bit restrictive, we would like to obtain some finite-sample results in dimension p = 2. Note that the latter dimension cannot be too high because of the curse of dimensionality linked with the necessary kernel smoothing (done in Algorithm 7 when creating the dataset of pairs). We also choose a simple dictionary ψ of functions, which consists of the two projections on the coordinates of Z. The performance of the estimators is still be assessed by the approximate error criterion (7.10). The corresponding z (j) are chosen as a grid of 400 points equispaced on the square [0.01, 0.99] 2 .

In this framework, we first choose block (iii) of the model : the conditional copula of X 1 and X 2 given Z will be Gaussian, and block (iv) :

). We will try different combinations for the remaining blocks (i) and (ii), as described as follows:

(1)

, and the copula of (Z 1 , Z 2 ) is Gaussian with a Kendall's tau equal to 0.5. Moreover, τ 1,2|Z=z = z 2 tanh(z 1 ). This model is interesting because the function z → τ 1,2|Z=z

will be far away from a linear function of ψ(z), and machine learning techniques should work better than logistic/probit regressions.

(2) We keep the same model as previously, but by setting g(τ 1,2|Z=z ) = z 1 + z 2 , using the function g in Example 7.1 so that we recover the parametric setting of Section 7.2.

(3) Z 1 ∼ Exp(1), Z 2 ∼ N (0, 1) and both variables are independent. Set τ 1,2|Z=z = exp(-z 1 |z 2 |).

Again, we have a misspecified nonlinear model that is far away from logit/probit models.

The results are given in

Choice of the number of neurons in the one-dimensional reference setting

We consider networks with different numbers of neurons, and study their performance, both statistically and computationally. The results are displayed in the following Table 7.7. We observe that increasing the number of neurons only seems to deteriorate the performance of the method.

Influence of the sample size n

In our one-dimensional reference setting, we fix all the parameters except n. For a grid of values of n we evaluate the performance of our estimators. We observe that, for most methods, the computation time increases and the error criterion decreases when the sample size increases. We note that the number of pairs is O(n(n-1)) (at most) and, therefore, the computation time should increase as O(n 2 ), which is coherent with the results of Table 7.8. The relative order of the performances does not seem to change with the sample size n: the same methods are the best ones with small or large n. Note that we have not tried to find an "optimal" fine-tuning of the parameters for each method and each choice of n. Indeed, finding optimal choices of tuning parameters is not an easy task (in a theoretical and practical sense). More accurate analysis are left for future research.

Influence of the lack of independence

In Section 7.3.5, we explain some theoretical considerations about the lack of independence in the dataset D and some consequences. The following simulation experiment complements this analysis with some empirical results.

Indeed, using Algorithm 7, we note that pairs of observations are not independent, and therefore, the elements of the dataset of pairs D are not independent from each other in general. This could damage the performance of our methods, compared to a situation where all elements would be independent. We now consider such a situation, in order to compare the performance of the methods in both cases. Note that the cardinality of D is n(n -1)/2. Therefore, we will compare the two following settings:

1. Reference situation: fix n = 3000, simulate n independent copies

Some basic definitions about copulas

Here, we recall the main concepts around copulas and conditional copulas. First, a d-dimensional copula is a cdf on [0, 1] d whose margins are uniform distributions. Sklar's theorem states that, for any d-dimensional distributions H, whose marginal cdfs' are denoted as F 1 , . . . , F d , there exists a copula C s.t.

for every (x 1 , . . . , x d ) ∈ R d . If the law of H is continuous, the latter C is unique, and it is called the copula associated to H. Inversely, for a given copula and some univariate cdfs'

The latter concept of copula is similarly related to any random vector X whose cdf is H, and there is no ambiguity by using the same term. Copulas are invariant w.r.t. strictly increasing transforms of the margins X k , k = 1, . . . , d. They provide very practical tools for modeling complex and/or highly dimensional distributions in a flexible way, by splitting the task into two parts: the specification of the marginal distributions on one side, and the specification of the copula on the other side. Therefore, a copula can be seen as a function that describes the dependence between the components of X, independently of the marginal distributions. Several popular dependence measures are functionals of the underlying copula only: Kendall's tau, Spearman's rho, Blomqvist coefficient, etc. The classical textbooks by Joe [START_REF] Joe | Dependence Modeling with copulas[END_REF] or Nelsen [START_REF] Nelsen | An introduction to copulas[END_REF] provide numerous and detailed results.

Numerous parametric families of copulas have been proposed in the literature: Gaussian, Student, Archimedean, Marshall-Olkin, extreme-value, etc. Several inference methods have been adapted to evaluate an underlying copula, possible without estimating the marginal cdfs' (Canonical Maximum Like-

tends to a constant when n → ∞ (independently of the choice of such indices).

A similar analysis can be led for the other terms. When i 1 = j 2 and j 1 = i 2 , we get

by setting

tends to a constant when n → ∞.

When j 1 = j 2 = j and i 1 = i 2 , we obtain

by setting

+ (p(y, z) -p(x, y, z)) log q(z, β) log(1 -q(x, β))

,j tends to a constant when n → ∞.

When j 1 = i 2 and i 1 = j 2 :

by setting

There are two cases of two equalities. If i 1 = i 2 = i and j 1 = j 2 = j, this yields Keywords: kernel smoothing, regression-type models, penalized estimation.

Part III

Other topics in inference

Based on [START_REF] Derumigny | Estimation of a regular conditional functional by conditional U-statistics regression[END_REF]: Derumigny, A., Estimation of a regular conditional functional by conditional Ustatistics regression. Arxiv preprint, arXiv:1903.10914, 2019.

Introduction

Let X be a random variable with values in a measurable space (X , A), and denote by IP X its law. A natural application is X = R p X , for a fixed dimension p X > 0. Often, we are interested in estimating a regular functional θ(IP X ) of the law of X, of the form

, n

Input: A finite collection of points z 1 , . . . , z n ∈ Z n , selected for estimation

k , . . . , z

Compute the estimator θ z σ(1) , . . . , z σ(k) using the sample (X i , Z i ), i = 1, . . . , n ; end Compute the minimizer β of (8.3) using the θ z σ(1) , . . . , z σ(k) , j = 1, . . . , n , estimated in the above step ;

Compute the prediction θ(z

k ) T β ; end Output: An estimator β and N predictions θ(z

Once an estimator β of β * has been computed, the prediction of all the conditional functionals is reduced to the computation of Λ (-1) ψ(z

The first term corresponds to the cost of evaluating each non-parametric estimator (8.1). The second term corresponds to the minimization of the convex optimization program (8.3), and the last one is the prediction cost. Note that it can provide a huge improvement compared to the previously available estimator with a cost in O(N n k ) when N → ∞, i.e. when we want to recover the full function θ(•,

Moreover, the speedup given by Algorithm 15 compared to the original conditional U-statistics (8.1) even increases with the sample size n, for moderate choices of n .

A similar model, called functional response has already been studied ; see, e.g. Kowalski and Tu [START_REF] Kowalski | Modern applied U-statistics[END_REF]Chapter 6.2]. They provide a method to estimate the parameter β * , using generalized estimating equations. However, they only provides asymptotic results for their estimator, and their algorithm needs to solve a multi-dimensional equation which has no reason to be convex.

In Section 8.2, we provide both asymptotic properties and non-asymptotic bounds for the nonparametric estimator θ. Section 8.3 is devoted to the theoretical study of the two-step parametric estimator β. Finally, we detail examples in Section 8.4. All proof have been postponed to the Appendix.

Theoretical properties of the nonparametric estimator θ(•)

where I ↑ k,n is the set of strictly increasing functions from {1, . . . , k} to {1, . . . , n}. As a consequence, the cost of the computation of θ↑ is reduced by a factor of k! compared to θ. The estimator θ is well-defined if and only if N k (z 1 , . . . , z k ) > 0, where

Non-asymptotic bounds for N k

To prove that our estimator θ(z 1 , . . . , z k ) exists with a probability that tends to 1, we will study the behavior of N k . We will need the following assumptions to control the behavior of the kernel K and of the density of Z.

Assumption 8.2.1. The kernel K(•) is bounded, i.e. there exists a finite constant

The kernel is of order α for some α > 0, i.e. for all j = 1, . . . , α -1 and all

Assumption 8.2.2. f Z is α-times continuously differentiable on Z and there exists a finite constant C K,α such that, for all z 1 , . . .

where

This Lemma is proved in Section 8.6.1. More can be said if the density f Z is bounded below. Therefore, we will use the following assumption.

Assumption 8.2.4.

There exists a constant f Z,min > 0 such that for every z ∈ Z, f Z (z) > f Z,min .

If for some > 0, we have C K,α h α /α! + t ≤ f Z,min -, then f (z) ≥ > 0 with probability larger than on the event whose probability is bound in Lemma 8.3. We should therefore choose the largest t possible, which yields the following corollary.

is strictly positive with probability larger than

guaranteeing the existence of the estimator θ(z 1 , . . . , z k ) on this event.

Non-asymptotic bounds in probability for θ

To establish bounds on θ for every fixed n, we will need some assumptions on the joint law of (X, Z).

Assumption 8.2.5.

There exists a measure µ on (X , A) such that IP X,Z is absolutely continuous with respect to µ ⊗ Leb p , where Leb p is the Lebesgue measure on R p . Assumption 8.2.9.

(iii) there exists a decreasing function H : R + → R + , and positive constants c 1 , c 2 such that

and

Proposition 8.6 (Consistency of θ, Theorem 2 in Stute [START_REF] Stute | Conditional U-statistics[END_REF]). Under Assumption 8.2.9, for IP ⊗k Z -almost all (z 1 , . . . , z k ), θ(z 1 , . . . , z k )

We introduce now a few more notation to state the asymptotic normality of θ. For 1 ≤ j, l, m ≤ k and

(ii) K is symmetric at 0 and bounded with compact support ;

(iii) θ j,l is continuous at (z 1 , . . . , z k ) for all 1 ≤ j, l ≤ k ;

(iv) θ is two times continuously differentiable in a neighborhood of (z 1 , . . . , z k ) ;

(v) θ j,l,m is bounded in a neighborhood of the point (z 1 , . . . , z k , z 1 , . . . , z k , z 1 , . . . , z k ) ∈ Z 3k for all

Proposition 8.7 (Asymptotic normality of θ, Corollary 2.4 in Stute [START_REF] Stute | Conditional U-statistics[END_REF]). Under Assumption 8.2.10,

where

Moreover, let N be a positive integer, and z

1 , . . . , z

k , . . . , z

Note that the second part of Proposition 8.7 above is a consequence of the first one. Indeed, for every (c 1 , . . . , c N ) ∈ R N , we can define θ z

The latter theorem gives some bounds that hold in probability for the prediction error ||Z ( β -β * )|| n and for the estimation error | β -β * | q with 1 ≤ q ≤ 2 under the specification (8.2). Note that the influence of n and r is hidden through the Restricted Eigenvalue number κ(s, 3). The result depends on three parameters to be chosen: γ, t, h. The choice of γ is easy, as a larger γ can only deteriorate the bound.

We therefore have an interest in choosing the smallest γ possible, i.e. γ = 3.

The choice of the parameters t and h is more difficult. Indeed, looking at Equation (8.8), we can see that a lower t gives a smaller bound, but at the same time the probability of this event (which probability is bounded in Equation (8.8)) is lower. This induces a trade-off between the probability of the event and the size of the bound, as we want the smallest possible bound with the highest possible probability.

Moreover, we cannot choose t too small, because of the lower bound (8.7): t is limited by a value proportional to h α . On the one hand, we could choose a small h, but on the other hand, the probability of the event in Equation (8.8) will decrease. Summing up this reasoning in other words, we get the following conclusion: low values of h and t give a smaller bound, while high values of h and t give a high probability. Therefore, there is some kind of compromise to do, depending of the kind of bound one is looking for.

Asymptotic properties of β when n → ∞ and for fixed n

In this part, n is still assumed to be fixed and we state the consistency and the asymptotic normality of β as n → ∞. As above, we adopt a fixed design: the z i are arbitrarily fixed or, equivalently, our reasoning are made conditionally on the second sample. In this section, we follow Derumigny and Fermanian [START_REF] Derumigny | About Kendall's regression[END_REF] by giving similar results. Proofs are identical and therefore omitted.

For n, n > 0, denote by βn,n the estimator (8.3) with h = h n and λ = λ n,n .

Lemma 8.9. We have βn,n = arg min

Theorem 8.10 (Consistency of β). Under Assumption 8.2.9, if n is fixed and λ = λ n,n → λ 0 , then, given z 1 , . . . , z n and as n tends to the infinity, βn,n

In particular, if λ 0 = 0 and < {ψ z σ(1) , . . . , z σ(k) :

Chapter 9. Confidence intervals for ratios of means: limitations of the delta method and honest confidence intervals 273

An upper bound on testable confidence levels

Theorem 9.13. For every n ∈ N * , and every α ∈ 0, α n,H , where α n,H : Note that for Theorems 9.11 and 9.12, it is required to know not only the length b X,n -a X,n but also the actual endpoints of the support, a X,n and b X,n . On the contrary, Theorem 9.13 does not require the latter. In that respect, the class of Theorem 9.13 is larger than the one of the two preceding theorems.

Proof of Theorems 9.11 and 9.12

The proofs are identical to those of Theorems 9.2 and 9.3, except for the Bienaymé-Chebyshev inequality that has to be replaced with the Hoeffding inequality. The latter can be used under Assumption 9.9.1.

Note also that E

Proof of Theorem 9.13

By Lemma 9.14, for every ξ < 1 ∧ (b Y,n -a Y,n )/l Y,n -1 , there exists a distribution P n, ξ ∈ P H satisfying Assumptions 9.2.1 and 9.9.1 such that P Y n = 0 ≥ αn,H (ξ). Denote its marginal distributions by P X,n, ξ and P Y,n, ξ . Therefore, P n, ξ satisfies Assumptions 9.2.1 and 9.9.1, and X n /Y n is undefined with probability greater than αn,H (ξ). Taking the supremum over ξ, we deduce that sup

This means that the random interval I * n := X n /Y n ± t cannot have coverage higher than 1 -α n,H since it may be undefined with a probability higher than α n,H .

Lemma 9.14. For each ξ in the interval

Proof of Lemma 9.14: We consider the following distribution on R

where c ∈ (0, n) is some constant to be chosen later,

Chapter 9. Confidence intervals for ratios of means: limitations of the delta method and honest confidence intervals 274

we have

which is true by assumption.

Consider now the following product measure on R 2 defined by P n := 0.5δ {0} + 0.5δ {b X,n -a X,n } ⊗

. The next step is to build a lower bound on the event

The assumption that (X i,n , Y i,n ) n i=1 i.i.d.

∼ P n and the construction of P n,l Y,n ,cn,ξ imply that

Chapter 9. Confidence intervals for ratios of means: limitations of the delta method and honest confidence intervals 275

Additional simulations

This section complements the simulations presented in the main body of the article. Figures show the same objects but with different specifications of the distribution used. We use those different specifications as the order of presentation in this section.

In this setting of simulations, we use the best bounds by setting the constants l Y,n and u Y,n that define our class of distributions equal to the actual corresponding moments (respectively the expectation for l Y,n and the second moment i.e. the expectation of the square for u Y,n ). That is we use

In practical settings, the rule-of-thumb will be to replace the theoretical and unknown moments by their empirical counterparts and use the n α obtained to appraise the reliability of the asymptotic CIs from the delta method. Specification: ∀n ∈ N * , PX,Y,n = N (1, 1) ⊗ N (0.025, 1). The nominal pointwise asymptotic level is set to 0.95. For a sample size n, the coverage is obtained as the mean over 5,000 repetitions. The dashed vertical line shows

Rule of thumb using α n

Symmetrically, we can consider the rule of thumb for a fixed sample size n and compare the desired nominal level 1 -α of the test to the rule of thumb 1 -α n , with the α n derived in Theorem 9.3.

Student distributions

The specification here is two Student distributions, both in the numerator and in the denominator. Standard Student distributions are centered. We use therefore translated versions by simply adding the expectations in order to avoid a null denominator for the ratio of expectations of interest. Below, T (µ, ν) denotes the distribution of a translated standard Student variable: µ + T where T is distributed according Specification: ∀n ∈ N * , PX,Y,n = N (1, 1) ⊗ N (0.05, 1). The nominal pointwise asymptotic level is set to 0.95. For a sample size n, the coverage is obtained as the mean over 5,000 repetitions. The dashed vertical line shows Specification: ∀n ∈ N * , PX,Y,n = N (1, 1) ⊗ N (0.25, 1). The nominal pointwise asymptotic level is set to 0.95. For a sample size n, the coverage is obtained as the mean over 5,000 repetitions. The dashed vertical line shows Specification: ∀n ∈ N * , PX,Y,n = N (1, 1) ⊗ N (0.75, 1). The nominal pointwise asymptotic level is set to 0.95. For a sample size n, the coverage is obtained as the mean over 5,000 repetitions. The dashed vertical line shows

Corr(X, Y ) = 0.5. The nominal pointwise asymptotic level is set to 0.90. For a sample size n, the coverage is obtained as the mean over 5,000 repetitions. The dashed vertical line shows nα := 2 uY,n -lY,n 2 / αl 2 Y,n , setting here Sample size n Maximal coverage (nominal level = 99%) Figure 9.17: Maximal coverage of asymptotic CIs based on the delta method and rule-of-thumb nα.

The nominal pointwise asymptotic level is set to 0.99. For a size n, the coverage is obtained as the mean over 5,000 repetitions. The dashed vertical line shows nα := 2 uY,n -lY,n 2 / αl 2 Y,n , setting here

The nominal pointwise asymptotic level is set to 0.95. For a size n, the coverage is obtained as the mean over 5,000 repetitions. The dashed vertical line shows nα := 2 uY,n -lY,n 2 / αl 2 Y,n , setting here

Chapter 9. Confidence intervals for ratios of means: limitations of the delta method and honest confidence intervals 280

For each nominal level 1 -α in the x-axis, we draw 5, 000 samples, compute the asymptotic CIs and see whether it covers or not the ratio of interest; we report the mean over the 5, 000 repetitions in the y-axis. The solid line is the first bisector y = x. The dashed vertical line shows αn

to a Student distribution with ν degrees of freedom. To satisfy Assumption 9.2.1, we need finite variance: we use degrees of freedom strictly higher than 2 for this purpose.

Exponential distributions

The specification here is two exponential distributions, both in the numerator and in the denominator.

In this setting of simulations, we use as previously the best bounds by setting the constants l Y,n and u Y,n that define our class of distributions equal to the actual corresponding moments (respectively the expectation for l Y,n and the second moment i.e. expectation of the square for u Y,n ). In other words, we

For reminder, in practical settings, the rule-of-thumb will be to replace the theoretical and unknown moments by their empirical counterparts and use the n α obtained to appraise the reliability of the asymptotic CIs from the delta method.

However, for exponential distributions, the variance is equal to the square of the expectation. Consequently, whatever the parameter of the exponential distribution in the denominator, we have n α = 4/α.

Previous simulations suggest that the closer the expectation in the denominator to 0, the larger the sample size required for the asymptotic approximation to hold. At first sight, We might be worried for the usefulness of our rule-of-thumb to get n α independent of E[Y ]. However, in the special case of exponential distributions, the lower the expectation, the lower too is the variance. Intuitively, the lower variance will compensate having an expectation closer to 0. The previous statement that links the closeness to 0 of the expectation in the denominator and the sample size required to reach the asymptotic approximation presupposes keeping fixed the variance. It cannot be anymore for exponential distributions.

The following simulations reveal that the convergence of the coverage of the asymptotic confidence

The nominal pointwise asymptotic level is set to 0.95. For a size n, the coverage is obtained as the mean over 5,000 repetitions. The dashed

The nominal pointwise asymptotic level is set to 0.95. For a size n, the coverage is obtained as the mean over 5,000 repetitions. The dashed vertical line shows nα := 2 uY,n -lY,n 2 / αl 2 Y,n , setting here

intervals toward their nominal level happens for n around one hundred and more and, furthermore, has the same pattern whatever the expectation of the exponential distribution in the denominator. Our ruleof-thumb n α appears to be a bit small. Nonetheless, it is coherent that it is constant across the value of

Pareto distributions

The specification here is two Pareto distributions, both in the numerator and in the denominator. Pareto distributions have support in R * + . They would fall in the easier case when the support of the denominator is well separated from 0. To assess the dependability of our rule-of-thumb in the general case, we use translated Pareto distributions. In what follows, the notation Pareto(E[Y ], τ, γ) denotes the distribution of a random variable that follows a Pareto distribution with shape parameter equal to γ translated such that its support is (τ, +∞) and its expectation is E[Y ]. A variable that is distributed according to

Bernoulli distributions

With discrete distributions for the variable at the denominator, it may happen that Y n = 0, all the more so as the expectation in the denominator and the sample size are low in the case of Bernoulli distributions.

In that situation, the confidence interval is said to be undefined and, for any arbitrary value, the statement that the CI contains that value is considered as false. Consequently, in the simulations with discrete distributions in the denominator, whenever the sample drawn is such that Y n = 0, we count the draw as a no coverage occurrence in the Monte Carlo estimation of the (maximal) coverage. Concretely, the maximal coverage displayed in the graph is computed as the mean over B repetitions. The repetitions With two Bernoulli variables in the numerator and the denominator, we are both in the BC and the Hoeffding cases. The following graphs illustrate the use of n α to appraise the reliability of the asymptotic confidence based on the delta method. We show both the one obtained in the BC case (Theorem 9.3) and the one obtained in the Hoeffding case (Theorem 9.12). Again, as in the main body of the paper, we follow a plug-in strategy to compute in practice n α and, in the setting of simulations, we simply use the known moments and bounds of the DGP used in the simulation.

Poisson distributions

The specification here considers two variables distributed according to a Poisson distribution, both in the numerator and in the denominator.

A Poisson distribution is entirely defined by its positive real parameter, which is equal to both its expectation and its variance. Consequently, to have denominator close to 0, we would need small variance too, as in the exponential specification (see Section 9.10.4). In order to disentangle expectation and variance, we use below translated Poisson variables. Precisely, the notation Poisson(µ, σ 2 ), µ ∈ R, σ 2 ∈ R * + , denotes a distribution alike to a Poisson, with parameter and variance equal to σ 2 but translated such that its expectation is µ. That is a variable distributed according to Poisson(µ, σ 2 ) is equal in distribution to P + (µ -σ 2 ) with P a standard Poisson distribution with parameter σ 2 -that is with density with respect to the counting measure equal to (σ 2 ) k exp(-σ 2 )/(k!) for every k ∈ N. Thus, a Poisson(µ, σ 2 ) has expectation µ and variance σ 2 . Chapter 9. Confidence intervals for ratios of means: limitations of the delta method and honest confidence intervals 288