Gildas Mazo

Maiage

François Portier

Parametric versus nonparametric: the fitness coefficient

Keywords: Goodness-of-fit, Density estimation, Semiparametric methods, Kernel smoothing, Likelihood methods

The fitness coefficient, introduced in this paper, results from a competition between parametric and nonparametric density estimators within the likelihood of the data. As illustrated on several real datasets, the fitness coefficient generally agrees with p-values but is easier to compute and interpret. Namely, the fitness coefficient can be interpreted as the proportion of data coming from the parametric model. Moreover, the fitness coefficient can be used to build a semiparamteric compromise which improves inference over the parametric and nonparametric approaches. From a theoretical perspective, the fitness coefficient is shown to converge in probability to one if the model is true and to zero if the model is false. From a practical perspective, the utility of the fitness coefficient is illustrated on real and simulated datasets.

Introduction

A challenge of data analysis is to assess the quality of a model. The traditional approach relies on goodness-of-fit tests where, loosely speaking, the ability of a model to fit the data is measured through distances between the observed values and the values expected under the model. Examples include the classical Pearson's chi-squared test [START_REF] Akritas | Pearson-type goodness-of-fit tests: the univariate case[END_REF], the Kolmogorov or Cramer-von-Mises goodness-of-fit tests [START_REF] Anderson | Two-sample test statistics for measuring discrepancies between two multivariate probability density functions using kernel-based density estimates[END_REF][START_REF] D'agostino | Goodness-of-Fit Techniques[END_REF][START_REF] Durbin | Weak convergence of the sample distribution function when parameters are estimated[END_REF], or likelihood-ratio based statistics [START_REF] Chen | A goodness-of-fit test for parametric and semi-parametric models in multiresponse regression[END_REF][START_REF] Claeskens | Goodness of fit via non-parametric likelihood ratios[END_REF][START_REF] Vuong | Likelihood ratio tests for model selection and non-nested hypotheses[END_REF] (see [START_REF] Chen | A goodness-of-fit test for parametric and semi-parametric models in multiresponse regression[END_REF] for an empirical likelihood approach).

In this context, p-values have emerged as natural instruments to measure the amount of evidence in favor of the model. However, the use of p-values is subjected to several difficulties: (i) their calculation might require computationally intensive strategies as the bootstrap [START_REF] Jiménez-Gamero | Empirical characteristic function tests for GARCH innovation distribution using multipliers[END_REF][START_REF] Kojadinovic | Goodness-of-fit testing based on a weighted bootstrap: a fast large-sample alternative to the parametric bootstrap[END_REF][START_REF] Stute | Bootstrap based goodness-of-fit-tests[END_REF]; (ii) interpretation is notoriously difficult [START_REF] Murtaugh | In defense of p values[END_REF][START_REF] Schervish | P values: what they are and what they are not[END_REF] as emphasized again in a recent ASA statement [START_REF] Wasserstein | The ASA's statement on p-values: context, process, and purpose[END_REF]; (iii) whenever some evidence has been found against the model, no information is delivered to improve inference.

In this paper, we introduce the fitness coefficient, a new criterion for simultaneously measuring the amount of evidence of a model and improving inference. Our goal is to provide an alternative approach to the use of p-values in goodness-of-fit testing that is no longer sensitive to the difficulties (i)-(iii).

Let X 1 , . . . , X n be independent d-variate observations with common density f 0 . Let P = {f θ : θ ∈ Θ} be a family of probability density functions representing the model. Given the maximum likelihood estimator f θn (based on the model), and the standard kernel density estimator fn (free from the model) with kernel K : R d → R + and bandwidth h n > 0, define the fitness coefficient αn as

αn ∈ argmax α∈[0,1] n i=1 log αf θn (X i ) + (1 -α) f LR i,n , (1)
where f LR i,n is called the leave-and-repair (LR) kernel estimate of f 0 (X i ) and is given by

f LR i,n =   1 (n -1)h d n j =i K X i -X j h n   + ∆ n q(X i ), (2)
with ∆ n ≥ 0 and q : R d → R + . The LR estimate is a modification of the well-known leave-oneout (LOO) estimate usually employed in cross-validation procedures [START_REF] Hall | On Kullback-Leibler loss and density estimation[END_REF] and semiparametric estimation [START_REF] Delecroix | On semiparametric M-estimation in singleindex regression[END_REF].

The fitness coefficient has the following advantages.

(i) The fitness coefficient is easy to compute. It is the minimizer of a simple one dimensional concave function.

(ii) The fitness coefficient is a measure of model quality. As seen in [START_REF] Akaike | A new look at the statistical model identification[END_REF], the fitness coefficient αn follows from a competition between the parametric and the nonparametric approach so as to maximize the likelihood of the observations. Hence, whenever the model is sufficiently true, we expect a value of αn relatively close to one. This is because the parametric estimator is likely to be more accurate than the nonparametric one. On the opposite, whenever the model is wrong, we expect a value of αn close to zero. Because αn f θn + (1 -αn ) fn is a mixture distribution between the parametric and the nonparametric estimates, the fitness coefficient is interpreted as the proportion of data distributed under the model. For instance, if one draws a bootstrap sample from the combination αn f θn + (1 -αn ) fn then the fitness coefficient αn is the proportion of data drawn from the fitted model f θn . Therefore, the less the value of αn , the less the bootstrap sample shall be "contaminated" by the nonparametric part of the combination.

To show the capability of the fitness coefficient, we compare it to p-values on a real data q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q 1e-09 1e-05 1e-01 0.0 0.2 0.4 0.6 0.8 1.0 p-values fitness coefficients example. Consider the problem of testing whether a given sample comes from a normal distribution. Specifically, we have 38 samples each consisting of n = 409 financial returns of a company from the French stock market CAC40 and we wish to measure the quality of the normal model for each of these samples. On the one hand a goodness-of-fit test based on the Cramer-Von-Mises statistic [START_REF] D'agostino | Goodness-of-Fit Techniques[END_REF] is carried out. On the other hand the fitness coefficient defined by ( 1) is computed with a Gaussian kernel K, ∆ n = 1/n, q(x) = t ν (x/100) where t ν is the density of a Student-t distribution with ν = 3 degrees of freedom, and h n given by [START_REF] Silverman | Density estimation for statistics and data analysis[END_REF] (p. 48).

In Figure 1, we plotted the values of the fitness coefficient against the p-values on a logarithmic scale. We can see a clear positive dependence relationship: large values for the fitness coefficient correspond to large p-values. This suggests that, if one had used p-values to assess the fitness of the normal model, he or she could have done so with the fitness coefficient.

The quality criterion induced by the fitness coefficient is different than that of information criteria [START_REF] Burnham | Model selection and multimodel inference: a practical information-theoretic approach[END_REF][START_REF] Claeskens | Model selection and model averaging[END_REF] such as the Akaike information criterion [START_REF] Akaike | A new look at the statistical model identification[END_REF] or the Bayesian information criterion [START_REF] Schwarz | Estimating the dimension of a model[END_REF] which focus on the relative performances between models. Note that convex parametric combinations recently have been proposed in the Bayesian literature [START_REF] Kamary | Testing hypotheses via a mixture estimation model[END_REF] to assess the fitness of a certain parametric model against another.

(iii) The fitness coefficient is useful to get robust semiparametric estimators. The fitness coefficient offers a natural semiparametric alternative αn f θn + (1 -αn ) fn for estimating the probability density function f 0 of the observations. The idea of forming such a convex combination to get an estimator robust to misspecification while retaining a performance comparable to parametric estimators when the true density is close to the model was originally developed by Olkin and Spiegelman [START_REF] Olkin | A semiparametric approach to density estimation[END_REF]. Their method, referred to as the OS method, consists of computing

αOS n ∈ argmax α∈[0,1] n i=1 log αf θn (X i ) + (1 -α) fn (X i ) , (3)
where fn is the standard kernel density estimator. The OS method and the LR method given in (1) differ in the choice of the nonparametric estimator in the combination. The OS method was noticed to be sensitive to the choice of the bandwidth [START_REF] Faraway | Implementing semiparametric density estimation[END_REF][START_REF] Rahman | A note on estimating the combining constant in semiparametric density estimation[END_REF]. Rather than considering the likelihood of the observations, some authors [START_REF] Lee | A simple formula for mixing estimators with different convergence rates[END_REF][START_REF] Rahman | A note on estimating the combining constant in semiparametric density estimation[END_REF][START_REF] Soleymani | A bootstrap procedure for local semiparametric density estimation amid model uncertainties[END_REF] investigate strategies based on the mean squared error between the combination αf θn + (1 -α) fn and the true density f 0 , but then the solution depends on the unknown distribution and hence heavy bootstrap methods need to be employed.

To improve inference, there exist also other approaches than that of forming a convex combination between the parametric and nonparametric estimators. The locally parametric nonparametric estimation is developed for instance in [START_REF] Hjort | Locally parametric nonparametric density estimation[END_REF][START_REF] Hjort | Hybrid combinations of parametric and empirical likelihoods[END_REF][START_REF] Talamakrouni | Parametrically guided local quasilikelihood with censored data[END_REF], but is less appealing from the point of view of model quality assessment because they do not provide any "fitness coefficient".

Main contributions. By introducing the fitness coefficient, we provide a new measure for assessing the quality of a model and an alternative to the OS method to get robust semiparametric estimators. Under mild conditions, the fitness coefficient αn is shown to converge in probability to one if the model is true and zero otherwise, a property called consistency. Even if the fitness coefficient is maximizing some objective function (over α ∈ [0, 1]), classical results from M-estimation theory does not apply because, when the model is true, the limiting objective function is independent from α. The proposed approach follows from a fine comparison between the rates of convergence of f θn and fn . We moreover provide examples of densities f 0 that satisfy our set of assumptions. Using some real data as well as extensive simulations, we observed that the LR approach is more stable than the OS approach and leads to more accurate inference. This is in agreement with our theoretical analysis, which cannot include the OS method as an example.

Outline. In Section 2, we introduce some quantities of interest and motivate the use of the LR estimator f LR i,n to compute the fitness coefficient αn . The consistency of the fitness coefficient is stated in Section 3 where some examples are given. In Section 4, numerical experiments are designed to measure the robustness of the fitness coefficient and the performance of the corresponding density estimators. All the proofs are postponed to the Appendix.

The leave-and-repair estimator

The aim of this section is to define and motivate the use of the leave-and-repair (LR) estimator f LR n (2) introduced in the definition of the fitness coefficient αn [START_REF] Akaike | A new look at the statistical model identification[END_REF]. Compared with the OS method given in (3), the use of the LR estimator might seem irrelevant at first view but in fact plays an important role to ensure the good behavior of the fitness coefficient.

The kernel density estimator of f 0 at x ∈ R d is given by fn

(x) = 1 nh d n n i=1 K x -X i h n .
For any h > 0, define the function f h as the convolution product between

K h (•) = K(•/h)/h d and f 0 , that is, f h (x) = (K h f 0 )(x), x ∈ R d . Note that f hn (x)
is the expected value of fn (x).

In other words,

f hn (x) = E[ fn (x)]. But since E[ fn (X i )|X i ] = f hn (X i ) + K(0)/(nh d n )
, we see that fn (X i ) has a positive h n -dependent bias when estimating f hn (X i ), conditionally on X i . When studying the estimator decomposition, this bias term spreads to the diagonal terms of some U -statistics and gives rise, in the end, to some non-negligible terms. This phenomenon is common in semiparametric statisitcs, and has been noticed for instance in Remark 4 in [START_REF] Portier | On the weak convergence of the empirical conditional copula under a simplifying assumption[END_REF].

To overcome the undesirable effects caused by this bias term, the leave-one-out (LOO) estimator of f hn (X i ), given by

f LOO n,i = 1 (n -1)h d n j =i K X i -X j h n ,
has been successfully used in several cross-validation procedures aiming at selecting the bandwidth, either based on the likelihood [START_REF] Habbema | A stepwise discriminant analysis program using density estimation[END_REF][START_REF] Marron | An asymptotically efficient solution to the bandwidth problem of kernel density estimation[END_REF][START_REF] Hall | On Kullback-Leibler loss and density estimation[END_REF] or on the mean squared error [START_REF] Rudemo | Empirical choice of histograms and kernel density estimators[END_REF][START_REF] Stone | An asymptotically optimal window selection rule for kernel density estimates[END_REF] (see [START_REF] Marron | A comparison of cross-validation techniques in density estimation[END_REF] for a comparison). Since then, LOO estimators have been frequently used in semiparametric studies [START_REF] Delecroix | On semiparametric M-estimation in singleindex regression[END_REF]. The LR estimator proposed in this paper is inspired, but different, from the LOO estimator. In view of (2), the LR estimator satisfies

f LR n,i = f LOO n,i + ∆ n q(X i ).
If ∆ n = 0 the LR estimator is equal to the LOO estimator. If q = K(0) and ∆ n = 1/((n -1)h d n ) the LR estimator is equal to (n/(n -1)) fn (X i ). In general, one can think of ∆ n as of order 1/n and q as a density, yielding that q(X 1 ) > 0 has probability 1.

The heuristic for using the LR estimator f LR n,i instead of the LOO estimator f LOO n,i is as follows. It is well-known that the Kullback-Leibler divergence of kernel density estimates depends crucially on the tails of the true distribution f 0 [START_REF] Hall | On Kullback-Leibler loss and density estimation[END_REF][START_REF] Schuster | On the nonconsistency of maximum likelihood nonparametric density estimators[END_REF]. As shown in [START_REF] Hall | On Kullback-Leibler loss and density estimation[END_REF], if the tail is too heavy and the kernel K(x) vanishes too quickly as x becomes large then the Kullback-Leibler divergence associated to the kernel density estimate goes to minus infinity. This is because some of the f LOO n,i , i = 1, . . . , n, might have very small values (possibly zero), leading to very large values (possibly infinite) for some of the log( f LOO n,i ). These values are involved in the computation of the Kullback-Liebler divergence and play an important role in our proofs when dealing with the likelihood of the nonparametric estimate. We built the LR estimator to overcome this issue by simply adding ∆ n q(X i ) to the LOO estimator. We coined the term leave-and-repair because the term ∆ n q(X i ) repairs the LOO estimator. Since f LR n,i ≥ ∆ n q(X i ), the LR estimator is not subjected to the difficulties of the LOO estimator. By adding the term ∆ n q(X i ) in (2), however, a biais is introduced: now one has

E[ f LR n,i |X i ] -f hn (X i ) = ∆ n q(X i ).
Thus, there is a biais-variance tradeoff controlled by the sequence ∆ n that must go to zero slowly enough to keep f LR n,i away from zero but also fast enough to keep the biais as small as possible. The right compromise is given in the conditions in Theorem 2 (for instance ∆ n = 1/n is one possibility).

Concerning the parametric estimator f θn , we follow [START_REF] Olkin | A semiparametric approach to density estimation[END_REF] by considering the maximum likelihood estimator. Let P = {f θ : θ ∈ Θ} be the parametric model where Θ ∈ R p is such that for each θ ∈ Θ, f θ : R d → R + is a measurable function satisfying f θ (x) dx = 1. The maximum likelihood estimator of f 0 based on P and X 1 , . . . , X n is f θn where θn (when it exists; this is further assumed) is defined as

θn ∈ argmax θ∈Θ n i=1 log(f θ (X i )).
The good behaviour of the maximum likelihood estimator is subjected to the assumption that f 0 ∈ P, that is, there exists θ 0 ∈ Θ such that f 0 = f θ 0 .

To conclude the section, we consider existence and uniqueness of the fitness coefficient αn . The existence follows from the use of the LR estimator f LR n,i . Uniqueness of αn is obtained under the mild requirement that the parametric and nonparametric estimators are distinguishable on the observed data. Proposition 1. Suppose that q(X 1 ) > 0 a.s. and f θn (X i ) = f LR n,i for at least one i ∈ {1, . . . , n}. Then the fitness coefficient exists and is unique.

The proof is given in Appendix A.

Consistency of the fitness coefficient

Recall that consistency of the fitness coefficient αn means αn → 1 in probability if f 0 ∈ P and αn → 0 if f 0 / ∈ P, where f 0 is the true underlying density and P is the parametric model. Section 3.1 and Section 3.2 contain the main consistency theorem and some examples satisfying our set of assumptions, respectively.

Assumptions and main result

S t = {x ∈ R d : f 0 (x) > t}, t ≥ 0.
We shall assume the following.

(H1) The density f 0 is bounded and continuous on R d and the gradient ∇f 0 of f 0 is bounded on R d , and satisfies, for every

x ∈ R d and u ∈ [-1, 1] d , |f 0 (x + u) -f 0 (x) -u T ∇f 0 (x)| ≤ u 2 2 g(x),
where g is positive, bounded, integrable and g(x) 2 /f 0 (x) dx < ∞.

(H2) The kernel function K : R d → R + integrates to 1 and takes one of the two following forms,

(a) K(x) ∝ K (0) ( x 2 ), or (b) K(x) ∝ d k=1 K (0) (|x k |),
where

K (0) : [0, 1] → R + is a bounded function of bounded variation. The sequence (h n ) n≥1 is such that nh 2d+4 n → 0, nh d n /| log(h n )| → ∞.
Whereas (H1) and (H2) are rather classical in the kernel smoothing littereature (see the remarks just below Theorem 2), the following assumption is specific to our approach. We shall see in Section 3.2 that this is satisfied for densities with classical tails.

(H3) The function q : R d → R + is bounded, integrable, and satisfies

E[| log(q(X 1 ))|] < ∞.
There exist β ∈ (0, 1] and c > 0 such that

S c t f 0 (x) dx ≤ ct β as t → 0. For any γ > 0, b n = γ(nh d n ) -1/β , there exists C > 0 such that, as n → ∞, sup x∈S bn sup u∈[-1,1] d f 0 (x + h n u) f 0 (x) ≤ C, and
h d n λ(S bn ) → 0.
For the sake of clarity, the (classical) assumptions dealing with the parametric model are postponed to the appendix: (A1) and (A2). They are taken from the monographs [START_REF] Van Der | Asymptotic statistics[END_REF] and [START_REF] Newey | Large sample estimation and hypothesis testing[END_REF], and they mainly ensure the asymptotic normality of θn whenever f 0 ∈ P. Theorem 2. Suppose that assumptions (H1) to (H3), and (A1) are fulfilled.

(i) When f 0 ∈ P, under (A2) and if (nh d n )∆ n → 0, it holds that αn → 1, in probability.

(ii) When

f 0 / ∈ P, if ∆ n → 0 and ( | log(h n )|/nh d n + h 2 n )| log(∆ n )| 1/β → 0, we have that αn → 0, in probability.
Appendix B is dedicated to the proof of Theorem 2. We did not follow the approach used in [START_REF] Olkin | A semiparametric approach to density estimation[END_REF], which, we believe, is unsatisfactory because they do not consider the case when αn lies in the border of [0, 1]. Actually, this is not straightforwardly remedied as the event αn = 0 or αn = 1 has a non-negligible probability (as illustrated in the numerical experiments in section 4.1). The smoothness assumption stated in (H1) and the symmetries in the kernel function ensure a control of order h 2 n of the bias f h (x) -f (x), uniformly in x ∈ R d (see Lemma B.7 stated in Appendix B.5). Such a rate could be improved by using higher order kernels but this is not necessary here. Assumption (H2), (a) and (b), are borrowed from the empirical process literature; see among others [START_REF] Nolan | U -processes: rates of convergence[END_REF][START_REF] Giné | Rates of strong uniform consistency for multivariate kernel density estimators[END_REF][START_REF] Einmahl | An empirical process approach to the uniform consistency of kernel-type function estimators[END_REF]. They permit to bound, uniformly in x ∈ R d , the variance term fn (x) -f h (x). The fact that the kernel has a compact support can be alleviated at the price of additional technicalities in the proof and assuming that the tails of the kernel are light enough. We did not include this analysis in the paper for reasons of clarity.

For any dimension d ≥ 1, there exists a couple of sequences (h n , ∆ n ) n≥1 that fulfills the restrictions (i), (ii) of Theorem 2 and (H2). For instance, the optimal bandwidth h n ∝ n -1/(d+4) , which minimizes the asymptotic mean integrated squared error [51, equation (2.5)], and ∆ n = 1/n, is one such sequence. This means that, in practice, one can choose the bandwidth according to the various methods of the literature, see e.g. [START_REF] Silverman | Density estimation for statistics and data analysis[END_REF].

An interesting point in Theorem 2 is the two opposite roles played by the sequence ∆ n in (i) and (ii), respectively. The consistency when f 0 ∈ P requires ∆ n to be as small as possible whereas when f 0 / ∈ P, ∆ n must not be too close to 0. In the proof, the case ∆ n = 0 (leave-oneout) as well as ∆ n q(X i ) = K(0)/(nh d ) (OS method) need to be excluded, suggesting that these other options are not consistent under our set of assumptions.

Distributions and bandwidth sequences satisfying (H3)

For densities f 0 with unbounded supports, the verification of Assumption (H3) only depends on some tail function g 0 associated to the density f 0 . The meaning of this is made precise in the following proposition. Proposition 3. Suppose that for any A > 0, inf x ≤A f 0 (x) > 0 and that there exists a function

g 0 such that f 0 (x)/g 0 (x) → 1 as x → ∞. Suppose that h n → 0 obeys nh d n → ∞. If there exist c 2 > 0 and β ∈ (0, 1] such that g 0 (x)≤t g 0 (x) dx ≤ c 2 t β , as t → 0 and if for any γ > 0, b n = γ(nh d n ) -1/β , there exists A > 0, C 2 > 0 such that sup x >A, g 0 (x)>bn sup u∈[-1,1] g 0 (x + h n u) g 0 (x) ≤ C 2 , and h d n λ(g 0 (x) > b n ) → 0, (4)
as n → ∞, then (H3) is valid for f 0 with the same value of β.

The proof of Proposition 3 is given in Appendix A. The function g 0 in Proposition 3, not necessarily a proper density function, represents the rate of decrease of

f 0 (x) as x → ∞. Example 1 (Mixture of densities). Let d = 1. Let f 0 (x) = π 1 f 1 (x) + π 2 f 2 (x), π 1 > 0, π 2 > 0, π 1 + π 2 = 1, where f 1 and f 2 are densities such that f 1 (x)/f 2 (x) → 0 as |x| → ∞. Take g 0 (x) = π 2 f 2 (x). Then, as |x| → ∞, f 0 (x) g 0 (x) = π 1 f 1 (x) π 2 f 2 (x) + 1 → 1.
Hence the verification of (H3) by f 0 only depends on the component f 2 .

Putting g 0 ∝ f 0 (the symbol ∝ stands for proportionality) in Proposition 3 amounts to check (H3) directly, which is done in the following examples.

Example 2 (Gaussian tails). Let d = 1 and g 0 (x) = κ 1 exp(-κ 2 x 2 ), with κ 1 > 0, κ 2 > 0. For clarity the computations are provided for κ 1 = κ 2 = 1 but can easily be extended for arbitrary values. Because exp(-x 2 )≤t exp(-x 2 ) dx ≤ t, as t → 0, we have that β = 1. Moreover, for 0 < b n < 1, we have

sup exp(-x 2 )>bn sup u∈[-1,1] exp(-(x + h n u) 2 + x 2 ) ≤ sup exp(-x 2 )>bn sup u∈[-1,1] exp(-2h n xu) = sup exp(-x 2 )>bn exp(2h n |x|) ≤ exp(2 -log(b n )h n ) Therefore, a sufficient condition on h n guaranteeing (4) is that h 2 n log(n) → 0, which is satisfied under (H2). Example 3 (Exponential tails). Let d = 1 and g 0 (x) = κ 1 exp(-κ 2 x), with κ 1 > 0, κ 2 > 0.
The computations are very similar to the one presented in the Gaussian case. We find β = 1 and the condition on h n becomes h n log(n) → 0 which is always true under (H2). Hence, as for Gaussian tails, when the tails are exponential, (H3) is automatically satisfied under (H2).

Example 4 (Polynomial tails)

. Let d = 1 and g 0 (x) = κ 1 |x| -k with κ 1 > 0, k > 1.
For simpicity, as in the Gaussian example, we focus on κ 1 = 1. We find that β = (k -1)/k. For h n < |A|, we have

sup |x|>A, |x|≤b -1/k n sup u∈[-1,1] d |x| k |x + h n u| k = sup |x|>A, |x|≤b -1/k n |x| k (|x| -h n ) k = 1 (1 -h n /A) k n→∞ → 1. Finally, since h n λ(g 0 > b n ) = 2b -1/k n = 2γ -1/k (nh n ) 1/(βk) , a sufficient condition on h n guaran- teeing (4) is that nh k n → 0.
The three examples considered above are informative on the interplay between the tails of f 0 and the choice of h n . For distribution with light enough tails, including Gaussian, exponential and polynomials with k ≥ 6, the conditions on h n required by (H3) are already fulfilled when assuming (H2). Consequently, the optimal bandwidth which has order n -1/5 is included by our set of assumptions. In contrast, as soon as k < 6 in the polynomial case, we have the additional condition that nh k n → 0.

Numerical illustrations

In all the simulation experiments, we have set ∆ n = 1/n, K(x) = exp(-x 2 /2)/ √ 2π and q(x) = t ν ((x -µ q )/σ q ) where t ν is the density of a Student-t distribution with ν = 3 degrees of freedom, µ q = 0 and σ q = 100. With such a large variance and heavy tails, this choice of q is non informative. We made µ q and σ q vary but the results were very similar, suggesting that the choice for q has little effect in practice (at least for light-tailed distribution). In all the experiments but those in Section 4.1, the bandwidth was chosen according to the well known rule of thumb given in [START_REF] Silverman | Density estimation for statistics and data analysis[END_REF] (p. 48, equation (3.31)). The choice of the bandwidth is discussed in Section 4.1. In Section 4.2, we study the behavior of the fitness coefficient and the performance of the estimators with respect to the amount of evidence of the model. In Section 4.3, we use the LR method for protection against misspecification. All the numerical experiments were carried out with the R software.

Sensitivity to the bandwidth: comparison of the fitness coefficient and the OS coefficient

In this section, we study how a change in the bandwidth affects the fitness coefficient and the OS coefficient. We reanalyze the data of Olkin and Spiegelman [START_REF] Olkin | A semiparametric approach to density estimation[END_REF]

f θ (x) = (x -µ)/σ -exp((x -µ)/σ),
where θ = (µ, σ), µ is a real location parameter and σ > 0 a dispersion parameter obeying var f θ = π 2 σ 2 /6 and E f θ = µ -σγ, where γ ≈ 0.58 is the Euler-Mascheroni constant. The maximum likelihood estimator is given by θn ≈ (62.1, 5.4). Let h denote the bandwidth. In [START_REF] Olkin | A semiparametric approach to density estimation[END_REF], it was arbitrarily chosen h = 0.7s, where s is the standard deviation of the data. This yields αOS n ≈ 0.8. But if h ≈ 0.43s, h ≈ 0.37s or h ≈ 0.21s then one gets αOS n ≈ 0. All the above values for h are grounded by well-known bandwidth selection methods, see the textbook [START_REF] Silverman | Density estimation for statistics and data analysis[END_REF] (p. 47, eqn (3.30) and p. 48, eqn (3.31)) and [START_REF] Sheather | A reliable data-based bandwidth selection method for kernel density estimation[END_REF]. By contrast, the fitness coefficient yields αn ≈ 1. These findings are summarized in Figure 2 (a) , where the coefficients are represented as functions of h. We see that the OS coefficient is sensitive to the choice of the bandwidth: a slight difference in h can yield a large difference in αOS n = αOS n (h) especially in the range 0.4 ≤ h ≤ 0.8. On the opposite, the fitness coefficient is more robust: the estimated value for αn (h) remains close to one in a large range for h. In Figure 2 (a), the fitness coefficient and the OS coefficient contradict each other and no more credit can be given to any one of them because the ground truth is unknown.

To observe the behavior of the coefficients when the model is known to be true, we simulated n = 400 observations according to a Gumbel distribution with mean and standard deviation equal to those of the wind speed data, that is, 59.1 and 6.55 respectively. The results are shown in Figure 2 (b). One has αn (h) ≈ 1 whatever h while αOS n (h) ≤ 0.2 for all h chosen by the bandwidth selection methods of the literature. These results tend to indicate that the fitness coefficient is consistent but the OS coefficient is not. Let us note that Figure 2 (a) and (b) are similar, making the Gumbel model plausible. The difference spotted in the range 0 ≤ h ≤ 0.2 can be explained by the ties of the wind speed data. (When h is small, one can see that (1) is close to the likelihood of a Bernouilli trials experiment, the maximizer of which is given by the proportion of untied observations, here one half.)

Whenever the model is wrong, we found on simulations that for most reasonable (that is, found in the literature as above) values of h, the values of the coefficients are close to zero, as expected. This is illustrated in Figure 2 (c): the model is still Gumbel, but the n = 400 data points were generated according to a Gaussian distribution with mean 59.1 and standard deviation 6.55.

Performance of the methods when the model and the truth intertwine

Parametric estimators perform better than kernel density estimators when the model is approximately true, but worse otherwise. Can the semiparametric combination be uniformly best? Does the fitness coefficient goes to unity as the model approaches the truth?

To get some insight, the following numerical experiment was done. We generated samples of size n = 400 according to a density f t , for several values t in a certain index set, representing the "distance" between f t and the model. Two settings have been tested.

Setting 1 The parametric model is given by f θ ∼ N (θ, 1) and the curve of true distributions is given by f t ∼ N (0, (1 + t) 2 ). The intersection between the model {f θ } and the family {f t } is given by f 0 ∼ N (0, 1); that is, θ = t = 0.

Setting 2 The parametric model is given by f θ ∼ N (0, θ 2 ) and the curve of true distributions is given by f t ∼ N (t, 1). The intersection between the model {f θ } the family {f t } is given by a N (0, 1) as well.

For each t, we compute the maximum likelihood estimator, the standard kernel density estimator, the fitness coefficient, the OS coefficient, and the semiparametric density estimator. The semiparametric density estimator is the combination between the maximum likelihood estimator and the kernel density estimator where the mixing coefficient can be either the fitness coefficient (LR method) or the OS coefficient (OS method). To assess the performances of the estimators, we compute the L2-distance to f t . The above procedure is repeated 500 times so that the errors are averaged over the repetitions.

Figure 3 summarizes the results for the first setting. The errors for the parametric estimator, shown in Figure 3 (a), shrink sharply as the model and the truth intersect. The error for the nonparametric estimator is approximately constant. We see that the OS method performs poorly: it fails to give accurate estimates near the truth. This behavior is explained in Figure 3 (b), where we see that the values of the OS coefficient barely exceed 0.1. This is not the case for the fitness coefficient; the values stretch entirely the range [0, 1] and are consistent with the proximity between the truth and the parametric model. As a consequence, coming back to Figure 3 (a), the error of the LR method is near the minimum of the parametric and nonparametric errors. This means that, in practice, however close our parametric model is to the truth, we never lose by choosing the LR method. Even more interestingly is the fact that in the region where the parametric and the nonparametric estimators perform similarly, the LR method performs better: this corresponds to the values t ≈ -0.10 and t ≈ 0.15. This fact is clearly seen in Figure 3 (c) which pictures the averaged error integrated in the interval [-t, t]: the LR method always has the lowest curve.

The results for n = 50, 100, 200 and for setting 2 are similar and not shown here to limit the length of the paper.

Application to multivariate density estimation

It is well known that building accurate multivariate parametric models is an uncertain and difficult task. One way of addressing this problem consists of decomposing the target density f 0 into a copula c and the marginal densities f 1 , . . . , f d , that is,

f (x 1 , . . . , x d ) = c(F 1 (x 1 ), . . . , F d (x d ))f 1 (x 1 ) • • • f d (x d )
(here the {F j } stand for the distribution functions). This decomposition, also known as Sklar's theorem, is unique provided that the {F j } are continuous; for more details about copulas, see e.g. [START_REF] Genest | Everything you always wanted to know about copula modeling but were afraid to ask[END_REF] or the books [START_REF] Nelsen | An introduction to copulas[END_REF][START_REF] Joe | Dependence Modeling with Copulas[END_REF]. The copula is assumed to belong to a parametric model {c ξ , ξ ∈ Ξ} and the true underlying parameter ξ is estimated [START_REF] Genest | A semiparametric estimation procedure of dependence parameters in multivariate families of distributions[END_REF] by

ξ = arg max ξ∈Ξ n i=1 log c ξ R i,1 n , . . . , R i,d n ,
where R i,j is the rank of X i,j among (X 1,j , . . . , X n,j ) and X i,j stands for the j-th coordinate of the i-th observation. The marginals are estimated in a separate step. If one of the marginals is misspecified, the estimation of the joint distribution is biased. In the following, a computer experiment illustrates that the LR method can help to reduce this bias by avoiding misspecification.

We have generated datasets of size n = 25, 50, 100, 150, . . . , 500 with a copula of the form (a so-called Gumbel copula)

C ξ (u 1 , u 2 ) = exp -(-log u 1 ) ξ + (-log u 2 ) ξ 1/ξ , ξ ≥ 1, (5)
with ξ = 3 and marginals

f 1 ∼ E(2), f 2 ∼ W (2, 1/2
) where E(λ) is an exponential distribution with mean 1/λ and W (a, b) is a Weibull distribution with shape a > 0 and scale b > 0, that is,

f 2 (x; a, b) = a b x b a-1 exp - x b a , x > 0.
For each of the simulated datasets, the copula parameter ξ was estimated as mentioned above and the marginals were estimated under three scenarios. In the first scenario we estimate them nonparametrically with the standard kernel density estimator. In the second scenario we do as if both marginals were exponentially distributed and compute the maximum likelihood estimator.

In the third scenario we form the convex combination with the maximum likelihood estimator and the standard kernel density estimators, where the mixing coefficient is the fitness coefficient (LR method).

The results for n = 200 and marginal estimation are shown in Figure 4. Figure 4 (a) pictures the estimated densities with the parametric, nonparametric and semiparametric methods for the first marginal, that is, when the parametric model is well specified. We see only two lines because the parametric, semiparametric and the true densities are very similar, indicating that α ≈ 1. Figure 4 (b) corresponds to the misspecified second marginal. Here this is the nonparametric and the semiparametric estimates which are nearly identical, indicating that α ≈ 0. Figure 5 shows the estimation for the bivariate joint density. In Figure 5 (b) we see that one marginal misspecification led to a poor estimation of the joint density, especially in the joint tails. Figure 5 (c) shows the estimated joint density with the nonparametric strategy for the marginals. Drawbacks of nonparametric estimation are easily spotted: the estimated density is multimodal and assumes positive values where it should be null. Visually, the best performance is achieved with the semiparametric strategy in Figure 5 (d). The figures for n = 50, 100, 500 are similar and not shown to limit the length of the paper.

The squared L2-distances between the true joint density and the estimators are shown in Figure 6. The semiparametric strategy performs best for all sample sizes.

Appendix A Proofs of the propositions

We define the mixture likelihood function

L n : [0, 1] → [ -∞, +∞) as L n (α) = n i=1 log αf θn (X i ) + (1 -α) f LR n,i .
The fitness coeficient αn in (1) is then defined as a maximizer of L n (α) over [0, 1].

A.1 Proof of Proposition 1

The presence of ∆ n q(X i ) in f LR n,i allows for L n (α) > -∞ for all α ∈ [0, 1). If for all i, f θn (X i ) > 0, i.e., L n (1) > -∞, then L n is continuos on [0, 1] and the extreme value theorem yields the existence of αn . Else, if L n (1) = -∞, there exists δ > 0 such that sup α∈[0,1-δ] L n (α) > sup α∈(1-δ,1] L n (α), meaning that the maximum is over [0, 1 -δ] and exists in virtue of the extreme value theorem. Whenever f θn (X i ) is not identically equal to f LR n,i for all i = 1, . . . , n, the function L n is strictly concave and so comes the unicity.

A.2 Proof of Proposition 3

Let 0 < < 1. By assumption, there exists à > 0, such that for all |x| > Ã, we have

(1 -)g 0 (x) ≤ f 0 (x) ≤ g 0 (x)(1 + ).
For t > 0 small enough (i.e., taking any t < inf |x|≤ Ã f 0 (x) implies that {|x| ≤ Ã} ⊂ {f 0 (x) > t}, or equivalently that S c t ⊂ {|x| > Ã}), it holds

S c t f 0 (x) dx = f 0 (x)≤t f 0 (x) dx ≤ (1 + ) (1-)g 0 (x)≤t g 0 (x) dx.
Consequently, we obtain that

S c t f 0 (x) dx ≤ t β c 2 (1 + )/(1 -) β . Remark that λ(S t ) ≤ λ({|x| ≤ Ã}) + λ({|x| > Ã} ∩ S t ) ≤ λ({|x| ≤ Ã}) + (1+ )g 0 (x)>t dx,
which is enough to obtain the last point of (H3).

Suppose that 0 < h n ≤ 1. By enlarging à (i.e., taking à := Ã+ √ d), we have, for all |x| > à and u ∈

[-1, 1] d , ( 1
-)g 0 (x + h n u) ≤ f 0 (x + h n u) ≤ g 0 (x + h n u)(1 + ). Let b n = γ(nh d n ) -1/β and A 1 = max(A, Ã). As soon as |x| ≤ A 1 , sup u∈[-1,1] d f 0 (x + h n u) f 0 (x) ≤ f 0 R d inf |x|≤A 1 f 0 (x)
.

Otherwise,

sup |x|>A 1 , f 0 (x)>bn sup u∈[-1,1] d f 0 (x + h n u) f 0 (x) ≤ (1 + ) (1 -) sup |x|>A 1 , (1+ )g 0 (x)>bn sup u∈[-1,1] d g 0 (x + h n u) g 0 (x) .
We conclude by remarking that the previous is bounded, by (1 + )C 2 /(1 -).

Appendix B Proof of Theorem 2

Theorem 2 follows from the application of two high-level results, corresponding respectively to the well-specified and misspecified case. Both high-level results take place in the following general framework: given a triangular sequence of non-negative real numbers ξ n,i , i = 1, . . . , n, n ≥ 1, we consider the mixture likelihood function given by

L n (α) = n i=1 log αf θn (X i ) + (1 -α)ξ n,i .
Here the sequence (ξ n,i ) is left unspecified in order to highlight the assumptions that we need on the nonparametric part. This random sequence could be the non-parametric estimator evaluated at X i , i.e., fn (X i ), the LOO estimate f LOO n,i

or the LR estimate f LR n,i with ∆ n > 0. In this slightly new context, we define αn as αn ∈ argmax α∈[0,1] L n (α).

In both cases, respectively, the misspecified and well-specified case, the approach taken is similar. We compare the empirical likelihood of the mixture to the one of the parametric estimate (in the well-specified case) or the nonparametric estimate (in the misspecified case).

In the proofs below, it is convenient to introduce the normalized version of L n (α), given by

Ln (α) = n i=1 log αf θn,i + (1 -α)ξ n,i f 0,i ,
where, for any real valued function f , we have introduced the short-cut notation f i for f (X i ).

B.1 Case (i) : the model is well-specified

We are based on some restricted mean quadratic error

Q (p) n (S) = n i=1 f θn,i -f 0,i f 0,i 2
1 {X i ∈S} , Q (np) n (S) = n i=1 ξ n,i -f 0,i f 0,i 2
1 {X i ∈S} ,
and some averaged linear error

M (p) n = n i=1 f θn,i -f 0,i f 0,i , M (np) n = n i=1 ξ n,i -f 0,i f 0,i .
The proof of the following theorem is given in Section B.3.1.

Theorem B.1. Suppose that f 0 ∈ P and let S ⊂ R d and b > 0 be such that for all x ∈ S, f 0 (x) > b. If the following convergences hold in probability, as n → ∞,

f θn -f 0 S → 0, max i=1,...,n, : X i ∈S |ξ n,i -f 0,i | → 0, (6)
Q (p) n (S) Q (np) n (S) → 0, |M (p) n | + |M (np) n | Q (np) n (S) → 0, (7)
then, αn → 1 as n → ∞, in probability.

We now verify the conditions of the previous theorem when ξ n,i is the LR sequence f LR n,i and when (H1), (H2), (H3), (A1), (A2) and nh d n ∆ n → 0 are fulfilled.

Condition [START_REF] Claeskens | Goodness of fit via non-parametric likelihood ratios[END_REF]. The first convergence in [START_REF] Claeskens | Goodness of fit via non-parametric likelihood ratios[END_REF] holds in virtue of ( 21) established in Section C. For the second one, it holds that

f LR n,i = n n -1 fn (X i ) - K(0) nh d n + ∆ n q(X i ).
Applying the first statement of Proposition C.1 in [START_REF] Portier | On the weak convergence of the empirical conditional copula under a simplifying assumption[END_REF] (which is a consequence of Theorem 2.1 in [START_REF] Giné | Rates of strong uniform consistency for multivariate kernel density estimators[END_REF]), we have, under (H1) and (H2), that fn -

f hn R d = O P | log(h n )| nh d n .
Together with Lemma B.9, we obtain that fn -

f 0 R d = O P ( | log(h n )|/nh d n ) + O(h 2 n ). Con- sequently, we get max i=1,...,n | f LR n,i -f 0,i | ≤ n n -1 fn -f 0 R d + K(0) nh d n + ∆ n q R d + f 0 R d n -1 = O P | log(h n )| nh d n + h 2 n + ∆ n . ( 8
)
The latter bound indeed goes to 0, in probability, as n → ∞.

Condition [START_REF] Claeskens | Model selection and model averaging[END_REF]. We proceed as follows, with ξ n,i = f LR n,i : All this together implies that (7) holds true.

B.2 Case (ii) : the model is misspecified

When the model is misspecified, i.e., f 0 / ∈ P, the following high-level conditions are enough to ensure the convergence in probability α n → 0. These conditions are easily implied by (A1), (H1), (H2), (H3) and [START_REF] Hjort | Hybrid combinations of parametric and empirical likelihoods[END_REF] holds) with Θ compact, that the envelop F Θ is such that E[log(F Θ,1 )] < +∞, and that for every x ∈ R d , θ → f θ (x) is a continuous function defined on Θ. Suppose that there exists β ∈ (0, 1] and c > 0 such that, as t → 0, S c t f 0 (x) ≤ ct β , and q : R d → R + such that

| log(∆ n )| 1/β ( | log(h n )|/nh d n + h 2 n ) as demonstrated below. The proof of this theorem is given in Section B.3.2. Theorem B.2. Suppose that f 0 / ∈ P and E[| log(f 0,1 )|] < ∞, that the class P is Glivenko- Cantelli (i.e.,
E[| log(q(X 1 ))|] < ∞, ξ n,1 ≥ ∆ n q 1 a
.s., and

| log(∆ n )| 1/β max i=1,...,n |ξ n,i -f 0 (X i )| → 0, then, αn → 0, as n → ∞, in probability.
We already argued that ( 20) is implied by (A1). The continuity of f θ is deduced from the continuity of log(f θ ) provided by (A1). The bound given in [START_REF] D'agostino | Goodness-of-Fit Techniques[END_REF] Because f 0 ∈ P, it holds that f θn,i > 0 for all i = 1, . . . , n, which guarantees the existence of a maximizer αn (as explained in the proof of Proposition 1). By definition of θn , n i=1 log (f 0,i ) ≤ n i=1 log f θn,i . Consequently, max α∈[0,1] Ln (α) ≥ 0 and for every > 0, the event max α∈[0,1-] Ln (α) < 0 implies that αn > 1 -. Thus, let ∈ (0, 1), the proof will be completed by showing that with probability going to 1,

together with | log(∆ n )| 1/β ( | log(h n )|/nh d n + h 2 n ) → 0 implies the stated convergence with ξ n,i = f LR n,i .

B.3 Proofs of the high-level Theorems

sup α∈[0,1-] Ln (α) < 0. A useful notation in the following is xi,n = 1 + α(f θn,i -f 0,i ) f 0,i + (1 -α)(ξ n,i -f 0,i ) f 0,i .
A useful technical detail is there exists a sequence n → 0 such that the event

{ max i=1...,n : X i ∈S f 0,i |x i,n -1| ≤ n }
has probability going to 1 as n → ∞. This is a consequence of ( 6). As we are establishing a result in probability, we can further suppose that this event is realized.

A key step in our approach is the following inequality, reminiscent of the Taylor development of the logarithm around 1,

log(x) -(x -1) ≤ -1 4 (x -1) 2 if 1/2 < x < 3/2 0 else ,
which might be derived by studying the concerned function. This kind of inequality is commonly used for studying likelyhood methods [START_REF] Grenander | Abstract inference[END_REF][START_REF] Murphy | Consistency in a proportional hazards model incorporating a random effect[END_REF]. Applied to xi,n , it gives

Ln (α) -(αM (p) n + (1 -α)M (np) n ) = n i=1 (log (x i,n ) -(x i,n -1)) ≤ - 1 4 n i=1 (x i,n -1) 2 1 {|x i,n -1|<1/2} ≤ - 1 4 n i=1 (x i,n -1) 2 1 {X i ∈S, |x i,n -1|<1/2} .
Note that whenever X i ∈ S, because it holds f 0,i |x i,n -1| ≤ n , we have (for n small enough) that |x i,n -1| < 1/2. This means that, for all i = 1, . . . , n, 1 {X i ∈S} ≤ 1 {|x i,n -1|<1/2} , and it follows

Ln (α) -(αM (p) n + (1 -α)M (np) n ) ≤ - 1 4 n i=1 (x i,n -1) 2 1 {X i ∈S} = - 1 4 (1 -α) 2 Q (np) n (S) + α 2 Q (p) n (S) + 2α(1 -α)U n ≤ - 1 4 (1 -α) 2 Q (np) n (S) 1 - 2α|U n | (1 -α)Q (np) n (S)
, where

U n = n i=1 (f θn,i -f 0,i )(ξ n,i -f 0,i ) f 2 0,i 1 {X i ∈S} .
Bounding the right-hand side with respect to α ∈ [0, 1 -] gives

sup α∈[0,1-] { Ln (α) -(αM (p) n + (1 -α)M (np) n )} ≤ - 1 4 2 Q (np) n (S) 1 -2 -1 |U n | Q (np) n (S)
.

By assumption, we have that

Q (p) n (S)/Q (np)
n (S) → 0 in probability. From the Cauchy-Schwartz inequality we get that

|U n | ≤ Q (np) n (S)Q (p) n (S), leading to |U n |/Q (np)
n (S) → 0, in probability. Consequently, we obtain that sup

α∈[0,1-] Ln (α) ≤ - 1 4 2 Q (np) n (S) 1 -2 -1 |U n | Q (np) n (S) -4 |M (p) n | + |M (np) n | 2 Q (np) n (S)
.

The term between brackets goes to 1, in probability, implying that for every δ > 0, with probability going to 1,

sup α∈[0,1-] Ln (α, θ) ≤ - 1 4 2 Q (np) n (S)(1 -δ).
Hence it remains to note that, by [START_REF] Claeskens | Model selection and model averaging[END_REF], with probability going to 1, Q

n (S) > 0.

B.3.2 Proof of Theorem B.2

Note that αn ∈ argmax α∈[0,1] Ln (α) exists because ξ n,i > 0 for all i, as explained in the proof of Proposition 1. Let > 0. The proof requires to show that with probability going to 1, αn < . This event is realized as soon as max α∈[ ,1] Ln (α) < Ln (0). We analyse both terms separately. First we show that Ln (0) → 0, in probability, and then that there exists δ > 0 such that, with probability going to 1,

sup α∈[ ,1] Ln (α) ≤ -δ. (9)
Let η > 0, b n = (η/| log(∆ n )|) 1/β and c n = max i=1,...,n |ξ n,i -f 0,i |. We assume further that b n + c n < 1 and ∆ n < 1. We have

| Ln (0)| ≤ n -1 n i=1 log ξ n,i f 0,i 1 {f 0,i >bn} + n -1 n i=1 log ξ n,i f 0,i 1 {f 0,i ≤bn} ≤ n -1 n i=1 log ξ n,i f 0,i 1 {f 0,i >bn} + n -1 n i=1 |log(∆ n q i )| 1 {f 0,i ≤bn} + |log(f 0,i )| 1 {f 0,i ≤bn} ≤ n -1 n i=1 log ξ n,i f 0,i 1 {f 0,i >bn} + |log(∆ n )| n -1 n i=1 1 {f 0,i ≤bn} + n -1 n i=1 (|log(q i )| + |log(f 0,i )|)1 {f 0,i ≤bn}
The expectation of the term in the middle is bounded by

| log(∆ n )|P(f 0 (X 1 ) ≤ b n ) of order | log(∆ n )|b β n = η,
by assumption. The corresponding term goes to 0, as η is arbitrarily small. The expectation of the term in the right is smaller than E[(| log(q 1 )| + | log(f 0,1 )|)1 {f 0,i ≤bn} ], which goes to 0 because | log(q 1 )| and | log(f 0,1 )| are integrable. Hence it remains to obtain that the term in the left goes to 0. The mean-value theorem gives

n -1 n i=1 log ξ n,i f 0,i 1 {f 0,i >bn} ≤ max i=1,...,n |ξ n,i -f 0,i | inf i=1,...,n : f 0,i >bn inf t∈[0,1] tξ n,i + (1 -t)f 0,i ≤ c n b n -c n → 0.
Now we establish ( 9) by obtaining one-sided inequalities. Take

b n = (1/| log(∆ n )|) 1/β , suppose that b n + c n < 1,
and use the monotonicity of the logarithm, to get that

n -1 n i=1 log αf θn,i + (1 -α)ξ n,i f 0,i 1 {f 0,i ≤bn} ≤ n -1 n i=1 log F Θ,i + b n + c n f 0,i 1 {f 0,i ≤bn} ≤ n -1 n i=1 | log F Θ,i + 1 f 0,i |1 {f 0,i ≤bn} .
Taking the expectation, we find a bound in E[| log

F Θ,1 +1 f 0,1
|1 {f 0,1 ≤bn} ] which goes to 0 as n → ∞ in virtue of the Lebesgue dominated convergence theorem. Then, taking 0 < η < 1, it holds that

Ln (α) ≤ n -1 n i=1 log αf θn,i + (1 -α)ξ n,i f 0,i 1 {f 0,i >bn} + o p (1) ≤ n -1 n i=1 log α(f θn,i + ηf 0,i ) + (1 -α)ξ n,i f 0,i 1 {f 0,i >bn} + o p (1)
The first term in the right-hand side is decomposed according to

n -1 n i=1 log α(f θn,i + ηf 0,i ) + (1 -α)ξ n,i α(f θn,i + ηf 0,i ) + (1 -α)f 0,i 1 {f 0,i >bn} + n -1 n i=1 log α(f θn,i + ηf 0,i ) + (1 -α)f 0,i f 0,i 1 {f 0,i >bn} .
By the mean value theorem, the term on the left is bounded by

(1 -α) max i=1,...,n |ξ n,i -f 0,i | (ηb n ) ∧ (b n -c n ) ,
which goes to 0, by assumption. For the term on the right, notice that {α(

f θ + ηf 0 ) + (1 -α)f 0 : α ∈ [ , 1], θ ∈ Θ} is Glivenko-Cantelli with envelop F Θ + 2f 0 .
Then applying Theorem 3 in [START_REF] Van Der Vaart | Preservation theorems for Glivenko-Cantelli and uniform Glivenko-Cantelli classes[END_REF], the class formed by log(α(

f θ + ηf 0 ) + (1 -α)f 0 ) is still Glivenko-Cantelli. Since for all θ ∈ Θ, α ∈ [ , 1], log( ηf 0 ) ≤ log(α(f θ + ηf 0 ) + (1 -α)f 0 ) ≤ log(F Θ + 2f 0 + 1), the function | log( ηf 0 )| + log(F Θ + 2f 0 + 1
) is an integrable envelop. Using again Theorem 3 in [START_REF] Van Der Vaart | Preservation theorems for Glivenko-Cantelli and uniform Glivenko-Cantelli classes[END_REF], the class formed by log(α(

f θ + ηf 0 ) + (1 -α)f 0 )1 {f 0 >b} , θ ∈ Θ, α ∈ [ , 1], 0 < b < 1, is still Glivenko-Cantelli with the same envelop. This implies that sup α∈[ ,1], θ∈Θ n -1 n i=1 log α(f θ,i + ηf 0,i ) + (1 -α)f 0,i f 0,i 1 {f 0,i >bn} -E log α(f θ,1 + ηf 0,1 ) + (1 -α)f 0,1 f 0,1 1 {f 0,1 >bn} → 0.
The integrability of the envelop and the fact that b n → 0 implies that

sup α∈[ ,1], θ∈Θ E log α(f θ,1 + ηf 0,1 ) + (1 -α)f 0,1 f 0,1 1 {f 0,1 ≤bn} → 0. It remains to use the inequality log(x) ≤ 2( √ x -1) to obtain that sup α∈[ ,1], θ∈Θ E log m θ,α,η f 0,1 ≤ sup α∈[ ,1], θ∈Θ 2 ( m θ,α,η f 0 -f 0 ) dλ where m θ,α,η = α(f θ + ηf 0 ) + (1 -α)f 0 . Since sup α∈[ ,1], θ∈Θ | (m θ,α,η -m θ,α,0 )f 0 dλ| = √ η, we get, using that √ a + b ≤ √ a + √ b, a ≥ 0, b ≥ 0, sup α∈[ ,1], θ∈Θ E log m θ,α,η f 0,1 ≤ 2 √ η + 2 sup α∈[ ,1], θ∈Θ ( m θ,α,0 f 0 -f 0 ) dλ = 2 √ η - inf α∈[ ,1], θ∈Θ ( √ m θ,α,0 -f 0 ) 2 dλ.
Using standard results about the Hellinger distance [START_REF] Pollard | Asymptopia[END_REF] (chapter 3) we obtain

sup α∈[ ,1], θ∈Θ E log m θ,α,η f 0,1 ≤ 2 √ η -(1/4) inf α∈[ ,1], θ∈Θ α 2 |f θ -f 0 | dλ 2 ≤ 2 √ η -( 2 /4) inf θ∈Θ |f θ -f 0 | dλ 2 .
Since f 0 / ∈ P and by the continuity assumption on f θ , it holds that inf θ∈Θ |f θ -f 0 |dλ > 0. Then, as η is arbitrary, the proof of (9) is complete.

B.4 Linear and quadratic error of parametric and nonparametric estimate

Important tools for dealing with the terms involving f LR n,i are coming from U -statistic theory. We call U -statistic of order p with kernel w : R p → R, any quantity of the kind

i 1 ,...,ip∈D w(X i 1 , . . . , X ip ),
where the summation is taken over the subset D formed by the (i 1 , . . . , i p ) ∈ {1, . . . , n} p such that i k = i , ∀k = . The number of terms in the summation is then n(n-1) . . . (n-p+1). When the kernel w is such that, for every k ∈ {1, . . . , p}, E[w(X 1 , . . . , X p ) | X 1 , . . . X k-1 , X k+1 , . . . , X p ] = 0, it is called a degenerate U -statistic. In the proofs, we shall rely on the so-called Hajek decomposition [START_REF] Van Der | Asymptotic statistics[END_REF]Lemma 11.11].

To establish the two following lemmas, Lemma B.3 and Lemma B.4, we are based on (H1), (H2) and (H3). One might note that the expressions (a) or (b) in (H2) on the kernel are not used in any of these lemmas.

Lemma B.3. Under assumptions (H1), (H2) and (H3), if nh d n ∆ n → 0, for any δ > 0 and any set S ⊂ R d such that inf x∈S f 0 (x) > b, we have with probability going to 1,

h d n n i=1 f LR n,i -f 0,i f 0,i 2
1 {X i ∈S} ≥ (1 -δ)v K λ(S).
where v K = K(u) 2 du.

Proof. Note that

E f LR n,i | X i = (n -1) -1 n j =i E h -d n K X i -X j h n | X i + ∆ n q i = f hn,i + ∆ n q i .
The proof follows from the decomposition

n i=1 f LR n,i -f 0,i f 0,i 2
1 {X i ∈S} = A n + B n + 2C n ,
where

A n = n i=1   f LR n,i -E f LR n,i | X i f 0,i   2 1 {X i ∈S} , B n = n i=1 f hn,i + ∆ n q i -f 0,i f 0,i 2
1 {X i ∈S} , C n = n i=1 f LR n,i -E f LR n,i | X i (f hn,i + ∆ n q i -f 0,i ) f 2 0,i 1 {X i ∈S} .
We will show that h d n A n → v K λ(S), in probability and that h d n C n → 0, in probability. This will be enough as B n ≥ 0, almost surely.

Proof that h d n A n → v K λ(S) in probability. Introduce the notation, for any h > 0,

a h (x, y) = K h (x -y) -f h (x) f 0 (x) , u h (x, y, z) = a h (x, y)a h (x, z)1 {x∈S} .
Developing, we find

A n = (n -1) -2 n i=1 n j =i n k =i u hn (i, j, k),
where u hn (i, j, k) is as short-cut for u hn (X i , X j , X k ). We treat A n relying on the Hajek projection of U -statistics. Up to a centering term, E [u hn (i, j, k) | X j , X k ], the U -statistic A n is a degenerate U -statistic. In the following we voluntary introduce this centering term in the summation to handle separately a degenerate U-statistic and another summation with less indices. By introducing, for any h > 0,

v h (j, k) = E [u h (i, j, k) | X j , X k ] , w h (i, j, k) = u h (i, j, k) -v h (j, k),
we obtain

A n = (n -1) -2 n i=1 n j =i n k =i w hn (i, j, k) + (n -1) -2 n i=1 n j =i n k =i v hn (j, k) = (n -1) -2 n i=1 n j =i n k =i, k =j w hn (i, j, k) + (n -1) -2 n i=1 n j =i
w hn (i, j, j)

+ (n -1) -2 n i=1 n j =i n k =i v hn (j, k). (10)
Treatment of the first term in [START_REF] Durbin | Weak convergence of the sample distribution function when parameters are estimated[END_REF]. Note that w hn (i, j, k) defines a degenerate U -statistic, i.e.,

E[w hn (i, j, k) | X i , X j ] = E[w hn (i, j, k) | X i , X k ] = E[w hn (i, j, k) | X j , X k ] = 0. Note that n i=1 n j =i n k =i, k =j w hn (i, j, k) = n i=1 n j>i n k>j w hn (i, j, k),
where w h is the symmetrized version of w h , i.e., for any triplet (x 1 , x 2 , x 3 ) of w h (x 1 , x 2 , x 3 ) = σ w h (x σ(1) , x σ(2) , x σ(3) ) where the sum is over all the 3! possible permutations of the set {1, 2, 3}. Using that the U -statistic with kernel w hn is degenerate, some algebra gives that

E     (n -1) -2 n i=1 n j =i n k =i, k =j w hn (i, j, k)   2   = (n -1) -4 n i=1 n j>i n k>j E[w hn (i, j, k) 2 ] = O(n -1 )E[w hn (1, 2, 3) 2 ].
We have, using Minkowski's inequality and the definition of the conditional expectation, that

E[w hn (1, 2, 3) 2 ] ≤ 3! E[w hn (1, 2, 3) 2 ] ≤ 3! E[u hn (1, 2, 3) 2 ].
Consequently, in virtue of [START_REF] Giné | Rates of strong uniform consistency for multivariate kernel density estimators[END_REF] in Lemma B.7, we have shown that

E     (n -1) -2 n i=1 n j =i n k =i, k =j w hn (i, j, k)   2   = O(n -1 h -2d n ).
The previous rate, multiplied by h 2d n , goes to 0, hence, this term is negligible. Treatment of the second term in [START_REF] Durbin | Weak convergence of the sample distribution function when parameters are estimated[END_REF]. We continue the study of A n by considering

(n -1) -2 n i=1 n j =i w hn (i, j, j) = (n -1) -2 n i=1 n j =i (w hn (i, j, j) -E[u hn (i, j, j) | X i ] + E[u hn (1, 2, 2)]) + (n -1) -1 n i=1 (E[u hn (i, j, j) | X i ] -E[u hn (1, 2, 2)]).
The first term is a degenerate U -statistic of order 2 whose order 2 moments satisfy

E     (n -1) -2 n i=1 n j =i (w hn (i, j, j) -E[u hn (i, j, j) | X i ] + E[u hn (1, 2, 2)])   2   = O(n -2 )E[u hn (1, 2, 2) 2 ] = O(n -2 h -3d n ).
This is obtained by following exactly the same lines as in the treatment of the U -statistic w n and using [START_REF] Hall | On Kullback-Leibler loss and density estimation[END_REF] in Lemma B.7. As n -2 h -3d n × h 2d n → 0, the previous term is negligible. The second term is a sum of centred independent random variables with variance smaller than, in virtue of [START_REF] Genest | A semiparametric estimation procedure of dependence parameters in multivariate families of distributions[END_REF] in Lemma B.7,

n(n -1) -2 E E[u hn (1, 2, 2) | X 1 ] 2 = O(n -1 h -2d n ).
This is the same rate as the rate obtained for the (negligible) U -statistic of order 3 with kernel w n .

Treatment of the third term in [START_REF] Durbin | Weak convergence of the sample distribution function when parameters are estimated[END_REF]. The study of A n continues by considering

(n -1) -2 n i=1 n j =i n k =i v hn (j, k) = (n -1) -2 n i=1 n j =i v hn (j, j) + (n -1) -2 n i=1 n j =i n k =i, k =j v hn (j, k) = (n -1) -1 n j=1 v hn (j, j) + (n -1) -2 (n -2) n j=1 n k =j v hn (j, k).
The term associated with double summation over j and k is a degenerate U -statistic, as

E[v hn (j, k) | X k ] = E[v hn (j, k) | X j ] = 0.
Consequently, following the same lines as in the treatment of the first term of A n , and using [START_REF] Hjort | Locally parametric nonparametric density estimation[END_REF] in Lemma B.7, we get

E     (n -1) -2 (n -2) n j=1 n k =j v hn (j, k)   2   = O(1)E[v hn (1, 2) 2 ] = O(h -d n )
which goes to 0, when multiplied by h 2d n . The remaining term is a sum of independent and identically distributed random variables. We have, by computing the variance of the centred average,

(n -1) -1 n j=1 (v hn (j, j) = O P (n -1/2 ) Ev hn (1, 1) 2 + n(n -1) -1 E[v hn (1, 1)],
where the first term, using [START_REF] Habbema | A stepwise discriminant analysis program using density estimation[END_REF] in Lemma B.7, is O(n -1/2 h -d n ) which goes to 0 when multiplied by h n . The dominating term is in fact the last one, as by Lemma B.8, it holds that

h d n E[v hn (1, 1)] → v K λ(S).
Proof that h d n C n → 0 in probability. We are based on similar decompositions as for A n involving U -statistics. Let hn (x) = (f hn (x) -f 0 (x) + ∆ n q(x))/f 0 (x) and note that in virtue of Lemma B.9, it holds

hn S ≤ b -1 h 2 n g R d u 2 2 K(u) du + ∆ n q R d .
Then

C n = (n -1) -1 n i=1 n j =i (a hn (i, j) hn,i 1 {X i ∈S} -b hn (j)) + n i=1 b hn (i), with b hn (j) = E[a hn (i, j) hn,i 1 {X i ∈S} | X j ].
The term on the left is a degenerate U -statistic for which it holds

E     (n -1) -1 n i=1 n j =i (a hn (i, j) hn,i 1 {X i ∈S} -b hn (j))   2   = O(1)E[a hn (1, 2) 2 2 hn,1 1 {X 1 ∈S} ].
Using [START_REF] Genest | Everything you always wanted to know about copula modeling but were afraid to ask[END_REF] in Lemma B.7 and the previous bound for hn S , we find

E[a hn (1, 2) 2 2 hn,1 1 {X 1 ∈S} ] = E V hn (X 1 ) 2 hn,1 f 2 0,1 1 {X 1 ∈S} = O(h -d n (h 2 n + ∆ n )),
where V h is defined in [START_REF] Faraway | Implementing semiparametric density estimation[END_REF]. The previous bound multiplied by h 2d n goes to 0. Using that E[a hn [START_REF] Akaike | A new look at the statistical model identification[END_REF][START_REF] Akritas | Pearson-type goodness-of-fit tests: the univariate case[END_REF] hn,1 1 {X 1 ∈S} ] = 0 and ( 16), the variance of the term on the right in C n is smaller than

nE[E[a hn (1, 2) hn,1 1 {X 1 ∈S} | X 2 ] 2 ] ≤ nE[E[|a hn (1, 2)| | X 2 ] 2 ] hn 2 S = O(n(h 4 n + ∆ 2 n ))
which, multiplied by h 2d n , goes to 0 by hypothesis. Hence h d n C n → 0, in probability and the proof is complete.

Lemma B.4. Under assumptions (H1), (H2) and (H3), if nh d n ∆ n → 0, we have

h d n n i=1 f LR n,i -f 0,i f 0,i = o P (1).
Proof. The decomposition is as follows

h d n n i=1 f LR n,i -f 0,i f 0,i = h d n (n -1) -1 n i=1 n j =i K hn (i, j) -f hn,i f 0,i + h d n n i=1 f hn,i -f 0,i f 0,i + h d n ∆ n n i=1 q i f 0,i . (11)
The expectation of the last term is nh d n ∆ n q(x) dx which goes to 0 by assumption. We can now focus on the first and second term of the decomposition. Treatment of the second term in [START_REF] Einmahl | An empirical process approach to the uniform consistency of kernel-type function estimators[END_REF]. Using that K(u) du = 1, the considered term is a centred empirical sum. Using Lemma B.9, its variance is then bounded by

E   h d n n i=1 f hn,i -f 0,i f 0,i 2   ≤ nh 2d n (f hn (x) -f 0 (x)) 2 f 0 (x) dx ≤ nh 2d+4 n g(x) 2 f 0 (x) dx u 2 K(u) du 2 ,
which goes to 0.

Treatment of the first term in [START_REF] Einmahl | An empirical process approach to the uniform consistency of kernel-type function estimators[END_REF]. Using that K(u) du = 1, one can verify that it is a degenerate U -statistic. Here the variance can not be computed directly because the leading term E

(K hn (1,2)-f hn,1 ) 2 f 2 0,1
is not necessarily finite. Hence we decompose according to the X i in S bn and the others, with b n = ( /nh d n ) 1/β where β is given in (H3) and > 0. We introduce k(x, y) = K hn (x, y) f 0 (x) , and define the linear operator Q P : L 2 (P) → L 2 (P) as

Q P [w](x, y) = w(x, y) -E[w(x, X 1 )] -E[w(X 1 , y)] + E[w(X 1 , X 2 )]. Because E[k(X 1 , y)] = E[k(X 1 , X 2 )] = 1 for all y ∈ R d , one sees that n i=1 n j =i K hn (i, j) -f hn,i f 0,i = n i=1 n j =i Q P (k) i,j = n i=1 n j =i Q P (k1 S bn ) i,j + Q P (k1 S c bn ) i,j .
Because the summation over Q P (k1 S bn ) is a degenerate U -statistics, we get that

E     h d n (n -1) -1 n i=1 n j =i Q P (k1 S bn ) i,j   2   = O(h 2d n )E (K hn (1, 2) -f hn,1 ) 2 f 2 0,1 1 S bn (X 1 ) .
Defining the kernel K = K 2 /v K and fh = f 0 Kh , we obtain

E (K hn (1, 2) -f hn,1 ) 2 f 2 0,1 1 S bn (X 1 ) ≤ E E[K hn (1, 2) 2 | X 1 ] f 2 0,1 1 S bn (X 1 ) = v K h -d n S bn fhn (x) f 0 (x) dx = v K h -d n S bn f (x -h n u) f 0 (x) K(u)du dx ≤ v K h -d n λ(S bn ) sup x∈S bn sup u∈[-1,1] d f (x + h n u) f 0 (x) .
For the term with Q P (k1 S c bn ), we obtain that

E   n i=1 n j =i Q P (k1 S c bn ) i,j   ≤ n(n -1)E[|Q P (k1 S c bn ) 1,2 |] ≤ 4n(n -1)E[|k 1,2 |1 S c bn (X 1 )] = 4n(n -1) S c bn f h (x) dx.
From Lemma B.10, we deduce that S c bn

f h (x) dx ≤ c 2 b β n = c 2 /nh d n .
To conclude, we have shown that there exists a constant C > 0 such that

E   h d n (n -1) -1 n i=1 n j =i Q P (k)   ≤ C h d n λ(S bn ) + nh d n b β n = C h d n λ(S bn ) + .
Invoking (H3) and because is arbitrarily small, the limit as n → ∞ is 0.

Lemma B.5. Under (A1) and (A2), we have

n i=1 f θn,i -f 0,i f 0,i 2
1 {X i ∈S} = O P (1).
Proof. Using ( 21), we have that, with probability going to 1,

n i=1 f θn,i -f 0,i f 0,i 2
1 {X i ∈S} ≤ n -1 n i=1 ˙ (X i ) 2 sup θ∈B(θ 0 ,δ) f θ (X i ) 2 f 2 0,i 1 {X i ∈S} n θn -θ 0 2 2 ≤ ˙ sup θ∈B(θ 0 ,δ) f θ 2 R d b -2 n θn -θ 0 2 2
which is a tight sequence in light of [START_REF] Joe | Dependence Modeling with Copulas[END_REF].

Lemma B.6. Under (A1) and (A2), we have

n i=1 f θn,i -f 0,i f 0,i = O P (1).
Proof. In virtue of (A2), the map θ → log f θ (x) is differentiable at θ 0 , for P -almost every x ∈ R d with derivative ˙ θ 0 (x) (this is obtained in [START_REF] Van Der | Asymptotic statistics[END_REF], in the proof of Theorem 5.39). Using stability properties for the composition, the map θ → f θ (x) = exp(log(f θ (x))) is differentiable at θ 0 , for P -almost every x ∈ R d with derivative ˙ θ 0 (x)f 0 (x). We are in position to apply Lemma 19.31 in [START_REF] Van Der | Asymptotic statistics[END_REF], with r n = √ n and m θ = f θ /f 0 . From the mentioned lemma, as

√ n( θn -θ 0 ) is tight, defining T i (θ) = f θ,i -f 0,i f 0,i -(θ -θ 0 ) T ˙ θ 0 ,i , t(θ) = f θ (x) -f 0 (x) -(θ -θ 0 ) T ˙ θ 0 (x)f 0 (x) dx, we obtain n i=1 {T i ( θn ) -t( θn )} = o P (1).
Actually, recalling that ˙ θ 0 (x)f 0 (x) dx = 0, we find that, for all θ ∈ Θ, t(θ) = 0. Hence, we obtain

n i=1 f θn,i -f 0,i f 0,i = n 1/2 ( θn -θ 0 ) T n -1/2 n i=1 ˙ θ 0 ,i + o P (1).
where the first term is a O P (1).

B.5 Auxiliary results

Recall some definitions, for any h > 0,

V h (X 1 ) = E[(K h (1, 2) -f h,1 ) 2 | X 1 ], (12)
a h (x, y) = K h (x -y) -f h (x) f 0 (x) , u h (x, y, z) = a h (x, y)a h (x, z)1 {x∈S} , as well as the short-cut g(i, j, k) for g(X i , X j , X k ).

Lemma B.7. Under (H1) and (H2), if S ⊂ R d is such that for all x ∈ S, f 0 (x) > b > 0, we have, for any h > 0,

V h (X 1 ) ≤ h d C 0 (13) E E[u h (1, 2, 2) | X 1 ] 2 ≤ h -2d C 1 (14)
E[u h (1, 2, 3) 2 ] ≤ h -2d C 1 (15)
E[|a h (1, 2)| | X 2 ] ≤ 2 (16) E[ E[u h (1, 2, 2) | X 2 ] 2 ] ≤ h -2d C 2 + C 3 (17) E[ u h (1, 2, 2) 2 ] ≤ h -3d C 4 + C 5 (18) E E[u h (1, 2, 3) | X 2 , X 3 ] 2 ≤ h -d C 6 + C 6 ( 19
)
where the constants C k , k = 0, . . . , 7 depends on K and f 0 only.

Proof. Remark that because K is bounded and |K(u)| du < ∞, we have |K(u)| k du < ∞, for any k ≥ 1. Note that, for every h > 0,

V h (X 1 ) ≤ E[K h (2, 1) 2 | X 1 ] ≤ h -d v K f 0 R d .
We obtain [START_REF] Genest | A semiparametric estimation procedure of dependence parameters in multivariate families of distributions[END_REF] by writing

E E[u h (1, 2, 2) | X 1 ] 2 = E V h (X 1 ) 2 f 4 0,1 1 {X 1 ∈S} ≤ h -2d v 2 K f 0 2 R d b -4 .
To establish [START_REF] Giné | Rates of strong uniform consistency for multivariate kernel density estimators[END_REF], note that

E[u h (1, 2, 3) 2 ] = E V h (X 1 ) 2 f 4 0,1 1 {X 1 ∈S} = E E[u h (1, 2, 2)|X 1 ] 2 .
For [START_REF] Grenander | Abstract inference[END_REF], write

E[|a h (1, 2)| | X 2 ] = |K h (x -X 2 ) -f h (x)| dx ≤ K h (x -X 2 ) dx + f h (x) dx = 2.
Inequality [START_REF] Habbema | A stepwise discriminant analysis program using density estimation[END_REF] follows from the lines

E[ E[u h (1, 2, 2) | X 2 ] 2 ] = (K h (x -y) -f h (x)) 2 f 0 (x) 1 {x∈S} dx 2 f 0 (y) dy ≤ 2 K h (x -y) 2 + f h (x) 2 f 0 (x) 1 {x∈S} dx 2 f 0 (y) dy ≤ 2b -2 K h (x -y) 2 + f h (x) 2 dx 2 f 0 (y) dy ≤ 2b -2 h -d v K + f h R d 2 f 0 (y) dy ≤ 4b -2 h -2d v 2 K + f 0 2 R d .
To show [START_REF] Hall | On Kullback-Leibler loss and density estimation[END_REF], write

E[u h (1, 2, 2) 2 ] = E (K h (1, 2) -f h,1 ) 4 f 4 0,1 1 {X 1 ∈S} .
Using that (a + b) 4 ≤ 8(a 4 + b 4 ), we obtain

E[u h (1, 2, 2) 2 ] ≤ 8E K h (1, 2) 4 + f 4 h,1 f 4 0,1 1 {X 1 ∈S} ≤ 8b -4 h -3d f 0 R d K(u) 4 du + f 0 4 R d .
For [START_REF] Hjort | Locally parametric nonparametric density estimation[END_REF], we have

E E[u h (1, 2, 3) | X 2 , X 3 ] 2 = E E K h (3, 1) -f h,3 f 0,3 K h (3, 2) -f h,3 f 0,3 1 {X 3 ∈S} | X 1 , X 2 2 .
We develop and compute bounds for each term. The larger term will be the one associated with the product of the kernels. We have, by Jensen's inequality, for any (y, z) ∈ R d × R d ,

ψ h (y, z) := K h (x -y)K h (x -z) f 0 (x) 1 {x∈S} dx 2 ≤ b -2 K h (x -y)K h (x -z) dx 2 = b -2 h -2d K(u)K((y -z)/h + u) du 2 ≤ b -2 h -2d K(u)K((y -z)/h + u) 2 du.
Then we obtain

E E K h (3, 1) f 0,3 K h (3, 2) f 0,3 1 {X 3 ∈S} | X 1 , X 2 2
= ψ h (y, z)f 0 (y)f 0 (z) dydz ≤ b -2 h -2d K(u)K((y -z)/h + u) 2 f 0 (y)f 0 (z) dydzdu = h -d b -2 K(u)K(v + u) 2 f 0 (z + hv)f 0 (z) dvdzdu

≤ h -d b -2 f 0 R d K(u)K(v + u) 2 dvdu = h -d b -2 f 0 R d v K .
Moreover, as

E K h (3, 1) f 0,3 f h,3 f 0,3 1 {X 3 ∈S} | X 1 , X 2 = K h (x -X 1 )f h (x) f 0 (x) 1 {x∈S} dx ≤ f 0 R d b -1 ,
and

E f h,3 f 0,3 2 1 {X 3 ∈S} | X 1 , X 2 = f h (x) 2 f 0 (x) 1 {x∈S} dx ≤ f 0 R d b -1 .
we finally obtain the result.

Lemma B.8. Under (H1) and (H2), if S ⊂ R d is such that for all x ∈ S, f 0 (x) > b > 0, we have that

lim h→0 h d E K h (1, 2) -f h,1 f 0,1 2 1 {X 1 ∈S} = v K λ(S). Proof. Write E K h (1, 2) -f h,1 f 0,1 2
1 {X 1 ∈S} = E V h (X 1 ) f 2 0,1 1 {X 1 ∈S} = E E[K h (1, 2) 2 | X 1 ] f 2 0,1 1 {X 1 ∈S} -E f 2 h,1 f 2 0,1 1 {X 1 ∈S} .
The right-hand side is bounded by b -2 f 0 R d , hence its participation in the stated limit is 0.

For the left-hand side term, use K = K 2 /v K and write

h d E E[K h (1, 2) 2 | X 1 ] f 2 0,1 1 {X 1 ∈S} = h d f 0 (y) f 0 (x) K h (x -y) 2 1 {x∈S} dydx = f 0 (x -hu) f 0 (x) K(u) 2 1 {x∈S} dudx = v K f 0 (x -hu) f 0 (x) K(u)1 {x∈S} dudx = v K λ(S) + v K (f 0 (x -hu) -f 0 (x)) f 0 (x) K(u)1 {x∈S} dudx.
It remains to note that the term in the right goes to 0, as h → 0, in virtue of the Lebesgue dominated convergence theorem.

Lemma B.9. Under (H1) and (H2), we have, for every x ∈ R d and h > 0,

|f 0 K h (x) -f 0 (x)| ≤ g(x)h 2 u 2 2 K(u) du.
Proof. Note that K(u) du = 1 and, by symmetry, uK(u) du = 0. Write Whenever f 0 ∈ P, it holds that θn → θ 0 , in probability [START_REF] Newey | Large sample estimation and hypothesis testing[END_REF]Theorem 2.1] or [START_REF] Van Der | Asymptotic statistics[END_REF]Lemma 5.35]. For now, asking the above Lipschitz condition to guarantee the Glivenko-Cantelli might seem a bit restrictive [START_REF] Newey | Large sample estimation and hypothesis testing[END_REF]Lemma 2.4], but this condition will also be required to derive asymptotic normality of θn as well as to obtain uniform convergence (over x ∈ R d ) of f θn (x) to f θ 0 (x). Indeed, we have that for any δ > 0, with probability going to 1, θn ∈ B(θ 0 , δ). Hence, using the mean-value theorem, we find |f θn (x) -f θ 0 (x)| ≤ θn -θ 0 2 ˙ (x) sup θ∈B(θ 0 ,δ)

f θ (x) (21)
for every x ∈ R d . Conclude using that ˙ × sup θ∈B(θ 0 ,δ) f θ is bounded and the convergence in probability of θn to θ 0 .

(A2) The true parameter θ 0 an interior point of Θ ⊂ R q . The model P is differentiable in quadratic mean at θ 0 , i.e., there exists a measurable vector-valued function ˙ θ 0 , with E[ ˙ θ 0 (X 1 ) 2 2 ], such that

f θ -f θ 0 - 1 2 (θ -θ 0 ) T ˙ θ 0 f θ 0 2 dλ = o( θ -θ 0 2
2 ).

The matrix I = E[ ˙ θ 0 (X 1 ) ˙ θ 0 (X 1 ) T ] is invertible.

As a consequence of the previous set of conditions [START_REF] Van Der | Asymptotic statistics[END_REF]Lemma 5.39], we have n 1/2 ( θn -θ 0 ) = I -1 n -1/2 n i=1 ˙ θ 0 (X i ) + o P (1). ( 22)

where E[ ˙ θ 0 (X 1 )] = 0. In particular, it holds that √ n( θn -θ 0 ) = O P (1).

Figure 1 :

 1 Figure 1: Estimated p-values (on a logarithmic scale) of the Cramer-Von-Mises goodness-of-fit normality test against the values of the fitness coefficient for the CAC40 data.

Let • 2

 2 be the Euclidean norm and for any set S ⊂ R d and any function f : S → R, define the sup-norm as f S = sup x∈S |f (x)|. Denote by λ the Lebesgue measure on R d . Introduce the density level sets

Figure 2 :

 2 Figure 2: Values of the fitness coefficient and the OS coefficient as a function of the bandwidth h, expressed as a proportion of the standard error of the data. Plain blue line: fitness coefficient. Dashed black line: OS coefficient. The red sticks indicate various bandwidth values chosen according to the literature (see text); (a) wind speed data, (b) simulations under a Gumbel model, (c) simulations under a Gaussian model. In all cases the fitted model is a Gumbel model.

Figure 3 :Figure 4 :

 34 Figure3: Performance of the methods when the truth {N (0, (1 + t) 2 ), -0.5 < t < 0.5} approaches the model {N (θ, 1), -∞ < θ < ∞} until they intersect at t = 0. The L2-distance averaged over the replications is pictured in (a) for the parametric estimator, the nonparametric estimator, the OS method and the LR method. The integrated averaged distance is pictured in (c). Figure (b) pictures the values of the fitness coefficient and the OS coefficient averaged over the replications.

Figure 5 :Figure 6 :

 56 Figure 5: Contour plots of the true (a) and the estimated joint densities with the parametric (b), nonparametric (c), and semiparametric (d) strategies. The size of the dataset is n = 200.

  (a) By Lemma B.3, stated in Section B.4, there exists c > 0 such that with probability going to 1, h d n Q (np) n (S) ≥ c. The set S is chosen equal to {f 0 (x) > b} where b > 0 is such that it is non-empty. (b) We show in Lemma B.4 that h d n |M (np) n | → 0 in probability. (c) In Lemma B.5 (resp. Lemma B.6), it is established, under (A1) and (A2), that Q (p) n (S) = O P (1) (resp. |M (p) n | = O P (1)).

B. 3 . 1

 31 Proof of Theorem B.1

|f 0

 0 K h (x) -f 0 (x)| = (f 0 (x -hu) -f 0 (x))K(u) du = (f 0 (x -hu) -f 0 (x) -(hu) T ∇f 0 (x))K(u) du ≤ |f 0 (x -hu) -f 0 (x) -(hu) T ∇f 0 (x)| K(u) du,and use (H1) to conclude. Lemma B.10. Under (H1), (H2) and (H3), there exists c 2 > 0 such that S c bn f hn (x) dx ≤ c 2 b β n .

  , consisting of yearly wind speed maxima taken in the north direction in Sheridan, Wyoming. There are 20 observations for the years 1958 to 1977: 70, 61, 61, 60, 61, 63, 61, 67, 61, 62, 47, 67, 61, 49, 55, 65, 57, 51, 47, 56. The parametric model is a Gumbel model, that is, log

Acknowledgments. The authors are grateful to Ingrid Van Keilegom, Anouar El Ghouch and Sylvie Huet for useful comments and references. The authors are also grateful to Christian Robert for some motivating remarks at the very beginning of this work.

The research by G. Mazo was partially funded by a "Projet de Recherche" of the "Fonds de la Recherche Scientifique -FNRS" (Belgium).

† The author gratefully acknowledges support from the Fonds de la Recherche Scientifique (FNRS) A4/5 FC 2779/2014-2017 No. 22342320, from the contract "Projet d'Actions de Recherche Concertées" No. 12/17-045 of the "Communauté française de Belgique".

Proof. Note that S c bn f h (x) dx = P(S c bn ) + S c bn (K hn f 0 -f 0 ) dx = P(S c bn ) + (K hn 1 S c bn (x) -1 S c bn (x))f 0 (x) dx.

The term in the left is bounded by cb β n as supposed in (H3). For the term in the right, define S bn,hn = {y + h n u : u ∈ [-1, 1] d , y ∈ S bn }. Note that, by (H2), as soon as x / ∈ S bn,hn , K hn 1 S c bn (x) = 1, hence

Moreover, for any x ∈ S bn,hn , we have, by (H3), that

Appendix C Parametric maximum likelihood estimator

In this section are reported some classical results on the maximum likelihood estimator of the density. When the model is well-specified, we need the consistency and the asymptotic normality of the estimated parameter θ 0 .

(A1) The set Θ ⊂ R q is compact. The model P = {f θ : θ ∈ Θ}, a collection of densities on R d , is identifiable, i.e., for every

There exists an R + -valued measurable function ˙ with E ˙ (X 1 ) 2 < ∞ for every x ∈ R d , for every θ 1 and θ 2 in Θ,

There exists δ > 0 such that the function ˙ × sup θ∈B(θ 0 ,δ) f θ is bounded.

It follows from (A1) that the class of functions P is Glivenko-Cantell [49, Theorem 2. (20)