F Comte

C Duval

Should we estimate a product of density functions by a product of estimators ?

Keywords: Primary 62G05, 62G07 Bandwidth selection, Density estimation, Kernel estimator, Model selection, Nonparametric estimation, Penalized Comparison to Overfitting, Projection estimator

In this paper, we consider the inverse problem of estimating the product f g of two densities, given a d-dimensional n-sample of i.i.d. observations drawn from each distribution. We propose a general method of estimation encompassing both projection estimators with model selection device and kernel estimators with bandwidth selection strategies. The procedures do not consist in making the product of each density estimator, but in plugging an overfitted estimator of one of the two densities, in an estimator based on the second sample. Our findings are a first step toward a better understanding of the good performances of overfitting in regression Nadaraya-Watson estimator.

Introduction

In this work, we consider that we have n observations X i , i = 1, . . . , n in R d independent and identically distributed (i.i.d.) with density f and independent from n additional observations Y i , i = 1, . . . , n in R d , i.i.d. with density g. We study the question of estimating the product function f g from these observations. Note that the resulting function is not -in general -a density, and none of the observations are directly related to this product. In that sense, we face an inverse problem. Our framework contains the case where f = g and the goal is to estimate f 2 by splitting a 2n-sample. These quantities may be of interest in some testing problems or as a first step for estimating the L 2 -norm of f , see [START_REF] Laurent | Adaptive estimation of a quadratic functional by model selection[END_REF]; other product problems are considered in [START_REF] Butucea | Fast adaptive estimation of log-additive exponential models in Kullback-Leibler divergence[END_REF].

However, we must explain that we considered this problem as a simplified setting (a toy-problem, in some sense) for a more complicated question. Let us explain it. Consider a regression model in dimension d = 1 with Y i = b(X i ) + ε i with i.i.d. and independent sequences (X i ) 1≤i≤n and (ε i ) 1≤i≤n . The question is to estimate the regression function b(•) from observations (X i , Y i ) 1≤i≤n . A popular proposal is the Nadaraya-Watson estimator (see [START_REF] Györfi | A distribution-free theory of nonparametric regression[END_REF])

b h (x) = 1 nh n i=1 Y i K Xi-x h 1 nh n i=1 K Xi-x h = n i=1 w n,i,h Y i , w n,i,h = K Xi-x h n i=1 K Xi-x h ,
where K is a kernel and h a bandwidth parameter. This estimator can be seen as a weighted combination of the Y i 's (second equality) or as a ratio of an estimator of bf , where f is still the density of the X i 's, divided by an estimator of f (first equality). In this last case, it is not clear that the same bandwidth h must be chosen for the numerator and the denominator. Surprisingly, Comte and Marie (2021) proposed sophisticated strategies for these two terms, but noticed in the simulation experiments that, if the numerical results obtained for both functions separately were excellent, the performance of the ratio was almost systematically defeated by the single bandwidth method selected from a least squares criterion relying on the weighted view of the question. The unique bandwidth selected in this case is small, but the ratio of these two bad and overfitted estimators is undoubtedly very good, at least for not too high noise level (see a related discussion in Section 4.1 of Bartlett et al. (2019)). This is why we wondered if the product of two functional estimators was a good estimator of the product of two functions; we took these functions as densities for simplicity. Thus our motivation is mainly theoretical, but we believe that the question is of general interest. Now, let us see why making a product of density estimators can be seen as an inadequate (sub-optimal) strategy. Assume that we set f g := f × g, where f and g are minimax optimal estimators of f and g respectively. To get an upper bound result, there is no other way than to separate the role of each estimate : both individual risks of f and g would emerge. Then, the resulting rate is the slowest between the rates of estimation of f and g: it is the rate induced by the less regular density between f and g, say g without loss or generality for the remaining of this discussion. Clearly, this is not optimal if the product f g is more regular than g. For instance, if d = 1 and for f a β(p, p) density with p ≥ 2, p integer and g a uniform density, i.e. a β(1, 1). Then on R, f has regularity p -1 and g regularity 0, but f g = f has regularity p -1. Therefore, one can wonder if in these cases it is possible to build an estimator directly adapted to the regularity of the product f g.

A related disadvantage of an upper bound separating the roles of f and g is that it does not treat this problem as an inverse problem : both individual regularities of f and g intervene whereas one expects that the sole regularity of f g should matter. Especially since, depending on the regularity classes which are considered, there is often no universal rule relating the regularities of f and g to the one of the product.

To complete this discussion, notice that it is easy to derive a lower bound result, inspired by the former example on beta distributions. Denote by Σ(s, L), where s = (s 1 , . . . , s d ) with positive s i , i = 1, . . . , d and L, a ball of radius L in space of functions with regularity s. Then it holds, for any measurable function

T of (X i , Y i ) 1≤i≤n , sup f g∈Σ(s,L) T -f g 2 ≥ sup f g∈Σ(s,L) Suppf ⊂[0,1] d g=1 [0,1] d T -f g 2 = sup f ∈Σ(s,L) Suppf ⊂[0,1] d T -f 2 . It follows that inf T sup f g∈Σ(s,L) T -f g 2 ≥ inf T sup f ∈Σ(s,L) Suppf ⊂[0,1] d T -f 2 , (1.1)
we recover on the right side the lower bound of the direct density estimation problem. To summarize, if the regularity set Σ(s, L) contains a [0, 1] d supported density f 0 , a lower bound for the product is given by a lower bound for the direct estimation of f 0 . This is enough to state that the upper bound results presented below are optimal. For instance in dimension d = 1, we recover rates in n -2s 2s+1 if (f g) ∈ Σ(s, L), a Sobolev class of regularity s, that are minimax.

The plan of the paper is the following. We propose in section 2.1 a general estimation strategy: we define a function estimator of the product f g and prove a non-asymptotic risk bound under general assumptions. This general strategy encompasses projection and kernel methods, for which we check the assumptions and present spectific results in Section 2.2. Then we study in a particular projection case the resulting rate, related to the regularity of f g: it can be reached for a well-chosen dimension of the projection space (see section 2.4). As this choice depends on unknown parameters, we then propose in Section 3.1 a general parameter selection strategy of [START_REF] Goldenshluger | Bandwidth selection in kernel density estimation: oracle inequalities and adaptive minimax optimality[END_REF] type, and prove that the resulting estimator automatically reaches the squared-bias/variance compromise. This general method applies to projection and kernel methods. However in both cases and in dimension 1 alternative methods are used in the simulations, which outperform Goldenshluger and Lepski procedures numerically; they are simpler to calibrate with faster execution times. Note that an invariable difficulty in the theoretical study of these procedures is that even if we have i.i.d. variables, our estimates are built from sums of dependent variables (see e.g. (2.6) or (2.7)). Numerical comparisons of the different methods and associated strategies for product estimators are conducted in section 4, and concur to our theoretical findings. Several additional questions are presented in the concluding remarks of section 5. Lastly, proofs are gathered in section 6 for the results of section 2 and in section 7 for the adaptive results stated in sections 3 and 4.

General estimates of the product and examples

Functional estimator and first risk bound

We consider a general family K of functions K : I 2 → R, where I ⊂ R d , that are symmetric (i.e. ∀x, y ∈ I, K(x, y) = K(y, x)).

In the sequel, we denote by ψ K the quantity ψ K (x) := K(x, y)ψ(y)dy for any function ψ. The collection K must be chosen such that ψ K can be a good approximation of ψ. Examples of possible collection K are provided in Section 2.2.

For such a function K * ∈ K, we define an estimator of f by

f K * (x) = 1 n n i=1 K * (X i , x), x ∈ I. ( 2
f K * ) = f K * is
expected to be a good approximation of f . Following an analogous way, we propose as an estimator of f g, for K ∈ K,

(f g) K,K * (x) = 1 n n i=1 f K * (Y i )K(Y i , x), x ∈ I. (2.2) Indeed, if f K * is near of f , and as E(f (Y 1 )K(Y 1 , x)) = (f g) K , we get a relevant estimator of f g if, again, for K ∈ K, ψ K is a good approximation of f g. Note that in definition (2.
2), X 1 and Y 1 must have the same dimension.

We set the following assumptions on K:

(A1) ∃C 1 > 0, ∀ψ ∈ L 2 (I), ψ K 2 = ψ(y)K(y, x)dy 2 dx ≤ C 1 ψ 2 (A2) L(K) := sup y∈I K 2 (y, x)dx < ∞.
Moreover, we require the following assumptions on f K * :

(A3) ∃C 3 > 0, E[ f K * (y)] 2 dy = f K * 2 ≤ C 3 . (A4) ∃C 4 > 0, Var f K * (y) dy ≤ C 4 . (A5) ∃C 5 > 0, ∀ψ ∈ L 2 (I), E ψ, f K * -E[ f K * ] 2 ≤ C 5 ψ 2 n .
These assumptions are related to K * and therefore to the density f only. Now, we can establish the following upper bound on the mean integrated risk of (f g) K,K * .

Proposition 2.1. Let K, K * belong to K. Assume that assumptions (A1)-(A5) hold and that g is bounded on I with bound denoted by g ∞ . Let (f g) K,K * be the estimator defined by (2.2). Then, we have

E[ (f g) K,K * -f g 2 ] ≤ 2 (f g) K -f g 2 +2C 1 g 2 ∞ f -f K * 2 +C(f, g) L(K) n , (2.3)
where

C(f, g) := g ∞ (C 3 + C 4 + C 5 ).
Strategy suggested by (2.3). The risk bound (2.3) contains two standard terms, the squared bias (f g) K -f g 2 and the variance C(f, g)L(K)/n, requiring a standard selection for K (see [START_REF] Lerasle | Optimal kernel selection for density estimation[END_REF]). It also involves the bias term f -f K * 2 which has no counterpart: thus K * can and should be chosen in order to make it negligible and f K * can thus be taken overfitted.

If in the initial problem, f and g have symmetric roles, this is no longer true in the definition (2.2) of the estimator, where one of the two densities is estimated first. As a matter of fact, Proposition 2.1 suggests to plug in the product estimator (2.2) an over-fitted estimator, eliminating a selection issue for K * . When possible we select for this over-fitted estimator the one corresponding to the smoother density. Indeed, this should make the additional bias term decrease faster. However, the information about which is smoother between f and g, is not available. From theoretical viewpoint, both f -f K * 2 and gg K * 2 are negligible by assuming a minimal regularity for f and g and choosing K * such that Assumptions (A1)-(A5) hold, makes the bias term the smallest as possible. From a practical point of view and in dimension 1, we propose to consider that the smoother density is the one for which a selection method for the direct density estimation of f and g leads to the smallest complexity L(K) (i.e. the smallest selected dimension in projection or the largest bandwidth for kernels, see section 2.2 hereafter).

In the sequel, we may write (f g) K instead of (f g) K,K * when there is no ambiguity, as K * can be fixed.

Assumptions (A1)-(A5) for projection and kernel estimators

We consider the following two examples:

[P ] Projection function: for a multi-index m = (m 1 , . . . , m d ) ∈ N d , let K m (x, y) = 0≤j≤m-1 ϕ j (x)ϕ j (y)
where (ϕ j ) 0≤j≤m-1 is an L 2 (I)-orthonormal basis, I = I 1 × . . . × I d , with

ϕ j (x) = (ϕ j1 ⊗ . . . ⊗ ϕ j d )(x) = ϕ j1 (x 1 ) × . . . × ϕ j d (x d ) and L(m) := 0≤j≤m-1 ϕ 2 j ∞ = d j=1   sup xj ∈Ij mj -1 kj =0 ϕ kj (x j ) 2   =: d j=1 L(m j ) < +∞.
We denote by S m the D m -dimensional linear subspace of L 2 (I) spanned by ϕ 0 , . . . , ϕ m-1 , where D m = d j=1 m j .

[Ker ] Kernel function:

K h (x, u) = K x -u h = d j=1 1 h j K x j -u j h j
for K an integrable and square-integrable symmetric function (K(-z) = K(z)) defined on R, such that I K = 1 and h ∈ [0, 1] d . We denote by Next we prove that Assumptions (A3)-(A5) are verified for the projection estimator of f

K h (x) := d j=1 h -1 j K(x j /h j ) and set L(h) = d j=1 L(h j ) where L(h) = K 2 /h.
f K m * := f m * = 0≤j≤m * -1 a j ϕ j , a j = 1 n n i=1 ϕ j (X i ), (2.4)
where 0≤j≤m * -1 stands for

m * 1 -1 j1=0 . . . m * d -1 j d =0
, and for the kernel estimator

f K h * := f h * (x) = 1 n n i=1 K h * (X i -x).
(2.5)

Proposition 2.3. Assume that f is bounded. Then the estimators of f :

• f m * in case [P] under L(m * ) ≤ n, • f h * in case [Ker] under L(h * ) ≤ n,
satisfy assumptions (A3)-(A5).

The conditions L(m * ) ≤ n and L(h * ) ≤ n represent a stronger version of Assumption (A2). Moreover f Km = f m = 0≤j≤m-1 a j ϕ j , with a j = f, ϕ j is the projection of f on S m : it is a good approximation of f for m large (in the sense that min 1≤j≤d m j is large). Analogously, f K h = f K h with u v denoting the convolution product of two L 2 (I) functions, gets close to f for small h (in the sense that max 1≤j≤d h j is small) under mild conditions, see e.g. [START_REF] Tsybakov | Introduction to nonparametric estimation[END_REF] or [START_REF] Goldenshluger | On adaptive minimax density estimation on R d[END_REF].

This allows to derive from Proposition 2.1 in the following section upperbound results for projection and kernel estimators that we introduce below.

A Corollary in the projection or kernel case

In a purely projection approach where we take K and K * in the same collection of the form [P], we obtain the following estimator of f g:

(f g) Km,K m * := (f g) m,m * = 0≤j≤m-1 a (m * ) j ϕ j , a (m * ) j = 1 n n i=1 ϕ j (Y i ) f m * (Y i ).
(2.6) Clearly, E( a (m * ) j ) = ϕ j , f m * g , which shows that our estimator is indeed close to f m * g, which in turn should be near of f g for large m * . Choosing m * large is possible since the variance of f m * does not appear in the risk bound. It is worth noting that the sum in the right hand side of (2.6) is composed of identically distributed but dependent variables: indeed, dependency appears through f m * , which is based on the X-sample.

In a purely kernel approach where we take K and K * in the same collection of the form [Ker], we consider the estimator of f g:

(f g) h,h * (x) = 1 n n i=1 f h * (Y i )K h (Y i -x). (2.7)
A straightforward consequence of Proposition 2.1 is the corollary:

Corollary 2.1. Assume that f and g are bounded.

Let (f g) m,m * be defined by (2.6). For any m * such that L(m * ) ≤ n, we have

E (f g) m,m * -f g 2 ≤ 2 (f g) m -f g 2 +2 g 2 ∞ f -f m * 2 +C 1 L(m) n , (2.8)
where f g = f g1 I , C 1 := g ∞ (1 + 2 f ∞ ) and (f g) m is the orthogonal projection of f g on S m , f m * the orthogonal projection of f on S m * . Let (f g) h,h * be defined by (2.7). For any h * such that L(h * ) ≤ n, we have

E (f g) h,h * -f g 2 ≤ 2 (f g) h -f g 2 + 2 g 2 ∞ K 2 1 f h * -f 2 + C 2 L(h) n , (2.9) where C 2 = g ∞ (1 + 2 K 2 1 f ∞ ).
It is also possible to consider a mixed strategy where K * is selected in collection [P] and K in [Ker], or vice-versa, and a similar result is obtained. This strategy is investigated in the numerical section. Then, the two bias terms (f g) K -f g 2 and f -f K * 2 refer to different regularity spaces, see the discussion below and the next section.

The bound (2.9) suggests to choose h * the smallest as possible, in order to make this term negligible. For instance if f belongs to a Nikols'ki ball with regularity parameter α = (α 1 , . . . , α d ) (see [START_REF] Goldenshluger | On adaptive minimax density estimation on R d[END_REF], Definition 1), f h * -f 2 has order d j=1 (h * j ) 2αj if the kernel K has order at least max j α j (where α is the largest integer such that α < α and the order of the kernel is understood as in section 3.2 of Goldenshluger and Lepski (2014)). It follows that if min j α j > 1/2 and h * j = 1/n, ∀j, this term has order less than 1/n and is negligible. Then, only the bandwidth h requires to be selected.

If in addition f g belongs to a Nikols'ki ball with regularity parameter β and K has order at least max j β j , then the estimator reaches the minimax rate

n -2β/(2β+d) for 1 β = 1 d d j=1 1 β j ,
(see Goldenshluger and Lepski (2014)), for h j chosen of order n -β/ βj (2β+1) . Such a choice of h is not feasible since β is unknown, a data driven procedure for selecting h must be proposed.

In the following section we provide a more precise evaluation of the rates of estimation in the projection case in dimension 1.

Projection strategy: Rates on Sobolev Hermite spaces

In this section we focus on the case d = 1 and give an example of rate induced by the bound (2.8), in the case of the Hermite basis and associated Sobolev spaces (see Comte and Lacour (2021) Definition 1 for the general case d ≥ 1). The Hermite functions (ϕ j ) j≥0 are defined from Hermite polynomials (H j ) j≥0 by: for x ∈ R ϕ j (x) = c j H j (x)e -x 2 /2 , H j (x) = (-1) j e x 2 d j dx j (e -x 2 ), c j = (2 j j! √ π) -1/2 .

(2.10) The Hermite polynomials (H j ) j≥0 are orthogonal with respect to the weight function e -x 2 , that is: R H j (x)H k (x)e -x 2 dx = 2 j j! √ πδ j,k (see [START_REF] Abramowitz | Handbook of mathematical functions with formulas, graphs, and mathematical tables[END_REF], chap 22.2.14). Therefore, the Hermite basis (ϕ j ) j≥0 is an orthonormal basis on R. We note also that ϕ j is bounded:

||ϕ j || ∞ = sup x∈R |ϕ j (x)| ≤ Φ 0 , with Φ 0 1, 086435/π 1/4 0, 8160 (2.11)
(see [START_REF] Abramowitz | Handbook of mathematical functions with formulas, graphs, and mathematical tables[END_REF] For s > 0, the Sobolev-Hermite ball (see [START_REF] Bongioanni | Sobolev spaces associated to the harmonic oscillator[END_REF]) is defined by :

W s H (D) = θ ∈ L 2 (R), k≥0 k s a 2 k (θ) ≤ D , D > 0, (2.12)
where

a k (θ) = θ, ϕ k . It is proved in Belomestny et al. (2019) that, for s an integer, s ≥ 1, f ∈ W s H = {θ ∈ L 2 (R), k≥0 k s a 2 k (θ)
< +∞} is equivalent to: f admits derivatives up to order s which satisfy: f , f , . . . , f (s) , x s-f ( ) for = 0, . . . , s -1 belong to L 2 (R). Moreover, for any function f ∈ W s H (D), we have f -f m 2 ≤ Dm -s . It is also easy to see that if, in addition, s > 1, then f is bounded. Indeed

| j≥0 a j ϕ j | ≤ Φ 0   |a 0 | + j≥1 (|a j |j s/2 )j -s/2   ≤ Φ 0   f + j≥1 j s a 2 j j≥1 j -s   .
As f and g are assumed to be bounded, it holds

| f g, ϕ j | ≤ min( f ∞ | g, ϕ j |, g ∞ | f, ϕ j |). Thus if f g ∈ W s H (D), f ∈ W s H (D )
E (f g) mopt,m * n -f g 2 ≤ C(D, D , f ∞ , g ∞ )n -2s 2s+1 .
We can conclude that the resulting rate is of order n -2s/(2s+1) , and is optimal, see (1.1).

Adaptive procedure

General adaption result

We propose a Goldenschluger and Lepski (2011) method. Define K n = {K τ } τ ∈Tn a family of symmetric functions indexed by a parameter τ , satisfying (A1)-(A2). For simplicity, we write ψ τ instead of ψ Kτ and L(τ ) instead of L(K τ ). For example in the previous examples the parameter τ is a d dimensional vector of integers m in the projection case and a bandwidth h ∈ [0, 1] d in the kernel context. We add the following assumptions

(A6) ∀x, y ∈ I, ∀τ, τ ∈ T n , K τ (x, z)K τ (z, y)dz = K τ (y, z)K τ (z, x)dz. (A7) f and g are bounded and ∀τ ∈ T n , f τ ∞ ≤ C 7 . (A8)
The collection of models is such that Card(T n ) ≤ n d , and ∀c > 0,

τ ∈Tn e -c √ L(τ ) ≤ Σ < +∞ where Σ = Σ(c) is a constant depending on c but not on n. (A9) ∀τ ∈ T n , sup x,y |K τ (x, y)| 2 ≤ C 9 L 2 (τ ). (A10) The parameter τ * is such that L(τ * ) ≤ f ∞ 2(d + 3)(1 + √ C 9 ) n log(n) . Define (f g) τ,τ (x) = K τ (y, x) (f g) τ (y)dy. Under (A6) note that (f g) τ,τ = (f g) τ ,τ . Now, set A(τ ) = sup τ ∈Tn (f g) τ,τ -(f g) τ 2 -κV (τ ) + , V (τ ) = κ(C 1 ∨ C 2 1 )( f 2 ∞ + g 2 ∞ ) L(τ ) n . (3.1)
The selection of τ is done by the rule

τ = arg min τ ∈Kn {A(τ ) + κ V (τ )}
for some positive constants κ and κ to be selected. The estimator f K τ * of f defined in (2.1) relies now on the symmetric function K τ * ∈ K n and is rewritten

f τ * with K * = K τ * and we note f K τ * by f τ * . Theorem 3.1. Assume that Assumptions (A1)-(A10) are fulfilled. Then we have, for κ ≥ κ, E( (f g) τ, τ -f g 2 ) ≤ C inf τ ∈Tn (C 1 ∨ 1) (f g) τ -f g 2 + κ V (τ ) +42(C 1 ∨ C 2 1 ) g 2 ∞ f -f τ * 2 + C n ,
where C is a numerical and C depends on f ∞ , g ∞ and on a constant specific to the collection K.

Theorem 3.1 shows that the adaptive procedure automatically realizes the squared bias-variance tradeoff up to negligible terms, as soon as τ * is such that f -f τ * 2 has order less than O(1/n). This result is general and allows to consider any collection K such that (A1)-(A10) are satisfied. For example, a mixed strategy where a kernel estimator of f is plugged in a projection estimator of the product is possible. It is worth stressing that the proof of Theorem 3.1 relies on the Talagrand inequality and does not involve the study of a U-statistics (see [START_REF] Lerasle | Optimal kernel selection for density estimation[END_REF] and our Theorem 4.2 below).

Assumptions (A6)-(A10) for projection and kernel estimators

In this section, we present assumptions ensuring (A1)-(A10) for the previous procedures. First, in the projection case (2.6), we define the collection of proposals for m as follows

M n = {m ∈ {1, . . . , n} d , L(m) ≤ n},
and set the following set of assumptions:

[P1] f and g are bounded on I.

[P2] The basis functions are bounded:

∀j ∈ N d , ∀x ∈ I, |ϕ j (x)| ≤ C ϕ . [P3] The model m * is such that L(m * ) ≤ f ∞ 4(d + 3) n log(n)
.

[P4] The collection of models is such that Card(M n ) ≤ n d , and ∀c > 0,

m∈Mn e -c √ L(m) ≤ Σ < +∞ where Σ = Σ(c) is a constant depending on c but not on n. [P5] There exist b ∈ (1, ∞) d and C b > 0, which need not to be known, such that j≥1 j b a 2 j (f ) ≤ C b < +∞, where j b = j b1 1 . . . j b d d . Corollary 3.1. In the projection case [P], assumptions [P1]-[P5] imply (A1)- (A10).
In Assumption [P3], the maximal value of m * depends on f ∞ . This constraint can be replaced by L(m * ) ≤ n/ log 3/2 (n) and the result follows for n large enough. Assumptions [P2] and [P4] are classical, for instance they are fulfilled by trigonometric and Hermite bases. Condition [P5] is a minimal regularity constraint. For instance for d = 1, it requires that the function f has a minimal regularity of 1/2 on Sobolev-Fourier spaces for I = [0, 1] and 1 on Hermite Sobolev spaces.

Second, for the kernel estimator (2.7), denote by T n a discrete collection H n of bandwidths in (1/n, 1) with cardinality less than n. We consider the following set of assumptions:

[K1] f and g are bounded on I.

[K2] The kernel K is even, bounded and integrable.

[K3] The bandwidth h * is such that

L(h * ) ≤ f ∞ 2(d+3)(1+ K ∞ K 2 ) n log(n) .
[K4] The discrete collection H n of bandwidths in (1/n, 1) d has cardinality less than n d and for any c 1 > 0, h∈Hn exp(-c 1 L(h)) ≤ Σ = Σ(c 1 ) < +∞.

Note that as

K = 1, K 1 ≥ 1. Similarly to [P3], We can replace in [K3] the bound by n log(n) -3/2 ≥ L(h * ), for large enough n, to get rid of the unknown constant f ∞ in the bound defining h * . Assumption [K4] is fulfilled for H n = h ∈ 1 n , 2 n , . . . , n -1 n , 1 d .
Contrary to the projection, the kernel method does not require any regularity constraint of type [P5].

Corollary 3.2. In the kernel case [K], assumptions [K1]-[K4] imply (A1)- (A10).

Numerical study

Numerically efficient adaptive procedures in dimension 1

For the numerical study we focus on dimension d = 1 and do not implement the above adaptive procedure. Indeed, the Goldenschluger and Lepski method is often difficult to calibrate from an implementation viewpoint (see Comte and Rebafka (2012)) and suffers from important computational costs. Indeed, it involves the calibration of two constants, κ and κ. This preliminary calibration step is difficult, probably because these constants act simultaneously on the bias and variance terms. In the kernel case, the "double" convolution which appears when computing f g h,h is numerically time consuming. Instead we propose two numerically efficient procedures model selection and PCO, for which the squared bias-variance tradeoff is also attained under [P1]-[P5] and [K1]-[K4] respectively.

Model selection for projection estimators

We present, under the above assumptions [P1]-[P5], a result for a more standard and simpler model selection procedure. More precisely, define

γ n (t) = t 2 - 2 n n i=1 f m * (Y i )t(Y i ). (4.1)
Then select m with the criterion

m := arg min m∈Mn min t∈Sm γ n (t) + pen(m) , pen(m) = κ( f 2 ∞ + g 2 ∞ ) L(m) n
where κ is a numerical constant. Note that

min t∈Sm γ n (t) = γ n ( (f g) m,m * ) = -(f g) m,m * 2 .
With the same tools as those used in the proof of Theorem 3.1, we can prove the following result.

Theorem 4.1. In the projection case [P], if Assumptions [P1]-[P5] hold, then, there exists κ 0 such that, for any κ ≥ κ 0 , we have

E( (f g) m,m * -f g 2 ) ≤ inf m∈Mn 3 f g -(f g) m 2 + 4κ( f 2 ∞ + g 2 ∞ ) L(m) n +16 g 2 ∞ f -f m * 2 + C n , (4.2)
where C is a constant depending on f ∞ , C a .

The proof is omitted but details can be found in the preprint version Comte and Duval (2022), version 1, which also indicates that κ 0 = 8 × 12 = 96 would suit. In practice, this theoretical value is always too large, and has to be calibrated on preliminary simulation experiments. Note that, the estimate (f g) m,m * is replaced by its positive part, for which the same risk bound holds. Moreover, it can be easily checked that this proof also hold in a mixed strategy where a kernel estimator for f is plugged in a projection estimator of the product, i.e. K * in [Ker] and K in [P]. This justifies why for numerical results we experiment this mixed strategy with a penalised criterion for adaptation.

The values f ∞ , g ∞ in the penalty term are unknown and must be replaced by estimates. The bound f ∞ can be estimated by the maximal value of a projection estimate of f on a middle-sized space, for instance sup

x∈I | f [ √ n] (x)|
and an analogous approach can be adopted for g ∞ . Let us denote these estimators by f ∞ and g ∞ . This strategy is theoretically studied in Theorem 12 p.594 (Appendix A: Random penalty) in [START_REF] Lacour | Adaptive estimation of the transition density of a Markov chain[END_REF]. We consider in the numerical Section the estimator f m,m * and adopt the following strategy. The penalty is obtained from the theory as the sum of the bounds on two terms, a bound on

1 n m-1 j=1 E ϕ 2 j (Y 1 )[ f m * (Y 1 )
] 2 and a bound on an additional term given by f ∞ g ∞ L(m)/n. Following ideas in [START_REF] Massart | Concentration inequalities and model selection[END_REF] (see also Theorem 7.6 p.216, in the density case), we replace the first term by

pen 1 (m) = 1 n 2 n i=1 m-1 j=0 f m * (Y i )ϕ j (Y i ) 2
and the second term by pen

2 (m) = f ∞ g ∞ L(m)/n.
(m) + κ f ∞ g ∞ √ m n . (4.3)
The constant κ is calibrated by preliminary simulations, see section 4.

Bandwidth selection with PCO

We describe here a method called "PCO" ("Penalized Comparison to Overfitting") introduced for kernel density estimation by [START_REF] Lacour | Estimator selection: a new method with applications to kernel density estimation[END_REF]. The PCO method is more complicated from theoretical point of view, because it involves the study of several U -statistics of order 2. But, it is much easier to calibrate and implement in practice.

Let us describe the PCO method. Denote by (f g) h (omitting h * for simplicity), we select

h = arg min h∈Hn (f g) h -(f g) hmin 2 + 2pen(h) with h min = min{h, h ∈ H n }, pen(h) = pen 1 (h)+pen 2 (h) where pen 1 (h) = 1 n 2 K h , K hmin n i=1 f 2 h * (Y i ), (4.4) pen 2 (h) = κ c 0 (f, g, K) nh , c 0 (f, g, K) = 4 K 3 1 K ∞ ( g 2 ∞ + f 2 ∞ ). (4.5)
Note that pen 1 and pen 2 are both of order 1/(nh). This is obvious for pen 2 ; for pen 1 , observe that

| K h , K hmin | ≤ K ∞ K 1 /h and that, under [K3], (1/n) n i=1 f 2 h * (Y i ) is
E (f g) h -f g 2 ≤ c 1 inf h∈Hn (f g) h -f g 2 + (1 + κ 1 + θ ) c 0 (f, g, K) nh +c 2 (f g) hmin -f g 2 + c 3 f h * -f 2 + C log(n) n ,
where c 3 and C are positive constants depending on f, g, K and (c 1 , c 2 ) = (3.6, 10.4) would suit.

The risk bound of Theorem 4.2 involves four terms. The first term in the first line is the minimal risk among the collection of estimators, up to multiplicative a constants. The two following terms, (f g) hmin -f g 2 and f h * -f 2 are bias terms corresponding to small bandwidths, they are negligible if h min is of order 1/n and h * of order log(n)/n (as required by assumption [K3]), and if the functions f and f g have regularity larger that 1/2. The last term log(n)/n has negligible order compared to the first one. Therefore, the adaptive estimator achieves the intended squared-bias variance compromise given at the end of section 2.3.

Description of the implemented procedures

We illustrate the performances of the projection estimator with Hermite basis (see section 2.4) and kernel estimator with kernel built as a Gaussian mixture defined by:

K(x) = 2n 1 (x) -n 2 (x), (4.6)
where n j (x) is the density of a centered Gaussian with a variance equal to j. This kernel is of order 3 (i.e. x j K(x)dx = 0, for j = 1, 2, 3). We consider four examples:

[BU] X ∼ f = B(7, 5) and Y ∼ g = U(0, 1), [GE] X ∼ f = Γ(4, 1/4) and Y ∼ g = E(1/4), [NL] X ∼ f = N (0, 3) and Y ∼ g Laplace, [NC] X ∼ f = N (0, 3) and Y ∼ g Cauchy.
We compute normalized L 2 -risks to allow the numerical comparaison of the different examples for which (f g) 2 varies a lot. Namely, we evaluate • Product : This estimator is obtained as the product of f g where each estimator is an adaptive optimal estimator. In the projection case, the product is f m1 g m2 , where f m is defined by (2.4) with

E[ (f g) -f g 2 ] f
m 1 = arg min m∈{1,...,Dmax}    -f m 2 + 4 n 2 n i=1 m-1 j=0 ϕ 2 j (X i )    ,
and g m2 is defined analogously. In the kernel case, f h1 g h2 , where f h1 is defined by (2.5) with

h 1 = arg min h∈{1/k,k=1,...,n} f h -f 1 n 2 + 4 n K h , K 1 n ,
and g h2 is defined analogously.

• First X: In all our examples f is smoother than g. The theoretical results suggest that one should consider for the preliminary estimate the dataset X which has density f . In the projection setting our estimate is (f g) m,m * of Theorem 4.1 and penalty given by (4.3), where

f 2 ∞ is estimated by sup x∈I | f 2 10 (x)|, g 2
∞ is estimated similarly, and with κ = 0.15 after calibration. In the kernel case we consider the estimator (f g) h of Theorem 4.2 with penalty given by (4.4) where pen 2 is replaced by

pen 2 = 0.32 f 2 ∞ + g 2 ∞ nh ,
where

f 2 ∞ is estimated by sup x∈I | f 2 log n/ √ n (x)|, g 2 ∞ is estimated similarly. Note that K 1 1.133 and K ∞ 0.516, K 3 1 K ∞ 0.75.
• Optimal first : As the information about compared smoothness of f and g is unavailable in practice, we have proposed an adaptive method for choosing which estimate is plugged in: we perform a classical penalized (resp. PCO) procedure (see step Product) to the datasets X and Y and we take as preliminary projection (resp. kernel) estimate the one for which m (resp. h) is the smallest (resp. largest). Indeed, the optimal dimension (resp. bandwidth) is asymptotically a decreasing (resp. increasing) function of the regularity. For instance, if m 1 < m 2 we proceed as in First X step, otherwise the roles of X and Y are switched. We count the number of times where Y is selected first; thus, when this count is zero, First X and Optimal first are the same and give the same result.

• Oracle (optimal first) : Our benchmark is computed as follows. We consider for all dimensions or bandwidths the estimators of the step Optimal first and select the oracle that minimizes

m → E[ (f g) m,m * -f g 2 ] or h → E[ (f g) h -f g 2 ]
. This quantity provides a numerical lower bound for the L 2 -risk of our procedure. Case [GE]: L 2 -risks with std in parenthesis (both multiplied by 10 2 ), Dmax = 100. For the Optimal first the bold sub-script is the number of times where Y is selected first.

Numerical results

Let us comment the results of Tables 1234. First, we compare separately projection and kernel procedures. Let us start with the two fully data driven methods Product and Optimal first. We observe that the results of the corresponding columns nicely confirm the theory: the risks of our procedure is almost systematically and significantly smaller (see Table 2 in particular). Besides, the risk of Optimal first is always comparable and even sometimes better than the risk of the First X method which uses the unavailable knowledge of the smoothest density. The risk of Optimal first has the same order as the Oracle even if a multiplicative factor larger than 2 is observed. Lastly, as the risks are normalized we can compare the risks of the different Tables; we see that the first two examples (Tables 1 and2) are slightly more difficult which was expected: these densities are less regular as functions on R.

Second, we can compare projection and kernel methods. The kernel method is much more time consuming that the projection method (by a factor more than 10). We can see that for the operational Optimal first method the kernel strategy seems better for the first two examples while the projection method wins in the two other cases. Nevertheless, the gap between the risks is never very large.

Mixed Kernel and projection strategy. Finally, we implement a mixed strategy where a kernel estimator for f , based on the kernel (4.6), is plugged in a Hermite projection estimator of the product f g. The selection procedure is performed via model selection as described above. Following the same strategies as previously, we display the estimated normalized L 2 -risks for the four examples. The adaptive Optimal first strategy is conducted by adaptively selecting the smoothest density with a model selection procedure, even if the plugged-in estimator is a kernel. Comparing the results displayed in Table 5 with above results, we observe an analog behavior for the mixed strategy as for the projection or kernel ones. Mixed Kernel-Projection : L 2 -risks with std in parenthesis (both multiplied by 10 2 ).

For the Optimal first the bold sub-script is the number of times where Y is selected first.

Concluding remarks

In this paper, we have shown that an optimal strategy for estimating a product of densities was not to make a product of estimators but to plug an overfitted estimator of one of the densities in the estimator of the product. This can be done both with projection and kernel estimators and adequate model or bandwidth selection methods are proved to deliver adaptive estimators. We have implemented these methods and proved their good numerical performances in dimension 1.

We assume that the two samples have the same sizes but the case where the X-sample has size n X and the Y sample size n Y is worth being studied, for instance if n X = γn, n Y = n, γ ∈ (0, ∞). Following the steps of the proof in the projection case suggests that the procedure can be adapted and leads to similar results with rate induced by the smallest sample size (1 ∧ γ)n.

We considered a product of two densities but generalizations to product of other functions or product of more than two densities may be worth studying. Lastly, it is likely that our methods would extend to dependent variables, provided that the two sequences remain independent, but this should be further investigated.

If we come back to the Nadaraya-Watson problem that initiated our question, we justified in our context that plugging an overfitted estimator is an optimal strategy. Other contexts where overfitting has been recognized as judicious exists (see Chinot and Lerasle (2020)). The next step, as the original problem is a ratio, is to address the question of estimating 1/f when f is a density.

Proofs of section 2

Proof of Proposition 2.1

Consider the classical bias variance decomposition

E[ (f g) K -f g 2 ] = E[ (f g) K -E[ (f g) K ] 2 ] + E[ (f g) K ] -f g 2 := V + B.
We first study the bias term B, note that E

[ (f g) K ] = (E[ f K * ]g) K = f K * g K , it follows that B ≤ 2 f K * g K -(f g) K 2 + (f g) K -f g 2 := 2(B 1 + B 2 ),
where B 2 = (f g) K -f g 2 is the standard integrated squared bias of f g. Using (A1) we write

B 1 = (f K * (y) -f (y))g(y)K(y, x)dy 2 dx ≤ C 1 (f K * -f )g 2 ≤ C 1 g 2 ∞ f K * -f 2 .
Next we split V to involve the conditional variance given (X 1 , . . . ,

X n ) = X V = E   1 n n i=1 f K * (Y i )K(Y i , .) -E f K * (Y 1 )K(Y 1 , .) X 2   + E ( f K * -f K * )g K 2 := V 1 + V 2 , as E f K * (Y 1 )K(Y 1 , .) X = ( f K * g) K .
For the first term, as the sum is composed of centered terms we get

V 1 = E Var 1 n n i=1 f K * (Y i )K(Y i , x) X dx ≤ 1 n E f K * (Y 1 ) 2 K(Y 1 , x) 2 dx = 1 n E f K * (Y 1 ) -f K * (Y 1 ) 2 K(Y 1 , x) 2 dx + E f K * (Y 1 ) 2 K(Y 1 , x) 2 dx ≤ 1 n E ( f K * (y) -f K * (y)) 2 K(y, x) 2 g(y)dydx + f K * (y) 2 K(y, x) 2 g(y)dydx = 1 n Var( f K * (y))g(y) K(y, x) 2 dxdy + f K * (y) 2 g(y) K(y, x) 2 dxdy ≤ g ∞ L(K) n (C 4 + C 3 ) ,
where we used successively (A2), (A4) and (A3). Finally, we write using (A5)

V 2 = E f K * -f K * , gK(., x) 2 dx ≤ C 5 n K(y, x) 2 g(y) 2 dydx ≤ C 5 n L(K) g 2 ,
where we used (A2) to obtain the last line. Using g 2 ≤ g ∞ and gathering all bounds completes the proof.

Proof of Proposition 2.2

In case [P], we have, denoting by ψ m the orthogonal projection of ψ on the linear space S m :=span(ϕ 0 , . . . , ϕ m-1 ),

  0≤j≤m-1 ϕ j (x)ϕ j (y)ψ(y)dy   2 dx =   0≤j≤m-1 ϕ j (y)ψ(y)dy ϕ j (x)   2 dx = 0≤j≤m-1 ϕ j (y)ψ(y)dy 2 = ψ m 2 ≤ ψ 2 .
So Assumption (A1) holds with C 1 = 1. On the other hand, for (A2), we have

sup y∈I   0≤j≤m-1 ϕ j (x)ϕ j (y)   2 dx = sup y∈I 0≤j≤m-1 ϕ 2 j (y) = L(m) < +∞.
In case [Ker], we denote by u v(x) = u(y)v(x -y)dy the convolution product. Then by the Young inequality (8.1), we get

ψ(y)K h (x -y)dy 2 dx = ψ K h 2 ≤ K h 2 1 ψ 2 = K 2d 1 ψ 2 .
Thus (A1) holds with

C 1 = K 2d 1 . As ∀j, K 2 h (y j -x j )dx j = K 2 (u)du/h j < +∞, Assumption (A2) is satisfied and L(K h ) = K 2d d j=1 h -1 j .

Proof of Proposition 2.3

In case [P], we have E[ f m * (y)] = f m * (y) where f m * is the projection of f on

S m * = span(ϕ 0 , . . . , ϕ m * -1 ). Therefore E[ f m * (y)] 2 dy = f m * 2 ≤ f 2 ≤
f ∞ , and (A3) holds with

C 3 = f 2 or C 3 = f ∞ . Next we write Var f m * (y) dy = Var   1 n n i=1   0≤j≤m * -1 ϕ j (X i )ϕ j (y)     dy = 1 n Var   0≤j≤m * -1 ϕ j (X 1 )ϕ j (y)   dy ≤ 1 n E      0≤j≤m * -1 ϕ j (X 1 )ϕ j (y)   2    dy = 1 n E   0≤j≤m * -1 ϕ 2 j (X 1 )   ≤ L(m * ) n .
Therefore (A4) holds with C 4 = 1. Lastly, recalling that a j = n -1 n i=1 ϕ j (X i ), and noting that a j = ϕ j , f = E( a j ), we have

E | ψ, f m * -E f m * | 2 = ψ(u)ψ(v)E   0≤j≤m * -1 ( a j -a j )ϕ j (u) 0≤k≤m * -1 ( a k -a k )ϕ k (v)   dudv = 1 n ψ(u)ψ(v) 0≤j,k≤m * -1 cov(ϕ j (X 1 )ϕ j (u), ϕ k (X 1 )ϕ k (v))dudv ≤ 1 n ψ(u)ψ(v) 0≤j,k≤m * -1 E(ϕ j (X 1 )ϕ j (u)ϕ k (X 1 )ϕ k (v))dudv
as the omitted term is the opposite of a nonnegative quantity (it can be written as the opposite of a square). It follows that

E | ψ, f m * -E f m * | 2 ≤ 1 n 0≤j,k≤m * -1 ψ(u)ψ(v)ϕ j (x)ϕ j (u)ϕ k (x)ϕ k (v)f (x)dudvdx = 1 n     0≤j≤m * -1 ϕ j (x)ϕ j (u)   ψ(u)du   2 f (x)dx ≤ f ∞ n     0≤j≤m * -1 ϕ j (x)ϕ j (u)   ψ(u)du   2 dx ≤ f ∞ n 0≤j,k≤m * -1 ψ(u)ψ(v) ϕ j (x)ϕ k (x)dx ϕ j (u)ϕ k (v)dudv = f ∞ n 0≤j≤m * -1 ψ(u)ϕ j (u)du 2 = f ∞ n ψ m * 2 ≤ f ∞ n ψ 2 .
We obtain that (A5) holds with C 5 = f ∞ . Now we consider case [Ker] and note that E(

f h * (x)) = f K h * (x). As by Young Inequality (8.1), f K h * 2 ≤ K h * 2 1 f 2 = K 2 1 f 2 , Assump- tion (A3) is satisfied with C 3 = K 2d 1 f 2 (or C 3 = K 2d 1 f ∞ ). A stan- dard bound (see Tsybakov (2009), proposition 1.4) yields Var f h * (x) dx ≤ K 2d /(n d j=1 h * j ) ≤ 1, so that (A4) holds with C 4 = 1. Lastly E ψ, 1 n n i=1 K h * (X i -•) -f K h * 2 = 1 n Var ( ψ, K h * (X 1 -.) ) ≤ 1 n E ψ, K h * (X 1 -.) 2 ≤ 1 n ψ(u)K h * (x -u)du 2 f (x)dx ≤ f ∞ n ψ K h * 2 ≤ f ∞ K 2 1 ψ 2 n = f ∞ K 2d 1 ψ 2 n ,
where we used Young's inequality. Therefore (A5) is holds with

C 5 = f ∞ K 2d 1 .

Proofs of adaptive results

Proof of Theorem 3.1

The proof starts by decompositions which are standard when studying Goldenschluger and Lepski (2011) methods and bounds. For κ ≥ κ, we get (see [START_REF] Comte | Nonparametric estimation[END_REF], sec 4.2)

E (f g) τ -f g 2 ≤ 3E( (f g) τ -f g 2 ) + 6κ V (τ ) + 6E(A(τ )). (7.1)
The first right-hand-side term has expectation controlled by applying Proposition 2.1 and the term V (τ ) can be associated with it. Only A(τ ) must be studied. We write

(f g) τ -(f g) τ,τ 2 ≤ 3( (f g) τ -(f g) τ 2 + (f g) τ -(f g) τ,τ 2 + (f g) τ,τ -(f g) τ,τ 2
).

The bound on the middle term is obtained with (A1)

(f g) τ -(f g) τ,τ 2 ≤ C 1 (f g) τ -(f g) 2
which refers to an adequate bias term. For the last term, we use (A1) and get

(f g) τ,τ -(f g) τ,τ 2 ≤ C 1 (f g) τ -(f g) τ 2 .
The expectation of this term is also studied in Proposition 2.1, with E(

(f g) τ - (f g) τ 2 ) = V + B 1 with the notation of this proof. Next (f g) τ -(f g) τ 2 ≤ 2( (f g) τ -(f τ * g) τ 2 + (f τ * g) τ -(f g) τ 2 )
where we have the bound (f

τ * g) τ -(f g) τ 2 ≤ C 1 g 2 ∞ f τ * -f 2 from (A1). Now we notice that (f g) τ -(f τ * g) τ 2 = sup t∈B(0,1) ν 2 n (t)
where B(0, 1) is a countable set of square integrable functions with t = 1 and the empirical process is defined by

ν n (t) = (f g) τ -(f τ * g) τ , t .
Therefore we have

(f g) τ -(f g) τ 2 ≤ 2 sup t∈B(0,1) ν 2 n (t) + C 1 g 2 ∞ f τ * -f 2 .
Reminding the definition of A(τ ) given by (3.1), we have

E(A(τ )) ≤ 6E sup τ ∈Kn sup t∈B(0,1) ν 2 n (t) - κ 6 V (τ ) + 3C 1 E( (f g) τ -(f g) τ 2 ) + 6C 2 1 g 2 ∞ f τ * -f 2 + 6C 1 (f g) τ -(f g) 2 . (7.2)
Thus, the result of Theorem 3.1 holds if we prove that, for two constants c 1 , c 2 , we have

τ ∈Kn E sup t∈B(0,1) ν 2 n (t) -c 1 V (τ ) ≤ C n . (7.3)
Indeed, plugging (7.3) in (7.2) and the result in (7.1) is the result of Theorem 3.1.

We prove (7.3). We split ν n in four terms:

ν n = ν n,1 + ν n,2 + ν n,3 + ν n,4
where for some positive constant c 0 to be defined in the sequel, we set

A(x) = {| f τ * (x) -f τ * (x)| < c 0 }, and
ν n,1 (t) = 1 n n i=1 [( f (Y i ) -f τ * (Y i ))1 A(Yi) K τ (Y i , .), t -(( f -f τ * )1 A g) τ , t ], ν n,2 (t) = 1 n n i=1 [( f (Y i ) -f τ * (Y i ))1 (A(Yi)) c K τ (Y i , .), t -(( f -f τ * )1 A c g) τ , t ], ν n,3 (t) = (( f -f τ * )g) τ , t = 1 n n i=1 ψ t (X i ), ψ t (X) = (K τ * (X, .)-f τ * g) τ , t , ν n,4 (t) = 1 n n i=1 [f τ * (Y i ) K τ (Y i , .), t -(f τ * g) τ , t ].

Study of ν n,2

We start by the study of ν n,2 as it leads to fix c 0 , and we first establish that E(sup t∈B(0,1) |ν n,2 (t)|) ≤ n -p for some positive p. It holds that

E sup t∈B(0,1) [ν 2 n,2 (t)] X ≤ E 1 n n i=1 [( f τ * (Y i ) -f τ * (Y i ))1 (A(Yi)) c K τ (Y i , .) -(( f τ * -f τ * )1 A c g) τ ] 2 X ≤ 1 n E ( f τ * (Y 1 ) -f τ * (Y 1 )) 2 1 (A(Y1)) c K τ (Y 1 , x) 2 X dx ≤ 2 L(τ ) n E ( f τ * (Y 1 ) 2 + f τ * (Y 1 )) 2 1 (A(Y1)) c X dx.
Using (A9) we obtain that f τ * (Y 1 ) 2 ≤ C 9 L 2 (τ * ) ≤ C 10 n 2 (where the last inequality follows from (A10)), and with (A7) that

E sup t∈B(0,1) [ν 2 n,2 (t)] X ≤ 2(C 10 n 2 + C 2 7 ) L(τ ) n P(A c (Y 1 )) ≤ 2(C 10 n 2 + C 2 7 )P(A c (Y 1 )).
Therefore,

E sup t∈B(0,1) [ν 2 n,2 (t)] ≤ 2(C 10 n + C 2 7 ) P(| f (y) -f τ * (y)| > c 0 )g(y)dy. (7.4)
We complete by applying the Bernstein inequality to Z i = K τ * (X i , y) yielding

P | f τ * (y) -f τ * (y)| > c 0 ≤ 2 exp - nc 2 0 2(v 2 2 + b 2 c 0 )
with v 2 2 a bound on Var(Z i ) and b 2 an a.s. bound on Z i . We find with (A9) that b 2 = √ C 9 L(τ * ) suits and with (A2) Var

(Z i ) ≤ f ∞ L(τ * ) = v 2 2 . Therefore, choosing c 0 = f ∞ , (7.5)
and using (A10) , it follows that

P | f τ * (y) -f τ * (y)| > c 0 ≤ 2n -(d+3) . (7.6)
Then, gathering (7.4) and (7.6) leads to E( sup t∈B(0,1)

ν 2 n,2 (t)) ≤ 4(C 10 n 2 +C 2 7 )n -(d+3) .
As a consequence under (A8), we get

τ ∈Kn E( sup t∈B(0,1) [ν 2 n,2 (t)]) ≤ C n . (7.7)
Study of ν n,1 . We apply the Talagrand inequality (see Lemma 8.1) to ν n,1 conditionally to X. Using that t → ν n,1 (t) is linear and the Cauchy-Schwarz inequality and t 2 = 1, we get

E sup t∈B(0,1) |ν n,1 (t)| |X 2 ≤ E sup t∈B(0,1) ν 2 n,1 (t)|X ≤ 1 n Var[( f τ * (Y 1 ) -f τ * (Y 1 ))1 A(Y1) K τ (Y 1 , y)|X]dy ≤ 1 n E[( f τ * (Y 1 ) -f τ * (Y 1 )) 2 1 A(Y1) K τ (Y 1 , y) 2 |X]dy ≤ c 2 0 L(τ ) n := H 2 1 .
Next, note that sup

x,t |( f τ * (x) -f τ * (x))1 A(x) K τ (x, .), t | ≤ c 0 sup t,x | K τ (x, .), t | ≤ c 0 K τ (x, .) = c 0 L(τ ) := b 1 (7.8) and sup t Var(( f τ * (Y 1 ) -f τ * (Y 1 ))1 A(Y1) K τ (Y 1 , .), t |X) ≤ sup t E[( f τ * (Y 1 ) 2 1 A(Y1) K τ (Y 1 , .), t |X) = sup t f τ * (u) 2 1 A(y) K τ (y, .), t |X)g(y)dy ≤ C 1 C 2 7 c 2 0 g ∞ := v 2 1 , (7.9)
where we used (A1) and (A7). Applying Lemma 8.1 with δ = 1 2 , it follows that

E sup t∈B(0,1) |ν n,1 (t)| 2 -4 c 2 0 L(τ ) n + X ≤ 4c 2 0 nK 1 C 1 C 2 7 g ∞ exp -K 1 L(τ ) 2 g ∞ + 49 K 1 C 2 (1/2) exp - K 1 C(1/2) 7 √ n , using L(τ ) ≤ n.
Since the latter bound does not depend on X, the inequality holds unconditionally in expectation. Therefore under (A8) and as L(τ ) ≥ 1, we get, for C a positive constant, and using (7.5),

τ ∈Kn E sup t∈B(0,1) |ν n,1 (t)| 2 -4 f 2 ∞ L(τ ) n + ≤ C n . ( 7
|ν n,3 (t)| 2 ≤ E ( f τ * -f τ * )g τ 2 = 1 n Var K τ * (X 1 , .)g τ (y) dy ≤ 1 n E K τ * (X 1 , .)g τ (y) 2 dy = 1 n K τ (x, y)K τ * (z, x)g(x)dx 2 f (z)dzdy ≤ C 1 f ∞ n g(x) 2 K τ (x, y) 2 dxdy ≤ C 1 f ∞ g 2 n L(τ ) := H 2 3 ,
using (A1) and (A2). Next, note that using Cauchy-Schwarz inequality and (A1)

sup x,t (K τ * (x, .) -f τ * g) τ , t ≤ C 1 sup x K τ * (x, .) -f τ * g ≤ C 1 g ∞ L(τ * ) + C 7 ≤ 2C 1 √ n g ∞ := b 3 , using (A7) with f τ * ≤ f τ * ∞ and (A10). Finally, (A1) enables to write sup t Var (K τ * (X, .) -f τ * g) τ , t ≤ sup t E K τ * (X, .) g) τ , t 2 ≤ C 1 f ∞ sup t K τ (u, v)t(u)du 2 g 2 (v)dv ≤ C 2 1 f ∞ g 2 ∞ := v 2 3 .
Applying Lemma 8.1 with δ = 1 2 , it follows that

E sup t∈B(0,1) |νn,3(t)| 2 -4C1 f ∞ g ∞ L(τ ) n + ≤ 4C 2 1 g 2 ∞ K1 n f ∞e -K 1 L(τ ) 2C 1 g ∞ + 49 × 4C 2 1 K1C 2 (1/2) e - K 1 C(1/2) √ f ∞ 14 √ C 1 g ∞ √ L(τ ) .
Therefore under (A8), we get, for C a positive constant,

τ ∈Kn E sup t∈B(0,1) |ν n,3 (t)| 2 -4C 1 f ∞ g ∞ L(τ ) n + ≤ C n . (7.11)
Study of ν n,4 Again we apply the Talagrand inequality, similar computations enable to derive from (A1) and (A7)

H 2 4 = C 1 f ∞ g ∞ L(τ )/n, v 2 4 = C 1 C 7 g ∞ and b 4 = C 7 √ n. It follows, by applying Lemma 8.1 with δ = 1 2 , that E sup t∈B(0,1) |νn,4(t)| 2 -4C1 f ∞ g ∞ L(τ ) n + ≤ 4 K1 C1C7 g ∞ n e -K 1 L(τ ) f ∞ 2C 7 + 49 × C 2 7 K1nC 2 (1/2) e - K 1 C(1/2) 14 √ C 1 f ∞ g ∞ C 7 √ n .
Therefore under (A8) we get, for C a positive constant,

τ ∈Kn E sup t∈B(0,1) |ν n,4 (t)| 2 -4C 1 f ∞ g ∞ L(τ ) n + ≤ C n . (7.12)
As a consequence, gathering (7.7)-(7.10)-(7.11) and (7.12) gives the result (7.3) for C a positive finite constant, depending on a, f ∞ , g ∞ , C 1 and C 7 .

Proof of Corollary 3.1

We already checked that [P] satisfies (A1) and (A2), and that for f bounded, Assumptions (A3)-(A5) hold under L(m * ) ≤ n, which is ensured by [P3]. For (A6), we write 0≤j≤m-1

ϕ j (x)ϕ j (z) 0≤k≤m -1 ϕ k (z)ϕ k (y)dz = 0≤j≤m-1 0≤k≤m -1 ϕ j (x)ϕ k (y) ϕ j (z)ϕ k (z)dz =δ j,k = 0≤j≤m∧m -1 ϕ j (x)ϕ j (y),
which is clearly symmetric in m, m . Therefore (A6) is fulfilled.

To check (A7) we have to bound f m * ∞ . Under [P2], we have

|f m * (x)| = a 0 (f )ϕ 0 (x) + 1≤j≤m * -1 a j (f )ϕ j (x) ≤ C ϕ   |a 0 | + j≥1 |a j (f )|   .
Then using [P5], we have

|f m * (x)| ≤ C ϕ   C ϕ + j≥1 j b a 2 j (f ) j≥1 j -b   ≤ C ϕ   C ϕ + C b j≥1 j -b   := C(b, ϕ) < +∞
since min 1≤j≤d b j > 1. Thus, Assumptions [P5] and [P2] imply that (A7) holds with C 7 = C(b, ϕ). Assumption [P4] is analogous to (A8). Lastly sup

x,y 0≤j≤m-1

ϕ j (x)ϕ j (y) 2 ≤ sup x   0≤j≤m-1 ϕ 2 j (x)   2 = L 2 (m),
so that (A9) holds with C 9 = 1, and therefore [P3] is (A10).

Proof of Corollary 3.2

We already checked that [Ker] satisfies (A1) and (A2), and that for f bounded, Assumptions (A3)-(A5) hold under L(h * ) ≤ n, which is ensured by [K3] (for n large enough). Next we compute

K h (x -z)K h (z -y)dz = K h (u)K h (x -u -y)du = K h (v -y)K h (x -v)dz,
where we made successively the affine changes of variables u = x -z and v = u + y. Using that K is even it follows that Assumption (A6) is fulfilled. Using [K1], we easily bound

f h * ∞ ≤ f ∞ K 1 , implying that (A7) holds with C 7 = f ∞ K 1 . Assumption [K4] implies (A8). Lastly, sup x,y |K h (x, y)| 2 ≤ L 2 (h) K 2 ∞ / K 4 , so that (A9) holds with C 9 = K 2 ∞ / K 4 and [K3] implies (A10).
(f g) h -f g 2 ≤ (f g) h -f g 2 + (pen(h) -ψ n (h)) -(pen( h) -ψ n ( h)) (7.13) with ψ n (h, h min ) = (f g) h -f g, (f g) hmin -f g .
As in Comte and Marie (2021), we decompose ψ n in

ψ n (h, h min ) = ψ 1,n (h, h min ) + ψ 2,n (h, h min ) + ψ 3,n (h, h min ).
First,

ψ 1,n (h, , h min ) := K h , K hmin n 2 n i=1 f 2 h * (Y i ) + U (h, h min ) n 2 = pen 1 (h) + U (h, h min ) n 2 ,
where

U n (h, h ) := 1≤i =j≤n f h * (Y i )K h (Y i -•) -(f g) h , f h * (Y j )K h (Y j -•) -(f g) h .
(7.14) Indeed,

K h , K hmin n 2 n i=1 f 2 h * (Y i ) = 1 n 2 n i=1 f h * (Y i )K h (Y i -•), f h * (Y i )K hmin (Y i -•) . Second, ψ2,n(h, hmin) := + 1 n (f g) h , (f g) h min - 1 n 2 n i=1 f h * (Yi)K h min (Yi -•), (f g) h + n i=1 f h * (Yi)K h (Yi -•), (f g) h min (7.15)
and lastly

ψ 3,n (h, h min ) := V n (h, h min ) + V n (h min , h) + (f g) h -f g, (f g) hmin -f g with V n (h, h ) := (f g) h -(f g) h , (f g) h -f g .
We state a series of Lemmas that permit to establish Theorem 4.2.

Lemma 7.1. Under the assumptions of Theorem 4.2, it holds that

E sup h,h ∈Hn |ψ 2,n (h, h )| ≤ C n ,
where C = C(f, g, K) is a positive constant depending on f , g, K.

Lemma 7.2. Under the assumptions of Theorem 4.2, for every ϑ ∈ (0, 1), it holds

E sup h,h |V n (h, h )| -ϑ (f g) h -f g 2 ≤ 1 2ϑ K 2 1 g ∞ f h * -f 2 +C log(n) n .
Lemma 7.3. Under the assumptions of Theorem 4.2, for every ϑ ∈ [0, 1], it holds

E sup h∈Hn |U n (h, h min )| n 2 -ϑ c 0 (f, g, K) nh ≤ ϑ f h * -f 2 + C log(n) n .
We deduce from (7.13) that

(f g) h -f g 2 ≤ (f g) h -f g 2 + 2 (pen 1 (h) -ψ n (h)) + 2pen 2 (h) -2(pen 1 ( h) -ψ n ( h)) -2pen 2 ( h). (7.16)
We have for all positive h

ψ n (h) -pen 1 (h) = U n (h, h min ) n 2 + ψ 2,n (h, h min ) + V n (h, h min ) + V n (h min , h) + (f g) h -f g, (f g) hmin -f g .
We note that for all positive θ

| (f g) h -f g, (f g) hmin -f g | ≤ θ 2 (f g) h -f g 2 + 1 2θ (f g) hmin -f g 2 .
Applying these for h = h we get

E ψ n ( h) -pen 1 ( h) -pen 2 ( h) ≤ E U n ( h, h min ) n 2 -θ c 0 (f, g, K) n h + E |V n ( h, h min )| - θ 2 (f g) hmin -f g 2 + E |V n (h min , h)| - θ 2 (f g) h -f g 2 + E θ (f g) h -f g 2 + (θ -κ) c 0 (f, g, K) n h + 1 2 (θ + 1 θ ) (f g) hmin -f g 2 + C n .
where we used Lemma 7.1. Now, using Lemmas 7.2 and 7.3, we get

E ψ n ( h) -pen 1 ( h) -pen 2 ( h) ≤ 1 2 (θ + 1 θ ) (f g) hmin -f g 2 + c 1 (f, g, K, θ) f h * -f 2 +θE (f g) h -f g 2 + (1 - κ θ ) c 0 (f, g, K) n h + C log(n) n . Observe that (f g) h -f g 2 = (f g) h -(f g) h 2 + (f g) h -f g 2 + 2V n (h, h). It follows that for all θ ∈ (0, 1 2 ), (1 -2θ) (f g) h -f g 2 + (1 - κ θ ) c 0 (f, g, K) nh -(f g) h -f g 2 = -2 V n (h, h) + θ (f g) h -f g 2 + (1 -2θ)(1 - κ θ ) c 0 (f, g, K) nh -(f g) h -(f g) h 2 ≤ -2 V n (h, h) + θ (f g) h -f g 2 ≤ 2(|V n (h, h)| -θ (f g) h -f g 2 )
provided that 1 -κ/θ ≤ 0. Therefore we choose κ ≥ θ and apply Lemma 7.2 again. We obtain

E ψ n ( h) -pen 1 ( h) -pen 2 ( h) ≤ 1 2 (θ + 1 θ ) (f g) hmin -f g 2 + c 2 (f, g, K) f h * -f 2 + θ 1 -2θ E (f g) h -f g 2 + C log(n) n .
Similarly, we get

E (|ψ n (h) -pen 1 (h)| + pen 2 (h)) ≤ 1 2 (θ + 1 θ ) (f g) hmin -f g 2 + c 2 (f, g, K) f h * -f 2 + θ (f g) h -f g 2 + (1 + κ θ ) c 0 (f, g, K) nh + C log(n) n .
Plugging the last two bounds in the expectation of (7.16) implies

1 - 2θ 1 -2θ E (f g) h -f g 2 ≤ E (f g) h -f g 2 + 2θ (f g) h -f g 2 + (1 + κ θ ) c 0 (f, g, K) nh + 2(θ + 1 θ ) (f g) hmin -f g 2 + 4c 2 (f, g, K) f h * -f 2 + C log(n) n .
Now applying Proposition 2.9 with a rougher bound on the variance (the constant is larger), we get for θ ∈ (0, 1/4), and κ ≥ θ,

1 - 2θ 1 -2θ E (f g) h -f g 2 ≤ 2(1 + θ) (f g) h -f g 2 + 2(1 + θ + κ) c 0 (f, g, K) nh + 2(θ + 1 θ ) (f g) hmin -f g 2 + 4c 2 (f, g, K) f h * -f 2 + C log(n) n .
We conclude that for κ ≥ 1/4 and all θ ∈ (0, 1/4),

E (f g) h -f g 2 ≤ 2(1 + c 1 (θ)) inf h∈Hn (f g) h -f g 2 + (1 + κ 1 + θ ) c 0 (f, g, K) nh +c 2 (θ) (f g) hmin -f g 2 + c 3 f h * -f 2 + C log(n) n ,
where

c 1 (θ) = 2θ(1 -θ)/(1 -3θ) > 0, c 2 (θ) = 2 (1 + θ 2 )(1 -2θ) θ(1 -2θ) > 0,
and c 3 and C are positive constants depending on θ, f, g, K. Taking θ = 0.2 we get the result given in Theorem 4.2.

7.4.1. Proof of Lemma 7.1

For the study of ψ 2,n , note that

| (f g) h , (f g) h | ≤ f g K h f g K h ≤ K 2 1 f g 2 and observe that E(| f h * (Y 1 )|) ≤ E( f 2 h * (Y 1 )) ≤ g ∞ ( K 2 + K 2 1 f 2 ).
As a consequence, for all positive h, h ,

E sup h,h ∈Hn 1 n n i=1 f h * (Y i )K h (Y i -•), (f g) h ≤ E | f h * (Y 1 )| sup h,h | K h (Y 1 -•), (f g) h | ≤ E(| f h * (Y 1 )|) K 2 1 f g ∞ ≤ g ∞ ( K 2 + K 2 1 f 2 ) K 2 1 f g ∞ .
From the definition of ψ 2,n given by (7.15), il follows that the result of Lemma 7.1 holds with C = (2 g ∞ (

K 2 + K 2 1 f 2 ) + g ∞ ) f g ∞ K 2 1 .
7.4.2. Proof of Lemma 7.2 and study of

V n (h, h ) We decompose V n (h, h ) = V n,1 (h, h ) + V n,2 (h, h ) + V n,3 (h, h ) with V n,1 (h, h ) = (f g) h -( f h * g) h , (f g) h -f g , V n,2 (h, h ) = ( f h * g) h -(f h * g) h , (f g) h -f g , V n,3 (h, h ) = (f h * g) h -(f g) h , (f g) h -f g .
We have for all positive θ

|V n,3 (h, h )| = | (f h * g) h -(f g) h , (f g) h -f g | ≤ ϑ 2 (f g) h -f g 2 + 1 2ϑ K 2 1 g 2 ∞ f h * -f 2 ,
implying that

E sup h,h |V n,3 (h, h )| - ϑ 2 (f g) h -f g 2 ≤ 1 2ϑ K 2 1 g ∞ f h * -f 2 .
(7.17) Next, we write

V n,2 (h, h ) = 1 n n i=1 (Z i -E(Z i )),
where

Z i -E(Z i ) := [(K h * (X i -•) -f h * (•)g(•)] K h , (f g) h -f

g and apply

Bernstein Inequality. Using that K is even, the variance bound is obtained by

VarZ 1 ≤ E(Z 2 1 ) = [(K h * (u -•) g(•)] K h , (f g) h -f g 2 f (u)du = [K h * (g [K h ((f g) h -f g)])(u)] 2 f (u)du ≤ f ∞ K h * (gK h ((f g) h -f g)) 2 ≤ f ∞ K h * 2 1 (gK h ((f g) h -f g)) 2 ≤ f ∞ g 2 ∞ K 4 1 (f g) h -f g) 2 := v 2 h,h .
On the other hand, we have

|Z 1 | ≤ sup z K h * (z -x)g(x)((f g) h -f g) K h )(x)dx ≤ g ∞ sup x |((f g) h -f g) K h )(x)| |K h * (z)|dz ≤ g ∞ K 1 sup x |((f g) h -f g)(x)| |K h (u)|du ≤ g ∞ K 2 1 (1 + K 1 ) f g ∞ := b h,h
. Therefore, Bernstein Inequality (8.2) implies that with probability larger than

1 -2e -λ |V n,2 (h, h )| ≤ 2λv 2 h,h n + λ n b h,h ≤ θ 4 (f g) h -f g) 2 + λ n f ∞ g 2 ∞ K 2 1 ( 2
K 2 1 θ + 1 + K 1 ).
This together with (8.3) and

|H n | = n leads to E sup h,h |V n,2 (h, h )| - ϑ 4 (f g) h -f g 2 ≤ C log(n) n . (7.18) For V n,1 , we write V n,1 (h, h ) = V b n,1 (h, h ) + V c n,1 (h, h ) with V b n,1 (h, h ) = 1 n n i=1 f b h * (Y i )K h (Y i -x) -( f b h * g) K h (x) ((f g) h -f g)(x)dx where f b h * (x) = f h * (x)1 | f h * (x)|≤d0 d 0 = 2 f ∞ K 1 .
Here, we apply the Bernstein inequality conditionally to X, with b

h,h = f ∞ f g ∞ K 2 1 (1 + K 1 ) and v 2 h,h = f 2 ∞ g ∞ K 4 1 (f g) h -f g 2 .
The orders of b h,h and v h,h being the same as for V n,2 and independent of X, the result for V b n,1 is :

E sup h,h |V b n,1 (h, h )| - ϑ 4 (f g) h -f g 2 ≤ C log(n) n . (7.19)
Lastly, by noticing that

| f h * (x)| ≤ | f h * (x) -f h * (x)| + f ∞ K 1 , we get P(| f h * (x)| > d 0 ) ≤ P(| f h * (x)-f h * (x)| > f ∞ K 1 ) ≤ P(| f h * (x)-f h * (x)| > c 0 ) as f ∞ K 1 ≥ f ∞ = c 0
by reminding of (7.5). Now using as in (7.6) Bernstein Inequality, we get, under [K3],

P(| f h * (x)| > d 0 ) ≤ 2n -4 . (7.20) Indeed condition [K3] is the translation of (A1) with d = 1 and C = K 2 ∞ / K 4 . Then as (f g) h -f g ∞ ≤ f g ∞ ( K 1 + 1) ≤ 2 f g ∞ K 1 , we get that E sup h,h ∈Hn |V c n,1 (h, h )| ≤ h,h ∈Hn 4 K ∞ h * f g ∞ K 1 P(| f h * (x)| > d 0 )g(x)dx ≤ C(f, g, K)|H n | 2 n 4 h * ≤ C n .
Note that the last bound is obtained using Warning. For the study of this term, in order to avoid burdensome technicalities, we assume that f h * is bounded by 2 K 1 f ∞ . We proved in the study of V n (see (7.20)) that that the probability of the complement is 1/n 4 under [K3].

|H n | ≤ n,
Recall that U n (h, h ) is defined by (7.14). We write that

f h * (Y i )K h (Y i -•) -(f g) h = f h * (Y i )K h (Y i -•) -( f h * g) K h (1) h + ( f h * g) K h -(f h * g) K h (2) h + (f h * g) K h -(f g) h (3)
h so that U n (h, h min ) can be splitted into 9 terms, denoted with obvious superindices (k, ) for k, ∈ {1, 2, 3}. These 9 terms can be reduced to 6 by symmetry arguments, denoted by U (i),(j) n (h, h min ) for i ≤ j ∈ {1, 2, 3}.

• Treatment of U

(1),( 1) n (h, h min ) we have, by analogy with Lemma 6.2 in Comte and Marie (2021), that, for every ϑ ∈ [0, 1],

E sup h∈Hn |U (1),(1) n |(h, h min ) n 2 - ϑ K 2 K 2 1 f 2 ∞ nh |X ≤ C log(n) n (7.21)
and it is easy to see that all bounds do note depend on X so de-conditioning is straightforward. 3) n (h, h min ) it is easy to handle thanks to the equality 2) n (h, h min ). Now, the process can be written as

• Treatment of U (3),(
U (3),(3) n (h, h min ) = n(n -1) (f h * g) K h -(f g) h , (f h * g) K hmin -(f g) hmin leading to the bound |U (3),(3) n (h, h min ) n 2 | ≤ [(f h * -f )g] K h [(f h * -f )g] K hmin ≤ K 2 1 g 2 ∞ f h * -f 2 . (7.22) • Treatment of U (2), (3) n
(h, h min ) first note that U (2), (3) n
(h, h min )/n 2 = [(n - 1)/n] (2) h , (3) hmin where (2) h , (3) hmin is equal to [( f h * g) -(f h * g)] K h ,((f h * -f )g) K hmin = [( f h * g) -(f h * g)], ((f h * -f )g) K hmin K h = [( f h * g) -(f h * g)], ((f h * -f )g) K h K hmin = [( f h * g) -(f h * g)] K hmin , ((f h * -f )g) K h and thus (2) h , (3) hmin = (2) hmin , (3) h , so that U (2),(3) n (h, h min ) = U (3),(
1 n n i=1 (Z 2,3 i -E(Z 2,3 i )), Z 2,3 i := K h * (X i -•)g, [(f h * -f )g] K h K hmin .
To apply Bernstein Inequality, we need to bound the variance and infinite norm of the Z 2,3 i 's. For the moment of order 2, we have

E[(Z 2,3 1 ) 2 ] = f (x) K h * (x -u)g(u)[(f -f h * )g] K h K hmin (u)du 2 dx ≤ f ∞ K h * [g[(f h * -f )g] K h K hmin ] 2 ≤ f ∞ K h * 2 1 [g[(f h * -f )g] K h K hmin ] 2 ≤ f ∞ g 2 ∞ K 6 1 f h * -f 2 := v.
For the upper bound, it holds:

sup x K h * (x -u)g(u)[(f -f h * )g] K h K hmin (u)du ≤ g ∞ sup u |[(f -f h * )g] K h K hmin (u)| sup x |K h * (x -u)|du ≤ g ∞ K 1 sup v ||[(f -f h * )g] K h (v)| sup u |K hmin (u)|du ≤ g 2 ∞ K 3 1 ( f ∞ + sup u |f K h * (u)|) ≤ g 2 ∞ K 3 1 (1 + K 1 ) f ∞ := b.
Then Bernstein Inequality implies that with probability larger than 1 -2e -λ , λ > 0, for any ϑ ∈ (0, 1),

|U (2),(3) n (h, h min )|/n 2 ≤ | (2) h , (3) hmin | ≤ 2vλ n + λ n b ≤ ϑ f -f h * 2 +C(K, f, g) λ ϑn .
As a consequence, we obtain

P sup h,h U (2),(3) (h, h ) n 2 -ϑ f -f h * 2 ≥ C(K, f, g) λ ϑn ≤ 2|H n | 2 e -λ .
Then it follows from (8.3) and

|H n | = n that E sup h,h U (2),(3) (h, h ) n 2 -ϑ f -f h * 2 + ≤ C log(n) n . (7.23)
• Treatment of U

(1),( 3) n (h, h min ) write that U

(1),( 3) n (h, h min )/n 2 = [(n-1)/n] (1) h , (3) hmin and (1) h , (3) hmin is

1 n n i=1 f h * (Y i )K h (Y i -•) -( f h * g) K h , [(f h * -f )g] K hmin .
We apply Bernstein Inequality conditionally to X, recalling that we consider

f h * bounded by 2 f ∞ K 1 . E f h * (Y i )K h (Y i -•), [(f h * -f )g] K hmin 2 |X = f h * (y)K h (y -u)[(f h * -f )g] K hmin (u)du 2 g(y)dy ≤ 4 K 2 1 f 2 ∞ g ∞ |K h | |(f h * -f )g| |K hmin | 2 ≤ 4 K 6 1 f 2 ∞ g 3 ∞ f h * -f 2 ,
by iterative application of Young Inequality. Next for the infinite norm

sup y | f h * (y)K h (y -•), [(f h * -f )g] K hmin | ≤ 2 f ∞ K 1 sup y |K h (y -u)||[(f h * -f )g] K hmin (u)|du ≤ 2 f ∞ K 1 |K h (v)|dv sup u |[(f h * -f )g] K hmin (u)| ≤ 2 f ∞ K 3 1 sup z |(f h * -f )(z)g(z)| ≤ 2 f 2 ∞ g ∞ K 3 1 (1 + K 1 ).
The bounds do not depend on X, it holds:

E sup h,h U (1),(3) (h, h ) n 2 -ϑ f -f h * 2 + ≤ C log(n) n (7.24)
for a constant C > depending on f, g, K. Moreover, the bounds do not depend on h, h min so the same bound hold for U

(3),( 1) n (h, h min )/n 2 .

• Treatment of U

(h, h min ) write U

(1),( 2)

n (h, h min )/n 2 = [(n-1)/n] (1) h , (2) hmin where (1) h , (2) hmin is 1 n n i=1 f h * (Y i )K h (Y i -•) -( f h * g) K h , [( f h * -f h * )g] K hmin .
First, we apply a Bernstein Inequality conditionally to X. The variance term is:

E f h * (Y i )K h (Y i -•), [( f h * -f h * )g] K hmin 2 ≤ 4 f ∞ g ∞ K 4 1 K h [( f h * -f h * )g] 2 := v.
For the upper bound, we get

sup y | f h * (y)K h (y -•), [( f h * -f h * )g] K hmin | ≤ 6 K 4 1 f 2 ∞ g ∞ := b
with usual tricks. Now, we can notice that

E K h [( f h * -f h * )g] 2 ≤ g ∞ K 2 1 K 2 nh . (7.25) So we write E sup h∈Hn U (1),(2) n (h, h min ) n 2 -4ϑ g ∞ K 2 1 K 2 nh ≤ E E sup h∈Hn U (1),(2) n (h, h min ) n 2 -ϑ K h [( f h * -f h * )g] 2 |X +ϑE sup h∈Hn K h [( f h * -f h * )g] 2 -4 g ∞ K 2 1 K 2 nh
The first term is bounded by taking the expectation of the conditional Bernstein, where constants are independent of the X i , which writes with the terms b, v:

E sup h∈Hn U (1),(2) n (h, h min ) n 2 -ϑ K h [( f h * -f h * )g] 2 ≤ C log(n) n . (7.26)
For the second, we use Talagrand Inequality, relying on the linear process

ν n (t) = K h [( f h * -f h * )g], t which fulfills sup t∈B(0,1) ν 2 n (t) = K h [( f h * -f h * )g] 2
where B(0, 1) is a countable dense subset of {t ∈ L 2 (R), t 2 = 1}. To apply Talagrand inequality, we compute H 2 , v, b. We have from (7.25) that

H 2 = g ∞ K 2 1 K 2 nh . Then we compute v 2 . sup t =1 Var K h (x -u)K h * (X 1 -u)t(x)dxdu ≤ sup t =1 E K h (x -u)K h * (X 1 -u)t(x)dxdu 2 = sup t =1 E (K h * K h t(X 1 )) 2 ≤ f ∞ sup t =1 K h * K h t 2 ≤ f ∞ K 4 1 := v 2 .
Next, for b, we find

sup t =1 sup y |K h * K h t(y)| ≤ sup t =1 sup y t 2 (x)dx (K h K h * (y -x)) 2 dx 1/2 = K h K h * ≤ K 1 K √ h := b.
The Talagrand Inequality gives

E sup h∈Hn K h [( f h * -f h * )g] 2 -4 g ∞ K 2 1 K 2 nh + ≤ C n h∈Hn exp(-c 1 /h) + card(H n ) exp(-C 2 √ n) .
As a consequence, as under [K4] and h ≤ 1, h∈Hn exp(-c 1 /h) ≤ Σ < +∞ and card(H n ) ≤ n, we get

E sup h∈Hn U (1),(2) n (h, h min ) n 2 -4ϑ g ∞ K 2 1 K 2 nh ≤ C log(n) n . (7.27) • Treatment of U (2),(2) n (h, h min ) write U (2),(2) n (h, h min )/n 2 = [(n-1)/n] (2) h , (2) hmin where (2) h , (2) hmin is [( f h * -f h * )g] K h , [( f h * -f h * )g] K hmin .
The decomposition of this term involves first a U-statistics related to X:

U X n (h, h min ) n 2 := 1 n 2 1≤i =j≤n (K h * (X i -u) -f K h * (u))g(u)K h (x -u)du × (K h * (X j -v) -f K h * (v))g(v)K hmin (x -v)dv dx
and terms corresponding to i = j that are studied separately:

1 n 2 n i=1 (K h * (X i -u) -f K h * (u))g(u)K h (x -u)du × (K h * (X i -v) -f K h * (v))g(v)K hmin (x -v)dv dx.
First, developing the latter product leads to the study of the four following terms. Two cross-terms that are are bounded by

1 n 2 n i=1 f (K h * g) K h K hmin (x)K h * (X i -x)g(x)dx ≤ g ∞ n sup x |f (K h * g) K h K hmin (x)| |K h * (z)|dz ≤ g ∞ K 2 1 n sup x |f (K h * g) K h (x)| ≤ g 2 ∞ K 3 1 n sup x |f K h * (x)| ≤ f ∞ g 2 ∞ K 4 1 n .
The product of last terms can be written

1 n (f K h * g) K h (x)(f K h * g) K hmin (x)dx ≤ 1 n (f K h * g) K h (f K h * g) K hmin ≤ 1 n K h 1 K hmin 1 g 2 ∞ f K h * 2 ≤ K 4 1 g 2 ∞ f 2 n .
Finally, the product of the first terms is

1 n 2 n i=1 K h * (X i -u)K h * (X i -v)K h (x-u)K hmin (x-v)g(u)g(v)dudvdx ≤ g 2 ∞ n 2 n i=1 |K h * | |K h |(X i -x)|K h * | |K hmin |(X i -x)dx = g 2 ∞ n |K h * | |K h |(z)|K h * | |K hmin |(z)dz ≤ g 2 ∞ n sup z |K h * | |K h |(z) |K h * | |K hmin |(z)dz ≤ g 2 ∞ K 3 1 K ∞ nh ,
implying that, for H i (u, v, x)

:= K h * (X i -u)K h * (X i -v)K h (x -u)K hmin (x -v) sup h 1 n 2 n i=1
Hi(u, v, x)g(u)g(v)dudvdx -

g 2 ∞ K 3 1 K ∞ nh ≤ 0.
Let us deal with the U-statistics U X n (h, h min ). We follow the line of the proof of Lemma 6.2 in Comte and Marie (2021) and write U X n (h, h min ) = 1≤i =j≤n G h,hmin (X i , X j ) where

G h,hmin (X i , X j ) = [(K h * (X i -•) -f h * )g] K h , [(K h * (X j -•) -f h * )g] K hmin .
Indeed, G h,hmin (X i , X j ) = G hmin,h (X i , X j ) as for all functions u, v it holds u K h , v K hmin = u, v K hmin K h = u, v K hmin K h = u K hmin , v K h .

We apply the deviation inequality for U-statistics of order 2, as in Lacour

sup x |[(K h * (z-•) -f h * )g] K hmin (x)| |[(K h * (X i -•) -f h * )g] K h (x)|dx ≤ sup z,z K hmin ∞ (K h * (z-•) -f h * )g 1 |[(K h * (X i -•) -f h * )g](x)|dx |K h (x)|dx ≤ 2 K ∞ h min g ∞ K 1 × 2 g ∞ K 2 1 = 4 g 2 ∞ K 3 1 K ∞ h min := a n . Thus a n λ 2 n 2 ≤ 4λ 2 g 2 ∞ K 3 1 K ∞ n .
Next b 2 n is a bound on n sup z E[G 2 h,hmin (z, X 1 )], we write

n sup z E[G 2 h,hmin (z, X 1 )] ≤ n sup z [(K h * (z -•) -f h * )g] K h 2 E( [(K h * (X 1 -•) -f h * )g] K hmin 2 ) ≤ K h 2 K hmin 2 sup z [(K h * (z -•) -f h * )g] 2 1 E( [(K h * (X 1 -•) -f h * )g] 2 1 ) ≤ 4n K 4 K 4 1 g 2 ∞ hh min := b 2 n .
We obtain

b n λ 3/2 n 2 ≤ 2λ 3/2 K 2 K 2 1 g ∞ √ hh min n 3/2 ≤ θ K 2 K 2 1 g 2 ∞ nh + λ 3 θ K 2 K 2 1 n 2 h min .
We compute c 2 n which is a bound on n 2 E G 2 h,hmin (X 1 , X 2 ) . Decompose

E G 2 h,hmin (X 1 , X 2 ) = E [(K h * (X 1 •) -f h * )g] K h , [(K h * (X 2 -•) -f h * )g] K hmin 2
into four squared terms. First,

f h * g K h , f h * g K hmin 2 ≤ K 4 1 f h * g 4 ≤ g 4 ∞ K 8 1 f 4 ≤ g 4 ∞ K 8 1 f 2 ∞ .
Second

f h * g K h , (K h * (X 2 -•)g) K hmin 2 ≤ sup z |f h * g K h (z)| |(K h * (X 2 -•)g) K hmin (z)|dz 2 ≤ K 1 sup z |(f h * g)(z)| K 1 |K h * (X 1 -u)g(u)|du 2 ≤ K 2 1 f ∞ g ∞ × g ∞ K 2 1 2 = ( f ∞ g 2 ∞ K 4 1 ) 2 .
The twin term in h min , h has clearly the same bound. Lastly

E (K h * (X 1 -•)g) K h , (K h * (X 2 -•)g) K hmin 2 = K h * (u -•)g K h (x)K h * (v -•)g K hmin (x)dx 2 f (u)f (v)dudv ≤ g 4 ∞ |K h * | |K h |(u -x)|K h * | |K hmin |(v -x)dx 2 f (u)f (v)dudv = g 4 ∞ [|K h * | |K h | |K h * | |K hmin |(u -v)] 2 f (u)f (v)dudv ≤ g 4 ∞ f 2 ∞ |K h * | |K h | |K h * | |K hmin | 2 ≤ g 4 ∞ f 2 ∞ K 6 1 K 2 h .
We get

c 2 n = n 2 h g 4 ∞ f 2 ∞ K 6 1 K 2 + 3 K 2 1 .
Thus , for all positive θ, λ it holds

c n √ λ n 2 ≤ θ g 4 ∞ f 2 ∞ K 6 1 nh + λ( K 2 + 3 K 2 1 ) 4n θ .
Lastly, the term d n is a bound on sup a,b 1≤i =j≤n

E [G h,hmin (X i , X j )a i (X i )b j (X j )] ,
where a k (•), b k (•) for k = 1, . . . , n is such that E(

n k=1 a 2 k (X k )) ≤ 1 and E( n k=1 b 2 k (X k )) ≤ 1.
Using the independence for i = j between functions of X i and functions of X j , we get that the term inside the sup is less than

n i=1 E (H i |K h ||a i (X i )|) , n i=1 E (H j |K hmin ||b j (X j )|) , (7.28)
where

H i = |K h * (X i -•)g -f h * g|. First we have n i=1 E (|K h * (X i -•)g -f h * g| |K h ||a i (X i )|) ≤ √ n n i=1 [E (|K h * (X i -•)g -f h * g| |K h ||a i (X i )|)] 2 1/2 ≤ √ n n i=1 E (|K h * (X i -•)g -f h * g| |K h |) 2 E(a 2 i (X i )) 1/2 As E (|K h * (X i -•)g -f h * g| |K h |) 2 ≤ f ∞ g 2 ∞ [ |K h * | |K h | 2 + f h * |K h | 2 ] we get n i=1 E (|K h * (X i -•)g -f h * g| |K h ||a i (X i )|) ≤ 2n f ∞ g ∞ K 1 K h .
Plugging this in formula (7.28), we get

n i=1 E (H i |K h ||a i (X i )|) , n i=1 E (H j |K hmin ||b j (X j )|) ≤ 2n f ∞ g ∞ K 1 K h n j=1 E |K h * (X j -•)g -f h * g| |K hmin (u)|du |b j (X j )| ≤ 2n f ∞ g ∞ K 1 K h × 2 K 2 1 g ∞   n j=1 E(b j (X j )|)   ≤ 2 2n f ∞ g 2 ∞ K 2 1 K h × √ n.
Therefore,

d n := 2 2 f ∞ g 2 ∞ K 2 1 K n √ h .
It follows that

d n λ n 2 ≤ θ f 2 ∞ g 4 ∞ K 4 1 K 2 nh + 2 λ 2 n .
Applying the deviation inequality for U-statistics of order

Appendix

In the paper we make an extensive use of the following:

• The Young Inequality: for u ∈ L p (R d ) and v ∈ L q (R d) , 1 ≤ p, q ≤ r ≤ ∞,

u v r ≤ u p v q , 1 r + 1 = 1 p + 1 q , ( 8

FirstProposition 2 . 2 .

 22 we state that the above functions fulfill assumptions (A1)-(A2): The functions K m defined in [P] and K h defined in [Ker] satisfy assumptions (A1)-(A2) and belong to K. The order of L(K m ) := L(m) depends on the choice of the basis. In dimension 1, for the trigonometric basis for m odd and I = [0, 1], it holds L(m) = m. For the Hermite basis where I = R, we have L(m) ≤ C H √ m (see Lemma 1 in Comte and Lacour (2021) and section 2.4). For the Legendre polynomial basis where I = [-1, 1] it holds that L(m) = m 2 , see Cohen et al. (2013, p.831). In any case, we consider that L(m) ≥ 1, which holds at least for m ≥ m 0 . Extension to dimension d is straightforward by tensorization of the bases, and L(m) = d i=1 L(m j ). In the kernel case, we simply have L(K h ) := L(h).

. 10 )

 10 Study of ν n,3 Similarly to ν n,1 we apply the Talagrand inequality E sup t∈B(0,1)

7. 4 .

 4 Proof of Theorem 4.2 Following the first step in Lacour et al. (2017), we write

  et al. (2017), see Theorem 3.4 in Houdré and Reynaud-Bouret (2003). Following the notations of the aforementionned papers, we have to compute four bounds a n , b n , c n , d n . First a n is a bound on sup z,z |G h,hmin (z, z )|. sup z,z |G h,hmin (z, z )| ≤ sup z,z

. 1 )••

 1 The Bernstein inequality: For i.i.d. random variables Z i , setS n = 1 n n i=1 (Z i -E[Z i ]). If E[Z 2 1 ] ≤ v and |Z 1 | ≤ b a.s. then with probability larger than 1 -2e -λ , for any λ > 0, Deriving a bound in expectation from a bound on probability: If P(Z ≥ κλ n ) ≤ n 2 e -λ for all λ > 0, then it holdsE[Z + ] 2 e -A ,and A = 2 log(n) gives the result.

  .1) This estimator is considered in Lerasle et al. (2016) and for specific choices of K * , it covers for instance projection estimators, kernel estimators or weighted projection estimators. The first two examples are detailed in Section 2.2. Remark that E(

  bounded with large probability, see(7.6). Theorem 4.2. Consider the kernel case[K]. Assume that Assumptions [K1]-[K4] hold and that 1/(nh min ) ≤ 1. Then, for any κ ≥ 1/4, we have

  g 2 and the associated deviations, from N = 100 independent datasets with different values of sample size n = 200, 1000 and 2000. All adaptive methods require the calibration of constants κ's in penalties. This is done by preliminary simulation experiments. For calibration strategies (dimension jump and slope heuristics), the reader is referred to Baudry et al. (2012), and to[START_REF] Lerasle | Optimal model selection in density estimation[END_REF] for theoretical justifications. Here, we test a grid of values of κ's, the tests are conducted on a set of densities which are different from the one considered hereafter, to avoid overfitting. The different estimators are computed on the same datasets and compared.

Table 1 Case

 1

	n	Product	First X	Optimal first	Oracle
		Hermite Kernel	Hermite Kernel	Hermite	Kernel	Hermite Kernel
	200	18.2	3.21	12.3	21.0	7.79	5.80	2.08	4.69
		(1.85)	(2.97)	(3.32)	(3.96)	(2.32,100)	(8.60,79)	(1.64)	(8.00)
	1000	4.41	1.22	3.79	1.05	2.19	0.93	0.54	0.74
		(0.57)	(1.05)	(1.09)	(0.68)	(0.66,100)	(0.6,26)	(0.40)	(0.54)
	2000	2.26	0.43	1.92	0.45	1.18	0.40	0.27	0.31
		(0.38)	(0.29)	(0.46)	(0.32)	(0.37,100)	(0.28,47)	(0.21)	(0.23)
	n	Product	First X	Optimal first	Oracle
		Hermite Kernel	Hermite Kernel	Hermite	Kernel	Hermite	Kernel
	200	27.4	16.6	10.5	5.21	10.5	5.21	4.27	3.79
		(8.13)	(9.11)	(5.07)	(3.36)	(5.07,0)	(3.36,0)	(2.87)	(2.18)
	1000	38.1	10.1	3.86	2.77	3.86	2.80	1.52 (1.02)	1.25
		(6.63)	(7.29)	(1.43)	(2.12)	(1.43,0)	(2.13,2)	(1.02)	(0.77)
	2000	36.2	11.0	2.62	1.75	2.62	1.75	0.90	0.80
		(4.98)	(10.8)	(0.80)	(1.57)	(0.80,0)	(1.57,0)	0.64)	(0.55)

[BU]: L 2 -risks with std in parenthesis (both multiplied by 10 2 ), Dmax = 130. For "Optimal first", the bold sub-script is the number of times where Y is selected first.

Table 2

 2

Table 3

 3 Case [NL]: L 2 -risks with std in parenthesis (both multiplied by 10 2 ), Dmax = 100. For the Optimal first the bold sub-script is the number of times where Y is selected first.

	n	Product	First X	Optimal first	Oracle
		Hermite Kernel	Hermite Kernel	Hermite	Kernel	Hermite Kernel
	200	4.60	49.8	3.24	5.19	3.24	5.18	2.43	2.65
		(2.59)	(71.8)	(2.14)	(2.48)	(2.14,48)	(2.48,23)	(1.67)	(1.86)
	1000	2.31	72.3	1.84	2.51	1.87	2.46	0.84	0.78
		(0.74)	(74.6)	(0.54)	(1.87)	(0.50,100)	(1.87,13)	(0.52)	(0.68)
	2000	0.92	64.1	0.73	1.88	0.83	1.77	0.43	0.39
		(0.74)	85.4)	(0.49)	(1.73)	(0.53,95)	(1.66,10)	(0.30)	(0.31)

Table 4

 4 Case

[NC]: L 2 -risks with std in parenthesis (both multiplied by 10 2 ), Dmax = 50. For the Optimal first the bold sub-script is the number of times where Y is selected first.

Table 5

 5

  2 (see Lacour et al. (2017) and Theorem 3.4 in Houdré and Reynaud-Bouret (2003)) leads thus to The result of Lemma 7.3 follows by gathering the bounds (7.21), (7.26), (7.27), (7.29), (7.24), (7.23), (7.22).

	E sup h∈Hn	U X n (h, h min ) n 2	-θ	f 2 ∞ g 4 ∞ K 4 1 K 2 nh	≤ C	log(n) n	.
	Therefore
	E sup h∈Hn	U	(2),(2) n n 2 (h, h min )	-θ	f 2 ∞ g 4 ∞ K 4 1 K 2 nh	≤ C	log(n) n	. (7.29)

The Talagrand inequality. The result below follows from the Talagrand concentration inequality given in Klein and Rio (2005) and arguments in Birgé and Massart (1998) (see the proof of their Corollary 2 page 354). Lemma 8.1. (Talagrand Inequality) Let Y 1 , . . . , Y n be independent random variables and let F be a countable class of uniformly bounded measurable functions. Consider ν n , the centered empirical process defined by

for f ∈ F. Assume there exists three positive constants M, H and v such that

Then, for any δ > 0 the following holds

By standard density arguments, this result can be extended to the case where F is a unit ball of a linear normed space, after checking that f → ν n (f ) is continuous and F contains a countable dense family.