Karine Bertin

Claire Lacour

Vincent Rivoirard

Adaptive pointwise estimation of conditional density function

Keywords: conditional density, adaptive estimation, kernel rules, projection estimates, oracle inequality, minimax rates, anisotropic Hölder spaces

In this paper we consider the problem of estimating f , the conditional density of Y given X, by using an independent sample distributed as (X, Y ) in the multivariate setting. We consider the estimation of f (x, .) where x is a fixed point. We define two different procedures of estimation, the first one using kernel rules, the second one inspired from projection methods. Both adapted estimators are tuned by using the Goldenshluger and Lepski methodology. After deriving lower bounds, we show that these procedures satisfy oracle inequalities and are optimal from the minimax point of view on anisotropic Hölder balls. Furthermore, our results allow us to measure precisely the influence of f X (x) on rates of convergence, where f X is the density of X. Finally, some simulations illustrate the good behavior of our tuned estimates in practice.

Introduction

Motivation

In this paper, we consider the problem of conditional density estimation. For this purpose, we assume we are given an i.i.d. sample (X i , Y i ) of couples of random vectors (for any i, X i ∈ R d1 and Y i ∈ R d2 , with d 1 ≥ 1 and d 2 ≥ 1) with common probability density function f X,Y and marginal densities f Y and f X : for any y ∈ R d2 and any x ∈ R d1 ,

f Y (y) = R d 1 f X,Y (u, y)du, f X (x) = R d 2 f X,Y (x, v)dv.
The conditional density function of Y i given X i = x is defined by

f (x, y) = f X,Y (x, y) f X (x)
for all y ∈ R d2 and x ∈ R d1 such that f X (x) > 0. Our goal is to estimate f using the observations (X i , Y i ). The conditional density is much more informative than the simple regression function and then its estimation has many practical applications: in Actuaries [START_REF] Efromovich | Oracle inequality for conditional density estimation and an actuarial example[END_REF]), Medicine [START_REF] With | Nonparametric conditional density estimation using piecewise-linear solution path of kernel quantile regression[END_REF]), Economy [START_REF] Hall | Cross-validation and the estimation of conditional probability densities[END_REF]), Meteorology [START_REF] Jeon | Using conditional kernel density estimation for wind power density forecasting[END_REF]) among others. In particular, due to recent advances in ABC methods, the problem of conditional density estimation in the multivariate setting is of main interest. Indeed, the ABC methodology, where ABC stands for approximate Bayesian computation, offers a resolution of untractable-yet-simulable models, that is models for which it is impossible to calculate the likelihood. The standard ABC procedure is very intuitive and consists in

• simulating a lot of parameters values using the prior distribution and, for each parameter value, a corresponding dataset,

• comparing this simulated dataset to the observed one; 1

• finally, keeping the parameter values for which distance between the simulated dataset and the observed one is smaller than a tolerance level.

That is a crude nonparametric approximation of the target posterior distribution (the conditional distribution of the parameters given the observation). Even if some nonparametric perspectives have been considered (see [START_REF] Blum | Approximate bayesian computation: a nonparametric perspective[END_REF] or Biau et al. ( 2012)), we easily imagine that, using the simulated couples (parameters and datasets), a good nonparametric estimation of the posterior distribution can be a credible alternative to the ABC method. Such a procedure has to consider that the conditional density has to be estimated only for the observed value in the conditioning.

All previous points clearly motivate our work and in the sequel, we aim at providing an estimate with the following 4 requirements:

1. The estimate has to be fully data-driven and implementable in a reasonable computational time.

2. The parameters of the method have to adapt to the function f in the neighborhood of x.

Tuning the hyperparameters of the estimate has to be an easy task.

3. The estimate should be optimal from the theoretical point of view in an asymptotic setting but also in a non-asymptotic one.

4. Estimating f in neighborhoods of points x where f X (x) is equal or close to 0 is of course a difficult task and a loss is unavoidable. Studying this loss and providing estimates that are optimal with respect to this problem are the fourth motivation of this paper.

To address the problem of conditional density estimation, the first idea of statisticians was to estimate f by the ratio of a kernel estimator of the joint density f X,Y and a kernel estimator of f X : see [START_REF] Rosenblatt | Conditional probability density and regression estimators[END_REF], [START_REF] Chen | The estimation of conditional densities[END_REF], or also [START_REF] Hyndman | Estimating and visualizing conditional densities[END_REF], De [START_REF] De Gooijer | On conditional density estimation[END_REF] for refinements of this method. A important work in this line is the one of [START_REF] Fan | Estimation of conditional densities and sensitivity measures in nonlinear dynamical systems[END_REF] who extend the Rosenblatt estimator by a local polynomial method (see also [START_REF] Hyndman | Nonparametric estimation and symmetry tests for conditional density functions[END_REF]). The estimators introduced in the ABC literature are also of this kind: a linear (or quadratic) adjustment is realized on the data before applying the classic quotient estimator [START_REF] Beaumont | Approximate bayesian computation in population genetics[END_REF] , [START_REF] Blum | Approximate bayesian computation: a nonparametric perspective[END_REF]). Other directions are investigated by [START_REF] Bouaziz | Conditional density estimation in a censored single-index regression model[END_REF] who use a single-index model, or [START_REF] Györfi | Nonparametric estimation of conditional distributions[END_REF] who partition the space and obtain a piecewise constant estimate. All these papers have in common to involve a ratio between two density estimates, though we can mention [START_REF] Stone | The use of polynomial splines and their tensor products in multivariate function estimation[END_REF] for a spline tensor based maximum likelihood estimator. An original approach which rather involves a product is the copula one of [START_REF] Faugeras | A quantile-copula approach to conditional density estimation[END_REF]. But his method depends on a bandwidth, that remains to select from the data. In particular, for all of these methods, the second requirement is not satisfied.

The practical choice of the bandwidth and cross-validation methods are studied in [START_REF] Bashtannyk | Bandwidth selection for kernel conditional density estimation[END_REF] and [START_REF] Fan | A crossvalidation method for estimating conditional densities[END_REF]. However, no theoretical result is associated to this study. The first adaptive results can be found in [START_REF] Clémençon | Adaptive estimation of the transition density of a regular Markov chain[END_REF] for the estimation of the transition density of a Markov chain, which is a very similar problem to the one of conditional density estimation (set Y i = X i+1 ). He uses thresholding of wavelet estimator. Afterwards, using different methods, the works of [START_REF] Brunel | Adaptive estimation of the conditional density in the presence of censoring[END_REF] or [START_REF] Efromovich | Conditional density estimation in a regression setting[END_REF] yield oracle inequalities and minimax rates of convergence for anisotropic conditional densities. The case of inhomogenous regularities is studied in [START_REF] Akakpo | Inhomogeneous and anisotropic conditional density estimation from dependent data[END_REF] or [START_REF] Sart | Estimation of the transition density of a Markov chain[END_REF] in the case of Markov chains. Still for global adaptive approach, we can cite [START_REF] Chagny | Warped bases for conditional density estimation[END_REF] who applies the Goldenshluger-Lepski methodology to warped bases and Le [START_REF] Le Pennec | Partition-based conditional density estimation[END_REF] who use a model selection approach with Kullback risk. All the previous authors use a global risk and either consider integration with respect to f X (x)dx or assume that f X is bounded from below by a constant (as it is done in regression estimation). We are interested in precisely studying this assumption to show that it is unavoidable in some sense.

Our strategy and our contributions

Our strategy to estimate f is based on the Goldenshluger and Lepski methodology proposed in the seminal papers Goldenshluger andLepski (2011, 2012) in the case of density estimation and extended to the white noise and regression models in [START_REF] Goldenshluger | General selection rule from a family of linear estimators[END_REF]. This strategy detailed in Section 2 allows us to derive two procedures: kernel and projection rules. If they seem different, they are based on similar ideas and they lead to quite similar theoretical results. Our method automatically selects a regularization parameter, and in particular a bandwidth for kernel rules. Note that the tolerance level in ABC methods can be reinterpreted as a regularization parameter.

Unlike most of previous works of the literature, we shall not use a global risk and we will evaluate the quality of an estimator f at a fixed point x ∈ R and in the L 2 -norm with respect to the variable y. In other words, we will use the risk

R x ( f , q) = E f -f q x,2 1 q , (1.1)
where for any function g,

g x,2 = R d 2
g 2 (x, y)dy

1/2
.

(1.

2)

The previously mentioned motivating applications show that the tuning parameter has to depend on x, which is not the case of other cited-above adaptive methods. As shown later, combined with the Goldenshluger and Lepski methodology, considering this risk allows us to derive estimates satisfying this property. Furthermore, for a given x, y → f (x, y) is a density, so it is natural for us to study the estimation pointwisely in x.

From the theoretical point of view, we establish non asymptotic meaningful oracle inequalities and rates of convergence on anisotropic Hölder balls H d (α, L). More precisely, in Proposition 1 and Theorem 4, we establish lower bounds in oracle and minimax settings. Then, upper bounds of the risk for our adaptive kernel procedure are established (see Theorems 1, 2 and 5). If the density f X is smooth enough, Corollary 1 shows that upper and lower bounds match up to constants in the asymptotic setting. Then, there is a natural question: is this assumption on the smoothness of f X mandatory? We prove that the answer is no by establishing the upper bound of the risk for our adaptive projection estimate (see Theorems 3 and 6). In particular, the latter achieves a polynomial rate of convergence on anisotropic Hölder balls with rate exponent ᾱ/(2 ᾱ + 1), where ᾱ is the classical anisotropic smoothness index. To our knowledge, this rate exponent is new in the conditional density estimation setting for the pointwise risk in x. Our result also explicits the dependence of the rate with respect to L on the one hand and to f X (x) on the other hand, which is not classical. Indeed, as previously recalled, estimation is harder when f X (x) is small and this is the reason why most of the papers assume that f X is bounded from below by a constant. For kernel rules, our study is sharp enough to measure precisely the influence of f X (x) on the performance of our procedure. Under some conditions and if the sample size is n, we show that the order of magnitude of minimax rates (that are achieved by our procedure), is (nf X (x)) ᾱ/(2 ᾱ+1) . We conclude that our setting is equivalent to the setting where f X is locally bounded from 0 by 1 but we observe nf X (x) observations instead of n.

Finally, we study our procedures from a practical point of view. We aim at completing theoretical results by studying tuning issues. More precisely, our procedures are data driven and tuning parameters depend on x and on an hyperparameter η, a constant that has to be tuned. We lead a precise study that shows how to choose η in practice. We also show that reconstructions for various examples and various values of n are satisfying. All these results show that our procedures fulfill requirements listed in Section 1.1.

Overview and notations

Our paper is organized as follows. In Section 2, we present the Goldenshluger and Lepski methodology in the setting of conditional density estimation. In Sections 4 and 5 respectively, kernel and projection rules are derived and studied in the oracle setting by using assumptions of Section 3. Rates of convergence on anisotropic Hölder balls are studied in Section 6. Then a simulation study is lead in Section 7, where we focus on tuning aspects of our procedures. Finally, in Section 8 and in Appendix, we prove our results. To avoid too tedious technical aspects, most of proofs are only given for d 1 = d 2 = 1 but can easily be extended to the general case. In the sequel, we assume that the sample size is 2n. The first n observations (X 1 , Y 1 ), . . . , (X n , Y n ) are used to estimate f , whereas X n+1 , . . . , X 2n are used to estimate f X when necessary. We recall that for any i, X i ∈ R d1 and Y i ∈ R d2 and we set d = d 1 + d 2 .

In addition to notations R x (•, •) and • x,2 introduced in (1.1) and (1.2), we use for any 1 ≤ q < ∞ • q , the classical L q -norm of any function g:

g q q = |g(x)| q dx.
Some assumptions on functions f and f X , specified in Section 3, will depend on the following neighborhood of x, denoted V n (x): Given A a positive real number and (k n ) n any positive sequence larger than 1 only depending on n and such that k n goes to +∞, we set:

V n (x) = d1 i=1 x i - 2A k n , x i + 2A k n .
Note that the size of V n (x) goes to 0. Then, we set

f ∞ = sup t∈Vn(x) sup y∈R d 2 f (t, y) ∈ [0, +∞], f X ∞ = sup t∈Vn(x) f X (t) ∈ [0, +∞] and δ = inf t∈Vn(x)
f X (t) ≥ 0.

Our results will strongly depend on these quantities. Finally, for any u ∈ R, we set {u} + = max(u, 0).

Methodology

The Goldenshluger-Lepski methodology

This section is devoted to the description of the Goldenshluger-Lepski methodology (GLM for short) in the setting of conditional density estimation. The GLM consists in selecting an estimate from a family of estimates, each of them depending on a parameter m. Most of the time, choosing this tuning parameter can be associated to a regularization scheme: if we take m too small, then the estimate oversmooths; if we take m too large, data are overfitted.

So, given a set of parameters M n , for any m ∈ M n , we assume we are given a smoothing linear operator denoted K m and an estimate fm . For any m ∈ M n , fm is related to K m (f ) via its expectation and we assume that E[ fm ] is close to (or equal to) K m (f ). The main assumptions needed for applying the GLM are

K m • K m ′ = K m ′ • K m (2.1) and K m ( fm ′ ) = K m ′ ( fm ) (2.2)
for any m, m ′ ∈ M n . The GLM is a convenient way to select an estimate among ( fm ) m∈Mn which amounts to selecting m ∈ M n and can be described as follows: For • a given norm and σ a function to be chosen later, we set for any m in M n ,

A(m) := sup m ′ ∈Mn fm ′ -K m ′ ( fm ) -σ(m ′ ) + .
Then we estimate f by using f := f m, where m is selected as follows:

m := arg min m∈Mn {A(m) + σ(m)} .
This choice can be seen as a bias-variance tradeoff, with σ(m) an estimator of the standard deviation of fm and A(m) an estimator of the bias (see later). Let us now fix m ∈ M n . Using (2.2), we have:

f -f = f m -f ≤ f m -K m( fm ) + K m ( f m) -fm + fm -f ≤ A(m) + σ( m) + A( m) + σ(m) + fm -f ≤ 2A(m) + 2σ(m) + fm -K m (f ) + K m (f ) -f . But A(m) = sup m ′ ∈Mn fm ′ -K m ′ ( fm ) -σ(m ′ ) + ≤ ξ(m) + B(m) with for any m ∈ M n , ξ(m) := sup m ′ ∈Mn ( fm ′ -K m ′ (f )) -(K m ′ ( fm ) -(K m ′ • K m )(f )) -σ(m ′ ) + and B(m) := sup m ′ ∈Mn K m ′ (f ) -(K m ′ • K m )(f ) .
We finally obtain:

f -f ≤ 2B(m) + 2σ(m) + fm -K m (f ) + f -K m (f ) + 2ξ(m). (2.3) Now, let us assume that |||K||| := sup m∈Mn |||K m ||| < ∞, (2.4)
where |||K m ||| is the operator norm of K m associated with • . In this case, B(m) is upper bounded by f -K m (f ) up to the constant |||K|||, which corresponds to the bias of fm if

K m (f ) = E[ fm ].
(2.5) Furthermore, using (2.1) and (2.2), for any m ∈ M n ,

ξ(m) ≤ sup m ′ ∈Mn (1+|||K|||) fm ′ -K m ′ (f ) -σ(m ′ ) + .
Then we choose σ such that, with high probability, for any m ∈ M n , fm -K m (f ) ≤ σ(m)/(|||K||| + 1).

(2.6) So, (2.3) gives that, with high probability,

f -f ≤ C inf m∈Mn { f -K m (f ) + σ(m)} , (2.7)
where C depends only on |||K|||. Since under (2.5), σ(m) controls the fluctuations of fm around its expectation, σ 2 (m) can be viewed as a variance term and the oracle inequality (2.7) justifies our procedure. Previous computations combined with the upper bound of A(m) also justify why A(m) is viewed as an estimator of the bias. Now, we illustrate this methodology with two natural smoothing linear operators: convolution and projection. The natural estimates associated with these operators are kernel rules and projection rules respectively. Next paragraphs describe the main aspects of both procedures and discuss assumptions (2.1), (2.2), (2.4) and (2.5) which are the key steps of the GLM.

Convolution and kernel rules

Kernel rules are the most classical procedures for conditional density estimation. To estimate f , the natural approach consists in considering the ratio of a kernel estimate of f X,Y with a kernel estimate of f X . Actually, we use an alternative approach and to present our main ideas, we assume for a while that f X is known and positive.

We introduce a kernel K, namely a bounded integrable function K such that K(u, v)dudv = 1 and K 2 < ∞. Then, given a regularization parameter, namely a d-dimensional bandwidth h belonging to a set H n to be specified later, we set

K h (u, v) = 1 d i=1 h i K u 1 h 1 , . . . , u d1 h d1 , v 1 h d1+1 , . . . , v d2 h d , u ∈ R d1 , v ∈ R d2 .
Then, we use the setting of Section 2.1 except that regularization parameters are denoted h, instead of m to match with usual notation of the literature. Similarly, the set of bandwidths is denoted by H n , instead of M n . For any h ∈ H n , we set:

∀ g ∈ L 2 , K h (g) = K h * g
where * denotes the standard convolution product and

fh (x, y) := 1 n n i=1 1 f X (X i ) K h (x -X i , y -Y i ).
(2.8)

The regularization operator K h corresponds to the convolution with K h . Note that

E[ fh (x, y)] = (K h * f )(x, y).
Therefore 3 of 4 assumptions of the GLM are satisfied, namely (2.1), (2.2) and (2.5). Unfortunately, (2.4) is satisfied with • the classic L 2 -norm but not with • x,2 , as adopted in this paper. We shall see how to overcome this problem later on. Another drawback of this description is that fh is based on the knowledge of f X . A kernel rule based on fX , an estimate of f X , is proposed in Section 4.2 where we define σ (see (2.6)) to apply the GLM methodology and then to obtain oracle inequalities similar to (2.7). Additional terms in oracle inequalities will be the price to pay for using fX instead of f X .

Projection

We introduce a collection of models (S m ) m∈Mn and for any m, we denote K m the projection on (S m , <, > X ) where <, > X is the scalar product defined by: ∀g, g ′ , < g, g ′ > X = g(u, y)g ′ (u, y)f X (u)dudy.

(2.9)

Of course, (2.1) is satisfied, but as for kernel rules, (2.4) is not valid with

K m • K m ′ (f ) = K m∧m ′ (f ), set for any (m, m ′ ) ∈ M 2 n , Km ( fm ′ ) = fm∧m ′ .
This operator is only defined on the set of the estimators fm but verifies (2.2). Now the previous reasoning can be reproduced and the GLM described in Section 2.1 can be applied by replacing K m ′ by Km ′ in A(m) and by setting

ξ(m) := sup m ′ ∈Mn ( fm ′ -K m ′ (f )) -( Km ′ ( fm ) -(K m ′ • K m )(f )) -σ(m ′ ) + .
In Section 5.2, we define σ such that for all m, m ′ ∈ M n , σ(m ∧ m ′ ) ≤ σ(m ′ ) and similarly to (2.6), with high probability, for any m ∈ M n ,

fm -K m (f ) ≤ σ(m) 2 .
Then, for all m, m ′ ∈ M n ,

Km ′ ( fm ) -(K m ′ • K m )(f ) = fm∧m ′ -K m∧m ′ (f ) ≤ σ(m ∧ m ′ ) 2 ≤ σ(m ′
) 2 so that ξ(m) vanishes with high probability. Thus, we shall be able to derive oracle inequalities in this case as well.

Discussion

We have described two estimation schemes for which the GLM is appropriate: kernel and projection rules. In these schemes, the main commutative properties of the GLM, namely (2.1) and (2.2), are satisfied. Due to the particular choice of the loss-function • x,2 , the property (2.4) is not satisfied. However in both schemes, we shall be able to prove that for any function g

K m (g) x,2 ≤ C sup t∈Vn(x) g t,2
(2.11)

where C is a constant, V n (x) is the neighborhood of x introduced in Section 1.3, and this property will allow us to control the bias term B(m), as well as the term ξ(m). In the sequel, we shall cope with the following specific features of each scheme:

• For kernel rules, when f X is known, (2.5) is satisfied and these estimates lead to straightforward application of the GLM. But, when f X is unknown, serious difficulties will arise.

• For projection rules, the dependence on the knowledge of f X will be weaker but since (2.5) is not satisfied, the control of the bias term will not be straightforward.

Beyond these aspects, our main task in next sections will be to derive for each estimation scheme a function σ that conveniently controls the fluctuations of preliminary estimates as explained in Section 2.1.

Assumptions

In this section, we state our assumptions on f and f X .

(H 1 ) The conditional density f is uniformly bounded on

V n (x) × R d2 : f ∞ < ∞. (H 2 ) The density f X is uniformly bounded on V n (x): f X ∞ < ∞.
(H 3 ) The density f X is bounded away from 0 on V n (x): δ > 0. In the sequel, without loss of generality, we assume that δ ≤ 1.

Assumptions (H 1 ) and (H 2 ) are very mild. Note that under (H 1 ), since f is a conditional density, for any t ∈ V n (x), f (t, v)dv = 1 and sup t∈Vn(x)

f 2 t,2 ≤ sup t∈Vn(x),y∈R d 2 f (t, y) f (t, v)dv = f ∞ < ∞. (3.1)
Assumption (H 3 ) is not mild but is in some sense unavoidable. As said in Introduction, one goal of this paper is to measure the influence of the parameter δ on the performance of the estimators of f . For the procedures considered in this paper, if f X is unknown, we need a preliminary estimator of f X denoted fX that is constructed with observations (X i ) i=n+1,...,2n . Then, we first assume that fX satisfies the following condition:

δ := inf t∈Vn(x) | fX (t)| > 0. (3.2)
For estimating f X , fX has to be rather accurate:

∀ λ > 0, P sup t∈Vn(x) f X (t) -fX (t) fX (t) > λ ≤ κ exp{-(log n) 3/2 }, (3.3)
where κ is a constant only depending on λ and f X . Theorem 4 in [START_REF] Bertin | Adaptive pointwise estimation of conditional density function[END_REF] proves the existence of an estimate fX satisfying these properties.

Kernel rules

In this section, we study the data-driven kernel rules we propose for estimating the conditional density f . They are precisely defined in Section 4.2 and their theoretical performances in the oracle setting are studied in Section 4.3. Before doing this, in Section 4.1, we establish a lower bound of the risk for any kernel estimate.

Lower bound for kernel rules

In this section, we consider the kernel estimate fh defined in (2.8) for h ∈ H n . In particular, f X is assumed to be known. For any fixed h ∈ H n , we provide a lower bound of the risk of fh with q = 2 by using the following bias-variance decomposition:

R 2 x ( fh , 2) = E fh -f 2 x,2 = K h * f -f 2 x,2 + var( fh (x, y))dy. Proposition 1. Assume that (H 1 ) is satisfied. Then if K(x, y) = K (1) (x)K (2) (y) with K (1) sup- ported by [-A, A] d1 , for any h ∈ H n , we have for any n, R 2 x ( fh , 2) ≥ K h * f -f 2 x,2 + K (2) 2 2 n d i=1 h i × [K (1) (s)] 2 f X (x -(s 1 h 1 , . . . , s d1 h d1 )) ds + C 1 n ,
where C 1 depends on K 1 and f ∞ . If we further assume that f X is positive and continuous on a neighborhood of x, then if max

H n → 0 when n → +∞, R 2 x ( fh , 2) ≥ K h * f -f 2 x,2 + K 2 2 f X (x)n d i=1 h i × (1 + o(1)) + O 1 n , (4.1)
when n → +∞.

The proof of Proposition 1 is given in Section 8. The lower bounds of Proposition 1 can be viewed as benchmarks for our procedures. In particular, our challenge is to build a data-driven kernel procedure whose risk achieves the lower bound given in (4.1). It is the goal of the next section where we modify fh by estimating f X when f X is unknown.

Kernel estimator

Let us now define more precisely our kernel estimator. We consider the kernel K defined in Section 2.2, but following assumptions of Proposition 1, we further assume until the end of the paper that following conditions are satisfied.

• The kernel K is of the form K(u, v) = K (1) (u)K (2) (v), u ∈ R d1 , v ∈ R d2 . • The function K (1) is supported by [-A, A] d1 .
Our data-driven procedure is based on fX (see Section 3) and is defined in the following way. We naturally replace fh defined in (2.8) with

fh (x, y) = 1 n n i=1 1 fX (X i ) K h (x -X i , y -Y i ). (4.2) Then, we set σ(h) = χ δn d i=1 h i with χ = (1 + η)(1 + K 1 ) K 2 , (4.3)
where δ is defined in (3.2) and η > 0 is a tuning parameter. The choice of this parameter will be discussed in Section 7 but all theoretical results are true for any η > 0. We also specify the set H n :

(CK) For any h = (h 1 , . . . , h d ) ∈ H n , we have for any i, h -1 i is a positive integer and

k n ≤ 1 h i , ∀ i ∈ {1, . . . , d 1 }, 1 d1 i=1 h i ≤ δn (log n) 3 and log 2 (n) ≤ 1 d i=d1+1 h i ≤ n.
The GLM described in Section 2.2 can be applied and we estimate f with f = fĥ where ĥ = ĥ(x) := arg min

h∈Hn {A(h) + σ(h)} , A(h) := sup h ′ ∈Hn fh ′ -fh,h ′ x,2 -σ(h ′ ) + , and fh,h ′ (x, y) = 1 n n i=1 fX (X i ) -1 (K h * K h ′ )(x -X i , y -Y i ) = (K h ′ * fh )(x, y). (4.4)
In the case where f X is known, fX is replaced by f X and δ by δ. In particular, we obtain the expressions of Section 2.2 except that now σ is specified.

Oracle inequalities for kernel rules

We establish in this section oracle inequalities for our estimator f with in mind the benchmarks given in (4.1). To shed lights on the performance of our procedure and on the role of δ, we first deal with the case where f X is known. We first state a trajectorial oracle inequality and then a control of the risk.

Theorem 1. Assume that the density f X is known so that fX = f X . We also assume that (H 1 ), (H 3 ) and (CK) are satisfied. If δn ≥ 1, we have with probability larger than

1 -C exp{-(log n) 5/4 }, f -f x,2 ≤ inf h∈Hn    C 1 sup t∈Vn(x) K h * f -f t,2 + C 2 δn d i=1 h i    , (4.5)
where

C 1 = 1 + 2 K 1 , C 2 = (1 + η) K 2 (3 + 2 K 1 )
and C depends on K, η and f ∞ . Furthermore, for any q ≥ 1,

R x ( f , q) ≤ C1 inf h∈Hn    sup t∈Vn(x) K h * f -f t,2 + 1 δn d i=1 h i    + C2 √ n , (4.6)
where C1 depend on K, η and q and C2 depends on K, η, f ∞ and q.

Due to the assumptions on H n , the last term of the right hand side of (4.6), namely C2 / √ n, is negligible with respect to the first one. Furthermore, since σ 2 (h) is proportional to (δn n i=1 h i ) -1 , the latter can be viewed as a variance term (see Section 2.1). Then right hand sides of (4.5) and (4.6) correspond to the best tradeoff between a bias term and a variance term, so (4.5) and (4.6) correspond indeed to oracle inequalities. Next, we can compare the (squared) upper bound of (4.6) and the lower bound of (4.1) when q = 2 and f X is continuous. We note that these bounds match up to leading constants, asymptotically negligible terms and up to the fact that terms of (4.6) are computed on V n (x) instead at x (note that the size of V n (x) goes to 0 when n → +∞ and δ and f X (x) are close). Actually, since (2.4) is not valid for • = • x,2 , we use Inequality (2.11). This explains why we need to compute suprema of the bias term on V n (x). Theorem 1 shows the optimality of our kernel rule.

From these results, we can also draw interesting conclusions with respect to the term δ that appears in the variance term. From (4.1), we already know that the term δ is unavoidable. Of course, the lower δ the worse the performance of f . Actually, in the oracle context, our setting is (roughly speaking) equivalent to the classical setting where f X is lower bounded by an absolute constant (see [START_REF] Brunel | Adaptive estimation of the conditional density in the presence of censoring[END_REF] for instance), but with δn observations to estimate f instead of n. A similar remark will hold in the minimax framework of Section 6.

The following theorem deals with the general case where f X is unknown and estimated by fX .

Theorem 2. We assume that (H 1 ), (H 2 ), (H 3 ), (CK) (3.2) and (3.3) are satisfied. If δn ≥ 1, we have with probability larger than

1 -C exp{-(log n) 5/4 }, f -f x,2 ≤ inf h∈Hn    C 1 sup t∈Vn(x) K h * f -f t,2 + C 2 δn d i=1 h i    + C 3 δ sup t∈Vn(x) | fX (t) -f X (t)|, (4.7)
where

C 1 = 1 + 2 K 1 , C 2 = (1 + η) K 2 (3 + 2 K 1 ), C 3 depends on K, η and f ∞ and C depends on K, η, f X and f ∞ . Furthermore, for any q ≥ 1, R x ( f , q) ≤ C1 inf h∈Hn    sup t∈Vn(x) K h * f -f t,2 + 1 δn d i=1 h i    + C2 δ E 1 q sup t∈Vn(x) | fX (t) -f X (t)| q + C3 √ n ,
where C1 depend on K, η and q, C2 depends on K, η, q and f ∞ and C3 depends on K, η, f X , f ∞ and q.

The main difference between Theorems 2 and 1 lie in the terms involving sup t∈Vn(x) | fX (t)f X (t)| in right hand sides. Of course, if f X is regular enough, we can build fX so that this term is negligible. But in full generality, this unavoidable term due to the strong dependence of fh on fX , may be cumbersome. Therefore, even if Theorem 1 established the optimality of kernel rules in the case where f X is known, it seems reasonable to investigate other rules to overcome this problem.

Projection rules

Unlike previous kernel rules that strongly depend on the estimation of f X , this section presents estimates based on the least squares principle. The dependence on fX is only expressed via the use of δ and fX ∞ := sup t∈Vn(x) | fX (t)|. For ease of presentation, we assume that d 1 = d 2 = 1 but following results can be easily extended to the general case (see Section 6.3).

Models

As previously, we are interested in the estimation of f when the first variable is in the neighborhood of x, so we still use V n (x) defined in Section 1.3. We introduce a collection of models (S m ) m∈Mn .

Definition 1. Let M n be a finite subset of {0, 1, 2, . . .} 2 . For each m = (m 1 , m 2 ) ∈ M n and given two L 2 (R)-orthonormal systems of bounded functions (ϕ m j ) j∈Jm and (ψ m k ) k∈Km , we set

F m1 = Span(ϕ m j , j ∈ J m ), H m2 = Span(ψ m k , k ∈ K m )
and the model S m is

S m = F m1 ⊗ H m2 =    t, t(x, y) = j∈Jm k∈Km a m j,k ϕ m j (x)ψ m k (y), a m j,k ∈ R    .
Finally, we denote

D m1 = |J m | and D m2 = |K m |,
respectively the dimension of F m1 and H m2 .

In this paper, we only focus on systems (ϕ m j ) j∈Jm based on Legendre polynomials. More precisely, the estimation interval [x -2A, x + 2A] is split into 2 m1 intervals of length 4A2 -m1 :

I l = I m l = x -2A + 4A(l -1)2 -m1 , x -2A + 4Al2 -m1 l = 1, . . . , 2 m1 .
Then J m = {(l, d), l = 1, . . . , 2 m1 , d = 0, . . . , r}, D m1 = (r + 1)2 m1 and for any u,

ϕ m j (u) = ϕ m l,d (u) = 2 m1 2A 2d + 1 2 P d (T l (u))1 I l (u)
where P d is the Legendre polynomial with degree d on [-1, 1], and T l is the affine map which transforms

I l into [-1, 1].
In the y-direction, we shall also take piecewise polynomials. In the sequel, we only use the following two assumptions : for all m, m

′ ∈ M n , D m2 ≤ D m ′ 2 ⇒ H m2 ⊂ H m ′ 2 ,
and there exists a positive real number φ 2 such that for all m ∈ M n for all u ∈ R,

k∈Km (ψ m k ) 2 (u) ≤ φ 2 D m2 .
Note that this assumption is also true for F m1 . Indeed the spaces spanned by the ϕ m j 's are nested and, for all u ∈ [x -2A, x + 2A],

2 m 1 l=1 r d=0 ϕ m l,d (u) 2 ≤ 2 m1 2A r d=0 2d + 1 2 = 2 m1 4A (r + 1) 2 = r + 1 4A D m1
using properties of the Legendre polynomials. Therefore, with φ 1 = (r + 1)/(4A), for any u ∈

[x -2A, x + 2A], j (ϕ m j ) 2 (u) ≤ φ 1 D m1 .

Projection estimator

As in [START_REF] Brunel | Adaptive estimation of the conditional density in the presence of censoring[END_REF] and following Section 2.3, we introduce the following empirical contrast:

γ n (t) = 1 n n i=1 R t 2 (X i , y)dy -2t(X i , Y i ) .
We have the following lemma whose proof is easy by using straightforward computations. We use the norm • X associated with the dot product , X defined in (2.9), so we have for any t,

t 2 X = t 2 (u, y)f X (u)dudy.
Lemma 1. Assume that the function j∈Jm k∈Km âm j,k ϕ m j ψ m k minimizes the empirical contrast function γ n on S m , then Ĝm Âm = Ẑm , (5.1)

where Âm denotes the matrix with coefficients

(â m j,k ) j∈Jm,k∈Km , Ĝm = 1 n n i=1 ϕ m j1 (X i )ϕ m j2 (X i ) j1,j2∈Jm
and Ẑm = 1

n n i=1 ϕ m j (X i )ψ m k (Y i ) j∈Jm,k∈Km . Similarly, if K m (f ) is the orthogonal projection of f on (S m , , X ), it minimizes on S m t -→ γ(t) = t -f 2 X -f 2 X = E(γ n (t)) and if K m (f ) = j∈Jm k∈Km a m j,k ϕ m j ψ m k then, G m A m = Z m ,
where A m denotes the matrix with coefficients

(a m j,k ) j∈Jm,k∈Km , G m = E( Ĝm ) = ϕ m j1 , ϕ m j2 X j1,j2∈Jm
and

Z m = E( Ẑm ) = ϕ m j (u)ψ m k (y)f (u, y)f X (u)dudy j∈Jm,k∈Km
.

From this lemma, we obtain that E(γ n (t)) is minimum when t = f , which justifies the use of γ n .

Then, to derive fm an estimate of f , we use (5.1) as a natural consequence of the minimization problem (2.10). But if Ĝm is not invertible, Âm can be not uniquely defined.

Since x is fixed, we can define, for each m = (m 1 , m 2 ), the index l m1 = l m1 (x) such that x belongs to I lm 1 (actually, since the estimation interval is centered in x, l m1 = 2 m1-1 + 1). Furthermore, since we use a piecewise polynomial system, the Gram matrix Ĝm is a block diagonal matrix with blocks Ĝ

(1) m , . . . , Ĝ(2 m 1 ) m , where Ĝ(l) m = 1 n n i=1 ϕ m l,d1 (X i )ϕ m l,d2 (X i ) 0≤d1,d2≤r
.

In the same way, we can define for l = 1, . . . ,

2 m1 Ẑ(l) m = 1 n n i=1 ϕ m l,d (X i )ψ m k (Y i ) 0≤d≤r,k∈Km
. Now, and by naturally using the blockwise representation of Ĝm , we define the collection of estimators ( fm ) m∈Mn as:

fm (x, y) = r d=0 k∈Km âm (lm 1 ,d),k ϕ m lm 1 ,d (x)ψ m k (y)
and

(â m (lm 1 ,d),k ) 0≤d≤r,k∈Km := Â(lm 1 ) m := ( Ĝ(lm 1 ) m ) -1 Ẑ(lm 1 ) m if min(Sp( Ĝ(lm 1 ) m )) > (1 + η) -2/5 δ 0 otherwise,
where η is a positive real number. Here, for a symmetric matrix M , Sp(M ) denotes the spectrum of M , i.e. the set of its eigenvalues. This expression allows us to overcome problems if Ĝm is not invertible. Note that, when r = 0, where r is maximal degree of Legendre polynomials, this estimator can be written

fm (x, y) = j∈Jm k∈Km n i=1 ϕ m j (X i )ψ m k (Y i ) n i=1 ϕ m j (X i ) 2 ϕ m j (x)ψ m k (y).
Now, to choose a final estimator among this collection, as explained in Section 2.3, we denote

m ∧ j = (m 1 ∧ j 1 , m 2 ∧ j 2 ) = (min(m 1 , j 1 ), min(m 2 , j 2 ))
and by using fX introduced in Section 3, we set

σ(m) = χ D m1 D m2 δn with χ2 = (1 + η) 2 (4φ 1 φ 2 (r + 1)) f X ∞ δ , (5.2)
where f X ∞ = fX ∞ and δ is defined in (3.2). We also specify the models we use: The following condition is the analog of (CK):

(CM ) For any m ∈ M n ,

k n (r + 1) ≤ D m1 ≤ δn (log n) 3 and log 2 (n) ≤ D m2 ≤ n.
The GLM described in Section 2.2 can be applied and we estimate f with f = f m where

m = m(x) := arg min m∈Mn {A(m) + σ(m)} and A(m) := sup m ′ ∈Mn fm ′ -fm ′ ∧m x,2 -σ(m ′ ) + .
The next section studies the performance of the estimate f .

Oracle inequality for projection estimators

We establish in this section oracle inequalities for the projection estimate in the same spirit as for the kernel rule. We recall that K m (f ) is the orthogonal projection of f on (S m , <, > X ) where <, > X is the dot product defined in (2.9). The following result is the analog of Theorem 2.

Theorem 3. We assume that (H 1 ), (H 2 ), (H 3 ), (CM ) (3.2) and (3.3) are satisfied. If δn ≥ 1, we have with probability larger than 1 -C exp{-(log n) 5/4 },

f -f x,2 ≤ inf m∈Mn C 1 sup t∈Vn(x) K m (f ) -f t,2 + 5 2 χ D m1 D m2 δn with χ defined in (5.2), C 1 = 1 + 2(r + 1)δ -1 f X ∞ and C depends on φ 1 , φ 2 , r, η, f ∞ and f X . Furthermore, for any q ≥ 1 R x ( f , q) ≤ C1 inf m∈Mn sup t∈Vn(x) K m (f ) -f t,2 + D m1 D m2 δn + C2 √ n
where C1 depends on φ 1 , φ 2 , r, η, f X ∞ , δ and q and C2 depends on φ 1 , φ 2 , r, η, f ∞ , f X and q.

As for Theorem 2, using the definition of σ, the right hand sides correspond to the best tradeoff between a bias term and a variance term. Note that unlike kernel rules, the performances of f do not depend on the rate of convergence of fX for estimating f X . But there is a price to pay: due to a rougher control of the bias term, χ depends on δ and the leading constants C 1 and C1 depend on δ. In particular, when f X is known, conclusions drawn from Theorem 1 do not hold here. However, in the case where r = 0 (the basis in the first coordinate is simply the histogram basis), we can use the simpler penalty term χ = (1 + η) √ 4φ 1 φ 2 and the previous result still holds. To prove this, it is sufficient to use the basis ( ϕ j -1 X ϕ j ⊗ ψ k ) j,k which is orthonormal for the scalar product ., . X .

Rates of convergence

In this section, minimax rates of convergence will be computed on Hölder balls H d (α, L). We recall that for two d-tuples of positive reals α = (α 1 , . . . , α d ) and L = (L 1 , . . . , L d ),

H d (α, L) = f : R d → R s.t. ∀ 1 ≤ i ≤ d ∂ m f ∂x m i ∞ ≤ L i , m = 0, . . . , ⌊α i ⌋ and for all t ∈ R ∂ ⌊αi⌋ f ∂x ⌊αi⌋ i (• + te i ) - ∂ ⌊αi⌋ f ∂x ⌊αi⌋ i (•) ∞ ≤ L i |t| αi-⌊αi⌋
where for any i, ⌊α i ⌋ = max{l ∈ N : l < α i } and e i is the vector where all coordinates are null except the ith one which is equal to 1. In the sequel, we use the classical anisotropic smoothness index defined by

ᾱ = d i=1 1 α i -1
and introduced in the seminal paper [START_REF] Kerkyacharian | Nonlinear estimation in anisotropic multiindex denoising[END_REF]. See also [START_REF] Goldenshluger | Universal pointwise selection rule in multivariate function estimation[END_REF].

Lower bound

We have the following result that holds without making any assumption. It is proved in Section 8.3 of [START_REF] Bertin | Adaptive pointwise estimation of conditional density function[END_REF].

Theorem 4. There exists a positive constant C not depending on L nor n such that, if n is large enough,

inf Tn sup (f,fX )∈ H(α,L) (f X (x)) 2 ᾱ 2 ᾱ+1 E f -T n 2 x,2 ≥ C d i=1 L 1 α i i 2 ᾱ 2 ᾱ+1 n -2 ᾱ 2 ᾱ+1 ,
where the infimum is taken over all estimators T n of f based on the observations (X i , Y i ) i=1,...,n and H(α, L) is the set such that the conditional density f belongs to H d (α, L) and the marginal density f X is continuous.

Note that we consider the ball H(α, L) which may be (slightly) smaller than the ball H(α, L). Actually, we wish to point out the dependence of the lower bound with respect to n, α and L as usual but also to f X (x), which is less classical. The goal in next sections is to show that our procedures achieve the lower bound of Theorem 4.

Upper Bound for kernel rules

In this section, we need an additional assumption on f . (H 4 ) There exists a compact set B, such that for all t ∈ V n (x), the function y → f (t, y) has a support included into B. We denote by |B| the length of the compact set B.

This assumption could be avoided at the price of studying the risk restricted on B. Moreover, to study the bias of the kernel estimator, we consider for any M = (M 1 , . . . , M d ) the following condition.

(BK M ) For any i ∈ {1, . . . , d}, for any 1 ≤ j ≤ M i , we have

R |x i | j |K(x)|dx i < ∞ and R x j i K(x)dx i = 0.
We refer the reader to [START_REF] Kerkyacharian | Nonlinear estimation in anisotropic multiindex denoising[END_REF] for the construction of a kernel K satisfying (BK M ) and previous required conditions. We obtain the following result showing the optimality of our first procedure from the minimax point of view, up to the rate for estimating f X .

Theorem 5. We assume that (H 1 ), (H 2 ), (H 3 ), (H 4 ), (CK), (3.2) and (3.3) are satisfied. Let

M = (M 1 , . . . , M d ) such that (BK M ) is satisfied. Then if f belongs to H d (α, L) such that ⌊α i ⌋ ≤ M i
for all i = 1, . . . , d, the kernel rule f satisfies for any q ≥ 1,

R q x ( f , q) ≤ C1 d i=1 L 1 α i i q ᾱ 2 ᾱ+1 (nδ) -qα 2α+1 + C2 δ q E sup t∈Vn(x) | fX (t) -f X (t)| q + C3 n -q 2 ,
where C1 depend on K, η and q, C2 depends on K, η, q and f ∞ and C3 depends on K, η, f X , f ∞ and q.

If the leading term in the last expression is the first one, then, up to some constants, the upper bound of Theorem 5 matches with the lower bound obtained in Theorem 4 (note that δ is close to f X (x)) when q = 2. In this case, our estimate is adaptive minimax. To study the second term, we can use Theorem 4 of [START_REF] Bertin | Adaptive pointwise estimation of conditional density function[END_REF] that proves that, in our setting, there exists an estimate fX achieving the rate

(log /n) β 2 β+1 if f X ∈ H d1 (β, L)
and we obtain the following corollary.

Corollary 1. We assume that (H 1 ), (H 2 ), (H 3 ), (H 4 ), (CK) and (BK M ) are satisfied. We also assume that f X ∈ H d1 (β, L) such that for any i = 1, . . . , d 1 , Li > 0 and 0

< β i ≤ β (m) i with some known β (m) i > 0. Then if f belongs to H d (α, L) such that ⌊α i ⌋ ≤ M i for all i = 1, . . . , d, the kernel rule f satisfies for any q ≥ 1, R q x ( f , q) ≤ C 1   d i=1 L 1 α i i q ᾱ 2 ᾱ+1 (nδ) -qα 2α+1 + 1 δ q log n n q β 2 β+1   + C 2 n -q 2 ,
where C 1 is a constant not depending on L, n and δ and C 2 is a constant not depending on L and n.

From the corollary, we deduce that if β > ᾱ and if δ is viewed as a constant, then the leading term is the first one. Furthermore, in this case, the rate is polynomial and the rate exponent is the classical ratio associated with anisotropic Hölder balls: ᾱ/(2 ᾱ + 1). Our result also explicits the dependence of the rate with respect to L and δ.

Upper bound for projection estimates

In the same way, we can control the bias for our second procedure of estimation in order to study the rate of convergence. Let us briefly explain how the procedure defined in Section 5 can be extended to the estimation of conditional anisotropic densities f : R d1 × R d2 → R with d 1 , d 2 ≥ 2. The contrast is still the same and the estimators fm have to be defined for m = (m 1 , . . . , m d ) with a polynomial basis on hyperrectangles : see [START_REF] Akakpo | Inhomogeneous and anisotropic conditional density estimation from dependent data[END_REF] for a precise definition. The model dimension is now

D m1 = d1 i=1 r i 2 mi
where r 1 , . . . , r d1 are the maximum degrees. Then, the selection rule to define f is unchanged, except that in (5.2)

χ2 = (1 + η) 2 4φ 1 φ 2 d1 i=1 r i f X ∞ δ
In order to control precisely the bias, we introduce the following condition.

(BM r ) H m2 is a space of piecewise polynomials with degrees bounded by r d1+1 , . . . , r d , with D m2 = d i=d1+1 r i 2 mi . This allows us to state the following result. Theorem 6. We assume that (H 1 ), (H 2 ), (H 3 ), (H 4 ), (CM ), (3.2) and (3.3) are satisfied. Let r = (r 1 , . . . , r d ) such that (BM r ) is satisfied. Then if f belongs to H d (α, L) such that α i < r i for all i = 1, . . . , d, the projection rule f satisfies for any q ≥ 1,

R q x ( f , q) ≤ C d i=1 L 1 α i i q ᾱ 2 ᾱ+1 n -qα 2α+1 ,
where C depend on A, |B|, r, α, δ and f X ∞ .

Thus, even if the control of δ is less accurate, the projection estimator achieves the optimal rate of convergence whatever the regularity of f X .

Simulations

In this section we focus on the numerical performances of our estimators. We first describe the algorithms. Then, we introduce the studied examples and we illustrate the performances of our procedures with some figures and tables.

Estimation algorithms

For both methods (kernel or projection), we need a preliminary estimator of f X . In order to obtain an accurate estimator of f X , we use a pointwise Goldenshluger Lepski procedure which consists in the following for estimating f X at x. This preliminary estimator is constructed using the sample (X i ) i=n+1,...,2n . Let us define for h > 0,

pen(n, h) = 2.2 K 2 (1 + K 1 ) | log h| fX (x) nh , (7.1)
where fX is a preliminary estimator of f X obtained by the rule of thumb (see [START_REF] Silverman | Density estimation for statistics and data analysis[END_REF]), and K is the classical Gaussian kernel. The value 2.2 is the adjusted tuning constant which was convenient on a set of preliminary simulations. Given H a finite set of bandwidths (actually H is a set of 10 bandwidths centered at the bandwidth obtained by the rule of thumb) and for h, h ′ ∈ H, consider

fh (x) = 1 n 2n i=n+1 K h (x -X i ) and fh,h ′ (x) = 1 n 2n i=n+1 (K h * K h ′ )(x -X i ).
We consider A(h, x) := max

h ′ ∈H fh,h ′ (x) -fh ′ (x) -pen(n, h ′ ) + .
Finally we define h 0 by h 0 := arg min h∈H {A(h, x) + pen(n, h)} (7.2) and we consider the following procedure of estimation: fX (x) = fh0 (x). Now, the algorithm for the kernel estimation of f is entirely described in Section 4.2 and we perform it with K the Gaussian kernel and a set of 10 bandwidths in each direction, that means that the size of H n is 10 d1+d2 . The quantity fh ′ -fh,h ′ x,2 is made easy to compute with some preliminary theoretical computations (in particular, note that for the Gaussian kernel

K h * K ′ h = K h ′′ with h ′′2 = h 2 + h ′2
). The only remaining parameter to tune is η which appears in the penalty term σ (see (4.3)).

In the same way, we follow Section 5.2 to implement the projection estimator. Matrix computations are easy to implement and make the implementation very fast. We only present the case of polynomials with degrees r = s = 0, i.e. histograms, since the performance is already good in this case. Again, the only remaining parameter to tune is η which appears in the penalty term σ (see (5.2)). Note that in the programs, it is possible to use non-integers m i and in fact this improves the performance of the estimation. However, to match with the theory we shall not tackle this issue.

Simulation study and analysis

We apply our procedures to different examples of conditional density functions with d 1 = d 2 = 1. More precisely, we observe (X i , Y i ) i=1,...,n such that Example 1 The X i 's are iid uniform variables on [0, 1] and

Y i = 2X 2 i + 5 + ε i (1.3 -|X i |) 1/2 , i = 1, . . . , n,
where the ε i 's are i.i.d. reduced and centered Gaussian variables, independent of the X i 's. Note that we also studied heavy-tailed noises in this example (i.e. the ε i 's are variables with a standard Cauchy distribution) and the results were almost identical.

Example 2 The X i 's are iid uniform variables on [0, 1] and the distribution of the Y i 's is a mixture of a normal distribution and an exponential distribution: Y i ∼ 0.75ε i + 0.25(2 + E i ), where ε i is a zero-mean normal distribution with standard deviation 2 + X i and E i is exponential with parameter 2.

Example 3 The X i 's are iid and their common distribution is a mixture of two normal distributions, 0.5N (0, 1/81) + 0.5N (1, 1/16) and

Y i = X 2 i + 1 + ε i (1.3 + |X i |) 1/2 , i = 1, . . . , n,
where the ε i 's are i.i.d. reduced and centered Gaussian variables, independent of the X i 's.

Example 4 The X i 's are iid and their common distribution is a mixture of two normal distributions, 0.5N (0, 1/81) + 0.5N (1, 1/16) and the distribution of the Y i 's is a mixture of a normal distribution and an exponential distribution: Y i ∼ 0.75ε i + 0.25(2 + E i ), where ε i is a zero-mean normal distribution with standard deviation 2 + X i and E i is exponential with parameter 2.

We simulate our observations for three sample sizes: n = 250, n = 500 and n = 1000. In Figure 1, we illustrate the quality of reconstructions for both estimates when f X is unknown. We use η = -0.2 for the projection estimator and η = 1 for the kernel estimator (see the discussion below). To go further, for each sample size, we evaluate the mean squared error of the estimators, in other words

MSE( f ) = f (x, y) -f (x, y) 2 dy,
where f is either the kernel rule or the projection estimate. In Appendix B, we give approximations of the MSE based on N = 100 samples for different values of η. Now, let us comment our results from the point of view of tuning, namely we try to answer the question: how to choose the parameter η? We first focus on kernel rules. Tables of Appendix B show that, often, the optimal value is η = 1. More precisely, it is always the case for Examples 1 and 2. For Examples 3 and 4, when η = 1 is not the optimal value, taking η = 1 does not deteriorate the risk too much. So, for kernel rules, the choice η = 1 is recommended even if larger values can be convenient in some situations. To shed more lights on these numerical results, in Figure 2, we draw the MSE for the kernel rule in function of the parameter η. We observe that the shape of the curve is the same whatever the example. If η is too small the risk blows up, which shows that the assumption η > 0 in theoretical results is unavoidable at least asymptotically. Furthermore, we observe that if η is too large, then the estimate oversmooths and the risk increases but without explosion for η not too far from the minimizer. Similar phenomena have already been observed for wavelet thresholding rules for density estimation (see Section 2.2 of [START_REF] Reynaud-Bouret | Adaptive density estimation: a curse of support?[END_REF]. Tuning kernel rules is then achieved. We now deal with projection rules. Unfortunately, the plateau phenomenon of Figure 2 does not happen for projection estimators. In this case, the optimal value for η seems to change according to the example. Tuning this procedure is not so obvious. Note that performances of kernel and projections rules are hardly comparable since they are respectively based on a Gaussian kernel function and piecewise constant functions.

For kernel rules, we study the influence of the knowledge of f X . Tables 1 and3 show that when f X is known results are a bit better as expected, but the difference is not very significant. Since projections rules are less sensitive to the estimate fX , we only show results with f X unknown. Finally, to study the dependence of estimation with respect to x, we focus on Tables 5 and6 that show that in Example 3 estimation is better at x = 0 and x = 1 than at x = 0.36. This was expected since the density design is smaller at x = 0.36 and this confirms the role of δ in the rate of convergence of both estimators (see Theorems 2 and 3). Similar conclusions can be drawn for Example 4. Finally, we wish to mention that the ratio between the risk of our procedures and the oracle risk (the upper bounds of Theorems 1, 2 and 3) remains bounded with respect to n, which corroborates our theoretical results.

Proofs

In this section, after giving intermediate technical results, we prove the results of our paper. Most of the time, as explained in introduction, we only consider the case d 1 = d 2 = 1. We use notations that we have previously defined. The classical Euclidian norm is denoted • . Except if the context is ambiguous, from now on, the • ∞ -norm shall denote the supremum either on R, on V n (x) or on V n (x) × R. We shall also use for any function g g ∞,2 := sup t∈Vn(x) g t,2 . This section is divided into two parts: Section 8.1 (respectively Section 8.2) is devoted to the proofs of the results for the kernel rules (respectively for the projection rules). We first prove in Section 8.1.1 the lower bound stated in Proposition 1. Main results for kernel rules, namely Theorems 1 and 2 are proved in Section 8.1.2. They depend on several intermediate results that are proved in Sections 8.1.3-8.1.6 (see the sketch of proofs in Section 8.1.2). Theorem 5 that derives rates for kernel rules is proved in Section 8.1.7. For projection rules, the main theorem, namely Theorem 3, is proved in Section 8.2.1. It is based on intermediate results shown in Sections 8.2.2-8.2.4. Finally, Theorem 6 that derives rates for projection rules is proved in Section 8.2.5. As usual in nonparametric statistics, our results are based on sharp concentration inequalities that are stated in Lemmas 2, 3 and 4. These lemmas and other technical results stated in Lemmas 5 and 6 are proved in Appendix A.

Lemma 2. [Bernstein Inequality] Let (U i ) be a sequence of i.i.d. variables uniformly bounded by a positive constant c and such that EU 2 1 ≤ v. Then

P 1 n n i=1 U i -E[U i ] ≥ ε ≤ 2 exp -min nε 2 4v , nε 4c
Note that Lemma 2 is a simple consequence of [START_REF] Birgé | Minimum contrast estimators on sieves: exponential bounds and rates of convergence[END_REF], p.366.

Lemma 3. [Talagrand Inequality] Let U 1 , . . . , U n be i.i.d. random variables and ν n (a) = 1 n n i=1 [τ a (U i )- E(τ a (U i ))
] for a belonging to A a countable subset of functions. For any ζ > 0, Let ρ > 1 and consider the event

P(sup a∈A |ν n (a)| ≥ (1 + 2ζ)H) ≤ 2 max exp - ζ 2 6 nH 2 v , exp - min(ζ,
Λ ρ = {ρ -1 δ ≤ δ ≤ ρδ} ∩ {ρ -2 f X ∞ ≤ fX ∞ ≤ ρ 2 f X ∞ }.
We have the following lemma.

Lemma 4. Condition (3.3) implies that

P(Λ c ρ ) ≤ B 1 e -(log n) 3/2
with some positive constant B 1 that depends on f X and ρ.

Lemma 5. For any integrable functions

f 1 and f 2 , if the support of u → f 2 (u, y) is included in [-2A/k n , 2A/k n ] d1
for all y, then we have

f 1 * f 2 x,2 ≤ sup t∈Vn(x) f 1 t,2 × f 2 1 ,
Lemma 6. We use notations of Definition 1. Let m = (m 1 , m 2 ) be fixed. For any function τ , the projection K m (τ ) of τ on S m verifies

K m (τ ) x,2 ≤ (r + 1) f X ∞ δ -1 sup t∈Vn(x) τ t,2 .
8.1 Proofs for the kernel estimator

Proof of Proposition 1

We just need to control:

var( fh (x, y))dy = 1 n var [f X (X 1 )] -1 K h (x -X 1 , y -Y 1 ) dy = 1 n E [f X (X 1 )] -2 K 2 h (x -X 1 , y -Y 1 ) -E [f X (X 1 )] -1 K h (x -X 1 , y -Y 1 ) 2 dy.
First, by using Lemma 5 and (3.1),

E [f X (X 1 )] -1 K h (x -X 1 , y -Y 1 ) 2 dy = (K h * f ) 2 (x, y)dy ≤ K h 2 1 × sup t∈Vn(x) f 2 t,2 ≤ K 2 1 f ∞ .
Furthermore,

E [f X (X 1 )] -2 K 2 h (x -X 1 , y -Y 1 ) dy = K 2 h (x -u, y -v)f (u, v)[f X (u)] -1 dudvdy = (K (1) h1 ) 2 (x -u)[f X (u)] -1 du × K (2) 2 2 h 2 = K (2) 2 2 h 1 h 2 × [K (1) (s)] 2 f X (x -sh 1 )
ds.

Now assume that f X is positive and continuous on a neighborhood of x. Since max H n → 0 when n → +∞, then h 1 → 0. Then we have

f X (x)[K (1) (s)] 2 f X (x -sh 1 ) ds -K (1) 2 2 ≤ [K (1) (s)] 2 f X (x) f X (x -sh 1 ) -1 ds ≤ max |v|≤Ah1 f X (x) f X (x + v) -1 [K (1) (s)] 2 ds = o(1).

Proof of Theorems 1 and 2

We introduce

g(x, y) = f X,Y (x, y) fX (x) = f X (x) fX (x) f (x, y).
We consider the set Γ = Γ 1 ∩ Γ 2 where

Γ 1 =    ∀h, h ′ ∈ H n : K h * fh ′ -K h * K h ′ * g x,2 ≤ χ 1 δnh ′ 1 h ′ 2    , Γ 2 =    ∀h ′ ∈ H n : fh ′ -K h ′ * g x,2 ≤ χ 2 δnh ′ 1 h ′ 2    and χ 1 = (1 + η) K 1 K 2 , χ 2 = (1 + η) K 2 .
We shall use following propositions that deal with the general case when f X is estimated by fX . When f X is known, it can easily be checked that these propositions also hold with g replaced by f and δ by δ. We also use the set Λ ρ studied in Lemma 4 with ρ = (1 + η/2) 2 .

Let us give a sketch of the proof. The main steps for proving Theorems 1 and 2 are the following. We first prove an oracle inequality for the function g on the set Γ (Proposition 2). Then, in Proposition 3, we prove that the event Γ occurs with large probability by using Lemma 3. Finally, Proposition 4 studies the impact of replacing g by f . Proposition 5 gives a polynomial control in n of our estimate that is enough to control its risk on Γ c by using Proposition 3 and Lemma 4.

Proposition 2. On the set Γ, we have the following result.

f -g x,2 ≤ inf h∈Hn C 1 K h * g -g ∞,2 + C 2 1 δnh 1 h 2 ,
where

C 1 = 1 + 2 K 1 and C 2 = (1 + η) K 2 (3 + 2 K 1 ).
Proposition 3. Under (H 1 ), (H 3 ) and (CK), we have:

P (Γ c ∩ Λ ρ ) ≤ C exp{-(log n) 5/4 }
where C depends on K, η and f ∞ .

Proposition 4. Assume that (H 1 ), (H 2 ) and (CK) are satisfied. On Λ ρ :

K h * g -g ∞,2 ≤ K h * f -f ∞,2 + Cδ -1 fX -f X ∞ , g -f x,2 ≤Cδ -1 fX -f X ∞ ,
where C depends on η, K, and f ∞ .

Proposition 5. Assume that (CK) is satisfied. For any h ∈ H n , fh x,2 ≤ K (1) ∞ K (2) 2 (log n) -3 n 3/2 .
The first part of Theorem 1 can be deduced from Propositions 2 and 3. Note that in the case of Theorem 1, since f X is known, g = f and P(Λ ρ ) = 1. The second part of Theorem 1 is a consequence of Proposition 5, (3.1) and (4.5). Since

f -f x,2 ≤ f -g x,2 + g -f x,2 and Γ ∩ Λ ρ = (Γ ∪ Λ c ρ ) ∩ Λ ρ
, the first part of Theorem 2 is a consequence of Propositions 2, 3 and 4 combined with Lemma 4. The second part of Theorem 2 is a consequence of Proposition 5, (3.1) and (4.7).

Proof of Proposition 2

We apply the GLM as explained in Section 2 with fh given in (4.2) for estimating g, M n = H n , . = . x,2 , σ(h) = χ/ δnh 1 h 2 , and the operator K h is the convolution product with K h . Note that (2.1), (2.2) and (2.5) are satisfied but not (2.4). But we have:

B(h) = sup h ′ ∈Hn K h ′ (g) -(K h ′ • K h )(g) x,2 ≤ K 1 sup t∈Vn(x) g -K h (g) t,2 ,
using Lemma 5 and the equality K h ′ 1 = K 1 . Let us fix h ∈ H n . We obtain Inequality (2.3) in our case:

f -g x,2 ≤ 2B(h) + 2σ(h) + fh -K h (g) x,2 + g -K h (g) x,2 + 2ξ(h) with ξ(h) = sup h ′ ∈Hn ( fh ′ -K h ′ (g)) -(K h ′ ( fh ) -(K h ′ • K h )(g)) x,2 -σ(h ′ ) + . But, on Γ, ∀h, h ′ ∈ H n , fh ′ -K h ′ (g) x,2 ≤ χ 2 / δnh ′ 1 h ′ 2 and K h ′ ( fh ) -(K h ′ • K h )(g) x,2 ≤ χ 1 / δnh ′ 1 h ′ 2 , so that ξ(h) = 0. Then, on Γ, f -g x,2 ≤ 2B(h) + 2σ(h) + χ 2 δnh 1 h 2 + g -K h (g) x,2 ≤ (2 K 1 + 1) sup t∈Vn(x) g -K h (g) t,2 + 2χ + χ 2 δnh 1 h 2 with 2χ + χ 2 = 2χ 1 + 3χ 2 = (1 + η)(2 K 1 + 3) K 2 .

Proof of Proposition 3

We respectively denote P and Ẽ the probability distribution and the expectation associated with

(X 1 , Y 1 ), . . . , (X n , Y n ). Thus Γ 1 =    ∀h, h ′ ∈ H n : fh,h ′ -Ẽ fh,h ′ x,2 ≤ χ 1 δnh ′ 1 h ′ 2    , Γ 2 =    ∀h ′ ∈ H n : fh ′ -Ẽ fh ′ x,2 ≤ χ 2 δnh ′ 1 h ′ 2    .
To prove Proposition 3, we study Γ c 1 ∩ Λ ρ and Γ c 2 ∩ Λ ρ . So first, let assume we are on the event Λ ρ . Note that on Λ ρ , we have δ-1 ≤ ρδ -1 and for all u ∈ V n (x), |g(u, v)| ≤ f (u, v)ρ (see the proof of Lemma 4). We denote for any x, y, u and v,

w(x, y, u, v) = [ fX (u)] -1 (K h * K h ′ )(x -u, y -v).
We can then write:

fh,h ′ (x, y) = 1 n n i=1 w(x, y, X i , Y i )
and with B the unit ball in L 2 (R) endowed with the classical norm and A a dense countable subset of B,

fh,h ′ -Ẽ fh,h ′ x,2 = sup a∈B a(y) fh,h ′ (x, y) -Ẽ[ fh,h ′ (x, y)] dy = sup a∈A a(y) fh,h ′ (x, y) -Ẽ[ fh,h ′ (x, y)] dy = sup a∈A 1 n n i=1 a(y) w(x, y, X i , Y i ) -Ẽ(w(x, y, X i , Y i )) dy.
Hence, one will apply the inequality of Lemma 3 with τ a,x (X i , Y i ) = a(y)w(x, y, X i , Y i )dy. First, we have:

Ẽ fh,h ′ -Ẽ fh,h ′ x,2 2 ≤ Ẽ fh,h ′ -Ẽ fh,h ′ 2 x,2 = Ẽ fh,h ′ (x, y) -Ẽ[ fh,h ′ (x, y)] 2 dy = var( fh,h ′ (x, y))dy = 1 n var [ fX (X 1 )] -1 (K h * K h ′ )(x -X 1 , y -Y 1 ) dy ≤ 1 n Ẽ [ fX (X 1 )] -2 (K h * K h ′ ) 2 (x -X 1 , y -Y 1 ) dy ≤ 1 δn (K h * K h ′ ) 2 (x -u, y -v)g(u, v)dudvdy.
But we have

(K h * K h ′ ) 2 (x -u, y -v) = K h ′ (x -u -s, y -v -t)K h (s, t)dsdt 2 ≤ K 2 h ′ (x -u -s, y -v -t)|K h (s, t)|dsdt × K 1 .
Therefore, since for any u, f (u, v)dv = 1 and K(x, y

) = K (1) (x)K (2) (y), Ẽ fh,h ′ -Ẽ fh,h ′ x,2 2 ≤ K 1 δn |K h (s, t)| K 2 h ′ (x -u -s, y -v -t)g(u, v)dudvdy dsdt = K 1 δn |K h (s, t)| (K (1) h ′ 1 ) 2 (x -u -s)(K (2) h ′ 2 ) 2 (y -v -t)dy g(u, v)dv du dsdt ≤ K 1 K (2) 2 2 ρ δnh ′ 2 |K h (s, t)| (K (1) h ′ 1 ) 2 (x -u -s)du dsdt = K 2 1 K (1) 2 2 K (2) 2 2 ρ δnh ′ 1 h ′ 2 = K 2 1 K 2 2 ρ δnh ′ 1 h ′ 2 . Consequently, we obtain Ẽ fh,h ′ -Ẽ fh,h ′ x,2 ≤ H, with H = K 1 K 2 ρ 1/2 δnh ′ 1 h ′ 2 . (8.1)
Now, let us deal with v which is an upper bound of sup a∈A var (τ a,x (X 1 , Y 1 )).

sup a∈A var (τ a,x (X 1 , Y 1 )) ≤ sup a∈A Ẽ a(y)w(x, y, X 1 , Y 1 )dy 2 ≤ sup a∈A Ẽ |w(x, y, X 1 , Y 1 )|dy a 2 (y)|w(x, y, X 1 , Y 1 )|dy ≤ sup u,v |w(x, y, u, v)|dy sup y Ẽ[|w(x, y, X 1 , Y 1 )|]. Now, sup u,v |w(x, y, u, v)|dy = sup u,v [ fX (u)] -1 (K h * K h ′ )(x -u, y -v) dy ≤ 1 δ sup u,v K (1) h ′ 1 (x -u -s)K (2) h ′ 2 (y -v -t)K h (s, t)dsdt dy ≤ 1 δ sup u,v |K h (s, t)| |K (1) h ′ 1 (x -u -s)||K (2) h ′ 2 (y -v -t)|dy dsdt ≤ K 1 K (2) 1 K (1) ∞ δh ′ 1 and sup y Ẽ[|w(x, y, X 1 , Y 1 )|] = sup y |w(x, y, u, v)|f X,Y (u, v)dudv = sup y |(K h * K h ′ )(x -u, y -v)|g(u, v)dudv ≤ g ∞ sup y |K h (x -u -s, y -v -t)||K h ′ (s, t)|dsdt dudv ≤ g ∞ K 2 1 ≤ f ∞ ρ K 2 1 since on Λ ρ , g ∞ ≤ ρ f ∞ and where g ∞ = sup (t,v)∈Vn(x)×R |g(t, v)|. Thus, we set v = K 3 1 K (2) 1 K (1) ∞ ρ f ∞ δh ′ 1 . (8.2)
Finally, we deal with M which has to be an upper bound of sup a∈A sup u sup v a(y)w(x, y, u, v)dy

sup a∈A sup u sup v a(y)w(x, y, u, v)dy = sup u,v w(x, ., u, v) 2 ≤ 1 δ sup u,v (K h * K h ′ ) 2 (x -u, y -v)dy 1/2 .
We have:

(K h * K h ′ ) 2 (x -u, y -v)dy = K h ′ (x -u -s, y -v -t)K h (s, t)dsdt 2 dy ≤ K 1 |K h (s, t)| K 2 h ′ (x -u -s, y -v -t)dy dsdt ≤ K 2 1 K (1) 2 ∞ K (2) 2 2 h ′2 1 h ′ 2 .
Therefore, we can set

M = K 1 K (1) ∞ K (2) 2 δh ′ 1 h ′ 2 . (8.3) So, since ρ = (1 + η/2) 2 , Lemma 3 implies that for any ζ > 0, P   fh,h ′ -Ẽ fh,h ′ x,2 ≥ (1 + 2ζ) K 1 K 2 (1 + η/2) δnh ′ 1 h ′ 2   ≤ 2 max exp - ζ 2 C 1 (K, f ∞ ) h ′ 2 , exp -ζ min(1, ζ)C 2 (K, η) nh ′ 1 δ ,
where C 1 (K, f ∞ ) and C 2 (K, η) are positive constants that depend on K and f ∞ and K and η respectively. Similarly we have for any ζ > 0,

P   fh ′ -Ẽ fh ′ x,2 ≥ (1 + 2ζ) K 2 (1 + η/2) δnh ′ 1 h ′ 2   ≤ 2 max exp - ζ 2 C 3 (K, η, f ∞ ) h ′ 2 , exp -ζ min(1, ζ)C 4 (K, η) nh ′ 1 δ ,
where C 3 (K, η, f ∞ ) and C 4 (K, η) are positive constants that depend on K, η and f ∞ and K and η respectively. Let ζ = η/(4 + 2η) so that (8.4) with C 5 (K, η, f ∞ ) and C 6 (K, η, f ∞ ) positive constants depending on K, η and f ∞ . We have a similar result for h ′ ∈Hn P fh ′ -Ẽ fh ′

(1 + 2ζ)(1 + η/2) = (1 + η). For (h ′ 1 , h ′ 2 ) ∈ H n , (log n) 3 ρn ≤ (log n) 3 ρδn ≤ (log n) 3 δn ≤ h ′ 1 < 1 and 1 n ≤ h ′ 2 < 1 (log n) 2 -1 . So, -nh ′ 1 δ ≤ -(log n) 3/2 and -1 h ′ 2 < -(log n) 2 + 1. Therefore, on Λ ρ , h,h ′ ∈Hn P   fh,h ′ -Ẽ fh,h ′ x,2 ≥ (1 + η) K 1 K 2 δnh ′ 1 h ′ 2   ≤ ρ 2 n 4 e -C5(K,η, f ∞ )(log n) 3/2 ≤ C 6 (K, η, f ∞ )e -(log n) 5/4 ,
x,2

≥ (1 + η) K 2 √ δnh ′ 1 h ′ 2
. Now to conclude, note that the right hand side of Inequality (8.4) is not random. This allows us to obtain the result of the proposition.

Proof of Proposition 4

We have the following decomposition

K h * g -g = K h * g -K h * f + K h * f -f + f -g. (8.5) Next, on Λ ρ , |K h * g(x, y) -K h * f (x, y)| = K h (x -u, y -v) (g(u, v) -f (u, v)) dudv = K h (x -u, y -v) f (u, v) fX (u) f X (u) -fX (u) dudv ≤ sup t∈Vn(x) f X (t) -fX (t) δ-1 |K h (x -u, y -v)|f (u, v)dudv ≤ sup t∈Vn(x) f X (t) -fX (t) δ -1 ρ |K h (x -u, y -v)|f (u, v)dudv.
Now by using (3.1), we have:

|K h (x -u, y -v)|f (u, v)dudv 2 dy ≤ K 1 |K h (x -u, y -v)|f 2 (u, v)dudvdy ≤ K 1 K (2) 1 |K (1) h1 (x -u)|f 2 (u, v)dudv ≤ f ∞ K 2 1 .
Then we deduce that (8.6) where C depends on ρ, f ∞ and K. Moreover we have on Λ ρ :

K h * g -K h * f ∞,2 ≤ Cδ -1 sup t∈Vn(x) f X (t) -fX (t) ,
g -f 2 t,2 = f 2 (t, y) f2 X (t) fX (t) -f X (t) 2 dy ≤ f ∞ δ-2 | fX (t) -f X (t)| 2 ≤Cδ -2 |f X (t) -fX (t)| 2 ,
where C depends on ρ and f ∞ . The last line, (8.5) and (8.6) allow us to conclude.

Proof of Proposition 5

For any h ∈ H n , we have 1

n δh 1 ≤ 1 (log n) 3 , 1 h 2 ≤ n.
Therefore,

fh 2 x,2 ≤ 1 n n i=1 fX (X i ) -1 1 h 1 K (1) x -X i h 1 1 h 2 K (2) y -Y i h 2 2 dy ≤ 1 n n i=1 1 δh 1 K (1) ∞ 1 h 2 K (2) y -Y i h 2 2 dy ≤ n(log n) -6 K (1) 2 ∞ n i=1 1 h 2 2 K (2) y -Y i h 2 2 dy ≤ n 3 (log n) -6 K (1) 2 ∞ K (2) 2
2 , which proves the result.

Proof of Theorem 5

We first assume that d 1 = d 2 = 1. Using conditions (BK M ), we then have:

(K h * f )(x, y) -f (x, y) = K(u, v) [f (x -uh 1 , y -vh 2 ) -f (x, y)] dudv = K(u, v) [f (x -uh 1 , y -vh 2 ) -f (x, y -vh 2 ) + f (x, y -vh 2 ) -f (x, y)] dudv = K(u, v) (-uh 1 ) ⌊α1⌋ ⌊α 1 ⌋! d ⌊α1⌋ dx ⌊α1⌋ f (x + ũh 1 , y -vh 2 ) - d ⌊α1⌋ dx ⌊α1⌋ f (x, y -vh 2 ) dudv + K(u, v) (-vh 2 ) ⌊α2⌋ ⌊α 2 ⌋! d ⌊α2⌋ dy ⌊α2⌋ f (x, y + ṽh 2 ) - d ⌊α2⌋ dy ⌊α2⌋ f (x, y) dudv where |ũ| ≤ |u| and |ṽ| ≤ |v|. If f ∈ H 2 (α, L), this implies that |(K h * f )(x, y) -f (x, y)| ≤ C 1 L 1 h α1 1 + C 2 L 2 h α2 2 ,
where C 1 and C 2 depend on α 1 , α 2 and K. We can easily generalize this result to the case d 1 , d 2 ≥ 2 and we obtain:

|(K h * f )(x, y) -f (x, y)| ≤ C d i=1 L i h αi i ,
with a constant C depending on α and K. Now taking

h i = L -1 α i i ∆ -1 α i n , ∆ n = d i=1 L 1 α i i -α 2α+1 (δn) α 2α+1 ,
we obtain that

1 δn d i=1 h i = ∆ -1 n and sup t∈Vn(x) K h * f -f t,2 ≤ C(δn) -ᾱ 2 ᾱ+1 d i=1 L 1 α i i ᾱ 2 ᾱ+1
, using (H 4 ) and where C is a positive constant that does not depend on δ, n and L. By using Theorem 2, this concludes the proof of Theorem 5.

Proofs for the projection estimator

The structure of the proof of the main theorem, namely Theorem 3, is similar to the structure of the proofs for kernel rules. It is detailed along Section 8.2.1.

Proof of Theorem 3

First, let

Γ = {∀m ∈ M n fm -K m (f ) x,2 ≤ σ(m)/2}.
To prove Theorem 3, we follow the GLM, as explained in Section 2, with . = . x,2 , and the operator K m is the projection on S m . In this case, using Lemma 6,

B(m) = sup m ′ ∈Mn K m ′ (f ) -(K m ′ • K m )(f ) x,2 ≤ (r + 1) f X ∞ δ -1 sup t∈Vn(x) f -K m (f ) t,2 . Moreover for all m, m ′ ∈ M n , K m ′ • K m = K m∧m ′ , with m ∧ m ′ = (min(m 1 , m ′ 1 ), min(m 2 , m ′ 2 )
), and σ(m ∧ m ′ ) ≤ σ(m ′ ). As already explained in Section 2, we introduce Km ( fm ′ ) = fm∧m ′ and

ξ(m) = sup m ′ ∈Mn ( fm ′ -K m ′ (f )) -( Km ′ ( fm ) -(K m ′ • K m )(f )) x,2 -σ(m ′ ) + .
Let us fix m ∈ M n . We obtain inequality (2.3) in our case:

f -f x,2 ≤ 2B(m) + 2σ(m) + fm -K m (f ) x,2 + f -K m (f ) x,2 + 2ξ(m). But, on Γ, for all m, m ′ in M n , fm ′ -K m ′ (f ) x,2 ≤ σ(m ′ )/2 and fm∧m ′ -K m∧m ′ (f ) x,2 ≤ σ(m ′ )/2, so that ξ(m) = 0. Then, on Γ, f -f x,2 ≤ 2B(m) + 2σ(m) + σ(m) 2 + f -K m (f ) x,2 ≤ (2(r + 1) f X ∞ δ -1 + 1) sup t∈Vn(x) f -K m (f ) t,2 + 5 2 σ(m). (8.7)
Now, let . n be the empirical norm defined by

t n = 1 n n i=1 t 2 (X i ) 1/2
and l m1 be the index such that x belongs to the interval I lm 1 . For ρ = (1 + η) 1/5 , let

Ω ρ = ∀ m, ∀t ∈ Span(ϕ m lm 1 ,d ) 0≤d≤r t 2 n ≥ ρ -1 t 2 (u)f X (u)du .
The heart of the proof of Theorem 3 is the following concentration result: Proposition 6. Assume that assumptions (H 1 ), (H 2 ), (H 3 ) and (CM ) are satisfied. There exists C > 0 only depending on η, φ 1 , φ 2 , r, f ∞ and f X ∞ and δ such that

P (Γ c ∩ Λ ρ ∩ Ω ρ ) ≤ C exp{-(log n) 5/4 }.
Proposition 6 and the following result show that the event Γ occurs with large probability.

Proposition 7. Assume that assumptions (H 2 ), (H 3 ) and (CM ) are satisfied. Then,

P(Ω c ρ ∩ Λ ρ ) ≤ C exp{-(log n) 5/4 },
where C is a constant only depending on ρ, φ 1 , r, f X ∞ and δ.

Then, using Lemma 4 and Propositions 6 and 7,

P(Γ c ) ≤ P((Γ ∩ Λ ρ ∩ Ω ρ ) c ) = P(Γ c ∩ Λ ρ ∩ Ω ρ ) + P(Ω c ρ ∩ Λ ρ ) + P(Λ c ρ ) ≤ Ke -log 5/4 (n) (8.8)
with K depending on η, φ 1 , φ 2 , r, f ∞ and f X . Then, the first part of Theorem 3 is proved. To deduce the second part, we use the following proposition.

Proposition 8. For all m ∈ M n ,

f -fm 2 x,2 ≤ 2 f ∞ + 2(1 + η) 4/5 δ-2 (r + 1)φ 2 1 φ 2 D 2 m1 D 2 m2 .
Using assumption (CM ), it implies that f -f m 2

x,2 ≤ C2 3 n 4 , where C3 depends on η, r, φ 1 , φ 2 and f ∞ . Then, by using (8.7) which is true on Γ ∩ Λ ρ we have

E f -f q x,2 = E f -f q x,2 1 Γ∩Λρ + E f -f q x,2 1 (Γ∩Λρ) c ≤ C4 sup t∈Vn(x) f -K m (f ) t,2 + f X ∞ δ D m1 D m2 δn q + Cq 3 n 2q P((Γ ∩ Λ ρ ∩ Ω ρ ) c ),
where C4 depends on η, φ 1 , φ 2 , r, f X ∞ and δ. Using (8.8), this concludes the proof of Theorem 3.

Proof of Proposition 6

First, we introduce some preliminary material. For any matrix M , we denote

M 2 = sup x =0 M x x , M F =   j,k |M j,k | 2   1 2
the operator norm and the Frobenius norm. We shall use that for any matrices M and N ,

M 2 ≤ M F , M N 2 ≤ M 2 N 2 , M N F ≤ M 2 N F .
Now we fix m ∈ M n . Then the index l m1 such that x belongs to the interval I lm 1 is fixed. For the sake of simplicity, we denote it by l. Note that

I l ⊂ V n (x), since 2 -m1 ≤ k -1 n . We set F (l) m1 = Span(ϕ m l,d ) 0≤d≤r . Moreover we denote Ĝ = Ĝ(l) m , Ẑ = Ẑ(l) m ,  = Â(l) m , ϕ d = ϕ m l,d , ψ k = ψ m k . The elements of  are denoted (â d,k ) d,k instead of (â m (lm 1 ,d),k ) d,k . We also introduce G = E( Ĝ) = ( ϕ d1 , ϕ d2 X ) 0≤d1,d2≤r and
Z = E( Ẑ) = ϕ d (u)ψ k (y)f (u, y)f X (u)dudy 0≤d≤r,k∈Km
.

By using Lemma 1, the coefficients (a m j,k ) of K m (f ) in the basis verify the matrix equation GA = Z where the coefficients of the matrix A are A d,k = a m (lm 1 ,d),k but are denoted a d,k for short. We shall use the following algebra result. If M is a symmetric matrix,

min(Sp(M )) = min u u * M u u * u . Then min(Sp(G)) = min u u * Gu u * u = min t∈F (l) m 1 t 2 (u)f X (u)du t 2 2 ≥ δ (8.9)
and, in the same way,

min(Sp( Ĝ)) = min u * Ĝu u * u = min t∈F (l) m 1 t 2 n t 2 2 , so that on Ω ρ min(Sp( Ĝ)) ≥ ρ -1 δ. (8.10)
Now, let us begin the proof of Proposition 6. Since

( fm -K m (f ))(x, y) = r d=0 k∈Km (â d,k -a d,k )ϕ d (x)ψ k (y) we deduce fm -K m (f ) 2 x,2 = k ( d (â d,k -a d,k )ϕ d (x)) 2 ≤ d ϕ 2 d (x) k d (â d,k -a d,k ) 2 ≤ φ 1 D m1 Â -A 2 F .
On Λ ρ , δ ≥ ρ -1 δ. Then, using (8.10), on Ω ρ ∩ Λ ρ , min(Sp( Ĝ)) ≥ ρ -2 δ = (1 + η) -2/5 δ, so we are in the case where  = Ĝ-1 Ẑ. From now on, we always assume that we are on Ω ρ ∩ Λ ρ . We have:

 -A F ≤ ( Ĝ-1 -G -1 )Z F + Ĝ-1 ( Ẑ -Z) F ≤ Ĝ-1 -G -1 2 Z F + Ĝ-1 2 Ẑ -Z F .
Since Ĝ is symmetric, Ĝ-1 2 is equal to the spectral radius of Ĝ-1 . And, using (8.10), its eigenvalues are positive, then Ĝ-1 2 = (min(Sp( Ĝ))) -1 ≤ ρδ -1 . In the same way, using (8.9),

G -1 2 = (min(Sp(G))) -1 ≤ δ -1 . Then, Ĝ-1 -G -1 2 = Ĝ-1 (G -Ĝ)G -1 2 ≤ ρδ -2 G -Ĝ 2 ≤ ρδ -2 G -Ĝ F . Thus  -A F ≤ ρδ -2 G -Ĝ F Z F + ρδ -1 Ẑ -Z F .
Moreover, since for any function s, d s, ϕ d 2 ≤ I l s 2 (u)du, where , denotes the standard L 2 dot product,

Z 2 F = r d=0 k∈Km ϕ d (u)f (u, .)f X (u)du, ψ k 2 ≤ r d=0 ϕ d (u)f (u, y)f X (u)du 2 dy ≤ I l f 2 (u, y)f 2 X (u)dudy ≤ f X 2 ∞ f ∞ (4A2 -m1 ). Finally (still on Ω ρ ∩ Λ ρ ), fm -K m (f ) x,2 ≤ C 3 Ĝ -G F + ρδ -1 φ 1 D m1 Ẑ -Z F .
Here

C 3 = f X ∞ ρδ -2 (r + 1) f ∞ . Thus, with P ρ (•) = P(• ∩ Λ ρ ∩ Ω ρ )
, we can write:

P ρ fm -K m (f ) x,2 ≥ σ(m) 2 ≤ P 1,m + P 2,m with          P 1,m = P ρ Ẑ -Z F ≥ σ(m) 2ρ 2 δ -1 φ 1 D m1 P 2,m = P ρ Ĝ -G F ≥ σ(m) 2ρC 3 (ρ -1) . [1] Study of P 1,m : Let ν n (t) = 1 n n i=1 t(X i , Y i ) -E(t(X i , Y i )) and
S (l) m = F (l) m1 ⊗ H m2 = t, t(x, y) = r d=0 k∈Km b d,k ϕ d (x)ψ k (y), b d,k ∈ R . Then, sup t∈S (l) m , t 2≤1 |ν n (t)| 2 = d,k |ν n (ϕ d ⊗ ψ k )| 2 = d,k 1 n n i=1 ϕ d (X i )ψ k (Y i ) -E(ϕ d (X i )ψ k (Y i )) 2 = Ẑ -Z 2 F .
We are reduced to bound:

P ρ sup t∈S (l) m , t 2 ≤1 |ν n (t)| ≥ σ(m) 2ρ 2 δ -1 φ 1 D m1 .
To deal with this term, we use Lemma 3. So, we consider A a dense subset of {t ∈ S (l) m , t 2 ≤ 1} and we compute M, H and v.

•

First, if t = d,k b dk ϕ d ⊗ ψ k then |t(u, v)| 2 = | d,k b dk ϕ d (u)ψ k (v)| 2 ≤ d,k b 2 dk d,k |ϕ d (x)ψ k (v)| 2 ≤ t 2 2 φ 1 D m1 φ 2 D m2 .
Thus sup t∈A t ∞ ≤ φ 1 φ 2 D m1 D m2 and we can take M = φ 1 φ 2 D m1 D m2 .

• Secondly, we recall

sup t∈S (l) m , t 2≤1 |ν n (t)| 2 = d,k 1 n n i=1 ϕ d (X i )ψ k (Y i ) -E(ϕ d (X i )ψ k (Y i )) 2 .
Since the data are independent,

Var 1 n n i=1 ϕ d (X i )ψ k (Y i ) = 1 n Var(ϕ d (X 1 )ψ k (Y 1 )).
We deduce:

k E 1 n n i=1 ϕ d (X i )ψ k (Y i ) -E(ϕ d (X i )ψ k (Y i )) 2 ≤ 1 n ϕ 2 d (u) k ψ 2 k (v)f X (u)f (u, v)dudv ≤ φ 2 D m2 n ϕ 2 d (u)f X (u) f (u, v)dv du ≤ φ 2 D m2 n f X ∞ .
Hence,

E sup t∈A ν 2 n (t) ≤ (r + 1) f X ∞ φ 2 D m2 n so that we can take H 2 = (r + 1) f X ∞ φ 2 D m2 /n. • Thirdly Var(t(X 1 , Y 1 )) ≤ E|t(X 1 , Y 1 )| 2 ≤ t 2 (u, v)f X (u)f (u, v)dudv ≤ t 2 2 f ∞ f X ∞ and then we can take v = f ∞ f X ∞ . Finally ζ 2 nH 2 6v = ζ 2 (r + 1)φ 2 6 f ∞ D m2 min(ζ, 1)ζnH 21M = min(ζ, 1)ζ (r + 1) f X ∞ 21 √ φ 1 n D m1 .
According to condition (CM ), on Λ ρ , since δ ≤ 1, D m1 ≤ ρn/(log n) 3 and D m2 ≥ (log n) 2 . Thus Talagrand's Inequality gives

P ρ sup t∈A |ν n (t)| ≥ (1 + 2ζ)H ≤ 2 exp(-C log 3/2 (n)) with C only depending on η, ζ, r, φ 1 , φ 2 , f ∞ , f X ∞ . Moreover, (1 + 2ζ)H = (1 + 2ζ) (r + 1) f X ∞ φ 2 D m2 n and, since δ > ρ -1 δ and f X ∞ > ρ -2 f X ∞ on Λ ρ , σ(m) 2ρ 2 δ -1 φ 1 D m1 ≥ ρ -4 (1 + η) (r + 1) f X ∞ φ 2 D m2 n .
Then, since ρ

5 = 1 + η, choosing ζ such that 1 + 2ζ = ρ gives σ(m) 2ρ 2 δ -1 φ 1 D m1 ≥ (1 + 2ζ)H
and then P 1,m ≤ 2 exp(-C log 3/2 (n)).

[2] Study of P 2,m : We now have to bound (with large probability) the term

Ĝ -G 2 F = d,d ′ 1 n n i=1 ϕ d ϕ d ′ (X i ) -E[ϕ d ϕ d ′ (X i )] 2 .
We use Bernstein's Inequality (Lemma 2): Since

sup u∈R |ϕ d (u)ϕ d ′ (u)| ∞ ≤ φ 1 D m1 and E|ϕ d ϕ d ′ (X 1 )| 2 ≤ ϕ 2 d ϕ 2 d ′ (u)f X (u)du ≤ φ 1 f X ∞ D m1 , the assumptions of Lemma 2 are satisfied with c = φ 1 D m1 and v = φ 1 f X ∞ D m1 . If we set ε = C 4 Dm 1 Dm 2 n , with C 4 = (ρ -1)(1 + η) φ 1 φ 2 f X ∞ /(ρ 3 C 3 δ √ r + 1) then, on Λ ρ , ε ≤ (ρ -1)σ(m) 2ρC 3 (r + 1) . Moreover on Λ ρ , since δ ≤ 1, nε 2 v = C 2 4 φ 1 f X ∞ D m2 ≥ C 2 4 φ 1 f X ∞ (log n) 2 nε c = C 4 φ 1 nD m2 D m1 ≥ C 4 φ 1 √ ρδ (log n) 5/2 ≥ C 4 φ 1 √ ρ (log n) 5/2 .
Then, using Lemma 2,

P ρ Ĝ -G F ≥ (ρ -1) σ(m) 2C 3 ρ ≤ d,d ′ P ρ 1 n n i=1 ϕ d ϕ d ′ (X i ) -E[ϕ d ϕ d ′ (X i )] ≥ (ρ -1)σ(m) 2ρC 3 (r + 1) ≤ 2(r + 1) 2 exp(-C 5 log 2 (n)),
with C 5 only depending on η, r, φ 1 , φ 2 , f ∞ , f X ∞ and δ. Finally, we denote

M n = {(m 1 , m 2 ), 2 m1 ≤ ρδn, D m2 ≤ n} which verifies M n ⊂ M n on Λ ρ .
Gathering all the terms together, we obtain

P ρ ∃m ∈ M n fm -K m (f ) x,2 ≥ σ(m) 2 ≤ P ρ ∃m ∈ M n fm -K m (f ) x,2 ≥ σ(m) 2 ≤ m∈Mn P 1,m + P 2,m ≤ m∈Mn 4(r + 1) 2 exp(-C 6 log 3/2 (n)) ≤ 4(r + 1) 2 ρδn 2 exp(-C 6 log 3/2 (n)), with C 6 depending on η, r, φ 1 , φ 2 , f ∞ , f X ∞ and δ, which yields Proposition 6. If Sp( Ĝ) ≥ (1 + η) -2/5 δ (otherwise  = 0), Ĝ-1 2 = ρ( Ĝ-1 ) = (min(Sp( Ĝ))) -1 ≤ (1 + η) 2/5 δ-1 .
Therefore, we have:

 2 F ≤ Ĝ-1 2 2 Ẑ 2 F ≤ (1 + η) 4/5 δ-2 d,k 1 n n i=1 ϕ d (X i )ψ k (Y i ) 2 ≤ (1 + η) 4/5 δ-2 d,k φ 1 φ 2 D m1 D m2
≤ (1 + η) 4/5 δ-2 (r + 1)φ 1 φ 2 D m1 D 2 m2 .

Finally

fm 2 x,2 = k∈Km r d=0 âdk ϕ d (x) 2 ≤ Â 2 F φ 1 D m1
≤ (1 + η) 4/5 δ-2 (r + 1)φ 2 1 φ 2 D 2 m1 D 2 m2 .

Proof of Theorem 6

We first assume that d 1 = d 2 = 1. We denote K 1 m the projection on F m1 endowed with the scalar product (g, h) X = g(z)h(z)f X (z)dz, and K 2 m the projection on H m2 endowed with the usual scalar product (g, h) us = g(z)h(z)dz. The projection K m (f ) can be written for any u and any y,

K m (f )(u, y) = k∈Km (f 1 (u, .), ψ m k ) us ψ m k (y) = K 2 m (f 1 (u, .))(y)
where f 1 (., y) = K 1 m (f (., y)). Thus we have the factorization (K m (f )f )(u, .) = K 2 m (f 1 (u, .)f (u, .)) + K 2 m (f (u, .))f (u, .)

and applying Pythagora's theorem K m (f )f 2 x,2 = K 2 m (f 1 (x, .)f (x, .)) 2 2 + K 2 m (f (x, .))f (x, .) 2 2 ≤ f 1 (x, .)f (x, .) 2 2 + K 2 m (f (x, .))f (x, .) 2 2 .

Now, we shall use the following result. Let τ be a univariate function belonging to the Hölder space H 1 (α, L) on a interval with length b. If S is the space of piecewise polynomials of degree bounded by r > α -1 based on the regular partition with 2 J pieces, then there exists a constant C(α, b) only depending on α and b such that

d ∞ (τ, S) := inf t∈S t -τ ∞ ≤ C(α, b)L2 -Jα
(see for example Lemma 12 in [START_REF] Barron | Risk bounds for model selection via penalization[END_REF]). Let K the orthogonal projection on S endowed with some scalar product. We denote

|||K||| = sup t∈L∞\{0} K(t) ∞ t ∞ .
Then, for all t ∈ S, since K(t) = t, τ -K(τ

) ∞ = τ -t + K(t -τ ) ∞ ≤ (1+|||K|||) t -τ ∞ .
We obtain:

τ -K(τ ) ∞ ≤ (1+|||K|||) inf t∈S t -τ ∞ ≤ (1+|||K|||)C(α, b)L2 -Jα .
It remains to bound |||K||| in the following cases.

• Case 1: S is the space of piecewise polynomials of degree bounded by r 1 , endowed with (., .) X (S = F m1 , K = K 1 m ). It is sufficient to apply Lemma 6 to the function τ (u, y) = t(u)ψ m k (y) to obtain |||K||| ≤ (r 1 + 1) f X ∞ δ -1 .

• Case 2: S is the space of piecewise polynomials of degree bounded by r 2 , endowed with the usual dot product (S = H m2 , K = K 2 m ). Then it is sufficient to apply the previous case with f X identically equal to 1, to obtain |||K||| ≤ (r 2 + 1).

Finally, we have obtained the following result: if τ is a univariate function belonging to the Hölder space H 1 (α, L) then

τ -K 1 m (τ ) ∞ ≤ C(α, A, r 1 , f X ∞ /δ)LD -α m1 , τ -K 2 m (τ ) ∞ ≤ C(α, |B|, r 2 )LD -α m2 .
Now f (x, .) belongs to the Hölder space H 1 (α 2 , L 2 ) then

K 2 m (f (x, .)) -f (x, .) ∞ ≤ C 2 L 2 D -α2 m2
with C 2 depending on α 2 , |B| and r 2 . Moreover, for all y ∈ B, f (., y) belongs to the Hölder space H 1 (α 1 , L 1 ) then

|f 1 (x, y) -f (x, y)| ≤ K 1 m (f (., y)) -f (., y) ∞ ≤ C 1 (α 1 , A, r, f X ∞ /δ)L 1 D -α1 m1
with C 1 not depending on y. Finally, since the support of f (x, .), f 1 (x, .), K 2 m (f (x, .)) is compact, we obtain

K m (f ) -f x,2 ≤ C 0 (L 1 D -α1 m1 + L 2 D -α2 m2 ).
with C 0 depending on A, |B|, r, α 1 , α 2 and f X ∞ and δ. We can easily generalize this result to the case d 1 , d 2 ≥ 2 and we obtain:

K m (f ) -f x,2 ≤ C d i=1 L i 2 -αimi
for C a constant. To conclude, by using Theorem 3, it remains to find (m 1 , . . . , m d ) that minimizes

(m 1 , . . . , m d ) -→ d i=1 L i 2 -αimi + d i=1 2 mi δn .
Solving this minimization problem shows that 2 mi has to be equal to L

1 α i i ∆ 1 α i
n up to a constant and

∆ n = d i=1 L 1 α i i -α 2α+1 (δn) α 2α+1 .
It gives the result.

Lemma 1 shows that the matrix of coefficients B = (b d,k ) 0≤d≤r,k∈Km verifies the equation GB = I(τ ), with G = E( Ĝ) = ( ϕ d1 , ϕ d2 X ) 0≤d1,d2≤r . Now, using (8.9), τ 2 (u, y)f 2 X (u)du dy using that for any function s, d ( sϕ d ) 2 ≤ I l s 2 . Next, using that I l is an interval with length 4A(r + 1)D -1 m1 ,

K m (τ ) 2 x,2 = k ( d b d,k ϕ d (x)) 2 ≤ d ϕ 2 d (x) k d b 2 d,k ≤ φ 1 D m1 B 2 F ≤ φ 1 D m1 G -1 2 2 I(τ ) 2 F ≤ φ 1 D m1 δ -2 I(τ ) 2 F .

Now we denote Proj

I(τ ) 2 F ≤ sup t∈I l τ 2 t,2 I l f 2 X (u)du ≤ 4A(r + 1)D -1 m1 f X 2 ∞ sup t∈I l τ 2 t,2 . Finally K m (τ ) 2 x,2 ≤ φ 1 D m1 δ -2 4A(r + 1)D -1 m1 f X 2 ∞ sup t∈I l τ 2 t,2 ≤ (r + 1) 2 f X 2 ∞ δ -2 sup t∈Vn(x) τ 2 t,2
and the lemma is proved.

B Tables for simulation results

In this appendix, for each example and each procedure, we give the approximated mean squared error based on N = 100 samples for different values of n, different values of the parameter η and different values of x. We give in bold red the minimal value of the approximated mean squared error. For the kernel estimator and Examples 1 and 2, we distinguish the case where f X is known or not.

Ex 1 f X known f X unknown η -0.2 0.5 1 2 3 -0.2 0.5 1 2 3 n = 250 1.285 0.061 0.017 0.020 0.029 1.368 0.033 0.028 0.042 0.062 n = 500 0.673 0.019 0.009 0.010 0.018 0.685 0.016 0.009 0.011 0.018 n = 1000 0.336 0.013 0.006 0.006 0.009 0.329 0.013 0.006 0.007 0.010 f X known f X unknown η -0.2 0.5 1 2 3 -0.2 0.5 1 2 3 n = 250 0.038 0.008 0.006 0.007 0.009 0.042 0.008 0.006 0.008 0.009 n = 500 0.021 0.006 0.004 0.005 0.006 0.025 0.006 0.004 0.005 0.007 n = 1000 0.01 0.004 0.003 0.004 0.005 0.012 0.004 0.003 0.004 0.005

Figure 1 :

 1 Figure 1: Plots of true function f (x, .) (plain line) versus kernel estimator f (x, .) (dashed line) and projection estimator f (x, .) (dot-dashed line) in x = 0 (n = 1000) for Example 3 (left) and Example 4 (right)

Figure 2 :

 2 Figure 2: MSE( f ) for n = 500, Example 1 (x = 0.5), Example 2 (x = 0.5), Example 3 (x = 0), Example 4 (x = 0)

  u)| ≤ M, E sup a∈A |ν n (a)| ≤ H, sup a∈A Var(τ a (U 1 )) ≤ v.

(

  Hm 2 the usual L 2 orthogonal projection on H m2 and (•, •) us the standard L 2 dot product. Notice that for any functions ∈ L 2 (R), k∈Km (s, ψ k ) 2 us = |Proj Hm 2 (s)| 2 (y)dy ≤ s 2 (y)dy. Then ϕ d (u)τ (u, .)f X (u)du, ψ k ) 2 us ≤ r d=0 ϕ d (u)τ (u, y)f X (u)du 2 dy ≤ I l

Table 1 :

 1 Mean squared error for the kernel estimator at x = 00.492 0.192 0.222 0.232 0.231 n = 500 0.087 0.076 0.119 0.211 0.229 n = 1000 0.051 0.047 0.055 0.070 0.138 Table 2: Mean squared error for the projection estimator at x = 0.5 for Example 1 Ex 2

  Therefore, we modify this approach to overcome this problem. The idea is the following. Let us denote S m∧m ′ = S m ∩ S m ′ . Taking inspiration from the fact that

	introduce the following empirical contrast:				• = • x,2 . Now, we
	for all function t, γ n (t) =	1 n	n i=1	R	t 2 (X i , y)dy -2t(X i , Y i ) ,
	so that E(γ n (t)) is minimum when t = f (see Lemma 1 in Section 5.2). Given m in M n , the conditional density can be estimated by:
	fm ∈ arg min t∈Sm	γ n (t).	(2.10)
	Unlike kernel rules, this estimate does not depend on f X but (2.2) and (2.5) are not satisfied even if for large values of n, K m (f ) ≈ E[ fm ].

Table 3 :

 3 Mean squared error for the kernel estimator at x = 0.5 for Example 2

	Ex 2	f X unknown
	η n = 250	0.5 0.154 0.104 0.128 0.152 0.158 1 2 3 -0.2
	n = 500 0.064 0.070 0.090 0.103 0.123
	n = 1000 0.047 0.060 0.063 0.074 0.088

Table 4 :

 4 Mean squared error for the projection estimator at x = 0.5 for Example 2

	Ex 3		f X unknown
	0.5 0.514 0.016 x\η -0.2 0	1 0.013 0.012 0.019 2 3
	n = 250 0.36 0.092 0.062 0.080	0.112	0.134
	1	1.709 0.015 0.009 0.009 0.016
	0	0.269 0.013	0.013 0.009 0.010
	n = 500 0.36 0.109 0.040 0.039 0.063	0.094
	1	0.601 0.010	0.009 0.006 0.008
	0	0.126 0.011	0.011	0.008 0.006
	n = 1000 0.36 0.104 0.029 0.024 0.037	0.056
	1	0.265 0.006	0.007 0.004 0.004

Table 5 :

 5 Mean squared error for the kernel estimator at x = 0, x = 0.36 and x = 1 for Example 3

Acknowledgements: The research of Claire Lacour and Vincent Rivoirard is partly supported by the french Agence Nationale de la Recherche (ANR 2011 BS01 010 01 projet Calibration). Karine Bertin has been partially supported by Project ECOS-CONICYT C10E03 and by the grant ANILLO ACT-1112, CONICYT-PIA, Chile. The authors wish to thank two anonymous referees who each made helpful suggestions that improved the presentation of the paper.

Proof of Proposition 7

In this Section, we denote t 2 X := t 2 (u)f X (u)du.

We recall that l m1 is the index such that x belongs to the interval I lm 1 and as in Section 8.2.2, we set: F

(lm 1 ) m1

= Span(ϕ m lm 1 ,d ) 0≤d≤r . We want to bound P(Ω c ∩ Λ ρ ) = P ∃m 1 , ∃t ∈ Span(ϕ m lm 1 ,d ) 0≤d≤r t 2 n < ρ -1 t 2 X and Λ ρ .

Under (CM ), we have: k n (r + 1) ≤ D m1 ≤ δn/(log n) 3 , and on Λ ρ , we have: 2 m1 ≤ ρδn. Let µ n be the empirical process defined by

Then, µ n (t 2 ) = t 2 nt 2 X , which implies that

But, for all t ∈ F (lm 1 ) m1

Using Lemma 2, we easily prove as in Section 8.2.2 that ∀m ∈ M n

with K depending on ρ, φ 1 , r, f X ∞ , δ. Then

which yields the result.

Proof of Proposition 8

First, as already noticed, f 2 x,2 ≤ f ∞ . Now let m be a fixed element of M n . Then we denote l = l m1 the index such that x belongs to the interval I l and moreover we denote

A Proofs of technical results

A.1 Proof of Lemma 3

We apply the Talagrand concentration inequality given in [START_REF] Klein | Concentration around the mean for maxima of empirical processes[END_REF] to the functions

and we obtain

Then we modify this inequality following Birgé and Massart (1998) Corollary 2 p.354. It gives

To conclude, we set λ = ζH.

A.2 Proof of Lemma 4

The lemma is a consequence of (3.3) used with

Thus, with probability 1κ exp(-

A.3 Proof of Lemma 5

We have:

A.4 Proof of Lemma 6

Let l the index such that x belongs to the interval I l . We denote