Gaëlle Chagny
email: gaelle.chagny@gmail.com

Angelina Roche
email: angelina.roche@parisdescartes.fr

ADAPTIVE ESTIMATION IN THE FUNCTIONAL NONPARAMETRIC REGRESSION MODEL

Keywords: Functional data analysis, Regression estimation, Nonparametric kernel estimators, Bandwidth selection AMS Subject Classication 2010: 62G08; 62H12

In this paper, we consider nonparametric regression estimation when the predictor is a functional random variable (typically a curve) and the response is scalar.

Starting from a classical collection of kernel estimates, the bias-variance decomposition of a pointwise risk is investigated to understand what can be expected at best from adaptive estimation. We propose a fully data-driven local bandwidth selection rule in the spirit of the Goldenshluger and Lepski method. The main result is a nonasymptotic risk bound which shows the optimality of our tuned estimator from the oracle point of view.

Convergence rates are also derived for regression functions belonging to Hölder spaces and under various assumptions on the rate of decay of the small ball probability of the explanatory variable. A simulation study also illustrates the good practical performances of our estimator.

1. Introduction 1.1. Statistical model. Nowadays, more and more advanced technologies make possible the recording of observations in such a way that the collected data may be considered as curves (or surfaces). This explains why developing methods for Functional Data Analysis (F.D.A.) is a great challenge for the statisticians, which has become more and more popular in the past decades, as it is highlighted by the monographies of [START_REF] Ramsay | puntionl ht enlysis[END_REF], Dabo-Niang and [START_REF] Dabo-Niang | puntionl nd opertoril sttistis[END_REF] and [START_REF] Ferraty | he yxford rndook of puntionl ht enlysis[END_REF]. Regression with functional covariate is one of the most studied problem. The functional linear model has rst been widely investigated (see, among all, [START_REF] Ramsay | Some tools for functional data analysis[END_REF][START_REF] Cai | Prediction in functional linear regression[END_REF][START_REF] Crambes | Smoothing splines estimators for functional linear regression[END_REF]) with its generalisations [START_REF] Cardot | Estimation in generalized linear models for functional data via penalized likelihood[END_REF][START_REF] Müller | Generalized functional linear models[END_REF]. However, studies of the nonparametric regression model, in which we are interested, are more recent. Consider

(1)

Y = m(X) + ε,
where Y is a real random variable, X a random variable which takes values in a separable innite-dimensional Hilbert space (H, •, • , • ) (it can be L 2 (I), the set of squaredintegrable functions on a subset I of R, or a Sobolev space), and m : H → R the target function to recover. The random variable ε stands for a noise term. We suppose that ε and X are independent and that ε is centred with E[ε 2 ] 1/2 = σ < ∞. The specicity thus stands in the dimension which is innite in two aspects: rst, the framework is a functional Date: December 29, 2014.

1 one (the covariate X is a stochastic process which lives in an innite-dimensional space), and then, no assumption is made on the form of the function m to estimate. The aim of this paper is to provide an adaptive optimal strategy to estimate the regression function m from a data sample {(X i , Y i ), i = 1, ..., n} distributed like the couple (X, Y ), that is Y i = m(X i ) + ε i , with ε i 's independent identically distributed (iFiFdF in the sequel) like ε, and independent from the X i 's. We explore the pointwise risk of a collection of kernel estimates, and propose to select the bandwidth in a data-driven way at a xed curve so as to obtain a theoretical non-asymptotic risk bound, which proves the optimality of our estimator in the collection.

1.2. State of the art and motivation. The starting point of our work is the following collection of estimators. Let K : R → R be a kernel function (that is R K(u)du = 1), K h : u → K(u/h)/h, for h > 0, and

(2)

m h (x) := n i=1 W (i) h (x)Y i where W (i) h (x) := K h ( X i -x ) n j=1 K h ( X j -x )
, for any (x, y) ∈ H × R. Inspired by the classical kernel methods of [START_REF] Nadaraya | On estimating regression[END_REF] and [START_REF] Watson | Smooth regression analysis[END_REF], these estimators have rst been introduced in the functional context by [START_REF] Ferraty | Dimension fractale et estimation de la régression dans des espaces vectoriels semi-normés[END_REF] and have then aroused considerable interest and attention. A rst order of magnitude for the risk is given by Ferraty and Vieu (2002), almost complete convergence is proved by [START_REF] Ferraty | Nonparametric models for functional data, with application in regression, time-series prediction and curve discrimination[END_REF], [START_REF] Ferraty | Nonparametric regression on functional data: inference and practical aspects[END_REF] deals with asymptotic normality and [START_REF] Ferraty | Rate of uniform consistency for nonparametric estimates with functional variables[END_REF] examine asymptotic expansions for the mean integrated squared error. Finally, a clear account about convergence rates (upper and lower bounds for the pointwise risk of (2) under concentration assumptions for the process X, as well as minimax rates) is provided by [START_REF] Mas | Lower bound in regression for functional data by representation of small ball probabilities[END_REF]. Moreover, this kind of estimates gives rise to other kernel-based strategies to recover m in the same functional context. Local linear methods in the spirit of local polynomial estimators have been explored by [START_REF] Baíllo | Local linear regression for functional predictor and scalar response[END_REF], [START_REF] Barrientos-Marin | Locally modelled regression and functional data[END_REF], [START_REF] Boj | Distance-based local linear regression for functional predictors[END_REF] and [START_REF] Berlinet | Local linear regression for functional data[END_REF], k-nearest neighbor kernels are examined by [START_REF] Burba | k-nearest neighbour method in functional nonparametric regression[END_REF], a recursive kernel approach is suggested by [START_REF] Amiri | Recursive estimation of nonparametric regression with functional covariate[END_REF] and Reproducing Kernel Hilbert Space-based methods are reported by [START_REF] Avery | RKHS-based functional nonparametric regression for sparse and irregular longitudinal data[END_REF]. Kernel methods have also been successfully extended to estimate the regression function m from dependent data (see, eFgF [START_REF] Masry | Nonparametric regression estimation for dependent functional data: asymptotic normality[END_REF][START_REF] Delsol | Advances on asymptotic normality in non-parametric functional time series analysis[END_REF] and references therein) or to consider the case of a functional response variable Y (in the model (1)), see [START_REF] Ferraty | Regression when both response and predictor are functions[END_REF].

All these methods have one point in common: they heavily depend on the choice of the smoothing parameter h (see ( 2)), the so-called bandwidth. Heuristically, a large value for h leads to an estimator with large bias (but small variance), while a too small value leads to high variability. It raises the question of dening a bandwidth selection criterion which is proved to automatically balance the bias-variance trade-o. However, not many investigations are concerned with this theoretical (the criterion should be theoretically justied) and practical (the bandwidth has to be chosen for application purpose) problem for F.D.A. The most commonly used method to select the bandwidth of a functional regression kernel estimate is the leave-one-out cross validation. The rst algorithm is proposed by Ferraty and Vieu (2002), used for dependent data by Ferraty et al. (2002) and Ferraty and Vieu (2006) and shown to be asymptotically optimal by [START_REF] Rachdi | Nonparametric regression for functional data: automatic smoothing parameter selection[END_REF]. It is a global method that do not depend on the point (curve) of estimation. A local version, also optimal in an asymptotic way, is proposed by [START_REF] Benhenni | Local smoothing regression with functional data[END_REF].

More recently, bayesian strategies have been studied [START_REF] Shang | Bayesian bandwidth estimation for a nonparametric functional regression model with unknown error density[END_REF][START_REF] Shang | Bayesian bandwidth estimation for a functional nonparametric regression model with mixed types of regressors and unknown error density[END_REF], but only for simulation purposes.

1.3. Contribution and overview. Few systematic studies have been undertaken to build a data-driven bandwidth selection method so as to obtain adaptive estimators and nonasymptotic risk bounds for functional regression estimation, while the same question is the subject of a wide litterature in classical nonparametric estimation. Only two papers, to our knowledge, focus on this problem. A method based on empirical prediction-risk minimisation is explored by [START_REF] Antoniadis | Bandwidth selection for functional time series prediction[END_REF]: oracle inequalities are proved, but the criterion is specic to functional time series prediction. Inspired both by the advances about the Lepski methods [START_REF] Goldenshluger | Bandwidth selection in kernel density estimation: oracle inequalities and adaptive minimax optimality[END_REF] and model selection, [START_REF] Chagny | Adaptive and minimax estimation of the cumulative distribution function given a functional covariate[END_REF] investigate a global bandwidth selection method to estimate a cumulative distribution function conditionally to a functional covariate, shown to be optimal both in the oracle and minimax sense, for an integrated error.

Motivated by this work and by the observation that a local bandwidth choice can improve signicantly the precision of estimation in a functional context (see the results of [START_REF] Benhenni | Local smoothing regression with functional data[END_REF], the goal of the present paper is to dene a local bandwidth selection criterion for the estimators (2) in functional regression in such a way that the resulting estimator is optimal in a nonasymptotic point of view. An upper-bound for a pointwise risk is rst derived in Section 2. For x 0 a xed point in H, the risk of an estimator m(x 0 ) computed at the curve x 0 , is, in the sequel,

E ( m(x 0 ) -m(x 0 )) 2 .
We obtain an exact bias-variance decomposition (Proposition 1) which permits to understand what we can expect at best from adaptive estimation, which is the subject of Section 3. The bandwidth selection, at a xed curve, is automatically performed in the spirit of [START_REF] Goldenshluger | Bandwidth selection in kernel density estimation: oracle inequalities and adaptive minimax optimality[END_REF]. The resulting estimator achieves the same performance as the one which would have been selected if the regularity index of the target function had been known, up to a constant and to a logarithm factor, as it is proved in our main result, Theorem 1. The result holds whatever the sample size. Convergence rates are also deduced in Section 3.4 for functions m belonging to Hölder spaces, and under various concentration assumptions on the process X. These assumptions are on the rate of decay of the small ball probability

ϕ x 0 : h > 0 → P( X -x 0 ≤ h), x 0 ∈ H,
which inuences the rate, as usual. The faster the small ball probability decreases, the smaller the rate of convergence of the estimator is. The rates (Proposition 2) are quite slow when X really lives in an innite-dimensional space. They nevertheless match with the lower bounds of [START_REF] Mas | Lower bound in regression for functional data by representation of small ball probabilities[END_REF]. This is the so-called "curse of dimensionality". Practical issues are discussed in Section 4 and the performances of our bandwidth selection criterion are compared with cross-validated criteria. Finally, the proofs are gathered in Section 5. Notice that we do not discuss in this work the choice of the norm . in the kernel estimators (2): we have shown in [START_REF] Chagny | Adaptive and minimax estimation of the cumulative distribution function given a functional covariate[END_REF] that the alternative choice of projection semi-norm does not improve the convergence rates, at least under our regularity assumption (H m dened below). Alternative regularity assumptions with data-driven semi-norms in the kernel could improve in practice the performances of kernel estimators. This is, to our opinion, an interesting perspective but this is far beyond the scope of this paper.

Risk of an estimator with fixed bandwidth

We provide in this section upper-bounds for the pointwise risk of any estimator (2) with xed bandwidth h.

2.1. Assumptions. The result will be obtained under the following assumptions. (H K ) The kernel K is of type I (Ferraty and Vieu, 2006) i.e. its support is in [0, 1] and there exist two constants c K , C K > 0 such that

c K 1 [0,1] ≤ K ≤ C K 1 [0,1] .
(H m ) There exists β ∈]0, 1], and a constant C m > 0 such that, for all x, x ∈ H,

|m(x) -m(x )| ≤ C m x -x β .
Assumption (H K ) is quite classical in kernel methods for functional data (see Ferraty et al. 2006;[START_REF] Burba | k-nearest neighbour method in functional nonparametric regression[END_REF][START_REF] Ferraty | Rate of uniform consistency for nonparametric estimates with functional variables[END_REF]. We are aware that this is a strong constraint on the choice of the kernel but alleviate it in a functional data context requires a lot of technical diculties and it is still, to our knowledge, an open problem. However, since the kernel is chosen by the statistician in practice, this is not a real problem. Assumption (H m ) is an Hölder-type regularity condition on the target function m to estimate, it is required to control the bias term of the risk.

2.2. Upper bound for the risk. The following theorem provides a bias-variance decomposition, in a non-asymptotic point of view.

Proposition 1. sf essumption @H m A is ful(lledD there exists onstnt C > 0 whih only depends on c K nd C K suh thtD for ll h > 0X

(3)

E ( m h (x 0 ) -m(x 0 )) 2 ≤ C h 2β + σ 2 nϕ x 0 (h)
.

The rst term of Inequality (3) is a bias term: it depends on the regularity of the function to estimate m and decreases when h decreases. The second term of Inequality (3) is a variance term which depends on the regularity of the process X through its small ball probability ϕ x 0 . This term increases when h decreases. This result is coherent with Ferraty et al. (2010, Theorem 2).

Adaptive estimation

We dene in this section a data-driven bandwidth selection method, which leads to an adaptive estimator.

3.1. Bandwidth selection. We have at our disposal the estimators m h dened by (2) for any h > 0. Let H n be a nite collection of bandwidths, with cardinality depending on n and properties precised below. Denote h max = max H n , h min = min H n . Our aim is to select a bandwidth h in the collection, only from the data, so as to obtain a resulting estimator which has a minimal risk, up to some constants or negligible terms, among the collection ( m h ) h∈Hn .

If one has access to the smoothness index β of m (see Assumption (H m )) and to the knowledge of the small ball probability ϕ x 0 (h), the best choice of bandwidth among the collection for a given curve x 0 ∈ H would be (4)

h * (x 0 ) = arg min h∈Hn h 2β + σ 2 nϕ x 0 (h) ,
thanks to the decomposition (3) in Proposition 1. This ideal choice, which realizes the best trade-o between the bias and variance terms is called the "oracle". It cannot be used in practice since it depends both on β and ϕ x 0 (h) which are generally unavailable for the statistician. The method we dene below is fully data-driven, and is in the spirit of the so-called Lepski methods: the most recent one, developed by [START_REF] Goldenshluger | Bandwidth selection in kernel density estimation: oracle inequalities and adaptive minimax optimality[END_REF], permits to derive the nonasymptotic results we want to obtain. The main idea is the following. Since the oracle h * depends on unknown quantities, let us "mimic" its choice by selecting

(5) h(x 0 ) = argmin h∈Hn A(h, x 0 ) + V (h, x 0 ) ,
where V (h, x 0 ) is an empirical counterpart for the variance term, and A(h, x 0 ) is an approximation of the bias term. To dene them, let us rst introduce an empirical version for the shifted small ball probability ϕ x 0 (h) = P( X -x 0 ≤ h) as follows ( 6)

ϕ x 0 (h) = 1 n n i=1 1 { X i -x 0 ≤h} .
It permits to set

(7) V (h, x 0 ) =    κσ 2 ln(n) n ϕ x 0 (h) if ϕ x 0 (h) = 0 +∞ otherwise,
where κ is a constant specied in the proofs which depends neither on h, nor on n, nor on m which is, in practice, xed once and for all (see Section 4.1.1). The idea of [START_REF] Goldenshluger | Bandwidth selection in kernel density estimation: oracle inequalities and adaptive minimax optimality[END_REF] is to dene A(h, x 0 ) by comparing two by two the estimators with xed bandwidth. In our functional context, the comparison can be done as follows:

(8)

A(h, x 0 ) = max h ∈Hn ( m h (x 0 ) -m h∨h (x 0 )) 2 -V (h , x 0 ) + .
This quantity is proved to be an approximation of the bias term (see Lemma 4). This motivates the following choice of the bandwidth ĥ(x 0 ), and the study of the selected estimator, at the point x 0 is m(x 0 ) := m h(x 0 ) (x 0 ): the theoretical results below establish that it really mimics the oracle. Notice that similar selection criteria, which are based on the used of auxiliary estimators with the bandwidth h ∨ h have been recently used by [START_REF] Chagny | Adaptive and minimax estimation of the cumulative distribution function given a functional covariate[END_REF] in a functional setting but for global selection purpose (the selected bandwidth does not depend on the curve x 0 ), or by Rebelles (2014) for a pointwise slection in multivariate density estimation.

3.2. Assumptions. We consider the following assumptions, in addition to the ones introduced in Section 2.1.

(H Hn ) The collection H n of bandwidths is such that: (H Hn,1 ) its cardinality is bounded by n,

(H Hn,2 ) for any h ∈ H n , ϕ x 0 (h) ≥ C 0 ln(n)/n, where C 0 > 0 is a purely numerical constant (specied in the proofs). (H ε ) For any integer l ≥ 2, E[|ε 1 | l ] ≤ C l ε σ l l!/2, for C ε > 1 a constant.
Assumption (H Hn ) means that the bandwidth collection should not be too large. Assumption (H ε ) is an integrability condition on the error. Bounded noise or Gaussian noise satisfy such kind of assumption.

Theoretical results.

Theorem 1. essume @H K AD @H m AD @H Hn A @@H Hn,1 A nd @H Hn,2 AA nd @H ε AF here exist two onstnts c, C > 0 depending on c

K D C K D C 0 D C m D C ε D h max D κ suh tht (9) E m(x 0 ) -m(x 0 ) 2 ≤ c min h∈Hn h 2β + σ 2 ln(n) nϕ x 0 (h) + C n .
This result proves that our selection rule (5) which permits to dene m(x 0 ) really mimics the oracle one (4). The selected estimator performs as well as the pseudo-estimator m h * (x 0 ) . The right-hand-side of (9) actually corresponds to the best compromise between the bias and the variance terms of the risk, since the term C/n is negligible with respect to the rst term of the right-hand-side of (9). Our procedure is thus adaptive, the selected estimator automatically adapts itself to the unknown smoothness of the target function m. We just have a logarithmic loss in the rate, by comparing the right-hand-side of ( 9) with (3). The loss is due to adaptation, and is known to be nevertheless adaptive optimal in many estimation problem studied with a pointwise risk. Notice also that the same phenomenon also occured to estimate the conditional cumulative distribution function in the same functional context, but with an integrated criterion, see [START_REF] Chagny | Adaptive and minimax estimation of the cumulative distribution function given a functional covariate[END_REF]. We see below that it does not aect the convergence rates in most of the cases.

Remark 1. The estimated variance term V (h, x 0 ) (see ( 7)) of the selection criterion cannot be used in practice, since it depends on the unknown noise variance

σ 2 = E[ε 2 1 ].
A solution consists in replacing it by an estimator, and to prove that the estimator selected with the new criterion still satises the adaptation property. The plug-in does not lead to specic diculties, and an upper-bound for the risk similar to (9) can be easily proved. We do not give all the theoretical details, since similar results and proofs can be found in Brunel and Comte (2005, Theorem 3.4) (for classical model selection method) and Chagny and Lacour (2014, Theorem 3) (for selection criterion in the spirit of Lepski's methodology). However, all the practical explanations can be found in Section 4.1.2.

3.4. Rates of convergence. Using the results of Theorem 1, we compute the rate of decrease of the risk of the selected estimator. Since the the variance term in the upperbound on the risk (Inequality (9)) depends on the small ball probability ϕ x 0 (h), the resulting rate will be related to its rate of decay when h → 0.

We proceed as in [START_REF] Chagny | Adaptive and minimax estimation of the cumulative distribution function given a functional covariate[END_REF]. Let us dene three classes of processes (we denote by c 1 , C 1 and c 2 some nonnegative constants): H X,L There exist some constants γ 1 , γ 2 ∈ R, and α > 0 such that

c 1 h γ 1 exp(-c 2 h -α ) ≤ ϕ x 0 (h) ≤ C 1 h γ 2 exp(-c 2 h -α ); H X,M There exist some constants γ 1 , γ 2 ∈ R, and α > 1, such that c 1 h γ 1 exp(-c 2 ln α (1/h)) ≤ ϕ x 0 (h) ≤ C 1 h γ 2 exp(-c 2 ln α (1/h)); H X,F There exists a constant γ > 0, such that c 1 h γ ≤ ϕ x 0 (h) ≤ C 1 h γ .
The class H X,F is typically the class of nite-dimensional processes. We can see this with the Karhunen-Loève decomposition of X,

(10) X = j≥1 λ j ξ j ψ j ,
which is simply the decomposition of X in the basis (ψ j ) j≥1 of the eigenfunctions of the covariance operator Γ :

f → E [ f, X X].
Here, (ξ j ) j≥1 is a sequence of uncorrelated centred standard random variables (the standardised principal component scores) and (λ j ) j≥1 is the sequence of eigenvalues associated to the eigenfunctions (ψ j ) j≥1 . The series (10) converges in terms of the norm • of H. Typically, if there is only a nite number of eigenvalues (λ j ) j≥1 which are non null, under mild assumptions on the law of X, we can prove that H X,F is veried and, in that case, X lies a.s. in the nite-dimensional space span{ψ j , λ j > 0} (Chagny and Roche, 2014) (note that, since Γ is a positive operator, λ j ≥ 0 for all j ≥ 1). Conversely, Mas (2012, Corollary 1, p. 10) proved that, under general conditions satised e.g. by Gaussian processes, the variable X can not veries Assumption H X,F if the set {j, λ j > 0} is innite.

A process X belongs to the class H X,M typically when the eigenvalues (λ j ) j≥1 decrease at an exponential rate. For instance, in the case of a Gaussian process with c exp(-2j)/j ≤ λ j ≤ C exp(-2j)/j, we have c 2 = 1/2 and α = 2 in H X,M (Homann-Jørgensen et al. 1979, Theorem 4.4 and example 4.7, pp. 333 and 336).

Finally, the class H X,L contains the processes such that the eigenvalues (λ j ) j≥1 decrease at a polynomial rate (Homann-Jørgensen et al., 1979, Theorem 4.4 and example 4.5, pp. 333 and 334). This is the case for instance for the Brownian motion (Ash and Gardner, 1975, pp. 41 and 42) or the Brownian bridge [START_REF] Macneill | Properties of sequences of partial sums of polynomial regression residuals with applications to tests for change of regression at unknown times[END_REF].

Proposition 2. uppose tht ll the ssumptions of heorem I re veri(edF he(neD for

β ∈]0, 1] nd C > 0D F C β e the following funtionl lssesX F C β = m : H → R, |m(x) -m(x )| ≤ C x -x β . • sf essumption H X,L is veri(edD then sup m∈F C β E m(x 0 ) -m(x 0 ) 2 ≤ C 1 (ln n) -β/α . • sf essumption H X,M is veri(edD then sup m∈F C β E m(x 0 ) -m(x 0 ) 2 ≤ C 2 exp - 2β c 1/α 2 ln 1/α (n) . • sf essumption H X,F is veri(edD then sup m∈F C β E m(x 0 ) -m(x 0 ) 2 ≤ C 3 ln n n 2β/(2β+γ) . rereD C 1 D C 2 nd C 3 re positive rel numers independent of nF
Since the bias-variance decomposition is very similar for the regression function estimation (Inequality ( 9)) and for the conditional cumulative distribution function estimation, we can easily adapt the proof of Chagny and Roche (2014, Section A.1.1) to prove Proposition 2.

The rates of decay of the risk are quite slow. Mas (2012, Theorem 3 and Corollary 2) has obtained very similar minimax convergence rates with dierent assumptions. Moreover, although slow, these rates are proved to be the optimal for conditional cumulative distribution function estimation in [START_REF] Chagny | Adaptive and minimax estimation of the cumulative distribution function given a functional covariate[END_REF] under the same assumptions on the small ball probability.

Numerical results

Our aim in this section is to illustrate the behaviour of our bandwidth selection device studied above and to compare its practical performances with cross-validation methods, to select a kernel estimator of the regression function. We rst explain how to implement our procedure, and detail specically how to tune the constant appearing in the term V (h, x 0 ) (which can be compared to a penalty constant).

4.1. Implementation of the estimator. For the simulation study, we choose the uniform kernel K(t) = 1 [0,1[ , and the following bandwidth collection

H n := {h max /k, k = 1, . . . , k max } . The norm • is the usual norm of L 2 ([0, 1]): f 2 := 1 0 f (t) 2 dt, for all f ∈ L 2 ([0, 1]).
With the denition of the kernel K, we see that, if h > max { X i -x 0 , i = 1, . . . , n}, K h ( X i -x 0 ) = 1. Hence, there is no need to consider bandwidths which are larger than max { X i -x 0 , i = 1, . . . , n}: we thus set h max = max { X i -x 0 , i = 1, . . . , n}. Note that a bandwidth collection depending on max { X i -x 0 , i = 1, . . . , n} is also considered in [START_REF] Amiri | Recursive estimation of nonparametric regression with functional covariate[END_REF]. Similarly, if h is taken too small, there are too few observations X i in the ball {x ∈ H, x -x 0 < h} and the variance of the estimator m h (x 0 ) is large.

To prevent this, we x k max such that ϕ x 0 (h max /k max ) ≥ ln n/n or, worded in a dierent way, k max is the largest integer such that h max /k max is greater than the empirical quantile of { X i -x 0 } i=1,...,n associated to the probability ln n/n. 4.1.1. glirtion of the onstnt ppering in the term V (h, x 0 )F The constant κ appearing in the term V (h, x 0 ) is a universal constant in the sense that it does not depend on the model parameters or on the estimation parameters. Hence, we must x it once and for all.

The method we propose is to evaluate the performances of our estimator on a grid of dierent values of κ, for dierent sets of parameters, and to keep the value of κ for which our estimator seems to have a reasonable mean squared error 1

M SE := E m(x 0 ) -m(x 0 ) 2 .
This calibration method is quite classical (see for instance [START_REF] Bertin | Adaptive estimation of conditional density function[END_REF][START_REF] Comte | Adaptive functional linear regression[END_REF].

The functional covariate is simulated in the following way:

X(t) := ξ 0 + J j=1 ξ j λ j ψ j (t),
where, for all j ≥ 1, ψ j (t) := √ 2 sin(π(j -0.5)t), (λ j ) j≥1 is a sequence of positive real numbers such that j≥1 λ j < +∞ and (ξ j ) j≥0 is an iFiFdF sequence of standard random variables and J is a positive integer. In the sequel, we consider three dierent settings:

(a) J = 150 and λ j = j -2 for all j ≥ 1. Here, J is suciently large to consider that

J j=1 ξ j λ j ψ j (t) ≈ j≥1 ξ j λ j ψ j (t).
Moreover, the choice of the sequence (λ j ) j≥1 , ensures that j≥1 ξ j λ j ψ j (t) veries H X,L .

1 Note that the optimal value of κ may vary in practice from one set of parameters to another. Then the idea is to realise a compromise between all the models and to take a reasonable value of the constant. (b) J = 150 and λ j = e -j /j for all j ≥ 1 which corresponds to a process verifying H X,M . (c) J = 2 and λ j = j -2 for all j ≥ 1 which corresponds to a process verifying H X,F . We consider also three regression models: (Ferraty and Vieu, 2002). Model 3: Y = 1 0 X(t) 2 dt + ε [START_REF] Amiri | Recursive estimation of nonparametric regression with functional covariate[END_REF]. For the three models, we set ε ∼ N (0, σ 2 ) with σ 2 = 0.01.

Model 1 : Y = 1 0 sin(4πt)X(t)dt
Figure 1 represents the empirical mean of the error as a function of κ. We choose the value κ = 0.1 for which the mean squared error of the estimator seems to be reasonable for all the chosen sets of parameters. 4.1.2. istimtion of the noise vrine σ 2 F Recall that our bandwidth selection criterion (5) depends on the noise variance σ 2 , see details in Remark 1. In the simulation study, we rst consider that the noise variance is known. However, in practice, we rarely know its value. Hence, we propose a plug-in estimator

σ 2 := 1 n n i=1 (Y i -m h min (X i )) 2 .
The idea beyond this proposition is the following: we want to propose an estimator which is as close as possible of the quantity n i=1 ε 2 i /n (we recall that the ε i 's are not observed). We have

1 n n i=1 ε 2 i = 1 n n i=1 (Y i -m(X i )) 2 ,
and we replace m(X i ) by an estimator m h (X i ) with a bandwidth h ∈ H n . A numerical study then allows us to establish that the more h is small, the more the quantity

n i=1 ε 2 i /n = n i=1 (Y i -m h (X i )) 2 /n is close to σ 2 .
The estimator m h(x 0 ) (x 0 ) with the bandwidth selected by the criterion (5) with σ 2 replaced by σ 2 is denoted m (ev) (x 0 ). Note that, in the simulation study, the replacement of σ 2 by the plug-in estimator σ 2 does not inuence signicantly the performances of the estimator (see Table 1 and Figures 2 (a) and 2 (b)).

Y (j) = m(X (j) i ). Figure (b): Y (j) = m (ve) (X (j) i ).

Simulation results.

We study here the behaviour of our estimator in two settings with Model 3, presented in Subsection 4.1.1:

First setting: The covariate X is simulated with the framework (b) (see Subsection 4.1.1) and the noise variance is xed to σ 2 = 0.2. Second setting: X is a standard Brownian motion and σ 2 = 0.01. We made a Monte-Carlo study over 50 replications of the sample (X i , Y i ) 1≤i≤n and the curve x 0 = X n+1 . We compare the performances of our selected kernel estimators m(x 0 ) and m (ev) (x 0 ) (bandwidth selected with the method introduced above in the spirit of Goldenshluger and Lepski 2011), with the same kernel estimator with bandwidth selected by leave-one-out and 10-fold cross-validations (CV). Both cross-validated methods consist in splitting the sample in two sub-samples (a learning sample and a test sample), evaluating the estimator at the points x 0 = X i for all X i in the test sample using only the data of the learning sample and comparing the predicted values Y (-ts) i of the Y i 's of the test sample with their real values. This procedure is repeated n times for the leave-one-out CV (the test sample contains only one observation) and 10 times for the 10-fold CV (the test sample contains about n/10 observations), in such a way that each observation is in the test sample only one time during the procedure. Finally, the selected bandwidth is the one which minimizes the mean squared prediction error : 2 presents the plot of the predicted values versus the observed values. For the four bandwidth selection devices, the prediction for a new value x 0 is quite accurate for most of the 50 evaluations of the estimators. Some values m(x 0 ) are badly predicted: these are isolated values, which means that only a very small number of curves X i 's in the sample are close to x 0 . This kind of isolated values is hard to estimate with kernel methods (which are local). Few dierences are visible between the dierent bandwidth selection devices. 4.2.2. wen qured irror @wiAF Table 1 presents the results of the Monte-Carlo study of the MSE of the fourth bandwidth selection devices presented here. We also give an estimation of the signal-to-noise ratio snr := σ 2 /Var(m(X)).

n i=1 Y i -Y (-ts) i 2 /n.

redited vluesF Figure

We rst note that, as expected, the mean squared prediction error decreases when the sample size increases. Then, remark that the performances, in terms of mean squared error, of our bandwidth selection devices (with known or estimated noise variance), are very similar to the performances of cross-validated methods. This conrms the rst ndings we made in view of Figure 2. 2 presents the mean time necessary to calculate the estimator and to select the bandwidth. We give here the total CPU time (system time and user time) necessary to compute an estimate of the value of m at a point x 0 . We performed these computations on a personal computer equipped with a processor Intel Core i5-4200, CPU: 1.60GHz (maximum speed: 2.30GHz), HD: 921Go, Memory: 7.88Go.

gomputtionl timeF Table

We observe that our bandwidth selection device outperforms cross-validation in terms of computation time: even with the estimation of the noise variance, which is time consuming, the computational time of the leave-one-out cross-validation is ranging from about 60 to 420 times that of our bandwidth selection device (50 to 370 times for the 10-fold crossvalidation). These dierences are due to the fact that cross-validated methods require the calculation of the estimator for each bandwidth and each curve X i of the sample (n × |H n | calls to the estimation function, where |H n | is the cardinality of the bandwidth collection H n ) while our method necessitates only one evaluation of the estimator per bandwidth (plus n evaluation of the estimate but for only one bandwidth for the calculation of the noise variance, giving a total of n + |H n | calls to the estimation function).

Note that the data-driven collection of bandwidths is larger (most of the time) for the simulation setting (b) than it is when X is a standard brownian motion (the small ball probability decreases faster to 0 when h goes to 0). This explains the observed dierences of computational time between the two settings. 5.1.1. he fernstein snequlitiesF Let us recall concentration results for empirical process, on which our proofs are based. We shall rst state the following useful lemma, which immediatly follows from (Birgé and Massart, 1998, p.366).

Lemma 1. @fernstein snequlityA vet T 1 , T 2 , . . . , T n e independent rndom vriles nd

S n (T ) = n i=1 (T i -E[T i ])F essume tht r(T 1 ) ≤ v 2 nd ∀l ≥ 2, 1 n n i=1 E |T i | l ≤ l! 2 v 2 b l-2 0 . henD for η > 0D P 1 n |S n (T )| ≥ η ≤ 2 exp - nη 2 /2 v 2 + b 0 η , ≤ 2 max exp - nη 2 4v 2 , exp - nη 4b 0 . ( 11
)
We deduce from (11) the following result, the proof of which is based on integration and is omitted.

Lemma 2. ith the sme nottions nd under the sme ssumptions s the ones of vemm ID for ny V > 0D

E S n (T ) n 2 -V + ≤ 2 max exp -n √ V 4b 0 32b 2 0 n 2 + 8b 0 √ V n , exp -nV 4v 2 4v 2 n .
5.1.2. he empiril proessF We apply in this section the previous concentration inequalities to bound the main empirical process involved in our context. We rst remark that Assumption (H K ) implies that, for all integer l > 0,

(12) c l K ϕ x 0 (h) ≤ E (K h ( X -x 0 )) l ≤ C l K ϕ x 0 (h).
One of the keys of the results is the control, in probability and expectation, of the quantity R x 0 h dened by:

(13) R x 0 h := 1 n n i=1 K h ( X i -x 0 ) E [K h ( X i -x 0 )]
.

The following lemma states the bounds.

Lemma 3. upposed tht @H K A is ful(lledF henD the following inequlity holdsD ( 14)

P |R x 0 h -1| > 1 2 ≤ 2 exp   - nϕ x 0 (h) 8 C 2 K c 2 K + C K 2c K   .
woreover if @H Hn,2 A is lso stis(edD for ll α > 0D

(15) E (R x 0 h -1) 2 -V R x 0 (h) + ≤ C n α , with V R x 0 (h) := κ R ln(n)/(nϕ x 0 (h))D κ R > max 4C K α c K √ C 0 2 , 4 C 2 K c 2 K α nd C > 0 depending only on C K D c K D C 0 nd κ R F Inequality (14) follows from Lemma 1, with T i = K h ( X i -x 0 )/E[K h ( X i -x 0 )] (E[T i ] = 1), which leads to S n (T )/n = R x 0 h -1.
The assumptions are fullled, with the parameters (computed mainly thanks to (12)),

v 2 = C 2 K c 2 k 1 nϕ x 0 (h) and b 0 = C K c k 1 nϕ x 0 (h)
.

Lemma 2 applied with the same process permits then to obtain Inequality (15).

Proof of Theorem 1.

A cornerstone decomposition for our main proofs is the following, for h ∈ H n ,

( m h (x 0 ) -m(x 0 )) 2 = n i=1 W (i) h (x 0 )Y i -m(x 0 ) 2 = n i=1 W (i) h (x 0 )m(X i ) + n i=1 W (i) h (x 0 )ε i -m(x 0 ) 2 = B h (x 0 ) + T h (x 0 ) (16) with (17) B h (x 0 ) = n i=1 W (i) h (x 0 ) (m(X i ) -m(x 0 )) 2 , T h (x 0 ) = n i=1 W (i) h (x 0 )ε i 2 .
We recall that n i=1 W

h (x 0 ) = 1 and that the independence of the ε i 's with the X i 's implies

E n i,j=1 W (i) h (x 0 )W (j) h (x 0 ) (m(X i ) -m(x 0 )) ε j = 0, which proves (16).
We rst bound the term B h (x 0 ). By Assumption (H m ):

B h (x 0 ) ≤ n i=1 W (i) h (x 0 ) |m(X i ) -m(x 0 )| 2 ≤ n i=1 W (i) h (x 0 )C m X i -x 0 β 2 . ( 18
)
Now we can remark that, by denition of W (i) h (x 0 ) and the fact that K is supported by [0, 1] (Assumption (H K )), we have

W (i) h (x 0 ) = 0 if X i -x 0 > h. This and Inequality (18) implies that (19) B h (x 0 ) ≤ C 2 m h 2β n i=1 W (i) h (x 0 ) 2 = C 2 m h 2β .
We turn now to the term T h (x 0 ):

E [T h (x 0 )] = E   n i=1 W (i) h (x 0 )ε i 2 1 {R x 0 h <1/2}   +E   n i=1 W (i) h (x 0 )ε i 2 1 {R x 0 h ≥1/2}   . (20)
For the rst term of Equation ( 20), remark that, by independence of ε i with ε j (for i = j) and X 1 , . . . , X n , and since ε is centered,

E   n i=1 W (i) h (x 0 )ε i 2 1 {R x 0 h <1/2}   = n i,j=1 E W (i) h (x 0 )W (j) h (x 0 )ε i ε j 1 {R x 0 h <1/2} = σ 2 n i=1 E W (i) h (x 0 ) 2 1 {R x 0 h <1/2} . Now we have, since the quantities (K h ( X i -x 0 )) 1≤i≤n are non-negative (Assump- tion (H K )), n i=1 W (i) h (x 0 ) 2 = n i=1 K 2 h ( X i -x 0 ) ( n i=1 K h ( X i -x 0 )) 2 ≤ 1 a.s.
This implies that

E   n i=1 W (i) h (x 0 )ε i 2 1 {R x 0 h <1/2}   ≤ σ 2 P (R x 0 h < 1/2) ≤ σ 2 P (|R x 0 h -1| > 1/2) .
Hence, by Lemma 3, we obtain

(21) E   n i=1 W (i) h (x 0 )ε i 2 1 {R x 0 h <1/2}   ≤ 2σ 2 exp   - nϕ x 0 (h) 8 C 2 K c 2 K + C K c K   ≤ Cσ 2 nϕ x 0 (h)
,

where C = 16e -1 C 2 K c 2 K + C K c K
by the fact that xe -x ≤ e -1 for all x > 0.

We turn now to the second term of Equation ( 20). By denition of

W (i) h (x 0 ) and R x 0 h E   n i=1 W (i) h (x 0 )ε i 2 1 {R x 0 h ≥1/2}   = E   n i=1 K h ( X i -x 0 ) nE [K h ( X -x 0 )] 1 R x 0 h ε i 2 1 {R x 0 h ≥1/2}   , ≤ 4E   n i=1 K h ( X i -x 0 ) nE [K h ( X -x 0 )] ε i 2   .
With the same arguments as the ones used for the rst term of ( 20),

E   n i=1 W (i) h (x 0 )ε i 2 1 {R x 0 h ≥1/2}   ≤ 4 n 2 (E [K h ( X -x 0 )]) 2 E n i=1 K 2 h ( X i -x 0 ) ε 2 i , = 4 n E K h ( X -x 0 ) E [K h ( X -x 0 )] ε 2 .
We conclude with Equation ( 12):

E   n i=1 W (i) h (x 0 )ε i 2 1 {R x 0 h ≥1/2}   ≤ 4C 2 K c 2 K σ 2 nϕ x 0 (h) . ( 22
)
Now Equation ( 16) and equations ( 19), ( 21) and ( 22) lead us to the expected result. 5.3. Proof of Theorem 1. We consider the set

Λ x 0 = h∈Hn ϕ x 0 (h) ϕ x 0 (h) -1 < 1 2 .
To prove Theorem 1, we study the loss function ( m(x 0 ) -m(x 0 )) 2 on the set Λ x 0 , and on its complementary (Λ x 0 ) c .

• Step 1. Upper bound for ( m(x 0 ) -m(x 0 )) 2 1 Λ x 0 .

Let h ∈ H n be a xed bandwidth. We rst split

m(x 0 ) -m(x 0 ) 2 ≤ 3 m h(x 0 ) (x 0 ) -m h(x 0 )∨h (x 0 ) 2 + 3 m h(x 0 )∨h (x 0 ) -m h (x 0 ) 2 +3 ( m h (x 0 ) -m(x 0 )) 2 .
We deduce from the denitions of A(h, x 0 ), A( h(x 0 ), x 0 ) and h(x 0 ) that

3 m h(x 0 ) (x 0 ) -m h(x 0 )∨h (x 0 ) 2 + 3 m h(x 0 )∨h (x 0 ) -m h (x 0 ) 2 ≤ 3 A(h, x 0 ) + V h(x 0 ), x 0 + 3 A h(x 0 ), x 0 + V (h, x 0 ) , ≤ 6 A (h, x 0 ) + V (h, x 0 ) . Thus, m(x 0 ) -m(x 0 ) 2 1 Λ x 0 ≤ 6 A(h, x 0 ) + 6 V (h, x 0 ) + 3 ( m h (x 0 ) -m(x 0 )) 2 1 Λ x 0 .
The idea is now to come down to the case of known small ball probability. To that aim, we dene

(23) V (h, x 0 ) = 2 3 κσ 2 ln(n) nϕ x 0 (h) , A(h, x 0 ) = max h ∈Hn ( m h (x 0 ) -m h∨h (x 0 )) 2 -V (h , x 0 ) + .
Compared to the data-driven counterparts ( 7) and ( 8), the variance term V (h, x 0 ) is deterministic here. We then split

A(h, x 0 ) = max h ∈Hn, V (h ,x 0 )<∞ ( m h∨h (x 0 ) -m h (x 0 )) 2 -V (h , x 0 ) + , ≤ max h ∈Hn, V (h ,x 0 )<∞ ( m h∨h (x 0 ) -m h (x 0 )) 2 -V (h , x 0 ) + + max h ∈Hn, V (h ,x 0 )<∞ V (h , x 0 ) -V (h , x 0 ) + , ≤ A(h, x 0 ) + max h ∈Hn V (h , x 0 ) -V (h , x 0 ) + , which gives m(x 0 ) -m(x 0 ) 2 1 Λ x 0 ≤ 6A(h, x 0 ) + 6V (h, x 0 ) + 3 ( m h (x 0 ) -m(x 0 )) 2 +6 max h ∈Hn V (h , x 0 ) -V (h , x 0 ) + + 6 V (h, x 0 ) -V (h, x 0 ) 1 Λ x 0 . ( 24
)
But we have, on the set Λ x 0 , for any h ∈ H n , | ϕ x 0 (h ) -ϕ x 0 (h )| < ϕ x 0 (h )/2. In particular, we thus have ϕ x 0 (h )-ϕ x 0 (h ) < ϕ x 0 (h )/2, that is ϕ x 0 (h ) < (3/2)ϕ x 0 (h ). This proves that V (h , x 0 ) -V (h , x 0 ) < 0, and hence max h ∈Hn V (h , x 0 ) -V (h , x 0 )

+ = 0. Moreover, on Λ x 0 , V (h, x 0 ) -V (h, x 0 ) = 2 3 κσ 2 ln(n) n 3 2 1 ϕ x 0 (h) - 1 ϕ x 0 (h) , ≤ 2 3 κσ 2 ln(n) n 2 1 ϕ x 0 (h) = 2V (h, x 0 ).
Gathering the two bounds in (24), and using that the expectation of the loss of the estimator m h in the right-hand-side of (24) has already been bounded (see Proposition 1) lead to

E m(x 0 ) -m(x 0 ) 2 1 Λ x 0 ≤ 6E [A (h, x 0 )] + 18V (h, x 0 ) + 3C h 2β + σ 2 nϕ x 0 (h) ,
where C is the constant involved in Equation (3).

The proof comes now down to establish an upper-bound for E[A(h, x 0 )]: we apply the following lemma (which proof is postponed to the following section).

Lemma 4. nder the ssumptions of heorem QD for ny h ∈ H n D there exists C > 0 suh tht

E [A (h, x 0 )] ≤ 2C 2 m h 2β + C n . he onstnt C depends on σ 2 , c K , C K , C 0 nd C ε F We thus obtain (25) E m(x 0 ) -m(x 0 ) 2 1 Λ x 0 ≤ c h 2β + σ 2 ln(n) nϕ x 0 (h) + C n ,
with c and C two constants, c depending on C m , σ 2 , c K and C K , and C depending on σ 2 , c K , C K , C 0 and C ε .

• Step 2. Upper bound for ( m(x 0 ) -m(x 0 )) 2 1 (Λ x 0 ) c . We introduce the terms B h (x 0 ) and T h (x 0 ) dened by (17):

m(x 0 ) -m(x 0 ) 2 1 (Λ x 0 ) c ≤ 2B h(x 0 ) (x 0 ) + 2T h(x 0 ) (x 0 ) 1 (Λ x 0 ) c . Next, thanks to (19), B h(x 0 ) (x 0 ) ≤ max h ∈Hn B h (x 0 ) ≤ max h ∈Hn 2C 2 m (h ) 2β . We deduce that E B h(x 0 ) (x 0 )1 (Λ x 0 ) c ≤ 2C 2 m h 2β max P ((Λ x 0 ) c ) . Moreover, E T h(x 0 ) (x 0 )1 (Λ x 0 ) c ≤ E max h ∈Hn T h (x 0 )1 (Λ x 0 ) c ≤ h ∈Hn E T h (x 0 )1 (Λ x 0 ) c , = h ∈Hn E n i=1 ε i K h ( X i -x 0 ) n i=1 K h ( X i -x 0 ) 2 1 (Λ x 0 ) c , = h ∈Hn E n i=1 ε 2 i K 2 h ( X i -x 0 ) ( n i=1 K h ( X i -x 0 )) 2 1 (Λ x 0 ) c , = σ 2 h ∈Hn E n i=1 K 2 h ( X i -x 0 ) ( n i=1 K h ( X i -x 0 )) 2 1 (Λ x 0 ) c , ≤ σ 2 h ∈Hn P ((Λ x 0 ) c ) ≤ σ 2 nP ((Λ x 0 ) c ) ,
where we have used the properties of ε i , X i and Assumption (H K ) in the same way as above, but also Assumption (H Hn,1 ). Therefore,

(26) E m(x 0 ) -m(x 0 ) 2 1 (Λ x 0 ) c ≤ 2C 2 m h 2β max + σ 2 n P ((Λ x 0 ) c ) .
We are reduced to bound P((Λ x 0 ) c ):

P((Λ x 0 ) c ) ≤ h∈Hn P 1 n n i=1 1 { X i -x 0 ≤h} -E 1 { X i -x 0 ≤h} ≥ ϕ x 0 (h) 2 .
We apply Bernstein's Inequality (Lemma 1), with T

i = 1 { X i -x 0 ≤h} and η = ϕ x 0 (h)/2. Since 0 ≤ T i ≤ 1, we set b 0 = 1, and v 2 = Var(T 1 ) = ϕ x 0 (h)(1 -ϕ x 0 (h)). Thus, P((Λ x 0 ) c ) ≤ 2 h∈Hn exp - n(ϕ x 0 (h)) 2 /8 ϕ x 0 (h)(1 -ϕ x 0 (h)) + ϕ(h)/2 = 2 h∈Hn exp - nϕ x 0 (h) 8(1 -ϕ x 0 (h)) + 4 ≤ 2 h∈Hn exp - nϕ x 0 (h) 12 .
This leads to P((Λ x 0 ) c ) ≤ 2n 1-C 0 /12 thanks to Assumptions (H Hn,1 ) and (H Hn,2 ). In (26), we obtain

E m(x 0 ) -m(x 0 ) 2 1 (Λ x 0 ) c ≤ 2 2C 2 m h 2β max + σ 2 n 2-C 0 /12 , (27) ≤ 2 2C 2 m h 2β max + σ 2 n -1 , (28)
as soon as C 0 > 36.

It remains to sum ( 25) and ( 27) to obtain Theorem 1. 5.4. Proof of Lemma 4. First remark that the denition (8) of A(h, x 0 ) can be written in the new way

A(h, x 0 ) = max h ∈Hn,h ≤h ( m h (x 0 ) -m h (x 0 )) 2 -V (h , x 0 ) + . For xed bandwidths h, h ∈ H n h ≥ h , let us introduce Ω x 0 h,h = {R x 0 h ≥ 1/2} ∩ {R x 0 h ≥ 1/2} with R x 0 dened by (13). We split, for h ≤ h, ( m h (x 0 ) -m h (x 0 )) 2 ≤ 2 ( m h (x 0 ) -m(x 0 )) 2 + 2 ( m h (x 0 ) -m(x 0 )) 2 ≤ 4 B h (x 0 ) + B h (x 0 ) + T h (x 0 )1 Ω x 0 h,h + T h (x 0 )1 Ω x 0 h,h +4T h (x 0 )1 (Ω x 0 h,h ) c + 4T h (x 0 )1 (Ω x 0 h,h ) c
, with B h (x 0 ) and T h (x 0 ) dened by ( 17) (the notations are valid with h or h ). We thus have the following decomposition, for any bandwidth h,

A(h, x 0 ) ≤ 4 max h ∈Hn,h ≤h B h (x 0 ) + B h (x 0 ) + 4 max h ∈Hn,h ≤h T h (x 0 )1 Ω x 0 h,h - V (h , x 0 ) 8 + +4 max h ∈Hn,h ≤h T h (x 0 )1 Ω x 0 h,h - V (h , x 0 ) 8 + + 4 max h ∈Hn,h ≤h (T h (x 0 ) + T h (x 0 )) 1 (Ω x 0 h,h ) c . Since V (h ) ≥ V (h) as soon as h ≤ h, max h ∈Hn,h ≤h T h (x 0 )1 Ω x 0 h,h - V (h , x 0 ) 8 + ≤ max h ∈Hn,h ≤h T h (x 0 )1 Ω x 0 h,h - V (h, x 0 ) 8 + , ≤ max h ∈Hn,h ≤h T h (x 0 )1 Ω x 0 h,h - V (h , x 0 ) 8 + .
Thus the splitting becomes

A(h, x 0 ) ≤ 4 max h ∈Hn,h ≤h B h (x 0 ) + B h (x 0 ) + 8 max h ∈Hn,h ≤h T h (x 0 )1 Ω x 0 h,h - V (h , x 0 ) 8 + +4 max h ∈Hn,h ≤h (T h (x 0 ) + T h (x 0 )) 1 (Ω x 0 h,h ) c . (29)
It remains to bound each of the terms. Two of them have not been centred: the terms involving B h (x 0 ) are bias terms, and the term involving 1 (Ω x 0 h,h ) c will be shown to be directly negligible. The last term, which depends on T h (x 0 ), is the more dicult term. We will control it by using Lemma 2.

• Upper-bound for the terms depending on B h (x 0 ). The term B h (x 0 ) is the rst term of the right-hand-side of ( 16) and has thus already been bounded (see ( 19)):

B h (x 0 ) ≤ C 2 m h 2β , for any bandwidth h. Thus, max h ∈Hn,h ≤h B h (x 0 ) + B h (x 0 ) ≤ max h ∈Hn,h ≤h C 2 m h 2β + C 2 m h 2β ≤ 2C 2 m h 2β . (30)
• Upper-bound for the term depending on 1 (Ω x 0 h,h ) c .

We roughly bound

max h ∈Hn,h ≤h (T h (x 0 ) + T h (x 0 )) 1 (Ω x 0 h,h ) c ≤ h ∈Hn T h (x 0 )1 (Ω x 0 h,h ) c + T h (x 0 )1 (Ω x 0 h,h ) c .
Moreover, by using the independence of X i and ε i and E[ε i ] = 0, we have

E T h (x 0 )1 (Ω x 0 h,h ) c = E ( n i=1 ε i K h ( X i -x 0 )) 2 ( n i=1 K h ( X i -x 0 )) 2 1 (Ω x 0 h,h ) c , = E n i=1 ε 2 i K 2 h ( X i -x 0 ) ( n i=1 K h ( X i -x 0 )) 2 1 (Ω x 0 h,h ) c ,
since the ε i 's are independent. Therefore, since the kernel is non-negative (Assumption (H K )), E T h (x 0 )1 (Ω x 0 h,h ) c ≤ σ 2 P (Ω x 0 h,h ) c . The inequality remains evidently valid with T h (x 0 ) replaced by T h (x 0 ), which leads to max h ∈Hn,h ≤h

(T h (x 0 ) + T h (x 0 )) 1 (Ω x 0 h,h ) c ≤ σ 2 h ∈Hn P (Ω x 0 h,h ) c .
Then, by applying twice Inequality (14) of Lemma 3,

P (Ω x 0 h,h ) c ≤ P |R x 0 h -1| > 1 2 + P |R x 0 h∨h -1| > 1 2 , ≤ 2    exp   - nϕ x 0 (h ) 8 C 2 K c 2 K + C K 2c K   + exp   - nϕ x 0 (h ∨ h ) 8 C 2 K c 2 K + C K 2c K      , ≤ 4n
- C 0 8 ( C 2 K /c 2 K +C K /2c K ) ,
since ϕ x 0 (h) ≥ C 0 ln(n)/n for any h ∈ H n (Assumption (H Hn,2 )). Then, as the cardinality of H n is bounded by n (Assumption (H Hn,1 )), we deduce

E max h ∈Hn,h ≤h (T h (x 0 ) + T h (x 0 )) 1 (Ω x 0 h,h ) c ≤ 4σ 2 n 1- C 0 8 ( C 2 K /c 2 K +C K /2c K ) ≤ 4σ 2 n , (31) if C 0 ≥ 16(C 2 K /c 2 K + C K /2c K ).
• Upper-bound for the terms depending on T h (x 0 ). First notice that

T h (x 0 )1 Ω x 0 h,h = n i=1 K h ( X i -x 0 ) nE[K h ( X i -x 0 ) 1 R x 0 h ε i 2 1 Ω x 0 h,h , ≤ 4 n i=1 K h ( X i -x 0 ) nE[K h ( X i -x 0 ) ε i 2 .
This implies that E max h ∈Hn,h ≤h

T h (x 0 )1 Ω x 0 h,h - V (h , x 0 ) 8 + ≤ h ∈Hn E      1 n n i=1 T i,h (x 0 ) -E [T i,h (x 0 )] 2 - V (h , x 0 ) 8    +   ,
with T i,h (x 0 ) = (K h ( X i -x 0 )/E[K h ( X i -x 0 ))ε i (we have used that E[T i,h (x 0 )] = 0, since ε i is centred, independent from X i ). Our aim is now to apply the Bernstein Inequality of Lemma 2. The random variables T i,h (x 0 ), i = 1, . . . , n are iFiFdF, and we compute the following parameters (see the assumptions of Lemma 1 for their denitions)

v 2 = C 2 ε C 2 K c 2 K 1 ϕ x 0 (h ) and b 0 = C ε C K c K 1 ϕ x 0 (h )
.

Their values are obtained by using (12) and Assumption (H ε ). Let V (h , x 0 ) = κ ln(n) nϕ x 0 (h ) , κ > 0. (32)

Lemma 2 shows that

h ∈Hn E      1 n n i=1 T i,h (x 0 ) -E [T i,h (x 0 )] 2 -V (h , x 0 )    +   ≤ A 1 (x 0 ) + A 2 (x 0 ) + A 3 (x 0 ), with A 1 (x 0 ) = 2 h ∈Hn 32b 2 0 n 2 exp   -n V (h , x 0 ) 4b 0   , A 2 (x 0 ) = 2 h ∈Hn 8b 0 V (h , x 0 ) n exp -n V (h , x 0 )4b 0 , A 3 (x 0 ) = 2 h ∈Hn 4v 2 n exp -n V (h , x 0 )4v 2 .
The strategy is now similar for each of these three terms: we use the denition of V and Assumption (H Hn ) (cardinality of H n bounded by n, and lower bound ϕ(h ) ≥ C 0 ln(n)/n, for any h ) to prove that the three terms have the order of magnitude O(1/n). More precisely, we prove that

A 1 (x 0 ) ≤ 64 C 2 K C 2 ε σ 2 c 2 k C 2 0 1 ln 2 (n) n 1- c K √ κC 0 4C K Cεσ , A 2 (x 0 ) ≤ 16 C K C ε σ √ κ c k C 3/2 0 1 ln 3/2 (n) n 1- c K √ κC 0 4C K Cεσ , A 3 (x 0 ) ≤ 8 C 2 K C 2 ε σ 2 C 0 c 2 k 1 ln(n) n 1- c 2 K κ 4C 2 K C 2 ε σ 2 . By choosing κ ≥ max(64C 2 K C 2 ε σ 2 /(C 0 c 2 K ), 8C 2 K C 2 ε σ 2 /c 2 K ), we nally obtain h ∈Hn E      1 n n i=1 T i,h (x 0 ) -E [T i,h (x 0 )] 2 -V (h , x 0 )    +   ≤ 64 C 2 K C 2 ε c 2 k C 2 0 + 6 C K C ε √ κ c k C 3/2 0 + 8 C 2 K C 2 ε C 0 c 2 k 1 n .
If 2κ/3 in the denition of V (h, x 0 ) (see ( 7)) is larger than 8κ, that is if κ ≥ 12 × 8C 2 ε C 2 K /c 2 K max(8/C 0 , 1), we also have V (h, x 0 )/8 ≥ V (h, x 0 ) and consequently, we have proved

E max h ∈Hn,h ≤h T h (x 0 )1 Ω x 0 h,h - V (h , x 0 ) 8 + ≤ C n , (33)
with C depending on c K , C K , C 0 , and C ε .

The proof of Lemma 4 is ended by gathering the inequalities (29), ( 30), (31), and (33).

Figure 1 .

 1 Figure 1. Plots of the empirical mean of the squared error calculated over 50 independent samples of size n = 500. Dotted lines: 95% condence interval on the mean squared error. Blue curves: process (a), green curves: process (b) and red curves : process (c).

  t)| ln(|X(t)|)dt + ε

Figure 2 .

 2 Figure 2. Plot of the predicted values Y (j) versus the observed values Y (j) . Figure (a): Y (j) = m(X

  Figures (c) and (d): predictions made with the estimator selected by leave-one-out cross-validation and 10-fold crossvalidation. Model 3 with X a standard Brownian motion, n = 500.

  X brownian motion n = 100 n = 200 n = 500 n = 100 n = 200 n = 500

  X brownian motion n = 100 n = 200 n = 500 n = 100 n = 200 n = 500 Known noise variance ( m(x 0 ))

Table 1 .

 1 Monte-Carlo study of the MSE over 50 replications.

	signal-to-noise ratio (snr)			0.055			0.016
	Known noise variance ( m(x 0 ))	0.669	0.387	0.082	0.155	0.115	0.032
	Est. noise variance ( m (ev)	(x 0 ))	0.669	0.392	0.084	0.155	0.115	0.032
	Leave-one-out CV		0.656	0.370	0.080	0.156	0.115	0.032
	10-fold CV		0.660	0.369	0.083	0.155	0.114	0.032

Table 2 .

 2 Mean CPU time in seconds (calculated over 50 runs) necessary for the calculation of the estimator (including bandwidth selection).

			0.04	0.11	0.35	0.03	0.06	0.17
	Est. noise variance ( m (ev)	(x 0 ))	0.39	1.50	9.45	0.33	1.24	7.95
	Leave-one-out CV		3.35	17.55	147.61	1.82	8.77	71.20
	10-fold CV		3.05	16.06	129.72	1.64	7.96	63.32