Fabienne Comte

Nicolas Marie

ON A PROJECTION ESTIMATOR OF THE REGRESSION FUNCTION DERIVATIVE

Keywords:

In this paper, we study the estimation of the derivative of a regression function in a standard univariate regression model. The estimators are defined either by derivating nonparametric least-squares estimators of the regression function or by estimating the projection of the derivative. We prove two simple risk bounds allowing to compare our estimators. More elaborate bounds under a stability assumption are then provided. Bases and spaces on which we can illustrate our assumptions and first results are both of compact or non compact type, and we discuss the rates reached by our estimators. They turn out to be optimal in the compact case. Lastly, we propose a model selection procedure and prove the associated risk bound. To consider bases with a non compact support makes the problem difficult.

Introduction

In this paper, we consider the random design regression model [START_REF] Abramowitz | Handbook of mathematical functions with formulas, graphs, and mathematical tables[END_REF] Y i = b(X i ) + ε i ; i ∈ {1, . . . , n}, n 1, where b(.) is the unknown continuously differentiable regression function, X 1 , . . . , X n are independent and identically distributed (i.i.d.) random variables with density f with respect to Lebesgue's measure, and the errors ε 1 , . . . , ε n are i.i.d, unobserved, centered with variance σ 2 , and independent of the X i 's. The observations are (X i , Y i ) 1 i n , and we assume that b is regular enough to admit a derivative. We are interested in nonparametric estimation of the derivative b of b, on a compact or a non-compact support.

1.1. Motivation and bibliographical elements. The question of nonparametric estimation of derivatives is not new and is studied in different contexts, such as density estimation or white noise model (see Efromovich [START_REF] Efromovich | Simultaneous sharp estimation of functions and their derivatives[END_REF]), and not only in regression. Indeed, there can be a lot of reasons for estimating not only a function but also its derivative, which may be of intrinsic interest as measure of slope for instance. Recently, Bercu et al. [START_REF] Bercu | Nonparametric recursive estimation of the derivative of the regression function with application to sea shores water quality[END_REF] studied this question in the concrete application setting of sea shores water quality. Precisely, they propose an estimator defined as the derivative of the well-known Nadaraya-Watson estimator. Dai et al. [START_REF] Dai | Optimal estimation of derivatives in nonparametric regression[END_REF] also mention applications to the modeling of human growth data (Ramsay and Silverman [START_REF] Ramsay | Applied Functional Data Analysis: Methods and Case Studies[END_REF]) or to Raman spectra of bulk materials (Charnigo et al. [START_REF] Charnigo | A generalized Cp criterion for derivative estimation[END_REF]).

Derivatives of a rate optimal estimate of the regression function are proved to be rate optimal estimates of the corresponding derivatives, see Stone [START_REF] Stone | Optimal rates of convergence for nonparametric estimators[END_REF][START_REF] Stone | Optimal Global Rates of Convergence for Nonparametric Regression[END_REF], who establishes optimal rates for local polynomial weighted estimators on a compact set. See also a discussion on the topic in Rice and Rosenblatt [START_REF] Rice | Smoothing splines: regression, derivatives and deconvolution[END_REF], for a fixed design model. Nonparametric estimation of the regression function derivative has been studied following different methods, relying on kernels, local polynomial regression, regression by smoothing splines, or difference quotients. We emphasize that the strategy for fixed design context, where X i are replaced by x i = i/n, relies on dedicated methods. Indeed, differences Y i -Y i-1 bring information on b , which is not the case for random design on non compact support. Kernel estimation of the regression function and its derivative is for instance studied by Gasser and Müller [START_REF] Gasser | Estimating regression functions and their derivatives by the kernel method[END_REF], in the fixed design case. In local polynomial regression, the derivative can be estimated by the coefficient of the derivative of the local polynomial regression fitted at given point x, as summarized in Tsybakov [START_REF] Tsybakov | Introduction to Nonparametric Estimation[END_REF], Chapter 1, see also Huang and Chan [START_REF] Huang | Local Polynomial and Penalized Trigonometric Series Regression[END_REF]. Stone [START_REF] Stone | Optimal Global Rates of Convergence for Nonparametric Regression[END_REF] showed that derivative estimation with splines can achieve the optimal L 2 -rate of convergence (proved in Stone [START_REF] Stone | Optimal rates of convergence for nonparametric estimators[END_REF]) under mild assumptions. Further asymptotic properties are obtained by Zhou and Wolfe [START_REF] Zhou | On derivative estimation in spline regression[END_REF], mainly in the fixed design setting on compact support: they rely on splines estimators, arguing that they avoid boundary problems of kernel estimators. Note that extensions to functional regressors have been conducted (see Hall et al. [START_REF] Hall | Estimation of functional derivatives[END_REF]). The smoothing parameter selection problem remained unanswered in the first papers. For kernel strategies, the bandwidth choice for the derivative estimator (based on a factor rule) is discussed in Fan and Gijbels [START_REF] Fan | Local polynomial modelling and its applications[END_REF], but not studied from theoretical point of view. Liu and Brabanter [START_REF] Liu | Smoothed nonparametric derivative estimation using weighted difference quotients[END_REF] propose a methodology which generalizes the difference quotient based estimator of the first order derivative to the random design setting, when X follows a uniform distribution on [0, 1]. They also discuss bandwidth selection in their setting. Lastly, we mention that an adaptive method in wavelet bases is studied in Chesneau [START_REF] Chesneau | A note on wavelet estimation of the derivatives of a regression function in a random design setting[END_REF], but it involves an estimate of f . As a consequence, the rate of estimation depends on the regularity of this function, which we want to avoid.

1.2. Contributions of the paper. In the present work, we consider a projection method and propose an estimator as a finite development in an orthonormal L 2 basis with m coefficients. We start from the least-square estimator studied by Baraud for the fixed design model in [START_REF] Baraud | Model selection for regression on a fixed design[END_REF] and the random design model in [START_REF] Baraud | Model selection for regression on a random design[END_REF]. These works consider compactly supported bases and assume that the density of the X i is lower bounded on the interval of estimation. The lower bound on the density is involved in the upper bound on the risk. These results have been extended to non compactly supported bases by Comte and Genon-Catalot [START_REF] Comte | Regression Function Estimation on Non-Compact Support as a Partly Inverse Problem[END_REF]; then, the assumption that the density is lower bounded can not be done, and the problem has to be handled differently. In some sense, regression function estimation in this setting has some characteristics of inverse problems.

Here, we show that two strategies can be considered to deduce from the least square estimator of b, an estimator of b (.), and these strategies do not coincide in general. We prove non asymptotic bounds on the integrated L 2 -risk of the estimators, for both strategies. The fact that our results are non asymptotic and global (and not pointwise), make them different from the literature mentioned previously. To our knowledge, these are the first results allowing for non compactly supported bases in the definition of the estimators. In the case of a trigonometric basis and compact support estimation, we recover the optimal rates given in Stone [START_REF] Stone | Optimal rates of convergence for nonparametric estimators[END_REF] under weak assumptions. We also obtain specific rates in the non compact Hermite basis setting. Therefore, our results contain previous ones, and extensions. Last but not least, we propose a model selection strategy relying on a Goldenshluger and Lepski [START_REF] Goldenshluger | Bandwidth selection in kernel density estimation: oracle inequalities and adaptive minimax optimality[END_REF] method and prove a risk bound for the final estimator: this result holds for sub-gaussian (bounded or gaussian) errors and implies that the estimator automatically reaches the optimal rate on regularity spaces, without requiring the knowledge of the regularity index of b. We discuss our assumptions, which remain rather weak.

The plan of the paper is the following. We define our notation and estimators in Section 2. In Section 3, we present our assumptions and prove two simple risk bounds allowing to compare our estimators. More elaborate bounds under a stability assumption (see Cohen et al. [START_REF] Cohen | On the Stability and Accuracy of Least Squares Approximations[END_REF][START_REF] Cohen | Correction to: On the stability and accuracy of least squares approximations[END_REF]) are also provided. Bases and spaces on which we can illustrate our assumptions and first results are described in Section 4. They are of compact (trigonometric basis) or non compact (HErmite basis) type, and we discuss the rates reached by our estimators. They are the optimal ones in the compact case. Section 5 is dedicated to the adaptive procedure: we prove a risk bound and deduce corollaries about adaptive rates. The possibility of non compact support makes the problem difficult, and even if the estimator seems to follow a standard Goldenshluger and Lepski [START_REF] Goldenshluger | Bandwidth selection in kernel density estimation: oracle inequalities and adaptive minimax optimality[END_REF] scheme, the proofs are delicate, due to an additional bias term. A numerical study shows that the collection of estimators contains relevant proposals and that the data driven estimator works in a satisfactory way, especially compared to the derivative of a Nadaraya-Watson estimator.

Definition of the estimators

Let B = (ϕ j ) j∈N\{0} be a Hilbert basis of L 2 (I, dx) with I R an interval. For the sake of readability, for every j ∈ N\{0}, the function x ∈ R → ϕ j (x)1 I (x) is also denoted by ϕ j . The following mean squares estimator of b I = b1 I , which is studied in Baraud [3] and in Comte and Genon-Catalot [START_REF] Comte | Regression Function Estimation on Non-Compact Support as a Partly Inverse Problem[END_REF], is defined by

b m (x) := m j=1 [ θ 1 m ] j ϕ j (x) ; x ∈ I,
where m ∈ N\{0},

θ 1 m = θ 1 m (X, Y) := 1 n Ψ -1 m Φ * m Y, M * denotes the transpose of M , Y := (Y 1 , . . . , Y n ) * , X := (X 1 , . . . , X n ) * , Φ m := (ϕ j (X i )) 1 i n,1 j m and Ψ m := 1 n Φ * m Φ m = ( ϕ j , ϕ k n ) 1 j,k m with ϕ, ψ n := 1 n n i=1 ϕ(X i )ψ(X i )
for every ϕ, ψ : R → R. The map (ϕ, ψ) → ϕ, ψ n is the empirical scalar product, and the associated norm is denoted by . n in the sequel. The theoretical analogue on

L 2 (R, f (x)dx) is (ϕ, ψ) -→ ϕ, ψ f := I ϕ(z)ψ(z)f (z)dz,
and the associated norm is denoted by . f . Notice that E( ϕ, ψ n ) = ϕ, ψ f . The reader can refer to Baraud [START_REF] Baraud | Model selection for regression on a fixed design[END_REF][START_REF] Baraud | Model selection for regression on a random design[END_REF], Cohen et al. [START_REF] Cohen | On the Stability and Accuracy of Least Squares Approximations[END_REF][START_REF] Cohen | Correction to: On the stability and accuracy of least squares approximations[END_REF], and Comte and Genon-Catalot [START_REF] Comte | Regression Function Estimation on Non-Compact Support as a Partly Inverse Problem[END_REF] for risk bounds on b m and an adaptive estimator.

Strategy 1. On the one hand, a natural estimator of b I is

(2) b ,1 m (x) := m j=1 [ θ 1 m ] j ϕ j (x)
with m ∈ N\{0}. Obviously,

( b ,1 m (X 1 ), . . . , b ,1 m (X n )) * = Φ m θ 1 m = 1 n Φ m Ψ -1 m Φ * m Y
with Φ m := (ϕ j (X i )) i,j . This requires to choose a regular basis. Note that, contrary to what may occur for the density estimator, this way is simpler than derivating the Nadaraya-Watson kernel based estimator as done in Bercu et al. [START_REF] Bercu | Nonparametric recursive estimation of the derivative of the regression function with application to sea shores water quality[END_REF]. Indeed, the latter involves the derivative of a quotient of two functions.

Strategy 2. On the other hand, when (bϕ j )(inf(I)) = (bϕ j )(sup(I)) for every j ∈ {1, . . . , m}, b , ϕ j = -b, ϕ j and the orthogonal projection

(b ) m of b on S m := span{ϕ 1 , . . . , ϕ m } in L 2 (I, dx) satisfies (b ) m (x) = - m j=1 b, ϕ j ϕ j (x).
Several of the basis we have in mind are such that the derivative of ϕ j can be expressed as a finite linear combination of the other ϕ k 's. Thus, if there exist (known) coefficients

d j,k such that ϕ j = m k=1 d j,k ϕ k , then b, ϕ j = m k=0 d j,k b, ϕ k .
A simple plug-in strategy leads thus to propose an estimate of b, ϕ j by replacing b, ϕ k in the above formula by

[ θ 1 m ] k . In other words, if there exists ∆ m,m+p ∈ M m,m+p (R) such that Φ m = Φ m+p ∆ *
m,m+p , one can consider a projection estimator of the derivative instead of derivating the projection estimator of b:

(3) b ,2 m (x) := m j=1 [ θ 2 m ] j ϕ j (x) with θ 2 m = - 1 n ∆ m,m+p Ψ -1 m+p Φ * m+p Y. Obviously, ( b ,2 m (X 1 ), . . . , b ,2 m (X n )) * = - 1 n Φ m+p ∆ m,m+p Ψ -1 m+p Φ * m+p Y.
We shall see in this paper that the two strategies are different and we will provide risk bounds that allow to compare the two methods.

Risk bounds

Notations and useful elementary properties:

• The operator norm of a matrix M is defined by M 2 op := λ max (M M * ), where we recall that M * is the transpose of M and λ max (M M * ) is the largest eigenvalue of the square matrix M M * , which are nonnegative. Note that for a square, symmetric and nonnegative matrix A, A op = λ max (A). Note also that if A and B are two matrices such that AB and BA are well defined, then λ max (AB) = λ max (BA). Finally, note that if A and B are two square, symmetric and nonnegative matrices, then Tr(AB)

A op Tr(B) = λ max (A)Tr(B), where Tr(M ) denotes the trace of a (square) matrix M .

• The Frobenius norm of a matrix M is defined by

M 2 F := Tr(M M * ) = Tr(M * M ).
• The natural scalar product on L 2 (I, f (x)dx), also called f -weighted scalar product, is denoted by ., . f , and the associated norm by . f . • For every ψ ∈ L 2 (I, dx), its orthogonal projection on

S m = {ϕ 1 , . . . , ϕ m } in L 2 (I, dx) is denoted by ψ m .
3.2. Preliminary rough risk bounds on b ,1 m and b ,2 m . In the sequel, we assume that b exists and is square integrable on I, and that the density function f fulfills the following assumption. First, we provide the following rough but general risk bound on b ,1 m . Proposition 3.2. Under Assumption 3.1,

E b ,1 m -b 2 n 3 f ∞ inf t∈Sm t -b 2 + 3E Φ m ( Φ * m Φ m ) -1 Φ * m 2 op b -b m 2 n + 3 b m -(b ) m 2 f + σ 2 n E Tr ( Φ * m Φ m ) -1 ( Φ m ) * Φ m ,
where b m is the L 2 (I, dx)-orthogonal projection of b on S m , and b m is its derivative, while (b ) m is the

L 2 (I, dx)-orthogonal projection of b on S m .
Let us comment the four terms in the previous bound:

(1) The first term is the bias term we could expect. It can be evaluated on regularity spaces. Without Assumption 3.1, this terms can be replaced by inf t∈Sm t -b 2 f . (2) The second term involves the bias related to b, which would be negligible compared to the previous one; but it is multiplied by a coefficient which has an order depending on m and will at least compensate the improvement. (3) The third term can be evaluated in the different bases: the procedure makes sense if the derivative of the projection and the projection of the derivative are close, for fixed m. Under Assumption 3.1, it is less than

3 f ∞ b m -(b ) m 2
, null in trigonometric spaces with odd dimensions, and of order less or equal than the first term in Laguerre or Hermite bases (see Proposition 4.1). ( 4) The last term is the variance term, and it is established in Proposition 3.3 that it increases with m as expected.

Proposition 3.3. The map m → E Tr ( Φ * m Φ m ) -1 ( Φ m ) * Φ m is increasing.
To sum up, we will have to make a compromise between decreasing bias term (1) and increasing variance term (4), with the specific difficulty related to nuisance terms (2) and [START_REF] Baraud | Model selection for regression on a random design[END_REF]. Now, we turn to the estimator b ,2 m and assume that there exists p ∈ N such that ϕ 1 , . . . , ϕ m fulfill the following assumption: Assumption 3.4 (m, p). For every j ∈ {1, . . . , m}, ϕ j ∈ S j+p .

Note that Assumption 3.4(m, p) implies that there exists ∆ m,m+p ∈ M m,m+p (R) such that [START_REF] Bercu | Nonparametric recursive estimation of the derivative of the regression function with application to sea shores water quality[END_REF] Φ m = Φ m+p ∆ * m,m+p . Trigonometric, Laguerre, Hermite and Legendre bases satisfy Assumption 3.4(m, p), see Section 4. More precisely, we have p = 0 for the Laguerre and Legendre bases and ∆ m,m is a lower triangular square matrix. We have p = 1 for Hermite and trigonometric bases with ∆ m,m+1 (j, k) = 0 for k j + p. For the trigonometric basis with an odd dimension, we can keep a square link ∆ m,m with a null first line followed by diagonal 2 × 2 blocks of type 0 -2πj 2πj 0 .

Assume that b and ϕ 1 , . . . , ϕ m fulfill also the following assumption. f is likely to be large, so the second strategy should be preferred.

E b ,2 m -b 2 n 2 f ∞ inf t∈Sm t -b 2 + 2E Φ m ∆ m,m+p ( Φ * m+p Φ m+p ) -1 Φ * m+p 2 op b -b m+p 2 n + σ 2 n E Tr ( Φ * m+p Φ m+p ) -1 ∆ * m,m+p Φ * m Φ m ∆ m,m+p .
Ψ m := E( Ψ m ) = ( ϕ j , ϕ k f ) j,k .
Assume also that it fulfills the following assumption called "stability assumption" by Cohen et al. [START_REF] Cohen | On the Stability and Accuracy of Least Squares Approximations[END_REF].

Λ m := L(m)( Ψ -1 m op ∨ 1) c n log(n) .
Then, let us establish elaborate risk bounds on b ,1 m and b ,2 m . Proposition 3.8. Under Assumptions 3.1, 3.4(m, p) and 3.

7(m+p), if E[b (X 1 ) 4 ] < ∞ and E(Y 4 1 ) < ∞, then E b ,1 m -b 2 n 3 f ∞ inf t∈Sm t -b 2 + 9 ∆ f,1 m,m+p 2 op b -b m 2 f + 3 b m -(b ) m 2 f + 2σ 2 n ∆ f,1 m,m+p 2 F + 2cn log(n) ∆ m,m+p 2 op E(Y 4 1 ) 1/2 + 3E(b (X 1 ) 4 ) 1/2 c 1/2 8.1 n 4 with ∆ f,1 m,m+p := Ψ 1/2 m+p ∆ * m,m+p Ψ -1/2 m
and ∆ m,m+p is defined in (4).

Proposition 3.9. Under Assumptions 3.1, 3.4(m, p), 3.5(m) and 3.7

(m + p), if E[b (X 1 ) 4 ] < ∞ and E(Y 4 1 ) < ∞, then E b ,2 m -b 2 n 2 f ∞ inf t∈Sm t -b 2 + 6 ∆ f,2 m,m+p 2 op b -b m+p 2 f + 2σ 2 n ∆ f,2 m,m+p 2 F + 2cn log(n) ∆ m,m+p op E(Y 4 1 ) 1/2 + 3E(b (X 1 ) 4 ) 1/2 c 1/2 8.1 n 4 with ∆ f,2 m,m+p := Ψ -1/2 m+p ∆ * m,m+p Ψ 1/2
m and ∆ m,m+p is defined in (4).

The coefficients involved in the bounds given in Propositions 3.8 and 3.9 are the theoretical ones instead of the empirical in Propositions 3.2 and 3.6. They will allow us to evaluate rates of convergence for the estimator, provided that the basis is specified. This is the point of the next section.

Let us conclude this section with the following proposition which allows to control the risk in norm . f of b ,1 m (resp. b ,2 m ) via its risk in empirical norm, already controlled several ways in Propositions 3.2 and 3.8 (resp. Propositions 3.6 and 3.9). Proposition 3.10. Under Assumptions 3.1, 3.4(m, p) and 3.7(m), if ∆ m,m+p 2 op m ∆ n 2 with m ∆ > 0 not depending on m and n, then

E( b ,1 m -b 2 f ) 5 f ∞ inf t∈Sm t -b 2 + 4E( b ,1 m -b 2 n ) + c 3.10 n
and, if in addition Assumption 3.5(m) is satisfied, then

E( b ,2 m -b 2 f ) 5 f ∞ inf t∈Sm t -b 2 + 4E( b ,2 m -b 2 n ) + c 3.10 n ,
where c 3.10 > 0 is a constant not depending on m and n.

The condition ∆ m,m+p

2 op
m ∆ n 2 is satisfied by the trigonometric basis and Hermite's basis (see Section 4).

Bases examples and explicit risk bounds

In this section, we describe more precisely several examples of bases. Then we evaluate, for each, the order of the term b m -(b ) m 2 , which represents the main difference between the risk bounds of the two estimates b ,1 m and b ,2 m . Lastly, we give explicit orders for all the terms involved in the bound of Proposition 3.9 in order to obtain from our nonasymptotic risk bound asymptotic rates of convergence. 4.1. Examples of bases. First of all, let us provide four usual bases which can be considered because the ϕ j 's are differentiable:

• The trigonometric basis:

Defined on I = [0, 1] by t 1 (x) := 1, t 2j (x) := √ 2 cos(2πjx) and t 2j+1 (x) := √ 2 sin(2πjx) for j = 1, . . . , p with m = 2p + 1. Thus, L(m) = m for (ϕ j ) 1 j m = (t j ) 1 j m .
• The Laguerre basis: Defined on I = R + , via Laguerre's polynomials L j , j 0, by

j (x) := √ 2L j (2x)e -x with L j (x) := j k=0 j k (-1) k x k k! .
It satisfies j , k = δ k,j (see Abramowitz and Stegun [START_REF] Abramowitz | Handbook of mathematical functions with formulas, graphs, and mathematical tables[END_REF], 22.2.13), where δ k,j is the Kronecker symbol. Then, ( j ) j 0 is an orthonormal family of L 2 (R + ) such that j (0) = √ 2 and

j ∞ = sup x∈R+ | j (x)| = √ 2.
Thus, L(m) = 2m for (ϕ j ) 1 j m = ( j-1 ) 1 j m . The j 's satisfy the following recursive formula (see Lemma 8.1 in Comte and Genon-Catalot [START_REF] Comte | Laguerre and Hermite bases for inverse problems[END_REF]):

(7) 0 = -0 and j = -j -2 j-1 k=0 k for j 1.
• The Hermite basis: Defined on I = R, via Hermite's polynomials H j , j 0, by

h j (x) := c h (j)H j (x)e -x 2 /2 with H j (x) := (-1) j e x 2 d j dx j (e -x 2 ) and c h (j) = (2 j j! √ π) -1/2 .
The family (H j ) j 0 is orthogonal for the e -x 2 -weighted scalar product and as

R H j (x)H k (x)e -x 2 dx = c 2 h (j)δ j,k ,
we get h j , h k = δ j,k , (see Abramowitz and Stegun [START_REF] Abramowitz | Handbook of mathematical functions with formulas, graphs, and mathematical tables[END_REF], 22.2.14). Moreover,

h j ∞ = sup x∈R |h j (x)| φ 0
with φ 0 = π -1/4 (see Abramowitz and Stegun [START_REF] Abramowitz | Handbook of mathematical functions with formulas, graphs, and mathematical tables[END_REF], 22.14.17 and Indritz [START_REF] Indritz | An inequality for Hermite polynomials[END_REF]). Thus, L(m) π -1/2 m, but it is proved in [START_REF] Comte | Adaptive estimation of the conditional density from direct or noisy data[END_REF] that there exists K > 0 such that

sup x∈R m-1 j=0 h j (x) 2 K √ m.
Therefore, we set L(m) = K √ m for (ϕ j ) 1 j m = (h j-1 ) 1 j m . The h j 's also satisfy a recursive formula (see Comte and Genon-Catalot [START_REF] Comte | Laguerre and Hermite bases for inverse problems[END_REF], Equation (52) in Section 8.2): ( 8)

h 0 = - 1 √ 2 h 1 and h j = 1 √ 2 ( jh j-1 -j + 1h j+1 ) for j 1.
• The Legendre basis: Defined on I = [-1, 1], via Legendre polynomials G j , j 0, by

g j (x) := 2j + 1 2 G j (x) with G j (x) := 1 2 j j! • d j dx j [(x 2 -1) j ]. As 1 -1 G j (x)G k (x)dx = 2 2j + 1 δ j,k , the family (g j ) j 0 is an orthonormal family of L 2 ([-1, 1]). For example, g 0 (x) = 1/ √ 2, g 1 (x) = 3/2x, g 2 (x) = 1/2 5/2(3x 2 -1)
, etc. Note that they are easy to compute numerically thanks to the recursive formula

g j (x) = 1 j [(2j -1)xg j-1 (x) -(j -1)g j-2 (x)]
, j 1, (see Formula 2.6.2 in [START_REF] Efromovich | Nonparametric curve estimation: methods, theory, and applications[END_REF]). Moreover,

g j ∞ 2j + 1 2 , which gives m-1 j=0 g j (x) 2 1 2 m-1 j=0 (2j + 1) = m 2 2
and L(m) = m 2 /2 (see also Cohen et al. [START_REF] Cohen | On the Stability and Accuracy of Least Squares Approximations[END_REF]) for (ϕ j ) 1 j m = (g j-1 ) 1 j m . The g j 's also satisfy a recursive formula (see Formula ( 22) p.10 in Lagrange [START_REF] Lagrange | Polynômes et fonctions de Legendre[END_REF]):

d dx g j+1 (x) = 2j + 3 [j/2] k=0 2(j -2k) + 1g j-2k (x),
which can be written

(9) g 2p+1 (x) = 4p + 3 p k=0 √ 4k + 1g 2k (x), g 2p+2 (x) = 4p + 5 p k=0 √ 4k + 3g 2k+1 (x).
Under Assumption 3.5(m), thanks to Equality (5) and to the recursive formulas available for each basis described above, we are able to compare the derivative b m of b m to the derivative of the projection (b ) m of b as follows:

Proposition 4.1. Under Assumption 3.5(m):

(i) If I = [0, 1] and ϕ j = t j (the trigonometric basis with an odd m), then b m -(b ) m 2 = 0. (ii) If I = R and ϕ j = h j-1 (the Hermite basis), then b m -(b ) m 2 = m 2 ( b, h m-1 2 + b, h m 2 ). (iii) If I = R + and ϕ j = j-1 (the Laguerre basis), then b m -(b ) m 2 = 4m m-1 k=0 b, k 2 . If in addition b(0) = 0, then b m -(b ) m 2 = 4m   k m b, k   2 . (iv) If I = [-1, 1], ϕ j = g j-1 (the Legendre basis) and m = 2p, then b m -(b ) m 2 = 3 p-1 k=0 √ 4k + 3 b, g 2k+1 2 + (4p -1) p-1 k=0 √ 4k + 1 b, g 2k 2 + p-1 j=0   4j + 3 p-1 k=j √ 4k + 3 b, g 2k+1 + 4j + 1 j k=0 √ 4k + 3 b, g 2k+1   2 + p-2 j=0   4j + 5 p-1 k=j+1 √ 4k + 1 b, g 2k + 4j + 2 j k=0 √ 4k + 1 b, g 2k   2 .
This implies that there exists a deterministic constant c 4.1 > 0, not depending on m and n, such

that b m -(b ) m 2 c 4.1 m 4 .
The cases are ordered from the simplest (the trigonometric one) to the most complicated (Legendre case for an even m). Proposition 4.1 shows that the term b m -(b ) m 2 which appears in the risk bound of b ,1 m importantly depends on the basis. Clearly, the first two bases are more convenient for this problem and we will focus on them in the sequel (for rates and simulation experiments).

4.2.

Explicit risk bound for the trigonometric basis. As the trigonometric basis has compact support, say I, we estimate in fact b := b1 I and we can assume that f (x) f 0 > 0 for every x ∈ I. Moreover, we assume that f is bounded (Assumption 3.1). We set I = [0, 1] for simplicity and assume that b(0) = b(1) (which ensures Assumption 3.5(m) for all m). Then, by considering models with an odd m, Assumption 3.4(m, p) is fulfilled for all m with p = 0. Moreover, we know from [START_REF] Comte | Regression Function Estimation on Non-Compact Support as a Partly Inverse Problem[END_REF] that

Ψ -1 m op 1/f 0 . Then, we get L(m) = m, ∆ m,m 2 op π 2 m 2 , ∆ f,1 m,m 2 op = ∆ f,2 m,m 2 op f ∞ f 0 m 2 , and ∆ f,1 m,m 2
F = ∆ f,2 m,m 2 F 1 f 0 m 3 .
The last bound comes from the following inequalities

∆ f,2 m,m 2
F = Tr[Ψ -1 m ∆ * m,m Ψ m ∆ m,m ] Ψ -1 m op Tr[∆ * m,m Ψ m ∆ m,m ] = Ψ -1 m op E Tr(∆ * m,m Ψ m ∆ m,m ) = 1 n Ψ -1 m op E Tr( Φ m ( Φ m ) * ) = 1 n Ψ -1 m op E   n i=1 m-1 j=0 ϕ j (X i ) 2   (10) m 2 n Ψ -1 m op E   n i=1 m-1 j=0 ϕ 2 j (X i )   = m 3 Ψ -1 m op m 3 f 0 ,
using that for ϕ j = t j , ϕ j = ±2πjϕ j±1 . So, the risk bound on b ,2 m established at Proposition 3.9 becomes

(11) E b ,2 m -b 2 n 2 f ∞ inf t∈Sm t -b 2 + 6 f 0 m 2 b -b m 2 + 2σ 2 nf 0 m 3 + c 1 n with c 1 >
W per (β, L) := {g ∈ C β ([0, 1]; R) : g (β-1) is absolutely continuous, 1 0 g (β) (x) 2 dx L 2 and g (j) (0) = g (j) (1), ∀j = 0, . . . , β -1}.
We obtain the following result:

Corollary 4.2. Consider the estimators b ,i m , i = 1, 2 computed in the trigonometric basis on I = [0, 1] with 0 < f 0 f (x) f ∞ < ∞, E[b (X 1 ) 4 ] < ∞ and E(Y 4 1 ) < ∞. If b ∈ W per (β, L) with β > 1, b(0) = b(1) and m opt = n 1/(2β+1) , then E b ,i mopt -b 2 n c i (L, β, f ∞ , f 0 , σ 2 )n -2(β-1)/(2β+1) , for i = 1, 2.
Proof. By Proposition 1.14 of [START_REF] Tsybakov | Introduction to Nonparametric Estimation[END_REF], a function f ∈ W per (β, L) admits a development

f = ∞ j=0 θ j ϕ j such that j 0 θ 2 j τ 2 j C(L),
where τ j = j β for even j, τ j = (j -1) β for odd j, and

C(L) := L 2 π -2β . Moreover, if b belongs to a the Sobolev ellipsoid W per (β, L) with β > 1, then b ∈ W per (β -1, 2πL). So, b -b m 2 c(L, β)m -2β and b -(b ) m 2 c(L, β)m -2(β-1) .
Therefore, plugging m = m opt = n 1/(2β+1) in [START_REF] Comte | Regression Function Estimation on Non-Compact Support as a Partly Inverse Problem[END_REF] gives the result of Corollary 4.2. Indeed, Propositions 3.8 and 3.9 apply because the required conditions are automatically satisfied by the trigonometric basis.

Note that we obtain the optimal rate for estimating the derivative of a regression function (see Stone [START_REF] Stone | Optimal Global Rates of Convergence for Nonparametric Regression[END_REF]). It coincides also with the rate of estimation for the derivative of a density (see Tsybakov [START_REF] Tsybakov | Introduction to Nonparametric Estimation[END_REF], Efromovich [START_REF] Efromovich | Simultaneous sharp estimation of functions and their derivatives[END_REF][START_REF] Efromovich | Nonparametric curve estimation: methods, theory, and applications[END_REF], recently Lepski [START_REF] Lepski | A new approach to estimator selection[END_REF] on general Nikolski's spaces, or Comte et al. [START_REF] Comte | Optimal adaptive estimation on R or R + of the derivatives of a density[END_REF]).

4.3.

Explicit risk bound for Hermite basis. Consider s, D > 0 and the Sobolev-Hermite ball of regularity s

(12) W s H (D) =    θ ∈ L 2 (R) : k 0 k s a 2 k (θ) D    ,
where a 2 k (θ) = θ, h k . In the Hermite case, the following bounds hold:

L(m) = K √ m, ∆ m,m+1 2 op 2m, ∆ f,1 m,m+1 2 op = ∆ f,2 m,m+1 2 op 2 f ∞ Ψ -1 m+1 op m and ∆ f,1 m,m+1 2
F = ∆ f,2 m,m+1 2 F 2K Ψ -1 m+1 op (m + 1) 3/2
. The last bound is obtained by following the line of the trigonometric case above, up to (10), using next formula [START_REF] Cohen | On the Stability and Accuracy of Least Squares Approximations[END_REF] for the derivative of the basis functions.

In this context, it is proved in [START_REF] Comte | Regression Function Estimation on Non-Compact Support as a Partly Inverse Problem[END_REF] that Ψ -1 m op is increasing with m. Therefore, we can state the following result.

(m + 1). Assume that b is square integrable, E[b (X 1 ) 4 ] < ∞, E(Y 4 1 ) < ∞ and that b ∈ W s H (D). If Ψ -1 m op
m γ for all m and s > 1 + γ, then by choosing

m opt = n 1/(s+1/2) yields E b ,i mopt -b 2 n c(D, s, f ∞ , σ 2 )n -2(s-1-γ)/(2s+1) for i = 1, 2.
Remark 4.4. The rate is deteriorated compared to n -2(s-1)/(2s+1) , which is the optimal rate of estimation for the derivative of a density in a similar non compact setting (see bounds [START_REF] Comte | Estimation non-paramétrique[END_REF] and ( 16) in [START_REF] Comte | Optimal adaptive estimation on R or R + of the derivatives of a density[END_REF]). However, we are in the framework of an inverse problem, due both to the derivative aspect and to the non compact support feature of the basis. If we compare the rate with the one found in [START_REF] Comte | Regression Function Estimation on Non-Compact Support as a Partly Inverse Problem[END_REF] for the estimation of b in the same context, n -s/(s+1) , we would expect n -(s-1)/(s+1) (which is larger). The deterioration is unavoidable as soon as the term Ψ -1 m op appears as multiplicative factor in the variance and the additional bias term. So the order obtained in Proposition 4.3 shows consistency but we do not know if it is optimal.

The main question is about the bounds ∆ f,1 m,m+1 2 op and ∆ f,1 m,m+1 2 F : the matrices in the norms involve both a matrix of type Ψ m and a matrix of type Ψ -1 m and if they could be associated, the factor Ψ -1 m op would not appear in the risk bound. The order of the additional bias term would be m -(s-1) and the variance would be of order m 2 /n. This seems to be true numerically.

The behavior of Ψ m is crucial for understanding our procedure. We want here to mention that in [START_REF] Comte | Regression Function Estimation on Non-Compact Support as a Partly Inverse Problem[END_REF], it is proved that, for all m, the matrix Ψ m computed in the Hermite basis is invertible and there exists a constant c such that, ( 13)

Ψ -1 m 2 op c m.
So, in the Hermite case, Inequality [START_REF] Comte | Adaptive estimation of the conditional density from direct or noisy data[END_REF] clearly implies that Ψ -1 m op cannot be uniformly bounded in m contrary to the case of compactly supported bases. Moreover, if we assume that f (x) c/(1

+ x 2 ) k for x ∈ R
a constant D = C(D) > D such that θ ∈ W s-1 H (D ). By Lemma 4.5, if b ∈ W s H (D), then b -b m 2 Dm -s , b -(b ) m 2 C(D)m -s+1
, and the risk bound on b ,2 m established at Proposition 3.9 becomes

E b ,2 m -b 2 n 2 f ∞ inf t∈Sm t -b 2 + 6 Ψ -1 m+1 op m b -b m+1 2 + 4Kσ 2 n Ψ -1 m+1 op (m + 1) 3/2 + c 1 n C(D) f ∞ [m -(s-1) + Ψ -1 m+1 op m(m + 1) -s ] + 4Kσ 2 n Ψ -1 m+1 op (m + 1) 3/2 + c 1 n (14) with c 1 > 0. Thus, if Ψ -1
m op = O(m γ ), for s > γ + 1, the estimator is consistent, and to plug the choice m = m opt = n 1/(s+1/2) in ( 14) gives the result of Corollary 4.3 for i = 2.

The risk E b ,1 m -b 2 n involves an additional term f ∞ b m -(b ) m 2 . From Proposition 4.1, (ii), we have b m -(b ) m 2 = (m/2)(a 2 m (b) + a 2 m-1 (b)) (1/2)( k m-1 a 2 k (b) + k m-1 ka 2 k (b)) m -(s-1)
under our assumptions, by writing that

k m-1 ka 2 k (b) = k m-1 k s a 2 k (b) × k -s+1 (m -1) -s+1 k 0 k s a 2 k (b) D(m -1) -(s-1) D2 s-1 m -(s-1) .
This gives the result of Corollary 4.3 for i = 1.

4.4.

Explicit risk bound for Legendre basis. By Proposition 2.6.1 in [START_REF] Efromovich | Nonparametric curve estimation: methods, theory, and applications[END_REF] (see also [START_REF] Devore | Constructive approximation. Grundlehren der Mathematischen Wissenschaften[END_REF], Section 7.6), it is known that if b ∈ C r ([-1, 1]; R) (r 1) and if there exists α ∈ (0, 1] such that

|b (r) (t) -b (r) (s)| Q|t -s| α ; ∀s, t ∈ [-1, 1], then there exists c > 0 such that b -b m 2 cm -2(r+α) and b -(b ) m 2 cm -2(r-1+α) .
The space of regularity β = r + α considered above will be called Hölder space and denoted by H(β, Q).

By Proposition 4.1, we can see that the first estimator may not be consistent as (b m ) -(b ) m 2 may not tend to zero. However, Formula [START_REF] Cohen | Correction to: On the stability and accuracy of least squares approximations[END_REF] shows that the Legendre basis satisfies Assumption 3.4(m, p) with p = 0 and triangular matrix ∆ m,m with null diagonal. As the basis is compactly supported, we can proceed as in the case of the trigonometric basis, assuming

I = [-1, 1], 0 < f 0 < f (x) < f ∞ < ∞ for every x ∈ I, and b(-1) = b(1) = 0. Then, L(m) = m 2 2 , ∆ m,m 2 op cm 4 , ∆ f,1 m,m 2 op = ∆ f,2 m,m 2 op c f ∞ f 0 m 4 , ∆ f,1 m,m 2
F = ∆ f,2 m,m 2 F c f 0 m 5 .
As a consequence, for b ∈ H(β, Q) with β > 2 and b(-1) = b(1) = 0, Proposition 3.9 implies that if

m opt = n 1/(2β+1) , then E b ,2 m -b 2 n c(Q, β, f ∞ , f 0 , σ 2 )n -2(β-2)/(2β+1) .
We mention this rate, but it is sub-optimal in the compact support case, specifically in comparison with the trigonometric basis.

A Goldenshluger-Lepski type adaptive estimator

The choice of the adequate m is crucial to reach the best order for the quadratic risk. However, this choice depends on unknown quantities, such as the order of regularity of the unknown function. This is why it is important to propose a way to select this dimension from the data. The problem is difficult, especially if we intend to bound the risk of the associated adaptive estimator. Penalty based model selection often rely on a contrast minimization, which seems not possible here. This is why we propose a Goldenshluger-Lepski type strategy, described in [START_REF] Goldenshluger | Bandwidth selection in kernel density estimation: oracle inequalities and adaptive minimax optimality[END_REF] for kernel estimators, and extended to dimension selection in Chagny [START_REF] Chagny | Warped bases for conditional density estimation[END_REF]. More precisely, consider the random collection

M n := m ∈ {1, . . . , n} : L(m + p)( Ψ -1 m+p 2 op ∨ 1) d n log(n)
where d > 0 is a constant depending on f ∞ (see the proof of Theorem 8.3), and the random penalty

V (m) := σ 2 m n ( Φ * m Φ m ) -1 ( Φ m ) * Φ m op .
This section deals with the adaptive estimator

(15) b := b ,1 m , where m = arg min m∈ Mn A(m) + κ 1 V (m) with A(m) := sup m ∈ Mn b ,1 m∧m -b ,1 m 2 n -κ 0 V (m ) + and κ 0 κ 1 .
Consider

M n := m ∈ {1, . . . , n} : L(m + p)( Ψ -1 m+p 2 op ∨ 1) d 4 • n log(n) ,
the theoretical counterpart of M n , and M + n with the same definition as M n but with d/4 replaced by 4d. The maximal element of M + n is denoted by M + n . Finally, let

V (m) := σ 2 m n ∆ f,1 m,m+p 2 op
be the theoretical version of V (m).

Lemma 5.1. The map m → V (m) is increasing.

A general Theorem is stated and proved in Section 8.9, the intested reader is refered to this section where comments are also provided. We emphasize that the procedure is general and does not depend on the basis. Moreover, the general results states an automatic squared-bias variance compromise. This is important in regard of Remark 4.4 above and Proposition 5.3 below, in case the rates would not be the best possible ones.

To avoid technicalities, we state two Propositions resulting from this theorem when considering the two main bases we previously described. More precisely, for the trigonometric case, we get the following result.

Proposition 5.2. Consider the estimator b computed in the trigonometric basis on

I = [0, 1] with 0 < f 0 f (x)
f ∞ < ∞ under Assumption 3.7(m). Moreover, assume that there exists κ > 0 such that E(exp(κε 2 1 )) < ∞ and that b is square-integrable on I. 2β+1) . Therefore, our data driven estimator automatically reaches the optimal rate, up to a multiplicative constant, in the compactly supported setting associated to the trigonometric basis. For the Hermite case, we obtain the following bound. 2s+1) . As a consequence, the Hermite estimator also reaches automatically the best rate we could expect, in the difficult context of non compact setting, but under stronger conditions. Note again that the general Theorem states that the data driven estimator performs the bias-variance compromise, whatever the effective orders of the terms are; this is why the estimator in the Hermite basis can numerically perform even better than in the trigonometric basis, see the next section.

If b ∈ W per (β, L) with β > 1, then E( b -b 2 n ) c(f 0 , f ∞ , L)n -2(β-1)/(
∈ W s H (D) with s > 2γ + 9/4, then E( b -b 2 n ) c(D, s, f ∞ , σ 2 )n -2(s-1-γ)/(

A numerical insight on the method

We consider the four simple functions [START_REF] Dai | Optimal estimation of derivatives in nonparametric regression[END_REF] b

1 (x) = 2 sin(πx), b 2 (x) = 0.5x exp(-x 2 /2), b 3 (x) = x 2 , b 4 (x) = 4x/(1 + x 2 ),
and we generate Y i = b(X i ) + ε i , i = 1, . . . , n, for i.i.d. X i ∼ N (0, 1), independent of the i.i.d Due to the theoretical difficulty of the question, in a model which looked rather simple at first sight, we first wondered if the strategy consisting in derivating the least squares regression estimator was relevant, and if numerical investigations could bring information about a good estimation strategy. This is why we first look at oracles: we compute all estimators of the collection and use the knowledge of the true function to compute the error associated to all of them in order to select the best one (the resulting "estimator" is called "oracle") in term of its L 2 -distance to the true. We also look at the associated dimensions.

ε i ∼ N (0, σ 2
We compute the L 2 -distance between each oracle estimator of b and the true b, and each oracle estimator of b and the true b , on an interval with bounds corresponding to the 3% and 97% quantiles of the X i 's, and finally take the average on 400 independent paths generated. Moreover, we average the selected dimensions for each sample. In other words, we retain the dimension and error corresponding in each case to the smallest error, and compute means and standard deviations. 15)) with Hermite basis (columns "Herm"), trigonometric basis (columns "Trigo") and for the derivative of NW estimator with oracle bandwidth (columns "NWO"), with their standard deviations multiplied by 100 ("std"). "dim": mean of the selected dimensions or oracle bandwidths. 400 repetitions and 3 sample sizes 250, 1000, 4000.

Table 1.

Table 1 shows that the MSE decreases when n increases, in all cases, and whether b or b is estimated. We can notice that function b 1 is chosen to be easy for the trigonometric basis, but the Hermite basis performs very well in this case too. On the contrary, the function b 2 is supposed to be easy for the Hermite basis, and it is, with small selected dimensions, but the trigonometric basis has a much worse performance. For the two other functions, the two bases perform similarly, with decreasing error when increasing n and simultaneous increase of the selected dimensions. This is expected from the theoretical formula giving the asymptotic optimal choice of m as a power of n, at least when the function under estimation does not admit a finite decomposition in the basis (like b 1 for the trigonometric basis or b 2 for the HErmite basis). What is puzzling in these results is the comparison of oracle dimensions for b and b : in each case, they are almost the same. This suggests to keep the selected model obtained for estimation of b by classical penalisation, and use this for b as well. This is coherent with the fact that the order of the optimal dimension are the same for b and b when using the trigonometric basis. We also implement the Goldenshluger-Lespki method with κ 1 = 2κ 0 and κ 0 = 1 for both Hermite and Trigonometric bases. We compare the performance or our estimator to the derivative of the Nadaraya-Watson estimator (with Gaussian kernel) with recursive computation in the spirit of Bercu el al. [START_REF] Bercu | Nonparametric recursive estimation of the derivative of the regression function with application to sea shores water quality[END_REF]. However, we took a fixed oracle bandwidth because their proposal of recursively varying bandwith h k = k -α with α = 0.3 does not work in our case and the method of selection of α is not given in their paper. So, we give the results for the best possible choice. The results are given in Table 2, and confirm that our method performs well. Obviously, the selected dimension are larger than the ones pointed by oracles in Table 1, and it is possible that other couples (κ 0 , κ 1 ) may be better. But it is now documented that the Goldenshluger and Lepski method is difficult to calibrate. The kernel estimator is generally less efficient in spite of its ideal bandwidth choice, even if its error gets very comparable to the other estimators when n increases. The orders associated with the MSE given are more concretely illustrated in Figures 1 to 4, and we can see that the estimations are very satisfactory. The estimators of the regression function b is obtained by penalisation as in Comte and Genon-Catalot [START_REF] Comte | Regression Function Estimation on Non-Compact Support as a Partly Inverse Problem[END_REF]. The Hermite basis performs globally very well, even to estimate a straight line as in Figure 3, which seems much more difficult for the trigonometric basis. Lastly, Figure 4 shows that there are a lot of side-effects for the estimation of b 4 , but it is probably due to "heavy tail" effects since it does not occur for b 2 which has faster decrease, see Figure 2 7. Concluding Remarks In this paper, we have defined two projection estimators of the derivative of b, based on observations (X i , Y i ) 1 i n drawn from Model [START_REF] Abramowitz | Handbook of mathematical functions with formulas, graphs, and mathematical tables[END_REF]. Under weak assumptions, we prove two simple risk bounds allowing to understand the differences between the two strategies. More elaborate bounds under a stability condition introduced by Cohen et al. [START_REF] Cohen | On the Stability and Accuracy of Least Squares Approximations[END_REF] are also given. These results are illustrated in the context of trigonometric, Legendre, Laguerre and Hermite bases, the first two ones being compactly supported, but not the last ones. Optimal rates are recovered with our method in the context of the trigonometric basis, but our setting is more general, which is a novelty. Lastly, we propose a model selection procedure and prove a general risk bound for the adaptive estimator. It automatically reaches the optimal rate in the trigonometric case. These last results are also new and not straightforward.

The method we propose is implemented and the few numerical experiments conducted shows that our estimator works well, in particular compared the derivative of a Nadaray-Watson kernel estimators; but more comparisons would be useful to confirm these practical results. Our investigation for simple examples shows that the collection of estimators always contains a good one. It also suggests that keeping for the estimation of b the dimension selected for b may be a safe simple strategy. Several extensions of this work may be of obvious interest: explanatory variables with higher dimensions may be studied in the spirit of Dussap [START_REF] Dussap | Nonparametric Multiple Regression by Projection on Non-compactly Supported Bases[END_REF], as well as higher order of derivatives, possibly only in the compactly supported case to begin with. Extensions to dependent contexts (the case of autoregressive models or the case of diffusion models) are also to be considered. As our proofs rely on results conditionally to the X i 's, thanks to their independence with the noise, dependency should imply theoretical difficulties.

Proofs

All the properties on matrix norms used in proofs are reminded in the Subsubsection 3.1 at the end of the introduction.

Moreover, we denote by E X the conditional expectation given X = (X 1 , . . . , , X n ).

8.1. Proof of Proposition 3.2. Note first that b ,1 m (X) = Φ m ( Φ * m Φ m ) -1 Φ * m Y, and since Y i = b(X i ) + ε i , X i is independent of ε i , and E(ε i ) = 0 for every i ∈ {1, . . . , n}, E X [ b ,1 m (X)] = Φ m ( Φ * m Φ m ) -1 Φ * m b(X) and E X ( Φ m ( Φ * m Φ m ) -1 Φ * m ε, Φ m ( Φ * m Φ m ) -1 Φ * m b(X) -b (X) n ) = 0.
Then,

E X ( b ,1 m -b 2 n ) = 1 n E X [ b ,1 m (X) -E X [ b ,1 m (X)] + E X [ b ,1 m (X)] -b (X) 2 2,n ] = 1 n [E X ( Φ m ( Φ * m Φ m ) -1 Φ * m ε 2 2,n ) + Φ m ( Φ * m Φ m ) -1 Φ * m b(X) -b (X) 2 2,n ] =: 1 n (A + B).
On the one hand,

A = E X ε * Φ m ( Φ * m Φ m ) -1 ( Φ m ) * Φ m ( Φ * m Φ m ) -1 Φ * m ε = σ 2 Tr Φ m ( Φ * m Φ m ) -1 ( Φ m ) * Φ m ( Φ * m Φ m ) -1 Φ * m = σ 2 Tr ( Φ * m Φ m ) -1 ( Φ m ) * Φ m .
On the other hand,

B 3 Φ m ( Φ * m Φ m ) -1 Φ * m b(X) -b m (X) 2 2,n + 3 b m (X) -(b ) m (X) 2 2,n + 3 (b ) m (X) -b (X) 2 2,n . So, 1 n E(B) 3 inf t∈Sm t -b 2 f + 3 b m -(b ) m 2 f + 3E(C) with C = 1 n Φ m ( Φ * m Φ m ) -1 Φ * m b(X) -b m (X) 2 2,n .
In order to manage this last term, note that b

m (X) = m j=1 b, ϕ j ϕ j (X) = Φ m ( b, ϕ j ) 1 j m . So, ( 17
) ( b, ϕ j ) 1 j m = ( Φ * m Φ m ) -1 Φ * m b m (X) and then, b m (X) = m j=1 b, ϕ j ϕ j (X) = Φ m ( b, ϕ j ) 1 j m = Φ m ( Φ * m Φ m ) -1 Φ * m b m (X).
Therefore,

C = 1 n Φ m ( Φ * m Φ m ) -1 Φ * m (b(X) -b m (X)) 2 2,n Φ m ( Φ * m Φ m ) -1 Φ * m 2 op b -b m 2 n .
This concludes the proof.

8.2.

Proof of Proposition 3.3. The ϕ j 's do not depend on m, so the S m 's are nested spaces, and then to establish the following equality is sufficient in order to conclude:

(18) E X sup t∈Sm: t n =1 ν n (t) 2 = σ 2 n Tr Ψ -1/2 m Ψ m Ψ -1/2 m with Ψ m := 1 n ( Φ m ) * Φ m and ν n (t) := ε, t n .
Let us prove Equality [START_REF] Dussap | Nonparametric Multiple Regression by Projection on Non-compactly Supported Bases[END_REF]. Consider t ∈ S m such that t n = 1. Necessarily (and sufficiently),

t = m j=1 a j ϕ j with a = Ψ -1/2 m u and u ∈ R m such that u 2,m = 1. Then, t = m k=1 u k m j=1 [ Ψ -1/2 m ] j,k ϕ j
and, thanks to Cauchy-Schwarz's inequality,

ν n (t) 2 = ε, t 2 n =   m k=1 u k ε, m j=1 [ Ψ -1/2 m ] j,k ϕ j n   2 m k=1 ε, m j=1 [ Ψ -1/2 m ] j,k ϕ j 2 n .
So,

sup t∈Sm: t n =1 ν n (t) 2 = sup u∈R m : u 2,m=1   m k=1 u k ε, m j=1 [ Ψ -1/2 m ] j,k ϕ j n   2 = m k=1 ε, m j=1 [ Ψ -1/2 m ] j,k ϕ j 2 n .
Therefore, since ε 1 , . . . , ε n are i.i.d, centered, and respectively independent of X 1 , . . . , X n , and since

Ψ -1/2 m
and Ψ m are symmetric matrices,

E X sup t∈Sm: t n =1 ν n (t) 2 = σ 2 n 2 m k=1 n i=1   m j=1 [ Ψ -1/2 m ] j,k ϕ j (X i )   2 = σ 2 n m j,k, =1 [ Ψ -1/2 m ] j,k [ Ψ -1/2 m ] ,k ϕ j , ϕ n = σ 2 n m j,k, =1 [ Ψ -1/2 m ] k,j [ Ψ m ] j, [ Ψ -1/2 m ] ,k = σ 2 n Tr Ψ -1/2 m Ψ m Ψ -1/2 m .
This concludes the proof.

8.3. Proof of Proposition 3.6. As in the proof of Proposition 3.2,

E X ( b ,2 m -b 2 n ) = 1 n -Φ m ∆ m,m+p ( Φ * m+p Φ m+p ) -1 Φ * m+p b(X) -b (X) 2 2,n + σ 2 n Tr ( Φ * m+p Φ m+p ) -1 ∆ * m,m+p Φ * m Φ m ∆ m,m+p 2 n [ -Φ m ∆ m,m+p ( Φ * m+p Φ m+p ) -1 Φ * m+p b(X) -(b ) m (X) 2 2,n + (b ) m (X) -b (X) 2 2,n ] + σ 2 n Tr ( Φ * m+p Φ m+p ) -1 ∆ * m,m+p Φ * m Φ m ∆ m,m+p .
On the one hand, as previously,

2 n E( (b ) m (X) -b (X) 2 2,n ) = 2 (b ) m -b 2 f 2 f ∞ inf t∈Sm t -b 2 .
On the other hand, thanks to Equalities ( 6) and ( 17),

(b ) m (X) = -Φ m ∆ m,m+p ( b, ϕ j ) 1 j m+p = -Φ m ∆ m,m+p ( Φ * m+p Φ m+p ) -1 Φ * m+p b m+p (X).
Then,

-Φ m ∆ m,m+p ( Φ * m+p Φ m+p ) -1 Φ * m+p b(X) -(b ) m (X) 2 2,n = -Φ m ∆ m,m+p ( Φ * m+p Φ m+p ) -1 Φ * m+p (b(X) -b m+p (X)) 2 2,n .
This concludes the proof.

Ω m := | t 2 n / t 2 f -1| 1 2 ; ∀t ∈ S m = Ψ -1/2 m Ψ m Ψ -1/2 m -I m op 1 2 .
The proof relies on the following lemma, borrowed from Comte and Genon-Catalot [11, Lemma 5].

Lemma 8.1. Under Assumption 3.7(m), there exists a deterministic constant c 8.1 > 0, not depending on m and n, such that

P(Ω c m ) c 8.1 n 8 and P(Λ c m ) c 8.1 n 8 .
First of all,

E b ,1 m -b 2 n = E b ,1 m -b 2 n 1 Λm+p + E( b 2 n 1 Λ c m+p ). Obviously, by applying Lemma 8.1, since E[b (X 1 ) 4 ] < ∞, E( b 2 n 1 Λ c m+p ) E[b (X 1 ) 4 ] 1/2 P(Λ c m+p ) 1/2 c 1/2 8.1 E[b (X 1 ) 4 ] 1/2 1 n 4 . Let us dissect b ,1 m -b 2 n 1 Λm+p via the event Ω m+p : E b ,1 m -b 2 n 1 Λm+p = E b ,1 m -b 2 n 1 Λm+p∩Ωm+p + E b ,1 m -b 2 n 1 Λm+p∩Ω c m+p E b ,1 m -b 2 n 1 Λm+p∩Ωm+p +2 E( b ,1 m 4 n 1 Λm+p ) 1/2 + E( b 4 n ) 1/2 P(Ω c m+p ) 1/2 =: S + T.
On the one hand, let us find suitable bounds on the two remaining terms:

• For every measurable function ψ : R → R and q ∈ [1, ∞[ such that E(ψ(X 1 ) 2q ) < ∞, by Jensen's inequality,

E( ψ 2q n ) = E 1 n n i=1 ψ 2 (X i ) q 1 n n i=1 E[(ψ(X i )) 2q ] = E(ψ 2q (X 1 )). (19)
Then,

E( b 4 n ) E(b (X 1 ) 4 ). • Recall that b ,1 m (X) = Φ m ( Φ * m Φ m ) -1 Φ * m Y = Φ m+p ∆ * m,m+p ( Φ * m Φ m ) -1 Φ * m Y. First, Φ m+p ∆ * m,m+p ( Φ * m Φ m ) -1 Φ * m 2 op = λ max ( Φ m+p ∆ * m,m+p ( Φ * m Φ m ) -1 ∆ m,m+p Φ * m+p ) = n -1 Φ m+p ∆ * m,m+p Ψ -1/2 m 2 op n -1 Ψ -1/2 m 2 op Φ m+p ∆ * m,m+p 2 op = Ψ -1 m op λ max (∆ m,m+p Ψ m+p ∆ * m,m+p ) Ψ -1 m op Ψ m+p op ∆ m,m+p 2 op . (20) Moreover, Ψ m+p op L(m + p) and L(m) Ψ -1 m op L(m + p) Ψ -1 m+p op cn/ log(n) on Λ m+p . Then, E( b ,1 m 4 n 1 Λm+p ) 1 n 2 E( Ψ -1 m 2 op Ψ m+p 2 op 1 Λm+p Y 4 2,n ) ∆ m,m+p 4 op c 2 n 2 log(n) 2 ∆ m,m+p 4 op E   1 n n i=1 Y 2 i 2   c 2 n 2 log(n) 2 ∆ m,m+p 4 op E(Y 4 1 ).
Thus, thanks to Lemma 8.1,

T = 2 E( b ,1 m 4 n 1 Λm+p ) 1/2 + E( b 4 n ) 1/2 P(Ω c m+p ) 1/2 2 cn log(n) ∆ m,m+p 2 op E(Y 4 1 ) 1/2 + E(b (X 1 ) 4 ) 1/2 c 1/2 8.1 n 4 .
On the other hand, with the exact same ideas as in the proof of Proposition 3.2,

S -3 f ∞ inf t∈Sm t -b 2 -3 b m -(b ) m 2 f 3E Φ m+p ∆ * m,m+p ( Φ * m Φ m ) -1 Φ * m 2 op b -b m 2 n 1 Λm+p∩Ωm+p + σ 2 n E Tr ( Φ * m Φ m ) -1 ∆ m,m+p Φ * m+p Φ m+p ∆ * m,m+p 1 Λm+p∩Ωm+p =: S 1 + S 2 .
Let us find suitable bounds on S 1 and S 2 :

• On Ω m+p , the eigenvalues of

Ψ -1/2 m+p Ψ m+p Ψ -1/2 m+p belong to [1/2, 3/2].
The same way, on Ω m , the eigenvalues of

Ψ -1/2 m Ψ m Ψ -1/2 m belong to [1/2, 3/2]
and then, those of the matrix Ψ

1/2 m Ψ -1 m Ψ 1/2 m belong to [2/3, 2]. So, on Ω m+p , S 1 := Φ m+p ∆ * m,m+p ( Φ * m Φ m ) -1 Φ * m 2 op satisfies S 1 = λ max ( Φ m+p ∆ * m,m+p ( Φ * m Φ m ) -1 ∆ m,m+p Φ * m+p ) = λ max (∆ * m,m+p Ψ -1 m ∆ m,m+p Ψ m+p ) = λ max (Ψ -1/2 m+p ∆ f,1 m,m+p Ψ 1/2 m Ψ -1 m Ψ 1/2 m (∆ f,1 m,m+p ) * Ψ -1/2 m+p Ψ m+p ) = λ max ((Ψ -1/2 m+p Ψ m+p Ψ -1/2 m+p ) 1/2 ∆ f,1 m,m+p Ψ 1/2 m Ψ -1 m Ψ 1/2 m (∆ f,1 m,m+p ) * (Ψ -1/2 m+p Ψ m+p Ψ -1/2 m+p ) 1/2 ) = (Ψ -1/2 m+p Ψ m+p Ψ -1/2 m+p ) 1/2 ∆ f,1 m,m+p Ψ 1/2 m Ψ -1 m Ψ 1/2 m (∆ f,1 m,m+p ) * (Ψ -1/2 m+p Ψ m+p Ψ -1/2 m+p ) 1/2 op Ψ -1/2 m+p Ψ m+p Ψ -1/2 m+p op ∆ f,1 m,m+p 2 op Ψ 1/2 m Ψ -1 m Ψ 1/2 m op 3 ∆ f,1 m,m+p 2 op . (21)
Thus,

S 1 9 ∆ f,1 m,m+p 2 op E( b -b m 2 n ) = 9 ∆ f,1 m,m+p 2 op b -b m 2 f . • As previously, since the eigenvalues of Ψ 1/2 m Ψ -1 m Ψ 1/2 m 1 Ωm+p belong to [2/3, 2], S 2 = σ 2 n E Tr Ψ 1/2 m Ψ -1 m Ψ 1/2 m (∆ f,1 m,m+p ) * Ψ -1/2 m+p Ψ m+p Ψ -1/2 m+p ∆ f,1 m,m+p 1 Λm+p∩Ωm+p σ 2 n E Ψ 1/2 m Ψ -1 m Ψ 1/2 m op Tr (∆ f,1 m,m+p ) * Ψ -1/2 m+p Ψ m+p Ψ -1/2 m+p ∆ f,1 m,m+p 1 Ωm+p 2σ 2 n Tr (∆ f,1 m,m+p ) * Ψ -1/2 m+p E( Ψ m+p )Ψ -1/2 m+p ∆ f,1 m,m+p = 2σ 2 n ∆ f,1 m,m+p 2
F .
The result follows by gathering all the terms. 8.5. Proof of Proposition 3.9. First of all,

E b ,2 m -b 2 n = E b ,2 m -b 2 n 1 Λm+p + E( b 2 n 1 Λ c m+p ).
Obviously, by applying Lemma 8.

1, since E[b (X 1 ) 4 ] < ∞, E( b 2 n 1 Λ c m+p ) E[b (X 1 ) 4 ] 1/2 P(Λ c m+p ) 1/2 c 1/2 8.1 E[b (X 1 ) 4 ] 1/2 1 n 4 . Let us dissect b ,2 m -b 2 n 1 Λm+p via the event Ω m+p : E b ,2 m -b 2 n 1 Λm+p = E b ,2 m -b 2 n 1 Λm+p∩Ωm+p + E b ,2 m -b 2 n 1 Λm+p∩Ω c m+p E b ,2 m -b 2 n 1 Λm+p∩Ωm+p +2 E( b ,2 m 4 n 1 Λm+p ) 1/2 + E( b 4 n ) 1/2 P(Ω c m+p ) 1/2 =: S + T.
On the one hand, let us find suitable bounds on the two remaining terms:

• As in the proof of Proposition 3.8, thanks to Inequality [START_REF] Efromovich | Simultaneous sharp estimation of functions and their derivatives[END_REF],

E( b 4 n ) E(b (X 1 ) 4 ). • Recall that b ,2 m (X) = Φ m ∆ m,m+p ( Φ * m+p Φ m+p ) -1 Φ * m+p Y.
First,

Φ m ∆ m,m+p ( Φ * m+p Φ m+p ) -1 Φ * m+p 2 op = λ max ( Φ m ∆ m,m+p ( Φ * m+p Φ m+p ) -1 ∆ * m,m+p Φ * m ) = n -1 Φ m ∆ m,m+p Ψ -1/2 m+p 2 op n -1 Ψ -1/2 m+p 2 op Φ m ∆ m,m+p 2 op = Ψ -1 m+p op λ max (∆ * m,m+p Ψ m ∆ m,m+p ) Ψ -1 m+p op Ψ m op ∆ m,m+p 2 op .
Moreover, Ψ m op L(m) L(m + p) and L(m

+ p) Ψ -1 m+p op cn/ log(n) on Λ m+p . Then, E( b ,2 m 4 n 1 Λm+p ) 1 n 2 E( Ψ -1 m+p 2 op Ψ m 2 op 1 Λm+p Y 4 2,n ) ∆ m,m+p 4 op c 2 n 2 log(n) 2 ∆ m,m+p 4 op E   1 n n i=1 Y 2 i 2   c 2 n 2 log(n) 2 ∆ m,m+p 4 op E(Y 4
1 ).

Thus, thanks to Lemma 8.1,

T = 2 E( b ,2 m 4 n 1 Λm+p ) 1/2 + E( b 4 n ) 1/2 P(Ω c m+p ) 1/2 2 cn log(n) ∆ m,m+p 2 op E(Y 4 1 ) 1/2 + E(b (X 1 ) 4 ) 1/2 c 1/2 8.1 n 4 .
On the other hand, with the exact same ideas than in the proof of Proposition 3.6,

S -2 f ∞ inf t∈Sm t -b 2 2E Φ m ∆ m,m+p ( Φ * m+p Φ m+p ) -1 Φ * m+p 2 op b -b m+p 2 n 1 Λm+p∩Ωm+p + σ 2 n E Tr ( Φ * m+p Φ m+p ) -1 ∆ * m,m+p Φ * m Φ m ∆ m,m+p 1 Λm+p∩Ωm+p =: S 1 + S 2 .
Let us find suitable bounds on S 1 and S 2 :

• On Ω m+p , the eigenvalues of Ψ

Ψ m Ψ -1/2 m belong to [1/2, 3/2]. So, on Ω m+p , S 1 := Φ m ∆ m,m+p ( Φ * m+p Φ m+p ) -1 Φ * m+p 2 op satisfies S 1 = λ max ( Φ m ∆ m,m+p ( Φ * m+p Φ m+p ) -1 ∆ * m,m+p Φ * m ) = λ max (∆ m,m+p Ψ -1 m+p ∆ * m,m+p Ψ m ) = λ max (Ψ -1/2 m (∆ f,2 m,m+p ) * Ψ 1/2 m+p Ψ -1 m+p Ψ 1/2 m+p ∆ f,2 m,m+p Ψ -1/2 m Ψ m ) = λ max ((Ψ -1/2 m Ψ m Ψ -1/2 m ) 1/2 (∆ f,2 m,m+p ) * Ψ 1/2 m+p Ψ -1 m+p Ψ 1/2 m+p ∆ f,2 m,m+p (Ψ -1/2 m Ψ m Ψ -1/2 m ) 1/2 ) = (Ψ -1/2 m Ψ m Ψ -1/2 m ) 1/2 (∆ f,2 m,m+p ) * Ψ 1/2 m+p Ψ -1 m+p Ψ 1/2 m+p ∆ f,2 m,m+p (Ψ -1/2 m Ψ m Ψ -1/2 m ) 1/2 op Ψ -1/2 m Ψ m Ψ -1/2 m op ∆ f,2 m,m+p 2 op Ψ 1/2 m+p Ψ -1 m+p Ψ 1/2 m+p op 3 ∆ f,2 m,m+p 2 op . Thus, S 1 6 ∆ f,2 m,m+p 2 op E( b -b m+p 2 n ) = 6 ∆ f,2 m,m+p 2 op b -b m+p 2 f . • As previously, since the eigenvalues of Ψ 1/2 m+p Ψ -1 m+p Ψ 1/2 m+p 1 Ωm+p belong to [2/3, 2], S 2 = σ 2 n E Tr Ψ 1/2 m+p Ψ -1 m+p Ψ 1/2 m+p ∆ f,2 m,m+p Ψ -1/2 m Ψ m Ψ -1/2 m (∆ f,2 m,m+p ) * 1 Λm+p∩Ωm+p σ 2 n E Ψ 1/2 m+p Ψ -1 m+p Ψ 1/2 m+p op Tr ∆ f,2 m,m+p Ψ -1/2 m Ψ m Ψ -1/2 m (∆ f,2 m,m+p ) * 1 Ωm+p 2σ 2 n Tr ∆ f,2 m,m+p Ψ -1/2 m E( Ψ m )Ψ -1/2 m (∆ f,2 m,m+p ) * = 2σ 2 n ∆ f,2 m,m+p 2
F .
The result follows by gathering all the terms.

2 f = I   m j=1 [ θ 1 m ] j ϕ j (x)   2 f (x)dx = ( θ 1 m ) * Ψ m θ 1 m Ψ m op θ 1 m 2 2,m with
Ψ m := ( ϕ j , ϕ k f ) j,k = 1 n E[( Φ m ) * Φ m ] = ∆ m,m+p Ψ m+p ∆ m,m+p .
Then,

Ψ m op ∆ m,m+p 2 op Ψ m+p op ∆ m,m+p 2
op L(m + p). Moreover, as established in the proof of Comte and Genon-Catalot [START_REF] Comte | Regression Function Estimation on Non-Compact Support as a Partly Inverse Problem[END_REF], Proposition 5,

θ 1 m 2 2,m 1 n Ψ -1 m op Y 2 2,n 1 n Ψ -1 m+p op Y 2 2,n
and then, on

Λ m+p , b ,1 m 4 f ∆ m,m+p 4 op c 2 log(n) 2 n i=1 Y 2 i 2 .
Since the Y i 's are independent and ∆ m,m+p

2 op m ∆ n 2 , E( b ,1 m 4 f ) 1/2 ∆ m,m+p 4 op c 2 n log(n) 2 E(Y 4 1 ) 1/2 mn 3 with m = cm ∆ E(Y 4 1 ) 1/2 .
Therefore, by Lemma 8.2,

E( b ,1 m -b 2 f ) 5 f ∞ inf t∈Sm t -b 2 + 4E( b ,1 m -b 2 n ) + c 3.10 (m, b ) n .
The risk bound in norm . f on b ,2 m is obtained via similar arguments.

Proof of Lemma 8.2. First of all, note that

E( ϕ -ϕ 2 f ) = E( ϕ -ϕ 2 f 1 Ωn ) + E( ϕ -ϕ 2 f 1 Ω c n ) =: T 1 + T 2 .
For any t ∈ L 2 (I, f (x)dx), let t (f ) be the orthogonal projection of t on S m for the theoretical norm . f . On the one hand, since

t 2 f 1 Ωn 2 t 2 n 1 Ωn for every t ∈ S m , ϕ -ϕ 2 f 1 Ωn = ( ϕ -ϕ (f ) 2 f + ϕ (f ) -ϕ 2 f )1 Ωn ϕ (f ) -ϕ 2 f + 2 ϕ -ϕ (f ) 2 n 1 Ωn inf t∈Sm t -ϕ 2 f + 4 ϕ -ϕ 2 n + 4 ϕ -ϕ (f ) 2 n . Since E( ϕ -ϕ (f ) 2 n ) = ϕ -ϕ (f ) 2 f , T 1 5 inf t∈Sm t -ϕ 2 f + 4E( ϕ -ϕ 2 n ) 5 f ∞ inf t∈Sm t -ϕ 2 + 4E( ϕ -ϕ 2 n ).
On the other hand, since P(Ω c m ) c 8.1 /n 8 by Lemma 8.1,

T 2 E( ϕ -ϕ 4 f ) 1/2 P(Ω c m ) 1/2 √ 8[ ϕ 2 f + E( ϕ 4 f ) 1/2 ] c 8.1 n 4 .
Finally, the condition E( ϕ 4 f ) 1/2 mn 3 implies that b, h j h j .

E( ϕ -ϕ 2 f ) 5 f ∞ inf t∈Sm t -ϕ 2 + 4E( ϕ -ϕ 2 n ) + √ 8c 8.1 ( ϕ 2 f + m) n .
Then, thanks to Equality [START_REF] Cohen | On the Stability and Accuracy of Least Squares Approximations[END_REF],

b m = 1 √ 2 m-1 j=0 b, h j ( jh j-1 -j + 1h j+1 ) = 1 √ 2   m-2 j=0 b, h j+1 j + 1h j - m j=1 b, h j-1 jh j   = 1 √ 2 m-2 j=0 j + 1 b, h j+1 -j b, h j-1 h j - m -1 2 b, h m-2 h m-1 + m 2 b, h m-1 h m .
On the other hand, if b is square integrable, then b = j 0 b , h j h j .

The usual integration by parts gives b , h j = -b, h j as soon as lim x→±∞ b(x)h j (x) = 0 (this holds because the h j 's have exponential decrease and b is square-integrable, thus bounded near infinity). So, the projection of b is

(b ) m = - m-1 j=0 b, h j h j = - 1 √ 2 m-1 j=0 b, jh j-1 -j + 1h j+1 h j = 1 √ 2 m-1 j=0 j + 1 b, h j+1 -j b, h j-1 h j .
All the components of b m and (b ) m are the same on S m-2 . So,

b m -(b ) m = - m 2 b, h m-1 h m - m 2 b, h m h m-1 ,
and then,

b m -(b ) m 2 = m 2 b, h m-1 2 + b, h m 2 .
If b belongs to a Hermite-Sobolev space with regularity index α > 1, then the term b m -(b ) m 2 is of order m -(α-1) , which is also the order of inf t∈Sm t -b 2 .

The Laguerre case. As previously, on the one hand, b m = m-1 j=0 b, j j , and thanks to [START_REF] Chesneau | A note on wavelet estimation of the derivatives of a regression function in a random design setting[END_REF],

b m = m-1 j=0 b, j -j -2 j-1 k=0 k = - m-1 j=0 b, j j -2 m-2 k=0   m-1 j=k+1 b, j   k .
On the other hand, if b is square integrable, then b = j 0 b , j j . Thus, since b , j = -b, j by Assumption 3.5 (true when b(0

) = 0), (b ) m = m-1 j=0 b, j j -2 j-1 k=0 b, k j .
Consequently,

(b ) m -b m = 2 m-1 j=0 m-1 k=0 b, k j = 2 m-1 k=0 b, k m-1 j=0 j ,
and then

(b ) m -b m 2 = 4m m-1 k=0 b, k 2 .
Moreover, by assuming that b(0) = 0,

k 0 b, k k (0) = √ 2 k 0 b, k = 0. So, m-1 k=0 b, k = -k m b, k , and then (b ) m -b m 2 = 4m   k m b, k   2 .
Finally, if b belongs to a Laguerre-Sobolev space with index α > 1, then the right-hand side in the previous equality is smaller than

k m k α b, k 2 = O(m -α+1 ) 8.8. Proof of Lemma 5.1. First, V (m) = σ 2 m n Ψ -1 m ( Φ m ) * Φ m op = σ 2 m n Ψ -1/2 m ( Φ m ) * Φ m Ψ -1/2 m op where Ψ -1/2 m
is a symmetric square root of Ψ -1 m . Now, as the matrix is symmetric,

Ψ -1/2 m ( Φ m ) * Φ m Ψ -1/2 m op = sup x∈R m x Ψ -1/2 m ( Φ m ) * Φ m Ψ -1/2 m x = n sup t∈Sm: t n =1 t 2 n . So, clearly, m → V (m) = σ 2 m/n
sup n∈N\{0}    1 log(n) m n L (m) L(m) [exp(-a 1 m) + exp(-a 2 L(m))]    < ∞ ; ∀a 1 , ∀a 2 > 0 with L (m) := sup x∈I m-1 j=0 ϕ j (x) 2 , (22)
and that there exists q ∈ N\{0} such that

(23) sup n∈N\{0}    1 n q/2 log(n) m n L (m) L(m)    < ∞.
Then, there exists a constant c 8.3 > 0, not depending on n, such that

E( b -b 2 n ) c 8.3 inf m∈Mn E( b ,1 m -b 2 n ) + κ 1 V (m) + ∆ f,1 m,m+p 2 op b -b m 2 f + sup m ∈M + n :m >m ∆ f,1 m ,m +p 2 op ( b M + n -b m 2 f + b -b M + n 2 ∞ ) + 9 2 f ∞ sup m ∈M + n :m >m b m -b m 2 + c 8.3 n .
Conditions ( 22) and ( 23) are fulfilled by all the bases we mentioned (trigonometric, Laguerre, Hermite, Legendre) because L(m) and L (m) have the order of powers of m. The condition on ε 1 is fulfilled by Gaussian random variables for any κ < 1/(2σ 2 ), and by random variables with a compactly supported distribution. The quantity

inf m∈Mn E( b ,1 m -b 2 n ) + κ 1 V (m) + ∆ f,1 m,m+p 2 op b -b m 2 f
has the order of the minimum risk over the estimators of the collection in this problem. The three additional terms are due to the bound on the bias term

E sup m ∈ Mn E X ( b ,1 m∧m ) -E X ( b ,1 m ) 2 n .
Concretely, Theorem 8.3 can be applied some of our specific bases.

8.9.2. Proof of Theorem 8.3. Throughout this subsection, for the sake of readability, we omit the superscript 1 and write b m instead of b ,1 m .

Following the lines of the proof of Theorem 2 in Comte and Genon-Catalot [START_REF] Comte | Regression Function Estimation on Non-Compact Support as a Partly Inverse Problem[END_REF], we consider the sets

Ξ n = {ω : M n ⊂ M n (ω) ⊂ M + n } and Ω n = m∈M + n Ω m ,
where

M + n := m ∈ {1, . . . , n} : L(m + p)( Ψ -1 m+p 2 op ∨ 1) 4c • n log(n) . First, E b -b 2 n 1 (Ωn∩Ξn) c c 1 n with c 1 > 0.
This follows from the proof of Proposition 3.8, using that P(Ξ c n ) c 3 /n 8 and P(Ω c n ) c 4 /n 8 . For these last probabilities, we refer to Comte and Genon-Catalot [START_REF] Comte | Regression Function Estimation on Non-Compact Support as a Partly Inverse Problem[END_REF], Lemmas 7 and 9, where the choice of

d = 1/[f( f ∞ ∨ 1 + 3 -1 )] with f = 192 is explained.
Here, the constant f has to be increased to obtain the power n -8 instead of n -2 . Now, we control the loss of b m on Ω n ∩ Ξ n . For any m ∈ M n , using that on Ξ n it also holds that m ∈ M n , we have

b m -b 2 n 3( b m -b m∧m 2 n + b m∧m -b m 2 n + b m -b 2 n ) 3(A(m) + κ 0 V ( m) + A( m) + κ 0 V (m) + b m -b 2 n ) 6(A(m) + κ 1 V (m)) + 3 b m -b 2 n as κ 0 κ 1 . (24) Moreover, A(m) 3 sup m∈ Mn b m -E X ( b m ) 2 n - κ 0 6 V (m) + +3 sup m ∈ Mn b m∧m -E X ( b m∧m ) 2 n - κ 0 6 V (m ) + +3 sup m ∈ Mn E X ( b m∧m ) -E X ( b m ) 2 n ,
and since

sup m ∈ Mn { • • • } = max sup m ∈ Mn:m m { • • • } ; sup m ∈ Mn:m m { • • • } , by Lemma 5.1 (m → V (m) is increasing), sup m ∈ Mn b m∧m -E X ( b m∧m ) 2 n - κ 0 6 V (m ) + max sup m ∈ Mn b m -E X ( b m ) 2 n - κ 0 6 V (m ) + ; b m -E X ( b m ) 2 n - κ 0 6 V (m) + sup m∈ Mn b m -E X ( b m ) 2 n - κ 0 6 V (m) + . Thus, (25)
A(m) 6 sup m∈ Mn b m -E X ( b m ) 2 n - κ 0 6 V (m) + + 3 sup m ∈ Mn E X ( b m∧m ) -E X ( b m ) 2 n .
The following lemma provides a suitable bound on the first term in the right-hand side of Inequality [START_REF] Huang | Local Polynomial and Penalized Trigonometric Series Regression[END_REF] obtained via the conditional Talagrand inequality.

Lemma 8.4. Let Assumption 3.1 be fulfilled. Let also Assumption 3.7(m + p) be fulfilled for every m ∈ M n . Moreover, assume that there exists κ > 0 such that E(exp(κε 2 1 )) < ∞, and that Conditions ( 22) and ( 23) hold. Then,

E sup m∈ Mn b m -E X ( b m ) 2 n - κ 0 6 V (m) + c 8.4 n ,
where c 8.4 > 0 is a deterministic constant not depending on n.

By Inequalities ( 24) and ( 25), and then by Lemma 8.4, for any m ∈ M n ,

E( b -b 2 n 1 Ωn∩Ξn ) 36E sup m∈ Mn b m -E X ( b m ) 2 n - κ 0 6 V (m) + +18E 1 Ξn∩Ωn sup m ∈ Mn E X ( b m∧m ) -E X ( b m ) 2 n +6κ 1 E( V (m)1 Ωn ) + 3E( b m -b 2 n ) 3E( b m -b 2 n ) + 6κ 1 c 2 V (m) + c 8.4 n +18E 1 Ξn∩Ωn sup m ∈ Mn E X ( b m∧m ) -E X ( b m ) 2 n with c 2 > 0. The inequality E( V (m)1 Ωn ) c 2 V (m)
is obtained via the same method than in the proof of Proposition 3.9.

Let us now control

B m,n := 1 n E 1 Ξn∩Ωn sup m ∈ Mn:m >m E X ( b ,1 m (X)) -E X ( b ,1 m (X)) 2 2,n .
Thanks to Equality [START_REF] Devore | Constructive approximation. Grundlehren der Mathematischen Wissenschaften[END_REF],

E X ( b ,1 m (X)) -E X ( b ,1 m (X)) = E X ( b ,1 m (X)) -b m (X) -(E X ( b ,1 m (X)) -b m (X)) + b m (X) -b m (X) = P m (b(X) -b m (X)) + P m (b(X) -b m (X)) + b m (X) -b m (X).
Since we are on Ω n , and since m, m ∈ M

+ n on Ξ n when m ∈ M n and m ∈ M n , Sp Ψ -1/2 m +p Ψ m +p Ψ -1/2 m +p ⊂ [1/2, 3/2] and Sp Ψ 1/2 m Ψ -1 m Ψ 1/2 m ⊂ [2/3, 2],
and the same for m instead of m . So, thanks to Inequality [START_REF] Fan | Local polynomial modelling and its applications[END_REF], This concludes the proof. We assume that j b, ϕ j 2 j 2β L with β ∈ N ∩ (1, ∞). First, 1) .

P m 2 op 3 ∆ f,1 m ,m +p 2 op . In the same way, P m 2 op 3 ∆ f,1 m,m+p 2 op . Thus, on Ω n , 1 n P m (b(X) -b m (X)) 2 2,n 3 ∆ f,1 m ,m +p 2 op b -b m 2 n 6 ∆ f,1 m ,m +p 2 op ( b -b Mn 2 n + b Mn -b m 2 n ) 6 ∆ f,1 m ,m +p 2 op ( b -b M + n 2 ∞ + 3/2 b M + n -b m 2
sup m<m n/ log(n) ∆ f,1 m ,m +p 2 op b M + n -b m 2 f f ∞ f 0 sup m<m n/ log(n) (m ) 2 j m b, ϕ j 2 f ∞ f 0 m -2(β-
So, this term is of same order than the bias term. Next, .

We have n -2β+3 n -2(β-1)/(2β+1) as soon as β (3 + √ 13)/4 1.65, which holds true when β ∈ N ∩ (1, ∞). In conclusion, this together with Theorem 8.3 and the orders given in Section 4.2 gives the announced result. 8.12. Proof of Proposition 5.3. Here, (ϕ j ) j∈N * is the Hermite basis, and for Ψ -1 m op = m γ , the constraint on the collection of models implies M + n = n 1/(2γ+1/2) . This is compatible with the choice of m opt = 1/n s+1/2 as s > 2γ + 9/4 > 2γ. Again, we have to study the orders of the additional terms of the bound in Theorem 8. So, this term is of same order than the bias term. Next, using Formula (8), by using the value of M + n . We have

- s -γ -2 2γ + 1/2 - s -1 -γ s + 1/2 if (s -γ -2) s + 1 2 -(s -γ -1) 2γ + 1 2 > 0. Since (s -γ -2) s + 1 2 -(s -γ -1) 2γ + 1 2 = (s -γ)(s -2γ -2) - 1 2 ,
s -γ > 2 and s -2γ -2 > 1/4, the constraint is fulfilled and this last term is negligible with respect to the rate. Considering the orders obtained in section 4.3, we get the result.

Assumption 3 . 1 .

 31 The density function f is bounded on I.

Assumption 3 . 5

 35 (m). For every j ∈ {1, . . . , m}, b(a)ϕ j (a) = b(a)ϕ j (a), where a := inf(I) and a := sup(I). Note that, for instance, Assumption 3.5(m) holds for every m ∈ N when b(a) = b(a) = 0. Under this additional condition, by the integration by parts formula, (5) b , ϕ j = -b, ϕ j ; ∀j ∈ {1, . . . , m}. j ϕ j (X) = -Φ m ∆ m,m+p ( b, ϕ j ) 1 j m+p , which legitimates the definition (3) of the alternative estimator b ,2 m of b . Let us establish a risk bound for this estimator. Proposition 3.6. Under Assumptions 3.1, 3.4(m, p) and 3.5(m),

2 f 2 f

 22 Now we have elements to compare the two estimators. Comparison of the two estimators. Note that for p = 0, this bound is almost the same as in Proposition 3.2, except that the undesirable term b m -(b ) m no longer appears. The counterpart is that the result of Proposition 3.6 requires the additional Assumptions 3.4 and 3.5. Thanks to Proposition 4.1 (see Section 4 for details): • In the specific case of the trigonometric basis, the additional term b m -(b ) m in the bound of Proposition 3.2 is null, and the first estimator requires less assumptions, so the first strategy is better. • In the case of the Hermite basis, Assumption 3.4(m, p), p = 1 and Assumption 3.5(m) are automatically fulfilled. However, it is difficult to determine which strategy is better. • In the Laguerre basis, Assumption 3.5(m) is satisfied for all m if b(0) = 0. If this holds, it follows from Proposition 4.1 (iii) that both strategies give the same rate. • In the case of the Legendre basis, the additional term b m -(b ) m

2

 2

3. 3 .

 3 Elaborate risk bounds on b ,1 m and b ,2 m . First of all, under Assumption 3.1, let us consider

Assumption 3 . 7 2 ) - 1 )/ 9 . 1

 372191 (m). The matrix Ψ m satisfies L(m) := sup x∈I m j=1 ϕ j (x) 2 < ∞ and L(m)( Ψ -1 m op ∨ 1) Since the ϕ j 's do not depend on m, the S m 's are nested spaces. Thus, since m → L(m) and m → Ψ -1 m op are increasing, if there exists m 0 ∈ N\{0} such that Assumption 3.7(m 0 ) is fulfilled, then Assumption 3.7(m) is fulfilled for every m m 0 . Now, consider the truncated estimators b ,1 m := b ,1 m Λm+p and b ,2 m := b ,2 m 1 Λm+p , where

1 m

 1 0 and odd m. Since p = 0 and b m = (b ) m for the trigonometric basis, the risk bound on b ,established at Proposition 3.8 is the same up to a multiplicative constant. Now, let us evaluate the rate of convergence of the estimator for b in some regularity space and well chosen m. Let β be a positive integer, L > 0 and define

Corollary 4 . 3 .

 43 Consider the estimators b ,i m , i = 1, 2 computed in the Hermite basis on I = R under Assumptions 3.1 and 3.7

Proposition 5 . 3 .

 53 Consider the estimator b computed in the Hermite basis on I = R under Assumptions 3.1 and 3.7(m + 1). Assume also that b is square integrable on I, E[b (X 1 ) 4 ] < ∞ and that there exists κ > 0 such that E(exp(κε 2 1 )) < ∞. If Ψ -1 m op m γ for every m ∈ {1, . . . , n}, and if b

  ), with σ = 0.25 and b = b j , j = 1, . . . , 4. For each sample, we compute the least squares estimator of b, together with its derivative, in the Hermite and in the trigonometric bases. We use what we call the "half" trigonometric basis, relying on functions x → √ 2 sin(πjx) and x → √ 2 cos(πjx) on [0, 1], rescaled to the interval [a, b]. For each function b, we considered K = 400 repetitions, and samples of sizes n = 250, 1000 and 4000.

Figure 1 .

 1 Figure 1. 40 estimated functions (dotted green) compared to the true (bold red), n = 1000. First line b 1 (see (16)) by penalisation, 100 MSE = 0.26 and 0.29, mean selected dimensions: 12.2 and 11.2. Second line b 1 with GL method, 100 MSE = 4.71 and 6.47, mean selected dimensions: 16.1 and 10.5. Left Hermite basis, right trigonometric basis.

Figure 2 .

 2 Figure 2. 40 estimated functions (dotted green) compared to the true (bold red), n = 1000. First line b 2 (see (16)) by penalisation, 100 MSE = 0.07 and 0.18, mean selected dimensions: 2.1 and 5.3. Second line b 2 with GL method, 100 MSE = 1.08 and 1.12, mean selected dimensions: 2.05 and 5.05. Left Hermite basis, right trigonometric basis.

Figure 3 .

 3 Figure 3. 40 estimated functions (dotted green) compared to the true (bold red), n = 1000. First line b 3 (see (16)) by penalisation, 100 MSE = 0.30 and 0.30, mean selected dimensions: 1.05 and 9.5. Second line b 3 with GL method, 100 MSE = 5.04 and 6.89, mean selected dimensions: 15.97 and 8.65. Left Hermite basis, right trigonometric basis.

Figure 4 .

 4 Figure 4. 40 estimated functions (dotted green) compared to the true (bold red), n = 1000. First line b 4 (see (16) by penalisation, 100 MSE = 0.29 and 0.33, mean selected dimensions: 9.85 and 10.5. Second line b 4 with GL method, 100 MSE = 11.8 and 6.28 mean selected dimensions: 16.1 and 10.6. Left Hermite basis, right trigonometric basis.

8. 4 .

 4 Proof of Proposition 3.8. Consider the following set

-1/ 2 m+p

 2 Ψ m+p Ψ -1/2 m+p belong to [1/2, 3/2] and then, those of the matrix Ψ 1/2 m+p Ψ -1 m+p Ψ 1/2 m+p belong to [2/3, 2]. The same way, on Ω m , the eigenvalues of Ψ -1/2 m

8. 6 .Lemma 8 . 2 .

 682 Proof of Proposition 3.10. The proof of Proposition 3.10 relies on the following general lemma. Consider ϕ ∈ L 2 (I, dx) and let ϕ be a measurable map from Ω × I into S m such that E( ϕ 4 f ) 1/2 mn 3 with m > 0 not depending on m and n. Under Assumptions 3.1 and 3.7(m),E( ϕ -ϕ 2 f ) 5 f ∞ inf t∈Sm t -ϕ 2 + 4E( ϕ -ϕ 2 n ) +c 3.10 (m, ϕ) n with c 3.10 (m, ϕ) = √ 8c 8.1 ( ϕ 2 f + m). The proof of Lemma 8.2 is postponed to the end of Subsection 8.6. Proposition 3.10 is obtained by applying Lemma 8.2 to ϕ = b and ϕ = b ,1 m first, and then to ϕ = b and ϕ = b ,2 m . First, b ,1 m

8. 7 .

 7 Proof of Proposition 4.1: The Hermite case. Consider a square integrable function b, and b m = m-1 j=0 b, h j h j its projection on S m = span{h 0 , . . . , h m-1 }. On the one hand, b m = m-1 j=0

+p 2 op ( b M + n -b m 2 f

 22 f ) where M + n is the maximal element of M + n , and1 n P m (b(X) -b m (X)) For the last term, on Ω n , b m (X) -b m (X) + b -b M + m ∈M + n :m >m b m -b m 2 .

8. 10 .••Ψ - 1 / 2 m( Φ m ) * Φ m b 2 - 1 m 1 ) 1 / 2 P(|ε 1 | > m n ) 1 2 with c 4 = 11 .

 101221112112411 Proof of Lemma 8.4. We emphasize that the lemma would be true for M n replaced by the weaker (and more natural) m ∈ {1, . . . , n} : L(m + p)( Ψ -1 m+p op ∨ 1) c n log(n) with c defined in Assumption 3.7(m + p). We only use this constraint in the following.First of all, for anym ∈ M n , since ψ 2 n = sup t∈Sm: t n =1 t, ψ 2 n for every ψ ∈ S m , b m -E X ( b m ) 2 n = sup t∈Sm: t n =1 ν n (t) 2 with, for any b = (b 1 , . . . , b m ) ∈ R m and t = m j=1 b j ϕ j , ν n (t) = 1 n t, Φ m Ψ -1 m Φ * m ε n = 1 n 2 [ Φ m Ψ -1 m Φ * m ] * t(X), ε 2,n = 1 n Θ t (X), ε 2,n , σ 2 m n 2 Ψ -1 m ( Φ m ) * Φ m op =: H 2 . Bound on B m,n . Since m ∈ M n , m Ψ -1 m ) * Φ m b] j ϕ j (x) 2m n n L(m) 1/2 sup b: Φm b 2,n= √ n Ψ -1 m ( Φ m ) * Φ m b m,2 2m n √ n L(m) Ψ -1 m ( Φ m ) * 2 op 2m n √ n L(m) Ψ -1 m op Ψ -1 m ( Φ m ) * Φ m op 2 cqκ -1 Ψ -1 m ( Φ m ) * Φ m op =: M. Bound on C m,n (X): C m,n (X) Φm b 2,n = √ n op L (m) e -c2m + q exp -c 3 √ q • L(m)where c 1 , c 2 , c 3 , c 1 , c 3 > 0 are universal constants, and thusS := E X sup m op L (m) e -c2m + q exp -c 3 √ q • L(m) cc 1 n log(n) m n L (m) L(m) e -c2m + q exp -c 3 √ q • L(m)thanks to the definition of M n . Thanks to Condition[START_REF] Gasser | Estimating regression functions and their derivatives by the kernel method[END_REF], this term is of order 1/n. On the other hand, since L(m) Ψ -1 m op cn/ log(n) for every m ∈ M n , and by Markov's inequality,Φm b 2,n = √ n Φ m Ψ -1 m ( Φ m ) * Φ m b cE(ε 4 1 ) 1/2 E(exp(κε 2 1)) 1/2 . Thanks to Condition[START_REF] Goldenshluger | Bandwidth selection in kernel density estimation: oracle inequalities and adaptive minimax optimality[END_REF], this term is of order 1/n. In conclusion, Proof of Proposition 5.2. Here, (ϕ j ) j∈N * is the trigonometric basis. Thus, with L(m) of order m and L (m) of order m 3 , Conditions (22) and (23) are obviously fulfilled. Moreover, under f (x) f 0 > 0, we know that Ψ -1 m op 1/f 0 , and then M + n has order n/ log(n). We take M + n = n/ log(n) for simplicity. The first terms of the bound in Theorem 8.3 have been already evaluated in the proof of Corollary 4.2, so we have to study the additional ones: sup m<m n/ log(n)

1 m

 1 sup m<m n/ log(n) b m -b m 2 j m [(2πj) b, ϕ j ] 2 Cm -2(β-1) .Lastly, for any m and x ∈ I,|(b -b m )(x)| -β+1/2 c(β, L)m -β+1/2 , which gives b -b m ∞ c(β, L)m -β+1/2 and sup m<m n/ log(n)

b m -b m 2 underj 2 f

 22 the regularity condition b ∈ W H s (L) with s > 2γ + 9/4 > 1. First, ∞ m -s+γ+1 .

sup m<m M + n b m -b m 2 j m j b, ϕ j 2 Cm - s+1 .∆ f, 1 m ,m +p 2 op b -b M + n 2 ∞

 22s+1122 This term is also of same order than the first bias term and is negligible with respect to the previous one.Lastly, b -b m 2 ∞ C(s, L)π -1/2 m -s+1, and thussup m<m M + n n -(s-γ-2)/(2γ+1/2)

  and k 1, then for m large enough, Ψ -1 m op Cm k . Numerical experiments seem to indicate that the order m k is sharp. Proof of Corollary 4.3. The following Lemma (Lemma 2.2 in Comte et al. [10]) gives a relationship between the regularity of θ ∈ W s H (D) and the regularity of its derivative. Lemma 4.5. Consider s 1 and D > 0. If θ ∈ W s

H (D) admits a square integrable derivative, then there exists

Table 1

 1 The results are reported in 100 MSE 0.91 (0.50) 0.82 (0.46) 0.23 (0.13) 0.23 (0.13) 0.06 (0.03) 0.07 (0.03) dim 10.7 (1.30) 4.94 (0.93) 13.1 (1.46) 6.13 (1.30) 15.2 (2.05) 7.40 (1.45) b 1 100 MSE 11.1 (6.27) 8.75 (5.11) 2.97 (1.79) 3.16 (1.92) 0.85 (0.47) 1.24 (0.62) dim 10.8 (1.01) 4.76 (0.76) 12.9 (1.37) 5.99 (1.12) 15.2 (2.01) 7.30 (1.34) b 2 100 MSE 0.18 (0.17) 0.47 (0.34) 0.05 (0.04) 0.12 (0.08) 0.01 (0.01) 0.03 (0.02) dim 2.15 (0.47) 3.08 (0.91) 2.15 (0.47) 3.08 (0.91) 2.18 (0.58) 3.46 (0.91) b 2 100 MSE 0.18 (0.19) 1.77 (1.07) 0.05 (0.05) 0.58 (0.36) 0.01 (0.01) 0.16 0.12) dim 2.26 (0.54) 2.50 (0.61) 2.26 (0.54) 3.01 (0.60) 2.22 (0.55) 3.28 (0.56) (1.15) 3.69 (1.16) 14.1 (1.35) 5.12 (1.77) 16.7 (1.44) 7.18 (2.07) b 3 100 MSE 18.8 24.7) 4.96 (9.84) 3.89 (2.69) 2.31 (2.33) 1.08 (0.71) 1.21 (0.73) dim 11.4 (1.15) 3.53 (1.01) 14.2 (1.30) 4.88 (1.43) 16.7 (1.42) 6.62 (1.66) b 4 100 MSE 0.68 (0.37) 0.82 (0.44) 0.21 (0.11) 0.24 (0.12) 0.06 (0.03) 0.07 (0.03) dim 8.71 (2.21) 5.20 (0.92) 11.6 (2.21) 6.30 (1.32) 14.9 (2.43) 7.42 (1.41) b 4 100 MSE 7.60 (3.48) 10.8 (5.48) 2.65 (1.18) 3.40 (1.57) 0.93 (0.43) 1.17 (0.55) dim 9.17 (2.02) 5.11 (0.75) 11.8 (1.91) 6.17 (1.11) 15.2 (1.99) 7.28 (0.94)

				n = 250		n = 1000		n = 4000
				Herm	Trigo	Herm	Trigo	Herm	Trigo
	b 1 b 3	MSE	1.14 (0.91) 0.64 (0.55) 0.25 (0.13) 0.19 (0.10) 0.06 (0.03) 0.06 (0.03)
		dim	11.3 n = 250			n = 1000			n = 4000
			Herm Trigo NWO Herm Trigo NWO Herm Trigo NWO
	b 1 MSE	78.7	13.6	335	22.0	7.13	128	1.73	3.11	46.8
		std	480	10	12.5	37	28	34	2	6	11
		dim	12.5	9.1	0.13	6.0	10.3	0.10	19.2	13.7	0.08
	b 2 MSE	0.27	4.31	3.98	0.06	1.07	1.87	0.03	0.27	0.89
		std	0.6	5.1	1.9	0.09	0.5	1	0.05	0.3	0.3
		dim	2.04	4.5	0.32	2.02	5.1	0.26	2.5	5.9	0.20
	b 3 MSE	20.4	17.3	62.1	5.87	9.75	23.9	1.71	11.8	9.03
		std	22	12	25	6	34	10	0.9	39	3
		dim	12.6	5.7	0.24	15.9	9.2	0.18	19.2	16.8	0.14
	b 4 MSE	22.3	37.7	36.1	7.61	6.07	15.1	3.35	2.88	6.60
		std	28	14	14	16	4	5	14	1	1
		dim	12.2	6.83	0.19	15.8	10.0	0.15	19.0	11.2	0.12

. "MSE": MSE of the oracle (for b and b , defined by (

16

)) multiplied by 100 with standard deviations (Std) multiplied by 100 in small parenthesis. "dim": mean of the oracle dimensions with Std in small parenthesis. Columns "Herm" correspond to the Hermite basis, columns "Trigo" to the half trigonometric basis. 400 repetitions

Table 2 .

 2 "MSE": MSE multiplied by 100 for the estimation of b , obtained by GL method and defined by (

  sup t∈Sm: t n =1 t 2 n is increasing. Theorem 8.3. Let Assumption 3.1 be fulfilled. Let also Assumption 3.7(m + p) be fulfilled for every m ∈ M n . Moreover, assume that there exists κ > 0 such that E(exp(κε 2 1 )) < ∞, that

	8.9. Theorem 8.3 and its proof.
	8.9.1. Statement of Theorem 8.3.

† Laboratoire Modal'X, Université Paris Nanterre, Nanterre, France Email address: nmarie@parisnanterre.fr

where

n (t), where ν

(1) andν (2)

On the one hand, in order to apply Talagrand's inequality to sup t∈Sm:

and let us find suitable bounds on each of these random quantities.

• Bound on A m,n (X). Note that var(ε