Fabienne Comte
email: fabienne.comte@parisdescartes.fr

Jan Johannes
email: jan.johannes@uclouvain.be

Adaptive functional linear regression

Keywords: Adaptation, Model selection, Lepski's method, linear Galerkin approach, prediction, derivative estimation, minimax theory AMS 2000 subject classifications: Primary 62GJ05; secondary 62G05, 62G20

We consider the estimation of the slope function in functional linear regression, where scalar responses are modeled in dependence of random functions. Cardot and Johannes [2010] have shown that a thresholded projection estimator can attain up to a constant minimax-rates of convergence in a general framework which allows to cover the prediction problem with respect to the mean squared prediction error as well as the estimation of the slope function and its derivatives. This estimation procedure, however, requires an optimal choice of a tuning parameter with regard to certain characteristics of the slope function and the covariance operator associated with the functional regressor. As this information is usually inaccessible in practice, we investigate a fully data-driven choice of the tuning parameter which combines model selection and Lepski's method. It is inspired by the recent work of Goldenshluger and Lepski [2011]. The tuning parameter is selected as minimizer of a stochastic penalized contrast function imitating Lepski's method among a random collection of admissible values. This choice of the tuning parameter depends only on the data and we show that within the general framework the resulting data-driven thresholded projection estimator can attain minimax-rates up to a constant over a variety of classes of slope functions and covariance operators. The results are illustrated considering different configurations which cover in particular the prediction problem as well as the estimation of the slope and its derivatives.

Introduction

In functional linear regression the dependence of a real-valued response Y on the variation of a random function X is studied. Typically the functional regressor X is assumed to be squareintegrable or more generally to take its values in a separable Hilbert space H with inner product

•, • H and norm • H . Furthermore, we suppose that Y and X are centered, which simplifies the notations and that the dependence between Y and X is linear in the sense that Y = β, X H + σε, σ > 0, (1.1)

for some slope function β ∈ H and error term ε with mean zero and variance one. Assuming an independent and identically distributed (iid.) sample of (Y, X), the objective of this paper is the construction of a fully data driven estimation procedure of the slope function β which still can attain minimax-optimal rates of convergence. Functional linear models have become very important in a diverse range of disciplines, including medicine, linguistics, chemometrics as well as econometrics (see for instance [START_REF] Ramsay | Functional Data Analysis[END_REF] and [START_REF] Ferraty | Nonparametric Functional Data Analysis: Methods, Theory, Applications and Implementations[END_REF], for several case studies, or more specific, [START_REF] Forni | Let's get real: A factor analytical approach to disaggregated business cycle dynamics[END_REF] and [START_REF] Preda | Pls regression on a stochastic process[END_REF] for applications in economics). The main class of estimation procedures of the slope function studied in the statistical literature are based on principal components regression (see e.g. [START_REF] Bosq | Linear Processes in Function Spaces[END_REF], [START_REF] Frank | A statistical view of some chemometrics regression tools[END_REF], [START_REF] Cardot | Functional linear model[END_REF], [START_REF] Cardot | CLT in functional linear regression models[END_REF] or [START_REF] Müller | Generalized functional linear models[END_REF] in the context of generalized linear models). The second important class of estimators relies on minimizing a penalized least squares criterion which can be seen as generalization of the ridge regression (c.f. [START_REF] Marx | Generalized linear regression on sampled signals and curves : a p-spline approach[END_REF] and [START_REF] Cardot | Spline estimators for the functional linear model[END_REF]). More recently an estimator based on dimension reduction and threshold techniques has been proposed by [START_REF] Cardot | Thresholding projection estimators in functional linear models[END_REF] which borrows ideas from the inverse problems community [START_REF] Efromovich | On inverse problems with unknown operators[END_REF] and [START_REF] Hoffmann | Nonlinear estimation for linear inverse problems with error in the operator[END_REF]). It is worth noting that all the proposed estimation procedures rely on the choice of at least one tuning parameter, which in turn, crucially influences the attainable accuracy of the constructed estimator. It has been shown, for example in [START_REF] Cardot | Thresholding projection estimators in functional linear models[END_REF], that the attainable accuracy of an estimator of the slope β is essentially determined by a priori conditions imposed on both the slope function and the covariance operator Γ associated to the random function X (defined below). These conditions are usually captured by suitably chosen classes F ⊂ H and G of slope functions and covariance operators respectively. Typically, the class F characterizes the level of smoothness of the slope function, while the class G specifies the decay of the sequence of eigenvalues of Γ. For example, [START_REF] Hall | Methodology and convergence rates for functional linear regression[END_REF] and [START_REF] Crambes | Smoothing splines estimators for functional linear regression[END_REF] consider differentiable slope functions and a polynomial decay of the eigenvalues of Γ. Furthermore, given a weighted norm • ω and the completion F ω of H with respect to • ω we shall measure the performance of an estimator β of β by its maximal F ω -risk over a class F ⊂ F ω of slope functions and a class G of covariance operators, that is

R ω [ β; F, G] := sup β∈F sup Γ∈G E β -β 2 ω .
This general framework with appropriate choice of the weighted norm • ω allows us to cover the prediction problem with respect to the mean squared prediction error (see e.g. [START_REF] Cardot | Spline estimators for the functional linear model[END_REF] or [START_REF] Crambes | Smoothing splines estimators for functional linear regression[END_REF]) and the estimation not only of the slope function (see e.g. [START_REF] Hall | Methodology and convergence rates for functional linear regression[END_REF]) but also of its derivatives. For a detailed discussion, we refer to [START_REF] Cardot | Thresholding projection estimators in functional linear models[END_REF]. Having these applications in mind the additional condition F ⊂ F ω only means that the estimation of a derivative of the slope function necessitates its existence. Assuming an iid. sample of (Y, X) of size n obeying model (1.1) [START_REF] Cardot | Thresholding projection estimators in functional linear models[END_REF] have derived a lower bound of the maximal weighted risk, that is

R * ω [n; F, G] C inf β R ω [ β; F, G]
for some finite positive constant C where the infimum is taken over all possible estimators β. Moreover, they have shown that a thresholded projection estimator β m * n in dependence of an optimally chosen tuning parameter m * n ∈ N can attain this lower bound up to a constant C > 0,

R ω [ β m * n ; F, G] C R * ω [n; F, G],
for a variety of classes F and G. In other words, R * ω [n; F, G] is the minimax rate of convergence and β m * n is minimax-optimal. The optimal choice m * n of the tuning parameter, however, follows from a classical squared-bias-variance compromise and requires an a-priori knowledge about the classes F and G, which is usually inaccessible in practice.

In this paper we propose a fully data driven method to select a tuning parameter m in such a way that the resulting data-driven estimator β m can still attain the minimax-rate R * ω [n; F, G] up to a constant over a variety of classes F and G. It is interesting to note that, considering a linear regression model with infinitely many regressors, Goldenshluger andTsybakov [2001, 2003] propose an optimal data-driven prediction procedure allowing sharp oracle inequalities. However, a straightforward application of their results is not obvious to us since they assume a priori standardised regressors, which in turn, in functional linear regression necessitates the covariance operator Γ to be fully known in advance. In contrast, given a jointly normally distributed regressor and error term, [START_REF] Verzelen | High-dimensional Gaussian model selection on a Gaussian design[END_REF] establishes sharp oracle inequalities for the prediction problem in case the covariance operator is not known in advance. Although, it is worth noting that considering the mean prediction error as risk eliminates the ill-posedness of the underlying problem, which in turn leads to faster minimax rates of convergences of the prediction error than, for examples, of the mean integrated squared error. On the other hand covering both of these two risks within the general framework discussed above [START_REF] Comte | Adaptive estimation in circular functional linear models[END_REF] consider functional linear regression with circular functional regressor which results in a partial knowledge of the associated covariance operator, i.e. its eigenfunctions are known in advance but the eigenvalues have to be estimated. In this situation [START_REF] Comte | Adaptive estimation in circular functional linear models[END_REF] have applied successfully a model selection approach which is inspired by the work of [START_REF] Barron | Risk bounds for model selection via penalization[END_REF] now extensively discussed in [START_REF] Massart | Concentration inequalities and model selection[END_REF]. In the circular case, it is possible to develop the unknown slope function in the eigenbasis of the covariance operator, which in turn, allows to derive an orthogonal series estimator in dependence of a dimension parameter. This dimension parameter has been chosen fully data driven by a model selection approach and it is shown that the resulting data-driven orthogonal series estimator can attain minimax-optimal rates of convergence up to a constant. Although, the proof crucially relies on the possibility to write the orthogonal series estimator as a minimizer of a contrast. In this paper we do not impose an a priori knowledge of the eigenbasis and, hence the orthogonal series estimator is no more accessible to us. Instead, we consider the thresholded projection estimator β m as presented in [START_REF] Cardot | Thresholding projection estimators in functional linear models[END_REF] which we did not succeed to write as a minimizer of a contrast. Therefore, our selection method combines model selection and Lepski's method (c.f. [START_REF] Lepski | On a problem of adaptive estimation in gaussian white noise[END_REF] and its recent review in Mathé [2006]) which is inspired by a bandwidth selection method in kernel density estimation proposed recently by [START_REF] Goldenshluger | Bandwidth selection in kernel density estimation: Oracle inequalities and adaptive minimax optimality[END_REF]. Selecting the dimension parameter m as minimizer of a stochastic penalized contrast function imitating Lepski's method among a random collection of admissible values we show that the fully data-driven estimator β m can attain the minimax-rate up to a constant

C > 0, that is R ω [ β m ; F, G] C • R ω [n; F, G] (1.2)
for a variety of classes F and G. We shall emphasize that the proposed estimator can attain minimax-optimal rates without specifying in advance neither that the slope function belongs to a class of differentiable or analytic functions nor that the decay of the eigenvalues is polynomial or exponential. The only price for this flexibility is in term of the constant C which is asymptotically not equal to one, i.e. the oracle inequality (1.2) is not sharp. The paper is organized as follows: in Section 2 we briefly introduce the thresholded projection estimator β m as proposed in [START_REF] Cardot | Thresholding projection estimators in functional linear models[END_REF]. We present the data driven method to select the tuning parameter and prove a first upper risk-bound for the fully data-driven estimator β m which emphasizes the key arguments. In section 3 we review the available minimax theory as presented in [START_REF] Cardot | Thresholding projection estimators in functional linear models[END_REF]. Within this general framework we derive upper risk-bounds for the fully-data driven estimator imposing additional assumptions on the distribution of the functional regressor X and the error term ε. Namely, we suppose first that X and ε are Gaussian random variables and second that they satisfy certain moment conditions. In both cases the proof of the upper risk-bound employs the key arguments given in Section 2, while more technical aspects are deferred to the appendix. The results in this paper are illustrated considering different configurations of classes F and G. We recall the minimax-rates in this situations and show that up to a constant these rates are attained by the fully-data driven estimator.

Methodology.

Consider the functional linear model (1.1) where the random function X and the error term ε are independent. Let the centered random function X, i.e., E X, h H = 0 for all h ∈ H, have a finite second moment, i.e., E X 2 H < ∞. Multiplying both sides in (1.1) by X, h H and taking the expectation leads to the normal equation

g, h H := E[Y X, h H ] = E[ β, X H X, h H ] =: Γβ, h H , for all h ∈ H, (2.1)
where g belongs to H and Γ denotes the covariance operator associated to the random function X. Throughout the paper we shall assume that there exists a solution β ∈ H of equation (2.1) and that the covariance operator Γ is strictly positive definite which ensures the identifiability of the slope function β (c.f. [START_REF] Cardot | Spline estimators for the functional linear model[END_REF]). However, due to the finite second moment of X the associated covariance operator Γ has a finite trace, i.e. it is nuclear. Thereby, solving equation (2.1) to reconstruct the slope function β is an ill-posed inverse problem with the additional difficulty that Γ is unknown and has to be estimated (for a detailed discussion of ill-posed inverse problems in general we refer to [START_REF] Engl | Regularization of inverse problems[END_REF]).

Thresholded projection estimator

In this paper, we follow [START_REF] Cardot | Thresholding projection estimators in functional linear models[END_REF] and consider a linear Galerkin approach to derive an estimator of the slope function β. Here and subsequently, let {ψ j , j ∈ N} be a prespecified orthornormal basis in H which in general does not correspond to the eigenbasis of the operator Γ defined in (2.1). With respect to this basis, we consider for all h ∈ H the development

h = ∞ j=1 [h] j ψ j where the sequence ([h] j ) j 1 with generic elements [h] j := h, ψ j H is square- summable, i.e., h 2 H = j 1 [h] 2 j < ∞.
We will refer to any sequence (a n ) n∈N as a whole by omitting its index as for example in «the sequence a». Furthermore, given m

∈ N denote [h] m := ([h] 1 , . . . , [h] m ) t (
where x t denotes the transpose of x) and let H m be the subspace of H spanned by the first m basis functions {ψ 1 , . . . , ψ m }. Obviously, if h ∈ H m then the norm of h equals the euclidean norm of its coefficient vector

[h] m , i.e., h H = ([h] t m [h] m ) 1/2 =: [h] m with a slight abuse of notations. An element β m ∈ H m is called a Galerkin solution of equation (2.1), if g -Γβ m H g -Γ β H , ∀ β ∈ H m .
(2.2)

Since the covariance operator Γ is strictly positive definite, it follows that the (m × m)-

dimensional covariance matrix [Γ] m := E([X] m [X] t m ) associated with the m-dimensional ran- dom vector [X] m is strictly positive definite too. Consequently, the Galerkin solution β m ∈ H m is uniquely determined by [β m ] m = [Γ] -1 m [g] m and [β m ] j = 0 for all j > m.
However, the Galerkin solution does generally not correspond to the orthogonal projection of the slope function onto the subspace H m . Moreover, let (bias m ) m 1 denote a sequence of approximation errors given by bias m := sup k m β k -β ω , m 1. It is important to note that in general without further assumptions the sequence bias does not converge to zero. Here and subsequently, however, we restrict ourselves to classes F and G of slope functions and covariance operators respectively which ensure this convergence. Obviously, this is a minimal regularity condition for us since we aim to estimate the Galerkin solution. Assuming a sample {(Y i , X i )} n i=1 of (Y, X) of size n, it is natural to consider the estimators

g := 1 n n i=1 Y i X i , and Γ := 1 n n i=1 •, X i H X i for g and Γ respectively. Moreover, let [ Γ] m := 1 n n i=1 [X i ] m [X i ] t m be the empirical (m × m)-dimensional covariance matrix and note that [ g] m = 1 n n i=1 Y i [X i ] m . Replacing in (2.
2) the unknown quantities by their empirical counterparts let β m ∈ H m be a Galerkin solution satisfying

g -Γ β m H g -Γ β H , ∀ β ∈ H m .
Observe that there exists always a solution β m , but it might not be unique. Obviously

, if [ Γ] m is non singular then [ β m ] m = [ Γ] -1 m [ g] m .
We shall emphasize the multiplication with the inverse of the random matrix [ Γ] m which may result in an unstable estimator even in case

[Γ] m is well conditioned. Let ✶ { [ Γ] -1 m s n} denote the indicator function which takes the value one if [ Γ] m is non-singular with spectral norm [ Γ] -1 m s := sup z =1 [ Γ]
-1 m z of its inverse bounded by n, and the value zero otherwise. The estimator β m of β proposed by [START_REF] Cardot | Thresholding projection estimators in functional linear models[END_REF] consists in thresholding the estimated Galerkin solution, that is,

β m := β m ✶ { [ Γ] -1 m s n} .
(2.3)

In the next paragraph we introduce a data-driven method to select the dimension parameter m ∈ N.

Data-driven selection of the dimension parameter

Our selection method combines model selection (c.f. [START_REF] Barron | Risk bounds for model selection via penalization[END_REF] and its discussion in [START_REF] Massart | Concentration inequalities and model selection[END_REF]) and Lepski's method (c.f. [START_REF] Lepski | On a problem of adaptive estimation in gaussian white noise[END_REF]) borrowing ideas from [START_REF] Goldenshluger | Bandwidth selection in kernel density estimation: Oracle inequalities and adaptive minimax optimality[END_REF]. We select the dimension parameter as minimizer of a penalized contrast function depending on the weighted norm • ω which we formalize next. Let (ω j ) j 1 be a strictly positive sequence of weights. We define for h ∈ H the weighted norm by [Id] m denotes respectively the m-dimensional diagonal matrix with diagonal entries (ω j ) 1 j m and the identity matrix where for all h ∈ H m we have

h 2 ω := ∞ j=1 ω j [h] 2 j . Furthermore, for m 1, [∇ ω ] m and
h 2 ω = [h] t m [∇ ω ] m [h] m = [∇ ω ] 1/2 m [h] m 2 . Given a sequence K := ([K] k ) k 1 of matrices, denote by ∆ m (K) := max 1 k m [∇ ω ] 1/2 k [K] -1 k [∇ ω ] 1/2 k s and δ m (K) := m ∆ m (K) log(∆ m (K) ∨ (m + 2)) log(m + 2) . (2.4) Take as an example, ∆ ω m := ∆ m (K) with K = ([Id] m ) m 1 which satisfies ∆ ω m = max 1 k m ω k . For n 1, set M ω n := max 1 m ⌊n 1/4 ⌋ : ∆ ω m n .
The dimension parameter is selected among a collection of admissible values {1, . . . , M } with random integer M given by

M := min 2 m M ω n : m ∆ ω m [ Γ] -1 m s > n 1 + log n -1, (2.5)
where we set M n := M ω n if the min runs over an empty set and ⌊a⌋ denotes as usual the integer part of a. Furthermore we define a stochastic sequence of penalties ( pen m ) 1 m Mn which takes its inspiration from [START_REF] Comte | Adaptive estimation in circular functional linear models[END_REF]. Let δ m := δ m (K) with K = ([ Γ] m ) m 1 and

pen m := 14 κ σ 2 m δ m n -1 with σ 2 m := 2 1 n n i=1 Y 2 i + [ g] t m [ Γ] -1 m [ g] m (2.6)
where κ is a positive constant to be chosen below. The random integer M and the stochastic penalties ( pen m ) 1 m Mn are used to define the sequence ( Ψ m ) 1 m Mn of contrast by

Ψ m := max m k M β k -β m 2 ω -pen k .
Setting arg min m∈A {a m } := min{m : a m a m ′ , ∀m ′ ∈ A} for a sequence (a m ) m 1 with minimal value in A ⊂ N, we select the dimension parameter

m := arg min 1 m M Ψ m + pen m .
(2.7)

The estimator of β is now given by β m and below we derive an upper bound for its risk. By construction the choice of the dimension parameter and hence the estimator β m do not rely on the regularity assumptions on the slope and the operator which we formalize in Section 3.

Upper risk bound for the data-driven thresholded projection estimator

The next assertion states the key argument in the proof of the upper risk-bound.

Lemma 2.1. Let (bias m ) m 1 be the sequence of approximation errors bias m = sup m k β kβ ω . Consider an arbitrary sequence of penalties (pen m ) m 1 , an upper bound M ∈ N, and the sequence (Ψ m ) m 1 of contrasts given by Ψm := max m k M β kβ m 2 ωpen k . If the subsequence (pen 1 , . . . , pen M ) is non-decreasing, then we have for the selected model m := arg min 1 m M {Ψ m + pen m } and for all 1 m M that

β m -β 2 ω 7 pen m +78 bias 2 m +42 max m k M β k -β k 2 ω - 1 6 pen k + (2.8)
where (a) + = max(a, 0).

Proof of Lemma 2.1. From the definition of m we deduce for all 1 m M that

β m -β 2 ω 3 β m -β m∧m 2 ω + β m∧m -β m 2 ω + β m -β 2 ω 3 Ψ m + pen m +Ψ m + pen m + β m -β 2 ω 6{Ψ m + pen m } + 3 β m -β 2 ω .
(2.9) Since (pen 1 , . . . , pen M ) is non-decreasing and 4 bias 2

m max m k M β k -β m 2 ω , it is for all 1 m M easily verified that Ψ m 6 sup m k M β k -β k 2 ω - 1 6 pen k + + 12 bias 2 m .
The last estimate allows us for all 1 m M to write

β m -β 2 ω 1 3 pen m +2 bias 2 m +2 sup m k M β k -β k 2 ω - 1 6 pen k + .
From the last inequality and (2.9), we obtain the assertion (2.8), which completes the proof.

In addition to the last assertion the proof of the upper risk bound requires two assumptions which we state next. For n 1 and a positive sequence a := (a m ) m 1 denote

M n (a) := min 2 m M ω n : m ∆ ω m a m > n 1 + log n -1 (2.10)
where we set M n (a) := M ω n if the set is empty. Observe that M given in (2.5) satisfies

M = M n (a) with a = ( [ Γ] -1 m s ) m 1 . Consider for m 1, δ Γ m := δ m (K) with K = ( [Γ] -1 m s ) m 1 and pen m := κ σ 2 m δ Γ m n -1 with σ 2 m := 2 EY 2 + [g] t m [Γ] -1 m [g] m (2.11)
which are obviously only theoretical counterparts of the random objects given in (2.6). In order to control the third right hand side term in the upper bound (2.8), the remainder term, we impose the following assumption, though we show in Section 3 under reasonable assumptions on the distribution of ε and X that it holds true for a wide range of classes F and G.

Assumption 2.1. There exist sequences (m ⋄ n ) n 1 and (M + n ) n 1 , and a constant K 1 such that

sup β∈F sup Γ∈G E sup m ⋄ n k M + n β k -β k 2 ω - 1 6 pen k + K 1 n -1 for all n 1.
In the following we decompose the risk with respect to an event E n where pen is comparable to its theoretical counterpart pen and M lies between m ⋄ n and M + n given by Assumption 2.1, and its complement E c n . To be precise, we define the event

E n := pen k pen k 72 pen k , ∀ m ⋄ n k M + n ∩ m ⋄ n M M + n (2.12)
and consider the elementary identity

sup β∈F sup Γ∈G E β m -β 2 ω = sup β∈F sup Γ∈G E β m -β 2 ω ✶E n + sup β∈F sup Γ∈G E β m -β 2 ω ✶E c n (2.13)
The conditions on the distribution of ε and X presented in the next section are also sufficient to show that the following assumption holds true.

Assumption 2.2. There exists a constant K 2 > 0 such that

sup β∈F sup Γ∈G E β m -β 2 ω ✶E c n K 2 n -1 for all n 1.
The next assertion provides an upper bound for the maximal F ω -risk over the classes F and G of the thresholded projection estimator β m with data-driven choice m given by (2.7).

Proposition 2.2. If Assumption 2.1 and 2.2 hold true, then we have

R ω [ β m ; F, G] 504 sup β∈F sup Γ∈G {pen m ⋄ n + bias 2 m ⋄ n } + (504 K 1 + K 2 )n -1 for all n 1.
Proof of Proposition 2.2. We make use of the elementary identity (2.13) and taking into account Assumption 2.2 we derive for all n 1

R ω [ β m ; F, G] sup β∈F sup Γ∈G E β m -β 2 ω ✶E n + K 2 n -1 .
(2.14)

We observe that the random subsequences ( σ 2 1 , . . . , σ 2

M

) and hence ( pen 1 , . . . , pen M ) are by construction monotonically non-decreasing. Indeed, for all

1 m k M the identity Γ( β k -β m ), ( β k -β m ) H = [ g] t k [ Γ] -1 k [ g] k -[ g] t m [ Γ] -1 m [ g] m holds true. Therefore, by using that Γ is positive definite it follows that [ g] t m [ Γ] -1 m [ g] m [ g] t k [ Γ] -1 k [ g] k , and hence σ 2 m σ 2 k . Consequently, Lemma 2.1 is applicable for all 1 m M and we obtain β m -β 2 ω 7 pen m +78 bias 2 m +42 max m k M β k -β k 2 ω - 1 6 pen k + .
On the event E n defined in (2.12) we deduce from the last bound that for all n 1

β m -β 2 ω ✶E n 504 pen m ⋄ n +78 bias 2 m ⋄ n +42 sup m ⋄ n k M + n β k -β k 2 ω - 1 6 pen k +
which by taking into account Assumption 2.1 implies that

sup β∈F sup Γ∈G E β m -β 2 ω ✶E n 504 sup β∈F sup Γ∈G {pen m ⋄ n + bias 2 m ⋄ n } + 504 K 1 n -1 for all n 1.
We obtain the claim of the proposition by combination of the last bound and (2.14).

Remark 2.1. The upper risk-bound given in the last assertion is strongly reminiscent of a variance/squared-biased decomposition of the F ω -risk associated with the estimator β m ⋄ n employing the dimension parameter m ⋄ n . Indeed, in many cases the penalty term pen m is in the same order as the variance of the estimator β m (c.f. Illustration 3.1 [P-P] and [E-P] below). In this situation we obviously wish that the parameter m ⋄ n just realize the balance between both the variance and the squared-biased term which in many cases can lead to an optimal estimation procedure. However, the construction of the penalty term is more involved to ensure that Assumption 2.1 and 2.2 can be satisfied (c.f. Illustration 3.1 [P-E]).

Minimax-optimality

In this section we recall first a general framework proposed by [START_REF] Cardot | Thresholding projection estimators in functional linear models[END_REF] which allows to derive minimax-optimal rates for the maximal F ω -risk, sup β∈F sup Γ∈G E β -β 2 ω , over the classes F and G. Placing us into this framework, we can derive the main results of this paper which state that the proposed data-driven procedure indeed can attain these minimax-rates.

Notations and basic assumptions

The additional regularity conditions β ∈ F and Γ ∈ G imposed on the slope function and the covariance operator, respectively, are characterized by different weighted norms in H with respect to the pre-specified orthonormal basis {ψ j , j ∈ N} in H, which we formalize now. Given a strictly positive sequence of weights b = (b j ) j 1 and a radius r > 0, let F b be the completion of H with respect to the weighted norm • b , then we consider in the following the ellipsoid

F r b := h ∈ F b : h 2 b
r as class of possible slope functions. Furthermore, as usual in the context of ill-posed inverse problems, we link the mapping properties of the covariance operator Γ and the regularity condition β ∈ F r b . Therefore, consider the sequence ( Γψ j , ψ j ) j 1 which sums up to E X 2 H , i.e. Γ is nuclear, and hence converges to zero. In what follows we impose restrictions on the decay of this sequence. Denote by N the set of all strictly positive nuclear operators defined on H. Given a strictly positive sequence of weights γ and a constant d 1 define the class G d γ ⊂ N of covariance operators by

G d γ := T ∈ N : f 2 γ 2 /d 2 T f 2 d 2 f 2 γ 2 , ∀f ∈ H
where arithmetic operations on sequences are defined element-wise, e.g. γ 2 = (γ 2 j ) j 1 . Let us briefly discuss the last definition. If T ∈ G d γ , then we have d -1 T ψ j , ψ j /γ j d, for all j 1. Consequently, the sequence γ is necessarily summable, because T is nuclear. Moreover, if λ denotes the sequence of eigenvalues of T then d -1 λ j /γ j d, for all j 1. In other words the sequence γ characterizes the decay of the eigenvalues of T ∈ G d γ . We do not specify the sequences of weights ω, b and γ, but impose from now on the following minimal regularity conditions.

Assumption 3.1. Let (ω j ) j 1 , (b j ) j 1 , and (γ j ) j 1 be strictly positive sequences of weights with b 1 = 1, ω 1 = 1, γ 1 = 1, and ∞ j=1 γ j < ∞ such that the sequences b -1 , ωb -1 , γ, and γ 2 ω -1 are monotonically non-increasing and converging to zero.

The last assumption is fairly mild. For example, assuming that ωb -1 is non-increasing, ensures that F r b ⊂ F ω . Furthermore, it is shown in [START_REF] Cardot | Thresholding projection estimators in functional linear models[END_REF] that the minimax rate

R * ω [n; F r b , G d γ ]
is of order n -1 for all sequences γ and ω such that γ 2 ω -1 is nondecreasing. We will illustrate all our results considering the following three configurations for the sequences ω, b and γ.

Illustration 3.1. In all three cases, we take ω j = j 2s , j 1. Moreover, let [P-P] b j = j 2p and γ j = j -2a , j 1, with p > 0, a > 1/2, and p > s > -2a;

[E-P] b j = exp(j 2p -1) and γ j = j -2a , j 1, with p > 0, a > 1/2, and s > -2a;

[P-E] b j = j 2p and γ j = exp(-j 2a + 1), j 1, with p > 0, a > 0, and p > s;

then Assumption 3.1 is satisfied in all cases.

Remark 3.1. In the configurations [P-P] and [E-P], the case s = -a can be interpreted as mean-prediction error (c.f. [START_REF] Cardot | Thresholding projection estimators in functional linear models[END_REF]). Moreover, if {ψ j } is the trigonometric basis and the value of s is an integer, then the weighted norm h ω corresponds to the L 2 -norm of the weak s-th derivative of h (c.f. [START_REF] Neubauer | When do Sobolev spaces form a Hilbert scale[END_REF]). In other words in this situation we consider as risk the mean integrated squared error when estimating the s-th derivative of β. Moreover, in the configurations [P-P] and [P-E], the additional condition p > s means that the slope function has at least p s + 1 weak derivatives, while for a value p > 1 in [E-P], the slope function is assumed to be an analytic function (c.f. [START_REF] Kawata | Fourier analysis in probability theory[END_REF]).

Minimax optimal estimation reviewed

Let us first recall a lower bound of the maximal F ω -risk over the classes F r b and G d γ due to [START_REF] Cardot | Thresholding projection estimators in functional linear models[END_REF]. Given an i.i.d. sample of (Y, X) of size n and sequences ω, b and γ satisfying Assumption 3.1 define

m * n := arg min m 1    max   ω m b m , m j=1 ω j nγ j      and R * n := max   ω m * n b m * n , m * n j=1 ω j nγ j   . (3.1) If in addition ξ := inf n 1 {(R * n ) -1 min(ω m * n b -1 m * n , m * n j=1 ω j (nγ j ) -1
)} > 0, then there exists a constant C := C(σ, r, d, ξ) > 0 depending on σ, r, d and ξ only such that

inf β R * ω [ β; F r b , G d γ ] C R * n for all n 1. (3.2)
On the other hand considering the dimension parameter m * n given in (3.1) [START_REF] Cardot | Thresholding projection estimators in functional linear models[END_REF] have shown that the maximal risk

R * ω [ β m * n ; F r b , G d γ ] of the estimator β m * n defined in (2.
3) is bounded by R * n up to constant for a wide range of sequences ω, b, and γ, provided the random function X and the error ε satisfy certain additional moment conditions. In other words

R * n = R * ω [n; F r b , G d γ ]
is the minimax-rate in this situation and the estimator β m * n is minimax optimal. Although, the definition of the dimension parameter m * n necessitates an a-priori knowledge of the sequences b and γ. In the remaining part of this paper we show that the data-driven choice of the dimension parameter constructed in Section 2 can automatically attain the minimax-rate R * n for a variety of sequences ω, b, and γ. Before, let us briefly illustrate the minimax result.

Illustration (continued) 3.2. Considering the three configurations (see Illustration 3.1), it has been shown in [START_REF] Cardot | Thresholding projection estimators in functional linear models[END_REF] that the estimator β m * n with m * n as given below attains the rate R * n up to a constant. We write for two strictly positive sequences

(a n ) n 1 and (b n ) n 1 that a n ∼ b n , if (a n /b n ) n 1
is bounded away from 0 and infinity.

[P-P] It is easily seen that m * n ∼ n 1/(2p+2a+1) if 2s+2a+1 > 0, m * n ∼ n 1/[2(p-s)] if 2s+2a+1 < 0 and m * n ∼ (n/ log n) 1/[2(p-s)] if 2a + 2s + 1 = 0, which in turn implies that R * n ∼ max(n -(2p-2s)/(2a+2p+1) , n -1 ), if 2s + 2a + 1 = 0 (and R * n ∼ log(n)/n if 2s + 2a + 1 = 0).
Observe that an increasing value of a leads to a slower minimax-rate R * n . Therefore, the parameter a is called degree of ill-posedness (c.f. [START_REF] Natterer | Error bounds for Tikhonov regularization in Hilbert scales[END_REF]).

[E-P] If 2a+2s+1 > 0, then m * n ∼ (log n-2a+1 2p log(log n)) 1/(2p) and R * n ∼ n -1 (log n) (2a+1+2s)/(2p) . Furthermore, if 2a + 2s + 1 < 0, then m * n ∼ (log n + (s/p) log(log n)) 1/(2p) and R * n ∼ n -1 , while R * n ∼ log(log n)/n if 2a + 2s + 1 = 0. [P-E] We have m * n ∼ (log n -2p+(2a-1) + 2a log(log n)) 1/(2a) . Thereby, R * n ∼ (log n) -(p-s)/a
. The parameter a reflects again the degree of ill-posedness since an increasing value of a leads also here to a slower minimax-rate R * n .

Minimax-optimality of the data-driven estimation procedure

Consider the thresholded projection estimator β m with data-driven choice m of the dimension parameter. Supposing that the joint distribution of the random function X and the error term ε satisfies certain additional conditions, we will prove below that the Assumptions 2.1 and 2.2 formulated in Section 2 hold true. These assumptions rely on the existence of sequences

(m ⋄ n ) n 1 and (M + n ) n
m ⋄ n := arg min 1 m M - n max ω m b m , δ γ m n and R ⋄ n := max ω m ⋄ n b m ⋄ n , δ γ m ⋄ n n satisfying m ⋄ n M - n M + n . Furthermore, let Σ := Σ(G d γ ) denote a finite constant such that Σ j 1 γ j and Σ m 1 ∆ γ m exp - 1 16(1 + log(d)) m log(∆ γ m ∨ (m + 2)) log(m + 2) (3.3)
which by construction always exists and depends on the class G d γ only. Let us illustrate the last definition by revisiting the three configurations for the sequences ω, b, and γ (Illustration 3.1).

Illustration (continued) 3.3. In the following we state the order of M - n and δ γ m which in turn are used to derive the order of m ⋄ n and R ⋄ n .

[

P-P] M - n ∼ n 1+log n 1+2a+(2s) + , δ γ m ∼ m 1+(2a+2s) + , and for all p > (s) + it follows m ⋄ n ∼ m 1/[1+2p-2s+(2a+2s) + and R ⋄ n ∼ n -2(p-s)/[1+2p-2s+(2a+2s) + ] ; [E-P] M - n ∼ n 1+log n 1+2a+(2s) + , δ γ m ∼ m 1+(2a+2s) + , and for all p > 0 it follows m ⋄ n ∼ (log n - 1+2(a+s) + -2s 2p log(log n)) 1/(2p) and R ⋄ n ∼ n -1 (log n) [1+2(a+s) + ]/(2p) ; [P-E] M - n ∼ (log n -1+2a+2(s) + 2a log(log n)) 1/(2a) , δ γ m ∼ m 1+2s+2a exp(m 2a
), and for all p > (s

) + it follows m ⋄ n ∼ (log n -1+2a+2p 2a log(log n)) 1/(2a) and R ⋄ n ∼ (log n) -(p-s)/a .
We proceed by formalizing the additional conditions on the joint distribution of ε and X which in turn are used to prove that the Assumptions 2.1 and 2.2 hold true.

Imposing a joint normal distribution. Let us first assume that X is a centered Gaussian H-valued random variable, that is, for all k 1 and for all finite collections {h 1 , . . . , h k } ⊂ H the joint distribution of the real valued random variables X, h 1 H , . . . , X, h k H is Gaussian with zero mean vector and covariance matrix with generic elements E h j , X H X, h l H , 1 j, l k. Moreover, suppose that the error term is standard normally distributed. The next assumption summarizes this situation.

Assumption 3.2. The joint distribution of the random function X and the error ε is normal.

The proof of the next assertion is more involved and hence deferred to Appendix C.

:= C 1 (σ 2 + r) Σ and K 2 := C 2 (σ 2 + r) Σ respectively.
By taking the value κ = 96 the random penalty pen and the random upper bound M given in (2.6) and (2.5) respectively depend indeed only on the data, and hence the choice m of the dimension parameter in (2.7) is fully data-driven. Moreover due to the last assertion we can apply Proposition 2.2 which in turn provides the key argument to prove the following upper risk-bound for the data-driven thresholded projection estimator β m with m given by (2.7).

Theorem 3.2. Let the assumptions of Proposition 3.1 be satisfied. There exists a finite constant

K := K(d) depending on d only such that R ω [ β m , F r b , G d γ ] K (σ 2 + r) R ⋄ n + Σ n -1 for all n 1.
Proof of Theorem 3.2. We shall provide in the appendix among others, the two technical Lemmas B.1 and B.2 which are used in the following. Moreover, denote by K := K(d) a constant depending on d only which changes from line to line. Making use of Proposition 3.1, i.e., Assumptions 2.1 and 2.2 are satisfied, we can apply Proposition 2.2, and hence for all n 1

R ω [ β m , F r b , G d γ ] 504 sup β∈F r b sup Γ∈G d γ {pen m ⋄ n + bias 2 m ⋄ n } + K (σ 2 + r) Σ n -1 . (3.4) Furthermore, if β ∈ F r b and Γ ∈ G d γ then firstly from (B.4) in Lemma B.1 follows that bias 2 m ⋄ n 34 d 8 r ω m ⋄ n b -1 m ⋄
n because γ 2 ω -1 and ωb -1 are non increasing due to Assumption 3.1. Secondly, by combination of (i) and (iv) in Lemma B.2, it is easily verified that pen

m ⋄ K (σ 2 +r)δ γ m ⋄ n -1 . Consequently, sup β∈F r b sup Γ∈G d γ {pen m ⋄ n + bias 2 m ⋄ n } K (σ 2 + r) R ⋄
n for all n 1 by combination of the last two estimates and the definition of R ⋄ n which in turn together with the upper bound (3.4) implies the assertion of the theorem.

Imposing moment conditions. We dismiss now the Assumption 3.2 and formalize in place conditions on the moments of the random function X and the error term ε. In particular we use that for all h ∈ H with Γh, h = 1, the random variable h, X is standardized, i.e. has mean zero and variance one.

Assumption 3.3. There exist a finite integer k 16 and a finite constant η 1 such that E|ε| 4k η 4k and that for all h ∈ H with Γh, h = 1 the standardized random variable h, X satisfies E| h, X | 4k η 4k .

It is worth noting that for any Gaussian random function X with finite second moment Assumption 3.3 holds true, since for all h ∈ H with Γh, h = 1 the random variable h, X is standard normally distributed and hence E| h, X | 2k = (2k-1)•. . .•5•3•1. The proof of the next assertion is again rather involved and deferred to Appendix D. It follows, however, along the general lines of the proof of Proposition 2.2 though it is not a straightforward extension. Take as an example the concentration inequality for the random variable

[Γ] 1/2 m ([ g] m -[ Γ] m [β m ] m ) in Lemma C.
K 1 := C 1 η 64 (σ 2 + r) Σ and K 2 := C 2 η 64 (σ 2 + r) Σ respectively.
We remark a change only in the constants when comparing the last proposition with Proposition 3.1. Note further that we need a larger value for the constant κ than in Proposition 3.1 although it is still a numerical constant and hence the choice m given by (2.7) is again fully data-driven. Moreover, both values for the constant κ, though convenient for deriving the theory, may be far too large in practice and instead be determined by means of a simulation study as in [START_REF] Comte | Penalized contrast estimator for density deconvolution[END_REF], for example. The next assertion provides an upper risk-bound for the data-driven thresholded projection estimator β m when imposing moment conditions.

R ω [ β m , F r b , G d γ ] K (σ 2 + r) R ⋄ n + K ′ η 64 Σ n -1 for all n 1.
⋄ := sup n 1 {R ⋄ n /R * n } < ∞ holds true, then we have for all n 1 R ω [ β m ; F r b , G d γ ] C • inf β R ω [ β; F r b , G d γ ]
where the infimum is taken over all possible estimators β and C is a finite positive constant.

Remark 3.2. In the last assertion ξ ⋄ = sup n 1 {R ⋄ n /R * n } < ∞ is for example satisfied if the following two conditions hold simultaneously true: (i) m * n M - n for all n 1 and (ii)

∆ γ m = max 1 j m ω j γ -1 j Cm -1 m j=1 ω j γ -1 j and log(∆ γ m ∨ (m + 2)) C log(m + 2) for all m 1. Observe that (ii) which implies δ γ m C m j=1 ω j γ j is satisfied in case ∆ γ
m is in the order of a power of m (e.g. Illustration 3.2 [P-P] and [E-P]). If this term has an exponential order with respect to m (e.g. Illustration 3.2 [P-E]), then a deterioration of the term δ γ m compared to the variance term m j=1 ω j γ j is possible. However, no loss in terms of the rate may occur, i.e., ξ ⋄ < ∞, when the squared-bias term ω m ⋄ n b -1 m ⋄ n dominates the variance term n -1 δ γ m ⋄ n (for a detailed discussion in a deconvolution context we refer to Butucea and Tsybakov [2007a,b]).

Let us illustrate the performance of the data-driven thresholded projection estimator β m considering the three configurations for the sequences ω, b, and γ (see Illustration 3.1 above).

Proposition 3.6. Assume an iid. n-sample of (Y, X) satisfying (1.1) and let either Assumption 3.2 or Assumption 3.3 hold true where we set respectively κ = 96 or κ = 288 in (2.6). The fully data-driven estimator β m attains the minimax-rates R * n (see Illustration 3.2), up to a constant, in the three cases introduced in the Illustration 3.1, if we additionally assume a+s 0 in the cases [P-P] and [E-P].

Proof of Proposition 3.6. Under the stated conditions it is easily verified that the assumptions of either Theorem 3.2 or Theorem 3.4 are satisfied. Moreover, the rates R * n (Illustration 3.2) and R ⋄ n (Illustration 3.3) are of the same order if we additionally assume a + s 0 in the cases [P-P] and [E-P]. Therefore we can apply Corollary 3.5 which implies the assertion.

Appendix

This section gathers preliminary technical results and the proofs of Proposition 3.1 and 3.3.

A Notations

We begin by defining and recalling notations to be used in all proofs. Given m 1, H m denotes the subspace of H spanned by the functions {ψ 1 , . . . , ψ m }. Π m and Π ⊥ m denote the orthogonal projections on H m and its orthogonal complement H ⊥ m , respectively. If K is an operator mapping H to itself and if we restrict Π m KΠ m to an operator from H m to itself, then it can be represented by a matrix [K] m with generic entries ψ j , Kψ l H =: [K] j,l for 1 j, l m. The spectral norm of [K] m is denoted by [K] m s and the inverse matrix of [K] m by [K] -1 m . Furthermore, keeping in mind the notations given in (2.4) and (2.10) we use for all m 1 and n 1

∆ ω m = ∆ m ([∇ ω ]), ∆ Γ m = ∆ m ([Γ]), Λ Γ m := log(∆ Γ m ∨ (m + 2)) log(m + 2) , δ Γ m = m∆ Γ m Λ Γ m = δ m ([Γ]), ∆ γ m = max 1 k m ω j γ -1 j = ∆ m ([∇ γ ]), Λ γ m := log(∆ γ m ∨ (m + 2)) log(m + 2) , δ γ m = m∆ γ m Λ γ m = δ m ([∇ γ ]), ∆ m = ∆ m ([ Γ]), Λ m := log( ∆ m ∨ (m + 2)) log(m + 2) , δ m := m ∆ m Λ m = δ m ([ Γ]), M = M n ( [ Γ] -1 m s ) m 1 , M - n = M n (16d 3 γ -1 ), M + n = M n ([4dγ] -1 ), pen m = κσ 2 m m∆ Γ m Λ Γ m n -1 and pen m = 14κ σ 2 m m ∆ m Λ m n -1 . (A.1) Recall that [ Γ] m = 1 n n i=1 [X i ] m [X i ] t m and [ g] m = 1 n n i=1 Y i [X i ] m where [Γ] m = E[X] m [X] t m and [g] m = EY [X] m . Given a Galerkin solution β m ∈ H m , m 1, of equation (1.2), let Z m := Y -β m , X H = σε+ β-β m , X H , and denote ρ 2 m := EZ 2 m = σ 2 + Γ(β-β m ), (β-β m ) H , σ 2 Y := EY 2 = σ 2 + Γβ, β H and σ 2 m = 2 σ 2 Y + [g] t m [Γ] -1 m [g] m
where we used that ε and X are uncorrelated. Define the random matrix [Ξ] m and random vector [W ] m respectively by

[Ξ] m := [Γ] -1/2 m [ Γ] m [Γ] -1/2 m -[Id] m , and [W ] m := [ g] m -[ Γ] m [β m ] m , where E[Ξ] m = 0, because E[ Γ] m = [Γ] m , and E[W ] m = [Γ(β -β m )] m = 0. Define further σ 2 Y := n -1 n i=1 Y 2 i , the events Ω m,n := { [ Γ] -1 m s n}, ℧ m,n := {8 [Ξ] m s 1}, A n := {1/2 σ 2 Y /σ 2 Y 3/2}, B n := { [Ξ] k s 1/8, ∀1 k M ω n }, C n := {[W ] t k [Γ] -1 k [W ] k 1 8 ([g] t k [Γ] -1 k [g] k + σ 2 Y ), ∀1 k M ω n }, (A.2)
and their complements Ω c m,n , ℧ c m,n , A c n , B c n , and C c n , respectively. Furthermore, we will denote by C universal numerical constants and by C(•) constants depending only on the arguments. In both cases, the values of the constants may change from line to line.

B Preliminary results

This section gathers preliminary results where we only exploit that the sequences ω, b and γ satisfy Assumption 3.1. The proof of the next lemma can be found in [START_REF] Johannes | On rate optimal local estimation in functional linear model[END_REF].

Lemma B.1. Let Γ ∈ G d
γ where the sequence γ satisfies Assumption 3.1, then we have

sup m∈N γ m [Γ] -1 m s 4d 3 , (B.1) sup m∈N [∇ γ ] 1/2 m [Γ] -1 m [∇ γ ] 1/2 m s 4d 3 , (B.2) sup m∈N [∇ γ ] -1/2 m [Γ] m [∇ γ ] -1/2
(i) d -1 γ m [Γ] -1 m s D, d -1 ∆ Γ m /∆ γ m D, (1 + log d) -1 Λ Γ m /Λ γ m
(1 + log D), and [Γ] -1 m s γ m D. Moreover, the monotonicity of γ implies

d -1 (1 + log d) -1 δ Γ m /δ γ m D(1 + log D) for all m 1, (ii) δ γ M + n n4D(1 + log D) and δ Γ M + n n4D 2 (1 + 2 log D) for n 1, (iii) n 2 max 1 m M + n [Γ] -1 m if n 2D and ∆ ω M + n M + n (1 + log n) 8D 2 . If in addition β ∈ F
d -1 γ M max 1 m M [Γ] -1 m s D.
From these estimates we obtain (i). Proof of (ii). Observe that

∆ γ M + n ∆ ω M + n γ -1 M + n . In case M + n = 1 the assertion is trivial, since ∆ ω 1 γ -1 1 = 1 due to Assumption 3.1. Thus, consider M ω n M + n > 1, which implies min 1 j M + n {γ j (j∆ ω j ) -1 } (1+log n)(4Dn) -1 , and hence M + n ∆ γ M + n 4Dn(1+log n) -1 , Λ γ M + n (1 + log D)(1 + log n), M + n ∆ Γ M + n 4D 2 n(1 + log n) -1 and Λ Γ M + n (1 + 2 log D)(1 + log n).
The assertion (ii) follows now by combination of these estimates. Proof of (iii). By employing that Dγ -1

M + n max 1 m M + n [Γ] -1 m , the assertion (iii) follows in case M + n = 1 from γ 1 = 1, while in case M + n > 1, we use M + n ∆ ω M + n γ -1 M + n 4Dn(1 + log n) -1 . Proof of (iv). Since ε and X are centered it follows from [β m ] m = [Γ] -1 m [g] m that ρ 2 m 2 EY 2 + E| β m , X H | 2 = 2 σ 2 Y + [g] t m [Γ] -1 m [g] m = σ 2 m .
Moreover, by employing successively the inequality of [START_REF] Heinz | Beiträge zur störungstheorie der spektralzerlegung[END_REF], i.e. Γ 1/2 β 2 d β 2 γ , and Assumption 3.1, i.e., γ and b -1 are non-increasing, the identity

σ 2 Y = σ 2 + Γβ, β H implies σ 2 Y σ 2 + d β 2 γ σ 2 + dr. (B.6)
Furthermore, from (B.3) and (B.5) in Lemma B.1 we obtain

[g] t m [Γ] -1 m [g] m d β m 2 γ 34d 9 r. (B.7)
The assertion (iv) follows now from (B.6) and (B.7), which completes the proof.

Lemma B.3. For all n, m 1 we have

1 4 < [ Γ] -1 m s [Γ] -1 m s 4, ∀ 1 m M ω n ⊂ M - n M M + n . Proof of Lemma B.3. Let τ m = [ Γ] -1 m -1
s and recall that 1 M M ω n with

M = M =              τ M +1 (M +1)∆ ω M +1 < 1+log n n , M = 1, min 2 m M τm m∆ ω m 1+log n n τ M +1 (M +1)∆ ω M +1 < 1+log n n , 1 M < M ω n , min 2 m M τm m∆ ω m 1+log n n , M = M ω n .
Given

τ m := [Γ] -1 m -1 s we have D -1 τ m /γ m d, m 1 due to (i) in Lemma B
.2 which we use to prove the following two assertions

M < M - n ⊂ min 1 m M ω n : τ m τ m < 1 4 , (B.8) M > M + n ⊂ max 1 m M ω n τ m τ m 4 . (B.9)
Obviously, the assertion of Lemma B.3 follows now by combination of (B.8) and (B.9). Consider (B.8) which is trivial in case

M - n = 1. If M - n > 1 we have min 1 m M - n γm mω + m 4D(1+log n) n
and, hence min

1 m M - n τm m∆ ω m 4(1+log n) n
. By exploiting the last estimate we obtain

M < M ω n ∩ M < M - n = M - n -1 M =1 M = M ⊂ M - n -1 M =1 τ M +1 (M + 1)∆ ω M +1 < 1 + log n n = min 2 m M - n τ m m∆ ω m < 1 + log n n ⊂ min 1 m M - n τ m τ m < 1/4 while trivially M = M ω n ∩ M < M - n = ∅, which proves (B.8) because M - n M ω n . Consider (B.9) which is trivial in case M + n = M ω n . If M + n < M ω n , then τ M + n +1 (M + n +1)∆ ω M + n +1 < (1+log n) 4n
, and hence

M > 1 ∩ M > M + n = M ω n M =M + n +1 M = M ⊂ M ω n M =M + n +1 min 2 m M τ m m∆ ω m 1 + log n n = min 2 m (M + n +1) τ m m∆ ω m 1 + log n n ⊂ τ M + n +1 τ M + n +1 4 while trivially { M = 1} ∩ { M > M + n } = ∅ which shows (B.9
) and completes the proof.

Lemma B.4. Let A n , B n and C n as in (A.2). For all n 1 it holds true that

A n ∩ B n ∩ C n ⊂ {pen k pen k pen k , 1 k M ω n } ∩ {M - n M M + n }. Proof of Lemma B.4. Let M ω n k 1. If [Ξ] k s 1/8, i.e. on the event B n , it is easily verified that ([Id] k + [Ξ] k ) -1 -[Id] k s 1/7 which we exploit to conclude 6/7 [∇ ω ] 1/2 k [Γ] -1 k [∇ ω ] 1/2 k s [∇ ω ] 1/2 k [ Γ] -1 k [∇ ω ] 1/2 k s 8/7 [∇ ω ] 1/2 k [Γ] -1 k [∇ ω ] 1/2 k s , 6/7 [Γ] -1 k s [ Γ] -1 k s 8/7 [Γ] -1 k s and 6/7x t [Γ] -1 k x x t [ Γ] -1 k x 8/7x t [Γ] -1 k x, for all x ∈ R k , (B.10)
and, consequently

(6/7)[ g] t k [Γ] -1 k [ g] k [ g] t k [ Γ] -1 k [ g] k (8/7)[ g] t k [Γ] -1 k [ g] k . (B.11)
Moreover, from [Ξ] k s 1/8 we obtain after some algebra,

[g] t k [Γ] -1 k [g] k 1 16 [g] t k [Γ] -1 k [g] k + 4[W ] k [Γ] -1 k [W ] k + 2[ g] t k [Γ] -1 k [ g] k , [ g] t k [Γ] -1 k [ g] k 33 16 [g] t k [Γ] -1 k [g] k + 4[W ] k [Γ] -1 k [W ] k .
Combining each of these estimates with (B.11) yields

(15/16)[g] t k [Γ] -1 k [g] k 4[W ] k [Γ] -1 k [W ] k + (7/3)[ g] t k [ Γ] -1 k [ g] k , (7/8)[ g] t k [ Γ] -1 k [ g] k (33/16)[g] t k [Γ] -1 k [g] k + 4[W ] k [Γ] -1 k [W ] k . If in addition [W ] t k [Γ] -1 k [W ] k 1 8 ([g] t k [Γ] -1 k [g] k + σ 2 Y ), i.e.
, on the event C n , then the last two estimates imply respectively

(7/16)([g] t k [Γ] -1 k [g] k + σ 2 Y ) (15/16)σ 2 Y + (7/3)[ g] t k [ Γ] -1 k [ g] k , (7/8)[ g] t k [ Γ] -1 k [ g] k (41/16)[g] t k [Γ] -1 k [g] k + (1/2)σ 2 Y ,
and hence in case 1/2 σ 2 Y /σ 2 Y 3/2, i.e., on the event A n , we obtain

(7/16)([g] t k [Γ] -1 k [g] k + σ 2 Y ) (15/8) σ 2 Y + (7/3)[ g] t k [ Γ] -1 k [ g] k , (7/8)([ g] t k [ Γ] -1 k [ g] k + σ 2 Y ) (41/16)[g] t k [Γ] -1 k [g] k + (29/16)σ 2 Y .
Combining the last two estimates we have

1 6 (2[g] t k [Γ] -1 k [g] k + 2σ 2 Y ) (2[ g] t k [ Γ] -1 k [ g] k + 2 σ 2 Y ) 3(2[g] t k [Γ] -1 k [g] k + 2σ 2 Y ).
Since on the event A n ∩ B n ∩ C n the last estimate and (B.10) hold for all 1 k M ω n it follows

A n ∩ B n ∩ C n ⊂ (1/6)σ 2 m σ 2 m 3σ 2 m and (6/7)∆ Γ m ∆ m (8/7)∆ Γ m , ∀1 m M ω n . From Λ m = log( ∆m∨(m+2)) log(m+2)
it is easily seen that (6/7) ∆ m /∆ Γ m (8/7) implies

1/2 (1 + log(7/6)) -1 Λ m /Λ Γ m (1 + log(8/7)) 3/2.
Taking into account the last estimates and the definitions

pen m = κσ 2 m m∆ Γ m Λ Γ m n -1 and pen m = 14κ σ 2 m m ∆ m Λ m n -1 we obtain A n ∩ B n ∩ C n ⊂ {pen m pen m 72 pen m , ∀1 m M ω n }. (B.12)
On the other hand, by exploiting successively (B.10) and Lemma B.3 we have

A n ∩ B n ∩ C n ⊂ 6 7 [ Γ] -1 m s [Γ] -1 m s 8 7 , ∀1 m M ω n ⊂ M - n M M + n . (B.13)
From (B.12) and (B.13) follows the assertion of the lemma, which completes the proof.

Lemma B.5. For all m, n 1 with n (8/7) [Γ] -1 m s we have ℧ m,n ⊂ Ω m,n .

Proof of Lemma B.5. Taking into account the identity

[ Γ] m = [Γ] 1/2 m {[Id] m + [Ξ] m }[Γ] 1/2 m we observe that [Ξ] m s 1/8 implies [ Γ] -1 m s (8/7) [Γ] -1 m s due to the usual Neumann series argument. If n (8/7) [Γ] -1
m s , then the last assertion implies ℧ m,n ⊂ Ω m,n , which proves the lemma.

C Proof of Proposition 3.1

We will suppose throughout this section that the conditions of Proposition 3.1 are satisfied and thus Assumption 3.1 particularly holds true which allows us to employ the Lemmas B.1-B.5 stated in Section B. Moreover, we show first technical assertions (Lemma C.1-C.5) where we exploit Assumption 3.2, i.e. X and ε are jointly normally distributed. They are used below to prove that the Assumptions 2.1 and 2.2 are satisfied (Proposition C.6 and C.7 respectively), which is the claim of Proposition 3.1. We begin by recalling elementary properties due to the Assumption 3.2 which are frequently used in this section. Given f ∈ H the random variable f, X H is normally distributed with mean zero and variance Γf, f H . Consider the Galerkin solution β m and h ∈ H m then the random variables ββ m , X H and h, X H are independent. Thereby, Z m = Yβ m , X H = σε + ββ m , X H and [X] m are independent, normally distributed with mean zero, and, respectively, variance ρ 2 m and covariance matrix

[Γ] m . Consequently, (ρ -1 m Z m , [X] t m [Γ] -1/2 m
) is a (m + 1)-dimensional vector of iid. standard normally distributed random variables. Let us further state elementary inequalities for Gaussian random variables.

Lemma C.1. Let {U i , V ij , 1
i n, 1 j m} be independent and standard normally distributed random variables. Then we have for all η > 0 and ζ 4m/n

P n -1/2 n i=1 (U 2 i -1) η exp - 1 8 η 2 1 + η n -1/2 ; (C.1) P n -1 n i=1 U i V i1 η ηn 1/2 + 1 ηn 1/2 exp - n 4 min η 2 , 1/4 ; (C.2) P   n -2 m j=1 n i=1 U i V ij 2 ζ   exp -n 16 + exp -ζn 64 ; (C.3)
and for all c 1 and a 1 , . . . , a m 0 that

E n i=1 U 2 i -2 c n + 16 exp -c n 16 (C.4) E   m j=1 n -1/2 n i=1 U i V ij 2 -4 c m   + 16 exp -c m 16 + 32 c m n exp -n 16 (C.5) E   m j=1 a j n i=1 U i V ij 2   2 = n(n + 2) m j=1 a 2 j + m j=1 a j ) 2 (C.6) Proof of Lemma C.1. Define W := n i=1 U 2 i and Z j := ( n i=1 U 2 i ) -1/2 n i=1 U i V ij . Obvi- ously, W has χ 2
n distribution with n degrees of freedom and Z 1 , . . . , Z m given U 1 , . . . , U n are independent and standard normally distributed, which we use below without further reference. From the estimate (C.1) given in [START_REF] Dahlhaus | Nonparametric quasi-maximum likelihood estimation for Gaussian locally stationary processes[END_REF] (Proposition A.1) follows

P   n -2 n i=1 U i V i1 2 η 2   P (n -1 W 2) + E P 2n -1 |Z 1 | 2 η 2 U 1 , . . . , U n exp - n 16 + 1 πη 2 n exp - η 2 n 4 ,
which implies (C.2). The estimate (C.3) follows analogously and we omit the details. By employing (C.1), 2c -1 c and n -1 (cn + t) 1 we obtain (C.4). Indeed,

E n i=1 U 2 i -2 c n + = ∞ 0 P n -1/2 n i=1 (U 2 i -1) n -1/2 (cn + t) dt ∞ 0 exp - 1 8 n -1 (cn + t) 2 1 + n -1 (cn + t) dt ∞ 0 exp - 1 16 (cn + t) dt = exp - cn 16 ∞ 0 exp - t 16 dt = 16 exp - cn 16
From the last estimate and (C.1) follows (C.5), because

E   m j=1 n -1/2 n i=1 U i V ij 2 -4 c m   + E   n -1 W E     m j=1 |Z j | 2 -2 c m   + U 1 , . . . , U n   + 2 c m n -1 (W -2n) +   16 exp - c m 16 E[n -1 W ] + 32 c m n exp - n 16 .
It remains to prove (C.6) which can be realized as follows (keep in mind that

E[W 2 ] = n(n+2)) E   m j=1 a j n i=1 U i V ij 2   2 = E   W 2 E   m j=1 a j |Z j | 2 2 U 1 , . . . , U n     = E[W 2 ] m j=1 a 2 j + m j=1 a j ) 2 .
Lemma C.2. For all n, m 1 we have

n 2 ρ -4 m E [W ] m 4 6 (E X 2 ) 2 . (C.7)
Furthermore, there exist a numerical constant C > 0 such that for all n 1

n 8 max 1 m ⌊n 1/4 ⌋ P [W ] t m [Γ] -1 m [W ] m ρ 2 m > 1 16 C; (C.8) n 8 max 1 m ⌊n 1/4 ⌋ P [Ξ] m s > 1/8 C;
(C.9)

n 7 P {1/2 σ 2 Y /σ 2 Y 3/2} c C. (C.10) Proof of Lemma C.2. Let n, m 1 be fixed, denote by (λ j , e j ) 1 j m an eigenvalue decom- position of [Γ] m . Define U i := (σε i + β -β m , X i H )/ρ m and V ij := (λ -1/2 j e t j [X i ] m
), 1 i n, 1 j m, where U 1 , . . . , U n , V 11 , . . . , V nm are independent and standard normally distributed random variables. Proof of (C.7) and (C.8). Taking into account m j=1 λ j E X 2 H and the identities

n 4 ρ -4 m [W ] m = ( m j=1 λ j ( n i=1 U i V ij ) 2 ) 2 and ([W ] t m [Γ] -1 m [W ] m )/ρ 2 m = n -2 m j=1 ( n i=1 U i V ij )
2 the assertions (C.7) and (C.8) follow, respectively, from (C.6) and (C.3) in Lemma C.1 (with a j = λ j ). Proof of (C.9). Since n

[Ξ] m s m max 1 j,l m | n i=1 (V ij V il -δ jl )
| we obtain due to (C.1) and (C.2) in Lemma C.1 that for all η > 0

P ( [Ξ] m s η) 1 j,l m P (|n -1 n i=1 (V ij V il -δ jl )| η/m) m 2 max P (|n -1 n i=1 V i1 V i2 | η/m), P (|n -1/2 n i=1 (V 2 i1 -1)| n 1/2 η/m) m 2 max (1 + m ηn 1/2 ) exp - n 4 min η 2 /m 2 , 1/4 , 2 exp - 1 8 nη 2 /m 2 1 + η/m .
Moreover, for all η m/2 the last bound simplifies to

P ( [Ξ] m s η) m 2 max 1 + 2m ηn 1/2 , 2 exp - 1 12 nη 2 m 2 .
and it is easily seen that the last bound implies (C.9). Proof of (C.10). Since Y 1 /σ Y , . . . , Y n /σ Y are independent and standard normally distributed, by exploiting that

{1/2 σ 2 Y /σ 2 Y 3/2} c ⊂ {|n -1 n i=1 Y 2 i /σ 2 Y -1| > 1/2}, (C.10) follows from (C.1) in Lemma C.1, which completes the proof.
Lemma C.3. We have for all c 1 and n, m 1

E n[W ] t m [Γ] -1 m [W ] m ρ 2 m -4 c m + 16 exp -c m 16 + 32 c m n exp -n 16 .
Proof of Lemma C.3. The assertion follows from (C.5) in Lemma C.1 and the identity n

[Γ] -1/2 m [W ] m 2 ρ -2 m = m j=1 (n -1/2 n i=1 U i V ij ) 2 derived
Γ∈G d γ M + n k=m ⋄ n ∆ Γ k E [W ] t k [Γ] -1 k [W ] k -4 σ 2 k k Λ Γ k n + C(d)(σ 2 + r)Σn -1 .
Proof of Lemma C.4. The key argument of the proof is the estimate given in Lemma C.3 with c = Λ Γ k . Taking into account this upper bound and that for all ii) and (iv) respectively) hold true, we obtain

β ∈ F r β and Γ ∈ G d γ the estimates ∆ Γ k 4d 3 ∆ γ k , (1 + log d) -1 Λ γ k Λ Γ k , δ Γ M + n nCd 6 (1 + log d) (recall that δ Γ m = m∆ Γ m Λ Γ m ) and ρ 2 k σ 2 k 2(σ 2 + 35d 6 r) (Lemma B.2 (i), (
M + n k=m ⋄ n ∆ Γ k E [W ] t k [Γ] -1 k [W ] k -4 σ 2 k k Λ Γ k n + M + n k=1 σ 2 k ∆ Γ k n E n[W ] t k [Γ] -1 k [W ] k ρ 2 k -4 k Λ Γ k + C(d)(σ 2 + r) n -1    M + n k=1 ∆ γ k exp - kΛ γ k 16(1 + log d) + M + n exp -n/16    .
Finally, exploiting that the constant Σ satisfies (3.3) and that M + n exp -n/16 C for all n 1 we obtain the assertion of the lemma, which completes the proof.

(σ 2 Y + [g] t k [Γ] -1 k [g] k ) = σ 2 k ρ 2 k ).
Combining these estimates we obtain (C.13), which completes the proof.

Proposition C.6. Let κ = 96 in the definition of the penalty pen given in (2.11). There exists a constant C(d) such that for all n 1 we have

sup β∈F r b sup Γ∈G d γ E sup m ⋄ n k M + n β k -β k 2 ω - 1 6 pen k + C(d)(σ 2 + r)Σ n -1 .
Proof of Proposition C.6. We use the identity

[ β k -β k ] k = [ Γ] -1 k [W ] k ✶Ω k,n -[β k ] k ✶Ω c
k,n , and obtain

β k -β k 2 ω = [∇ ω ] 1/2 k [ Γ] -1 k [W ] k 2 ✶Ω k,n + β k 2 ω ✶Ω c k,n . (C.14) Exploiting further ([Id] k + [Ξ] k ) -1 s ✶℧ k,n 2, the identity [ Γ] k = [Γ] 1/2 k {[Id] k + [Ξ] k }[Γ] 1/2 k and the definition of ∆ Γ k it follows that [∇ ω ] 1/2 k [ Γ] -1 k [W ] k 2 ✶℧ k,n 4∆ Γ k [Γ] -1/2 k [W ] k 2 .
On the other hand, we have

[∇ ω ] 1/2 k [ Γ] -1 k [W ] k 2 ✶Ω k,n ∆ ω k n 2 [W ] k 2 .
From these estimates and

β k ω β k b (ωb -1
is non-increasing due to Assumption 3.1) we deduce for all k 1

β k -β k 2 ω 4∆ Γ k [Γ] -1/2 k [W ] k 2 + ∆ ω k n 2 [W ] k 2 ✶℧ c k,n + β k 2 b ✶Ω c k,n .
Taking into account this upper bound, the notations ∆ Γ k and Λ Γ k given in (A.1), and the definition

pen k = 96σ 2 k k∆ Γ k Λ Γ k n -1 we obtain for all β ∈ F r b and Γ ∈ G d γ that E sup m ⋄ n k M + n β k -β k 2 ω - 1 6 pen k + 4 M + n k=m ⋄ n ∆ Γ k E [Γ] -1/2 k [W ] k 2 -4 σ 2 k k Λ Γ k n + + M + n k=m ⋄ n n 3 E [W ] k 4 1/2 P (℧ c k,n ) 1/2 + M + n k=m ⋄ n β k 2 b P (Ω c k,n )
Consider the second and third right hand side term. By exploiting, respectively, (C.7) in Lemma C.2 and (B.5) in Lemma B.1 together with ρ 2 m 2(σ 2 + 35d 6 r) (Lemma B.2 (iv)) these two terms are bounded by

6(σ 2 + 35d 6 r)E X 2 n 2 M + n max m ⋄ n k M + n P (℧ c k,n ) 1/2 + 34d 8 rM + n max m ⋄ n k M + n P (Ω c k,n ).
Combining this upper bound, the property E X 2 d j 1 γ j dΣ and the estimates given in Lemma C.5 we deduce for all β ∈ F r b and Γ ∈ G d γ that

sup β∈F r b sup Γ∈G d γ E sup m ⋄ n k M + n β k -β k 2 ω - 1 6 pen k + C(d)(σ 2 + r)Σ n -1 + 4 sup β∈F r b sup Γ∈G d γ M + n k=m ⋄ n ∆ Γ k E [Γ] -1/2 k [W ] k 2 -4 σ 2 k k Λ Γ k n +
The result of the proposition follows now by replacing the last right hand side term by its upper bound given in Lemma C.4, which completes the proof.

Proposition C.7. Let κ = 96 in the definition of pen and pen given in (2.11) and (2.6) respectively. There exists a constant C(d) such that for all n 1 we have

sup β∈F r b sup Γ∈G d γ E( β m -β 2 ω ✶E c n ) C(d)(σ 2 + r)Σ n -1 .
Proof of Proposition C.7. Taking into account the decomposition (C.14) and the estimate

[∇ ω ] 1/2 k [ Γ] -1 k [W ] k 2 ✶Ω k,n ∆ ω k n 2 [W ] k 2
given in the proof of Proposition C.6 we conclude

β k -β 2 ω 2∆ ω k n 2 [W ] k 2 + 2 β k 2 ω + 2 β 2 ω , for all k 1.
By exploiting (B.5) in Lemma B.1 together with β k ω β k b (ωb -1 is non-increasing due to Assumption 3.1) we obtain for all β ∈ F r b and Γ ∈ G d γ and for all k 1 that

β k -β 2 ω 2∆ ω k n 2 [W ] k 2 + 2(34d 8 r + r).
Since 1 m M ω n and max

1 k M ω n ∆ ω k n it follows for all β ∈ F r b and Γ ∈ G d γ that E( β m -β 2 ω ✶E c n ) 2n 3 M ω n max 1 k M ω n E [W ] k 4 1/2 |P (E c n )| 1/2 + 2(34d 8 r + r)M ω n P (E c n ).
From (C.7) in Lemma C.2 together with ρ 2 m 2(σ 2 + 35d 6 r) (Lemma B.2) and E X 2 dΣ we conclude for all β ∈ F r b and Γ ∈ G d γ that

E( β m -β 2 ω ✶E c n ) 12(σ 2 + 35d 6 r)dΣn 2 M ω n |P (E c n )| 1/2 + 2(34d 8 r + r)M ω n P (E c n ).
The result of the proposition follows now from M ω n ⌊n 1/4 ⌋ and by replacing the probability P (E c n ) by its upper bound Cn -7 given in Lemma C.5, which completes the proof.

Proof of Proposition 3.1. The assertion follows from Proposition C.6 and Proposition C.7 and we omit the details.

D Proof of Proposition 3.3

We assume throughout this section that the conditions of Proposition 3.3 are satisfied which allows us to employ the Lemma B.1-B.5 stated in Section B. We formulate first preliminary results (Proposition D.1 and Lemma D.2-D.5) which rely on the moment conditions imposed through Assumption 3.3. They are used below to prove that the Assumptions 2.1 and 2.2 are satisfied (Proposition D.6 and D.7 respectively), which is the claim of Proposition 3.3. We begin by gathering elementary bounds due to Assumption 3.3. Let k be given by Assumption 3.3 then for all m 1 we have

E|Z m | 4k ρ 2 m η 4k , E|Y | 4k σ 4k Y η 4k , max 1 j m E|([Γ] -1/2 m [X] m ) j | 4k η 4k , E β -β m , X H 4k Γ 1/2 (β m -β) 4k H η 4k , E [X] t m [Γ] -1 m [X] m 2k m 2k η 4k .
Moreover, if V is a non negative random variable with EV k < ∞ then the elementary inequality EV ✶ {V t} t -k+1 EV k holds true for all t > 0. Taking into account this estimate we obtain under Assumption 3.3, that for all m, n 1

Eε 2 ✶ {|ε|>n 1/6 } η 32 n -5 , E β -β m , X H 2 ✶ {| β-β m ,X H |> Γ 1/2 (β m -β) H n 1/6 } η 32 Γ 1/2 (β m -β) 2 H n -5 , E|[X] t m [Γ] -1 m [X] m | 2 ✶ {[X] t m [Γ] -1 m [X]m>mn 1/3 } η 32 m 2 n -14/3 (D.1)
and by employing Markov's inequality

P (|ε| > n 1/6 ) n -16/3 η 32 , P (| β -β m , X H | > Γ 1/2 (β m -β) H n 1/6 ) n -16/3 η 32 . (D.2)
We exploit these bounds in the following proofs. Moreover, the key argument used in the proof of Lemma D.3 is the following inequality due to [START_REF] Talagrand | New concentration inequalities in product spaces[END_REF] (see e.g. [START_REF] Klein | Concentration around the mean for maxima of empirical processes[END_REF]).

Proposition D.1 (Talagrand's Inequality). Let T 1 , . . . , T n be independent T -valued random variables and ν

* s = (1/n) n i=1 ν s (T i ) -E[ν s (T i )] ,
for ν s belonging to a countable class {ν s : s ∈ S} of measurable functions. Then, for ε > 0, Var(ν s (T i )) v.

E sup s∈S |ν * s | 2 -2(1 + 2ε)H 2 + C v n exp(-K 1 ε nH 2 v ) + h 2 n 2 C 2 (ε) exp(-K 2 C(ε) √ ε nH h ) with K 1 = 1/6, K 2 = 1/(21 √ 2), C(ε) = √ 1 + ε -
Lemma D.2. There exist a numerical constant C > 0 such that for all n 1

n 2 sup m 1 ρ -4 m E [W ] m 4 Cη 8 (E X 2 H ) 2 ; (D.3) n 8 max 1 m ⌊n 1/4 ⌋ P [W ] t m [Γ] -1 m [W ] m ρ 2 m > 1 16 Cη 64 ; (D.4) n 8 max 1 m ⌊n 1/4 ⌋ P [Ξ] m s > 1/8 C(η); (D.5) n 7 P {1/2 σ 2 Y /σ 2 Y 3/2} c Cη 64 . (D.6)
Proof of Lemma D.2. Let n, m 1 be fixed, denote by (λ j , e j ) 1 j m an eigenvalue decomposition of

[Γ] m . Define U i := (σε i + β -β m , X i H )/ρ m and V ij := (λ -1/2 j e t j [X i ] m ), 1 i n, 1 j m. Keep in mind that E|U i | 4k η 4k , E|V ij | 4k η 4k and E|U i V ij | 2k η 4k
for some k 16 due to Assumption 3.3 and U 1 V 1j , . . . , U n V nj are independent and centered random variables for all 1 j m. Proof of (D.3) and (D.4). Consider the identities n

4 ρ -4 m [W ] m 4 = ( m j=1 λ j ( n i=1 U i V ij ) 2 ) 2 and ([W ] t m [Γ] -1 m [W ] m )/ρ 2 m = n -2 m j=1 ( n i=1 U i V ij ) 2 .
We apply successively Minkowski's (re-spectively Jensen's) inequality and Theorem 2.10 in [START_REF] Petrov | Limit theorems of probability theory. Sequences of independent random variables[END_REF], which leads to

n 2 ρ -4 m E [W ] m 4 = n -2 E m j=1 λ j ( n i=1 U i V ij ) 2 2 n -2 m j=1 λ j E n i=1 U i V ij 4 1/2 2 C m j=1 λ j max 1 i n E U i V ij 4 1/2 2 Cη 8 m j=1 λ j 2 ; m -k n k ρ -2k m E [Γ] -1/2 m [W ] m 2k = n -k E m -1 m j=1 ( n i=1 U i V ij ) 2 k n -k m -1 m j=1 E n i=1 U i V ij 2k C(k)m -1 m j=1 max 1 i n E U i V ij 2k C(k)η 4k .
The first estimate implies (D.3) since m j=1 λ j E X 2 H . By employing Markov's inequality the second estimate with k = 16 implies (D.4), that is

max 1 m ⌊n 1/4 ⌋ P [W ] t m [Γ] -1 m [W ] m 2 m > 1 16 Cn -16 η 64 max 1 m ⌊n 1/4 ⌋ m 16 Cn -12 η 64 .
Proof of (D.5). Since V 1j V 1lδ jl , . . . , V nj V nlδ jl are independent and centered random variables with

E|V ij V il -δ jl | 2k
Cη 4k for all 1 j, l m it follows from Theorem 2.10 in [START_REF] Petrov | Limit theorems of probability theory. Sequences of independent random variables[END_REF]

that n k E n -1 n i=1 (V ij V il -δ jl ) 2k C(k)η 4k . By employing the elementary inequality [Ξ] m 2 s 1 j,l m |V ij V il -δ jl | 2 ,
2 i /σ 2 Y -1 2k C(k)η 4k it follows from Theorem 2.10 in Petrov [1995] that E n -1 n i=1 Y 2 i /σ 2 Y -1 2k C(k)n -k η 4k
. Employing Markov's inequality and the last bound with k = 16 we deduce P |n

-1 n i=1 Y 2 i /σ 2 Y -1| > 1/2
Cn -16 η 64 . Thereby, the assertion (C.10) follows from the last bound by exploiting that

{1/2 σ 2 Y /σ 2 Y 3/2} c ⊂ {|n -1 n i=1 Y 2 i /σ 2 Y -1| > 1/2}, which completes the proof. Lemma D.3. Let ς m := σ + η 2 Γ 1/2 (β m -β) H , m 1.
There exists a numerical constant C such that for all ⌊n 1/4 ⌋ m 1 we have

E [Γ] -1/2 m [W n ] m 2 -12ς 2 m mΛ Γ m n + C ς 2 m n exp - mΛ Γ m 6 + exp - n 1/6 100 + η 32 n 2 .
Proof. Let 1 m n be fixed and S m := {z ∈ R m : z t z 1}. Define the subsets

E n := {e ∈ R : |e| n 1/6 }, X 1n := {x ∈ H : | β -β m , x H | Γ 1/2 (β -β m ) H n 1/6
(σe + β -β m , x H ) 2 [x] t m [Γ] -1 m [x] m ✶ {e∈En,x∈Xn} (σ + Γ 1/2 (β m -β) H ) 2 n 2/3 m ς 2 m n 2/3 m =: h 2 (D.8)
By employing the independence of ε and X it is easily seen that

n E sup s∈S m |ν * s | 2 σ 2 m + E| β -β m , X H | 2 [X] t m [Γ] -1 m [X] m , sup s∈S m 1 n n i=1 Var(ν s (ε i , X i )) σ 2 + sup s∈S m E| β -β m , X H | 2 |s t [Γ] -1/2 m [X] m | 2 .
By applying the Cauchy-Schwarz inequality together with E

[Γ] -1/2 m [X] m 4 m 2 η 4 and E β - β m , X H 4 Γ 1/2 (β m -β) 4 H η 4 we obtain E sup s∈S m |ν * s | 2 m n (σ 2 + Γ 1/2 (β -β m ) 2 H η 4 ) ς 2 m mΛ Γ m n =: H 2 , (D.9)
and taking in addition into account that E|s

t [Γ] -1/2 m [X] m | 4 η 4 for all s ∈ S m we obtain sup s∈S m 1 n n i=1 Var(ν s (ε i , X i )) σ 2 + Γ 1/2 (β m -β) 2 H η 4 ς 2 m =: v. (D.10)
Combining (D.8), (D.9), (D.10) due to Talagrand's inequality (Lemma D.1 with ε = 1) follows

E sup s∈S m |ν * s | 2 -6ς 2 m mΛ Γ m n + C ς 2 m n exp - mΛ Γ m 6 + ς 2 m m n 4/3 exp - n 1/6 100 C ς 2 m n exp - mΛ Γ m 6 + exp - n 1/6 100 (D.11)
where we used that m ⌊n 1/4 ⌋. Consider T 2 on the right hand side of (D.7). By employing [X] m [Γ] -1 m [X] m ✶ {X∈X 2,n } mn 1/3 and X n = X 1n ∩ X 2n we have nE sup

s∈S m |R * s | 2 E(σε + β -β m , X H ) 2 [X] m [Γ] -1 m [X] m (1 -✶ {ε∈En,X∈Xn} ) E(σε + β -β m , X H ) 2 [X] m [Γ] -1 m [X] m ✶ {X ∈X 2,n } + mn 1/3 E(σε + β -β m , X H ) 2 (✶ {ε ∈En} + ✶ {X ∈X 1n } ).
Taking into account that E(σε + ββ m , X H ) 4 (σ 2 + Γ 1/2 (ββ m ) 2 H ) 2 η 4 , Eε 2 = 1 and

E| β -β m , X H | 2 = Γ 1/2 (β -β m ) 2
H from the independence between ε and X follows nE sup

s∈S m |R * s | 2 (σ 2 + Γ 1/2 (β -β m ) 2 H )η 2 E|[X] m [Γ] -1 m [X] m | 2 ✶ {X ∈X 2,n } 1/2
+ mn 1/3 σ 2 Eε 2 ✶ {ε ∈En} + Γ 1/2 (β -β m ) 2

H P (ε ∈ E n ) + σ 2 P ∈ X 1n ) + E| β -β m , X H | 2 ✶ {X ∈X 1n } .
We exploit now the estimates given in (D.1) and (D.2). Thereby, we obtain

nE sup s∈S m |R * s | 2 C(σ 2 + Γ 1/2 (β -β m ) 2 H )η 32 mn -7/3 Cς 2 m η 32 n -2
where we used that m ⌊n 1/4 ⌋. Keeping in mind the decomposition (D. Proof of Proposition D.6. We follow line by line the proof of Proposition C.6 . Keeping in mind that pen k = 288σ 2 k k∆ Γ k Λ Γ k n -1 we obtain

E sup m ⋄ n k M + n β k -β k 2 ω - 1 6 pen k + 4 M + n k=m ⋄ n ∆ Γ k E [Γ] -1/2 k [W ] k 2 -12σ 2 k kΛ Γ k n + + M + n k=m ⋄ n n 3 E [W ] k 4 1/2 P (℧ c k,n ) 1/2 + M + n k=m ⋄ n β k 2 ω P (Ω c k,n ).
The second and third right hand side term we bound due to Lemma D. The assertion follows now with help of Lemma D.5, which completes the proof.

Proof of Proposition 3.3. The assertion follows from Proposition D.6 and Proposition D.7 and we omit the details.

  1 which amongst others we define now referring only to the classes F r b and G d γ . Keep in mind the notations given in (2.4) and (2.10). For m 1 and K = ([∇ γ ] m ) m 1 define ∆ γ m := ∆ m (K) and δ γ m := δ m (K) where ∆ γ m = max 1 k m ω j γ -1 j . Moreover, for n 1 we set M - n := M n (a) with a = (16d 3 γ -1 m ) m 1 and M + n := M n (a) with a = ([4dγ m ] -1 ) m 1 . Taking into account these notations we define for n 1

  Proposition 3.1. Assume an iid. n-sample of (Y, X) obeying (1.1) and Assumption 3.2. Consider sequences ω, b and γ satisfying Assumption 3.1 and in the definition (2.6) and (2.11) of the penalty pen and pen respectively set κ = 96. For the classes F r b and G d γ , there exist finite constants C 1 := C 1 (d) and C 2 := C 2 (d) depending on d only such that the Assumptions 2.1 and 2.2 hold true, with K 1

  3 in Appendix C which due to Assumption 3.2 is shown by employing elementary inequalities for Gaussian random variables. In contrast, the proof of an analogous result under Assumption 3.3 given in Lemma D.3 in Appendix D is based on an inequality due to[START_REF] Talagrand | New concentration inequalities in product spaces[END_REF] (Proposition D.1 in the appendix states a version as presented in[START_REF] Klein | Concentration around the mean for maxima of empirical processes[END_REF]). Proposition 3.3. Assume an iid. n-sample of (Y, X) obeying (1.1) and Assumption 3.3. Consider sequences ω, b and γ satisfying Assumption 3.1 and in the definition (2.6) and (2.11) of the penalty pen and pen respectively, set κ = 288. For the classes F r b and G d γ , there exist finite constants C 1 := C 1 (σ, η, F r b , G d γ ) depending on σ, η and the classes F r b and G d γ only, and C 2 := C 2 (d) depending on d only, such that Assumptions 2.1 and 2.2 hold true with

Theorem 3. 4 .

 4 Let the assumptions of Proposition 3.3 be satisfied. There exist finite constants K := K(d) depending on d only and K ′ := K ′ (σ, η, F r b , G d γ ) depending on σ, η and the classes F r b and G d γ only such that

  Proof of Theorem 3.4. Taking into account Proposition 3.3 rather than Proposition 3.1 we follow line by line the proof of Theorem 3.2 and hence we omit the details.Minimax-optimality.A comparison of the upper bounds in both Theorem 3.2 and Theorem 3.4 with the lower bound displayed in (3.2) shows that the data-driven estimator β m attains up to a constant the minimax-rate R * n = min 1 m<∞ max ωm bm , all m 1. The next assertion is an immediate consequence of Theorem 3.2 and Theorem 3.4 and we omit its proof.Corollary 3.5. Let the assumptions of either Theorem 3.2 or Theorem 3.4 be satisfied. If in addition ξ

  Let in addition β ∈ F r b with sequence b satisfying Assumption 3.1. If β m denotes a Galerkin solution of g = Γβ then for each strictly positive sequence w := (w j ) j 1 such that w/b is non increasing we obtain for all m ∈ N β -Lemma B.2. Let D := (4d 3 ) and let Assumption 3.1 be satisfied. If Γ ∈ G d γ then it holds

  Proof of Lemma B.2. Proof of (i). Due to (B.1) and (B.3) in Lemma B.1, we have for all Γ ∈ G d γ and for all m 1 that [Γ] Thus, given D = (4d 3 ) for all m 1 we have d -1

  in the proof of Lemma C.2. Lemma C.4. There exists a constant C(d) only depending on d such that for all n

Lemma C. 5 .

 5 There exist a numerical constant C and a constant C(d) only depending on d such that for all n 1 we have sup Proof of Lemma C.5. Since M + n ⌊n 1/4 ⌋ and ℧ c m,n = [Ξ] m > 1/8 the assertion (C.11) follows from (C.8) in Lemma C.2. Consider (C.12). Let n o := n o (d) := exp(128d 6 ) 8d 3 , and consequently ∆ ω M + n (M + n log n) 128d 6 for all n n o . We distinguish in the following the cases n < n o and n n o . First, consider 1 n n o . Obviously, we have M + n max 1 m M + n P (Ω c m,n ) 4 and n o depends on d only. On the other hand, if n n o then from Lemma B.2 (iii) follows n 2 max 1 m M + n [Γ] -1 m , and hence ℧ m,n ⊂ Ω m,n for all 1 m M + n by employing Lemma B.5. From (C.11) we conclude M+ n max 1 m M + n P (Ω c m,n ) M + n max 1 m M + n P (℧ c m,n ) Cn -3. By combination of the two cases we obtain (C.12). It remains to show (C.13). Consider the events A n , B n and C n defined in (A.2), whereA n ∩ B n ∩ C n ⊂ E n due to Lemma B.4. Moreover we have n 7 P (A c n ) C, n 7 P (B c n ) C,and n 7 P (C c n ) C, due to (C.10), (C.9) and (C.8) in Lemma C.2 respectively (keep in mind that ⌊n 1/4 ⌋ M ω n and 2

E

  7) the last bound and (D.11) imply together the claim of Lemma D.3 which completes the proof. Lemma D.4. There exists a constant K := K(σ, η, F r b , G d γ ) depending on σ, η and the classes F r b and G d γ only such that for all n 1 we have sup (σ 2 + r) Σ n -1 . Proof. We begin our proof with the observation that there exists an integer n o := n o (σ, η, F r b , G d γ ) depending on σ, η and the classes F r b and G d γ only such that for all n n o and for all m m ⋄ n we have ς 2 m2(σ 2 + Γ 1/2 β 2 H +[g] t m [Γ] -1 m [g] m ) = 2(σ 2 Y +[g] t m [Γ] -1 m [g] m ) = σ 2 m . Indeed, we have 1/m ⋄ n = o(1) as n → ∞ and |ς 2 mσ 2 | = o(1) as m → ∞ because ς m = σ + η 2 Γ 1/2 (β mβ) H and Γ 1/2 (β mβ) 2 H 34 d 9 r γ m b -1 m due to (B.5) in Lemma B.1. We distinguish in the following the cases n < n o and n n o . First, consider n < n o . Due to (D.3) in Lemma D.2 and ρ 2 m 2(σ 2 + 35d 6 r) (Lemma B.2 (iv)) we have for all m 1 o η 4 (σ 2 + r). The last bound implies the assertion of the lemma for all 1 n < n o because n o depends on σ, η and the classes F r b and G d γ only. Consider now n n o where we have ς 2 that Σ = Σ(G d γ ) satisfies (3.3) and n exp -n 1/6 /100 C which in turn implies the claim of the lemma for all n n o , i.e., η 32 (σ 2 + r) Σ n -1 . Combining the cases n < n o and n n o completes the proof. Lemma D.5. There exist a numerical constant C and a constant C(d) only depending on d such that for all n 1 we have sup Proof of Lemma D.5. By employing Lemma D.2 rather than Lemma C.2 the proof of the lemma follows along the lines of the proof of Lemma C.5, and we omit the details. Proposition D.6. Let κ = 288 in the definition of the penalty pen given in (2.11). There exists a constant K := K(σ, η, F r b , G d γ ) depending on σ, η and the classes F r b and G d γ only such that for all n 1 we have sup (σ 2 + r) Σ n -1 .

ω

  employing the bound given in Lemma D.4 we complete the proof. Proposition D.7. Let κ = 288 in the definition of pen and pen given in (2.11) and (2.6) respectively. There exists a constant C(d) such that for all n 1 we have sup d) η 64 (σ 2 + r) Σ n -1 . Proof of Proposition D.7. Taking into account (D.3) in Lemma D.2 rather than (C.7) in Lemma C.2 we follow line by line the proof of Proposition C.7 and conclude that sup ✶E c ) C(d)(σ 2 + r)η 8 Σn 5

  Jensen's inequality and the last bound we obtainm -2k n k E [Ξ] m

		2k s	C(k)η 4k . Applying Markov's inequality and the last bound with k = 16
	we conclude
	max 1 m ⌊n 1/4 ⌋	P [Ξ] m s >	1 8	Cn -16 η 64	max 1 m ⌊n 1/4 ⌋	m 32 Cn -8 η 64
	which proves the assertion (D.5).
	Proof of (D.6). Since Y 2 1 /σ 2 Y -1, . . . , Y 2 n /σ

Y -1 are independent and and centered random variables with E Y

  },where we bound the terms T 1 and T 2 on the right hand side separately. Consider first T 1 which we estimate by employing Talagrand's inequality. Obviously, we have

	sup e∈R,x∈H	sup s∈S m	|ν s (e, x)| 2 = sup e∈R,x∈H

This work was supported by the IAP research network no. P6/03 of the Belgian Government (Belgian Science Policy).