Ouerdia Arkoun
email: ouerdia.arkoun@supbiotech.fr

Jean-Yves Brua

Serguei Pergamenchtchikov

Sequential Model Selection Method for Nonparametric Autoregression

Keywords: Non-parametric estimation, Non parametric autoregresion, Non-asymptotic estimation, Robust risk, Model selection, Sharp oracle inequalities. Subject Classifications: primary 62G08, secondary 62G05

In this paper for the first time the nonparametric autoregression estimation problem for the quadratic risks is considered. To this end we develop a new adaptive sequential model selection method based on the efficient sequential kernel estimators proposed by Arkoun and Pergamenshchikov (2016). Moreover,

Introduction

One of the standard linear models in general theory of time series is the autoregressive model (see, for example, [START_REF] Anderson | The Statistical Analysis of Time Series[END_REF] and the references therein). Natural extensions for such models are nonparametric autoregressive models which are defined by

y k = S(x k )y k-1 + ξ k and x k = a + k(b -a) n , (1.1)
where S(•) ∈ L 2 [a, b] is unknown function, a < b are fixed known constants, 1 ≤ k ≤ n, the initial value y 0 is a constant and the noise (ξ k ) k≥1 is i.i.d. sequence of unobservable random variables with Eξ 1 = 0 and Eξ 2 1 = 1. The problem is to estimate the function S(•) on the basis of the observations (y k ) 1≤k≤n under the condition that the noise distribution is unknown.

It should be noted that the varying coefficient principle is well known in the regression analysis. It permits the use of more complex forms for regression coefficients and, therefore, the models constructed via this method are more adequate for applications (see, for example, [START_REF] Fan | Statistical Methods with Varying Coefficient Models[END_REF], [START_REF] Luo | Nonparametric Estimation of the Production Function with Time-Varying Elasticity Coefficients[END_REF]). In this paper we consider the varying coefficient autoregressive models (1.1). There is a number of papers which consider these models such as Dahlhaus (1996a), Dahlhaus (1996b) and [START_REF] Belitser | Recursive Estimation of a Drifted Autoregressive Parameter[END_REF]. In all these papers, the authors propose some asymptotic (as n → ∞) methods for different identification studies without considering optimal estimation issues. To our knowledge, for the first time, the minimax estimation problem for the model (1.1) has been treated in [START_REF] Arkoun | Nonparametric Estimation for an Autoregressive Model[END_REF] and [START_REF] Moulines | On Recursive Estimation for Time Varying Autoregressive Processes[END_REF] in the nonadaptive case, i.e. for the known regularity of the function S. Then, in [START_REF] Arkoun | Sequential Adaptive Estimators in Nonparametric Autoregressive Models[END_REF] it is proposed to use the sequential analysis method for the adaptive pointwise estimation problem in the case where the unknown Hölder regularity is less than one, i.e when the function S is not differentiable. Also it should be noted (see, [START_REF] Arkoun | Sequential Adaptive Estimators in Nonparametric Autoregressive Models[END_REF]) that for the model (1.1), the adaptive pointwise estimation is possible only in the sequential analysis framework. That is why we study sequential estimation methods for the smooth function S. In this paper we consider the quadratic risk defined as

R p ( S n , S) = E p,S S n -S 2 , S 2 = b a S 2 (x)dx , (1.2)
where S n is an estimator of S based on observations (y k ) 1≤k≤n and E p,S is the expectation with respect to the distribution law P p,S of the process (y k ) 1≤k≤n given the distribution density p and the coefficient S. Moreover, taking into account that the distribution p is unknown, we use the robust nonparametric estimation approach proposed in Galtchouk and Pergamenshchikov (2006a). To this end we set the robust risk as R * ( S n , S) = sup p∈P R p ( S n , S) , (1.3)

where P is a family of the distributions defined in Section 2.

In order to estimate the function S in model (1.6) we make use of the estimator family ( S λ , λ ∈ Λ), where S λ is a weighted least square estimator with the Pinsker weights. For this family, similarly to Galtchouk and Pergamenshchikov (2009a), we construct a special selection rule, i.e. a random variable λ with values in Λ, for which we define the selection estimator as S * = S λ . Our goal in this paper is to show the non asymptotic sharp oracle inequality for the robust risks (1.3), i.e. to show that for any ˇ > 0 and n ≥ 1

R * ( S * , S) ≤ (1 + ˇ ) min λ∈Λ R * ( S λ , S) + B n ˇ n , (1.4)
where B n is a rest term such that for any δ > 0,

lim n→∞ B n n δ = 0 . (1.5)
In this case the estimator S * is called optimal in the oracle inequality sense. In this paper, in order to obtain this inequality for model (1.1) we develop a new model selection method based on the truncated sequential procedures developed in [START_REF] Arkoun | Sequential Robust Estimation for Nonparametric Autoregressive Models[END_REF] for the pointwise efficient estimation.

Then we use the non asymptotic analysis tool proposed in Galtchouk and Pergamenshchikov (2009a) based on the non-asymptotic studies from [START_REF] Baron | Risk bounds for model selection via penalization[END_REF] for a family of least-squares estimators and extended in [START_REF] Fourdrinier | Improved model selection method for a regression function with dependent noise[END_REF] to some other estimator families. To this end we use the approach proposed in [START_REF] Galtchouk | Adaptive Sequential Estimation for Ergodic Diffusion Processes in Quadratic Metric[END_REF], i.e. we pass to a discrete time regression model by making use of the truncated sequential procedure introduced in [START_REF] Arkoun | Sequential Robust Estimation for Nonparametric Autoregressive Models[END_REF]. To this end, at any point (z l ) 1≤l≤d of a partition of the interval [a, b], we define a sequential procedure (τ l , S * l ) with a stopping rule τ l and an estimator S * l . For Y l = S * l with 1 ≤ l ≤ d, we come to the regression equation on some set Γ ⊆ Ω:

Y l = S(z l ) + ζ l , 1 ≤ l ≤ d .
(1.6)

Here, in contrast with the classical regression model, the noise sequence (ζ l ) 1≤l≤d has a complex structure, namely,

ζ l = ξ * l + l , (1.7)
where (ξ * l ) 1≤l≤d is a "main noise" sequence of uncorrelated random variables and ( l ) 1≤l≤n is a sequence of bounded random variables.

We will use the oracle inequality (1.4) to prove the asymptotic efficiency for the proposed procedure, using the same method as it is been used in Galtchouk and Pergamenshchikov (2009b). The asymptotic efficiency means that the procedure provides the optimal convergence rate and the asymptotically minimal rate normalized risk which coincides with the Pinsker constant. It should be emphasized that only sharp oracle inequalities of type (1.4) allow to synthesis efficiency property in the adaptive setting.

The paper is organized as follows: In Section 2 we state the main conditions for the model (1.1). In Section 3 we describe the passage to the regression scheme. In Section 4 we describe the sequential model selection procedure. In Section 5 we announce the main results. In Section 6 we study the properties of the obtained regression model (1.6). In Section 7 we prove all basic results. In Appendix A we give all the auxiliary and technical tools.

Main Conditions

As in [START_REF] Arkoun | Sequential Robust Estimation for Nonparametric Autoregressive Models[END_REF] we assume that in the model (1.1) the i.i.d. random variables (ξ k ) k≥1 have a density p (with respect to Lebesgue measure) from the functional class P defined as

P := p ≥ 0 : +∞ -∞ p(x) dx = 1 , +∞ -∞ x p(x) dx = 0 , +∞ -∞ x 2 p(x) dx = 1 and sup k≥1 +∞ -∞ |x| 2k p(x) dx ς k (2k -1)!! ≤ 1    , (2.1)
where ς ≥ 1 is some parameter, which may be a function of the number observation n, i.e. ς = ς(n), such that for any δ > 0

lim n→∞ ς(n) n δ = 0 . (2.2)
Note that the (0, 1)-Gaussian density belongs to P. In the sequel we denote this density by p 0 . It is clear that for any q > 0

m * q = sup p∈P E p |ξ 1 | q < ∞ , (2.3)
where E p is the expectation with respect to the density p from P. To obtain the stable (uniformly with respect to the function S ) model (1.1), we assume that for some fixed 0 < < 1 and L > 0 the unknown function S belongs to the ε -stability set introduced in [START_REF] Arkoun | Sequential Robust Estimation for Nonparametric Autoregressive Models[END_REF] as

Θ ,L = S ∈ C 1 ([a, b], R) : |S| * ≤ 1 - and | Ṡ| * ≤ L , (2.4)
where

C 1 ([a, b], R) is the Banach space of continuously differentiable [a, b] → R functions and |S| * = sup a≤x≤b |S(x)|.
3 Passage to a discrete time regression model

We will use as a basic procedure the pointwise procedure from Arkoun and Pergamenchtchikov (2016) at the points (z l ) 1≤l≤d defined as

z l = a + l d (b -a) and d = [ √ n] , (3.1)
where [a] is the integer part of a number a. So we propose to use the first ι l observations for the auxiliary estimation of S(z l ). We set

S ι l = 1 A ι l ι l j=1 Q l,j y j-1 y j , A ι l = ι l j=1 Q l,j y 2 j-1 , (3.2)
where Q l,j = Q(u l,j ) and the kernel Q(•) is the indicator function of the interval [-1; 1], i.e. Q(u) = 1 [-1,1] (u). The points (u l,j ) are defined as

u l,j = x j -z l h . (3.3)
Note that to estimate S(z l ) on the basis of kernel estimator with the kernel Q we use only the observations (y j ) k 1,l ≤j≤k 2,l from the h -neighborhood of the point z l , i.e.

k 1,l = [n z l -n h] + 1 and k 2,l = [n z l + n h] ∧ n , (3.4)
where z l = (z l -a)/(b -a) and h = h/(b -a). Note that, only for the last point z d = b, we take k 2,d = n. We choose ι l in (3.2) as

ι l = k 1,l + q and q = q n = [(n h) µ 0 ] (3.5)
for some 0 < µ 0 < 1. In the sequel for any 0 ≤ k < m ≤ n we set

A k,m = m j=k+1 Q l,j y 2 j-1 and A m = A 0,m . (3.6)
Next, similarly to [START_REF] Arkoun | Sequential Adaptive Estimators in Nonparametric Autoregressive Models[END_REF], we use a kernel sequential procedure based on the observations (y j ) ι l ≤j≤n . To transform the kernel estimator in a linear function of observations and we replace the number of observations n by the following stopping time

τ l = inf{ι l + 1 ≤ k ≤ k 2,l : A ι l ,k ≥ H l } , (3.7)
where inf{∅} = k 2,l and the positive threshold H l will be chosen as a positive random variable which is measurable with respect to the σ-field {y 1 , . . . , y ι l }. Now we define the sequential estimator as

S * l = 1 H l   τ l -1 j=ι l +1 Q l,j y j-1 y j + κ l Q l,τ l y τ l -1 y τ l   1 Γ l , (3.8)
where Γ l = {A ι l ,k 2,l -1 ≥ H l } and the correcting coefficient 0 < κ l ≤ 1 on this set is defined as

A ι l ,τ l -1 + κ 2 l Q l,τ l y 2 τ l -1 = H l .
(3.9)

Note that, to obtain the efficient kernel estimator of S(z l ) we need to use all k 2,lι l -1 observations. Similarly to [START_REF] Konev | Estimate of the Number of Observations in Sequential Identification of the Parameters of Dynamical Systems[END_REF], one can show that τ l ≈ γ l H l as H l → ∞, where

γ l = 1 -S 2 (z l ) . (3.10)
Therefore, one needs to choose H l as (k 2,l -ι l -1)/γ l . Taking into account that the coefficients γ l are unknown, we define the threshold H l as

H l = 1 - γ l (k 2,l -ι l -1) and = 1 2 + ln n , (3.11)
where

γ l = 1 -S 2 ι l
and S ι l is the projection of the estimator S ι l in the interval ] -1 + , 1 -[, i.e.

S ι l = min(max( S ι l , -1 + ), 1 -) .
(3.12)

To obtain the uncorrelated stochastic terms in kernel estimator for S(z l ) we choose the bandwidth h as

h = b -a 2d . (3.13)
As to the estimator S ι l , we can show the following property.

Theorem 3.1. The convergence rate in probability of the estimator (3.12) is more rapid than any power function, i.e. for any b > 0

lim n→∞ n b max 1≤l≤d sup S∈Θ ,L sup p∈P P p,S | S ι l -S(z l )| > 0 = 0 , (3.14) where 0 = 0 (n) → 0 as n → ∞ such that lim n→∞ n δ 0 = ∞ for any δ > 0.
Now we set

Y l = S * l (z l )1 Γ and Γ = ∩ d l=1 Γ l . (3.15)
Using the convergence (3.14), we study the probability properties of the set Γ in the following theorem.

Theorem 3.2. For any b > 0, the probability of the set Γ satisfies the following asymptotic equality

lim n→∞ n b sup S∈Θ ,L P p,S (Γ c ) = 0 . (3.16)
In view of this theorem we can shrink the set Γ c . So, using the estimators (3.15) on the set Γ we obtain the discrete time regression model (1.6) in which

ξ * l = τ l -1 j=ι l +1 Q l,j y j-1 ξ j + κ l Q(u l,τ l ) y τ l -1 ξ τ l H l (3.17)
and l = 1,l + 2,l , where

1,l = τ l -1 j=ι l +1 Q l,j y 2 j-1 šl,j + κ 2 l Q(u l,τ l ) y 2 τ l -1 šl,τ l H l , šl,j = S(x j ) -S(z l )
and

2,l = (κ l -κ 2 l ) Q(u l,τ l ) y 2 τ l -1 S(x τ l ) H l .
Note that in the model (1.6) the random variables (ξ * j ) 1≤j≤d are defined only on the set Γ. For technical reasons we need to define these variables on the set Γ c as well. To this end, for any j ≥ 1 we set

Ql,j = Q l,j y j-1 1 {j<k 2,l } + H l Q l,j 1 {j=k 2,l } (3.18)
and Ǎι l ,m = m j=ι l +1 Q2 l,j . Note, that for any j ≥ 1 and l = m Ql,j Qm,j = 0 .

(3.19)

and Ǎι l ,k 2,l ≥ H l . So now we can modify the stopping time (3.7) as

τl = inf{k ≥ ι l + 1 : Ǎι l ,k ≥ H l } . (3.20)
Obviously, τl ≤ k 2,l and τl = τ l on the set Γ for any 1 ≤ l ≤ d. Now similarly to (3.9) we define the correction coefficient as

Ǎι l ,τ l -1 + κ2 l Q2 l,τ l = H l . (3.21)
It is clear that 0 < κl ≤ 1 and κl = κ l on the set Γ for 1 ≤ l ≤ d. Using this coefficient we set

η l = τl -1 j=ι l +1 Ql,j ξ j + κl Ql,τ l ξ τl H l . (3.22)
Note that on the set Γ, for any 1 ≤ l ≤ d, the random variables η l = ξ * l . Moreover (see Lemma A.2), for any 1 ≤ l ≤ d and p ∈ P

E p,S (η l |G l ) = 0 , E p,S η 2 l |G l = σ 2 l and E p,S η 4 l |G l ≤ mσ 4 l , (3.23)
where

σ l = H -1/2 l , G l = σ{η 1 , . . . , η l-1 , σ l } and m = 4(144/ √ 3) 4 m * 4 . It is clear that σ 0, * ≤ min 1≤l≤d σ 2 l ≤ max 1≤l≤d σ 2 l ≤ σ 1, * , (3.24)
where

σ 0, * = 1 -2 2(1 -)nh and σ 1, * = 1 (1 -)(2nh -q -3) . Now, taking into account that | 1,l | ≤ Lh, for any S ∈ Θ ,L we obtain that sup S∈Θ ,L E p,S 1 Γ 2 l ≤ L 2 h 2 + υn (nh) 2 , (3.25)
where υn = sup p∈P sup S∈Θ ,L E p,S max 1≤j≤n y 4 j . The behavior of this coefficient is studied in the following theorem. ) is approximately as h -2 when n → ∞. We will use this in the oracle inequalities below.

Remark 3.2. It should be emphasized that to estimate the function S in (1.1) we use the approach developed in [START_REF] Galtchouk | Adaptive Sequential Estimation for Ergodic Diffusion Processes in Quadratic Metric[END_REF] for the sequential drift estimation problem in the stochastic differential equation. On the basis of the efficient sequential kernel procedure developped in [START_REF] Galtchouk | Nonparametric Sequential Minimax Estimation of the Drift Coefficient in Diffusion Processes[END_REF], [START_REF] Galtchouk | Asymptotically Efficient Sequential Kernel Estimates of the Drift Coefficient in Ergodic Diffusion Processes[END_REF] and [START_REF] Galtchouk | Efficient pointwise estimation based on discrete data in ergodic nonparametric diffusions[END_REF] with the kernel-indicator, the stochastic differential equation is replaced by regression model. It should be noted that to obtain the efficient estimator one needs to take the kernel-indicator estimator. By this reason, in this paper, we use the kernel-indicator in the sequential estimator (3.8). It also should be noted that the sequential estimator (3.8) which has the same form as in [START_REF] Arkoun | Sequential Robust Estimation for Nonparametric Autoregressive Models[END_REF], except the last term, in which the correction coefficient is replaced by the square root of the coefficient used in [START_REF] Konev | On One Property of Martingales with Conditionally Gaussian Increments and Its Application in the Theory of Nonasymptotic Inference[END_REF]. We modify this procedure to calculate the variance of the stochastic term (3.17).

Model selection

In this section we consider the nonparametric estimation problem in the non asymptotic setting for the regression model (1.6) for some set Γ ⊆ Ω. The design points (z l ) 1≤l≤d are defined in (3.1). The function S(•) is unknown and has to be estimated from observations the Y 1 , . . . , Y d . Moreover, we assume that the unobserved random variables (η l ) 1≤l≤d satisfy the properties (3.23) with some nonrandom constant m > 1 and the known random positive coefficients (σ l ) 1≤l≤d satisfy the inequality (3.24) for some nonrandom positive constants σ 0, * and σ 1, * Concerning the random sequence = ( l ) 1≤l≤n we suppose that

u * d = E p,S 1 Γ 2 d < ∞ . (4.1)
The performance of any estimator S will be measured by the empirical squared error

S -S 2 d = ( S -S, S -S) d = b -a d d l=1 ( S(z l ) -S(z l )) 2 .
Now we fix a basis (φ j ) 1≤j≤n which is orthonormal for the empirical inner product:

(φ i , φ j ) d = b -a d d l=1 φ i (z l )φ j (z l ) = 1 {i=j} . (4.2)
For example, we can take the trigonometric basis (φ j ) j≥ 1 in L 2 [a, b] defined as

φ 1 = 1 , φ j (x) = 2 b -a Tr j (2π[j/2]l 0 (x)) , j ≥ 2 , (4.3)
where the function Tr j (x) = cos(x) for even j and Tr j (x) = sin(x) for odd j, To this end, we replace the function S by these observations, i.e.

θ j,d = b -a d d l=1 Y l φ j (z l ) . (4.5)
From (1.6) we obtain immediately the following regression scheme

θ j,d = θ j,d + ζ j,d with ζ j,d = b -a d η j,d + j,d , (4.6)
where

η j,d = b -a d d l=1 η l φ j (z l ) and j,d = b -a d d l=1 l φ j (z l ) .
Note that the upper bound (3.24) and the Bounyakovskii-Cauchy-Schwarz inequality imply that

| j,d | ≤ d φ j d = d . (4.7)
We estimate the function S on the grid (3.1) by the weighted least-squares estimator

S λ (z l ) = d j=1 λ(j) θ j,d φ j (z l ) 1 Γ , 1 ≤ l ≤ d , (4.8)
where the weight vector λ = (λ(1), . . . , λ(d)) belongs to some finite set Λ ⊂ [0, 1] d , the prime denotes the transposition. We set for any

a ≤ t ≤ b S λ (t) = d l=1 S λ (z l )1 {z l-1 <t≤z l } . (4.9)
Moreover, denoting λ 2 = (λ 2 (1), . . . , λ 2 (n)) we define the following sets In order to obtain a good estimator, we have to write a rule to choose a weight vector λ ∈ Λ in (4.8). We define the empirical squared risk as

Λ 1 = {λ 2 , λ ∈ Λ} and Λ 2 = Λ ∪ Λ 1 . ( 4
Err d (λ) = S λ -S 2 d .
Using (4.4) and (4.8) we can rewrite this risk as

Err d (λ) = d j=1 λ 2 (j) θ 2 j,d -2 d j=1 λ(j) θ j,d θ j,d + d j=1 θ 2 j,d . (4.11)
Since the coefficient θ j,d is unknown, we need to replace the term θ j,d θ j,d by some of its estimators which we choose as

θ j,d = θ 2 j,d - b -a d s j,d with s j,d = b -a d d l=1 σ 2 l φ 2 j (z l ) .
(4.12)

Note that from (3.24) -(4.2) it follows that

s j,d ≤ σ 1, * . (4.13)
Similarly to [START_REF] Galtchouk | Adaptive Sequential Estimation for Ergodic Diffusion Processes in Quadratic Metric[END_REF] one needs to introduce a penalty term in the cost function to compensate such modification of the empirical risk. We choose it as

P d (λ) = b -a d d j=1 λ 2 (j)s j,d . (4.14)
Finally, we define the cost function in the following form

J d (λ) = d j=1 λ 2 (j) θ 2 j,d -2 d j=1 λ(j) θ j,d + δP d (λ) . (4.15)
where 0 < δ < 1 is some positive constant which will be chosen later. We set

λ = argmin λ∈Λ J d (λ) (4.16)
and define an estimator of S(t) of the form (4.9):

S * (t) = S λ (t) for a ≤ t ≤ b .
(4.17)

Remark 4.1. We use the procedure (4.17) to estimate the function S in the autoregressive model (1.1) through the regression scheme (1.6) generated by the sequential procedures (3.15).

Main results

In this section we formulate all main results. First we obtain the sharp oracle inequality for the selection model procedure (4.17) for the general regression model (1.6).

Theorem 5.1. There exists some constant l * > 0 such that for any weight vectors set Λ, any p ∈ P, any n ≥ 1 and 0 < δ ≤ 1/12, the procedure (4.17), satisfies the following oracle inequality

E p,S S * -S 2 d ≤ 1 + 4δ 1 -6δ min λ∈Λ E p,S S λ -S 2 d + l * νς 2 δ σ 2 1, * σ 0, * d + u * d + δ 2 P S (Γ c ) .
(5.1)

Using now Lemma A.7 we obtain the oracle inequality for the quadratic risks (1.2).

Theorem 5.2. There exists some constant l * > 0 such that for any weight vectors set Λ, any continuously differentiable function S, any p ∈ P, any n ≥ 1 and 0 < δ ≤ 1/12, the procedure (4.17) satisfies the following oracle inequality Theorem 5.3. Assume that the conditions (2.2) and (5.3) hold. Then for any p ∈ P, S ∈ Θ ,L , n ≥ 3 and 0 < δ ≤ 1/12, the procedure (4.17 We obtain the same inequality for the robust risks Theorem 5.4. Assume that the conditions (2.2) and (5.3) hold. Then for any n ≥ 3, any S ∈ Θ ,L and any 0 < δ ≤ 1/12, the procedure (4.17) satisfies the following oracle inequality

R p ( S * , S) ≤ (1 + 4δ)(1 + δ) 2 1 -6δ min λ∈Λ R p ( S λ , S) + l * ς 2 ν δ Ṡ 2 d 2 + σ 2 1, * σ 0, * d + u * d + δ 2 P S (Γ c ) . ( 5
R * ( S * , S) ≤ (1 + 4δ)(1 + δ) 2 1 -6δ min λ∈Λ R * ( S λ , S) + B * n δn , (5.5)
where the term Bn is such that for any δ > 0

lim n→∞ B * n n δ = 0 .
It is well known that to obtain the efficiency property we need to specify the weight coefficients (λ(j)) 1≤j≤n (see, for example, Galtchouk and Pergamenshchikov (2009b)). Consider for some fixed 0 < ε < 1 a numerical grid of the form A = {1, . . . , k * } × {ε, . . . , mε} ,

(5.6)

where m = [1/ε 2 ]. We assume that both parameters k * ≥ 1 and ε are functions of n, i.e. k * = k * (n) and ε = ε(n), such that

       lim n→∞ k * (n) = +∞ , lim n→∞ k * (n) ln n = 0 , lim n→∞ ε(n) = 0 and lim n→∞ n δ ε(n) = +∞ (5.7)
for any δ > 0. One can take, for example, for n ≥ 2

ε(n) = 1 ln n and k * (n) = k * 0 + √ ln n , (5.8)
where k * 0 ≥ 0 is some fixed constant. For each α = (β, l) ∈ A, we introduce the weight sequence λ α = (λ α (j)) 1≤j≤n

with the elements

λ α (j) = 1 {1≤j<j * } + 1 -(j/ω α ) β 1 {j * ≤j≤ω α } , (5.9)
where

j * = 1 + [ln n], ω α = (d β l n) 1/(2β+1) and d β = (β + 1)(2β + 1) π 2β β .
Now we define the set Λ as Λ = {λ α , α ∈ A} .

(5.10)

Note that these weight coefficients are used in [START_REF] Konev | On One Property of Martingales with Conditionally Gaussian Increments and Its Application in the Theory of Nonasymptotic Inference[END_REF]Pergamenshchikov (2012, 2015) for continuous time regression models to show the asymptotic efficiency. It will be noted that in this case the cardinal of the set Λ is ν = k * m. It is clear that properties (5.7) imply condition (5.3).

6 Properties of the regression model (1.6)

In order to prove the oracle inequalities we need to study the conditions introduced in [START_REF] Konev | Efficient Robust Nonparametric Estimation in a Semimartingale Regression Model[END_REF] for the general semi-martingale model. To this end we set for any λ ∈ R d the functions

B(λ) = b -a √ d d j=1 λ(j) η j,d , η j,d = η 2 j,d -E p,S η 2 j,d . ( 6
= (λ 1 , . . . , λ d ) ∈ R d E p,S 1 Γ B 2 (λ) ≤ 10(b -a)σ 1, * m E p,S P d (λ) . (6.2)
where m is defined in (3.23).

Proof. First note that the random variable η j,d can be represented as

η j,d = b -a d d l=1 φ 2 j (z l ) ηl + 21 {l≥2} υj,l η l ,
where ηl = η 2 l -σ 2 l and υj,l = φ j (z l )

l-1 r=1 φ j (z r )η r . Therefore, we can rewrite the term B(λ) as

B(λ) = B 1 (λ) + 2B 2 (λ) .
The terms B 1 (λ) and B 2 (λ) are defined as

B 1 (λ) = (b -a) 2 d √ d d l=1 ψ 1,l (λ)η l and B 2 (λ) = (b -a) 2 d √ d d l=2 ψ 2,l (λ)η l ,
where

ψ 1,l (λ) = d j=1 λ(j)φ 2 j (z l ) and ψ 2,l (λ) = d j=1 λ(j)υ j,l . So, E p,S B 2 (λ) ≤ 2E p,S B 2 1 (λ) + 8E p,S B 2 2 (λ) . (6.3)
Taking into account property (4.2) and Bounyakovskii -Cauchy -Schwarz inequality we get

ψ 2 1,l (λ) ≤ d j=1 λ 2 (j)φ 2 j (z l ) d j=1 φ 2 j (z l ) = d b -a d j=1 λ 2 (j)φ 2 j (z l ) .
In view of properties (3.23) we obtain that

E p,S B 2 1 (λ) = (b -a) 4 d 3 d l=1 ψ 2 1,l (λ) E p,S η2 l ≤ (b -a) 4 d 3 d l=1 ψ 2 1,l (λ) E p,S η 4 l ≤ σ 1, * m(b -a) 3 d 2 E p,S d j=1 λ 2 (j) d l=1 σ 2 l φ 2 j (z l ) = σ 1, * (b -a) m E p,S P d (λ) .
To estimate the last term in the right hand side of the inequality (6.3), noting that the term ψ 2,l can be represented as

ψ 2,l (λ) = l-1 r=1 g l,r η r ,
where g l,r = d j=1 λ(j) φ j (z l )φ j (z r ), we use properties (3.23) to obtain

E p,S B 2 2 (λ) = (b -a) 4 d 3 d l=2 E p,S ψ 2 2,l (λ) η 2 l ≤ σ 1, * (b -a) 4 d 3 d l=2 E p,S ψ 2 2,l (λ) = σ 1, * (b -a) 4 d 3 d l=2 l-1 r=1 g 2 l,r E p,S σ 2 r ≤ σ 1, * (b -a) 4 d 3 d r=1 E p,S σ 2 r d l=1 g 2 l,r = σ 1, * (b -a) E p,S P d (λ) .
Hence Property 6.1. 2 Now we need the following moment bound.

Property 6.2. For any non random v

1 , . . . , v d E   d j=1 v j η j,d   2 ≤ σ 1, * d j=1 v 2 j . (6.4) Proof. Note that E   d j=1 v j η j,d   2 = b -a d E d l=1 σ 2 l   d j=1 v j φ j (z l )   2 ≤ σ 1, * (b -a) d d l=1   d j=1 v j φ j (z l )   2 .
By applying the orthonormal property (4.2) we obtain the desired inequality. Hence Property 6.2. 2

Proofs

7.1 Proof of Theorem 3.1

First recall that

S ι l = 1 A ι l ι l j=1 Q l,j y j-1 y j and S ι l = min(max( S ι l , -1 + ˜ ), 1 -˜ ) ,
where ˜ = 1/(2 + ln n). Note that for sufficiently large n, for which we have ˜ < and then S(z l ) ∈ [-1 + ˜ ; 1 -˜ ]. We can write

| S ι l -S(z l )| ≤ | S ι l -S(z l )| ≤ ι l j=k 1,l y 2 j-1 |S(x j ) -S(z l )| ι l j=k 1,l y 2 j-1 + |I n | ,
where

I n = ι l j=k 1,l y j-1 ξ j / ι l j=k 1,l y 2 j-1 . Taking into account that |x j -z l | ≤ h for k 1,l ≤ j ≤ k 2,l , we obtain that for any S ∈ Θ ,L , | S ι l -S(z l )| ≤ Lh + |I n | .
So, for sufficiently large n

P p,S | S ι l -S(z l )| > 0 ≤ P p,S I n > 0 2 ≤ P p,S I n > 0 2 , Ξ + P p,S (Ξ c ) , (7.1)
where

Ξ = Υ m 0 ,m 1 (z l ) ≤ 1/2 , m 0 = k 1,l -2, m 1 = ι l -1 and Υ m 0 ,m 1 (z l ) is defined in (A.7
). Hence we obtain the following inequality on the set Ξ:

ι l j=k 1,l y 2 j-1 = (ι l -k 1,l + 1) 1 1 -S 2 (z l ) + Υ m 0 ,m 1 (z l ) ≥ q 2 .
Therefore, for any p > 2,

P p,S I n > 0 2 , Ξ ≤ P p,S   ι l j=k 1,l y j-1 ξ j > q 2   ≤ 2 p q p E p,S ι l j=k 1,l y j-1 ξ j p . (7.2)
Using here the correlation inequality (A.2) and the bound (A.6), we obtain that

max 1≤l≤d sup S∈Θ ,L sup p∈P E p,S ι l j=k 1,l y j-1 ξ j p ≤ c p q p/2 .
3.1. Applying this bound in (7.1) and using Lemma A.6 we obtain Theorem 3.1.

7.2 Proof of Theorem 3.2

First note, that

P p,S (Γ c ) ≤ d l=1 P p,S A ι l ,k 2,l -1 < H l .
Moreover, note that in view of definition (A.7) the term A ι l ,k 2,l -1 can be represented as

A ι l ,k 2,l -1 = (m 1,l -m 0,l ) 1 γ l + Υ m 0,l ,m 1,l (z l ) ,
where m 0,l = ι l -1 and m 1,l = k 2,l -2. Taking into account the definition of H l in (3.11) and the fact that 0 < γ l , γ l ≤ 1 and that | γ l -γ l | ≤ 2| S ι l -S(z l )|, we obtain

P p,S A ι l ,k 2,l -1 < H l = P p,S 1 γ l + Υ m 0,l ,m 1,l (z l ) < 1 - γ l ≤ P p,S 1 γ l - 1 γ l > 2 + P p,S Υ m 0,l ,m 1,l (z l ) > 2 ≤ P p,S S ι l -S(z l ) > 3 4 + P p,S Υ m 0,l ,m 1,l (z l ) > 2 .
Applying here Theorem 3.1 and Lemma A.6 we obtain Theorem 3.2.

Proof of Theorem 3.3

Note that, for any m ≥ 1

E p,S max 1≤j≤n y 4 j ≤ n b/2 + n j=1 +∞ n b/2 P p,S y 4 j ≥ z dz ≤ n b/2 + n max 1≤j≤n E p,S |y j | 4m +∞ n b/2 z -m dz = n b/2 + max 1≤j≤n E p,S |y j | 4m n 1-b(m-1)/2 m -1 .
Choosing here m > 1 + 2/b and using the bound (A.6) we obtain the property (3.26). Hence Theorem 3.3.

Proof of Theorem 5.1

First of all, note that on the set Γ we can represent the empirical squared error Err d (λ) in the form

Err d (λ) = J d (λ) + 2 d j=1 λ(j)θ j,d + S 2 d -δ P d (λ) (7.3)
with θ j,d = θ j,d -θ j,d θ j,d . From (4.6) we find that

θ j,d = θ j,d ζ j,d + b -a d η j,d + 2 b -a d η j,d j,d + 2 j,d ,
where η j,d = η 2 j,d -s j,d . Now putting

M (λ) = d j=1 λ(j) θ j,d ζ j,d , (7.4)
we rewrite (7.3) as follows

Err d (λ) = J d (λ) + 2M (λ) + 2 1 √ d B(λ) + 2∆(λ) + S 2 d -δ P d (λ) , (7.5)
where

B(λ) is given in (6.1), ∆(λ) = ∆ 1 (λ) + ∆ 2 (λ), ∆ 1 (λ) = d j=1 λ(j) 2 j,d and ∆ 2 (λ) = 2 b -a d d j=1 λ(j) η j,d j,d .
In view of Property 6.1, for any λ ∈ R d , For ∆ 1 (λ), taking into account the properties of Fourier coefficients we obtain that sup

E p,S 1 Γ B 2 (λ) ≤ 10σ 1, * (b -a) m E p
λ∈[0,1] d |∆ 1 (λ)| ≤ d j=1 2 j,d = 2 d .
(7.8)

To estimate the term ∆ 2 (λ) we recall that, for any ε > 0 and any x, y ∈ R 2xy ≤ εx 2 + ε -1 y 2 . (7.9)

Therefore, for some 0 < ε < 1,

|∆ 2 (λ)| ≤ ε b -a d d j=1 λ 2 (j) η 2 j,d + 2 d ε = εP d (λ) + ε |B(λ 2 )| √ d + 2 d ε ,
where the vector λ 2 ∈ Λ 1 as in (4.10). Thus, for any λ ∈

[0, 1] d , |∆(λ)| ≤ εP d (λ) + ε B(λ 2 )| √ d + 2ε -1 2 d . (7.10) Putting M 1 (λ) = 2 B(λ) √ d + 2∆(λ) ,
we can rewrite the empirical risk (7.5) as

Err d (λ) = J d (λ) + 2M (λ) + M 1 (λ) + S 2 d -δ P d (λ) . (7.11)
From (7.10) we obtain

|M 1 (λ)| ≤ 2 |B(λ)| √ d + 2 |B(λ 2 )| √ d + 2εP d (λ) + 4ε -1 2 d .
Moreover, setting

B * = sup λ∈Λ B 2 (λ) P d (λ) + B 2 (λ 2 ) P d (λ 2 )
and taking into account that P d (λ 2 ) ≤ P d (λ) for any λ ∈ Λ, we get

2 |B(λ)| √ d + 2 |B(λ 2 )| √ d ≤ 2εP d (λ) + ε -1 B * d .
By choosing ε = δ/4 we find

|M 1 (λ)| ≤ δP d (λ) + 16 δ Υ d , Υ d = B * 4d + 2 d .
(7.12)

Now from (7.11) we obtain that, for some fixed λ 0 from Λ,

Err d ( λ) -Err d (λ 0 ) = J d ( λ) -J d (λ 0 ) + 2 M ( µ) + M 1 ( λ) -δP d ( λ) -M 1 (λ 0 ) + δP d (λ 0 ) ,
where µ = λ -λ 0 . By the definition of λ in (4.16) we obtain on the set Γ

Err d ( λ) ≤ Err d (λ 0 ) + 2 M ( µ) + 32 Υ d δ + 2δP d (λ 0 ) . (7.13)
From (7.6) and (7.7) it follows that

E p,S 1 Γ B * ≤ λ∈Λ E p,S 1 Γ B 2 (λ) P d (λ) + B 2 (λ 2 ) P d (λ 2 ) ≤ 10σ 1, * (b -a) m λ∈Λ P 1,d (λ) P 0,d (λ) + P 1,d (λ 2 ) P 0,d (λ 2 ) = 20 m(b -a)ν σ * .
and σ * = σ 2 1, * σ 0, * . Therefore, for 0 < δ < 1 this inequality allows to bound Υ d as (7.14) where u * d is given by (4.1). Now we study the second term on the right-hand side of inequality (7.13). For any weight vector λ ∈ Λ we set µ = λ -λ 0 . Then we decompose this term as Note now that from (7.17) it follows that

E p,S 1 Γ Υ d ≤ 5 m(b -a)σ * ν d + u * d ,
M (µ) = Z(µ) + V (µ) (7.
E p,S 1 Γ Z * ≤ µ∈Λ-λ 0 E p,S 1 Γ Z 2 (µ) D(µ) ≤ νσ 1, * (b -a) . (7.20)
Now we estimate the first term on the right-hand side of the inequality (7.19). On the set Γ we have Šµ

2 d -S µ 2 d = d j=1 µ 2 (j)(θ 2 j,d -θ 2 j,d ) ≤ -2 d j=1 µ 2 (j) θ j,d ζ j,d = -2Z 1 (µ) -2V 1 (µ) , (7.21)
where

Z 1 (µ) = b -a d d j=1 µ 2 (j)θ j,d η j,d and V 1 (µ) = d j=1 µ 2 (j) θ j,d j,d .
Taking into account that |µ(j)| ≤ 1, similarly to inequality (7.17), we find

E p,S 1 Γ Z 2 1 (µ) ≤ σ 1, * D(µ) .
Moreover, for the random variable

Z * 1 = sup µ∈Λ-λ 0 Z 2 1 (µ) D(µ) ,
we obtain the same upper bound as in (7.20), i.e. Therefore, on the set Γ Šµ

2 d ≤ S µ 2 d + 2δ Šµ 2 d + Z * 1 dδ + 2 d δ , i.e. Šµ 2 d ≤ 1 1 -2δ S µ 2 d + 1 (1 -2δ)δ Z * 1 d + 2 d . ( 7
= Z * + Z * 1 yield on the set Γ 2M ( µ) ≤ 2δ 1 -2δ S µ 2 d + 1 δ(1 -2δ) Z * 2 d + 2 d ≤ 4δ(Err d ( λ) + Err d (λ 0 )) 1 -2δ + 1 δ(1 -2δ) Z * 2 d + 2 d .
Therefore from the preceding inequality and (7.13) we obtain

Err d ( λ)1 Γ ≤ 1 + 2δ 1 -6δ Err d (λ 0 )1 Γ + 32(1 -2δ) δ(1 -6δ) Υ d 1 Γ + 1 δ(1 -6δ) Z * 2 d + 2 d 1 Γ + 2δ(1 -2δ) 1 -6δ P d (λ 0 )1 Γ
and through the inequalities (7.14), (7.20) and (7.22) we estimate the empirical risk as

E p,S Err d ( λ)1 Γ ≤ 1 + 2δ 1 -6δ E p,S Err d (λ 0 )1 Γ + 32(1 -2δ) δ(1 -6δ) 5 mσ * ν(b -a) d + u * d + 1 δ(1 -6δ) 2νσ 1, * (b -a) d + u * d + 2δ(1 -2δ) 1 -6δ E p,S 1 Γ P d (λ 0 ) .
Taking into account that σ * ,1 ≤ σ * and that 1 -6δ > 1/2 for 0 < δ < 1/12, we get

E p,S Err d ( λ)1 Γ ≤ 1 + 2δ 1 -6δ E p,S Err d (λ 0 )1 Γ + 320 δ ( m + 1)σ * ν(b -a) d + u * d + 2δ(1 -2δ) 1 -6δ E p,S 1 Γ P d (λ 0 ) .
By applying Lemma A.4 with ε = 2δ we get that

E p,S Err d ( λ)1 Γ ≤ 1 + 4δ 1 -6δ E p,S Err d (λ 0 )1 Γ + 320 δ ( m + 1)σ * ν(b -a) d + 3u * d + 10δ σ 1, * mP p,S (Γ c ) .
Taking (Γ c ) respectively, we come to the inequality (5.1). Hence Theorem 5.1.

A Appendix

A.1 Burkhölder inequality

We need the following from [START_REF] Shiryaev | Probability[END_REF].

Lemma A.1. Let (M k ) 1≤k≤n be a martingale. Then for any q > 1

E |M n | q ≤ b * q E   n j=1 (M j -M j-1 ) 2   q/2 , (A.1)
where the coefficient b * q = 18(q) 3/2 /(q -1) 1/2 .

A.2 Properties of the sequential procedures

Lemma A.2. The properties (3.23) hold for the random variables (η l ) 1≤l≤d defined in (3.22).

Proof. First, we set F j = σ{ξ 1 , . . . , ξ j } for 1 ≤ j ≤ n and as usual F 0 = {Ω, ∅}. Moreover, note that

η l = n j=1
ťl,j ξ j and ťl,j = σ 2 l 1 {ι l ≤j<τ l } Ql,j + 1 {j=τ l } κl Ql,τ l .

Taking into account that ťl,j is F j-1 -measurable for any 1 ≤ j ≤ n and

A.3 Correlation inequality

Now we give the correlation inequality from [START_REF] Galtchouk | Uniform concentration inequality for ergodic diffusion processes observe at discrete times[END_REF].

Lemma A.3. Let (Ω, F, (F j ) 1≤j≤n , P) be a filtered probability space and (X j , F j ) 1≤j≤n be a sequence of random variables such that for some p ≥ 2 Moreover, taking into account that E p,S I 1 = 0 and in view of Theorem (6.2) E p,S I 2 1 ≤ σ 1, * S 2 d . So, recalling that that S d ≤ b -a, we estimate E p,S I 1 1 Γ as E p,S I 1 1 Γ = E p,S I 1 1 Γ c ≤ σ 1, * P p,S (Γ c ) .

Hence Lemma A.4.

  Theorem 3.3. For any b > 0 the sequence (υ n ) n≥1 satisfies the following limiting equality lim n→∞ n -b υn = 0 . (3.26) Remark 3.1. It should be noted that the property (3.26) means that the asymptotic behavior of the upper bound (3.25

  [x] denotes the integer part of x. and l 0 (x) = (x -a)/(b -a). Note that, using the orthonormality property (4.2) we can represent for any 1 ≤ l ≤ d the function S as S(z l ) = d j=1 θ j,d φ j (z l ) and θ j,d = S, φ j d . (4.4) So, to estimate the function S we have to estimate the Fourier coefficients (θ j,d ) 1≤j≤d .

. 2 )

 2 Now we assume that the cardinal ν of Λ and the parameter ς in the density family (2.1) are functions of the number observations n, i.e. ν = ν(n) and ς = ς(n) such that for any δ > 2 -3.3 and the bounds (3.24) -(3.25) we obtain the oracle inequality for the estimation problem for the model (1.1).

. 1 )

 1 Property 6.1. For any d ≥ 1 and any λ

  ,S P d (λ) . (7.6) Note that the inequalities (3.24) imply that P 0,d (λ) ≤ P d (λ) ≤ P 1,d (λ) , (7.7) where P 0,d (λ) = σ 0, * (b -a)|λ| 2 d and P 1,d (λ) = σ 1, * (b -a)|λ| 2 d .

  ) θ j,d η j,d and V (µ) = d j=1 µ(j) θ j,d j,d .We define now the weighted discrete Fourier transformation of S, i.e. Property 6.2 we can estimate the term Z(µ) asE p,S 1 Γ Z 2 (µ) ≤ σ 1, * (b -a) d Šµ 2 d := σ 1, * (b -a)D(µ) .(7.17)Moreover, by the inequalities (7.9) with ε = δ and (7.8) we can estimate V (µ

E

  p,S Z * 1 1 Γ ≤ νσ 1, * (b -a) . (7.22) Furthermore, similarly to (7.18) we estimate the second term in (7.21) as 2|V 1 (µ)| ≤ δ Šµ

  Note also that G l = σ{η 1 , . . . , η l-1 , σ l , } ⊂ F ι l . Noting thatE η l |F ι l = 0 and E η 2 l |F ι l = 1 ,we obtain the first two equalities in (3.23). As to the last inequality, note that through (A.1) we can write E last bound in (3.23). Hence Lemma A.2. 2

max 1≤j≤n E

 1≤j≤n |X j | p < ∞ . bound for the penalty term Lemma A.4. For sufficiently large n and 0 < ε < 1,E p,S P d (λ) ≤ 1 1 -ε E p,S Err d (λ)1 Γ + u * mP p,S (Γ c ) .Proof. Indeed, by the definition of Err d (λ) on the set Γ we have Err d (λ(j) -1)θ j,d η j,d andI 2 = d j=1 λ 2 (j) j,d η j,d ,we get on the set Γ the following lower bound for the empirical riskErr d (λ) ≥ bj) s j,d -E p,S 1 Γ c d j=1 λ 2 (j) η 2 j,d .Using the correlation inequality (A.2) and the upper bound for the fourth moment in (3.23) we obtain E p,S η 4 j,d ≤ 64 m σ 2 j) s j,d -8σ 1, * d m P p,S (Γ c ) . ≥ E p,S P d (λ) -8(b -a)σ 1, * mP p,S (Γ c ) .

  into account the definition of m in (3.23) and that m * 4 ≤ 3ς 2 , then by replacing E p,S Err d ( λ)1 Γ and E p,S Err d (λ 0 )1 Γ by E p,S S * -S 2 d -S 2 d P p,S (Γ c ) and E p,S S λ 0 -S 2 d -S 2 d P p,S

Acknowledgements. The last author was partially supported by the Russian Federal Professor program (project no. 1.472.2016/1.4, the Ministry of Education and Science of the Russian Federation), the research project no. 2.3208.2017/4.6 (the Ministry of Education and Science of the Russian Federation), by RFBR Grant 16-01-00121 A and by "The Tomsk State University competitiveness improvement program" grant 8.1.18.2018.

This work was supported by the RSF grant 17-11-01049 (National Research Tomsk State University).

A.5 Properties of the model (1.1) Lemma A.5. For all t ∈ N * and 0 < < 1, the random variables y k in (1.1) satisfy the following :

Proof. This lemma is shown in Arkoun (2011) (Lemma A.1). We set

+ is positive part of a and γ l is defined in (3.10).

Lemma A.6. Assume that the bounds m 0 and m 1 in (A.7) are such that for some

Then, for any b > 0

where 0 = 0 (n) → 0 as n → ∞ is such that lim n→∞ n δ 0 = ∞ for any δ > 0. Proof. This lemma is shown in Arkoun (2011) (Lemma A.2). where c j are some constants. Then for all ε > 0, the function ∆ = f -g satisfies the following inequalities

A.6 Properties of the norms

Proof. Lemma A.7 is proven in [START_REF] Konev | Robust model selection for a semimartingale continuous time regression from discrete data[END_REF]. (Lemma A.2.)