Karim Benhenni
email: karim.benhenni@univ-grenoble-alpes.fr

Didier A Girard
email: didier.girard@univ-grenoble-alpes.fr

Sana Louhichi
email: sana.louhichi@univ-grenoble-alpes.fr

On the excess of average squared error for data-driven bandwidths in nonparametric trend estimation

We consider the problem of the optimal selection of the smoothing parameter h in kernel estimation of a trend in nonparametric regression models with strictly stationary errors. We suppose that the errors are stochastic volatility sequences. Three types of volatility sequences are studied: the log-normal volatility, the Gamma volatility and the log-linear volatility with Bernoulli innovations. We take the weighted average squared error (ASE) as the global measure of performance of the trend estimation using h and we study two classical criteria for selecting h from the data, namely the adjusted generalized cross validation and Mallows-type criteria. We establish the asymptotic distribution of the gap between the ASE evaluated at one of these selectors and the smallest possible ASE. A Monte-Carlo simulation for a log-normal stochastic volatility model illustrates that this asymptotic approximation can be accurate even for small sample sizes.

Introduction

Nonparametric trend estimation is a very popular field of research in Statistics and is used in different domains of application. There are several nonparametric estimates of the trend in time series models or the mean function of stochastic processes. Many of these estimates are constructed from a kernel function that depends on a smoothing parameter h known, in nonparametric statistical literature, as a bandwidth. The choice of this parameter is crucial since it has an important impact on the performance of the kernel estimates. Some criteria are available for choosing this parameter but there are mostly based on models with independent errors; the plug-in method, see for instance [START_REF] Ruppert | An effective bandwidth selector for local least squares regression[END_REF] and [START_REF] Fan | A study of variable bandwidth selection for local polynomial regression[END_REF], the cross-validation (CV) and the generalized cross-validation (GCV), see for instance [START_REF] Rice | Bandwidth choice for nonparametric regression[END_REF], [START_REF] Härdle | How far are automatically chosen regression smoothing parameters from their optimum[END_REF] and [START_REF] Girard | Asymptotic comparison of (partial) cross-validation, GCV and randomized GCV in non-parametric regression[END_REF] among others. However, in the case of dependent errors, there are very limited available results in the literature concerning the selection methods of the smoothing parameter, see for instance the review by [START_REF] Opsomer | Nonparametric regression with correlated errors[END_REF]. [START_REF] Hall | On bandwidth choice in nonparametric regression with both short and long-range dependent errors[END_REF] develop bootstrap and cross-validation methods to select the smoothing parameter under short and long range dependance.

In [START_REF] Benhenni | On bandwidth selection problems in nonparametric trend estimation under martingale difference errors[END_REF], we considered dependent strictly stationary martingale difference errors with an application to ARCH(1) errors. Based on four criteria defining, via minimization, two optimal and two data-driven smoothing parameters (or "two selectors"), we showed that the minimizers of these four criteria are "first-order equivalent" in probability (using now a classical terminology as specified in equation (9) below). Moreover, we gave a normal asymptotic behavior of the difference between, in particular, the minimizer of the average squared error (ASE, in short, defined by equation (3) below) and that of the Mallows-type criterion. In this paper, we are mainly interested in studying the excess of the ASE for such selectors. Let us recall that the excess of the ASE ("ASE-excess" in short) associated with any particular selector, is defined as the increase of ASE when one compares it with its lowest possible value when varying h for the available data. Early studies of the ASE-excess, [START_REF] Hall | Extent to which least-squares cross-validation minimises integrated square error in nonparametric density estimation[END_REF], and [START_REF] Mammen | A short note on optimal bandwidth selection for kernel estimators Statistics and Probability letters[END_REF], put an emphasis on the fact that the asymptotic behavior of this excess is interpretable, more than the asymptotic behavior of the difference between selectors.

In this paper we provide some extensions to the theoretical results of Benhenni, Girard and Louhichi (2021) (abbreviated as BGL henceforth) in two directions. Firstly, we show, under the same assumptions as in Theorem 3.1 of BGL, that the asymptotic behavior of the ASE-excess which has been established by [START_REF] Härdle | How far are automatically chosen regression smoothing parameters from their optimum[END_REF] for independent and identically distributed (i.i.d., in short) errors, still holds. Secondly, we show that the required assumptions for the sequence of errors are fulfilled by three common time series volatility models. Volatility has been one of the most active areas of research in time series econometrics and economic forecasting. It may be modeled as an unobserved component following some latent stochastic process, such as autoregression. The resulting models are called stochastic volatility (SV) models and have been the focus of considerable attention, see for instance [START_REF] Taylor | Modelling stochastic volatility: a review and comparative study[END_REF], [START_REF] Ghysels | Stochastic volatility[END_REF], [START_REF] Shephard | Time Series Models with Econometric, Finance and Other Applications[END_REF] and [START_REF] Billo | Stochastic Volatility Models: A Survey with Applications to Option Pricing and Value at Risk[END_REF]. Here, we concentrate on three stochastic volatility models (SV): the log-normal volatility, the Gamma volatility and the log-linear volatility with Bernoulli innovations. The first two SV models satisfy a strong mixing condition with a decreasing power bound whereas the Bernoulli SV model does not satisfy any mixing condition.

In addition, an extensive Monte-Carlo simulation study for the log-normal stochastic volatility model with various model parameters is provided in section 4 and demonstrates that the asymptotic behavior established here can be quite realistic for a moderate number of observations, n, over a large range of parameters.

The paper is organized as follows. Section 2 introduces the nonparametric model, defines the different criteria for the selection of the smoothing parameter h, explains the main assumptions and recalls previous results that are required for the present study. Section 3 states the main results of this paper. Theorem 3.1 studies the asymptotic distribution of the ASE-excess for optimal bandwidths. Proposition 3.1 is an application of Theorem 3.1 to the SV models. Corollaries 3.1, 3.2 and 3.3 study, respectively, the log-normal SV, the Gamma SV and the loglinear SV with Bernoulli innovations. Section 4 is the Monte Carlo simulation study mentioned above. Finally, Section 5 is devoted to the proofs.

Model, selection criteria and useful tools

Let ( i ) i≥0 be a strictly stationary sequence of centered random variables with finite second moment. Let σ 2 = Var( 1 ) and R be the correlation matrix of the vector ( 1 , • • • , n ) t (where v t denotes the transpose of the vector v). Consider the following regression model, defined for

i = 1, • • • , n, by Y i = r(x i ) + i , x i = i n , (1)
where r is an unknown regression function with second order continuous derivative and the x i 's are equally spaced fixed design. We are interested, here, in the Priestley-Chao estimator of r defined, for x ∈ IR, by

r(x) = n i=1 l i (x)Y i , with l i (x) = 1 nh K x -x i h ,
where K is a compactly supported, even kernel, with class C 2 ([-1, 1]) and h is a positive bandwidth smaller than 0.5. The above curve estimator can also be written in the following matrix form:

r = LY, (2)
with r = (r(x 1 ), • • • , r(x n )) t , Y = (Y 1 , • • • , Y n ) t
and L = (l j (x i )) 1≤i,j≤n is known as the smoothing matrix or the hat matrix. Since the estimator r depends on a smoothing parameter h, some criteria are needed for choosing h. One first criterion is the ASE, T n (h), defined by

T n (h) = 1 n n i=1 u(x i )(r(x i ) -r(x i )) 2 , (3)
where u = u (for a fixed positive less than 0.5) is a known positive function of class C 1 and [ , 1 -]-compactly supported. This function is introduced in order to eliminate the boundary effects of the compactly supported kernel K. We know from Lemma 2.1, in BGL, that if ∞ k=1 k|Cov( 0 , k )| < ∞ then the mean weighted average squared error, IE(T n (h)), is very close to the quantity D n (h), defined by

D n (h) = h 4 4 1 0 u(x)r 2 (x)dx 1 -1 t 2 K(t)dt 2 + 1 nh ( 1 0 u(x)dx) 1 -1 K 2 (y)dy σ 2 + 2 ∞ k=1 Cov( 0 , k ) . (4) Let h * n ∈ argmin h>0 D n (h).
We suppose in the sequel that

1 0 u(x)r 2 (x)dx = 0. Then clearly, h * n = cn -1/5 , with c = ( 1 0 u(x)dx) 1 -1 K 2 (y)dy (σ 2 + 2 ∞ k=1 Cov( 0 , k )) 1 0 u(x)r 2 (x)dx( 1 -1 t 2 K(t)dt) 2 1/5 . ( 5
)
Let H n be a neighborhood of h * n , i.e, H n = [an -1/5 , bn -1/5 ] for some fixed a < c < b. Define also, ĥn ∈ argmin h∈Hn T n (h).

Both those two criteria (D n (h) and T n (h)) depend, in particular, on the unknown function r, so they cannot, of course, directly provide a data-driven selection of bandwidth. A well-known criterion, which is an unbiased estimator (up to an additive constant) of the mean weighted average squared error IE(T n (h)), is the following variant of Mallows' criterion introduced for other purposes than ours and adapted to dependent errors with known covariance matrix σ 2 R:

CL(h) = n -1 U 1/2 (I -L)Y 2 + 2σ 2 n -1 tr(U RL), ( 6
)
where I is the identical matrix, U is the diagonal matrix defined by diag(u(x 1 ),

• • • , u(x n )), Y = (Y 1 , • • • , Y n ) t , L = (l j (x i ))
1≤i,j≤n as defined by (2), and tr(M ) denotes the trace of a square matrix M . We consider, according to our purpose, ĥM to be the minimizer of this dependent version of the Mallows-type criterion:

ĥM ∈ argmin h∈Hn CL(h).

Finally, let G X (h) be the classical GCV-type criterion and ĥG be its minimizer:

ĥG ∈ argmin h∈Hn G X (h) with G X (h) = n -1 U 1/2 (I -L)Y 2 × Ξ X tr(U L) tr(U ) , (7)
where Ξ X satisfies, for small values of |t|, Ξ X (t) = 1 + 2t + O(t 2 ) with second derivative Ξ X bounded on a neighborhood of 0.

(8)

This GCV-type criterion G X is well adapted to strictly stationary uncorrelated dependent errors.

The bandwidth h * n has the advantage of being explicitly derived, but its drawback is that it cannot be implemented in practice since it depends on unknown quantities such as σ 2 and r . The bandwidth ĥG has no explicit expression but on the other hand it can be implemented in practice since it is obtained by minimizing an observable criterion. Because ĥn is recognized as the ideal, albeit unobservable bandwidth, then a comparison of ĥG (or ĥM ) with ĥn has been an initial subject of interest for many authors.

Since we are interested in this study to move out of the basic framework where the errors ( i ) 1≤i≤n form a sequence of i.i.d. random variables, a dependency condition should be assumed.

We consider here the rather general condition of "martingale difference sequences" (MDS, see Conditions (A) below) since, on the one hand, there are many models satisfying this notion (see the next section) and since, on the other hand, studying this framework of dependency makes it possible to study the most general case of strictly stationary random variables as is done by [START_REF] Peligrad | A maximal Lp-inequality for stationary sequences and its applications[END_REF]. Note also, that MDS are uncorrelated dependent random variables, so that the above GCV-type criterion, G X , is well adapted. We also need, for technical reasons, a higher moment condition on the distribution of 1 . All these assumptions are summarized in the following Conditions (A).

Conditions (A). Assume that the errors ( i ) i≥0 form a strictly stationary MDS with respect to some natural filtration (F i ) i≥1 , i.e, for any i > 0, i is F i -measurable and IE( i |F i-1 ) = 0 almost surely (a.s., in short). Suppose also that IE( 2p 1 ) < ∞ for some p > 8.

Under Conditions (A), the bandwidths h * n , ĥn , ĥM and ĥG are first-order equivalent in probability (and then both the CL criterion and the GCV-type criterion enjoy the "asymptotic optimality" property), this means that ĥn

h * n , ĥM h * n , ĥG h * n (9)
all converge in probability to 1 as n tends to infinity (see Proposition 3.1 in BGL).

Theorem 3.1 in BGL gives the rate at which ĥMĥn and ĥGĥn converge in distribution to a centered normal law, and it needs an additional dependence condition, Condition (B) below,

Condition (B). There exists a positive decreasing function Φ defined on IR

+ satisfying ∞ s=1 s 4 Φ(s) < ∞,
and for any positive integer q ≤ 6, 1

≤ i 1 ≤ • • • ≤ i k < i k+1 ≤ • • • ≤ i q ≤ n such that i k+1 -i k ≥ max 1≤l≤q-1 (i l+1 -i l ), |Cov( i 1 • • • i k , i k+1 • • • iq )| ≤ Φ(i k+1 -i k ), (10)
where

i 1 • • • i k denotes the product k =1 i (and likewise for i k+1 • • • iq ).
Condition (B) is known in the literature (see [START_REF] Doukhan | A new weak dependence condition and applications to moment inequalities[END_REF]). It allows, in particular, to control the higher order moments of partial sums of dependent random sequences. Now, if both Conditions (A) and (B) are satisfied, then it holds that, by Theorem 3.1 in BGL, n 3/10 ( ĥMĥn ) and n 3/10 ( ĥGĥn ) (11)

both converge in distribution to a centered normal law with variance Σ 2 given by

Σ 2 = 4σ 6/5 5 2 A 8/5 B 2/5 ( t 2 K(t)dt) 2 1 0 u 2 (x)r 2 (x)dx + 8σ 6/5 5 2 A 3/5 B 7/5 1 0 u 2 (x)dx (K -G) 2 (u)du, ( 12
)
where σ 2 = IE( 2 1 ), G is the function defined for any x ∈ IR by G(x) = -xK (x) and

A = 1 0 u(x)r 2 (x)dx t 2 K(t)dt 2 , B = 1 0 u(x)dx K 2 (t)dt.
3 Main results with applications to stochastic volatility models

We have thus studied sufficient conditions on the sequence of errors under which these four bandwidths ( ĥM , ĥn , ĥG and ĥn ) are asymptotically equivalent and their differences have asymptotic normal distributions. Now, let us recall that selecting the bandwidth is only a means to an end: minimising the error in the estimation of r. Since the measure of error that we consider is the ASE, as it is often the case in supervised learning, it is now important to compare these selectors via their corresponding ASEs rather than comparing them directly with each other.

Our purpose is thus to study the asymptotic behaviors of the gaps, called the ASE-excesses, T n ( ĥM ) -T n ( ĥn ) and T n ( ĥG ) -T n ( ĥn ). That is, we are interested in the study of the deviations of the error T n ( ĥM ) (respectively T n ( ĥG )) from the minimal possible value of the ASE, which is T n ( ĥn ).

The following theorem gives, under Conditions (A) and (B), the rate at which these gaps tend to 0, extending the results stated by [START_REF] Härdle | How far are automatically chosen regression smoothing parameters from their optimum[END_REF] for i.i.d. errors.

Theorem 3.1. Suppose that Conditions (A) and (B) are satisfied. Then both n(T n ( ĥM ) -T n ( ĥn )) and n(T n ( ĥG ) -T n ( ĥn ))

converge in distribution to a CX 2 (1) law, where X 2 (1) is the chi-square distribution with one degree of freedom and C is the positive constant given by,

C = 2σ 2 5A t 2 K(t)dt 2 1 0 u 2 (x)r 2 (x)dx + 2A B 1 0 u 2 (x)dx (K -G) 2 (t)dt ,
where σ 2 = IE( 2 1 ), G, A and B are defined as in (12). Remark. As expected, the constant C in this asymptotic distribution is the same as in the i.i.d. case established by [START_REF] Härdle | How far are automatically chosen regression smoothing parameters from their optimum[END_REF]. If we choose the weight function u to be proportional to u 2 , then it is easy to check that C gets so simplified that it no longer depends on the function r. In this case and as an immediate consequence of Theorem 3.1, it is possible to construct asymptotic prediction intervals for the excess error using the quantiles of the chi-square distribution. See Section 4 for an example of such a simplified value of C, since, in this simulation study, u is trivially proportional to u 2 . We give, in the sequel, examples of strictly stationary MDS of errors satisfying the requirements of Theorem 3.1 (and also the asymptotic optimality (9) and the asymptotic normality (11)-( 12). Both results ( 9) and ( 11)-( 12) are proved in BGL but the class of examples below has not been considered there. More precisely, BGL only studies standard ARCH(1) sequences. Let us recall that an ARCH(1) sequence satisfies the Conditions (A) and (B) provided its so-called "persistence" parameter stays lower than 2025027 -1/8 ≈ 0.162796). The examples we consider here belong to the class of stochastic volatility processes. Recall that a Stochastic Volatility process ( i ) i∈IN , SV in short, is defined as

i = σ i Z i , i ∈ IN, ( 13
)
where the volatility sequence (σ i ) i∈IN is a strictly stationary sequence of positive random variables which is independent of the i.i.d. centered noise sequence (Z i ) i∈IN . We refer, for instance, to Davis and Mikosh (2009) for the main properties of SV models.

The following proposition gives conditions under which the requirements of Theorem 3.1 are satisfied for SV error processes.

Proposition 3.1. Let ( i ) i∈IN be as defined in (13). Suppose that, there exists a positive decreasing function Φ defined on

IR + satisfying ∞ s=1 s 4 Φ(s) < ∞,
and for any positive integer q ≤ 6, 1

≤ i 1 ≤ • • • ≤ i k < i k+1 ≤ • • • ≤ i q ≤ n such that i k+1 -i k ≥ max 1≤l≤q-1 (i l+1 -i l ), |Cov(σ i 1 • • • σ i k , σ i k+1 • • • σ iq )| ≤ Φ(i k+1 -i k ), and moreover |IE(Z i 1 • • • Z iq )| < ∞. ( 14
)
If IE( 2p 1 ) < ∞ for some p > 8 then the asymptotic optimality (9), the asymptotic normality (11)-( 12) and the conclusions of Theorem 3.1 hold.

The following corollaries give more explicit examples of SV models satisfying the assumptions of Proposition 3.1, so that Theorem 3.1 holds true in these cases.

Log-normal volatility sequences

The log-normal SV models are due to [START_REF] Taylor | Modelling Financial Time Series[END_REF]. For these models, the volatility sequence (σ i ) i∈IN is an exponential weight of a Gaussian moving average. They are a basic alternative to ARCH-type processes, since unlike ARCH-type models, their variances always remain positive without the need of further conditions. Corollary 3.1 below proves in particular that for log-normal SV models the volatility sequence (σ i ) i∈IN is a strictly stationary strong mixing sequence in the sense of [START_REF] Rosenblatt | A central limit theorem and a strong mixing condition[END_REF]. Recall that (σ i ) i∈IN is a strongly mixing sequence if its strong mixing coefficient (α s ) s≥0 defined by

α s = sup k∈IN α(σ(σ i , i ≤ k), σ(σ i , i ≥ k + s)),
tends to 0 as s tends to infinity, where for two sigma-fields A and B,

α(A, B) = sup A∈A,B∈B |Cov(1I A , 1I B )|.
Corollary 3.1. Suppose that the volatility sequence (σ i ) i∈IN is defined for i ∈ IN, by σ i = β exp ∞ j=0 γ j η i-j with |γ| < 1, β > 0 and (η i ) i∈Z Z is an i.i.d. centered sequence distributed as a Gaussian law with finite variance. Suppose also that Z 1 follows a standard Gaussian law. Then the process ( i ) i∈IN , as defined in (13), is a strictly stationary MDS, with finite all integer moments, strongly mixing with α s = O(|γ| 2 3 s ), and the asymptotic optimality (9), the asymptotic normality ( 11)-( 12) together with the conclusions of Theorem 3.1 hold.

We illustrate the conclusions of Corollary 3.1 in Section 4.

Gamma volatility sequences

We consider here the Gamma stochastic model as defined by (13), with (Z i ) i∈IN being a sequence of i.i.d. standard normal random variables and for i ∈ IN, σ i = √ h i where (h i ) i∈IN is a positive time-homogeneous strictly stationary Markov chain. We suppose that the marginal distribution of (h i ) i∈IN is a Gamma Γ(p, λ) distribution, i.e., denoting by π the invariant measure of this Markov chain,

π(dx) = f (x)dx, f (x) = λ p Γ(p) x p-1 e -λx 1I x≥0 , p, λ > 0,
where Γ(p) = ∞ 0 u p-1 e -u du. Suppose that this Markov chain is geometrically ergodic. This means that there exists a positive constant c and a Borel positive function a(•) such that the following inequality holds for any π-almost everywhere x ∈ IR: for any n ∈ IN \ {0}, and Borel set B

|P n (x, B) -π(B)| ≤ a(x)e -cn , (15)
recall that the transition probability P is defined, for suitable set A and x, by

P (x, A) = IP (h 1 ∈ A|h 0 = x) and for n ∈ IN \ {0} P n (x, A) = IP (h n ∈ A|h 0 = x) .
In this case, it is well-known that the Markov chain (h i ) i∈IN is β-mixing with geometrically decaying mixing coefficients (β n ) n≥1 (cf. for instance Theorem 3.7 in [START_REF] Bradley | Basic Properties of Strong Mixing Conditions. A Survey and Some Open Questions[END_REF] and the references therein). Recall that, for a sequence (X n ) n∈IN , the β-mixing coefficients (β n ) n≥1 are defined by (see for instance [START_REF] Doukhan | Mixing: Properties and Examples[END_REF] (Sec 1.1))

β n = sup m∈IN IE sup B∈σ(X i , i≥m+n) |IP(B|σ(X 0 , • • • , X m )) -IP(B)| .
The Gamma AR(1) process, stated in the example below, is a Markov chain satisfying all the above assumptions.

Example. Let h 0 be distributed as Γ(p, λ) distribution. Define, for ρ ∈]0, 1[, h n recursively by,

h n = ρh n-1 + ξ n , where (ξ n ) n is an i.i.d. sequence of random variables with characteristic function IE e -itξ 1 = λ-it λ-itρ -p
. The process (h n ) n≥1 is then a strictly stationary Markov chain with Gamma Γ(p, λ) univariate marginal distribution (see for instance [START_REF] Gaver | First-Order Autoregressive Gamma Sequences and Point Processes[END_REF]). This Markov chain is also geometrically ergodic in the sense of (15) (see for instance [START_REF] Kesten | Renewal Theory for Functionals of a Markov Chain with General State Space[END_REF]).

The following corollary gives conditions under which the Gamma stochastic volatility models satisfy the requirements of Proposition 3.1 and Theorem 3.1.

Corollary 3.2. Suppose that the volatility sequence (σ i ) i∈IN is defined for i ∈ IN, by σ i = √ h i where (h i ) i∈IN is a positive time-homogeneous strictly stationary and geometrically ergodic Markov chain with marginal Gamma Γ(p, λ) distribution. Let (Z i ) i∈IN be a sequence of i.i.d. standard normal random variables. Then the process ( i ) i∈IN , as defined by ( 13), is a strictly stationary MDS, β-mixing with β n = O(e -ρn ), for some ρ > 0, and with finite all integer moments. The asymptotic optimality (9), the asymptotic normality ( 11)-( 12) and the conclusions of Theorem 3.1 hold.

Log-linear volatility sequences with Bernoulli innovations

The following corollary studies another class of SV models, ( i ) i∈IN as introduced in (13), for which the conclusions of Theorem 3.1 still hold. In these models, we suppose that (log(σ i )) i∈IN is a linear process with Bernoulli innovation having coefficients (2 -k ) k∈IN . Unlike the log-normal SV, in this case the volatility sequence (σ i ) i∈IN is not strongly mixing (see [START_REF] Bradley | Basic properties of strong mixing conditions[END_REF]) but it is associated in the sense of [START_REF] Esary | Association of random variables with applications[END_REF]. Recall that a sequence (σ i ) i∈IN is said to be associated if for any non-decreasing and bounded functions f and g,

Cov(f (σ 1 , • • • , σ n ), g(σ 1 , • • • , σ n )) ≥ 0. ( 16
)
Corollary 3.3. Suppose that the volatility sequence (σ i ) i∈IN is defined for i ∈ IN, by σ i = exp ∞ j=0 2 -j η i-j where (η i ) i∈Z Z is an i.i.d. centered sequence distributed as a Bernoulli B(1/2) distribution. Suppose also that Z 1 follows a standard normal law. Then the volatility sequence (σ i ) i∈IN is associated and the asymptotic optimality (9), the asymptotic normality (11)-( 12) together with the conclusions of Theorem 3.1 hold.

A Monte-carlo simulation study for a "trend plus a log-normal SV process"

The purpose of this section is to illustrate by simulations the asymptotic optimality (a.o, in short) as stated in (9), and to check whether the asymptotic normality ( 11)-( 12) and the asymptotic scaled X 2 (1) distribution of the ASE-excess, as stated in Theorem 3.1, are realistic (and thus useful) descriptions. Our experiments are similar to those in BGL, the main difference being that the noise sequence is now a common log-normal SV sequence, as analysed in Corollary 3.1, instead of an ARCH(1). Precisely, we consider the smooth function r(x) = (4x(1 -x)) 3 as "deterministic trend", an equispaced design and a noise level σ for which the noise-to-signal ratio is "moderate" (see Figure 1 for n = 512) and finally K is taken as the well-known biweight kernel. Notice that this trend offers a simple solution to the boundary effect issue: since it is "smoothly periodic", it allows us to consider a circular design, that is, as is explained in Härdle, [START_REF] Härdle | How far are automatically chosen regression smoothing parameters from their optimum[END_REF], the estimation of r near i = 1 is done by setting, for i ≤ 0, y i := y n-i and similarly at the other end (such a "periodic padding" is used by many other authors). We are then allowed to take u(•) equals to the characteristic function on [0, 1] as a weight function. Moreover, an important appealing consequence of this periodicity assumption, it that the main computation for any criterion discussed here can be reduced to fast Fourier transforms and this makes very affordable extensive large-scale simulations. See BGL for more details. To guard against possible local minimisers, the numerical minimisation of the ASE (T n (h)) or of the Mallows criterion CL(h) consists of a global search over a dense enough grid (precisely, of size 401) of equispaced (on log-scale) values of h. We used Mathematica for these simulations; in fact the code is very similar to the one published in the demo [START_REF] Girard | Nonparametric Curve Estimation by Kernel Smoothers: Efficiency of Unbiased Risk Estimate and GCV Selector[END_REF] except for the noise generation, of course, since this demo is restricted to i.i.d. errors. As parameters, in order to define the noise process, in addition to σ and γ (the serial correlation introduced in Corollary 3.1), we introduce τ := Var(η 1 )/(1 -γ 2 ).

(17)

Let us remark that, to generate a noise process with variance σ 2 , it is easy to check that the parameter β (used in Corollary 3.1) has to be set to σ exp (-τ 2 ) . The advantage of using (σ, τ 2 , γ) instead of (β, Var(η 1 ), γ) is deduced from the fact that τ is the unique shape-parameter for the marginal density of the sequence of conditional variances σ 2 i 's. Such a parameterization is common (see e.g. [START_REF] Taylor | Modelling stochastic volatility: a review and comparative study[END_REF]).

We consider three values for τ , and three values for the serial-correlation parameter γ, precisely τ ∈ {0.2, 0.4, 0.75}, γ ∈ {0.01, 0.9, 0.98}, with a common value σ = 0.32. Note that the intermediate value 0.4 for τ is representative of values often obtained by fitting such a log-normal SV model to real econometric series; see [START_REF] Taylor | Modelling stochastic volatility: a review and comparative study[END_REF] (especially its Section 3.4, where τ is denoted by β) for an interesting review. Any large value of τ (say, greater than 1) implies a very fat tail for the marginal density of the amplitude of the noise | i | which may cause a large instability of the classical kernel curveestimate (a "robust" version kernel smoothing would be much more appropriate in such case).

On the other hand, recall that a value very close to 0 for τ would imply that the density of the conditional variance σ 2 i is concentrated around 1 and thus the serial correlation would have virtually no impact on the dependence between the i 's (which is then a "quasi-i.i.d.-normal" sequence). Thus we restrict the present study to the range [0.2, 0.75] for τ .

The a.o. property. As is well-known, a result like (9) generally stems from a uniform relative accuracy result which states that CL(h) -n -1 U 1/2 (Y -r) 2 uniformly approximates T n (h) (or its expectation, say MASE(h)) with a small (in probability and in sup norm over the domain of candidate h's) error, "small" being defined relatively to MASE(h).

We illustrate in Figure 2 that a uniform relative accuracy is well observed for all the considered values of τ and γ. Note that the results for γ = 0.01 and τ ∈ {0.2, 0.4, 0.75} have produced plots very similar to the plot for γ = 0.9, τ = 0.2 (top-left panel in Figure 2) so they are not included in Figure 2. Only a slight deterioration is observed for (τ, γ) = (0.75, 0.98). Notice that this in contrast with the Monte Carlo study for ARCH(1) noise sequences in BGL where a large sample a.o. was no longer observed when the chosen persistence parameter (denoted, there, by α) was not small enough; indeed, even for n = 2 15 it was required that α < 0.75 (precisely, a reasonable "uniform relative accuracy" was not observed for α ∈ {0.75, 0.9, 0.98}; we refer to that paper for a discussion of this). Figure 2 : n = 2 15 . These 6 panels only differ by (τ, γ) varying in {0.2, 0.4, 0.75} × {0.9, 0.98}. In each panel, the dashed blue curve is the "empirical MASE", precisely the average (over the 3000 replicates) of the T n (h) curves. Each of the 21 boxplots (located at 21 fixed discrete values for h) are built from the first 100 replicates of CL(h

○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ 0
) -n -1 U 1/2 (Y -r) 2 .
Asymptotic normal distribution. This Section aims to assess the usefulness of the theoretical asymptotic normal approximation stated in ( 11)-( 12) for reasonable dataset sizes n. We are going to demonstrate that both τ and γ have an impact on the speed of convergence (with respect to n) toward this approximation. Figure 3 : τ = 0.2. . These 6 panels only differ by n (= 2 9 in the top row and 2 15 in the bottom row) and by γ varying in {0.01, 0.9, 0.98}. In each panel, the displayed normalized histogram is that of the 3000 replicates of ĥMĥn . The superposed blue curve is the normal distribution of ĥMĥn as predicted by the asymptotic theory.

Let us first consider τ = 0.2. By inspecting Figure 3, we clearly see, in the three top panels, that the asymptotic approximation fits rather well already for n = 2 9 and for any γ ∈ {0.01, 0.9, 0.98}. For n = 2 15 the three bottom panels illustrate that the asymptotic theory provides a very accurate prediction of the finite sample "truth". Notice that, as expected the accuracy for n = 2 12 (not displayed here) is observed to be intermediary between the one for n = 2 9 and that for n = 2 15 , and is thus also quite good.

Notice that, again as expected, the range of the abscissae (h-differences) increases by moving from n = 2 15 (bottom) to n = 2 9 (top).

It is good news that the approximation given by ( 11) and ( 12) is very useful for n as small as 512. Figure 4 : τ = 0.4. These 9 panels only differ by n (= 2 9 in the top row, 2 12 (middle) and 2 15 in the bottom row) and by γ varying in {001, 0.9, 0.98}. In each panel, the displayed normalized histogram is that of the 3000 replicates of ĥMĥn . The superposed blue curve is the normal distribution of ĥMĥn as predicted by the asymptotic theory.

The simulation results for τ = 0.4 are described in Figure 4. Here we add the three panels corresponding to n = 2 12 . The analog figure for τ = 0.75 is Figure 5. Now one clearly sees that, for γ = 0.01 (first column in these two 3 × 3 arrays of histograms) the smallest value of n (= 2 9 ) is always sufficient for the usefulness of the asymptotic normal approximation -although there is a slight deterioration for τ = 0.75 (precisely the histogram in the topleft panel in Figure 5 exhibits a non-negligible proportion of "too large" negative values for ĥMĥn which almost always are associated with too-small ĥM 's). One observes that the latter deterioration is softened if n is increased to 2 12 (middle panel of first column of Figure 5). Next, an inspection of the second column (thus γ = 0.9) of both these two arrays shows that n = 2 9 is "just sufficient" only for the smaller τ = 0.4 and provided one accepts a slight inaccuracy of the same type as the one mentioned above. But n = 2 12 is clearly required for τ = 0.75. Notice that, because of the observed jump in the observed accuracy by going from (τ, γ) = (0.75, 0.01) to (0.75, 0.9), we also repeated the same simulations for the case (τ, γ) = (0.75, 0.5) : they produced a histogram rather close to the one for (0.75, 0.01); this demonstrates that it is only for "large" γ (that is, near 0.9 or above) that the asymptotic approximation is not accurate for n = 2 9 .

Next, the third columns (that is, for γ = 0.98) shows that n = 2 12 is required for τ = 0.4, and n = 2 15 is required for τ = 0.75. (middle) and n = 2 15 (bottom). These 9 panels only differ by n and by γ varying in {001, 0.9, 0.98}. In each panel, the displayed normalized histogram is that of the 3000 replicates of ĥMĥn . The superposed blue curve is the normal distribution of ĥMĥn as predicted by the asymptotic theory.

All these experiments are thus well in agreement with ( 11)-( 12). But for certain settings, which are not un-common in practice (see [START_REF] Taylor | Modelling stochastic volatility: a review and comparative study[END_REF]), this normal approximation is accurate only for quite large n (for example, n larger than 2 12 is required for (τ, γ) = (0.4, 0.98)). And this so-required value for n is shown to be an increasing function of both τ and γ.

Asymptotic scaled X 2 (1) distribution of the ASE-excess. Now let us look at the usefulness of the asymptotic approximation stated in Theorem 3.1 for the excess of ASE. The constants in this approximation can be quite simplified in the context of this Monte Carlo study. Indeed, because we have taken u ≡ 1, by a simple algebra and using the fact that for the bi-weight kernel we have (K -G) 2 = G 2 = K 2 (see the proof of Lemma 3.1 in [START_REF] Girard | Estimating the accuracy of (local) cross-validation via randomised GCV choices in kernel or smoothing spline regression[END_REF] for the first equality, and Table 1 there, for the second), we can check that the constant C in Theorem 3.1 has the simplified expression C = (6/5)σ 2 . As a side remark, the fact that the asymptotic approximation is thus not a function of r when u ≡ 1, is quite appealing in practice: the distribution of the possible excesses of ASE can then be predicted for n "large enough", requiring only the value of σ (recall that this distribution could also be estimated in much more general contexts via the simulation of randomized choices; cf [START_REF] Girard | Estimating the accuracy of (local) cross-validation via randomised GCV choices in kernel or smoothing spline regression[END_REF] for the large-n theoretical justification of this approach in the i.i.d. case). A natural question for a practitioner is thus the meaning of "large enough" in the above discussion. We summarize in Figure 6 the simulation results for 4 settings: the "worst" setting, precisely n = 512 and (τ, γ) = (0.75, 0.98) in panel A, and three other settings, where "worst" means that the goodness of the approximation of the observed histogram by the asymptotic scaled chi-square law of Theorem 3.1 is the worst in the 9 configurations of (τ, γ)values. This goodness can be of course improved (in agreement with the theoretical result) in distribution, as n tends to infinity. We get (using the same arguments as for the proof of Lemma 2.1 in BGL), for any h sufficiently small and for any n sufficiently large,

IE(T n (h)) = 3h 2 1 0 u(x)r 2 (x)dx 1 -1 t 2 K(t)dt 2 + 2σ 2 nh 3 ( 1 0 u(x)dx) 1 -1 K 2 (y)dy +o(h 2 ) + O( 1 n 2 h 4 ) + o( 1 nh 3 ).
We deduce (recall that h * n = cn -1/5 ),

n 2/5 IE(T n (h * n )) = 3c 2 1 0 u(x)r 2 (x)dx 1 -1 t 2 K(t)dt 2 + 2σ 2 c 3 ( 1 0 u(x)dx) 1 -1 K 2 (y)dy + o(1). Hence, lim n→∞ n 2/5 IE(T n (h * n )) = 5σ 4/5 B 2/5 A 3/5 , and lim n→∞ Σ 2 2 n 2/5 IE(T n (h * n )) = C, (21)
with

C = 2σ 2 5A t 2 K(t)dt 2 1 0 u 2 (x)r 2 (x)dx + 2A B 1 0 u 2 (x)dx (K -G) 2 (t)dt .
We obtain collecting ( 19), ( 20) and ( 21), n 2 ( ĥĥn )

2 IE(T n (h * n )) =⇒ CX 2 (1), (22)
in distribution, as n tends to infinity.

Control of

n 2 ( ĥ -ĥn ) 2 T n (h * ) -IE(T n (h * n )) . Clearly, n 2 ( ĥ -ĥn ) 2 T n (h * ) -IE(T n (h * n )) = n 3/10 ( ĥ -ĥn ) 2 n 2/5 2 T n (h * ) -IE(T n (h * n )) . (23)
We have,

n 2/5 T n (h * ) -IE(T n (h * n )) (24) ≤ n 2/5 T n (h * ) D n (h * ) -1 D n (h * ) + n 2/5 D n (h * ) -IE(T n (h * n )) , and
n 2/5 T n (h * ) D n (h * ) -1 D n (h * ) ≤ sup h∈Hn T n (h) D n (h) -1 n 2/5 sup h∈Hn D n (h) . (25)
We need the following lemma proved in BGL, Lemma 5.1. It holds, under the previous notations,

lim sup n→∞ n 2/5 sup h∈Hn IE(T n (h)) < ∞, lim sup n→∞ n 2/5 sup h∈Hn D n (h) < ∞ lim n→∞ sup h∈Hn D n (h) IE(T n (h)) -1 = 0 lim n→∞ sup h∈Hn T n (h) D n (h) -1 = 0, in probability.
We deduce, using Lemma 5.1 and (25),

lim n→∞ n 2/5 T n (h * ) D n (h * ) -1 D n (h * ) = 0, in probability. ( 26
)
Now,

n 2/5 D n (h * ) -IE(T n (h * n )) ≤ n 2/5 D n (h * ) -D n (h * n ) + n 2/5 IE(T n (h * n )) D n (h * n ) IE(T n (h * n )) -1 ≤ cst h * h * n -1 + n 2/5 sup h∈Hn IE(T n (h)) sup h∈Hn D n (h) IE(T n (h)) -1 ≤ cst ĥ h * n -1 + cst ĥG ĥM -1 + n 2/5 sup h∈Hn IE(T n (h)) sup h∈Hn D n (h) IE(T n (h)) -1 . ( 27
)
The asymptotic optimality property (9) allows to deduce that ĥ We deduce, collecting ( 24), ( 26) and ( 28 The proof of Theorem 3.1 is complete by collecting ( 18), ( 22) and (30).

Proof of Proposition 3.1

The sequence ( i ) i∈IN , as defined in (13), is strictly stationary. This property follows from the strictly stationary property of the sequence (σ i ) i∈IN . Define, now, the sequence of filtration (F i ) i≥0 by, for i ∈ IN, F i = σ(Z 0 , • • • , Z i , σ i , σ i+1 ). Then i is F i -measurable. Since σ i is F i-1 -measurable, Z i is independent of F i-1 and IE(Z i ) = 0, we have

IE ( i |F i-1 ) = σ i IE (Z i |F i-1 ) = 0,
almost surely. This sequence ( i ) i∈IN is then a strictly stationary MDS. We have, by the requirements of Proposition 3.1, that IE( 2p 1 ) < ∞ for some p > 8. Conditions (A) are then satisfied. Our task now is to check Condition (B), i.e., the inequality in (10). We have, by the definition of SV model, (denoting by P the distribution of Z 1 ), for i

1 ≤ • • • ≤ i k < i k+1 ≤ • • • ≤ i q , Cov( i 1 • • • i k , i k+1 • • • iq ) = Cov(σ i 1 Z i 1 • • • σ i k Z i k , σ i k+1 Z i k+1 • • • σ iq Z iq ) = z i 1 • • • z iq Cov(σ i 1 • • • σ i k , σ i k+1 • • • σ iq )dP (z i 1 ) • • • dP (z iq ) = IE(Z i 1 • • • Z iq )Cov(σ i 1 • • • σ i k , σ i k+1 • • • σ iq ).
Consequently,

Cov( i 1 • • • i k , i k+1 • • • iq ) = |IE(Z i 1 • • • Z iq )||Cov(σ i 1 • • • σ i k , σ i k+1 • • • σ iq )| ≤ cst |Cov(σ i 1 • • • σ i k , σ i k+1 • • • σ iq )|. ( 31
)
Condition ( 10) is then satisfied from the last inequality together with Condition (14). Condition (B) holds. The proof of Proposition 3.1 is complete since all the requirements of Theorem 3.1 are satisfied (the asymptotic optimality (9), the asymptotic normality (11)-( 12) are already proven in BGL.)

Proof of Corollary 3.1

All the higher moments of σ 1 and 1 are finite, we refer for instance to Cox, Hinkley and Barndorff-Nielsen (1996) (page 22). We have now to check that ( i ) i∈IN is strongly mixing and that Condition ( 14) is satisfied. Since the density of η 0 is in IL 1 , the linear process ∞ j=0 γ j η i-j i is strongly mixing (see [START_REF] Pham | Some mixing properties of time series models[END_REF]) with,

α s ≤ K j≥s k≥j |γ| k 2/3 = O(|γ| 2 3 s ),
for some constant K. Similarly, the sequence (σ i ) i≥0 is still strongly mixing with the same mixing coefficients (α s ) s . We deduce, from (31), that ( i ) i≥0 is also strongly mixing with

For 1 ≤ i 1 ≤ • • • ≤ i k < i k+1 ≤ • • • ≤ i q ≤ n
such that, letting s = i k+1 -i k , s ≥ max 1≤l≤q-1 (i l+1 -i l ), we have using [START_REF] Birkel | The invariance principle for associated processes[END_REF], and the above bound,

|Cov(σ i 1 • • • σ i k , σ i k+1 • • • σ iq )| ≤ e 2(q-2) i k i=i 1 iq l=i k+1
Cov(σ i , σ l ) ≤ cst 2 -s .

Condition ( 14) is then satisfied, since ∞ s=1 s 4 2 -s < ∞. Now, we have since σ 1 is bounded and Z 1 is normally distributed IE( 2p 1 ) < ∞ for any p ∈ IN. All the requirements of Proposition 3.1 are satisfied and therefore the asymptotic optimality (9), the asymptotic normality ( 11)-( 12) and the conclusions of Theorem 3.1 hold.

Figure 1 :

 1 Figure1: n = 2 9 . Each of these 2 panels displays one data set Y and the "smooth" deterministic trend r(•) when the noise is a log-normal SV sequence. In the 2 panels τ = 0.75, and they only differ by γ = 0.01 (left) and γ = 0.98 (right); see(17) for the definition of these two parameters.

Figure 5 :

 5 Figure5: τ = 0.75. n = 2 9 (top), n = 2 12 (middle) and n = 2 15 (bottom). These 9 panels only differ by n and by γ varying in {001, 0.9, 0.98}. In each panel, the displayed normalized histogram is that of the 3000 replicates of ĥMĥn . The superposed blue curve is the normal distribution of ĥMĥn as predicted by the asymptotic theory.

Figure 6 :

 6 Figure 6 : Assessment of the simplified 6 5n σ 2 X 2 (1) approximation of the ASE-excess. In each panel, the displayed normalized histogram is that of the 3000 replicates of T n ( ĥM ) -T n ( ĥn ). The superposed blue curve is the 6 5n σ 2 X 2 (1) density as suggested by the asymptotic theory. (A): n = 2 9 and (τ, γ) = (0.75, 0.98). (B): n = 2 12 and (τ, γ) = (0.75, 0.98). (C): n = 2 9 and (τ, γ) = (0.75, 0.9). (D): n = 2 9 and (τ, γ) = (0.4, 0.98).

  , in probability, to 0 as n tends to infinity. Consequently, the bound (27) together with Lemma 5.1, givelim n→∞ n 2/5 D n (h * ) -IE(T n (h * n )) = 0, in probability. (28)

  5 T n (h * ) -IE(T n (h * n )) = 0, in probability. ĥn ) 2 T n (h * ) -IE(T n (h * n )) = 0, in probability.(30)

Acknowledgements

We thank the reviewer for his/her accurate and constructive comments. This led to a significant improvement of the original manuscript. This paper was developed in the framework of Grenoble Alpes Data Institute (ANR-15-IDEX-02).

when n is increased: Panel B, compared to A, demonstrates that n = 4096 is sufficient for (τ, γ) = (0.75, 0.98), while n = 512 is not. But we also observed two facts: 1) such a goodness is also restored by decreasing γ. Panel C demonstrates that γ = 0.9 is sufficient even for n = 512; and we always observed this behavior for lower γ. 2) the shape parameter τ has the same impact. Indeed Panel D, where, again, γ = 0.98, demonstrates that for τ = 0.4 the asymptotic scaled X 2 (1) approximation is quite good even for n = 512. For the configurations (τ, γ) ∈ {(0.75, 0.01), (0.4, 0.01), (0.4, 0.9), (0.2, 0.01), (0.2, 0.9), (0.2, 0.98)}, the results are not displayed, for sake of place, because the figures would be quasi-identical to those in either Panel B, C or D.

By comparing with the experimental results for the normal approximation of the differences of bandwidths, this latter configuration (τ, γ) = (0.4, 0.98) is one of many where the scaled X 2 (1) approximation for the excess of ASE does not require very large n, while it was not the case for the bandwidths' differences.

Proofs

In all the proofs, we denote by cst a positive constant that may be different from line to line.

Proof of Theorem 3.1

Let ĥ be either ĥM or ĥG . We know, using T n ( ĥn ) = 0, that there exists h * between ĥ and ĥn for which,

where h * n is as defined in (5) and is the minimizer of D n (h) (defined in (4)). Let us control the two terms of the right hand side of (18).

Control of n

2 ( ĥĥn

where Σ is as defined in (12). We know from ( 11) and ( 12) that Σ -1 n 3/10 ( ĥĥn ) =⇒ N (0, 1), in distribution, as n tends to infinity and then Σ -1 n 3/10 ( ĥĥn )

such that, defining j = i k+1 -i k , j ≥ max 1≤l≤q-1 (i l+1 -i l ), and for s, l, r strictly positive reals for which 1/s + 1/l + 1/r = 1,

(see [START_REF] Davydov | Convergence of distributions generated by stationary stochastic processes[END_REF]). Condition ( 14) is then satisfied since j≥1 j 4 |γ| 2j 3r < ∞ (recall that in this model all the moments of σ 1 are finite). Therefore the requirements of Proposition 3.1 are satisfied so that the asymptotic optimality (9), the asymptotic normality ( 11)-( 12) and the conclusions of Theorem 3.1 hold.

Proof of Corollary 3.2

This sequence ( n ) n≥1 has finite moments at any order. In particular, for r ∈ IN,

(see for instance [START_REF] Abraham | Gamma stochastic volatility models[END_REF] and the references therein). Now the sequence (σ i ) i is also β-mixing (since, by definition, it's the square root of a β-mixing positive sequence (h i ) i≥1 ) so it is also strongly mixing with

for some positive real numbers a 1 and ρ. This bound together with (32) prove that, for 1

for some fixe r > 1. Clearly,

Condition ( 14) is then satisfied from ( 33) and (34). As in the proof of Corollary 3.1, we deduce that all the requirements of Proposition 3.1 are satisfied. Therefore the asymptotic optimality (9), the asymptotic normality ( 11)-( 12) and the conclusions of Theorem 3.1 hold.

Proof of Corollary 3.3

The sequence (σ i ) i∈IN is associated since it is a non-decreasing function of independent random variables (see [START_REF] Esary | Association of random variables with applications[END_REF]). The random variable σ i is bounded, |σ i | ≤ e 2 and, by (see [START_REF] Birkel | The invariance principle for associated processes[END_REF]),

2 -k Cov(η i-j , η l-k ) ≤ cst 2 -|i-l| .