Bernstein Polynomials

RECURSIVE AND NON-RECURSIVE REGRESSION ESTIMATORS USING

Keywords: 2010 Mathematics Subject Classification. 62G08, 62L20. Key words and phrases. Nonparametric regression, Stochastic approximation algorithm, Bernstein polynomial

. In addition, through some simulation studies, we show that our non-recursive estimator has the lowest integrated root mean square error (ISE) in most of the considered cases. Finally, using a set of real data, we demonstrate how our non-recursive and recursive regression estimators can lead to very satisfactory estimates, especially near the boundaries.

Introduction

The goal in any data analysis is to extract from raw information the accurate estimation. One of the most important and common questions concerning if there is a statistical relationship between a response variable (Y ) and an explanatory variable (X i ).

An option to answer this question is to employ regression analysis in order to model this relationship.

Let (X, Y ) , (X 1 , Y 1 ) , . . . , (X n , Y n ) be independent, identically distributed pairs of random variables with joint density function g (x, y), and let f denote the probability density of X. There were many ways to estimate the regression function r :

x → E[Y |X = x].
The most known are the kernel regression estimators. On the non-recursive approach, we refer, among many others, to the estimator proposed by Nadaraya [START_REF] Nadaraya | On Estimating Regression[END_REF] and Watson [START_REF] Watson | Smooth regression analysis[END_REF], the alternative kernel estimators given by Priestley and Chao [START_REF] Priestley | Non-parametric function fitting[END_REF] and Gasser and Müller [START_REF] Gasser | Kernel estimation of regression functions, Smoothing Techniques for Curve Estimation[END_REF]. On the other hand, the recursive estimation was widely discussed, we refer to the approach of Révész [START_REF] Révész | Robbins-Monro procedure in a Hilbert space and its application in the theory of learning processes I[END_REF][START_REF] Révész | How to apply the method of stochastic approximation in the non-parametric estimation of a regression function[END_REF] and Tsybakov [START_REF] Tsybakov | Recurrent estimation of the mode of a multidimensional distribution[END_REF] which was studied by Mokkadem et al. [START_REF] Mokkadem | Revisiting Révész's stochastic approximation method for the estimation of a regression function[END_REF], Slaoui [START_REF] Slaoui | Moderate deviation principles for recursive regression estimators defined by stochastic approximation method[END_REF][START_REF] Slaoui | Large and moderate deviation principles for recursive regression estimators defined by stochastic approximation method[END_REF][START_REF] Slaoui | Plug-in bandwidth selector for recursive kernel regression estimators defined by stochastic approximation method[END_REF], also we find the semi-recursive approach introduced by Slaoui [START_REF] Slaoui | Optimal bandwidth selection for semi-recursive kernel regression estimators[END_REF]. The advantage of the recursive estimator is the update of the estimation, which is computationally cost-effective when new data appear in the sample, this advantage given an additional motivation of the present work. Each of the last estimators has its own particular strengths and weaknesses. However, the common problem is the edge effect. In fact, when the regression function has bounded support, kernel estimates often overspill the boundaries and are consequently biased at and near these edges. To overcome this problem, many works are devoted to reducing the effects, we can list Gasser and Müller [START_REF] Gasser | Kernel estimation of regression functions, Smoothing Techniques for Curve Estimation[END_REF], Gasser et al. [START_REF] Gasser | Kernels for nonparametric curve estimation[END_REF], Granovsky and Müller [START_REF] Granovsky | Optimizing kernel methods: a unifying variational principle[END_REF] and Müller [START_REF] Müller | Smooth optimal kernel estimators near endpoints[END_REF] discuss boundary kernel methods. Djojosugito and Speckman [START_REF] Djojosugito | Boundary bias correction in nonparametric density estimation[END_REF] investigated boundary bias reduction based on a finite-dimensional projection in a Hilbert space. In this work, we propose a non-recursive and recursive approach of regression estimation using Bernstein polynomials.

The estimation using Bernstein polynomial for density and distribution functions have been widely discussed in several frameworks. See, for instance, the original work of Vitale [START_REF] Vitale | A Bernstein polynomial approach to density function estimation[END_REF] and extensions given by Tenbusch [START_REF] Tenbusch | Two-dimensional Bernstein polynomial density estimation[END_REF], Ghosal [START_REF]Convergence rates for density estimation with Bernstein polynomials[END_REF], Kakizawa [START_REF] Kakizawa | Bernstein polynomial probability density estimation[END_REF][START_REF] Kakizawa | A note on generalized Bernstein polynomial density estimators[END_REF], Igarashi and Kakizawa [START_REF] Igarashi | On improving convergence rate of Bernstein polynomial density estimator[END_REF], Rao [START_REF] Rao | Estimation of distribution and density functions by generalized Bernstein polynomials[END_REF], Leblanc [START_REF] Leblanc | A bias-reduced approach to density estimation using Bernstein polynomials[END_REF][START_REF] Leblanc | On estimating distribution function using Bernstein polynomials[END_REF][START_REF] Leblanc | On the boundary properties of Bernstein polynomial estimators of density and distribution functions[END_REF], Babu et al. [START_REF] Babu | Application of Bernstein polynomials for smooth estimation of a distribution and density function[END_REF], Babu and Chaubey [START_REF] Babu | Smooth estimation of a distribution function and density function on a hypercube using Bernstein polynomials for dependent random vectors[END_REF], Jmaei et al. [START_REF] Jmaei | Recursive distribution estimator defined by stochastic approximation method using Bernstein polynomials[END_REF] and more recently Slaoui and Jmaei [START_REF] Slaoui | Recursive density estimators based on Robbins-Monro's scheme and using Bernstein polynomials[END_REF].

The layout of the present paper is as follows. In Section 2, we list our assumptions and notations. In Section 3, we introduce our non-recursive estimator and we compute its bias, variance, mean squared error (M SE), the mean integrated squared error (M ISE) and we establish a weak convergence rate. In Section 4 we introduce our recursive estimator and we state the main theoretical results. Section 5 is devoted to some numerical studies : first, a simulation study is presented in Subsection 5.1 and, then, an application to a real dataset is described in Subsection 5.2. Finally, we discuss our conclusion in Section 6. To avoid interrupting the flow of this paper, all mathematical developments are relegated to the Appendix.

Assumptions and Notations

Let us first define the class of positive sequences that will be used in the statement of our assumptions. Definition 2.1. Let γ ∈ R and (v n ) n≥1 be a nonrandom positive sequence. We say that

(v n ) ∈ GS(γ) if lim n→+∞ n 1 - v n-1 v n = γ.
This condition was introduced by Galambos and Seneta [START_REF] Galambos | Regularly varying sequences[END_REF] to define regularly varying sequences (see also Bojanic and Seneta [START_REF] Bojanic | A unified theory of regularly varying sequences[END_REF]). Typical sequences in GS(γ) are, for b ∈ R, n γ (log n) b , n γ (log log n) b , and so on.

To obtain the behavior of our estimators, we make to the following assumptions : : (A1) (γ n ) ∈ GS (-α), α ∈ 3 4 , 1 . : (A2) (m n ) ∈ GS(a), a ∈ 1-α 4 , 2 3 α . : (A3) (i) g (s, t) is twice continuously differentiable with respect to s.

(ii) For q ∈ {0, 1, 2}, s → R t q g (s, t) dt is a bounded function continuous at s = x. For q ∈ [2, 3], s → R |t| q g (s, t) dt is a bounded function.

(iii) For q ∈ {0, 1}, R |t| q ∂g ∂x (x, t) dt < ∞, and s → R t q ∂ 2 g ∂s 2 (s, t) dt is a bounded function continuous at s = x. Assumption (A1) on the stepsize was used in the recursive framework for the estimation of the density function (see Mokkadem et al. [20] and [START_REF] Slaoui | Large and moderate principles for recursive kernel density estimators defined by stochastic approximation method[END_REF][START_REF] Slaoui | Bandwidth selection for recursive kernel density estimators defined by stochastic approximation method[END_REF]), for the estimation of the distribution function (see Slaoui [START_REF] Slaoui | The stochastic approximation method for the estimation of a distribution function[END_REF]) and for the estimation of the regression function (see Mokkadem et al. [START_REF] Mokkadem | Revisiting Révész's stochastic approximation method for the estimation of a regression function[END_REF] and [START_REF] Slaoui | Plug-in bandwidth selector for recursive kernel regression estimators defined by stochastic approximation method[END_REF][START_REF] Slaoui | Optimal bandwidth selection for semi-recursive kernel regression estimators[END_REF]). This assumption ensures that n≥1 γ n = ∞ and n≥1 γ 2 n < ∞, which are two classical assumptions for obtaining the convergence of Robbins-Monro's algorithm (see [START_REF] Duflo | Random Iterative Models[END_REF]).

Assumption (A2) on (m n ) was introduced similarly to the assumption on the bandwidth used for the recursive kernel regression estimator (see Mokkadem et al. [20,[START_REF] Mokkadem | Revisiting Révész's stochastic approximation method for the estimation of a regression function[END_REF]), to ensure the application of the technical lemma given in the appendix A.

Assumption (A3) on the density of the couple (X, Y ) was used in the nonrecursive framework for the estimation of the regression function (see Nadaraya [START_REF] Nadaraya | On Estimating Regression[END_REF] and Watson [START_REF] Watson | Smooth regression analysis[END_REF]) and in the recursive framework (see Mokkadem et al. [21] and Slaoui [START_REF] Slaoui | Moderate deviation principles for recursive regression estimators defined by stochastic approximation method[END_REF][START_REF] Slaoui | Large and moderate deviation principles for recursive regression estimators defined by stochastic approximation method[END_REF][START_REF] Slaoui | Plug-in bandwidth selector for recursive kernel regression estimators defined by stochastic approximation method[END_REF]).

Remark 2.1. The intuition behind the use of such order (m n ) belonging to GS (a) is that the ratio m n-1 /m n is equal to 1 -a/n + o (1/n), then using such order and using the assumption on the stepsize, which ensures that γ n-1 /γ n is equal to 1 + α/n + o (1/n). The application of the technical lemma given in the appendix A ensures that the bias and the variance will depend only on m n and γ n and not on m 1 , . . . , m n and γ 1 , . . . , γ n , then the M ISE will depend also only on m n and γ n , which will be helpful to deduce an optimal order and an optimal stepsize. Throughout this paper we will use the following notations :

∆ 1 (x) = 1 2 (1 -2x)f ′ (x) + x(1 -x)f (2) (x) , ψ(x) = (4πx(1 -x)) -1/2 , ξ = lim n→∞ (nγ n ) -1 , N (x) = r (x) f (x) , ∆ 2 (x) = 1 2 (1 -2x)N (x) + x(1 -x)N ′ (x) , ∆(x) = 1 2 x(1 -x)r (2) (x) + (1 -2x) + 2x(1 -x) f ′ (x) f (x) r ′ (x) , C 1 = 1 0 ∆ 2 (x)dx, C 2 = 1 0 V ar[Y |X = x] f (x) ψ(x)dx, K 1 = 1 0 ∆(x)f (x) f (x) -aξ 2 dx, K 2 = 1 0 2f (x)ψ(x)Var [Y |X = x] 4f (x) -(2α -a)ξ dx.
Moreover, we denote by o x the pointwise bound in x (i.e., the error is not uniform in x ∈ [0, 1]).

Estimators based on the Bernstein polynomials

Let (X, Y ), (X 1 , Y 1 ), . . . , (X n , Y n ) be independent, identically distributed pairs of random variables with joint density function g(x, y), and let f denote the probability density of X which is supported on [0, 1]. We follow the approach of Vitale [START_REF] Vitale | A Bernstein polynomial approach to density function estimation[END_REF] and Leblanc [START_REF] Leblanc | A bias-reduced approach to density estimation using Bernstein polynomials[END_REF][START_REF] Leblanc | On estimating distribution function using Bernstein polynomials[END_REF], used for distribution and density estimation, to define a Bernstein estimator of the regression r :

x → E [Y |X = x] at a given point x ∈ [0, 1] such that f (x) ̸ = 0 r n (x) = n i=1 Y i mn-1 k=0 1 { k mn <Xi≤ k+1 mn } b k (m n -1, x) n i=1 mn-1 k=0 1 { k mn <Xi≤ k+1 mn } b k (m n -1, x) , ( 1
)
where b k (m, x) = m k x k (1 -x) m-k
is the Bernstein polynomial of order m. This estimator can be viewed as a generalization of the estimator proposed in Tenbusch [START_REF] Tenbusch | Nonparametric Curve Estimation with Bernstein estimates[END_REF], in which the order m n is chosen to be equal to n.

The following proposition gives the bias, the variance and the M SE of r n (x), for

x ∈ [0, 1] such that f (x) > 0. Proposition 3.1. Let Assumptions (A2) and (A3) hold. For x ∈ [0, 1], such that f (x) > 0, we have E [ r n (x)] -r(x) = ∆(x)m -1 n + o m -1 n . (2) Var [ r n (x)] =    m 1/2 n n Var[Y |X=x] f (x) ψ(x) + o x m 1/2 n n if x ∈ (0, 1), mn n Var[Y |X=x] f (x) + o x mn n if x = 0, 1.
(3)

M SE [ r n (x)] =    ∆ 2 (x)m -2 n + m 1/2 n n Var[Y |X=x] f (x) ψ(x) + o m -2 n + o x m 1/2 n n if x ∈ (0, 1), ∆ 2 (x)m -2 n + mn n Var[Y |X=x] f (x) + o m -2 n + o x mn n if x = 0, 1.
To minimize the M SE of r n , for x ∈ [0, 1] such that f (x) > 0, the order m must equal to

m opt =      4∆ 2 (x)f (x) Var[Y |X=x]ψ(x) 2/5 n 2/5 if x ∈ (0, 1), 2∆ 2 (x)f (x) Var[Y |X=x] 1/3 n 1/3 if x = 0, 1, then M SE r n,mopt (x) =    5(∆(x)) 2/5 (Var[Y |X=x]ψ(x)) 4/5 (4f (x)) 4/5 n -4/5 + o n -4/5 if x ∈ (0, 1), 3(∆(x)Var[Y |X=x]) 2/3 (2f (x)) 2/3 n -2/3 + o n -2/3 if x = 0, 1.
Remark 3.1. Clearly, our first proposed estimator converge to the true regression function. The rate of convergence is bigger near of the edge (x ∈ {0, 1}) than inside the interval, however, as it was shown, the non-parametric kernel estimation near of the edge fails (see for instance Jmaei et al. [START_REF] Jmaei | Recursive distribution estimator defined by stochastic approximation method using Bernstein polynomials[END_REF], Slaoui and Jmaei [START_REF] Slaoui | Recursive density estimators based on Robbins-Monro's scheme and using Bernstein polynomials[END_REF]).

The following proposition gives the M ISE of r n Proposition 3.2. Let Assumptions (A2) and (A3) hold, we have

M ISE( r n ) = C 2 m 1/2 n n + C 1 m -2 n + o m 1/2 n n + o m -2 n . (4)
Hence, the asymptotically optimal choice of m is

m opt = 4C 1 C 2 2/5 n 2/5 ,
for which we get

M ISE r n,mopt = 5C 1/5 1 C 4/5 2 4 4/5 n -4/5 + o n -4/5 .
Let us now state the following theorem which gives the weak convergence rate of the estimator r n (x) defined in (1), for x ∈ [0, 1] such that f (x) > 0.

Theorem 3.1. (Weak pointwise convergence rate). Let Assumptions (A2) and (A3) hold.

When x ∈ (0, 1), and m n is chosen such that nm -5/2 n → c for some constant c ≥ 0, we have

n 1/2 m -1/4 n ( r n (x) -r (x)) D → N √ c∆(x), Var [Y |X = x] ψ(x) f (x) . When x ∈ {0, 1}, and m n is chosen such that nm -3 n → c for some constant c ≥ 0, we have n m n ( r n (x) -r (x)) D → N √ c∆(x), Var [Y |X = x] f (x) .
When x ∈ (0, 1) and m n is chosen such that nm

-5/2 n → ∞ or when x ∈ {0, 1} and m n is chosen such that nm -3 n → ∞ we have m n ( r n (x) -r (x)) P → ∆(x),
where D → denotes the convergence in distribution, N the Gaussian-distribution and P → the convergence in probability. The next corollary is an immediate consequence of the previous Theorem on which we give the the weak convergence rate of the estimator r n (x), for x ∈ [0, 1] such that f (x) > 0 in the case when m n is chosen such that nm -5/2 n → 0 for x ∈ (0, 1) and nm -3 n → 0 for x ∈ {0, 1}. Corollary 3.1. Let Assumptions (A2) and (A3) hold.

When x ∈ (0, 1), and m n is chosen such that nm

-5/2 n → 0, then n 1/2 m -1/4 n ( r n (x) -r (x)) D → N 0, Var [Y |X = x] ψ(x) f (x) . When x ∈ {0, 1}, and m n is chosen such that nm -3 n → 0, then n m n ( r n (x) -r (x)) D → N 0, Var [Y |X = x] f (x) .

Recursive estimator

In order to construct a stochastic algorithm for the estimation of the regression function r : x → E [Y |X = x] at a point x such as f (x) ̸ = 0, Révész [START_REF] Révész | Robbins-Monro procedure in a Hilbert space and its application in the theory of learning processes I[END_REF] defines an algorithm, which approximates the zero of the function h : y → f (x)r(x) -f (x)y. Using the procedure proposed in Robbins and Monro [START_REF] Robbins | A Stochastic Approximation Method[END_REF], the considered algorithm is defined by setting r 0 (x) ∈ R and for n ≥ 1

r n (x) = r n-1 (x) + γ n W n (x),
where γ n is the stepsize and W n is an observation of the function h at the point r n-1 (x). We define W n , using Bernstein polynomials

W n (x) = m n Y n mn-1 k=0 1 { k mn <Xn≤ k+1 mn } b k (m n -1, x) -m n mn-1 k=0 1 { k mn <Xn≤ k+1 mn } b k (m n -1, x)r n-1 (x),
then, the estimator r n can be rewritten as

r n (x) = 1 -γ n m n mn-1 k=0 1 { k mn <Xn≤ k+1 mn } b k (m n -1, x) r n-1 (x) + γ n m n Y n mn-1 k=0 1 { k mn <Xn≤ k+1 mn } b k (m n -1, x), = (1 -γ n f (x)) r n-1 (x) + γ n f (x) -m n mn-1 k=0 1 { k mn <Xn≤ k+1 mn } b k (m n -1, x) r n-1 (x) + γ n m n Y n mn-1 k=0 1 { k mn <Xn≤ k+1 mn } b k (m n -1, x).
We set

Z n (x) = m n mn-1 k=0 1 { k mn <Xn≤ k+1 mn } b k (m n -1, x), W n (x) = m n Y n mn-1 k=0 1 { k mn <Xn≤ k+1 mn } b k (m n -1, x).
Then, the proposed algorithm can be rewritten as follows:

r n (x) = (1 -γ n f (x)) r n-1 (x) + γ n (f (x) -Z n (x)) r n-1 (x) + γ n W n (x). (5)
In order, to establish the asymptotic behaviour of r n , we introduce the auxiliary stochastic approximation algorithm defined by setting ρ n (x) = r(x) for all n ≤ n 0 -2, ρ n0-1 (x) = r n0-1 (x), and, for n ≥ n 0 ,

ρ n (x) = (1 -γ n f (x)) ρ n-1 (x) + γ n (f (x) -Z n (x)) r(x) + γ n W n (x). ( 6
)
We first give the behaviour of ρ n . Then, we show how the behaviour of r n can be deduced from that of ρ n . 4.1. Within the interval [0, 1]. To obtain the bias, the variance and the M SE of r n (x), for x ∈ (0, 1) such that f (x) > 0, we set

: (A4) lim n→∞ (nγ n ) ∈ min a f (x) , 2α-a 4f (x) , ∞ . Proposition 4.1. Let Assumptions (A1)-(A4) hold. For x ∈ (0, 1), such that f (x) > 0,
we have

E[r n (x)] -r(x) = f (x)∆(x) f (x) -aξ 1 {a∈( 1-α 4 , 2 5 α]} m -1 n + 1 {a∈( 2 5 α, 2 3 )} o γ n m 1/2 n +o m -1 n + γ n m 1/2 n , (7) V ar[r n (x)] = 2f (x)ψ(x)Var [Y |X = x] 4f (x) -(2α -a)ξ 1 {a∈[ 2 5 α, 2 3 )} γ n m 1/2 n + 1 {a∈( 1-α 4 , 2 5 α)} o m -2 n +o γ n m 1/2 n + m -2 n , (8)
and

M SE [r n (x)] = f 2 (x)∆ 2 (x) (f (x) -aξ) 2 1 {a∈( 1-α 4 , 2 5 α)} m -2 n + 2f (x)Var [Y |X = x] ψ(x) 4f (x) -(2α -a)ξ 1 {a∈( 2 5 α, 2 3 )} γ n m 1/2 n + o m -2 n + γ n m 1/2 n .
Remark 4.1. When lim n→∞ (nγ n ) > max a f (x) , 2α-a 4f (x) , the equations ( 7) and (8) hold simultaneously.

To minimize the M SE of r n (x), for x ∈ (0, 1) such that f (x) > 0, the stepsize (γ n ) must be chosen in GS(-1) and (m n ) must must be in GS (2/5) such that

4 3/5 f (x) - 2 5 ξ -2/5 f (x)∆ 2 (x) Var [Y |X = x] ψ(x) 2/5 γ -2/5 n , then M SE [r n (x)] = 5(f (x)) 6/5 (∆(x)) 2/5 (Var [Y |X = x] ψ(x)) 4/5 4 6/5 f (x) -2 5 ξ 6/5 γ 4/5 n + o γ 4/5 n .
Let us now state the following theorem, which gives the weak convergence rate of the estimator r n (x) defined in [START_REF] Djojosugito | Boundary bias correction in nonparametric density estimation[END_REF], for x ∈ (0, 1) such that f (x) > 0. → c for some constant c ≥ 0, then

γ -1/2 n m -1/4 n (r n (x) -r (x)) D → N √ c f (x)∆(x) f (x) -aξ , 2f (x)Var [Y |X = x] ψ(x) 4f (x) -(2α -a)ξ . (2) If γ -1 n m -5/2 n → ∞, then m n (r n (x) -r (x)) P → f (x)∆(x) f (x) -aξ ,
where D → denotes the convergence in distribution, N the Gaussian-distribution and P → the convergence in probability.

The next corollary is an immediate consequence of the previous Theorem on which we give the the weak convergence rate of the estimator r n (x), for x ∈ (0, 1) in the case when m n is chosen such that nm -5/2 n → 0 for x ∈ (0, 1). Corollary 4.1. Let Assumptions (A1)-(A4) hold, when x ∈ (0, 1), and m n is chosen such that nm

-5/2 n → 0, then γ -1/2 n m -1/4 n (r n (x) -r (x)) D → N 0, 2f (x)Var [Y |X = x] ψ(x) 4f (x) -(2α -a)ξ .
4.2. The edges of the interval [0, 1]. For the case x ∈ {0, 1}, such that f (x) > 0, we need to consider the following additional Assumption

: (A ′ 4) lim n→∞ (nγ n ) ∈ min a f (x) , α-a 2f (x) , ∞ .
The following proposition gives the bias, the variance and the M SE of r n (x), for x ∈ {0, 1}.

Proposition 4.2. Let Assumptions (A1)-(A ′ 4) hold. For x ∈ {0, 1}, such that f (x) > 0, we have E[r n (x)] -r(x) = f (x)∆(x) f (x) -aξ 1 {a∈( 1-α 4 , α 3 ]} m -1 n + 1 {a∈( α 3 , 2 3 α)} o ( √ γ n m n ) +o m -1 n , (9)
V ar[r n (x)] = f (x)Var [Y |X = x] 2f (x) -(α -a)ξ 1 {a∈[ α 3 , 2 3 α)} γ n m n + 1 {a∈( 1-α 4 , α 3 ]} o m -2 n +o (γ n m n ) , (10)
and

M SE [r n (x)] = f 2 (x)∆ 2 (x) (f (x) -aξ) 2 1 {a∈( 1-α 4 , α 3 ]} m -2 n + f (x)Var [Y |X = x] 2f (x) -(α -a)ξ 1 {a∈( 1-α 4 , α 3 ]} γ n m n + o m -2 n + γ n m n . Remark 4.2. (1) When lim n→∞ (nγ n ) > max a f (x) , α-a 2f (x)
, the equations ( 9) and (10) hold simultaneously.

(2) To minimize the M SE of r n , for x ∈ {0, 1} such that f (x) > 0, the stepsize (γ n ) must be chosen in GS(-1) and (m n ) must be in GS (1/3) such that

2 2/3 f (x) - 1 3 ξ -1/3 f (x)∆ 2 (x) Var [Y |X = x] 1/3 γ -1/3 n , then M SE [r n (x)] = 3(f (x)) 4/3 (∆(x)Var [Y |X = x]) 2/3 2 4/3 f (x) -1 3 ξ 4/3 γ 2/3 n + o γ 2/3 n .
Let us now state the following theorem, which gives the weak convergence rate of the estimator r n (x) defined in [START_REF] Djojosugito | Boundary bias correction in nonparametric density estimation[END_REF], for x ∈ {0, 1} such that f (x) > 0.

Theorem 4.2. (Weak pointwise convergence rate). Let Assumption (A1)-(A ′ 4) hold, we have

(1) If γ -1 n m -3 n → c for some constant c ≥ 0, then γ -1/2 n m -1/2 n (r n (x) -r (x)) D → N √ c f (x)∆(x) f (x) -aξ , f (x)Var [Y |X = x] ψ(x) 2f (x) -(α -a)ξ , (2) If γ -1 n m -3 n → ∞, then m n (r n (x) -r (x)) P → f (x)∆(x) f (x) -aξ ,
where

D
→ denotes the convergence in distribution, N the Gaussian-distribution and P → the convergence in probability.

The next corollary is an immediate consequence of the previous Theorem on which we give the the weak convergence rate of the estimator r n (x), for x ∈ {0, 1} in the case when m n is chosen such that nm

-5/2 n → 0 for x ∈ {0, 1}. Corollary 4.2. Let Assumption (A1)-(A ′ 4) hold, when x ∈ {0, 1}, and m n is chosen such that nm -3 n → 0, then γ -1/2 n m -1/2 n (r n (x) -r (x)) D → N 0, f (x)Var [Y |X = x] ψ(x) 2f (x) -(α -a)ξ . 4.3. The M ISE of r n .
To obtain the M ISE of r n , we add the following assumption

: (A ′′ 4) Set φ = inf x∈[0,1] f (x) > 0, we demand that lim n→∞ (nγ n ) ∈ min a φ , 2α -a 4φ , ∞ . Proposition 4.3. Let Assumptions (A1) -(A ′′ 4) hold, we have M ISE (r n ) = K 1 1 {a∈( 1-α 4 , 2 5 α]} m -2 n + K 2 1 {a∈[ 2 5 α, 2 3 α} γ n m 1/2 n + o m -2 n + γ n m 1/2 n
The following result is a consequence of the previous proposition which gives the optimal order (m n ) of the estimator r n introduced in (5) and the corresponding M ISE.

Corollary 4.3. Let Assumptions (A1) -(A ′′ 4) hold. To minimize the M ISE of r n , the stepsize (γ n ) must be chosen in GS(-1) and (m n ) must be in GS(2/5) such that

4K 1 K 2 2/5 γ -2/5 n ,
and then

M ISE (r n ) = 5K 1/5 1 K 4/5 2 4 4/5 γ 4/5 n + o γ 4/5 n .
Remark 4.3. We can claim that our two proposed estimators converge to the true regression function. It is true that the rate of convergence is bigger near of the edge (x ∈ {0, 1}) than inside the interval, however, as it was shown previously (see for instance Jmaei et al.

[13], Slaoui and Jmaei [START_REF] Slaoui | Recursive density estimators based on Robbins-Monro's scheme and using Bernstein polynomials[END_REF]) the non-parametric kernel estimation near of the edge fails.

Applications

We recall the regression function's kernel estimator proposed by Nadaraya [START_REF] Nadaraya | On Estimating Regression[END_REF] and Watson [START_REF] Watson | Smooth regression analysis[END_REF], for x ∈ R such that f (x) ̸ = 0

r N W n (x) = n i=1 Y i K x -X i h n i=1 K x -X i h , (11)
where

K : R → R is a nonnegative, continuous, bounded function satisfying R K(z)dz = 1,
R zK(z)dz = 0 and R z 2 K(z)dz < ∞ known as kernel and h = (h n ) is a bandwidth (that is, a sequence of positive real numbers that goes to zero). We also recall the recursive estimator of a regression function which is a generalized version of Révész's estimator (see Révész [START_REF] Révész | Robbins-Monro procedure in a Hilbert space and its application in the theory of learning processes I[END_REF][START_REF] Révész | How to apply the method of stochastic approximation in the non-parametric estimation of a regression function[END_REF]) and was studied by Mokkadem et al. [START_REF] Mokkadem | Revisiting Révész's stochastic approximation method for the estimation of a regression function[END_REF]

r GR n (x) = 1 -γ n h -1 n K x -X n h n r GR n-1 (x) + γ n h -1 n Y n K x -X n h n . ( 12
)
The major limitation of these two estimators occurs at the edges of the support. In fact, these estimators are inconsistent at the boundary. This effectively restricts their application to values of x in the interior of the support of the estimated regression function.

The purpose of this section is to provide a comparative study between Nadaraya-Watson's estimator r N W n defined in [START_REF] Granovsky | Optimizing kernel methods: a unifying variational principle[END_REF], the generalized Révész's estimator r GR n defined in [START_REF] Igarashi | On improving convergence rate of Bernstein polynomial density estimator[END_REF], our non-recursive estimator r n defined in (1) and our recursive estimator r n introduced in (5).

Simulations. We consider the regression model

Y = r(X) + ε,
where ε ∼ N (0, 1).

When using the estimators r N W n and r GR n , we choose the kernel K(x) = (2π) -1/2 exp -x 2 /2 and the bandwidth equal to (h n ) = n -1/5 (ln(n + 1)) -1 . When using our proposed Bernstein estimators r n and r n , we choose the order equal to m n = ⌊n 2/5 (ln(n + 1))⌋ and we choose two stepsize (γ n ) = (n -0.9 ) and (γ n ) = (n -1 ).

We consider three sample sizes n = 50, n = 100 and n = 500, three regression functions (a) r(x) = cos(x), (b) r(x) = 0.3 exp -x 2 /2 + 0.7 exp -(x -1) 2 /2 , (c) r(x) = 1 + 0.6x, and three densities of X, the beta density B(3, 5), the beta mixture density 0.5B(2, 1) + 0.5B [START_REF] Babu | Application of Bernstein polynomials for smooth estimation of a distribution and density function[END_REF][START_REF] Boukabour | Semiparametric Bayesian networks for continuous data[END_REF] and the truncated standard normal density N [0,1] (0, 1). We consider six estimators: our non-recursive Bernstein estimator r n defined in (1), Nadaraya-Watson's estimator r N W n proposed in [START_REF] Granovsky | Optimizing kernel methods: a unifying variational principle[END_REF], two proposed recursive Bernstein estimators r n,1 and r n,2 introduced in (5) with stepsize (γ n ) = (n -1 ) and (γ n ) = (n -0.9 ) respectively and finally two Generalized Révész's estimators r GR n,1 and r GR n,2 defined in (12) using the same stepsizes as the one previously used in r n . For each model and sample of size n, we approximate the average integrated squared error (ISE) of the estimator using N = 500 trials of sample size n , r n,1 and r n,2 correspond to our recursive estimator with the choice (γ n ) = (n -1 ) and (γ n ) = (n -0.9 ) respectively, and r GR n,1 and r GR n,2 correspond to the generalized Révész's estimator with the choice (γ n ) = (n -1 ) and (γ n ) = (n -0.9 ) respectively.

ISE = 1 N N k=1 ISE [r k ] , Regression function Density of X n rn r N W n rn,1 rn,2 r GR n,1 r GR n,2 (a)
where rk is the estimator computed from the k th sample, and

ISE [r k ] = 1 0 {r(x) -r(x)} 2 dx.
In Table 1 we give qualitative comparison between our non-recursive estimator r n defined in (1) and our recursive estimator r n given in [START_REF] Djojosugito | Boundary bias correction in nonparametric density estimation[END_REF] with (γ n ) = (n -1 ). We then conclude that:

• In all the considered models, the average ISE of our non-recursive regression estimator r n defined in (1) is the smallest, except the cases with X ∼ B [START_REF] Bojanic | A unified theory of regularly varying sequences[END_REF][START_REF] Djojosugito | Boundary bias correction in nonparametric density estimation[END_REF] where the the average ISE of Nadaraya-Watson's estimator r GR n given in [START_REF] Granovsky | Optimizing kernel methods: a unifying variational principle[END_REF] is the smallest when the size is n = 50 and the cases with X ∼ N [0,1] (0, 1) where the average ISE of the generalized Révész's estimator estimator is the smallest.

• In all the models with X ∼ N [0,1] (0, 1), the average ISE of our recursive regression estimator r n defined in [START_REF] Djojosugito | Boundary bias correction in nonparametric density estimation[END_REF] with the choice (γ n ) = (n -1 ) is smaller than that of our non-recursive regression estimator r n introduced in (1). • In all the models, the average ISE of our recursive regression estimator r n,1 with the choice (γ n ) = (n -1 ) is smaller than that of our recursive regression estimator r n,2 with the choice (γ n ) = (n -0.9 ). • The average ISE decreases as the sample size increases.

Figure 1, shows regression estimates plotted for 500 simulated samples from the model r(x) = cos(x) with X ∼ N [0,1] (0, 1) of sizes n = 100 (left panel) and n = 500 (right panel). From Figure 1, we conclude that:

• Both our estimators are close to the true regression function. • Our recursive regression estimator r n defined in (5) using the stepsize (γ n ) = (n -1 ) is closer to the true regression function than that of the proposed nonrecursive estimator r n given in (1) especially with the size n = 500. • When the sample size increases, we get closer estimation of the true regression function.

Real dataset.

In order to illustrate our two proposed estimators by using a set of real data. We consider the CO2 dataset which is available in the R package Stat2Data and contained 237 observations on two variables; Day and CO2. Scientists at a research station in Brotjacklriegel, Germany recorded CO2 levels, in parts per million, in the atmosphere for each day from the start of April through November in 2001. First, to estimate an unknown regression function, it is critical to have a reliable data-dependent rule for order selection. One popular and practical approach is cross-validation. First, we compute the leave-one-out residuals:

∀i ∈ {1, . . . , n} , e -i = Y i -r-i (X i ),
where r-i is the regression estimate without the data point (X i , Y i ). Then, the smoothing parameter is chosen by minimizing

CV (m n ) = 1 n n i=1 e 2 -i .
We then apply our proposed estimators r n defined in (1) and r n given in (5) on this model. For convenience, we assume that the minimum of days is 90 and the maximum is 335 (the Day data are such that min i (x i ) = 91 and max i (x i ) = 334). Finally, we used the Cross-validation method to obtain m n = 220 for our non-recursive estimator r n and m n = n for our recursive estimator. We observe from Figure 2 that the two proposed estimators give better estimation compared to Nadaraya-Watson's (11) especially near the boundaries, which corroborate remarks 3.1 and 4.3f.

Conclusion

In this paper, we propose a non-recursive and recursive estimator of regression function based on Bernstein polynomials and stochastic algorithm derived from the Robbins-Monro's scheme. We first study their theoretical behavior. Then, we conduct a simulation study and analyse a real data application on CO2 data. For all the models, the average ISE of our non-recursive regression estimator r n defined in (1) is the smallest, except the cases with X ∼ B [START_REF] Bojanic | A unified theory of regularly varying sequences[END_REF][START_REF] Djojosugito | Boundary bias correction in nonparametric density estimation[END_REF] where the average ISE of Nadaraya-Watson's estimator r GR n given in [START_REF] Granovsky | Optimizing kernel methods: a unifying variational principle[END_REF] is smaller in the case when the sample size is n = 50 and the cases with X ∼ N [0,1] (0, 1) where the average ISE of the generalized Révész's estimator give better results in terms of average ISE. In addition, a major advantage of our recursive estimator is that its update, when new sample points are available, requires less computational cost than Nadaraya-Watson estimator. Finally, the two estimators have nice features and satisfactory improvement in comparison to Kernel estimators especially near the boundaries.

In conclusion, the estimation using Bernstein polynomials allowed us to overcome the edge problem and obtain quite similar results as Nadaraya-Watson's estimator. Moreover, we plan to make extensions of our method in the future and to consider the functional data (see, Slaoui [START_REF] Slaoui | Wild Bootstrap Bandwidth Selection of Recursive Nonparametric Relative Regression for Independent Functional Data[END_REF][START_REF] Slaoui | Recursive nonparametric regression estimation for independent functional data[END_REF]) to built a semi-recursive Bernstein estimator for regression function. We plan also to extend our estimators by considering the recursive nonparametric estimation for Bayesian networks (see for instance the recent paper in this subject Boukabour and Masmoudi [START_REF] Boukabour | Semiparametric Bayesian networks for continuous data[END_REF]).

Appendix A. Outlines of the proofs

In this section, we present proofs for the results given in the paper. First, we recall a series of results, which are proven in Leblanc [START_REF] Leblanc | A bias-reduced approach to density estimation using Bernstein polynomials[END_REF], linked to different sums of Bernstein polynomial, defined by

S mn (x) = mn k=0 b 2 k (m n , x),
These results are given in the following lemma.

Lemma A.1. We have

: (i) 0 ≤ S mn (x) ≤ 1, ∀x ∈ [0, 1], : (ii) S mn (x) = m -1/2 n [ψ(x) + o x (1)], ∀x ∈ (0, 1) : (iii) S mn (0) = S mn (1) = 1.
Let g be any continuous function on

[0, 1]. Then : (iv) m 1/2 n 1 0 g(x)S mn (x)dx = 1 0 g(x)ψ(x)dx + o(1),
We start by proving the characteristics of our non-recursive estimator r n defined by (1). To do so, we note

N n (x) = m n n n i=1 Y i mn-1 k=0 1 { k mn <Xi≤ k+1 mn } b k (m n -1, x).
Then, we may rewrite r n as

r n (x) = N n (x) f n (x) ,
where f n is the Vitale's estimator of the density f defined, for all x ∈ [0, 1], by

f n (x) = m n n n i=1 mn-1 k=0 1 { k mn <Xi≤ k+1 mn } b k (m n -1, x) = m mn-1 k=0 F n k + 1 m n -F n k m n b k (m n -1, x),
with F n is the empirical distribution function of the variable X.

A.1. Prooof of Proposition 3.1. We start by giving the bias and the variance of N n (x)

E [N n (x)] = m n E Y mn-1 k=0 1 { k mn <X≤ k+1 mn } b k (m n -1, x) , = m n mn-1 k=0 k+1 mn k mn R yg(z, y)dy dzb k (m n -1, x), = m n mn-1 k=0 k+1 mn k mn r(z)f (z)dz b k (m n -1, x).
Using Taylor expansion, we have

r(z)f (z) = r(x) + (z -x)r ′ (x) + (z -x) 2 2 r (2) (x) + o (z -x) 2 × f (x) + (z -x)f ′ (x) + (z -x) 2 2 f (2) (x) + o (z -x) 2 , = r(x)f (x) + (z -x) (r ′ (x)f (x) + r(x)f ′ (x)) + (z -x) 2 2 r (2) (x)f (x) + f (2) (x)r(x) + 2r ′ (x)f ′ (x) + o (z -x) 2 .
Since N (x) = r(x)f (x), then we obtain

E [N n (x)] = r(x)f (x)m n mn-1 k=0 k + 1 m n - k m n b k (m n -1, x) + (r ′ (x)f (x) + r(x)f ′ (x)) m n 2 mn-1 k=0 k + 1 m n -x 2 - k m n -x 2 × b k (m n -1, x) + r (2) (x)f (x) + f (2) (x)r(x) + 2r ′ (x)f ′ (x) m n 6 mn-1 k=0 k + 1 m n -x 3 - k m n -x 3 × b k (m n -1, x) + o m mn-1 k=0 k + 1 m n -x 3 - k m n -x 3 b k (m n -1, x) = N (x) + (r ′ (x)f (x) + r(x)f ′ (x)) m n 2 mn-1 k=0 m -2 n (2k + 1 -2m n x)b k (m n -1, x) + r (2) (x)f (x) + f (2) (x)r(x) + 2r ′ (x)f ′ (x) m n 6 mn-1 k=0 m -3 n (k + 1 -m n x) 2 + (k -m n x) 2 + (k + 1 -m n x)(k -m n x) b k (m n -1, x)[1 + o(1)] = N (x) + (r ′ (x)f (x) + r(x)f ′ (x)) m -1 n 2 {2T 1,mn-1 (x) + (1 -2x)T 0,mn-1 (x)} + r (2) (x)f (x) + f (2) (x)r(x) + 2r ′ (x)f ′ (x) m -2 n 6 mn-1 k=0 3(k -m n x) 2 + 3(k -mx) + 1 b k (m n -1, x)[1 + o(1)] = N (x) + (r ′ (x)f (x) + r(x)f ′ (x)) m -1 n 2 {2T 1,mn-1 (x) + (1 -2x)T 0,mn-1 (x)} + r (2) (x)f (x) + f (2) (x)r(x) + 2r ′ (x)f ′ (x) m -2 n 6 3T 2,mn-1 (x) + 3(2x + 1)T 1,mn-1 (x) + (x 2 + 3x + 1)T 0,mn-1 (x) [1 + o(1)],
where

T j,mn (x) = mn-1 k=0 (k -m n x) j b k (m n , x), ∀j ∈ N.
Note that it is easy to obtain

T 0,mn (x) = 1, T 1,mn (x) = 0, T 2,mn (x) = m n x(1 -x),
then, we have

E [N n (x)] = N (x) + ∆ 2 (x)m -1 n + o m -1 n . (13) Moreover, we have Var [N n (x)] = E N 2 n (x) -E 2 [N n (x)] ,
where

N 2 n (x) = m 2 n n 2 n i=1 Y 2 i mn-1 k=0 1 { k mn <Xi≤ k+1 mn } b k (m n -1, x) 2 + m 2 n n 2 n i,j=1 i̸ =j Y i Y j mn-1 k=0 1 { k mn <Xi≤ k+1 mn } b k (m n -1, x) × mn-1 k=0 1 { k mn <Xj ≤ k+1 mn } b k (m n -1, x) .
Then, we get

E N 2 n (x) = m 2 n n E   Y 2 mn-1 k=0 1 { k mn <X≤ k+1 mn } b k (m n -1, x) 2   + m 2 n n(n -1) n 2 E 2 Y mn-1 k=0 1 { k mn <X≤ k+1 mn } b k (m n -1, x) , = m 2 n n E   Y 2 mn-1 k=0 1 { k mn <X≤ k+1 mn } b k (m n -1, x) 2   + 1 - 1 n E 2 [N n (x)] ,
and

Var [N n (x)] = m 2 n n E   Y 2 mn-1 k=0 1 { k mn <X≤ k+1 mn } b k (m n -1, x) 2   - 1 n E 2 [N n (x)] , = m 2 n n E Y 2 mn-1 k=0 1 { k mn <X≤ k+1 mn } b 2 k (m n -1, x) - 1 n E 2 [N n (x)] , = m 2 n n mn-1 k=0 k+1 mn k mn R y 2 g(z, y)dy dzb 2 k (m n -1, x) - 1 n E 2 [N n (x)] , = m 2 n n mn-1 k=0 k+1 mn k mn E[Y 2 |X = z]f (z)dz b 2 k (m n -1, x) - 1 n E 2 [N n (x)] , = m n n E[Y 2 |X = x]f (x)S mn (x) - 1 n E 2 [N n (x)] .
The application of Lemma A.1 (ii) and (iii), ensures that

Var [N n (x)] = m 1/2 n n E[Y 2 |X = x]f (x)ψ(x) + o x m 1/2 n n for x ∈ (0, 1), mn n E[Y 2 |X = x]f (x) + o x mn n for x = 0, 1. (14)
Furthermore, we have

Cov (f n (x), N n (x)) = E [f n (x)N n (x)] -E [f n (x)] E [N n (x)] , = m 2 n n E   Y mn-1 k=0 1 { k mn <X≤ k+1 mn } b k (m n -1, x) 2   + n(n -1)m 2 n n 2 E 2 Y mn-1 k=0 1 { k mn <X≤ k+1 mn } b k (m n -1, x) -E [f n (x)] E [N n (x)] , = m 2 n n E   Y mn-1 k=0 1 { k mn <X≤ k+1 mn } b k (m n -1, x) 2   - 1 n E [f n (x)] E [N n (x)] , = m 2 n n mn-1 k=0 k+1 mn k mn R yg(z, y)dy dzb 2 k (m n -1, x) - 1 n E [f n (x)] E [N n (x)] , = m n n r(x)f (x)S mn (x) - 1 n E [f n (x)] E [N n (x)] .
The application of Lemma A.1 (ii) and (iii), ensures that

Cov (f n (x), N n (x)) = m 1/2 n n r(x)f (x)ψ(x) + o x m 1/2 n n for x ∈ (0, 1), mn n r(x)f (x) + o x mn n for x = 0, 1. (15)
To compute the bias of r n (x), we let h(x, y) = y x and we apply Taylor's expansion, we get

h(x n , y n ) = h(x, y) + (x n -x, y n -y)∇h T (x, y) + 1 2 (x n -x, y n -y)H(x, y)(x n -x, y n -y) T + o ||(x n -x, y n -y)|| 2 ,
where ∇h is the gradient of h and H is its hessian matrix.

∇h(x, y) = - y x 2 , 1 x H = 2y x 3 -1 x 2 -1 x 2 0 .
Then, we have

y n x n = y x - y x 2 (x n -x) + 1 x (y n -y) + y x 3 (x n -x) 2 - 1 x 2 (x n -x)(y n -y) + o (x n -x) 2 + (x n -x)(y n -y) .
We set (x n , y n ) = (f n (x), N n (x)) and (x, y) = (f (x), N (x)), we infer that

r n (x) = r(x) - r(x) f (x) (f n (x) -f (x)) + 1 f (x) (N n (x) -N (x)) + r(x) {f (x)} 2 (f n (x) -f (x)) 2 - 1 {f (x)} 2 (f n (x) -f (x)) (N n (x) -N (x)) +o (f n (x) -f (x)) 2 + (f n (x) -f (x)) (N n (x) -N (x)) , then E [ r n (x)] = r(x) - r(x) f (x) (E [f n (x)] -f (x)) + 1 f (x) (E [N n (x)] -N (x)) + r(x) {f (x)} 2 (E [f n (x)] -f (x)) 2 - 1 {f (x)} 2 E [(f n (x) -f (x)) (N n (x) -N (x))] + o E (f n (x) -f (x)) 2 + E [(f n (x) -f (x)) (N n (x) -N (x))] .
Let us recall, that for the Vitale's estimator f n , we have

E [f n (x)] = f (x) + ∆ 1 (x) m n + o m -1 n , ∀x ∈ [0, 1], (16)
and

Var [f n (x)] = m 1/2 n n f (x)ψ(x) + o x m 1/2 n n for x ∈ (0, 1), mn n f (x) + o x mn n for x = 0, 1. (17)
The combination of ( 16) and ( 13), ensures that

E [ r n (x)] = r(x) + 1 f (x) ∆ 2 (x) - r(x) f (x) ∆ 1 (x) m -1 n + o m -1 n , = r(x) + ∆(x)m -1 n + o m -1 n , ∀x ∈ [0, 1]
and we obtain (2) of Proposition 3.1. Now, in order to compute the variance of r n (x), we use the fact that

Var [h(x n , y n )] = ∇h(x, y)V ar(x n , y n )∇h T (x, y)[1 + o(1)],
which ensures that

Var [ r n (x)] = ∇h(x, y)Σ [f n (x), N n (x)] ∇h T (x, y)[1 + o(1)].
The combination of ( 14), ( 15) and ( 17), ensures that

Σ [f n (x), N n (x)] =            m 1/2 n n f (x) f (x)r(x) f (x)r(x) f (x)E(Y 2 |X = x) ψ(x) + o x m 1/2 n n for x ∈ (0, 1), mn n f (x) f (x)r(x) f (x)r(x) f (x)E(Y 2 |X = x) + o mn n for x = 0, 1.
We infer that, for x ∈ (0, 1), we have

Var [ r n (x)] = - N (x) {f (x)} 2 , 1 f (x) × f (x) f (x)r(x) f (x)r(x) f (x)E(Y 2 |X = x) × - N (x) {f (x)} 2 , 1 f (x) T × m 1/2 n ψ(x) + o x m 1/2 n n , = - N (x) f (x) + r(x), - N (x)r(x) f (x) + E(Y 2 |X = x) × - N (x) {f (x)} 2 , 1 f (x) T × m 1/2 n n ψ(x) + o x m 1/2 n n , = 1 f (x) 0, -E 2 (Y |X = x) + E(Y 2 |X = x) × (-1, 1) T m 1/2 n n ψ(x) + o x m 1/2 n n , = 1 f (x) E(Y 2 |X = x) -E 2 (Y |X = x) × m 1/2 n n ψ(x) + o x m 1/2 n n , = m 1/2 n n Var [Y |X = x] f (x) ψ(x) + o x m 1/2 n n ,
and, For x ∈ {0, 1}, we have

Var [ r n (x)] = m n n Var [Y |X = x] f (x) + o x m n n .
which gives (3) of Proposition 3.1.

A.2. Proof of Proposition 3.2. First, we have

M ISE( r n ) = 1 0 Var [ r n (x)] + Bias 2 [ r n (x)] dx = 1 0 Var [ r n (x)] dx + C 1 m -2 n + o m -2 n .
Moreover, we have

Var [ r n (x)] ≃ - N (x) {f (x)} 2 , 1 f (x) × Var [f n (x)] Cov [f n (x), N n (x)] Cov [f n (x), N n (x)] Var [N n (x)] × - N (x) {f (x)} 2 , 1 f (x) T , = r 2 (x) {f (x)} 2 Var [f n (x)] -2 r(x) {f (x)} 2 Cov [f n (x), N n (x)] + 1 {f (x)} 2 Var [N n (x)] [1 + o(1)], then 1 0 Var [ r n (x)] dx = 1 0 r 2 (x)dx Var [f n (x)] {f (x)} 2 dx -2 1 0 r(x) Cov [f n (x), N n (x)] {f (x)} 2 dx + 1 0 Var [N n (x)] dx {f (x)} 2 [1 + o(1)]. (18)
Since, we have for x ∈ [0, 1],

Var [f n (x)] = 1 n A mn (x) -f 2 mn (x) , f 2 mn (x) = E 2 [f n (x)] = f 2 (x) + O m -1 n , A mn (x) = m 2 n mn-1 k=0 F k + 1 m n -F k m n b 2 k (m n -1, x), = m n f (x)S mn-1 (x) + O (H mn-1 (x)) + O m -1 n , H mn (x) = mn-1 k=0 k m n -x b 2 k (m n -1, x).
The application of Cauchy-Schwarz inequality together with the fact that 0 ≤ b k (m n , x) ≤ 1 and

mn k=0 k m n -x 2 b k (m n , x) = x(1 -x) m n ≤ 1 4m n ,
gives for all m n ≥ 1 and x ∈ [0, 1]

H mn (x) ≤ m k=0 k m n -x 2 b k (m n , x) 1/2 mn k=0 b 3 k (m n , x) 1/2 ≤ S mn (x) 4m n 1/2 .
Moreover, the application of Jensen's inequality together with Lemma A.1 (iv), ensures that, for any continuous function g

1 0 g(x)H mn (x)dx ≤ 1 0 g(x) S mn (x) 4m n 1/2 dx, ≤ 1 0 g(x)dx 1/2 1 4m 3/2 n 1 0 g(x)ψ(x)dx + o m -3/2 n 1/2 = O m -3/4 n .
Then, we infer that

1 0 r 2 (x) Var [f n (x)] {f (x)} 2 dx = 1 n 1 0 r 2 (x) A mn (x) -f 2 mn (x) {f (x)} 2 dx, = 1 n 1 0 r 2 (x) A mn (x) {f (x)} 2 dx - 1 0 r 2 (x)dx + O 1 m n n , = m n n 1 0 r 2 (x) {f (x)} 2 S mn-1 (x) + O (H mn-1 (x)) + O m -1 n dx - 1 n 1 0 r 2 (x)dx + O 1 m n n , = m n n 1 0 r 2 (x) f (x) S mn-1 (x)dx + O m -3/4 n - 1 n 1 0 r 2 (x)dx + O 1 m n n .
Moreover, the application of Lemma A.1 (iv), gives

1 0 r 2 (x) Var [f n (x)] {f (x)} 2 dx = m 1/2 n n 1 0 r 2 (x) f (x) ψ(x)dx - 1 n 1 0 r 2 (x)dx +o m 1/2 n n + O 1 m n n . (19)
Further, we have

Cov [f n (x), N n (x)] = 1 n m 2 n mn-1 k=0 k+1 mn k m r(z)f (x)dz b 2 k (m n -1, x) -E [f n (x)] E [N n (x)] , = 1 n m 2 n mn-1 k=0 k+1 mn k mn [r(x)f (x) + O(z -x)]dz b 2 k (m n -1, x) -f (x)N (x) + O 1 m n n , = m n n r(x)f (x)S mn-1 (x) + O (H mn-1 (x)) + O m -1 n - 1 n f (x)N (x) + O 1 m n n .
Then, using the same argument for H mn-1 (x) as previously, we obtain (20)

1 0 r(x) Cov [f n (x), N n (x)] {f (x)} 2 dx = m n n 1 0 r 2 (x) f (x) S mn-1 (x)dx + O m -3/4 n - 1 n 1 0 r 2 (x)dx + O 1 m n n = m 1/2 n n 1 0 r 2 (x) f (x) ψ(x)dx - 1 n 1 0 r 2 (x)dx + o m 1/2 n n + O 1 m n n .
Moreover, we have

Var [N n (x)] = m 2 n n mn-1 k=0 k+1 mn k mn E[Y 2 |X = z]f (z)dz b 2 k (m n -1, x) - 1 n E 2 [N n (x)] , = m 2 n n mn-1 k=0 k+1 mn k mn [E[Y 2 |X = x]f (x) + O (z -x)]dz b 2 k (m n -1, x) - 1 n N 2 (x) + O 1 m n n , = m n n E[Y 2 |X = x]f (x)S mn-1 (x) + O (H mn-1 (x)) + O m -1 n - 1 n N 2 (x) + O 1 m n n , then, (21) 1 0 Var
[N n (x)] {f (x)} 2 dx = m n n 1 0 E[Y 2 |X = x] f (x) S mn-1 (x)dx + O m -3/4 n - 1 n 1 0 r 2 (x)dx + O 1 m n n , = m 1/2 n n 1 0 E[Y 2 |X = x] f (x) ψ(x)dx - 1 n 1 0 r 2 (x)dx + o m 1/2 n n + O 1 m n n .
Finally, substituting [START_REF] Mokkadem | Compact law of the iterated logarithm for matrix normalized sums of random vectors[END_REF], ( 20) and ( 21) into (18), gives

1 0 Var [ r n (x)] dx = 1 0 E[Y 2 |X = x] f (x) ψ(x)dx - 1 0 E 2 [Y |X = x] f (x) ψ(x)dx m 1/2 n n +o m 1/2 n n , = 1 0 E[Y 2 |X = x] -E 2 [Y |X = x] f (x) ψ(x)dx m 1/2 n n + o m 1/2 n n , = 1 0 V ar[Y |X = x] f (x) ψ(x)dx m 1/2 n n +o m 1/2 n n .
Then, we obtain the result in (4) of Proposition 3.2.

A.3. Proof of Theorem 3.1. To prove the convergence, for x ∈ (0, 1), we use the fact that ( 22)

n 1/2 m -1/4 n ( r n (x) -E [ r n (x)]) D → N 0, Var [Y |X = x] f (x) ψ(x) ,
which will be proved later. We have

n 1/2 m -1/4 n ( r n (x) -r (x)) = n 1/2 m -1/4 n r n (x) -E [ r n (x)] + n 1/2 m -1/4 n (E [ r n (x))] -r(x) , = n 1/2 m -1/4 n ( r n (x) -E [ r n (x)]) + n 1/2 m -5/4 n ∆(x)[1 + o(1)],
then, when nm → ∞, we have

m n ( r n (x) -r (x)) = m n ( r n (x) -E [ r n (x)]) + m n (E [ r n (x)] -r (x)) , = n -1/2 m 5/4 n n 1/2 m -1/4 n ( r n (x) -E [ r n (x)]) + ∆(x)[1 + o(1)].
Since we have n 1/2 m 5/4 n → 0, Part 2 of Theorem 3.1 follows from [START_REF] Müller | Smooth optimal kernel estimators near endpoints[END_REF]. Now let us prove [START_REF] Müller | Smooth optimal kernel estimators near endpoints[END_REF]. First, we let

w i = mn-1 k=0 1 { k mn <Xi≤ k+1 mn } b k (m n -1, x) n i=1 mn-1 k=0 1 { k mn <Xn≤ k+1 mn } b k (m n -1, x) .
Clearly we have

r n (x) = n i=1 w i Y i , and
r n (x) -E [ r n (x)] = n i=1 (w i Y i -E [w i Y i ]) . Noting that 0 ≤ w i ≤ 1, for all p > 0, we have E |w i Y i | 2+p = O(1) and n i=1 E |w i Y i | 2+p = O(n).
Moreover, for x ∈ (0, 1), we have

v 2 n = n i=1 Var [w i Y i ] = V ar[Y |X = x] f (x) ψ(x)nm 1/2 n + o nm 1/2 n , then, we have 1 v 2+p n n i=1 E |w i Y i | 2+p = O n n 2+p 2 m 2+p 4 n , = O n -p 2 m -2+p 4 n = o(1).
Then the convergence in [START_REF] Müller | Smooth optimal kernel estimators near endpoints[END_REF] follows from the application of Lyapounov's theorem. Now to prove the convergence, For x ∈ {0, 1}, we use the fact that

(23) n m n ( r n (x) -E [ r n (x)]) D → N 0, Var [Y |X = x] f (x) ,
which will be proved later. We have

n m n ( r n (x) -r (x)) = n m n r n (x) -E [ r n (x)] + n m n (E [ r n (x))] -r(x) , = n m n ( r n (x) -E [ r n (x)]) + n 1/2 m -3/2 n ∆(x)[1 + o(1)],
we infer that, when nm -3 n → c for some constant c ≥ 0, then Part 3 of Theorem 3.

1 follows. Now, if nm -3 n → ∞, we have m n ( r n (x) -r (x)) = m n ( r n (x) -E [ r n (x)]) + m n (E [ r n (x)] -r (x)) , = n 1/2 m 3/2 n n m n ( r n (x) -E [ r n (x)]) + ∆(x)[1 + o(1)],
and then Part 4 of Theorem 3.1 follows from ( 23) and the fact that n 1/2 m 5/4 n → 0. To prove [START_REF] Nadaraya | On Estimating Regression[END_REF], For x ∈ {0, 1}, we have

v 2 n = n i=1 Var |w i Y i -E [w i Y i ]| 2+p = V ar[Y |X = x] f (x) nm n + o (nm n ) , hence 1 v 2+p n n i=1 E |w i Y i -E [w i Y i ]| 2+p = O n n 2+p 2 m 2+p 2 n , = O n -p 2 m -2+p 2 n = o(1).
Then the convergence in [START_REF] Nadaraya | On Estimating Regression[END_REF] follows from the application of Lyapounov's theorem.

A.4. Proof of the results obtained for r n . First, we set n 0 ≥ 3 such that ∀k ≥ n 0 , γ k ≤ (2 ∥f ∥ ∞ ) -1 and γ k m k ≤ 1. Moreover, we introduce the following notations:

s n = n k=n0 γ k Π n (s) = n j=n0 (1 -sγ j ) for s > 0, U k,n (s) = Π n (s)Π -1 k (s) for s > 0. Further, we define the sequences (λ n ), ( λ n ), (β n ) and ( β n ) by setting (λ n ) =    γ n m 1/2 n if lim n→∞ γ n m 5/2 n = ∞, m -1 n otherwise. ( λ n ) =    γ n m 1/2 n ln n if lim n→∞ γ n m 5/2 n ln n = ∞, m -1 n otherwise. (β n ) = √ γ n m n if lim n→∞ γ n m 3 n = ∞, m -1 n otherwise. (24)
A.4.2. Proof of Lemma A.3. We have, for n ≥ n 0 ,

ρ n (x) -r(x) = (1 -γ n f (x)) (ρ n-1 (x) -r(x)) + γ n (W n (x) -r(x)Z n (x)) , = Π n (f (x)) n k=n0 Π -1 k (f (x))γ k (W k (x) -r(x)Z k (x)) +Π n (f (x))(ρ n0-1 (x) -r(x)), = T n (x) + R n (x).
Remark A.1.

(1) Since ρ n0-1 (x) = r n0-1 (x), we have

T n (x) = n k=n0 U k,n (f (x))γ k (W k (x) -r(x)Z k (x)) , R n (x) = Π n (f (x))(r n0-1 (x) -r(x)).
(2) Since |r n0-1 (x) -r(x)| = O(1) a.s. The application of Lemma A.2, ensures that

R n (x) = O (Π n (f (x))) a.s. (29) = o (λ n ) for x ∈ (0, 1), o (β n ) for x ∈ {0, 1} . a.s.
We infer that Lemma A.3 hold when ρ n (x) is replaced by T n (x). Then, for x ∈ [0, 1] such that f (x) > 0, we have

E [T n (x)] = n k=n0 U k,n (f (x))γ k (E [W k (x)] -r(x)E [Z k (x)]) ,
where

E [Z k (x)] = m k m k -1 k=0 F k + 1 m k -F k m k b k (m -1, x), = f (x) + ∆ 1 (x)m -1 k + o m -1 k , and
E [W k (x)] = m k E Y m k -1 k=0 1 k m k <X≤ k+1 m k b k (m k -1, x) , = r(x)f (x) + ∆ 2 (x)m -1 k + o m -1 k .
Then, we obtain

E [T n (x)] = f (x)∆(x) n k=n0 U k,n (f (x))γ k m -1 k + o m -1 k .
Moreover, we have

Var [T n (x)] = n k=n0 U 2 k,n (f (x))γ 2 k {Var [W k (x)] + r 2 (x)Var [Z k (x)] -2r(x)Cov (W k (x), Z k (x)) ,
where

Var [W k (x)] = E[Y 2 |X = x]f (x)ψ(x)m 1/2 k + o x m 1/2 k for x ∈ (0, 1), E[Y 2 |X = x]f (x)m k + o x (m k ) for x = 0, 1. Var [Z k (x)] = f (x)ψ(x)m 1/2 k + o x m 1/2 k for x ∈ (0, 1), f (x)m k + o (m k ) for x = 0, 1. Cov (W k (x), Z k (x)) = r(x)f (x)ψ(x)m 1/2 k + o x m 1/2 k for x ∈ (0, 1), r(x)f (x)m k + o x (m k ) for x = 0, 1.
which gives

Var [T n (x)] =            f (x)Var [Y |X = x] ψ(x) n k=n0 U 2 k,n (f (x))γ 2 k m 1/2 k + o x m 1/2 k for x ∈ (0, 1), f (x)Var [Y |X = x] n k=n0 U 2 k,n (f (x))γ 2 k [m k + o x (m k )] for x = 0, 1.
Remark A.2.

(1) For x ∈ (0, 1), the application of Lemma A.2 gives ( 25) and ( 26). ( 2) For x ∈ {0, 1}, the application of Lemma A.2 gives ( 27) and [START_REF] Robbins | A Stochastic Approximation Method[END_REF].

The following lemma gives the weak convergence rate of the estimator ρ n defined in [START_REF] Galambos | Regularly varying sequences[END_REF], for x ∈ [0, 1] such that f (x) > 0.

Lemma A.4. (Weak convergence rate of ρ n )

Let Assumption (A1) -(A4) hold. For x ∈ (0, 1), we have

(1) If γ -1 n m -5/2 n → c for some constant c ≥ 0, then γ -1/2 n m -1/4 n (ρ n (x) -r (x)) D → N √ c f (x)∆(x) f (x) -aξ , 2f (x)Var [Y |X = x] ψ(x) 4f (x) -(2α -a)ξ , (2) If γ -1 n m -5/2 n → ∞, then m n (ρ n (x) -r (x)) P → f (x)∆(x) f (x) -aξ .
Let Assumption (A1)-(A ′ 4) hold. For x ∈ {0, 1}, we have

(1) If γ -1 n m -3 n → c for some constant c ≥ 0, then γ -1/2 n m -1/2 n (ρ n (x) -r (x)) D → N √ c f (x)∆(x) f (x) -aξ , f (x)Var [Y |X = x] ψ(x) 2f (x) -(α -a)ξ , (2) If γ -1 n m -3 n → ∞, then m n (ρ n (x) -r (x)) P → f (x)∆(x) f (x) -aξ .
A.4.3. Proof of Lemma A.4. To prove Lemma A.4, for x ∈ (0, 1), we use the fact that if a ∈ [ 2 5 α, 2 3 α), we have

γ -1/2 n m -1/4 n (ρ n (x) -E [ρ n (x)]) D → N 0, 2f (x)Var [Y |X = x] ψ(x) 4f (x) -(2α -a)ξ , (30)
which will be proved later.

Remark A.3.

(1) The result in (30) hold if we replace ρ n (x) by T n (x). (2) Part 1 of Lemma A.4 follows from the combination [START_REF] Rao | Estimation of distribution and density functions by generalized Bernstein polynomials[END_REF] and [START_REF] Slaoui | Bandwidth selection for recursive kernel density estimators defined by stochastic approximation method[END_REF].

(3) Part 1 and 2 of Lemma A.4 follows from the combination of ( 27) and [START_REF] Slaoui | Bandwidth selection for recursive kernel density estimators defined by stochastic approximation method[END_REF] Now, we set

η k (x) = Π -1 k (f (x))γ k (W k (x) -r(x)Z k (x)) . (31)
Then,

T n (x) -E [T n (x)] = Π n (f (x)) n k=n0 (η k (x) -E [η k (x)]) .
Moreover, for x ∈ (0, 1), we have

Var [η k (x)] = Π -2 k (f (x))γ 2 k m 1/2 k [f (x)ψ(x)Var [Y |X = x] + o(1)
] , and, since lim n→∞ (nγ n ) > (2α -a)/(4f (x)), Lemma A.2 ensures that

v 2 n = n k=n0 Var [η k (x)] = n k=n0 Π -2 k (f (x))γ 2 k m 1/2 k [f (x)ψ(x)Var [Y |X = x] + o(1)] = 2Π -2 n (f (x))γ n m 1/2 n 4f (x) -(2α -a)ξ [f (x)ψ(x)Var [Y |X = x] + o(1)] . (32)
Further, for all p > 0 and x ∈ [0, 1], we make use of Lemma A.1 (ii) to ensure that

E   |Y k -r(x)| 2+p m k -1 i=0 1 i m i <Xn≤ i+1 m i b k (m i -1, x) 2+p   ≤ m k -1 i=0 b 2 k (m i -1, x) (2+p)
Moreover, since lim n→∞ (nγ n ) > (α -a/2) /(2f (x)), there exists a p > 0 such that lim n→∞ (nγ

n ) > (1 + p) (α -a/2) /(2 + p)(f (x)) > (1+p)α-(3(2+p)/4)a 2+p
, then the application of Lemma A.2 gives

n k=n0 E |η k (x)| 2+p = O n k=n0 Π -2-p k (f (x))γ 2+p k m 2+p k ×E   |Y k -r(x)| 2+p m k -1 i=0 1 i m i <Xn≤ i+1 m i b k (m i -1, x) 2+p     = O n k=n0 Π -2-p k (f (x))γ 2+p k m 3(2+p)/4 k = O γ 1+p n m 3(2+p)/4 n Π 2+p n (f (x))
, we infer that

1 v 2+p n n k=n0 E |η k (x)| 2+p = O m n (γ n m n ) p/2 ,
and the convergence in [START_REF] Slaoui | Bandwidth selection for recursive kernel density estimators defined by stochastic approximation method[END_REF] follows from the application of Lyapounov's Theorem. Now, to prove Lemma A.4, For x ∈ {0, 1}, we use the fact that when a ∈ [ α 3 , 2 3 α), we have

γ -1/2 n m -1/2 n (ρ n (x) -E [ρ n (x)]) D → N 0, f (x)Var [Y |X = x] 2f (x) -(α -a)ξ , (34)
which will be proved later.

Remark A.4.

(1) Part 3 and 4 of Lemma A.4 follows from the combination of ( 26) and ( 34).

(2) The result in (34) hold if we replace ρ n (x) by T n (x).

In the case when x = 0, 1, we have

Var [η k (x)] = Π -2 k (f (x))γ 2 k m k [f (x)Var [Y |X = x] + o(1)] . Since lim n→∞ (nγ n ) > (α -a)/(2f (x)), we make use of Lemma A.2 to ensure that v 2 n = n k=n0 Var [η k (x)] = n k=n0 Π -2 k (f (x))γ 2 k m k [f (x)Var [Y |X = x] + o(1)] = 2Π -2 n (f (x))γ n m n 4f (x) -(2α -a)ξ [f (x)Var [Y |X = x] + o(1)] .
Moreover, there exists a p > 0 such that

n k=n0 E |η k (x)| 2+p = O γ 1+p n m 3(2+p)/4 n Π 2+p n (f (x)) , then, 1 v 2+p n n k=n0 E |η k (x)| 2+p = O m 1/2 n γ n m 1/2 n p/2
, and the convergence in [START_REF] Slaoui | Plug-in bandwidth selector for recursive kernel regression estimators defined by stochastic approximation method[END_REF] follows from the application of Lyapounov's Theorem.

The following lemma gives the strong pointwise convergence rate of ρ n , for x ∈ [0, 1] such that f (x) > 0.

Lemma A.5. (Strong pointwise convergence rate of ρ n ) Let Assumption (A1) -(A4) hold. For x ∈ (0, 1), we have

(1) If γ -1 n m -5/2 n / ln(s n ) → c for some constant c ≥ 0, then, with probability one, the sequence   γ -1 n m -1/2 n 2 ln(s n ) (ρ n (x) -r(x))
  is relatively compact and its limit set is the interval

c 2 f (x)∆(x) f (x) -aξ - 2f (x)Var [Y |X = x] ψ(x) 4f (x) -(2α -a)ξ , c 2
f (x)∆(x) f (x) -aξ + 2f (x)Var [Y |X = x] ψ(x) 4f (x) -(2α -a)ξ .
(

) If γ -1 n m -5/2 n / ln(s n ) → ∞, then with probability one lim n→∞ m n (ρ n (x) -r (x)) = f (x)∆(x) f (x) -aξ . 2
Let Assumption (A1)-(A ′ 4) hold. For x ∈ {0, 1}, we have (1) If γ -1 n m -3 n / ln(s n ) → c for some constant c ≥ 0, then, with probability one, the sequence

  γ -1 n m -1 n 2 ln(s n ) (ρ n (x) -r (x))
  is relatively compact and its limit set is the interval

c 2 f (x)∆(x) f (x) -aξ - f (x)Var [Y |X = x] ψ(x) 2f (x) -(α -a)ξ , c 2
f (x)∆(x) f (x) -aξ + f (x)Var [Y |X = x] ψ(x) 2f (x) -(α -a)ξ .
(

) If γ -1 n m -3 n / ln(s n ) → ∞, then, with probability one lim n→∞ m n (ρ n (x) -r (x)) = f (x)∆(x) f (x) -aξ . 2
A.4.4. Proof of Lemma A.5.

Remark A.5.

(1) In view of (30), Lemma A.5 hold when ρ n (x) -r (x) is replaced by T n (x) (2) We give the proof in the case when x ∈ (0, 1), the case x = 0, 1 can be proven by following similar steps.

First, we set

B n (x) = n k=n0 (η k (x) -E [η k (x)]) ,
where η k is defined in [START_REF] Slaoui | The stochastic approximation method for the estimation of a distribution function[END_REF].

• We consider the case a ≥ 2 5 α (in which lim n→∞ (nγ n ) > (α -a/2)/(2f (x))). We set

H 2 n (f (x)) = Π 2 n (f (x))γ -1 n m -1/2 n
, and note that, since γ

-1 n m -1/2 n ∈ GS(α -a/2), we have ln H -2 n (f (x)) = -2 ln (Π n (f (x))) + ln n k=n0 γ -1 k-1 m -1/2 k-1 γ -1 k m -1/2 k + ln γ n0-1 m 1/2 n0-1 = -2 n k=n0 ln (1 -f (x)γ k ) + n k=n0 ln 1 - α -a/2 k + o 1 k + ln γ n0-1 m 1/2 n0-1 = n k=n0 (2f (x)γ k + o (γ k )) - n k=n0 ((α -a/2)ξγ k + o (γ k )) + ln γ n0-1 m 1/2 n0-1 = (2f (x) -(α -a/2)ξ) s n + o (s n ) . ( 35
) Since 2f (x) -(α -a/2)ξ > 0, it follows in particular that lim n→∞ H -2 n (f (x)) = ∞. Moreover, we have lim n→∞ H 2 n (f (x))/H 2 n-1 (f (x)) = 1
, and the application of Lemma A.2 together with [START_REF] Slaoui | Moderate deviation principles for recursive regression estimators defined by stochastic approximation method[END_REF], ensures that

lim n→∞ H 2 n (f (x)) n k=n0 Var [η k (x)] = f (x)ψ(x)Var [Y |X = x] (2f (x) -(α -a/2)ξ) .
set ε ∈ (0, α -5a/2) such that lim n→∞ (nγ n ) > a/f (x) + ε/2. Then, the application of Lemma A.2 together with [START_REF] Slaoui | Moderate deviation principles for recursive regression estimators defined by stochastic approximation method[END_REF], ensures that

H 2 n (f (x)) n k=n0 Var [η k (x)] = O Π 2 n (f (x))m 2 n ln ln Π -2 n (f (x))m -2 n n k=n0 Π -2 k (f (x))γ k m 1/2 k = O Π 2 n (f (x))m 2 n ln ln Π -2 n (f (x))m -2 n n k=n0 Π -2 k (f (x))γ k o m -2 k k -ε = o(1).
Moreover, in view of ( 33)

E |η k (x)| 3 = O Π -3 n (f (x))γ 3 n m 9/4 n ,
and thus in view of (37), we get

1 n √ n n k=n0 E |H n (f (x))η k (x)| 3 = O H 3 n (f (x)) n √ n ln ln Π -2 n (f (x))m -2 n 3/2 n k=n0 Π -3 k (f (x))γ 3 k m 9/4 k = O Π 3 n (f (x))m 3 n n √ n ln ln Π -2 n (f (x))m -2 n 3/2 n k=n0 Π -3 k (f (x))γ k o m -3 k = O ln ln Π -2 n (f (x))m -2 n 3/2 n √ n = o ln H -2 n (f (x)) -1 .
The application of Theorem 1 in Mokkadem and Pelletier [START_REF] Mokkadem | Compact law of the iterated logarithm for matrix normalized sums of random vectors[END_REF] then ensures that, with probability one

lim n→∞ H n (f (x))B n (x) 2 ln ln H -2 n (f (x)) = lim n→∞ m -1 n ln ln Π -2 n (f (x))m -2 n 2 ln ln H -2 n (f (x)) (T n (x) -E [T n (x)]) = 0.
Noting that (37) ensures that

lim n→∞ ln ln H -2 n (f (x)) / ln ln Π -2 n (f (x))m -2 n = 1, then, lim n→∞ m -1 n (T n (x) -E [T n (x)]) = 0 a.s.
and Lemma A.5 in the case when a < 2α/5 follows from [START_REF] Rao | Estimation of distribution and density functions by generalized Bernstein polynomials[END_REF].

A.4.7. Asymptotic behaviour of r n . We show in this section how to deduce the asymptotic behaviour of r n from that of ρ n . To do so, we set

δ n (x) = r n (x) -ρ n (x),
and we prove that δ n is negligible in front of ρ n . Note that, in view of ( 5) and ( 6), and since ρ n0-1 = r n0-1 , we have, for n ≥ n 0 δ n (x) = (1 -γ n f (x)) δ n-1 (x) + γ n (f (x) -Z n (x)) (r n-1 (x) -r(x))

= n k=n0
U k,n (f (x))γ k (f (x) -Z k (x)) (r k-1 (x) -r(x)) . [START_REF] Vitale | A Bernstein polynomial approach to density function estimation[END_REF] To obtain an upper bound of δ n , we must have an upper bound of r n -r. To do so, we use the following property given by Mokkadem et al. [START_REF] Mokkadem | Revisiting Révész's stochastic approximation method for the estimation of a regression function[END_REF] (P) : if (r n -r) is known to be bounded almost surely by a sequence (w n ), then it can be shown that (δ n ) is bounded almost surely by a sequence (w ′ n ) such that lim n→∞ w ′ n w n = 0, which may allow to upper bound r n -r by a sequence smaller than (w n ).

We thus proceed as follows. We first establish an upper bound of (r n -r). Then, we apply the Property (P) several times until we obtain an upper bound which allows to prove that δ n is negligible in front of ρ n .

The proof of the results given in Section 4 relies on the repeated application of the following lemma. = O (λ n w n ) a.s.

Further, we set

ε k (x) = E [Z k (x)] -Z k (x), G k (x) = r k (x) -r(x), Q n (x) = n k=n0 Π -1 k (f (x))γ k ε k (x)G k-1 (x),
and F k = σ ((X 1 , Y 1 ) , . . . , (X k , Y k )). Since, we have

Var [Z k (x)] = f (x)ψ(x)m 1/2 k + o m 1/2 k
, ∀x ∈ (0, 1), and of Lemma A.2, the increasing process of the martingale (Q n (x)) satisfies • Let us first consider the case when the sequence (nγ n ) is bounded. In this case we have Π -1 n (f (x)) ∈ GS(ξ -1 f (x)), and thus ln (< Q > n (x)) = O (ln n) a.s. Theorem 1.3.15 in Duflo [START_REF] Duflo | Random Iterative Models[END_REF] then ensures that, for any δ > 0, + o (λ n ) a.s.

< Q > n (x) = n k=n0 E Π -2 k (f (x))γ 2 k ε 2 k (x)G 2 k-1 (x)|F k-1 = n k=n0 Π -2 k (f (x))γ 2 k G 2 k-1 (
|Q n (x)| = o < Q > 1 2 n (x) (ln < Q > n (x))
which concludes the proof of Lemma A.7 in this case.

• Let us now consider the case lim n→∞ (nγ n ) = ∞. In this case, for all δ > 0, we have It follows that ln Π -2 n (f (x)) = O n 1+δ γ n . The sequence (λ n w n ) being in GS(-λ * + w * ), we deduce that, for all δ > 0, we have ln (< Q > n (x)) = O n 1+δ γ n a.s.

Theorem 1.3.15 in Duflo [START_REF] Duflo | Random Iterative Models[END_REF] then ensures that, for any δ > 0,

|Q n (x)| = o < Q > 1 2
n (x) (ln < Q > n (x)) which concludes the proof of Lemma A.7 for x ∈ (0, 1).

Remark A.6.

(1) For x ∈ {0, 1}, we use the same steps as in the case x ∈ (0, 1) with the sequence (β n ) defined in [START_REF] Priestley | Non-parametric function fitting[END_REF].

(2) To use Lemma A.7, we must establish an upper bound for r n (x) -r(x). For this purpose, we follow similar steps as Mokkadem et al. [START_REF] Mokkadem | Revisiting Révész's stochastic approximation method for the estimation of a regression function[END_REF] to prove Proposition 4.1, Theorem 4.1 and Proposition 4.3 in the case (nγ n ) is bounded and in the case lim n→∞ (nγ n ) = ∞ (see the supplementary material). (3) We use the same idea to prove Proposition 4.2 and Theorem 4.2 which correspond to the case x = 0, 1.

Theorem 4 . 1 . 1 )

 411 (Weak pointwise convergence rate). Let Assumptions (A1)-(A4) hold, we have (If γ

Figure 1 .

 1 Figure 1. Qualitative comparison between the two proposed regression estimators r n given in (5) with stepsize (γ n ) = (n -1 ) (solid line) and r n given in (1) (dashed line), the true regression function (circle line) for 500 samples respectively of size 100 (left panel) and of size 500 (right panel) of the model r(x) = cos(x) with X ∼ N [0,1] (0, 1).

Figure 2 .

 2 Figure2. The daily carbon dioxide measurements data using Nadaraya-Watson's estimator[START_REF] Granovsky | Optimizing kernel methods: a unifying variational principle[END_REF] (dotted line) and our proposed Bernstein estimators r n defined in (1) (dashed line) and r n given in[START_REF] Djojosugito | Boundary bias correction in nonparametric density estimation[END_REF] with stepsize (γ n ) = (n -1 ) (solid line).

-5/ 2 n

 2 → c for some constant c ≥ 0, Part 1 of Theorem 3.1 follows immediately. Now, when nm -5/2 n

  2+p g(z, y)dy + |r(x)| 2+p R g(z, y)dy dz = O m -(2+p)/4 n .

Lemma A. 7 . 1 ) 2 +( 2 ) 2 a 1 ) 2 +( 2 )n w n n 1+δ γ n 1+δ 2 a.s. A. 4 . 8 ..

 71222122248 Let Assumptions (A1) -(A3) hold, and assume that there exists (w n ) ∈ GS(w * ) such that |r n (x) -r(x)| = O(w n ) a.s. For x ∈ (0, 1), we have(If the sequence (nγ n ) is bounded, if lim n→∞ (nγ n ) > min {a/f (x), (2α -a)/(4f (x))} ,and if w * > 0, then, for all δ > 0,|δ n (x)| = O λ n w n (ln n) 1+δ o (λ n ) a.s. If lim n→∞ (nγ n ) = ∞, then, for all δ > 0, |δ n (x)| = O λ n w n n 1+δ γ n 1+δ .s.For x ∈ {0, 1}, we have(If the sequence (nγ n ) is bounded, if lim n→∞ (nγ n ) > min {a/f (x), (α -a)/(2f (x))} ,and if w * > 0, then, for all δ > 0,|δ n (x)| = O β n w n (ln n) 1+δ o (β n ) a.s. If lim n→∞ (nγ n ) = ∞, then, for all δ > 0, |δ n (x)| = O β Proof of Lemma A.7.To prove Lemma A.7, we use the following decomposition, which can be deduced from[START_REF] Vitale | A Bernstein polynomial approach to density function estimation[END_REF] δ n (x) = δ(1) n (x) + δ(2) n (x), withδ (1) n (x) = n k=n0 U k,n (f (x))γ k (E [Z k (x)] -Z k (x)) (r k-1 (x) -r(x)) , δ (2) n (x) = n k=n0 U k,n (f (x))γ k (f (x) -E [Z k (x)]) (r k-1 (x) -r(x)) .Moreover, we have for x ∈ [0, 1]E [Z k (x)] = f (x) + ∆ 1 (x)m -1 k + o m -1 kThen, the application of Lemma A.2, ensures thatδ (2) n (x) = O Π n (f (x)) n k=n0 Π -1 k (f (x))γ k m -1 k w k a.s. = O Π n (f (x)) n k=n0 Π -1 k (f (x))γ k O (λ k ) w k a.s.

  x)Var [Z k (x)]

1+δ 2 + 2 +

 22 O(1) a.s.= o Π -1 n (f (x))λ n w n (ln n) 1+δ O(1) a.s.It follows that, for any δ > 0, δ(1) n (x) = o λ n w n (ln n)

1+δ 2 +

 2 O (Π n (f (x))) a.s. = o λ n w n (ln n)

γn γ n n δ n k=1 γ k k δ = 1 -

 1 k k δ . Moreover, since γ n n δ ∈ GS(-(α -δ)) with (α -δ) < 1, we have lim n→∞ (α -δ).

1+δ 2 + 2 + 2 += o λ n w n n 1+δ γ n 1+δ 2 a

 2222 O(1) a.s.= o Π -1 n (f (x))λ n w n n 1+δ γ n 1+δ O(1) a.s.The application of Lemma A.2 then ensures that, for any δ > 0,δ (1) n (x) = o λ n w n n 1+δ γ n 1+δ O (Π n (f (x))) a.s. .s.

Table 1 .

 1 The average integrated squared error (ISE) of our nonrecursive estimator r

		B(3, 5)	50	0.055	0.055	0.104 0.115 0.038 0.047
			200	0.032	0.034	0.079 0.081 0.022 0.028
			500	0.026	0.028	0.066 0.069 0.016 0.019
		0.5B(2, 1) + 0.5B(1, 4)	50	0.068	0.067 0.125 0.134	0.108	0.106
			200 0.038	0.042	0.106 0.104	0.090	0.085
			500 0.030	0.033	0.099 0.101	0.084	0.075
		N [0,1] (0, 1)	50	0.064	0.063	0.064 0.081 0.041 0.048
			200	0.038	0.041	0.025 0.033 0.017 0.024
			500	0.029	0.031	0.012 0.018 0.009 0.014
	(b)	B(3, 5)	50	0.036	0.036 0.092 0.095	0.050	0.052
			200 0.014	0.016	0.064 0.063	0.032	0.032
			500 0.009	0.011	0.057 0.057	0.027	0.028
		0.5B(2, 1) + 0.5B(1, 4)	50	0.030	0.030	0.085 0.092	0.042	0.048
			200 0.012	0.013	0.061 0.063	0.029	0.029
			500 0.007	0.008	0.057 0.057	0.024	0.024
		N [0,1] (0, 1)	50	0.051	0.050	0.060 0.078 0.038 0.048
			200	0.020	0.023	0.023 0.033 0.017 0.023
			500	0.013	0.015	0.012 0.018 0.010 0.014
	(c)	B(3, 5)	50	0.067	0.067 0.113 0.107	0.120	0.105
			200 0.041	0.043	0.078 0.079	0.086	0.076
			500 0.036	0.037	0.074 0.070	0.079	0.064
		0.5B(2, 1) + 0.5B(1, 4)	50	0.055	0.054 0.098 0.105	0.094	0.086
			200 0.039	0.039	0.079 0.072	0.068	0.070
			500 0.033	0.034	0.066 0.063	0.065	0.052
		N [0,1] (0, 1)	50	0.082	0.082	0.065 0.085 0.055 0.059
			200	0.051	0.053	0.024 0.034 0.021 0.025
			500	0.039	0.042	0.012 0.017 0.010 0.014

n , Nadaraya-Watson's estimator r N W n

Acknowledgements

The authors would like to thank the Editor, the associate Editor and the referee for their very helpful comments, which led to a considerable improvement of the original version of the paper and a more sharply focused presentation. This work benefited from the financial support of the GDR 3477 GeoSto.

Note that (λ n ) and ( λ n ) belong to GS(-λ * ) with λ * = min 2α-a 4 , a , and (β n ) and ( β n ) belong to GS(-β * ) with β * = min α-a 2 , a . To establish the characteristics of our recursive estimator r n defined by [START_REF] Djojosugito | Boundary bias correction in nonparametric density estimation[END_REF], we state the following technical lemma, which is proved in Mokkadem et al. [START_REF] Mokkadem | Revisiting Révész's stochastic approximation method for the estimation of a regression function[END_REF], and which will be used throughout the demonstrations.

Moreover, for all positive sequence (α n ) such that lim n→∞ α n = 0, and all C,

We first give the asymptotic behavior of (ρ n ) defined in [START_REF] Galambos | Regularly varying sequences[END_REF]. Then, we follow similar steps as Mokkadem et al. [START_REF] Mokkadem | Revisiting Révész's stochastic approximation method for the estimation of a regression function[END_REF] to show how the asymptotic behavior of (r n ) (6) can be deduced from that of (ρ n ).

A.4.1. Asymptotic behavior of ρ n . The following Lemma gives the bias and the variance of the estimator ρ n defined in [START_REF] Duflo | Random Iterative Models[END_REF]. (1) Assume that Assumptions (A1) -(A4) hold. For x ∈ (0, 1), such that f (x) > 0,

we have

(

we have

Moreover, in view of (33), we have

∈ GS(α -a/2), the application of Lemma A.2 together with [START_REF] Slaoui | Optimal bandwidth selection for semi-recursive kernel regression estimators[END_REF], ensures that

The application of Theorem 1 in Mokkadem and Pelletier [START_REF] Mokkadem | Compact law of the iterated logarithm for matrix normalized sums of random vectors[END_REF] then ensures that, with probability one, the sequence

  is relatively compact and its limit set is the interval

In view [START_REF] Slaoui | Optimal bandwidth selection for semi-recursive kernel regression estimators[END_REF], we have lim n→∞ ln ln H -2 n (f (x)) / ln(s n ) = 1. It, follows that, with probability one, the sequence γ -1 n m 1 /2 n (T n (x) -E [T n (x)]) / 2 ln(s n ) is relatively compact and its limit set is the interval given in [START_REF] Slaoui | Recursive density estimators based on Robbins-Monro's scheme and using Bernstein polynomials[END_REF]. The application of (25) concludes the proof of Lemma A.5 in the cases a ≥ 2α/5.

• We consider the case a < 2 5 α (in which lim n→∞ (nγ n ) > a/f (x)). We set

, and note that, since

A.4.6. Proof of Lemma A.6. We have

Var [ρ n (x)] dx.

• In the case when a ∈ ( 1-α 4 , 2 5 α], we use the fact that for all x ∈ (0, 1), lim

we have

• In the case when a ∈ ( 2 5 α, 2 3 α), we have

On the other hand, we note that

Using the same argument as in the proof of Proposition 3.2, we obtain • In the case when a ∈ [ 2 5 α, 2 3 α), since, for all x ∈ (0, 1), lim n→∞ (nγ n ) > (2αa)/(4φ) > (2α -a)/(4f (x)), Lemma A.2 gives