Fadoua Balabdaoui

Jon A Wellner

ESTIMATION OF A K-MONOTONE DENSITY: LIMIT DISTRIBUTION THEORY AND THE SPLINE CONNECTION

Keywords: Primary 62G05, 60G99; secondary asymptotic distribution, completely monotone, convex, Hermite interpolation, inversion, k-fold integral of Brownian motion, least squares, maximimum likelihood, minimax risk, mixture models, multiply monotone, nonparametric estimation, rates of convergence, shape constraints, splines

We study the asymptotic behavior of the Maximum Likelihood and Least Squares Estimators of a k-monotone density g0 at a fixed point x0 when k > 2. We find that the j-th derivative of the estimators at x0 converges at the rate n -(k-j)/(2k+1) for j = 0, . . . , k -1.

The limiting distribution depends on an almost surely uniquely defined stochastic process H k that stays above (below) the k-fold integral of Brownian motion plus a deterministic drift, when k is even (odd). Both the MLE and LSE are known to be splines of degree k -1 with simple knots. Establishing the order of the random gap

n where τ ± n denote two successive knots, is a key ingredient of the proof of the main results. We show that this "gap problem" can be solved if a conjecture about the upper bound on the error in a particular Hermite interpolation via odd-degree splines holds.

Introduction.

1.1. The estimation problem and motivation. A density function g on R + is monotone (or 1-monotone) if it is nonincreasing. It is 2-monotone if it is nonincreasing and convex, and k-monotone for k ≥ 3 if and only if (-1) j g (j) is non-negative, nonincreasing, and convex for j = 0, . . . , k -2.

We write D k for the class of all k-monotone densities on R + , and M k for the class of all k-monotone functions (without the density restriction).

Suppose that g 0 ∈ D k and that X 1 , . . . , X n are i.i.d. with density g 0 . We write G n for the empirical distribution function of X 1 , . . . , X n . Our main interest is in the Maximum Likelihood Estimators (or MLE) ĝn of g 0 ∈ D k .

When k = 1, it is well known that the maximum likelihood estimator ĝn of g 0 ∈ D 1 is the Grenander (1956) estimator; i.e. the left-derivative of the least concave majorant G n of G n , and if g 0 (x 0 ) < 0 with g 0 continuous in a neighborhood of x 0 , then

n 1/3 (ĝ n (x 0 ) -g 0 (x 0 )) → d 1 2 g 0 (x 0 )|g 0 (x 0 )| 1/3 2Z, (1.1)
where 2Z is the slope at zero of the greatest convex minorant of two-sided Brownian motion +t 2 , t ∈ R; see Prakasa [START_REF] Rao | Estimation of a unimodal density[END_REF], [START_REF] Groeneboom | Estimating a monotone density[END_REF], and Kim and Pollard (1990).

When k = 2, Groeneboom, Jongbloed, and Wellner (2001b) considered both the MLE and LSE and established that if the true convex and nonincreasing density g 0 satisfies g 0 (x 0 ) > 0 (and g 0 is continuous in a neighborhood of x 0 ), then n 2/5 (ḡ n (x 0 )g 0 (x 0 ))

n 1/5 (ḡ n (x 0 ) -g (x 0 )) → d 1 24 g 2 0 (x 0 )g 0 (x 0 ) 1/5 H (2) (0)
1 24 3 g 0 (x 0 )g 0 (x 0 ) 3 1/5 H (3) (0)

, (1.2)
where ḡn is either the MLE or LSE and H is a random cubic spline function such that H (2) is convex and H stays above integrated two-sided Brownian motion +t 4 , t ∈ R, and touches exactly at those points where H (2) changes its slope (see Groeneboom, Jongbloed, and Wellner (2001a)).

imsart-aos ver. 2005/05/19 file: tr462R.tex date: January 30, 2007 Our main interest in this paper is in establishing a generalization of the pointwise limit theory given in (1.1) and (1.2) for general k ∈ N, k ≥ 1.

Beyond the obvious motivation of extending the known results for k = 1 and k = 2 as listed above, there are several further reasons for considering such extensions:

(a) Pointwise limit distribution theory for natural nonparametric estimators of the piecewise smooth regression models of smoothness k considered by Mammen (1991) is only available for k ∈ {1, 2}. Similar models (with just one element in the partition) have been proposed for software reliability problems by Miller and Sofer (1986). Similarly, pointwise limit distribution theory is still lacking for the locally adaptive regression spline estimators considered by Mammen and van de Geer (1997).

(b) The classes of densities D k have mixture representations as scale mixtures of Beta(1, k) densities: as is known from Williamson (1956) (see also [START_REF] Lévy | Extensions d'un théorème de D. Dugué et M. Girault[END_REF], [START_REF] Gneiting | Radial positive definite functions generated by Euclid's hat[END_REF], and Balabdaoui and Wellner (2004a)), g ∈ D k if and only if there is a distribution function F on (0, ∞) such that

g(x) = ∞ 0 k y k (y -x) k-1 + dF (y) = ∞ 0 w 1 - wx k k-1 + d F (w) (1.3)
where z + ≡ z1{z ≥ 0} and F = F (k/•). The second form of the mixture representation in the last display makes it clear that the limiting class of densities as k → ∞, namely D ∞ , is the class of scale mixtures of exponential distributions. In view of Feller (1971), pages 232-233, this is just the class of completely monotone densities; see also Widder (1941) and [START_REF] Gneiting | On the Bernstein-Hausdorff-Widder conditions for completely monotone functions[END_REF]. To the best of our knowledge, there is no pointwise limit distribution theory available for the MLE in any class of mixed densities based on a smooth mixing kernel, including this particular case in which the kernel (or mixture density) is the exponential scale family as studied by Jewell (1982). On the other hand, maximum likelihood estimators in various classes of mixture models with smooth kernels have been proposed in a wide range of applications including pharmacokinetics (Mallet (1986), Mallet, Men- tre, Steimer, and Lokiec (1988), and [START_REF] Davidian | Smooth nonparametric maximum likelihood estimation for population pharmacokinetics, with application to Quinidine[END_REF]), demography [START_REF] Vaupel | The impact of heterogeneity in individual frailty on the dynamics of mortality[END_REF]), and shock models and variations in hazard rates (Harris and Singpurwalla (1968), Doyle, Hansen, and McNolty (1980), [START_REF] Hill | Maximum likelihood estimation for mixtures[END_REF]).

(c) The whole family of mixture models D k corresponding to k ∈ (0, ∞) in (1.3) might be of some interest eventually, especially since the family of distributions corresponding to the classical Wicksell problem is contained in the class D 1/2 ; see e.g. [START_REF] Groeneboom | Isotonic estimation and rates of convergence in Wicksell's problem[END_REF].

(d) The sub-class of k-monotone densities with mixing distribution F sat-

isfying g (k-1) (0) = k! ∞ 0 y -k dF (y) < ∞
can be regarded as the distributions arising in a generalization of Hampel's bird watching problem (Hampel (1987)) in which birds are captured k-times, but only one "inter-catch" time is recorded. Based on those observed inter-catch times, the goal is to estimate the true distribution F of the resting times Y of the migrating birds, which we assume to have a density f with k-th moment µ k (f ) < ∞. Furthermore, we assume that the time points of capture form the arrival time points of a Poisson process with rate λ, and given Y = y, the number of captures by time y is Poisson(λy) with λ small enough so that exp(-λy) ≈ 1, and the probability of catching a bird more than k times is negligible (see also Hampel (1987) and [START_REF] Anevski | Estimating the derivative of a convex density[END_REF]). If S k,1 denotes the elapsed time between the first and second captures (the only observed inter-catch time), then it follows by a derivation analogous to Hampel's that the density of the time S k,1 is given by

g(x) = 1 µ k (f ) ∞ 0 k(y -x) k-1 + f (y)dy
which is clearly k-monotone. We obtain F , the probability distribution of Y , by inverting the previous mixture representation; that is

F (t) = 1 - g (k-1) (t) g (k-1) (0+)
at any point of continuity t > 0 of F.

In connection with (a), it is interesting to note that the definition of the family D k is equivalent to g ∈ D k if and only if (-1) k-1 g (k-1) (where g (k-1) imsart-aos ver. 2005/05/19 file: tr462R.tex date: January 30, 2007

is either the left-or right-derivative of g (k-2) ) is nonincreasing. This follows from Lemma 4.3 of Gneiting (1999) since Gneiting's condition lim x→∞ g(x) = 0 is automatic for densities. Thus the equivalent definition of D k has a natural connection with the work of Mammen (1991) in the nonparametric regression setting. In parallel to the treatment of convex regression estimation given by Groeneboom, Jongbloed, and Wellner (2001b), it seems clear that pointwise distribution theory for nonparametric least squares estimators for the regression problems in (a) could be developed if adequate theory were available for the Maximum Likelihood and Least Squares estimators of densities in the class D k , so we focus exclusively on the density case in this paper. In Section 5 we comment further on the difficulties in obtaining corresponding limit theory for the smooth kernel cases discussed in (b).

1.2. Description of the key difficulty: the gap problem. The key result that Groeneboom, Jongbloed, and Wellner (2001b) used to establish

(1.2) is that τ + n -τ - n = O p (n -1/5 ) as n → ∞
, where τ - n and τ + n are two successive jump points of the first derivative of ḡn in the neighborhood of x 0 . Such a result was already proved by Mammen (1991) (see Lemma 8) in the context of nonparametric regression, where the true regression curve, m, is piecewise concave/convex or convex/concave such that m is twice continuously differentiable in the neigborhood of x 0 , and m (x 0 ) = 0. Furthermore, [START_REF] Mammen | Nonparametric regression under qualitative smoothness assumptions[END_REF] conjectured the right form of the asymptotic distribution of his Least square estimator, which was later established by Groeneboom, Jongbloed, and Wellner (2001b).

To obtain the stochastic order n -1/5 for the gap, [START_REF] Groeneboom | Estimation of convex functions: characterizations and asymptotic theory[END_REF] used the characterizations of the estimators together with the "mid-point property" which we review in Section 4. For k = 1, the same property can be used to establish that n -1/3 is the order of the gap. As a function of k, it is natural to conjecture that n -1/(2k+1) is the general form of the order of the gap. In the problem of nonparametric regression via splines, Mammen and van de Geer (1997) have conjectured that n -1/(2k+1) is the order of the distance between the knot points of their regression spline m under the assumption that the true regression curve m 0 satisfies our same working assumptions, but the question was left open (see [START_REF] Mammen | Locally adaptive regression splines[END_REF], page 400). In this manuscript, we refer to the problem of establishing the order of τ + nτ - n as the gap problem. In Section 4, we show that when k > 2, the gap problem is closely related to a "non-classical" Hermite interpolation problem via odd-degree splines.

To put the interpolation problem encountered in the next section in context, it is useful to review briefly the related complete interpolation problem for odd-degree splines which is more "classical" and for which error bounds uniform in the knots are now available. Given a function f ∈ C (k-1) [0, 1] and an increasing sequence 0 = y 0 < y 1 < • • • < y m < y m+1 = 1 where m ≥ 1 is an integer, it is well-known that there exists a unique spline, called the complete spline and denoted here by Cf , of degree 2k -1 with interior knots y 1 , . . . , y m that satisfies the 2k + m conditions

(Cf )(y i ) = f (y i ), i = 1, . . . , m (Cf ) (l) (y 0 ) = f (l) (y 0 ), (Cf ) (l) (y m+1 ) = f (l) (y m+1 ), l = 0, . . . , k -1;
see Schoenberg (1963), de Boor (1974), or [START_REF] Nürnberger | Approximation by Spline Functions[END_REF], page 116, for further discussion. If j ∈ {0, . . . , k} and f ∈ C (k+j) [0, 1], then there exists

c k,j > 0 such that sup 0<y 1 <•••<ym<1 f -Cf ∞ ≤ c k,j f (k+j) ∞ . (1.4)
For j = k, this "uniform in knots" bound in the complete interpolation problem was first conjectured by de Boor (1973) for k > 4 as a generalization that goes beyond k = 2, 3 and 4 for which the result was already established (see also de Boor (1974)). By a scaling argument, the bound

(1.4) implies that, if f ∈ C (2k) [a, b], a < b ∈ R, the interpolation error in
the complete interpolation problem is uniformly bounded in the knots, and that the bound is of the order of (ba) 2k . One key property of the complete spline interpolant Cf is that (Cf ) (k) is the Least Squares approximation of imsart-aos ver. 2005/05/19 file: tr462R.tex date: January 30, 2007 f (k) when f (k) ∈ L 2 ([0, 1]); i.e., if S k (y 1 , . . . , y m ) denotes the space of splines of order k (degree k -1) and interior knots y 1 , . . . , y m , then

1 0 (Cf ) (k) -f (k) (x) 2 dx = min S∈S k (y 1 ,...,ym) 1 0 S(x) -f (k) (x) 2 dx (1.5)
(see e.g. [START_REF] Schoenberg | On interpolation by spline functions and its minimal property[END_REF][START_REF] De Boor | Bounding the error in spline interpolation[END_REF], [START_REF] Nürnberger | Approximation by Spline Functions[END_REF]). Consequently, if L ∞ denotes the space of bounded functions on [0, 1], then the properly defined map

C (k) [0, 1] → S k (y) f (k) → (Cf ) (k)
where y = (y 1 , . . . , y m ), is the restriction of the orthoprojector P S k (y) from L ∞ to S k (y) with respect to the inner product g, h = 1 0 g(x)h(x)dx which assigns to a function g ∈ L ∞ the k-derivative of the complete spline interpolant of any primitive of g of order k (note that the difference between two primitives of g of order k is a polynomial of degree k -1).

de [START_REF] De Boor | Bounding the error in spline interpolation[END_REF] pointed out that, in order to prove the conjecture, it is enough to prove that

sup y P S k (y) ∞ = sup y sup g∈L∞ P S k (y) (g) ∞ g ∞
is bounded, and this was successfully achieved by [START_REF] Shadrin | The L ∞ -norm of the L 2 -spline projector is bounded independently of the knot sequence: A proof of de Boor's conjecture[END_REF].

The Hermite interpolation problem which arises naturally in Section 4 appears to be another variant of interpolation problems via odd-degree splines which has not yet been studied in the approximation theory or spline literature. More specifically, if f is some real-valued function in C (j) [0, 1] for some j ≥ 1, 0 = y 0 < y 1 < • • • < y 2k-4 < y 2k-3 = 1 is a given increasing sequence, then there exists a unique spline H k f of degree 2k -1 and interior knots y 1 , . . . , y 2k-4 satisfying the 4k -4 conditions

(H k f )(y i ) = f (y i ), and (H k f ) (y i ) = f (y i ), i = 0, . . . , 2k -3. (1.6)
It turns out that deriving the stochastic order of the distance between two successive knots of the MLE and LSE in the neighborhood of the point of estimation is very closely linked to bounding the error in this new Hermite interpolation independently of the locations of the knots of the spline interpolant. More precisely, if g t (x) = (xt) k-1 + /(k -1)! is the power truncated function of degree k -1 with unique knot t, then we conjecture that there is a constant d k > 0 such that sup t∈(0,1)

sup 0<y 1 <•••<y 2k-4 <1 g t -H k g t ∞ ≤ d k . (1.7)
As shown in Balabdaoui and Wellner (2005), the preceding formulation implies that boundedness of the error independently of the knots of the spline interpolant holds true for any f ∈ C (k+j) , that is sup

0<y 1 <•••<y 2k-4 <1 f -H k f ∞ ≤ d k,j f (k+j)
∞ .

If j = k and f (2k) ∞ ≤ 1, it follows from Proposition 1 of [START_REF] Balabdaoui | Conjectures of error boundedness in a new Hermite interpolation problem via splines of odd-degree splines[END_REF] that the interpolation error must be bounded above by the error for interpolating the perfect spline

S * (t) = 1 (2k)! t 2k + 2 2k-4 i=1 (-1) i (t -τ j ) 2k + .
For a definition of perfect splines, see e.g. Bojanov, Hakopian and Sahakian (1993), Chapter 6. Based on large number of simulations, we found that sup

0<y 1 <•••<y 2k-4 <1 S * -H k S * ∞ ≤ 2 (2k)!
for fairly large values of k (see the last column in Table 2 in Balabdaoui and Wellner (2005)). The latter strongly suggests that for f ∈ C (2k) [0, 1] we have sup

0<y 1 <•••<y 2k-4 <1 f -H k f ∞ ≤ 2 (2k)! f (2k) ∞ . (1.8)
Based on Conjecture (1.7), we will prove that the distance between two consecutive knots in a neighborhood of x 0 is O p (n -1/(2k+1) ).

After a brief introduction of the MLE and LSE and their respective characterizations, we give in Section 3 a statement of our main result which imsart-aos ver. 2005/05/19 file: tr462R.tex date: January 30, 2007

gives the joint asymptotic distribution of the successive derivatives of the MLE and LSE. The obtained convergence rate n -(k-j)/(2k+1) for the j-th derivative of any of the estimators was found by Balabdaoui and Wellner (2004a) to be the asymptotic minimax lower bound for estimating g (j) 0 (x 0 ), j = 0, . . . , k -1 under the same working assumptions. The limiting distribution depends on the higher derivatives of H k , an almost surely uniquely defined process that stays above (below) the (k -1)-fold integral of Brownian motion plus the drift (k!/(2k)!) t 2k , when k is even (odd), and whose derivative of order 2k -2 is convex (H k is also said to be (2k -2)-convex).

The process H k is studied separately in Balabdaoui and Wellner (2004c).

Proving the existence of H k relies also on our conjecture in (1.7) since the key problem, also referred to as the gap problem, depends on a very similar Hermite interpolation problem, except that the knots of the estimators are replaced by the points of touch between the (k -1)-fold integral of Brownian motion plus the drift (k!/(2k)!) t 2k and H k . For more discussion of the background and related problems, see Balabdaoui and Wellner (2004a).

For a discussion of algorithms and computational issues, see Balabdaoui and Wellner (2004b).

The estimators and their characterization.

Let X 1 , • • • , X n be n independent observations from a common k-monotone density g 0 . We consider nonparametric estimation of g 0 via the Least Squares and Maximum Likelihood methods, and that of its mixture distribution F 0 , that is the distribution function on (0, ∞) such that

g 0 (x) = ∞ 0 k(t -x) k-1 + t k dF 0 (t), x > 0.
In other words, g 0 is a scale mixture of Beta(1, k) densities. The mixing distribution is furthermore given at any point of continuity t by the inversion formula where G 0 (t) = t 0 g 0 (x)dx. An estimator for F 0 can be obtained by simply plugging in estimators of G (j) 0 = g (j-1) 0

F 0 (t) = k j=0 (-1) j t j j! G (j) 0 (t) (2.9)
, j = 0, . . . , k, in the inversion formula (2.9). We call estimation of the (mixed) k-monotone density g 0 the direct problem, and estimation of the mixing distribution function F 0 the inverse problem. For more technical details on the mixture representation and the inversion formula, see Lemma 2.1 of Balabdaoui and Wellner (2004a). Now, we give the definition of the Least Squares and Maximum Likelihood estimators; these were already considered in the case k = 2 by [START_REF] Groeneboom | Estimation of convex functions: characterizations and asymptotic theory[END_REF]. The LSE, gn , is the minimizer of the criterion function

Φ n (g) = 1 2 ∞ 0 g 2 (t)dt - ∞ 0 g(t)dG n (t)
over the class M k , whereas the MLE, ĝn , maximizes the "adjusted" loglikelihood function, i.e.

l n (g) = ∞ 0 log g(t)dG n (t) - ∞ 0 g(t)dt
over the same class. In Balabdaoui and Wellner (2004a), we find that both estimators exist and are splines of degree k -1, i.e., their (k -1)-st derivative is stepwise. Furthermore, as shown in Balabdaoui and Wellner (2004a), the LSE's and MLE's are characterized as follows: let Hn and Y n be the processes defined for all x ≥ 0 by

Y n (x) = x 0 t k-1 0 • • • t 2 0 G n (t 1 )dt 1 dt 2 . . . dt k-1 (2.10) = x 0 (x -t) k-1 (k -1)! dG n (t), and
Hn (x) = x 0 t k 0 • • • t 2 0 gn (t 1 )dt 1 dt 2 . . . dt k (2.11) = x 0 (x -t) k-1 (k -1)! gn (t)dt.
imsart-aos ver. 2005/05/19 file: tr462R.tex date: January 30, 2007

Then the k-monotone function gn is the LSE if and only if

Hn (x)        ≥ Y n (x), for all x ≥ 0 = Y n (x), if (-1) k-1 g(k-1) n (x-) < (-1) k-1 g(k-1) n (x+).
(2.12)

For the MLE we define the process

H n (x, g) = x 0 k(x -t) k-1 x k ĝn (t) dG n (t) (2.13)
for all x ≥ 0 and g ∈ D k . Then, a necessary and sufficient condition for the k-monotone function ĝn to be the MLE is given by

H n (x, ĝn )        ≤ 1, for all x ≥ 0 = 1, if (-1) k-1 ĝ(k-1) n (x-) < (-1) k-1 ĝ(k-1) n (x+).
(2.14)

These characterizations are crucial for understanding the local asymptotic behavior of the LSE and MLE. They were exploited in Balabdaoui and Wellner (2004a) to show uniform strong consistency of the estimators on intervals of the form [c, ∞), c > 0. Here, they prove to be once again very useful for establishing their limit theory in both the direct and inverse problems.

The asymptotic distribution.

3.1. The main convergence theorem. To prepare for a statement of the main result, we first recall the following theorem from Balabdaoui and Wellner (2004c) giving existence of the processes H k .

Theorem 3.1 For all k ≥ 1, let Y k denote the stochastic process defined by

Y k (t) =    t 0 (t-s) k-1 (k-1)! dW (s) + (-1) k k! (2k)! t 2k , t ≥ 0 0 t (t-s) k-1 (k-1)! dW (s) + (-1) k k! (2k)! t 2k , t < 0.
If Conjecture (1.7) holds (also see the discussion in Balabdaoui and Well- ner (2004c)), then there exists an almost surely uniquely defined stochastic process H k characterized by the following four conditions:

(i) The process H k stays everywhere above the process Y k :

H k (t) ≥ Y k (t), t ∈ R. (ii) (-1) k H k is 2k-convex; i.e. (-1) k H (2k-2) k
exists and is convex.

(iii) The process H k satisfies ∞ -∞ (H k (t) -Y k (t)) dH (2k-1) k (t) = 0. (iv) If k is even, lim |t|→∞ (H (2j) k (t) -Y (2j) k (t)) = 0 for j = 0, . . . , (k -2)/2; if k is odd, lim t→∞ (H k (t) -Y k (t)) = 0 and lim |t|→∞ (H (2j+1) k (t) - Y (2j+1) k (t)) = 0 for j = 0, . . . , (k -3)/2.
Now we are able to state the main result of this paper which generalizes Theorem 6.2 of Groeneboom, Jongbloed, and Wellner (2001b) for estimating convex (2-monotone) densities:

Theorem 3.2 Let x 0 > 0 and g 0 be a k-monotone density such that g 0 is k-times differentiable at x 0 with (-1) k g (k)
0 (x 0 ) > 0 and assume that g

      n k 2k+1 (ḡ n (x 0 ) -g 0 (x 0 )) n k-1 2k+1 (ḡ (1) n (x 0 ) -g (1) 0 (x 0 )) . . . n 1 2k+1 (ḡ (k-1) n (x 0 ) -g (k-1) 0 (x 0 ))        → d        c 0 (x 0 )H (k) k (0) c 1 (x 0 )H (k+1) k (0) . . . c k-1 (x 0 )H (2k-1) k (0)        and n 1 2k+1 ( Fn (x 0 ) -F (x 0 )) → d (-1) k x k 0 k! c k-1 (x 0 )H (2k-1) k (0)
where

c j (x 0 ) = (g 0 (x 0 )) k-j (-1) k g (k) 0 (x 0 ) k! 2j+1 1 2k+1 , for j = 0, . . . , k -1.
g 0 ∈ D k satisfies g (k) 0 (x 0 ) = 0 and Conjecture (1.7) holds, then τ 2k-3 -τ 0 = O p (n -1/(2k+1) ) where τ 0 < • • • < τ 2k-3 are 2k -2 successive jump points of ḡ(k-1) n in a neighborhood of x 0 .
Using Lemma 3.1 we can establish the rate(s) of convergence of the estimators gn and ĝn and their derivatives viewed as local processes in n -1/(2k+1) neighborhoods of the fixed point x 0 . This is accomplished in Proposition 3.1.

Y a,σ (t) =    σ t 0 s k-1 0 • • • s 2 0 W (s 1 )ds 1 . . . ds k-1 + a(-1) k k! (2k)! t 2k , t ≥ 0 σ 0 t 0 s k-1 • • • 0 s 2 W (s 1 )ds 1 . . . ds k-1 + a(-1) k k! (2k)! t 2k , t ≤ 0 with σ = g(x 0 ), a = (-1) k g (k)
0 (x 0 )/k!, and W is a two-sided Brownian motion process starting from 0.

• The localized processes Hloc

n and H loc n satisfy Fenchel (inequality and equality) relations relative to the localized processes Y loc n and Y loc n respectively.

• We then show via tightness that the localized processes Hloc n and H loc n (and all their derivatives up to order 2k -1) converge to a limit process satisfying the conditions (i) -(iv) of Theorem 3.1, and hence the limit process in both cases is just H k (up to scaling by constants). When specialized to t = 0 this gives the conclusion of Theorem 3.2.

Here is the key rates of convergence proposition.

Proposition 3.1 Fix x 0 > 0 and let g 0 be a k-monotone density such that (-1) k g (k)

0 (x 0 ) > 0. Let ḡn denote either the MLE ĝn or the LSE gn . If Conjecture (1.7) holds, then for each M > 0 we have,

sup |t|≤M ḡ(j) n (x 0 + n -1/(2k+1) t) - k-1 i=j n -(i-j)/(2k+1) g (i) 0 (x 0 ) (i -j)! t i-j = O p (n -(k-j)/(2k+1) ) for j = 0, . . . , k -1. (3.1)
For the LSE, we define the local Y n and Hn -processes respectively by

Y loc n (t) = n 2k 2k+1 x 0 +tn -1/(2k+1) x 0 v k-1 x 0 • • • v 2 x 0 G n (v 1 ) -G n (x 0 ) - v 1 x 0 k-1 j=0 (u -x 0 ) j j! g (j) 0 (x 0 )du Π k-1 i=1 dv i ,
and

Hloc n (t) = n 2k 2k+1 x 0 +tn -1/(2k+1) x 0 v k x 0 • • • v 2 x 0 gn (v 1 ) - k-1 j=0 (v 1 -x 0 ) j j! g (j) 0 (x 0 ) dv 1 . . . dv k + Ãk-1,n t k-1 + Ãk-2,n t k-2 + • • • + Ã1,n t + Ã0,n ,
where

Ãj,n = n (2k-j)/(2k+1) j! H(j) n (x 0 ) -Y (j) n (x 0 ) , j = 0, . . . , k -1.
Let r k ≡ 1/(2k + 1). In the case of the MLE, the local processes Y loc n and H loc n are defined as and

Y loc n (t) g 0 (x 0 ) = n 2kr k x 0 +tn -r k x 0 v k-1 x 0 • • • v 1 x 0 g 0 (v) -k-1 j=0 (v-x 0 ) j j! g (j) 0 (x 0 ) ĝn (v) dvdv 1 . . . dv k-1 + n 2kr k x 0 +tn -r k x 0 v k-1 x 0 • • • v 1 x 0 1 ĝn (v) d(G n -G 0 )(v) dv 1 . . . dv k-1 imsart-
H loc n (t) g 0 (x 0 ) = n 2kr k x 0 +tn -r k x 0 v k-1 x 0 • • • v 1 x 0 ĝn (v) -k-1 j=0 (v-x 0 ) j j! g (j) 0 (x 0 ) ĝn (v) dvdv 1 . . . dv k-1 + A k-1,n t k-1 + • • • + A 0,n where A j,n = - n (2k-j)r k (k -1)!j! g 0 (x 0 ) H (j) n (x 0 ) - (k -1)! (k -j)! x k-j 0 , j = 0, . . . , k -1.
In the following lemma, we will give the asymptotic distribution of the local process Y loc n and Y loc n in terms of the (k -1)-fold integral of two-sided Brownian motion, g 0 (x 0 ), and g (k) 0 (x 0 ) assuming that the true density g 0 is k-times continuously differentiable at x 0 . We denote by Ȳloc n either Y loc n or Y loc n .

Lemma 3.2 Let x 0 be a point where g 0 is continuously k-times differentiable in a neighborhood of x 0 with (-1) k g

(k) 0 (x 0 ) > 0. Then Ȳloc n ⇒ Y a,σ in C[-K, K] for each K > 0 where Y a,σ (t) = σ t 0 s k-1 0 • • • s 2 0 W (s 1 )ds 1 . . . ds k-1 + a(-1) k k! 2k! t 2k , t ≥ 0 σ 0 t 0 s k-1 • • • 0 s 2 W (s 1 )ds 1 . . . ds k-1 + a(-1) k k! 2k! t 2k , t < 0
where W is standard two-sided Brownian motion starting at 0, σ = g 0 (x 0 ),

and a = (-1) k g (k) 0 (x 0 )/k!. Now,
Y a,σ (t) d = 1 s 1 Y 1,1 t s 2
where

s 1 = 1 g 0 (x 0 ) (-1) k g (k) 0 (x 0 ) k! g 0 (x 0 ) (2k-1)/(2k+1) (3.2) s 2 = g 0 (x 0 ) (-1) k g (k) 0 (x 0 ) k! 2/(2k+1) . (3.3)
To show that the derivatives of Hloc n are tight, we need the following lemma. Hl

n (t) = s 1 Hloc n (s 2 t), Ȳl n (t) = s 1 Ȳloc n (s 2 t)
where s 1 and s 2 are given by (3.2) and (3.3) respectively.

Lemma 3.6 Let c > 0. Then

(( Hl n ) (0) , ( Hl n ) (1) , . . . , ( Hl n ) (2k-1) ) ⇒ (H (0) k , H (1) k , . . . , H (2k-1) k ) in (D[-c, c]) 2k
where H k is the stochastic process defined in Theorem 3.1. To keep the manuscript to a reasonable length, proofs of the results of Subsection 3.2 and of the main convergence Theorem 3.2 can be found in [START_REF] Balabdaoui | Estimation of a k-monotone density: limit distribution theory and the spline connection, with complete proofs[END_REF], Appendix 1. The arguments there are constructed along the lines of Groeneboom, Jongbloed, and Wellner (2001b). However, those arguments had to be adapted and further developed to be able to treat k-monotonicity for an arbitrary integer k ≥ 2. In this general case, we found that it is very useful to consider perturbation functions to learn about the asymptotic behavior of the estimators. Such are knots of either the LSE or MLE in the neighborhood of x 0 are found to be the most sensible perturbation functions to consider. For a definition of B-splines, see e.g. Nürnberger (1989), Theorem 2.2. For technical details on the use of B-splines for constructing perturbations, see e.g. Proposition 6.1 in Balabdaoui and Wellner (2006), Appendix 1.

The gap problem -Spline connection.

Recall that it was assumed that g 0 is k-times continuously differentiable at x 0 and that (-1) k g (k) 0 (x 0 ) > 0. Under a weaker assumption, Balabdaoui and Wellner (2004a) proved strong consistency of the (k -1)-st derivative of the MLE and LSE. This consistency result together with the above assumptions imply that the number of jump points of this derivative, in a small neighborhood of x 0 , diverges to infinity almost surely as the sample size n → ∞. This "clustering" phenomenon is one of the most crucial elements in studying the local asymptotics of the estimators. The jump points form then a sequence that converges to x 0 almost surely and therefore the distance between two successive jump points, for example located just before and after x 0 , converges to 0 as n → ∞. But it is not enough to know that the "gap" between these points converges to 0: an upper bound for this rate of convergence is needed.

To prove Lemma 3.1, we will focus first on the LSE because it is somewhat easier to handle through the simple form of its characterization. The arguments for the MLE could be built upon those used for the LSE, but in this case, one has to deal with some extra difficulties due to the non-linear nature of its characterization.

We start first by describing the difficulties of establishing this result for the general case k > 2.

4.1. Fundamental differences. Let τ - n and τ + n be the last and first jump points of the (k -1)-st derivative of the LSE gn , located before and after x 0 respectively. To obtain a better understanding of the gap problem, we describe the reasoning used by Groeneboom, Jongbloed, and Wellner (2001b) in order to prove that τ + nτ - n = O p (n -1/5 ) for the special case k = 2. The characterization of the estimator is given by Hn

(x) ≥ Y n (x), x ≥ 0 = Y n (x), if x is a jump point of g n (4.1)
where Hn (x) = x 0 (xt)g n (t)dt and Y n (x) = x 0 G n (t)dt. On the interval [τ - n , τ + n ), the function g n is constant since there are no more jump points in this interval. This implies that Hn is polynomial of degree 3 on [τ - n , τ + n ). But, from the characterization in (4.1), it follows that

Hn (τ ± n ) = Y n (τ ± n ), H n (τ ± n ) = Y n (τ ± n ) .
These four boundary conditions allow us to fully determine the cubic polynomial Hn on [τ - n , τ + n ]. Using the explicit expression for Hn and evaluating it at the mid-point τ = (τ - n + τ + n )/2, Groeneboom, Jongbloed, and Wellner (2001b) established that

Hn (τ n ) = Y n (τ - n ) + Y n (τ + n ) 2 - G n (τ + n ) -G n (τ - n ) 8 (τ + n -τ - n ).
imsart-aos ver. 2005/05/19 file: tr462R.tex date: January 30, 2007 [START_REF] Groeneboom | Estimation of convex functions: characterizations and asymptotic theory[END_REF] refer to this as the "midpoint property". By applying the first condition (the inequality condition) in (4.1), it follows that

Y n (τ - n ) + Y n (τ + n ) 2 - G n (τ + n ) -G n (τ - n ) 8 (τ + n -τ - n ) ≥ Y n (τ n ).
The inequality in the last display can be rewritten as

Y 0 (τ - n ) + Y 0 (τ + n ) 2 - G 0 (τ + n ) -G 0 (τ - n ) 8 (τ + n -τ - n ) ≥ E n
where G 0 and Y 0 are the true counterparts of G n and Y n respectively, and E n is a random error. Using empirical process theory, [START_REF] Groeneboom | Estimation of convex functions: characterizations and asymptotic theory[END_REF] showed that

|E n | = O p (n -4/5 ) + o p ((τ + n -τ - n ) 4 ). (4.2)
On the other hand, Groeneboom, Jongbloed, and Wellner (2001b) established that there exists a universal constant C > 0 such that

Y 0 (τ - n ) + Y 0 (τ + n ) 2 - G 0 (τ + n ) -G 0 (τ - n ) 8 (τ + n -τ - n ) = -Cg 0 (x 0 )(τ + n -τ - n ) 4 + o p ((τ + n -τ - n ) 4 ). (4.3)
Combining the results in (4.2) and (4.3), it follows that

τ + n -τ - n = O p (n -1/5 ).
The problem has two main features that make the above arguments work.

First of all, the polynomial Hn can be fully determined on [τ - n , τ + n ] and therefore it can be evaluated at any point between τ - n and τ + n . Second of all, it can be expressed via the empirical process Y n and that enables us to "get rid of" terms depending on gn whose rate of convergence is still unknown at this stage. We should also add that the problem is symmetric around τn , a property that helps establishing the formula derived in (4.3).

When k > 2, it follows from the characterization of the LSE given in (2.12), that for any two successive jump points of g(k-1)

n , τ - n , τ + n , the four equalities Hn (τ ± n ) = Y n (τ ± n ), and H n (τ ± n ) = Y n (τ ± n )
still hold. However, these equations are not enough to determine the polynomial Hn , now of degree 2k -1, on the interval [τ - n , τ + n ]. One would need 2k conditions to be able to achieve this. [We would be in this situation if we had equality of the higher derivatives of Hn and Y n at τ - n and τ + n , that is

H(j) n (τ - n ) = Y (j) n (τ - n ), H(j) n (τ + n ) = Y (j) n (τ + n ) (4.4)
for j = 0, . . . , k -1, but the characterization (2.12) does not give this much.] Thus it becomes clear that two jump points are not sufficient to determine the piecewise polynomial Hn . However, if we consider p > 2 jump points τ n,0 < • • • < τ n,p-1 (all located e.g. after x 0 ), then Hn is a spline of degree 2k-1 with interior knots τ n,1 , . . . , τ n,p-2 ; that is, Hn is a polynomial of degree 2k -1 on (τ n,j , τ n,j+1 ) for j = 0, . . . , p -2 and is (2k -2)-times differentiable at its knot points τ n,0 , . . . , τ n,p-1 . In the next subsection, we prove that if p = 2k -2, the spline Hn is completely determined on [τ n,0 , τ n,2k-3 ] by the conditions

Hn (τ n,i ) = Y n (τ n,i ), and H n (τ n,i ) = Y n (τ n,i ), i= 0, . . . , 2k -3 . (4.5)
This result proves to be very useful for determining the stochastic order of the distance between two successive jump points in a small neighborhood of x 0 if our Conjecture (1.7) on the uniform boundedness of the error in the "non-classical" Hermite interpolation problem via splines of odd-degree defined in (1.6) holds.

The gap problem for the LSE -Hermite interpolation.

In the next lemma, we prove that given 2k-2 successive jump points

τ n,0 < • • • < τ n,2k-3 of g(k-1) n
, Hn is the unique solution of the Hermite problem given by (4.5).

In the following, we will omit writing the subscript n explicitly in the knots, but their dependence on the sample size should be kept in mind.

• • < τ 2k-3 , the (2k -1)-th spline Hn is uniquely determined on [τ 0 , τ 2k-3 ]
by the values of the process Y n and of its derivative Y n at τ 0 , . . . , τ 2k-3 .

Proof. We know that for any jump point τ of H(2k-1) n , we have

Hn (τ ) = Y n (τ ) and H n (τ ) = Y n (τ ).
This can be viewed as a Hermite interpolation problem if we consider that the interpolated function is the process Y n and that the interpolating spline is Hn (see e.g. [START_REF] Nürnberger | Approximation by Spline Functions[END_REF], Definition 3.6, pages 108 and 109).

Existence and uniqueness of the spline interpolant follows easily from the Schoenberg-Whitney-Karlin-Ziegler Theorem (Schoenberg and Whitney (1953); Theorem 3, page 529, [START_REF] Karlin | Chebyshevian spline functions[END_REF]; or see Theorem 3.7, page 109, Nürnberger (1989); or Theorem 9.2, page 162, DeVore and Lorentz ( 1993)).

In the following lemma, we prove a preparatory result that will be used later for deriving the stochastic order of the distance between successive knots, τ 0 , .

(t) = t 2k (2k)! -H k x 2k (2k)! (t) then g (k) 0 (τ )e k (τ ) ≤ E n + R n (4.6)
where E n defined in (4.8) is a random error and R n defined in (4.9) is a remainder that both depend on the knots τ 0 , . . . , τ 2k-3 and the point τ .

Proof. Let τ ∈ ∪ 2k-4 i=0 (τ i , τ i+1 ). From the characterization in (2.12) and the fact that Hn

= H k [Y n ] on [τ 0 , τ 2k-3 ], it follows that H k [Y n ](τ ) ≥ Y n (τ ).
Let Y 0 be the true counterpart of Y n ; i.e., Y 0 (x) = x 0 (xt) k-1 g 0 (t)dt/(k -1)!. Then, we can rewrite the previous inequality as

H k [Y 0 ](τ ) -Y 0 (τ ) ≥ -E n (τ ) (4.7)
where

E n = H k [Y n -Y 0 ](τ ) -[Y n -Y 0 ](τ ). (4.8)
Based on the working assumptions, the function Y 0 is (2k)-times continuously differentiable in a small neighborhood of x 0 . Now, Taylor expansion of Y 0 (t), with integral remainder, around τ up to the order 2k yields

Y 0 (t) = 2k-1 j=0 (t -τ ) j j! Y (j) 0 (τ ) + τ 2k-3 τ (t -u) 2k-1 + (2k -1)! g (k) 0 (u)du for all t ∈ [τ 0 , τ 2k-3 ].
Using this expansion along with the fact that the operator H k is linear and does preserve polynomials of degree 2k -1, we can rewrite the inequality in (4.7) as

1 (2k -1)! τ 2k-3 τ H k [(t -u) 2k-1 + ](τ ) g (k) 0 (u)du ≥ -E n .
In the previous display, H k [(tu) 2k-1 + ](τ ) is the Hermite spline interpolant of the truncated power function t → (tu) 2k-1 + (u is fixed), evaluated at the point τ . Now, we can rewrite the left side of the previous inequality as

τ 2k-3 τ 1 (2k -1)! H k [(t -u) 2k-1 + ](τ ) g (k) 0 (u)du = g (k) 0 (τ ) 1 (2k -1)! τ 2k-3 τ H k [(t -u) 2k-1 + ](τ )du + 1 (2k -1)! τ 2k-3 τ H k [(t -u) 2k-1 + ](τ ) g (k) 0 (u) -g (k) 0 (τ ) du = g (k) 0 (τ ) 1 (2k -1)! H k τ 2k-3 τ [(t -u) 2k-1 + ]du (τ ) + R n , (4.9)
imsart-aos ver. 2005/05/19 file: tr462R.tex date: January 30, 2007 using once again linearity of the operator H k . The remainder R n is equal to the Hermite interpolant of the function

t → 1 (2k -1)! t τ (t -u) 2k-1 (2k -1)! (g (k) 0 (u) -g (k) 0 (τ ))du
at the point τ . On the other hand, we can further rewrite the integral term in (4.9) as

1 (2k -1)! H k τ 2k-3 τ (t -u) 2k-1 + du (τ ) = 1 (2k -1)! H k t τ (t -u) 2k-1 du (τ ) = 1 (2k)! H k (t -τ ) 2k (τ ).
In other words, the integral term in (4.9) is nothing but the value of the Hermite spline interpolant of the function t → (tτ ) 2k /(2k)! at the point τ . As claimed in the lemma, this value is also equal to -e k (τ ), where e k the error of the Hermite interpolation of the function x 2k /(2k)!. Indeed, let

P 2k-1 (t) = (t -τ ) 2k /(2k)! -t 2k /(2k)!. Since P 2k-1 is a polynomial of degree 2k -1, we have H k (x -τ ) 2k (2k)! (t) = H k x 2k (2k)! (t) + P 2k-1 (t). If t = τ , P 2k-1 (τ ) = 0 -τ 2k /(2k)! = -τ 2k /(2k)!, which implies that H k (x -τ ) 2k (2k)! (τ ) = H k x 2k (2k)! (τ ) - τ 2k (2k)! = -e k (τ ).
The error e k defined in Lemma 4.2 can be recognized as a monospline of degree 2k with 2k -2 simple knots τ 0 , . . . , τ 2k-3 . For a definition of monosplines, see e.g. Michelli (1972), Bojanov, Hakopian and Sahakian (1993), Nürnberger (1989), page 194 or DeVore and Lorentz (1993), page 136.

In the next lemma, we state an important property of e k .

Lemma 4.3 The function x → e k (x) has no other zeros than τ 0 , . . . , τ 2k-3

in [τ 0 , τ 2k-3 ]. Furthermore, (-1) k e k ≥ 0 on [τ 0 , τ 2k-3 ].
Proof. See [START_REF] Balabdaoui | Estimation of a k-monotone density: limit distribution theory and the spline connection, with complete proofs[END_REF], Appendix 3.

In Lemma 4.2, the key inequality in (4.6) can be rewritten as

(-1) k g (k) 0 (τ ) • (-1) k e k (τ ) ≤ E n + R n , (4.10)
where the first factor on the right side is already known to be positive by k-monotonicity of g 0 . Lemma 4.4 and Lemma 4.5 are the final steps toward establishing the order of the gap for the LSE based on the Conjecture (1.7).

Lemma 4.4 If Conjecture (1.7) holds, then E n in (4.6) of Lemma 4.2 sat- isfies |E n | = O p (n -k/(2k+1) ) + o p ((τ 2k-3 -τ 0 ) 2k ).
Proof. We have

E n = H k [Y n -Y 0 ](τ ) -[Y n -Y 0 ](τ ).
Using (generalized) Taylor expansions of Y n and Y 0 around the point τ up to the order k -1 yields

Y n (t) -Y 0 (t) = k-1 j=0 (t -τ ) j j! [Y (j) n (τ ) -Y (j) 0 (τ )] + t τ (t -x) k-1 (k -1)! d(G n -G 0 )(x),
and therefore,

E n = H k t τ 1 (k -1)! (t -x) k-1 d(G n -G 0 )(x) (τ ) = H k τ 2k-3 τ g t (x)d(G n -G 0 )(x) (τ ), where g t (x) = (t-x) k-1 + (k-1)! = τ 2k-3 τ H k [g t (x)](τ )d(G n -G 0 )(x), by linearity of H k = τ 2k-3 τ 0 f τ (x)d(G n -G 0 )(x).
imsart-aos ver. 2005/05/19 file: tr462R.tex date: January 30, 2007

Given x ∈ [τ , τ 2k-3 ], f τ (x) = H k [g t (x)](τ )1 [τ ,τ 2k-3 ] (x), where H k [g t (x)](τ ) is
the value at τ of the Hermite spline interpolant of the function t → g t (x) = (tx) k-1 + /(k -1)!. Thus, f τ (x) depends on the knots τ 0 , . . . , τ 2k-3 , and the point s = τ ∈ [τ 0 , τ 2k-3 ], and can be viewed as an element of the class of functions

F y 0 ,R = {f s (x) = f s,y 0 ,...,y 2k-3 (x) : x ∈ [y 0 , y 2k-3 ], s ∈ [y 0 , y 2k-3 ], x 0 -δ ≤ y 0 < y 1 < • • • < y 2k-3 ≤ y 0 + R ≤ x 0 + δ}, (4.11)
where δ > 0 is a fixed small number. In view of Conjecture (1.7) together with the triangle inequality, there exists a constant C > 0 depending only on k such that

|f s (x)| ≤ C(y 2k-3 -y 0 ) k-1 1 [y 0 ,y 2k-3 ] (x)
and hence the collection F y 0 ,R has envelope function F y 0 ,R given by

F y 0 ,R (x) = CR k-1 1 [y 0 ,y 0 +R] (x).
Furthermore, F y 0 ,R is a VC-subgraph collection of functions (see Proposition 6.1 in Appendix, for a detailed argument), and hence by van der Vaart and Wellner (1996), Theorem 2.6.7, page 141

sup Q N ( F Q,2 , F y 0 ,R , L 2 (Q)) ≤ K V k for 0 < < 1 where V k = 2(V (F y 0 ,R ) -1) with V (F y 0 ,R ) the VC-dimension
of the collection of subgraphs and the constant K depends only on V (F y 0 ,R ) (note that from our proof of Proposition 6.1, it is clear that V (F y 0 ,R ) depends only of k). It follows that sup

Q 1 0 1 + log N ( F y 0 ,R Q,2 , F y 0 ,R , L 2 (Q))d < ∞.
On the other hand, with M ≡ g 0 (x 0δ). Application of Lemma 6.1 with d = k yields -2k/(2k+1) ).

EF 2 y 0 ,R (X 1 ) = C 2 R 2(k-1) y 0 +R y 0 g 0 (x)dx ≤ C 2 MR 2k-
|E n | = o p ((τ 2k-3 -τ 0 ) 2k ) + O p (n
Lemma 4.5 If the bound in (1.8) holds, then R n of Lemma 4.2 satisfies

|R n | = o p ((τ 2k-3 -τ 0 ) 2k ).
Proof. By definition, R n is the value at τ of the Hermite spline interpolant of the function

t → t τ (t -u) 2k-1 (2k -1)! (g (k) 0 (u) -g (k)
0 (τ ))du (4.12) By (1.8), there exists a constant D > 0 depending only on k such that

|R n | ≤ D sup t∈[τ 0 ,τ 2k-3 ] |g (k) 0 (t) -g (k) 0 (τ )| (τ 2k-3 -τ 0 ) 2k .
In the previous bound, we used the fact that the (2k)-times derivative of the function in (4.12) is g

(k) 0 (t) -g (k)
0 (τ ). But, note that this derivative is o p (1), which follows from uniform continuity of g (k) 0 on compacts. This in turn implies the claimed bound.

Proof of Lemma 3.1 for the LSE. Let j 0 ∈ {0, . . . , 2k -4} be such that [τ j 0 , τ j 0 +1 ] is the largest knot interval; i.e., τ j 0 +1τ j 0 = max 0≤j≤2k-4 (τ j+1τ j ). Let a = τ 0 , b = τ 2k-3 . Using the inequality in (4.10) and since the bounds on R n and E n are independent of the choice of τ in ∪ 2k-4 j=0 (τ j , τ j+1 ), it follows that sup τ ∈(τ j 0 ,τ j 0 +1 )

(-1) k e k (τ ) ≤ O p (n -2k/(2k+1) ) + o p ((τ 2k-3 -τ 0 ) 2k ).
Now, on the interval [τ j 0 , τ j 0 +1 ], the Hermite spline interpolant of the function x 2k /(2k)! reduces to a polynomial of degree 2k -1. On the other hand, imsart-aos ver. 2005/05/19 file: tr462R.tex date: January 30, 2007 the best uniform approximation of the function x 2k on [τ j 0 , τ j 0 +1 ] from the space of polynomials of degree ≤ 2k -1 is given by the polynomial

x → x 2k - τ j 0 +1 -τ j 0 2 2k 1 2 2k-1 T 2k 2x -(τ j 0 + τ j 0 +1 ) τ j 0 +1 -τ j 0 , (4.13)
where T 2k is the Chebyshev polynomial of degree 2k (defined on [-1, 1]), see, e.g., Nürnberger (1989), Theorem 3.23, page 46 or DeVore and Lorentz (1993), Theorem 6.1, page 75. It follows that sup

τ ∈(τ j 0 ,τ j 0 +1 ) (-1) k e k (τ ) ≥ T 2k 2 4k-1 (2k)! ∞ (τ j 0 +1 -τ j 0 ) 2k (4.14) = 1 2 4k-1 (2k)! (τ j 0 +1 -τ j 0 ) 2k since T 2k ∞ = 1. But, τ 2k-3 -τ 0 = 2k-4 j=0 (τ j+1 -τ j ) ≤ (2k -3)(τ j 0 +1 -τ j 0 ).
Hence, sup τ ∈(τ j 0 ,τ j 0 +1 )

(-1) k e k (τ ) ≥ 1 (2k -3) 2k 2 4k-1 (2k)! (τ 2k-3 -τ 0 ) 2k .
Combining the results obtained above, we conclude that Then,

(-1) k g (k) 0 (x 0 ) (2k -3) 2k 2 4k-1 (2k)! (τ 2k-3 -τ 0 ) 2k ≤ O p (n -2k/(2k+1) ) + o p ((τ 2k-3 -τ 0 ) 2k ) which implies that τ 2k-3 -τ 0 = O p (n -1/(2k+1) ).
H k [Y n ] -Y n ≥ g 0 (τ 0 ) fn -H k [ fn ] + ∆ n -H k [∆ n ] on [τ 0 , τ 2k-3 ]
, where Y n is the same empirical process introduced in (2.10),

fn (x) ≡ - t τ 0 (x -t) k-1 (k -1)! 1 ĝn (t) - 1 g 0 (τ 0 ) d( Ĝn (t) -G 0 (t))
and

∆ n (x) ≡ x τ 0 (x -t) k-1 (k -1)! 1 ĝn (t) - 1 g 0 (τ 0 ) d(G n (t) -G 0 (t)).
Proof. Let Ĝn (x) =

x 0 ĝn (s)ds. The characterization in (2.14) can be rewritten as

x 0 (x -t) k-1 ĝn (t) d( Ĝn (t) -G n (t)) ≥ 0, for x > 0 = 0, if x is a jump point of ĝ(k-1) n .
(4.15)

Note that when x is a jump point of ĝ(k-1) n , the two parts of (4.15) imply that the first derivative of the function on the right side is equal to 0 at the jump point x; i.e.,

x 0 (x -t) k-2 ĝn (t) d( Ĝn (t) -G n (t)) = 0. (4.16) For x > 0, let Ĥn (x) = x 0 (x -t) k-1 (k -1)! d Ĝn (t).
Note that Ĥn = H n defined in (2.13), and on [τ 0 , τ 2k-3 ], Ĥn is a spline of degree 2k -1 with knots τ 0 , . . . , τ 2k-3 . For x ∈ [τ 0 , τ 2k-3 ], we can write

x 0 (x -t) k-1 ĝn (t) d( Ĝn (t) -G n (t)) = 1 g 0 (τ 0 ) x 0 (x -t) k-1 d( Ĝn (t) -G n (t))
imsart-aos ver. 2005/05/19 file: tr462R.tex date: January 30, 2007

+ x 0 (x -t) k-1 1 ĝn (t) - 1 g 0 (τ 0 ) d( Ĝn (t) -G n (t)) = ( Ĥn (x) -Y n (x)) g 0 (τ 0 ) + τ 0 0 (x -t) k-1 1 ĝn (t) - 1 g 0 (τ 0 ) d( Ĝn (t) -G n (t)) + x τ 0 (x -t) k-1 1 ĝn (t) - 1 g 0 (τ 0 ) d( Ĝn (t) -G 0 (t)) + x τ 0 (x -t) k-1 1 ĝn (t) - 1 g 0 (τ 0 ) d(G 0 (t) -G n (t)) = 1 g 0 (τ 0 ) ( Ĥn (x) -Y n (x)) + p n (x) -fn (x) -∆ n (x).
Note that

p n (x) ≡ τ 0 0 (x -t) k-1 1 ĝn (t) - 1 g 0 (τ 0 ) d( Ĝn (t) -G n (t))
is a polynomial of degree k -1. From (4.15) and (4.16), it follows that Ĥn is the Hermite spline interpolant of the function

Y n + g 0 (τ 0 ) -p n + fn + ∆ n such that Ĥn ≥ Y n + g 0 (τ 0 )(-p n + fn + ∆ n ).
Hence,

H k Y n + g 0 (τ 0 ) -p n + fn + ∆ n ≥ Y n + g 0 (τ 0 ) -p n + fn + ∆ n on [τ 0 , τ 2k-3 ], or equivalently H k [Y n ] -Y n ≥ g 0 (τ 0 ) fn -H k [ fn ] + ∆ n -H k [∆ n ] .
Since H k [Y n ] -Y n has already been studied for proving the order of the gap in the case of the LSE, the final step is to evaluate each of the interpolation errors

E 1 = fn -H k [ fn ] and E 2 = ∆ n -H k [∆ n ]. (4.17)
hope was that we might be able to learn about the problem with k = ∞ by studying the problem for fixed k, and then taking limits as k → ∞. Unfortunately, we now believe that new tools and methods will be needed. Here is the state of affairs as we understand it now.

In terms of the rates of convergence, and localization properties, our development here shows that the local behavior of the estimators near a fixed point x 0 > 0 becomes dependent on an increasing number of jump points or knots in the spline problem. In other words, one needs to consider 2k-2 consecutive jump points (knots) τ 0,n < • • • < τ n,2k-3 of the (k -1)-st derivative of the estimators in a neighborhood of x 0 in order to be able to find a bound on τ n,j+1τ n,j , j = 0, . . . , 2k -4 as n → ∞. Thus the problem becomes increasingly "less local" with increasing k, and this leads us to suspect that the situation in the k = ∞ (or completely monotone) problem might be only "weakly local" or perhaps even "completely non-local" in senses yet to be precisely defined.

Another aspect of this problem is that although the MLE is asymptotically equivalent to the (mass unconstrained) LSE for each fixed k if our conjecture (1.7) holds, they seem to differ increasingly as k increases. For k = 1, the MLE and the LSE are identical; for k = 2, the MLE differs from the (mass unconstrained), but the LSE always has total mass 1. For k ≥ 3, the MLE and LSE differ, and, moreover, the total amount of mass in the

unconstrained LSE for n = 1 is M k = ((2k -1)/k)(1 -1/(2k -1)) k-1
2e -1/2 ≈ 1.21306 . . . = 1 as k → ∞. We do not know how the mass of the unconstrained LSE behaves jointly in n and k, even though (by consistency) the mass of the LSE converges to 1 as n → ∞ for fixed k. We also do not even know if the unconstrained LSE exists for the scale mixture of exponentials, even though it is clear that the constrained estimator (defined by the least squares criterion minimized over D k rather than M k ) with mass 1 does exist. Since our current proof techniques rely so heavily on showing equivalence between the MLE and the (unconstrained) LSE, it seems likely that new methods will be required. We do not know if the (mass) constrained LSE's and the MLE's are asymptotically equivalent either for finite k or for k = ∞. Our current plan is to study the constrained LSE's with total mass constrained to be 1 for finite sample sizes, to investigate the asymptotic equivalence of these mass-constrained LSE's and the MLE's, and to (perhaps) extend this study to k = ∞ via limits on k. We do not yet know the "right" Gaussian version of the estimation problem in the completely monotone case.

Another way to view these difficulties might be to take the following perspective: since more knowledge is available concerning the MLE's for the families D k with k finite, and since D ∞ is the intersection of all the D k 's (and hence well-approximated by D k with k large), we can fruitfully consider estimation via model selection, choosing k based on the data, over the collection ∪ ∞ k=1 D k . In summary, we have tried to shed some more light on the local behavior of two nonparametric estimators of a k-monotone density, the Maximum Likelihood and Least Squares estimators. We have shown that they are both adaptive splines of degree k -1, with knots determined by the data and their corresponding criterion functions. When (-1) k g (k) 0 (x 0 ) > 0, the distance between their knots in a neighborhood of a point x 0 > 0 was shown to be n -1/(2k+1) if a conjecture concerning the uniform boundedness of the interpolation error in a new Hermite interpolation problem holds, and once this control of the distance between the knots is available, pointwise limit distribution theory follows via a route paralleling previous results for k = 1, 2. Although we do not exclude the possibility that this order could be established via other different approaches, we hope that the techniques developed here demonstrate that there could still be many interesting and powerful connections between statistics and approximation theory.

6. Appendix-Proofs from Empirical processes theory. The following proposition is a slight generalization of Lemma 4.1 of Kim and Pol- lard (1990), page 201. Proof. By van der Vaart and Wellner (1996), theorem 2.14.1, page 239, it follows that

E sup fx,y∈F x,R |(G n -G 0 )(f x,R )| 2 ≤ K n EF 2 x,R (X 1 ) = O n -1 R 2d-1 (6.20)
for some constant K > 0 depending only on x 0 , δ, and the entropy integral in (6.18). For any f x,y ∈ F x,R , we write 1≤j≤jn n 2(k+d)/(2k+1) n -1 n -(2d-1)/(2k+1) j 2d-1 ( (j -1) k+d + m) 2 = C 1≤j≤jn j 2d-1 ( (j -1) k+d + m) 2 ≤ C ∞ j=1 j 2d-1 ( (j -1) k+d + m) 2 0 as m ∞ where C > 0 is a constant that depends only on x 0 , δ. Therefore it follows that (6.19) holds.

(P n -P 0 )(f x,y ) = (G n -G 0 )(f x,
In the following, we present VC-subgraph proofs for Lemma 4.4. (1996), problem 11, page 152, it suffices to show that the "between graphs"

C 1 = {(t, c) ∈ R + × R : 0 ≤ c ≤ f t (x) or f t (x) ≤ c ≤ 0} :
x ∈ [y 0 , y 2k-3 ], x 0δ ≤ y 0 < y 1 < • • • < y 2k-3 ≤ y 0 + R ≤ x 0 + δ} is a VC class of sets. Let

C 1,j = (t, c) ∈ R + × R : 0 ≤ c ≤ f t (x)1 [y j-1 ,y j ] (t)
or f t (x)1 [y j-1 ,y j ] (t) ≤ c ≤ 0 :

x ∈ [τ 0 , τ 2k-3 ], x 0δ ≤ y 0 < y 1 < • • • < y 2k-3 ≤ y 0 + R ≤ x 0 + δ} for j = 1, . . . , 2k -3. Since t → f t (x)1 [y j-1 ,y j ] (t) is a polynomial of degree at most k -1 for each j = 1, . . . , k, the classes C 1,j are all VC classes. Also note that

C 1 ⊂ C 1,1 . . . C 1,2k-3 ≡ C k .
By Dudley (1999), theorem 2.5.3, page 153, C k is a VC class (or see [START_REF] Van Der Vaart | Weak Convergence and Empirical Processes[END_REF], Lemma 2.6.17, part (iii), page 147). Hence C 1 is a VC class and F y 0 ,R is a VC -subgraph class.

  neighborhood of x 0 . Let ḡn denote either the LSE gn or the MLE ĝn and let Fn be the corresponding mixing measure defined in terms of Ḡn = • 0 ḡn (s)ds via (2.9). If Conjecture (1.7) holds, then 

  imsart-aos ver. 2005/05/19 file: tr462R.tex date: January 30, 2007 3.2. The key results and outline of the proofs. Our proof of Theorem 3.2 proceeds by solving the key gap problem assuming that our Conjecture (1.7) holds. This is carried out in Section 4 in which the main result is: Lemma 3.1 Let k ≥ 3 and ḡn denote either the LSE gn or the MLE ĝn . If

Lemma 3. 5

 5 For all j ∈ {0, . . . , k -1}, let Āj,n denote either Ãj,n or A j,n . If Conjecture (1.7) holds, then Āj,n = O p (1). (3.4) Now we rescale the processes Ȳloc n and Hloc n so that the rescaled Ȳloc n converges to the canonical limit process Y k defined in Lemma 3.4. Since the scaling of Ȳloc n will be exactly the same as the one we used for Y k , we define Hl n and Ȳl n by

4. 3 .

 3 The gap problem for the MLE. To show Lemma 3.1 for the MLE, one needs to deal with an extra difficulty posed by the nonlinear form of the characterization of this estimator, given in (2.14). In the following, we show how one can get around this difficulty. The main idea is to "linearize" the characterization of the MLE, and hence be able to re-use the arguments developed for the LSE in the previous subsection.Lemma 4.6 Let τ 0 , . . . , τ 2k-3 be 2k -2 successive jump points of ĝ(k-1) n .

1 0

 1 imsart-aos ver. 2005/05/19 file: tr462R.tex date: January 30, 2007 Lemma 6.1 Let F be a collection of functions defined on [x 0δ, x 0 + δ],with δ > 0 small. Suppose that for a fixed x ∈ [x 0δ, x 0 + δ] and R > 0such that [x, x + R] ⊆ [x 0δ, x 0 + δ], the collection F x,R = f x,y ≡ f 1 [x,y] , f ∈ F, x ≤ y ≤ x + R admits an envelope F x,R such that EF 2 x,R (X 1 ) ≤ KR 2d-1 , R≤ R 0 ,for some d ≥ 1/2 where K > 0 depends only on x 0 and δ. Moreover, suppose thatsup Q log N (η F x,R Q,2 , F x,R , L 2 (Q))dη < ∞. (6.18)Then, for each > 0 there exist random variablesM n of order O p (1) such that |(G n -G 0 )(f x,y )| ≤ |y -x| k+d + n -k+d 2k+1 M n for |y -x| ≤ R 0 .(6.19)

  y ) and define M n byM n = inf D > 0 : (P n -P 0 )(f x,y ) ≤ (yx) k+d + n -(k+d)/(2k+1) D, for all f x,y ∈ F x,Rimsart-aos ver. 2005/05/19 file: tr462R.tex date: January 30, 2007and M n = ∞ if no D > 0 satisfies the required inequality. For 1 ≤ j ≤ Rn 1/(2k+1) = j n , we haveP (M n > m) ≤ P (P n -P 0 )(f x,y ) > (yx) k+d + n -(k+d)/(2k+1) m for some f x,y ∈ F x,R ≤ 1≤j≤jn P n (k+d)/(2k+1) (P n -P 0 )(f x,y ) > (j -1) k+d + m for some f x,y ∈ F x,R , (j -1)n -1/(2k+1) ≤ yx ≤ jn -1/(2k+1) ≤ 1≤j≤jn n 2(k+d)/(2k+1) E sup y:0≤y-x<jn -1/(2k+1) (P n -P 0 )(f x,y-x ) 2 ( (j -1) k+d + m) 2 = 1≤j≤jn n 2(k+d)/(2k+1) E sup f x,y-x ∈F x,jn -1/(2k+1) (P n -P 0 )(f x,y-x ) 2 ( (j -1) k+d + m) 2 ≤ C

Proposition 6. 1

 1 For k ≥ 2, the class of functions F y 0 ,R given in (4.11) is a VC-subgraph class.Proof. We show first that the class of subgraphsC = {{(t, c) ∈ R + × R : c < f t (x)} : x ∈ [τ 0 , τ 2k-3 ], x 0δ ≤ y 0 < y 1 < • • • < y 2k-3 ≤ y 0 + R ≤ x 0 + δ} imsart-aos ver. 2005/05/19 file: tr462R.tex date: January 30, 2007is a VC class of sets in R + × R. If we show this, then the class of functions (4.11) is VC-subgraph. Alternatively, fromvan der Vaart and Wellner

  Once the rates have been established, we define for the LSE localized versions

	Y loc n , Hloc

n of the processes Y n , Hn given in (2.10) and (2.11) respectively, and Y loc n , H loc n related to the process H n given in (2.13) in the case of the MLE. The proof then proceeds by showing that: • The localized processes Y loc n and Y loc n converge weakly to Y a,σ where

  The limit process Y a,σ in Lemma 3.2 satisfies

	Hloc n (t) -Ȳloc n (t) ≥ 0	for all t ≥ 0,
	with equality if x 0 + tn -1/(2k+1) is a jump point of	ḡ(k-1)

let Hloc n denote either Hloc n or H loc n . Lemma 3.3 The localized processes Ȳloc n and Hloc n , satisfy n . imsart-aos ver. 2005/05/19 file: tr462R.tex date: January 30, 2007 Lemma 3.4

  perturbation functions need of course to be permissible, i.e., the resulting perturbed function must belong to the k-monotone class, but they need also to have a compact support to suit the local nature of the current es-

timation problem. It turned out that choices are rather limited, and that B-splines with degree k -1 and support [τ n,1 , τ n,k+1 ], where τ n,1 , . . . , τ n,k+1

imsart-aos ver. 2005/05/19 file: tr462R.tex date: January 30, 2007

Acknowledgments:

We gratefully acknowledge helpful conversations with Carl de Boor, Nira Dyn, Tilmann Gneiting, Piet Groeneboom and Alexei Shadrin. We also would like to thank an Associate Editor and two referees for their comments and suggestions that helped to greatly improve the presentation of the paper.

* Research supported in part by National Science Foundation grant DMS-0203320 † Research supported in part by National Science Foundation grant DMS-0203320, NI-AID grant 2R01 AI291968-04, and an NWO Grant to the Vrije Universiteit, Amsterdam

Lemma 4.7 Let E 1 and E 2 be the interpolation errors defined in (4.17).

Then,

Proof. A detailed proof can be found in Balabdaoui and Wellner (2006), Appendix 3.

Proof of Lemma 3.1 for the MLE. From our study of the distance between the knots of the LSE, and using very similar caluculations we can show that for all τ ∈ ∪ 2k-4 j=0 (τ j , τ j+1 )

which implies that by the results obtained for the LSE

for some constant D > 0 depending on k and x 0 . Hence, it follows from Lemma 4.7 that

which yields the order n -1/(2k+1) for the distance between the knots of the MLE in the neighborhood of x 0 .

Conclusions and discussion.

As noted in Section 1, one of the motivations for this work was to try to approach the problem of pointwise limit theory for the MLE's in both the forward and inverse problems for the family of completely monotone densities on R + . This is one very important special case of the family of nonparametric mixture models with a smooth kernel as was mentioned in part (b) of our discussion in Section 1. Jewell (1982)