M Olteanu

J Rynkiewicz Samos-Matisse

ASYMPTOTIC PROPERTIES OF AUTOREGRESSIVE REGIME-SWITCHING MODELS

) to autoregressive models with regimes switches according to a Markov chain. In this case, it is shown that the marginal likelihood is no longer a contrast function and cannot be used to select the number of regimes. Some numerical examples illustrate the results and their convergence properties.

Introduction

Autoregressive regime-switching models are being widely used in modelling financial and economic time series such as business cycles [START_REF] Hamilton | A new approach to the economic analysis of nonstationary time series and the business cycle[END_REF]; [START_REF] Lam | The Hamilton model with a general autoregressive component: estimation and comparison with other models of economic time series[END_REF], exchange rates (Engle and [START_REF] Hamilton | Analysis of time series subject to changes in regime[END_REF], financial panics [START_REF] Schwert | Business cycles, financial crises and stock volatility[END_REF] or stock prices [START_REF] Wong | On a mixture autoregressive model[END_REF].

When the number of regimes is fixed, the statistical inference is relatively straightforward [START_REF] Hamilton | Analysis of time series subject to changes in regime[END_REF]) and the asymptotic properties of the estimates have already been established [START_REF] Francq | Ergodicity of autoregressive processes with Markovswitching and consistency of the maximum likelihood estimator[END_REF]; Krishnamurthy and Ryden, 1998; Douc R., Moulines E. and Rydén T., 2004). However, the problem of selecting the number of regimes is far less obvious and hasn't been completely answered yet. When the number of regimes is unknown, identifiability problems arise and the likelihood ratio test statistic (LRTS hereafter) is no longer convergent to a χ 2distribution. Some partial answers were proposed by [START_REF] Hansen | The likelihood ratio test under nonstandard conditions: testing the Markov switching model of GNP[END_REF]Hansen ( ,1996aHansen ( , 1996b) ) and [START_REF] Garcia | Asymptotic null distribution of the likelihood ratio test in Markov switching models[END_REF]. Hansen derived an asymptotic bound for the distribution of the LRTS based on empirical processes techniques, while Garcia obtained the asymptotic distribution of the LRTS, but under some very restrictive hypothesis. Let us also mention that the consistency of the estimate of the number of regimes was proven recently in a Bayesian framework [START_REF] Rios | Penalized estimate of the number of states in Gaussian linear AR with Markov regime[END_REF].

In the particular case of mixture models, several ideas and methods were proposed to estimate the number of components: non-parametric techniques as in [START_REF] Henna | On estimating the number of constituents of a finite mixture of continuous distributions[END_REF], [START_REF] Roeder | A graphical technique for determining the number of components in a mixture of normals[END_REF] or Izenman and Sommer (1998), moment techniques in [START_REF] Lindsay | Moment matrices: application in mixtures[END_REF] or Dacunha-Castelle and Gassiat (1997) and penalized maximumlikelihood in Leroux(1992a), [START_REF] Keribin | Consistent estimation of the order of mixture models[END_REF] and [START_REF] Liu | Asymptotics for likelihood ratio tests under loss of identifiability[END_REF]. Furthermore, [START_REF] Gassiat | Likelihood ratio inequalities with applications to various mixtures[END_REF] proved that in the case of hidden Markov models, the number of regimes can be estimated using a marginal penalized-likelihood estimate. The aim of this paper is to extend the existing results for mixtures and hidden Markov models to the case where the mean of the observed process is replaced by a regression function.

In Section 2, the results on the LRTS for mixture models are extended to autoregressive regime-switching models with independent regime switches. We give sufficient conditions for the tightness of the LRTS and compute its asymptotic distribution. Section 3 is devoted to verifying the result of the previous section in the case where the noise is Gaussian and the regression functions are linear. The last section handles the case where regime switches are Markovian. Once the result in the independent case was established, it seemed natural to generalize it by using a cost function close to the marginal likelihood, as defined in [START_REF] Gassiat | Likelihood ratio inequalities with applications to various mixtures[END_REF]. Yet, it can be seen right away that this is no longer a contrast function and the convergence is achieved only in the particular cases of constant autoregressive functions (hidden Markov models) or independent regime switches (autoregressive mixture models).

2. LRTS for autoregressive mixture models 2.1. The observations. Let us briefly recall the definition of strong mixing processes which will be needed hereafter. For a more detailed review, refer to [START_REF] Doukhan | Mixing: properties and examples[END_REF] and [START_REF] Bradley | Basic properties of strong mixing conditions. A survey and some open questions[END_REF].

Let (Y k ) k∈Z be a strictly stationary sequence of random variables defined on a probability space (Ω, K, P). For every n ≥ 1, define the β-mixing coefficients

β n = β F 0 -∞ , F ∞ n where F 0 -∞ = σ (Y k , k ≤ 0), F ∞ n = σ (Y k , k ≥ n), as β (A, B) = 1 2 sup
(Ai) i∈I ,(Bj ) j∈J (i,j)∈I×J |P (A i ∩ B j ) -P (A i ) P (B j )|

where (A i ) i∈I (resp. (B j ) j∈J ) ranges over the set of A(resp. B) measurable partitions.

The sequence (Y k ) k∈Z is called β-mixing if lim n→∞ β n = 0.

Throughout the rest of the paper, we will assume that the observations are a realization of a stationnary process (Y k ). Moreover, (Y k ) will be assumed to be geometrically β-mixing. This assumption may seem strong, but actually it is fulfilled by a wide class of processes.

Finally, let us denote by µ the stationnary measure of the vector (Y k+1 , Y k ).

2.2. The model. Let P = {g θ , θ ∈ Θ} be a set of densities with respect to some positive measure ν, where Θ is a compact finite-dimensional set.

Let us consider an observed sample {y 1 , ..., y n } of the series Y k . For every y k , the true density conditionally to y k-1 is

g 0 (y k | y k-1 ) = p0 i=1 π 0 i g θ 0 i (y k | y k-1 ) ,
where g θ 0 i ∈ P, π 0 i > 0 and

p0 i=1 π 0 i = 1.
This model is a generalization of mixture models. Several regression models can be written in this way, for example mixtures of linear regressions with Gaussian noise, which are particularly important in econometrics (see Hansen 1996) and will be studied in Section 3 :

(1)

Y t = a 0 Xt Y t-1 + b 0 Xt + σ 0 Xt ε t ,
where • X t is an i.i.d. sequence of random variables valued in a finite space {1, ..., p 0 } and with probability distribution

π 0 = π 0 1 , • • • , π 0 p0 • a 0 1 , • • • , a 0 p0 , b 0 1 , • • • , b 0 p0 are real numbers • σ 0 1 , • • • , σ 0 p0 are strictly positive real numbers • ε t is an i.i.d. noise N (0, 1), independent of (Y t-k ) k≥1 .
Let us remark that if a 0 1 , • • • , a 0 p0 are all zero, the model is a simple Gaussian mixture.

Approximation of the LRTS.

Let G be the set of possible conditional densities:

G = g (y k | y k-1 ) = p i=1 π i g θi (y k | y k-1 ) , π i ∈ [0; 1], p i=1 π i = 1, g θi ∈ P, p ∈ N ⋆
If p ≤ p 0 , there are no identification issues. Therefore, we will assume that p > p 0 in the sequel.

Let

l n (g) = n k=2 ln g (y k | y k-1 ) be the log-likelihood function of (y 1 , • • • , y n ), conditionally to y 1 .
The LRTS is defined as:

(2)

2λ n = 2 sup g∈G ln(g) -ln(g 0 ) = 2 sup g∈G n k=2 p i=1 π i g θi (y k | y k-1 ) n k=2 p0 i=1 π 0 i g θ 0 i (y k | y k-1 )
We establish a theorem giving an approximation of the LRTS. Some notations and definitions are needed first:

• For an η > 0, denote (3)
G η := g ∈ G, g -g 0 L 2 (µ) ≤ η .
The extended set of score-functions S η is defined as:

(4)

S η =      s g = g g 0 -1 g g 0 -1 L 2 (µ) , g ∈ G η      . • Let us define the limit-set of scores D D = d ∈ L 2 (µ) | ∃(g n ) ∈ G, g n -g 0 g 0 L 2 (µ) ----→ n→∞ 0, d -s gn L 2 (µ) ----→ n→∞ 0 .
By putting g t = g n for t ∈ [0, 1] and n ≤ 1 t < n + 1, we obtain that, for all d ∈ D, there exists a parametric path

(g t ) 0≤t≤1 such that ∀t ∈ [0, 1], g t ∈ G, t → gt-g 0 g 0 L 2 (µ) is continuous, gt-g 0 g 0 L 2 (µ) ---→ t→0 0 and d-s gt L 2 (µ) ---→ t→0 0.
• We recall the definition of the L 2,β (P)-space and the notion of bracketing entropy. Consider Z k a strictly stationary sequence, β-mixing and such that n≥1 β n < ∞. The L 2,β (P)-space is defined as

L 2,β (P) = f, f 2,β < ∞ , f 2,β = 1 0 β -1 (u) [Q f (u)] 2 du where -β (u) is the càdlàg extension of β n by considering β (u) = β [u] and β 0 = 1 -ϕ -1 (u) = inf {t ∈ R, ϕ (t) ≤ u}, if ϕ is a non-increasing function -Q f is the quantile function of |f (Z 0 )|, that is the inverse of t → P (|f (Z 0 )| > t) Consider the extended set of score-functions S η endowed with the norm • 2,β . For every ε > 0, we define an ε-bracket by [l, u] = {f ∈ F, l ≤ f ≤ u} such that u -l 2,β < ε. The ε-bracketing entropy is H [•] ε, S η , • 2,β = ln N [•] ε, S η , • 2,β ,
where

N [•] ε, S η , • 2,β is the minimum number of ε-brackets necessary to cover S η .
With the previous notations, we introduce the following assumption (B): Assume that G is Glivenko-Cantelli and that there exists η > 0 such that

1 0 H [•] ε, S η , • 2,β dε < ∞.
Then, according to [START_REF] Doukhan | Mixing: properties and examples[END_REF], the set S η is Donsker under (B). Now, let us state the following theorem which generalizes the result of [START_REF] Gassiat | Likelihood ratio inequalities with applications to various mixtures[END_REF]. The proof is given in the Appendix.

Theorem 1 :

Under the assumption (B),

2λ n = sup d∈D max 1 √ n n i=2 d(Y i , Y i-1 ); 0 2 + o P (1)
Although this result may be applied to more general models, this paper is restricted to autoregressive mixture models.

2.4. Asymptotic law of the LRTS. This section is a direct application of Theorem 1.

We give sufficient conditions for which the Donsker assumption (B) holds in the case of autoregressive mixture models.

Usually, for parametric models, a Lipschitz condition on θ is sufficient to show that S is Donsker. However, if g depends on the parameter θ, the score-function

θ → s g = g g 0 -1 ' ' ' g g 0 -1 ' ' ' L 2 (µ)
may not be continuous, thus not Lipschitz, in θ 0 .

The following theorem shows that assumption (B) holds for autoregressive mixture models under some general hypothesis. Furthermore, we prove that the limit set of scores D is complete and has continuous parametric paths. Hence, the asymptotic behavior of the LRTS may be completely described.

Assumptions for the tightness of LRTS.

H-1:

The set G is Glivenko-Cantelli and the set of possible parameters:

{π 1 , • • • , π p ∈ [0, 1], θ 1 , • • • , θ p ∈ Θ}
contains a neighborhood of the parameters defining the true conditional density g 0 . H-2: There exists η > 0 such that for all g ∈ G with gg 0 L 2 (µ) ≤ η,

g g 0 -1 L 2 (µ)
< ∞ H-3: By denoting l θi := g θ i g 0 and, with a slight abuse of notation, ∂ q ∂θ q j the derivative of order q with respect to all components of θ j , we assume the existence of a square-integrable function h and of a neighborhood N of

(θ 0 1 , • • • , θ 0 p 0 ) such that, for all (θ 1 , • • • , θ p 0 ) ∈ N , ∂l θj ∂θ j (θ j ) ≤ h, ∂ 2 l θj ∂θ 2 j (θ j ) ≤ h and ∂ 3 l θj ∂θ 3 j (θ j ) ≤ h.
H-4: With the following notations:

l ′ j := ∂l θj ∂θ j θ 0 j , l ′′ := ∂ 2 l θj ∂θ 2 j θ 0 j we assume that for distinct (θ i ) 1≤i≤p (l θi ) 1≤i≤p , (l ′ i ) 1≤i≤p 0 , (l ′′ i ) 1≤i≤p 0
are linearly independent in the Hilbert space L 2 (µ).

Let us define Ω : L 2 (P ) → L 2 (µ) by Ω(g) = g g 2 , for g = 0. Now, we can state the following theorem, which generalizes theorem 4.1 of Liu and Shao (2003) :

Theorem 2 : Let d be the parametric dimension of the regression functions. Under the assumptions H-1, H-2, H-3 and H-4, there exists a centered Gaussian process {W S , S ∈ F} with continuous sample path and covariance kernel

P (W S1 W S2 ) = P (S 1 S 2 ) such that lim n→∞ 2λ n = sup S∈F (max(W S , 0)) 2 .
The index set F is defined as F = ∪ t F t , with the union running over

t = (t 0 , • • • , t p0 ) ∈ N p0+1 with 0 = t 0 < t 1 < • • • < t p0 ≤ p and F t = Ω p0 i=1 ζ i l θ 0 i + p i=p0+1 ζ i l θi + p0 i=1 λ T i l ′ i + δ p0 i=1 ti j=ti-1+1 γ T j l ′′ i γ j , λ 1 , • • • , λ p0 , γ 1 , • • • , γ tp 0 ∈ R d ; ζ 1 , • • • , ζ p ∈ R, θ tp 0 +1 , • • • , θ p ∈ Θ -θ 0 1 , • • • , θ 0 p0
where δ = 1 if there exists a vector q such that:

q j ≤ 0, ti j=ti-1+1 q j = 1, ti j=ti-1+1
√ q j γ t j = 0 for i = 1, • • • , p 0 ; and δ = 0 otherwise.

Note that the asymptotic law of the LRTS depends on the true parameters of the model. The next two sections illustrate important consequences of this theorem.

2.5. Penalized-likelihood estimate for the number of regimes. For p ∈ N ⋆ , let us denote

G p = g (y k | y k-1 ) = p i=1 π i g θi (y k | y k-1 ) , π i ∈ [0; 1], p i=1 π i = 1, g θi ∈ P .
For some fixed P ∈ N ⋆ sufficiently large, we shall consider the following class of functions

G P = P p=1 G p
For every g ∈ G P we define the number of regimes as

p (g) = min {p ∈ {1, ..., P } , g ∈ G p } .
With this definition, p 0 = p g 0 is the number of regimes of the true model.

The estimate of the number of regimes p can now be defined as p ∈ {1, ..., P } maximizing the penalized criterion:

(5)

T n (p) = sup g∈Gp l n (g) -a n (p)
where

l n (g) = n k=2 ln g (y k | y k-1 )
is the conditional log-likelihood with respect to y 1 and a n (p) is a penalty term.

With the previous definitions, the following result can be stated:

Corollary 1

Suppose the following assumptions are true:

• Assumptions H-1, H-2, H-3 and H-4 are true. • (A) a n (•) is an increasing function of p, a n (p 1 ) -a n (p 2 ) ----→ n→∞ ∞ for every p 1 > p 2 and an(p) n ----→ n→∞ 0 for every p.
Then, p maximizing the penalized criterion defined by (5) converges in probability,

p P ----→ n→∞ p 0 .

Proof

The result and its proof are inspired by [START_REF] Gassiat | Likelihood ratio inequalities with applications to various mixtures[END_REF] and [START_REF] Keribin | Consistent estimation of the order of mixture models[END_REF].

First, let us show that p does not overestimate p 0 .

P (p > p 0 ) ≤ P p=p0+1 P (T n (p) > T n (p 0 )) = P p=p0+1 P sup g∈Gp l n (g) -a n (p) > l n g 0 -a n (p 0 ) ≤ P p=p0+1 P (λ n > a n (p) -a n (p 0 ))
Since λ n is tight (Theorem 1) and according to assumption (A),

P (λ n > a n (p) -a n (p 0 )) → 0 Thus, P (p > p 0 ) → 0.
Let us now prove that p does not underestimate p 0 :

P (p < p 0 ) ≤ p0-1 p=1 P (T n (p) > T n (p 0 )) = p0-1 p=1 P 1 n-1 sup g∈Gp l n (g) -l n g 0 > an(p)-an(p0) n-1
For all p < p 0 , we shall prove that 1 n-1 sup g∈Gp l n (g)l n g 0 converges in probability to a strictly negative value. Then, according to the hypothesis (A), the proof will be complete.

Hypothesis (H-1) ensures that E µ (ln g) < ∞ for all g ∈ G p . Let us define K g 0 , G p = inf g∈Gp K g 0 , g , where K g 0 , g = E µ ln g 0 g .
Since the set of parameters is compact and K g 0 , g is continuous with respect to the parameters, K g 0 , G p attains its infimum for some g ∈ G p and, according to the hypothesis (H-4), K g 0 , g > 0.

By the definition of the maximum likelihood,

1 n -1 sup g∈Gp l n (g) -l n g 0 ≥ 1 n -1 l n (g) -l n g 0 Since (Y k , Y k-1
) is strictly stationary and geometrically ergodic,

1 n -1 l n (g) -l n g 0 P ----→ n→∞ -E µ ln g 0 g = -K g 0 , g Thus, lim inf n→∞ 1 n -1 sup g∈Gp l n (g) -l n g 0 ≥ -K g 0 , g It remains to prove that lim sup n→∞ 1 n -1 sup g∈Gp l n (g) -l n g 0 ≤ -K g 0 , g
Since the set of parameters is compact, for all η > 0, it may be covered by a finite number of balls N η , centered in c i = (π i , θ i ) p , i = 1, ..., N η and with radius η 2 . Let us now define

m η ((y 1 , y 2 ) , G p ) = sup d((π1,θ1) p ,(π2,θ2) p )≤η |l n g (π1,θ1) (y 1 , y 2 ) -l n g (π2,θ2) (y 1 , y 2 ) | Now, we can write 1 n-1 sup g∈Gp l n (g) -l n g 0 = = sup g∈Gp 1 n-1 n k=2 l n (g (Y k , Y k-1 )) -l n g 0 (Y k , Y k-1 ) ≤ sup i=1,...,Nη l n (g ci ) -l n g 0 + 1 n-1 n k=2 m η ((Y k , Y k-1 ) , G p ) n→∞ ----→ sup i=1,...,Nη -K g 0 , g ci + E µ (m η ((Y k , Y k-1 ) , G p ))
On the one hand,

sup i=1,...,Nη -K g 0 , g ci = -inf i=1,...,Nη K g 0 , g ci ≤ -K g 0 , g on the other hand, if η → 0, m η ((y 1 , y 2 ) , G p ) → 0 and E µ (m η ((Y k , Y k-1 ) , G p )) → 0 Thus lim sup n→∞ 1 n -1 sup g∈Gp l n (g) -l n g 0 ≤ -K g 0 , g

Application to linear autoregressive models with Gaussian noise

In this section we are interested in checking whether the assumptions (H1)-(H4) in subsection 2.4 hold in the case of a very popular autoregressive regime-switching model. We shall consider that the process (X t , Y t ) follows the true model

(4) Y t = a 0 Xt Y t-1 + b 0 Xt + σ 0 Xt ε t where
• X t is an i.i.d. sequence of random variables valued in a finite space {1, ..., p 0 } and with probability distribution

π 0 := π 0 1 , • • • , π 0 p0 • for every i ∈ {1, ..., p 0 }, a 0 i , b 0 i , σ 0 i are real numbers with a 0 i < 1 and σ 0 i > 0 • (ε t ) t∈N is a sequence of i.i.d. standard Gaussian variables, independent of (Y t-k ) k≥1 .
This model is obviously a special case of the more general model in section 2.2.

The following result which ensures strict stationarity and ergodicity can be stated. The proof may be found in the Appendix.

Proposition 1 : If a 0 i < 1 for every i ∈ {1, ..., p 0 }, (X t , Y t ) is strictly stationary, geometrically ergodic and, in particular, geometrically β-mixing. Moreover, there exists δ > 0 such that E µ e δY 2 t < ∞.

The set of possible conditional densities is the following :

G = g | g (y 2 | y 1 ) = p i=1 π i 1 √ 2πσ 2 i e -1 2σ 2 i (y2-(aiy1+bi)) 2
, p = 1, ..., P,

(a i , b i , σ 2 i ) ∈ Θ ⊂ R × R × R ⋆ + , π i ∈ [0, 1] , p i=1 π i = 1 , with Θ a compact set.
Within this framework, G is Glivenko-Cantelli and the assumption H-1 is true. Moreover, in a neighborhood of the true parameter θ 0 i , the second and third derivatives of

l θi (y 1 , y 2 ) = 2πσ 0 i 2 2πσ 2 i e -1 2σ 2 i (y2-(aiy1+bi)) 2 + 1 2σ 0 i 2 (y2-(a 0 i y1+b 0 i )) 2
exist and are dominated by a square integrable function, hence the assumption H-3 also holds.

Next, we check whether the generalized score functions are well defined (assumption H-2):

g g 0 -1 L 2 (µ)
< ∞, ∀g such that gg 0 ≤ η.

Conditions for the existence of the extended score functions

Consider the true conditional distribution

g 0 (y 2 | y 1 ) = p0 j=1 π 0 j f θ 0 j y 2 -F θ 0 j (y 1 )
and let the possible conditional distributions be

g (y 2 | y 1 ) = p i=1 π i f θi (y 2 -F θi (y 1 )) .

One can prove by direct computations that

Proposition 2 (the proof is available in the Appendix)

g g 0 -1 L 2 (µ)
< ∞ if for every i ∈ {1, ..., p}, there exists k ∈ {1, ..., p 0 }such that

σ 2 i < 2 σ 0 k 2 and a i -a 0 k < δ 2 (σ 0 k ) 2 -σ 2 i for δ > 0 verifying E e δY 2 t < ∞.
This sufficient condition states that the possible models should not be too different from the real one so that the convergence holds.

The consequences of this condition will be discussed later. For the moment, we will assume it is fulfilled.

Finally, let us check the assumption H-4:

Lemma 1 (the proof is straightforward and will be omitted)

The functions

g θi , i = 1, • • • , p, ∂g θ 0 i ∂a i , ∂g θ 0 i ∂b i , 1 σ 0 i ∂g θ 0 i ∂σ i + ∂ 2 g θ 0 i ∂b 2 i , ∂ 2 g θ 0 i ∂a 2 i , ∂ 2 g θ 0 i ∂σ 2 i , ∂ 2 g θ 0 i ∂a i ∂σ i , ∂ 2 g θ 0 i ∂a i ∂b i , ∂ 2 g θ 0 i ∂b i ∂σ i , i = 1, ..., p 0 are linearly independent.
Hence, the assumptions H-1, H-2, H-3 and H-4 are fulfilled if the possible parameters of the regression function are not too far from the true ones. Since the true regression function is not known, it seems very difficult to assume such hypothesis.

If the parameter set is not restricted, we will see in the next section that the LRTS will be divergent.

3.1. Simple regression mixture example. Let G be the set of possible conditional densities:

G = g (y k | y k-1 ) = πg θ (y k | y k-1 ) + (1 -π)g 0 (y k | y k-1 ) , π ∈ [0; 1], g θ ∈ P with P = g θ (y k |y k-1 ) = 1 √ 2π e -1 2 (y k -θy k-1 ) 2 , θ ∈ Θ ⊂ R the set of conditional densities and g 0 (y k | y k-1 ) = 1 √ 2π e -1 2 y 2
k . This model is clearly a particular case of the general mixture of expert model and is a simple example of mixture of regressions with Gaussian noise. Let

l n (g) = n k=2 ln g (y k | y k-1 )
be the conditional log-likelihood function of (y 1 , • • • , y n ). We want to know whether the true model is really a mixture regression model (i.e. θ = 0 and π = 0) or the observations are independent (θ = 0 or ∀x, π = 0). The LRTS is defined as: (6)

2λ n = 2 sup g∈G ln(g) -ln(g 0 ) = 2 sup g∈G n k=1 ln πg θ (y k | y k-1 ) + (1 -π)g 0 (y k | y k-1 ) g 0 (y k | y k-1 )
In order to derive the behaviour of the LRTS, two cases have to be analyzed. The first one is if π can be close to 0. The second one is when ∃δ > 0 such that π ≥ δ.

Divergence of LRTS. The LRTS can be divergent if π is not constraint. Indeed, for such sequence we can have E µ ln(g)ln(g 0 ) → 0 with θ = 0. The score functions are well defined if the quantity

g θ (Y2|Y1) g 0 (Y2|Y1) -1 L 2 (µ) = exp -θ 2 2 Y 2 1 + θY 2 Y 1 -1 L 2 (µ) is finite. So, exp -θ 2 2 Y 2 1 + θY 2 Y 1 -1 2 L 2 (µ) = 1 2π exp -θ 2 2 y 2 1 + θy 2 y 1 -1 2 exp -1 2 y 2 1 exp -1 2 y 2 2 dy 1 dy 2 = 1 2π exp -θ 2 y 2 1 + 2θy 2 y 1 -2 exp -θ 2 2 y 2 1 + θy 2 y 1 + 1 exp -1 2 y 2 1 exp -1 2 y 2 2 dy 1 dy 2
The integral of the dominant term (the first) is:

I (θ) = 1 2π exp -θ 2 y 2 1 + 2θy 2 y 1 exp -1 2 y 2 1 exp -1 2 y 2 2 dy 1 dy 2 = 1 2π exp -θ 2 + 1 2 y 2 1 + 2θy 1 y 2 -1 2 y 2 2 dy 1 dy 2 = 1 2π exp - θ 2 + 1 2 y 1 - θ √ θ 2 + 1 2 y 2 2 -1 2 -θ 2 θ 2 + 1 2 y 2 2 dy 1 dy 2 = √ 2θ 2 +1 √ 2π exp -1 2 -θ 2 θ 2 + 1 2 2 2 dy 2 Finally for -1 √ 2 < θ < 1 √ 2 , exp -θ 2 2 Y 2 1 + θY 2 Y 1 -1 L 2 (µ) < +∞
and the score function is well defined.

Note that the distribution of the LRTS 2λ n for a finite number of possible parameters θ 1 , • • • , θ m will always converge to the square of a m-dimensional normal distribution with covariance

E ν θi (Y 1 , Y 2 ) ν θj (Y 1 , Y 2 ) 1≤i,j≤m .
Suppose that an arbitrary number of"almost" uncorrelated random variables can be found, then λ n can take an arbitrarily large value since the maximum of m independent samples from standard normal distribution is approximately √ 2 log m. Hence, [START_REF] Fukumizu | Likelihood ratio of unidentifiable models and multilayer neural networks[END_REF]

has shown that if a sequence θ 1 , • • • , θ m , • • • exists so that lim m→∞ ν θm (Y 1 , Y 2 ) P → 0
then the likelihood ratio T n diverges to infinite. Here, we get

lim θ→ 1 √ 2 ,θ< 1 √ 2 exp -θ 2 2 Y 2 1 + θY 2 Y 1 -1 L 2 (µ) = +∞ So, for each sphere B of R 2 , centered on the origin, if (Y 1 , Y 2 ) ∈ B:
lim θ→ 1 √ 2 ,θ< 1 √ 2 exp " -θ 2 2 Y 2 1 +θY2Y1 " -1 ' ' 'exp " -θ 2 2 Y 2 1 +θY2Y1 " -1 ' ' ' L 2 (µ) = 0 and exp " -θ 2 2 Y 2 1 +θY2Y1 " -1 ' ' 'exp " -θ 2 2 Y 2 1 +θY2Y1 " -1 ' ' ' L 2 (µ) converges to 0 in probability for θ → 1 √ 2 , θ < 1 √ 2 .
With the choice θ m = 1 Convergence of LRTS. If π is greater or equal than a δ > 0 then, necessary, the maximum likelihood estimator θ converges to θ 0 = 0, otherwise lim n→∞ λ n = sup g∈G E µ ln(g)ln(g 0 ) can not be close to 0. Thus, the model is identifiable in θ and unidentifiable in π. Since ∂ ∂θ g θ (y 2 | y 1 ) = y 1 (y 2θy 1 ) g θ , we have the following Taylor expansion around θ 0 = 0:

s g = g θ g 0 -1 g θ g 0 -1 L 2 = (θ-θ0) ∂ ∂θ g θ g 0 (θ0)+o(|θ-θ0|) (θ-θ0) ∂ ∂θ g θ g 0 (θ0)+o(|θ-θ0|) L 2 = Y1Y2+o(1) Y1Y2+o(1) L 2
Hence, the LRTS converges to the square of the maximum of a Gaussian process which convariance function is identically equal to 1 i.e. the classical χ 2 law.

In conclusion, if the mixture weights can be as small as possible, the likelihood ratio tends to infinity and in order to avoid this divergence, it is required to constraint the parameters in a neighborhood of the true walue, which does not make much sense if the model is unknown. But if the mixture weights are bounded by below, then all parameters of regression density g θ converge to some true one. It is thus possible, in the analysis of the asymptotics of the criterion, to restrict the set of scores as in proposition 2 and then apply theorem 2 with restricted set of score functions as well as corollary 1.

Generalization to autoregressive Markov-switching models?

The aims of the section is to study the generalization of the previous results to Markov switching models.

Let us consider the more general case where the process (X t , Y t ) follows the true model

(5) Y t = F θ 0 X t (Y t-1 ) + ε θ 0 X t (t)
where • X t is a homogeneous Markov chain, irreducible and aperiodic, with finite state-space {1, ..., p 0 } and stationary probability distribution π 0 := π 0 1 , ..., π 0 p0

• for every i ∈ {1, ..., p 0 }, F θ 0 i (y) ∈ F, where F = F θ , θ ∈ Θ, Θ ⊂ R d compact set is the family of possible regression functions. We suppose throughout the rest of this section that F θ 0 i are sublinear, that is they are continuous and ∃ a 0 i , b

0 i ∈ R 2 + such that F θ 0 i (y) ≤ a 0 i |y| + b 0 i , (∀) y ∈ R ; • for every i ∈ {1, ..., p 0 }, (ε θi (t)) t is an i.i.d. noise so that ε θi (t) is in- dependent of (Y t-k ) k≥1 . Moreover, ε θ 0 i (t)
has density g 0 i ∈ P , where P = g θ , θ ∈ Θ, Θ ⊂ R l is a family of strictly positive densities with respect to the Lebesgue measure.

According to [START_REF] Yao | On stability of nonlinear AR processes with Markov switching[END_REF], a unique strictly-stationary and geometricallyergodic solution (X t , Y t ) exists under the hypothesis

(HS) (∃) s ≥ 1 so that ∀i ∈ {1, • • • , p 0 }, E ε θ o i s < ∞ and the spectral radius ρ (Q s ) < 1, with Q s =    a 0 1 s π 0 11 • • • a 0 p0 s π 0 1p0 . . . . . . . . . a 0 1 s π 0 p01 • • • a 0 p0 s π 0 p0p0   
where a 0 i are the leading coefficients in the linear functions dominating F θ 0 i and π 0 ij are the entries of the transition matrix of X t , i, j ∈ {1, ..., p 0 }. The hypothesis (HS) is clearly verified whenever a 0 i < 1, for all i ∈ {1, ..., p 0 }. Considering an observed n-sample of Y t , one would naturally attempt to extend the methods in the previous sections to the case where the invariant measure of the hidden Markov chain is lower bounded by a strictly positive constant. Several problems arise: on the one hand, the non-identifiability issue and on the other hand, the dependence structure of X t . This dependence will not allow an explicit form for the conditional density, marginally in X t :

g 0 (y k | y k-1 , ..., y 0 ) = p0 i=1 P (X k = i | y k-1 , ..., y 0 ) g θ 0 i y k -F θ 0 i (y k-1 )
since P (X k = i | y k-1 , ..., y 0 ) has to be computed recursively. However, since X t is stationary and following the same idea as [START_REF] Gassiat | Likelihood ratio inequalities with applications to various mixtures[END_REF], a cost function which involves the invariant probability measure of the hidden Markov chain can be defined.

The class of possible mixture densities is:

G = g | g (y 1 , y 2 ) = p i=1 π i g θi (y 2 -F θi (y 1 )) , θ i ∈ Θ
where Θ is a compact set.

The cost function is defined as

C n (g) = 1 n n k=2 ln g (y k | y k-1 ) = 1 n -1 n k=2 ln p i=1 π i g θi (y 2 -F θi (y 1 )) .
One may notice that C n (g) is similar to the conditional likelihood marginal in X t and may expect it to be maximized by g = g 0 , where "the true conditional density" is now written as

g 0 (y k | y k-1 ) = p0 i=1 π 0 i g θ 0 i y k -F θ 0 i (y k-1 ) ,
where π 0 i is the expectation of the hidden state i under the true invariant distribution.

Let us check if C n (g) is a contrast function with the maximum reached at g 0 . Let (X, Y 2 , Y 1 ) be a generic variable having the stationary measure of the extended Markov-chain

(X k , Y k , Y k-1 ) as distribution. Since C n (g) is an additive function of the Markov chain (X k , Y k , Y k-1 ) 1≤k≤n and p0 i=1 1 {X=i} (X) = 1, we have C n (g) = 1 n n k=2 ln p j=1 π j g θj y k -F θj (y k-1 ) a.s. → E ln p j=1 π j g θj Y 2 -F θj (Y 1 ) = E p0 i=1 1 {X=i} (X) ln p j=1 π j g θj Y 2 -F θj (Y 1 ) = p0 i=1 π 0 i R 2 ln p j=1 π j g θj y 2 -F θj (y 1 ) g θ 0 i y 2 -F θ 0 i (y 1 ) λ i (y 1
) dy 1 dy 2 where λ i (y 1 ) is the stationary measure of Y 1 conditionally to X = i.

Then,

E ln(g) -ln(g 0 ) = p0 i=1 P (X = i) E ln g g 0 | X = i = = p0 i=1 π 0 i ln P p j=1 πj g θ j (y2-Fθ j (y1)) P p 0 j=1 π 0 j g 0 θ j " y2-F 0 θ j (y1) " g θ 0 i y 2 -F θ 0 i (y 1 ) λ i (y 1 ) dy 1 dy 2
and, by Fubini's theorem, E ln(g)ln(g 0 ) = = ln P p j=1 πj f θ j (y2-Fθ j (y1))

P p 0 j=1 π 0 j f 0 θ j " y2-F 0 θ j (y1) " p0 i=1 π 0 i f θ 0 i y 2 -F θ 0 i (y 1 ) λ i (y 1 ) dy 1 dy 2
The last term can be proven immediately to be negative in either of the following cases:

• λ i (y 1 ) = λ (y 1 ) for all i ∈ {1, ..., p 0 } which leads to autoregressive mixture models already considered in Section 2.

• F θj (y 1 ) and F θ 0 i (y 1 ) are constant functions for j ∈ {1, ..., p}, i ∈ {1, ..., p 0 }, but this corresponds to hidden Markov chains already studied in [START_REF] Gassiat | Likelihood ratio inequalities with applications to various mixtures[END_REF]. Note that Theorem 2 applies in this case and it gives the probability distribution of the marginal likelihood ratio which is exactly the same as for mixture models (see [START_REF] Liu | Asymptotics for likelihood ratio tests under loss of identifiability[END_REF].

In the general case, however, there is no reason for the last integral to be negative. Some simulation results are presented for illustrating this last assertion.

Simulation results

Several two-regime models were considered, with transition matrices:

M 1 =
0.5 0.5 0.5 0.5 , M 2 = 0.9 0.1 0.1 0.9 and M 3 = 0.9 0.5 0.1 0.5 . The first transition matrix corresponds to independent regime switches. The regression functions are either linear, or constant. The latter cases correspond to hidden Markov chains.

The noise was considered normally distributed N 0, (0.5) 2 and the likelihood was penalized according to the BIC criterion. For every model, several sample sizes were considered (from 200 up to 2000 input values) and for each model and sample size, twenty different samples were simulated. In each case, Tables 1 and2 contain the estimated number of regimes (the maximum was fixed at three).

Simulation results prove that the penalized estimate p diverges when the true model is, for instance, a two-regime autoregressive Markov-switching model. This means that the cost function that was considered as a generalization of the "marginal likelihood" does not have the right properties to be a contrast function and the problem of estimating p 0 remains open in the general case of autoregressive Markov switching models. Table 1. Results for the "marginal-loglikelihood" BIC-penalized cost-function Table 2. Results for the exact-loglikelihood BIC-penalized cost-function

M 2 M 3 n p = 1 p = 2 p = 3 p = 1 p = 2 p = 3 F 0 1 (y) = 0.
1 n -1 n k=2 s (Y k-1 , Y k ) 2 = O P (1)
with S η defined by equation ( 4). Moreover, S η ⊂ L 2 (µ), thus S 2 η ⊂ L 1 (µ) and using the L 2 -entropy condition

S 2 _ = (s) 2 _ , g ∈ G η , with (s) _ (y k-1 , y k ) = min (0, s (y k-1 , y k )), is Glivenko-Cantelli. Since (Y k-1 , Y k
) is ergodic and strictly stationary, we obtain the following uniform convergence in probability:

inf s∈Sη 1 n -1 n k=2 (s) 2 _ (Y k-1 , Y k ) -→ n→∞ inf s∈Sη (s) _ 2 2
.

The following lemma is a straightforward adaptation of the inequality 1.1 in [START_REF] Gassiat | Likelihood ratio inequalities with applications to various mixtures[END_REF].

Lemma 2.

Under (B)

(8) sup g∈Gη :ln(g)-ln(g 0 )≥0 g -g 0 g 0 2 ≤ 2 sup g∈Gη n t=2 s g (Y t-1 , Y t ) n t=2 (s g ) 2 _ (Y t-1 , Y t )
One may apply this inequality to obtain [START_REF] Francq | Ergodicity of autoregressive processes with Markovswitching and consistency of the maximum likelihood estimator[END_REF] sup g∈Gη :ln(g)-ln(g 0 )≥0

g -

g 0 g 0 2 = O P n -1/2
Taylor expansion gives that ln(1 + u) = u -u 2 2 + u 2 R(u), with lim u←0 R(u) = 0. Thus, for any g,

ln(g) -ln(g 0 ) = g-g 0 g 0 2 n t=2 s g (Y t-1 , Y t ) -1 2 g-g 0 g 0 2 2 n t=2 (s g (Y t-1 , Y t )) 2 + g-g 0 g 0 2 2 n t=2 (s g (Y t-1 , Y t )) 2 R g-g 0 g 0 2 n t=2 s g (Y t-1 , Y t ) By (B), 1 n-1 n t=2 (s g (Y t-1 , Y t )) 2 = O P (1)
. Now, we have the following lemma:

Lemma 3 Let (F (X 1 ), • • • , F (X n )) be stationnary sequence of real random variables in L 2 then max i∈{1,••• ,n} (F (X i )) = o P ( √ n) Proof of lemma 3 Let us show that ∀ε > 0, lim n→∞ P (| max i∈{1,••• ,n} (F (X i ))| > ε √ n) = 0
We have

P (| max i∈{1,••• ,n} (F (X i ))| > ε √ n) ≤ P ({|F (X 1 )| > ε √ n} ∪ • • • ∪ {|F (X n )| > ε √ n}) ≤ nP (|F (X 1 )| > ε √ n) Now, since F (X 1 ) ∈ L 2 , lim n→∞ ∞ ε √ n F (x) 2 dP (x) + ε √ n -∞ F (x) 2 dP (x) = 0 Hence lim n→∞ n×P (|F (X 1 )| > ε √ n) ≤ lim n→∞ 1 ε 2 ∞ ε √ n F (x) 2 dP (x) + ε √ n -∞ F (x) 2 dP (x) = 0
Furthermore, since S η admits a square integrable envelop function F and using [START_REF] Francq | Ergodicity of autoregressive processes with Markovswitching and consistency of the maximum likelihood estimator[END_REF] we have:

sup g∈Gη :ln(g)-ln(g 0 )≥0 g -g 0 g 0 2 2 n t=2 (s g (Y t-1 , Y t )) 2 R g -g 0 g 0 2 n t=2 s g (Y t-1 , Y t ) = o P (1).
Thus,

sup g∈Gη ln(g) -ln(g 0 ) = sup g∈Gη g-g 0 g 0 2 n t=2 s g (Y t-1 , Y t ) -1 2 g-g 0 g 0 2 2 n t=2 (s g (Y t-1 , Y t )) 2 + o P (1), which implies that sup g∈Gη ln(g) -ln(g 0 ) ≤ sup g∈Gη:ln(g)-ln(g 0 )≥0 max P n t=2 sg (Yt-1,Yt) √ n ; 0 2 P n t=2 (sg (Yt-1,Yt)) 2 n + o P (1).
Since S η 2 is Glivenko-Cantelli:

sup g∈Gη n t=2 (s g (Y t-1 , Y t )) 2 n -1 = o P (1), and
2 sup g∈Gη ln(g) -ln(g 0 ) ≤ sup g∈Gη :ln(g)-ln(g 0 )≥0 max n t=2 s g (Y t-1 , Y t ) √ n ; 0 2 +o P (1)
.

Let G η n = g ∈ G η : g-g 0 g 0 ≤ n -1/4
. Using [START_REF] Francq | Ergodicity of autoregressive processes with Markovswitching and consistency of the maximum likelihood estimator[END_REF], we obtain that

2 sup g∈Gη ln(g) -ln(g 0 ) ≤ sup g∈Gη n max n t=2 s g (Y t-1 , Y t ) √ n ; 0 2 + o P (1). Now, sup g∈Gη n s g -D 2 ----→ n→∞ 0
, thus for a sequence u n decreasing to 0, and with

∆ n = s g -d : g ∈ G η n , d ∈ D, s g -d 2 ≤ u n ,
we obtain that 2 sup g∈Gη ln(g)ln(g 0 ) ≤ max sup d∈D

P n t=2 d(Yt-1,Yt) √ n + sup δ∈∆n P n t=2 δ(Yt-1,Yt) √ n ; 0 2 + o P (1).
Under (B), thanks to Theorem 3 of Doukhan (1995) the empirical process indexed by S η has the property of asymptotic stochastic equicontinuity, so:

sup δ∈∆n n t=2 δ (Y t-1 , Y t ) √ n = o P (1), and
2 sup g∈Gη ln(g) -ln(g 0 ) ≤ sup d∈D max n t=2 d (Y t-1 , Y t ) √ n ; 0 2 + o P (1).
Moreover, using classical normal asymptotic properties along the parametric paths, one obtains that, for a sequence of finite subsets D k increasing to D,

2 sup g∈Gη (ln(g) -ln(f )) ≥ sup d∈D k max n t=2 d (Y t-1 , Y t ) √ n ; 0 2 + o P (1).
for any k. Therefore, Theorem 1 is true.

4.2. Proof of Theorem 2. Let η > 0 be a real number. Consider Ĝn = ∅ the set of functions which maximize the log-likelihood. Since, under H-1, G is Glivenko-Cantelli, for n large enough, gg 0 L 2 (µ) < η for g ∈ Ĝn so Ĝn ⊂ G η . Let us remark that, under assumption H-2, the score function s g ∈ S η is well defined in a compact neighborhood of the true density function g 0 .

Proving that for an η > 0, a parametric family like S η is Donsker is not so easy. The problems arise when g → g 0 and the limits of s g in L 2 (µ) have to be computed. To achieve our proof, let us split S into two classes of functions.

For a sufficiently small ε > 0, we consider F 0 ⊂ G η , a neighborhood of g 0 ,

F 0 = g ∈ G, g g 0 -1 L 2 (µ) ≤ ε, g = g 0 . S is splitted into S 0 = {s g , g ∈ F 0 }
and S η \ S 0 .

On S η \ S 0 , it can be easily seen that

g1 g 0 -1 g1 g 0 -1 L 2 (µ) - g2 g 0 -1 g2 g 0 -1 L 2 (µ) L 2 (µ) ≤ 2 g1 g 0 -g2 g 0 L 2 (µ) g1 g 0 -1 L 2 (µ)
for every g 1 , g 2 ∈ G η \ F 0 and, moreover, by the definition of S 0 ,

g1 g 0 -1 g1 g 0 -1 L 2 (µ) - g2 g 0 -1 g2 g 0 -1 L 2 (µ) L 2 (µ) ≤ 2 ε g 1 g 0 - g 2 g 0 L 2 (µ)
On the other hand, by the assumption H-3, g g 0 has square-integrable partialderivatives of order one and, using the result 19.7 on parametric classes of functions in [START_REF] Van Der Vaart | Asymptotic Statistics[END_REF], we get:

N [•] (ε, S \ S 0 , • 2 ) = O 1 ε 2 D ,
where D is the number of parameters in the model.

It remains to prove that the bracketing number is a polynom of ( 1 ε ) for S 0 . The idea is to reparameterize the model in a convenient manner which will allow a Taylor expansion around the identifiable part of the true value of the parameters.

Let us recall that it is assumed that p 0 < p.

When g g 0 -1 = 0, by the linear independence of the functions g θj , a vector of positive integers t = (t i ) 0≤i≤p0 , t 0 = 0 exists so that:

θ ti-1+1 = ... = θ ti = θ 0 i , ti j=ti-1+1 π j = π 0 i , i ∈ {1, ..., p 0 }
With this remark, one can define in the general case s = (s i ) 1≤i≤p0 and q = (q j ) 1≤j≤p so that, for every i ∈ {1, ..., p 0 } , j ∈ {t i-1 + 1, ..., t i },

s i = ti j=ti-1+1 π j -π 0 i , q j = π j ti l=ti-1+1 π l
and a new parameterization will be

Θ t = (φ t , ψ t ) , φ t = (θ j ) 1≤j≤tp 0 , (s i ) 1≤i≤p0-1 , (π j ) p j=tp 0 +1 , ψ t = (q j ) 1≤j≤p , (θ j ) p j=tp 0 +1
with φ t containing all the identifiable parameters of the model and ψ t the nonidentifiable ones. Then, for g = g 0 , we will have:

φ 0 t = (θ 0 1 , ..., θ 0 1
, ..., θ 0 p0 , ..., θ 0 p0 , 0, ..., 0 0, ..., 0

t 1 t p0 -t p0-1 p 0 -1 p -t p0
) T

This reparameterization allows to write a second-order Taylor expansion of g g 0 -1 at φ 0 t . With the notations introduced in assumptions H, the density ratio becomes:

g g 0 -1 = p0 i=1 s i + π 0 i ti j=ti-1+1 q j l θj + p j=tp 0 +1 π j l θj -1
and since s p0 = -p0-1 i=1 s i ,

g g 0 -1 = p0-1 i=1 s i + π 0 i ti j=ti-1+1 q j l θj + π 0 p0 - p0-1 i=1 s i tp 0 j=tp 0 -1 +1 q j l θj + p j=tp 0 +1 π j l θj -1
By remarking that when φ t = φ 0 t , g g 0 does not vary with ψ t , we will study the variation of this ratio in a neighborhood of φ 0 t and for fixed ψ t . We can state the following result:

Proposition 3 Let us denote D (φ t , ψ t ) = g (φ t ,ψ t ) g 0 -1 L 2 (µ)
. With the notations of assumptions H-3 and H-4, for any fixed ψ t , the second-order Taylor expansion at φ 0 t exists such as

g g 0 -1 = φ t -φ 0 t T l ′ (φ 0 t ,ψt) + 1 2 φ t -φ 0 t T l ′′ (φ 0 t ,ψt) φ t -φ 0 t -1 + o (D (φ t , ψ t )) with φ t -φ 0 t T l ′ (φ 0 t ,ψt) = p0 i=1 π 0 i   ti j=ti-1+1 q j θ j -θ 0 i   T l ′ i + p0 i=1 s i l θ 0 i + p j=tp 0 +1 π j l θj and φ t -φ 0 t T l ′′ (φ 0 t ,ψt) φ t -φ 0 t = p0 i=1   2si   ti j=ti-1+1 q j θ j -θ 0 i   T l ′ i + +π 0 i ti j=ti-1+1 q j θ j -θ 0 i T l ′′ i θ j -θ 0 i   Moreover, φ t -φ 0 t T l ′ (φ 0 t ,ψt) + 1 2 φ t -φ 0 t T l ′′ (φ 0 t ,ψt) φ t -φ 0 t = 0 ⇔ φ t = φ 0 t Proof of Proposition 3
The first term in the development can be computed easily by remarking that the gradient of g g 0 -1 at φ 0 t , ψ t is:

• for i ∈ {1, ..., p 0 } and j ∈ {t i-1 + 1, ..., t i },

∂ "

g g 0 -1 " ∂θj φ 0 t , ψ t = π 0 i q j l ′ i • for i ∈ {1, ..., p 0 -1}, ∂ " g g 0 -1 " ∂si φ 0 t , ψ t = ti j=ti-1+1 q j l θ 0 i - tp 0 j=tp 0 -1+1 q j l θ 0 p 0 = l θ 0 i -l θ 0 p 0 • for j ∈ {t p0 + 1, ..., p}, ∂ " g g 0 -1 " ∂πj φ 0 t , ψ t = l j
The term of second order can be obtained by direct computations once the Hessian is computed at φ 0 t , ψ t :

• ∂ 2 " g g 0 -1 " ∂θ 2 j φ 0 t , ψ t = π 0 i q j l ′′ i , i = 1, ..., p 0 and j = t i-1 + 1, ..., t i • ∂ 2 " g g 0 -1 " ∂θj ∂θ l φ 0 t , ψ t = 0 , j, l = 1, ..., p and j = l • ∂ 2 " g g 0 -1 " ∂si∂s k φ 0 t , ψ t = 0 , i, k = 1, ..., p 0 -1 • ∂ 2 " g g 0 -1 " ∂si∂θj φ 0 t , ψ t = q j l ′ i , i = 1, ..., p 0 -1 and j = t i-1 + 1, ..., t i • ∂ 2 " g g 0 -1 " ∂si∂θj φ 0
t , ψ t = -q j l ′ p0 , i = 1, ..., p 0 -1 and j = t p0-1 + 1, ..., t p0 • the other crossed derivatives of s i and θ j are zero It still has to be proven that the rest is o φ tφ 0 t . As it can be easily seen that the third derivative of g g 0 -1 can be expressed in terms of partial derivatives of order two and three of l θ 0 j , j = 1, ..., p 0 , the result follows from the assumption H-3 and the linear independence in H-4.

Using the Taylor expansion above, we can now show that S 0 \ g 0 is a Donsker class, using the next result:

Proposition 4

Let d be the dimension of the parameter indexing the functions g θ . The number of ε-brackets N

[•] (ε, S 0 , • 2 ) covering S 0 is O 1 ε p0×(2d)+p .

Proof of Proposition 4

The idea of this proof is to bound N [•] (ε, S 0 , • 2 ) by the number of ε-brackets covering a wider class of functions. For every g ∈ F 0 , we will consider the reparameterization Φ = (φ t , ψ t ) which allows to write a second-order development of the density ratio:

g (φt,ψt) g 0 -1 = φ t -φ 0 t T l ′ (φ 0 t ,ψt) + 1 2 φ t -φ 0 t T l ′′ (φ 0 t ,ψt) φ t -φ 0 t + o (D (φ t , ψ t ))
Then, by remarking that the first two terms in the Taylor expansion are linear combinations of l θ 0 i , l ′ i , l ′′ i , i = 1, ..., p 0 and l θj , j = t p0 + 1, • • • , p, the density ratio can be written also as:

g (φt,ψt) g 0 -1 = p0 i=1 α i l θ 0 i + p j=tp 0 +1 α j l θj + p0 i=1 β T i l ′ i + p0 i=1 γ T i l ′′ i γ i + o (D (φ t , ψ t ))
where

(α i ) 1≤i≤p ∈ R, (β i ) 1≤i≤p0 and (γ i ) 1≤i≤p0 ∈ R d .
Now, using the linear independence, ∃m > 0 ,so that, for every

α j , j = 1, • • • , p, β i , γ i γ T i , i = 1, • • • , p 0 of norm 1, p0 i=1 α i l θ 0 i + p j=tp 0 +1 α j l θj + p0 i=1 β T i l ′ i + p0 i=1 γ T i l ′′ i γ i L 2 (µ) ≥ m.
At the same time, since

g (φ t ,ψ t ) g 0 -1 g (φ t ,ψ t ) g 0 -1 L 2 (µ) L 2 (µ) = 1
we will obtain that the Euclidean norm of the coefficients in the second-order development of

g (φ t ,ψ t ) g 0 -1 ' ' ' g (φ t ,ψ t ) g 0 -1 ' ' ' L 2 (µ)
is upper bounded by 1 m . This fact implies that S 0 can be included in

H = p0 i=1 α i l θ 0 i + β T i l ′ i + γ T i l ′′ i γ i + p j=tp 0 +1 α j l θj + o (1) , α j , j = t p0 + 1, • • • , p, β i , γ i γ T i , i = 1, • • • , p 0 ≤ 1 m and then obviously N [•] (ε, H, • 2 ) = O 1 ε p0×2d+p+1 .
Since the set S η was proved to be Donsker, it remains to identify the asymptotic index set of score functions.

Asymptotic index set. The set of limit score functions F is defined as the set of functions d so that one can find a sequence g n satisfying gn-f f 2 → 0 and ds gn 2 → 0.

Let us define the two principal behaviors for the sequences g n which influence the form of functions d :

• If the second order term is negligible with respect to the first one :

g n g 0 -1 = (Φ n -Φ 0 ) T l ′ (Φ 0 t ,ψn) + o(D(Φ n , ψ n )).
• If the second order term is not negligible with respect to the first one :

gn g 0 -1 = (Φ n -Φ 0 ) T l ′ (Φ 0 t ,ψn) + 0.5(Φ n -Φ 0 ) T l ′′ (Φ 0 ,ψn) (Φ n -Φ 0 ) + o(D(Φ n , ψ n )).
In the first case, a set t = (t 0 , • • • , t p0 ) exists so that the limit function of s gn will be in the set:

D t 1 = Ω p0 i=1 ζ T i l θ 0 i + p i=p0+1 ζ T i l θi + p0 i=1 λ T i l ′ i λ 1 , • • • , λ p0 ∈ R d ; ζ 1 , • • • , ζ p ∈ R θ tp 0 +1 , • • • , θ p ∈ Θ -θ 0 1 , • • • , θ 0 p0
In the second case, an index i exists so that : ti j=ti-1+1 q j (θ jθ 0 i ) = 0, Otherwise, the second order term will be negligible compared to the first one, so ti j=ti-1+1 √ q j × √ q j (θ jθ 0 i ) = 0.

Hence, a set a set t = (t 0 , • • • , t p0 ) exists so that the set of functions d will be:

Ω p0 i=1 ζ i l θ 0 i + p i=p0+1 ζ i l θi + p0 i=1 λ T i l ′ i +δ p0 i=1 ti j=ti-1+1 γ T j l ′′ i γ j λ 1 , • • • , λ p0 , γ 1 , • • • , γ tp 0 ∈ R d ; ζ 1 , • • • , ζ p ∈ R θ tp 0 +1 , • • • , θ p ∈ Θ -θ 0 1 , • • • , θ 0 p0
where δ = 1 if there exists a vector q exists so that: q j ≤ 0, ti j=ti-1+1 q j = 1, ti j=ti-1+1 √ q j γ t j = 0 for i = 1, • • • , p 0 ; and δ = 0 otherwise. So, the limit functions will belong to F. Conversely, let d be an element of F, as functions d belong to the Hilbert sphere, one of their components is not equal to 0. Let us assume that this component is ζ 1 , but the proof would be similar with any other component. The norm of d is 1, so any component of d is determined by the ratio: ζ2 ζ1 , • • • , 1 ζ1 γ p0 .

Then, by assumption H-1, the set of possible parameters contains a neighborhood of the parameters realizing the true conditional density function g 0 , we can chose the parameters of g n so that:

∀i ∈ {2, • • • , p 0 } :

P t i j=t i-1 +1 π n j -π 0 i P t 1 j=1 π n j -π 0 1 n→∞ -→ ζi ζ1 ,
∀i ∈ {1, • • • , p 0 } :

P t i j=t i-1 +1 q n j (θ n j -θ 0 i ) P t 1 j=1 π n j -π 0 1 n→∞ -→ 1 ζ1 λ i , ∀j ∈ {1, • • • , t p0 } : √ q n j P t 1 j=1 π n j -π 0 1 θ n j -θ 0 i n→∞ -→ 1 ζ1 γ j , ∀i ∈ {p 0 + 1, • • • , p} : π n i P t 1 j=1 π n j -π 0 1 n→∞ -→ 1 ζ1 ζ i .

Proof of Proposition 1

Since the noise is Gaussian and a 0 i < 1 for every i ∈ {1, ..., p 0 }, by [START_REF] Yao | On stability of nonlinear AR processes with Markov switching[END_REF], there exists a unique strictly stationary and geometrically ergodic solution, which in particular will be geometrically β-mixing.

On the other hand, the Gaussian noise implies the existence of moments of any order. Now let us prove the existence of an exponential moment for Y t . By denoting σ = max i=1,...,p0 σ 0 i , ρ = max i=1,...,p0 a 0 i < 1, b = max i=1,...,p0 b 0 i and for s ∈ N ⋆ , one has : Since ρ < 1 and E |ε t | 2s ≥ E ε 2 t = 1, we finally obtain

|Y t | 2s = F 0 θX t (Y t-1 ) + ε θX t (t)
E |Y t | 2s 1 2s ≤ b + σE |ε t | 2s 1 2s 1 -ρ ≤ b + σ 1 -ρ E |ε t | 2s 1 2s
The exponential moment can be computed then by

E e δY 2 t = ∞ k=0 E |Y t | 2k k! δ k ≤ ∞ k=0 E |ε t | 2k k! δ b + σ 1 -ρ 2 k
The last term being the moment generating function of a χ 2 (1)-distribution, it will be finite for any δ such that 0 < δ < 1 2 1-ρ b+σ

2

.

Proof of Proposition 2

The norm of the generalized score function is for all i ∈ {1, ..., p}. On the other hand, since p0 j=1 π 0 j f 0 θj y 2 -F 0 θj (y 1 ) ≥ π 0 k f 0 θ k y 2 -F 0 θ k (y 1 ) for every k ∈ {1, ..., p 0 }, the generalized score function is well defined if for every i ∈ {1, ..., p}, there exists k ∈ {1, ..., p 0 } such that

g g 0 -1 L 2 (µ) = g 2 (

√ 2 - 1 m

 21 , we get lim m→∞ ν θm (Y 1 , Y 2 ) P → 0 and the LRTS is divergent.

2s≤ρ

  (ρ |Y t-1 | + b + σ |ε t |) 2s ≤ ... ≤ ≤ b + σ |ε t | + ∞ k=1 ρ k (b + σ |ε t-k |) b + σ |ε t-k |) k b + σE |ε t-k | 2s 1 2s

  y 1 , y 2 ) g 0 (y 1 , y 2 )dy 2 dλ (y 1 ) -1 = = ( p i=1 π i f θi (y 2 -F θi (y 1 ))) -F 0 θj (y 1 ) dy 2 dλ (y 1 ) -1and by the inequality( p i=1 π i f θi (y 2 -F θi (y 1 ))) 2 ≤ p i=1 π i f 2 θi (y 2 -F θi (y 1 )), the integral will be finite if f 2 θi (y 2 -F θi (y 1 ))

	2
	p0 j=1 π 0 j f 0 θj y 2 p0 j=1 π 0 j f 0 θj y 2 -F 0

θj (y 1 ) dy 2 dλ (y 1 ) < ∞

Since the set G is Glivenko-Cantelli, for all η > 0 and for n large enough: ĝn ∈ G η , where G η is defined by equation (3). Now, using Theorem 1 of [START_REF] Doukhan | Mixing: properties and examples[END_REF], under (B) [START_REF] Doukhan | Invariance principles for absolutely regular empirical processes[END_REF] sup s∈Sη


Olivier Strauss

Sebastien Destercke

F-boxes for filtering

Keywords: Possibility measures, maxitive kernels, pboxes, clouds, signal filtering

Selecting a particular summative kernel (i.e., formally equivalent to a probability distribution) when filtering a digital signal can be a difficult task. An obvious solution to this problem is to filter with multiple kernels rather than with a single one, but the computing cost of such a solution can quickly become prohibitive (especially in real-time applications). Another alternative, the one studied in this paper, is to consider kernels modeled by imprecise probabilistic representations. Considering such representations makes the use of numerical tools coming from imprecise probability theory possible, such as the Choquet integral, and allows one to work with multiple kernels without multiplying too much the number of required computations. In this paper, we propose to use the well-known p-box representation to filter a digital signal. We show that the use p-boxes allows making more precise inferences than those obtained with possibility distributions and clouds. We then discuss the practical aspect of computing a filtered signal with pboxes, and finish by some experiments.

Introduction

Reconstructing a continuous signal from a set of sampled and possibly corrupted observations is a common problem in both digital analysis and signal processing [START_REF] Jan | Digital Signal Filtering, Analyses and Restoration[END_REF]. In this context, kernel-based methods can be used for different purposes: reconstruction, impulse response modelling, interpolation, linear and non-linear transformations, stochastic or band-pass filtering, etc.

Most of the kernels used in signal processing are summative kernels, or linear combination of summative kernels. A summative kernel is a positive function with an integral equal to one. A summative kernel can therefore be associated to a particular probability distribution. However, how to choose a particular kernel and its parameters to filter a given signal is often a tricky question. In this paper, we focus more particularly on symmetric and unimodal kernels, as they are often used in practice.

One solution to overcome this difficulty is to use imprecise probabilistic models to model in a single representation a whole set of possible kernels. This can be done thanks to the correspondence between sets of summative kernels and sets of probabilities. Existing propositions [START_REF] Loquin | On the granularity of summative kernels[END_REF] have explored the use of representations such as possibility distributions [START_REF] Dubois | Possibility Theory: An Approach to Computerized Processing of Uncertainty[END_REF][START_REF] Dubois | When upper probabilities are possibility measures[END_REF] or clouds [START_REF] Neumaier | Clouds, fuzzy sets and probability intervals[END_REF] to derive respectively so-called maxitive and cloudy kernels.

These representations have the important advantage of being computationally tractable and are particularly interesting when the kernels to model all share a common modal value.

This paper is focused on another kernel based on an imprecise probabilistic representation, namely p-boxes. Indeed, the interest of using p-boxes for filtering purposes have not been investigated yet, despite the fact that they present the same advantages as clouds and possibility distributions (simplicity and tractability). P-boxes consist in pairs of cumulative distributions, and have the same computational advantages as clouds or possibility distributions. We will show in this paper that they can also provide more precise results than maxitive or cloudy kernels.

Filtering with such representations require to use imprecise extensions of expectation operators, which result in an an interval-valued filtered signal rather than a classical single-valued one. This signal corresponds to the lower and upper envelopes of filtered signals that would have been obtained by filtering with every single summative kernel contained in the imprecise probabilistic representation. Using such representations is therefore particularly interesting to achieve a robustness analysis in one operation.

Section 2 presents the material concerning summative kernels and imprecise probabilistic representations needed in this paper. The capacity of p-boxes to model symmetric kernels and their relation with other representations is studied in Section 3. Section 4 deals with the computational aspects of p-box based filtering. Finally, Section 5 presents some basic experiments on simulated noisy signals.

Summative and imprecise kernels

In this section, we recall the basics about summative kernels, before introducing kernels based on imprecise probabilistic representations. We restrict ourselves to representations defined on the real line R and its discretization X .

Summative kernels and probability sets

A summative kernel κ is formally equivalent to a Lebesgue-measurable probability distribution κ : R → R + and can be interpreted as such. The associated probability measure P κ : B → [0, 1] defined on the real Borel agebra B is such that, for any measurable subset A ⊆ R (also called an event), P κ (A) = A κ(x)dx. Some typical summative kernels are recalled and represented in Table 1.

Name

κ Shape Epanechnikov κ(x) = 1 ∆ 3 4 (1 -( x ∆ ) 2 )I ∆ x 0 Triangular κ ∆ (x) = (1 -| x ∆ |)I ∆ x 0 Uniform κ ∆ (x) = 1 2∆ I ∆ x 0 Table 1: Some classical summative kernels
As said in the introduction, the choice of a particular summative kernel to filter a given signal can be tricky. This is why kernels based on imprecise probability representations can help in filtering with sets of kernels at once.

We call imprecise kernel a convex set P of summative kernels. Since summative kernels are equivalent to probability distributions, P is equivalent to a convex set of probability distributions. Similarly to the probability measure of a summative kernel, to any event A ⊆ R can be associated two measures, i.e., the lower and upper probability measures induced by P, which are such that P(A) = inf κ∈P P(A) and P(A) = sup κ∈P P(A).

These two measures are dual, in the sense that P(A) = 1 -P(A c ), with A c the complement of A. Hence it is sufficient in practice to work with only one of them specified over all events. Handling a generic imprecise kernel P can be a difficult task, however simpler imprecise probabilistic representations that satisfy some properties can make this handling much easier. We introduce such easy-to-use representations in the next sections.

Possibility distributions and maxitive kernels

A maxitive kernel π is defined as a possibility distribution [START_REF] Dubois | Possibility Theory: An Approach to Computerized Processing of Uncertainty[END_REF], that is a mapping π : R → [0, 1] with at least one x ∈ R such that π(x) = 1. It induces two (lower and upper) confidence measures, respectively called necessity and possibility measures. They are such that, for any event A ⊆ R, we have:

Π(A) = max x∈A π(x) N(A) = 1 -Π(A c ). (1)
A maxitive kernel π induces an imprecise kernel P π including all summative kernels dominated by the possibility measure Π, i.e., P π = {κ ∈ P R |∀A ⊆ R, P(A) ≤ Π(A)}, with P R the set of all summative kernels over R. If a summative kernel κ is in P π , we say by a small abuse of language that π includes κ. This interpretation makes maxitive kernels instrumental tools to filter signals when the identification of a single summative kernel is difficult.

Clouds and cloudy kernels

Maxitive kernels are very simple imprecise kernels, since they consist in a single mapping over elements x ∈ R. While this makes them very easy to build and to compute with, it also limits their expressiveness. For instance, they cannot model sets of summative kernels whose bandwidth is both lower-and upper-bounded. Clouds [START_REF] Neumaier | Clouds, fuzzy sets and probability intervals[END_REF] are a slightly more complex representation with increased expressiveness, but they remain simple enough so that computing with them remains tractable. A cloudy kernel or cloud is defined as a pair of mappings [π, η] from R to the unit interval [0, 1] such that η ≤ π and there is at least one element x ∈ R such that π(x) = 1 and one element y ∈ R such that η(y) = 0.

A cloudy kernel [π, η] induces an imprecise kernel P [π,η] such that

P [π,η] = {κ ∈ P R |P κ (η α ) ≤ 1 -α ≤ P κ (π α )}, (2)
where η α = {x|η(x) ≥ α} and π α = {x|π(x) > α}.

Maxitive kernels are special kinds of cloudy kernels, as a maxitive kernel π is equivalent to a cloudy kernel [π, η] where η = 0. Also note that, mathematically speaking, clouds are equivalent to interval-valued fuzzy sets having boundary conditions (i.e., π(x) = 1 and η(y) = 0 for some (x, y) ∈ R 2 ). A family of clouds that will be of particular interest here are the comonotonic clouds [START_REF] Destercke | Unifying practical uncertainty representations: II. Clouds[END_REF]. They are defined as follows:

Definition 1. A cloud is comonotonic if ∀x, y ∈ R, π(x) < π(y) ⇒ η(x) ≤ η(y)
Indeed, such clouds possess the properties [START_REF] Destercke | Unifying practical uncertainty representations: II. Clouds[END_REF] that makes cloudy kernels tractable representations.

P-boxes and f-boxes

Let us now introduce the imprecise kernels based on pboxes, that we call f-boxes (for filter boxes).

A f-box [F, F] is formally equivalent to a p-box [START_REF] Ferson | Constructing probability boxes and dempster-shafer structures[END_REF][START_REF] Destercke | Unifying practical uncertainty representations: I. Generalized p-boxes[END_REF], and is defined as a pair of cumulative distributions, a lower one F : R → [0, 1] and an upper one F : R → [0, 1], such that F ≤ F and there is at least one element x ∈ R with F(x) = F(x) = 1. A f-box induces an imprecise kernel P [F,F] defined as follows

P [F,F] = {κ ∈ P R |∀x ∈ R, F(x) ≤ P κ ([-∞, x]) ≤ F(x)}.
A f-box encompasses every summative kernel whose cumulative distribution lies between F and F.

Modeling symmetric kernels with imprecise kernels

As said in the introduction, in this paper we are mainly concerned by imprecise kernels modeling sets of kernels that are symmetric and unimodal. We assume that the shape of the kernels to model is known and parametrised by a bandwidth (i.e., the support of the corresponding probability). We also assume that this bandwidth is illknown, and that it is only lower and upper bounded. We therefore assume from now one that κ belongs to a family of symmetric kernel centered around zero (i.e., κ(x) = κ(-x)). A basic symmetric kernel κ has [-1, 1] for support. From this basic kernel it is easy to derive a summative kernel κ ∆ with bandwidth ∆ and support [-∆, ∆] using the transformation κ ∆ (x) = 1 ∆ κ( x ∆ ) (see Table 1 for illustrations).

Maxitive and cloudy kernels modeling: a reminder

First, recall that the cumulative distribution

F κ ∆ : [-∆, ∆] → [0, 1] of κ ∆ is such that F κ ∆ (0) = 1/2 and F(x) + F(-x) = 1.
Let us now briefly remind how a set of kernels κ ∆ whose bandwidth is only known to lie in the interval [∆] := [∆ inf , ∆ sup ] can be modeled by maxitive or cloudy kernels. We will denote by κ [∆] this set of kernels.

There are many ways to build a maxitive kernel including the set κ [∆] [START_REF] Baudrit | Practical representations of incomplete probabilistic knowledge[END_REF], but the most specific solution is given by the Dubois-Prade transformation [START_REF] Dubois | On possibility/probability transformations[END_REF]. For κ [∆] , this transformation gives the distribution π

[∆] such that π [∆] (x) = 2 * F ∆ sup (x) if x ≤ 0 2 * (1 -F ∆ sup (x)) if x > 0. (3)
We will denote by π

+ [∆] , π - [∆] the following functions π - [∆] (x) = π [∆] (x) if x ≤ 0 1 if x > 0 π + [∆] (x) = 1 if x ≤ 0 π [∆] (x) if x > 0. ( 4
)
The induced imprecise kernel

P π [∆] includes, among others, all summative kernels κ ∆ with ∆ ∈ [0, ∆ sup ] [9].
This means that maxitive kernels alone cannot take account of the lower bound ∆ inf , and include for example the Dirac function. This can be a problem, for instance in those cases where it is desirable to smoothen a signal or where the interval-valued signal resulting from an imprecise filtering should not envelope the initial noisy signal.

By supplementing the (upper) distribution π with a (lower) distribution η, cloudy kernels can solve this insufficiency. In [START_REF] Destercke | Using cloudy kernels for imprecise linear filtering[END_REF], we proposed the following cloudy

x 0 1 η -η + π - π + ∆ inf ∆ sup Figure 1: Example of cloudy kernel x 0 1 F F ∆ inf ∆ sup F = max{F ∆ sup , F ∆ inf } F = min{F ∆ sup , F ∆ inf } Figure 2: Example of f-box kernel [π [∆] , η [∆] ] to approximate κ [∆] : π [∆] (x) = 2 * F ∆ sup (x) if x ≤ 0 2 * (1 -F ∆ sup (x)) if x ≥ 0 (5) η [∆] (x) = 2 * F ∆ inf (x) if x ≤ 0 2 * (1 -F ∆ inf (x)) if x ≥ 0 (6)
Similarly to maxitive kernels, we denote by η

+ [∆] , η - [∆] the functions such that η - [∆] (x) = η [∆] (x) if x ≤ 0 1 if x > 0 η + [∆] (x) = 1 if x ≤ 0 η [∆] (x) if x > 0. (7)
A cloudy kernel corresponding to a family of triangular kernel κ is pictured in Figure 1 . Cloudy kernels therefore allow to solve some issues related to maxitive kernels (but have problems of their own, as we shall see in Section 5).

F-box kernels

As illustrated by Figure 1, both maxitive and cloudy kernels are symmetric and are built around the modal value (here, 0). Rather than considering this modal value as a central point, the f-box approach focuses on events of the kind [-∞, x]. Such a view suggests different approximations of the set κ [∆] , that we explore here. Given the set of kernels κ [∆] , the most specific f-box

(F [∆] , F [∆]
) one can build that encompasses this set is

F [∆] (x) = max{F ∆ sup (x), F ∆ inf (x)}, (8)
F [∆] (x) = min{F ∆ sup (x), F ∆ inf (x)}, (9)
as shows the next theorem:

Theorem 1. (F [∆] , F [∆]
) is the most specific f-box including the set of kernels κ [∆] , and does not include any kernel

κ ∆ with ∆ ∈ [∆]. Proof. (Sketch) Consider a kernel κ ∆ with ∆ ∈ [∆]. It is included in (F [∆] , F [∆] ), as min{F ∆ sup , F ∆ inf } ≤ F ∆ ≤ max{F ∆ sup , F ∆ inf },
meaning that both κ ∆ sup and κ ∆ inf are in the kernel. To show that it is the most specific, assume ex absurdo that there is a more specific f-box

(F , F ) including κ [∆] . This means that ∃x such that F (x) > F [∆] (x) or F (x) < F [∆] (x), implying that either F ∆ sup ∈ (F [∆] , F [∆] ) or F ∆ inf ∈ (F [∆] , F [∆] ), which is a contradiction since both κ ∆ sup and κ ∆ inf are in κ [∆] . To show that no kernel κ ∆ with ∆ ∈ [∆] is in (F [∆] , F [∆]
), simply takes the same argument (there is an x for which

F ∆ (x) < F [∆] (x) or F ∆ (x) > F [∆] (x)).
Figure 2 shows a f-box given by Eq. ( 8) and ( 9) for triangular kernels. We can see in this picture that, although there is some symmetry in the representation, it is not symmetric around the 0 coordinate axis.

The next theorem shows that f-boxes modeling symmetric unimodal kernels are more specific than maxitive or cloudy kernels given by Eq. ( 3) and ( 6), and will therefore provide more precise results when filtering. around events [-x, x] (see [START_REF] Neumaier | Clouds, fuzzy sets and probability intervals[END_REF] and [START_REF] Couso | The necessity of the strong alpha-cuts of a fuzzy set[END_REF] for details).

Theorem 2. (F [∆] , F [∆] ) is strictly more specific than the maxitive kernel π [∆] or the cloudy kernel [π [∆] , η [∆] ], in the sense that P (F [∆] ,F [∆] ) ⊆ P [π [∆] ,η [∆] ] ⊆ P π [∆] ,
To show that (

F [∆] , F [∆]
) is more precise than both of them, it is then sufficient to show that lower and upper probabilities induced by (F [∆] , F [∆] ) over events [-x, x] are tighter or coincide with the one induced by π

[∆] and [π [∆] , η [∆] ].
Using Eq. ( 5) and ( 6), we get that

P π [∆] ([-x, x]) = P [π [∆] ,η [∆] ] ([-x, x]) = 1 -2F ∆ sup (-x) = F ∆ sup (x) -F ∆ sup (-x) and P [π [∆] ,η [∆] ] ([-x, x]) = 1 -2F ∆ inf (-x) = F ∆ inf (x) - F ∆ inf (-x) (the upper probability bound for π [∆]
on such events is one).

Figure 3 shows that the lower probability

P (F [∆] ,F [∆] ) has F ∆ sup (x) -F ∆ sup (-x)
for upper bound (in fact, this is the exact value). The same reasoning can be done for the upper probability.

To see that the inclusion is usually strict, remark that the kernel which has function π

- [∆] for cumulative dis- tribution is not in (F [∆] , F [∆] ), while it is in π [∆] and [π [∆] , η [∆] ].
Finally, we can show that any kernel whose cumulative distributions lies in Proof. (Sketch) First consider the minimal bandwidth

(F [∆] , F [∆] ) has a bandwidth in [∆] x 0 1 F F -x x P (F [∆] ,F [∆] ) ([-x, x])
∆ inf . Since for all x ∈ [-∆ inf , ∆ inf ], 0 < F [∆] (x) < F [∆] (x) < 1, no cumulative distribution in (F [∆] , F [∆] ) can have a bandwidth lower than ∆ inf . Consider now ∆ sup . As F [∆] (x) = 0 for any x < -∆ sup and F [∆] (x) = 1 for any x > ∆ sup , no cumulative distri- bution in (F [∆] , F [∆]
) can have a bandwidth higher than ∆ sup .

Practical computations

We can now discuss how filtering can actually be achieved with f-boxes while maintaining a low computational complexity. This can be done by extending classical expectation operators (equivalent to filter with summative kernels) through the use of Choquet integrals. We first recall what is a Choquet integral [START_REF] Choquet | Theory of capacities[END_REF][START_REF] Denneberg | Fuzzy Measures and Integrals -Theory and Applications, chapter Non-additive measure and integral, basic concepts and their role for applications[END_REF] and its links with expectation operators, summative kernels and maxitive kernels. We then propose an efficient algorithm allowing to compute this Choquet integral for f-boxes. To make notations shorter, we will from now on denote F [∆] , F [∆] by F and F, assuming that a kernel family as well as maximal and minimal bandwidth have been specified.

Since computations are achieved on a discretised space, we consider in this section that we are working on a finite domain X of N elements. In our case, this space corresponds to a finite sampling of the (continuous) signal.

Expectation operator and Choquet integral

Let X = {x 1 , . . . , x N } be the discretized domain, where x 1 corresponds to the last sampled time and x N is the sampled item with time horizon N. Consider a realvalued function f (here, the sampled values of the signal) on X , together with a discretized summative kernel κ i , i = 1, . . . , N, where κ i = κ(x i ).

Classical causal convolution between the kernel κ and the sampled signal f is equivalent to apply an expectation operator, i.e. to compute E κ ( f ) such that

E κ ( f ) = N ∑ i=1 κ i f (x i )
Now, when working with a set P of kernels defined on X , expectation operator E becomes imprecise, and its result when applied to f is an interval-valued expectation

[E( f ), E( f )] such that E( f ) = inf κ∈P E κ ( f ) E( f ) = sup κ∈P E κ ( f ) (10)
In general, these bounds are not easy to compute. However in some specific cases practical tools exists that makes them computable. In particular, if there is no loss of information when considering the set of summative kernels dominated by the lower probability of P on events alone, and if the lower probability P satisfies a property of 2-monotinicity, that is if for any pair {A, B} ⊆ X we have P(A ∩ B) + P(A ∪ B) ≥ P(A) + P(B), then Eq. ( 10) can be solved by using the Choquet integral.

Consider a positive bounded function 1 f on X . If we denote by () a reordering of elements of X such that f (x (1) ) ≤ . . . ≤ f (x (N) ), Choquet Integrals giving lower and upper expectations are given by

C P ( f ) = E( f ) = N ∑ i=1 ( f (x (i) ) -f (x (i-1) )P(A (i) ), C P ( f ) = E( f ) = N ∑ i=1 ( f (x (i) ) -f (x (i-1) )P(A (i) ),
with f (x (0) ) = 0 and A (i) = {x (i) , . . . , x (N) }. The main difficulty then becomes to compute the lower and upper confidence measures for the N sets A i .

Imprecise expectations with f-boxes

Since p-boxes, or equivalently f-boxes induce lower confidence measures that are ∞-monotone [START_REF] Destercke | Unifying practical uncertainty representations: I. Generalized p-boxes[END_REF] (an even more restrictive property than 2-monotonicity), Choquet integral can be used to compute their expectation bounds. Let us now see how the lower confidence measure value on various events can be computed efficiently (upper confidence measure values can then be computed by duality).

In the continuous case, when a f-box is defined by two continuous fucntions F and F over R, the lower probability P([x, y]) of an interval C = [x, y] is given by the following formula

P(C) = max{0, F(y) -F(x)},
and the lower probability of the union A of any set of n disjoint intervals C i = [x i , y i ], i = 1, . . . , n is given by

P(A = n i=1 C i ) = n ∑ i=1 P(C i ). ( 11
)
Note that any event A can be written in this form.

In the discretised case where X = {x 1 , . . . , x N }, an interval C = [x, y] becomes a set C = [x i , x j ] where C = {x k |i ≤ k ≤ j} contains every discretised elements x k 1 Assuming positivity is not constraining here, since if c is a constant E( f + c) = E( f ) + c and the same holds for E. Therefore any bounded function can be made positive by a simple translation. between x i and x j . In this case, the lower probability of C becomes P(C) = max{0, F(x j ) -F(x i-1 )}, and Eq. ( 11) remains the same, and again any event A can be written as the union of disjoint intervals (two intervals

[x i , x j ], [x k , x l ] are considered disjoint if j + 1 < k, otherwise if k = j + 1 their union is no longer disjoint).
Algorithm 1 describes how to compute lower confidence measures and the incremental summation giving the lower expectation, while Example 1 provides an illustration of the process. At each step, the disjoint intervals of the form C 1 , . . . ,C M i forming A (i) are extracted and the corresponding lower confidence measure is computed. The value of the Choquet integral is then incremented. Note that two orderings and indexing are used in the algorithm: the one where elements are ordered by values of f , denoted by (), and the other where elements are ordered w.r.t. to their sampling time ( f (x 1 ) being the first sampled value and f (x N ) the last one).

x 0

1 x 8 x 7 x 6 x 5 x 4 x 3 x 2 x 1 F(x 6 ) F(x 6 )
for i = 1, . . . , N do Compute f (x (i) ) -f (x (i-1) ) ; Extract intervals C 1 , . . . ,C M i such that A (i) = C 1 ∪ . . . ∪C M i ; With C j = [x j , x j ] ; Compute P(A (i) ) = ∑ M i j=1 max(0, F(x j ) -F(x j-1 )) ; E = E + ( f (x (i) ) -f (x (i-1) ) × P(A (i) )
Example 1. Consider the situation pictured in Fig. 4. For each sampled item x and associated sampled value f (x), we consider that the corresponding distribution values are [F(x), F(x)] (More conservative discretizations can be adopted, but do not change the problem). The values of discretized f , F and F are summarized in Table 2.

If we now apply Algorithm 1 with the lower probability, we get the following steps:

1. i=1, f (x (i) ) -f (x (i-1) ) = 2, A (i) = X , P(A (i) ) = 1 → E = 2 2. i=2, f (x (i) ) -f (x (i-1) ) = 3, A (i) = C 1 = {x 2 , . . . , x 8 }, P(A (i) ) = max{0, F(x 8 ) -F(x 1 )} = 0.936 → E = 4.
x (5) x (4) x (6) x (2) x (3) x (7) x (8)
Table 2: Values for example 1

3. i=3, f (x (i) ) -f (x (i-1) ) = 1, A (i) = C 1 ∪ C 2 = {x 2 , . . . , x 4 } ∪ {x 6 , x 7 , x 8 }, P(A (i) ) = max{0, F(x 4 )-F(x 1 )}+max{0, F(x 8 )-F(x 5 )} = 0.469 → E = 5.277 4. i=4, f (x (i) ) -f (x (i-1) ) = 2, A (i) = C 1 ∪ C 2 = {x 2 , . . . , x 4 } ∪ {x 7 , x 8 }, P(A (i) ) = max{0, F(x 4 ) - F(x 1 )} + max{0, F(x 8 ) -F(x 6 )} = 0.436 → E = 5.277 + 2 × 0.436 = 6.149 5. i=5, f (x (i) ) -f (x (i-1) ) = 1, A (i) = C 1 ∪ C 2 ∪ C 3 = {x 2 } ∪ {x 4 } ∪ {x 7 , x 8 }, P(A (i) ) = max{0, F(x 2 ) - F(x 1 )}+max{0, F(x 4 )-F(x 3 )}+max{0, F(x 8 )- F(x 6 )} = 0.061 → E = 6.21 6. i=6, f (x (i) ) -f (x (i-1) ) = 1, A (i) = C 1 ∪ C 2 = {x 4 }∪{x 7 , x 8 }, P(A (i) ) = max{0, F(x 4 )-F(x 3 )}+ max{0, F(x 8 ) -π(x 6 )} = 0 → E = 6.21 7. i=7, stop, since P(A (i-1) ) = 0
The lower expectation is finally 6.21.

Experiments

This section aims at illustrating the behaviours of the different imprecise kernels presented in Section 2 when filtering a noisy signal. We compare four approaches: the conventional precise method (using only one sumkernel) and the imprecise filtering method performed with maxitive, cloudy and f-box kernels.

Experiment description

The comparison is achieved on the signal shown in Figure 5. This signal is made of 10 sine signals whose frequencies were randomly chosen (in a manner that subsequent sampling would respect the Shannon sampling theorem [START_REF] Elwood | Communication in the presence of noise[END_REF]), with each signal weighted by a randomly chosen weighting factor. Finally, the signal has been quantized and altered by a Poisson noise (so that the noise is neither ergodic nor time invariant). Figure 5 gives the plot of the noisy signal s (in cyan) and of the original signal f (in black).

Assume that the practitioner has decided that the appropriate kernel shape was of the form κ ∆ (x) = e The maxitive kernel chosen for this experiment is the most specific that dominates κ 0.05 . It is built after Eq. (3) (see [START_REF] Loquin | On the granularity of summative kernels[END_REF] for a more generic procedure using the same arguments).

The cloudy kernel is built after Eq. ( 5) and ( 6), as explained in [START_REF] Destercke | Using cloudy kernels for imprecise linear filtering[END_REF]. The lower distribution η is built using κ 0.045 , while the upper distribution π is built using κ 0.05 .

We finally compute a f-box kernel according to the method described in this paper. Using Eq.( 8) and ( 9), we define F and F as follows.

∀x ∈ R, let F 0.045 (x) = x -∞ κ 0.045 (u)du and F 0.050 (x) =

x -∞ κ 0.050 (u)du, then F(x) = min(F 0.045 (x), F 0.050 (x)) and F(x) = max(F 0.045 (x), F 0.050 (x)).

Discussion on experiment results.

The use of imprecise filtering (with maxitive, cloudy or f-box kernels) is recommended in those cases where the practitioner does not know the exact shape of the impulse response of the filter. When the practitioner approximately knows the shape of the filter but can only give its upper bandwidth, then the most appropriate representation is the maxitive kernel. When the practitioner can also give a lower bandwidth, then the cloudy kernel is an appropriate and more specific representation that leads to a more specific interval-valued filtered output. However, the cloudy representation has some theoretical and practical limitations. First, like the maxitive representation, it is built around a modal value and its extension to other situations often provide unsatisfying results. Second, experimental practice shows that it can lead to very chaotic bounds, even if a smoothing filter has been chosen initially by the practitioner. As we shall see here, the f-box representation seems to overcome these problems and leads to a more specific and less chaotic imprecise valued filtered signal. This can be explained by the better properties, in terms of approximation of the set κ [∆] , of f-box kernels. cise valued filtered signals.

As expected, the f-box based imprecise valued filtered signal is included in the imprecise valued filtered signals obtained with the maxitive (see Figure 8) and the cloudy kernels (see Figure 9). The inclusion of the cloudy based imprecise valued filtered signal into the maxitive based imprecise valued filtered signal (see Figure 7) has been proved in [START_REF] Destercke | Using cloudy kernels for imprecise linear filtering[END_REF]. Similarly, we proved in this paper that the f-box filtered signal would be included in the two others.

The inclusion of the Dirac measure in the maxitive kernel gives very large upper and lower filtered bounds, that encompass the whole signal (i.e., the signal is always in the interval provided by the maxitive kernel). Compared to maxitive kernels, cloudy kernels are more appropriate to represent the available knowledge on the kernel shape and bandwidth. However, as it can be remarked in Figures 7 and9, the bounds of the cloudy based imprecise filtered signal are very chaotic and are not similar to the general shape of the precise filtered signals (in cyan on the figures). This provides results that may be difficult to interpret.

On the other hand, the f-box filtered signal is smoother and well follows the general trend of precise signals filtered with different summative kernels. It is also a bit more precise than the cloudy filtered signal. Hence, the use of f-box kernels appears as a promising solution in this kind of situation.

Final remarks and perspectives

In this paper, we have introduced f-boxes as convenient models to achieve filtering when the bandwidth of the desired filter is ill-known. F-boxes allow us to model in a single representation a set of (here symmetric and unimodal) kernels whose bandwidth is lower-and upperbounded. Thanks to their particular properties (i.e., they induce ∞-monotone lower probabilities that possess particular additive properties), we have developed an efficient algorithm, based on the Choquet integral, that computes the lower and upper envelope of sets of filtered signals.

We have also shown that when it comes to model sets of symmetric and unimodal summative kernes, f-boxes display particularly good properties, of theoretical as well as of practical nature. They thus constitute a good approximation and interesting tool to achieve imprecise filtering.

Also, compared to other kernels derived from imprecise probabilistic models, such as maxitive or cloudy kernels, f-boxes are well adapted to the case of causal filtering, mainly because the p-box representation use bounding probabilities over sets [-∞, x], rather than over sets centred around a modal value. In future works, we intend to explore the potential of f-boxes to model sets of causal filters as well.

We would also like to remark that this model directly extends to the case of positive filters with a fixed gain higher or lower than one. Indeed, since lower and upper expectations are such that E(a • f ) = a • E( f ) and E(a • f ) = a • E( f ), it is sufficient to consider the kernel such that a • κ, where κ is an unknown summative kernel and to do process exactly as before.

  with the inclusion being usually strict. Proof. (Sketch) In the case of symmetric unimodal kernels around 0, imprecise kernels induced by maxitive kernel π [∆] and cloudy kernel [π [∆] , η [∆] ] correspond to sets of summative kernels that satisfy lower and upper probabilistic contraints induced by π [∆] and [π [∆] , η [∆] ]

Figure 3 :Theorem 3 .

 33 Figure 3: Lower probability on event [-x, x]

Figure 4 : 1 :

 41 Figure 4: Discretisation of a f-filter with 8 sampled values

1 ( x ∆ 2

 12 -1) , but that he could only "guess" that the appropriate bandwidth ∆ would lie somewhere in the interval [∆] = [∆ inf , ∆ sup ] = [0.045, 0.050]. The two extreme kernels

Figure 5 :Figure 6 :

 56 Figure 5: Original signal (black) superimposed on the noisy signal (cyan).

Figure 7 :

 7 Figure 7: Ten summative-based filtered signals (cyan) superimposed with the maxitive-based filtered signal (dotted blue and red), the cloud-based filtered signal (plain blue and red) and the original signal (black).

Figures 7 ,

 7 8 and 9 display a zoomed window of the resulting filtered output of the different filter. Each Figure presents two imprecise valued filtered signals (blue upper and red lower), the original signal (in black) and the output of 50 summative (conventional) based precise filtered signals involving a kernel of the form κ ∆ with ∆ ∈ [0.045, 0.050]. As expected by the theory (i.e., imprecise kernels all include the set κ [∆] ), all the 50 precise filtered signals belong to the enveloppe of each imprecise valued filtered signals. Moreover, since the filter has been correctly designed, ∀x ∈ R there is always a value ∆ ∈ [0.045, 0.050] such that E x κ ∆ (s) = f (x), i.e., the exact original signal is retrieved for some value of ∆. Thus, the real signal is always included in all the impre-

Figure 8 :

 8 Figure 8: Ten summative-based filtered signals (cyan) superimposed with the maxitive-based filtered signal (dotted blue and red), the cloud-based filtered signal (plain blue and red) and the original signal (black).

Figure 9 :

 9 Figure 9: Ten summative-based filtered signals (cyan) superimposed with the maxitive-based filtered signal (dotted blue and red), the cloud-based filtered signal (plain blue and red) and the original signal (black).