Jacques Touboul
email: jack_touboul@hotmail.com

PROJECTION PURSUIT THROUGH RELATIVE ENTROPY MINIMISATION

Keywords: projection pursuit, minimum relative entropy, maximum relative entropy, elliptic distribution, rate of convergence. MSC: 62H40 62F03 62F10 62F30

Consider a dened density on a set of very large dimension. It is quite dicult to nd an estimate of this density from a data set. However, it is possible through a projection pursuit methodology to solve this problem. In his seminal article, Huber (see "Projection pursuit", Annals of Statistics, 1985) demonstrates the interest of his method in a very simple given case. He considers the factorization of density through a Gaussian component and some residual density. Huber's work is based on maximizing relative entropy. Our proposal leads to a new algorithm. Furthermore, we consider the case when the density to be factorized is estimated from an i.i.d. sample. In this case, we will propose a test for the factorization of the estimated density.

step of Huber's algorithm amounts to dening a vector a 1 and a density f (1) by

a 1 = arg inf a∈R d * K(f g a f a , g) and f (1) = f g a 1 f a 1 , (1)
where R d * is the set of non null vectors of R d , where f a (resp. g a ) stands for the density of a X (resp. a Y ) when f (resp. g) is the densiy of X (resp. Y ). More exactly, this results from the maximisation of a → K(f a , g a ) since K(f, g) = K(f a , g a ) + K(f ga f a , g) and it is assumed that K(f, g) is nite. In a second step, Huber replaces f with f (1) and goes through the rst step again. By iterating this process, Huber thus obtains a sequence (a 1 , a 2 , ...) of vectors of R d * and a sequence of densities f (i) .

Remark 1. Huber stops his algorithm when the relative entropy equals zero or when his algorithm reaches the d th iteration, he then obtains an approximation of f from g : When there exists an integer j such that K(f (j) , g) = 0 with j ≤ d, he obtains f (j) = g, i.e.

f = gΠ j i=1 f (i-1) a i ga i since by induction f (j) = f Π j i=1 g a i f (i-1) a i
. Similarly, when, for all j, Huber gets K(f (j) , g) > 0 with j ≤ d, he assumes g = f (d) in order to derive f = gΠ d i=1 f

(i-1) a i g a i
. He can also stop his algorithm when the relative entropy equals zero without the condition j ≤ d is met. Therefore, since by induction we have f

(j) = f Π j i=1 g a i f (i-1) a i with f (0) = f , we obtain g = f Π j i=1 g a i f (i-1) a i
. Consequently, we derive a representation of f as f = gΠ j i=1 f

(i-1) a i g a i .
Finally, he obtains K(f (0) , g) ≥ K(f (1) , g) ≥ ..... ≥ 0 with f (0) = f . At present, let us illustrate this through the following example:

Example 1 (From Huber's article). Let (X 1 , ..., X d ) be a random vector with density function f .

We want to approximate f with a product of Gaussian densities. Let g be a Gaussian density on R d with same mean and variance as f . In the sequel, for all a ∈ R d * , we will call f a the density of a X and f 1 , f 2 ,...,f d the marginals of f . Thus, let us consider the two following hypotheses:

• Let us assume that the X 1 , ..., X d are independent, i.e. f (x) = Π d j=1 f j (x j ), and

• K(f 1 , g 1 ) ≥ K(f 2 , g 2 ) ≥ ... ≥ K(f d , g d ) ≥ 0.
We assume the relative entropy between the marginals of identical ranking are well-ordered. Consequently, g d and f d are the closest densities and g 1 and f 1 the least similar densities. Thus, on page 458 of [START_REF] Peter | Projection pursuit[END_REF], Huber rst shows that, for all b in R d * , K(f b , g b ) ≤ K(f 1 , g 1 ). He sets f (1) (x) = f (x) g 1 (x 1 ) f 1 (x 1 ) , he therefore gets a 1 = (1, 0, ..., 0), f (1) (x) = Π d j=1 f

(1) j (x j ) -where f

(1) 1 = g 1 and f

(1) j = f j for j = 2, ..., d -and K(f, g) = K(f 1 , g 1 ) + K(f (1) , g) through relationship (1). In the second step, Huber shows that, for all b in R d * , K([f (1) ] b , g b ) ≤ K(f 2 , g 2 ). Hence, he gets a 2 = (0, 1, 0, 0, .., 0), f (2) (x) = Π d j=1 f

(2) j (x j ) -where f

(2)

1 = g 1 , f
(2) 2 = g 2 and f

(2) j = f j for j = 3, ..., das well as K(f (1) , g) = K(f

(1)
2 , g 2 ) + K(f (2) , g), i.e. K(f, g) = K(f 1 , g 1 ) + K(f 2 , g 2 ) + K(f (2) , g), since f

(1) 2 = f 2 . And so on. Huber shows that, for all b in R d * , K([f (d-1) ] b , g b ) ≤ K(f d , g d ). Therefore, he obtains : • a d = (0, 0, 0, 0, .., 1),

• f (d) (x) = Π d j=1 f (d) j (x j )
, where f = g j for j = 1, ..., d -1, i.e. f (d) = g, and • K(f (d-1) , g) = K(f (d-1) d , g d ) + K(f (d) , g) = K(f d , g d ) since f (d) = g implies K(f (d) , g) = 0 and since f (d-1) d = f d , i.e. K(f, g) = K(f 1 , g 1 ) + K(f 2 , g 2 ) + ... + K(f d , g d ).

We therefore obtain f = gΠ d j=1 f j g j , which is the representation of f we have been looking for.

1.2 Huber's synthetic approach Keeping the notations of the above section, we start with performing the K(f, g) = 0 test; should this test turn out to be positive, then f = g and the algorithm stops, otherwise, the rst step of his algorithm would consist in dening a vector a 1 and a density g (1) by

a 1 = arg inf a∈R d * K(f, g f a g a
) and g (1) = g f a 1 g a 1 .

(2)

More exactly, this optimisation results from the maximisation of a → K(f a , g a ) since K(f, g) = K(f a , g a ) + K(f, g fa g a ) and it is assumed that K(f, g) is nite. In a second step, Huber replaces g with g (1) and goes through the rst step again. By iterating this process, Huber thus obtains a sequence (a 1 , a 2 , ...) of vectors of R d * and a sequence of densities g (i) .

Remark 2. First, in a similar manner to the analytic approach, this methodology enables us to approximate and even to represent f from g:

To obtain an approximation of f , Huber either stops his algorithm when the relative entropy equals zero, i.e. K(f, g (j) ) = 0 implies g (j) = f with j ≤ d, or when his algorithm reaches the d th iteration, i.e. he approximates f with g (d) .

To obtain a representation of f , Huber stops his algorithm when the relative entropy equals zero, since K(f, g (j) ) = 0 implies g (j) = f . Therefore, since by induction we have g (j) = gΠ j i=1 fa i g (i-1) a i with g (0) = g, we then obtain f = gΠ j i=1 f a i

g (i-1) a i
.

Second, he gets K(f, g (0) ) ≥ K(f, g (1) ) ≥ ..... ≥ 0 with g (0) = g.

Finally, in [START_REF] Zhu | On the forward and backward algorithms of projection pursuit[END_REF], Mu Zhu shows that, even if the above two algorithms are exclusively based on the maximisation of the relative entropy K(f a , g a ), beyond d iterations, the data processing of these methodologies evidences signicant dierences, i.e. that past d iterations, the two methodologies are no longer equivalent. We will therefore only consider Huber's synthetic approach since g is known and since we want to nd a representation of f .

Proposal

We start with performing the K(f, g) = 0 test; should this test turn out to be positive, then f = g and the algorithm stops, otherwise, the rst step of our algorithm would consist in dening a vector a 1 and a density g (1) by

a 1 = arg inf a∈R d * K(g f a g a , f
) and g (1) = g f a 1 g a 1 .

(3)

In the second step, we will replace g with g (1) , and we will repeat the rst step. And so on, by iterating this process, we will end up obtaining a sequence (a 1 , a 2 , ...) of vectors in R d * and a sequence of densities g (i) . We will prove that a 1 simultaneously optimises (1), (2) and (3). We will also prove that the underlying structures of f evidenced through this method are identical to the ones obtained through Huber's methods.

Remark 3. As in Huber's algorithms, we perform an approximation and a representation of f :

To obtain an approximation of f , we stop the algorithm when the relative entropy equals zero, i.e. we approximate f with g (j) , or when the algorithm reaches the d th iteration, i.e. we approximate f with g (d) .

To obtain a representation of f , we stop the algorithm when the relative entropy equals zero. Therefore, since by induction we have g

(j) = gΠ j i=1 f a i g (i-1) a i with g (0) = g, we then obtain f = gΠ j i=1 f a i g (i-1) a i
.

Finally, we have K(g (0) , f ) ≥ K(g (1) , f ) ≥ ..... ≥ 0 with g (0) = g.

Let us study the following example:

Example 2. Let f be a density dened on R 3 by f (x 1 , x 2 , x 3 ) = n(x 1 , x 2 )h(x 3 ), with n being a bi-dimensional Gaussian density, and h being a non Gaussian density. Let us also consider g, a Gaussian density with same mean and variance as f . The function a → K(g f a g a , f ) reaches zero for e 3 = (0, 0, 1) . Indeed, since g(

x 1 , x 2 /x 3 ) = n(x 1 , x 2 ), we have K(g f 3 g 3 , f ) = K(n.f 3 , f ) = K(f, f ) = 0 as f 3 = h. We therefore obtain that g(x 1 , x 2 /x 3 ) = f (x 1 , x 2 /x 3 ).
To recapitulate our method, if K(g, f ) = 0, we derive f from the relationship f = g; should a sequence (a i ) i=1,...j , j < d, of vectors in R d * dening g (j) and such that K(g (j) , f ) = 0 exist, then f (./a i x, 1 ≤ i ≤ j) = g(./a i x, 1 ≤ i ≤ j), i.e. f coincides with g on the complement of the vector subspace generated by the family {a i } i=1,...,j -see also section 2.1.2 for more detailed explanations.

In this paper, after having claried the choice of g, we will consider the statistical solution to the representation problem, assuming that f is unknown and X 1 , X 2 ,... X m are i.i.d. with density f . We will provide asymptotic results pertaining to the family of optimizing vectors a k,m -that we will dene more precisely below -as m goes to innity. Our results also prove that the empirical representation scheme converges towards the theoretical one. As an application, section 3.4 permits a new test of t pertaining to the copula of an unknown density f and section 3.5 gives us an estimate of a density deconvoluted with a Gaussian component. Finally, we will compare Huber's optimisation methods with ours and we will present simulations.

The algorithm

The model

As explained by Friedman [START_REF] Friedman | Projection pursuit density estimation[END_REF], Diaconis [START_REF] Diaconis | Asymptotics of graphical projection pursuit[END_REF] and Hwang [START_REF] Hwang | Nonparametric multivariate density estimation: a comparative study[END_REF], the choice of g depends on the family of distribution one wants to nd in f . Until now, the choice has been only to use the class of Gaussian distributions. This can be extended to the class of elliptic distributions.

Elliptic laws

The interest of this class that it is such that conditional densities with elliptic distributions are also elliptic -see [START_REF] Cambanis | On the theory of elliptically contoured distributions[END_REF], [START_REF] Landsman | Tail conditional expectations for elliptical distributions[END_REF]. This very property allows us to use this class in our algorithm -and in Huber's algorithms.

Denition 1. X is said to abide by a multivariate elliptic distribution -noted

X ∼ E d (µ, Σ, ξ d ) - if X presents the following density, for any x in R d : f X (x) = c d |Σ| 1/2 ξ d 1 2 (x -µ) Σ -1 (x -µ)
• with Σ, being a d × d positive-denite matrix and with µ, being an d-column vector, • with ξ d , being referred as the "density generator",

• with c d , being a normalisation constant, such that c d = Γ(d/2) (2π) d/2 ∞ 0 x d/2-1 ξ d (x)dx -1 , with ∞ 0 x d/2-1 ξ d (x)dx < ∞.
Property 1. 1/ For any X ∼ E d (µ, Σ, ξ d ), for any A, being a m × d matrix with rank m ≤ d, and for any b, being an m-dimensional vector, we have AX + b ∼ E m (Aµ + b, AΣA , ξ m ). Therefore, any marginal density of multivarite elliptic distribution is elliptic, i.e. [START_REF] Cambanis | On the theory of elliptically contoured distributions[END_REF] states that conditional densities with elliptic distributions are also elliptic.

X = (X 1 , X 2 , ..., X d ) ∼ E d (µ, Σ, ξ d ) ⇒ X i ∼ E 1 (µ i , σ 2 i , ξ 1 ), f X i (x) = c 1 σ i ξ 1 1 2 ( x-µ i σ ) 2 , 1 ≤ i ≤ d. 2/ Corollary 5 of
Indeed, if X = (X 1 , X 2 ) ∼ E d (µ, Σ, ξ d ), with X 1 (resp. X 2 ) being a size d 1 < d (resp. d 2 < d), then X 1 /(X 2 = a) ∼ E d 1 (µ , Σ , ξ d 1 ) with µ = µ 1 + Σ 12 Σ -1 22 (a -µ 2 ) and Σ = Σ 11 -Σ 12 Σ -1 22 Σ 21 , with µ = (µ 1 , µ 2 ) and Σ = (Σ ij ) 1≤i,j≤2 .
Remark 4. In [START_REF] Landsman | Tail conditional expectations for elliptical distributions[END_REF], the authors show that the multivariate Gaussian distribution derives from ξ d (x) = e -x . They also show that if X = (X 1 , ..., X d ) has an elliptic density such that its marginals verify E(X i ) < ∞ and E(X 2 i ) < ∞ for 1 ≤ i ≤ d, then µ is the mean of X and Σ is the covariance matrix of X. Consequently, from now on, we will assume that we are in this case.

Denition 2. Let t be an elliptic density on R k and let q be an elliptic density on R k . The elliptic densities t and q are said to belong to the same family -or class -of elliptic densities, if their generating densities are ξ k and ξ k respectively, which belong to a common given family of densities.

Example 3. Consider two Gaussian densities N (0, 1) and N ((0, 0), Id 2 ). They are said to belong to the same elliptic families as they both present x → e -x as generating density.

Choice of g

Let us begin with studying the following case: Let f be a density on R d . Let us assume there exists d not null independent vectors a j , with

1 ≤ j ≤ d, of R d , such that f (x) = n(a j+1 x, ..., a d x)h(a 1 x, ..., a j x), (4)
with j < d, with n being an elliptic density on R d-j-1 and with h being a density on R j , which does not belong to the same family as n. Let X = (X 1 , ..., X d ) be a vector presenting f as density. Dene g as an Elliptic distribution with same mean and variance as f . For simplicity, let us assume that the family {a j } 1≤j≤d is the canonical basis of R d :

The very denition of f implies that (X j+1 , ..., X d ) is independent from (X 1 , ..., X j ). Hence, property 1 enables us to derive that the density of (X j+1 , ..., X d ) given (X 1 , ..., X j ) is n.

Let us assume that K(g (j) , f ) = 0, for some j ≤ d. We then get

f (x) f a 1 f a 2 ...f a j = g(x) g (1-1) a 1 g (2-1) a 2 ...g (j-1) a j
, since, by induction, we have g (j) (x) = g(x)

fa 1 g (1-1) a 1 fa 2 g (2-1) a 2 ... fa j g (j-1) a j .
Consequently, the fact that conditional densities with elliptic distributions are also elliptic as well as the above relationship enable us to state that n(a j+1 x, ., a

d x) = f (./a i x, 1 ≤ i ≤ j) = g(./a i x, 1 ≤ i ≤ j).
In other words, f coincides with g on the complement of the vector subspace generated by the family {a i } i=1,...,j .

At present, if the family {a j } 1≤j≤d is no longer the canonical basis of R d , then this family is again a basis of R d . Hence, lemma 7 -page 23 -implies that g(./a 1 x, ..., a j x) = n(a j+1 x, ..., a d x) = f (./a 1 x, ..., a j x),

(5) which is equivalent to having K(g (j) , f ) = 0 -since by induction g (j) = g

f a 1 g (1-1) a 1 f a 2 g (2-1) a 2 ... f a j g (j-1) a j
.

The end of our algorithm implies that f coincides with g on the complement of the vector subspace generated by the family {a i } i=1,...,j . Therefore, the nullity of the relative entropy provides us with information on the density structure. In summary, the following proposition claries the choice of g which depends on the family of distribution one wants to nd in f :

Proposition 1. With the above notations, K(g (j) , f ) = 0 is equivalent to g(./a 1 x, ..., a j x) = f (./a 1 x, ..., a j x)

More generally, the above proposition leads us to dening the co-support of f as the vector space generated from vectors a 1 , ..., a j . Denition 3. Let f be a density on R d . We dene the co-vectors of f as the sequence of vectors a 1 , ..., a j which solves the problem K(g (j) , f ) = 0 where g is an Elliptic distribution with same mean and variance as f . We dene the co-support of f as the vector space generated from vectors a 1 , ..., a j .

Stochastic outline of the algorithm

Let X 1 , X 2 ,..,X m (resp. Y 1 , Y 2 ,..,Y m ) be a sequence of m independent random vectors with same density f (resp. g). As customary in nonparametric Kullback-Lieber optimizations, all estimates of f and f a (resp. g and g a ) are being performed using a subsample X 1 , X 2 ,..,X n (resp. Y 1 , Y 2 ,..,Y n ) extracted from X 1 , X 2 ,..,X m (resp. Y 1 , Y 2 ,..,Y m ) such that the estimates are bounded below by some positive deterministic sequence θ m -see Annex B. Let P n be the empirical measure of the subsample X 1 , X 2 ,..,X n . Let f n (resp. g n , f a,n , g a,n for any a in R d * ) be the kernel estimate of f (resp. g, f a , g a ) which is built from X 1 , X 2 ,..,X n (resp. Y 1 , Y 2 ,..,Y n ). As dened in section 1.3, we introduce the following sequences (a k ) k≥1 and (g (k) ) k≥1 :

• a k is a non null vector of R d such that a k = arg min a∈R d * , K(g (k-1) fa g (k-1) a , f ), • g (k) is the density such that g (k) = g (k-1) f a k g (k-1) a k with g (0) = g.
The stochastic setting up of the algorithm uses f n and g n instead of f and g. Thus, at the rst step, we build the vector ǎ1 which minimizes the relative entropy between f n and g n f a,n ga,n and which estimates a 1 : Proposition 11 page 19 and lemma 11 page 24 enables us to minimize the relative entropy between f n and g n f a,n ga,n . Dening ǎ1 as the argument of this minimization, proposition 4 page 8 shows us that this vector tends to a 1 in n. Finally, we dene the density ǧ(1) as ǧ(1) = g n f ǎ1 ,n g ǎ1 ,n which estimates g (1) through theorem 1. At the second step, we build the vector ǎ2 which minimizes the relative entropy between f n and ǧ(1) . Dening ǎ2 as the argument of this minimization, proposition 4 page 8 shows us that this vector tends to a 2 as n tends to innity. Finally, we dene the density ǧ(2) as ǧ(2) = ǧ(1) f ǎ2 ,n ǧ(1) ǎ2 ,n which estimates g (2) through theorem 1.

And so on, we will end up obtaining a sequence (ǎ 1 , ǎ2 , ...) of vectors in R d * estimating the co-vectors of f and a sequence of densities (ǧ (k) ) k such that ǧ(k) estimates g (k) through theorem 1.

Results

Convergence results

Hypotheses on f

In this paragraph, we dene the set of hypotheses on f which we can possibly use in our work. Discussion of several on these hypotheses can be found in Annex D. In this section, to be more legible we replace g with g (k-1) . Let

Θ = R d * , M (b, a, x) = ln( g(x) f (x) f b (b x) g b (b x) )g(x) f a (a x) ga(a x) dx -( g(x) f (x) f b (b x) g b (b x) -1), P n M (b, a) = M (b, a, x)dP n , PM (b, a) = M (b, a, x)f (x)dx,
where P being the probability measure of f . Similarly as in chapter V of [VDW], we dene : (H 1) : For all ε > 0, there is η > 0, such that for all c ∈ Θ verifying c -a k ≥ ε, we have PM (c, a) < PM (a k , a) -η, with a ∈ Θ. (H 2) : There is a neighborhood of a k , V , and a positive function H, such that, for all c ∈ V we have |M (c, a k , x)| ≤ H(x) (P -a.s.) with PH < ∞, (H 3) : There is a neighborhood V of a k , such that for all ε, there is a η such that for all c ∈ V and a ∈ Θ, verifying a -a k ≥ ε, we have PM (c, a k ) < PM (c, a) -η.

Putting I a k = ∂ 2 ∂a 2 K(g fa k g a k , f ), and x → ρ(b, a, x) = ln( g(x)f b (b x) f (x)g b (b x) ) g(x)f a (a x)
ga(a x) , we consider now three new hypotheses: (H 4) : There is a neighborhood V k of (a k , a k ) such that, for all (b, a) of V k , the gradient ∇( g(x)f a (a x) g a (a x) ) and the Hessian H( g(x)f a (a x) g a (a x) ) exist (λ_a.s.), and the rst order partial derivative g(x)f a (a x) g a (a x) and the rst and second order derivative of (b, a) → ρ(b, a, x) are dominated (λ_a.s.) by integrable functions. (H 5) : The function (b, a) → M (b, a, x) is C 3 in a neighborhood V k of (a k , a k ) for all x; and all the partial derivatives of order 3 of (b, a) → M (b, a, x) are dominated in V k by a P_integrable function H(x). (H 6) : P ∂ ∂b M (a k , a k ) 2 and P ∂ ∂a M (a k , a k ) 2 are nite and the expressions P ∂ 2 ∂b i ∂b j M (a k , a k ) and I a k exist and are invertible. Finally, we dene (H 7) : There exists k such that PM (a k , a k ) = 0. (H 8) : (V ar P (M (a k , a k ))) 1/2 exists and is invertible. (H 0): f and g are assumed to be positive and bounded.

Estimation of the rst co-vector of f

Let R be the class of all positive functions r dened on R and such that g(x)r(a x) is a density on R d for all a belonging to R d * . The following proposition shows that there exists a vector a such that f a g a minimizes K(gr, f ) in r:

Proposition 2. There exists a vector a belonging to R d * such that

arg min r∈R K(gr, f ) = f a g a and r(a x) = f a (a x) g a (a x) .
In the following of sections 3.1 and 3.2, we will use the kernel estimate of f (resp. g, f a and g a for any a in R d * ) instead of f (resp. g, f a and g a for any a in R d * ). We will keep the notation "f " (resp. "g", "f a " and "g a ") to designate this estimate. Following [BROKEZ], let us introduce the estimate of K(g f a g a , f ), through

Ǩ(g f a g a , f ) = M (a, a, x)dP n (x) Proposition 3. Let ǎ := arg inf a∈R d * Ǩ(g f a g a , f ).
Then, ǎ is a strongly convergent estimate of a as dened in proposition 2.

Let us also introduce the following sequences (ǎ k ) k≥1 and (ǧ (k) ) k≥1 -for any given n:

• ǎk is an estimate of a k as dened in proposition 3 with ǧ(k-1) instead of g -see section 2.2 -,

• ǧ(k) is such that ǧ(0) = g, ǧ(k) (x) = ǧ(k-1) (x) f ǎk (ǎ k x) [ǧ (k-1) ] ǎk (ǎ k x) , i.e. ǧ(k) (x) = g(x)Π k j=1 f ǎj (ǎ j x)
[ǧ (j-1) ] ǎj (ǎ j x) . We also note that ǧ(k) is a density.

Convergence study at the k th step of the algorithm:

In this paragraph, we will show that the sequence (ǎ k ) n converges towards a k and that the sequence (ǧ (k) ) n converges towards g (k) . Let čn (a) = arg sup c∈Θ P n M (c, a), with a ∈ Θ, and γn = arg inf a∈Θ sup c∈Θ P n M (c, a). We state Proposition 4. It holds 1/ sup a∈Θ čn (a) -a k tends to 0 a.s. 2/ γn tends to a k a.s. . Finally, the following theorem shows that ǧ(k) converges almost everywhere towards g (k) : Theorem 1. It holds

ǧ(k) → n g (k) a.s.

Asymptotic Inference at the k th step of the algorithm

The following theorem shows that ǧ(k) converges towards g (k) at the rate O P (m -1 d=1 -4 4+d 1 d>1 ) in three dierents cases, namely for any given x, with the L 1 distance and with the relative entropy: Theorem 2. It holds

|ǧ (k) (x) -g (k) (x)| = O P (m -1 d=1 -4 4+d 1 d>1 ), |ǧ (k) n (x) -g (k) (x)|dx = O P (m -1 d=1 -4 4+d 1 d>1 ), |K(ǧ (k) n , f ) -K(g (k) , f )| = O P (m -1 d=1 -4 4+d 1 d>1 ).
Then, the following theorem shows that the laws of our estimators of a k , namely čn (a k ) and γn , converge towards a linear combination of Gaussian variables.

Theorem 3. It holds

√ nA.(č n (a k ) -a k ) Law → B.N d (0, P ∂ ∂b M (a k , a k ) 2 ) + C.N d (0, P ∂ ∂a M (a k , a k ) 2 ) and √ nA.(γ n -a k ) Law → C.N d (0, P ∂ ∂b M (a k , a k ) 2 ) + C.N d (0, P ∂ ∂a M (a k , a k ) 2 ) where A = P ∂ 2 ∂b∂b M (a k , a k )(P ∂ 2 ∂a∂a M (a k , a k ) + P ∂ 2 ∂a∂b M (a k , a k )), C = P ∂ 2 ∂b∂b M (a k , a k ) and B = P ∂ 2 ∂b∂b M (a k , a k ) + P ∂ 2 ∂a∂a M (a k , a k ) + P ∂ 2 ∂a∂b M (a k , a k ).

A stopping rule for the procedure

In this paragraph, we will call

ǧ(k) n (resp. ǧ(k) a,n ) the kernel estimator of ǧ(k) (resp. ǧ(k) a ). We will rst show that g (k)
n converges towards f in k and n. Then, we will provide a stopping rule for this identication procedure.

Estimation of f

Through remark 6 page 13 and as explained in section 14 of [START_REF] Peter | Projection pursuit[END_REF], the following lemma shows that K(g

(k-1) n f a k ,n g (k-1) a k ,n
, f a k ,n ) converges almost everywhere towards zero as k and then as n go to innity:

Lemma 1. We have

lim n lim k K(ǧ (k) n fa k ,n [ǧ (k) ] a k ,n , f n ) = 0 a.s.
Consequently, the following proposition provides us with an estimate of f :

Theorem 4. We have lim n lim k ǧ(k) n = f a.s.

Testing of the criteria

In this paragraph, through a test of the criteria, namely a → K(ǧ

(k) n fa,n [ǧ (k) ] a,n , f n ),
we will build a a stopping rule for this identication procedure. First, the next theorem enables us to derive the law of the criteria: Theorem 5. For a xed k, we have

√ n(V ar P (M (č n (γ n ), γn ))) -1/2 (P n M (č n (γ n ), γn )-P n M (a k , a k )) Law → N (0, I), as n goes to innity,
where k represents the k th step of the algorithm, where I is the identity matrix in R d .

Note that k is xed in theorem 5 since γn = arg inf a∈Θ sup c∈Θ P n M (c, a) where M is a known function of k -see section 3.1.1. Thus, in the case where K(g

(k-1) f a k g (k-1) a k , f ) = 0, we obtain Corollary 1. We have √ n(V ar P (M (č n (γ n ), γn ))) -1/2 (P n M (č n (γ n ), γn )) Law → N (0, I).
Hence, we propose the test of the null hypothesis

(H 0 ) : K(g (k-1) f a k g (k-1) a k , f ) = 0 versus the alternative (H 1 ) : K(g (k-1) f a k g (k-1) a k , f ) = 0.
Based on this result, we stop the algorithm, then, dening a k as the last vector generated, we derive from corollary 1 a α-level condence ellipsoid around a k , namely

E k = {b ∈ R d ; √ n(V ar P (M (b, b))) -1/2 P n M (b, b) ≤ q N (0,1) α }
where q

N (0,1) α
is the quantile of a α-level reduced centered normal distribution and where P n is the empirical measure araising from a realization of the sequences (X 1 , . . . , X n ) and (Y 1 , . . . , Y n ). Consequently, the following corollary provides us with a condence region for the above test:

Corollary 2. E k is a condence region for the test of the null hypothesis (H 0 ) versus (H 1 ).

Goodness-of-t test for copulas

Let us begin with studying the following case: Let f be a density dened on R 2 and let g be an Elliptic distribution with same mean and variance as f . Assuming rst that the algorithm leads us to having K(g (2) , f ) = 0 where family (a i ) is the canonical basis of R 2 . Hence, we have g

(2) (x) = g(x) f 1 g 1 f 2 g (1) 2 = g(x) f 1 g 1 f 2 g 2 -through lemma 14 page 25 -and g (2) = f . Therefore, f = g(x) f 1 g 1 f 2 g 2 , i.e. f f 1 f 2 = g g 1 g 2 , and then ∂ 2 ∂x∂y C f = ∂ 2 ∂x∂y C g where C f (resp. C g ) is the copula of f (resp. g).
At present, let f be a density on R d and let g be the density dened in section 2.1.2. Let us assume that the algorithm implies that K(g (d) , f ) = 0. Hence, we have, for any

x ∈ R d , g(x)Π d k=1 f a k (a k x) [g (k-1) ] a k (a k x) = f (x), i.e. g(x) Π d k=1 g a k (a k x) = f (x) Π d k=1 f a k (a k x) -through lemma 14.
Moreover, the family (a i ) i=1...d is a basis of R d -see lemma 15 page 25. Hence, putting A = (a 1 , ..., a d ) and dening vector y (resp. density f , copula Cf of f , density g, copula Cg of g) as the expression of vector x (resp. density f , copula C f of f , density g, copula C g of g) in basis A, the above equality implies ∂ d ∂y 1 ...∂y d Cf = ∂ d ∂y 1 ...∂y d Cg . Finally, we perform a statistical test of the null hypothesis

(H 0 ) : ∂ d ∂y 1 ...∂y d Cf = ∂ d ∂y 1 ...∂y d Cg versus (H 1 ) : ∂ d ∂y 1 ...∂y d Cf = ∂ d ∂y 1 ...∂y d Cg .
Since under (H 0 ) we have K(g (d) , f ) = 0, then, as explained in section 3.3.2, corollary 2 provides us with a condence region for the test.

Theorem 6. Keeping the notations of corollary 2, we infer that E d is a condence region for the test of the null hypothesis (H 0 ) versus the alternative (H 1 ).

Rewriting of the convolution product

In the present paper, we rst elaborated an algorithm aiming at isolating several known structures from initial datas. Our objective was to verify if for a known density on R d , there exists a known density n on R d-j-1 such that, for d > 1

f (x) = n(a j+1 x, ..., a d x)h(a 1 x, ..., a j x), (6)
with j < d, with (a 1 , . . . , a d ) being a basis of R d and with h being a density on R j . Secondly, our next step has been to build an estimate (resp. a representation) of f without necessarily assuming that f meets relationship (6) -see theorem 4 (resp. proposition 13 page 21). Consequently, let us consider Z 1 and Z 2 , two random vectors with respective densities h 1 and h 2which is Elliptic -on R d . Let us consider a random vector X such that X = Z 1 + Z 2 and let f be its density. This density can then be written as :

f (x) = h 1 * h 2 (x) = R d h 1 (x)h 2 (t -x)dt.
Then, the following property enables us to represent f under the form of a product and without the integral sign Proposition 5. Let φ be a centered Elliptic density with σ 2 .I d , σ 2 > 0, as covariance matrix, such that it is a product density in all orthogonal coordinate systems and such that its characteristic

function s → Ψ( 1 2 |s| 2 σ 2 ) is integrable -see [LANDS03].
Let f be a density on R d which can be deconvoluted with φ, i.e.

f = f * φ = R d f (x)φ(t -x)dt,
where f is some density on R d . Let g (0) be the Elliptic density belonging to the same Elliptic family as f and having same mean and variance as f . Then, the sequence (g (k) ) k converges uniformly a.s. and in L 1 towards f in k, i.e.

lim k→∞ sup x∈R d |g (k) (x) -f (x)| = 0, and lim k→∞ R d |g (k) (x) -f (x)|dx = 0.
Finally, along with the notations of section 3.3 and of proposition 5, the following theorem enables us to estimate any convolution product of a multivariate Elliptic density φ with a continuous density f :

Theorem 7. It holds lim n lim k ǧ(k) n = f * φ a.s.

Comparison of all the optimisation methods

In this section, we will study Huber's algorithm in a similar manner to sections 2 and 3. We will then be able to compare our methodologies.

Remark 5 (Huber's algorithm). First, until now, the choice has only been to use the class of Gaussian distributions. Here and similarly to section 2.1, we extend this choice to the class of elliptic distributions. Moreover, using the subsample X 1 , X 2 ,..., X n -see Annex B -and using the protocol of section 2.2 with K(g a , f a ) -see section 4.2 -instead of K(g g a f a , f ), proposition 11 page 19, lemma 11 page 24 and remark 6 page 13 enable us to perform Huber's algorithm :

• we dene â1 and the density ĝ(1) such that â1 = arg max a∈R d * K(g a,n , f a,n ) and ĝ(1) = g n f â1 ,n g â1 ,n , • we dene â2 and the density ĝ(2) such that â2 = arg max a∈R d * K(ĝ

(1) a,n , f a,n ) and ĝ(2) = ĝ(1) f â2 ,n ĝ(1) â2 ,n
, and so on, we will end up obtaining a sequence (â 1 , â2 , ...) of vectors in R d * and a sequence of densities ĝ(k) .

Hypotheses on f

In this paragraph, we dene the set of hypotheses on f which we can possibly use in this work. First, denote g in lieu of g (k-1) . Let

Θ 1 a = {b ∈ Θ | ( g b (b x) f b (b x) -1)f a (a x) dx < ∞}, m(b, a, x) = ln( g b (b x) f b (b x) )g a (a x) dx -( g b (b x) f b (b x) -1), P a m(b, a) = m(b, a, x)f a (a x) dx and P n m(b, a) = m(b, a, x) f a (a x)
f (x) dP n , with P a being the probability measure with density f a . Similarly as V an der V aart, in chapter V of [VDW], let us dene : (H1) : For all ε > 0, there is η > 0 such that, for all b ∈ Θ 1 a verifying b -a k ≥ ε for all a ∈ Θ, we have P a m(b, a) < P a m(a k , a) -η, (H2) : There is a neighborhood of a k , V , and a positive function H, such that,for all b ∈ V , we have |m(b, a k , x)| ≤ H(x) (P a -a.s.) with P a H < ∞, (H3) : There is a neighborhood V of a k , such that for all ε, there is a η such that for all b ∈ V and a ∈ Θ, verifying a -a k ≥ ε, we have

P a k m(b, a k ) -η > P a m(b, a). Moreover, dening x → υ(b, a, x) = ln( g b (b x) f b (b x)
)g a (a x), putting: (H4) : There exists a neighborhood of (a k , a k ), that we will name V k , such that, for all (b, a) of V k , the gradient ∇( ga(a x) f a (a x) ) and the Hessian H( ga(a x) f a (a x) ) exist (λ -a.s.) and the rst order partial derivative g a (a x) fa(a x) and the rst and second order derivative of order 3 of (b, a) → υ(b, a, x) are dominated (λ_a.s.) by integrable functions.

(H5) : The function (b, a) → m(b, a) is C 3 in a neighborhood V k of (a k , a k )

The rst co-vector of f simultaneously optimizes four problems

Let us rst study Huber's analytic approach. Let R be the class of all positive functions r dened on R and such that f (x)r -1 (a x) is a density on R d for all a belonging to R d * . The following proposition shows that there exists a vector a such that fa g a minimizes K(f r -1 , g) in r:

Proposition 6 (Analytic Approach). There exists a vector a belonging to

R d * such that arg min r∈R K(f r -1 , g) = f a ga , and r(a x) = fa(a x) g a (a x) and K(f, g) = K(f a , g a ) + K(f ga f a , g).
Let us also study Huber's synthetic approach: Let R be the class of all positive functions r dened on R and such that g(x)r(a x) is a density on R d for all a belonging to R d * . The following proposition shows that there exists a vector a such that f a ga minimizes K(gr, f ) in r:

Proposition 7 (Synthetic Approach). There exists a vector a belonging to R d * such that arg min r∈R K(f, gr) = f a ga , and r(a x) = f a (a x) g a (a x)

and K(f, g) = K(f a , g a ) + K(f, g fa g a ).

In the meanwhile, the following proposition shows that there exists a vector a such that f a ga minimizes K(g, f r -1 ) in r.

Proposition 8. There exists a vector a belonging to R d * such that arg min r∈R K(g, f r -1 ) = fa g a , and r(a x) = f a (a x) ga(a x)

Moreover, we have K(g, f ) = K(g a , f a ) + K(g, f g a fa ).

Remark 6. First, through property 4 page 18, we get

K(f, g f a g a ) = K(g, f g a f a ) = K(f g a f a , g) and K(f a , g a ) = K(g a , f a )
. Thus, proposition 8 implies that nding the argument of the maximum K(g a , f a ) amounts to nding the argument of the maximum K(f a , g a ). Consequently, the criteria of Huber's methodologies is a → K(g a , f a ). Second, our criteria is a → K(g g a fa , f ) and property 4 implies K(g, f ga f a ) = K(g fa g a , f ). Consequently, since [BROKEZ] takes into account the very form of the criteria, we are then in a position to compare Huber's methodologies with ours.

To recapitulate, the choice of r = fa g a enables us to simultaneously solve the following four optimisation problems, for a ∈ R d * : First, nd a such that a = arginf a∈R d * K(f g a fa , g) -pertaining to the analytic approach -Second, nd a such that a = arginf a∈R d * K(f, g f a g a ) -pertaining to the synthetic approach -Third, nd a such that a = argsup a∈R d * K(g a , f a ) -to compare Huber's methods with ours -Fourth, nd a such that a = arginf a∈R d * K(g fa g a , f ) -pertaining to our method.

4.3 On the sequence of the transformed densities (g (j) )

As already explained in the introduction section, the Mu Zhu article leads us to only consider Huber's synthetic approach. Moreover, in this section, we will use the kernel estimate of f (resp. g, f a and g a for any a in R d * ) instead of f (resp. g, f a and g a for any a in R d * ). We will keep the notation "f " (resp. "g", "f a " and "g a ") to designate this estimate.

Estimation of the rst co-vector of f

Using the subsample X 1 , X 2 ,..,X n -see Annex B -and following [BROKEZ], let us introduce the estimate of K(g a , f a ), through K(g a , f a ) = m(a, a, x)( fa(a x) f (x) )dP n Proposition 9. Let â := arg sup a∈R d * K(g a , f a ). Then, â is a strongly convergent estimate of a as dened in proposition 8.

Finally, let us dene the following sequences (â k ) k≥1 and (ĝ (k) ) k≥1 -for any given n :

• âk is an estimate of a k as dened in proposition 9 with ĝ(k-1) instead of g -see remark 5 -,

• ĝ(k) is such that ĝ(0) = g and ĝ(k) (x) = ĝ(k-1) (x) f âk (â k x) [ĝ (k-1) ] âk (â k x) , i.e. ĝ(k) (x) = g(x)Π k j=1 f âj (â j x)
[ĝ (j-1) ] âj (â j x) . We also note that ĝ(k) is a density. Finally, the following theorem shows that ĝ(k) converges almost everywhere towards g (k) :

Theorem 8. For any given k, it holds ĝ(k) → n g (k) a.s.

Asymptotic Inference at the k th step of the algorithm

The following theorem shows that ĝ(k) converges towards g (k) at the rate O P (m -1 d=1 -4 4+d 1 d>1 ) in three dierents cases, namely for any given x, with the L 1 distance and with the relative entropy: Theorem 9. It holds |ĝ (k)

(x)-g (k) (x)| = O P (m -1 d=1 -4 4+d 1 d>1 ), |ĝ (k) (x)-g (k) (x)|dx = O P (m -1 d=1 -4 4+d 1 d>1 ), and |K(f, ĝ(k) ) -K(f, g (k) )| = O P (m -1 d=1 -4 4+d 1 d>1 ).
The following theorem shows that the laws of Huber's estimators of a k , namely bn (a k ) and βn , converge towards a linear combination of Gaussian variables.

Theorem 10. It holds

√ nD.( bn (a k ) -a k ) Law → E.N d (0, P ∂ ∂b m(a k , a k ) 2 ) + F.N d (0, P ∂ ∂a m(a k , a k ) 2 ) and √ nD.( βn -a k ) Law → G.N d (0, P ∂ ∂a m(a k , a k ) 2 ) + F.N d (0, P ∂ ∂b m(a k , a k ) 2 ) where E = P ∂ 2 ∂a 2 m(a k , a k ), F = P ∂ 2 ∂a∂b m(a k , a k ), G = P ∂ 2 ∂b 2 m(a k , a k ) and D = (P ∂ 2 ∂b 2 m(a k , a k )P ∂ 2 ∂a 2 m(a k , a k ) -P ∂ 2 ∂a∂b m(a k , a k )P ∂ 2 ∂b∂a m(a k , a k )) > 0.
4.4 A stopping rule for the procedure

We will rst give an estimation of f . We will then provide a stopping rule for this identication procedure. We note that, in this section, we will call f n (respectively

ĝ(k) n , ∀a ∈ Θ ĝ(k) a,n , f a,n ) the kernel estimator of density f (respectively ĝ(k) , ∀a ∈ Θ ĝ(k) a , f a ).
Remark 7. In the case where f is known, then, as explained in section 14 of [START_REF] Peter | Projection pursuit[END_REF], the sequence (K(g

(k-1) a k
, f a k )) k≥1 converges towards zero. Many authors have studied this hypothesis and its consequences. For example, Huber deducts that, if f can be deconvoluted with a Gaussian component, (K(g

(k-1) a k
, f a k )) k≥1 converges toward 0. He then shows that g (i) uniformly converges in L 1 towards f -see propositions 14.2 and 14.3 page 461 of his article. Similarly, Friedman in [START_REF] Friedman | Projection pursuit density estimation[END_REF], page 19, and Hwang, in [START_REF] Hwang | Nonparametric multivariate density estimation: a comparative study[END_REF] pages 16 and 17, develop other typical examples.

Estimation of f

The following lemma shows that lim k K(ĝ (k) a k ,n , f a k ,n ) converges towards zero as k and then as n:

Lemma 2. We have

lim n lim k K(ĝ (k) a k ,n , f a k ,n ) = 0, a.s.
Then, the following theorem enables us to provide simulations through an estimation of f Theorem 11. We have

lim n lim k ĝ(k) n = f, a.s.

Testing of the criteria

In this paragraph, through a test of Huber's criteria, namely a → K(ĝ

(k)
a,n , f a,n ), we will build a stopping rule for the procedure. First, the next theorem gives us the law of Huber's criteria.

Theorem 12. For a xed k, we have √ n(V ar P (m( bn ( βn ), βn ))) -1/2 (P n m( bn ( βn ), βn ) -P n m(a k , a k )) Law → N (0, I), as n goes to innity, where k represents the k th step of the algorithm, where I is the identity matrix in R d .

Note that k is xed in theorem 12 since βn = arg sup a∈Θ sup b∈Θ P a n m(b, a) where m is a known function of k -see section 4.1. Thus, in the case where K(g

(k)
a , f a ) = 0, we obtain Corollary 3. We have √ n(V ar P (m( bn ( βn ), βn ))) -1/2 (P n m( bn ( βn ), βn ))

Law → N (0, I).
Hence, we propose the test of the null hypothesis (H 0 ) : K(g

(k-1) a k , f a k ) = 0 versus the alternative (H 1 ) : K(g (k-1) a k , f a k ) = 0.
Based on this result, we stop the algorithm, then, dening a k as the last vector generated from Huber's algorithm, we derive from corollary 3 a α-level condence ellipsoid around a k , namely

E k = {b ∈ R d ; √ n(V ar P (m(b, b))) -1/2 P n m(b, b) ≤ q N (0,1) α } where q N (0,1) α
is the quantile of a α-level reduced centered normal distribution and where P n is the empirical measure araising from a realization of the sequences (X 1 , . . . , X n ) and (Y 1 , . . . , Y n ). Consequently, the following corollary provides us with a condence region for the above test: Corollary 4. E k is a condence region for the test of the null hypothesis (H 0 ) versus (H 1 ).

Simulations

We will illustrate this section by detailing several examples. In each example, the rst part of the program will follow our algorithm and will aim at creating a sequence of densities (g (j) ), j = 1, .., k, k < d, such that g(0) = g, g (j) = g (j-1) f a j /[g (j-1) ] a j and K(g (k) , f ) = 0, where K is the relative entropy and a j = arg inf b K(g (j-1) f b /[g (j-1) ] b , f ), for all j = 1, ..., k. Moreover, in a second step, the program will follow Huber's method and will create a sequence of densities (g (j) ), j = 1, .., k, k < d, such that g(0) = g, g (j) = g (j-1) f a j /[g (j-1) ] a j and K(f, g (k) ) = 0, where K is the relative entropy and a j = argsup b K([g (j-1) ] b , f b ), for all j = 1, ..., k. Example 1 : We are in dimension 3(=d), let us consider a sample of 50(=n) values of a random variable X with a density law f dened by, f (x) = N ormal(x 1 + x 2 ).Gumbel(x 0 + x 2 ).Gumbel(x 0 + x 1 ), where the Gumbel law parameters are (-3, 4) and (1, 1) and where the normal distribution parameters are (-5, 2). Let us then generate a Gaussian random variable Y -that we will name g -with a density which presents the same mean and variance as f . In the rst part of the program, we theoretically obtain k = 2, a 1 = (1, 0, 1) and a 2 = (1, 1, 0) (or a 2 = (1, 0, 1) and a 1 = (1, 1, 0) which leads us to the same conclusion). To get this result, we perform the following test (H 0 ) : (a1, a2) = ((1, 0, 1), (1, 1, 0)) versus (H 1 ) : (a1, a2) = ((1, 0, 1), (1, 1, 0)). Moreover, if i represents the last iteration of the algorithm, then √ n(V ar

P (M (c n (γ n ), γ n ))) (-1/2) P n M (c n (γ n ), γ n )
Law → N (0, 1), and then we estimate (a 1 , a 2 ) with the following 0.9(=α) level condence ellipsoid

E i = {b ∈ R 3 ; (V ar P (M (b, b))) -1/2 P n M (b, b) ≤ q N (0,1) α / √ n 0, 2533/7.0710678 = 0.03582203}.
Indeed, if i = 1 represents the last iteration of the algorithm, then a 1 ∈ E 1 , and if i = 2 represents the last iteration of the algorithm, then a 2 ∈ E 2 , and so on, if i represents the last iteration of the algorithm, then a i ∈ E i . Now, if we follow Huber's method, we also theoretically obtain k = 2, a 1 = (1, 0, 1) and a 2 = (1, 1, 0) (or a 2 = (1, 0, 1) and a 1 = (1, 1, 0) which leads us to the same conclusion). To get this result, we perform the following test:

(H 0 ) : (a 1 , a 2 ) = ((1, 0, 1), (1, 1, 0)) versus (H 1 ) : (a 1 , a 2 ) = ((1, 0, 1), (1, 1, 0)). The fact that, if i represents the last iteration of the algorithm, then

√ n(V ar P (m(b n (β n ), β n ))) (-1/2) P n m(b n (β n ), β n )
Law → N (0, 1), enables us to estimate our sequence of (a i ), reduced to (a 1 , a 2 ), through the following 0.9(=α)

Critics of the simulations

We note that as the approximations accumulate and according to the power of the calculators used, we might obtain results above or below the value of the thresholds of the dierent tests. Moreover, in the case where f is unknown, we will never be sure to have reached the minimum or the maximum of the relative entropy: we have indeed used the simulated annealing method to solve our optimisation problem, and therefore it is only when the number of random jumps tends in theory towards innity that the probability to get the minimum or the maximum tends towards 1. We also note that no theory on the optimal number of jumps to implement does exist, as this number depends on the specicities of each particular problem.

Finally, we choose the 50 -4 7 (resp. 50 -2 3 ) for the AMISE of example 1 (resp. example 2). This choice leads us to simulate 50 random variables -see [START_REF] Scott | Multivariate density estimation. Theory, practice, and visualization[END_REF] page 151 -, none of which have been discarded to obtain the truncated sample.

Conclusion

Projection Pursuit is useful in evidencing characteristic structures as well as one-dimensional projections and their associated distributions in multivariate data sets. Huber, in [START_REF] Peter | Projection pursuit[END_REF], shows us how to achieve it through maximization of the relative entropy. The present article demonstrates that our relative entropy minimisation method constitutes a good alternative to Huber's. Indeed, the convergence results and simulations we carried out convincingly fullled our expectations regarding our methodology.

A Reminders

Let us call h a the density of a Z if h is the density of Z, and K the relative entropy or Kullback-Lieber distance. The function K is dened by -considering P and Q, two probabilities:

K(Q, P ) = ϕ( ∂Q ∂P ) dP if P << Q and K(Q, P ) = +∞ otherwise, where ϕ : x → xln(x) -x + 1 is strictly convex.
Let us present some well-known properties of the relative entropy.

Property 2. We have

K(P, Q) = 0 ⇔ P = Q. Property 3. The application Q → K(Q, P ) is : • convex,
• lower semi-continuous (l.s.c.) for the topology that makes all the applications of the form Q → f dQ continuous where f is bounded and continuous and • l.s.c. for the topology of the uniform convergence, and greater than the L 1 distance.

Moreover, corollary (1.29), page 19 of [LIVAJ], enables us to derive: Property 4.

If T : (X, A) → (Y, B) is measurable and if K(P, Q) < ∞, then K(P, Q) ≥ K(P T -1 , QT -1 ), with equality being reached when T is surjective for (P, Q).

A.1 Useful lemmas

Through a reductio ad absurdum argument, we derive lemmas 3 and 4 : Lemma 3. Let f be a density in R d bounded and positive. Then, any projection density of f -that we will name f a , with a ∈ R d * -is also bounded and positive in R.

Lemma 4. Let f be a density in R d bounded and positive. Then any density f (./a x), for any a ∈ R d * , is also bounded and positive.

By induction and from lemmas 3 and 4, we have Lemma 5. If f and g are positive and bounded densities, then g (k) is positive and bounded.

Finally we introduce a last lemma Lemma 6. Let f be an absolutely continuous density, then, for all sequence (a n ) tending to a in R d * , sequence f a n uniformly converges towards f a .

Proof : For all a in R d * , let F a be the cumulative distribution function of a X and ψ a be a complex function dened by ψ a (u, v) = F a (Re(u + iv)) + iF a (Re(v + iu)), for all u and v in R. First, the function ψ a (u, v) is an analytic function, because x → f a (a x) is continuous and as a result of the corollary of Dini's second theorem -according to which "A sequence of cumulative distribution functions which pointwise converges on R towards a continuous cumulative distribution function F on R, uniformly converges towards F on R"we deduct that, for all sequence (a n ) converging towards a, ψ a n uniformly converges towards ψ a . Finally, the Weierstrass theorem, (see proposal (10.1) page 220 of the "Calcul innitésimal" book of Jean Dieudonné), implies that all sequences ψ a,n uniformly converge towards ψ a , for all a n tending to a. We can therefore conclude. 2

B Study of the sample

Let X 1 , X 2 ,..,X m be a sequence of independent random vectors with same density f . Let Y 1 , Y 2 ,..,Y m be a sequence of independent random vectors with same density g. Then, the kernel estimators f m , g m , f a,m and g a,m of f , g, f a and g a , for all a ∈ R d * , almost surely and uniformly converge since we assume that the bandwidth h m of these estimators meets the following conditions (see [BOLE]):

(Hyp):

h m m 0, mh m m ∞, mh m /L(h -1 m ) → m ∞ and L(h -1 m )/LLm → m ∞, with L(u) = ln(u ∨ e). Let us consider A 1 (m, a) = 1 m Σ m i=1 ln{ ga,m(a Y i ) f a,m (a Y i ) } ga,m(a Y i ) g m (Y i ) , A 2 (m, a) = 1 m Σ m i=1 ( ga,m(a X i ) f a,m (a X i ) -1) fa,m(a X i ) f m (X i ) , B 1 (m, a) = 1 m Σ m i=1 ln{ fa,m(a Y i ) ga,m(a Y i ) g m (Y i ) f m (Y i ) } fa,m(a Y i ) ga,m(a Y i ) and B 2 (m, a) = 1 m Σ m i=1 (1 -{ fa,m(a X i ) ga,m(a X i ) g m (X i ) f m (X i ) }).
Our goal is to estimate the maximum of K(g a , f a ) and the minimum of K(g f a ga , f )). To achieve this, it is necessary for us to truncate the samples: Let us consider now a sequence θ m such that θ m → 0, and y m /θ 2 m → 0, where y m is the almost sure convergence rate of the kernel density estimator, i.e. y m = O P (m -2 4+d ) -see lemma 12. We will generate f m , g m and g b,m from the starting sample and we will select the X i and Y i vectors and such that f m (X i ) ≥ θ m and g m (Y i ) ≥ θ m , for all i and for all b ∈ R d * -for Huber's algorithm -and such that f m (X i ) ≥ θ m and g b,m (b Y i ) ≥ θ m , for all i and for all b ∈ R d * -for our algorithm. The vectors meeting these conditions will be called X 1 , X 2 , ..., X n and Y 1 , Y 2 , ..., Y n . Consequently, the next proposition provides us with the condition required to obtain our estimates Proposition 11. Using the notations introduced in [BROKEZ] and in sections 3.1.1 and 4.1, it holds sup

a∈R d * |(A 1 (n, a) -A 2 (n, a)) -K(g a , f a )| → 0 a.s., (7)
sup

a∈R d * |(B 1 (n, a) -B 2 (n, a)) -K(g f a g a , f )| → 0 a.s. ( 8
)
Remark 8. We can take for θ m the expression m -ν , with 0 < ν < 1 4+d .

C Case study : f is known

In this Annex, we will study the case when f and g are known. We will then use the notations introduced in sections 3.1.1 and 3.1.2 with f and g, i.e. no longer with their kernel estimates.

C.1 Convergence study at the k th step of the algorithm:

In this paragraph, when k is less than or equal to d, we will show that the sequence (ǎ k ) n converges towards a k and that the sequence (ǧ (k) ) n converges towards g (k) . Both γn and čn (a) are M-estimators and estimate a k -see [BROKEZ]. We state Proposition 12. Assuming (H 1) to (H 3) hold. Both sup a∈Θ čn (a) -a k and γn tends to a k a.s.

Finally, the following theorem shows us that ǧ(k) converges uniformly almost everywhere towards g (k) , for any k = 1..d.

Theorem 14. Assumimg (H 1) to (H 3) hold. Then, ǧ(k) → n g (k) a.s. and uniformly a.e.

C.2 Asymptotic Inference at the k th step of the algorithm

The following theorem shows that ǧ(k) converges at the rate O P (n -1/2 ) in three dierents cases, namely for any given x, with the L 1 distance and with the relative entropy:

Theorem 15. Assuming (H 0) to (H 3) hold, for any k = 1, ..., d and any x ∈ R d , we have

|ǧ (k) (x) -g (k) (x)| = O P (n -1/2 ), ( 9
)
|ǧ (k) (x) -g (k) (x)|dx = O P (n -1/2 ), ( 10
)
|K(ǧ (k) , f ) -K(g (k) , f )| = O P (n -1/2 ). ( 11
)
The following theorem shows that the laws of our estimators of a k , namely čn (a k ) and γn , converge towards a linear combination of Gaussian variables.

Theorem 16. Assuming that conditions (H 1) to (H 6) hold, then

√ nA.(č n (a k ) -a k ) Law → B.N d (0, P ∂ ∂b M (a k , a k ) 2 ) + C.N d (0, P ∂ ∂a M (a k , a k ) 2 ) and √ nA.(γ n -a k ) Law → C.N d (0, P ∂ ∂b M (a k , a k ) 2 ) + C.N d (0, P ∂ ∂a M (a k , a k ) 2 ) where A = (P ∂ 2 ∂b∂b M (a k , a k )(P ∂ 2 ∂a i ∂a j M (a k , a k ) + P ∂ 2 ∂a i ∂b j M (a k , a k ))), C = P ∂ 2 ∂b∂b M (a k , a k ) and B = P ∂ 2 ∂b∂b M (a k , a k ) + P ∂ 2 ∂a i ∂a j M (a k , a k ) + P ∂ 2 ∂a i ∂b j M (a k , a k ).

C.3 A stopping rule for the procedure

We now assume that the algorithm does not stop after d iterations. We then remark that, it still holds -for any i > d:

• g (i) (x) = g(x)Π i k=1 f a k (a k x) [g (k-1) n ]a k (a k x)
, with g (0) = g.

• K(g (0) , f ) ≥ K(g (1) , f ) ≥ K(g (2) , f )... ≥ 0.

• Theorems 14, 15 and 16. Moreover, through remark 6 page 13 and as explained in section 14 of [START_REF] Peter | Projection pursuit[END_REF], the sequence (K(g (k-1) fa k g (k-1) a k , f )) k≥1 converges towards zero. Then, in this paragraph, we will show that g (i) converges towards f in i. And nally, we will provide a stopping rule for this identication procedure.

C.3.1 Representation of f

Under (H 0), the following proposition shows us that the probability measure with density g (k) converges towards the probability measure with density f : Proposition 13. We have lim k g (k) = f a.s.

C.3.2 Testing of the criteria

Through a test of the criteria, namely a → K(g (k-1) f a g (k-1) a , f ), we will build a stopping rule for this procedure. First, the next theorem enables us to derive the law of the criteria.

Theorem 17. Assuming that (H 1) to (H 3), (H 6) and (H 8) hold. Then,

√ n(V ar P (M (č n (γ n ), γn ))) -1/2 (P n M (č n (γ n ), γn ) -P n M (a k , a k )) Law → N (0, I),
where k represents the k th step of the algorithm and where I is the identity matrix in R d .

Note that k is xed in theorem 17 since γn = arg inf a∈Θ sup c∈Θ P n M (c, a) where M is a known function of k -see section 3.1.1. Thus, in the case where K(g (k-1) f a k g (k-1) a k , f ) = 0, we obtain Corollary 5. Assuming that (H 1) to (H 3), (H 6), (H 7) and (H 8) hold. Then,

√ n(V ar P (M (č n (γ n ), γn ))) -1/2 (P n M (č n (γ n ), γn )) Law → N (0, I).
Hence, we propose the test of the null hypothesis (H 0 ) :

K(g (k-1) fa k g (k-1) a k , f ) = 0 versus (H 1 ) : K(g (k-1) f a k g (k-1) a k
, f ) = 0. Based on this result, we stop the algorithm, then, dening a k as the last vector generated, we derive from corollary 5 a α-level condence ellipsoid around a k , namely

E k = {b ∈ R d ; √ n(V ar P (M (b, b))) -1/2 P n M (b, b) ≤ q N (0,1) α }, where q N (0,1) α
is the quantile of a α-level reduced centered normal distribution. Consequently, the following corollary provides us with a condence region for the above test: Corollary 6. E k is a condence region for the test of the null hypothesis (H 0 ) versus (H 1 ).

D Hypotheses' discussion

D.1 Discussion on (H 2).

We verify this hypothesis in the case where :

• a 1 is the unique element of R d * such that f (./a 1 x) = g(./a 1 x), i.e. K(g(./a 1 x)f a 1 (a 1 x), f ) = 0, (1)
• f and g are bounded and positive, (2)

• there exists a neighborhood V of a k such that, for all b in V and for all positive real A, there exists S > 0 such that g(./b x) ≤ S.f (./b x) with x > A (3).

Let us remark that we obtain the same proof with f , g (k-1) and a k .

First, (1) implies that g

f a 1 ga 1 = f . Hence, 0 > ln( g f f c gc )g f a 1 ga 1 dx = -K(g f c gc , f ) > -K(g, f )
by the very construction of g f c g c . Besides, (2) and (3) imply that there exists a neighborhood V of a k such that, for all c in V , there exists S > 0 such that, for all x in R d , g(./c x) ≤ S.f (./c x). Consequently, we get

|M (c, a 1 , x)| ≤ | -K(g, f )| + | -( g(./c x) f (./c x) -1)| ≤ K(g, f ) + S + 1. Finally, we obtain the existence a neighborhood V of a k such that, for all c in V , |M (c, a k , x)| ≤ H(x) = K(g, f ) + S + 1 (P -a.s.) with PH < ∞.

D.2 Discussion on (H 3).

We verify this hypothesis in the case where a 1 is the unique element of R d * such that f (./a 1 x) = g(./a 1 x), i.e. K(g(./a 1 x)f a 1 (a 1 x), f ) = 0 -we obtain the same proof with f , g (k-1) and a k . Preliminary (A):

Shows that A = {(c, x) ∈ R d * \{a 1 } × R d ; f a 1 (a 1 x) g a 1 (a 1 x) > f c (c x)
gc(c x) and g(x) f c (c x) gc(c x) > f (x)} = ∅ through a reductio ad absurdum, i.e. if we assume A = ∅. Thus, our hypothesis enables us to derive

f (x) = f (./a 1 x)f a 1 (a 1 x) = g(./a 1 x)f a 1 (a 1 x) > g(./c x)f c (c x) > f since fa 1 (a 1 x) ga 1 (a 1 x) ≥ fc(c x) g c (c x) implies g(./a 1 x)f a 1 (a 1 x) = g(x) fa 1 (a 1 x) ga 1 (a 1 x) ≥ g(x) fc(c x) g c (c x) = g(./c x)f c (c x), i.e. f > f . We can therefore conclude. Preliminary (B): Shows that B = {(c, x) ∈ R d * \{a 1 } × R d ; f a 1 (a 1 x)
ga 1 (a 1 x) < fc(c x) g c (c x) and g(x) fc(c x) g c (c x) < f (x)} = ∅ through a reductio ad absurdum, i.e. if we assume B = ∅. Thus, our hypothesis enables us to derive

f (x) = f (./a 1 x)f a 1 (a 1 x) = g(./a 1 x)f a 1 (a 1 x) < g(./c x)f c (c x) < f
We can therefore conclude as above. Let us now prove (H 3):

We have P M (c, a 1 )-P M (c, a) = ln( g(x)f c (c x) gc(c x)f (x) ){ fa 1 (a 1 x) g a 1 (a 1 x) -f c (c x) gc(c x) }g(x)dx. Moreover, the logarithm ln is negative on {x ∈ R d * ; g(x)f c (c x) gc(c x)f (x) < 1} and is positive on {x ∈ R d * ; g(x)f c (c x) gc(c x)f (x) ≥ 1}. Thus, the preliminary studies (A) and (B) show that ln( g(x)f c (c x) gc(c x)f (x) ) and { fa 1 (a 1 x) g a 1 (a 1 x) -f c (c x) gc(c x)
} always present a negative product. We can therefore conclude, since (c, a) → P M (c, a 1 ) -P M (c, a) is not null for all c and for all a = a 1 . 2

E Proofs

This last section includes the proofs of most of the lemmas, propositions, theorems and corollaries contained in the present article.

Remark 9. 1/ (H 0) -according to which f and g are assumed to be positive and bounded -through lemma 5 (see page 19) implies that ǧ(k) and ĝ(k) are positive and bounded.

2/ remark 4 implies that f n , g n , ǧ(k) and ĝ(k) are positive and bounded since we consider a Gaussian kernel.

Proof of propositions 6 and 7. Let us rst study proposition 7.

Without loss of generality, we will prove this proposition with x 1 in lieu of a X. Let us dene g * = gr. We remark that g and g * present the same density conditionally to x 1 . Indeed,

g * 1 (x 1 ) = g * (x)dx 2 ...dx d = r(x 1 )g(x)dx 2 ...dx d = r(x 1 ) g(x)dx 2 ...dx d = r(x 1 )g 1 (x 1
). Thus, we can demonstrate this proposition. We have g(.|x 1 ) = g(x 1 ,...,xn)

g 1 (x 1 )
and g 1 (x 1 )r(x 1 ) is the marginal density of g * . Hence, g * dx = g 1 (x 1 )r(x 1 )g(.|x 1 )dx = g 1 (x 1 ) f 1 (x 1 ) g 1 (x 1 ) ( g(.|x 1 )dx 2 ..dx d )dx 1 = f 1 (x 1 )dx 1 = 1 and since g * is positive, then g * is a density. Moreover,

K(f, g * ) = f {ln(f ) -ln(g * )}dx, (12)
= f {ln(f (.|x 1 )) -ln(g * (.|x 1 )) + ln(f 1 (x 1 )) -ln(g 1 (x 1 )r(x 1 ))}dx, = f {ln(f (.|x 1 )) -ln(g(.|x 1 )) + ln(f 1 (x 1 )) -ln(g 1 (x 1 )r(x 1 ))}dx, (13)
as g * (.|x 1 ) = g(.|x 1 ). Since the minimum of this last equation ( 13) is reached through the mini-

mization of f {ln(f 1 (x 1 )) -ln(g 1 (x 1 )r(x 1 ))}dx = K(f 1 , g 1 r), then property 2 necessarily implies that f 1 = g 1 r, hence r = f 1 /g 1 . Finally, we have K(f, g) -K(f, g * ) = f {ln(f 1 (x 1 )) -ln(g 1 (x 1 ))}dx = K(f 1 , g 1 ),
which completes the demonstration of proposition 7.

Similarly, if we replace f * = f r -1 with f and g with g * , we obtain the proof of proposition 6. 2

Proof of proposition 8. The demonstration is very similar to the one for proposition 7, save for the fact we now base our reasoning at row (12) on g{ln(g * ) -ln(f

)}dx instead of K(f, g * ) = f {ln(f ) -ln(g * )}dx. 2
Proof of proposition 2. The demonstration is also very similar to the one for proposition 7, save for the fact we now base our reasoning at row 12 on K(g

* , f ) = g * {ln(f ) -ln(g * )}dx instead of K(f, g * ) = f {ln(f ) -ln(g * )}dx. 2
Proof of lemma 7.

Lemma 7. If the family

(a i ) i=1...d is a basis of R d then g(./a 1 x, ..., a j x) = n(a j+1 x, ..., a d x) = f (./a 1 x, ..

., a j x).

Putting A = (a 1 , .., a d ), let us determine f in the A basis. Let us rst study the function dened by ψ : R d → R d , x → (a 1 x, .., a d x). We can immediately say that ψ is continuous and since A is a basis, its bijectivity is obvious. Moreover, let us study its Jacobian.

By denition, it is

J ψ (x 1 , . . . , x d ) = ∂ψ 1 ∂x 1 • • • ∂ψ 1 ∂x d • • • • • • • • • ∂ψ d ∂x 1 • • • ∂ψ d ∂x d = a 1,1 • • • a 1,d • • • • • • • • • a d,1 • • • a d,d = |A| = 0 since A is a basis. We can therefore infer : ∀x ∈ R d , ∃!y ∈ R d such that f (x) = |A| -1 Ψ(y), i.e. Ψ (resp. y)
is the expression of f (resp of x) in basis A, namely Ψ(y) = ñ(y j+1 , ..., y d ) h(y 1 , ..., y j ), with ñ and h being the expressions of n and h in basis A. Consequently, our results in the case where the family {a j } 1≤j≤d is the canonical basis of R d , still hold for Ψ in the A basis -see section 2.1.2. And then, Lemma 13.

sup b∈R d * | 1 n Σ n i=1 { g b,n (b X i ) f b,n (b X i ) -1} f b,n (b X i ) f n (X i ) -{ g b (b x) f b (b x) -1}f b (b x)dx| → 0 a.s., sup b∈R d * | 1 n Σ n i=1 ln{ g a,n (a Y i ) f a,n (a Y i ) } g a,n (a Y i ) gn(Y i )
-ln( ga(a x) f a (a x) )g a (a x) dx| → 0 a.s. and

lim n→∞ sup a∈R d * |(B 1 (n, a) -B 2 (n, a)) -K(g f a ga , f )| = 0 a.s. with B 1 (n, a) = 1 n Σ n i=1 ln{ f a,n (a Y i ) g a,n (a Y i ) g n (Y i ) fn(Y i ) } f a,n (a Y i ) g a,n (a Y i ) and B 2 (n, a) = 1 n Σ n i=1 (1 -{ f a,n (a X i ) g a,n (a X i ) g n (X i ) fn(X i ) }). Proof : Let us remark that | 1 n Σ n i=1 { g b,n (b X i ) f b,n (b X i ) -1} f b,n (b X i ) fn(X i ) -{ g b (b x) f b (b x) -1}f b (b x)dx| = | 1 n Σ n i=1 { g b,n (b X i ) f b,n (b X i ) -1} f b,n (b X i ) f n (X i ) -1 n Σ n i=1 g b (b X i ) f b (b X i ) -1} f b (b X i ) f (X i ) + 1 n Σ n i=1 g b (b X i ) f b (b X i ) -1} f b (b X i ) f (X i ) -{ g b (b x) f b (b x) -1}f b (b x)dx| ≤ | 1 n Σ n i=1 { g b,n (b X i ) f b,n (b X i ) -1} f b,n (b X i ) f n (X i ) -1 n Σ n i=1 g b (b X i ) f b (b X i ) -1} f b (b X i ) f (X i ) | +| 1 n Σ n i=1 g b (b X i ) f b (b X i ) -1} f b (b X i ) f (X i ) -{ g b (b x) f b (b x) -1}f b (b x)dx| Moreover, since |{ g b (b x) f b (b x) -1}f b (b x)|dx ≤ 2
, the law of large numbers enables us to state:

| 1 n Σ n i=1 g b (b X i ) f b (b X i ) -1} f b (b X i ) f (X i ) -{ g b (b x) f b (b x) -1}f b (b x)dx| → 0 a.s.. Moreover, | 1 n Σ n i=1 { g b,n (b X i ) f b,n (b X i ) -1} f b,n (b X i ) f n (X i ) -1 n Σ n i=1 g b (b X i ) f b (b X i ) -1} f b (b X i ) f (X i ) | ≤ 1 n Σ n i=1 |{ g b,n (b X i ) f b,n (b X i ) -1} f b,n (b X i ) fn(X i ) -{ g b (b X i ) f b (b X i ) -1} f b (b X i ) f (X i ) | and |{ g b,n (b X i ) f b,n (b X i ) -1} f b,n (b X i ) f n (X i ) -{ g b (b X i ) f b (b X i ) -1} f b (b X i ) f (X i ) | = | g b,n (b X i )-f b,n (b X i ) f n (X i ) -g b (b X i )-f b (b X i ) f (X i ) | ≤ 1 |f (X i )|.|f n (X i )| {|f (X i )|.|g b,n (b X i ) -g b (b X i )| + |f (X i ) -f n (X i )|.|g b (b X i )| +|f (X i )|.|f b,n (b X i ) -f b (b X i )| + |f (X i ) -f n (X i )|.|f b (b X i )|}, through the introduction of terms g b f -g b f and f f b -f f b , ≤ O P (1).y n θ n .(θ n -y n ) = O P (1) 1 θ 2 n y n -θn
, by denitions of θ n and y n , → 0, a.s. because, y n θ 2 n → 0 a.s., by hypothesis on θ n .

Consequently,

1 n Σ n i=1 |{ g b,n (b X i ) f b,n (b X i ) -1} f b,n (b X i ) fn(X i ) -{ g b (b X i ) f b (b X i ) -1} f b (b X i ) f (X i ) | → 0,
as it is a Cesàro mean. This enables us to conclude. Similarly, we show that :

• sup b∈R d * | 1 n Σ n i=1 ln{ g a,n (a Y i ) f a,n (a Y i ) } g a,n (a Y i ) gn(Y i ) -ln( ga(a x) f a (a x)
)g a (a x) dx| → 0 a.s. and then limit 7 page 20 holds,

• lim m→∞ sup a∈R d * |(B 1 (m, a) -B 2 (m, a)) -K(g f a g a , f )| = 0 a.

Proof :

As it is equivalent to prove either our algorithm or Huber's, we will only develop here the proof for our algorithm. Assuming, without any loss of generality, that the a i , i = 1, .., p, are the vectors of the canonical basis, since g (p-1) (x) = g(x) f 1 (x 1 )

g 1 (x 1 ) f 2 (x 2 )
g 2 (x 2 ) ...

f p-1 (x p-1 )
g p-1 (x p-1 ) we derive immediately that g (p-1) p = g p . We note that it is sucient to operate a change in basis on the a i to obtain the general case. 2

Proof of lemma 15.

Lemma 15. If there exits p, p ≤ d, such that K(g (p) , f ) = 0, then the family of (a i ) i=1,..,p -derived from the construction of g (p) -is free and orthogonal.

Proof :

Without any loss of generality, let us assume that p = 2 and that the a i are the vectors of the canonical basis. Using a reductio ad absurdum with the hypotheses a 1 = (1, 0, ..., 0) and that a 2 = (α, 0, ..., 0), where α ∈ R, we get g (1) (x) = g(x 2 , ..,

x d /x 1 )f 1 (x 1 ) and f = g (2) (x) = g(x 2 , .., x d /x 1 )f 1 (x 1 ) fαa 1 (αx 1 ) [g (1)
] αa 1 (αx 1 ) . Hence f (x 2 , .., x d /x 1 ) = g(x 2 , .., x d /x 1 )

fαa 1 (αx 1 ) [g (1)
] αa 1 (αx 1 ) . It consequently implies that f αa 1 (αx 1 ) = [g (1) ] αa 1 (αx 1 ) since

1 = f (x 2 , .., x d /x 1 )dx 2 ...dx d = g(x 2 , .., x d /x 1 )dx 2 ...dx d f αa 1 (αx 1 ) [g (1) ] αa 1 (αx 1 ) = f αa 1 (αx 1 ) [g (1)
] αa 1 (αx 1 ) . Therefore, g (2) = g (1) , i.e. p = 1 which leads to a contradiction. Hence, the family is free. Moreover, using a reductio ad absurdum we get the orthogonality. Indeed, we have f (x)dx = 1 = +∞ = n(a j+1 x, ..., a d x)h(a 1 x, ..., a j x)dx. 2

Proof of lemma 16.

Lemma 16. We have

Θ = {b ∈ Θ | ( g(x) f (x) f b (b x) g b (b x) -1)f (x)dx < ∞}.
We get the result since

( g(x)f b (b x) f (x)g b (b x) -1)f (x) dx = ( g(x)f b (b x) g b (b x) -f (x)) dx = 0. 2
Proof of propositions 12. In the same manner as in Proposition 3.4 of [BROKEZ], we prove this proposition through lemma 16. 2

Proof of propositions 4 and 10. Proposition 4 comes immediately from proposition 11 page 19 and lemma 12 page 20. Similarly, we prove proposition 10 since both sup a∈Θ bn (a) -a k and βn converge toward a k a.s in the case where f is known -see also in annex C where the case when f is known has been carried out in our algorithm. 2

Proof of theorem 14. Using lemma 6 page 19 and since, for any k, g

(k) = g (k-1) f a k g (k-1) a k
, we prove this theorem by induction. 2

Proof of theorems 1 and 8. We prove the theorem by induction. First, by the very denition of the kernel estimator ǧ(0) n = g n converges towards g. Moreover, the continuity of a → f a,n and a → g a,n and proposition 4 imply that ǧ(1)

n = ǧ(0) n f a,n ǧ(0) a,n
converges towards g (1) . Finally, since, for any

k, ǧ(k) n = ǧ(k-1) n f ǎk ,n ǧ(k-1) ǎk ,n
, we conclude similarly as for ǧ(1) n . In a similar manner, we prove theorem 8. 2

Proof of theorem 15. relationship (9). Let us consider

Ψ j = { f ǎj ( ǎj x) [ǧ (j-1) ] ǎj ( ǎj x) - f a j (a j x)
[g (j-1) ]a j (a j x) }. Since f and g are bounded, it is easy to prove that from a certain rank, we get, for any

x given in R d |Ψ j | ≤ max( 1 [ǧ (j-1) ] ǎj ( ǎj x) , 1 [g (j-1) ] a j (a j x) )|f ǎj ( ǎj x) -f a j (a j x)|.
Remark 10. First, based on what we stated earlier, for any given x and from a certain rank, there is a constant R>0 independent from n, such that max(

1 [ǧ (j-1) ] ǎj ( ǎj x) , 1 [g (j-1) ]a j (a j x) ) ≤ R = R(x) = O(1). Second, since ǎk is an M -estimator of a k , its convergence rate is O P (n -1/2 ).
Thus using simple functions, we obtain an upper and lower bound for f ǎj and for f a j and we reach the following conclusion:

|Ψ j | ≤ O P (n -1/2 ). (14)
We nally obtain:

|Π k j=1 f ǎj ( ǎj x) [ǧ (j-1) ] ǎj ( ǎj x) -Π k j=1 f a j (a j x) [g (j-1) ] a j (a j x) | = Π k j=1 f a j (a j x) [g (j-1) ] a j (a j x) |Π k j=1 f ǎj ( ǎj x) [ǧ (j-1) ] ǎj ( ǎj x) [g (j-1) ] a j (a j x) f a j (a j x) -1|.
Based on relationship (14), the expression

f ǎj ( ǎj x)
[ǧ (j-1) ] ǎj ( ǎj x)

[g (j-1) ] a j (a j x) fa j (a j x)

tends towards 1 at a rate of

O P (n -1/2 ) for all j. Consequently, Π k j=1 f ǎj ( ǎj x) [ǧ (j-1) ] ǎj ( ǎj x) [g (j-1) ]a j (a j x) f a j (a j x)
tends towards 1 at a rate of O P (n -1/2 ). Thus from a certain rank, we get

|Π k j=1 f ǎj ( ǎj x) [ǧ (j-1) ] ǎj ( ǎj x) -Π k j=1 f a j (a j x) [g (j-1) ] a j (a j x) | = O P (n -1/2 )O P (1) = O P (n -1/2 ). In conclusion, we obtain |ǧ (k) (x)-g (k) (x)| = g(x)|Π k j=1 f ǎj ( ǎj x) [ǧ (j-1) ] ǎj ( ǎj x) -Π k j=1 f a j (a j x) [g (j-1) ] a j (a j x) | ≤ O P (n -1/2 ).
relationship (10). The relationship 9 of theorem 15 implies that | ǧ(k) (x)

g (k) (x) -1| = O P (n -1/2 ) because, for any given x, g (k) (x)| ǧ(k) (x) g (k) (x) -1| = |ǧ (k) (x) -g (k) (x)|. Consequently, there exists a smooth function C of R d in R + such that lim n→∞ n -1/2 C(x) = 0 and | ǧ(k) (x) g (k) (x) -1| ≤ n -1/2 C(x), for any x. We then have |ǧ (k) (x) -g (k) (x)|dx = g (k) (x)| ǧ(k) (x) g (k) (x) -1|dx ≤ g (k) (x)C(x)n -1/2 dx. Moreover, sup x∈R d |ǧ (k) (x) -g (k) (x)| = sup x∈R d g (k) (x)| ǧ(k) (x) g (k) (x) -1| = sup x∈R d g (k) (x)C(x)n -1/2
→ 0 a.s., by theorem 14. This implies that sup x∈R d g (k) (x)C(x) < ∞ a.s., i.e. sup x∈R d C(x) < ∞ a.s. since g (k) has been assumed to be positive and bounded -see remark 9. Thus, g (k)

(x)C(x)dx ≤ sup C. g (k) (x)dx = sup C < ∞ since g (k) is a density, therefore we can conclude |ǧ (k) (x) -g (k) (x)|dx ≤ sup C.n -1/2 = O P (n -1/2 ).
2 relationship (11). We have

K(ǧ (k) , f ) -K(g (k) , f ) = f (ϕ( ǧ(k) f ) -ϕ( g (k) f ))dx ≤ f S| ǧ(k) f -g (k) f |dx = S |ǧ (k) -g (k)
|dx with the line before last being derived from theorem 13 page 18 and where ϕ : x → xln(x) -x + 1 is a convex function and where S > 0. We get the same expression as the one found in our Proof of Relationship (10) section, we then obtain K(ǧ (k) Proof :

, f ) -K(g (k) , f ) ≤ O P (n -1/2 ). Similarly, we get K(g (k) , f ) -K(ǧ (k) , f ) ≤ O P (n -1/2
Let us rst study the Huber's case.

Let N be the randon variable such that N = Σ m j=1 1 {f m (X j )≥θ m , g m (Y j )≥θ m } . The very denition of X 1 , . . . X m and of Y 1 , . . . Y m and the fact that the X j the Y j are independent, enable us to derive that n = m.P

(f m (X j ) ≥ θ m , g m (Y j ) ≥ θ m ) = m.P(f m (X j ) ≥ θ m ).P(g m (Y j ) ≥ θ m ).
Consequently, let us study P(f m (X i ) ≥ θ m ). Let (ξ i ) i=1...m be the sequence such that, for any i and any

x in R d , ξ i (x) = Π d l=1 1 (2π) 1/2 h l e -1 2 ( x l -X il h l ) 2 -Π d l=1 1 (2π) 1/2 h l e -1 2 ( x l -X il h l ) 2 f (x)dx.
Hence, for any given j and conditionally to X 1 , . . . , X j-1 , X j+1 , . . . , X m , the variables (ξ i (X j ))

i (X j )| ≤ Π d l=1 1 (2π) 1/2 h l + Π d l=1 1 (2π) 1/2 h l |f (x)|dx = 2.(2π) -d/2 Π d l=1 h -1 l since sup x e -1 2 x 2 ≤ 1. Moreover, noting that f m (x) = 1 m Σ m i=1 ξ i (x) + (2π) -d/2 1 m Σ m i=1 Π d l=1 h -1 l e -1 2 ( x l -X il h l ) 2 f (x)dx, we have f m (X j ) ≥ θ m ⇔ 1 m Σ m i=1 ξ i (X j ) + (2π) -d/2 1 m Σ m i=1 Π d l=1 h -1 l e -1 2 ( x l -X il h l ) 2 f (x)dx ≥ θ m ⇔ 1 m -1 Σ m i=1 i =j ξ i (X j ) ≥ (θ m -(2π) -d/2 1 m Σ m i=1 Π d l=1 h -1 l e -1 2 ( x l -X il h l ) 2 f (x)dx - 1 m ξ j (X j )) m m -1 with ξ j (X j ) = 0. Then, dening t (resp. ε) as t = 2.(2π) -d/2 Π d l=1 h -1 l (resp. ε = (θ m -(2π) -d/2 Π d l=1 h -1 l 1 m Σ m i=1 Π d l=1 e -1 2 ( x l -X il h l ) 2 f (x)dx) m m-1 ), the Bennet's inequality -[DEVGY85] page 160-implies that P( 1 m-1 Σ m i=1 i =j ξ i (X j ) ≥ ε/X 1 , . . . , X j-1 , X j+1 , . . . , X m ) ≤ 2.exp(-(m-1)ε 2 4t 2
).

Finally, since the X i are i.i.d. and since ( Π d l=1 e

-1 2 ( x l -y l h l ) 2 f (x)dx)f (y)dy < 1, then the law of large numbers says that 1 m Σ m i=1 Π d l=1 e -1 2 ( x l -X il h l ) 2 f (x)dx → m Π d l=1 e -1 2 ( x l -y l h l ) 2 f (x)f (y)dxdy a.s. Con- sequently, since 0 < ν < 1 4+d , we then obtain -(m-1)ε 2 4t 2 ∼ ∞ -m. 1 4 Π d l=1 e -1 2 ( x l -y l h l ) 2 f (x)f (y)dxdy, i.e. the limit of -(m-1)ε 2 4t 2
is -∞. Thus, from a certain rank, we have e am = O(1), i.e.

P(f m (Y j ) ≥ θ m ) = O(1). Similarly, we get P(g m (Y j ) ≥ θ m ) = O(1). In conclusion, we can say that n = m.P(f m (X j ) ≥ θ m ).P(g m (Y j ) ≥ θ m ) = O(m 2 ).
In the same manner, we derive the same result as above for our method. 2

Proof of theorems 2 and 9. First, from lemma 12, we derive that, for any x, sup a∈R d * |f a,n (a x) -

f a (a x)| = O P (n -2 4+d ). Then, let us consider Ψ j = f ǎj ,n ( ǎj x) ǧ(j-1) ǎj ,n ( ǎj x) - fa j (a j x) g (j-1) a j (a j x)
, we have

Ψ j = 1 ǧ(j-1) ǎj ,n ( ǎj x)g (j-1) a j (a j x) ((f ǎj ,n ( ǎj x)-f a j (a j x))g (j-1) a j (a j x)+f a j (a j x)(g (j-1) a j (a j x)-ǧ (j-1) ǎj ,n ( ǎj x))), i.e. |Ψ j | = O P (n -1 2 1 d=1 -2 4+d 1 d>1 ) since f a j (a j x) = O(1) and g (j-1) a j (a j x) = O(1)
. We can therefore conclude similarly as in theorem 15 and through lemma 17. Similarly, we prove theorem 9. 2

Proof of theorem 16. First of all, let us remark that hypotheses (H 1) to (H 3) imply that γn and čn (a k ) converge towards a k in probability.

Hypothesis (H 4) enables us to derive under the integrable sign after calculation,

P ∂ ∂b M (a k , a k ) = P ∂ ∂a M (a k , a k ) = 0, P ∂ 2 ∂a i ∂b j M (a k , a k ) = P ∂ 2 ∂b j ∂a i M (a k , a k ) = ϕ"( gf a k f ga k ) ∂ ∂a i gf a k f ga k ∂ ∂b j gf a k f ga k f dx, P ∂ 2 ∂a i ∂a j M (a k , a k ) = ϕ ( gf a k f ga k ) ∂ 2 ∂a i ∂a j gf a k f ga k f dx, P ∂ 2 ∂b i ∂b j M (a k , a k ) = -ϕ"( gf a k f ga k ) ∂ ∂b i gf a k f ga k ∂ ∂b j gf a k f ga k f dx,
and consequently P ∂ 2 ∂b i ∂b j M (a k , a k ) = -P ∂ 2 ∂a i ∂b j M (a k , a k ) = -P ∂ 2 ∂b j ∂a i M (a k , a k ), which implies, ∂ 2 ∂a i ∂a j K(g fa k g a k , f ) = P ∂ 2 ∂a i ∂a j M (a k , a k ) -P ∂ 2 ∂b i ∂b j M (a k , a k ), = P ∂ 2 ∂a i ∂a j M (a k , a k ) + P ∂ 2 ∂a i ∂b j M (a k , a k ), = P ∂ 2 ∂a i ∂a j M (a k , a k ) + P ∂ 2 ∂b j ∂a i M (a k , a k ).
By denition of the estimators γn and čn (a k ), we have Proof of theorems 3 and 10. We immediately get the theorem through proposition 11 and theorem 16. 2

Proof of proposition 13. Let us consider ψ, ψ a , ψ (k) , ψ (k) a the characteristic functions of densities f , f a , g (k-1) and [g (k-1) ] a . We have |ψ(ta) -ψ (k-1) (ta)| = |ψ a (t) -ψ (k-1) a (t)| ≤ |f a (a x) -[g (k-1) ] a (a x)|dx, and then sup a |ψ a (t) -ψ (k-1) a (t)| ≤ sup a |f a (a x) -[g (k-1) ] a (a x)|dx ≤ sup a K([g (k-1) ] a , f a ) since ψ(ta) = E(e ita x ) = ψ a (t) -where t ∈ R and a ∈ R d * -and since the relative entropy is greater than the L 1 distance. Therefore, since, as explained in section 14 of Huber's article, we have lim k K([g (k-1) ] a k , f a k ) = 0 we then get lim k g (k) = f -which is the Huber's representation of f . Moreover, we have |ψ(t) -ψ (k) (t)| ≤ |f (x) -g (k) (x)|dx ≤ K(g (k) , f ). As explained in section 14 of Huber's article and through remark 6 page 13 and through the additive relation of proposition 6, we can say Gaussian distributions. As derived at remark 4 of page 5, for all k, the determinant of the covariance of the random vector -with density g (k) -is greater than or equal to the product of a positive constant times the determinant of the covariance of the random vector with density f . Consequently, the form of the kernel estimate implies that there exists an integrable function ϕ such that, for any given k and any given n, we have |g Hence √ n(P n M (č n (a k ), γn ) -PM (a k , a k )) abides by the same limit distribution as √ n(P n M (a k , a k ) -PM (a k , a k )), which is N (0, V ar P (M (a k , a k ))). 2

Proof of theorems 5 and 12. Through proposition 11 and theorem 17, we derive theorem 5.

Similarly, we get theorem 12. 2

Proof of proposition 5. Let us rst show by induction the following assertion P(k) = {g (k) allows a deconvolution g (k) = g (k) * φ} Initialisation : For k = 0, we get the result since g = g (0) is elliptic. Going from k to k + 1 : Let us assume P(k) is true, we then show that P(k + 1). Since the family of a i , i ≤ k + 1 is free -see lemma 15 -then, we dene B as the basis of R d such that its k + 1 rst vectors are the a i , i ≤ k + 1 -see the incomplete basis theorem for its existence. Thus, in B and using the same procedure to prove lemma 7 page 23, we have g (k) (x) = g (k) (./x k+1 )g (k) k+1 (x k+1 ). Consequently, the very denition of the convolution product, the Fubini's theorem and the hypothesis on the Elliptical family used imply that g (k) (x) = g (k) (./x k+1 )g (k) k+1 (x k+1 ) with g (k) (./x k+1 ) = g (k) (./x k+1 ) * E d-1 (0, σ 2 I d-1 , ξ d-1 ) and with g

(k) k+1 (x k+1 ) = g (k)
k+1 (x k+1 ) * E 1 (0, σ 2 , ξ 1 ). Finally, replacing g (k) k+1 with f k+1 = f k+1 * E 1 (0, σ 2 , ξ 1 ), we conclude this induction with g (k+1) = g (k) (./x k+1 )f k+1 (x k+1 ). Now, let us consider ψ (rep. ψ, ψ (k) , ψ (k) ) the characteristic function of f (resp. f , g (k) , g (k) ). We then have ψ(s) = ψ(s)Ψ( 1 2 σ 2 |s| 2 ) and ψ (k) (s) = ψ (k) (s)Ψ( 1 2 σ 2 |s| 2 ). Hence, ψ and ψ (k) are less or equal to Ψ( 1 2 σ 2 |s| 2 ) which is integrable by hypothesis, i.e. ψ and ψ (k) are absolutely integrable. We then obtain g (k) (x) = (2π) -d ψ (k) (s)e -is x ds and f (x) = (2π) -d ψ(s)e -is x ds. Moreover, since the sequence (ψ (k) ) uniformly converges and since ψ and ψ (k) are less or equal to Ψ( 1 2 σ 2 |s| 2 ), then the dominated convergence theorem implies that lim k |f (x) -g (k) (x)| ≤ (2π) -d lim k |ψ(s) -ψ (k) (s)|ds = 0 a.s. i.e. lim k sup x |f (x) -g (k) (x)| = 0 a.s. Finally, since, by hypothesis, (2π) -d |ψ(s) -ψ (k) (s)|ds ≤ 2(2π) -d Ψ( 1 2 σ 2 |s| 2 )ds < ∞, then the above limit and the dominated convergence theorem imply that lim k |f (x) -g (k) (x)|dx = 0. 2 Proof of theorem 7. We immediately get the proof through theorem 4.

2

d

  = g d and f (d) j

  for all x and all the partial derivatives of (b, a) → m(b, a) are dominated in V k by a P_integrable function H(x). (H6) : P ∂ ∂b m(a k , a k ) 2 and P ∂ ∂a m(a k , a k ) 2 are nite and the quantities P ∂ 2 ∂b i ∂b j m(a k , a k ) and P ∂ 2 ∂a i ∂a j m(a k , a k ) are invertible. (H7) : there exists k such that Pm(a k , a k ) = 0. (H8) : (V ar P (m(a k , a k ))) 1/2 exists and is invertible.

4. 3 . 2

 32 Convergence study at the k th step of the algorithm Let bn (a) = arg sup b∈Θ P a n m(b, a), with a ∈ Θ, and βn = arg sup a∈Θ sup b∈Θ P a n m(b, a). We state Proposition 10. Both sup a∈Θ bn (a) -a k and βn converge toward a k a.s.

Figure 1 :Figure 2 :

 12 Figure 1: Graph of the distribution to estimate (red) and of our own estimate (green).

  And nally, according to theorem III.4 of [AZE97], we have Theorem 13. Let f : I → R be a convex function. Then f is a Lipschitz function in all compact intervals [a, b] ⊂ int{I}. In particular, f is continuous on int{I}.

  b, a) = 0P n ∂ ∂a M (b(a), a) = 0 ie P n ∂ ∂b M (č n (a k ), γn ) = 0 P n ∂ ∂a M (č n (a k ), γn ) + P n ∂ ∂b M (č n (a k ), γn ) ∂ ∂a čn (a k ) = 0, i.e. P n ∂ ∂b M (č n (a k ), γn ) = 0 (E0) P n ∂ ∂a M (č n (a k ), γn ) = 0 (E1) .Under (H 5) and (H 6), and using a Taylor development of the (E0) (resp. (E1)) equation, we infer there exists(c n , γ n ) (resp. (c n , γn )) on the interval [(č n (a k ), γn ), (a k , a k )] such that -P n ∂ ∂b M (a k , a k ) = [(P ∂ 2 ∂b∂b M (a k , a k )) + o P (1), (P ∂ 2 ∂a∂b M (a k , a k )) + o P (1)]a n . (resp. -P n ∂ ∂a M (a k , a k ) = [(P ∂ 2 ∂b∂a M (a k , a k )) + o P (1), (P ∂ 2 ∂a 2 M (a k , a k )) + o P (1)]a n ) with a n = ((č n (a k ) -a k ) , (γ n -a k ) ).

  ϕ. Finally, the dominated convergence theorem enables us to say that lim n lim k g(k) n = lim n f n = f , since f n converges towards f and since |[lim n lim k g (k) n (x)] -lim n f n (x)|dx = 0.Similarly, we prove theorem 11. 2Proof of theorem 17. Through a Taylor development of P n M (č n (a k ), γn ) of rank 2, we get at point (a k , a k ):P n M (č n (a k ), γn ) = P n M (a k , a k ) + P n ∂ ∂a M (a k , a k )(γ n -a k ) + P n ∂ ∂b M (a k , a k )(č n (a k ) -a k ) + 1 2 {(γ n -a k ) P n ∂ 2 ∂a∂a M (a k , a k )(γ n -a k ) + (č n (a k ) -a k ) P n ∂ 2 ∂b∂a M (a k , a k )(γ n -a k ) +(γ n -a k ) P n ∂ 2 ∂a∂b M (a k , a k )(č n (a k ) -a k ) + (č n (a k ) -a k ) P n ∂ 2 ∂b∂b M (a k , a k )(č n (a k ) -a k )}The lemma below enables us to conclude.Lemma 19. Let H be an integrable function and let C = H dP andC n = H dP n , then, C n -C = O P ( 1 √ n ).Thus we getP n M (č n (a k ), γn ) = P n M (a k , a k ) + O P ( 1 n ), i.e.√ n(P n M (č n (a k ), γn ) -PM (a k , a k )) = √ n(P n M (a k , a k ) -PM (a k , a k )) + o P (1).

  s. and then limit 8 page 20 holds. 2

	Proof of lemma 14.
	Lemma 14. For any p ≤ d, we have f (p-1) ap	= f a p -see Huber's analytic method -, g	(p-1) ap	= g a p -
	see Huber's synthetic method -and g ap (p-1)	= g a

p -see our algorithm.

  ). We can therefore conclude. 2 Proof of lemma 17. Lemma 17. With the notations introduced in Appendix B. It holds n = O(m 2 ).

  Thus we get√ na n = √ n P ∂ 2 ∂b 2 M (a k , a k ) P ∂ 2 ∂a∂b M (a k , a k ) P ∂ 2 ∂b∂a M (a k , a k ) P ∂ 2 ∂a 2 M (a k , a k ) , f ) P ∂ 2 ∂b∂b M (a k , a k ) P ∂ 2 ∂b∂b M (a k , a k ) P ∂ 2 ∂b∂b M (a k , a k )Moreover, the central limit theorem implies:P n ∂ ∂b M (a k , a k ) (0, P ∂ ∂a M (a k , a k ) 2 ), since P ∂ ∂b M (a k , a k ) = P ∂ ∂a M (a k , a k ) = 0, which leads us to the result. Finally, if f is known, we similarly prove theorem 10. 2

						-1	-P n -P n	∂ ∂b M (a k , a k ) ∂ ∂a M (a k , a k )	+ o P (1)
		=	√ n(P ∂ 2 ∂b∂b M (a k , a k ) ∂ 2 ∂a∂a K(g	fa k g a k	, f )) -1
		.	P ∂ 2 ∂b∂b M (a k , a k ) + ∂ 2 ∂a∂a K(g	fa k g a k	.	-P n -P n	∂ ∂b M (a k , a k ) ∂ ∂a M (a k , a k )	+ o P (1)
						Law → N d (0, P ∂ ∂b M (a k , a k ) 2 ),
	P n	∂ ∂a M (a k , a k )

Law

→ N d

level condence ellipsoid E i = {b ∈ R 3 ; (V ar P (m(b, b))) -1/2 P n m(b, b) ≤ q N (0,1) α / √ n 0.03582203}. Indeed, if i = 1 represents the last iteration of the algorithm, then a 1 ∈ E 1 , and if i = 2 represents the last iteration of the algorithm, then a 2 ∈ E 2 , and so on, if i represents the last iteration of the algorithm, then a i ∈ E i . Finally, we obtain Our Algorithm Huber's Algorithm

Projection Study number 0 : minimum : 0.317505 maximum : 0.715135 at point : (1.0,1.0,0) at point : (1.0,1.0,0) P-Value : 0.99851 P-Value : 0.999839 Test :

H 0 : a 1 ∈ E 1 : False H 0 : a 1 ∈ E 1 : False Projection Study number 1 : minimum : 0.0266514 maximum : 0.00727748 at point : (1.0,0,1.0) at point : (1,0.0,1.0) P-Value : 0.998852 P-Value : 0.999835 Test :

0.444388 0.794124 Therefore, we conclude that f = g (2) . Example 2 : We are in dimension 2(=d), let us consider a sample of 50(=n) values of a random variable X with a density law f dened by, f (x) = Cauchy(x 0 ).N ormal(x 1 ), where the Cauchy law parameters are -5 and 1 and where the normal distribution parameters are (0, 1). Our reasoning is the same as in Example 1. In the rst part of the program, we theoretically obtain k = 1 and a 1 = (1, 0). To get this result, we perform the following test:

(H 0 ) : a 1 = (1, 0) versus (H 1 ) : a 1 = (1, 0). We estimate a 1 by the following 0.75(=α) level condence ellipsoid

Now, if we follow Huber's method, we also theoretically obtain k = 1 and a 1 = (1, 0). To get this result, we perform the following test: (H 0 ) : a 1 = (1, 0) versus (H 1 ) : a 1 = (1, 0). Hence, using the same reasoning as in Example 1, we estimate a 1 through the following 0.75 (=α) level condence ellipsoid

2.32331 Therefore, we conclude that f = g (1) . if g is the expression of g in basis A, we have g(./y 1 , ..., y j ) = ñ(y j+1 , ..., y d ) = Ψ(./y 1 , ..., y j ), i.e. g(./a 1 x, ..., a j x) = n(a j+1 x, ..., a d x) = f (./a 1 x, ..., a j x). 2

Proof of lemma 8. By denition of the closure of a set, we have Lemma 8. The set Γ c is closed in L 1 for the topology of the uniform convergence.

Proof of lemma 9. Since K is greater than the L 1 distance, we have Lemma 9. For all c > 0, we have

Proof of lemma 10. The denition of the closure of a set and lemma 6 (see page 19) imply Lemma 10. G is closed in L 1 for the topology of the uniform convergence.

Proof of lemma 11.

Lemma 11.

From lemmas 8, 9 and 10 (see page 24), we get Γ c ∩ G is a compact for the topology of the uniform convergence, if Γ c ∩ G is not empty. Hence, and since property 3 (see page 18) tells us that Q → K(Q, P ) is lower semi-continuous in L 1 for the topology of the uniform convergence, then the inmum is reached in L 1 . (Taking for example c = K(g, f ), Ω is necessarily not empty because we always have

Proof of lemma 12.

Lemma 12. For any continuous density f , we have

Moreover, from page 150 of [START_REF] Scott | Multivariate density estimation. Theory, practice, and visualization[END_REF], we derive that b m (x) = O P (Σ d j=1 h 2 j ) where h j = O P (m -1 4+d ). Then, we obtain b m (x) = O P (m -2 4+d ). Finally, since the central limit theorem rate is O P (m -1 2 ), we then obtain that

Proof of proposition 11. Let us rst remark that we have f

for all i and for all b ∈ R d * , thanks to the uniform convergence of the kernel estimators. Indeed, we have f

, by denition of y n , and then f (X i ) ≥ -y n + θ n , by hypothesis on f n (X i ). This is also true for g n and g b,n . This entails:

Proof of lemmas 1 and 2. Let us apply our algorithm between f and g. There exists a sequence of densities (g (k) ) k such that 0 = K(g (∞) , f ) ≤ .. ≤ K(g (k) , f ) ≤ .. ≤ K(g, f ), (*) where g (∞) = lim k g (k) which is a density by construction. Moreover, let (g

n is the kernel estimate of g (k) . Since we derive from remark 9 page 22 an integrable upper bound of g

n , for all k, which is greater than f -see also the denition of ϕ in the proof of theorem 4 -, then the dominated convergence theorem implies that, for any k, lim n K(g (k) , f ), i.e., from a certain given rank n 0 , we have 0 ≤ .. ≤ K(g

Consequently, through lemma 18 page 29, there exists a k such that

Finally, through the dominated convergence theorem and taking the limit as n in (***) we get

The dominated convergence theorem enables us to conclude:

n , f n ). Similarly, we get lemma 2. 2

Proof of lemma 18.

Lemma 18. With the notation of the proof of lemma 1, we have

a k+1 , g a k+1 ). Moreover, through remark 6 page 13, we also derive that K(f (k) , g) = K(g (k) , f ). Then, K(f (k) a k+1 , g a k+1 ) is the decreasing step of the relative entropies in (*) and leading to 0 = K(g (∞) , f ). Similarly, through the construction of (**), we obtain that K(f (k) a k+1 ,n , g a k+1 ,n ) is the decreasing step of the relative entropies in (**) and leading to K(g

Second, through the conclusion of the section 4.2 and lemma 14.2 of Huber's article, we obtain that K(f (k) a k+1 ,n , g a k+1 ,n ) converges -in decreasing and in k -towards a positive function of n -that we will call ξ n . Third, the convergence of (g (k) ) k -see proposition 13 -implies that, for any given n, the sequence (K(g

n , f n )) k is not nite. Then, through relationship ( * * ), there exists a k such that 0 < K(g Proof of theorems 4 and 11. We recall that g (k) n is the kernel estimator of ǧ(k) . Since the relative entropy is greater than the L 1 -distance, we then have lim n lim k K(g n is a convex combination of multivariate