Jacques Touboul
email: jack_touboul@hotmail.com

PROJECTION PURSUIT THROUGH Φ-DIVERGENCE MINIMISATION

Keywords: Projection Pursuit, minimum Φ-divergence, Elliptical distribution, goodness-of-fit, copula, regression. 2000 MSC: 94A17 62F05 62J05 62G08

Consider a defined density on a set of very large dimension. It is quite difficult to find an estimate of this density from a data set. However, it is possible through a projection pursuit methodology to solve this problem. In his seminal article, Huber (see "Projection pursuit", Annals of Statistics, 1985) demonstrates the interest of his method in a very simple given case. He considers the factorization of density through a Gaussian component and some residual density. Huber's work is based on maximizing relative entropy. Our proposal leads to a new algorithm. Furthermore, we will also consider the case when the density to be factorized is estimated from an i.i.d. sample. We will then propose a test for the factorization of the estimated density. Applications include a new test of fit pertaining to the Elliptical copulas.

Outline of the article

The objective of Projection Pursuit is to generate one or several projections providing as much information as possible about the structure of the data set regardless of its size:

Once a structure has been isolated, the corresponding data are eliminated from the data set. Through a recursive approach, this process is iterated to find another structure in the remaining data, until no futher structure can be evidenced in the data left at the end. [START_REF] Friedman | Projection pursuit density estimation[END_REF][START_REF] Friedman | Exploratory projection pursuit[END_REF] and [START_REF] Peter | Projection pursuit[END_REF] count among the first authors to have introduced this type of approaches for evidencing structures. They each describe, with many examples, how to evidence such a structure and consequently how to estimate the density of such data through two different methodologies each. Their work is based on maximizing relative entropy. For a very long time, the two methodologies exposed by each of the above authors were thought to be equivalent but Mu [START_REF] Zhu | On the forward and backward algorithms of projection pursuit[END_REF] showed it was in fact not the case when the number of iterations in the algorithms exceeds the dimension of the space containing the data. In the present article, we will therefore only focus on Huber's study while taking into account Mu Zhu remarks.

At present, let us briefly introduce Huber's methodology. We will then expose our approach and objective.

Huber's analytic approach

Let f be a density on R d . We define an instrumental density g with same mean and variance as f . Huber's methodology requires us to start with performing the K( f, g) = 0 test -with K being the relative entropy. Should this test turn out to be positive, then f = g and the algorithm stops. If the test were not to be verified, the first step of Huber's algorithm amounts to defining a vector a 1 and a density f (1) by

a 1 = arg inf a∈R d * K( f g a f a , g) and f (1) = f g a 1 f a 1 , (1.1)
where R d * is the set of non null vectors of R d , where f a (resp. g a ) stands for the density of a ⊤ X (resp. a ⊤ Y) when f (resp. g) is the density of X (resp. Y). More exactly, this results from the maximisation of a → K( f a , g a ) since K( f, g) = K( f a , g a ) + K( f g a f a , g) and it is assumed that K( f, g) is finite. In a second step, Huber replaces f with f (1) and goes through the first step again. By iterating this process, Huber thus obtains a sequence (a 1 , a 2 , ...) of vectors of R d * and a sequence of densities f (i) .

Remark 1.1. Huber stops his algorithm when the relative entropy equals zero or when his algorithm reaches the d th iteration, he then obtains an approximation of f from g : When there exists an integer j such that K( f ( j) , g) = 0 with j ≤ d, he obtains f ( j) = g, i.e. f = gΠ j i=1 f (i-1) a i g a i since by induction f ( j) = f Π j i=1 g a i f (i-1) a i . Similarly, when, for all j, Huber gets K( f ( j) , g) > 0 with j ≤ d, he assumes g = f (d) in order to derive f = gΠ d i=1 f (i-1) a i g a i . He can also stop his algorithm when the relative entropy equals zero without the condition j ≤ d is met. Therefore, since by induction we have f ( j) = f Π j i=1 g a i f (i-1) a i with f (0) = f , we obtain g = f Π j i=1 g a i f (i-1) a i

. Consequently, we derive a representation of f as f = gΠ j i=1 f (i-1) a i g a i .

Finally, he obtains K( f (0) , g) ≥ K( f (1) , g) ≥ ..... ≥ 0 with f (0) = f .

Huber's synthetic approach

Keeping the notations of the above section, we start with performing the K( f, g) = 0 test; should this test turn out to be positive, then f = g and the algorithm stops, otherwise, the first step of his algorithm would consist in defining a vector a 1 and a density g (1) by

a 1 = arg inf a∈R d * K( f, g f a g a
) and g (1) = g f a 1 g a 1 .

(1.2) More exactly, this optimisation results from the maximisation of a → K( f a , g a ) since K( f, g) = K( f a , g a ) + K( f, g f a g a ) and it is assumed that K( f, g) is finite. In a second step, Huber replaces g with g (1) and goes through the first step again. By iterating this process, Huber thus obtains a sequence (a 1 , a 2 , ...) of vectors of R d * and a sequence of densities g (i) .

Remark 1.2. First, in a similar manner to the analytic approach, this methodology enables us to approximate and even to represent f from g: To obtain an approximation of f , Huber either stops his algorithm when the relative entropy equals zero, i.e. K( f, g ( j) ) = 0 implies g ( j) = f with j ≤ d, or when his algorithm reaches the d th 2 iteration, i.e. he approximates f with g (d) .

To obtain a representation of f , Huber stops his algorithm when the relative entropy equals zero, since K( f, g ( j) ) = 0 implies g ( j) = f . Therefore, since by induction we have g ( j) = gΠ j i=1 f a i g (i-1) a i with g (0) = g, we then obtain f = gΠ j i=1 f a i g (i-1) a i

.

Second, he gets K( f, g (0) ) ≥ K( f, g (1) ) ≥ ..... ≥ 0 with g (0) = g.

Proposal

Let us first introduce the concept of Φ-divergence. Let ϕ be a strictly convex function defined by ϕ : R + → R + , and such that ϕ(1) = 0. We define a Φ-divergence of P from Q -where P and Q are two probability distributions over a space Ω such that Q is absolutely continuous with respect to P -by

Φ(Q, P) = ϕ( dQ dP )dP.
Throughout this article, we will also assume that ϕ(0) < ∞, that ϕ ′ is continuous and that this divergence is greater than the L 1 distance -see also Annex A.1 page 18. Now, let us introduce our algorithm. We start with performing the Φ(g, f ) = 0 test; should this test turn out to be positive, then f = g and the algorithm stops, otherwise, the first step of our algorithm would consist in defining a vector a 1 and a density g (1) by

a 1 = arg inf a∈R d * Φ(g f a g a , f
) and g (1) = g f a 1 g a 1 .

(1.3)

Later on, we will prove that a 1 simultaneously optimises (1.1), (1.2) and (1.3).

In our second step, we will replace g with g (1) , and we will repeat the first step. And so on, by iterating this process, we will end up obtaining a sequence (a 1 , a 2 , ...) of vectors in R d * and a sequence of densities g (i) . We will thus prove that the underlying structures of f evidenced through this method are identical to the ones obtained through the Huber's method. We will also evidence the above structures, which will enable us to infer more information on f -see example below.

Remark 1.3. As in the previous algorithm, we first provide an approximate and even a represention of f from g: To obtain an approximation of f , we stop our algorithm when the divergence equals zero, i.e. Φ(g ( j) , f ) = 0 implies g ( j) = f with j ≤ d, or when our algorithm reaches the d th iteration, i.e. we approximate f with g (d) . To obtain a representation of f , we stop our algorithm when the divergence equals zero. Therefore, since by induction we have g ( j) = gΠ j i=1

f a i g (i-1) a i with g (0) = g, we then obtain f = gΠ j i=1 f a i g (i-1) a i . Second, he gets Φ(g (0) , f ) ≥ Φ(g (1) , f ) ≥ ..... ≥ 0 with g (0) = g.
Finally, the specific form of relationship (1.3) establishes that we deal with M-estimation. We can therefore state that our method is more robust than Huber's -see [START_REF] Victor | Optimal robust estimates using the Kullback-Leibler divergence[END_REF], [START_REF] Toma | Optimal robust M-estimators using divergences[END_REF] as well as [START_REF] Peter | Robust Statistics[END_REF].

At present, let us study two examples:

Example 1.1. Let f be a density defined on R 3 by f (x 1 , x 2 , x 3 ) = n(x 1 , x 2 )h(x 3 ), with n being a bi-dimensional Gaussian density, and h being a non Gaussian density. Let us also consider g, a Gaussian density with same mean and variance as f . Since g(x 1 , x 2 /x 3 ) = n(x 1 , x 2 ), we then have Φ(g

f 3 g 3 , f ) = Φ(n. f 3 , f ) = Φ( f, f ) = 0 as f 3 = h, i.e. the function a → Φ(g f a g a , f
) reaches zero for e 3 = (0, 0, 1) ′ . We therefore obtain g(x 1 , x 2 /x 3 ) = f (x 1 , x 2 /x 3 ).

Example 1.2. Assuming that the Φ-divergence is greater than the L 2 norm. Let us consider (X n ) n≥0 , the Markov chain with continuous state space E. Let f be the density of (X 0 , X 1 ) and let g be the normal density with same mean and variance as f . Let us now assume that Φ(g (1) , f ) = 0 with g (1) (x) = g(x) f 1 g 1 , i.e. let us assume that our algorithm stops for a 1 = (1, 0) ′ . Consequently, if (Y 0 , Y 1 ) is a random vector with g density, then the distribution law of X 1 given X 0 is Gaussian and is equal to the distribution law of Y 1 given Y 0 . And then, for any sequence

(A i ) -where A i ⊂ E -we have P X n+1 ∈ A n+1 | X 0 ∈ A 0 , X 1 ∈ A 1 , . . . , X n-1 ∈ A n-1 , X n ∈ A n = P (X n+1 ∈ A n+1 | X n ∈ A n ) , based on the very definition of a Markov chain, = P (X 1 ∈ A 1 | X 0 ∈ A 0 ) , through the Markov property, = P (Y 1 ∈ A 1 | Y 0 ∈ A 0 )
, as a consequence of the above nullity of the Φ-divergence.

To recapitulate our method, if Φ(g, f ) = 0, we derive f from the relationship f = g; should a sequence (a i ) i=1,... j , j < d, of vectors in R d * defining g ( j) and such that Φ(g ( j) , f

) = 0 exist, then f (./a ⊤ i x, 1 ≤ i ≤ j) = g(./a ⊤ i x, 1 ≤ i ≤ j), i.e.
f coincides with g on the complement of the vector subspace generated by the family {a i } i=1,..., j -see also section 2 for a more detailed explanation.

In this paper, after having clarified the choice of g, we will consider the statistical solution to the representation problem, assuming that f is unknown and X 1 , X 2 ,... X m are i.i.d. with density f . We will provide asymptotic results pertaining to the family of optimizing vectors a k,m -that we will define more precisely below -as m goes to infinity. Our results also prove that the empirical representation scheme converges towards the theoretical one. As an application, section 3.4 permits a new test of fit pertaining to the copula of an unknown density f , section 3.5 gives us an estimate of a density deconvoluted with a Gaussian component and section 3.6 presents some applications to the regression analysis. Finally, we will present simulations.

The algorithm

The model

As explained by [START_REF] Friedman | Projection pursuit density estimation[END_REF][START_REF] Friedman | Exploratory projection pursuit[END_REF] and [START_REF] Diaconis | Asymptotics of graphical projection pursuit[END_REF], the choice of g depends on the family of distribution one wants to find in f . Until now, the choice has only been to use the class of Gaussian distributions. This can be extended to the class of elliptic distributions with almost all Φ-divergences.

Elliptical laws

The interest of this class lies in the fact that conditional densities with elliptical distributions are also elliptical -see [START_REF] Cambanis | On the theory of elliptically contoured distributions[END_REF], [START_REF] Landsman | Tail conditional expectations for elliptical distributions[END_REF]. This very property allows us to use this class in our algorithm. Definition 2.1. X is said to abide by a multivariate elliptical distribution -noted X ∼ E d (µ, Σ, ξ d ) -if X presents the following density, for any x in R d :

f X (x) = c d |Σ| 1/2 ξ d 1 2 (x -µ) ′ Σ -1 (x -µ)
• with Σ, being a d × d positive-definite matrix and with µ, being an d-column vector,

• with ξ d , being referred as the "density generator",

• with c d , being a normalisation constant, such that c d = Γ(d/2) (2π) d/2 ∞ 0 x d/2-1 ξ d (x)dx -1 , with ∞ 0 x d/2-1 ξ d (x)dx < ∞. Property 2.1. 1/ For any X ∼ E d (µ, Σ, ξ d ),
for any A, being a m × d matrix with rank m ≤ d, and for any b, being an m-dimensional vector, we have AX [START_REF] Cambanis | On the theory of elliptically contoured distributions[END_REF] states that conditional densities with elliptical distributions are also elliptic. [START_REF] Landsman | Tail conditional expectations for elliptical distributions[END_REF] shows that multivariate Gaussian distributions derive from ξ d (x) = e -x . He also shows that if X = (X 1 , ..., X d ) has an elliptical density such that its marginals verify E(X i ) < ∞ and E(X 2 i ) < ∞ for 1 ≤ i ≤ d, then µ is the mean of X and Σ is the covariance matrix of X. Consequently, from now on, we will assume that we are in this case. Definition 2.2. Let t be an elliptical density on R k and let q be an elliptical density on R k ′ . The elliptical densities t and q are said to belong to the same family -or class -of elliptical densities, if their generating densities are ξ k and ξ k ′ respectively, which belong to a common given family of densities.

+ b ∼ E m (Aµ + b, AΣA ′ , ξ m ). Therefore, any marginal density of multivarite elliptical distribution is elliptic, i.e. X = (X 1 , X 2 , ..., X d ) ∼ E d (µ, Σ, ξ d ) ⇒ X i ∼ E 1 (µ i , σ 2 i , ξ 1 ), f X i (x) = c 1 σ i ξ 1 1 2 ( x-µ i σ ) 2 , 1 ≤ i ≤ d. 2/ Corollary 5 of
Indeed, if X = (X 1 , X 2 ) ′ ∼ E d (µ, Σ, ξ d ), with X 1 (resp. X 2 ) being a size d 1 < d (resp. d 2 < d), then X 1 /(X 2 = a) ∼ E d 1 (µ ′ , Σ ′ , ξ d 1 ) with µ ′ = µ 1 + Σ 12 Σ -1 22 (a -µ 2 ) and Σ ′ = Σ 11 -Σ 12 Σ -1 22 Σ 21 , with µ = (µ 1 , µ 2 ) and Σ = (Σ i j ) 1≤i, j≤2 . Remark 2.1.
Example 2.1. Consider two Gaussian densities N(0, 1) and N((0, 0), Id 2 ). They are said to belong to the same elliptical families as they both present x → e -x as generating density.

Choice of g

Let us begin with studying the following case: Let f be a density on R d . Let us assume there exists d not null independent vectors a j , with

1 ≤ j ≤ d, of R d , such that f (x) = n(a ⊤ j+1 x, ..., a ⊤ d x)h(a ⊤ 1 x, ..., a ⊤ j x), (2.1)
with j < d, with n being an elliptical density on R d-j-1 and with h being a density on R j , which does not belong to the same family as n. Let X = (X 1 , ..., X d ) be a vector presenting f as density. Define g as an Elliptical distribution with same mean and variance as f . For simplicity, let us assume that the family {a j } 1≤ j≤d is the canonical basis of R d :

The very definition of f implies that (X j+1 , ..., X d ) is independent from (X 1 , ..., X j ). Hence, the density of (X j+1 , ..., X d ) given (X 1 , ..., X j ) is n.

Let us assume that Φ(g ( j) , f ) = 0, for some j ≤ d. We then get f (x)

f a 1 f a 2 ... f a j = g(x) g (1-1) a 1 g (2-1) a 2 ...g ( j-1) a j
, since, by induction, we have g

( j) (x) = g(x) f a 1 g (1-1) a 1 f a 2 g (2-1) a 2
...

f a j g ( j-1) a j
.

Consequently, the fact that conditional densities with elliptical distributions are also elliptical as well as the above relationship enable us to infer that n(a ⊤ j+1 x, .,

a ⊤ d x) = f (./a ⊤ i x, 1 ≤ i ≤ j) = g(./a ⊤ i x, 1 ≤ i ≤ j).
In other words, f coincides with g on the complement of the vector subspace generated by the family {a i } i=1,..., j . Now, if the family {a j } 1≤ j≤d is no longer the canonical basis of R d , then this family is again a basis of R d . Hence, lemma F.1 -page 24 -implies that

g(./a ⊤ 1 x, ..., a ⊤ j x) = n(a ⊤ j+1 x, ..., a ⊤ d x) = f (./a ⊤ 1 x, ..., a ⊤ j x), (2.2)
which is equivalent to having Φ(g ( j) , f ) = 0 -since by induction g

( j) = g f a 1 g (1-1) a 1 f a 2 g (2-1) a 2
...

f a j g ( j-1) a j
.

The end of our algorithm implies that f coincides with g on the complement of the vector subspace generated by the family {a i } i=1,..., j . Therefore, the nullity of the Φ-divergence provides us with information on the density structure. In summary, the following proposition clarifies our choice of g which depends on the family of distribution one wants to find in f :

Proposition 2.1. With the above notations, Φ(g ( j) , f

) = 0 is equivalent to g(./a ⊤ 1 x, ..., a ⊤ j x) = f (./a ⊤ 1 x, ..., a ⊤ j x)
More generally, the above proposition leads us to defining the co-support of f as the vector space generated from vectors a 1 , ..., a j . Definition 2.3. Let f be a density on R d . We define the co-vectors of f as the sequence of vectors a 1 , ..., a j which solves the problem Φ(g ( j) , f ) = 0 where g is an Elliptical distribution with same mean and variance as f . We define the co-support of f as the vector space generated from vectors a 1 , ..., a j .

Remark 2.2. Any (a i ) family defining f as in (2.1), is an orthogonal basis of R d -see lemma F.2

Stochastic outline of our algorithm

Let X 1 , X 2 ,..,X m (resp. Y 1 , Y 2 ,..,Y m ) be a sequence of m independent random vectors with same density f (resp. g). As customary in nonparametric Φ-divergence optimizations, all estimates of f and f a as well as all uses of Monté Carlo's methods are being performed using subsamples X 1 , X 2 ,..,X n and Y 1 , Y 2 ,..,Y n -extracted respectively from X 1 , X 2 ,..,X m and Y 1 , Y 2 ,..,Y msince the estimates are bounded below by some positive deterministic sequence θ m -see Annex B. Let P n be the empirical measure of the subsample X 1 , X 2 ,.,X n . Let f n (resp. f a,n for any a in R d * ) be the kernel estimate of f (resp. f a ), which is built from X 1 , X 2 ,..,X n (resp. a ⊤ X 1 , a ⊤ X 2 ,..,a ⊤ X n ). As defined in section 1.3, we introduce the following sequences (a k ) k≥1 and (g (k) ) k≥1 : k) is the density such that g

• a k is a non null vector of R d such that a k = arg min a∈R d * Φ(g (k-1) f a g (k-1) a , f ), (2.3) • g (
(k) = g (k-1) f a k g (k-1) a k with g (0) = g.
The stochastic setting up of the algorithm uses f n and g (0) n = g instead of f and g (0) = g -since g is known. Thus, at the first step, we build the vector ǎ1 which minimizes the Φ-divergence between f n and g f a,n g a and which estimates a 1 : Proposition B.1 page 20 and lemma F.6 page 25 enable us to minimize the Φ-divergence between f n and g f a,n g a . Defining ǎ1 as the argument of this minimization, proposition 3.3 page 8 shows us that this vector tends to a 1 . Finally, we define the density ǧ(1) m as ǧ(1) m = g f ǎ1 ,m g ǎ1 which estimates g (1) through theorem 3.1. Now, from the second step and as defined in section 1.3, the density g (k-1) is unknown. Consequently, once again, we have to truncate the samples: All estimates of f and f a (resp. g (1) and g (1) a ) are being performed using a subsample X 1 , X 2 ,..,X n (resp. Y (1) 1 , Y (1) 2 ,..,Y (1) n ) extracted from X 1 , X 2 ,..,X m (resp. Y (1) 1 , Y (1) 2 ,..,Y (1) m -which is a sequence of m independent random vectors with same density g (1) ) such that the estimates are bounded below by some positive deterministic sequence θ m -see Annex B. Let P n be the empirical measure of the subsample X 1 , X 2 ,..,X n . Let f n (resp. g (1) n , f a,n , g (1) a,n for any a in R d * ) be the kernel estimate of f (resp. g (1) and f a as well as g (1) a ) which is built from X 1 , X 2 ,..,X n (resp. Y (1) 1 , Y (1) 2 ,..,Y (1) n and a ⊤ X 1 , a ⊤ X 2 ,..,a ⊤ X n as well as

a ⊤ Y (1) 1 , a ⊤ Y (1) 2 ,..,a ⊤ Y (1) n ).
The stochastic setting up of the algorithm uses f n and g (1) n instead of f and g (1) . Thus, we build the vector ǎ2 which minimizes the Φ-divergence between f n and g (1) n f a,n g (1) a,n -since g (1) and g (1) a are unknown -and which estimates a 2 . Proposition B.1 page 20 and lemma F.6 page 25 enable us to minimize the Φ-divergence between f n and g (1) n f a,n g (1) a,n . Defining ǎ2 as the argument of this minimization, proposition 3.3 page 8 shows us that this vector tends to a 2 in n. Finally, we define the density ǧ( 2)

n as ǧ(2) n = g (1) n f ǎ2 ,n g (1)
ǎ2 ,n which estimates g (2) through theorem 3.1.

And so on, we will end up obtaining a sequence (ǎ 1 , ǎ2 , ...) of vectors in R d * estimating the covectors of f and a sequence of densities ( ǧ(k)

n ) k such that ǧ(k) n estimates g (k) through theorem 3.1.

Results

Convergence results

Hypotheses on f

In this paragraph, we define the set of hypotheses on f which could possibly be of use in our work. Discussion on several of these hypotheses can be found in Annex E. In this section, to be more legible we replace g with g (k-1) . Let

Θ = R d , Θ Φ = {b ∈ Θ | ϕ * (ϕ ′ ( g(x) f (x) f b (b ⊤ x) g b (b ⊤ x) ))dP < ∞}, M(b, a, x) = ϕ ′ ( g(x) f (x) f b (b ⊤ x) g b (b ⊤ x) )g(x) f a (a ⊤ x) g a (a ⊤ x) dx -ϕ * (ϕ ′ ( g(x) f (x) f b (b ⊤ x) g b (b ⊤ x) )), P n M(b, a) = M(b, a, x)dP n , PM(b, a) = M(b, a, x)dP,
where P is the probability measure presenting f as density. Similarly as in chapter V of [START_REF] Van Der Vaart | Asymptotic statistics, volume 3 of Cambridge Series in Statistical and Probabilistic Mathematics[END_REF], let us define : (H1) : For all ε > 0, there is η > 0, such that for all c ∈ Θ Φ verifying c -

a k ≥ ε, we have PM(c, a) -η > PM(a k , a), with a ∈ Θ. (H2) : ∃ Z < 0, n 0 > 0 such that (n ≥ n 0 ⇒ sup a∈Θ sup c∈{Θ Φ } c P n M(c, a) < Z) (H3) :
There is a neighbourhood V of a k , and a positive function H, such that, for all c ∈ V, we have |M(c, a k , x)| ≤ H(x) (Pa.s.) with PH < ∞, (H4) : There is a neighbourhood V of a k , such that for all ε, there is a η such that for all c ∈ V and a ∈ Θ, verifying aa k ≥ ε, we have PM(c, a k ) < PM(c, a)η. ) , let us now consider three new hypotheses: (H5) : The function ϕ is C 3 in (0, +∞) and there is a neighbourhood

Putting I a k = ∂ 2 ∂a 2 Φ(g f a k g a k , f ), and x → ρ(b, a, x) = ϕ ′ ( g(x) f b (b ⊤ x) f (x)g b (b ⊤ x) ) g(x) f a (a ⊤ x) g a (a ⊤ x
V ′ k of (a k , a k ) such that, for all (b, a) of V ′ k , the gradient ∇( g(x) f a (a ⊤ x) g a (a ⊤ x)
) and the Hessian H( g(x) f a (a ⊤ x) g a (a ⊤ x) ) exist (λ_a.s.), and the first order partial derivatives g(x) f a (a ⊤ x) g a (a ⊤ x) and the first and second order derivatives of (b, a) → ρ(b, a, x) are dominated (λ_a.s.) by λ-integrable functions. (H6) : The function (b, a) → M(b, a) is C 3 in a neighbourhood V k of (a k , a k ) for all x; and the partial derivatives of (b, a) → M(b, a) are all dominated in V k by a P_integrable function H(x). (H7) : P ∂ ∂b M(a k , a k ) 2 and P ∂ ∂a M(a k , a k ) 2 are finite and the expressions P ∂ 2 ∂b i ∂b j M(a k , a k ) and I a k exist and are invertible. Finally, we define (H8) : There exists k such that PM(a k , a k ) = 0. (H9) : (Var P (M(a k , a k ))) 1/2 exists and is invertible. (H0): f and g are assumed to be positive and bounded.

Estimation of the first co-vector of f

Let R be the class of all positive functions r defined on R and such that g(x)r(a ⊤ x) is a density on R d for all a belonging to R d * . The following proposition shows that there exists a vector a such that f a g a minimizes Φ(gr, f ) in r: Proposition 3.1. There exists a vector a belonging to

R d * such that arg min r∈R Φ(gr, f ) = f a g a and r(a ⊤ x) = f a (a ⊤ x) g a (a ⊤ x) .
Remark 3.1. This proposition proves that a 1 simultaneously optimises (1.1), (1.2) and ( 1.3).

In other words, it proves that the underlying structures of f evidenced through our method are identical to the ones obtained through Huber's methods -see also Annex D.

Following [START_REF] Broniatowski | Parametric estimation and tests through divergences and the duality technique[END_REF], let us introduce the estimate of Φ(g

f a,n g a , f n ), through Φ(g f a,n g a , f n ) = M(a, a, x)dP n (x) Proposition 3.2. Let ǎ be such that ǎ := arg inf a∈R d * Φ(g f a,n g a , f n ).
Then, ǎ is a strongly convergent estimate of a, as defined in proposition 3.1.

Let us also introduce the following sequences (ǎ k ) k≥1 and ( ǧ(k) n ) k≥1 , for any given n -see section 2.2.:

• ǎk is an estimate of a k as defined in proposition 3.2 with ǧ(k-1)

n instead of g, • ǧ(k) n is such that ǧ(0) n = g, ǧ(k) n (x) = ǧ(k-1) n (x) f ǎk ,n (ǎ ⊤ k x) [ǧ (k-1) ] ǎk ,n (ǎ ⊤ k x) , i.e. ǧ(k) n (x) = g(x)Π k j=1 f ǎ j ,n (ǎ ⊤ j x) [ǧ ( j-1) ] ǎ j ,n (ǎ ⊤ j x) .
We also note that ǧ(k)

n is a density.

Convergence study at the k th step of the algorithm:

In this paragraph, we will show that the sequence (ǎ k ) n converges towards a k and that the sequence ( ǧ(k) n ) n converges towards g (k) . Let čn (a) = arg sup c∈Θ P n M(c, a), with a ∈ Θ, and γn = arg inf a∈Θ sup c∈Θ P n M(c, a). We state Proposition 3.3. Both sup a∈Θ čn (a)a k and γn converge toward a k a.s.

Finally, the following theorem shows that ǧ(k)

n converges almost everywhere towards g (k) :

Theorem 3.1. It holds ǧ(k) n → n g (k) a.s.

Asymptotic Inference at the k th step of the algorithm

The following theorem shows that ǧ(k) n converges towards g (k) at the rate O P (n -2 2+d ) in three differents cases, namely for any given x, with the L 1 distance and with the relative entropy:

Theorem 3.2. It holds | ǧ(k) n (x) -g (k) (x)| = O P (n -2 2+d ), | ǧ(k) n (x) -g (k) (x)|dx = O P (n -2 2+d ) and |K( ǧ(k) n , f ) -K(g (k) , f )| = O P (n -2 2+d ).
Remark 3.2. With the relative entropy, we have n = O(m 1/2 ) -see lemma F.13. The above rates consequently become O P (m -1 2+d ).

The following theorem shows that the laws of our estimators of a k , namely čn (a k ) and γn , converge towards a linear combination of Gaussian variables.

Theorem 3.3. It holds √ nA.(č n (a k ) -a k ) Law → B.N d (0, P ∂ ∂b M(a k , a k ) 2 ) + C.N d (0, P ∂ ∂a M(a k , a k ) 2 ) and √ nA.( γn -a k ) Law → C.N d (0, P ∂ ∂b M(a k , a k ) 2 ) + C.N d (0, P ∂ ∂a M(a k , a k ) 2 ) where A = P ∂ 2 ∂b∂b M(a k , a k )(P ∂ 2 ∂a i ∂a j M(a k , a k ) + P ∂ 2 ∂a i ∂b j M(a k , a k )), C = P ∂ 2 ∂b∂b M(a k , a k ) and B = P ∂ 2 ∂b∂b M(a k , a k ) + P ∂ 2 ∂a i ∂a j M(a k , a k ) + P ∂ 2 ∂a i ∂b j M(a k , a k ).

A stopping rule for the procedure

In this paragraph, we will call ǧ(k)

n (resp. ǧ(k) a,n ) the kernel estimator of ǧ(k) (resp. ǧ(k) a )
. We will first show that g (k) n converges towards f in k and n. Then, we will provide a stopping rule for this identification procedure.

Estimation of f

The following proposition provides us with an estimate of f :

Theorem 3.4. We have lim n lim k ǧ(k) n = f a.s.
Consequently, the following corollary shows that Φ(g (k-1)

n f a k ,n g (k-1) a k ,n , f a k ,n ) converges towards zero
as k and then as n go to infinity:

Corollary 3.1. We have lim n lim k Φ( ǧ(k) n f a k ,n [ǧ (k) ] a k ,n , f n ) = 0 a.s.

Testing of the criteria

In this paragraph, through a test of our criteria, namely a → Φ( ǧ(k)

n f a,n [ǧ (k) ] a,n , f n ),
we will build a stopping rule for this procedure. First, the next theorem enables us to derive the law of our criteria:

Theorem 3.5. For a fixed k, we have √ n(Var P (M(č n ( γn ), γn ))) -1/2 (P n M(č n ( γn ), γn ) -P n M(a k , a k )) Law → N(0, I),
where k represents the k th step of our algorithm and where I is the identity matrix in R d .

Note that k is fixed in theorem 3.5 since γn = arg inf a∈Θ sup c∈Θ P n M(c, a) where M is a known function of k -see section 3.1.1. Thus, in the case when Φ(g

(k-1) f a k g (k-1) a k , f ) = 0, we obtain Corollary 3.2. We have √ n(Var P (M(č n ( γn ), γn ))) -1/2 P n M(č n ( γn ), γn ) Law → N(0, I).
Hence, we propose the test of the null hypothesis

(H 0 ) : Φ(g (k-1) f a k g (k-1) a k , f ) = 0 versus the alternative (H 1 ) : Φ(g (k-1) f a k g (k-1) a k , f ) 0.
Based on this result, we stop the algorithm, then, defining a k as the last vector generated, we derive from corollary 3.2 a α-level confidence ellipsoid around a k , namely

E k = {b ∈ R d ; √ n(Var P (M(b, b))) -1/2 P n M(b, b) ≤ q N(0,1) α }
where q N(0,1) α is the quantile of a α-level reduced centered normal distribution and where P n is the empirical measure araising from a realization of the sequences (X 1 , . . . , X n ) and (Y 1 , . . . , Y n ). Consequently, the following corollary provides us with a confidence region for the above test:

Corollary 3.3. E k is a confidence region for the test of the null hypothesis (H 0 ) versus (H 1 ).

Goodness-of-fit test for copulas

Let us begin with studying the following case: Let f be a density defined on R 2 and let g be an Elliptical distribution with same mean and variance as f . Assuming first that our algorithm leads us to having Φ(g (2) , f ) = 0 where family (a i ) is the canonical basis of R 2 . Hence, we have g

(2) (x) = g(x) f 1 g 1 f 2 g (1) 2 = g(x) f 1 g 1 f 2 g 2 -through lemma F.7 page 26 -and g (2) = f . Therefore, f = g(x) f 1 g 1 f 2 g 2 , i.e. f f 1 f 2 = g g 1 g 2 ,

and then

∂ 2 ∂x∂y C f = ∂ 2 ∂x∂y C g
where C f (resp. C g ) is the copula of f (resp. g).

At present, let f be a density on R d and let g be the density defined in section 2.1.2. Let us assume that our algorithm implies that Φ(g (d) , f ) = 0. Hence, we have, for any

x ∈ R d , g(x)Π d k=1 f a k (a ⊤ k x) [g (k-1) ] a k (a ⊤ k x) = f (x), i.e. g(x) Π d k=1 g a k (a ⊤ k x) = f (x) Π d k=1 f a k (a ⊤ k x) , since lemma F.7 page 26 implies that g (k-1) a k = g a k if k ≤ d.
Moreover, the family (a i ) i=1...d is a basis of R d -see lemma F.8 page 26. Hence, putting A = (a 1 , ..., a d ) and defining vector y (resp. density f , copula C f of f , density g, copula Cg of g) as the expression of vector x (resp. density f , copula C f of f , density g, copula C g of g) in basis A, the above equality implies

∂ d ∂y 1 ...∂y d C f = ∂ d ∂y 1 ...∂y d Cg .
Finally, we perform a statistical test of the null hypothesis (H 0 ) :

∂ d ∂y 1 ...∂y d C f = ∂ d ∂y 1 ...∂y d Cg versus the alternative (H 1 ) : ∂ d ∂y 1 ...∂y d C f ∂ d ∂y 1 ...∂y d Cg .
Since, under (H 0 ), we have Φ(g (d) , f ) = 0, then, as explained in section 3.3.2, corollary 3.3 provides us with a confidence region for our test.

Theorem 3.6. Keeping the notations of corollary 3.3, we infer that E d is a confidence region for the test of the null hypothesis (H 0 ) versus the alternative hypothesis (H 1 ).

Rewriting of the convolution product

In the present paper, we first elaborated an algorithm aiming at isolating several known structures from initial datas. Our objective was to verify if for a known density on R d , a known density n on R d-j-1 such that, for d > 1, -which is Elliptical -on R d . Let us consider a random vector X such that X = Z 1 + Z 2 and let f be its density. This density can then be written as :

f (x) = n(a ⊤ j+1 x, ..., a ⊤ d x)h(a ⊤ 1 x, ..., a ⊤ j x), ( 3
f (x) = h 1 * h 2 (x) = R d h 1 (x)h 2 (t -x)dt.
Then, the following property enables us to represent f under the form of a product and without the integral sign Proposition 3.4. Let φ be a centered Elliptical density with σ 2 .I d , σ 2 > 0, as covariance matrix, such that it is a product density in all orthogonal coordinate systems and such that its characteristic function s → Ψ( 1 2 |s| 2 σ 2 ) is integrable -see [START_REF] Landsman | Tail conditional expectations for elliptical distributions[END_REF]. Let f be a density on R d which can be deconvoluted with φ, i.e.

f = f * φ = R d f (x)φ(t -x)dt,
where f is some density on R d . Let g (0) be the Elliptical density belonging to the same Elliptical family as f and having same mean and variance as f . Then, the sequence (g (k) ) k converges uniformly a.s. and in L 1 towards f in k, i.e.

lim k→∞ sup x∈R d |g (k) (x) -f (x)| = 0, and lim k→∞ R d |g (k) (x) -f (x)|dx = 0.
Finally, with the notations of section 3.3 and of proposition 3.4, the following theorem enables us to estimate any convolution product of a multivariate Elliptical density φ with a continuous density f :

Theorem 3.7. It holds lim n lim k ǧ(k) n = f * φ a.s.

On the regression

In this section, we will study several applications of our algorithm pertaining to the regression analysis. We define (X 1 , ..., X d ) (resp. (Y 1 , ..., Y d )) as a vector with density f (resp. g -see section 2.1.2).

Remark 3.3. In this paragraph, we will work in the L 2 space. Then, we will first only consider the Φ-divergences which are greater than or equal to the L 2 distance -see [START_REF] Vajda | χ α -divergence and generalized Fisher's information[END_REF]. Note also that the co-vectors of f can be obtained in the L 2 space -see lemma F.6 and proposition B.1.

The basic idea

In this paragraph, we will assume that Θ = R 2 * and that our algorithm stops for j = 1 and a 1 = (0, 1) ′ . The following theorem provides us with the regression of X 1 on X 2 : Theorem 3.8. The probability measure of X 1 given X 2 is the same as the probability measure of Y 1 given Y 2 . Moreover, the regression between X 1 and X 2 is

X 1 = E(Y 1 /Y 2 ) + ε, where ε is a centered random variable orthogonal to E(X 1 /X 2 ). Remark 3.4. This theorem implies that E(X 1 /X 2 ) = E(Y 1 /Y 2 )
. This equation can be used in many fields of research. The Markov chain theory has been used for instance in example 1.2. Moreover, if g is a Gaussian density with same mean and variance as f , then [START_REF] Gilbert | Probabilités, analyse des données et statistique[END_REF]

implies that E(Y 1 /Y 2 ) = E(Y 1 ) + Cov(Y 1 ,Y 2 ) Var(Y 2 ) (Y 2 -E(Y 2 )
) and then

X 1 = E(Y 1 ) + Cov(Y 1 , Y 2 ) Var(Y 2 ) (Y 2 -E(Y 2 )) + ε.

General case

In this paragraph, we will assume that Θ = R d * and that our algorithm stops with j for j < d. Lemma F.9 implies the existence of an orthogonal and free family

(b i ) i= j+1,..,d of R d * such that R d = Vect{a i } ⊥ ⊕ Vect{b k } and such that g(b ⊤ j+1 x, ..., b ⊤ d x/a ⊤ 1 x, ..., a ⊤ j x) = f (b ⊤ j+1 x, ..., b ⊤ d x/a ⊤ 1 x, ..., a ⊤ j x). (3.2)
Hence, the following theorem provides us with the regression of b ⊤ k X, k = 1, ..., d, on (a ⊤ 1 X, ..., a ⊤ j X): Theorem 3.9. The probability measure of (b ⊤ j+1 X, ..., b ⊤ d X) given (a ⊤ 1 X, ..., a ⊤ j X) is the same as the probability measure of

(b ⊤ j+1 Y, ..., b ⊤ d Y) given (a ⊤ 1 Y, ..., a ⊤ j Y). Moreover, the regression of b ⊤ k X, k = 1, ..., d, on (a ⊤ 1 X, ..., a ⊤ j X) is b ⊤ k X = E(b ⊤ k Y/a ⊤ 1 Y 1 , ..., a ⊤ j Y)+b ⊤ k ε, where ε is a centered random vector such that b ⊤ k ε is orthogonal to E(b ⊤ k X/a ⊤ 1 X, ..., a ⊤ j X
). Corollary 3.4. If g is a Gaussian density with same mean and variance as f , and if Cov(X i , X j ) = 0 for any i j, then, the regression of b

⊤ k X, k = 1, ..., d, on (a ⊤ 1 X, ..., a ⊤ j X) is b ⊤ k X = E(b ⊤ k Y)+b ⊤ k ε, where ε is a centered random vector such that b ⊤ k ε is orthogonal to E(b ⊤ k X/a ⊤ 1 X, ..., a ⊤ j X).

Simulations

Let us study four examples. The first involves a χ 2 -divergence, the second a Hellinger distance, the third a Cressie-Read divergence (still with γ = 1.25) and the fourth a Kullback Leibler divergence.

In each example, our program will follow our algorithm and will aim at creating a sequence of densities (g ( j) ), j = 1, .., k, k < d, such that g (0) = g, g ( j) = g ( j-1) f a j /[g ( j-1) ] a j and Φ(g (k) , f ) = 0, with Φ being a divergence and a j = arg inf b Φ(g ( j-1) f b /[g ( j-1) ] b , f ), for all j = 1, ..., k. Moreover, in the second example, we will study the robustness of our method with two ouliers. In the third example, defining (X 1 , X 2 ) as a vector with f as density, we will study the regression of X 1 on X 2 . And finally, in the fourth example, we will perform our goodness-of-fit test for copulas.

Simulation 4.1 (With the χ 2 divergence). We are in dimension 3(=d), and we consider a sample of 50(=n) values of a random variable X with a density law f defined by : f (x) = Gaussian(x1 + x2). Gaussian(x0 + x2). Gumbel(x0 + x1), where the Normal law parameters are (-5, 2) and (1, 1) and where the Gumbel distribution parameters are -3 and 4. Let us generate then a Gaussian random variable Y -that we will name g -with a density presenting the same mean and variance as f . We theoretically obtain k = 1 and a 1 = (1, 1, 0). To get this result, we perform the following test: H0 : a 1 = (1, 1, 0) versus (H 1 ) : a 1 (1, 1, 0). Then, corollary 3.3 enables us to estimate a 1 by the following 0.9(=α) level confidence ellipsoid H 0 : a 1 ∈ E 1 : True χ 2 (Kernel Estimation of g (1) , g (1) ) 6.1726

E 1 = {b ∈ R 3 ; (Var P (M(b, b))) (-1/2) P n M(b, b) ≤ q N(0,1) α / √ n ≃ 0,
Therefore, we conclude that f = g (1) .

Simulation 4.2 (With the Hellinger distance H). We are in dimension 20(=d). We first generate a sample with 100(=n) observations, namely two outliers x = (2, 0, . . . , 0) and 98 values of a random variable X with a density law f defined by f (x) = Gumbel(x 0 ).Normal(x 1 , . . . , x 9 ), where the Gumbel law parameters are -5 and 1 and where the normal distribution is reduced and centered.

Our reasoning is the same as in Simulation 4.1.

In the first part of the program, we theoretically obtain k = 1 and a 1 = (1, 0, . . . , 0). To get this result, we perform the following test (H 0 ) : a 1 = (1, 0, . . . , 0) versus (H 1 ) : a 1 (1, 0, . . . , 0). We estimate a 1 by the following 0.9(=α) level confidence ellipsoid 1) , g (1) ) 3.042174

E i = {b ∈ R 2 ; (Var P (M(b, b))) -1/2 P n M(b, b) ≤ q N(0,
0 : a 1 ∈ E 1 : True H(Estimate of g (
Therefore, we conclude that f = g (1) .

Simulation 4.3 (With the Cressie-Read divergence (Φ)).

We are in dimension 2(=d), and we consider a sample of 50(=n) values of a random variable X = (X 1 , X 2 ) with a density law f defined by f (x) = Gumbel(x 0 ).Normal(x 1 ), where the Gumbel law parameters are -5 and 1 and where the normal distribution parameters are (0, 1). Let us generate then a Gaussian random variable Y -that we will name g -with a density presenting same mean and variance as f . We theoretically obtain k = 1 and a 1 = (1, 0). To get this result, we perform the following test: H0 : a 1 = (1, 0) versus (H 1 ) : a 1 (1, 0). Then, corollary 3.3 enables us to estimate a 1 by the following 0.9(=α) level confidence ellipsoid H 0 : a 1 ∈ E 1 : True Φ(Kernel Estimation of g (1) , g (1) ) 6.47617

E 1 = {b ∈ R 2 ; (Var P (M(b, b))) (-1/2) P n M(b, b) ≤ q N(0,
Therefore, we conclude that f = g (1) .

- At present, keeping the notations of this simulation, let us study the regression of X 1 on X 2 . Our algorithm leads us to infer that the density of X 1 given X 2 is the same as the density of y 1 given Y 2 . Moreover, property A.1 implies that the co-factors of f are the same with all divergence. Consequently, we can use theorem 3.8, i.e. it implies that X 1 = E(Y 1 /Y 2 ) + ε, where ε is a centered random variable orthogonal to E(X 1 /X 2 ). Thus, since g is a Gaussian density, remark 3.4 implies that

5 -4 -3 -2 -1 0 1 2 3 4 -4 -3 -2 -1 0
X 1 = E(Y 1 ) + Cov(Y 1 , Y 2 ) Var(Y 2 ) (Y 2 -E(Y 2 )) + ε.
Now, using the least squares method, we estimate α 1 and α 2 such that X 1 = a 1 + a 2 X 2 + ε. Thus, the following table presents the results of our regression and of the least squares method if we assume that ε is Gaussian.

Our Regression

E(Y 1 ) -4.545483 Cov(Y 1 , Y 2 ) 0.0380534 Var(Y 2 ) 0.9190052 E(Y 2 ) 0.3103752 correlation coefficient (Y 1 , Y 2 ) 0.02158213 Least squares method a 1 -4.34159227 a 2 0.06803317 correlation coefficient (X 1 , X 2 ) 0.04888484 -2 -1 0 1 2 -6 -5 -4 -3 -2 -1 X_2 X_1
Figure 3: Graph of the regression of X1 on X2 based on the least squares method (red) and based on our theory (green).

Simulation 4.4 (With the relative entropy K). We are in dimension 2(=d), and we use the relative entropy to perform our optimisations. Let us consider a sample of 50(=n) values of a random variable X with a density law f defined by : f (x) = c ρ (F Gumbel (x 0 ), F Exponential (x 1 )).Gumbel(x 0 ).Exponential(x 1 ), where :

• c is the Gaussian copula with correlation coefficient ρ = 0.5, • the Gumbel distribution parameters are -1 and 1 and • the Exponential density parameter is 2. Let us generate then a Gaussian random variable Y -that we will name g -with a density presenting the same mean and variance as f . We theoretically obtain k = 2 and (a 1 , a 2 ) = ((1, 0), (0, 1)). To get this result, we perform the following test:

(H 0 ) : (a 1 , a 2 ) = ((1, 0), (0, 1)) versus (H 1 ) : (a 1 , a 2 ) ((1, 0), (0, 1)). Then, theorem 3.6 enables us to verify (H 0 ) by the following 0.9(=α) level confidence ellipsoid H 0 : a 2 ∈ E 2 : True K(Kernel Estimation of g (2) , g (2) ) 4.0680 Therefore, we can conclude that H 0 is verified.

E 2 = {b ∈ R 2 ; (Var P (M(b, b))) (-1/2) P n M(b, b) ≤ q N(0,1) α / √ n ≃ 0,

Critics of the simulations

In the case where f is unknown, we will never be sure to have reached the minimum of the Φ-divergence: we have indeed used the simulated annealing method to solve our optimisation problem, and therefore it is only when the number of random jumps tends in theory towards infinity that the probability to reach the minimum tends to 1. We also note that no theory on the optimal number of jumps to implement does exist, as this number depends on the specificities of each particular problem. Moreover, we choose the 50 -4 4+d (resp. 100 -4 4+d ) for the AMISE of simulations 4.1, 4.2 and 4.3 (resp. simulation 4.4). This choice leads us to simulate 50 (resp. 100) random variables -see [START_REF] Scott | Multivariate density estimation. Theory, practice, and visualization[END_REF] page 151 -, none of which have been discarded to obtain the truncated sample. Finally, we remark that some of the key advantages of our method over Huber's consist in the fact that -since there exist divergences smaller than the relative entropy -our method requires a considerably shorter computation time and also in the in the superiority in robustness of our method.

Conclusion

Projection Pursuit is useful in evidencing characteristic structures as well as one-dimensional projections and their associated distributions in multivariate data. [START_REF] Peter | Projection pursuit[END_REF] shows us how to achieve it through maximization of the relative entropy. The present article shows that our Φ-divergence method constitutes a good alternative to Huber's particularly in terms of regression and robustness as well as in terms of copula's study. Indeed, the convergence results and simulations we carried out, convincingly fulfilled our expectations regarding our methodology.

A. Reminders

A.1. Φ-Divergence

Let us call h a the density of a ⊤ Z if h is the density of Z. Let ϕ be a strictly convex function defined by ϕ : R + → R + , and such that ϕ(1) = 0. Definition A.1. We define the Φ-divergence of P from Q, where P and Q are two probability distributions over a space Ω such that Q is absolutely continuous with respect to P, by

Φ(Q, P) = ϕ( dQ dP )dP. (A.1)
The above expression (A.1) is also valid if P and Q are both dominated by the same probability.

The most used distances (Kullback, Hellinger or χ 2 ) belong to the Cressie-Read family (see [START_REF] Cressie | Multinomial goodness-of-fit tests[END_REF], [START_REF] Csiszár | On topology properties of f -divergences[END_REF] and the books of [START_REF] Friedrich | Convex statistical distances[END_REF]Igor (1987), Pardo Leandro (2006) and [START_REF] Zografos | ϕ-divergence statistics: sampling properties and multinomial goodness of fit and divergence tests[END_REF]). They are defined by a specific ϕ. Indeed, -with the relative entropy, we associate ϕ(x) = xln(x)x + 1 -with the Hellinger distance, we associate ϕ(x) = 2( √ x -1) 2 -with the χ 2 distance, we associate ϕ(x) = 1 2 (x -1) 2 -more generally, with power divergences, we associate ϕ(x) = x γ -γx+γ-1 γ(γ-1) , where γ ∈ R \ (0, 1) -and, finally, with the L 1 norm, which is also a divergence, we associate ϕ(x) = |x -1|. In particular we have the following inequalities:

d L 1 (g, f ) ≤ K(g, f ) ≤ χ 2 (g, f ).
Let us now present some well-known properties of divergences.

Property A.1. We have Φ(P, Q) = 0 ⇔ P = Q.

Property A.2. The application Q → Φ(Q, P) is greater than the L 1 distance, convex, lower semi-continuous (l.s.c.) -for the topology that makes all the applications of the form Q → f dQ continuous where f is bounded and continuous -as well as l.s.c. for the topology of the uniform convergence.

Property A.3 (corollary (1.29), page 19 of Friedrich and Igor (1987)). If T : (X, A) → (Y, B) is measurable and if K(P, Q) < ∞, then K(P, Q) ≥ K(PT -1 , QT -1 ), with equality being reached when T is surjective for (P, Q). By induction and from lemmas A.1 and A.2, we have lemme A.3. If f and g are positive and bounded densities, then g (k) is positive and bounded.

Finally we introduce a last lemma lemme A.4. Let f be an absolutely continuous density, then, for all sequences (a n ) tending to a in R d * , sequence f a n uniformly converges towards f a .

Proof. For all a in R d * , let F a be the cumulative distribution function of a ⊤ X and ψ a be a complex function defined by ψ a (u, v) = F a (Re(u + iv)) + iF a (Re(v + iu)), for all u and v in R. First, the function ψ a (u, v) is an analytic function, because x → f a (a ⊤ x) is continuous and as a result of the corollary of Dini's second theorem -according to which "A sequence of cumulative distribution functions which pointwise converges on R towards a continuous cumulative distribution function F on R, uniformly converges towards F on R"we deduct that, for all sequences (a n ) converging towards a, ψ a n uniformly converges towards ψ a . Finally, the Weierstrass theorem, (see proposal (10.1) page 220 of the "Calcul infinitésimal" book of Jean Dieudonné), implies that all sequences ψ ′ a,n uniformly converge towards ψ ′ a , for all a n tending to a. We can therefore conclude.

B. Study of the sample

Let X 1 , X 2 ,..,X m be a sequence of independent random vectors with same density f . Let Y 1 , Y 2 ,..,Y m be a sequence of independent random vectors with same density g. Then, the kernel estimators f m , g m , f a,m and g a,m of f , g, f a and g a , for all a ∈ R d * , almost surely and uniformly converge since we assume that the bandwidth h m of these estimators meets the following conditions (see [START_REF] Bosq | Theorie De L'Estimation Fonctionnelle[END_REF]):

(Hyp):

h m ց m 0, mh m ր m ∞, mh m /L(h -1 m ) → m ∞ and L(h -1 m )/LLm → m ∞, with L(u) = ln(u ∨ e). Let us consider B 1 (n, a) = 1 n Σ n i=1 ϕ ′ { f a,n (a ⊤ Y i ) g a,n (a ⊤ Y i ) g n (Y i ) f n (Y i ) } f a,n (a ⊤ Y i ) g a,n (a ⊤ Y i ) and B 2 (n, a) = 1 n Σ n i=1 ϕ * {ϕ ′ { f a,n (a ⊤ X i ) g a,n (a ⊤ X i ) g n (X i )
f n (X i }}. Our goal is to estimate the minimum of Φ(g f a g a , f ). To do this, it is necessary for us to truncate our samples: Let us consider now a positive sequence θ m such that θ m → 0, y m /θ 2 n → 0, where y m is the almost sure convergence rate of the kernel density estimator -y m = O P (m -2 4+d ), see lemma F.10y (1) m /θ 2 m → 0, where y (1) m is defined by n is defined by

|ϕ( g m (x) f m (x) f b,m (b ⊤ x) g b,m (b ⊤ x) ) -ϕ( g(x) f (x) f b (b ⊤ x) g b (b ⊤ x) )| ≤ y (1)
|ϕ ′ ( g m (x) f m (x) f b,m (b ⊤ x) g b,m (b ⊤ x) ) -ϕ ′ ( g(x) f (x) f b (b ⊤ x) g b (b ⊤ x) )| ≤ y (2)
m for all b in R d * and all x in R d . We will generate f m , g m and g b,m from the starting sample and we will select the X i and Y i vectors such that f m (X i ) ≥ θ m and g b,m (b ⊤ Y i ) ≥ θ m , for all i and for all b ∈ R d * . The vectors meeting these conditions will be called X 1 , X 2 , ..., X n and Y 1 , Y 2 , ..., Y n . Consequently, the next proposition provides us with the condition required for us to derive our estimations Proposition B.1. Using the notations introduced in [START_REF] Broniatowski | Parametric estimation and tests through divergences and the duality technique[END_REF] and in section 3.1.1, it holds

lim n→∞ sup a∈R d * |(B 1 (n, a) -B 2 (n, a)) -Φ(g f a g a , f )| = 0. Remark B.1.
With the relative entropy, we can take for θ m the expression m -ν , with 0 < ν < 1 4+d .

C. Case study : f is known

In this Annex, we will study the case when f and g are known. We will then use the notations introduced in sections 3.1.1 and 3.1.2 with f and g, i.e. no longer with their kernel estimates.

C.1. Convergence study and Asymptotic Inference at the k th step of the algorithm

In this paragraph, when k is less than or equal to d, we will show that the sequence (ǎ k ) n converges towards a k and that the sequence ( ǧ(k) ) n converges towards g (k) . Both γn and čn (a) are M-estimators and estimate a k -see [START_REF] Broniatowski | Parametric estimation and tests through divergences and the duality technique[END_REF]. We state Proposition C.1. Assuming (H1) to (H3) hold. Both sup a∈Θ čn (a)a k and γn tends to a k a.s.

Finally, the following theorem shows us that ǧ(k) converges uniformly almost everywhere towards g (k) , for any k = 1..d.

Theorem C.1. Assumimg (H1) to (H3) hold. Then, ǧ(k) → n g (k) a.s. and uniformly a.e.

The following theorem shows that ǧ(k) converges at the rate O P (n -1/2 ) in three differents cases, namely for any given x, with the L 1 distance and with the Φ-divergence: Theorem C.2. Assuming (H0) to (H3) hold, for any k = 1, ..., d and any x ∈ R d , we have

| ǧ(k) (x) -g (k) (x)| = O P (n -1/2 ), (C.1) | ǧ(k) (x) -g (k) (x)|dx = O P (n -1/2 ), (C.2) |K( ǧ(k) , f ) -K(g (k) , f )| = O P (n -1/2 ). (C.3)
The following theorem shows that the laws of our estimators of a k , namely čn (a k ) and γn , converge towards a linear combination of Gaussian variables.

Theorem C.3. Assuming that conditions (H1) to (H6) hold, then √ nA.(č n (a k ) -a k ) Law → B.N d (0, P ∂ ∂b M(a k , a k ) 2 ) + C.N d (0, P ∂ ∂a M(a k , a k ) 2 ) and √ nA.( γn -a k ) Law → C.N d (0, P ∂ ∂b M(a k , a k ) 2 ) + C.N d (0, P ∂ ∂a M(a k , a k ) 2 ) where A = (P ∂ 2 ∂b∂b M(a k , a k )(P ∂ 2 ∂a i ∂a j M(a k , a k ) + P ∂ 2 ∂a i ∂b j M(a k , a k ))), C = P ∂ 2 ∂b∂b M(a k , a k ) and B = P ∂ 2 ∂b∂b M(a k , a k ) + P ∂ 2 ∂a i ∂a j M(a k , a k ) + P ∂ 2 ∂a i ∂b j M(a k , a k ). 20

C.2. A stopping rule for the procedure

We now assume that the algorithm does not stop after d iterations. We then remark that, it still holds -for any i > d:

•

g (i) (x) = g(x)Π i k=1 f a k (a ⊤ k x) [g (k-1) n ] a k (a ⊤ k x) , with g (0) = g. • K(g (0) , f ) ≥ K(g (1) , f ) ≥ K(g (2) , f )... ≥ 0. • Theorems C.1, C.2 and C.3.
Moreover, as explained in section 14 of [START_REF] Peter | Projection pursuit[END_REF] for the relative entropy, the sequence (Φ(g (k-1) f a k g (k-1) a k , f )) k≥1 converges towards zero. Then, in this paragraph, we will show that g (i) converges towards f in i. And finally, we will provide a stopping rule for this identification procedure.

C.2.1. Representation of f

Under (H0), the following proposition shows us that the probability measure with density g (k) converges towards the probability measure with density f :

Proposition C.2 (Representation of f ). We have lim k g (k) = f a.s.

C.2.2. Testing of the criteria

Through a test of the criteria, namely a → Φ(g (k-1) f a g (k-1) a , f ), we will build a stopping rule for this procedure. First, the next theorem enables us to derive the law of the criteria. Hence, we propose the test of the null hypothesis (H 0 ) :

K(g (k-1) f a k g (k-1) a k , f ) = 0 versus (H 1 ) : K(g (k-1) f a k g (k-1) a k , f ) 0.
Based on this result, we stop the algorithm, then, defining a k as the last vector generated, we derive from corollary C.1 a α-level confidence ellipsoid around a k , namely

E k = {b ∈ R d ; √ n(Var P (M(b, b))) -1/2 P n M(b, b) ≤ q N(0,1)
α }, where q N(0,1) α is the quantile of a α-level reduced centered normal distribution. Consequently, the following corollary provides us with a confidence region for the above test:

Corollary C.2. E k is a confidence region for the test of the null hypothesis (H 0 ) versus (H 1 ).

D. The first co-vector of f simultaneously optimizes four problems

Let us first study Huber's analytic approach. Let R ′ be the class of all positive functions r defined on R and such that f (x)r -1 (a ⊤ x) is a density on R d for all a belonging to R d * . The following proposition shows that there exists a vector a such that f a g a minimizes K( f r -1 , g) in r:

Proposition D.1 (Analytic Approach). There exists a vector a belonging to R d * such that arg min r∈R ′ K( f r -1 , g) = f a g a , and r(a ⊤ x) = f a (a ⊤ x) g a (a ⊤ x) as well as K( f, g) = K( f a , g a ) + K( f g a f a , g). Let us also study Huber's synthetic approach: Let R be the class of all positive functions r defined on R and such that g(x)r(a ⊤ x) is a density on R d for all a belonging to R d * . The following proposition shows that there exists a vector a such that f a g a minimizes K(gr, f ) in r:

Proposition D.2 (Synthetic Approach

). There exists a vector a belonging to R d * such that arg min r∈R K( f, gr) = f a g a , and r(a

⊤ x) = f a (a ⊤ x) g a (a ⊤ x) as well as K( f, g) = K( f a , g a ) + K( f, g f a g a ).
In the meanwhile, the following proposition shows that there exists a vector a such that f a g a minimizes K(g, f r -1 ) in r.

Proposition D.3. There exists a vector a belonging to R d

* such that arg min r∈R ′ K(g, f r -1 ) = f a g a , and r(a

⊤ x) = f a (a ⊤ x) g a (a ⊤ x) as well as K(g, f ) = K(g a , f a ) + K(g, f g a f a ).
Remark D.1. First, through property A.3 page 18, we get K( f, g f a g a ) = K(g, f g a f a ) = K( f g a f a , g) and K( f a , g a ) = K(g a , f a ). Thus, proposition D.3 implies that finding the argument of the maximum of K(g a , f a ) amounts to finding the argument of the maximum K( f a , g a ). Consequently, the criteria of Huber's methodologies is a → K(g a , f a ). Second, if the Φ-divergence is the relative entropy, then our criteria is a → K(g g a f a , f ) and property A.

3 implies K(g, f g a f a ) = K(g f a g a , f ).
To recapitulate, the choice of r = f a g a enables us to simultaneously solve the following four optimisation problems, for a ∈ R d * : First, find a such that a = arginf a∈R d * K( f g a f a , g), Second, find a such that a = arginf a∈R d * K( f, g f a g a ), Third, find a such that a = argsup a∈R d * K(g a , f a ), Fourth, find a such that a = arginf a∈R d * K(g f a g a , f ).

E. Hypotheses' discussion

E.1. Discussion of (H2).

Let us work with the relative entropy and with g and a 1 .

For all b ∈ R d * , we have ϕ * (ϕ ′ ( g(x) f b (b ⊤ x) f (x)g b (b ⊤ x) )) f (x)dx = ( g(x) f b (b ⊤ x) f (x)g b (b ⊤ x) -1) f (x)dx = 0, since, for any b in R d * , the function x → g(x) f b (b ⊤ x) g b (b ⊤ x) is a density. The complement of Θ Φ in R d
* is ∅ and then the supremum looked for in R is -∞. We can therefore conclude. It is interesting to note that we obtain the same verification with f , g (k-1) and a k .

E.2. Dicussion of (H4). This hypothesis consists in the following assumptions:

• We work with the relative entropy, (0)

• We have f (./a ⊤ 1 x) = g(./a ⊤ 1 x), i.e. K(g f 1 g 1 , f ) = 0 -we could also derive the same proof with f , g (k-1) and a k -(1) Preliminary (A): Shows that A = {(c, x) ∈ R d * \{a 1 }×R d ; f a 1 (a ⊤ 1 x) g a 1 (a ⊤
through a reductio ad absurdum, i.e. if we assume A ∅. Thus, our hypothesis enables us to derive

f (x) = f (./a ⊤ 1 x) f a 1 (a ⊤ 1 x) = g(./a ⊤ 1 x) f a 1 (a ⊤ 1 x) > g(./c ⊤ x) f c (c ⊤ x) > f since f a 1 (a ⊤ 1 x) g a 1 (a ⊤ 1 x) ≥ f c (c ⊤ x) g c (c ⊤ x) implies g(./a ⊤ 1 x) f a 1 (a ⊤ 1 x) = g(x) f a 1 (a ⊤ 1 x) g a 1 (a ⊤ 1 x) ≥ g(x) f c (c ⊤ x) g c (c ⊤ x) = g(./c ⊤ x) f c (c ⊤ x), i.e. f > f . We can therefore conclude. Preliminary (B): Shows that B = {(c, x) ∈ R d * \{a 1 }×R d ; f a 1 (a ⊤ 1 x) g a 1 (a ⊤ 1 x) < f c (c ⊤ x) g c (c ⊤ x) , g(x) f c (c ⊤ x) g c (c ⊤ x) < f (x)} = ∅ through a reductio
ad absurdum, i.e. if we assume B ∅. Thus, our hypothesis enables us to derive

f (x) = f (./a ⊤ 1 x) f a 1 (a ⊤ 1 x) = g(./a ⊤ 1 x) f a 1 (a ⊤ 1 x) < g(./c ⊤ x) f c (c ⊤ x) < f
We can therefore conclude as above.

Let us now verify (H4):

We have PM(c, a 1 )-PM(c, a) = ln( g(x) f c (c ⊤ x) g c (c ⊤ x) f (x) ){ f a 1 (a ⊤ 1 x) g a 1 (a ⊤ 1 x) -f c (c ⊤ x) g c (c ⊤ x) }g(x)dx. Moreover, the logarithm ln is negative on {x ∈ R d * ; g(x) f c (c ⊤ x) g c (c ⊤ x) f (x) < 1} and is positive on {x ∈ R d * ; g(x) f c (c ⊤ x) g c (c ⊤ x) f (x) ≥ 1}. Thus, the preliminary studies (A) and (B) show that ln( g(x) f c (c ⊤ x) g c (c ⊤ x) f (x) ) and { f a 1 (a ⊤ 1 x) g a 1 (a ⊤ 1 x) -f c (c ⊤ x) g c (c ⊤ x)
} always present a negative product. We can therefore conclude, since (c, a) → PM(c, a 1 ) -PM(c, a) is not null for all c and for all a -with a a 1 .

F. Proofs

This last section includes the proofs of most of the lemmas, propositions, theorems and corollaries contained in the present article.

Remark F.1. 1/ (H0) -according to which f and g are assumed to be positive and boundedthrough lemma A.3 (see page 19) implies that ǧ(k) and ĝ(k) are positive and bounded. 2/ remark 2.1 page 5 implies that f n , g n , ǧ(k) and ĝ(k) are positive and bounded since we consider a Gaussian kernel.

Proof of propositions D.1 and D.2. Let us first study proposition D.2. Without loss of generality, we will prove this proposition with x 1 in lieu of a ⊤ X. Let us define g * = gr. We remark that g and g * present the same density conditionally to x 1 . Indeed, g * 1 (x 1 ) = g * (x)dx 2 ...dx d = r(x 1 )g(x)dx 2 ...dx d = r(x 1 ) g(x)dx 2 ...dx d = r(x 1 )g 1 (x 1 ). Thus, we can demonstrate this proposition. We have g(.|x 1 ) = g(x 1 ,...,x n ) g 1 (x 1 ) and g 1 (x 1 )r(x 1 ) is the marginal density of g * . Hence, g * dx = g 1 (x 1 )r(x 1 )g(.|x 1 )dx = g 1 (x 1 ) f 1 (x 1 ) g 1 (x 1 ) ( g(.|x 1 )dx 2 ..dx d )dx 1 = f 1 (x 1 )dx 1 = 1 and since g * is positive, then g * is a density. Moreover,

K( f, g * ) = f {ln( f ) -ln(g * )}dx, (F.1) = f {ln( f (.|x 1 )) -ln(g * (.|x 1 )) + ln( f 1 (x 1 )) -ln(g 1 (x 1 )r(x 1 ))}dx, = f {ln( f (.|x 1 )) -ln(g(.|x 1 )) + ln( f 1 (x 1 )) -ln(g 1 (x 1 )r(x 1 ))}dx, (F.2)
as g * (.|x 1 ) = g(.|x 1 ). Since the minimum of this last equation (F.2) is reached through the minimization of f {ln( f 1 (x 1 ))ln(g 1 (x 1 )r(x 1 ))}dx = K( f 1 , g 1 r), then property A.1 necessarily implies that f 1 = g 1 r, hence r = f 1 /g 1 .

Finally, we have K( f, g) -K( f, g * ) = f {ln( f 1 (x 1 ))ln(g 1 (x 1 ))}dx = K( f 1 , g 1 ), which completes the demonstration of proposition D.2.

Similarly, if we replace f * = f r -1 with f and g with g * , we obtain the proposition D.1. 2 Proof of proposition D.3. The demonstration is very similar to the one for proposition D.2, save for the fact we now base our reasoning at row (F.1) on g{ln(g * )ln( f

)}dx instead of K( f, g * ) = f {ln( f ) -ln(g * )}dx.
2 Proof of proposition 3.1. Without loss of generality, we reason with x 1 in lieu of a ⊤ x. Let us define g * = gr. We remark that g and g * present the same density conditionally to x 1 . Indeed,

g * 1 (x 1 ) = g * (x)dx 2 ...dx d = h(x 1 )g(x)dx 2 ...dx d = h(x 1 ) g(x)dx 2 ...dx d = h(x 1 )g 1 (x 1
). We can therefore prove this proposition. First, since f and g are known, then, for any given function h : x 1 → h(x 1 ), the application T , which is defined by: T : g(./x 1 ) h(x 1 ) f 1 (x 1 )

g 1 (x 1 )
→ g(./x 1 ) f 1 (x 1 ), T : f (./x 1 ) f 1 (x 1 ) → f (./x 1 ) f 1 (x 1 ) is measurable. Second, the above remark implies that Φ(g * , f ) = Φ(g * (./x 1 ) g 1 (x 1 )h(x 1 ) f 1 (x 1 ) , f (./x 1 ) f 1 (x 1 )) = Φ(g(./x 1 ) g 1 (x 1 )h(x 1 ) f 1 (x 1 ) , f (./x 1 ) f 1 (x 1 )). Consequently, property A.3 page 18 infers : Φ(g(./x 1 ) g 1 (x 1 )h(x 1 )

f 1 (x 1 ) , f (./x 1 ) f 1 (x 1 )) ≥ Φ(T -1 (g(./x 1 ) g 1 (x 1 )h(x 1 ) f 1 (x 1 ) ), T -1 ( f (./x 1 ) f 1 (x 1 ))) = Φ(g(./x 1 ) f 1 (x 1 ), f (./x 1 ) f 1 (x 1 )), by the very definition of T . = Φ(g f 1 g 1 , f
), which completes the proof of this proposition.

2 Proof of lemma F.1. lemme F.1. We have g(./a ⊤ 1 x, ..., a ⊤ j x) = n(a ⊤ j+1 x, ..., a ⊤ d x) = f (./a ⊤ 1 x, ..., a ⊤ j x). Putting A = (a 1 , .., a d ), let us determine f in basis A. Let us first study the function defined by ψ : R d → R d , x → (a ⊤ 1 x, .., a ⊤ d x). We can immediately say that ψ is continuous and since A is a basis, its bijectivity is obvious. Moreover, let us study its Jacobian.

By definition, it is J ψ (x 1 , . . . , Let us note first that we will prove this proposition for k ≥ 2, i.e. in the case where g (k-1) is not known. The initial case using the known density g (0) = g, will be an immediate consequence from the above. Moreover, going forward, to be more legible, we will use g (resp. g n ) in lieu of g (k-1) (resp. g (k-1) n

x d ) = ∂ψ 1 ∂x 1 • • • ∂ψ 1 ∂x d • • • • • • • • • ∂ψ d ∂x 1 • • • ∂ψ d ∂x d = a 1,1 • • • a 1,d • • • • • • • • • a d,1 • • • a d,d = |A| 0 since A is a basis. We can therefore infer : ∀x ∈ R d , ∃!y ∈ R d such that f (x) = |A| -1 Ψ(
). We can therefore remark that we have f

(X i ) ≥ θ n -y n , g(Y i ) ≥ θ n -y n and g b (b ⊤ Y i ) ≥ θ n -y n ,
for all i and for all b ∈ R d * , thanks to the uniform convergence of the kernel estimators. Indeed, we have f

(X i ) = f (X i ) -f n (X i ) + f n (X i ) ≥ -y n + f n (X i )
, by definition of y n , and then f (X i ) ≥ -y n + θ n , by hypothesis on f n (X i ). This is also true for g n and g b,n . This entails sup b∈R

d * | 1 n Σ n i=1 ϕ ′ ( f a,n (a ⊤ Y i ) g a,n (a ⊤ Y i ) g n (Y i ) f n (Y i ) ). f a,n (a ⊤ Y i ) g a,n (a ⊤ Y i ) -ϕ ′ ( g(x) f (x) f b (b ⊤ x) g b (b ⊤ x) )g(x) f a (a ⊤ x) g a (a ⊤ x) dx| → 0 a.s. Indeed, let us remark that | 1 n Σ n i=1 {ϕ ′ { f a,n (a ⊤ Y i ) g a,n (a ⊤ Y i ) g n (Y i ) f n (Y i ) } f a,n (a ⊤ Y i ) g a,n (a ⊤ Y i ) } -ϕ ′ ( g(x) f (x) f b (b ⊤ x) g b (b ⊤ x) ) g(x) f a (a ⊤ x) g a (a ⊤ x) dx| = | 1 n Σ n i=1 ϕ ′ { f a,n (a ⊤ Y i ) g a,n (a ⊤ Y i ) g n (Y i ) f n (Y i ) } f a,n (a ⊤ Y i ) g a,n (a ⊤ Y i ) -1 n Σ n i=1 ϕ ′ { f a (a ⊤ Y i ) g a (a ⊤ Y i ) g(Y i ) f (Y i ) } f a (a ⊤ Y i ) g a (a ⊤ Y i ) + 1 n Σ n i=1 ϕ ′ { f a (a ⊤ Y i ) g a (a ⊤ Y i ) g(Y i ) f (Y i ) } f a (a ⊤ Y i ) g a (a ⊤ Y i ) -ϕ ′ ( g(x) f (x) f b (b ⊤ x) g b (b ⊤ x) ) g(x) f a (a ⊤ x) g a (a ⊤ x) dx| ≤ | 1 n Σ n i=1 ϕ ′ { f a,n (a ⊤ Y i ) g a,n (a ⊤ Y i ) g n (Y i ) f n (Y i ) } f a,n (a ⊤ Y i ) g a,n (a ⊤ Y i ) -1 n Σ n i=1 ϕ ′ { f a (a ⊤ Y i ) g a (a ⊤ Y i ) g(Y i ) f (Y i ) } f a (a ⊤ Y i ) g a (a ⊤ Y i ) | +| 1 n Σ n i=1 ϕ ′ { f a (a ⊤ Y i ) g a (a ⊤ Y i ) g(Y i ) f (Y i ) } f a (a ⊤ Y i ) g a (a ⊤ Y i ) -ϕ ′ ( g(x) f (x) f b (b ⊤ x) g b (b ⊤ x) ) g(x) f a (a ⊤ x) g a (a ⊤ x) dx| Moreover, since |ϕ ′ ( g(x) f (x) f b (b ⊤ x) g b (b ⊤ x) ) g(x) f a (a ⊤ x)
g a (a ⊤ x) |dx < ∞, as implied by lemma A.3, and since we assumed g such that Φ(g, f ) < ∞ and Φ( f, g) < ∞ and since b ∈ Θ Φ , the law of large numbers enables us to state that

| 1 n Σ n i=1 ϕ ′ { f a (a ⊤ Y i ) g a (a ⊤ Y i ) g(Y i ) f (Y i ) } f a (a ⊤ Y i ) g a (a ⊤ Y i ) -ϕ ′ ( g(x) f (x) f b (b ⊤ x) g b (b ⊤ x) ) g(x) f a (a ⊤ x) g a (a ⊤ x) dx| → 0 a.s. Furthermore, | 1 n Σ n i=1 ϕ ′ { f a,n (a ⊤ Y i ) g a,n (a ⊤ Y i ) g n (Y i ) f n (Y i ) } f a,n (a ⊤ Y i ) g a,n (a ⊤ Y i ) -1 n Σ n i=1 ϕ ′ { f a (a ⊤ Y i ) g a (a ⊤ Y i ) g(Y i ) f (Y i ) } f a (a ⊤ Y i ) g a (a ⊤ Y i ) | ≤ 1 n Σ n i=1 |ϕ ′ { f a,n (a ⊤ Y i ) g a,n (a ⊤ Y i ) g n (Y i ) f n (Y i ) } f a,n (a ⊤ Y i ) g a,n (a ⊤ Y i ) -ϕ ′ { f a (a ⊤ Y i ) g a (a ⊤ Y i ) g(Y i ) f (Y i ) } f a (a ⊤ Y i ) g a (a ⊤ Y i ) | and |ϕ ′ { f a,n (a ⊤ Y i ) g a,n (a ⊤ Y i ) g n (Y i ) f n (Y i ) } f a,n (a ⊤ Y i ) g a,n (a ⊤ Y i ) -ϕ ′ { f a (a ⊤ Y i ) g a (a ⊤ Y i ) g(Y i ) f (Y i ) } f a (a ⊤ Y i )
g a (a ⊤ Y i ) | → 0 as a result of the hypotheses intially introduced on θ n . Consequently,

1 n Σ n i=1 |ϕ ′ { f a,n (a ⊤ Y i ) g a,n (a ⊤ Y i ) g n (Y i ) f n (Y i ) } f a,n (a ⊤ Y i ) g a,n (a ⊤ Y i ) -ϕ ′ { f a (a ⊤ Y i ) g a (a ⊤ Y i ) g(Y i ) f (Y i ) } f a (a ⊤ Y i ) g a (a ⊤ Y i ) | → 0,
as it is a Cesàro mean. This enables us to conclude. Similarly, we obtain Proof of lemma F.4. Since Φ is greater than the L 1 distance, we have lemme F.4. For all c > 0, we have

sup b∈R d * | 1 n Σ n i=1 ϕ * {ϕ ′ { f a,n (a ⊤ X i ) g a,n (a ⊤ X i ) g n (X i ) f n (X i ) }} - ϕ * (ϕ ′ ( g(x) f (x) f b (b ⊤ x) g b (b ⊤ x) )) f (x)
Γ c ⊂ B L 1 ( f, c), where B L 1 ( f, c) = {p ∈ L 1 ; f -p 1 ≤ c}.
Proof of lemma F.5. The definition of the closure of a set and lemma A.4 (see page 19) imply lemme F.5. G is closed in L 1 for the topology of the uniform convergence.

Proof of lemma F.6. lemme F.6. inf a∈R d * Φ(g * , f ) is reached when the Φ-divergence is greater than the L 1 distance as well as the L 2 distance.

Proof. Indeed, let G be {g f a g a ; a ∈ R d * } and Γ c be Γ c = {p; K(p, f ) ≤ c} for all c>0. From lemmas F.3, F.4 and F.5 (see page 25), we get Γ c ∩G is a compact for the topology of the uniform convergence, if Γ c ∩ G is not empty. Hence, and since property A.2 (see page 18) implies that Q → Φ(Q, P) is lower semi-continuous in L 1 for the topology of the uniform convergence, then the infimum is reached in L 1 . (Taking for example c = Φ(g, f ), Ω is necessarily not empty because we always have Φ(g f a g a , f ) ≤ Φ(g, f )). Moreover, when the Φ-divergence is greater than the L 2 distance, the very definition of the L 2 space enables us to provide the same proof as for the L 1 distance.

Proof of lemma F.7. lemme F.7. For any p ≤ d, we have f (p-1) a p = f a p -see Huber's analytic method -, g (p-1) a p = g a psee Huber's synthetic method -and g (p-1) a p = g a p -see our algorithm.

Proof. As it is equivalent to prove either our algorithm or Huber's, we will only develop here the proof for our algorithm. Assuming, without any loss of generality, that the a i , i = 1, .., p, are the vectors of the canonical basis, since g (p-1) (x) = g(x) f 1 (x 1 )

g 1 (x 1 ) f 2 (x 2 ) g 2 (x 2 ) ... f p-1 (x p-1 )
g p-1 (x p-1 ) we derive immediately that g (p-1) p = g p . We note that it is sufficient to operate a change in basis on the a i to obtain the general case.

Proof of lemma F.8. lemme F.8. If there exits p, p ≤ d, such that Φ(g (p) , f ) = 0, then the family of (a i ) i=1,..,p -derived from the construction of g (p) -is free and orthogonal.

Proof. Without any loss of generality, let us assume that p = 2 and that the a i are the vectors of the canonical basis. Using a reductio ad absurdum with the hypotheses a 1 = (1, 0, ..., 0) and that a 2 = (α, 0, ..., 0), where α ∈ R, we get g (1) (x) = g(x 2 , .., x d /x 1 ) f 1 (x 1 ) and f = g (2) (x) = g(x 2 , .., x d /x 1 ) f 1 (x 1 )

f αa 1 (αx 1 ) [g (1)
] αa 1 (αx 1 ) . Hence f (x 2 , .., x d /x 1 ) = g(x 2 , .., x d /x 1 )

f αa 1 (αx 1 ) [g (1)
] αa 1 (αx 1 ) . It consequently implies that f αa 1 (αx 1 ) = [g (1) ] αa 1 (αx 1 ) since 1 = f (x 2 , .., x d /x 1 )dx 2 ...dx d = g(x 2 , .., x d /x 1 )dx 2 ...dx d f αa 1 (αx 1 ) [g (1) ] αa 1 (αx 1 ) = f αa 1 (αx 1 ) [g (1) ] αa 1 (αx 1 ) . Therefore, g (2) = g (1) , i.e. p = 1 which leads to a contradiction. Hence, the family is free. Moreover, using a reductio ad absurdum we get the orthogonality. Indeed, we have f (x)dx = 1 +∞ = n(a ⊤ j+1 x, ..., a ⊤ d x)h(a ⊤ 1 x, ..., a ⊤ j x)dx. The use of the same argument as in the proof of lemma F.2, enables us to infer the orthogonality of (a i ) i=1,..,p .

Proof of lemma F.9. lemme F.9. If there exits p, p ≤ d, such that Φ(g (p) , f ) = 0, where g (p) is built from the free and orthogonal family a 1 ,...,a j , then, there exists a free and orthogonal family

(b k ) k= j+1,...,d of vectors of R d * , such that g (p) (x) = g(b ⊤ j+1 x, ..., b ⊤ d x/a ⊤ 1 x, ..., a ⊤ j x) f a 1 (a ⊤ 1 x)... f a j (a ⊤ j x) and such that R d = Vect{a i } ⊥ ⊕ Vect{b k }.
Proof. Through the incomplete basis theorem and similarly as in lemma F.8, we obtain the result thanks to the Fubini's theorem.

Proof of lemma F.10. lemme F.10. For any continuous density f , we have y

m = | f m (x) -f (x)| = O P (m -2 4+d ).
where

Ψ (∞) n,k is a density such that Ψ (∞) n,k = lim k g (k)
n . Finally, through the dominated convergence theorem and taking the limit as n in (***) we get 0

= K(g (∞) , f ) = lim n K(g (∞) n , f n ) ≥ lim n K(Ψ (∞) n,k , f n ) ≥ 0. The dominated convergence theorem enables us to conclude: 0 = lim n K(Ψ (∞) n,k , f n ) = lim n lim k K(g (k) n , f n ).
Proof of lemma F.12. lemme F.12. With the notation of the proof of lemma F.11, we have 0

≤ .. ≤ K(Ψ (∞) n,k , f n ) ≤ .. ≤ K(g (∞) n , f n ) ≤ .. ≤ K(Ψ (∞) n,k-1 , f n ) ≤ .. ≤ K(g n , f n ), (***)
Proof. First, as explained in section D, we have

K( f (k) , g) -K( f (k+1) , g) = K( f (k) a k+1 , g a k+1 )
. Moreover, through remark D.1 page 22, we also derive that K( f (k) , g) = K(g (k) , f ). Then, K( f (k) a k+1 , g a k+1 ) is the decreasing step of the relative entropies in (*) and leading to 0 = K(g (∞) , f ). Similarly, the very construction of (**), implies that K( f (k) a k+1 ,n , g a k+1 ,n ) is the decreasing step of the relative entropies in (**) and leading to K(g (∞) n , f n ). Second, through the conclusion of the section D and lemma 14.2 of Huber's article, we obtain that K( f (k) a k+1 ,n , g a k+1 ,n ) converges -in decreasing and in k -towards a positive function of n -that we will call ξ n . Third, the convergence of (g (k) ) k -see proposition C.2 -implies that, for any given n, the sequence (K(g (k) n , f n )) k is not finite. Then, through relationship ( * * ), there exists a k such that 0 -1) ] a j (a ⊤ j x) }. Since f and g are bounded, it is easy to prove that from a certain rank, we get, for any

< K(g (k-1) n , f n ) -K(g (∞) n , f n ) < ξ n . Thus, since Q → K(Q, P) is l.s.
j = { f ǎ j ( ǎ j ⊤ x) [ǧ ( j-1) ] ǎ j ( ǎ j ⊤ x) - f a j (a ⊤ j x) [g ( j
x given in R d |Ψ j | ≤ max( 1 [ǧ ( j-1) ] ǎ j ( ǎ j ⊤ x) , 1 [g ( j-1) ] a j (a ⊤ j x) )| f ǎ j ( ǎ j ⊤ x) -f a j (a ⊤ j x)|.
Remark F.2. First, based on what we stated earlier, for any given x and from a certain rank, there is a constant R>0 independent from n, such that max(

1 [ǧ ( j-1) ] ǎ j ( ǎ j ⊤ x) , 1 [g ( j-1) ] a j (a ⊤ j x) ) ≤ R = R(x) = O(1). Second, since ǎk is an M-estimator of a k , its convergence rate is O P (n -1/2 ).
Thus using simple functions, we infer an upper and lower bound for f ǎ j and for f a j . We therefore reach the following conclusion:

|Ψ j | ≤ O P (n -1/2 ). (F.3)
We finally obtain

|Π k j=1 f ǎ j ( ǎ j ⊤ x) [ǧ ( j-1) ] ǎ j ( ǎ j ⊤ x) -Π k j=1 f a j (a ⊤ j x) [g ( j-1) ] a j (a ⊤ j x) | = Π k j=1 f a j (a ⊤ j x) [g ( j-1) ] a j (a ⊤ j x) |Π k j=1 f ǎ j ( ǎ j ⊤ x) [ǧ ( j-1) ] ǎ j ( ǎ j ⊤ x) [g ( j-1) ] a j (a ⊤ j x) f a j (a ⊤ j x) -1|.
Based on relationship (F.3), the expression

f ǎ j ( ǎ j ⊤ x) [ǧ ( j-1) ] ǎ j ( ǎ j ⊤ x) [g ( j-1) ] a j (a ⊤ j x) f a j (a ⊤ j x)
tends towards 1 at a rate of O P (n -1/2 ) for all j. Consequently,

Π k j=1 f ǎ j ( ǎ j ⊤ x) [ǧ ( j-1) ] ǎ j ( ǎ j ⊤ x) [g ( j-1) ] a j (a ⊤ j x) f a j (a ⊤ j x)
tends towards 1 at a rate of O P (n -1/2 ). Thus from a certain rank, we get (k) (x)C(x)n -1/2 → 0 a.s., by theorem C.1. This implies that sup x∈R d g (k) (x)C(x) < ∞ a.s., i.e. sup x∈R d C(x) < ∞ a.s. since g (k) has been assumed to be positive and bounded -see remark F.1. Thus, g (k) (x)C(x)dx ≤ sup C. g (k) (x)dx = sup C < ∞ since g (k) is a density, therefore we can conclude | ǧ(k) (x)g (k) (x)|dx ≤ sup C.n -1/2 = O P (n -1/2 ).

|Π k j=1 f ǎ j ( ǎ j ⊤ x) [ǧ ( j-1) ] ǎ j ( ǎ j ⊤ x) -Π k j=1 f a j (a ⊤ j x) [g ( j-1) ] a j (a ⊤ j x) | = O P (n -1/2 )O P (1) = O P (n -1/2 ). In conclusion, we obtain | ǧ(k) (x) -g (k) (x)| = g(x)|Π k j=1 f ǎ j ( ǎ j ⊤ x) [ǧ ( j-1) ] ǎ j ( ǎ j ⊤ x) -Π k j=1 f a j (a ⊤ j x) [g ( j-1) ] a j (a ⊤ j x) | ≤ O P (n -1/2 ). relationship (C.2). The relationship C.1 of theorem C.2 implies that | ǧ(k) (x) g (k) (x) -1| = O P (n -1/2 ) because, for any given x, g (k) (x)| ǧ(k) (x) g (k) (x) -1| = | ǧ(k) (x) -g (k) (x)|. Consequently, there exists a smooth function C of R d in R + such that lim n→∞ n -1/2 C(x) = 0 and | ǧ(k) (x) g (k) (x) -1| ≤ n -1/2 C(x), for any x. We then have | ǧ(k) (x) -g (k) (x)|dx = g (k) (x)| ǧ(k) (x) g (k) (x) -1|dx ≤ g (k) (x)C(x)n -1/2 dx. Moreover, sup x∈R d | ǧ(k) (x) -g (k) (x)| = sup x∈R d g (k) (x)| ǧ(k) (x) g (k) (x) -1| = sup x∈R d g
2 relationship (C.3). We have k) |dx with the line before last being derived from theorem A.1 page 18 and where ϕ : x → xln(x)-x+1 is a convex function and where S > 0. We get the same expression as the one found in our Proof of Relationship (C.2) section, we then obtain K( ǧ(k) , f ) -K(g (k) , f ) ≤ O P (n -1/2 ). Similarly, we get K(g (k) , f ) -K( ǧ(k) , f ) ≤ O P (n -1/2 ). We can therefore conclude. Proof. Let N be the random variable such that N = Σ m j=1 1 { f m (X j )≥θ m , g(Y j )≥θ m } . Since the events { f m (X j ) ≥ θ m } and {g(Y j ) ≥ θ m } are independent from one another and since {g(Y j ) ≥ θ m } ⊂ {g m (Y j ) ≥ -y m + θ m }, we can say that n = m.P( f m (X j ) ≥ θ m , g(Y j ) ≥ θ m ) ≤ m.P( f m (X j ) ≥ θ m ).P(g m (Y j ) ≥ -y m + θ m ). Consequently, let us study P( f m (X i ) ≥ θ m ). Let (ξ i ) i=1...m be the sequence such that, for any i and any x in R d , ξ i (x) = Π d l=1 1 (2π) 1/2 h l e -1 2 (

K( ǧ(k) , f ) -K(g (k) , f ) = f (ϕ( ǧ(k) f ) -ϕ( g (k) f ))dx ≤ f S | ǧ(k) f -g (k) f |dx = S | ǧ(k) -g (
x l -X il h l

) 2

-Π d l=1 1 (2π) 1/2 h l e -1 2 (

x l -X il h l

) 2 f (x)dx. Hence, for any given j and conditionally to X 1 , . . . , X j-1 , X j+1 , . . . , X m , the variables (ξ i (X j )) i j i=1...m are i.i.d. and centered, have same second moment, and are such that

|ξ i (X j )| ≤ Π d l=1 1 (2π) 1/2 h l + Π d l=1 1 (2π) 1/2 h l | f (x)|dx = 2.(2π) -d/2 Π d l=1 h -1 l since sup x e -1 2 x 2 ≤ 1. Moreover, noting that f m (x) = 1 m Σ m i=1 ξ i (x) + (2π) -d/2 1 m Σ m i=1 Π d l=1 h -1 l e -1 2 ( x l -X il h l ) 2 f (x)dx, we have f m (X j ) ≥ θ m ⇔ 1 m Σ m i=1 ξ i (X j ) + (2π) -d/2 1 m Σ m i=1 Π d l=1 h -1 l e -1 2 ( x l -X il h l ) 2 f (x)dx ≥ θ m ⇔ 1 m-1 Σ m i=1 i j ξ i (X j ) ≥ (θ m -(2π) -d/2 1 m Σ m i=1 Π d l=1 h -1 l e -1 2 ( x l -X il h l ) 2 f (x)dx -1 m ξ j (X j )) m m-1
with ξ j (X j ) = 0. Then, defining t (resp. ε) as t = 2.(2π

) -d/2 Π d l=1 h -1 l (resp. ε = (θ m -(2π) -d/2 Π d l=1 h -1 l 1 m Σ m i=1 Π d l=1 e
-1 2 (

x l -X il h l

) 2 f (x)dx) m m-1 ), the Bennet's inequality -Devroye (1985) page 160-implies that P( 1 m-1 Σ m i=1 i j ξ i (X j ) ≥ ε/X 1 , . . . , X j-1 , X j+1 , . . . , X m ) ≤ 2.exp(-(m-1)ε 2 4t 2 ).

Finally, since the X i are i.i.d. and since ( Π d l=1 e

-1 2 (

x l -y l h l

) 2 f (x)dx) f (y)dy < 1, then the law of large numbers implies that 1 m Σ m i=1 Π d l=1 e

-1 2 (

x l -X il h l

) 2 f (x)dx → m Π d l=1 e
-1 2 (

x l -y l h l

) 2 f (x) f (y)dxdy a.s. Consequently, since 0 < ν < 1 4+d -see remark B.1 -and since e -x ≤ x -1 2 when x > 0, we obtain, after calculation, that, from a certain rank, exp(-(m-1)ε 2 4t 2 ) = O(m -1 4 ), i.e., from a certain rank, P( f m (Y j ) ≥ θ m ) = O(m -1 4 ). Similarly, we infer P(g(Y j ) ≥ θ m ) = O(m -1 4 ). In conclusion, we can say that n = m.P( f m (X j ) ≥ θ m ).P(g m (Y j ) ≥ θ m ) = O(m 1 2 ). Similarly, we derive the same result as above for any step of our method.

Proof of theorem 3.2. First, from lemma F.10, we derive that, for any x,

sup a∈R d * | f a,n (a ⊤ x)-f a (a ⊤ x)| = O P (n -2 4+d
). Then, let us consider Ψ j = f ǎ j ,n ( ǎ j ⊤ x) ǧ( j-1) ǎ j ,n ( ǎ j ⊤ x) -f a j (a ⊤ j x) g ( j-1) a j (a ⊤ j x)

, we have

Ψ j = 1 ǧ( j-1)
ǎ j ,n ( ǎ j ⊤ x)g ( j-1) a j (a ⊤ j x)

(( f ǎ j ,n ( ǎ j ⊤ x)f a j (a ⊤ j x))g

( j-1) a j (a ⊤ j x) + f a j (a ⊤ j x)(g

( j-1)
a j (a ⊤ j x)ǧ( j-1) ǎ j ,n ( ǎ j ⊤ x))),

i.e. |Ψ j | = O P (n -1 2 1 d=1 -2 4+d 1 d>1 ) since f a j (a ⊤ j x) = O(1) and g ( j-1) a j (a ⊤ j x) = O(1). We can therefore conclude similarly as in theorem C.2.

2 Proof of theorem 3.3. We get ththeorem through theorem C.3 and proposition B.1.

2 Proof of theorem C.3. First of all, let us remark that hypotheses (H1) to (H3) imply that γn and čn (a k ) converge towards a k in probability. Hypothesis (H4) enables us to derive under the integrable sign after calculation, P ∂ ∂b M(a k , a k ) = P ∂ ∂a M(a k , a k ) = 0, P ∂ 2 ∂a i ∂b j M(a k , a k ) = P ∂ 2 ∂b j ∂a i M(a k , a k ) = ϕ"( , f ) = P ∂ 2 ∂a i ∂a j M(a k , a k ) -P ∂ 2 ∂b i ∂b j M(a k , a k ), = P ∂ 2 ∂a i ∂a j M(a k , a k ) + P ∂ 2 ∂a i ∂b j M(a k , a k ), = P ∂ 2 ∂a i ∂a j M(a k , a k ) + P ∂ 2 ∂b j ∂a i M(a k , a k ).

g f a k f g a k ) ∂ ∂a i g f a k f g a k ∂ ∂b j g f a k f g a k f dx, P ∂ 2 ∂a i ∂a j M(a k , a k ) = ϕ ′ ( g f a k f g a k ) ∂
The very definition of the estimators γn and čn (a k ), implies that

Figure 1 :Figure 2 :

 12 Figure 1: Graph of the distribution to estimate (red) and of our own estimate (green).

Figure 4 :

 4 Figure4: Graph of the estimate of (x 0 , x 1 ) → c ρ (F Gumbel (x 0 ), F Exponential (x 1 )).

Theorem A. 1

 1 (theorem III.4 of Azé (1997)). Let f : I → R be a convex function. Then f is a Lipschitz function in all compact intervals [a, b] ⊂ int{I}. In particular, f is continuous on int{I}.A.2. Useful lemmasThrough a reductio ad absurdum argument, we derive lemmas A.1 and A.2 : lemme A.1. Let f be a density in R d bounded and positive. Then, any projection density of fthat we will name f a , with a ∈ R d * -is also bounded and positive in R.lemme A.2. Let f be a density in R d bounded and positive. Then any density f (./a ⊤ x), for any a ∈ R d * , is also bounded and positive.

m

  for all b in R d * and all x in R d , and finally y (2)

Theorem C. 4 .

 4 Assuming that (H1) to (H3), (H6) and (H8) hold. Then, √ n(Var P (M(č n ( γn ), γn ))) -1/2 (P n M(č n ( γn ), γn ) -P n M(a k , a k )) Law → N(0, I), where k represents the k th step of the algorithm and with I being the identity matrix in R d . Note that k is fixed in theorem C.4 since γn = arg inf a∈Θ sup c∈Θ P n M(c, a) where M is a known function of k -see section 3.1.1. Thus, in the case where Φ(g (k-1) f a k g (k-1) a k , f ) = 0, we obtain Corollary C.1. Assuming that (H1) to (H3), (H6), (H7) and (H8) hold. Then, √ n(Var P (M(č n ( γn ), γn ))) -1/2 (P n M(č n ( γn ), γn )) Law → N(0, I).

  dx| → 0 a.s. 2 Proof of lemma F.3. By definition of the closure of a set, we have lemme F.3. The set Γ c is closed in L 1 for the topology of the uniform convergence.

  c. -see property A.2 page 18 -relationship (**) implies (***).Proof of theorem 3.1. First, by the very definition of the kernel estimator ǧ(0) n = g n converges towards g. Moreover, the continuity of a → f a,n and a → g a,n and proposition 3.3 imply that ǧLet us consider Ψ

  We keep the notations introduced in Appendix B. It holds n = O

  č n (a k ), γn ) = 0 P n ∂ ∂a M(č n (a k ), γn ) + P n ∂ ∂b M(č n (a k ), γn ) ∂ ∂a čn (a k ) = 0, i.e. P n ∂ ∂b M(č n (a k ), γn ) = 0 (E0) P n ∂ ∂a M(č n (a k ), γn ) = 0 (E1) .Under (H5) and (H6), and using a Taylor development of the (E0) (resp. (E1)) equation, we infer there exists (c n , γ n ) (resp. (c n , γn )) on the interval [(č n (a k ), γn ), (a k , a k )] such that-P n ∂ ∂b M(a k , a k ) = [(P ∂ 2 ∂b∂b M(a k , a k )) ⊤ + o P (1), (P ∂ 2 ∂a∂b M(a k , a k )) ⊤ + o P (1)]a n . (resp. -P n ∂ ∂a M(a k , a k ) = [(P ∂ 2 ∂b∂a M(a k , a k )) ⊤ + o P (1), (P ∂ 2 ∂a 2 M(a k , a k )) ⊤ + o P (1)]a n ) with a n = ((č n (a k )a k ) ⊤ , ( γna k ) ⊤

  .1) did indeed exist, with j < d, with (a 1 , . . . , a d ) being a basis of R d and with h being a density on R j . Secondly, our next step consisted in building an estimate (resp. a representation) of f without necessarily assuming that f meets relationship (3.1) -see theorem 3.4. Consequently, let us consider Z 1 and Z 2 , two random vectors with respective densities h 1 and h 2

  2533/7.0710678 = 0.03582203}.

	And, we obtain
	Our Algorithm
		minimum : 0.0201741
	Projection Study 0 :	at point : (1.00912,1.09453,0.01893)
		P-Value : 0.81131
	Test :

  y), i.e. Ψ (resp. y) is the expression of f (resp of x) in basis A, namely Ψ(y) = ñ(y j+1 , ..., y d ) h(y 1 , ..., y j ), with ñ and h being the expressions of n and h in basis A. Consequently, our results in the case where the family {a j } 1≤ j≤d is the canonical basis of R d , still hold for Ψ in basis A -see section 2.1.2. And then, if g is the expression of g in basis A, we have g(./y 1 , ..., y j ) = ñ(y j+1 , ..., y d ) = Ψ(./y 1 , ..., y j ),

	i.e. g(./a ⊤ 1 x, ..., a ⊤ j x) = n(a ⊤ j+1 x, ..., a ⊤ d x) = f (./a ⊤ 1 x, ..., a ⊤ j x). Proof of lemma F.2. lemme F.2. Should there exist a family (a i ) i=1...d such that f (x) = n(a ⊤ j+1 x, ..., a ⊤ d x)h(a ⊤ 1 x, ..., a ⊤ j x), 2 with j < d, with f , n and h being densities, then this family is a orthogonal basis of R d . Using a reductio ad absurdum, we have f (x)dx = 1 +∞ = n(a ⊤ j+1 x, ..., a ⊤ d x)h(a ⊤ 1 x, ..., a ⊤ j x)dx. We can therefore conclude. 2 Proof of proposition B.1.

  ∂ 2 ∂b i ∂b j M(a k , a k ) = -P ∂ 2 ∂a i ∂b j M(a k , a k ) = -P ∂ 2 ∂b j ∂a i M(a k , a k ), which implies,

	2 ∂a i ∂a j ∂b g f a k P ∂ 2 f g a k ) ∂ ∂b i	g f a k f g a k g f a k f g a k	f dx, ∂ ∂b j g f a k f g a k	f dx,
	and consequently P ∂ 2 f a k ∂a i ∂a j K(g g a k

i ∂b j M(a k , a k ) =ϕ"(

  ). Thus we get M(a k , a k )P ∂ 2 ∂a∂b M(a k , a k ) P ∂ 2 ∂b∂a M(a k , a k ) P ∂ 2 ∂a 2 M(a k , a k )

	=	√ na n = √ n(P ∂ 2 ∂b∂b M(a k , a k ) ∂ 2 √ n       P ∂ 2 ∂b 2       -1 ∂a∂a K(g f a k g a k , f )) -1	-P n -P n	∂ ∂b M(a k , a k ) ∂ ∂a M(a k , a k )	+ o P (1)
	.	      	P ∂ 2 ∂b∂b M(a k , a k ) + ∂ 2 ∂a∂a K(g P ∂ 2 ∂b∂b M(a k , a k )	f a k g a k	, f ) P ∂ 2 ∂b∂b M(a k , a k ) P ∂ 2 ∂b∂b M(a k , a k )	       .	-P n -P n	∂ ∂b M(a k , a k ) ∂ ∂a M(a k , a k )	+ o P (1)

x) > f c (c ⊤ x) g c (c ⊤ x) , g(x) f c (c ⊤ x) g c (c ⊤ x) > f (x)} = ∅

Defining b m (x) as b m (x) = |E( f m (x))f (x)|, we have y m ≤ | f m (x) -E( f m (x))| + b m (x). Moreover, from page 150 of [START_REF] Scott | Multivariate density estimation. Theory, practice, and visualization[END_REF], we derive that b m (x) = O P (Σ d j=1 h 2 j ) where h j = O P (m -1 4+d ). Then, we obtain b m (x) = O P (m -2 4+d ). Finally, since the central limit theorem rate is O P (m -1 2 ), we infer that y m ≤ O P (m -1 2 ) + O P (m -2 4+d ) = O P (m -2 4+d ). 2 Proof of proposition 3.3. Proposition 3.3 comes immediately from proposition B.1 page 20 and lemma C.1 page 20.

2 Proof of proposition 3.4. Let us first show by induction the following assertion P(k) = {g (k) allows a deconvolution g (k) = g (k) * φ} Initialisation : For k = 0, we get the result since g = g (0) is elliptic. Going from k to k + 1 : Let us assume P(k) is true, we then show that P(k + 1). Since the family of a i , i ≤ k + 1 is free -see lemma F.8 -then, we define B as the basis of R d such that its k + 1 first vectors are the a i , i ≤ k + 1 -see the incomplete basis theorem for its existence. Thus, in B and using the same procedure to prove lemma F.1 page 24, we have g (k) (x) = g (k) (./x k+1 )g (k) k+1 (x k+1 ). Consequently, the very definition of the convolution product, the Fubini's theorem and the hypothesis made on the Elliptical family imply that g (k)

) and with g (k) k+1 (x k+1 ) = g (k) k+1 (x k+1 ) * E 1 (0, σ 2 , ξ 1 ). Finally, replacing g (k) k+1 with f k+1 = f k+1 * E 1 (0, σ 2 , ξ 1 ), we conclude this induction with g (k+1) = g (k) (./x k+1 ) f k+1 (x k+1 ). Now, let us consider ψ (rep. ψ, ψ (k) , ψ (k) ) the characteristic function of f (resp. f , g (k) , g (k) ). We

Hence, ψ and ψ (k) are less or equal to Ψ( 1 2 σ 2 |s| 2 ) which is integrable by hypothesis, i.e. ψ and ψ (k) are absolutely integrable. We then obtain g (k)

Moreover, since the sequence (ψ (k) ) uniformly converges and since ψ and ψ (k) are less or equal to Ψ( 1 2 σ 2 |s| 2 ), then the dominated convergence theorem implies that

)ds < ∞, then the above limit and the dominated convergence theorem imply that lim k | f (x)g (k) (x)|dx = 0. 2 Proof of corollary 3.1. Through the dominated convergence theorem and through theorem 3.4, we get the result using a reductio ad absurdum.

2 Proof of lemma F.11. lemme F.11. Let consider the sequence (a i ) defined in (2.3) page 6. We then have

Proof. Trough the relationship (2.3) and through remark D.1 page 22 as well as the additive relation of proposition D.1, we can say that 0

where g (∞) = lim k g (k) which is a density by construction. And through proposition C.2, we obtain that K(g (∞) ,

n ) k be the sequence of densities such that g (k) n is the kernel estimate of g (k) . Since we derive from remark F.1 page 23 an integrable upper bound of g (k) n , for all k, which is greater than f -see also the definition of ϕ in the proof of theorem 3.4 -, then the dominated convergence theorem implies that, for any k, lim n K(g (k) n , f n ) = K(g (k) , f ), i.e., from a certain given rank n 0 , we have 0

*). Consequently, through lemma F.12 page 28, there exists a k such that 0

Moreover, the central limit theorem implies:

which leads us to the result.

2 Proof of proposition C.2. Let us consider ψ (resp. ψ (k) ) the characteristic function of f (resp. g (k-1) ). Let also consider the sequence (a i ) defined in (2.3) page 6. We have |ψ(t) - (k) , f ). As explained in section 14 of Huber's article and through remark D.1 page 22 as well as through the additive relation of proposition D.1, we can say that lim k K(g

Proof of theorem 3.4. We recall that g (k) n is the kernel estimator of ǧ(k) . Since the relative entropy is greater than the L 1 -distance, we then have

n (x)]lim n f n (x)|dx = 0. Moreover, for any given k and any given n, the function g (k) n is a convex combination of multivariate Gaussian distributions. As derived at remark 2.1 of page 5, for all k, the determinant of the covariance of the random vector -with density g (k) -is greater than or equal to the product of a positive constant times the determinant of the covariance of the random vector with density f . The form of the kernel estimate therefore implies that there exists an integrable function ϕ such that, for any given k and any given n, we have |g (k) n | ≤ ϕ. Finally, the dominated convergence theorem enables us to say that lim n lim k g (k) n = lim n f n = f , since f n converges towards f and since |[lim n lim k g (k) n (x)]lim n f n (x)|dx = 0. 2 Proof of theorem C.4. Through a Taylor development of P n M(č n (a k ), γn ) of rank 2, we get at point (a k , a k ):

The lemma below enables us to conclude.

lemme F.14. Let H be an integrable function and let C = H dP and C

Thus we get

2 Proof of theorem 3.5. Through proposition B.1 and theorem C.4, we derive theorem 3.5.. 2 Proof of theorem 3.7. We immediately get the proof from theorem 3.4.

2 Proof of theorem 3.8. Since Φ(g (1) , f ) = 0, then, through lemma F.9, we deduct that the density of b ⊤ 2 X/a ⊤ 1 X, with a 1 = (0, 1) ′ and b 2 = (1, 0) ′ , is the same as the one of b ⊤ 2 Y/a ⊤ 1 Y. Hence, we derive that E(X 1 /X 2 ) = E(Y 1 /Y 2 ) and also that the regression between X 1 and X 2 is

where ε is a centered random variable such that it is 31 orthogonal to E(X 1 /X 2 ).

2 Proof of theorem 3.9. We infer this proof similarly to the proof of theorem 3.8 section.

2 Proof of corollary 3.4. Assuming first that the b k and the a i are the canonical basis of R d . Then, for any i j, Y i is independent from Y j , i.e. E(Y k /Y 1 , ..., Y j ) = E(Y k ). Consequently, the regression between X k and (X 1 , ..., X j ) is given by X k = E(Y k ) + ε k where ε is a centered random variable such that it is orthogonal to E(X k /X 1 , ..., X j ). At present, we derive the general case thanks to the methodology used in the proof of lemma F.1 section with the transformation matrix B = (a 1 , ..., a j , b j+1 , ..., b d ).

2