Julien Reygner
email: julien.reygner@enpc.fr

Adrien Touboul
email: adrien.touboul@enpc.fr

Reweighting samples under covariate shift using a Wasserstein distance criterion

Keywords: Reweighting, Covariate shift, Wasserstein distance, Uncertainty Quantification, Nearest neighbor regression, Nearest neighbor distance

Considering two random variables with different laws to which we only have access through finite size i.i.d samples, we address how to reweight the first sample so that its empirical distribution converges towards the true law of the second sample as the size of both samples goes to infinity. We study an optimal reweighting that minimizes the Wasserstein distance between the empirical measures of the two samples, and leads to an expression of the weights in terms of Nearest Neighbors. The consistency and some asymptotic convergence rates in terms of expected Wasserstein distance are derived, and do not need the assumption of absolute continuity of one random variable with respect to the other. These results have some application in Uncertainty Quantification for decoupled estimation and in the bound of the generalization error for the Nearest Neighbor regression under covariate shift.

1. Introduction 1.1. Regression under covariate shift. This article is dedicated to the study of a method aimed at approximating the law of a random variable

(1) Y = f (X, Θ),
where X ∈ R d , Θ ∈ Θ are independent random variables, with respective laws denoted by µ X and µ Θ , and f : R d × Θ → R e is a measurable function. The space Θ is only assumed to be measurable. The specificity of the problem at stake is that we assume to be provided with:

• a training sample (X ′ j , Y ′ j ) j∈ 1,m of i.i.d observations Y ′ j = f (X ′ j , Θ j ) where Θ j has law µ Θ and is independent from X ′ j , but the law µ X ′ of X ′ j may differ from µ X ; • an evaluation sample (X i ) i∈ 1,n with i.i.d observations distributed according to µ X . This situation is known as covariate shift in the statistical learning literature [START_REF] Braun | Development and application of the collaborative optimization architecture in a multidisciplinary design environment[END_REF][START_REF] Quionero-Candela | Dataset shift in machine learning[END_REF]. This problem is motivated by the study of decomposition-based uncertainty quantification (UQ) methods in complex industrial systems, as is detailed in Subsection 5.2 below. In this context, the overall objective is to approximate a quantity of interest of the form

(2) QI = E[φ(Y )]
for some function φ : R e → R. Following previous works in this direction [START_REF] Amaral | A decomposition approach to uncertainty analysis of multidisciplinary systems[END_REF][START_REF] Amaral | A decomposition-based approach to uncertainty analysis of feed-forward multicomponent systems[END_REF][START_REF] Amaral | Optimal L2-norm empirical importance weights for the change of probability measure[END_REF], our estimator of QI assumes the form

(3) QI m,n = 1 m m j=1 w j φ(Y ′ j ),
where the vector of weights w m = (w 1 , . . . , w m ) is chosen so that the weighted empirical measure

µ wm X ′ m := 1 m m j=1
w j δ X ′ j of the training sample X ′ m = (X ′ 1 , . . . , X ′ m ) be close, in a sense which will be made precise below, to the empirical measure

µ Xn := 1 n n i=1 δ X i
of the evaluation sample X n = (X 1 , . . . , X n ). Such a reweighting procedure is a standard approach to the problem of density ratio estimation in the statistical learning literature [START_REF] Sugiyama | Density ratio estimation in machine learning[END_REF], the purpose of which is to estimate the density dµ X /dµ X ′ from the samples X n and X ′ m , without estimating separately the measures µ X and µ X ′ . While the theoretical analysis of such methods almost always requires this density to exist, in the UQ context which motivates the present study it is desirable not to assume that any of the measures µ X and µ X ′ be absolutely continuous with respect to the other, see in particular Remark 5.3.

The first step of our work is thus the computation of optimal weights w m for the problem

min wm W q µ Xn , µ wm X ′ m , (4)
∀j ∈ 1, m , w j ≥ 0, and

m j=1 w j = m, (5)
where W q denotes the Wasserstein distance of order q ≥ 1 on R d . The reason for the choice of this distance is that unlike criteria already studied in the density ratio estimation literature, such as moment/kernel matching, L 2 distance, Kullback-Leibler divergence (see [START_REF] Sugiyama | Density ratio estimation in machine learning[END_REF] and the references therein), it is not sensitive to absolute continuity conditions and therefore it is well suited to our UQ motivation. On the other hand, this choice makes the problem closely related to the fields of optimal quantization [START_REF] Graf | Foundations of quantization for probability distributions[END_REF] and Nearest Neighbor (NN) estimation [START_REF] Biau | Lectures on the nearest neighbor method[END_REF].

More precisely, for any k ∈ 1, m , denote by ψ (k) m (x) the k-NN estimator of the regression function [START_REF] Billingsley | Convergence of probability measures[END_REF] ψ(x)

:= E [φ(Y )|X = x] = E [φ(f (x, Θ))] ,
defined from the observation of the training sample, and then consider the Monte Carlo estimator

QI (k) m,n = 1 n n i=1 ψ (k) m (X i )
of QI. Then the vector of weights w m which are optimal for (4)- [START_REF] Biau | Lectures on the nearest neighbor method[END_REF] turns out not to depend on the value of q, and the associated estimator QI m,n defined by (3) coincides with the 1-NN estimator QI

m,n . For this reason, we shall denote by w [START_REF] Allaire | Surrogate modeling for uncertainty assessment with application to aviation environmental system models[END_REF] m the vector of optimal weights for (4)- [START_REF] Biau | Lectures on the nearest neighbor method[END_REF], and more generally by w (k) m the vector of weights induced by the k-NN estimator of ψ. The main results of this paper describe the asymptotic behavior, as the respective sizes m and n of the training and evaluation samples grow to infinity, of both the Wasserstein distance W q ( µ Xn , µ w 1.2. Outline of the article. The derivation of the Wasserstein optimal weights w m is detailed in Section 2, where we also highlight connections between our results and various topics in numerical probability and statistical learning. The asymptotic behavior of W q ( µ Xn , µ w (k) m X ′ m ) and QI (k) m,n are respectively studied in Sections 3 and 4. Applications to decomposition-based UQ and the generalization error for NN regression under covariate shift, as well as numerical illustrations, are presented in Section 5.

1.3. Notation. We denote by N the set of the natural integers including zero and by N * = N \ {0} the set of positive integers. Given two integers n 1 ≤ n 2 , the set of the integers between n 1 and n 2 is written n 1 , n 2 = {n 1 , . . . , n 2 }. For x ∈ R, ⌈x⌉ (resp. ⌊x⌋) is the unique integer verifying x ≤ ⌈x⌉ < x + 1 (resp. x -1 < ⌊x⌋ ≤ x). For (x, y) ∈ R 2 , we use the join and meet notation x ∧ y = min(x, y) and

x ∨ y = max(x, y). Last, we denote by (x) + := 0 ∨ x and (x) -:= 0 ∨ (-x) the nonnegative and nonpositive parts of x ∈ R.

We fix a norm | • | on R d , which need not be the Euclidean norm. The supremum norm of φ :

R d → R is denoted by φ ∞ = sup x∈R d |φ(x)|. The distance between a point x ∈ R d and a subset A ⊂ R d is denoted by dist(x, A). Last, for all x ∈ R d and r ≥ 0, we denote B(x, r) := {x ′ ∈ R d : |x -x ′ | ≤ r},
and recall that the support of a probability measure ν ∈ P(R d ) is defined by

supp(ν) := x ∈ R d : ∀r > 0, ν(B(x, r)) > 0 .

Wasserstein distance minimization and NN regression

2.1. Optimal weights for Wasserstein distances. We begin by recalling the definition of the Wasserstein distance.

Definition 2.1 (Wasserstein distance). Let P(R d ) be the set of probability measures on R d and, for any

q ∈ [1, +∞), let P q (R d ) = ν ∈ P(R d ) : R d |x| q dν(x) < +∞ .
The Wasserstein distance of order q between µ and ν ∈ P q (R d ) is defined as

W q (µ, ν) = inf R d ×R d |x -x ′ | q dγ(x, x ′ ) : γ ∈ Π(µ, ν) 1/q
, where Π(µ, ν) is the set of probability measures on R d × R d with marginals µ and ν.

We refer to [START_REF] Villani | Optimal transport, volume 338 of Grundlehren der mathematischen Wissenschaften[END_REF]Section 6] for a general introduction to Wasserstein distances. This definition allows for an explicit resolution of the minimization problem (4)- [START_REF] Biau | Lectures on the nearest neighbor method[END_REF], which relies on the notion of Nearest Neighbor (NN). For x ∈ R d and k ∈ 1, m , we denote by NN with lowest index j. We omit the superscript notation (k) when referring to the 1-NN, i.e.

NN X ′ m (x) = NN (1) X ′ m (x).
In the next statement, for any i ∈ 1, n and l ∈ 1, m , we denote by j

(l) i the (lowest) index j such that X ′ j = NN (l) X ′ m (X i ).
Proposition 2.2 (Optimal vector of weights). Let the k-NN vector of weights w

(k) m = (w (k) 1 , . . . , w (k)
m ) be defined by, for all j, k ∈ 1, m , [START_REF] Boucheron | Moment inequalities for functions of independent random variables[END_REF] w

(k) j := m kn n i=1 k l=1 1 {j=j (l) i } .
The vector w (k) m satisfies (5) and verifies, for all q ∈ [1, +∞),

(8) W q q µ Xn , µ w (k) m X ′ m ≤ 1 kn n i=1 k l=1 X i -NN (l) X ′ m (X i ) q .
For k = 1, the equality is reached

(9) W q q µ Xn , µ w (1) m X ′ m = 1 n n i=1 X i -NN X ′ m (X i ) q ,
and the vector is optimal for (4) in the sense that for any w m = (w 1 , . . . , w m ) which also satisfies (5),

we have

(10) W q µ Xn , µ w (1) m X ′ m ≤ W q µ Xn , µ wm X ′ m .
In other words, for a given j ∈ 1, m , w

j is proportional to the number of points X i of which X ′ j is one of the first k NN. We refer to [START_REF] Loog | Nearest neighbor-based importance weighting[END_REF] for a numerical illustration of the use of the vector of weights w

m in the context of classification under covariate shift.

Proof. For a general vector of weights w m = (w 1 , . . . , w m ) which satisfies (5), the Wasserstein distance

W q q ( µ Xn , µ wm X ′ m
) is the solution of the following optimal transport problem inf

(γ i,j ) (i,j)∈ 1,n × 1,m n i=1 m j=1 γ i,j |X i -X ′ j | q , ∀i ∈ 1, n , m j=1 γ i,j = 1 n (marginal condition on µ Xn ), ∀j ∈ 1, m , n i=1 γ i,j = w j m (marginal condition on µ wm X ′ m ), ∀(i, j) ∈ 1, n × 1, m , γ i,j ≥ 0, (11)
where γ i,j is the coefficient of the discrete transport plan between δ X i and δ X ′ j . For the k-NN vector of weights w (k) m defined by [START_REF] Boucheron | Moment inequalities for functions of independent random variables[END_REF], the transport plan

γ (k) i,j = 1 kn k l=1 1 {j=j (l) i }
satisfies the two marginal conditions. Reordering the terms in the associated cost gives the upper bound of Equation [START_REF] Braun | Development and application of the collaborative optimization architecture in a multidisciplinary design environment[END_REF].

We now prove the equality [START_REF] Brécheteau | A k-points-based distance for robust geometric inference[END_REF] and optimality [START_REF] Buttazzo | On the Wasserstein distance between mutually singular measures[END_REF] of w [START_REF] Allaire | Surrogate modeling for uncertainty assessment with application to aviation environmental system models[END_REF] m at the same time. On the one hand, it is clear that for any vector of weights w m = (w 1 , . . . , w m ) and any transport plan (γ i,j

) (i,j)∈ 1,n × 1,m between µ Xn and µ wn X ′ m , we have n i=1 m j=1 γ i,j |X i -X ′ j | q ≥ n i=1 m j=1 γ i,j |X i -NN X ′ m (X i )| q = 1 n n i=1 |X i -NN X ′ m (X i )| q ,
therefore taking the infimum over all transport plans yields

W q q µ Xn , µ wm X ′ m ≥ 1 n n i=1 |X i -NN X ′ m (X i )| q .
On the other hand, taking w m = w

m in the left-hand side and combining this inequality with (8) for k = 1, we obtain both the equality (9) and optimality [START_REF] Buttazzo | On the Wasserstein distance between mutually singular measures[END_REF].

Remark 2.3. In order to alleviate notation, we now write µ is the pushforward of µ Xn by NN X ′ m , and that this transport map yields an optimal coupling between µ Xn and µ

(k) X ′ m = µ w (k) m X ′ m . 2.
(1) X ′ m in Definition 2.1, for any q ≥ 1. The idea to associate each

X i with NN X ′ m (X i ) = X ′ j (1) i
is the basis of the theory of optimal quantization [START_REF] Graf | Foundations of quantization for probability distributions[END_REF][START_REF] Kloeckner | Approximation by finitely supported measures[END_REF][START_REF] Pagès | CEMRACS 2013-modelling and simulation of complex systems: stochastic and deterministic approaches[END_REF]. In this context, the sample X ′ m plays the role of the quantization grid, and NN X ′ m is known to be the optimal quantization function. The right-hand side of (9) then corresponds to the L q mean quantization error induced by the grid X ′ m for the measure µ Xn .

2.2.2. Link with geometric inference. When q = 2, the right-hand side of (8) rewrites

x∈R d d 2 µ X ′ m ,k/m (x) µ Xn (dx),
where d µ,α (•) is the distance function to µ with parameter α introduced by Chazal, Cohen-Steiner and Mérigot in [START_REF] Chazal | Geometric inference for probability measures[END_REF]Definition 3.2] in order to perform geometric and topological inference for set estimation, see also [START_REF] Chazal | Rates of convergence for robust geometric inference[END_REF][START_REF] Brécheteau | A k-points-based distance for robust geometric inference[END_REF] for robust inference. In particular, [START_REF] Chazal | Geometric inference for probability measures[END_REF]Proposition 3.3] shows that for any

x ∈ R d , (12) d 2 µ X ′ m ,k/m (x) = inf wm W 2 2   1 m m j=1 w j δ X ′ j , δ x   ,
w m = (w 1 , . . . , w m ) satisfies (5) and w j ≤ m/k for all j.

This result may be directly compared with the estimates (8) and ( 9), at least in the case where µ X = δ x .

In this case, if k = 1, then the supplementary constraint w j ≤ m/k is necessarily implied by [START_REF] Biau | Lectures on the nearest neighbor method[END_REF] and therefore, combining [START_REF] Chazal | Rates of convergence for robust geometric inference[END_REF] with (9), we recover the optimality result [START_REF] Buttazzo | On the Wasserstein distance between mutually singular measures[END_REF]. For arbitrary values of k, the combination of ( 12) with [START_REF] Braun | Development and application of the collaborative optimization architecture in a multidisciplinary design environment[END_REF] shows that the vector w (k) m need not be optimal for [START_REF] Buttazzo | On the Wasserstein distance between mutually singular measures[END_REF], but yields a solution which is lower than any solution with the supplementary constraint that w j ≤ m/k.

2.2.3.

A more general problem. Proposition 2.2 may appear as a specific instance, restricted to empirical measures, of the following problem: given two probability measures µ and ν on R d , and assuming that µ ∈ P q (R d ), compute the infimum of W q (µ, ρdν) over all probability densities ρ with respect to ν. Similar questions were recently addressed in [START_REF] Buttazzo | On the Wasserstein distance between mutually singular measures[END_REF]. First, it is clear that if µ is absolutely continuous with respect to ν, then taking ρ = dµ/dν shows that this minimum is 0. Next, following the proof of Proposition 2.2, it is easily seen that for any ρ,

W q q (µ, ρdν) ≥ E [dist(X, supp(ν)) q ] , X ∼ µ.
To show that the right-hand side actually matches with the infimum of the left-hand side when ρ varies, we keep following the proof of Proposition 2.2. Since supp(ν) is closed, for any x ∈ R d the set Ψ(x) :=

{x ′ ∈ supp(ν) : |x -x ′ | = dist(x, supp(ν))
} is nonempty and closed. Besides, the multifunction Ψ is weakly measurable 1 , therefore by the Kuratowski-Ryll-Nardzewski theorem it admits a measurable selection which we denote by nn ν . We denote by µ * the associated pushforward of µ by nn ν . Then it is clear that supp(µ * ) ⊂ supp(ν) on the one hand, and that

W q q (µ, µ * ) ≤ E [dist(X, supp(ν)) q ] , X ∼ µ,
on the other hand. We finally deduce from the approximation result stated in Lemma 2.4 below that

inf ρ W q q (µ, ρdν) = W q q (µ, µ * ) = E [dist(X, supp(ν)) q ] , X ∼ µ,
which thereby generalizes the results of Proposition 2.2. Notice that there may not exist a minimizer ρ for this problem as the measure µ * need not be absolutely continuous with respect to ν.

Lemma 2.4 (W q approximation by absolutely continuous measures). Let µ * and ν be two probability measures on R q such that supp(µ * ) ⊂ supp(ν). For any ǫ > 0, there exists a probability density ρ with respect to ν such that, for all q ≥ 1, W q (µ * , ρdν) < ǫ.

Proof. Let ǫ > 0 and X ∈ R d be a random variable with law µ * . Almost surely, X ∈ supp(µ * ) ⊂ supp(ν) and therefore ν(B(X, ǫ)) > 0, which allows to draw Y with conditional distribution dν(y|B(X, ǫ)) := 1 {y∈B(X,ǫ)} dν(y) ν(B(X, ǫ)) .

On the one hand, the random variable Y has density ρ(y) :=

x∈R d 1 {y∈B(x,ǫ)} dµ * (x) ν(B(x, ǫ))
with respect to ν, and on the other hand we have |X -Y | < ǫ, almost surely, which ensures that W q (µ * , ρdν) < ǫ for any q ≥ 1.

1 Let us denote S = supp(ν) and fix U ⊂ R d an open set, which we write as the countable union of closed sets (Fn) n≥1 .

Then {x ∈ R d : Ψ(x) ∩ U = ∅} = ∪ n≥1 {x ∈ R d : dist(x, (S ∩ Fn)) = dist(x, S)} and it is easily seen that each set in the right-hand side is measurable.

Analysis of the Wasserstein distance

W q ( µ Xn , µ (k) X ′ m )
In this section, we study the asymptotic behavior of E[W q q ( µ Xn , µ

(k) X ′ m )]
when n, m → +∞. To this aim, we first notice that by Proposition 2.2, we have [START_REF] Chen | Explaining success of nearest neighbor methods in prediction[END_REF] E W q q µ Xn , µ

X ′ m = E 1 n n i=1 X i -NN X ′ m (X i ) q = E X -NN X ′ m (X) q , for k = 1, and (1)
E W q q µ Xn , µ (k) X ′ m ≤ E 1 kn n i=1 k l=1 X i -NN (l) X ′ m (X i ) q = 1 k k l=1 E X -NN (l) X ′ m (X) q , for k ≥ 2. (14)
Observe that the right-hand sides of both ( 13) and ( 14) no longer depend on n.

3.1. Consistency. Our first main result is a consistency result. Before stating it in Theorem 3.3, we formulate two crucial assumptions.

Assumption 3.1 (Support condition). We have supp(µ X ) ⊂ supp(µ X ′ ).

Assumption 3.2 (Min-integrability).

There exists an integer m 0 ≥ 1 such that

E min j∈ 1,m 0 |X ′ j | < +∞.
Theorem 3.3 (Consistency). Let Assumptions 3.1 and 3.2 hold. For all q ∈ [1, +∞) such that E[|X| q ] < +∞, and any sequence of positive integers (k m ) m≥1 such that k m /m → 0 when m → +∞, we have

lim m→+∞ E W q q µ Xn , µ (km) X ′ m = 0, uniformly in n. Theorem 3.3 is proved in Subsection 3.3.
Remark 3.4 (On Assumption 3.2). Assumption 3.2 is obviously satisfied if X ′ has a finite first order moment, but also for some heavy-tailed distributions. It writes under the equivalent form

+∞ 0 P(|X ′ | > r) m 0 dr < +∞,
which may be easier to check. An example of a random variable which does not satisfy this assumption,

in dimension d = 1, is X ′ = exp(1/U )
E W q q µ Xn , µ (1)
X ′ m = E [dist(X, supp(µ X ′ )) q ] .
3.2. Rates of convergence. The next step of our study consists in complementing Theorem 3.3 with a rate of convergence. We first discuss the case k = 1. Following [START_REF] Chen | Explaining success of nearest neighbor methods in prediction[END_REF], we start by writing

E W q q µ Xn , µ (1) X
′ m = E X -NN X ′ m (X) q = E E X -NN X ′ m (X) q X , (15)
and observe that for any

x ∈ supp(µ X ), |x -NN X ′ m (x)| = min j∈ 1,m |x -X ′ j |.
If there is an open set U of R d containing x and such that µ X ′ (• ∩ U ) has a density p X ′ with respect to the Lebesgue measure which is continuous at x, then an elementary computation shows that, for all r ≥ 0,

lim m→+∞ P m 1/d min j∈ 1,m |x -X ′ j | > r = exp -r d v d p X ′ (x) ,
where v d denotes the volume of the unit sphere of

R d for the norm | • |. If p X ′ (x) > 0 then this indicates that the correct order of convergence in Theorem 3.3 should be m -q/d . If p X ′ (x) = 0, or if the measure µ X ′ (• ∩ U )
is not absolutely continuous with respect to the Lebesgue measure, it is easy to construct elementary examples yielding different rates of convergence; see also [START_REF] Biau | Lectures on the nearest neighbor method[END_REF]Chapter 2] for the singular case. We leave these peculiarities apart and work under the following strengthening of the support condition of Assumption 3.1.

Assumption 3.6 (Strong support condition). There exists an open set

U ⊂ R d which contains supp(µ X )
and such that:

(i) the measure µ X ′ (• ∩ U ) has a density p X ′ with respect to the Lebesgue measure;

(ii) the density p X ′ is continuous and positive on U ;

(iii) there exist κ ∈ (0, 1] and r κ > 0 such that, for any x ∈ U , for any r ∈ [0, r κ ],

P X ′ ∈ B(x, r) ≥ κp X ′ (x)v d r d .
Obviously, Assumption 3.6 implies Assumption 3.1 because then supp(µ X ) ⊂ U ⊂ supp(µ X ′ ).

Part (iii) of Assumption 3.6 was introduced in [START_REF] Gadat | Classification in general finite dimensional spaces with the k-nearest neighbor rule[END_REF] in the context of Nearest Neighbor classification, and called Strong minimal mass assumption there. Similar assumptions are commonly used in set estimation, geometric inference and quantization, such as standardness [START_REF] Cuevas | A approach to support estimation[END_REF] or Ahlfors regularity [START_REF] Kloeckner | Approximation by finitely supported measures[END_REF].

Under Assumption 3.6, for all x ∈ supp(µ X ), a positive random variable Z such that

P(Z > r) = exp(-r d v d p X ′ (x)) has moments E [Z q ] = Γ(1 + q/d) (v d p X ′ (x)) q/d
, where Γ denotes Euler's Gamma function. Therefore, as soon as the sequence

m q/d min j∈ 1,m X -X ′ j q
is uniformly integrable, the normalized quantity

m q/d E W q q µ Xn , µ (1)
X ′ m converges to Γ(1 + q/d) v q/d d E 1 p X ′ (X) q/d
, when m goes to infinity. This statement appears for example in the literature of stochastic optimal quantization [START_REF] Graf | Foundations of quantization for probability distributions[END_REF]Theorem 9.1]. Here, we provide an explicit moment condition ensuring uniform integrability. Assumption 3.7 (Moments). In addition to Assumption 3.6, the condition

E 1 + |X| q p X ′ (X) q/d < +∞ holds.
Assumptions 3.6 and 3.7 are discussed in more detail below. We now state our second main result.

Theorem 3.8 (Convergence rates for k = 1). Let Assumptions 3.2 and 3.6 hold, and let q ∈ [1, +∞) be such that Assumption 3.7 holds. Then we have

lim m→+∞ m q/d E W q q µ Xn , µ (1)
X ′ m = Γ(1 + q/d) v q/d d E 1 p X ′ (X) q/d . Theorem 3.8 is proved in Subsection 3.3.
We now discuss the estimation of µ Xn by the weighted empirical measure µ [START_REF] Buttazzo | On the Wasserstein distance between mutually singular measures[END_REF], we first observe that we always have

(k) X ′ m for an arbitrary k ∈ 1, m . By
W q µ Xn , µ (k) X ′ m ≥ W q µ Xn , µ (1) X ′ m ,
so that the estimation of µ Xn is deteriorated by increasing the number of neighbors. Still, in the asymptotic regime of Theorem 3.3, a bound of the same order of magnitude as Theorem 3.8 may be obtained. Corollary 3.9 (Convergence rates for k-NN). Under the assumptions of Theorem 3.8, for any nonde-

creasing sequence of positive integers (k m ) m≥1 such that k m /m → 0 when m → +∞, we have lim sup m→+∞ m k m q/d E W q q µ Xn , µ (km) X ′ m ≤ c d,q Γ(1 + q/d) v q/d d E 1 p X ′ (X) q/d , with some constant c d,q > 1.
Corollary 3.9 is proved in Subsection 3.3, where the expression of the constant c d,q is also given. Remark 3.10 (Optimal choice of µ X ′ ). When X has a density p X with respect to the Lebesgue measure, an interesting fact is that the minimum of the quantity E 1/p X ′ (X) q/d over the probability measure p X ′ is not reached when p X ′ = p X . Instead, according to [START_REF] Zador | Development and evaluation of procedures for quantizing multivariate distributions[END_REF], the minimum is attained when p X ′ (x) ∝ p X (x) d/(q+d) . Remark 3.11 (NN distance without covariate shift). In the case where µ X = µ X ′ , the quantity

E W q q µ Xn , µ (1) X
′ m = E X -NN X ′ m (X)
q is called Nearest Neighbor distance. It naturally arises in the theoretical study of NN regression and classification [START_REF] Biau | Lectures on the nearest neighbor method[END_REF]Chapter 2]. Previous works on the topic focus mainly on the convergence when q = 2 and assume that X has a bounded support [START_REF] Biau | Lectures on the nearest neighbor method[END_REF][START_REF] Evans | Asymptotic moments of near-neighbour distance distributions[END_REF][START_REF] Liitiäinen | Bounds on the mean power-weighted nearest neighbour distance[END_REF][START_REF] Penrose | Laws of large numbers and nearest neighbor distances[END_REF]. Some works [START_REF] Chen | Explaining success of nearest neighbor methods in prediction[END_REF][START_REF] Kohler | Rates of convergence for partitioning and nearest neighbor regression estimates with unbounded data[END_REF] consider some random variables X with unbounded support in the context of k-NN regression, but make the assumption of a bounded regression function ψ.

In this perspective, a direct corollary from Theorem 3.8 is the following statement: let X have a density p X for which the strong minimal mass assumption 3.6 (iii) holds with U = R d and

R d (1 + |x| q )p X (x) 1-q/d dx < +∞.
Let X n = (X 1 , . . . , X n ) be an i.i.d sample from p X , independent from X, and let NN Xn denote the NN among X n . We have

E [|X -NN Xn (X)| q ] ∼ n→+∞ Γ(1 + q/d) v q/d d n q/d R d p X (x) 1-q/d dx.
This extends the results of the literature by ensuring the asymptotic equivalence for random variables with unbounded support.

Let us conclude this subsection with some comments on Assumptions 3.6 and 3.7. When X has a compact support, Assumptions 3.6 and 3.7 are verified as soon as µ X ′ has a continuous density p X ′ which is bounded from below and above on an open set U ′ containing the support of µ X . Indeed, in that case there exist ǫ > 0 and an open subset

U of U ′ such that U contains supp(µ X )
and U ′ contains the ǫ-neighborhood of U . Then, Assumption 3.6 (iii) is verified with r κ < ǫ and

κ = inf x∈U ′ p X ′ (x)/ sup x∈U ′ p X ′ (x).
Assumptions 3.6 and 3.7 also hold in some nontrivial noncompact cases. An example of a sufficient condition for Assumption 3.6, which does not depend on µ X , is given in the next statement and is proved in Subsection 3.3. We also refer to [20, Section 2.4] for a discussion of this assumption.

Assumption 3.7 gives a relationship between µ X and p X ′ to ensure the convergence. In essence, it asserts that the tail of µ X must be quite lightweight compared to the tail of p X ′ . For instance, if X and X ′ are centered Gaussian vectors with respective covariance σ 2 I d and σ ′2 I d , then by Lemma 3.12, Assumption 3.6 is satisfied with U = R d , and it is easy to check that for q ∈ [1, +∞), Assumption 3.7 holds if and only if σ ′2 > σ 2 q/d.

Proofs.

In this subsection, we present the proofs of Theorems 3.3 and 3.8, Corollary 3.9 and Lemma 3.12.

Proof of Theorem 3.3. We begin our proof with the constant case k m = 1 for all m and then extend it to the general case. We recall that by ( 13),

E W q q µ Xn , µ (1) X
′ m = E X -NN X ′ m (X) q = E min j∈ 1,m |X -X ′ j | q .
By Assumption 3.1, X ∈ supp(µ X ′ ) almost surely, so that we deduce from Lemma 2.2 in [5, Chapter 2] that

min j∈ 1,m |X -X ′ j | q a.s -→ m→+∞ 0.
Let m 0 be the integer given by Assumption 3.2, we have

min j∈ 1,m |X -X ′ j | q ≤ 2 q-1 |X| q + min j∈ 1,m |X ′ j | q .
The random variable |X| q is integrable by assumption and for m ≥ ⌈q⌉m 0 , the inequality

E min j∈ 1,m X ′ j q ≤ E min j∈ 1,m X ′ j ⌈q⌉ q/⌈q⌉ ≤ E min j∈ 1,m 0 X ′ j min j∈ m 0 +1,2m 0 X ′ j • • • min j∈ (⌈q⌉-1)m 0 +1,⌈q⌉m 0 X ′ j q/⌈q⌉ ≤ E min j∈ 1,m 0 X ′ j q < +∞
holds. Then by the dominated convergence theorem,

E min j∈ 1,m |X -X ′ j | q -→ m→+∞ 0.
For the general case k m /m → 0, we adapt directly the proof of [START_REF] Biau | Lectures on the nearest neighbor method[END_REF]Theorem 2.4] to the context

µ X = µ X ′ . Let us fix l ∈ 1, m/2 and partition the set {X ′ 1 , . . . , X ′ m } into 2l sets of size m 1 , . . . , m 2l with, for all j ∈ 1, 2l , ⌊m/2l⌋ ≤ m j ≤ ⌊m/2l⌋ + 1.
We denote by NN , there are at least l subsets j for which

|X -NN (l) X ′ m (X)| ≤ |X -NN (1,j) X ′ m (X)|, therefore |X -NN (l) X ′ m (X)| q ≤ 1 l 2l j=1 |X -NN (1,j) X ′ m (X)| q ,
and consequently

E X -NN (l) X ′ m (X) q ≤ 2E X -NN X ′ ⌊m/2l⌋ (X) q .
Finally, we deduce from ( 14) that, as soon as k m ≤ m/2,

E W q q µ Xn , µ (km) X ′ m ≤ 1 k m km l=1 E X -NN (l) X ′ m (X) q ≤ 2 k m km l=1 E X -NN X ′ ⌊m/2l⌋ (X) q ≤ 2E X -NN X ′ ⌊m/2km⌋ (X) q , (16)
which goes to 0 as a consequence of the first part of the proof when m/2k m goes to infinity.

Proof of Theorem 3.8. By (13), we have

E m q/d W q q ( µ Xn , µ (1)
X ′ m ) = E E m q/d min j∈ 1,m X -X ′ j q X = R d +∞ 0 P m q/d min j∈ 1,m |x -X ′ j | q > t dtdµ X (x) = R d +∞ 0 P(m q/d |x -X ′ | q > t) m dtdµ X (x), (17)
by independence of the X ′ j . The proof consists in computing the pointwise limit of P(m p/d |x -Y | p > t) m for (x, t) ∈ supp(µ X ) × R + and then establishing the convergence of the integral via the dominated convergence theorem.

Pointwise convergence. We have

P(m q/d |x -X ′ | q > t) m = 1 -P(|x -X ′ | ≤ t 1/q /m 1/d ) m = exp m log 1 -P(|x -X ′ | ≤ t 1/q /m 1/d ) .
By Assumption 3.6, we have

P(|x -X ′ | ≤ t 1/q /m 1/d ) = p X ′ (x)v d t d/q /m + o(1/m),
with v d the volume of the unit sphere. Thus

m log 1 -P(|x -X ′ | ≤ t 1/q /m 1/d ) = -p X ′ (x)v d t d/q + o(1),
and we conclude that

P(m q/d |x -X ′ | q > t) m -→ m→+∞ exp -p X ′ (x)v d t d/q .
Dominated convergence. Let r κ > 0 be given by Assumption 3.6. We split the integral in the righthand side of [START_REF] Evans | Asymptotic moments of near-neighbour distance distributions[END_REF] and study each term separately

R d +∞ 0 P(m q/d |x -X ′ | q > t) m dtdµ X (x) = I + II with I := R d r q κ m q/d 0 P(|x -X ′ | > t 1/q /m 1/d ) m dtdµ X (x), II := R d +∞ r q κ m q/d P(|x -X ′ | > t 1/q /m 1/d ) m dtdµ X (x).
Convergence of I. For t ∈ [0, r q κ m q/d ], we have t 1/q /m 1/d ≤ r κ and thus

P(|x -X ′ | > t 1/q /m 1/d ) m = 1 -P(|x -X ′ | ≤ t 1/q /m 1/d ) m ≤ 1 - p X ′ (x)v d κt d/q m m by Assumption 3.6.
Using the elementary inequality (1a/n) n ≤ exp(-a) for a ≤ n, we can write

P(|x -X ′ | > t 1/q /m 1/d ) m ≤ exp(-κv d p X ′ (x)t d/q ).
This bound does not depend on m and the integral

R d +∞ 0 exp -κv d p X ′ (x)t d/q dtdµ X (x) = R d Γ(1 + q/d) (κv d p X ′ (x)) q/d dµ X (x) = Γ(1 + q/d) (κv d ) q/d E 1 p X ′ (X) q/d
is finite by Assumption 3.7. We therefore deduce from the dominated convergence theorem that

lim m→+∞ I = R d +∞ 0 exp -p X ′ (x)v d t d/q dtdµ X (x) = Γ(1 + q/d) v q/d d E 1 p X ′ (X) q/d .
Convergence of II. Let m ≥ 2(q + 1)m 0 . Using the change of variable r q = t/m q/d , we have

II = q R d +∞ rκ m q/d r q-1 P(|x -X ′ | > r) m drdµ X (x) ≤ q R d +∞ rκ V m (x, r)drdµ X (x), with V m (x, r) := m q/d P(|x -X ′ | > r κ ) m-(q+1)m 0 r q-1 P(|x -X ′ | > r) (q+1)m 0 .
As P(|x -X ′ | > r κ ) < 1 for all x in U , by Assumption 3.6, V m (x, r) is pointwise convergent to 0 on the support of µ X . We check that V m (x, r) is bounded from above by an integrable function which does not depend on m. Let us denote m ′ = m -(q + 1)m 0 ≥ m/2 and rewrite

m q/d P(|x -X ′ | > r κ ) m-(q+1)m 0 = m m ′ q/d m ′q/d P(|x -X ′ | > r κ ) m ′ ≤ 2 q/d m ′q/d 1 -P(|x -X ′ | ≤ r κ ) m ′ ≤ 2 q/d m ′q/d exp -m ′ κp X ′ (x)v d r d κ ,
where we have used Assumption 3.6 and the elementary above inequality at the third line. We deduce that

m q/d P(|x -X ′ | > r κ ) m-(q+1)m 0 ≤ C 1 p X ′ (x) q/d , C 1 := 2 q/d (κv d r d κ ) q/d sup u≥0 (u q/d e -u ), so that (18) V m (x, r) ≤ V (x, r) := C 1 p X ′ (x) q/d r q-1 P(|x -X ′ | > r) (q+1)m 0 .
To complete the proof, we verify that V (x, r) is integrable on U × [r κ , +∞). We first fix x ∈ R d and estimate the integral of V (x, r) in r. Using the fact that if |x -

X ′ | > r then |X ′ | > r -|x|, we first write +∞ rκ r q-1 P(|x -X ′ | > r) (q+1)m 0 dr ≤ +∞ 0 r q-1 P(|X ′ | > r -|x|) (q+1)m 0 dr = +∞ -|x| (r + |x|) q-1 P(|X ′ | > r) (q+1)m 0 dr. On the interval [-|x|, 0], we have 0 -|x| (r + |x|) q-1 P(|X ′ | > r) (q+1)m 0 dr = 0 -|x| (r + |x|) q-1 dr = |x| q q .
On the interval [0, +∞), we first rewrite

+∞ 0 (r + |x|) q-1 P(|X ′ | > r) (q+1)m 0 dr = +∞ 0 (r + |x|) q-1 P min j∈ 1,m 0 |X ′ j | > r q+1 dr,
and recall from Assumption 3.2 that

C 2 := E[min j∈ 1,m 0 |X ′ j |] < +∞.
As a consequence, we deduce from Markov's inequality that the right-hand side in the previous equality is bounded from above by

|x|∨1 0 (r + |x|) q-1 dr + C q+1 2 +∞ |x|∨1 (r + |x|) q-1 r q+1 dr.
If |x| ≤ 1 then this expression is bounded from above. If |x| > 1, then we have |x| 0 (r + |x|) q-1 dr ≤ 2 q-1 |x| q on the one hand, and

+∞ |x| (r + |x|) q-1 r q+1 dr = 1 |x| +∞ 1 (u + 1) q-1 u q+1 du,
which is bounded from above, on the other hand. Overall, we conclude that there exists a constant C 3 such that ( 19)

+∞ rκ r q-1 P(|x -X ′ | > r) (q+1)m 0 dr ≤ C 3 (1 + |x| q ).
As a consequence, the combination of ( 18) and ( 19) yields

R d +∞ rκ V (x, r)drdµ X (x) ≤ C 1 C 3 E 1 + |X| q p X ′ (X) q/d ,
which by Assumption 3.7 allows to apply the dominated convergence theorem to show that II goes to 0, and thereby completes the proof.

Proof of Corollary 3.9. We start from the second line of Equation ( 16) and estimate its right-hand side

E W q q ( µ Xn , µ (km) X ′ m ) ≤ 2 k m km l=1 E X -NN X ′ ⌊m/2l⌋ (X) q = 2 k m km l=1 2k m m l k m m 2l q/d E X -NN X ′ ⌊m/2l⌋ (X) q = k m m q/d 2 q/d+1 k m km l=1 l k m q/d F m 2l with F (u) = u q/d E[|X -NN X ′ ⌊u⌋ (X)| q ].
Let ǫ > 0. By Theorem 3.8, there exists u ǫ ≥ 0 such that, for all u ≥ u ǫ ,

F (u) - Γ(1 + q/d) v q/d d E 1 p X ′ (X) q/d ≤ ǫ.
We can remark that for m ∈ N * and l ∈ 1, k m ,

m 2l ≥ m 2k m -----→ m→+∞ +∞.
Thus, if we take m ǫ such that for all m ≥ m ǫ , m 2km ≥ u ǫ , we have

F m 2l - Γ(1 + q/d) v q/d d E 1 p X ′ (X) q/d ≤ ǫ for any m ≥ m ǫ and l ≤ k m . Consequently, 1 k m km l=1 l k m q/d F m 2l - Γ(1 + q/d) v q/d d E 1 p X ′ (X) q/d ≤ ǫ 1 k m km l=1 l k m q/d ≤ ǫ, so that lim m→+∞ 2 q/d+1 k m km l=1 l k m q/d F m 2l = c d,q Γ(1 + q/d) v q/d d E 1 p X ′ (X) q/d , where c d,q := lim m→+∞ 2 q/d+1 k m km l=1 l k m q/d =          2 q/d+1 k k l=1 l k q/d if sup m≥1 k m = k < +∞, 2 q/d+1 1 0 u q/d du = 2 q/d+1 q/d + 1 if sup m≥1 k m = +∞,
because k m is nondecreasing. This concludes the proof.

Proof of Lemma 3.12. Obviously, it suffices to check that p X ′ satisfies (iii) in Assumption 3.6. Let us denote by •, • and B(x, r) respectively the inner product and the ball of center x and radius r associated to • . We set x 0 = 0 without loss of generality. As h is positive and nonincreasing, we may fix r 0 > 0 and define

κ := h(r 0 ) h(0) ∈ (0, 1].
If x ≤ r 0 /2, then for all y ∈ B(0, r 0 /2), the monotonicity of h ensures that p X ′ (x + y) ≥ κp X ′ (x).

By the equivalence of the norms, there exist C ≥ c > 0 such that for any x ∈ R d and any r ≥ 0,

B(x, cr) ⊂ B(x, r) ⊂ B(x, Cr). Thus ∀r ≤ r 0 /2c, P X ′ ∈ B(x, r) ≥ P X ′ ∈ B(x, cr) ≥ (c/C) d v d κp X ′ (x)r d .
If x > r 0 /2, let us introduce the half-cone

C x = x ′ ∈ R d : x ′ -x, -x ≥ x ′ -x x 2 ,
and notice that for all r ≤ r 0 /2 and x ′ ∈ C x ∩ B(x, r),

x ′ 2 = x 2 + x ′ -x 2 + 2 x ′ -x, x ≤ x 2 + x ′ -x 2 -x ′ -x x ≤ x 2 + x ′ -x 2 -x ′ -x 2 = x 2 .
Thus, for all x ′ ∈ C x ∩ B(x, r), p X ′ (x ′ ) ≥ p X ′ (x). For a given r, the sets C x ∩ B(x, r) have the same volume for all x, which we denote by αv d r d for some α ∈ (0, 1/C d ). Finally, we have

∀r ≤ r 0 /2c, P(X ′ ∈ B(x, r)) ≥ P(X ′ ∈ B(x, cr) ∩ C x ) ≥ αc d v d p X ′ (x)r d .
If we take κ = (c/C) d min(αC d , κ) and r κ = r 0 /2c, we obtain the point (iii) of Assumption 3.6.

Convergence of QI

k = k m ∈ 1, m , (20) E W q q µ X , µ (km) X ′ m ≤ 2 q-1 E W q q (µ X , µ Xn ) + E W q q µ Xn , µ (km) X ′ m .
Under the assumptions of Corollary 3.9, the second term has order of magnitude at most (k m /m) q/d .

The study of the first term, namely the rate of convergence of the expected W q distance (taken to the power q) between the empirical measure of iid realizations and their common distribution, has been the subject of several works. Under the condition that there exists s > 2q such that E[|X| s ] < +∞, we have

from [19, Theorem 1] (21) E W q q (µ X , µ Xn ) =      O n -1/2 if q > d/2, O n -1/2 log(1 + n) if q = d/2, O n -q/d if q < d/2.
These estimates may be improved if more assumptions are made on µ X . For example, if this measure possesses a lower and upper bounded density on some bounded subset of R d , then the rate is known to be n -q/d even if q > d/2 [START_REF] García Trillos | On the rate of convergence of empirical measures in ∞-transportation distance[END_REF]. This rate may even be improved if µ X concentrates on a low-dimensional submanifold of R d [START_REF] Weed | Sharp asymptotic and finite-sample rates of convergence of empirical measures in Wasserstein distance[END_REF][START_REF] Divol | Measure estimation on manifolds: an optimal transport approach[END_REF], which is particularly relevant in the UQ context which motivates this study, see Remark 5.3. In order to make the use of our results as flexible as possible, from now on we shall denote by (τ q,d (n)) n≥1 a sequence such that

E W q q (µ X , µ Xn ) = O (τ q,d (n)) ,
and thus

E W q q µ X , µ (km) X ′ m = O (τ q,d (n)) + O k m m q/d .
As is sketched in the discussion above, the precise order of τ q,d (n) depends on properties of the measure µ X .

In the sequel, where we study the convergence of QI (k) m,n to QI, the W 1 distance plays a specific role, due to the Kantorovitch duality formula [START_REF] Villani | Optimal transport, volume 338 of Grundlehren der mathematischen Wissenschaften[END_REF]Remark 6.5]

(22) W 1 (µ, ν) = sup |ϕ| Lip ≤1 R d ϕ(x)dµ(x) - R d ϕ(x)dν(x) ,
where |ϕ| Lip denotes the Lipschitz constant of ϕ.

We shall need the following estimate.

Lemma 4.1 (W q 1 estimate). If E[|X| q ] < +∞ with q ≥ 2, then E [W q 1 (µ X , µ Xn )] = O 1 n q/2 + τ 1,d (n) q .
Proof. For any vector (x 1 , . . . , x n ) ∈ (R d ) n , let us define

W (x 1 , . . . , x n ) := W 1 1 n n i=1 δ x i , µ X = sup |ϕ| Lip ≤1 1 n n i=1 ϕ(x i ) - R d ϕ(x)µ X (dx)
thanks to [START_REF] Ghanem | Handbook of uncertainty quantification[END_REF]. Then for any i ∈ 1, n and x ♭ i ∈ R d , using the identity above and the fact that | sup fsup g| ≤ sup |f -g|, we get

W (x 1 , . . . , x n ) -W (x 1 , . . . , x i-1 , x ♭ i , x i+1 , . . . , x n ) ≤ sup |ϕ| Lip ≤1 ϕ(x 1 ) + • • • + ϕ(x n ) n - ϕ(x 1 ) + • • • + ϕ(x ♭ i ) + • • • + ϕ(x n ) n = 1 n sup |ϕ| Lip ≤1 ϕ(x i ) -ϕ(x ♭ i ) = 1 n |x i -x ♭ i |.
As a consequence, letting X n = (X 1 , . . . , X n ) and X ♭ n = (X ♭ 1 , . . . , X ♭ n ) be two independent samples from µ X , we deduce that the random variables V + and V -defined by

V ± := E n i=1 W (X 1 , . . . , X n ) -W (X 1 , . . . , X i-1 , X ♭ i , X i+1 , . . . , X n ) 2 ± |X n
satisfy the bound

V ± ≤ 1 n 2 n i=1 E |X i -X ♭ i | 2 |X n .
As a consequence, for any q ≥ 2 we have by Jensen's inequality

E (V ± ) q/2 ≤ 1 n q/2 E   1 n n i=1 E |X i -X ♭ i | 2 |X q/2   ≤ 1 n q/2 E |X -X ♭ | q ≤ 2 q-1 n q/2 E [|X| q ] .
We therefore deduce from the higher-order Efron-Stein inequality [7, Theorem 2] that there exists a

universal constant C q such that E [|W (X n ) -E [W (X n )]| q ] ≤ C q E (V -) q/2 + (V + ) q/2 ≤ 2 q C q n q/2 E [|X| q ]
. We conclude the proof by writing, using Jensen's inequality again,

E [W (X n ) q ] = E [|W (X n ) -E [W (X n )] + E [W (X n )]| q ] ≤ 2 q-1 (E [|W (X n ) -E [W (X n )]| q ] + E [W (X n )] q ) ,
which yields the claimed estimate. Remark 4.2. With the estimates given by (21), τ 1,d (n) is of order n -1/d as soon as d ≥ 3. Thus, in this case, we get from Lemma 4.1 that E[W q 1 (µ X , µ Xn )] is of order n -q/d . When q > d/2, this is a faster rate of decay to 0 than what one would have obtained bounding W 1 by W q .

Rate of convergence of QI (km)

m,n in the noiseless case. We assume that Y = f (X) and study the rate of convergence of QI

(km) m,n to QI. When φ • f is L-Lipschitz continuous, we deduce from (22) that QI -QI (km) m,n q = R d φ • f (x)dµ X (x) - R d φ • f (x)d µ (km) X ′ m (x) q ≤ L q W q 1 µ X , µ (km) X ′ m .
We therefore obtain the following result. and let the assumptions of Corollary 3.9 hold for some q ≥ 2. Then

(23) E QI -QI (km) m,n q 1/q = O 1 √ n + τ 1,d (n) + O k m m 1/d .
There is no need for k m to go to infinity and thus k m = 1 is optimal.

These computations can be adapted to cases other than φ • f Lipschitz continuous. For instance, if A ⊂ R e , φ(y) = 1 {y∈A} and f is globally Lipschitz continuous, it is possible to use the margin assumption of [START_REF] Tsybakov | Optimal aggregation of classifiers in statistical learning[END_REF] to deduce theoretical rates of convergence in the estimation of QI = P(Y ∈ A).

Rate of convergence of QI

(km) m,n in the noisy case. We now study the convergence of QI (km) m,n to QI when Y = f (X, Θ). A first striking result is then that even under the assumptions of Theorem 3.3, the estimator QI

(1) m,n need not be consistent. Indeed, consider the case where X is actually deterministic and always equal to some x 0 ∈ R d . Then we have

QI (1) m,n = 1 n n i=1 φ(Y ′ j (1) i ),
where j

(1) i is the index of the closest X ′ j to X i . But since X i = x 0 for all i, all indices j

(1) i are equal to some j (1) and the estimator rewrites

QI (1) m,n = φ(Y ′ j (1) ) = φ(f (X ′ j (1) , Θ j (1)
)).

While Assumption 3.1 ensures that X ′ j (1) converges to x 0 when m → +∞, in general the corresponding sequence of Θ j (1) does not converge.

As is evidenced on this example, the presence of an atom in the law of X makes the estimator QI (1) m,n depend on a single realization of Θ and therefore prevents this estimator from displaying an averaging behavior with respect to the law of Θ. In Proposition 4.4, we clarify this point by exhibiting a necessary and sufficient condition for the estimator QI [START_REF] Allaire | Surrogate modeling for uncertainty assessment with application to aviation environmental system models[END_REF] m,n to be consistent, while in Proposition 4.5, we show that replacing QI m,n with k m → +∞ allows to recover such an averaging behavior and makes the estimator consistent, even when µ X has atoms. In the latter case, we also provide rates of convergence in Proposition 4.6.

We recall that ψ(x) = E[φ(f (x, Θ))] is defined in Equation ( 6). In the next statement, we denote by A X the set of atoms of µ X , that is to say the set of x ∈ R d such that P(X = x) > 0, and introduce the notation ϑ(x) := Var(φ(f (x, Θ))).

m,n -QI ------→ m,n→+∞ 0 if and only if ∀x ∈ A X , ϑ(x) = 0.
In particular, under the above assumptions, if the law of X has no atom, i.e. A X = ∅, then QI

(1) m,n converges to QI.

Proof. Let us write

QI (1) m,n -QI = QI (1)
m,n -QI

m,n + QI (1) m,n -QI , with QI (1)
m,n = 1 m m j=1 w (1)
j ψ(X ′ j ). (1)
Using the Lipschitz continuity of ψ, the duality formula ( 22) and Theorem 3.3, we get that QI

m,n -QI converges to 0 when m, n → +∞, in L 1 . Therefore, QI

m,n -QI converges to 0 if and only if QI

(1) m,n - QI (1) m,n converges to 0. Let us rewrite QI (1) m,n -QI (1) m,n = 1 m m j=1 w (1) j φ f X ′ j , Θ j -ψ X ′ j = 1 n n i=1 φ f X ′ j (1) i , Θ j (1) i -ψ X ′ j (1) i
, introduce the notation A + X := {x ∈ A X : ϑ(x) > 0}, and denote

e 1 := 1 n n i=1 φ(f (X ′ j (1) i , Θ j (1) i )) -ψ(X ′ j (1) i ) 1 {X i ∈A + X } , e 2 := 1 n n i=1 φ(f (X ′ j (1) i , Θ j (1) i )) -ψ(X ′ j (1) i ) 1 {X i ∈A + X } .
In Step 1 below, we prove that

E [|e 1 |] ------→ n,m→+∞ 0,
demonstrating at the same time the direct implication of the convergence when

A + X = ∅. In Step 2, we show that if A + X = ∅ then E[|e 2 |]
does not converge to 0, which implies that in this case, QI

m,n -QI (1) m,n (1)
does not converge to 0 in L 1 .

In both steps, we shall use the following preliminary remark: given a measurable subset A of R d , taking the conditional expectation with respect to (X n , X ′ m ) it is easy to see that for i ∈ 1, n ,

E (φ(f (X ′ j (1) i , Θ j (1) i )) -ψ(X ′ j (1) i
))1 {X i ∈A} = 0, and for (i

1 , i 2 ) ∈ 1, n 2 , E (φ(f (X ′ j (1) i 1 , Θ j (1) i 1 )) -ψ(X ′ j (1) i 1
))

1 {X i 1 ∈A} (φ(f (X ′ j (1) i 2 , Θ j (1) i 2 )) -ψ(X ′ j (1) i 2 ))1 {X i 2 ∈A} = E 1 {j (1) i 1 =j (1) i 2 } ϑ(X ′ j (1) i 1 )1 {X i 1 ∈A,X i 2 ∈A} .
Therefore,

E[|e 1 | 2 ] = 1 n 2   n i=1 E ϑ(X ′ j (1) i )1 {X i ∈A + X } + i 1 =i 2 E 1 {j (1) i 1 =j (1) i 2 } ϑ(X ′ j (1) i 1
)

1 {X i 1 ∈A + X ,X i 2 ∈A + X }   = 1 n E ϑ(X ′ j (1) 1 )1 {X 1 ∈A + X } + n -1 n E 1 {j (1) 1 =j (1) 2 } ϑ(X ′ j (1) 1 )1 {X 1 ∈A + X ,X 2 ∈A + X } , and a similar expression holds for E[|e 2 | 2 ].
Step 1. Thanks to the boundedness of φ, and thus of ϑ, it is immediate that

1 n E[ϑ(X ′ j (1) 1 )1 {X 1 ∈A + X } ] -----→ n→+∞ 0,
uniformly in m. Therefore, to show that E[|e 1 | 2 ] converges to 0, it suffices to prove that

E 1 {j (1) 1 =j (1) 2 } ϑ(X ′ j (1) 1 )1 {X 1 ∈A + X ,X 2 ∈A + X } -----→ m→+∞ 0.
In this purpose, let us first write

E 1 {j (1) 1 =j (1) 2 } ϑ(X ′ j (1) 1
)1

{X 1 ∈A + X ,X 2 ∈A + X } ≤ E 1 {NN X ′ m (X 1 )=NN X ′ m (X 2 )} ϑ(NN X ′ m (X 1 ))1 {X 1 ∈A + X }
, and recall that, by Assumption 3.1 and Lemma 2.2 in [5, Chapter 2], NN X ′ m (X 1 ) converges to X 1 and

NN X ′ m (X 2 ) converges to X 2 , almost surely. As a consequence, if X 1 ∈ A X \ A +
X then ϑ(X 1 ) = 0 and by the continuity of ϑ and the boundedness of φ, the dominated convergence theorem shows that

E 1 {X 1 ∈A X \A + X } 1 {NN X ′ m (X 1 )=NN X ′ m (X 2 )} ϑ(NN X ′ m (X 1 )) -----→ m→+∞ 0.
On the other hand, if X 1 ∈ A X , then almost surely X 1 = X 2 , and therefore

1 {NN X ′ m (X 1 )=NN X ′ m (X 2 )}
converges to 0 almost surely. Using the boundedness of φ and the dominated convergence theorem again, we deduce that

E 1 {X 1 ∈A X } 1 {NN X ′ m (X 1 )=NN X ′ m (X 2 )} ϑ(NN X ′ m (X 1 )) -----→ m→+∞ 0, which shows that E[|e 1 | 2 ],
and thus E[|e 1 |], converge to 0.

Step 2. Let us now assume that A + X is nonempty and show that e 2 does not converge to 0 in L 1 . We shall actually prove that e 2 does not converge to 0 in L 2 : since e 2 is bounded then this prevents the convergence from occuring in L 1 . From the preliminary remark, we write

E[|e 2 | 2 ] = 1 n E ϑ(X ′ j (1) 1 )1 {X 1 ∈A + X } + n -1 n E 1 {j (1) 1 =j (1) 2 } ϑ(X ′ j (1) 1 )1 {X 1 ∈A + X ,X 2 ∈A +
X } , and we prove that

lim inf m→+∞ E 1 {j (1) 1 =j (1) 2 } ϑ(X ′ j (1) 1 )1 {X 1 ∈A + X ,X 2 ∈A + X } > 0. Let x ∈ A + X . Obviously, E 1 {j (1) 1 =j (1) 2 } ϑ(X ′ j (1) 1 )1 {X 1 ∈A + X ,X 2 ∈A + X } ≥ E 1 {j (1) 1 =j (1) 2 } ϑ(X ′ j (1) 1 )1 {X 1 =X 2 =x} = E ϑ(NN X ′ m (x))1 {X 1 =X 2 =x} .
By Assumption 3.1 and Lemma 2.2 in [5, Chapter 2] again, NN X ′ m (x) converges to x almost surely, therefore using the continuity and boundedness assumptions on ϑ, the dominated convergence theorem

shows that

E ϑ(NN X ′ m (x))1 {X 1 =X 2 =x} -----→ m→+∞ ϑ(x)µ X ({x}) 2 > 0,
which completes the proof.

We now study the estimator QI (km) m,n and show that it is unconditionnally consistent as soon as k m → +∞. We provide L 2 convergence rates in Proposition 4.6. Proof. We decompose the error as [START_REF] Kloeckner | Approximation by finitely supported measures[END_REF] QI As ψ is globally Lipschitz continuous and does not depend on Θ, we have

E QI (km) m,n -QI 2 1/2 ≤ LE W 2 1 µ , µ (km) X ′ m 1/2 ≤ 2L E W 2 1 (µ X , µ Xn ) + E W 2 1 ( µ Xn , µ (km) X ′ m ) 1/2
by Jensen's inequality, with L the Lipschitz constant of ψ. The second term is bounded from above by

E[W 2 2 ( µ Xn , µ (km) X ′ m
)], which goes to 0 by Theorem 3.3. For the first term, the same arguments as in the proof of Lemma 4.1 show that

E W 2 1 (µ X , µ Xn ) = O 1 n + E [W 1 (µ X , µ Xn )] 2 .
Since W 1 (µ X , µ Xn ) converges to 0 in probability [START_REF] Panaretos | Statistical aspects of Wasserstein distances[END_REF] and the assumption that E[|X| 2 ] < +∞ ensures that this sequence is uniformly integrable, we deduce that its expectation converges to 0 [6, Section 5].

Thus, the second part of the right-hand side of (24) converges to 0 in L 2 .

Let us consider the first part in the right-hand side of [START_REF] Kloeckner | Approximation by finitely supported measures[END_REF]. We write the quadratic error

E QI (km) m,n -QI (km) m,n 2 = E     1 m m j=1 w (km) j (ψ(X ′ j ) -φ(f (X ′ j , Θ j )))   2   = E   1 m 2 m j=1 w (km)2 j ψ(X ′ j ) -φ(f (X ′ j , Θ j )) 2   + E   m -1 m 2 j =l w (km) j w (km) l ψ(X ′ j ) -φ(f (X ′ j , Θ j )) ψ(X ′ l ) -φ(f (X ′ l , Θ l ))   .
Using the fact that E[w

(km) j f (X ′ j , Θ j )|X n , X ′ m ] = w (km) j
ψ(X ′ j ) by definition and the independence of the Θ j , the cross terms vanish. The remaining quadratic term is

E   1 m 2 m j=1 w (km)2 j ψ(X ′ j ) -φ(f (X ′ j , Θ j )) 2   = 1 m 2 m j=1 E w (km)2 j (ψ(X ′ j ) -φ(f (X ′ j , Θ))) 2 ≤ 4 m 2 m j=1 E w (km) j 2 φ 2 ∞ . (25)
We remark that

m j=1 w (km) j 2 = m 2 n 2 k 2 m n i 1 ,i 2 =1 km l 1 ,l 2 =1
1 {j (l 1 )

i 1 =j (l 2 ) i 2 }
and that for some fixed i 1 ,i 2 and l 1 , there exists exactly one l 2 ∈ 1, m such that j

(l 1 ) i 1 = j (l 2 ) i 2 as (j (l)
i 2 ) 1≤l≤m is a permutation of 1, m . Therefore, there exists at most one l 2 ∈ 1, k m verifying this property and, consequently,

m j=1 w (km) j 2 ≤ m 2 n 2 k 2 m n i 1 ,i 2 =1 km l 1 =1 1 = m 2 k m .
We can then bound the second term by

E     1 m m j=1 w (km) j (ψ(X ′ j ) -φ(f (X ′ j , Θ j )))   2   1/2 ≤ 2 k 1/2 m φ ∞ ,
which converges to 0 when k m goes to infinity.

In order to complement Proposition 4.5 with a rate of convergence, we restart from the decomposition [START_REF] Kloeckner | Approximation by finitely supported measures[END_REF]. Under the additional assumptions of Corollary 3.9 with q = 2, the same arguments as in the proof of Proposition 4.3 yield

E QI (km) m,n -QI 2 1/2 = O 1 √ n + τ 1,d (n) + O k m m 1/d
, while we still have

E QI (km) m,n -QI (km) m,n 2 1/2
= O 1 √ k m from the proof of Proposition 4.5. As a consequence,

E QI -QI (km) m,n 2 1/2 = O 1 √ n + τ 1,d (n) + O k m m 1/d + 1 √ k m .
Optimizing in k m , we get the following statement.

Proposition 4.6 (Rates of convergence in the noisy case). Under the assumptions of Proposition 4.5 and Corollary 3.9 with q = 2, we have

E QI -QI (km) m,n 2
1/2 = O 1 √ n + τ 1,d (n) + O 1 m 1/(d+2) for k m ∼ m 2/(d+2) .
The loss of convergence order with respect to Proposition 4.3 is similar to the NNR context, in which it deteriorates from the rate 1/d in the noiseless case to the rate of 1/(d + 2) in the noisy case [5, Section 14.6 and Section 15.3].

Applications and numerical illustration

We present a reformulation of our results in a standard framework for k-NN regression in Subsection 5.1, and then provide a detailed account of the original motivation of this work by decompositionbased UQ in Subsection 5.2. Last, numerical illustrations of our main results in a simple setting are reported in Subsection 5.3; we refer to [START_REF] Touboul | Model of margin, margin sensitivity analysis and uncertainty quantification in graphs of functions in complex industrial systems[END_REF]Chapter 11] for an application in an industrial context.

= E [Y |X = x] = E [f (x, Θ)] ,
and the k-NN estimator of r(x) is defined from the training set by

r (k) m (x) = 1 k k l=1 Y ′ j (l) (x) ,
where j (l) (x) denotes the (smallest) index j such that X

′ j = NN (l) X ′ m (x).
We are no longer interested in some quantity E[φ(Y )] but rather in the (L 2 ) generalization error under covariate shift

E r(X) -r (k) m (X) 2 , X ∼ µ X .
For the sake of simplicity we assume that Y , r(X), etc. take scalar values.

Theorem 5.1 (L 2 generalization error of the k-NN regression under covariate shift). Let µ X and µ X ′ verify the assumptions of Theorem 3.8 for q = 2. Assume in addition that f is Lipschitz continuous in x, uniformly in Θ, and that Var(f (x, Θ)) ≤ σ 2 < +∞ for all x ∈ supp(µ X ′ ).

When the sequence (k m ) m≥1 satisfies the assumptions of Corollary 3.9, with k m ∼ m 2/(d+2) , there exists C ≥ 0 such that

lim sup m→+∞ m 1/(2+d) E r(X) -r (km) m (X) 2 1/2 ≤ σ + CE 1 p X ′ (X) 2/d 1/2 .
We retrieve essentially the same orders of convergence as in the case without covariate shift. The quantity E[1/p X ′ (X) 2/d ] 1/2 seems to be the relevant bound of the loss due to the use of µ X ′ instead of µ X and we expect that the greater this quantity is, the slower the convergence will be.

Proof. The proof is an adaptation of [5, Theorem 14.5], using elements of the proofs of Theorem 3.8 and Corollary 3.9. We can decompose the L 2 error E r(X)r (km) m (X)

2 1/2 ≤ E r(X) -r (km) m (X) 2 1/2 + E r (km) m (X) -r (km) m (X) 2 1/2 with r (km) m (x) = 1 k m km l=1 E f (NN (l) X ′ m (x), Θ) .
By Jensen's inequality, the first term can be bounded by

E r(X) -r (km) m (X) 2 ≤ L 2 1 k m km l=1 E X -NN (l) X ′ m (X) 2 ,
where L is the Lipschitz constant of f , and then following the proof of Corollary 3.9, we get

lim sup m→+∞ m k m 1/d E r(X) -r (km) m (X) 2 1/2 ≤ CE 1 p X ′ (X) 2/d 1/2 , with C := L(c d,2 Γ(1 + 2/d)/v 2/d d ) 1/2 . The second term is bounded by E r (km) m (X) -r (km) m (X) 2 1/2 = 1 k m E km l=1 f (NN (l) X ′ m (X), Θ j (l) (X) ) -E[f (NN (l) X ′ m (X), Θ)|X] 2 1/2 ≤ 1 k 1/2 m σ.
The optimal rate is k m ∼ m 2/(2+d) , leading to

lim sup m→+∞ m 1/(2+d) E r(X) -r (km) m (X) 2 1/2 ≤ σ + CE 1 p X ′ (X) 2/d 1/2
, which completes the proof. 5.2. Application to decomposition-based UQ. In the UQ context, the relation (1) represents a computer simulation [START_REF] Fang | Design and modeling for computer experiments[END_REF][START_REF] De Rocquigny | Uncertainty in industrial practice: a guide to quantitative uncertainty management[END_REF]: the random variable X is the input of the simulation, the random variable Θ describes the set of its parameters, the function f is the numerical model and the random variable Y is the output of the simulation. The function φ involved in the definition (2) of the quantity of interest QI is the observable.

The fact that we assume that both X and Θ may be random, but with distinct sources of uncertainty (which is modeled by their statistical independence), comes from the study of uncertainty propagation in complex networks of numerical models [START_REF] Allaire | Surrogate modeling for uncertainty assessment with application to aviation environmental system models[END_REF][START_REF] Amaral | A decomposition-based approach to uncertainty analysis of feed-forward multicomponent systems[END_REF][START_REF] Marque-Pucheu | Efficient sequential experimental design for surrogate modeling of nested codes[END_REF][START_REF] Sanson | Systems of Gaussian process models for directed chains of solvers[END_REF][START_REF] Ming | Integrated emulators for systems of computer models[END_REF]. In this context, several computer codes, representing various disciplines, are connected with each other by the fact that the outputs of certain codes are taken as inputs of other codes. Then f represents one discipline, with 'internal' uncertain parameters Θ whose law is known by the agent in charge of the simulation, and 'external' uncertain parameters X which are the output of possibly several upstream numerical simulations. Independently from our complex system context, assuming that the internal parameter Θ may be random is a standard practice to take aleatoric or epistemic uncertainty into account [START_REF] De Rocquigny | Uncertainty in industrial practice: a guide to quantitative uncertainty management[END_REF][START_REF] Ghanem | Handbook of uncertainty quantification[END_REF].

If X is deterministic then the computation of a global quantity of interest can be treated by the so-called Collaborative Optimization methods [START_REF] Braun | Development and application of the collaborative optimization architecture in a multidisciplinary design environment[END_REF][START_REF] Yao | Review of uncertainty-based multidisciplinary design optimization methods for aerospace vehicles[END_REF] in Multidisciplinary Analysis and Optimization.

However, if X is random, a direct Monte Carlo evaluation of QI is often impossible to implement in practice. Indeed, if the number of interacting disciplines is large and each code evaluation is costly, then one cannot wait for a sample X 1 , . . . , X n to be generated by the upstream simulations before starting running one's own simulation. Therefore, decomposition-based UQ methods have been introduced in the literature in order to allow disciplines to run their numerical simulations independently. Basically, these methods work in two phases. In an offline phase, each discipline generates it own synthetic sample X ′ 1 , . . . , X ′ m according to some user-chosen probability measure µ X ′ on R d (or possibly other designs of experiment). The numerical model f is then evaluated on the sample (X ′ 1 , Θ 1 ), . . . , (X ′ m , Θ m ) to obtain a corresponding set of realizations Y ′ 1 , . . . , Y ′ m . Once actual realizations X 1 , . . . , X n become available in a subsequent online phase, they have to be used in combination with the synthetic sample to construct an estimator of QI, but evaluations of the numerical model f are no longer allowed.

We refer to [START_REF] Amaral | A decomposition approach to uncertainty analysis of multidisciplinary systems[END_REF][START_REF] Amaral | A decomposition-based approach to uncertainty analysis of feed-forward multicomponent systems[END_REF][START_REF] Amaral | Optimal L2-norm empirical importance weights for the change of probability measure[END_REF] for examples and background on these methods. The k-NN reweighting scheme introduced in the present article is yet another possible approach to this problem. More general nonparametric regression methods, such as Nadaraya-Watson estimators, may also be considered. A more systematic study of such approaches, based on linear reweighting, as well as their generalization to the estimation of quantities of interest defined on a graph of numerical models, may be found in [START_REF] Touboul | Model of margin, margin sensitivity analysis and uncertainty quantification in graphs of functions in complex industrial systems[END_REF]Chapter 11] and will be the object of a future publication. Remark 5.2 (Stochastic simulators). Our framework is also suited to the situation where Θ does not represent well-identified parameters, but must rather be interpreted as the inherent randomness of the numerical model. In the UQ literature, such models are called stochastic simulator (see for instance [START_REF] Zhu | Global sensitivity analysis for stochastic simulators based on generalized lambda surrogate models[END_REF] and the references therein) and their emulation is closely related with the regression problem addressed in this article, interpreting Θ as a noise term. Remark 5.3 (A simple example with low-dimensionally supported data). In the multidisciplinary context introduced above, consider the simple setting in which a random variable X 1 is taken as an input by two distinct disciplines, represented by two numerical models f 1 and f 2 . Assume in addition that the output

Y 1 = f 1 (X 1
) of the former is taken as an input by the latter, so that Y 2 = f 2 (X 1 , Y 1 ). In the decomposition-based approach, the second discipline has to design a synthetic sample (X ′1 j , Y ′1 j ) j∈ 1,m without the actual knowledge of f 1 . Therefore it is unlikely that this sample be absolutely continuous with respect to the true law of (X 1 , Y 1 ), the support of which lies in the manifold {(x, f 1 (x))}. on the segment {(u, u), u ∈ [0, 1]}. We can see that the greater s corr is, the better the reconstruction looks like. From a more quantitative point of view, this observation is confirmed in Figure 3, where we plot the evolution of E[W 2 2 ( µ Xn , µ ) is computed (with n = 100) thanks to the formula [START_REF] Brécheteau | A k-points-based distance for robust geometric inference[END_REF], and its expectation is estimated through Monte Carlo average over 700 replications. The theoretical rate of m -q/d = m -1 is attained for each experiment.

  of QI. While taking k = 1 is optimal for the convergence of µ w (k) m X ′ m to µ Xn , one may expect from the theory of NN regression that the estimator QI (k) m,n display better convergence properties if k is chosen to grow to infinity with m. Therefore we shall study both regimes k = 1 and k = k m → +∞.

  the k-th Nearest Neighbor (k-NN) of x among the sample X ′ m , that is to say the k-th closest point to x among X ′ 1 , . . . , X ′ m for the norm | • |. If there are several such points, we define NN (k) X ′ m (x) to be the point X ′ j

Lemma 3 . 12 (

 312 Radial density -Sufficient condition for Assumption 3.6). Let • be a norm on R d , induced by an inner product and not necessarily identical to | • |. If µ X ′ has a density p X ′ with respect to the Lebesgue measure on R d , which writes p X ′ (x) = h( xx 0 ) for some x 0 ∈ R d and h : [0, +∞) → R continuous, positive and nonincreasing, then Assumption 3.6 holds with U = R d .

1 -

 1 NN among the subset j. By the definition of NN (l) X ′ m

4 . 1 .

 41 to QI This section is dedicated to the study of the convergence of QI (k) m,n to QI. As a preliminary step, we complement the results from Section 3 by deriving rates of convergence for the Wasserstein distance between µ (k) X ′ m and µ X in Subsection 4.1. We then distinguish between the noiseless case in which Y = f (X), addressed in Subsection 4.2, and the noisy case Y = f (X, Θ), addressed in Subsection 4.3. Convergence of µ (k) X ′ m to µ X . Let us fix q ∈ [1, +∞) and use Jensen's inequality to write, for

Proposition 4 . 3 (

 43 Rates of convergence in the noiseless case). Assume that:(i) the function f does not depend on Θ, (ii) the function φ • f is globally Lipschitz continuous,

Proposition 4 . 4 (

 44 Consistency of the 1-NN in the noisy case). Assume that: (i) the function φ is bounded, (ii) the function ψ is globally Lipschitz continuous, (iii) the function ϑ is continuous, and let the assumptions of Theorem 3.3 hold. We have E QI (1)

Proposition 4 . 5 (

 45 Consistency in the noisy case). Assume that (i) the function φ is bounded, (ii) the function ψ is globally Lipschitz continuous, and let the assumptions of Theorem 3.3 hold with q = 2. As soon as k m goes to infinity with m and k m /m → 0, we have

5. 1 .

 1 Generalization error of k-NN regression under covariate shift. In this subsection, we address the k-NN regression problem under covariate shift from the following more standard point of view: the quantity of interest is directly the regression function r(x) :

(c) s corr = 0. 9 FIGURE 1 .Figure 2

 912 FIGURE 1. Plot of the support of X and 1500 i.i.d realizations of X ′ for different values of s corr .

5 . 3 . 2 .

 532 a function of m. We can see that although this quantity converges at the theoretical rate m -1 , the multiplicative constant decreases with s corr . Influence of µ X ′ on the convergence of QI (km) m,n . We now concentrate on the impact on the efficiency of QI (km) m,n . We keep the framework of the previous paragraph, and we try to estimate the quantity of interestQI = E[φ(f (X, Θ))], f ((x 1 , x 2 ), θ) = sin(2πx 1 ) sin(2πx 2 )(1 + θ) with Θ ∼ U ([-1, 1]) and φ(y) = y. The L 2 error E QI (km) m,n -QI 2 1/2 = E QI (km) m,n -0.5 2 1/2is computed by Monte Carlo estimation. As highlighted in Figure4, the closeness of µ X to µ X ′ is an important factor for the efficiency of the estimator.

FIGURE 2 .

 2 FIGURE 2. Trace of a kernel smoothing of µ (1) X ′ m on the segment {(u, u), u ∈ [0, 1]} for different values of s corr .

FIGURE 3 .

 3 FIGURE 3. Evolution of E[W 2 2 ( µ Xn , µ(1)X ′ m)] with respect to m, for various values of s corr . For each value of m and s corr , the quantity W 2 2 ( µ Xn , µ(1)X ′ m

FIGURE 4 .

 4 FIGURE 4. Estimation of the L 2 error with respect to s corr for n = m = 900 and k m = 4, averaged on 2000 replications.

  2. Comments on Proposition 2.2. In this subsection, we discuss the relation between the result of

	Proposition 2.2 and other fields in numerical probability and statistical learning, as well a generalization
	of this result to a more general framework.
	2.2.1. Link with optimal quantization. It is clear from Proposition 2.2 that µ	(1) X ′ m

Acknowledgements. This work was motivated by a collaboration with P. Benjamin, F. Mangeant and M. Yagoubi. We also benefited from fruitful discussions with G. Biau and A. Guyader. Last, we thank two anonymous referees for their careful reading of the article, and their numerous suggestions which allowed to greatly improve the presentation of this work.

This research work has been carried out under the leadership of the Technological Research Institute SystemX, and therefore granted with public funds within the scope of the French Program "Investissements d'Avenir".

5.3. Numerical illustrations. This subsection investigates numerically the influence of the choice of the synthetic distribution µ X ′ on the quality of the respective approximations of µ X by µ . We investigate how the relationship between µ X and µ X ′ impacts the convergence of µ

with µ = 0.5, σ = 0.3 and various s corr in (-1, 1). Intuitively, the closer s corr is to 1, the closer µ X ′ is to µ X , as illustrated in Figure 1.