Arthur Stéphanovitch
email: stephanovitch@dma.ens.fr

Clément Eddie Aamari
email: eddie.aamari@ens.fr

Clément Levrard
email: clement.levrard@univ-rennes1.fr

Adversarial Networks are Minimax Optimal Distribution Estimators. 2023. hal-04315916 WASSERSTEIN GENERATIVE ADVERSARIAL NETWORKS ARE MINIMAX OPTIMAL DISTRIBUTION ESTIMATORS

Keywords: Wasserstein Generative Adversarial Networks are Minimax Optimal Distribution Estimators Primary 62G05, secondary 62E17 Minimax rate, generative model, distribution estimation, manifold, interpolation inequality. 1 Primary 62G05, secondary 62E17 Minimax rate, generative model, distribution estimation, manifold, interpolation inequality

published or not. The documents may come

1. Introduction. Let X 1 , ..., X n be i.i.d. random points drawn from a probability measure µ with support in R p . The inference of µ is a fundamental problem in statistics and machine learning, for which numerous methods have been developed [START_REF] Alexandre | Introduction to nonparametric estimation[END_REF]. Recent years have witnessed the advent of generative methodologies based on Generative Adversarial Networks (Goodfellow et al., 2014), with outstanding achievements in the fields of image [START_REF] Karras | Aliasfree generative adversarial networks[END_REF], video (Vondrick et al., 2016), and text generation [START_REF] Yu | SeqGAN: Sequence generative adversarial nets with policy gradient[END_REF]. In this paper we focus on the Wasserstein GAN (WGAN) approach of [START_REF] Arjovsky | Wasserstein generative adversarial networks[END_REF], which uses the 1-Wasserstein distance as an alternative to the Jensen-Shannon divergence implemented in traditional GAN. Over the years, WGAN and their derivatives have gained popularity in the machine learning community. They are today considered as one of the most successful generative techniques, achieving state-of-the art results in difficult problems [START_REF] Karras | Aliasfree generative adversarial networks[END_REF] while improving the stability and getting rid of unpleasant issues such as mode collapse (Gulrajani et al., 2017). Although WGANs have shown excellent properties in numerous empirical studies reported in the machine learning literature [START_REF] Liu | Wasserstein gan-based smallsample augmentation for new-generation artificial intelligence: A case study of cancer-staging data in biology[END_REF], [START_REF] Luo | Eeg data augmentation for emotion recognition using a conditional wasserstein gan[END_REF], [START_REF] Stanczuk | Wasserstein gans work because they fail (to approximate the wasserstein distance[END_REF]), many of their theoretical properties remain to be studied.

The present work establishes the minimax optimality of the WGAN estimator. To set the scene and notation, the generative problem is to use the data X 1 , ..., X n to learn µ and, simultaneously, be able to sample from a distribution close to it. Aiming to solve this problem, a generative adversarial network consists of a class of generator functions G and a class of discriminators D. Given a known easy-to-sample distribution ν on a latent space Z, the generator G ∋ g : Z → R p approximates µ by trying to minimize over G a certain Integral Probability Metric (IPM) [START_REF] Mroueh | Integral probability metrics and their generating classes of functions[END_REF]:

(1.1) d D (µ, g #ν ) := sup D∈D E µ [D(X)] -E ν [D(g(Z))],
where g #ν stands for the pushforward measure of ν by g. The goal of the discriminator D ∋ D : R p → R is to distinguish between the true distribution and the fake one g #ν , by maximizing over D the quantity

L(g, D) := E µ [D(X)] -E ν [D(g(Z))].
The Generative Adversarial Networks min-max problem can then be written as

(1.2) inf g∈G sup D∈D E µ [D(X)] -E ν [D(g(Z))].
One can see the class D as a subset of a larger class F , with d F being a metric over distributions. Various types of classes F have been used in the GAN literature. This includes Lipschitz continuous function (WGAN, [START_REF] Arjovsky | Wasserstein generative adversarial networks[END_REF]), Sobolev functions (Sobolev GAN, [START_REF] Mroueh | Integral probability metrics and their generating classes of functions[END_REF]) and reproducing kernel Hilbert space (MMD GAN, [START_REF] Li | Mmd gan: Towards deeper understanding of moment matching network[END_REF]). These discrepancies are to be contrasted with the more historically used ones in classical non parametric density estimation such as L p distance, Hellinger distance and Kullback-Leibler divergence [START_REF] Alexandre | Introduction to nonparametric estimation[END_REF], which are only applicable when the model is dominated. In the present paper, we work in a general setting where the target measure µ can have a low dimensional structure which makes these usual discrepancy measures irrelevant. We choose the discriminative class F to be the Hölder class H γ 1 (R p , R) corresponding to the unit ball of functions of smoothness γ ≥ 1. Note that the case γ = 1 is equivalent to the Wasserstein case when µ has compact support.

Optimal rates of estimation of a measure µ with support in R p are known to decay exponentially as the ambient dimension p increases. To overcome this curse of dimensionality, some low-dimensional structural assumptions on µ need to be imposed. In this work, we suppose that there exists a map g ⋆ ∈ H β+1 K (T d , R p ) with T d the d-dimensional torus, such that µ = g ⋆ #U with U ∼ U([0, 1] d ) uniform random variable on the cube. In particular, there exist Y 1 , ..., Y n i.i.d. such that Y i ∼ U([0, 1] d ) and g ⋆ (Y i ) = X i . Note that the Y i are unknown, we only have access to the X i . In this context, the inference problem consists in trying to find an estimator μ of µ = g ⋆ #U based on the sample (X i ) i=1,...,n , such that the expected error

E Xi∼g ⋆ #U [d H γ 1 (g ⋆ #U , μ(X 1 , ..., X n ))
], is as small as possible. The estimator μ is said to be minimax optimal if there is no estimator that achieves a better rate of convergence uniformly over the model. Formally, it means that there exists a constant C > 0 independent of n such that

sup g ⋆ ∈H β+1 K E Xi∼g ⋆ #U [d H γ 1 (g ⋆ #U , μ(X 1 , ..., X n ))] ≤ C inf T sup g ⋆ ∈H β+1 K E Xi∼g ⋆ #U [d H γ 1 (g ⋆ #U , T (X 1 , ..., X n ))].
In [START_REF] Divol | Measure estimation on manifolds: an optimal transport approach[END_REF], the author provides a (non-generative) minimax estimator of densities in the manifold setting for the Wasserstein distance. It uses a local polynomial manifold estimator from [START_REF] Aamari | Non-asymptotic rates for manifold, tangent space, and curvature estimation[END_REF] coupled with a kernel density estimator. However, the local polynomial estimator is very computationally expensive and therefore cannot be used in high dimension. In [START_REF] Tang | Minimax rate of distribution estimation on unknown submanifold under adversarial losses[END_REF], the author also provide a minimax estimator of densities in the manifold setting but for the distance d H γ 1 . The estimator uses a regularization of the empirical measure µ n = 1 n n i=1 δ Xi with a manifold estimator coupled with a truncated wavelet regularization. This estimator is purely theoretical and would be extremely computationally expensive to be implemented in practice. Therefore, the existence of a tractable minimax estimator is still an open question and is crucial to be answered in order to provide efficient tools for concrete applications.

As the Wasserstein GAN [START_REF] Arjovsky | Wasserstein generative adversarial networks[END_REF] approach has been proven to be easily implementable and provide state-of-the-art results in various field, we focus in this paper on the GAN estimator

(1.3) ĝ ∈ arg min g∈G sup D∈D 1 n n i=1 D(X i ) -D(g(U i ))
for

U i ∼ U([0, 1] d ) i.i.d., G ⊂ H β+1 K (T d , R p ) and D ⊂ H γ 1 (R p , R) with γ ∈ [1, β + 1]
. The estimator ĝ is to be understood as an empirical approximation of the solution of (1.2) based on the data X 1 , ..., X n . The probability measures ĝ#U is then naturally our estimator of the target g ⋆ #U . Recall that as D ⊂ H γ , the case γ = 1 corresponds to the classical WGAN estimator. One of the strengths of the GAN estimator (1.3) is that it carries out both support and density estimations at the same time, which allows in particular to avoid the use of any manifold estimator like in [START_REF] Divol | Measure estimation on manifolds: an optimal transport approach[END_REF] and [START_REF] Tang | Minimax rate of distribution estimation on unknown submanifold under adversarial losses[END_REF].

Main contributions. We build tractable classes of neural networks G and D, such that (1.4) sup

g ⋆ ∈H β+1 K E Xi∼g ⋆ #U [d H γ 1 (g ⋆ #U , ĝ#U )] ≤ C 1 (log n) C2 n -β+γ 2β+d ∨ n -1 2 ,
with C 1 , C 2 > 0 some constants independent of n. This rate has been proven to be minimax optimal in [START_REF] Tang | Minimax rate of distribution estimation on unknown submanifold under adversarial losses[END_REF] up to the logarithmic factors. To the best of our knowledge, this is the first result showing that the GAN estimator attains minimax rates for the Wasserstein/Hölder distances. This result improves, in particular, on the rates obtained in [START_REF] Chen | Distribution approximation and statistical estimation guarantees of generative adversarial networks[END_REF], [START_REF] Schreuder | Statistical guarantees for generative models without domination[END_REF] and Chae (2022). Minimax convergence rates of the vanilla version of the GAN (Goodfellow et al., 2014), have recently been obtained by Belomestny et al. (2023) for the Jensen-Shannon divergence. Their result only deals with the full dimensional setting d = p, as the Jensen-Shannon divergence is only non-trivial when the measures to be compared are not singular with respect to each other. We obtain the minimax rate of convergence (1.4) over measures µ = g ⋆ #U that have a density with respect to the volume measure of an unknown submanifold. In the process, minimax rates are also obtained for two intermediate statistical models:

• (Model 1, Theorem 4.1) We first treat a general low dimensional setting where the target measure may have atoms and its support is not necessarily a manifold. In this case, the class of discriminators D used is theoretical, meaning it is not computable in practice. This model is studied to discuss the limitation of the assumptions of the general case. • (Model 2, Proposition 4.7) We prove minimax rates in the full dimensional setting p = d, where the target measure has a density with respect to the p-dimensional Lebesgue measure. This case is treated to understand in the easier setting of full dimension, how additional assumptions can help us obtain a tractable estimator.

• (Model 3, Theorem 5.8) Adapting the method developed in the full dimensional case to the submanifold case, we propose a tractable GAN estimator achieving minimax rates for all γ ∈ [1, β + 1] simultaneously.

The main result is proven using a new interpolation inequality (Theorem 5.1) that bounds the distance d H γ 1 (g #U , g ⋆ #U ) by the quantity d H β+1 1 (g #U , g ⋆ #U ) β+γ 2β+1 up to logarithmic factors. A detailed overview of the main results is provided in Section 2.2.

Organization of the paper. In Section 2, we set the notation, properly state our main results, and compare our contributions to the existing works. In Section 3, we introduce premilinary tools that we will use to show the minimax optimality of the GAN. In Section 4, we gain some insights by treating the general low dimensional setting and the full dimensional setting of Models 1 and 2. Finally, in Section 5, we treat the low dimensional submanifold setting of Model 3. Technical points of the proofs are gathered in the supplementary material.

2. Overview of the main results and comparison with other works.

2.1. Notation. Let n ≥ 1 be the sample size, p ≥ 2 be the dimension of the ambient space, d ∈ {1, ..., p} the intrinsic dimension of the data, β ≥ 1 the regularity of the data, γ ∈ [1, β + 1] the regularity of the discriminator and K > 1 a bounding constant for several parameters to come.

We write ⟨•, •⟩ the canonical dot product on R p , ∥x∥ the Euclidean norm of a vector x ∈ R p and B p (0, K) the p-dimensional ball of radius K. Let N 0 = {0, 1, ..., } be the set of nonnegative integers. For η > 0, X , Y two subsets of Euclidean spaces and f = (f 1 , ..., f p ) ∈ C ⌊η⌋ (X , Y) the set of ⌊η⌋ := max{k ∈ N 0 | k ≤ η} times differentiable functions, denote

∂ ν f i = ∂ |ν| fi ∂x ν 1 1 ...∂x
H η K (X , Y) = f ∈ C ⌊η⌋ (X , Y) max i |ν|≤⌊η⌋ ∥∂ ν f i ∥ L ∞ (X ,Y) + |ν|=⌊η⌋ ∥∂ ν f i ∥ η-⌊η⌋ ≤ K
denote the ball of radius K of the Hölder space H η (X , Y), the set of functions f : X → Y of regularity η. To lighten the notation we will write H γ 1 instead of H γ 1 (R p , R) and H β+1 K instead of H β+1 K (T d , R p ) when the context is clear. For a uniform random variable U ∼ U([0, 1] d ) on the d-dimensional unit cube and g ∈ H β+1 K (T d , R p ), we denote by g #U the push forward measure of U by g, i.e. ∀f ∈ H 0 1 (R p , R),

R p f (x)dg #U (x) := [0,1] d f (g(u))dλ d (u),
with λ d being the d-dimensional Lebesgue measure. For a d-dimensional submanifold M of R p and ϵ > 0, we denote by M ϵ := {x ∈ R p | d(x, M) < ϵ} the set of points such that their distance to the manifold d(x, M) := inf y∈M ∥x -y∥, is less than ϵ. The reach r M (Federer, 1959) of M corresponds to the largest ϵ ≥ 0 such that its orthogonal projection π M is well defined from M ϵ to M. The projection on T x (M) the tangent space at the point x ∈ M, will be denoted by P Tx(M) . We will write M f (x)dλ M (x) the integral of a function f : M → R with respect to the volume measure on M, i.e. the d-dimensional Hausdorff measure.

Given a differentiable map f : R k → R l we denote by ∇f its differential and by ∥∇f (x)∥ its operator norm at x. We denote by (∇f (x)) ⊤ its transpose matrix. If k ≤ l, λ min ((∇f (x)) ⊤ ∇f (x)) corresponds to the smallest eigenvalue of the matrix (∇f (x)) ⊤ ∇f (x).

The distance we use to measure discrepancy between two probability measures µ, ν on R p is the Hölder Integral Probability Metric (Müller, 1997) :

(2.1)

d H γ 1 (µ, ν) := sup D∈H γ 1 E X∼µ,Y ∼ν [D(X) -D(Y )].
In the case where γ = 1, the Wasserstein and the d H 1 1 distances are equivalent on compact domains. This can be shown using the dual formulation of the 1-Wasserstein distance W 1 [START_REF] Villani | Optimal transport: old and new[END_REF]: for two probability measures µ, ν with support in the ball B p (0, K), we have

d H 1 1 (µ, ν) ≤ W 1 (µ, ν) ≤ (2K + 1)d H 1 1 (µ, ν).
Therefore, in the case where γ = 1 and the discriminator class D is a subset of H 1 1 , the GAN estimator (1.3) is called the Wasserstein GAN estimator. We will denote |A| the cardinal of a set A. For a, b ∈ R, a ∧ b and a ∨ b will denote the minimum and maximum value between a and b respectively. Throughout, letters C, C 2 , C 3 ... denote constants that can vary from line to line and that only depend on p, d, β and K.

2.2. Overview of the main results. As mentioned above, the goal of the present paper is to build a tractable GAN estimator (1.3) that attains minimax rates for the distance d H γ 1 with γ ∈ [1, β + 1]. We cover a total of three different statistical models. The first two models are studied in order to get insights on how to construct a minimax and tractable estimator in the third model, as well as to highlight the key technical points to do so. Having gathered insights from these analysis, we obtain the desired result under the assumptions of the third model. 2.2.1. A (too) general low dimensional model. The first model allows the target measure to have an intrinsic low dimensional structure as well as a non smooth support. MODEL 1. There exists g ⋆ ∈ H β+1 K (T d , R p ) such that the target measure µ verifies µ = g ⋆ #U .

Supposing that µ is the push-forward measure of a low dimensional space by a smooth map is a classical assumption in the GAN literature (Chae (2022), [START_REF] Schreuder | Statistical guarantees for generative models without domination[END_REF]). The latent space is usually the cube [0, 1] d or the whole R d space when the target measure is sub-Gaussian. Like in [START_REF] González-Prieto | Dynamics of fourier modes in torus generative adversarial networks[END_REF], we use the torus T d = R d /Z d for the latent space so that the target measure g ⋆ #U is compactly supported and we can use harmonic analysis tools such as wavelets (Giné and Nickl, 2015) to describe g ⋆ . Model 1 is very general as it does not have assumptions on the smoothness of support of µ besides that it is compact and connected. Under these assumptions, we obtain in Section 3 the following result.

Theorem 4.1 For µ verifying the assumptions of Model 1 and γ ∈ [1, β + 1], there exist theoretical classes of functions G and D γ such that the GAN estimator ĝ of (1.3) verifies

E d H γ 1 (ĝ #U , µ) ≤ C log(n) C2 n -β+γ 2β+d ∨ n -1 2 .
Theorem 4.1 asserts that the GAN estimator is minimax optimal up to the logarithmic factor for the distance

d H γ 1 with γ ∈ [1, β + 1].
To the best of our knowledge, this is the first result showing that the GAN is minimax for that kind of distance. In particular this proves that the Wasserstein GAN (γ = 1) is also minimax optimal.

The minimax rate is attained in the setting of general push forward maps which allows the target measure to have a low dimensional structure and therefore, be singular to the Lebesgue measure of the ambient space. Furthermore, the only assumption being the regularity of the map g ⋆ , it even allows the target measure g ⋆ #U to be singular to the d-dimensional Hausdorff measure. Indeed, g ⋆ #U could have some parts that have finite k-dimensional Hausdorff measure for any k ∈ {0, ..., d}, as for example with g ⋆ (x) = (x 1 , ..., x k , 0, ..., 0). On the other hand, the support of g ⋆ #U could have some strong irregularities like corners or intersections. However, the discriminator class D γ we use in Theorem 4.1 to estimate µ in Model 1 is not tractable. To use a tractable class, we will suppose that µ has a smooth density with respect to the volume measure of a submanifold. To get some insights from a simple setting, we first treat the "full dimensional case" where the submanifold is the whole ambient space R p . 2.2.2. A full dimensional density-based model. In the full dimensional case d = p, we focus on measures that satisfy the following assumptions.

MODEL 2. The target measure µ is absolutely continuous with respect to the pdimensional Lebesgue measure, is compactly supported in B p (0, K) and its density f ⋆ belongs to

H β K (R p , R).
Note that the densities of push forward measures g #U with U ∼ U([0, 1] p ) are discontinuous on the boundary of their support. Therefore, measures in Model 2 cannot be push forward of the Lebesgue measure on the cube [0, 1] p . In the density case, we hence use an estimator that is different from the GAN estimator (1.3). Given n i.i.d. data X i ∈ R p of law f ⋆ , the adversarial density estimator is defined as

(2.2) f ∈ arg min f ∈F max D∈D n i=1 D(X i ) -D(U i )f (U i ),
for U i ∈ R p i.i.d. uniform random variables on the ball B p (0, K). Note that this is not the typical GAN estimator where usually the samples of candidate densities f ∈ F , are generated by push forward distributions. Such a GAN-like formulation, which uses densities as generators, has already been proposed by [START_REF] Liang | How well generative adversarial networks learn distributions: A nonparametric view[END_REF] and [START_REF] Singh | Nonparametric density estimation under adversarial losses[END_REF]. The drawback of this method is that it does not provide a direct way to sample from the approximate distribution f . We do not develop on the generation from the estimator (2.2) as this estimator is only a by-product of the method we developed to solve the low dimensional case of Model 3. We obtain in Section 4.2 the following result. Proposition 4.7: For µ verifying the assumptions of Model 2, there exist tractable neural network classes F and D such that the density adversarial estimator f of (2.2), verifies for all γ ∈ [1, β] simultaneously:

E d H γ 1 ( f , µ) ≤ C log(n) C2 n -β+γ 2β+p ∨ n -1 2 .
Proposition 4.7 asserts that the adversarial density estimator of 2.2 also attains minimax rates (up to the logarithmic factor) for the classical density estimation problem.

2.2.3.

A non-degenerate manifold model. We now come back to the low dimensional case and define the assumptions we add to Model 1, so that we can use tractable classes of generators and discriminators for the GAN estimator

(1.3). DEFINITION 2.1. Let g ∈ H β+1 K (T d , R p ). i)
The map g is said to verify the K-manifold regularity condition if it is injective and its image M g is a d-dimensional submanifold with reach r g greater than K -1 . ii) The map g is said to verify the K-density regularity condition if the differential of g verifies

inf u∈[0,1] d λ min ((∇g(u)) ⊤ ∇g(u)) 1/2 ≥ K -1 .
Using this definition, we define the central model of the paper, under which we obtain a tractable and minimax estimator of measures having a low dimensional structure. MODEL 3. There exists g ⋆ ∈ H β+1 K (T d , R p ) that verifies the K-manifold and K-density regularity conditions such that the target measure µ verifies µ = g ⋆ #U .

A target measure µ satisfying the assumptions of Model 3 has a support M g ⋆ = g ⋆ (T d ) that is a d-dimensional manifold. Furthermore, it has a density with respect to the volume measure on M g ⋆ and this density belongs to H β C (M g ⋆ , R). On the other hand, as M g ⋆ always has empty boundary (in the manifold sense), Model 3 is non-empty only when d < p. Under these assumptions, we obtain in Section 5.2 the main result of the paper.

Theorem 5.8 For µ verifying the assumptions of Model 3, there exist tractable neural network classes G and D such that the GAN estimator ĝ of (1.3) verifies for all γ ∈ [1, β + 1] simultaneously:

E d H γ 1 (ĝ #U , µ) ≤ C log(n) C2 n -β+γ 2β+d ∨ n -1 2 .
To the best of our knowledge, the GAN estimator (1.3) is the first tractable estimator attaining minimax rates up to logarithmic factors in the manifold setting. Furthermore, it is also the first one being minimax for several IPMs simultaneously. Indeed, as in the other comparable results in the literature, Theorem 4.1 provides a minimax estimator ĝγ that is different for each a γ ∈ [1, β + 1] as it uses a discriminator class D γ ⊂ H γ 1 . In contrast, Theorem 5.8 provides a single GAN estimator ĝβ+1 computed with a class D β+1 , that attains minimax rates for all the distances d H γ 1 , γ ∈ [1, β + 1] simultaneously, up to logarithmic factors. To obtain this uniform minimax optimality, we show an interpolation inequality that allows to compare the different IPMs. That is, we obtain in Section 5.1 the following result.

Theorem 5.1 Let g, g ⋆ ∈ H β+1 K (T d , R p ) verifying the K-manifold and K-density regularity conditions. There exists constants

C, C 2 > 0 such that if d H β+1 1 (g #U , g ⋆ #U ) ≤ C -1
, then for all ϵ ∈ (0, 1) and γ ∈ [1, β + 1], we have

d H γ 1 (g #U , g ⋆ #U ) ≤ C 2 log(ϵ -1 ) 4 d H β+1 1 (g #U , g ⋆ #U ) β+γ 2β+1 + ϵ .
This new result of functional analysis allows to show that if an estimator is minimax for the distance d H β+1 1 , then it will also be minimax for the distances d H γ 1 with γ ∈ [1, β + 1] up to logarithmic factors. Theorem 5.1 requires the functions g and g ⋆ to verify the Kmanifold and K-density regularity conditions. This is the reason why we add these extra assumptions in Model 3 compared to Model 1. The proof of the Theorem strongly relies on wavelet theory (Giné and Nickl, 2015). All along the paper, we try to show how wavelets provide a key tool to describe Hölder regularity and therefore are perfectly suited for the estimation of measures with smoothness properties.

2.3.

Comparison with other works. Let us compare more extensively our results with other works on GAN and minimax estimators. Specifically we focus on:

• The settings in which the results are obtained • The rates of convergence of the error up to logarithmic factors • The computational tractability of the methods Our result. We treat two different cases: the general setting (Model 1) where µ is not necessarily supported on a manifold and can have atoms, and the setting where µ has a density with respect to the volume measure of a submanifold (Models 2 and 3).

In both settings we obtain the minimax optimal rate of O(n -β+γ 2β+d ∨ n -1 2 ) where n is the sample size, β determines the smoothness of the target measure µ, γ is the smoothness of the discriminator class and d ≤ p is the intrinsic dimension of µ. In the first setting the class D used for the GAN estimator (1.3) is theoretical, meaning that it is not computable in practice. In the second setting, we provide tractable classes of neural networks which lead to a single GAN being minimax optimal for all γ ∈ [1, β + 1] simultaneously.

GAN estimators.

Let us first compare our result with the already existing rates obtained for the GAN estimator in the Hölder/Wasserstein discrimination setting.

• In [START_REF] Chen | Distribution approximation and statistical estimation guarantees of generative adversarial networks[END_REF], assuming that the support of the target measure is convex, the authors treat the full dimension case p = d and the case where the target measure is supported on a linear subspace case. They obtain the rate O(n

-γ 2γ+p ∨ n -β+1 2(β+1)+p ) (for the distance d H γ
1 with γ ≥ 1) in the full dimension case and the rate O(n -1 2+d ) (for the Wasserstein distance W 1 ) in the linear subspace case. These rates are unfortunately always slower than the minimax rates O(n -β+γ 2β+p ∨ n -1 2 ) and O(n -1 d ) respectively. Their method is tractable, and uses ReLU feedforward neural networks.

• In [START_REF] Schreuder | Statistical guarantees for generative models without domination[END_REF], the authors treat the case β +1 = γ, in noisy settings and suppose that the target measure is supported on a manifold. They provide the first result assessing the influence of the noise and of the contamination on the error of generative Modeling. They obtain the rate O(n

-β+1 d ∨ n -1 2 ) which is always strictly slower than the minimax rate O(n -2β+1 2β+d ∨ n -1 2 ) when β + 1 < d/2.
They do not explicit the class G in their method and the class D is taken as the whole H γ 1 , which is not tractable. • In Chae (2022), the authors treat the Wasserstein case γ = 1. They obtain the rate O(n -β 2β+d ) which is always strictly slower than the minimax rate O(n -β+1 2β+d ). They use a set of neural networks for the class G, but use the set of optimal discriminators in H 1 1 between functions in G, as the class D. We use the same discriminator class in our first setting but this class of functions is way to expensive to compute in practice.

In a setting extending all these 3 works, our results show that the GAN estimator can be minimax for properly chosen classes G and D.

Minimax rates. Let us now compare our work with estimators that are not the GAN estimator but that attain the minimax optimal rate.

• In [START_REF] Liang | How well generative adversarial networks learn distributions[END_REF], the authors treat the full dimensional case p = d with Sobolev regularity instead of Hölder. They obtain the minimax rate of convergence O(n

-β+γ 2β+p ∨ n -1 2 ) for the Sobolev discrimination metric d W γ 1 , γ ∈ N 0 with W γ
1 the unit ball of the Sobolev space W γ . They provide the first formalization of nonparametric estimation under the adversarial framework. The estimator used is adversarial but is not the GAN estimator as in the GAN loss (1.3), it does not use samples of the generator and uses a regularization of the empirical measure. The discriminator class is the whole Sobolev class W γ 1 , which is not computable in practice. Furthermore, they compute integrals with respect to the pdimensional Lebesgue measure, which is not tractable for large ambient dimension p.

• In [START_REF] Divol | Measure estimation on manifolds: an optimal transport approach[END_REF], the author treats the Wasserstein case γ = 1 when the measure admits a β-regular density with respect to the volume measure of a d-dimensional closed (compact boundaryless) manifold. They propose the first estimator attaining the minimax rate of convergence O(n -β+1 2β+d ) in the manifold case. The estimator uses a local polynomial estimator from [START_REF] Aamari | Non-asymptotic rates for manifold, tangent space, and curvature estimation[END_REF] that is not tractable for large ambient dimension p.

• In [START_REF] Tang | Minimax rate of distribution estimation on unknown submanifold under adversarial losses[END_REF], the authors treat the case where the measure admits an α-regular density with respect to the volume measure of a d-dimensional β-regular closed submanifold (β ≥ α -1). They obtain the minimax rate of convergence O(n

-α+γ 2α+d ∨ n -βγ d ∨ n -1 2 ) for γ ∈ (0, ∞).
In particular they cover the case γ ∈ (0, 1) which sheds light on the fact that when γ is small, the distance d H γ 1 tends to be more sensitive to the misalignment between the supports of the measures than to the discrepancy between the densities on their respective supports. Although it is adversarial, the estimator is very different from the GAN. Indeed, for all

D ∈ H γ 1 (R p , R), they compute an intermediate estimator Ĵ (D) of E µ [D(X)], such that the quantity sup D∈H γ 1 E µ [D(X)] -Ĵ (D), achieves the minimax rate. Then, an adversarial estimator μ ∈ arg min ν∈S sup D∈H γ 1 E ν [D(Y )] -Ĵ (D) is com- puted.
As they suppose that the target measure µ belongs to the class S and the intermediate estimator Ĵ (D) already attains the minimax rate, then naturally, μ also attains this rate. However, the real estimation is realised through the computation of the intermediate estimator Ĵ (D), not the adversarial estimator μ. To compute the estimator Ĵ (D), they use a manifold estimator that is as computationally heavy as in [START_REF] Divol | Measure estimation on manifolds: an optimal transport approach[END_REF]. Furthermore, the estimator computes several infima over the whole Hölder class H β 1 , the wavelet expansion of each discriminator D ∈ H γ 1 , and computes all their derivatives up to the order γ. Unfortunately, all these steps make the estimator not tractable.

The fact that we can obtain minimax rates for the GAN estimator (1.3), shows in particular that it is not necessary to regularize the empirical measure in the GAN loss like in [START_REF] Liang | How well generative adversarial networks learn distributions[END_REF] and [START_REF] Tang | Minimax rate of distribution estimation on unknown submanifold under adversarial losses[END_REF]. Taking a discriminator class D much smaller than the Hölder functions, actually suffices to implicitly regularize the measures as this class cannot distinguish well between discrete and smooth measures. In particular, it makes our estimator tractable in contrast to these three works.

See Table 1 for a summary of the different results discussed previously.

, γ ≥ 1 ✓ ✗ ✓ Schreuder et al. (2021) g ⋆ #U , g ⋆ : [0, 1] d → R p d H γ 1 , γ = β ✓ ✗ ✗ Chae (2022) g ⋆ #U , g ⋆ : [0, 1] d → R p W 1 ✓ ✗ ✗ Liang (2021) density d W γ 1 , γ ∈ N 0 ✗ ✓ ✗ Divol (2021) density on a closed manifold Wp, p ≥ 1 ✗ ✓ ✗
Tang and Yang (2022)

density on a closed manifold d H γ 1 , γ > 0 ✗ ✓ ✗ Theorem 5.8 g ⋆ #U , g ⋆ : T d → R p d H γ 1 , γ ∈ [1, β + 1] simultaneously ✓ ✓ ✓ TABLE 1
Comparison of the results on the estimation of a target measure of regularity β.

3. Preliminary tools. This section gathers preliminary results used to build the minimax GAN estimators of the following sections. These tools are both of statistical nature (Section 3.1), and of analytical nature (Section 3.2).

3.1. Bias-variance trade-off for GANs. In this Section, we suppose that we are in the setting of Model 1 (which includes the setting of Model 3). We exhibit a general bound on the objective and give the required values of the approximation errors and covering numbers of the classes G and D, so that this bound provides minimax rates.

3.1.1. A general bound. Consider two closed sets of functions: a set of generators G ⊂ H β+1 K (T d , R p ) and a set of discriminators D ⊂ H γ 1 (B p (0, K), R). The approximation error of the class G is defined as

(3.1) ∆ G := inf g∈G d H γ 1 (g #U , g ⋆ #U ).
The error ∆ G corresponds to how well the class G approximates g ⋆ #U for the cost d H γ 1 . It is crucial for this error to be small since

∆ G ≤ E Xi∼g ⋆ #U d H γ 1 (ĝ #U , g ⋆ #U
) . On the other hand, the approximation error of the class D is defined as

(3.2) ∆ D := sup g∈G d H γ 1 (g #U , g ⋆ #U ) -d D (g #U , g ⋆ #U ) .
It corresponds to how well the class D approximates the discrimination made by H γ 1 between the measures g ⋆ #U and g #U , for all g ∈ G. As opposed to ∆ G , it is not crucial that this error be small. Indeed the role of the class D is only to associate a score d D (g #U , g ⋆ #U ) to each function g ∈ G, in order to find a good potential minimizer ĝ#U . However, this score does not have to be at the same scale as d H γ 1 (g #U , g ⋆ #U ). For example, we show in Section 5 that under suitable assumptions on g, g ⋆ we have

d H γ 1 (g #U , g ⋆ #U ) ≤ d H β+1 1 (g #U , g ⋆ #U ) β+γ 2β+1
up to logarithmic factors. Therefore, having a class D that well approximates H β+1 1 allows to obtain a minimax estimator although ∆ D is big in this case. Nevertheless, if ∆ D is small, it ensures that the discrimination done by the class D is relevant and therefore with high probability, ĝ is a good candidate among the class G.

Let us now define the notion of covering number which is the other feature of G and D that intervenes in the bound on the objective. For ϵ > 0, a minimal ϵ-covering F ϵ of a class of functions F is defined as:

(3.3) F ϵ := arg min{|A| | ∀f ∈ F, ∃f ϵ ∈ A, ∥f -f ϵ ∥ ∞ ≤ ϵ}.
The covering numbers |G ϵ | and |D ϵ | characterise the sizes of the classes of functions. The larger the classes are, the most likely the estimators ĝ and Dĝ (see Definition B.5) will overfit the data (X i ) i . The following result exhibits a classical bias/variance trade off between the approximation errors and covering numbers.

THEOREM 3.1. If g ⋆ ∈ H β+1 K (T d , R p ) and G ⊂ H β+1 K (T d , R p ), D ⊂ H γ 1 (B p (0, K), R), then the GAN estimator (1.3) verifies E d H γ 1 (ĝ #U , g ⋆ #U ) -∆ G -∆ D ≤ C min δ∈[0,1] (δ + 1/n) 2 log(n|G 1/n ||D 1/n |) n + 1 √ n (1 + δ (1-d 2γ ) + log(δ -1 )1 {2γ=d} ) .
The proof of Theorem 3.1 can be found in Section B.2.1. It gives a bound on the error of the estimator in terms of covering number and approximation errors of the classes G and D. It is a WGAN version of Theorem 1 in Belomestny et al. (2023), designed for Jensen-Shannon GANs.

3.1.2. Expected growth of optimal generators and discriminators classes. Let us discuss the values of the approximation errors and covering numbers that are required by Theorem 3.1 to achieve minimax optimality. From [START_REF] Tang | Minimax rate of distribution estimation on unknown submanifold under adversarial losses[END_REF], we know that the minimax convergence rate is O(n -β+γ 2β+d ∨ n -1 2 ). Therefore, if β+γ 2β+d < 1/2, in light of Theorem 3.1 we need to take δ ≥ Cn -γ 2β+d so that δ 1-

d 2γ √ n ≤ Cn -β+γ 2β+d
. Then, we also need

(3.4) log(|G 1/n |) ∨ log(|D 1/n |) ≤ Cn d 2β+d so that δ log(|G1/n||D1/n|) n ≤ Cn -β+γ 2β+d ∨ n -1 2
. Lastly, we need to take G and D such that

(3.5) ∆ G ∨ ∆ D ≤ C n -β+γ 2β+d ∨ n -1 2 .
On the other hand, we can prove that the covering number of a class of functions is lower bounded by its precision of approximation of the Hölder classes.

PROPOSITION 3.2. Let be η, ϵ > 0 and F ⊂ L 2 (T d , R) a class of functions such that sup

f ∈H η 1 inf g∈F ∥f -g∥ ∞ ≤ ϵ, then for all θ ∈ (0, ϵ/3) we have log(|F θ |) ≥ Cϵ -d η .
The proof of Proposition 3.2 can be found in Section B.2.2. From the previous proposition we see that although we need the approximation errors of the classes G and D to verify

∆ G ∨ ∆ D ≤ Cn -β+γ 2β+d ∨ n -1
2 in order to be minimax, we also need

(3.6) sup g ⋆ ∈H β+1 K inf g∈G ∥g ⋆ -g∥ ∞ ≥ Cn -β+1 2β+d
and sup

D ⋆ ∈H γ 1 inf D∈D ∥D ⋆ -D∥ ∞ ≥ Cn -γ 2β+d d p so that log(|G 1/n |) ∨ log(|D 1/n |) ≤ Cn d 2β+d
as desired. We are going to build classes of functions verifying these properties using a wavelet approximation produced by neural networks.

3.2. Wavelets: a key tool to describe regularity. In this section, we introduce the wavelet framework (Giné and Nickl, 2015) and use it to build the classes G and D of our GAN estimator in the setting of Model 1.

3.2.1. Wavelets, Besov spaces and regularity trade-off. Let ψ, ϕ ∈ H ⌊β⌋+3 (R, R) be a compactly supported scaling and wavelet function respectively (see (Daubechies, 1988) on Daubechies wavelets). For ease of notation, the functions ψ, ϕ will be written ψ 0 , ψ 1 respectively. Then for j ∈ N 0 , l ∈ {1, ..., 2 p -1}, w ∈ Z p , the family of functions

ψ 0w (x) = p i=1 ψ 0 (x i -w i ) , ψ jlw (x) = 2 jp/2 p i=1 ψ li (2 j x i -w i )
form an orthonormal basis of L 2 (R p , R) (with l i the ith digit of the base-2-decomposition of l). Let q 1 , q 2 ≥ 1, s > 0 and b ≥ 0 such that ⌊β⌋ + 3 > s. The Besov space B s,b q1,q2 (R p , R) consists of functions f : R p → R that admit a wavelet expansion in L 2 :

f (x) = w∈Z p α f (w)ψ 0w (x) + ∞ j=0 2 p -1 l=1 w∈Z p α f (j, l, w)ψ jlw (x), equipped with the norm ∥f ∥ B s,b q 1 ,q 2 = w∈Z p |α f (w)| q1 q2/q1 + ∞ j=0 2 jq2(s+p/2-p/q1) (1 + j) bq2 2 p -1 l=1 w∈Z p |α f (j, l, w)| q1 q2/q1 1/q2
.

Note that for b = 0, B s,0 q1,q2 (R p , R) coincides with the classical Besov space B s q1,q2 (R p , R) (Giné and Nickl, 2015). The Besov spaces can be generalized for any s ∈ R as a subspace of the space of tempered distribution S ′ (R p ). Indeed for f ∈ S ′ (R p ) and C > 0, writing α f (j, l, w) = ⟨f, ψ jlw ⟩, the Besov space for s ∈ R is defined as

B s,b q1,q2 (R p , R, C) = {f ∈ S ′ (R p )| ∥f ∥ B s,b q 1 ,q 2 ≤ C}.
In the following we will use intensively the connection between Hölder and Besov spaces.

LEMMA 3.3. (Proposition 4.3.23 (Giné and Nickl, 2015), (4.63) [START_REF] Haroske | Envelopes and sharp embeddings of function spaces[END_REF])) If α > 0 is a non integer, then

H α (R p , R) = B α ∞,∞ (R p , R) with equivalent norms. If α ≥ 0 is an integer, then ∀ϵ > 0 B α,1+ϵ ∞,∞ (R p , R) - → H α (R p , R) - → B α ∞,∞ (R p , R)
, where we write A -→ B if the function space A compactly injects in the function space B.

The p-dimensional wavelet basis (ψ jlw ) jlw will allow us to describe functions in H γ 1 (R p , R) that appear in the d H γ 1 IPM. On the other hand, to describe our generators g ∈ G ⊂ H β+1 K (T d , R p ), we will use the periodised wavelet basis (see Giné and Nickl (2015)). One can construct a wavelet basis on L 2 (T d ) by using the periodised scaling and wavelet functions on [0, 1]/Z:

(3.7) ϕ per j (s) = 2 j/2 k∈Z ϕ(2 j (s -k)) and ψ per j (s) = 2 j/2 k∈Z ψ(2 j (s -k)).
Note that it is a well known fact that k∈Z ϕ(s -k) = 1 (Section 4.3.4 in Giné and Nickl (2015)). For ease of notation, ϕ per j and ψ per j will be written ψ per j0 , ψ per j1 . Then

1, ψ per jlz = d i=1 ψ per jli (• -2 -j z i ) | j ∈ N 0 , l ∈ {1, ..., 2 d }, z ∈ {0, ..., 2 j -1} d ,
is a wavelet basis of L 2 (T d ). We will use this basis to describe functions in H β+1 (T d , R p ) as Lemma 3.3 still holds for the periodised Besov spaces Giné and Nickl (2015)). Let us define an operator that allows to change the regularity of a function by modifying its wavelet coefficients. DEFINITION 3.4. Let X be either R η or the torus [0, 1] η /Z η , and s, b, γ, c ∈ R.

B β+1 ∞,∞ (T d , R p ) (Section 4.3.4 in
For f ∈ B s,b ∞,∞ (X , R) a tempered distribution, define the tempered distribution Γ γ,c (f ) ∈ B s+γ,b+c ∞,∞ (X , R) by its wavelets coefficients ⟨Γ γ,c (f ), ψ jlz ⟩ := 2 jγ (1 + j) c ⟨f, ψ jlz ⟩.
The tempered distribution Γ γ,c (f ) is a "regularization" of f if γ < 0 or γ = 0 and c < 0. For ease of notation, we will write Γ γ instead of Γ γ,0 . A first use of this operator is in the following lemma. LEMMA 3.5. For all η ≥ 1, g ∈ H η K , γ ∈ [1, η + 1] and ϵ ∈ (0, 1) we have

d H γ 1 (g #U , g ⋆ #U ) ≤ C log(ϵ -1 ) 1/2 p i=1 ∥Γ -(γ-1) (g i ) -Γ -(γ-1) (g ⋆ i )∥ L 2 ([0,1] d ,R) + ϵ .
The proof of Lemma 3.5 can be found in Section B.3.1. This lemma asserts that the distance between g #U and g ⋆ #U for the norm d H γ 1 is controlled by the L 2 distance between the γ -1 regularization of g and g ⋆ . Therefore, to have a small generator error ∆ G , we do not have to control ∥g -g ⋆ ∥ ∞ , but only the quantity ∥Γ -(γ-1),-2 (g) -Γ -(γ-1),-2 (g ⋆ )∥ L 2 . This is a key result that will help us obtain the right bound on ∆ G . Indeed, from (3.5) we know that we need ∆ G ≤ Cn -β+γ 2β+d ∨ n -1 2 , but from (3.6) we know that will we have at best inf

g∈G ∥g ⋆ -g∥ ∞ ≥ Cn -β+1 2β+d
. Therefore, we need to take advantage of the γ-smoothness of the discriminator in order to obtain

∆ G = inf g∈G d H γ 1 (g #U , g ⋆ #U ) ≤ Cn -β+γ 2β+d ∨ n -1 2 .
Let us note that in [START_REF] Tang | Minimax rate of distribution estimation on unknown submanifold under adversarial losses[END_REF] it is supposed that the target measure µ belongs to the class S used in the adversarial training (see Section 2.3), which allows them to get around this problem as they have

∆ S = inf ν∈S d H γ 1 (ν, µ) = d H γ 1 (µ, µ) = 0.
Furthermore, to utilize the γ-smoothness of the discriminator, they compute a high order estimator of the type

J (D) = n i=1 1≤|j|≤⌊γ⌋ 1 j! ∇ j D(X i ) ĝ • q(X i ) -X i j
which makes them having to compute all the derivatives up to the order γ of every D ∈ H γ 1 (R p , R). Lemma 3.5 gives a simple way to utilize the γ-smoothness of the discriminator using wavelet theory and that allows us not to have to compute high order estimators.

To obtain Lemma 3.5, the main idea is to write

E D(g(U )) -D(g ⋆ (U )) = E 1 0 ∇D g ⋆ (U ) + t(g(U ) -g ⋆ (U )) , g(U ) -g ⋆ (U ) dt ,
and to transfer the regularity of the discriminator ∇D ∈ H γ-1 1 to the generators g, g ⋆ by modifying their wavelets coefficients. This result is a particular case of Proposition B.1 that shows that in a general setting, we can trade the regularity of the distributions in the L 2 scalar product. This result will be used multiple times along the paper.

3.2.2. From GANs to wavelets and back. Let us now discuss how we are going to use wavelet theory to construct our classes of generators and discriminators. For η ∈ (0, β + 1), δ ∈ (0, 1), and

C η > 0 such that H η 1 (T d , R p ) ⊂ B η ∞,∞ (T d , R p , C η ), define (3.8) F η,δ per := {f ∈ B η ∞,∞ (T d , R p , C η K)| ⟨f i , ψ per jlz ⟩ L 2 = 0, ∀j ≥ log 2 (δ -1 )}
the set of functions in B η ∞,∞ that do not have frequencies higher than log 2 (δ -1 ). We will later build our class of generators G as an approximation by neural network of the class F η,δ per . From Lemma 3.5, approximation error and covering numbers of the class F η,δ per can be derived.

PROPOSITION 3.6. We have

∆ F η,δ per := sup f ⋆ ∈H η K inf f ∈F η,δ per d H γ 1 (f #U , f ⋆ #U ) ≤ C log(δ -1 ) C2 δ η-1+γ ,
and ∀ϵ > 0, log(|F η,δ per | ϵ ) ≤ Cδ -d log(δ -1 ) log(ϵ -1 ).

The proof of Proposition 3.6 can be found in Section B.3.2. Choosing δ = δ n := n -1 2β+d and η = β + 1 in Proposition 3.6, we obtain (3.9)

∆ F β+1,δn per ≤ C log(n) C2 n -β+γ 2β+d
and log(|F β+1,δn per

| 1/n ) ≤ Cn d 2β+d log(n) 2 .
From (3.4) and (3.5), we see that these properties make F β+1,δn per ideal to be our class of generators G.

To make the method tractable, we provide a way to approximate F β+1,δn per in practice using ReQU neural networks (Belomestny et al., 2022). Fix L > 0 and let be φ : R → R such that

(3.10) ∥ϕ -φ∥ H ⌊β⌋+2 ≤ CL -1 ,
for ϕ the Daubechies scaling function. The existence of a REQU neural network φ satisfying (3.10) is guaranteed by Lemma A.1 in Section A.2.1. We also detail in this section how to compute ψper jlz (A.10), the approximation by neural networks of the periodised Daubechies wavelet ψ per jlz using φ. Define the class of function Fη,δ per by

Fη,δ per = f ∈ L 2 (T d , R p )| f i = log 2 (δ -1 ) j=0 2 d l=1 z∈{0,...,2 j -1} d α(j, l, w) i ψper jlz with |α(j, l, z) i | ≤ C η K2 -j(η+d/2) , ∀i ∈ {1, ..., p} , (3.11) for C η the constant such that ∥ • ∥ B η ∞,∞ ≤ C η ∥ • ∥ H η . The class Fη,δ
per is an approximation of the class F η,δ per defined in (3.8) using the ReQu neural network approximation ψper jlz (A.10) of the periodised wavelet (3.7). Using propositions 3.6 we obtain in the next proposition that Fη,δ per inherits the same properties as F η,δ per in terms of approximation error and covering numbers.

PROPOSITION 3.7. If φ : R → R is such that (3.10) holds with L ≥ δ -(⌊β⌋+1+γ) , then

∆ F η,δ per := sup f ⋆ ∈H η K inf f ∈ F η,δ per d H γ 1 (f #U , f ⋆ #U ) ≤ C log(δ -1 ) C2 δ η+γ-1 and ∀ϵ ∈ (δ η-1 L -1 , 1), log(| Fη,δ per | ϵ ) ≤ Cδ -d log(⌊δ -1 ⌋) log(ϵ -1 ).
The proof of Proposition 3.7 can be found in Section B.3.3. Using this result, we have from (3.9) that the class Fβ+1,n -1 2β+d per is perfectly suited to be our class of generators. The choice of the discriminator class is however more delicate as we do not have a result such as Lemma 3.5 enabling to exploit the (β + 1)-smoothness of the generators to bound ∆ D (3.2). Indeed, we used Lemma 3.5 to show that we can transfer the regularity of the discriminators to the generators in the IPM d H γ 1 (g #U , g ⋆ #U ), and therefore bound ∆ G by the approximation error of the β + γ-smooth map Γ -(γ-1) (g ⋆ ), for the L 2 distance. We will see in the following sections that it is not straightforward that we can do the same thing for ∆ D (meaning giving the regularity of generators to discriminators) as the measures g #U and g ⋆ #U

do not have any regularity from the point of view of the p-dimensional Lebesgue measure.

In the next section, we study two different settings to get insights on how to solve this problem. We show that we can either bypass it by taking a non tractable class D, or add more assumptions on g ⋆ (Model 3) so that the measure g ⋆ #U has some regularity from the point of view of the p-dimensional Lebesgue measure.

4. Model generality vs. tractability: two sides models. In this section we treat two different settings to get some insights on how to build a minimax and tractable GAN estimator. We first treat the general case of Model 1 and discuss the limits of the assumptions. Then, we treat the case of Model 2 to understand in the easier setting of full dimension, how additional assumptions can help us obtain a tractable estimator.

4.1. Model 1: A tractable generator class in the general low dimensional case. In this section, we fix a γ ∈ [1, β + 1] and define two classes of function G and D such that the associated GAN estimator is minimax for d H γ 1 in the setting of Model 1. Fix the approximation of the scaling function in (3.10) to be at precision L -1 = n -1 and take as the generator class (4.1)

G := Fβ+1,n -1 2β+d per .
We detail in Section A.2.2 how to implement the class Fβ+1,n -1 2β+d per in practice. Let us take as the discriminator class (4.2)

D := {D ⋆ g,g ′ | g, g ′ ∈ G 1/n }, with D ⋆ g,g ′ ∈ arg max D∈H γ 1 E[D(g(U )) -D(g ′ (u))].
The class of discriminators is constructed specifically to well approximate the IPMs between the measures g #U with g ∈ G, without being too massive. This class D is not really computable in practice, but we use it here to illustrate that we can bypass the problems risen by the lack of regularity of the target measure g ⋆ #U , at the cost of the tractability of the method. Having defined the classes G and D, we can state the final bound for the GAN estimator (1.3) in the setting of Model 1.

THEOREM 4.1. For all g ⋆ ∈ H β+1 K (T d , R p ) and γ ∈ [1, β + 1],
the GAN estimator ĝ of (1.3) with G and D defined in (4.1) and (4.2), satisfies

E d H γ 1 (ĝ #U , g ⋆ #U ) ≤ C log(n) C2 n -β+γ 2β+d ∨ n -1 2 .
The proof of Theorem 4.1 can be found in Section C.1.1. As usual in nonparametric estimation, an extra logarithmic term in Theorem 4.1 is to be a attributed to a concentration bound (Theorem 3.1). More interestingly, another one arises from functional inequalities when β is not an integer (Lemma 3.5). We do not keep track of the extra exponent C 2 to avoid unnecessary heaviness, as the obtained exponent is very likely suboptimal.

As pointed out in remark 4 of [START_REF] Tang | Minimax rate of distribution estimation on unknown submanifold under adversarial losses[END_REF], some authors consider a more general setting than ours, by allowing the observed data to contain noise. It usually consists in supposing that instead of observing data X i ∼ µ i.i.d, we observe Y i = X i + ϵ i with ϵ i i.i.d. errors. In this setting, Genovese et al. (2012) showed that when the error follows the standard Gaussian distribution, the minimax rate for estimating a manifold under the Hausdorff distance is lower bounded by O(log(n) -1 ). To bypass this slow convergence, previous work like [START_REF] Divol | Measure estimation on manifolds: an optimal transport approach[END_REF], assume the noise variance to decay with the number of data so that it does not impact the rate of convergence. In our setting, supposing we observe noisy data with ∥ϵ 1 ∥ ≤ n -β+γ 2β+d ∨ n -1 2 almost surely, will not impact Theorem 4.1 as the noise scales at the minimax rate.

Theorem 4.1 shows that the GAN estimator is minimax optimal (up to the logarithmic factor) for the distance d H γ 1 . Nevertheless, this result does not really translate to practical applications as the class of discriminators D (4.2) we used is not tractable. Due to the lack of smoothness of the target measure g ⋆ #U , we had to use the fact that the class G is small rather than it is of smoothness β + 1. Supposing that the target measure has a smooth density with respect to the volume measure of a submanifold, we show in the next sections that we can exploit this regularity and therefore, use a tractable class of discriminators.

Model 2:

A tractable D in full dimension. In this section we move away from the case of transport maps from the torus and treat the setting of Model 2 where the target measure µ has a density f ⋆ with respect to the p-dimensional Lebesgue measure. The goal is to understand how to build a tractable estimator of densities on submanifolds, in the simple case where the submanifold is the whole R p space.

Let us first state an analog of Theorem 3.1 in the case of densities.

COROLLARY 4.2. If the target f ⋆ belongs to H β K (R p , R), has support included in B p (0, K) and F ⊂ B β ∞,∞ (B p (0, K), R, K), D ⊂ B γ ∞,∞ (B p (0, K), R, 1) with γ ∈ [1, β], then the density adversarial estimator (2.2) verifies E sup D∈H γ 1 R p D(x)( f (x) -f ⋆ (x))dλ p (x) -∆ F -∆ D ≤ C min δ∈[0,1] (δ + 1/n) 2 log(n|F 1/n ||D 1/n |) n + 1 √ n (1 + δ (1-p 2γ ) + log(δ -1 )1 {2γ=p} ) .
The proof of Corollary 4.2 can be found in Section C.2.1. Let us now build the classes of functions F and D by adapting to the non periodic case, the construction of the class of generators (4.1) from the previous section. For η ∈ (0, β + 1) and δ ∈ (0, 1), define

F η,δ = {f ∈ B η ∞,∞ (R p , R, C η K)| ⟨f, ψ jlw ⟩ L 2 = 0, ∀j ≥ log 2 (δ -1 ) and supp(f ) ⊂ B p (0, K)} the set of functions in B η
∞,∞ that do not have high frequencies and have compact support in B p (0, K). As in Section 4.1, we are going to approximate the class F η,δ by neural networks. Let φ, ψ be the approximations by neural networks of the scaling and wavelet Daubechies function from (A.4) and (A.5) at precision L -1 = n -1 . For ease of notation φ, ψ will be written ψ0 , ψ1 respectively and define the multi-dimensional approximation of the wavelet as

ψ0w (x) = p i=1 ψ0 (x i -w i ) , ψjlw (x) = 2 pj/2 p i=1 ψli (2 j x i -w i ).
Using these approximated wavelets, Define the class of function F by (4.3)

Fη,δ := log 2 (δ -1 ) j=0 2 p l=1 w∈{-K2 j ,...,K2 j } p α(j, l, w) ψjlw |α(j, l, w)| ≤ C η K2 -j(η+p/2) .
As in the periodic case, we obtain that Fη,δ and F η,δ have the same bounds on their approximation errors and covering numbers.

4.3. If ∥ϕ -φ∥ H ⌊β⌋+2 ≤ L -1 with L ≥ δ -(⌊β⌋+2+γ) , then ∆ F η,δ := sup h ⋆ ∈H η 1 inf f ∈ F η,δ sup D∈H γ 1 R p D(x)(f (x) -h ⋆ (x))dλ p (x) ≤ Cδ η+γ and ∀ϵ ∈ (L -1 δ η-1 , 1), log(| Fη,δ | ϵ ) ≤ Cδ -p log(⌊δ -1 ⌋) log(ϵ -1 ).
The proof of Proposition 4.3 can be found in Section C.2.2. Here, we need our class of generators to approximate at a scale O(n

-β+γ 2β+p ) the density f ⋆ ∈ H β K (R p , R), while hav- ing a covering number that does not exceed O(n p 2β+p ) (see (3.4)) up to logarithmic factors. Therefore, choosing η = β and δ n = n -1 2β+p in Proposition 4.3, we have inf f ∈ F β,δn sup D∈H γ 1 R p D(x)(f (x) -f ⋆ (x))dλ p (x) ≤ Cn -β+γ 2β+p and log(| Fβ,δn 1/n |) ≤ Cn p 2β+p log(n),
which makes Fβ,δn perfectly suited to be our class of generators.

We now develop a method that allows to exploit the smoothness of the target measure. We are going to define a class of discriminators D such that the density adversarial estimator (2.2) attains minimax rates for all

d H γ 1 , γ ∈ [1, β] simultaneously.
4.4. Let be f, f ⋆ ∈ H β K (R p , R) compactly supported in B p (0, K).
Then for all α ≥ γ > 0, we have

d B γ,2 ∞,∞ (1) (f, f ⋆ ) ≤ Cd B α,2 ∞,∞(1) (f, f ⋆ ) β+γ β+α .
The proof of Corollary 4.4 can be found in Section C.2.3. This is the key result that will allow us to prove that if f attains the minimax rate for d H α 1 , then it will also attain the minimax rate for d H γ 1 for all γ ∈ [1, α]. Indeed, by injection of Besov spaces into Holder spaces (up to logarithmic terms), Corollary 4.4 implies that for all ϵ ∈ (0, 1),

d H γ 1 (f, f ⋆ ) ≤ C log(ϵ -1 ) 2 d H α 1 (f, f ⋆ ) β+γ β+α + ϵ, as detailed in Section C.2.6. Therefore, if for a certain α ∈ [1, β] we obtain d H α 1 ( f , f ⋆ ) ≤ O(n -β+α 2β+p ), then ∀γ ∈ [1, α] we will have up to logarithmic terms, (4.4) d H γ 1 ( f , f ⋆ ) ≤ Cd H α 1 ( f , f ⋆ ) β+γ β+α ≤ O(n -β+α 2β+p ) β+γ β+α = O(n -β+γ 2β+p ).
Hence, by taking the regularity of the discriminator class to be equal to β, if we obtain

d H β 1 ( f , f ⋆ ) ≤ O(n -β+β 2β+p ), we would have minimax rates for d H γ 1 for all γ ∈ [1, β].
However, in the case where β > p/2 the minimax rate for the distance d

H β 1 is O(n -1/2 ) not O(n -β+β 2β+p
), so equation (4.4) does not work anymore. Therefore, we choose the regularity of the discriminator class D to be

β := β ∧ p/2.
At the end of the day, we choose (4.5)

F := Fβ,n -1 2β+p and D := F β,n -1 2β+p
as classes of generators and discriminators. We are first going to show that the density adversarial estimator (2.2) with these classes attains minimax rates for the distance d H β 1

. Then applying Corollary 4.4, we will obtain that it also attains minimax rates for all

d H γ 1 , γ ∈ [1, β]. Now to obtain the rate for the distance d H β 1
, we need to bound the approximation error of the class D which is defined in the density case as

∆ D := sup f ∈F sup D ⋆ ∈H β 1 inf D∈D R p (D ⋆ (x) -D(x))(f (x) -f ⋆ (x))dλ p (x).
Here, in contrast to the assumptions of Model 1, the target measure f ⋆ has some regularity from the point of view of the p-dimensional Lebesgue measure. Therefore, as illustrated in the following lemma, we can give the β-regularity of the function f -f ⋆ to the generators D ⋆ -D and bound ∆ D by the approximation error of the β + β-smooth map Γ -β (D ⋆ ).

LEMMA 4.5. For D defined in (4.5) we have

∆ D ≤ C sup h∈B 0 ∞,∞ sup D ⋆ ∈H β 1 inf D∈D R p (Γ -β (D ⋆ )(x) -Γ -β (D)(x))h(x)dλ p (x).
The proof of Lemma 4.5 can be found in Section C.2.4. Putting the bounds of Lemma 4.5 and Proposition 4.3 in Corollary 4.2, we obtain that the density adversarial estimator is minimax for the distance d H β 1 . PROPOSITION 4.6. For f the density adversarial estimator (2.2) using the classes F and D defined in (4.5), we have that

E d H β 1 ( f , f ⋆ ) ≤ C log(n) C2 n -β+ β 2β+p .
The proof of Proposition 4.6 can be found in Section C.2.5. Using Corollary 4.4, we can now show the final bound for the density adversarial estimator (2.2) in the setting of Model 2. PROPOSITION 4.7. For all f ⋆ ∈ H β+1 K (R p , R), the density adversarial estimator f of (2.2) using the classes F and D defined in (4.5) verifies for all γ ∈ [1, β]

E d H γ 1 ( f , f ⋆ ) ≤ C log(n) C2 n -β+γ 2β+p ∨ n -1 2 .
The proof of Proposition 4.7 can be found in Section C.2.6. We can conclude that the density adversarial estimator attains minimax optimal rates up to logarithmic factors in the density case. Note that we had to distinguish whether β ≤ p/2 or not.

• In the case β < p/2, the estimator f is computed using a discriminator class of regularity β and attains minimax rates for all γ ∈ [1, β] at the same time. • In the case β ≥ p/2, the estimator f is computed using a discriminator class of regularity p/2 and attains minimax rates for all γ ∈ [1, ∞) at the same time, as for

γ ≥ p/2, d H γ 1 ( f , f ⋆ ) ≤ d H p/2 1 ( f , f ⋆ ).
As mentioned in Section 2.2, the density adversarial estimator (2.2) does not provide a direct way of sampling from the approximate density f . However, one could obtain such estimator by supposing that the target density f ⋆ is the push forward measure of a known easy to sample density ν on R p . In this case, defining the GAN estimator as

ĝ ∈ arg min g∈G sup D∈D 1 n n i=1 D(X i ) -D(g(Y i )) for Y i ∼ ν i.i.d. and G ⊂ H β+1 K (R p , R p )
, one could prove (following the same method developed above) that this estimator attains minimax rates of convergence. We do not develop further on this estimator as the main purpose of this paper is to provide results for the manifold case of Model 3. Having developed a method for the full dimensional case, we adapt it in the following section to the case where the submanifold is of dimension d < p.

5. The manifold case. In this section, we suppose that we are in the setting of Model 3. Using this hypothesis we have that g ⋆ #U admits a density with respect to the volume measure of the manifold M g ⋆ = g ⋆ (T d ) and that its density

f g ⋆ belongs to H β C (M g ⋆ , R).
In order to adapt the method of the preceding section, we first prove an interpolation inequality similar to Corollary 4.4 in this setting. Then, we use it to prove minimax optimality of the GAN estimator.

5.1. Interpolation inequality for smooth densities on manifolds. In this section we want to prove an analog version of Corollary 4.4 but in the context of Model 3. The difficulty comes from the fact that although our push forward maps g, g ⋆ : T d → R p belong to H β+1 , the densities they produce on their respective images do not have regularity from the point of view of the p-dimensional Lebesgue measure. We will see that in order to exploit the regularity of the push forward maps, we can split the cost E[D(g(U )) -D(g ⋆ (U ))] into two terms: one characterizing the distance between the support of g #U and g ⋆ #U , and the other characterizing their difference in terms of density. Let us first state the desired result and then detail its proof. THEOREM 5.1. Let g, g ⋆ ∈ H β+1 K (T d , R p ) that verify the K-manifold regularity condition with g ⋆ that verifies the K-density regularity condition of Definition 2.1. There exists constants

C, C 2 > 0 such that if d H β+1 1 (g #U , g ⋆ #U ) ≤ C -1 , then for all ϵ ∈ (0, 1) and γ ∈ [1, β + 1], we have d H γ 1 (g #U , g ⋆ #U ) ≤ C 2 log(ϵ -1 ) 4 d H β+1 1 (g #U , g ⋆ #U ) β+γ 2β+1 + ϵ .
Compared to Corollary 4.4, g #U and g ⋆ #U need to be already close in d H β+1 1 distance for Theorem 5.1 to apply. This assumption will imply that there exists a diffeomorphism between their supports with several key properties (Proposition D.5). The precise definition of these properties is gathered into the notion of (g #U , g ⋆ #U )-compatible map (Definition D.3) and can be found in the appendix Section D.1. In a nutshell, for t

> d H β+1 1 (g #U , g ⋆ #U ), a map T : M t g ⋆ → M g ⋆ being (g #U , g ⋆ #U )-compatible is a projection onto M g ⋆ such that its restriction to M g is a C β+1 -diffeomorphism.
Let us now prove Theorem 5.1 through

(g #U , g ⋆ #U )-compatibility. For D ∈ H γ 1 (R p , R) and T being (g #U , g ⋆ #U )-compatible, write E U ∼U ([0,1] d ) D(g(U )) -D(g ⋆ (U )) = A 1 + A 2 ,
with

A 1 := E U ∼U ([0,1] d ) D(g(U )) -D(T (g(U )))
characterizing the distance between the supports, and

A 2 := E U ∼U ([0,1] d ) D(T (g(U ))) -D(g ⋆ (U ))
characterizing the distance between the densities. Splitting the cost this way allows to finely exploit the regularity of g and g ⋆ :

• The term A 2 writes as an integral of a product of functions defined on the same support M g ⋆ . Indeed, denoting by f Tg the density of (T • g) #U with respect to the volume measure λ M g ⋆ , we have

A 2 = M g ⋆ D(x)(f Tg (x) -f g ⋆ (x))dλ M g ⋆ (x).
Since the only support involved is M g ⋆ , wavelet-based expansions allow to transfer the regularity of the densities on M g ⋆ to the discriminator, as in the the full-dimensional case (Corollary 4.4). • The term A 1 writes similarly, through the Taylor expansion

A 1 = p i=1 [0,1] d 1 0 ∂ i D(g(u) + t(T (g(u)) -g(u)))(T (g(u)) -g(u)) i dtdλ d (u).
Here, a similar wavelet-like expansion is made possible using the compatibility properties of T : M g → M g ⋆ . However, extra technical difficulties arise to deal with the evaluation of a regularised version of D on the ambient points g(u) + t(T (g(u)) -g(u) 0≤t≤1 , while the wavelet construction naturally takes place on the torus T d .

More formally, the bound on A 1 comes from the following result.

LEMMA 5.2. Let g, g ⋆ ∈ H β+1 K (T d , R p ) that verify the K-manifold regularity condition with g ⋆ that verifies the K-density regularity condition. There exist constants

C, C 2 > 0 such that, if d H β+1 1 (g #U , g ⋆ #U ) ≤ C -1
, then for all ϵ ∈ (0, 1) and γ ∈ [1, β + 1], we have

d H γ 1 ((T • g) #U , g #U ) ≤ C 2 log(ϵ -1 ) 4 sup D∈H β+1 1 ,D•T =0 E U ∼U ([0,1] d ) [D(g(U ))] β+γ 2β+1 + ϵ .
The proof of Lemma 5.2 can be found in Section D.3. Note that the right hand side only depends on g ⋆ through the constraint D • T = 0, i.e. that D |M g ⋆ = 0. As a result, this term is oblivious to the mass distribution of g ⋆ #U over M g ⋆ , and only measures distances between the support M g and M g ⋆ .

To deduce a bound on A 1 , observe that

sup D∈H β+1 1 ,D•T =0 E U ∼U ([0,1] d ) [D(g(U ))] = sup D∈H β+1 1 ,D•T =0 E U ∼U ([0,1] d ) [D(g(U )) -D(g ⋆ (U ))] ≤ d H β+1 1 (g #U , g ⋆ #U
), which finally leads to

A 1 ≤ C 2 log(ϵ -1 ) 4 d H β+1 1 (g #U , g ⋆ #U ) β+γ 2β+1 + ϵ .
The term A 2 may be bounded using the following result.

LEMMA 5.3. Let g, g ⋆ ∈ H β+1 K (T d , R p ) that verify the K-manifold regularity condition with g ⋆ that verifies the K-density regularity condition. There exist constants

C, C 2 > 0 such that if d H β+1 1 (g #U , g ⋆ #U ) ≤ C -1
, then for all ϵ ∈ (0, 1) and γ ∈ [1, β + 1], we have

d H γ 1 ((T • g) #U , g ⋆ #U ) ≤ C 2 log(ϵ -1 ) sup D∈H β+1 1 (M g ⋆ ,R) E U [D(T (g(U ))) -D(g ⋆ (U ))] β+γ 2β+1 + ϵ.
The proof of Lemma 5.3 can be found in Section D.4. For D ∈ H β+1

1 (M g ⋆ , R), we have D • T ∈ H β+1 C (M t g ⋆ , R
). Thus, using Theorem D.20, we can show (like for Proposition D.17) that D • T can be extended globally to a map in H β+1 C (R p , R). In particular, we have sup

D∈H β+1 1 (M g ⋆ ,R) E U ∼U ([0,1] d ) [D(T (g(U ))) -D(g ⋆ (U ))] ≤ Cd H β+1 1 (g #U , g ⋆ #U ).
Hence, from Lemma 5.3 we deduce that

A 2 ≤ C 2 log(ϵ -1 )d H β+1 1 (g #U , g ⋆ #U ) β+γ 2β+1 + ϵ,
that concludes the proof of Theorem 5.1. Just as in the full-dimensional case (Section 4.2), the next section builds upon this interpolation bound to show that GAN estimators attain optimal estimation rates for all the distances d H γ 1 γ∈[1,β+1] simultaneously.

5.2.

A tractable GAN estimator in the manifold case. We are now in position to define our classes of generators and discriminators for the GAN estimator in the setting of Model 3. We use the class of generators G from Section 4.1 and the class of discriminators D from Section 4.2 (both computable in practice). As in Section 4.2, we first show that the GAN estimator attains minimax rates for the distance d H β+1 1 (for β defined in (5.2)). Then applying Theorem 5.1, we obtain that it also attains minimax rates for all d H γ 1 , γ ∈ [1, β + 1]. In order to apply Theorem 5.1 to the GAN estimator ĝ, we need it to verify the manifold regularity condition of Definition 2.1. We define in the Appendix Section E.1, the χnumerical regularity condition that implies the manifold regularity condition. In a nutshell, it is an easy to compute condition that ensures that if two points are far in the torus, then their images by ĝ are not too close. We choose to be the class of discriminators. The specific discriminator regularity β + 1 comes naturally in light of Theorem 5.1. Using the same argument as above (4.4) for the full-dimensional case, β + 1 is a regularity cap for the discriminator class, over which no gain may be expected in terms of convergence of the GAN (recall that the convergence rate in terms of d H γ 1 cannot be faster than n -1/2 ).

However, the use of this discriminator class raises two problems.

• The logarithm of its 1/n-covering is of order O(n p 2 β+d ), which is too large as it depends on the ambient dimension p.

• It only approximates H β+1 1 up to an error O(n -β+1 2 β+d ) in sup norm, so it is not clear whether we can obtain that ∆ D is of order O(n -2 β+1 2 β+d ).

To bypass these problems, we sharpen the bound given by Theorem 3.1 through a finer analysis of the bias and variance terms of the class D. The quantities log(|D 1/n |) and ∆ D will be replaced using the following finer (but more involved) quantities.

• For ϵ ∈ (0, 1), let

|(D G ) ϵ | := sup g∈G |({D • g | D ∈ D}) ϵ | , be the largest ϵ-covering number of the class {D • g | D ∈ D} among g ∈ G. • For g ∈ G, define D ⋆ g ∈ arg max D∈B β+1 ∞,∞(C ) E[D(g(U )) -D(g ⋆ (U ))] and D g ∈ arg min D∈D ∥D -D ⋆ g ∥ B 0 ∞,∞ ,
as the optimal discriminator within B β+1 ∞,∞ and its approximation by D. Let us write

∆ g D := E[D ⋆ g (g(U )) -D ⋆ g (g ⋆ (U )) -(D g (g(U )) -D g (g ⋆ (U )))] for the discrimination error between g #U and g ⋆ #U of the class D compared to the class B β+1
∞,∞ (C). Using these two quantities, Theorem 3.1 can be sharpened as follows.

THEOREM 5.4. For all g ⋆ ∈ H β+1 K (T d , R p ) verifying the K-manifold and K-density regularity conditions of Definition 2.1, the GAN estimator ĝ of (1.3), with G and D defined in (5.1) and (5.3), satisfies

E d H β+1 1 (ĝ #U , g ⋆ #U ) -∆ G -E[∆ ĝ D ] ≤C min δ∈[0,1] (δ + 1/n) 2 log(n|G 1/n ||(D G ) 1/n |) n + 1 √ n (1 + δ (1-d 2( β+1) ) + log(δ -1 )1 {β+1=d/2} ) .
The proof of Theorem 5.4 can be found in Section E.3. Contrary to ∆ D , the term E[∆ ĝ D ] depends only on the approximation error of D for the candidate minimizer ĝ. On the other hand, in contrast to log(|D 1/n |), the term log(|(D G ) 1/n |) does not depend on the dimension of the ambient space p as shown in the following result.

LEMMA 5.5.

For g ∈ Lip K (T d , [0, 1] p ), there exists an ϵ-covering (N g ) ϵ of the class N g = {D • g | D ∈ F β+1,δ }, such that log(|(N g ) ϵ |) ≤ Cδ -d log(δ -1 ) log(ϵ -1 ).
The proof of Lemma 5.5 can be found in Section E.4. For δ = δn = n -1 2 β+d , we deduce that log

(|(D G ) 1/n |) ≤ C log(n) 2 n d 2 β+d , so it scales roughly like log(|G 1/n |). Let us now explain how to bound the term E[∆ ĝ D ]. Compared to Model 1, g ⋆ #U
has some regularity with respect to a smooth submanifold. Therefore, this regularity can be exploited with Theorem 5.1 in the following manner.

LEMMA 5.6. There exists a constant C > 0 such that, if g ∈ G verifies the χ-manifold regularity condition and d H β+1 (g #U , g ⋆ #U ) ≤ C -1 , then we have

∆ g D ≤ C 2 log(n) C3 ∥∇D ⋆ g -∇D g ∥ ∞ d H β+1 1 (g #U , g ⋆ #U ) β+1 2 β+1 + 1 n .
The proof of Lemma 5.6 can be found in Section E.5. It shows that E[∆ ĝ D ] depends both on the precision of the approximation of H β+1 1 by D, and on the d

H β+1 1 distance between ĝ#U and g ⋆ #U . Therefore, if ĝ#U is close to g ⋆ #U , the term E[∆ ĝ D ]
is much smaller than ∆ D . From Lemmas 5.5 and 5.6, we obtain that the GAN estimator built with classes G and D defined in (5.1) and (5.3) is minimax for the distance d H β+1 . THEOREM 5.7. For all g ⋆ ∈ H β+1 K (T d , R p ) verifying the K-manifold and K-density regularity conditions of Definition 2.1, the GAN estimator ĝ of (1.3), with G and D defined in (5.1) and (5.3), satisfies

E d H β+1 1 (ĝ #U , g ⋆ #U ) ≤ C log(n) C2 n -2 β+1 2 β+d .
The proof of Theorem 5.7 can be found in Section E.6. Finally, combining Theorem 5.7 with the interpolation bound of Theorem 5.1, the main result follows: over Model 3, GAN estimators attain minimax optimal rates for all the distances d H γ 1 γ∈[1,β+1] simultaneously.

THEOREM 5.8. For all g ⋆ ∈ H β+1 K (T d , R p ) verifying the K-manifold and K-density regularity conditions of Definition 2.1, the GAN estimator ĝ (1.3), with G and D defined in (5.1) and (5.3), satisfies, for all γ ∈ [1, β + 1],

E d H γ 1 (ĝ #U , g ⋆ #U ) ≤ C log(n) C2 n -β+γ 2β+d ∨ n -1 2 .
As in the full dimensional case, we have:

• When β + 1 < d/2, ĝ is computed using a discriminator class of regularity β + 1 and attains minimax rates up to logarithmic factors for all γ ∈ [1, β + 1] at the same time. • When β + 1 ≥ d/2, ĝ is computed using a discriminator class of regularity d/2 and attains minimax rates up to logarithmic factors for all γ ∈ [1, ∞) at the same time.

To the best of our knowledge, the GAN estimator (1.3) is the first tractable estimator attaining minimax rates up to logarithmic factors in the manifold setting. It is also the first one to be proved to be minimax for several IPMs simultaneously. On the other hand, the fact that we attain minimax rates shows in particular that Theorem 5.1 is sharp up to the logarithmic factors.

In [START_REF] Liang | How well generative adversarial networks learn distributions[END_REF], the author shows that in the case γ < d/2, if we take D = H γ 1 as the discriminator class for the GAN estimator (1.3), then the rate of error is O(n -γ d ) which is strictly slower than the minimax. This is due to the fact that the class H γ 1 is so massive that it is able to well discriminate between empirical and smooth measures. Therefore, it will be more sensitive to the distance between the data itself than the underlying measures behind it. This is well illustrated by the fact that for any measure ν with a smooth density with respect to a closed d-dimensional manifold M ⊂ R p , we have (Lemma 13 in [START_REF] Tang | Minimax rate of distribution estimation on unknown submanifold under adversarial losses[END_REF])

C 1 log(n) -1 n -γ d ≤ E Xi,Yi∼ν d H γ 1 ( 1 n n i=1 δ Xi , 1 n n i=1 δ Yi ) ≤ C 2 n -γ d .
Although the X i 's and Y i 's are i.i.d. with the same distribution, the d H γ 1 distance between their empirical distribution is strictly greater than the minimax rate. To overcome this problem, [START_REF] Liang | How well generative adversarial networks learn distributions[END_REF] and [START_REF] Tang | Minimax rate of distribution estimation on unknown submanifold under adversarial losses[END_REF] use as estimator

μ ∈ arg min ν∈S sup D∈H γ 1 E ν [D(X)] -Ĵ (D),
where Ĵ a regularization of the integration of D under the empirical measure

µ n = 1 n n i=1 δ Xi .
Theorem 5.8 shows that actually, it is not necessary to regularize µ n to obtain minimax rates. Taking a discriminator class D that is much smaller than the Hölder functions suffices to implicitly regularize the measures, as this class is not able to distinguish between discrete and smooth measures. This result gives a theoretical explanation of the well known fact that WGANs work because the discriminator class fails to approximate the Wasserstein distance as studied in [START_REF] Stanczuk | Wasserstein gans work because they fail (to approximate the wasserstein distance[END_REF].

Discussion. In this paper, we provided minimax rates of convergence of the GAN estimator in a general setting. This is a step towards a theoretical understanding of why Generative Adversarial Networks obtain state of the art results in various fields. On the other hand, it shows that it is not necessary to regularize the empirical measure in the GAN estimator (1.3) (like in [START_REF] Liang | How well generative adversarial networks learn distributions[END_REF] and [START_REF] Tang | Minimax rate of distribution estimation on unknown submanifold under adversarial losses[END_REF]) to obtain minimax rates.

Our analysis suggests that if the target measure has some some higher Hölder regularity than the class used in the IPM, one should use discriminators having the same regularity than the target. This is supported by our inequality between the discrimination of different regularities (Theorem 5.1). It shows that the Hölder class of the same regularity than the target distinguishes just as efficiently as the classes of weaker regularity (in particular the Wasserstein).

Our results strongly highlight the use of wavelets to describe function's regularity. The final GAN estimator uses classes of neural networks that are designed to well approximate low wavelets frequencies. These classes allow to connect with the "Network" part of the GAN, but any class allowing to well approximate the Daubechies wavelet would work.

This suggests that a "Generative Adversarial Wavelet" estimator would be perfectly suited to estimate smooth densities on manifolds. Future research will be focused on extending our analysis of this type of estimator and on numerical experiments to see how it performs compared to the state of the art. Furthermore, we could also consider moving away from the classical GAN setting with push-forward measures and extend our results to the case of manifolds with several charts.

APPENDIX A: ADDITIONAL NOTATION AND NEURAL NETWORK CONSTRUCTION

In this section, we define some additional notation needed for the poofs and we detail the construction of the Neural Networks used by our classes of generators and discriminators.

A.1. Additional notation. We will write Lip 1 the set of 1-Lipschitz functions. The support of a function f will be denoted by supp(f ). For a path γ : [0, 1] → T d , we write γ its differential. For u, v ∈ T d in the d-dimensional torus, we write ∥u -v∥ = min{

1 0 ∥ γ(t)∥dt | γ : [0, 1] → T d , γ(0) = u, γ (
1) = v} as the periodised Euclidean norm. The d-dimensional ball of radius ϵ > 0 in the torus for this distance will be written B d (0, ϵ). The Hausdorff distance between two subset A, B ⊂ R p will be denoted H(A, B). For x a point belonging either to R p , T d or R, we will write O(x) to denote a quantity that is bounded by C∥x∥ for C > 0 independent of x.

To lighten the notation, for a function f : R p → R that admits a wavelet expansion in L 2 :

f (x) = w∈Z p α f (w)ψ 0w (x) + ∞ j=0 2 p -1 l=1 w∈Z p α f (j, l, w)ψ jlw (x),
we will write

f (x) = ∞ j=0 2 p l=1 w∈Z p α f (j, l, w)ψ jlw (x),
with the convention that ψ 02 p w = ψ 0w and for all j ≥ 1, ψ j2 p w = 0. Likewise, the wavelet expansion of a map f : T d → R will be written

f (u) = ∞ j=0 2 d l=1 z∈{0,...,2 j -1} d α f (j, l, w)ψ per jlz (u),
with the convention that ψ 02 d 0 = 1 and for all z ̸ = 0, j ≥ 1, ψ per 02 d z = ψ per j2 d z = ψ per j2 d 0 = 0. Recalling the definition of Besov spaces from Section 3.2, in the case

q 1 = q 1 = ∞, the norm ∥ • ∥ B s,b q 1 ,q 2 of a function f : R p → R value ∥f ∥ B s,b ∞,∞ = sup j≥0 2 j(s+p/2) (1 + j) b 2 p l=1 sup w∈Z p |α f (j, l, w)|.
Likewise, for a function f :

T d → R we have ∥f ∥ B s,b ∞,∞ = sup j≥0 2 j(s+p/2) (1 + j) b 2 d l=1 sup z∈{0,...,2 j -1} d |α f (j, l, z)|.
A.2. Approximation of low wavelet frequencies by Neural Networks.

A.2.1. Theoretical construction of the approximated wavelet. Let us describe how to approximate the class F η,δ per of (3.8) by neural networks. The first ingredient we need, is to have a good approximation φ of the Daubechies scaling function ϕ ∈ H ⌊β⌋+3 (R, R) (Daubechies, 1988), that will then give us a good approximation of the wavelet function

(A.1) ψ = +∞ k=-∞ λ k ϕ(2 • -k).
The classical technique to approximate ϕ is to use that it is solution to the two-scale difference equation [START_REF] Daubechies | Two-scale difference equations. i. existence and global regularity of solutions[END_REF]:

(A.2) ϕ(x) = +∞ k=-∞ h k ϕ(2x -k)
for h k = 2 ϕ(y)ϕ(2y -k)dy. From Theorem 4.2.10 in (Giné and Nickl, 2015), there exists ϕ with compact support in [0, N ] that belongs to H ⌊β⌋+3 (R, R) with N ≤ 12(⌊β⌋ + 3). Then (A.2) can be rewritten as

(A.3) ϕ(x) = N k=0 h k ϕ(2x -k).
Using this property, the classical methods to approximate ϕ are iterative schemes as the "cascade algorithm" [START_REF] Daubechies | Two-scale difference equations. i. existence and global regularity of solutions[END_REF]. Here we would like to use neural networks, so we are going to use the ReQU construction of (Belomestny et al., 2022).

LEMMA A.1 (Theorem 2, Belomestny et al. (2022)). For all α > 2 and L > 0, there exists a class of neural networks with ReQu activation function

N α L ⊂ H ⌊α⌋ 1 ([0, 1] d , R p ) with width CL d and depth C such that for any f ∈ H α 1 ([0, 1] d , B p (0, 1)), there exists a neural network h f ∈ N α L such that ∀l ∈ {0, ..., ⌊α⌋} we have ∥f -h f ∥ H l ≤ C 1 L α-l .
In Belomestny et al. (2023) they also provide a bound on the covering number of the neural network class N α L . We will not use it as it depends on the number of parameters of the network which is to big here to obtain minimax rates. We will instead give a smaller covering using the wavelet decomposition of the functions.

Fix L > 0 and define φ ∈ N ⌊β⌋+2 L

as the neural network from Lemma A.1 such that (A.4) ∥ϕ -φ∥ H ⌊β⌋+2 ≤ CL -1 .

Recalling (A.1), define also

(A.5) ψ = 1 k=-N +1 λ k φ(2 • -k),
with λ k = (-1) k+1 h 1-k for the approximation by neural networks of the wavelet function.

Using these neural networks, let us approximate the periodised wavelets from (3.7). Recalling that ϕ has support in [0, N ] and ψ in [-N/2 + 1/2, N/2 + 1/2], then for s ∈ [0, 1] and z ∈ {0, ..., 2 j -1} we have

(A.6) ϕ per j (s -2 -j z) = 2 j/2 0 k=⌈-1-2 -j N ⌉ ϕ(2 j (s -k) -z) and (A.7) ψ per j (s -2 -j z) = 2 j/2 ⌊1+2 -(j+1) (N -1)⌋ k=⌈-1-2 -(j+1) (N +1)⌉ ψ(2 j (s -k) -z).
Note that if 2 j > N then

(A.8) ϕ per j (s -2 -j z) = ϕ(2 j s -z) + ϕ(2 j (s + 1) -z)
and (A.9) ψ per j (s -2 -j z) = ψ(2 j (s -1) -z) + ψ(2 j s -z) + ψ(2 j (s + 1) -z). Define the periodised wavelet approximations

φper j (s) = 2 j/2 0 k=⌈-1-2 -j N ⌉ φ(2 j (s -k))
and

ψper j (s) = 2 j/2 ⌊1+2 -(j+1) (N -1)⌋ k=⌈-1-2 -(j+1) (N +1)⌉ ψ(2 j (s -k)).
for φ and ψ from (A.4) and (A.5) . For ease of notation φper and ψper j will be written ψper j0 , ψper j1 respectively. We finally define the multi-dimension approximation of the wavelet as

(A.10) ψper 02 d 0 (u) = 1 , ψper jlz (u) = d i=1 ψper jli (u i -2 -j z i ) l = 1, ..., 2 d , z ∈ {0, ..., 2 j -1} d .
We use the convention that for all z ̸ = 0, j ≥ 1, ψper

02 d z = ψper j2 d z = ψper j2 d 0 = 0. Now taking a function a function f = log 2 (δ -1 ) j=0 2 d l=1 z∈{0,...,2 j -1} d αf (j, l, z) ψper jlz , the next
proposition states that the true wavelet coefficients α f (j, l, z) are close to its approximate coefficients αf (j, l, z).

PROPOSITION A.2. For any f ∈ L 2 (T d , R), if there exists a sequence (α f (j, l, z)) with |α f (j, l, z)| ≤ C2 -j(η+d/2) such that

f = log 2 (δ -1 ) j=0 2 d l=1 z∈{0,...,2 j -1} d αf (j, l, z) ψper jlz ,
then ∀j ≤ log 2 (δ -1 ) we have

|α f (j, l, z) -αf (j, l, z)| ≤ CL -1 2 -j(⌊β⌋+2+d/2) δ -(⌊β⌋+2-η) log 2 (δ -1 ) + 2 -j(η+d/2)
and ∀j > log 2 (δ -1 ) we have

|α f (j, l, z)| ≤ CL -1 2 -j(⌊β⌋+2+d/2) δ -(⌊β⌋+2-η) log 2 (δ -1 ).
The proof of Proposition A.2 follows the same arguments than the proof of the following proposition.

PROPOSITION A.3. For any f ∈ L 2 (R p , R), if there exists a sequence ( αf (j, l, w)) with |α f (j, l, w)| ≤ C2 -j(η+p/2) such that

f = log 2 (δ -1 ) j=0 2 p l=1 w∈{-K2 j ,...,K2 j } p αf (j, l, w) ψjlw , then ∀j ≤ log 2 (δ -1 ) we have |α f (j, l, w) -αf (j, l, w)| ≤ CL -1 (2 -j(⌊β⌋+2+p/2) δ -(⌊β⌋+2-η) log 2 (δ -1 ) + 2 -j(η+p/2) )
and ∀j > log 2 (δ -1 ) we have

|α f (j, l, w)| ≤ CL -1 2 -j(⌊β⌋+2+p/2) δ -(⌊β⌋+2-η) log 2 (δ -1 ).
PROOF. Let (j, l, w) ∈ {1, ..., log(δ -1 )} × {1, ..., 2 p } × {-K2 j , ..., K2 j } p , we have

∥ ψjlw -ψ jlw ∥ H ⌊β⌋+2 ≥ ∥ ψjlw -ψ jlw ∥ B ⌊β⌋+2 ∞,∞ = sup (j ′ ,l ′ ,w ′ ) 2 j ′ (⌊β⌋+2+p/2) |α ψjlw (j ′ , l ′ , w ′ ) -1 {(j ′ ,l ′ ,w ′ )=(j,l,w)} |. Furthermore ∥ ψjlw -ψ jlw ∥ H ⌊β⌋+2 = 2 jp/2 ∥ p i=1 ψli (2 j • i -w i ) - p i=1 ψ li (2 j • i -w i )∥ H ⌊β⌋+2 = 2 jp/2 ∥ p i=1 ( ψli (2 j • i -w i ) -ψ li (2 j • i -w i )) p r>i ψlr (2 j • r -w r ) p s<i ψ ls (2 j • s -w s )∥ H ⌊β⌋+2 ≤ 2 j(⌊β⌋+2+p/2) p i=1 ∥( ψli (• i -w i ) -ψ li (• i -w i )) p r>i ψlr (• r -w r ) p s<i ψ ls (• s -w s )∥ H ⌊β⌋+2 ≤ C2 j(⌊β⌋+2+p/2) L -1 , recalling that ∥ ψli -ψ li ∥ H ⌊β⌋+2 ≤ CL -1 for l i ∈ {0, 1}.
We deduce that for all (j ′ , l ′ , w ′ ) ̸ = (j, l, w) we have

|α ψjlw (j ′ , l ′ , w ′ )| ≤ C2 (j-j ′ )(⌊β⌋+2+p/2) L -1
and

|α ψjlw (j, l, w) -1| ≤ CL -1 . Let (j ′ , l ′ , w ′ ) ∈ N ⋆ 0 × {1, ..., 2 p } × Z p , as supp( ψjlw ), supp(ψ jlw ) ⊂ p i=1 [2 -j (w i -N ), 2 -j (w i + N )],
we have that if

p i=1 [2 -j (w i -N ), 2 -j (w i + N )] p i=1 [2 -j ′ (w ′ i -N ), 2 -j ′ (w ′ i + N )] = ∅, then ⟨ ψjlw , ψ j ′ l ′ w ′ ⟩ = 0. Let f = log 2 (δ -1 ) j=0 2 p
l=1 w∈{-K2 j ,...,K2 j } p αf (j, l, w) ψjlw ∈ Fη,δ , taking αf (j, l, w) := 0 for j ≥ log 2 (δ -1 ), we have

|α f (j ′ , l ′ , w ′ ) -αf (j ′ , l ′ , w ′ )| =| log 2 (δ -1 ) j=0 2 p l=1 w∈{-K2 j ,...,K2 j } p αf (j, l, w)⟨ ψjlw , ψ j ′ l ′ w ′ ⟩ -αf (j ′ , l ′ , w ′ )| ≤| (j,l,w)̸ =(j ′ ,l ′ ,w ′ ) |α f (j, l, w)⟨ ψjlw , ψ j ′ l ′ w ′ ⟩| + |α f (j ′ , l ′ , w ′ )||⟨ ψj ′ l ′ w ′ , ψ j ′ l ′ w ′ ⟩ -1|1 {j ′ ≤log 2 (δ -1 )} ≤ (j,l,w)̸ =(j ′ ,l ′ ,w ′ ) supp( ψjlw)∩supp(ψ j ′ l ′ w ′ )̸ =∅ C2 -j(η+p/2) 2 (j-j ′ )(⌊β⌋+2+p/2) L -1 + C2 -j ′ (η+p/2) L -1 1 {j ′ ≤log 2 (δ -1 )} =CL -1 2 -j ′ (⌊β⌋+2+p/2) (j,l,w)∈{1,...,log 2 (δ -1 )}×{1,...,2 p }×{-K2 j ,...,K2 j } p supp( ψjlw)∩supp(ψ j ′ l ′ w ′ )̸ =∅ 2 j(⌊β⌋+2-η) + C2 -j ′ (η+p/2) L -1 1 {j ′ ≤log 2 (δ -1 )} ≤CL -1 2 -j ′ (⌊β⌋+2+p/2) j∈{1,...,log 2 (δ -1 )} 2 j(⌊β⌋+2-η) + C2 -j ′ (η+p/2) L -1 1 {j ′ ≤log 2 (δ -1 )} ≤CL -1 (2 -j ′ (⌊β⌋+2+p/2) δ -(⌊β⌋+2-η) log 2 (δ -1 ) + 2 -j ′ (η+p/2) 1 {j ′ ≤log 2 (δ -1 )} ).
A.2.2. Implementation of the class Fβ+1,n -1 2β+d per (3.11). Let us first give a method to find a ReQu neural network φ from (A.4) that approximates the true scaling function at a precision n -1 . This method is largely inspired by the Section 4 of [START_REF] Daubechies | Two-scale difference equations. i. existence and global regularity of solutions[END_REF]. The idea is to use an iterative scheme based on the two-scale difference equation (A.3). The unique solution ϕ of this equation, is a fixed point fixed point V f = f of the linear operator

(A.11) V f (x) = N k=0 h k f (2x -k). Take a function f 0 ∈ H ⌊β⌋+3 (R, R) such that f (l) 0 (k) = ϕ (l) (k)
for all k ∈ {0, 1, ..., N } and l ∈ {0, ..., ⌊β⌋ + 3}. Then, writing V j f 0 for the application of the operator V a total of j times on the function f 0 , Theorem 4.2 in [START_REF] Daubechies | Two-scale difference equations. i. existence and global regularity of solutions[END_REF] states that: -l) , ∀l ∈ {0, ..., ⌊β⌋ + 3}. We are going to use ReQU neural networks of Belomestny et al. (2022) to approximate such a f 0 . Based on the data (k, ϕ (l) (k)) k∈{0,...,N } [START_REF] Daubechies | Two-scale difference equations ii. local regularity, infinite products of matrices and fractals[END_REF] give a precise method to compute the values ϕ (l) (k)), find a ReQU neural network f0 ∈ N ⌊β⌋+3 CN using classical gradient descent optimization on the weights of the network, such that (A.12) , 2). Then, the neural network fj = V j f0 , which is a linear combination of scalings and translations of f0 , verifies

For f j = V j f 0 , we have ∥f (l) j -ϕ (l) ∥ ∞ ≤ C2 -j(⌊β⌋+3
⌊β⌋+3 l=0 N k=0 | f (l) 0 (k) -ϕ (l) (k)| ≤ n -log 2 (Ch)-1 , for C h = max k∈{0,...,N } |h k | ∈ (1
∥ f (l) j -ϕ (l) ∥ ∞ ≤ C 2 -j(⌊β⌋+3-l) + C j h n -log 2 (Ch)-1 , ∀l ∈ {0, ..., ⌊β⌋ + 3}.
Finally, for j = log 2 (n) we obtain a ReQU neural network φ = fj such that ∥ϕ -φ∥ H ⌊β⌋+2 ≤ Cn -1 .

Note that as we need a smooth approximation of ϕ, we cannot use classical ReLU neural networks as it will not give us a function that also approximates the derivatives of ϕ. Having the approximation φ of the scaling function, we can compute the approximated periodised wavelets ψper jlz as described in the precedent Section (A.10). The class Fβ+1,n -1 2β+d per (3.11) can then be implemented as the set of neural networks being linear combinations of the ψper jlz , for j ∈ {0, ..., log 2 (n 1 2β+d )}, l ∈ {1, ..., 2 d } and z ∈ {0, ..., 2 j -1} d . Let us sum up the different steps of the construction and then detail their tractability.

1. Find by gradient descent a ReQU neural network f0 ∈ N ⌊β⌋+3 CN verifying (A.12). 2. Compute the neural network fj = V j f0 = k1,...,kj∈{0,...,N } j h k1 ...h kj f0 (2 j x -

j i=1 2 i-1 k i )
for j = log 2 (n).

3. Build p neural networks f i of the shape

f i = log 2 (n 1 2β+d ) j=0 2 d l=1 z∈{0,...,2 j -1} d αf (j, l, z) i ψper jlz ,
for ψper jlz from (A.10).

Step 1. Although we need to compute the derivatives of f0 up to the order ⌊β⌋ + 3, this is not very costly as f0 is a uni-dimensional neural network with width and depth that do not depend on the number of data. Furthermore, we only need to compute the error with ϕ (l) at the points k ∈ {1, ..., N } which also does not depend on the number of data. Finally we need to obtain an error O(n -log 2 (Ch)-1 ) which is strictly greater than O(n -2 ). To the best of our knowledge, there are not existing results on the number of operations it would take for a ReQU neural network to obtain such error. Nevertheless, if one would use classical splines (instead of neural network), then it would take a finite number of steps to have an error of 0 at the points k ∈ {1, ..., N }. Therefore, one could hope that it would take as most O(log(n)) operation for a ReQU neural network to obtain an error O(n -2 ).

Step 2. Computing the operator V only requires a finite number of products and sums. Therefore this step takes O(log(n)) operations.

Step 3. This step requires the computations of the ψper jlz , but for u ∈ [0, 1] d , and fixed j, there are only a finite number of ψper jlz (u) that are not 0. The computation of each ψper jlz using the neural network φ = flog(n) from step 2, is done in a finite time. Therefore this step takes O(log(n)) operations.

Note that once the neural network class Fβ+1,n -1 2β+d per has been implemented, it is then very easy to optimize it in the GAN loss (1.3) as the neural network depends linearly on the weights αf (j, l, z) i .

APPENDIX B: PROOFS OF THE TECHNICAL TOOLS OF SECTION 3

In this section, we gather the proofs of Section 3. We start by stating some preliminaries results of interpolation theory that will be used in the proofs. Then, we give the proofs of each result in the order in which they are stated in the paper.

B.1. Some useful interpolation inequalities. Let us state a general interpolation inequality that shows that we can trade of the regularity of the distributions in the L 2 scalar product.

PROPOSITION B.1. Let X be either R η or [0, 1] η /Z η , h 1 ∈ B s1,b1 ∞,∞ (X , R p , 1) and h 2 ∈ B s2,b2 ∞,∞ (X , R p , 1) for some s 1 , s 2 , b 1 , b 2 ∈ R.
Then for τ ∈ R, t, r ∈ [0, 1], q > 1 and 1/q + 1/q ⋆ = 1 we have

h 1 , h 2 L 2 (X ,R p ) ≤ Γtτ (h 1 ), Γ (1-t)τ (h 2 ) 1 q L 2 (X ,R p ) Γ-r q ⋆ q τ (h 1 ), Γ -(1-r) q ⋆ q τ (h 2 ) 1 q ⋆ L 2 (X ,R p )
where Γtτ (h 1 ) ∈ B s1+tτ,b1

∞,∞ and Γ-r q ⋆ q τ (h 1 ) ∈ B s1-r q ⋆ q ,b1 ∞,∞ are defined by ⟨ Γtτ (h 1 ) i , ψ jlz ⟩ = S(j, l, w) i ⟨Γ tτ (h 1 ) i , ψ jlz ⟩ and ⟨ Γ-t q ⋆ q τ (h 1 ) i , ψ jlz ⟩ = S(j, l, w) i ⟨Γ -r q ⋆ q τ (h 1 ) i , ψ jlz ⟩ with S(j, l, w) i ∈ {-1, 1} the sign of ⟨h 1i , ψ jlz ⟩⟨h 2i , ψ jlz ⟩, i = 1, ..., p.

PROOF. The proof of this proposition is a simple application of Hölder's inequality on the wavelets coefficients of h 1 and h 2 . For i ∈ {1, ..., p} and k ∈ {1, 2}, writing α hk (j, l, w) i = ⟨h ki , ψ j,l,w ⟩ L 2 (X ,R p ) we have

h 1 , h 2 L 2 (X ,R p ) = p i=1 ∞ j=0 2 η l=1 w∈Z η α h1 (j, l, w) i α h2 (j, l, w) i ≤ p i=1 ∞ j=0 2 η l=1 w∈Z η 2 j τ q |α h1 (j, l, w) i α h2 (j, l, w) i | 1/q 2 -j τ q |α h1 (j, l, w) i α h2 (j, l, w) i | 1/q ⋆ ≤   p i=1 ∞ j=0 2 η l=1 w∈Z η 2 jτ |α h1 (j, l, w) i α h2 (j, l, w) i |   1 q ×   p i=1 ∞ j=0 2 η l=1 w∈Z η 2 -τ q ⋆ q |α h1 (j, l, w) i α h2 (j, l, w) i |   1 q ⋆ = Γtτ (h 1 ), Γ (1-t)τ (h 2 ) 1 q L 2 (X ,R p ) Γ-r q ⋆ q τ (h 1 ), Γ -(1-r) q ⋆ q τ (h 2 ) 1 q ⋆ L 2 (X ,R p )
.

This proposition is a key result that we will use numerous times all along the paper. An useful corollary is the following. COROLLARY B.2. Let µ, ν be two probability measures with compact supports in R p . Then for all θ, θ 1 , θ 2 > 0 with θ 1 , θ 2 non integers such that θ 1 < θ < θ 2 , we have

sup D∈H θ 1 E X∼µ Y ∼ν [D(X) -D(Y )] ≤C sup D∈H θ 1 1 E X∼µ Y ∼ν [D(X) -D(Y )] θ 2 -θ θ 2 -θ 1 × sup D∈H θ 2 1 E X∼µ Y ∼ν [D(X) -D(Y )] θ-θ 1 θ 2 -θ 1 .
We will use this result in the proof of minimax optimality of the GAN estimator of Section 4. Before giving the proof of this result, let us first define the notion of envelope of a measure. DEFINITION B.3. Let µ be a probability measure compactly supported on R p . For ϵ ∈ (0, 1], we call the ϵ-envelope of µ, the probability measure µ ϵ defined for any Borel set A by

(B.1) µ ϵ (A) = 1 λ p (B p (0, ϵ)) R p λ p (A ∩ B(x, ϵ))dµ(x).
Then for γ > 0 and D ∈ H γ 1 , we have

R p D(x)(dµ -dµ ϵ )(x) = R p D(x) - 1 λ p (B p (0, ϵ)) B p (x,ϵ) D(y)dλ p (y) dµ(x) ≤ ϵ 1∧γ .
Furthermore, µ ϵ is absolutely continuous with respect to λ p and its density f ϵ µ equals to

f ϵ µ (y) = 1 λ p (B p (0, ϵ)) R p 1 {y∈B p (x,ϵ)} dµ(x)
≤ λ p (B p (0, ϵ)) -1 , so f ϵ µ is bounded and therefore belongs to L 2 . Let us now give the proof of Corollary B.2.

PROOF OF COROLLARY B.2. For ϵ ∈ (0, 1], let µ ϵ , ν ϵ be the ϵ-envelope of µ, ν respectively and f ϵ µ , f ϵ ν the densities of µ ϵ and ν ϵ . For all D ∈ H θ 1 we have

E X∼µ [D(X)] -E Y ∼ν [D(Y )] ≤ E Xϵ∼µϵ [D(X ϵ )] -E Yϵ∼νϵ [D(Y ϵ )] + 2ϵ 1∧θ = R p D(x)(f ϵ µ (x) -f ϵ ν (x))dλ p (x) + 2ϵ 1∧θ . Using Proposition B.1 for h 1 = D, h 2 = f ϵ µ -f ϵ ν , s 1 = θ, τ = θ -θ 1 , s 2 = b 1 = b 2 = 0, t = r = 1 and q = θ2-θ1 θ2-θ we have R p D(x)(f ϵ µ (x) -f ϵ ν (x))dλ p (x) ≤ R p Γθ-θ1 (D)(x)(f ϵ µ (x) -f ϵ ν (x))dλ p (x) θ 2 -θ θ 2 -θ 1 R p Γθ-θ2 (D)(x)(f ϵ µ (x) -f ϵ ν (x))dλ p (x) θ-θ 1 θ 2 -θ 1 ≤C sup D∈B θ 1 ∞,∞(1) R p D(x)(f ϵ µ (x) -f ϵ ν (x))dλ p (x) θ 2 -θ θ 2 -θ 1 sup D∈B θ 2 ∞,∞(1) R p D(x)(f ϵ µ (x) -f ϵ ν (x))dλ p (x) θ-θ 1 θ 2 -θ 1
.

From Lemma 3.3 we have that B θ1 ∞,∞ (1) ⊂ H θ1 C and for any D ∈ H θ1 C we have

R p D(x)(f ϵ µ (x) -f ϵ ν (x))dλ p (x) ≤ 2ϵ 1∧θ1 + E X∼µ [D(X)] -E Y ∼ν [D(Y )].
Doing the same thing for D ∈ H θ2 C and letting ϵ go to 0, we get the result.

Let us now show that we can gain some weak Besov regularity by paying a logarithmic term.

PROPOSITION B.4. Let X be either R p or T d , f ∈ L 1 (X , R, 1) ∩ L 2 (X , R) and g ∈ H γ 1 (X , R) with γ > 0.
Then for all τ > 0, ϵ ∈ (0, 1) we have

X f (x)g(x)dλ X (x) ≤ C log(ϵ -1 ) τ X f (x) Γ0,-τ ϵ (g)(x)dλ X (x) + C γ ϵ for C γ such that B γ/2 ∞,∞ (X , R, 1) ⊂ H 0 Cγ (X , R) and Γ0,-τ ϵ (g)(x) = log(ϵ -2/γ ) j=0 2 dim(X ) l=1 w∈Z dim(X )
(1 + j) -τ α g (j, l, w) α g (j, l, w)α f (j, l, w) |α g (j, l, w)α f (j, l, w)| ψ j,l,w (x).

PROOF. Let us write M the dimension of the space X . For k ≥ 1 we have

∞ j=k 2 M l=1 w∈Z M α f (j, l, w)α g (j, l, w) ≤ ∞ j=k 2 M l=1 w∈Z M C2 -j(γ+M/2) |α f (j, l, w)| ≤ C2 -kγ/2 ∞ j=k 2 M l=1 w∈Z M 2 -j(γ+M )/2 |α f (j, l, w)| = C2 -kγ/2 κ(x)f (x)dλ p (x) for κ(x) = ∞ j=k 2 M l=1 w∈Z M 2 -j(γ+M )/2 S(j, l, w)ψ j,l,w (x) with S(j, l, w) = αf (j,l,w) |αf (j,l,w)| . As κ ∈ B γ/2 ∞,∞ (1) ⊂ H 0 Cγ , we have κ(x)f (x)dλ p (x) ≤ ∥κ∥ ∞ ∥f ∥ L1 ≤ C γ .
Let us write γ ϵ = ⌊log 2 (ϵ -2/γ )⌋, we have

X f (x)g(x)dλ X (x) = ∞ j=0 2 M l=1 w∈Z M α g (j, l, w)α f (j, l, w) ≤ γϵ j=0 2 M l=1 w∈Z M α g (j, l, w)α f (j, l, w) + C γ ϵ ≤ (γ ϵ + 1) τ γϵ j=0 2 M l=1 w∈Z M (1 + j) -τ |α g (j, l, w)α f (j, l, w)| + C γ ϵ ≤ C log(ϵ -1 ) τ X f (x) Γ0,-τ ϵ (g)(x)dλ X (x) + C γ ϵ B.
2. Proof of the bias-variance trade-off for GANs (Section 3.1). Let us define some useful notations for the GAN estimator. DEFINITION B.5. For D ∈ H γ 1 and g ∈ H β+1 K (T d , R p ) define: • the discrimination score under D of the push-forward measure g #U and g ⋆ #U as L(g, D)

:= E U ∼U ([0,1] d ) [D(g(U )) -D(g ⋆ (U ))],
• the discretized version of the discrimination score

L n (g, D) := 1 n n i=1 D(X i ) -D(g(U i )), with U i ∼ U([0, 1] d ) and X i ∼ g ⋆ #U i.i.d. random variables, i = 1, ..., n, • the optimal discriminator in H γ 1 between g #U D ⋆ g ∈ arg max D∈H γ 1 L(g, D),
• the optimal discriminator in D of the empirical approximation of the expectation

Dg ∈ arg max D∈D L n (g, D).
When the context is clear we will write

E[D(g(U ))] instead of E U ∼U ([0,1] d ) [D(g(U ))].
B.2.1. Proof of Theorem 3.1. Let us state a first proposition that gives a bound on the expected difference between E[D(g(U ))] and its empirical approximation

1 n n i=1 D(g(U i )),
for all g ∈ G and D ∈ D.

PROPOSITION B.6. If G ⊂ H β+1 K (T d , R p ) and D ⊂ H γ 1 (B p (0, K), R), for i.i.d. random variables U and U i that are uniform on [0, 1] d , we have E Ui sup g∈G D∈D E U [D(g(U )] - 1 n n i=1 D(g(U i )) ≤ C min δ∈[0,1] (δ + 1/n) 2 log(n|G 1/n ||D 1/n |) n + 1 √ n (1 + δ (1-d 2γ ) + log(δ -1 )1 {2γ=d} ) .
PROOF. We are first going to use Dudley's inequality bound (Vaart and Wellner (2023), chapter 2). For g ∈ H β+1 K and D ∈ H γ 1 , we have from the Faa di Bruno formula that D

• g ∈ H γ C . Let us take Θ = {D • g ∈ H γ C | D ∈ D, g ∈ G} equipped with the metric M = ∥ • ∥ ∞ . For θ = D • g ∈ Θ,
we define the zero-mean sub-Gaussian process (e.g. (van Handel, 2016), chapter 5) as

X θ = 1 √ n n i=1 D(g(U i )) - E[D(g(U ))] with U i ∼ U([0, 1] d ) i.i.d. Then as sup θ,θ ′ ∈Θ M (θ, θ ′ ) ≤ 2, using Dudley's bound we obtain ∀δ ∈ [0, 2], E sup g∈G D∈D E[D(g(U )] - 1 n n i=1 D(g(U i )) ≤2E sup g∈G, D∈D ∥D•g∥∞≤δ E[D(g(U ))] - 1 n n i=1 D(g(U i )) + 16 √ n 2 δ/4 log(|Θ ϵ |)dϵ. (B.2)
Let us take care of the first term of (B.2). Let G 1/n and D 1/n be minimal 1/n covering of the class G and D for the distance ∥ • ∥ ∞ . Therefore, for any g ∈ G, D ∈ D, ∃g n ∈ G 1/n and

D n ∈ D 1/n such that ∥g -g n ∥ ∞ ≤ 1 n and ∥D -D n ∥ ∞ ≤ 1 n . As H γ 1 ⊂ Lip 1 , we have for α > 0 P sup g∈G, D∈D ∥D•g∥∞≤δ E[D(g(U ))] - 1 n n i=1 D(g(U i )) ≥ α ≤ P sup g∈G, D∈D ∥D•g∥∞≤δ E[D n (g n (U ))] - 1 n n i=1 D n (g n (U i )) ≥ α - 4 n ≤ P ∃ gn∈G1/n,Dn∈D1/n ∥Dn•gn∥∞≤δ+ 2 n with E[D n (g n (U ))] - 1 n n i=1 D n (g n (U i )) ≥ α - 4 n ≤ gn∈G1/n,Dn∈D1/n ∥Dn•gn∥∞≤δ+ 2 n P E[D n (g n (U ))] - 1 n n i=1 D n (g n (U i )) ≥ α - 4 n ≤ |G 1/n ||D 1/n | exp - n(α -4 n ) 2 2(δ + 2 n ) 2
, applying Hoeffding's inequality to the random variables

Z i = D n (g n (U i )), i = 1, ..., n. Then for τ ∈ (0, 1], solving |G 1/n ||D 1/n | exp - n(α-4 n ) 2 2(δ+ 2 n ) 2
= τ with respect to α, we obtain that with probability at least 1 -τ

sup g∈G, D∈D ∥D•g∥∞≤δ E[D(g(U ))] - 1 n n i=1 D(g(U i )) ≤ C (δ + 1/n) 2 log(|G 1/n ||D 1/n |/τ ) n + 1 n .
Taking τ = 1/n, we deduce that

E sup g∈G, D∈D ∥D•g∥∞≤δ E[D(g(U ))] - 1 n n i=1 D(g(U i )) ≤ C (δ + 1/n) 2 log(|G 1/n ||D 1/n |n) n + 1 n .
For the second term of (B.2), as Θ ⊂ H

) ϵ |) ≤ Cϵ -d γ so 16 √ n 2 δ/4 log(|Θ ϵ |)dϵ ≤ C √ n 2 δ/4 ϵ -d 2γ dϵ = C1 {2γ̸ =d} √ n(1 -d 2γ ) 2 1-d 2γ - δ 4 (1-d 2γ ) + C1 {2γ=d} √ n (log(2) -log(δ/4)).
Therefore we have

16 √ n 2 δ/4 log(|Θ ϵ |)dϵ ≤ C √ n (1 + δ (1-d 2γ ) + log(4δ -1 )1 {2γ=d} ).
We see that having small classes of functions allows the empirical approximation to be close to the expectation. As we would like our estimator ĝ ∈ arg min

g∈G max D∈D n i=1 D(X i ) - D(g(U i )) to obtain a small error E U ∼U ([0,1] d ) [D(ĝ(U )) -D(g ⋆ (U ))] for all D ∈ D, it is then crucial that E sup g∈G D∈D E[D(g(U )] -1 n n i=1 D(g(U i ))
L(ĝ, D ⋆ ĝ ) =L(ĝ, D ⋆ ĝ ) -L(ĝ, D ĝ) + L(ĝ, D ĝ) -L(g, D ⋆ g ) + L(g, D ⋆ g ) ≤∆ D + L(ĝ, D ĝ) -L n (ĝ, D ĝ) + L n (ĝ, D ĝ) -L(g, D g ) + L(g, D g ) -L(g, D ⋆ g ) + ∆ G ≤∆ D + L(ĝ, D ĝ) -L n (ĝ, D ĝ) + L n (ĝ, Dĝ ) -L(g, D g ) + ∆ G ≤∆ D + L(ĝ, D ĝ) -L n (ĝ, D ĝ) + L n (g, Dg ) -L(g, D g ) + ∆ G ≤∆ D + L(ĝ, D ĝ) -L n (ĝ, D ĝ) + L n (g, Dg ) -L(g, Dg ) + ∆ G
using the definition of ĝ and D. Then, we have

E L(ĝ, D ĝ) -L n (ĝ, D ĝ) =E E[D ĝ(ĝ(U )) -D ĝ(g ⋆ (U ))] - 1 n n i=1 D ĝ(ĝ(U i )) -D ĝ(g ⋆ (Y i )) ≤E sup g∈G D∈D E[D(g(U )] - 1 n n i=1 D(g(U i )) + E sup D∈D E[D(g ⋆ (U )] - 1 n n i=1 D(g ⋆ (Y i )) .
The first term of the precedent inequality can be controlled by Proposition B.6. As Y i ∼ U([0, 1] d ), doing the same derivation as in the proof of Proposition B.6 for a fixed g ⋆ ∈ H β+1 1 , we can also bound the second term by the same quantity. Then, doing the same thing for E L n (g, Dg ) -L(g, Dg ) , we get the result. B.2.2. Proof of Proposition 3.2. This result is already well known (Theorem 4.2 in De-Vore et al. ( 1989)) but we give a new proof in the case η non integer using wavelet decomposition. The case η integer can also be proven the same way but at the cost of losing a factor log(ϵ) 2 coming from the fact that B η ∞,∞ does not inject in

H η but B η,2 ∞,∞ does. PROOF OF PROPOSITION 3.2. Let f ∈ H η 1 and g ∈ F , as H 0 - → B 0 ∞,∞ we have that there exists C 0 > 0 such that ∥f -g∥ ∞ ≥ C -1 0 ∥f -g∥ B 0 ∞,∞ = C -1 0 sup j≥0 2 jd/2 2 d l=1 sup z∈{0,...,2 j -1} d |α g (j, l, z) -α f (j, l, z)|.
Therefore, ∀f ∈ H η 1 , there exists g ∈ F such that

2 d l=1 2 jd/2 |α g (j, l, z) -α f (j, l, z)| ≤ C 0 ∥f -g∥ ∞ ≤ C 0 ϵ,
for all j ≥ 0 and z ∈ {0, ..., 2 j -1} d . On the other hand, as

B η ∞,∞ = H η , there exists C η > 0 such that, if f verifies 2 d l=1 |α f (j, l, z)| ≤ C η 2 -j(η+d/2) , ∀j ≥ 0, z ∈ {0, ..., 2 j -1} d , then f ∈ H η 1 .
Therefore, for all k ∈ {0, ..., ⌊2 -jη-(d+2) C η (C 0 ϵ) -1 ⌋} 2 d , as

2 d l=1 k l 2 2-jd/2 C 0 ϵ ≤ 2 d l=1 C η 2 -j(η+d/2)-d ≤ C η 2 -j(η+d/2) ,
then there exists g ∈ F such that for all j ≥ 0, z ∈ {0, ..., 2 j -1} d ,

2 jd/2 2 d l=1 |α g (j, l, z) -k l 2 2-jd/2 C 0 ϵ| ≤ C 0 ϵ.
Furthermore, we have

2 jd/2 2 d l=1 |α g (j, l, z) -k l 2 2-jd/2 C 0 ϵ| ≤ C 0 ϵ implies 2 jd/2 |α g (j, l, z) -k l 2 2-jd/2 C 0 ϵ| ≤ C 0 ϵ, ∀l ∈ {1, ..., 2 d },
which implies

2 jd/2 |α g (j, l, z) -(k l + 1)2 2-jd/2 C 0 ϵ| > C 0 ϵ, ∀l ∈ {1, ..., 2 d }.
Then for each sequence

α ∈ log 2 (⌊(C -1 η C0ϵ) -1 η ⌋)-d/η j=1 z∈{0,...,2 j -1} d {0, ..., ⌊2 -jη-(d+2) C η (C 0 ϵ) -1 ⌋} 2 d ,
there exists a different g ∈ F such that ∥g -

log 2 (⌊(C -1 η C0ϵ) -1 η ⌋) j=1 2 d l=1 z∈{0,...,2 j -1} d α jlz C 0 2 2-jd/2 ϵ ψ per jlk ∥ ∞ ≤ ϵ.
We stop at the frequency

j = log 2 (⌊(C -1 η C 0 ϵ) -1 η ⌋) -d/η, as for f ∈ H η 1 and j ≥ log 2 (⌊(C -1 η C 0 ϵ) -1 η ⌋) -d/η we have 2 jd/2 2 d l=1 |α f (j, l, z)| ≤ 2 jd/2 C η 2 -j(η+d/2) = C η 2 d-jη = C 0 ϵ,
so F does not have to cover these frequencies. Then for θ ∈ (0, C0ϵ 3 ), the θ-covering number of F verifies

|F θ | ≥ log 2 (⌊(C -1 η C0ϵ) -1 η ⌋)-(d+3)/η j=0 z∈{0,...,2 j -1} d ⌊2 -jη-(d+2) C η (C 0 ϵ) -1 ⌋ 2 d ≥ z∈{0,...,2 log 2 (⌊(C -1 η C 0 ϵ) -1 η ⌋)-(d+3)/η } d 2 = z∈{0,...,⌊(C -1 η C0ϵ) -1 η ⌋2 -(d+3)/η } d 2 ≥ C2 ⌊(C -1 η C0ϵ) -1 η ⌋ d .
We then deduce that

log(|F θ |) ≥ Cϵ -d η .
B.3. Proofs of the results on the properties of F η,δ per (Section 3.2).

B.3.1. Proof of Lemma 3.5.

PROOF. Let g ∈ B η ∞,∞ (K) and D ∈ H γ 1 , using a Taylor expansion we have

E D(g(U )) -D(g ⋆ (U )) = E 1 0 ⟨∇D(g ⋆ (U ) + t(g(U ) -g ⋆ (U ))), g(U ) -g ⋆ (U )⟩dt = 1 0 p i=1 E ∂ i D(g ⋆ (U ) + t(g(U ) -g ⋆ (U )))(g(U ) -g ⋆ (U )) i dt.
Let us fix a t ∈ [0, 1] and write the functions in their (periodized) wavelets expansions:

D t i (u) := ∂ i D(g ⋆ (u) + t(g(u) -g ⋆ (u))) = ∞ j=0 2 d l=1 z∈{0,...,2 j -1} d α D t i (j, l, z)ψ per jlz (u), g(u) i = ∞ j=0 2 d l=1 z∈{0,...,2 j -1} d α g (j, l, w) i ψ per jlz (u),
and

g ⋆ (u) i = ∞ j=0 2 d l=1 z∈{0,...,2 j -1} d α g ⋆ (j, l, w) i ψ per jlz (u). As g -g ⋆ ∈ H η K ⊂ B η ∞,∞ (C) and D t i ∈ H γ-1 C ⊂ B γ-1 ∞,∞ (
C), we have that for all i ∈ {1, ...p} and j ≥ 0, -1+η+d) . Then, using the wavelet expansions we get

α D t i (j, l, z)(α g (j, l, w) i -α g ⋆ (j, l, w) i ) ≤ C2 -j(γ
E ∂ i D(g ⋆ (U ) + t(g(U ) -g ⋆ (U )))(g(U ) -g ⋆ (U )) i = ∞ j=0 2 d l=1 z∈{0,...,2 j -1} d α D t i (j, l, z)(α g (j, l, w) i -α g ⋆ (j, l, w) i ) ≤ log 2 (⌊ϵ -1 ⌋) j=0 2 d l=1 z∈{0,...,2 j -1} d α D t i (j, l, z)(α g (j, l, w) i -α g ⋆ (j, l, w) i ) + ∞ j=log 2 (⌊ϵ -1 ⌋)+1 2 d l=1 z∈{0,...,2 j -1} d C2 -j(γ-1+η+d) ≤ log 2 (⌊ϵ -1 ⌋) j=0 2 d l=1 z∈{0,...,2 j -1} d α D t i (j, l, z)(α g (j, l, w) i -α g ⋆ (j, l, w) i ) + Cϵ η+γ-1 .
Applying twice the Cauchy-Schwarz inequality we get

log 2 (⌊ϵ -1 ⌋) j=0 2 d l=1 z∈{0,...,2 j -1} d α D t i (j, l, z)(α g (j, l, w) i -α g ⋆ (j, l, w) i ) ≤ log 2 (⌊ϵ -1 ⌋) j=0   2 d l=1 z∈{0,...,2 j -1} d |α D t i (j, l, z)| 2   1/2 ×   2 d l=1 z∈{0,...,2 j -1} d |α g (j, l, w) i -α g ⋆ (j, l, w) i | 2   1/2 ≤   log 2 (⌊ϵ -1 ⌋) j=0 2 2j(γ-1) 2 d l=1 z∈{0,...,2 j -1} d |α D t i (j, l, z)| 2   1/2 ×   log 2 (⌊ϵ -1 ⌋) j=0 2 -2j(γ-1) 2 d l=1 z∈{0,...,2 j -1} d |α g (j, l, w) i -α g ⋆ (j, l, w) i | 2   1/2 ≤   log 2 (⌊ϵ -1 ⌋) j=0 2 2j(γ-1) 2 d l=1 z∈{0,...,2 j -1} d |C2 -j(γ-1+d/2) | 2   1/2 × ∥Γ -γ+1 (g i ) -Γ -γ+1 (g ⋆ i )∥ B 0 2,2 ≤C log 2 (⌊ϵ -1 ⌋) 1/2 ∥Γ -γ+1 (g i ) -Γ -γ+1 (g ⋆ i )∥ L 2 ,
as ∀s ≥ 0, B s 2,2 = W s,2 the L 2 Sobolev space of regularity s.

B.3.2. Proof of Proposition 3.6.

PROOF. For f ∈ F η,δ per we have f ∈ H η C log(δ -1 ) 2 (we do not distinguish between η integer or not) so using Lemma 3.5 with ϵ = δ, for any f ⋆ ∈ H η K we have

d H γ 1 (f #U , f ⋆ #U ) ≤ C log(δ -1 ) C2 p i=1 ∥Γ -(γ-1) (f i ) -Γ -(γ-1) (f ⋆ i )∥ L 2 + Cδ η-1+γ .
Then, for f such that ∀j ≤ log 2 (δ -1 ) and i ∈ {1, ..., p}, α fi (j, l, z) = α f ⋆ i (j, l, z), we have -1+η) .

∥Γ -(γ-1) (f i ) -Γ -(γ-1) (f ⋆ i )∥ 2 L 2 ≤ C∥Γ -(γ-1) (f i ) -Γ -(γ-1) (f ⋆ i )∥ 2 B 0 2,2 = C ∞ j=0 2 d l=1 z∈{0,...,2 j -1} d 2 -2j(γ-1) |α fi (j, l, z) -α f ⋆ i (j, l, z)| 2 = C ∞ j≥log 2 (δ -1 ) 2 d l=1 z∈{0,...,2 j -1} d 2 -2j(γ-1) |α f ⋆ i (j, l, z)| 2 ≤ C ∞ j≥log 2 (δ -1 ) 2 -2j(γ+η-1) ≤ C2 -log 2 (δ -1 )2(γ-1+η) = Cδ 2(γ
Let us now bound the covering number for a certain ϵ ∈ (0, 1). Define

(F η,δ per ) ϵ = {f ∈ F η,δ per | ∀i, j, l, z, ∃k ∈ [-KC η ϵ -1 2 -jη+d (1 + j) 2 , ..., KC η ϵ -1 2 -jη+d (1 + j) 2 ], ⟨f i , ψ per j,l,z ⟩ L 2 = kϵ (1 + j) 2 2 jd/2+d }.
Then for f ∈ F η,δ per and f ϵ ∈ (F η,δ per ) ϵ such that

⟨(f ϵ ) i , ψ per j,l,z ⟩ L 2 = ϵ (1 + j) 2 2 jd/2+d ⌊⟨f i , ψ per j,l,z ⟩ L 2 ϵ -1 (1 + j) 2 2 jd/2+d ⌋,
we have

∥f i -(f ϵ ) i ∥ ∞ ≤ C∥f i -(f ϵ ) i ∥ B 0,2 ∞,∞ = C sup j∈N0 2 jd/2 2 d l=1 sup z∈{0,...,2 j -1} d (1 + j) 2 |α fi (j, l, z) -α (fϵ)i (j, l, z)| ≤ C sup j∈N0 2 jd/2 2 d l=1 sup z∈{0,...,2 j -1} d (1 + j) 2 ϵ (1 + j) 2 2 jd/2+d = Cϵ so (F η,δ per ) ϵ is an Cϵ covering of F η,δ per . Furthermore (F η,δ per ) ϵ = log 2 (⌊δ -1 ⌋) j=0 2 d l=1 z∈{0,...,2 j -1} d 2 -jη+d+1 (1 + j) 2 C η Kϵ -1 ≤ C log 2 (⌊δ -1 ⌋) j=0 2 d l=1 z∈{0,...,2 j -1} d ϵ -1 = C log 2 (⌊δ -1 ⌋) j=0 (ϵ -1 ) 2 jd+d ≤ C(ϵ -1 ) log 2 (⌊δ -1 ⌋)2 log 2 (⌊δ -1 ⌋)d+d = C(ϵ -1 ) log 2 (⌊δ -1 ⌋)2 d δ -d , so log( (F η,δ per ) ϵ ) ≤ Cδ -d log(⌊δ -1 ⌋) log(ϵ -1 ). B.3.3. Proof of Proposition 3.7.
PROOF. For g ∈ Fη,δ per and D ∈ H γ 1 , we have

E[D(g(U )) -D(g ⋆ (U ))] = E[⟨ 1 0 ∇D(g(U ) + t(g ⋆ (U ) -g(U ))dt, g ⋆ (U ) -g(U )⟩] ≤ C sup f ∈H γ-1 1 E[⟨f (U ), g ⋆ (U ) -g(U )⟩].
Then the rest of the proof proceeds as the proof of Proposition 4.3 Section C.2.2.

APPENDIX C: PROOFS OF THE BOUNDS OF SECTION 4

C.1. Proof of the bound in the general low dimensional case (Section 4.1). In this section, we give the proofs of the results of Section 4 in the order in which they are stated in the paper.

C.1.1. Proof of Theorem 4.1.

PROOF. Using Proposition 3.7 for η = β + 1, δ = n -1 2β+d and ϵ = 1/n we have that

∆ G ≤ C log(n) C2 n -β+γ 2β+d and log(|G 1/n |) ≤ Cn d 2β+d log(n) 2 .
Let us now take care of ∆ D . Write g ∈ arg min g∈G L(g, D ⋆ g ), then for g ∈ G, as D ⋆ g,g ∈ D, we have

sup D∈D L(g, D ⋆ g ) -L(g, D) ≤L(g, D ⋆ g ) -L(g, D ⋆ g,g ) =E D ⋆ g (g(U )) -D ⋆ g (g ⋆ (U )) -D ⋆ g,g (g(U )) + D ⋆ g,g (g ⋆ (U )) =E D ⋆ g (g(U )) -D ⋆ g (g(U )) -D ⋆ g,g (g(U )) + D ⋆ g,g (g(U )) + E D ⋆ g (g(U )) -D ⋆ g (g ⋆ (U )) -D ⋆ g,g (g(U )) + D ⋆ g,g (g ⋆ (U )) ≤E D ⋆ g (g(U )) -D ⋆ g (g(U )) -D ⋆ g,g (g(U )) + D ⋆ g,g (g(U )) + 2∆ G ≤2∆ G ≤ C log(n) C2 n -β+γ 2β+d .
On the other hand, by definition of D, we have that

|D 1/n | ≤ |D| ≤ |G 1/n | 2 . Using Theorem 3.1 we have E L(ĝ, D ⋆ ĝ ) ≤C min δ∈[0,1] (δ + 1/n) 2 log(n|G 1/n ||D 1/n |) n + 1 √ n (1 + δ (1-d 2γ ) + log(δ -1 )1 {2γ=d} ) + ∆ G + ∆ D ≤C min δ∈[0,1] (δ + 1/n) 2 n d 2β+d n + 1 √ n (1 + δ (1-d 2γ ) + log(δ -1 )1 {2γ=d} ) + log(n) C2 n -β+γ 2β+d .
Now taking δ = n -γ 2β+d we have

(δ + 1/n) 2 n d 2β+d n ≤ 2n -γ 2β+d n d/2 2β+d n -β+d/2 2β+d = 2n -β+γ 2β+d and 1 √ n (1 + δ (1-d 2γ ) ) = n -1 2 + n -γ 2β+d γ-d/2 γ -β+d/2 2β+d = n -1 2 + n -β+γ 2β+d ,
so we finally obtain

E L(ĝ, D ⋆ ĝ ) ≤ C log(n) C2 n -β+γ 2β+d ∨ n -1 2 .
C.2. Proofs of the bound in the full dimensional case (Section 4.2).

C.2.1. Proof of Corollary 4.2.

PROOF. Following the proof of Proposition B.6, for

Θ = {Df ∈ H γ C | D ∈ D, f ∈ F } and X θ = 1 √ n n i=1 D(U i )f (U i ) -D(x)f (x)dλ p (x),
with U i i.i.d. uniform random variables on B p (0, K), we get using Dudley's inequality ∀δ ∈ [0, 2K],

E sup f ∈F D∈D D(x)f (x)dλ p (x) - n i=1 D(U i )f (U i ) ≤2E sup f ∈G, D∈D ∥Df ∥∞≤δ D(x)f (x)dλ p (x) - 1 n n i=1 D(U i )f (U i ) + 16 √ n 2K δ/4 log(|Θ ϵ |)dϵ.
Doing as in the proof of Proposition 3.6, we can show that log

(|(B γ ∞,∞ ) ϵ |) ≤ ϵ -d γ log(ϵ -1
). The rest of the proof follows the same arguments than the end of the proof of Proposition B.6 and the proof of Theorem 3.1 (see Section B.2.1). C.2.2. Proof of Proposition 4.3 . First, let us give the approximation error and covering number of the class F η,δ . PROPOSITION C.1. We have

∆ F η,δ := sup h ⋆ ∈H η 1 inf f ∈F η,δ sup D∈H γ 1 R p D(x)(f (x) -h ⋆ (x))dλ p (x) ≤ Cδ η+γ ,
and ∀ϵ ∈ (0, 1),

log(|F η,δ | ϵ ) ≤ Cδ -p log(⌊δ -1 ⌋) log(ϵ -1 ). PROOF. Let h ⋆ ∈ H η
1 and f ∈ F η,δ be such that for all j ≤ log 2 (δ -1 ), ⟨h ⋆ , ψ j,l,w ⟩ = ⟨f, ψ j,l,w ⟩. Then for D ∈ H γ 1 we have

R p D(x)(f (x) -h ⋆ (x))dλ p (x) = ∞ j=0 2 p l=1 w∈Z p α D (j, l, w)(α f (j, l, w) -α h ⋆ (j, l, w)) = ∞ j≥log2(δ -1 ) 2 p l=1 w∈Z p α D (j, l, w)α h ⋆ (j, l, w) ≤ ∞ j≥log2(δ -1 ) 2 p l=1 w∈Z p K2 -j(p+η+γ) 1 {supp(ψjlw)∩B p (0,K)̸ =∅} ≤ ∞ j≥log2(δ -1 ) C2 -j(η+γ) = Cδ η+γ .
For the bound on the covering number see the proof of Proposition 3.6, Section B.3.2.

We can now give the proof of Proposition 4.3.

PROOF OF PROPOSITION 4.3. Let h ⋆ ∈ H η 1 and define f ∈ Fη,δ by f = log 2 (δ -1 ) j=0 2 p l=1 w∈{-K2 j ,...,K2 j } p α h ⋆ (j, l, w) ψjlw ,
for α h ⋆ (j, l, w) = ⟨h ⋆ , ψ j,l,w ⟩. Then, from Proposition A.3 there exist coefficients (α f (j, l, w)) such that

f = ∞ j=0 2 p l=1 w∈{-K2 j ,...,K2 j } p α f (j, l, w)ψ jlw ,
with ∀j ≤ log 2 (δ -1 ),

|α f (j, l, w) -αf (j, l, w)| ≤ CL -1 (2 -j(⌊β⌋+2+p/2) δ -(⌊β⌋+2-η) log 2 (δ -1 ) + 2 -j(η+p/2) )
and ∀j > log 2 (δ -1 ),

|α f (j, l, w)| ≤ CL -1 2 -j(⌊β⌋+2+p/2) δ -(⌊β⌋+2-η) log 2 (δ -1 ).
As L ≥ δ -(⌊β⌋+2+γ) , we have in particular that

|α f (j, l, w) -α h ⋆ (j, l, w)| ≤ C2 -j(η+p/2) δ η+γ , for j ≤ log 2 (δ -1 )
and

|α f (j, l, w)| ≤ C2 -j(⌊β⌋+2+p/2) δ η+γ , for j > log 2 (δ -1 ).
Therefore, for D ∈ H γ 1 we have

R p D(x)(f (x) -h ⋆ (x))dλ p (x) = ∞ j=0 2 p l=1 w∈Z p α D (j, l, w)(α f (j, l, w) -α h ⋆ (j, l, w)) = log2(δ -1 ) j=0 2 p l=1 w∈Z p |α D (j, l, w)(α f (j, l, w) -α h ⋆ (j, l, w))| + ∞ j>log2(δ -1 ) 2 p l=1 w∈Z p |α D (j, l, w)(α f (j, l, w) -α h ⋆ (j, l, w))| ≤C ∞ j≤log2(δ -1 ) 2 p l=1 w∈Z p 2 -j(γ+p/2) 2 -j(η+p/2) δ η+γ 1 {supp(ψjlw)∩B p (0,K)̸ =∅} + ∞ j>log2(δ -1 ) 2 p l=1 w∈Z p |α D (j, l, w)|(|α f (j, l, w)| + |α h ⋆ (j, l, w)|) ≤Cδ η+γ + C ∞ j>log2(δ -1 ) 2 p l=1 w∈Z p 2 -j(γ+p/2)
(2 -j(⌊β⌋+2+p/2) δ η+γ + 2 -j(η+p/2) )1 {supp(ψjlw)∩B p (0,K)̸ =∅} ≤Cδ η+γ .

Let us now prove the bound on the covering number. Let

F = {f ∈ B η ∞,∞ (B p (0, K), R, K)| |α f (j, l, w)| ≤ CL -1 2 -j(⌊β⌋+2+p/2) δ -(⌊β⌋+2-η) log 2 (δ -1 ), ∀j > log 2 (δ -1 )}.
Then from Proposition A.3, we have that ∀ϵ ∈ (0, 1),

log(|( Fη,δ ) ϵ |) ≤ C log(|(F) ϵ |).
Let ϵ > L -1 δ η-1 and (F η,δ ) ϵ an ϵ-covering of F η,δ . For all f ∈ F , writing

f = log2(δ -1 ) j=0 2 p l=1 w∈Z p α f (j, l, w)ψ jlw , we have f ∈ F η,δ . Then there exist f ϵ ∈ (F η,δ ) ϵ such that ∥ f -f ϵ ∥ ∞ ≤ ϵ.
Then,

∥f -f ϵ ∥ ∞ ≤ ϵ + ∥f -f ∥ ∞ ≤ ϵ + C∥f -f ∥ B 0,2 ∞,∞ = ϵ + C sup j≥log2(δ -1 ) (1 + j) 2 2 jp/2 2 d l=1 sup w∈Z p |α f (j, l, w)| ≤ ϵ + C sup j≥log2(δ -1 ) (1 + j) 2 2 jp/2 2 d l=1 L -1 2 -j(⌊β⌋+2+p/2) δ -(⌊β⌋+2-η) log 2 (δ -1 ) ≤ ϵ + C log(δ -1 ) C2 L -1 δ η .
Therefore we deduce that (F η,δ ) ϵ is a Cϵ-covering of |F| so

log(|(F) ϵ |) ≤ C log(|(F η,δ ) ϵ |).
Finally, using Proposition C.1 we get the result.

C.2.3. Proof of Corollary 4.4.

PROOF. Let D ∈ B γ,2 ∞,∞ (1), then applying Proposition B.1 for h 1 = D, h 2 = f -f ⋆ , τ = γ -α, t = 1, r = γ
β+γ and q = β+α β+γ , we get

R p D(x)(f (x) -f ⋆ (x))dλ p (x) ≤ R p Γγ-α (D)(x)(f (x) -f ⋆ (x))dλ p (x) β+γ β+α × R p Γγ (D)(x)Γ β (f -f ⋆ )(x)dλ p (x) α-γ β+α . Furthermore, R p Γγ (D)(x)Γ β (f -f ⋆ )(x)dλ p (x) = ∞ j=0 2 p l=1 w∈Z p 2 j(β+γ) |α D (j, l, w)(α f (j, l, w) -α f ⋆ (j, l, w))| ≤ ∞ j=0 2 p l=1 w∈Z p 2 -jp (1 + j) -2 C1 {supp(ψjlw)∩B p (0,K)̸ =∅} ≤ C ∞ j=0 (1 + j) -2 .
C.2.4. Proof of Lemma 4.5.

PROOF. From Proposition

A.3 we have that Fβ,n -1 2β+p ⊂ B β ∞,∞ (R p , R, C). Then, for f ∈ F, D ⋆ ∈ H β 1 and D ∈ D, we have R p (D ⋆ (x) -D(x))(f (x) -f ⋆ (x))dλ p (x) = ∞ j=0 2 p l=1 w∈Z p (α D ⋆ (j, l, w) -α D (j, l, w))(α f (j, l, w) -α f ⋆ (j, l, w)) ≤ ∞ j=0 2 p l=1 w∈Z p C2 -jp/2 2 -jβ (α D ⋆ (j, l, w) -α D (j, l, w)).
The map h : R p → R having wavelets coefficients

α h (j, l, w) = 2 -jp/2 , belongs to B 0 ∞,∞
which gives the result.

C.2.5. Proof of Proposition 4.6.

PROOF. We bound ∆ D following the proof of Proposition C.1. For f ∈ F and D f ∈ D such that ⟨D f , ψ j,l,w ⟩ = ⟨D ⋆ f , ψ j,l,w ⟩ for all j ≤ log 2 (n 1 2β+p ), we have

R p (D f (x) -D ⋆ f (x))(f (x) -f ⋆ (x))dλ p (x) = ∞ j≥log 2 (n 1 2β+p ) 2 p l=1 w∈Z p (α D ⋆ f (j, l, w)(α f (j, l, w) -α f ⋆ (j, l, w)) ≤ Cn -β+ β 2β+p .
Furthermore, from Proposition 4.3 we have

∆ G ≤ Cn -β+ β 2β+p and log(|G 1/n ||D 1/n |) ≤ log(n) 2 n p 2β+p .
Then, we get the result using Corollary 4.2 with δ = n -β 2β+p .

C.2.6. Proof of Proposition 4.7.

PROOF. If β ≥ β and γ ∈ [ β, β] we have E d H γ 1 ( f , f ⋆ ) ≤ E d H β 1 ( f , f ⋆ ) ≤ C log(n) C2 n -1 2 ,
using Proposition 4.6. Otherwise, for D ∈ H γ 1 , we have

R p D(x)(f (x) -f ⋆ (x))dλ p (x) = ∞ j=0 2 p l=1 w∈Z p α D (j, l, w)(α f (j, l, w) -α f ⋆ (j, l, w)) ≤ log(⌊n⌋) j=0 2 p l=1 w∈Z p α D (j, l, w)(α f (j, l, w) -α f ⋆ (j, l, w)) + ∞ j=log(⌊n⌋)+1 2 p l=1 w∈Z p 2 -j(p+β+γ) K1 {supp(ψjlw)∩B p (0,K)̸ =∅} ≤(1 + log(n)) 2 log(⌊n⌋) j=0 (1 + j) -2 2 p l=1 w∈Z p |α D (j, l, w)(α f (j, l, w) -α f ⋆ (j, l, w))| + ∞ j=log(⌊n⌋)+1 2 -j(β+γ) C ≤(1 + log(n)) 2 sup D∈B γ,2 ∞,∞ (1) R p D(x)(f (x) -f ⋆ (x))dλ p (x) + C/n.
Then using Corollary 4.4 with α = β we have ) compatibility. We first define the notion of approximate Jacobian (Definition 2.10 in Federer (1959)). DEFINITION D.1. Let M be a d-dimensional submanifold of R p and a map T : R p → M of regularity C 1 . For x ∈ R p , using the matrix of ∇T (x) with respect to orthonormal bases of R p and T T (x) (M), the approximate Jacobian of T , noted ap d (∇T (x)), is equal to the square root of the sum of the squares of the determinants of the d by d minors of this matrix.

sup D∈B γ,2 ∞,∞(1) R p D(x)(f (x) -f ⋆ (x))dλ p (x) ≤ C sup D∈B β,2 ∞,∞(1) R p D(x)(f (x) -f ⋆ (x))dλ p (x) β+γ β+ β ≤ C log(n) C2 n -β+ β 2β+p β+γ β+ β ≤ C log(n) C2 n -β+γ 2β+p ,
We will use multiple times the next proposition. PROPOSITION D.2. (Theorem 3.2.22, Federer (2014)) For t > 0, T : M t → M of regularity C 1 and D : M t → R, we have

M t D(x)ap d (∇T (x))dλ p (x) = M T -1 ({z}) D(x)dλ p-d (x)dλ M (z),
for λ M the volume measure on the submanifold M.

Writing M g := g(T d ) for the image of the torus by a map g, we can now define the notion of (g #U , g ⋆ #U ) compatibility.

DEFINITION D.3. For g, g ⋆ ∈ H β+1 K (T d , R p ) and t ∈ (0, r g ⋆ /2) such that H(M g , M g ⋆ ) < t. We say that a map T :

M t g ⋆ → M g ⋆ is (g #U , g ⋆ #U ) KT -compatible with a radius t if: i) T ∈ H β+1 KT (M t g ⋆ , M g ⋆ ) and the restriction T |M g : M g → M g ⋆ is a diffeomorphism such that (T • g) #U admits a density f T ∈ H β KT (M g ⋆ , R) with respect to the volume measure on M g ⋆ . ii) For all x ∈ M g ⋆ , T -1 ({x}) -x ⊂ E x a p -d dimensional subspace of R p and (T -1 ({x}) -x) ∩ B p (x, t) = B Ex (0, t). iii) For any function f ∈ H β+1 1 (M t g ⋆ , R), the map x → T -1 ({x}) f (y)dλ p-d Ex (y) belongs to H β+1 KT (M g ⋆ , R). iv) For T -1 g : M g ⋆ → M g the inverse application such that T -1 g • T |M g = Id |M g , T -1 g • T ∈ H β+1 KT (M t g ⋆ , M g ). Furthermore, the approximate Jacobian of T (see Definition D.1) x → ap d (∇T (x)) is bounded below by K -1 T .
Requiring this compatibility seems to be quite a strong property but we will show in Proposition D.5 that actually, as long as the d H β+1 1 distance between the measures is relatively small, there always exists a (g #U , g ⋆ #U )-compatible map. The idea behind Definition D.3 is that we would like to project M g onto M g ⋆ with a C β+1 -diffeomorphism. From [START_REF] Leobacher | Existence, uniqueness and regularity of the projection onto differentiable manifolds[END_REF], we know that the orthogonal projection on a manifold of regularity β +1 is only of regularity β so unfortunately, we cannot use it here. Therefore, we define the notion of (g #U , g ⋆ #U )-compatible map as a "pseudo" projection of regularity β + 1. This notion regroup technical properties we will need in the proof of Theorem 5.1. They are essentially properties that the orthogonal projection onto M g ⋆ would verify if M g ⋆ was of regularity β + 2 and M g , M g ⋆ are close enough. Indeed, to have the notion of compatibility, we will need the sets M g and M g ⋆ to be close in Hausdorff distance. First, let us show that the Hausdorff distance between the image of g and g ⋆ is controlled by the d H β+1

1 norm. PROPOSITION D.4. Let g, g ⋆ ∈ H 1 K (T d , R p ), then for all t > 0, if sup D∈H β+1 1 E U ∼U ([0,1] d ) [D(g(U )) -D(g ⋆ (U ))] ≤ t (d+1)(2β+1) , then W 1 (g #U , g ⋆ #U ) ≤ Ct d+1 and H(M g , M g ⋆ ) < Ct.
PROOF. Suppose first that β is not an integer. Then, using Corollary B.2 with θ = 1, θ 1 = 1/2 and θ 2 = β + 1, we have

W 1 (g #U , g ⋆ #U ) ≤ 2K sup D∈H 1 1 E U ∼U ([0,1] d ) [D(g(U )) -D(g ⋆ (U ))] ≤ C sup D∈H 1/2 1 (E[D(g(U )) -D(g ⋆ (U ))]) 2β 2β+1 sup D∈H β+1 1 (E[D(g(U )) -D(g ⋆ (U ))]) 1 2β+1 ≤ C sup D∈H β+1 1 (E[D(g(U )) -D(g ⋆ (U ))]) 1 2β+1 .
If now β is an integer, then using Corollary B.2 with θ = 1, θ 1 = β-1/2 2β and θ 2 = β + 3/2, we get the same result. Then by hypothesis we obtain

W 1 (g #U , g ⋆ #U ) ≤ Ct d+1 . Suppose that max x∈Mg d(x, M g ⋆ ) ≥ δ. For u ∈ [0, 1] d such that d(g(u)
, M g ⋆ ) ≥ δ, using the dual formulation of the 1-Wasserstein distance, we have that

W 1 (g #U , g ⋆ #U ) = sup D∈Lip 1 E U ∼U ([0,1] d ) [D(g(U )) -D(g ⋆ (U ))] ≥ [0,1] d d(g(z), M g ⋆ )dλ d (z) > B d (u, δ 2K ) δ 2 dλ d (z) ≥ Cδ d+1 .
Therefore we can deduce that

H(M g , M g ⋆ ) ≤ Ct.
In this proposition, the exponents are not optimal but are sufficient for our purpose. Using this result, let show that if the distance

d H β+1 1 (g #U , g ⋆ #U
) is small enough, then there exists a (g #U , g ⋆ #U )-compatible map.

PROPOSITION D.5. Let g, g ⋆ ∈ H β+1 K (T d , R p ) that verify the K-manifold regularity condition with g ⋆ that verifies the K-density regularity condition. There exist constants C, C 2 > 0 such that if

sup D∈H β+1 1 E U ∼U ([0,1] d ) [D(g(U )) -D(g ⋆ (U ))] ≤ C -1 ,
then g verifies the C 2 -density regularity condition and there exists a

(g #U , g ⋆ #U ) KT - compatible map with radius t ≥ C -1 2 and K T ≤ C 2 .
The proof of Proposition D.5 can be found in the following Section.

D.2. Proof of Proposition D.5. The proof of proposition D.5 breaks down as follow: we first approximate M g ⋆ by a manifold M gs of regularity β + 2 using convolution and show that its reach is bounded below. Using that g #U is close to g ⋆ #U for the d H β+1 1 distance, we show that g is a local diffeomorphism. We then show that π s|M g ⋆ : M g ⋆ → M gs the restriction to M g ⋆ of π s the orthogonal projection onto M gs , is a C β+1 diffeomorphism. Finally we define our map T as T (x) = (π s|M g ⋆ ) -1 • π s (x) and show that it verifies all the conditions to be (g #U , g ⋆ #U )-compatible.

D.2.1. Approximation of M g ⋆ by a manifold M gs of regularity β + 2. Let s ∈ (0, r g ⋆ /2) and R s : T d → R defined by R s (u) = 0 ∨ (1 -∥u∥/s)/C s with C s such that ∥R s ∥ L 1 = 1. Then define g s : T d → R p by g s := g ⋆ * R s .
We have that g s belongs to

H β+1 K (T d , R p ) ∩ H β+2 Ks -1 (T d , R p ), verifies that ∥g ⋆ -g s ∥ ∞ ≤ Ks and ∥g ⋆ -g s ∥ H ⌊β⌋+1 ≤ Cs. Write C 0 for the smallest C > 0 such that ∥g ⋆ -g s ∥ H ⌊β⌋+1 ≤ C 0 s.
We do not fix the value of s yet because we will lower it all along the proof to suit our needs. Let us show that g s is a diffeomorphism. PROPOSITION D.6. There exists a constant C > 0 such that if s ∈ (0, C -1 ), the map g s is a diffeomorphism on its image.

PROOF. For u ∈ T d and w ∈ R d , we have

∥∇g s (u) w ∥w∥ ∥ ≥ ∥∇g ⋆ (u) w ∥w∥ ∥ -C 0 s ≥ K -1 -C 0 s ≥ 1 2K for s ≤ (4KC 0 ) -1 , so g s is a local diffeomorphism. Suppose there exists u, v ∈ T d , such that g s (u) = g s (v) and u ̸ = v. Then as ∥g s (u) -g s (v)∥ = ∥∇g s (u)(v -u) + 1 0 ∇ 2 g s (u + t(v -u))(v -u) 2 (1 -t)dt∥ ≥ ∥∇g ⋆ (u)(v -u)∥ -∥ 1 0 ∇ 2 g ⋆ (u + t(v -u))(v -u) 2 (1 -t)dt∥ -2C 0 s∥u -v∥ ≥ K -1 ∥u -v∥ - K 2 ∥u -v∥ 2 -2C 0 s∥u -v∥ = K 2 ∥u -v∥(2K -2 -4 C 0 K s -∥u -v∥), we deduce that ∥u -v∥ ≥ 2K -2 -4 C0 K s. Then for s ≤ (16C 0 K 3 ) -1 , we have ∥g ⋆ (u) -g ⋆ (v)∥ ≤ ∥g s (u) -g s (v)∥ + 2C 0 s ≤ (8K 3 ) -1 .
As ∥u -v∥ ≥ K -2 , applying Lemma D.19, we obtain that r g ⋆ ≤ (16K 3 ) -1 which is impossible as g ⋆ verifies the K-manifold regularity condition. Therefore, g s is injective and so is a diffeomorphism.

Let us show that the submanifold M gs = g s ([0, 1] d ) as a stricly positive reach. LEMMA D.7. There exists a constant C > 0 such that if s ∈ (0, C -1 ), we have that the reach r gs of M gs , verifies

r gs ≥ C -1 . PROOF. Let u, v ∈ T d , recall from the proof of proposition D.6 that for s ≤ (4KC 0 ) -1 , we have ∥∇g s (v)(u -v)∥ ≥ ∥u-v∥
2K . We are going to use characterization of the reach for submanifolds of Lemma D.18. Using a Taylor expansions on g s we have

∥g s (u) -g s (v)∥ 2 2d(g s (u) -g s (v), T gs(v) (M gs )) = ∥∇g s (v)(u -v) + O(∥u -v∥ 2 )∥ 2 2d(∇g s (v)(u -v) + O(∥u -v∥ 2 ), T gs(v) (M gs )) ≥ ∥∇g s (v)(u -v)∥ 2 + 2⟨∇g s (v)(u -v), O(∥u -v∥ 2 )⟩ + ∥O(∥u -v∥ 2 )∥ 2 2d(∇g s (v)(u -v), T gs(v) (M gs )) + 2d(O(∥u -v∥ 2 )∥, T gs(v) (M gs )) ≥ K -2 4 ∥u -v∥ 2 -2K 2 ∥u -v∥ 3 -K 2 ∥u -v∥ 4 2K∥u -v∥ 2 ≥ 1 8K 3 (1 -K 4 (8∥u -v∥ + 4∥u -v∥ 2 )), (D.1)
where we used that d(∇g

s (v)(u -v), T gs(v) (M gs )) = 0. Then ∥u -v∥ ≤ (32K 4 ) -1 implies that ∥g s (u) -g s (v)∥ 2 2d(g s (u) -g s (v), T gs(v) (M gs )) > (16K 3 ) -1 .
Therefore, if r gs ≤ (16K 3 ) -1 we have

r gs = min ∥u-v∥≥(32K 4 ) -1 ∥g s (u) -g s (v)∥ 2 2d(g s (u) -g s (v), T gs(v) (M gs ))
. PROOF. Let t ∈ (0, r gs /2) and

Let (u ⋆ , v ⋆ ) ∈ arg min ∥u-v∥≥(32K 4 ) -1 ∥g s (u) -g s (v)∥ 2 2d(g s (u) -g s (v), T gs(v) (M gs )) , then ∥g ⋆ (u ⋆ ) -g ⋆ (v ⋆ )∥ ≤ ∥g s (u ⋆ ) -g s (v ⋆ )∥ + 2C 0 s ≤ ∥g s (u ⋆ ) -g s (v ⋆ )∥ 2 d(g s (u ⋆ ) -g s (v ⋆ ), T gs(v ⋆ ) (M gs )) + 2C 0 s = 2r gs + 2C 0 s. Then, if r gs ∨ C 0 s < (2 8 K 5 ) -1 , we have ∥g ⋆ (u ⋆ ) -g ⋆ (v ⋆ )∥ < (2 6 K 5 ) -1 and as ∥u ⋆ -v ⋆ ∥ ≥ (32K 4 ) -1 , applying Lemma D.19, we obtain r g ⋆ ≤ (2 7 K 5 ) -1 which is impossible as r g ⋆ ≥ K -1 . Therefore, taking s < (C 0 2 8 K 5 ) -1 we have r gs ≥ (2 8 K 5 ) -1 . D.2.2.
x ∈ M t gs , h ∈ T πs(x) (M gs ) ⊥ with ∥h∥ ≤ t , u = g -1 s • π s (x) and v ∈ [0, 1] d . Define γ : [0, 1] → T d by γ(r) = g -1 s • π s (x + rh).
Then, we have that γ is 4Kh-Lipschitz as π s is 2-Lipschitz on M t gs (see formula (D.3) and equation (D.4)) and g -1 s is 2K-Lipschitz. We would like to show that π

s (x + h) = π s (x) i.e ∥x + h -g s (u + v)∥ > ∥x + h -g s (u)∥.
Then as γ is 4Kh-Lipschitz it is sufficient to prove it for v ∈ B d (0, 4Kh). We have

∥x + h -g s (u + v)∥ 2 =∥x + h -g s (u)∥ 2 + 2⟨x + h -g s (u), -∇g s (u)v + O(∥v∥ 2 )⟩ + ∥ -∇g s (u)v + O(∥v∥ 2 )∥ 2 .
On the ohter hand,

x + h -g s (u) = x + h -π s (x) ∈ Im(∇g s (u)) ⊥ the orthogonal of the image of ∇g s (u), so ∥x + h -g s (u + v)∥ 2 =∥x + h -g s (u)∥ 2 + 2⟨x + h -g s (u), O(∥v∥ 2 )⟩ + ∥∇g s (u)v∥ 2 -2⟨∇g s (u)v, O(∥v∥ 2 )⟩ + O(∥v∥ 4 ) ≥∥x + h -g s (u)∥ 2 -2∥x + h -g s (u)∥K∥v∥ 2 + K -2 4 ∥v∥ 2 -2K∥v∥K∥v∥ 2 -K 2 ∥v∥ 4 ≥∥x + h -g s (u)∥ 2 + ∥v∥ 2 ( K -2 4 -4tK -2K 2 ∥v∥ -K 2 ∥v∥ 2 ) ≥∥x + h -g s (u)∥ 2 + ∥v∥ 2 ( K -2 4 -K 3 (8t + 4Kt 2 ))
where we used that ∥v∥ ≤ 4Kh ≤ 4Kt. Then for t = K -5 /2 6 we have ∥x

+ h -g(u + v)∥ > ∥x + h -g(u)
∥, so π s verifies ii) for a radius t.

LEMMA D.9. The map π s|M g ⋆ : M g ⋆ → M gs , i.e. the restriction of the projection π s to the manifold M g ⋆ , is injective.

PROOF. Suppose there exist u, v ∈ T d different such that π s (g ⋆ (u)) = π s (g ⋆ (v)). We have

∥g s (u) -g s (v)∥ ≤ ∥g ⋆ (u) -g ⋆ (v)∥ + 2C 0 s ≤ ∥g ⋆ (u) -π s (g ⋆ (u))∥ + ∥g ⋆ (v) -π s (g ⋆ (u))∥ + 2C 0 s ≤ 4C 0 s Suppose first that ∥u -v∥ ≥ (32K 4 ) -1 and let (x, y) ∈ arg min ∥a-b∥≥(32K 4 ) -1 ∥g s (a) -g s (b)∥.
As detailed in the proof of Lemma D.7, we have that g s (x) -g s (y) ∈ T gs(y) (M gs ) ⊥ and

r gs = min ∥a-b∥≥(32K 4 ) -1 ∥g s (a) -g s (b)∥ 2 2d(g s (a) -g s (b), T gs(b) (M gs )) ≤ ∥g s (x) -g s (y)∥ 2 2d(g s (x) -g s (y), T gs(y) (M gs )) = 1 2 ∥g s (x) -g s (y)∥ ≤ 1 2 ∥g s (u) -g s (v)∥ ≤ 2C 0 s.
As from Lemma D.7 we have r gs ≥ C -1 , supposing s > 0 is small enough, this is impossible and so ∥u -v∥ ≤ (32K 4 ) -1 .

Let

F : [0, ∞) → R p defined by F (t) = g ⋆ (u + t(u -v)) then |d(F (t), M gs ) -d(g ⋆ (u) + t(g ⋆ (u) -g ⋆ (v)), M gs )| ≤ ∥F (t) -(g ⋆ (u) + t(g ⋆ (u) -g ⋆ (v)))∥ ≤ t∥∇g ⋆ (u)(u -v) -(g ⋆ (u) -g ⋆ (v))∥ + K∥u -v∥ 2 t 2 ≤ Kt∥u -v∥ 2 + K∥u -v∥ 2 t 2 = K∥u -v∥ 2 (t + t 2 ). (D.2) We have ⟨g ⋆ (v) -π s (g ⋆ (v)), g ⋆ (v) -g ⋆ (u)⟩ = ⟨g ⋆ (v) -g ⋆ (u) + g ⋆ (u) -π s (g ⋆ (u)), g ⋆ (v) -g ⋆ (u)⟩ = ∥g ⋆ (v) -g ⋆ (u)∥ 2 -⟨g ⋆ (u) -π s (g ⋆ (u)), g ⋆ (u) -g ⋆ (v)⟩,
so without loss of generality we can suppose that ⟨g ⋆ (u) -π s (g

⋆ (u)), g ⋆ (u) -g ⋆ (v)⟩ ≥ 0 (otherwise we would have ⟨g ⋆ (v) -π s (g ⋆ (v)), g ⋆ (v) -g ⋆ (u)⟩ ≥ 0). Let C 1 > 0 be the constant from Proposition D.8 such that π s verifies the point ii) of Definition D.3 on M C -1 1 gs . Supposing s ≤ (2C 0 C 1 ) -1 we have H(M gs , M g ⋆ ) ≤ C 0 s < C -1 1 /2, then for t ∈ [0, C -1 1 2∥g ⋆ (u)-g ⋆ (v)∥ ], using Proposition D.8 we have π s (g ⋆ (u) + t(g ⋆ (u) -g ⋆ (v))) = π s (g ⋆ (u)). Then d(g ⋆ (u) + t(g ⋆ (u) -g ⋆ (v)), M gs ) 2 = ∥g ⋆ (u) + t(g ⋆ (u) -g ⋆ (v)) -π s (g ⋆ (u))∥ 2 = ∥g ⋆ (u) -π s (g ⋆ (u))∥ 2 + 2t⟨g ⋆ (u) -π s (g ⋆ (u)), g ⋆ (u) -g ⋆ (v)⟩ + t 2 ∥g ⋆ (u) -g ⋆ (v)∥ 2 ≥ t 2 ∥g ⋆ (u) -g ⋆ (v)∥ 2 . Using (D.2) for t = 2C0s ∥g ⋆ (u)-g ⋆ (v)∥ , we have d(F ( 2C 0 s ∥g ⋆ (u) -g ⋆ (v)∥ ), M gs ) ≥d(g ⋆ (u) + 2C 0 s ∥g ⋆ (u) -g ⋆ (v)∥ (g ⋆ (u) -g ⋆ (v)), M gs ) -K∥u -v∥ 2 2C 0 s ∥g ⋆ (u) -g ⋆ (v)∥ + (2C 0 s) 2 ∥g ⋆ (u) -g ⋆ (v)∥ 2 . Then as ∥g ⋆ (u) -g ⋆ (v)∥ ≤ ∥g ⋆ (u) -π s (g ⋆ (u))∥ + ∥g ⋆ (v) -π s (g ⋆ (v))∥ ≤ 2C 0 s, we have 2C0s ∥g ⋆ (u)-g ⋆ (v)∥ ≤ 2C0s ∥g ⋆ (u)-g ⋆ (v)∥ 2 so d(F ( 2C 0 s ∥g ⋆ (u) -g ⋆ (v)∥ ), M gs ) ≥ 2C 0 s ∥g ⋆ (u) -g ⋆ (v)∥ ∥g ⋆ (u) -g ⋆ (v)∥ -2K∥u -v∥ 2 2C 0 s ∥g ⋆ (u) -g ⋆ (v)∥ 2 ≥2C 0 s -8KC 2 0 s 2 ∥u -v∥ ∥g ⋆ (u) -g ⋆ (v)∥ 2 ≥2C 0 s(1 -4KC 0 s ∥u -v∥ K -1 ∥u -v∥ -K∥u -v∥ 2 2 ) ≥2C 0 s(1 -16K 3 C 0 s)
where we used that ∥u -v∥ ≤ (32K 4 ) -1 implies that ∥u -v∥ -K 2 ∥u -v∥ 2 ≥ ∥u -v∥/2. Then taking s ≤ (2 6 K 3 C 0 ) -1 we have d(F ( 2C0s ∥g ⋆ (u)-g ⋆ (v)∥ , M gs ) > C 0 s which is impossible by definition of g s .

From [START_REF] Leobacher | Existence, uniqueness and regularity of the projection onto differentiable manifolds[END_REF] we know that the canonical projection π onto a submanifold M with strictly positive reach r, verifies

(D.3) ∇π(x) = Id Tπ(x)(M) -∥x -π(x)∥L π(x),v -1 P Tπ(x)(M) ,
for L π(x),v the shape operator in the direction v = x-π(x) ∥x-π(x)∥ . From Corollary 3 in [START_REF] Leobacher | Existence, uniqueness and regularity of the projection onto differentiable manifolds[END_REF], we also have that

(D.4) ∥ Id Tπ(x)(M) -∥x -π(x)∥L π(x),v -1 ∥ ≤ (1 -∥x -π(x)∥/r) -1 .
Furthermore if M is of regularity β + 2, there exists a constant C > 0 depending on the H β+2 norm of the charts such that

(D.5) ∥π | M r/2 ∥ H β+1 ≤ C
Using these results, we obtain in the next proposition that the restriction of π s to the manifold M g ⋆ is a diffeomorphism.

PROPOSITION D.10. There exists a constant C > 0 such that if s ≤ C -1 , then the map π s|M g ⋆ is a diffeomorphism.

PROOF. From (D.5) we have

(D.6) ∥π s| M rg s /2 gs ∥ H β+1 ≤ Cs -1 .
To show that π s|M g ⋆ is a local diffeomorphism, let us show that if there existed u such that λ min (∇(π s • g ⋆ )(u) ⊤ ∇(π s • g ⋆ )(u)) was too small, then the Hausdorff distance between M gs and M g ⋆ would be too big. Let u ∈ [0, 1] d and h > 0, then for v ∈ R d such that ∥v∥ = 1 and

⟨π s (g ⋆ (u)) -g ⋆ (u), ∇(π s • g ⋆ -g ⋆ )(u)v⟩ ≥ 0, we have ∥π s (g ⋆ (u + hv)) -g ⋆ (u + hv)∥ ≥ ∥π s (g ⋆ (u)) -g ⋆ (u) + ∇(π s • g ⋆ -g ⋆ )(u)hv∥ -2Kh 2 = ∥π s (g ⋆ (u)) -g ⋆ (u)∥ 2 + 2⟨π s (g ⋆ (u)) -g ⋆ (u), ∇(π s • g ⋆ -g ⋆ )(u)hv⟩ + ∥∇(π s • g ⋆ -g ⋆ )(u)hv∥ 2 1/2 -2Kh 2 ≥h∥∇(π s • g ⋆ -g ⋆ )(u)v∥ -2Kh 2 ≥h(∥∇g ⋆ (u)v∥ -∥∇(π s • g ⋆ )(u)v∥) -2Kh 2 ≥h(K -1 -∥∇(π s • g ⋆ )(u)v∥) -2Kh 2 . As H(M gs , M g ⋆ ) ≤ Ks we deduce with h = (K -1 -∥∇(π s • g ⋆ )(u)v∥)/4K that (K -1 -∥∇(π s • g ⋆ )(u)v∥) 2 8K ≤ Ks so λ min (∇(π s • g ⋆ )(u) ⊤ ∇(π s • g ⋆ )(u)) 1/2 ≥ K -1 - √ 8sK. (D.7)
In particular π s • g ⋆ is an immersion and therefore π s|M g ⋆ : M g ⋆ → M gs is a local diffeomorphism. As M g ⋆ is compact without boundary, π s (M g ⋆ ) is a compact boundaryless submanifold of M gs so being of same dimension, there are equal. We can conclude that π s|M g ⋆ is a diffeomorphism. D.2.3. Density regularity of g. As for all u ∈ T d we have λ min ((∇g ((∇g(u)) ⊤ ∇g(u)) was to small, then the distane

⋆ (u)) ⊤ ∇g ⋆ (u)) 1/2 ≥ 1/K, let us show that if inf u∈[0,1] d λ min
d H β+1 1 (g #U , g ⋆ #U
) would be too big.

PROPOSITION D.11. There exist constants C, δ > 0 such that for all τ > δ, if 2β+1) .

inf u∈T d inf w∈R d ∥∇g(u) w ∥w∥ ∥ ≤ K -τ , then d H β+1 1 (g #U , g ⋆ #U ) ≥ CK -δ(d+1)(
PROOF. Let τ > 3 and suppose that there exists u 0 ∈ T d and w

0 ∈ R d with ∥w 0 ∥ = 1 such that ∥∇g(u 0 )w 0 ∥ ≤ K -τ . For δ ∈ (3, τ ), define D δ : M K -2 /2 g ⋆ → R by D δ (x) = ( K -δ 8 ∧ ( K -δ 4 -∥π(x) -π(g(u 0 ))∥)) ∨ 0 for π the canonical projection onto M g ⋆ . As D δ is 2-Lipschitz (see (D.3)), we have W 1 (g #U , g ⋆ #U ) ≥ 1 2 E[D δ (g(U )) -D δ (g ⋆ (U ))] = 1 2 E[D δ (g(U ))1 {∥π(g(U ))-π(g(u0))∥≤ K -δ 4 } ] (D.8) - 1 2 E[D δ (g ⋆ (U ))1 {∥π(g ⋆ (U ))-π(g(u0))∥≤ K -δ 4 } ] ≥ 1 2 E[ K -δ 8 1 {∥π(g(U ))-π(g(u0))∥≤ K -δ 8 } - K -δ 4 1 {∥π(g ⋆ (U ))-π(g(u0))∥≤ K -δ 4 } ] (D.9)
Let us first bound below the quantity E[1 {∥π(g(U ))-π(g(u0))∥≤ K -δ 8 } ]. Let v ∈ R d and write v 0 = ⟨v, w 0 ⟩w 0 , v ⊥ 0 = v -v 0 . Using a Taylor expansion on the path ζ : [0, 1] → R d defined by

ζ(t) = u 0 + 4( t 2 ∧ 1 2 )v 0 + 4(( t 2 - 1 2 ) ∨ 0)v ⊥ 0 ,
we have

∥g(u 0 + v) -g(u 0 )∥ = ∥ 1 0 ∇g(ζ(t)) ζ(t)dt∥ = ∥ 1 0 ∇g(u 0 + tv 0 )v 0 + ∇g(u 0 + v 0 + tv ⊥ 0 )v ⊥ 0 dt∥ = ∥ 1 0 ∇g(u 0 )v 0 + 1 0 ∇ 2 g(u 0 + stv 0 )tv 2 0 )ds + ∇g(u 0 + v 0 + tv ⊥ 0 )v ⊥ 0 dt∥ ≤ K -τ ∥v 0 ∥ + K 2 ∥v 0 ∥ 2 + K∥v ⊥ 0 ∥. Therefore, if ∥v 0 ∥ ≤ K -δ+1 2 /2 6 and ∥v ⊥ 0 ∥ ≤ K -(δ+1) /2 5 then ∥π(g(u 0 + v)) -π(g(u 0 ))∥ ≤ 2∥g(u 0 + v) -g(u 0 )∥ ≤ K -δ 8 .
Writing C d-1 the volume of the d -1-dimensional ball, we can conclude that

E[1 {∥π(g(U ))-π(g(u0))∥≤ K -δ 8 } ] ≥ C d-1 K -(δ+1) 2 5 d-1 K -δ+1 2 2 6 .
Let us now bound the quantity

E[1 {∥π(g ⋆ (U ))-π(g(u0))∥≤ K -δ 4 } ]. For u ∈ T d , we have ∥π(g ⋆ (u)) -π • g(u 0 )∥ =∥g ⋆ (u) -g ⋆ • g ⋆-1 • π • g(u 0 )∥ ≥∥∇g ⋆ (g ⋆-1 • π • g(u 0 ))(u -g ⋆-1 • π • g(v 0 ))∥ -K(∥u -g ⋆-1 • π • g(u 0 )∥ 2 ) ≥K -1 ∥u -g ⋆-1 • π • g(u 0 )∥ -K∥u -g ⋆-1 • π • g(u 0 )∥ 2 . (D.10) As r g ⋆ ≥ K -1 , from Lemma D.19 we have ∥g ⋆ (u) -g ⋆ (v)∥ ≤ K -3 /4 imply ∥u -v∥ ≤ K -2 /2, then, (D.11) ∥g ⋆ (u) -g ⋆ • g ⋆-1 • π • g(u 0 )∥ ≤ K -δ 4 imply ∥u -g ⋆-1 • π • g(u 0 )∥ ≤ K -2 /2. Solving K -1 x -Kx 2 ≤ K -δ 4 , we find x / ∈ ( K -1 - √ K -2 -K -δ+1 2K , K -1 + √ K -2 -K -δ+1 2K
) so from (D.10),

∥g ⋆ (u) -π • g(u 0 )∥ ≤ K -δ 4 implies ∥u -g ⋆-1 • π • g(u 0 )∥ / ∈ ( K -1 - √ K -2 -K -δ+1 2K , K -1 + √ K -2 -K -δ+1 2K
).

From (D.11), we have that ∥g

⋆ (u) -π • g(u 0 )∥ ≤ K -δ 4 implies ∥u -g ⋆-1 • π • g(u 0 )∥ ≤ K -2 /2 < K -1 + √ K -2 -K -δ+1 2K , so finally ∥g ⋆ (u) -π • g(u 0 )∥ ≤ K -δ 4 implies that ∥u -g ⋆-1 • π • g(u 0 )∥ ≤ K -1 - √ K -2 -K -δ+1 2K ≤ K -δ+2 2K = K -δ+1 2 .
Writing C d the volume of the d-dimensional ball, we can conclude that

E[1 {∥π(g ⋆ (U ))-π(g(u0))∥≤ K -δ 4 } ] ≤ C d K -δ+1 2 d .
Finally, from (D.8) we have

W 1 (g #U , g ⋆ #U ) ≥ 1 2 E[ K -δ 8 1 {∥π(g(U ))-π(g(u0))∥≤ K -δ 8 } - K -δ 4 1 {∥π(g ⋆ (U ))-π(g(u0))∥≤ K -δ 4 } ] ≥ 1 2 K -δ 8 C d-1 K -(δ+1) 2 5 d-1 K -δ+1 2 2 6 - K -δ 4 C d K -δ+1 2 d = C d-1 K -δ(d+1) 2 5(d+1) (K δ/2-d+1/2 - C d 2 4d+2 K d C d-1
).

Then for δ > 3 such that

K δ/2-d+1/2 -Cd2 4d+2 K d Cd-1 ≥ 2 5(d+1)
Cd-1 we have 2β+1) .

W 1 (g #U , g ⋆ #U ) ≥ K -δ(d+1) so from Proposition D.4 we have d H β+1 1 (g #U , g ⋆ #U ) ≥ CK -δ(d+1)(
Recalling that in the assumptions of Proposition D.5 we suppose that d H β+1 1 (g #U , g ⋆ #U ) ≤ C -1 , then supposing C > 0 large enough, using proposition D.11, we obtain (D.12) inf

u∈[0,1] d λ min ((∇g(u)) ⊤ ∇g(u)) 1/2 ≥ C -1 . D.2.4. The (g #U , g ⋆ #U
)-compatible map. We can now define the map T :

M rg s /4 g ⋆ → M g ⋆ by T (x) = (π s|M g ⋆ ) -1 • π s (x),
which is well defined because H(M gs , M g ⋆ ) ≤ Ks < r gs /4. Let us now state that T is the map we are interested in: PROPOSITION D.12. The map T :

M t g ⋆ → M g ⋆ defined by T (x) = (π s|M g ⋆ ) -1 • π s (x), is (g #U , g ⋆ #U ) KT -compatible with radius t ≥ C -1 and K T ≤ C.
PROOF. Let us start by proving the point i). We have ∇T (x) = ∇(π s|M g ⋆ ) -1 (π s (x)) • ∇π s (x). (D.13) From (D.7), as g ⋆ is K-Lipschitz, we have that for x ∈ M g ⋆ and h ∈ T x (M g ⋆ ),

(D.14) ∥∇π s|M g ⋆ (x)h∥ ≥ K -1 (K -1 - √ 8sK)∥h∥.
Then, as π s ∈ H Let us now show that the density of (T •g) #U with respect to the volume measure on M g ⋆ , is of regularity β. As H(M gs , M g ) ≤ H(M g , M g ⋆ ) + H(M gs , M g ⋆ ) ≤ 2Ks we show the same way as we did with g ⋆ that π s • g is a diffeomorphism and therefore, T • g is also a diffeomorphism. Let us write f T the density of (T • g) #U with respect to the volume measure on M g ⋆ . We have

f T (x) = det ∇(T • g)((T • g) -1 (x)) ⊤ ∇(T • g)((T • g) -1 (x)) -1/2 .
Using (D.12) and doing the same derivations as with g ⋆ in the proof of Proposition D.10, we can show that there exists a constant C > 0 such that for all u ∈ T d ,

λ min (∇(π s • g)(u) ⊤ ∇(π s • g)(u)) 1/2 ≥ C -1 - √ 16sK.
Furthermore, for all u ∈ T d , as ∀h ∈ T πs(g(u)) (M gs ),

∥∇π s|M g ⋆ (T • g(u))h∥ ≤ 2∥h∥,
we have

(D.15) λ min ∇(T • g)((T • g) -1 (x)) ⊤ ∇(T • g)((T • g) -1 (x)) 1/2 ≥ 1 2 (C -1 - √ 16sK).
Then for s > 0 small enough, using the Faa di Bruno formula we have that f T ∈ H β Cs -C 2 (M g ⋆ , R). Therefore point i) is verified. Proposition D.8 asserts that there exists a constant C > 0 such that π s verifies ii) for a radius C -1 . Then, as H(M g ⋆ , M gs ) ≤ Ks ≤ C -1 /2 for s > 0 small enough, we deduce that T verifies ii) for a radius t = C -1 /4.

Let us prove point iii). Let f ∈ H β+1 Cs -1 (M gs , (R p ) p-d ) such that (f 1 (x), ..., f p-d (x)) is an orthonormal basis of T x (M gs ) ⊤ . For x ∈ M gs , define R x : R p-d → R p a linear map such that R x (e i ) = f i (x) for (e 1 , ..., e p-d ) the canonical basis of R p-d . Then for h ∈ H β+1 1 (M t g ⋆ , R) and x ∈ M g ⋆ , we have

T -1 ({x}) h(y)dλ p-d Ex (y) = Tπ s (x) (Mg s ) ⊥ ∩B d (x,t) h(y)dλ p-d Tπ s (x) (Mg s ) ⊥ (y) = R p-d ∩B d (0,t) h(x + R πs(x) (z))dλ p-d (z).
Then as ∀z ∈ R p-d , the map x → x + R πs(x) (z) belongs to H β+1 Cs -1 (M g ⋆ , R p ), we have that T verifies iii).

Let us prove iv). We have

T -1 g • T = (π s|M g ) -1 • π s (x)
so as H(M gs , M g ) ≤ 2Ks we show the same we showed that

T ∈ H β+1 Cs -C 2 (M t gs , M g ⋆ ), that T -1 g • T ∈ H β+1 Cs -C 2 (M t gs , M g ).
Let us now lower bound the approximate Jacobian of T . For x ∈ M t g ⋆ , using (D.14) we have that

inf z∈TT (x)(M ⋆ g ) ∥∇π s (x) z ∥z∥ ∥ ≥ 1 K (K -1 - √ 8sK), so as π s is 2-Lipschitz on M g ⋆ , we deduce that inf z∈TT (x)(M ⋆ g ) ∥∇T (x) z ∥z∥ ∥ ≥ 1 2K (K -1 - √ 8sK).
Therefore, we deduce that the approximate Jacobian of T verifies

ap d (∇T (x)) ≥ 1 2K (K -1 - √ 16sK) d ,
so T verifies the point iv) for s > 0 small enough.

D.3. Proof of Lemma 5.2. The proof breaks down as follow. We first prove that we only need to treat the case γ = 1. Then, we build a function H ∈ H β+1 C (A, R) for A a neighborhood of M g ∪ M g ⋆ , and prove that d

H 1 1 (g #U , (T • g) #U ) ≤ E[H(g(U ))] β+1 2β+1
. Finally, we extend H to the whole R p space. D.3.1. Proof that we only need to treat the case γ = 1. Let us first show an interpolation inequality on the distance d H γ 1 .

LEMMA D.13. For γ ∈ (1, β + 1) we have for all ϵ ∈ (0, 1),

d H γ 1 (g #U , (T • g) #U ) ≤C log(ϵ -1 ) 2 d H 1 1 (g #U , (T • g) #U ) β+1-γ β × d H β+1 1 (g #U , (T • g) #U ) γ-1 β + C log(ϵ -1 ) 2 ϵ.
PROOF. The proof follows the same arguments than the proof of Corollary B.2 except we are first going to give additional regularity to the discriminator so that its regularization belongs to Hölder spaces. Let ϵ > 0 and f ϵ g , f ϵ T the densities of the ϵ-envelopes of g #U , (T • g) #U (see Definition B.3) respectively. For D ∈ H γ 1 we have

E U ∼U ([0,1] d ) [D(g(U )) -D(T (g(U )))] ≤ R p D(x)(f ϵ g (x) -f ϵ T (x))dλ p (x) + 2ϵ.
Using Proposition B.4 we have that

R p D(x)(f ϵ g (x) -f ϵ T (x))dλ p (x) ≤ C log(ϵ -1 ) 2 X Γ0,-2 ϵ (D)(x)(f ϵ g (x) -f ϵ T (x))dλ X (x) + Cϵ ≤ C log(ϵ -1 ) 2 E[ Γ0,-2 ϵ (D)(g(U )) -Γ0,-2 ϵ (D)(T (g(U )))] + Cϵ .
Let us write F = Γ0,-2 ϵ (D) and let ξ > 0, then

E[F (g(U )) -F (T (g(U )))] ≤ R p F (x)(f ξ g (x) -f ξ T (x))dλ p (x) + 2ξ. Using Proposition B.1 for h 1 = F , h 2 = f ξ g -f ξ T , s 1 = γ, τ = γ -1, b 1 = -2, s 2 = b 2 = 0, t = r = 1 and q = β+1-γ β we have R p F (x)(f ξ g (x) -f ξ T (x))dλ p (x) ≤ Γγ-1 (F )(f ξ g (x) -f ξ T (x)) β+1-γ β × Γγ-(β+1) (F )(f ξ g (x) -f ξ T (x)) γ-1 β ≤E[ Γγ-1 (F )(g(U )) -Γγ-1 (F )(T (g(U )))] β+1-γ β × E[ Γγ-(β+1) (F )(g(U )) -Γγ-(β+1) (F )(T (g(U )))] γ-1 β + C(ξ β+1-γ β + ξ γ-1 β ). As Γγ-1 (F ) ∈ B 1,2 ∞,∞ (C) ⊂ H 1 C and Γγ-(β+1) (F ) ∈ B β+1,2 ∞,∞ (C) ⊂ H β+1 C , letting ξ tend to 0 we get the result. Now if we prove, d H 1 1 (g #U , (T • g) #U ) ≤ C log(ϵ -1 ) 2 d H β+1 1 (g #U , (T • g) #U ) β+1 2β+1 + Cϵ , then as β+1-γ β β+1 2β+1 + γ-1 β = β+γ 2β+1
, using lemma D.13 we will have that

d H γ 1 (g #U , (T • g) #U ) ≤ C log(ϵ -1 ) 4 d H β+1 1 (g #U , (T • g) #U ) β+γ 2β+1 + Cϵ .
Therefore, we can conclude that we only need to treat the case γ = 1. D.3.2. Construction of a good potential of regularity β + 1. Let us write

ω g = inf u∈T d inf w∈R d ∥∇g(u) w ∥w∥ ∥ ∧ 1.
From Proposition D.11, we have that ω g ≥ C -1 so g is a local diffeomorphism and as it is injective, it is a diffeomorphism. Let us show that g is a C β+1 diffeomorphism.

LEMMA D.14. We have g -1 ∈ H β+1 C (M g , T d ).

PROOF. As ∇g -1 (x) = (∇g) -1 (g -1 (x)) and ω g ≥ C, we deduce from the Faa di Bruno formula that g -1 ∈ H ⌊β⌋+1 C (M g , T d ). Furthermore, we have ∇ ⌊β⌋+1 (g -1 )(x) = F ((∇g) -1 , ∇ 2 g, ..., ∇ ⌊β⌋+1 g)(g -1 (x)) for F multilinear that can be computed from the Faa di Bruno formula. Therefore we have u → F ((∇g) -1 , ∇ 2 g, ..., ∇ ⌊β⌋+1 g)(u) belongs to H β-⌊β⌋ C

. Let us now show that there exists a constant

C 0 > 0 such that g -1 is C 0 -Lipschitz for the Euclidean norm. Let x, y ∈ M g , if ∥x -y∥ ≥ C -1 0 then ∥g -1 (x) -g -1 (y)∥ ≤ 1 ≤ C 0 ∥x -y∥. If now ∥x -y∥ ≤ C -1 0 and suppose that ∥g -1 (x) -g -1 (y)∥ ≥ (4K) -1 ω g . Then taking C -1 0 ≤ 1 4 K -1 w 2 g we have from Lemma D.19 that r g ≤ 1 2 ∥g(g -1 (x)) -g(g -1 (y))∥ = 1 2 ∥x -y∥ ≤ 1 8 K -1 w 2 g
which is impossible as r g ≥ 1/K by hypothesis. Therefore we have ∥g

-1 (x) -g -1 (y)∥ < (4K) -1 ω g , so ∥x -y∥ = ∥g(g -1 (x)) -g(g -1 (y))∥ ≥ ω g ∥g -1 (x) -g -1 (y)∥ -K∥g -1 (x) -g -1 (y)∥ 2 ≥ 1 2 ω g ∥g -1 (x) -g -1 (y)∥, so we conclude that g -1 is C 0 -Lipschitz. Finally we have |∇ ⌊β⌋+1 g -1 (x) -∇ ⌊β⌋+1 g -1 (y)| = |F ((∇g) -1 , ∇ 2 g, ..., ∇ ⌊β⌋+1 g)(g -1 (x)) -F ((∇g) -1 , ∇ 2 g, ..., ∇ ⌊β⌋+1 g)(g -1 (y))| ≤ C∥g -1 (x) -g -1 (y)∥ β-⌊β⌋ ≤ C∥x -y∥ β-⌊β⌋ .
Applying Proposition D.5, we have that there exists a map T being (g #U , g ⋆ #U ) KTcompatible with radius t ≥ C -1 and K T ≤ C. For the lightness of the derivations, let us write

X(u) = g(u) -T (g(u)). Then for D ∈ H 1 1 we have (D.16) E U ∼U ([0,1] d ) D(g(U )) -D(T (g(U ))) ≤ CE U ∼U ([0,1] d ) [∥X(U )∥] .
Let ξ > 0, and define L :

T d → R p by L(u) = X(u) ∥X(u)∥ 1 -(1 -∥X(u)∥/ξ)1 {∥X(u)∥<ξ} . (D.17) Let us write K X = ∥X∥ H β+1 ∨ 1 ≤ C and δ u = ∥X(u)∥ for u ∈ T d . We have that L ∈ H 0 1 (T d , R p
), let us show that L has some additional Hölder regularity.

PROPOSITION D.15. For L : T d → R p defined in (D.17), we have for all α ∈ [0, 1] and u

, v ∈ T d , ∥L(u) -L(v)∥ ≤ 8K X min(δ u , δ v ) -α ∥u -v∥ α . PROOF. Let Γ0,-2 ϵ (L i )(u) = log(⌊ϵ -2 ⌋) j=0 2 d l=1 z∈{0,...,2 j -1} d (1 + j) -2 α Li (j, l, z)S(j, l, w) i ψ per jlz (u)
with S(j, l, w) i = αL i (j,l,z)αX i (j,l,z) |αL i (j,l,z)αX i (j,l,z)| . Then using Proposition B.4, we have

E U ∼U ([0,1] d ) [⟨L(U ), X(U )⟩] ≤ C log(ϵ -1 ) 2 E U ∼U ([0,1] d ) [⟨ Γ0,-2 ϵ (L)(U ), X(U )⟩] + Cϵ. Applying Proposition B.1 for h 1 = Γ0,-2 ϵ (L), h 2 = X, s 1 = 0, b 1 = -2, s 2 = β + 1, b 2 = 0, τ = -β, t = 1, r = 0 and q = 2β+1 β+1 we get E U [⟨ Γ0,-2 ϵ (L)(U ), X(U )⟩] ≤ Γ-β ( Γ0,-2 ϵ (L)), X β+1 2β+1 L 2 Γ0 ( Γ0,-2 ϵ (L)), Γ β+1 (X) β 2β+1 L 2 ≤ Γ-β,-2 (L), X L 2 + Cϵ 2β+1 β+1 2β+1 Γ0,-2 ϵ (L), Γ β+1 (X) β 2β+1 L 2 , where we used that Γ-β ( Γ0,-2 ϵ (L)), X L 2 = Γ-β,-2 ϵ (L), X L 2 ≤ Γ-β,-2 (L), X L 2 + Cϵ 2β+1 . We have that Γ β+1 (X) ∈ B 0 ∞,∞ (C) and Γ0,-2 ϵ (L) ∈ B 0,2 ∞,∞ (C) so p i=1 ∞ j=0 2 d l=1 z∈{0,...,2 j -1} d α Γ0,-2 ϵ (Li) (j, l, z)α Γ β+1 (Xi) (j, l, z) ≤ p i=1 ∞ j=0 2 d l=1 z∈{0,...,2 j -1} d (1 + j) -2 C2 -jd ≤ C p i=1 ∞ j=0 (1 + j) -2 ≤ C.
Let us show that ∇ ⌊β⌋+1 Γ-β,-2 ϵ (L) exists and has some Hölder regularity. Let u ∈ T d be such that δ u > ξ and define L u : B d (u, δu 4KX ) → R p being equal to L. We then extend L u to the whole cube by

(D.20) L u (v) = L u (π Bu (v))(0 ∨ (1 -3∥v -π Bu (v)∥)),
for π Bu the projection on B d (u, δu 4KX ) (recall that it is the periodised Euclidean norm we are using). For all v ∈ B d (u, δu 4KX ), we have δ v ≥ δ u /2 so from Proposition D.15 with α = 1 -(β -⌊β⌋) we get

L u ∈ H 0 1 (B d (u, δ u 4K X ), R p ) ∩ H 1-(β-⌊β⌋) 16KX δ -(1-(β-⌊β⌋)) u (B d (u, δ u 4K X ), R p ).
Therefore, we also have that

L u ∈ H 0 1 (T d , R p ) ∩ H 1-(β-⌊β⌋) 16KX δ -(1-(β-⌊β⌋)) u (T d , R p ).
Let us look at the size of the support of the periodised wavelet. Recall from (A.9) that if 2 j > N , for s ∈ [0, 1] and r ∈ {0, ..., 2 j -1} , the one-dimensional periodised scaling and wavelet functions equal

ϕ per j (s -2 -j z) = ϕ(2 j s -z) + ϕ(2 j (s + 1) -z) and ψ per j (s -2 -j z) = ψ(2 j (s -1) -z) + ψ(2 j s -z) + ψ(2 j (s + 1) -z). For l i ∈ {0, 1}, as supp(ψ li (2 j • -r)) ⊂ [(r -N )2 -j , (r + N )2 -j ] we have: if z i ≥ 2 j -N , [0, 1] ∩ supp(ψ li (2 j • -z i )) ⊂ [(z i -N )2 -j , 1] [0, 1] ∩ supp(ψ li (2 j (• + 1) -z i )) ⊂ [0, (z i + N )2 -j -1] and if z i ≤ N , [0, 1] ∩ supp(ψ li (2 j (• -1) -z i )) ⊂ [1 + (z i -N )2 -j , 1] [0, 1] ∩ supp(ψ li (2 j • -z i )) ⊂ [0, (z i + N )2 -j ]. Then supp(ψ per li (2 j • -z i )) ⊂ [(z i -N )2 -j , (z i + N )2 -j
]/Z so finally we can conclude that supp(ψ per j,l,z ) ⊂ B d (z2 -j , N d2 -j ), recalling that this is the ball for the periodised Euclidean distance.

Let us write (α (Lu)i (j, l, z)) (j,l,z) the wavelet coefficients of (L u ) i for i = 1, ..., p. As support(ψ per jlz ) ⊂ B d (2 -j z, dN 2 -j ), we have that for any (j, l, z) such that j

> ⌊log 2 (dN 4K X δ -1 u )⌋ + 1 and support(ψ per jlz ) ∩ B d (u, δu 8KX ) ̸ = ∅ then α (Lu)i (j, l, z) = α Li (j, l, z).

Let us note

ξ u = ⌊log 2 (dN 4K X δ -1 u )⌋ + 1 and define the map Γ-β,-2 (L u ) i : T d → R by Γ-β,-2 (L u ) i (v) = ∞ j=0 2 d l=1 z∈{0,...,2 j -1} d 2 -jβ (1 + j) -2 S(j, l, z) i α (Lu)i (j, l, z)ψ per j,l,z (v), the β, 2 wavelet regularization of L ui . Then Γ-β,-2 (L u ) ∈ H β C (T d , R p ) ∩ H ⌊β⌋+1 CKX δ -(1-(β-⌊β⌋)) u (T d , R p ).
Let i 1 , ..., i ⌊β⌋+1 ∈ {1, ..., d}, we have

|∂ ⌊β⌋+1 i1,...,i⌊β⌋+1 Γ-β,-2 (L) i (u)| = |∂ ⌊β⌋+1 i1,...,i⌊β⌋+1 ∞ j=0 2 d l=1 z∈{0,...,2 j -1} d α Γ-β,-2 (Li) (j, l, z)ψ per j,l,z (u)| ≤|∂ ⌊β⌋+1 i1,...,i⌊β⌋+1 ξu j=0 2 d l=1 z∈{0,...,2 j -1} d (α Γ-β,-2 (Li) (j, l, z) -α Γ-β,-2 (Lu)i (j, l, z))ψ per j,l,z (u)| + |∂ ⌊β⌋+1 i1,...,i⌊β⌋+1 ξu j=0 2 d l=1 z∈{0,...,2 j -1} d α Γ-β,-2 (Lu)i (j, l, z)ψ per j,l,z (u) + ∂ ⌊β⌋+1 i1,...,i⌊β⌋+1 ∞ j=ξu+1 2 d l=1 z∈{0,...,2 j -1} d α Γ-β,-2 (Li) (j, l, z)ψ per j,l,z (
(α Γ-β,-2 (Li) (j, l, z) -α Γ-β,-2 (Lu)i (j, l, z))ψ per j,l,z (u)| =| ξu j=0 2 d l=1 z∈{0,...,2 j -1} d (α Γ-β,-2 (Li) (j, l, z) -α Γ-β,-2 (Lu)i (j, l, z)) × 2 j(⌊β⌋+1+d/2) d k=1 m∈Z ∂ θk ψ lk (2 j (u k -m) -z k )| ≤C ξu j=0 2 d l=1 z∈{0,...,2 j -1} d 2 j(⌊β⌋+1-β) | d k=1 m∈Z ∂ θk ψ lk (2 j (u k -m) -z k )| ≤C ξu j=0 z∈{0,...,2 j -1} d 2 j(⌊β⌋+1-β) 1 {u∈supp(ψ per jlz )} ≤C ξu j=0 2 j(⌊β⌋+1-β) ≤ C2 ξu(⌊β⌋+1-β) ≤ CK X δ -(⌊β⌋+1-β) u .
For the second term we have

|∂ ⌊β⌋+1 i1,...,i⌊β⌋+1 ξu j=0 2 d l=1 z∈{0,...,2 j -1} d α Γ-β,-2 (Lu)i (j, l, z)ψ per j,l,z (u) + ∂ ⌊β⌋+1 i1,...,i⌊β⌋+1 ∞ j=ξu+1 2 d l=1 z∈{0,...,2 j -1} d α Γ-β,-2 (Li) (j, l, z)ψ per j,l,z (u)| = |∂ ⌊β⌋+1 i1,...,i⌊β⌋+1 ∞ j=0 2 d l=1 z∈{0,...,2 j -1} d α Γ-β,-2 (Lu)i (j, l, z)ψ per j,l,z (u)| ≤ ∥∂ ⌊β⌋+1 i1,...,i⌊β⌋+1 Γ-β,-2 (L u ) i ∥ ∞ ≤ CK X δ -(⌊β⌋+1-β) u .
The same way, let us show that for u, v ∈ T d we have

(D.22) |∂ ⌊β⌋+1 i1,...,i⌊β⌋+1 Γ-β,-2 (L) i (u) -∂ ⌊β⌋+1 i1,...,i⌊β⌋+1 Γ-β,-2 (L) i (v)| ≤ CK X (δ -1 u ∨ δ -1 v )∥u -v∥ β-⌊β⌋ . Suppose that ∥u -v∥ ≤ δu∨δv 8KX , then define L u,v coinciding with L on B d (arg max w∈{u,v} δ w , δu∨δv 8KX ). Otherwise, if ∥u -v∥ > δu∨δv 8KX define L u,v coinciding with L on B d (u, δu 16KX ) ∪ B d (v, δv 16KX ). In both cases we have L u,v ∈ H 1 16KX δ -1 u ∨δ -1 v .
In the first case extend L u,v by projection on the ball as we did with L u in (D.20), in the second case we extend it with the sum of the projections:

L u,v (w) =L u,v (π Bu (w))(0 ∨ (1 -24K X δ -1 u ∥w -π Bu (w)∥)) + L u,v (π Bv (w))(0 ∨ (1 -24K X δ -1 v ∥w -π Bv (w)∥)).
In both cases we also have L

u,v ∈ H 1 Cδ -1 u ∨δ -1 v .
Then by splitting the wavelet coefficients of ∂

⌊β⌋+1 i1,...,i⌊β⌋+1 Γ-β,-2 (L) i (u) -∂ ⌊β⌋+1 i1,...,i⌊β⌋+1
Γ-β,-2 (L) i (v) as in (D.21), we obtain (D.22).

Let A and H : A → R defined in (D.18) and (D.19). We have g

-1 • T -1 g • T ∈ H β+1 C (A, T d ), Id -T ∈ H β+1 KT (R p , R p ) and Γ-β,-2 (L) ∈ H β C (T d , R p
), so we get from the Faa di Bruno formula that H ∈ H β C (A, R). Let i 1 , ..., i ⌊β⌋+1 ∈ {1, ..., p} and write P ({i 1 , ..., i ⌊β⌋+1 }) the set of subsets of {i 1 , ..., i ⌊β⌋+1 }. We have

∂ ⌊β⌋+1 i1,...,i⌊β⌋+1 H(x) = S∈P ({i1,...,i⌊β⌋+1}) ∂ |S| i∈S ( Γ-β,-2 (L) • g -1 • T -1 g • T )(x) , ∂ ⌊β⌋+1-|S| i / ∈S (Id -T )(x) .
For all S ∈ P ({i 1 , ..., i ⌊β⌋+1 }) with |S| < ⌊β⌋ + 1, we have that

x → ∂ |S| i∈S ( Γ-β,-2 (L) • g -1 • T -1 g • T )(x) , ∂ ⌊β⌋+1-|S| i / ∈S (Id -T )(x) ∈ H β-⌊β⌋ C (A, R).
From the previous calculations, we know that

|∂ ⌊β⌋+1 i1,...,i⌊β⌋+1 Γ-β,-2 (L) i (g -1 • T -1 g • T (x))| ≤ CK X δ -(⌊β⌋+1-β) g -1 •T -1 g •T (x) = CK X ∥T -1 g • T (x) -T (T -1 g • T (x))∥ -(⌊β⌋+1-β) = CK X ∥T -1 g • T (x) -T (x)∥ -(⌊β⌋+1-β) ≤ CK X ∥x -T (x)∥ -(⌊β⌋+1-β) so | ∂ ⌊β⌋+1 i1,...,i⌊β⌋+1 ( Γ-β,-2 (L) • g -1 • T -1 g • T )(x) , (Id -T )(x) | ≤ CK C3 X ∥x -T (x)∥ β-⌊β⌋ . Let x, y ∈ A with ∥x -T (x)∥ ≤ ∥y -T (y)∥, from (D.22) we get | ∂ ⌊β⌋+1 i1,...,i⌊β⌋+1 ( Γ-β,-2 (L) • g -1 • T -1 g • T )(x) , (Id -T )(x) -∂ ⌊β⌋+1 i1,...,i⌊β⌋+1 ( Γ-β,-2 (L) • g -1 • T -1 g • T )(y) , (Id -T )(y) | ≤| ∂ ⌊β⌋+1 i1,...,i⌊β⌋+1 ( Γ-β,-2 (L) • g -1 • T -1 g • T )(x) -∂ ⌊β⌋+1 i1,...,i⌊β⌋+1 ( Γ-β,-2 (L) • g -1 • T -1 g • T )(y) , (Id -T )(x) | + | ∂ ⌊β⌋+1 i1,...,i⌊β⌋+1 ( Γ-β,-2 (L) • g -1 • T -1 g • T )(y) , (Id -T )(x) -(Id -T )(y) | ≤CK C3 X min(∥x -T (x)∥, ∥y -T (y)∥) -1 ∥x -y∥ β-⌊β⌋ ∥x -T (x)∥ + ∥T -1 g • T (y) -T (y)∥ -(1-(β-⌊β⌋)) ∥(Id -T )(x) -(Id -T )(y)∥ ≤CK C3 X ∥x -y∥ β-⌊β⌋ + ∥y -T (y)∥ -(1-(β-⌊β⌋)) ∥(Id -T )(x) -(Id -T )(y)∥ ≤C∥x -y∥ β-⌊β⌋ + ∥y -T (y)∥ -(1-(β-⌊β⌋)) 2∥y -T (y)∥ 1-(β-⌊β⌋) ∥(Id -T )(x) -(Id -T )(y)∥ β-⌊β⌋ ≤C∥x -y∥ β-⌊β⌋ .
Therefore we have finally that H ∈ H β+1 C (A, R).

D.3.3. Extension of the potential to the whole R p space. Let us now prove that we can extend H to the whole R p using Whitney's extension Theorem (Theorem D.20).

PROPOSITION D.17. The map H ∈ H β+1 C (A, R) can be extended into a map in H β+1 C (R p , R).

PROOF. Let x 1 , ..., x k ∈ A and define

P i (x) = ⌊β⌋+1 l=0 ∇ l H(x i ) (x -x i ) l l! ,
with the notations ∇ l H(x i ) the l-differential of H and

∇ l H(x i )(x -x i ) l = ∇ l H(x i ) x -x i , x -x i , ..., x -x i ).
Then the P i check conditions a) and b) for M = ∥H∥ H β+1 (A,R) . Let show that they also verify condition c). First, for α = ⌊β⌋ + 1 we have

∥∇ ⌊β⌋+1 P i (x i ) -∇ ⌊β⌋+1 P j (x i )∥ = ∥∇ ⌊β⌋+1 H(x i ) -∇ ⌊β⌋+1 H(x j )∥ ≤ C∥x i -x j ∥ β-⌊β⌋ .
Let α ∈ {0, ..., ⌊β⌋} and γ : [0, 1] → A continuous and differentiable almost everywhere such that γ(0) = x j and γ(1) = x i . Then, from Lemma D.21 we get

∥∇ α P i (x i ) -∇ α P j (x i )∥ = ∥∇ α H(x i ) - ⌊β⌋+1-α l=0 ∇ l+α H(x j ) (x i -x j ) l l! ∥ ≤ C∥ γ∥ β+1-α L 1
.

Let us now define a path from x i to x j . Suppose first that ∥x i -x j ∥ ≤ t/4 (for t > 0 the radius of definition of T : M t g ⋆ → M g ⋆ ), and define P A : M t g ⋆ → A by

P A (x) =    x if x ∈ A x-T (x) ∥x-T (x)∥ ∥T -1 g • T (x) -T (x)∥ + T (x) if x / ∈ A For x / ∈ A we have ∇P A (x) = ∥T -1 g • T (x) -T (x)∥ ∥x -T (x)∥ ∇(Id -T )(x) - x -T (x) ∥x -T (x)∥ (x -T (x)) ⊤ ∥x -T (x)∥ ∇(Id -T )(x) + x -T (x) ∥x -T (x)∥ (x -T (x)) ⊤ ∥x -T (x)∥ ∇(T -1 g • T -T )(x) + ∇T (x), so ∥∇P A (x)∥ ≤ 4K 2 T . Let γ : [0, 1] → A defined by γ(s) = P A (x i + s(x j -x i )),
which is well defined as for s ∈ [0, 1],

d(x i + s(x j -x i ), M g ⋆ ) ≤ ∥x i + s(x j -x i ) -x i ∥ + d(x i , M g ⋆ ) ≤ t/4 + t/2.
We have ∥ γ(s)∥ ≤ 4K 2 T ∥x i -x j ∥ so we deduce that

∥∇ α P i (x i ) -∇ α P j (x i )∥ ≤ C∥x i -x j ∥ β+1-α .
Suppose now that ∥x i -x j ∥ ≥ t/4 and define

γ(s) =              x i + 3s(π g ⋆ (x i ) -x i ) if s ∈ [0, 1/3] g ⋆ g ⋆-1 (π g ⋆ (x i )) + 3(s -1/3)(g ⋆-1 (π g ⋆ (x j )) -g ⋆-1 (π g ⋆ (x i )) if s ∈ (1/3, 2/3) π g ⋆ (x j ) + 3(s -2/3)(x j -π g ⋆ (x j )) if s ∈ [2/3, 1],
the path from x i to x j that starts by projecting x i onto M g ⋆ , then circulates along M g ⋆ until it reaches the projection of x j , and finally go to x j in a straight line.

For s ∈ [0, 1/3] ∪ [2/3, 1] we have ∥ γ(s)∥ ≤ 3 max(∥π g ⋆ (x i ) -x i ∥, ∥π g ⋆ (x j ) -x j ∥) ≤ 3t ≤ 12∥x i -x j ∥ and for s ∈ (1/3, 2/3) ∥ γ(s)∥ ≤ 3K 2 ∥π g ⋆ (x i ) -π g ⋆ (x j )∥ ≤ 3K 3 ≤ 12K 3 t -1 ∥x i -x j ∥.
We then get as in the case where ∥x i -x j ∥ ≤ t/4, that ∥∇ α P i (x i ) -∇ α P j (x i )∥ ≤C∥x i -x j ∥ β+1-α .

Therefore using the sharp form of Whitney's extension Theorem D.20, H can be extended to a map in H β+1 C (R p , R).

We conclude the proof of Lemma 5.2 by noticing that (1 + j) -2 S(j, l, w)α D (j, l, w)ψ jlw (x)

E U [⟨ Γ-β,-
for S(j, l, w) = α D (j,l,w)(α f t T (j,l,w)-α f t g ⋆ (j,l,w)) |α D (j,l,w)(α f t T (j,l,w)-α f t g ⋆ (j,l,w))| . Then using Proposition B.4, we have L 2 (R p )

Γγ,-2

ϵ (D), Γ β (f t T -f t g ⋆ ) β+1-γ 2β+1 L 2 (R p )
.

Like in the proof of lemma 5.2, we have As T verifies point iii) of Definition D.3, we have F D ∈ H β+1 C (M g ⋆ , R) so we can conclude that LEMMA D.18 (Federer (1959), Theorem 4.18). The reach r ∈ [0, ∞) of a submanifold M verifies r = inf q̸ =p∈M ∥q -p∥ 2 2d(q -p, T p (M))

E U ∼U ([0,1] d ) [D(T (g(U ))) -D(g ⋆ (U ))] ≤ C sup f ∈H β+1 1 (M g ⋆ ,R) E U ∼U ([0,1] d ) [f (T (g(U ))) -f (g ⋆ (U ))]
.

Using this result, let us show an upper bound on the reach.

LEMMA D.19. Let g ∈ H β+1 K (T d , R) injective that verifies the K 2 -regularity density condition and such that M g = g([0, 1] d ) is a submanifold. Then if there exist δ ∈ (0, (2K 2 K) -1 ) and u ⋆ , v ⋆ ∈ T d such that ∥u ⋆ -v ⋆ ∥ ≥ δ and ∥g(u ⋆ ) -g(v ⋆ )∥ < K -1 2 δ -Kδ 2 , then r g ≤ ∥g(u ⋆ ) -g(v ⋆ )∥/2. PROOF. For u, v ∈ T d we have ∥g(u) -g(v)∥ = ∥∇g(v)(u -v) -O(∥u -v∥ 2 )∥ ≥ K -1 2 ∥u -v∥ -K∥u -v∥ 2 so ∥u -v∥ = δ implies that ∥g(u) -g(v)∥ ≥ K -1 2 δ -Kδ 2 . Let (x, y) ∈ arg min ∥u-v∥≥δ ∥g(u) -g(v)∥, then as ∥g(x) -g(y)∥ ≤ ∥g(u ⋆ ) -g(v ⋆ )∥ < K -1 2 δ -Kδ 2 , we have ∥x -y∥ > δ. Defining F (u, v) = ∥g(u) -g(v)∥ 2 , we have that ∇F (x, y) = 0 so we deduce that g(x) -g(y) ∈ T g(y) (M g ) ⊥ . Then using Lemma D.18, we have r g = inf q̸ =p∈Mg ∥q -p∥ 2 2d(q -p, T p (M g ))

≤ ∥g(x) -g(y)∥ 2 2d(g(x) -g(y), T g(y) (M g ))

=

∥g(x) -g(y)∥ 2 ≤ ∥g(u ⋆ ) -g(v ⋆ )∥/2. D.5.2. Extension results. To extend maps that are defined only on a subspace of R p , we will use the following version of Whitney's extension Theorem (Theorem A in [START_REF] Charles L Fefferman | A sharp form of whitney's extension theorem[END_REF] and "Whitney's extension Theorem" in [START_REF] Fefferman | Extension of cˆm,ω-smooth functions by linear operators[END_REF]). THEOREM D.20. Given β, p ≥ 1, there exists k ∈ N 0 depending only on p and β, for which the following holds. Let f : E → R with E an arbitrary subset of R p . Suppose that for any k distinct points x 1 , ..., x k ∈ E, there exists ⌊β⌋ + 1 degree polynomials P 1 , ..., P k on R p , satisfying a) P i (x i ) = f (x i ) for i = 1, ..., k; b) ∥∇ α P i (x i )∥ ≤ M for i = 1, ..., k and |α| ≤ ⌊β⌋ + 1 and c) ∥∇ α (P i -P j )(x i )∥ ≤ M ∥x i -x j ∥ β+1-α for i, j = 0, ..., k and |α| ≤ ⌊β⌋ + 1 with M independent of x 1 , ..., x k

Then f extends to H β+1 C (R p , R).

Let us now prove a bound on the growth of smooth functions defined on arbitrary subset of R p . LEMMA D.21. Let E ⊂ R p , x 1 , x 2 ∈ E and γ : [0, 1] → E continuous and differentiable almost everywhere with γ(0) = x 2 and γ(1) = x 1 . Then for all η > 0, k ∈ N and map F ∈ H η 1 (E, R k ), we have

∥F (x 1 ) - ⌊η⌋ i=0 ∇ i F (x 2 ) (x 1 -x 2 ) i i! ∥ ≤ ∥ γ∥ η L 1 .
PROOF. This result is proven by induction. The case ⌊η⌋ = 0 is straightforward. Let us prove the case ⌊η⌋ ≥ 1 supposing that the result is true for ⌊η⌋ -1. We have

∥F (x 1 ) - ⌊η⌋ i=0 ∇ i F (x 2 ) (x 1 -x 2 ) i i! ∥ E.2. Proof of proposition E.2.
PROOF. Let us first show that g is injective. As g verifies the χ-density regularity condition and belongs to H 2 χ , we have for all u, v ∈ T d , ∥g(u) -g(v)∥ ≥ χ -1 ∥u -v∥ -χ∥u -v∥ 2 .

Then for ∥u -v∥ ≤ 1 2χ 2 we have, ∥g(u) -g(v)∥ ≥ 1 2χ ∥u -v∥.

Suppose there exists u, v ∈ T d different such that g(u) = g(v), then from the previous derivation we have ∥u -v∥ > 1 2χ 2 . Define z u , z v ∈ {0, ..., [2 5 √ dK 2 χ 2 ]} d such that ∥u -

z u 2 5 √ dK 2 χ 2 ∥ + ∥v - z v 2 5 √ dK 2 χ 2 ∥ ≤ 1 16K 2 χ 2 .
Then, we have

∥ z u 2 5 √ dK 2 χ 2 - z v 2 5 √ dK 2 χ 2 ∥ ≥ ∥u -v∥ -(∥u - z u 2 5 √ dK 2 χ 2 ∥ + ∥v - z v 2 5 √ dK 2 χ 2 ∥) ≥ 1 2χ 2 - 1 16K 2 χ 2 ≥ 1 4χ 2 and ∥g( z u 2 5 √ dK 2 χ 2 ) -g( z v 2 5 √ dK 2 χ 2 )∥ ≤∥g(u) -g(v)∥ + K(∥u - z u 2 5 √ dK 2 χ 2 ∥ + ∥v - z v 2 5 √ dK 2 χ 2 ∥) ≤ 1 16Kχ 2 ,
which is impossible by definition of the χ-numerical regularity condition. Therefore, we have that g is injective and being a local diffeomorphism, it is a diffeomorphism so in particular M g is a submanifold.

Let us now show that its reach is greater than (Cχ 5 ) -1 . We are going to use the characterisation of the reach for submanifold (Lemma D.18). Doing the same derivation as (D.1), we obtain that for all u, v ∈ T d , ∥g(u) -g(v)∥ 2 2d(g(u) -g(v), T g(v) (M g ))

≥ 1 χ (χ -2 -χ 2 (4∥u -v∥ + ∥u -v∥ 2 )), so ∥u -v∥ ≤ 1 2 6 χ 4 implies that ∥g(u) -g(v)∥ 2 2d(g(u) -g(v), T g(v) (M g ))

≥ 1 2χ 3 .

Let now consider the case ∥u -v∥ ≥ 1 2 6 χ 4 . We have ∥g(u) -g(v)∥ 2 2d(g(u) -g(v), T g(v) (M g ))

≥ ∥g(u) -g(v)∥ 2 , with |α f (j, l, w)| ≤ (C η K + Cδ η+γ )2 -j(η+d/2) , for j ≤ log 2 (δ -1 ) and |α f (j, l, w)| ≤ C2 -j(⌊β⌋+2+d/2) δ η+γ , for j > log 2 (δ -1 ).

Then if η is not an integer, we have ∥f ∥ H η ≤ C as B η ∞,∞ ⊂ H η . Now if η is an integer we have (1 + j) 2 + Cδ ⌊β⌋+2-η log(δ -1 ) 2 ≤ C log(δ -1 ) 2 .

∥f ∥ H η ≤ C∥f ∥ B η,2 ∞,∞ = C sup
Using Proposition E.3, we have that if β is not an integer then ) to be small. LEMMA E.4. For all g ⋆ ∈ H β+1 K (T d , R p ), the GAN estimator ĝ of (1.3) with G and D defined in (5.1) , verifies that with probability at least 1/n we have ≤ sup

D ⋆ ∈H β+1 1 (B p (0,K),R) inf D∈D ∥D -D ⋆ ∥ ∞ ≤ sup D ⋆ ∈H β+1 1 (B p (0,K),R) inf D∈D ∥D -D ⋆ ∥ B 0,2 ∞,∞ ≤ C log(n) 2 n -β 2 β+d .
Finally, taking δ = 1 and putting everything in the bound of Theorem 5.4, we get the result.

As discussed Section E.1, using Lemma E.4 and Proposition E.2 for n large enough, we have that ĝ verifies the (Cχ 5 ∨ ∥ĝ∥ H β+1 )-manifold regularity condition. E.6.2. Approximation error of the class G. From Proposition 3.7, we know that (E.2)

inf g∈ F β+1,n - 1 2β+d per d H β+1 1 (g #U , g ⋆ #U ) ≤ C log(n) C2 n -β+ β+1 2 β+d .
Therefore it suffices to prove that there exists g ∈ Fβ+1,n -1 2β+d per that verifies both (E.2) and the χ-numerical regularity condition. We show in the proof of the next result that the map being the low wavelet frequency approximation of g ⋆ by neural networks, verifies these conditions. PROPOSITION E.5. For all g ⋆ ∈ H β+1 K (T d , R p ) that verifies the K-density regularity condition of Definition 2.1, the class G defined in (5.1) verifies for α g ⋆ (j, l, z) = ⟨g ⋆ , ψ per jlz ⟩. Then, from Proposition A.2 there exist coefficients (α g (j, l, z)) such that g = ∞ j=0 2 d l=1 z∈{0,...,2 j -1} d α g (j, l, z)ψ per jlz , with |α g (j, l, w) -α g ⋆ (j, l, w)| ≤ Cn -β+ β+2 2β+d 2 -j(β+1+d/2) , for j ≤ log 2 (n 1 2β+d ) and |α g (j, l, w)| ≤ C2 -j(⌊β⌋+2+d/2) n -β+ β+2 2β+d , for j > log 2 (n 1 2β+d ).

∆ G = inf g∈G d H β+1 1 (g #U , g ⋆ #U ) ≤ C log(n) C2 n -
Let u, v ∈ T d with ∥u -v∥ ≥ (4χ 2 ) -1 , we have ≥ ∥g ⋆ (u) -g ⋆ (v)∥ -C log(n) 2 n -β+1 2β+d .

∥g(u) -g(v)∥ ≥ ∥g ⋆ (u) -g ⋆ (v)∥ -2∥g -g ⋆ ∥ ∞ ≥∥g ⋆ (u) -g ⋆ (v)∥ -2C∥g -g ⋆ ∥ B 0,2 ∞,∞ ≥∥g ⋆ (u) -g ⋆ (v)∥ -C sup
Using Lemma D.19 with δ = (4χ 2 ) -1 , we have that if ∥g ⋆ (u) -g ⋆ (v)∥ ≤ K -1 (4χ 2 ) -1 -K(4χ 2 ) -2 , Then, using Theorem 5.4 and Lemma 5.6, we have

E d H β+1 1 (ĝ #U , g ⋆ #U ) ≤E[∆ ĝ D ] + C log(n) C2 n -2β+1 2β+d ≤C log(n) C3 n -β 2β+d E d H β+1 1 (ĝ #U , g ⋆ #U ) β+1 2β+1 + n -β+1 2β+d . Now if E d H β+1 1 (ĝ #U , g ⋆ #U ) ≤ n -2β+1 2β+d
we have the result, otherwise

n -β+1 2β+d ≤ E d H β+1 1 (ĝ #U , g ⋆ #U ) β+1 2β+1 so E d H β+1 1 (ĝ #U , g ⋆ #U ) ≤ C log(n) C3 n -β 2β+d 2E d H β+1 1 (ĝ #U , g ⋆ #U ) β+1 2β+1
, so

E d H β+1 1 (ĝ #U , g ⋆ #U ) β 2β+1 ≤ C log(n) C3 n -β 2β+d
which gives

E d H β+1 1 (ĝ #U , g ⋆ #U ) ≤ C log(n) C3 n -2β+1 2β+d .
Let us treat now the case β + 1 ≥ d/2. For δ = n -d/2 2 β+d , we obtain

∆ G + min δ∈[0,1] (δ + 1/n) 2 log(n|G 1/n ||(D G ) 1/n |) n + 1 √ n (1 + log(δ -1 )) ≤ C log(n) C2 n -1 2 ,
and as for the case β + 1 < d/2, using Theorem 5.4 and Lemma 5.6, we have

E d H d/2 1 (ĝ #U , g ⋆ #U ) ≤ C log(n) C3 n -β 2 β+d E d H d/2 1 (ĝ #U , g ⋆ #U ) β+1 2 β+1 + n -β+1 2 β+d . Now if E d H d/2
1 (ĝ #U , g ⋆ #U ) ≤ n -1 2 we have the result, otherwise

E d H d/2 1 (ĝ #U , g ⋆ #U ) ≤ C log(n) C3 n -β 2 β+d E d H d/2 1 (ĝ #U , g ⋆ #U ) β+1 2 β+1 .
We conclude the same way by

E d H d/2 1 (ĝ #U , g ⋆ #U ) β 2 β+1 ≤ C log(n) C3 n -β 2 β+d
which gives

E d H d/2 1 (ĝ #U , g ⋆ #U ) ≤ C log(n) C3 n -2 β+1 2 β+d = C log(n) C3 n -1 2 .

  operator for any multi-index ν = (ν 1 , ..., ν d ) ∈ N d 0 with |ν| := ν 1 + ... + ν d ≤ ⌊η⌋. Write ∥f i ∥ η-⌊η⌋ = sup x̸ =y fi(x)-fi(y) min{1,∥x-y∥ η-⌊η⌋ } and let

PROPOSITION

  Then, the question of what should be the regularity of the class of discriminators D arises. Let us first show using Proposition B.1, that IPMs of high regularity control IPMs of lower regularity.

COROLLARY

  (5.1) G := {g ∈ Fβ+1,n -1 2β+d per | g verifies the χ-numerical regularity condition} to be the class of generators. As mentioned in Section 4.1, the approximation error and covering numbers of the class Fβ+1,n -1 2β+d per are well suited for the GAN to attain minimax rates (see Proposition 3.7). Proposition E.5 shows that requiring the generators to verify the χ-numerical regularity condition does not deteriorate the approximation error of the class G. Define (5.2) β + 1 := (β + 1) ∧ d/2, write δn := n -

  be small. From Proposition B.6 we can now give the proof of Theorem 3.1. PROOF OF THEOREM 3.1 . Let D g ∈ arg max D∈D L(g, D) and g ∈ arg min g∈G L(g, D ⋆ g ), we can decompose the objective as

  using Proposition 4.6. APPENDIX D: PROOF OF THE INTERPOLATION INEQUALITY IN THE MANIFOLD CASE (SECTION 5.1) D.1. Definition of the (g #U , g ⋆ #U

  Properties of the projection onto M gs . Define π s : M rg s gs → M gs as the orthogonal projection. Let us first show that π s verifies the point ii) of Definition D.3 on a neighborhood of M gs . PROPOSITION D.8. There exists a constant C > 0 such that the projection π s verifies the point ii) of Definition D.3 on M C -1gs .

  u)|. (D.21) For k ∈ {1, ..., d}, let us write θ k = ⌊β⌋+1 r=1 1 {ir=k} . For the first term of (D.21) we have

E

  2 (L)(U ), g(U ) -T (g(U ))⟩] = E U [H(g(U )) -H(T (g(U )))] U [f (g(U )) -f (T (g(U )))].D.4. Proof of Lemma 5.3. PROOF. By the point i) of definition of the (g #U , g ⋆ #U ) compatibility (Defintion D.3), we have that T • g #U admits a density f T ∈ H β KT (M g ⋆ , R) with respect to the volume measure on M g ⋆ . Define the t-envelope of f T and f g ⋆ as in Definition B.3 but in the case where the densities have some regularity:f t T (x) = f T (T (x))Θ(4∥x -T (x)∥ 2 /t 2 )|ap d (∇T (x))| R p-d Θ(4∥y∥ 2 /t 2 )dλ p-d (y) and f t g ⋆ (x) = f g ⋆ (T (x))Θ(4∥x -T (x)∥ 2 /t 2 )|ap d (∇T (x))| R p-d Θ(4∥y∥ 2 /t 2 )dλ p-d (y), for ap d the approximate jacobian from Definition D.1 andΘ ∈ H β C (R, R + ) such that Θ(x) = Θ(-x), Θ(0) = 1 and Θ |(1,∞) = 0.As ∇T belongs to H β KT (R p , L(R p , R p )) and ap d (∇T (x)) is bounded below by C -1 (T verifies point (iv) of Definition D.3), then from the Faa di Bruno formula, we deduce that ap d (∇T ) belongs toH β C (M t g ⋆ , R). Let D ∈ H γ C (R p , R) and define D : M t g ⋆ → R by D(x) = D(T (x))κ(2∥x -T (x)∥ 2 /t 2 ), for κ ∈ H γ C (R, R) such that κ |(-∞,1/2) = 1 and κ |(1,∞) = 0.Then, extending D by 0 to the whole space R p , we have that D ∈ H γ C (R p , R). From Proposition D.2, we haveE U [D(T (g(U ))) -D(g ⋆ (U ))] = M g ⋆ D(x)(f T (x) -f g ⋆ (x))dλ M g ⋆ (x) = M g ⋆ D(x)(f T (x) -f g ⋆ (x)) R p-d Θ(4∥y∥ 2 /t 2 )dλ p-d (y) T -1 ({x}) Θ(4∥z -T (z)∥ 2 /t 2 )dλ p-d (z)dλ M g ⋆ (x) = M g ⋆ T -1 ({x})D(T (z))(f T (T (z)) -f g ⋆ (T (z)))Θ(4∥z -T (z)∥ 2 /t 2 ) R p-d Θ(4∥y∥ 2 /t 2 )dλ p-d (y)dλ p-d (z)dλ M g ⋆

  -f t g ⋆ (x))dλ p (x) + Cϵ.Applying Proposition B.1 for h1 = Γ0,-2 ϵ (D), h 2 = f t T -f t g ⋆ , s 1 = γ, b 1 = 2, s 2 = β, b 2 = 0, τ = -(β + 1 -γ), t = 1, r = γ β+γ and q = 2β+1 β+γ we get (x)(f t T (x) -f t g ⋆ (x))dλ p (x) ≤ Γγ-(β+1)

F

  (j, l, w)||α fT (j, l, w) -α f g ⋆ (j, l, w)| j) -2 C2 -jp 1 {supp(ψjlw)∩B p (0,K)̸ =∅} ≤ C ∞ j=0 (1 + j) -2 ≤ C. Define F D : M g ⋆ → R by -T (z)∥ 2 /t 2 ) R p-d Θ(4∥y∥ 2 /t 2 )dλ p-d (y) dλ T -1 ({x}) (z). -f t g ⋆ (z))|ap d (∇T (z))| -1 dλ p-d (z)dλ M g ⋆ (T (z)) -f g ⋆ (T (z)))Θ(4∥z -T (z)∥ 2 /t 2 ) R p-d Θ(4∥y∥ 2 /t 2 )dλ p-d (y) dλ p-d (z)dλ M g ⋆ (x) = M g ⋆ F D (x)(f T (x) -f g ⋆ (x))dλ M g ⋆ (x).

  Results on the reach of submanifolds. Let us give the characterization of the reach for submanifolds.

  j) 2 |α f (j, l, z)| + Cδ ⌊β⌋+2-η log(δ -1 ) 2 ≤ C sup j≤log(δ -1 ))

  U , g ⋆ #U ) ≤ Cn -β 2β+d .PROOF. We are going to use the bound given by Theorem 5.4. Using Lemma 5.5 we have that log(|(D G ) 1/n |) ≤ C log(n) 2 n d 2 β+d ,and from Proposition 3.7 we have |G 1/n | ≤ C log(n) 2 n d 2β+d . On the other hand, from Proposition E.5 we have that ∆G ≤ C log(n) C2 n -([0,1] d ) [D(g(U )) -D(g ⋆ (U ))]

  ...,2 j -1} d α g ⋆ (j, l, z) ψper jlz (u),

  ..,2 j -1} d |α g (j, l, w) -α g ⋆ (j, l, w)| ≥∥g ⋆ (u) -g ⋆ (v)∥ -

  γ C we have log(|Θ ϵ |) ≤ C log(|(H γ 1 ) ϵ |) and from Donoho et al. (1998) we know that log(|(H γ 1

  Using this result, let us give a first (sub-optimal) bound ond H β+1 1 (ĝ #U , g ⋆ #U) that will allow us to use the different results that require the distance d H β+1 1 (ĝ #U , g ⋆ #U

	if β is an integer Fβ+1,n -1 2β+d per	⊂ H β+1 C log(n) 2 .	Fβ+1,n -1 2β+d per	⊂ H β+1 C , and

PROOF. Suppose first that ∥u -v∥ ≥ min(δ u , δ v )/(4K X ). Then ∥L(u) -L(v)∥ ≤ 2 = 2∥u -v∥ -α ∥u -v∥ α ≤ 8K X min(δ u , δ v ) -α ∥u -v∥ α . Now if ∥u -v∥ ≤ min(δ u , δ v )/(4K X ), then for u s = u + s(v -u) with s ∈ [0, 1], we have

Furthermore, the differential of L is defined almost everywhere and values

Note that we have

so we are now going to focus on finding a function

up to logarithmic terms. Let us define

for T -1 g : M g ⋆ → M g the inverse application from Definition D.3. We have the following result on H. PROPOSITION D.16. The application H belongs to H β+1 C (A, R) and for all ϵ ∈ (0, 1) we have

, where we applied the induction hypothesis to the function ∇F . APPENDIX E: PROOFS OF THE BOUND IN THE MANIFOLD CASE (SECTION 5.2) E.1. Numerical regularity condition. Let us define an easy to compute condition that will ensure that ĝ verifies the manifold regularity condition. DEFINITION E.1. For χ > K, we say that a map g ∈ H 2

This condition ensures that if two points are far in the torus, then their images by g are not too close. The following proposition asserts that the numerical and density regularity conditions together, imply the manifold regularity condition.

) verify the χnumerical regularity and χ-density regularity conditions. Then g is injective and its image M g has a reach lower bounded by (Cχ 5 ) -1 .

The proof of Proposition E.2 can be found in Section E.2. Let us now choose the value of the parameter χ that we use for our class of generators. Let C 2,β+1 > 0 such that Fβ+1,n

For n large enough, as with high probability we will have that d

), then ĝ will verify the assumptions of Proposition E.2 and therefore it will verify the (Cχ 5 ∨ ∥ĝ∥ H β+1 )-manifold regularity condition.

Note that in practice, the numerical regularity condition of Definition E.1 can be checked by adding to the loss of the GAN estimator (1.3) the term

Finally, using Lemma D.18 we get the result. E.3. Proof of Theorem 5.4.

PROOF. Recalling the notations of Definition B.5 for γ = β +1, define g ∈ arg min g∈G L(g, D ⋆ g ). Doing as in the proof of Theorem 3.1 we obtain that

For g ∈ G, write

Recalling the proof of Proposition B.6 we have for all δ > 0

Then solving with respect to α the equation

Now doing as in the proof of Theorem 3.1, we obtain that

and A k1,...,kd = [u k1,...,kd 1 , u k1,...,kd

For D ∈ F β+1,δ , we have by definition of D that there exist weights

As supp( ψjlw ) ⊂ p i=1

[2 -j (w i -C), 2 -j (w i + C)] and g is K-Lipschitz, we have for all i ∈ {1, ..., p} and j ≤ log 2 (⌊δ -1 ⌋),

which implies

and as |2 j-1 δ| < 1 they are at most 2 + 2C integers w i ∈ Z such that supp( ψjlw ) ∩ g i (A k1,...,kd ) ̸ = ∅. Therefore, D |g(Ak 1 ,...,k d ) the restriction of D to g(A k1,...,kd ) can be represented by log(δ -1 )(2 p (2 + 2C)) p weights and so D |g([0,1] d ) can be represented by (2Kδ -1 ) d log(δ -1 )(2 p (2 + 2C)) p weights. We can then conclude that ∀ϵ ∈ (0, 1),

).

E.5. Proof of Lemma 5.6.

PROOF. Let π g be an optimal transport plan for the Euclidean distance between g #U and g ⋆ #U . We have

Writing

β+1 β+ β+1 + 1 n E.6. Proof of Theorem 5.7. The proof breaks down as follow. First, we determine the smoothness of the functions in the class of generators G and show that with high probability, ĝ verifies the density and manifold regularity conditions. Then, we bound the term of approximation error of the generator class. Finally we prove that the GAN estimator attains the minimax rate for the distance d H β+1 1 using Theorem 5.4. E.6.1. Regularity of ĝ. Let us first determine the regularity of the functions in the class G.

PROPOSITION E.3. For all η ∈ (0, β + 1] and δ ∈ (0, 1), we have that if η is not an integer then Fη,δ per ⊂ H η C , and if η is an integer Fη,δ per ⊂ H η C log(δ -1 ) 2 .

PROOF. Let f ∈ Fη,δ per , then from Proposition A.2 there exist coefficients (α f (j, l, z)) such that

Furtermore,

) -1 , we would have r g ⋆ ≤ 3 8 (4Kχ 2 ) -1 < K -1 , which is impossible as g ⋆ verifies the K-manifold regularity condition. Therefore we deduce that

for n large enough. We conclude that g verifies the χ-numerical condition and thus belongs to G. Finally we can show like in the Proof of Proposition 3.6 Section B.3.2, that

E.6.3. Putting everything together. We now give the proof of Theorem 5.7.

PROOF OF THEOREM 5.7. By stability of Besov spaces for the differentiation (proposition 4.3.19, Giné and Nickl (2015)) we have

Therefore, by Lemma 5.6, we have

Suppose first that β + 1 < d/2. Then, using the bounds on the approximation errors and covering numbers derived previously, for δ = n -β+1 2β+d we have