François Bachoc
email: francois.bachoc@math.univ-toulouse.fr

Agnès Lagnoux
email: lagnoux@univ-tlse2.fr

Posterior contraction rates for constrained deep Gaussian processes in density estimation and classication

Keywords: Gaussian priors, deep Gaussian priors, Bayesian inference, nonparametric density estimation, classication, posterior contraction, smoothness classes, Matérn covariance functions. AMS 2000 subject classication: 60G15, 62G05, 62F15

We provide posterior contraction rates for constrained deep Gaussian processes in non-parametric density estimation and classication. The constraints are in the form of bounds on the values and on the derivatives of the Gaussian processes in the layers of the composition structure. The contraction rates are rst given in a general framework, in terms of a new concentration function that we introduce and that takes the constraints into account. Then, the general framework is applied to integrated Brownian motions, Riemann-Liouville processes, and Matérn processes and to standard smoothness classes of functions. In each of these examples, we can recover known minimax rates.

Introduction

Gaussian processes are widely used in statistics and machine learning to model a wide range of data [START_REF] Rasmussen | Gaussian Processes for Machine Learning[END_REF]. They have become a popular method for a large range of applications, such as geostatistics [START_REF] Bevilacqua | Estimating space and space-time covariance functions for large data sets: a weighted composite likelihood approach[END_REF][START_REF] Matheron | La théorie des variables régionalisées et ses applications[END_REF][START_REF] Porcu | Spatio-temporal covariance and crosscovariance functions of the great circle distance on a sphere[END_REF], numerical code approximation [START_REF] Sacks | Design and analysis of computer experiments[END_REF][START_REF] Santner | The Design and Analysis of Computer Experiments[END_REF][START_REF] Bachoc | Improvement of code behavior in a design of experiments by metamodeling[END_REF], calibration [START_REF] Paulo | Calibration of computer models with multivariate output[END_REF][START_REF] Bachoc | Calibration and improved prediction of computer models by universal Kriging[END_REF], and global optimization [START_REF] Jones | Ecient global optimization of expensive black box functions[END_REF]. In particular, they have been adopted in nonparametric Bayesian methods for constructing prior distributions for innite-dimensional statistical models in several settings. Many related theoretical contributions were obtained since the late 2000's [START_REF] Van Der Vaart | Rates of contraction of posterior distributions based on Gaussian process priors[END_REF] and practical contributions have been developed even before [START_REF] Lenk | The logistic normal distribution for Bayesian, nonparametric, predictive densities[END_REF]. We refer to the following books on the topic [START_REF] Rasmussen | Gaussian Processes for Machine Learning[END_REF][START_REF] Ghosal | Fundamentals of nonparametric Bayesian inference[END_REF] and the references therein. For instance, when considering the nonparametric estimation of a density relative to some measure µ from a sample (X 1 , . . . , X n ), one may consider as prior distribution the random density p Z (x) = e Z(x) e Z(y) dµ (y) ,

where (Z(x)) x∈X is a Gaussian process indexed by the space X of the observations. Here the exponential form forces the prior to weight only nonnegative functions and the renormalization permits to integrate to unity. We refer to Section 3.1 for more details on this context and also to, e.g., [START_REF] Rasmussen | Gaussian Processes for Machine Learning[END_REF][START_REF] Lenk | Towards a practicable Bayesian nonparametric density estimator[END_REF][START_REF] Tokdar | Posterior consistency of logistic Gaussian process priors in density estimation[END_REF][START_REF] Ghosal | Posterior consistency of Gaussian process prior for nonparametric binary regression[END_REF]. Dealing with classication is also of interest and can be handled again using Gaussian processes. Here the estimation of the binary regression function P(Y = 1|X = x) can be done from a bivariate sample ((X 1 , Y 1 ), . . . , (X n , Y n )) and using priors of the form

f Z (x) = Ψ(Z(x)),
where Ψ is a xed measurable function from R to (0, 1). We refer to Section 3.2 and [START_REF] Choudhuri | Nonparametric binary regression using a Gaussian process prior[END_REF][START_REF] Wood | A Bayesian approach to robust binary nonparametric regression[END_REF] for more details on this context. Other settings have also been considered in the literature as regression with xed covariates [START_REF] Van Der Vaart | Rates of contraction of posterior distributions based on Gaussian process priors[END_REF][START_REF] Choi | On posterior consistency in nonparametric regression problems[END_REF], and white noise models [START_REF] Van Der Vaart | Rates of contraction of posterior distributions based on Gaussian process priors[END_REF] to name a few. In the literature, several examples of Gaussian priors have been considered as the integrated Brownian motion [START_REF] Van Der Vaart | Rates of contraction of posterior distributions based on Gaussian process priors[END_REF], the Riemann-Liouville process [START_REF] Van Der Vaart | Rates of contraction of posterior distributions based on Gaussian process priors[END_REF][START_REF] Castillo | Lower bounds for posterior rates with Gaussian process priors[END_REF], the Matérn process [START_REF] Van Der | Information rates of nonparametric Gaussian process methods[END_REF], and the exponential process [START_REF] Ying | Asymptotic properties of a maximum likelihood estimator with data from a Gaussian process[END_REF]. Now given a prior as illustrated in the previous paragraph and the observations, Bayes' rule leads to a posterior distribution on the function of interest dening the unknown data distribution. One is then typically interested in proving posterior consistency, meaning that the posterior distribution converges to this function of interest as the sample size goes to innity (see, e.g., [START_REF] Barron | The consistency of posterior distributions in nonparametric problems[END_REF]). Another question of interest is the one of rates of contraction of posterior distributions based on Gaussian process priors. In a nutshell, the rate of contraction of the posterior corresponds to an ε n as small as possible such that the posterior probability of the ball centered at the function of interest and of radius ε n still converges to one in probability. Other directions have been investigated. In particular, while upper bounds are avalaible and express in terms of a concentration function involving the reproducing kernel Hilbert space (RKHS) associated to the Gaussian prior, the author of [START_REF] Castillo | Lower bounds for posterior rates with Gaussian process priors[END_REF] exhibits lower bound counterparts. More recently, more exibility has been allowed considering a randomly rescaled smooth Gaussian eld as a prior that adapts to smoothness [START_REF] Van Der Vaart | Adaptive Bayesian estimation using a Gaussian random eld with inverse gamma bandwidth[END_REF].

In this article, we consider deep Gaussian priors to reach further generality, similarly as deep neural networks are exploited to go beyond standard shallow neural networks. The reader is referred to [START_REF] Lee | Deep neural networks as Gaussian processes[END_REF][START_REF] Garriga-Alonso | Deep convolutional networks as shallow Gaussian processes[END_REF][START_REF] Matthews | Gaussian process behaviour in wide deep neural networks[END_REF][START_REF] Dunlop | How deep are deep Gaussian processes[END_REF] for some references on both deep Gaussian processes and deep neural networks. Deep Gaussian processes, introduced in [START_REF] Damianou | Deep Gaussian processes[END_REF], are non-Gaussian stochastic processes, constructed from a network of Gaussian processes, similarly as neural networks. A deep Gaussian process is then a stochastic process from R d to R of the form

Z H • Z H-1 • • • • • Z 1 , (1)
where, for h = 1, . . . , H, Z h is a (multivariate) Gaussian process from R d h to R d h+1 , with the convention d 1 = d and d H+1 = 1. In analogy to neural networks, Z 2 , . . . , Z H-1 may be interpreted as hidden layer processes and H may be called the number of layers. In particular, since deep Gaussian processes are based on composing Gaussian processes, their mathematical analysis becomes challenging. Deep Gaussian processes are commonly used as a prior for a function that is observed exactly. For instance, [START_REF] Hebbal | Bayesian optimization using deep Gaussian processes with applications to aerospace system design[END_REF] tackles optimization and [START_REF] Radaideh | Surrogate modeling of advanced computer simulations using deep Gaussian processes[END_REF] deals with surrogate modeling of computer models with applications to nuclear engineering. Deep Gaussian processes are exploited as well with noisy/indirect function observations [START_REF] Damianou | Deep Gaussian processes[END_REF][START_REF] Salimbeni | Doubly stochastic variational inference for deep Gaussian processes[END_REF]. From a methodological point of view, deep Gaussian processes are also an ecient way to obtain more nonstationarity than simple Gaussian processes. This need for non-stationarity is largely acknowledged in the Gaussian process literature (see e.g. [START_REF] Gramacy | Bayesian treed Gaussian process models with an application to computer modeling[END_REF][START_REF] Marmin | Warped Gaussian processes and derivative-based sequential designs for functions with heterogeneous variations[END_REF]). Hence, because of its practical impact, the theoretical analysis of deep Gaussian processes is benecial. In this work, several Gaussian processes are constrained (in terms of bounded norms for the processes themselves and their rst derivatives) and then composed to form constrained deep Gaussian processes. The constraints are necessary for our theoretical analysis (see more details in Section 2.1 in the discussion of ( 2) and ( 3)) and may be useful in practice to exclude very irregular realizations from the Bayesian model. Note that these bounds and derivative constraints may also be enforced to standard Gaussian processes, for a better modeling in some situations, especially with additional available expertise [START_REF] Golchi | Monotone emulation of computer experiments[END_REF][START_REF] López-Lopera | Finite-dimensional Gaussian approximation with linear inequality constraints[END_REF][START_REF] Maatouk | Gaussian process emulators for computer experiments with inequality constraints[END_REF]. In the same vein of previous works, we investigate posterior contraction for density estimation and classication. We establish original general rates of contraction extending those available for classical Gaussian processes (see Theorems 3.1 and 3.2). The proofs are original and exploit the results established in [START_REF] Van Der Vaart | Rates of contraction of posterior distributions based on Gaussian process priors[END_REF]. In [START_REF] Van Der Vaart | Rates of contraction of posterior distributions based on Gaussian process priors[END_REF], the authors consider a single Gaussian process dened on a compact space and valued in R. A prior step before proving our results is then the construction of a single global Gaussian process from the collection of the Gaussian processes involved in the deep Gaussian process prior (see Appendix B).

In addition, we study several examples of deep Gaussian priors: integrated Brownian motions (Section 4.1) and Riemann-Liouville processes (Section 4.2) in dimension one, and Matérn processes in general dimension (Section 4.3). It appears that the optimal rates are recovered in these examples for stantard classes of functions. The proofs of these results rely on the proofs of the analog results in the context of classical (single) Gaussian processes [START_REF] Van Der Vaart | Rates of contraction of posterior distributions based on Gaussian process priors[END_REF][START_REF] Van Der | Information rates of nonparametric Gaussian process methods[END_REF]. We would like to mention that, while preparing this article, we have been aware of the independent work of Finocchio and Schmidt-Hieber [START_REF] Finocchio | Posterior contraction for deep Gaussian process priors[END_REF] on the same topic. Their paper presents many interesting results. In contrast to our work, they consider the problem of regression rather than density estimation and classication. They address adaptivity with respect to the smoothness and structure of the function of interest, which we do not. Their proofs are independent of the proofs of the present paper and the techniques used are also dierent. To our knowledge, [START_REF] Finocchio | Posterior contraction for deep Gaussian process priors[END_REF] is the only already existing work providing posterior contraction results for deep Gaussian processes. The paper is organized as follows. In Section 2, we present the setting and some notation. Section 3 is dedicated to posterior contraction for both density estimation and classication.

Examples of rates of contraction for specic function classes and priors are given in Section 4. Appendix A explains how to allow for more exibility on the constraints considered in Section 2 using linear transformations of inputs and outputs. In Appendix B, we transform our deep Gaussian process into a single global real-valued Gaussian process dened on a compact space in order to apply the results of [START_REF] Van Der Vaart | Rates of contraction of posterior distributions based on Gaussian process priors[END_REF]. The proofs of the results of Sections 3 and 4, and of Appendix B are postponed to Appendix C.

Z H • • • • • Z 1 .
Such a process gives a prior on the continuous functions from R d to R. An illustration is provided in Figure 1.

We consider the input space [-1, 1] d for the deep Gaussian process prior

Z H • • • • • Z 1 .
For our proof techniques, we constrain the process Z h , h = 2, . . . , H, to also have the xed input domain [-1, 1] d h . The reader is referred to Appendix B for more details. To do so, we condition Z 1 , . . . , Z H-1 by the following constraints, that we call value constraints: for

all h = 1, . . . , H -1, Z h ∞,[-1,1] d h 1, (2)
where we dene for any k 1 , k 2 ∈ N * , for any A ⊂ R k 1 , and for any f This set of constraints has strictly positive probability (see Lemma C.3). In addition, for technical reasons again (in particular for the proof of Lemma C.7), we consider a second set of constraints, that we call derivative constraints: for h = 2, . . . , H, for i = 1, . . . , d h+1 , and for j = 1, . . . , d h , there exists a xed deterministic constant K h,i,j such that

= (f 1 , . . . , f k 2 ) : A → R k 2 , f ∞,A = sup t∈A max =1,...,k 2 |f (t)|. t 1 t 2 t = (t 1 , t 2 ) Z 1,1 Z 1,3 Z 1,2 Z 1 = (Z 1,1 , Z 1,2 , Z 1,3 ) Z 2 Z 3
∂Z h,i ∂x j ∞,[-1,1] d h K h,i,j . (3)
This second set of constraints has again strictly positive probability (see Lemma C.3 applied to the derivatives of Z h,i for h = 2, . . . , H and for i = 1, . . . , d h+1 ). As explained, the previous constraints are necessary for our theoretical analysis and may also be useful in practice to exclude very irregular realizations from the Bayesian model. Observe that the form of the constraints in (2) can be extended to more exible bounds on the values of the components of Z 1 , . . . , Z H-1 (see Appendix A). Outside of Appendix A, we consider the constraints (2) for convenience of exposition. Now we index all the univariate components of Z 1 , . . . , Z H , dening the nite set

I = {(h, i); h ∈ {1, . . . , H}, i ∈ {1, . . . , d h+1 }}.
In the sequel, for (h, i) ∈ I, we denote by Z c,h,i a stochastic process from [-1, 1] d h to R which law is that of the process Z h,i conditioned by (2) and/or (3). More precisely, for any Borel set B, for h = 2, . . . , H -1, and for i = 1, . . . , d h+1 ,

P (Z c,h,i ∈ B) = P Z h,i ∈ B Z h,i ∞,[-1,1] d h 1, ∂Z h,i /∂x j ∞,[-1,1] d h K h,i,j , j = 1, . . . , d h (4)
and

   P (Z c,1,i ∈ B) = P Z 1,i ∈ B Z 1,i ∞,[-1,1] d 1 1 for i = 1, . . . , d 2 , P (Z c,H ∈ B) = P Z H ∈ B ∂Z H /∂x j ∞,[-1,1] d H K H,1,j , j = 1, . . . , d H . (5)
For h = 1, . . . , H, we let Z c,h = (Z c,h,1 , . . . , Z c,h,d h+1 ). Note that the independence of the processes Z h,i , for (h, i) ∈ I, yields that, for any Borel sets B 1 , . . . , B H ,

P (Z c,h ∈ B h , h = 1, . . . , H) =P Z h ∈ B h , h = 1, . . . , H Z h,i ∞,[-1,1] d h 1, h = 1, . . . , H -1, i = 1 . . . , d h+1 , ∂Z h,i /∂x j ∞,[-1,1] d h K h,i,j h = 2, . . . , H, i = 1 . . . , d h+1 , j = 1 . . . , d h .
For h = 1, . . . , H, let B h be the Banach space of the continuous functions from [-1, 1] d h to R endowed with the uniform norm. In addition, for any h = 2, . . . , H -1 and for any i = 1, . . . , d h+1 , we dene the subset B c,h,i of B h composed by the continuously dierentiable functions z ∈ B h satisfying (2) and (3) (with Z replaced by z). Similarly, for i = 1, . . . , d 2 , B c,1,i stands for the subset of the functions z ∈ B 1 satisfying (2) only and B c,H,1 stands for the subset of the continuously dierentiable functions z ∈ B H satisfying (3) only. Hence (4) and ( 5) simply rewrite, for any Borel set B ⊂ B h and for any (h, i) ∈ I,

P(Z c,h,i ∈ B) = P(Z h,i ∈ B|Z h,i ∈ B c,h,i ).
The constrained deep Gaussian process prior is then given by

Z c,H • • • • • Z c,1 .

A concentration function for constrained deep Gaussian processes

For a centered Gaussian process X on a space E, we dene its RKHS H X on E from the covariance function of

f ∞,1 = f ∞ + j=1 ∂f ∂x j ∞ . (6)
In [START_REF] Van Der Vaart | Rates of contraction of posterior distributions based on Gaussian process priors[END_REF], the rates of contraction for classical Gaussian processes are given in terms of a concentration function involving the RKHS associated to the Gaussian process prior. More precisely, the concentration function for a single Gaussian process X from a compact space E to R, with a continuous function x 0 on E, and ε > 0, is given by

φ x 0 (ε) = inf h∈H X h-x 0 ∞ <ε h 2 H X -log P ( X ∞ < ε) (7)
where • ∞ is the uniform norm for continuous functions from E to R. This function is interpreted as measuring the concentration around the xed function x 0 .

Here, we propose a novel concentration function adapted to the deep context and to the constraints. First, for h = 2, . . . , H and for i = 1, . . . , d h+1 , we consider z 0,h,i in the closure of

H h,i in (C 1 ([-1, 1] d h , R), • ∞,1
) and for i = 1, . . . , d 2 , we consider z 0,1,i in the closure of

H 1,i in (C 0 ([-1, 1] d 1 , R), • ∞ ). Second, we dene, for ε > 0, Φ c,z 0 (ε) := d 2 i=1    3 2 inf g∈H 1,i g-z 0,1,i ∞ <ε g 2 H 1,i -2 log P Z 1,i ∞ < ε    + (h,i)∈I h 2 3 2 inf g∈H h,i g-z 0,h,i ∞ < ε 2 ∂g/∂x j -∂z 0,h,i /∂x j ∞ < K min 4 , j=1,...,d h g 2 H h,i -2 log P Z h,i ∞ ε 2 -2 d h j=1 log P ∂Z h,i /∂x j ∞ K min 4 , (8)
where

K min = min h=2,...,H min i=1,...,d h+1 min j=1,...,d h K h,i,j . (9)
This function is interpreted as evaluating the concentration of the processes Z h,i around the xed functions z 0,h,i for (h, i) ∈ I, taking the constraints into account.

3 Posterior contraction in density estimation and classication

Density estimation

Consider the observation of an i.i.d. sample X 1 , . . . , X n from a xed unknown probability density function

p 0 : [-1, 1] d → (0, +∞) (such that [-1,1] d p 0 (t)dt = 1). For z : [-1, 1] d → R, let p z : [-1, 1] d → (0, +∞) be given by p z (t) = e z(t) [-1,1] d e z(s) ds , for t ∈ [-1, 1] d . ( 10
)
Suppose that p 0 can be written as p z 0,H •••••z 0,1 where z 0,h = (z 0,h,1 , . . . , z 0,h,d h+1 ) for h = 1, . . . , H and such that z 0,h,i ∈ B c,h,i for (h, i) ∈ I. As above, assume that, for h = 2, . . . , H and for i = 1, . . . , d h+1 , z 0,h,i is in the closure of

H h,i in (C 1 ([-1, 1] d h , R), • ∞,1 ) and for i = 1, . . . , d 2 , z 0,1,i is in the closure of H 1,i in (C 0 ([-1, 1] d , R), • ∞ ).
We consider the prior p Z c,H •••••Z c,1 on p 0 , where, for any h = 1, . . . , H, Z c,h has been dened in Section 2.1. Then the posterior is given by

P(p Z c,H •••••Z c,1 ∈ • |X 1 , . . . , X n ).
The posterior is a random measure on (C 0 ([-1, 1] d , R) that depends on the observations X 1 , . . . , X n and we are interested in its convergence to p 0 as n → ∞.

Before stating the main result of this section, we recall the denition of the Hellinger distance h: for f, g :

[-1, 1] d → [0, ∞) with [-1,1] d f (t)dt = [-1,1] d g(t)dt = 1, h(f, g) = [-1,1] d f (t) -g(t) 2 dt. (11)
The next theorem is an extension of [START_REF] Van Der Vaart | Rates of contraction of posterior distributions based on Gaussian process priors[END_REF]Theorem 3.1] to constrained deep Gaussian priors.

Theorem 3.1. Let Φ c,z 0 be the function dened in [START_REF] Castillo | Lower bounds for posterior rates with Gaussian process priors[END_REF] and recall K min in [START_REF] Chen | Quadratic functionals and small ball probabilities for the m-fold integrated Brownian motion[END_REF]. Assume that, for h = 1, . . . , H -1 and for i = 1, . . . , d h+1 , z 0,h,i ∞ < 1 and for h = 2, . . . , H, for i = 1, . . . , d h+1 , and for j = 1, . . . , d h , ∂z 0,h,i /∂x j ∞ K min /2. Then, for a sequence

(ε c,n ) n∈N satisfying ε c,n > 0, ε c,n → 0, nε 2 c,n → ∞, and Φ c,z 0 (ε c,n ) nε 2 c,n , the posterior distribution satises P h(p Z c,H •••••Z c,1 , p 0 ) > M n ε c,n |X 1 , . . . , X n → n→∞ 0 (12)
in probability, for any sequence

(M n ) such that M n → ∞.
Remark that in Theorem 3.1, the conditions z 0,h,i ∞ < 1 and ∂z 0,h,i /∂x j ∞ K min /2 are not restrictive since, for a given p 0 , z 0,H can be taken with large enough values and derivatives values, to compensate for the bounded values of z 0,1 , . . . , z 0,H-1 , and then the K h,i,j in (9) can be taken large enough. Recall also that Appendix A shows that the value bound 1 can be replaced by arbitrary large bounds, up to linear changes of inputs and outputs of the Gaussian processes.

Classication

Consider the observation of an i.i.d. sample

(U 1 , V 1 ), . . . , (U n , V n ) distributed as (U, V ) where U is a random variable in [-1, 1] d and V is a binary random variable (V ∈ {0, 1}) such that P(V = 1|U ) = f 0 (U ), with a xed unknown function f 0 : [-1, 1] d → (0, 1). Consider a function Ψ : R → (0, 1) such that Ψ has a bounded derivative Ψ on R and such that Ψ /(Ψ(1 -Ψ)) is bounded on R. For instance, one may use the standard logistic function Ψ(x) = e x /(1 + e x ). For z : [-1, 1] d → R, let f z : [-1, 1] d → (0, 1) be given by f z (t) = Ψ(z(t)), for t ∈ [-1, 1] d . ( 13
)
Suppose that f 0 can be written as

f z 0,H •••••z 0,1
where z 0,h = (z 0,h,1 , . . . , z 0,h,d h+1 ) for h = 1, . . . , H and such that z 0,h,i ∈ B c,h,i for (h, i) ∈ I. Further, as above, assume that, for h = 2, . . . , H and for i

= 1, . . . , d h+1 , z 0,h,i is in the closure of H h,i in (C 1 ([-1, 1] d h , R), • ∞,1 ) and for i = 1, . . . , d 2 , z 0,1,i is in the closure of H 1,i in (C 0 ([-1, 1] d , R), • ∞ ). We consider the prior f Z c,H •••••Z c,1
, where, for h = 1, . . . , H, Z c,h has been dened in Section 2.1. Then the posterior is given by

P(f Z c,H •••••Z c,1 ∈ • |(U 1 , V 1 ), . . . , (U n , V n )).
Again, the posterior is a random measure on

(C 0 ([-1, 1] d , R) that depends on the observa- tions (U 1 , V 1 ), . . . , (U n , V n ) and we are interested in its convergence to f 0 as n → ∞. We let g 2,U be the L 2 norm of a function g : [-1, 1] d → R with respect to the law of U .
The next theorem is an extension of [45, Theorem 3.2] (see also [START_REF] Ghosal | Fundamentals of nonparametric Bayesian inference[END_REF]Theorem 11.22]) to constrained deep Gaussian priors.

Theorem 3.2. Consider the same setting and the same assumptions as in Theorem 3.1 for Φ c,z 0 , K min , and

(z 0,h,i ) (h,i)∈I . Then, for a sequence (ε c,n ) n∈N satisfying ε c,n > 0, ε c,n → 0, nε 2 c,n → ∞, and Φ c,z 0 (ε c,n ) nε 2 c
,n , the posterior distribution satises

P f Z c,H •••••Z c,1 -f 0 2,U > M n ε c,n |(U 1 , V 1 ), . . . , (U n , V n ) → n→∞ 0 (14)
in probability, for any sequence

(M n ) such that M n → ∞.

Examples of rates of contraction for specic function classes and specic priors

In this section, we apply Theorems 3.1 and 3.2 to classical examples of Gaussian processes for Z 1 , . . . , Z H and to classical examples of function classes for z 0,1 , . . . , z 0,H . These examples are by no means exhaustive, and Theorems 3.1 and 3.2 would enable to obtain contraction rates in other settings as well.

Integrated Brownian motion processes in dimension one

First consider independent Brownian motions as studied in [START_REF] Van Der Vaart | Rates of contraction of posterior distributions based on Gaussian process priors[END_REF] for standard (non deep) Gaussian processes. Here, we let

d 1 = • • • = d H+1 = 1
, that is we consider the composition of univariate processes. Thus we simply have Z h = Z h,1 for h = 1, . . . , H and the constants K h,i,j involved in the constraints (3) will simply be denoted by K h . For x ∈ R, let x be the largest integer strictly smaller than x. [START_REF] Azaïs | Level sets and extrema of random processes and elds[END_REF] to R that have derivatives up to order β and which derivative of order β is β -β -Hölder. The space F β ([-1, 1], R) is called the Hölder space of order β.

For β > 0, let F β ([-1, 1], R) be the set of functions from [-1,
For a continuous function g : [0, ∞) → R, we let I 0 g = g and, for ∈ N * , we dene by induction

I g : [0, ∞) → R by, for t ∈ [0, ∞), (I g)(t) = t 0 (I -1 g)(s)ds. We let N 1 ∈ N
and, for h = 2, . . . , H, N h ∈ N . We also let Z h be the Gaussian process dened by

Z h (t) = (I N h B h )(t + 1) + N h =0 X h, (t + 1) ! , for t ∈ (-1, ∞),
where B h is a standard Brownian motion on [0, ∞) and where X h,0 , . . . , X h,N h are inde- pendent standard Gaussian variables that are also independent of B h . Remark that Z h is then dened on (-1, ∞), is N h -times dierentiable, and has value and derivatives values at t = -1 given by X h,0 , . . . , X h,N h . In order to t with the formalism of Sections 2 and 3, we now extend Z 1 , . . . , Z h to the real line. This extension is done simply with a constant or a linear function. This choice is arbitrary and has no impact on the constrained deep Gaussian process priors constructed as in Sections 2 and 3. If N 1 = 0, we extend the denition of Z 1 (t) for t ∈ (-∞, -1) by taking the value at t = -1. If N h 1, we extend the denition of Z h (t) for t ∈ (-∞, -1) by a linear function based on the values of Z h and its derivative at t = -1. After this, Z h is a continuous Gaussian process on R (continuously dierentiable if N h 1).

As in Sections 2 and 3, we let Z 1 , . . . , Z H be independent. The next theorem then provides the rates of contraction given by Theorems 3.1 and 3.2.

Theorem 4.1. First consider the setting of Theorem 3.1. Assume that p 0 is strictly positive and belongs to 3) can be chosen large enough so that the conclusion of Theorem 3.1 holds with

F β ([-1, 1], R) for a xed β > 0. Assume that β N 1 +1/2 and that β N h for h = 2, . . . , H. Then the xed constants K 1 , . . . , K H in (
ε c,n = C sup n -β/(2N 1 +2)
for some constant C sup that does not depend on n. Second, consider the setting of Theorem 3.2, with f 0 = f g 0 where g 0 belongs to F β ([-1, 1], R), with the same conditions on β, N 1 , . . . , N H . Then the rst conclusion again holds, with the same expression for ε c,n .

From Theorem 4.1 with the condition β N 1 + 1/2, the rate of contraction is fastest when N 1 + 1/2 = β (which is possible when β = k + 1/2, k ∈ N, and means that the smoothness of Z 1 matches that of the xed unknown function), in which case we recover the classical minimax rate n -β/(2β+1) (see [START_REF] Van Der Vaart | Rates of contraction of posterior distributions based on Gaussian process priors[END_REF]Theorem 4.1 and below]). For a larger N 1 , the rate deteriorates (oversmooth prior). Note that the processes Z 2 , . . . , Z H are chosen smoother than the xed unknown function from the condition N h β for h = 2, . . . , H.

Riemann-Liouville processes in dimension one

While the smoothness index of the Brownian motion is necessarily of the form integer + 1/2, any smoothness index can be reached with the Riemann-Liouville process as explained in [START_REF] Van Der Vaart | Rates of contraction of posterior distributions based on Gaussian process priors[END_REF]. As in Section 4.1, in the sequel, we let

d 1 = • • • = d H+1 = 1, we simply have Z h Z h,1
, and we simply denote K h for K h,1,1 in (3), h = 1, . . . , H. For x ∈ R, let x be the largest integer smaller or equal to x. For h = 1, . . . , H, we set α 1 > 0 and for h = 2, . . . , H, α h > 1. We let B h be a standard Brownian motion on [0, ∞).

We dene R h as the Gaussian process on [0, ∞) dened by R h (t) = t 0 (t -s) α h -1/2 dB h (s) and then the Gaussian process

Z h on [-1, ∞) by Z h (t) = R h (t + 1) + α h +1 =0 X h, (t + 1) ! , for t ∈ [-1, ∞)
and where X h,1 , . . . , X h,α h +1 are independent standard Gaussian variables.

We then extend Z h on (-∞, 1) as in Section 4.1. Again as in Section 4.1, we let Z 1 , . . . , Z H be independent. The next theorem then provides the rates of contraction given by Theorems 3.1 and 3.2.

Theorem 4.2. First consider the setting of Theorem 3.1. Assume that p 0 is strictly positive and belongs to

F α 1 ([-1, 1], R).
Assume that for h = 2, . . . , H, α h α 1 . Then the xed constants K 1 , . . . , K H in (3) can be chosen large enough so that the conclusion of Theorem 3.1 holds with

ε c,n = C sup n -α 1 /(2α 1 +1)
for some constant C sup that does not depend on n. Second, consider the setting of Theorem 3.2, with f 0 = f g 0 where g 0 belongs to

F α 1 ([-1, 1], R)
, with the same conditions on α 1 , . . . , α H . Then the rst conclusion again holds, with the same expression for ε c,n .

Note that the smoothness of Z 1 exactly matches that of the xed unknown function and the processes Z 2 , . . . , Z H have a smoothness that is larger than or equal to that of the xed unknown function. This enables to recover the standard minimax rate n -α 1 /(2α 1 +1) (see [START_REF] Van Der Vaart | Rates of contraction of posterior distributions based on Gaussian process priors[END_REF]Theorem 4.3 and below]).

Matérn processes in general dimension

Here we consider Gaussian processes with Matérn covariance functions, which allows for both arbitrary input dimension and arbitrary smoothness. These processes were studied in [44, Section 3.1] for standard (non deep) Gaussian processes. Let us rst extend the denition of F β ([-1, 1], R) to larger dimensions. For β > 0 and ∈ N * , we let F β ([-1, 1] , R) be the set of functions from [-1, 1] to R for which all the partial derivatives of order (a 1 , . . . , a ) exist for all a 1 , . . . , a ∈ N with a 1 + • • • + a β and which partial derivatives of order β are β -β -Hölder.

For (h, i) ∈ I, we let Z h,i have Matérn covariance function, that is, for u, v ∈ R d h , Cov (Z h,i (u), Z h,i (v)) = R d e iλ (u-v) m h,i (λ)dλ, where m h,i (λ) = 1 (1 + λ 2 ) α h,i +d/2 (15)
is called the spectral density and α h,i > 0 is called the smoothness parameter. A Gaussian process on [-1, 1] ( ∈ N ) with Matérn covariance function with α h,i = α in (15) has sample paths in

F β ([-1, 1] , R) for any β < α.
Then, as in [START_REF] Van Der | Information rates of nonparametric Gaussian process methods[END_REF], we dene the Sobolev space H β ([-1, 1] , R) as the set of functions f from [-1, 1] to R that are restrictions of functions f from R → R with Fourier transform f (λ) = (2π) -e iλ t f (t)dt such that, dening

f 2 β := R d (1 + λ 2 ) β f 2 (λ) dλ < ∞.
The next theorem provides the rates of contraction given by Theorems 3.1 and 3.2.

Theorem 4.3. First consider the setting of Theorem 3.1. Assume that p 0 is strictly positive and belongs to

F β ([-1, 1] d , R)∩H β ([-1, 1] d , R) for some β > 0. Assume that, for (h, i) ∈ I, β α h,i .
Then the xed constants K h,i,j for h = 2, . . . , H, for i = 1, . . . , d h+1 , and for j = 1, . . . , d h in (3) can be chosen large enough such that the conclusion of Theorem 3.1 holds with

ε c,n = C sup n -β/(2α 1,min +d) ,
where α 1,min = min(α 1,1 , . . . , α 1,d 2 ) and C sup is some constant that does not depend on n. Second, consider the setting of Theorem 3.2, with f 0 = f g 0 where g 0 belongs to

F β ([-1, 1] d , R)∩ H β ([-1, 1] d , R)
, with the same conditions on β, K h,i,j , α h,i for h = 1, . . . , H, for i = 1, . . . , d h+1 , and for j = 1, . . . , d h . Then the rst conclusion again holds, with the same expression for ε c,n .

Above, the Gaussian process priors are taken smoother than the xed unknown function. With the appropriate smoothness α 1,min = β for the rst layer, we recover the classic optimal rate n -β/(2β+d) as pointed out in [44, Theorem 5].

Conclusion

In this paper, we have provided rates of contraction for the posterior distribution of a deep Gaussian process prior, with constraints on the values and the derivatives. These results are the rst to address density estimation and classication, to the best of our knowledge. Recently, [START_REF] Finocchio | Posterior contraction for deep Gaussian process priors[END_REF] addressed contraction rates for deep Gaussian processes in regression. These general rates (Theorems 3.1 and 3.2) measure in terms of a new concentration function, that takes the constraints into account. The proofs of these general rates are based on constructing a single global Gaussian process prior in order to exploit existing results for standard Gaussian processes [START_REF] Van Der Vaart | Rates of contraction of posterior distributions based on Gaussian process priors[END_REF]. Some of the techniques for taking the constraints into account could be considered of independent interest (in particular the proofs in Section C.1). We show in three examples how the general rates enable to recover existing minimax rates for standard function classes and Gaussian processes. These examples are not exhaustive. In future work, it would be benecial to exploit Theorems 3.1 and 3.2 in other settings, with more involved function classes based on composition structures, in the aim of obtaining contraction rates that are out of reach for a single Gaussian process prior, thus further illustrating the exibility benet brought by deep Gaussian processes. Finally, adaptivity to function smoothness is an important topic for future work as well.

A More exibility on the constraints by linear transformations of inputs and outputs

Here, we consider that Z 1 , . . . , Z H satisfy constraints of the form (3) but the constraints

Z h ∞,[-1,1] d h
1 are replaced by more general constraints: for h = 1, . . . , H -1 and for i = 1, . . . , d h+1 , we assume

Z h,i ∞, =1,...,d h [-L h, ,L h, ] L h+1,i (16)
where the constants L h+1,i > 0 (for h = 1, . . . , H -1 and for i = 1, . . . , d h+1 ) are arbitrary and

L 1,1 = • • • = L 1,d 1 = 1.
We show that we can construct processes Y 1 , . . . , Y H that provide the same composition

Y H • • • • • Y 1 = Z H • • • • • Z 1 ,
that have constraints of the form ( 2) and ( 3), and that are obtained from linear transformations of inputs and outputs. Indeed, dene the Gaussian processes Y 1 , . . . , Y H as follows.

For h = 1, . . . , H -1, for t 1 , . . . , t d h ∈ R, and for i = 1, . . . , d h+1 , let

Y h,i (t 1 , . . . , t d h ) = 1 L h+1,i Z h,i (L h,1 t 1 , . . . , L h,d h t d h ) . (17)
Recall that d H+1 = 1 and let

Y H (t 1 , . . . t d H ) = Z H (L H,1 t 1 , . . . , L H,d H t d H ) , for t 1 , . . . , t d H ∈ R. (18)
The next lemma then shows the composition equality.

Lemma A.1. For any t 1 , . . . , t d ∈ R, we have

Y H • • • • • Y 1 (t 1 , . . . , t d ) = Z H • • • • • Z 1 (
Y h,i • Y h-1 • • • • • Y 1 (t 1 , . . . , t d ) = 1 L h+1,i Z h,i • Z h-1 • • • • • Z 1 (t 1 , . . . , t d ). (19)
For h = 1, this is true from [START_REF] Ghosal | Posterior consistency of Gaussian process prior for nonparametric binary regression[END_REF] with

L 1,1 = • • • = L 1,d 1 = 1 and d 1 = d.
Assume that this is true for some h ∈ {1, . . . H -2}. Thus by [START_REF] Golchi | Monotone emulation of computer experiments[END_REF] and then by [START_REF] Ghosal | Posterior consistency of Gaussian process prior for nonparametric binary regression[END_REF], for i = 1, . . . , d h+1 ,

Y h+1,i • • • • • Y 1 (t 1 , . . . , t d ) = Y h+1,i Y h,1 • Y h-1 • • • • • Y 1 (t 1 , . . . , t d ), . . . , Y h,d h+1 • Y h-1 • • • • • Y 1 (t 1 , . . . , t d ) = Y h+1,i 1 L h+1,1 Z h,1 • Z h-1 • • • • • Z 1 (t 1 , . . . , t d ), . . . , 1 L h+1,d h+1 Z h,d h+1 • Z h-1 • • • • • Z 1 (t 1 , . . . , t d ) = 1 L h+2 , i Z h+1,i Z h,1 • Z h-1 • • • • • Z 1 (t 1 , . . . , t d ), . . . , Z h,d h+1 • Z h-1 • • • • • Z 1 (t 1 , . . . , t d ) = 1 L h+2 , i Z h+1,i • Z h • • • • • Z 1 (t 1 , . . . , t d ).
This concludes the proof of ( 19) by induction. Finally, from ( 19) and ( 18), we obtain, for t 1 , . . . , t d ∈ R,

Y H • • • • • Y 1 (t 1 , . . . , t d ) = Y H (Y H-1,1 • Y H-2 • • • • • Y 1 (t 1 , . . . , t d ), . . . , Y H-1,d H • Y H-2 • • • • • Y 1 (t 1 , . . . , t d )) = Y H 1 L H,1 Z H-1,1 • Z H-2 • • • • • Z 1 (t 1 , . . . , t d ), . . . , 1 L H,d H Z H-1,d H • Z H-2 • • • • • Z 1 (t 1 , . . . , t d ) = Z H (Z H-1,1 • Z H-2 • • • • • Z 1 (t 1 , . . . , t d ), . . . , Z H-1,d H • Z H-2 • • • • • Z 1 (t 1 , . . . , t d )) = Z H • Z H-1 • • • • • Z 1 (t 1 , . . . , t d ).
The next lemma shows that Y 1 , . . . , Y H satisfy constraints of the form (2) when Z 1 , . . . , Z H satisfy the more general ones given in [START_REF] Garriga-Alonso | Deep convolutional networks as shallow Gaussian processes[END_REF].

Lemma A.2. The two following assertions are equivalent. 1. For h = 1, . . . , H -1 and for i = 1, . . . , d h+1 ,

Z h,i ∞, =1,...,d h [-L h, ,L h, ] L h+1,i .

For

h = 1, . . . , H -1 and for t ∈ [-1, 1] d h , Y h (t) ∈ [-1, 1] d h+1 .
Proof of Lemma A.2. The second assertion can be written as: for h = 1, . . . , H -1 and

i = 1, . . . , d h+1 , Y h,i ∞,[-1,1] d h 1.
Hence, to prove the lemma, it is sucient to show that, for h = 1, . . . , H -

1 and i = 1, . . . , d h+1 , Z h,i ∞, =1,...,d h [-L h, ,L h, ] L h+1,i ⇔ Y h,i ∞,[-1,1] d h 1.
The latter equivalence follows from [START_REF] Ghosal | Posterior consistency of Gaussian process prior for nonparametric binary regression[END_REF].

Finally, the next lemma shows that Y 1 , . . . , Y H and Z 1 , . . . , Z H equivalently satisfy constraints of the form (3).

Lemma A.3. For h = 2, . . . , H, for i = 1, . . . , d h+1 , and for j = 1, . . . , d h ,

∂Z h,i ∂x j ∞, =1,...,d h [-L h, ,L h, ] K h,i,j ⇐⇒ ∂Y h,i ∂x j ∞,[-1,1] d h L h,j L +1,i K h,i,j .
Proof of Lemma A.3. Let L H+1,1 = 1 by convention. From ( 17) and ( 18), we have, for h = 2, . . . , H, for i = 1, . . . , d h+1 , j = 1, . . . , d h , and for t 1 , . . . , t

d h ∈ R, ∂ ∂x j Y h,i (t 1 , . . . , t d h ) = ∂ ∂t j 1 L h+1,i Z h,i (L h,1 t 1 , . . . , L h,d h t d h ) = L h,j L h+1,i ∂ ∂x j Z h,i (L h,1 t 1 , . . . , L h,d h t d h ) .

Hence we have

∂Z h,i ∂x j ∞, =1,...,d h [-L h, ,L h, ] K h,i,j ⇐⇒ ∂Y h,i ∂x j ∞,[-1,1] d h L h,j L h+1,i K h,i,j .
which concludes the proof.

B Preliminary notation and intermediate results

The proofs will exploit the results established in [START_REF] Van Der Vaart | Rates of contraction of posterior distributions based on Gaussian process priors[END_REF] for a single Gaussian process. Thus we will consider the setting of a single Gaussian process dened on a compact space and valued in R. The following section is dedicated to the construction of this process.

B.1 Introduction of a single global Gaussian prior

Let d max = max(d 1 , . . . , d H ) and let X be the compact subspace [-1, 1] dmax × I with the distance d dened by

d((t, h, i), (t , h , i )) = (t, σ(h, i)) -(t , σ(h , i )) , for (t, h, i) and (t , h , i ) in X
where the norm is the Euclidean norm in dimension d max + 1 and σ is any xed bijection from I to {1, . . . , |I|} with |I| the cardinality of I. We introduce the centered Gaussian process W , dened by

W (t, h, i) = Z h,i (T h (t)), for (t, h, i) ∈ X , (20)
where T h (t) is the vector of length d h obtained from the d h rst coordinates of t. Then W has continuous trajectories from X to R.

Let B be the Banach space of the continuous functions from X to R endowed with the uniform norm given by

w ∞,X := ∨ (h,i)∈I w(•, h, i) ∞,[-1,
P h,i (w)(t) = w((t, 0), h, i), for t ∈ [-1, 1] d h ,
where the vector (t, 0) has dimension d max . Consequently, for (h, i) ∈ I and for t ∈

[-1, 1] d h , P h,i (W )(t) = Z h,i (t).
Now we dene the subset B c of B composed by the functions w ∈ B such that the three following conditions (corresponding to (2) and ( 3)) hold.

• For h = 1, . . . , H -1, i = 1, . . . , d h+1 , P h,i (w) ∞,[-1,1] d h 1.

• For h = 2, . . . , H, i = 1, . . . , d h+1 , P h,i (w) is continuously dierentiable on [-1, 1] d h .

• For h = 2, . . . , H, i = 1, . . . , d h+1 , j = 1, . . . , d h , ∂P h,i (w)/∂x j ∞,[-1,1] d h K h,i,j .

Then we consider a process W c valued in B with distribution dened by

P(W c ∈ B) = P(W ∈ B| W ∈ B c ), (21)
C ψ (t) = C ψ,H • • • • • C ψ,1 (t). (22)
Remark that, for t ∈

[-1, 1] d , when W ∈ B c , C W (t) = Z H • • • • • Z 1 (t). (23)
Hence, considering the conditioned version W c of W (as dened in ( 21)), C Wc has the distribution of the deep Gaussian process

Z H • • • • • Z 1
where the univariate Gaussian processes Z h,i are conditioned to (2) and/or (3).

Finally, we let k be the covariance function of W on X × X and H be the RKHS of W (that is dened as in [46, Section 2.1]) with RKHS-norm • H .

B.2 The global concentration function is upper bounded

For w 0 in the closure of H in B (with respect to • ∞ ), we dene, for ε > 0,

φ c,w 0 (ε) = inf h∈H h-w 0 ∞ <ε h 2 H -log P ( W c ∞ < ε) -log P( W -w 0 ∞ < 2ε, W ∈ B c ). (24)
The function φ c,w 0 is interpreted as the concentration function of W around w 0 and is an extension of the function φ x 0 already dened in [START_REF] Bevilacqua | Estimating space and space-time covariance functions for large data sets: a weighted composite likelihood approach[END_REF] and in [45, Equation (1.

2)], that takes the bound constraints in W c into account. The next lemma shows that the presence of the constraints increases the concentration function.

Proof of Lemma B.1. For any ε > 0, let us consider the dierence between the two concentration functions

φ c,w 0 (ε) -φ w 0 (ε) = -log P ( W c ∞ < ε) -log P( W -w 0 ∞ < 2ε, W ∈ B c ) + log P ( W ∞ < ε) . Since P ( W c ∞ < ε) = P ( W ∞ < ε, W ∈ B c ) /P(W ∈ B c ), P ( W ∞ < ε) P ( W ∞ < ε, W ∈ B c ) and P(W ∈ B c ) P( W -w 0 ∞ < 2ε, W ∈ B c ),
the latter dierence is non-negative.

The next theorem shows that the global φ c,w 0 dened in ( 24) is upper bounded by Φ c,w 0 dened in [START_REF] Castillo | Lower bounds for posterior rates with Gaussian process priors[END_REF].

Theorem B.2. Let w 0 in B. For (h, i) ∈ I, assume that w 0 (•, h, i) = P h,i (w 0 )(T h (•)). For h = 2, . . . , H and for i = 1, . . . , d h+1 , assume that

P h,i (w 0 ) is in the closure of H h,i in (C 1 ([-1, 1] d h , R), • ∞,1 ). Recall that • ∞,1 is dened in (6). For i = 1, . . . , d 2 , assume that P 1,i (w 0 ) is in the closure of H 1,i in (C 0 ([-1, 1] d 1 , R), • ∞ ). Then w 0 is in the closure of H in (B, • ∞ ).
Now consider ε ∈ (0, 1] and assume that, for h = 1, . . . , H -1, for i = 1, . . . , d h+1 ,

P h,i (w 0 ) ∞ + 2ε 1 (25)
and, for h = 2, . . . , H, for i = 1, . . . , d h+1 , and for j = 1, . . . , d h ,

∂P h,i (w 0 ) ∂x j ∞ K min 2 (26)
where K min is dened in [START_REF] Chen | Quadratic functionals and small ball probabilities for the m-fold integrated Brownian motion[END_REF]. Then we have

φ c,w 0 (ε) Φ c,w 0 (ε)
where Φ c,w 0 is as in [START_REF] Castillo | Lower bounds for posterior rates with Gaussian process priors[END_REF] with z 0,h,i = P h,i (w 0 ) for (h, i) ∈ I.

B.3 Contraction rates with the global concentration function

The next theorem is an adaptation of [45, Theorem 2.1]. For a metric space (E, d), any subset A ⊂ E, and any ε > 0, we let N (ε, A, d) be the minimum number of balls of radius ε needed to cover A.

Theorem B.3. Consider any sequence (ε c,n ) n∈N , satisfying nε 2 c,n → ∞ and φ c,w 0 (ε c,n ) nε 2
c,n for φ c,w 0 dened in [START_REF] Lataªa | Royen's proof of the Gaussian correlation inequality[END_REF], and any C > 1 with e -Cnε 2 c,n < 1/2. Then there exists an integer n C 0 and a sequence of measurable sets

(B c,n ) n∈N with B c,n ⊂ B c such that, for n n C , log N (6 √ Cε c,n , B c,n , • ∞ ) n(12 √ Cε c,n ) 2 , ( 27
)
P(W c / ∈ B c,n ) e -Cnε 2 c,n , (28)
P( W c -w 0 ∞ < 2ε c,n ) e -nε 2 c,n . (29)
Theorem B.3 enables to obtain posterior contraction rates at speed ε c,n satisfying φ c,w 0 (ε c,n ) nε 2 c,n similarly as done in [START_REF] Van Der Vaart | Rates of contraction of posterior distributions based on Gaussian process priors[END_REF]. This directly allows to prove Theorems 3.1 and 3.2, that is contraction rates at speed ε c,n satisfying Φ c,z 0 (ε c,n ) nε 2 c,n from the bound φ c,w 0 Φ c,z 0 of Theorem B.2.

C Proofs

In the rest of the appendix, we write C sup for a nite constant which value is allowed to change between occurrences.

C.1 Proof of Theorem B.2

Before proving Theorem B.2, let us establish several useful lemmas. We do not exclude that results similar to Lemma C.1 could be known by the experts, but we have not found any reference. We prove Lemma C.1 after its statement for self-suciency.

Lemma C.1. The RKHS H of the covariance function k is equal to the set of functions g : X → R such that, for (h, i) ∈ I,

g(•, h, i) = P h,i (g)(T h (•)) and P h,i (g) ∈ H h,i . (30)
Furthermore, for g ∈ H, one has

g 2 H = (h,i)∈I P h,i (g) 2 H h,i . (31)
Proof of Lemma C.1. For h = 1, . . . , H and for t ∈ [-1, 1] dmax , recall that T h (t) ∈ [-1, 1] d h is composed of the d h rst coordinates of t. For (h, i) ∈ I, let H 0,h,i be the set of functions of the form

t ∈ [-1, 1] d h → N h,i a=1 α h,i,a k h,i (t h,i,a , t), (32)
for

N h,i ∈ N * , for t h,i,1 , . . . , t h,i,N h,i ∈ [-1, 1] d h , and for α h,i,1 , . . . , α h,i,N h,i ∈ R.
Also, let H 0 be the set of functions of the form

(t, h, i) ∈ X → N a=1 γ a k((t a , h a , i a ), (t, h, i)), (33)
for N ∈ N * , for (t 1 , h 1 , i 1 ), . . . , (t N , h N , i N ) ∈ X , and for γ 1 , . . . , γ N ∈ R. Then, from Moore-Aronszajn theorem (see [6, Theorem 3]), we have H 0,h,i ⊂ H h,i for (h, i) ∈ I and H 0 ⊂ H.

We have, for any function g 0 ∈ H 0 of the form (33), using the independence of the Z h,i for (h, i) ∈ I, that P h,i (g 0 ) is the function

t ∈ [-1, 1] d h → N a=1 (ha,ia)=(h,i) γ a k h,i (T h,i (t a ), t)
and thus belongs to H 0,h,i . Hence we have, using the independence of the Z h,i for (h, i) ∈ I, again,

g 0 2 H = N a,b=1 γ a γ b k((t a , h a , i a ), (t b , h b , i b ))) = (h,i)∈I N a,b=1 (ha,ia)=(h b ,i b )=(h,i) γ a γ b k h,i (T ha (t a ), T h b (t b )) = (h,i)∈I P h,i (g 0 ) 2 H h,i . (34)
Now let g ∈ H. Again from Moore-Aronszajn theorem, there exists a sequence (g N ) N ∈N * of elements of H 0 , that is a Cauchy sequence with • H converging pointwise to g. This implies that the rst property in [START_REF] Maatouk | Gaussian process emulators for computer experiments with inequality constraints[END_REF] holds for g since it holds for g N of the form [START_REF] Matthews | Gaussian process behaviour in wide deep neural networks[END_REF]. For (h, i) ∈ I, let g N,h,i = P h,i (g N ). From [START_REF] Paulo | Calibration of computer models with multivariate output[END_REF] and the linearity of P h,i , (g

N →∞ P h,i (g N )(t) = lim N →∞ g N,h,i (t) = g h,i (t).
Hence the second property in [START_REF] Maatouk | Gaussian process emulators for computer experiments with inequality constraints[END_REF] holds and g can be written as in [START_REF] Maatouk | Gaussian process emulators for computer experiments with inequality constraints[END_REF]. Conversely, let a function g : X → R that satises [START_REF] Maatouk | Gaussian process emulators for computer experiments with inequality constraints[END_REF]. For (h, i) ∈ I, from Moore-Aronszajn theorem, there exist (g N,h,i ) N ∈N * as in [START_REF] Matheron | La théorie des variables régionalisées et ses applications[END_REF] that is a Cauchy sequence of elements of H 0,h,i with the norm • H h,i and such that, as

N → ∞, g N,h,i → P h,i (g) pointwise on [-1, 1] d h . Then let us show that the function g N : X → R dened by, for (t, h, i) ∈ X , g N (t, h, i) = g N,h,i (T h (t)) , (35)
belongs to H 0 . We have

g N (t, h, i) = N h,i a=1 α h,i,a k h,i (t h,i,a , T h (t)) = N h,i a=1 α h,i,a k(((t h,i,a , 0), h, i), (t, h, i)) = (h ,i )∈I N h ,i a=1 α h ,i ,a k(((t h ,i ,a , 0), h , i ), (t, h, i)),
using the independence of the P h,i (W ) for (h, i) ∈ I. Hence, g N belongs to H 0,N and thus to H 0 . Observe that we have, for (h, i) ∈ I, P h,i (g N ) = g N,h,i and thus from [START_REF] Paulo | Calibration of computer models with multivariate output[END_REF], as N → ∞, g N is a Cauchy sequence of elements in H 0 with the norm • H . Hence g N also converges pointwise on X and the pointwise limit function lim N →∞ g N belongs to H from Moore-Aronszajn theorem. We also have, for xed

(t, h, i) ∈ X , lim N →∞ g N (t, h, i) = lim N →∞ g N,h,i (T h (t)) = P h,i (g) (T h (t)) = g(t, h, i),
using at the end the rst equality in [START_REF] Maatouk | Gaussian process emulators for computer experiments with inequality constraints[END_REF]. Hence, any g : X → R satisfying (30) does belong to H. This shows that the RKHS H is as indicated in the lemma. Finally, let us prove [START_REF] Marmin | Warped Gaussian processes and derivative-based sequential designs for functions with heterogeneous variations[END_REF]. Let g : X → R in H and thus satisfying [START_REF] Maatouk | Gaussian process emulators for computer experiments with inequality constraints[END_REF] . Then, for (h, i) ∈ I, there exists a sequence (g N,h,i ) N ∈N * of functions in H 0,h,i that converges to P h,i (g) with the norm • H h,i . Dene g N as in [START_REF] Porcu | Spatio-temporal covariance and crosscovariance functions of the great circle distance on a sphere[END_REF]. We have shown that (g N ) N ∈N * is a Cauchy sequence of elements in H 0 with the norm • H that converges pointwise to g. Hence g N also converges with the norm • H to g and we have, using [START_REF] Paulo | Calibration of computer models with multivariate output[END_REF],

g 2 H = lim N →∞ g N 2 H = lim N →∞ (h,i)∈I P h,i (g N ) 2 H h,i .
Because of ( 35), we have P h,i (g N ) = g N,h,i and thus

g 2 H = lim N →∞ (h,i)∈I g N,h,i 2
H h,i = (h,i)∈I P h,i (g) 2 H h,i .
This concludes the proof.

Lemma C.2. Consider a centered Gaussian process X, valued in a Banach space (E, • ) composed of functions from a set T to R. Let H X be the RKHS of X, with RKHS-norm

• H X . Let f ∈ E.
Then we have, for any ε > 0,

-log P ( X -f < 2ε) 1 2 inf g∈H X g-f <ε g 2 H X -log P ( X ε) ,
inequality ( [START_REF] Royen | A simple proof of the Gaussian correlation conjecture extended to multivariate gamma distributions[END_REF], see also [START_REF] Lataªa | Royen's proof of the Gaussian correlation inequality[END_REF]), we obtain

P
|∂X(t j )/∂x i | b i → P sup t∈T |X(t)| a × i=1 P sup t∈T |∂X(t)/∂x i | b i ,
as n → ∞, again from the dominated convergence theorem and Lemma C.4. This concludes the proof.

Now we are able to prove Theorem B.2.

Proof of Theorem B.2. From the assumptions of the theorem and Lemma C.1, we obtain that w 0 is indeed in the closure of H in B (with respect to • ∞ ). Then, using Lemma C.1, ε ∈ (0, 1] and (25), we obtain

φ c,w 0 (ε) = inf g∈H g-w 0 ∞ <ε g 2 H -log P ( W c ∞ < ε) -log P( W -w 0 ∞ < 2ε, W ∈ B c ) inf {g h,i ;(h,i)∈I} for all (h,i),g h,i ∈H h,i , g h,i -P h,i (w 0 ) ∞ <ε   (h,i)∈I g h,i 2 H h,i   -log P Z h,i ∞ < ε for (h, i) ∈ I and ∂Z h,i /∂x j ∞ K h,i,j for h = 2, . . . , H, for i = 1, . . . , d h+1 , for j = 1, . . . , d h -log P Z h,i -P h,i (w 0 ) ∞ < 2ε for (h, i) ∈ I and ∂Z h,i /∂x j ∞ K h,i,j for h = 2, . . . , H, for i = 1, . . . , d h+1 , for j = 1, . . . , d h .
Using now the independence between the Z h,i 's for (h, i) ∈ I, we obtain φ c,w 0 (ε)

d 2 i=1 A i + (h,i)∈I h 2 B h,i , (36)
where we have dened

A i = inf g∈H 1,i g-P 1,i (w 0 ) ∞ <ε g 2 H 1,i -log P Z 1,i ∞ < ε -log P Z 1,i -P 1,i (w 0 ) ∞ < 2ε and B h,i = inf g∈H h,i g-P h,i (w 0 ) ∞ <ε g 2 H h,i -log P Z h,i ∞ < ε, ∂Z h,i /∂x j ∞ K h,i,j for j = 1, . . . , d h -log P Z h,i -P h,i (w 0 ) ∞ < 2ε, ∂Z h,i /∂x j ∞ K h,i,j for j = 1, . . . , d h . First let us bound A i . Since P 1,i (w 0 ) is in the closure of H 1,i in (C 0 ([-1, 1] d 1 ), R), • ∞ ), using Lemmas C.2, C.3,

and C.4, we have

A i inf g∈H 1,i g-P 1,i (w 0 ) ∞ <ε g 2 H 1,i -log P Z 1,i ∞ < ε + 1 2 inf g∈H 1,i g-P 1,i (w 0 ) ∞ <ε g 2 H 1,i -log P Z 1,i ∞ < ε = 3 2 inf g∈H 1,i g-P 1,i (w 0 ) ∞ <ε g 2 H 1,i -2 log P Z 1,i ∞ < ε . (37)
Second let us bound B h,i . We have, from [START_REF] Lenk | The logistic normal distribution for Bayesian, nonparametric, predictive densities[END_REF],

B h,i inf g∈H h,i g-P h,i (w 0 ) ∞ <ε g 2 H h,i -log P Z h,i ∞ < ε, ∂Z h,i /∂x j ∞ K min , j = 1, . . . , d h -log P Z h,i -P h,i (w 0 ) ∞ < 2ε, ∂Z h,i /∂x j -∂P h,i (w 0 )/∂x j ∞ K min /2, j = 1, . . . , d h inf g∈H h,i g-P h,i (w 0 ) ∞ <ε g 2 H h,i -log P Z h,i ∞ < ε, ∂Z h,i /∂x j ∞ K min , j = 1, . . . , d h -log P Z h,i -P h,i (w 0 ) ∞ ε, 2ε ∂Z h,i /∂x j -∂P h,i (w 0 )/∂x j ∞ /K min ε, j = 1, . . . , d h . (38)
Let us dene the Banach space B h as the set of continuous functions from

[-1, 1] d h × {0, 1, . . . , d h }, equipped with the norm • ∞,ε dened by, for z ∈ B h , z ∞,ε = z(•, 0) ∞ ∨ 2ε K min z(•, 1) ∞ ∨ • • • ∨ 2ε K min z(•, d h ) ∞ .
We consider the map

M from (C 1 ([-1, 1] d h , R), • ∞,1 ) to (B h , • ∞,ε ) dened by, for f ∈ C 1 ([-1, 1] d h , R), (M f )(•, 0) = f , (M f )(•, 1) = ∂f /∂x 1 , ... ,(M f )(•, d h ) = ∂f /∂x d h .
Let us dene the Gaussian process Z on B h by Z = M (Z h,i )and similarly the function w 0 in B h by w 0 = M (P h,i (w 0 )). We write H Z the RKHS of the Gaussian process Z and • H Z its RKHS-norm.

Lemma C.6. Recall that h = 2, . . . , H. We have

H Z = {M (g); g ∈ H h,i } and, for g ∈ H h,i , g H h,i = M (g) H Z .
The proof of Lemma C.6 is postponed after the proof of Theorem B.2. Then, using the previous notation and Lemmas C.2 and C.6, we obtain

-log P Z h,i -P h,i (w 0 ) ∞ ε, 2ε K min ∂Z h,i /∂x j -∂P h,i (w 0 )/∂x j ∞ ε for j = 1, . . . , d h = -log P Z -w 0 ∞,ε ε 1 2 inf h∈H Z h-w 0 ∞,ε < ε 2 h 2 H Z -log P Z ∞,ε ε 2 = 1 2 inf g∈H h,i g-P h,i (w 0 ) ∞ < ε 2 ∂g/∂x j -∂P h,i (w 0 )/∂x j ∞ < K min 4 , j=1,...,d h g 2 H h,i -log P Z h,i ∞ ε 2 , ∂Z h,i /∂x j ∞ K min 4 for j = 1, . . . , d h .
Now using the above display and ( 38), together with Lemmas C.4, and C.5, we obtain

B h,i 3 2 inf g∈H h,i g-P h,i (w 0 ) ∞ < ε 2 ∂g/∂x j -∂P h,i (w 0 )/∂x j ∞ < K min 4 , j=1,...,d h g 2 H h,i -2 log P Z h,i ∞ ε 2 -2 d h j=1 log P ∂Z h,i /∂x j ∞ K min 4 .
Then the proof is concluded using [START_REF] Radaideh | Surrogate modeling of advanced computer simulations using deep Gaussian processes[END_REF] and [START_REF] Rasmussen | Gaussian Processes for Machine Learning[END_REF].

It remains to prove Lemma C.6.

Proof of Lemma C.6. By assumption, the Gaussian process Z h,i , with input space [-1, 1] d h , has continuously dierentiable realizations. Hence, Z h,i can be viewed as a random element of the Banach space

C 0 ([-1, 1] d h , R) of continuous functions on [-1, 1] d h , equipped with the norm • ∞ . The space (H h,i , • H h,i ) is dened as in [46, Section 2.1] (from the covariance function of Z h,i ). From [46, Theorem 2.1], H h,i can be equivalently dened as in [46, Section 2.2] from the law of Z h,i in C 0 ([-1, 1] d h , R) and from the norm • ∞ . The identity map from (C 1 ([-1, 1] d h , R), • ∞,1 ) to (C 0 ([-1, 1] d h , R), • ∞ ) is a one-to-
([-1, 1] d h , R), • ∞,1
). Now we consider the Banach space B h equipped with the norm • ∞,ε . We consider the map M introduced above. Then M is a one-to-one, continuous and linear application from 7)) and since φ c,w 0 is non-increasing, we have

(C 1 ([-1, 1] d h , R), • ∞,1 ) to (B h , • ∞,ε
φ w 0 (ε n ) φ c,w 0 (ε n ) φ c,w 0 (ε c,n ) nε 2 c,n nε 2 n .
Furthermore, since ε 2 n ε 2 c,n , we have e -Cnε 2 n < 1/2. Hence we can apply [45, Theorem 2.1] to W with ε n and C. Thus, there exists a sequence of measurable sets

(B n ) n∈N , B n ⊂ B for all n ∈ N, such that log N (3ε n , B n , • ∞ ) 6Cnε 2 n , (39)
P(W / ∈ B n ) e -Cnε 2 n , (40)
P( W -w 0 ∞ < 2ε n ) e -nε 2 n . (41
√ Cε c,n , B c,n , • ∞ ) log N (3 √ Cε n , B c,n , • ∞ ) since ε c,n ε n /2 log N (3ε n , B n , • ∞ ) since B c,n ⊂ B n and C > 1 6Cnε 2 n from (39) 24Cnε 2 c,n since ε c,n ε n /2 n(12 √ Cε c,n ) 2 .
Hence [START_REF] Lenk | Towards a practicable Bayesian nonparametric density estimator[END_REF] holds. As for [START_REF] Li | Gaussian processes: inequalities, small ball probabilities and applications[END_REF], from [START_REF] Salimbeni | Doubly stochastic variational inference for deep Gaussian processes[END_REF], one gets

P(W c / ∈ B c,n ) = P(W c / ∈ B n ) = P(W / ∈ B n |W ∈ B c ) e -Cnε 2 n P(W ∈ B c ) = e -Cnε 2 c,n
so that (28) holds. Last, one has

P( W c -w 0 ∞ < 2ε c,n ) P( W -w 0 ∞ < 2ε c,n , W ∈ B c ) e -φc,w 0 (εc,n) e -nε 2 c,n
so that (29) holds.

Lemma C.8. Recall the Hellinger distance h from [START_REF] Choudhuri | Nonparametric binary regression using a Gaussian process prior[END_REF], the function ψ → C ψ from [START_REF] Ibragimov | Gaussian Random Processes[END_REF], and the function z → p z from [START_REF] Choi | On posterior consistency in nonparametric regression problems[END_REF]. For any functions v and w in B c , we have

h(p Cv , p Cw ) K H v -w ∞ e K H v-w ∞ /2 , K(p Cv , p Cw ) cK 2 H v -w 2 ∞ (1 + K H v -w ∞ )e K H v-w ∞ ,
and

V (p Cv , p Cw ) cK 2 H v -w 2 ∞ (1 + K H v -w ∞ ) 2 e K H v-w ∞ ,
where K H is as in Lemma C.7 and c 1 is a nite constant.

Proof of Lemma C.8. By [START_REF] Van Der Vaart | Rates of contraction of posterior distributions based on Gaussian process priors[END_REF]Lemma 3.1], one has, with a nite constant c 1,

h(p Cv , p Cw ) C v -C w ∞ e Cv-Cw ∞ /2 , (44)
K(p Cv , p Cw ) c C v -C w 2 ∞ (1 + C v -C w ∞ ) e Cv-Cw ∞ , (45)
V (p Cv , p Cw ) c C v -C w 2 ∞ (1 + C v -C w ∞ ) 2 e Cv-Cw ∞ . ( 46
)
Thus we conclude the proof from Lemma C.7.

Now we are able to prove Theorem 3.1.

Proof of Theorem 3.1. Consider a sequence (ε c,n ) n∈N satisfying the conditions of the theorem. Dene w 0 ∈ B by, for (h, i) ∈ I and for t ∈ [-1, 1] dmax , w 0 (t, h, i) = z 0,h,i (T h (t)).

Then, since ε c,n → 0 for n large enough, from the assumptions of the theorem on the functions z 0,h,i , the conditions ( 25) and ( 26 Hence,

C c 4t = C + 4. ( 47
) Let ε n = 2 √ tε c,n and ε n = 14 √ C c K H ε c,n ε n .
Let us apply [18, Theorem 8.9], with the constant C, the sequences (ε n ) n∈N and (ε n ) n∈N , and the prior p C Wc . Let us thus check that the assumptions of [START_REF] Ghosal | Fundamentals of nonparametric Bayesian inference[END_REF]Theorem 8.9] are satised, using Theorem B.3. Since nε 2 c,n → ∞, there exists N ∈ N * such that, for n N , e -Ccnε 2 c,n < 1/2. Hence, the assumptions of Theorem B.3 hold, with C there given by C c and with (ε c,n ) n N as considered above. Hence, from this theorem, there exists an integer n C 0 and a sequence of sets (B c,n ) n n C such that the conclusions ( 27), [START_REF] Li | Gaussian processes: inequalities, small ball probabilities and applications[END_REF], and ( 29) hold (with C replaced by C c ). Then, in [START_REF] Ghosal | Fundamentals of nonparametric Bayesian inference[END_REF]Theorem 8.9], we choose P n,1 = {p Cw ; w ∈ B c,n }.

• First, let w 1 , . . . , w N be a • ∞ -distance covering of B c,n with radius 6

√ C c ε c,n with N = N (6 √ C c ε c,n , B c,n , • ∞ )
. Now let x ∈ P n,1 (so that x = p Cw for some w ∈ B c,n ). There is i ∈ {1, . . . , N } such that w -w i ∞ 6 √ C c ε c,n . Then, by Lemma C.8, one has

h(p Cw , p Cw i ) K H w -w i ∞ e K H w-w i ∞ /2 6K H C c ε c,n e 3K H √ Ccεc,n 7K H C c ε c,n ,
for n large enough, since ε c,n → 0. This leads to, for n large enough,

log N 7 C c K H ε c,n , P n,1 , h log N 6 C c ε c,n , B c,n , • ∞ n(12 C c ε c,n ) 2 from (27) with C replaced by C c n(14 C c K H ε c,n ) 2 since K H > 1.
Hence, we obtain, for n large enough,

log N (ε n /2, P n,1 , h) nε 2 n . (48)
• Second, using [START_REF] Li | Gaussian processes: inequalities, small ball probabilities and applications[END_REF] in Theorem B.3 (with C replaced by C c ) and then [START_REF] Van Der Vaart | Adaptive Bayesian estimation using a Gaussian random eld with inverse gamma bandwidth[END_REF],

P(p C Wc ∈ P n,1 ) = P(W c / ∈ B c,n ) e -Ccnε 2 c,n = e -(C+4)nε 2 n . (49)
• Third, for p : [-1, 1] d → (0, ∞) with [-1,1] d p(x)dx = 1, let, with K and V dened in [START_REF] Tokdar | Posterior consistency of logistic Gaussian process priors in density estimation[END_REF] and ( 43) respectively,

V 2,0 (p 0 ; p) = [-1,1] d log p 0 (t) p(t) -K(p 0 , p) 2 p 0 (t)dt V (p 0 , p).
Then, using Lemma C.8 and (29) in Theorem B.3, one has, for n large enough,

P K(p 0 , p C Wc ) < ε 2 n , V 2,0 (p 0 ; p C Wc ) < ε 2 n = P K(p 0 , p C Wc ) < 4tε 2 c,n , V 2,0 (p 0 ; p C Wc ) < 4tε 2 c,n P K(p 0 , p C Wc ) < 4tε 2 c,n , V (p 0 , p C Wc ) < 4tε 2 c,n P cK 2 H W c -w 0 2 ∞ (1 + K H W c -w 0 ∞ )e K H Wc-w 0 ∞ < 4tε 2 c,n , cK 2 H W c -w 0 2 ∞ (1 + K H W c -w 0 ∞ ) 2 e K H Wc-w 0 ∞ < 4tε 2 c,n P 2cK 2 H W c -w 0 2 ∞ < 4tε 2 c,n = P W c -w 0 ∞ < 2ε c,n exp{-nε 2 c,n } = exp -n 1 4t ε 2 n = exp -nCε 2 n . (50
: [-1, 1] d → R, we let L w : [-1, 1] d × {0, 1} → R be dened by L w (u, v) = f w (u) v (1 -f w )(u) 1-v for (u, v) ∈ [-1, 1] d × {0, 1}. (51)
Notice that, if f w (dened in ( 13)) is a candidate function for f 0 , then L w (U, V ) is the likelihood function of (U, V ) with respect to the measure L U ⊗ µ. Extend the denition of the Kullback-Liebler divergence K in [START_REF] Tokdar | Posterior consistency of logistic Gaussian process priors in density estimation[END_REF] and that of V in [START_REF] Tsirelson | The density of the distribution of the maximum of a Gaussian process[END_REF], replacing the input space

[-1, 1] d by [-1, 1] d × {0, 1} and Lebesgue measure dt by L U ⊗ µ. For f : [-1, 1] d → R,
write also f 2,U for the L 2 norm of f with respect to L U .

Lemma C.9. For any functions v and w in B c , we have

L Cv -L Cw 2,U = √ 2 f Cv -f Cw 2,U √ 2K H Ψ ∞ v -w ∞ K(L Cw , L Cw 0 ) K 2 H Ψ Ψ(1 -Ψ) ∞ ∨ 1 w -w 0 2 ∞ ,
and

V (L Cw , L Cw 0 ) K 2 H Ψ Ψ(1 -Ψ) ∞ ∨ 1 2 w -w 0 2 ∞ ,
using the denitions ( 22), [START_REF] Damianou | Deep Gaussian processes[END_REF], and (51) and where K H is as in Lemma C.7.

Proof. Lemma 3.2 of [START_REF] Van Der Vaart | Rates of contraction of posterior distributions based on Gaussian process priors[END_REF], with the observation that S(w, w 0 ) there is bounded by

Ψ Ψ(1 -Ψ) ∞ ∨ 1 yields L Cv -L Cw 2,U = √ 2 f Cv -f Cw 2,U √ 2 Ψ ∞ C v -C w 2,U K(L Cw , L Cw 0 ) Ψ Ψ(1 -Ψ) ∞ ∨ 1 C w -C w 0 2 2,U , and
V (L Cw , L Cw 0 ) Ψ Ψ(1 -Ψ) ∞ ∨ 1 2 C w -C w 0 2 2,U .
The proof is concluded using that

• 2,U • ∞ and that C v -C w ∞ K H v -w ∞ for v, w ∈ B c from Lemma C.7.
Proof of Theorem 3.2. We proceed as in the proof of Theorem 3.1, using [18, Theorem 8.9], with d there given by d(L Cv , L Cw ) = L Cv -L Cw 2,U /2 for v and w ∈ B c . Remark that this choice of d indeed satises the assumption (8.2) in [START_REF] Ghosal | Fundamentals of nonparametric Bayesian inference[END_REF], because of the item (vi) in [START_REF] Ghosal | Fundamentals of nonparametric Bayesian inference[END_REF]Lemma B.1] (see the paragraph after the assumption (8.2) in [START_REF] Ghosal | Fundamentals of nonparametric Bayesian inference[END_REF]). Remark that instead of using Lemma C.8 as in the proof of Theorem 3.1, we use Lemma C.9 here. This concludes the proof by also noting that L

Cw -L z 0,H •••••z 0,1 2,U = √ 2 f Cw -f z 0,H •••••z 0,1 2,U = √ 2 f Cw -f 0 2,U for w ∈ B c from Lemma C.9.
C.

h of Z h is H N h +1 ([-1, 1]) with squared RKHS-norm, for g ∈ H N h +1 ([-1, 1]), g 2 H h = 1 -1 (g (N h +1) (t)) 2 dt + N h i=0 g (i) (-1) 2 .
Here g (0) = g and g (1) , . . . , g (N h ) are the derivatives of g. First consider the setting of Theorem 3.1. Dene z 0,1 , . . . , z 0,h by z 0,1 = log(p 0 )/2 log(p 0 ) ∞ , z 0,h = id for h = 2, . . . , H -1, and z 0,H = 2 log(p 0 ) ∞ id. Here id is the identity function. Then z 0,H •• • ••z 0,1 = log(p 0 ). Furthermore, consider that the constants K 1 , . . . , K H are selected large enough such that the functions z 0,1 , . . . , z 0,H satisfy the conditions of Theorem 3.1. Recall K min dened in [START_REF] Chen | Quadratic functionals and small ball probabilities for the m-fold integrated Brownian motion[END_REF].

Lemma C.10. We have, for ε ∈ (0, 1],

inf

g∈H 1 g-z 0,1 <ε g 2 H 1 C sup ε -2(N 1 -β+1)/β
and, for h = 2, . . . , H,

inf g∈H h g-z 0,h ∞ < ε 2 ∂g/∂x 1 -∂z 0,h /∂x 1 ∞ < K min 4 g 2 H h C sup . Proof of Lemma C.10. The function z 0,1 = log(p 0 )/2 log(p 0 ) ∞ is a function in F β ([-1, 1], R) that does not depend on ε. Hence, because H 1 = H N 1 +1 ([-1, 1
]) as seen above, from the proof of Theorem 4.1 in [START_REF] Van Der Vaart | Rates of contraction of posterior distributions based on Gaussian process priors[END_REF], we obtain the rst inequality of the lemma.

To prove the second inequality, remark that z 0,h is a linear function from [-1, 1] to R and thus it belongs to H h . Hence the inmum in the second inequality of the lemma is smaller than z 0,h 2 H h < ∞.

Lemma C.11. We have, for ε ∈ (0, 1] and for h = 1, . . . , H,

-log P ( Z h ∞ ε) C sup ε -1/(N h +1/2) .
Proof of Lemma C.11. We have, for h = 1, . . . , H,

-log P ( Z h ∞ ε) = -log P sup t∈[0,2] I N h B h (t) + N h =0 X h, t ! ε .
The right-hand side above is bounded by C sup ε -1/(N h +1/2) as stated in the proof of Theorem 4.1 in [START_REF] Van Der Vaart | Rates of contraction of posterior distributions based on Gaussian process priors[END_REF] (this follows in particular from Theorem 1.3 in [START_REF] Chen | Quadratic functionals and small ball probabilities for the m-fold integrated Brownian motion[END_REF]).

Recall Φ c,z 0 from (8

∈ (0, 1], Φ c,z 0 (ε) C sup ε -2(N 1 -β+1)/β .
Hence, without loss of generality, one may assume that C sup > 1 and thus we have, for n large enough, Φ c,z 0 C sup n -β/(2N 1 +2) nC 2 sup n -2β/(2N 1 +2) . Hence, the conclusion of Theorem 3.1 holds with ε c,n = C sup n -β/(2N 1 +2) . The proof for the conclusion of Theorem 3.2 is the same.

C.6 Proof of Theorem 4.2

The proof is similar to the proof of Theorem 4.1 As done previously, we write for simplicity, for h = 1, . . . , H, z 0,h,1 = z 0,h , Z h,1 = Z h , and H h,1 = H h . For any α > 0 and for any measurable function g on [-1, 1], we dene the (left-sided) Riemann-Liouville integral of g of order α (if it exists) by, for t ∈ [-1, 1], (I α 0+ g)(t) = 1 Γ(α) t -1 (t -s) α-1 g(s)ds where Γ stands for the standard Gamma function. As shown in [START_REF] Van Der Vaart | Rates of contraction of posterior distributions based on Gaussian process priors[END_REF]Theorem 4.2] and with a linear change of input variables (see [START_REF] Van Der Vaart | Reproducing kernel Hilbert spaces of Gaussian priors[END_REF]Lemma 7.1]), using the fact that the RKHS of a sum of independent Gaussian processes is the sum of their RKHS's (see [START_REF] Berlinet | Reproducing Kernel Hilbert Spaces in Probability in Statistics[END_REF]Theorem 5]), for h = 1, . . . , H, the RKHS H h of Z h is the set of functions of the form

I α h +1/2 0+ (g) + P α h +1 , ( 52
)
where g is square integrable on [-1, 1] and where P α h +1 is a polynomial of degree α h + 1.

First consider the setting of Theorem 3.1. Dene z 0,1 , . . . , z 0,H as in the proof of Theorem 4.1 with in particular z 0,H • • • • • z 0,1 = log(p 0 ). Also as in the proof of Theorem 4.1, let the constants K 1 , . . . , K H be selected large enough such that the functions z 0,1 , . . . , z 0,H satisfy the conditions of Theorem 3.1.

3 2 inf g∈H 1 g-z 0,1 ∞ <ε g 2 H 1 -2 log P ( Z 1 ∞ < ε) C sup ε -1/α 1
and, for h = 2, . . . , H,

3 2 inf g∈H h g-z 0,h ∞ < ε 2 ∂g/∂x 1 -∂z 0,h /∂x 1 ∞ < K min 4 , g 2 H h -2 log P Z h ∞ ε 2 -2 log P ∂Z h /∂x 1 ∞ K min 4 C sup ε -1/α h .
Proof of Lemma C.12. The rst part of the lemma comes with a straightforward application of [START_REF] Van Der Vaart | Rates of contraction of posterior distributions based on Gaussian process priors[END_REF]Theorem 4.3] (note that the arguments of its proof would be the same if the support [0, 1] there was replaced by [-1, 1]).

Then the second inmum in the statement of the lemma is bounded because z 0,h (a multiple of identity) belongs to H h since H h contains the polynomials of degrees up to α h + 1. The quantity -2 log P Z h ∞ ε/2 is smaller than the term displayed in Lemma C.12 (with Z 1 there replaced by Z h ) and thus this quantity is bounded by C sup ε -1/α h from this lemma. Finally, the quantity -2 log P ∂Z h /∂x j ∞ K min /4 does not depend on ε.

Hence, we obtain from α h α 1 , for h = 1, . . . , H, that Φ c,z 0 (ε) C sup ε -1/α 1 . Then, as in the proof of Theorem 4.1 and without loss of generality, one may assume that C sup > 1 and thus we have, for n large enough, Φ c,z 0 C sup n -α 1 /(2α 1 +1) nC 2 sup n -2α 1 /(2α 1 +1) .

Hence, the conclusion of Theorem 3.1 holds with ε c,n = C sup n -α 1 /(2α for Ψ : R d → R such that R d Ψ 2 (λ)m h,i (λ)dλ < ∞, with m h,i as in [START_REF] Finocchio | Posterior contraction for deep Gaussian process priors[END_REF]. First consider the setting of Theorem 3.1. Dene z 0,h,i for (h, i) ∈ I as follows. Without loss of generality (up to swapping Z 1,1 , . . . , Z 1,d 2 ), we may assume that α 1,1 = α 1,min . Then we let z 0,1,1 = log(p 0 )/2 log(p 0 ) ∞ , z 0,1,2 = • • • = z 0,1,d 2 = 0. For h = 2, . . . , H -1, we let z 0,h,1 (u 1 , . . . , u d h ) = u 1 for (u 1 , . . . , u d h ) ∈ [-1, 1] d h and we let z 0,h,2 = • • • = z 0,h,d h+1 = 0. Finally, we let z 0,H,1 = (u 1 , . . . , u d H ) = 2u 1 log(p 0 ) ∞ for (u 1 , . . . , u d H ) ∈ [-1, 1] d H . Then z 0,H • • • • • z 0,1 = log(p 0 ). Furthermore, consider that the constants K h,i for (h, i) ∈ I are selected large enough such that the functions z 0,h,i for (h, i) ∈ I satisfy the conditions of Theorem 3.1.

Lemma C.13. We have, for ε ∈ (0, 1], Then the two last inma in the statement of the lemma are bounded because z 0,h,i (either zero or a linear function) belongs to H h,i . Indeed, z 0,h,i , dened on [-1, 1] d h can be written as the restriction of an innitely dierentiable compactly supported function on R d h , the latter function thus satisfying (53). The quantity -2 log P Z h,i ∞ ε/2 is bounded by C sup ε -d/α h,i from [44, Lemma 3]. Finally, the quantity log P ∂Z h,i /∂x j ∞ K min /4 does not depend on ε.

3 2 inf g∈H 1,1 g-z 0,1,1 ∞ <ε g 2 H 1,1 -2 log P Z 1,1 ∞ < ε C sup ε -d/α
Hence, we obtain from α h,i β for (h, i) ∈ I that Φ c,z 0 (ε) C sup ε -d/ min (h,i)∈I α h,i + C sup ε -(2α 1,1 +d-2β)/β C sup ε -(2α 1,1 +d-2β)/β .

Hence, as in the proof of Theorems 4.1 and 4.2, without loss of generality, one may assume that C sup > 1 and thus we have Φ c,z 0 C sup n -β/(2α 1,min +d) nC 2 sup n -2β/(2α 1,min +d) . Hence, the conclusion of Theorem 3.1 holds with ε c,n = C sup n -β/(2α 1,min +d) . The proof for the conclusion of Theorem 3.2 is the same.

Figure 1 :

 1 Figure 1: Example of a deep Gaussian process from R 2 to R. Here H = 3, d 1 = d = 2, d 2 = 3, d 3 = 1, and d H+1 = d 4 = 1.

)

  Now let B c,n = B n ∩ B c . Clearly, B c,n ⊂ B n . Furthermore, since nε 2 c,n → ∞, there exists a xed n C ∈ N * such that, for n n C , ε c,n ε n /2. Thus, for n n C , log N (6

  ) of Theorem B.2 hold. Then Theorem B.2 provides φ c,w 0 (ε c,n ) Φ c,z 0 (ε c,n ) and thus φ c,w 0 (ε c,n ) nε 2 c,n . Let, with c and K H as in Lemma C.8, t = 2cK 2 H , C c = 1 + 16t and C = 1 4t .

  for any Borel set B ⊂ B. Hence W c corresponds to the Gaussian process W conditioned by the constraints (2) and/or (3). For any function ψ ∈ B c , let C ψ be the function from [-1, 1] d to R dened by the follow-

ing. For h = 1, . . . , H, we let the function C ψ,h = (P h,1 (ψ), . . . , P h,d h+1 (ψ)) be dened on

[-1, 1] d h and valued in [-1, 1] d h+1 . Then we let, for t ∈ [-1, 1] d ,

  |∂X(t j )/∂x i | b i , i = 1, . . . , = P X(t 1 ), . . . , X(t n ), ∂X(t 1 )/∂x 1 , . . . , ∂X(t n )/∂x 1 , . . . , ∂X(t 1 )/∂x , . . . , ∂X(t n )/∂x ∈ A ∩ ∩ i=1 B i P X(t 1 ), . . . , X(t n ), ∂X(t 1 )/∂x 1 , . . . , ∂X(t n )/∂x 1 , . . . , ∂X(t 1 )/∂x , . . . , ∂X(t n )/∂x ∈ A ), . . . , X(t n ), ∂X(t 1 )/∂x 1 , . . . , ∂X(t n )/∂x 1 , . . . , ∂X(t 1 )/∂x , . . . , ∂X(t n )/∂x ∈ B i

	max j=1,...,n	|X(t j )| a, max j=1,...,n
	× P X(t 1 = P max i=1 j=1,...,n |X(t j )| a ×	i=1	P max j=1,...,n

  one, continuous and linear application. Hence, from[START_REF] Van Der Vaart | Reproducing kernel Hilbert spaces of Gaussian priors[END_REF] Lemma 7.1], the RKHS H h,i can also be equivalently dened as in[START_REF] Van Der Vaart | Reproducing kernel Hilbert spaces of Gaussian priors[END_REF] Section 2.2], but this time by seeing Z h,i as a random element of (C 1

  ). Also recall that we have dened Z = M (Z h,i ).Hence, from[START_REF] Van Der Vaart | Reproducing kernel Hilbert spaces of Gaussian priors[END_REF] Lemma 7.1], the RKHS H Z of the Gaussian process Z (dened as in[START_REF] Van Der Vaart | Reproducing kernel Hilbert spaces of Gaussian priors[END_REF] Section 2.2], by seeing Z as a random element of (B h ,• ∞,ε )), is equal to {M (g); g ∈ H h,i }. Furthermore, for g ∈ H h,i , we have g H h,i = M (g) H Z, where we recall that • H Z is the norm of the RKHS H Z . From [46, Theorem 2.1], this latter denition of H Z coincides with the denition of the RKHS H Z in the statement of the lemma (from [46, Section 2.1], from the covariance function of Z). The two denitions of the corresponding norms also coincide (again from[START_REF] Van Der Vaart | Reproducing kernel Hilbert spaces of Gaussian priors[END_REF] Theorem 2.1]). Proof of Theorem B.3. Let C > 1 and let (ε c,n ) n∈N * satisfy the conditions of the theorem. -log P(W ∈ B c )/Cn ε 2 c,n . By Lemma B.1 (with the denition of φ w 0 in (

	C.2 Proof of Theorem B.3
	We set ε 2 n = ε 2 c,n

  .4 Proof of Theorem 3.2 With L U the law of U and µ the counting measure on {0, 1}, for any L : [-1, 1] d × {0, 1} → R, we write L 2,U for the L 2 -norm of L with respect to the measure L U ⊗ µ. For w

	)
	Hence, we can apply [18, Theorem 8.9], with the sequences (ε n ) and (ε n ) (starting at a
	n large enough) and the constant C. Indeed, (48) (resp. (49) and (50)) here enables to
	show Equation (8.

5) (resp. Equation (8.6) and Equation (8.4)) in [18, Theorem 8.9]. As a consequence, [18, Theorem 8.9] implies (12).

C

  5 Proof of Theorem 4.1Since Z 1 , . . . , Z H are univariate here, we write for simplicity, for h = 1, . . . , H, z 0,h,1 = z 0,h (when we will apply Theorems 3.1 and 3.2 below), Z h,1 = Z h , and H h,1 = H h . For ∈ N and for (a, b) ∈ R 2 such that a < b, let H +1 ([a, b]) be the Sobolev space of -times continuously dierentiable functions h on [a, b] with -th derivative function h ( ) that is the integral of a square integrable function h ( +1) on [a, b]: h ( ) (x) = For h = 1, . . . , H, as shown in[46, Section 10] and with a linear change of input variables (see[START_REF] Van Der Vaart | Reproducing kernel Hilbert spaces of Gaussian priors[END_REF] Lemma 7.1]), the RKHS H

	for x ∈ [a, b].	x a h ( +1) (t)dt,

  ). From Lemmas C.10, C.11, and C.3 and from β N 1 + 1/2 and β N h , for h = 2, . . . , H, we obtain, for ε

  1 +1) . The proof for the conclusion of Theorem 3.2 is the same.C.7 Proof of Theorem 4.3The proof is similar to the proof of Theorems 4.1 and 4.2. By [47, Lemma 4.1] (as in[44, (11)]), the RKHS H h,i of Z h,i is the set of restrictions to [-1, 1] d h of all real parts of the functions h : R d h → R of the form

	h(t) =	e iλ t Ψ(λ)m h,i (λ)dλ	(53)
	R d

  1,1 + C sup ε -2α 1,1 +d-2β β and, for (h, i) ∈ I\{(1, 1)}, if h = 1, ∂Z h,i /∂x j ∞ K min 4 C sup ε -d/α h,i .Proof of Lemma C.13. A straightforward application of[START_REF] Van Der | Information rates of nonparametric Gaussian process methods[END_REF] Lemmas 3 and 4] leads to the rst part of the lemma.

	3 2	inf g∈H h,i	g 2 H h,i -2 log P Z h,i ∞	ε 2	C sup ε -d/α h,i
		g-z 0,h,i ∞	< ε 2
	and for h = 2, . . . , H,
	3 2	inf g∈H h,i	g 2 H
	< ε 2 ∂g/∂x K min g-z 0,h,i ∞ 4 ,
		j=1,...,d h

j -∂z 0,h,i /∂x j ∞ < h,i -2 log P Z h,i ∞ ε 2 -2 log P

Lemma B.1. Let φ w 0 (ε) be as in[START_REF] Bevilacqua | Estimating space and space-time covariance functions for large data sets: a weighted composite likelihood approach[END_REF] with x 0 = w 0 , X = W and E = X . One has φ w 0 (ε) φ c,w 0 (ε) for any ε > 0.

Acknowledgments

We are very grateful to Jean-Marc Azaïs, Ismaël Castillo, Michel Ledoux and Thibault Randrianarisoa for fruitful discussions, technical advices and constructive comments. We acknowledge support from the ANR, with the project GAP.

with the convention that the inmum above is equal to +∞ if there are no g ∈ H X such that g -f < ε. Proof of Lemma C.2. If there is no g ∈ H X such that g -f < ε, then the inequality of the lemma is trivially true. If there exists one g ∈ H X with g -f < ε, we have, for such a g, using rst the triangle inequality and then [START_REF] Li | Gaussian processes: inequalities, small ball probabilities and applications[END_REF]Theorem 3.1],

-log P ( X -f < 2ε) -log P ( X -g ε) 1 2 g 2 H X -log P ( X < ε) .

This concludes the proof.

Lemma C.3. Let T be a compact metric set and let X be a continuous centered univariate Gaussian process indexed by T . Then, for u > 0,

Proof of Lemma C.3. This result is often stated implicitly in the literature (for instance in [START_REF] Li | Gaussian processes: inequalities, small ball probabilities and applications[END_REF]) but we are not aware of an explicit statement. Here is a proof. Let u > 0 and let (t i ) i∈N * be a sequence of elements in T that is dense in T . For n ∈ N * , let F n be the σ-algebra generated by X(t 1 ), . . . , X(t n ). Then, from [5, Proposition 2.9] for instance, E[X(•)|F n ] → X uniformly on T and almost surely. Hence, one has

as n → ∞. Using the triangular inequality and from the independence between X -E[X(•)|F n ] and E[X(•)|F n ] (as the conditional distribution of the rst process given F n is deterministic), we obtain

The rst probability above is non-zero for n large enough, as seen before. The second probability is non-zero for any n ∈ N * , because

Lemma C.4. Let X be a centered univariate Gaussian process with continuous realizations, indexed by a compact metric set T . Then, for all u > 0,

In other words, the random variable sup t∈T |X(t)| is absolutely continuous on the half-line

Proof of Lemma C.4. Let T be a dense countable subset of T . By density, since T is dense, and X has continuous realizations, we have, for u > 0,

where we have used the symmetry of the law of a centered Gaussian process. This last probability is zero from Tsirelson's theorem ( [START_REF] Tsirelson | The density of the distribution of the maximum of a Gaussian process[END_REF], see also [START_REF] Azaïs | Level sets and extrema of random processes and elds[END_REF]Theorem 7.1]), together with Lemma C.3.

Lemma C.5. Let X be a centered univariate Gaussian process indexed by [-1, 1] for some ∈ N * with continuously dierentiable realizations. Then, for a > 0 and for b 1 , . . . , b > 0, we have

Proof of Lemma C.5. Let T = [-1, 1] and (t j ) j∈N * be a sequence of elements of T that is dense in T . From the dominated convergence theorem and Lemma C.4, as n → ∞,

1 , . . . , z (1) n , . . . , z

and, for i = 1, . . . ,

1 , . . . , z (1) n , . . . , z

Then A and B are convex symmetric subsets of R 2n . Hence, from the Gaussian correlation C.3 Proof of Theorem 3.1

Before proving Theorem 3.1, let us establish the following lemma and dene

Recall the denition of ψ → C ψ in [START_REF] Ibragimov | Gaussian Random Processes[END_REF].

Lemma C.7. For w, z ∈ B c , we have

where

Proof of Lemma C.

Hence, we can obtain the lemma by a descending induction from H to 1.

Now we dene the Kullback-Leibler divergence: for f, g :

if [-1,1] d |log(f (t)/g(t))| f (t)dt < +∞ and K(f, g) = +∞ else. We also introduce

Then we have the following lemma, that follows from Lemma C.