Max Dit

Clément Hadrien Charles

Arnaud Ainsi Qu'

Eugène, Younès, Florent, David Jeanne Dimitri

Caroline

François Camille Thibault

Aline Toto

Laura

Malau, Zaza, Héléna Charlène A Nicolas

Caroline Chantal

Sylvie

Denis

Lulu

Pascale

Keywords: Bayesian nonparametrics, Spike and Slab prior, thresholding, Pólya tree, Bernstein-von Mises theorems Bayésien non paramétrique, a priori Spike and Slab, seuillage, arbre de Pólya, théorèmes Bernstein-von Mises

Lourde tâche que de remercier tous les gens qui ont pu m'accorder leur soutien ces dernières années, j'espère n'oublier personne. Et toi que j'aurais oublié, sache que je te remercie quand même.

La première personne que je souhaite remercier est bien sûr Ismaël, mon directeur de thèse, qui a toujours su faire preuve de gentillesse, de patience et a su être un soutien inconditionnel même dans mes moments les plus difficiles (et Dieu sait qu'il y en a eu). Sa passion pour les mathématiques et son extrême rigueur sont ce qui m'ont le plus marqué chez lui et j'aimerais toujours m'en inspirer à l'avenir, et il y a en réalité peu de mots pour dire toute l'admiration que j'ai pour lui, à la fois sur le plan scientifique et le plan humain.

Je remercie également les personnes qui ont accepté de faire partie de mon jury : Cristina Butucea, Julyan Arbel, Pierre Alquier, mes rapporteurs Aad van der Vaart et Vincent Rivoirard pour avoir eu la gentillesse de lire en détails ma thèse, et aussi tout particulièrement Stéphane Boucheron qui aura eu pour moi le rôle d'un second mentor au long de ma thèse.

Je souhaite aussi remercier mes collègues de Paris Diderot et particulièrement Ratha qui fût un peu une seconde mère pour moi et les autres membres de notre bureau Zakaria, Dominique et Inès que je remercie pour les bons moments passés ensemble, Ziaad avec qui on aura échangés quelques (nombreuses) heures d'enseignement et Yann, dont l'amitié remonte à bien plus loin que le début de ma thèse. Je n'oublie pas également Raphaël Lefevere et les élèves de MIASHS avec qui j'aurai passé de bons moments d'enseignement.

Merci aussi à tous les affamés de Jussieu pour tous ces bons moments passés au restaurant des personnels, appelé plus sobrement "la cantine", à savoir Eric, Alexandre, Nicolas, Nicolas, Rancy, Thibault, Michel, Paul, Carlo, Henri, Sarah, Olga, PA et d'autres bien sûr. Mes profondes amitiés aux membres du bureau 203 (où le bon mot du jour aura été chaque jour : "j'ai faim") qui devraient se reconnaître : Burrito pour toutes les distractions (mais aussi conseils et astuces scientifico-informatiques, je dois le rappeler quand même) dont tu as eu connaissance et que tu as bien sûr partagées, Léo dit Rasemoohret pour avoir été la garantie du bon goût dans ce bureau, un Zèbre Lambda aussi (que je dédeste en fait), ainsi que l'homme sage du bureau dont aucune blague ne sera tombée à l'eau (de Co...) j'ai nommé.. Curve ? Tant que j'y suis merci aussi aux Missplayers pour leurs..

Résumé détaillé

Ce document rassemble les travaux que j'ai effectués sous la direction d'Ismaël Castillo pendant la durée de ma thèse centrée sur l'utilisation dans un cadre bayésien de l'a priori Spike and Slab dans des modèles de dimension grande ou infinie, et des propriétés asymptotiques qui en découlent. Ce travail est divisé en 4 chapitres, un chapitre introductif et 3 chapitres qui font l'objet d'articles (un paru pour le deuxième chapitre, et deux à soumettre pour les suivants).

Analyse par bayésien empirique de lois a posteriori Spike and Slab.

On considère le modèle de suite gaussienne parcimonieuse, où l'on observe X 1 , . . . , X n des variables aléatoires telles que pour tout i ∈ {1, . . . , n} X i = θ i + ε i avec le bruit ε tel que ses coordonnées ε i suivent la loi normale standard (de densité notée ϕ) et θ ∈ R n le paramètre à estimer. On suppose que ce paramètre θ est parcimonieux, c'est-à-dire qu'il appartient à la classe ℓ 0 [s n ] suivante :

ℓ 0 [s n ] = {θ ∈ R n , #{i; θ i ̸ = 0} ≤ s n }
avec (s n ) n une suite qui tend vers l'infini mais telle que s n /n → 0 quand n → ∞. On considère la convergence de lois a posteriori bayésiennes de lois a priori Spike and Slab :

Π α = n i=1
(1 -α)δ 0 + αΓ, où Γ est une loi à densité notée γ sur R. La famille de lois Π α permet de modéliser des vecteurs parcimonieux grâce au paramètre de parcimonie α ∈]0; 1[. Ce paramètre est xii calibré par une approche bayésienne empirique : on le remplace par un estimateur α construit en maximisant la vraisemblance marginale bayésienne empirique : n i=1 ((1 -α)ϕ(X i ) + αϕ * γ(X i )). [START_REF] Johnstone | Needles and straw in haystacks: empirical Bayes estimates of possibly sparse sequences[END_REF] ont montré que la médiane a posteriori avec plug-in de α converge à vitesse optimale au sens minimax pour la perte quadratique sur la classe des vecteurs parcimonieux ℓ 0 [s n ], dès que la loi Γ (dite Slab) a des queues de distribution au moins Laplace.

Dans ce travail, on considère la loi a posteriori plug-in complète Π α(•|X ). On s'intéresse principalement au moment d'ordre 2 a posteriori ˆ∥θ -θ 0 ∥ 2 dΠ α(θ|X ).

On montre que, sous certaines conditions sur Γ, le moment d'ordre 2 a posteriori converge lui aussi à vitesse minimax optimale pour la perte quadratique. De façon surprenante, ce n'est pas le cas pour Γ la loi Laplace : on montre qu'il est en effet nécessaire que Γ ait des queues polynomiales (plus lourdes que x -3 , par exemple Γ Cauchy) pour que le moment d'ordre 2 a posteriori converge à vitesse optimale. On montre que cette sous-optimalité pour un Slab Laplace n'est pas dûe au second moment puisqu'elle se traduit également sur la loi a posteriori entière. Par ailleurs, on montre que des résultats similaires (à un facteur logarithmique près) sont vrais pour la classe de lois dite Spike and Slab LASSO récemment introduite par [START_REF] Ročková | The spike-and-slab LASSO[END_REF] et [START_REF] Ročková | Bayesian estimation of sparse signals with a continuous spike-and-slab prior[END_REF].

Constante exacte pour l'a posteriori Spike and Slab calibré par bayésien empirique.

Ce travail se situe dans le même cadre que le chapitre précédent et poursuit l'étude de la loi a posteriori plug-in complète Π α(•|X ). Les résultats d'optimalité évoqués ci-dessus le sont à constante près. Ainsi, pour θmed (X) la médiane a posteriori, [START_REF] Johnstone | Needles and straw in haystacks: empirical Bayes estimates of possibly sparse sequences[END_REF] montrent que pour n assez grand et pour une constante C > 0 assez grande, sup

θ 0 ∈ℓ 0 [sn] E θ 0 ∥ θmed (X) -θ 0 ∥ 2 ≤ Cs n log( n s n )(1 + o(1)).
Il est connu que la vitesse minimax pour ce problème est 2s n log(n/s n ))(1+o(1)) quand n → ∞. Il est possible de montrer que l'a posteriori Spike and Slab dans lequel on fait un Résumé détaillé xiii plug-in d'un paramètre α oracle fait atteindre la vitesse minimax avec constante exacte 2 au moment d'ordre 2, et ce même pour un Slab Laplace. On peut donc naturellement se demander si le second moment a posteriori avec plug-in du maximum de vraisemblance peut lui aussi converger à vitesse minimax, cette fois adaptative. On montre qu'en effet, pour un choix approprié de la loi slab Γ (celui-ci doit avoir des queues très lourdes, de l'ordre de x -1 log -2 (x)), il est possible d'atteindre cette vitesse minimax exacte :

sup

θ 0 ∈ℓ 0 [sn] E θ 0 ˆ∥θ -θ 0 ∥ 2 dΠ α(θ|X ) ≤ 2s n log( n s n )(1 + o(1)).
Chapter 1 Introduction 1.1 General Frame : the non-parametric, frequentist Bayesian approach 1.1.1 The Bayesian approach Take (X , A) a measurable space, where A is a σ-field over X and (Θ, d) a subset of a separable Banach space. Consider a statistical experiment where one observes some data X ∈ X , a random object whose law will be interpreted using a model, defined here as follows

P = {P θ , θ ∈ Θ}, (1.1.1)
where the P θ are probability measures on A.

The model depends on an unknown parameter θ, let us consider this parameter θ as a random variable too. Namely θ will follow the law Π, which is called the a priori law (or simply prior).

On the other hand, one views P θ as the law of X|θ. This gives us the following Bayesian diagram X|θ ∼ P θ θ ∼ Π.

(1.1.2)

We will henceforth assume that P θ and Π are absolutely continuous relatively to fixed σ-finite measures µ and ν. Denoting by f θ and π their densities, the joint law (θ, X) has a density h(θ, x) = f θ (x)π(θ) and X has a density h(x) = ˆΘ f θ (x)π(θ)dν(θ). Under standard measurability conditions, see pages 6-7 of [START_REF] Ghosal | Fundamentals of Nonparametric Bayesian Inference[END_REF], Bayes' formula gives the following density for θ|X

Π(θ|X) = f θ (X)π(θ) h(X) 1l h(X)>0
(1.1.4)

In the classical approach, one generally builds a point estimator θ(X) ∈ Θ. The Bayesian approach provides the user with an entire probability distribution which depends on our observations X and not just a point estimator. It also provides estimators which are "aspects" of the a posteriori law: if they exist, the mean of the a posteri law ˆθdΠ(θ|X), the posterior median, or the posterior mode(s) for instance. It can be used to find credible sets (which can turn out to be confidence sets), or to make tests H 0 versus H 1 using the quantities Π(H 0 |X) and Π(H 1 |X).

From now on we will assume that we have n ∈ N observations X = X (n) = (X 1 , . . . , X n ). Score and Fisher Information in i.i.d. parametric models. A model P as above is said to be differentiable in quadratic mean (abbreviated DQM ) at θ if there exists a vector l θ (called the score at θ) of k functions such that, when h → 0

ˆ f θ+h -f θ - 1 2 h T l θ f θ 2 dµ = o(∥h∥ 2 ) (1.1.5)
The score is centered and has a variance I θ which is called the Fisher Information. It is shown in [START_REF] Van Der Vaart | Asymptotic statistics, volume 3 of Cambridge Series in Statistical and Probabilistic Mathematics[END_REF] that this also implies that the model is locally asymptotically normal (abbreviated LAN).

Frequentist Bayesian

We will follow the Frequentist approach by assuming that a true parameter θ 0 exists and has to be estimated ∃θ 0 ∈ Θ such that X ∼ P θ 0

(1.1.6)

The sequence (Π(•|X)) n∈N * is said to be P θ 0 -consistent with respect to the distance d if, for every ε > 0 as n → ∞ Π(d(θ, θ 0 ) ≤ ε|X) → 1 in P θ 0 -probability (1.1.7) This result is equivalent to the sometimes more convenient version, denoting by E θ 0 = E P θ 0 the expectation under P θ 0

E θ 0 [Π(d(θ, θ 0 ) ≤ ε|X)] → 1 (1.1.8)
The sequence (Π(•|X)) n∈N * will be strongly P θ 0 -consistent if the previous convergence is P θ 0 -almost surely.

Point Estimators. Let θ be an estimator derived from the posterior (like the posterior mean θ = ˆθdΠ(θ|X) for example). One says that is θ is consistent (uniformly in θ 0 ∈ Θ) if, as n → ∞ sup

θ 0 ∈Θ E θ 0 [d( θ, θ 0 )] → 0 (1.1.9)
Minimax convergence rate. In terms of rate of convergence, one would like to build estimators converging to the true parameter 'as fast as possible' . To do so, one defines the minimax rate r * n over the set Θ of parameters with respect to the loss function (here a distance) d, as

r * n = inf θ sup θ∈Θ E θ [d(θ, θ)],
(1.1.10)

where the infimum is taken over all estimators of the parameter. One says that θ converges at minimax rate if there exists N ∈ N such that ∀n ≥ N sup

θ 0 ∈Θ E θ 0 [d( θ, θ 0 )] ≤ Cr * n (1.1.11)
Actually the entire a posteriori law can converge at minimax rate (uniformly in

θ 0 ∈ Θ), namely if, as n → ∞ sup θ 0 ∈Θ E θ 0 [Π(d(θ, θ 0 ) ≤ Cr * n |X)] → 1 (1.1.12)
One can define a credible set of level at least 1 -γ by replacing the = by a ≥ in the definition.

In general, one may want (this may not always be possible for complex models) the diameter of a credible set to be rate-optimal, in a minimax sense, as n → ∞

sup θ 0 ∈Θ E θ 0 [Diam(C)] ≍ r * n (1.1.14)
One would naturally ask if credible sets can be used as confidence sets, namely if lim inf n→∞ inf

θ 0 ∈Θ P θ 0 (θ 0 ∈ C) ≥ 1 -γ (1.1.15)
If Θ ⊂ R k , it turns out that for quantile-type sets and i.i.d. data, one can positively answer that question using the following theorem Theorem 1 (Bernstein-von Mises). Consider a model P = {P ⊗n θ , θ ∈ Θ} such that X 1 , . . . , X n |θ ∼ P ⊗n θ . Assume that the density π of the prior is positive and continuous at θ 0 , the model P is DQM (see (1.1.5)) at the point θ 0 with an invertible Fisher Information I θ 0 . Assume also that for every ε > 0 there exists a sequence (ϕ n ) n of tests such that lim It can be checked that this theorem implies that, asymptotically, quantile-type credible sets built from the a posteriori law are confidence sets and have optimal diameter.

High and Infinite Dimension Models

Nonparametric prior distributions are harder to build and choose than in a parametric setting, as one has to define a distribution on a much larger space. Often the posterior distribution will still strongly depend on the choice of the prior distribution. One has to aim at posterior consistency at minimax rate, and it can be significantly harder than in parametric settings, where good consistency is often obtained as soon as the prior puts positive mass around the true parameter (in nonparametric setting, the precise amount of mass in vanishing neighbourhoods of the truth typically matters). The object one usually estimates in nonparametric Bayesian inference is a function or a density, for instance through the analysis of an infinite sequence of its wavelet coefficients, and building a flexible enough prior (for instance to achieve adaptive results) will require some care. Tuning the involved parameters may also demand significantly more work than in parametric settings. Nonparametric and high-dimensional models include the Gaussian sequence model (which is the main focus of Chapters 2 and 3), the Gaussian White Noise model and the Density Estimation model (which is the main focus of Chapter 4).

Estimation. In i.i.d. settings, Ghosal, Ghosh and van der Vaart developped a general framework to derive posterior rates with respect to certain distances on the parameter space (later generalised in [START_REF] Ghosal | Convergence rates of posterior distributions for noniid observations[END_REF] to non i.i.d. settings) Theorem 2 [START_REF] Ghosal | Convergence rates of posterior distributions[END_REF]). Let Π = Π n be a sequence of a priori laws and assume that X are i.i.d. with density f θ 0 . Let ε n be a sequence of positive reals such that ε n → 0 and √ nε n → ∞ as n → ∞. Assume the existence of some constants C and L such that

Π θ ∈ Θ; -E θ 0 [log( f θ f θ 0 (X))] ≤ ε 2 n , E θ 0 [log( f θ f θ 0 (X)) 2 ] ≤ ε 2 n ≥ e -Cnε 2 n and Π(Θ \ Θ n ) ≤ Le -(C+4)nε 2 n
for a sequence Θ n ⊂ Θ such that there exist tests ψ n = ψ(X 1 , . . . , X n ) such that ∀n ∈ N and M > 0 large enough

E θ 0 [ψ n ] → 0 and sup θ∈Θn;d(f θ ,f θ 0 )≥M εn E θ [1 -ψ n ] ≤ Le -(C+4)nε 2 n Then Π(d(f θ , f θ 0 ) > M ε n |X) → 0 as n → ∞ in P θ 0 -probability.
This result provides qualitative conditions such as the existence of tests (an entropy condition via ε-covering numbers of the Θ n can also be used) for the minimax convergence of the a posteriori law. Directly using this theorem may be delicate to get more precise conditions on some a priori laws (such as the Spike and Slab introduced in the following section) or for some choices of metric. In the cases where no analog of this result have been proven one sometimes needs to use more direct reasonings on the posterior distribution.

Nonparametric Bernstein-von Mises and Uncertainty Quantification. A nonparametric Bernstein-von Mises result would take the following form : (1.1.16) where one has to ask several questions, whose answers may be unclear at first but certainly depend on the situation. Firstly, what is the limiting distribution D? Secondly, what is the sense of the convergence in the result? Finally, which centering estimator T n do we choose to get this convergence result? Let us consider the Gaussian White Noise model as an example. For f ∈ L 2 ([0, 1]),

L( √ n(θ -T n )|X) → D
t ∈ [0, 1] and dW the standard Gaussian White Noise, the model is

dX (n) (t) = f (t)dt + 1 √ n dW (t).
(1.1.17)

If one chooses a wavelet basis ϕ, (ψ lk ) l∈N,0≤k<2 l (say the Haar basis to fix ideas), using the notation f lk = ⟨f, ψ lk ⟩ = ´1 0 f (t)ψ lk (t)dt, one can write (setting ψ -1,-1/2 = ϕ and letting l ≥ -1 in what follows)

ˆ1 0 ψ lk (t)dX (n) (t) = ˆ1 0 ψ lk (t)f (t)dt + 1 √ n ˆ1 0 ψ lk (t)dW (t)
that we can rewrite X lk = f lk + 1 √ n ε lk . One now has

X (n) = f + 1 √ n W
so, as √ n(X (n) -f ) = W, one would naturally take the centering T n = X (n) in (1.1.16) and the limiting distribution D = L(W) := N the law of white noise. We set τ : f → √ n(f -X (n) ) and denote by Π n the shifted posterior distribution Π(•|X (n) ) • τ . Recall also that, by definition of white noise, ∀f, g ∈ L 2 ([0, 1]), one has E[W(f )W(g)] = ⟨f, g⟩.

To establish a nonparametric BVM result, one has to consider larger spaces (here larger than L 2 ([0, 1])) as one needs a 1/ √ n rate that can only be achieved with weaker metrics. The impossibility to obtain a BVM result in L 2 has been shown by [START_REF] Cox | An analysis of bayesian inference for nonparametric regression[END_REF] and [START_REF] Freedman | Wald lecture: On the bernstein-von mises theorem with infinitedimensional parameters[END_REF]. Consider, for s > 0 the Sobolev space H -s 2 defined as

H -s 2 = {f ; ∥f ∥ 2 s,2 = l≥0 2 -2ls 2 l -1 k=0 |⟨ψ lk , f ⟩| 2 < ∞} (1.1.18)
For every s > 0, L 2 ⊂ H -s 2 . Now, one builds a 'logarithmic' Sobolev space to be the 'smallest' containing W, somewhat taking the limiting case s = 1/2. For that, one usually uses an 'admissible' sequence ω = (ω l ) l≥0 . Here we take, for δ > 1, ω l = l 2δ and set (1.1.19) This set was built to ensure that W belongs to it, as for δ > 1/2,

H(ω) = {f ; ∥f ∥ 2 ω = l≥0 2 -l ω l 2 l -1 k=0 |⟨ψ lk , f ⟩| 2 < ∞}
E[∥W∥ 2 ω ] = l≥0 2 -l ω l 2 l -1 k=0 E[ε 2 lk ] ≤ l≥0 2 -l ω l 2 l ≤ l≥0 l -2δ < ∞
We now have to state the convergence in (1.1.16). For that, we use the following metric.

Bounded Lipschitz metric. Let (S, d) be a metric space. The bounded Lipschitz metric β S on probability measures of S is defined as follows, for any µ, ν probability measures of S β S (µ, ν) = sup

F ;∥F ∥ BL ≤1
ˆS F (x)(dµ(x) -dν(x)) , (1.1.20) where F : S → R and .1.21) This metric metrizes the convergence in distribution: µ n → µ in distribution as n → ∞ if and only if β S (µ n , µ) → 0 as n → ∞.

∥F ∥ BL = sup x∈S |F (x)| + sup x̸ =y |F (x) -F (y)| d(x, y) . ( 1
Bernstein-von Mises phenomenon. One will say that the model satisfies a Bernstein-von Mises phenomenon if, as n → ∞ β H(ω) (Π n , N ) → 0 in P f 0 -probability.

(1.1.22) [START_REF] Castillo | Nonparametric bernstein-von mises theorems in gaussian white noise[END_REF] have shown that result for the Gaussian White Noise model and series priors in their Theorem 8.

In the Density Estimation model where the observations X 1 , . . . , X n are i.i.d. random variables of density f 0 assumed to be α-Hölder, one recenters the function with the help of a smoothed version of the empirical estimator 1 n n i=1 δ X i to get a convergence in the Bounded Lipschitz metric of a larger space M 0 (ω) to the law of the Gaussian White Bridge, see Section 1.3 for more details about the BVM phenomenon in density estimation.

Uncertainty Quantification. Bernstein-von Mises results are useful to build Confidence sets from Credible sets (recall the definitions (1.1.13), (1.1.14) and (1.1.15)), but in nonparametric models this will not always work. Theorem 1 of [START_REF] Castillo | Nonparametric bernstein-von mises theorems in gaussian white noise[END_REF] states that this works in the Gaussian White Noise model for fixed regularity, namely the credible set is built using the true regularity of the function which therefore assumed to be known. To get adaptive results, one often needs more conditions on the parameter to estimate, such as so-called polished-tails condition or self-similarity conditions. As seen in [START_REF] Szabó | Rejoinder to discussions of "frequentist coverage of adaptive nonparametric bayesian credible sets[END_REF], one will often need to use a blow up factor to ensure that the credible sets are confidence sets. Ray (2017) derive adaptive confidence sets from credible sets for the Gaussian White Noise model under a self-similarity condition and Spike and Slab priors.

One can use the following approach to quantify uncertainty via inflated credible balls.

Choose a consistent estimator θ of the parameter θ 0 and let r(X) = ˆ∥θ -θ∥ 2 dΠ(θ|X), which is the second posterior moment if one chooses θ = θ the posterior mean. The credible ball is defined as

C L = {θ, ∥θ -θ∥ 2 ≤ M τ Lr(X)}
with L ≥ 1 a blow-up factor. By Markov's inequality, one has Π(C L |X) ≥ 1 -τ as long as M τ ≥ 1/τ . One needs now to prove that this credible set is a confidence set (1.1.15) and has an optimal diameter (1.1.14), which is the same as proving that the second posterior moment is consistent at minimax rate if θ = θ. This approach has been used for instance in [START_REF] Castillo | Spike and slab empirical Bayes sparse credible sets[END_REF].

Tuning the parameters

In the Bayesian approach, it frequently happens that the a priori put on θ also depends on a parameter. In this section, we will assume that θ ∼ Π α , with α an additionnal parameter, which is often called a hyperparameter. One problem that arises is how to choose a decent value for α. Usually, one uses one of the two following methods to handle this problem.

Hierarchical Bayes. The first natural method is to adopt an even more Bayesian approach and consider the paramater α random and put a prior π on it. This results in the following Bayesian diagram

X|θ, α ∼ P θ θ|α ∼ Π α α ∼ π (1.1.23)
Even though one has to choose another a priori law, which may in turn depend on other parameters, the randomization it provides on α is often enough to correctly choose α in order to get optimal (or nearly optimal) rates in a majority of examples.

Empirical Bayes. Another natural idea is to choose α as α the maximiser of the marginal likelihood of the α in the model, namely the likelihood integrated over the entire space of parameters Θ. Simply put, this is the marginal distribution of α|X.

α = arg max α ˆΘ n k=1 f θ (X i ) π α (θ)dθ
(1.1.24)

One now uses this quantity α to form a prior by plugging α in Π α , resulting in the following diagram

X|θ ∼ P θ θ ∼ Π α (1.1.25)
These two methods are of prime interest in the following, especially the Empirical Bayes method.

Gaussian Sequence Model and Thresholding

Definition of the Model

We can write the Gaussian Sequence Model as follows, with X the observed vector of R n

X i = θ 0,i + ε i , i = 1, . . . n, (1.2.1)
where ε 1 , . . . , ε n are independent and identically distributed (iid) random variables following the N (0, 1) law (whose density will be denoted ϕ), and the parameter θ 0 = (θ 0,1 , . . . , θ 0,n ) belongs to the class ℓ 0 [s n ] defined by

ℓ 0 [s n ] = {θ ∈ R n , |{i ∈ {1, . . . n}, θ i ̸ = 0}| ≤ s n }, for 0 ≤ s n ≤ n,
where |A| is the number of elements in the set A.

One commonly assumes that

s n = o(n) when n → ∞. We denote by ∥ • ∥ the euclidean norm, ∥v∥ 2 = n i=1 v 2 i for v ∈ R n .
We are interested in finding estimators of θ 0 that converge to θ 0 at the minimax rate of the class ℓ 0 [s n ], which is, as proven in [START_REF] Donoho | Maximum entropy and the nearly black object[END_REF] Theorem 3 (Donoho,Hoch,Johnstone,Stern,1992). Let r n be the minimax rate for estimating θ in ℓ 0 [s n ] with respect to ∥.∥. Then,

r n = r n,2 (ℓ 0 [s n ]) = inf θ sup θ∈ℓ 0 [sn] 1 n n i=1 E θ ( θi -θ i ) 2 = 2s n n log( n s n ) (1 + o(1))
when n → ∞

For an estimator θ of θ 0 , it is then desirable that sup

θ 0 ∈ℓ 0 [sn] 1 n E θ 0 ∥ θ -θ 0 ∥ 2 2 ≤ C 2s n n log( n s n ) (1 + o(1)) , (1.2.2)
where C is a positive constant, that we ideally would like to be equal to 1 (but this could represent a lot of additional work on its own).

In fact, we are mostly interested in more general results for the entire a posteriori law, namely sup

θ 0 ∈ℓ 0 [sn] E θ 0 Π(∥θ -θ 0 ∥ 2 > 2Cs n log( n s n )|X) → 0 (1.2.3)
and for the posterior second moment sup

θ 0 ∈ℓ 0 [sn] E θ 0 ˆ∥θ -θ 0 ∥ 2 2 dΠ(θ|X) ≤ 2Cs n log( n s n )(1 + o(1)) (1.2.4)
The second moment will be a main focus in the following, as good results of convergence for the posterior second moment imply good results for the complete a posteriori law and lead to Uncertainty Quantification via inflated credible balls, as seen in 1.1.3. Also if ones has (1.2.4), the posterior mean (denoted by θ) will satisfy (1.2.2). Indeed, using the Jensen inequality, one has for every

θ 0 ∈ ℓ 0 [s n ] ∥ θ -θ 0 ∥ 2 2 = ∥ ˆθdΠ(θ|X) -θ 0 ∥ 2 2 ≤ ´∥θ -θ 0 ∥ 2 2 dΠ(θ|X)
which leads to sup

θ 0 ∈ℓ 0 [sn] 1 n E θ 0 ∥ θ -θ 0 ∥ 2 2 ≤ C 2s n n log( n s n ) (1 + o(1)).
The natural way to handle the sparsity of the model and produce consistant estimators is to use thresholding.

Thresholding. The first idea is to estimate θ by keeping the observations larger than some threshold t n , and set the remaining coordinates to zero, this is the hard thresholding estimator : θi = X i 1l {|X i |>tn} for i ∈ {1, . . . , n}.

One has then to choose the threshold t n . The oracle choice, namely if the maximum number of nonzero coordinates of the true signal s n is known, is t n = 2 log(n/s n ). It can be checked that θ concentrates around the true signal θ 0 at minimax rate. As s n is unknown, one can not choose this threshold. However, the choice t n = 2 log(n) provides a near-minimax rate, only missing the true minimax rate by a constant or a logarithmic factor. Such fixed thresholds are actually not flexible enough. Indeed, if one chooses a rather large t n but the true signal happens to be too dense, too much observations will be set to 0, and if one chooses a rather small t n but the true signal is too sparse, the estimator will keep too much observations. A good threshold should therefore adapt to the effective sparsity of the signal. Furthermore, one may also want the threshold to be stable to small changes of the data. We will see in what follows that a suitable (possibly empirical) choice of prior on θ leads to a thresholding estimator (the posterior median) which has a threshold with all these desirable properties.

Penalization and other frequentist methods. The hard thresholding estimator can in fact be viewed as an ℓ 0 -penalized estimator, which was introduced in the context of model selection (see for instance [START_REF] Birgé | Gaussian model selection[END_REF]). Another useful penalty is the ℓ 1 -norm of θ, which leads to the LASSO estimator.

The LASSO estimator is defined as follows θLASSO = argmin

θ∈R n { n i=1 (θ i -X i ) 2 + λ n i=1 |θ i |}
where λ ≥ 0 is the regularization parameter. The second term is called the ℓ 1 penalty and is what makes the LASSO work, as it allows the estimator to continuously shrink the coefficients. The larger λ the closer to 0 are the coefficients. The LASSO, which leads to a good prediction accuracy by providing to the user a bias-variance trade-off, has been largely studied over the years. Among many others, one can cite [START_REF] Tibshirani | Regression shrinkage and selection via the lasso[END_REF], [START_REF] Bickel | Simultaneous analysis of lasso and dantzig selector[END_REF], [START_REF] Zhang | General empirical bayes wavelet methods and exactly adaptive minimax estimation[END_REF] and [START_REF] Zou | The adaptive lasso and its oracle properties[END_REF]. Several other frequentist methods have been developed, one can cite, among many other methods, estimates based on False Discovery Rate thresholds (see [START_REF] Abramovich | Adapting to unknown sparsity by controlling the false discovery rate[END_REF]), who used the Benjamini and Hochberg threshold.

Bayesian approach and the Spike and Slab Prior

We will follow the approach introduced in 1.1, and view the parameter θ as a random variable following an a priori law that we now have to choose. The first natural law to think of may just be a product of Gaussian densities, for we know that this is a conjugate prior and that the a posteriori law will also be Gaussian. Let us try this prior and assume first that for every i ∈ {1, . . . , n}

θ i ∼ N (0, σ 2 i ),
After some quick computing, one finds out that the a posteriori law is also a product of Gaussian densities with updated means and variances so that, for every i ∈ {1, . . . , n}

θ i |X i ∼ N σ 2 i 1 + σ 2 i X i , σ 2 i 1 + σ 2 i
Therefore the posterior mean estimator is θmean = σ 2 1+σ 2 X. Now if for example one assumes that, for every i ∈ {1, . . . , n}, 1/2 ≤ σ 2 i ≤ 1, let us consider θ 0 = 0 ∈ ℓ 0 [s n ] and look at its quadratic risk

E θ 0 [∥ θmoy -θ 0 ∥ 2 ] = n i=1 σ 2 i 1 + σ 2 i 2 E θ 0 [X 2 i ] ≥ n 9
which is far from the minimax rate 2s n log( n sn ). This shows that this choice of prior does not properly take account of the sparsity of the model. This choice also faces issues for large signals. Indeed if one assumes that for every i ∈ {1, . . . , n}, σ 2 i = 1, the posterior mean becomes θmoy = X 2 , so if the real θ 0 has a first coordinate equal to 1000, the first coordinate of the estimator will be around 500. One sees that the estimator shrinks the signal too much to be useful, and it suggests that a density with heavier tails may also be useful.

A second idea is to use another continuous distribution and use a product of Laplace distributions instead. In this view we will now assume that for every i ∈ {1, . . . , n}

θ i ∼ Lap(0, λ).
The posterior density can be written as a constant times e

-1 2 n i=1 (θ i -X i ) 2 - n i=1 λ|θ i | so the posterior mode M is M = argmax θ∈R n {e -1 2 n i=1 (θ i -X i ) 2 - n i=1 λ|θ i | } = argmin θ∈R n { n i=1 (θ i -X i ) 2 + 2λ n i=1 |θ i |} = θLASSO
Therefore the posterior mode will show good consistency properties. But this represents only one aspect of the full a posteriori law, which has actually been shown in Theorem 7 in Castillo et al. (2015) to not contract at the same rate as its mode. Namely, for the standard choice λ = λ n = √ 2 log n the posterior distribution will not put any mass on balls around the true signal of radius √ n/ √ 2 log n. Thus this choice of prior is not very appropriate especially if one also aims at Uncertainty Quantification through the full posterior distribution.

Another idea that seems very natural and that will not use a continuous prior is to reflect the parcimonious nature of the model directly in the a priori law, which is done in the Spike and Slab prior.

The Spike and Slab Prior. Since the model is sparse, we already know that a certain number (in fact, most) of coordinates are equal to zero, the natural idea behind the Spike and Slab prior is to force some coordinates of θ to be equal to 0 and model the rest of the coordinates as an arbitrary signal (even possibly small).

θ ∼ Π α := n i=1 (1 -α)δ 0 + αΓ (1.2.5)
with δ 0 the Dirac in 0, Γ a probability law to be chosen which is absolutely continuous relatively to the Lebesgue measure and whose density will be noted γ, and α ∈ [0; 1] a parameter to be chosen too. Both because of their graphical representations, the part with the Dirac mass at 0 is called the Spike and the part with the density which is meant to have heavy tails is called the Slab. The closer α is to 0 the sparsier the model is, and one usually calls α the smoothing parameter.

Posterior Distribution. The a posteriori law is also a product. Indeed, writing

g = ϕ * γ Π α (θ|X) = n i=1 [(1 -α)δ 0 (θ i ) + αγ(θ i )]ϕ(X i -θ i ) (1 -α)ϕ(X i ) + αg(X i ) .

Introduction

Thus we obtain

Π α (θ|X) = n i=1 [(1 -a(X i ))δ 0 (θ i ) + a(X i )ψ X i (θ i )] (1.2.6) with a(X i ) = a α (X i ) = αg(X i ) (1 -α)ϕ(X i ) + αg(X i )
and the density

ψ X i = ϕ(X i -•)γ(•) g(X i )
Note that, for the moment, each θ i |X only depends on the observation X i and actually

L(θ i |X) = L(θ i |X i ).
Firstly, one has now to specify the choice of the parameters γ and α. If we first wish to choose the Slab density γ, one may want to use Gaussian densities.

Case where γ is N (0, σ 2 ). In that case, for every i ∈ {1, . . . , n}, ψ X i also is the density of a normal law, whose mean is σ 2 1+σ 2 X i and whose variance is σ 2 1+σ 2 .

Taking α = 2 in Theorem 2.8 of [START_REF] Castillo | Needles and straw in a haystack: posterior concentration for possibly sparse sequences[END_REF] shows that if the true signal has coordinates that are too large, the posterior distribution will asymptotically not put any mass around the true signal. This shows that choosing γ Gaussian is not suitable. In fact, the hypotheses of the following properties used by Johnstone and Silverman also exclude the Gaussian case.

Hypotheses on the Slab. Following [START_REF] Johnstone | Needles and straw in haystacks: empirical Bayes estimates of possibly sparse sequences[END_REF], we would like the density γ to have heavy enough tails, that is why we will choose in the following a standard Laplace density instead of a standard normal law. Precisely, one assumes that

sup u>0 | d du log γ(u)| = Λ < ∞ (1.2.7)
This gives us that, ∀u > 0, log γ(u) ≥ log γ(0) -Λu and therefore, ∀u > 0, γ(u) ≥ γ(0)e -Λ|u| , which prevents us from choosing a gaussian γ.

One will furthermore assume that u → u 2 γ(u) is bounded and that there exists

κ ∈ [1, 2] such that, when y → ∞ 1 γ(y) ˆ∞ y γ(u)du ≍ y κ-1 (1.2.8)
Properties of the coordinate-wise posterior median. Under these hypotheses, the posterior median, denoted by θmed , has the following properties Proposition 1 [START_REF] Johnstone | Needles and straw in haystacks: empirical Bayes estimates of possibly sparse sequences[END_REF]. The posterior median θmed = θmed (x, α) is an increasing function in x, antisymmetric and is a thresholding rule:

∀x ≥ 0 , 0 ≤ θmed (x, α) ≤ x.
Moreover it is a thresholding estimator: there exists t = t(α) > 0 (see below for more on this threshold) such that θmed (x, α) = 0 ⇔ |x| ≤ t(α).

(1.2.9)

It also has a bounded shrinkage property : there exists b > 0 such that, for t(α) as above

∀x, ∀α, | θmed (x, α) -x| ≤ t(α) + b. (1.2.10)
Link between α and the threshold t = t(α). We will see t as a function of α defined as follows t :

     (0, 1) -→ (0, +∞)
α -→ The threshold of the posterior median obtained with prior Π α

(1.2.11)

Using the following notation ∀x ∈ R, g

+ (x) = ˆ+∞ 0 ϕ(x -u)γ(u)du and g -(x) = ˆ0 -∞
ϕ(x -u)γ(u)du and recalling (1.2.6), we have

P (θ > 0|X = x) = αg + (x) (1 -α)ϕ(x) + αg(x) , so that 2αg + (t) = (1 -α)ϕ(t) + αg(t) and therefore 1 α = 1 + g + (t) -g -(t) ϕ(t) = 1 + 2 ˆ+∞ 0 sinh(tu)e -u 2 2 γ(u)du. (1.2.12)
This gives us a threshold which is a continuous function of α, decreasing from +∞ when α equals 0 to 0 when α equals 1.

One can further show that t(α) is of order 2 log(1/α) independently of the choice of the density γ (as long as γ satisfies (1.2.7) and (1.2.8)). One can refer back to Lemma 14 of [START_REF] Castillo | On spike and slab empirical Bayes multiple testing[END_REF] for an even finer result (as

ζ 2 (α) -C < t 2 (α) < ζ 2 (α)
Introduction as shown in ( 52) and ( 53) of [START_REF] Johnstone | Needles and straw in haystacks: empirical Bayes estimates of possibly sparse sequences[END_REF]).

As an oracle choice of threshold is 2 log(n/s n ), one sees that an oracle choice of α would be α * = s n /n. Since s n is still an unknown quantity, one may now ask how to properly choose α.

Choice of α.

First note that if one chooses α constant in (0, 1), the results will not be more satisfying than the case α = 1 which is a product of continuous densities. The intuition behind this is if one looks at the a priori law, the expected number of nonzero coordinates αn will be of order n, which is too high as we want it to be of order s n . One therefore has to make α depend on n and have it tend to 0 with n to be able to handle with the sparsity of the model properly.

Taking this into account, one can expect that choosing α of order 1/n will provide better results. Indeed, for a Spike and Slab prior with α = 1/n, it is shown in [START_REF] Mismer | Master's thesis[END_REF] (available on author's webpage) that the posterior law concentrates itself around the true signal θ 0 at a near-minimax rate : sn n log n (this is the correct rate (3) only up to a logarithmic factor).

For better results (both in theory and practice), one may consider an automatic procedure to select α, namely to use a Hierarchical Bayes or an Empirical Bayes approach.

To be even more Bayesian, one can use the Hierarchical method and consider α itself as a random variable, and put a prior on it. As α ∈ (0, 1), a natural prior is a Beta distribution. If α ∼ Beta(a, b), the Beta distribution of parameters a, b ∈ R + * which has as density b

(x) = x a-1 (1 -x) b-1 1l [0,1] (x)Γ(a + b)/(Γ(a)Γ(b))
, the expected number of nonzero coordinates for Π α is a a+b n. Ideally this number should be s n but as seen before with the choice α = 1/n reasonable rates can be achieved if this expected number is smaller than s n , which suggests that the quantity a a+b has to belong in (c 1 n , C sn n ). This suggests to take a small and b larger, and in this view one may take α ∼ β(1, n + 1), as in [START_REF] Castillo | Needles and straw in a haystack: posterior concentration for possibly sparse sequences[END_REF]. This choice leads to the minimax concentration of the corresponding posterior distribution, as can be seen in the paper of Castillo and van der Vaart (2012) (They actually derive a concentration result for a general prior in their Theorem 2.2, the Spike and Slab prior only being a special case treated in Example 2.2, more details on the general prior can be found in 1.2.3). Note that the rate of convergence has the right logarithm part log(n/s n ) even though we were not able to choose the first parameter equal to s n as it is unknown (this would have led to an expected number of nonzero coordinates of order s n ). This shows that the Hierarchical Bayes approach provides more flexibility than, for instance, just taking α = 1/n. Choosing α by Empirical Bayes. We will now introduce the Empirical Bayes approach for our Spike and Slab prior, which will be the main focus for the results presented in this document for the Gaussian Sequence model. The idea of [START_REF] Johnstone | Needles and straw in haystacks: empirical Bayes estimates of possibly sparse sequences[END_REF] is to estimate α by maximising the marginal likelihood in α in the Bayesian model, which is the density of α | X. The log-marginal likelihood in α can be written as

ℓ(α) = ℓ n (α; X) = n i=1 log((1 -α)ϕ(X i ) + αg(X i )).
(1.2.13)

Let α be defined as the maximiser of the log-marginal likelihood

α = argmax α∈An ℓ n (α; X), (1.2.14)
where the maximisation is restricted to

A n = [α n , 1], with α n defined, with t(α) as in (1.2.11), by t(α n ) = 2 log n.
The reason for this restriction is that one does not need to take α smaller than α n , which would correspond to a choice of α 'more conservative' than hard-thresholding at threshold level √ 2 log n. The a priori law that will be therefore considered is the Spike and Slab where we have 'plugged' the value α :

θ ∼ Π α := n i=1 (1 -α)δ 0 + αΓ (1.2.15)
One will also denote the threshold of our new 'plug-in' posterior median, recalling (1.2.11),

t = t(α) (1.2.16)
The first result obtained with this approach is the following, which shows that some point estimators derived from the Empirical Bayes a posteriori law converge to the true signal at minimax rate as appears in Theorem 3 Theorem 1.2.1 [START_REF] Johnstone | Needles and straw in haystacks: empirical Bayes estimates of possibly sparse sequences[END_REF]. Let μ be a thresholding rule (see (1.2.9)) with threshold t and with the bounded shrinkage property (see (1.2.10)). For n large enough we have, for C a large enough constant, and provided

s n ≥ log 2 n sup θ∈ℓ 0 [sn] 1 n E θ ∥μ -θ∥ 2 ≤ Cr n
With a slab γ verifying (1.2.7) and (1.2.8), the posterior median is a thresholding rule with the bounded shrinkage property. [START_REF] Johnstone | Needles and straw in haystacks: empirical Bayes estimates of possibly sparse sequences[END_REF] also prove that the result also holds for the posterior mean even though it only has the bounded shrinkage property. The parameter α obtained by Empirical Bayes is computationally very tractable, and the authors developped the package EBayesThresh to compute the quantities involved in their results.

Looking at this theorem, one can now ask whether the entire a posteriori law will also concentrate around the true signal at minimax rate. The focus will be put on the posterior second moment, for which one would like to derive results of the form sup

θ 0 ∈ℓ 0 [sn] E θ 0 ˆ∥θ -θ 0 ∥ 2 2 dΠ α(θ | X) ≤ Cr n (1.2.17)
This topic was one of the main interests of this work and is further addressed in 1.2.5.

Other choices of a priori laws

There are of course many other choices of a priori laws in this sparse setting, and this section aims to introduce a few of them.

Spike and Slab LASSO(SSL)

. [START_REF] Ročková | Bayesian estimation of sparse signals with a continuous spike-and-slab prior[END_REF] and [START_REF] Ročková | The spike-and-slab LASSO[END_REF] used a slightly different prior, which will also be considered further in this document. The idea is to replace the Dirac mass by a probability distribution to make the whole prior absolutely continuous relatively to the Lebesgue measure

θ ∼ Π α := n i=1 (1 -α)Γ 0 + αΓ 1 (1.2.18)
where the densities γ 0 and γ 1 are Laplace ((λ i /2) exp(-λ i |x|)) where parameters λ 0 and λ 1 serve very different purposes : the first is larger than the second, making the first density look like a Dirac (in a continuous way) and the second like a classical Slab. The Spike and Slab LASSO prior can be interpreted as linking the Spike and Slab prior and the frequentist LASSO, as the Spike and Slab is obtained by letting λ 0 → ∞ and the LASSO is obtained by setting λ 0 = λ 1 and considering the posterior mode. Note that in the SSL case, the posterior median is not a thresholding estimator anymore.

The modes of the a posteriori law are then well defined, and the mode is even unique as soon as (λ 1 -λ 0 ) 2 ≤ 4. [START_REF] Ročková | Bayesian estimation of sparse signals with a continuous spike-and-slab prior[END_REF] shows that the posterior global mode converges to the true signal at minimax rate with the oracle choice α = s n /(s n + n) (so this is not an adaptive result) as long as λ 1 < e -2 and for the choice λ 0 = n/s n + 4. However, the author shows an adaptive result in a particular regime for the entire posterior law using a Hierarchical approach, setting α ∼ β(1, 4n) and λ 0 = (1 -α)/α. The additional assumption on the signal is that all the nonzero coordinates have to be greater than (s n /n) log(n/s n ).

Horseshoe. The Horseshoe prior is a scale mixture of Gaussian distributions. It is the distribution (which has a density π) such that, ∀i ∈ {1, . . . , n}, with each λ i following the standard half-Cauchy on the positive reals law (denoted by C + (0, 1)) and τ a global hyperparameter,

θ i |λ i , τ ∼ N (0, λ 2 i τ 2 ) λ i ∼ C + (0, 1) (1.2.19)
The name Horsehoe, as stated by [START_REF] Carvalho | The horseshoe estimator for sparse signals[END_REF], comes from the fact that, with

κ i = 1 1+λ i , E[θ i |X] = ˆ1 0 (1 -κ i )X i Π(κ i |X)dκ i = (1 -E[κ i |X])X i
The quantity E[κ i |X] can be seen as the a posteriori amount of shrinkage towards 0. Since the λ i 's are half-Cauchy, each shrinkage coefficient κ i follows the β(1/2, 1/2) law, which has the shape of a horseshoe.

Its density π satisfies the following inequality, proven by Carvalho et al. ( 2010)

1 2τ log(1 + 4τ 2 θ 2 i ) ≲ π(θ i ) ≲ 1 τ log(1 + 2τ 2 θ 2 i ), θ i ̸ = 0
The density π therefore has a pole at zero and Cauchy tails, which makes the Horseshoe and the Spike and Slab (with Cauchy Slab) strikingly similar, and the parameter τ seems to play the same role as the parameter α of the Spike and Slab. van der Pas et al. (2017a) show adaptive near-minimax (without the log(n/s n ) part) rates of convergence for the posterior distribution for Empirical Bayes and Hierarchical Bayes under certain conditions. van der Pas et al. (2017b) give credible sets derived from the horseshoe posterior that can be used, asymptotically in n, as confidence sets.

A more general Prior. [START_REF] Castillo | Needles and straw in a haystack: posterior concentration for possibly sparse sequences[END_REF] use a more global a priori form, of which the Spike and Slab is just a particular case. The general prior is obtained through the following method

• Draw a dimension s using a law π(s) on the set {0, 1, . . . , n}

• Draw a support S ⊂ {1, . . . , n} uniformly on the sets of cardinal s : Π(S) = π(s)

( n s )

• This leads to the following prior on θ θ ∼ S⊂{1,...,n}

Π(S)    i∈S Γ ⊗ i / ∈S δ 0    (1.2.20)
with Γ probability distributions which are absolutely continuous relative to the Lebesgue measure and with density γ.

Note that in this general setting, the a posteriori does not shape as a product anymore, which can make the proofs harder. One says that π has exponential decrease if there exist C > 0 and D < 1 such that

π(s) ≤ Dπ(s -1) (1.2.21)
for s > Cs n . If this is satisfied with C = 0 then π is said to has strict exponential decrease.

One may now ask what to choose for the prior π on the dimension, here are some examples :

Binomial prior. If π is the binomial Bin(n, α), then the prior on θ is the Spike and Slab. This prior π has exponential decrease for α ≲ sn n . Hierarchical approach using a Beta prior. As before, we want s|α to follow Bin (n, α). In this aim take α ∼ Beta(κ, λ) and set

π(s) = n s β(κ + s, λ + n -s) Beta(κ, λ) ∝ Γ(κ + s)Γ(λ + n -s) s!(n -s)!
For κ = 1 and λ = n + 1, we have π(s) ∝ 2n-s n , which has strict exponential decrease with D = 1 2 . More generally and as seen before one can set κ = 1 and λ = κ 1 (n + 1), which leads to π(s)

∝ (κ 1 +1)n-s κ 1 n .
Complexity prior. This prior has the form π(s) ∝ e -as log( bn s ) . It shows to be quite fitting for the problem. Indeed, as e s log( n s ) ≤ n s ≤ e s log( en s ) , it is inversely proportional to the number of models of size s and seems good to lessen the complexity of the problem. It has exponential decrease as soon as b > 1 + e.

One is not able to use Theorem 2 as our observations X i are not i.i.d.. To get to their results, [START_REF] Castillo | Needles and straw in a haystack: posterior concentration for possibly sparse sequences[END_REF] first prove a result on the dimension : Theorem 4 [START_REF] Castillo | Needles and straw in a haystack: posterior concentration for possibly sparse sequences[END_REF]. If π has exponential decrease and γ is centered with a finite second moment, then there exists M > 0 such that n → ∞ :

sup θ 0 ∈ℓ 0 [sn] E θ 0 [Π(|S θ | > M s n |X)] → 0
This further leads to their main result Theorem 5 [START_REF] Castillo | Needles and straw in a haystack: posterior concentration for possibly sparse sequences[END_REF]. If π has exponential decrease and γ is centered with a finite second moment which can be written e h with h such that ∀x, y ∈ R,

|h(x) -h(y)| ≲ 1 + |x -y|, then, with r * n such that r * 2 n ≥ (s n log( n s n )) ∨ (log( 1 π(s n ) ))
and

M > 0 large enough, for n → ∞ sup θ 0 ∈ℓ 0 [sn] E θ 0 [Π(∥θ -θ 0 ∥ 2 > M r * 2 n |X)] → 0
The hypothesis on γ is verified for Laplace, and the 3 priors on dimension seen before in the examples (so including the Spike and Slab) verify the hypotheses of the theorem. Moreover, considering the complexity prior, the authors showed that the posterior mean converge to the true signal at minimax rate and that convergence of the second posterior moment is obtained.

There are several other Bayesian methods in the Gaussian sequence setting, such as non-local priors (as in [START_REF] Johnson | On the use of non-local prior densities in bayesian hvoothesis tests hypothesis[END_REF]), Gaussian mixture priors (see [START_REF] George | Calibration and empirical Bayes variable selection[END_REF]), or adopting a fractional likelihood perspective (see [START_REF] Martin | Asymptotically minimax empirical bayes estimation of a sparse normal mean vector[END_REF]).

Exact constant

In the setting of the sparse sequence model, we say that the posterior distribution converges at minimax rate with exact constant (or converges at sharp minimax rate) with respect to the L 2 -norm loss if sup

θ 0 ∈ℓ 0 [sn] E θ 0 ˆ∥θ -θ 0 ∥ 2 2 dΠ(θ|X) ≤ 2s n log( n s n )(1 + o(1)). (1.2.22)
This is (1.2.4) with the constant C = 1, implying that this is a finer result. The definition immediately implies using Jensen's inequality that the posterior mean (denoted here by θ) converges at minimax rate with exact constant to the true signal sup

θ 0 ∈ℓ 0 [sn] E θ 0 ∥ θ -θ 0 ∥ 2 2 ≤ 2s n log( n s n )(1 + o(1)). (1.2.23)
Another application is that if one uses a randomised estimator θ = θ(X, U ) using the data X and uniform variables U on [0, 1] to simulate from the a posteriori law, namely θ

such that L( θ(X, U )|X) = Π(•|X); stating (1.2.22
) is exactly stating the convergence to θ 0 at sharp minimax rate of θ. In the present setting, in order to effectively sample such a θ and have L( θ(X, U )|X) = Π(•|X), as the Spike and Slab a posteriori law is a product, one can take, denoting by

F θ i |X the cumulative distribution function of each θ i |X i , θ(X, U ) = (F -1 θ 1 |X (U 1 ), . . . , F -1 θn|X (U n ))
The convergence at minimax rate with exact constant for the Spike and Slab will require a specific choice of Slab, as will be seen in the following section.

Contributions using the Empirical Bayes method for the

Spike and Slab prior

The following work, which is treated in more details in Chapters 2 and 3, was motivated by pursuing the work seen in 1.2.2 of [START_REF] Johnstone | Needles and straw in haystacks: empirical Bayes estimates of possibly sparse sequences[END_REF]. Using the Empirical Bayes approach (1.2.14), they derived convergences at minimax rate for the posterior median and the posterior mean, as seen in Theorem 1.2.1, for suiting densities γ. Aiming at Uncertainty Quantification (which was later treated by [START_REF] Castillo | Spike and slab empirical Bayes sparse credible sets[END_REF] based on the present work), a natural question was to know if the second moment of the posterior law (1.2.15) behaved the same way. Namely, the form of the desired results is sup

θ 0 ∈ℓ 0 [sn] E θ 0 ˆ∥θ -θ 0 ∥ 2 2 dΠ α(θ | X) ≤ Cr n (1.2.24)
Suboptimality of the Laplace Slab. The first investigations were conducted with Γ taken as a standard Laplace distribution, and led to a quite surprising result. The posterior second moment for a Laplace Slab does not converge at minimax rate uniformly in θ ∈ ℓ 0 [s n ], even though the posterior median and mean do so (as was proved by [START_REF] Johnstone | Needles and straw in haystacks: empirical Bayes estimates of possibly sparse sequences[END_REF] and noted above) Theorem 6. Let Π α be the Spike and Slab prior distribution (1.2.5) with Slab distribution Γ equal to the Laplace distribution Lap(1). Let Π α[• | X] be the corresponding plug-in posterior distribution given by (1.2.15), with α chosen by the empirical Bayes procedure (1.2.14). There exist D > 0, N 0 > 0, and c 0 > 0 such that, for any n ≥ N 0 and any s n with 1 ≤ s n ≤ c 0 n, there exists θ 0 ∈ ℓ 0 [s n ] such that,

E θ 0 ˆ∥θ -θ 0 ∥ 2 2 dΠ α[θ | X] ≥ Ds n e √ log (n/sn) .
One can now ask whether this suboptimality result only comes from considering an integrated L 2 -moment, instead of simply asking for a posterior convergence result in probability like (1.2.3). It is actually not the case, as the entire a posteriori law is also suboptimal for the Laplace Slab.

Theorem 7. Under the same notation as in Theorem 6, if Π α is a Spike and Slab distribution with a slab Γ taken as the standard Laplace distribution, there exists m > 0 such that for any s n with s n /n → 0 and log

2 n = O(s n ) as n → ∞, there exists θ 0 ∈ ℓ 0 [s n ] such that, as n → ∞, E θ 0 Π α ∥θ -θ 0 ∥ 2 2 ≤ ms n e √ 2 log (n/sn) | X = o(1).
This is a stronger result than Theorem 6, but with an additional mild condition s n ≳ log 2 n. The fact that this result implies the preceding one follows from bounding from below the integral in the display of Theorem 6 by the integral restricted to the set where ∥θ -θ 0 ∥ 2 is larger than the target lower bound rate. The intuition behind these two results is that the Empirical Bayes provides (for some specific signals) a parameter α somewhat larger than the oracle parameter α * = s n /n (here α ≳ sn n e √ log (n/sn) ). One sees through this example that the behaviour of some aspects of an a posteriori law (such as the median or the mean) does not drive the behaviour of the complete a posteriori law.

One can also note that this is an example where the Empirical Bayes and the Hierarchical Bayes methods deliver different results, as the a posteriori law in the Hierarchical approach with a Laplace Slab does not show any suboptimality and converges at minimax rate, as seen in Theorem 5.

Theorem 8. Let Π α be the Spike and Slab prior distribution (1.2.5) with Slab distribution Γ equal to the standard Cauchy distribution. Let Π α[• | X] be the corresponding plug-in posterior distribution given by (1.2.15), with α chosen by the empirical Bayes procedure (1.2.14). There exist C > 0, N 0 > 0, and c 0 , c 1 > 0 such that, for any n ≥ N 0 , for any s n such that there exist constant c 0 , c 1 such that c 1 log 2 n ≤ s n ≤ c 0 n,

sup θ 0 ∈ℓ 0 [sn] E θ 0 ˆ∥θ -θ 0 ∥ 2 2 dΠ α(θ | X) ≤ Cr n .
Actually, any Slab density γ with tails of the order x -1-δ with δ ∈ (0, 2) gives the same result. These densities are particularly suitable if one wants to consider d q -distances instead of the d 2 -distance (see [START_REF] Castillo | Spike and slab empirical Bayes sparse credible sets[END_REF]).

This result shows once again that heavy tails are crucial to make the Empirical Bayes method succeed and get minimax results.

Sharp minimax convergence.

To go even further and get the exact constant 2 in the minimax rate, we use the special Slab density γ on R given by

γ(x) = 1 2 ∆(1 + |x|), ∆(u) = u -1 (1 + log(u)) -2 for u > 0, ( 1
0 , c 1 such that c 1 log 2 n ≤ s n ≤ c 0 n, for n → ∞ sup θ 0 ∈ℓ 0 [sn] E θ 0 ˆ∥θ -θ 0 ∥ 2 2 dΠ α(θ | X) ≤ 2s n log( n s n )(1 + o(1)
).

An intuition for why it works is that one may decompose the L 2 -norm in two parts, depending on whether the components of the true signal are different from zero or not. The nonzero signal part contributes for 2s n log( n sn )(1 + o( 1)). The other part is more dependent on the choice of the Slab. This part indeed depends on α, which is too far from the oracle parameter α * = s n /n in the Laplace case, resulting in a zero signal contribution larger than the minimax rate. In the Cauchy case (actually also with tails

x -1-δ , δ ∈ (0, 2)), the zero signal contribution appears to be exactly of the order of the minimax rate. With the special Slab (1.2.25), this contribution becomes lower than the minimax rate, finally resulting in 2s n log( n sn )(1 + o(1)).

Results for the Spike and Slab LASSO prior (SSL). Deriving analog results for the SSL prior, which is the continuous counterpart to the Spike and Slab, was also of particular interest. The prior on θ is the following

θ ∼ Π α := n i=1 (1 -α)Γ 0 + αΓ 1 , (1.2.26)
with Γ 0 a Laplace distribution with parameter λ 0 , but here we will not restrict the choice of Γ 1 to a Laplace. As seen in section 1.2.3, it is convenient to let λ 0 depend on n, here we set, mostly for more convenience in the proofs (see Chapter 2)

λ 0 = 5n √ 2π (1.2.27)
Let α be defined as the maximiser of the log-marginal likelihood α = argmax α∈An ℓ n (α; X), (1.2.28)

where the maximisation is restricted to A n = [α n , 1], with α n defined, in view of (1.2.11), by

t(α n ) = 2 log n.
The a priori law that will be therefore considered is the Spike and Slab where we have 'plugged' the value α :

θ ∼ Π α := n i=1 (1 -α)δ 0 + αΓ (1.2.29)
Theorem 10. Let Π α be the SSL prior distribution (1.2.26) with Cauchy slab and parameters λ 0 given by (1.2.27) and λ 1 = 0.05. Let Π α[• | X] be the corresponding plug-in posterior distribution given by (1.2.29), with α chosen by the Empirical Bayes procedure (1.2.28). There exist C > 0, N 0 > 0, for any n ≥ N 0 , for any s n such that there exist constant c 0 , c 1 such that c 1 log 2 n ≤ s n ≤ c 0 n, then sup

θ 0 ∈ℓ 0 [sn] E θ 0 ˆ∥θ -θ 0 ∥ 2 dΠ α(θ | X) ≤ Cs n log n.

Density Estimation and Pólya Trees

Definition of the Model

We can write the Density Estimation Model as follows, with X the observed vector of R n X 1 , . . . , X n i.i.d. ∼ P (1.3.1)

where P belongs to the model P = {P ; dP = f dµ} with µ the Lebesgue measure on [0, 1].

The goal here is to estimate the true density function f 0 . We make the two following assumptions

f 0 is bounded away from 0 and ∞ (1.3.2) ∃α ∈ (0, 1] such that f 0 ∈ C α ([0, 1]) (1.3.3) where C α ([0, 1]) = {f : [0, 1] → R; sup x̸ =y∈[0,1] |f (x) -f (y)| |x -y| α < ∞} is the set of α-Hölder
functions of [0, 1]. One would like to estimate f 0 in an adaptive way, namely a way that does not depend on the unknown parameter α.

Minimax rate. As proven by [START_REF] Ibragimov | An estimate of the density of a distribution[END_REF], the minimax rate when estimating densities in C α ([0, 1]) using the supremum norm as the loss function is

ε * n,α = log n n α 2α+1
.

(1.3.4)

Haar Basis. The Haar wavelet basis is {ϕ,

ψ lk , 0 ≤ k < 2 l , l ≥ 0}, where ϕ = 1l [0,1] and, for ψ = -1l (0,1/2] + 1l (1/2,1] , ψ lk (•) = 2 l/2 ψ(2 l • -k), 0 ≤ k < 2 l , l ≥ 0 (1.3.5)
As we focus on density functions on [0, 1], which are nonnegative functions g such that ˆ1 0 gϕ = ˆ1 0 g = 1, the first Haar-coefficient is always 1. That means that one only needs to consider the basis functions ψ lk and the Haar basis will simply be denoted as {ψ lk } in the following.

If a function g belongs to C α ([0, 1]), with α ∈ (0, 1], then the sequence of its Haar wavelet coefficients ⟨g, ψ lk ⟩ satisfies sup

0≤k<2 l ,l≥0 2 l(1/2+α) |⟨g, ψ lk ⟩| < ∞.
(1.3.6)

Bayesian approach. To adopt a Bayesian perspective, one has to put a prior on P , therefore one has to build a probability distribution on probability distributions. A rather common choice would be a Dirichlet process, but as its draws are discrete almost surely it will not be suitable to estimate objects as smooth as a density. A more convenient distribution on distributions with densities is the Pólya tree, which is introduced in what follows. (Actually the Dirichlet process is a particular case of Pólya tree, but with the α ε going to 0, see below)

The Pólya Tree Prior

Dyadic partitions. For any fixed indexes l ≥ 0 and 0 ≤ k < 2 l , the rational number r = k2 -l can be written in a unique way as ε(r) := ε 1 (r) . . . ε l (r), its finite expression of length l in base 1/2 (note that it can end with one or more 0). That is, ε i ∈ {0, 1} and

k2 -l = l i=1 ε i (r)2 -i .
Let E := l≥0 {0, 1} l ∪ {∅} be the set of finite binary sequences. We write |ε| = l if ε ∈ {0; 1} l and |∅| = 0. Let us introduce a sequence of partitions I = {(I ε ) |ε|=l , l ≥ 0} of the unit interval. Set I ∅ = (0, 1] and, for any ε ∈ E such that ε = ε(l; k) is the expression in base 1/2 of k2 -l , set

I ε := k 2 l , k + 1 2 l := I l Introduction Y ε0 ⨿ Y ε ′ 0 (1.3.7) Y ε0 ∼ Beta(α ε0 , α ε1 ) (1.3.8) Y ε1 = 1 -Y ε0
(1.3.9)

P (I ε ) = |ε| i=1 Y ε 1 ...ε i (1.3.10)
One can then use a tree representation (see Figure 1.1) to visually compute P (I ε ). One follows the path ε 1 , ε 1 ε 2 , . . . , ε 1 ε 2 . . . ε |ε|-1 , ε alongside ε, resulting in a product of Beta variables with parameters depending on whether one goes left on the tree (ε j = 0) or right (ε j = 1) This defines a random probability distribution on the distributions of [0, 1], so that the Pólya tree can be used as an a priori law on P in the Density Estimation Model. The Pólya tree prior has a conjugacy property, namely if one observes i.i.d. X 1 , . . . , X n following a probability distribution P itself following a P T (A) on I, the a posteriori law of P |X 1 , . . . , X n is also a Pólya tree P T (A * ), where

P (I ε ) = |ε| j=1,ε j =0 Y ε 1 ,...,ε j-1 0 × |ε| j=1,ε j =1 (1 -Y ε 1 ,...,ε j-1 0 ) (1.3.11)
A * = {α * ε = α ε + n i=1 1l {X i ∈Iε} , ε ∈ E}.
A proof of this result can be found in the book of [START_REF] Ghosal | Fundamentals of Nonparametric Bayesian Inference[END_REF].

The set of parameters A offers a large variety of choices, which leads to a large variety of different Pólya trees. However, the most common choice is to take the same parameters α ε at each level. In the following, for any level l ≥ 1, one takes

∀ε ∈ E such that |ε| = l, α ε = a l (1.3.12)
In other words, one chooses in the following A = (a l ) l≥1 a sequence of positive numbers. Note that if one takes a l = 2 -l , the corresponding Pólya tree is a Dirichlet process (see [START_REF] Ferguson | A bayesian analysis of some nonparametric problems[END_REF]).

As shown by [START_REF] Kraft | A class of distribution function processes which have derivatives[END_REF], if on the contrary one chooses a l tending to ∞ as l → ∞, more precisely if

∞ l=1 a -1 l < ∞ (1.3.13)
the corresponding Pólya tree on the canonical dyadic partition on [0, 1] is absolutely continuous relatively to the Lebesgue measure on [0, 1]. Therefore one will assume (1.3.13) in what follows.

For more details on Pólya trees, one can refer to [START_REF] Lavine | Some aspects of polya tree distributions for statistical modelling[END_REF] or [START_REF] Mauldin | Polya trees and random distributions[END_REF].

One may note that, unlike other classical estimators such as kernel estimators (in case the kernel takes negative values) or wavelet density estimators, Pólya tree priors always sit on densities, so that the posterior is itself automatically a density. Furthermore, as we will see below, there is a natural way to equip the prior with a natural built-in choice of the regularity hyperparameter, which will allow for adaptive inference. (1.3.15) with parameters α ε ∈ N to be chosen and a real parameter π ε (later to be taken of the form 2 -l2 e -Cl , where we wrote l = |ε|). There are multiple probability distributions on Borelians of [0, 1] that coincide on dyadic intervals I ε with P (I ε ) resulting from the above construction. We consider the specific one that is absolutely continuous relatively to the Lebesgue measure on [0, 1] with a constant density on each I ε , |ε| = L + 1. So, both prior and posterior are histograms on dyadic intervals at depth L.

Introduction ε ∈ E , Y ε0 ∼ (1 -π ε0 )δ 1 2 + π ε0 Beta(α ε0 , α ε1 ),
Definition. The prior distribution with parameters α ε , π ε , as above is called Spike and Slab Pólya tree and denoted Π(α ε , π ε ). This prior is based on an idea of Ghosal and van der Vaart, which is referred as Evenly Split Pólya tree in their book [START_REF] Ghosal | Fundamentals of Nonparametric Bayesian Inference[END_REF]. First note that the Haar coefficients f lk of a density f can be expressed as

f lk = ⟨f, ψ lk ⟩ = 2 l 2 P (I ε )(1 -2Y ε0 ) (1.3.16)
The Spike and Slab Pólya tree can therefore be seen as a 'thresholding prior', as the thresholding takes place on the sequence of Haar coefficients of the function where

Y ε0 = 1 2 .
Using this Spike and Slab prior can be seen as taking a Hierarchical approach. The usual Pólya tree (PT) prior on densities (under (1.3.13)) leads to the following Bayesian diagram

X|f ∼ f f ∼ P T ((Y ε0 )) with Y ε0 ∼ Beta(α ε0 , α ε1 ),
so the Y ε0 have fixed (Beta) distributions, whereas the Spike and Slab Pólya tree (SSPT) prior leads to the diagram

X|f ∼ f f ∼ SSP T ((Y ε0 )) with Y ε0 ∼ (1 -π ε0 )δ1 X|f ∼ f f |(γ ε0 ) ∼ SSP T ((Y ε0 )) with Y ε0 ∼ (1 -γ ε0 )δ1 2 + γ ε0 Beta(α ε0 , α ε1 ) γ ε0 ∼ Be(π ε0 )
So in this case the distributions followed by the Y ε0 are random, hence this approach can be viewed as hierarchical.

The a posteriori law. Proposition 6 of Chapter 4 states that the Spike and Slab type Pólya tree still satisfies conjugacy. Indeed, for every ε ∈ E, the a posteriori law of Y ε0 knowing X 1 , . . . , X n is

Y ε0 |X ∼ (1 -πε0 )δ1 2 + πε0 Beta(α ε0 (X), α ε1 (X)) (1.3.17)
where the quantities πε , T = T (ε, X) and α ε (X) all depend on the observations. Note that if π ε = 1, meaning that the prior is also a product of Beta variables, we get that the posterior is a product of Beta variables too.

An adaptive concentration result. The following Theorem shows that the a posteriori law obtained with a Spike and Slab type Pólya tree prior concentrates around the true density f 0 at minimax rate for the supremum-norm loss.

Theorem 11. Let f 0 ∈ C α [0, 1], for α ∈ (0, 1] and suppose ∥ log f 0 ∥ ∞ < ∞. Let X 1 , .
. . , X n be i.i.d. random variables on [0, 1] following P f 0 . Let Π be the prior on densities induced by a Spike and Slab Polya Tree prior Π(α ε , π ε ) with the choices

α ε = a π ε = 2 -l 2 e -κl , l = |ε|
for κ large enough constant and a > 0 constant. Then for any

M n → ∞, in P f 0 -probability Π   ∥f -f 0 ∥ ∞ ≤ M n log n n α 2α+1 | X   → 1
This theorem is an adaptive version of Theorem 1 of [START_REF] Castillo | Polya tree posterior distributions on densities[END_REF]. There are few results so far in the literature in density estimation for the supremum-norm loss, among those are the results from [START_REF] Castillo | On bayesian supremum norm contraction rates[END_REF][START_REF] Hoffmann | On adaptive posterior concentration rates[END_REF] and [START_REF] Yoo | Supremum norm posterior contraction and credible sets for nonparametric multivariate regression[END_REF] for multivariate regression.

A Bernstein Von Mises result.

To establish a nonparametric Bernstein Von Mises (BVM) result, one has first to find a space M 0 large enough to have convergence at rate √ n of the posterior density to a Gaussian process. One can then derive results for some other space F using continuous mapping for continuous functionals ψ : M 0 → F. A space that combines nicely with supremum norm structure was introduced by Castillo and Nickl (2014) and defined as follows, using an 'admissible' sequence ω

= (ω l ) l≥0 such that ω l / √ l → ∞ as l → ∞ M 0 = M 0 (ω) = x = (x lk ) l,k ; lim l→∞ max 0≤k<2 l |x lk | ω l = 0 (1.3.18) Equipped with the norm ∥x∥ M 0 = sup l≥0 max 0≤k<2 l |x lk | ω l
, this is a separable Banach space. In a slight abuse of notation, we will write f ∈ M 0 if the sequence of its Haar wavelet coefficients belongs in that space : (⟨f, ψ lk ⟩) l,k ∈ M 0 .

P -white bridge process. For P a probability distribution in [0, 1], one defines the P -white bridge process, denoted by G P . This is the Gaussian process indexed by the

Hilbert space L 2 (P ) = {f : [0, 1] → R; ˆ1 0 f 2 dP < ∞} with covariance E[G P (f )G P (g)] = ˆ1 0 (f - ˆ1 0 f dP )(g - ˆ1 0 gdP )dP (1.3.19)
We will denote by N the law of G P 0 (with P 0 = P f 0 ). The main purpose of the admissible sequence ω is to ensure that G P ∈ M 0 . Intuitively, if one does not use these weights w l , the maximum over 2 l Gaussian variables is of order 2 log(2 l ) = C √ l and does not tend to 0 as l → ∞, see Remark 1 of [START_REF] Castillo | On the bernstein von mises phenomenon for nonparametric bayes procedures[END_REF] for a precise proof of this result.

Recentering the distribution. To establish our BVM result, one also has to find a suitable way to center the posterior distribution. In this view, denote by P n the empirical measure

P n = 1 n n i=1 δ X i . (1.3.20)
Let us also consider C n , which is a smoothed version of P n , defined by

⟨C n , ψ lk ⟩ =    ⟨P n , ψ lk ⟩ if l ≤ L 0 if l > L, (1.3.21)
where L is our original cutoff, defined by (4.1.5).

We finally introduce T n , which depends on the true parameter α, defined by

⟨T n , ψ lk ⟩ =    ⟨P n , ψ lk ⟩ if l ≤ L n 0 if l > L n , (1.3.22)
where we defined L n to be the integer such that

2 Ln = ⌊c 0 n log n 1 1+2α ⌋ (1.3.23)
for a suitable constant c 0 ∈ R + * , whose precise value is made clear below.

Weak BVM result. We have the following Bernstein-von Mises phenomenon for f 0 in Hölder-type balls (standard Hölder balls are subsets of the following ones)

H(α, R) := {f = (f lk ) : |f lk | ≤ R2 -(α+1/2) , ∀l ≥ 0, 0 ≤ k < 2 l }
Theorem 12. Let N be the law of G P 0 . Let C n be the centering defined in (4.2.6). Let l 0 (n) be an increasing and diverging sequence. We define the prior Π such that

Y ε0 ∼ Beta(a, a) for |ε| ≤ l 0 Y ε0 ∼ (1 -π ε0 )δ1 2 + π ε0 Beta(a, a) for l 0 < |ε| ≤ L
where π ε = 2 -l 2 e -κ|ε| with κ a large enough constant. The posterior distribution then satisfies a weak BvM : for every α, R > 0, recalling β S from (1.1.20), sup

f 0 ∈H(α,R) E f 0 β M 0 (ω) (Π(•|X) • τ -1 Cn , N ) → 0 as n → ∞ and for any admissible sequence ω = (ω l ) with ω l 0 (n) / log(n) → ∞.
The choice of recentering of the distribution is quite flexible, as it can be checked that the result also holds if one replaces C n by the posterior mean fn or by T n which depends on α. Actually, the only required condition on where one cuts the empirical measure is to satisfy Theorem 1 of [START_REF] Castillo | On the bernstein von mises phenomenon for nonparametric bayes procedures[END_REF]. One can see that the cutoff L is exactly the furthest one can go according to that theorem.

Using the methods of [START_REF] Castillo | On the bernstein von mises phenomenon for nonparametric bayes procedures[END_REF], this result leads to several applications, for instance derivation of BVM theorems for semiparametric functionals via the continuous mapping theorem and Donsker-type theorems, which do not appear here for the sake of

Chapter 2

Empirical Bayes analysis of spike and slab posterior distributions

Introduction

In the sparse normal means model, one observes a sequence X = (X 1 , . . . , X n )

X i = θ i + ε i , i = 1, . . . , n, (2.1.1)
with θ = (θ 1 , . . . , θ n ) ∈ R n and ε 1 , . . . , ε n i.i.d. N (0, 1). Given θ, the distribution of X is a product of Gaussians and is denoted by P θ . Further, one assumes that the 'true' vector

θ 0 belongs to ℓ 0 [s n ] = {θ ∈ R n , #{i : θ i ̸ = 0} ≤ s n } ,
the set of vectors that have at most s n nonzero coordinates, where 0 ≤ s n ≤ n. A typical sparsity assumption is that s n is a sequence that may grow with n but is 'small' compared to n (e.g. in the asymptotics n → ∞, one typically assumes s n /n = o(1) and

s n → ∞). A natural problem is that of estimating θ with respect to the euclidean loss ∥θ -θ ′ ∥ 2 = n i=1 (θ i -θ ′ i ) 2 .
A benchmark is given by the minimax rate for this loss over the class of sparse vectors ℓ 0 [s n ]. Denoting [START_REF] Donoho | Maximum entropy and the nearly black object[END_REF] show that the minimax rate equals (1 + o(1))r n as n → ∞.

r n := 2s n log(n/s n ),
Taking a Bayesian approach, one of the simplest and arguably most natural classes of prior distributions in this setting is given by so-called spike and slab distributions,

θ ∼ n i=1 (1 -α)δ 0 + αG,
where δ 0 denotes the Dirac mass at 0, the distribution G has density γ with respect to Lebesgue measure and α belongs to [0, 1]. These priors were introduced and advocated in a number of papers, including [START_REF] Mitchell | Adaptive bernstein von mises theorems in gaussian white noise[END_REF]; [START_REF] George | The variable selection problem[END_REF]; [START_REF] George | Calibration and empirical Bayes variable selection[END_REF]; [START_REF] Yuan | Efficient empirical Bayes variable selection and estimation in linear models[END_REF]. One important point is the calibration of the tuning parameter α, which can be done in a number of ways, including: deterministic n-dependent choice, data-dependent choice based on a preliminary estimate α, fully Bayesian choice based on a prior distribution on α. Studying the behaviour of the posterior distributions in sparse settings is currently the object of a lot of activity. A brief (and by far not exhaustive) overview of recent works is given below. Given a prior distribution Π on θ, and interpreting P θ as the law of X given θ, one forms the posterior distribution Π[• | X] which is the law of θ given X. The frequentist analysis of the posterior distribution consists in the study of the convergence of Π[• | X] in probability under P θ 0 , thus assuming that the data has actually been generated from some 'true' parameter θ 0 .

In the present paper, we follow this path and are more particularly interested in obtaining a uniform bound on the posterior squared L 2 -moment of the order of the optimal minimax rate, that is in proving, with C a large enough constant, sup

θ 0 ∈ℓ 0 [sn] E θ 0 ˆ∥θ -θ 0 ∥ 2 dΠ(θ | X) ≤ Cr n (2.1.2)
for Π a prior distribution constructed using a spike and slab approach, whose prior parameters may be calibrated using the data, that is following an empirical Bayes method. This is of interest for at least three reasons

• this provides adaptive convergence rates for the entire posterior distribution, using a fully data-driven procedure. This is more than obtaining convergence of aspects of the posterior such that posterior mean or mode, and in fact may require different conditions on the prior, as we shall see below.

• the inequality (2.1.2) automatically implies convergence of several commonly used point estimators derived from the posterior Π[• | X]: it implies convergence at rate

Cr n of the posterior mean ´θdΠ(θ | X) (using Jensen's inequality, see e.g. [START_REF] Castillo | Needles and straw in a haystack: posterior concentration for possibly sparse sequences[END_REF]), but also of the coordinatewise posterior median (see the supplement of [START_REF] Castillo | Needles and straw in a haystack: posterior concentration for possibly sparse sequences[END_REF] for details) and in fact of any fixed posterior coordinatewise quantile, for instance the quantile 1/4 of Π[• | X]. It also implies, using Tchebychev's inequality, convergence of the posterior distribution at rate M n r n for ∥ • ∥ 2 as in (2.1.3) below with M = M n , for any M n → ∞.

• knowing (2.1.2) is a first step towards results for uncertainty quantification, in particular for the study of certain credible sets. Indeed, (2.1.2) suggests a natural way to build such a set, that is

C ⊂ R n with Π[C | X] ≥ 1 -α for a given α ∈ (0, 1).
Namely, define C = {θ : ∥θ -θ∥ 2 ≤ r X }, with θ the posterior mean (or another suitable point estimate of θ) and r X a large enough multiple of the (1

-α)-quantile of ´∥θ -θ∥ 2 dΠ(θ | X).
The present work is the first of a series of papers where we study aspects of inference using spike and slab prior distributions. In particular, based on the present results, the behaviour of the previously mentioned credible sets is studied in the forthcoming paper [START_REF] Castillo | Spike and slab empirical Bayes sparse credible sets[END_REF].

Previous results on frequentist analysis of spike and slab type priors. In a seminal paper, Johnstone and Silverman [START_REF] Johnstone | Needles and straw in haystacks: empirical Bayes estimates of possibly sparse sequences[END_REF] considered estimation of θ using spike and slab priors combined with an empirical Bayes method for choosing α. They chose α = α based on a marginal maximum likelihood approach to be described in more details below. Denoting θ the associated posterior median (or posterior mean), [START_REF] Johnstone | Needles and straw in haystacks: empirical Bayes estimates of possibly sparse sequences[END_REF] established that sup

θ 0 ∈ℓ 0 [sn] E θ 0 ∥ θ -θ 0 ∥ 2 ≤ Cr n ,
thereby proving minimaxity up to a constant of this estimator over ℓ 0 [s n ]. The estimator is adaptive, as the knowledge of s n is not required in its construction.

In [START_REF] Castillo | Needles and straw in a haystack: posterior concentration for possibly sparse sequences[END_REF], convergence of the posterior distribution is studied in the case α is given a prior distribution. If α ∼ Beta(1, n + 1), Π is the corresponding hierarchical prior, and Π[• | X] the associated posterior distribution, it is established in [START_REF] Castillo | Needles and straw in a haystack: posterior concentration for possibly sparse sequences[END_REF] that for large enough M , as n → ∞,

sup θ 0 ∈ℓ 0 [sn] E θ 0 Π[∥θ -θ 0 ∥ 2 ≤ M r n | X] → 1.
(2.1.3)

In [START_REF] Martin | Asymptotically minimax empirical bayes estimation of a sparse normal mean vector[END_REF], Martin and Walker use a fractional likelihood approach to construct a certain empirical Bayes spike and slab prior, where the idea is to reweight the standard spike and slab prior by a power of the likelihood. They derive rate-optimal concentration results for the corresponding posterior distribution and posterior mean.

A related class of prior distributions recently put forward by Ročková Ročková (2018) and [START_REF] Ročková | The spike-and-slab LASSO[END_REF], is given by

θ ∼ n i=1 (1 -α)G 0 + αG 1 ,
where both distributions G 0 , G 1 have densities with respect to Lebesgue measure. The authors in particular consider the choices G 0 = Lap(λ 0 ) and G 1 = Lap(λ 1 ), where Lap(λ) denotes the Laplace (double-exponential) distribution. Taking λ 0 large enough enables one to mimic the spike of the standard spike and slab prior, and the fact that both G 0 , G 1 are continuous distributions offers some computational advantages, especially when working with the posterior mode. One can also note that the posterior mode when α = 1 leads to the standard LASSO estimator. For this reason, the authors in [START_REF] Ročková | Bayesian estimation of sparse signals with a continuous spike-and-slab prior[END_REF]; [START_REF] Ročková | The spike-and-slab LASSO[END_REF] call this prior the spike and slab LASSO prior. It is shown in [START_REF] Ročková | Bayesian estimation of sparse signals with a continuous spike-and-slab prior[END_REF], Theorem 5.2 and corollaries, that a certain deterministic n-dependent choice of α, λ 0 , λ 1 (but independent on the unknown s n ) leads to posterior convergence at near-optimal rate s n log n, while putting a prior on α can yield [START_REF] Ročková | Bayesian estimation of sparse signals with a continuous spike-and-slab prior[END_REF], Theorem 5.4) the minimax rate for the posterior, if a certain condition on the strength of the true non-zero coefficiencents of θ 0 is verified.

Other priors and related work. We briefly review other options to induce sparsity using a Bayesian approach. One option considered in [START_REF] Castillo | Needles and straw in a haystack: posterior concentration for possibly sparse sequences[END_REF] is first to draw a subset S ⊂ {1, . . . , n} at random and then to draw nonzero coordinates on this subset only. That is, sample first a dimension k ∈ {0, . . . , n} at random according to some prior π. Given k, sample S uniformly at random over subsets of size k and finally set

θ i ∼ G i ∈ S θ i = 0 i / ∈ S.
Under the assumption that the prior π on k is of the form, referred to as the complexity prior,

π(k) = ce -ak log(nb/k) , (2.1.4)
Castillo and van der Vaart (2012) show that under this prior, both (2.1.3) and (2.1.2) are satisfied. However, such a 'strong' prior on the dimension is not necessary at least for (2.1.3) to hold: it can be checked for instance, for π the prior on dimension induced by the spike and slab prior on

θ with α ∼ Beta(1, n + 1), that π(s n ) ≍ exp(-cs n ) ≫ exp(-cs n log(n/s n ))
. So in a sense the complexity prior 'penalises slightly more than necessary'.

Another popular way to induce sparsity is via the so-called horseshoe prior, which draws a θ from a continous distribution which is itself a mixture. As established in van der Pas et al. (2017a)-van der Pas et al. (2017b) the horseshoe yields the nearly-optimal rate s n log n uniformly over the whole space ℓ 0 [s n ], up again to the correct form of the logarithmic factor. In a different spirit but still without using Dirac masses at 0, the paper [START_REF] Jiang | General maximum likelihood empirical Bayes estimation of normal means[END_REF] shows that, remarkably, it is also possible to adopt an empirical Bayes approach on the entire unknown distribution function F of the vector θ, interpreting θ as sampled from a certain distribution, and the authors derive oracle results over ℓ p , p > 0, balls for the plug-in posterior mean (not including the case p = 0 though). We also note the interesting work van der Pas et al. ( 2016) that investigates necessary and sufficient conditions for sparse continuous priors to be rate-optimal. However the latter is for a fixed regularity parameter s n , while the results decribed in Section 2.2 (in particularity the suboptimality phenomenon, but also upper-bounds using the empirical Bayes approach) are related to adaptation.

Using complexity-type priors on the number of non-zero coordinates, Belitser and co-authors [START_REF] Babenko | Oracle convergence rate of posterior under projection prior and Bayesian model selection[END_REF][START_REF] Babenko | Oracle convergence rate of posterior under projection prior and Bayesian model selection[END_REF][START_REF] Belitser | Needles and straw in a haystack: robust empirical Bayes confidence for possibly sparse sequences[END_REF] consider Gaussian priors on non-zero coefficients, with a recentering of the posterior mean at the observation X i -for those coordinates i that are selected-to adjust for overshrinkage. In [START_REF] Belitser | Needles and straw in a haystack: robust empirical Bayes confidence for possibly sparse sequences[END_REF], oracle results for the corresponding posterior are derived, that in particular imply convergence at the minimax rate up to constant over ℓ 0 [s n ], and the authors also derive results on uncertainty quantification by studying the frequentist coverage of credible sets using their procedure.

For further references on the topic, in particular about relationships between spike and slab priors and absolutely continuous counterparts such as the horseshoe or the spike and slab LASSO, we refer to the paper van der Pas et al. ( 2017b) and its discussion by several authors of the previously mentioned works.

Overview of results and outline. This paper obtains the following results.

1. For the spike and slab prior, in Section 2.2.2 we establish lower bound results that show that the popular Laplace slab yields suboptimal rates when the complete empirical Bayes posterior is considered.

2. In Sections 2.2.3 and 2.2.6, we establish rate-optimal results for the posterior squared L 2 -moment for the usual spike and slab with a Cauchy slab, when the prior hyperparameter is chosen via a marginal maximum likelihood method.

3. In Section 2.2.4, the spike and slab LASSO prior is considered and we provide a near-optimal adaptive rate for the corresponding complete empirical Bayes posterior distribution.

Section 2.2 introduces the framework, notation, and the main results, ending with a brief simulation study in Section 2.2.5 and discussion. Section 2.3 gathers the proofs of the lower-bound results as well as upper-bounds on the spike and slab prior. Technical lemmas for the spike and slab prior can be found in Section 2.4, while Sections 2.5-2.6 contain the proof of the result for the spike and slab LASSO prior.

For real-valued functions f, g, we write f ≲ g if there exists a universal constant C such that f (x) ≤ Cg(x), and f ≳ g is defined similarly. When x is a positive real number or an integer, we write f (x) ≍ g(x) if there exists positive constants c, C, D such that for

x ≥ D, we have cf (x) ≤ g(x) ≤ Cf (x). For reals a, b, one denotes a ∧ b = min(a, b) and a ∨ b = max(a, b).

Framework and main results

Empirical Bayes estimation with spike and slab prior

In the setting of model (2.1.1), the spike and slab prior on θ with fixed parameter

α ∈ [0, 1] is Π α ∼ ⊗ n i=1 (1 -α)δ 0 + αG(•), (2.2.1)
where G is a given probability measure on R. We consider the following choices

G =            Lap(1)
or

Cauchy(1)
where Lap(λ) denotes the Laplace (double exponential) distribution with parameter λ and Cauchy(1) the standard Cauchy distribution. Different choices of parameters and prior distributions are possible (a brief discussion is included below) but for clarity of exposition we stick to these common distributions. In the sequel γ denotes the density of G with respect to Lebesgue measure. By Bayes' formula the posterior distribution under (2.1.1) and (2.2.1) with fixed

α ∈ [0, 1] is Π α [• | X] ∼ ⊗ n i=1 (1 -a(X i ))δ 0 + a(X i )G X i (•), (2.2.2)
where, denoting by ϕ the standard normal density and g(x) = ϕ * G(x) = ´ϕ(x -u)dG(u) the convolution of ϕ and G at point x ∈ R, the posterior weight a(X i ) is given by, for any i,

a(X i ) = a α (X i ) = αg(X i ) (1 -α)ϕ(X i ) + αg(X i ) . (2.2.3)
The distribution G X i has density

γ X i (•) := ϕ(X i -•)γ(•) g(X i ) (2.2.4)
with respect to Lebesgue measure on R. The behaviour of the posterior distribution Π α [• | X] heavily depends on the choices of the smoothing parameters α and γ. It turns out that some aspects of this distribution are thresholding-type estimators, as established in [START_REF] Johnstone | Needles and straw in haystacks: empirical Bayes estimates of possibly sparse sequences[END_REF].

Posterior median and threshold t(α). The posterior median θmed α (X i ) of the ith coordinate has a thresholding property: there exists t(α) > 0 such that θmed α (X i ) = 0 if and only if |X i | ≤ t(α). A default choice can be α = 1/n; one can check that this leads to a posterior median behaving similarly as a hard thresholding estimator with threshold √ 2 log n. One can significantly improve on this default choice by taking a well-chosen data-dependent α.

In order to choose α, in this paper we follow the empirical Bayes method proposed in [START_REF] Johnstone | Needles and straw in haystacks: empirical Bayes estimates of possibly sparse sequences[END_REF]. The idea is to estimate α by maximising the marginal likelihood in α in the Bayesian model, which is the density of α | X. The log-marginal likelihood in α can be written as

ℓ(α) = ℓ n (α; X) = n i=1 log((1 -α)ϕ(X i ) + αg(X i )).
(2.2.5)

Let α be defined as the maximiser of the log-marginal likelihood

α = argmax α∈An ℓ n (α; X), (2.2.6)
where the maximisation is restricted to

A n = [α n , 1], with α n defined by t(α n ) = 2 log n.
The reason for this restriction is that one does not need to take α smaller than α n , which would correspond to a choice of α 'more conservative' than hard-thresholding at threshold level √ 2 log n.

In [START_REF] Johnstone | Needles and straw in haystacks: empirical Bayes estimates of possibly sparse sequences[END_REF], Johnstone and Silverman prove that the posterior median αmed (X i ) has remarkable optimality properties, for many choices of the slab density γ. For γ with tails 'at least as heavy as' the Laplace distribution, then this point estimator converges at the minimax rate over ℓ 0 [s n ]. More precisely, it follows from Theorem 1 in [START_REF] Johnstone | Needles and straw in haystacks: empirical Bayes estimates of possibly sparse sequences[END_REF] that there exists constants C, c 0 , c 1 such that if

c 1 log 2 n ≤ s n ≤ c 0 n, (2.2.7)
then the posterior median θmed

α = ( θmed α (X i )) 1≤i≤n is rate optimal sup θ∈ℓ 0 [sn] E θ ∥ θmed α -θ∥ 2 ≤ Cs n log(n/s n ). ( 2
) = √ 2 log n.
In the present paper for simplicity of exposition we first work under the condition (2.2.7). In Section 2.2.6, we show that the lower bound part of the condition can be removed when working with the modified estimator as in [START_REF] Johnstone | Needles and straw in haystacks: empirical Bayes estimates of possibly sparse sequences[END_REF].

Plug-in posterior distribution. The posterior we consider in this paper is Π α[• | X], that is the distribution given by (2.2.2), where α has been replaced by its empirical Bayes (EB) estimate α given by (2.2.6). This posterior is called complete EB posterior in the sequel. The value α is easily found numerically, as implemented in the R package EbayesThresh, see [START_REF] Johnstone | EbayesThresh: R Programs for Empirical Bayes Thresholding[END_REF]. As noted in [START_REF] Johnstone | Needles and straw in haystacks: empirical Bayes estimates of possibly sparse sequences[END_REF], the posterior median αmed (X i ) displays excellent behaviour in simulations. However, the entire posterior distribution Π α[• | X] has not been studied so far. It turns out that the behaviour of the posterior median does not always reflect the behaviour of the complete posterior, as is seen in the next subsection.

Suboptimality of the Laplace slab for the complete EB posterior distribution

Theorem 13. Let Π α be the spike and slab prior distribution (2.2.1) with slab distribution G equal to the Laplace distribution Lap(1). Let Π α[• | X] be the corresponding plug-in posterior distribution given by (2.2.2), with α chosen by the empirical Bayes procedure (2.2.6). There exist D > 0, N 0 > 0, and c 0 > 0 such that, for any n ≥ N 0 and any s n with 1 ≤ s n ≤ c 0 n , there exists θ 0 ∈ ℓ 0 [s n ] such that,

E θ 0 ˆ∥θ -θ 0 ∥ 2 dΠ α[θ | X] ≥ Ds n e √ log (n/sn) .
Theorem 13 implies that taking a Laplace slab leads to a suboptimal convergence rate in terms of the posterior squared L 2 -moment. This result is surprising at first, as we know by (2.2.8) that the posterior median converges at optimal rate r n . The posterior mean also converges at rate r n uniformly over ℓ 0 [s n ], by Theorem 1 of [START_REF] Johnstone | Needles and straw in haystacks: empirical Bayes estimates of possibly sparse sequences[END_REF]. So at first sight it would be quite natural to expect that so does the posterior second moment.

One can naturally ask whether the suboptimality result from Theorem 13 could come from considering an integrated L 2 -moment, instead of simply asking for a posterior convergence result in probability, as is standard in the posterior rates literature following [START_REF] Ghosal | Convergence rates of posterior distributions[END_REF]. We now derive a stronger result than Theorem 13 under the mild condition s n ≳ log 2 n. The fact that the result is stronger follows from bounding from below the integral in the display of Theorem 13 by the integral restricted to the set where ∥θ -θ 0 ∥ 2 is larger than the target lower bound rate.

Theorem 14. Under the same notation as in Theorem 13, if Π α is a spike and slab distribution with as slab G the Laplace distribution, there exists m > 0 such that for any s n with s n /n → 0 and log

2 n = O(s n ) as n → ∞, there exists θ 0 ∈ ℓ 0 [s n ] such that, as n → ∞, E θ 0 Π α ∥θ -θ 0 ∥ 2 ≤ ms n e √ 2 log (n/sn) | X = o(1).
Theorem 14, by providing a lower bound in the spirit of [START_REF] Castillo | Lower bounds for posterior rates with Gaussian process priors[END_REF], shows that the answer to the above question is negative, and for a Laplace slab, the plug-in posterior Π α[• | X] does not converge at minimax rate uniformly over ℓ 0 [s n ].

Note that the suboptimality occuring here does not result from an artificially constructed example (we work under exactly the same framework as [START_REF] Johnstone | Needles and straw in haystacks: empirical Bayes estimates of possibly sparse sequences[END_REF]) and that this has important (negative) consequences for construction of credible sets. Due to the rate-suboptimality of the EB Laplace-posterior, typical credible sets derived from it (such as, e.g., taking quantiles of a recentered posterior second moment) will inherit the suboptimality in terms of their diameter, and therefore will not be of optimal size. Fortunately, it is still possible to achieve optimal rates for certain spike and slab EB posteriors: the previous phenomenon indeed disappears if the tails of the slab in the prior distribution are heavy enough, as seen in the next subsection.

Optimal posterior convergence rate for the EB spike and

Cauchy slab

The next result considers Cauchy tails, although other examples can be covered, as discussed below. In the sequel, we abbreviate by SAS prior a spike and slab prior with Cauchy slab.

Theorem 15. Let Π α be the SAS prior distribution (2.2.1) with slab distribution G equal to the standard Cauchy distribution. Let Π α[• | X] be the corresponding plug-in posterior distribution given by (2.2.2), with α chosen by the empirical Bayes procedure (2.2.6). There exist C > 0, N 0 > 0, and c 0 , c 1 > 0 such that, for any n ≥ N 0 , for any s n such that (2.2.7) is satisfied for such c 0 , c 1 , sup

θ 0 ∈ℓ 0 [sn] E θ 0 ˆ∥θ -θ 0 ∥ 2 dΠ α(θ | X) ≤ Cr n .
If one only assumes s n ≤ c 0 n in (2.2.7), then the last statement holds with the bound Cr n replaced by Cr n + C log 3 n.

Theorem 15 confirms that the empirical Bayes plug-in posterior, with α chosen by marginal maximum likelihood, converges at optimal rate with precise logarithmic factor, at least under the mild condition (2.2.7), if tails of the slab distribution are heavy enough. Inspection of the proof of Theorem 15 reveals that any slab density γ with tails of the order x -1-δ with δ ∈ (0, 2) gives the same result. Sensibility to the tails, in particular in view of posterior convergence in terms of d q -distances, will be further investigated in [START_REF] Castillo | Spike and slab empirical Bayes sparse credible sets[END_REF].

We note that the horseshoe prior on θ considered in van der Pas et al. (2017a)van der Pas et al. (2017b) also has Cauchy-like tails, which seems to confirm that for empirical Bayes-calibrated (product-type) sparse priors, heavy tails are important to ensure optimal or near-optimal behaviour, see also the discussion Castillo (2017a).

The lower bound in condition (2.2.7) is specific to the estimate α. Note that in the very sparse regime where s n ≤ c 1 log 2 n, the rate is no more than C log 3 n, thus missing the minimax rate by at most a logarithmic factor. This lower bound on s n can be removed and the minimax rate obtained over the whole range of sparsities s n if one modifies slightly α, where the estimator is changed if α is too close to the lower boundary of the maximisation interval, see Section 2.2.6.

Posterior convergence for the EB spike and slab LASSO

Now consider the following prior on θ with fixed parameter α ∈ [0, 1]

Π α ∼ ⊗ n i=1 (1 -α)G 0 (•) + αG 1 (•), (2.2.9)
where for k = 0, 1, G k is given by

G 0 = Lap(λ 0 ), G 1 =            Lap(λ 1 )
or Cauchy(1/λ 1 ), which leads to the spike and slab LASSO prior of [START_REF] Ročková | The spike-and-slab LASSO[END_REF] in the case of a Laplace G 1 , and to a heavy-tailed variant of the spike and slab LASSO if

G 1 is Cauchy(1/λ 1 ), that is if its density is γ 1 (x) = (λ 1 /π)(1 + λ 2 1 x 2 ) -1 .
In this setting γ 0 , γ 1 denote the densities of G 0 , G 1 with respect to Lebesgue measure. We call SSL prior a spike and slab LASSO prior with Cauchy slab.

By Bayes' formula the posterior distribution under (2.1.1) and (2.2.9) with fixed

α ∈ [0, 1] is Π α [• | X] ∼ ⊗ n i=1 (1 -a(X i ))G 0,X i (•) + a(X i )G 1,X i (•), (2.2.10) where g k (x) = ϕ * G k (x) = ´ϕ(x -u)dG k (u)
is the convolution of ϕ and G k at point

x ∈ R for k = 0, 1, the posterior weight a(X i ) is defined through the function a(•) given by

a(x) = a α (x) = αg 1 (x) (1 -α)g 0 (x) + αg 1 (x) ,
and if G k has density γ k with respect to Lebesgue measure, the distribution G k,X i has density

γ k,X i (•) := ϕ(X i -•)γ k (•) g k (X i ) .
In slight abuse of notation, we keep the same notation in the case of the SSL prior for quantities such as a(x) or α below, as it will always be clear from the context which prior we work with. We consider the following specific choices for the constants λ 0 , λ 1

     λ 0 = L 0 n, L 0 = 5 √ 2π, λ 1 = L 1 , L 1 = 0.05.
(2.2.11)

The choice of the constants L 0 , L 1 is mostly for technical convenience, and is similar to that of, e.g. Corollary 5.2 in [START_REF] Ročková | Bayesian estimation of sparse signals with a continuous spike-and-slab prior[END_REF]. Any other constant L 0 (resp. L 1 ) larger (resp. smaller) than the above value also works for the following result. The above numerical values may not be optimal. Let α be defined as the maximiser of the log-marginal likelihood,

α = argmax α∈[C log n/n,1] ℓ n (α; X), (2.2.12)
for C = C 0 (γ 0 , γ 1 ) a large enough constant to be chosen below (this ensures that α belongs to an interval on which we can verify that β is increasing, see (2.5.2)). This time we do not have access to the threshold t, since for the SSL prior the posterior median is not a threshold estimator, so here C log n/n plays the role of an approximated version of α n in (2.2.6).

Theorem 16. Let Π α be the SSL prior distribution (2.2.9) with Cauchy slab and parameters (λ 0 , λ 1 ) given by (2.2.11). Let Π α[• | X] be the corresponding plug-in posterior distribution given by (2.2.10), with α chosen by the empirical Bayes procedure (2.2.12). There exist C > 0, N 0 > 0, and c 0 , c 1 > 0 such that, for any n ≥ N 0 , for any s n such that (2.2.7) is satisfied for such c 0 , c 1 , then sup

θ 0 ∈ℓ 0 [sn] E θ 0 ˆ∥θ -θ 0 ∥ 2 dΠ α(θ | X) ≤ Cs n log n.
If one only assumes s n ≤ c 0 n in (2.2.7), then the last bound holds with Cs n log n replaced by C(s n log n + log 3 n).

This result is an SSL version of Theorem 15. It shows that a spike and slab LASSO prior with heavy-tailed slab distribution and empirical Bayes choice of the weight parameter leads to a nearly optimal contraction rate for the entire posterior distribution. Hence it provides a theoretical guarantee of a fully data-driven procedure of calibration of the smoothing parameter in SSL priors.

A brief numerical study

Theorems 13-14 imply that the posterior distribution for the spike and slab prior and Laplace(1) slab does not converge at optimal rate and the discrepancy between the actual rate and the minimax rate for some 'bad' θ 0 s is at least of order

R n = exp 2 log(n/s n ) log(n/s n ) ,
up to a multiplicative constant factor, as both lower and upper bounds are up to a constant. Note that R n grows more slowly than a polynomial in n/s n , so the suboptimality effect will typically be only visible for quite large values of n/s n . For instance, if n = 10 4 and s n = 10, one has R n ≈ 6, which is quite small given that an extra multiplicative constant is also involved.

For the present simulation study we took n = 10 7 , s n = 10, for which R n ≈ 13.9, and the non-zero values of θ 0 equal to {2 log(n/s n )} 1/2 , as the lower bound proof of Theorems 13-14 suggests. We computed α using the package EBayesThresh of Johnstone and Silverman [START_REF] Johnstone | EbayesThresh: R Programs for Empirical Bayes Thresholding[END_REF] and computed ´∥θ -θ 0 ∥ 2 2 dΠ α(X ) using its explicit expression, which can be obtained in closed form for a Laplace slab, with similar computations as in [START_REF] Johnstone | EbayesThresh: R Programs for Empirical Bayes Thresholding[END_REF], Section 6.3. We then took the empirical average over 100 repetitions to estimate the target expectation

R 2 := E θ 0 ´∥θ -θ 0 ∥ 2 2 dΠ α(X )
. We first took γ = Lap(1) a standard Laplace slab and obtained R2 ≈ 1110. For comparison, we computed the empirical quadratic risk Rmean for the posterior mean (approximating E θ 0 ∥ θmean -θ 0 ∥ 2 ) and Rmedian the posterior median of the same posterior, obtaining Rmean ≈ 158 and Rmedian ≈ 167. So, in this case R2 is already 6 to 7 times larger than the risk of either mean or median.

To further illustrate the 'blow-up' in the rate for the posterior second moment R 2 , we took a Laplace slab Lap(a) with inverse-scale parameter a, for which the numerator in the definition of R n becomes exp{a 2 log(n/s n )} (let us also note that the multiplicative constant we refer to above also depends on a). The same simulation experiment as above was conducted, with the standard Laplace slab replaced by a Lap(a) slab, for different values of a. The numerical results are presented in Table 2.1, which feature a noticeable increase in the second moment R2 , while the risks of posterior mean and median stay around the same value, as expected. We also performed the same experiments for the quasi-Cauchy slab prior introduced in Johnstone and Silverman (2004)-Johnstone and Silverman (2005) (it is very close to the standard Cauchy slab -in particular it has the same Cauchy tails -but more convenient from the numerical perspective, see [START_REF] Johnstone | EbayesThresh: R Programs for Empirical Bayes Thresholding[END_REF], Section 6.4). We found Rmedian ≈ 192, Rmean ≈ 191 for the posterior mean and R2 ≈ 287 for the posterior second moment. This time, as expected, the posterior second moment is not far from the two other risks.

a 0.

Modified empirical Bayes estimator

For n ≥ 3 and A ≥ 0, let us set t 2 n = 2 log n -5 log log n and t A = 2(1 + A) log n. For Π α the SAS prior with a Cauchy slab, let as before t(α) be the posterior median threshold for fixed α. It is not hard to check that t(•) is continuous and strictly decreasing so has an inverse (see [START_REF] Johnstone | Needles and straw in haystacks: empirical Bayes estimates of possibly sparse sequences[END_REF], Section 5.3). In a similar fashion as in [START_REF] Johnstone | Needles and straw in haystacks: empirical Bayes estimates of possibly sparse sequences[END_REF], Section 4, let us introduce a modified empirical Bayes estimator as, for A ≥ 0 and t := t(α), α

A := t -1 (t A ), αA =      α, if t ≤ t n , α A , if t > t n .
(2.2.13)

Theorem 17. Let Π α be the SAS prior distribution with slab distribution G equal to the standard Cauchy distribution. For a fixed A > 0, let Π αA [• | X] be the corresponding plug-in posterior distribution, with αA the modified estimator (2.2.13). There exist C, c 0 > 0, N 0 > 0, such that, for any n ≥ N 0 , for any s n such that s n ≤ c 0 n,

sup θ 0 ∈ℓ 0 [sn] E θ 0 ˆ∥θ -θ 0 ∥ 2 dΠ α(θ | X) ≤ Cr n .
Theorem 17 shows that the plug-in SAS posterior distribution using the modified estimator (2.2.13), A > 0, and a Cauchy slab attains the minimax rate of convergence r n even in the very sparse regime s n ≲ log 2 n, for which the unmodified estimate of Theorem 15 may lose a logarithmic factor.

Discussion

In this paper, we have developped a theory of empirical Bayes choice of the hyperparameter of spike and slab prior distributions. It extends the work of Johnstone and Silverman [START_REF] Johnstone | Needles and straw in haystacks: empirical Bayes estimates of possibly sparse sequences[END_REF] in that here the complete EB posterior distribution is considered. One important message is that such a generalisation preserves optimal convergence rates at the condition of taking slab distributions with heavy enough tails. If the tails of the slab are only moderate (e.g. Laplace), then the complete EB posterior rate may be suboptimal. This is in contrast with the hierarchical case considered in [START_REF] Castillo | Needles and straw in a haystack: posterior concentration for possibly sparse sequences[END_REF], where a Laplace slab combined with a Beta distributed prior on α was shown to lead to an optimal posterior rate. On the one hand, the empirical Bayes method often leads to simpler or/and more easily tractable practical algorithms; on the other hand, we have illustrated here that the complete EB posterior may in some cases need slightly stronger conditions to conserve optimal theoretical guarantees. This phenomenon had not been pointed out so far in the literature, to the best of our knowledge.

We also note that Theorem 15 (or Theorem 17 if one allows for very sparse signals) enables one to recover the optimal form of the logarithmic factor log(n/s n ) in the minimax rate. This entails significant work, as one needs to control the empirical Bayes weight estimate α both from above and below. This could work too in the SSL setting of Theorem 16, although this seems to need substantial extra technical work.

Looking at Theorems 13 and 14, it is natural to wonder why the Empirical Bayes approach fails for the Laplace slab where the full Bayes approach succeeds as seen in Castillo and van der Vaart (2012) Theorem 2.2. The reason why the hierarchical Bayes version works also for γ Laplace is the extra penalty in model size induced by the hierarchical prior on dimension. Indeed, in the full Bayes approach, the posterior distribution of α given X has density

f α | X (α) ∝ p(X | α)π(α),
where p(X | α) is the marginal density one maximises when considering the MMLE α. Hence adding a term log π(α) for well-chosen π -for instance that arising from a Beta(1, n + 1) prior on α as considered in Castillo and van der Vaart (2012) -to the log-marginal likelihood one maximises forces α to concentrate on smaller values. For instance, in the present setting, one could consider a penalised log-marginal maximum likelihood, which would force the estimate α to concentrate on slightly smaller values, which would allow one to avoid the extra e √ log n/sn term arising in Theorems 13-14.

The present work can also serve as a basis for constructing confidence regions using spike-and-slab posterior distributions. This question is considered in the forthcoming paper [START_REF] Castillo | Spike and slab empirical Bayes sparse credible sets[END_REF].

Proofs for the spike and slab prior

Let us briefly outline the ingredients of the proofs to follow. For Theorems 13 and 15, our goal is to bound the expected posterior risk R n (θ 0 ) = E θ 0 ´∥θ -θ 0 ∥ 2 dΠ α(θ | X). There are three main tools. First, after introducing notation and basic bounds in Section 2.3.1, bounds on the posterior risk for fixed α are given in Section 2.3.2, as well as corresponding bounds for random α. Let us note that the corresponding upper bounds are different from those obtained on the quadratic risk for the posterior median in [START_REF] Johnstone | Needles and straw in haystacks: empirical Bayes estimates of possibly sparse sequences[END_REF] (and in fact, must be, in view of the negative result in Theorem 13). Second, inequalities on moments of the score function are stated in Section 2.3.3. As a third tool, we obtain deviation inequalities on the location of α in Section 2.3.4. One of the bounds sharpens the corresponding bound from [START_REF] Johnstone | Needles and straw in haystacks: empirical Bayes estimates of possibly sparse sequences[END_REF] in case the signal belongs to the nearly-black class ℓ 0 [s n ] which we assume here.

Proofs of Theorems 13 and 15 are given in Sections 2.3.5 and 2.3.6. For Theorem 15, we also needed to slightly complete the proof of one of the inequalities on thresholds stated in [START_REF] Johnstone | Needles and straw in haystacks: empirical Bayes estimates of possibly sparse sequences[END_REF], see Lemma 11. The proof of Theorem 14, which uses ideas from both previous proofs, is given in Section 2.3.7. Proofs of technical lemmas for the SAS prior are given in Section 2.4.

Notation and tools for the SAS prior

Expected posterior L 2 -squared risk. For a fixed weight α, the posterior distribution of θ is given by (2.2.2). On each coordinate, the mixing weight a(X i ) is given by (2.2.3) and the density of the non-zero component γ X i by (2.2.4). In the sequel we will obtain bounds on the following quantity, already for a given α ∈ [0, 1],

ˆ∥θ -θ 0 ∥ 2 dΠ α (θ | X) = n i=1 ˆ(θ i -θ 0,i ) 2 dΠ α (θ i | X i ).
To do so, we study r 2 (α, µ, x)

:= ´(u -µ) 2 dπ α (u | x), where π α (• | x) ∼ (1 -a(x))δ 0 + a(x)γ x (•). By definition r 2 (α, µ, x) = (1 -a(x))µ 2 + a(x) ˆ(u -µ) 2 γ x (u)du.
This quantity is controlled by a(x) and the term involving γ x . From the definition of a(x), bounding the denominator from below by one of its two components, and using a(x) ≤ 1 yields, for any real x and α ∈ [0, 1],

α g g ∨ ϕ (x) ≤ a(x) ≤ 1 ∧ α 1 -α g ϕ (x). (2.3.1)
The marginal likelihood in α. By definition, the empirical Bayes estimate α in (2.2.6) maximises the logarithm of the marginal likelihood in α in (2.2.5). In case the maximum is not taken at the boundary, α is a zero of the derivative (score) of the previous likelihood. Its expression is S(α) = n i=1 β(X i , α), where following Johnstone and Silverman (2004) we set, for 0 ≤ α ≤ 1 and any real x,

β(x, α) = β(x) 1 + αβ(x) , β(x) = g ϕ (x) -1.
The study of α below uses in a crucial way the first two moments of β(X i , α), so we introduce the corresponding notation next. Let E τ , for τ ∈ R n , denote the expectation

under θ 0 = τ . Define m(α) = -E 0 β(X, α) (2.3.2)
and further denote

m 1 (τ, α) = E τ [β(X, α)] = ˆ∞ -∞ β(t, α)ϕ(t -τ )dt. m 2 (τ, α) = E τ [β(X, α) 2 ].
The thresholds ζ(α), τ (α) and t(α). Following [START_REF] Johnstone | Needles and straw in haystacks: empirical Bayes estimates of possibly sparse sequences[END_REF], we introduce several useful thresholds. From Lemma 1 in [START_REF] Johnstone | Needles and straw in haystacks: empirical Bayes estimates of possibly sparse sequences[END_REF], we know that g/ϕ, and therefore β = g/ϕ -1, is a strictly increasing function on R + . It is also continuous, so given α, a pseudo-threshold ζ = ζ(α) can be defined by

β(ζ) = 1 α . (2.3.3)
Further one can also define τ (α) as the solution in x of

Ω(x, α) := a(x) 1 -a(x) = α 1 -α g ϕ (x) = 1. Equivalently, a(τ (α)) = 1/2. Also, β(τ (α)) = α -1 -2 so τ (α) ≤ ζ(α). Define α 0 as τ (α 0 ) = 1 and set τ (α) = τ (α ∧ α 0 ). (2.3.4)
Recall from Section 2.2 that t(α) is the threshold associated to the posterior median for given α. It is shown in [START_REF] Johnstone | Needles and straw in haystacks: empirical Bayes estimates of possibly sparse sequences[END_REF], Lemma 3, that t(α) ≤ ζ(α).

Finally, the following bound in terms of τ (α), see [START_REF] Johnstone | Needles and straw in haystacks: empirical Bayes estimates of possibly sparse sequences[END_REF] p. 1623, is also useful for large x,

1 -a(x) ≤ 11l |x|≤τ (α) + e -1 2 (|x|-τ (α)) 2 1l |x|>τ (α) .
(2.3.5)

Posterior risk bounds

Recall the notation r 2 (α, µ, x) = ´(u -µ) 2 dΠ α (u).

Lemma 1. Let γ be the Cauchy or Laplace density. For any x and α ∈ [0, 1/2],

r 2 (α, 0, x) ≤ C 1 ∧ α 1 -α g ϕ (x) (1 + x 2 ) r 2 (α, µ, x) ≤ (1 -a(x))µ 2 + Ca(x)((x -µ) 2 + 1).
Let γ be the Cauchy density. For any real x and α ∈ [0, 1/2],

E 0 r 2 (α, 0, x) ≤ Cτ (α)α E µ r 2 (α, µ, x) ≤ C(1 + τ (α) 2 ).
The following lower bound is used in the proof of Theorem 13.

Lemma 2. Let γ be the Laplace density. There exists

C 0 > 0 such that, for x ∈ R and α ∈ [0, 1] r 2 (α, 0, x) ≥ C 0 α.
We now turn to bounding r 2 (α, µ, x). This is the quantity r 2 (α, µ, x), where α (which comes in via a(x) = a α (x)) is replaced by α. This is done with the help of the threshold τ (α).

Lemma 3 (no signal or small signal). Let γ be the Cauchy density. Let α be a fixed non-random element of (0, 1). Let α be a random element of [0, 1] that may depend on x ∼ N (0, 1) and on other data. Then there exists C 1 > 0 such that

Er 2 (α, 0, x) ≤ C 1 ατ (α) + P (α > α) 1/2 .
There exists C 2 > 0 such that for any real µ, if x ∼ N (µ, 1),

Er 2 (α, µ, x) ≤ µ 2 + C 2 .
Lemma 4 (signal). Let γ be the Cauchy density. Let α be a fixed non-random element of (0, 1). Let α be a random element of [0, 1] that may depend on x ∼ N (µ, 1) and on other data and such that τ (α) 2 ≤ d log(n) with probability 1 for some d > 0. Then there exists C 2 > 0 such that for all real µ,

Er 2 (α, µ, x) ≤ C 2 1 + τ (α) 2 + (1 + d log n)P (α < α) 1/2 .

Moments of the score function

The next three lemmas are borrowed from [START_REF] Johnstone | Needles and straw in haystacks: empirical Bayes estimates of possibly sparse sequences[END_REF] and apply to any density γ such that log γ is Lipschitz on R and satisfies

γ(y) -1 ˆ∞ y γ(u)du ≈ y κ-1 , as y → ∞. (2.3.6)
Both Cauchy and Laplace densities satisfy (2.3.6), with κ = 2 and κ = 1 respectively, and their logarithm is Lipschitz.

Lemma 5. For κ ∈ [1, 2] as in (2.3.6), as α → 0, m(α) ≍ ζ κ-1 g(ζ).
Also, the function α → m(α) is nonnegative and increasing in α.

Lemma 6. The function α → m 1 (µ, α) is decreasing in α. Also, m 1 (ζ, α) ∼ 1/(2α) as α → 0. For small enough α, m 2 (µ, α) ≤ Cα -1 m 1 (µ, α), µ ≥ 1.
Lemma 7. There exist a constant c 1 such that for any x and α,

|β(x, α)| ≤ 1 α ∧ c 1 ,
and constants c 2 , c 3 , c 4 such that for any α, and κ as in (2.3.6),

m 1 (µ, α) ≤ -m(α) + c 2 ζ(α)µ 2 , |µ| ≤ 1/ζ(α) m 1 (µ, α) ≤ (α ∧ c 3 ) -1 for all µ and m 2 (µ, α) ≤ c 4 m(α) ζ(α) κ α |µ| ≤ 1/ζ = 1/ζ(α) m 2 (µ, α) ≤ (α ∧ c 3 ) -2
for all µ.

In-probability bounds for α

Lemma 9 below implies that, for any possible θ 0 , the estimate α is smaller than a certain α 1 with high probability. One can interpret this as saying that α does not lead to too much undersmoothing (i.e. too many nonzero coefficients). On the other hand, if there is enough signal in a certain sense, α does not lead to too much oversmoothing (i.e. too many zero coefficients), see Lemma 10.

Although we generally follow the approach of [START_REF] Johnstone | Needles and straw in haystacks: empirical Bayes estimates of possibly sparse sequences[END_REF], there is one significant difference. One needs a fairly sharp bound on α 1 below. Using the definition from [START_REF] Johnstone | Needles and straw in haystacks: empirical Bayes estimates of possibly sparse sequences[END_REF] would lead to a loss in terms of logarithmic factors for the posterior L 2 -squared moment. So we work with a somewhat different α 1 , and shall thus provide a detailed proof of the corresponding Lemma 9. For the oversmoothing case, one can borrow the corresponding Lemma of [START_REF] Johnstone | Needles and straw in haystacks: empirical Bayes estimates of possibly sparse sequences[END_REF] as is.

Let α 1 = α 1 (d) be defined as the solution of the equation, with

η n = s n /n, dα 1 m(α 1 ) = η n , (2.3.7)
where d is a constant to be chosen (small enough for Lemma 9 to hold). A solution of (2.3.7) exists, as using Lemma 5, α → α m(α) is increasing in α, and equals 0 at 0. Also, provided η n is small enough, α 1 can be made smaller than any given arbitrary constant.

The corresponding threshold ζ 1 is defined by

β(ζ 1 ) = α -1 1 . From Lemma 5, we have m(α 1 ) ≍ ζg(ζ 1 ) if γ is Cauchy and m(α 1 ) ≍ g(ζ 1 ) if γ is Laplace.
Lemma 8. Let κ be the constant in (2.3.6). Let α 1 be defined by (2.3.7) for d a given constant and let ζ 1 be given by β(ζ 1 ) = α -1 1 . Then there exist real constants c 1 , c 2 such that for large enough n,

log(n/s n ) + c 1 ≤ ζ 2 1 2 ≤ log(n/s n ) + κ -1 2 log log n + c 2 ,
with κ as in (2.3.6). Also, ζ 2 1 ∼ 2 log(n/s n ) as n/s n goes to ∞. Lemma 9. Let α 1 be defined by (2.3.7) for d a given small enough constant and let ζ 1 be given by β(ζ 1 ) = α -1 1 . Suppose (2.2.7) holds. Then for some constant C > 0,

sup θ∈ℓ 0 [sn] P θ [ ζ < ζ 1 ] ≤ exp(-Cs n ).
For the oversmoothing case, one denotes the proportion of signals above a level τ by

π(τ ; µ) = 1 n #{i : |µ i | ≥ τ }. (2.3.8)
We also set, recalling that α 0 is defined via τ (α 0 ) = 1,

α(τ, π) = sup{α ≤ α 0 : πm 1 (τ, α) ≥ 2 m(α)}.
(2.3.9)

One defines ζ τ,π as the corresponding pseudo-threshold β -1 (α(τ, π) -1 ).

Lemma 10 (Johnstone and Silverman ( 2004), Lemma 11). There exists C and π 0 such that if π < π 0 , then for all τ ≥ 1, sup θ: π(τ ;θ)≥π

P θ [ ζ > ζ τ,π ] ≤ exp{-Cnϕ(ζ τ,π )}.

Proof of Theorem 13

Proof. Let α * be defined as the solution in α of the equation,

α m(α) = η n /4, (2.3.10)
where

η n = s n /n (that is α * = α 1 (d) with d = 4 in (2.3.7)). Let ζ * be defined via β(ζ * ) = α * .
Let θ 0 be the specific signal defined by, for α * , ζ * as in (2.3.10),

θ 0,i =      ζ * , 1 ≤ i ≤ s n 0, s n < i ≤ n .
Using Lemma 5, one gets m(α

* ) ≍ g(ζ * ) ≍ γ(ζ * ) as ζ * → ∞. Lemma 8 implies ζ * 2 ≥ 2 log(1/η n ) + C,
) + C ≥ log(1/η n ).
We next prove that, for α given by (2.2.6), for small enough c > 0,

P θ 0 [ α < α * ] ≤ e -csn .
(2.3.12)

If α * ≤ α n the probability at stake is 0, as α belongs to [α n , 1] by definition. For α * > α n , we have {α

< α * } = {S(α * ) < 0}. With A = n i=1 m 1 (µ i , α * ), P θ 0 [ α < α * ] = P θ 0 [S(α * ) < 0] = P θ 0 n i=1 β(θ 0,i + Z i , α * ) -m 1 (θ 0,i , α * ) < -A Setting W i = m 1 (θ 0,i , α * ) -β(θ 0,i + Z i , α * ), we have |W i | ≤ 2C/α * =: M and W i are independent.
So by Bernstein's inequality,

P θ 0 n i=1 W i > A ≤ exp - 1 2 A 2 V + 1 3 M A ,
where V is an upper-bound for n i=1 Var(W i ). The term A equals

A = (n -s n )(-m(α * )) + s n m 1 (ζ * , α * ).
The function α → α m(α) is increasing, as m(•) is (Lemma 5), so by its definition (2.3.10), α * can be made smaller than any given positive constant, provided c 0 in (2.2.7) is small enough, ensuring η n = s n /n is small enough. Using Lemma 6, m 1 (ζ, α) ∼ 1/(2α) as α → 0. So, using (2.3.10), one obtains, for small enough c 0 ,

A ≥ s n 3α * - s n 4α * = s n 12α * .
On the other hand, the last part of Lemma 7 implies

V ≤ i / ∈S 0 m 2 (0, α * ) + i∈S 0 m 2 (ζ * , α * ) ≤ C(n -s n ) m(α * ) ζ * α * + C s n α * 2 .
Using the definition of α * , one deduces V ≲ s n /α * 2 and from this

V A 2 + M A 3A 2 ≲ 1 s n ,
which in turn implies (2.3.12), as then

P θ 0 [ n i=1 W i > A] ≤ exp[-cs n ]. Next one writes ˆ∥θ -θ 0 ∥ 2 dΠ α[θ | X] ≥ ˆ∥θ -θ 0 ∥ 2 dΠ α[θ | X]1l α≥α * ≥ i / ∈S 0 ˆθ2 i dΠ α(θ | X)1l α≥α * Lemma 2 implies, for any possibly data-dependent weight α, that ´θ2 i dΠ α (θ | X) ≳ α, so ˆ∥θ -θ 0 ∥ 2 dΠ α[θ | X] ≥ (n -s n )α1l α≥α * ≥ (n -s n )α * 1l α≥α * . As (n -s n )α * P θ 0 [ α ≥ α * ] ≳ Cnα * (1 -e -csn
), an application of (2.3.11) concludes the proof.

Proof of Theorem 15

Let us decompose the risk R n (θ 0 ) = E θ 0 ´∥θ -θ 0 ∥ 2 dΠ α(θ | X) according to whether coordinates of θ correspond to a 'small' or 'large' signal, the threshold being

ζ 1 = β -1 (α -1 1 ), with α 1 defined in (2.3.7). One can write R n (θ 0 ) = i: θ 0,i =0 + i: 0<|θ 0,i |≤ζ 1 + i: |θ 0,i |>ζ 1 E θ 0 ˆ(θ i -θ 0,i ) 2 dΠ α(θ i | X).
We next use the first part of Lemma 3 with α = α 1 and the second part of the Lemma to obtain, for any

θ 0 in ℓ 0 [s n ], i: θ 0,i =0 + i: 0<|θ 0,i |≤ζ 1 E θ 0 ˆ(θ i -θ 0,i ) 2 dΠ α(θ i | X) ≤ C 1 i: θ 0,i =0 [α 1 τ (α 1 ) + P θ 0 (α > α 1 )] + i: 0<|θ 0,i |≤ζ 1 (θ 2 0,i + C) ≤ C 1 (n -s n )α 1 τ (α 1 ) + (n -s n )e -c 1 log 2 n + (ζ 2 1 + C)s n ,
where for the last inequality we use Lemma 9 and (2.2.7). From (2.3.7) one gets, with

η n = s n /n, nα 1 ≲ nη n ζ -1 1 g(ζ 1 ) -1 ≲ s n ζ 1 .
Now using Lemma 8 and the fact that τ (α 1 ) ≤ ζ 1 , one obtains that the contribution to the risk of the indices i with |θ 0,i | ≤ ζ 1 is bounded by a constant times s n log(n/s n ).

It remains to bound the part of the risk for indexes i with |θ 0,i | > ζ 1 . To do so, one uses Lemma 4 with α chosen as α = α 2 := α(ζ 1 , π 1 ) and π 1 = π(ζ 1 ; θ 0 ), following the definitions (2.3.8)-(2.3.9). One denotes by ζ 2 the pseudo-threshold associated to α 2 . The following estimates are useful below

ζ 2 1 < ζ 2 2 (2.3.13) π 1 ζ 2 2 ≤ Cη n log(1/η n ). (2.3.14)
These are established in a similar way as in [START_REF] Johnstone | Needles and straw in haystacks: empirical Bayes estimates of possibly sparse sequences[END_REF], but with the updated definition of α 1 , ζ 1 from (2.3.7), so we include the proof below for completeness.

One can now apply Lemma 4 with α = α 2 ,

i: |θ 0,i |>ζ 1 E θ 0 ˆ(θ i -θ 0,i ) 2 dΠ α(θ i | X) ≤ C 2 nπ 1 1 + ζ 2 2 + (1 + d log n)P θ 0 (α < α 2 ) 1/2 ≤ C 2 nπ 1 1 + ζ 2 2 + (1 + d log n)P θ 0 ( ζ > ζ 2 ) 1/2 .
Let us verify that the term in brackets in the last display is bounded above by We now check that (2.3.13)-(2.3.14) hold. We first compare α 1 and α 2 . For small enough α, the bound on m 1 from Lemma 7 becomes 1/α, so that, using the definition

C(1 + ζ 2 2 ). If ζ 2 > log n, this is immediate by bounding P θ 0 ( ζ > ζ 2 ) by 1. If ζ 2 ≤ log n, Lemma 10 implies P θ 0 ( ζ > ζ 2 ) ≤ exp(-Cnϕ(ζ 2 )) ≤ exp(-C √ n),
(2.3.7) of α 1 , m 1 (ζ 1 , α 1 ) m(α 1 ) ≤ 1 α 1 η n dα 1 -1 ≤ d η n ≤ d π 1 ,
using the rough bound π 1 ≤ η n . Note that both functions m(•) -1 and m 1 (ζ 1 , •) are decreasing via Lemmas 5-6, and so is their product on the interval where both functions are positive. As d < 2, by definition of α 2 this means α 2 < α 1 that is

ζ 1 < ζ 2 .
To prove (2.3.14), one compares ζ 2 first to a certain

ζ 3 = ζ(α 3 ) defined by α 3 (largest) solution of Φ(ζ(α 3 ) -ζ 1 ) = 8 π 1 α 3 m(α 3 ), with Φ(x) = P [N (0, 1) > x].
Using Lemma 11, which also gives the existence of

ζ 3 , one gets m 1 (ζ 1 , α 3 ) m(α 3 ) ≥ 1 4 β(ζ 3 ) Φ(ζ 3 -ζ 1 ) m(α 3 ) = 1 4α 3 8α 3 m(α 3 ) π 1 m(α 3 ) = 2 π 1 .
This shows, reasoning as above, that [START_REF] Johnstone | Needles and straw in haystacks: empirical Bayes estimates of possibly sparse sequences[END_REF], one distinguishes two cases to further bound ζ 3 .

α 3 ≤ α 2 , that is ζ 2 ≤ ζ 3 . Following
If

ζ 3 > ζ 1 + 1, using ζ 2 2 ≤ ζ 2 3 and m(α 3 ) ≲ ζ 3 g(ζ 3 ), π 1 ζ 2 2 ≤ ζ 2 3 8α 3 m(α 3 ) Φ(ζ 3 -ζ 1 ) ≲ ζ 3 3 g(ζ 3 ) β(ζ 3 ) ζ 3 -ζ 1 ϕ(ζ 3 -ζ 1 ) ≤ Cζ 4 3 ϕ(ζ 3 ) ϕ(ζ 3 -ζ 1 ) = Cζ 4 3 ϕ(ζ 1 )e -(ζ 3 -ζ 1 )ζ 1 ≤ C(ζ 1 + 1) 4 e -ζ 1 ϕ(ζ 1 ),
where for the last inequality we have used that

x → x 4 e -(x-ζ 1 )ζ 1 is decreasing for x ≥ ζ 1 +1. Lemma 8 now implies that ϕ(ζ 1 ) ≲ η n . As ζ 1 goes to ∞ with n/s n , one gets π 1 ζ 2 2 ≲ η n . If ζ 1 ≤ ζ 3 ≤ ζ 1 + 1, let ζ 4 = ζ(α 4 ) with α 4 solution in α of Φ(1) = 8α m(α)π -1 1 .
By the definition of ζ 3 , since Φ(1) ≤ Φ(ζ 3 -ζ 1 ), we have 8α 4 m(α 4 ) ≤ 8α 3 m(α 3 ) so that α 4 ≤ α 3 . Using Lemma 5 as before,

Φ(1) ≲ g(ζ 4 ) β(ζ 4 ) π -1 1 ≲ ϕ(ζ 4 )π -1 1 .
Taking logarithms this leads to

ζ 2 4 ≤ C + 2 log(π -1 1 ).
In particular, ζ 2 2 ≤ 2 log(π -1 1 ) + C. As x → x log(1/x) is increasing, one gets, using

π 1 ≤ η n , π 1 ζ 2 2 ≤ 2η n log(1/η n ) + Cη n ,
which concludes the verification of (2.3.13)-(2.3.14) and the proof of Theorem 15.

In checking (2.3.14), one needs a lower bound on m 1 . In [START_REF] Johnstone | Needles and straw in haystacks: empirical Bayes estimates of possibly sparse sequences[END_REF], the authors mention that it follows from their lower bound (82), Lemma 8. But this bound cannot hold uniformly for any smoothing parameter α (denoted by w in Johnstone and Silverman ( 2004)), as m 1 (0, w) = -m(w) < 0 if w ̸ = 0. So, although the claimed inequality is correct, it does not seem to follow from (82). We state the inequality we use now, and prove it in Section 2.4.3.

Lemma 11. Let Φ(t) = ´∞ t ϕ(u)du. For π 1 , ζ 1 as above, a solution 0 < α ≤ α 1 to the equation Φ(ζ(α) -ζ 1 ) = 8π -1 1 α m(α).
(2.3.15) exists. Let α 3 be the largest such solution. Then for c 0 in (2.2.7) small enough,

m 1 (ζ 1 , α 3 ) ≥ 1 4 β(ζ 3 ) Φ(α 3 -ζ 1 ).
(2.3.16)

Proof of Theorem 14

Let θ 0 , α * , ζ * be defined as in the proof of Theorem 13. Below we show that the event

A = {α ∈ [α * , cα * ]}
, for c a large enough constant, has probability going to 1, faster than a polynomial in 1/n. Recall from the proof of Theorem 13 that, if α ≥ α * , so in particular on A, we have

V X ≥ (n -s n )α * ≥ nα * /2 ≥ C 1 s n g(ζ * ) -1 . Denote v n = ms n g(ζ * ) -1 V X = ˆ∥θ -θ 0 ∥ 2 dΠ α(θ | X),
where m is chosen small enough so that v n ≤ V X /2 on A. Then,

Π α ∥θ -θ 0 ∥ 2 < v n | X 1l A = Π α ∥θ -θ 0 ∥ 2 -V X < v n -V X | X 1l A ≤ Π α ∥θ -θ 0 ∥ 2 -V X < -V X /2 | X ≤ 4V -2 X ˆ{∥θ -θ 0 ∥ 2 -V X } 2 dΠ α(θ | X),
where the second line follows from Markov's inequality. One now writes the L 2 -norm in the previous display as sum over coordinates and one expands the square, while noting that given X the posterior Π α[• | X] makes the coordinates of θ independent

ˆ{∥θ -θ 0 ∥ 2 -V X } 2 dΠ α(θ | X) = ˆ i,j (θ i -θ 0,i ) 2 -ˆ(θ i -θ 0,i ) 2 dΠ α(θ | X) × (θ j -θ 0,j ) 2 -ˆ(θ j -θ 0,j ) 2 dΠ α(θ | X) dΠ α(θ | X) = n i=1 ˆ (θ i -θ 0,i ) 2 -ˆ(θ i -θ 0,i ) 2 dΠ α(θ | X) 2 dΠ α(θ | X) ≤ n i=1 ˆ(θ i -θ 0,i ) 4 dΠ α(θ | X).
The last bound is the same as in the proof of the upper bound Theorem 15, except the fourth moment replaces the second moment. Denote

r 4 (α, µ, x) = ´(u -µ) 4 dπ α (u | x), then r 4 (α, µ, x) = (1 -a(x))µ 4 + a(x) ˆ(u -µ) 4 γ x (u)du.
In a similar way as in the proof of Lemma 1, one obtains ´(u-µ) 4 γ x (u)du ≤ C(1+(x-µ) 4 ).

Next, noting that since now γ is Laplace so g has Laplace tails, x → (1 + x 4 )g(x) is integrable, proceeding as in the proof of Lemma 1, one gets E 0 r 4 (α, 0, x) ≲ α as well as

E µ r 4 (α, µ, x) ≲ 1 + τ (α) 4
, for any fixed α. Similarly as in Lemmas 3-4, one then derives the following random α bounds

Er 4 (α, 0, x) ≲ cα * + P (α > cα * ) 1/2
and, for any µ,

Er 4 (α, µ, x) ≲ 1 + τ (α * ) 4 + (1 + log 2 n)P (α < α * ) 1/2 .
By using that the probabilities in the last displays go to 0 faster than 1/n, which we show below, and gathering the bounds for all i,

E θ 0 n i=1 ˆ(θ i -θ 0,i ) 4 dΠ α(θ | X) ≲ s n (1 + τ (α * ) 4 ) + nα * .
From this deduce that

E θ 0 Π α ∥θ -θ 0 ∥ 2 < v n | X ≲ P [A c ] + [s n (1 + τ (α * ) 4 ) + nα * ]/(s n g(ζ * ) -1 ) 2 ≲ P [A c ] + s -1 n (1 + τ (α * ) 4 )g(ζ * ) + s -1 n g(ζ * ).
The last bound goes to 0, as τ (α * ) ≤ ζ α * = ζ * and g has Laplace tails. To conclude the proof, we show that P θ 0 (α ∈ [α * , cα * ]) is small. From the proof of Theorem 13, one already has

P θ 0 [ α < α * ] ≤ exp(-cs n ), which is a o(1/n) using s n ≳ log 2 n.
To obtain a bound on P θ 0 [ α > cα * ], one can now revert the inequalities in the reasoning leading to the Bernstein bound in the proof of Theorem 13. With A = n i=1 m 1 (µ i , α), we have

P θ 0 [ α > cα * ] = P θ 0 [S(cα * ) > 0] = P θ 0 n i=1 β(θ 0,i + Z i , cα * ) -m 1 (θ 0,i , cα * ) > -A . But here, -A = (n -s n ) m(cα * ) -s n m 1 (ζ * , cα * ). As α → m(α) is increasing, m(cα * ) ≥ m(α * ). Now by Lemma 7, m 1 (ζ * , cα * ) ≤ (cα * ∧ c 3 ) -1 ≤ 1 cα * ,
provided α * ≤ c 3 /c = c 3 /16, which is the case for η n small enough. Since by definition n m(α * ) = s n /(4α * ), we have -A ≥ s n /(8α * ). From there one can carry over the same scheme of proof as for the previous Bernstein inequality, with now à = -A and Ṽ the variance proxy which is bounded by

Ṽ ≤ (n -s n )m 2 (0, cα * ) + s n m 2 (ζ * , cα * ) ≲ n m(cα * ) ζ cα * cα * + s n (cα * ) 2 . Now m(cα * ) ≲ Cg(ζ cα * ).
Using bounds similar to those of Lemma 8, one can check that

C 1 + ζ 2 α * ≤ ζ 2 cα * ≤ C 2 + ζ 2 α * , which implies that m(cα * )/ζ cα * ≲ m(α * )/ζ * ≲ m(α * ). From this one deduces, with M ≤ C/s n , Ṽ Ã2 + M Ã 3 Ã2 ≲ C ′ s n ,
which by Bernstein's inequality implies P θ 0 [ α > cα * ] ≤ exp[-Cs n ], which completes the proof of Theorem 14.

Technical lemmas for the SAS prior

Proofs of posterior risk bounds: fixed α

Proof of Lemma 1. First one proves the first two bounds. To do so, we derive moment bounds on γ x . Since γ x (•) is a density function, we have for any x, ´γx (u)du = 1. This implies (log g) ′ (x) = ´(u -x)γ x (u)du = ´uγ x (u)du -x. In Johnstone and Silverman ( 2004), the authors check, see p. 1623, that ´uγ x (u)du =: m1 (x) is a shrinkage rule, that is 0 ≤ m1 (x) ≤ x for x ≥ 0, so by symmetry, for any real x,

| ˆuγ x (u)du| ≤ |x|. Decomposing u 2 = (u -x) 2 + 2x(u -x) + x 2 and noting that ´(u -x) 2 γ x (u)du = g ′′ (x)/g(x) + 1, ˆu2 γ x (u)du = g ′′ g (x) + 1 + 2x g ′ g (x) + x 2 .
Note that for γ Laplace or Cauchy, we have |γ ′ | ≤ c 1 γ and |γ ′′ | ≤ c 2 γ. This leads to

|g ′ (x)| = | ˆγ′ (x -u)ϕ(u)du| ≤ c 1 ˆγ(x -u)ϕ(u)du = c 1 g(x)
and similarly |g ′′ | ≤ c 2 g, so that ´u2 γ x (u)du ≤ C(1 + x 2 ) which gives the first bound using (2.3.1). We note, en passant, that the one but last display also implies for any real

x that ˆu2 γ x (u)du ≥ 1 -c 2 -2c 1 |x| + x 2 , (2.4.1)
which implies that ´u2 γ x (u)du goes to ∞ with x. Also, for any real µ,

ˆ(u -µ) 2 γ x (u)du = (x -µ) 2 + g ′′ g (x) + 1 + 2(x -µ) g ′ g (x).
Now using again g ′ /g ≤ c 1 and g ′′ /g ≤ c 2 leads to

ˆ(u -µ) 2 γ x (u)du ≤ C(1 + (x -µ) 2 ).
By using the expression of r 2 (α, µ, x), this yields the second bound of the lemma. We now turn to the bounds in expectation. For a zero signal µ = 0, one notes that x = τ (α) is the value at which both terms in the minimum in the first inequality of the lemma are equal. So

E 0 r 2 (α, 0, x) ≲ ˆ1l |x|≤τ (α) α 1 -α g ϕ (x)ϕ(x)(1 + x 2 )dx + ˆ1l |x|>τ (α) (1 + x 2 )ϕ(x)dx.
For γ Cauchy, g has Cauchy tails and x → (1 + x 2 )g(x) is bounded, so one gets, with

α ≤ 1/2, E 0 r 2 (α, 0, x) ≲ α ˆ1l |x|≤τ (α) dx + τ (α)ϕ(τ (α)) + ϕ(τ (α))/τ (α) ≲ τ (α)α + τ (α)ϕ(τ (α)) ≲ τ (α)α + τ (α)αg(τ (α)) ≲ τ (α)α.
Turning to the last bound of the lemma, we distinguish two cases. Set for the remaining of the proof T := τ (α) for simplicity of notation. The first case is |µ| ≤ 4T , for which

E µ r 2 (α, µ, x) ≤ µ 2 + C ≤ C 1 (1 + T 2 ).
The second case is |µ| > 4T . We bound the expectation of each term in the second bound of the lemma (that for r 2 (α, µ, x)) separately. First,

E[a(x)(1 + (x -µ) 2 )] ≤ C. It thus suffices to bound µ 2 E µ [1 -a(x)].
To do so, one uses the bound (2.3.5) and starts by noting that, if Z ∼ N (0, 1),

E[1l |Z+µ|≤T ] ≤ P [|Z| ≥ |µ| -T ] ≤ P [|Z| ≥ |µ|/2].
This implies, with Φ(u) = ´∞ u ϕ(t)dt ≤ ϕ(u)/u for u > 0,

E µ [µ 2 1l |x|≤T ] ≤ C 2 |µ|ϕ(|µ|) ≤ C 3 . If A = {x, |x -µ| ≤ |µ|/2} and A c denotes its complement, √ 2πE µ [e -1 2 (|x|-T ) 2 ] ≤ ˆAc e -1 2 (x-µ) 2 dx + ˆA e -1 2 (|x|-T ) 2 dx.
The first term in the last sum is bounded above by 2 Φ(|µ|/2). The second term, as A ⊂ {x, |x| ≥ |µ|/2}, is bounded above by 2 Φ(|µ|/4). This implies, in the case |µ| > 4T , that

E µ r 2 (α, µ, x) ≤ C 4 + 4µ 2 Φ(|µ|/4) + 5 ≤ C.
The last bound of the lemma follows by combining the previous bounds in the two cases.

Proof of Lemma 2. From the expression of r 2 (α, 0, x) it follows

r 2 (α, 0, x) ≥ a(x) inf x∈R ˆu2 γ x (u)du ≥ α g ϕ ∨ g (x) inf x∈R ˆu2 γ x (u)du ≥ α inf x∈R g ϕ ∨ g (x) inf x∈R ˆu2 γ x (u)du ≥ C 0 α,
where c 0 > 0. Indeed, both functions whose infimum is taken in the last display are continuous in x, are strictly positive for any real x, and have respective limits 1 and +∞ as |x| → ∞, using (2.4.1), so these functions are bounded below on R by positive constants.

Proofs of posterior risk bounds: random α

Proof of Lemma 3. Using the bound on r 2 (α, 0, x) from Lemma 1,

r 2 (α, 0, x) = r 2 (α, 0, x)1l α≤α + r 2 (α, 0, x)1l α>α ≤ α 1 - α g ϕ (x) ∧ 1 (1 + x 2 )1l α≤α + C(1 + x 2 )1l α>α ≤ α 1 -α g ϕ (x) ∧ 1 (1 + x 2 )1l α≤α + C(1 + x 2 )1l α>α .
For the first term in the last display, one bounds the indicator from above by 1 and proceeds as in the proof of Lemma 1 to bound its expectation by Cατ (α). The first part of the lemma follows by noting that E[(1

+ x 2 )1l α>α ] is bounded from above by (2 + 2E 0 [x 4 ]) 1/2 P (α > α) 1/2 ≤ C 1 P (α > α) 1/2
by Cauchy-Schwarz inequality. The second part of the lemma follows from the fact that using Lemma 1,

r 2 (α, µ, x) ≤ (1 -a(x))µ 2 + Ca(x)((x -µ) 2 + 1) ≤ µ 2 + C(x -µ) 2 + C for any α.
Proof of Lemma 4. Combining (2.3.5) and the third bound of Lemma 1,

r 2 (α, µ, x) ≤ µ 2 1l |x|≤τ ( α) + e -1 2 (|x|-τ ( α)) 2 1l |x|>τ ( α) + C((x -µ) 2 + 1).
Note that it is enough to bound the first term on the right hand side in the last display, as the last one is bounded by a constant under E µ . Let us distinguish the two cases α ≥ α and α < α.

In the case α ≥ α, as τ (α) is a decreasing function of α,

1l |x|≤τ ( α) + e -1 2 (|x|-τ ( α)) 2 1l |x|>τ ( α) 1l α≥α ≤ 1l |x|≤τ ( α) + 1l τ ( α)<|x|≤τ (α) + e -1 2 (|x|-τ ( α)) 2 1l |x|>τ (α) 1l α≥α ≤ 1l |x|≤τ (α) + e -1 2 (|x|-τ (α)) 2 1l |x|>τ (α) ,
where we have used e -1 2 v 2 ≤ 1 for any v and that e

-1 2 (u-c) 2 ≤ e -1 2 (u-d) 2 if u > d ≥ c.
As a consequence, one can borrow the fixed α bound obtained previously so that

E [r 2 (α, µ, x)1 α≥α ] ≤ 2E µ r 2 (α, µ, x) ≤ C 1 + τ (α) 2 .
In the case α < α, setting b n = √ d log n and noting that τ (α) ≤ b n with probability 1 by assumption, proceeding as above, with b n now replacing τ (α), one can bound

1l |x|≤τ ( α) + e -1 2 (|x|-τ ( α)) 2 1l |x|>τ ( α) ≤ 1l |x|≤bn + e -1 2 (|x|-bn) 2 1l |x|>bn .
From this one deduces that

E µ 2 1l |x|≤τ ( α) + e -1 2 (|x|-τ ( α)) 2 1l |x|>τ ( α) 1l α<α ≤ C E µ µ 4 1l |x|≤bn + µ 4 e -(|x|-bn) 2 1/2 P (α < α) 1/2 .
Using similar bounds as in the fixed α case, one obtains

E µ µ 4 1l |x|≤bn + µ 4 e -(|x|-bn) 2 ≤ C(1 + b 4 n ).
Taking the square root and gathering the different bounds obtained concludes the proof.

Proofs on pseudo-thresholds

Proof of Lemma 8. For small α, or equivalently large ζ, we have

(g/ϕ)(ζ) = β(ζ) + 1 ≍ β(ζ).
Deduce that for large n, using η n = dα 1 m(α 1 ) and Lemma 5 on m,

η n ≍ α 1 ζ κ-1 1 g(ζ 1 ) β(ζ 1 ) β(ζ 1 ) ≍ ζ κ-1 1 ϕ(ζ 1 ) ≍ ζ κ-1 1 e -ζ 2 1 /2 .
From this deduce that

| log c + (κ -1) log ζ 1 - ζ 2 1 2 + log(1/η n )| ≤ C.
In particular, using log ζ ≤ a + ζ 2 /4 for some constant a > 0 large enough, one gets

ζ 2 1 ≤ 4(C + log(1/η n )) ≤ 4(C + log n).
Inserting this back into the previous inequality leads to

ζ 2 1 /2 ≤ log(1/η n ) + C + (1/2)(κ -1) log log n.
The lower bound is obtained by bounding (κ -1) log(ζ 1 ) ≥ 0, for small enough α 1 .

Proof of Lemma 9. Using (2.2.7), log(1/η n ) ≤ log(n) -2 log log n, and the bound on ζ from Lemma 8 gives

ζ 2 1 ≤ 2 log n -3 2 log log n, so that t(α 1 ) ≤ ζ(α 1 ) = ζ 1 ≤ √ 2 log n = t(α n ).
It follows that α 1 belongs to the interval [α n , 1] over which the likelihood is maximised.

Then one notices that { ζ < ζ 1 } = {α > α 1 } = {S(α 1 ) > 0}, regardless of the fact that the maximiser α is attained in the interior or at the boundary of [α n , 1]. So

P θ [ ζ < ζ 1 ] = P θ [S(α 1 ) > 0].
The score function equals S(α) = n i=1 β(X i , α), a sum of independent variables. By Bernstein's inequality, if W i are centered independent variables with |W i | ≤ M and n i=1 Var(W i ) ≤ V , then for any A > 0,

P n i=1 W i > A ≤ exp{- 1 2 A 2 /(V + 1 3 M A)}.
Set W i = β(X i , α 1 ) -m 1 (θ 0,i , α 1 ) and A = -n i=1 m 1 (θ 0,i , α 1 ). Then one can take M = c 3 /α 1 , using Lemma 7. One can bound -A from above as follows, using the definition of α 1 ,

-A ≤ - i / ∈S 0 m(α 1 ) + i∈S 0 c α 1 ≤ -(n -s n ) m(α 1 ) + cs n /α 1 ≤ -n m(α 1 )/2 + cdn m(α 1 ) ≤ -n m(α 1 )/4,
provided d is chosen small enough and, using again the definition of α 1 ,

V ≤ i / ∈S 0 m 2 (0, α 1 ) + i∈S 0 m 2 (θ 0,i , α 1 ) ≤ C α 1 (n -s n ) m(α 1 )ζ -κ 1 + cs n /α 1 ≤ Cα -1 1 n m(α 1 )ζ -κ 1 /2 + cdn m(α 1 ) ≤ C ′ dn m(α 1 )/α 1 ,
where one uses that ζ -1 1 is bounded. This leads to

V + 1 3 M A A 2 ≤ C ′ d nα 1 m(α 1 ) + 4c 3 3nα 1 m(α 1 ) ≤ c -1 5 nα 1 m(α 1 ) . One concludes that P [ α > α 1 ] ≤ exp{-c 5 nα 1 m(α 1 )} = exp{-Cs n } using (2.3.7).
Proof of Lemma 10. It is the same proof as Lemma 11 of [START_REF] Johnstone | Needles and straw in haystacks: empirical Bayes estimates of possibly sparse sequences[END_REF], but one has actually to be careful as one needs a positive lower bound on m 1 (1, α) (which cannot be true for every µ) to prove that m 2 (µ, α) ≤ Cm 1 (µ, α)/α. For more details, we refer to [START_REF] Castillo | Spike and slab empirical Bayes sparse credible sets[END_REF]'s proof of Lemma 18 or Lemma 24 and Lemma 25 of Chapter 3.

Proof of Lemma 11. First we check the existence of a solution. Set

ζ α = ζ(α) and R α := Φ(ζ α -ζ 1 )/(α m(α)). For α → 0 we have ζ α -ζ 1 → ∞ so by using Φ(u) ≍ ϕ(u)/u
as u → ∞ one gets, treating terms depending on ζ 1 as constants and using ϕ(

ζ α ) ≍ αg(ζ α ), Φ(ζ α -ζ 1 ) ≍ ϕ(ζ α -ζ 1 ) ζ α -ζ 1 ≍ αg(ζ α )e ζαζ 1 . As m(α) ≍ ζ α g(ζ α ), one gets R α ≍ e ζαζ 1 /ζ α → ∞ as α → 0.
On the other hand, with

π 1 ≤ s n /n and α 1 m(α 1 ) = ds n /n, R α 1 = 1 2α 1 m(α 1 ) = dn 2s n ≤ 8 π 1 d 16 , so that R α 1 < 8/π 1 as d < 2
. This shows that the equation at stake has at least one solution for α in the interval (0, α 1 ). By definition of m 1 (µ, α), for any µ and α, and

ζ = ζ(α), m 1 (µ, α) = ˆζ -ζ β(x) 1 + αβ(x) ϕ(x -µ)dx + ˆ|x|>ζ β(x) 1 + αβ(x) ϕ(x -µ)dx = (A) + (B).
By definition of ζ, the denominator in (B) is bounded from above by 2αβ(x) so

(B) ≥ 1 2α ˆ|x|>ζ ϕ(x -µ)dx ≥ 1 2 β(ζ) Φ(ζ -µ).
One splits the integral (A) in two parts corresponding to β(x) ≥ 0 and β(x) < 0. Let c be the real number such that g/ϕ(c) = 1. By construction the part of the integral (A)

with c ≤ |x| ≤ ζ is nonnegative, so, for α ≤ |β(0)| -1 /2, (A) ≥ ˆc -c β(x) 1 + αβ(x) ϕ(x -µ)dx ≥ - ˆc -c |β(0)| 1 -α|β(0)| ϕ(x -µ)dx ≥ -2|β(0)| ˆc -c ϕ(x -µ)dx,
where one uses the monotonicity of y → y/(1+αy). For µ ≥ c, the integral ´c -c ϕ(x-µ)dx is bounded above by 2 ´c 0 ϕ(x -µ)dx ≤ 2cϕ(µ -c). To establish (2.3.16), it thus suffices to show that (i

) := 4|β(0)|cϕ(ζ 1 -c) ≤ 1 4 β(ζ 3 ) Φ(ζ 3 -ζ 1 ) =: (ii).
The right hand-side equals 2 m(α 3 )/π 1 by definition of ζ 3 . Since γ is Cauchy, Lemma 5 gives m(α

3 ) ≍ ζ 3 g(ζ 3 ) ≍ ζ -1 3 . It is enough to show that (π 1 ζ 3 ) -1 is larger than Cϕ(ζ 1 -c), for suitably large C > 0.
Let us distinguish two cases. In the case ζ 3 ≤ 2ζ 1 , the previous claim is obtained, since ζ 1 goes to infinity with n/s n by Lemma 8 and ϕ(ζ

1 -c) = o(ζ -1
1 ). In the case ζ 3 > 2ζ 1 , we obtain an upper bound on ζ 3 by rewriting the equation defining it. For

t ≥ 1, one has Φ(t) ≥ Cϕ(t)/t. Since ζ 3 -ζ 1 > ζ 1 in the present case, it follows from the equation defining ζ 3 that C ϕ(ζ 3 -ζ 1 ) ζ 3 -ζ 1 ≤ 8α 3 m(α 3 )/π 1 .
This can be rewritten using

ϕ(ζ 3 -ζ 1 ) = √ 2πϕ(ζ 3 )ϕ(ζ 1 )e ζ 1 ζ 3 , as well as ϕ(ζ 3 ) = g(ζ 3 )α 3 /(1 + α 3 ) ≳ α 3 g(ζ 3 ) and m(α 3 ) ≍ ζ 3 g(ζ 3
). This leads to

e ζ 1 ζ 3 ζ 2 3 ≤ C π 1 e ζ 2 1 /2 .
By using e x /x 2 ≥ Ce x/2 for x ≥ 1 one obtains

ζ 2 1 e ζ 1 ζ 3 /2 ≤ e ζ 2 1 /2 C/π 1 , that is, using ζ 2 1 ≥ 1, π 1 ζ 3 ≤ π 1 ζ 1 + π 1 log(C/π 1 ) ζ 1 ≤ π 1 ζ 1 + C ≤ C ′ ζ 1 ,
using that u → u log(1/u) is bounded on (0, 1). So the previous claim is also obtained in this case, as ϕ(ζ 1 -c) is small compared to (C ′ ζ 1 ) -1 for large ζ 1 .

Proof of the convergence rate for the modified estimator

Proof of Theorem 17. The proof is overall in the same spirit as that of Theorem 2 in [START_REF] Johnstone | Needles and straw in haystacks: empirical Bayes estimates of possibly sparse sequences[END_REF] and goes by distinguishing the two cases s n ≥ log 2 n and s n < log 2 n. The main difference is that here we work with the full posterior distribution, and the risk bounds require Lemmas 1-4, that bound the posterior risk in various settings, as well as a result, Lemma 13 below, in the same vein. Also, we need to work with a modified version of ζ 1 , to make sure that the probability in Lemma 9 goes to 0 fast enough. We note that this version of ζ 1 is the one used in [START_REF] Johnstone | Needles and straw in haystacks: empirical Bayes estimates of possibly sparse sequences[END_REF] for both their Theorems 1 and 2 (in our Theorem 15, such a modification is not needed and we worked with the simpler version there). To do so, one replaces η n = s n /n in the definition (2.3.7) of α 1 by ηn = max η n , log 2 n n .

To keep notation simple, we still denote the corresponding threshold by ζ 1 . In the first part of the proof below, η n ≥ log 2 (n)/n, so this is the same version as in definition (2.3.7).

In the second part of the proof, we have ηn = log 2 n/n and we now indicate the relevant properties of the corresponding modified threshold ζ 1 . First, the statement of Lemma 8 becomes, with κ = 2 (as γ is Cauchy),

log(1/η n ) + c 1 ≤ ζ 2 1 2 ≤ log(1/η n ) + 1 2 log log n + c 2 . ( 2
P [ ζ < ζ 1 ] ≤ exp{-C ′ nα 1 m(α 1 )} ≤ exp{-Cnη n } ≤ e -C log 2 n .
(2.4.3)

We are now ready for the proof of Theorem 17. First consider the case s n ≥ log 2 n and let us show that the risk of the empirical Bayes posterior Π αA [• | X] is not larger than that of the non-modified one. One decomposes

E θ 0 ˆ∥θ -θ 0 ∥ 2 dΠ αA (θ | X) = E θ 0 ˆ∥θ -θ 0 ∥ 2 dΠ α(θ | X)1 t≤tn + E θ 0 ˆ∥θ -θ 0 ∥ 2 dΠ αA (θ | X)1 t>tn ≤ E θ 0 ˆ∥θ -θ 0 ∥ 2 dΠ α(θ | X) + E θ 0 ˆ∥θ -θ 0 ∥ 2 dΠ α A (θ | X)1 t>tn = (I) + (II).
The term (I) corresponds to the risk of the unmodified estimator, so is bounded as in Theorem 15. For (II), one splits it according to small and large signals θ 0,i : (II) = S + S, with

S = i: |θ 0,i |≤ζ 1 E θ 0 ˆ(θ i -θ 0,i ) 2 dΠ α A (θ i | X)1 t>tn ,
and S = (II) -S. From Lemma 1, one knows that r 2 (α A , µ, x) ≤ µ 2 + C(1 + (x -µ) 2 ), while for µ = 0, one can use the bound in expectation E 0 r 2 (α, 0, x) ≤ Cατ (α), so that

S ≤ { i: |θ 0,i |=0 + i: 0<|θ 0,i |≤ζ 1 }E θ 0 ˆ(θ i -θ 0,i ) 2 dΠ α A (θ i | X) ≤ Cnα A τ (α A ) + Cs n ζ 2 1 .
We now use the definition of α A to bound α A and τ (α A ). To bound τ (α A ), note that for any α ∈ (0, 1), by definition a(τ (α)) = 1/2, so for a signal of amplitude τ (α), the posterior puts 1/2 of its mass at zero, which means the posterior median is 0, implying

τ (α) ≤ t(α), so that τ (α A ) ≤ t A .
Combining with the bound for α A of Lemma 12,

nα A τ (α A ) ≤ Cn -A t 3 A .
For any fixed A > 0, this goes to 0 with n so it is a o(s n ζ 2 1 ), while s n ζ 2 1 is bounded by Cs n log(n/s n ) as follows from Lemma 8. Now to bound S, one adapts the last bound of Lemma 1 to accommodate for the indicator 1 t>tn . This is done in Lemma 13 whose bound (2.4.5) implies S ≤ Cs n t 2 A P ( t > t n ) 1/2 . This bound coincides up to a universal constant with the corresponding bound (128) in Johnstone and Silverman (2004) (taken for p = 0, p = 1 and q = 2, which corresponds to our setting, i.e. working with ℓ 0 classes and quadratic risk). So the remaining bounds of [START_REF] Johnstone | Needles and straw in haystacks: empirical Bayes estimates of possibly sparse sequences[END_REF] for the case s n > c log 2 n can be used directly (the distinction of the three cases as in [START_REF] Johnstone | Needles and straw in haystacks: empirical Bayes estimates of possibly sparse sequences[END_REF] p. 1646-1647 can be reproduced word by word, and is omitted for brevity), leading to S ≤ Cs n log(n/s n ).

Second, consider the case where s n ≤ log 2 n. We note that for this regime of s n , the inequalities (2.4.2) become, using that by definition ηn = log 2 n/n,

log n -2 log log n + c 1 ≤ ζ 2 1 2 ≤ log n - 3 2 log log n + c 2 . (2.4.4)
Let us show that the risk of the plug-in posterior using the modified estimator is at most of the order of the minimax risk. For ζ 1 as above,

E θ 0 ˆ∥θ -θ 0 ∥ 2 dΠ αA (θ | X) = i: θ 0,i =0 + i: 0<|θ 0,i |≤ζ 1 + i: |θ 0,i |>ζ 1 E θ 0 ˆ(θ i -θ 0,i ) 2 dΠ αA (θ i | X) =: (i) + (ii) + (iii).
For the terms (i) and (ii), apply respectively each bound of Lemma 3 with α

= α A to get (ii) ≤ Cs n [ζ 2 1 + 1] ≤ C ′ s n ζ 2 1 ≲ s n log n using (2.4.
2), which is bounded from above by Cs n log(n/s n ) in the regime s n ≤ log 2 n. Also,

(i) ≤ Cn α A τ (α A ) + P [ αA > α A ] 1/2 .
For large enough n, we have τ (α A ) = τ (α A ) which is less than t(α A ) = t A as noted above. Now α A is bounded using Lemma 12, so that

nα A τ (α A ) ≲ t A (1 + A)(log n)n -A = o(1) for A > 0.
We now bound the probability 53) in [START_REF] Johnstone | Needles and straw in haystacks: empirical Bayes estimates of possibly sparse sequences[END_REF]). Using (2.4.4), we have ζ 2 1 ≥ 2 log n -4 log log n + 2c 1 so, writing in slight abuse of notation t(ζ 1 ) = t(α 1 ) seeing t(•) as a function of

P [ αA > α A ] 1/2 . Recall the inequality t(α) 2 ≥ ζ(α) 2 -C (see e.g. (
ζ 1 instead of α 1 , t(ζ 1 ) 2 ≥ t 2 n + log log n -C + 2c 1 so that t(ζ 1 ) ≥ t n for n large enough. Deduce {α A > α A } = { t < t n } ⊂ { t < t(ζ 1 )} = { ζ < ζ 1 }. Using (2.4.3), we have P [ ζ < ζ 1 ] ≤ e -C
] = 0. Thus (iii) ≤ i: |θ 0,i |>ζ 1 E θ 0,i r 2 (α A , θ 0,i , X i ) ≤ C i: |θ 0,i |>ζ 1 (1 + τ (α A ) 2 + 0) ≤ Cs n τ (α A ) 2 , which is no more than 2Cs n (1 + A) log n ≤ C ′ s n log n. As s n ≤ c log 2 n, we have log n ≲ log(n/s n ) so (iii) ≤ Cs n log(n/s n ).
Putting the previous bounds together, one gets (i) + (ii) + (iii) ≤ Cs n log(n/s n ), which concludes the proof.

Lemma 12. For A ≥ 0, with t 2 A = 2(1 + A) log n and α A = t -1 (t A ), there exist N 0 > 0 and C > 0 both independent of A such that for n ≥ N 0 ,

α A ≤ C(1 + A)(log n)n -1-A . Proof. First recall the bound t(α) < ζ(α). Setting α = t -1 (u) in this inequality leads, using ζ(u) = β -1 (1/u), to u < β -1 (1/t -1 (u)). As β is increasing on R + , one has t -1 (u) < 1/β(u), so α A < 1 β(t A ) = g ϕ -g (t A ) ϕ g (t A ) ≤ 2 ϕ g (t A ) ≤ Ct 2 A e -t 2 A ,
where we use that g has Cauchy tails. The result follows by using the expression of t A .

Lemma 13. For any real µ, for B := { t > t n }, and α A , t A as above,

E µ [r 2 (α A , µ, x)1l B ] ≤ C(t 2 A + 1)P (B) 1/2 .
(2.4.5)

Proof. Similar to the proof of Lemma 1, one sets T := τ (α A ) and distinguishes two cases:

if |µ| ≤ 4T , Lemma 1 implies r 2 (α A , µ, x) ≤ µ 2 + (1 + (x -µ) 2
), so using Cauchy-Schwarz inequality,

E µ [r 2 (α A , µ, x)1l B ] ≤ CT 2 P (B) + P (B) + E µ [(x -µ) 4 ] 1/2 P (B) 1/2 ≤ C(1 + T 2 )P (B) 1/2 .
If |µ| > 4T , one uses the bound on r 2 from Lemma 1 again keeping the dependence in a(x). First,

E[a(x){1 + (x -µ) 2 }1l B ] ≤ E[{1 + (x -µ) 2 } 2 ] 1/2 P (B) 1/2 ≤ CP (B) 1/2 .
Let us now focus on

E µ [(1 -a(x))µ 2 1l B ] ≤ E µ [{1 |x|≤T + e -(|x|-T ) 2 /2 1 |x|>T }1l B ]. The first term, using P µ [|x| < T ] ≤ Φ(|µ|/2), is bounded by µ 2 Φ(|µ|/2) 1/2 P (B) 1/2 ≤ CP (B) 1/2 .
The second term is bounded by µ 2 {E µ [e (-|x|-T ) 2 ]} 1/2 P (B) 1/2 . In the proof of Lemma 1, we showed that E µ [e (-|x|-T ) 2 /2 ] 1/2 is bounded by a universal constant times Φ(|µ|/4). As e -y 2 ≤ e -y 2 /2 , the term at stake is bounded from above by µ 2 Φ(|µ|/4)P (B) 1/2 ≤ CP (B) 1/2 , which implies (2.4.5).

Proof of Theorem 16: the SSL prior

Recall that we use the notation of the SAS case, keeping in mind that every instance of g is replaced by g 1 and (some of the) ϕs by g 0 . Similarly, β(x, α), m, m 1 and m 2 are defined as in Section 2.3.1, but with β(x) = g 1 /g 0 -1.

The main steps of the proof generally follow those of Theorem 15, although technically there are quite a few differences. In the SSL case, we do not know whether the function β = g 1 /g 0 -1 is nondecreasing over the whole R + . Yet, we managed to show that β, which is an even function, is nondecreasing on the interval

J n = [2λ 1 , 2 log n],
see Proposition 2 below. This allows us to define its inverse β -1 = β |Jn -1 on this interval.

Further, we prove in Lemma 20 that β crosses the horizontal axis on the previous interval, is strictly negative on [0, 2λ 1 ] and tends to ∞ when x → ∞. As β is continuous, the graph of the function crosses any given horizontal line y = c, for any c > 0.

The threshold ζ in the SSL case. For every α ∈ (0, 1), one sets

ζ = ζ(α) = min{s > 0, β(s) = 1/α}. (2.5.1)
This is well defined by the property noted in the previous paragraph. Now one notes that g 0 ≤ 2ϕ for x ≤ λ 0 /2, see Lemma 19, and that the function g 1 /ϕ takes a value at √ 2 log n not smaller than Cn/ log n, since g 1 ≲ γ 1 has Cauchy tails. This implies the existence of a constant C > 1 such that

β( 2 log n) ≥ n/(C log n).
(2.5.2)

Now we claim that for any α ∈ (C log n/n, 1], we have the identity ζ(α) = β -1 (α -1 ). To see this, first note that for any α ∈ (C log n/n, 1], by (2.5.2) and β(2λ 1 ) < 0, we have

α -1 ∈ β(J n )
. This shows that t = β -1 (α -1 ) solves β(t) = α -1 . Also, it is the smallest possible solution t > 0, as β takes negative values on [0, 2λ 1 ], which establishes the identity.

The threshold ζ 1 in the SSL case. In the SSL case, the function α → m(α) = -E 0 [β(X, α)] is still nondecreasing, since for any real z, the map M z : α → z/(1 + αz) is nonincreasing and β(X, α) = M β(X) (α). By Proposition 3, we also have that m is positive for α ≥ C log n/n and is of the order of a constant for α = 1. So, the map α → α m(α) is nondecreasing on [C log n/n, 1], its value at C log n/n is less than C ′ log n/n, and its value at one is of the order of a constant. This shows, using s n ≥ c 1 log 2 n by (2.2.7), that the following equation has a unique solution α 1 ∈ (C log n/n, 1) (2.5.3) with d a small enough constant to be chosen later (see the proof of Lemma 21). Thus we can set

α 1 m(α 1 ) = ds n /n,
ζ 1 = β -1 (α -1 1 ),
and by the above arguments we have

ζ 1 ∈ J n . So Proposition 3 gives α -1 1 ≍ n sn ζ 1 g 1 (ζ 1 ) ≍ n snζ
1 . Now we can follow the same proof as in Lemma 8, replacing up to constants instances of g 0 (ζ 1 ) by ϕ(ζ 1 ) thanks to Lemma 17 and (2.6.5) (as

ζ 1 ≤ √ 2 log n < λ 0 /2), to obtain ζ 2 1 ≲ C log(n/s n ).
Defining τ (α) and τ (α). In the SSL case, we set

Ω(x, α) = α 1 -α 2g 1 ϕ (x).
This definition is as in the SAS case except that g is replaced by 2g 1 . We still use the same notation for simplicity. As g 1 satisfies the same properties as g, one defines τ (α) and τ (α) similarly to the SAS case. More precisely, τ (α) is the unique solution to the equation Ω(τ (α), α) = 1, whenever α ≤ α * , where Ω(0, α * ) = 1. One sets τ (α) = 0 for α ≥ α * and τ (α) = τ (α ∧ α 0 ) with τ (α 0 ) = λ 1 (this slightly differs from the SAS case).

As in the proof of Theorem 15, one can now decompose the risk

R n (θ 0 ) = E θ 0 ´∥θ - θ 0 ∥ 2 dΠ α(θ | X)
according to whether coordinates of θ correspond to a 'small' or 'large' signal, the threshold being ζ 1 that we define next. One can write

R n (θ 0 ) = i: θ 0,i =0 + i: 0<|θ 0,i |≤ζ 1 + i: |θ 0,i |>ζ 1 E θ 0 ˆ(θ i -θ 0,i ) 2 dΠ α(θ i | X).
We next use the first part of Lemma 16 with α = α 1 and the second part of the Lemma to obtain, for any θ 0 in ℓ 0 [s n ],

i: θ 0,i =0 + i: 0<|θ 0,i |≤ζ 1 E θ 0 ˆ(θ i -θ 0,i ) 2 dΠ α(θ i | X) ≤ C i: θ 0,i =0 α 1 τ (α 1 ) + P θ 0 (α > α 1 ) + λ -2 0 + i: 0<|θ 0,i |≤ζ 1 (θ 2 0,i + C) ≤ C(n -s n ) α 1 τ (α 1 ) + e -C log 2 n + λ -2 0 + (ζ 2 1 + C)s n ,
where for the last inequality we use Lemma 21. From (2.5.3) one gets

nα 1 ≲ s n ζ -1 1 g(ζ 1 ) -1 ≲ s n ζ 1 .
Let us now check that τ (α

1 ) ≤ ζ 1 . First, β(ζ 1 ) = α -1 1 > α -1 1 -1. By definition of τ (α 1 ), using ϕ ≤ 2g 0 by Lemma 17, α -1 1 -1 = 2(g 1 /ϕ)(τ (α 1 )) ≥ β(τ (α 1 )) + 1.
This gives us that β(ζ 1 ) ≥ β(τ (α 1 )) + 1 which implies the result as β is increasing here. Now with the previous bound on ζ 1 one obtains that the contribution to the risk of the indices i with |θ 0,i | ≤ ζ 1 is bounded by a constant times s n log(n/s n ).

It remains to bound the part of the risk for indexes i with |θ 0,i | > ζ 1 . To do so, one uses the second part of Lemma 16 with α chosen as α ′ 2 = C(log n/n), with C as in (2.5.2). By definition of α in (2.2.12), the probability that α is smaller than α ′ 2 equals zero. Also, one has τ (α ′

2

) 2 ≤ C log n. Indeed, setting ζ ′ 2 = β -1 (α ′ 2 -1
), we have as before

τ (α ′ 2 ) ≤ ζ ′ 2 ≤ √ 2 log n. This implies i: |θ 0,i |>ζ 1 E θ 0 ˆ(θ i -θ 0,i ) 2 dΠ α(θ i | X) ≤ Cs n log n,
which concludes the proof of Theorem 16.

Technical lemmas for the SSL prior

Fixed α bounds

As in the SAS case, we use the notation

r 2 (α, µ, x) = ˆ(u -µ) 2 dπ α (u | x)
, where now

π α (• | x)
is the posterior on one coordinate (X 1 , say) for fixed α in the SSL case, given

X 1 = x.
Lemma 14. For a zero signal µ = 0, we have for any x and α ∈ [0, 1/2],

r 2 (α, 0, x) ≤ C 1 ∧ α 1 -α g 1 ϕ (x) (1 + x 2 ) + ˆu2 γ 0,x (u)du E 0 r 2 (α, 0, x) ≤ Cτ (α)α + 4/λ 2 0 .
For an arbitrary signal µ ∈ R, we have that for any real x and α ∈ [0, 1/2],

r 2 (α, µ, x) ≤ (1 -a(x)) ˆ(u -µ) 2 γ 0,x (u)du + Ca(x)((x -µ) 2 + 1) E µ r 2 (α, µ, x) ≤ C(1 + τ (α) 2 ).
Proof. By definition, in the SSL case, r 2 (α, 0, x) = (1-a(x)) ˆu2 γ 0,x (u)du+a(x) ˆu2 γ 1,x (u)du.

Similar to Lemma 1, we have a

(x) ˆu2 γ 1,x (u)du ≤ C 1 ∧ α 1 -α g 1 g 0 (x) (1 + x 2
). The first bound now follows from the inequality g 0 ≥ ϕ/2 obtained in Lemma 17. For the bound in expectation,

E 0 ˆu2 γ 0,x (u)du = ˆ ˆu2 ϕ(x -u)γ 0 (u) g 0 (x) du ϕ(x)dx ≤ 2 ˆu2 ˆϕ(x -u)dxγ 0 (u)du = 2 ˆu2 γ 0 (u)du = 4/λ 2 0 ,
and one then proceeds as in Lemma 1 to obtain the bound for zero signal.

Now for a general signal µ, the bound for r 2 (α, µ, x) follows from the definition and the previous bound. For the bound in expectation, by symmetry one can assume µ ≥ 0. Also note that the term with the a(x) factor is bounded in expectation by a constant, by using a(x) ≤ 1. To handle the term with 1 -a(x), we distinguish two cases. First, one assumes that µ ≤ λ 0 /2. We have, using

(a + b) 2 ≤ 2a 2 + 2b 2 , (1 -a(x)) ˆ(u -µ) 2 γ 0,x (u)du ≲ (1 -a(x))µ 2 + (1 -a(x)) ˆu2 ϕ(x -u) γ 0 (u) g 0 (x) du.
For the first term we proceed as in Lemma 1, for the second using g 0 ≥ ϕ/2 from Lemma 17,

E µ (1 -a(x)) ˆu2 ϕ(x -u) γ 0 (u) g 0 (x) du ≤ 2 ˆu2 γ 0 (u) ˆϕ(x -u)ϕ(x -µ) ϕ(x) dxdu ≲ ˆu2 γ 0 (u) ˆe-(x-(u+µ)) 2 /2+uµ dxdu ≲ λ 0 ˆu2 e -λ 0 |u|+uµ du.
As µ ≤ λ 0 /2, this is in turn bounded by a constant times (λ 0 ) -2 . Now in the case that µ > λ 0 /2, recall from the proof of Lemma 1 that for any real x,

ˆ(u -µ) 2 γ 0,x (u)du = (x -µ) 2 + 1 + g ′′ 0 g 0 (x) + 2(x -µ) g ′ 0 g 0 (x).
(2.6.1)

The first two terms are, in expectation, bounded by a constant. Next one writes

E µ (1 -a(x)) g ′′ 0 g 0 (x) = ˆ(1 -a(x)) g ′′ 0 g 0 (x)ϕ(x -µ)dx
By Lemma 17, we have |g ′′ 0 | = λ 2 0 |g 0 -ϕ| ≤ 1. One splits the integral on the last display in two parts. For |x| ≤ µ/4, one uses that g ′′ 0 is bounded together with the bound g 0 ≥ ϕ/2. For |x| > µ/4, one uses g ′′ 0 /g 0 = λ 2 0 (g 0 -ϕ)/g 0 ≤ λ 2 0 together with 1-a(x) ≤ (g 0 /g 1 )(x)/α, which follows from the expression of a(x). This leads to

E µ (1 -a(x)) g ′′ 0 g 0 (x) ≤ ˆ|x|≤µ/4 e xµ-µ 2 2 dx + λ 2 0 α ˆ|x|>µ/4 g 0 g 1 (x)ϕ(x -µ)dx.
The first term in the last expression is bounded. The second one is bounded by a constant given our choice of λ 0 by combining the following: α -1 ≤ n from (2.2.12), g 0 ≲ γ 0 for µ > λ 0 /8 from (2.6.6) and g 1 ≳ γ 1 .

To conclude the proof, for the last term in (2.6.1), using (2.6.4), the bound on 1-a(x) from Lemma 15 below, and the fact that

x → xϕ(x) is bounded, E µ 2(1 -a(x))(x -µ) g ′ 0 g 0 (x) is bounded by 2 ˆ(1 -a(x))| g ′ 0 g 0 (x)||(x -µ)ϕ(x -µ)|dx ≲ ˆ(1 -a(x))|x|dx ≲ ˆ|x|≤τ(α) |x|dx + ˆτ(α)≤|x|≤ λ 0 2 |x|e -(|x|-τ (α)) 2 2 dx + ˆ|x|≥ λ 0 2 |x|(1 -a(x))dx ≲ τ (α) 2 + 2(1 -e -( λ 0 2 -τ (α)) 2 2 ) + τ (α) + ˆ|x|≥ λ 0 2 n 3 |x| γ 0 γ 1 (x)dx ≲ 1 + τ (α) 2 .
Lemma 15. For any x ∈ [0, λ 0 /2] and α ∈ [0, 1],

1 -a(x) ≤ 1l |x|≤τ (α) + 4e -1 2 (|x|-τ (α)) 2 1l |x|>τ (α) .
Proof. One first notes that 1 -a(x) ≤ 4Ω(x, α) -1 for x ≤ λ 0 /2, using the fact that for such x, g 0 (x) ≤ 2ϕ(x) as found in Lemma 19. The following inequalities hold for τ (α) ≤ x ≤ λ 0 /2, using τ (α) ≥ λ 1 by definition and that |(log g 1 ) ′ | ≤ λ 1 as seen in (2.6.3),

Ω(x, α) = Ω(τ (α), α) exp ˆx τ (α) ((log g 1 ) ′ (u) -(log ϕ) ′ (u))du ≥ exp ˆx τ (α) (u -λ 1 )du ≥ exp ˆx τ (α) (u -τ (α))du = e (x-τ (α)) 2 2
.

Random α bounds

Lemma 16. Let α be a fixed non-random element of (0, 1). Let α be a random element of [0, 1] that may depend on x ∼ N (0, 1) and on other data. Then there exists C 1 > 0 such that

Er 2 (α, 0, x) ≤ C 1 ατ (α) + P (α > α) 1/2 + 4 λ 2 0 .
There exists C 2 > 0 such that for any real µ, if x ∼ N (µ, 1),

Er 2 (α, µ, x) ≤ µ 2 + C 2 .
Suppose now that τ (α) 2 ≤ d log(n) with probability 1 for some d > 0, and that x ∼ N (µ, 1). Then there exists C 2 > 0 such that for all real µ,

Er 2 (α, µ, x) ≤ C 2 1 + τ (α) 2 + (1 + d log n)P (α < α) 1/2 .
Proof of Lemma 16. For the first two inequalities, the proof is the same as in the SAS case in Lemma 3, the only difference being the presence of the term 4/λ 2 0 coming from Lemma 14 for the first inequality. For the third inequality , it follows from Lemma 14 that

r 2 (α, µ, x) ≤ (1 -a α(x)) ˆ(u -µ) 2 γ 0,x (u)du + C[(x -µ) 2 + 1].
In expectation the last term is constant. For the first term, with Lemma 15,

1 -a α(x) ≤ 1l |x|≤τ ( α) + 4e -1 2 (|x|-τ ( α)) 2 1l λ 0 2 ≥|x|>τ ( α) + 1l |x|≥ λ 0 2 n g 0 g 1 (x),
where the last estimate uses the bound α ≥ 1/n. As in Lemma 4, let us distinguish the two cases α ≥ α and α < α. In the case α ≥ α, as τ (α) is a decreasing function of α,

1l |x|≤τ ( α) + 4e -1 2 (|x|-τ ( α)) 2 1l λ 0 2 ≥|x|>τ ( α) 1l α≥α ≲ 1l |x|≤τ ( α) + 1l τ ( α)<|x|≤τ (α) + e -1 2 (|x|-τ ( α)) 2 1l λ 0 2 ≥|x|>τ (α) 1l α≥α ≲ 1l |x|≤τ (α) + e -1 2 (|x|-τ (α)) 2 1l λ 0 2 ≥|x|>τ (α) ,
where we have used e -1 2 v 2 ≤ 1 for any v and that e

-1 2 (u-c) 2 ≤ e -1 2 (u-d) 2 if u > d ≥ c. For the third term, we have to control E µ 1l |x|≥ λ 0 2 n g 0 g 1 (x) ˆ(u -µ) 2 γ 0,x (u)du .
To do so, one uses (2.6.1). In expectation, the term in factor of (x -µ) 2 + 1 is bounded by a constant. Using (2.6.6) and the fact that g ′′ 0 /g 0 ≤ λ 2 0 , the term in factor g ′′ 0 /g 0 is bounded by

λ 2 0 n ˆ|x|≥ λ 0 2 g 0 g 1 (x)ϕ(x -µ)dx ≲ n 3 ˆ|x|≥ λ 0 2 γ 0 γ 1 (x)dx ≲ n 4 ˆ|x|≥ λ 0 2
x 2 e -λ 0 |x| dx ≲ n 4 e -Cn 2 .

Finally, using (2.6.4) and the fact that x → xϕ(x) is bounded, one obtains

E µ 1l |x|≥ λ 0 2 n g 0 g 1 (x)(x -µ) g ′ 0 g 0 (x) ≤ ˆ|x|≥ λ 0 2 n g 0 g 1 (x)|x||(x -µ)ϕ(x -µ)|dx ≲ ˆ|x|≥ λ 0 2 n g 0 g 1 (x)|x|dx.
As a consequence, one can borrow the fixed α bound obtained previously so that

E [r 2 (α, µ, x)1 α≥α ] ≲ E µ r 2 (α, µ, x) ≲ 1 + τ (α) 2 .
In the case α < α, setting b n = √ d log n and noting that τ (α) ≤ b n with probability 1 by assumption, proceeding as above, with b n now replacing τ (α), one can bound

1l |x|≤τ ( α) + 4e -1 2 (|x|-τ ( α)) 2 1l λ 0 2 ≥|x|>τ ( α) + 1l |x|≥ λ 0 2 n g 0 g 1 (x) ≲ 1l |x|≤bn + e -1 2 (|x|-bn) 2 1l λ 0 2 ≥|x|>bn + 1l |x|≥ λ 0 2 n g 0 g 1 (x).
From this one deduces that E (1 -a α(x)) ˆ(u -µ) 2 γ 0,x (u)du is bounded from above by a constant times

  E µ   ˆ(u -µ) 2 γ 0,x (u)du 2 [1l |x|≤bn + e -(|x|-bn) 2 ]     1/2 P (α < α) 1/2 .
Using the same bounds but squared as in the fixed α case, one obtains that the expectation in the last display is bounded from above by C(1 + b 4 n ). Taking the square root and gathering the different obtained bounds concludes the proof.

Properties of the functions g 0 and β for the SSL prior

Recall the notation ϕ, γ 0 , g 0 from Section 2.2. For any real x, we also write ψ(x) = ´∞ x e -u 2 /2 du. Our key result on β is the following.

Proposition 2. β = g 1 g 0 -1 is strictly increasing on [2λ 1 ; √ 2 log n].
We next state and prove some Lemmas used in the proof of Proposition 2 below.

Lemma 17. The convolution g 0 = ϕ * γ 0 satisfies g ′′ 0 = λ 2 0 (g 0 -ϕ) as well as

1 g 0 ≤ 2 ϕ and |g 0 -ϕ| ≤ 1 λ 2 0 .
Proof. The first identity follows by differentiation. One computes g 0 (x) by separating the integral in a positive and negative part to get, for any real x,

g 0 (x) = λ 0 e λ 2 0 2 2 √ 2π e λ 0 x ψ(λ 0 + x) + e -λ 0 x ψ(λ 0 -x) .
(2.6.2)

Now combining the standard inequality (1 -x -2 )e -x 2 /2 ≤ xψ(x) ≤ e -x 2 /2 , for x > 0, with the expression of g 0 (0) obtained from (2.6.2), we get 1 2 ≤ g 0 ϕ (0) ≤ 1 for large enough n. By [START_REF] Johnstone | Needles and straw in haystacks: empirical Bayes estimates of possibly sparse sequences[END_REF], Lemma 1, the function g 0 /ϕ is increasing, which implies the first inequality of the lemma.

The approximation property of ϕ by g 0 is obtained by a Taylor expansion. For any

x, u ∈ R, there exists c between x and x -u such that ϕ

(x -u) -ϕ(x) = uxϕ(x) + u 2 (c 2 -1)ϕ(c)/2, so that 2(g 0 (x) -ϕ(x)) = ˆ(2uxϕ(x) + u 2 (c 2 -1)ϕ(c))γ 0 (u)du = ˆu2 (c 2 -1)ϕ(c)γ 0 (u)du,
whose absolute value is bounded by ´u2 |c 2 -1|ϕ(c)γ 0 (u)du. This is less than ´u2 γ 0 (u)du = λ -2 0 .

Lemma 18. Let L 0 = 5 √ 2π. Then for all x ∈ [0; 2 log(λ 0 /L 0 )],

(log g 0 ) ′ (x) ≤ -x/2.

Proof. Let g o+ (x) = ˆ∞ 0 ϕ(v + x)γ 0 (v)dv and g o-(x) = ˆ0 -∞ ϕ(v + x)γ 0 (v)dv.
First we check that for any x in the prescribed interval, we have

λ 0 (g o+ -g o-)(x) ≤ -x(ϕ(x) -2/λ 0 ) ≤ 0.
For any real x, using the inequality

e v ≥ 1 + v, g o-(x) = ˆ∞ 0 ϕ(x -u)γ 0 (u)du = ˆ∞ 0 ϕ(x + u)e 2xu γ 0 (u)du ≥ ˆ∞ 0 ϕ(x + u)(1 + 2xu)γ 0 (u)du ≥ g o+ (x) + λ 0 x ˆ∞ 0 uϕ(x + u)e -λ 0 u du.
Setting ∆(x) = ˆ∞ 0 uϕ(x + u)e -λ 0 u du, one can write

∆(x) = ˆ∞ 0 u(ϕ(x + u) -ϕ(x))e -λ 0 u du + ϕ(x) ˆ∞ 0 ue -λ 0 u du = ˆ∞ 0 u(ϕ(x + u) -ϕ(x))e -λ 0 u du + ϕ(x)/λ 2 0 .
As ϕ is 1-Lipshitz, one can bound from below ϕ(x + u) -ϕ(x) ≥ -u, which leads to, for any x ≥ 0,

∆(x) ≥ - ˆ∞ 0 u 2 e -λ 0 u du + ϕ(x)/λ 2 0 ≥ -2/λ 3 0 + ϕ(x)/λ 2 0 .
This leads to inequality on g o+ -g o-above, using that x belongs to the prescribed interval to get the nonpositivity. From this one deduces

g ′ 0 (x) = λ 0 (g o+ -g o-)(x) ≤ -x(ϕ(x) -2/λ 0 ). This now implies g ′ 0 g 0 (x) ≤ -x ϕ(x) -2λ -1 0 ϕ(x)+λ -2 0
On the prescribed interval ϕ(x) ≥ 5/λ 0 , so using that t → (t -a)/(t + b) is increasing,

g ′ 0 g 0 (x) ≤ -x 5λ -1 0 -2λ -1 0 5λ -1 0 + λ -2 0 = - 3x 5 + λ -1 0 ≤ - x 2 ,
for large enough n, which concludes the proof.

Proof of Proposition 2. We will firstly note that if G 1 has a Cauchy(1/λ 1 ) law,

|(log g 1 ) ′ (x)| ≤ λ 1 .
(2.6.3) Indeed, for any real x, recalling that γ

1 (x) = (λ 1 /π)(1 + λ 2 1 x 2 ) -1 , one sees that γ ′ 1 (x)/γ 1 (x) = (-2λ 2 1 x)/(1 + 2λ 2 1 x 2 ) and |γ ′ 1 (x)/γ 1 (x)| ≤ 2 √ 2λ 1 /3. This implies (2.6.3), as |(log g 1 ) ′ (x)| = | ˆϕ(x -u)γ ′ 1 (u)du|/g 1 (x) ≤ 2 √ 2 3 λ 1 ˆϕ(x -u)γ 1 (u)du/g 1 (x) ≤ 2 √ 2 3 λ 1 ≤ λ 1 .
Let (x, y) ∈ [2λ 1 ; λ 0 /4] 2 with x ≤ y. Using Lemma 18 one can find c ∈ [x; y] with log(g 0 (x)/g 0 (y)) = (x -y)(log g 0 ) ′ (c) ≥ (x -y)(-c/2) ≥ (y -x)x/2. On the other hand, by (2.6.3) one deduces that for some c ∈ [x; y], we have log(g 1 (x)/g 1 (y)) = (x -y)(log g 1 ) ′ (c) ≤ (y -x)λ 1 . Thus for any x, y as before,

g 1 (x) g 1 (y)
≤ e (y-x)λ 1 and e (y-x) x 2 ≤ g 0 (x) g 0 (y) .

As x ≥ 2λ 1 by assumption, this leads to the announced inequality.

Lemma 19. For n large enough, recalling that λ 0 depends on n, we have (log g 0 ) ′ (x) ≥ -x for any x > 0, (2.6.4)

g 0 (x) ≤ 2ϕ(x)
for any 0 ≤ x ≤ λ 0 /2, (2.6.5)

g 0 (x) ≲ γ 0 (x)
for any x ≥ λ 0 /8.

(2.6.6)

Proof. For any real x, we set µ 0,1 (x) = ˆuϕ(x -u)γ 0 (u)

g 0 (x) du, the expectation of γ 0,x . A direct computation shows, for x > 0 that (log g 0 ) ′ (x) = -x + µ 0,1 (x). But µ 0,1 (x) = ˆ∞ 0 u λ 0 ϕ(x -u)e -λ 0 u 2g 0 (x) du + ˆ0 -∞ u λ 0 ϕ(x -u)e λ 0 u 2g 0 (x) du = ˆ∞ 0 u λ 0 e -λ 0 u 2g 0 (x) (ϕ(x -u) -ϕ(x + u))du = ˆ∞ 0 u λ 0 e -λ 0 u 2g 0 (x) ϕ(x + u)(e 2xu -1)du ≥ 0,
which leads to (2.6.4). For the second point, we first prove the identity, for x > 0,

g 0 (x) = e λ 2 0 /2 √ 2π ψ(λ 0 )γ 0 (x)+ϕ(x) λ 0 2 e (λ 0 -x) 2 /2 (ψ(λ 0 -x) -ψ(λ 0 )) + e (λ 0 +x) 2 /2 ψ(λ 0 + x) . Indeed, g 0 (x) = ˆ∞ 0 ϕ(u)(γ 0 (x+u)+γ 0 (x-u))du = γ 0 (x) ˆ∞ 0 ϕ(u)e -λ 0 u du+ ˆ∞ 0 ϕ(u)γ 0 (x- u)du, for x > 0. The first term equals e λ 2 0 /2 ψ(λ 0 )γ 0 (x)/ √ 2π.
The second one equals

ˆ∞ -x ϕ(x + v)γ 0 (v)dv = ϕ(x) ˆ∞ -x e -v 2 2 -vx γ 0 (v)dv = ϕ(x) λ 0 2 ˆ0 -x e -v 2 2 -vx+λ 0 v dv + ˆ∞ 0 e -v 2 2 -vx-λ 0 v dv = ϕ(x) λ 0 2 ˆx 0 e -v 2 2 +vx-λ 0 v dv + e (x+λ 0 ) 2 2 ˆ∞ 0 e -(v+x+λ 0 ) 2 2 dv = ϕ(x) λ 0 2 e (λ 0 -x) 2 2 ˆλ0 λ 0 -x e -u 2 2 du + e (x+λ 0 ) 2 2 ψ(x + λ 0 )
which gives the announced identity. If x ≤ λ 0 /2, using the inequality yψ(y) ≤ e -y 2 /2 for y > 0, we have

g 0 (x) ≤ λ -1 0 γ 0 (x)/ √ 2π + ϕ(x)(λ 0 /2) (λ 0 -x) -1 + (λ 0 + x) -1 .
This leads, using

γ 0 (x)/λ 0 ≤ e -λ 2 0 /2 for x ≤ λ 0 /2, to g 0 (x) ≤ ϕ(x)(1/2 + 1 + 1/2) = 2ϕ(x).
For the third point, if x ≥ λ 0 /8, the first term is bounded as follows:

λ 0 e λ 2 0 /2 e λ 0 x ψ(λ 0 + x) ≤ λ 0 e λ 2 0 /2 e λ 0 x e -λ 2 0 /2-x 2 /2-λ 0 x (λ 0 + x) -1 ≤ λ 0 (λ 0 + x) -1 e -x 2 /2 ≤ λ 0 (9λ 0 /8) -1 e -x 2 /2 . Now ψ(λ 0 -x) ≤ e -λ 2 0 /2-x 2 /2+λ 0 x (λ 0 -x) -1 ≤ 4e -λ 2 0 /2-x 2 /2+λ 0 x λ -1 0 if λ 0 /8 ≤ x ≤ 3λ 0 /4, which leads to g 0 (x) ≲ ϕ(x). If x ≥ 3λ 0 /4 one bounds the second term by λ 0 e λ 2 0 /2-λ 0 x ≤ λ 0 e 2λ 0 x/3-λ 0 x ≤ λ 0 e -λ 0 x/3 , so that, for x ≥ λ 0 /8, g 0 (x) ≲ γ 0 (x). The next lemma is useful to control β outside [2λ 1 , √ 2 log n].
Lemma 20. Set λ 1 = 0.05. For n large enough, for some C > 0, we have

(g 1 /g 0 )(2λ 1 ) < 0.25, β(x) < 0 for all x ∈ [0, 2λ 1 ], β(x) ≳ n/log n, for all 2 log n ≤ x ≤ λ 0 /2, β(x) ≳ e Cn 2 γ 1 (n)/n for all x ≥ λ 0 /8. Proof. 1) We have g 1 g 0 (2λ 1 ) ≤ λ 1 √ 2π λ 0 ´e-(u-2λ 1 ) 2 /2 e -λ 0 |u| du
. For the denominator, we have

ˆe-(u-2λ 1 ) 2 /2 e -λ 0 |u| du ≥ ˆ∞ 0 e -(u-2λ 1 ) 2 /2-λ 0 u du ≥ e λ 2 0 /2-2λ 1 λ 0 ˆ∞ 0 e -(u-(2λ 1 -λ 0 )) 2 /2 du ≥ e -2λ 2 1 ψ(λ 0 -2λ 1 )/(λ 0 -2λ 1 ) ≥ e -2λ 2 1 (λ 0 -2λ 1 ) -1 (1 -(λ 0 -2λ 1 ) -2 ) ≥ 0.99e -2λ 2 1 (λ 0 -2λ 1 ) -1 for n large enough .
This implies (g 1 /g 0 )(2λ 1 ) < 0.25 for λ 1 = 0.05.

2) Let x ∈ [0, 2λ 1 ], using Lemma 17, we have β ≤ 2g 1 /ϕ -1. As the last function is increasing as we know from the SAS case, we have β(x) ≤ 2(g 1 /ϕ)(2λ 1 ) -1. With (2.6.5) we end up with β(x) ≤ 4(g 1 /ϕ)(2λ 1 ) -1, which is strictly negative by the first point.

3

) Let x ∈ [ √ 2 log n, λ 0 /2]. With (2.6.5), we have β(x) ≥ (g 1 /2ϕ)(x) -1 ≥ (g 1 /2ϕ)( √ 2 log n) -1, and as g 1 ≳ γ 1 , we end up with β(x) ≳ n/ log n. 4) For x ≥ λ 0 /8, via (2.6.6) we have β(x) + 1 ≥ (γ 1 /γ 0 )(x) ≥ (γ 1 /γ 0 )(λ 0 /8) which gives the result.

Bounds on moments of the score function

Recall that, for all

k ≥ 1, µ ∈ R and α ∈ [0, 1], m k (µ, α) = E[β(Z + µ) k ] where Z ∼ N (0, 1), and m(α) = -m 1 (0, α) = -2 ´∞ 0 β(z, α)ϕ(z)dz. Proposition 3. With κ as in (2.3.6), there exist constants D 1 and D 2 such that for α ∈ (C log n/n, 1], D 1 ζ κ-1 g 1 (ζ) ≤ m(α) ≤ D 2 ζ κ-1 g 1 (ζ). Also, c ≤ m(1) ≤ C with c, C independent of n. Proof. Recall that for α ∈ (C log n/n, 1], we have ζ = β -1 (α -1 ) and ζ ≤ √ 2 log n. m(α) = -2 ˆ∞ 0 β(z) 1 + αβ(z) ϕ(z)dz = -2 ˆ∞ 0 β(z)ϕ(z)dz + 2 ˆ∞ 0 αβ 2 (z) 1 + αβ(z) ϕ(z)dz = -2 ˆ∞ 0 β(z)ϕ(z)dz + 2 ˆζ 0 αβ 2 (z) 1 + αβ(z) ϕ(z)dz +2 ˆ∞ ζ αβ 2 (z) 1 + αβ(z) ϕ(z)dz := A + B + C
• For the first term, with K a positive constant one can write

A = 2 ˆ∞ 0 (ϕ - g 1 g 0 ϕ) = 2 ˆ∞ 0 (ϕ - g 1 g 0 (ϕ -g 0 + g 0 )) = 2 ˆ∞ 0 (ϕ -g 1 ) + 2 ˆ∞ 0 g 1 (g 0 -ϕ) g 0 = 0 + 2 ˆKζ 0 g 1 (g 0 -ϕ) g 0 + 2 ˆ∞ Kζ g 1 (g 0 -ϕ) g 0 := (i) + (ii).
Using the fact that g 1 /ϕ is increasing, we have

|(i)| ≤ 2λ -2 0 ˆKζ 0 g 1 /g 0 ≤ 4λ -2 0 ˆKζ 0 g 1 /ϕ ≤ 4Kζg 1 (Kζ)λ -2 0 /ϕ(Kζ) ≲ Kn K 2 -2 ζg 1 (Kζ)
Taking K = 6/5, we end up with |(i)| ≲ ζn -2/5 g 1 (6ζ/5) and this term is strictly dominated by ζ κ-1 g 1 (ζ). By Lemma 17, and the fact that g 1 ≍ γ 1 , we have :

|(ii)| ≤ 2 ˆ∞ Kζ g 1 (1 + ϕ/g 0 ) ≤ 6 ˆ∞ Kζ g 1 ≲ (6ζ/5) κ-1 g 1 (6ζ/5) using (2.3.6)
This term too is dominated by ζ κ-1 g 1 (ζ).

• For the second term, we use the fact that on (0, ζ), α|β| < 1, so 1 + b 0 ≤ 1 + αβ ≤ 2, where b 0 = g 1 (2λ 1 )/2ϕ(0) -1 does not depend on n, so that

B ≍ ˆζ 0 αβ 2 (z)ϕ(z)dz
We will now use the fact that, with h := g 2 1 /ϕ, ´ζ 0 h(z)dz ≤ 16h(ζ)/ζ. This is a direct corollary of lemma 4 in [START_REF] Johnstone | Needles and straw in haystacks: empirical Bayes estimates of possibly sparse sequences[END_REF]. We have, also using (2.6.5):

ˆζ 0 β 2 (z)ϕ(z)dz ≲ ˆζ 0 (g 2 1 /g 2 0 )ϕ ≲ ˆζ 0 g 2 1 /ϕ ≲ g 2 1 (ζ)/(ζϕ(ζ)) ≲ β(ζ)g 1 (ζ)/ζ ≲ g 1 (ζ)(αζ) -1 hence B ≲ g 1 (ζ)ζ -1 , dominated by ζ κ-1 g 1 (ζ).
• For the last term, we first use the fact that αβ(z

) < 1 + αβ(z) , so that C ≲ ˆ∞ ζ β(z)ϕ(z)dz. C ≲ ˆ∞ ζ g 1 ϕ/g 0 ≲ ˆ∞ ζ g 1 (z)dz using Lemma 17 ≍ ζ κ-1 g 1 (ζ) using (2.3.6)
For an upper bound we write

C = 2 ˆλ0 /2 ζ αβ 2 (z) 1 + αβ(z) ϕ(z)dz + 2 ˆ∞ λ 0 /2 αβ 2 (z) 1 + αβ(z) ϕ(z)dz =: (i) + (ii).
For the first term, using (2.6.5), we have for every

z ∈ [ζ, λ 0 /2], β(z) ≥ g 1 2ϕ (z) -1 ≥ g 1 4ϕ (z) and α g 1 4ϕ (z) ≳ α n log n ≳ 1, so that (i) ≥ 2 ˆλ0 /2 ζ α(g 2 1 /16ϕ 2 )(z) 1 + α(g 1 /4ϕ)(z) ϕ(z)dz ≳ ˆλ0 /2 ζ g 1 (z)dz ≳ ζ κ-1 g 1 (ζ)
For the second term, we have

(ii) ≲ ˆ∞ λ 0 /2 β(z)ϕ(z)dz ≲ ˆ∞ λ 0 /2 g 1 (z)dz ≲ λ κ-1 0 g 1 (λ 0 ) ≲ λ -1 0 .
Putting the bounds together finally leads to m(α)

≍ g 1 (ζ)ζ κ-1 . To prove m(1) ≤ ϕ(0)/g 1 (2λ 1 ), write m(1) = -2 ˆ∞ 0 ϕ + 2 ˆ∞ 0 ϕ/(1 + β). Now ˆ∞ 0 ϕ/(1 + β) = ˆ2λ 1 0 ϕ/(1 + β) + ˆλ0 /2 2λ 1 ϕ/(1 + β) + ˆ+∞ λ 0 /2 ϕ/(1 + β).
Using that on [0, 2λ 1 ], 1 + β ≥ 1 + b 0 = g 1 (2λ 1 )/2ϕ(0) and (2.6.5) and (2.6.6), we have

ˆ∞ 0 ϕ/(1 + β) ≤ ˆ2λ 1 0 ϕ/(1 + b 0 ) + ˆλ0 /2 2λ 1 ϕ 2 /g 1 + ˆ∞ λ 0 /2 γ 0 ϕ/g 1 ≤ ˆ2λ 1 0 ϕ/(1 + b 0 ) + ˆ∞ 2λ 1 ϕ 2 /g 1 + ˆ∞ 0 ϕ/g 1 ≤ C.
For the lower bound, recall that m(1) = -2

ˆ∞ 0 ϕ + 2 ˆ∞ 0 ϕ/(1 + β) and use Lemma 17 to write 2 ˆ∞ 0 ϕ/(1 + β) ≥ ˆ∞ 0 ϕ 2 /g 1 which does not depend on n. Proposition 4. Let α ∈ [C log n/n, 1]. 1) For small enough α, we have m 2 (0, α) ≲ m(α)(αζ κ ) -1
2) For k = 1 or 2, for all µ and all α small enough,

m k (µ, α) ≤ (α ∧ |B 0 |/(1 + B 0 )) -k with B 0 = g 1 (0)/2ϕ(0) -1. Proof. 1) Let α ∈ [0; 1], we have m 2 (0, α) = 2 ˆ∞ 0 β 2 (z) (1 + αβ(z)) 2 ϕ(z)dz = 2 ˆζ 0 β 2 (z) (1 + αβ(z)) 2 ϕ(z)dz + 2 ˆ∞ ζ β 2 (z) (1 + αβ(z)) 2 ϕ(z)dz
For the first term, as in Proposition 3, and using Proposition 17, we have

ˆζ 0 β 2 (z) (1 + αβ(z)) 2 ϕ(z)dz ≲ ˆζ 0 β 2 (z)ϕ(z)dz ≲ g 1 (ζ)(αζ) -1
For the last term, by the fact that β is increasing on [ζ, √ 2 log n], (2.6.5) and (2.6.6) we have that

β > 0 on [ζ, ∞] so that ˆ∞ ζ β 2 (z) (1 + αβ(z)) 2 ϕ(z)dz ≲ 1/α 2 ˆ∞ ζ ϕ(z)dz ≲ β 2 (ζ)ϕ(ζ)/ζ ≲ β(ζ)g 1 (ζ)/ζ hence m 2 (0, α) ≲ g 1 (ζ) αζ . Yet m(α) ≍ ζ κ-1 g 1 (ζ) when α → 0, which yields the first point. 2) Recall the definition m k (µ, α) = ´ β(t) 1+αβ(t) k ϕ(t -µ)dt. If β(t) ≥ 0, β(t)
1+αβ(t) ≤ 1/α. Otherwise we have |t| < λ 0 /2 so using (2.6.5) for the numerator leads to β(t) ≥ g 1 (0)/2ϕ(0) -1 = B 0 and for the denominator

|1 + αβ(t)| = 1 + αβ(t) ≥ 1 + β(t) ≥ 1 + B 0 .

In-probability bounds

Lemma 21. We take α = α 1 and ζ = ζ 1 as defined by (2.5.3). There exists C > 0 such that sup θ∈ℓ 0 (sn)

P θ ( ζ < ζ) ≤ exp(-C(log n) 2 ).
Proof. First note that, almost surely, α-1 ≥ 1 > β(2λ 1 ) with the help of the first point of Lemma 20, so

ζ = β -1 (α -1 ) > 2λ 1 . Since β is increasing on (2λ 1 , √ 2 log n) and ζ ≤ √ 2 log n, we have { ζ < ζ} = {α > α}, so P ( ζ < ζ) = P (α > α) = P (α > α ∩ S(α) > 0) + P (α > α ∩ S(α) ≤ 0).
Let us now focus on the event {α > α}∩{S(α) ≤ 0}. If S(α) ≤ 0, since S is decreasing,

S < 0 on ]α, α]. So the likelihood l is decreasing on ]α, α[. It implies that there exists α ′ ∈]α, α[ such that l(α ′ ) > l(α). But this contradicts the maximality of α. Therefore {α > α} ∩ {S(α) ≤ 0} = ∅. Hence P ( ζ < ζ) = P (α > α ∩ S(α) > 0) ≤ P (S(α) > 0). The score function S(α) = n i=1 β(θ i + Z i , α
) is a sum of independent random variables, each bounded by α -1 . We have

P (S(α) > 0) = P ( n i=1 W i > A), with A = -n i=1 m 1 (θ i , α) and W i = β(θ i + Z i , α) -m 1 (θ i , α) centered variables, bounded by M = (1 + c)/α using the second point of Proposition 4. Setting V = n i=1 var(W i ), Bernstein's inequality gives P (S(α) > 0) ≤ exp( -A 2 2(V + M A 3 )
).

Moreover, proceeding as in Lemma 9 in the SAS case, we have -A ≲ -n m(α) and

V ≲ n m(α) α , so A 2 2(V + M A 3 ) -1 = V A 2 + M 3A ≤ C αn m(α) + C ′ αn m(α) ≲ (αn m(α)) -1 therefore A 2 2(V + M A 3 ) ≳ αn m(α) ≳ s n ≳ (log n) 2 and finally P (S(α) > 0) ≤ exp(-C(log n) 2 ).

Posterior convergence at sharp minimax rate

One defines a notion of posterior convergence at the sharp minimax rate, or convergence at the minimax rate with exact constant, with respect to the L 2 -norm loss, as follows sup

θ 0 ∈ℓ 0 [sn] E θ 0 ˆ∥θ -θ 0 ∥ 2 2 dΠ(θ|X) ≤ 2s n log( n s n )(1 + o(1)) (3.1.2) If (3.1.
2) holds, then at least two estimators (one of these randomised) converge at the minimax rate with exact constant in the usual sense. First, using the Jensen inequality, it implies that the posterior mean (denoted here by θ) converges at minimax rate with exact constant to the true signal sup

θ 0 ∈ℓ 0 [sn] E θ 0 ∥ θ -θ 0 ∥ 2 2 ≤ 2s n log( n s n )(1 + o(1)) (3.1.3)
Second, let us consider a draw from the posterior distribution. More formally, it is a θ = θ(X, U ), using the data X and uniform variables U on [0, 1], such that

L( θ(X, U )|X) = Π(•|X), stating (3.1.
2) is exactly stating the convergence to θ 0 at minimax rate with exact constant of θ. To construct such a θ in practice in the setting of the sequence model with a Spike and Slab prior, as the a posteriori law is a product, one can take, denoting by

F θ i |X the cumulative distribution function of each θ i |X i , θ(X, U ) = (F -1 θ 1 |X (U 1 ), . . . , F -1 θn|X (U n ))

Spike and Slab prior

Prior

The spike and slab prior with smoothing parameter α is given by

Π α ∼ n i=1 (1 -α)δ 0 + αG(•), (3.1.4)
where δ 0 denotes the Dirac mass at 0 and G is a given probability measure of density γ, often taken to be the Laplace distribution or some heavy tailed distribution.

Posterior

The posterior distribution under (3.1.1)-

(3.1.4) is Π α [• | X] ∼ n i=1 (1 -a α (X i ))δ 0 + a α (X i )γ X i (•), (3.1.5)
where we have set, denoting ϕ the standard normal density and ϕ * G(x) = ´ϕ(x-u)dG(u) the convolution of ϕ and G,

g(X i ) = (ϕ * G)(X i ), γ X i (•) = ϕ(X i -•)γ(•) g(X i ) , a α (X i ) = αg(X i ) (1 -α)ϕ(X i ) + αg(X i )

A special Slab density γ

Consider the unimodal symmetric density γ on R given by

γ(x) = 1 2 ∆(1 + |x|), ∆(u) = u -1 (1 + log(u)) -2 , ( 3.1.6)
The purpose of this new density is to have sufficiently heavy tails, heavier than Cauchy.

Apart from this specific tail property, γ still satisfies

sup u>0 d du log γ(u) =: Λ < ∞. (3.1.7)
Let us denote by g = ϕ * γ the convolution of the heavy-tailed γ given by (3.1.6) and the noise density ϕ. Basic properties of g are gathered in Lemma 22, while Lemma 23 provides bounds on corresponding moments of the score function.

Useful Thresholds

Let us recall the following useful threshold properties already used in Chapter 2. As noted in [START_REF] Castillo | Spike and slab empirical Bayes sparse credible sets[END_REF], the properties established by [START_REF] Johnstone | Needles and straw in haystacks: empirical Bayes estimates of possibly sparse sequences[END_REF] are extended without difficulties to slabs with heavier tails than Cauchy.

Posterior median and threshold t(α). The a posteriori median θmed

α (X) = θmed i (X i ) i∈{1,...,n}
has the following thresholding property: there exists t(α) > 0, depending on the smoothing parameter α of (3.1.4) such that θmed i

(X i ) = 0 if and only if |X i | ≤ t(α). A default
choice can be α = 1/n, which leads to a posterior median behaving similarly as a hard thresholding estimator with threshold √ 2 log n. One can significantly improve on this choice by taking a well-chosen data-dependent α, as will be seen in 3.1.5.

The thresholds ζ(α), τ (α) and τ (α). Following [START_REF] Johnstone | Needles and straw in haystacks: empirical Bayes estimates of possibly sparse sequences[END_REF], we introduce several useful thresholds. From Lemma 1 in [START_REF] Johnstone | Needles and straw in haystacks: empirical Bayes estimates of possibly sparse sequences[END_REF], we know that g/ϕ, and therefore B = g/ϕ -1, is a strictly increasing function on R + . It is also continuous, so given α, a pseudo-threshold ζ = ζ(α) can be defined by

B(ζ) = 1 α . (3.1.8)
It is shown in [START_REF] Johnstone | Needles and straw in haystacks: empirical Bayes estimates of possibly sparse sequences[END_REF], Lemma 3, that

t(α) ≤ ζ(α). (3.1.9)
Further one can also define τ (α) as the solution in x of

Ω(x, α) := a(x) 1 -a(x) = α 1 -α g ϕ (x) = 1.
Equivalently, a(τ (α)) = 1/2. Also, B(τ

(α)) = α -1 -2 so τ (α) ≤ ζ(α). Define α 0 as τ (α 0 ) = 1 and set τ (α) = τ (α ∧ α 0 ). (3.1.10)
In the sequel, one can always take α small enough, so it will be silently understood that α ≤ α 0 so that τ (α) = τ (α). These thresholds are useful to understand the behaviour of the a posteriori law, in particular to have bounds on the updated smoothing parameter a α (X).

Empirical Bayes choice of α

The log-marginal likelihood in α can be written as

ℓ(α) = ℓ n (α; X) = n i=1 log((1 -α)ϕ(X i ) + αg(X i )).
(3.1.11)

Let α be defined as the maximiser of the log-marginal likelihood α = argmax α∈An ℓ n (α; X), (3.1.12)

where the maximisation is restricted to A n = [α n , 1], with α n defined, in view of (1.2.11), by

t(α n ) = 2 log n.
The reason for this restriction is that one does not need to take α smaller than α n , which would correspond to a choice of α 'more conservative' than hard-thresholding at threshold level √ 2 log n. The a priori law that will be therefore considered is the Spike and Slab where we have 'plugged' the value α :

θ ∼ Π α := n i=1 (1 -α)δ 0 + αΓ (3.1.13)
One will also denote the threshold of our new 'plug-in' posterior median, in view of 3.1.4,

t = t(α) (3.1.14)
the threshold of the posterior median corresponding to the Spike and Slab prior with plugged-in parameter α.

Main result

Our specific choice of Slab density leads the following result, assuming that s n satisfies the mild condition that there exist constants c 0 , c 1 such that

c 1 log 2 n ≤ s n ≤ c 0 n. (3.2.1)
As in Chapter 2, we note that it is under this condition that the rate in Theorem 1 of [START_REF] Johnstone | Needles and straw in haystacks: empirical Bayes estimates of possibly sparse sequences[END_REF] is optimal.

Theorem 18. Let Π α be the Spike and Slab prior distribution (3.1.4) with Slab density γ given by (3.1.6). Let Π α[• | X] be the corresponding plug-in posterior distribution given by (3.1.13), with α chosen by the empirical Bayes procedure (3.1.12). For any

s n verifying (3.2.1) , for n → ∞ sup θ 0 ∈ℓ 0 [sn] E θ 0 ˆ∥θ -θ 0 ∥ 2 2 dΠ α(θ | X) ≤ 2s n log( n s n )(1 + o(1)).
Theorem 18 states that the second moment of the a posteriori law whose smoothing parameter has been chosen with Empirical Bayes and whose Slab is the specific density (3.1.6) converges to 0 at sharp minimax rate, i.e. where one obtains the exact constant 2. We note that similarly to Chapter 2 or to Theorem 2 of [START_REF] Johnstone | Needles and straw in haystacks: empirical Bayes estimates of possibly sparse sequences[END_REF], one could also consider a modified estimator for α. As in Chapter 2, we believe that working with this modified estimator α should enable one to remove condition (3.2.1) and get the sharp minimax rate also in the regime s n ≲ log 2 n.

Why it works

Let us first consider the case where α is a fixed constant in (0, 1) to get an intuition on why it is possible at all to obtain this sharp minimax rate result already in case the regularity parameter s n is given to us. In the quantity

ˆ∥θ -θ 0 ∥ 2 dΠ α (θ | X) = n i=1 ˆ(θ i -θ 0,i ) 2 dΠ α (θ i | X i ),
let us distinguish two parts: the coordinates of θ 0 that are just equal to zero on one hand, and the nonzero coordinates on the other hand. For α ∈ (0, 1), µ ∈ R and x ∈ R, we use the following notation

r 2 (α, µ, x) = (1 -a(x))µ 2 + a(x) ˆ(u -µ) 2 γ x (u)du. so that ˆ∥θ -θ 0 ∥ 2 dΠ α (θ | X) = n i=1 r 2 (α, θ 0,i , X i ).
For the part where the signal is zero, we have the following result, for α small enough

E 0 r 2 (α, 0, x) ≲ ατ (α) 2 {1 + log(1 + τ (α))} -2 (3.2.2)
This result immediately follows from Lemma 27, as τ (α) goes to infinity as α → 0.

For the nonzero coordinates, the following bound directly follows from Lemma 28

E µ [r 2 (α, µ, x)] ≤ τ (α) 2 (1 + o(1)) (3.2.3)
Expected posterior squared norm at fixed α. Putting together the previous bounds leads to, with S 0 the support of θ 0 ,

E θ 0 ˆ∥θ -θ 0 ∥ 2 dΠ α (θ | X) = n i=1 E θ 0 ˆ(θ i -θ 0,i ) 2 dΠ α (θ i | X i ) ≤ i / ∈S 0 Cατ (α) 2 / log 2 (τ (α)) + i∈S 0 τ (α) 2 (1 + o(1)) ≤ Cnατ (α) 2 / log 2 (τ (α)) + s n τ (α) 2 (1 + o(1))
which leads to the following result for fixed α Proposition 5. For a fixed α ∈ [s n /n; log(log n/s n )s n /n], the Spike and Slab prior with parameter α and γ as in (3.1.6) yields the exact constant for the posterior squared norm, that is, for n → ∞

sup θ 0 ∈ℓ 0 [sn] E θ 0 ˆ∥θ -θ 0 ∥ 2 dΠ α (θ | X) ≤ 2s n log(n/s n )(1 + o(1))
Note that for α as in Proposition 5 (and as soon as α ≥ (s n /n) η for η > 0 for instance) the nonzero part of the signal always contribute for 2s n log(n/s n ). The part of zero signal is more dependent on the choice of the Slab. When α is data-driven, this part may interfere with the nonzero part. In the Laplace case, one can check that α is too far from the oracle parameter α * = s n /n, resulting in a zero signal contribution larger than the minimax rate. In the Cauchy case, the zero signal contribution becomes exactly of the order of the minimax rate. With the special Slab (3.1.6), we shall prove that for data-driven α this contribution becomes lower than the minimax rate, finally resulting in 2s n log( n sn )(1 + o( 1)).

Proofs

Thresholds and Useful Bounds

The following bounds are borrowed from Johnstone and Silverman (2004) (again, they extend without difficulty to the heavy-tailed γ)

Bounds on a α (x). For any real x and α ∈ [0, 1],

α g g ∨ ϕ (x) ≤ a α (x) ≤ 1 ∧ α 1 -α g ϕ (x). (3.3.1)
The following bound in terms of τ (α), see [START_REF] Johnstone | Needles and straw in haystacks: empirical Bayes estimates of possibly sparse sequences[END_REF] p. 1623 (one has τ (α) = τ (α) for any α ≤ α 0 ), is useful for large x,

1 -a α (x) ≤ 1l |x|≤τ (α) + e -1 2 (|x|-τ (α)) 2 1l |x|>τ (α) . (3.3.2)

Properties of g and moments of the score function

While qualitative properties of g and of the score function (such as g/ϕ or m are increasing functions) do not change with the present heavy-tailed choice of γ, some of the equivalents of g and moments of the score function change, in a way that we describe now.

Lemma 22. For γ defined by (3.1.6) and g = ϕ * γ, as x → ∞,

g(x) ≍ γ(x) g(x) -1 ˆ∞ x g(u)du ≍ x log x.
Also, g/ϕ is strictly increasing from (g/ϕ)(0) < 1 to +∞ as x → ∞.

The monotonicity property in Lemma 22 enables one to define a pseudo-threshold, still denoted ζ, from the function

B = (g/ϕ) -1 as ζ(α) = B -1 (α -1 ).
The posterior median, by the same proof as Lemma 2 in [START_REF] Johnstone | Needles and straw in haystacks: empirical Bayes estimates of possibly sparse sequences[END_REF], is a threshold rule: there exists t(α) > 0 such that the posterior median on coordinate i is 0 if and only if |X i | ≤ t(α). Also, by the same proof as in [START_REF] Johnstone | Needles and straw in haystacks: empirical Bayes estimates of possibly sparse sequences[END_REF], one has t(α) 2 < ζ(α) 2 and ϕ(t) < Cϕ (ζ).

Now turning to the moments of the score function, let us denote

B(x, α) = B(x) 1 + αB(x) ,
and similarly as for m, m 1 , m 2 , let us set

m(α) = -E 0 B(X, α), m 1 (µ, α) = E µ B(X, α), m 2 (µ, α) = E µ B(X, α) 2 .
Lemma 23. The function α → m(α) is nonnegative and increasing in α. As α → 0,

m(α) ≍ (log ζ) -1 . (3.3.3) m 1 (µ, α) ≤                  -m(α) + C ζ(α) log ζ(α) 2 µ 2 , for |µ| < 1/ζ(α), C ϕ(ζ/2) α , for |µ| < ζ(α)/2 (α ∧ c) -1 , for all µ. m 2 (µ, α) ≤                C log ζ(α) 1 ζ(α) 2 m(α) α , for |µ| < 1/ζ(α), C ζ ϕ(ζ/2) α 2 , for |µ| < ζ(α)/2 (α ∧ c) -1 , for all µ.
Proof of Lemma 22. The derivative (log γ) ′ (u) is bounded in absolute value for |u| ≥ 1 by a universal constant, and γ is unimodal and symmetric, so g ≍ γ and the monotonicity of g/ϕ follow from the proof of Lemma 1 in [START_REF] Johnstone | Needles and straw in haystacks: empirical Bayes estimates of possibly sparse sequences[END_REF]. The second estimate is immediate using the first one and the fact that ´∞ y (u log 2 u) -1 du = (log y) -1 .

Proof of Lemma 23. Using that ´Bϕ = 0, one can rewrite, as in Johnstone and Silverman ( 2004), m as

m(α) = 2 ˆ∞ 0 αB(z) 2 1 + αB(z) ϕ(z)dz from which it follows that, separating into z ≤ ζ anf z > ζ, m(α) ≍ ˆζ 0 αB(z) 2 ϕ(z)dz + ˆ∞ ζ B(z)ϕ(z)dz.
The first term is dealt with using the estimate of Corollary 1 of [START_REF] Johnstone | Needles and straw in haystacks: empirical Bayes estimates of possibly sparse sequences[END_REF], valid for any γ log-Lipshitz on R which is the case here, which leads to, for ζ large enough, or equivalently α small enough,

α ˆζ 0 B(z) 2 ϕ(z)dz ≤ α C ζ g(ζ) 2 ϕ(ζ) ≲ C g(ζ) ζ .
For the second term, noting that Bϕ ∼ g and using Lemma 22 for the tail bound on (minus) the primitive of g, one gets that this term is asymptotic to g(ζ)ζ log ζ ≍ log -1 ζ and always dominates the first term. This proves the claim on m. Now turning to m 1 and m 2 , note that the global bounds directly follow from the fact that |B(x, α)| ≤ C ∨ α -1 . The intermediate bounds are derived as in [START_REF] Johnstone | Needles and straw in haystacks: empirical Bayes estimates of possibly sparse sequences[END_REF], since the proofs involve only the log-Lipschitz property of γ.

For small signals |µ| ≤ 1/ζ and the first moment, one proceeds as in Johnstone and Silverman ( 2004) by (Taylor-) expanding the function µ → m 1 (µ, α) at the order 2 around µ = 0. The first derivative in µ is 0, since the function is symmetric. The following bound on the second derivative is as in [START_REF] Johnstone | Needles and straw in haystacks: empirical Bayes estimates of possibly sparse sequences[END_REF]: on [-ζ, ζ]

one bounds ϕ ′′ (u) by C(1 + u 2 )ϕ(u) and uses ϕ(z -µ) ≤ Cϕ(z) thanks to the fact that |µ| ≤ ζ -1 , ∂ 2 ∂µ 2 m 1 (µ, α) ≤ ˆ∞ -∞ |B(z, α)ϕ ′′ (z -µ)|dz ≤ ˆζ -ζ |B(z)|(1 + z 2 )ϕ(z)dz + 2 α ˆ|z|>ζ ϕ ′′ (z -µ)dz ≤ ˆζ -ζ g(z)(1 + z 2 )dz + CB(ζ)ϕ(ζ) = (i) + (ii).
The term (ii) is bounded by a constant times ζg(ζ) ≤ C(log ζ) -2 . The integral defining (i) can be separated in |z| ≤ 2, for which it is bounded by a constant, and |z| > 2, part on which one integrates by part to obtain

ˆζ 2 g(z)z 2 dz ≤ C ˆζ 2 z log 2 z dz = ζ 2 2 log 2 ζ - 4 2 log 2 2 - ˆζ 2 z 2 log 3 z dz ≤ ζ 2 2 log 2 ζ .
One concludes that the term (i) dominates in the expression of the second derivative, and the bound for m 1 follows by a Taylor expansion.

The bound for the second moment is obtained by separating again |z| ≤ ζ and |z| > ζ to obtain

m 2 (µ, α) ≤ C ˆζ -ζ B(z) 2 ϕ(z -µ)dz + 1 α 2 ˆ|z|>ζ ϕ(z -µ)dz ≤ C ˆζ 0 B(z) 2 ϕ(z)dz + 2 α 2 ϕ(z -µ) z -µ ≤ C g(ζ) ζ 1 α + C g(ζ) ζ 1 α .
By using the estimate g(ζ) ≍ m(α)/(ζ log ζ) which follows from the estimate on m, the bound on m 2 for small µ follows.

Let us now state a simplified version of Lemma 26 of [START_REF] Castillo | On spike and slab empirical Bayes multiple testing[END_REF]. Note that the authors introduce a quantity T µ (α) not appearing here since T µ (α) ≥ 1 which will be sufficient in what follows. Another proof of Lemma 24 can be obtained using a slightly different approach, see Lemma 18 of [START_REF] Castillo | Spike and slab empirical Bayes sparse credible sets[END_REF].

Lemma 24. Let Φ(t) = ´∞ t ϕ(u)du. There exist M 0 > 0 and a 0 ∈ (0, 1) such that ∀µ ≥ M 0 and ∀α ≤ a 0

m 1 (µ, α) ≥ 1 4 B(ζ) Φ(ζ -µ)
.

Proof. We follow the proof of Lemma 26 in [START_REF] Castillo | On spike and slab empirical Bayes multiple testing[END_REF] that stays valid for our special Slab (3.1.6) as it only needs g to be decreasing and (log g) ′ to be bounded which is stated in (3.1.7). In the page 59 of their paper, one has the following inequalities

∀α ≤ a 0 , ∀M 0 ≤ µ ≤ ζ -1, m 1 (µ, α) ≥ 1 2 B(ζ) Φ(ζ -µ) + CB(ζ) ϕ(ζ -µ) µ and ∀α ≤ a 0 , ∀µ ≥ ζ -1, m 1 (µ, α) ≥ 1 4 B(ζ) Φ(ζ -µ)
which leads to the result.

We will also need the following result Lemma 25. Let M 0 and a 0 be the constants appearing in Lemma 24. For every µ ≥ M 0 and α ≤ a 0 ,

m 2 (µ, α) ≲ m 1 (µ, α) α . Proof. We have E[|B(µ + Z, α)|] = m 1 (µ, α) + E[|B(µ + Z, α)| -B(µ + Z, α)] with Z ∼ N (0, 1).
We first use Lemma 24 to show that, for µ ≥ M 0 and α ≤ a 0 , m 1 (µ, α) ≳ 1.

For M 0 ≤ µ < ζ, we have, for small enough α

m 1 (µ, α) ≳ B(ζ) Φ(ζ -µ) ≳ g(ζ) ϕ(ζ) 1 ζ - 1 ζ 3 ϕ(ζ -µ) ≳ g(ζ) ζ 2 -1 ζ 3 e -µ 2 2 +µζ ≳ g(ζ) ζ 3 e -µζ 2 +µζ ≳ 1 ζ 4 log 2 (ζ) e µζ 2 ≳ 1
For µ > ζ, we have, for small enough α

m 1 (µ, α) ≳ B(ζ) Φ(ζ -µ) ≳ g(ζ) ϕ(ζ) 1 2 + ˆµ-ζ 0 ϕ(t)dt ≳ 1 2 g(ζ) ϕ(ζ) ≳ 1
Using the fact that, as noted in (88) of [START_REF] Johnstone | Needles and straw in haystacks: empirical Bayes estimates of possibly sparse sequences[END_REF],

|B(x, α)| ≤ c = B(0)/(1 + B(0)) if B(x) < 0 and |B(x, α)| ≤ 1/α if B(x) ≥ 0, we have, for µ ≥ M 0 and α ≤ a 0 , as m 1 (µ, α) ≳ 1, E[|B(µ + Z, α)|] ≤ m 1 (µ, α) + 2c ≲ m 1 (µ, α).
We also have, for α small enough, |B(µ

+ Z, α)| ≤ 1 α . Hence m 2 (θ, α) ≤ E[|B(µ+Z, α)| 2 ] ≤ 1 α E[|B(µ+Z, α)|] ≲ m 1 (µ,α) α
for α small enough.

Bounds for posterior moments and fixed α

Here we study r 2 (α, µ, x) := ´(u -µ) 2 dπ α (u | x), where for x real we denote in slight abuse of notation

π α (• | x) ∼ (1 -a α (x))δ 0 + a α (x)γ x (•).
For any real µ and α ∈ [0, 1], by definition (3.3.4) We first need to study the integral ´(u -µ) 2 γ x (u)du. The following Lemma is a general result on densities that one could check for our special Slab γ but also for Laplace, Cauchy or other classical choices of Slab densities.

r 2 (α, µ, x) = (1 -a α (x))µ 2 + a α (x) ˆ(u -µ) 2 γ x (u)du.
Lemma 26. Let γ be any density on R such that there exist positive constants c 1 and c 2 such that |(log γ) .3.5) which leads to the following upper bound

′ | ≤ c 1 and |(log γ) ′′ | ≤ c 2 ˆ(u -µ) 2 γ x (u)du = (x -µ) 2 + g ′′ g (x) + 1 + 2(x -µ) g ′ g (x). ( 3
ˆ(u -µ) 2 γ x (u)du ≤ (x -µ) 2 + c 2 + 2c 1 |x -µ| ≤ 2(x -µ) 2 + c 3 . (3.3.6)
Proof. Since γ x (•) is a density function, for any x, ˆγx (u)du = 1.

Noting that for any x, ´uγ x (u)du = x + (log g) ′ (x) and that this quantity is in absolute value less than |x| (check, cf before), one obtains

| ˆuγ x (u)du| ≤ |x|. Decomposing u 2 = (u -x) 2 + 2x(u -x) + x 2 and noting that ´(u -x) 2 γ x (u)du = g ′′ (x)/g(x) + 1, ˆu2 γ x (u)du = g ′′ g (x) + 1 + 2x g ′ g (x) + x 2 .
Using that |γ ′ | ≤ c 1 γ and |γ ′′ | ≤ c 2 γ, this leads to

|g ′ (x)| ≤ ˆ|γ ′ (x -u)|ϕ(u)du ≤ c 1 ˆγ(x -u)ϕ(u)du = c 1 g(x)
and similarly |g ′′ | ≤ c 2 g, so that ´u2 γ x (u)du ≤ C(1 + x 2 ). Similarly, for any real µ,

ˆ(u -µ) 2 γ x (u)du = (x -µ) 2 + g ′′ g (x) + 1 + 2(x -µ) g ′ g (x). (3.3.7)
Bounds for zero signal.

Lemma 27. Let γ be as in (3.1.6) and let r 2 (α, µ, x) be as in (3.3.4).

E 0 r 2 (α, 0, x) ≲ ατ (α) 2 {1 + log(1 + τ (α))} -2
Proof. Suppose for now that x ∼ N (0, 1). One has r 2 (α, 0, x) = a α (x) ˆu2 γ x (u)du.

Using the simple bound (3.3.1) for a α (x), this implies for all x r 2 (α, 0,

x) ≲ 1 ∧ α 1 -α g ϕ (x) (1 + x 2 ). (3.3.8)
By taking the expectation and noticing that τ (α) is the number that makes both sides of the infimum of the last display equal, one obtains

E 0 r 2 (α, 0, x) ≲ ˆ1l |x|>τ (α) (1 + x 2 )ϕ(x)dx + ˆ1l |x|≤τ (α) α 1 -α g ϕ (x)ϕ(x)(1 + x 2 )dx.
As ˆ∞ M x 2 ϕ(x)dx ≤ CM ϕ(M ), the first term of the last display is bounded by

ϕ(τ (α))/τ (α) + τ (α)ϕ(τ (α)).
For γ as in (3.1.6), as g ≍ γ by Lemma 22, the second term of the last display is bounded by a constant times

α ˆτ(α) -τ (α) γ(x)(1 + x 2 )dx = α ˆτ(α) 0 1 + x 2 (1 + x)(1 + log(1 + x)) 2 dx ≤ α ˆ1+τ(α)
1 y 2 y(1 + log y) 2 dy, setting y = 1 + x and using that 1 + x 2 ≤ y 2 for y ≥ 1. An integration by parts gives ˆ1+τ(α)

1 y (1 + log y) 2 dy = y 2 2(1 + log y) 2 1+τ (α) 1 + ˆ1+τ(α) 1 y (1 + log y) 3 dy.
The map y → y/(1 + log y) 3 is increasing for y ≥ e 2 . This implies, for C > 0 a universal constant, ˆ1+τ(α)

1 y (1 + log y) 3 dy ≤ C + τ (α) 1 + τ (α) {1 + log(1 + τ (α))} 3 .
Bounds for nonzero signal. The definition of r 2 (α, µ, x) and the moment bound (3.3.6) lead to

r 2 (α, µ, x) ≤ (1 -a α (x))µ 2 + a α (x)(2(x -µ) 2 + C).
(3.3.9)

The weight 1 -a α (x) is bounded using (3.3.2), so that

r 2 (α, µ, x) ≤ µ 2 1l |x|≤τ (α) + e -1 2 (|x|-τ (α)) 2 1l |x|>τ (α) + 2(x -µ) 2 + C. (3.3.10)
Let us prove the following lemma Lemma 28. Let a 0 be the solution of the equation τ (α) = 20. There exist C 1 , C 2 > 0 such that for any 0 < α ≤ a 0 (and alpha small enough for τ = τ ) and any real µ,

E µ r 2 (α, µ, x) ≤ (µ 2 + C 1 )1l |µ|≤τ (α)+ √ τ (α) + C 2 1l |µ|>τ (α)+ √ τ (α) .
In particular, there exists C 3 > 0 such that for any 0 < α ≤ a 0 and any real µ,

E µ r 2 (α, µ, x) ≤ τ (α) 2 1 + τ (α) -1/2 1l |µ|≤τ (α)+ √ τ (α) + C 3 .
Proof. Let us set as shorthand notation for the proof

T := τ (α), ϵ = T -1/2 , δ = ϵ 2(1 + ϵ) .
The condition on α implies ϵ < 1/4. Let us distinguish two cases. First, if |µ| ≤ (1 + ϵ)T , one simply bounds the first term on the right hand side of (3.3.9) by µ 2 and the expectation of the second one by a constant, which leads to the first term displayed in the upper-bound of the lemma.

In the case that |µ| > (1 + ϵ)T , one uses (3.3.10) to get

E µ r 2 (α, µ, x) ≤ µ 2 P µ [|x| ≤ T ] + E µ [e -1 2 (|x|-T ) 2 ] + C.
Under the present assumption on µ, note that |µ| -

T = 2δ|µ| + (1 -2δ)|µ| -T > 2δ|µ|.
The triangle inequality implies

P µ [|x| ≤ T ] ≤ P µ [|ε| ≥ |µ| -T ] = 2 Φ (2δ|µ|) .

Let us consider the interval

A = [µ -δ|µ|, µ + δ|µ|]. One can split E µ [e -(|x|-T ) 2 2 ] = ˆe-(|x|-T ) 2 2 e -(x-µ) 2 2 dx √ 2π ≤ ˆAc e -(x-µ) 2 2 dx √ 2π + ˆA e -(|x|-T ) 2 2 dx √ 2π .
By definition of A, the first term of the last bound is 2 Φ (δ|µ|). Moreover, on A, we have

|x| ≥ (1 -δ)|µ|, so |x| -T ≥ (1 -δ -1 1+ϵ )|µ| = δ|µ|. This leads to ˆA e -(|x|-T ) 2 2 dx √ 2π ≤ 2δ|µ|e -δ 2 µ 2 /2 ≤ Ce -δ 2 µ 2 /4 ,
where one uses xe -x 2 /2 ≤ Ce -x 2 /4 for x ≥ 0. Putting the previous bounds together implies

E µ r 2 (α, µ, x) ≤ Cµ 2 Φ (δ|µ|) + ϕ(δµ/2) + C ≤ Cµ 2 ϕ(δµ/2) + C ≤ Cδ -2 ϕ(δµ/4) + C ≤ CT e -dT 1/2 + C,
where we have used that ve -v ≤ 2e -v/2 for v ≥ 0 and where d = 1/256. The last bound in the previous display is bounded by a universal constant, which leads to the second term displayed in the upper-bound of the lemma.

This finally leads to the bound, for α = o(1) and any µ

E µ [r 2 (α, µ, x)] ≤ τ (α) 2 (1 + o(1)) (3.3.11)

Risk bound for fixed α: proof of Proposition 5

Proof of Proposition 5. Using the facts that τ 2 (α) ≤ ζ 2 (α) ≤ 2 log(1/α) + o(log(1/α)) (use for example Lemma 14 of [START_REF] Castillo | On spike and slab empirical Bayes multiple testing[END_REF]) and that, for α small enough, there exist c 1 > 1 and c 2 > c 1 two constants such that 2c 1 log(1/α) ≤ τ (α) ≤ 2c 2 log(1/α), we have, for α small enough

E θ 0 ´∥θ -θ 0 ∥ 2 dΠ α (θ | X) ≤ Cnατ (α) 2 / log 2 (τ (α)) + s n τ (α) 2 (1 + o(1)) ≤ 2c 1 Cnα log(1/α)/ log 2 (2c 2 log(1/α)) + 2s n log(1/α)(1 + o(1)) ≤ 2c 1 Cnα log(1/α)/ log 2 (2c 2 log(1/α)) + 2s n log(n/s n )(1 + o(1))
Lemma 30 (signal). Let α be a fixed non-random element of (0, 1) (chosen small enough so that τ (α) = τ (α)). Let α be a random element of [0, 1] that may depend on x ∼ N (µ, 1) and on other data and such that τ (α) 2 ≤ d log(n) with probability 1 for some d > 0.

Then there exists C 2 > 0 such that for all real µ,

Er 2 (α, µ, x) ≤ τ (α) 2 (1 + o(1)) + C 2 (1 + d log n)P (α < α) 1/2 .
Proof. Combining (3.3.2) and (3.3.9),

r 2 (α, µ, x) ≤ µ 2 1l |x|≤τ ( α) + e -1 2 (|x|-τ ( α)) 2 1l |x|>τ ( α) + C((x -µ) 2 + 1).
Note that it is enough to bound the first term on the right hand side in the last display, as the last one is bounded by a constant under E µ . Let us distinguish the two cases α ≥ α and α < α.

In the case α ≥ α, as τ (α) is a decreasing function of α,

1l |x|≤τ ( α) + e -1 2 (|x|-τ ( α)) 2 1l |x|>τ ( α) 1l α≥α ≤ 1l |x|≤τ ( α) + 1l τ ( α)<|x|≤τ (α) + e -1 2 (|x|-τ ( α)) 2 1l |x|>τ (α) 1l α≥α ≤ 1l |x|≤τ (α) + e -1 2 (|x|-τ (α)) 2 1l |x|>τ (α) ,
where we have used e -1 2 v 2 ≤ 1 for any v and that e -1 2 (u-c) 2 ≤ e -1 2 (u-d) 2 if u > d ≥ c. As a consequence, one can borrow the fixed α bound (3.3.11) obtained previously so that

E [r 2 (α, µ, x)1 α≥α ] ≤ E µ µ 2 (1l |x|≤τ (α) + e -1 2 (|x|-τ (α)) 2 1l |x|>τ (α) ) + C((x -µ) 2 + 1) ≤ τ (α) 2 (1+o(1)).
In the case α < α, setting b n = √ d log n and noting that τ (α) ≤ b n with probability 1 by assumption, proceeding as above, with b n now replacing τ (α), one can bound

1l |x|≤τ ( α) + e -1 2 (|x|-τ ( α)) 2 1l |x|>τ ( α) ≤ 1l |x|≤bn + e -1 2 (|x|-bn) 2 1l |x|>bn .
From this one deduces that

E µ 2 1l |x|≤τ ( α) + e -1 2 (|x|-τ ( α)) 2 1l |x|>τ ( α) 1l α<α ≤ C E µ µ 4 1l |x|≤bn + µ 4 e -(|x|-bn) 2 1/2 P (α < α) 1/2 .
Using similar bounds as in the fixed α case, one obtains

E µ µ 4 1l |x|≤bn + µ 4 e -(|x|-bn) 2 ≤ C(1 + b 4 n ).
Taking the square root and gathering the different bounds obtained concludes the proof.

Undersmoothing

Let α 1 be defined as the solution in α of the equation, (3.3.12) where d is a constant to be chosen small enough, see below, and ηn = (s n ∨ log 2 n)/n.

dα m(α) = ηn ,
Note that under (3.2.1), we have η = s n /n. As in Johnstone and Silverman ( 2004) we note that α → α m(α) is increasing in α for α small enough, and equals 0 at 0. So a solution of (3.3.12) exists. Also, provided η n is small enough, it can be made smaller than any given arbitrary constant. Let ζ 1 be defined via B(ζ 1 ) = 1/α 1 for α 1 as in (3.3.12). As stated in (3.3.3), m(α) ≍ (log ζ) -1 , as α = o(1). Note that in [START_REF] Johnstone | Needles and straw in haystacks: empirical Bayes estimates of possibly sparse sequences[END_REF], ζ 1 is defined as the solution in ζ of α m(α) = ηn ζ κ . The following result is an adaptation of Lemma 10 in [START_REF] Johnstone | Needles and straw in haystacks: empirical Bayes estimates of possibly sparse sequences[END_REF] to accommodate this different choice. As already noted in Chapter 2, it seems choosing α = α 1 as in (3.3.12) is necessary to obtain a sharp posterior integrated squared rate.

Lemma 31. There exist universal constants C and η 0 such that if η ≤ η 0 and n/ log 2 (n) ≥ η -1 0 , then sup

θ∈ℓ 0 [η] P θ [ ζ < ζ 1 ] ≤ exp(-Cnη n ).
Proof. Using log(1/η n ) ≤ log(n) -2 log log n, the bound on ζ from Lemma 32 gives that

ζ 2 1 ≤ 2 log n -3 2 log log n, so that t(α 1 ) ≤ ζ(α 1 ) = ζ 1 ≤ √ 2 log n = t(α n )
, so α 1 belongs to the interval [α n , 1] over which the likelihood is maximised. For the rest of the proof let us denote α = α 1 .

Then one notices that

{ ζ < ζ 1 } = {α > α 1 } = {S(α 1 ) > 0} as well as { ζ > ζ 1 } =
{S(α 1 ) < 0}: the sign of S at any particular w determines on which side of α the given

α lies. So, P θ [ ζ < ζ 1 ] = P θ [S(α 1 ) > 0].
The score function equals S(α) = n i=1 β(X i , α), a sum of independent variables. By Bernstein's inequality, if W i are centered independent variables with |W i | ≤ M and n i=1 Var(W i ) ≤ V , then for any A > 0,

P n i=1 W i > A ≤ exp{- 1 2 A 2 /(V + 1 3 M A)}.
Set W i = B(X i , α)-m 1 (θ 0,i , α) and A = -n i=1 m 1 (θ 0,i , α). Then one can take M = c 3 /α, using Lemma 7. One can bound -A from above as follows (3.3.13) where one uses that ζ -1 is bounded [in fact, goes to 0 if η 0 = o( 1)]. This leads to

-A ≤ - i / ∈S 0 m(α) + i∈S 0 c α ≤ -(n -s n ) m(α) + cs n /α ≤ -n m(α)/2 + cdn m(α) ≤ -n m(α)/4, and
V (α) ≤ i / ∈S 0 m 2 (0, α) + i∈S 0 m 2 (θ 0,i , α) ≤ c 4 (n -s n ) m(α) ζ κ α + cs n /α 2 ≤ C α (n -s n ) m(α)ζ -κ + cs n /α ≤ Cα -1 n m(α)ζ -κ /2 + cdn m(α) ≤ C ′ dn m(α)/α,
V + 1 3 M A A 2 ≤ C ′ d nα m(α) + 4c 3 3nα m(α) ≤ c -1 5 nα m(α) .
Deduce that

P [S(α) > 0] ≤ exp{-c 5 nα m(α)}.
Lemma 32 (Basic bounds on ζ(α), τ (α) and t(α)). Let α = α 1 be defined by (3.3.12) for d a given constant and ηn small enough, and let ζ(α) be given by B(ζ(α)) = α -1 . Then for some constants c 1 , c 2 ,

log(1/η n ) + c 1 ≤ ζ(α) 2 2 ≤ log(1/η n ) + 1 2 log(1 + log(1/η n )) + c 2 .
The same upper and lower bound hold (with possible different constants c 1 and c 2 ) for τ (α) 2 and t(α). In particular,

ζ(α) 2 ∼ τ (α) 2 ∼ t(α) 2 ∼ 2 log(1/η n ) as ηn → 0.
Proof. But for small ηn , we have α small, or equivalently ζ large, so that (g/B)(ζ) ∼ ϕ(ζ). Now from the definition (3.3.12) of α combined with (3.3.3), one has

ηn ≍ dαζ log(ζ) g(ζ) B(ζ) B(ζ) ≍ ζ log(ζ)ϕ(ζ) ≍ ζ log(ζ)e -ζ 2 /2 .
From this deduce that

| log c + log ζ + log (log(ζ)) - ζ 2 2 + log(1/η n )| ≤ C. | log c + C ′ log ζ - ζ 2 2 + log(1/η n )| ≤ C.
In particular, using log ζ ≤ a + ζ 2 /4 for some constant a > 0 large enough, one gets

ζ 2 ≤ 4(C + log(1/η n )).
Inserting this back into the previous inequality leads to

ζ 2 /2 ≤ log(1/η n ) + C + 1 2 log(1 + log(1/η n )).
To prove that the same statement holds for τ (α) and t(α) note that following from the definition of τ (α) and ζ(α) we have ζ(α/2) ≤ τ (α) ≤ ζ(α) and from page 1622 of [START_REF] Johnstone | Needles and straw in haystacks: empirical Bayes estimates of possibly sparse sequences[END_REF]

we have that ζ(α) 2 -c ≤ t(α) 2 ≤ ζ(α) 2 .

Oversmoothing

Following [START_REF] Johnstone | Needles and straw in haystacks: empirical Bayes estimates of possibly sparse sequences[END_REF], Section 8.3, let us define (3.3.14) We also set, recalling that α 0 is defined via τ (α 0 ) = 1, (3.3.15) One also defines ζ τ,π as the corresponding pseudo-threshold B -1 (α(τ, π) -1 ).

π(τ ; µ) = 1 n #{i : |µ i | ≥ τ }.
α(τ, π) = sup{α ≤ α 0 : πm 1 (τ, α) ≥ 2 m(α)}.
Lemma 33. There exists C and π 0 such that if π < π 0 , then for all τ ≥ 1, sup θ: π(τ ;θ)≥π

P θ [ ζ > ζ τ,π ] ≤ exp{-Cnϕ(ζ τ,π )}.
Proof. This is the same proof as in [START_REF] Johnstone | Needles and straw in haystacks: empirical Bayes estimates of possibly sparse sequences[END_REF], Lemma 11, where we use Lemma 23 to bound m 2 (0, α) and Lemma 25 for m 2 (τ, α).

Proof of Theorem 18

Let us decompose the risk R n (θ 0 ) = E θ 0 ´∥θ -θ 0 ∥ 2 dΠ α(θ | X) according to whether coordinates of θ correspond to a 'small' or 'large' signal, the threshold being ζ 1 = β -1 (α -1 1 ), with α 1 defined in (3.3.12). One can write

R n (θ 0 ) = i: θ 0,i =0 + i: 0<|θ 0,i |≤ζ 1 + i: |θ 0,i |>ζ 1 E θ 0 ˆ(θ i -θ 0,i ) 2 dΠ α(θ i | X).
We next use the first part of Lemma 29 with α = α 1 to obtain, for any θ 0 in ℓ 0 [s n ], (I) :=

i: θ 0,i =0 E θ 0 ˆ(θ i -θ 0,i ) 2 dΠ α(θ i | X) ≤ C 1 i: θ 0,i =0 α 1 τ (α 1 ) 2 (1 + log(1 + τ (α 1 )) -2 + P θ 0 (α > α 1 ) ≤ C 1 (n -s n )α 1 τ (α 1 ) 2 (1 + log(1 + τ (α 1 )) -2 + (n -s n )e -c 1 log 2 n ,
where for the last inequality we use Lemma 31 and (3.2.1). From (3.3.12) one gets, with

η n = s n /n, nα 1 ≲ nη n ζ -1 1 g(ζ 1 ) -1 ≲ s n log(ζ 1 ).
Therefore, as in Proposition 5, the term (I) is a o(s n log(n/s n ))

For the 'intermediate' signal part, using the second part of the Lemma 29, we have

i: 0<|θ 0,i |≤ζ 1 E θ 0 ˆ(θ i -θ 0,i ) 2 dΠ α(θ i | X) ≤ i: 0<|θ 0,i |≤ζ 1 (θ 2 0,i + C) ≤ (ζ 2 1 + C)#{i : 0 < |θ 0,i | ≤ ζ 1 },
Now using Lemma 32 and the fact that τ (α 1 ) ≤ ζ 1 , one obtains that the contribution to the risk of the indices i with 0

< |θ 0,i | ≤ ζ 1 is bounded by 2 log(n/s n )(1 + o(1))#{i : 0 < |θ 0,i | ≤ ζ 1 } (3.3.16)
It remains to bound the part of the risk for indexes i with |θ 0,i | > ζ 1 . To do so, one uses Lemma 30 with α chosen as α = α 2 := α(ζ 1 , π 1 ) and π 1 = π(ζ 1 ; θ 0 ) the proportion of components of the true signal above ζ 1 , following the definitions (3.3.14)-(3.3.15). Also, τ (α 2 ) = τ (α 2 ). One denotes by ζ 2 the pseudo-threshold associated to α 2 .

Let us first compare α 1 and α 2 . For small enough α, the bound on m 1 from Lemma 23 becomes 1/α, so that, using the definition (3.3.12) of α 1 ,

m 1 (ζ 1 , α 1 ) m(α 1 ) ≤ 1 α 1 η n dα 1 -1 ≤ d η n ≤ d π 1 ,
using the rough bound π 1 ≤ η n . Note that both functions m(•) -1 and m 1 (ζ 1 , •) are decreasing via Lemma 23, and so is their product on the interval where both functions are positive. As d < 2, by definition of α 2 this means α

2 < α 1 that is ζ 1 < ζ 2 .
One can now apply Lemma 30 with α = α 2 and use the fact that τ

(α 2 ) ≤ ζ 2 i: |θ 0,i |>ζ 1 E θ 0 ˆ(θ i -θ 0,i ) 2 dΠ α(θ i | X) ≤ nπ 1 τ (α 2 ) 2 (1 + o(1)) + C 2 (1 + d log n)P θ 0 (α < α 2 ) 1/2 ≤ nπ 1 ζ 2 2 (1 + o(1)) + C 2 (1 + d log n)P θ 0 ( ζ > ζ 2 ) 1/2 .
Let us verify that this term in the last display is bounded above by

nπ 1 ζ 2 2 (1 + o(1)). If ζ 2 > log n, this is immediate by bounding P θ 0 ( ζ > ζ 2 ) by 1. If ζ 2 ≤ log n, Lemma 33 implies P θ 0 ( ζ > ζ 2 ) ≤ exp(-Cnϕ(ζ 2 )) ≤ exp(-C √ n),
m 1 (ζ 1 , α 3 ) m(α 3 ) ≥ 1 4 B(ζ 3 ) Φ(ζ 3 -ζ 1 ) m(α 3 ) = 1 4α 3 8α 3 m(α 3 ) π 1 m(α 3 ) = 2 π 1 .
This shows, reasoning as above, that α 3 ≤ α 2 , that is ζ 2 ≤ ζ 3 . Following [START_REF] Johnstone | Needles and straw in haystacks: empirical Bayes estimates of possibly sparse sequences[END_REF], one distinguishes two cases to further bound

ζ 3 . If ζ 3 > ζ 1 + 1, using ζ 2 2 ≤ ζ 2 3 and m(α 3 ) ≲ ζ 3 g(ζ 3 ), π 1 ζ 2 2 ≤ ζ 2 3 8α 3 m(α 3 ) Φ(ζ 3 -ζ 1 ) ≲ ζ 3 3 g(ζ 3 ) B(ζ 3 ) ζ 3 -ζ 1 ϕ(ζ 3 -ζ 1 ) ≤ Cζ 4 3 ϕ(ζ 3 ) ϕ(ζ 3 -ζ 1 ) = Cζ 4 3 ϕ(ζ 1 )e -(ζ 3 -ζ 1 )ζ 1 ≤ C(ζ 1 + 1) 4 e -ζ 1 ϕ(ζ 1 ),
where for the last inequality we have used that

x → x 4 e -(x-ζ 1 )ζ 1 is decreasing for x ≥ ζ 1 +1.
Lemma 32 now implies that ϕ(ζ 1 ) ≲ η n . As ζ 1 goes to ∞ with n/s n , one gets π 1 ζ 2 2 ≲ η n . In this case, gathering the three different bounds leads us to

R n (θ 0 ) ≤ o(s n log(n/s n )) + 2 log(n/s n )(1 + o(1))#{i : 0 < |θ 0,i | ≤ ζ 1 } + s n ≤ o(s n log(n/s n )) + 2s n log(n/s n )(1 + o(1)) + s n ≤ 2s n log(n/s n )(1 + o(1)) If ζ 1 ≤ ζ 3 ≤ ζ 1 + 1, let ζ 4 = ζ(α 4 ) with α 4 solution in α of Φ(1) = 8α m(α)π -1 1 . By the definition of ζ 3 , since Φ(1) ≤ Φ(ζ 3 -ζ 1 ), we have 8α 4 m(α 4 ) ≤ 8α 3 m(α 3 ) so that α 4 ≤ α 3 , that is also ζ 3 ≤ ζ 4 . Using (3.3.3) as before, Φ(1) ≲ g(ζ 4 ) B(ζ 4 ) π -1 1 ≲ ϕ(ζ 4 )π -1 1 .
Taking logarithms this leads to

ζ 2 4 ≤ C + 2 log(π -1 1 ). In particular, ζ 2 2 ≤ 2 log(π -1 1 ) + C. It follows that nπ 1 ζ 2 2 ≤ 2nπ 1 log(π -1 1 ) + Cnπ 1 = 2nπ 1 log(n/s n ) + 2nπ 1 log(s n /(nπ 1 )) + Cnπ 1 ≤ 2nπ 1 log(n/s n ) + 2s n log(n/s n ) nπ 1 sn log(s n /(nπ 1 ))(log(n/s n )) -1 + C nπ 1 sn (log(n/s n )) -1 ≤ 2nπ 1 log(n/s n ) + 2s n log(n/s n ) [C ′ (log(n/s n )) -1 ] ,
Where the last line uses that nπ 1 ≤ s n and that u ∈ [1; +∞] → log(u)/u is bounded. Now gathering the three different bounds and using the fact that #{i : 0

< |θ 0,i | ≤ ζ 1 } + nπ 1 = #{i : 0 < |θ 0,i | ≤ ζ 1 } + #{i : |θ 0,i | ≥ ζ 1 } ≤ s n leads us to R n (θ 0 ) ≤ o(s n log(n/s n )) + 2 log(n/s n )#{i : 0 < |θ 0,i | ≤ ζ 1 } + 2nπ 1 log(n/s n ) ≤ 2s n log(n/s n )(1 + o(1))
which concludes the proof of Theorem 18.

Lemma 34. Let Φ(t) = ´∞ t ϕ(u)du. For π 1 , ζ 1 as above, a solution 0 < α ≤ α 1 to the equation Φ(ζ(α) -ζ 1 ) = 8π -1 1 α m(α).
(3.3.17) exists. Let α 3 be the largest such solution. Then for c 0 in (3.2.1) small enough,

m 1 (ζ 1 , α 3 ) ≥ 1 4 B(ζ 3 ) Φ(α 3 -ζ 1 ). ( 3
(ζ α ) ≍ αg(ζ α ), Φ(ζ α -ζ 1 ) ≍ ϕ(ζ α -ζ 1 ) ζ α -ζ 1 ≍ αg(ζ α )e ζαζ 1 . As m(α) ≍ ζ α g(ζ α ), one gets R α ≍ e ζαζ 1 /ζ α → ∞ as α → 0.
On the other hand, with

π 1 ≤ s n /n and α 1 m(α 1 ) = ds n /n, R α 1 = 1 2α 1 m(α 1 ) = dn 2s n ≤ 8 π 1 d 16 , so that R α 1 < 8/π 1 as d < 2
. This shows that the equation at stake has at least one solution for α in the interval (0, α 1 ). Finally (3.3.18) is a direct consequence of Lemma 24.

Let E := l≥0 {0, 1} l ∪ {∅} be the set of finite binary sequences. We write |ε| = l if ε ∈ {0; 1} l and |∅| = 0. We also use the notation

ε ′ = ε 1 ε 2 . . . ε l-1 (1 -ε l ).
Let us introduce a sequence of partitions I = {(I ε ) |ε|=l , l ≥ 0} of the unit interval. Here we will consider regular partitions, as defined below. This is mostly for simplicity of presentation, and other partitions, based for instance on quantiles of a given distribution, could be considered as well. Set I ∅ = (0, 1] and, for any ε ∈ E such that ε = ε(l; k) is the expression in base 1/2 of k2 -l , set

I ε := k 2 l , k + 1 2 l := I l k
For any l ≥ 0, the collection of all such dyadic intervals is a partition of (0, 1]. A random probability measure P follows a Pólya tree distribution P T (A) with parameters A = {α ε ; ε ∈ E} on the sequence of partitions I if there exist random variables 0 ≤ Y ε ≤ 1 such that, 1. the variables Y ε0 for ε ∈ E are mutually independent and Y ε0 follows a Beta(α ε0 , α ε1 ) distribution. 2. for any ε ∈ E, we have Y ε1 = 1 -Y ε0 3. for any l ≥ 0 and ε = ε 1 . . . ε l ∈ {0; 1} l , we have

P (I ε ) = l j=1 Y ε 1 ...ε j (4.1.1)
. This construction can be visualised using a tree representation, see Figure 4.1 : to compute the random mass that P assigns to the subset I ε of [0, 1], one follows a dyadic tree along the expression of ε : ε 1 ; ε 1 ε 2 , . . . , ε 1 ε 2 . . . ε l = ε. The mass P (I ε ) is a product of Beta variables whose parameters depend on whether one goes 'left' (ε j = 0) or 'right' (ε j = 1) along the tree :

P (I ε ) = l j=1,ε j =0 Y ε 1 ,...,ε j-1 0 × l j=1,ε j =1 (1 -Y ε 1 ,...,ε j-1 0 ) (4.1.2)
This construction uniquely defines a random probability distribution on distributions on [0, 1]. For more details we refer to [START_REF] Ferguson | Prior distributions on spaces of probability measures[END_REF] and [START_REF] Lavine | Some aspects of polya tree distributions for statistical modelling[END_REF].

The corresponding object, the class of Pólya tree distributions, is quite flexible : different behaviours of the sequence of parameters (α ε ) ε∈E give a Pólya tree with different properties. A standard assumption is that the parameters α ε only depend on the depth |ε|, so that ∀ε ∈ E, α ε = a l for any l ≥ 1 and a sequence (a l ) l≥1 of positive numbers, which will be assumed henceforth.

Paths along the tree. A given ε = ε 1 , . . . , ε l ∈ E gives rise to a path ε

1 → ε 1 ε 2 → ε 1 ε 2 . . . ε l .
We denote I [i] ε := I ε 1 ...ε i , for any i in {1, . . . , l}. Similarly, denote

Y [i] ε = Y ε 1 ...ε i
Conversely, any pair (l, k) with l ≥ 0 and k ∈ {0, . . . 2 l -1} is associated with a unique ε = ε(l, k), the expression of length l in base 1/2 of k2 l .

Function spaces and wavelets

We briefly introduce some standard notation appearing in the statements below.

Haar basis. The Haar wavelet basis is {ϕ, ψ lk , 0 ≤ k < 2 l , l ≥ 0}, where ϕ = 1l [0,1] and, for ψ = -1l (0,1/2] + 1l (1/2,1] ,

ψ lk (•) = 2 l/2 ψ(2 l • -k), 0 ≤ k < 2 l , l ≥ 0.
In this paper our interest is in density functions, that is nonnegative functions g with ˆ1 0 gϕ = ˆ1 0 g = 1, so that their first Haar-coefficient is always 1. So, we will only need to consider the basis functions ψ lk and simply write informally (ψ lk ) for the Haar basis.

Function classes. Let L 2 = L 2 [0, 1] denote the space of square-integrable functions on [0, 1] relative to Lebesgue measure equipped with the ∥ .

• ∥ 2 -norm. For f, g ∈ L 2 , denote ⟨f, g⟩ := ⟨f, g⟩ 2 = ˆ1 0 f g. Let L ∞ = L ∞ [0,
(4.1.4) This is the minimax rate for estimating a density function in a ball of α-Hölder functions, when the supremum norm is considered as a loss, see [START_REF] Ibragimov | An estimate of the density of a distribution[END_REF] and [START_REF] Khas'minskii | A lower bound on the risks of non-parametric estimates of densities in the uniform metric[END_REF].

Spike and Slab prior distributions 'truncated' at a certain level L.

In the following, one defines the cutoff L max = log 2 (n) and L the largest integer such that

2 L L ≤ n (4.1.5) Note that L ≤ L max for every n. Let X (n) = (X 1 , • • • , X n ) be i.i.d
. from law P with density f . Let Π be the prior on densities generated as follows. One keeps the Pólya tree random measure with respect to the canonical dyadic partition of [0, 1] construction up to level L, replacing the Beta distributions by

ε ∈ E , Y ε0 ∼ (1 -π ε0 )δ 1 2 + π ε0 Beta(α ε0 , α ε1 ), (4.1.6)
with parameters α ε ∈ N to be chosen and a real parameter π ε (later to be taken of the form 2 -l 2 e -Cl , where we wrote l = |ε|). There are multiple probability distributions on Borelians of [0, 1] that coincide on dyadic intervals I ε with P (I ε ) resulting from the above construction. We consider the specific one that is absolutely continuous relatively to the Lebesgue measure on [0, 1] with a constant density on each I ε , |ε| = L + 1. So, both prior and posterior are histograms on dyadic intervals at depth L.

Definition. The prior distribution with parameters α ε , π ε , as above is called Spike and Slab Pólya tree and denoted Π(α ε , π ε ). This prior is based on an idea of Ghosal and van der Vaart, which is referred as Evenly Split Pólya tree in their book [START_REF] Ghosal | Fundamentals of Nonparametric Bayesian Inference[END_REF]. First note that the Haar coefficients f lk of a density f can be expressed as

f lk = ⟨f, ψ lk ⟩ = 2 l 2 P (I ε )(1 -2Y ε0 ) (4.1.7)
The Spike and Slab Pólya tree can therefore be seen as a 'thresholding prior', as the thresholding takes place on the sequence of Haar coefficients of the function where Y ε0 = 1 2 . Using this Spike and Slab prior can be seen as taking a Hierarchical approach. The usual Pólya tree (PT) prior on densities (under (1.3.13)) leads to the following Bayesian diagram

X|f ∼ f f ∼ P T ((Y ε0 )) with Y ε0 ∼ Beta(α ε0 , α ε1 ),
so the Y ε0 have fixed (Beta) distributions, whereas the Spike and Slab Pólya tree (SSPT) prior leads to the diagram

X|f ∼ f f ∼ SSP T ((Y ε0 )) with Y ε0 ∼ (1 -π ε0 )δ1 2 + π ε0 Beta(α ε0 , α ε1 )
which can be seen as the following diagram, using a sequence (γ ε0 ) ε of Bernoulli variables.

X|f ∼ f f |(γ ε0 ) ∼ SSP T ((Y ε0 )) with Y ε0 ∼ (1 -γ ε0 )δ1 2 + γ ε0 Beta(α ε0 , α ε1 ) γ ε0 ∼ Be(π ε0 )
So in this case the distributions followed by the Y ε0 are random, hence this approach can be viewed as hierarchical.

The a posteriori law. In the following, we will write

N X (I ε ) = n i=1 1l X i ∈Iε , p α ε0 = (α ε0 + α ε1 -1)! (α ε0 -1)!(α ε1 -1)! (4.1.8) and p X = (N X (I ε ) + α ε0 + α ε1 -1)! (N X (I ε0 ) + α ε0 -1)!(N X (I ε1 ) + α ε1 -1)! (4.1.9)
The following proposition shows that the Spike and Slab Pólya tree prior, as does the classical Pólya tree prior, is conjugate in the Density Estimation model.

Proposition 6. For every ε ∈ E with |ε| ≤ L, the a posteriori law of Y ε0 knowing X 1 , . . . , X n is Y ε0 |X ∼ (1 -πε0 )δ1 2 + πε0 Beta(α ε0 (X), α ε1 (X))
where πε0 is defined via a quantity T as follows

πε0 = π ε0 T (1 -π ε0 ) + π ε0 T , T =T (ε, X) = 2 N X (Iε) p α ε0 p X
and where Iε) .

α ε (X) = N X (I ε ) + α ε . Proof. We have Π(Y |X) ∝ f (X 1 , . . . , X n ) L-1 |ε|=0 (1 -π ε0 )δ1 2 (Y ε0 ) + π ε0 p α ε0 Y α ε0 -1 ε0 (1 -Y ε0 ) α ε1 -1 with f (X 1 , . . . , X n ) = n i=1 |ε|=L (2 L P (I ε )) 1l X i ∈Iε = (2 L ) n |ε|=L P (I ε ) N X (

Introduction

123

Noticing that

L-1 |ε|=0 2 N X (Iε) = 2 nL , we have f (X 1 , . . . , X n ) = (2 L ) n L-1 |ε|=0 Y N X (I ε0 ) ε0 (1 -Y ε0 ) N X (I ε1 ) = L-1 |ε|=0 2 N X (Iε) Y N X (I ε0 ) ε0 (1 -Y ε0 ) N X (I ε1 ) .
This gives us that

Π(Y |X) = 1 A L-1 |ε|=0 (1 -π ε0 )2 N X (Iε) Y N X (I ε0 ) ε0 (1 -Y ε0 ) N X (I ε1 ) δ1 2 (Y ε0 )+ 2 N X (Iε) π ε0 p α ε0 Y N X (I ε0 )+α ε0 -1 ε0 (1 -Y ε0 ) N X (I ε1 )+α ε1 -1 = 1 A L-1 |ε|=0 (1 -π ε0 )δ1 2 (Y ε0 ) + 2 N X (Iε) π ε0 p α ε0 Y N X (I ε0 )+α ε0 -1 ε0 (1 -Y ε0 ) N X (I ε1 )+α ε1 -1 with A = L-1 |ε|=0 (1 -π ε0 ) + 2 N X (Iε) π ε0 p α ε0 p X
, which concludes the proof.

Note that if π ε = 1, meaning that the prior is also a product of Beta variables, one recovers the standard conjugacy for the (truncated at L) usual Pólya tree.

We will henceforth use the following notations 1. Tilded notation, posterior distribution. We denote by P a distribution sampled from the posterior distribution and by Ỹ the corresponding variables Y in (4.1.1). In particular, the variable Ỹε0 is distributed following the marginal a posteriori law

Ỹε0 ∼ (1 -πε0 )δ 1 2 + πε0 Beta(α ε0 (X), α ε1 (X)).
2. Bar notation, posterior mean. Let f = ˆf dΠ(f |X) denote the posterior mean density and P the corresponding probability measure. We use the notation Ȳ for the variables defining P via (4.1.1).

Main results

By definition, we take as prior as above the realisation of the Spike and Slab Pólya tree P that is absolutely continuous with respect to Lebesgue's measure with density equal to a histogram and histogram heights equal to P (I ε ). The posterior is, by Proposition 6, again a Spike and Slab Pólya tree with density w.r.t. Lebesgue equal to a histogram and histogram heights equal to P (I ε ). In particular, it induces a posterior on densities that we consider in the main results below.

An adaptive concentration result

The following Theorem shows that the a posteriori law obtained with a Spike and Slab type Pólya tree prior concentrates around the true density f 0 at minimax rate for the supremum-norm loss.

Theorem 19. Let f 0 ∈ C α [0, 1], for α ∈ (0, 1] and suppose ∥ log f 0 ∥ ∞ < ∞. Let X 1 , . . . , X n be i.i.d. random variables on [0, 1] following P f 0 . Let Π be the prior on densities induced by a Spike and Slab Polya Tree prior Π(α ε , π ε ) with the choices

α ε = a π ε = 2 -l 2 e -κl , l = |ε|
for κ large enough constant and a > 0 constant. Then for any

M n → ∞, in P f 0 -probability Π   ∥f -f 0 ∥ ∞ ≤ M n log n n α 2α+1 | X   → 1
This theorem is an adaptive version of Theorem 1 of [START_REF] Castillo | Polya tree posterior distributions on densities[END_REF]. There are few results so far in the literature in density estimation for the supremum-norm loss, among those are the results from [START_REF] Castillo | On bayesian supremum norm contraction rates[END_REF][START_REF] Hoffmann | On adaptive posterior concentration rates[END_REF] and [START_REF] Yoo | Supremum norm posterior contraction and credible sets for nonparametric multivariate regression[END_REF] for multivariate regression.

A Bernstein Von Mises result

To establish a nonparametric Bernstein Von Mises (BVM) result, one has first to find a space M 0 large enough to have convergence at rate √ n of the posterior density to a Gaussian process. One can then derive results for some other space F using continuous mapping for continuous functionals ψ : M 0 → F. A space that combines nicely with supremum norm structure was introduced by Castillo and Nickl (2014) and defined as follows, using an 'admissible' sequence ω = (ω l ) l≥0 such that ω

l / √ l → ∞ as l → ∞ M 0 = M 0 (ω) = x = (x lk ) l,k ; lim l→∞ max 0≤k<2 l |x lk | ω l = 0 (4.2.1) Equipped with the norm ∥x∥ M 0 = sup l≥0 max 0≤k<2 l |x lk | ω l
, this is a separable Banach space. In a slight abuse of notation, we will write f ∈ M 0 if the sequence of its Haar wavelet coefficients belongs in that space : (⟨f, ψ lk ⟩) l,k ∈ M 0 .

P -white bridge process. For P a probability distribution in [0, 1], one defines the P -white bridge process, denoted by G P . This is the Gaussian process indexed by the

Hilbert space L 2 (P ) = {f : [0, 1] → R; ˆ1 0 f 2 dP < ∞} with covariance E[G P (f )G P (g)] = ˆ1 0 (f - ˆ1 0 f dP )(g - ˆ1 0 gdP )dP (4.2.2)
We will denote by N the law of G P 0 (with P 0 = P f 0 ). The main purpose of the admissible sequence ω is to ensure that G P ∈ M 0 . Intuitively, if one does not use these weights w l , the maximum over 2 l Gaussian variables is of order 2 log(2 l ) = C √ l and does not tend to 0 as l → ∞, see Remark 1 of [START_REF] Castillo | On the bernstein von mises phenomenon for nonparametric bayes procedures[END_REF] for a precise proof of this result.

Bounded Lipschitz metric. Let (S, d) be a metric space. The bounded Lipschitz metric β S on probability measures of S is defined as, for any µ, ν probability measures of S,

β S (µ, ν) = sup F ;∥F ∥ BL ≤1 ˆS F (x)(dµ(x) -dν(x)) , (4.2.3)
where F : S → R and

∥F ∥ BL = sup x∈S |F (x)| + sup x̸ =y |F (x) -F (y)| d(x, y) . ( 4

.2.4)

This metric metrizes the convergence in distribution:

µ n → µ in distribution as n → ∞ if and only if β S (µ n , µ) → 0 as n → ∞.
Recentering the distribution. To establish our BVM result, one also has to find a suitable way to center the posterior distribution. In this view, denote by P n the empirical measure

P n = 1 n n i=1 δ X i . (4.2.5)
Let us also consider C n , which is a smoothed version of P n , defined by

⟨C n , ψ lk ⟩ =    ⟨P n , ψ lk ⟩ if l ≤ L 0 if l > L, (4.2.6)
where L is our original cutoff, defined by (4.1.5).

We finally introduce T n , which depends on the true parameter α, defined by

⟨T n , ψ lk ⟩ =    ⟨P n , ψ lk ⟩ if l ≤ L n 0 if l > L n , (4.2.7)
where we defined L n to be the integer such that

2 Ln = ⌊c 0 n log n 1 1+2α ⌋ (4.2.8)
for a suitable constant c 0 ∈ R + * , whose precise value is made clear below.

Weak BVM result. We have the following Bernstein-von Mises phenomenon for f 0 in Hölder balls (standard Hölder balls are subsets of the following ones)

H(α, R) := {f = (f lk ) : |f lk | ≤ R2 -(α+1/2) , ∀l ≥ 0, 0 ≤ k < 2 l }
Theorem 20. Let N be the law of G P 0 . Let C n be the centering defined in (4.2.6). Let l 0 (n) be an increasing and diverging sequence. We define the prior Π such that

Y ε0 ∼ Beta(a, a) for |ε| ≤ l 0 Y ε0 ∼ (1 -π ε0 )δ1 2 + π ε0 Beta(a, a) for l 0 < |ε| ≤ L
where π ε = 2 -l 2 e -κ|ε| with κ a large enough constant. The posterior distribution then satisfies a weak BvM : for every α, R > 0 sup

f 0 ∈H(α,R) E f 0 β M 0 (ω) (Π(•|X) • τ -1 Cn , N ) → 0
as n → ∞ and for any admissible sequence ω = (ω l ) with ω l 0 (n) / log(n) → ∞.

The choice of recentering of the distribution is quite flexible, as it can be checked that the result also holds if one replaces C n by the posterior mean fn or by T n which depends on α. Actually, the only required condition on where one cuts the empirical measure is to satisfy Theorem 1 of [START_REF] Castillo | On the bernstein von mises phenomenon for nonparametric bayes procedures[END_REF]. One can see that the cutoff L is exactly the furthest one can go according to that theorem.

Using the methods of [START_REF] Castillo | On the bernstein von mises phenomenon for nonparametric bayes procedures[END_REF], this result leads to several applications, for instance derivation of BVM theorems for semiparametric functionals via the continuous mapping theorem and Donsker-type theorems, which do not appear here for the sake of brievity. It may also lead to the construction of adaptive credible sets although it may require substantial additional work.

Proofs

Preliminaries and notation

The following notations will be used throughout the proofs.

For a given distribution P with distribution function F and density f on [0, 1], denote P (B) = F (B) = ˆB f , for any measurable subset B of [0, 1]. In particular under the "true" distribution, we denote P 0 (B) = F 0 (B) = ˆB f 0 . We will also denote by p ε the quantity P (I ε ).

In the sequel C denotes a universal constant whose value only depends on other fixed quantities of the problem.

For a function f in L 2 , and L n an integer, denote by f Ln the L 2 -projection of f onto the linear span of all elements of the basis {ψ lk } up to level l = L n . Also, denote f L c n the projection of f onto the orthocomplement V ect{ψ lk , l > L n }. In the proofs, we shall use the decomposition f = f Ln + f L c n , which holds in L 2 and L ∞ under prior and posterior as f is truncated at level L so has a finite Haar expansion under prior and posterior.

Recall the definition of L n from (4.2.8).

We will write, for i ∈ {1, . . . , l},

y i = F 0 (I [i] ε ) F 0 (I [i-1] ε ) and ∆ i = C 0 L n 2 i n (4.3.1)
. This gives us that, for ε * n,α as in (4.1.4),

∥ f Ln -f Ln 0 ∥ ∞ ≲ Ln l=0 2 l 2 max 0≤k<2 l | flk -f 0,lk | ≲ 1 n Ln l=0 2 l( 1 2 -α) + L n n Ln l=0 2 -lα + L n n 2 Ln ≲ ε * n,α ,
where we have used |f 0,lk | ≲ 2 -l( 1 2 +α) .

Term f Lnf Ln

Consider the event

A = {∀i ≤ L n , |ε| = l ≤ L n , | Ȳ [i] ε -Ỹ [i] ε | ≤ r [i] ε } with r [i] ε = M L n nF 0 (I [i] ε ) . ( 4
E X [∥f L c n ∥ ∞ ] ≤ log 2 (n) l=Ln+1 2 l 2 E X [ max 0≤k<2 l |f lk |] ≤ log 2 (n) l=Ln+1 2 l 2   2 l -1 k=0 E X [|f lk |]   We have E X [|f lk |] = 2 l 2 E X [p ε ]E X [|1 -2 Ỹε0 |].
On one hand, we have

E X [|1 -2 Ỹε0 |] = (1 -πε ) ˆ(1 -2u)δ1 2 (u) + πε E[|1 -2Z|] = πε E[|1 -2Z|],
with Z drawn from a Beta(α ε0 , α ε1 ). This gives us that, using Lemma 40

E X [|1 -2 Ỹε0 |] ≤ πε ≲ e -C 2 l √ n .
On the other hand, we have

E X [p ε ] = l-1 i=0 Q X,ε (i, 0) with Q X,ε (i, 0) = (1 -π[i+1] ε ) 1 2 + π[i+1] ε l + N X (I [i+1] ε ) 2l + N X (I [i] ε )
using the notations I [i] ε = I ε 1 ε 2 ...ε i and π[i] ε = πε 1 ε 2 ...ε i . Let us distinguish two regimes, when i ≤ L n and when i > L n . For the former, we have that, writing as before for i ∈ {1, . . . , l},

y i = F 0 (I [i] ε ) F 0 (I [i-1] ε )
and ∆ i = C 0 L n 2 i n .

(1

-π[i+1] ε ) 1 2 ≤ (1 -π[i+1] ε )(y i+1 + | 1 2 -y i+1 |) ≤ (1 -π[i+1] ε )y i+1 + 1 n + ∆ i .
This gives us that there exists C 1 , C ′ 1 > 0 such that :

Q X,ε (i, 0) ≤ y i+1 1 n +∆ i y i+1 + (1 -π[i+1] ε ) + π[i+1] ε 1 + C 1 (l+1)2 i + √ in2 i 2 n ≤ F 0 (I [i+1] ε ) F 0 (I [i] ε ) 1 + C ′ 1 (l+1)2 i + √ in2 i 2 n
This implies that

Ln i=0 Q X,ε (i, 0) ≲ F 0 (I [Ln+1] ε ) Ln i=0   1 + C 2 l2 i + √ in2 i 2 n   ≲ 1 2 Ln exp   C 4 Ln i=0 l2 i + √ in2 i 2 n   ≲ 1 2 Ln exp   C 5 L n 2 Ln + √ L n n2 Ln 2 n   ≲ 1 2 Ln .
When i > L n , we have, Q X,ε (i, 0) ≤ 1 2 (1 + 2 √ n ), therefore :

l i=Ln+1 Q X,ε (i, 0) ≲ 2 Ln 2 l l i=Ln+1 (1 + 2 √ n ) ≲ 2 Ln 2 l (1 + 2 √ n ) l-Ln
We finally have

E X [p ε ] ≤ 1 2 l (1 + 2 √ n ) l-Ln This leads to E X [∥f L c n ∥ ∞ ] ≤ 1 √ n log 2 (n) l=Ln+1 2 l e -C 2 l (1 + 2 √ n ) l-Ln , therefore E X [∥f L c n ∥ ∞ ] ≤ e (Ln+1)(log 2-C 2 ) √ n (1 + 2 √ n ) 1-(1+ 2 √ n )e log 2-C 2 log 2 (n)-Ln 1-(1+ 2 √ n )e log 2-C 2 ≲ e (Ln+1)(log 2-C 2 )
√ n which tends to 0 faster than any power of n provided C 2 is chosen large enough.

Conclusion

Gathering the different bounds, one obtains that

E f 0 Π ∥f -f 0 ∥ ∞ ≥ M n log n n α 2α+1 | X is bounded by E f 0 [E X (∥f -f 0 ∥ ∞ )] M n ε * n,α ≲ 1 M n + 2 Ln e -CLn M n ε * n,α + e (Ln+1)(log 2-C 2 ) M n ε * n,α √ n
This tends to 0 when n → ∞, which concludes the proof oh Theorem 19.

Proof of Theorem 20

In what follows, one will denote, for l ≥ 0, by π l the projection onto the finite-dimensional subspace V l of L 2 defined by

V l = span{ψ l ′ k : 0 ≤ l ′ ≤ l, 0 ≤ k < 2 l ′ }.
One will also similarly denote by π >l the projection onto span{ψ l ′ k : l ′ > l, 0 ≤ k < 2 l ′ }.

The proof uses a similar approach as Ray ( 2017), but the argument has to be adapted to the density estimation model and to the specific Spike and Slab procedure considered here. Let us introduce the sets J n (γ) = (l, k); |f 0,lk | > γ log n/n (4.3.8)

for every γ ∈ R + * . Note that, recalling (4.2.8) and for f 0 ∈ C α ([0, 1]), (l, k) ∈ J n (γ) implies l ≤ L n . We will also denote by S the support of f : S = {(l, k); f lk ̸ = 0} (4.3.9)

One will firstly need the following tightness result.

A tightness result

Theorem 21. Under the assumptions of Theorem 20, for every η > 0, R > 0 and α ∈ (0, 1), there exist M > 0 and n 0 ∈ N such that for every n ≥ n 0 sup f 0 ∈H(α,R)

E f 0 Π(∥f -f 0 ∥ M 0 ≥ M/ √ n|X) < η
Proof. Fix η > 0. Consider the event

A n = {S c ∩ J n (γ) = ∅} ∩ {S ∩ {l > L n } ̸ = ∅} ∩ { max (l,k):l≤Ln |f 0,lk -f lk | ≤ γ log(n)/n}
By Lemma 41, there exist γ > 0 such that for every α ∈ (0, 1) there exists B > 0 such that, for every f 0 ∈ H(α, R) , E f 0 [Π(A c n |X)] ≲ n -B . Now with D > 0 to be chosen, E f 0 [Π(∥f -f 0 ∥ M 0 ≥ M/ √ n|X)] is bounded above by T 1 + T 2 + T 3 , where

T 1 = E f 0 [Π({∥f -f 0 ∥ M 0 ≥ M/ √ n} ∩ {∥π l 0 (f -f 0 )∥ M 0 ≤ D/ √ n} ∩ A n |X)] T 2 = E f 0 [Π({∥f -f 0 ∥ M 0 ≥ M/ √ n} ∩ {∥π l 0 (f -f 0 )∥ M 0 > D/ √ n} ∩ A n |X)] T 3 = E f 0 [Π(A c n |X)
The last term is a o(1). The first term is bounded by

E f 0 Π({∥π >l 0 (f -f 0 )∥ M 0 ≥ (M -D)/ √ n} ∩ A n |X) .
We now proceed as in [START_REF] Hoffmann | On adaptive posterior concentration rates[END_REF]. As f 0 ∈ H(α, R), there exists J n (α) with 2 Jn(α) ≲ (n/ log n) 1/(2α+1) such that J n (γ) ⊂ {(l, k) : l ≤ J n (α), 0 ≤ k < 2 l } and since by hypothesis ω l 0 ≥ c √ log n. This gives us that, on A n , ∥π >l 0 (f -f 0 )∥ M 0 = O(n -1/2 ) for every f 0 ∈ H(α, R). We therefore choose M = M (η) to make the term T 1 smaller than η/2. The term T 2 is bounded by

E f 0 [Π( √ n∥π l 0 (f -f 0 )∥ M 0 > D|X)],
which is bounded, by Markov's inequality, by

√ n D E f 0 E Π (∥π l 0 (f -f 0 )∥ M 0 |X) . We have √ n D E f 0 E Π (∥π l 0 (f -f 0 )∥ M 0 |X) = √ n D E f 0 E Π (sup l≤l 0 1 ω l max k |f l,k -f 0,l,k | |X) ≤ √ n D E f 0 E Π (sup l≤l 0 1 ω l max k |⟨f -T n , ψ l,k ⟩| |X) + E f 0 E Π (sup l≤l 0 1 ω l max k |⟨f 0 -T n , ψ l,k ⟩| |)
The first expectation is bounded by C/ √ n as in Castillo (2017b) (see Lemma 8 and the proof of tightness starting page 2091). Indeed, when l ≤ l 0 our prior is only a Beta, just as in [START_REF] Castillo | Polya tree posterior distributions on densities[END_REF], except that the parameters a l ≡ a of the Beta are constant in our case and do not decay to 0, but this decline to 0 is irrelevant to frequencies l ≤ l 0 . The second term can be bounded, following the approach of [START_REF] Castillo | On the bernstein von mises phenomenon for nonparametric bayes procedures[END_REF] in their first theorem, using that (ω l ) l is admissible and with κ > 0 large enough, by

1 D sup l≤l 0 √ l ω l E f 0 sup l≤l 0 1 √ l max k | √ n⟨f 0 -T n , ψ lk ⟩| ≲ κ D + 1 D ˆ∞ κ P f 0 sup l≤l 0 1 √ l max k | √ n⟨f 0 -T n , ψ lk ⟩| > u du ≲ κ D + 1 D l≤l 0 ,k ˆ∞ κ P f 0 | √ n⟨f 0 -T n , ψ lk ⟩| > √ lu du ≲ κ D + 1 D l≤l 0 2 l ˆ∞ κ e -Clu du ≲ κ D + 1 D l≤l 0 e -C ′ κl ≲ 1 D
where the third inequality follows from an application of Bernstein's inequality. This finally gives us that by taking D = D(η) large enough the second term can be made smaller than η/2, which concludes the proof of Theorem 21.

Proof of Theorem 20

Fix η > 0 and denote Πn = Π(•|X) • τ -1 Cn . By the triangle inequality, uniformly over the relevant class of functions, for fixed l > 0, we have

β M 0 ( Πn , N ) ≤ β M 0 ( Πn , Πn • π -1 l ) + β M 0 ( Πn • π -1 l , N • π -1 l ) + β M 0 (N • π -1 l , N ) (4.3.10)
Let us now look more precisely at the first term of (4.3.10). Take a function F : M 0 → R such that ∥F ∥ BL ≤ 1, F n a random variable following Πn and (ω l ′ ) an admissible sequence such that ωl ′ /ω l ′ → 0 as l ′ → ∞. Let us also consider the events

D = {∥f ∥ M 0 ≤ M } and D n = {∥f -C n ∥ M 0 ≤ M/ √ n}
where M is large enough to have E f 0 [Π(∥f -f 0 ∥ M 0 ≥ M/ √ n|X)] < η/9 as in Theorem 21. One has

ˆM0 F d Πn - ˆM0 F d Πn • π -1 l ≤E Πn [|F (F n ) -F (π l (F n ))||X] ≤E Πn [|F (F n ) -F (π l (F n ))|(1l D + 1l D c )|X] ≤E Πn [∥F n -π l (F n )∥ M 0 ∥F ∥ BL 1l D |X] + 2 Πn (D c |X) ≤E sup l ′ >l 1 ω l ′ max 0≤k<2 l ′ | √ n⟨f -C n , ψ l ′ k ⟩|1l Dn |X + 2Π(∥f -C n ∥ M 0 ≥ M/ √ n|X) ≤ sup l ′ >l ωl ′ ω l ′ E sup l ′ >l 1 ωl ′ max 0≤k<2 l ′ | √ n⟨f -C n , ψ l ′ k ⟩|1l Dn |X + 2Π(∥(f -f 0 ) + (f 0 -C n )∥ M 0 ≥ M/ √ n|X) ≤ sup l ′ >l ωl ′ ω l ′ M + 2Π(∥f -f 0 ∥ M 0 ≥ M/(2 √ n)|X) + 2Π(∥f 0 -C n ∥ M 0 ≥ M/(2 √ n)|X)
The first term can be made smaller than η/9 by taking l large enough. Using Theorem 21, the expectation of the second term can be made smaller than η/9 by taking n large enough. The last term can be handled as in Theorem 1 of [START_REF] Castillo | On the bernstein von mises phenomenon for nonparametric bayes procedures[END_REF] (as here j n in that statement corresponds to our cutoff L) and be made smaller than η/9 by taking n large enough. Besides, one can note that the result holds when replacing C n by T n as in that case j n would correspond to L n which satisfies the required condition of Theorem 1 of [START_REF] Castillo | On the bernstein von mises phenomenon for nonparametric bayes procedures[END_REF]. This gives us that the first term of (4.3.10) is smaller than η/3. A similar result holds for the last term (see the proof of Theorem 1 of [START_REF] Castillo | On the bernstein von mises phenomenon for nonparametric bayes procedures[END_REF]). For the middle term, note that l 0 (n) ≥ l for n large enough. For such n, the projected prior onto the first l coordinates is a product of Beta variables and we are exactly in the setting of [START_REF] Castillo | Polya tree posterior distributions on densities[END_REF], except that the parameters a l ≡ a of the Beta are constant in our case and do not decay to 0. Since l is fixed, the fact that the parameters of our Beta do not depend on l does not change the outcome. Therefore, following the proof of the convergence of the finite-dimensional projections from page 2089 to page 2091 of [START_REF] Castillo | Polya tree posterior distributions on densities[END_REF], the middle term can be made smaller than η/3, which concludes the proof.

Technical Lemmas

Recall the notation, for i ∈ {1, . . . , l},

y i = F 0 (I [i] ε ) F 0 (I [i-1] ε )
and ∆ i = C 0 L n 2 i n .

(4.3.11)

Lemma 36. For l ≤ L n , on the event B, for ε ∈ E with |ε| = l, there exist some nonnegative real constants C and C ′ such that :

1 -π[i] ε ≤ (1 -π[i] ε )1l |y i -1 2 |≤∆ i + C 2 -i 2 √ n π [i] ε e -C ′ nF 0 (I [i-1] ε )∆ 2 i 1l |y i -1 2 |>∆ i .
In particular

1 -π[i] ε ≲ 1l |y i -1 2 |≤∆ i + 1 n 1l |y i -1 2 |>∆ i .
Proof. Let us write

s = s X = N X (I [i-1] ε
) + 2a -2 and q = q X = N X (I [i] ε ) + a -1 (4.3.12) so that we can rewrite N X (I [i-1] ε )+2a-1 p X = q!(s-q)! s!

. Using the fact that √ 2πn n+ 1 2 e -n ≤ n! ≤ √ 2πn n+ 1 2 e -n+ 1 12n for any integer n, this gives us that N X (I [i-1] ε ) + 2a -1 p X ≥ 2π q(s -q) s ( q e ) q ( s-q e ) s-q ( s e ) s e -1 12s .

We have, denoting by B(a) the Bernoulli distribution of parameter a and KL(P, Q) the Kullback-Leibler divergence between distributions P and Q, that 2 s ( q e ) q ( s-q e ) s-q ( s e ) s = e s( q s log( 2q s )+(1-q s ) log(2(1-q s ))) = e sKL(B( q s )||B( 12 )) .

We also know that KL(B( q s )||B( 1 2 ))

≥ 1 4 ∥B( q s ) -B( 1 2 )∥ 2 L 1 = 1 4 (2| q s -1 2 |) 2 .
Recalling T from Proposition 6, this leads to

T ≳ 1 √ s + 1 q(s -q) s(s + 1) e s| q s -1 2 | 2 -1 12s
On B, we can write N X (I

[i] ε ) = nF 0 (I [i] ε ) + δ i,ε with |δ i,ε | ≲ nLn 2 i .
We have

| q s -1 2 | = y i -1 2 + y i δ i,ε nF 0 (I [i] ε ) - δ i-1,ε nF 0 (I [i-1] ε ) 1+ δ i-1,ε nF 0 (I [i-1] ε ) ≥ |y i -1 2 | -y i | δ i,ε nF 0 (I [i] ε ) - δ i-1,ε nF 0 (I [i-1] ε ) | 1+ δ i-1,ε nF 0 (I [i-1] ε )
.

Since δ i,ε nF 0 (I

[i] ε )
tends to 0 when n → ∞, we have | q s -1 2 | ≳ |y i -1 2 |.

We now have T ≳ e CnF 0 (

I [i-1] ε )|y i -1 2 | 2 2 -i 2 √ n
, which concludes the proof, since 1-π

[i] ε = 1-π [i] ε (1-π [i] ε )+π [i] ε T
.

Lemma 37. For l ≤ L n , ε ∈ E such that |ε| = l, on the event B, we have

pε p 0,ε -1 ≤ C   l i=1 2 i n + L n 2 l n   .
Proof. We have p 0,ε = l i=1 y i and pε = l i=1 w i with

w i = 1 2 (1 -π[i] ε ) + π[i] ε N X (I [i] ε )+l N X (I [i-1] ε )+2l
. We can write

w i y i -1 = (1 -π[i] ε ) 1 2y i -1 + πε N X (I [i] ε ) + a y i (N X (I [i-1] ε ) + 2a) -1 .
Using Lemma 42, the second term is bounded by a constant times ( 2 i n + Ln2 i n ). The first term is bounded, by Lemma 36, by ∆ i + 2 Proof. We have, with y ε0 = F 0 (I ε0 ) F 0 (Iε) , Ȳε0

y ε0 -1 ≤ (1 -πε0 ) 1 2y ε0 -1 + πε0 N X (I ε0 ) + a y ε0 (N X (I ε ) + 2a) -1
The second term is bounded using Lemma 44 by a constant times ( 2 2l+ l 2 n + Ln2 l n ). The first term is bounded by Ln2 l n + 1 n , which concludes the proof.

We also have, using Lemma 45 Proof. As in Lemma 36, we will write s = N X (I [i-1] ε ) + 2a -2 and q = N X (I [i] ε ) + a -1, and δ = N X (I [i] ε ) -N X (I

(II)1l {|y i -1 2 |≤∆ i } ≤ 1l {|y i -1 2 |≤∆ i } P   |Z -E[Z]| ≥ C ′ L n 2 i n   ≲ e -M
[i]
ε ′ ). We can note first that q = s+δ 2 . Let M l = M nl 2 l ∨ l be the constant appearing in (4.3.3) when l > L n .

We have |δ| ≤ n|F 0 (I ε ) -F 0 (I ε ′ )| + M l ≲ n2 -l(α+1) + M l ≲ M l because f 0 is α-Hölder.

As in Lemma 36, T ≲ 1 √ s + 1 q(s -q) s(s + 1) e s( q s log( 2q s )+(1-q s ) log(2(1-q s )))

≲ 1 √ s + 1 e s( 1 2 (1+ δ s ) log(1+ δ s )+ 1 2 (1-δ s ) log(1-δ s )) ≲ 1 √ s + 1 e δ 2 2s
As l ≤ L with L defined as in (4.1.5), we have

s ≳ n 2 l +   l + L n 2 l n ∨ (l + L n )   .
which leads, as l ≤ L, to

δ 2 2s ≲ 2 l n nl 2 l ∨ l 2
This gives us that T ≲ 2 Lemma 41. Let J n (γ) be defined in (4.3.8). There exist γ > 0 such that for every α ∈ (0, 1), there exists B > 0 such that, for every f 0 ∈ H(α, R),

1)E f 0 [Π(S c ∩ J n (γ) ̸ = ∅|X)] ≲ n -2α/(1+2α) (log n) 1/(2α+1) 2)E f 0 [Π(S ∩ {l > L n } ̸ = ∅|X)] ≲ 1/ √ n
3)E f 0 Π( max This means that, for n large enough, the event in the probability in (4.3.13) is a subset of the event B c . Using Lemma 35, the probability in the last display is therefore bounded by a constant times e -BLn . The second term of (4.3.13) is bounded by

E f 0   Π(| flk -f lk | > γ 2 log(n) n |X)   .
As in (4.

  n→∞ E θ 0 [ϕ n ] = 0 and lim n→∞ sup ∥θ-θ 0 ∥≥ε E θ [ϕ n ] = 0. Then, as n → ∞, ∥L( √ n(θ -θ 0 )|X 1 , . . . , X n ) -N (∆ n (θ 0 ), I -1 θ 0 )∥ T V = o P θ 0 (1),with ∆ n (θ 0 ) = I (X i ) and ∥ • ∥ T V the total variation distance between two probability measures.

Fig. 1 . 1

 11 Fig. 1.1 Indexed binary tree with levels l ≤ 2 represented. The nodes index the intervals I ε . Edges are labelled with random variables Y ε .

  so this is also the case.One now compares ζ2 first to a certain ζ 3 = ζ(α 3 ) defined by α 3 (largest) solution of Φ(ζ(α 3 ) -ζ 1 ) = 8 π 1 α 3 m(α 3 ), with Φ(x) = P [N (0, 1) > x].Using Lemma 34, which also gives the existence of ζ 3 , one gets

  .3.18) Proof. First we check the existence of a solution. Set ζ α = ζ(α) and R α := Φ(ζ αζ 1 )/(α m(α)). For α → 0 we have ζ α -ζ 1 → ∞ so by using Φ(u) ≍ ϕ(u)/u as u → ∞ one gets, treating terms depending on ζ 1 as constants and using ϕ

Fig. 4 . 1

 41 Fig. 4.1 Indexed binary tree with levels l ≤ 2 represented. The nodes index the intervals I ε . Edges are labelled with random variables Y ε .

n

  Ln e -CLn , which tends to 0 when n → ∞. Now using similar arguments as in the proof of Theorem 1 of[START_REF] Castillo | Polya tree posterior distributions on densities[END_REF] where the bound for the Pólya tree given in Lemma 2 is replaced by the bound for the Spike and Slab Pólya tree given in Lemma 38, we have, on A and B, that Ỹε0 is a Beta variable and is therefore bounded by 1, we have|f lk -flk | = 2 l 2 |(p ε -pε ) + 2p ε ( Ȳε0 -Ỹε0 ) + 2 Ỹε0 (p ε -pε )| ≤ 2 l 2 |p ε -pε | + 2|p ε ( Ȳε0 -Ỹε0 )| + 2|p ε -pε | ≤ 2 l 2 pε 3| pε pε -1| + | Ȳε0 -Ỹε0 | Using the fact that pε ≲ 2 -l , | Ȳε0 -Ỹε0 | ≲ Ln2 l n on Aand (4.3.6), we obtain that, on A and B,|f lk -flk | ≲ on A and B, to ∥f Lnf Ln ∥ ∞ ≲ Ln2 Ln nWe have, denoting by E X the expectation under the posterior distribution,

  It now remains to bound the part with the frequencies l 0 < l ≤ J n (α). On A n , we have sup l 0 <l≤Jn(α)

  uses Lemma 43 to conclude the proof. Lemma 38. For l ≤ L n ,on the event B, ε ∈ E with |ε| = l, |f 0,lk | + nL n )

2 Ln 16 .

 16 Lemma 40. There existsC 2 > 0 such that for any l > L n , ε ∈ E with |ε| = l, on B, πε ≲ e -C 2 l √ n .

l 2 √

 2 n e C 1 l . Choosing κ = C 1 + C 2 in the definition of π l leads us to πε ≲ π l T ≲ e -C 2 l √ n .

  lk -f lk | > γ log(n)/n|X) ≲ 1/n BProof. 1) We have, on B, using Lemma 36Π(S c ∩ J n (γ) ̸ = ∅|X) ≤ (l,k)∈Jn(γ) Π(f lk = 0|X)The first term is in fact 0, as|y l -1 2 | = |f 0,l,k | 2 -l/2 2F 0 (I l k ) ≳ 2 l/2 log(n)/n > ∆ l for (l, k) ∈ J n (γ) with γ large enough. This finally gives us that Π(S c ∩ J n (γ) ̸ = ∅|X) ≲ l≤Ln 2 l /n2) We have, on B, using Lemma 40Π(S ∩ {l > L n } ̸ = ∅|X) ≤ (l,k):l>Ln Π(f lk ̸ = 0|X) 0,lk -f lk | > γ log(n) n |X)Looking at the expectation under f 0 of each term, and in view of recentering by the posterior mean f , one writesE f 0   Π(|f 0,lk -f lk | > |f 0,lk -f lk | > γ log(n) n |X)1l{|f 0,lk -flk |≤ γ B, using (4.3.4), we have that|f 0,lk -flk | ≲ 2 -l(α+1/2) n/ log n) 1/(2α+1)) n if α < 1/2. If α ≥ 1/2, we have |f 0,lk -flk | ≲ 2 -l(α+1/2)

  3.7), |f lk -flk | ≲ Ln n on A and B, so the second term is bounded byP n f 0 (B c ) + Π(A c |X) ≲ e -CLn. This finally leads toE f 0   Π( max (l,k)∈Jn(γ) |f 0,lk -f lk | > γ log(n) n |X)   ≲ (l,k):l≤Ln e -CLn ≲ 2 -Ln

Table 2

 2

		5	1	1.5	2	2.5	3	3.5
	Second moment 394 1110 2847 5716 8093 16530 34791
	Median	173 167 169 174 185	209	219
	Mean	157 158 166 172 182	224	336

.1 Empirical risks R2 , Rmed , Rmean for Laplace slabs Lap(a) and a ∈ [0.5, 3.5]

  so this is also the case. Conclude that the last display is bounded above by Cnπ1 (1 + ζ 2 2 ) ≤ C ′ nπ 1 ζ 2 2 .Using (2.3.14), this term is itself bounded by Cs n log(n/s n ), which concludes the proof of the Theorem, given (2.3.13)-(2.3.14).

  1] denote the space of all measurable functions on [0, 1] that are bounded up to a set of Lebesgue measure 0, equipped with the (essential) supremum norm ∥ • ∥ ∞ . The class C α [0, 1], α ∈ (0, 1], of Hölder functions on the interval [0, 1] is the set of functions g on [0, 1] such that sup x̸ =y∈[0,1] |g(x) -g(y)|/|x -y| α is finite.Let us recall that if a function g belongs to C α , α ∈ (0, 1], then the sequence of its Haar-wavelet coefficients ⟨g, ψ lk ⟩ satisfies sup

	2 l(1/2+α) |⟨g, ψ lk ⟩| < ∞.	(4.1.3)
	0≤k<2 l ,l≥0
	For a given α > 0, and n ≥ 1, define
	ε * n,α =	n log n	α 2α+1

Optimal concentration for Cauchy Slab. The next direction was to use a standard Cauchy Slab instead of a standard Laplace, and this led to the following result, showing optimal concentration uniformly in θ ∈ ℓ 0 [s n ]

+ π ε0 Beta(α ε0 , α ε1 ) which can be seen as the following diagram, using a sequence (γ ε0 ) ε of Bernoulli variables.

Remerciements

Chapter 3

Sharp asymptotic minimaxity of spike and slab empirical Bayes procedures

Introduction

Model

Consider the sequence model (1.2)

with θ = (θ 1 , . . . , θ n ) ∈ R n and ε 1 , . . . , ε n i.i.d. N (0, 1). Suppose as before that the 'true' vector θ 0 belongs to

Recall that the minimax rate over ℓ 0 [s n ] for the Euclidean norm (not renormalized by n) is 2s n log( n sn )(1 + o( 1)).

The first term, as it involves an increasing function of α (as here α is small enough), is bounded by 2c 1 Cs n log ((n/s n ) log(log n/s n )) / log 2 (2c 2 log(n/s n log(log n/s n )))), which is bounded by

which completes the proof.

Random α bounds

For convenience we work with the 'threshold' τ (α) (as we take α ≤ α 0 such that τ (α) = τ (α)), although other choices t(α), ζ(α) should be essentially equivalent.

Lemma 29 (no signal or small signal). Let α be a fixed non-random element of (0, 1). Let α be a random element of [0, 1] (chosen small enough so that τ (α) = τ (α)) that may depend on x ∼ N (0, 1) and on other data. Then there exists C 1 > 0 such that

There exists C 2 > 0 such that for any real µ, if x ∼ N (µ, 1),

Proof. Using the bound (3.3.8) on r 2 (α, 0, x),

For the first term in the last display, one bounds the indicator from above by 1 and proceeds as in the proof of (3.2.2) to bound its expectation by Cτ (α)

The first part of the lemma follows by noting that E[(1

The second part of the lemma follows from the fact that using (3.3.9),

Chapter 4

Adaptive Pólya trees on densities using a Spike and Slab type prior

Introduction

The paper [START_REF] Castillo | Polya tree posterior distributions on densities[END_REF] showed that, for well chosen parameters, Pólya trees are able to model smooth functions and to induce posterior distributions with optimal convergence rates in the minimax sense for a range of Hölder regularities and also derived a Bernstein-von Mises theorem as well as a Donsker-type theorem, but the chosen parameters depend on the Hölder regularity of the true density. Here, as in Castillo (2017b), we will follow a multiscale approach to obtaining adaptive rates and limiting shape results, introduced in Castillo and Nickl (2013), [START_REF] Castillo | On the bernstein von mises phenomenon for nonparametric bayes procedures[END_REF], [START_REF] Castillo | On bayesian supremum norm contraction rates[END_REF] with connections to semiparametric functionals [START_REF] Castillo | A bernstein-von mises theorem for smooth functionals in semiparametric models[END_REF].

Definition of a Pólya tree

Here we recall the construction of a standard Pólya tree.

First let us introduce some notation relative to dyadic partitions. For any fixed indexes l ≥ 0 and 0 ≤ k < 2 l , the rational number r = k2 -l can be written in a unique way as ε(r) := ε 1 (r) . . . ε l (r), its finite expression of length l in base 1/2 (note that it can end with one or more 0). That is, ε i ∈ {0, 1} and

Define B an event on the dataspace on which, simultaneously for the countable family of indexes l ≥ 1, 0 ≤ k < 2 l , for M large enough to be chosen,

We recall Lemma 4 in [START_REF] Castillo | Polya tree posterior distributions on densities[END_REF] which ensures that one can restrict to the event B in the following. We note that, as here the levels l > L are truncated in the prior and posterior, one actually only needs the control (4.3.2) for l ≤ L.

Lemma 35. Let X 1 , . . . , X n be i.i.d. of density f 0 on [0, 1], with f 0 bounded away from 0 and infinity. Then for M large enough there exists B > log(2) such that for every positive integer n

Proof of Theorem 19

Let us recall the definition of L n in (4.2.8).

Since we have f -

), the proof will be split in four parts, where we study each of these terms separately.

Term

Term f

.

Then, for M > 0 large enough, on the event B, there exists C > 2 log(2) such that

Proof. We will write Z a random variable drawn from a Beta(a ε (X), a ε ′ (X)), whose density will be noted as b.

Note first that the set {|u -Ȳε | > B} can be written as

For the first term, due to Lemma 36 we have

The first term is 0 so (I)

i ≲ e -CLn given our choice of ∆ i .

For the second term, we first write that, for Z as above,

The first probability is 0 and the second one is bounded by e -M 2 Ln 16 using Lemma 45.

Proofs

provided C is chosen greater than 2 log(2), which concludes the proof.

The following Lemmas are borrowed from [START_REF] Castillo | Polya tree posterior distributions on densities[END_REF].

Lemma 42. On the event B,

Proof. This is the quantity w i y i -1 in Lemma 1 of [START_REF] Castillo | Polya tree posterior distributions on densities[END_REF].

Lemma 43 (Lemma 3 of Castillo (2017b)). Let {y i } 1≤i≤L , {w i } 1≤i≤L be two sequences of positive real numbers such that there are constants c 1 , c 2 with

Then there exists c 3 depending on c 1 , c 2 only such that

Lemma 44. On the event B N X (I ε0 ) + a y ε0 (N X (I ε ) + 2a)

Proof. This is the quantity Ȳε0 y ε0 -1 in Lemma 2 of [START_REF] Castillo | Polya tree posterior distributions on densities[END_REF].

Lemma 45 (Lemma 6 of Castillo (2017b)). Let ϕ, ψ belong to (0, ∞). Let Z follow a Beta(ϕ, ψ) distribution. Suppose, for some reals c 0 , c 1 , 0 < c 0 ≤ ϕ/(ϕ + ψ) ≤ c1 < 1 and ϕ ∧ ψ > 8

Then there exists D > 0 depending on c 0 , c 1 only such that for any x > 0,