Diala Ivan

Emma, Julien, Matthieu, Mareva, Cécile, Boupi Alice Simon

Joran Théo

Charley

Keywords: inverse problems, Plug & Play methods, Langevin algorithms, Markov chain, Monte-Carlo methods, stochastic gradient descent, denoising, deblurring, interpolation. iii

 in 2013, Plug & Play (PnP) methods are often applied for image restoration in a Bayesian context. These methods aim at computing Minimum Mean Square Error (MMSE) or Maximum A Posteriori (MAP) for inverse problems in imaging by combining an explicit likelihood and an implicit a-priori defined by a denoising algorithm. In the literature, PnP methods differ mainly in the iterative scheme used for both optimization and sampling. In the case of optimization algorithms, recent works guarantee the convergence to a fixed point of a certain operator, fixed point which is not necessarily the MAP. In the case of sampling algorithms in the literature, there is no evidence of convergence. Moreover, there are still important open questions concerning the correct definition of the underlying Bayesian models or the computed estimators, as well as their regularity properties, necessary to ensure the stability of the numerical scheme. The aim of this thesis is to develop simple but efficient restoration methods while answering some of these questions. The existence and nature of MAP and MMSE estimators for PnP prior is therefore a first line of study. Three methods with convergence results are then presented, PnP-SGD for MAP estimation and PnP-ULA and PPnP-ULA for sampling. A particular interest is given to denoisers encoded by deep neural networks. The efficiency of these methods is demonstrated on classical image restoration problems such as denoising, deblurring or interpolation. In addition to allowing the estimation of MMSE, sampling makes possible the quantification of uncertainties, which is crucial in domains such as biomedical imaging. Lastly, the influence of the denoiser on the posterior is investigated and a comparison between the Bayesian probabilities reported by the model and the frequentist probabilities arising from a large number of repetitions of an experiment is drawn.

Résumé

Ce manuscrit de thèse est consacré à l'étude des méthodes Plug & Play appliquées à des problèmes inverses rencontrés en restauration d'images. Depuis les travaux de [START_REF] Venkatakrishnan | Plug-and-play priors for model based reconstruction[END_REF] en 2013, les méthodes Plug & Play (PnP) sont souvent appliquées pour la restauration d'image dans un contexte Bayésien. Ces méthodes visent à calculer les estimateurs Minimum Mean Square Error (MMSE) ou Maximum A Posteriori (MAP) pour des problèmes inverses en imagerie en combinant une vraisemblance explicite et un a-priori implicite défini par un algorithme de débruitage. Dans la littérature, les méthodes PnP diffèrent principalement par le schéma itératif utilisé que cela soit pour l'optimisation ou l'échantillonnage. Dans le cas des algorithmes d'optimisation, des travaux récents garantissent la convergence vers un point fixe d'un certain opérateur, point fixe qui n'est pas nécessairement le MAP. Dans le cas des algorithmes d'échantillonnage de la littérature, il n'existe pas de preuves de convergence. Par ailleurs, il reste d'importantes questions ouvertes portant sur la bonne définition des modèles Bayésiens sous-jacents ou encore des estimateurs calculés, ainsi que leurs propriétés de régularité, nécessaires pour assurer la stabilité du schéma numérique. Le but de cette thèse est de développer des méthodes de restauration simples mais efficaces tout en répondant à ces interrogations. L'existence et la nature des estimateurs MAP et MMSE pour des a-priori PnP constitue donc un premier axe d'étude. Deux méthodes avec des résultats de convergence sont alors présentées, PnP-SGD pour l'estimation du MAP et PnP-ULA pour l'échantillonnage. Un intérêt particulier est porté aux débruiteurs encodés par des réseaux de neurones profonds. L'efficacité de ces méthodes est démontrée sur des problèmes classiques de restauration d'image tels le débruitage, le défloutage ou l'interpolation. En plus de permettre l'estimation du MMSE, l'échantillonnage rend possible la quantification d'incertitudes, ce qui est crucial dans des domaines tels que l'imagerie biomédicale. Enfin, l'influence du débruiteur sur l'a-posteriori estimée est questionnée et une comparaison entre les probabilités données par notre modèle et les probabilités fréquentistes provenant d'un grand nombre d'expériences est faite. Mots-Clefs : problèmes inverses, méthodes Plug & Play, algorithmes de Langevin, chaîne de Markov, méthodes de Monte-Carlo, descente de gradient stochastique, débruitage, défloutage, interpolation. iv que je suis au bout du rouleau et que t'es en plein boulot compte beaucoup pour moi. T'es mon roc. Avec Adva, un bel ami, votre appartement est une deuxième maison, un pied à terre pour la famille. J'ai aussi une pensée particulière pour Elliot avec qui tout a commencé et Thomas F. un altruiste forcené de la route.

J'ai une pensée toute particulière pour Maxime. Ca va faire 10 ans qu'on se connaît et depuis 10 ans, q'on ne se lâche pas. Quand je pense à nous, je pense à de deux gros tocards toujours prêts à inventer de nouvelles histoires, à s'attaquer aux questions les plus existentielles qui soient (la réponse est souvent dans la question en fait). Quand je pense à toi, je vois un ami fidèle et sûr. Un vrai ami qui me pousse toujours à me dépasser, gentiment, patiemment. On trouvera notre voie, c'est sûr ! J'ai encore une pensée pour mes amis/amies de Munich. Vous êtes une merveilleuse bande de copains/copines et nous voir encore actifs/actives à Paris et ailleurs me fait tellement plaisir. Avec vous je me sens vraiment bien. J'adore nos barbecues à la poêle, nos raclettes aux pates, nos découvertes auvergnates. Merci à Manon C. (quelle humanité !), Manon G. (quel humour !), Coralie, Hélène (quelle coloc !), Romain (que de profondeur dans nos conversations !), Thibaud (quelle empathie !). J'ai une pensée particulière pour mon amie Josépha. Tu me manques j'avoue et parfois c'est frustrant de ne pas te voir plus. Mais j'adore nos appels impromptus. Et que dire de Clément D., le dresseur de chats/chattes aux converses rouges et aux t-shirts de prisonnier ? Vivement nos soirées charentaises (au McDo ?) avec Eléonore (que j'adore) et Rooney (!!!!!). Hâte de rencontrer Poppy ! J'ai aussi une pensée pour Pierre que j'aimerais voir plus aujourd'hui. Merci pour l'astuce de l'éclairage. Et Anne, ma partenaire de bibliothèque à Munich, mais surtout une amie pour la vie. J'aimerais qu'on se voie plus, tellement plus. Et Nini, Madame jamais en avance ! Ta paire de ciseaux retrouvée n'a pas aidé. T'es trop drôle et grâce à toi je vais parfois à la MEP (cher lecteur si tu ne connais pas, tu as gagné le mépris de Nina). Coucou Hadrien ! Anne-Katharina, j'espère que tu crois que je t'ai oubliée parce qu'il est évident que non. Parce que tu dois faire partie des noms auxquels je pense depuis le début, ou plutôt toujours (comme Dédé avec Mbappé). On s'aide à se réaliser mutuellement, à tatons. C'est beau. On trouve quand même le temps d'aborder des questions de fond avec le sourire (Quels sont les meilleurs raviolis Giovanni Rana ?). Pour tout ça, je te suis sincérement reconnaissant. Alors merci. Et à bientôt ! Un petit mot aussi pour mes camarades de randonnée Jean-Baptiste et Clément P., amateurs de croquettes. Avec vous souffrir n'a jamais été aussi beau. Hâte de partir avec vous pour de nouvelles aventures. C'est quoi le prochain plan ? La traversée du Vercors ? la descente de la Loire en canoë ? Une bière dans le 18 ème ? Nous approchons de la fin et j'ai l'impression que cette section n'aboutira malheureusement pas à un petit article de conférence... A défaut, je continue sur ma lancée et en profite pour remercier mes amis de lycée. Alexandre K., Edouard, Gauthier, Louise, Tom et Violette, c'est vrai qu'on sait poser le cerveau et s'amuser tous ensemble. A votre contact, je me vois évoluer. J'ai une pensée pour Quiz avec qui je partage des "déjeuners" d'une rare intensité (on devrait plutôt parler de demi-journées je pense). On a encore tant de choses à analyser, décortiquer ensemble, ça promet ! J'ai une pensée pour Adélie, ma fidèle partenaire de TP, qui a toujours su m'accompagner, me rassurer. Parfois je me dis que c'est moi qui ai deux ans de moins... Merci aussi à Edith, que je vois peu mais qui sait être là quand j'en ai besoin. Même si on a bien évolué, je suis heureux de t'avoir pour amie après toutes ces années. Léa, je ne sais pas trop quoi dire tellement j'ai à raconter. Te retrouver après toutes ces années m'a fait tellement de bien. J'avais oublié. Avec toi, je peux être moi et ça, ça n'a pas de prix. Merci. Je n'oublie pas Lucie, mon petit biscuit, avec qui j'aime partager un thé, prendre le goûter et parfois le petit-déjeuner. Une amie qui chaque jour me fait viii prendre conscience qui je suis. J'ai aussi une pensée pour Aurélie, Lorène et A-P Gignac. Nos retrouvailles sont trop rares mais toujours mémorables. Un coucou à Clara Zaza parce que je ne sais pas où parler de toi.

Je tiens à remercier la famille Paetzold pour son chaleureux accueil et ma famille bien sûr. Sans votre soutien, tout ça n'aurait jamais été possible. Merci à mon père pour son goût de l'effort dans le sport. C'est une passion, parfois dévorante, mais qui me permet de m'évader. Aujourd'hui je crois avoir trouver la manière de m'exprimer dedans, mais sans toi papa, cela n'aurait pas été possible. Merci à ma soeur, Céline, que j'admire. Si j'ai fait une thèse, tu n'y es pas étrangère. J'aime nos rares discussions au cours desquelles on refait le monde (et la famille). Un océan et un continent ne sont pas assez pour nous séparer. Merci à ma mère. Pour tout, pour rien, partout, pour toujours. Aimante, attentionnée, inquiète, sensible, déterminée, courageuse. La meilleure part de moi. On en a fait du chemin depuis mon exposé de troisième sur le tungstène. Je vous aime. J'ai une pensée pour mon grandpère (forcément) parti trop tôt et ma cousine Héloïse que je suis heureux de découvrir un peu chaque jour. Heureux aussi pour François et le petit Samuel.

Et puis et puis et puis il y a Clara qui est belle comme un soleil et qui m'aime pareil que moi j'aime Clara même qu´on se dit souvent qu'on aura une maison avec des tas de fenêtres avec presque pas de murs et qu'on vivra dedans et qu´il fera bon y être et que si c'est pas sûr c'est quand même peut-être. Clara, c'est une ode à l'amour un peu tous les jours. Clara ce sont ces rencontres ambiguës si plaisantes, ces escapades romantiques décidées sur un coup de tête, ces promesses lancées sur un morceau d'oreiller, ce sentiment d'immortalité, ces rires qui prouvent le silence et ces jeux toujours plus déjantés. Clara ce sont aussi ces réveils matinaux difficiles, ces corvées de ménage partagées, ces discussions parfois pénibles une fois la nuit tombée, des pleurs parfois. C'est cette envie de rester avec toi pour et malgré tout ça. C'est ce que j'appelle Amour. Merci à toi, qui me soutiens et m'aides à traverser les épreuves de la vie comme personne. Parce que, oui la vie est dure mais elle est aussi très belle. Surtout avec toi. Je t'aime. Je te rappelle aussi que tu es Bombur au cas où tu l'aurais oublié. Je te le rappellerai sans doute ce soir en rentrant ou demain au réveil. ix Ce manuscrit est dédié à ma mère, qui a toujours su m'accompagner. J'aimerais que cela ne s'arrête jamais.

Il est aussi dédié à mon ami Baptiste L. Ce manuscrit, c'est une sorte de mausolée, ma manière de ne jamais t'oublier.

x En effet, la solution de droite présente une lésion contrairement à celle de gauche. Quelle solution doit-on choisir ? Quelle confiance avons-nous dans la lésion ? Ces questions peuvent être répondues dans le cadre bayésien. Images tirées de [START_REF] Repetti | Scalable bayesian uncertainty quantification in imaging inverse problems via convex optimization[END_REF]. . . . . . . . . . . . . . . . . . . . . . . . . . . 1.2 Principe d'un DPPM. p θ correspond à la distribution de débruitage que nous apprenons par inférence variationnelle et q à la distribution de bruitage. Image issue de [START_REF] Kawar | Denoising diffusion restoration models[END_REF]. . . . . . . . . . . . . . . . . . . . . . . . . . 1.3 llustration du principe des méthodes d'unrolling. Dans ce cas, le réseau de neurones vise à résoudre un problème inverse de super-résolution via un schéma ADMM linéarisé. Il existe trois modules constitutifs : le module apriori P qui est appris pendant l'apprentissage, le module relatif aux données D comprend des informations sur le processus d'observation et le module de mise à jour U. Image tirée de [START_REF] Laroche | Deep model-based super-resolution with nonuniform blur[END_REF]. . . . . . . . . . . . . . .

Conclusion and

2.1

Illustration of the advantage of the Bayesian framework over the variational paradigm. Two possible solutions to an inverse problem in medical imaging leading to two possible different diagnoses. Indeed, the solution on the right presents a lesion whereas the one on the left does not. Which solution should we select ? How confident are we on the lesion ? These questions can be answered in the Bayesian framework. Illustrations from [START_REF] Repetti | Scalable bayesian uncertainty quantification in imaging inverse problems via convex optimization[END_REF]. 2.2 Principle of a DPPM. p θ corresponds to the denoising diffusion distribution we learn by variational inference and q to the noising process distribution. Image taken from [START_REF] Kawar | Denoising diffusion restoration models[END_REF]. . . . . . . . . . . . . . . . . . . . . . . 2.3 Illustration of the deep-unrolling principles. In this case, the neural network aims at solving a super-resolution inverse problem via a linearised ADMM scheme. There are three constitutive modules: the prior module P that is learnt during the training, the data module D which incorporates information about the observation process and the update module U. Image taken from [START_REF] Laroche | Deep model-based super-resolution with nonuniform blur[END_REF] (5/255) 2 with α = 0.25 and TV -L 2 initialization. Left: Evolution of the average PSNR computed for K = 10 independent noise realizations for each image. A thousand of iterations seem to be sufficient to leave the burnin phase and enter the stationary phase. The decay of the discretization step-size δ k does not alter the results, which suggests that the algorithm has converged. Right: Evolution of the average gradient norm of the log-posterior computed over the 10 experiments for each image. In less than 500 iterations, it stabilizes around 0.4 for each image. These plots suggest that the algorithm has converged. The decrease observed after 5000 iterations is explained by the decay of the discretization step-size δ k and does not alter the final result. 4.6 Plug & Play deblurring. Image are blurred with a 9 × 9 uniform kernel, a Gaussian noise of standard deviation σ 2 = (1/255) 2 is added. The denoiser D ε is trained at ε = (5/255) 2 . The plots shows mean and standard deviation values of PSNR and SSIM over K=10 independent noise realizations for each of the six images and different values of the regularization parameter α. Initialization plays a very minor role in this case and all algorithms achieve similar (nearly optimal) performance for α = 0.3, except for FBS which requires a larger (sub-optimal) α to converge. . . . . . . . . . . . . . . for all images in the pixel domain. After 5e5 iterations, sample pixels are nearly uncorrelated in all spatial directions for the images Traffic, Alley, Bridge and Goldhill. For the images Cameraman and Simpson, in the slowest direction, samples need more iterations to become uncorrelated. . . . . . . . . 5.7 Log-standard deviation maps in the Fourier domain for the Markov chains defined by PnP-ULA for the deblurring problem. First line: images Cameraman, Simpson, Traffic. Second line: images Alley, Bridge and Goldhill. For the first three images, we clearly see that uncertainty is observed on frequencies that are near the kernel of the blur filter (shown on the right), and is also higher around high frequencies (i.e. around edges and textured areas in images). For the last three images, very high uncertainty is observed around some specific frequencies. In the direction of these frequencies, the Markov chain is moving very slowly and the mixing time of the chain is particularly slow, as shown on Figure 5.9. . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.8 Evolution of the L 2 distance between the final MMSE estimate and the samples generated by PnP-ULA for the deblurring problem after the burn-in phase. For images as Cameraman or Simpson, samples randomly oscillate around the MMSE. On the contrary, for images as Bridge or Goldhill, the plot is structured, meaning that samples are still correlated. . . . . . . . . . . xvi 5.9 ACF for the deblurring problem. The ACF are shown for lags up to 1.75e5 for the three images Cameraman, Simpson and Traffic (see the two plots to the left) and independence seems to be achieved in all directions. For the three other images, independence is not achieved in the slowest direction (corresponding to the most uncertain frequency of the samples in the Fourier domain) even after 1e6 iterations. . . . . . . . . . . . . . . . . . . . . . . . . . 5.10 Left: PSNR evolution of the estimated MMSE for the interpolation problem.

After 5e5 iterations, the convergence of the first order moment of the posterior distribution seems to be achieved for all images. Middle and right: PSNR evolution of the estimated MMSE for the deblurring problem. The convergence for the posterior mean can be fast for simple images such as Cameraman, Simpson, and Traffic (for these images the PSNR evolution is shown for the first 5e5 iterations). Increasing δ increases the convergence speed for these images by a factor close to 2. For more complex images, such as Alley or Goldhill, the convergence is much slower and is still not achieved after 3e6 iterations with PPnP-ULA for δ = 6δ th . . . . . . . . . . . . . . . . . . . . . . De nombreux problèmes nécessitent d'inférer un signal à partir de données partiellement observées et souvent bruitées. Typiquement, ces problèmes apparaissent en ingénierie biomédicale, en géophysique, en astronomie ou encore en finance. Ces problèmes sont appelés problèmes inverses car les résoudre revient à inverser le processus d'observation de la quantité d'intérêt afin de recupérer un signal à partir des données observées. De tels problèmes sont souvent modélisés par l'Equation (1.1)

Results comparison for the interpolation task of the images presented in

y = Ax + n , (1.1)
où y correspond aux données observées, x à la quantité que nous souhaitons inférer, A à l'opérateur de dégradation, qui modélise le processus d'observation en l'absence de bruit et n au bruit de mesure.

Les problèmes inverses rencontrés sont souvent mal-posés, c'est-à-dire qu'une petite pertubation au moment de l'observation peut produire de grandes erreurs dans le signal que l'on souhaite retrouver ou qu'il existe plusieurs signaux possibles consistants avec les données observées. Tenir compte de ce caractère mal-posé est crucial, spécifiquement dans des domaines où des décisions sont prises à partir du signal restauré, comme en imagerie médicale. Par ailleurs, les problèmes inverses en imagerie sont souvent des problèmes en haute dimension, ce qui rend leur résolution d'autant plus difficile. Dans beaucoup d'applications, les signaux admissibles appartiennent à un petit sousensemble de l'espace ambiant. Par exemple en imagerie, il est communément supposé que les images naturelles sont concentrées sur une variété de l'espace ambiant [START_REF] Fefferman | Testing the manifold hypothesis[END_REF]. Ceci a motivé le développement de régularisateurs qui apportent une connaissance a priori sur le signal que l'on souhaite restaurer.

Les approches variationnelles sont très populaires pour résoudre les problèmes inverses en imagerie. Elles commencent par construire une fonction objectif que l'on cherche à minimiser. Cette fonction objectif résulte généralement de la somme de deux termes, le terme d'attache aux données qui mesure la distance entre l'observation y et la solution à notre problème et le terme de régularisation qui promeut des images avec certaines propriétés recherchées ou qui se trouvent dans le voisinage d'une variété. Comme exemples de régularisateurs classiques, on peut citer la variation totale (TV) [START_REF] Rudin | Nonlinear total variation based noise removal algorithms[END_REF] qui favorise des images avec des gradients parcimonieux (en particulier les images constantes par morceaux), ou encore les régularisateurs favorisant des solutions parcimonieuses dans des domaines transformés tels que les bases d'ondelettes [START_REF] Donoho | Ideal spatial adaptation by wavelet shrinkage[END_REF]. Bien que beaucoup de progrés aient été réalisés dans ce domaine, les régularisateurs explicites ne parviennent généralement pas à capturer la complexité des images naturelles. De plus, ces régularisateurs sont souvent convexes afin de tirer profit des méthodes d'optimisation idoines, ce qui peut limiter l'expressivité des modèles associés.

Les problèmes inverses mal posés peuvent également être abordés dans un cadre bayésien. Dans ce cadre, les données observées et le signal à reconstruire sont considérés comme des réalisations de variables aléatoires. Dans le paradigme Bayésien, nous cherchons à déterminer la distribution a posteriori, i.e. la distribution de x sachant les données observées y en appliquant la loi de Bayes

p(x|y) = p(x)p(y|x)/ p(z)p(y|z)dz , (1.2)
où p(y|x) est appelée la vraisemblance et exprime notre connaissance de la dégradation subie par l'image x pour aboutir à l'observation y, et où p(x) est appelé la distribution a priori et encode l'information dont nous disposons à propos de x avant d'observer y. La distribution a posteriori décrit toutes les solutions possibles au problème inverse considéré, compte tenu des données observées. Elle représente une solution bien plus complète que celle proposée dans le contexte variationnel qui fournit une simple estimation de la quantité d'intérêt x. En effet, connaître la distribution a posteriori permet tout d'abord de proposer différentes solutions au problème inverse. De plus, il est possible de quantifier l'incertitude des solutions proposées, ce qui est un avantage indéniable, notamment dans les applications où la prise de décision est basée sur la restauration proposée. Par exemple, la Figure 1.1 montre les limites du paradigme variationnel puisque deux solutions à Ces questions peuvent être répondues dans le cadre bayésien. Images tirées de [START_REF] Repetti | Scalable bayesian uncertainty quantification in imaging inverse problems via convex optimization[END_REF].

un problème inverse en imagerie médicale conduisent à deux diagnostics distincts, l'une des restaurations présentant une lésion et l'autre pas. Cependant, les images étant des objets vivant dans des espaces de haute dimension, il est impossible d'un point de vue computationnel d'estimer cette distribution a posteriori. Par ailleurs, se pose également la question du choix de la distribution a priori. En pratique, elle doit encoder des informations significatives sur la quantité d'intérêt x et ne pas impliquer des opérations de calcul trop intensives. C'est pourquoi les a priroris explicites non-appris ont longtemps été limités à des modèles log-concaves simples [START_REF] Bardsley | Mcmc-based image reconstruction with uncertainty quantification[END_REF][START_REF] Louchet | Posterior expectation of the total variation model: Properties and experiments[END_REF][START_REF] Durmus | Efficient bayesian computation by proximal markov chain monte carlo: when langevin meets moreau[END_REF]. Le développement récent des réseaux de neurones constitue une véritable avancée en imagerie. Les méthodes basées sur l'apprentissage profond permettent aujourd'hui d'obtenir des résultats de pointe dans de nombreux domaines, tels que la vision par ordinateur, la reconnaissance du langage ou encore les prévision météorologiques, et cela pour une grande variété de tâches. Ces approches consistent à adapter un modèle générique à un problème spécifique en entraînant notre modèle sur des données d'entraînement et cela de manière agnostique, c'est-à-dire sans connaissance sur la manière dont sont générées les données. Grâce aux ressources informatiques dont nous disposons, notre modèle apprend automatiquement une structure au sein des données fournies. Le succès des méthodes par apprentissage profond est fondé sur l'abondance des données d'apprentissage. Cependant, les données relatives à un problème inverse ne sont pas toujours disponibles en abondance. En effet, elles peu-vent coûter très cher à générer. En outre, ces méthodes purement axées sur les données sont très spécifiques au problème. Un petit changement dans le processus d'observation implique un nouveau processus d'apprentissage qui est coûteux en temps et en calcul. Ceci limite l'utilisation de ces approches pour la résolution de problèmes inverses. Cependant, depuis les années 2010, de nombreux efforts ont été faits pour développer des méthodes combinant les approches basées sur les données et celles exploitant les informations que nous avons sur le problème inverse pour résoudre les problèmes inverses.

Un première famille de méthodes consiste à apprendre un régularisateur/a-priori à partir des données. Dans ce qui suit nous décrivons certaines de ces approches.

• Méthodes Plug & Play (PnP): [START_REF] Venkatakrishnan | Plug-and-play priors for model based reconstruction[END_REF] proposent d'utiliser des réseaux de neurones afin de définir un régularisateur implicite via un algorithme de débruitage tout en conservant une vraisemblance explicite. Cette dernière est généralement supposée connue et calibrée [START_REF] Arridge | Solving inverse problems using datadriven models[END_REF]. L'idée vient du fait que de nombreux algorithmes classiques d'optimisation font intervenir l'opérateur proximal du potentiel a-priori qui agit comme un débruiteur. Les approches Plug & Play mettent en relation un algorithme de débruitage avec un opérateur proximal ou un gradient associé à la densité a-priori. Elles sont principalement utilisées pour de l'estimation ponctuelle [START_REF] Ryu | Plug-and-play methods provably converge with properly trained denoisers[END_REF][START_REF] Sun | An online plug-and-play algorithm for regularized image reconstruction[END_REF][START_REF] Sun | Scalable plug-and-play admm with convergence guarantees[END_REF][START_REF] Xu | Provable Convergence of Plug-and-Play Priors with MMSE denoisers[END_REF][START_REF] Zhang | Plug-and-play image restoration with deep denoiser prior[END_REF]Hurault et al., 2022a,b). Elles obtiennent souvent des résultats de pointe sur une grande variété de tâches [START_REF] Zhang | Plug-and-play image restoration with deep denoiser prior[END_REF]. Les méthodes Plug & Play sont également appliquées pour l'échantillonnage comme dans [START_REF] Kadkhodaie | Solving linear inverse problems using the prior implicit in a denoiser[END_REF][START_REF] Guo | Agem: Solving linear inverse problems via deep priors and sampling[END_REF]. Ces méthodes sont flexibles et ne nécessitent que l'entraînement d'un réseau de neurons de débruitage, qui est léger par rapport à d'autres réseaux neurones. Leurs fondements théoriques sont un domaine de recherche actif et si nous commenccons à mieux les comprendre pour l'estimation ponctuelle, à notre connaissance, aucun résultat de convergence n'a été donné pour l'échantillonnage avant les travaux présentés dans ce manuscrit.

• Méthodes basées sur le score-matching:

Le score-matching a été conçu à l'origine pour l'apprentissage de modèles statistiques non normalisés basés sur des échantillons i.i.d. et provenant d'une distribution de données inconnue [START_REF] Hyvärinen | Estimation of non-normalized statistical models by score matching[END_REF]. Cependant, dans sa forme originale, le scorematching n'est pas adapté aux problèmes de haute dimension. C'est pourquoi [START_REF] Bengio | Generalized denoising auto-encoders as generative models[END_REF] propose une variante du score-matching original pour estimer le score d'une densité cible légèrement bruitée. Pour éviter les gradients mal définis, [START_REF] Song | Generative modeling by estimating gradients of the data distribution[END_REF] entraîne un réseau de neurones qui approxime le score de différentes densités cibles bruitées et l'incorpore dans un schéma de Langevin recuit afin de générer de nouveaux échantillons à partir de la distribution empirique associée aux données d'entraînement. (Kawar et al., 2021b,a) adaptent cette méthode afin de résoudre des problèmes inverses classiques d'imagerie en échantillonnant la distribution a-posteriori. Bien que ces méthodes produisent des résultats impressionnants, le réseau de neurones utilisé pour approximer les scores est souvent très lourd et exigeant en termes de calcul. En effet, il vise à approximer directement le score de la densité a-priori induite par un ensemble de données. Il s'agit d'une tâche plus complexe que le simple débruitage. De plus, dans les méthodes actuelles, le score de l'a-priori est associé à un jeu de données spécifique (comme les chambres CelebA-HQ ou LSUN). Ainsi, l'image que nous souhaitons estimer pour résoudre le problème inverse considéré doit appartenir à la même classe d'images que les images de l'ensemble d'entraînement. Enfin, il n'est toujours pas clair quelles distributions sont échantillonnées. inférence variationnelle et q à la distribution de bruitage. Image issue de [START_REF] Kawar | Denoising diffusion restoration models[END_REF].

• Modèles probabilistes de diffusion du bruit (DDPMs) (Ho et al., 2020) :

Ils ont été développés dans un contexte génératif. Leur objectif est de produire de nouveaux échantillons à partir d'un jeu de données. Ces échantillons doivent être issus de la même distribution que celle des échantillons du jeu de données. Un DDPM est une chaîne de Markov paramétrée et entraînée par inférence variationnelle pour produire des échantillons correspondant aux données d'entraînement. Les transitions de cette chaîne sont apprises pour inverser un processus de diffusion de bruit. Ce processus de bruitage est généralement une chaîne de Markov qui ajoute progressivement du bruit aux données d'apprentissage jusqu'à ce que celles-ci ressemblent à du bruit pur. Ensuite, après avoir appris le processus de débruitage, nous pouvons, à partir du bruit pur, générer de nouveaux échantillons. Dans le cas le plus simple, le processus de bruitage consiste à ajouter de petites quantités de bruit gaussien et les noyaux de transitions conditionnelles sont également gaussiens. Cela permet un paramétrage particulièrement simple du réseau de neurones.

Fort du succès de ces approches pour la génration d'images, [START_REF] Kawar | Denoising diffusion restoration models[END_REF][START_REF] Saharia | Image superresolution via iterative refinement[END_REF] adaptent les méthodes DDPM en ajoutant des informations sur les données observées dans le modèle afin de résoudre différents problèmes inverses. Ils obtiennent d'excellents résultats pour ces différents problèmes inverses. [START_REF] Kawar | Denoising diffusion restoration models[END_REF] réussissent même à résoudre des problèmes inverses avec des images très différentes de l'ensemble d'entraînement. Cependant, comme pour les méthodes basées sur le score-matching, ces approches ne présentent aucune garantie de convergence. Cela signifie que nous ne savons pas à quelle distribution appartiennent les échantillons générés. En outre, ces méthodes sont souvent exigeantes en termes de calcul et difficiles à entraîner car elles comportent de nombreux hyper paramètres.

Un autre type d'approches, appelé Deep-Unrolling, consiste à entraîner un réseau de neurones, dont l'architecture est inspirée d'un schéma optimisation, afin d'approcher un opérateur qui résoud un problème inverse.

• Deep unrolling:

L'opérateur est généralement défini via un schéma itératif avec un nombre fini d'itérations N . Ce type de méthodes est associé au paradigme variationnel puisque l'opéra- teur est conçu pour minimiser une fonction objectif. Les méthodes basées sur l'unrolling optimisent les paramètres de cet algorithme itératif et apprennent une régularisation de bout en bout en minimisant cette fonctionnelle sur un ensemble de données d'apprentissage. Elles produisent d'excellents résultats en moins d'itérations que la simple application de l'algorithme d'optimisation. Les schémas d'optimisation classiques tels que Quadratic Qplitting [START_REF] Afonso | Fast image recovery using variable splitting and constrained optimization[END_REF] ou Alternating Direction Method of Multipliers (ADMM) [START_REF] Glowinski | Sur l'approximation, par éléments finis d'ordre un, et la résolution, par pénalisation-dualité d'une classe de problèmes de dirichlet non linéaires[END_REF][START_REF] Boyd | Distributed optimization and statistical learning via the alternating direction method of multipliers[END_REF] sont souvent appliqués. Figure 1.3 illustre le principe de l'unrolling.

Chaque couche du réseau de neurones correspond à une opération de l'algorithme d'optimisation. Les méthodes d'unrolling incorporent directement le modèle de dégradation dans le processus d'apprentissage. Ainsi celui-ci n'est pas agnostique. Cela permet au réseau de neurones d'extraire plus d'informations des données. Ces méthodes ont prouvé leur efficacité sur de nombreuses applications : [START_REF] Gregor | Learning fast approximations of sparse coding[END_REF][START_REF] Chen | Trainable Nonlinear Reaction Diffusion: A Flexible Framework for Fast and Effective Image Restoration[END_REF][START_REF] Diamond | Unrolled optimization with deep priors[END_REF]Adler and Öktem, 2018a;[START_REF] Gilton | Neumann networks for inverse problems in imaging[END_REF][START_REF] Zhang | Deep unfolding network for image super-resolution[END_REF][START_REF] Laroche | Deep model-based super-resolution with nonuniform blur[END_REF]. Cependant, comme les couches du réseau de neurones comprennent des informations sur le processus de dégradations, elles sont spécifiques au problème.

Dans le cadre de cette thèse, nous nous concentrons sur les méthodes Plug & Play. Ces méthodes présentent de nombreux avantages. Elles sont flexibles et facilement adaptables à tout problème inverse car elles découplent le termes associé au processus d'observation de celui-ci du régularisateur,ce qui les rend très génériques. En outre, elles ne nécessitent pas de ressources informatiques aussi importantes que pour les autres méthodes, car il est possible d'intégrer presque n'importe quel algorithme de débruitage pré-entraîné dans leurs schémas. Pour l'informatique embarquée dans les petits appareils, la possibilité de résoudre plusieurs problèmes de restauration à l'aide d'un seul réseau de débruitage est par exemple d'un grand intérêt. Par ailleurs, il est également possible de concevoir et d'entraîner son propre débruiteur, ce qui reste moins coûteux que d'entraîner un réseau neurones pour apprendre le score d'une densité.

L'objectif de cette thèse est de mieux comprendre les méthodes Plug & Play aussi bien d'un point de vue théorique que pratique. D'un point de vue théorique, il est important de savoir dans quelles conditions les modèles Bayésiens cadre Plug & Play sont bien définis et bien posés par exemple. D'importantes questions sur la convergence de ces algorithmes doivent également être abordées. Par exemple, les sous-ensembles vers lesquels ces algorithmes convergent lors de l'estimation de ponctuelle ne sont pas toujours clairement définis sous des hypothèses réalistes. Lors de l'échantillonnage à partir de la distribution a-posteriori, il n'existe tout simplement aucune garantie de convergence. [START_REF] Zhang | Plug-and-play image restoration with deep denoiser prior[END_REF] ou alors celles-ci sont dérivées sous des hypothèses irréalistes [START_REF] Sun | An online plug-and-play algorithm for regularized image reconstruction[END_REF][START_REF] Sun | Scalable plug-and-play admm with convergence guarantees[END_REF][START_REF] Ryu | Plug-and-play methods provably converge with properly trained denoisers[END_REF] ou du moins difficilement vérifiables [START_REF] Cohen | Regularization by denoising via fixed-point projection (red-pro)[END_REF]

y = Ax + n , (2.1)
where y corresponds to the observed data, x the quantity we wish to recover/to infer, A to the forward operator that models the physics behind the observation process in the absence of noise and n to the measurement noise. These inverse problems encountered in real life are often ill-posed. It means that a little perturbation in the measurements can lead to large errors in the signal we wish to reconstruct or that there exist several possible signal values that are consistent with the observed data. It is a question of prime importance especially in fields where decisions are taken based on this reconstruction like for image-guided diagnosis in medicine. Furthermore, imaging inverse problems involve high-dimensional objects, making the task more challenging. In many applications, the admissible signals belong to a small subset of the ambient space. For example, in modern imaging science we tend to believe that natural images are concentrated on a low-dimensional manifold of the ambient space [START_REF] Fefferman | Testing the manifold hypothesis[END_REF]. It has motivated the development of regularizers/priors that incorporate prior knowledge on the data we wish to retrieve.

Variational approaches are very popular to deal with inverse problems in imaging science. They consist in building an objective function we seek to minimize. This objective function is generally the sum of two terms, the data-fidelity term that measures the discrepancy between the observation y and the proposed restoration and the regularization term which promotes images with desired properties or which are located in the neighbourhood of some sub-manifold. As classical regularizers, we can cite the Total Variation (TV) semi-norm [START_REF] Rudin | Nonlinear total variation based noise removal algorithms[END_REF]) that promotes images with sparse gradients, hence favors images that are piecewise-constant, or regularizers enforcing sparsity in transformed domains as wavelet basis [START_REF] Donoho | Ideal spatial adaptation by wavelet shrinkage[END_REF]. Although a lot of progress has been made, handcrafted regularizers do not succeed in capturing the whole complexity in natural images. In addition, these regularizers are often designed to be convex in order to take advantage of convex optimization tools, which might limit their effectiveness. For instance, solutions provided by the TV regularizer within the variational framework can suffer from staircasing [START_REF] Louchet | Posterior expectation of the total variation model: Properties and experiments[END_REF].

Ill-posed inverse problems can also be tackled in a Bayesian framework. In this framework both the observed data and the signal to be reconstructed are the result of random phenomena. Both are seen as the realizations of random variables. In the Bayesian paradigm, we seek to determine the posterior distribution, ie the distribution of the model parameter x given the observed data y by applying the Bayes' rule

p(x|y) = p(x)p(y|x)/ p(z)p(y|z)dz , (2.2)
where p(y|x) is called the likelihood and is directly related to the noise distribution and p(x) is the prior which encodes the information we have about x. The posterior distribution describes all possible solutions to the inverse problem considered, given observed data. It represents a far more complete solution to the inverse problem than retrieving the quantity of interest x as in the variational paradigm, and offers a lot of possibilities. First of all, knowing the posterior distribution allows to propose different estimates of the posterior distribution as a solution to the inverse problem. In addition, it allows to perform uncertainty quantification studies on the proposed solutions. It is a huge advantage, especially in applications where decision making is based on a proposed restoration. For example, Figure 2.1 shows the limitations of the variational paradigm as two solutions to an inverse problem in medical imaging lead to two distinct diagnoses as one of the restorations presents a lesion. However, recovering the whole posterior in imaging science is rarely doable from a computational point of view, as we have to deal with highdimensional objects. It also raises the question of the choice of the prior distribution. To solutions to an inverse problem in medical imaging leading to two possible different diagnoses. Indeed, the solution on the right presents a lesion whereas the one on the left does not. Which solution should we select ? How confident are we on the lesion ? These questions can be answered in the Bayesian framework. Illustrations from [START_REF] Repetti | Scalable bayesian uncertainty quantification in imaging inverse problems via convex optimization[END_REF].

be useable, it has to contain meaningful information about the quantity of interest x and not to imply too intensive computational operations. In this framework, hand-crafted priors were restricted for a long time to simple log-concave models [START_REF] Bardsley | Mcmc-based image reconstruction with uncertainty quantification[END_REF][START_REF] Louchet | Posterior expectation of the total variation model: Properties and experiments[END_REF][START_REF] Durmus | Efficient bayesian computation by proximal markov chain monte carlo: when langevin meets moreau[END_REF]. The recent development of neural networks is a real breakthrough in imaging science. Deep Learning based methods achieve today state-of-the-art results in a lot of different fields such as computer vision, speech recognition, weather forecasting and playing games and for a large variety of tasks. These approaches consist in adapting a generic model to a specific problem through learning against training data. Thanks to the computational resources we have at our disposal, they automatically learn a structure within the data they are fed with. Their success is based on the abundance of training data and the agnosticism from a prior knowledge of how these data are generated. However, data related to an inverse problem are not always available in abundance. Because they can cost too much to produce. Besides, these purely data-driven methods are very problem specific. A small change in the observation process implies a new time-consuming and computationally expensive learning process. It limits the use of these approaches to tackle inverse problems. However, since the 2010's, a lot of effort has been made to develop methods combining data-and knowledgedriven approaches for solving inverse problems.

A first type of methods consists in learning a regularizer/prior from the data. In the following, we describe some of these approaches.

• Plug & Play (PnP) methods: [START_REF] Venkatakrishnan | Plug-and-play priors for model based reconstruction[END_REF] propose to use neural networks in order to define an implicit regularizer via a denoising algorithm while keeping an explicit likelihood density, which is usually assumed to be known and calibrated [START_REF] Arridge | Solving inverse problems using datadriven models[END_REF]. The idea comes from the fact that many classical optimization algorithms involve the proximal operator of the prior potential which acts as a denoiser. Plug & Play approaches either relate a denoising algorithm to a proximal operator or a gradient associated with the prior density. They are mostly used to perform point estimation [START_REF] Ryu | Plug-and-play methods provably converge with properly trained denoisers[END_REF][START_REF] Sun | An online plug-and-play algorithm for regularized image reconstruction[END_REF][START_REF] Sun | Scalable plug-and-play admm with convergence guarantees[END_REF][START_REF] Xu | Provable Convergence of Plug-and-Play Priors with MMSE denoisers[END_REF][START_REF] Zhang | Plug-and-play image restoration with deep denoiser prior[END_REF]Hurault et al., 2022a,b) where they often achieve state-ofthe-art results on a large variety of tasks [START_REF] Zhang | Plug-and-play image restoration with deep denoiser prior[END_REF] but are also applied for sampling as in [START_REF] Kadkhodaie | Solving linear inverse problems using the prior implicit in a denoiser[END_REF][START_REF] Guo | Agem: Solving linear inverse problems via deep priors and sampling[END_REF]. These methods are flexible and only require to train a denoising neural network which is light in comparison to other neural networks. Their theoretical foundations are an active field of research and if we begin to better understand them for point estimation, to the best of our knowledge, no convergence result has been given for sampling before the work presented in this manuscript.

• Score-matching based methods: Score-matching is originally designed for learning nonnormalized statistical models based on i.i.d. samples from an unknown data distribution [START_REF] Hyvärinen | Estimation of non-normalized statistical models by score matching[END_REF]. However, in its original form, it scales poorly with the dimension. That is why [START_REF] Bengio | Generalized denoising auto-encoders as generative models[END_REF] propose a variant of the original score matching to estimate the score of a slightly perturbed target density. To avoid illdefined gradients, [START_REF] Song | Generative modeling by estimating gradients of the data distribution[END_REF] train a neural network that approximated the score of different perturbed target densities and plug it into an annealed Langevin scheme to generate new samples from the empirical distribution associated with the training data. (Kawar et al., 2021b,a) adapt this method in order to solve classical imaging inverse problems by sampling from the posterior distribution. Although, it delivers impressive results, the neural network used to approximate the scores is often very heavy and computationally demanding. Indeed, it aims to directly approximate the score of the prior density induced by a dataset. It is a more complex task than simple denoising. Furthermore, in the current methods the score of the prior is associated with a specific dataset (such as CelebA-HQ or LSUN bedrooms). Then, the image we wish to retrieve to solve the considered inverse problem must be in the same class as images of the training set. Finally, it is still not clear which distributions are sampled exactly.

• Denoising Diffusion Probabilistic Models (DDPMs) (Ho et al., 2020): were developed in a generative context. Their goal is to produce new samples from a distribution given a dataset. A DDPM is a parameterized Markov chain trained using variational inference to produce samples matching the training data. Transitions of this chain are learned to reverse a noising diffusion process. This noising process is usually a Markov chain that gradually adds noise to the training data until the training data looks like pure noise. Then, after learning the denoising process we are able given pure noise to generate new samples. In the simplest case, the noising process consists in adding small amounts of Gaussian noise and the conditional transition kernels are Gaussian too. It allows for a particularly simple neural network parameterization.

Based on the success of these approaches in generative modelling, [START_REF] Kawar | Denoising diffusion restoration models[END_REF]; Principle of a DPPM. p θ corresponds to the denoising diffusion distribution we learn by variational inference and q to the noising process distribution. Image taken from [START_REF] Kawar | Denoising diffusion restoration models[END_REF]. [START_REF] Saharia | Image superresolution via iterative refinement[END_REF] adapt this framework in order to solve different inverse problems by adding information about the observed data in the model. They achieve great results for the different inverse problems. [START_REF] Kawar | Denoising diffusion restoration models[END_REF] even succeed to solve inverse problems with images very different from the training set. However, as for score-matching based methods, these approaches do not have any convergence guarantees. It means that we do not know to which distribution the generated samples belong. In addition, theses methods are often computationally demanding and difficult to train as they have a lot of hyper parameters.

A second kind of approaches, called Deep-Unrolling, consists in training a neural network, whose architecture is inspired by optimization scheme, to approximate an operator that solves a given inverse problem.

• Deep unrolling: The operator is generally defined via an iterative scheme with a given finite number of iterations N . It is associated with the variational framework as the operator is designed to minimize a functional. The unrolling based methods both optimize the parameters of this iterative algorithm and learn a regularization in an end-to-end manner by minimizing this functional over a training set. It produces great results in fewer iterations than simply applying the optimization algorithm. Classical optimization schemes such as half-quadratic splitting [START_REF] Afonso | Fast image recovery using variable splitting and constrained optimization[END_REF] or Alternating Direction Method of Multipliers (ADMM) [START_REF] Glowinski | Sur l'approximation, par éléments finis d'ordre un, et la résolution, par pénalisation-dualité d'une classe de problèmes de dirichlet non linéaires[END_REF][START_REF] Boyd | Distributed optimization and statistical learning via the alternating direction method of multipliers[END_REF] are considered. Figure 2.3 illustrates the principle of deep-unrolling methods.

Each layer of the neural network corresponds to an operation of the iterative optimization algorithm. Deep unfolding methods directly incorporate the degradation model into the learning process, making it not agnostic any more. It allows the neural network to extract more information from the data. These methods have proved their efficiency over many applications [START_REF] Gregor | Learning fast approximations of sparse coding[END_REF][START_REF] Chen | Trainable Nonlinear Reaction Diffusion: A Flexible Framework for Fast and Effective Image Restoration[END_REF][START_REF] Diamond | Unrolled optimization with deep priors[END_REF]Adler and Öktem, 2018a;[START_REF] Gilton | Neumann networks for inverse problems in imaging[END_REF][START_REF] Zhang | Deep unfolding network for image super-resolution[END_REF][START_REF] Laroche | Deep model-based super-resolution with nonuniform blur[END_REF]. However, because the neural network layers include information about the measurement process, they are very problem-specific.

In this thesis, we focus on Plug & Play methods. These methods have numerous advantages. They are flexible and easily adaptable to any inverse problem as they decouple the In this case, the neural network aims at solving a super-resolution inverse problem via a linearised ADMM scheme. There are three constitutive modules: the prior module P that is learnt during the training, the data module D which incorporates information about the observation process and the update module U . Image taken from [START_REF] Laroche | Deep model-based super-resolution with nonuniform blur[END_REF].

observation related terms from the plugged regularizer. It makes them really universal. In addition, they do not require massive computational resources as we can plug almost any pre-trained denoising algorithm within their schemes. For embedded computing in small devices, being able to solve several restoration problems by using a single denoising network is of huge interest for instance. Besides, it is also possible to design and train its own denoising neural network, which is still less expensive than training a score matching neural network.

The goal of this thesis is to better understand Plug & Play methods both from a theoretical and a practical point of view and to pave the way for further studies. From a theoretical point of view, it is important to know under which conditions the Bayesian models derived within a Plug & Play framework are well defined and well posed for instance. There are also important questions about the convergence of these algorithms that need to be addressed. For instance, the subsets towards which these algorithms converge when performing point estimation is not always clearly defined under realistic assumptions. When sampling from the posterior distribution, there are simply no convergence guarantees. This thesis is divided in 5 chapters.

In Chapter 3 we present the main concepts appearing in this thesis. We firstly introduce inverse problems and their ill-posed nature that makes them difficult to solve. After reviewing historical methods to tackle them, we focus on Plug & Play approaches for point estimation and sampling.

Chapter 4 introduces PnP-SGD, a gradient descent based scheme that aims at estimating the MAP with a Plug & Play prior. After proving the existence, the stability of the MAP, we show that this problem is well-posed under realistic assumptions. PnP-SGD converges towards a point in the vicinity of the set of the stationary points of the posterior distribution.

Chapter 5 presents two sampling algorithms, PnP-ULA and PPnP-ULA, with a Plug & Play prior. Theoretical questions for the MMSE estimation problem are addressed under realistic assumptions. Then, covergence results and non-asymptotic error bounds are pre-sented. Finally, the efficiency of the methods is proved on a classical inverse problems and a first uncertainty quantification study is delivered.

Chapter 6 investigates the influence of deep priors in the posterior distribution. First, we study the solutions proposed by the posterior distributions induced by different denoisers from a quantitative point of view. Then, we seek to interpret these results in terms of potential. It allows us to better understand the solutions promoted by our denoiser. Finally, we investigate the accuracy of the sampled distribution from a frequentist point of view, in order to determine if the distribution we sampled from is realistic and properly models the reality.

Chapter 7 concludes this thesis and proposes perspectives for further studies. In this section, we detail the contributions of this thesis.

On Maximum-a-Posteriori estimation with Plug & Play priors and stochastic gradient descent

We address the problem of solving an inverse problem by computing the MAP estimator with a Plug & Play prior using a stochastic gradient descent (SGD) scheme. A state-ofthe-art Convolutionnal neural network is used in place of the prior [START_REF] Ryu | Plug-and-play methods provably converge with properly trained denoisers[END_REF]. The algorithm developed is called Plug & Play SGD (PnP-SGD). We emphasize the fact that it is a highly non-convex optimization problem. Where previous approaches show impressive results, they either lack convergence proofs [START_REF] Zhang | Plug-and-play image restoration with deep denoiser prior[END_REF] or they are derived under unrealistic [START_REF] Sun | An online plug-and-play algorithm for regularized image reconstruction[END_REF][START_REF] Sun | Scalable plug-and-play admm with convergence guarantees[END_REF][START_REF] Ryu | Plug-and-play methods provably converge with properly trained denoisers[END_REF] or at least not easily checkable [START_REF] Cohen | Regularization by denoising via fixed-point projection (red-pro)[END_REF] assumptions. In this chapter, we firstly address key theoretical questions related the existence of such an estimator, its stability and its well-posedness. We show that it is a (weakly) well-posed problem. In addition, convergence results are presented under realistic assumptions on the denoiser used. Asymptotically, PnP-SGD converges towards points in the vicinity of the set of stationary point of the posterior distribution. We finally report a range of imaging experiments demonstrating PnP-SGD as well as comparisons with other PnP schemes. We show that PnP-SGD provides good results in comparison with state-ofthe-art PnP methods that are clearly interpretable. This work will be published on Journal of Mathematical Imaging and Vision (JMIV) under a special issue.

Bayesian imaging using Plug & Play priors: when Langevin meets Tweedie

Plug & Play priors within Monte Carlo sampling schemes for general Bayesian computation are not very common in imaging science although they begin to be an active field of research. To the best of our knowledge, there is no proof of convergence. Algorithm convergence issues aside, as for the MAP estimation problem, there are important open questions regarding whether the underlying Bayesian models and estimators are well defined, well-posed, and have the basic regularity properties required to support efficient Bayesian computation schemes. This chapter develops theory for Bayesian analysis and computation with PnP priors. We introduce Plug & Play Unadjusted Langevin Algorithm (PnP-ULA) for Monte Carlo sampling and Minimum Mean Squared Error (MMSE) estimation. Using recent results on the quantitative convergence of Markov chains, we establish detailed convergence guarantees for this algorithm under realistic assumptions on the denoising operators used, with special attention to denoisers based on deep neural networks. We also show that these algorithms approximately target a decision-theoretically optimal Bayesian model that is well-posed. PnP-ULA is demonstrated on several canonical problems such as image deblurring and inpainting, where it is used for point estimation as well as for uncertainty visualisation and quantification.

This work was published on SIAM Journal on Imaging Science.

In-depth study of data-driven priors for sampling

If the Bayesian framework gives an access to the posterior distribution, which underpins all inference about the signal we wish to recover from the observed data, it is interesting to determine what solutions are promoted by denoisers with different properties. In addition, checking if the estimated Bayesian model is meaningful from a frequentist point of view is also a question of prime importance. Indeed, we aim at verifying if the Bayesian model is accurate and correctly models the empirical posterior distribution. From a practical point of view, we check if the computed probabilities with our Bayesian model match the empirical ones. We perform this task on three different MMSE denoisers inducing different image priors.

Publications and Preprints

Inverse problems

From a practical viewpoint solving an inverse problem boils down to determine causes from an observed phenomenon. It consists of retrieving the model parameter x ∈ X from observed data y ∈ Y where x and y are generally related by the following equation

* y = A(x) + n (3.1)
where X is the parameter space and Y is the data space. Both spaces are vector spaces with appropriate topologies. Elements of both spaces consist in possible parameter and data vectors. A : X → Y is an operator called the forward operator and maps parameters to data. Classically it is assumed to be known and continuous. n ∈ Y is the realization of a Y-valued random variable that characterizes the observation process stochasticity.

In imaging science, the model parameter x is usually an image in R d . We consider two main types of representation for images:

• Pixel-wise representation x ∈ R H×W ×C where H and W are respectively the height and the width of the picture whereas C corresponds to the number of channels. For instance, C = 1 if we are dealing with grayscale images and

C = 3 if it is with RGB images. Eventually, d = HW C. • Coordinate-based representation x = u θ (x 1 , x 2 ) ∈ R C , with (x 1 , x 2 ) ∈ Ω 2 ⊂ R 2 . u θ are the intensity values of the C channels located in (x 1 , x 2 )
and is parametrized by θ ∈ Θ with Θ the set of possible parameters for u θ .

Classically, images are encoded by the discrete pixel-wise representation. It is the representation we will use in the following. However, we point out that recent works aim at developing neural networks learning a continuous intensity map from spatial locations. They succeed in accurately modelling natural scenes and globally improve the results when dealing with high-frequency structures as proved in [START_REF] Sitzmann | Implicit neural representations with periodic activation functions[END_REF] for example.

Examples of inverse problems in imaging science

Most inverse problems in imaging aim at reconstructing an unknown image x ∈ R d from a degraded observation y ∈ C m under some assumptions on their relationship. In this case A is called the degradation operator and models deterministic instrumental aspects of the observation process, and n is an unknown (stochastic) noise term taking values in C m . In this section, we introduce some classical inverse problems we may deal with in this manuscript. These problems differ from each other because of the degradation operator A, which is often assumed to be linear.

3.1.2.a Image denoising

In denoising, we have A = Id and (3.1) becomes

y = x + n (3.2)
Optical blur

Motion blur In the simplest case, the noise distribution is known. However, we can face a correlated or a spatially varying noise distribution, what increases the difficulty.

From a practical point of view, a digital photography can suffer from noise if the acquisition time is too short or if the light intensity in the scene is too low.

3.1.2.b Image deblurring

In image deblurring, we have ∀x ∈ R d , Ax = k * x where k is a blurring kernel and * stands for the convolution operator.

There are two major types of blur, the motion blur and the optical blur. Motion blur arises when either the acquisition system or the photo's subject is moving during the acquisition time. Optical blur is caused by an incorrect focus on some elements of the picture, by light diffraction and by chromatic aberrations in the lens. Figure 3.1 illustrates these two types of blur. More realistic blur operators are encoded by spatially-varying kernel. In addition, the convolution kernel k is not necessarily known, making this problem even more to difficult to solve as we need to firstly or jointly estimate k. In this case we talk about blind deblurring.

3.1.2.c Image interpolation

The degradation operator A is a masking operator that masks certain pixels in a (random) manner. A is the identity matrix with random rows missing. Then, (3.1) becomes

y = x| mask + n (3.3)

3.1.2.d Image inpainting

The inpainting problem is very much related to the interpolation problem. The difference is that a whole pixel region is masked. This problem is more complicated than interpolation.

Ill-posedness

The inverse problems we are facing in imaging science are usually ill-posed. It means that they are sensitive to variations in the data vector y. The paternity of the term "ill-posed" is attributed to the French mathematician Jacques Hadamard who firstly defined a well-posed problem.

Definition 3.1.1 (Well-Posedness) An inverse problem is called well-posed if 1. There exists at least one solution. (Existence)

2. There is at most one solution. (Uniqueness)

The solution depends continuously on data. (Stability)

According to Hadamard, an inverse problem is said to be ill-posed, if one of these conditions does not hold. If the two first conditions appear to be normal, the third one can be easily explained from a practical point of view. If the inverse operator of A is either unbounded or discontinuous, the additive observation noise can be dramatically amplified during the inversion process. Then, two distinct but close observations ỹ and ȳ can lead to very different reconstructions x and x, what is not desirable.

Ill-posedness is often encountered. One can easily prove that every compact operator between 2 infinite-dimensional Hilbert spaces with infinite range has a discontinuous inverse and then its associated inverse problem is ill-posed. If A is linear, one can also interpret instability in terms of singular-values. The faster the decay of the singular values, the more ill-posed is the inverse problem.

Let us consider some special cases where the degradation operator A is linear and the parameter and data spaces involved are Euclidean.

1. Assume A : R d → I(A) R m with d < m and there exists a unique inverse operator A -1 : I(A) → R d . As the observation noise n does not necessarily belong to I(A), we cannot simply invert A although its inverse exists.

Assume

A : R d → R m with d > m.
In this case, we have more unknowns that equations and the system is said to be underdetermined. Consequently, there are several possible solutions to one observation y.

3. Assume A : R d → R d and there exists A -1 : R d → R d . We note λ 1 and λ d respectively the smallest and largest eigenvalue. If κ = λ d /λ 1 is large, then the matrix is nearly singular and the problem is sensitive to small perturbations in the observation y. The problem is then said to be ill-conditioned. Thus, the naive restoration x = A -1 y = x + A -1 n is dominated by A -1 n and does not constitute a good restoration.

In addition if n = 0 in Equation (3.1), the problem is ill-posed if A is not invertible. Subsequently, it is clear that the interpolation and inpainting inverse problems are ill-posed.

Regularization

When facing an ill-posed inverse problem, Hadamard suggests in [START_REF] Hadamard | Lectures on cauchy's problem in linear partial differential equations[END_REF] to model it differently instead of trying to solve it. However, major inverse problems faced in mathematical physics are usually unstable as showed in shown in [START_REF] Calderon | Uniqueness in the cauchy problem for partial differential equations[END_REF][START_REF] Calderon | On the Existence of Certain Singular Integrals[END_REF][START_REF] Zygmund | On singular integrals[END_REF] for instance.

The goal of regularization is to develop stable methods in order to estimate x from data y assuming the knowledge of the degradation operator A and to prove some properties of the estimated solution. In the fact, regularization aims at computing a mapping R θ : Y → X which is continuous in Y for a fixed regularization parameter θ and such that R θ (y) → x as y → Ax.

There exist 4 major types of regularization methods.

• Approximate analytic inversion The idea behind these methods is to stabilize A -1 by smoothing the inverse operator. It is often very problem-specific. The Filtered Back-Projection introduced by [START_REF] Natterer | The Mathematics of Computerized Tomography[END_REF][START_REF] Natterer | Mathematical methods in image reconstruction[END_REF] used in CT reconstruction is a good example of such methods.

• Iterative methods with early stopping With this kind of methods, we typically aim at minimizing the functional z → Az -y 2 2 . Applying gradient based methods, this functional usually decreases before growing up. We talk about semi-convergent behaviour. The goal of these methods is to design a stopping criterion that plays the role of a regularizer (see [START_REF] Natterer | Mathematical methods in image reconstruction[END_REF] for example).

• Discretization The idea is to look for an approximate solution of (3.1) in a specific subspace using Projection or Galerkin methods (see [START_REF] Natterer | Regularisierung schlecht gestellter probleme durch projektionsverfahren[END_REF] for example).

• Variational methods This approach boils down to solve an optimization problem

R θ (y) = arg min z∈X {L(Az, y) + S θ (z)} .
with L : Y × Y → R a function quantifying the similarity between two elements of the data space, often called the the data-fidelity term, S θ : X → R a regularization function associated with the regularization parameter θ. This term allows to promote solutions with desirable features. It stabilizes the algorithm by encoding a-priori information about x. Designing a good regularizer consists in a challenging task and is an active field of research. The classical Tikhonov regularization uses Hilbert-space norms to regularize the inverse problem and was introduced in [START_REF] Tikhonov | On the stability of inverse problems[END_REF]. The value of θ has also a huge impact on the proposed solution and it can be learned or set manually. The data-fidelity term is often chosen with respect to the observation noise distribution and generally as an affine transformation of the negative log-likelihood in order to have a statistical interpretation. Typically in digital photography or in tomography applications, the observation noise is modelled by a Poisson distribution, which leads to a Kullback-Leibler (KL) divergence for the data-fidelity term. Otherwise, the observation noise is often modelled by a Gaussian distribution.

It is a very adaptive framework with a plug-and-play structure as the degradation operator A, the data-fidelity term and the regularizer are chosen in accordance to the properties of the tackled inverse problem. It also involves terms with clearly defined roles.

Variational Approaches in imaging science and their regularizers

In this section we focus on variational approaches in imaging science, which can usually be written

R λ (y) = arg min z∈X {L(Az, y) + λS(z)} , (3.4)
where λ > 0 and balances the trade-off between the regularization and data-fidelity.

As previously explained, adding a regularization term allows to add information about the properties of the solution of the tackled inverse problem. This supplementary piece of information can favour a solution over another and can turn an ill-posed inverse problem into a well-posed one. The appeal for these methods in imaging is due to two main reasons. First, they are very adaptable as explained before. Second, with the progress in (convex) optimization, they scale very well to high dimensional problems. Finding a good regularizer capturing the richness of images consists in an active field of research.

• Total variation (TV). TV was introduced in [START_REF] Rudin | Nonlinear total variation based noise removal algorithms[END_REF] for the denoising inverse problem. It reads

∀z ∈ R d , S(z) = TV(z) = Ω d|Dz| (3.5)
where Ω ⊂ R d and corresponds to the image domain.

Discretizing (3.5), we get

∀z ∈ R d , TV(z) = (i,j)∈[0,nrows]×[0,n cols ] ∇ i,j z p (3.6) with p = 1 or 2 and ∇ i,j z = [z i+1,j -z i,j , z i,j+1 -z i,j ].
It is a regularization widely used in imaging science as it promotes images with sparse gradients and assumes that images are piecewise constant objects. However, it struggles to restore images with high-frequency structures and tends to add jumps. We talk about staircasing effect. In order to alleviate this issue, other regularization terms based on TV were designed such as the Total Generalized Variation (TGV) [START_REF] Bredies | Total generalized variation[END_REF] or the Infimal-Convolution Total Variation (ICTV) [START_REF] Chambolle | Image recovery via total variation minimization and related problems[END_REF].

• Sparsity in transformed domains. The regularizer can be written S(z) = W z 1 with W : R d → Z. This regularization favours sparsity of the image coefficients in a representation space Z. Z is typically the space spanned by a learned dictionary [START_REF] Elad | Image denoising via sparse and redundant representations over learned dictionaries[END_REF] or by a wavelet basis [START_REF] Beck | A fast iterative shrinkage-thresholding algorithm for linear inverse problems[END_REF] or a wavelet frame [START_REF] Donoho | Ideal spatial adaptation by wavelet shrinkage[END_REF].

• Expected Patch Log-Likelihood (EPLL). This approach is intrinsically related to the Bayesian approach described in Section 3.2. It consists in learning a prior p on a set of image patches and then to minimize (3.4) with S(z) =i log p(P i z) and P i is a matrix extracting the i-th path from the image in vectorized form out of all overlapping patches. The authors of [START_REF] Zoran | From learning models of natural image patches to whole image restoration[END_REF] used a Gaussian mixture model prior p whose parameters have been learned from a patch dataset with an Expectation-Maximization (EM) algortihm.

The variational approach consists in solving an optimization problem. There rarely exist closed-form solutions except in simple cases like, for example, when the data-fidelity and regularization terms are both quadratic. With more complex regularizers, we have to apply iterative schemes. The recent progress in convex optimization allows to efficiently tackle high-dimensional optimization problems even if the objective function is not smooth. As iterative schemes commonly used in imaging, we can cite the Alternating Direction Method of Multipliers (ADMM) [START_REF] Glowinski | Sur l'approximation, par éléments finis d'ordre un, et la résolution, par pénalisation-dualité d'une classe de problèmes de dirichlet non linéaires[END_REF][START_REF] Boyd | Distributed optimization and statistical learning via the alternating direction method of multipliers[END_REF], the first-order primal-dual algorithm [START_REF] Chambolle | A first-order primal-dual algorithm for convex problems with applications to imaging[END_REF]. However, the convergence guarantees only concern convex regularizers, which limits their reliability in more general contexts. The Bayesian paradigm is a complete statistical inferential methodology providing a natural framework to regularise inverse problems so as to deliver accurate and well-posed solutions. It is the framework of this thesis and it includes most of the variational methods. In this context, both data and model parameters are considered as the realization of some random variables. Accordingly, the relationship between x and y is described by a statistical model with likelihood function p(y|x) or by the potential F (x, y) = -log p(y|x). The knowledge about x before observing y is encoded by the marginal or prior distribution for x, typically specified via a density function p or by its potential U (x) = -log p(x). Unless explicitly stated otherwise, we henceforth assume that all densities are defined w.r.t. to the appropriate Lebesgue measure. The likelihood and prior define the joint distribution with density p(x, y) = p(y|x)p(x), from which we derive the posterior distribution with density

Bayesian

p(x|y) = p(y|x)p(x)/p(x, y) = p(y|x)p(x)/ R d p(y|z)p(z)dz .
(3.7)

However, the Bayesian framework proposes far more than a simple reconstruction based on an observation y, as it produces (after applying the Bayes' theorem) the posterior distribution. It describes all the possible solutions given the observation y. In addition, having access to the posterior distribution allows to estimate uncertainty. Most imaging methods seek to derive estimators reaching some kind of consensus between prior and likelihood and summarizing the posterior distribution, for instance the Minimum Mean Square Error (MMSE) or Maximum A Posteriori (MAP) estimators

xmap = arg max x∈R d p(x|y) = arg min x∈R d {F (x, y) + U (x)} , (3.8) xmmse = arg min u∈R d E[ x -u 2 |y] = E[x|y] = R d xp(x|y)dx . (3.9)
Computing the MAP is very appealing as it boils down to solve an optimization problem as in the variational case. Consequently, computing the MAP under a specific prior p(x) ∝ e -λS(x) is equivalent to solve an optimization with the regularization term λS(x).

Estimating the MMSE is on the contrary more difficult as it implies estimating highdimensional integrals. Although computing the MAP is attractive from a computational point of view, it is still not correctly understood from a theoretical point of view in the Bayesian framework. Indeed, an estimator is a Bayes estimator if it minimizes the expected cost for a given cost function C under the posterior distribution p(x|y) x)p(z|y)dz p(y) dy . (3.10)

R C (x) = E p(y) E p(z|y) [C(z, x)] = y∈Y z∈X C(z,
As explained in [START_REF] Pereyra | Revisiting Maximum-a-Posteriori estimation in log-concave models[END_REF], if the posterior is not log-concave, we do not know if the MAP minimizes (3.10) under a cost function C. On the contrary, the MMSE estimator minimizes (3.10) with C :

X × X → R, (z 1 , z 2 ) → z 1 -z 2 2 2 .
At this point, we point out the fact that if both the prior and the likelihood are Gaussian, then the MAP and the MMSE estimators are equal.

The quality of the inference about x given y depends on how accurately the specified prior represents the true marginal distribution for x. Most works in the Bayesian imaging literature consider relatively simple priors promoting sparsity in transformed domains or piece-wise regularity (e.g., involving the 1 norm or the total-variation pseudo-norm [START_REF] Rudin | Nonlinear total variation based noise removal algorithms[END_REF][START_REF] Chambolle | An algorithm for total variation minimization and applications[END_REF][START_REF] Louchet | Posterior expectation of the total variation model: Properties and experiments[END_REF][START_REF] Pereyra | Proximal Markov chain Monte Carlo algorithms[END_REF]), Markov random fields (MRF, 2011), or learning-based priors like patch-based Gaussian or Gaussian mixture models [START_REF] Zoran | From learning models of natural image patches to whole image restoration[END_REF][START_REF] Yu | Solving inverse problems with piecewise linear estimators: From gaussian mixture models to structured sparsity[END_REF][START_REF] Aguerrebere | A Bayesian Hyperprior Approach for Joint Image Denoising and Interpolation, With an Application to HDR Imaging[END_REF]Teodoro et al., 2018b;[START_REF] Houdard | High-dimensional mixture models for unsupervised image denoising (hdmi)[END_REF]. Special attention is given in the literature to models that have specific factorisation structures or that are log-concave, as this enables the use of Bayesian computation algorithms that scale efficiently to high-dimensions and which have detailed convergence guarantees, [START_REF] Pereyra | Proximal Markov chain Monte Carlo algorithms[END_REF][START_REF] Durmus | Efficient bayesian computation by proximal markov chain monte carlo: when langevin meets moreau[END_REF][START_REF] Repetti | Scalable bayesian uncertainty quantification in imaging inverse problems via convex optimization[END_REF][START_REF] Girolami | Riemann manifold langevin and hamiltonian monte carlo methods[END_REF][START_REF] Chen | Stochastic gradient Hamiltonian Monte Carlo[END_REF].

Methods related to the Bayesian approach are described in Section 3.5. With the development of computational resources, a lot of efforts have been made to develop artificial neural networks and their applications. They are mapping inspired by the working of human brains. They consist in interconnected nodes called neurons. As in the human brain, artificial neurons are trained to indirectly react to some stimuli and fire when they perceive one. Their expressivity makes them a powerful tool to solve inverse problems [START_REF] Cybenko | Approximation by superpositions of a sigmoidal function[END_REF][START_REF] Hornik | Approximation capabilities of multilayer feedforward networks[END_REF]. In this section, we expose some basics components about neural networks. We are only interested in feedforward neural networks.

Learning Approach with

Neural networks

Neural networks are oriented graphs where each node is called a neuron. Neurons are grouped into layers. The first layer is called the input layer and the last one the output layer. Every layers between theses layers are called hidden layers.

When using a neural network, we have to determine an architecture, ie basic components composing our neural network such as the activation functions, the connection type between layers, the number of layers, etc, ... There exist different types of architectures. In this section, we present different types of architectures used in imaging science when performing point estimation. FEEDFORWARD NEURAL NETWROK Let us consider a neural network with H layers and H -2 hidden layers. We denote {n 0 , n 1 , ..., n H } ∈ N H+1 each layer size and f h (x) the result obtained when computing the result of the h-th layer for the input x ∈ R n0 . The mapping f h : R n0 → R n0 is defined by (3.11) where W h ∈ R n h-1 ×n h is a matrix with entries corresponding to the weights of each edge, b h ∈ R n h is the bias added by the h-th layer and σ is an activation function, usually nonlinear, like the Rectified Linear Unit (ReLU) or the sigmoid function. (W h , b h ) h∈{1,2,..H,} are parameters to learn.

f h (x) = x if h = 0 σ(W h f h-1 (x) + b h ) otherwise,
If every neuron from a layer are connected to each neuron of the following one, we talk about fully-connected neural network.

The depth of neural network corresponds to its number of hidden layers. Works such as [START_REF] Eldan | The power of depth for feedforward neural networks[END_REF] and [START_REF] Daniely | Depth separation for neural networks[END_REF] support the idea, the deeper is a neural network, the more expressive it is.

CONVOLUTIONAL NEURAL NETWORK (CNN) However, when dealing with images, which are high-dimensional objects, fully-connected neural networks imply too many parameters to set. We might want to opt for a different architecture. In this case, we prefer to use convolutional neural networks. They are feedforward neural networks with a specific structure. The equation (3.11) also describes their working process, except that (W h ) h∈{1,...,H} results from the concatenation of convolutional operators. Then a neuron in the h-th layer results from a combination of only a few neurons of the h -1-th layer located in the same region. The weights performing these convolutions are shared between neurons that is why they involve less parameters. In addition to have a lot less parameters, CNNs are approximately shiftinvariant (due to boundary effects), which is a desirable property when dealing with images. Indeed, important features can be located anywhere in the input space and CNNs can then detect feature regardless their locations. On the contrary, fully-connected neural networks for instance are not shift-invariant. Then, to be able to detect a specific feature, they have to see this feature in different locations to be sure that the neurons of each region can spot it.

Moreover, within a layer, other operations can be added. A very common operation is max-pooling that allows to reduce the dimension of the input signal by summarizing a zone by its largest element. Furthermore, we emphasize the use of U-Nets that were firstly developed for image segmentation in the biomedical context (see [START_REF] Ronneberger | U-net: Convolutional networks for biomedical image segmentation[END_REF]). These CNNs have successive layers where pooling operations are replaced by upsampling operators allowing the network to propagate context information to higher resolution layers.

RESIDUAL NEURAL-NETWORK (RES-NET) They were presented in [START_REF] He | Deep residual learning for image recognition[END_REF]. Skip connections are used to jump over some layers. Typical ResNet models are implemented with double-or triple-layer skips that contain nonlinearities like ReLU and batch normalization in between. The equation (3.12) describes the working process of such networks.

f h (x) = x if h = 0 σ(W h σ(W h-1 f h-2 (x) + b h-1 ) + b h + f h-2 (x)) otherwise.
(3.12)

Adding skip connections allows us to avoid the problem of vanishing gradients that considerably slows down the neural netowrk training phase. Then it eases the neural networks learning. In addition, skip connections mitigates the degradation problem sometimes encountered when training a neural network as reported by [START_REF] He | Deep residual learning for image recognition[END_REF]. Indeed, with an increasing depth, which is often associated with a better expressivity, the neural network accuracy gets saturated and finally degrades.

Learning process with neural networks

Training a neural network means, given a specific architecture, learning the best set of parameters θ = W h , b hh∈{1,...,H} to solve our inverse problem. In supervised learning, we seek to determine it over a training set (x i , y i ) {i=1,...,N } consisting in input-output examples, where x i s and y i s are respectively the target solutions and their associated observations. These pairs are considered as i.i.d samples from the joint distribution with density p(x, y). The goal is to find θ * such that

θ * = arg min θ∈Θ N i=1 L(x i , f θ (x i ) , (3.13)
where Θ is the set of admissible parameters, L is the cost function. This cost function is often quadratic.

The optimal parameter θ * is often computed using stochastic optimization algorithms such as the stochastic gradient descent [START_REF] Goodfellow | Deep Learning[END_REF] or Adam [START_REF] Kingma | A method for stochastic optimization[END_REF].

If the size of the training set N is large enough, according to the law of large numbers, we have

E (X,Y ) [L(x, f θ (y))] ≈ N i=1 L(x i , f θ (y i )).

Neural networks for point estimation

In the last few years, deep neural networks have become ubiquitous to solve inverse problems in imaging, showing unmatched performance for point estimation for some specific problems like image denoising. Deep networks can be trained without explicitly using the knowledge of the forward model (3.1) [START_REF] Dong | Learning a deep convolutional network for image super-resolution[END_REF][START_REF] Zhang | Beyond a gaussian denoiser: Residual learning of deep cnn for image denoising[END_REF][START_REF] Zhang | Ffdnet: Toward a fast and flexible solution for cnn-based image denoising[END_REF][START_REF] Gharbi | Deep joint demosaicking and denoising[END_REF][START_REF] Schwartz | Deepisp: Toward learning an end-to-end image processing pipeline[END_REF][START_REF] Gao | Dynamic scene deblurring with parameter selective sharing and nested skip connections[END_REF] or on the contrary can use this model explicitly via unrolled optimization techniques [START_REF] Gregor | Learning fast approximations of sparse coding[END_REF][START_REF] Chen | Trainable Nonlinear Reaction Diffusion: A Flexible Framework for Fast and Effective Image Restoration[END_REF][START_REF] Diamond | Unrolled optimization with deep priors[END_REF][START_REF] Gilton | Neumann networks for inverse problems in imaging[END_REF]. Then, the goal is to learn a deep neural network approximating an operator R : Y → X that is implicitly defined via an iterative scheme. This scheme often aims at finding the minimum of a functional.

Also, imaging approaches based on neural networks struggle to support more advanced inference by comparison to a Bayesian treatment by Monte Carlo sampling, which can support a wide breadth of statistical analyses beyond point estimation, particularly Bayesian decision-theoretic approaches to deal with advanced forms of uncertainty quantification (e.g., hypothesis tests, p-values, model misspecification tests), as well as approaches to deal with automatic calibration of partially unknown models and objective model comparison [START_REF] Robert | The Bayesian Choice: From Decision-Theoretic Foundations to Computational Implementation[END_REF].

Neural networks for sampling

MONTE-CARLO DROPOUT In order to go further than simple point estimation, [START_REF] Gal | Dropout as a bayesian approximation: Representing model uncertainty in deep learning[END_REF] develop a new theoretical framework casting dropout training in deep neural networks where dropout consists in randomly omitting each hidden unit with probability P . Then, a neural network with dropout can be interpreted as a variational Bayesian approximation and allows to perform uncertainty quantification.

GENERATIVE MODELING WITH NEURAL NETWORKS

To go further than simple point estimation, generative neural networks were developed. These neural networks are trained in order to sample from a target distribution π ∈ P(R d ) given an empirical distribution π = (1/N ) N i=1 δ xi where the x i are samples of the target distribution π. These methods consist in learning a mapping g that transforms samples of a easy-to-sample distribution π 0 ∈ P(R p ), like a Gaussian distribution, into samples from the target distribution π. There exist 3 major types of neural networks to sample from a distribution namely Variational Auto-Encoders (VAEs) [START_REF] Kingma | An Introduction to Variational Autoencoders[END_REF], Normalizing Flows (NFs) and Generative Adversarial Networks (GANs). In this section, we briefly introduce these neural networks.

VAEs have a similar structure to autoencoders which consist of two networks.

• The Encoder G : R d → R p maps the samples from π 0 to a latent variable in R p .

Usually, p < d.

• The Decoder D : R p → R d maps back latent variables in R p to images in R d .

The latent space corresponds to a space where it is easier to manipulate the data. It supports the idea that images are concentrated on a low-dimensional sub-manifold.

The idea behind VAE is to model the latent space in a probabilistic manner. Then the encoder and the decoder are both associated with distributions as shown in Figure 3.4. In this case we sample a latent variable z from π 0 in the latent space and use the decoder to map into an image.

We want to encourage the latent space to follow the distribution π 0 which is usually Gaussian.

GANs consist of two neural networks trained in a competitive manner [START_REF] Goodfellow | Generative Adversaria l Networks[END_REF].

• The Generator G : R p → R d maps the samples from π 0 to images in R d .

• The Discriminator D : R d → [0, 1] tries to distinguish samples from π for which D(x i ) = 1 from artificial images generated from π 0 and for which D(G(z)) = 0.

The loss reads min

G max G {E x log D(x) + E z log[1 -G(D(z))]} .
NFs are such that p = d. The goal is to train an invertible neural network T that maps images to latent representation z = T (x) classically normally distributed [START_REF] Papamakarios | Normalizing Flows for Probabilistic Modeling and Inference[END_REF]. Then, we have

π(x) = π 0 (T (x)) × | det J T (x)| .
(3.14)

We have an explicit expression of the target density π. In order to be exploitable, | det J T | has to be easily computable. That is why T is often composed of simple transformations with triangular Jacobians.

In addition, stochastic normalizing flows (SNFs) were introduced in [START_REF] Sun | Scalable plug-and-play admm with convergence guarantees[END_REF] and consist of a sequence of deterministic flow transformations and stochastic sampling methods with tractable paths, such as Markov Chain Monte Carlo (MCMC) or overdamped Langevin dynamics.

There are several survey papers for VAEs [START_REF] Kingma | An Introduction to Variational Autoencoders[END_REF], GANs [START_REF] Creswell | Generative adversarial networks: An overview[END_REF][START_REF] Wang | Generative adversarial networks: introduction and outlook[END_REF] and NFs [START_REF] Papamakarios | Normalizing Flows for Probabilistic Modeling and Inference[END_REF][START_REF] Kobyzev | Normalizing flows: An introduction and review of current methods[END_REF] providing a comprehensive review of the literature for distribution learning.

Sampling the posterior distribution Based on the success of the generative neural networks previously introduced, a lot of effort has been made to sample from the posterior distribution using a neural network in order to solve inverse problems. The basic idea is to add an input to the networks for the observation y. [START_REF] Mirza | Conditional generative adversarial nets[END_REF] introduced conditional GANs (cGANs), constructed by simply feeding the observation y, we wish to condition on to both the generator and discriminator. cGANS were applied to solve inverse problems such as image reconstruction in ultra low dose 3D helical CT (Adler and Öktem, 2018b) or Partial-Differential-Equation (PDE) based inverse problems [START_REF] Ray | The efficacy and generalizability of conditional gans for posterior inference in physics-based inverse problems[END_REF]. [START_REF] Goh | Solving bayesian inverse problems via variational autoencoders[END_REF] proposes to train a VAE to solve a PDE-based inverse problem and to perform uncertainty quantification. Eventually, [START_REF] Lugmayr | Srflow: Learning the superresolution space with normalizing flow[END_REF] is an example where a NF is trained to solve an ill-posed inverse problem in imaging, super-resolution. The network learns the conditional distribution of the solution to the inverse problem given the low-resolution input y. SNFs were firstly applied to tackle inverse problems in [START_REF] Hagemann | Stochastic normalizing flows for inverse problems: a markov chains viewpoint[END_REF].

Limitations of the pure neural network based approaches

One disadvantage of using neural networks to solve imaging inverse problems is that in order to achieve state-of-the-art performance it is usually necessary to train the network for a specific problem configuration. Then, the network must be retrained if the forward model, ie A or the noise distribution change, or any model parameters change significantly. Solutions encoded by end-to-end neural networks are mostly problem specific and not easily adapted to reflect changes in the problem (e.g., in instrumental settings). There also exist concerns regarding the stability of such approaches for general reconstruction problems [START_REF] Antun | On instabilities of deep learning in image reconstruction and the potential costs of ai[END_REF][START_REF] V. Antun | Can stable and accurate neural networks be computed[END_REF].

Furthermore sampling neural networks generally lack convergence proofs. Then, it is not clear which distribution we are sampling from.

In order to address these limitations, we consider Plug & Play methods for posterior sampling and Maximum-a-Posteriori estimation. Related work on PnP MAP estimation is discussed in Section 3.4 and our contributions to this subject are detailed in Chapter 4. Related work on PnP posterior sampling is discussed in Section 3.6 and our contributions to this subject are detailed in Chapters 5 and 6. Plug & Play methods try to combine the strengths of the Bayesian paradigm and the neural networks. It consists to specify an implicit prior defined via a denoiser. These data-driven regularisation approaches learn an implicit representation of the prior density p(x) (or its potential U (x) = -log p(x)) while keeping an explicit likelihood density, which is usually assumed to be known and calibrated [START_REF] Arridge | Solving inverse problems using datadriven models[END_REF] In the context of imaging inverse problems, Plug & Play methods aim at using a carefully chosen denoiser D ε : R d → R d to implicitly define an image prior. This is achieved by relating D ε to a proximal operator or a gradient associated with the prior density. In the first case, D ε replaces a MAP estimator for a denoising problem. In the second case, D ε replaces a Minimum Mean Square Error (MMSE) estimator for a denoising problem, related to the gradient of a log-prior via Tweedie's identity † [START_REF] Robbins | An empirical bayes approach to statistics[END_REF][START_REF] Miyasawa | An empirical bayes estimator of the mean of a normal population[END_REF][START_REF] Efron | Tweedie's formula and selection bias[END_REF]. † Notice that although it is conceptually helpful to distinguish these two cases (in order to make a historical and practical survey of the subject), there are clear theoretical connections between the two approaches. Indeed, under regularity conditions on the Bayesian model involved, MAP denoisers can be expressed as MMSE denoisers under an alternative (albeit often unknown) Bayesian model [START_REF]Should penalized least squares regression be interpreted as maximum a posteriori estimation[END_REF]. However this equivalence can not always be exploited in practice and has been mostly ignored in the literature on Plug & Play methods until very recently with the work of Xu et al. [START_REF] Xu | Provable Convergence of Plug-and-Play Priors with MMSE denoisers[END_REF] to be presented later.

A survey of

In what follows, we describe how these approaches have been widely used to compute the MAP estimator as a solution to the considered inverse problem. In our discussion, we pay particular attention to questions related to algorithmic convergence, and to the interpretation of the computed solutions, as this has been an important focus of the literature.

Plug & Play MAP estimators using proximal splitting

Let D † ε denote the MAP estimator to recover x from a noisy observation x ε ∼ N (x, ε Id) under the assumption that x has marginal density p(x) ∝ exp[-U (x)]; that is,

D † ε (x ε ) = arg min x∈R d { 1 2 x ε -x 2 + εU (x)} = prox εU (x ε ) .
When we set the PnP denoiser D ε such that D ε = D † ε , any optimization scheme making use of a proximal descent on the prior can be used to solve (3.8) via D ε .

For instance, the alternating direction method of multipliers (ADMM) [START_REF] Glowinski | Sur l'approximation, par éléments finis d'ordre un, et la résolution, par pénalisation-dualité d'une classe de problèmes de dirichlet non linéaires[END_REF][START_REF] Boyd | Distributed optimization and statistical learning via the alternating direction method of multipliers[END_REF] writes the augmented Lagrangian of (3.8) as

E ε (x, z, v) = F (x, y) + x -z 2 /(2ε) + v (x -z) + U (z) .
The joint optimization of the augmented Lagrangian is given by

(x map , ẑmap ) = arg min x,z∈R d max v∈R d E ε (x, z, v).
This provides the solution xmap = ẑmap of (3.8) when ε → 0.

In practice, the joint optimization is solved by an alternate minimization scheme on x and z and a gradient ascent on u = εv,

x k+1 = arg min x E ε (x, z k , u k /ε) = prox εF (•,y) (z k -u k ) , (3.15) z k+1 = arg min z E ε (x k+1 , z, u k /ε) = prox εU (x k+1 + u k ) = D ε (x k+1 + u k ) , (3.16) u k+1 = u k + x k+1 -z k+1 .
(3.17)

Similarly, when F (., y) is differentiable, the simpler Forward-backward splitting (FBS) scheme [START_REF] Combettes | Proximal splitting methods in signal processing[END_REF], which only requires to compute ∇F , can be written in a Plug-and-Play fashion as

x k+1 = prox εU (x k -ε∇F (x k , y)) = D ε (x k -ε∇F (x k , y)) . (3.18)
A fully proximal version of this algorithm, called Backward-backward splitting (BBS) (Combettes and Pesquet, 2011), writes

x k+1 = prox εU (prox εF (x k )) = D ε (prox εF (x k )) . (3.19)
BBS aims at solving a slightly modified version of (3.8) where F is replaced by its Moreau envelope with parameter ε. The same algorithm can be derived using half-quadratic splitting to solve (3.8). When U is convex, such splitting schemes and many variants (including primal-dual methods, ISTA or FISTA, etc.) are well understood and proved to converge to the global optimum [START_REF] Boyd | Distributed optimization and statistical learning via the alternating direction method of multipliers[END_REF]. They have also been successfully used for non-convex U like patch-based Gaussian mixture models (GMM) as pioneered for external learning by [START_REF] Zoran | From learning models of natural image patches to whole image restoration[END_REF]. The use of splitting schemes with non-convex GMM priors was later refined with convergence guarantees for scene-adapted learning (Teodoro et al., 2018a).

Following the seminal work of [START_REF] Venkatakrishnan | Plug-and-play priors for model based reconstruction[END_REF], this kind of splitting schemes have become ubiquitous in cases where U (and hence D † ε ) are unknown and unspecified, but a denoiser D ε is available and assumed to be a good approximation of D † ε = prox εU . As popular and efficient these methods have become, their convergence properties have remained largely unknown. Indeed, for most denoisers D ε , there is no guarantee that there exists a potential U such that D ε = prox εU . [START_REF] Sreehari | Plug-and-Play Priors for Bright Field Electron Tomography and Sparse Interpolation[END_REF] establish some sufficient conditions for this to happen: D ε must be differentiable, and its Jacobian J Dε should be symmetric with eigenvalues within the [0, 1] interval to ensure non expansiveness. These assumptions hold for transform-domain thresholding denoisers and for variants of Non Local Means (Buades et al., 2005a) where symmetry is explicitly enforced [START_REF] Sreehari | Plug-and-Play Priors for Bright Field Electron Tomography and Sparse Interpolation[END_REF]. More recently, it has also been shown [START_REF] Nair | Fixed-point and objective convergence of plug-and-play algorithms[END_REF]) that a special class of linear denoisers (including Non Local Means (Buades et al., 2005a)) are proximal operators of some closed, proper functions. This approach necessitates to work with a non standard inner product though. The previous proofs do not hold for most popular denoisers, including BM3D [START_REF] Dabov | Image denoising with block-matching and 3d filtering[END_REF], Non Local Bayes [START_REF] Lebrun | A nonlocal bayesian image denoising algorithm[END_REF] and neural networks denoisers like DnCNN [START_REF] Zhang | Beyond a gaussian denoiser: Residual learning of deep cnn for image denoising[END_REF], as observed in [START_REF] Reehorst | Regularization by denoising: Clarifications and new interpretations[END_REF].

CONSENSUS EQUILIBIRUM / FIXED POINT INTERPRETATION. Since it remains difficult to show that PnP schemes converge to the MAP or even a critical point of (3.8), several authors have proposed to analyse these schemes from a consensus equilibrium point of view [START_REF] Buzzard | Plug-and-play unplugged: Optimization-free reconstruction using consensus equilibrium[END_REF][START_REF] Ahmad | Plug-and-play methods for magnetic resonance imaging: Using denoisers for image recovery[END_REF], or similarly to consider and analyse these approaches as fixed-point algorithms [START_REF] Sun | An online plug-and-play algorithm for regularized image reconstruction[END_REF][START_REF] Ryu | Plug-and-play methods provably converge with properly trained denoisers[END_REF]. The fixed points attained by these algorithms cannot be interpreted as MAP estimators, but should be seen as solving a set of equilibrium equations involving both the denoiser and the data term. For instance, for PnP-FBS, the idea is to show convergence to the set of points x satisfying x = D ε (x-ε∇F (x, y)). It can easily be shown that the fixed points of several of these PnP algorithms (in particular PnP-ADMM and PnP-FBS) coincide [START_REF] Meinhardt | Learning proximal operators: Using denoising networks for regularizing inverse imaging problems[END_REF][START_REF] Sun | An online plug-and-play algorithm for regularized image reconstruction[END_REF].

Assuming that such fixed points exist, [START_REF] Sun | An online plug-and-play algorithm for regularized image reconstruction[END_REF] show convergence of PnP-ISTA (which is equivalent to PnP-FBS above) under the assumptions that ∇F is L y -Lipschtitz, εL y 1 and D ε is θ-averaged, see (Bauschke et al., 2011, Definition 4.33) for a definition. This assumption on the denoiser is probably too strong, since most denoisers cannot be considered as averaged operators. [START_REF] Sun | Scalable plug-and-play admm with convergence guarantees[END_REF] reformulate PnP-ADMM with different convergence conditions, and still assume quite restrictive conditions on the denoiser D ε ‡ . [START_REF] Ryu | Plug-and-play methods provably converge with properly trained denoisers[END_REF] propose a convergence analysis of PnP-ADMM, PnP-FBS and PnP-DRS (PnP Douglas-Rachford Splitting), based on the weaker assumption that the residual operator D ε -Id is L-Lipschitz with a Lipschitz constant which depends both on the data fitting term F and the denoiser D ε . The proof also requires F to be µ-strongly convex (which excludes all cases where A is not full rank and de facto excludes some of the applications considered in [START_REF] Ryu | Plug-and-play methods provably converge with properly trained denoisers[END_REF]) and it imposes quite restrictive assumptions on relative values of µ, ε and L.

In a similar direction, [START_REF] Xu | Provable Convergence of Plug-and-Play Priors with MMSE denoisers[END_REF] very recently proposed a convergence study for PnP-ISTA, with the assumption that ∇F is L y -Lipschitz with εL y 1. However, they assume that

D ε is an exact MMSE denoiser, i.e. D ε (x ε ) = E[X|X ε = x ε ]
, where X ∼ p and X ε -X ∼ N (0, ε Id). Therefore their theoretical results do not carry to many classical denoisers, such that those learned from training data and implemented by neural networks.

ASSUMPTIONS ON ALGORITHM PARAMETERS. Most of the convergence proofs for PnP algorithms impose restrictive assumptions on the choice of parameters used in the iterative schemes. This may exclude interesting ranges of parameters for several inverse problems. For instance, for PnP-FBS, the parameter ε (which can be interpreted as the step of the proximal or gradient descents) and the Lipschitz parameter L y of ∇F must typically be chosen such that L y ε C with C ∈ [1, 2] (see [START_REF] Ryu | Plug-and-play methods provably converge with properly trained denoisers[END_REF][START_REF] Xu | Provable Convergence of Plug-and-Play Priors with MMSE denoisers[END_REF], the exact value of C depends on the convergence proof).

If F (x, y) = 1 2ασ 2 Ax -y 2 , with A ≤ 1, it implies that 1 α σ 2
ε . The parameter ε is imposed by the denoiser D ε (the denoiser is trained for a noise of variance ε), and σ is given by the quantity of noise in the forward model. If, for instance, the forward model involves a noise standard deviation σ which is 5 times smaller than the one used for the denoiser D ε , it means that the penalty α (which balances the respective weights of the data and prior terms) should be chosen larger than 25, which implies that the algorithm will only converges for huge regularizations. We will see in Section 4.3 that for this kind of reason the PnP-FBS algorithm often fails to converge for classical imaging inverse problems, or converges only for values of α which are not interesting in practice. Fully proximal algorithms such as PnP-ADMM or PnP-BBS are much more robust in practice, even when the conditions of their theoretical convergence are not fully met. The PnP-SGD algorithm that we will introduce in the following does not suffer from the same convergence limitations.

AMP ALGORITHMS. It is worth mentioning at this point that the Plug-and-Play framework has also been shown to be very efficient with Approximate Message Passing algorithms [START_REF] Ahmad | Plug-and-play methods for magnetic resonance imaging: Using denoisers for image recovery[END_REF]. These algorithms have excellent convergence properties for data terms of the form Ax -y 2 with A belonging to specific classes of random matrices. This restriction on A does not hold for the inverse problems considered in this thesis so we focus instead on classical optimization scheme such as the ones described above.

Plug & Play MAP estimators using gradient descent

Now, assume that D ε = D ε , where D ε is the MMSE estimator to recover x from the noisy observation x ε with (X ε |X = x) ∼ N (x, ε Id) when X has marginal density p; that is,

D ε (x ε ) = E[X|X ε = x ε ] = R d zp(z)G ε (x ε -z)dz/ R d p(z)G ε (x ε -z)dz , (3.20)
where G ε is a Gaussian kernel with variance ε, meaning that for all x ∈ R d ,

G ε (x) = (2πε) -d/2 exp[-x 2 /(2ε)] .
We introduce the following class of smooth approximations of p(x), defined for any

x ∈ R d by p ε (x) = R d p(x)G ε (x -x)dx . (3.21)
In this case, Tweedie's identity [START_REF] Efron | Tweedie's formula and selection bias[END_REF] establishes the following relationship between the MMSE denoiser D ε and (3.21), for any

x ∈ R d ∇U ε (x) = -∇ log p ε (x) = (x -D ε (x))/ε , (3.22)
where U ε = -log(p ε ). This relation can be used to plug the MMSE denoiser D ε in any gradient descent scheme involving ∇U ε as follows

X k+1 = X k -δ∇F (X k , y) -δ∇U ε (X k ) + δZ k+1 , (3.23) or X k+1 = X k + δ∇ log p(y|X k ) + δ ε (D ε (X k ) -X k ) + δZ k+1 , (3.24)
where {Z k : k ∈ N} is a sequence of i.i.d Gaussian random variables with zero mean and identity covariance matrix and δ > 0 is the gradient descent step-size.

It is at the core of the algorithm PnP-SGD presented in Chapter 4.

Similarly to the MAP denoiser D † ε , the MMSE denoiser D ε is usually not known, so PnP methods rely on other denoisers D ε that are believed to be good approximations of D ε . Observe that CNN denoisers are usually trained to minimize an empirical quadratic risk on a large database of natural images. As a consequence, they naturally produce good approximations of MMSE denoisers D ε for realistic image priors. This makes approaches based on Tweedie's identity particularly attractive. On the other hand, learning mechanisms to produce good approximations of MAP denoisers D † ε are much less widespread, although under some conditions, MMSE denoisers can be shown to be MAP denoisers on a different prior (see [START_REF]Should penalized least squares regression be interpreted as maximum a posteriori estimation[END_REF][START_REF] Xu | Provable Convergence of Plug-and-Play Priors with MMSE denoisers[END_REF]Hurault et al., 2022b)).

A similar relation is derived by [START_REF] Romano | The little engine that could: Regularization by denoising (red)[END_REF] where they present the Regularization by Denosing (RED) method, which proposes an insightful Bayesian formulation of denoiser-based priors as image-adaptive Laplacian regularisations. Instead of using Tweedie's identity, the RED method solves (3.8) via different optimization algorithms (including gradient descent and ADMM) with explicit regularization

U ε (x) = (1/2) x, x -D ε (x) .
As shown in [START_REF] Reehorst | Regularization by denoising: Clarifications and new interpretations[END_REF], under the assumptions that D ε is locally homogeneous and has symmetric Jacobian, this implies that for any

x ∈ R d , ∇U ε (x) = x -D ε (x)
, which is (up to a scaling factor 1/ε) the same expression as Tweedie's identity in (3.22). Unfortunately, as pointed out before, these assumptions on D ε are not strictly satisfied by most commonly used denoisers [START_REF] Reehorst | Regularization by denoising: Clarifications and new interpretations[END_REF], although we note that Jacobian symmetry can be explicitely enforced [START_REF] Milanfar | Symmetrizing Smoothing Filters[END_REF]. The convergence of the RED algorithms for denoisers that do not verify the above-mentioned assumption remains unproven. As an alternative interpretation the RED algorithm can be seen as a way to approximate the score ∇U ε by (x -D ε (x))/ε in the optimality equation ∇F + ∇U ε = 0. Here the optimal MMSE denoiser D ε is again replaced by some other denoiser.

More recently, [START_REF] Cohen | Regularization by denoising via fixed-point projection (red-pro)[END_REF]) studies a projected RED estimator which seeks to minimize a data fidelity term subject to the constraint that the solution belongs to the set of fixed points {x ∈ R d : x = D ε (x)}, thus sharing strong link with the consensus equilibrium interpretation of proximal-based PnP estimators. It is reported in [START_REF] Cohen | Regularization by denoising via fixed-point projection (red-pro)[END_REF] that when D ε is a demi-contractive mapping, its fixed points define a convex set, which allows the construction of provably convergent algorithms for this alternative RED estimator. However, as pointed out in [START_REF] Pesquet | Learning maximally monotone operators for image recovery[END_REF], verifying that a given denoising operator is demicontractive is not easy and, to be the best of our knowledge, it is not yet clear what denoisers verify this property. Furthermore, from a Bayesian inference viewpoint, additional studies would be required in order to determine when this projected RED estimator defines or approximates a MAP estimator for a suitable Bayesian model.

In a similar direction, two very recent works (Hurault et al., 2022a,b) show how to train efficiently a denoiser that explicitly satisfies D ε (x) = x -∇g ε (x) for some functional g ε . Plugging this denoiser in appropriate PnP schemes, they are able to prove convergence to stationary points of an explicit cost function.

The PnP-SGD optimisation algorithm that will be presented in Chapter 4 is very close to the gradient descent version of RED presented in [START_REF] Romano | The little engine that could: Regularization by denoising (red)[END_REF]. We will show that it converges to the vicinity of the solution of (3.8) under much milder conditions than previously assumed. In particular, the convergence guarantees hold even when D ε is not an exact MAP or MMSE denoiser, which is often the case in practice. Importantly, our convergence guarantees hold for the neural network denoiser used in [START_REF] Ryu | Plug-and-play methods provably converge with properly trained denoisers[END_REF]) (a variant of DnCNN [START_REF] Zhang | Beyond a gaussian denoiser: Residual learning of deep cnn for image denoising[END_REF] with a contractive residual) and also for the native Non Local Means (Buades et al., 2005b). In this section we review some of the methods commonly used in imaging for sampling. There is a vast literature on Bayesian computation methodology for models related to imaging sciences (see, e.g., [START_REF] Pereyra | A survey of stochastic simulation and optimization methods in signal processing[END_REF]). Here, we briefly summarise efficient high-dimensional Bayesian computation strategies derived from the Langevin stochastic differential equation (SDE)

Posterior

dX t = ∇ log p(X t |y) + √ 2dB t = ∇ log p(y|X t ) + ∇ log p(X t ) + √ 2dB t , (3.25)
where (B t ) t 0 is a d-dimensional Brownian motion. When p(x|y) is proper and smooth, with x → ∇ log p(x|y) Lipschitz continuous § , then, for any initial condition X 0 ∈ R d , the SDE (3.25) has a unique strong solution (X t ) t 0 that admits the posterior of interest p(x|y) as unique stationary density [START_REF] Roberts | Exponential convergence of langevin distributions and their discrete approximations[END_REF]. In addition, for any initial condition X 0 ∈ R d the distribution of X t converges towards the posterior distribution in total variation. Although solving (3.25) in continuous time is generally not possible, we can use discrete time approximations of (3.25) to generate samples that are approximately distributed according to p(x|y). A natural choice is the Unadjusted Langevin algorithm (ULA) Markov chain (X k ) k 0 obtained from an Euler-Maruyama discretisation of (3.25), given by X 0 ∈ R d and the following recursion for all k ∈ N

X k+1 = X k + δ∇ log p(y|X k ) + δ∇ log p(X k ) + √ 2δZ k+1 , (3.26)
where {Z k : k ∈ N} is a family of i.i.d Gaussian random variables with zero mean and identity covariance matrix and δ > 0 is a step-size which controls a trade-off between asymptotic accuracy and convergence speed [START_REF] Dalalyan | Theoretical guarantees for approximate sampling from smooth and logconcave densities[END_REF][START_REF] Durmus | Nonasymptotic convergence analysis for the unadjusted Langevin algorithm[END_REF]. The approximation error involved in discretizing (3.25) can be asymptotically removed at the expense of additional computation by combining (3.26) with a Metropolis-Hastings correction step, leading to the so-called Metropolis-adjusted Langevin Algorithm (MALA) [START_REF] Roberts | Exponential convergence of langevin distributions and their discrete approximations[END_REF].

It is interesting to draw comparison between ULA and SGD. As for the SGD update rule, the red term in the ULA update rule points towards the target distribution modes. However, the square root on on δ in the green term allows exploration and consists in the main difference with SGD.

When the prior density p(x) is log-concave but not smooth, one can still use ULA by approximating the gradient of U (x) = -log p(x) in (3.26) by the gradient of the smooth Moreau-Yosida envelope U λ (x), given for any x ∈ R d and λ > 0 by ∇U λ (x) = 1 λ (xprox λ U (x)). ¶ For example, one could use the Moreau-Yosida ULA [START_REF] Durmus | Efficient bayesian computation by proximal markov chain monte carlo: when langevin meets moreau[END_REF], § That is, there exists L 0 such that for any

x 1 , x 2 ∈ R d , ∇ log p(x 1 |y) -∇ log p(x 2 |y) L x 1 -x 2 ¶ Recall: The Moreau-Yosida envelope is defined as U λ (x) = inf x U (x) + 1 2λ x -x 2 and the proximal operator is defined as prox λ U (x) = arg min x∈R d U (x) + 1 2λ x -x 2 2 .
given by X 0 ∈ R d and the following recursion for all k ∈ N

X k+1 = X k + δ∇ log p(y|X k ) + δ λ prox λ U (X k ) -X k + √ 2δZ k+1 . (3.27)
Notice that prox λ U is equivalent to MAP denoising under the prior p(x), for additive white Gaussian noise with noise variance λ. The Plug & Play ULA methods studied in Chapter 5 are closely related to (3.27), with a state-of-the-art Gaussian denoiser "plugged" in lieu of prox λ U . However, instead of approximating ∇U via a Moreau-Yosida envelope as above, we use Tweedie's identity (3.22) relating ∇U to an MMSE denoiser (see Section 5.2). (2017); Bigdeli and Zwicker (2017) based on the Tweedie's formula using a denoising algorithm D ε within a Monte Carlo sampling scheme. To the best of our knowledge, the idea of leveraging a denoising algorithm to approximate the score ∇U within an iterative Monte Carlo scheme was first proposed in the seminal paper [START_REF] Alain | What Regularized Auto-Encoders Learn from the Data-Generating Distribution[END_REF] in the context of generative modelling with denoising auto-encoders, where the authors present a Monte Carlo scheme that can be viewed as an approximate Plug & Play MALA. This scheme was recently combined with an expectation maximisation approach and applied to Bayesian inference for inverse problems in imaging in [START_REF] Guo | Agem: Solving linear inverse problems via deep priors and sampling[END_REF] where the goal was to tackle blind imaging inverse problems. Similarly, the recent work [START_REF] Kadkhodaie | Solving linear inverse problems using the prior implicit in a denoiser[END_REF] proposes to solve imaging inverse problems by using a Plug & Play stochastic gradient strategy that has close connections to an unadjusted version of the MALA scheme of [START_REF] Alain | What Regularized Auto-Encoders Learn from the Data-Generating Distribution[END_REF]. However, (Kadkhodaie and Simoncelli, 2020) only deals with noiseless inverse problems. (Kawar et al., 2021b,a) also suggest to plug an MMSE denoiser into an annealed Langevin scheme instead of a score-matching neural network as in [START_REF] Song | Generative modeling by estimating gradients of the data distribution[END_REF] in order to sample from the posterior distribution. However, they only show results obtained with NCSNv2 network proposed by [START_REF] Song | Generative modeling by estimating gradients of the data distribution[END_REF]. While these approaches have shown some remarkable empirical performance, they rely on hybrid algorithms that are not always well understood and that in some cases fail to converge. Indeed, their convergence properties remain an important open question, especially when D ε is implemented as a neural network that is not a gradient mapping. Consequently, the generated samples do not necessarily represent the posterior of interest p(x|y). In contrast, PnP-ULA and PPnP-ULA that are presented in Chapter 5 allow to sample from the posterior distribution of an inverse problem with a deep Plug & Play prior also providing convergence guarantees and non-asymptotic error bounds. As explained in Section 3.4, several recent works have proposed and studied the use of PnP methods in order to tackle inverse problems. Most of the literature focuses on MAP estimation. If these methods can deliver accurate results, particularly when combined with state-of-the art denoisers, their theoretical analysis stayed for a long time poorly understood. Moreover, they often relied on unrealistic assumptions on the properties of the image denoiser. This chapter is mostly inspired by the preprint article [START_REF] Laumont | On maximum-a-posteriori estimation with plug & play priors and stochastic gradient descent[END_REF] and aims at developing efficient PnP algorithms with convergence guarantees under reasonable assumptions in order to perform MAP estimation for Bayesian models with PnP priors. First, we address key questions related to the existence, the stability and the wellposedness of the inverse problem in Sections 4.2 and 4.2.1. Then, a convergence proof for MAP computation by PnP stochastic Gradient Descent (PnP-SGD) under realistic assumptions is presented in Section 4.2.2. Finally, the efficiency of the algorithm is demonstrated over a range of classical inverse problems such as denoising, deblurring and interpolation in Section 4.3.

A survey of

Proofs ans convergence results were derived by Valentin De Bortoli and are exposed in Appendix A for the sake of completeness.

Analysis of maximum-a-posteriori estimation with PnP priors

We are interested in MAP estimation for Bayesian models involving PnP priors that are defined implicitly by an image denoising algorithm D ε . We pay special attention to the highly practically relevant case in which D ε approximates the optimal MMSE denoiser D ε associated to p, i.e.,

D ε = E[X|X ε = x ε ] for (X ε |X = x) ∼ N (x, ε
i } N i=1 with X i |X i = x i ∼ N (x i , ε Id)
, and choosing D ε to approximately minimize the empirical MSE loss

N i=1 D ε (x i )-x i 2 .
Similarly, many stateof-the-art patch-based image denoisers are also designed to approximate D ε .

The fact that D ε is only an approximation of D ε leads to several complications in the analysis and computation of MAP solutions. For example, unlike D ε , D ε does not define a gradient mapping in general, and key results such as Tweedie's identity [START_REF] Efron | Tweedie's formula and selection bias[END_REF] do not hold. Moreover, in the case of neural network denoisers trained from samples {x i } N i=1 from p, the model is unknown as it is only available through {x i } N i=1 , making it difficult to check that basic regularity properties required for MAP estimation are satisfied.

Rather than imposing strong assumptions on D ε , we address these difficulties by formulating our analysis in the M-complete Bayesian framework, in which we assume that the posterior p(x|y) associated with the true prior p(x) exists but remains largely unknown, and all inference on x|y are conducted by using operational approximations of this true model [START_REF] Bernardo | Bayesian Theory[END_REF]. In particular, we focus on the class of smooth approximations of p(x|y) given for any ε > 0 and x ∈ R d by

p ε (x|y) = p ε (x)p(y|x) R d p ε (x)p(y|x)dx , (4.1)
where p ε (x) is the smooth approximation of the prior p(x) defined in (3.21). We will study MAP estimation for p ε (x|y) to establish that the procedure is well defined, well posed, amenable to efficient computation, and that it provides a useful approximation to MAP estimation with the true posterior p(x|y). Following on from this, Section 4.2.2 will study the computation of MAP solutions for p ε (x|y) by using PnP SGD with a generic denoiser D ε that approximates D ε , where we will pay particular attention to the conditions on D ε required to ensure convergence, as well as to the bias introduced by using D ε instead of D ε .

It is established in Chapter 5 that, under basic assumptions on the likelihood function p(y|x) detailed in H1 below, the posterior approximation p ε (x|y) is well defined, proper, and can be made as close to the true posterior p(x|y) as desired by reducing the value of ε, with the approximation error vanishing as ε → 0. Crucially, Chapter 5 also establishes that, under H1 and mild assumptions on the optimal MMSE denoiser D ε (essentially, that the denoising problem underlying D ε is well posed in the sense of Hadamard), then x → p ε (x|y) is differentiable with x → ∇ log p ε (x|y) Lipschitz continuous. We conclude that the approximation p ε (x|y) is well defined and amenable to computation by first-order schemes, such as SGD to compute critical points of p ε (x|y) and perform MAP estimation.

H1 For any y ∈ R m , sup x∈R d p(y|x) < +∞, p(y|•) ∈ C 1 (R d , (0, +∞)). In addition, there exists L y > 0 such that ∇ log p(y|•) is L y Lipschitz continuous.
With the above-mentioned properties of p ε (x|y) in mind, we wonder if computing a MAP solution for p ε (x|y) provides useful information about a MAP solution for p(x|y). More precisely, we study if critical points for p ε (x|y) are stable w.r.t. variations in ε, and if they converge to critical points of p(x|y) as ε → 0. Proposition 4.2.1 below establishes that this is indeed the case. In words, MAP solutions computed with p ε (x|y) are in the neighbourhood of MAP solutions for p(x|y), with ε controlling a trade-off between the computational efficiency of first-order schemes and the accuracy of the delivered solutions w.r.t. p(x|y). When ε is large, the approximation of the posterior is smoother so gradient descent can be used with larger steps to improve convergence speed (as the gradients have a smaller Lipschitz constant).

Formally, we investigate the stability of the set of stationary points S ε,K = {x ∈ K : ∇ log p ε (x|y) = 0} w.r.t. ε > 0, where K is a compact set. We show that for any sequence (ε n , x n ) n∈N such that lim n→+∞ ε n = 0 and for any n ∈ N, x n ∈ S εn,K every cluster point of (x n ) n∈N belongs to S K = {x ∈ K : ∇ log p(x|y) = 0}. In other words, we show that the stationary points of the approximate posterior are close to the ones of the true posterior. Proposition 4.2.1 Assume H1 and that p ∈ C 1 (R d , (0, +∞)) with p ∞ + ∇p ∞ < +∞. Then for any compact set K ⊂ R d and (x εn ) n∈N such that lim n→+∞ ε n = 0 and for any n ∈ N, x εn ∈ S εn,K , we have that any cluster point x of x εn satisfies x ∈ S K .

Proof: The proof is postponed to Appendix A.1. Note that the above result can be strengthened to show the convergence at the levels of sets. More precisely, we can show that any cluster point of {S K,ε } ε>0 is a subset of S K in the sense of the Haussdorff distance, see [START_REF] Munkres | Topology[END_REF] for a definition.

As a third and final point in our analysis, we study if MAP estimation for p ε (x|y) is a wellposed estimation procedure, which is an essential requirement for meaningful inference. One would ideally seek to establish the existence of a unique global maximiser that is Lipschitz continuous w.r.t. perturbations of the observed data y. Unfortunately, this is not possible without imposing very strong assumptions on the model. Instead, Proposition 4.2.2 below shows that, under some assumptions on the likelihood p(y|x), the set of critical points of p ε (x|y) is locally Lipschitz continuous w.r.t. perturbations of y, which is a weaker form of well-posedness. Notice that the assumptions on the likelihood can be relaxed when D ε is contractive, but this is usually unrealistic. This highlights a limitation of MAP estimation by comparison to other Bayesian estimators, namely MMSE estimation, which is shown in Chapter 5 to be well-posed under significantly weaker assumptions.

Proposition 4.2.2 Assume H1 and that

(x, y) → p(y|x) ∈ C 2 (R d × R m , R). Let ε > 0, we have that (x, y) → p ε (x|y) ∈ C 2 (R d × R m , R + ). Let y 0 ∈ R m denote
some observed data and x y0 ∈ R d a local maximiser of the posterior x → p ε (x|y 0 ) with ∇ 2 log p ε (x y0 |y 0 ) negative. Then there exists an open set V 0 ⊂ R m and a function x (y) ∈ C 1 (V 0 , R d ) such that y 0 ∈ V 0 and for any y ∈ V 0 , x (y) is a strict local maximizer of x → p ε (x|y).

Proof: The proof is postponed to Appendix A.2.

To conclude, a major challenge in understanding Bayesian inference with PnP priors and providing guarantees for the delivered solutions is that the underlying prior and posterior densities p(x) and p(x|y) are unknown. Also, the image denoiser D ε used to construct PnP schemes is not usually directly related to the model. Instead, when it approximates the optimal MMSE denoiser D ε , it is indirectly related to the model via Tweedie's identity and the smooth approximations p ε (x) and p ε (x|y). We establish that these operational approximations are useful for MAP inference for x|y, in the sense that they are well defined, proper, and MAP solutions for p ε (x|y) can be made arbitrarily close to the true MAP solutions through the choice of ε. Importantly, under some assumptions, MAP solutions for p ε (x|y) are well posed and amenable to efficient computation by first order optimisation methodology.

PnP-SGD and convergence

We are now ready to study the computation of MAP solutions for p ε (x|y) by using PnP SGD with a generic denoiser D ε that approximates D ε . We pay particular attention to the conditions on D ε required to ensure convergence, and to the bias introduced by using D ε instead of D ε .

We begin by using Tweedie's identity to express SGD to compute critical points of p ε (x|y) as the following sequence: X 0 ∈ R d and for any k ∈ N

X k+1 = X k -δ k ∇F (X k , y) -δ k /ε(X k -D ε (X k )) + δ k Z k+1 , (4.2)
where (δ k ) k∈N ∈ (R + ) N is a sequence of step-sizes, ε > 0, and {Z k : k ∈ N} a family of i.i.d. Gaussian random variables with zero mean and identity covariance matrix. We recall that the sequences (X k ) k∈N and (Z k ) k∈N are defined on an underlying probability space (Ω, F, P).

As mentioned previously, in most practically relevant cases D ε is an abstract quantity that cannot be computed. Instead, we have a different denoiser D ε that can be assumed to be a good approximation of D ε . For example, when we have access to samples {x i } N i=1 from p we can consider a noisy version of these samples {x i } N i=1 with level ε > 0 and train a neural network based denoiser D ε to minimize the loss

N i=1 D ε (x i ) -x i 2 . This loss corresponds to the empirical version of E[ D ε (x ε ) -x 2 ] (with x ∼ p and X ε ∼ N (x, ε Id) condtionally to x) whose minimizer is the MMSE D ε .
Using a generic denoiser D ε in our SGD scheme in lieu of D ε we obtain the Plug & Play SGD algorithm associated with following recursion: X 0 ∈ R d and for any k ∈ N

X k+1 = X k + δ k (b ε (X k ) + Z k+1 ) , (4.3) b ε (x) = ∇ log(p(y|x)) + α(D ε (x) -x)/ε , (4.4)
where we note that we have introduced a regularization parameter α > 0 that controls the amount of regularisation enforced by D ε . The original SGD algorithm is recovered by setting α = 1 and

D ε = D ε . Algorithm 1 PnP-SGD Require: n, n burnin ∈ N, y ∈ R m , ε, α, δ > 0 Initialization: Set X 0 = x and k = 0. for k = 0 : N do Z k+1 ∼ N (0, Id) if k n burnin then X k+1 = X k + δ 0 ∇ log(p(y|X k )) + (δ 0 α/ε)(D ε (X k ) -X k ) + δ 0 Z k+1 end if if k > n burnin then X k+1 = X k + δ k ∇ log(p(y|X k )) + (δ k α/ε)(D ε (X k ) -X k ) + δ k Z k+1 δ k+1 = δ 0 (k + 1 -n burnin ) -0.8 end if end for return X N
We now turn to the proof of convergence of PnP-SGD. The asymptotic estimates we derive in this chapter are only valid for sequences which remain in a compact set K, which is a classical assumption in stochastic approximation [START_REF] Tadić | Asymptotic bias of stochastic gradient search[END_REF][START_REF] Delyon | Convergence of a stochastic approximation version of the em algorithm[END_REF][START_REF] Delyon | General results on the convergence of stochastic algorithms[END_REF][START_REF] Metivier | Applications of a kushner and clark lemma to general classes of stochastic algorithms[END_REF]. Under tighter conditions on x → log p ε (x|y) this limitation can be circumvented using the global asymptotic results of (Tadić et al., 2017, Theorem A1.1). Another way to remove this restriction would be to consider an additive term of the form x → (x-Π C (x))/λ in b ε (where Π C is the projection onto some compact convex set C and λ > 0 some hyperparameter) which ensures the stability of the numerical scheme. We leave this analysis for future work. In practice, we have not observed any stability issues for PnP-SGD provided that the stepsize is chosen appropriately see Section 4.3.3. In what follows, we show that the bias of PnP-SGD depends on the distance between D ε and the MMSE estimator D ε , using recent results from [START_REF] Tadić | Asymptotic bias of stochastic gradient search[END_REF].

H2 Assume that there exist ε 0 > 0, L 0 and a function M : R + → R + such that for any ε ∈ (0, ε 0 ], R 0, x 1 , x 2 ∈ R d and x ∈ B(0, R) we have

D ε (x 1 ) -D ε (x 2 ) L x 1 -x 2 , D ε (x) -D ε (x) M(R) , (4.5)
where we recall that

D ε (x 1 ) = R d x g ε (x|x 1 )dx , (4.6
)

with x → g ε (x|x) the probability density of X given X ε = x where X ε ∼ N (X, ε Id) condi- tionally to X and X ∼ p.
The first part of (4.5) regarding the smoothness property of the denoiser can be explicitly verified for a certain class of neural networks by adding a spectral regularization term for each layer of the neural network, see [START_REF] Ryu | Plug-and-play methods provably converge with properly trained denoisers[END_REF][START_REF] Miyato | Spectral normalization for generative adversarial networks[END_REF]. The second condition follows from carefully selecting the loss of the neural network as in the Noise2Noise network introduced in [START_REF] Lehtinen | Noise2noise: Learning image restoration without clean data[END_REF] and controlling the population error. We refer the reader to Chapter 5 for more details regarding the role of the bounding function M(R).

In particular, for neural network denoisers, Proposition 3.1 in Chapter 5 explains how to promote low values of M(R) during training by using a particular loss function. In addition, Chapter 5 makes connections with universal approximation results (see e.g., (Bach, 2017, Section 4.7)).

We are now ready to state Proposition 4.2.3 which ensures that stable PnP-SGD sequences are close to the set of stationary points of x → log p ε (x|y) where x → log p ε (x|y) is given in (4.1). The distance to this set of stationary points is controlled by the approximation error of the D ε .

H3 For any y ∈ R m , x → -log p(y|x) is real-analytic * † ‡ .
In the following, d stands for the distance induced by the Euclidean norm.

∈ (0, ε 0 ]. Assume that lim k→+∞ δ k = 0, k∈N δ k = +∞ and k∈N δ 2 k < +∞. Let R > 0, K ⊂ B(0, R) be a compact set, X 0 ∈ R d and A ε,K ∈ F given by A ε,K = {ω ∈ Ω : there exists k 0 ∈ N such that for any k k 0 , X k (ω) ∈ K.} , (4.7)
where (X k ) k∈N is given by (4.3). Then there exist C ε,K 0 and r ε,K ∈ (0, 1)

such that lim sup k→+∞ d(X k (ω), S ε,K ) C ε,K M(R) r ε,K for any ω ∈ A ε,K , with S ε,K = {x ∈ K : ∇ log p ε (x|y) = 0} , (4.8)
where x → p ε (x|y) is given in (4.1).

Proof: The proof is postponed to Appendix A.3.

The proof can be extended to the case where Z k = 0 using (Tadić et al., 2017, Theorem 2.1). In this case the assumption that k∈N δ 2 k < +∞ can be replaced by lim k→+∞ δ k = 0. The following experimental section demonstrates the PnP-SGD algorithm on three canonical imaging inverse problems, namely image denoising, deblurring and interpolation, along with other standard PnP algorithms. In this section, we study the behaviour of several PnP algorithms for three classical inverse problems: denoising, deblurring and interpolation. We recall that in each of these problems we consider a prior model p(x) ∝ exp[-U (x)] which is unknown and that the inference x|y is obtained by approximation of this model. For the deblurring and denoising problems, the log-posterior of the degradation model can be written for any x, y ∈ R d as

-log p(x|y) = Ax -y 2 /(2σ 2 ) + αU (x) + C , (4.9) * A function f : R d → R is said to be real-analytic if for any x 0 = (x 1 0 , . . . , x d 0 ) ∈ R d there exists (an 1 ,...,n d ) n 1 ,...,n d ∈N ∈ R N d and r > 0 such that for any x = (x 1 , . . . , x d ) ∈ B(x 0 , r) f (x) = n 1 ∈N • • • n d ∈N an 1 ,...,n d d j=1 (x j -x j 0 ) n j . †
The assumption that x → log(p(y|x)) is real-analytic is satisfied in all of our experiments since there exists A ∈ R p×d and σ > 0 such that for any x ∈ R d and y ∈ R m , log p(y|x) = Ax -y 2 /(2σ 2 ).

‡ From Liouville's theorem one could think that the simultaneously verifying that ∇ log p(y|•) is Lipschitz continuous and that x → log(p(y|x)) is real-analytic restricts our analysis to models for which ∇ 2 log p(y|•) is constant (i.e., Gaussian models), but this is not the case because Liouville's theorem applies entire functions, which are a subclass of the real-analytic class.

where A is a d × d matrix, C 0 is a constant and the parameter α 0 balances the weights of the log-likelihood F (x, y) and the log-prior U . In this case, we have for any x, y ∈ R d , F (x, y) = Ax -y 2 /(2σ 2 ). In our interpolation experiments, we change the likelihood so that pixels are either visible or hidden. In this case the log-posterior can be written for any x ∈ R d and y ∈ R m as

-log p(x|y) = ι Qx=y + αU (x) + C , with ι C (x) = 0 if x ∈ C +∞ otherwise, ( 4

Image dataset

In Figures 4.1 and 4.2 we present the 6 original images used in the experiments. These images contain both geometric structures, constant areas and textured regions. On the same figures, we display degraded versions of each image for each set of experiments. For the denoising experiment, the level of the Gaussian noise is fixed to σ 2 = (30/255) 2 . In the case of deblurring, the operator A correponds to a 9 × 9 uniform blur operator, and we add Gaussian noise with variance σ 2 = (1/255) 2 . Finally, in the context of interpolation, we hide 80% of the pixels.

Algorithms

In this section, we evaluate PnP-SGD (Algorithm 1) along with three other classical PnP algorithms: PnP-ADMM (Algorithm 2), PnP-FBS (Algorithm 3) and PnP-BBS (Algorithm 4). Note that in the case of interpolation, the log-likelihood is not differentiable, since ι C is not differentiable. In Section 4.3.6 we will present an extension of these PnP algorithms to this setting using proximal operators.

In order to take into account the parameter α > 0 into Algorithms 2 to 4, we slightly modify the target function. Instead of minimizing x → -log p(x|y) we aim at minimizing x → -log p(x|y)/α. Doing so, the parameter α > 0 can be included in the parameters of the log-likelihood which becomes (x, y) → F (x, y)/α. All algorithms are implemented using Python and the PyTorch library. Our experiments are run on an Intel Xeon CPU E5-2609 server with an Nvidia Titan XP graphic card.

Algorithm 2 PnP-ADMM

Require: n ∈ N, y ∈ R m , ε > 0, α > 0, x 0 ∈ R d Initialization: Set x 0 = z 0 , and u k = 0. for k = 0 : N do x k+1 = prox (ε/α)F (•,y) (z k -u k ) z k+1 = D ε (x k+1 + u k ) u k+1 = u k + (x k+1 -z k+1 ) end for return x N +1

Parameters settings and convergence conditions

In this section, we recall and discuss the choice of the different parameters, as well as the convergence conditions for PnP-SGD. We also discuss the convergence properties of PnP- For denoising, we add a Gaussian noise with variance σ 2 = (30/255) 2 . For deblurring, the operator A correponds to a 9 × 9 uniform blur operator, and we add Gaussian noise with variance σ 2 = (1/255) 2 . For interpolation, we hide 80% of the pixels.

Algorithm 3 PnP-FBS ADMM and PnP-FBS following the guidelines of [START_REF] Ryu | Plug-and-play methods provably converge with properly trained denoisers[END_REF][START_REF] Xu | Provable Convergence of Plug-and-Play Priors with MMSE denoisers[END_REF].

Require: n ∈ N, y ∈ R m , ε > 0, α > 0, x 0 ∈ R d for k = 0 : N do x k+1 = D ε (x k -(ε/α)∇F (x k , y)) end for return x N +1 Algorithm 4 PnP-BBS Require: n ∈ N, y ∈ R m , ε > 0, α > 0, x 0 ∈ R d for k = 0 : N do x k+1 = D ε (prox (ε/α)F (•,y) (x k )) end for return x N +1
Recall that from (4.1), we denote L y the Lipschitz constant of the log-likelihood gradient x → ∇F (., y). For F (x, y) = Ax -y 2 /(2σ 2 ), L y = A A /σ 2 , with A the adjoint of A. F is µ-strongly convex if and only if A is invertible, in which case µ = λ min (A) 2 /σ 2 , where λ min (A) is the smallest singular value of A. In our experiments we have λ min = 1 for denoising and λ min = 0 for deblurring and interpolation. In our experiments, the operator A is always chosen such that A A = 1. Note that if F is replaced by F /α, as it the case in Algorithms 2 to 4, we have that L y and µ are replaced by L y /α and µ/α.

DENOISER.

In all experiments, the denoising operator D ε is chosen as the pretrained denoising neural network introduced in [START_REF] Ryu | Plug-and-play methods provably converge with properly trained denoisers[END_REF]. This denoiser is trained so that Id -D ε is L-Lipschitz with L < 1. Note that this corresponds to the first part of (4.5) in H2. In [START_REF] Ryu | Plug-and-play methods provably converge with properly trained denoisers[END_REF] three pretrained denoisers, at noise level ε = (5/255) 2 , (15/255) 2 , (40/255) 2 are proposed. In this work, we only use the first one in our denoising and deblurring experiments. The interpolation problem requires a more subtle strategy relying on a coarse to fine approach, described in Section 4.3.6.

PNP-SGD.

In Algorithm 1, we consider a burn-in regime with a constant step δ 0 until some iteration n burnin . After this initial phase, we set (δ k ) k∈N to be a decreasing sequence satisfying the conditions of Proposition 4.2.3. In the case of denoising or deblurring, δ 0 is given by

δ 0 = δ stable /6, where δ stable := 2/L tot , L tot = αL/ε + A * A /σ 2 , (4.11)
where L tot is the Lipschitz constant of ∇ log p(.|y). Note that setting δ 0 = δ stable ensures that the deterministic scheme: x 0 ∈ R d and for any k ∈ N, x k+1 = x k + δ 0 ∇ log p(x k |y), satisfies that (log p(x k |y)) k∈N is non-decreasing. After the burn-in period, we use a decreasing sequence of step-sizes (δ k ) k∈N such that for any k ∈ N we have (4.12) which satisfies the conditions required in Proposition 4.2.3 for convergence. Note that contrary to existing work, any value of α > 0 can be used in Algorithm 1 provided that δ 0 is defined accordingly using (4.11).

δ k := δ 0 × (k -n burnin ) -0.8 ,
PNP-ADMM. The convergence results of [START_REF] Ryu | Plug-and-play methods provably converge with properly trained denoisers[END_REF] for PnP-ADMM require the strong convexity of F . In our experiments, this condition is met for denoising experiments (since A = Id), but not for interpolation nor deblurring if the blur operator is not invertible (which is the case for a 9 × 9 uniform blur). In the denoising case, following [START_REF] Ryu | Plug-and-play methods provably converge with properly trained denoisers[END_REF], PnP-ADMM converges to a fixed point if L ∈ [0, 1) and L/(1

+ L(1 -2L)) < ε/(ασ 2 ).
In practice, L and ε being set, this condition can only be satisfied for small values of the regularisation parameter α, which often lead to poor-quality results. However, Algorithm 2 experimentally converges to a fixed point with interesting visual properties for larger values of α. This suggests that it might be possible to prove the convergence of PnP-ADMM under weaker conditions than the ones of [START_REF] Ryu | Plug-and-play methods provably converge with properly trained denoisers[END_REF].

PNP-FBS.

Similarly to PnP-ADMM the convergence results obtained by [START_REF] Ryu | Plug-and-play methods provably converge with properly trained denoisers[END_REF] for PnP-FBS are only valid in a strongly convex setting. In our case this corresponds to the denoising experiment here. The condition on the Lipschitz constant of the denoiser D ε is L/(1 + L) < ε/(ασ 2 ) < (L + 2)/(L + 1). In Section 4.3.4, we show that these conditions are not met in our experiments. In practice, we still observe convergence of the algorithm for the denoising experiments. This is no longer case in non-strongly convex problems, see Section 4.3.5 and Section 4.3.6. In [START_REF] Xu | Provable Convergence of Plug-and-Play Priors with MMSE denoisers[END_REF], convergence towards the set of stationary points of the log-posterior is established for PnP-FBS provided that D ε = D ε , i.e. D ε is the optimal MMSE. In addition, [START_REF] Xu | Provable Convergence of Plug-and-Play Priors with MMSE denoisers[END_REF] require that εL y 1. This condition implies that ε A A ασ 2 . Since A A = 1 for all our experiments, this implies α ε/σ 2 . In experiments with large noise level (as it is the case for our denoising setting), this leads to acceptable values of α. However, when σ is small in comparison to ε (which is the case for deblurring), the regularisation parameter α for which the convergence is ensured is too highlighted in Section 4.3.3.

Denoising

For these denoising experiments, we add a Gaussian noise of variance σ 2 = (30/255) 2 (see the second row of Figures 4.1 and 4.2 for examples of degraded images). In this experiment we use a denoiser D ε trained for a noise level ε = (5/255) 2 on a dataset {x i , x i } N i=1 with x i ∼ p and x i ∼ N (x i , ε Id) for any i ∈ {1, . . . , N }. Using this denoiser in Algorithms 1 to 4, we aim at denoising y with noise level σ 2 .

We run all algorithms for several values of the regularization parameter α and for two different initializations: first a TV-L 2 initialization, i.e. applying a simple TV-L 2 restoration to the noisy image following [START_REF] Rudin | Nonlinear total variation based noise removal algorithms[END_REF][START_REF] Chambolle | A first-order primal-dual algorithm for convex problems with applications to imaging[END_REF], and second an oracle initialization (using the original image without degradation). Although the noisy observation y is a natural initialization, we observed that initializing at y usually leads to unsatisfactory results for all PnP schemes with a small value of ε. We believe that this arises from the high non-convexity of the problem. Our goal here is to assess the dependency of the algorithm on initialization, since the log-posterior we study is highly non-convex.

For PnP-SGD, the initial step-size δ 0 and the sequence (δ k ) k∈N are defined as explained in Section 4.3.3. For these denoising experiments, the resulting value of δ 0 is already quite small, such that decreasing δ k after the burn-in phase effectively stops the search for a better optimum and does not change the result. The number of iterations n burnin for the burn-in was set between 5000 and 25000 for SGD. Within that range, we stop this phase as soon as |PSNR(X k+1 ) -PSNR(X k )| < 0.1 × δ 0 . This conservative choice allows to make sure that the algorithm reaches its steady state, so that the oracle initialization (starting from an overestimated value of PSNR) does not overestimate the global maximum and the non-oracle initializations (starting from an underestimated value of PSNR) do not under-estimate it. In practice, convergence is reached after a few hundreds of iterations in most cases and only rarely did the algorithm iterate beyond 5000. Increasing δ 0 to δ 0 = 0.9 × δ stable also permits to achieve faster convergence, but in this case adding a decreasing phase for (δ k ) k∈N after the burn-in regime is important to achieve the same asymptotic results.

For the splitting-based algorithms (ADMM, BBS, FBS), practical convergence is very fast and 100 iterations are largely sufficient in all cases. Observe that since we use a denoiser trained for a noise level ε = (5/255) 2 , and our denoising experiments are run for σ 2 = (30/255) 2 , theoretical convergence of PnP-ADMM following [START_REF] Ryu | Plug-and-play methods provably converge with properly trained denoisers[END_REF] requires that α < (1+L(1-2L))/36L. The exact value of L for the denoising considered in [START_REF] Ryu | Plug-and-play methods provably converge with properly trained denoisers[END_REF] is not available, but our experiments suggest that L ≈ 1. This implies that only drastically small values of α meet the previous condition. As a result, this condition is not satisfied with the choices of α that are experimentally optimal but does not prevent the algorithm to converge in practice. In the same way, provided that L ∈ [0, 1), convergence of PnP-FBS following [START_REF] Ryu | Plug-and-play methods provably converge with properly trained denoisers[END_REF] implies that α is at least larger than 18, see Section 4.3.3. Yet, interesting values of α for this denoising experiment are far smaller. The condition provided in [START_REF] Xu | Provable Convergence of Plug-and-Play Priors with MMSE denoisers[END_REF], α ε/σ 2 = 1/3 gives more realistic values for α but we remind that in this case we must assume that D ε = D ε .

Figure 4.3 summarizes the results of this denoising experiment on 10 independent random noise realizations on each of the 6 images in the dataset, for PnP-SGD, PnP-ADMM and PnP-BBS (PnP-FBS is not shown here for the sake of clarity, but it shows a very similar behavior). We first observe that initialization seems to play a very minor role for all the algorithms considered in this problem. A TV-L 2 initialization is sufficient to reach virtually the same reconstruction quality as the oracle initialization. This might be explained by the fact that denoising is a relatively simple inverse problem. Second, all algorithms produce very similar results, with an optimal value of α around 0.25, see Figure 4.3. Table 4.1 summarizes the denoising results of all algorithms (including PnP-FBS) obtained for this nearly optimal setting of α = 0.25. In Figure 4.4 we display the results of the different algorithms for this denoising experiment. If the PSNR values are quite close, it seems that the algorithms make different compromises in terms of visual results. For example, the estimator obtained with PnP-ADMM seems to exhibit sharper edges. However, it also seems to hallucinate more false structures than other algorithms.

TV-L 2 init

TV-L 2 vs. oracle init SGD vs ADMM vs BBS SGD ADMM BBS This impression is confirmed by the evolution of the average gradient norm of the logposterior, as after 1000 iterations, a plateau around 0.4 is reached. The decay after 5000 iterations is due to the forced decay of the δ k as the algorithm has left the burn-in phase during which the discretization step-size was held constant. and TV -L 2 initialization. Left: Evolution of the average PSNR computed for K = 10 independent noise realizations for each image. A thousand of iterations seem to be sufficient to leave the burn-in phase and enter the stationary phase. The decay of the discretization step-size δ k does not alter the results, which suggests that the algorithm has converged. Right: Evolution of the average gradient norm of the log-posterior computed over the 10 experiments for each image. In less than 500 iterations, it stabilizes around 0.4 for each image. These plots suggest that the algorithm has converged. The decrease observed after 5000 iterations is explained by the decay of the discretization step-size δ k and does not alter the final result.

Deblurring

We now turn to the deblurring problem. In this section, images are blurred with a uniform 9 × 9 kernel, and a small Gaussian noise of standard deviation σ = 1/255 is added in order to define the degradation model. We now compare the behavior of Algorithms 1 to 3. Experiments with PnP-SGD follow the same rules as for the denoising problem and the same observations are valid. When running PnP-ADMM we use approximately 200 iterations to ensure the convergence whereas for PnP-FBS and PnP-BBS, we use approximately 500 iterations. Except for PnP-SGD (using Proposition 4.2.3), these PnP algorithms are not guaranteed to converge according to [START_REF] Ryu | Plug-and-play methods provably converge with properly trained denoisers[END_REF] since A is not invertible. In practice PnP-FBS indeed converges only for very large values of the regularization parameter α, whereas other PnP algorithms converge for all our experiments. As highlighted in Section 4.3.3 this suggests that convergence for PnP-ADMM and PnP-FBS occur under weaker conditions than the ones prescribed in [START_REF] Ryu | Plug-and-play methods provably converge with properly trained denoisers[END_REF].

Figure 4.6 summarizes the results of deblurring on 10 independent random noise realizations on each of the 6 images in the dataset, for PnP-SGD, PnP-ADMM and PnP-BBS (PnP-FBS is not shown here because it does not converge most of the time), for TV-L 2 and oracle initializations. Again, initialization appears to play a minor role in the final results.

Observe that all algorithms show very similar performance (when they converge) for these deblurring experiments. While PnP-SGD is slower to converge, it is ensured to approximate the MAP theoretically. Table 4.2 summarizes the deblurring results of all algorithms (including PnP-FBS) obtained for the nearly optimal setting of α = 0.3. In Figure 4.7 we display the results of the different algorithms for this deblurring experiment. Interestingly, we note that visual results for this deblurring problem are much more similar to each other than for denoising experiments.

TV-L 2 init

TV-L 2 vs. oracle init SGD vs ADMM vs BBS SGD ADMM BBS As for the denoising problem, we study the convergence of PnP-SGD studying the evolution of the average PSNR and gradient norm of the log-posterior over the iterations. Figure 4.8 shows the evolution of the average PSNR computed for each image over the 10 different experiments. It suggests that 4000 of iterations seem to be needed to have a stable PSNR for all images except Cameraman, which requires on average 1.5e4 iterations. This difference between the other images could be explained by the fact that α = 0.3 is a more sub-optimal regularization parameter for this image. This impression is confirmed by the evolution of the average gradient norm of the log-posterior, as after 6000 iterations, a plateau around 0.2 is reached. These plots also suggest that the convergence is slower for deblurring than for denoising.

Interpolation

The interpolation problem consists in trying to recover x ∈ R d from a small proportion of its pixels, namely from the measurements vector y = Qx, where Q is a m × d matrix consisting of m random lines from the d × d identity matrix, and m = qd d. In our experiments we set q = 20%. In this case, since measurements are not affected by noise, the data-fitting term takes the form of a hard constraint, i.e. for any x ∈ R d and y ∈ R m we have F (x, y) = ι Cy (x), where C y = {x : y = Qx} . iterations seem to be required to enter the stationary phase for all images except Cameraman, that needs on average 1.5e4 iterations. Right: Evolution of the average gradient norm of the log-posterior computed over the 10 experiments for each image. For all images, the gradient norm stabilizes around 0.2.

The non-differentiability of F is not problem when using ADMM and BBS since in this case the proximal operator of γF (•, y) is not only defined but admits a closed-form (which is independent of γ = ε/α). More precisely, we have for any x ∈ R d and y ∈ R m , prox γι C (x) = P Px + Q * y in terms of the (d -m) × d matrix P containing all the lines of the identity matrix which are not contained in Q. However, SGD and FBS cannot be directly applied to this problem because they require F to be differentiable. Nevertheless we can apply these algorithms to an equivalent formulation in the reduced space R d-m of unknown pixels, as shown in the following subsection.

4.3.6.a Adapting SGD to the non-differentiable inpainting problem

In what follows, we denote by x := Px ∈ R n the vector of n = d -m unknown pixels in x. Given the unknown pixels x = Px and the measurements y = Qx we can reconstruct x via the affine mapping f y : R n → R d defined for any x ∈ R d and y ∈ R m by f y (x) = P * x + Q * y.

The solution of the original problem x map = arg min x F (x, y) + U (x) can then be written as (4.13) and xmap can be found by gradient descent on Ũ = U •f y . Using the chain rule and Tweedie's formula, we have that the gradient of Ũ is given for any x ∈ R d and y ∈ R m by

x map = arg min x∈Cy U (x) = f y ( arg min x U (f y (x))) , xmap = arg min x U (f y (x)) ,
∇ Ũ (x) = P∇U (f y (x)) = (1/ε)P(Id -D ε ) • f y (x) . (4.14)
Finally, since the affine operators P and f y are 1-Lipschitz we have that L (1/ε), where L is the Lipschitz constant of ∇ Ũ .

4.3.6.b Parameter settings and results

The interpolation problem we consider is extremely ill-posed since 80% of the pixels are only constrained by the image prior. Since our implicit prior p ε (x) is most likely far from log-concave, the posterior shows a particularly large number of local optima. For this reason all methods are extremely sensitive to the initial condition. The initial conditions used in the previous experiments may misguide both ADMM and SGD to a wrong local optimum.

To deal with this more difficult case, we consider a different approach, combining:

• A coarse to fine scheme where we start by solving the MAP problem for large values of ε, and then use the result of this coarse MAP as an initialization for the next smaller value of ε. In our experiments we used ε = (40/255) 2 , (15/255) 2 , (5/255) 2 , both for ADMM and for SGD;

• For each value of ε, a burn-in phase of 2000 iterations with δ 0 = 2.5δ stable , followed by a phase of 1000 decreasing steps, as defined in (4.12).

Table 4.3 summarizes the results of different algorithmic strategies to solve our inpainting problem, on our set of 6 images with K = 4 random realizations for each image, and Figure 4.9 shows an example of results on the Simpsons image.

We can observe in Table 4.3 that the coarse-to-fine scheme is beneficial to both SGD and ADMM, allowing to reach a reconstruction quality which comes very close to the oracle initialization. This benefit is also clear on the visual results shown on Figure 4.9. In the case of a random initialization, the coarse to fine strategy is needed to avoid the apparition of spurious geometric structure in the background. In the case of the TV -L 2 initialization, it yields better continuity in the fine black lines of the image. This holds both for ADMM and SGD.

In these interpolation experiments, we also observed that using larger initial step-sizes at the beginning and using the stochastic gradient descent instead of a simple gradient descent are important to obtain good MAP estimates. This could be explained by the non-convex nature of this problem: the stochastic term and the larger step sizes are required to avoid getting trapped in spurious local optima. In this chapter we studied MAP estimation in Bayesian imaging models with PnP priors defined by image denoising algorithms. First, we sought to better understand, from a theoretical point of view, MAP estimation for solving imaging inverse problems. That is why we first addressed key questions about the definition, the stability and the wellposedness of this problem. We established that, under mild conditions, MAP solutions are well-posed. For computation, we proposed a PnP-SGD optimisation method that is provably convergent under mild and realistic assumptions on the denoiser. It paves the way for further studying more elaborate PnP schemes to estimate the MAP. The proposed approach was then illustrated on a range of imaging inverse problems by using a deep network denoiser that satisfied our conditions for convergence.

In future work, we would like to continue our theoretical and empirical investigation of Bayesian PnP models, methods and algorithms. First, we would like to develop provably convergent accelerated algorithms based on more elaborated optimization schemes. Furthermore, developing methods to automatically adjust the regularisation parameter α directly from the observed data y in a manner akin to [START_REF] Vidal | Maximum likelihood estimation of regularization parameters in high-dimensional inverse problems: An empirical bayesian approach part i: Methodology and experiments[END_REF], is also a crucial question. So far, we set this parameter by experimenting, which is time consuming, computationally demanding and not theoretically grounded. It can also lead to model miss-specification that hurts the proposed MAP restoration. It certainly explains the results obtained for Cameraman on the deblurring task with α = 0.25. It would also be interesting to extend the decision-theoretic foundation of MAP estimation in log-concave models of [START_REF] Pereyra | Revisiting Maximum-a-Posteriori estimation in log-concave models[END_REF] to encompass MAP estimation in (not log-concave) PnP Bayesian models. Indeed, computing the MAP to solve an inverse problem (in the general case where the prior is not necessarily log-concave) is still not understood from a Bayesian point of view, as we do not know if it minimizes any cost function under the posterior distribution [START_REF] Bassett | Maximum a posteriori estimators as a limit of bayes estimators[END_REF][START_REF] Pereyra | Revisiting Maximum-a-Posteriori estimation in log-concave models[END_REF].

In the following chapter, we go further than simple point estimation. Indeed, as explained in Chapter 2, although it is practical to summarize the posterior distribution by one point, we have no information about the uncertainty on the proposed solution. It can be problematic, especially in contexts where a decision is based on the proposed restoration like in medicine. In the next chapter, we propose two Plug & Play sampling algorithms with detailed convergence guarantees and non-asymptotic error bounds under realistic assumptions to tackle this issue. As explained in Chapter 2, although computing the MAP is appealing, it is only wellunderstood in log-concave cases. We may wish to compute other estimators such as the MMSE, which is theoretically well-founded within the Bayesian framework. Furthermore, we may want to better exploit the resources provided by the Bayesian framework and not to simply summarize the posterior distribution to 1 point. For example, we could wish to quantify uncertainty on the proposed restoration or perform model calibration.

In this chapter we present our work on Monte Carlo sampling with PnP priors for general Bayesian computation. First, we answer primordial questions concerning the well-posedness, and the stability of the related Bayesian models and estimators in Section 5.2. Then, we present the Plug-and-Play Unadjusted Langevin Algorithm (PnP-ULA) and Projected Plugand-Play Unadjusted Langevin Algorithm (PPnP-ULA), two Monte-Carlo sampling algorithms with detailed convergence guarantees under realistic assumptions on the denoising operators used. A special attention is given to deep neural network denoisers (see Section 5.3). Section 5.4 illustrates the methods on classical imaging inverse problems such as denoising, deblurring and interpolation where it is used to perform MMSE point estimation and as well as for uncertainty visualization and quantification. Finally, we propose to speedup the state-space exploration of the proposed sampling algorithms using a state-of-the-art discretization scheme and show the results in Section 5.5.

Convergence results were derived by Valentin De Bortoli and are exposed in Appendix B for the sake of completeness.

This chapter is mostly based on the article [START_REF] Laumont | Bayesian imaging using plug & play priors: When langevin meets tweedie[END_REF] published in SIAM Journal on Imaging Sciences. Section 5.5 is entirely new.

Bayesian inference with

Bayesian modelling and analysis with Plug & Play priors

This section presents the same formal framework for Bayesian analysis and computation with Plug & Play priors as in Section 4.2.1. In order to address ill-posedness, we also introduce prior knowledge about x by specifying an image denoising operator D ε for recovering x from a noisy observation x ε with (X ε |X = x) ∼ N (x, ε Id) with noise variance ε > 0. A case of particular relevance in this context is when D ε is implemented by a neural network, trained by using a set of clean images {x i } N i=1 . As explained in Section 4.2.1, a central challenge in the formalisation of Bayesian inference with Plug & Play priors is that the denoiser D ε used is generally not directly related to a marginal distribution for x, so it is not possible to derive an explicit posterior for x|y from D ε . As a result, it is not clear that plugging D ε into gradient-based algorithms such as ULA leads to a well-defined or convergent scheme that is targeting a meaningful Bayesian model.

To overcome this difficulty, in a manner akin to Chapter 4, we analyse in this chapter the Bayesian models through the prism of M-complete Bayesian modelling [START_REF] Bernardo | Bayesian Theory[END_REF]. We recall that, in this paradigm, there exists a true -albeit unknown and intractable-marginal distribution for x on (R d , B(R d )), where B(R d ) denotes the Borel σfield of R d , noted µ, and a posterior distribution for x|y. If it were possible, basing inferences on these two distributions would be optimal both in terms of point estimation and in terms of delivering Bayesian probabilities accurate in a frequentist viewpoint. When µ admits a density w.r.t. the Lebesgue measure on R d , we denote it by p . In the latter case, the posterior distribution for x|y associated with the marginal µ also admits a density * that is given for any x ∈ R d and y ∈ R m by p (x|y) = p(y|x)p (x)/ R d p(y|x)p (x)dx .

(5.1)

Unlike most Bayesian imaging approaches that operate implicitly in an M-closed manner and treat their postulated Bayesian models as true models (see [START_REF] Bernardo | Bayesian Theory[END_REF] for more details), we explicitly regard p (or more precisely µ) as a fundamental property of the unknown x, and models used for inference as operational approximations of p specified by the practitioner (either analytically, algorithmically, or from training data). This distinction will be useful for using the oracle posterior (5.1) as a reference, and Plug & Play Bayesian algorithms based on a denoiser D ε as approximations to reference algorithms to perform inference w.r.t. p . The accuracy of the Plug & Play approximations will depend chiefly on the closeness between D ε and an optimal denoiser D ε derived form p that we define shortly.

In this conceptual construction, the marginal µ naturally depends on the imaging application considered. It could be the distribution of natural images of the size and resolution of x, or that of a class of images related to a specific application. And in problems where there is training data {x i } N i=1 available, we regard {x i } N i=1 as samples from µ. Lastly, we note that the posterior for x|y remains well defined when µ does not admit a density; this is important to provide robustness to situations where p is nearly degenerate or improper. For clarity, our presentation assumes that p exists, although this is not strictly required † .

Notice that because µ is unknown, we cannot verify that p (x|y) satisfies the basic desiderata for gradient-based Bayesian computation: i.e., p (x|y) need not be proper and differentiable, with ∇ log p (x|y) Lipschitz continuous. To guarantee that gradient-based algorithms that target approximations of p (x|y) are well defined by construction, we introduce a regularised oracle µ ε obtained via the convolution of µ with a Gaussian smoothing kernel with bandwidth ε > 0. Indeed, by construction, µ ε has a smooth proper density p ε given for any x ∈ R d and ε > 0 by

p ε (x) = (2πε) -d/2 R d exp [-x -x 2 2 /(2ε)]p (x)dx .
Equipped with this regularised marginal distribution, we use Bayes' theorem to involve the likelihood p(y|x) and derive the posterior density p ε (x|y), given for any ε > 0 and

x ∈ R d by p ε (x|y) = p(y|x)p ε (x)/ R d p(y|x)p ε (x)dx , (5.2)
which inherits the regularity properties required for gradient-based Bayesian computation when the likelihood satisfies H1, which for presentation clarity we recall below.

H1 For any y ∈ R m , sup x∈R d p(y|x) < +∞, p(y|•) ∈ C 1 (R d , (0, +∞)) and there exists L y > 0 such that ∇ log(p(y|•)) is L y Lipschitz continuous.
More precisely, Proposition 5.2.1 below establishes that the regularised prior p ε (x) and posterior p ε (x|y) are proper, smooth, and that they can be made arbitrarily close to the original oracle models p (x) and p (x|y) by reducing ε, with the approximation error vanishing as ε → 0. Proposition 5.2.1 Assume H1. Then, for any ε > 0 and y ∈ R m , the following hold: important perspective for future work.

† Operating without densities requires measure disintegration concepts that are technical (Schwartz).

(a) p ε and p ε (•|y) are proper.

(b) For any k ∈ N, p ε ∈ C k (R d ). In addition, if p(y|•) ∈ C k (R d ) then p ε (•|y) ∈ C k (R d , R). (c) Let k ∈ N. If R d x k p (x)dx < +∞ then R d x k p ε (x|y)dx < +∞. (d) lim ε→0 p ε (•|y) -p (•|y) 1 = 0.
(e) In addition, if there exist κ, β 0 such that for any

x ∈ R d , p -p (•-x) 1 x β , then there exists C 0 such that p ε (•|y) -p (•|y) 1 Cε β/2 .
Proof: The proof is postponed to Appendix B.8.2.

Under H1 and p(y|•) ∈ C 1 (R d ), x → ∇ log p ε (x|y) is well-defined and continuous. However, x → ∇ log p ε (x|y) might not be Lipschitz continuous and hence the Langevin SDE (3.25) might not have a strong solution. This requires an additional assumption on µ.

To study the Lipschitz continuity of x → ∇ log p ε (x|y), as well as to set the grounds for Plug & Play methods that define priors implicitly through a denoising algorithm, we use again the oracle MMSE denoiser D ε defined in (3.20). For presentation clarity, we recall it below.

∀ (x, ε) ∈ R d × R * + , D ε (x) = (2πε) -d/2 R d x exp [-x -x 2 /(2ε)]p (x)dx .
Under the assumption that the expected mean square error (MSE) is finite, D ε is the MMSE estimator to recover an image x ∼ µ from a noisy observation x ε ∼ N (x, ε Id) [START_REF] Robert | The Bayesian Choice: From Decision-Theoretic Foundations to Computational Implementation[END_REF]. Again, this optimal denoiser is a fundamental property of x and it is generally intractable. Motivated by the fact that state-of-the-art image denoisers are close-to-optimal in terms of MSE, in Section 5.3 we will characterise the accuracy of Plug & Play Bayesian methods for approximate inference w.r.t. p ε (x|y) and p (x|y) as a function of the closeness between the denoiser D ε used and the reference D ε .

To relate the gradient x → ∇ log p ε (x) and D ε , we use Tweedie's identity [START_REF] Efron | Tweedie's formula and selection bias[END_REF]) which states that for all (5.4) where g ε (•|x ε ) is the density of the conditional distribution of the unknown image x ∈ R d with marginal µ, given a noisy observation x ε ∼ N (x, ε Id). See Section 5.3.2 for details.

x ∈ R d ε∇ log p ε (x) = D ε (x) -x , ( 5
∈ R d R d x -D ε (x ε ) 2 g ε (x|x ε )dx C ,
Proof: The proof is postponed to Lemma B.6.2.

These results can be generalised to hold under the weaker assumption that the expected MSE for D ε is finite but not uniformly bounded, as in this case x → ∇ log p ε (x|y) is locally instead of globally Lipschitz continuous (this technical extension is postponed to future work). The pathological case where D ε does not have a finite MSE arises when µ is such that the denoising problem does not admit a Bayesian estimator w.r.t. to the MSE loss. In summary, the gradient x → ∇ log p ε (x|y) is Lipschitz continuous when µ carries enough information to make the problem of Bayesian image denoising under Gaussian additive noise well posed.

Notice that by using Tweedie's identity, we can express a ULA recursion for sampling approximately from p ε (x|y) as follows: .5) where we recall that {Z k : k ∈ N} are i.i.d standard Gaussian random variables on R d and δ > 0 is a positive step-size. Under standard assumptions on δ, the sequence generated by (5.5) is a Markov chain which admits an invariant probability distribution with a density provably close to p ε (x|y), with δ controlling a trade-off between asymptotic accuracy and convergence speed. In the following section we present Plug & Play ULAs that arise from replacing D ε in (5.5) with a denoiser D ε that is tractable. Before concluding this section, we study whether the oracle p (x|y) is itself well-posed, i.e., if p (x|y) changes continuously w.r.t. y under a suitable probability metric (see [START_REF] Latz | On the well-posedness of bayesian inverse problems[END_REF]). We answer positively to this question in Proposition 5.2.3 which states that, under mild assumptions on the likelihood, p (x|y) is locally Lipschitz continuous w.r.t. y for an appropriate metric. This stability result implies, for example, that the MMSE estimator derived from p (x|y) is locally Lipschitz continuous w.r.t. y, and hence stable w.r.t. small perturbations of y. Note that a similar property holds for the regularised posterior p ε (x|y). In particular, Proposition 5.2.3 holds for Gaussian likelihoods (see Section 5.3 for details). (5.6) and for any c > 0,

X k+1 =X k + δ∇ log p(y|X k ) + (δ/ε) (D ε (X k ) -X k ) + √ 2δZ k+1 . ( 5
(Φ 1 (x) + Φ 2 (y 1 ) + Φ 2 (y 2 )) y 1 -y 2 ,
R d (1 + Φ 1 (x)) exp[cΦ 1 (x)]p (x)dx < +∞. Then y → p (•|y) is locally Lipschitz w.r.t • 1 , i.e.
, for any compact set K there exists C K 0 such that for any

y 1 , y 2 ∈ K , p (•|y 1 ) -p (•|y 2 ) 1 C K y 1 -y 2 .
Proof: The proof is a straightforward application of Proposition B.5.1.

To conclude, starting from the decision-theoretically optimal model p (x|y), we have constructed a regularised approximation p ε (x|y) that is proper and smooth by construction, with gradients that are explicitely related to denoising operators by Tweedie's formula. Under mild assumptions on p(y|x), the approximation p ε (x|y) is well-posed and can be made arbitrarily close to the oracle p (x|y) by controlling ε. Moreover, we established that x → ∇ log p ε (x) is Lipschitz continuous when the problem of Gaussian image denoising for µ under the MSE loss is well posed. This allows imagining convergent gradient-based algorithms for performing Bayesian computation for p ε (x|y), setting the basis for Plug & Play ULA schemes that mimic these idealised algorithms by using a tractable denoiser D such as neural network, trained to optimise MSE performance and hence to approximate the oracle MSE denoiser D .

Bayesian computation with Plug & Play priors

We are now ready to study Plug & Play ULA schemes to perform approximate inference w.r.t. p ε (x|y) (and hence indirectly w.r.t. p (x|y)). We use (5.5) as starting point, with D ε replaced by a surrogate denoiser D ε , but also modify (5.5) to guarantee geometrically fast convergence ‡ to a neighbourhood of p ε (x|y). In particular, geometrically fast convergence is achieved here by modifying far-tail probabilities to prevent the Markov chain from becoming too diffusive as it explores the tails of p ε (x|y). We consider two alternatives to guarantee geometric convergence with markedly different bias-variance trade-offs: one with excellent accuracy guarantees but that requires using a small step-size δ and hence has a higher computational cost, and another one that allows taking a larger step-size δ to improve convergence speed at the expense of weaker guarantees in terms of estimation bias.

First, in the spirit of Moreau-Yosida regularised ULA [START_REF] Durmus | Efficient bayesian computation by proximal markov chain monte carlo: when langevin meets moreau[END_REF], we define Plug & Play ULA (PnP-ULA) as the following recursion: given an initial state X 0 ∈ R d and for any k ∈ N, (5.7) where C ⊂ R d is some large compact convex set that contains most of the prior probability mass of x, Π C is the projection operator onto C w.r.t the Euclidean scalar product on R d , and λ > 0 is a tail regularisation parameter that is set such that the drift in PnP-ULA satisfies a certain growth condition as x → ∞ (see Section 5.3 for details).

(PnP-ULA) X k+1 =X k + δ∇ log p(y|X k ) + (δ/ε) (D ε (X k ) -X k ) + (δ/λ)(Π C (X k ) -X k ) + √ 2δZ k+1 ,
An alternative strategy (which we call Projected PnP-ULA, i.e. PPnP-ULA, see Algorithm 6) is to modify PnP-ULA to include a hard projection onto C, i.e. (X k ) k∈N is defined by X 0 ∈ C and the following recursion for any k ∈ N (5.8) where we notice that, by construction, the chain cannot exit C because of the action of the projection operator Π C . The hard projection guarantees geometric convergence with weaker restrictions on δ and hence PPnP-ULA can be tuned to converge significantly faster than PnP-ULA, albeit with a potentially larger bias. These two schemes are summarised in Algorithm 5 and Algorithm 6 below. Note the presence of a regularisation parameter α in these algorithms, which permits to balance the weights between the prior and data terms.

X k+1 = Π C X k + δ∇ log p(y|X k ) + (δ/ε)(D ε (X k ) -X k ) + √ 2δZ k+1 ,
For the sake of simplicity, this parameter is set to α = 1 in Section 5.3 and Section 5.4 but will be taken into account in the supplementary material Appendix B. Section 5.3.2 and Section 5.3.3 present detailed convergence results for PnP-ULA and PPnP-ULA. Implementation guidelines, including suggestions for how to set the algorithm parameters of PnP-ULA and PPnP-ULA are provided in Section 5.4. Lastly, it is worth mentioning that Algorithm 5 and Algorithm 6 can be straightforwardly modified to incorporate additional regularisation terms. More precisely, one could consider a prior defined as the (normalised) product of a Plug & Play term and an explicit analytical term. In that case, one should simply modify the recursion defining the Markov chain by adding the gradient associated with the analytical term. In a manner akin to [START_REF] Durmus | Efficient bayesian computation by proximal markov chain monte carlo: when langevin meets moreau[END_REF], analytical terms that are not smooth are involved via their proximal operator.

Before concluding this section, it is worth emphasising that, in addition to being important in their own right, Algorithm 5 and Algorithm 6 and the associated theoretical results

Notation

Denote by B(R d ) the Borel σ-field of R d , and for f :

R d → R measurable, f ∞ = sup x∈R d |f (x)|. For µ a probability measure on (R d , B(R d )) and f a µ-integrable function, denote by µ(f ) the integral of f w.r.t. µ. For f : R d → R measurable and V : R d → [1, ∞) measurable, the V -norm of f is given by f V = sup x∈R d |f (x)|/V (x).
Let ξ be a finite signed measure on (R d , B(R d )). The V -total variation distance of ξ is defined as

ξ V = sup f V 1 R d f (x)dξ(x) .
(5.11) ) the set of R m -valued k-differentiable functions, respectively the set of compactly supported R m -valued and k-differentiable functions. Let f : U → R, we denote by ∇f , the gradient of f if it exists. f is said to be m-convex with m 0 if for all x 1 , x 2 ∈ R d and t ∈ [0, 1],

If V = 1, then • V is
f (tx 1 + (1 -t)x 2 ) tf (x 1 ) + (1 -t)f (x 2 ) -mt(1 -t) x 1 -x 2 2 /2 . ( 5

Convergence of PnP-ULA

In this section, we fix ε > 0 and derive quantitative bounds between the iterates of PnP-ULA and π ε with respect to the total variation distance. To address this issue, we first show that PnP-ULA is geometrically ergodic and establish non-asymptotic bounds between the corresponding Markov kernel and its invariant distribution. Second, we analyse the distance between this stationary distribution and π ε .

For any ε > 0 we define

g ε : R d × R d → [0, +∞) for any x 1 , x 2 ∈ R d by g ε (x 1 |x 2 ) = p (x 1 ) exp[-x 2 -x 1 2 /(2ε)] R d p (x) exp[-x 2 -x 2 /(2ε)]dx . (5.13)
Note that g(•|X ε ) is the density with respect to the Lebesgue measure of the distribution of X given X ε , where X is sampled according to the prior distribution µ (with density p ) and X ε = X + ε 1/2 Z where Z is a Gaussian random variable with zero mean and identity covariance matrix. Throughout, this section, we consider the following assumption on the family of denoising operators {D ε : ε > 0} which will ensure that PnP-ULA approximately targets π ε .

H4 (R)

We have that R d x 2 p (x)dx < +∞. In addition, there exist ε 0 > 0, M R 0 and L 0 such that for any ε ∈ (0, ε 0 ], x 1 , x 2 ∈ R d and x ∈ B(0, R) we have (5.14) where we recall that

(Id -D ε )(x 1 ) -(Id -D ε )(x 2 ) L x 1 -x 2 , D ε (x) -D ε (x) M R ,
D ε (x 1 ) = R d x g ε (x|x 1 )dx .
(5.15)

H2 and H4 are very similar, except that the Lipschitz continuity condition concerns Id -D ε in H4 and D ε in H2. The Lipschitz continuity condition in (5.14) will be useful for establishing the stability and geometric convergence of the Markov chain generated by PnP-ULA. This condition can be explicitly enforced during training by using an appropriate regularization of the neural network weights [START_REF] Ryu | Plug-and-play methods provably converge with properly trained denoisers[END_REF][START_REF] Miyato | Spectral normalization for generative adversarial networks[END_REF]. Regarding the second condition in (5.14), M R is a bound on the error involved in using D ε as an approximation of D ε for images of magnitude R (i.e., for any x ∈ B(0, R)), and it will be useful for bounding the bias resulting from using PnP-ULA for inference w.r.t. π ε (recall that the bias vanishes as M R → 0 and δ → 0). For denoisers represented by neural networks, one can promote a small value of M R during training by using an appropriate loss function. More precisely, consider a neural network f w : R d → R d , parameterized by its weights and bias gathered in w ∈ W where W is some measurable space, for any ε > 0, one could target empirical approximation of a loss of the form ε : W → [0, +∞) given for any w ∈ W by

ε (w) = R d ×R d x -f w (x ε ) 2 p ε (x ε )g ε (x|x ε )dx ε dx.
Note that such a loss is considered in the Noise2Noise network introduced in [START_REF] Lehtinen | Noise2noise: Learning image restoration without clean data[END_REF].

With regards to the theoretical limitations stemming from representing D ε by a deep neural network, universal approximation theorems (see e.g., (Bach, 2017, Section 4.7)) suggest that M R could be arbitrarily low in principle. For a given architecture and training strategy, and if there exists MR 0 such that inf w∈W sup x∈B(0,R)

M-1 R f w (x) -D ε (x) } 1 then the second condition in (5.14) holds upon letting D ε = f w † for an appropriate choice of weights w † ∈ W. This last inequality can be established using universal approximation theorems such as (Bach, 2017, Section 4.7). Moreover, for any other w ∈ W, (5.15). Consider w † ∈ W obtained after numerically minimizing ε and satisfying ε (w † ) ε + η with η > 0. In this case, the following result ensures that (5.14) is satisfied with M R of order η 1/(2d+2) for any R > 0 and letting D ε = f w † .

ε (w) R d ×R d x -D ε (x ε ) 2 p ε (x ε )g ε (x|x ε )dxdx ε = ε , since for any x ε ∈ R d , D ε (x ε ) = R d x g ε (x|x ε )dx, see

Proposition 5.3.1 Assume that for any

w ∈ W R d ( x 2 + f w (x ε ) 2 )p ε (x ε )g ε (x|x ε )dxdx ε < +∞ . (5.16) Let R, η > 0 and w † ∈ W such that ε (w ) ε + η. In addition, assume that sup x1,x2∈B(0,2R) x 2 -x 1 -1 ( f w † (x 2 ) -f w † (x 1 ) + D ε (x 2 ) -D ε (x 1 ) ) < +∞ , (5.17)
where D ε is given in (5.15). Then there exists C R , ηR 0 such that if η ∈ (0, ηR ] then for any

x ∈ B(0, R), f w † (x) -D ε (x) C R η 1/(2d+2) .
Proof: The proof is postponed to Appendix B.6.1.

Note that (5.16) is satisfied if for any w ∈ W, sup x∈R d f w (x) (1 + x ) -1 < +∞ and H4 holds.

We recall that PnP-ULA, see Algorithm 5, is given by the following recursion: X 0 ∈ R d and for any k ∈ N

X k+1 = X k + δb ε (X k ) + √ 2δZ k+1 , (5.18) b ε (x) = ∇ log p(y|x) + P ε (x) + (prox λ (ι C )(x) -x)/λ , P ε (x) = (D ε (x) -x)/ε , (5.19)
where δ > 0 is a step-size, ε, λ > 0 are hyperparameters of the algorithm, C ⊂ R d is a closed convex set, {Z k : k ∈ N} a family of i.i.d. Gaussian random variables with zero mean and identity covariance matrix and prox λ (ι C ) the proximal operator of ι C with step-size λ, see (Bauschke et al., 2011, Definition 12.23), where

ι C is the convex indicator of C defined for x ∈ R d by ι C = +∞ if x / ∈ C and 0 if x ∈ C. Note that for any x ∈ R d we have prox λ (ι C )(x) = Π C (x), where Π C is the projection onto C.
In what follows, for any δ > 0 and C ⊂ R d closed and convex, we denote by R ε,δ : R d × B(R d ) → [0, 1] the Markov kernel associated with the recursion (5.18) and given for any .20) Note that for ease of notation, we do not explicitly highlight the dependency of R ε,δ and b ε with respect to the hyperparameter λ > 0 and C.

x ∈ R d and A ∈ B(R d ) by R ε,δ (x, A) = (2π) -d/2 R d 1 A (x + δb ε (x) + √ 2δz) exp[-z 2 /2]dz . ( 5
Here we consider the case where x → log p(y|x) satisfies a one-sided Lipschitz condition, i.e. we consider the following condition.

H5 There exists m ∈ R such that for any x 1 , x 2 ∈ R d we have

∇ log p(y|x 2 ) -∇ log p(y|x 1 ), x 2 -x 1 -m x 2 -x 1 2 .
(5.21)

We refer to the supplementary material Appendix B.3 for refined convergence rates in the case where x → log p(y|x) is strongly m-concave. Note that if H5 is satisfied with m > 0 then x → log p(y|x) is m-concave. Assume H1 then H5 holds for m = -L y . However, it is possible that m > -L y which leads to better convergence rates for PnP-ULA. As a result even when H1 holds we still consider H5. In order to deal with H5 in the case where m 0, we set C ⊂ R d to be some convex compact set fixed by the user. Doing so, we ensure the stability of the Markov chain. The choice of C in practice is discussed in Section 5.4. In our imaging experiments, we recall that for any x ∈ R d we have, p(y|x) ∝ exp[-Ax -y 2 /(2σ 2 )]. If A is not invertible then x → log p(y|x) is not m-concave with m > 0. This is the case, in our deblurring experiment when the convolution kernel has zeros in the Fourier domain.

We start with the following result which ensures that the Markov chain (5.18) is geometrically ergodic under H4 for the Wasserstein metric W 1 and in V -norm for V :

R d → [1, +∞) given for any x ∈ R d by V (x) = 1 + x 2 .
(5.22) Proposition 5.3.2 Assume H1, H4(R) for some R > 0 and H5. (5.24) where V is given in (5.22).

Let λ > 0, ε ∈ (0, ε 0 ] such that 2λ(L y + L/ε -min(m, 0)) 1 and δ = (1/3)(L y + L/ε + 1/λ) -1 . Then for any C ⊂ R d convex and compact with 0 ∈ C, there exist A 1,C 0 and ρ 1,C ∈ [0, 1) such that for any δ ∈ (0, δ], x 1 , x 2 ∈ R d and k ∈ N we have δ x1 R k ε,δ -δ x2 R k ε,δ V A 1,C ρ kδ 1,C (V 2 (x 1 ) + V 2 (x 2 )) , (5.23) W 1 (δ x1 R k ε,δ , δ x2 R k ε,δ ) A 1,C ρ kδ 1,C x 1 -x 2 ,
The constants A 1,C and ρ 1,C do not depend on the dimension d but only on the parameters m, L, L y , ε and C. Note that a similar result can be established for W p for any p ∈ N * instead of W 1 . Under the conditions of Proposition 5.3.2 we have for any

ν 1 , ν 2 ∈ P 1 (R d ) ν 1 R k ε,δ -ν 2 R k ε,δ V A 1,C ρ kδ 1,C R d V 2 (x)dν 1 (x) + R d V 2 (x)dν 2 (x) , (5.25)
W 1 (ν 1 R k ε,δ , ν 2 R k ε,δ ) A 1,C ρ kδ 1,C R d x dν 1 (x) + R d
x dν 2 (x) .

(5.26)

First, (P 1 (R d ), W 1 ) is a complete metric space (Villani, 2009, Theorem 6.18). Second, for any δ ∈ (0, δ], there exists m ∈ N * such that f m is contractive with f :

P 1 (R d ) → P 1 (R d )
given for any ν ∈ P 1 (R d ) by f(ν) = νR ε,δ using Proposition 5.3.2. Therefore we can apply the Picard fixed point theorem and we obtain that R ε,δ admits an invariant probability measure π ε,δ ∈ P 1 (R d ).

Therefore, since π ε,δ is an invariant probability measure for R ε,δ and π ε,δ ∈ P 1 (R d ), using (5.25), we have for any

ν ∈ P 1 (R d ) νR k ε,δ -π ε,δ V A 1,C ρ kδ 1,C R d V 2 (x)dν(x) + R d V 2 (x)dπ ε,δ (x) , (5.27)
W 1 (νR k ε,δ , π ε,δ ) A 1,C ρ kδ 1,C R d x dν(x) + R d x dπ ε,δ (x) . (5.28)
Combining this result with the fact that for any t 0, (1 -e -t ) -1 1 + t -1 , we get that for any n ∈ N * and h : R d → R measurable such that sup

x∈R d {(1 + x 2 ) -1 |h(x)|} < +∞ n -1 n k=1 E[h(X k )] - R d h(x)dπ ε,δ (x) (5.29) A 1,C ( δ + log -1 (1/ρ 1,C )) V 2 (x) + R d V 2 (x)dπ ε,δ (x) (nδ) , (5.30)
where (X k ) k∈N is the Markov chain given by (5.18) with starting point

X 0 = x ∈ R d .
In the rest of this section we evaluate how close the invariant measure π ε,δ is to π ε . Our proof will rely on the following assumption which is necessary to ensure that x → log p ε (x) has Lipschitz gradients, see Proposition 5.2.2.

H6

For any ε > 0, there exists K ε 0 such that for any

x ∈ R d , R d x - R d x g ε (x |x)dx 2 g ε (x|x)dx K ε , (5.31)
with g ε given in (5.13).

We emphasize that H6 is not needed to establish the convergence of the Markov chain. However, we impose it in order to compare the stationary distribution of PnP-ULA with the target distribution π ε . Depending on the prior distribution density p , H6 may be checked by hand. Finally, note that H6 can be extended to cover the case where the prior distribution µ does not admit a density with respect to the Lebesgue measure.

In the following proposition, we show that we can control the distance between π ε,δ and π ε based on the previous observations. Proposition 5.3.3 Assume H1, H4(R) for some R > 0, H5 and H6. Moreover, let ε ∈ (0, ε 0 ] and assume that

R d (1 + x 4 )p ε (x)dx < +∞. Let λ > 0 such that 2λ(L y + (/ε) max(L, 1 + K ε /ε) -min(m, 0))
1 and δ = (1/3)(L y + L/ε + 1/λ) -1 . Then for any δ ∈ (0, δ] and C convex and compact with 0 ∈ C, R ε,δ admits an invariant probability measure π ε,δ . In addition, there exists B 0 0 such that for any C convex compact with B(0, R C ) ⊂ C and R C > 0, there exists B 1,C 0 such that for any δ ∈ (0, δ]

π ε,δ -π ε V B 0 R -1 C + B 1,C (δ 1/2 + M R + exp[-R]) , (5.32)
where V is given in (5.22).

Proof: The proof is postponed to Appendix B.6.3.

We now combine Proposition 5.3.2 and Proposition 5.3.3 in order to control the bias of the Monte Carlo estimator obtained using PnP-ULA. In the supplementary material Appendix B.4 we also provide bounds on

|n -1 n k=1 E[h(X k )]- R d h(x)dπ(x)| by controlling π -π ε V .
Proposition 5.3.4 Assume H1, H4(R) for some R > 0, H5 and H6. Moreover, let > 0, ε ∈ (0, ε 0 ] and assume that

R d (1 + x 4 )p ε (x)dx < +∞. Let λ > 0 such that 2λ(L y + (1/ε) max(L, 1 + K ε /ε) -min(m, 0)) 1 and δ = (1/3)(L y + L/ε + 1/λ) -1 . Then there exists C 1,ε > 0 such that for any C convex compact with B(0, R C ) ⊂ C and R C > 0 there exists C 2,ε such that for any h : R d → R measurable with sup x∈R d {|h(x)| (1 + x 2 ) -1 } 1, n ∈ N * , δ ∈ (0, δ] we have n -1 n k=1 E [h(X k )] - R d h(x)dπ ε (x) C 1,ε R -1 C + C 2,ε,C (δ 1/2 + M R + exp[-R] + (nδ) -1 ) (1 + x 4 ) . (5.

33)

Proof: The proof is straightforward combining Proposition 5.3.2 and Proposition 5.3.3.

Convergence guarantees for PPnP-ULA

We now study the Projected Plug & Play Unadjusted Langevin Algorithm (PPnP-ULA). It is given by the following recursion: X 0 ∈ C and for any k ∈ N

X k+1 = Π C (X k + δb ε (X k ) + √ 2δZ k+1 ) , (5.34) b ε (x) = ∇ log p(y|x) + P ε (x) , P ε (x) = (D ε (x) -x)/ε , (5.35)
where .36) Note that for ease of notation, we do not explicitly highlight the dependency of Q ε,δ and b ε with respect to the hyperparameter C. First, we have the following result which ensures that PPnP-ULA is geometrically ergodic for all step-sizes. Proposition 5.3.5 Assume H1, H4(R) for some R > 0. Let λ, ε, δ > 0. Then for any C ⊂ R d convex and compact with 0 ∈ C, there exist ÃC 0 and ρC ∈ [0, 1) such that for any δ ∈ (0, δ], x 1 , x 2 ∈ C and k ∈ N we have

δ > 0 is a step-size, ε > 0 is an hyperparameter of the algorithm, C ⊂ R d is a closed convex set, {Z k : k ∈ N}
∈ B(R d ) by Q ε,δ (x, A) = (2π) -d/2 R d 1 Π -1 C (A) (x + δb ε (x) + √ 2δz) exp[-z 2 /2]dz . ( 5
δ x1 Q k ε,δ -δ x2 Q k ε,δ TV ÃC ρkδ C .
(5.37)

Proof: The proof is postponed to Appendix B.7.1.

In particular Q ε,δ admits an invariant probability measure π C ε,δ . The next proposition ensures that for small enough step-size δ the invariant measures of PnP-ULA and PPnP-ULA are close if the compact convex set C has a large diameter. Proposition 5.3.6 Assume H1, H4(R) for some R > 0 and H5. In addition, assume that there exists m, c > 0 such that for C = R d and for any ε > 0 and

x ∈ R d , b ε (x), x -m x 2 + c. Let λ > 0, ε ∈ (0, ε 0 ] such that 2λ(L y + L/ε -min(m, 0)) 1.
Then there exist Ā 0 and η, δ > 0 such that for any C ⊂ R d convex and compact with 0 ∈ C and

B(0, R C /2) ⊂ C ⊂ B(0, R C ) and δ ∈ (0, δ] we have π ε,δ -π C ε,δ TV Ā exp[-ηR C ] ,
(5.38)

where π ε,δ is the invariant measure of R ε,δ and π C ε,δ is the invariant measure of Q ε,δ . Proof: The proof is postponed to Appendix B.7.2.
It is worth mentioning at this point that in our experiments, see Section 5.4, the probability of the iterates (X n ) n∈N leaving C with PnP-ULA or with PPnP-ULA is so low that the projection constraint is not activated. As a result, if implemented with the same stepsize both algorithms produce the same results. We do not suggest completely removing the constraints as this is important to theoretically guarantee the geometric ergodicity of the algorithms.

Regarding the choice of the step-size, we observe that the bound δ = (1/3)(L y + L/ε + 1/λ) -1 used in PnP-ULA is conservative and our experiments suggest that PnP-ULA is stable for larger step-sizes. This section illustrates the behaviour of PnP-ULA and PPnP-ULA with two classical imaging inverse problems: non-blind image deblurring and interpolation. For these two problems, we first analyse in detail the convergence of the Markov chain generated by PnP-ULA for different test images. This is then followed by a comparison between the MMSE Bayesian point estimator, as calculated by using PnP-ULA and PPnP-ULA and the MAP estimator provided by the recent PnP-SGD method presented in Chapter 4. To simplify comparisons, for all experiments and algorithms, the operator D ε is chosen as the pretrained denoising neural network introduced in [START_REF] Ryu | Plug-and-play methods provably converge with properly trained denoisers[END_REF], for which

(D ε -Id) is L-Lipschitz with L < 1.
For the deblurring experiments, the observation model takes the form

y = Ax + n , (5.39)
where x ∈ R d is the unknown original image, y ∈ R m the observed image, n is a realization of a Gaussian i.i.d. centered noise with variance σ 2 Id (with σ 2 = (1/255) 2 ), and A is a 9 × 9 box blur operator. The log-likelihood for this case writes log p(y|x) = -Ax -y 2 /(2σ 2 ).

In the interpolation experiments, we seek to recover x ∈ R d from y = Ax where the matrix A is an m × d matrix containing m randomly selected rows of the d × d identity matrix. We focus on a case where 80% of the image pixels are hidden and the observed pixels are measured without any noise. Because the posterior density for x|y is degenerate, we run PnP-ULA on the posterior x|y where x := Px ∈ R n denotes the vector of n = d -m unobserved pixels of x, and map samples to the pixel space by using the affine mapping f y : R n → R d defined for any x ∈ R n and y ∈ R m by f y (x) = P x + A y.

Note that we can write the log-posterior Ũε (x) = -log p ε (x|y) on the set R n of hidden pixels in terms of f y and the log-prior U ε (x) = -log p ε (x) on the set R d :

Ũε = U ε • f y .
Using the chain rule and Tweedie's formula, we have that for any

x ∈ R d and y ∈ R m b ε (x) = -∇ Ũε (x) = -P∇U ε (f y (x)) = (1/ε)P(D ε -Id)(f y (x)) .
(5.40)

Since P and f y are 1-Lipschitz, b ε = -∇ Ũε is also Lipschitz with constant L (L/ε). Figure 5.1 shows the six test images of size 256 × 256 pixels that were used in the experiments. We have selected these six images for their diversity in composition, content and level of detail (some images are predominantly composed of piece-wise constant regions, whereas others are rich in complex textures). This diversity will highlight strengths and limitations of the chosen denoiser as an image prior. Figure 5.2 depicts the corresponding blurred images and Figure 5.3 the images to interpolate.

Cameraman.

Simpson. Traffic.

Alley.

Bridge. Goldhill.

Implementation guidelines and parameter setting

In the following, we provide some simple and robust rules in order to set the parameters of the different algorithms, in particular the discretization step-size δ and the tail regularization parameter λ.

CHOICE OF THE DENOISER

The theory presented in Section 5.3 requires that D ε satisfies H4(R). As default choice, we recommend using a pretrained denoising neural network such as the one described in [START_REF] Ryu | Plug-and-play methods provably converge with properly trained denoisers[END_REF]. The Lipschitz constant of the network is controlled during training by using spectral normalization and therefore the first condition of H4(R) holds. Moreover, the loss function used to train the network is given by ε as introduced in Section 5.3.2. Therefore, under the conditions of Proposition 5.3.1, we get that the second condition of H4(R) holds.

STEP-SIZE δ

The parameter δ controls the asymptotic accuracy of PnP-ULA and PPnP-ULA, as well as the speed of convergence to stationarity. This leads to the following biasvariance trade-off. For large values of δ, the Markov chain has low auto-correlation and converges quickly to its stationary regime. Consequently, the Monte Carlo estimates computed from the chain exhibit low asymptotic variance, at the expense of some asymptotic bias. On the contrary, small values of δ produce a Markov chain that explores the parameter space less efficiently, but more accurately. As a result, the asymptotic bias is smaller, but the variance is larger. In the context of inverse problems that are high-dimensional and ill-posed, properly exploring the solution space can take a large number of iterations. For this reason, we recommend using large values of δ, at the expense of some bias. In addition, in PnP-ULA, δ is also subject to a numerical stability constraint related to the inverse of the and the Lipschitz constant of the log-likelihood gradient. In our experiments, L = 1 and L y = A A /σ 2 , so we choose δ just below the upper bound δ th = 1/3(Lip(b ε )) -1 where A is the adjoint of A. For PPnP-ULA, we set δ < (L/ε + L y ) -1 (resp. δ < (L/ε) -1 for interpolation) to prevent excessive bias.

PARAMETER λ The parameter λ controls the tail behaviour of the target density. As previously explained, it must be set so that the tails of the target density decay sufficiently fast to ensure convergence at a geometric rate, a key property for guaranteeing that the Monte Carlo estimates computed from the chain are consistent and subject to a Central Limit Theorem with the standard O( √ k) rate. More precisely, we require λ ∈ (0, 1/2(L/ε + 2L y )). Within this admissible range, if λ is too small this limits the maximal δ and leads to a slow Markov chain. For this reason, we recommend setting λ as large as possible below

(2L/ε + 4L y ) -1 . OTHER PARAMETERS The compact set C is defined as C = [-1, 2]
d , even if in practice no samples where generated outside of C in all our experiments, which suggests that the tail decay conditions hold without explicitly enforcing them. In all our experiments, we set the noise level of the denoiser D ε to ε = (5/255) 2 . The initialization X 0 can be set to a random vector. In our experiments (where m = d), we chose X 0 = y in order to reduce the number of burn-in iterations. For m = d we could use X 0 = A y instead. Concerning the regularization parameter α, by default we set α = 1, but in some cases it is possible to marginally improve the results by fine tuning it. All algorithms are implemented using Python and the PyTorch library, and run on an Intel Xeon CPU E5-2609 server with a Nvidia Titan XP graphic card or on Idris' Jean-Zay servers featuring Intel Cascade Lake 6248 CPUs with a single Nvidia Tesla V100 SXM2 GPU. Reported running times correspond to the Xeon + Titan XP configuration.

Convergence analysis of PnP-ULA in non-blind image deblurring and inpainting

When using a sampling algorithm such as PnP-ULA on a new problem, it is essential to check that the state space is correctly explored. In order to provide a thorough convergence study, we first run the algorithm for 25 × 10 6 iterations. We use a burn-in period of 2.5 × 10 6 iterations, and consider only the samples computed after this burn-in period to study the Markov chain in close-to-stationary regime. In Section 5.4.3, we will see that much less iterations are required if the goal is only to compute point estimators with PnP-ULA. For simplicity, the algorithm is always initialized with the observation y in our experiments with PnP-ULA (for interpolation, this means that unknown pixels are initialized to the value 0).

There is no fully comprehensive way to empirically characterise the convergence properties of a high-dimensional Markov chain, as different statistics computed from the same chain align differently with the eigenfunctions of the Markov kernel and hence exhibit different convergence speeds. In problems of small dimension, we would calculate and analyse the ddimensional multivariate autocorrelation function (ACF) of the Markov chain, but this is not feasible in imaging problems. In problems of moderate dimension, one could characterise the range of convergence speeds by first estimating the posterior covariance matrix (which, for 256×256 images, would be a 256 2 ×256 2 matrix) and then performing a principal component analysis on this matrix to identify the directions with smallest and largest uncertainty, as these would provide a good indication of the subspaces where the chain converges the fastest and the slowest. However, computing the posterior covariance matrix is also not possible in imaging problems because of the dimensionality involved. Here we focus on approximations of the posterior covariance which make sense for the particular inverse problem we study. More precisely, we use the diagonalization basis of the inverse operator, i.e. the Fourier basis for the deblurring experiments, and the basis formed by the unknown pixels for the inpainting experiments. Under the assumption that the posterior covariance is mostly determined by the likelihood, this strategy allows broadly identifying the linear statistics that converge fastest and slowest, without requiring the estimation and manipulation of prohibitively large matrices.

INTERPOLATION We first focus on the interpolation problem. Figure 5.4 shows a map of the pixel-wise marginal standard deviations, for all images. We observe that pixels in homogeneous regions have low uncertainty, while pixels on textured regions, edges, or complex structures (a reflection on the window shutter in the Alley image for instance) are the most uncertain.

For the same experiments, Figure 5.5 shows the Euclidean distance between the final MMSE estimate (computed using all samples) and the samples of the chain, every 2500 samples (after the burn-in period, and hence in what is considered to be a close-to-stationary regime). Fluctuations around the posterior mean and the absence of temporal structure in the plots of Alley or Goldhill are a first indication that the chain explores the solution space with ease. However, in some other cases such as the Simpson image, we observe metastability, where the chain stays in a region of the space for millions of iterations and then Uncertainty is located around edges and in textured areas.

jumps to a different region, again for millions of iterations. This is one of the drawbacks of operating with a posterior distribution that is not log-concave and that may exhibit several modes. Lastly, Figure 5.6 displays the sample ACFs of the fastest and slowest converging statistics associated with the interpolation experiments (as estimated by identifying, for each image, the unknown pixels with lowest and highest uncertainty). These ACF plots measure how fast samples become uncorrelated. A fast decay of the ACF is associated with good Markov chain mixing, which in turn implies accurate Monte Carlo estimates. On the contrary, a slow decay of the ACF indicates that the Markov chain is moving slowly, which leads to Monte Carlo estimates with high variance. As mentioned previously, because computing and visualising a multivariate ACF is difficult, here we show the ACF of the chain along the slowest and the fastest directions in the spatial domain (for completeness, we also show the ACF for a pixel with median uncertainty). We see that independence is reached very fast in the subspaces of low or median uncertainty, and is much slower for the few very uncertain pixels.

DEBLURRING We now focus on the non-blind image deblurring experiments, where, as explained previously, we perform our convergence analysis by using statistics associated with the Fourier domain. Figure 5.7 depicts the marginal standard deviation of the Fourier coefficients (in absolute value), for all images. For the three images Cameraman, Simpsons and Traffic, all the standard deviations have a similar range of values, and the largest values are observed around frequencies in the kernel of the blur filter (shown on the right of the same figure) and for high frequencies. Conversely, for the three images Alley, Bridge and Goldhill, very high uncertainty is observed in the vicinity of four specific frequencies. This suggests that the denoiser used is struggling to regularise these specific frequencies, and Cameraman.

Simpson.

Traffic.

Alley. Bridge. Goldhill. For the first three images, we clearly see that uncertainty is observed on frequencies that are near the kernel of the blur filter (shown on the right), and is also higher around high frequencies (i.e. around edges and textured areas in images). For the last three images, very high uncertainty is observed around some specific frequencies. In the direction of these frequencies, the Markov chain is moving very slowly and the mixing time of the chain is particularly slow, as shown on Figure 5.9.

Fastest direction Median direction Slowest direction

consequently the posterior distribution is very spread along these directions and difficult to explore by Markov chain sampling as a result. Interestingly, this phenomenon is only observed in the images that are rich in texture content. Moreover, Figure 5.8 depicts the Euclidean distance between the MMSE estimator computed from entire chain (i.e. all samples) and each sample (we show one point every 2500 samples). We notice that many of the images exhibit some degree of meta-stability or slow convergence because of the presence of directions in the solution space with very high uncertainty. Again, this is consistent with our convergence theory, which identifies posterior multimodality and anisotropy as key challenges that future work should seek to overcome.

Lastly, we show on Figure 5.9 the sample ACFs for the slowest and the fastest directions in the Fourier domain § . Again, in all experiments, independence is achieved quickly in the fastest direction. The behaviour of the slowest direction for the three images Alley, Bridge and Goldhill suggests that the Markov chain is close to the stability limit and exhibits highly oscillatory behaviour as well as poor mixing.

Point estimation for non-blind image deblurring and interpolation

We are now ready to study the quality of the MMSE estimators delivered by PnP-ULA and PPnP-ULA and report comparisons with MAP estimation by PnP-SGD introduced in Chapter 4.

Cameraman.

Simpson.

Traffic.

Alley. Bridge. Goldhill.

Fast direction Slow direction Fast direction Slow direction

Figure 5.9: ACF for the deblurring problem. The ACF are shown for lags up to 1.75e5 for the three images Cameraman, Simpson and Traffic (see the two plots to the left) and independence seems to be achieved in all directions. For the three other images, independence is not achieved in the slowest direction (corresponding to the most uncertain frequency of the samples in the Fourier domain) even after 1e6 iterations.

QUANTITATIVE RESULTS Figure 5.10 illustrates the evolution of the PSNR of the mean of the Markov chain (the Monte Carlo estimate of the MMSE solution), as a function of the number of iterations, for the six images of Figure 5.1. These plots have been computed by using a step-size δ = δ th that is just below the stability limit and a 1-in-2500 thinning. We observe that the PSNR between the MMSE solution as computed by the Markov chain and the truth stabilises in approximately 10 5 iterations in the experiments where the chain exhibits fast convergence, whereas over 10 6 are required in experiments that suffer from slow convergence (e.g., deblurring of Alley, Bridge and Goldhill). Moreover, we observe that using PPnP-ULA with a larger step-size can noticeably reduce the number of iterations required to obtain a stable estimate of the posterior mean, particularly in the image deblurring experiments. VISUAL RESULTS Figures 5.11 to 5.14 show the MMSE estimate computed by PnP-ULA on the whole chain including the burn-in for the 6 images, for the interpolation and deblurring experiments. We also provide the MAP estimation results computed by using PnP-SGD (see Chapter 4), which targets the same posterior distributions. We report the Peak Signal-To Noise Ratio (PSNR) and the Structural Similarity Index (SSIM) [START_REF] Wang | Mean squared error: Love it or leave it? a new look at signal fidelity measures[END_REF][START_REF] Wang | Image quality assessment: from error visibility to structural similarity[END_REF] for all these experiments. For the interpolation experiments, PnP-SGD struggles to converge when initialized with the observed image (see Section 4.3). For this reason, we warm start PnP-SGD by using an estimate of x obtained by minimizing the Total Variation pseudo-norm under the constraint of the known pixels. For simplicity, PnP-ULA is initialized with the observation y. We observe in Figure 5.11 and Figure 5.12 that the results obtained by computing the MMSE Bayesian estimator with PnP-ULA are visually and quantitatively superior to the ones delivered by MAP estimation with PnP-SGD. In particular, the sampling approach seems to better recover the continuity of fine structures and lines in the different images. For the deblurring experiments, the results of PnP-SGD are provided by using a regularisation parameter α = 0.3 (which was shown to yield optimal results on this set of images in Section 4.3) and for α = 1, which recovers the model used by PnP-ULA. Observe that for the three first images (shown on Figure 5.13), the MMSE result is much sharper than the best MAP result, and the PSNR / SSIM results also show a clear advantage for the MMSE. For the other three images (results are shown on Figure 5.14), the quality of the MMSE solutions delivered is slightly deteriorated by the slow convergence of the Markov chain and the poor regularisation of some specific frequencies, which leads to a common visual artefact (a rotated rectangular pattern). Using a different denoiser more suitable for handling textures, or combining a learnt denoiser with an analytic regularisation term, might correct this behaviour and will be the topic of future work.

A partial conclusion from this set of comparisons is that the sampling approach of PnP-ULA, when it samples the space correctly, seems to provide much better results than the MAP estimator for the same posterior. Of course, this increase in quality comes at the cost of a much higher computation time.

Deblurring and interpolation: uncertainty visualisation study

One of the benefits of sampling from the posterior distribution with PnP-ULA is that we can probe the uncertainty in the delivered solutions. In the following, we present an uncertainty visualisation analysis that is useful for displaying the uncertainty related to image structures of different sizes and located in different regions of the image (see [START_REF] Cai | Uncertainty quantification for radio interferometric imaging -i. proximal MCMC methods[END_REF] for more details). The analysis proceeds as follows. First, Figure 5.4 and Figure 5.15 show the marginal posterior standard deviation associated with each image pixel, as computed by PnP-ULA over all samples, for the interpolation and deblurring problems. As could be expected, we observe for both problems that highly uncertain pixels are concentrated around directions of the space are not explored with the same efficiency. [START_REF] Pereyra | Accelerating proximal Markov chain Monte Carlo by using an explicit stabilized method[END_REF] propose to apply an orthogonal Runge-Kutta-Chebyshev based stochastic approximation presented in [START_REF] Abdulle | Optimal explicit stabilized integrator of weak order 1 for stiff and ergodic stochastic differential equations[END_REF]. This method allows us to take larger step-size δ, which leads to better exploring faculty. It requires s posterior score evaluations at specific points determined by Chebyshev polynomial extrapolation where s belongs to {1, ..., 15}. We recall that Chebyshev polynoms of first order are determined as follows:

∀x ∈ R, T k (x) =    1 if k = 0 X if k = 1 2xT k-1 (x) -T k-2 (x) otherwise.
The Plug & Play SKROCK algorithm is detailed in Algorithm 7. The more we evaluate posterior scores, the larger we can set the discretization step-size. Indeed, we have: δ SKROCK,s = l s δ P nP -U LA , with l s = (s -0.5) 2 (2 -4/3η) and η = 0.05. Table 5.1 compares the different discretization step-sizes useable when applying SKROCK. Applying SKROCK allows us to take way greater step-sizes. However if increasing the number of posterior score evaluations s stabilizes the scheme, what guarantees a faster state space exploration, it also increases the asymptotic bias. In the following, we will take δ = 0.9δ SKROCK,s and let run the the algorithm during N /s iterations with N = 1e7 in order to guarantee the same number of forward pass into the neural network denoiser. It is less iterations than with PnP-ULA but it corresponds to a greater absolute time where the absolute time is defined as follows T = δ × t, with t the current iteration.

Interpolation.

Deblurring. us to apply this algorithm. In addition as SKROCK covers the state-space more rapidly than PnP-ULA, it can also highlight more quickly undesirable properties of the Markov chain. For instance, Figure 5.20 shows the evolution of the Euclidean distance between the original image and the samples generated by SKROCK. After 1e7 neural network applications, the Markov chain is still not ergodic. It highlights the difficulties encountered with the prior plugged into our scheme. This chapter presented theory, methods, and computation algorithms for performing Bayesian inference with Plug & Play priors. This mathematical and computational framework is rooted in the Bayesian M-complete paradigm and adopts the view that Plug & Play models approximate a regularised oracle model. We established clear conditions ensuring that the involved models and quantities of interest are well defined and well posed. Following on from this, we studied two Bayesian computation algorithms related to biased approximations of a Langevin diffusion process, for which we provide detailed convergence guarantees under easily verifiable and realistic conditions. For example, our theory does not require the denoising algorithms representing the prior to be gradient or proximal operators. We also studied the estimation error involved in using these algorithms and models instead of the oracle model, which is decision-theoretically optimal but intractable. To the best of our knowledge, this is the first Bayesian Plug & Play framework with this level of insight and guarantees on the delivered solutions. We illustrated the proposed framework with two Bayesian image restoration experiments -deblurring and interpolation -where we computed point estimates as well as uncertainty visualisation and quantification analyses and highlighted how the limitations of the chosen denoiser manifest in the resulting Bayesian model and estimates. From a Bayesian computation viewpoint, we also test the efficiency of accelerated algorithm with SKROCK [START_REF] Pereyra | Accelerating proximal Markov chain Monte Carlo by using an explicit stabilized method[END_REF] and prove their use although no convergence proofs are currently available. It is still an active field of research.

In future work, we would like to continue our theoretical and empirical investigation of Bayesian Plug & Play models, methods and algorithms. From a modelling viewpoint, it would be interesting to consider priors that combine a denoiser with an analytic regularisation term. It could allow us to promote samples with some desired properties and enrich the prior. Furthermore, it could compensate for a lack of regularization by the neural network.

In the same direction, we would like to consider other neural network based priors. This problematic is at the core of Chapter 6. But we could also consider generative priors as in [START_REF] Bora | Compressed sensing using generative models[END_REF] or autoencoder-based priors as in [START_REF] González | Solving Inverse Problems by Joint Posterior Maximization with Autoencoding Prior[END_REF]. Another field of research concerns the possible generalization to other smoothings and their properties in the context of Bayesian inverse problems. We are also very interested in strategies for training denoisers that automatically verify the conditions required for exponentially fast convergence of the Langevin SDE, for example by using the framework recently proposed in [START_REF] Pesquet | Learning maximally monotone operators for image recovery[END_REF] to learn maximally monotone operators, or the data-driven regularizers described in [START_REF] Kobler | Total deep variation for linear inverse problems[END_REF][START_REF] Mukherjee | Learned convex regularizers for inverse problems[END_REF]. The recent works of (Hurault et al., 2022a,b) also offer interesting perspectives from a theoretical point of view. (Hurault et al., 2022a,b) learn a denoiser which results from the gradient of some known non-convex functional. It could be interesting to see how it affects the convergence results. Besides, with regards to experimental work, we intend to study the application of this framework to uncertainty quantification problems, e.g., in the context of medical imaging. The idea would be to train a denoising neural network on a specific dataset related to the inverse problem considered and to apply SKROCK in order to sample from the posterior distribution and perform detailed uncertainty quantification study.

If PnP-ULA and PPnP-ULA allow to sample from the posterior distribution and consequently to estimate any posterior probabilities, it is interesting to check if these probabilities coincide with the empirical probabilities we can compute. The idea behind is to check if the posterior model is accurate and models the true unavailable model. These questions are at the core of the last chapter of this thesis. In this chapter, we are interested in the role of the prior p and its associated denoiser D ε in sampling algorithms. We focus on deep denoisers trained on a dataset in order to automatically learn a-priori information about the data we deal with. Eventually, we wish to investigate if there are denoisers more likely to model the true albeit unknown posterior model This chapter is motivated by the analysis of the results exposed in Section 5.4 for the deblurring problem, where a meta-stability behaviour was observed for images with highfrequency structures. Some PnP-ULA restorations present artefacts like a grid-pattern. Visualizing the standard deviation of the marginal posterior at each pixel, we note these areas are highly uncertain, meaning that the algorithm is not confident in the proposed structures. The goal of this chapter is to test the efficiency of different denoisers and to infer information about the posterior sampled distribution. To do so, we are going further than simply performing point estimation. A closer look will be given at posterior credible sets or credible regions. They are the regions of the state space in which most of the posterior distribution mass lies [START_REF] Robert | The Bayesian Choice: From Decision-Theoretic Foundations to Computational Implementation[END_REF]. A set C β is a posterior credible region with confidence 1 -β if P (x ∈ C β |y) = 1 -β. For every β ∈ (0, 1), there exist an infinite number of credible regions associated with the confidence level 1 -β. The region C * β that has the minimum volume is called the highest posterior density region (HPD) [START_REF] Robert | The Bayesian Choice: From Decision-Theoretic Foundations to Computational Implementation[END_REF] and is associated with a positive scalar M β such that

C * β = {x ∈ R d , p(x|y) M β } . (6.1)
In the case where the target posterior distribution p(x|y) admits a positive density w.r.t the Lebesgue measure x → exp[-U (x)], the HPD is given by

C * β = {x ∈ R d , U (x) η β } , (6.2)
where η β = log(M β ). The quantity U is generally called the potential.

Consequently, the HPD region with confidence (1-β) is only characterised by the threshold η β . Let us point out that when we want to estimate posterior credible sets and HPD regions we have to solve very high-dimensional integrals of the form C β p(x|y)dx. They will be estimated using Monte-Carlo methods. A first interrogation concerns the presence or not of the original image in the posterior credible sets.

In this chapter, we consider three different denoisers:

• The Spectral Normalized Deep Convolutional Neural Network (SN-DnCNN) [START_REF] Ryu | Plug-and-play methods provably converge with properly trained denoisers[END_REF]: It is the denoiser used in the experiments of Chapters 4 and 5. It is a 17-layers convolutional neural network with ReLU activation functions and batch normalization that learns the residual mapping. It is trained with L 2 -loss so that the residual mapping is contractive, ie with a Lipschitz constant smaller than 1. To do so, at each forward pass of the neural network during the training, the spectral norm of each layer operator is estimated with the power method and then the layers are normalized by their estimated spectral norm.

The code of SN-DnCNN is available at https://github.com/uclaopt/Provable_ Plug_and_Play/.

• The proximal gradient-step denoiser (Prox-GSD) (Hurault et al., 2022b): This denoiser D ε is trained to act like the gradient-step of an explicit functional g ε . We have then for all x ∈ R d D ε (x) = x -∇g ε (x). g ε corresponds to the potential of the approximate prior p ε introduced in Section 5.2. In (Hurault et al., 2022b), g ε is defined such that ∀x ∈ R d , g ε (x) = 1 2 x -N (x, ε) 2 2 with N (., ε) a neural network. This potential was previously introduced in [START_REF] Romano | The little engine that could: Regularization by denoising (red)[END_REF]Bigdeli and Zwicker, 2017). Although considered but disregarded in [START_REF] Romano | The little engine that could: Regularization by denoising (red)[END_REF], it has the advantage to be sufficiently general without requiring too restrictive hypotheses on N (., ε) to get convergent PnP-optimization schemes. In (Hurault et al., 2022b), N (., ε) is a light In order to be differentiable, it has softplus activation functions. Let us point out that this denoiser, contrary to SN-DnCNN takes the noisy image and the noise level map as inputs.

The training is performed in two steps. The first step consists in training the denoiser with the classical L 2 -loss. The second step enforces the non-expansivity of the residual denoiser with the following loss function

L(ε) = E x∼p,nε∼N (0,ε) [ D ε (x + n ε ) -x 2 + µmax( J Id-Dε (x + n ε ) S , 1 -ν)] (6.3)
where S denotes the spectral norm, ν = 0.1 and µ = 0.001.

It is worth mentioning at this point that this denoiser is also the proximal operator of some non-convex potential Φ ε as shown in (Hurault et al., 2022b).

The main advantage of using this denoiser is that it is the gradient of some known potential g ε . Consequently, it would allow us to estimate exact HPD regions or to perform automatic model calibration using maximum likelihood maximization as in [START_REF] Vidal | Maximum likelihood estimation of regularization parameters in high-dimensional inverse problems: An empirical bayesian approach part i: Methodology and experiments[END_REF].

The code of Prox-GSD is available at https://github.com/samuro95/Prox-PnP.

• The Firmly Non-Expansive (FINE) network [START_REF] Pesquet | Learning maximally monotone operators for image recovery[END_REF] This network is inspired by the DnCNN architecture detailed in [START_REF] Zhang | Beyond a gaussian denoiser: Residual learning of deep cnn for image denoising[END_REF]. The batch normalization layers have been removed and the ReLU activation functions have been replaced by LeakyReLUs. This FINE network is incorporated within the framework of Maximally Monotone Operators (MMO). We consider a multivalued operator B defined on a Hilbert space H, B : H → 2 H , with 2 H the family of all subsets of H. 2 H is called the power set of H.The notation B : H → 2 H means that H maps every point x ∈ H to a set Bx ⊂ H. B is characterized by its graph gra B = {(x, u) ∈ H × H|u ∈ Bx}. It concretely means that there is no maximal operator B which contains gra B. For example, if B is not maximal, then there exists (x, u) / ∈ B such that B = B ∪ {(x, u)} is monotone. Furthermore, a multivalued operator B is fully characterized by its resolvent defined as J B = (Id -B) -1 where the inverse corresponds to the inversion of a graph. The authors of [START_REF] Pesquet | Learning maximally monotone operators for image recovery[END_REF] exploit the fact that an operator B is maximally monotone if and only if there exists a non-expansive operator, ie 1-Lipschitz, Q such that J B = 1 2 (Id + Q). In the case where B corresponds to the subdifferential of a certain convex potential g, the resolvent is the proximal operator of g. The authors of [START_REF] Pesquet | Learning maximally monotone operators for image recovery[END_REF] learn a non-expansive operator Q so that the operator J = 1 2 ((Id + Q)) corresponds to the resolvent of a maximally monotone operator B * . Finally, the training loss of this neural network reads .5) with y ε = x + n ε , z ε,ρ = ρx + (1 -ρ) J(x + n ε ), λ = 0.002 and ν = 5e -2.

L = E x∼p,nε∼N (0,ε),ρ∼U [ 0,1] [ J(y ε ) -x 2 2 + λ max{ ∇Q(z ε,ρ ) 2 S , 1 -ν}] ( 6
The code of the FINE denoiser is available at https://github.com/basp-group/ PnP-MMO-imaging.

We consider these three denoisers because they all allow to estimate the Lipschitz constant L of the residual operator D ε -Id.

We did not perform any training and only considered pretrained neural networks. That is why we will show in the following color images for Prox-GSD, which is only trained on a color base.

If not specified, the compact ensuring the strong convexity in the tails is

C = [-1, 2] d .
In a first part, we draw a comparison of the point estimators computed using the different denoisers with PnP-ULA in Section 6.2. Then, we look at results obtained in terms of potential using an approximated potential when we do not know if the denoiser is associated with a potential in Section 6.3. Doing so, we can better understand the images promoted by the plugged prior. Finally, in Section 6.4 we perform a coverage study with the FINE and SN-DnCNN induced priors to see if they are accurate from a frequentist point of view, meaning that they deliver probabilities which are coherent with the true posterior model.

SN-DnCNN

FINE

Prox-GSD √ ε 5/255 2.25/255 15/255 Table 6.1: Noise levels of the different denoisers used within the PnP-ULA framework. These denoiser noise levels were found to achieve the best results from a perceptual and quantitative point of view for the non-blind deblurring inverse problem. In this section, we aim at analysing the results produced by our sampling algorithms from a visual perspective on a non-blind deblurring inverse problem presented in (5.39) with different denoiser induced priors. This section is motivated by the observation of metastable phenomena for certain images such as Goldhill or Alley when dealing with this inverse problem. The proposed MMSE restorations, although achieving good PSNR scores, present a grid-pattern that harms the visual impression. It urges to look for other priors derived from other denoisers. First, we draw a comparison between the results generated with the SN-DnCNN and FINE induced priors. In both cases, the standard deviation of the additive Gaussian white noise is σ = 1/255. The comparison is drawn on Simpson and Goldhill presented in Figure 5.1. Then, we look at the results when the prior score is derived from a Prox-GSD denoiser. In this case, we cannot draw a direct comparison with other denoiser induced priors as the Prox-GSD denoiser only works with RGB images and we did not perform training on grayscale images. In order to speed up the convergence, as an application of Prox-GSD is 4 times slower than for the other two denoisers, we set the additive Gaussian white noise standard deviation to σ = 5/255.

The PnP-ULA discretization step-size is set such that δ = 0.9δ stable where δ stable is defined as in Section 5.4. Table 6.1 gives the different denoiser noise levels applied into the PnP-ULA framework. They correspond to the noise levels achieving the best results from a perceptual and quantitative point of view for the non-blind deblurring inverse problem. Figure 6.4 compares different restorations for the deblurring inverse problem on Simpson and Goldhill. The SN-DnCNN induced posterior distribution produces the MMSE restorations which always reach the best scores in terms of PSNR or SSIM. If these quantitative results are confirmed by the visual impression on Simpson, the MMSE restoration for Goldhill presents an abnormal grid pattern that negatively affects it. This pattern is found after around 5e5 iterations in the PnP-ULA samples and spreads to the whole image along the iterations. It only appears for images with high-frequency structures. At this point the Markov chain generated by PnP-ULA is not ergodic and we should not compute MMSE or standard deviations. The FINE MMSEs never exhibit such pattern for either image. However, their edges look smoother, which explains the lower PSNR and SSIM scores. In addition, for Simpson, an almost piecewise constant picture, PnP-ULA with SN-DnCNN tends to generate more diverse samples. These differences are visible around contours such as clouds. Figure 6.5 shows the standard deviation of the marginal posterior at each pixel computed with PnP-ULA for the SN-DnCNN and FINE denoisers. With FINE, the uncertainty magnitude is lower than with SN-DnCNN. For example, it is 3 and 6.4 times smaller for respectively Simpson and Goldhill. It means that the FINE posterior distribution generate less diverse samples and it could interpret as a distribution concentrated. Furthermore, the grid pattern appearing over time for Goldhill causes uncertainty that propagates even on piecewise constant areas. Besides, uncertainty is more spread out with FINE than with SN-DnCNN. It seems that PnP-ULA with a FINE induced prior is more uncertain on the location of contours whereas with the SN-DnCNN prior uncertainty concerns more the edge magnitude. At this point, it is difficult to explain these differences. If the properties of SN-DnCNN limit less its expressivity in comparison with FINE and could explain the larger variety of the generated samples, we cannot explain the strange structures produced with the SN-DnCNN induced prior. It could be due to the neural network properties or a lack of training. To answer this question, we should retrain both denoisers on the same training set. This is currently under investigation.

We also test PnP-ULA with the Prox-GSD denoiser introduced by (Hurault et al., 2022b) on RGB images. The tackled inverse problem is slightly more difficult as the additive Gaussian white noise has a larger standard deviation σ = 5/255. The goal was to speed-up the convergence of the sampling algorithm as applying the Prox-GSD denoiser is computationally more expensive. Figure 6.6 shows the MMSE and marginal posterior distribution standard deviation at each pixel estimated after 1e7 iterations for Color Simpson. The MMSE is contaminated by packs of exploding pixels organized in lines that seriously ruin the restoration quality. The standard deviation plot reveals that if contours are more uncertain, piecewise constant areas are also corrupted by a lot of noise. Figure 6.7 and Figure 6.8 allow to better understand the phenomenon that deteriorates the estimated MMSE. Figure 6.7 shows samples generated by PnP-ULA with the Prox-GSD denoiser plugged into the prior score. After 5e5 iterations, samples are noisy deblurred solutions to the deblurring inverse problem. After 1.63e6 iterations, the magnitude of some pixels increases and goes outside [0, 1]. This phenomenon arises at an edge, here the boat mast. Then, it steadily diffuses to the whole image and corrupts it. Figure 6.8 shows the cumulative histogram of the pixel values for different samples. The pixel magnitude increases over time, and so does the number of pixels greater than 1 and smaller than 0. In the end, after 1e7 iterations, 20% of the pixels go outside [0, 1].

To alleviate this pixel magnitude increase, we decide to exploit the knowledge we have about the original image that the denoiser does not seem to take into account. We know that it belongs to [0, 1] d . Consequently, we apply P-PnP-ULA with C = [0, 1] d . Adding a projection onto the convex compact C = [0, 1] d after an ULA step constrains the sample to belong to [0, 1] d . Figure 6.9 exposes the estimated MMSE and the marginal posterior standard deviation derived using P-PnP-ULA. After 5e6 iterations for the same inverse problem. The computed MMSE is a much better solution than the MMSE derived with PnP-ULA both from a visual and a quantitative point of view. It indeed achieves a great PSNR score although it is a little smooth. The uncertainty magnitude is reasonable and comparable to the one estimated for Simpson with PnP-ULA and SN-DnCNN. Edges are the most uncertain parts of the restoration.

Samples produced applying PPnP-ULA are presented in Figure 6.10. They appear to be good although noisy, and they do not exhibit any of the default observed with PnP-ULA. We point out here that applying PPnP-ULA allows to pick a larger step-size and consequently to more quickly and better explore the state-space. Whereas after 5e6 iterations, samples generated by PnP-ULA with Prox-GSD were totally corrupted and did not constitute possible solutions to the deblurring inverse problem, samples with PPnP-ULA are far better.

PPnP-ULA does not solve all the problems when plugging Prox-GSD into the prior score. Figure 6.11 shows the empirical first and second order moment of the posterior distribution sampled using PPnP-ULA with Prox-GSD for Fox and computed after 1e6 iterations. Fox n = 1e6, PSNR=24.25 n = 4e6, PSNR=24.13 n = 5e6, PSNR=24.25 is an interesting image for it has high-frequency textures. The estimated MMSE exhibits stripes that severely damages the restoration and makes it a bad solution to the inverse problem we aim at solving. The stripes mask is also perceptible on the standard deviation plot and constitutes the main source of uncertainty in the restoration. It reminds the grid pattern observed for Goldhill, Alley and Bridge when sampling from the posterior with the SN-DnCNN induced prior. As explained in Figure 5.7, high uncertainty was observed for some frequencies in the kernel of the blur operator for the samples generated by PnP-ULA. The denoiser used did not succeed in regularizing these frequencies. Figure 6.12 shows that these stripes come from high-frequency textures present around the fox snout and that rapidly spread over the fox coat over time. After only 1e5 iterations samples are already corrupted by these motifs.

Although we cannot draw a direct comparison between Prox-GSD and the other denoisers previously tested as they only work on grayscale images, we apply PnP-ULA to a grayscale version of Fox. The idea is to see if these denoisers also struggle with this image. Figure 6.13 shows that these two denoisers induced posteriors do not struggle when dealing with this image. It would rather highlight the extreme sensitivity of the Prox-GSD induced posterior when dealing with images with high-frequency textures.

For completeness, we also show the Euclidean distance between the MMSE estimator computed from entire chain (i.e. all samples) and each stored sample (we show one point every 1000 samples) with both denoisers in Figure 6.14. Whereas samples generated with the FINE prior look totally uncorrelated, we observe a light meta-stability behavior for those generated with the SN-DnCNN prior. This could be explained by a very anisotropic posterior distribution induced by this denoiser. In this section we aim at understanding how the results exposed in Section 6.2 are explained in terms of potentials. As a neural network denoiser does not necessarily derive from a potential, we firstly present an approximation of the prior potential introduced in [START_REF] Romano | The little engine that could: Regularization by denoising (red)[END_REF]. It allows us to build a posterior potential U red for the SN-DnCNN and FINE denoisers and to see the posterior potential evolution over time. As Prox-GSD is the gradient of some known potential g ε , we can perform this study with the real potential U . These results are interesting as they allow to see the images promoted by our posterior distribution with our plug-and-play prior.

Fox

Blurry Observation (PSNR=21.69)

MMSE

Standard deviation When we use a classical denoiser, we do not know if there exists such a potential U associated with the posterior distribution. To alleviate this issue, we consider an approximated potential U red introduced by Romano et al. in [START_REF] Romano | The little engine that could: Regularization by denoising (red)[END_REF] and given for any

x ∈ R d by ∀x ∈ R d , U red (x) = Ax -y 2 /(2σ 2 ) + (α/2)x T (x -D ε (x)). (6.6)
This potential is interesting because, if α ← α/ε, its gradient is equal to the gradient of the approximated log-posterior density considered in PnP-ULA and detailed in Algorithm 5. Indeed, if the denoiser D ε is locally homogeneous, non-expansive and its Jacobian J Dε is symmetric † , we have according to [START_REF] Romano | The little engine that could: Regularization by denoising (red)[END_REF] and [START_REF] Reehorst | Regularization by denoising: Clarifications and new interpretations[END_REF] that ∇U red = -∇ log p Y |X (y|x) + α/ε(x -D ε (x)). (6.7) Figure 6.15 shows the evolution of the approximated potential of the samples generated by PnP-ULA with SN-DnCNN or FINE for Simpson and Goldhill on the deblurring inverse problem introduced in Section 6.2. For the FINE induced prior, the original image Simpson is explained by this approached potential as its value is below the potential values of the samples generated by PnP-ULA. It is not the case for Goldhill, as the potential values of the samples is below the potential of the original image. It can be explained by a wrong model-specification. With the SN-DnCNN induced prior, it seems that the original image does not look like an admissible solution as the potential of the original image is higher than the ones of the samples. For Goldhill, it seems that this posterior strongly promotes images presenting a grid-pattern.

The main advantage of using the Prox-GSD prior is that we have access to the posterior potential. We can then have an idea of the types of images that are promoted by this prior. Based on Figure 6.8, we can see that this posterior tends to promote images outside [0, 1] d . The results observed in Figures 6.6 and 6.7 are not due to instabilities in the PnP-ULA • Considering the clean image x i , we generate an observation y i ∼ p(y|x = x i ). The observation process considered is the same as in Section 4.3.5 where the degradation operator A encodes a 9×9 bloc-filter with additive Gaussian white noise with standard deviation σ = 1/255.

• For each observation y i , we sample from the posterior distribution using PnP-ULA over 5e5 iterations. It is initialized at the observation y i . Then, we empirically estimate the posterior credible sets for different levels β using the RED potential U red . The objective is to estimate the regions A β,i such that P(x ∈ A β,i ) = 1 -β, ie the (1 -β)-quantiles of the posterior distribution for β ∈ {0.05, 0.10, 0.15, 0.20}.

• Finally, we check the proportion of clean images that belong to the credible sets of level β. The prior is considered to be true in the frequentist sense, if this proportion is close to 1 -β.

Figure 6.17 shows that the estimated posterior distributions induced by both denoisers are not accurate in the frequentist sense. The observed frequencies do not match the posterior probabilities. Our posterior models are conservative and tend to overestimate uncertainty. It is partially explained by the ill-posedness of the deblurring inverse problem we are dealing with, that makes exploration difficult in some directions of the state-space. It could also mean that the prior is misspecified. In this case, the regularization parameters used are maybe not the best to explain the true model. One way to compensate this could be to calibrate the posterior model by maximum likelihood optimization as in [START_REF] Vidal | Maximum likelihood estimation of regularization parameters in high-dimensional inverse problems: An empirical bayesian approach part i: Methodology and experiments[END_REF]. Finally, it is also possible that our denoisers did not properly learn the structure underlying the natural images, ie the sub-manifold where they live. If it is due to their structures that limit their expressivity or a lack of training is currently under investigation. In this chapter, we investigated the influence of deep priors on the solutions of an ill-posed non-blind deblurring problem. We considered three denoisers with different properties. The comparison was drawn under different scopes. First, we looked at the generated results from a visual point of view. Then, we compared them in terms of potential in order to better understand which properties were promoted with the induced priors. Finally, we checked the frequentist accuracy of the FINE and SN-DnCNN induced priors.

The FINE posterior produces over-smooth samples but does not generate solutions with unrealistic artefacts. On the other hand, the SN-DnCNN posterior generates more diverse samples both from a visual and quantitative point of view in comparison to the FINE posterior. However, it can create artefacts that ruin the restoration for images with more detailed areas. Neither the SN-DnCNN posterior nor the FINE posterior is accurate in a frequentist sense and both tend to be conservative and to overestimate uncertainty. The Prox-GSD posterior really struggles with high-frequency structures.

At this point, we cannot explain these differences as the neural networks were trained on different training sets. For future work, we would like to go further in the prior analysis. First, we would like to train the FINE and SN-DnCNN denoiser on MNIST in order to see if their architectures and properties allow them to learn simple data structures. Then, we would like to train SN-DnCNN on ImageNet as FINE to see if their differences are due to their different properties. This is a major point that needs to be clarified as it could change the denoising neural netowrks we should use within Plug & Play sampling algorithms. Finally, we would like to test frequentist accuracy of different sampling methods such as scorematching and DDRM to better understand from which distribution they sample from.

Conclusion and Perspectives

In this thesis, we paved the way for a better understanding of the Bayesian Plug & Play methods for solving inverse problems. In Chapter 3 we introduced inverse problems and their inherent difficulties. We proposed a short review of the different recent strategies to combine deep neural networks and optimization schemes to solve such inverse problems. Our focus on Plug & Play methods was explained by their simplicity, their versatility, their ability to work with pre-trained networks. The mathematical and computational framework developed in this thesis is rooted in the Bayesian M-complete paradigm and adopts the view that Plug & Play models approximate a regularised oracle model. This is a major difference with the literature on Plug & Play methods that generally postulate and plug a prior assumed to be the true prior.

Chapter 4 exposed our first contribution on posterior maximization in Bayesian imaging with PnP priors. First, it clarified some theoretical major points about MAP estimation with PnP priors. Then it proposed an algorithm for MAP estimation, PnP-SGD (Algorithm 1), with convergence guarantees under realistic and checkable assumptions. For instance, we demonstrated that PnP-SGD converges to points in the vicinity of stationary points of the true posterior even if only an approximated MMSE denoiser is plugged into the SGD scheme. It is a major difference with most works on Plug & Play, which either proves convergence towards fixed points of some operator [START_REF] Sun | An online plug-and-play algorithm for regularized image reconstruction[END_REF][START_REF] Sun | Scalable plug-and-play admm with convergence guarantees[END_REF][START_REF] Ryu | Plug-and-play methods provably converge with properly trained denoisers[END_REF] or towards stationary points of a distribution [START_REF] Xu | Provable Convergence of Plug-and-Play Priors with MMSE denoisers[END_REF]Hurault et al., 2022a,b).

Chapter 5 presented our second contribution on posterior sampling in Bayesian imaging with PnP priors. It focused on MMSE restorations for inverse problems. First, it dealt with fundamental interrogations such as existence, stability and well-posedness of MMSE estimators under clear conditions. Then, two sampling algorithms were proposed, PnP-ULA (Algorithm 5) and PPnP-ULA (Algorithm 6). Convergence guarantees and non-asymptotic error bounds were derived under realistic assumptions. To the best of our knowledge, this is the first Bayesian Plug & Play framework with this level of insight and guarantees on the delivered solutions.

Finally, Chapter 6 investigated the role of the plugged prior when sampling from the posterior. The goal was to interrogate the importance of the denoiser properties and the solutions they promote for a given inverse problem. SN-DnCNN [START_REF] Ryu | Plug-and-play methods provably converge with properly trained denoisers[END_REF] and FINE [START_REF] Pesquet | Learning maximally monotone operators for image recovery[END_REF] induced posteriors were shown to be both over-confident and inaccurate in a frequentist sense, although they deliver excellent MMSE restorations. The differences between the results obtained with both denoisers cannot currently be precisely explained, as they were not trained on the same training set. This is an important question that we plan to investigate further in the future.

During the period of this Phd thesis, several advances have been made in the inverse problem literature both for point estimation and sampling. Among those advances, we can cite neural networks performing point estimation not trained to solve an inverse problem but a whole class of inverse problems. It allows to generalize end-to-end approaches. For example, [START_REF] Zhang | Ffdnet: Toward a fast and flexible solution for cnn-based image denoising[END_REF][START_REF] Zhang | Plug-and-play image restoration with deep denoiser prior[END_REF] proposed denoising neural networks taking a noise map as input in addition to the noisy image. These neural networks adapt to any Gaussian denoising inverse problem. Furthermore, [START_REF] Debarnot | Deep-blur: Blind identification and deblurring with convolutional neural networks[END_REF] proposed a neural network based method to directly estimate a spatially-variant blur kernel. Advances were also proposed in Bayesian sampling with the outbreak of score matching [START_REF] Song | Generative modeling by estimating gradients of the data distribution[END_REF] and DDPM [START_REF] Kawar | Denoising diffusion restoration models[END_REF] based methods. However, we still believe that Plug & Play methods remain relevant for their simplicity and their strong theoretical foundations.

One limitation of Plug & Play methods is their slowness. There are several possible directions to improve this aspect. Although, there are no convergence results available, SKROCK, presented in Section 5.5, takes a first step in this direction. To go further we could think of different discretization schemes of the Langevin SDE, which incorporate second order moment information for instance [START_REF] Panloup | Unajusted langevin algorithm with multiplicative noise: Total variation and wasserstein bounds[END_REF]. Another possibility to speed up the convergence of PnP-ULA that we considered consists in plugging an invertible denoiser such as NFs [START_REF] Lugmayr | Srflow: Learning the superresolution space with normalizing flow[END_REF][START_REF] Gritsenko | On the relationship between normalising flows and variational-and denoising autoencoders[END_REF] or other invertible neural networks [START_REF] Liu | Are deep neural architectures losing information? invertibility is indispensable[END_REF]. We would benefit from the greater stability of implicit schemes and it would allow us to take larger step-sizes. Another possible deep prior could be the generative VAE proposed by [START_REF] González | Solving Inverse Problems by Joint Posterior Maximization with Autoencoding Prior[END_REF]. In this case, the log-posterior is quasi bi-concave and exhibits a more sampling-friendly geometry. It could also be applied for MAP estimation.

In Chapter 6, we saw that the Plug & Play induced posteriors were not accurate. One explanation could be that the parameters (α, ε) were not the best ones explaining the posterior model. [START_REF] Vidal | Maximum likelihood estimation of regularization parameters in high-dimensional inverse problems: An empirical bayesian approach part i: Methodology and experiments[END_REF] propose a method to automatically set the regularization parameters by maximizing the likelihood in an empirical fashion. It is an axis of improvement we consider for future work.

The posterior inaccuracy raises other fundamental questions. Under which conditions can a denoiser accurately model the true posterior ? Or more, generally, can a Plug & Play posterior model accurately represent the posterior distribution ? In the literature, particular attention has been given to enforcing the Lipschitz constant of the denoiser or its residual [START_REF] Ryu | Plug-and-play methods provably converge with properly trained denoisers[END_REF][START_REF] Sun | An online plug-and-play algorithm for regularized image reconstruction[END_REF][START_REF] Xu | Provable Convergence of Plug-and-Play Priors with MMSE denoisers[END_REF][START_REF] Pesquet | Learning maximally monotone operators for image recovery[END_REF]. In Chapter 6 we began to answer these questions from an experimental point of view. Building a (firmly) non-expansive denoiser does not seem to be enough. We plan to go further in this direction in future works.

As explained in Chapter 2, score-matching and DDPM based methods do not come with convergence guarantees and non-asymptotic error bounds but their results are very promising. So do neural networks based methods for sampling. However, the analysis is often limited to sample presentation. We could wish to analyse these distributions from a more statistical perspective and check if they are accurate in a frequentist sense for instance.

We may also wonder how these methods explore state-space in comparison with PnP-ULA. [START_REF] Andrle | Invertible neural networks versus mcmc for posterior reconstruction in grazing incidence x-ray fluorescence[END_REF] recently showed that invertible neural networks for sampling produced equivalent results to MCMC methods but outperformed them in terms of computational time for a specific inverse problem.

From a more practical point of view, we wish to adapt our MAP estimation and sampling methods to more realistic and concrete inverse problems. We thought for example of low-photon imaging problems where the data exhibit statistical properties that cannot be reflected by the Gaussian model. They arise when the number of photons emitted or reflected by an object or scene of interest is measured. They concern numerous areas of imaging science such as emission tomographic imaging [START_REF] Hohage | Inverse problems with poisson data: statistical regularization theory, applications and algorithms[END_REF], fluorescence microscopy [START_REF] Hohage | Inverse problems with poisson data: statistical regularization theory, applications and algorithms[END_REF][START_REF] Bertero | Inverse imaging with poisson data[END_REF], astronomical imaging [START_REF] Hohage | Inverse problems with poisson data: statistical regularization theory, applications and algorithms[END_REF][START_REF] Starck | Astronomical image and data analysis[END_REF], and single-photon light detection and ranging (LIDAR) [START_REF] Altmann | Lidar waveform-based analysis of depth images constructed using sparse single-photon data[END_REF][START_REF] Halimi | Restoration of intensity and depth images constructed using sparse single-photon data[END_REF][START_REF] Rapp | A few photons among many: Unmixing signal and noise for photon-efficient active imaging[END_REF][START_REF] Shin | Photon-efficient computational 3-d and reflectivity imaging with single-photon detectors[END_REF]. Mildly low-photon problems generally have Poisson statistics, whereas more challenging problems exhibit approximately Bernoulli/binomial or geometric data (Altmann et al., 2017a,b). These inverse problems are often associated with severe identifiability issues, poor stability, high uncertainty about the solution and poor regularity conditions. We also thought to tackle problems with additive spatially varying noise, which can be met in medical imaging. In parallel Magnetic Resonance Imaging (pMRI), which is an MRI allowing a faster acquisition time, we cannot assume a stationary noise model [START_REF] Aja-Fernandez | Spatially variant noise estimation in mri: A homomorphic approach[END_REF]. Eventually, in RAW digital photography and satellite imaging, it is common to use a Gaussian noise model in addition to a Poisson noise and to approximate such noise model by a Gaussian distribution with a spatially-varying variance (Aguerrebere, 2014, Chapter 2).

Eventually, the Plug & Play models presented in this thesis approximate a regularised oracle model. The regularization is obtained by convolving the true but inaccessible prior p with a Gaussian kernel G ε . However, we could think of other types of smoothing which might be more adapted to specific types of inverse problems with different measurement noise distributions. We start by considering a slightly more general framework than the one previously introduced. More precisely, instead of p we consider a general distribution p and instead of considering p ε as a prior we consider a tamed version of this density by introducing another hyperparameter α > 0. In what follows, we describe this setting in details. We start by recalling a mild assumption on the likelihood.

H1 For any y

∈ R m , sup x∈R d p(y|x) < +∞, p(y|•) ∈ C 1 (R d , (0, +∞)) and there exists L y > 0 such that ∇ log(p(y|•)) is L y Lipschitz continuous.
For any ε > 0 we recall that p ε is given by the Gaussian smoothing of p with level ε, for any

x ∈ R d by p ε (x) = (2πε) -d/2 R d exp[-x -x 2 /(2ε)] p(x)dx . (B.1)
One typical example of likelihood function that we consider in our numerical illustration, see Section 5.4, is p(y|x) ∝ exp[-Ax -y 2 /(2σ 2 )] for any x ∈ R d with σ > 0 and A ∈ R m×d . Before turning to the analysis of the convergence of the introduced algorithms we state the following proposition which ensures the regularity of the posterior model w.r.t to the observation y.

We consider the following assumption on x → p(y|x) and the prior p for some hyperparameter α > 0 and an observation y ∈ R m .

H7

The following hold:

(dπ/dLeb)(x) = p(y|x)p α (x) R d p(y|x)p α (x)dx . (B.2)
Note that for ease of notation, we do not explicitly highlight the dependency of the posterior distribution π with respect to the hyperparameter α > 0, since it is fixed in the rest of this section. We also consider the family of probability distributions {π ε : ε > 0} given for any ε > 0 and x ∈ R d by

(dπ ε /dLeb)(x) = p(y|x)p α ε (x) R d p(y|x)p α ε (x)dx . (B.3)
We also recall the assumption on the denoiser D ε , see Section 5.3.2 for details.

H2 There exist ε 0 > 0, M R 0 and L 0 such that for any ε ∈ (0,

ε 0 ], x 1 , x 2 ∈ R d and x ∈ B(0, R) we have (Id -D ε )(x 1 ) -(Id -D ε )(x 2 ) L x 1 -x 2 , D ε (x) -D ε (x) M R , (B.4)
where we recall that We now present an improvement on the results of Section 5.3.2 in the case where the loglikelihood x → log p(y|x) is strongly concave. We recall that the Markov chain is given by the following recursion: X 0 ∈ R d and for any k ∈ N

D ε (x 1 ) = R d x g ε (x|x 1 )dx . (B.
X k+1 = X k + δb ε (X k ) + √ 2δZ k+1 , (B.6) b ε (x) = ∇ log p(y|x) + αP ε (x) + (prox λ (ι C )(x) -x)/λ , P ε (x) = (D ε (x) -x)/ε , (B.7)
In the strongly concave setting we set

C = R d , i.e. ∀x ∈ C, prox λ (ι C )(x) = x.
We recall that in our image processing applications, we have that for any

x ∈ R d , p(y|x) ∝ exp[-Ax -y 2 /(2σ 2
)] and that x → p(y|x) is strongly log-concave if and only if A is invertible. This is the case for denoising tasks where A = Id and for deblurring tasks with convolution kernels which have full Fourier support.

We start with the following result which ensures that the Markov chain (B.6) is geometrically ergodic under H4 for the Wasserstein metric W 1 and in V -norm for V :

R d → [1, +∞) given for any x ∈ R d by V (x) = 1 + x 2 . (B.8)
The following proposition is the counterpart of Proposition 5.3.2.

Proposition B.3.1 Assume H1, H7 and H4(R) for some R > 0. Let α > 0 and ε ∈ (0, ε 0 ].

If there exists m > 0 such that log(p(y|•)) is m-concave with m 2αL/ε then there exist A 1 0 and ρ 1 ∈ [0, 1) such that for any δ ∈ (0, δ], x 1 , x 2 ∈ R d and k ∈ N we have

δ x1 R k ε,δ -δ x2 R k ε,δ V A 1 ρ kδ 1 (V 2 (x 1 ) + V 2 (x 2 )) , (B.9) W 1 (δ x1 R k ε,δ , δ x2 R k ε,δ ) A 1 ρ kδ 1 x 1 -x 2 , (B. 10
)
where V is given in (B.8) and δ = m(L y + αL/ε) -2 /2.

Proof: The proof is postponed to Appendix B.6.2.

We recall the assumption on g ε which ensures that x → log(p ε (x)) has Lipschitz gradients.

H6 For any ε > 0, there exists K ε 0 such that for any

x ∈ R d , R d x - R d x g ε (x |x)dx 2 g ε (x|x)dx K ε , (B.11)
with g ε given in (5.13).

The following proposition is the counterpart of Proposition 5.3.3.

Proposition B.3.2 Assume H1, H7, H4(R) for some R > 0 and H6. Moreover, let α > 0, ε ∈ (0, ε 0 ] and assume that R d (1 + x 4 )p α ε (x)dx < +∞. In addition, if there exists m > 0 such that log(p(y|•)) is m-concave with m (2α/ε) max(L, 1+K ε /ε) and δ = m(L y +αL/ε) -2 /2, then for any δ ∈ (0, δ], R ε,δ admits an invariant probability measure π ε,δ and there exists B 1 0 such that for any δ ∈ (0, δ]

π ε,δ -π ε V B 1 (δ 1/2 + M R + exp[-R]) , (B.12)
where V is given in (B.8) and B 1 does not depend on R.

123

Proof: The proof is postponed to Appendix B.6.3.

The bound appearing in (B.12) depends on an extra hyperparameter R > 0 which may be optimized if H2(R) holds for any R > 0 and {M R : R > 0} can be expressed in a closed form. In particular if there exists M ∈ (0, 1) such that for any R > 0, M R = M × R then there exists B 1 0 such that for any δ ∈ (0, δ] and R > 0 B.13) by setting R = log(1/M). Similarly if there exists M > 0 such that for any R > 0, M R = M then there exists B 1 0 such that for any δ ∈ (0, δ] and R > 0 Proposition B.3.3 Assume H1, H7, H2(R) for some R > 0 and 6. Moreover, let α > 0, ε ∈ (0, ε 0 ] and assume that In particular, applying Proposition B.3.3 to the family {h i } d i=1 where for any i ∈ {1, . . . , d}, h i (x) = x i we get that

π ε,δ -π ε V B 1 (δ 1/2 + M log(1/M)) , (
π ε,δ -π ε V B 1 (δ 1/2 + M) , (
R d (1 + x 4 )p α ε (x)dx < +∞. In addition, if there exists m > 0 such that log(p(y|•)) is m-concave with m (2α/ε) max(L, 1+K ε /ε) and δ = m(L y +αL/ε) -2 /2, then there exists C 1,ε 0 such that for any h : R d → R measurable with sup x∈R d {|h(x)| (1 + x 2 ) -1 } 1, n ∈ N * , δ ∈ (0, δ] we have n -1 n k=1 E [h(X k )] - R d h(x)dπ ε (x) C 1,ε (δ 1/2 + M R + exp[-R] + (nδ) -1 )(1 + x 4 ) .
n -1 n k=1 E [X k ] - R d xdπ ε (x) C 1,ε (δ 1/2 + M R + exp[-R] + (nδ) -1 )(1 + x 4 ) , (B.16)
and n -1 n k=1 X k is an approximation of the MMSE given by We consider the following general regularity assumption.

H8 (α)

There exist κ 0, β > 0 and q : R d → (0, +∞) such that R d q(x)dx = 1, q ∞ < +∞ and for almost every

x ∈ R d , R d |p(x) -p(x -x)| q min(1-1/α,0) (x)dx e κ(1+ x 2 ) x β . with B(0, R C ) ⊂ C and R C > 0, there exists C 2,ε,C 0 such that for any h : R d → R measurable with sup x∈R d {|h(x)| (1 + x 2 ) -1 } 1, n ∈ N * , δ ∈ (0, δ] and R > 0 we have n -1 n k=1 E [h(X k )] - R d h(x)dπ(x) C 0 ε β min(α,1)/4 + C 1,ε R -1 C + C 2,ε,C (δ 1/2 + M R + exp[-R] + (nδ) -1 ) (1 + x 4 ) . (B.18) In addition, if there exists m > 0 such that log(p(y|•)) is m-concave with m 2(α/ε) max(L, 1+ K ε /ε) and δ = m(L y + αL/ε) -2 /2, then there exists C 1,ε 0 such that for any h : R d → R measurable with sup x∈R d {|h(x)| (1 + x 2 ) -1 } 1, n ∈ N * , δ ∈ (0, δ] and R > 0 we have n -1 n k=1 E [h(X k )] - R d h(x)dπ(x) C 0 ε β min(α,1)/4 + C 1,ε (δ 1/2 + M R + exp[-R] + (nδ) -1 )(1 + x 4 ) . (B.

19)

Proof: In the general case where log(p(y|•)) is not assumed to be m-concave with m > 0, the proof is completed upon combining Proposition 5.3.4, Proposition B.4.2 and the fact that for any probability distribution ν 1 , ν 2 , ν 1 - where (B t ) t 0 is a d-dimensional Brownian motion and X 0 = x. In this case, (B.20) defines a Markov semi-group (P t ) t 0 for any x ∈ R d and A ∈ B(R d ) by P t (x, A) = P(X t ∈ A) where (X t ) t 0 is the solution of (B.20) with X 0 = x. Consider now the generator of (P t ) t 0 , defined for any f ∈ C The following two lemmas are classical, see for instance (Bortoli et al., 2020, Lemma 18, Lemma 19). We recall these results and their proofs for the sake of completeness. Proof: We divide the proof into two parts.

ν 2 V ν 1 -ν 2 1/2 TV (ν 1 [V 2 ] + ν 2 [V 2 ]) 1/2
(a) Let ∈ N * and γ ∈ (0, γ] with γ = m/(4L 2 ). Let T γ (x) = x -γb(x). In the sequel, for any k ∈ {1, . . . , }, c, ck 0 and λ, λk ∈ [0, 1) are constants independent of γ which may take different values at each appearance. Let ε ∈ (0, 1/2). Using (B.26), the fact that for any a, b 0, (a + b) 2 (1 + ε)a 2 + (1 + ε -1 )b 2 and the fact that for any a, b 0 we have (a + b) 1/2 a 1/2 + b 1/2 , we get that for any x ∈ R d with x (2c/(εm)) 1/2

T γ (x) = x 2 + 2γ b(x), x + γ 2 b(x) Using that λ γ -λ γ/2 -log(1/λ)γλ γ/2 /2, we get that for any γ ∈ (0, γ], R γ satisfies D d (W, λ γ , cγ). We now show that there exist ζ > 0 and β 0 such that (P t ) t 0 satisfies D c (W, ζ, β). First, for any x ∈ R d we have Proof: There exists C c 0 such that for any x ∈ R d and t 0, P t W (x) C c W (x) using (Bortoli and Durmus, 2020, Lemma 25-(b)). Using that for any t 0, (1 -e -t ) -1 1 + 1/t we get that for any γ ∈ (0, γ], x ∈ R d and k ∈ N * We recall that the Markov chain (X k ) k∈N , defined in (B.6), is given by In this section, we prove the convergence of PnP-ULA and control the bias of its invariant measure in the general framework introduced in Appendix B.2 (i.e. α = 1) under two different assumptions on the posterior: either the posterior is log-concave as in Appendix B.3 or the posterior satisfies a more general one-sided Lipschitz condition as in Section 5.3.2. Note that in Section 5.3.2 the results are only stated for α = 1. The statements of the propositions can be generalized to α > 0 by replacing 2λ(L y + L/ε -min(m, 0)) 1 and δ = (1/3)(L y +L/ε+1/λ) -1 by 2λ(L y +αL/ε-min(m, 0)) 1 and δ = (1/3)(L y +αL/ε+1/λ) -1 in Proposition 5.3.2 and 2λ(L y +(a/ε) max(L, 1+K ε /ε)-min(m, 0)) 1 and δ = (1/3)(L y +L/ε+ 1/λ) -1 by 2λ(L y + (α/ε) max(L, 1 + K ε /ε) -min(m, 0)) 1 and δ = (1/3)(L y + αL/ε + 1/λ) -1 in Proposition 5.3. Let R > 0. Let X and Z be random variables with distribution µ and zero mean Gaussian with identity covariance matrix. Let X ε = X + ε 1/2 Z. We recall that the distributions of X and X ε have density with respect to the Lebesgue measure given by p and p ε respectively. In addition, the conditional density of X given X ε is given by g ε . By definition D ε (X ε ) = E[X|X ε ] and therefore we have

X k+1 = X k + δb ε (X k ) + √ 2δZ k+1 , (B.
ε (w † ) = E X -f w † (X ε ) 2 (B.76) = E X -D ε (X ε ) 2 + 2E [ X -D ε (X ε ), D ε (X ε ) -f w † (X ε ) ] + E f w † (X ε ) -D ε (X ε ) 2 (B.77) = E X -D ε (X ε ) 2 + E f w † (X ε ) -D ε (X ε ) 2 = ε + E f w † (X ε ) -D ε (X ε ) 2 .
(B.78)

Combining this result, the condition that ε (w † ) ε +η and the Cauchy-Schwarz inequality we get that

E[ f w † (X ε ) -D ε (X ε ) ] √ η . (B.79)
Since f w † and D ε are locally Lipschitz, there exists C R 0 such that for any x 1 , x 2 ∈ B(0, 2R) we have

| f w † (x 2 ) -D ε (x 2 ) -f w † (x 1 ) -D ε (x 1 ) | C R x 2 -x 1 .
(B.80)

Assume that sup x∈B(0,R) f w † (x) -D ε (x) > η with = (2d + 2) -1 and denote x R ∈ B(0, R) such that we have sup x∈B(0,R) f w † (x) -D ε (x) = f w (x R ) -D ε (x R ) . Using (B.80) we have

E[ f w † (X ε ) -D ε (X ε ) ] B(0,2R)∩B(x R ,C -1 R η ) f w † (x) -D ε (x) p ε (x)dx (B.81) ( f w † (x R ) -D ε (x R ) -η ) B(0,2R)∩B(x R ,C -1 R η ) p ε (x)dx . (B.82)
Combining this result and (B.79) we obtain that

f w † (x R ) -D ε (x R ) η 1/2 B(0,2R)∩B(x R ,C -1 R η ) p ε (x)dx -1
+ η , (B.83)

Setting M R = η 1/2 ( B(0,2R)∩B(x R ,C -1 R η ) p ε (x)dx) -1 +η concludes the first part of the proof. Denote v d the volume of the unit d-dimensional ball. We have that Leb(B(x

R , C -1 R η )) = C -d R η d v d .
Using the Fubini theorem, the Lebesgue differentiation theorem (Bogachev, 2007, Theorem 5.6.2), the dominated convergence theorem and the fact that for η ∈ (0, (C R R) 1/ ], B(0, 2R) We conclude the proof of Proposition 5.3.2 upon using Lemma B. 5.1, Lemma B.5.2, (Bortoli and Durmus, 2020, Corollary 2) with γ ← (4λ) -1 (L y + αL/ε + 1/λ) -2 δ and the fact that for any probability distribution ν 1 , ν 2 , Using the dominated convergence theorem we obtain that log(p ε ) ∈ C ∞ (R d , R). In particular we have for any

∩ B(x R , C -1 R η ) = B(x R , C -1 R η ) we get that lim η→0 Leb(B(x R , C -1 R η )) -1 R d 1 B(x R ,C -1 R η )∩B(x R ,C -1 R η ) (x)p ε (x)dx (B.84) = lim η→0 R d |B(x R , C -1 R η )| -1 (2πε) -d/2 R d 1 B(x R ,C -1 R η ) (x)
ν 1 -ν 2 V ν 1 -ν 2 1/2 TV (ν 1 [V 2 ] + ν 2 [V 2 ])
x ∈ R d ∇ 2 log(p ε (x)) = -ε -1 Id +ε -2 R d (x -x) ⊗2 g ε (x|x)dx -ε -2 R d (x -x)g ε (x|x)dx ⊗2 (B.103) = -ε -1 Id +ε -2 R d x - R d x g ε (x |x)dx ⊗2 g ε (x|x)dx (B.104)
Therefore, using H6 we obtain that for any x ∈ R d we have In what follows we prove Proposition 5.3.3. The proof of Proposition B.3.2 is similar and left to the reader.

∇ 2 log(p ε (x)) 2 ε -1 + ε -2 K ε , (
Proof of Proposition 5.3.3 Let λ > 0 such that 2λ(L y + αL/εm) 1 and δ = (1/3)(L y + αL/ε + 1/λ) -1 . We divide the proof into two parts. First, we show that for any C convex compact with 0 ∈ C there exists B 1,C 0 such that for any δ ∈ (0, δ] and R > 0 with (B t ) t 0 a d-dimensional Brownian motion. bε is Lipschitz continuous using Lemma B.6.2, hence this SDE admits a unique strong solution for any initial condition X 0 with E[ X 0 2 ] < +∞, see (Karatzas and Shreve, 1991, Chapter 5, Theorem 2.9). We denote by (P t,ε ) t 0 the semigroup associated with the strong solutions of (B.110). Similarly to the proof of Proposition B.3.1, replacing (Bortoli and Durmus, 2020, Corollary 2) by (Bortoli and Durmus, 2020, Corollary 22), there exist ÃC 0 and ρC ∈ [0, 1) such that that for any x 1 , x 2 ∈ R d and t 0 ) is a complete metric space and the Picard fixed point theorem we obtain that for any δ ∈ (0, δ] there exist π ε,δ , πε ∈ P 1 (R d ) such that π ε,δ R ε,δ,C = π ε,δ and for any t 0, πε P t,ε = πε . Note that by [START_REF] Roberts | Exponential convergence of langevin distributions and their discrete approximations[END_REF] Finally, using Lemma B.5.1, Lemma B.5.2, the logarithmic Sobolev inequality (Boucheron et al., 2013, Theorem 5.5), the Cauchy-Schwarz inequality and the Markov inequality, there exists Bb 0 such that for any t 0 and x ∈ R d P( X

π ε,δ -πε V B 1,C (δ 1/2 + M R + exp[-R]) , (
δ x1 P t,ε -δ x2 P t,ε V ÃC ρt C (V
δ x R m 1/m,C -δ x P 1,C V 2B b ( √ δ + M R + exp[-R])(1 + x 4 ) exp[Φ(x)] (B.
C c x 2 dπ ε (x) P 1/2 (X / ∈ C) E 1/2 [ X 4 ] E[ X 4 ]R -2 C . (B.133)
with X a random variable with distribution π ε . We conclude using the generalized Pinsker inequality (Durmus and Moulines, 2017, Lemma 24). where is the cumulative distribution function of the univariate Gaussian distribution with zero mean and unit variance. In addition, using that the coupling is absorbing, we have that for any k ∈ N, B.137) Combining this result and (B.135), we get that for any k ∈ N

E 1 ∆ c R d (X k , Y k ) E 1 ∆ c R d (X k/ 1/δ 1/δ , Y k/ 1/δ 1/δ ) , (
δ x1 Q k ε,δ -δ x2 Q k ε,δ TV E 1 ∆ c R d (X k , Y k ) (1 -β) k/ 1/δ . (B.138)
Using that k/ 1/δ kδ/(1 + δ) -1 concludes the proof upon letting ρC = (1 -β) 1/(1+ δ) and ÃC = (1 -β) -1 . B.7.2 Proof of Proposition 5.3.6 Let α, λ > 0, ε ∈ (0, ε 0 ] such that 2λ(L y + αL/ε -min(m, 0)) 1 and δ1 = (1/3)(L y + αL/ε + 1/λ) -1 . Recall that for any

x 1 , x 2 ∈ R d b ε (x 1 ) -b ε (x 2 ) (L y + αL/ε + 1/λ) x 1 -x 2 . (B.139)
Using this result, the fact that for any x ∈ R d , b ε (x), x -m x 2 + c and (Douc et al., 2019, Theorem 19.4.1) there exist δ2 > 0, B 0 and ρ ∈ (0, 1] such that for any δ ∈ (0, δ2 ],

x ∈ R d and k ∈ N

δ x R k ε,δ -π ε,δ V + δ x Q k ε,δ -π C ε,δ V B ρkδ V (x) , (B.140)
with B and ρ which do not depend on R. In addition, using Lemma B.5.1, for any k ∈ N and δ ∈ (0, δ2 ] we have R k ε,δ V (x) λkδ V (x) + cδ , (B.141) with λ ∈ [0, 1) and c > 0 which do not depend on R 0. For any δ ∈ (0, δ2 ] we have

λ δ + cδ λ δ (1 + cδλ -δ2 ) (λ exp[cλ -δ2 ]) δ . (B.142) Let A = λ exp[cλ -δ2 ],
we have that for any x ∈ R d , R ε,δ V (x) A δ V (x). Therefore we get that (V (X n )A -n ) n∈N is a supermartingale. Hence using Doob maximal inequality and Markov inequality we get that P sup k∈{0,...,n} The first part of the proposition is straightforward. Using Pinsker's inequality (Boucheron et al., 2013, Theorem 4.19) For the second part of the proof, since there exist c 1 , > 0 and c 2 ∈ R such that for any x ∈ R d , U (x) c 1 x + c 2 then for any k ∈ N * and α > 0, R d (1 + x ) k p(x) < +∞. Let q(x) = (1+ x ) -(d+1) / R d (1+ x ) -(d+1) dx. Then using that for any t 0, |e t -1| |t| e |t| we get that for any x ∈ R d R d |p(x) -p(x -x)| q 1-1/α (x)dx

X k R V (
C γ x γ exp[C γ x γ ] R d
(1 + x ) (d+1)(1/α-1) p(x)dx First we show the following technical lemma.

Lemma B.8.1 For any x, y 0 and β > 0, (x + y) β -x β 2 β (y β + x (β-1)∧0 y).

Proof: The result is straightforward if β ∈ (0, 1], since in this case (x + y) β

x β + y β . Assume that β > 1. If x = 0 the result holds. Now assume that x > 0. If y x then (x + y) β -x β 2 β y β . Assume that y x. Since f : t → (1 + t) β -1 is convex we obtain that for any t ∈ [0, 1], f(t) 2 β t. Using this result we have (x + y) β -x β x β f (y/x) 2 β x β-1 y , (B.147) which concludes the proof.

Before proving Proposition B.4.2 we state the following lemma.

Lemma B.8.2 Let π 1 , π 2 two probability measures and q 1 , q 2 : R d → [0, +∞) two measurables functions such that for any x ∈ R d , (dπ i /dLeb)(x) = q i (x)/c i with c i = R d q i (x)dx. Denote D = R d |q 1 (x) -q 2 (x)|. We have π 1 -π 2 TV 2c -1 1 D . (B.148)

Proof: We have (B.158) If α < 1 then using that q ∞ < +∞, we get that |p(x) -p(x -x)| q min(1-1/α,0) (x)(2πε)

π 1 -π 2 TV = R d q 1 (x) c 1 - q 2 (x) c 2 dx c -1 1 (D + |c 2 -c 1 |) , (
R d |p(x) -pε (x)| dx 2 α p(y|•) ∞ (1 + q 1/α-1 ∞ )(1 + p (α-1)∧0 ∞ ) × R d |p(x) -p ε (x)| q 1-1/α (x)dx + R d |p(x) -p ε (x)| q 1-1/α (x)dx

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.2 Position du problème . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.3 Contenu . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.4 Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.4.1 Sur l'estimation du Maximum A-Posteriori avec des a-prioris Plug-&-Play pour la descente de gradient stochastique 1.4.2 Méthodes Bayésiennes utilisant des a-prioris Plug & Play: quand Lagevin rencontre Tweedie 1.4.3 Etude approfondie des a-prioris PnP pour l'échantillonnage 1. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.2 Problem statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.3 Outline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.4 Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.4.1 On Maximum-a-Posteriori estimation with Plug & Play priors and stochastic gradient descent 2.4.2 Bayesian imaging using Plug & Play priors: when Langevin meets Tweedie 2.4.3 In-depth study of data-driven priors for sampling 2.4.4 Publications and Preprints 2.4.5 List of presentations 3 Background 3.1 Inverse problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Approaches in imaging science and their regularizers 3.2 Bayesian Approach in imaging science . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3 Learning Approach with Deep Neural Networks in imaging science . . . . . . . . . . . . . . 3.3.1 Neural networks 3.3.2 Learning process with neural networks 3.3.3 Neural networks for point estimation 3.3.4 Neural networks for sampling xi 6.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.2 Qualitative comparison . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.3 Potential analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.4 Coverage ratio analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.5 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.1 Examples of the different types of blur. In both cases the blur kernel is spatially-varying, with a mix of sharp and blurry objects for both images. . . 3.2 Example of imaging inverse problems. . . . . . . . . . . . . . . . . . . . . . . 3.3 Simplified neural network. Each node corresponds to a neuron and each edge corresponds to a weight. Neurons of the same colour are grouped into layers. Image from Wikipedia. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiv 3.4 Structure of a VAE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.1 Dataset (part 1): First three images in our dataset, and examples of degraded images for the three inverse problems considered in this chapter. For denoising, we add a Gaussian noise with variance σ 2 = (30/255) 2 . For deblurring, the operator A correponds to a 9 × 9 uniform blur operator, and we add Gaussian noise with variance σ 2 = (1/255) 2 . For interpolation, we hide 80% of the pixels. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2 Dataset (part 2): Last three images in our dataset, and examples of degraded images for the three inverse problems considered in this chapter. For denoising, we add a Gaussian noise with variance σ 2 = (30/255) 2 . For deblurring, the operator A correponds to a 9 × 9 uniform blur operator, and we add Gaussian noise with variance σ 2 = (1/255) 2 . For interpolation, we hide 80% of the pixels. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.3 Plug & Play denoising for σ 2 = (30/255) 2 with the prior implicit in D ε for ε = (5/255) 2 and different values of the regularization parameter α. This table shows means and standard deviations for PSNR and SSIM values over K=10 independent noise realizations for each of the six images and different values of the regularization parameter α. Initialization plays a very minor role in this case and all algorithms achieve similar (nearly optimal) performance for α = 0.25. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.4 Plug & Play denoising for σ 2 = (30/255) 2 , ε = (5/255) 2 and with α = 0.25. Although the results obtained by the different methods are close from a quantitative point of view, they look for different compromises. For example, PnP-ADMM looks for sharper edges than PnP-SGD but tends to hallucinate structures. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.5 Convergence diagnosis for Plug & Play denoising for σ 2 = (30/255) 2 , ε =

4 . 7

 47 Plug & Play deblurring, for a 9 × 9 kernel, an additive Gaussian noise of standard deviation σ 2 = (1/255) 2 , for ε = (5/255) 2 and for the nearly optimal value of α = 0.3. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xv 4.8 Convergence diagnosis for Plug & Play deblurring with σ 2 = (1/255) 2 , ε = (5/255) 2 , α = 0.3 and TV -L 2 initialization. Left: Evolution of the average PSNR computed for K = 10 independent noise realizations for each of the 6 images. As expected the convergence is slower for the deblurring problem. 4000 iterations seem to be required to enter the stationary phase for all images except Cameraman, that needs on average 1.5e4 iterations. Right: Evolution of the average gradient norm of the log-posterior computed over the 10 experiments for each image. For all images, the gradient norm stabilizes around 0.2. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.9 Interpolation results for the Simpson's image with p = 0.8, σ = 0 each column corresponds to a different initial condition. . . . . . . . . . . . . . . . . . . . 5.1 Original images used for the deblurring and interpolation experiments. . . . . 5.2 Images of Figure 5.1, blurred using a 9 × 9-box-filter operator and corrupted by an additive Gaussian white noise with standard deviation σ = 1/255. . . . 5.3 Images of Figure 5.1, with 80% missing pixels. . . . . . . . . . . . . . . . . . . 5.4 Marginal posterior standard deviation of the unobserved pixels for the interpolation problem. Uncertainty is located around edges and in textured areas. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.5 Evolution of the L 2 distance between the final MMSE estimate and the samples generated by PnP-ULA for the interpolation problem after the burn-in phase. Samples randomly oscillate around the MMSE. It means that they are uncorrelated. For the images Cameraman, Simpson or Bridge, we note a change of range for the L 2 distance. It could be interpreted as a mode switching as our posterior is likely not log-concave. . . . . . . . . . . . . . . . 5.6 ACF for the interpolation problem. The ACF are shown for lags up to 5e5

  Figure 5.3 using PnP-ULA (first row) and PnP-SGD initialized with a TVL2 restoration (second row). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.12 Results comparison for the interpolation task of the images presented in Figure 5.3 using PnP-ULA (first row) and PnP-SGD initialized with a TVL2 restoration (second row). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.13 Results comparison for the deblurring task of the images presented in Figure 5.2 using PnP-ULA with α = 1 (first row), PnP-SGD with α = 0.3 (second row) and α = 1 (third row). PnP-ULA was initialized with the observation y (see Figure 5.2) whereas PnP-SGD was initialised with a TVL2 restoration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.14 Results comparison for the deblurring task of the images presented in Figure 5.2 using PnP-ULA with α = 1 (first row), PnP-SGD with α = 0.3 (second row) and α = 1 (third row). PnP-ULA was initialized with the observation y (see Figure 5.2) whereas PnP-SGD was initialised with a TVL2 restoration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.15 Marginal posterior standard deviation for the deblurring problem. On simple images such as Simpson (see fig. 5.1), most of the uncertainty is located around the edges. For the images Alley, Bridge and Goldhill, associated with a highly correlated Markov chain in some directions, some areas are very uncertain. They correspond to the zones where the rotated rectangular pattern appears in the MMSE estimate. . . . . . . . . . . . . . . . . . . . . . 5.16 Evolution of the Root Mean Squared Error (RMSE) between the final standard deviation and the estimated current standard deviation for the interpolation and deblurring problems. . . . . . . . . . . . . . . . . . . . . . . . . . . 5.17 Marginal posterior standard deviation of the Alley and Simpson images for the interpolation problem at different scales. The scale i corresponds to a downsampling by a factor 2i of the original sample size. . . . . . . . . . . . . 5.18 Marginal posterior standard deviation of the images Alley and Simpson for the deblurring problem at different scales. The scale i corresponds to a downsampling by a factor 2i of the original sample size. . . . . . . . . . . . . . . . xvii 5.19 Results obtained with SKROCK for s = 10 and s = 15 gradient evaluations on a deblurring inverse problem with A a 9×9 bloc filter and with an additive Gaussian white noise with standard deviation σ = 1/255. SKROCK achieves similar results to PnP-ULA in term of MMSE and standard deviation point estimations. In order to draw a fair comparison, we let run the algorithm during n/s iterations with n = 1e7. . . . . . . . . . . . . . . . . . . . . . . . . 5.20 Ergodicity test of the Markov chain generated by SKROCK for the deblurring inverse problem. The 2 plots on the left correspond to ACF computed in the Fourier domain with s = 10 and s = 15. A similar meta-stability behaviour is observed as with PnP-ULA. However, it is less pronounced and the number of posterior score evaluation seems to decrease this phenomenon. The plot on the right corresponds to the evolution of the Euclidean distance between the samples generated by the SKROCK Markov chain and the original image over time δ × t. None of these Markov chains is ergodic. . . . . . . . . . . . .

1. 4 . 1

 41 Sur l'estimation du Maximum A-Posteriori avec des a-prioris Plug-&-Play pour la descente de gradient stochastique 1.4.2 Méthodes Bayésiennes utilisant des a-prioris Plug & Play: quand Lagevin rencontre Tweedie 1.4.3 Etude approfondie des a-prioris PnP pour l'échantillonnage 1.4.4 Publications et Pré-publications 1.4.5 Liste des présentations 1.1 Contexte . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

1. 2

 2 Position du problème . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Figure 1 . 1 :

 11 Figure 1.1: Illustration des limites du paradigme variationnel. Deux solutions possibles à un problème inverse en imagerie médicale délivrent 2 diagnostics différents. En effet, la solution de droite présente une lésion contrairement à celle de gauche. Quelle solution doit-on choisir ? Quelle confiance avons-nous dans la lésion ?Ces questions peuvent être répondues dans le cadre bayésien. Images tirées de[START_REF] Repetti | Scalable bayesian uncertainty quantification in imaging inverse problems via convex optimization[END_REF].

Figure 1 . 2 :

 12 Figure 1.2: Principe d'un DPPM. p θ correspond à la distribution de débruitage que nous apprenons par inférence variationnelle et q à la distribution de bruitage. Image issue de (Kawar et al., 2022).

Figure 1 . 3 :

 13 Figure 1.3: llustration du principe des méthodes d'unrolling. Dans ce cas, le réseau de neurones vise à résoudre un problème inverse de super-résolution via un schéma ADMM linéarisé. Il existe trois modules constitutifs : le module a-priori P qui est appris pendant l'apprentissage, le module relatif aux données D comprend des informations sur le processus d'observation et le module de mise à jour U . Image tirée de (Laroche et al., 2022).

1. 3

 3 Contenu . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

2. 2

 2 Problem statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Figure 2 . 1 :

 21 Figure 2.1: Illustration of the advantage of the Bayesian framework over the variational paradigm. Two possible solutions to an inverse problem in medical imaging leading to two possible different diagnoses. Indeed, the solution on the right presents a lesion whereas the one on the left does not. Which solution should we select ?How confident are we on the lesion ? These questions can be answered in the Bayesian framework. Illustrations from[START_REF] Repetti | Scalable bayesian uncertainty quantification in imaging inverse problems via convex optimization[END_REF].

Figure 2 . 2 :

 22 Figure 2.2: Principle of a DPPM. p θ corresponds to the denoising diffusion distribution we learn by variationalinference and q to the noising process distribution. Image taken from[START_REF] Kawar | Denoising diffusion restoration models[END_REF].

Figure 2 . 3 :

 23 Figure 2.3: Illustration of the deep-unrolling principles. In this case, the neural network aims at solving a super-resolution inverse problem via a linearised ADMM scheme. There are three constitutive modules: the prior module P that is learnt during the training, the data module D which incorporates information about the observation process and the update module U . Image taken from (Laroche et al., 2022).

2. 3

 3 Outline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

•

  R. Laumont, V. D. Bortoli, A. Almansa, J. Delon, A. Durmus, and M. Pereyra. Bayesian imaging using plug & play priors: When langevin meets tweedie. SIAM Journal on Imaging Sciences, 15(2):701-737, 2022. doi: 10.1137/21M1406349. URL https:// doi.org/10.1137/21M1406349 • R. Laumont, V. de Bortoli, A. Almansa, J. Delon, A. Durmus, and M. Pereyra. On maximum-a-posteriori estimation with plug & play priors and stochastic gradient descent. 2021. URL https://hal.archives-ouvertes.fr/hal-03348735/document 2.4.5 List of presentations• R. Laumont, V. D. Bortoli, A. Almansa, J. Delon, A. Durmus, and M. Pereyra. Bayesian imaging using Plug & Play priors. Invited presentation at Centre de Mathématiques et Leurs Applications (CMLA), ENS Paris-Saclay, February 10th 2020. • R. Laumont, V. D. Bortoli, A. Almansa, J. Delon, A. Durmus, and M. Pereyra. Bayesian imaging using Plug & Play priors. Invited presentation at Groupe de Travail des Jeunes Doctorants (GTTJD), Université Paris-Cité, February 14th 2020. • R. Laumont, V. D. Bortoli, A. Almansa, J. Delon, A. Durmus, and M. Pereyra. Bayesian imaging using Plug & Play priors. Invited presentation at GTTI-Centre Borelli, ENS Paris-Saclay, March 10th 2021. • R. Laumont, V. D. Bortoli, A. Almansa, J. Delon, A. Durmus, and M. Pereyra. Bayesian imaging using Plug & Play priors. Invited presentation at Groupe de Travail des Jeunes Doctorants (GTTJD), Université Paris-Cité, May 21rst 2021. • R. Laumont, V. D. Bortoli, A. Almansa, J. Delon, A. Durmus, and M. Pereyra. Bayesian imaging using Plug & Play priors. Invited presentation at 10 eme Biénnale de Mathématiques Appliquées et Industrielles, La Grande Motte, June 24th 2021. • R. Laumont, V. D. Bortoli, A. Almansa, J. Delon, A. Durmus, and M. Pereyra. Bayesian imaging using Plug & Play priors. Invited presentation at Journées du groupe Modélisation Aléatoire et Statistique (MAS) 2020 de la Société de Mathématiques Appliquées et Industrielles (SMAI), Université d'Orléans, August 27th 2021. • R. Laumont, V. D. Bortoli, A. Almansa, J. Delon, A. Durmus, and M. Pereyra. Bayesian imaging using Plug & Play priors. Invited presentation at Congrès des Jeunes Chercheuses et Chercheurs en Mathématiques Appliquées (CJC-MA), Ecole Polytechnique, October 29th 2021. • R. Laumont, V. D. Bortoli, A. Almansa, J. Delon, A. Durmus, and M. Pereyra. Bayesian imaging using Plug & Play priors. Invited presentation at 3 rd IMA Conference on Inverse Problems from Theory to Application, Bayes Center (Scotland), May 5th 2022. • R. Laumont, V. D. Bortoli, A. Almansa, J. Delon, A. Durmus, and M. Pereyra. Bayesian imaging using Plug & Play priors. Invited presentation at Workshop: Imaging With Uncertainty Quantification (IUQ), Konventum Conference Center (Denmark), September 27th 2022. problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Approaches in imaging science and their regularizers 3.2 Bayesian Approach in imaging science . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3 Learning Approach with Deep Neural Networks in imaging science . . . . . . . . . . . . . . 3.3.1 Neural networks 3.3.2 Learning process with neural networks 3.3.3 Neural networks for point estimation 3.3.4 Neural networks for sampling 3.3.5 Limitations of the pure neural network based approaches 3.4 A survey of Plug & Play methods for estimating the MAP in imaging . . . . . . . . . . . . .3.4.1 Plug & Play MAP estimators using proximal splitting 3.4.2 Plug & Play MAP estimators using gradient descent 3.5 Posterior sampling in imaging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.6 A survey of Plug & Play methods for sampling the posterior distribution . . . . . . . . . . . 3.1 Inverse problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Figure 3 . 1 :

 31 Figure 3.1: Examples of the different types of blur. In both cases the blur kernel is spatially-varying, with a mix of sharp and blurry objects for both images.

FigureFigure 3 . 2 :

 32 Figure 3.2: Example of imaging inverse problems.

  Approach in imaging science . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Figure 3 . 3 :

 33 Figure 3.3: Simplified neural network. Each node corresponds to a neuron and each edge corresponds to a weight. Neurons of the same colour are grouped into layers. Image from Wikipedia.

Figure 3 . 4 :

 34 Figure 3.4: Structure of a VAE

  Plug & Play methods for estimating the MAP in imaging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

  sampling in imaging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

  Plug & Play methods for sampling the posterior distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A natural strategy to reconcile the strengths of the Bayesian paradigm and neural networks is provided by Plug & Play approaches. As explained in Section 3.4, Plug & Play approaches seek to derive an approximation of the gradient ∇U (called the Stein score) Bigdeli et al.

4. 1

 1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

4. 2

 2 PnP maximum-a-posteriori estimation: analysis and computation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Proposition 4 . 2 . 3

 423 Assume H1, H2 and H3. Let α > 0 and ε

4. 3

 3 Experimental study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

  .10) with Q an m × d matrix consisting of m random lines from the d × d identity matrix.

Figure 4 . 1 :

 41 Figure 4.1: Dataset (part 1): First three images in our dataset, and examples of degraded images for the three inverse problems considered in this chapter.For denoising, we add a Gaussian noise with variance σ 2 = (30/255) 2 . For deblurring, the operator A correponds to a 9 × 9 uniform blur operator, and we add Gaussian noise with variance σ 2 = (1/255) 2 . For interpolation, we hide 80% of the pixels.

Figure 4 . 2 :

 42 Figure 4.2: Dataset (part 2): Last three images in our dataset, and examples of degraded images for the three inverse problems considered in this chapter.For denoising, we add a Gaussian noise with variance σ 2 = (30/255) 2 . For deblurring, the operator A correponds to a 9 × 9 uniform blur operator, and we add Gaussian noise with variance σ 2 = (1/255) 2 . For interpolation, we hide 80% of the pixels.

Figure 4 . 3 :

 43 Figure 4.3: Plug & Play denoising for σ 2 = (30/255) 2 with the prior implicit in Dε for ε = (5/255) 2 and different values of the regularization parameter α. This table shows means and standard deviations for PSNR and SSIM values over K=10 independent noise realizations for each of the six images and different values of the regularization parameter α. Initialization plays a very minor role in this case and all algorithms achieve similar (nearly optimal) performance for α = 0.25.

Figure 4 .

 4 Figure 4.5 delivers a first convergence diagnosis of PnP-SGD for the denoising task. The evolution of the average PSNR computed for each image over the 10 different experiments suggests that only a thousand of iterations seem to be needed to reach the stationary regime.This impression is confirmed by the evolution of the average gradient norm of the logposterior, as after 1000 iterations, a plateau around 0.4 is reached. The decay after 5000 iterations is due to the forced decay of the δ k as the algorithm has left the burn-in phase during which the discretization step-size was held constant.

Figure 4 . 4 :

 44 Figure 4.4: Plug & Play denoising for σ 2 = (30/255) 2 , ε = (5/255) 2 and with α = 0.25. Although the results obtained by the different methods are close from a quantitative point of view, they look for different compromises. For example, PnP-ADMM looks for sharper edges than PnP-SGD but tends to hallucinate structures.

Figure 4 . 5 :

 45 Figure 4.5: Convergence diagnosis for Plug & Play denoising for σ 2 = (30/255) 2 , ε = (5/255) 2 with α = 0.25and TV -L 2 initialization. Left: Evolution of the average PSNR computed for K = 10 independent noise realizations for each image. A thousand of iterations seem to be sufficient to leave the burn-in phase and enter the stationary phase. The decay of the discretization step-size δ k does not alter the results, which suggests that the algorithm has converged. Right: Evolution of the average gradient norm of the log-posterior computed over the 10 experiments for each image. In less than 500 iterations, it stabilizes around 0.4 for each image. These plots suggest that the algorithm has converged. The decrease observed after 5000 iterations is explained by the decay of the discretization step-size δ k and does not alter the final result.

Figure 4 . 6 :

 46 Figure 4.6: Plug & Play deblurring. Image are blurred with a 9 × 9 uniform kernel, a Gaussian noise of standard deviation σ 2 = (1/255) 2 is added. The denoiser Dε is trained at ε = (5/255) 2 .The plots shows mean and standard deviation values of PSNR and SSIM over K=10 independent noise realizations for each of the six images and different values of the regularization parameter α. Initialization plays a very minor role in this case and all algorithms achieve similar (nearly optimal) performance for α = 0.3, except for FBS which requires a larger (sub-optimal) α to converge.

Figure 4 . 7 :

 47 Figure 4.7: Plug & Play deblurring, for a 9 × 9 kernel, an additive Gaussian noise of standard deviation σ 2 = (1/255) 2 , for ε = (5/255) 2 and for the nearly optimal value of α = 0.3.

Figure 4 . 8 :

 48 Figure 4.8: Convergence diagnosis for Plug & Play deblurring with σ 2 = (1/255) 2 , ε = (5/255) 2 , α = 0.3 and TV -L 2 initialization. Left: Evolution of the average PSNR computed for K = 10 independent noise realizations for each of the 6 images. As expected the convergence is slower for the deblurring problem. 4000iterations seem to be required to enter the stationary phase for all images except Cameraman, that needs on average 1.5e4 iterations. Right: Evolution of the average gradient norm of the log-posterior computed over the 10 experiments for each image. For all images, the gradient norm stabilizes around 0.2.

4. 4

 4 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Figure 4 . 9 :

 49 Figure 4.9: Interpolation results for the Simpson's image with p = 0.8, σ = 0 each column corresponds to a different initial condition.

5. 1

 1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Proposition 5 . 2 . 3

 523 Assume that there exist Φ 1 : R d → [0, +∞) and Φ 2 : R m → [0, +∞) such that for any x ∈ R d and y 1 , y 2 ∈ R m log(p(y 1 |x)) -log(p(y 2 |x))

  the total variation denoted by • TV . Let U be an open set of R d . For any pair of measurable spaces (X, X ) and (Y, Y), measurable function f : (X, X ) → (Y, Y) and measure µ on (X, X ) we denote by f # µ the pushforward measure of µ on (Y, Y) given for any A ∈ Y by f # µ(A) = µ(f -1 (A)). We denote P(R d ) the set of probability measures over (R d , B(R d )) and for any m ∈ N, P m (R d ) = {ν ∈ P(R d ) : R d x m dν(x) < +∞}. We denote by C k (U, R m ) and C k c (U, R m

  .12) For any a ∈ R d and R > 0, denote B(a, R) the open ball centered at a with radius R. Let (X, X ) and (Y, Y) be two measurable spaces. A Markov kernel K is a mapping K : X × Y → [0, 1] such that for any x ∈ X, P(x, •) is a probability measure and for any A ∈ Y, P(•, A) is measurable. For any probability measure µ on (X, X ) and measurable function f : Y → R + we denote µP = X P(x, •)dµ(x) and Pf = Y f (y)P(•, dy). In what follows the Dirac mass at x ∈ R d is denoted by δ x. For any x ∈ R d , we denote τ x : R d → R d the translation operator given for any x ∈ R d by τ x(x ) = x -x. The complement of a set A ⊂ R d , is denoted by A c . All densities are w.r.t. the Lebesgue measure (denoted Leb) unless stated otherwise. For all convex and closed set C ⊂ R d , we define Π C the projection operator onto C w.r.t the Euclidean scalar product on R d . For any matrix a ∈ R d1×d2 with d 1 , d 2 ∈ N, we denote a ∈ R d2×d1 its adjoint.

  a family of i.i.d. Gaussian random variables with zero mean and identity covariance matrix and where Π C is the projection onto C. In what follows, for any δ > 0 and C ⊂ R d closed and convex, we denote by Q ε,δ : R d × B(R d ) → [0, 1] the Markov kernel associated with the recursion (5.34) and given for any x ∈ R d and A

5. 4

 4 Experimental study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Figure 5 . 1 :

 51 Figure 5.1: Original images used for the deblurring and interpolation experiments.

Figure 5 . 2 :

 52 Figure 5.2: Images of Figure 5.1, blurred using a 9 × 9-box-filter operator and corrupted by an additive Gaussian white noise with standard deviation σ = 1/255.

Figure 5 . 3 :

 53 Figure 5.3: Images of Figure 5.1, with 80% missing pixels.

Figure 5 . 4 :

 54 Figure 5.4: Marginal posterior standard deviation of the unobserved pixels for the interpolation problem.Uncertainty is located around edges and in textured areas.

Figure 5 . 5 :

 55 Figure 5.5: Evolution of the L 2 distance between the final MMSE estimate and the samples generated by PnP-ULA for the interpolation problem after the burn-in phase. Samples randomly oscillate around the MMSE. It means that they are uncorrelated. For the images Cameraman, Simpson or Bridge, we note a change of range for the L 2 distance. It could be interpreted as a mode switching as our posterior is likely not log-concave.

Figure 5 . 6 :

 56 Figure 5.6: ACF for the interpolation problem. The ACF are shown for lags up to 5e5 for all images in the pixel domain. After 5e5 iterations, sample pixels are nearly uncorrelated in all spatial directions for the images Traffic, Alley, Bridge and Goldhill. For the images Cameraman and Simpson, in the slowest direction, samples need more iterations to become uncorrelated.

Figure 5 . 7 :

 57 Figure 5.7: Log-standard deviation maps in the Fourier domain for the Markov chains defined by PnP-ULA for the deblurring problem. First line: images Cameraman, Simpson, Traffic. Second line: images Alley, Bridge and Goldhill.For the first three images, we clearly see that uncertainty is observed on frequencies that are near the kernel of the blur filter (shown on the right), and is also higher around high frequencies (i.e. around edges and textured areas in images). For the last three images, very high uncertainty is observed around some specific frequencies. In the direction of these frequencies, the Markov chain is moving very slowly and the mixing time of the chain is particularly slow, as shown on Figure5.9.

Figure 5 . 8 :

 58 Figure 5.8: Evolution of the L 2 distance between the final MMSE estimate and the samples generated by PnP-ULA for the deblurring problem after the burn-in phase. For images as Cameraman or Simpson, samples randomly oscillate around the MMSE. On the contrary, for images as Bridge or Goldhill, the plot is structured, meaning that samples are still correlated.

Figure 5 . 11 :

 511 Figure 5.11: Results comparison for the interpolation task of the images presented in Figure 5.3 using PnP-ULA (first row) and PnP-SGD initialized with a TVL2 restoration (second row).

  /SSIM=0.79 PSNR=26.16/SSIM=0.80 PSNR=26.76/SSIM=0.74 PnP-SGD . PSNR=26.45/SSIM=0.75 PSNR=24.71/SSIM=0.77 PSNR=25.96/SSIM=0.72

Figure 5 . 12 :

 512 Figure 5.12: Results comparison for the interpolation task of the images presented in Figure 5.3 using PnP-ULA (first row) and PnP-SGD initialized with a TVL2 restoration (second row).

Figure 5 . 15 :

 515 Figure 5.15: Marginal posterior standard deviation for the deblurring problem. On simple images such as Simpson (see fig. 5.1), most of the uncertainty is located around the edges. For the images Alley, Bridge and Goldhill, associated with a highly correlated Markov chain in some directions, some areas are very uncertain.They correspond to the zones where the rotated rectangular pattern appears in the MMSE estimate.

Figure 5 . 16 :Figure 5 . 17 :Figure 5 . 20 :

 516517520 Figure 5.16: Evolution of the Root Mean Squared Error (RMSE) between the final standard deviation and the estimated current standard deviation for the interpolation and deblurring problems.

6. 1

 1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Figure 6 . 1 :

 61 Figure 6.1: Architecture of the SN-DnCNN. Figure taken from (Ryu et al., 2019).

Figure 6 . 2 :

 62 Figure 6.2: Architecture of the light DRUNet. Figure taken from(Hurault et al., 2022a)

Figure 6 . 3 :

 63 Figure 6.3: Architecture of the FINE network. Here C corresponds to the number of channels. C = 1 for grayscale images, C = 3 for color images. Figure taken from (Pesquet et al., 2020).

Figure 6 . 4 :

 64 Figure 6.4: MMSE estimates and two samples obtained with PnP-ULA using SN-DnCNN and FINE induced priors for the deblurring problem for Simpson and Goldhill. The blur operator A is a 9 × 9 block-filter and the observation noise is an additive Gaussian white noise with standard deviation σ = 1/255. Samples generated with the FINE priors are less diverse but more robust to instabilities. The Markov chain computed with SN-DnCNN is not ergodic.

Figure 6 . 10 :

 610 Figure 6.10: Samples generated by P-PnP-ULA for Color Simpson for Prox-GSD.

6. 3

 3 Potential analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Figure 6 .

 6 Figure 6.11: MMSE and marginal posterior standard deviation obtained with P-PnP-ULA using the Prox-GSD induced prior for the deblurring problem inverse for Color Fox with the projection set C = [0, 1] d . The blur operator A is 9 × 9 bloc-filter and the observation noise is an additive Gaussian white noise with standard deviation σ = 5/255. A grid pattern on the MMSE ruins the restoration and increases uncertainty.

Figure 6 . 12 :Figure 6 . 14 :

 612614 Figure 6.12: Samples generated by P-PnP-ULA for Fox with Prox-GSD. A quick deterioration of the samples is observed. It seems to come from high-frequency motifs in the fox coat that eventually propagates to the whole image.

Figure 6 . 17 :

 617 Figure 6.17: Coverage ratio analysis for the posterior induced by the SN-DnCNN and the FINE denoisers. Both posterior distributions are not accurate in the frequentist sense as the empirically estimated posterior probabilities do not match the theoretical ones. Both posterior models are over-confident.

6. 5

 5 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

  in Appendix B.6 and Appendix B.7 respectively. Finally, proofs of Appendix B.4 are given in Appendix B.8. B.2 A general framework . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

  (a) R d p(y|x)p α (x)dx < +∞ and for any ε > 0, R d p(y|x)p α ε (x)dx < +∞. (b) R d x 2 p(x)dx < +∞. Note that if α = 1, H7-(a) hold under H1, see Proposition 5.2.1. Under H7-(a), define π the target probability distribution for any x ∈ R d by

  B.14) by letting R → +∞.We now combine Proposition B.3.1 and Proposition B.3.2 in order to control the bias of the Monte Carlo estimator obtained using PnP-ULA. This proposition is the counterpart ofProposition 5.3.4.

  The proof is straightforward upon combining Proposition B.3.1 and Proposition B.3.2.

  . The proof is similar in the case where log(p(y|•)) is m-concave upon replacing Proposition 5.3.4 by Proposition B.3.3. B.5 Technical results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . In this section, we gather technical results which will be used throughout our analysis. Let b ∈ C(R d , R d ) such that for any x ∈ R d , the following Stochastic Differential Equation admits a unique strong solution dX t = b(X t )dt + √ 2dB t , (B.20)

  2 (R d , R) by Af = ∇f, b(x) + ∆f . (B.21)We say that a Markov semi-group (P t ) t 0 on R d × B(R d ) with extended infinitesimal generator (A, D(A)) (see e.g.[START_REF] Meyn | Stability of Markovian processes. III. Foster-Lyapunov criteria for continuous-time processes[END_REF] for the definition of (A, D(A))) satisfies a continuous drift condition D c (W, ζ, β) if there exist ζ > 0, β 0 and a measurable functionW : R d → [1, +∞) with W ∈ D(A) such that for all x ∈ R d AW (x) -ζW (x) + β . (B.22)Similarly, we consider the Markov chain (X k ) k∈N given by the following recursion for anyk ∈ N and x ∈ R d X k+1 = X k + γb(X k ) + 2γZ k , (B.23)with X 0 = x, γ > 0 and {Z k : k ∈ N} a family of i.i.d Gaussian random variables with zero mean and identity covariance matrix. We define its associated Markov kernel R γ :R d × B(R d ) → [0, 1] as follows for any x ∈ R d and A ∈ B(R d ) R γ (x, A) = R d 1 A (x + γb(x) + 2γz) exp[-z 2 /2]dz . (B.24) We say that R γ satisfies a discrete drift condition D d (W, λ, c) if there exist λ ∈ [0, 1), c 0 and a measurable function W : R d → [1, +∞) such that for all x ∈ R d R γ W (x) λW (x) + c . (B.25)

Lemma B. 5 . 1

 51 Assume that there exist L, c 0 and m > 0 such that for anyx 1 , x 2 ∈ R d we have b(x 1 ), x 1 -m x 1 2 + c , b(x 1 ) -b(x 2 ) L x 1 -x 2 . (B.26) Let γ = m/L 2 .Then the following results hold:(a) For any ∈ N * there exist λ ∈ (0, 1], c, β 0 and ζ > 0 such that for any γ ∈ (0, γ], R γ satisfies D d (W, λ γ , cγ) and(P t ) t 0 satisfies D c (W, ζ, β) with W (x) = 1 + x 2 .(b) For any > 0, there exist λ ∈ (0, 1], c, β 0 and ζ > 0 such that for any γ ∈ (0, γ],R γ satisfies D d (W, λ γ , cγ) and (P t ) t 0 satisfies D c (W, ζ, β) with W (x) = exp[ 1 + x 2 ].

2 T 2 ( 2 ( 2 (

 2222 2γm + (1 + ε)γ 2 L 2 ) x 2 + 2γc + (1 + ε -1 )γ 2 b(0) γm + (1 + ε)γ 2 L 2 ) x 2 + (1 + ε -1 )γ 2 b(0) γ((2 -ε)m -(1 + ε)L 2 γ)/2] x + (1 + ε -1/2 )γ b(0) . (B.30) Note that (2 -ε)m -(1 + ε)L 2 γ < 0 since ε ∈ (0, 1/2) and γ = m/L 2 .On the other hand using (B.26) and the fact that for any a, b 0 with a b and e a -e b e a (a -b), we have for anyx ∈ R d with x (2c/(εm)) 1//(εm)) 1/2 exp[γL] -exp[-γ((2 -ε)m -(1 + ε)L 2 γ)/2] + γ b(0) (B.33) exp[-γ((2 -ε)m -(1 + ε)L 2 γ)/2] x + γ(2c/(εm)) 1/2 exp[γL](L + 2m) + γ b(0) . (B.34)Combining (B.27) and (B.34), there exist λ ∈ [0, 1) and c 0 such that for any γ ∈ (0, γ] and x ∈ R d , T γ (x)λ γ x + γc . (B.35)Note that using (B.35), for any k ∈ {1, . . . , 2 } there exist λk ∈ (0, 1) and ck 0 such thatT γ (x) k { λγ k x + γc k } k (B.36) λγk k x k + γ2 k max(c k , 1) k max(γ, 1) k-1 {1 + x k-1 } (B.37) λγ k x k + ck γ{1 + x k-1 } (1 + x k )(1 + ck γ) . (B.38)Therefore, combining (B.36) and the Cauchy-Schwarz inequality we obtain that for any γ ∈ (0, γ] andx ∈ R d R d (1 + y 2 )R γ (x, dy) = 1 + E[( T γ (x) 2 + 2 2γ T γ (x), Z + 2γ Z -k) 2 (3k-)/2 γ (k+ )/2 E[ T γ (x), Z k-Z 2 ] (B.40) 1 + T γ (x) -k) γ (k+ )/2 E[ T γ (x), Z k-Z 2 ]1 {(1,0)} c (k, )

  , the Cauchy-Schwarz inequality and (B.26), we obtain that for anyx ∈ R d AW (x) = ∇W (x), b(x) + ∆W (x)

  R k γ W (x) W (x) + cγ k∈N λ kγ (1 + c(γ + log(1/λ)))W (x) , (B.65) which concludes the proof upon letting C = C c + 1 + c(γ + log(1/λ). Proposition B.5.1 Assume that there exist Φ 1 : R d → [0, +∞) and Φ 2 : R m → [0, +∞) such that for any x ∈ R d and y 1 , y 2∈ R m log(q y1 (x)) -log(q y2 (x)) (Φ 1 (x) + Φ 2 (y 1 ) + Φ 2 (y 2 )) y 1 -y 2 , (B.66)and for any c > 0,R d (1 + Φ 1 (x)) exp[cΦ 1 (x)]p(x)dx < +∞.Then y → π y is locally Lipschitz w.r.t the total variation • TV , where for any x ∈ R d , y ∈ R m we have(dπ y /dLeb)(x) = q y (x)p(x) R d q y (x)p(x)dx .(B.67)Proof: Let y 1 , y 2 ∈ K with K a compact set. Let y 0 ∈ K and D K be the diameter of K. Using Lemma B.8.2 we get thatπ y1 -π y2 TV 2c y1 R d |q y1 (x) -q y2 (x)| p(x)dx , (B.68) with c y1 = R d q y1 (x)p(x)dx.Combining this result with the fact that for any a, b ∈ R we have e a -e b |a -b| max(e a , e b ) we get thatπ y1 -π y2 TV 2c y1 R d |q y1 (x) -q y2 (x)| p(x)dx (B.69) 2c y1 R d (Φ 1 (x) + Φ 2 (y 1 ) + Φ 2 (y 2 )) y 1 -y 2 (B.70) × exp[(2Φ 1 (x) + Φ 2 (y 1 ) + Φ 2 (y 0 ) + Φ 2 (y 2 ))D K ]p(x)dx (B.71)2c y1 (Φ 2 (y 1 ) + Φ 2 (y 2 )) exp[Φ 2 (y 1 ) + Φ 2 (y 0 ) + Φ 2 (y 2 )] Φ 1 (x)) exp[2D K Φ 1 (x)]p(x)dx × y 1 -y 2 ,

  74) b ε (x) = ∇ log(p(y|x)) + α(D ε (x) -x)/ε + (x -Π C (x))/λ , (B.75)where δ > 0 is a stepsize, α, ε, λ > 0 are hyperparameters of the algorithm, C ⊂ R d is a closed convex set with 0 ∈ C, Π C is the projection on C and {Z k : k ∈ N} a family of i.i.d. Gaussian random variables with zero mean and identity covariance matrix.

  3 and Proposition 5.3.4. B.6.1 Proof of Proposition 5.3.1

  exp[-x -x 2 /(2ε)]p(x)dxdx (B.85) = R d (2πε) -d/2 exp[-x R -x 2 /(2ε)]p(x)dxdx = p ε (x R ) > 0 .(B.86)Using this result we have,lim sup η→0 η -M R = 1 + lim sup η→0 η 1/2-(d+1) η d B(0,2R)∩B(x R ,C -1 R η ) p ε (x)dx -1 (B.87) = 1 + C d R v d p -1 ε (x R ) < +∞ , (B.88)which concludes the proof.B.6.2 Proof of Proposition 5.3.2 and Proposition B.3.1We divide this section into two parts. First, we prove the general case where log(p(y|•)) is not assumed to be strongly concave but only satisfying a one-sided Lipschitz condition, i.e. Proposition 5.3.2. Then we turn to the proof of Proposition B.3.1.(a) Let λ > 0 such that 2λ(L y + αL/ε) 1 and δ = (1/3)(L y + αL/ε + 1/λ) -1 . Let C be a compact convex set with 0 ∈ C. Using H2, (B.6) and that Id -Π C is non-expansive we have for anyx 1 , x 2 ∈ R d b ε (x 1 ) -b ε (x 2 ) (L y + αL/ε + 1/λ) x 1 -x 2 . (B.89) Denote R C = sup{ x 1 -x 2 : x 1 , x 2 ∈ C}. Using (B.6), the Cauchy-Schwarz inequality and that 2λ(αL/εm) 1 we have for anyx 1 , x 2 ∈ R d b ε (x 1 ) -b ε (x 2 ), x 1 -x 2 (-m + αL/ε) x 1 -x 2 2 -x 1 -x 2 2 /λ + R C x 1 -x 2 /λ (B.90) -x 1 -x 2 2 /(2λ) + R C x 1 -x 2 /λ . (B.91) Hence, for any x 1 , x 2 ∈ R d with x 1 -x 2 4R C we obtain that b ε (x 1 ) -b ε (x 2 ), x 1 -x 2 -x 1 -x 2 2 /(4λ). We also have that for any x ∈ R d b ε (x), x -x 2 /(4λ) + sup x∈R d (R C /λ + b(0) ) x -x 2 /(4λ) . (B.92)

2 g

 2 B.105) which concludes the first part of the proof. Reciprocally, since x → ∇ log(p ε (x)) is Lipschitzcontinuous with constant K 0 we get that for any basis vector (e i ) i∈{1,...,d} we have that e i ∇ 2 log(p ε (x))e i K. Combining this result with (B.103), we get that ε (x|x)dx Kd + ε -1 d , (B.106) which concludes the proof.

  B.107) with πε given by(dπ ε /dLeb)(x) ∝ exp[-d 2 (x, C)/(2λ)]p(y|x)p α ε (x) , (B.108)Second, we show that there exists B 0 0 such that for anyC convex compact with 0 ∈ C π ε -πε V B 0 diam -1/4 (C) , (B.109)which concludes the proof upon using the triangle inequality. (a) Let C convex compact with 0 ∈ C. We introduce ( Xt ) t 0 solution of the following Stochastic Differential Equation (SDE): X0 = X 0 and d Xt = bε ( Xt )dt + √ 2dB t , (B.110) bε (x) = ∇ log(p(y|x)) + α∇ log(p ε (x)) + prox λ (ι C )(x) , (B.111)

2 ( 2 ( 2 R

 222 , Theorem 2.1) we have for anyx ∈ R d (dπ ε /dLeb)(x) ∝ exp[-d 2 (x, C)/(2λ)]p(y|x)p α ε (x) , (B.114) since prox λ (ι C ) = ∇d 2 (•, C)/(2λ). Let f : R d → R measurable and such that for any x ∈ R d , |f (x)| V (x). Let m ∈ N * such that m δ-1 , x ∈ R d and k ∈ N we have δ x R km ε,1/m [f ]δ x P km km,ε [f ] = m -P 1,ε )P k-j-1,ε [f ] (B.115)Using (B.112), Lemma B.5.1 and Lemma B.5.2 there exists B a 0 such that for any x ∈ R d and k ∈ N we haveδ x P k,ε,C [f ] -πε [f ] B a ρk C V 2 (x) . (B.116) Let T = 1, b 1 (t, (w t ) t∈[0,T ] ) = m-1 j=0 1 [j/m,(j+1)/m) (t)b ε (w jδ ) and b 2 (t, (w t ) t∈[0,T ] = bε (w t ). Let X solution of dX t = b(t, (X t ) t∈[0,1] ) + √ 2B t with X 0 = x with x ∈ R d and b = b 1 , respectively b = b 2 . Note that (X(2)t ) t 0 = ( Xt ) t 0 and (X(1)k/m ) = (X k ) k∈N . For any i ∈ {1, 2}, denote P (i) t the Markov semigroup associated with X (i) t . For any x ∈ R d we have δ x R m ε,1/m,C -δx P 1and the fact that for any a, b 0, (a + b) 2 2(a 2 + b 2 ), we have for any t ∈ [j/m, (j + 1)/m), j ∈ {0, . . . , m -1} and(w t ) t∈[0,1] ∈ C([0, 1] , R d ) b 1 (t, (w t ) t∈[0,1] ) -b 2 (t, (w t ) t∈[0,1] ) 2 = b ε (w j/m ) -bε (w t ) (w j/m ) -b ε (w t ) 2 + 2 bε (w t ) -b ε (w t ) /ε 2 + 4α 2 1 B(0,R) c ( w t )/ε 2 , (B.120)where L b is the Lipschitz constant associated with b ε . In addition using Itô's isometry we have for any t ∈ [j/m, (j + 1)/m)

  125) 48B b ( √ δ + M R + exp[-R]) exp[2Φ(x)] ,(B.126) Combining this result and (B.116) we obtain that for any k ∈ N, j ∈ {0, . . . , k -1}, x ∈ R d and R > 0 we have(δ x R m 1/m,Cδ x P 1,C )P k-j-1,C [f ] B a B b ( √ δ + M R + exp[-R])ρ k-j-1 C exp[2Φ(x)] . (B.127)Using this result, Lemma B.5.1, Lemma B.5.2 and (B.115) we obtain that there exists B c 0 such that for any m ∈ N * with m -1 δπ ε,1/m,C -πε V lim sup k→+∞ δ 0 R km ε,1/m,Cδ 0 P km km,ε,C V B c ( √ δ +M R +exp[-R]) . (B.128)The proof in the general case where δ ∈ (0, δ] is similar and we obtain that there exists B c 0 such that for any δ ∈ (0, δ]π ε,δ -πε V B c ( √ δ + M R + exp[-R]) . (B.129) (b)For any C compact convex with 0 ∈ C we define πε and ρ ε,C such that for anyx ∈ R d ρ ε,C (x) = exp[-d 2 (x, C)/(2λ)]p(y|x)p α ε (x) , (dπ ε /dLeb)(x) = ρ ε,C (x) R d ρ ε,C (x)dx . (B.130) Similarly, define ρ ε and π ε such that for any x ∈ R d ρ ε (x) = p(y|x)p α ε (x) , (dπ ε /dLeb)(x) = ρ ε (x) R d ρ ε (x)dx . (B.131) Since for any x ∈ R d , ρ ε,C (x) ρ ε (x) we get R d ρ ε,C (x)dx R d ρ ε (x)dx. Hence we obtain using the Cauchy-Schwarz inequality and the Markov inequality KL(π ε |π C ) R d log(ρ ε (x)/ρ ε,C (x))dπ ε (x) (B.132)

  Let α, λ, ε, δ > 0, δ ∈ (0, δ] and C ⊂ R d convex and compact with 0 ∈ C. For anyx 1 , x 2 ∈ R d we have b ε (x 1 ) -b ε (x 2 ) (L y + αL/ε) x 1 -x 2 . (B.134) Denote (X n , Y n ) n∈N theMarkov chain obtained using the coupling described in (Bortoli and Durmus, 2020, Section 3) with initial condition (x 1 , x 2 ) ∈ C. Using (Bortoli and Durmus, 2020, Corollary 7-(b)) we get that for any ∈ NE 1 ∆ c R d (X ( +1) 1/δ , Y ( +1) 1/δ ) (1 -β)E 1 ∆ c R d (X 1/δ , Y 1/δ ) , (B.135) where ∆ R d = {(x, x) : x ∈ R d } and β ∈ (0, 1) with β = 2{-(1 + δ)(1 + L y + (αL/ε))diam(C)} ,(B.136)

  x)A nδ exp[-R] . (B.143) Therefore, we get that for any k ∈ N π ε,δ -π C ε,δ TV (V (0) + cδ 2 )A kδ exp[-R] + B ρkδ V (0) . (B.144)We conclude upon letting k = r/(2 log(A)δ) .

  we have for anyx ∈ R d µ -(τ x ) # µ 2 TV 2KL((τ x ) # µ|µ) 2 R d U (x + x) -U (x) dµ(x) 2C γ x γ . (B.145)

  B.149) which concludes the proof using that |c 2 -c 1 | D.We now give the proof of Proposition B.4.2.Proof: Let α > 0. For any ε > 0 and x ∈ R d denote p(x) = p(y|x)p α (x) and pε (x) = (py|x)p α ε (x), where we recall that for anyx ∈ R d p ε (x) = (2πε) -d/2 R d p(x) exp[-x -x 2 /(2ε)]dx . (B.150)For any ε > 0 we haveR d |p(x) -pε (x)| dx p(y|•) ∞ R d |p α (x) -p α ε (x)| dx . (B.151) Using Lemma B.8.1 and that p ε ∞ p ∞ < +∞, we have for any ε > 0 and x ∈ R d R d |p(x) -pε (x)| dx 2 α p(y|•) ∞ (1 + p (α-1) -p ε (x)| dx + R d |p(x) -p ε (x)| α dx . (B.153)Using Jensen's inequality, for any q : R d → (0, +∞) withR d q(x)dx = 1 we have pε (x)| dx 2 α p(y|•) ∞ (1 + p (α-p ε (x)| dx + R d |p(x) -p ε (x)| q 1-1/α (x)dx α . (B.155) If α 1, choosing q such that q ∞ 1 we get R d |p(x) -pε (x)| dx 2 α p(y|•) ∞ (1 + p (α-1p ∈ L 1 (R d ) and {x → (2πε) -d/2 exp[-x 2 /(2ε)] : ε > 0} is afamily of mollifiers, we have lim ε→0 R d |p(x) -p ε | dx = 0. Combining this result, (B.157) and Lemma B.8.2 concludes the first part of the proof. Now let α > 0 and assume H8(α). If α 1 then using (B.152) we have R d |p(x) -pε (x)| dx 2 α (1 + 2 α-1 ) p(y|•) ∞ (1 + p (α-1)∧0 ∞ ) R d |p(x) -p ε (x)| dx .

  ) -p ε (x)| q min(1-1/α,0) (x)dx + R d |p(x) -p ε (x)| q min(1-1/α,0) (x)dx α .(B.160) Using Jensen's inequality and the change of variable x → ε 1/2 x, we have for any ε ∈ (0, (4κ) -1 ] R d |p(x) -p ε (x)| q min(1-1/α,0) (x)dx (B.161) R d R d

  Proof of Proposition 4.2.1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A.2 Proof of Proposition 4.2.2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A.3 Proof of Proposition 4.2.3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Organization of the supplementary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B.2 A general framework . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B.3 Strongly log-concave case . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B.4 Posterior approximation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B.5 Technical results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B.6 Proofs of Section 5.3.2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B.6.1 Proof of Proposition 5.3.1 B.6.2 Proof of Proposition 5.3.2 and Proposition B.3.1 B.6.3 Proof of Proposition 5.3.3 and Proposition B.3.2 B.7 Proofs of Section 5.3.3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B.7.1 Proof of Proposition 5.3.5 B.7.2 Proof of Proposition 5.3.6 B.8 Proofs of Appendix B.4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Illustration des limites du paradigme variationnel. Deux solutions possibles à un problème inverse en imagerie médicale délivrent 2 diagnostics différents.

	A Proofs of Chapter 4
	A.1 B Proofs of Chapter 5 List of Figures B.1 B.8.1 Proof of Proposition B.4.1 1.1
	B.8.2 Proof of Proposition B.4.2
	Perspectives
	xii xiii

  The Prox-GSD induced prior seems to promote images with these unnatural structures as the sample potential is at its lowest after 5e6 iterations and the data fitting term do not penalize this evolution. . . . . . .6.8 Cumulative histograms of the pixel values of different samples generated by PnP-ULA for Color Simpson with the Prox-GSD induced prior. The pixel magnitude of the generated samples tend to increase over time. After 5e6 iterations, at least 30% of the pixels are outside [0, 1]. The Prox-GSD induced prior does not regularize enough the posterior distribution tails. It seems that the neural network did not learn the range of values of the images. . . . . . xviii 6.9 MMSE and marginal posterior standard deviation obtained with P-PnP-ULA using the Prox-GSD induced prior for the deblurring problem inverse for Color Simpson with the projection convex compact set C = [0, 1] d . The blur operator A is 9×9 bloc-filter and the observation noise is an additive Gaussian white noise with standard deviation σ = 5/255. The hard projection onto C allows to alleviate diverging samples. Consequently the MMSE does not expose parasite structures. The log standard deviation is showed as uncertainty is very low. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.10 Samples generated by P-PnP-ULA for Color Simpson for Prox-GSD. . . . . 6.11 MMSE and marginal posterior standard deviation obtained with P-PnP-ULA using the Prox-GSD induced prior for the deblurring problem inverse for Color Fox with the projection set C = [0, 1] d . The blur operator A is 9 × 9 bloc-filter and the observation noise is an additive Gaussian white noise with standard deviation σ = 5/255. A grid pattern on the MMSE ruins the restoration and increases uncertainty. . . . . . . . . . . . . . . . . . . . . . . . 6.12 Samples generated by P-PnP-ULA for Fox with Prox-GSD. A quick deterioration of the samples is observed. It seems to come from high-frequency motifs in the fox coat that eventually propagates to the whole image. . . . . . 6.13 Comparison of the results generated by PnP-ULA with the SN-DnCNN and the FINE induced priors for a grayscale version of Fox after 1e7 iterations. .16 Evolution of the exact posterior potential U for Color Simpson and Fox for the deblurring inverse problem and PnP-ULA or P-PnP-ULA with Prox-GSD. Without the hard projection onto the convex compact set C = [0, 1] d , the posterior induced by the Prox-GSD promotes samples as in Figure 6.7 with diverging pixel values. A hard projection onto C alleviates this issue. However, it still promotes samples with poor-perceptual quality. . . . . . . . 6.17 Coverage ratio analysis for the posterior induced by the SN-DnCNN and the FINE denoisers. Both posterior distributions are not accurate in the frequentist sense as the empirically estimated posterior probabilities do not match the theoretical ones. Both posterior models are over-confident. . . . . . .1 Plug & Play denoising for σ 2 = (30/255) 2 with the prior in D ε for ε = (5/255) 2 . This table shows mean PSNR values over K=10 independent noise realizations for each of the six images. The regularization parameter α = 0.25 is nearly optimal for all algorithms. . . . . . . . . . . . . . . . . . . . . . . . . 4.2 Plug & Play deblurring. Image are blurred with a 9 × 9 uniform kernel, a Gaussian noise of standard deviation σ = 1/255 is added. The denoiser D

6.1 Architecture of the SN-DnCNN. Figure taken from

[START_REF] Ryu | Plug-and-play methods provably converge with properly trained denoisers[END_REF]

. . . . . 6.2 Architecture of the light DRUNet. Figure taken from

(Hurault et al., 2022a) 6.

3 Architecture of the FINE network. Here C corresponds to the number of channels. C = 1 for grayscale images, C = 3 for color images. Figure taken from

[START_REF] Pesquet | Learning maximally monotone operators for image recovery[END_REF]

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.4 MMSE estimates and two samples obtained with PnP-ULA using SN-DnCNN and FINE induced priors for the deblurring problem for Simpson and Goldhill. The blur operator A is a 9 × 9 block-filter and the observation noise is an additive Gaussian white noise with standard deviation σ = 1/255. Samples generated with the FINE priors are less diverse but more robust to instabilities. The Markov chain computed with SN-DnCNN is not ergodic. . . 6.5 Marginal posterior standard deviation computed for the deblurring problem with the SN-DnCNN and FINE induced priors. For the Sn-DnCNN prior uncertainty is more concentrated around edges and higher, whereas for the FINE prior uncertainty is more diffuse but 4 times smaller. . . . . . . . . . . 6.6 MMSE and marginal posterior standard deviation obtained with PnP-ULA using the Prox-GSD induced prior for the deblurring problem for Color Simpson. The blur operator A is a 9 × 9 block-filter and the observation noise is an additive Gaussian white noise with standard deviation σ = 5/255. 6.7 Samples generated by PnP-ULA for Color Simpson and their associated potential values for Prox-GSD. If produced samples look good in a first phase, they deteriorate over time. Abnormal structures appear around highfrequency areas. Both restorations do not exhibit artefacts ruining the visual impression as with Prox-GSD (see Figure 6.11). The SN-DnCNN induced posterior proposes an MMSE restoration with sharper edges and better quantitative results. The associated uncertainty is higher and less spread than with the FINE induced posterior. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.14 Evolution of the L 2 distance between the final MMSE estimate and the samples generated by PnP-ULA with the SN-DnCNN and the FINE induced posterior for the deblurring problem after the burn-in phase for Fox. . . . . 6.15 Evolution of the approximated posterior potential U red for Simpson and Goldhill with the SN-DnCNN and FINE induced priors for the deblurring inverse problem. For the FINE prior, the original image potential is not always below the potential of the generated samples. For Goldhill, the orignal image is not a good solution for this inverse problem in terms of potential. With the SN-DnCNN induced prior, this potential does not regard the original images as a good solution. for Goldhill, it seems to promote images with the grid pattern. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6xix List of Tables 4ε is trained at ε 2 = (5/255) 2 . This table shows mean PSNR values over K=10 independent noise realizations for each of the six images. The regularization parameter α = 0.30 is nearly optimal for all algorithms. . . . . . . . . . . . .

4.3 Interpolation with p = 0.8, σ = 0 with random, TV-L 2 and oracle initialization. Mean and standard deviation of PSNR and SSIM measures computed on K=4 random tests for each of the 6 images. Note the effectiveness of the coarse-to-fine scheme with either random or TV-L 2 initialization: Coarse to fine SGD is only 0.33 dB away from the solution obtained with oracle init, which should be quite close to the global optimum. ADMM is only 0.22 dB away from the solution obtained with oracle init. . . . . . . . . . . . . . . . . 5.1 Largest discretization step-sizes useable for PnP-ULA (without enforcing the strong convexity in the tails), SKROCK with s = 10 and SKROCK with s = 15. SKROCK allows us to take way larger step-size. It makes the the state-space exploration faster and it should guarantee a faster sample decorrelation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.1 Noise levels of the different denoisers used within the PnP-ULA framework. These denoiser noise levels were found to achieve the best results from a perceptual and quantitative point of view for the non-blind deblurring inverse problem. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xx 1 Introduction (en franccais) 1.1 Contexte . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.2 Position du problème . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.3 Contenu . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.4 Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

un point de vue fréquentiste, afin de déterminer si la distribution que nous avons échantillonnée est réaliste et modélise correctement la réalité. Le Chapitre 7 conclut cette thèse et propose des perspectives pour de futures études. 1.4 Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

	problème d'optimisation hautement non convexe. Alors que les approches concurrentes pro-
	duisent d'excellents résultats, elles manquent souvent de preuves de convergence
	Cette thèse est divisée en cinq chapitres.
	Dans le Chapitre 3, nous présentons les principaux concepts apparaissant dans cette thèse.
	Nous introduisons tout d'abord les problèmes inverses et leur nature mal posée qui les rend
	difficiles à résoudre. Après avoir passé en revue les méthodes historiques permettant de
	les résoudre, nous nous concentrons sur les approches Plug & Play pour l'estimation de
	ponctuelle et l'échantillonnage.
	Le Chapitre 4 présente PnP-SGD, un schéma basé sur la descente de gradient qui vise à
	estimer la MAP avec un a priori Plug & Play. Après avoir prouvé l'existence et la stabilité
	de l'estimateur MAP, nous montrons que ce problème est bien posé sous des hypothèses réal-
	istes. PnP-SGD converge vers un point au voisinage de l'ensemble des points stationnaires
	de la distribution a-posteriori.
	Le Chapitre 5 présente deux algorithmes d'échantillonnage avec un a priori Plug & Play,
	PnP-ULA et PPnP-ULA. Les questions théoriques pour le problème d'estimation MMSE
	sont abordées sous des hypothèses réalistes. Ensuite, des résultats de convergence et sur
	les bornes d'erreurs non-asymptotiques sont présentés. Enfin, l'efficacité de ces méthodes
	est prouvée sur des problèmes inverses classiques et une première étude de quantification de
	l'incertitude est effectuée.
	Le Chapitre 6 étudie l'influence des a-prioris profonds sur la distribution a-posteriori
	générée par PnP-ULA. Dans un premier temps, nous étudions les solutions proposées par
	les distributions a-posteriori induites par différents débruiteurs d'un point de vue quantitatif.
	Ensuite, nous cherchons à interpréter ces résultats en termes de potentiel. Cela nous permet
	de mieux comprendre les solutions promues par notre débruiteur. Enfin, nous étudions la
	précision de la distribution échantillonnée d'Dans cette section, nous détaillons les différentes contributions de cette thèse.
	1.4.1 Sur l'estimation du Maximum A-Posteriori avec des a-prioris Plug-&-Play pour
	la descente de gradient stochastique
	Nous abordons la résolution d'un problème inverse en calculant l'estimateur MAP avec un a-
	priori Plug & Play et sous un schéma de descente de gradient stochastique (SGD). Le réseau
	de neurones convolutionnels (Ryu et al., 2019) est utilisé comme a-priori. L'algorithme
	développé est appelé Plug & Play SGD (PnP-SGD). Nous soulignons le fait qu'il s'agit d'un

au Centre de Mathé- matiques et Leurs Applications (CMLA), ENS

  Paris-Saclay, le 10 Février 2020.

	1.4.4 Publications et Pré-publications
	• R. Laumont, V. D. Bortoli, A. Almansa, J. Delon, A. Durmus, and M. Pereyra. Bayesian
	imaging using plug & play priors: When langevin meets tweedie. SIAM Journal on . Dans ce Imaging Sciences, 15(2):701-737, 2022. doi: 10.1137/21M1406349. URL https:// chapitre, nous abordons tout d'abord des questions théoriques importantes liées notamment à l'existence de l'estimateur MAP, sa stabilité et son caractère bien posé. Nous montrons doi.org/10.1137/21M1406349
	qu'il s'agit d'un problème (faiblement) bien posé. De plus, des résultats de convergence sont • R. Laumont, V. de Bortoli, A. Almansa, J. Delon, A. Durmus, and M. Pereyra. On présentés sous des hypothèses réalistes sur le débruiteur utilisé. Asymptotiquement, PnP-maximum-a-posteriori estimation with plug & play priors and stochastic gradient de-SGD converge vers des points dans le voisinage de l'ensemble des points stationnaires de la distribution a-posteriori. Nous rapportons enfin une série d'expériences démontrant l'effi-scent. 2021. URL https://hal.archives-ouvertes.fr/hal-03348735/document
	cacité de PnP-SGD et comparant cet algorithme avec d'autres schémas PnP. Nous montrons que PnP-SGD fournit de bons résultats par rapport aux méthodes PnP. 1.4.5 Liste des présentations
	Ce travail va être publié dans le Journal of Mathematical Imaging and Vision (JMIV) • R. Laumont, V. D. Bortoli, A. Almansa, J. Delon, A. Durmus, and M. Pereyra. dans un numéro spécial. 1.4.2 Méthodes Bayésiennes utilisant des a-prioris Plug & Play: quand Lagevin ren-Bayesian imaging using Plug & Play priors. Présentation • R. Laumont, V. D. Bortoli, A. Almansa, J. Delon, A. Durmus, and M. Pereyra. contre Tweedie Bayesian imaging using Plug & Play priors. Présentation
	Les a-prioris Plug & Play incorporés dans les schémas d'échantillonnage de Monte Carlo ne
	sont pas très courants en imagerie, bien qu'ils commencent à être étudié. À notre connais-
	sance, il n'existe aucune preuve de convergence pour de tels schémas. En plus des problèmes
	de convergence de ces algorithmes, comme pour le problème de l'estimation MAP, d'im-
	portantes questions restent ouvertes quant à savoir si les modèles et estimateurs bayésiens
	sous-jacents sont bien définis, bien posés et possèdent les propriétés de régularité de base
	requises pour effectuer de calculs dans le cadre Bayésien. Ce chapitre développe la théorie de
	l'analyse et du calcul bayésiens avec des a-prioris PnP. Nous présentons l'algorithme Plug &
	Play Unadjusted Langevin Algorithm (PnP-ULA) pour l'échantillonnage de Monte-Carlo et
	l'estimation de l'erreur quadratique moyenne minimale (MMSE). En utilisant des résultats
	récents sur la convergence quantitative des chaînes de Markov, nous établissons des garanties
	de convergence détaillées pour cet algorithme sous des hypothèses réalistes sur les opérateurs
	de débruitage utilisés. Une attention particulière aux débruiteurs basés sur les réseaux de
	neurones profonds est portée. Nous montrons également que ces algorithmes ciblent approx-
	imativement un modèle bayésien optimal en théorie de la décision et bien posé. L'efficacité
	de PnP-ULA est démontrée sur plusieurs problèmes inverses classiques en imagerie tels que
	le déflouttage et l'interpolation.
	Ce travail a été publié dans SIAM Journal on Imaging Science.
	1.4.3 Etude approfondie des a-prioris PnP pour l'échantillonnage
	Si le cadre Bayésien donne accès à la distribution a-posteriori, qui sous-tend toute inférence
	sur le signal que nous souhaitons récupérer à partir des données observées, il est intéressant
	de déterminer quelles solutions sont promues par des débruiteurs aux propriétés différentes.
	En outre, vérifier si le modèle bayésien estimé est significatif d'un point de vue fréquentiste
	est également une question de première importance. En effet, nous cherchons à vérifier si
	le modèle Bayésien est exact et modélise correctement la distribution a-posterior empirique.
	D'un point de vue pratique, nous vérifions si les probabilités calculées avec notre modèle
	bayésien correspondent aux probabilités empiriques. Nous effectuons cette tâche avec trois
	débruiteurs MMSE différents induisant des a-prioris d'image différents.

au Groupe de Travail des Jeunes Doctorants (GTTJD), Université

  Paris-Cité, le 14 Février 2020.

	• R. Laumont, V. D. Bortoli, A. Almansa, J. Delon, A. Durmus, and M. Pereyra.
	Bayesian imaging using Plug & Play priors. Présentation au GTTI-Centre Borelli,
	ENS Paris-Saclay, le 10 Mars 2021.
	• R. Laumont, V. D. Bortoli, A. Almansa, J. Delon, A. Durmus, and M. Pereyra.
	Bayesian imaging using Plug & Play priors. Présentation

au Groupe de Travail des Jeunes Doctorants (GTTJD), Université

	Paris-Cité, le 21 Mai 2021.
	• R. Laumont, V. D. Bortoli, A. Almansa, J. Delon, A. Durmus, and M. Pereyra.
	Bayesian imaging using Plug & Play priors. Présentation à la 10

eme Biénnale de Mathématiques Appliquées et Industrielles, La

  Grande Motte, le 24 Juin 2021.

	• R. Laumont, V. D. Bortoli, A. Almansa, J. Delon, A. Durmus, and M. Pereyra.
	Bayesian imaging using Plug &

Play priors. Présentation à la Journées du groupe Modélisation Aléatoire et Statistique (MAS) 2020 de la Société de Mathé- matiques Appliquées et Industrielles (SMAI), Université

	d'Orléans, le 27 Août
	27 2021.
	• R. Laumont, V. D. Bortoli, A. Almansa, J. Delon, A. Durmus, and M. Pereyra.
	Bayesian imaging using Plug &

Play priors. Présentation au Congrès des Jeunes Chercheuses et Chercheurs en Mathématiques Appliquées (CJC-MA), Ecole

	Polytechnique, le 29 Octobre 2021.
	• R. Laumont, V. D. Bortoli, A. Almansa, J. Delon, A. Durmus, and M. Pereyra.
	Bayesian imaging using Plug & Play priors. Présentation à l' 3

rd IMA Confer- ence on Inverse Problems from Theory to Application, Bayes

  Context . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 2.2 Problem statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 2.3 Outline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 2.4 Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

	2
	Introduction
	2.1 17
	2.4.1 On Maximum-a-Posteriori estimation with Plug & Play priors and stochastic gra-
	dient descent	17
	2.4.2 Bayesian imaging using Plug & Play priors: when Langevin meets Tweedie	17
	2.4.3 In-depth study of data-driven priors for sampling	18
	2.4.4 Publications and Preprints	18
	2.4.5 List of presentations	18
	Center (Ecosse),
	le 5 Mai 2022.
	• R. Laumont, V. D. Bortoli, A. Almansa, J. Delon, A. Durmus, and M. Pereyra.
	Bayesian imaging using Plug & Play priors. Présentation au Workshop: Imaging
	With Uncertainty Quantification (IUQ), Konventum Conference Center (Den-
	mark), le 27 September 2022.

2.1 Context . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

	In several areas of science and industry, there is a need to reliably infer a signal from noisy
	observed data. Typical examples are encountered in biomedical engineering, geophysics,
	astronomy or finance. These problems are called inverse problems because they consist in
	inverting the observation process, to retrieve a signal given observed data. Such problems
	can often be modelled by Equation (2.1)

Table 4 . 1 :

 41 Plug & Play denoising for σ 2 = (30/255) 2 with the prior in Dε for ε = (5/255) 2 . This table shows mean PSNR values over K=10 independent noise realizations for each of the six images. The regularization parameter α = 0.25 is nearly optimal for all algorithms.

SGD (PSNR=26.58 dB, SSIM=0.69) ADMM (PSNR=26.17 dB, SSIM=0.68) BBS (PSNR=26.33 dB, SSIM=0.64) FBS (PSNR=26.31 dB, SSIM=0.67)

Table 4 . 3 :

 43 Interpolation with p = 0.8, σ = 0 with random, TV-L 2 and oracle initialization. Mean and standard deviation of PSNR and SSIM measures computed on K=4 random tests for each of the 6 images. Note the effectiveness of the coarse-to-fine scheme with either random or TV-L 2 initialization: Coarse to fine SGD is only 0.33 dB away from the solution obtained with oracle init, which should be quite close to the global optimum. ADMM is only 0.22 dB away from the solution obtained with oracle init.

	PSNR	SSIM

  .3) and hence x → ∇ log p ε (x|y) is Lipschitz continuous if and only if D ε has this property. We argue that this is a natural assumption on D ε , as it is essentially equivalent to assuming that the denoising problem underpinning D ε is well-posed in the sense of Hadamard (recall that an inverse problem is said to be well posed if its solution is unique and Lipschitz continuous w.r.t to the observation[START_REF] Stuart | Inverse problems: A bayesian perspective[END_REF]). As established in Proposition 5.2.2 below, this happens when the expected MSE involved in using D ε to recover x from x ε ∼ N (x, ε Id), where x has marginal µ, is finite and uniformly upper bounded for all x ε ∈ R d . Assume H1. Let ε > 0. ∇ log p ε is Lipschitz continuous if and only if there exists C 0 such that for any x ε

	Proposition 5.2.2

  1/2 . (B.93) In the following lemma, we show that under H6, ∇ log(p ε ) is Lipschitz continuous. Assume H6. Then for any x 1 , x 2 ∈ R d we have∇ log(p ε (x 1 )) -∇ log(p ε (x 2 )) (1 + K ε /ε) x 1 -x 2 /ε . (B.101)Reciprocally, if there x → ∇ log(p ε (x)) is Lipschitz-continuous then H6.Proof: Let ε > 0. We recall that for any x ∈ R d we havep ε (x) =

	Lemma B.6.2 R d	exp[-x -x	2 /(2ε)]p(x)dx .	(B.102)

  2 (x 1 ) + V 2 (x 2 )) , (B.112) W 1 (δ x1 P t,ε , δ x2 P t,ε ) ÃC ρt C x 1 -x 2 . (B.113) Combining (B.112), Proposition B.3.1, the fact that (P 1 (R d ), W 1

  = tm and Φ(x) = 1 + x 2 . Combining this result, (B.120), (B.117), (B.121) and Lemma B.6.1, we obtain that there exists B b 0 such that for any x ∈ R d and R > 0

	t	1)	R) exp[-2R]E exp[2 X	(1) t	(B.122)
			exp[-2R]E 1/2 exp 4 √	2	t t/m dB t	E 1/2 [exp[4 X t ]	(B.123)
			Bb exp[-2R] exp[2Φ(x)] ,	(B.124)
	where t

* There also exist other types of inverse problems where the noise n is not additive but multiplicative for instance. However, they are out of the scope of this thesis. See(Aguerrebere, 2014, Chapter 2) or[START_REF] Dunlop | Multiplicative noise in bayesian inverse problems: Well-posedness and consistency of map estimators[END_REF] for example.

‡ In[START_REF] Sun | Scalable plug-and-play admm with convergence guarantees[END_REF], the residual Id -Dε is assumed to be firmly non expansive, which is equivalent to say that Dε is firmly non expansive, see(Bauschke et al., 2011, Proposition 4.4).

* Strictly speaking, the true likelihood p (y|x) may also be unknown, this is particularly relevant in the case of blind or myopic inverse imaging problems. For simplicity, we restrict our experiments and theoretical development to the case where p(y|x) represents the true likelihood. Generalizations of our approach to the blind or semi-blind setting are discussed, e.g. by[START_REF] Guo | Agem: Solving linear inverse problems via deep priors and sampling[END_REF] -formalising these generalisations is an

‡ Geometric convergence is highly desirable property in large-scale problems and guarantees that the generated Markov chains can be used for Monte Carlo integration.

§ The slowest direction corresponds to the Fourier coefficient with the highest (real or imaginary) variance.

x -y, Bx -By . This assumption is stronger than ensuring the residual non-expansivity.

† In[START_REF] Reehorst | Regularization by denoising: Clarifications and new interpretations[END_REF], it is proved that if the Jacobian symmetry assumption does not hold, then there does not exist any potential U red associated with (6.7). Furthermore the authors conducted a study to know for which denoisers this assumption holds. Interestingly, it is proved that for classical denoisers such as BM3D[START_REF] Dabov | Image denoising with block-matching and 3d filtering[END_REF], Non Local Means (NLM)(Buades et al., 2005a), TNRD[START_REF] Chen | Trainable Nonlinear Reaction Diffusion: A Flexible Framework for Fast and Effective Image Restoration[END_REF] or DnCNN[START_REF] Zhang | Beyond a gaussian denoiser: Residual learning of deep cnn for image denoising[END_REF], the assumption on the Jacobian symmetry does not hold.

Remerciements

Algorithm 5 PnP-ULA Require: n ∈ N, y ∈ R m , ε, λ, α, δ > 0, C ⊂ R d convex and compact Ensure: 2λ(2L y + αL/ε) 1 and δ < (1/3)(L y + 1/λ + αL/ε) -1

Initialization: Set X 0 ∈ R d and k = 0. for k = 0 : N do Z k+1 ∼ N (0, Id)

and compact

Initialization: Set X 0 ∈ C and k = 0. for k = 0 : N do Z k+1 ∼ N (0, Id)

set the grounds for analysing more advanced stochastic simulation and optimisation schemes for performing Bayesian inference with Plug & Play priors, in particular accelerated optimisation and sampling algorithms [START_REF] Pereyra | Accelerating proximal Markov chain Monte Carlo by using an explicit stabilized method[END_REF]. This is an important perspective for future work. In this section, we provide a theoretical study of the long-time behaviour of PnP-ULA, see Algorithm 5 and PPnP-ULA, see Algorithm 6. For any ε > 0 we recall that p ε is given by the Gaussian smoothing of p with level ε, for any x ∈ R d by

R d exp[-x -x 2 /(2ε)] p (x)dx .

(5.9)

One typical example of likelihood function that we consider in our numerical illustration, see Section 5.4, is p(y|x) ∝ exp[-Ax -y 2 /(2σ 2 )] for any x ∈ R d with σ > 0 and A ∈ R m×d . We define π the target posterior distribution given for any x ∈ R d by (dπ/dLeb)(x) = p (x|y). We also consider the family of probability distributions {π ε : ε > 0} given for any ε > 0 and x ∈ R d by

(5.10)

Note that in the supplementary material Appendix B we investigate the general setting where p ε is replaced by (p ε ) α for some α > 0 that acts as a regularisation parameter. We divide our study into two parts. We recall that π ε is well-defined for any ε > 0 under H1, see Proposition 5.2.1. We start with some notation in Section 5.3.1. We then establish non-asymptotic bounds between the iterates of PnP-ULA and π ε with respect to the total variation distance for any ε > 0, in Section 5.3.2. Finally, in Section 5.3.3 we establish similar results for PPnP-ULA.

PnP-ULA, δ = δ the edges of the reconstructed images, but also on textured areas. The dynamic range of the pixel standard deviations is larger for the interpolation problem than for deblurring, which suggests that the problem has a higher level of intrinsic uncertainty. Figure 5.16 shows the evolution of the RMSE between the standard deviation computed along the samples and its asymptotic value, respectively for the interpolation and deblurring problems. Estimating these standard deviation maps necessitates to run the chain longer than to estimate the MMSE, as could be expected for second order statistical moment.

Following on from this, to explore the uncertainty for structures that are larger than one pixel, Figure 5.17 and Figure 5.18 report the marginal standard deviation associated with higher scales. More precisely, for different values of the scale i, we downsample the stored samples by a factor 2i before computing the standard deviation. This downsampling step permits quantifying the uncertainty of larger or lower-frequency structures, such as the bottom of the glass in Simpson for the deblurring experiment. At each scale, we see that the uncertainty of the estimate is much more localized for the interpolation problem (resulting in higher uncertainty values in some specific regions) and more spread out for deblurring, PnP-ULA, α = 1. If PnP-ULA shows interesting results for point estimation, we saw previously that it can have some difficulties to correctly explore the state space. In addition, samples generated by the PnP-ULA Markov chain can be highly correlated, which should be avoided if we want accurate Monte-Carlo estimates Figure 5.9. These problems can be partially explained by a too coarse discretization scheme of the Langevin SDE. If the Euler-Maruyama scheme is straightforward to apply, it does not take into account the geometry of the posterior distribution. Indeed, the discretization step-size δ is the same in all the directions of the space and is only determined by the Lipschitz constant of the posterior score. If the posterior distribution is very anisotropic, then all the Scale 1 Scale 2

Scale 3 Scale 4 Algorithm 7 SK-ROCK In the following, we tackle the non-blind deblurring inverse problem presented in Sections 4.3 and 5.4 on Goldhill. We are particularly interested in this image as PnP-ULA struggles to properly restore this picture and does not generate an ergodic Markov chain.

The point estimation results obtained with SKROCK are very similar to the ones computed with PnP-ULA as we can see on Figure 5.19. A grid pattern, ruining the visual impression, is still present for the MMSEs. The marginal posterior standard deviations estimated with SKROCK also suffer from high uncertainty on piecewise constant areas. It is due to this grid pattern that appears around high-frequency structures and which spreads out. We point out that the uncertainty magnitude is higher when using SKROCK instead of PnP-ULA. The state-space exploration is actually quicker. Then, the higher magnitude could be explained by the exploration of an area of the state-space unseen by PnP-ULA.

Figure 5.20 shows the ACF plots of the SKROCK Markov chains with s = 10 and 15 computed in the Fourier domain as for PnP-ULA. It is interesting to note that the meta-stable behaviour of the Markov chain observed with PnP-ULA is attenuated. Although, we do not have any convergence guarantees as for PnP-ULA, it is a positive side-effect that should urge The Prox-GSD induced prior seems to promote images with these unnatural structures as the sample potential is at its lowest after 5e6 iterations and the data fitting term do not penalize this evolution. For the FINE prior, the original image potential is not always below the potential of the generated samples. For Goldhill, the orignal image is not a good solution for this inverse problem in terms of potential. With the SN-DnCNN induced prior, this potential does not regard the original images as a good solution. for Goldhill, it seems to promote images with the grid pattern.

Color Simpson (PnP-ULA)

Color Simpson (P-PnP-ULA) Fox (P-PnP-ULA) scheme but by the nature of the prior. Although this denoiser was trained with images in [0, 1] d , it firstly seems that it has not learnt the image range of values. Consequently, it does not regularize enough the posterior tails and does not sufficiently constrain the mass probability inside [0, 1] d . The hard projection onto [0, 1] d allows to alleviate this issue (even if it adds some bias). Adding this constraint, the original image appears to be a good solution to our inverse problem for color Simpson, although the sample potentials are higher. Even if we constrain the samples to belong to [0, 1] d , it does not necessarily produce good-looking restorations. Fox is a good example of this phenomenon. Of course, the original image is now an admissible solution to the inverse problem as we enforce the posterior probability mass to belong to [0, 1] d . However the posterior still struggles when dealing with high-frequency structures as seen in Figure 6.12 and none of the samples generated appears to be a good solution to the inverse problem considered. If the FINE and SN-DnCNN priors produce good restorations, it is interesting to question the quality of these priors from a frequentist point of view. The posterior probabilities computed are true only in the paradigm defined by the model. The aim of this section is then to check if they also are meaningful in a frequentist sense. This part is very much inspired by the work [START_REF] Holden | Bayesian imaging with data-driven priors encoded by neural networks[END_REF]. As explained in [START_REF] Holden | Bayesian imaging with data-driven priors encoded by neural networks[END_REF], in a frequentist approach, the posterior probabilities should match the observed frequency over a large number of trials. It means that for m observations of the modelled quantities {x 1 , x 2 , ..., x m }, we should have if m is sufficiently large

6.4

x|y)p(y)dy

In this section we test the coverage of the credible intervals derived by the posterior model induced by the SN-DnCNN and the FINE denoisers. To do so, we consider a dataset D = {x i } m i=1 of clean images resulting from the concatenation of the datasets set3c, CBSD10, CBSD68 and proceed as follows: Proof: Let K ⊂ R d be a compact set and (x n , ε n ) n∈N such that lim n→+∞ ε n = 0 and for any n ∈ N, x n ∈ S εn,K . Let x ∈ S a cluster point of (x n ) n∈N . Hence, for any n ∈ N * there exist an increasing sequence

In what follows, we show that lim n→+∞ ∇ log(p ε kn (x kn )) = ∇ log p(x ). First, we show that lim

where we recall that for any x ∈ R d , G ε (x) is a Gaussian kernel with variance ε. For ease of notation, we define G = G 1 . Let η > 0. Then, there exists R > 0 such that for any ε > 0 we have

We have that K is compact and therefore f is uniformly continuous on K . Hence there exists ξ > 0 such that for any x ∈ K, ε ∈ (0, ξ] and y ∈ B(0, R) we have

Hence, combining (A.3) and (A.4) we get that for any x ∈ K and ε ∈ (0, ξ]

Combining this result, the fact that lim n→+∞ x kn = x and that p > 0, we get that lim n→+∞ ∇ log(p ε kn )(x kn ) = ∇ log p(x ). Indeed, we have that for any n ∈ N

We conclude using (A.9) and that log p ∈ C(R d , R). Finally, we obtain that

Hence, x ∈ S K . Proof: First, using that p ∈ C(R d , R + ) we have that for any v ∈ R d and c ∈ R there exists A ∈ B(R d ) such that A | x, v -c| p(x)dx > 0, meaning that there is no lower-dimensional affine space of R d to which x belongs almost surely. Hence, we can apply (Gribonval, 2011, Lemma II.1) and

A.2 Proof of

We have that for any x ∈ R d and y ∈ R m , ∇ log p ε (x|y) = ∇ log p(y|x)+D ε (x). Hence, we get that (x, y)

We have that ∇ x (∇ x log p ε )(x y0 |y 0 ) is invertible. Therefore using the implicit function theorem, there exist V 0 ⊂ R m open and x ∈ C 1 (V 0 , U 1 ) such that for any y ∈ V 0 , ∇ log p ε (x (y)|y) = 0, i.e. x (y) is a strict local maximizer of x → log p ε (x|y), since -∇ 2 log p ε (x (y)|y) is positive, which concludes the proof.

A.3 Proof of

Proof:

In order to prove this theorem we are going to apply (Tadić et al., 2017, Theorem 2.1). In particular, in order to follow the notation of (Tadić et al., 2017, Theorem 2.1), we

Hence, we obtain that [START_REF] Tadić | Asymptotic bias of stochastic gradient search[END_REF], Assumption 2.1, Assumption 2.2) are satisfied.

In what follows, we show that (Tadić et al., 2017, Assumption 2.3.c) holds. We have that for any

, where * denotes the convolution product. Since

Hence, p * G ε ∈ L 1 (R d ) and we obtain that for almost every

In the rest of the proof, we denote pε :

We have that pε is analytic using the dominated convergence theorem. Since for any

We conclude using (Tadić et al., 2017, Theorem 2.1). In this supplementary chapter we present some extensions and gather the proofs related to Chapter 5 for completeness. The main author of this chapter is Valentin de Bortoli. In this chapter, we first introduce a more general framework in Appendix B.2. Then in Appendix B.3 we present our improved convergence results in the case where the loglikelihood is strongly log-concave. Posterior approximation bounds in our general setting are gathered in Appendix B.4. Then we turn to the proof of these results. We first derive technical results in Appendix B.5. Proofs of Section 5.3.2 and Section 5.3.3 are presented In the case where α 1, H8(α) is equivalent to the following assumption: there exist κ 0 and β > 0 such that for almost every

B.1 Organization

β , where we recall that µ is the probability distribution with density with respect to the Lebesgue measure proportional to p and that for any x ∈ R d , τ x (x) = x-x. Note that since p ∈ L 1 (R d ) we have lim x→0 µ -(τ x ) # µ TV = 0. In H8(α) for α < 1 we assume more regularity for x → (τ x ) # µ in total variation in order to obtain explicit bounds between π ε and π.

In the following proposition we provide easy-to-check conditions on the density of the prior distribution µ so that H8(α) holds. x) dx. Assume that U is γ-Hölder, i.e. there exists C γ > 0 such that for any

Proposition B.4.1 Assume that there exists

. Then H8(α) is satisfied for α 1. In addition, assume that γ 2 and that there exist c 1 , > 0 and c 2 ∈ R such that for any x ∈ R d , U (x) c 1 x + c 2 then H8(α) holds for any α > 0.

Under H8(α) we establish the following result which ensures that π ε is close to π in total variation for small values of ε. (c) Assume that p ∞ < +∞ and H8(α) then there exist ε 1 > 0 and A 0 0 such that for any ε ∈ (0, ε 1 ] we have π ε -π TV A 0 ε β min(α,1)/2 .

Note that a related result in the case where p(x) = e -U (x) / R d e -U (x) dx with U Lipschitz continuous and α = 1 can be found in [START_REF] Vono | Asymptotically exact data augmentation: models, properties and algorithms[END_REF], Corollary 1) with explicit dependency with respect to the dimension d. However, note that Proposition B.4.2 differs from [START_REF] Vono | Asymptotically exact data augmentation: models, properties and algorithms[END_REF], Corollary 1) since the Gaussian smoothing approximation is applied to the prior distribution and the estimate is given on the posterior distribution in Proposition B.4.2, whereas in (Vono et al., 2019, Corollary 1) the Gaussian smoothing approximation is applied to the posterior distribution and the estimate is given on the posterior distribution as well.

The following proposition is an extension of Proposition B.3.3 and Proposition 5.3.4. The main difference is that the approximation is expressed with respect to the true posterior π and not π ε for some value ε > 0. Let ε 1 > 0 be given by Proposition B.4.2. In order to state this proposition, we recall the following assumption which is a relaxation of the strongly log-concave condition.

H5 There exists m ∈ R such that for any x 1 , x 2 ∈ R d we have

Note that the posterior is strongly log-concave if and only if m > 0.

Proposition B.4.3 Assume H1, H7, H2, H6 and H5. Let α > 0 and assume that for any ε ∈ (0, min(ε 0 , ε 1 )], R d (1+ x 4 )(p α ε +p α )(x)dx < +∞ and H8(α). Then there exists C 0 0 such that for any ε > 0 and λ > 0 such that 2λ(L y + (α/ε) max(L, 1 + K ε /ε) -min(m, 0)) 1 and δ = (1/3)(L y + αL/ε + 1/λ) -1 , there exists C 1,ε 0 such that for any C convex compact

Using the first part of the proof, there exist λ 0 ∈ [0, 1) and c 0 0 such that for any γ ∈ (0, γ] with γ = m/(4L 2 ) we have that R γ satisfies D d (W 2 , λ γ 0 , c 0 γ). Using Jensen's inequality we obtain that for any γ ∈ (0, γ] and

0 Φ(x) . (B.54) In addition, using that for any a, b 0 with a b we have e a -e b e a (b -a), we get for any

Hence, there exist λ 1 ∈ [0, 1) and c 1 0 such that for any γ ∈ (0, γ] we have that R γ satisfies

Using the logarithmic Sobolev inequality (Boucheron et al., 2013, Theorem 5.5) we get for any γ ∈ (0, γ] and

In addition, using that for any a, b 0 with a b we have e a -e b e a (b -a), we get for any

Therefore, there exist λ ∈ [0, 1) and c 0 such that for any γ ∈ (0, γ] we have that R γ satisfies D d (W, λ γ , cγ). We now show that there exist ζ > 0 and β 0 such that (P t ) t 0 satisfies D c (W, ζ, β). First, for any x ∈ R d we have

which concludes the proof.

Lemma B.5.2 Assume that there exist λ ∈ (0, 1], c, β 0, ζ, γ > 0 such that for any γ ∈ (0, γ], R γ satisfies D d (W, λ γ , cγ) and (P t ) t 0 satisfies D c (W, ζ, β). Then, there exists C 0 such that for any x ∈ R d , t 0 and k ∈ N * we have

This concludes the proof of Proposition B.3.1 upon using (Bortoli and Durmus, 2020

t ) -b 2 (t, X B the distribution of (B t ) t∈[0,T ] witgh B 0 = x. Using the generalized Pinsker inequality (Durmus and Moulines, 2017, Lemma 24) and the transfer theorem (Kullback, 1997, Theorem 4.1) we get that (Liptser and Shiryaev, 2001, Theorem 7.7) with C 1 = C0 (C 0 + C α 0 ). Let ε 1 = min((cC 1 ) -2/β /2, (cC 1 ) -2/(βα) /2, (4κ) -1 ) and c = R d p(x)dx. Combining (B.166) with Lemma B.8.2, we get that for any ε ∈ (0, ε 1 ] π -π ε TV 2c -1 C 1 (ε β/2 + ε βα/2 ) , (B.167) which concludes the proof upon letting A 0 = 2c -1 C 1 .