Cette aventure qui a été la thèse commence bien au-delà de janvier 2019. Le mot "recherche" évoque en moi un désir de connaissance, un questionnement, l'envie d'aller toujours au fond des choses, une quête éternelle qui bien se marie avec le métier de chercheur et de chercheuse, mais qui ne se limite pas à la sphère professionnelle. La thèse étant l'accomplissement d'une étape très symbolique pour ma vie personnelle et étant une personne très réservée, l'exercice des remerciements m'est difficile.

Ayant cherché de ne jamais lésiner mes "merci", "grazie" ou "thank you" aux personnes qui m'ont aidé dans ce parcours, ne m'en voudrons pas ceux qui ne seront pas mentionnés ici, parce que je me dois quand-même de mentionner quelques personnes pour moi importantes dans ce parcours.

Je souhaite tout d'abord remercier Florence Tupin et Loïc Denis. Vous avez été pour moi bien plus que des encadrants. Florence, ta vision scientifique, ton ouverture d'esprit, te fermeté de caractère, les réponses que tu as su toujours apporter à mes questions scientifiques et non, les nombreuses discussions que l'on a eu autour de la machine à café, lors d'un point d'avancement ou en marge d'un Conseil de laboratoire ont eu un effet rassurant pour moi dans les quelques moments où la frustration pour un résultat non satisfaisant m'empêchait d'avoir une vision à long terme. Loïc, malgré une longue période de déplacements limités, tu ne m'as jamais fait manquer ta présence : tu as été toujours disponible pour une discussion en visio, pour relire un article, proposer de brillantes idées pour se tirer d'une impasse. Merci pour ta gentillesse, ta rigueur scientifique, ton exigence : je te dois beaucoup, tu as été un point de repère au cours de ces annèes. Merci aussi de m'avoir plusieurs fois accueilli à Saint-Etienne : je garde de très bons souvenirs de mes visites et notamment de notre cross dans la neige. Vous m'avez beaucoup appris sur l'imagerie radar, sur le fonctionnement du monde académique et vous m'avez également appris le français, en me donnant confiance quand sa maîtrise était plus loin d'être gagnée qu'elle ne l'est maintenant.

Merci à vous deux de m'avoir donné la possibilité de présenter les fruits de ces trois ans de travail devant un jury composé par des chercheurs et chercheuses que j'estime beaucoup. Je remercie par la même occasion les membres du jury : Bertrand Le Saux, Nicolas Thome, Andrés Almansa, Richard Bamler, Luisa Verdoliva et Charles Soussen. C'est pour moi un honneur que vous ayez consacré une partie précieuse de votre temps à rapporter mon manuscrit et à écouter ma présentation.

Je tiens à remercier, une fois de plus, Floriana. Tu sais plus que personne l'importance de cette étape et tu as toujours été présente pour moi depuis le début de ce parcours.

Merci aussi à mes parents. Vous avez toujours essayé, à votre manière, de me soutenir dans mes choix, confiants que toutes les décisions que j'ai prises ont été pour mon mieux.

IV VIII CONTENTS hypothèse de speckle temporellement décorrélé pour obtenir des réalisations indépendantes. Les changements sont compensés en recourant à une stratégie d'entraînement itérative. Le réseau SAR2SAR est donc entraîné sur des images dont le speckle est corrélé spatialement et peut être par conséquent appliqué directement sur les images radar, donnant des performances de très bonne qualité en termes de préservation de la résolution spatiale.

L'apprentissage de SAR2SAR reste cependant lourd : la stratégie se déroule en plusieurs étapes pour compenser les changements et un jeu de données contenant des piles d'images doit être constitué. Avec l'approche MERLIN (chapitre 5) (Dalsasso et al., 2022b), nous relâchons ces contraintes en proposant une méthode d'apprentissage auto-supervisée mono-image. En effet dans les images SAR complexes, les partie réelles et imaginaires sont mutuellement indépendantes et elles peuvent être naturellement utilisées pour entraîner un réseau de manière auto-supervisée. Nous montrons la simplicité de mise en oeuvre d'un tel cadre en entraînant un réseau pour trois modalités d'acquisitions, présentant des différences en termes de résolution spatiale, de textures et de corrélation spatiale du speckle.

Dans un souci de science ouverte, le code associé aux méthodes développées est disponible en accès libre.

Mi casa tiene mar y tierra, mi mujer tiene grandes ojos color de avellana silvestre, cuando viene la noche el mar se viste de blanco y de verde y luego la luna en la espuma sueña como novia marina.

No quiero cambiar de planeta.

El perezoso, Pablo Neruda

Ma maison compte mer et terre et ma femme a d'immenses yeux couleur des noisettes sylvestres, lorsque survient la nuit la mer se pare de blanc et de vert bientôt la lune dans l'écume rêve en fiancée océane.

Je ne veux changer de planète.

Le paresseux, Pablo Neruda

La mia casa ha mare e terra, la mia donna ha grandi occhi color nocciola selvatica, quando si fa notte il mare si veste di bianco e di verde, e la luna tra le schiume sogna come una sposa marina.

Non voglio cambiare pianeta.

Il pigro, Pablo Neruda

Synthèse des travaux

Le Radar à Synthèse d'Ouverture (RSO, aussi appelé SAR en anglais) permet d'acquérir des données pour l'observation de la Terre de jour comme de nuit, quelles que soient les conditions météorologiques. Grâce notamment au programme Copernicus de l'Agence Spatiale Européenne nous disposons aujourd'hui d'un grand nombre de données distribuées librement. Cependant, l'exploitation de données satellitaires radar est limitée par la présence de très fortes fluctuations du signal rétrodiffusé par la scène imagée. En effet, les images SAR sont entachées par un phénomène intrinsèque aux systèmes d'imagerie cohérente : le chatoiement, communément appelé speckle en anglais.

Dans cette thèse, nous visons à faciliter l'interprétation des images SAR grâce au développement de techniques de réduction de speckle. Les approches existantes reposent sur le modèle de Goodman, décrivant le speckle comme un bruit multiplicatif et spatialement non corrélé (chapitre 1). Dans le domaine de la vision par ordinateur, les méthodes de débruitage s'appuyant sur un réseau neuronal convolutif (approches d'apprentissage profond) ont permis de grandes avancées et représentent aujourd'hui l'état de l'art (chapitre 2). Nous proposons donc d'utiliser les techniques de débruitage basées sur les algorithmes d'apprentissage profond pour la réduction de speckle dans les images SAR (méthodes de despeckling). Premièrement, nous étudions l'adaptation des techniques dites supervisées, c.à.d. visant à minimiser l'écart, selon un certain critère, entre l'estimation fournie par le réseau et une image de référence, dite vérité terrain (chapitre 3) [START_REF] Dalsasso | SAR Image Despeckling by Deep Neural Networks: from a pre-trained model to an endto-end training strategy[END_REF]. Nous proposons la création d'une base de données d'images de référence en moyennant des piles d'images multi-temporelles acquises sur la même zone. Des paires d'images pour entraîner un réseau peuvent être générées en synthétisant du speckle selon le modèle de Goodman. Cependant, dans les images réelles le speckle est spatialement corrélé. La corrélation peut être typiquement réduite par un sous-échantillonnage d'un facteur 2, mais cela engendre une perte de résolution.

Au vu des limites des approches supervisées, inspirés par la méthode auto-supervisée noise2noise, nous proposons d'entraîner un réseau directement sur des données réelles (chapitre 4). Le principe des méthodes de débruitage auto-supervisées est le suivant : si un signal contient une composante déterministe et une composante aléatoire, un réseau entraîné à prédire une nouvelle réalisation de ce signal à partir d'une première réalisation indépendante ne pourra prédire que la composante déterministe, c.à.d. la scène sous-jacente, apprenant ainsi à supprimer le speckle. Dans la méthode que nous développons, SAR2SAR (Dalsasso et al., 2021b), nous utilisons des séries multi-temporelles sous Remote Sensing (RS) encompasses a wide variety of imaging technologies ranging from passive sensors (e.g. optical, multispectral, hyperspectral) to active sensors (e.g. radar, laser), which are capable of acquiring signals about the atmosphere and Earth's surface. Those signals, which can often be visualized and processed as images, are of fundamental importance to efficiently monitor our planet. The possibility of acquiring such data from space allows to carry on some analysis both at a local and at a global scale. Having access to different sources of information makes it possible to address multiple applications linked to Earth Observation (EO).

Historically, RS for EO was born with photography. 1972 is the year of the first integral photo of Earth (see Figure 1.1). 1972 is also the year of the first environmental movement and the year of publication of "The Limits to Growth" [START_REF] Meadows | The Limits to Growth; A Report for the Club of Rome's Project on the Predicament of Mankind[END_REF]. In this report, the authors have constructed a model "built specifically to investigate five major trends of global concern: accelerating industrialization, rapid population growth, widespread malnutrition, depletion of nonrenewable resources, and a deteriorating environment". It calls for a shift towards a more sustainable and ecological way of living, ensuring global stability. Without a dramatic change in resource consumption, the population would face an "uncontrollable decline".

Since then, 26 United Nations (UN) Climate Change Conferences (COP) and 5 Intergovernmental Panel on Climate Change (IPCC) reports followed, with the 6th scheduled for 2022. The Kyoto Protocol (1997) and the Paris Agreement (2015) present long term strategies to reduce CO 2 emissions and limit climate change. To face such environmental, social and economical challenges, RS systems play a key role by enabling a huge variety of applications: local environmental monitoring (e.g. urban expansion, deforestation, marine pollution), global environmental monitoring (e.g. biomass estimation, global warming analysis, CO 2 emissions, global sea level rise), risk and damage assessment (e.g. flooding, avalanches, earthquakes), production of semantic maps (e.g. land cover, change detection), agricultural applications (e.g. crops estimations), renewable resources monitoring are some of the many things that, through a quantitative Figure 1.1 -"The Blue Marble": this iconic image is the first integral photo of Earth. It was taken on 17/12/1972 from the Apollo 17 shuttle by the crew traveling towards the moon. Source: http://www.lpi.usra.edu/resources/apollo/frame/ ?AS17-148-22727 representation of our planet led us to a better understanding of the geophysics of the Earth. Indeed, through RS sensors one can get an efficient and quantitative monitoring of changes occurring to the environment, may it be for natural causes (e.g. floods, fires, growth of trees, snowfalls) or anthropogenic causes (e.g. construction of structures such as buildings, dams, bridges or consumption of natural resources).

The radiation theory states that every body at a temperature T greater then the absolute zero (0[K] = -273.16[°C]):

• emits its own radiation depending on the temperature T and the surface characteristics

• reflects, absorbs or is penetrated by Electromagnetic (EM) radiations coming from the outside.

RS refers to an acquisition process acquiring information about an object by means of a remote sensor. The two main natural sources that can be used in RS are the Sun and the Earth itself. We refer to systems exploiting natural resources as passive sensors. Such sensors are sensitive to the chemical constituents of the observed objects. Among them, one can cite: multi-or hyper-spectral sensors, photometers, radiometers. Sometimes, instead of using the energy naturally emitted by the Sun or reflected by the Earth, instruments produce their own radiations, providing their own source of illumination of the scene: we refer to those as active sensors.

RAdio Detection and Ranging (RADAR) consists of an active system that radiates energy into the space and detects the signal that is back-scattered by different objects towards the radar. By controlling the source of energy, it can effectively acquire data (almost) independently from atmospheric and meteorological conditions. Through a comparison of the transmitted and received signal, some information about the object that interacted with the wave can be retrieved (e.g. roughness, humidity,..). An EM wave travels in the vacuum at the speed of light c 3 × 10 8 [m/s]. It is characterized by a certain frequency f , which is linked to the wavelength λ and its speed by the following relation:

c = λ × f (1.1)
The radar is typically composed of a transmitter and a receiver. The transmitting antenna sends a radar signal that is shaped by the antenna pattern. The portion of the signal that is reflected in the direction of the radar antenna is measured by the receiver and detected: the backscattered signal carries some characteristics of the target illuminated by the transmitted wave. A radar for RS typically operates within the range of microwaves, i.e. between 0.25[GHz] and 17.6 [GHz]. The advantage of using microwaves over optical waves (used in passive systems) is their capability of seeing through clouds, haze and rain. The most exploited frequency bands are P, L, C, X and Ku (see Table 1.1). Typically, a RS process consists of three main steps: acquisition (aimed at acquiring the data), processing (aimed at improving data quality) and interpretation (where information is extracted from the data). In this Ph.D thesis, I will present some contributions related to the processing step of the particular RS process regarding SAR data 1 , namely speckle suppression techniques.

Introduction to Synthetic Aperture Radars

Carried on board satellites or aircrafts, SAR sensors provide imaging capabilities. Being active systems, they can collect information at any time of the day and with (almost) any meteorological conditions, providing a global and continuous coverage of the Earth' surface. This powerful feature of SARs allow to access areas covered by clouds, such as the tropics and the subtropics, subject to long wet seasons with frequent rainfalls.

The main drawback of SARs is the so-called speckle phenomenon. The coherent summation of the echoes backscattered by several elementary scatterers within a resolution cell results in strong signal fluctuations. This limits the interpretation of SAR images and the direct application of image processing techniques. The speckle model will be detailed in chapter 2.

In the following subsections, the basics of a SAR system are briefly recalled: the reader can refer to [START_REF] Bamler | SAR Data Acquisition and Image Formation[END_REF] and [START_REF] Maître | Processing of synthetic aperture radar (SAR) images[END_REF] for more details on the mathematical developments of equations presented hereafter.

Geometry of a RAR

A SAR operates as a Real Aperture Radar (RAR). A RAR is an active imaging system constituted of a mono-static antenna: a single antenna emits an EM wave and then records the signal backscattered from objects on the ground. The distance between the radar and the target generating the backscattered signal is retrieved from the time delay between the transmitted and received wave. A Side-Looking Aperture Radar (SLAR) is characterized by a side-looking geometry: indeed, if the wave was transmitted straight on the ground (i.e., at nadir), it would not be possible to distinguish between objects lying on the right side and on the left side at the same distance from the radar. The main diffraction lobe of an antenna of size d A transmitting a wave of wavelength λ is characterized by an angle β = 2λ/d A , resulting in an azimuth resolution δ A which depends on the radar-target distance R:

δ A = R β 2 = Rλ d A , (1.2)
meaning that two targets cannot be resolved if they are inside the half antenna beam at the same time.

On the range direction, two targets can be resolved if the pulse backscattered from the farther target is received after the reception of the first echo. For a non-modulated rectangular pulse of time duration τ , the range resolution δ R is:

δ R = cτ 2 . (1.3)
By projecting this resolution in the ground range geometry, one obtains:

δ gR = cτ 2 sin θ , (1.4)
with θ being the look angle. What this relation is suggesting is that one would need to transmit a shorter pulse to achieve a better resolution. This would imply that a smaller amount of power can be sent, resulting in a degradation of the Signal-to-noise ratio (SNR). This limitation is overcome by sending instead a frequency-modulated rectangular pulse, i.e. a chirp waveform. The chirp is defined as: s(t) = e jπKt 2 e j2πf 0 t rect τ (t) ,

and has a constant amplitude during the pulse duration τ and a linearly varying instantaneous frequency:

f (t) = Kτ + f 0 , t ∈ - τ 2 , τ 2 , (1.6)
Thus, the instantaneous frequency varies linearly within the following range:

f ∈ f 0 - Kτ 2 , f 0 + Kτ 2 (1.7)
and has as a result a bandwidth B = Kτ , resulting in an apparent time duration τ = 1/B = 1/(Kτ ). Therefore, the resolution of a compressed pulse is:

δ R,chirp = cτ 2 = c 2Kτ . (1.8)
This equation shows that one can control the compression factor K in order to have τ τ . This way, the Pulse Compression Ratio (PCR) between the range resolution of an uncompressed pulse of length τ and that of a compressed pulse of same length and bandwidth B is:

PCR = δ R δ R,chirp = cτ /2 cτ /2 = τ τ = Kτ 2 .
(1.9)

From RAR to SAR: the synthetic aperture principle

To give an order of magnitude to the resolution obtained through Equation 1.2, let us consider an antenna of size d A = 10[m] carried on board a satellite and sending a wave with an incidence angle θ = 23°. If R = 1000 [km] is the distance between the target and the radar, one would obtain an azimuth resolution δ A = 5[km]. To improve the resolution up to 5[m] with a RAR, one would need to build an antenna having a side 10[km] long. In 1951, Carl Wiley overcame this limitation with the invention of the synthetic antenna [START_REF] Carl | Pulsed doppler radar methods and apparatus[END_REF][START_REF] Carl | Synthetic aperture radars[END_REF]. While the radar moves along-track, it acquires repeatedly echoes of the same target as long as it is illuminated by the diffraction lobe of the antenna, i.e. for a distance ∆y = Rβ = 2λR/d A (see Fig. 1.4). The phase of each echo is measured with respect to a common reference (i.e. coherently, essential for SAR interferometry (Yagüe-Martínez et al., 2016)): not only the intensity of the echo is recovered, but the whole complex amplitude (its model is presented in the next chapter). Thus, signals acquired by the radar can be post-processed by delaying the received echoes of a time delay corresponding to the round-trip traveled by the wave. This is the equivalent of having a physical antenna of length d A,synth = ∆y, thus:

d A,synth = ∆y = 2λR d A .
(1.10)

The azimuth resolution then becomes: .11) This signal-processing technique is the synthetic aperture principle, leading to the acronym SAR. Interestingly, under Fresnel's approximations, this boils down to building an azimuth impulse response having a linearly modulated frequency: both the range and the azimuth direction have a transfer function described by a chirp (see Fig. 1.6) Thanks to this post-processing technique, the resolution of the radar in the above example becomes δ A = 5[m]. Interestingly, the azimuth resolution is independent from the distance between the radar and the target.

δ A = λR d A,synth = λR 2λR/d A = d A 2 . ( 1

The impulse response of a SAR system

The synthesis of a radar image, also called focusing, consists in the implementation of a matched filter in both directions. The transmitted range chirp and the natural-induced azimuth chirp are convolved with their conjugated time-reversed versions. This step, referred to as chirp compression, returns a cardinal sine (or sinc) waveform presenting a main lobe and several sidelobes: see Fig. 1.5. Bright targets are often narrow enough to produce a response well approximated by the impulse response, i.e. by a sinc function. In order to obtain an image with limited sidelobes, spectral windowing is applied (Massonnet and Souyris, 2008) (Fig. 1.6). Indeed, a SAR image presents a high dynamic range due to the strong response produced by human-made structures (buildings, cars, ships), which is of several orders of magnitude higher than the response generated by vegetated or flat areas such as roads. Around these strong targets, the sidelobes of the impulse response of a SAR imaging system are clearly visible, hiding the much weaker echoes in their vicinity. Therefore, by spectrally apodizing the image, the rectangular profile spectrum of the chirp is modified to have a bell-shaped profile. This way, the widening of the main lobe of the sinc is traded for a reduction of sidelobes: see Fig. 1.6. Figure 1.6 -Formation of a SAR image with a single target: after the signal is acquired, the image undergo a focusing step both in the range and azimuth direction, improving the resolution. The response of the system being a sinc function, strong sidelobes are visible around the bright target. To reduce them, the image is spectrally apodized. Moreover, it is sampled at a frequency higher than the Nyquist frequency.

As it will be detailed in the next chapter, the SAR signal is impaired by the speckle, a physical phenomenon responsible for strong signal fluctuations. Goodman's model of speckle is the founding statistical model of the speckle reduction techniques (also called despeckling) developed in the last decades. The assumption that pixels are independent and identically distributed (i.i.d.) is certainly the least representative of real SAR imaging systems [START_REF] Argenti | A tutorial on speckle reduction in Synthetic Aperture Radar images[END_REF]. Indeed, the SAR acquisition system includes an apodization function and an oversampling step whose impulse response spatially correlates the data. This sensor-specific procedure thus introduces spatial correlations in the speckle component.

Speckle spatial correlations severely impair the performances of despeckling algorithms developed under the hypothesis of white speckle. Chapter 3 will illustrate the problem, while two methods aiming at reducing speckle and learning the system Point Spread Function (PSF) (squared modulus of the impulse response) altogether are presented in chapters 4 and 5.

The use of SAR time series

In the last two decades we have witnessed an increased interest on the analysis of time series, whether it be multi-temporal data acquired by passive or active sensors. Two main factors have contributed to this trend: (i) the reduction of revisit time, i.e. the period of time between two acquisitions of the same point on Earth taken by a satellite (or a family of satellites) orbiting around Earth; (ii) the new open policy data distribution of some images providers (e.g. Landsat or the Sentinel family operated by the European Space Agency (ESA)). This enables to tackle a series of problems that can be all categorized as "change detection": binary change detection (e.g. detection of abrupt changes such as floods, fires), multi-class change detection (e.g. update of thematic maps), change detection in long time series (e.g. land cover monitoring for yield prediction and vegetation evolution).

In the context of the ESA Copernicus program, the Sentinel-1 constellation of two satellites carrying a SAR sensor allows a continuous and frequent coverage of the Earth surface, delivering an image of the same area every few days (6 days at most) from a slightly different look angle (allowing to perform an interferometric analysis [START_REF] Bamler | Synthetic aperture radar interferometry[END_REF]). Its open data policy has ensured the availability of long Sentinel-1 SAR time series. For SAR data, time-series analysis is intimately related to image restoration. On the one hand, one needs to reduce signal fluctuations caused by speckle to effectively detect changes between images of the same area. On the other hand, each acquired date can provide a new speckle realization and can be combined to produce an image with reduced speckle. However, stacks of co-registered images can be treated as independent speckle realizations of the same underlying clean signal only in the absence of changes, requiring a prior change-detection step to select stable areas. Moreover, as the speckle depends on the scene geometry, acquisitions would need to be sufficiently far apart to allow a high degree of temporal decorrelation but sufficiently close to minimize the number of changed areas. Due to temporal correlation, the averaging of L images of the same zone leads to an image having an Equivalent Number Of Looks (ENL) smaller than L. In other words, fluctuations are reduced by a factor smaller than L, as images are not independent.

In this Ph.D, SAR time series will be exploited at three levels:

• To produce a dataset of groundtruth data (Chapter 3). Speckle-free images are obtained by temporally averaging (i.e. multilooking on the temporal axes) stacks of co-registered SAR multi-temporal data.

• To develop a learning algorithm able to reduce speckle by only looking at specklecorrupted real data (Chapter 4). A change detection step is performed to compensate changes between different dates: this way, each image of the stack represents the same, but differs in the speckle component.

• The information contained in multi-temporal SAR series is used to improve the performances of single-image speckle reduction techniques (Chapter 4).

Context of the Ph.D

The main objective of this Ph.D is to improve the interpretation of SAR images by exploring the potential of speckle reduction techniques based on neural networks.

As it will be seen in next chapter, the speckle can be statistically described as a multiplicative noise. A big leap in natural image denoising was made possible thanks to (possibly deep) Convolutional Neural Network (CNN) [START_REF] Zhang | Beyond a gaussian denoiser: Residual learning of deep CNN for image denoising[END_REF], a datadriven machine learning technique relying upon a network of (possibly several) layers of convolutional kernels [START_REF] Goodfellow | Deep Learning[END_REF]

Challenge (2): Open code

Weights of the trained models described in the literature were not publicly available.

Thus, reproducing published works required a considerable effort by the researchers, making it more difficult to build on existing methods and compare with them.

As a matter of fact, at the beginning of my Ph.D thesis, I concentrated my effort on building a dataset of speckle-free SAR images. This set of groundtruth data served to train a supervised model inspired by the work of [START_REF] Chierchia | SAR image despeckling through convolutional neural networks[END_REF], allowing to produce state-of-the-art results. This step was necessary to create a baseline and to build on it.

While satisfying results can be obtained on simulated data corrupted by spatially uncorrelated speckle, real data have undergone a spectral apodization that spatially correlates image pixels, therefore leading to a new challenge:

Challenge (3): Speckle spatial correlations

In real SAR images, the speckle component is spatially correlated due to the transfer function of the system. Speckle reduction algorithms require a fine modeling of the speckle to take into account the spatial correlations. A model for spatially correlated speckle, however, does not exist.

As as consequence of Challenge (3) Speckle spatial correlations, the despeckling community started to move towards training strategies allowing to learn to suppress speckle by only looking at corrupted examples (so-called self-supervised approaches.) Thus, the second part of my Ph.D thesis was devoted to the study of deep learning methods robust to spatially correlated speckle.

Organization of the manuscript

This Ph.D manuscript is organized as follows. Chapter 2 presents Goodman's model of speckle. While providing an overview of speckle suppression techniques, the main contributions proposed in this Ph.D are highlighted and contextualized within the the state-of-the-art. Chapter 3 first introduces SAR-CNN (Dalsasso et al., 2020), a supervised deep learning algorithm trained with a dataset of speckle-free images (obtained from stacks of co-registered SAR images acquired at different dates) corrupted by synthetically generated speckle, then studies some solutions to improve robustness to spatial correlations of speckle. It thus provides some answers to Challenge ( 1 All along my Ph.D thesis, I have associated with each submitted article the open code of the proposed algorithm: an example on SAR2SAR is presented in appendix E. This initiative, related to Challenge (2) Open code, is devoted to let every researcher take advantage of the latest development in the field on SAR image despeckling, either for application purposes, comparison or to build on it even more sophisticated algorithms, contributing to the advancement of the RS community towards its goals. It is present in every chapter of this Ph.D manuscript and, even more than that, it is a precept I believe in.

The next chapters are based on already published research articles. These will be cited at the beginning of each chapter. Nevertheless, these have been rearranged to provide a consistent manuscript and enriched with complementary information and research paths that have been explored since and that are not covered in the published papers.

Chapter 2

State-of-the-art of SAR despeckling

Associated publications:

Journal papers:

• B.

Image Restoration in Remote Sensing

Remote sensing data restoration attempts to recover an image from its corrupted version. The recovered image improves further analysis of the remote sensing images. Remote sensing images can be degraded by three major sources: atmospheric perturbation, imaging systems, and instrumental noises. The atmosphere can have several major impacts on remote sensing data (particularly the ones captured by passive sensors) such as absorption, scattering, and reflection of the solar radiation in the atmosphere. Imaging systems induce artifacts and noise such as speckle in SARs and striping in Hyperspectral Images (HSI) push-broom imaging systems. Instrumental (sensor) noise includes thermal (Johnson) noise, quantization noise, and shot noise (for optical imagery). To compensate for the atmospheric effects, atmospheric corrections should be applied. The noises and artifacts induced by imaging systems and the instruments often are treated by image processing and machine learning techniques. Fig. 2.1 shows the dynamics of the important subject of denoising and restoration in the remote sensing community. The reported numbers include magazines, scientific journals, and conference papers published by IEEE on this particular subject using "remote sensing" and "(denoising, restoration, or noise reduction)". In order to highlight the number of papers, the time period has been split into a number of equal time slots (i.e., 2000-2002, 2003-2005, 2006-2008, 2009-2011, 2012-2014, 2015-2017, and 2018-2020 (December 31st)). The steadily increasing number of papers reveals the popularity of this subject.

In this Ph.D thesis, we will study the particular case of degradation caused by speckle to SAR images and propose some techniques aimed at reducing it, i.e. despeckling techniques.

The speckle model

SAR is a coherent imaging technique. The coherent acquisition of the radar signal is essential: echoes collected at different locations along the track of the radar antenna can be numerically combined to refocus the wave field diffracted by the scene, the phase shift between SAR echoes captured under slightly different angles or dates reveals 3D locations and displacements in SAR interferometry. The downside of coherent imaging techniques is the speckle phenomenon: measurements correspond to the coherent summation of several elementary responses ρ n exp(jϕ n ), these responses may either add constructively (leading to a large echo) or destructively (leading to a very low echo). The outcome of this coherent summation is intimately related to the geometrical configuration of elementary scatterers and is modeled, for surfaces that are rough at the scale of the radar wavelength, by the Goodman's model [START_REF] Goodman | Speckle phenomena in optics : Theory and applications[END_REF]. The hypotheses considered in Goodman's model are:

• Phases ϕ n and amplitudes ρ n are i.i.d. random variables and they are statistically The single-look complex image contains spatially-correlated speckle components that are independent in the real and imaginary parts. The SAR transfer function shown here corresponds to Sentinel-1 stripmap mode. For visualization purposes, a non-linear look-up table is used to display intensity images. independent from one another (i.e., knowledge of the phase brings no information about the amplitude of the same elementary scatterer, and vice versa)

• The surface roughness is large compared to the radar wavelength. As a result, each elementary echo has a phase which is uniformly distributed in [-π, π], i.e. all phase values are equally likely.

• There is a large number of elementary scatterers within a resolution cell It follows from the central limit theorem that the distribution of z = n ρ n exp(jϕ n ) converges to a circular complex Gaussian distribution [START_REF] Goodman | Speckle phenomena in optics : Theory and applications[END_REF]:

p Z (z) = 1 πR exp(-|z| 2 /R) , (2.1)
where z is the complex amplitude at a given pixel and R > 0 is the SAR reflectivity at that pixel.

The multiplicative nature of speckle phenomenon becomes clear by writing z under the form z = ν √ R, with R the reflectivity of the homogeneous area and ν a complex random variable distributed according to:

p N (ν) = 1 π exp(-|ν| 2 ) . (2.2)
The decomposition of z into its real and imaginary parts, z = a + jb, leads to:

p Z (z) = p Z (a + jb) = 1 πR exp(-(a 2 + b 2 )/R) = 1 √ 2π R/2 exp(-a 2 /R) N (0,R/2) 1 √ 2π R/2 exp(-b 2 /R) N (0,R/2) , (2.3)
which shows that the real and imaginary parts of the complex amplitude are i.i.d. according to a Gaussian distribution with variance R/2, or equivalently, that the real and imaginary parts of the complex-speckle component ν in the multiplicative model are i.i.d. and Gaussian-distributed with a variance equal to 1/2. This multiplicative stochastic model is illustrated in the left part of Fig. 2.2. From one pixel to the next, the realization of speckle is independent and the random field ν ∈ C K of a K-pixels image is a white Gaussian field.

Depending on the acquisition mode, the chosen pixel size, and the spectral apodization applied to reduce sidelobes around bright targets, a specific SAR system response then transforms the spatially uncorrelated field z into a spatially correlated field z, see Fig. 2.2 (center):

z = Hz , (2.4)
with H the spatial-domain operator associated to the SAR transfer function. If the SAR system H is real-valued (for a shift-invariant system, this corresponds to a frequency response with Hermitian symmetry), then by linearity of H we get: z = ã + j b with ã = Ha and b = Hb .

(2.5) ã and b are spatially correlated but mutually independent random fields (appendix A derives a slightly more general condition on H to obtain statistically independent components ã and b). This property will be at the heart of Chapter 5.

Goodman's model provides a statistical characterization of the intensity image I = |z| 2 = a 2 + b 2 , i.e. before the application of the operator H correlating image pixels. The intensity of the SAR echo I at a given pixel is perturbed by a randomly fluctuating variable S,

I = S × R , (2.6)
In this multiplicative noise model, S is distributed according to a gamma law:

p(S) = L L Γ(L) S L-
y = s + x , (2.8)
where the log-speckle s follows a Fisher-Tippett distribution:

p(s) = L L Γ(L) e Ls • exp(-Le s ) , (2.9)
and the variance is constant:

Var[s] = ψ(1, L) (ψ(1, L)
is the polygamma function of order L, see e.g. [START_REF] Abramowitz | Handbook of mathematical functions: with formulas, graphs, and mathematical tables[END_REF]). Averaging speckle samples in log domain requires an adequate debiasing step since the log-speckle has a non-zero mean:

E[s] = ψ(L) -log(L) (ψ is the digamma function).
Beyond the intensity of the back-scattered echoes, SAR images can also capture information of the phase and polarization of the wave. Such additional information is central in several applications of SAR imaging based on polarimetric analysis and classification, interferometry, differential interferometry and tomography. At each pixel, a diffusion vector 6 in Polarimetric and Interferometric SAR (PolInSAR), k ∈ C N with N ≥ 3 in SAR tomography). Due to the speckle phenomenon, k ∈ C K fluctuates according to a complex circular Gaussian distribution:

k is collected (k ∈ C 2 in SAR Interferometry (InSAR), k ∈ C 3 in SAR Polarimetry (PolSAR), k ∈ C
p(k|Σ) = 1 π K |Σ| exp(-k † Σ -1 k) , (2.10)
where the covariance matrix Σ ∈ C K×K characterizes the image surface (i.e., it contains the interferometric and/or polarimetric information), k † is the Hermitian transpose of column vector k, and |Σ| is the determinant of the covariance matrix Σ. To access that information, the sample covariance C is typically computed by averaging over a small local window of L pixels:

C = 1 L L =1 k k † . (2.11)
Due to speckle, the sample covariance matrix is noisy: it fluctuates according to Wishart's distribution:

p(C|Σ, L) = L LK |C| L-K Γ K (L)|Σ| L exp(-Ltr(Σ -1 C)) .
(2.12)

For large values of L, speckle fluctuations are limited but this corresponds to averaging many pixels which represents a dramatic resolution loss.

The statistical models of speckle given in equations (2.7), (2.9), (2.10), and (2.12) are at the core of SAR restoration techniques.

Figure 2.3 -Summary of the statistical models of a single channel SAR image. The Single-Look complex SAR image z = a + jb follows a complex circular Gaussian distribution. SAR intensities can possibly be averaged incoherently, with L being the number of looks (i.e. the number of measurements that are combined). To compress the dynamic range and stabilize speckle variance, one often applies a logarithmic transform to the data, turning the speckle into an additive noise.

SAR speckle reduction is a challenging problem for several reasons:

• SAR images have a high-dynamic range, with contrasts that span several orders of magnitude between scattering surfaces with moderate roughness and strong echoes produced by multiple reflections on man-made structures (dihedral and trihedral configurations of the ground and building walls, metal fences, power poles);

• due to the heavy-tailed distribution of speckle (eq. 2.7, Fig. 2.3), maximum a posteriori estimators require solving non-convex minimization problems;

• multi-variate SAR modalities like InSAR, PolSAR, PolInSAR, and SAR tomography involve the estimation of complex-valued covariance matrices.

• For real images, due to the application of the transfer function H the delivered intensity image is spatially correlated:

Ĩ = |z| 2 = |Hz| 2 = |H ã| 2 + |H b| 2 .
Adapting denoising algorithms developed for Additive White Gaussian Noise (AWGN) reduction in natural images is far from easy. The specificities of SAR data require building specific denoising algorithms adapted to speckle statistics, called despeckling algorithms. A non-exhaustive overview is given in the next section. To provide a broader picture of the evolution of despeckling approaches in the last decades, the reader can refer to Fig. 2.4.

Conventional Despeckling Techniques

The strong, signal-dependent, non-Gaussian fluctuations corrupting SAR intensity images have driven the development of numerous restoration approaches. The early approaches were based on local filters, i.e., on the averaging of intensities within a small window. Due to the strong speckle fluctuations, the use of small windows is not sufficient to reach a satisfying level of smoothing. Increasing the size of the averaging window, however, leads to an unacceptable resolution loss. Some mechanism is required to prevent the blurring of point-like structures and edges. Lee's σ filter [START_REF] Lee | Digital image smoothing and the sigma filter[END_REF] weighted averaging:

R i = j∈N i w i,j I j and Σ i = j∈N i w i,j C j , (2.13)
where the strategy to derive the weights w i,j from the data varies according to the pixel-selection method.

Rather than explicitly selecting similar pixels, variational approaches (second block of Fig. 2.4) define the estimator as that achieving a trade-off between data fidelity and adequacy with the prior model, i.e., a regularization term. Edge-preserving priors such as the Total Variation (TV) ( et al., 2009) have been considered for the restoration of intensity images. The datafidelity can be derived from the gamma distribution (Eq.(2.7)) or the Fisher-Tippett distribution (Eq.(2.9)). The latter has the advantage of being convex, and thus, easier to minimize. The regularization can either be applied to the reflectivities or to the log-transformed reflectivities, leading to minimization problems of the form

R = arg min R L n i=1 R i + I i R i + λφ(R), (2.14) or x = arg min x L n i=1 (x i -y i + exp(y i -x i )) + λφ(x), (2.15)
where R ∈ R n + is the n-pixels restored reflectivity image, I ∈ R n + is the n-pixels noisy SAR intensity image, in linear scale, and x ∈ R n and y ∈ R n are their equivalent in log-scale.

Patch-based and variational methods can be combined, which is particularly beneficial in the context of interferometric phase estimation to reconstruct buildings [START_REF] Ferraioli | PARISAR: Patch-based estimation and regularized inversion for multibaseline SAR interferometry[END_REF].

The specificities of speckle in SAR images have justified the design of dedicated restoration methods, often inspired by the continuing progress in the field of natural image denoising. Adapting those methods can, however, be a tedious task. A way to circumvent these adaptations is to tackle speckle reduction using a plugin Alternating Directions Method of Multipliers (ADMM) approach, i.e., decomposing the image reconstruction process into an alternation of non-linear steps to account for speckle statistics and Gaussian denoising steps that can be performed by any off-the-shelf Gaussian denoiser [START_REF] Deledalle | MuLoG, or how to apply Gaussian denoisers to multi-channel SAR speckle reduction[END_REF].

Overview of deep learning-based despeckling techniques

Deep neural networks have the capability to learn application-specific patterns and adapt to non-Gaussian corruptions. Unsurprisingly, deep learning applications to the restoration of SAR images have flourished these last years, see bottom part of Fig. 2.4. When designing a neural network, special care must be paid to the specificities of SAR imagery, in particular the high dynamic range of SAR intensities, the complex-valued definite-positive covariance matrices that characterize the interferometric or polarimetric information, the non-stationary noise variance, or the spatial correlations of speckle due to spectral apodization.

Training deep models requires a huge amount of data in order to generalize well. However, in SAR image denoising, there is a shortage of ground truth information to train supervised models, and ad-hoc datasets have thus to be built to this aim. In this context, the typical signature of bright scatterers and the geometrical distortion linked to the lateral view of the radar (such as layover, shadowing and foreshortening) require a careful application of canonical data augmentation techniques (e.g. rotation and mirroring). Alternatively, pre-trained models can be integrated into the MUltichannel LOgarithm with Gaussian denoising (MuLoG) framework mentioned in the previous paragraph (Yang et employs a pre-trained Gaussian denoiser in an iterative scheme accounting for speckle statistics; supervised training, using ground-truth images that match the speckled images provided as input to the network; self-supervised training, using co-registered pairs of SAR images captured at different dates; a single image and a masking strategy: the network is trained to correctly infer the masked pixels of the input image; a single image in Single-Look Complex format and exploiting the mutual independence of its real and imaginary part.

). Only the areas not affected by temporal changes are kept and the supervised training is performed using pairs of speckle-corrupted images and multi-temporal averages. . Self-supervised approaches that build upon the noise2noise framework [START_REF] Lehtinen | Noise2noise[END_REF] rely on the intuition that, given the random nature of noise (assuming a perfect temporal decorrelation), a model trained to predict, from a noisy acquisition, another noisy realization ends up predicting the common component: the underlying reflectivity. However, in practice, the two acquisitions must be sufficiently separated in time for temporal speckle decorrelation to occur. Changes are then also more likely to arise. A patch-similarity measure can be used to weight the loss function [START_REF] Ma | SAR image despeckling by noisy reference-based deep learning method[END_REF]. As an alternative, changes can be not only detected but also compensated. This will be at the heart of the second contribution of this Ph.D:

Contribution (2): SAR2SAR (Dalsasso et al., 2021b)
In SAR2SAR, we propose to apply a compensation to make sure that only the speckle component differs between the two dates. Training a network on actual SAR images makes it possible to learn the actual statistics of speckle and to capture the spatial correlations induced by the impulse response of the SAR imaging system. We show that networks trained with this self-supervised strategy are readily applicable to actual SAR images: the reconstructed images do not suffer from artifacts due to the spatial correlations of speckle. This training framework is depicted in Fig. 2.5 ). ). This approach is at the crossroad between Bayesian modeling and deep learning. With the blind-spot CNN, the clean value of a pixel is obtained by combining the observed value at that pixel and an estimation based solely on the values of the neighbouring pixels. The network is trained to predict the central value of a window from the values of the neighbouring pixels. The quality of this estimation is evaluated by comparing the predicted value with the actual (noisy) value, which provides a reference-less way to train the network.

This contribution answers to

In this unsupervised algorithm, there is no need for a SAR time series and the training set can thus easily be created. However, it requires speckle to be spatially uncorrelated (so that the speckle realization at the central pixel be independent from the speckle realizations at neighboring pixels). To improve the robustness to residual speckle correlations, the network in (Molini et al., 2021) is trained with a central spot of variable size, which prevents from relying too heavily on pixels in the immediate vicinity of the target pixel. However, to preserve the advantages of training a model using single images while being robust to speckle spatial correlations, one needs to rely on some form of speckle whitening. In the third contribution of this Ph.D, we show that it is possible to use a single-image learning algorithm to remove spatially correlated speckle by exploiting another form of diversity: Contribution (3): MERLIN (Dalsasso et al., 2022b) A neural network can be trained in an unsupervised way on a single image by exploiting the independence between real and imaginary part of a SLC SAR image. MERLIN shows that it is possible to train deep models by feeding them with the real part and using the imaginary part to supervise the learning (and vice versa). As the framework only assumes real/imaginary part statistical independence, it can handle spatially correlated speckle: thus, MERLIN is trained with images that underwent the SAR transfer function (Fig. 2.5 ).

This contribution answers to

Context of the work

The question motivating this work is the following: do neural networks need to be trained specifically on SAR data for speckle reduction and is it worth the effort of producing a high-quality dataset? This chapter is mainly based on our paper SAR Image Despeckling by Deep Neural Networks: from a Pre-Trained Model to an End-to-End Training Strategy [START_REF] Dalsasso | SAR Image Despeckling by Deep Neural Networks: from a pre-trained model to an endto-end training strategy[END_REF]. The objective of this chapter is to shed light on the advantages and disadvantages of making a significant effort to create training sets and learning SARspecific CNNs rather than readily applying generic networks pre-trained for AWGN removal on natural images to SAR despeckling. In this PhD, we consider this question for intensity images as we believe that it is enlightening for the more difficult case of multi-channel SAR despeckling that arises in SAR polarimetry, SAR interferometry or SAR tomography.

Figure 3.1 -Illustration of the three speckle reduction approaches described in this chapter: the first two apply a CNN trained to remove AWGN from natural images, the last approach consists of training a CNN specifically to the suppression of speckle in (log-transformed) SAR images.

In order to discuss this matter, we consider three different SAR despeckling frameworks based on CNNs. The first one consists of applying a CNN pre-trained on AWGN removal from natural images. Many CNNs have been proposed for AWGN suppression. The extension to SAR imagery requires to account for the statistics of speckle noise. This can be performed by an iterative scheme recently introduced for speckle reduction: MuLoG algorithm [START_REF] Deledalle | MuLoG, or how to apply Gaussian denoisers to multi-channel SAR speckle reduction[END_REF], which is based on the plug-in ADMM strategy (Chan et al., 2017). To investigate the sensitivity of the neural network to the training set, we describe a new procedure to generate a high-quality training set of speckle-free SAR images, in an attempt to address Challenge (1) Training set. We use this dataset to retrain the CNN for AWGN removal on SAR images and integrate the network within MuLoG: we refer to this technique as the hybrid approach. In the final approach, a network is trained specifically to suppress speckle on SAR images. We use the same dataset as before and corrupt the images with synthetically generated i.i.d. speckle. We consider a network architecture similar to that used in the work of Chierchia et al. [START_REF] Chierchia | SAR image despeckling through convolutional neural networks[END_REF] and discuss the influence of the number of layers and of the loss function on the despeckling performance of the CNN, trained and tested on our datasets.

We consider this step as necessary to create a baseline and, at a later stage, build on it. To facilitate comparisons of future despeckling methods with our work (see Challenge (2) Open code, we provide an open-source code that includes the network weights for Sentinel-1 image despeckling (see Section 3.3).

Finally, we provide a comparison of existing strategies to understand which one is the most effective at taking into account speckle spatial correlations. We thus begin an analysis of Challenge (3) Speckle spatial correlations.

SAR Despeckling Using Supervised CNNs

Recall: Speckle model

The intensity I measured by a SAR sensor is linked to the hidden reflectivity R and the speckle component S by a multiplicative relation [START_REF] Goodman | Speckle phenomena in optics : Theory and applications[END_REF]:

I = R × S,
where the speckle component follows a Gamma distribution with unitary mean and variance inversely proportional to the number of looks L:

p(S) = L L Γ(L) S L-1 exp (-LS) .
(2.7)

The log-speckle s = log(S) is distributed according to a Fisher-Tippett distribution of mean E[s] = ψ(L) -log(L) and variance Var[s] = ψ(1, L):

p(s) = L L Γ(L)
e Ls exp(-Le s ).

(2.9)

Architecture of the CNN

Name N out Configuration Layer 1 64 3 × 3 CONV, ReLU Layer 2 to (D-1) 64 3 × 3 CONV, Batch Norm., ReLU Layer D 1 3 × 3 CONV
In the field of AWGN removal, the residual learning method DnCNN introduced in (Zhang et al., 2017) is a reference method for which models pre-trained on natural images at various signal-to-noise ratios are available. The loss function that is minimized during the training step is the L 2 loss (i.e., the sum of squared errors, averaged over the whole training set). To train the DnCNN, 400 natural images of size 180×180 pixels with gray levels in the range [0, 1] were used for training. Patches of size 40 × 40 pixels were extracted at random locations from these images. Different networks were trained for simulated additive white Gaussian noise levels equal to σ = 10 255 , 15 255 , ..., 75

255

(14 different networks each corresponding to a given noise standard deviation).

We consider two different ways to apply a pre-trained CNN to speckle noise reduction: a homomorphic filter that processes log-transformed intensities with DnCNN (section 3.2.1) and the embedding of DnCNN within the iterative scheme MuLoG (Deledalle et al., 2017) (3.2.2). Note that our approach is general and other pre-trained CNNs than the DnCNN could readily be applied. Finally, in section 3.2.3 we propose to train a CNN specifically for speckle reduction. The three approaches are summarized in Fig. 3.1.

Homomorphic Filtering with a Pre-Trained CNN

Please note that the work described in this subsection has been primarily done by Xiangli Yang. It is however recalled in this section as it allows to study the need for a specific training of deep neural networks for speckle removal.

The simplest approach to applying a pre-trained CNN acting as a Gaussian denoiser is the homomorphic filtering depicted at the top of Figure 3.1, and hereafter denoted homomorphic-CNN. This approach consists of approximating the noise term s in logtransformed data as an additive white Gaussian noise with non-zero mean. As recalled at the beginning of this Section, log-transformed speckle is not Gaussian but follows a Fisher-Tippett distribution under Goodman's fully developed speckle model (see Figure 3.3). Hence, the homomorphic approach is built on a rather coarse statistical approximation. Under this approximation, the log-transformed SAR image can be restored by first applying the pre-trained CNN, then correcting for the bias ψ(L) -log L due to the non-centered noise component.

In order to successfully apply a pre-trained CNN model, it is crucial to properly normalize the data so that the range of input data matches the range of data used during the training step (neural networks are highly non-linear). This method maps the range [q m , q M ] by an affine transform to the [0, 1] range, with q m and q M corresponding to the 0.3% and 99.7% quantiles of the log-transformed intensities. After this normalization, the standard deviation of the log-transformed noise is σ = ψ(1, L)/(q M -q m ). This value can be used to select the network trained for the closest noise standard deviation σ train that is less or equal to σ. The normalized image is then multiplied by σ train /σ so that the noise standard deviation exactly matches that of the images in the training set of the network.

Iterative Filtering with MuLoG and a Pre-Trained Model

Please note that the work described in this subsection has been primarily done by Xiangli Yang. It is however recalled in this section as it allows to study the need for a specific training of deep neural networks for speckle removal.

The MuLoG framework [START_REF] Deledalle | MuLoG, or how to apply Gaussian denoisers to multi-channel SAR speckle reduction[END_REF] accounts for the Fisher-Tippett distribution of log-transformed speckle (see Figure 3.3) with an iterative scheme that alternates the application of a Gaussian denoiser (namely, a proximal operator) and of a non-linear correction. Figure 3.1, second row, illustrates that, by embedding a CNN trained as a Gaussian denoiser within an iterative scheme, a Fisher-Tippett denoiser is obtained. Throughout the iterations, the parameter of the Gaussian denoiser evolves, which requires to apply the network selection and image normalization strategy described in the previous paragraph.

Despeckling with a CNN Specifically Trained on SAR Images

The architecture that we consider for our CNN trained on SAR images (SAR-CNN) has been described in Section 3.2. In this section, we describe in details a pipeline for producing high-quality ground-truth images for the training step of the network. Then, we show how this dataset can be used to train a CNN for speckle reduction.

Training-Set Generation

Deep learning models need a lot of data to generalize well. This is an issue in deep learning-based SAR image despeckling techniques, due to the lack of truly specklefree SAR images. The reference image has to be therefore created through an ad-hoc procedure, in order to produce images taking into account the content of SAR data (strong backscattering scatterers, real radiometric contrasts of SAR data). This is done by exploiting series of SAR images. Speckle noise can be strongly reduced by multi-temporal multilooking (i.e., averaging the intensity of images acquired at different dates, assuming that no changes occurred and that images are temporally decorrelated). Multi temporal stacks have therefore been considered in this study. Due to the coherence of some regions, some speckle fluctuations are remaining after this temporal multi-looking procedure. These images are further improved by applying a MuLoG+BM3D denoising step [START_REF] Deledalle | MuLoG, or how to apply Gaussian denoisers to multi-channel SAR speckle reduction[END_REF] with an equivalent number of looks estimated from selected homogeneous regions. The images obtained are then considered speckle-free and serve as a ground truth. Synthetic speckle noise is simulated based on the statistical models described in Section 3.2 in order to produce the noisy/clean image pairs necessary for the training of the network. Although Goodman's fully developed model is not verified everywhere in the images (the Rician distribution [START_REF] Goodman | Speckle phenomena in optics : Theory and applications[END_REF][START_REF] Eltoft | The Rician Inverse Gaussian Distribution: A New Model for Non-Rayleigh Signal Amplitude Statistics[END_REF] Nicolas and Tupin, 2019) could instead be used for resolution cells with a dominant scatterer) and assumes spatially uncorrelated speckle, it has been the funding model of most SAR despeckling methods developed these last four decades [START_REF] Argenti | A tutorial on speckle reduction in Synthetic Aperture Radar images[END_REF]. Thus, we find it relevant to simulate 1-look speckle noise based on Goodman's model. In Section 3.4, its limitations are discussed.

A ground truth image is depicted in Figure 3.4, where we show the progress from the 1-look SAR image to the clean reference used in our training step. The temporal average of large temporal stacks of finely registered SAR images leads to images with limited speckle fluctuations and, after denoising, these images retain the characteristics of SAR images (bright points, sharp edges, textures) with almost no residual fluctuations due to speckle. Since the denoising operation is applied on an image already temporally multilooked, only small fluctuations have to be suppressed by the denoising step (i.e., this denoising step helps but is not crucial). This method thus proposes a realistic way to create speckle-free SAR images. A description of the training set is then given in Table 3.2. The network is easier to train on log-transformed data since the noise is then additive and white (and coarsely Gaussian distributed, as it can be seen from Figure 3. To train the network, several loss functions have been considered. Experiments (see Section 3.3) suggest the L 1 loss function to be preferable to the smoothed L 1 loss function of Chierchia et al. and to L 2 loss function. This matches other studies that have shown a reduction of artifacts and an improvement of the convergence when using the L 1 loss (Zhao et al., 2017). We, therefore, used the following loss function:

N i=1 f CNN (y i ) -x i -(ψ(L) -log(L)) • 1 1 (3.1)
where the sum is carried over all N images from (a batch sampled from) the training set, f CNN (•) represents the action of the CNN on some log-transformed input data, the boldface is used to denote images (the i-th noisy image I i and the corresponding specklefree reference R i ). The term (ψ(L) -log(L)) • 1 is a constant image that corresponds to the bias correction. Its role is to center Fisher-Tippett distribution. By making this term explicit, it is easier to perform transfer learning, i.e., to re-train a network to a different number of looks L by warm-starting the optimization from the values obtained for the previous number of looks.

The Training of the Network

The training set is formed by 7 Sentinel-1 speckle-free images produced by filtering 7 different multi-temporal stacks of size between 1024 × 1536 and 1024 × 8192 pixels. The images are selected so that to cover urban areas, forests, a coast with some water

Hybrid Approach: MuLoG + Trained CNN

A hybrid approach is also considered. In this method, the dataset that we have constructed is used to retrain the CNN described in Section 3.2.3 to remove Gaussian noise from SAR images. Account for the Fisher-Tippett distribution is also made possible by embedding this network within the MuLoG framework. By doing so, we aim at investigating the influence of the content of the training images on the restoration performances.

Experimental Results

The following paragraphs provide a comparison of the different strategies for CNN-based despeckling both on images with simulated speckle and on single-look Sentinel-1 images. First, the impact of the loss function and of the number of layers on the performance of SAR-CNN is illustrated. Network architecture is described in Sections 3.2.2 and 3.2.3 (and more in details in the original article (Zhang et al., 2017)). A graphic illustration is also provided in Figure 3.5. The network has been trained in a supervised way using a dataset created as explained in Section 3.2.3. The three proposed algorithms are summarized in Table 3.3. The image corresponds to one of the speckle-free images of our testing set. We magnify 6 regions in order to illustrate cases where the initial CNN architecture fails to recover some structures (regions 1, 2, 3 and 6) that are present in the ground truth and at least partially recovered applying the proposed modifications to the CNN, and cases were some spurious structures appear (regions 4 and 5) with the first CNN architecture employed, but are not created with the modifications of the CNN depth and loss function that we considered. All the three architectures are trained for 50 epochs on the high-quality database of SAR images we have created, and conclusions are drawn after qualitative evaluation of our testing images. Indeed, as already claimed, a comparison with the work of Chierchia et al. is not applicable.

Quantitative Comparisons on Images with Simulated Speckle

Two common image quality criteria are used to evaluate the quality of despeckling obtained with different methods: the Peak-Signal-to-Noise Ratio (PSNR), related to the mean squared error, which is relevant in terms of evaluation of the estimated reflectivities (bias and variance of the estimator), and the Structural SIMilarity (SSIM) which better captures the perceived image quality. For each speckle-free image from the testing set, several versions corrupted by synthetic single-look speckle are generated, in order to report both the PSNR and SSIM mean values, and their standard deviations over different noise realizations.

Seven different images are used in our testing set. We report the performance of the approaches proposed in this chapter as well as the performance of SAR-BM3D (Parrilli To qualitatively evaluate the quality of denoising, we display in Figure 3.7 the results obtained by the different methods on image "Marais 1". To better analyze the results, the residual intensity images (i.e., the ratio between the noisy and the restored images) are displayed below the restored images. Almost no structure can be identified by visual analysis of the residual images, which means that the compared methods preserve very well the geometrical content of the original images (limited over-smoothing). We suggest evaluating the bias E[ R -R], where the average is computed on 20 restoration results obtained from 20 different speckle realizations. Of course, the estimator should also reduce the noise level. This reduction can be locally analyzed by displaying the standard deviation of the estimator R normalized by R (relative standard deviation). In constant regions, the despeckling is very effective and we observe a small value of σ[ R]/R. Close to edges or in very textured regions, the despeckling is less efficient: the estimator is influenced by the speckle fluctuations and larger values of σ[ R]/R are observed. Structures that are too thin to be preserved when filtering a single look image are visible in the bias image (i.e., they are removed in the restorations for all/most speckle realizations). The area encircled in red in the bias images of figure 6 corresponds to a portion of a road (visible as a dark thin line in the ground truth SAR image) that is generally slightly better preserved by the SAR-CNN method. In the bias images, the bias is closer to zero in this area for the SAR-CNN method.

It can be observed both on the quantitative results reported in the tables and in the qualitative analysis that the CNN methods (the two versions of MuLoG+CNN and SAR-CNN) perform better than MuLoG+BM3D which we use as reference algorithm in SAR despeckling before the introduction of CNN methods. SAR-CNN removes speckle from the images while preserving the details, such as edges, at the cost of introducing small but noticeable artifacts in homogeneous areas. Instead, MuLoG+BM3D and MuLoG+CNN generate blurry edges, over-smoothing some areas where the details are lost, even when the CNN is pre-trained on SAR images. This can be observed comparing the denoised images of Figure 3.7, where SAR-CNN preserves better the details of the urban area at the bottom of the image compared to the three other denoising methods. The quality of the details can be attributed to the richness of information captured by the network when learning on many SAR image patches.

It also appears by analyzing the bias and relative standard deviation figures that MuLoG+CNN and SAR-CNN are the ones that introduce the smallest biases in the estimated images. Pre-training the CNN on SAR images brings only a small improvement over pre-training on natural images. However, a smaller bias is introduced: over the whole image, the sum of the bias is close to zero. The relative standard deviation is the smallest for SAR-CNN. Figure 3.8 shows the results on one speckle realization of images listed in table 3.4 and 3.5. This allows to have a qualitative interpretation of the scores provided in the tables and to appreciate the fidelity of the despeckled image to the reference noise-free images.

Despeckling of Real Single-Look SAR Images: reduction of the spatial correlations

In this section, the denoising performance of SAR-CNN for Sentinel-1 image despeckling and MuLoG+CNN are evaluated on real single-look SAR images acquired during Sentinel-1 mission. To test our deep learning-based denoiser, we focused on some of the areas of the images analyzed in the above tables, picking one of the multitemporal instances used to generate the ground truth images for the training of SAR-CNN and making sure that these areas do not belong to the training set.

Unlike synthetically generated noisy SAR images, in real acquisitions, pixels are spatially correlated. SAR images undergo an apodization (and over-sampling) process [START_REF] Pastina | Effect of apodization on SAR image understanding[END_REF][START_REF] Herbert C Stankwitz | Nonlinear apodization for sidelobe control in SAR imagery[END_REF][START_REF] Abergel | Subpixellic Methods for Sidelobes Suppression and Strong Targets Extraction in Single Look Complex SAR Images[END_REF] aimed at reducing the sidelobes of strong targets, by introducing some spectral weighting (a more detailed discussion on this matter is provided in Section 3.4.1). Thus, we subsample real SAR acquisitions by a factor of 2 to reduce speckle spatial correlations. As already observed in the case of synthetic SAR data, the images that are restored with the proposed methods show significant improvements on the denoising performance over the reference despeckling algorithm MuLoG+BM3D, with SAR-CNN being the one that provides the best visual result. Even when compared to SAR-BM3D and NL-SAR, which do not require the image to undergo a downsampling step, our results are visually more pleasant, with a better preservation of fine structures. NL-SAR, indeed, gives its best in polarimetric and interferometric configurations. Figure 3.9 shows the restoration results and the residual images (obtained by forming the ratio noisy/denoised) obtained with the different methods. In contrast to the simulated case, some structures can be visually identified in the residual images. These structures correspond to thin roads. The downsampling operation necessary to reduce the speckle correlations make the preservation of these structures very difficult for all methods. SAR-CNN seems to be the most effective at preserving those structures. Indeed, the residual image does not present clear textures nor constant areas, meaning that no structures have been removed and that noise there is no noise left in the restored image (the residual image represents what is suppressed from the noisy image). Visual analysis of the restored images compared to the real image also seems to indicate fewer artifacts with SAR-CNN. Since the ground-truth reflectivity is not available, to measure the performance of the proposed method the ENL is estimated on manually selected homogeneous areas. The homogeneous regions chosen for the ENL estimation are shown with red boxes, and the ENL values are given in Table 3.6. Then, this analysis has been extended to a TerraSAR-X acquisition. In this case, the study aims at assessing the generalization capabilities of the trained SAR-CNN on images from a different sensor and a different spatial resolution. While MuLoG+CNN is not dedicated to a specific sensor (see Table 3.7), SAR-CNN is sensor specific. As such, to give its best on TerraSAR-X images, it should be retrained using images acquired from TerraSAR-X: a dataset can be constructed as described in Section 3.2.3. As it can be seen from Figure 3.10, MuLoG+CNN seems the approach that provides visually the best results. The estimated ENL indicate that all despeckling methods are very effective in homogeneous regions. It seems that MuLoG+CNN produces an image with a slightly better perceived resolution, which may indicate a better generalization property compared to SAR-CNN. While the latter would benefit from a dedicated training on TerraSAR-X images, it achieves satisfying performance on this data without any finetuning, which indicates that it could be readily applied to images from other sensors in the absence of time series to build a ground truth for retraining. The analysis of the residual images, like in Figure 3.9, indicates that thin linear structures are attenuated in restored images due to the downsampling step.

Discussion

A CNN trained to suppress additive white Gaussian noise encodes a very generic model of natural images in the form of a proximal operator related to an implicit prior. Like other models of natural images that were successfully applied to the problem of speckle reduction in SAR imaging (wavelets, total variation), they are relevant to SAR imagery because they capture structures (points, edges, corners) and textures. Yet, the specificities of SAR images make it beneficial to train a model specifically on SAR images. This is done naturally by patch-based methods that use the content of the image itself as a model (repeating patches). In this chapter, we have shown a notable improvement with a CNN model trained on SAR images provided that a high-quality training set is built, enough layers are used to capture large scale structures and an adequate loss function is selected. Moreover, once trained, SAR-CNN exhibits the best runtime performance when using a Graphics Processing Unit (GPU) (see Table 3.7). When considering the real-life case of partially correlated speckle or images from different sensors, the plugging of a network trained on natural images within MuLoG provides better generalization properties.

The extension to multi-channel SAR images represents a real challenge. Speckle reduction in multi-channel images requires modeling the correlations between channels (the interferometric and polarimetric information). In order to learn those correlations directly from the data, a dataset that contains speckle-free images covering the whole diversity of polarimetric responses, interferometric phase differences and the whole range of coherences for typical geometrical structures (points, lines, corners, homogeneous regions, textured regions) must be formed. Needless to say, this is far more challenging than collecting single-channel SAR images to cover only the diversity of geometric structures. Failure to correctly include all cases in the training set implies that the network, instead of performing a high-dimensional interpolation, performs a high-dimensional extrapolation, which puts the user at high risk of experiencing large prediction errors. This difficulty justifies the relevance of using pre-trained networks within MuLoG framework which has been designed to apply single-channel restoration methods to multi-channel SAR images. A summary of the advantages and drawbacks of the proposed methods are reported in Table 3.8. Only a little gain in performance is observed with MuLoG+CNN when the network is pre-trained specifically on SAR images. Thus, if creating a dataset is possible, it is advisable to use it to train an end-to-end model Figure 3.11 -Results obtained by filtering with SAR-CNN some real Sentinel-1 SAR images, whose ENL is given in table 3.6. To reduce speckle spatial correlations, images have been subsampled by a factor of 2 before feeding them to the network. Bilinear interpolation has then been used to keep the original image size. such as SAR-CNN. This conclusion is a first answer to Challenge (1) Training set.

To offer the possibility to use the presented SAR-CNN for testing and comparison, addressing Challenge (2) Open code, we released an open-source code of the network trained on our dataset: https://gitlab.telecom-paris.fr/RING/SAR-CNN. Indeed, replicating results of a published work is not an easy task and may represents up to months of work. Therefore, by sharing our code, we hope to help other researchers and users of SAR images to easily apply our CNN-based denoiser on single-look Sentinel-1 images, and possibly compare the restoration performance with their own methods. A simple method to reduce the impact of the spatial correlations of speckle is to downsample the SAR image, as it has been done in the previous section. A good compromise between reduction of correlation and preservation of resolution is to apply a downsampling factor of 2, as shown in Fig. 3.12, right column. The artifacts are suppressed, however the resolution is not as good as the resolution achieved on the numerical simulation where no downsampling was necessary.

Another example of artifacts due to spatial correlations of the speckle can be seen in the paper (Wang et al., 2017b), where PPB algorithm and SAR-CNN do not perform as expected because no downsampling is applied to the real image.

These examples show that it is essential to ensure that the speckle has no significant spatial correlations prior to applying some speckle reduction methods from the literature. In the next section, alternative solutions to downsampling are analyzed.

Existing strategies to denoise SAR images with spatially correlated speckle

One possibility to successfully apply algorithms based on Goodman's fully developed model and the i.i.d. assumption is to revert the steps performed by the data providers.

In [START_REF] Abergel | A complex spectrum based SAR image resampling method with restricted target sidelobes and statistics preservation[END_REF] and [START_REF] Abergel | Resolution-preserving speckle reduction of sar images: The benefits of speckle decorrelation and targets extraction[END_REF], the authors provide a method that preserves the spatial resolution by correctly resampling SAR images and extracting bright targets. A so-called pseudo-raw image is then recovered, which is the image that would have been acquired if the data was sampled at the Shannon-Nyquist sampling frequency and no spectral weighting was applied (i.e., no apodization), see Figure 3.14.

Based on the knowledge of the parameters of the sensor contained in the metadata of the images, deramping, demodulation and deapodization can be carefully computed to obtain an image where, in homogeneous regions, speckle presents almost no spatial correlation [START_REF] Abergel | Resolution-preserving speckle reduction of sar images: The benefits of speckle decorrelation and targets extraction[END_REF]. The pseudo-raw image can then be filtered using standard speckle reduction methods, as illustrated in Figure 3.13. The over-sampling factor be re-applied to the resulting image in order to obtain an image comparable to the original image. Then, the discrete Diracs initially extracted can be re-injected to produce the final denoised estimate, avoiding the side-lobes of strong scatterers to hide close structures. Some experiments have been performed during the Master-2 internship of Max Muzeau that I co-supervised together with Florence Tupin and Loïc Denis. Some results of this approach are displayed in Figure 3.15. While filtering an image with the target component can lead to a loss of some information, removing strong scatterers before decorrelation and filtering allows to preserve the bright points. However, reintroducing the discrete Diracs directly to the resampled denoised result leads to an image less interpretable: indeed, human observers are used to look at the sidelobes to spot the presence of strong reflectors. This approach applies pre-and post-processing steps that require the knowledge of sensor's parameters and are sensor-specific. In [START_REF] Lapini | Blind speckle decorrelation for SAR image despeckling[END_REF], Lapini et al. propose a blind speckle decorrelation algorithm. After having detected and removed the point targets according to a threshold empirically set, least square (LS) optimization is performed to estimate and invert the point spread function (PSF) of a SAR acquisition system. At this step, the reflectivity can be estimated by filtering the image with a despeckling algorithm developed under the uncorrelated speckle hypothesis. As the PSF is applied independently on each polarimetric channel, Arienzo et al. [START_REF] Arienzo | Accurate despeckling and estimation of polarimetric features by means of a spatial decorrelation of the noise in complex PolSAR data[END_REF] have extended this method to PolSAR data. These methods, however, come at a computational cost. If one wants to process an image in a more automatic and straightforward way, the despeckling algorithm has to include some robustness to spatial correlations.. No standard exist yet in deep learning for SAR image despeckling, which also makes it difficult to compare different architectures and reproduce the published results. In the next section, we analyze the impact of including in the training loss the TV, defined in equation (3.3), to attenuate the effect of the correlation. This loss is proposed by (Wang et al., 2017b) and we investigate its effectiveness in handling speckle spatial correlations by relying on the dataset and the training strategy described in section 3.2.3.

Deep Learning for SAR image despeckling: training methodology and robustification with TV

To obtain the results that are illustrated in this section, we implemented the training strategy presented in Section 3.2.3: a speckle-free reference is created by averaging a multitemporal stack of images acquired over the same scene and the remaining speckle fluctuations are suppressed using MuLoG+BM3D with the appropriate equivalent number of looks. Synthetically generated speckled images are then created by following Goodman's model and the i.i.d. assumption and using the speckle-free reference images as images of the reflectivities R.

To study the role of a TV term in improving robustness to speckle correlations, we consider analyzing the performances of SAR-CNN architecture. While images with simulated speckle are used during training, real single-look images (acquired over areas not belonging to the training set) are used in the testing set. The network is trained using two different loss functions in order to analyze their differences. In the first case, an 1 loss is used:

L 1 = N i=1 f CNN (y i ) -x i -(ψ(L) -log(L)) • 1 1 (3.2)
where y i and x i are a pair of log-transformed noisy and clean amplitude images. Bias is corrected at the network's output. We define the smoothed TV term on the denoised image f CNN (y i ) by:

L TV = p,q (f CNN (y i ) p+1,q -f CNN (y i ) p,q ) 2 + (f CNN (y i ) p,q+1 -f CNN (y i ) p,q ) 2 + 2 1/2 , (3.
3) where is a parameter that is small compared to the typical contrast between logtransformed values. A joint loss that combines the two previous losses is defined by:

L 1 +TV = L 1 + λL TV (3.4)
The use of the TV term is justified by its effectiveness in reducing spurious details, characterized by a high total variation, that strongly affect the estimations of the speckled image when speckle is correlated. Penalizing reconstructions with a large Total Variation improves the robustness while preserving sharp edges (which have a low total variation).

The two loss functions L 1 and L 1 +TV are tested on images with synthetic noise (Fig. 3.16) and real images (Fig. 3.17). Adding the total variation terms has the effect of smoothing the results. In the simulations with synthetic speckle, this leads to the loss of some small details. However, when real images with correlated speckle are considered, the network trained with the combined loss L 1 +TV leads to results with far fewer artifacts. In order to obtain this robustness, a large value of λ has been chosen (λ = 1.2). Compared to the sub-sampling strategy, the results seem slightly worse. In particular the bright targets in Fig. 3.17 are well-preserved by SAR-CNN applied to a down-sampled image while they disappear when the TV term is employed.

Figure 3.17 -Results of MuLoG+BM3D and the deep learning method SAR-CNN on a real 1-look Sentinel-1 image

Discussion

In section 3.4 we investigate different strategies already proposed in the literature that one can adopt when dealing with correlated SAR images. Downsampling the image may seem to be a crude method, as it leads to an image with a poorer resolution, yet at the end it provides the best results when single-look images are considered.

When training a neural network with the TV term in the loss function as proposed by (Wang et al., 2017b), the robustness to speckle correlations is improved, but this requires choosing a high value for the regularization parameter to avoid spurious structures in single look images. As a consequence, small details are not well preserved. This strategy seems more suitable when the number of looks is larger (see results proposed in (Wang et al., 2017b)): this is the case of Ground Range Detected (GRD) images, whose ENL is around 4. A much lower regularization parameter is then required to obtain results that are immune to the spatial correlations of speckle, and hence much fewer small details are lost in the restoration process.

This study shows the interest of train a deep learning model specifically to denoise correlated data. The strategy proposed to create an ad-hoc training set of SAR images can be extended to include the resampling method discussed in section 3.4.1, allowing the learning from images generated with a correlated speckle noise. However, each SAR sensor has a specific apodization function. Moreover, some image modalities, such as TerraSAR-X SpotLight [START_REF] Eineder | Spaceborne spotlight sar interferometry with terrasar-x[END_REF] and Sentinel-1 TOPS (Yagüe-Martínez et al., 2016), are not linear spatial-invariant system, making the creation of realistic SAR images a difficult task. On top of that, to create reference images one needs to have access to long co-registered SAR time series (of at least 25 images). Depending on the policy of the data provider, this is not always feasible.

An alternative approach would be to exploit couples (or series, when available) of SAR images to learn to reduce speckle noise directly from the real SAR images. A preliminary work relying on self-supervised approaches has been done by [START_REF] Boulch | Learning speckle suppression in SAR images without groundtruth: application to Sentinel-1 time series[END_REF]. In the next chapter, we will see how the recent advances in self-supervised approaches for image denoising [START_REF] Lehtinen | Noise2noise[END_REF] offer a possibility to adapt such framework to suppress speckle from real SAR data, thus with spatially correlated speckle.

Summary of the chapter

Summary: Training of supervised models for SAR despeckling SAR time series are used to produce a dataset of speckle-free reference images. Such groundtruth images have been used to:

• train a CNN in an end-to-end fashion for speckle reduction

• develop a hybrid model: a network for AWGN suppression is trained on SAR-like images, learning specific priors, and is then integrated in a despeckling framework Speckle spatial correlations seriously limit the performances of methods that do not account for system's PSF. We have studied the performances of three existing strategies:

• Adding a TV term in the loss function integrates knowledge of speckle spatial correlations during training. However, a network trained with such a loss fails to recover fine details.

• Removing the sensor's transfer function requires to properly re-introduce strong scatterers. When despeckling the decorrelated image, the background is well recovered, but the discrete Diracs added in a post-processing phase produce a non-natural result.

• Correlation is reduced by subsampling the image by a factor of 2. The subsampled image is denoised and then upsampled. While halving the resolution, the method is fast and, however not satisfying, seems to be a good compromise between quality and complexity. To overcome the shortcomings pointed out in Section 3.4, a neural network can be trained on images corrupted by spatially correlated speckle. Two learning paradigms can be considered to this aim: supervised approaches and self-supervised approaches.

In a supervised training strategy, one needs to produce speckle-free images used as groundtruth for the learning algorithm. This would imply recovering a pseudo-raw reference image, i.e. a speckle-free image which is not spectrally apodized nor oversampled. Section 3.4.1 illustrates the difficulties of re-introducing bright targets on whitened images. Moreover, fine knowledge of the SAR system is required to properly synthesize images with spatially correlated speckle. Therefore, in this chapter we propose instead to move towards a semi-supervised algorithm: SAR2SAR. By learning how to suppress speckle directly from real SAR images, the neural network automatically develops robustness to spatially correlated speckle.

A preliminary study on a learning strategy relying only on speckle-corrupted SAR images is done in [START_REF] Boulch | Learning speckle suppression in SAR images without groundtruth: application to Sentinel-1 time series[END_REF]. Meanwhile, noise2noise [START_REF] Lehtinen | Noise2noise[END_REF] demonstrates that by only looking at corrupted examples it is possible to suppress AWGN as efficiently as if clean examples were accessible during training. Inspired by these two works, in this chapter we address the lack of noise-free references by extending the noise2noise approach in order to take into account the peculiarities of SAR data. Thanks to a well-grounded loss function formulation, which derives from the speckle model, large stacks of multi-temporal images acquired over the same area are exploited in two ways. In the first instance, we rely on the dataset of speckle-free images described in chapter 3 and our U-Net model is trained with synthetic speckle, following Goodman's fully developed speckle model [START_REF] Goodman | Speckle phenomena in optics : Theory and applications[END_REF], see Section 2.2. At a later stage, we feed the network with real acquisitions, allowing learning of the spatial correlation introduced by the SAR system, namely spectral apodization and oversampling. This answers to Challenge [START_REF] Miranda | Definition of the TOPS SLC deramping function for products generated by the S-1 IPF[END_REF] Training set and Challenge (3) Speckle spatial correlations. The problem of temporal changes is addressed by a strategy for change compensation. This ensures robustness and generalization of the algorithm, since any temporal series of SAR images can be used at this transfer learning step.

Contribution (2): Semi-supervised learning of spatially correlated speckle

• Training of a network that learns to suppress speckle by only looking at speckle-corrupted images

• Assessment of the quality of SAR2SAR with an example of thin river detection from GRD SAR products

• Extension of the proposed method to multi-temporal SAR despeckling

• Publication of an open-source code for testing and comparison on SLC and GRD data (see appendix E).

Semi-supervised image restoration: from Gaussian denoising to speckle reduction

Recall: Speckle model

The intensity I measured by a SAR sensor is linked to the hidden reflectivity R and the speckle component S by a multiplicative relation [START_REF] Goodman | Speckle phenomena in optics : Theory and applications[END_REF]:

I = R × S,
where the speckle component follows a Gamma distribution with unitary mean and variance inversely proportional to the number of looks L:

p(S) = L L Γ(L) S L-1 exp (-LS) .
(2.7)

The log-speckle s = log(S) is distributed according to a Fisher-Tippett distribution of mean E[s] = ψ(L) -log(L) and variance Var[s] = ψ(1, L):

p(s) = L L Γ(L)
e Ls exp(-Le s ).

(2.9) In [START_REF] Lehtinen | Noise2noise[END_REF], the authors adopt a modified U-Net model. Originally developed for medical image segmentation [START_REF] Ronneberger | U-net: Convolutional networks for biomedical image segmentation[END_REF], it proved to be effective for denoising purposes.

To study the relevance of using the U-Net for SAR despeckling, we first employ the architecture depicted in Figure 4.1 in the supervised strategy presented in the previous chapter and compare it to the performances obtained with SAR-CNN (Section 3.3). To do so, both networks are trained in a residual fashion on the same dataset of specklefree images, that are corrupted with synthetically generated speckle to produce pairs of training images. Then, an extensive evaluation of the results in carried out, such as in Section 3.3. By looking at the PSNR quality score of Table 4.1, U-Net achieves an overall better performance than SAR-CNN. Visual inspection of the results of Figure 4.2 confirms what is inferred from Table 4.1.

Moreover, from our experiments we observed that U-Net is 16 times faster at training time (reducing the computational burden needed to complete the training step) and generalizes well, which is an interesting property for tasks such as SAR despeckling where the number of training samples can be limited. Thus, to develop the training strategy that we present in this chapter, we rely on the U-Net architecture of In the supervised learning setting, pairs (x, y) of noiseless and noisy images are available for training. A common approach to estimate the parameters θ of an estimator f θ :

y → x = f θ (y) is to minimize the L 2 loss function: θ(L 2 ) supervised ∈ argmin θ E X E Y |X f θ (y) -x 2 , (4.1)
where x is a random realization of the random vector X and y is a random realization under the conditional distribution p Y |X . The self-supervised approach noise2noise introduced by Lehtinen et al. [START_REF] Lehtinen | Noise2noise[END_REF] considers only noisy pairs (y 1 , y 2 ), where y 1 and y 2 are two independent realizations drawn under the same conditional distribution p Y |X . The authors suggest replacing the unknown realization x with the noisy observation y 2 given that it is much easier to obtain additional noisy measurements of a static scene rather than very high quality measurements (i.e., virtually noise-free images):

θ(L 2 ) self-supervised ∈ argmin θ E X E Y |X f θ (y 1 ) -y 2 2 . (4.2)
The rationale behind this substitution comes from the conditional expectation of the expansions of the squared L 2 norms in equations (4.1) and (4.2):

E Y |X f θ (y) -x 2 = E Y |X f θ (y) 2 -2x t E Y |X f θ (y) + x 2 , (4.3) E Y |X f θ (y 1 ) -y 2 2 = E Y |X f θ (y 1 ) 2 -2 E Y |X y 2 t E Y |X f θ (y 1 ) + E Y |X y 2 2 . (4.4)
Provided that the noise is centered, i.e. E Y |X y 2 = x, the two expansions differ only by a term that is constant with respect to the parameters θ. Therefore, if the training set is large enough, parameters estimated with the self-supervised procedure of (4.1) are equivalent to parameters estimated with the supervised procedure (4.2).

In practice, training sets are limited and it is therefore necessary to consider how fast the self-supervised estimator converges to the supervised estimator. Under non-Gaussian noise, other loss functions may be more efficient. This is in particular the case of the co-log-likelihood:

θ(lik) self-supervised ∈ argmin θ E (X,Y ) -log p(y 2 |f θ (y 1 )) . (4.5)
Among the M-estimators, i.e. methods to estimate parameters θ based on the minimization of a loss function over the training set, the maximum likelihood estimators are known to be efficient [START_REF] Harry | Detection, estimation, and modulation theory, part I: detection, estimation, and linear modulation theory[END_REF]). This is illustrated in the case of speckle in figure 4. [START_REF] Baqué | RAMSES-NG: New performances of the flexible multi-spectral airborne remote sensing research platform[END_REF], where the root mean square error1 of the log-intensity is reported for the L 2 and the log-likelihood loss functions. The minimizer of the log-likelihood loss converges more quickly to x, which indicates that it should be preferred as a loss function for self-supervised training of a despeckling network and is confirmed in our experiments described in section 4.3.

Application to SAR image despeckling: a semi-supervised approach

When y 1 and y 2 are speckle-corrupted log-intensity images, Section 2.2 and Figure 2.3 recalled that the conditional distribution p Y |X (y|x) is a Fisher-Tippett distribution.

The application of the operator H associated to the SAR transfer function spatially correlates the speckle. We denote with y1 and y1 log-intensity images corrupted by spatially correlated speckle, i.e. images y 1 and y 2 after the application of the operator H. The marginal distribution of the log-intensity yk = log Ĩk at pixel k is still Fisher-Tippett The loss function in (4.5) takes the form:

L f θ (y 1 ), y2 = - k log p([y 1 ] k |f θ ([y 2 ] k )) = k f θ ([y 1 ] k ) -[y 2 ] k + exp [y 2 ] k -f θ ([y 1 ] k ) , (4.6)
where the constant offset and the multiplicative factor L are dropped since they are irrelevant in the minimization problem (4.5), and the sum involves all the pixel values The proposed SAR restoration method, named SAR2SAR since it extends on the original idea of noise2noise, considers several SAR time series, each accurately co-registered, and performs the training of a despeckling network using both the idea of the self-supervised loss of equation 4.6 and of change compensation. Figure 4.5 summarizes the principle of the method: the restoration is performed in the log-domain by a deep network. Since the change compensation requires the availability of preestimated reflectivities, the training of the network is performed in 3 steps: (A) first on images with synthetically generated speckle, (B) then on pairs of images extracted randomly from a time-series, the second image being compensated for changes based on reflectivites estimated with the network trained in (A), (C) finally a refinement step is performed where the network weights in (B) are used to obtain a better compensation for changes. As part (A) is not, in the strict sense of the term, self-supervised, we refer to SAR2SAR as a semi-supervised algorithm.

Experiments

In our set of experiments, our model is the U-Net (Ronneberger et al., 2015) described in [START_REF] Lehtinen | Noise2noise[END_REF], trained in a residual fashion [START_REF] He | Deep residual learning for image recognition[END_REF]. Images are fed to the network after a log transform. Thus, the network reproduces the noise, which is subtracted from the input image. The despeckled image is obtained as a result.

Synthetic speckle noise

One of the main issues when using deep learning algorithms on SAR images is the scarcity of training data. To achieve the desired level of generalization and given that the application of the presented algorithm to time-series needs an accurate adaptation, training is initially carried out on images corrupted by synthetic speckle noise. At each iteration, two independent speckle realizations (following the model described in section 2.2) are used to create two noisy images, one being the input image and the other one to compute the loss. The images are divided into patches of 256 × 256 pixels, with a stride of 32. 3014 batches of 4 images compose our training set. The network has been trained for 30 epochs using the Adam optimizer, with a learning rate set as 0.001 and decreased by a factor of 10 after the first 5 epochs and by a factor of 100 after the first 10 epochs. The loss function of our SAR2SAR method is adapted to the distribution of SAR images using equation 4.6.

Creating synthetic images from noise-free references, moreover, allows a quantitative evaluation. Results of several despeckling filters are presented in table 4.2. The PSNR values on the SAR2SAR are not only comparable to those obtained with SAR-CNN (Dalsasso et al., 2020), but are superior to the use of L 2 (with the proper debiasing step, as discussed in section 4.2.3), justifying the adaptation of the loss: see the last three columns of table 4.2. Results on image Lelystad and Saclay are displayed in figure 4.6. While the use of SAR2SAR is motivated by its direct application on real SAR images, even on images corrupted with synthetic speckle noise it achieves state-of-the-art results.

Real SAR images

To fine-tune the network on real images, the SAR time series composing the training need to be denoised to generate the images used to compensate for changes. An estimation can be obtained by using the network trained on synthetic speckle, after subsampling the images to reduce the effect of the correlation ( As the compensation images used at this step required a subsampling operation, they have a poor resolution that impact the results produced by SAR2SAR. To overcome this issue, an iterative process has been studied: every 10 epochs the compensation images given to the network are updated with the results of SAR2SAR itself. Given that, asymptotically, the function learned by the network tends to the identity, we found experimentally that one iteration is a good compromise between speckle reduction and improvement of the resolution (see table 4.3). Results are shown in figure 4.7. In the absence of reference images, standard image quality metrics cannot be com-puted. A quantitative evaluation can be performed by estimating the ENL, defined as ENL = E[ R] 2 /Var[ R], on a manually selected homogeneous area. The higher this metric, the stronger the speckle reduction. Since the ENL favors filters that perform an over-smoothing, it cannot be used as a sole indicator for the quality of noise reduction. Together with visual inspection, it seems fair to say that SAR2SAR leads to the best restoration quality. Additional results comparing SAR-CNN and SAR2SAR are available in Appendix D. Visual inspection of images shown in figure 4

Ablation study

To demonstrate the impact of the pre-training step with synthetically generated speckle noise and the importance of change compensation, an ablation study is conducted and results on two Sentinel-1 image patches are shown in Figure 4.9.

The use of a change compensation mechanism plays a key role in the effective exploitation of multi-temporal SAR images. Training directly on image pairs without compensating for change leads to poor results with notable bias in some areas (Fig. 4.9.a and Fig. 4.9.d). Even when only the closest date is selected to minimize changes, the lack of change compensation is penalizing and additional artifacts also occur due to the non-negligible temporal correlation of speckle (Fig. 4.9.b and Fig. 4.9.e). In particular, the result of Fig. 4.9.b has been obtained by training the network in a scenario close to that considered in [START_REF] Boulch | Learning speckle suppression in SAR images without groundtruth: application to Sentinel-1 time series[END_REF], who were the first proposing a selfsupervised strategy for speckle reduction. An example of time series of SAR images acquired approximately every 12 days is provided in Appendix C: frequent changes can be observed, showing the necessity of taking them into account.

When the network is not pre-trained, i.e., there is no step A with synthetic speckle and only pairs of actual SAR images are used, the restoration performance worsens (left part of Fig. 4.9, in particular Fig. 4.9.c). A reason for this drop in image quality may be that the learning is guided uniquely by the change-compensated image. Since the method to compensate the changes is not perfect, the images that drive the estimation of the network weights are of lesser quality: indeed, they have been subsampled before being filtered with SAR-CNN. This could possibly be mitigated by considering much larger training sets or by improving the change compensation step (this would however require to restore well a SAR image, resulting in a "chicken and egg" problem).

Warm-starting the network using simulated data allows to create a virtually unlimited number of noisy image pairs of the same area. When trained this way, the model is competitive with other state-of-the-art despeckling filters (see Table 4.2) and only needs a slight adjustment to gain robustness to the spatial correlations of the speckle. It can then be used to produce pre-estimates of the SAR reflectivities to compensate for temporal changes, and refined on real images with our two-steps self-supervised strategy, where step (C) removes the initial limitations due to the down-sampling procedure used to decorrelate the speckle.

Discussion

Single-look SAR images are difficult to denoise due to the strong speckle variance and a mismatch between Goodman's model of speckle and the real model (Challenge (3) Speckle spatial correlations). The proposed SAR2SAR algorithm learns how to suppress real speckle directly from the data, by devising a self-supervised algorithm leveraging deep learning and multi-temporal stacks of images.

While providing state-of-the-art results on images with synthetic speckle noise, it is on real single-look images that SAR2SAR shows a clear improvement over existing despeckling algorithms. The methods developed under fully developed speckle model assumptions, indeed, need a careful adaptation in order to properly deal with correlated data (Dalsasso et al., 2021a). As discussed in Section 3.4, a careful pre-processing to whiten the speckle requires knowledge of the sensor's parameters and adds a computational burden. Moreover, target re-introduction produces an artificial result: see second column of Fig. 4.10. If a subsampling step is applied, images with a poorer resolution are produced. SAR2SAR learns to suppress speckle directly from the data, making it readily applicable on real images. The advantage of deep learning algorithms, moreover, is that they are computationally fast once they are trained.

The general formulation of SAR2SAR suggests that it can be extended to GRD images (as we will show in Section

Adaptation to GRD data

In various applications related to Earth observation where one does not need interferometric information from the SLC images, Sentinel 1 Ground Range Detected Interferometric Wide swath High definition Dual polarization (GRD IW HD) images provided by ESA are very popular because of their large availability and ease of use compared to other SAR data. GRD products are Level-1 data, just as SLC data. However, the phase information is lost as the GRD product is obtained by incoherent combination (multi-looking) of pixels of a focused SAR data in order to project the image to ground range, relying on a Digital Elevation Model (DEM) and on the Earth ellipsoid model. As a result, speckle reduction is traded for a loss of resolution: GRD data are characterized by an ENL of 4.4 and have approximately square pixels. Although fluctuations are reduced, GRD products remain challenging to exploit, as speckle is still a limit to their interpretation. Removing speckle from GRD images can be sensitive, especially given the spatially variable correlation of speckle. Thus, specific techniques need to be built to this aim.

One can ask whether such a denoising step can be beneficial even for approaches that have been designed to be robust to speckle noise. In this section, we present a modified river detection method that is based on a first denoising step adapted from SAR2SAR (Dalsasso et al., 2021b) and a detection performed on the denoised images using a method that uses exogenous information to guide the river detection (Gasnier et al., 2021b).

The proposed method consists of two steps: the first step filters the speckle from GRD images using an adaptation of SAR2SAR. The second step segments the narrow rivers on these filtered images.

Adaptation of SAR2SAR to GRD images

Following the same principle outlined in Section 4.2.3, a U-Net [START_REF] Ronneberger | U-net: Convolutional networks for biomedical image segmentation[END_REF] network has been initially trained on the same speckle-free reference images on which speckle with L = 4 is synthetically created. In this way, pairs of GRD images having the same content but independent speckle realizations are obtained and the network is trained only using speckle-corrupted images (step A). In order to learn specific characteristics of GRD images, in particular the spatially-varying correlations, the network is fine-tuned on co-registered stacks of GRD images (step B). To compensate for changes occurring between images y 1 and y 2 , the target image is replaced by y 2 -x2 + x1 . The estimates of the reflectivities x1 and x2 are obtained thanks to network A. Since network A was trained only on decorrelated speckle, Sentinel-1 images are downsampled by a factor 2 to reduce the impact of speckle correlations. The outputs of network A are then up-sampled by a factor 2 to produce estimates x1 and x2 . After this first fine-tuning step on real data, the network learned at step B is used to produce more accurate estimates x1 and x2 in the change compensation formula (the downsampling is no more necessary since network B, directly trained on Sentinel-1 images, is robust to speckle correlations). A last network is obtained at the end of this refinement: network C. At step A, the image has been pre-processed through a subsampling by a factor of 2 to reduce speckle spatial correlation. At steps B and C, the network is refined on real images and it is thus robust to spatially correlated speckle: thus, images are processed at their full resolution, obtaining a higher level of detail preservation and an enriched information.

Detection of narrow rivers in denoised GRD images

Please note that the work described in this subsection has been carried out by Nicolas Gasnier. It is however recalled in this section as it allows to draw some conclusions on the quality of images estimated using SAR2SAR.

The first step of out method consists in suppressing speckle from GRD images. The goal of the second step is to get an accurate segmentation of the rivers in the image. To achieve this, we adapted a framework (Gasnier et al., 2021b) that uses first a linear features detector and exogenous information to retrieve the river centerline. Then the river is segmented around the centerline using a specific Conditional Random Field (CRF) approach. The exogeneous information on the river consists of control points and can be found in prior databases such as Global River Widths from Landsat (GRWL) [START_REF] Allen | Global extent of rivers and streams[END_REF] in which the river centerlines are stored as sets of nodes. Such information can be of a great help to distinguish rivers from other dark linear structures that are present in Sentinel-1 images, like large roads. However, due to discrepancies between the database centerline and the actual rivers on the image, both in shape and position, these river centerlines from the database cannot be directly used to segment the river, hence the need for a repositioning stage.

Linear features detection on denoised GRD images

The linear features map is computed using a linear features detector based on the Generalized Likelihood Ratio Test (GLRT) (Gasnier et al., 2021c). As this detector is agnostic to the number of looks L, no modification was needed to adapt it to denoised GRD images.

River centerline determination as the least cost path between control points

As the linear features detected in the previous step can correspond to actual rivers, but also to roads, layover areas, shadows, or even to false-positive detections caused by corner reflectors, the second stage uses exogenous information from a river database to determine the shape and the position of the river centerline in the image. This stage is identical to its counterpart in (Gasnier et al., 2021b) and consists in detecting the centerline as the least-cost path between two nodes that belong to the same river in the exogenous database and are a few kilometers apart. The cost array is computed from the previously computed linear features detector response using the same parameter N pow = 10, as for noisy GRD images and with D max being the maximum value of D in the image.

River segmentation around the centerline

The last stage of our method consists in segmenting the river around the centerline obtained in the previous step, using a CRF approach adapted from (Gasnier et al., 2021b), with a simplified expression. The CRF energy is defined as the sum of a data term, a term that forces centerline pixels to be classified as water, and a regularization term that all depend on the class (land or river) of each pixel.

The data term depends on the denoised image intensity I and on the class: for the river class, the data term is quadratic: (log(I) -R log ) 2 , where R log is the estimated log-reflectivity of the water. This distribution accounts for the fluctuations of river pixel intensities caused by the remaining speckle, if any, and also for the fluctuations of the river reflectivities caused by varying roughness of the water surface.

The data term for noisy SAR images processing usually derives from a gamma distribution (to model speckle fluctuations). Variations of intensity caused by the spatial evolution of water reflectivity are considered negligible compared to that of speckle. This is no longer true when considering despeckled images. We therefore prefer a quadratic data term to account for spatial changes in water reflectivities. In the absence of a model for the land class and in order to prevent a bias toward it, the data term for the land class is set constant and proportional to the mean value of the data term, computed on all pixels of the centerline of the river, excluding the highest values. We keep the same regularization term as in the method (Gasnier et al., 2021b): it consists in a weighted total variation penalization where weights are inversely proportional to the magnitude of the spatial gradient of the SAR image. This penalizes transitions between river and land, except where the gradient is strong. The orientation of the gradient is also considered to align the segmentation only if the direction is right, i.e., corresponding to a river to land transition.

Experimental results

In this section, we present the detection results we obtained by applying the proposed approach (despeckling with SAR2SAR and river detection) to various Sentinel 1 GRD IW HD images with narrow rivers. To quantitatively assess the performance of our proposed method and to compare it with the original approach that does not use a denoising step, we compute three metrics using a manually-defined ground truth: Recall (Rec), which is the proportion of actual water pixels that are classified as water, Precision (Pre) that is the proportion of actual water among all the pixels classified as water, and F-score that is the harmonic mean of the precision and the recall.

The quantitative results are presented in table 4.4 with a comparison with the baseline method (Gasnier et al., 2021b) that does not use any despeckling. For each metric and each image, the best result is in bold font. The comparison shows that the proposed approach improves the detection result over the baseline for all images but one (Redon) for which the F-scores are very close.

The results of the different steps of the method for a crop of image 3 (Gaoual) that shows the confluence between rivers Tomine and Koumba near Gaoual (Guinea) are presented in figures 4.12. Figure 2.D shows the ratio between the noisy image crop and its denoised counterpart, illustrating the very good despeckling performance of the neural network: no noticeable structure is present in the ratio image and the denoised image is very smooth.

Discussion

The approach presented in this section combines and adapts two existing methods to improve narrow rivers detection. First, SAR2SAR has been re-trained with Sentinel-1 GRD time series (Challenge (1) Training set), learning the non-uniform spatial correlation of speckle directly from real data (Challenge (3) Speckle spatial correlations) and how to efficiently reduce its fluctuations. SAR2SAR proves to achieve despeckling results of a high quality: it suppresses speckle noise while preserving fine structures without introducing notable artifacts. This is crucial for many remote sensing applications, among which narrow river detection. The images denoised with SAR2SAR-GRD are used as input for a modified narrow rivers detection method. Qualitative and quantitative experiments have shown that a preliminary denoising step improves the results on the adapted river detection approach, compared to the original method working on noisy data. Once the network is trained, the computational burden for applying a prior despeckling step is not heavy and can thus be easily paid for an overall gain in detection performances.

In the light of these conclusions, the code of SAR2SAR-GRD was made publicly available 2 , to let the remote sensing community benefit from its use to face the numerous challenges in EO applications (Challenge (2) Open code).

Application to multi-temporal time series

Please note that the work described in this subsection has been done in collaboration with Inès Meraoumia.

SAR2SAR has proven to achieve accurate results for speckle reduction in SAR amplitude images. The wide availability of multi-temporal stacks of SAR images can improve even further the quality of denoising. In this section, we propose a simple yet efficient way to integrate temporal information into a deep neural network for speckle suppression. Archives provide access to long time-series of SAR images, from which multi-temporal averages can be computed with virtually no remaining speckle fluctuations: see Section 3.2.3. The proposed method combines this multi-temporal average and the image at a given date in the form of a ratio image and uses SAR2SAR to remove the speckle in this ratio image. This simple strategy is shown to offer a noticeable improvement compared to filtering the original image without knowledge of the multi-temporal average.

Incorporating multi-temporal information with deep learning can potentially significantly improve denoising processes. Notably, stable structures in observed areas such as buildings or roads are clearly visible in multi-temporal averages, even when the contrast with the surrounding area is weak. RABASAR [START_REF] Zhao | Ratio-based multitemporal SAR images denoising: RABASAR[END_REF]) is a framework designed to extend single-image speckle reduction techniques by including a high-quality multi-temporal average image. The multi-temporal mean is referred to as a "super-image", owing to its high signal-to-noise ratio, and represents a summary of the multi-temporal SAR stack. RABASAR combines a speckled image with the superimage by forming the ratio between the two images. This ratio image has a reduced informational content (only speckle fluctuations and changes with respect to the superimage are present) and is thus easier to denoise. Most of the single-image despeckling methods are designed for spatially uncorrelated speckle and require a spatial resampling (see Section 3.4) when applied to actual SAR data. Utilizing SAR2SAR (Dalsasso et al., 2021b) avoids sub-sampling altogether which preserves the spatial resolution of the restored images.

Ratio-based filtering

The proposed extension is based on RABASAR, a multi-temporal despeckling method (see [START_REF] Zhao | Ratio-based multitemporal SAR images denoising: RABASAR[END_REF]) for a complete description). The key idea is the use of the ratio between a speckled image and the temporal mean of the SAR image stack (named in the following "the super-image" Σ).

The first step of this method is to compute the super-image Σ. Averaging the multitemporal series strongly reduces the speckle. If some noticeable speckle fluctuations remain, a slight spatial filtering may be necessary to obtain a very high-quality superimage.

For any image I in the stack, the ratio τ = I/Σ is then computed and denoised by a state-of-the-art speckle reduction method. In the absence of speckle fluctuations in the super-image, the noise distribution is similar to the distribution of a single-look SAR image and a traditional despeckling algorithm can be applied. If speckle fluctuations remain in the super-image image, an adaption of the despeckling algorithm is necessary (a generic method, called RuLoG, is proposed in [START_REF] Zhao | Ratio-based multitemporal SAR images denoising: RABASAR[END_REF]). For all despeckling methods that are sensitive to speckle correlations, a sub-sampling of the ratio image is necessary, which alters the resolution of the restored ratio image T . Finally, the denoised estimation of R is retrieved by multiplying T with the super-image Σ: R = T × Σ.

In this section, we propose to replace the denoising step of the ratio image by the SAR2SAR approach, which avoids the sub-sampling procedure required by standard despeckling filters.

Adaptation of SAR2SAR to ratio images

Owing to their non-linear nature, neural networks are very sensitive to the dynamic range of their inputs. Significantly shifting the dynamic range of input images between training and testing most often leads to catastrophic results. Ratio images have very different ranges compared to SAR intensity images: in the absence of significant changes between the image I and the super-image Σ, the expected value of the ratio is 1. It is then necessary to appropriately rescale them in order to use the SAR2SAR network trained on SAR images (i.e., not specifically trained on ratio images). We experimented with several normalization strategies and describe the one that worked best. SAR2SAR processes log-transformed intensities y = log(I) that are approximately rescaled to the [0, 1] range by a fixed affine transform y → (y-m)/(M -m) where m and M respectively correspond to the minimum and maximum log-transformed intensity computed over the whole training set. In order to preserve the original range, in log-domain, of the image I when processing the ratio τ , we normalize the super-image Σ: Σ = Σ/λ. The normalization factor λ is chosen such that the average log-transformed intensity of the super-image is equal to 0: λ = exp(µ[log(Σ)]), with µ[•] the average value computed over all pixels of an image. The modified ratio image T = I/Σ is processed by SAR2SAR. The obtained despeckled ratio T is then multiplied by the normalized super-image to produce the final estimate of the restored image: I = T • Σ .

Experimental results

We illustrate our method on single-look Sentinel-1 images of an area near Lelystad, Netherlands. A stack of 25 images was spatially co-registered and temporally averaged. Remaining speckle fluctuations were suppressed with MuLoG+BM3D (Deledalle et al., 2017), using an equivalent number of look estimated in a homogeneous area (see Section 3.2.3). A single-look amplitude image and the super-image are shown in Fig. 4.13, left column. Fig. 4.13 compares restoration results obtained by several strategies: the top block gives single-image restoration results and the bottom block shows how the use of a super-image improves the despeckling. Images (a) and (d) suffer from artifacts due to the application of a despeckling method that is sensitive to spatial correlations of speckle directly on a Sentinel-1 image. Downsampling the images reduces speckle correlation and suppresses these artifacts ((b) and (e)). This comes at the cost of a noticeable resolution loss, somewhat mitigated by the use of a super-image. SAR2SAR is robust to speckle correlations. It gives superior results in the single-image scenario (image c) and offers a restoration with an improved preservation of details such as thin roads or field edges using the proposed multi-temporal approach (image f).

The formulation of RABASAR makes it readily applicable not only to single-look SAR images, but also to stacks of GRD data. In section 4.6, we extended SAR2SAR by training a model specifically on GRD data: we will refer to this network as SAR2SAR-GRD. Following what is done for single-look images, one can improve the performances of SAR2SAR-GRD by integrating the model within RABASAR to naturally exploit the multi-temporal information of GRD time series. As GRD data have already been multilooked (they have an ENL close to 4.4), the superimage obtained through multitemporal multi-looking does not necessarily need to be refined with a mild denoising step. In Fig. 4.14, as we can observe from the result produced, the integration of the multi-temporal information leads to a result richer in terms of details: thin roads that were lost with the single-image approach are very well restored when SAR2SAR-GRD is integrated in a multi-temporal framework for SAR despeckling.

Summary of the chapter

Summary: SAR2SAR

To develop a deep learning despeckling algorithm, we used SAR time series to:

• Produce groundtruth images and pre-train the model using synthetic speckle noise to generate two independent speckle-corrupted realizations of the same image.

• Fine-tune the network on real images to adapt to speckle spatial correlations. Each date of the time series is treated as an independent sample, using the pre-trained model to compensate for temporal changes.

Two extensions of the proposed algorithm were derived:

• Adaptation of SAR2SAR to GRD images. To this aim, the model has been re-trained from scratch. First, we considered a 4-look synthesized speckle. Then, the model has been fine-tuned on stacks of GRD image to learn the spatially varying speckle correlations.

• Integrate the model in a multi-temporal framework for SAR despeckling, namely RABASAR: both models, for single-look SAR images and for GRD images, have been tested and they both showed to benefit from the multitemporal information provided under the form of a superimage.

Context of the work

The question motivating this work is the following: can we reduce the burden of building a dataset to train a self-supervised deep learning model for SAR despeckling?

This chapter is mainly based on our paper As if by magic: self-supervised training of deep despeckling networks with MERLIN (Dalsasso et al., 2022b). The objective of this chapter is to propose a new hassle-free way of building training sets for SAR speckle reduction (Challenge (1) Training set). In Section 2.2, we derived the statistical model of a SLC SAR image: under certain hypothesis, the real and imaginary part are mutually uncorrelated. In this chapter, we will see that thanks to this property, building a dataset to train deep models for speckle suppression has never been that easy: a single image can naturally be decomposed into two independent subsets and a neural network trained with self-supervision directly on actual SAR images, leading to robustness to speckle spatial correlations (Challenge (3) Speckle spatial correlations).

To train a CNN with self-supervision, one needs to produce two statistically independent sets of measurements. In the previous chapter, we have seen that this can be achieved by resorting to SAR time series: provided that the deterministic component does not change and that perfect temporal speckle decorrelation occurs, each date can be treated as an independent speckle realization of the same underlying signal. As an alternative, if one assumes the speckle to be a white process, a network can be trained on single images by applying some pixel-masking strategy (Molini et al., 2021). However, the former requires the collection of several (at least two) images acquired over the same area, a fine co-registration step and a pre-trained network to compensate for temporal changes. The latter allows to train a model on single images, but it requires to decorrelate the speckle (see Section 3.4.1) and constrains the network architecture to have a specific structure in order to exclude masked pixels from its receptive field. This chapter describes a new training framework, applicable to all kinds of network architectures. This strategy is fully unsupervised: it only requires SLC images to perform the training or to process new data once the network is trained. In contrast to other existing works, it does not require additional hypotheses like the absence of spatial correlations of the speckle, or temporal stability throughout a time series. The phase information of SLC images is often considered irrelevant when only the intensity is of interest (i.e., apart from the context of SAR interferometry). Finding an interest in the real and imaginary components for the purpose of restoring the intensity images may even seem disconcerting. In section 2.2, we derived a statistical model of SLC images showing that two independent and identically distributed images can be extracted from an SLC image. This paves the way to the application of a self-supervised training strategy inspired by noise2noise [START_REF] Lehtinen | Noise2noise[END_REF]: MERLIN. Results obtained on images at different spatial resolutions confirm the ability of MERLIN to produce high-quality restoration results at medium, high, and very high spatial resolutions.

MERLIN: complex self-supervised despeckling

Recall: Complex amplitude speckle statistics

The complex amplitude at a given pixel z = ρ exp(jϕ) can be decomposed into its real and imaginary parts, z = a + jb, leading to:

p Z (z) = p Z (a + jb) = 1 πR exp(-(a 2 + b 2 )/R) = 1 √ 2π R/2 exp(-a 2 /r) N (0,r/2) 1 √ 2π R/2 exp(-b 2 /R) N (0,R/2) , (2.3)
which shows that the real and imaginary parts of the complex amplitude are i.i.d. according to a Gaussian distribution with variance R/2 where R is the SAR reflectivity at that pixel. Depending on the acquisition mode, the chosen pixel size, and the spectral apodization applied to reduce sidelobes around bright targets, a specific SAR system response then transforms the spatially uncorrelated field z into a spatially correlated field z:

z = Hz , (2.4)
with H the spatial-domain operator associated to the SAR transfer function. If the SAR system H is real-valued (for a shift-invariant system, this corresponds to a frequency response with Hermitian symmetry), then by linearity of H we get: z = ã + j b with ã = Ha and b = Hb .

(2.5)

The real and imaginary parts ã and b are spatially correlated but mutually independent random fields (appendix A derives a slightly more general condition on H to obtain statistically independent components ã and b).

As depicted in Fig. 2.2 and recalled above from Section 2.2, the statistical model of SAR image formation shows that the two components ã2 and b2 , corresponding to the squared real and imaginary part of a SLC image that are added to form the SAR intensity image, are independent and identically distributed. Each component contains half of the information, or, in other words, has a SNR that is 1/ √ 2 times the SNR of the intensity image (the intensity corresponds to the sum of these two independent components, its variance is halved, which corresponds to a SNR improvement by a factor √ 2).

Since SLC images provide two i.i.d. components that contain half the information1 from the intensity image, a self-supervised training strategy can be built by processing one component (e.g., the real part) and evaluating the restoration quality on the other component (e.g., the imaginary part). This corresponds to an ideal application case of the noise2noise principle [START_REF] Lehtinen | Noise2noise[END_REF] in which a deep neural network is trained to predict a noisy image from another independent noisy realization. Since the realization-specific random perturbation can not be guessed by the network, it tends to remove the noise in the input image even if no noiseless image is provided to the loss function.

We follow a similar approach to train networks by performing a coMplex sElf-supeRvised despeckLINg (MERLIN). Our approach is graphically summarized in Fig. 5.1: during the training phase (step A of the figure), the network is trained to process only the real part and a loss function is evaluated to measure how close the estimated reflectivity is to the imaginary component. Once the network is trained, it can be applied to reduce speckle noise in SLC images (step B of the figure). This time, both the real and imaginary parts are independently processed using the same network weights (i.e., a single network is trained in step A). The two estimations are then combined to produce the final estimation (by averaging). When despeckling SLC images, all the information is used, i.e., both the real and imaginary parts. In order to define the loss function used during the training phase (step A), it is necessary to decide which parameters should be estimated. The SAR transfer function has a significant impact on the image appearance: the 0-padding controls the pixel size while the spectral apodization sets the height of the sidelobes. Rather than inverting the SAR transfer function, we consider producing an image with the same characteristics (pixel size and bright point signature). With an ideal SAR transfer function H = I (the identity matrix of dimension K × K), the real and imaginary parts ã and b have a variance equal to R/2. With a non-ideal transfer function, the variance corresponds to the diagonal of matrix 1 2 Hdiag(R)H , i.e., the variance of the k-th pixel is Rk /2 with Rk = H 2 k R . With MERLIN, we aim at estimating the values Rk for each pixel. The marginal distribution of ãk and bk (the real and imaginary parts at pixel k of the SLC image) is a centered Gaussian with variance Rk /2. We thus define the following loss function L: which corresponds, up to an additive constant, to the sum over all pixels of the opposite of the log-likelihood of the marginal distribution. To reduce the dynamic range, it is beneficial that the network inputs and outputs be expressed in log-scale. We introduce ǎk = log |ã k |, bk = log | bk | and recall that xk = log Rk , to define the equivalent loss function expressed with log-scale images:

L( R, b) = k 1 2 log Rk + b2 k Rk , (5.1)
L log (x, b) = k 1 2 xk + exp 2 bk -xk . (5.2)
Note that since the real and imaginary parts are i.i.d. and given that we use the same network weights in step B to process both the real and imaginary parts, the training phase (step A) can not only be performed with the real part as input and the loss L( R, b), but also with the imaginary part as input and the loss L( R, ã) (only the former case is represented in Fig. 5.1 for simplicity reasons while both are applied in practice).

Experimental validation

In contrast to the family of self-supervised methods derived from the concept of blindspot (Laine et al., 2019) that require the receptive field of the network to exclude the central pixel(s), MERLIN imposes no constraint on the type of neural network used to perform the estimation. In the following experiments, we kept the same residual U-Net architecture employed within SAR2SAR (Chapter 4). 1.0 1.0 0.5 1.0 learning rate 10 -3 10 -3 10 -3 10 -3 10 -4 after 6 epochs 10 -4 after 4 epochs 10 -4 after 4 epochs 10 -4 after 3 epochs 10 -5 after 20 epochs 10 -5 after 20 epochs 10 -5 after 20 epochs 10 -5 after 20 epochs

To limit the dynamic range of the images at the input of the network, images ã and b are log-transformed and normalized using a fixed affine transform.

In the following set of images, the CNN has been trained on SLC SAR images. The experiments have been conducted both on images with synthetic speckle noise and on real SAR images. The hyper-parameters used to train the network for each imaging modality are listed in table 5.1. In the spirit of Challenge (2) Open code, the weights of the trained models are made available for testing at https://gitlab.telecom-paris. fr/RING/MERLIN.

Evaluation of MERLIN on images with synthetic speckle

We first evaluate the capability of MERLIN to train a network to restore images synthetically corrupted by speckle. The training set of speckle-free images has been built according to Section 3.2.3. We consider an ideal SAR transfer function: H = I so that many different despeckling techniques built on Goodman's model can be applied and compared. with MERLIN can not perform as well as when trained with the supervised training (i). The degradation in image quality remains limited, however, as seen on figure 5.2.

Table 5.2 gives PSNR values, expressed on amplitude images √ R, for several despeckling methods. Depending on the image, the U-Net trained with MERLIN performs at least as well or better than methods like SAR-BM3D [START_REF] Parrilli | A nonlocal SAR image denoising algorithm based on LLMMSE wavelet shrinkage[END_REF] or NL-SAR [START_REF] Deledalle | Nl-sar: A unified nonlocal framework for resolution-preserving (pol)(in) sar denoising[END_REF] that are not based on deep neural networks. The performance seems comparable on average to that of SAR-CNN, presented in Chapter 3. Numerical values confirm our analysis of figure 5.2: when trained with MERLIN, the U-Net produces results that are slightly worse than when real and imaginary parts are processed jointly and the network is trained in a supervised fashion. Compared to the self-supervised method Speckle2Void (Molini et al., 2021) which uses a specific network architecture to obtain a receptive field with a central blind spot, the performance of the U-Net network trained with MERLIN is notably better. We show in the next paragraph that, when applied to actual SAR images, the gain brought by self-supervision with MERLIN becomes very appealing.

Restoration of actual SAR images

When actual SAR images are considered, it is beneficial to train a network specifically for a given sensor and a particular imaging mode. The SAR transfer function varies from one imaging mode to the other (appendix B illustrate a pre-processing technique one must apply to decorrelate ã and b when the operator H is not real), as well as the spatial resolution and, thus, the structures that can be resolved. In this paragraph, we show results obtained with the same network architecture but different trainings each performed on images of the same type. Figure 5.4 gives results obtained on a TerraSAR-X image acquired in High Resolution SpotLight (HS) mode over an urban area: a small area of the city of Berlin, Germany. The content in this area is very different from the previous region shown in Fig. 5.3: there are many bright targets and the images have a higher spatial resolution. Bright targets are preserved while homogeneous areas are smoothed. As highlighted in the zoom-ins of Fig. 5.4, point-like scatterers having a deterministic phase appear differently in the real and imaginary part, some of them are even visible only in one of the two parts. Yet, when the intermediate estimations provided by applying the network to the two parts separately are combined together, these targets are correctly restored in the final denoised image. When many point-like targets are aligned horizontally or vertically (i.e., in the direction of the sidelobes), the network tends to merge the targets into a line. This phenomenon could possibly be reduced by considering a larger training set and/or a different network architecture. Textures are well recovered by Speckle2Void but a slight bias can be observed in the lake of Fig. 5.6.b. Conversely, the blind-spot structure of the network employed in Speckle2Void makes it hard to recover isolated bright points, as it is difficult to predict their existence from the neighboring pixels. This behavior is exacerbated in dense urban areas, such as that of Figure 5.7. The images of the residuals appear spatially more stationary with MERLIN than with the other restoration techniques: almost no structure can be identified. Being robust to speckle spatial correlations and relying on all the pixels in the receptive field of the CNN, MERLIN produces a pleasant result with a good preservation of both geometrical structures and detailed textures, whilst strongly suppressing speckle noise. Point-like scatterers seem to be well restored as well.

To allow a more extensive evaluations of MERLIN, additional restoration results are provided below, as well as at https://gitlab.telecom-paris.fr/RING/MERLIN. Images below display results obtained on images acquired in Stripmap mode and HS mode. Their visual inspection attests the quality achieved by MERLIN for different image modalities and resolutions. Although real and imaginary parts have a worst SNR than the intensity image, the reflectivity estimated from each part separately is restored really well and it is slightly improved when the two estimations are finally combined to produce the final output. A major difficulty to incorporate such models within MERLIN's loss function is that they depend on additional parameters that would require to be locally set to account for the content of each resolution cell (level of heterogeneity in the cell). From a pragmatic point of view, the qualitative analysis of the results produced by MERLIN on high-resolution images (Fig. 5.4) seems to indicate that the behavior of the network is satisfactory even in the very-high-resolution regime. Performing a more in-depth analysis would probably require using high-quality SAR simulators to provide ground truths for quantitative validation.

Note that, to obtain independent real and imaginary the condition that is presented in appendix A must be verified: images can be pre-processed as illustrated in appendix B. Moreover, due to the phase modulation applied to perform Terrain Observation with Progressive Scans SAR (TOPSAR) in most of Sentinel-1 acquisition modes (in particular, Interfeometric Wide swath (IW)), a direct application of MERLIN is not possible. It is mandatory that these SLC images be deramped before processing, see [START_REF] Miranda | Definition of the TOPS SLC deramping function for products generated by the S-1 IPF[END_REF]. Additional preprocessing also seems necessary to fully decorrelate the real and imaginary parts of these images which will be considered in future works and presented in the perspectives of this thesis.

One may wonder if MERLIN could possibly work on intensity-only images, by generating fake phase information (a phase could be drawn at each pixel according to a uniform distribution in [-π, π]). This would work perfectly well in the case of an ideal SAR transfer function: the results presented in section 5.3.1 would not change if only the intensity was provided to MERLIN and a random phase was generated afterward. This is analogous to what is done in the Recorrupted2Recorrupted framework presented in [START_REF] Pang | Recorrupted-to-recorrupted: Unsupervised deep learning for image denoising[END_REF], where only unpaired images corrupted by AWGN are available: to generate two images with independent noise from a single noisy observation, the available image is recorrupted, leading to two noisy images having independent noise components. However, when real SAR images are considered, the SAR transfer function is no longer ideal and the statistical distribution of the actual phase differs from a random white field. Figure 5.8.c shows the restored images obtained by a network trained on TerraSAR-X Stripmap intensity images, with a phase generated randomly. There are remaining speckle fluctuations and artifacts in the form of a high-frequency texture due to the mismatch between the spatial correlations of the intensity and the Another point worth discussion is the inversion of the SAR transfer function. The loss functions in Eqs.(5.1) and (5.2) were derived from the marginal distributions. Starting from the full distribution, a different loss function would be obtained:

L full (R, b) = k 1 2 log R k + b Hdiag(R)H -1 b , (5.4)
which boils down to Eq.(5.1) when H = I. The loss function of Eq.(5.4) is much more costly to evaluate since H corresponds to a Toeplitz-block Toeplitz matrix (a 2D convolution in the direct domain, or a product in the Fourier domain). The inversion Hdiag(R)H -1 can not be derived in closed form, which makes the training much more costly (several iterations of a conjugate gradients algorithm would typically be necessary for each evaluation of the loss function). Moreover, some form of regularization on R would be necessary since the data ã and b are not sufficient to constrain R beyond the cutoff frequency of the SAR system.

As illustrated in Fig. 5.2 and commented in section 5.3.1, the self-supervision with MERLIN comes at a cost: the real and imaginary components are processed separately. This limits the performance compared to joint processing of both components since the network is forced to handle images with a worse SNR (by a factor 1/ √ 2). This is partially compensated when the restorations computed separately on each component are finally combined. We think that this drawback is amply compensated by the good adaptation of the network to the SAR transfer function and the ability with MERLIN to use very large training sets (possibly, entire archives from a sensor).

Finally, compared to other self-supervision approaches, MERLIN imposes no limitation to the architecture of the network and does not assume a spatially decorrelated speckle. Together with the possibility to straightforwardly include huge archives of images in the training set, this opens new possibilities to consider highly expressive network architectures (e.g., very deep) for despeckling.

Summary of the chapter

Summary: MERLIN

We have shown that single-look complex images offer an ideal framework to selftrain despeckling networks using the real and imaginary parts of the data. The proposed generic training approach:

• Imposes no constrain on the architecture of the network.

• Completely suppresses the hassle of building training sets with reference images.

Using this framework, called MERLIN, we trained three models specific to:

• TerraSAR-X data in Stripmap mode.

• TerraSAR-X data in High Resolution SpotLight mode.

• High-resolution images acquired by SETHI.

This shows that networks specific to a sensor/acquisition mode can be easily trained, reaching a higher performance than general-purpose networks. Relieved from the worry of building training sets, future work can focus on designing clever network architectures. With MERLIN, very large scale training using entire archives of SAR images produced with a specific sensor mode can be contemplated, which could potentially lead to unprecedented despeckling performances.

Conclusion and perspectives

Conclusion

The main objective of this thesis was to leverage deep learning techniques to improve the interpretation of SAR images. Impaired by strong fluctuations known as speckle, SAR data require the development of despeckling techniques to ease their exploitation. This thesis has shown that training despeckling networks directly on SAR images, without ground truth, offers several advantages:

• building the training set is much easier, it can thus include much larger amounts of images;

• there is no domain-shift between network training and network application since the network is trained directly on actual speckle, provided that images with a similar content are present in the training set.

A synopsis on self-supervised techniques

Associated publications:

Conference papers: 5). These approaches are summarized in Fig. 6.1 and each one has different advantages and limitations, which are discussed hereafter.

Temporal diversity:

A natural way to obtain two speckle realizations of the same radar scene consists of collecting two images at two dates with a temporal separation sufficient to ensure the temporal decorrelation of the speckle. With increasing temporal separations, more changes might also occur within the scene. Let us consider for now that the radar scene remains basically unchanged except for the small changes that cause the speckle decorrelation. A training sample then corresponds to a pair of noisy intensities ( Ĩt 1 k , Ĩt 2 k ) of speckled images corresponding to the same (unavailable) ground truth reflectivity R k . The splitting of a training sample can either be y in k = Ĩt 1 k and y val k = Ĩt 2 k or the converse. In practice, it is necessary to handle the changes in the radar scene that occurred between times t 1 and t 2 . Two approaches are possible: if these changes are of limited spatial expansion, they can be detected (using a change-detection method) and the pixels in the changed area discarded from the computation of the loss function [START_REF] Vitale | Analysis on the building of training dataset for deep learning SAR despeckling[END_REF]. A more refined way to account for changes is pro-posed in Chapter 4: we apply a change compensation operation to image Ĩt 2 k . This way, all changes, even the smallest, are accounted for. Since change compensation requires a method to estimate the reflectivities Rt 1 and Rt 2 , the algorithm SAR2SAR (Dalsasso et al., 2021b), performs several training steps to progressively refine the despeckling network to benefit from improved change compensations.

Spatial diversity:

In the absence of additional images of the same scene at other dates, data splitting can be spatial. The idea is to mask some observed values and to train the network to predict the values that are masked by computing the unsupervised loss on those masked values. Several techniques can be used to perform the masking: masked values can be replaced by a local average, a purely random value, zero, or the network architecture can be designed in order to achieve a "blind-spot", i.e., the central area of the patch is unconnected to the output (the receptive field of the neural net does not contain this area). The latter approach leads to effective training because all observed pixels appear in turn in the loss function. Direct application of a network with a blind spot would be disappointing, especially in SAR imaging where point-like structures are common, since such punctual elements would be lost when falling within the blind spot. In order to also account for the observations inside the blind spot in the reflectivity estimate, the algorithm Speckle2Void (Molini et al., 2021) follows the Bayesian approach of (Laine et al., 2019): the network predicts a parametric distribution, which characterizes the reflectivity [R k ] at pixel , based on the observed intensities V I k ( ) in the neighborhood of pixel (the neighborhood excludes pixel which represents the blind spot). It defines a prior that can then be combined with the actual observed intensity [I k ] , for example using the posterior mean estimator.

An important requirement to apply a spatial splitting is that the speckle be spatially uncorrelated. If it is correlated, then the network will be able to recover the masked noisy intensity [I k ] using only the neighborhood V I k ( ) and the resulting estimation will still be corrupted by speckle. To prevent this, speckle correlations must be reduced either by subsampling the image or by applying a spectral equalization (and possibly a resampling if the image has been oversampled): some approaches [START_REF] Lapini | Blind speckle decorrelation for SAR image despeckling[END_REF][START_REF] Abergel | Subpixellic Methods for Sidelobes Suppression and Strong Targets Extraction in Single Look Complex SAR Images[END_REF]Dalsasso et al., 2021a) are detailed in Section 3.4.1. Moreover, to obtain a blind spot centered on pixel , the architecture of the neural network must be carefully designed, which severely constrains the choice of the network.

Real/imaginary part diversity:

In our latest work (Dalsasso et al., 2022b), detailed in Chapter 5, we have suggested another possible splitting of SLC training data for unsupervised training: the decomposition into the real and imaginary parts. Under reasonable assumptions (a SAR system with a real-valued response, i.e., with a transfer function with Hermitian symmetry), the real and imaginary parts of an SLC image are indeed statistically independent. In contrast to the approach based on temporal diversity, there is no issue with possible changes between the two components since they are acquired simultaneously.

The splitting of a sample from the training set takes either the form y in k = ãk and y val k = bk or the converse (real and imaginary parts can be swapped randomly during training), where we recall that ãk and bk are the real and imaginary parts of the SLC image zk .

In order to ensure that the real and imaginary parts ãk and bk are independent, a preprocessing may be necessary to achieve a 0-Doppler shift and possibly to correct for spectrum asymmetries, as depicted at the bottom right of figure 6 The following figure displays restoration results for each method as well as the residual noise, i.e., the ratio image I k / R k . Upon successful estimation of the reflectivity image R k , the residual I k / R k should correspond solely to the speckle component and contain no structure from the original intensity image. By close visual inspection, it appears that the baseline method, SAR-BM3D, produces some artifacts in smooth areas (fields, high vegetation) or around strong scatterers in the form of streaks or oscillations. SAR2SAR provides the sharpest restorations but suppresses less strongly the speckle fluctuations: remaining fluctuations can still be observed in some areas. Speckle2Void restores satisfyingly smooth areas but has difficulties to handle strong punctual scatterers. Finally, MERLIN seems to offer the smoothest estimates for vegetated areas and produces an estimation that is slightly more blurry than SAR2SAR on urban areas. The residual noise images display more structure in the images produced by SAR-BM3D and Speckle2Void than with MERLIN or SAR2SAR. It seems that the latter removes almost no significant structure from the original image: no extended pattern can be identified in the residual image.

Beyond the analysis of restoration quality, there are pros and cons for each approach that are summarized in the Table at the end of this section. Exploiting temporal diversity requires pairs of registered intensity images with sufficient temporal separation to ensure speckle decorrelation. Such image pairs may not be available when considering airborne SAR systems, which limits the applicability of the approach. Moreover, the change detection and/or compensation must be reliable otherwise a bias may appear in the estimates. Updating the change compensation method with the despeckling network leads to a training in several steps which makes it heavier than other self-supervised approaches. Since training requires pairs of images at different dates, fine tuning the network on a new image (with a content that differs from the training set) may not be possible if only a single image is available. Speckle2Void, based on spatial diversity, requires only intensity images, which simplifies the building of the training set and the fine tuning to new images. The requirement of spatially uncorrelated speckle leads to a preprocessing that changes the image appearance (a resampling may be necessary and the spectral apodization is removed, see Section 3.4.1). Only very specific network architectures lead to a blind spot in the receptive field, this limits the design choices and our experiments indicate that the network requires more care to reach convergence than a simpler U-Net network. MERLIN self-supervised training strategy only requires SLC images. The preprocessing step ensuring that the spectrum is symmetrical has a very limited impact on the image appearance and on the computational burden (see Appendix B). Training is easy and in practice can be performed even on a single image. Unlike Speckle2Void, any network architecture can be considered, which allows the straightforward transfer of evolved network architectures developed in the field of natural image restoration.

Concluding remarks

The work presented in this Ph.D thesis allows to conclude that self-supervised training of deep neural networks offers many advantages over conventional supervised training techniques. With respect to SAR-CNN, starting point and baseline for this analysis, with self-supervised techniques building the training set is greatly simplified and generalization to unseen data is largely improved given that actual SAR images are used in the training phase rather than images with synthetic speckle. There now exists several approaches to perform self-supervision, based on common ideas of splitting each training sample into two parts: one part fed to the neural network and the other used to compute the self-supervised training loss. This thesis introduced two algorithms: SAR2SAR, which belongs to the family of methods relying on temporal separation, and MERLIN, a unique and innovative approach demonstrating that the separation can be performed in the complex domain, i.e. by splitting real and imaginary part of a SLC image.

As the reference image is not needed, with self-supervised methods it is possible to fine-tune the network on a given area of interest leading to improved performances at test time on that specific area. This is particularly true for strategies requiring spatial diversity or real/imaginary part diversity, where a single image is sufficient to supervise the training.

Open issues and perspectives

End-to-end multi-temporal despeckling

The information contained in time series of SAR images is a major lever to the reduction of speckle. Under the assumption of perfect temporal decorrelation, Quegan filter [START_REF] Quegan | Filtering of multichannel sar images[END_REF] proposes to apply a temporal-multilooking, preceded by a change-compensation step requiring to pre-estimate the reflectivities (possibly through a CNN). RABASAR [START_REF] Zhao | Ratio-based multitemporal SAR images denoising: RABASAR[END_REF] encodes the multi-temporal information under the form of a superimage, i.e. the temporal average of a stack. The ratio between the date of interest and the superimage is computed, leading to an image easier to be filtered thanks to its increased stationarity. Speckle can be suppressed from the ratio using a deep learning approach. By re-combining the despeckled ratio and the superimage, the final estimate is obtained. These approaches however require long time series to obtain good restoration performances. Moreover, a neural network could find a more efficient non-linear combination of images composing the stack. In a preliminary study, we re-trained step A of SAR2SAR on images with synthetically generated speckle with two inputs: a 1-look image and a 20-looks image, representing an ideal superimage: the speckle is perfectly temporally decorrelated and no changes occur. However, as the result of Fig. 6.3 shows, the restored image of the so called multi-temporal SAR2SAR seems of poorer quality than the estimate obtained with the single-image method. The network seems to show a "lazy" behavior, bypassing the 1-look image and outputting a result close to the 20-look image, probably because it gets stuck on some local minima.

A possible way to circumvent this issue is to provide a network with several inputs being different dates of an image stack. As such, the network could learn the best encoding of the multi-temporal information leading to improved despeckling performances. Some works on this subject are ongoing, in collaboration with Inès Meraoumia.

Interpretability of the results

If one wanted to criticize the despeckling results produced by neural networks, he/she could argue that the estimated image is "too good to be true". Is the reflectivity retrieved by a CNN reliable or does the restored image contain spurious information? Bringing an answer to these questions is critical to unleash the full potential of deep learning algorithms. A possible answer could be provided by associating to a despeckled image a measure of uncertainty for each predicted pixel [START_REF] Gawlikowski | A survey of uncertainty in deep neural networks[END_REF][START_REF] Valsesia | Permutation invariance and uncertainty in multitemporal image super-resolution[END_REF]. We conducted a preliminary analysis by extending step A of SAR2SAR. We trained the model by adding to the self-supervised loss an additional loss defined as:

L uncertainty f θ (y 1 ), y 2 , β = k [y 2 ] k -f θ ([y 1 ] k ) -E[s] 2 Var[s] β k -1
where β k is the uncertainty associated to estimated pixel k, E[s] and Var[s] are the theoretical mean and variance of log-transformed speckle, respectively. The CNN is thus trained to simultaneously produce the restored image and the uncertainty associated to its prediction, as depicted in the following figure. The results obtained on images corrupted with synthetic speckle seem promising. The method has to prove its effectiveness on real SAR images, where the speckle is spatially correlated and the change compensation must be carefully done to avoid that possible errors made in the pre-estimations propagate in the uncertainty measured by the network.

In the light of the improvements brought by MERLIN in neural network training, the problem can be reformulated within this self-supervised framework by relying on a Bayesian modeling to formalize the uncertainty as a parametric distribution (Gal and Ghahramani, 2016).

Despeckling of multi-channel complex SAR images

Adapting neural networks to complex data is challenging. In classical deep models, inputs are processed separately. However, the real and imaginary part of multi-channel complex SAR data are mutually correlated and they have to be jointly processed. Sica et al. [START_REF] Sica | φ-net: Deep residual learning for insar parameters estimation[END_REF] propose an adaptation of the U-Net to interferometric SAR data. Simulated data have been used for this aim. In (Mullissa et al., 2021), a first attempt to suppress speckle from PolSAR data has been done. The adaptation of deep models to multi-channel SAR images is still at its early stages. The Bayesian modeling proposed in (Molini et al., 2021) could be extended to PolSAR data, whose model is given in the second chapter of this manuscript. To deal with complex data, one could either work in a transformed space or rely on deep complex networks (Trabelsi et al., 2017).

Application of MERLIN to deramped Sentinel-1 images in TOPS mode

In the Sentinel-1 TOPS imaging modality (opposite to the SPOT modality), the steering of the antenna in the azimuth direction reduces the azimuth resolution while increasing the swath coverage. Due to the beam steering during the acquisition process, the images undergo a linear frequency modulation. In order to train a network with MERLIN to suppress speckle from Sentinel-1 TOPS products, images must first be deramped and demodulated. The deramping operation inverts the linear frequency modulation, while demodulation centers the support of the complex spectrum on the 0-Doppler frequency. Nevertheless, these steps seems insufficient to obtain a symmetrical frequency response, which is a necessary condition for the independence between the real and the imaginary part of a complex SAR image.

Deramped and demodulated azimuth profile spectrum of a Sentinel-1 TOPS product at the scale of a 1000 × 1000 image patch. The blue line represents the original azimuth profile and the orange one its symmetrical with respect to the 0-Doppler frequency. Indeed, as it can be observed in the above illustration, the azimuth spectrum presents an asymmetry. The images below show the results obtained with MERLIN on such images, in comparison with SAR2SAR. The former smooths out small structures, while the latter reconstruct many more details. This is in contrast to what is observed on images corrupted by synthetic speckle (see Chapter 5) and on TerraSAR-X stripmap images, where the two algorithms were providing comparable results. This might be due to the asymmetry observed in the azimuth spectrum, resulting in a complex spatial operator that partially correlates the real and the imaginary parts of the data. The azimuth spectrum suggests that a possible correction would be to strengthen the right side of the spectrum, while attenuating the left side.

Finding the best pre-processing step allowing to have independent real and imaginary parts to train deep models with MERLIN on Sentinel-1 TOPS images is still under investigation. Answering to this open question is of utmost importance for the RS community, given the widespread use of such data for a broad range of applications that would benefit from speckle suppression.

89 5. 1

 1 Context of the work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89 5.2 MERLIN: complex self-supervised despeckling . . . . . . . . . . . . . . . 91 5.3 Experimental validation . . . . . . . . . . . . . . . . . . . . . . . . . . . 94 5.3.1 Evaluation of MERLIN on images with synthetic speckle . . . . . 95 5.3.2 Restoration of actual SAR images . . . . . . . . . . . . . . . . . 96 5.4 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104 5.5 Summary of the chapter . . . . . . . . . . . . . . . . . . . . . . . . . . . 106

Chapter 1 Introduction 1 . 1

 111 Context: Remote Sensing

Figure 1 .

 1 Figure 1.3 -Acquisition geometry of a side-looking aperture radar (SLAR)

Figure 1 . 4 -

 14 Figure 1.4 -Illustration of the Synthetic Aperture Antenna principle. The top image shows how the same target is observed by the radar several times, leading to multiple acquisitions of the same object from different positions. Below, one can see the antenna lobe obtained thanks to the synthetic aperture principle: a smaller antenna pattern leads to an improved azimuth resolution.

Figure 1 . 5 -

 15 Figure 1.5 -Shape of a 2D cardinal sine (sinc) function: the main lobe is surrounded by several secondary lobes (or sidelobes) whose power is -13[dB] less than the main sinc lobe.

Figure 1 . 7 -

 17 Figure 1.7 -Given a time series of co-registered SAR images, time-series analysis aimed at detecting multi-temporal changes and SAR speckle reduction mutually benefit from each other.

  . A preliminary analysis on the subject of speckle reduction through deep CNN for SAR images started in April 2018. At that time, I was in internship at Laboratoire Traitement et Communication de l'Information (LTCI), Télécom Paris, in the Images, Modélisation, Analyse, GEométrie et Synthèse team (IMAGES team) working on my Master's thesis in Information and Communication Engineering at the University of Trento (Italy). My Ph.D officially started in January 2019. By integrating the IMAGES team I joined a group active in the field of speckle reduction from SAR images. Beside the essential supervision of Florence Tupin and Loïc Denis, I have also had the chance to work with other Ph.D candidates: the fruit of the collaborations done with Xiangli Yang, Nicolas Gasnier and Inès Meraoumia will be presented in this manuscript. When I started working on SAR despeckling, only few articles on deep learning for speckle suppression were published (Chierchia et al., 2017; Wang et al., 2017b; Wang et al., 2017a; Zhang et al., 2018), as the subject was still at its infancy. The SAR community was thus facing some difficulties, that I can summarize in two main points: Challenge (1): Training set A well-established dataset to train deep models did not exist.

  ) Training set and shows the need of providing solutions to Challenge (3) Speckle spatial correla-tions. In Chapter 4, time series of SAR data are instead exploited to obtain different spatially-correlated speckle realization to develop SAR2SAR (Dalsasso et al., 2021b), a despeckling algorithm robust to speckle spatial correlations. Challenge (1) Training set and Challenge (3) Speckle spatial correlations are addressed. MERLIN (Dalsasso et al., 2022b), presented in Chapter 5, proposes a different training strategy showing that a model can learn to suppress speckle without requiring time-series: it exploits the independence between the real and imaginary part of a Single-Look Complex (SLC) SAR image. As SAR2SAR, it is trained on real images and addresses Challenge (3) Speckle spatial correlations, but it does a further step towards the solution of Challenge (1) Training set.

Figure 2 . 1 -

 21 Figure 2.1 -The number of papers published by IEEE on the subject of remote sensing data restoration.

Figure 2

 2 Figure 2.2 -A statistical model of speckle in SAR image: the intensity image on the right is a corrupted version of the reflectivity image shown on the left. The single-look complex image contains spatially-correlated speckle components that are independent in the real and imaginary parts. The SAR transfer function shown here corresponds to Sentinel-1 stripmap mode. For visualization purposes, a non-linear look-up table is used to display intensity images.

Figure 2 . 4 -

 24 Figure 2.4 -Temporal evolution of some representative methods for speckle reduction in SAR images, from conventional methods which started emerging in the 1980s to the more recent deep learning methods. Image credits: IGARSS 2021 presentation of Loïc Denis of our article A Review of Deep-Learning Techniques for SAR Image Restoration.

  Figure2.5 -The strategies to apply deep neural networks for speckle suppression: employs a pre-trained Gaussian denoiser in an iterative scheme accounting for speckle statistics; supervised training, using ground-truth images that match the speckled images provided as input to the network; self-supervised training, using co-registered pairs of SAR images captured at different dates; a single image and a masking strategy: the network is trained to correctly infer the masked pixels of the input image; a single image in Single-Look Complex format and exploiting the mutual independence of its real and imaginary part.

Challenge ( 1 )

 1 Training set, Challenge (2) Open code and Challenge (3) Speckle spatial correlations, section 1.3. It is presented in Chapter 4. Instead of learning from multi-temporal SAR series, Molini et al. (Molini et al., 2021) proposed Speckle2Void, an adaptation of the blind-spot CNN (Laine et al., 2019) to single-look intensity SAR image despeckling, by extending the analysis carried out in (Molini et al., 2020) (see Fig 2.5

Challenge ( 1 )•

 1 Training set, Challenge (2) Open code and Challenge (3) Speckle spatial correlations, described in section 1.3. It is presented in Chapter 5. Dalsasso, E., Yang, X., Denis, L., Tupin, F., & Yang, W. (2020). SAR Image Despeckling by Deep Neural Networks: from a pre-trained model to an end-to-end training strategy. Remote Sensing, 12(16), 2636. (Dalsasso et al., 2020) International conferences: • Dalsasso, E., Denis, L., & Tupin, F. (2021). How to handle spatial correlations in SAR despeckling? Resampling strategies and deep learning approaches. In EUSAR 2021; 13th European Conference on Synthetic Aperture Radar (pp. 1-6). VDE. (Dalsasso et al., 2021a)

Contribution ( 1 )

 1 : A dataset for supervised methods • Creation of a high-quality dataset of speckle-free SAR images to train supervised deep models for speckle suppression • Training of a network for SAR despeckling serving as baseline • Publication of an open-source code for testing and comparison

Figure 3 .

 3 Figure 3.2 and Table3.1 illustrate the architecture of the network DnCNN proposed by Zhang et al.[START_REF] Zhang | Beyond a gaussian denoiser: Residual learning of deep CNN for image denoising[END_REF]. This architecture is also used in the work by[START_REF] Chierchia | SAR image despeckling through convolutional neural networks[END_REF]. We choose to keep the same network structure as it allows us to perform a fair comparison to their work and to assess the quality of the training set we produced.

Figure 3 . 2 -

 32 Figure 3.2 -The architecture of DnCNN.

0Figure 3

 3 Figure 3.3 -Comparison of the Fisher-Tippett distribution (with different number of looks L, represented by the continuous line) and the Gaussian distribution (dashed line) with the same mean and variance.

Figure 3 . 4 -

 34 Figure 3.4 -(a) A 1-look SAR image acquired by Sentinel-1 ©ESA, (b) result of multitemporal averaging of 45 dates, (c) virtually speckle-free image obtained by denoising image (b) with MuLoG+BM3D.

  3).Compared to the suppression of AWGN, the networks need to learn how to separate log-transformed SAR reflectivities from log-transformed speckle, distributed according to the Fisher-Tippett distribution given in Equation (2.7). In the follwing, a discussion about minor changes to the network architecture is carried out. It is worth to point out that, given the impossibility to reproduce the work proposed in (Chierchia et al., 2017) (the weights are not available and the datasets are not the same), we intend by no means to compare our results to those of Chierchia et al. Instead, we have always followed our training strategy and drawn conclusions based on visual inspection of our testing set.In this study, it has been experimentally found that increasing the depth D of the network compared to the depth used by Zhang et al. and Chierchia et al. was improving the performance on the testing set. We used 19 layers, each layer involving spatial convolutions with 3 × 3 kernels (see Figure3.5). The receptive field of our network then corresponds to a patch of size 39 × 39.

Figure 3

 3 Figure 3.5 -The proposed SAR-CNN for Sentinel-1 image despeckling.

Figure 3 .

 3 Figure 3.6 shows the impact of the loss function (L 1 versus the smoothed L 1 used by Chierchia et al. (Chierchia et al., 2017)) and of the network depth in terms of despeck-

Figure 3 . 6 -

 36 Figure 3.6 -Illustration of the influence of the loss function and of the number of layers of the CNN on the despeckling performance: (a) the ground truth image obtained by temporal+spatial filtering of a stack of 45 Sentinel-1 images; (b) the same image corrupted by a synthetic speckle; (c) restoration obtained by training the CNN architecture proposed by Chierchia et al. on our training set; (d) restoration obtained when using an L 1 loss instead of the loss used by Chierchia et al.; (e) restoration obtained with an L 1 loss and two additional layers.

Figure 3

 3 Figure 3.7 -Restoration results and ratio images (noisy/denoised) on an image from the testing set.

  et al., 2012), NL-SAR (Deledalle et al., 2015) and MuLoG+BM3D in Tables 3.4 and 3.5.

Figure 3 . 8 -

 38 Figure 3.8 -Results obtained by filtering with SAR-CNN images corrupted by 1-look synthetic speckle noise. While a single speckle realization is considered in this figure, the average quality scores on 20 noisy images are provided in tables 3.4 and 3.5.

Figure 3

 3 Figure 3.9 -Despeckling results on a single-look Sentinel-1 image: (a) the single-look SAR image, (b) a restoration result obtained by NL-SAR, (c) restoration obtained by SAR-BM3D, (d) restoration obtained by MuLoG+BM3D, (e) restoration obtained with the pre-trained CNN and MuLoG framework, (f ) restoration obtained with a CNN trained on SAR images and MuLoG framework, (g) restoration obtained with our network trained for speckle removal on Sentinel-1 images. Images at the bottom row give the residuals of the restoration results. The red box indicates the area that has been used to estimate the equivalent number of looks to evaluate the quality of denoising.

Figure 3 . 10 -

 310 Figure 3.10 -Despeckling results on a single-look TerraSAR-X image: (a) a TerraSAR-X image in Stripmap mode (1 m × 2 m spatial resolution), (b) a restoration result obtained by NL-SAR, (c) restoration obtained by SAR-BM3D, (d) restoration obtained by MuLoG+BM3D, (e) restoration obtained with the pre-trained CNN and MuLoG framework, (f ) restoration obtained with a CNN trained on SAR images and MuLoG framework, (g) restoration obtained with our network trained for speckle removal on Sentinel-1 data. Images at the bottom row give the residuals of the restoration results. As in Figure 3.9, the red box indicates the area used to estimate the equivalent number of looks.

3. 4

 4 How to handle speckle spatial correlations: illustration of the problem and strategies to alleviate it Most recent speckle reduction techniques are quite effective at removing most of speckle fluctuations, when evaluated on synthetic speckle, see left column of Fig.3.12. Direct application of these methods on a real image, however, leads to serious artifacts, see the despeckling results obtained on a Sentinel-1 image shown on the central column of Fig.3.12. Those artifacts are due to the mismatch between the theoretical model of speckle considered (spatially i.i.d. speckle component) and the actual speckle component. Indeed, as described in Section 2.2, the application of spectral windowing and oversampling through the linear operator H introduces spatial correlations in the speckle component.

Figure 3 .

 3 Figure3.12 -Recent speckle reduction methods such as MuLoG or SAR-CNN perform very well on simulated speckle (left column, in blue). Direct filtering of real images however leads to many artifacts, as illustrated on this single-look Sentinel 1 image (central column, in green). Sub-sampling the image by a factor 2 reduces the speckle correlations and lead to a much better restoration, at the cost of a resolution loss (right column, in green). Source: ©IEEE.

Figure 3 .

 3 Figure3.13 -The SAR image can be decomposed into bright targets and the speckle (background) component. The first one is resampled to reduce the sidelobes of bright targets, by placing the strong reflector at the center of the pixel. To the latter, spectral zero-padding and windowing are removed to spatially decorrelate the speckle. After filtering the image with a despeckling algorithm, the result is resampled to the original resolution and discrete Diracs are re-introduced.

Figure 3 .

 3 Figure 3.14 -Illustration of the decorrelation steps: (a) the original SAR image in stripmap mode over the Île de la Réunion, France; (b) the image resampled at the Shannon-Nyquist sampling frequency; (c) the deapodized image where pixels are spatially i.i.d; (d) the decorrelated SAR image obtained in the absence of bright targets.

Figure 3 .

 3 Figure3.15 -Results obtained on some Sentinel-1 SAR stripmap image patches over the Île de la Réunion by pre-processing the images to decorrelate them before suppressing speckle using an algorithm developed under assumptions of spatially uncorrelated pixels, such as SAR-CNN.

Figure 3 .

 3 Figure 3.16 -Results of MuLoG+BM3D and the deep learning method SAR-CNN on an image with synthetic 1-look uncorrelated speckle

Figure 4 . 1 -

 41 Figure 4.1 -Structure of the U-Net considered in this chapter and inspired from the architecture used in the noise2noise framework, plus a residual connection: assuming the additive speckle model for SAR images in log domain, the network learns the speckle component, which is subtracted from the input image to produce the final estimate.

Figure 4

 4

Figure 4 . 2 -

 42 Figure 4.2 -Results obtained by filtering images corrupted by synthetic speckle noise with SAR-CNN and with a U-Net. Both networks are trained using the same dataset until they reach convergence.

Figure 4 . 3 -

 43 Figure 4.3 -Principle of the noise2noise approach: the network relies on independent noisy realizations of the same underlying clean signal that one wishes to recover.

Figure 4 . 4 -

 44 Figure 4.4 -Evolution of the root mean square estimation error as a function of the number of samples. Source: ©IEEE.

[y 1 ]Figure 4 . 5 -

 145 Figure 4.5 -The proposed despeckling algorithm SAR2SAR is based on a U-Net trained in a residual learning fashion to restore log-transformed SAR images. The training is done in 3 steps: (A) on images with simulated speckle, (B) on pairs of actual SAR images, with changes compensated for based on the network trained in (A), (C) a final refinement of the network is performed with a better compensation for changes. Source: ©IEEE.

Figure 4 . 6 -

 46 Figure 4.6 -Results of state-of-the art despeckling filters on an image corrupted with synthetic 1-look uncorrelated speckle.

Figure 4 .

 4 Figure 4.7 -Results of state-of-the art despeckling filters on a real Sentinel-1 SAR image. SAR2SAR B refers to the first fine-tuning step, when images with a poorer resolution are used to compensate for changes.

  .8 demonstrate the effectiveness of the algorithm in different situations: textures in the forest area are not over-smoothed (in particular image Marais 1, Fig.4.8) and fine details are well reconstructed (see the narrow river of image Nirapdur, Fig.4.8), without leaving residuals of speckle noise in the filtered images.

Figure 4 . 8 -

 48 Figure 4.8 -Restoration of Sentinel-1 images with SAR2SAR. Real single-look SAR images are shown on the left side, with the corresponding despeckled result on the right side.

Figure 4 .

 4 Figure 4.9 -Ablation study of SAR2SAR, showing the restoration of two Sentinel-1 images. In the left column, a self-supervised training conducted only on real images is considered. On the right column, the network is initially trained on images with synthetic speckle noise. Different strategies are considered, showing the importance of change compensation and the relevance of the proposed semi-supervised approach. Source: ©IEEE.

  4.6) and to any sensor (e.g. TerraSAR-X, Sentinel-1 stripmap), once the training data are collected, addressing Challenge (1) Training set. To produce the result of the last column of Fig.4.10, SAR2SAR is fine-tuned on a stack of 11 images of size 1446 × 1048 pixels acquired over the Île de Réunion, France in Stripmap mode with resolution of 5 × 5[m]. Although SAR2SAR provides a pleasant result, removing the speckle affecting the SAR image, the fine-tuned network shows that even more details can be restored if the model is adapted to the given image modality and resolution.The weights of the trained model (and additional results) are released along with our article 2 , contributing to the need for testing code presented as Challenge (2) Open code (an illustration of the shared notebook is given in appendix E).

Figure 4 . 10 -

 410 Figure 4.10 -Restoration results obtained on two crops of a SAR image acquired over the Île de la Réunion, France, in Sentinel-1 Stripmap mode at 5m × 5m spatial resolution. SAR2SAR allows to filter the image at full resolution, and a fine-tuning of the network (originally developed for Sentinel-1 TOPS data at 5m × 20m) achieves even better results. Data provided by Thierry Koleck, CNES

Figure 4 . 11 -

 411 Figure 4.11 -Despeckled estimates on two GRD SAR images at the three stages of SAR2SAR. At step A, the image has been pre-processed through a subsampling by a factor of 2 to reduce speckle spatial correlation. At steps B and C, the network is refined on real images and it is thus robust to spatially correlated speckle: thus, images are processed at their full resolution, obtaining a higher level of detail preservation and an enriched information.

Figure 4 .

 4 Figure 4.12 -Some steps of the proposed method on a crop of Gaoual image. (a): Full noisy GRD image. The 3 pairs of a priori nodes (A1,A2), (B1,B2), and (C1,C2) are displayed in blue. (b-f) are cropped as presented in the red square in (a). (b): noisy GRD HD image. (c): Image denoised using the proposed denoising approach. (d): Ratio between the noisy image and the denoised image. (e) Response of the linear structures detector. (f): Result of the rivers detection. In (f), the true positive pixels are displayed in blue, the false negative in red, the false positive in yellow, and the true negative pixels as the actual denoised GRD image. The GRD images are displayed in amplitude with cropped dynamic for better visualization. The ratio image is displayed after logarithmic transformation. Source: ©IEEE.

Figure 4 .

 4 Figure 4.13 -Comparison of several despeckling strategies for single-image and multitemporal processing. Source: ©IEEE (Dalsasso et al., 2021c).

Figure 4 .

 4 Figure 4.14 -A stack of 17 GRD images is temporally averaged to produce the superimage encoding the multi-temporal information. If perfect speckle decorrelation has occurred through time, one would obtain a superimage having an ENL= 74.8. The obtained ratio image is more homogeneous than the initial image I, and it thus easier to suppress speckle from it. This result has been produced by Lucie Jandet in the context of a Master's project that I co-supervised together with Florence Tupin.

•

  Dalsasso, E., Denis, L., & Tupin, F. (2022). As if by magic: self-supervised training of deep despeckling networks with MERLIN. IEEE Transactions on Geoscience and Remote Sensing. 60, 1-13. (Dalsasso et al., 2022b)

Contributions•

  Derivation from the SAR image model of a suitable loss function for selfsupervised image despeckling exploiting the complex nature of SAR data • Training of a model for three different image modalities • Publication of an open-source code for testing and comparison for TerraSAR-X data in Stripmap mode and in HS-Spotlight mode

Figure 5 . 1 -

 51 Figure 5.1 -The principle of MERLIN: during step A, the despeckling network is trained to estimate the reflectivity based solely on the real part. The loss function evaluates the likelihood of the predictions according to the imaginary part. Once the network is trained, it can be used as shown in B: the real and imaginary parts are processed separately using networks with the same weights. The outputs are combined to form the final estimation. Note that, to simplify the figure, step A is illustrated only with the real part as input but real and imaginary parts can be swapped during training to increase the number of training samples, see text. Source: ©IEEE (Dalsasso et al., 2022b).

Figure 5

 5 Figure 5.2 -Despeckling results on images corrupted by synthetic speckle: the first column shows the noisy images obtained by multiplying the ground truth images √ r shown in the second column by a white speckle field. The third column gives despeckling results obtained with a U-Net trained in a supervised fashion (SAR2SAR, step A). The last column gives despeckling results obtained with the same network trained with MERLIN. Compared to the supervised training, MERLIN is penalized because it only has access to a single speckle realization through the real or imaginary part. The images shown in each row are regions of interest extracted from the images Lely, Limagne, and Marais1 that appear in table 5.2. Source: ©IEEE (Dalsasso et al., 2022b). Additional images can be seen on https://gitlab.telecom-paris.fr/RING/MERLIN.

  Figure 5.2 compares restoration results on 3 different images with simulated speckle, for the same network architecture but two different training strategies: (i) a supervised training where the network has access to the full intensity image (i.e., ã2 + b2 ) and the loss function is evaluated on a different, independent, speckle realization drawn from the same groundtruth image (i.e., step A of SAR2SAR algorithm, SAR2SAR A : see Section 4.2.3); (ii) a self-supervised training with MERLIN where the network only has access to either ã2 or b2 and the loss function is L log (x, b) or L log (x, ǎ), respectively.Note that in the approach (i) the loss function used during training corresponds to:L log (x, ǎ ) + L log (x, b ) = k xk + exp y k -xk ,(5.3)with 2ǎ = log ã 2 , 2 b = log b 2 , and y = log(ã 2 + b 2 ) the log-transformed versions of the square of the real and imaginary parts, and the intensity of the second noisy realization. Obviously, with half the information available, the same network trained

Figure 5

 5 Figure 5.3 -Application of a U-Net trained with MERLIN on a TerraSAR-X image near Serre-Ponçon dam, in the French Alps, acquired in stripmap mode. Additional despeckling results on TerraSAR-X images in stripmap mode can be seen on https: //gitlab.telecom-paris.fr/RING/MERLIN. Source: ©IEEE (Dalsasso et al., 2022b).

Figure 5 .

 5 Figure 5.3 shows results obtained on a TerraSAR-X image acquired in stripmap mode over an agricultural area in the French Alps. The first row of the figure illustrates the decomposition of the intensity image into its squared real ã2 and imaginary b2 components. The second row shows the estimations produced by the network trained by MERLIN from each component and the final estimate. A qualitative analysis of the results shows a very good restoration of fine details and textures as well as bright targets.

Figure 5 . 4 -

 54 Figure 5.4 -Application of a U-Net trained with MERLIN on a TerraSAR-X image of Berlin, Germany, acquired in high-resolution spotlight mode. Additional despeckling results on TerraSAR-X images in High Resolution SpotLight (HS) mode can be seen on https://gitlab.telecom-paris.fr/RING/MERLIN. Source: ©IEEE (Dalsasso et al., 2022b).

Figure 5

 5 Figure 5.5 -Application of a U-Net trained with MERLIN on a 35cm spatial resolution airborne image on an agricultural area near Nîmes, France, acquired in 2014 by SETHI sensor (©ONERA). The corresponding 20cm resolution optical images (source: Geoportail ©IGN), shown on the left, date back from 2018. Source: ©IEEE (Dalsasso et al., 2022b).

Figure 5 .

 5 Figure 5.5 shows how MERLIN performs in very-high-resolution airborne imaging. The same U-Net network as previously is trained on a single image (9 130 × 10 000 pixels) captured with SETHI (Baqué et al., 2017) by the French aerospace laboratory ONERA in 2014. The image has a pixel size of 13cm in range and 19cm in azimuth (the spatial resolution is about 35cm). Two regions of interest are displayed together with optical images at 20cm resolution (orthorectified image by the French geographic institute (IGN)). Processing such an image is challenging for a despeckling algorithm because of the spatial correlations of speckle and the strong sidelobes around bright targets. Vegetation seems to be well restored: the stripes visible both in optical and SAR images are preserved and the low-contrasted tree response in the bottom row of Fig.5.5 is recovered. Few distortions seem to be applied to bright targets in the top row of Fig.5.5.

Figure 5 . 6 -

 56 Figure 5.6 -Restoration results of three despeckling filters on two TerraSAR-X images near Serre-Ponçon dam, in the French Alps, acquired in stripmap mode. The residual intensity images (i.e. the ratio noisy/denoised) is provided to assist in the visual analysis. Source: ©IEEE (Dalsasso et al., 2022b).

Figure 5 . 6

 56 Figure 5.6 and figure 5.7 compare the results produced by MERLIN with two other despeckling filters: SAR-BM3D (Cozzolino et al., 2013) and Speckle2Void (Molini et al., 2021). For each restoration result, the residual image (ratio between the noisy and the denoised image) is also shown. To account for speckle spatial correlations, before applying SAR-BM3D and Speckle2Void, images have been decorrelated. The blind speckle decorrelator proposed by Lapini et al. (Lapini et al., 2013) has been used, as suggested by the authors of Speckle2Void (Molini et al., 2021). Visual inspection of the residual images suggests that some structures have been attenuated by SAR-BM3D, with the edges appearing a bit fuzzy. Moreover, some artifacts arise in homogeneous areas.

Figure 5

 5 Figure 5.7 -Restoration results of three despeckling filters on a TerraSAR-X images of Berlin, in Germany, acquired in high-resolution spotlight mode. The residual intensity images (i.e. the ratio noisy/denoised) is provided to assist in the visual analysis. Source: ©IEEE. (Dalsasso et al., 2022b)

Figure 5 . 8 -

 58 Figure 5.8 -(a) A TerraSAR-X image in stripmap mode. (b) Image restored with MERLIN. (c)-(d) Images restored by a U-Net trained with self-supervision by replacing the real phase with a random phase sampled uniformly in [-π, π]. To produce the result shown in (d) input images are subsampled by a factor of two both at training time and at test time; pixels of the filtered image are then interpolated to recover the original image size. Source: ©IEEE (Dalsasso et al., 2022b).

Figure 6 . 1 -

 61 Figure 6.1 -Three approaches for self-supervised training strategies are possible: using temporal diversity (pairs of images captured at different dates); using spatial diversity (by masking part of the spatial information); and using the real / imaginary part diversity (by feeding only the real or imaginary part to the network and supervising the training with the other component). Each strategy requires a different preprocessing of the data to ensure that the statistical guarantees for self-supervised training are fulfilled.

  .1 and in Appendix B. We illustrate in this summary each self-supervised training strategy by despeckling TerraSAR-X Stripmap images with the following algorithms: • SAR2SAR (Dalsasso et al., 2021b), Chapter 4, trained using the temporal diversity (strategy ); • Speckle2Void (Molini et al., 2021), trained using the spatial diversity (strategy ); • MERLIN (Dalsasso et al., 2022b), Chapter 5, trained using the real / imaginary part diversity (strategy ). The networks in SAR2SAR and MERLIN share the same architecture derived from the U-Net of (Lehtinen et al., 2018). The network used in Speckle2Void has a specific architecture in order to enforce a blind spot at the center of the receptive field. To ensure a fair comparison, all three training strategies are applied on a dataset of TerraSAR-X Stripmap images. As a baseline, we also include the results of SAR-BM3D (Parrilli et al., 2011) which is not based on a deep neural network.

Figure 6 .

 6 Figure 6.2 -Results on two TerraSAR-X Stripmap images. For each method, the ratio noisy/denoised is shown to check if some structures are removed from the original image by the despeckling step.

Figure 6 .

 6 Figure 6.3 -Results of SAR2SAR and Multi-temporal SAR2SAR on an image corrupted by 1-look synthetic speckle. Both networks refer to step A of the algorithm. In the latter, the model is trained with an additional input being a superimage, i.e. a groundtruth image corrupted with 20-look synthetic speckle.

Figure 6 . 4 -

 64 Figure 6.4 -Preliminary result of uncertainty estimation performed by extending the SAR2SAR framework. Together with the despeckled image, the network outputs an uncertainty map β associated to its prediction. The map suggests that the model is confident on the reflectivity predicted in homogeneous areas and might have partially failed to restore the low contrasted structure on the bottom-left part of the image.

Figure B. 2 -

 2 Figure B.2 -The SpotLight acquisition mode requires to pre-process the images to obtain independents real and imaginary parts. If the training is carried out on the data presenting a degree of correlation between real and imaginary parts, the network can guess part of the speckle component: this explains the residual fluctuations that are left on the images of the central column. When the data is pre-processed at a patch level (both at training and at test time), ã and b are completely de-correlated and a network can be trained with MERLIN: results are shown on the right column.

Figure E. 3 -

 3 Figure E.3 -Second step: the compatible version of tensorflow, a popular deep learning framework, is installed.

Figure E. 4 -

 4 Figure E.4 -Third step: the network is applied on all images contained in the "test data" folder, where one can upload any single-look Sentinel-1 SAR image to be despeckled.

Figure E. 5 -

 5 Figure E.5 -In the test folder, the original images are saved together with the despeckled ones, both as numpy array containing the amplitude image in single-precision floatingpoint format floating and as png image files. They are ready to be displayed and downloaded.

Figure E. 6 -

 6 Figure E.6 -The png file of the despeckled image can be easily visualized.

  4.2.1 Network Architecture . . . . . . . . . . . . . . . . . . . . . . . . 64 4.2.2 Self-supervised learning . . . . . . . . . . . . . . . . . . . . . . . 67 4.2.3 Application to SAR image despeckling: a semi-supervised approach 68 4.3 Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70 4.4 Ablation study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75 4.5 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76 4.6 Adaptation to GRD data . . . . . . . . . . . . . . . . . . . . . . . . . . 77 4.6.1 Adaptation of SAR2SAR to GRD images . . . . . . . . . . . . . 78 4.6.2 Detection of narrow rivers in denoised GRD images . . . . . . . . 80 4.6.3 Experimental results . . . . . . . . . . . . . . . . . . . . . . . . . 81 4.6.4 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82 4.7 Application to multi-temporal time series . . . . . . . . . . . . . . . . . 83 4.7.1 Ratio-based filtering . . . . . . . . . . . . . . . . . . . . . . . . . 84 4.7.2 Adaptation of SAR2SAR to ratio images . . . . . . . . . . . . . . 84 4.7.3 Experimental results . . . . . . . . . . . . . . . . . . . . . . . . . 86 4.8 Summary of the chapter . . . . . . . . . . . . . . . . . . . . . . . . . . . 88 5 As if by magic: MERLIN

Table 1 .

 1 1 -Most used frequency bands in Radar Remote Sensing and the associated wavelength in the vacuum.

	Frequency band	P	L	C	X	Ku
	Frequency [GHz] [0.25; 0.5]	[1; 2]	[3.75; 7.5] [7.5; 12] [12; 17.6]
	Wavelength [cm] [60; 120] [15; 30]	[4; 8]	[2.5; 4] [1.7; 2.5]

Common types of RS Radar sensors are altimeters, sounders (also known as Ground Penetrating Radars), scatterometers, Synthetic Aperture Radar (SAR), Polarimeters and Interferometers.

  Cozzolino et al., 2013; Deledalle et al., 2010; Chen et al., 2010; Deledalle et al., 2014; Torres et al., 2014; Deledalle et al., 2015). All these pixel-selection methods form an estimate Ri (or Σ i in PolSAR and InSAR) at pixel i based on the noisy observed values I j (or C j ) in the neighborhood N i centered on pixel i, using a

	prevents the mixing of large intensities and lower intensities by restricting the averaging to values belonging to a range centered on the value of the central pixel. The local selection of an oriented averaging window (Lopes et al., 1990) or the selection of pixels by region growing (Vasile et al., 2006) help to reduce the blurring of edges. A robust approach to identify similar pixels consists of comparing image patches (Buades et al., 2005; Deledalle et al., 2011; Deledalle et al., 2012). So-called non-local methods perform a weighted averaging with weights derived from patch similarities (Deledalle et al., 2009; Parrilli et al., 2011; MERLIN

  Aubert and Aujol, 2008; Denis et al., 2009; Bioucas-Dias and Figueiredo, 2010), image decomposition priors like TV+ 1 and TV+ 0 (Denis et al., 2010; Lobry et al., 2016), or wavelet and curvelet transforms (Xie et al., 2002; Durand

homogeneous areas. The oversampling and spectral windowing operations are indeed not taken into account by Goodman's speckle model. Recent advances in self-supervised approaches for AWGN suppression (Lehtinen et al., 2018; Krull et al., 2019; Laine et al., 2019) allow training deep models directly on noisy data

  . Adapting them to SAR images is, therefore, of utmost importance, given the inherent scarcity of speckle-free images. SAR time-series can be exploited to this aim (Boulch et al., 2018; Ma et al., 2020; Dalsasso et al., 2021b): see Fig 2.5

	Contribution (1): A dataset for supervised methods
	(Dalsasso et al., 2020) Due to speckle temporal correlations, the average image
	generated from a multi-temporal stack may still present residual speckle fluc-
	tuations. To mitigate this phenomenon, we propose to apply a mild denoising
	step to the multi-looked image. Speckled images used to train CNNs are then
	produced by drawing random fluctuations according to the theoretical speckle
	distribution of Eq.(2.7), and a deep model is trained in a supervised fashion (see
	Fig.2.5 ).
	This contribution answers to Challenge (1) Training set and Challenge (2)
	Open code, section 1.3. It is presented in Chapter 3.
	Alternatively, large datasets can be built by adding simulated speckle noise to nat-
	ural images (see Image Despeckling -Convolutional Neural Network (ID-CNN) (Wang
	et al., 2017b), Image Denoising -Generative Adversarial Network (ID-GAN) (Wang et
	al., 2017a), SAR -Dilated Residual Network (SAR-DRN) (Zhang et al., 2018)). The
	statistical distribution of natural images is, however, quite different from that of SAR
	images (where point-like and linear structures are much more frequent), causing a prob-
	lem referred to as domain gap. Moreover, methods developed under the assumption of
	i.i.d. speckle suffer from artifacts due to the spatial correlation of speckle when applied
	to actual SAR images (Dalsasso et al., 2021a). Combining a spatial loss with a spectral
	term (Vitale et al., 2019) along with an edge-preserving term (Vitale et al., 2020) still
	produces artifacts in
	The network architecture, called SAR-CNN, is inspired from the Denoising Convolu-
	tional Neural Network (DnCNN) (Zhang et al., 2017). Cozzolino et al. (Cozzolino et al.,
	2020) introduced a method to combine a non-local filtering based on patches and deep
	learning: NL-CNN (Plötz and Roth, 2018), using a similar supervised training strat-
	egy. Building such a training set is far from easy: temporal changes are frequent and, if
	not properly accounted for, the network may end up producing a biased result. As an
	alternative, simulated speckle noise can be added to a ground-truth image obtained by
	temporal-averaging (Lattari et al., 2019) (see Fig. 2.5 ). Yet, due to speckle temporal
	correlations, this reference image may still present residual speckle fluctuations. The
	first contribution described in this Ph.D manuscript has been proposed in this context:

Table 3 .

 3

1 -Configuration of a DnCNN with D layers.

Table 3 .

 3

	Images	Number of Dates Number of Patches
	Marais 1	45	40194
	Limagne	53	40194
	Saclay	69	7227
	Lely	25	14850
	Rambouillet	69	39168
	Risoul	72	9648
	Marais 2	45	40194

2 -Description of the training-set of the proposed SAR-CNN. For each image, the number of dates composing the multi-temporal stack and the number of patches extracted are given. Network Architecture and the Effect of the Loss Function

  surfaces, fields and mountainous areas. Patches of size 40 × 40 are extracted from these images, with a stride of 10 pixels between patches. Mini-batches of 128 patches are used. In order to improve the network generalization capability, standard data augmentation techniques are used: vertical, horizontal flipping and ±90 • and 180 • rotations are applied on the patches. 11968 batches of 128 patches are processed during an epoch. A total of 50 epochs were used with ADAM stochastic gradient optimization method, with an initial learning rate of 0.001. The convergence of the learning and prevention for over-fitting were checked by monitoring the decrease of the loss function throughout the epochs as well as the performance over the test set. The deep learning framework used for the implementation is tensorflow 1.1.12. Training is carried out with an Intel Xeon CPU at 3.40 GHz and an Nvidia K80 GPU and took approximately 7 h.

Table 3 .

 3 3 -Description of the proposed algorithms.

	Algorithm MuLoG+CNN	MuLoG+CNN (Pretrained on SAR)	SAR-CNN
	Input	Natural images	SAR dataset	SAR dataset
	Noise type	Gaussian	Gaussian	Speckle
	Architecture DnCNN, D = 17	DnCNN, D = 17	DnCNN, D = 19
	Loss function	L 2	L 2	L

[START_REF] Miranda | Definition of the TOPS SLC deramping function for products generated by the S-1 IPF[END_REF]

Influence of the Loss Function and of the Network Depth

Table 3 .

 3 4 -Comparison of denoising quality in terms of PSNR on amplitude images. For each ground truth image, 20 noisy instances are generated. 1σ confidence intervals are given. Per-method averages are given at the bottom. For visual inspection, results on one noisy example for each image are provided in figure 3.8.

	Images	Noisy	SAR-BM3D	NL-SAR	MuLoG+BM3D MuLoG+CNN	MuLoG+CNN (Pretrained on SAR)	SAR-CNN
	Marais 1	10.05 ± 0.014 23.56 ± 0.134 21.71 ± 0.126		23.46 ± 0.079	23.39 ± 0.061	23.63 ± 0.068	24.65 ± 0.086
	Limagne	10.87 ± 0.047 21.47 ± 0.309 20.25 ± 0.196		21.47 ± 0.218	21.16 ± 0.025	21.85 ± 0.127	22.65 ± 0.291
	Saclay	15.57 ± 0.134 21.49 ± 0.368 20.40 ± 0.270		21.67 ± 0.245	21.88 ± 0.220	22.77 ± 0.240	23.47 ± 0.228
	Lely	11.45 ± 0.005 21.66 ± 0.445 20.54 ± 0.330		22.25 ± 0.437	22.17 ± 0.270	22.97 ± 0.367	23.79 ± 0.491
	Rambouillet 8.81 ± 0.069 23.78 ± 0.198 22.28 ± 0.113		23.88 ± 0.169	23.30 ± 0.114	23.30 ± 0.163	24.73 ± 0.080
	Risoul	17.59 ± 0.036 29.98 ± 0.264 28.69 ± 0.201		30.99 ± 0.376	30.85 ± 0.184	31.03 ± 0.201	31.69 ± 0.283
	Marais 2	9.70 ± 0.093 20.31 ± 0.783 20.07 ± 0.755		21.59 ± 0.757	21.00 ± 0.489	22.12 ± 0.679	23.36 ± 0.807
	Average	12.00	23.17	21.99		23.62	23.39	23.95	24.91
	Images	Noisy	SAR-BM3D	NL-SAR		MuLoG+BM3D MuLoG+CNN	MuLoG+CNN (Pretrained on SAR)	SAR-CNN
	Marais 1	0.3571 ± 0.0015 0.8053 ± 0.0018 0.7471 ± 0.0029	0.8003 ± 0.0020	0.7955 ± 0.0027	0.8072 ± 0.0024	0.8333 ± 0.0016
	Limagne	0.4060 ± 0.0021 0.8091 ± 0.0027 0.7493 ± 0.0033	0.8011 ± 0.0030	0.8055 ± 0.0027	0.8147 ± 0.0023	0.8327 ± 0.0029
	Saclay	0.5235 ± 0.0019 0.8031 ± 0.0032 0.7478 ± 0.0040	0.7734 ± 0.0034	0.7956 ± 0.0033	0.8156 ± 0.0030	0.8314 ± 0.0024
	Lely	0.3654 ± 0.0013 0.8473 ± 0.0023 0.8062 ± 0.0023	0.8552 ± 0.0025	0.8659 ± 0.0019	0.8703 ± 0.0018	0.8856 ± 0.0019
	Rambouillet 0.2886 ± 0.0017 0.7831 ± 0.0028 0.7364 ± 0.0031	0.7798 ± 0.0029	0.7706 ± 0.0095	0.7821 ± 0.0073	0.8002 ± 0.0026
	Risoul	0.4362 ± 0.0017 0.8306 ± 0.0024 0.7671 ± 0.0028	0.8345 ± 0.0030	0.8291 ± 0.0027	0.8341 ± 0.0024	0.8493 ± 0.0018
	Marais 2	0.2628 ± 0.0017 0.8506 ± 0.0026 0.8222 ± 0.0022	0.8561 ± 0.0025	0.8594 ± 0.0111	0.8677 ± 0.0097	0.8866 ± 0.0025
	Average	0.3771	0.8184	0.7680		0.8143	0.8173	0.8273	0.8460

Table

3

.5 -Comparison of denoising quality evaluated in terms of SSIM on amplitude images. For each ground truth image, 20 noisy instances are generated. 1σ confidence intervals are given. Per-method averages are given at the bottom. For visual inspection, results on one noisy example for each image are provided in figure 3.8.

Table 3 .

 3 6 -ENL estimation on denoised 1-look Sentinel-1 SAR acquisitions and on a TerraSAR-X image.

	Images	SAR-BM3D NL-SAR MuLoG+BM3D MuLoG+CNN	MuLoG+CNN (Pretrained on SAR)	SAR-CNN
	Sentinel-1:
	Marais 1	226.48	165.24	132.30	288.70	210.17	177.72
	Lely	166.60	75.19	349.32	82.24	145.07	289.03
	Rambouillet	262.47	171.42	139.62	413.09	383.81	295.30
	Marais 2	119.99	213.45	84.67	146.33	182.44	206.93
	TerraSAR-X:
	Saint Gervais	40.01	39.70	39.37	45.18	129.66	59.21

Table 3 .

 3 7 -Time for despeckling a 500 × 500 clip. Experiments were carried out with an Intel Xeon CPU at 3.40 GHz and an Nvidia K80 GPU. For NL-SAR, the radius of the smallest/largest search window size is set to 1/20 and the half-width of the smallest/largest patches as 0/10.

	SAR-BM3D NL-SAR MuLoG+BM3D MuLoG+CNN SAR-CNN
	73.89 s	116.28 s	59.82 s	80.43 s	0.19 s

Table 3 .

 3 8 -Advantages and disadvantages of the CNN-based despeckling strategies considered in this chapter.

		•	No specific training is needed
			(improvement is small when the CNN is trained on
			SAR images)
	Pros	•	Straightforward adaptation to multiple looks
	MuLoG+CNN	•	Straightforward generalization to polarimetric and/or in-
			terferometric SAR images
		•	Straightforward adaptation to different SAR sensors
	Cons •	High runtime due to its iterative procedure
		•	Provides the highest performances
		•	Fastest runtime performances once the network is trained
	Pros	•	Possible adaptation to multiple looks (requires re-training)
	SAR-CNN	•	Possible adaptation to different SAR sensors (requires re-training)
		•	Requires a training: formation of a dataset of speckle-free
			SAR images
	Cons

•

Generalization to multi-channel SAR images (polarimetric and/or interferometric) raises dimensionality issues: very large training set to sample the diversity of radar images

  This chapter is mainly inspired from our paper SAR2SAR: A Semi-Supervised Despeckling Algorithm for SAR Images(Dalsasso et al., 2021b). SAR despeckling algorithms generally rely on the hypothesis of spatially uncorrelated speckle components. However, this is not the case for real SAR images[START_REF] Argenti | A tutorial on speckle reduction in Synthetic Aperture Radar images[END_REF][START_REF] Abergel | Subpixellic Methods for Sidelobes Suppression and Strong Targets Extraction in Single Look Complex SAR Images[END_REF]. In the previous chapter, we have discussed the shortcomings of existing approaches to handle spatially correlated speckle(Dalsasso et al., 2021a).

Table 4 .

 4 1 -Comparison of denoising quality in terms of PSNR on amplitude images. For each ground truth image, 20 noisy instances are generated. 1σ confidence intervals are given. Per-method averages are given at the bottom.

	Images	Noisy	SAR-CNN	U-Net
	Marais 1	10.05 ± 0.0141 24.65 ± 0.0860 25.61± 0.1089
	Limagne	10.87 ± 0.0469 22.65 ± 0.2914 24.25 ±0.1178
	Saclay	15.57 ± 0.1342 23.47 ± 0.2276 23.39±0.3991
	Lely	11.45 ± 0.0048 23.79 ± 0.4908 23.38±0.6130
	Rambouillet 8.81 ± 0.0693 24.73 ± 0.0798 23.92±0.3900
	Risoul	17.59 ± 0.0361 31.69 ± 0.2830 30.25±0.2561
	Marais 2	9.70 ± 0.0927	23.36 ± 0.8068 26.52 ±0.1942
	Average	12.00	24.91	25.33

Table 4

 4

	Images	Noisy	SAR-BM3D	NL-SAR	MuLoG+BM3D	SAR-CNN	noise2noise SAR2SAR A
	Marais 1	10.05±0.014 23.56±0.134 21.71±0.126	23.46±0.079	24.65±0.086 25.31±0.108 25.73±0.125
	Limagne	10.87±0.047 21.47±0.309 20.25±0.196	21.47±0.218	22.65±0.291 24.08±0.110 24.45±0.119
	Saclay	15.57±0.134 21.49±0.368 20.40±0.270	21.67±0.245	23.47±0.228 23.50±0.385 23.60±0.437
	Lely	11.45±0.005 21.66±0.445 20.54±0.330	22.25±0.437	23.79±0.491 23.25±0.367 23.67±0.542
	Rambouillet 8.81±0.069 23.78±0.198 22.28±0.113	23.88±0.169	24.73±0.080 23.73±0.388 24.16±0.385
	Risoul	17.59±0.036 29.98±0.264 28.69±0.201	30.99±0.376	31.69±0.283 29.93±0.216 30.68±0.230
	Marais 2	9.70±0.093 20.31±0.783 20.07±0.755	21.59±0.757	23.36±0.8070 26.15±0.211 26.63±0.215
	Average	12.00	23.17	21.99	23.62	24.91	25.13	25.56

.2 -Comparison of denoising quality in terms of PSNR on amplitude images. For each ground truth image, 20 noisy instances are generated. 1σ confidence intervals are given. Per-method averages are given at the bottom.

Table 4 .

 4 3 -At each step of the iterative process, the Wasserstein distance is computed to measure the distance between the theoretical speckle distribution of equation 2.7 and the empirical distribution of the residual speckle in intensity format (the lower the better).

	Training proceeds for 20 more epochs with a learning rate decreased by a factor of 100
	w.r.t. the initial value. Five time series of respectively 53 (Limagne), 45 (Marais 1),
	45 (Marais 2), 69 (Rambouillet) and 25 (Lely) dates compose the training set. 2896
	image patches are organized into 724 batches of 4 patches each. Learning is thereby
	transferred to correlated speckle.
				SAR2SAR C
	restoration method	SAR2SAR A SAR2SAR B	iter 1 iter 2 iter 3 iter 4 iter 5
	Wasserstein distance	0.147	0.063	0.086 0.109 0.128 0.144 0.158

Dalsasso et al., 2021a)

.

Table 4 .

 4 4 -Comparison of results between the proposed method and the baseline method

		Image		Baseline method	Proposed Method
	#-name	Polarization Size (pixels) Pre % Rec% F-score Pre % Rec % F-score
	1 -Des Moines	VH	1313×1750	92.44 93.35	92.89	92.54 94.54 93.53
	2 -Sunar	VH	1026 × 923 82.36 81.71	82.03	79.12 86.17 82.49
	3 -Gaoual	VH	927×1854	92.51 89.09	90.77	93.90 90.00 91.91
	4 -Garonne	VV	1109×1704	97.60 82.44	89.38	97.69 83.23 89.89
	5 -Redon	VH	618×773	90.71 92.34 91.52	90.41 92.60	91.49
	6 -Régina	VH	553×1216	89.33 82.95	86.02	90.92 83.98 87.31

Table 5 .

 5 1 -Description of the training parameters for all experiments carried out with a residual U-Net trained with MERLIN.

		Synthetic speckle	TerraSAR-X stripmap	TerraSAR-X HS spotlight	SETHI
	# images	7	3	4	1
	patch size	256 × 256	256 × 256	256 × 256	256 × 256
	batch size	12	12	12	12
	# patches	12420	50604	27048	76260
	# batches	1035	4217	2254	6355
	# epochs	30	30	30	30
	gradient norm (?)

Table 5 .

 5 2 -Comparison of denoising quality in terms of PSNR on amplitude images. For each ground truth image, 20 noisy instances are generated. 1σ confidence intervals are given. Per-method averages are indicated at the bottom.

	Images	Noisy	SAR-BM3D	NL-SAR	MuLoG+BM3D	SAR-CNN	SAR2SARA	Speckle2Void	MERLIN
			(patch-based)	(patch-based)	(patch-based) (deep network) (deep network) (deep network) (deep network)
						(supervised)	(supervised) (self-supervised) (self-supervised)

  The statistical model for SAR image formation presented in Section 2.2 and figure 2.2 served as a basis to derive the loss function used to train networks with MERLIN, Eqs.(5.1) and (5.2). It is based on Goodman's speckle model which is well-established for homogeneous areas imaged at a medium to high resolution. It is known to be less relevant when very high-resolution images are considered, especially in urban areas due to the presence of strong scatterers that dominate the responses within a resolution cell. Many alternative statistical models were proposed in the literature(Ovarlez et al., 2015) (Sportouche et al., 2017) (Nicolas and Tupin, 2019) (Yue et al., 2021).

	Intensity image	real part squared Re 2 {z}	imaginary part squared Im 2 {z}
		estimated reflectivity	estimated reflectivity
	MERLIN's final estimate	(network applied to Re 2 {z})	(network applied to Im 2 {z})

Results on Sendai, patch 1, Stripmap mode Intensity image real part squared Re 2 {z} imaginary part squared Im 2 {z} estimated reflectivity estimated reflectivity MERLIN's final estimate (network applied to Re 2 {z}) (network applied to Im 2 {z}) Results on Sendai, patch 2, Stripmap mode Intensity image real part squared Re 2 {z} imaginary part squared Im 2 {z} estimated reflectivity estimated reflectivity MERLIN's final estimate (network applied to Re 2 {z}) (network applied to Im 2 {z}) Results on Salon-de-Provence, patch 7, High Resolution SpotLight (HS) mode Intensity image real part squared Re 2 {z} imaginary part squared Im 2 {z} estimated reflectivity estimated reflectivity MERLIN's final estimate (network applied to Re 2 {z}) (network applied to Im 2 {z})

Results on Salon-de-Provence, patch 9, High Resolution SpotLight (HS) mode

5.4 Discussion

  2021b) (presented in Chapter 4), spatial (Molini et al., 2021), or even real / imaginary parts of SLC images (Dalsasso et al., 2022b) (described in Chapter

	• Dalsasso, E., Denis, L., Muzeau, M., & Tupin, F. (2022). Self-supervised
	training strategies for SAR image despeckling with deep neural networks.
	To be presented at the 14h European Conference on Synthetic Aperture
	Radar (EUSAR 2022). (Dalsasso et al., 2022a)
	Self-supervised despeckling methods are based on a common principle: if a signal
	contains a deterministic and a stochastic component, only the deterministic component
	can be predicted given an independent realization of the signal. Self-supervised training
	then starts by splitting the data into subsets between which speckle is statistically
	independent: y in k and y val k . A network is then trained to predict, from a first subset of
	the data y in k , the observations of the second subset y val k . The splitting of the data into y in

k and y val k can exploit different kinds of information diversity: temporal (Dalsasso et al.,

  Dalsasso et al., 2021b) Speckle2Void (Molini et al., 2021) MERLIN (Dalsasso et al., 2022b)

		Temporal diversity	Spatial diversity	Real/imaginary diversity
	SAR2SAR (training data requirement pairs of registered intensity images	intensity images	SLC images
	preprocessing impact	bias if imperfect change detection/compensation	spectral equalization changes image appearance	limited impact
	training complexity	+++	++	+
	flexibility of net architecture	+++	+	+++
	fine tuning to new images	+	+++	+++

Summary of the advantages and limitations of each self-supervised training strategy.

As data acquired by SAR systems are often treated with Image Processing techniques, from now on I will interchangeably use the term SAR image and SAR data.

CHAPTER 2. STATE-OF-THE-ART OF SAR DESPECKLING

since the log-speckle is not centered, a compensation is added to prevent a bias with the L2 loss, which is replaced by f θ (y 1 ) -y

+ ψ(L) -log L 2

https://gitlab.telecom-paris.fr/RING/SAR2SAR

we consider here that the raw phase angle(z) is non-informative, which of course is no longer true when considering multiple SLC images in interferometric configuration

2015-10-06 2015-10-18 2015-10-30 2015-11-23 2015-12-17 2016-01-10 2016-02-03 2016-02-15 2016-02-27 2016-03-22 2016-04-15 2016-04-27 2016-05-09 2016-05-21 2016-06-02 2016-06-14 2016-07-20 2016-08-01 2016-08-13 2016-09-06 SAR time series acquired over Lelystad, Netherlands.

Remerciements Acknowledgments

The airborne SAR images processed in this chapter were provided by ONERA, the French aerospace lab, within the project ALYS ANR-15-ASTR-0002 funded by the DGA (Direction Générale à l'Armement) and the ANR (Agence Nationale de la Recherche). Some of the TerraSAR-X images have been provided by the German Space Agency DLR for the project DLR-MTH0232. COP Climate Change Conference. 1 CRF Conditional Random Field. 79, 80 DEM Digital Elevation Model. 77 DnCNN Denoising Convolutional Neural Network. 24, 32 EM Electromagnetic. 2-5 ENL Equivalent Number Of Looks. 10, 44, 45, 58, 72, 77, 85 EO Earth Observation. 1, 82 ESA European Space Agency. 9, 76 GLRT Generalized Likelihood Ratio Test. 79 GPU Graphics Processing Unit. 47 GRD Ground Range Detected. 58, 62, 76, 77, 79, 81, 85, 87 GRD IW HD Ground Range Detected Interferometric Wide swath High definition Dual polarization. 76, 80 GRWL Global River Widths from Landsat. 79 HS High Resolution SpotLight. 97, 100 HSI Hyperspectral Images. 15 ID-CNN Image Despeckling -Convolutional Neural Network. 25 ID-GAN Image Denoising -Generative Adversarial Network. 25 IGN French geographic institute. 98 i.i.d. independent and identically distributed. 9, 16, 18, 25, 30, 50, 55, 91, 93, 121 IMAGES team Images, Modélisation, Analyse, GEométrie et Synthèse team. 11 InSAR SAR Interferometry. 19, 21 IPCC Intergovernmental Panel on Climate Change. 1 IW Interfeometric Wide swath. 103 LTCI Laboratoire Traitement et Communication de l'Information. 11 MuLoG MUltichannel LOgarithm with Gaussian denoising. 23, 30, 32, 33, 37, 47 PCR Pulse Compression Ratio. 6 PolInSAR Polarimetric and Interferometric SAR. 19, 21 PolSAR SAR Polarimetry. 19, 21, 115 PSF Point Spread Function. 9 PSNR Peak-Signal-to-Noise Ratio. 38, 64, 95 RADAR RAdio Detection and Ranging. 3 RAR Real Aperture Radar. 4, 6 RS Remote Sensing. 1-3, 13, 116 SAR Synthetic Aperture Radar. 3, 4, 6-13, 15, 16, 18, 19, 21, 23, 25-27, 29-31, 33-35, 37-41, 47, 49, 50, 55, 57, 58, 61-64, 67, 68, 71, 74-76, 80, 82, 83, 85, 87, 89-91, 93-96, 98, 103-105, 107, 109, 111-116, 121-123 SAR-DRN SAR -Dilated Residual Network. 25 SLAR Side-Looking Aperture Radar. 4 SLC Single-Look Complex. 13, 27, 62, 76, 89-93, 103, 104, 107, 109, 110, 112 SNR Signal-to-noise ratio. 6, 91, 100, 104 SSIM Structural SIMilarity. 38 TOPSAR Terrain Observation with Progressive Scans SAR. 103 TV Total Variation. 22, 55-57, 59 UN United Nations. 1

Context of the work

The question motivating this work is the following: how to learn speckle spatial correlations during training of a neural network for speckle reduction?

List of Publications

Journal papers International conferences:

Appendices

Appendix A

Condition on the SAR response to obtain independent real and imaginary parts MERLIN self-supervised training procedure is based on the statistical independence of ã and b, the real and imaginary parts of the single-look complex SAR image. Under Goodman's fully-developed speckle model, the real and imaginary parts a and b obtained with an ideal SAR system are i.i.d. under a Gaussian distribution N 0, 1 2 diag(R) , i.e., z ≡ a + jb follows a circular complex Gaussian distribution with covariance 1 2 diag(R). A radar system modeled by the linear (possibly complex-valued) spatial-domain operator H produces correlated samples z = Hz distributed according to a circular complex Gaussian distribution with covariance matrix C = Hdiag(R)H (where • denotes the conjugate transpose). The real and imaginary parts ã and b are jointly distributed according to a Gaussian distribution:

with Σ the covariance matrix defined by blocks:

and H = M + jN. The expansion of equation (A.2) leads to the following expressions for the 4 blocks:

This shows that ã and b are independent if and only if

and by choosing R = e i (the i-th elementary vector, i.e., r k = 0 if k = i and r i = 1), equation (A.4) gives:

Matrix m i n i is thus symmetric and at most of rank one. Vectors m i and n i are thus collinear: there exist a couple of scalars (λ i , τ i ) and a vector q i ∈ R K such that m i = λ i q i and n i = τ i q i (setting λ i or τ i to 0 leads to a rank 0 matrix). Therefore, condition (A.4) implies that matrices M and N take the form:

with λ and τ any vector of R K . It can be easily checked that this necessary condition is sufficient for equation (A.4) to hold. Real and imaginary components ã and b are thus statistically independent if and only if the SAR system response H can be written under the form:

where Q and Q are real-valued linear operators and exp(jϕ) is a K-dimensional phase vector (i.e., a vector of K complex values, each with unit magnitude). Equation (A.8) shows that the SAR response must take the form of a per-pixel phase shift followed by a real-valued linear operator. If the SAR system has a shift-invariant response, then all ϕ i are equal and H = exp(jϕ)Q (where ϕ is a scalar): a constant phase shift is applied to the whole image. Linear operators H and Q can then be diagonalized by the 2D discrete Fourier transform: H = F diag( h)F, with h the frequency response of the SAR system, and

| is even and the gain of the SAR system | h| must also be even in order for the system to produce independent real and imaginary parts ã and b.

Appendix B

Pre-processing of image patches to obtain independent real and imaginary parts

Due to a non-zero Doppler centroid or an acquisition mode with a time-varying squint angle, the frequency response of the SAR system may be asymmetrical, leading to correlated real and imaginary parts in the SAR images. This would prevent the application of MERLIN. To avoid this problem, we preprocess each patch to recenter its spectrum. The U-Net employed in Chapter 5 takes as input patches with 256 × 256 pixels. These patches are preprocessed as follows.

To correct for a non-zero Doppler at the scale of the patch, we compute the azimuth and range 1D profiles by averaging the magnitude of the 2D Fourier transform of the patch along the range and azimuth directions, respectively. Let p denote one such profile. We look for the shift δ such that the translated profile T δ {p} and its symmetric S{T δ {p}} superimpose at best: The airborne image shown in Fig. 5.5 has a spectrum with an asymmetrical support. After centering the spectrum, we built a mask corresponding to all frequencies (ν x , ν y ) such that their symmetric counterparts (-ν x , ν y ), (ν x , -ν y ), and (-ν x , -ν y ) are all within the bandwidth of the SAR system (i.e., non-zero). We then cut all frequencies off beyond our symmetrical mask to ensure the symmetry of the spectrum.

Appendix C

Example of SAR time series

A key step of the SAR2SAR algorithm (Chapter 4 and (Dalsasso et al., 2021b)) is the change compensation step. The ablation study shows that, if changes are not accounted for, the network produces biased results. Hereafter, a multi-temporal stack of 20 SAR images acquired approximately every 12 days is displayed. Changes are frequent and even between close dates many pixels are likely to be affected by changes, especially in the crop fields.

Appendix D

Supplementary results of SAR-CNN and SAR2SAR on Sentinel-1 images

Noisy SAR-CNN SAR2SAR

Results on Marais 1, date 4

Noisy SAR-CNN SAR2SAR

Results on Marais 1, date 5

Noisy SAR-CNN SAR2SAR

Results on Lelystad, date 3

Noisy SAR-CNN SAR2SAR

Results on Rambouillet, date 2

Noisy SAR-CNN SAR2SAR

Results on Marais 2, date 4

Noisy SAR-CNN SAR2SAR

Results on Marais 2, date 5

Appendix E

Example of shared code: Jupyter Notebook of SAR2SAR for single-look SAR data

The code associated to each developed method has been made freely available on the Gitlab of our research team: https://gitlab.telecom-paris.fr/RING. In the following images, the code associated to SAR2SAR is shown through a series of screenshots.

The shared code can be intended both as a demo (some images are provided with the code) and as a tool that any researcher can use for his/her purposes on any Sentinel-1 image. Codes associated to SAR-CNN, SAR2SAR for GRD images and MERLIN for stripmap images and High-Resolution SpotLight images follow the same framework. Observation purposes regardless of the daylight or cloud cover. Nowadays, thanks to the Copernicus program of the European Space Agency, a huge amount of SAR data is freely available. However, the exploitation of satellite SAR images is limited by the presence of strong fluctuations in the backscattered signal. Indeed, SAR images are corrupted by speckle, a phenomenon inherent to coherent imaging systems. In this Ph.D thesis, we aim to improve the interpretation of SAR images by resorting to speckle reduction techniques. Existing approaches are based on Goodman's model, which describes the speckle component as a spatially uncorrelated multiplicative noise. In the computer vision field, denoising methods relying on Convolutional Neural Networks (deep learning approaches) have led to great improvements and provide nowadays state-of-the-art results. We propose to use deep learning-based denoising techniques to reduce speckle from SAR images (despeckling methods). At first, we study the adaptation of supervised techniques that minimize a certain distance between the estimation provided by the CNN and a reference image, also called "groundtruth". We propose to create a dataset of reference images by averaging multi-temporal images acquired over the same area. Pairs of reference and corrupted images can be generated by synthetizing speckle following Goodman's model. However, in real images the speckle component is spatially correlated which typically requires subsampling these images by a factor 2 to reduce the spatial correlations, which also degrades the spatial resolution.

Given the limits of supervised approaches and inspired by noise2noise, a self-supervised denoising method, we propose to train our networks directly on actual SAR images. The principle of self-supervised denoising methods is the following : if a signal contains a deterministic component and a random component, then a network trained to predict a new signal realization from a first independent signal realization will only predict the deterministic component, i.e., the underlying scene, thereby suppressing the speckle. In the method we have developed, SAR2SAR, we leverage multi-temporal SAR series to obtain independent realizations of the same scene, under the hypothesis of temporally decorrelated speckle. Changes are compensated by devising an iterative training strategy. SAR2SAR is thus trained directly on images with spatially correlated speckle and can readily be applied on SAR images without subsampling, providing high-quality results. The training of SAR2SAR is quite heavy : it is articulated in several steps to compensate changes and a dataset comprising stacks of images must be built. With our approach "MERLIN", we alleviate the training by proposing a single-image learning strategy. Indeed, in single-look-complex SAR images, real and imaginary parts are mutually independent and can be naturally exploited to train CNNs with self-supervision. We show the potential of this training framework for three imaging modalities, different in terms of spatial resolution, textures, and speckle spatial correlation. For the sake of open science, the code associated to each algorithm developed is made freely available.

Institut Polytechnique de Paris 91120 Palaiseau, France