Andre Perez

Johan Rothman

Lydie Mathieu

Nicolas Baier

Marc Be- Ranger

Viviane Cattin

Alessandro Meoli

Donatas Miklusis

Irene Huertas

Marti Vilella

Martin Schwendener

Planetary exploration missions require some challenging phases during the landing of a spacecraft or a payload, such as a rover. The study and exploration of the Moon, Mars and other extraterrestrial bodies are expected to be performed in the near future by unmanned missions and therefore an autonomous procedure is essential for the precise and fast navigation operations. The support of a 3D vision for the Guidance Navigation and Control (GNC) system allows the creation of a Digital Elevation Map (DEM) of the target terrain, used to identify and avoid dangerous sites and adapt the course of the spacecraft to a safe location. This ability can ultimately grant the possibility to explore challenging reliefs, such as craters, more efficiently.

Flash Light Detection and Ranging (LiDAR) systems are being considered one of the best alternatives for such 3D vision sensors and a key technology in the future of relative proximity navigation. These systems are constituted by a detector composed by a large array of Avalanche PhotoDiode (APD) and a LASER. The differential of the Flash LiDAR is that the light pulse emitted by the LASER illuminates the entire target terrain at once without employing a scanning device, making the system more compact and reliable.

This thesis describes the optimization of a Flash LiDAR system based on a Mercury Cadmium Telluride (MCT)-APD detector developed by CEA-LETI. MCT-APDs have characteristics that allow the fast detection of the returned LASER pulse with a good Signal-to-Noise Ratio (SNR). The material provides a high gain at a low reverse bias and a low excess noise factor, which makes it a good candidate for applications with expected low number of photons, such as long distance measurements. The Read-Out Integrated Circuit (ROIC) responsible for controlling the detector is designed to output two frames per pulse: a 2D, or intensity, image corresponding to the integration of the light pulse and a 3D, or Time-of-Flight (TOF), image that can be translated into a range measurement.

At a first moment the constituent parts, apart from the detector, were designed and characterized aiming the miniaturization of the system. This optimization phase included the integration of a new and powerful pulsed LASER, the design of an optical system and the development of a dedicated electronic control board. The established constraints of weight and size were taken into consideration in the creation of this new prototype for the future experimental campaign.

The prototype was used to investigate the major faults that corrupt the quality of the image. A set of imaging processing algorithm and methods were developed in order to improve the performance of the range measurements according to two criteria: accuracy and precision. Finally, a final test campaign was performed at the European Space Agency (ESA) campsite in Noordwijk, Netherlands. The prototype system was mounted onto a robotic arm to simulate the landing on a target terrain according to different motion profiles.

Two of the calibration methods developed in the context of this thesis were presented at

Résumé

Les missions d'exploration planétaire ont des phases difficiles lors de l'atterrissage d'un vaisseau spatial ou d'une charge utile, telle qu'un rover. L'étude et l'exploration de la Lune, de Mars ou d'autres corps extraterrestres devraient être effectuées prochainement par des missions sans assistance humaine; une procédure autonome est donc essentielle pour les opérations de navigation précises et rapides. L'utilisation d'une caméra 3D dans le système de guidage, de navigation et de contrôle (GNC) permet la création d'une carte numérique d'élévation (DEM) du terrain ciblé, utilisé afin d'identifier et éviter les sites dangereux et adapter le parcours du vaisseau spatial à un endroit sûr. Inversement cela peut permettre d'explorer plus efficacement des reliefs difficiles, comme les cratères.

Les systèmes Flash LiDAR sont considérés comme l'une des meilleures alternatives pour ces capteurs de vision 3D. C'est une technologie clé dans l'avenir de la navigation à différentes distances. Ces systèmes sont constitués d'un détecteur composé d'une matrice de photodiodes à avalanche (APD) et d'un LASER. L'avantage principal du Flash LiDAR réside dans le fait que l'impulsion lumineuse émise par le LASER éclaire tout le terrain cible sans utiliser de balayage mécanique, ce qui rend le système plus compact et fiable.

Cette thèse décrit l'optimisation d'un système Flash LiDAR basé sur un détecteur de tellurure de mercure-cadmium (MCT) développé par le CEA-LETI. Les MCT-APDs ont des caractéristiques leur permettant la détection rapide de l'impulsion LASER envoyée avec un bon rapport signal sur bruit (SNR). En effet, le MCT permet la réalisation d'APD ayant un gain élevé pour une tension de polarisation modeste. Ces photodiodes possèdent également un faible facteur d'excès de bruit, ce qui en fait un bon candidat pour les applications. Le circuit intégré de lecture (ROIC) chargé de contrôler le détecteur est conçu pour enregistrer deux trames par impulsion: une image 2D, ou d'intensité, correspondant à l'intégration de l'impulsion lumineuse et une image 3D, ou de temps de vol (TOF), pouvant être traduite en mesure de distance.

Dans un premier temps chaque composant, en dehors du détecteur, a été identifié et caractérisés dans le but de miniaturiser le système. Cette phase d'optimisation comprenait l'intégration d'un LASER plus puissant, ainsi que la conception d'un système optique et le développement d'une carte de commande électronique dédiée. Les contraintes de poids et de taille établies ont été prises en compte lors de la création de ce nouveau prototype pour la future campagne expérimentale.

Le prototype a été utilisé pour étudier les principaux défauts qui altèrent la qualité de l'image. Un ensemble d'algorithmes et de méthodes de traitement d'image a été mis au point afin d'améliorer les performances des mesures de distance en fonction de deux critères: justesse et fidélité. Enfin, une dernière campagne de test a été réalisée sur le site de l'Agence Spatiale Européenne (ESA) à Noordwijk, aux Pays-Bas. Le prototype a été monté sur un bras robotisé pour simuler l'atterrissage sur un terrain cible selon différents profils de mouvement.

Deux des méthodes d'étalonnage développées dans le cadre de cette thèse ont été présentées à la Conférence internationale sur l'optique spatiale (ICSO 2018) et au Symposium international sur la géoscience et la détection à distance (IGARSS 2019).

List of Figures

List of Acronyms

Introduction

Autonomous navigation has become a central technology for space missions with applications varying from rover operations to the descent and landing of spacecraft. In the past decades, traditional 2D passive cameras have been broadly used for this task [Mourikis 2009], sometimes with the aid of active sensors, such as microwave radars [Clark 1998]. The development of 3D camera technology, capable of providing range measurements, is leading to a promising option to meet the challenging requirements of space exploration, mainly in terms of reliability, frame rate, power consumption and demands to the Central Processing Unit (CPU). A lot of effort has been dedicated on the evaluation of the most suitable strategy to record the range measurements and compose the 3D frame. In this context, Light Detection and Ranging (LiDAR) systems are rising as a prominent alternative, satisfying both the performance specifications and restrictive conditions of space operations. Generally speaking, two kinds of LiDAR systems exist. Scanning LiDARs, on one hand, have few photodetectors and need a scanning mechanism to form the 3D frame of a scene. Flash LiDARs, on the other hand, illuminate the whole scene at a time and detect the reflected pulse with a Focal Plane Array (FPA). As a result, there is no need for a moving scanning mechanism or complex data processing to form the 3D frame, which is a clear advantage for space applications.

Commissariat à l'énergie atomique et aux énergies alternatives (CEA) has developed a prototype of a Flash LiDAR detector operating at the Medium-Wavelength Infra-Red (MWIR) and based on the Mercury Cadmium Telluride (MCT) technology. MCT avalanche photodiodes provide high gains at low reverse bias with a low excess noise factor, being particularly well suited for scenarios with low incoming flux of photons, as expected in some space applications. This detector provides the Time-of-Flight (TOF) measurement (3D information) and the returned intensity of the LASER pulse (2D information) at the same instant for every acquisition frame. This feature could be used to improve the overall quality of the 3D measurements, in terms of accuracy and precision, by fusing both 3D and 2D information. Note that the detector used in this work is a test component, thus it has a larger number of defects than it would have as a final product. However, its properties are still well relevant for the evaluation of the technology at the designated scenario.

The focus of this work is to optimize a Flash LiDAR system prototype targeting space applications, specifically the operation of landing on extraterrestrial bodies. The optimization of the system follows the development of a set of algorithms to improve the quality of the measurements according to some predefined criteria. In a first moment, the main components of the system (light source, optical elements and electronics) are integrated with the detector in a compact fashion aiming to meet some weight and size requirements. Second, calibration techniques and data treatment schemes are developed and tested in laboratory conditions. Finally, a validation campaign using a robotic arm to simulate descent to a target, at the European Space Agency (ESA)'s campsite, is performed to demonstrate the system performance.

The description of the principle of operation of a LiDAR, as well as the different approaches for the 3D frame formation, are discussed in section 1.1. In section 1.2 we detail how this technology can be applied for space operations and which are the existing systems in this context. Finally, section 1.3 presents the content of this work.

3D Cameras and LiDARs

A 3D camera is a device that enables the perception of depth in images. This concept has vast applications in all sort of fields: it is used to make high-resolution maps [START_REF] Vosselman | [END_REF]]; for control and navigation of autonomous cars [START_REF] Hsu | [END_REF]]; for atmospheric remote sensing and meteorology [START_REF] Wilkerson | [END_REF]]; for military [START_REF] Harney | Military applications of coherent infrared radar[END_REF]] and law enforcement [Solomon 2006]; and for video games [START_REF] Ten | How Kinect depth sensor works-stereo triangulation[END_REF]] and digital modeling of objects and buildings [START_REF] Rocchini | [END_REF]]. This section describes the various strategies that have been proposed in the literature to assess the particular needs of different applications.

Stereo cameras capture 3D images by simulating human binocular vision using two or more lenses with distinct sensors. It has been implemented as a principle for 3D movies, for example, and as ranging system for cars driven with autonomous intelligence [START_REF] Bertozzi | [END_REF]]. Depth is inferred by computing a disparity field at each pixel location. Although stereo cameras have been used to obtain morphologic and topographic information of extraterrestrial bodies [Neukum 2004], its computational and memory requirements could be prohibitive at a scenario where high frame rates are of major concerns. Active sensors measure depth by illuminating the scene with a light source and can be divided in two main categories: projected-light and TOF sensors. Projected-light sensors, like the first version of Kinect (Figure 1.2), use a light pattern in combination with a standard 2D camera to measure distance via triangulation [Furht 2008]. They are used mainly for 3D scanning and modeling of objects. Due to their structure, power and sensor detection accuracy, they are typically applied for short distance measurements (tens of meters) and it is thus unsuitable for descent and landing, which requires 3D measurement capabilities at hundreds of meters.

TOF cameras, on the other hand, resolve distance based on the known speed of light, by measuring the delay between light emission and its detection [START_REF] Hansard | Time-of-flight cameras: principles, methods and applications[END_REF]]. The light source of these cameras are usually a Light Amplification by Stimulated Emission of Radiation (LASER) device. This direct approach decreases the need for processing power and is therefore suitable for applications with restrictions on CPU capability. This category of cameras can be subdivided in two: phase-based or pulse-based systems.

In phase-based system, the LASER operates in Continuous Wave (CW) mode and is modulated with a frequency ranging from 10 to 100 MHz. The distance is determined by measuring the phase difference between the signal sent and the one received after been reflected from the target. The relationship between the range to the surface and the measured phase shift is given by the following equation [START_REF] Oggier | [END_REF]]:

d = cφ 4πf (1.1)
where c is the speed of light; φ is the phase difference and f is the modulated frequency. As a consequence of this measurement strategy, phase-based systems have distance ambiguities for scenes deeper than the modulated wavelength. The ambiguity range (AR) is determined by:

AR = c 2f (1.2)
This peculiarity is problematic for long distance applications because there is a tradeoff between range and Signal-to-Noise Ratio (SNR) needed to resolve the ambiguities. At short distances, on the other hand, phase-based systems have good SNR as a result of the extended integration time [START_REF] Christy | [END_REF]]. The disadvantages of longer integration times is the higher susceptibility to background illumination and multiple reflections. Pulse-based systems, also called LiDAR systems, determine distance directly from the time delay between the emission of a light pulse and the detection of its reflection. This TOF measurement τ is related to the distance d by the following simple equation:

d = cτ 2 (1.
3)

The first 3D LiDAR systems were composed by few individual detectors, so a scanning procedure was necessary to obtain an image with a large number of pixels [McManamon 2012]. The so called Scanning LiDARs require a high repetition rate LASER in order to provide acceptable frame rates. Its moving parts also compromise the reliability of the system. Nonetheless, due to its low cost compared to Flash LiDARs and its higher resolution compared to radars, Scanning LiDARs are the main sensors currently in use for driverless cars [START_REF] Buehler | [END_REF]]. In opposition to the latter case, 3D Flash LiDAR sensors are composed by a matrix of photodetectors, a FPA, and so are capable of recording full 3D images with a single LASER pulse, without the need of a scanning mechanism. The consequence is ultimately higher frame rates; the lack of need for motion-compensation algorithms within a frame; less weight and independence from moving mechanical parts. These features are all desirable for space applications. Figure 1.5 shows the principle of operation of a Flash LiDAR system and its main constituents: the pulsed LASER and the 3D camera itself, linked by a trigger to account for the time span between light emission and detection. Passive optical elements, as lenses, and electronics were left out of the schematics for simplicity.

The amount of LASER power needed per pulse is usually higher in comparison to a scanning LiDAR, due to the need to illuminate the entire scene at once. One approach to reduce power requirement is to use Avalanche PhotoDiode (APD), a type of photodetector capable of providing gain through a process called avalanche multiplication. By increasing an applied reverse bias voltage to a semiconductor diode, the gain, or multiplication factor, is also increased until the so-called breakdown voltage is reached.

A Single-Photon Avalanche Diode (SPAD) is a type of detector that operates well above the breakdown voltage and so can have a very large avalanche gain for any photon received [START_REF] Cova | Avalanche photodiodes and quenching circuits for single-photon detection[END_REF]]. This kind of operation is also called Geiger-mode in the literature. Since Geigermode APDs do not need a lot of energy to obtain a response, it is possible to keep energy per pulse low and the repetition rate high. However, the dead time limits the acquisition rate. After the SPAD undergoes an avalanche, the detector is blocked and is unable to detect any received photons until after the dead time. Another disadvantage is that, in order to acquire gray-scale (intensity images), multiple pulses are necessary, increasing the amount of energy spent [McManamon 2017]. Due to this requirement, SPADs are not so efficient if we are also interested in 2D intensity images. Geiger-mode Flash LiDARs are commercially available in a 32 x 32 pixels format (Figure 1.6). As opposed to Geiger-mode, Linear-mode APDs are designed to operate below the breakdown voltage and so have an adjustable gain depending on the applied reverse bias voltage. The energy required to image an area is typically higher for Linear-mode receivers than for Geiger-mode, because the photocurrent generated by the photoelectric effect must be distinguishable from the electronic noise level of the detector. Due to this power requirement, LASERs used for Linear-mode will generally have lower repetition rate than for Geiger-mode. Apart from that, this type of camera is capable of providing gray-scale measurements from a single pulse, by accumulating photons from the reflected light pulse. There is also no dead time after each detection. Development in the direction of increasing the sensitivity of linear-mode APDs are underway and so the main advantage of Geiger-mode is becoming less relevant.

Geiger-mode APDs are usually made of Indium Gallium Arsenide (InGaAs) as the semiconductor material for long distance applications. Compared to photodiodes made of Germanium, for instance, InGaAs APDs have faster time response, higher quantum efficiency and lower dark current [START_REF] Wilkerson | [END_REF]]. InGaAs technology can also be used for Linear-mode APDs and are typically operated at gains up to 20, limited by excess noise and breakdown issues. Linear-mode APDs based on InGaAs were made commercially available by Advanced Scientific Concepts Inc (ASC). The company sells a 128 x 128 pixels LiDAR camera that is at the final test stages in the context of the National Aeronautics and Space Administration (NASA)'s Autonomous Landing and Hazard Avoidance (ALHAT) project (Figure 1.7). Another promising semiconductor material for APDs is MCT. The material is commonly used for Fourrier-transform infrared spectroscopy, infrared astronomy and night vision for military purposes. MCT APDs have been show to exhibit high linear gains up to 1000 or more with extremely low excess noise factor and low dark current [McManamon 2017]. These parameters are explained in more details in section 2.1. Recent research has shown Linear-mode MCT APDs with single photon sensitivity [START_REF] Sun | HgCdTe e-APD detector arrays with single photon sensitivity for space lidar applications[END_REF]], thus matching the main advantage of Geiger-mode detectors whithout having a dead time issue. The two main disadvantages of MCT-based detectors is that they need to be cooled to temperatures near that of liquid nitrogen for standard MWIR or Long-Wavelength Infra-Red (LWIR) detectors (77 K), in order to reduce noise due to thermally excited current carriers; and cost. Short-Wavelength Infra-Red (SWIR) MCT cameras can be thermoelectric cooled (at higher temperature than the previous case) [ Rothman 2014]. Although Linear-mode MCT cameras have been demonstrated by DRS, Raytheon and others, they are not available as a commercial product, only as custom devices with a specified development. Despite those disadvantages, MCT-based LiDAR systems present themselves as a competitive alternative for applications with great properties in terms of gain, noise and reliability.

CEA has developed a prototype of an MCT-based Linear-mode Flash LiDAR detector with a 320 x 256 pixels FPA operating at 80K [ de Borniol 2010]. Due to its exceptional characteristics, this prototype serves as a proof-of-concept of the applicability of this technology to space scenarios on this study.

3D Cameras in Space

The fast and accurate determination of the spacecraft's position and orientation is a crucial requirement for its navigation on numerous space missions. Non-exhaustive examples of applications could include:

• Controlled landing of spacecrafts on extraterrestrial bodies;

• Topographic modelling of asteroids;

• Rendezvous and docking of spacecrafts;

• Navigation of rovers;

• Formation flying;

• Space debris removal. 3D cameras have been described as the main sensor for these missions [START_REF] Do Carmo | Imaging lidars for space applications[END_REF], since they directly provide depth measurement.

Scanning LiDARs have been used historically for space applications, especially for Terrain Relative Navigation (TRN) and rendezvous between spacecrafts. Its calibration is relatively easy because of the few amount of detectors [START_REF] Christian | [END_REF]] and it is possible to use a narrow LASER beam with reduced power compared to Flash LiDARs. Despite that, as discussed in section 1.1, its moving parts compromises the reliability of the system while limiting the potential for miniaturization. These limitations raised the motivation for the development of Flash LiDARs dedicated for space exploration missions.

Space applications have serious constraints of size, mass and power consumption of devices and, as a consequence, Flash LiDARs have to be specifically designed to meet this requirements [START_REF] Do Carmo | Imaging lidars for space applications[END_REF]. Another obstacle is the amount of flux of photons reaching the detector which could be very low notably at operations at long distances, e.g. the initial phases of descent and landing.

Present Developments on 3D Cameras for Space Applications

NASA's development on Flash LiDAR for space exploration has been done in the context of the Autonomous Landing Hazard Avoidance Technology (ALHAT) project. The project was launched in 2011 with the objective of instrument new landers with the ability to automatically recognize their desired landing site and adjust the course of descent accordingly to potential landing hazards. The Flash LiDAR system based on the technology developed by Advanced Scientific Concepts (ASC) was chosen as the primary landing system sensor. The ASC Tiger Eye Camera used is an InGaAs-APD detector operating in linear mode hybridized to a Read-Out Integrated Circuit (ROIC) constituting the 128x128 pixels FPA. The sensor specifications are described in Table 1 The system was tested on the Morpheus Autonomous Rocket-Propelled Lander and was able to identify landing hazards as small as 30cm from a slant range of 450m. Those flight tests served as the Technology Readiness Level (TRL) 6 demonstration of the system.

In Europe, ESA conducted a study to evaluate the suitability of TOF cameras for space applications, gathering also Thales Alenia Space, Terma and SINTEF. A small part of the listed requirements for potential space missions summarized in this study are reproduced on Table 1.2. Please refer to [START_REF] Christy | [END_REF]] for full details. The study analyzed phase-based systems; gate-based systems; and Flash LiDARs and selected the second one. The study concludes that phase-based systems are unsuitable due to range ambiguity related problems, as described in section 1.1. Flash LiDARs were rejected with the justification of being an immature technology in Europe. Gated-based systems integrate the reflected pulse between the actuation of two gates. These gates can be controlled to adjust the range of depth measurement. Distance d is derived from the integrated light as follows:

Mission

d = c 2 (τ delay + s 2 s 1 + s 2 τ pulse ) (1.4)
where here c is the speed of light; τ delay is the shutter delay; τ pulse is the width of the pulse; and s 1 and s 2 are the number of electrons integrated over the two main gates. The chosen sensor for tests was developed by TriDiCam GmbH, but the company, unfortunately, is presently out of business. Since that, ESA's applications for LiDAR systems were focused on missions related to wind measurements; cloud and aerosol measurements and a LASER altimeter that will provide absolute topographic height and position with respect to Mercury [ESA ].

Imaging 3D LiDAR for Landing Applications

So far, all space missions chose completely safe landing areas, considering the available knowledge. This is an important requirement due to the high budget necessary for space exploration and the safety of the crew in case of human presence in the ship. On the other hand, it reduces the available area for exploration to a few hundred meters around the chosen landing spot and increases time and fuel necessary for rovers to reach more scientifically interesting areas [Brady 2010].

One such example is the lunar poles, which are now thought to be more enriched in water and other volatiles than the equatorial regions sampled during the Apollo missions. The presence of water in permanent shaded regions near the poles could facilitate a semipermanent settlement of humans on the moon for detailed scientific study of the soil or even as an intermediate station for planetary launches. Due to the very small axial tilt of the Moon, the South Pole has peaks of near-permanent lighting that could provide a reliable and almost constant source of energy. These two very attractive features of the poles are linked to craters (volatiles) and mountains (lighting) and so a future lander has to be able to descent in a nearby area, possibly with rough terrain [Lemelin 2014].

CONTENT OF THE WORK

Flash LiDARs are being considered as the key sensor for landing on extraterrestrial bodies due to advantages already specified in section 1.1: capability of acquiring 3D images at any lightning conditions; reliability by virtue of the independence of moving mechanical parts; and high gain possibility to deal with low income flux of photons at long distances. The sensor may also be employed on different functions such as: Altimetry, TRN, Hazard Detection and Avoidance (HDA) and Hazard Relative Navigation (HRN). Table 1 The Flash LiDAR prototype developed by CEA-LETI with its large FPA and high gain is a good candidate to assess the MCT technology for landing applications. Aiming at the demanding requirements of precision and resolution, the goal of this study is to optimize the assembly of the system into a compact prototype and to improve the overall quality of the 3D measurements by using image processing techniques.

Content of the work

Chapter 2 describes in details the MCT LiDAR system used in this study. The characteristics of the main components are discussed as well as the choices that led to the definition of the final system being tested. Chapter 3 presents the set of processing algorithms used in order to improve the overall quality of the images acquired by the system. The algorithms are grouped as a processing workflow. The miniaturization and optimization of the system described in chapter 2 and the development and study of algorithms in chapter 3 are combined in chapter 4, in which we present the final experimental test campaign performed at ESA and its main results. Finally, chapter 5 concludes this study with the main discussions of the application of this technology and prospects for future works.

Chapter 2

Flash LiDAR System Implementation

In 2009, the Commissariat à l'énergie atomique et aux énergies alternatives (CEA) developed a 320x256 hybrid Focal Plane Array (FPA) for Flash Light Detection and Ranging (LiDAR). This detector is the core and starting point from which the LiDAR system developed in this work was built. The other elements of the system were designed or chosen aiming the miniaturization and overall optimization of the system.

The block diagram of the LiDAR system with its main subparts is depicted in figure 2.1. A brief description of the elements follows: • Image Sensor and Read-Out Integrated Circuit (ROIC): As described in the previous chapter, the sensor was developed along with its respective hybridized ROIC. Section 2.1 provides a complete description of the detector subsystem as well as the pixel architecture.

• Pulsed LASER: the choice of the light source is of fundamental importance to meet application requirements. Critical parameters will be further detailed in section 2.2.

• LASER driver: driver LASER electronics also have to be considered in the choice of the pulsed LASER. Power consumption, weight and possible delays are the main parameters of interest.

• Control electronics: This sub-part controls trigger signals to command the LASER and the voltage ramp necessary for the 3D operation of the detector. It also controls the operation of the detector itself, including setting voltage bias of the photodetectors, clocks and finally image/video acquisition. Section 2.3 provides more details of its operation.

• Optics: an overview of the lenses used for detection and for expansion of the LASER beam will be described in section 2.4. Hot and cold filters used to mitigate the influence of some sources of noise are also described.

• Power Source: provided by the spacecraft in the space scenario, the power consumption should be as low as possible.

This chapter provides a comprehensive description of each constituent part of the LiDAR system; the constraints imposed by the test campaign; and the hardware optimization process that lead to the final device.

Image Sensor and ROIC

As mentioned in chapter 1, this study was conducted with a Flash LiDAR imaging FPA. The detector used in this project came from a 2009 batch and no major modifications were made. As the main component for the LiDAR system, this section describes its main characteristics and limitations.

The integrated ROIC consists in a 320 x 256 pixels array with 30µm pitch. Each pixel is capable of recording the 2D (intensity) and 3D (time-of-flight) information of the scene for every LASER pulse. A detailed description of the pixel operation and the ROIC design is provided in section 2.1.2.

As described in section 1.1, the chosen semiconductor material for the avalanche photodiodes was Mercury Cadmium Telluride (MCT), also commonly refered by its chemical formula HgCdTe. MCT Avalanche PhotoDiode (APD) detectors operate on linear mode and have a high quantum efficiency, i.e., a large percentage of input photons contributes to current generation. This type of APD has an electron-to-hole ionization coefficient close to zero, which means that almost all carriers generated during an avalanche are electrons. The consequence is that the excess noise generated due to the statistical nature of the avalanche process is negligible, meaning that the excess noise factor is close to one. This allows very high linear gains, approaching single photon detection, without exceeding the avalanche breakdown voltage. In order to understand the importance of these parameters on the system sensitivity, section 2.1.1 describes the Signal-to-Noise Ratio (SNR) calculation for avalanche photodetectors.

SNR calculation for MCT-APD detectors

SNR is defined as the ratio between the power of a signal and the power of noise. In the case of photodetectors, the signal is the current generated by the incoming flux of photons. It is possible to reduce the signal power by reducing the influence of the sources of noise while maintaining a target SNR necessary for the adequate operation of a system. The main sources of noise during the process of photodetection are the following:

• Shot Noise: It comes from the statistical nature of the number of photons detected during a certain period of time. Shot noise is especially important if the number of photons detected is low, e.g. at a long-distance measurement, and it could be statistically described as a Poisson distribution [START_REF] Horowitz | [END_REF]]. For large numbers of photons, the Poisson distribution approaches a normal distribution.

• Background Illumination: The shot noise described in the item above comes from the arrival of the photons belonging to the actual light pulse, so it is, in this sense, inevitable. However, the detector will receive photons from other unwanted sources, like the reflection of the sun light on the target's surface. The light coming from other sources than the LASER could produce erroneous Time-of-Flight (TOF) measurements. These photons will also contribute to shot noise and this is the origin of the background illumination noise. We can mitigate this problem by using band-pass filters centred on the LASER wavelength, but the unwanted photons at this wavelength will continue to be a source of noise.

• Dark Noise: Even in the absence of light, random generation of electrons and holes pairs occurs in photosensitive devices. The small current generated in this process, called "dark current", will also produce a shot noise and degrade the SNR.

• ROIC Noise: Finally, the read-out circuit will also contribute as a portion of the total noise. Thermal noise, or Johnson-Nyquist noise, is one of its main constituents and is provoked by electron agitation related to the temperature. Other noise sources could also influence, such as coupled noise. In opposition to the previous cases, ROIC noise is not influenced by avalanche gain since it is not generated at the photodiode level.

Taking into account the signal and noise sources mentioned above, the resulting signalto-noise ratio is given by:

SN R = i 2 s σ 2 s + σ 2 bi + σ 2 d + σ 2 ROIC (2.1)
where i s is the current generated by the incoming flux of photons; σ 2 s , σ 2 bi , σ 2 d are the shot noise variance due to the signal, background illumination and dark current respectively; and σ 2 ROIC is the ROIC noise variance. Now taking in consideration the avalanche gain process, this expression can be described as:

SN R = (Gi s ) 2 2qBG 2 F (i s + i bi + i d ) + σ 2 ROIC = i 2 s 2qBF (i s + i bi + i d ) + σ 2 ROIC G 2 (2.2)
where G is the avalanche gain; q is the charge of the electron; B is the electrical bandwidth of the receiver and F is the excess noise factor. If the sensor has an integrating input stage, like a Capacitive Transimpedance Amplifier (CTIA), B = 1 /(2 × Tint), where T int is the integration time. The last part of equation 2.2 shows explicitly the importance of avalanche gain: if it is high enough, the ROIC becomes a negligible source of noise. It also shows the importance of a low excess noise factor F for the system's sensitivity. This equation demonstrates some of the most advantageous features of MCT technology over Indium Gallium Arsenide (InGaAs): it has higher avalanche gain, lower excess noise factor and less dark current [McManamon 2017].

ROIC Design and pixel operation

The MCT APDs designed at CEA are manufactured using liquid phase epitaxial grown epi-layer [De Borniol 2014] with constant cadmium composition adjusted to provide a cutoff wavelength of 4.6µm at 80K [START_REF] Borniol | [END_REF]. The array of MCT APDs, each with 30µm pitch, is hybridized with a ROIC fabricated on a standard 0.18µm Complementary Metal-Oxide-Semiconductor (CMOS) process as depicted in Figure 2.2.

The APD bias is controlled through the circuit and will typically be adjusted to provide a gain between 20 and 100. Table 2.1 sums up some of the ROIC main specifications. A simplified schematic of the pixel's architecture is depicted on Figure 2.3 [de Borniol 2010]. The operation is based on a CTIA followed by a comparator. Two feedback capacitors are connected to the CTIA: one with a small capacitance value (C 3D ) used for fast pulse detection; and a bigger one (C 2D ) used for the purpose of flux integration. C 2D offers 3.6 × 10 6 electrons of storage capacity. C 3D is responsible for quickly drive node N 1 to a voltage level higher than the one stored by the comparator (V th ) at the arrival of the first photons of the LASER pulse. The track and hold stage then samples an analog voltage ramp which serves as a time base. Those samples, along with the knowledge of the slope of the voltage ramp, can be translated as a time-of-flight (3D) measurement. At this moment, the switch connects C 2D to the Operational Transconductance Amplifier (OTA) in order to allow the integration of the LASER whole pulse. The final node voltage at N 1 then corresponds to the intensity (2D). Integration time can be adjusted by the system operator via control electronics. This acquisition phase is followed by a serial readout phase that produces two frames of data, representing intensity (2D) and TOF (3D) measurements.

The voltage ramp is externally generated and transmitted to the ROIC. At first, an arbitrary waveform generator by Tektronix served as a ramp generator and it was latter replaced by a dedicated board designed at CEA for this project, described in section 2.3.1. The target requirements demands a very stable high quality ramp, notably in terms of jitter and amplitude noise. The minimum voltage ramp duration is 200 ns by design, which is equivalent to observe a depth range of 30 m. At this arrangement and without considering calibration and data treatment processes, the observed range noise (1-σ) is 30 cm. A longer depth range can be accomplished with a lower ramp slope, leading to a possible loss in resolution. The ROIC is reasonably flexible in terms of control: the system can, for example, work as a passive imager by disabling the TOF evaluation blocks and increasing the integration time.

Finally, every pixel column has an amplifier and the entire array is read in series (2D and 3D data) to form a single analog voltage output. Please refer to appendix A for more details on the operation of the pixel.

Pulsed LASER

The other key component of a Flash LiDAR system is the light source, in this case, a LASER. Its parameters are directly linked to the capabilities of the system in terms of maximum range, resolution and accuracy. Notably, the optical power delivered by the LASER will influence the number of photons arriving at the detector and thus the SNR. Figure 2.4 presents some of these parameters in a simplified example to understand their relation with the number of arriving photons. The LASER illuminates an area A illum bigger than the target area A t in order to mitigate the impact caused by the non-ideal beam profile. We can then assume that A t is uniformly illuminated and that the power P SC backscattered by the target is spatially constant. P SC is proportional to the power transmitted by the LASER P T , attenuated by the travelled distance R and the atmospheric absorption represented by the coefficient η atm . Finally, P R is the power received by the detector of area A REC and related to P SC in the same manner subject to the system's efficiency η sys . It is possible to show that the relationship between power received P R and power transmitted P T is two ratio of areas times some efficiency parameters [McManamon 2012]. Equation 2.3 describes this relationship, where ρ T is the reflectivity of the target:

P R = P T A T A illum A REC πR 2 ρ T η 2 atm η sys (2.3)
To calculate the number of expected photons for every pixel, we manipulate equation 2.3 to account for the energy received during one pulse detection, as follows:

N pixel = 1 S pixels λ hcE T A T A illum A REC (πR 2 ) ρ T η 2 atm η sys (2.4)
where N pixel is the number of arriving photons per pixel; S pixels is the number of pixels; λ is the LASER's wavelength; h is Planck's constant; c is the speed of light and E T is the energy per pulse. Equation 2.4 demonstrates the expected relationship that the number of arriving photons decays with the square of the distance to the target. This is the main reason why the gain provided by avalanche photodiodes is so relevant at long distances. The wavelength choice also impacts the amount of photons. If we make assumptions for these parameters as described in Table 2.2, the expected amount of arriving photons would be 260 at a distance of 500 meters. This result indicates the need for avalanche photodiodes. Although not explicitly described in equation 2.4, the pulse width is relevant to the detection process due to the threshold described in section 2.1.2. For the same amount of LASER power, shorter pulses will have more energy and therefore the threshold will be reached sooner. The wavelength on which the LASER operates is also of great interest for two reasons: first, since the sun can be perceived as a blackbody [Iqbal 2012], its spectral radiance is lower at higher wavelengths. Figure 2.5 depicts the solar irradiance with respect to the wavelength [Thuillier 2003]. The second reason is that a small Short-Wavelength Infra-Red (SWIR) band is considered to be eye-safe for humans and it is therefore more secure for prototype tests or even at space mission where humans could be present.

Finally, LASER jitter is another parameter that will unequivocally influence the range measurement precision. Jitter is generally described as random deviation from the true period of a signal. In LiDAR systems it means an imprecision on the true moment of pulse emission and this will reflect on an imprecision on the TOF measured. 6. An Optical Parametric Oscillator (OPO) converts the pulse wavelength from 1064nm into 1570nm, in the eye-safe region of the spectrum. The pulse width is 8ns, its maximum energy is 8mJ and the reported jitter at emission is 1ns. Despite these parameters of this LASER being satisfactory for the purpose of this study, it has the disadvantage of being water-cooled. This property makes it too heavy to be mounted on the robotic arm used in the final test campaign. This weight constraint is better detailed in section 2.6. A specification list for the purchase of a LASER for future experiments was prepared (Table 2.3). The wavelength should stay within the eye-safe region, while the pulse energy should be as high as possible for a pulse duration as low as possible. The ROIC limits the rate of our system to 20Hz, so the LASER should fire at least at this repetition rate. The triggering of the LASER must be external, so to be electronically controlled and so that the system have a precise knowledge of the instant of pulse emission. Ideally, the pulse format would be top-hat, which means that the spatial distribution of energy would be uniform. The weight should be as low as possible, indicating the need for thermoelectric cooling. Finally, as previously stated, jitter should be as low as possible for a precise range measurement. A comparison between different LASER suppliers was prepared based on these specifications and it is depicted in Table 2

Supplier

Control Electronics

As described in Figure 2.1, the ROIC is connected to a control electronics that provides power, bias, and clocks necessary for the sensor operation. It is also responsible for acquiring data and transmit it to a computer for visualization and storage.

The signals are provided by a control rack, as shown in Figure 2.8, connected to a computer (not shown in the image) in which the operational instructions are programmed by the user. The voltage ramp is provided by an arbitrary waveform generator, triggered by the control rack.

Following the developments towards a miniaturize system, the entire control rack depicted in Figure 2.8 was replaced by a system composed by a Field-Programmable Gate Array (FPGA), a generic Mother Card and a specific Daughter Card (Figure 2.9). The FPGA used is commercially available and the generic motherboard was designed by CEA to provide all necessary clocks and bias for the operation of infrared ROICs.

Voltage Ramp Generation

Similarly to the LASER, jitter is a vital point of concern for the ramp generation. A random error on the precise moment the ramp starts will lead to inconsistent measurements between pulses for the same distance and, in the case of a LiDAR system, the tolerance is very low. Just 1 nanosecond of jitter is equivalent to 15 centimeters of imprecision. Initially, a Tektronix's arbitrary waveform generator was set up to provide a precise voltage ramp (section 2.1.2), triggered by the Daughter Card. It was verified by the aid of an oscilloscope that the jitter was less than 1 nanosecond as required. Although being able to generate a precise ramp that meets the requirements of the tests, the size and weight of the wave form generator makes its use unfeasible for a compact system. In the context of this work, we have developed a electronic board to replace the wave form generator.

The schematic diagram of the "ramp generator" board is presented in Figure 2.10. It operates through a current source that charges a polystyrene capacitor. Before the start of the ramp, the analog switch U 3 bypasses the capacitor C1 and derives the current of the generator.

The current generator consists of R2, Q3 and U 2. The amplifier U 2 drives the transistor Q3 so that the voltage across R2 is equal to U J3 -U J6 . Thus, the current I = (UJ3 -UJ6) /R2. The slope of the voltage ramp is adjustable, allowing the modification of the range of distance measurements. The "coarse" adjustment of the ramp actually changes the value of C1 (the switch J4 provides two options: 220pF or 2.2nF ). The "fine" adjustment is made adpting the value of R2 // R19. The ramp voltage is provided by the follower amplifier U 5, followed by a resistance of 50Ω.

The board can be powered by a 230 / 28V power supply, a filter and two regulators + and -5V. It is also possible to power the board upstream of the regulators by a + and -12V supply (J9, J10 and J19) for example or directly in + and -5V downstream (J12).

Jitter of the ramp generator board was evaluated using an oscilloscope on persistence time mode, where several periods are overlapped in a single image. A descendant voltage ramp generated by the card is depicted in Figure 2.12. The peak-to-peak jitter value can be approximately inferred from the image as being slightly higher than 2ns. For a zero-mean sine wave, the relationship between the root mean square (RMS) value and peak-to-peak amplitude is:

Jitter RM S = Jitterpeak-to-peak /2 √ 2.
Thus, the RMS jitter value of the designed voltage ramp is estimated to be less than 1ns, as desired.

Optics

An optical system consisted of multiple lenses is placed right after the LASER, adapting the Field-of-Illumination (FOI) as specified for the application, e.g., at different stages of a spacecraft descent. This optical system was also design to guarantee the safety of the staff during the experimental campaigns performed both at CEA and the European Space Agency (ESA). The calculation of the safe requirements is demonstrated in Appendix B.1.

Similarly to the case of LASER optics, lenses at reception could be conformed to the tests needs. Lenses are important at detection level to effectively increase the amount of photons reaching the sensor. Objectives composed of multiple lenses that could adjust the focus during descent could be evaluated if necessary, as well as telescope lenses for long-distance measurements. In this study a single 100-mm-focus-length biconvex lens manufactured by ThorLabs was used on the detector side.

The design of the Dewar imposes that the detector faces down. A large right-angle mirror is placed below the detector so that the Field-of-View (FOV) of the detector matches the FOI of the LASER. The 75mm silver coated mirror is manufactured by Edmond Optics and it has a reported reflectivity at 1570nm of 98%.

Thermal and parasitic radiations on the sensor are limited by a cold low-pass filter and a hot band-pass filter. The band-pass filter is centred on the LASER wavelength and therefore also mitigates the effect of background illumination. Figure 2.13 depicts the transmittance of the hot band-pass filter centred at 1570nm. Note that all wavelengths lower than 1550nm and higher than 1600nm are basically eliminated.

Cryogenic Cooler

As previously discussed, one of the main drawbacks of MCT FPAs is the need of cooling at 80K. Alternatives could be implemented to make it function at higher temperatures, for example by limiting the integration time. This could also be achieved by fabricating a FPA with a shorter cut-off wavelength. Nevertheless, this study will focus on the present device that needs to operate at very low temperatures.

In laboratory tests and field trials this temperature is reached using liquid nitrogen that needs to be stored in a cryogenic cooler, or a dewar. Dewars are vacuum-insulated containers for cryogens. In this work we use a versatile generic dewar, depicted in Figure 2.14, that can be easily reconfigured for different detectors and applications. As a final product to space applications, a customized version considering size, weight and cost should be designed, but this will not be evaluated in this current work. In space applications, a different cooling system would be envisaged in which the detector would be connected to a radiator facing the outside of the ship.

A special lid was designed to prevent spillage of liquid nitrogen during the test campaign while the system is moving. The lid is made of Polytetrafluoroethylene (PTFE), a synthetic polymer that maintains high strength, toughness and self-lubrication at low temperatures down to 5K [START_REF] Teflon | Properties handbook. DuPont Fluoroproducts[END_REF]]. Spectral filters, as the ones described in section 2.4, as well as FOV defining apertures are components of the dewar -the cold filter is also cooled by the liquid nitrogen. The income flux of photons reaches the FPA through a window mounted at the dewar, which could also be used as a band-pass filter.

The recommended safety measures when dealing with liquid nitrogen are described in Appendix B.2.

Final System Assembly

Detector Selection

The first experimental tests demonstrated higher noise levels than expected. This fact motivated the hybridization and characterization of three other detectors. Among them, one exhibited superior performance in terms of stability and noise level.

An experimental comparison under the same conditions is made in order to evaluate the difference on the noise level between detectors. First we low the threshold V th , described in section 2.1.2, so that there is an immediate detection on the noise and the voltage ramp is instantly sampled. This procedure permits the evaluation of the electrical response of the detector free from the influence of the photodiodes. An acquisition of 400 images is made for each detector, allowing the evaluation of the temporal standard deviation for each pixel, which is related to the electrical temporal noise.

Figure 2.15 shows the resulting standard deviation for each detector, measured in mV . Detector 1 presents a structural pattern noise that it is not present on the Detector 2. More than that, the mean standard deviation on the first image is 5mV , while for the new detector we measured 1.5mV , i.e., more than three times lower. This important difference on the noise level permits a lower threshold level V th , thus improving the linear response of the system, as detailed in section 3.2.3.

Detector 2 was therefore selected for the final system assembly and was used for the development of the set of algorithms discussed in chapter 3. Unfortunately, this detector eventually stopped working due to a malfunctioning connection on the ROIC and so Detector 1 was used during the final experimental campaign. During the Dynamic Tests Phase, explained in details in chapter 4, the LiDAR system was mounted onto a KUKA robotic manipulator. The robotic arm was programmed to move towards different targets in order to simulate the descent of a spacecrafts and evaluate the performance of the system. The assembly of the LiDAR system, i.e., all of its constituent parts should obey some weight and dimensions constraints for the safe operation of the robotic arm. Figure 2.16 depicts the loading curve corresponding to the maximum load capacity. The system's center of mass should be inside the envelope related to its weight. The overall weight should not be heavier than 10 kg.

LiDAR System Model

In order to better estimate the weight and center of gravity of the system, every component was modeled in Solid Works (2018 version). Besides the main components, this model also contains the power supply of the control electronics, the mirror and its support at detector level, the LASER's radiator and the table and structures that hold the system together. The 3D model of the cryogenic cooler, LASER, radiator, ventilator and mirror were all given by their respective manufactures and its measures were independently verified. The control electronics, the table and the power supply did not have a previous model and were drawn in Solid Works after being measured and weighted. Liquid nitrogen was also considered in the model, as if the dewar was completely full. The final model is depicted in Figure 2.17. Based on each component's weight and their models it was possible to estimate the center of gravity of the entire system (represented in pink in Figure 2.17). The total weight was estimated to be 8.05 kg. and the center of mass is located at X = -8.84, Y = 2.08 and Z = 90.08 (millimeters) with respect to the flange of the robotic arm. According to this model and taking the Payload Diagram (Figure 2.16) in consideration, the system is respecting the constraints for safe operation of the robotic arm with a safety margin of approximately 40 mm.

System Implementation and Model Comparison

The system was implemented as described by the model and is depicted in Figure 2.18. The cables connecting the control electronics are the only components not specified in the system model. The final system weights 8.85kg, i.e., 800g more than predicted by the model, most of it related to the cables.

The table was custom-made to save as much material as possible and thus not significantly influence the weight of the system. The material of choice was the Polymethyl methacrylate (PMMA), also known as acrylic or plexiglass. PMMA is a strong, tough, and lightweight material. It is thus rigid enough to maintain the alignment between the LASER and the detector while keeping the overall system light.

The LASER was not equipped with a cooling system, and so it had to be bought separately. The system, composed of radiator and ventilator, was designed so that the cooling As briefly described in section 2.2, the LASER has a control box too heavy to be put together with the rest of the system. The solution found was to design an aluminium cage to hold the control box attached to the "shoulder" of the robotic manipulator. The control box thus follow the system without producing additional moment of force. Figure 2.19 shows the final system mounted on the robotic arm and the control box attached separetely. The estimated center of gravity of the LiDAR system is highlighted in red in Figure 2.16. Since most of the exceeding weight comes from the cables, which are close to the center, this difference will not significantly move the center of gravity beyond the 40 mm of margin to the limit. In conclusion, the constraints of weight and center of gravity were met by the LiDAR system and the robotic arm can be operated safely.

Software Elements

The operation of the system is performed on a computer via a LabView (2015 SP1) interface. The connection to the control electronics (described in section 2.3) is made via an ethernet cable linked to the FPGA. Figure 2.20 shows Labview's subVIs responsible for sending power and clock signals and operate image acquisition. The data processing is performed offline using Matlab (R2016b). Chapter 3 provides a comprehensive description of the algorithms used to improve the performance of the system.

Chapter 3

Data Processing for Flash LiDAR Systems

The Read-Out Integrated Circuit (ROIC), whose architecture is described in details in section 2.1.2, provides two outputs for every pulse emitted by the LASER: a 2D image, related to the total amount of photons detected; and a 3D image, the measurement of the Time-of-Flight (TOF). The left side of Figure 3.1 shows an example of a scene with some card boxes, on which the red square delimits the area imaged by the LiDAR system. The right side depicts the equivalent 2D and 3D output images. The TOF measurement has a direct use for the target application since it hands over the information on which the navigation system relies to create a Digital Elevation Map (DEM) of the landing site. However, several imperfections on the system compromise image quality. In Figure 3.1, for example, the black foam present on the background is not visible on the 3D frame and the boarders of the image are corrupted.

CHAPTER 3. DATA PROCESSING FOR FLASH LIDAR SYSTEMS

LiDAR intensity data is used in numerous applications such as segmentation and object detection and recognition [START_REF] Kashani | [END_REF]]. 2D images can also provide an indirect range measurement, since the amount of photons detected decreases quadratically with the distance. Intensity images present a superior richness of details derived from the more elevated Signalto-Noise Ratio (SNR), as it can be noted in Figure 3.1.

The combination of 2D and 3D information could then be used for the implementation of a variety of image processing techniques in order to improve the quality of the range measurement and meet the resolution / precision requirements of space applications. Some of the methods presented in this chapter demonstrate the advantages of this interaction.

Section 3.1 describes the major defects on image formation linked to electronic issues. These problems can be identified in laboratory conditions and are not connected to specific features of the scene. The imperfections of the system as a whole, on the other hand, are usually scene-dependent and are discussed on section 3.2. Finally, section 3.3 details the techniques employed to overcome those limitations.

The final result of this process is an improved image / video that could be used as an input for Hazard Detection and Avoidance (HDA) and Safe Site Selection (SSS) algorithms that interact with the Guidance Navigation and Control (GNC) unit to adapt the course of during landing. These high-level algorithms are, however, outside of the scope of this study and an integration with them remains as a future work.

Electronic Faults

Defective Pixels

Pixels that do not work or whose response vary greatly from the mean are defined as bad or defective. Defective pixels are a common problem in infrared Focal Plane Array (FPA) related to the difficult manufacturing process for microbolometer arrays. This is the case for Mercury Cadmium Telluride (MCT) and Indium Gallium Arsenide (InGaAs) detectors employed in Flash LiDAR systems. On the other hand, cameras acting on the visible spectrum are constructed from silicon and are much less prominent to such defects. A non exhaustive list of common pixel defects include:

• Dead pixels do not respond to light at all and do not provide any information. They are usually black (minimum value) or white (maximum value), but could also have any intermediate value.

• Hot pixels respond to light normally but suffer from excessive dark current and can easily become saturated.

• RTS pixels respond to light and can provide valid information, but they flip up and down between different discrete states 1 .

• Blinking pixels can be either dead blinking if they jump randomly between two dead states or blinking operating if they jump between the correct value and a dead state.

Bad pixels can be present in infrared FPAs not only individually but also in clusters. A cluster of bad pixels is a group of at least two defective pixels that are adjacent to each other. As mentioned in chapter 1, the detector used in this study is a prototype manufactured ten years ago and thus presents an elevated number of bad pixels, including several clusters. Figure 3.2 shows an example of two 2D images registered while the shutter was closed. There was no incoming flux of photons and therefore the image was expected to be uniform. No gain was applied on the left one, while a mean gain of 35 was employed on the other. These images were formed by taking the median value of 50 frames in order to reduce the impact of non-stationary temporal noise. The identification of pure dead pixel is the simplest between the categories described, because their value presents a great discrepancy to the mean and are invariant with time or gain applied. Black pixels on Figure 3.2 form clusters on the bottom right side and upper left that are compatible with possible scratches on the FPA. These pixels present the same defect on both images, indicating that they are dead pixels. The presence of a much larger number of black pixels on the right image, on the other hand, reveals the presence of hot pixels.

Blinking pixels are much harder to identify because the blinking frequency can be very low. Dead pixels are unresponsive in both 2D and 3D images, but hot and blinking pixels tend CHAPTER 3. DATA PROCESSING FOR FLASH LIDAR SYSTEMS to be more present on intensity images. Since 2D images are directly linked to the integrated flux of photons, the pixels on this image are more likely to become saturated. The method for identification and correction of defective pixels is presented in section 3.3.1.

Nonuniformity of the FPA

The FPA is a matrix of individual avalanche photodetectors, each one acting as an active pixel of the final composed image. While we should expect that the pixels in a camera all behave equally, in practice each photodetector has a different response to the incoming flux of photons in terms of gain and offset [Milton 1985]. This so-called Fixed Pattern Noise (FPN) can seriously reduce the performance of infrared detectors and methods to overcome this problem have been proposed since the late 1970's.

Those deviations come from an imperfect fabrication process and several factors could influence this response discrepancy, such as the temperature in which the system is operated and the age of the device. ROICs may also be a source of nonuniformity response by introducing artifacts such as stripping [Schowengerdt 2006]. In section 3.1.1 a shutter was used to forbid the detection of photons and the resulting images were expected to be uniform. Instead, we notice on the left image of Figure 3.2 a column-wise nonuniformity when no gain is applied on the Avalanche PhotoDiode (APD). As described in section 2.1.2, the ROIC has an amplifier for each column of pixels. Different values of gain and offset in these amplifiers would explain this type of nonuniformity.

The application of gain on the APDs results in even stronger effect, as seen on the right image of Figure 3.2. A difference on the thickness of the material used on the hybridization process causes a spatial variation of the responsivity of the detectors. Because of that, different regions of the FPA respond differently to the application of the reverse bias voltage that generates the avalanche gain effect and the result is a nonuniform image. The impact of radiation on the nonuniformity is assessed using a blackbody. Blackbodies are excellent instruments for laboratory tests and calibration of infrared photodetectors. An ideal blackbody absorbs all electromagnetic radiation received, regardless of wavelength and angle of incidence. When in thermal equilibrium, a blackbody emits radiation according to Planck's Law: the spectrum of the radiation depends only on the temperature. The blackbody as a source of illumination to the detector provides a uniform flux of photons and therefore all pixels should present the same value. Figure 3.4 depicts the test bench used to evaluate the nonuniformity of the detector.

The system is operated in passive mode, since no LASER is present. The integration time is set to 400ns to detect a greater number of photons. The APD gain was set to 25. Figure 3.5 depicts images of the blackbody at two temperatures. The scales are set to present pixels that satisfy the "3σ condition". Although the zones of nonuniformity are basically the same on both images, at 45°C the standard deviation is much higher, indicating that the nonuniformity increases with the flux of photons. As in the previous section, these images represent the mean value of 50 frames. The method for compensation of the nonuniformity in FPAs is presented in section 3.3.2.

Low Frequency Electrical Noise

So far, the images used to present the electric faults were formed by averaging a sequence of frames. This technique was used to suppress temporal noise that varies independently from frame to frame. During the operation of the system, however, this method would imply in a blurring effect on the images if the scene is dynamic and / or on an effective frame rate loss. Therefore, the analysis and mitigation on each independent frame is important to improve image quality while preventing this consequences. Figure 3.6 shows four consecutive frames of a video sequence after the processes of NonUniformity Correction (NUC) and Bad Pixel Replacement (BPR), presented in sections 3.3.1 and 3.3.2 respectively. The shutter was closed for the acquisition of this sequence and therefore there is no incoming flux of photons. Although no correlation between frames of the sequence is discernible, we notice a clearly spatially-structured noise pattern. The source of this noise seems to be related to the control electronics (section 2.3), likely the Mother Card or the power supply. In the latter case, the noise was expected to be temporally correlated on a base frequency of 50Hz, the European grid frequency. However, since the frequency of frame acquisition is quite low (around 7Hz), the pattern between different images is poorly correlated over time.

The spatially periodic characteristic of the noise is explored on the method for the mitigation of this problem, described in section 3.3.3.

Scene-dependent Faults

APD Depolarization

The photodetectors on the FPA are reverse polarized with a voltage controlling the avalanche gain. The user of the system can set this tension and therefore adapt its value to the application envisioned. While every pixel represents a cathode, the common anode is connected on the borders of the FPA. This means that the closer to the center of the array, the further the pixel is from the anode, implying in a higher resistance due to the semiconductor material. When the Flash LiDAR system is imaging a flat area, a large portion of the FPA receive photons at the same time. The instant peak of photocurrent that is produced might be too elevated to sustain the applied reverse voltage. Pixels on the center of the FPA suffer more from this effect due to high resistance loss. Figure 3.7 depicts the impact on the image: pixels around the center of the FPA present a lower value than expected in comparison to the borders.

The response to the depolarization effect is different on the 2D and 3D images. This fact is explored on the compensation method described in section 3.3.6.

Non Uniform Illumination

As discussed in the introduction of this chapter, the intensity information on 2D frames can be useful for improving TOF measurements, as is described in sections 3.3.5 and 3.3.6. The mitigation of the effects that could deteriorate this information is thus of vital importance for the success of its contribution.

Scanning LiDAR systems customarily consider that the target is uniformly illuminated due to the relatively small area covered by a single LASER pulse. This is not the case for Flash LiDARs, in which case the entire target scene is illuminated at once. A common mitigation measure is to adjust the LASER's Field-of-Illumination (FOI) to cover a larger area than the camera's Field-of-View (FOV). However, this procedure leads to wasting a part of the LASER energy that could be meaningful for long distance measurements.

The output of most LASERs are modeled as Gaussian beams, which means that its transverse intensity profile is given by the Gaussian function [START_REF] Svelto | [END_REF]]. Nevertheless, this simplified expression does not account for other possible sources of imperfections in the pulse generation, at the optical components used to diverge the LASER beam or the misalignment between the LASER and the detector. The assembly of these effects could be visualized by imaging a blank wall as depicted in Figure 3.8. Since the parameters that produced such effects are unknown, an empirical method based on captured data is more suitable. The calibration method developed to mitigate these problems were first presented at the International Geoscience and Remote Sensing Symposium (IGARSS 2019) and are detailed in section 3.3.4.

Nonlinear TOF measurements

The architecture of the pixel and the operations involved at the moment of the detection are described in section 2.1.2. When the reflected pulse arrives, the current generated by the APD is integrated at the 3D capacitance C 3D until reaching a threshold level V th . At this moment, the analog voltage ramp that serves as a time base is sampled and this value is hold as a measurement of the TOF (3D). Figure 3.9 is a representation of the time base used to measure the time-of-flight. At time 0, the LASER is fired and the voltage ramp is triggered to start increasing its output voltage level. At time t 0 the reflected pulse reaches the detector (represented by the gray area in Figure 3.9), but the system takes until t th to reach the threshold level V th (orange area). In this prototype, V th can be set by the operator of the system and can be adjusted in order to reject background and dark noise, described in section 2.1.1. The consequence is a delay on the correct instant of sampling that will depend mainly on the threshold level V th , the gain provided by the APD and the reflected pulse energy.

Figure 3.9 shows that the 3D measurement is a linear function of the time needed to reach the threshold, t th . This indeed refers to the sum of the actual TOF of the pulse and the extra time needed to integrate the pulse until V th is reached, as shown in equation 3.1.

V 3D = a × (t 0 + t extra ) + b (3.1)
The actual time-of-flight (t 0 ) is simply a function of the distance to the target (D) according to equation 3.2, where c is the speed of light.

t 0 = 2 × D c (3.2)
On the other hand, t extra can be seen as the time needed to accumulate a threshold number of electrons at the C 3D capacitor, which, in its turn, can be changed in our system through three variables: the established threshold level itself (V th ); the gain provided by the avalanche photodiode; and the reflected pulse energy (P ref lected ). Considering the first two variables as constant during the landing process for simplicity, t extra is expected to increase with lower reflected P ref lected . Thus, as a first order approximation, t extra can be considered proportional to the square of the distance to the target, as presented in equation 3. [START_REF]Standard deviation of sequence of images before and after electrical noise filtering 3.2[END_REF], where e is a constant and R is the reflectivity of the target. This approximation assumes an ideal capacitance that charges linearly with time and a constant flux of photons during the integration of the light pulse.

t extra ∼ 1 P ref lected = e × D 2 R (3.3)
The consequence is that the 3D measurement V 3D is actually a quadratic function of the distance whose parameters can not be completely assessed just from the voltage ramp properties. This nonlinear behavior tends to be aggravated in applications with low number of incoming photons, which is expected at long distances in space. The calibration method that accounts for this distortion is described in section 3.3.5.

Processing Methods and Algorithms

Bad Pixel Replacement

As previous discussed, 2D images present more defective pixels than TOF images. Therefore the labeling process is made on a 2D image and applied to both. The approach used to identify defective pixels may vary a lot according to the applications. In the case of this study, the processing of labeling a bad pixel is made in two steps.

First dead and hot pixels are identified together using the median value of 50 images registered while the shutter is closed as shown in Figure 3.2. The APD gain is set to 35 in order to saturate the hot pixels thus simplifying ther identification. If a pixel value does not satisfy the "3σ condition", it is labeled as a defective pixel. Figure 3.10 demonstrates the identification of those pixels, shown in white on the right image. In this example, 2.07% of the pixels were labeled dead or hot pixels. Blinking pixels are determined next by registering 500 images. The evolution of each pixel along this set of image is then analysed, instead of a median image. Pixels jumping to extreme values frequently should be labeled as blinking pixels.

In a normal distribution, 0.27% of the samples are expected to not satisfy the "3σ condition". Following this idea, if much more samples go beyond this limit, the pixel must be blinking. In this study, the empirically chosen frequency threshold was 1%. According to this criterion, 0.36% of the pixels are labeled as blinking pixels. Adding up dead and hot pixels, the total amount of identified defective pixels is 2.43%. Note that this relatively high value is related to the fact that the detector belongs to a rather old manufacturing lot.

The strategy for replacement of these bad pixels is based on the nearest neighborhood algorithm, which is weighted by a Gaussian weight function of the Euclidean distance to the current pixel [START_REF] Isoz | [END_REF]]. Only valid neighbors pixels are considered in the calculation.

Let the pixel at spatial location (x, y) on the image plane being labeled as bad, its new value p(x, y) will be estimated as: p(x, y) =

x ,y ∈S w d(x, x , y, y ), σ p(x , y )

x ,y ∈S w d(x, x , y, y ), σ

(3.4)
where S is a discrete set of good pixels coordinates in the neighborhood of (x, y) coordinate, and w is a kernel Gaussian function: In order to overcome clusters, the size of the window that compose S must increase if the number of valid pixels is low. Let K × K be the amount of of pixels used to evaluate S and V be the number of valid pixels. The initial value for K is 3. As an adaptive rule for K we established that while V ≤ 0.6K 2 then K ← K + 1. Finally, 3.11 presents the result of the application of the method. The clusters of dead pixels were identified and replaced correctly. However some defective pixels seems to have escaped identification. Their impact shall be mitigated using the other techniques described in this chapter.

w(d, σ) = exp - d 2 σ = e -

Nonuniformity Correction

The techniques for the correction of the nonuniformity on infrared FPA can be roughly divided into calibration-based methods [START_REF] Schulz | [END_REF] or scene-based methods [Torres 2003]. Scene-based methods evaluate the different on the pixel value between subsequent frames, e.g., by moving the camera. This technique has the advantages of being insensible to camera drifts and does not need external references. Calibration-based methods, on the contrary, require the use of homogeneous radiance sources to serve as reference. The method can be employed on all kinds of image data, including static scenes.

In this work, we perform the Two Point Correction (TPC), a simple, calibration-based method widely used to correct the nonuniformity response of infrared FPAs [Vincent 2015]. The calibration is performed in laboratory conditions using a blackbody at two different temperatures. The method estimates two correction parameters per pixel, gain and offset, that are later applied on regular images. Note that these parameters have to be estimated for both 2D and 3D images.

The TOF evaluation blocks of the ROIC were initially disabled so the system would function as a passive imager and we acquired 50 images at seven different temperatures, between 15°C and 45°C. These images are then averaged to suppress noise and the result is one mean image for each temperature. Figure 3.5 shows the first and last image of this sequence. Two of them are chosen to compute the correction parameter for each pixel, cumulatively producing a Gain Image and an Offset Image. Figure 3.12 shows an example of the parameters images, computed when the images at T = 20°C and T = 40°C served as input for the calibration. The result of the TPC method adopted is shown in Figure 3.13(b): a much more uniform result but still with some defective pixels that do not behave as their neighbours. The BPR strategy described in section 3.3.1 was performed and the result is illustrated in Figure 3.13(c) with a clean uniform image. The correction results can also be appreciated in terms of the reduction of the standard deviation. Figure 3.14 shows that the standard deviation is greatly diminished after NUC operation and becomes naturally zero for the two points used as parameters for the correction (images at T = 20°C and T = 40°C). Note that the standard deviation actually increases after NUC on the image correspondent to T = 15°C. The reason is that the TPC may result in large values in defective pixels. In this example, about 2.43% of the pixels were marked as bad pixel after NUC. Since the calibration parameters are evaluated in laboratory conditions, it is necessary to assess if they hold over time. A first set of images is acquired, after waiting for the appropriate time for the detector to cool down, to serve as reference. During the same cooling routing, i.e. by refilling the liquid nitrogen and not letting the detector warm up, three new sets of images are acquired after 90, 180 and 240 minutes of the reference. The result is presented in Figure 3.15. The standard deviation of the calibrated images remained close to the reference, with a minor offset error. Between them, the results are even closer, indicating that the system has no major tendency of changing its parameters if it remains cooled. During a mission, however, it might me necessary to calibrate the detector on Earth and only cooling it down during operation in space. We tested the application of these parameters on four consecutive cooling routines. The calibration parameters represented on the Gain and Offset Images shown in Figure 3.12 are set as reference. A waiting time of at least 30 minutes was respected between each routine so that the FPA would be at room temperature before being cooled down again. The standard deviation for each image after compensation is evaluated and presented in Figure 3.16. The result of the correction is not as good for other cooling routines as for the reference. However, the standard deviation is still significantly reduced compared to the original images. In conclusion, the calibration parameters are still valid over time, the periodical re-calibration of the system might improve the results. During long field campaigns, where uniform radiance sources like a blackbody might not be available the solution is to perform a one-point correction, therefore considering only the offset parameter. A shutter is used to block all incoming photons to the detector and 50 consecutive frames are registered. The median value of each pixel through these images are calculated to create an Offset Image ( Figure 3.17). The column-wise pattern present on the image indicates the compensation of an electric offset of the column amplifiers of the ROIC, as expected.

Previous works [ de Borniol 2010] demonstrate the importance of NUC for TOF data. In order to perform the calibration, we should image a scene where every pixel is expected to have the same TOF measurement, e.g., a wall. However, the depolarization effect discussed in section 3.2.1 would cause the saturation of pixels that does not reflect the nonuniformity effect on an image of a regular scene. The image would also suffer from other scene-dependent impairments, as discussed in section 3.2. In consequence, the identification of the nonuniformity due solely to imperfections on the detector would be impossible. In a similar way to 2D images, however, the nonuniformity provoked by the ROIC can be independently identified and corrected. The first step is to set a very low threshold of detection V th , described in section 2.1.2. As soon as the system starts, this threshold is reached by the electronic noise of the system and the voltage ramp is sampled. From a different perspective, this method forces the system to detect a TOF of approximately zero for all pixels. Second, we vary the ramp start time, allowing the acquisition of 3D images at several points on the ramp, equivalent to different TOF distances. The result is a complete characterization of the voltage ramp as perceived by the APD array, depicted in Figure 3.18.

The column-structured noise is also present in these images. Two images of the linear regime of the voltage ramp serve as input for the applications of the TPC and the result of the compensation is shown in image 3.19.

As stated before, this method does not account for the effect of different levels of radiation on the nonuniformity of the images, so its performance on real scenes is limited. Nevertheless, it is noticeable on Figure 3.19 that the column-structure noise disappeared after correction. However, a low-frequency electrical noise still remains clear.

Electrical Noise Filtering

In the Fourier domain image, each point represents a particular frequency contained in the spatial domain [Jain 1989]. This frequency domain image is a complex function, whose magnitude represents the amount of that frequency present in the spatial domain function. The electrical noise is well-represented on the magnitude of the frequency domain image since the noise has a strong periodic pattern on the spatial domain image. Figure 3.20 presents the magnitude of the frames depicted on Figure 3.6 in the frequency domain. A logarithmic scale is used for better visualization.

The result shows that the images contain components of all frequencies, but usually low frequencies contain more image information than the higher ones. The low frequencies are represented around the center of the frames shown in Figure 3.20. Therefore we expect that most of the information of a noiseless image is concentrated on the center peak. However, several other peaks are present and their position change from one frame to the other. These other peaks are related to the electrical noise. Since the noise is more easily identifiable on the frequency domain, the method adopted for its mitigation is to simply filter the identified noise on the frequency domain and then reverse the new image to the spatial domain. First, following the assumption that the frequencies around the center of the frequency image contain useful information, a window is applied to spare these frequencies from filtering. Second, a threshold is calculated based on the mean magnitude value of the frequencies outsided the mentioned window. All frequencies magnitudes above this threshold are truncated.

Although simple, this noise suppression method proved to be efficient while preserving valuable information on the frames. The result on the 2D intensity image is shown in Figure 3.22. The method is also applied to 3D TOF images with similar results. 3.22 indicates that the filtering process greatly reduce the structure pattern that was present. The result can be quantitatively measured in terms of the reduction of the standard deviation, as done for the NUC and BPR procedures. The filtering method is applied to 50 consecutive frames and the result is presented in Table 3 The spatial noise was remarkably reduce on 3D frames from a mean value of 5.8mV to 2.3mV . Considering a voltage ramp with a slope of 10mV /ns and keeping in mind that 1ns of TOF is equivalent to 15cm, this filtering method would improve the spatial precision from 8.7cm to 3.75cm.

The application of this filtering method on real-like scenarios is discussed on chapter 4.

Calibration of Intensity (2D) Images

A lot of work has been done to identify the parameters that play a major role on the calibration of a LiDAR intensity data [START_REF] Kashani | [END_REF]] and the majority of current intensity calibration methods are dedicated for range and angle of incidence. Nevertheless, it is usually assumed that the scene is uniformly illuminated. In the case of large FPAs, this assumption fails to represent reality. The goal of this calibration procedure is to provide a Normalization Factor image that, when applied to the raw 2D intensity data, would mitigate the effect of the nonuniform illumination. Unfortunately, a simple blank wall target, as depicted in Figure 3.8, does not serve well for this task due to the depolarization of the APDs explained in section 3.2.1. This electronic fault produces an undesired noise effect on the image with saturated pixel values that does not reflect the spatial power dispersion. In order to overcome this issue, this calibration procedure uses a large checkerboard panel as target (Figure 3.23). The reflectivity difference between the white and black squares of the checkerboard reduces the impairment previously described. The target is translated in front of the camera at a fixed distance of 15m, to cover as much of the FOV area as possible and to mitigate local incongruities. At every position, 50 sample frames are acquired and prepossessed using the BPR and NUC methods described in sections 3.3.1 and 3.3.2 respectively. Finally, the temporal noise is then reduce by application of a median filter for each pixel.

After this preprocessing stage, a mask is created to separate white and black squares, as depicted in Figure 3.24. This step is necessary so the difference in reflectivity between the two types of square do not influence the final calibration image. Finally, consider P b the pixels belonging to black squares and P w to white ones, a mean black m b and white m w values are calculated, disregarding outliers that does not satisfy the "3σ condition". These values serve as normalization factors for each pixel, as described in the equations below:

p norm (x, y) =      m w p(x, y) , if p(x, y) ∈ P w m b p(x, y) , if p(x, y) ∈ P b (3.6)
The normalized pixels are then combined and the resulting image is filtered using a spatial median filter to obtain the smooth calibration image presented in Figure 3.25. Four calibrated targets with known reflectivity of 5%, 10%, 18% and 50% at the LASER wavelength were used in order to evaluate the overall performance of the proposed calibration method, as depicted in Figure 3.26 (a). The raw intensity images are prepocessed and the final image is cropped within the red rectangle for better visualization (Figure 3.26 (b)). Finally the result of the compensation procedure is shown in Figure 3.26 (c) and the overall change on the uniformity of the image is perceivable. As a means to quantify the improvement, the standard deviation for each calibrated target is evaluated before and after calibration, the result is presented in Table 3.2. The method successfully improved the uniformity of the intensity data in all cases.

Target Reflectivity 5% 10% 18% 50% Before Calibration 0.5 0.7 1.4 2.4 After Calibration 0.3 0.6 1.0 1.5

Calibration of TOF (3D) Images

The calibration method described in this section and its results were first published in [START_REF] Parahyba | [END_REF]]. The combination of equation 3.1, 3.2 and 3.3 establishes the following quadratic relationship between the sampled value V 3D , the distance to the target D and its reflectivity R, where p 1,2,3 are coefficients to be determined via calibration.

V 3D = p 1 × D 2 R + p 2 × D + p 3 (3.7)
Equation 3.7 has the inconvenience of expressing V 3D as a function of the Reflectivity R, which is generally not known with great precision in a real scenario. In order to overcome this ambiguity, an empirical model for the 2D measurement (intensity) was derived as presented in equation 3.8.

V 2D = q 1 × R + q 2 D 2 + q 3 × R + q 4 (3.8)
Where q 1,2,3,4 are also coefficients to be determined via calibration. By applying equations 3.7 and 3.8 it is possible to simultaneously resolve the reflectivity of the target R and its distance to the detector D using the detection measurements V 2D and V 3D . However, this set of nonlinear equations does not have a simple closed form solution. In this work, we use a gradient descent algorithm to solve the set of nonlinear equations represented by 3.7 and 3.8 [START_REF] Haykin | Adaptive filter theory[END_REF]].

The experimental setup used to evaluate the impact of t extra on the TOF measurement V 3D is illustrated in Figure 3.27. The LASER manufactured by Quantel and described in section 2.2 served as a light source for this experiment. Three calibrated targets with reflectivity of 5%, 10% and 50% at the LASER wavelength were used on the tests. The distance between the LiDAR system and the targets is varied between 4m and 24m, using a 50cm step between each acquisition. The voltage ramp duration was set to 160ns, allowing the observation of objects within a distance of 24m. All this set of measurements was performed with a mean APD gain of 35. A commercial range finder is attached to the imaging system and used to measure the ground truth. The accuracy of the empirical model for the 2D measurement is demonstrated in Figure 3.28, where the full lines represent the model described by equation 3.8. In parallel to the 2D image, figure 3.29 presents the TOF measurement with respect to the distance, in which the impact of the reflectivity on the 3D measurement becomes clear. In a situation with a high number of photons, as demonstrate with the 50% reflectivity target, the 3D measure could be considered linear with respect to the distance. Ideally, all targets should behave exactly in this way, but as the reflectivity of the target decreases, so the number of photons received, inducing a more nonlinear response of the system. Figure 3.30 shows the results after applying the calibration method. The method was able to correct the nonlinear distortion for all targets. The Root Mean Square Error (RMSE) between 6m and 20m was 21cm. It is noticeable that at short distance (less than 6 meters), the calibrated distance is erroneous for the 50% reflectivity target. This effect is attributed to a noisier detection at a high flux of photons situation, in which multipath reflections may be prominent in the laboratory conditions. Above 20m and 22m, the results for the 5% and 10% reflectivity targets, respectively, are saturated as expected due to the impossibility of retrieving useful information on those conditions. The calibration parameters calculated in laboratory conditions could be used on a flight mission, provided that the optical system would not change. However, for simplicity, we assume that the reflectivity of the target is uniform. In chapter 4, when applied to the final experimental campaign, equations 3.7 and 3.8 are simplified as:

V 3D = p 1 × D 2 + p 2 × D + p 3 (3.9) V 2D = q 1 D 2 + q 2 (3.10)

Confidence Filtering

The effect of APD depolarization, presented in section 3.2.1 impacts the 2D and 3D frames differently. As discussed in section 2.1, the energy generated when photons reaches an APD is first accumulated into a small capacitance C 3D . When the voltage threshold in this capacitance is reached, the analog voltage ramp is sampled and the value is stored as a range measurement. The depolarization effect ultimately causes a delay on the detection of the pulse, leading to an erroneous measurement. On the other hand, the total energy of the pulse detection is stored on capacitance C 2D and presented as an intensity measurement.

The effect of the depolarization of the APD is less noticeable on the 2D frame since the pulse was accumulated for a longer period of time. Figure 3.31 depicts the flat target, a blank wall, and the correspondent 2D and 3D frames. The LASER is not illuminating the entire scene and therefore the border of the beam can be seen on the image. The frames were normalized so that the effect could be compared. Note that the 3D frame is heavily impacted by the depolarization, but the 2D frame remains relatively uniform. The contrast can be further appreciated by calculating the local standard deviation images where each output pixel (Figure 3.32) contains the standard deviation value of the 7-by-7 neighborhood around the corresponding pixel of the input image depicted in Figure 3.31. The borders are highlighted due to the strong standard deviation of pixels close to an edge, but the center of the target remains relatively flat for the 2D image.

We observe from Figures 3.31 and 3.32 that, in the presence of the depolarization effect, the 2D information seems more spatially reliable than the 3D information. Therefore, the local standard deviation in 3D could be used as an outlier detection function in the filtering process. Based on this assumption, we propose a filtering method that is a variation of the bilateral filter [Tomasi 1998]. Each pixels on the 3D frame is replaced with a weighted average of pixels values in its neighborhood. The weights computation for a pixel depends on three parameters: (a) the Euclidean distance from the pixel to its neighbors; (b) the 2D intensity similarity of the pixel w.r.t. its neighbors; and (c) the local standard deviation of the pixels in the 3D frame. The weights are based on a Gaussian window . The filtering function is defined by Equation 3.11.

I f iltered 3D (x, y) = 1 W p x ,y ∈S I 3D (x , y )g(||(x , y ) -(x, y)||, σ a ) g(||I 2D (x , y ) -I 2D (x, y)||, σ b )g(||ST D 3D (x , y )||, σ c ) (3.11)
where

I f iltered

3D

is the 3D filtered frame; I 3D is the original 3D frame; I 2D is the 2D frame, (x, y) are the coordinates of the current pixel to be filtered; ST D 3D is the local standard deviation image represented in Figure 3.32; ||a|| represents the norm of vector a; S is a discrete set of pixels coordinates in the neighborhood of (x, y) and g is the Gaussian kernel, defined by Equation 3.5.

The bandwidth parameter σ is used for all pixels and optimized independently for each kernel in equation 3.11. This optimization aims to minimize the RMSE of range measurement of the pixels of a calibration flat target with respect to the ground truth provided by the Range Finder, as discussed in the previous section. The normalization term W p is defined as: The combination of the three kernels results on the filtering image presented in Figure 3.34. As expected, a higher weight is given to neighbors pixels belonging to the target and less affected by the depolarization.

W p = x ,
Finally, the result of this Confidence Filter is presented in Figure 3.35. The method successfully reduced the nonuniformity of the 3D measurement linked to the depolarization problem. The evaluation of the method in terms of accuracy and precision is discussed in Chapter 4.

Image Processing Workflow

The algorithms and methods presented in this chapter are combined in a single workflow sequence presented in Figure 3.36. The compensation of the electric faults, characterized in section 3.1, is referenced as a preprocessing phase needed to obtain good reference images used for calibration on the next phase. This image processing sequence was applied and evaluated on the experimental test campaign described in chapter 4.

Experimental Test Campaign and Results

Performing fully autonomous landings on heavenly bodies requires the on-board control system to map the designated landing area, perceive possible obstacles and correct course, if the originally planed site is deemed too risky. The earlier in the approach a topographical map can be generated, the more time the control system has to look for alternatives. The major challenge is hence to perform a three dimensional scan of a designated area. The implementation of a Flash LiDAR system with this purpose was described in chapter 2 and a set of image processing methods for the system was developed and discussed in chapter 3. Finally, this chapter describes the final experimental test campaign in the GNC Rendez-Vous And Landing Simulator (GRALS) facility of the Orbital Robotics and Guidance Navigation and Control (GNC) Laboratory at the European Space Research and Technology Centre (ESTEC) Campus. The system was mounted onto a KUKA robotic manipulator, which approached a landscape model at various velocity and noise profiles. The main goal was to acquire a new set of data; to test the efficiency of the image processing algorithms; and to assess the current performance of the system. This experimental test campaign serves as empirical evaluation of the Flash LiDAR system developed in this study co-sponsored by ESA/ESTEC. The results and conclusions derived from this campaign could be used for the further development of the FLASH LiDAR system for 3D imaging and space applications.

Test Environment

GRALS Facility

The main tests took place at GRALS, illustrated in Figure 4.2 below. The room is approximately 35 meters long, sufficient to assess the entire dynamic range of the analog voltage ramp described in section 2.1.

System Assembly on the Robotic Arm

The system, assembled as described in section 2.6, was mounted to the end effector of the robotic arm in accordance with the procedure given in the GRALS Operation Manual, as shown in Figure 4.1.

The wired connections that are required are the power supply of the Control Electronics and the LASER. These connections were tied together to form a wire train, and it was ensure that it cannot be tensioned. An ethernet cable, necessary to establish the communication link between the control electronics and the computer commanding the system, is already present on the robot and passes through the caterpillar.

During measurements, the relative position of the robot was streamed to Matlab's SIMULINK online by means of an RSI connection. This information is useful as an independent "ground truth" for the data acquired by the LiDAR system.

Test Plan

The test campaign was organized in three main phases, namely: Configuration and Calibration; Static Tests and Dynamic Tests. The first phase aims at setting up the system, verify the operation of its basic functionalities after transportation and perform a detailed calibration that is used in the next phases. The static phase refers to the evaluation of the system still, without any movement of the camera or the target. The objective is to assess the precision and accuracy of the system and arrange the target scene to be used on the final phase. Finally, the dynamic tests was performed using the robotic arm and its goal was to evaluate the system working in a video form with different moving configuration.

Configuration and Calibration

The first phase consists basically on reassure that the system is completely operational after transportation from France to the Netherlands. At the very beginning, the detector was tested in passive mode, i.e., without the LASER, in order to verify the correct behavior of the electronic system and data acquisition. Bias, clocks and power signals were all verified. A first characterization of the analog voltage ramp was also performed.

Next, the calibration procedures described in chapter 3 were performed. All security measurements discussed in Appendix B were put in place, regarding the use of the LASER and the correct handling of the liquid nitrogen.

Static Tests

The second phase of the test campaign involved the creation of a scene serving as target for the evaluation of the performance of the system. The target to be imaged by the system was built at ESA using polystyrene and was divided in four approximately equal parts (1 m x 1 m) containing different types of relief.

Figure 4.3 depicts the targets and its different features. Target 1 is not painted in order to verify the relative reflectivity the paint induces on the targets. Target 2 has a simple cross profile in order to investigate the system performance in zones of discontinuity (borders). The several differences in height on Target 3 serves to evaluate the minimum contrast perceived by the system. Finally, Target 4 has defined Gaussian profile in order to test the system capacity to reconstruct an accurate Digital Elevation Map (DEM) of a terrain. During this phase, the targets were imaged at several distances while using a Range Finder as ground truth. The optical system in front of the LASER was adjusted to a fixed Field-of-Illumination (FOI) that would be satisfactory from a distance of 30m to 15m. These tests were performed again after the system was mounted on the robotic arm and is referenced as a "discrete" trajectory on the following section.

Dynamic Tests

The final test phase consists of assessing the system performance in a dynamic environment, i.e., while moving with respect to the target. The goal is to simulate the descent of the system on an extraterrestrial body. The data was acquired on video form, as opposed to single images in the Static Tests, which could allow the future development of algorithms that take advantage of the redundancy between frames to improve the precision of the system. KUKA's robotic manipulator was set to move according to the following profiles:

• Discrete: starting at a distance of 31.60 meters to the target, the robot would move and stop every 1 meter for the acquisition of images later used for calibration. The end point is at a distance of 15.60, which yields 17 stops in total.

• Constant velocity: The robot moves with constant speeds of 10, 20, 30 or 40 cm/s towards the target without stopping.

• Constant deceleration: The robot has a initial speed of 50 cm/s and decelerates until it stops at the end point, corresponding to a mean accelaration of -7.81 mm/s 2 .

In order to simulate a random motion behaviour, the robot was programmed to perform pseudo random movements in the plane perpendicular to the main motion direction. Measurements of all velocity profiles thus are either "clean" or "random". Figure 4.4 depicts an example of a noise profile used on this experiment: the robot is set to transition between 76 different positions according to the speed pattern described in the items above. The X-axis and Y-axis are orthogonal to the main movement direction of the robotic arm and the rotation angle is with respect to the Y-axis. The reference zero in each case is with respect to a "clean" trajectory.

Performance Criteria

Unless otherwise specified, the results of this work are evaluated according to two criteria: Accuracy and Precision. Accuracy is here defined as the proximity of a measurement to the ground truth value. Precision is the level of reproducibility of a measurement, i.e., how close are the results under the same conditions. The temporal precision refers to variance of a pixel value over time under stable conditions. The spatial precision is here defined as the similarity of the values of different pixels in the image that, for a particular scene, should present the same result. Pixels representing a flat wall are an example. Unless otherwise stated, the criteria used on this study is of spatial precision.

Figure 4.5 provides a visual explanation of these concepts based on the probability density of a measurement. The evaluation of the accuracy in this study is made via the calculation of the Root Mean Square Error (RMSE), as described in Equation 4.1. The value for every pixel p in a set of P pixels is given by v p and G is the ground truth.

Accuracy = P p=1 (v p -G) 2 P (4.1)
As for the precision, the simplest evaluation method would be the standard deviation, described in Equation 4.2, where µ is the mean value of the P pixels expected to correspond to the same distance.

P recision

= P p=1 (v p -µ) 2 P -1 (4.2)

Image Processing Results and Evaluation

The image processing algorithms developed in this study and presented in chapter 3 were applied to the video sequences acquired in this experimental test campaign. The order of application of the compensation methods was presented in section 3.3.7 in the form of a workflow, here again reproduce for better reference. At first, the methods are applied to a sequence of static scenes in order to create the reference images that are used for calibration. The calibration results are presented in section 4.4.1. Finally, the reference images are used along with the algorithms to improve the quality of the video sequence of the dynamic tests according to the criteria established in section 4.3. The final results are presented in section 4.4.2.

Calibration

In order to perform the calibration procedures described in chapter 3, the back of a polystyrene mock-up was used as a target (Figure 4.7). The surface is relatively uniform and has the same reflectivity as the scene. The range finder used for ground truth indicates that the system is place at 31.56 m from the target. The robotic arm is set to move according to the discrete trajectory (section 4.2.3). At every step of one meter 50 frames are acquired, until the camera reaches the distance of 15.56 m to the target. The experiment was repeated for three values of Avalanche PhotoDiode (APD) reverse bias: 0V , -1V and -2V , corresponding to three different gain levels. This example demonstrates some of the problems faced during the execution of the test campaign. First, note that the divergence of the LASER was adjusted so that an area slightly bigger than the target was illuminated. This measure was taken as an effort to save photons while maintaining an uniform illumination, as discussed in section 2.2. Nevertheless, the upper-left part of the illuminated circle area is apparently receiving more LASER power. We suspect that this effect is caused by the misalignment of the LASER optics. The center of the illuminated area does not match the center of the image, which is explained by the fact that the mirror on the detector side is not at the same height as the LASER, also compromising the uniformity of the scene. Finally, a parasitic signal caused by the reflection on the rail of the robotic arm is visible on the right side of the image. Some of the electric and scene-dependent faults described in chapter 3 are also apparent in Figure 4.8. Defective pixels probably linked to scratches are present in the upper-left and down-right corners on both images; the low-frequency spatial noise appears strong on the 2D image; and the depolarization of the APD causes a nonuniformity effect on the center of the target on the 3D image.

Preprocessing Calibration

Following the image processing described in Figure 4.6, the first compensation step is to perform the offset correction on the 2D Raw Image and the electric NonUniformity Correction (NUC) on the 3D Raw Image, as detailed in section 3.3.2. Figure 4.9 depicts the result of this method. The images are cropped for better visualization, since the LASER FOI is much smaller than the Field-of-View (FOV) of the detector. The column-wise pattern that was presented on the 2D Raw Image disappeared and the defective pixels are highlighted for the next compensation step. The result on the 3D frame is less noticeable due to the prominence of the depolarization effect.

The next processing step is the Bad Pixel Replacement (BPR), detailed in section 3.3.1. As previously stated, the total amount of identified defective pixels for the detector used in this experimental campaign is 2.43%. Figure 4.10 shows that the method successfully replaced all apparent defective pixels, which in turn reduced the global standard deviation of the images. Finally, the impact of the low-frequency spatial noise noticeable on the 2D frame is reduced by applying the method developed on section 3.3.3. The result presented on Figure 4.11 shows that the noise level is significantly reduced, although some features are still apparent. The improvement can be quantified in terms of the standard deviation of the value of the pixels correspondent to the target. Table 4.1 summarises the result for this example. The disparity was reduced for a factor of 7 on the 2D image, an important enhancement; but for the 3D image the reduction is only of a factor of 2. This fact is largely due to the elevated impact of the depolarization of the APDs on the formation of the 3D image.

Electrical Noise Filtering Before After 2D Image 66.9 9.4 3D Image 471.11 220.5 As previously stated, the methods illustrated in this section with a specific example were applied to the whole sequence of images that served as calibration for the system.

Illumination Correction Model Estimation

After the necessary preprocessing steps described in last section, the first calibration method to be performed is the intensity correction model, detailed in section 3.3.4. During the experimental campaign, however, a decision was taken to not use a large checkboard target, but to use the same sequence of images that will later be used for the TOF calibration. This way, a sole set of images is adopted for the entire calibration sequence.

As previously stated, 17 sequences of 50 frames are recorded at every step of 1 meter. The preprocessing methods are applied to all images and then the temporal noise is further reduced by applying a median filter for every pixel, as exemplified in Figure 4.12. Pixels correspondent to the target are then cropped from these median images and a single mean value for each one is calculated, disregarding outliers that does not satisfy the "3 σ condition". These values serve as normalization factors for each pixel, as described in Equations 3.6, forming 17 normalization images.

On chapter 3 the checkboard was translated in front of the camera at a fixed distance. Here, the target covers approximately the same pixels, but at several distances. The consequence is that a smaller area of the image is calibrated, but this calibration tends to be more independent from the distance of the target. These images are combined using a median filter, which is possible because they are all normalized and thus represent the same range of values. The resulting image is filtered using a spatial filter to obtain the smooth calibration image presented in Figure 4.13. The same procedure is repeated for a reverse bias voltage of 0V , -1V and -2V so that the Intensity Calibration Image is dependent on the APD gain.

In order to proper evaluate the method, half of the images were used as input for the creation of the Intensity Calibration images and the result was applied to the other half. Similarly to what was done in section 3.3.4, the calibration result is evaluated in terms of the standard deviation of the pixels corresponding to the target. Figure 4.14 presents this measure before (dashed lines) and after 2D Calibration (full lines) with respect to the distance and the reverse bias voltage. First, note that the standard deviation is higher when the camera is close to the target, due to the elevated number of photons. The nouniformity also increases with the APD gain, as observed in section 3.1.2. The method successfully reduced the standard deviation in all cases, as expected. The 2D Calibrated Image, along with the 3D Electric Noise Filtered Image, serve as input for the procedure of Confidence Filtering, detailed in section 3.3.6. At first, the 3D image is linearly translated into TOF using the slope of the voltage ramp and subsequently to distance in meters. The voltage ramp was characterized in order to perform the NUC described in section 3.3.2 and depicted in Figure 3.18. The bandwidth parameter σ of each kernel is optimized independently in order to minimize the mean RMSE of all sequences with V AP D = 0V . The impact of V AP D was not evaluated and remains as prospects for future research. Finally, Figure 4.15 presents the result of the filtering method. The accuracy of the system was successfully increased for all distances and APD gain variations. The strong impact of the APD depolarization causes a high RMSE at short range, which decreases with the distance. Before filtering, the RMSE tends to be lower when APD gain is increased, which indicates that linearity assumption of the 3D measurement and the distance may not hold for small signals. After the application of the method, however, this difference is not appreciable. The method is also successful on reducing the standard deviation of the pixel values corresponding to the target and therefore increasing the precision of the measurement, as observed in Figure 4.16.

TOF Correction Model Estimation

The last processing step is the TOF calibration, detailed in section 3.3.5. The calibration parameters defined in equations 3.7 and 3.8 referent to the 2D and 3D measurements are determined using the result of the previous processing steps of 2D Calibration and Confidence Filtering. Figure 4.17 presents the 2D measurement with respect to the distance along with the model fitting. The model matches the points with great accuracy. Similarly, Figure 4.18 depicts the 3D measurement and the model derived from the estimated parameters. Note that for V AP D = 0V the fitting curve of the model is quite close to the measured points, but for V AP D = -2V the model error is higher. This fact may result from an increasing nonuniformity of the image at higher APD gains, which is not taken into account by the electric NUC procedure described in section 3.3.2. Finally, the improvement provided by the TOF calibration method can also be quantified in terms of Accuracy and Precision. Figure 4.20 presents the RMSE of the 3D measurement to the ground truth before calibration, i.e. assuming a linear relationship between the sampled value and the distance, and after using the method developed in this study. The accuracy was significantly increased in most cases and stayed below 1 meter through the experimental range of distances. The standard deviation of the pixels corresponding to the target did not significantly changed with the TOF Calibration and therefore the precision of the system remained as depicted in Figure 4.16.

The final result of the application of the set of processing methods on the example framed used throughout this chapter is presented in Figure 4.21. The improvement on image quality in terms of accuracy at every processing step is summarized on Table 4.2 for V AP D = 0V . The sequences 1 to 17 represent the median image of the set of 3D frames acquired at a distance varying from 15.56m to 31.56m with an onemeter step. The values on each column represents the RMSE in meters with respect to the ground truth verified with the Range Finder after each processing method. The last column represents the overall improvement defined by A linear transformation based on the slope of the voltage ramp was used to translate the output values of volts to meters. Note that the Illumination Correction is not mentioned on Table 4.2 because the method does not act directly on the 3D frame. Nevertheless, it serves as input for the Confidence Filtering method. The late made the major contribution on the reduction of the RMSE for this V AP D . The TOF Calibration contribution is comparetively limited. This result could indicate that for this test campaign the nonlinearity of the TOF measurement is not the major impairment for the quality of the images.

Video Sequence Results

Linear Motion Profiles

After a few tests with the targets depicted in Figure 4.3, we arrived to the conclusion that the grey paint used is too absorbent of the light at a wavelength of 1571nm. In order to overcome this issue, the targets were covered with white sheets of paper (Figure 4.22). In the first dynamic test performed, the KUKA's robotic manipulator was set to move with constant velocity, as described in section 4.2.3. Five speed profiles were studied: 10 cm/s, 20 cm/s, 30 cm/s and 40 cm/s. The reverse APD bias voltage was set to -2V so that in principle the Signal-to-Noise Ratio (SNR) would be the best between the curves tested during calibration. The system was set to record images as soon as the robot started moving. The maximum amount of images allowed by the memory of the system was 500 and that number was reached for the slowest speed profiles. Unfortunately some images presented an electrical interference, especially on 2D frames, as depicted in Figure 4.24. Those outlying images where automatically discarded during static tests with the use of median filtering. These interference may be linked to a ground loop issue that was detected between the LASER electronics and the control card of the detector. The images that presented this feature were identified and rejected. The images flagged as valid were then filtered to mitigate the electrical noise described in section 3.3.3. Next, the parameters of the scene-dependent calibration methods derived in last section were applied to all reminiscent images of the video sequence. Figure 4.25 presents the result of the application of the intensity calibration and confidence filtering on the example frames. Note that although the depolarization effect was in fact mitigated on the 3D image, it also became harder to distinguish the target from the background. Finally, TOF calibration parameters are applied to the 2D and 3D frames of Figure 4.25 in order to have the proper range measurement of the target. Figure 4.26 shows that parts of the target that are not completely covered by the paper sheet are easily identifiable on the 3D image because they appear to be further distant than the rest of the image, indication of the low reflectivity of the paint.

The maximum height difference on the target is of 20 cm. The pyramidal profile of the target is hardly identifiable on Figure 4.26 suggesting that the experimental conditions and/or the system need to be further adjusted to allow the identification of detail features in the terrain.

Henceforth the results are focused on the measured distance of a flat part of the target, e.g. the center square of the cross, with respect to the ground truth provided by the KUKA robotic manipulator. Table 4.3 summarizes the evaluation of the measurements in terms of RMSE with respect to the ground truth. The highest incremental improvement was achieved with the Confidence Filtering method, as had been the case of Table 4.2. The overall improvement provided by the set of compensation methods is significant, although being far from ideal. This result demonstrates that the performance of the system needs a significant improvement in order to identify features of 20 cm on the target. The results in terms of precision are summarized on Table 4.4. In this case, the precision is measured as the mean standard deviation of the pixels corresponding to the center square of the target for all images of a sequence. As for the accuracy, the Confidence Filtering method provided the greatest enhancement among the studied processing steps. This is an expected result, since the Confidence Filtering is a low-pass spatial filter and therefore reduces the local standard deviation. Nevertheless, the performance of the method in both criteria greatly improved the overall quality of the measurements.

Sequence

Accelerated Motion Profiles

A similar performance improvement was found for the accelerated motion profiles. Figure 4.29 presents the comparison between the Raw and Calibrated 3D measurement throughout the video sequence. The robot starts moving at 50 cm/s and progressively decelerates. The memory limit of the FPGA is reached before the robot arrives at the end point. The calibration measurement curve is much closer to the ground truth, especially at the beginning of the movement (low amount of photons) and at the end (high depolarization effect). Some local peaks indicate issues that were not completely resolved by the processing algorithms. Table 4.5 summarizes the accuracy results for the accelerated motion profiles in terms of RMSE. Finally, Figure 4.30 depicts a sequence of consecutive 3D frames of the random motion profile series. The reflection of the robotic arm's rail is present in some of these images. The set of imaging processing methods were in principle successful in improving the overall quality of the image, but the fact that the background has approximately the same reflectivity as the targets makes it difficult to isolate the measurements in order to compare with the ground truth.

Criteria

Conclusion

This final chapter is an opportunity to review all the work developed in this thesis. We first present a quick summary of the different contributions to the implementation of a Flash LiDAR prototype system for space applications before proposing some ways to continue this research.

Summary

In the introduction of this thesis, we presented the evolution of 3D cameras in general and in the context of space exploration missions in particular. The principle of operation of a Flash LiDAR is discussed along with its main advantages with other 3D vision solutions. First of all, LiDARs are active devices, allowing the possibility of operations without the presence of sun light and limiting the noise caused by the later. These devices record full 3D images with a single LASER pulse, eliminating the need for a scanning device. Finally, the detector is composed by an array of avalanche photodiodes, capable of providing a gain to the small signals generated by a low incoming flux of photons and therefore mitigating the impact of the electronic noise. The detector used in this thesis is based on a Mercury Cadmium Telluride (MCT) substrate allowing high gain with low excess noise factor; the major drawback is that the detector needs to be cooled down with liquid nitrogen.

The detector developed by CEA-LETI was the base for the implementation of a Flash LiDAR prototype detailed in Chapter 2. The main constituents of the system were described and optimized for the experimental test campaign, as follows:

• A new LASER was specified, purchased and integrated in the prototype.

• A new electronic control system was designed based on a FPGA and three boards: the mother card, the daughter card and the voltage ramp generator. This step significantly reduced the size of the system and made it independent of an external ramp generation source.

• An optical system based on two converging lenses was integrated to the output of the LASER in order to guarantee the safety of the laboratory users and to regulate the size of the light beam. Thermal and parasitic radiations on the sensor were limited by the use of filters.

The system was designed to respect the constraints of weight and size for the safe usage of the robotic arm during the experimental evaluation campaign. The final prototype assembly is presented and discussed in section 2.6.

The limits and major defects of the prototype are discussed in Chapter 3. The main work developed on this thesis was the development of a set of calibration methods and image processing algorithms to mitigate these faults. The list of studied and developed algorithms include:

• NonUniformity Correction (NUC) for the mitigation of the Fixed Pattern Noise (FPN);

• Bad Pixel Replacement (BPR) for the identification and replacement of defective pixels;

• Electrical Noise Filtering based on the Fourier transform of the image for the mitigation of a low-frequency spatial noise;

• Intensity Calibration in order to compensate for the nonuniform illumination of the scene;

• Confidence Filtering based on the fact that the 2D image present less spatial variance than the Time-of-Flight (TOF) image, this methods aims the mitigation of the APD depolarization defect.

• TOF Calibration for the mitigation of the slightly nonlinear relationship between the true distance and the 3D measurement.

An image processing workflow was established and is reproduced in Finally, an experimental test campaign was realized at the European Space Agency (ESA) Research and Technology Center in Noordwijk, the Netherlands. This was the first opportunity to assess the prototype along with the complete set of processing methods and evaluate its performance and limitations, described in Chapter 4. The system was mounted onto a robotic manipulator that followed several programmed motion profiles towards polystyrene targets built for the test.

The calibration parameters for the processing methods were derived first by imaging a blank wall at a series of fixed distances to the camera. In a second phase, the system acquired a series of video sequences on which the processing methods were applied using the calibration parameters determined priorly. Two criteria, namely the accuracy and the precision, were used to evaluate the quality of the measurements and the incremental improvement at each stage of the image processing workflow.

The overall improvement was higher than 30% in accuracy and 80 % in precision for the studied cases.

Prospects for Future Developments

As a first attempt of this prototype, there are numerous opportunities for improvement. The set of amelioration action can be divided in three categories, discussed in this section: system; processing methods and the test environment.

System Improvements

As previously discussed, the batch of detectors used on this thesis were several years old. This contributed to an abnormal amount of defective pixels and possibly to a stronger nonuniform response through the Focal Plane Array (FPA). Between the five detectors that were tested at some point, only one remained operational until the end of the experimental campaign. A new round of manufacturing could not only provide less defective detectors but also could be used as an opportunity to review the Read-Out Integrated Circuit (ROIC) design to improve the detection threshold. A different ratio of material in the MCT alloy could allow a lower cutoff wavelength leading to a less demanding operational temperature for the detector and a thermoelectric cooling system rather than liquid nitrogen or cryocooling. The evolution of the fabrication process of the APDs could in turn help preventing the depolarization effect described in section 3.2.1.

The LASER, although more powerful than the previous one, had some unexpected issues that could be addressed on a future prototype. First, it generates an output clock after the light pulse is fired with low jitter (less than 2 ns), but with a high delay. The consequence is that the current system is incapable of measuring distances less than 15 meters. The performance of the system at close range remains a matter for a future study. Second, there was a problem with the integration of the clock signals and the occurrence of a possible ground loop which lead to interference on the images that needed to be rejected, as discussed in section 4.4.2.1.

As for the optical system, a better attachment method would improve the quality of the alignment of the lenses allowing a more uniform distribution of the LASER power. The Fieldof-Illumination (FOI) of the LASER and the Field-of-View (FOV) of the detector could also be better matched to avoid energy waste. The employ of a dewar facing towards the target would prevent the need of a mirror which would make the prototype 500 g more light.

Finally, the control electronics was a great development in comparison to the old version in terms of size and capabilities, including here the conception of the voltage ramp generator board. Nevertheless, the level of electrical noise contaminating the images was higher than expected. An examination of the causes of this problem could reduce the influence of the low-frequency spatial noise discussed in section 3.1.3.

Processing Methods Improvements

The effect of the NUC procedure was limited because of the depolarization issue. In the case of the mitigation of this problem with a reviewed version of the ROIC design, the method could be applied on the 3D frames by acquiring images covering the entire FPA at two different distances to the target. The impact of this new NUC and the stability of the calibration method would need to be reevaluated.

The results evaluated on Chapter 4 indicates that the Confidence Filtering is the most efficient method on the improvement of the accuracy of the system. For the future development of this method, an investigation of the impact of the bandwidth parameters σ could lead to a superior performance. The current implementation of this method is at the moment quite slow and could be optimized.

The image processing during this study was completely done offline. The next step would be to implement a real-time version of the set of methods and assess its complexity in terms of number of operations and memory requirements.

The integration of the processing methods discussed in this thesis with Hazard Detection and Avoidance (HDA) and Safe Site Selection (SSS) algorithms that interact with the Guidance Navigation and Control (GNC) unit to adapt the course of during landing remains as an interesting topic for a future work.

Test Environment Improvements

The experimental test campaign allowed the evaluation of the system and the set of processing algorithms at close range. A few adjustments would, nevertheless, improve the quality of the images. First, the rail onto which the robotic arm is attached and moves reflects the light pulse, causing parasitic interference and multipath reflections that degrades the images. A black paint could mitigate this effect. Second, the black curtains that served as a background for the target proved to be reflective at the wavelength of the light pulse. Ideally, it should be absorbent in order to create a better contrast with the target.

As for the target, it proved to not be suited for the experiment. The paint used was too absorbent of the light pulse and the temporary solution was to cover them with sheets of paper. A previous study of the reflectivity of the paint must be performed, ideally to match the known reflectivity of an extraterrestrial body, such as the Moon. The size of the features in the targets were too small to be identified with the system current resolution. In future works, a higher range of features sizes would help to better evaluate the system capability to identify dangerous terrain zones.

Finally, the system is conceived for long distance measurements and therefore an indoor laboratory environment limits the complete evaluation of the system capabilities. A future step on the prototype development should include tests at hundreds of meters between the camera and the target, including the coupling to a flight module to investigate the system performance on a real terrain. The maximum time considered possible for unintentional exposure of an eye in front of a laser without movement of the person, head or eyes is 10 seconds. The MPE for an exposure of This experiment is estimated to need 2 liters of liquid nitrogen per day, which will be stored in a conventional transportation dewar. So V g = 2 × 683 = 1366l = 1.366m 3 . This amount of gas is considered to be safe if the room volume satisfies the condition:

0.2095 × (V R -V g ) V R > 20% (B.7) V R > 30.12m 3 (B.8)
The length of the room is 35 meters and the width and height are higher than 2 meters. The conclusion is that amount of liquid nitrogen used in this experiment does not represent an asphyxiation risk, since the room volume is much larger than the minimum V R calculated.

The other risk liquid nitrogen represents besides asphyxiation are cryogenic burns. Liquid nitrogen can cause cryogenic burns if the substance itself, or surfaces which are or have been in contact with the substance (e.g. metal transfer hoses), come into contact with the skin. Local pain may be felt as the skin cools, though intense pain can occur when cold burns thaw and, if the area affected is large enough, the person may go into shock. In order to mitigate this hazard, persons manipulating the dewar should use these personal protective equipment:

• Hands: non-absorbent insulated gloves must always be worn when handling anything that is or has been in recent contact with liquid nitrogen. Cryogenic gloves are designed to be used in the vapor phase only and should not be immersed into liquid nitrogen under any circumstances.

• Face: Security googles should be used to protect the eyes where splashing or spraying may occur and, in particular, where operations are carried out at eye level.

• Feet: sturdy shoes with a re-enforced toecap are recommended for handling liquid nitrogen vessels. Open toed shoes should not be worn under any circumstances.

1. 1

 1 Example of a 3D Stereo Camera [3DC 2019]. . . . . . . . . . . . . . . . . . . 1.2 Example of a Projected-Light sensor: Kinect™ for XBOX 360. . . . . . . . . 1.3 Example of a phase-based camera system: Basler's time-of-flight camera. . . 1.4 Example of a Scanning LiDAR system -Velodyne® LiDAR HDL-64E. . . . 1.5 Principle of Operation of a Flash LiDAR Camera. . . . . . . . . . . . . . . . 1.6 Example of a Geiger-mode Flash LiDAR: Kestrel from Priceton Lightwave™. 1.7 Example of a Linear-mode Flash LiDAR: GoldenEye developed by ASC™. . 2.1 Block Diagram for a LiDAR system with main subparts. . . . . . . . . . . . 2.2 Hybridization between the MCT-APD array and the silicon ROIC. . . . . . 2.3 Simplified schematic of the pixel. . . . . . . . . . . . . . . . . . . . . . . . . 2.4 Calculating the received power: parameters. . . . . . . . . . . . . . . . . . . 2.5 Absolute solar spectral irradiance from 200 to 2400 nm after normalization by 1.4% [Thuillier 2003] . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.6 Quantel's ULTRA LASER system . . . . . . . . . . . . . . . . . . . . . . . . 2.7 (a) AMS's Optical LASER Head and (b) Control Box . . . . . . . . . . . . . 2.8 Control Electronics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.9 New Control System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.10 Schematic Diagram of the Voltage Ramp Generator board . . . . . . . . . . 2.11 Voltage Ramp Generator Card with its connection points . . . . . . . . . . . 2.12 Voltage Ramp visualized on an oscilloscope on persistence time mode. The width of the signal is related to the jitter . . . . . . . . . . . . . . . . . . . . 2.13 Transmittance of the hot band-pass filter between 1.5 µm and 1.8 µm . . . . 2.14 Dewar containing liquid nitrogen to cool down the FPA: condensated water from the air can be noted on the top of the component . . . . . . . . . . . . 2.15 Temporal Standard Deviation for each pixel between 400 images for two detector samples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.16 Robotic arm's payload diagram with the estimated Center of Gravity highlighted in red . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.17 3D Model of the LiDAR System. The center of mass is represented in pink. 2.18 LiDAR system: (a) front view (b) back view . . . . . . . . . . . . . . . . . . 2.19 LiDAR system mounted on KUKA's robotic manipulator along with the LASER's control box . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.20 Screenshot of Labview's subVIs responsible for system operation . . . . . . . 3.1 Example of Flash LiDAR output, including the 2D intensity and 3D TOF frames. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2 Example of 2D images containing bad pixels, individual and clusters. . . . . 3.3 Value of a blinking pixel through a sequence of images. . . . . . . . . . . . . viii LIST OF FIGURESix 3.4 Testbench for the evaluation of the nonuniformity: the detector faces a blackbody emitting an uniform flux of photons. . . . . . . . . . . . . . . . . . . . 3.5 2D images of a blackbody at 15°C and 45°C demonstrating an increasing nonuniformity with the incoming flux of photons . . . . . . . . . . . . . . . . 3.6 Sequence of 2D frames presenting the low frequency electrical noise on the detector. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.7 The effect of depolarization in the center of the image of a flat area. . . . . . 3.8 Blank wall image captured by the LiDAR system. Nonuniformity is provoked by a set of undesired effects such as APD depolarization, non uniform illumination of the LASER and imperfections on the optical system. . . . . . . . . 3.9 Representation of the analog voltage ramp (time base) . . . . . . . . . . . . 3.10 Identification of dead and hot pixels. . . . . . . . . . . . . . . . . . . . . . . 3.11 Result of Bad Pixel Replacement . . . . . . . . . . . . . . . . . . . . . . . . . 3.12 Gain Image and Offset Image . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.13 (a) Example of a passive image taken in front of a blackbody; (b) same image after nonuniformity correction and (c) after bad pixel replacement. . . . . . 3.14 Result of the application of NUC (green) and NUC + BPR (blue) on a set of images taken in front of a blackbody . . . . . . . . . . . . . . . . . . . . . . . 3.15 Result of the application of the NonUniformity Correction (NUC) parameters in the same cooling routine over time. . . . . . . . . . . . . . . . . . . . . . . 3.16 Result of the application of the NUC parameters in other cooling routines. . 3.17 Offset image used in one-point correction. . . . . . . . . . . . . . . . . . . . 3.18 Voltage ramp characterization (yellow line represents fitting) . . . . . . . . . 3.19 Effect of the electric NUC procedure. Left: raw image; right: compensated image. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.20 Sequence of frames on the frequency domain . . . . . . . . . . . . . . . . . . 3.21 Lines of a frequency domain image indicating the noise harmonics . . . . . . 3.22 Noise Filtering -2D intensity image . . . . . . . . . . . . . . . . . . . . . . . 3.23 Checkerboard calibration target. . . . . . . . . . . . . . . . . . . . . . . . . . 3.24 Masked images. Left: white squares, right: black squares. . . . . . . . . . . . 3.25 Surface plot of the Intensity Calibration Image . . . . . . . . . . . . . . . . . 3.26 The LiDAR system prototype, on the left, and the checkerboard calibration target, on the right. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.27 Experimental Setup for the evaluation of the TOF calibration method . . . . 3.28 Intensity measurement with respect to the distance for three targets with different reflectivities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.29 Uncalibrated 3D measurements of three targets with known reflectivity with respect to the distance evaluated by the range finder. . . . . . . . . . . . . . 3.30 Calibrated measured distance of three targets with respect to distance evaluated by the range finder . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.31 Normalized 2D and 3D frames of a flat target depicting the APD depolarization effect. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.32 2D and 3D local standard deviation image. . . . . . . . . . . . . . . . . . . . 3.33 Example kernel for a pixel in the center of the flat target (red cross): (a) Euclidean distance; (b) Similarity on 2D frame; (c) local standard deviation in 3D frame . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.34 Final filter image composed by the kernels depicted in Figure 3.33 . . . . . . 3.35 Result of the application of the Confidence Filtering method . . . . . . . . . 3.36 Image Processing Workflow . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.1 The Flash LiDAR is mounted onto the end effector of the robotic arm. . . . x LIST OF FIGURES 4.2 GRALS Facility, as seen from (a) the South-West end and (b) the North-East 4.3 Polystyrene targets used to assess the system's resolution . . . . . . . . . . . 4.4 Profile of the random movement of the robotic arm . . . . . . . . . . . . . . 4.5 Definition of Accuracy and Precision [Wikipedia ] . . . . . . . . . . . . . . . 4.6 Image Processing Workflow . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.7 Wall target used for system calibration . . . . . . . . . . . . . . . . . . . . . 4.8 Raw 2D and 3D example frames of a blank wall target at 23.56m . . . . . . 4.9 Result of the application of the nonuniformity compensation method on the example frames of Figure 4.8 . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.10 Result of the application of the bad pixel replacement method on the example frames of Figure 4.8 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.11 Result of the application of the electrical noise filtering on the example frames of Figure 4.8 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.12 Example of a single image of a blank wall target at 23.56m (left) and the median of 50 similar images (right) . . . . . . . . . . . . . . . . . . . . . . . 4.13 Surface plot of the Intensity Calibration Image for a reverse bias voltage of 0V 4.14 Standard deviation of the target before (dashed lines) and after 2D Calibration (full lines) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.15 Accuracy of the 3D measurement before (dashed lines) and after Confidence Filtering (full lines) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.16 Precision of the 3D measurement before (dashed lines) and after Confidence Filtering (full lines) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.17 Intensity measurement with respect to the distance for different APD gains. 4.18 Uncalibrated 3D measurement with respect to the distance for different APD gains. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.19 Calibrated 3D measurement with respect to the distance for different APD gains. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.20 Accuracy of the 3D measurement before (dashed lines) and after TOF Calibration (full lines) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.21 3D Raw Image before the set of processing algorithms (left) and the final calibrated image (right) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.22 Polystyrene target 2 covered with white sheets of paper . . . . . . . . . . . . 4.23 Example of 2D and 3D frames after the application of NUC and BPR procedures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.24 Example of 2D and 3D frames presenting electrical interference. . . . . . . . 4.25 Example of 2D and 3D frames after Intensity Calibration and Confidence Filtering. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.26 The target (left) and the final cropped 3D image after TOF Calibration . . . 4.27 Results for the linear movement profile at 10 cm/s, before and after the application of the processing methods. . . . . . . . . . . . . . . . . . . . . . . . . 4.28 Results for the linear motion profile with respect to the ground truth. . . . . 4.29 Results for the accelerated motion profile with respect to the ground truth. . 4.30 Sequence of frames demonstrating the random linear motion . . . . . . . . . 5.1 Image Processing Workflow . . . . . . . . . . . . . . . . . . . . . . . . . . . . A.1 Schematics of the Pixel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A.2 Schematics of the Operational Transconductance Amplifier (OTA) . . . . . . A.3 Schematics of the Comparator and locking block . . . . . . . . . . . . . . . . B.1 Designed LASER Optical System for Operational Safety . . . . . . . . . . .

ADC

  Analog-to-Digital Converter APD Avalanche PhotoDiode BPR Bad Pixel Replacement CEA Commissariat à l'énergie atomique et aux énergies alternatives CMOS Complementary Metal-Oxide-Semiconductor CPU Central Processing Unit CTIA Capacitive Transimpedance Amplifier CW Continuous Wave DEM Digital Elevation Map ESA European Space Agency ESTEC European Space Research and Technology Centre FOI Field-of-Illumination FOV Field-of-View FPA Focal Plane Array FPGA Field-Programmable Gate Array FPN Fixed Pattern Noise GNC Guidance Navigation and Control GRALS GNC Rendez-Vous And Landing Simulator HDA Hazard Detection and Avoidance HRN Hazard Relative Navigation InGaAs Indium Gallium Arsenide LASER Light Amplification by Stimulated Emission of Radiation LiDAR Light Detection and Ranging LWIR Long-Wavelength Infra-Red MCT Mercury Cadmium Telluride MPE Maximum Permissible Exposure MWIR Medium-Wavelength Infra-Red NASA National Aeronautics and Space Administration ND:YAG Neodymium-Doped Yttrium Aluminium Garnet NOHD Nominal Ocular Hazard Distance NUC NonUniformity Correction xiii xiv LIST OF TABLES OPO Optical Parametric Oscillator OTA Operational Transconductance Amplifier PMOS P-type metal-oxide-semiconductor RMSE Root Mean Square Error ROIC Read-Out Integrated Circuit SNR Signal-to-Noise Ratio SPAD Single-Photon Avalanche Diode SSS Safe Site Selection SWIR Short-Wavelength Infra-Red TOF Time-of-Flight TPC Two Point Correction TRL Technology Readiness Level TRN Terrain Relative Navigation Chapter 1

Figure 1 .

 1 1 presents an example of a commercial 3D Stereo Camera.

Figure 1 . 1 :

 11 Figure 1.1: Example of a 3D Stereo Camera [3DC 2019].

Figure 1 . 2 :

 12 Figure 1.2: Example of a Projected-Light sensor: Kinect™ for XBOX 360.

  Figure 1.3 shows an example of a phase-based camera.

Figure 1 . 3 :

 13 Figure 1.3: Example of a phase-based camera system: Basler's time-of-flight camera.

Figure 1 .

 1 4 shows an example of a scanning LiDAR sensor designed for obstacle detection and navigation of autonomous vehicles.

Figure 1 . 4 :

 14 Figure 1.4: Example of a Scanning LiDAR system -Velodyne® LiDAR HDL-64E.

Figure 1 . 5 :

 15 Figure 1.5: Principle of Operation of a Flash LiDAR Camera.

Figure 1 . 6 :

 16 Figure 1.6: Example of a Geiger-mode Flash LiDAR: Kestrel from Priceton Lightwave™.

Figure 1 . 7 :

 17 Figure 1.7: Example of a Linear-mode Flash LiDAR: GoldenEye developed by ASC™.

Figure 2 . 1 :

 21 Figure 2.1: Block Diagram for a LiDAR system with main subparts.

Figure 2 . 2 :

 22 Figure 2.2: Hybridization between the MCT-APD array and the silicon ROIC.

Figure 2 . 3 :

 23 Figure 2.3: Simplified schematic of the pixel.

Figure 2 . 4 :

 24 Figure 2.4: Calculating the received power: parameters.

  Energy per pulse (E T ) 5 mJ Illuminated area (A illum ) 110% of Target area (A T ) Radius of the detector optics 50 mm Target Reflectivity (ρ T ) 12% System Performance (η sys ) 50% Atmospheric absorption (η atm ) Negligible

Figure 2 . 5 :

 25 Figure 2.5: Absolute solar spectral irradiance from 200 to 2400 nm after normalization by 1.4% [Thuillier 2003]

Figure 2 .

 2 Figure 2.6: Quantel's ULTRA LASER system

  .4. For its superior performance in terms of pulse energy and a relatively short pulse duration, AMS Technologies was chosen. The LASER model is IFL-N1530-OPO-EO (Figure 2.7 a) and it is manufactured by DPLE group. Despite the LASER head being relatively light, 2.15 kg, and being thermoelectric cooled, it is associated with a control box having 3.1 kg (Figure 2.7 b). Due to the weight constraints imposed by the robotic arm in the final test campaign (section 2.6.2), this component was handled separately.

Figure 2 .

 2 Figure 2.7: (a) AMS's Optical LASER Head and (b) Control Box

Figure 2

 2 Figure 2.8: Control Electronics

Figure 2 . 9 :

 29 Figure 2.9: New Control System

Figure 2 .

 2 Figure 2.10: Schematic Diagram of the Voltage Ramp Generator board

Figure 2 .

 2 11 depicts the manufactured voltage ramp generator card and its connection points. The start of the ramp is controlled by a +3.3V logic at the CLK input. The inverter U 9 controls the opening of the switch U 3. The current of Q3 then discharges C1. The end of discharge voltage is controlled by D4, Q2 and the voltage of J6. Voltage of J6 = endof dischargevoltage + 1.2V .

Figure 2 .

 2 Figure 2.11: Voltage Ramp Generator Card with its connection points

Figure 2 .

 2 Figure 2.12: Voltage Ramp visualized on an oscilloscope on persistence time mode. The width of the signal is related to the jitter

Figure 2 .

 2 Figure 2.13: Transmittance of the hot band-pass filter between 1.5 µm and 1.8 µm

Figure 2 .

 2 Figure 2.14: Dewar containing liquid nitrogen to cool down the FPA: condensated water from the air can be noted on the top of the component

Figure 2 .

 2 Figure 2.15: Temporal Standard Deviation for each pixel between 400 images for two detector samples

Figure 2 .

 2 Figure 2.16: Robotic arm's payload diagram with the estimated Center of Gravity highlighted in red

Figure 2 .

 2 Figure 2.17: 3D Model of the LiDAR System. The center of mass is represented in pink.

Figure 2 .

 2 Figure 2.18: LiDAR system: (a) front view (b) back view

Figure 2 .

 2 Figure 2.19: LiDAR system mounted on KUKA's robotic manipulator along with the LASER's control box

Figure 2 .

 2 Figure 2.20: Screenshot of Labview's subVIs responsible for system operation

Figure 3 . 1 :

 31 Figure 3.1: Example of Flash LiDAR output, including the 2D intensity and 3D TOF frames.

Figure 3 . 2 :

 32 Figure 3.2: Example of 2D images containing bad pixels, individual and clusters.

Figure 3 .

 3 3 presents the value of a pixel through 500 images. The mean value is represented by µ and the standard deviation by σ. In this work, we define as extreme, or outliers, pixel values that do not satisfy the condition |p(x, y) -µ| ≤ 3σ. Henceforth this definition is called the "3σ condition". On Figure3.3 the pixel jumps to this extreme values in more occasions than expected, indicating a blinking behavior.

Figure 3 . 3 :

 33 Figure 3.3: Value of a blinking pixel through a sequence of images.

Figure 3 . 4 :

 34 Figure 3.4: Testbench for the evaluation of the nonuniformity: the detector faces a blackbody emitting an uniform flux of photons.

Figure 3 . 5 :

 35 Figure 3.5: 2D images of a blackbody at 15°C and 45°C demonstrating an increasing nonuniformity with the incoming flux of photons

Figure 3 . 6 :

 36 Figure 3.6: Sequence of 2D frames presenting the low frequency electrical noise on the detector.

Figure 3 . 7 :

 37 Figure 3.7: The effect of depolarization in the center of the image of a flat area.

Figure 3 . 8 :

 38 Figure 3.8: Blank wall image captured by the LiDAR system. Nonuniformity is provoked by a set of undesired effects such as APD depolarization, non uniform illumination of the LASER and imperfections on the optical system.

Figure 3 . 9 :

 39 Figure 3.9: Representation of the analog voltage ramp (time base)

Figure 3 .

 3 Figure 3.10: Identification of dead and hot pixels.

  σ being the bandwidth of the Gaussian window.

Figure 3 .

 3 Figure 3.11: Result of Bad Pixel Replacement

Figure 3 .

 3 Figure 3.12: Gain Image and Offset Image

Figure 3 .

 3 Figure 3.13: (a) Example of a passive image taken in front of a blackbody; (b) same image after nonuniformity correction and (c) after bad pixel replacement.

Figure 3 .

 3 Figure 3.14: Result of the application of NUC (green) and NUC + BPR (blue) on a set of images taken in front of a blackbody

Figure 3 .

 3 Figure 3.15: Result of the application of the NUC parameters in the same cooling routine over time.

Figure 3 .

 3 Figure 3.16: Result of the application of the NUC parameters in other cooling routines.

Figure 3 .

 3 Figure 3.17: Offset image used in one-point correction.

Figure 3 .

 3 Figure 3.18: Voltage ramp characterization (yellow line represents fitting)

Figure 3 .

 3 Figure 3.19: Effect of the electric NUC procedure. Left: raw image; right: compensated image.

Figure 3 .

 3 Figure 3.20: Sequence of frames on the frequency domain

Figure 3 .

 3 Figure 3.21: Lines of a frequency domain image indicating the noise harmonics

Figure 3 .

 3 Figure 3.22: Noise Filtering -2D intensity image

Figure

  Figure 3.22 indicates that the filtering process greatly reduce the structure pattern that was present. The result can be quantitatively measured in terms of the reduction of the standard deviation, as done for the NUC and BPR procedures. The filtering method is applied to 50 consecutive frames and the result is presented in Table3.1.

Figure 3 .

 3 Figure 3.23: Checkerboard calibration target.

Figure 3 .

 3 Figure 3.24: Masked images. Left: white squares, right: black squares.

Figure 3 .

 3 Figure 3.25: Surface plot of the Intensity Calibration Image

Figure 3 .

 3 Figure 3.26: The LiDAR system prototype, on the left, and the checkerboard calibration target, on the right.

Figure 3 .

 3 Figure 3.27: Experimental Setup for the evaluation of the TOF calibration method

Figure 3 .

 3 Figure 3.28: Intensity measurement with respect to the distance for three targets with different reflectivities

Figure 3 .

 3 Figure 3.29: Uncalibrated 3D measurements of three targets with known reflectivity with respect to the distance evaluated by the range finder.

Figure 3 .

 3 Figure 3.30: Calibrated measured distance of three targets with respect to distance evaluated by the range finder

Figure 3 .

 3 Figure 3.31: Normalized 2D and 3D frames of a flat target depicting the APD depolarization effect.

Figure 3 .

 3 Figure 3.32: 2D and 3D local standard deviation image.

  y ∈S g(||(x , y ) -(x, y)||, σ a )g(||I 2D (x , y ) -I 2D (x, y)||, σ b )g(||ST D 3D (x , y )||, σ c ) (3.12) Figure 3.33 illustrates an example of a weights map that compose the Confidence Filter for a pixel on the center of the target. Each one represents the parameters that compose the filtering method.

Figure 3 .

 3 Figure 3.33: Example kernel for a pixel in the center of the flat target (red cross): (a) Euclidean distance; (b) Similarity on 2D frame; (c) local standard deviation in 3D frame

Figure 3 .

 3 Figure 3.34: Final filter image composed by the kernels depicted in Figure 3.33

Figure 3 .

 3 Figure 3.35: Result of the application of the Confidence Filtering method

Figure 3 .

 3 Figure 3.36: Image Processing Workflow

Figure 4 . 1 :

 41 Figure 4.1: The Flash LiDAR is mounted onto the end effector of the robotic arm.

Figure 4 . 2 :

 42 Figure 4.2: GRALS Facility, as seen from (a) the South-West end and (b) the North-East

Figure 4 . 3 :

 43 Figure 4.3: Polystyrene targets used to assess the system's resolution

Figure 4 . 4 :

 44 Figure 4.4: Profile of the random movement of the robotic arm

Figure 4 . 5 :

 45 Figure 4.5: Definition of Accuracy and Precision [Wikipedia ]

Figure 4 . 6 :

 46 Figure 4.6: Image Processing Workflow

Figure 4 . 7 :

 47 Figure 4.7: Wall target used for system calibration

Figure 4 .

 4 Figure 4.8 presents an example of an intensity and Time-of-Flight (TOF) frames before preprocessing at 23.56m from the target, the APD reverse bias was set to 0V in this case. The colorbar range of the images presented here follow henceforth the "3σ condition" first discussed in section 3.1.1. On a 2D Raw image a low voltage means a stronger signal, the colors are inverted after Offset compensation.This example demonstrates some of the problems faced during the execution of the test campaign. First, note that the divergence of the LASER was adjusted so that an area slightly bigger than the target was illuminated. This measure was taken as an effort to save photons while maintaining an uniform illumination, as discussed in section 2.2. Nevertheless, the

Figure 4 . 8 :

 48 Figure 4.8: Raw 2D and 3D example frames of a blank wall target at 23.56m

Figure 4 . 9 :

 49 Figure 4.9: Result of the application of the nonuniformity compensation method on the example frames of Figure 4.8

Figure 4 .

 4 Figure 4.10: Result of the application of the bad pixel replacement method on the example frames of Figure 4.8

Figure 4 .

 4 Figure 4.11: Result of the application of the electrical noise filtering on the example frames of Figure 4.8

Figure 4 .

 4 Figure 4.12: Example of a single image of a blank wall target at 23.56m (left) and the median of 50 similar images (right)

Figure 4 .

 4 Figure 4.13: Surface plot of the Intensity Calibration Image for a reverse bias voltage of 0V

Figure 4 .

 4 Figure 4.14: Standard deviation of the target before (dashed lines) and after 2D Calibration (full lines)

Figure 4 .

 4 Figure 4.15: Accuracy of the 3D measurement before (dashed lines) and after Confidence Filtering (full lines)

Figure 4 .

 4 Figure 4.16: Precision of the 3D measurement before (dashed lines) and after Confidence Filtering (full lines)

Figure 4 .

 4 Figure 4.17: Intensity measurement with respect to the distance for different APD gains.

Figure 4 .

 4 Figure 4.18: Uncalibrated 3D measurement with respect to the distance for different APD gains.

Figure 4 .

 4 Figure 4.19 shows the results after applying the calibration method. As expected from

Figure 4 .

 4 Figure 4.19: Calibrated 3D measurement with respect to the distance for different APD gains.

Figure 4 .

 4 Figure 4.20: Accuracy of the 3D measurement before (dashed lines) and after TOF Calibration (full lines)

Figure 4 .

 4 Figure 4.21: 3D Raw Image before the set of processing algorithms (left) and the final calibrated image (right)

Figure 4 .

 4 Figure 4.22: Polystyrene target 2 covered with white sheets of paper

Figure 4 .

 4 Figure 4.23: Example of 2D and 3D frames after the application of NUC and BPR procedures

Figure 4 .

 4 23 depicts an example of 2D and 3D frames of this test after the application of these methods. The target is unfortunately barely distinguishable from the background. We reached the conclusion that the black curtains do not absorb well the light at 1571nm, thus hindering the image contrast. The bright spots on the right side of the images are relative to the reflection of the light pulse on the rail of the robotic arm (Figure4.2).

Figure 4 .

 4 Figure 4.24: Example of 2D and 3D frames presenting electrical interference.

Figure 4 .

 4 Figure 4.25: Example of 2D and 3D frames after Intensity Calibration and Confidence Filtering.

Figure 4 .

 4 27 presents the comparison of the Raw and Calibrated 3D

Figure 4 .

 4 Figure 4.26: The target (left) and the final cropped 3D image after TOF Calibration

Figure 4 .

 4 Figure 4.27: Results for the linear movement profile at 10 cm/s, before and after the application of the processing methods.

Figure 4 .

 4 Figure 4.28 summarizes the calibration results for the linear movement profiles at the studied velocities. Note that the measurements successfully follow the respective ground truth for all profiles, although the error remains relatively high.Table4.3 summarizes the evaluation of the measurements in terms of RMSE with respect to the ground truth. The highest incremental improvement was achieved with the Confidence Filtering method, as had been the case of Table4.2. The overall improvement provided by the set of compensation methods is significant, although being far from ideal. This result demonstrates that the performance of the system needs a significant improvement in order to identify features of 20 cm on the target.

Figure 4 .

 4 Figure 4.28: Results for the linear motion profile with respect to the ground truth.

Figure 4 .

 4 Figure 4.29: Results for the accelerated motion profile with respect to the ground truth.

Figure 4 .

 4 Figure 4.30: Sequence of frames demonstrating the random linear motion

  Figure 5.1.

Figure 5 . 1 :

 51 Figure 5.1: Image Processing Workflow

Figure A. 3 :

 3 Figure A.3: Schematics of the Comparator and locking block

Figure B. 2 :

 2 Figure B.2: MPE at the cornea for direct exposure to LASER radiation

.1 adapted from [Roback 2016].

	Parameter	Value
	Max operational range		1.3 km
	Min operational range	Defocus limit Saturation limit	250 m (2 pixels defocus) 100m
	Range precision		8 cm (1-σ)
	Range accuracy		< 35 cm (1-σ)
	Ground footprint (at	Full FOV	7.5m x 7.5m
	430m), normal target	1 pixel (GSD)	5.9cm
	Size	Laser + Detector System Electronics	28Hx34.3Dx33.7W cm 24Hx36Dx33W cm
	Weight	Laser + Detector System Electronics	16.3kg 16.3kg
	Power		450W
	Table 1.1: NASA's FLASH LiDAR (developed by ASC) specifications

Table 1 . 2 :

 12 Typical requirements per mission for time-of-flight cameras

	Range	Accuracy (3-σ)

Table 2 .

 2

1: ROIC specifications

Table 2 .

 2

2: Hypothesis for transmission parameters

  2.

	Operating Wavelength	1.5 -1.7 µm
	Output Pulse Energy (@λ)	≥ 5 mJ
	Pulse Repetition Rate	≥ 20 Hz
	Pulse Duration	≤ 10 ns
	Clock Triggering	External
	Pulse Format	Top-hat
	Weight	≥ 5 kg
	Weight	≥ 5 kg

Table 2 . 3

 23

: LASER's list of specifications

Table 2 . 4

 24

		Operating	Output	Pulse	Pulse	Jitter
		Wave-	Pulse	Repetition	Duration
		length	Energy	Rate
			(@λ)
	AMS Tech-	1571 ± 2 nm	15 mJ	20 Hz	≤ 5 ns	1 ns
	nologies
	Spectra	1550 ± 2 nm	45 µJ	35 -500 kHz	≤ 3 ns	Unknown
	Physics
	IPG	1645 nm	3 mJ	Up to 10	5 ns	Unknown
	Photonics			kHz
	Bright	1550 nm	4 mJ	Up to 1 kHz	≤ 1.5 ns	"Low
	Solutions					Jitter"
						option
	Quantel	1570 nm	8 mJ	20 Hz	≤ 8 ns	≤ 2 ns

: LASER specification from different suppliers

  .1.

	Standard deviation (mV) min max mean
	Before Filtering	5.6	5.9	5.8
	After Filtering	2.2	2.3	2.3

Table 3 . 1 :

 31 Standard deviation of sequence of images before and after electrical noise filtering

Table 3 .

 3

2: Standard deviation before and after calibration (mV)

Table 4 . 1 :

 41 Standard deviation before and after electrical noise filtering (mV)

Table 4 . 2 :

 42 Summary of Calibration Results in terms of Accuracy for V AP D = 0V

Table 4 . 3 :

 43 Summary of Accuracy Results for the linear motion profile V AP D = -2V

Table 4 . 4 :

 44 Summary of Precision Results for the linear motion profileV AP D = -2V

Table 4 .

 4

5: Summary of Results for the accelerated motion profile 4.4.2.3 Random Motion Profiles

Acknowledgements

This thesis is based on the research work developed at CEA-LETI (France) in partnership with ESA-ESTEC (Netherlands

Appendix A

Pixel Schematics

The diagram of this pixel is shown below in Figure A.1. The circuit includes:

• A Capacitive Transimpedance Amplifier (CTIA) with two return capacitors (C 3D for pulse detection and C 2D for load integration). This stage integrates a precharge system and connection switches around capacitors.

• A relative threshold comparator with a dispersion compensation system.

• A logic block for locking the decision at the output of the comparator.

• An analog ramp sampler that serves as a time base for flight time measurement.

• An addressed pixel follower that allows one of its two inputs (2D / Time-of-Flight (TOF)) to be transferred to the column bus. This circuit has some specificities to take into account during its implementation. It uses a common source P-type metal-oxide-semiconductor (PMOS) amplifier (see Figure A.2). The use of this stage allows its operation with a static polarization current lower than the diode current (output for a photodiode n / p). This is necessary to ensure a fast response time for pulse detection over the wide current range from the Avalanche PhotoDiode (APD) (up to 30µA) while limiting static pixel consumption. In return, it is no longer possible in this APPENDIX A. PIXEL SCHEMATICS case to provide the necessary signal excursion with a reset-to-zero integration capacitor by a parallel switch. A precharge system of CTIA has therefore been realized. It is interesting to be able to adjust its threshold as close as possible to the initial level after precharging at the output of CTIA (V init ) in order to detect the pulse as quickly as possible. For this, it is necessary to use for the comparator a relative threshold that compensates for technological dispersion. The latter lead to a variation from one pixel to the other of the initial level at the output of the CTIA and the offset of the comparator. A compensation system has been implemented at the comparator. It compensates for these two effects as well as some of the charge injection phenomena related to precharging. This system samples a relative threshold on a capacitor at the reference input of the comparator. The threshold is established by first starting the auto-zero of the comparator in follower mode. This makes it possible to sample a voltage which superimposes the intrinsic offset of the comparator at the output of CTIA (V init ). In a second step, a small voltage difference (∆V externally controlled) is added. It is this relative voltage difference that sets the effective differential threshold.

Appendix B

LiDAR Risk Assessment

The two main identified hazards for this experiment are the LASER, as the light source for the system operation, and the detector's cooling system that uses liquid nitrogen. This purpose of this appendix is to evaluate those risks and establish mitigation measurements for the experiment preparation and during its execution.

B.1 LASER Safety Calculation and Risk Assessment

The reference text to be followed in the field of LASER safety is the European standard "Safety of LASER products: Equipment Classification, requirements and user guide" referenced NF-EN 60825 (initial document: NF-EN 60825-1 of 1994 and update NF-EN 60825-1 / A2 of 2006). The aim here is to calculate two physical characteristics of the LASER, Maximum Permissible Exposure (MPE) and Nominal Ocular Hazard Distance (NOHD), and to deduce the precautions for a safe LASER usage accordingly. The LASER considered for calculation here is fabricated by the DPLE group, linked to the B. I. Stepanov Institutes of Physics, and distributed by AMS Technologies. Its main properties are described in section 2.2. It operates in the infrared wavelength of 1.571 µm (eye-safe). Although considered a Class IV LASER, an optical system is attached to the output in order to guarantee operation safety. The main parameters used for the safety calculation are presented in Table B.1.

Operating

The objective was to illuminate with a laser a scene located at least 25 m from the system with a spot of more than 2m in diameter while ensuring maximum eye safety. In order to assure this double requirement, the optical systm comprises two lenses as depicted in Figure B.1. The first one has a short 8-mm-focus-length to rapidly diverge the output beam and ensure eye-safety. The second lens adjusts the final divergence for the adequate illumination of the target. The optical system is completely covered with tubes.

The MPE is the level of radiation to which, under normal circumstances, a person may be exposed without suffering adverse effects. Its value depends on several parameters including APPENDIX B. LIDAR RISK ASSESSMENT 10 seconds is as follows, where T is the time of exposure and F max is the maximum repetition rate:

Finally, the last MPE calculation takes into account the average value of a pulse in a pulse train of infinite duration, where N is the total number of pulses in 10 seconds

The most restrictive criterion between these three is the one that must be applied to this evaluation, in this case, MPE = 33.34J • m -2 .

The NOHD is the distance at which the beam irradiance or radiant exposure equals the appropriate corneal MPE. This value is calculated by the following equation:

Where E LASER is the mean energy density, a is the beam output diameter and g is the divergence.

When the laser is not attached to an optical system to expand the beam, the NOHD is 2.2 meters. This result justifies the need for the first lens to expand the beam. From equation B.4 the condition for the safe exposure to this laser beam at any range is:

The optical system was then designed so that a ≈ 5cm at its output, thus ensuring safety for the laboratory personnel.

B.2 Liquid Nitrogen Safety Calculation and Risk Assessment

Liquid nitrogen is a colorless, odorless liquid with a boiling point of -196 °C. At low temperatures the gas / vapor is heavier than air. Small amounts of liquid vaporize rapidly to produce large volumes of gas (1 liter of liquid nitrogen will produce 0.7 m 3 of gas). Nitrogen gas is invisible -the cloudy vapor which appears when liquid nitrogen is exposed to air is condensed moisture, not the gas itself.

Liquid nitrogen represents a risk of asphyxiation where ventilation is inadequate and the nitrogen gas evolved can build up and displace oxygen from the local atmosphere. An atmosphere containing less than 18% oxygen is potentially hazardous and entry into atmospheres containing less than 20% should be avoided. Oxygen deficiency resulting from a large spillage of liquid nitrogen or sudden rapid release of nitrogen gas from a pressurized vessel may be calculated as follows -this is the 'worst case scenario':

Where V R is the room volume and V g is the maximum gas release, which is the liquid volume capacity of the vessel V x gas expansion factor.