Impact of Laser Mode Partition Noise on Optical

Heterodyning at Millimeter-Wave Frequencies Hamza Hallak Elwan 1 , Ramin Khayatzadeh 1 , Tong Shao 2 , Julien Poette 1 , Beatrice Cabon 1 , and Liam P. Barry 2 , Senior Member, IEEE

Abstract-This paper examines the impact of laser mode partition noise (LMPN) during optical heterodyning process. The system presented here employs a passively mode-locked laser diode (PMLLD) as a multi-mode source for millimeter-wave (mmwave) generation. A comprehensive theoretical and experimental study of LMPN at low frequency and, for the first time, in mmwave frequency band is demonstrated. Simulations are presented to investigate the distribution of mode partition noise across optical modes. LMPN appears as an impairment in performance on a radio-over-fiber (RoF) communication system by increasing the intensity noise when the number of optical modes detected by a photodiode (PD) is limited. The effect of LMPN for various number of modes is demonstrated by coherent and incoherent receivers, and it is shown that 25 dB degradation in signal power occurs due to noise when not all modes are detected. Index Terms-Error vector magnitude (EVM), laser mode partition noise (LMPN), millimeter-wave (mm-wave), optical heterodyne, passively mode-locked laser diodes (PMLLD), phase noise, radio-over-fiber (RoF), relative intensity noise (RIN).

I. INTRODUCTION

T HE push towards very high bit rate wireless communi- cation systems requires substantial spectral bandwidth. This has motivated the use of millimeter-wave (mm-wave) frequencies [START_REF] Kleine-Ostmann | A review on terahertz communications research[END_REF] and in particular, the unlicensed band from 57 to 64 GHz [START_REF] Guo | 60 GHz millimeter-wave radio: principle, technology and new results[END_REF], [START_REF] Yong | 60 GHz Technology for GBPS WLAN and WPAN from theory to practice[END_REF]. Wireless communications in this frequency band are employed for short distance or indoor systems due to enormous attenuation in the atmosphere at 60 GHz [START_REF] Smulders | Exploiting the 60 GHz band for local wireless multimedia access: prospects and future directions[END_REF]. Therefore, the prominent interest in microwave photonic generation of mm-wave signals such as optical heterodyning is growing rapidly [START_REF] Jia | Key enabling technologies for optical-wireless networks: optical millimeter-wave generation, wavelength reuse, and architecture[END_REF], [START_REF] Stohr | 60 GHz radio-over-fiber technologies for broadband wireless services[END_REF], where two or more optical modes beat on a high-speed photodiode (PD). An optical fiber is used to transmit the generated optical signals and data from the central station (CS) to the base station (BS), and owing to heterodyne process on PD, the electrical mm-wave signals are generated before distribution to mobile stations.

Optical sources employed in mm-wave radio-over-fiber (RoF) communication system generate optical noise which degrades the system performance. Optical noise predominantly originates from laser dynamics and spontaneous emission that induce fluctuations in both amplitude and phase of the optical source field. For two independent semiconductor lasers with linewidths of several MHz, this exhibits in large phase noise on the RF beat signal. A mode-locked laser diode (MLLD) can be employed for heterodyne mm-wave generation, and thanks to mode locking and correlation of phase noise between the optical lines. This source does not require any external locking scheme to partially alleviate phase noise on RF signals [START_REF] Brendel | PLL-stabilized optical communications in millimeterwave RoF systems[END_REF]- [START_REF] Stohr | Millimeter-wave photonic components for broadband wireless systems[END_REF], which results in better system stability and cost efficient mm-wave generation [START_REF] Braun | Optical microwave generation and transmission experiments in the 12-and 60-GHz region for wireless communications[END_REF]. The influence of phase noise on mm-wave and sub-THz RoF system was investigated [START_REF] Khayatzadeh | Impact of phase noise in 60 GHz radio-over-fiber communication system based on passilevy mode-locked laser[END_REF]- [START_REF] Shao | Phase Noise Investigation of Multicarrier Sub-THz Wireless Transmission System Based on an Injection-Locked Gain-Switched Laser[END_REF]. We also demonstrated the impact of relative intensity noise (RIN) on the generation of mm-wave signals for two different kinds of lasers, and an envelope detector (ED) was proposed to only detect the intensity noise on mm-wave [START_REF] Hallak Elwan | Relative intensity noise in optical heterodyning applied to millimeter wave systems[END_REF]- [START_REF] Khayatzadeh | Impact of amplitude noise in millimeter-wave radio-over-fiber systems[END_REF] and W band signals [START_REF] Khayatzedeh | 100 GHz RoF System Based on Two Free Running Lasers and Non-coherent Receiver[END_REF]. The use of multi-mode sources can lead to larger power fluctuations in each mode due to laser mode partition noise (LMPN) with relatively constant total optical power from source. When filtering a single optical mode of multi-mode laser, LMPN results the in poor system performance for data transmission and communication links [START_REF] Robert | Laser mode partition noise in lightwave systems using dispersive optical fiber[END_REF], but LMPN is neglected for the entire optical spectrum since intensity fluctuations are eliminated on optical lines [START_REF] Ogawa | Analysis of mode partition noise in laser transmission systems[END_REF], [START_REF] Ogawa | Measurements of mode partition noise of laser diodes[END_REF].

In this paper, LMPN is both theoretically and experimentally studied in order to characterize the power fluctuation among optical modes and its impact on mm-wave frequency band. Therefore, a 60 GHz RoF system employing a passively mode-locked laser diode (PMLLD) is investigated with two different receivers used for electrical detection: an RF coherent receiver using a mixer, and an incoherent receiver using an envelope detector. Simulations of mode partition noise are provided to clearly understand the LMPN distribution between different optical modes. Experimental measurements have been realized and are in a good agreement with simulation results. With increase in a number of modes, the LMPN impact decreases, however, the chromatic dispersion instigated due to increase in a number of modes, degrades the performance of 60 GHz RoF system. The influence of LMPN on error vector magnitude (EVM) is evaluated at a data rate of 100 Mbps quadrature phase shifting keying (QPSK) modulation format at a 60 GHz carrier.

This paper is organized as follows: section II presents theoretical analysis of LMPN. In section III, experimental setup, simulation of LMPN, and measured results at both low and mm-wave frequency are discussed. Section IV investigates the transmission experiment and LMPN impact on EVM at 60 GHz, and a conclusion is provided in section V.

II. THEORETICAL ANALYSIS OF LMPN

The principle of generating mm-wave signals through optical heterodyning requires beating of at least two optical tones on a PD with a frequency difference between modes equals to the desired mm-wave frequency. The electrical field E(t) of a multi-mode laser, (M ) modes, can be defined as:

E(t) = M i=1 A i 1 + γ i (t) + ψ i (t) exp j 2π f 0 + ifr)t + φ i (t) (1)
where A i , (γ i +ψ i ), and φ i are field amplitude, total amplitude noise, and phase noise of ith optical mode, respectively. f 0 is the carrier frequency of first mode, and f r is the beating frequency corresponding to the free spectral range (FSR) of laser. The total amplitude noise is sum of relative intensity noise (RIN) expressed from laser rate equations which is represented by (γ i ) and laser mode partition noise (LMPN) expressed by (ψ i ). Since the repetition frequency (f r ) of PMLLD used is 58.63 GHz, the spectral terms of frequency equal or higher than (2f r ) are filtered out by PD of 70 GHz bandwidth. After some simple manipulations, the photodetected current I(t) for (M ) modes can be expressed as:

I(t) = M i,j α I i I j × 1 + (γ i (t) + γ j (t)) + (ψ i (t)+ ψ j (t)) cos 2π(f j -f i ))t + (φ j (t) -φ i (t)) (2)
where I i is DC photodetected current for ith mode, (f j -f i ) is mm-wave frequency of the generated beating between modes i and j, and (φ j -φ i ) is phase noise exhibited on the mmwave carrier. For (i = j), α = 1 and first sum in (2) is the total DC current from sum of each DC contribution from each mode while second sum (γ i + γ j ) and third sum (ψ i + ψ j ) represent RIN and LMPN from DC to a few GHz, respectively. Whenever (i =j), α = 2, first term is sum of all beating terms. Second sum corresponds to RIN contribution of beating term at frequency of (f j -f i ), while third sum is associated to noise partition effect of beating between modes i and j. In (2), second order terms of RIN and LMPN have been neglected. In order to calculate power spectral density (PSD) of the detected photocurrent, auto-correlation function R I (τ ) of the photocurrent is derived, assuming the fluctuations of amplitude noise are stationary [START_REF] Haus | Noise of mode-locked lasers[END_REF]. Using the small angle approximation, R I (τ ) can be defined as:

R I (τ ) = I(t)I(t + τ ) = M i,j=1 M m,n=1 α 2 I i I j I m I n × A i,j,m,n (t, t + τ ) φ i,j,m,n (t, t + τ ) (3)
where is mean value over time, and τ is a positive time delay in realistic system. Whereas each beating corresponds to interaction between 2 optical modes, four indexes are used in this equation to represent the correlations between the beatings of pairs of modes. Consequently, correlation with indexes i, j, m, n is associated with beating between modes i and j and beating between modes m and n. A i,j,m,n (t, t + τ ) and φ i,j,m,n (t, t + τ ) are correlations of amplitude noise and phase noise of beatings produced by modes (i, j) and (m, n), respectively. In case where both amplitude and phase noise of the mm-wave signal are uncorrelated, separate averages can be considered on both amplitude and phase noise. Therefore, the average of amplitude noise correlation can be derived after neglecting the mean value of amplitude noise of a specific mode, γ = ψ = 0. According to laser dynamic, RIN is produced up to a few GHz, limited by the relaxation frequency of laser. Due to coupling between modes, power fluctuation among longitudinal modes causes mode partition noise [START_REF] Robert | Laser mode partition noise in lightwave systems using dispersive optical fiber[END_REF]- [START_REF] Ogawa | Measurements of mode partition noise of laser diodes[END_REF], [START_REF] Govind | Lightwave systems[END_REF]. Therefore, RIN and LMPN can be considered as having independent origins, and subsequently their correlation functions are neglected in this approach γ × ψ = 0. The average of amplitude noise correlation can be expressed as:

Ai,j,m,n(t, t + τ ) = 1 + γi(t)γm(t + τ ) + γi(t)γn(t + τ ) + γj(t)γm(t + τ ) + γj(t)γn(t + τ ) + ψi(t)ψm(t + τ ) + ψi(t)ψn(t + τ ) + ψj(t)ψm(t + τ ) + ψj(t)ψn(t + τ ) (4)
For (i = m and j = n), summation of auto-correlation functions are linked to amplitude noise on individual modes while for all other cases, summation of cross-correlation functions are related to amplitude noise among different modes i.e. beatings. From ( 2), (3), and (4), classical definition of RIN and LMPN of the beating between modes i and j are linked to the parameters ψ i,j and γ i,j using:

RIN ij (t, t + τ ) = 4γ i (t)γ j (t + τ ) (5) LM P N ij (t, t + τ ) = 4ψ i (t)ψ j (t + τ ) (6)
The average of phase noise correlation can be expressed as:

φ i,j,m,n (t, t + τ ) = 1 2 cos(2π(f n -f m )τ ) + ∆φ i,j (t) -∆φ m,n (t + τ ) sin(2π(f n -f m )τ ) (7)
The PSD of the detected photocurrent S I (f ) can be calculated by the Fourier transform of its auto-correlation in (3):

SI (f ) = 1 2 i,j,m,n α 2 IiIjImIn Ai,j,m,n(f ) × δ [2π(fn -fm)] + G A,φ (f ) × δ [2π(fn -fm)] ( 8
)
where δ is the dirac function, and,

Ai,j,m,n(f ) = F T Ai,j,m,n(t, t + τ ) G A,φ (f ) = F T Ai,j,m,n(t, t+τ )× ∆φi,j(t)-∆φm,n(t, t+τ )
When intensity noise and phase noise are considered statistically independant, this expression can be simplified as:

Ai,j,m,n(t, t + τ ) × ∆φi,j(t) -∆φm,n(t, t + τ ) = Ai,j,m,n(t, t + τ ) × ∆φi,j(t) -∆φm,n(t, t + τ ) (9)
It can be observed from ( 8) that the impact of both RIN and LMPN for a multi-mode laser degrades the performance of RoF system at baseband and mm-wave frequencies by generating intensity noise through parameters A i,j,m,n .

III. EXPERIMENTAL INVESTIGATION OF LMPN AT LOW FREQUENCY AND AT MILLIMETER-WAVE BAND

This section experimentally analyzes and examines the impact of laser mode partition noise on baseband and mm-wave frequency band. The electrical power spectrum distribution of LMPN is measured for different number of optical modes using an optical filter. This study is implemented in both (300 MHz-18 GHz) and mm-wave frequency ranges.

A. Experimental Setup

The configuration of the experimental setup for LMPN analysis is shown in Fig. 1. PMLLD generating a low phase noise signal at 58.63 GHz by heterodyning is used. Considering chromatic dispersion effect, optimum length of 72m single mode optical fiber (SMF) G.652 is added to obtain the maximum electrical power of mm-wave signal. For identifying the impact of LMPN, number of detected modes is varied by filtering optical tones using a wavelength selective switch (WSS) employed as a wave shaper 1000S programmable optical filter from Finisar to select limited number of optical tones at point (b). For investigating LMPN at baseband and mm-wave frequencies, two different paths are used. The first detection system (path 1) is dedicated to low frequency analysis in (300 MHz-18 GHz) frequency range as uses a bias tee of (0.3-26.5) GHz bandwidth for suppressing the DC component, and a 40 dB gain amplifier having a 18 GHz bandwidth. The second detection system (path 2) is used for mm-wave measurements where a 19.5 dB gain erbium-doped fiber amplifier (EDFA) after a PMLLD is employed when reducing the number of modes. In path 2, a 35 dB gain amplifier in (50-62) GHz range is used and the amplified mmwave signal is passed to a mixer or an envelope detector. When using the mixer, a local oscillator (LO) at 52 GHz downconverts mm-wave signal to 6.63 GHz. Since an envelope detector only detects intensity, the noise detected is due to sum of RIN and LMPN in a PSD of photocurrent, and thus there is no impact of phase noise on beating signal [START_REF] Hallak Elwan | Impact of Relative Intensity Noise on 60 GHz Radio-over-Fiber Wireless Transmission Systems millimeter-wave radio-over-fiber systems[END_REF], [START_REF] Khayatzadeh | Impact of amplitude noise in millimeter-wave radio-over-fiber systems[END_REF].

In both schemes, Anritsu electrical spectrum analyzer (ESA) is then used to measure the PSD of the photodetected current.

The optical spectrum at point (a) of Fig. 1 is presented in Fig. 2 In experimental measurements, thermal noise of the system and shot noise have been measured, and their contributions have been de-correlated from experimental results because they have different and independent physical origins from LMPN [START_REF] Poette | Highly sensitive measurement technique of relative intensity noise and laser characterization[END_REF]. After measuring S parameters of the opto-electric (O/E) system by a vector network analyzer (VNA), frequency response of detection system from photodiode to ESA has been accounted for in results.

B. Mode Partition Noise Simulation and Distribution

After theoretical analysis of LMPN in section II, the simulation is described here to highlight noise behavior which depends on mode partition within the optical spectrum. The number of beating contributions is (M -1), but LMPN influences all modes, and therefore it possesses (M ) contributions. Simulations are conducted using the Math-Work's MATLAB software. From theoretical analysis of LMPN expressed in section II, there are two contributions:

1) Noise Contribution from Each Mode: In optical comb, each mode (i) has its own contribution (ψ ii ) to the noise induced by mode partition. Its level mainly depends on average optical mode power. The noise due to mode partition on a given mode (i) is compensated by sum of noise (ψ ji ) induced by mode (i) on other modes (j). The summation of (ψ ii ) and (ψ ji ) is therefore equal to zero, as can be described:

ψ ii + M j =i ψ ji = M j=1 ψ ji = 0 (10)
Whenever (ψ ii ) is positive or negative, (ψ ji ) contribution will be opposite in this model. Different distributions of this compensation can be considered: compensation across the entire spectrum (first scenario) or over a limited number of neighboring modes (second scenario).

2) Total LMPN of Mode (i): The influence of other modes (j) produces contributions (ψ ij ) on a given mode (i). From mathematical analysis, the contribution (ψ ii ) represents auto-correlation function of mode partition for mode (i) while the contributors (ψ ji ) and (ψ ij ) correspond to cross-correlation function of mode partition between mode (i) and mode (j), respectively. The total noise ψ i due to mode partition of a given mode (i) is sum of its own contribution (ψ ii ) and contribution (ψ ij ) from other modes (j), expressed as:

ψ i = ψ ii + M j =i ψ ij = M j=1 ψ ij (11)
Table I describes an example of LMPN distribution between 3 optical modes. According to [START_REF] Braun | Optical microwave generation and transmission experiments in the 12-and 60-GHz region for wireless communications[END_REF], sum of terms in each column should equal zero, while from [START_REF] Khayatzadeh | Impact of phase noise in 60 GHz radio-over-fiber communication system based on passilevy mode-locked laser[END_REF], sum of all the terms in each row gives the total mode partition noise of the concerned mode ψ i = 0.

TABLE I DISTRIBUTION LMPN ACROSS THREE OPTICAL MODES

Table I shows that when all modes are detected, the total contribution of mode partition on noise ψ 1 + ψ 2 + ψ 3 equals zero, and consequently no impact of mode partition can be seen. The following simulation parameters have been selected as follows to correspond to the experimental investigation: the number of modes (M ) of the optical spectrum is 41, and non-normalized mode powers I i are considered to exhibit the effects of power spectrum distribution. Fig. 3 represents the experimental measurements and simulation results of LMPN at low frequency where curves describe the cumulative power of LMPN versus an increasing number of detected modes. Curve (1) illustrates experimental measurements of mode partition obtained in (300 MHz-18 GHz) using path 1 of Fig. 1, and simulation results in curves (2) and (3) are performed for a random LMPN distribution. The difference between two simulation curves (2) and (3) in Fig. 3 depends on the distribution of noise: noise from one mode (i) can be distributed over all modes (curve 2) or on a limited number of modes (6 modes here for curve 3).

Curve (2) increases as a function of number of modes to reach a maximum for 15 modes and then decreases to 0 when all modes are aggregated. It is important to notice that accumulation of more than 27 modes modifies the curve slope due to integration of side modes that have a reduced power compared to others central modes. Curve (3) corresponds to a distribution of noise over only 6 neighboring modes. Simulation results shown in curve [START_REF] Guo | 60 GHz millimeter-wave radio: principle, technology and new results[END_REF] where noise is distributed over all modes is in close agreement with experimental measurements of curve [START_REF] Kleine-Ostmann | A review on terahertz communications research[END_REF]. This indicates that all modes, even those having low power like side modes, need to be detected in order to suppress the effect of mode partition. It can be concluded that, in this study, noise is experimentally distributed over all modes.

C. LMPN Effects in (300 MHz-18 GHz) Frequency Range

For measurements of mode partition noise at low frequency, experiments has been carried out using path 1 of Fig. 1. In this case, EDFA is not used, and the total optical power of 9 dBm is used. Fig. 4 depicts the PSD normalized to optical power of detected signal in (300 MHz-18 GHz) for different number of optical modes detected, but displayed up to 7 GHz for better clarity. This normalization is performed to compensate optical power increase when detecting a larger number of modes.

Noise measured for frequencies lower than 300 MHz is due to detection stage, and therefore, it should not be considered here. LMPN is responsible for noise increase as high as 25 dB from 300 MHz to 2 GHz. In this experiment, WSS selects a specific number of optical modes that have different optical powers. Fig. 4, curve [START_REF] Kleine-Ostmann | A review on terahertz communications research[END_REF] shows that the normalized PSD of noise is maximum when detecting only one optical mode. LMPN contribution decreases proportionally to number of modes selected and reaches a minimum when all modes are selected. Fig. 4 reveals the minimum level of LMPN can be achieved and corresponds to the noise floor level of system when all optical modes (41 modes) are in WSS bandwidth.

In mm-wave RoF system of Fig. 1, an EDFA is used to compensate for optical power losses when selecting a small number of modes by WSS. In order to study the effect of amplified spontaneous emission (ASE) noise induced by EDFA, we have carried out experiments with, and without, EDFA at low frequency (path 1 of Fig. 1). The measured PSD is depicted in Fig. 5 with EDFA (curves (1) and ( 2)) and without EDFA (curves (3) and ( 4)). The comparison is made for 2 cases: for only 2 modes, curves (1) and ( 3) and for all modes (curves ( 2) and ( 4)). For 2 modes case, the contribution of mode partition on PSD can be clearly identified by noise increase in frequency range below 2 GHz. It can be noticed in Fig. 5 that the measured PSD results, with and without EDFA, have the same behavior but differ by 19.5 dB corresponding to EDFA gain. Therefore, there is no additional degradation of noise thanks to good saturation of EDFA and a WSS employed after EDFA.

D. LMPN Effects in mm-wave RoF System

An investigation of laser mode partition noise on mm-wave signals is performed. Fig. 1, path 2 describes the schematic diagram of experimental setup using both coherent and incoherent receivers, for a total optical power of 9 dBm. EDFA is used to compensate for optical losses induced by optical filtering performed by the WSS when selecting different number of optical modes.

For coherent receiver, an LO signal of 52 GHz is applied to a mixer for down-conversion as represented in path 2 of Fig. 1. Since phase noise of LO is extremely low (-140 dBc/Hz for frequency offset above 10 MHz) [START_REF] Agilent | D PSG Microwave Analog Signal Generator Datasheet[END_REF], the influence of phase noise of mixing process on IF signal is neglected. Fig. 6 illustrates electrical power spectrum density of received mm-wave signals normalized to mm-wave carrier for various optical bandwidth of WSS. It can be observed from the linewidth of mm-wave signals that phase noise is reasonably constant while the LMPN has different levels. This confirms that intensity noise from mode partition can be observed on mm-wave carrier and degrades the system performance. For non-normalized PSD and with an increasing number of modes, LMPN would slowly increase with increase in power of RF carrier. Measurements in Fig. 6 clearly indicate that LMPN contribution decreases when optical spectrum is extended, and thus the behavior of mode partition noise at mm-wave signal is comparable to that at low frequency as shown in Fig. 4.

When using incoherent receiver, with an envelope detector, the solely amplitude noise impact is measured and investigated. The detected normalized PSD of mode partition noise is depicted in Fig. 7 after detection. Measurements are implemented for an optical photodetected power of 0 dBm with different number of modes in optical spectrum. Due to limited bandwidth of Schottky diode used, results are presented up to 2 GHz. A good agreement with results of Fig. 4 is obtained: when all modes are detected, the impact of noise mode partition is minimal. Here, curve [START_REF] Jia | Key enabling technologies for optical-wireless networks: optical millimeter-wave generation, wavelength reuse, and architecture[END_REF] in Fig. 7 corresponds to a residual amplitude noise which only originates from RIN, which is filtered out by ED response. Similar result can be observed by detecting a signal in a system where no mode partition noise exists, measuring beating between two independent mono-mode lasers, e.g. distributed feedback laser [START_REF] Hallak Elwan | Impact of Relative Intensity Noise on 60 GHz Radio-over-Fiber Wireless Transmission Systems millimeter-wave radio-over-fiber systems[END_REF], [START_REF] Khayatzadeh | Impact of amplitude noise in millimeter-wave radio-over-fiber systems[END_REF]. Fig. 7 clearly shows an increase of mode partition noise detected in mm-wave frequency band when reducing number of optical modes. We can conclude that LMPN also influences amplitude noise on mm-wave signals, as described in [START_REF] Khawaja | Wireless hybrid mode locked lasers for next generation radio-over-fiber[END_REF], and degrades system performance. The heterodyne process induces LMPN in generated signal, even at mm-wave frequency.

IV. DIGITAL MODULATION EXPERIMENT OF 60 GHZ ROF TRANSMISSION

We have then analyzed the effect of LMPN on EVM for 60 GHz RoF system with digital modulation.

A. Experimental Setup

The schematic diagram of experimental setup used to investigate system performance is presented in Fig. 8 for 100 Mbps QPSK modulation at 1 GHz subcarrier onto mm-wave carrier for analysis of LMPN impact on EVM. In the central station (CS), the previous PMLLD is utilized, and the same SMF length of 72 m is used as before. A polarization controller (PC) is added to adjust polarization state of optical signal. A Mach-Zehnder modulator (MZM) biased at quadrature point is employed for applying data stream from an arbitrary waveform generator (AWG). An EDFA is used for compensation of modulator losses and optical power variation when reducing the optical spectrum bandwidth using WSS. WSS is designated to filter out some modes from optical comb and to control transmitted optical power since it can also be used as an optical attenuator. For a fiber-optic transmission, the base station (BS) detects optical signal on a high-speed PD where the heterodyne mm-wave signal is generated at 58.63 GHz. This signal is amplified by an RF amplifier before being detected using ED.

Both down-converted signal and data are captured and sampled by a high-speed digital sampling oscilloscope (DSO) from Agilent 54855A of 6 GHz bandwidth. EVM is then evaluated in on-line from the measurements using a vector signal analyzer (VSA) installed on the DSO. More information can be found in [START_REF] Hallak Elwan | Impact of Relative Intensity Noise on 60 GHz Radio-over-Fiber Wireless Transmission Systems millimeter-wave radio-over-fiber systems[END_REF], [START_REF] Khayatzadeh | Impact of amplitude noise in millimeter-wave radio-over-fiber systems[END_REF], where theory and mathematical equations of amplitude noise were discussed for a system based on an ED.

B. EVM Dependance on A Number of Modes

The measurements are carried out for EVM behavior at 1 GHz subcarrier frequency of 60 GHz carrier. Fig. 9 depicts two cases where 2 optical modes are selected from flatness power modes using WSS (square points), or all modes are detected (circle points). For both cases, WSS also operates as a variable optical attenuator to control the received RF power.

From the measured PSD of mode partition noise using mixer of Fig. 6 and using an ED of Fig. 7, LMPN measurements for 2 modes demonstrate more influence than all modes. Thus, EVM of 2 modes is higher than all modes, as can be seen in Fig. 9. For 2 optical modes detected, the lowest EVM achieved is 8.1 % for a received RF power of -26.7 dBm. Since LMPN depends on an optical power, the same results are observed for other sets of two adjacent modes are considered where the optical lines are flat. When detecting the entire optical spectrum, EVM decreases to 6.2 % at the same received RF power. The EVM trends for 2 modes and entire spectrum are closely congruent. It can again be concluded that for reducing the impact of LMPN at baseband and on 60 GHz signals, it is shown that it is necessary to increase number of optical modes. Another solution could be a careful choice of the frequency subcarrier with respect to generated mmwave carrier to avoid the effect of laser mode partition. When increasing fiber length, chromatic dispersion will induce a time delay between different beatings. This can be interpreted by a de-correlation between optical modes, and consequently noise induced by LMPN will be de-correlated as well. Therefore, this delay degrades the compensation of mode partition noise which can not fully be compensated when detecting several beatings. The frequency limit at which the compensation is possible depends on fiber length and wavelength difference between beatings, and number of beatings should also be considered.

For entire optical spectrum, we encounter a trade-off between laser mode partition noise and chromatic dispersion that both degrade system performance and limit the transmission length.

V. CONCLUSION

We have studied and demonstrated the impact of laser mode partition noise on baseband and mm-wave RoF communication systems. Applications concern PMLLD possessing a low phase noise for generation of a beating signal. The study of LMPN performed on optical heterodyning has been investigated by theoretical analysis and experimental results, where simulations are implemented to describe the behavior of LMPN across the optical spectrum. Impact of LMPN on generated mm-wave carrier has been experimentally demonstrated. Very close agreement is found with measured results.

The experiments have been carried out on mm-wave RoF systems utilizing both coherent and incoherent receivers for down-conversion, and electrical spectrums of LMPN have been presented. EVM versus RF power is measured using data rates of 100 Mbps and QPSK modulation on 1 GHz subcarrier for the case of two filtered modes and the entire optical spectrum. The results conclude that, for a larger number of modes, LMPN impact is lowered. It is worth mentioning that even side modes of low power have a strong impact on LMPN level. Careful attention should be made when designing system to avoid partition noise effect that could degrade signal transmission and considering chromatic dispersion effect. We We would like to highlight that this study and the model can be applicable to any kind of optical heterodyne generation system.

Fig. 1 .

 1 Fig. 1. Schematic of experimental setup for RoF system based on PMLLD, components indicated in dash line are used for mm-wave band.

  (a) measured by an optical spectrum analyzer (OSA) of 0.05 nm resolution bandwidth, showing a very large number of modes with a flat amplitude distribution. The instances of optical spectrum at point (b) are shown in Fig. 2: when filtering two modes in (b.1), when filtering seven modes in (b.2), or when detecting all modes except its left side modes at lower power in (b.3).

Fig. 2 .

 2 Fig. 2. Optical spectrum of PMLLD for different bandwidth of WSS at biased current 209 mA.

Fig. 3 .

 3 Fig. 3. LMPN of measurements and simulation results with different LMPN distribution between optical modes.

Fig. 4 .

 4 Fig. 4. The PSD of LMPN as a function of frequency for a different number of optical modes.

Fig. 5 .

 5 Fig. 5. The PSD comparison between experiments using EDFA and experiments without EDFA for two optical modes and all optical modes.

Fig. 6 .

 6 Fig. 6. The normalized mode partition noise PSD of un-modulated mm-wave signal based on mixer.

Fig. 7 .

 7 Fig. 7. The normalized mode partition noise PSD of un-modulated mm-wave signal after down-conversion based on envelope detector.

Fig. 8 .

 8 Fig. 8. The configuration of experimental setup for 60 GHz RoF communication system based on incoherent receiver.

Fig. 9 .

 9 Fig. 9. Measured EVM as a function of received RF power for filtered two optical modes and entire optical spectrum at 100 Mbps of QPSK modulation.

The authors[START_REF] Kleine-Ostmann | A review on terahertz communications research[END_REF] are with Grenoble Alpes University, Institut de Microélectronique Electromagnétisme et Photonique-LAboratoire d'Hyperfréquences et de Caractérisation, IMEP-LAHC, F-38000 Grenoble, France (e-mail: hamza.hallak-elwan@imep.grenoble-inp.fr; khayatzr@minatec.inpg.fr; julien.poette@phelma.grenoble-inp.fr; beatrice.cabon@grenoble-inp.fr).The authors (2) are with Radio and Optical Communication Lab, Rince Institute, Dublin City University, Dublin, Ireland (e-mail: tong.shao@dcu.ie; liam.barry@dcu.ie).

ACKNOWLEDGMENT

The authors acknowledge that this work was co-funded by Ulysses Project, the French Ministry of Foreign Affairs and International Development (MAEDI) and French Ministry for High Education and Research (MENESR). The authors wish to thank III-V lab for providing the PMLLD chip.


Ramin Khayatzadeh

Hamza Hallak Elwan

Julien Poette

Beatrice Cabon

00 GHz RoF System Based on Two Free Running Lasers and Non-coherent Receiver

Keywords: Envelope detector, Error Vector Magnitude (EVM), Distributed Feedback Laser (DFB), phase noise, Radioover-Fiber (RoF)

A 100 GHz radio over fiber (RoF) system based on two free running distributed feed back (DFB) lasers at the transmitter and a non-coherent receiver is proposed and experimentally demonstrated. Different modulation techniques such as QPSK, 16QAM and OOK for the data rate of more than 1 Gbps have been demonstrated for this RoF system. At the central station, one of the two modes is intensity modulated using external modulator and then the two modes propagate through single mode fiber to the base station. The millimeter wave at 100 GHz is produced by heterodyning of two optical modes at a pin photo-diode. At the receiver, an envelope detector (ED) has been used in order to down-convert data to intermediate frequency (IF). The experimental results show that the impact of phase and frequency instability of the system based on two free running lasers on Error Vector Magnitude (EVM) for the communication system based on non-coherent detection is completely vanished.

I. INTRODUCTION

F UTURE wireless communication systems will utilize carriers at millimeter wave (mm-wave) frequencies to overcome the saturation of different frequency bands in combination with very high bit rates. In this field, W frequency band is attracting more attentions due to its wide available bandwidth (75-110 GHz). There are many applications using W frequency band such as; satellite communication, W band radar research, radar targeting and tracking and also non military applications for high data rate systems ([1]- [START_REF] Kanno | 40 Gb/s W-band (75-110 GHz) 16-QAM radio-overfiber signal generation and its wireless transmission[END_REF]).

Mm-wave signal's generation and delivery difficulties bring research works to use microwave-photonics systems such as RoFs. RoF system suggests optical fiber as a solution for loss and bandwidth limitations of the other propagation mediums at mm-wave frequencies. In RoF systems, the two or more optical modes with the frequency distances equal to mm-wave frequency range are produced at the central station (CS) using lasers, and then transmitted through the optical fiber to the base stations (BS) where utilizing heterodyne receiver the mmwave carrier is generated. Then, from the BSs the wireless delivery of the mm-wave signal to the mobile stations is performed. One of the main issues of the mm-wave frequency generation using heterodyning technique is the phase noise and frequency instability of the generated carrier signal. In order to remove this problem many techniques have been developed. These techniques can be divided into two main groups. The techniques in group one are those that focus on stabilizing the lasers or the transmitter by using phase locked loops at the All the authors are with L'Institut de Microlectronique Electromagntisme et Photonique et le LAboratoire d'Hyperfrquences et de Caractrisation, IMEP-LAHC,3, rue Parvis Louis Nel -CS 50257 -38016 Grenoble, France email: khayatzr@minatec.inpg.fr transmitter to stabilise the lasers' modes frequency distance [START_REF] Brendel | PLL-stabilised optical communications in millimeterwave RoF systems[END_REF] or by using phase mode locked lasers such as passively mode locked lasers [START_REF] Khayatzadeh | Impact of phase noise in 60-GHz Radio-over-Fiber communication system based on passively modelocked laser[END_REF]. The techniques of the second group are those using which one can remove the impact of phase noise from the received signal, for example, by using a non-coherent down-conversion technique [START_REF] Khayatzadeh | Coherent and non-coherent receivers in 60-GHz RoF system based on passively modelocked laser[END_REF] or phase recovery techniques [START_REF] Yi | Frequency offset compensation and carrier phase recovery for differentially encoded 16-QAM vector signal in a 60-GHz RoF system[END_REF].

In this paper, a 100 GHz RoF system based on two free running DFB lasers and a non-coherent detector are presented. Many RoF systems based on coherent detection at W band have been studied recently ([4]- [START_REF] Pang | 25 Gbit/s QPSK hybride fiber-wireless transmission in the W-band (75-110 GHz) with remote antenna unit for in-building wireless networks[END_REF]) but few works contributed to RoF in W band using non-coherent receiver, to our acknowledge. Using a non-coherent receiver at the receiver side, we could successfully overcome the phase fluctuations caused by two free running lasers in this communication system. Furthermore, using non-coherent receiver the down-conversion is done without utilizing any local oscillator at 100 GHz which highly reduces the cost and complexity of the system.

This paper is organized into four sections. Following section I, the experimental setup is given in section II. In section III, the obtained results are under discussion. And Finally, the paper is concluded in section IV.

II. EXPERIMENTAL SETUP

Experimental setup of the proposed RoF communication system based on two free running DFB lasers and noncoherent receiver is illustrated in Fig. 1. In order to generate the signal at 100 GHz in the system presented in this figure, two DFB lasers (# 1 and # 2) are used and the frequency distance between the two modes is controlled by two temperature controllers (TC1 and TC2). An Mach-Zehnder modulator (MZM) is used to modulate the light at the output of laser # 2 externally. The data at IF is applied to the MZM using an Arbitrary Waveform Generator (AWG) that is able to generate signals with different modulation schemes such as; 16QAM, QPSK and OOK. A variable optical attenuator (VOA) is placed before the photo-diode, so that total input optical power of the photo-detector and the electrical power at the input of envelope detector (ED) can be controlled. The optical mode beating is done using a pin photo-diode of 100 GHz bandwidth, which also acts as a filter for higher order mode beatings. The IF is set at 4.5 GHz so the beating between the laser # 1's mode and the upper side band of the signal at the output of MZM will generate a frequency at 95.5 GHz at the output of the photo-diode. At the receiver, a zero bias ED (75-110 GHz), utilizing schottky barrier beam lead diode, is used to down convert the signal from mm-wave frequency to IF. Before the ED a W-band low noise amplifier (LNA) is used to increase the signal level so that the ED can detect the signal.

III. RESULTS AND DISCUSSION

The system performance is verified for various bit rates and received optical powers. For a fixed length of optical fiber, the received optical power is varied by changing attenuation value of the VOA and the EVM values are monitored by plotting them versus different values of the received optical power. Using this technique, one can determine impact of different noise on the performance of the RoF system under test. This technique is well studied in [START_REF] Khayatzadeh | Impact of amplitude noise in millimeter-wave Radio-over-Fiber systems[END_REF], where we proposed a method to determine the origin of the noise which has the most impact on EVM. This method is based on analyzing the trends of EVM versus received optical power using (1) [START_REF] Georgiadis | Gain, phase imbalance, and phase noise effects on error vector magnitude[END_REF]:

EV M = 1 SN R + 2 -2 exp( -σ 2 2 ) 1 P AP R ( 1
)
where SNR is signal to noise ratio, σ is rms value of phase noise and PAPR is the peak to average power ratio for the considered modulation scheme. In an RoF system the SNR can be modelled as follows:

SN R = R 2 P 2 opt 4kT F ∆f R L + 2qRP opt ∆f + 10 RIN/10 R 2 P 2 opt ∆f (2)
where k: Boltzmann constant, T: temperature, ∆f : received signal bandwidth, F: amplifier noise figure, q: electron charge, Fig. 2. EVM versus received optical power for RoF system at 100 GHz and 16QAM modulation scheme.

P opt : received optical power, RIN: relative intensity noise and R L is the load resistance. By substituting SNR to [START_REF] Essen | SUMATRA, a W-band SAR for UAV application[END_REF], one can model the EVM for an RoF system. Fig. 2 shows the trend of EVM values versus received optical power for different scenarios. The EVM values are measured at 200 Mbps (square points) and 1 Gbps (circle points) for 16QAM modulation scheme and IF of 4.5 GHz. It can be seen that for the same received optical power, the EVM value for 200 Mbps (-17 dB for -0.5 dBm optical power) is much lower than the EVM value for 1 Gbps (-14 dB for -0.5 dBm optical power). The reason the EVM value increases is that for measuring EVM for the signals with higher data rates we need to increase the bandwidth of the VSA while receiving the same optical power which reduces SNR. For example, for data rate of 200 Mbps the VSA's bandwidth is set to 200 MHz while for 1 Gbps its bandwidth is set to 500 MHz. As can be seen in this figure, the models with no phase noise impact (phase noise variance σ = 0) well match the measurement results for both 200 Mbps and 1 Gbps data rates. By setting σ to 20 × 10 -3 rad and 30 × 10 -3 rad, which correspond to rms phase jitter of a signal generator with broadband phase noise equals -120 dBc/Hz over 200 MHz and 500 MHz bandwidths [START_REF] Kester | Converting oscillator phase noise to time jitter[END_REF], one can see that for the same received optical power the EVMs are increased to much higher values, especially for large values of received optical power. This shows that using the non-coherent receiver in this RoF system we could successfully overcome the phase noise impact on performance of the system. In this figure, the star point line shows the EVM evolution versus the received optical power when there is no phase and no optical (RIN, shot) noise impact on the received signal. In this case, the only noise that impacts the performance of the system is the receiver thermal noise.

The constellation diagram of the detected signal is presented in Figure 3 (A) for 16QAM modulation scheme with 200 Mbps and (B) QPSK modulation scheme with data rate of 1 Gbps. Observing the constellation diagram, we can see that the phase noise impact on the received symbols are vanished using noncoherent receiver, as it is expected. The system performance is also tested for OOK modulation scheme. The eye diagrams of the detected signal are presented in Figure 4 IV. CONCLUSION In this paper, we propose a 100 GHz RoF communication system based on two free running DFB lasers and a noncoherent receiver. Different digital modulation schemes such as 16QAM, QPSK and OOK are successfully tested for the data rates of more than 1 Gbps using the proposed communication system. The phase noise impact on the performance of the system is completely vanished using non-coherent receiver. The simplicity of the proposed communication system comparing to the systems based on coherent detection, in which different stabilisation techniques are needed to stabilize the mm-wave carrier, is a further strength point of this system.

Fig. 1 .

 1 Fig. 1. Experimental setup of RoF communication system based on two DFB lasers and non-coherent receiver, TC: Temperature Controller, PC: Polarization Controller, MZM: Mach-Zehnder Modulator, AWG: Arbitrary Waveform Generator, VOA:Variable Optical Attenuator, LNA: Low Noise Amplifier, ED: Envelope Detector, DSO: Digital Storage Oscilloscope.

  for different data rates (A) 100 Mbps (B) 400 Mbps (C) 1 Gbps (D) 2 Gbps. The results are showing well open eye upto 1 Gbps. By increasing the data rate to 2 Gbps, the eye opening is reducing which is caused by the limited bandwidth of the ED.

Fig. 3 .

 3 Fig. 3. Constellation diagram of the detected signal (A) 200 Mbps 16QAM (B) 400 Mbps QPSK.

Fig. 4 .

 4 Fig. 4. Eye diagram of the detected signal (A) 200 Mbps (B) 400 Mbps (C) 1 Gbps (D) 2 Gbps.


Pierre Biasutti

Jean-François Aujol

Mathieu Brédif

Aurélie Bugeau

Diusion and inpainting of reectance and height LiDAR orthoimages

This paper presents a fully automatic framework for the generation of so-called LiDAR orthoimages (i.e. 2D raster maps of the reectance and height LiDAR samples) from ground-level LiDAR scans. Beyond the Digital Surface Model (DSM or heightmap) provided by the height orthoimage, the proposed method cost-eectively generates a reectance channel that is easily interpretable by human operators without relying on any optical acquisition, calibration and registration. Moreover, it commonly achieves very high resolutions (1cm 2 per pixel), thanks to the typical sampling density of static or mobile LiDAR scans. Compared to orthoimages generated from aerial datasets, the proposed Li-DAR orthoimages are acquired from the ground level and thus do not suer occlusions from hovering objects (trees, tunnels and bridges), enabling their use in a number of urban applications such as road network monitoring and management, as well as precise mapping of the public space e.g. for accessibility applications or management of underground networks. Its generation and usability however faces two issues : (i) the inhomogeneous sampling density of LiDAR point clouds and (ii) the presence of masked areas (holes) behind occluders, which include, in a urban context, cars, tree trunks, poles or pedestrians (i) is addressed by rst projecting the point cloud on a 2D-pixel grid so as to generate sparse and noisy reectance and height images from which dense images estimated using a joint anisotropic diusion of the height and reectance channels. (ii) LiDAR shadow areas are detected by analysing the diusion results so that they can be inpainted using an examplar-based method, guided by an alignment prior. Results on real mobile and static acquisition data demonstrate the eective-

Introduction

Orthophotographies and Digital Surface Models (DSM), dened respectively as the color and ground height orthoimages (i.e. raster maps dened on a regular horizontal grid), are ubiquitous products in modern cartography. They are widely used in many application elds such as remote sensing, geographical information and earth observation, mapping and environmental studies. Such orthoimages are traditionnaly computed from an aerial perspective (satellites, planes and more recently unmanned aerial vehicles (UAVs)). Although aerial imagery approaches provide a very well known and common approach to the problem of orthoimage generation, they may be limited in terms of accuracy and resolutions and they certainly suer from occlusions caused by the natural and urban environnement such as trees, tunnels, overhangs or tall buildings (Fig. 1.a).

These limitations prevent orthoimages generated by above-ground datasets to be used for a whole new set of applications that rely on a precise mapping of the ground and which cannot suer from such large occlusions. These applications include, mostly in a urban context, accessibility assessment for soft mobilities (disabled, wheelchairs and strollers) and itinerary computations [START_REF] Serna | Urban accessibility diagnosis from mobile laser scanning data[END_REF], precise mapping of road marks [START_REF] Hervieu | Road marking extraction using a model&data-driven RJ-MCMC[END_REF], road limits or curbs (29; 19; 51; 23), road inventory [START_REF] Pu | Recognizing basic structures from mobile laser scanning data for road inventory studies[END_REF], road surface modelling and quality measurements [START_REF] Hervieu | Road side detection and reconstruction using LIDAR sensor[END_REF], mobile mapping registrations on aerial images [START_REF] Tournaire | Towards a sub-decimetric georeferencing of groundbased mobile mapping systems in urban areas: Matching ground-based and aerial-based imagery using roadmarks[END_REF] or image based localization using ground landmarks [START_REF] Qu | Vehicle localization using monocamera and geo-referenced trac signs[END_REF]. Moreover, recent legislations in European Figure 2: Full orthoimage production pipeline from MLS. Framed rectangles are processing steps, arrows are exchanged data. h 0 and u 0 are the projections of each point height and reectance respectively onto an horizontal grid. M proj is a binary mask of pixels where at least one point was projected.

countries call for a subdecimetric accuracy mapping of underground networks (water pipes, gaz pipes, internet wires and phone wires) as the lack of accurate data has lead to accidents and delays in many public works. Very high resolution orthoimaging with limited occlusions could help in meeting the requirements of these legislations as it would provide sub-centimetric accuracy mapping of the ground.

To maximize orthoimage resolution and to minimize occlusions, we propose to leverage ground-level LiDAR scans acquired by Mobile Laser Scanning vehicles (MLS) or from xed stations. The proximity of the acquisition ensures a high resolution as well as a diminution of occluded areas. As in [START_REF] Vallet | Road orthophoto/DTM generation from mobile laser scanning[END_REF], we propose to derive a gray image from the reectance attribute of the LiDAR samples (which measures backscattered energy) instead of relying on optical imagery (which would introduce diculties in dynamic environments and require precise co-registration) to produce sub-centimetric orthoimages.

Problem statement and related works

The projection of a ground-level point cloud at centrimetric resolutions creates a sparse image due to its inhomogeneous sampling density (Fig. 1

.b).

The problem of lling in the holes created by the lack of information in the raster image created by the projection of a point cloud has been scarsely investigated [START_REF] Vallet | Road orthophoto/DTM generation from mobile laser scanning[END_REF]. This problem is strongly related to DSM generation from LiDAR data, especially airborne LIDAR data, which has been widely studied over the past decades [START_REF] Chen | State-of-the-art: DTM generation using airborne LIDAR data[END_REF]. DSMs are mostly represented either by Triangular Irregular Network (50; 20; 14) or by raster images (27; 45; 41; 13). In both cases, the main challenges of DSM generation still remains ground point ltering and interpolation. Ground point ltering aims at removing points that belong to the ground (e.g. earth surface in case of airborne DSMs) from points that belong to elevated structures (such as buildings, trees, cars, fences or poles). For airborne LiDAR data, it is done by dening slope operators in order to follow the ground surface (49; 42; 25). However, these methods are developped in order to extract the ground on large scale, for terrain with high relief variation. In the case of urban scenarios, these methods fail to distinguish correctly small objects (bikes, pedestrians) from the ground. DSM interpolation approaches depends on the nal product representation. For TIN, the interpolation is either done by Delaunay Triangulation [START_REF] Xiaobo | A new study of delaunay triangulation creation[END_REF] or by plane tting, as proposed in [START_REF] Bitenc | Evaluation of a lidar land-based mobile mapping system for monitoring sandy coasts[END_REF]. However, these approaches are not relevant to our problem as we aim at generating orthoimages as well as raster DSMs. The interpolation of raster images DSMs has already been done by coarse to ne interpolation approach [START_REF] Kraus | Advanced DTM generation from LIDAR data[END_REF], by using moving least squares in [START_REF] Shan | Topographic laser ranging and scanning: principles and processing[END_REF] and more recently using Poisson interpolation [START_REF] Pérez | Poisson image editing[END_REF] to deal with the high variations of density in the raster images derived from MLS [START_REF] Vallet | Road orthophoto/DTM generation from mobile laser scanning[END_REF]. The generation of DSMs does not require to preserve textures as the surface model is textureless. However in our context, we aim at generating orthoimages from the reectance as well as DSMs. Therefore, the preservation of texture is a key point of our problem, which requires to use other approaches for the raster image interpolation.

Contributions

We propose a framework that combines diusion inpainting and examplarbased inpainting for the joint production of high resolution and photo-realistic reectance and height orthoimages. Our main contribution is to provide a fully automatic framework with an ecient set of default parameters. First, we introduce a novel anisotropic diusion technique that uses both texture and height information. Second, we propose an examplar-based inpainting method that is modied in order to take advantage of the specic attributes present in LiDAR datasets.

Outline of the paper

In this paper, we propose in Section 2 a novel approach for the reconstruction of ground orthoimages that produces very satisfying results in terms of visual quality and coherence. We rst explain in Section 3 how the point cloud can be projected on a 2D-pixel grid after ltering ground points. We present a brief state-of-the-art on diusion algorithms and we introduce a diffusion model for the computation of dense images from the sparse projections in Section 4. In Section 5, we present dierent examplar-based inpainting techniques before introducing an inpainting method that takes height information into account as well as assumptions about the alignment between structures. Finally, in Section 6, we validate our framework by presenting various examples of reconstruction made on real acquisition data.

Framework description

Orthoimage generation from LiDAR scans aquired at ground level as been scarcely studied in the past. Nevertheless, the relation between LiDAR reectance and optical acquisition has already been used for dierent applications such as depth map generation from point cloud [START_REF] Bevilacqua | Joint inpainting of depth and reectance with visibility estimation[END_REF], which shows the correlation between both modalities. [START_REF] Vallet | Road orthophoto/DTM generation from mobile laser scanning[END_REF] rst extract the ground points by considering the lowest points projected in each pixel. Next, they perform Poisson interpolation to connect sparse pixels. In this paper, we introduce an ecient and fully automatic pipeline to reconstruct an aerial image from a LiDAR point cloud. The proposed framework is summed up in gure 2.

From the point cloud, we need to extract the ground points (eg. points that do not belong to a mobile object or to an object that is lying on the ground). This is done by computing an envelop Γ (see section 3). The reectance and height values of these ground points are then projected in two 2D-images respectively: u 0 and h 0 . This projection is done by removing the z (height) coordinate and rounding the coordinates to the chosen resolution. We also build a mask M proj of the pixels where at least one point was projected.

At this point, the projections u 0 and h 0 are sparse as they do not cover all the pixels of the images. Figure 3 presents an example of the dierent kinds of missing pixels that results from the projection. Some parts of the projection correspond to the inside of a building (gure 3.c in orange), under sampling holes appear in between lines of acquisition (gure 3.c in blue) and an occlusion is caused by a pole blocking the laser beams (gure 3.c in green). In order to reconstruct the missing information of the orthoimage, we rst perform diusion on both u 0 and h 0 by coupling reectance and height in an anisotropic diusion algorithm in order to remove holes due to undersampling. The resulting images are respectively called u d 0 and h d 0 . After this step, there are still some large holes remaining. Their locations M o cc are retrieved through mathematicl morphology. Finally, we can reconstruct occlusion holes using an examplar-based inpainting method that uses both reectance and height information, as well as an assumption about the alignment between structures to inpaint.

Projection of LiDAR point cloud

The projection of a point cloud onto a 2D pixel grid is a typical discretization problem. It mostly requires to dene a mapping between the point cloud metric frame and the 2D-pixel grid. However, in the case of Digital Terrain Model, it is also needed to lter out o-ground points (trees, urban structures, cars). We introduce a novel approach for ground point ltering in section 3.1 and explain how the projections are done in section 3.2. More details about the parametrization of the projection can be found in 3.3.

Filtering ground points

The denition of ground-points in a point cloud can be tedious as we have to lter groups of point that represent relatively planar structures and which do not belong to any other objects than the ground itself. Ground ltering is a typical DTM generation problem [START_REF] Meng | Ground ltering algorithms for airborne LiDAR data: A review of critical issues[END_REF]. Traditional aerial DTMs generally model the scene at large scale. In order to correctly include details of urban scenes (pavements, steps or any lightly elevated structure that belongs to the ground), it is necessary to model the ground at a nest scale. In urban scenario, plane tting is often use as primary ground segmentation. Although it allows a fast and simple estimation of ground points, considering horizontal planes relatively to the acquisition system can be ambiguous. Indeed, modern MLSs tend to be accurate enough to acquire ceilings through windows, creating false positives. Vertical planes are also relevant (pavements, stairs), but not in every cases (trucks, billboards). This problem has been investigated by considering it as a classication problem [START_REF] Rottensteiner | A new method for building extraction in urban areas from high-resolution LIDAR data[END_REF] or by performing advanced structural analysis (27; 9). However, these solutions have shown their limitations when the scene presents high diversity of objects. In particular, they lack of precision when aiming at estimating the boundaries of the ground in urban scenes because other objects (cars, ceilings) are often considered as the ground as they share common structural properties.

We propose a novel approach for ground point ltering based on the way the acquisition is done. We aim to lter out hovering object or any point that is over another one. As the points are acquired with a certain margin of error, direct comparison is not suitable as the likelyhood of two points having the exact same (x, y) coordinates is negligible. We rst create an empty envelop of the size of the projection where each pixel has an innite value. This envelop will help dening the boundaries of the 2D region that represents the ground while ensuring that all the points that fall into the envelop really are ground points. We then consider segments made by each point and its relative emission point. Each segment is discretized in the envelop using the Bresenham line algorithm [START_REF] Bresenham | Algorithm for computer control of a digital plotter[END_REF]. As the beam is perfectly straight, we can estimate the height of the segment at any position of the segment. Each pixel is then lled with the lowest height value of segments that cross it. Note that in our case, only points below the sensor are considered. This reduce the amount of data to process while ensuring to discard only o-ground points. However, this is only suitable for MLS in urban scenarios. Figure 4 shows a slice of the maximal envelop Γ computed on a set of beams that overlaps. We can see that for every overlapping beams, only the portion of the lowest one is kept in the envelop. Finally, we lter the point cloud by taking only points that are under the envelop and the threshold, with an epsilon margin.

Sparse projections

Using the ltered point cloud, we want to produce two sparse images corresponding to the reectances and the heights in the sensor frame: u 0 and h 0 dened on the mask M proj . The values for each pixel in u 0 is the mean of the reectances of every points that is being projected in it. The values in h 0 are the same using the height in the sensor frame. Finally, an M proj image is produced where pixels are valued 1 where at least one point was projected and 0 elsewhere. Note that at high resolution (1px per square centimeter), the use of the mean is relevant on our data as only few points (less than 5) project in each pixel. However, if the amount of points that projects in each pixel increases a lot (when working at lower resolution for example), one can consider using the median instead of the mean to remove outliers. Note that the computational cost of the median is higher than the cost of the mean. Thus, its use will signicantly increase the running time of the projection step.

Parameters

The choice of the mapping between real coordinates and pixels mostly depends on the density of the point cloud. In our case, with an acquisition done using a RIEGL LMS-Q120i which produces 300 000 points per second, the maximal acceptable resolution was 1px = 1cm is arbitrary but in the case of a urban scenario, it should be kept under the height of the aquisition vehicle. More details about the parameters are provided in 6.1.

Dependency to the sensor

It is important to point out that the type of missing data are directly related to the chosen resolution as well as the type of sensor. The holes due to the acquisition sampling are less likely to appear when chosing a lower resolution. Moreover, the missing values in between acquisition lines are specic to the sensor mentionned above. They are quite homogeneous and create a regular pattern. With a panoramic sensor such as the one used in (1), the missing pixels will appear in a random pattern, but will create a more dense image for the same resolution, which makes our pipeline still suitable for this type of data.

Diusion of sparse images

The two images obtained in the previous section are sparse in the sense that they do not cover every pixels of the DTM. Therefore, we need to interpolate the images in order to get a dense representation of them. The goal is to ll in gaps between relatively close pixels that are due to the acquisition undersampling. In this section, we rst explain what are the requirements that the lling method needs to meet. Then we introduce a modication to existing methods in order to enhance the results. Finally we show a comparison of dierent methods to validate our proposed modication.

Choice of the approach and requirements

A typical approach for lling small holes by interpolation is to use diusion algorithms. Several diusion techniques exist such as the total variation [START_REF] Chambolle | A rst-order primal-dual algorithm for convex problems with applications to imaging[END_REF], the generalized total variation (8), structure tensor diusion (47; 5) or partial dierential equation diusion (2) and extended to multi-modal data [START_REF] Zhuang | Fast hyperspectral image denoising and inpainting based on low-rank and sparse representations[END_REF].

Here, we focus on iterative solving methods which are more exible. A basic diusion algorithm is the so called Gaussian diusion which is an isotropic technique that consists in updating the image with its own Laplacian [START_REF] Koenderink | The structure of images[END_REF]. However in the case of a urban scenario, an anisotropic diusion is more relevant as very high gradients appear at the edge of dierent structures (roads, pavements, stairs) and need to be preserved.

The Perona-Malik algorithm ( 32) is a well known algorithm for anisotropic diusion. It is partially inspired from the Gaussian diusion and is dened as follows:

   ∂u ∂t -div(c(|∇u|)∇u) = 0 in Ω × (0, t) ∂u ∂N = 0 in ∂Ω × (0, T ) u(0, x) = u 0 (x) in Ω (1)
where u 0 ∈ Ω is the input image, div is the divergence operator, ∇ is the gradient operator, N is the normal vector to the boundary of Ω and c is an increasing function. A common choice for c is the weighting function

c(|∇u|) = 1 √ 1+(|∇u|/α) 2 , α
being a weighting factor that quanties how much the gradient information needs to be considered. This technique ensures the preservation of edges while ensuring smooth transitions between sampled scan lines. Nevertheless, this technique only takes into account the gradients of a single channel. In our context, the diusion needs to be blocked in case of a high gradient in the reectance image as well as in the case of a high gradient in the height image that could correspond to the junction between the road and a pavement, or steps of stairs. Therefore, we need to modify equation (1) in order to take both channels into account.

Our proposed algorithm

We propose here a modication to the Perona-Malik equation ( 1) by coupling heights and reectances as follows, using previously introduced notations:

               ∂u ∂t -div(f(|∇u|, |∇h|)∇u) = 0 in Ω × (0, t) ∂h ∂t -div(f(|∇u|, |∇h|)∇h) = 0 in Ω × (0, t) ∂u ∂N = 0 in ∂Ω × (0, T ) ∂h ∂N = 0 in ∂Ω × (0, T ) u(0, x) = u 0 (x) in Ω h(0, x) = h 0 (x) in Ω (2)
where we recall that u 0 is the reectance image and h 0 is the height image. We introduce the new weighting function f that emerges from the one used in equation ( 1) as follows:

f (|∇u|, |∇h|) = 1 1 + |∇u| 2 α 2 + |∇h| 2 β 2 (3)
having α, β as weighting constants quantifying how gradients of reectance and height need to be considered. The choice of coupling both reectance and height information into the same model is motivated by the fact that reectance and height gradients are not always at the same locations and therefore, are complementary. Note that coupling various modalities in a model has already been proposed in (3) for coupling multi-spectral images, however in that case authors present a model specically designed for merging multiple images representing the same object at dierent wavelengths. Using our method, we can now take into account gradients coming from both u 0 and h 0 .

Comparison with other diusion techniques

In this section, we propose an evaluation of the performances of our model against gaussian diusion and closest neighbors diusion. Projecting a point cloud acquired at very low speed provides a dense image locally. Therefore, we can dene a ground truth using this region of the projection. We dene a set of 20 masks of same dimension as the ground truth and we randomly set 80% of the pixels to 1. For each method and each mask, we recover pixels of the ground truth where the mask is valued 1, using the rest of the image. Note that the percentage of missing pixel (here, 80%) is dened as the average missing pixels ratio of our dataset. Finally, we compute the average of classical similarity metrics (MSSIM and MPSNR, which are respectively the mean of the SSIMs and the mean of the PSNRs) for each methods on the reconstructed images compared to the ground truth. The results are summed up Table 1 in which we can see that our method outperforms the two other diusion methods. Figure 6 presents one set of results. We can see that the Gaussian model as well as our model better succeed in recovering the aspect of the original image. Our method outperforms the Gaussian diusion by recovering sharper edges.

Parameters

In practice, the proposed diusion technique was implemented by solving the PDE system with a rst order explicit Euler scheme with respect to the time variable. The number of iterations has to be chosen in order to ll in stripe holes. It depends on the chosen resolution as very sparse images will require more iterations to fully ll the image. Moreover, a good speed-up can be obtained by using the result of the closest neighbors diusion of both u 0 and h 0 as the initialization for the proposed model as it drastically lowers the number of required iterations. The weighting term for the reectances α should be higher than the one for height β in order to completely block the diusion in case of large height variation while connecting close pixels. Practical details will be given in section 6.1. Note that only unknown pixels regarding M proj should be updated to prevent an oversmoothing of the nal images.

Inpainting of occlusions

After the projection, some holes are not only caused by some undersampling but also by the beam being blocked by an object (cars, poles, lights, pedestrians or bikes) before reaching the ground. This leads to a ground projection with a lot of information at the edge turned toward the sensor, but nothing when going further. As occlusion holes are wider than stripe holes, the diusion algorithm proposed above is not suitable in order to reach a visually satisfying result. In this section, we rst see how occlusions holes are detected in the image. We then present the problem of texture synthesis in our case and we give a rst solution. Finally, we introduce an improvement to this solution based on assumptions made on the urban scenario.

Occlusion hole detection

The occlusion detection consists in dening which holes are caused by the sampling rate and which holes are caused by a blocking of the laser beams. This can be done by applying mathematical morphology on the projection mask M proj before diusion where each known pixel is valued 1 and all other pixels are valued 0. At this point, everything with the 0 value is considered as occlusion holes.

Having M proj , a simple morphological operation known as closing ( 40) is enough to detect occlusions and build the occlusion mask M occ . The closing consists in applying a dilation of a certain radius to the mask and then to apply an erosion of the same radius. This leads to a closing of small 0-labelled areas surrounded by 1s. Choosing wisely the radius of the closing ensures that undersampling holes are eliminated while preserving the shape and the position of the occlusion holes.

Unfortunately, the resulting mask does not consider the boundaries of the scene, and tends to extend further. We recall that when projecting the point cloud (Section 3), a Γ envelop is computed in order to dene the boundaries of the scene. Thus, we consider the intersection of the computed mask and the Γ envelop to prevent the mask from expending outside of the ground region, typically inside of buildings or in regions too far from the sensor (Figure 3).

Examplar-based inpainting

Among the variety of dierent inpainting algorithms, examplar-based algorithms are known for being more eective and more reliable in lling large areas (with large internal radius). Examplar-based inpainting consists in trying to nd the best candidate in the known region of the image for the patch centered on a pixel lying on the border of the hole. Once found, the candidate is used to ll the unknown part of the image by copying the color in its central pixel [START_REF] Efros | Texture synthesis by non-parametric sampling[END_REF] or the full patch [START_REF] Criminisi | Region lling and object removal by exemplar-based image inpainting[END_REF]. The operation is repeated until the hole is fully closed. More recent approaches, such as (17; 46) reconstruct the texture using both color information and depth information. However these algorithms require dierent acquisitions of the same view, which is not applicable in our case as we aim at performing the reconstruction on a single acquisition pass.

The urban scenario presents a huge variety of structures (roads, pavements, stairs, gutters) as well as many dierent textures (roads, cobbles, oor tiles). Thus, we decided to base our work on the Criminisi et al. [START_REF] Criminisi | Region lling and object removal by exemplar-based image inpainting[END_REF] algorithm that was designed for the good preservation of the structures in the reconstruction. More complex approaches exist that rely on the work presented in ( 16) such as ( 11) and ( 28) however it would have been less intuitive to adapt them to our context. In [START_REF] Criminisi | Region lling and object removal by exemplar-based image inpainting[END_REF], authors put forward the idea that the order in which areas are reconstructed have a high impact in the nal result. They introduce a priority term that takes into account the strength and the direction of the image's gradient at the border of the unlled area. A patch that contains a strong gradient in the direction orthogonal to the border of the region to reconstruct is evaluated before more uniform patches.

Modication to the original algorithm

Coupling reectances and heights. The algorithm presented in ( 16) oers a very good technique for region lling. However, it can fail when the area to ll is very large. Therefore, we introduce a modication to the algorithm by taking the height information into account as a guide for the reconstruction. The idea is to use the height information to restrain the selection of best candidate patches to the areas of similar height by computing the Sum of Squared Dierences of the candidate patch in both the reectance and the height images. The SSD (Sum of Squared Dierences) is dened as follows:

SSD(P 1 , P 2 ) = i,j∈Ω (P 1 (i, j) -P 2 (i, j)) 2 (4) a. b.
c.

Figure 7: Comparison of ( 16) and our proposed modication on the junction between the road and a pavement. (a) is the original unlled image where the dark region is being reconstructed using examplar-based inpainting, (b) the result from ( 16), (c) our proposed optimization. The result is clearly better in (c) as the reconstruction conserves the structures of the image without creating new artifacts such as the one appearing on the left of (b).

having P 1 , P 2 the two 2D-patches that are compared and Ω the domain of denition of the image. In our modication, and for each candidate, a score is attributed by combining both channels as follows:

S p (P t , P c ) = SSD(P R t , P R c ) + η × SSD(P H t , P H c ) (5)
where P t is the target patch to be lled and P c is a candidate patch. P c can be any patch in the image that has no pixel that belongs to an occlusion hole. However, for speed-up purpose, we can limit the selection of P c to be in a certain radius around P t . η is a regularization parameter and the superscripts R, H denote that the patch is taken in the reectance image or the height map respectively. The regularization parameter only appears for the height map SSD as the height information is less important in order to reach a visually satisfying result. The impact of the use of the height map in the synthesis is very noticeable in gure 7. The structure of the road is well preserved using the proposed modication compared to the original algorithm in which artifacts appear after some iterations. These artifacts mislead the reconstruction and the result is visually incoherent.

Taking advantage of urban environment. Although the current modication of the algorithm provides a very good solution for lling occlusion holes, the reconstruction can fail sometimes when the hole is very large. This happens for holes that are caused by cars or trucks where the area to reconstruct is signicantly larger than regular holes (10 6 pixels at a 1px = 1cm 2 resolution for a standard car and the portion of pavement behind it) and it can become a common issue. Indeed, at the center of the holes the nearest known information is too far away and the error accumulated along the iterations is likely to fail the reconstruction. To improve the results in the concerned areas, we advocate that the structure of a urban environment is very likely to evolve in a similar way to the vehicle path as illustrated in gure 8. Therefore, we can constrain the selection of candidates to patches that are at a similar distance to the sensor than the current patch. The range attribute of the LiDAR image provides this information for each point.

We dene the new score equation as follows, using previously introduced notations:

S f (P t , P c ) = 1 + |d(P t ) -d(P c )| γ 2 × S p (P t , P c ) (6)
having d(P ) the distance between the sensor and the center of the patch P and γ a regularization parameter that constrains the selection of patch to a range interval around the current range. The range can be accessed everywhere in the image by precomputing a signed distance map of the area to the path of the vehicle (eg. where the range is the lowest).

Large patches and artifacts. When the reconstruction is done at a very high resolution, large patches (10 3 px) are likely to be required in order to correctly represent the structural elements of the image. This might lead to abrupt junctions between reconstructed patches. Therefore, we propose to enhance the copy of the patch by performing the seam carving using graphcuts presented in [START_REF] Rubinstein | Improved seam carving for video retargeting[END_REF]. The goal is to compute the optimal cut between P t and P c where they overlap to obtain a seamless result.

Parameters

η should be kept under 1 to ensure the visual coherence of the reconstruction. Parameter γ depends on the size of the occlusion. When γ = 1, the regularization is very strong and the selection of the candidate patch is constrained on a narrow band of same distance to the sensor point. When the value of the parameter is highly increased (γ > 10 4 ), no regularization operates and the algorithm behaves as if the range was not taken into account. Therefore, one can alternate between these two values for γ depending on the internal radius of the occlusion (see next section).

Results

We conclude this paper by presenting dierent results obtained using the proposed framework. We rst present a general set of parameters for an automatic reconstruction of a set of orthoimages. We then demonstrate the eciency of the solution by showing various results and comparison to existing methods. After that, we validate the quality of the framework using numerical criterions. Finally, some details about the computation time are drawn.

Parameters

In the same way as other pipelines, this one comes with a set of parameters that was used for producing every images displayed in this paper.

Projection. The objective of this study was to provide very high quality orthoimages. Therefore, all reconstructions were done at the maximal possible resolution of our sensor: 1px = 1cm 2 . A threshold of 60cm from the road level was used to lter out points after the computation of the envelop.

Diusion. For the diusion step, we found the best balance of results by setting α = 5, β = 0.7 with 3 iterations and by rst interpolating u 0 and h 0 using the nearest neighbor algorithm.

Mask extraction. In this step, a closing radius of 6px was enough to ll stripe holes while leaving occlusions intact. Inpainting. At 1px = 1cm 2 , the chosen patch size was 43x43px to t the smallest structuring element (cobbles). In all our experiment, η = 0.2 ended up being a very good choice. Finally, we set the value of γ to 0.3 or 10 6 , the choice being made by automatically checking whether the internal radius of the evaluated occlusion was higher than 50cm or not.

Qualitative analysis

A quick glance at the dierence between traditional aerial orthophotography and MLS orthoimage using our framework is given in gure 9. The resolution provided by a typical aerial camera is about 50cm 2 per pixel, where our reconstruction is done at 1cm 2 per pixel. Fine textures and very precise details are noticeable in the reconstruction whereas only main structures can be seen in the aerial orthophotography. Moreover, the aerial orthophotography presents various occlusions such as trees that do not appear in our result. In gure 10, we show a visual comparison between the proposed framework and the method introduced in (44) which is the state of the art for the production of orthoimages using MMS LiDAR point cloud. We can see that both algorithms perform about the same for stripe holes, but our solution gives more satisfying results for large occlusions. The texture is better reconstructed using our method. This will be later discussed in Section 6.3.

More reconstruction results are displayed in Figure 11. Each step of the pipeline is illustrated. We can see on Figure 11 top that the framework performs a very good reconstruction on ne details such as cobbles. In Figure 11 bottom, 25% (∼ 5.10 5 px) of the area is occluded, mostly due to the presence of cars and poles. However, our framework succeeds plausible reconstruction of the scene, leading to a result that is much more understandable than initially. Finally, gure 12 shows an extreme scenario where the use of the range is relevant as the structure of the scene follows the same path as the road. The environment is fully reconstructed (16%, ∼ 10 6 px) while preserving the structure of the road.

In Figure 13, the framework is applied on data provided by the Seman-tic3D dataset [START_REF] Hackel | Semantic3D.net: A new large-scale point cloud classication benchmark[END_REF]. This dataset is aquired using a static LiDAR sensor. There, we can see that the area under the sensor as well as occlusion on the ground are successfully recovered while preserving the ne cobble texture.

The purpose of this pipeline is to generate both reectance and height orthoimages. In gure 14, we show how the two outputs can be combined in order to obtain a 3D model of the road. Areas that present large occlusions are highlighted in green. (c) the nal results of our method. In both result, the orthoimage is successfully reconstructed while improving the understandability of the scene.

scene with the junction of the road and a pavement.

Quantitative analysis

Apart from the visual results, we also provide a numerical comparison between the proposed framework and the one of [START_REF] Vallet | Road orthophoto/DTM generation from mobile laser scanning[END_REF]. Measuring similarities between two images is a tough task as the plethora of dierent metrics are all designed for a single aspect of the image (color variation, gradient similarity and correlation). In the case of texture synthesis, the similarity cannot be measure of the standard deviation and the distance between histograms, also known as Wasserstein metric in [START_REF] Rabin | Wasserstein barycenter and its application to texture mixing[END_REF], provide simple and ecient metrics for evaluating the quality of our results. Table 2 sums up the comparison of the inpainting step on two examples: an image where the hole has been manually removed and an image where the ground truth is available as the vehicle did a second pass in which the occlusion disappear. For each example, we compute the standard deviation of the region reconstructed by examplar-based inpainting. We also compute the distance between the normalized histrograms of the ground truth and a b c each output. For both examples, our method provides a standard deviation that is very close to the ground truth resulting in visually similar textures.

As the proposed framework also reconstructs the height map of the aquired area, we provide a numerical analysis of this aspect. The choice of the metric in that case is quite easier as the height map is more homogeneous than the reectance image, especially in a urban scenario as can be seen in gure 15. Therefore, the Normalized Mean Square Error is enough to estimate how good the reconstruction is. We found out that in general the mean square error was below 1cm. This validates the proposed framework for the reconstruction of height map.

Computational speed

The performances of the framework in terms of computational speed are mostly aected by the amount of occlusions and the resolution at which the reconstruction is being made. As the framework is composed of several steps, we present the computation time of each step as well as the total time of processing. All the results are given using MATLAB 2015a on a single thread with an Intel Core i5 CPU at 3.40GHz.

The speed of computation is summed up in table 3. The evaluation is done for the reconstruction of the same point set at dierent resolutions. The choice of resolution and the amount of stripe holes do not aect much the computation time in proportion. However, the inpainting of large occlusions drastically increases the time of computation in the case of very high resolution. The computation speed of this step might be largely improved by using approaches derivated from Patch-Match (4). Moreover, the framework can be run in parallel as each step is independent of the next ones.

Conclusion and future work

We have proposed a complete framework to reconstruct high quality ground orthoimage from a point cloud aquired with LiDAR. This framework consists of several steps, which make use of classical modern imaging techniques. By taking into account the multi-modal nature of the data, we propose several modications of these methods, leading to signicantly better results.

The framework is designed to work automatically with a set of parameters that ensures satisfying results on a large variety of input data as demonstrated by the results. Our approach performs at least as well as previous techniques. In case of large occlusions or complex textures, it drastically outperforms earlier works in terms of visual quality. Moreover, robustness towards edges and structures conservation in both reectance and height domain has been demonstrated.

Although the average results of the method are more than acceptable, it can underperform in some specic cases. Indeed, the recognition of the area to reconstruct using the envelop can sometimes fail when a massive nonstatic object in the scene is considered (i.e. a building), this area therefore not being reconstructed. Moreover, the use of examplar-based inpainting introduces common issues such as aberrant synthesis when no similar patches are available.

In the future, we will focus on improving the current framework to better distinguish static structures and mobile objects, based on the work presented in [START_REF] Schoenberg | Segmentation of dense range information in complex urban scenes[END_REF]. Moreover, we also want to perform labelling on the scene in order to produce a more relevant metric for evaluating the results. Finally, we aim at using aerial optical data in order to provide colored orthoimages, which promise very interesting challenges.

Acknowledgement

This study has been carried out with nancial support from the French State, managed by the French National Research Agency (ANR) in the frame of the Investments for the future Programme IdEx Bordeaux (ANR-10-IDEX-03-02). J-F. Aujol is a member of Institut Universitaire de France.

Figure 3 :

 3 Figure 3: Highlighting of the dierent types of holes in the sparse projection. (a) is the original point cloud colored with the reectance, (b) is the sparse projection of the white rectangle in (a) after extracting ground points, (c) is the sparse projection labelled with the dierent kinds of holes.

Figure 4 :

 4 Figure 4: Slice of an envelop Γ obtained by evaluating several aligned beams coming from the sensor S until they hit an object of the scene. The red line is the nal threshold applied to the envelop to exclude too high points.

Figure 5 :

 5 Figure 5: Comparison of dierent diusion techniques for lling stripe holes. (a) is the point cloud, (b) its projection (rotated for clarity purpose), (c) is the Gaussian diusion result, (d) is the Perona-Malik algorithm result and (e) is the result of our proposed modication. We can see that our modication provides a better conservation of big holes while lling perfectly the stripe holes.

Figure 6 :

 6 Figure 6: Result of the dierent diusion models on degraded ground truth. (a) original image, (b) original image with 80% of pixels removed, (c) Closest Neighbors result, (d) Gaussian diusion result and (e) Our result. We can see that the Gaussian diusion and our model better recover the aspect of the image. Our method succeed in a ner edge recovery.

Figure 8 :

 8 Figure 8: Illustration of the assumption that the urban environment evolves in a similar way than the path of the sensor. The straight green line shows the path of the sensor. Each dashed line represents areas of same distance to the sensor.

Figure 9 :

 9 Figure 9: Comparison between aerial orthophotography with a standard resolution (50cm 2 per pixel) (top) and MLS orthoimage using our model at 1cm 2 per pixel (bottom). Traditional orthophotography provides limited resolution and suers from occlusions brought by the coverage of trees and other structures whereas our model provides unobstructed, hight resolution orthoimages. The aerial image comes from Geoportail.

Figure 10 :

 10 Figure 10: Comparison between our proposed framework (a) and the one introduced in (44) (b). Texture is better preserved using our framework.

  Figure 14.a and 14.b are the reectance image and the height image of the area that is being modelled in gure 14.c. We can see that the 3D model respects the topography of the a. b.c.

Figure 11 :

 11 Figure 11: Various results on dierent urban scenes. (a) shows the original point clouds projected on an horizontal grid (sparse). (b) are the results after stripe holes were lled.Areas that present large occlusions are highlighted in green. (c) the nal results of our method. In both result, the orthoimage is successfully reconstructed while improving the understandability of the scene.

Figure 12 :

 12 Figure 12: Exemple of scene that follows the vehicule path. In this case, the use of the range information is very relevant. The green dashed line denotes the vehicle path.

Figure 13 :Figure 14 :

 1314 Figure 13: Example of reconstruction on the Semantic 3D dataset. (a) is the area from which the orthoimage is aquired, (b) is the projection of the ground points on an horizontal grid, (c) is the nal result. The nal results provides a plausible estimation of the area under the acquisition sensor.

Figure 15 :

 15 Figure 15: Comparison between height images with and without occlusion on the junction between a road and a pavement. (a) is the original height map, (b) is the reconstruction of an occlusion in the same area. The occlusion corresponds to the darkest region of (c). The mean square error of the reconstruction (b) compared to (a) on the occlusion region is 2mm.

Table 1 :

 1 Evaluation of dierent diusion algorithms

	Metric	Closest Neighbors Gaussian Proposed Model
	MSSIM	0.8056	0.8550	0.8591
	MPSNR (dB)	33.21	34.59	35.08

Table 2 :

 2 Numerical comparison between reconstructions

	Image	Articial occlusion STD Hist. dist. STD Hist. dist Real occlusion
	Ground truth	4.51	-	4.79	-
	Proposed framework 4.56	0.14	4.29	0.19
	(44)	1.87	0.78	2.05	0.80

Table 3 :

 3 Comparison of computation speed compared to the resolution

	Image size	600x550px 2400x2200px
	Image resolution	1px = 4cm 2 1px = 1cm 2
	Percentage of stripe holes	13%	61%
	Percentage of occlusion holes	22%	25%
	2D Projection	2.13s	3.78s
	Diusion	1.54s	3.27s
	Mask extraction	0.18s	0.91s
	Examplar-based inpainting	23.81s	6.31min
	Total	27.66s	6min38s