Member, IEEE Mohsen Koohestani
email: mohsen.koohestani@eseo.fr

Jean-Luc Levant
email: jeanluc.levant@microchip.com

Senior Member, IEEE Mohamed Ramdani
email: mohamed.ramdani@eseo.fr

R Koohestani

M Perdriau

Ramdani

A Novel Passive Cost-Effective Technique to Improve Radiated Immunity on PCBs

Keywords: Dielectric loading, far field, integrated circuits (ICs), near field, radiated immunity, transverse electromagnetic (TEM) cell

This paper proposes a novel simple cost-effective technique to improve the potential immunity of integrated circuits (ICs) to radiated electromagnetic (EM) disturbances. It is based on dielectric loading which lessens the susceptibility of ICs independently of frequency by confining the reactive fields inside the dielectric load. This has no side effects on the IC performance.

To highlight the importance of the proposed technique, immunity testing was performed numerically both in near and far fields. A loaded IC (a microcontroller) was compared to an unloaded one through far-field measurements inside a transverse EM cell to indirectly assess the suitability of the proposed approach. Results show that loading the IC with non-ferromagnetic dielectric material not only effectively decreases its susceptibility to electric field, but also to magnetic field. Moreover, a significant difference of ∼48% less failure rate was achieved for the loaded compared to the unloaded IC demonstrating the usefulness of this technique as a potential solution for radiated immunity improvement.

I. INTRODUCTION

E LECTROMAGNETIC (EM) immunity (EMI) and EM compatibility (EMC) testing are of great importance with the fast increasing functionality and complexity of integrated circuits (ICs) [START_REF] Ben Dhia | Electromagnetic Compatibility of Integrated Circuits: Techniques for Low Emission and Susceptibility[END_REF]- [START_REF] Reddy | A novel segmentation approach for modeling of radiated emission and immunity test setups[END_REF]. A higher clock frequency or any electrical transition with sharp edges (e.g., addressing or controlling data) gives rise to EMI issues [START_REF] Kim | Spread spectrum clock generator with delay cell array to reduce electromagnetic interference[END_REF].

In an embedded system, a microcontroller (μC) plays a major role in governing specific operations. μCs are subjected to a partial or a complete malfunction in presence of an EM disturbance [START_REF] Yuan | Detection of electromagnetic interference in microcontrollers using the instability of an embedded phase-lock loop[END_REF]. Moreover, printed circuit boards (PCBs) having relatively long tracks connected to IC pins suffer more in terms of radiation, as they act as antennas driven by IC activity. These have led the EMC/EMI regulatory commissions (e.g., international electrotechnical commission IEC [START_REF]Integrated Circuits, Measurement of Electromagnetic Immunity, 150 kHz to 1 GHz: General Conditions and Definitions[END_REF]) to define limits on EM radiation produced by ICs.

Besides, the industrial trends in the EMC of μCs show an increasing demand for IC miniaturization technology, which leads to speed up their functionality at the cost of a higher power consumption and stronger switching currents [START_REF] Op't Land | Integrated circuit immunity modeling beyond 1 GHz[END_REF]. To counteract the latter effect, voltage supply is reduced, which in return lowers the noise margin of digital circuitry and consequently IC immunity [START_REF] Ramdani | The electromagnetic compatibility of integrated circuits-past, present, and future[END_REF]. This fact fostered some interest in the wireless communications community for ways to improve immunity of ICs to radiated disturbances.

A most commonly used approach to improve the EMI field generated by today's high-speed ICs is shielding [START_REF] Parker | Electromagnetic interference: A tutorial[END_REF]. Another prevalent solution is to take benefit from flexible polymer absorbers [START_REF] Valente | Setup for EMI shielding effectiveness tests of electrically conductive polymer composites at frequencies up to 3.0 GHz[END_REF], [START_REF]RFI Absorbers[END_REF]. They are made of either soft metal flakes filler in polymer resin or high permeability magnetic flake particles. The latter is usually preferred as it is non-conductive and can be directly applied to circuit tracks.

Here, we propose a new simple passive approach to improve the radiated immunity on PCBs using dielectric loading. It is worth mentioning that dielectric loading has been mainly employed by antenna engineers as an effective technique to reduce antenna dimensions [START_REF] Moreira | Ultra wideband dielectric sandwich loaded antennas[END_REF], to introduce notch frequency bands [START_REF] Koohestani | Bandreject ultra-wideband monopole antenna using patch loading[END_REF], and, in general, to improve antenna behavior with no side effects [START_REF] Koohestani | Influence of dielectric loading on the fidelity factor of an ultra wideband monopole antenna[END_REF]. It has also been used to reduce interactions between antenna and human body by making the antenna less sensitive to human body proximity effects [START_REF] Koohestani | Bandwidth enhancement of a wearable UWB antenna near a human arm[END_REF]. It consists in gluing small pieces of demetalized commercially available substrate with a proper permittivity around/over the IC. Simulations and measurements were conducted to confirm the adequacy of the proposed technique.

There exist several near and far field measurements techniques to evaluate the radiated EM immunity of ICs. The most commonly used near-field technique is the IEC 62132-9 scan method [START_REF] Lacrampe | Radiated suceptibility investigation of electronic board from near field scan method[END_REF], which is a reciprocal use of IEC 61967-3 employed for immunity testing. A transverse EM (TEM) cell, which approximates a uniform and linearly polarized plane wave, is widely used for far-field EMI/EMC tests of radio frequency (RF) devices [START_REF] Stojanovic | Determination of equivalent coupling surface of passive components using the TEM Cell[END_REF]. In general, measurements in far field indicate the PCB radiation with respect to its orientation and to frequency. However, they cannot determine the internal sources of emissions. Conversely, near-field techniques overcome this limitation providing a precise location of radiation sources. Here, for the sake of completeness, the suitability of the proposed approach was investigated both in near and far field zones.

The paper is organized as follows. Section II describes the materials used, the methodology employed, and the simulation and experimental setup configurations. Section III details the analysis of the numerical results, while Section IV addresses the experimental results of the proposed technique. Concluding contributions of this study are provided in Section V.

II. MATERIALS AND METHODS

This section deals with the research material and methodology to study the influence of dielectric loading on the radiated immunity on PCBs.

A. Simulation Setup and Method

As described in IEC 62132 [START_REF]Integrated Circuits, Measurement of Electromagnetic Immunity, 150 kHz to 1 GHz: General Conditions and Definitions[END_REF], a simple 10 cm × 10 cm square PCB was considered with a 3 cm × 3 cm square metal patch in its center representing a rough approximation of an IC (see Fig. 1). The patch is printed on a low-cost FR4 epoxy substrate ( r = 4.4, thickness = 0.8 mm, and tan δ = 0.02). The other side of the PCB is coated with metal serving as ground plane. The PCB is mounted 1 cm above a 50 Ω transmission line (TL) of 0.37 cm width to be in near-field conditions (see Fig. 1). The other side of the TL substrate (FR4, thickness = 1.6 mm) is a metallic ground plane. All the metallic parts in the numerical model are considered to be copper.

A 5 cm × 5 cm solid piece of demetalized low-loss commercial substrate (RT/Duroid 6002) with a permittivity of ε r = 2.94, thickness of 0.762 mm, and tan δ = 0.001 was used as the dielectric load to fully cover the square patch. This small low-loss load confines the reactive (non-radiating) fields, which would result in lower electric field intensity while maintaining magnetic field, with no EM power dissipation. To evaluate the importance of the proposed dielectric loading technique, simulations were performed using Ansys HFSS full-wave solver both in near and far EM field zones. In EMC, the limits for near and far field zones are usually fixed at λ 2π and 2D 2 λ from the source, respectively, where λ and D are the wavelength and the largest radiator dimension, respectively.

B. Measurement Setup and Method

In order to perform experiments, an indirect measurement was considered due to the impossibility of measuring the EM fields directly at the PCB level under the dielectric pad. As modern μCs are generally equipped with embedded phase lock loops (PLLs) to accurately protect the system clock from jitter and instabilities, they are very sensitive to EM disturbances [START_REF] Yuan | Detection of electromagnetic interference in microcontrollers using the instability of an embedded phase-lock loop[END_REF], [START_REF] Orietti | Electromagnetic susceptibility analysis on a digital pulse width modulator for SMPSs[END_REF], [START_REF] Lee | Analysis and measurement of timing jitter induced by radiated EMI noise in automatic test equipment[END_REF]; in [START_REF] Yuan | Detection of electromagnetic interference in microcontrollers using the instability of an embedded phase-lock loop[END_REF] and [START_REF] Lee | Analysis and measurement of timing jitter induced by radiated EMI noise in automatic test equipment[END_REF], the μC malfunction due to injecting magnetic field on PLL power supply pins has been reported. Here, a 32-bit μC (ATMEL SAM3S4B) [START_REF]ATEMEL SAM3S ARM-based Flash MCU series microcontrollers[END_REF] initially designed for a previous research project was used. It was loaded by a piece of dielectric to check whether it improves the immunity of the μC's clock PLL to the EM interferences. A TEM cell as the far-field free space condition emulator was also employed for radiated immunity testing. We used a standard TEM cell (Model: FCC-TEM-JM2, dc up to 1.6 GHz) for the purpose of this study [START_REF]FCC-TEM-JM2 TEM cell[END_REF].

The thermal conductivity of RT/Duroid 6002 is 0.6 W m .K [START_REF]RT/Duroid 6002 low-loss high frequency laminates[END_REF]. For the considered load dimension of 50 mm × 50 mm × 0.762 mm layer, the thermal resistance is 0.5 K W , which would lead to a low temperature difference between both sides of the load even for large size ICs (e.g., dissipating about 15 W, such as Intel Core TM i3-7020U laptop processors). In case of small ICs, such as SAM3S4B, with 10 mm × 10 mm package, the thermal resistance would be 12.5 K W (25 × 0.5 K W ), considering the worst case scenario where no heat would be dissipated in the horizontal plane. With the SAM3S4B dissipating about 132 mW (40 mA × 3.3 V), the difference between the upper and lower sides remains very low (0.132 × 12.5 = 1.65 • C).

III. NUMERICAL RESULTS AND DISCUSSION

This section presents the impact of dielectric loading on the PCB behavior in near and far field scenarios. A comparison is made between the loaded and unloaded ICs (an equivalent patch) considering the distribution of wave impedance, i.e., the ratio of the norms electric to magnetic field ( E / H ), as the criterion to provide an index of the relative strength between the electric and magnetic fields. The magnitude of the electric field decreases under the load due to the dielectric properties of the material. As far as the magnetic field is concerned, coupling with the pins of an IC depends on the orientation of the tangential component: The more parallel to the pins, the lesser coupling and consequently lower is the disturbance injected into lowimpedance pins (such as power supply pins).

In a TEM cell, magnetic field lines are supposed to be all in the same direction. In the case presented in the paper, it must be noted that the purpose of loading is not to ensure the most homogeneous magnetic field below the load surface, but to decrease the IC susceptibility. Therefore, when the IC is replaced by an equivalent patch, it is necessary to check out the following, when the patch is loaded:

1) If the direction of the tangential magnetic field around the patch at a given frequency remains unchanged, the magnitude of that field decreases (or, at least, does not increase). 2) If the direction of that magnetic field changes, the latter becomes more perpendicular to the patch sides (i.e., parallel to IC pins). If those conditions are met, it is possible to claim that the immunity of the IC to magnetic field would either decrease or remain unchanged with the dielectric load.

A. Near Field

Fig. 1 shows the simulation setup; one port is excited, while the other one is terminated with a 50 Ω load. The IEC standard for the voltage standing wave ratio (VSWR) with 50 Ω termination has to be lower than 1.25, which is the case here over a wide range of frequencies [see Fig. 3(a)]. To evaluate the importance of the dielectric loading, a comparison was made with the unloaded PCB.

Fig. 2 shows the wave impedance distribution on the PCB surface with and without the dielectric loads at two gigahertzrange resonance frequencies, i.e., 1 and 1.85 GHz [see Fig. 3(a)]. Note that loading neither detunes the frequency nor changes the level and, therefore, the variation in impedance distribution is not a result of a frequency change due to the load. In Fig. 2, the colors representing the E / H ratio distribution in linear scale go from dark blue (low impedance) to green, yellow, and red (high impedance). As observed in Fig. 2(b), we can clearly see that the highest impedance at the bottom side of the IC is ∼800 Ω for the loaded case, whereas for the unloaded one, it ranges over relatively higher values from 1500 to 2500 Ω. This is a result of reactive field confinement in the dielectric loads. The effectiveness of the loading technique was checked through simulations and found to remain unchanged for a similar scenario, but with the PCB flipped down to have the patch facing the TL.

Fig. 4 plots the simulated distribution of the vector magnetic field around the patch for both frequencies, with and without the load. At 1 GHz, the orientation is almost the same in both cases, with a worst-case coupling (the field is often parallel to a side and, consequently, perpendicular to IC package pins) over a very significant part of the patch perimeter, while the magnitude decreases and becomes much more uniform in presence of the load. At 1.85 GHz, the orientation does not change with the load either (but is the same on the four sides of the patch), while the magnitude distribution exhibits low-color spots at different locations; however, that fact can be neglected, as field lines are perpendicular to the patch left and right sides where magnitude increases (hence, negligible coupling to IC pins); conversely, where field lines are parallel to the top and bottom sides, magnitude is reduced.

In both cases, it can thus be considered that the susceptibility of an IC to magnetic field would decrease (most of the time) or remain unchanged depending on the frequency and the location of the sensitive pins. Meanwhile, it should be remembered that, in all cases, the susceptibility to electric field would decrease.

On the one hand, as most of the μCs tend to have an emission spectrum and a susceptibility frequency bandwidth over the 10-100 MHz frequency range, a study was conducted at 60 MHz to check the suitability of the proposed technique at lower frequencies (see Fig. 5). Following the previous reasoning, improvements in immunity as a result of lower EM fields due to loading are evident; the E / H ratio at 60 MHz reduces from E / H ratio for unloaded and loaded (ε r = 2.94) PCBs for an input power of 1 W at 60 MHz. The scale is the same as in Fig. 2. Fig. 6.

E / H ratio for a loaded PCB with ε r = 10.2 PCB for 1-W input power at (a) 1 GHz and (b) 1.85 GHz. The scale is the same as in Fig. 2. 1400 to 200 Ω (by 85%) for a loaded compared to an unloaded PCB.

On the other hand, since the amount of stored energy in a dielectric is directly proportional to its permittivity, it is expected to have even higher immunity in a loaded IC with higher ε r values. Fig. 6 shows results for a case study with RT/Duroid 6010 loads (ε r = 10.2) demonstrating a significant improvement in wave impedance around the IC where the load is located; it reduces from 800 to 200 Ω (by 75%) for a dielectric load with a higher ε r of 10.2 compared to 2.94.

B. Far Field

A standard TEM cell as the far-field free space condition emulation was used for far-field testing of radiated immunity [START_REF]FCC-TEM-JM2 TEM cell[END_REF]. Loaded and unloaded versions of the 10 cm × 10 cm PCB, shown in Fig. 1, were placed on the TEM cell aperture. The IEC standard for VSWR lower than 1.25 was fully met at the TEM cell resonant frequencies. Fig. 7 shows the E / H ratio distribution over the PCB at the first two resonance frequencies, i.e., 0.9 and 1.4 GHz [see Fig. 3(b)]. Note that, unlike the septum used in simulation, the one of the real TEM cell is asymmetric, which allows much lower VSWR. As observed, loading neither detunes the frequency nor changes the VSWR level and, hence, the variation in impedance distribution is independent of the load impact on the resonant frequency change.

Following the above-mentioned reasoning at the beginning of the current Section, a first glance at the results in Fig. 7 clearly shows the effectiveness of the proposed dielectric loading technique to improve IC radiated immunity independently of frequency. At both considered frequencies, much lower impedance was observed in the loaded compared to the unloaded PCB around the IC where the dielectric loads are attached; the wave Fig. 7.

E / H ratio for unloaded and loaded (ε r = 2.94) PCBs inside a standard TEM cell for an input power of 1 W at (a) 0.9 GHz and (b) 1.4 GHz. impedance at higher frequency is reduced by 46.5% (from 511 to 273 Ω).

A plot similar to Fig. 4 for near field is provided in Fig. 8 for far field. As observed, at both frequencies, the magnetic field on left and right sides of the patch has the same magnitude and orientation with or without any load. On top and bottom sides, higher magnitudes can be observed with the load, but can be neglected due to the magnetic field lines being perpendicular to the patch sides. Note that those simulations were performed at the resonant frequencies of the TEM cell, which is indeed the worst case in terms of field homogeneity. Consequently, the same conclusions can be drawn in a TEM cell: Loading would decrease the susceptibility of an IC both to electric and magnetic fields (or, in the worst case, leave the latter unchanged).

At 60 MHz, thanks to dielectric loading, a similar behavior of the E / H ratio to that of higher frequencies (around 1 GHz) E / H ratio for unloaded and loaded (ε r = 2.94) PCBs for an input power of 1 W at 60 MHz. The scale is the same as in Fig. 7.

was achieved observing lower impedance and consequently less field strength around the IC (see Fig. 9). Therefore, the dielectric loading technique is also effective at lower MHz frequencies where most μC tend to radiate.

A plot similar to Fig. 6 for near field is provided in Fig. 10 for far field in a TEM cell. It can be seen that the higher permittivity (ε r = 10.2) does not provide any E / H improvement compared to the lower value (ε r = 2.94).

IV. EXPERIMENTAL RESULTS AND DISCUSSION

Due to the impossibility to measure the EM field directly under the dielectric layer, an indirect measurement technique was set up. It consisted in using an IC (a μC) with an observable criterion for radiated immunity. A loaded IC was compared to an unloaded one through measurements using a standard TEM cell (see Fig. 11). E / H ratio for a loaded PCB with ε r = 10.2 for an input power of 1 W at (a) 0.9 GHz and (b) 1.4 GHz. The scale is the same as in Fig. 7. A 10 cm × 10 cm PCB with a 32-bit μC (ATMEL SAM3S4B) [START_REF]ATEMEL SAM3S ARM-based Flash MCU series microcontrollers[END_REF] was employed for the experiments (see Fig. 11). The μC was configured to generate a 48-MHz core clock from its internal PLL referenced by an external 18.432 MHz crystal. It was then programmed with simple software toggling one output pin at a fixed frequency (about 1.5 kHz).

A previous measurement campaign highlighted that the clock generation circuitry of that μC was sensitive to continuous-wave interference at 223 MHz. Therefore, immunity could be assessed by measuring the jitter of the output signal through automatic mask testing, thanks to a Keysight DSO6014L oscilloscope fitted with the appropriate option. The time-domain limit of the mask was fixed at a 5% deviation in signal period in both directions (lower and higher). The immunity criterion was the statistical failure rate (i.e., the output signal leaving the area of the oscilloscope mask) averaged over a one-minute time slot.

The PCB was secured on the aperture of the TEM cell and exposed to radiated interference through continuous-wave power injection into the septum [see Fig. 11(d)], thanks to a Keysight N5183A generator feeding a Prâna AP32DT120 20 W power amplifier. The other port of the TEM cell was terminated with a 50-Ω load. Incident and reflected powers were monitored with a Keysight E4419B dual-channel power meter through the built-in directional coupler of the amplifier. As mentioned in Section II, the PLL of a μC is highly sensitive to a tangential magnetic field coupled to its power supply pins (namely V DDPLL ), provided that the field component is in the same direction as the loops formed by the pins on the PCB: The PLL is perturbed when the interference is propagated into its power supply rail.

The incident power was measured at 43 dBm (20 W upper limit of the amplifier) while the reflected power was monitored at 15 dBm (31 mW). Of course, the injection frequency was chosen at 223 MHz, in order to make use of the weakness in immunity mentioned previously.

Three different injections were considered: (1) without any dielectric loading [see Fig. 11(a)]; (2) with dielectric loading covering only the PCB traces and μC pins [see Fig. 11(b)]; and (3) with dielectric loading over the whole μC (package and pins) and the PCB traces [see Fig. 11(c)]. They were carried out exactly in the same conditions (same RF power, same PCB location, and orientation). The orientation of the PCB was chosen to maximize magnetic field coupling to the power supply pins. That was demonstrated by the fact that no errors could be observed by rotating the PCB by 90 • . Results are shown in Fig. 12 observing a 7.14% failure rate in case [START_REF] Ben Dhia | Electromagnetic Compatibility of Integrated Circuits: Techniques for Low Emission and Susceptibility[END_REF] [see Fig. 12(a)], 4.97% in case [START_REF] Pythoud | Proficiency testing in EMC radiated immunity[END_REF] [see Fig. 12(b)], and 3.70% in case [START_REF] Reddy | A novel segmentation approach for modeling of radiated emission and immunity test setups[END_REF] [see Fig. 12(c)]. These results clearly demonstrate the improvements in radiated immunity due to dielectric loading in a significant manner. The results also show that dielectric loading not only makes the IC and PCB immune to the electric field, but also decreases the susceptibility of the μC to external magnetic field. From an economic point of view, a dielectric or a non-ferromagnetic material would be a better option compared to a ferrite-based material.

V. CONCLUSION

A dielectric loading technique was proposed to improve the radiated immunity of ICs, which has no side effects on their performances. The behavior of an electronic circuit with a loaded IC was compared to an unloaded one both in near and far fields. It was found that, independently of frequency, a loaded circuit is less susceptible compared to an unloaded one due to the reactive field confinement in the dielectric load without inducing side effects on its performance. Moreover, loading an IC with non-ferromagnetic dielectric material proved to be effective to enhance its susceptibility to both magnetic and electric field, which makes it possible to reduce the cost of the technique compared to the ubiquitous use of ferrites. An experimental show case study using a TEM cell proved a ∼48% less failure rate for a loaded μC (ATMEL SAM3S4B) compared to the unloaded IC. The obtained results demonstrate the suitability of the proposed technique as a potential candidate to improve the radiated immunity of electronic devices.

Fig. 1 .

 1 Fig. 1. Schematic geometry used in simulation.

A c c eFig. 3 .

 3 Fig. 2. E / H ratio for unloaded and loaded (ε r = 2.94) PCBs for 1-W input power at (a) 1 GHz and (b) 1.85 GHz.

Fig. 4 .

 4 Fig. 4. Simulated vector magnetic field distribution around the unloaded (top) and loaded (bottom) patch under the load at (a) 1 GHz and (b) 1.85 GHz. Black arrows indicate the privileged orientation.

Fig. 5 .

 5 Fig. 5.E / H ratio for unloaded and loaded (ε r = 2.94) PCBs for an input power of 1 W at 60 MHz. The scale is the same as in Fig.2.

Fig. 8 .

 8 Fig. 8. Simulated vector magnetic field distribution inside the TEM cell around the unloaded (top) and loaded (bottom) patch under the load at (a) 0.9 GHz and (b) 1.4 GHz. Black arrows indicate the privileged orientation.

Fig. 10 .

 10 Fig. 10.E / H ratio for a loaded PCB with ε r = 10.2 for an input power of 1 W at (a) 0.9 GHz and (b) 1.4 GHz. The scale is the same as in Fig.7.

Fig. 11 .

 11 Fig. 11. Measurement setup with a loaded and unloaded 32-bit μC. (a) Unloaded. (b) Partially loaded. (c) Fully loaded. (d) TEM cell used for the purpose of this study [22].

  A c c e p t e d M a n u s c r i p t

Fig. 12 .

 12 Fig. 12. Measured μC failure rate averaged over a one-minute time slot for an input power of 20 W at 223 MHz. (a) Unloaded. (b) Partially loaded. (c) Fully loaded.

A c c e

  p t e d M a n u s c r i p t

ACKNOWLEDGMENT

The authors would like to cordially thank V. Arboux for his assistance with prototype construction.

Mohsen Koohestani (S'13-M'15) received the Ph.D. degree (with honor) in electromagnetics from the École polytechnique fédérale de Lausanne (EPFL), Lausanne, Switzerland, and the Universidade de Lisboa (ULISBOA), Lisbon, Portugal, in 2014.

From 2014 to 2018, he was a Postdoctoral Fellow with the Institut d'Electronique et de Télécommunications de Rennes (IETR), Université de Rennes 1, France, working mainly on biomedical applications of wireless power transfer systems (WPTs). He is currently an Associate Professor with ESEO, Angers, France, in the field of RF/microwave wireless systems. He is also an Associate Researcher with IETR. He has authored over 40 peer-reviewed scientific journal papers. His research interests include novel design and analysis of antennas and microwave components, design and characterization of human body models, dosimetry and exposure assessment, and WPT and energy harvesting.


Friederike Peteler
email: friederike.peteler@map.cnrs.fr

Eloi Gattet
email: eloi.gattet@map.cnrs.fr

Philippe Bromblet
email: philippe.bromblet@cicrp.fr

Odile Guillon
email: odile.guillon@cicrp.fr

Jean-Marc Vallet
email: jean-marc.vallet@cicrp.fr

Livio De Luca
email: livio.deluca@map.cnrs.fr

Analyzing the evolution of deterioration patterns: A first step of an image-based approach for comparing multitemporal data sets

Keywords: Change measurements, multitemporal data set, depth map, ortho image, evolution of deterioration patterns

When documenting and analyzing cultural heritage, the monument states can be described by multitemporal data sets, which however present a complication for the elaboration and examination process. This difficulty leads to the necessity to improve the analyze process in order to expand the documentation process and help experts to enrich and share information about the historical buildings. Therefore an approach of change measurement, which supports the chronical comprehension of a building by visualizing and quantifying the dimensional temporal effects was elaborated, where the analyze process of a multitemporal data set was based on the interpretation of depth map images. These maps were obtained by generating ortho images of an object that was created on purpose by setting up an experimentation to acquire a multitemporal data set.

This approach is the first step of a wider ongoing research about change detection processes on multitemporal data sets.

I. INTRODUCTION

In the field of preservation and valuation of cultural heritage, the monument states can be described by various sets of data like multitemporal data, obtained through diverse acquisition techniques, in different light conditions, composed of various size and resolution. The difficulty to elaborate, compare, examine and validate these data prior to restauration, leads to the necessity to improve the analyze process. The documentation process has to be expanded in order to help experts to create, enrich and share information about the historical buildings [START_REF] Manuel | An approach for precise 2D/3D semantic annotation of spatially-oriented images for in situ visualization applications[END_REF]. Change detection is an important analyze method which helps the comprehension of a building. During the process of change detection, an object is observed in subsequent time periods to identify the differences in the surface state of the object and to quantify the chronological effects on the multitemporal data set [START_REF] Singh | Digital change detection techniques using remotelysensed data[END_REF].

Photographs testify the state of preservation of a building at a specific time and constitute therefore an important value for a change detection process. As well with the development of automated image-based-modeling techniques, large sets of images can be spatially oriented, allowing a very precise computation of change measurements. Nowadays the analysis of historical buildings is performed mostly on 3D point clouds, consenting a direct three-dimensional approach [START_REF] Girardeau-Montaut | Détection de changement sur des données géometriques tridimensionelles[END_REF] but they present as well the difficulty to occupy very large disk space, being composed of a lot of data. Moreover a 3D viewer appears to be problematic to be handled in the field and by non-experts of point cloud processing.

Therefore, it was decided to elaborate here a more end-user friendly approach by using images such as depth maps of ortho images for the chronical change computation. An experimentation was set up by simulating various alteration types on an artificial object and creating a multitemporal data set in order to test this specific method.

II. ACQUISITION AND DATA PROCESSING

This approach of change measurement, by using depth maps of the ortho images, is based essentially on a set of tools and procedures concerning the on-going development of an automated image-based 3D reconstruction method [START_REF] Micmac | Pastis and Other Beverages in a Nutshell! Tutorial[END_REF]. The process consists of an automated calibration and orientation of images, a dense multi-view correlation which leads to an ortho image generation in order to compare the morphological evolution of the deterioration patterns.

A. Study case

The experiment was carried out on a small artificial wall with L shape in cellular concrete (ca. 1.50 x 1.50 x 0.75 m), placed in an environment that allowed to perform different operations that simulate the nearest real conditions for three different types of illumination (raking light under tungsten lightening, front illumination with fluorescent light tubes and zenithal illumination with neon colored day light) and many of the degradation patterns according to Vergès-Belmin [START_REF]Illustrated glossary on stone deterioration patterns[END_REF] on stone deterioration.

The several degradation types were performed in four different steps, creating manually the loss and the adding of material within intervals of a few days. In a first state (T0), the wall was intact, in its new original shape, then in a second moment (T1), several types of alterations simulating deteriorations belonging to different families of the reference glossary like detachment, features induced by material loss, discoloration, deposits and biological colonization, were realized by means of some tools (sand paper, chisel, drilling device) and products (paintings, gypsum plaster, black lamp, vegetation). Then followed step three (T2) and four (T3), where each degradation pattern was successively intensified (thickness, depth, superficial expansion, chromatic saturation, etc.) (Fig. 1).

B. Acquisitions

276 photographs were taken in the four different time periods, 23 for each alteration step and illumination condition. The acquisition was carried out with a Nikon D800E with an f/8 aperture and a focal length of 35mm, following the acquisition protocol needed for a correct multi-view stereo correlation [START_REF] Martin-Beaumont | Photographer-friendly work-flows for image-based modeling of heritage artefacts[END_REF]. The position of the camera didn't have to be exactly the same thanks to the robustness of the orientation tool that was used afterwards. Nevertheless the acquisition positions were chosen quite similar in each phase to allow a more precise confrontation of the different steps and conditions.

In each time period was performed also one scan with a Faro Scan to extract later the Ground Control Points (GCP) information to scale the final orientation.

C. Automated orientation of images

The whole set of photographs was computed with the SIFT algorithm, in order to perform the tie point recognition for a matching between the images. The acquisitions of the three different light conditions, which simulate the diachronic impact on illumination (Fig. 2) were all calculated together. The SIFT algorithm is robust enough to find descriptors between the different illuminations. To optimize the orientation process, the set of photographs was divided into six different groups, one group for each significant correlation (COR) and orientation (ORI) position and a sixth group taking all the pictures at one time period (T2) with a specific light condition (halo rasant) (Fig. 3). For the second processing step, the recognition of the camera's geometric model internal parameters, a camera calibration was used, which had been created previously through a calibration performed on a test set.

The final image orientation was computed automatically adding one by one all the images and proceeding a bundle adjustment to refine each image orientation (Fig. 4).

Fig. 4. SIFT points and image orientation

To scale the set of photographs in correspondence to the acquired point clouds from the laser scanner, significant points were chosen on the scan point cloud and their GCP information was transferred into the image processing project. The image orientation was recalculated at this point creating a new global transformation of the orientation.

D. Creation of ortho images and depth maps

Now that the images were oriented in relation with each other, ortho images of each face of the wall and of each time period and light condition could be generated. Ortho images are the best approach for a comparison since they give a plane for the reference geometry. The points which have to be compared, get projected by the least mean square method on a plane, which can be used then to produce as well the depth maps, which show the distances of each point to this reference plane.

Therefore an adjacent plane was created on each face of the wall by drawing a mask on the pictures of face A and a mask on face B. The reference system could be recalculated according to the recognized tie points belonging to the mask and the adjacent midplane was computed with the least square fitting estimation.

A first collage of the interested photographs was obtained where it was possible do draw a mask on each result of each face of the wall to indicate which part of the scene should be used for the following ortho image creation. The final ortho image was computed by a dense matching-base surface reconstruction automatic method [START_REF] Pierrot-Desseiligny | Apero, an open source bundle adjustment software for automatic calibration and orientation of a set of images[END_REF][START_REF] Pierrot-Desseiligny | Automated image-based procedures for accurate artifacts 3D modeling and orthoimage generation[END_REF], that generates, on each relevant image, a depth map, a mask and an ortho image. These data were merged together by computing for each individual ortho image a radiometric equalization using the least mean square, generating the final ortho image.

In addition to this final ortho image, also a 16 bits depth map image of each ortho image was created. These depth map images of the different time periods were used for the following change measurement process. (Fig. 5).

III. CHANGE MEASUREMENTS

Once the oriented depth maps of the different time periods were generated, the process of change measurement could be set up. An approach was chosen to simply compute an image differencing between the spatially registered depth maps of the various time periods. By computing an image difference, a further image is produced that represents the variation between the two different times.

A numeric computation software can read the 16 bit depth images as matrixes and compute the distance of each z-value of the depth map. The matrixes representing the spatially oriented depth maps of two ortho images of two time periods are subtracted, generating a new matrix which describes the modifications and can be visualized in form of an image of change quantification. (Fig. 6).

The two main problems that can appear while computing the change quantifications on the various depth maps, are when the different images are not oriented spatially on the same plane and are not identical in size and resolution.

The orientation problem was resolved in this specific approach thanks to the generation of the adjacent midplane on each face of the wall in the previous elaboration phase when the ortho images were generated. The same midplane was used for the generation of each ortho image of each time period and therefore the resulting depth maps of each ortho image were spatially oriented in the same reference system and had the same coordinates.

The size problem was fixed during the same step of this workflow, when a mask was drawn on the collage of the interested photographs prior to the creation of the final ortho image. The same mask was used for the generation of each ortho image of each time and therefore the same part of the pictures was taken in consideration, generating images of the same size. Nevertheless the depth maps appeared to be composed by various resolutions. After having computed a resize of the resolution of the depth map images in order to obtain four depth images of the same size and resolution, it was possible to visualize a change quantification, computing the z-distance differences between each image. With the help of colors are visualized the differences computed, caused by added and subtracted material (Fig. 7).

A critical element of the image differencing method is deciding where to place the threshold boundaries between change and no-change pixel displayed in the histogram which has to be evaluated probably for each specific survey case and depending on the precision with whom the differences want to be recognized. In this case the boundaries were set in an interval of 10 cm to show the adding and loss of material with a high precision expressed in a multicolored map.

Analyzing the outcomes of the different light conditions, it results that the restitution of the acquisitions made during the tungsten raking light illumination showed less noise in comparison to the others. However the SIFT algorithm used, was able to perform a robust bundle adjustment with each different light condition for the orientation of the images.

IV. CONCLUSION

The procedure for a change measurement described in this work is the first approach of a new elaboration of a complete process of quantitative change monitoring on a multitemporal data set. It is based on the interpretation of non-threedimensional data, in order to facilitate the storage, the handling and the elaboration of this data and to make it more user-friendly for non-experts of 3D-point cloud processing. Therefore the complete process is not finished here and much work remains, but some applications can be already considered by using this first approach. The conservation of historic buildings and archeological excavations could be monitored with this method, quantifying the dimensional changes during various time periods.

The experiment conducted here will be enriched with the addition of a chromatic change measurement by elaborating an approach using the ortho images even of the different light simulations, to be able to monitor not only the dimensional alterations and degradations but also the chromatic deterioration patterns.

Furthermore the approach by now validates the geometric point of view, but future informatics implementations will require more in-depth analysis of several issues, that have to be taken into account for the ongoing development.

A future challenge will be also to carry out the process of change measurement on a multitemporal data set of an actual building where the alterations and degradations are unknown in order to test the efficiency of this approach. The aim will be to monitor quantitatively the evolution of the deterioration patterns in different resolutions, starting from images that were taken from various points of view, with different expositions and in dissimilar time periods. Computing the development of the deterioration patterns will be of great interest for conservations, starting from the diagnose procedures to the following elaborations that will be applied during the restauration process.

Fig. 1 .

 1 Fig. 1. The wall on which the experiment was carried out in time period T0 (left) and time period T3 (right).

Fig. 2 .

 2 Fig. 2. The three simulations of the diachronic illumination (from left to right): front illumination with fluorescent light tubes (fluo_ldj), zenithal illumination with neon colored day light (fluo_zenital) and raking light under tungsten lightening (halo_rasant).

Fig. 3 .

 3 Fig. 3. The six orientation groups divided by acquisition positions, time periods and light conditions.

Fig. 5 .

 5 Fig. 5. The four depth maps of time period T0, T1, T2 and T3 of face A and their histograms.

Fig. 6 .

 6 Fig. 6. The various changes between the different time periods: in the first row are visible the changes going from T0 to T1, T0 to T2 and T0 to T3.In the second row are visible the intermediary changes going from time period T1 to T2, T1 to T3 and T2 to T3.

Fig. 8 .

 8 Fig. 8. The change computation in detail: the first row shows parts of the ortho images in the four different states of deterioration, the second and third row show the depth maps and the corresponding grey shaded images, while the fourth row shows the images of change measurement of each time period in correspondence to T0.

ACKNOWLEDGMENT

We would like to thank Sébastien Aze for supplying the materials and building the experimental wall.