Haodong Jiang

Wengfeng Cai

Ke Li

Ming Cheng

Vineet Kumar

Zhen Yin

Davy Gé

Dan Luo

Quanquan Mu

Yan Jun Liu
email: yjliu@sustech.edu.cn

Holographically Fabricated, Highly Reflective Nanoporous Polymeric Distributed Bragg Reflectors with Red, Green, and Blue Colors

Keywords: distributed Bragg reflector, nanoporous reflection grating, holographic fabrication, phase separation, refractive index sensing

We report holographic fabrication of nanoporous distributed Bragg reflector (DBR) films with periodic nanoscale porosity via a single-prism configuration. The nanoporous DBR films result from the phase separation in a material recipe, which consists of a polymerizable acrylate monomer and nonreactive volatile solvent. By changing the interfering angle of two laser beams, we achieve the nanoporous DBR films with highly reflective RGB colors. The reflection band of the nanoporous DBR films can be tuned by further filling different liquids into the pores inside the films, resulting in the color change accordingly. Experimental results show that such kind of nanoporous DBR films could be potentially useful for many applications, such as color filters and refractive index sensors.

A distributed Bragg reflector (DBR) [START_REF] Lova | Advances in functional solution processed planar 1D photonic crystals[END_REF] is a periodic structure formed from alternating dielectric layers that can be used to achieve near unit reflectance within a range of frequencies. By designing DBR films with different periods and refractive indices, one can selectively achieve reflection or transmission of specific wavelength of light [START_REF] Puzzo | Color from colorless nanomaterials: Bragg reflectors made of nanoparticles[END_REF]. In the past decades, the DBR films have been widely used in sensors [START_REF] Kersey | Fiber grating sensors[END_REF], displays [START_REF] Chen | Resonant-enhanced full-color emission of quantum-dot-based display technology using a pulsed spray method[END_REF], lasers [START_REF] Jayaraman | Theory, design, and performance of extended tuning range semiconductor lasers with sampled gratings[END_REF], LEDs [START_REF] Neyts | Semitransparent metal or distributed Bragg reflector for wide-viewing-angle organic light-emitting-diode microcavities[END_REF] and other optoelectronic devices [START_REF] Shelykh | Interface photonic states at the boundary between a metal and a dielectric Bragg mirror[END_REF][START_REF] Brand | Optical Tamm states above the bulk plasma frequency at a Bragg stack/metal interface[END_REF]. Thus far, the DBR films are typically fabricated via evaporation and coating techniques, such as physical/chemical deposition, spin-coating, Langmuir-Blodgett coating, etc. However, these techniques usually require complicated multi-step processes, long fabrication time, or costly facilities, hence limiting further development of DBR applications. Therefore, it is highly desired to develop a simple, fast and cost-effective technique to fabricate the DBR films.

Along this line, the laser holographic technique could be the best option since it provides a single-step and rapid fabrication process for highly ordered structures. A typical holographic process starts with a photopolymerizable prepolymer material.

Upon holographic exposure, the cured photopolymer gives rise to periodic index modulations that are governed by the interference pattern of coherent laser beams [START_REF] Liu | Holographic fabrication of azo-dye-functionalized photonic structures[END_REF][START_REF] Liu | Azo-dye-doped absorbing photonic crystals with purely imaginary refractive index contrast and all-optically switchable diffraction properties[END_REF]. Depending on the interference configurations, one-, two-, or three-dimensional (1D, 2D or 3D) periodic structures can be created. By adding the nonreactive liquid crystals (LCs) into the prepolymer mixture, one can therefore achieve the ordered polymer-LC structures with tunable optical properties, resulting in many useful photonic applications [START_REF] Tondiglia | Holographic formation of electro-optical polymer-liquid crystal photonic crystals[END_REF][START_REF] Escuti | Holographic photonic crystals[END_REF][START_REF] Liu | Electrically tunable two-dimensional holographic photonic crystals fabricated by a single diffractive element[END_REF][START_REF] Liu | Electrically tunable three-dimensional holographic photonic crystals made of polymer-dispersed liquid crystal[END_REF][START_REF] Zheng | Single-step exposure for two-dimensional electrically-tuneable diffraction grating based on polymer dispersed liquid crystal[END_REF][START_REF] Zheng | Electrically controlled optical choppers based on holographic polymer dispersed liquid crystal gratings[END_REF][START_REF] Chen | Photoinitiation and inhibition under monochromatic green light for storage of colored 3D images in holographic polymer-dispersed liquid crystals[END_REF][START_REF] Zhao | Crosstalk-free patterning of cooperative-thermoresponse images by the synergy of the AIEgen with the liquid crystal[END_REF][START_REF] Luo | Reconfigurable polymer-templated liquid crystal holographic gratings via visible-light recording[END_REF]. This LC-involving patterning process is widely known as the holographic polymer-dispersed liquid crystal (HPDLC) technique [START_REF] Bunning | Holographic polymer-dispersed liquid crystals (H-PDLCs)[END_REF][START_REF] Liu | Holographic polymer-dispersed liquid crystals: Materials, formation, and applications[END_REF].

Similarly, the addition of other additives such as nanoparticles into these holographic formulations has been also investigated [START_REF] Tomita | Holographic manipulation of nanoparticle distribution morphology in nanoparticle-dispersed photopolymers[END_REF][START_REF] Pramitha | Silver-doped photopolymer media for holographic recording[END_REF][START_REF] Li | Holographic kinetics for mixed volume gratings in gold nanoparticles doped photopolymer[END_REF]. Simultaneous addition of LCs and nanoparticles in the HPDLC formulation is expected to achieve the ordered structures with enhanced optical properties [START_REF] Hinojosa | Effects of gold nanoparticles on electro-optical properties of a polymer-dispersed liquid crystal[END_REF][START_REF] Zhang | Electro-optical characteristics of holographic polymer dispersed liquid crystal gratings doped with nanosilver[END_REF][START_REF] Liu | Diffusion kinetics investigations of Nano Ag-doped holographic polymer dispersed liquid crystal gratings[END_REF].

The 1D HPDLC periodic structures (i.e., gratings) are natural DBRs [START_REF] Sutherland | Bragg gratings in an acrylate polymer consisting of periodic polymer-dispersed liquid-crystal planes[END_REF][START_REF] Caputo | Development of a new kind of switchable holographic grating made of liquid-crystal films separated by slices of polymeric material[END_REF][START_REF] Liu | A polarization insensitive 22 optical switch fabricated by liquid crystal-polymer composites[END_REF][START_REF] Liu | Optically switchable gratings based on azo-dye-doped, polymer-dispersed liquid crystals[END_REF][START_REF] Yuan | HPDLC color reflective displays[END_REF][START_REF] Natarajan | Switchable holographic polymer-dispersed liquid crystal reflection gratings based on thiol-ene photopolymerization[END_REF][START_REF] Park | Reflective mode of HPDLC with various structures of polyurethane acrylates[END_REF], which have been extensively investigated for many applications [START_REF] Jakubiak | Electrically switchable lasing from pyrromethene 597 embedded holographic-polymer dispersed liquid crystals[END_REF][START_REF] Hsiao | High contrast switching of distributed-feedback lasing in dye-doped H-PDLC transmission grating structures[END_REF][START_REF] Liu | Low-threshold and narrow-linewidth lasing from dye-doped holographic polymer-dispersed liquid crystal transmission gratings[END_REF][START_REF] Liu | Effect of liquid crystal concentration on the lasing properties of dye-doped holographic polymer-dispersed liquid crystal transmission gratings[END_REF]. Particularly, the 1D HPDLC reflection gratings have long been studied for optical filters and reflective displays [START_REF] Yuan | HPDLC color reflective displays[END_REF]. For the grating fabrication, the nature of coherent light illumination and the viscosity of the prepolymer syrup affect the phase separation of LC and polymer significantly during the photopolymerization process. Usually, upon the LCs partially separate from the polymer, numerous optically anisotropic LC droplets are formed, which will further cause nonnegligible light scattering. As a result, the strong light scattering could kill the phase separation due to polymerization in patterned periodic fashion [START_REF] Liu | Improvement of the diffraction properties in holographic polymer dispersed liquid crystal Bragg gratings[END_REF][START_REF] Caputo | Characterization of the diffraction efficiency of new holographic gratings with a nematic film-polymer-slice sequence structure[END_REF][START_REF] Liu | Effect of surfactant on the electro-optical properties of holographic polymer dispersed liquid crystal Bragg gratings[END_REF], leading to formation of a poor-quality grating. This effect can be greatly suppressed by elevating the polymerization temperature above the clearing point of LCs. The elevated temperature decreases not only the anisotropy effect of LCs, but also the viscosity of the prepolymer syrup that greatly facilitate the phase separation [START_REF] Liu | Improvement of the diffraction properties in holographic polymer dispersed liquid crystal Bragg gratings[END_REF][START_REF] Caputo | Characterization of the diffraction efficiency of new holographic gratings with a nematic film-polymer-slice sequence structure[END_REF]. Another possible way is to add the nonreactive additive. Hsiao and coworkers have reported porous reflection gratings fabricated from an acetone-added HPDLC formulation using visible laser (wavelength: 514 nm) holography [START_REF] Hsiao | Optical microfabrication of highly reflective volume Bragg gratings[END_REF][START_REF] Hsiao | Nanoporous polymeric photonic crystals by emulsion holography[END_REF][START_REF] Hsiao | Influence of non-reactive solvent on optical performance, photopolymerization kinetics and morphology of nanoporous polymer gratings[END_REF].

Applications including humidity sensors have been also demonstrated [START_REF] Hsiao | Organic solvent vapor detection using holographic photopolymer reflection gratings[END_REF][START_REF] Shi | Nanoporous polymeric transmission gratings for high-speed humidity sensing[END_REF][START_REF] Shi | Humidity sensing based on nanoporous polymeric photonic crystals[END_REF].

However, only red-color reflection grating can be achieved at the normal incidence using the visible laser holography technique. In addition, the reflectance of nanoporous DBRs is still not high enough and the reflection band is a bit wide due to the strong scattering, which greatly limits their further applications.

In this work, we report highly reflective nanoporous DBRs with a modified HPDLC formulation via a single-prism-based ultraviolet (UV) interference setup. By changing the interference angle, nanoporous DBRs with three primary colors (red, green, and blue) are achieved after the removal of LCs. Such a kind of nanoporous DBRs is potentially useful for various applications including sensors, filters, holograms, etc.

In our experiments, the prepolymer syrup consists of [START_REF] Liu | A polarization insensitive 22 optical switch fabricated by liquid crystal-polymer composites[END_REF] The experimental setup of the single-prism-based holography system is schematically illustrated in Fig. 1a. An Ar + laser (Model: Innova 306C, Wavelength: 363.6 nm, Coherent, USA) was used as the light source. The output laser beam was subsequentially filtered, expanded, and then collimated with a beam size of 2 cm in diameter. The collimated laser beam was then impinged on a designed prism. At a certain incident angle, total internal reflection took place for the incident laser beam at the glass-air interface of the prism. As a result, the incident beam and its own total internally reflected one interfered each other and formed a periodic interference pattern. The designed prism is made of quartz and has its apex angle of  = 64° with a purpose to achieve the red-color nanoporous DBR films and have the ease of adjusting the incident angle as well. A LC cell filled with the prepolymer syrup was in optical contact with the base of the prism using an index-matching oil to further form the interference pattern inside the cell. Fig. 1b shows the specific optical path of laser in the prism and LC cell. With the help of the matching oil, the laser beam entered the LC cell and totally reflected at the glass-air interface. The prism was placed on a rotation stage for the ease of changing the incident angle  of the collimated laser beam. By doing so, the period of the interference pattern can be conveniently changed.

As a result, we can achieve the polymeric DBR films with different periods. All the experiments were carried out at room temperature. Upon exposure to the interference patterning with the beam intensity of 10 mW/cm 2 for 2 min, the LC cells were then exposed to the UV light for ~30 mins with the exposure intensity of 10 mW/cm 2 and then post-cured for 24 hr in air to ensure the complete polymerization. Upon opening the LC cell to separate the cured sample from one cover slide, the acetone inside the cured sample will evaporate rapidly and a nanoporous polymeric DBR film situated on the other cover slide was then achieved.

It is worth mentioning that when the cell was opened, the DBR films tended to situate on the cover slide that was nearest to the incoming light. We noticed that there was a big difference on the appearance of the sample before and after the cell opening. The sealed LC cells after the UV laser exposure demonstrate a totally transparent state, as shown in Fig. 3a1 and3a2. The total transparency can be mainly attributed to the index matching due to the presence of isotropic solvent (i.e., acetone). However, the nanoporous polymeric DBR film presents a translucent state (see Fig. 3b1 and3b2).

The evaporation of acetone leads to numerous air nanopores inside the polymers, which further cause strong scattering, making the DBR film opaque. Fig. 3a3 and 3a3

shows the measured transmission and reflection spectra of the three samples before and after the cell opening. From Fig. 3a3, it is obvious that the three samples before the cell opening have quite high transmittance (near 80%) and reflectance (near 20%) across the whole visible range. In contrast, after the cell opening, there is significant decrease of the transmission and slight decrease of the reflection, as shown in Fig. 3b3.

Moreover, When the acetone evaporated after the cell opening, air nanopores were created. Meanwhile, the formed liquid crystal nanodroplets were still left inside the polymer matrix. These air nanopores and LC nanodroplets will create a slightly larger index modulation within the film compared to the originally acetone and LC nanodroplets. As a result, upon the acetone evaporation, clearly observable peaks/dips appeared on the measured refection/transmission spectra for blue and green films. It is worth mentioning that we used different references in our spectral measurement. For transmittance measurement, we used the free-space light transmission in air as the reference, while for reflectance measurement, we used an Ag mirror as the reference.

Since the Ag mirror doesn't have 100% reflection, the measured reflection of the DBR films will be higher than the theoretical prediction. As a result, we observed relatively high reflectance.

Figure 3. Transparency-tested photos for a blue film before (a1 and a2) and after (b1 and b2) the cell opening, which were taken from the black (a1 and b1) and white (a2 and b2) backgrounds, respectively.

Measured transmission and reflection spectra for blue, green and red films before (a3) and after (b3) the cell opening, respectively.

We have to mention that although the acetone evaporated after the LC cell was opened, the LCs were still left inside the films. To further remove the LCs trapped inside the films, we used the ethanol to wash away the LCs. When the LC nanodroplets were washed away, complete air nanopores were created, resulting in a much larger index modulation. Therefore, after the removal of LCs, the nanoporous polymeric DBR films demonstrate a colored appearance. In our experiments, at three different incident angles of 7°, 14° and 21°, we have achieved blue, green and red colored DBR films, respectively, as shown in Fig. 4a1&a2, b1&b2 and c1&c2. We can clearly see the shiny blue, green and red colors both macroscopically and microscopically from the samples. It is worth mentioning that a macroscopic look at DBR films shows nonuniform colors with fingerprint-like patterns, which could be mainly attributed to the nonuniform thickness of the index-matching oil in the optical contact. By accurately controlling the uniform thickness of the index-matching oil, this fingerprint-like pattern could be completely avoided, and hence a uniform color can be achieved accordingly. To further check the morphological details of the DBR films, we have carried out the SEM investigation on three colored samples. Fig. 3a3-c3 shows the typically observed cross-sectional SEM images that reflect the detailed morphologies of the nanoporous polymeric DBR films. It can be clearly seen that the films have a multi-layered structure that consists of alternative polymer-rich and nanopore-rich layers. Therefore, they can highly reflect a certain wavelength range of the light. We can also see that as the reflection color changes from blue to red, the period of the DBR films increases gradually. The measured periods of the blue, green and red films are ~350, ~458, and ~663 nm, respectively. A close look at the morphologies reveals that the nanopore diameters range from a few nanometers to tens of nanometers, and the porosity of the polymeric DBR structure is in a range of 20-40%.

As mentioned, the observed visible colors originate from the high reflection of the nanoporous DBR films. The reflection spectra of the films were therefore measured using a UV-Vis-NIR microspectrophotometer. Fig. 5 shows the measured reflection spectra for the blue, green and red samples, respectively. The red and green films have the reflection of larger than 90%, while the blue one has comparatively low reflectance but still higher than 70%. The relatively low reflection for the blue film is mainly attributed to the slightly high absorption of the polymer materials at the blue range. In addition, we can also see from Fig. 5 that the full-width at half-maximum (FWHM) gradually decreases as the reflection band shifts from blue to red. The FWHM of the reflection bands are 46, 42, and 30 nm for blue, green, and red films, respectively. The larger FWHM for the blue film might be caused by the nanopore-induced relatively stronger scattering in the blue range. As the light wavelength in the red range becomes larger, the nanopore-induced scattering becomes weak, hence resulting in a smaller FWHM. polymer-rich and nanopore-rich layers. The other data used in our calculations were tabulated in Table 1. It is worth mentioning that in our calculations, the wavelength dispersion of the refractive indices of the composed materials is not considered because their refractive index changes in the visible range are small enough to be ignored. We have also taken into account the shrinkage of 10-20% of the polymerization and nanopores due to the evaporation of the solvents. It is evident that the calculated curves are in reasonable agreement with the measured ones. In principle, the reflection band of the nanoporous polymeric DBR films is mainly determined by their filling ratio and refractive index contrast between the polymer-rich and nanopore-rich layers at the fixed period. It is quite straightforward that the reflection band will be shifted when the air nanopores are filled with other materials that have different refractive indices. Therefore, the achieved nanoporous polymeric DBRs are potentially useful for optical sensing applications. In this regard, we carried out a refractive index sensing test using our nanoporous polymeric DBRs.

We have investigated the changes of the reflection spectra for the blue, green and red films that infiltrated with four different solvents. The measured spectral changes are shown in Fig. 6. Overall, the measured reflection spectra for these three colored DBR films show a similar trend after the filling with the four liquids. As the refractive index increases, the reflection peak shows an obvious redshift for each film. In addition, the reflection peak intensity decreases as well. Fig. 6d summarizes the peak and intensity change as the function of the refractive index. A linear change can be well observed for the blue and green films. There is slight variation for the red one as the reflection peak becomes low and broad, causing that the measurement is not accurate enough. Using the Bragg's Law, the center wavelength of the reflection spectrum can be determined by [START_REF] Hsiao | Nanoporous polymeric photonic crystals by emulsion holography[END_REF] ( )

Bragg 1 1 2 2 2 n d n d  =+ (1)
where for our case, n1 and d1 are the refractive index and thickness of the polymeric layer, n2 and d2 are the refractive index and thickness of the nanoporous layer. For the refractive index sensing test, the refractive index change of the nanoporous layer (i.e., n2) plays a dominant role. When n2 increases, the reflected center wavelength increases as well. As a result, we observed a clear redshift of the reflection peak. On the other hand, in the extreme case, when n2 increases to be equal to n1 (i.e., reach the complete index matching condition), the DBR film will become totally transparent.

There is no reflection caused by the index mismatch between the polymer-rich and nanopore-rich layers. Therefore, we also observed the intensity decrease of the reflection peak. In summary, we have demonstrated nanoporous polymeric DBR films with periodic nanoscale porosity via a single-prism holography technique. The nanoporous DBR films consisted of alternative polymer-rich and air-nanopore-rich layers. By changing the interfering angle of two laser beams, we have achieved highly reflective nanoporous DBR films with red, green, and blue colors. Their reflectance can be larger than 70% for blue, and 90% for green and red wavelength range. We have also experimentally confirmed that the reflection band of the nanoporous DBR films can be deliberately tuned by further filling different solvents into the pores inside the films, indicating that they are highly sensitive to the refractive index. Such kind of nanoporous polymeric DBR films could be potentially useful for many applications, such as color filters and refractive index sensors.

  ) using the optical adhesive NOA65. The cell gap was controlled to be 10 µm using the SiO2 ball spacers. The prepolymer syrup was then injected into the LC cell via the capillary action.The reflection spectra of nanoporous DBR films were measured using a UV-Vis-NIR microspectrophotometer (CRAIC 20/30 PV™) with a 75W broadband Xenon light source. The probe light beam was focused to have a detecting area of 15 × 15 μm 2 using a 36× objective lens combined with a variable aperture. The morphologies of nanoporous DBR films were investigated by the field emission scanning electron microscopy (FESEM, Merlin, Zeiss) at an acceleration voltage of 5 kV. For refractive index sensing test, four solvents with different refractive indices were used. They were methanol (nm = 1.328), ethanol (ne = 1.361), n-propanol (np = 1.384), n-butanol (nb = 1.399), respectively.

Figure 1 .

 1 Figure 1. (a) Experimental setup for fabricating DBR films. (b) Schematic optical interference inside the LC cell via the single-prism configuration.

Figure 2 .

 2 Figure 2. Simulated interference patterns with three different incident angles of (a) 7°, (b) 14°, and (c) 21°, respectively.

Figure 4 .

 4 Figure 4. Camera photos (a1, b1 and c1), optical microscopic images (a2, b2 and c2), SEM images (a3, b3 and c3) of the blue (a1-a3), green (b1-b3), and red (c1-d3) films, respectively.

Figure 5 .

 5 Figure 5. Measured and simulated reflection spectra for the blue, green, and red films, respectively.

Figure 6 .

 6 Figure 6. Measured spectral changes of the reflection for the blue (a), green (b), and red (c) films that infiltrated with methanol, ethanol, n-propanol, and n-butanol, respectively. (d) Summarized peak and intensity change as the function of the refractive index.

Table 1 .

 1 The estimated thickness of the polymer-rich and nanopore-rich layers for the blue, green and red films.

	Sample	Total Period (nm)	Polymer Layer (nm)	Air Layer (nm)
	Blue	350	204	146
	Green	458	237	221
	Red	663	319	344

Acknowledgements

This work was supported in part by National Natural Science Foundation of China (Grant No. 61805113), Natural Science Foundation of Guangdong Province (Grant No. 2017A030313034 and 2018A030310224), Shenzhen Science and Technology Innovation Commission (Grant No. JCYJ20180305180635082, JCYJ20170817111349280, and GJHZ20180928155207206), Open Fund of State Key Laboratory of Applied Optics (Grant No. SKLAO-201904), and Guangdong Innovative and Entrepreneurial Research Team Program (Grant No. 2017ZT07C071).

We acknowledge the help and discussion from Prof. Vincent Hisao at Taiwan Chi Nan University. The authors also acknowledge the assistance of SUSTech Core Research Facilities.


Muhammad Arslan

Christophe Cruz

come

Understanding the human behaviors in dynamic urban areas of interest

Muhammad Arslan 1 , Christophe Cruz 2 , Dominique Ginhac 3 1,2 Laboratoire d'Informatique de Bourgogne (LIB) -EA 7534, Univ. Bourgogne Franche-Comte, Dijon, France 3 Laboratoire Imagerie et Vision Artificielle (ImViA) -EA 7535, Univ. Bourgogne Franche-Comte, Dijon, France 1 muhammad.arslan@u-bourgogne.fr, 2 christophe.cruz@ubfc.fr, 3 dominique.ginhac@ubfc.fr

Extended Abstract

The fast growth of urban populations across evolving cities is resulting in new types of technical, physical, material, and safety-related challenges and constraints [START_REF] Chen | Understanding the dynamics of urban areas of interest through volunteered geographic information[END_REF]. Humans perceive, move, and perform actions (called behaviors) in different urban areas, and make interactions with the objects, the other humans, and the environment for achieving their goals for attaining the desired level of satisfaction [START_REF] Wagner | Exploring Occupant Behavior in Buildings[END_REF][START_REF] Chen | Definition of occupant behavior in residential buildings and its application to behavior analysis in case studies[END_REF]. More formally, human behaviors are observable actions or reactions of humans in response to external or internal stimuli [START_REF] Wagner | Exploring Occupant Behavior in Buildings[END_REF][START_REF] Hong | An ontology to represent energy-related occupant behavior in buildings. Part I: Introduction to the DNAs framework[END_REF][START_REF] Yan | Occupant behavior modeling for building performance simulation: Current state and future challenges[END_REF]. Over the last decade, human behavior understanding has attracted the strong interest of the researchers [START_REF] Wagner | Exploring Occupant Behavior in Buildings[END_REF] for better monitoring and controlling the facilities in urban areas. This massive interest in behavior understanding is because of its numerous potential applications which can be divided into three major domains as shown in Fig. 1.

To understand the human behaviors for different applications, in the existing literature [START_REF] Wagner | Exploring Occupant Behavior in Buildings[END_REF][START_REF] Chen | Definition of occupant behavior in residential buildings and its application to behavior analysis in case studies[END_REF][START_REF] Hong | An ontology to represent energy-related occupant behavior in buildings. Part I: Introduction to the DNAs framework[END_REF][START_REF] Yan | Occupant behavior modeling for building performance simulation: Current state and future challenges[END_REF], an extensive range of different types of sensors (wired and wireless) for monitoring humans and the environment are present to acquire rich information for modeling human behaviors and their interactions (energy consumption, carbon footprint, etc.). The sensors are often low-cost nodes and typically connected to the internet for data transmission. After collecting the required sensory data, this data needs to be mapped with the corresponding facility information for analyzing human behaviors (Z. [START_REF] Yan | Semantic trajectories: computing and understanding mobility data[END_REF]Arslan et al., 2019). To employ the facility information, building information model (BIM)-based platforms are utilized and preferred over traditional 3D CAD-based systems in the Architecture, Engineering, and Construction (AEC) industry. A BIM model is a digital representation of the physical and functional characteristics of a facility providing a source of shared knowledge of facility information for the facility managers to use and maintain information throughout its lifecycle (NBS-US, 2019). The adoption rate of BIM in the US AEC industry has risen from 17% in 2007 to 71% in 2012. Whereas, in the UK AEC industry, it has improved to 74% in 2018 [START_REF] Maltese | Sustainability assessment through green BIM for environmental, social and economic efficiency[END_REF]. Integration of BIM data with real-time sensory data acquired from the Internet of things (IoT), Radio Frequency Identification (RFID), 3D scanning, GPS, etc. has provided us a centralized digital city model to study different types of human behaviors for the management of urban areas [START_REF] Tang | A review of building information modeling (BIM) and the internet of things (IoT) devices integration: Present status and future trends[END_REF]. However, this process of understanding human behaviors gets more complicated when the dynamicity of the urban areas gets incorporated into the behavior extraction process (Cruz Fig. 1 Applications of understanding human behaviors [START_REF] Wagner | Exploring Occupant Behavior in Buildings[END_REF][START_REF] Chen | Definition of occupant behavior in residential buildings and its application to behavior analysis in case studies[END_REF][START_REF] Hong | An ontology to represent energy-related occupant behavior in buildings. Part I: Introduction to the DNAs framework[END_REF][START_REF] Yan | Occupant behavior modeling for building performance simulation: Current state and future challenges[END_REF] et al., 2017). Our physical environments are dynamic environments that evolve in terms of geometry and contextual information.

To address these requirements of dynamic urban environments, an integrated human behavioral framework is discussed (see Fig. 2). There are six different processes which are involved in constructing this framework that are; 1) Defining human behaviors using DNAS ontology; In DNAS, 'Drivers' indicate the environmental factors which contribute to stimulating occupants to achieve a physical, physiological or psychological need. 'Needs' define the requirements (physical or non-physical) that must be fulfilled to achieve desired satisfaction. 'Actions' are the set of interactions with the urban systems to achieve the desired comfort, and 'Systems' represent the urban area. 2) Capturing behaviors of humans; As human` movements and presence are the prerequisites for any kind of human behavior understanding, our framework specifically deals with capturing the spatio-temporal movements of human in dynamic urban areas. 3) Preprocessing the behaviors; Once the spatio-temporal movements of humans are captured in the form of trajectories, these movements are filtered, segmented (i.e. based on the use case e.g. walk and run segments), and the facility locations having the stay locations of humans are identified (again, it is dependent on the use-case application). 4) Semantic enrichment and storing the movements; After preprocessing the human movements, the movements are contextually enriched with the corresponding urban areas to add more meaning to the collected trajectories using the context extracted from BIM or an OSM file. Later, these spatio-temporal movements are stored in the STriDE (Semantic Trajectories in Dynamic Environments) data model. 5) Modeling behaviors; Based on the use-case requirements, human behavior modeling techniques are applied to the processed spatio-temporal data along with the supplementary data of the human or the facility to achieve the desired classification of the human behaviors. Primarily, there exist four major types of approaches to model human behaviors which are: static-deterministic, staticstochastic, dynamic-deterministic and dynamic-stochastic [START_REF] Wagner | Exploring Occupant Behavior in Buildings[END_REF][START_REF] Yan | Occupant behavior modeling for building performance simulation: Current state and future challenges[END_REF][START_REF] Chen | Definition of occupant behavior in residential buildings and its application to behavior analysis in case studies[END_REF][START_REF] Hong | An ontology to represent energy-related occupant behavior in buildings. Part I: Introduction to the DNAs framework[END_REF]. Using these probabilistic approaches, the computation of probabilities of different movements of humans in terms of the evolving urban area of interest is used for enriching a DNAS ontology for an enhanced understanding of human behaviors by categorizing their actions based on their movements. 6) Visualizing the classified human behaviors; Based on the literature survey (Arslan et al., 2019), BIM is chosen for visualizing the classified movements of humans.

The objective of the framework is to facilitate the development of new systems to enable the standardization of human behaviors` descriptions by incorporating the real-life dynamicity of urban environments during simulations for an improved understanding of human behaviors (Arslan et al., 2019). Here, dynamicity refers to any information about physical space, time and environment utilized for categorizing the situation of humans in urban areas [START_REF] Yan | Occupant behavior modeling for building performance simulation: Current state and future challenges[END_REF]. Changes in the purpose or the position of the locations in the facility will result in different human behaviors [START_REF] Yan | Occupant behavior modeling for building performance simulation: Current state and future challenges[END_REF]. The updated context about the locations along with the previous contextual information will be captured using our framework for studying the behaviors of the humans in detail concerning the changes that occurred in urban areas. The historicization of an urban area along with human behavioral (i.e. location) data will help facility managers for conducting 'cause and effect' analysis (Arslan et al., 2019a). The resulted analysis can be helpful for a broad range of applications in the area of city monitoring [START_REF] Volk | Building Information Modeling (BIM) for existing buildings-Literature review and future needs[END_REF][START_REF] Costin | RFID and BIM-enabled worker location tracking to support real-time building protocol and data visualization[END_REF] relating to the Fig. 2 Human behavioral framework having different processes management of a changing environment which involves moving objects as well as evolving building geometries such as construction planning, risk and hazard assessment, clash detection, construction logistics, building automation, spatial analysis, etc. (Arslan et al., 2019a).

Keywords:

Smart cities; spatio-temporal; BIM; movements; semantics; dynamic environments


Markus Schuhmann

Kathrin Altwegg

Hans Balsiger

Jean-Jacques Berthelier

Johan De Keyser

Stephen A Fuselier

Sébastien Gasc

Tamas I Gombosi

Nora Hänni

Martin Rubin

Thierry Sémon

Chia-Yu Tzou

Susanne F Wampfler

la diffusion de documents scientifiques de niveau

Keywords: Jupiter-Family Comets, Rosetta, Organics, Mass Spectrometry, Solar System, Space Missions

recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Introduction

Until the Rosetta mission, space missions targeting comets were generally brief fly-bys through the coma or tail. As a consequence, although comets are considered to contain many important details about the evolution of our solar system, much of this information was yet to be revealed.

Most data on the chemical composition of cometary nuclei and comae were obtained using remote sensing methods from ground. While ground-based measurement techniques have the advantage of being able to observe many cometary comae, these techniques are usually limited to chemical compounds with relatively large dipole moments. For compounds with small dipole moments, these techniques result in large uncertainties in the molecular abundances.

The Rosetta spacecraft, launched in 2004, followed a new approach of long-term encounter and close-up investigation of comet 67P/Churyumov-Gerasimenko (hereafter 67P). The spacecraft followed the comet throughout most of its orbit around the Sun, thus including observations during essential events like the in-and outbound equinoxes and perihelion. With Rosetta, new data about comets were collected, with many of the physical and chemical properties revealed for the first time. 1,2 For example, the complex shape of the nucleus, its very dark surface, and the huge variability in the coma were surprising. Further, the richness and complexity of the comet's organic composition were detailed during the two-year encounter. Although a fly-by of the Giotto spacecraft already showed the presence of organic molecules up to 100 u/e in comets 3 , the increased mass resolution and sensitivity of the instruments onboard Rosetta revealed a diversity of complex organic molecules far beyond expectations. One year after arrival at the comet, Le Roy et al. 4 published a chemical inventory of molecules found early in the mission, at a solar distance of 3 au, confirming the presence of many organic compounds previously only detected via remote ground-based methods.

The instruments onboard the Rosetta spacecraft included several mass spectrometers. The lander Philae had two mass spectrometers, COSAC 5 and Ptolemy 6 , which collected information directly from the comet surface. Onboard the Rosetta Orbiter, the mass spectrometer COSIMA studied the composition of dust grains in the coma, while ROSINA mainly observed the gaseous phase of the coma. The ROSINA suite included two mass spectrometers that observed the coma at a wide range of distances from the comet surface throughout the two-year encounter. All groups of mass spectrometers performed studies on the chemical composition, and their combined observations exhibited both similarities and differences. Altwegg et al. [START_REF] Altwegg | Organics in comet 67Pa first comparative analysis of mass spectra from ROSINA-DFMS, COSAC and Ptolemy[END_REF] provided a detailed summary of these studies, including discussion of the presence of CH-, HCN-, and CHO-compounds in the comet.

In addition, a detailed study on hydrocarbons across several mission phases was performed by Schuhmann et al. [START_REF] Schuhmann | Aliphatic and Aromatic Hydrocarbons in Comet 67P/Churyumov-Gerasimenko Seen by ROSINA[END_REF] , showing the presence of a range of aliphatic and aromatic compounds in the coma.

In this study, a calibration campaign was performed on several CHO-bearing molecules to obtain their fragmentation patterns and the detector sensitivity for ROSINA-DFMS mass spectrometer. This campaign forms the basis of a detailed identification and quantification of CHO-bearing molecules up to 100 u/e in the coma of comet 67P. Furthermore, by combining the laboratory and space data, the bulk abundances of CHO-bearing molecules relative to water are determined. Data presented here were compared to results from other cometary studies, as well as modelling results of the interstellar medium. Section 2 describes the instrumentation and the laboratory calibration.

Section 3 applies this laboratory calibration to the space data. Section 4 discusses the conclusions.

Methods and Observations

The ROSINA instrument suite onboard of the Rosetta spacecraft consists of three different sensors.

The COmetary Pressure Sensor (COPS), measuring the overall density, the Reflectron-type Time-Of-Flight Mass Spectrometer (RTOF), and the Double Focusing Mass Spectrometer (DFMS). [START_REF] Balsiger | ROSINA -Rosetta Orbiter Spectrometer for Ion and Neutral Analysis[END_REF] This work focuses on data obtained with DFMS. Consisting of a Mattauch-Herzog-configuration 10 , DFMS has a high mass resolution (3000 at 1% peak height) and a high sensitivity, which together with the instrument's long operation time onboard of Rosetta provide ideal conditions for detailed analysis of the comet's organic composition. DFMS makes measurements in high-and lowresolution mode. However, for this study high-resolution mode data are used exclusively. There are two nearly exact copies of DFMS: the flight model onboard of the spacecraft and a laboratory model with nearly identical properties and identical settings to the flight model. All calibrations discussed below were performed with the laboratory model, while the flight model collected space data.

Laboratory Calibration

For calibration the University of Bern calibration facility CASYMIR (Calibration System for the Mass Spectrometer Instrument ROSINA) was used. This facility was developed for measurements under space-equivalent conditions. In particular, it is operated in very low density conditions from 10 -10 to 10 -6 mbar. 11, [START_REF] Graf | A Cometary Neutral Gas Simulator for Gas Dynamic Sensor and Mass Spectrometer Calibration[END_REF] For calibration of DFMS, a series of measurements were performed at three different pressure levels, ranging from 10 -9 to 10 -7 mbar, to derive the fragmentation behavior of the compounds and the sensitivity of the instrument to the compounds.

Calibration of Fragmentation

DFMS uses electron impact ionization to ionize very low energy cometary molecules. Besides charging neutral molecules positively, molecular chains may be fragmented into smaller pieces of unsaturated or even saturated species through this electron bombardment. Simulation and preflight testing of the DFMS ion source showed that the best performance was achieved at an ionization energy of 45 eV. This energy is lower than the 70 eV ionization energy traditionally used in the laboratory to develop molecular fragmentation databases. Fragmentation patterns of molecules are affected by the ionization energy. The patterns directly obtained in the DFMS calibrations were thus the primary source for this study. More traditional databases provided fragmentation patterns as a secondary option when DFMS calibration measurements could not be performed. There were several reasons why some calibration measurements could not be performed: (I) Compounds had vapor pressures that were too low, so that vacuum conditions applied would not be sufficient for creating the level of gas phase required for calibration. (II) Compounds vapor pressures were too high, in some cases leading to unstable pressure conditions, difficult to correct with the thermal valve control elements used, or (III) Compounds were corrosive and/or poisonous and therefore too dangerous to study. For compounds not measured in the DFMS, the Database of the National Institute for Standards and Technology 13 (further referred to as NIST) were used. DFMS fragmentation patterns are obtained either relative to the total amount of ions or to the most abundant fragment. An example of the fragmentation behaviour of methanol in DFMS is provided in Table 1 with the corresponding fragmentation pattern in Figure 1.A list of all fragmentation values used for this study is found in the appendix.

High quality fragmentation patterns are essential for identification of molecules in the DFMS space data, as fragmentation processes can also lead to formation of molecules present in the volatile phase of the comet (e.g. CH2O from CH3OH). This substantially complicates the identification of CHO-bearing molecules in the coma. Only molecules showing at least the parent and major fragments, as determined from the lab calibration, are classified here as present in the space data.

In addition, any impact from other compounds sharing fragments of the same mass must be subtracted according to the fragmentation pattern.

Calibration of DFMS Sensitivity

The DFMS detector consists of a microchannel plate (MCP) with a position sensitive linear anode (Linear Electron Detector Array, LEDA) that has a resolution of 512 pixels. Detailed information on the detector is found in Nevejans et al. [START_REF] Nevejans | The LEDA512 Integrated Circuit Anode Array for the Analog Recording of Mass Spectra[END_REF] and Balsiger et al. [START_REF] Balsiger | ROSINA -Rosetta Orbiter Spectrometer for Ion and Neutral Analysis[END_REF] . The detector together with the analyzer show a species (mass)-dependent sensitivity for which a compensation factor, the socalled sensitivity factor (Appendix A), is derived from calibration measurements. This factor (given in cm 3 ) relates the density of the compound in DFMS's ion source multiplied by the ratio of the electron emission current to the ion current on the detector at each pressure level. A detailed description of the DFMS sensitivity derivation is found in Le Roy et al. All calibration compounds are present in liquid phase at atmospheric pressure and room temperature, thus requiring a transformation into gaseous phase for the DFMS measurements. In the calibration chamber, samples were exposed to vacuum (< 10 -4 mbar) at temperatures of < 353.15 K, allowing higher evaporation rates. For this study, calibration of a broad range of CHObearing molecules was performed. However, as the calibration capacity is limited, the main focus was on CHO-compounds that were detected in the coma of the comet cf 7 and on compounds expected to be present in the coma due to their presence in the interstellar medium or in other comets.

Data obtained from separately performed background measurements were subtracted from the calibration output, removing any contribution and contamination from previous calibration runs in CASYMIR or in the instrument. Sensitivity is derived from all ions detected as a function of pressure level. Sensitivity calculation for three different pressure levels showed a linear correlation. This correlation is approximated using least-square fit in the form of a linear regression, with the slope representing the sensitivity factor.

Space Data Selection

For this work, DFMS space data from the May 2015 post-equinox period (date: May 21 to June 1, 2015; mission phase: ESC2-MTP016, STP058 -VSTP111) were selected. This mission phase was considered to be the best representation of the organic bulk composition of comet 67P. At a heliocentric distance of 1.53 au, the comet was considered close enough to the Sun to enable a sufficient outgassing rate, whilst unstable outburst conditions such as those occurring around perihelion could be avoided. The distance between the comet and the spacecraft was relatively large (~200 km), which is known to have an impact on the DFMS spectra intensities. For the derived abundances, the time period from May 21 to June 1 was used, as the spacecraft was located over the comet's southern (summer) hemisphere. Data obtained during spacecraft maneuvers were excluded as previous studies indicate that maneuvers lead to the desorption of larger amounts of molecules from the spacecraft surface. 15

Data Treatment

The DFMS raw spectra consist of 512 abscissa points (representing the LEDA pixels) with the number of detector counts at each pixel. DFMS data analysis requires a use of a mass scale (m/z) and a calculation of the number of ions from the detector counts. The number of ions is generally calculated over 20s, referring to the default 20s integration time. Also, an individual pixel gain is applied, compensating for local variation in the sensitivity of the LEDA pixels. In addition, a general gain and a mass dependent detector yield value are applied. The number of ions per species is derived from integration of the peak over each pixel. However, in this study a single-Gaussian fitting routine was used to fit the peak shapes (fig. 2) as this represents the core of the peak very well and is a simpler and preferable procedure to separate overlapping peaks in spectra when compared to summing the number of ions over each pixel around the peak. The relative abundances of molecules were calculated using both the DFMS laboratory and space data. Therefore, fragmentation and sensitivity, as obtained in calibration, and the number of ions per species, as obtained from the space data, are taken into account. The procedure for computing the abundance ratios is described in detail for DFMS by Gasc et al. representing formic acid (HCOOH) and ethanol (C2H5OH). In terms of fragmentation, acetaldehyde and ethanol show a pattern similar to methanol and formaldehyde, that is, a loss of two single-bonded H-atoms under electron impact causes formation of C2H4O from C2H6O. But again, this process is not sufficient to explain the presence of all C2H4O and the high levels of C2H4O essentially confirm that acetaldehyde is present in the coma in May 2015. The peak at 42

u/e is identified as C2H2O. However, in this case, a fragmentation contribution from almost all CHO-bearing molecules of higher mass present is possible. Therefore C2H2O is categorized here as a fragmentation product. For the selected measurement period, the presence of propanol and butanol is considered to be likely, as data from other periods shows propanol and butanol in the coma of the comet 67P. Le Roy et al. 4 and Altwegg et al. [START_REF] Altwegg | Organics in comet 67Pa first comparative analysis of mass spectra from ROSINA-DFMS, COSAC and Ptolemy[END_REF] confirmed the presence of propanol.

However, neither propanol nor butanol were detected in this study from only the May 2015 data.

The failure to detect butanol here is considered to be due to low abundances of both molecules and the large distance between the comet and the spacecraft, resulting in lower intensities. Both species are thus expected to be present, but they are considered hidden in the background and are excluded from the calculation of the relative abundances. At mass 58 u/e, C3H6O is present (fig. 3.e). For this peak, identification based on structural features is difficult as several isomers could be present.

The most feasible candidate is probably acetone, in agreement with Altwegg et al. [START_REF] Altwegg | Organics in comet 67Pa first comparative analysis of mass spectra from ROSINA-DFMS, COSAC and Ptolemy[END_REF] , who identified acetone as well. Furthermore, the presence of acetone in the interstellar medium and on comets is very well documented, and therefore in contrast to other isomers like propanal that could be present. Differentiation of both compounds via fragmentation pattern should be possible. Propanal produces high quantities of fragmentation products around 29 u/e, while acetone shows only very little fragmentation around this mass/charge ratio. Instead, for acetone, high quantities of fragmentation products occur at 43 u/e. Comparing DFMS spectra at 29 u/e and 43 u/e from May 2015, the CHO-peak at 29 u/e appears to be higher than the peak at 43 u/e by roughly a factor 10.

Hence, the fragmentation values and the sensitivity of propanal were used for calculation of the relative abundance. However, the peak at 29u/e has contributions from fragmentation from all major CHO-bearing compounds present, hence doesn´t allow unequivocal isomer identification.

Most likely there is a mixture of both (or more) isomers and the exact ratio must remain undetermined.

A peak at 60 u/e refers to another CHO-bearing compound (fig. 3.f). It confirms the presence of acetic acid, which proves together with above discussed formic acid, the existence of carboxylic acids in the coma of the comet 67P. At 62 u/e, low amounts of C2H6O2 are present (fig. 3.g), potentially belonging to ethylene glycol.

Another peak at 72 u/e (fig. 3.h) is identified as C4H8O. For this peak, the fragmentation pattern and sensitivity of butanal were used in the analysis. Again, one of several isomers, or a combination of them, can be responsible for the peak. In addition to butanal, there is the possible presence of its isomers butanaone and propanal, 2-methyl-(isobutanal). The NIST fragmentation pattern of butanal and isobutanal are similar. Besides the rather small differences in the fragmental abundances, the only major difference is the lack of a fragmentation product at 44 u/e in the isobutanal-pattern. Hence, differentiation between these isomers is not possible under the present circumstances.

Another peak at 74 u/e (fig. 3.i) is identified as C3H6O2, which would be a known fragmentation product of propylene glycol. However, propylene glycol could not be identified in this study (see following section), instead there may be the presence of methyl acetate or an isomer such as methyl formate at 74 u/e. Methyl formate is a good candidate because of its presence in the ISM.

However, the NIST fragmentation pattern of methyl formate shows a different picture, as no observable parent peak or peaks of close-by fragments are present in the fragmentation pattern. In contrast, the NIST fragmentation pattern of methyl acetate clearly indicates the presence of a parent peak. As this is in agreement with the DFMS space data, the fragmentation pattern and sensitivity of methyl acetate were used for this study.

3.2 Presence of CHO-Species Higher than 75 u/e Species of higher masses are difficult to investigate as fluxes from the comet decrease with increasing masses and the instrument sensitivities decline. Besides that, the larger distances between the comet and the spacecraft in May 2015 result in rather lower intensities compared to other mission phases. Nevertheless, the identification campaign was performed up to 100 u/e. The conditions for detection of glycolic acid and propylene glycol were considered to be especially favorable in May 2015 compared to other periods when strong contributions of C2H4OS and C6H4

prevented their detection. However, their presence could not be confirmed for the May 2015 period and both compounds were considered to be absent or below the estimated detection limit. Further investigations were made regarding the presence of glycerol at 92 u/e. Several investigations of glycerol in the comet 67P have been performed previously. Altwegg et al. [START_REF] Altwegg | Organics in comet 67Pa first comparative analysis of mass spectra from ROSINA-DFMS, COSAC and Ptolemy[END_REF] suggested the presence of toluene instead of glycerol. In fact, the mass/charge ratio of toluene and glycerol are comparable, even the measurements in the DFMS high-resolution mode were not sufficient for clear separation of both peaks. The calibration of glycerol preformed in this study; however, revealed that fragmentation behavior of glycerol is not consistent with the data from May 2015, as major fragments are missing. This confirms the findings of Altwegg et al. [START_REF] Altwegg | Organics in comet 67Pa first comparative analysis of mass spectra from ROSINA-DFMS, COSAC and Ptolemy[END_REF] , suggesting the peak at 92 u/e is due to toluene instead. elements might appear. Isomerism may occur but cannot be considered here. Therefore, the names of the species identified here may not be unique unless identified in earlier studies; see also 4,7 . 17

Bulk Abundances of CHO-bearing Molecules

Studies of the abundances of CHO-bearing compounds using ground-observations can be found for most types of comets. Also, Le Roy et al. 4 calculated abundances relative to water for the comet 67P at larger heliocentric distance in 2014, shortly after the spacecraft's arrival. Table 3 lists the abundances relative to water for all CHO-molecules detected up to 100 u/e in the May 2015 postequinox period. Formaldehyde and methanol show the highest abundances. Their abundances of 0.32% and 0.21%, respectively are within 2σ of each other. Other CHO-molecules appear to be lower by approximately a factor of 10 compared to methanol and formaldehyde. They cover a relatively small range from 0.01% to 0.05% relative to water. In this second group, ethanol and acetaldehyde show relatively high abundances of close to 0.05%. Methyl acetate, propanal (or acetone), acetic acid and butanal show much lower abundances of less than 0.01% relative to water. remote sensing spectroscopic observations. Formaldehyde and Methanol show the highest abundances, which is in agreement with our study, but broadly range between 0.6 -6.2 % (methanol-to-water) and 0.13-2.4 % (formaldehyde-to-water). For other CHO-compounds, the upper limit appears to be significantly lower, e.g. 0.18 % for formic acid and 0.12% for ethanol.

Bockelée-Morvan and Biver 17 further point out the differences of molecular abundances between various types of comets. This is in agreement with values we found in further existing literature:

For comet Halley (1P) the ratios of methanol to water were found to be 1.8% by Bockelée-Morvan et al. [START_REF] Bockelée-Morvan | On the Origin of the 3.2 to 3.6-Micron Emission Features in Comets[END_REF] and Eberhardt et al. [START_REF] Eberhardt | Methanol and hydrogen sulfide in comet P/Halley[END_REF] , and 1.7 % by Rubin et al. [START_REF] Rubin | Monte Carlo Modeling of Neutral Gas and Dust in the Coma of Comet 1P/Halley[END_REF] . The ratio of formaldehyde to water ranges in Halley-type comets from 1.5% 20 to 4% [START_REF] Bockelée-Morvan | On the Origin of the 3.2 to 3.6-Micron Emission Features in Comets[END_REF] . Furthermore, methanol-to-water ratios for several long-period comets range from 1.48% in comet C2/2012 F6 21 , to 3.9% in comet C/2001 A2 [START_REF] Magee-Sauer | The Organic Composition of Comet C/2001 A2 (LINEAR). I. Evidence for an Unusual Organic Chemistry[END_REF] .

For the majority of long-periodic comets methanol-to-water ratios around 2.5% were derived, such as for comet Hale-Bopp (C/1995) by Bockelée-Morvan et al. [START_REF] Bockelée-Morvan | New Molecules found in Comet C/1995 O1 (Hale-Bopp). Investigating the Link between Cometary and Interstellar Material[END_REF] . For these long period comets, formaldehyde-to-water ratios are generally lower, around 1% for comet Hale-Bopp 23 and for comet Hyakutake (C/1996-B2) [START_REF] Biver | Radio Wavelength Molecular Observations of Comets C/1999 T1 (McNaught-Hartley)[END_REF] .

The ratios of several other CHO-bearing molecules to water, including formic acid, acetaldehyde, and ethylene glycol, have been determined for long-periodic comets. Generally, the highest abundances for ethylene glycol-to-water are around 0.3%: Comet Hale-Bopp (0.25%) [START_REF] Crovisier | Ethylene Glycol in Comet C/1995 O1 (Hale-Bopp)[END_REF] , comet Lemmon (0.24%), and comet Lovejoy (0.35%) [START_REF] Biver | Complex Organic Molecules in Comets C/2012 F6 (Lemmon) and C/2013 R1 (Lovejoy): Detection of Ethylene Glycol and Formamide[END_REF] . Besides these molecules, most other CHObearing molecules in long periodic comets show abundances lower than 0.1% with respect to water. Relevant for this study are ratios derived from Jupiter-family comets, given similar short periodic orbit and evolutionary conditions to the comet 67P. In particular, Le Roy et al. 4 noted differences among the hemispheres of the comet and they used data from a period when the comet was beyond 3.1 au from the Sun and when the southern hemisphere was in winter. Furthermore, among others Hässig et al. [START_REF] Hässig | Time variability and heterogeneity in the coma of 67P/Churyumov-Gerasimenko[END_REF] , Fougere et al. [START_REF] Fougere | Direct Simulation Monte Carlo modelling of the major species in the coma of comet 67P/Churyumov-Gerasimenko[END_REF] , Läuter et al. [START_REF] Läuter | Surface localization of gas sources on comet 67P/Churyumov-Gerasimenko based on DFMS/COPS data[END_REF] , and Biver et al. [START_REF] Biver | Long-term monitoring of the outgassing and composition of comet 67P/Churyumov-Gerasimenko with the Rosetta/MIRO instrument[END_REF] showed how relative abundances of a multitude of species varied as a function of heliocentric distance and spacecraft location at 67P. In earlier ROSINA studies such as Altwegg et al. [START_REF] Altwegg | Organics in comet 67Pa first comparative analysis of mass spectra from ROSINA-DFMS, COSAC and Ptolemy[END_REF] , the presence of large amounts of unsaturated species was discussed. Furthermore, this study shows that fragmentation processes in the ion source are not sufficient for production of this amounts of molecules with unsaturated C=O bonds occurring in form of aldehydes and carboxylic acids. The presence of these compounds is important as it might provide hints at a common origin of the material in comets and the ISM.

Hasegawa and Herbst 32 show models of gas-grain chemistry in dense interstellar clouds with complex organic molecules. Besides a high number of different organic species investigated in this study, some CHO-bearing molecules were taken into account. Figure 4

Conclusions

A laboratory calibration campaign of ROSINA-DFMS on CHO-bearing molecules was conducted to determine the molecular fragmentation pattern and sensitivity of the instrument to the molecules. Based on these lab results, a detailed identification and quantification campaign of CHO-bearing molecules in the DFMS space data for May 2015 post-equinox period was performed. This campaign revealed the presence of CHO-bearing molecules most-likely in the form of aldehydes, alcohols, and carboxylic-acids. The abundances of all compounds present were calculated relative to water. Formaldehyde and methanol are the most abundant of all CHObearing molecules present, followed by acetaldehyde and ethanol. Other molecules present are significantly less abundant and have a maximum of 0.01 % abundance relative to water. The ratios of methanol and formaldehyde with respect to water are similar to values derived for other Jupiter-Family comets but are at the lower end of the range. The abundances are generally lower than the ones derived by Le Roy et al. 4 for comet 67P using DFMS data from early in the encounter when the comet was far from the Sun and relative water outgassing was lower. cf. [START_REF] Fray | Sublimation of ices of astrophysical interest: A bibliographic review[END_REF] The observations here are exclusively from a period when the comet was much closer to the Sun and the southern hemisphere was in summer.

The time period here is closer to perihelion, when most ground-based observations of comets occur. Therefore, based on the comparison with ground-based measurements of other comets, 67P

is classified as relatively poor in CHO-bearing species. Therefore, it is important that there are future rendezvous with other Jupiter-family comets and detailed comparison between in situ and ground-based observations. Similarly, ground-based observations of comets far from the perihelion would help confirm the differences in the in situ measurements of CHO-bearing species.

Figure 1 :

 1 Figure 1: DFMS fragmentation pattern of methanol. Bars represent the degree of fragmentation relative to the most abundant fragment (CH3).

Figure 2 :

 2 Figure 2: CH4O peak in DFMS high-resolution spectra. For analysis of the CHO-bearing molecules a single-Gaussian fitting routine was applied in the spectra.

16 3.

 16 Application to Space Data 3.1 Presence of CHO-species lighter than 75 u/e Analysis of the DFMS space data from May 2015 inbound post-equinox reveals the presence of several CHO-bearing compounds. High amounts of CH2O and CH4O are found in the DFMS data at 30 u/e (fig.3.a) and 32 u/e (fig.3.b) and can be identified as methanol (CH3OH) and formaldehyde (H2CO). Formaldehyde is also a fragmentation product of methanol; however, as shown in Figure 1, the level of fragmentation of methanol into formaldehyde is not high enough to explain the large amounts present. At 44 u/e, smaller amounts of C2H4O can be found (fig.3.c) and identified as acetaldehyde (CH3CHO), while CH2O2 and C2H6O are present at 46 u/e (fig.3.d),

Figure 3

 3 Figure 3.a-i: DFMS spectra of CHO-bearing molecules detected in May 2015 post-equinox-period

  For those comets, relative abundance determinations for minor CHO-bearing molecules are rare. Le Roy et al.4 performed a study on the comet 67P, revealing abundances of 0.01% or lower for minor CHO-bearing molecules.Like the long period comets, the abundances of methanol and formaldehyde in Jupiter-family comets are discussed broadly in the literature. Dello Russo et al.[START_REF] Russo | Emerging trends and a comet taxonomy based on the volatile chemistry measured in thirty comets with high-resolution infrared spectroscopy between 1997 and[END_REF] provides a detailed and recent overview on molecular abundances of CHO-bearing molecules relative to water in Jupiter-family comets. The ratios appear to vary significantly among different Jupiter-family comets. The average methanol-to-water ratios range from 0.49 % in comet 73P/Schwassmann-Wachmann-C and 0.54 % in 73P/Schwassmann-Wachmann-B up to 3.48 % in comet 2P/Encke. Values available for comet 67P in 2014 show ratios on the lower side of this range, 0.31 to 0.55%, depending on the comet hemisphere.4 The abundance of methanol in May 2015 is with 0.21%, hence low compared to the 2014 measurement and low compared to other Jupiter family comets. The differences between our results and Le Roy et al.4 implies changes in the ratios relative to water over the Rosetta investigation period. This is caused by changing conditions when the spacecraft first encountered the comet and the post-equinox and pre-perihelion period in May 2015 analyzed here.

Figure 4 .

 4 Figure 4. Comparison of Hasegawa/Herbst ISM model and ROSINA-DFMS. The plot

  shows a comparison of the CHO-bearing molecules investigated in this study and in the Hasegawa/Herbst model (using results from Model A from Hasegawa and Herbst 32 ). The values are normalized to methanol (CH3OH) for better comparison and isomers occurring in the Hasegawa/Herbst model are appended to the compounds of same mass. Comparison of the Hasegawa/Herbst model to the ROSINA-DFMS results confirms that the investigated compounds are present in both studies, including saturated and unsaturated molecules. As can be seen in Figure 4, results from the Hasegawa/Herbst model show higher amounts for CHO, H2CO, and CH2O2, with formaldehyde being especially enriched compared to ROSINA. CH2HO has similar abundance, while formic acid (C2H4O) and ethanol (C2H5OH) are more abundant in the ROSINA data.

Finally

  , concerning CHO-bearing species in comets and the ISM, abundant unsaturated CHObearing species were detected in form of aldehydes and carboxylic acids with unsaturated C=O bonds in comet 67P. A comparison to the model of Hasegawa and Herbst 32 , showing the evolution of organics in the ISM, highlights the similarities to the ROSINA data from comet 67P on CHObearing molecules, with a difference in abundance especially for formaldehyde. This difference might indicate the conservation of CHO-compounds in comets from earlier solar system formation stages.

Table 1 : DFMS fragmentation of methanol

 1

	Mass (u)	Fragment	Abundance (%)	Error (%)
	13	CH	4.05	1.2
	14	CH2	12.60	3.6
	15	CH3	100.00	0.0
	16	O	0.52	0.2
	17	OH	1.23	0.4
	28	CO	14.89	4.3
	29	CHO	50.09	9.8
	30	CH2O	7.22	2.7
	31	CH3O	63.12	11.9
	32	CH4O	39.18	9.7
	33	13 CH4O	0.52	0.2

4

Table 2 : CHO-bearing molecules present in May 2015

 2 Table 2 provides a list with all CHO-bearing molecules detected in May 2015.

	Integer mass (u)	compound	Interpretation
	29	COH	Fragment
	30	CH2O	Formaldehyde
	31	CH3O	Fragment
	32	CH4O	Methanol
	33	CH5O	Fragment
	41	C2HO	Fragment
	42	C2H2O	Fragment
	43	C2H3O	Fragment
	44	C2H4O	Acetaldehyde
	45	C2H5O	Fragment
	46	CH2O2	Formic Acid
	46	C2H6O	Ethanol
	57	C3H5O	Fragment
	58	C3H6O	Propanal or Acetone*
	60	C2H4O2	Acetic Acid
	62	C2H6O2	Ethylene Glycol
	70	C4H6O	Fragment
	71	C4H7O	Fragment
	72	C4H8O	Butanal*
	74	C3H6O2	Methyl Acetate*

* Molecular identification is based on mass/charge ratios, hence the definition of the exact structural group is difficult as several stable or semi-stable modifications with the same set of

Table 3 : Abundances of CHO-bearing molecules (rel. to H2O)

 3 Comparison to Other Comets and the ISM Abundances of methanol and formaldehyde have been summarized broadly for several types of comets. Most recently, Bockelée-Morvan and Biver 17 published a review of molecular abundances in cometary atmospheres, in which values of several CHO-bearing molecules are summarized from

	Integer Mass (u)	CHO-molecule	Abundance
	30	Formaldehyde	(3.2 ± 1.0)•10 -3
	32	Methanol	(2.1 ± 0.6)•10 -3
	44	Acetaldehyde	(4.7 ± 1.7)•10 -4
	46	Formic acid	(1.3 ± 0.8)•10 -4
	46	Ethanol	(3.9 ± 2.3)•10 -4
	58	Propanal (Acetone)	(4.7 ± 2.4)•10 -5
	60	Acetic acid	(3.4 ± 2.0)•10 -5
	62	Ethylene glycol	(1.1 ± 0.7)•10 -4
	72	Butanal	(9.9 ± 3.2)•10 -5
	74	Methyl acetate	(2.1 ± 0.7)•10 -5
	18	Water	1.00


D Botina-Monsalve
email: johan.botina-monsalve@u-bourgogne.fr

Y Benezeth

R Macwan

P Pierrart

F Parra

K Nakamura

R Gomez

J Miteran

Long Short-Term Memory Deep-Filter in Remote Photoplethysmography

Remote photoplethysmography (rPPG) is a recent technique for estimating heart rate by analyzing subtle skin color variations using regular cameras. As multiple noise sources can pollute the estimated signal, post-processing techniques, such as bandpass filtering, are generally used. However, it is often possible to see alterations in the filtered signal that have not been suppressed, although an experienced eye can easily identify them. From this observation, we propose in this work to use an LSTM network to filter the rPPG signal. The network is able to learn the characteristic shape of the rPPG signal and especially its temporal structure, which is not possible with the usual signal processingbased filtering methods. The results of this study, obtained on a public database, have demonstrated that the proposed deep-learning-based filtering method outperforms the regular post-processing ones in terms of signal quality and accuracy of heart rate estimation.

Introduction

Heart rate (HR) measurement has made it possible to monitor the behavior of the heart in humans, and analyze physiological conditions. There are mainly two ways to measure HR, by means of electrocardiography (ECG) and photoplethysmography (PPG). ECG measures the electrical field induced by the heart activity in the chest, and PPG measures reflection and absorption of light on skin tissues, which indirectly captures the blood volume variations caused by the periodic beating of the heart. During cardiac contraction (systole), the volume of blood increases, and during the cardiac relaxation (diastole), the volume of blood decreases. These contact-based HR measurement methods may cause discomfort, hygiene issue, or even be impossible on fragile skins [START_REF] Qiu | Evm-cnn: Real-time contactless heart rate estimation from facial video[END_REF].

Verkruysse et al. [START_REF] Verkruysse | Remote plethysmographic imaging using ambient light[END_REF] demonstrated that these perfusion variations, associated with heartbeats are not noticeable by the naked eye, but can be measured from a standard video camera to measure PPG signals remotely, using ambient light as an illumination source. This remote Photoplethysmography (rPPG) technique is a low-cost and noninvasive way to measure HR. RPPG aims to estimate color changes in the frames. It is also worth mentioning here that there is another technique to obtain the HR remotely using a camera, by Ballistography (BCG) such as in [START_REF] Pino | Bcg algorithm for unobtrusive heart rate monitoring[END_REF]. BCG measures the periodic head movements due to ejections of blood into great vessels along with each heartbeat and, that way extracts HR signals.

With PPG or rPPG measurement, several biomedical parameters can potentially be computed: vascular occlusion, peripheral vasomotor activity, breathing rate, blood pressure by pulse transit time estimation, blood oxygen level, heart rate variability (HRV), and obviously heart rate [START_REF] Allen | Photoplethysmography and its application in clinical physiological measurement[END_REF]. As a consequence, the applications are numerous, including control of vital signs in the elderly and newborns, mixed reality, lie detection in criminals, physiological measurements of drivers, face anti-spoofing, and automatic skin detection, to mention a few [START_REF] Chaichulee | Cardiorespiratory signal extraction from video camera data for continuous non-contact vital sign monitoring using deep learning[END_REF][START_REF] Bousefsaf | 3d convolutional neural networks for remote pulse rate measurement and mapping from facial video[END_REF][START_REF] Allen | Photoplethysmography and its application in clinical physiological measurement[END_REF]. First of all, it can be noted that the advent of non-contact measurements has opened the way to many new applications. Secondly, it is also important to note here that many of these applications use a fairly advanced analysis of the temporal signal (e.g. detection of systolic and diastolic peaks [START_REF] Mcduff | Remote detection of photoplethysmographic systolic and diastolic peaks using a digital camera[END_REF]), which is considerably more complex than frequency analysis over long periods (usually ten to twenty seconds to measure heart rate for example).

In the literature, multiple ways of rPPG estimation can be found. Initially, only the green channel was used [START_REF] Verkruysse | Remote plethysmographic imaging using ambient light[END_REF]. Later approaches can be categorised in two groups: the first ones are based on Blind Source Separation techniques (BSS): ICA [START_REF] Hyvarinen | Fast and robust fixed-point algorithms for independent component analysis[END_REF], PCA [START_REF] Lewandowska | Measuring pulse rate with a webcam-a non-contact method for evaluating cardiac activity[END_REF], PVM [START_REF] Macwan | Periodic variance maximization using generalized eigenvalue decomposition applied to remote photoplethysmography estimation[END_REF], EVM [START_REF] Wu | Eulerian video magnification for revealing subtle changes in the world[END_REF]. The second ones are based on a light interaction model to determine a projection vector: CHROM [START_REF] De Haan | Robust pulse rate from chrominance-based rppg[END_REF], POS [START_REF] Wang | Algorithmic principles of remote ppg[END_REF], PBV [START_REF] De Haan | Improved motion robustness of remote-ppg by using the blood volume pulse signature[END_REF]. Readers interested in learning more about these techniques can refer to the state of the art reviews presented in [START_REF] Mcduff | A survey of remote optical photoplethysmographic imaging methods[END_REF][START_REF] Hassan | Heart rate estimation using facial video: A review[END_REF][START_REF] Sun | Photoplethysmography revisited: from contact to noncontact, from point to imaging[END_REF]. End-to-end approaches based on deep learning have also been used recently [START_REF] Chen | Deepphys: Video-based physiological measurement using convolutional attention networks[END_REF][START_REF] Qiu | Evm-cnn: Real-time contactless heart rate estimation from facial video[END_REF][START_REF] Chaichulee | Cardiorespiratory signal extraction from video camera data for continuous non-contact vital sign monitoring using deep learning[END_REF][START_REF] Bousefsaf | 3d convolutional neural networks for remote pulse rate measurement and mapping from facial video[END_REF][START_REF] Yu | Remote photoplethysmograph signal measurement from facial videos using spatio-temporal networks[END_REF]. One of the main advantages of these CNN-based measurements is that it allows achieving good results without the need for the designer to analyze the problem in depth [START_REF] Zhan | Analysis of cnnbased remote-ppg to understand limitations and sensitivities[END_REF]. In addition, it is no longer necessary to use a pipeline-based framework where regions of interest (ROI) are first detected and tracked over frames, RGB channels are then combined to estimate the pulse signal, which is filtered and analyzed to extract physiological parameters such as heart rate or respiration rate. As a consequence, CNN-based approaches are less prone to error propagation in the pipeline. However, as noted in [START_REF] Zhan | Analysis of cnnbased remote-ppg to understand limitations and sensitivities[END_REF], recent works in this field have focused on performance rather than understanding. Consequently, it is often hard to predict the limitations of the system, and it is well known that the training dataset used is critical.

RPPG signals estimated from a video usually contain some noise due to motion, illumination variations, internal noise of the digital camera, and also due to the estimation technique. This generally calls for post-processing steps to smooth the signal and remove unnecessary information, such as frequencies out of the regular physiological range of interest. This smoothing operation is habitually performed by a bandpass filter (e.g. in [START_REF] Niu | Rhythmnet: Endto-end heart rate estimation from face via spatial-temporal representation[END_REF]). Some works also used wavelet filters [START_REF] Bousefsaf | Continuous wavelet filtering on webcam photoplethysmographic signals to remotely assess the instantaneous heart rate[END_REF][START_REF] Macwan | Periodic variance maximization using generalized eigenvalue decomposition applied to remote photoplethysmography estimation[END_REF] or Singular Spectrum Analysis (SSA) [START_REF] Wang | Quality metric for camera-based pulse rate monitoring in fitness exercise[END_REF]. Although these filtering methods do smooth the signals, they do not necessarily eliminate particular signal alteration, which can, however, be easily identified by experts. An example of one rPPG signal extracted and filtered by a band pass filter and a wavelet filter is presented in figure 1. RPPG signals filtered by both methods show significant alterations compared with their PPG ground truth signal. This kind of noise may alter the accuracy of heart rate measurement but more gravely prevent further advanced analysis of the rPPG signals that are based on pulse shape characteristics or peak detection on the temporal signal. For example, it has been shown that even a single heart period artifact, within a 2-minutes recording, can lead to errors of heart rate variability features that are considerably larger than typical effect size in psychological studies [START_REF] Berntson | Ecg artifacts and heart period variability: don't miss a beat![END_REF]. Therefore, in this work, we want to benefit from the advantages of deep learning-based methods to improve the filtering of rPPG signals while relying on a handcrafted pipeline thanks to which it is possible to control the system limits better.

Recurrent Neural Networks (RNNs) have been used in numerous applications with sequential information as inputs, like Natural Language Processing (NLP) and speech processing. RNNs are relevant in applications where the structure embedded in the data sequence transfers valuable knowledge, much like an expert would do to identify the wrong peaks in an rPPG signal. However, these networks show a problem during backpropagation: they are sensitive to vanishing and exploding gradients. This implies that the gradients can fade or explode as a consequence of the multiplication of many small or large derivatives during training. As a result, although their goal is to learn long-term dependencies, literature shows that RNNs cannot store information for very long. Long Short-Term Memory networks (LSTMs), on the other hand, provide memory blocks in its recurrent connections, storing more information for a longer period of time and avoiding the problem presented in RNN networks [START_REF] Lecun | Deep learning[END_REF][START_REF] Bousefsaf | 3d convolutional neural networks for remote pulse rate measurement and mapping from facial video[END_REF]. This kind of networks have already been used for biomedical signal analysis. For example, Oh et al. To the best of our knowledge, no deep learning approach has been used to filter biomedical signals and especially to filter rPPG signals. In this article we propose an LSTMbased deep-filter to reduce noise in rPPG signals, to improve signal quality and in consequence, improve HR or HRV measurement. We validate our methodology using the public database Multimodal Spontaneous Emotion Corpus -Heart Rate database (MMSE-HR) [START_REF] Zhang | Multimodal spontaneous emotion corpus for human behavior analysis[END_REF]. This paper is organized as follows: In section 2 an overview of the methods used are presented, the Periodic Variance Maximization (PVM) used to acquire rPPG signals from video, and the Long Short-Term Deep-Filter (LSTM-DF). In section 3 the experiments implemented are explained, with the specification of the LSTM-DF architecture and its hyperparameters. In section 4 a comparison is made between band pass filtering, wavelet filtering and LSTM-DF filtering. Finally, conclusions and perspectives are presented in section 5.

Method

The proposed method aims to filter rPPG signals with the implemented LSTM-DF filter. The workflow of the procedure is presented in Figure 2. The first step consists of the rPPG signal estimation from a video sequence. As a second step, the rPPG signals are filtered with our LSTM-DF filter. The main components of the framework are explained in the following subsections.

rPPG signal estimation

The first step of the rPPG signal extraction procedure consists in the selection of the Region Of Interest (ROI). For each video frame, face is detected and tracked using the reduced ResNet-10 model and the MedianFlow Tracker of the OpenCV library [START_REF] Bradski | The OpenCV Library[END_REF]. Skin detection, as formulated by Conaire et al. [START_REF] Ciarán Ó Conaire | Detector adaptation by maximising agreement between independent data sources[END_REF], is then performed to select the skin pixels, which subsequently are spatially averaged to obtain a triplet of RGB values per frame and concatenated to obtain the RGB temporal trace. The RGB temporal traces are then preprocessed to remove the DC component of the signals dividing samples by their mean over a temporal interval. DC-normalized RGB signals are then band-pass filtered using Butterworth filter (with cut-off frequencies of 0.7 and 3.5Hz).

Then, we decided to use the recently proposed PVM algorithm to estimate the rPPG signals because of its efficiency and good performances. The PVM method has been extensively described and evaluated in [START_REF] Macwan | Periodic variance maximization using generalized eigenvalue decomposition applied to remote photoplethysmography estimation[END_REF]; we preset below the main concepts. The algorithm extracts the rPPG signal by estimating the time period τ * that maximizes the periodic information embedded in the temporal mixture x, of size 3 × M , obtained by spatial averaging of RGB video frames over a temporal window of M samples. τ * is in turn found by searching for the optimum 3 × 3 weighting matrix W that maximizes periodicity over the range of different lags τ i corresponding to the human heart rate in the range [. 7 3] Hz. The ideal weighting matrix W τi at each τ i is estimated by performing GEVD on the pair of the covariance and lagged covariance matrices C x = xx T and P x = xx T τ respectively. Here x τ is the signal mixture x lagged by τ samples, representing the original signal mixture x evolved over time. The 3×1 generalized eigenvector w corresponding to the strongest generalized eigenvalue is used to extract the temporal signal y(t), represented as the vector y of size 1 × M , using y = w T x.

This formulation is based on the fact that the similarity between the quasi-periodic y and its lagged version y τ is strongly related to its periodicity and can be expressed in terms of a periodicity metric given by

P(τ, w) . = yy T τ yy T = w T P x w w T C x w . (1)

Long Short-Term Deep-Filter (LSTM-DF)

In order to train the LSTM-DF network that will filter rPPG signals, we have created a data matrix Y composed of a set of rPPG signals with a fixed length. The same process is applied on the corresponding PPG signals to create the ground truth data matrix Ŷ. y i is the rPPG signal of the video i with length T i , where i ∈ [1, N ] and N is the number of videos in the database, and therefore the number of rPPG signals. Y and Ŷ are created with a simple sliding window procedure; the algorithm is presented in Algorithm 1, with L being the length of the sliding window and s being the step size.

Algorithm 1: RPPG training dataset building count ← 1; N ← 79; L ← 5; s ← 0.04; for i ← 1 : N do for t ← 1 : s : T i -L do Y(count) ← y i,train (t : t + L); Ŷ(count) ← ŷi,train (t : t + L); count = count + 1; end end
A filtered rPPG signal ỹ is predicted by the LSTM-DF from an rPPG signal y, this is carried out as a regression problem. An LSTM network is composed of an input, a memory block, and an output layer. The input is the rPPG signal y. The memory block consists of: one input gate that learns which information should be stored in the memory block, a forget gate that learns how much information must be forgotten or withheld from the memory block, and an output gate that takes care of learning when the collected information can be used. The architecture of the LSTM-DF is shown in Figure 3 where b is the batch size, L is the length of y, and Return Sequence (RS) is an argument that decides whether a layer outputs each time step or its final time step. The size of the input signal is defined to be b × L×1. The output size of the first two layers remains b×L× 125 due to their number of units and their return sequence argument. The third and fourth layers decrease the output to b×L×1 representing, as a result, the predicted filtered rPPG signal ỹ. A dropout step is done on the first three layers to avoid the overfitting problem. The mean squared error (MSE) metric is chosen as the loss function for measuring the difference between ỹ and ŷ. The batch size b and the length of the signals L are discussed in the experimental section.

Figure 2. System framework: rPPG signal estimation is done in a sequence of videos using face and skin detection followed by the PVM algorithm [START_REF] Macwan | Periodic variance maximization using generalized eigenvalue decomposition applied to remote photoplethysmography estimation[END_REF]. Then an rPPG data set for training LSTM-DF network is built, concatenating fixed-length rPPG signal windows. Finally, LSTM-DF is compared with bandpass and wavelet fiters in rPPG filtering application. A cross-validation procedure is used to assess the performance of the LSTM-DF.

Experiments

Database

The LSTM-DF was trained and tested using the database Multimodal Spontaneous Emotion Corpus -Heart Rate database, which was made with emotion recognition in mind, so the videos include a large number of facial movements and expressions.The length of each video is between 30 to 60 seconds with a resolution of 1040 × 1392 pixels and a frame rate of 25 fps. The ground truth signal is the PPG signal with 1 kHz of sampling rate, obtained by the BIOPAC 150 data acquisition system [START_REF] Zhang | Multimodal spontaneous emotion corpus for human behavior analysis[END_REF]. N = 79 videos were used in our experiments with a 5-fold cross validation procedure.

Metrics

We compare our proposed filtering method with two classical approaches, namely a bandpass (BP) filtering with cutoff frequencies in 0.7 and 3.5 Hz, and a wavelet (WV) filtering technique using the same parameters as in [START_REF] Macwan | Periodic variance maximization using generalized eigenvalue decomposition applied to remote photoplethysmography estimation[END_REF]. For all the experiments, we evaluate the performance of our algorithm in terms the quality of the heart rate estimation and also in terms of the quality of the estimated rPPG signal. Therefore, we use Mean Absolute Error (MAE) in beats per minute (bpm) and Pearson's correlation coefficient (r) to evaluate the heart rate estimation performance and Signal to Noise Ratio (SNR) and Template Matching Correlation (TMC) [START_REF] Orphanidou | Signal-quality indices for the electrocardiogram and photoplethysmogram: Derivation and applications to wireless monitoring[END_REF] to evaluate the quality of the estimated signal. Metrics are calculated using heart rates estimated from the PPG signal and the filtered rPPG signals by the three filtering methods. 15-second window length was used to compute HR.

LSTM-DF setup

The proposed method was implemented on a Windows PC equipped with Intel Xeon 2.4 GHz CPU, with a RAM of 16GB, and with a NVIDIA GeForce RTX 2070 GPU, using the Keras (v.2.2.4) library with the Tensorflow (v.2.0) backend. LSTM layers were set with default hyperbolic tangent activation function, and the dense layer with a linear activation function. The first two LSTM layers had a dropout equal to 0.2, the loss function was mean squared error, and the Adam optimizer was set with learning rate of 0.002. The random weights initialization were made with the Glorot uniform initializer, also called Xavier uniform initializer with seed=1.

First, we conducted an experiment to determine the best hyperparameters of our network varying the batch sizes and the number of epochs. Table 1 presents the results of this experiment using the architecture proposed in Figure 3 with a 5-fold cross validation procedure. Results given in the table are the average of the 5 folds. For these experiments, it is possible to observe that the obtained results are quite stable. Therefore, in all the following experiments, we decided to use a batch size of 32 and a number of epochs of 100.

Batch size

LSTM-DF filtering

A re-sampling procedure on ŷ from 1 KHz to 25 Hz was required in order to match with y, albeit without any interpolation or perturbation in the signal shape, as 25 Hz is a multiple of 1 KHz. Each signal was also normalized between -1 and 1. Then, Algorithm 1 is applied with y train,i and ŷtrain,i signals to create the matrices Y and Ŷ, with a L = 5 seconds and s = 0.04 seconds (c.f. the rPPG training dataset building section in Figure 2) . For each of the 5 folds a Y and Ŷ was used to train the LSTM-DF network. Then, filtered rPPG signals ỹtest,i are obtained by filtering y test,i with the deep filter. The number of rPPG signals used for training and testing in fold number 1 was 64 and 15 respectively, while for folds from 2 through 5, it was 63 and 16 respectively.

Results and discussion

We compare in Table 2 our filtering method with a bandpass filter (BP) and a wavelet filter (WV). The results are presented as the average of each measure for the 5 folds, the last row of the table is the average of all values in the 5 folds. The best result for each metric and each fold are depicted in bold.

As mentioned previously, we use M AE to evaluate the heart rate estimation performance along with SN R and Template Matching Correlation (T M C) to evaluate the signal quality. First, it is possible to observe that the wavelet and bandpass filtering methods show very similar resluts in terms of M AE and that these results are already quite good, with M AE below 2bpm in average. Interestingly, the proposed method presents the lowest M AE values for all the folds as well as in average. This observation confirms that our LSTM-DF improves the accuracy of the heart rate measurements. This is probably due to the removal of false peaks in the rPPG signals (as seen in Figure 5), allowing the FFT method to find the dominant frequency more precisely.

Figure 4 shows the correlation plots of the HR estimations for the three filtering methods. Pearson correlation metric r is also presented in the figure. Similar conclusions can be drawn with these correlation plots. Even if the distribution of the HR estimations of BP and WV are already quite good, the results of LSTM-DF is even better with a fitting line closer to the 45 • line as compared to BP and WV. The Pearson correlation coefficient is also improved with LSTM-DF from 0.88 and 0.89 for BP and WV to 0.96.

With respect to the evaluation of the signal quality, Table 2 shows the SN R and the T M C metrics. Regarding to SN R, the wavelet filter shows a slight signal quality improvement over the bandpass filter. The LSTM-DF filter, on the other hand, presents a considerable improvement in the SN R metric with respect to the other two methods for all the folds as well as their average. The same behavior is present in the results of the T M C metric, where the proposed method surpasses the other filtering methodologies in all the folds, and therefore in their average.

With the proposed method of filtering, rPPG signals are smoothed in a different way than with wavelet or bandpass filers. To be more specific, in Figure 5, three filtered rPPG signals are depicted, the first one was filtered by BP, the second one by WV, and the last one by our LSTM-DF. WV and BP filtering methods, give a filtered rPPG signal with a good approximation of the number of peaks present in the PPG signal. However, it can be seen that these methods also remove the characteristic shape of the PPG signal, specifically, its dichrotic notch [START_REF] Sahni | Noninvasive monitoring by photoplethysmography[END_REF]. Consequently, rPPG signals filtered by BP and WV filters appear to be sinusoidal in shape. On the other hand, LSTM-DF allows not only to preserve the frequency and the number of peaks of the PPG signal, but also preserves the characteristic shape of the ground truth PPG signal, which is also quantified by the T M C metric in Table 2. In Figure 5 the the LSTM-DF filtering (c.f. the blue circles).

Conclusions and perspectives

The Long Short-Term Memory Deep-Filer (LSTM-DF) was presented in this paper to filter rPPG signals as an alternative to conventional signal processing techniques that cannot encapsulate the characteristic shape of the PPG signal neither its temporal structure. We have demonstrated, on a public database, that the LSTM-DF increases the accuracy of heart rate measurements as well as the quality of the estimated signal. Interestingly, the LSTM-DF filter preserves the characteristic shape of a PPG signal and paves the road to an advanced analysis of the temporal signal. It is important to note that the same methodology can be applied to the filtering of other temporal signals with characteristic shapes (e.g. ECG signals).

However, there is still a lot of experimentation to be car-ried out to validate our methodology further. For example, it would be interesting to validate our work on other databases and to evaluate the robustness during a cross-database validation. Finally, even if we postulate that our filtering better preserves the shape of the PPG signal and thus allows to estimate more easily advanced temporal characteristics, it now seems necessary to validate this assertion experimentally.

Figure 1 .

 1 Figure 1. Example of noise in two remote photoplethysmographic signals after bandpass and wavelet filtering. The PPG ground truth signal is presented in red while the filtered rPPG signal is presented in black.

  [START_REF] Oh | Automated diagnosis of arrhythmia using combination of cnn and lstm techniques with variable length heart beats[END_REF] proposed a CNN-LSTM model to detect five different classes of arrhythmias. Yildirim[START_REF] Yildirim | A novel wavelet sequence based on deep bidirectional lstm network model for ecg signal classification[END_REF] a bidirectional LSTM to classify ECG signals. Tan et al. [25] used convolutional neuronal networks (CNNs) and LSTMs to classify normal versus coronary artery disease ECG signals, the CNN is applied to reduce the number of data points in 5-second ECG signals, and the LSTM is applied to extract the temporal features of the signals. Thus, RNNs are indeed capable of handling of heartbeat-related signals.

Figure 3 .

 3 Figure 3. LSTM-DF architecture. Where numbers in red represent the size of each data. RS=Return sequences, T=True, F=False.

Figure 4 .

 4 Figure 4. Correlation plots of the HR estimation obtained with (from left to right) bandpass filtering, wavelet filtering and the proposed LSTMD-DF filtering.

Table 1 .

 1 Results obtained varying the batch size and number of epochs of the LSTM network.

		No. epochs HR-MAE SNR TMC
	8	30	1.50	9.69	0.71
	32	30	1.51	9.59	0.70
	32	100	1.32	9.44	0.72
	64	100	1.32	9.50	0.70

Table 2 .

 2 , it is possible to see that the characteristic dichrotic notch is preserved with 5-fold cross validation comparison in HR-measurement and signal quality improvement in rPPG signals filtered by: Bandpass filter (BP), Wavelet filter (WV), and the method proposed (LSTMD-F).

		HR measurement			Signal Quality
	Fold		HR-MAE [bpm]		SNR [dB]	TMC [%]
		BP	WV LSTM-DF BP	WV LSTM-DF BP	WV LSTM-DF
	1	1.53 1.52	0.53	7.41 7.44	10.47	0.49 0.50	0.82
	2	2.82 2.61	2.24	5.54 5.63	8.42	0.49 0.50	0.64
	3	1.80 1.79	1.43	6.52 6.61	9.95	0.32 0.33	0.71
	4	0.99 1.00	0.84	7.04 7.20	9.59	0.64 0.65	0.75
	5	2.31 2.40	1.38	4.88 4.92	9.00	0.43 0.44	0.65
	Average 1.88 1.85	1.31	6.27 6.36	9.44	0.48 0.49	0.71