Emma Carrigan

Ludovic Hoyet

Rachel Mcdonnell

Quentin Avril

A Preliminary Investigation into the Impact of Training for Example-Based Facial Blendshape Creation

Keywords: Descriptors (according to ACM CCS): I, 3, 3 [Computer Graphics]: Picture/Image Generation-Line and curve generation CCS Concepts, Computing methodologies → Mesh models; Animation;

des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Introduction

The process of creating blendshapes is still very reliant on artists. Although there is a lot of research done on scanning and rigging faces, any facial blendshapes or rigs that are created using these methods require extensive editing for use in games and movies. Additionally, some methods require a large amount of facial scans or motion capture, which can be costly depending on the technology required or the amount of time for which an actor must be hired. Currently, these are necessary costs for any AAA game or blockbuster.

One method which is used in production is Example-Based Facial Rigging [LWP10], which is an extension of Deformation Transfer [SP04]. This method uses a generic blendshape rig template, i.e. a neutral face with a number of blendshape target faces, as well as the neutral face of the character which you want to create the rig for, and a number of facial poses of this character. The algorithm recreates each of the blendshapes of the generic rig for the desired target face, while also incorporating facial details which it learns from the supplied facial poses.

While this method is used in professional pipelines, companies still need to hire an actor to create numerous poses and 3D modelling artists to clean the final blendshapes. However, we believe that we can improve the blendshape creation process to cut down the dependency on actors by finding the optimal types of poses to supply to the system. Our contribution is a preliminary perceptual study into the effect of a set of input scans on the EBFR system. From this, we can see what areas of the face were most improved by the algorithm and which areas were least improved. This gives us an idea of the impact of our supplied training poses and is a basis for further research into reducing the number of scans needed to attain suitable blendshape rigs using EBFR.

Related Work

Sumner and Popović define a method of deformation transfer for triangle meshes. This method deforms two meshes similarly given a source mesh, a deformation of the source mesh, a target mesh, and correspondence between the two [SP04]. The transfer is achieved through solving a constrained optimisation for the target mesh topology that matches the source deformations as closely as possible, while maintaining consistency constraints. Expanding on this, Ben-Chen et al. describe a spatial deformation transfer technique that allows deformation transfer to be applied to more than just single component manifold triangle meshes [START_REF] Ben- | Spatial deformation transfer[END_REF].

Li et al. propose a method for generating facial blendshapes given a generic facial rig and a neutral pose for the target mesh [LWP10]. This method improves upon Sumner and Popović's Deformation Transfer technique by supplying example poses of the target mesh to train the blendshape generation. This method uses a generic template model with all of the blendshapes that are to be created for the target model.

Xu et al. propose a method for facial animation transfer that transfers detailed animations and allows for quick user-editing of the spatial-temporal domain [XCLT14]. This approach splits the high-fidelity facial performance into high-level facial feature lines, large-scale facial deformation, and fine-scale motion details. It then transfers and reconstructs them to create the retargeted animation.

Ribero et al. propose a method of facial retargeting that takes into account the range of motion of the source and target characters in order to allow retargeting between characters of significantly different styles and proportions [RZL * 17].

Method

Our idea was to use all the common training poses available across a number of actors, which would theoretically create the best possible trained rigs for our data using EBFR. We then ran an experiment comparing these trained rigs, as well as untrained rigs created using the Deformation Transfer method, to a ground truth facial scan. The participants chose which of DT or EBFR faces best resembled the ground truth, then described how close their chosen pose was to the ground truth on a 5-point Likert scale. This showed us which parts of the face were most improved or disimproved by EBFR. An example of the stimuli can be seen in Figure 1.

Stimuli

We used the Bosphorus Database to get data for our experiment [SAD * 08]. The data is provided as point clouds with textures. We meshed, cleaned, normalized and registered this data to create meshes with consistent topology for ease of use for our experiment. After this preprocessing step, we had a large number of facial expressions from different actors which we could use both as ground truth and as input to EBFR. An example of the cleaned data we used can be seen in Figure 2. The Bosphorus database is a database of facial scans of over , however due to the difficulty of activating certain facial muscles in isolation, most scans are actually a combination of AUs. Fortunately, each scan in the database has been annotated by a FACS expert. Using this information, we can attempt to recreate the scans using our facial rigs created using DT and EBFR, as the blendshapes of the rigs we used were based on FACS.

We selected 4 female and 4 male actors from this database, and created trained and untrained rigs for each selected actor. The untrained rig was created using Deformation Transfer using a generic facial animation rig whose blendshapes were based on FACS AUs. The trained rig was created using Example-Based Facial Rigging, using the same generic model and neutral pose as the untrained rig, but including 19 additional facial scans of the actor with different expressions as training poses.

The expressions in the scans used as training were the same across all actors. These expressions consisted of the 19 Action Units as detailed in Table 1. These expressions were chosen because they were the most commonly represented expressions in the database and we required a common set of expressions across all actors. They were also chosen to convey information from all the different areas of the face, e.g., mouth, nose, eyes.

Participants and Procedure

Participants were presented with 152 trials: 2 Actor Sex (Female, Male) × 4 Actors × 19 Expressions. In each trial, participants were presented with the ground truth face (scan) in the middle of the screen, and both the trained and untrained faces randomly presented on the left or right side of the screen (Figure 1). Participants could rotate the faces simultaneously using the arrow keys on the keyboard, to a maximum of 30 degrees in each direction. For each stimulus they were asked "Which face resembles the middle face most?", and answered using the S and F keyboard keys. They saw the faces for a maximum of 10s, after which they were forced to provide an answer. Then they were asked to rate how close the face they selected was to the middle face on a scale from 1 (Not at all) to 5 (Identical) using the keyboard. The trials in the experiment were presented in blocks: each actor of one gender was presented in a random order, then the actors of the other gender. The genders were presented in a randomized order. All the expressions for one actor were presented in a random order before moving to the next actor.

We included training stimuli at the beginning of the experiment, identical across participants and using an actor who did not appear in the experiment. The participants used these stimuli to become familiar with the experiment and the buttons needed to answer our questions. Responses for these stimuli were not recorded. A screen was shown between the training and real experiment to warn the participants that their responses would begin to be recorded.

Twenty-three participants took part in our experiment (5 female, 17 male and 1 other, aged 23-61 years). They viewed the experiment on a 24" display of resolution 1920x1200. Each participant was given an information sheet and consent form to sign. The information was repeated on the screen at the beginning of the experiment. The participant was then asked to input some demographics information before they began the experiment.

Results

To assess whether trained (EBFR) faces were preferred to untrained (DT) faces, as well as whether differences appear for different parts of the faces, we performed a one-way repeated measures Analysis of Variance (ANOVA) with within-subject factors Expression on the percentage of times EBFR was preferred over DT. To analyse these results, each participant's results were averaged across all the actors for each condition. All effects are reported at p < 0.05. When we found main or interaction effects, we further explored the cause of these effects using Newman-Keuls (p < 0.05) post-hoc tests for pairwise comparisons.

First, we found a main effect of Expression (F 18,396 = 41.65, p ≈ 0), where post-hoc analysis showed that EBFR was clearly preferred for some expressions, and less for others (Figure 3). To further explore these effects, we conducted single t-tests against 50% to evaluate if preference was above chance level (p < 0.05). Results showed 3 categories of expression, which are listed below:

Improved by EBFR: 0, 1, 2, 3, 5, 6, 7, 8, 9, 14 and 15

No preference between EBFR and DT: 4, 10, 12, 17 and 18 EBFR worsened the results: 11, 13, 16

Excluded Results

We found that, for some of the expressions, participants preferred the untrained faces across all actors, which was unusual as we expected the trained faces to be equal or better in every case. In order to understand why, we manually examined the stimuli and found some artifacts across almost all actors for certain expressions.

There were texture artifacts for expressions 16 and 18, with FACS names Outer Brow Raiser and Eyes Closed, as can be seen on the left in Figure 4a. Although our interest was purely morphological and we asked participants to ignore texture artifacts to the best of their ability, we found these artifacts to be too noticeable to ignore. For this reason, we chose to exclude expressions 16 and 18 from our analysis.

While we could have avoided these issues by removing the textures on every stimulus, we found that the meshes with no texture were unnatural and might have affected the perception of participants, as they were too unlike real faces. As we are interested in human facial perception, we decided to include the textures to ensure the faces looked as human as possible. We also found that the trained stimulus for expression 13 (Mouth Stretch) was often unnatural looking, which we found to be caused by an error in scaling the scan from the database. In our data cleaning process, we scaled the faces to be of unit length. This had a strong negative effect on expression 13, as the actor opens their mouth as wide as possible, which causes the face to be a lot longer than when at rest. In the training process, we were essentially telling our algorithm to make the neutral actor scan (Figure 4b Left) shrink to match the scanned expression 13 (Figure 4b Centre). This resulted in an unnatural face (Figure 4b Right). For this reason, we excluded expression 13 from our results.

Analysis

After removing the results that were caused by artifacts, we can separate the results into groups as shown in We excluded the Mouth Stretch expression, as our data cleaning algorithm scaled the faces so the meshes would be unit-length from top to bottom. This made Mouth Stretch smaller than it should have been. However, we did not exclude Jaw Drop as there were no obvious artifacts, although it appears a similar issue may have happened. Jaw Drop is a slightly longer than normal face, so the scaling should have affected this expression unfavourably as well.

Brow Lowerer was the only upper face expression that remained after we excluded results. We had a noticeable lack of upper face expressions to choose from, and two of the three expressions we had were excluded due to artifacts. Brow Lowerer's neutral result may be caused by not having enough upper face training poses.

Interestingly, for Lip Corner Depressor and Lip Pressor, expressions which cause the lips to be pushed together and stretched, it seems that the algorithm simply has a hard time recreating these. For Lip Pressor, the lips become quite thin, which confused our training algorithm and seemed to accentuate some sharp edges around the lip contour, and sometimes caused overlapping faces. For Lip Corner Depressor, the downward movement seemed to make the mouth open slightly in some cases, and stretch the bottom lip to make it look slightly larger in other cases. Lips Part had a similar issue in that it often made the contour of the lips slightly sharper. These small errors seem to be enough to affect the perception of these expressions.

Discussion

We created a number of facial rigs using EBFR and showed that EBFR produces perceptually better facial rigs than Deformation Transfer. We found that artifacts caused by the algorithm that af-fected the contour of the lips were more noticeable than artifacts that affected the other areas of the face. Our results indicate that the lip area is important when creating facial rigs. However, it is possible this was caused by the lack of an internal mouth structure. This caused any opening of the mouth to be very apparent. Future work will investigate the importance of the internal structure.

More interesting is the fact that the Lips Part expression was noticeably affected. This expression had the mouth slightly open, so we see that it is not the difference between an open and closed mouth that is noticeable, but the actual shape of the lips, specifically the edge between the lip and the inside of the mouth.

Our main limitation came from the database of facial scans we used. We chose it because we already had a pipeline for processing meshes from this database, however we found it difficult to get a wide sample of facial expressions across many actors. This was the reason we were lacking in upper-face expressions in our study, we simply did not find a subset of actors in the database that had scans of the same upper-face expressions.

Our initial goal for this project was to identify what facial expressions are important to use as training when using Example-Based Facial Rigging to create facial rigs. While our work here has indicated certain parts of the face that might require more attention when automatically creating blendshapes, there is room for more investigation into this topic. We would like to be able to specify a subset of facial expressions that would be considered the "ideal" subset to use for training the EBFR algorithm. To do this, we would need to create multiple rigs with separate subsets and compare them. This fell out of the scope of our study, but we feel it is an important next step.

Figure 1 :

 1 Figure 1: An example of stimuli shown to participants comparing the facial rigs created without training, and those with training.

Figure 2 :

 2 Figure 2: Examples of scanned poses from the Bosphorus database.

Figure 3 :

 3 Figure 3: Main effect of Expression on preference of EBFR over DT.

( a )

 a Figure 4: (a) The texture artifact which affected expressions 16 and 18. (b) The artifact which affected expression 13.

Table 1 :

 1 The

	AU No.	FACS Name	Expression No.
	9	Nose Wrinkler	0
	10	Upper Lip Raiser	1
	12	Lip Corner Puller	2
	14	Dimpler	3
	15	Lip Corner Depressor	4
	16	Lower Lip Depressor	5
	17	Chin Raiser	6
	18	Lip Pucker	7
	22	Lip Funneler	8
	23	Lip Tightener	9
	24	Lip Pressor	10
	25	Lips Part	11
	26	Jaw Drop	12
	27	Mouth Stretch	13
	28	Lip Suck	14
	34	Cheek Puff	15
	2	Outer Brow Raiser	16
	4	Brow Lowerer	17
	43	Eyes Closed	18

Action Units we used in our experiment with their FACS names. The third column shows the numbers we used for them in our experiment. Expressions 0-15 are lower face expressions, 16-18 are upper face. a hundred actors attempting to recreate the Action Units (AUs) as described in Ekman et al.'s Facial Action Coding System (FACS) [EF78]

Table 2 .

 2 The expressions where DT was significantly preferred.

	AU		FACS Name	Exp.
	9		Nose Wrinkler	0
	10		Upper Lip Raiser	1
	12		Lip Corner Puller	2
	14		Dimpler	3
	16	Lower Lip Depressor	5
	17		Chin Raiser	6
	18		Lip Pucker	7
	22		Lip Funneler	8
	23		Lip Tightener	9
	28		Lip Suck	14
	34		Cheek Puff	15
	(a) The expressions where EBFR was significantly preferred.
	AU		FACS Name	Exp.
	15	Lip Corner Depressor	4
	24		Lip Pressor	10
	26		Jaw Drop	12
	4		Brow Lowerer	17
	(b) The expressions where there was no significant difference be-
	tween EBFR and DT.
		AU FACS Name Exp.
		25	Lips Part	11
	(c)

Table 2 :

 2 Results grouped by ratio of trained to untrained responses.

c 2018 The Author(s) Eurographics Proceedings c 2018 The Eurographics Association.

(2014), 42. 2 c 2018 The Author(s) Eurographics Proceedings c 2018 The Eurographics Association.

Acknowledgements

This research was partially funded by Science Foundation Ireland as part of the Game Face (13/CDA/2135) project.


Pierre Aumond

Léo Jacquesson

Arnaud Can

Probabilistic modeling framework for multisource sound mapping

Keywords: Soundscape, Multisource sound mapping, Stochastic sound modeling

HAL is

Introduction

The representation of urban sound environments through noise maps became standard since the enactment of the European Noise Directive 2002/49/EC [START_REF] Ec | Directive 2002/49/EC of the European parliament and the Council of 25 June 2002 relating to the assessment and management of environmental noise[END_REF]. The modeling framework to generate noise maps is now well established: long-term aggregated indicators are calculated based on a collection or estimation of road, air and rail traffic variables, followed by sound emission and propagation calculations [START_REF] Kephalopoulos | Advances in the development of common noise assessment methods in Europe: The CNOSSOS-EU framework for strategic environmental noise mapping[END_REF]. This framework however disregards the sound levels variations, and hence prevents the production of statistical or emergence indicators that improve the characterization of urban sound environments [START_REF] Liu | Spatiotemporal variability of soundscapes in a multiple functional urban area[END_REF][START_REF] Can | Accounting for traffic dynamics improves noise assessment: Experimental evidence[END_REF][START_REF] Brocolini | Measurements of acoustic environments for urban soundscapes: choice of homogeneous periods, optimization of durations, and selection of indicators[END_REF].

In addition, limiting sound maps to traffic sources truncates the reality. A wider range of sound sources, including natural sounds, intervenes when people are asked to describe urban sound environments and evaluate their quality [START_REF] Nilsson | Acoustic Indicators of Soundscape Quality and Noise Annoyance in Outdoor Urban Areas[END_REF][START_REF] Ricciardi | Sound quality indicators for urban places in Paris cross-validated by Milan data[END_REF][START_REF] Brown | Towards standardization in soundscape preference assessment[END_REF]. Some models link perceptual variables, especially soundscape pleasantness, to the time of presence of typical urban sound sources such as birds or voices and experience shows that natural sounds are often perceived as impacting positively the soundscape pleasantness [START_REF] Lavandier | The contribution of sound source characteristics in the assessment of urban soundscapes[END_REF][START_REF] Aumond | Modeling Soundscape Pleasantness Using perceptual Assessments and Acoustic Measurements Along Paths in Urban Context[END_REF][START_REF] Guastavino | The Ideal Urban Soundscape: Investigating the Sound Quality of French Cities[END_REF].

The relative impact of sound sources is however more complex: the positive effect of voices can be for instance annihilated when voices are too numerous or misfit the context [START_REF] Pérez-Martínez | Soundscape assessment of a monumental place: A methodology based on the perception of dominant sounds[END_REF]. Interactions between sources are also of importance, as shown in [START_REF] Morel | Noise annoyance assessment of various urban road vehicle pass-by noises in isolation and combined with industrial noise: A laboratory study[END_REF] between road traffic and industrial noise, or between natural sound sources and road traffic noise [START_REF] Coensel | Effects of natural sounds on the perception of road traffic noise[END_REF][START_REF] Hao | Assessment of the masking effects of birdsong on the road traffic noise environment[END_REF][START_REF] You | Evaluating water sounds to improve the soundscape of urban areas affected by traffic noise[END_REF][START_REF] Jeon | Perceptual assessment of quality of urban soundscapes with combined noise sources and water sounds[END_REF].

In consequence, sound mapping recently moved towards multi-sources approaches [START_REF] Licitra | Noise mapping in the EU: models and procedures[END_REF][START_REF] Kang | Ten questions on the soundscapes of the built environment[END_REF]. In [START_REF] Hong | Exploring spatial relationships among soundscape variables in urban areas: A spatial statistical modelling approach[END_REF], the presence of sources, categorized into traffic, water, human and bird sounds, is evaluated perceptually at sampled locations and interpolated to create sources-oriented noise maps. A modeling approach is followed in [START_REF] Aletta | Soundscape approach integrating noise mapping techniques: a case study in Brighton, UK[END_REF]: specific noise maps are built for road traffic, fountains, and birds, which are placed by default at trees locations. In recent years, acoustic emission and propagation models have been integrated directly into Geographic Information Systems (GIS) [START_REF] Fortin | An opensource tool to build urban noise maps in a GIS[END_REF][START_REF] De Kluijver | Noise mapping and GIS: optimising quality and efficiency of noise effect studies[END_REF] and sources-oriented noise maps can benefit from this environment [START_REF] Hong | Soundscape mapping in urban contexts using GIS techniques[END_REF]. Lavandier et al. related the perceived loudness of the global sound environment and perceived presence time ratio of some sound sources of interest, namely traffic, voices and birds, to geographical indicators [START_REF] Lavandier | Urban soundscape maps modelled with georeferenced data[END_REF]. If these first researches prove the feasibility and interest of producing source-oriented noise maps, they need further investigations to better consider the sound sources localization and temporal activities, as well as their specific levels and spectral contents. For supporting such researches, it will be helpful that all the source-oriented noise maps are produced using independent emission models for each type of sound source and sharing an identic acoustic propagation model. So that, the new advances would not affect other sound sources emission model and the propagation modeling framework. In addition, the described mapping experiences were based on static approaches that hinder the masking that occurs between sound sources. Indeed, the respective temporal variations of sources are known to strongly impact the fact of hearing or not a sound source within an urban sound mixture, through for instance auditory attention processes [START_REF] Oldoni | A computational model of auditory attention for use in soundscape research[END_REF].

We propose a probabilistic modeling framework for producing source-oriented sound maps within an open-source GIS, which shares the same structure, described in section 2, whatever the sound source is.

The model is illustrated for the production of traffic, fountains, voices and birds sound maps in section 3.

As shown in Section 4, the original probabilistic approach makes it possible to investigate the competition between sound sources, that is to say the probability that a sound source, or a group of sound sources, will be louder than other ones.

Method

Modeling framework 2.1.1. General presentation

The modeling consists of producing source-oriented sound maps. Our approach is stochastics: a set of n x k sound maps is created, corresponding to n representations of the possible instantaneous sound environment for each of the k considered sound source. Each sound map i k can be seen as the photography at one instant of the possible encountered equivalent 1-second sound environment for the contribution of a given sound source. Statistics are done on a sufficient representative number of maps to characterize the sound environment where the model input parameters are stable (e.g. constant bird's density during the study period and area). The objective is thus to account for the 1-s time variability of the sound environment to compute original sound environment indicators. The time evolution of sound environments, that is the coherence between two consecutive iterations i, is not a target output of the modeling. We follow the same modeling framework in four steps whatever the sound source is.

Although the modeling framework contains the same four steps whatever the sound source is, the content of each step differs from one sound source to the other, based on knowledge on its characteristics.

In terms of modeling, some sound sources are very well documented in terms of both sound emission and spatial repartition, while other ones require specific modeling. The interest of the approach lies here in the fact that: (i) new sound sources can easily be implemented following the same modeling approach, (ii) each step can be improved through further research without compromising the rest of the modeling chain.

Step 1

The spatial repartition of the potential sound sources (humans, birds, cars, etc.) is defined. To limit the calculation time, the spatial repartition correlates to the density of the sound sources within each zone for a given time period. For instance, in the example illustrated in Figure 1 birds are located within the park where they are more prone to be found.

Step 2

The matrix of attenuation per octave band (63-8000 Hz), between each couple receivers and potential sound source, is calculated (see Figure 2). We consider that for the stable considered period of time, the acoustic propagation remains unchanged. A set of n sound maps is created for each sound source k. Each sound map is calculated by activating randomly a proportion of the potential sound sources with respect to knowledge on the activity of the sound source objects. For instance if it is assumed that 30% of the present humans are speaking during the same second, only 30% of the located humans randomly selected on the map are activated for a sound map i.

A 1-second equivalent sound power level and spectrum is assigned to each activated sound source, based on knowledge on the sound source: 1-s variability in sound power levels, sound spectrum, prosody for the voices, etc. Each sound map i is calculated by summing the contribution, at each receiver, of each sound source activated at i. The individual contribution of each sound source is the sum between the sound power level and the sound propagation attenuation calculated at step 2.

Note again that the activated sound sources vary at each iteration, but that the attenuation matrix is only calculated once, hence strongly limiting the computation time. A proportion of the birds previously located on the map (step 1) are activated randomly.

The contributions of all the activated sound sources are summed to compute n maps as n possibilities of bird songs sound maps.

Step 4

Sound level indicators are calculated at each receiver based on the sound maps. The original sound indicators can be statistical levels for each modeled sound sources (L 10 , L 50 and L 90 , which stand as the sound level exceeded 10, 50 and 90% of the time, respectively). But also, it can be the proportion of the combinations between the n x k sound maps computed for each sound source where the sound level of a defined sound source is superior to the other ones. For instance, for x% of the n x k computed sound maps the birds sound level is superior to the sound level of the road traffic. A graphical summary is shown in Original sound indicators can be calculated from the sound maps generated for each sound source but also from the combination between all the generated sound maps.

Software environment and sound propagation modeling

All the modeling steps are implemented within the open-source GIS software OrbisGis1 . OrbisGIS allows researchers to share their results and build a common platform to analyze sustainable urban development. OrbisGis is compatible with the use of Open Street Map2 (OSM) databases (i.e., buildings, roads, etc.) and facilitates the representation of the produced sound maps. In this study, all the geographic information has been imported from the Geofabrik3 portal except the traffic data which own to the Nantes city council.

The sound propagation done during the step 2 presented above utilizes the free and open-source Noisemodelling plugin4 , which has already been developed as an OrbisGis plugin to produce static road traffic noise maps [START_REF] Fortin | Urban noise maps in a GIS[END_REF]. Noisemodelling is designed to produce environmental noise maps on very large urban areas, with few computational resources. The method developed in Noisemodelling is very close to the European CNOSSOS Method for the sound attenuation calculations, see details in [START_REF] Fortin | Urban noise maps in a GIS[END_REF]. A resume of the user-defined parameters that have been used for this study is given in Table 1.

Ground absorption G=0

Wall absorption G=0.23

Study area and period

The site is located in Nantes, France. It was chosen because it presents a wide variety of sound

Specifications for each sound source and for our study case

The probabilistic modeling framework is illustrated in this paper for road traffic, fountains, voices and bird songs, which are four sound sources that are known to strongly affect the quality of urban sound environments, and that have been the matter of numerous studies. Details of the modeling are presented for each of the four sound sources in this section. For each sound source, we detail the modeling for the steps one and three described previously. The sound propagation (step 2) is not detailed as it does not differ from one sound source to the other. The indicators calculation (step 4) is detailed in Section 4, which presents the final sound maps.

Road traffic

The negative impact of road traffic on urban sound environments is perceptually clearly established [START_REF] Aumond | Modeling Soundscape Pleasantness Using perceptual Assessments and Acoustic Measurements Along Paths in Urban Context[END_REF][START_REF] Nilsson | Soundscape quality in suburban green areas and city parks[END_REF]. Note finally that road traffic noise has a secondary negative effect, not clearly quantified yet, which is its masking over the positively perceived sources such as bird songs (29).

Spatial repartition (Step 1)

The possible positions for road traffic correspond to the roads documented within Open Street Map in our case. Roads are discretized into point sources as recommended for road traffic noise mapping [START_REF] Kephalopoulos | Advances in the development of common noise assessment methods in Europe: The CNOSSOS-EU framework for strategic environmental noise mapping[END_REF]. We choose a 10m-space resolution, which corresponds to the maximum density encountered on the road network, so that vehicle noise emissions can be individualized. These point sources become potential sound sources for each map iteration, where combined with vehicle densities to activate road traffic sources on the network. The height of the point sources is set to 0.05m in accordance with the CNOSSOS standard. Traffic densities are derived from flow rates Q and speeds V, for different vehicle categories.

The key here stands in the access to the traffic data inputs, meaning flow rates, speeds per vehicle category and road segment, which are not necessarily freely available. In this work, data were made available by the city council of Nantes, France, as other cities do.

Sound emission (Step 3)

At each computation i, the probability to activate a point source is the ratio between the density of the possible sources on the road network (for instance K max = 0.2 veh/m if the sources are placed with a space granularity of 5 meters), and the calculated density. For instance with Q = 900 veh/h and V = 30 km/h, the deduced density is K = 0.03 veh/m, and the probability of activation is 15%. Thus, this stochastic approach permits to reproduce the intermittent sound levels encountered within streets where the traffic density is low.

Once activated, the sound power level and the sound spectrum of a sound source is computed based on the state-of-the-art NMPB noise emission model, in terms of the vehicle category and vehicle speed [START_REF] Dutilleux | NMPB-Routes-2008: The Revision of the French Method for Road Traffic Noise Prediction[END_REF].

The implementation results in a map of 1-s emissions, which is extended by means of the attenuation matrix into the noise map i for road traffic.

Fountains

Fountains have a different status from the other elements of street furniture. History has given them strong social, aesthetic and artistic dimensions. Fountains thus constitute key elements of visual amenity and contribute to the identity of gardens and cities. Perceptual studies reveal that fountain sounds are the most favorite ones within urban soundscapes [START_REF] Kang | On the diversity of urban waterscape[END_REF]. They are also studied as a potential mask for road traffic noise and thus a potential urban facility for improving the sound environments quality by reducing the negative effect of traffic noise (32).

Spatial repartition (Step 1)

The fountains are represented within Open Street Map and thus easy to position. In the case study only one fountain is present. In this study, the height of the point sources for fountain is set to 0.5 m.

Sound emission (Step 3)

The fountain is set to active during all the period of the sound environment we aim to model. The sound spectrum of fountains has been the matter of researches [START_REF] Galbrun | Acoustical and perceptual assessment of water sounds and their use over road traffic noise[END_REF][START_REF] Watts | Measurement and Subjective Assessment of Water Generated Sounds[END_REF], which highlighted its high frequency component, between 1 kHz and 4 kHz. More in details, the spectrum depends on factors such as the type of fountain, its shape, height of fall, or the material of impact of the water, which often stand as not available information. Under absence of detailed information, we propose by default an average spectrum, inspired from [START_REF] Galbrun | Acoustical and perceptual assessment of water sounds and their use over road traffic noise[END_REF][START_REF] Watts | Measurement and Subjective Assessment of Water Generated Sounds[END_REF] and normalized at 90 dB. This corresponds to the spectrum given in Table 2, which is used by default. The implementation results in a map of 1-s emissions, which is extended by means of the attenuation matrix into the sound map i for fountains. In this study case, we consider the fountain as constant, but without any supplementary effort, we could add a variation on the sound level spectra for example, or on the time variation (for instance if during 10 seconds each minute the fountain stops emitting sound).

Human voices

Sounds derived from human activities are often appreciated components of the urban sound environments, and constitute its identity. Perceptual tests show that amongst human activities, voices are the most quoted of human sounds to describe an ideal soundscape [START_REF] Guastavino | The Ideal Urban Soundscape: Investigating the Sound Quality of French Cities[END_REF]. Finally, voices are generally associated to mode lively and animated soundscapes [START_REF] Axelsson | A principal components model of soundscape perception[END_REF].

Spatial repartition (Step 1)

In an urban environment, most of pedestrians are located on sidewalks and pedestrian zones. Sidewalks are not directly available within Open Street Map database. In our case, they are obtained by subtracting to a large buffer around roads, a narrow buffer around roads and buildings areas. The resulting possible locations for pedestrians are depicted in orange in Figure 6.

The spatial distribution of humans in a city center highly depends on morphologic parameters, such as the functionality of the zone, the shopping areas or the transportation network and soft mode transportation facilities. Space syntax approaches, based on the streets connectivity are found in the literature as good predictors of the traffic volumes within the city [START_REF] Jiang | Ranking spaces for predicting human movement in an urban environment[END_REF][START_REF] Raford | Space Syntax: Innovative Pedestrian Volume Modeling Tool for Pedestrian Safety[END_REF]. However, this approach has detractors as it disregards precious metric information and is rather limiting [START_REF] Ratti | Space Syntax: Some Inconsistencies[END_REF]; it furthermore does not explicitly account for the functionality of the zones. A microscopic approach based on activity patterns and scheduling and utility maximization is proposed in [START_REF] Hoogendoorn | Pedestrian route-choice and activity scheduling theory and models[END_REF], which however requires specific modeling and higher resources.

As a first approximation, we propose a macroscopic estimation of pedestrian volumes based on

available open data furnished by Open street Map. The model parameters are calibrated using open traffic volumes data from the city of Montreal 5 (more than a hundred counting points). As proposed in [START_REF] Lavandier | Urban soundscape maps modelled with georeferenced data[END_REF], the model relies on the density of points of interest (which are shops, railway stations, etc.) within radius of different sizes. Densities are expressed through quadratic kernel functions. For a given item (for instance shops), the density is expressed as in Equation 1.

∑ [ 15 16 * 𝑅 * (1 - 𝐷 𝑗 2 𝑅 2 )] 𝑚 𝑗=1 Equation 1
with D j the distance of each item j to the point of interest, R the limit radius (the density being null at The model correlates with the pedestrian flowrates from the Montreal dataset (R² adj. = 56%, p<.05), so it will be considered as sufficient for a first approximation of pedestrian flow rates, in the absence of more refined modeling.

D j = R),
The pedestrian flow rates determined over the area with Equation 2are then converted into pedestrian densities. First, the area is partitioned into a grid of 30 m of side, with a surface S grid = 900 m². Then, the number of pedestrians N ped within each element of the grid is calculated, as

Nped = (Qped / Vped ) * (Sgrid /lpp) Equation 3
with Q ped the pedestrian flow rate measured within a band of l pp width set to 20 m (Q ped will increase as the l pp increases), and the walking speed V ped = 1.25 m.s -1 . The number of pedestrians placed in the walkable zones is defined based on the ratio within the walkable zones and the grid. These pedestrians are located within the walkable zones of each element of the grid. This ensures that the density increase due to streets tightening is well reproduced. The resulting pedestrian positioning is depicted in Figure 6. The height of the point sources for pedestrian is set to 1.5 m.

Sound emission (Step 3)

A relation between the pedestrian density and the number of talking pedestrians has been elaborated by Qi and Kang [START_REF] Meng | The influence of crowd density on the sound environment of commercial pedestrian streets[END_REF]. It suggests that 30% of pedestrians talking in a dense crowd is a good approximation.

As a first approximation, this number is retained in the modeling, and for each generated sound map i, 30% of the pedestrian point sources are randomly activated.

The sound spectrum for human voices, categorized into man, woman and child voices, can be found for instance in [START_REF]Speech Levels in Various Noise Environments[END_REF]. It shows a maximum of energy within the 250-500 Hz band frequencies. The sound power level of talking persons varies according to the background sound levels. According to Lazarus [START_REF] Lazarus | Prediction of Verbal Communication is Noise-A review: Part 1[END_REF], raised normal speaking conditions is 60 dB at 1 meter from the sound source. The Table 3 presents the chosen sound power emissions per octave band associated to voices. The implementation results in a map of 1-s emissions, which is extended by means of the attenuation matrix into the sound map i for pedestrian voices.

Bird songs

Bird songs often constitute one of the only contacts with sounds from the bio-phony that urban dwellers experience in their daily life. This explains why bird songs are known to improve the quality of urban sound environments [START_REF] Aumond | Modeling Soundscape Pleasantness Using perceptual Assessments and Acoustic Measurements Along Paths in Urban Context[END_REF], and potentially reduce stress (43).

Spatial repartition (Step 1)

Indicators of urban morphology, especially the building plan area fraction, the green area dispersion index, and the green area perimeters, are found to be correlated to the presence of birds and to the diversity of encountered species [START_REF] Hao | Integrated effects of urban morphology on birdsong loudness and visibility of green areas[END_REF][START_REF] Pellissier | Birds are also sensitive to landscape composition and configuration within the city centre[END_REF]. Thus, without additional knowledge about the bird's spatial repartition, they are placed by default within zones that cover trees located within streets, parks, and the center of urban blocks, which often correspond to private gardens. These zones shown in Figure 7 Birds areas (light green areas), final repartition of birds (green dots) and receivers area (dotted line).

Sound emission (Step 3)

The proportion of activated birds corresponds to the number of birds singing at the same time. Without additional knowledge, we consider that 10% of the potential sound sources are emitting sound.

Calder and Willam linked the acoustic power P (mW) of singing passerine birds to their weight m (g), through the formula P = 0.042.m 1.14 [START_REF] Calder | The Scaling of Sound Output and Territory Size: Are They Matched?[END_REF]. The singing center frequency f c (kHz) of passerine birds also depends on their weight, through the formula f c = 7.204.m -0.24 [START_REF] Wallschläger | Correlation of song frequency and body weight in passerine birds[END_REF]. For instance, for sparrows whose weight is approximately 35g, these formula correspond to a sound power level L w = 85.8 dB and a singing center frequency f c = 3.1 kHz. The Table 4 presents the sound power emission level per octave band used in this study to model bird songs. The implementation results in a map of 1-s emissions, which is extended by means of the attenuation matrix into the sound map i for birds.

Probabilistic modeling

1-s equivalent sound environment

The n instantaneous (1-s equivalent) sound maps are generated through the implementation of the presented modeling for fountains, traffic, birds and voices. An example of a predicted instantaneous sound environment is depicted in Figure 8. The stochastic process is visible for instance in Figure 8 (b), in which bird sounds are not homogeneously distributed within the park, because of the randomized activation process. This randomization effect is also visible for road traffic on Figure 8 (d), which shows sound peaks along the main roads, as well as sections with lower sound levels. The effect of the sound attenuation is also visible, for instance sound levels due to road traffic sound sources are very low within the inner courts of buildings.

Statistical maps

The compilation of n iterations (n=60) of possible instantaneous sound environments helps calculating statistical sound indicators as described in step 4, which are not accessible through the standard static noise mapping approaches. The number of 60 iterations was chosen because the maps produced were relatively stable after this number of iterations. An example of L 50 map is provided in Figure 9. Compared with the previous sound maps, and as expected the sound level repartition is much more homogeneous.

These kinds of maps can be useful to evaluate for example the sound levels range of variation in a street for a specific sound source. For instance for road traffic noise, the L 10 -L 90 indicator is a meaningful indicator which is usually small for large boulevard, but important in small streets with sporadic car passages. This indicator could also be calculated within the proposed framework.

Combined sound maps

In addition, the confrontation of the instantaneous sound maps makes it possible to estimate the probability that a sound source emerges from the global environment sound mixture. For instance, in Figure 8(b) one can expect that bird sounds, which are not very loud in the south of the park, will be masked by the fountain if we combine these two sound maps.

The criteria to define emergences are as following:

 Bird sounds emerge, for a given instantaneous sound map i and at a given receiver, if the sound pressure level from the birds sound map exceeds the energetic sum of the sound pressure level due to road traffic and fountains at 4 kHz and exceeds 40dB (limit set for background urban sound level);

 Voice sounds emerge, for a given instantaneous sound map i and at a given receiver, if the sound pressure level from the voices sound map exceeds the energetic sum of the sound pressure level due to road traffic and fountains at 250 or 500 Hz and exceeds 40dB (limit set for background urban sound level);

 Road traffic sounds emerge, for a given instantaneous sound map i and at a given receiver, if at least one octave band of the sound pressure level from the road traffic sound map exceeds the sound pressure level due to fountains and exceeds 40dB (limit set for background urban sound level). This third map aims to evaluate road traffic noise masking through fountains.

The computation of 360 combined sound maps gives first approximation of the percentage of the time when a sound source emerges from the global environment sound mixture. Results are depicted in In Figure 9 (b), bird sounds are quite homogeneously present over the area (even if louder in parks). In Figure 10 (b), the map proportion where bird sounds emerges is low near the major axis at the south and close to the fountain.

In Figure 10 (a), voices are very present in the south west of the study area which is a pedestrian zone.

In the other areas, the competition with the road traffic is very important.

In Figure 10 (c), road traffic noise emerges in nearly all the area. This informs that the road traffic sound levels emerge within the global environment sound mixture. In such situation, the presence of the source is not sufficient to describe the sound environment, but it is very complementary to the sound level indicators produced and illustrated in Figure 9 (d). Interestingly, it can be observed that the propagated sound from the road to the park, at the south-west, is covered by the fountain sound.

Discussion

Each block of the modeling chain relies inevitably on simplifications. The sound source spatial repartitions and sound power level and spectrum are approximations done to illustrate the framework with realistic values based on the literature and available data. But for other study cases, where results closer to the ground truth are expected, all parameters can be refined using for instance in situ pedestrians, traffic or birds counting, more complex density modeling, but also acoustic measurements of the sound sources of interest. For example, among the outstanding questions we may wonder whether the pedestrian positioning model, derived from the Montreal database, is not too erratic when applied to a European city like Nantes. Also, criteria such as the proportion of birds singing simultaneously (10%) should be supported by appropriate studies that have not yet been found in the literature.

The modeling blocks are for now completely independent. Thus, interactions between the sound sources are not modeled, except what concerns their masking. However, it is known for instance that the presence of road traffic noise affect for instance birds both in the presence and singing pitches [START_REF] Calder | The Scaling of Sound Output and Territory Size: Are They Matched?[END_REF][START_REF] Gentry | Evidence of suboscine song plasticity in response to traffic noise fluctuations and temporary road closures[END_REF].

This effect can amplify the masking of bird sounds through road traffic noise, what would require modeling some interactions, and thus prioritize the calculation of road traffic noise.

A one-second step was chosen because it is a commonly used value in environmental acoustics and it is thus easier to compare the results obtained with measurement databases. To study masking phenomena, smaller temporal resolution could probably be used, however, the modeling framework proposed in this article will remain valid.

Other sources can be added, such as wind on the trees or sea sounds, to adapt to the sound environment of any studied urban area. The addition of these two listed sound sources should not be problematic since they are fixed sources, with documented sound spectrums [START_REF] Bolin | Prediction Method for Wind-Induced Vegetation Noise[END_REF][START_REF] Bolin | Air-borne sound generated by sea waves[END_REF]; but the candidate sources for addition is wide, including bell sounds, rolling suitcases near train stations, etc.

The sound source emergence from a sound mixture is particularly interesting for soundscape studies and it would be of great interest to also include the masking effects into the final step, to present cartography even closer to the perception. Also, linking the produced indicators of emergence to perceived times of presence could be useful as this parameter intervenes for instance within sound pleasantness models [START_REF] Aumond | Modeling Soundscape Pleasantness Using perceptual Assessments and Acoustic Measurements Along Paths in Urban Context[END_REF].

The stochastic approach permits to add variability on input parameter. This can be used to add associated uncertainty as proposed in [START_REF] Courtois | Statistical ambient noise maps from traffic at world and basin scales[END_REF]. For instance, variability on voices sound source parameters could be included using 50% of men and 50 % of women voice spectrums.

Finally, in order to promote reproducible research and the reuse of the framework, all data and code are available on Github 6 .

Conclusion

A modeling framework adaptable to any of the sound sources encountered in urban areas has been The proposed framework integrates independent modeling blocks that leave open the improvements at each step of the modeling, or the choice between different models based on the available input data. The stochastic approach also enables the estimation of sound levels variability. The final original proposed sound maps permit to the reader to observe if a specific sound source is emerging from the global sound mixture, and so give very complementary information to the well-known sound level maps. 6 https://github.com/pierromond/NoiseModelling/wiki/12---Stochastic-Multi-Source-Sound-Mapping

Figure 1

 1 Figure 1 Graphical resume of Step 1. The potential sound sources, here birds, are placed in the park.

Figure 2

 2 Figure 2 Graphical resume of Step 2. All the propagation paths are calculated between the couples sound sources / receivers

Figure 3

 3 Figure 3 Graphical resume of Step 3.

Figure 4 .

 4 Figure 4.

Figure 4

 4 Figure 4 Graphical resume of Step 4.

Figure 5

 5 Figure 5 Case study area (a). Set of receivers (blue dots) (b)

Figure 6

 6 Figure 6 Spatial distribution of human voice sound sources within the study area: zones of walkability (light orange areas), final repartition of pedestrians (orange dots) and receivers area (dotted line).

  are found through a combination of Open Street Map layers. Without additional knowledge, one at firsts considers homogeneous spatial distribution of birds as shown in Figure 7. The height of the point sources for birds is set to 4 m.

Figure 7

 7 Figure 7 Spatial distribution of bird sound sources within the study area:

Figure 8

 8 Figure 8 Four 1-s equivalent sound maps i for (a) pedestrian voices; (b) birds; (c) fountains; (d) traffic

Figure 9

 9 Figure 9 Four L50 sound maps for (a) pedestrian voices; (b) birds; (c) fountains; (d) traffic

Figure 10

 10 Figure10.

  proposed and applied on a city district of Nantes, France. It consists of four steps: (Step 1) a spatial distribution of the potential sound source of interest; (Step 2) the calculation of a sound propagation matrix; (Step 3) the stochastic activation of the sound sources for n iterations of the sound map; (Step 4) the calculation of original sound indicators, some of which coming from the combination of computed sound maps.

Table 1

 1 Values given to user defined parameters used in this study for sound propagation

	User defined parameters	User configuration for this study
	Reflection order	2
	Diffraction Order	1
	Maximum propagation distance	250 m
	Maximum seeking wall	50 m
	Buildings height	10 m
	Receivers height	1.5 m

Table 2

 2 Study case sound spectrum for fountains (in dB)

		Global	63 Hz	125 Hz	250 Hz	500 Hz	1 kHz	2 kHz	4 kHz	8 kHz
	L W (dB)	90	43	51	62	76	81	85	83	84

  midday data), through multiple linear regressions. Equation 2 shows the constructed model, which expresses the pedestrian flow rate Q (pedestrian per hour), based on the density of shops K sh,300 in a radius of 300m, the density of public transport stations (metros and tramways) K PT,400 in a radius of 400m, and the density of restaurants K rt,400 in a radius of 400 m.

	Q = 5593 Ksh,300 + 36922 KPT,400 + 3042 Krt,400	Equation 2

and m the number of items within the radius R. The model is constructed on the Montreal dataset 5 http://donnees.ville.montreal.qc.ca/ (

Table 3

 3 Study case sound spectrum for voices (in dB)

		Global	63 Hz	125 Hz	250 Hz	500 Hz	1 kHz	2 kHz	4 kHz	8 kHz
	L W (dB)	71.5	35	50	53	57	53	50	45	40

Table 4

 4 Study case sound spectrum for bird songs (in dB)

		Global	63 Hz	125 Hz	250 Hz	500 Hz	1 kHz	2 kHz	4 kHz	8 kHz
	L W (dB)	85.5	-	-	-	-	54	80	84	68

http://orbisgis.org

https://www.openstreetmap.org

http://download.geofabrik.de

http://noise-planet.org/en/noisemodelling.html

Acknowledgments

The authors also would like to thank Solène Croci, and Jean-Michel Auberlet for fruitful discussion on birds and pedestrian spatial repartition in urban environment.