Tomi Kinnunen

Rosa González Hautamäki

Ville Vestman

Md Sahidullah

CAN WE USE SPEAKER RECOGNITION TECHNOLOGY TO ATTACK ITSELF? ENHANCING MIMICRY ATTACKS USING AUTOMATIC TARGET SPEAKER SELECTION

Keywords: Speaker verification, mimicry, spoofing

We consider technology-assisted mimicry attacks in the context of automatic speaker verification (ASV). We use ASV itself to select targeted speakers to be attacked by human-based mimicry. We recorded 6 naive mimics for whom we select target celebrities from VoxCeleb1 and VoxCeleb2 corpora (7,365 potential targets) using an i-vector system. The attacker attempts to mimic the selected target, with the utterances subjected to ASV tests using an independently developed x-vector system. Our main finding is negative: even if some of the attacker scores against the target speakers were slightly increased, our mimics did not succeed in spoofing the xvector system. Interestingly, however, the relative ordering of the selected targets (closest, furthest, median) are consistent between the systems, which suggests some level of transferability between the systems.

INTRODUCTION

It is well known that representation attacks [START_REF] Ratha | Enhancing security and privacy in biometrics-based authentication systems[END_REF][START_REF]Information technology -biometric presentation attack detection -part 1: Framework[END_REF] -also known as spoofing attacks -cast a shadow over the security of biometric systems. A spoofing attack involves an adversary (attacker) who aims at masquerading oneself as another targeted user to gain illegitimate access. Unprotected automatic speaker verification (ASV) systems can easily be spoofed using replay, voice conversion and text-to-speech attacks [START_REF] Wu | Spoofing and countermeasures for speaker verification: A survey[END_REF]. This has sparked research into spoofing countermeasures aimed at detecting the attacks from given audio. Community-driven benchmarks such as ASVspoof [START_REF] Wu | ASVspoof 2015: the first automatic speaker verification spoofing and countermeasures challenge[END_REF] and AVspoof [START_REF] Ergunay | On the vulnerability of speaker verification to realistic voice spoofing[END_REF] were launched for an organized study of countermeasures. In the context of security, continual arms race between attacks and their defenses is well known [START_REF] Biggio | Wild patterns: Ten years after the rise of adversarial machine learning[END_REF]: to develop effective countermeasures, it is necessary to understand the attacks. The speech synthesis community has independently launched voice conversion challenge [START_REF] Toda | The voice conversion challenge 2016[END_REF][START_REF] Lorenzo-Trueba | The voice conversion challenge 2018: Promoting development of parallel and nonparallel methods[END_REF] to advance VC methods. Within the past few years, active and dynamic communities both at the 'attack' and 'defense' sides of ASV, focused on technological attacks, have emerged.

In this study we focus on a nearly-forgotten ASV attackmimicry (impersonation). Unlike the technology-induced attacks, mimicry involves human-based modification of one's voice production. The question of recognizer vulnerability against mimicry was addressed at least around half a century ago [START_REF] Luck | Automatic speaker verification using cepstral measurements[END_REF][START_REF] Endres | Voice spectrograms as a function of age, voice disguise, and voice imitation[END_REF] and has remained a cursory topic within the ASV field [START_REF] Lau | Vulnerability of speaker verification to voice mimicking[END_REF][START_REF] Lau | Testing voice mimicry with the YOHO speaker verification corpus[END_REF][START_REF] Mariéthoz | Can a professional imitator fool a GMM-based speaker verification system?[END_REF][START_REF] Eriksson | The disguised voice: imitating accents or speech styles and impersonating individuals[END_REF][START_REF] Hautamäki | Automatic versus human speaker verification: The case of voice mimicry[END_REF][START_REF] Farrús | Voice disguise in automatic speaker recognition[END_REF]. While ASV vulnerability caused by technical attacks is widely reported, less (reliable) information is available on effectivess of mimicry, due to adoption of small, proprietary datasets. The authors are fully aware of the difficulties in collecting mimicry data from professional artists [START_REF] Hautamäki | Automatic versus human speaker verification: The case of voice mimicry[END_REF], whose prevalance in the general population is arguably very low. Nonetheless, if mimicry attacks could be shown to be a threat to ASV, it would be conceivably challenging to devise countermeasures: natural human speech lacks processing artifacts that enable detection of technical attacks. Thus, we argue that it is important to keep mimicry also in the list of potential attacks against ASV. Of particular interest in this work are mimicry attacks against celebrities whose voice data is available in massive amounts in the public domain. In line with the recent EU's General Data Protection Regulation (GDPR), intended to protect the privacy of its citizens, it is important to assess risks associated with multimedia data in the public domain. A recent study [START_REF] Lorenzo-Trueba | Can we steal your vocal identity from the internet? initial investigation of cloning obama's voice using GAN, WaveNet and low-quality found data[END_REF] has attempted voice cloning of celebrity voices based on found data using a pre-defined target speaker. The cloned voice samples were, however, detectable as spoofed speech.

We focus on technology-assisted mimicry attacks that uses ASV itself to identify potential target speakers to be subjected to mimicry attacks. The idea is to identify targets whose voice is similar to that of the attacker's voice, as this could potentially involve fewer articulatory or voice source modifications. Two related prior studies are [START_REF] Lau | Testing voice mimicry with the YOHO speaker verification corpus[END_REF] and [START_REF] Panjwani | Crowdsourcing attacks on biometric systems[END_REF] which involve search of either targets [START_REF] Lau | Testing voice mimicry with the YOHO speaker verification corpus[END_REF] or attackers [START_REF] Panjwani | Crowdsourcing attacks on biometric systems[END_REF] from a large set of candidates. The authors of [START_REF] Lau | Testing voice mimicry with the YOHO speaker verification corpus[END_REF] used a Gaussian mixture model (GMM) system to find closest, furthest and median targets from YOHO corpus for a few naive impersonators, leading to substantially increased false acceptance rate. In [START_REF] Panjwani | Crowdsourcing attacks on biometric systems[END_REF], the authors selected impersonators (rather than targets) through a commercial crowd-sourcing platform based on self-judgment and further refinement using ASV.

Our study can be seen as an attempt to reproduce the findings of [START_REF] Lau | Testing voice mimicry with the YOHO speaker verification corpus[END_REF] using up-to-date ASV technology and a far larger target candidate set (7, 365 celebrities pooled from VoxCeleb1 [START_REF] Nagrani | VoxCeleb: A large-scale speaker identification dataset[END_REF] and Vox-Celeb2 [START_REF] Chung | VoxCeleb2: Deep speaker recognition[END_REF]). A key methodological difference, however, is that unlike [START_REF] Lau | Testing voice mimicry with the YOHO speaker verification corpus[END_REF] that used a single GMM recognizer, we include two different ASV systems (Fig. 1). We argue that it may be unrealistic for the attacker to interact many times with the targeted ASV, but he/she may develop an offline substitute ASV that, after optimization, hopefully behaves similar to the attacked one. Our work bears some resemblance to black box attacks [START_REF] Papernot | Practical black-box attacks against machine learning[END_REF] in adversial machine learning [START_REF] Biggio | Wild patterns: Ten years after the rise of adversarial machine learning[END_REF], though our adversary is not a machine learning algorithm but a human. Further, those methods use either classifier output score or decision to optimize the attacks, while we assume that the attacker receives no feedback from the attacked system in any form. Thus, we expect that such attacks are not strong, but we argue that they are realistic given the abundance of multimedia data of celebrity speakers and various public-domain ASV implementations. We seek to answer the question in the title of this work, using a specifically collected 'attacker' data and VoxCeleb celebrity targets.

ASV-ASSISTED MIMICRY ATTACKS

Attack implementation

Let T = {Tj} J j=1 denote a set of unique, publicly known target speaker identities and let A = {A k } K k=1 denote a set of attacker identities. Given a pair of arbitrary utterances (or a pair of collections of many utterances), (Ui, Uj), a automatic speaker verification (ASV) system (speaker detector), D(Ui, Uj) computes a score s ∈ R with an arbitrary scale but higher relative values indicating stronger support that the source of Ui and Uj is the same speaker.

We consider two different types of ASV systems. The first one, attacker's ASV (Dpub), is a public-domain, known ASV implementation, and the latter, black-box ASV (Dblack), is the system an attacker attempts to spoof but whose internal workings, scores and decisions are inaccessible to the attacker. The proposed attack proceeds as follows:

ASV-assisted mimicry attack 1. Attacker A ∈ A records his/her natural voice sample, Unat.

2.

A uses Dpub to compute scores {sj} J j=1 between Unat and all the targets in a public database. A picks the closest target, j * = arg max J j=1 Dpub(Unat, Uj), where Uj contains the known recordings of Tj.

3.

A continues to use Dpub to pick the best-matching utterances of Tj * .

4. The attacker listens to the selected utterance(s) and attempts to adjust his/her voice towards the target. Once completed practicing, A submits a mimicked test utterance Umimic to Dblack(Umimic, Uj * ), the aim of being authenticated as Tj * .

Public-domain (attacker's) ASV system

The attacker's ASV uses i-vector front-end and probabilistic discriminant analysis (PLDA) back-end to compute speaker similarity scores. We extract 20 mel-frequency cepstral coefficients (MFCCs) using 20 filters1 , leading to 60 features per frame after including deltas and double-deltas. The features are processed with RASTA filtering and cepstral mean and variance normalization (CMVN). Non-speech frames are omitted using energy-based speech activity detector (described in Section 5.1 of [START_REF] Kinnunen | An overview of text-independent speaker recognition: From features to supervectors[END_REF]).

To train the i-vector extractor, we compute sufficient statistics using a universal background model (UBM) of 512 Gaussians. To train the UBM, we choose randomly 10,000 speech utterances from SI-284 subset of the Wall Street Journal corpus (WSJ0 and WSJ1) and 10,000 speech utterances from train-clean-360 and train-clean-100 subsets of the Librispeech [START_REF] Panayotov | Librispeech: An ASR corpus based on public domain audio books[END_REF] corpus. To train the T-matrix with 400 total factors, we randomly choose 20,000 speech utterances from the same subset of WSJ and 20,000 speech utterances from the same subset of Librispeech. Finally, the PLDA is trained on the entire SI-284 subset (from WSJ0 and WSJ1) and entire train-clean subset (from Librispeech) consisting 169,969 speech utterances from a total 1,455 speakers. The 400-dimensional i-vectors are further reduced to 250 dimensions with linear discriminant analysis (LDA) using the same data as in PLDA training, followed by centering and whitening. We use a simplified PLDA with 200-dimensional speaker subspace. We adopt MATLAB-based MSR Identity Toolkit2 to train the attacker's ASV.

The above development data and parameter selections are based on preliminary ASV experiments on the AVOID corpus (collected in [START_REF] Hautamäki | Acoustical and perceptual study of voice disguise by age modification in speaker verification[END_REF]) and VoxCeleb (1 and 2). For the AVOID corpus, we obtained EERs of 3.30% and 4.52% for text-dependent scenario with two English sentences separately. Further, we obtained EERs of 10.50% and 16.98% for text-independent ASV on two subsets of VoxCeleb1 and VoxCeleb2. These custom protocols were created by randomly choosing 60 speakers from VoxCeleb1 (6,163 target and 363,617 non-target trials) and VoxCeleb2 (9,118 target and 537,962 non-target trials).

Attacked ASV systems

In our experiments, we regard x-vector systems [START_REF] Snyder | X-vectors: Robust DNN embeddings for speaker recognition[END_REF] based on pretrained Kaldi [START_REF] Povey | The Kaldi Speech Recognition Toolkit[END_REF] recipes as ASV systems to be attacked. To emulate the scenario of attacker's limited knowledge of this system, the attacker's ASV is made intentionally different from the attacked ASV systems in terms of feature extractor set-up, embedding type, and development corpora (Table 1). Both attacked systems are based on Kaldi recipes for VoxCeleb and NIST Speaker Recognition Evaluation 2016, while the attacker's system uses i-vectors.

CORPUS OF TARGET SPEAKERS: VOXCELEB

The attacker's ASV is used as a voice search tool to find the closest speakers from the combination of VoxCeleb1 [START_REF] Nagrani | VoxCeleb: A large-scale speaker identification dataset[END_REF] and Voxceleb2 [START_REF] Chung | VoxCeleb2: Deep speaker recognition[END_REF] to each of the locally recruited subjects (described in Section 4). The combined VoxCeleb corpus contains about 1.3 million speech excerpts extracted from more than 170,000 YouTube videos from J = 7, 365 unique speakers. This totals to about 2,800 hours of audio material that is, for the most part, active speech. Both VoxCeleb corpora were collected using automated pipeline exploiting face verification and active speaker verification technologies [START_REF] Chung | VoxCeleb2: Deep speaker recognition[END_REF].

VoxCeleb1 contains mostly English speech, while VoxCeleb2 is more diverse in nationalities and languages. The nationality information of the target speakers was of our interest, as the recruited local speakers are Finnish and we wanted to see if Finnish people do better job at imitating Finnish targets than non-Finnish. According to the VoxCeleb1 metadata, there were no Finnish targets in Vox-Celeb1. The VoxCeleb2 did not include nationality metadata, so we Table 1. Details of the speaker verification systems used to simulate targeted impersonation attack against automatic speaker verification. The attacker is assumed to not have information about the attacked systems, and hence the attacker's system differs from the attacked systems. made a script to automatically obtain nationalities using Google's Knowledge Graph API 3 . With this strategy we found 44 Finnish speakers from VoxCeleb2.

LOCALLY RECRUITED ATTACKERS

Speakers and recording gear

We recruited K = 6 voluntary local speakers (4M + 2F) to serve as 'attackers'. All are native Finnish speakers and one of them is a co-author of this study. All the six speakers, selected based on their availability, took part in 2015 to the recordings of [START_REF] Hautamäki | Acoustical and perceptual study of voice disguise by age modification in speaker verification[END_REF], currently under release with the name AVOID corpus 4 . We adopt the same recording setup and part of text prompts from [START_REF] Hautamäki | Acoustical and perceptual study of voice disguise by age modification in speaker verification[END_REF] but otherwise the two studies are unrelated. All the subjects signed an informed consent form to use their speech data for research, and were rewarded with movie and coffee tickets. Two of the male subjects knew the specific goals of our study while the remaining four subjects were not informed that the text and target speakers were tailored for them, nor where do the voices originate from. They were not informed that the study relates to ASV vulnerability, but were merely asked to mimic the targets as accurately as they could.

As illustrated in Fig. 1, the subjects took part to three recording sessions. The first session, produced in the subject's natural voice, is used for VoxCeleb target speaker selection, while the remaining two sessions serve for vulnerability analysis of the attacked systems. The tasks in the recording sessions differed, while the recording setup was the same: recordings took place in a silent laboratory room with a portable Zoom H6 Handy Recorder using an omnidirectional headset mic (Glottal Enterprises M80) with 44.1 kHz sampling and 16-bit quantization. Three other channels (two smartphones, electroglottograph) were also collected, but are not used in this study.

The first recording session (data for target search)

The first session, used for the targeted VoxCeleb speaker search, consists of four tasks in the speaker's natural voice. The tasks consisted of spontaneous speech and read text (13 sentences) in both Finnish and English. The read texts in Finnish are the same used in [START_REF] Hautamäki | Acoustical and perceptual study of voice disguise by age modification in speaker verification[END_REF] and their corresponding English versions were added for this study. We have approximately 6 minutes of speech (before speech activity detection) per speaker from Session 1.

Attacked target speaker search and utterance selection

For the purpose of targeted speaker search, we compute a single averaged i-vector for each of the six speakers resulting from 28 individual utterances from Session 1. Similar to [START_REF] Lau | Testing voice mimicry with the YOHO speaker verification corpus[END_REF], we use the ASV system to pick for each attacker the closest, median, and furthest speakers among the VoxCeleb speakers. The closest one is most relevant for vulnerability analysis while the other two serve for reference purposes. We do this ASV-assisted search separately for all the VoxCeleb speakers (unconstrained search from 7,365 speakers) and for the subset of 44 Finnish speakers. We pool all the speech data of the VoxCeleb speakers to compute average i-vector per target.

In addition to the three ASV-selected targets, we include common target matched with the speaker's gender, in both Finnish and English. The common Finnish targets are Päivi Räsänen (F, politician) and Ilkka Kanerva (M, politician), and the common English targets are Hillary R. Clinton (F, politician) and Leonardo DiCaprio (M, actor). Even if well-known persons, from the viewpoint of ASV they are random targets with no strong presuppositions how similar their voices are to our attackers.

In summary, for each of our 4 male and 2 female subjects, we select 6 customized targets (3 ASV-ranks × 2 languages) and 2 common gender-matched ones (one Finnish, one English). This gives a theoretical total of 3 × 2 × 4 male + 2 common male + 3 × 2 × 2 female + 2 common female = 40 target speakers. Not all of the ASV-selected targets are unique, however: one male Finnish celebrity was the closest target for three attackers, one male Finnish celebrity repeated as the median speaker for two male attackers, and one female Finnish celebrity is the furthest speaker for the two female attackers. The final number of unique celebrity targets is 36.

For each of the 36 target speakers, we selected at minimum 30 seconds of active speech to evaluate the ASV system attacks. This duration of speech was collected from multiple shorter utterances for two reasons: First, the segments in VoxCeleb corpora are typically about 5 to 10 seconds long. Secondly, as we were going to ask the recruited attackers to imitate the target speakers, shorter utterances would be easier to impersonate.

We utilized Attacker's ASV also for the utterance selection. For the closest targets, we selected the highest scoring utterances, while for the furthest targets, the utterances with the lowest scores where selected. For the median speakers, we selected the utterances close to mean. This was further accompanied by manual inspection: if the audio quality (determined subjectively by listening) was not deemed high enough, we discarded it and moved on to the next ones in the ranked list.

Speech transcription and the mimicry recordings

Unlike the first recording session (common to all subjects), the second and third sessions were customized for each subject. This process involves the use of speech transcripts of the selected target utterances. To this end, we used Amazon's Mechanical Turk5 (MTurk), a commercial crowdsourcing service, to transcribe the English language audio. The Finnish transcripts were produced by two native Finnish speakers. The 35 MTurk crowdworkers and the 2 Finnish transcribers were asked to transcribe all the nuances of conversational speech, including repetitions, hesitations, filler words etc. Finally, two reviewers audited the quality of all the transcripts. In Session 2, which took place 5 to 6 weeks after Session 1, the subject was provided with the transcripts of the selected target utterance(s) and was asked to read the sentences twice in his or her natural voice. The speaker was not informed whose speech the transcripts corresponded to; even the coauthor taking part to the study was unaware whose transcripts he was reading. The rationale of including this session was to familiarize each attacker with the target speaker sentences. We adopted the general idea to include a session with reference text only and another one with audio from the design used in [START_REF] Mariéthoz | Can a professional imitator fool a GMM-based speaker verification system?[END_REF].

In the last session, which took place 2 to 6 days after Session 2, the subjects were provided with the same transcript as in Session 2, but in addition they had now access to the actual target speaker audio excerpts played through headphones. The transcripts were provided on a printed paper and the audio reference from a tablet computer that the subject was able to interact with; he/she could play the target utterance(s) as many times as needed, and he/she then tried to mimic the voice according to their best skills. Again, the subject was asked to mimic each sentence twice. In the experiments, we use only the second recording of each sentence.

RESULTS

In the following, we evaluate effectiveness of the mimicry attacks. The target speaker models used in the experiments were enrolled using all available segments except those selected for testing as described in Section 4.3.

Figure 2 displays how the attacker's PLDA scores compare to the target scores. The general findings are as expected. First, the order of the closest, the median, and the furthest speakers transfers from the attacker's ASV system to the attacked ASV systems implying that the ASV-assisted speaker selection can help in ASV attacks. Second, in general, the mimicry attempts were not successful as the attacker's natural and mimicry scores are significantly (and substantially) below the target scores. Additionally, we find no significant difference between the natural and impersonated versions. Finally, as the recruited attackers were Finnish, attackers' scores against the Finnish targets were higher than for non-Finnish targets.

To look at the effect of mimicry closer, we analyze the difference of mimicked and natural speech scores (Table 2). Interestingly, and contradictory to what we assumed, if the target speaker's voice is already close to the attacker's voice, the impersonation attempts degrade the score. The same finding was noted in situations where the target is a well known public figure like the targets in the common category are. We suspect that the effect might be due to people having higher tendency to overact someone they already know well. However, in the case of the targets that are not close to the attackers and, on average, are not so well known (median and furthest categories), impersonation is helpful. Figure 3 shows a sample of the best and worst attackers for the common targets, with similar findings as above. Our attackers are native Finnish speakers recorded with a specific set-up which could be different from the VoxCeleb conditions. This raises a question whether our mimicry attacks were unsuccessful simply due to domain mismatch. To address this concern, we ran an additional experiment, in which we scored attacker's test segments against attacker's own speaker models. If our ASV systems are able to cope with domain mismatch, we expect high detection scores similar to the VoxCeleb target scores). The results shown in Figure 4 confirm this as, similarly to Figure 2, the averaged log likelihood ratios are close to 50 for the attacked ASV system 1. From Figure 4 we also find that impersonation lowers the scores, showing that the ASV system is not robust against disguise (the act of attempting to be not recognized as oneself). This finding was not a surprise to us [START_REF] Hautamäki | Acoustical and perceptual study of voice disguise by age modification in speaker verification[END_REF]. Attacked ASV system 1

Attacker's natural voice Attacker's impersonated voice

CONCLUSION

Can one use ASV technology to attack itself? To answer this, we collected 6 native Finnish mimics and used ASV to locate customized targets from VoxCeleb. Our preliminary analysis reveals that, unlike in [START_REF] Lau | Testing voice mimicry with the YOHO speaker verification corpus[END_REF], our mimicry attempts were unsuccessful. In fact, the ASV scores even degraded, specifically when the impersonator's natural voice was already close to the target speaker's voice. Further work is required to analyze the reasons, specifically in terms of acoustic modifications implemented by our naive impersonators. The relative ordering of the closest, median and furthest speaker was, however, preserved across the attacker's and attacked ASV systems, with higher relative scores obtained for Finnish targets. Though our assisted attacks did not succeeed to spoof state-of-the-art x-vector technology, selection of imposters from a larger set of speakers (e.g. using crowd-sourcing [START_REF] Panjwani | Crowdsourcing attacks on biometric systems[END_REF]) may help in spoofing ASV systems.

1 :

 1 Search of potential celebrities to attack. SESSION 2: Rehearsal of target text, no mimicry.

SESSION 3 :

 3 Mimicry (from transcript and audio).

Fig. 1 .

 1 Fig.1. Automatic speaker verification (ASV) assisted mimicry attack: attacker uses a public-domain ASV system to select target speakers matched with his/her voice from a public celebrity data. The attacker then practices target speaker mimicry, intended to attack another independently developed ASV system.

Fig. 2 .

 2 Fig.2. Comparison of attackers' ASV scores (log likelihood ratios) to the targets' scores for all the three ASV systems involved in the study. The scores are averaged over all attackers and all speech segments. The error bars represent 95 % confidence intervals for the means.

Fig. 3 .

 3 Fig.3. Scores against the common celebrity targets for the best and the worst impersonators (Color codes are equal to Fig.2).

Fig. 4 .

 4 Fig.[START_REF] Wu | ASVspoof 2015: the first automatic speaker verification spoofing and countermeasures challenge[END_REF]. Attacker's scores in the case where attacker's test sentences are tested against attackers' own speaker models instead targets' speaker models as in Figure2.

Table 2 .

 2 Score differences between attacks with impersonated voices and attacks with natural voices. Differences are averaged over attackers, target nationalities, and utterances. ± indicates 95 % confidence intervals. In the case of the closest target speakers, impersonation attempts are counterproductive.

	ASV system	Closest	Median	Furthest	Common
	Attacker	-9.7 ± 5.2	2.2 ± 4.3 5.9 ± 7.1 -7.2 ± 4.3
	Attacked1 -5.2 ± 3.9	9.2 ± 3.3 6.1 ± 4.3 -0.5 ± 3.8
	Attacked2 -3.7 ± 5.5 15.0 ± 7.0 4.7 ± 7.4 -4.0 ± 7.7

http://cs.joensuu.fi/~sahid/codes/AntiSpoofing_ Features.zip

https://www.microsoft.com/en-us/download/ details.aspx?id=52279

https://developers.google.com/knowledge-graph/

http://urn.fi/urn:nbn:fi:lb-2018060621

https://www.mturk.com/

The work was supported by Academy of Finland (project no. 309629).


Alain Bourmaud
email: alain.bourmaud@univ-ubs.fr

David Siniscalco

Loic Foucat

Camille Goudenhooft

Xavier Falourd

Bruno Pontoire

Olivier Arnould

Johnny Beaugrand

Christophe Baley

Loïc Foucat

Evolution of flax cell wall ultrastructure and mechanical properties during the retting step

Keywords:

à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Introduction

Flax (Linum Usitatissimum L.) is probably one of the oldest fibre plants used by humans, especially by the prehistoric cultures of Europe, but also in Egypt, ancient Babylonia, Palestine and along the shores of the Black Sea [START_REF] Heer | Prehistoric culture of flax[END_REF]. Wild flax fibres have been identified in the ruins of the oldest Neolithic lacustrine villages in Switzerland, while 30,000-year-old flax fibres have been discovered in Upper Palaeolithic layers in Georgia [START_REF] Kvavadze | 30,000-year-old wild flax fibers[END_REF]. Thus, prehistoric huntergatherers used flax fibres to manufacture cords for hafting stone tools, baskets and clothing. To extract fibres and separate them from the woody part of the stem, ancient civilisations developed water retting which is one of the oldest known bioprocesses. A first description of the retting stage can be found in the Sumerian poem, The Bridal Sheet (Poem,3000BC), and archaeological discoveries indicate that this process was carried out at various places and periods in Antiquity.

Over the last few years, flax fibres have demonstrated their great potential as reinforcements of composite materials and their use is developing in several industrial domains, particularly in the transport sector [START_REF] Witayakran | Natural fiber composites: Review of recent automotive trends[END_REF]. Nevertheless, the need for ever-increasing productivity and the problem of storage leads to new consideration about the control and industrialization of the retting process. Indeed, this bioprocess often takes place under very different environmental conditions from one year to another, and its effects on the cell wall ultrastructure and fibre performance are poorly known. During retting, the various microorganisms present in the soil, due to environmental conditions and in particular the alternation of rain, dew and sun, colonize the flax stems [START_REF] Djemiel | Characterization of bacterial and fungal community dynamics by high-throughput sequencing (HTS) metabarcoding during flax dew-retting[END_REF][START_REF] Sharma | Effect of sample preparation and heating rate on the differential thermogravimetric analysis of flax fibres[END_REF]. The enzymes they secrete, in particular polygalacturonases and xylanases, gradually degrade the pectic compounds mainly located in the middle lamellae of the fibre bundles but also in the interfacial areas with the cortical parenchyma or xylem. This damage to the plant tissues structure facilitates the extraction the fibre bundles during the scutching stage. The fibre bundles are also easier to separate [START_REF] Akin | elementary fibres of flax and glass: Analysis of reproducibility and scattering[END_REF], which favours good composite mechanical properties due to the improved fibre individualization and the reduction in the number of bundles [START_REF] Coroller | Effect of flax fibres individualisation on tensile failure of flax/epoxy unidirectional composite[END_REF]. Sharma [START_REF] Sharma | Chemical retting of flax using chelating compounds[END_REF] showed that the limiting factor of retting is the degradation of homogalacturonans, in particular when these constituents are located in the epidermis; over-retting results in the loss of the cellulose that makes up the fibres [START_REF] Placet | The influence of unintended field retting on the physicochemical and mechanical properties of industrial hemp bast fibres[END_REF], arguably due to the well-known secretion of a second type of enzyme (cellulase).

Although the impact of retting on the properties and performances of plant fibres has already been investigated, the conclusions of these studies are unfortunately somewhat contradictory. [START_REF] Van De Weyenberg | Influence of processing and chemical treatment of flax fibres on their composites[END_REF] obtained similar mechanical properties for fibre bundles of green, half-retted and retted flax, with stresses at break of 68.9, 72.8 and 71 cN.tex -1 , respectively, which indicate no significant change in performances. The same trend was highlighted by [START_REF] Alix | Pectinase treatments on technical fibres of flax: Effects on water sorption and mechanical properties[END_REF]). By contrast, other authors [START_REF] Martin | Influence of the degree of retting of flax fibers on the tensile properties of single fibers and short fiber/polypropylene composites[END_REF][START_REF] Van De Velde | Thermal and mechanical properties of flax fibres as potential composite reinforcement[END_REF] have measured higher tensile strength at break and Young's modulus values for the most retted single flax fibres.

The aim of the present study is to explore the evolution of the flax cell wall behaviour during a controlled dew retting phase. Structural and mechanical investigations were conducted at the cell wall scale using X-Ray Diffraction (XRD) and solid-state Nuclear Magnetic Resonance (NMR), while nanomechanical measurements were performed through nanoindentation and AFM Peak Force Quantitative Nano-Mechanical property mapping (AFM PF-QNM).

Materials and methods

Flax fibres

Flax plants of the Alizée variety were grown in France (Hauts-de-France region) in 2011 and were provided by Van Robaeys Frères® (Killem, France); they were carefully stored at controlled humidity and temperature (48% RH and 23 °C). Fibres samples are labelled R1, R2, R3, R4, R5 and R6, which corresponds to retting times of 1, 5, 9, 14, 16 and 19 days, respectively. Flax stems were pulled at the end of June and dew-retted in the field for different durations. Additional information about cultivation and climatic conditions are provided in [START_REF] Martin | Influence of the degree of retting of flax fibers on the tensile properties of single fibers and short fiber/polypropylene composites[END_REF].

Sample embedding preparation

A series of ethanol/deionized water concentration (50%, 75%, 90% and 100%) were used to dehydrate samples. After this step, to maintain the fibre and the cell-wall structure during surface preparation, samples were embedded in a mixture containing increasing proportions of LR (London Resin) White acrylic resin/ethanol (25%, 50%, 75% and 100%). Then, the polymerization of the resin was finalised in an oven at 60 °C for one night. The aramid fibres were introduced into an epoxy resin (Struers Epofix) for 24 h and polymerization took place at room temperature. To obtain reliable measurements with the atomic force microscope, we must minimize their roughness at the nanometer scale. Consequently, the samples were prepared using an ultramicrotome (Leica Ultracut R) with two consecutive diamond knives (Diatome Histo and Ultra AFM), to cut very thin sections (about 50 nm) at low cutting speed (≈1 mm/s) and thus reduce sample deformations.

Nanoindentation and AFM peak force QNM measurements

A commercial nanoindentation system Nanoindenteur XP (MTS Nano Instruments) was used at room temperature. All the samples were tested by using a Berkovich indenter (three-side pyramid), with a maximum depth of 150 nm (loading rate of 0.05 s -1 (i.e., 1 μN/s)). When the maximum load was reached, a holding segment of 20 s was performed before unloading (unloading rate of 10 μN/s), followed by a thermal drift correction. The indentation modulus was calculated using the approach of Oliver and Pharr [START_REF] Oliver | An improved technique for deterimining hardness and elastic modulus using load and displacement sensing indentation experiments[END_REF]; the values reported here are equal to the means of measurements obtained over at least 30 valid indentation locations.

Mechanical mapping of the sample surfaces was performed with a Multimode AFM instrument (Bruker Corporation, USA) by using the PF-QNM imaging mode. In this mode, the vertical motion was set at a frequency of 2 kHz (well below the cantilever resonance frequency, around 525 kHz). Mechanical property mapping was carried out using a RTESPA-525 (Bruker) probe with a cantilever spring constant around 140 N/m. Due to its high value, this cantilever stiffness was calibrated using the Sader method [START_REF] Sader | Spring constant calibration of atomic force microscope cantilevers of arbitrary shape[END_REF], as well as Scanning Electron Microscope (Jeol JSM 6460 L V) images for the measurements of the cantilever dimensions (i.e., length and width). The tip radius, around 30 nm, was adjusted on a Highly Oriented Pyrolytic Graphite (HOPG) reference sample, providing an indentation modulus of around 18 GPa. For all the measurements, the applied force was set at 200 nN and the curves obtained were adjusted by a DMT contact model which takes into account the adhesion force [START_REF] Barthel | Adhesive elastic contacts: JKR and more[END_REF].

Undulations or waves are sometimes visible on the mechanical mapping images obtained (Fig. 1). This is due to the preparation of the section of cellulose microfibril. This work is based on the total cellulosic surfaces of amorphous cellulose and a microfibril model with cellulosic chains 0.57 nm wide [START_REF] Newman | Estimation of the lateral dimensions of cellulose crystallites using C-13 NMR signal strengths[END_REF]. Finally, CP/MAS 13 C variable contact time (VCT) spectroscopy was used to characterize the molecular dynamics of the samples. Twenty CP/MAS spectra were acquired using contact times from 10 μs to 9 ms, and accumulation of 920 scans for each spectrum. We then used a two-proton reservoir model [START_REF] Kolodziejski | Kinetics of cross-polarization in solid-state NMR: A guide for chemists[END_REF][START_REF] Paris | NMR local range investigations in amorphous starchy substrates: II-Dynamical heterogeneity probed by 1H/13C magnetization transfer and 2D WISE solid state NMR[END_REF] to follow the evolution of carbon peak areas as a function of contact time.

Statistical analysis

The mean values of the indentation modulus of the different batches were evaluated in terms of statistical difference. A bilateral t-test with a 95% confidence interval was used and the value of the P-parameter was determined. When it is less than 0.05, the difference between two batches is statistically significant.

SEM analysis

A Jeol JSM 6460 L V scanning electron microscope was used to analyze the surface of the fibre bundles corresponding to the different degrees of retting. After being cut into 5 mm pieces, the flax fibre bundles were glued to a sample holder using a conductive adhesive. They were then metallized with a thin layer of gold using an Edwards Scancoat Six device during 180 s.

Results and discussion

Use of AFM PF-QNM and nanoindentation to explore structure and indentation modulus of flax cell walls

For all levels of retting, AFM PF-QNM modulus mapping (Fig. 1) reveals a homogenous indentation modulus across the flax cell walls without any gradient [START_REF] Goudenhooft | Investigation of the mechanical properties of flax cell walls during plant development: The relation between performance and cell wall structure[END_REF], indicating the maturity of the bast flax fibres, even for the R1 level obtained at the pulling-out stage.

Fig. 2 shows the evolution of indentation modulus obtained by both AFM PF-QNM and nanoindentation tests. Here, nanoindentation is used to validate the AFM PF-QNM mechanical measurements; we note a good correlation between the two sets of data, with the same general tendency despite an underestimation of the indentation modulus [START_REF] Arnould | Towards a better understanding of wood cell wall characterisation with contact resonance atomic force microscopy[END_REF]. AFM PF-QNM and nanoindentation measurements were not performed on the same fibre, which could lead to slight differences in stiffness values due to the natural character of flax cell walls.

To compare the indentation modulus values with literature data, it is appropriate to consider measurements on the R6 batch, corresponding to conventional retting before scutching. For this sample, nanoindentation and AFM PF-QNM indentation moduli are 19.3 ± 3.8 GPa and 17.2 ± 1.5 GPa, respectively, which is in good agreement with previous measurements on mature and dew-retted flax cell walls [START_REF] Alix | Pectinase treatments on technical fibres of flax: Effects on water sorption and mechanical properties[END_REF]. The indentation moduli are of the same order of magnitude as values obtained on the mature G-layer [START_REF] Goudenhooft | Investigation of the mechanical properties of flax cell walls during plant development: The relation between performance and cell wall structure[END_REF]; in addition, the non-mature Gn-layer cannot be distinguished through AFM PF-QNM investigations (Fig. 1), demonstrating that the fibres are well-matured. Interestingly, we find a significant increase of the indentation modulus with retting level. Differences between R1 and R6 are +17% and +23% with AFM PF-QNM and nanoindentation measurements, respectively; to the authors' best knowledge, this stiffening has never been demonstrated at the cell wall scale. This increasing tendency is confirmed by the t-test statistical analysis, shown in Table 1. Sample sets were compared in pairs and values found to be statistically significant (P < 0.05), excepting the values comparing R2 to R4 and R3 to R5.

surface with the microtome. This is a consequence of the AFM laser and its reflection on the samples surface, which causes interferences, especially here as the sample surface reflection q uality i s h igh d ue t o its preparation. These interferences induce a bias, especially in the measurement of the adhesion force [START_REF] Kassies | Removing interference and optical feedback artifacts in atomic force microscopy measurements by application of high frequency laser current modulation[END_REF][START_REF] Méndez-Vilas | Optical interference artifacts in contact atomic force microscopy images[END_REF]. In our case, for some measurements, nothing could be done to reduce or remove them, but the way we analysed the data reduces their effect o n t he v alues r eported h ere. T he c ell wall stiffness d istribution w as i nferred b y a djusting, w ith a G aussian distribution (OriginPro 2017 software), the height distribution function (Gwyddion 2.39 software) calculated from the indentation modulus maps. This fitted Gaussian distribution yields the mean value (equal to the value at the maximum of the distribution here as this latter is symmetric) and the standard deviation of the indentation modulus.

Finally, these images are used to determine the thickness of the interface between fibres. T his p rocedure i s c arried o ut a long almost three lines per interface as shown in Fig. 1. These interfaces are initially composed of middle lamellae, but these are gradually replaced by embedding resin during progression of the retting process.

X-ray diffraction

Flax fibre bundles (diameter ∼ 150 μm) were extracted from plants and mounted on a Bruker-AXS D8 Discover diffractometer (Karlsruhe, Germany). Cu Kα 1 radiation (λ = 1.5405 Å), generated in a sealed tube at 40 kV and 40 mA, was parallelized using a Gobël mirror parallel optics system and collimated to produce a 500 μm incident beam diameter. Diffraction p atterns were o btained b y r ecording s cans (10 min for each sample) in the 2θ angle range between 10° and 35°. The cellulose crystallinity index (I c ) of the fibre bundle was calculated using the common method described by Segal et al. [START_REF] Segal | An empirical method for estimating the degree of crystallinity of native cellulose using the X-ray diffractometer[END_REF].

NMR investigations

First, the flax s amples w ere h ydrated w ith 4 0% w ater ( w/w) in order to improve the spectral resolution. Solid state 1 H/ 13 C CP/MAS NMR experiments were performed on a Bruker Avance III 400 MHz spectrometer with a 13 C frequency of 100.62 MHz by using a double resonance H/X CP/MAS 4 mm probe. The samples were spun at room temperature and at a rate of 9 kHz. The cross-polarisation pulse sequence parameters were as follows: 3.0 μs proton 90° pulse, 1.75 ms contact time at 37 kHz and 10 s recycle time. Results were obtained after the accumulation of 5120 scans. The chemical shift of the 13 C NMR spectra was calibrated thanks to the use of the carbonyl signal of glycine (176.03 ppm), which was used as an external standard. All data were processed using Gaussian multiplication parameters of L B = -20 Hz and GB = 0.05 and then Fourier transform was performed.

The approach of [START_REF] Larsson | A CP/MAS13C NMR investigation of molecular ordering in celluloses[END_REF]) was used to determine the cellulose crystallinity from the peak deconvolution of the C4 region (77-92 ppm. In the crystalline region the L arsson & al. approach is performed thanks to the use of three Lorentzian peaks corresponding to cellulose Cr (I α ) (89.4 ppm), cellulose Cr (I α+β ) (88.7 ppm) and cellulose Cr (I β ) (87.7-87.8 ppm). In addition, an additional Gaussian peak representing the para-crystalline (PCr) contribution (88.5-88.6 ppm) is used. Three others peaks were exploited in the amorphous region, two Gaussian peaks corresponding to the accessible surface (AS; 83.2 and 84.1-84.2 ppm) and another one for the inaccessible surface (IAS; 83.5-83.7 ppm).

The chemical shift, peak half-width and peak area of the different peaks were determined with a least squares fitting m ethod using Peakfit® s oftware. T he c ellulose c ell w all c rystallinity r ate w as calculated by dividing the area of the four peaks of the crystalline region by those of the seven peaks for the cellulose C4 region. The lateral dimensions of the fibrils ( L FD) a nd t he l ateral d imensions o f t he fibril aggregates (FLAD) were then estimated using a square model of a cross-However, some similar results have been obtained on the same fibre batches in another study focused on the tensile characterization of single flax fibres [START_REF] Martin | Influence of the degree of retting of flax fibers on the tensile properties of single fibers and short fiber/polypropylene composites[END_REF]. In this latter study, the Young's modulus of the fibre was shown to increase by +44% between levels R1 and R6. The increase difference between Young's modulus and indentation modulus is due to the anisotropic nature of the cell wall [START_REF] Eder | Experimental micromechanical characterisation of wood cell walls[END_REF] and the difference in the investigation scale (i.e., whole fibre for the tensile test compared to local measurements at the cell wall scale in the present study). Therefore, there is a significant evolution of the overall fibre and cell wall stiffness during the retting process, evidenced by both tensile tests; nanoindentation and AFM. Although [START_REF] Martin | Influence of the degree of retting of flax fibers on the tensile properties of single fibers and short fiber/polypropylene composites[END_REF] did not conduct a structural investigation, these authors used TGA analysis and hot-water/EDTA extraction to show a decrease in cortical parenchyma and middle lamella components, which is consistent with a dew-retting process. Similar conclusions were obtained for hemp fibres by Placet et al. [START_REF] Placet | The influence of unintended field retting on the physicochemical and mechanical properties of industrial hemp bast fibres[END_REF], who found a significant decrease of pectic components. Moreover, these latter authors observed a decrease of xylan component which plays a determining role in the cell wall architecture and could potentially be linked to changes in the stiffness of the cell wall. According to these authors, this major structural evolution is linked to the worsened mechanical performance of single fibres, demonstrating that the tested hemp batch was over-retted and exposed to a profound, and therefore detrimental, enzyme degradation. These authors also highlight the great uncertainty in monitoring hemp retting compared to flax cultivation. In the latter case, exogenous parameters such as temperature are constantly checked by calculating accumulated temperature data [START_REF] Lefeuvre | Tensile properties of hemp fiber mechanical properties and their scattering (Fedora 17)[END_REF]. In view of these considerations, we can exclude over-retting in the present case and the positive evolution of cell wall mechanical properties should be linked rather to a change in the cell wall ultrastructure or conformation, such as an evolution of the cellulose crystallinity. These points are explored further below.

AFM PF-QNM investigations also provide interesting information about the bundle structure of scutched fibres during the retting process. As shown in Fig. 1, the middle lamella is clearly visible for the first three retting levels (R1-R3) and appears to be cohesive with the surrounding fibres, especially at the junctions between fibres; with the increase of retting level and development of enzyme action, this middle lamella progressively disappears until it represents only a few fragments, as in the case of the R4 sample where it is still present but not physically linked with the primary wall of the fibre. At the same time, the inter-fibre space is filled with embedding resin, indirectly showing the importance of retting for enhancing the composite impregnation. Fig. 3.A illustrates the qualitative evolution of the thickness of the interface between fibres (i.e., middle lamella and/or embedding resin, see measurement lines in Fig. 1) throughout the retting process and the impact on fibre divisibility. Distance values correspond to the average interface length between fibres calculated from ten profile measurements represented by blue lines on Fig. 1. These observations are supported by additional mechanical investigations (Fig. 3B) In fact, local measurements performed by AFM PF-QNM in the middle lamella region yield indentation moduli of 12.5 ± 0.7 GPa, 13.6 ± 0.8 GPa and 12.4 ± 1.7 GPa, for R1, R2 and R3 samples, respectively. On the other hand, similar investigations carried out on the most retted samples (R4, R5 and R6) give results between 5.4 ± 0.2 GPa and 6.0 ± 0.3 GPa, which correspond to the resin indentation modulus. The resin indentation modulus is slightly higher here than the literature value [START_REF] Goudenhooft | Investigation of the mechanical properties of flax cell walls during plant development: The relation between performance and cell wall structure[END_REF], presumably due to the presence of a residual middle lamella fraction in the inter-fibre space, especially in R4 and R5 samples. Values obtained in the middle lamella region are slightly higher than those reported in the literature for wood by nanoindentation in the cell corner middle lamella [START_REF] Gindl | The significance of the elastic modulus of wood cell walls obtained from nanoindentation measurements[END_REF]. A better knowledge of the middle lamellae is of great interest here, especially to highlight their cohesive function in fibres bundles; nevertheless, these data must be considered with caution, being only of relative value to compare cell wall indentation moduli, and should not be used, for example, to establish an absolute value of the longitudinal Young's modulus in numerical or micromechanical modelling.

Monitoring of crystallinity degree during retting using X-ray diffraction and NMR analysis

In the literature, an increase in the crystallinity of plant fibres during retting has been demonstrated from crystallinity index measurements obtained by X-Ray diffraction. [START_REF] Zafeiropoulos | A study of transcrystallinity and its effect on the interface in flax fibre reinforced composite materials[END_REF] report an increase of the crystallinity index from 64.55% for green flax to 71.64% for dew retted flax. Moreover, [START_REF] Marrot | Analysis of the[END_REF] report an increase in the degree of crystallinity of hemp fibres following the retting of the plant in the field. The crystallinity index increased from 45.8% for green hemp, to 71.3%, 71.2% and 74.5% after 14, 31 and 45 days of ground retting, respectively.

In the present case, the cell wall cellulose crystallinity of R1 and R6 was investigated by both NMR and XRD. We chose to focus only on retting levels R1 and R6; the important variations in mechanical properties lead us to focus on these two extreme cases. Fig. 4 shows the 13 C CP/MAS NMR spectra obtained for R1 and R6 samples. These spectra are characteristic of type-I cellulose [START_REF] Dinand | Mercerization of primary wall cellulose and its implication for the conversion of cellulose I{\textrightarrow}cellulose II[END_REF] for the two samples studied. Characteristic hemicelluloses signals are also observed as part of the main constituents of flax plant cell walls. To obtain information on the supramolecular structure of cellulose fibrils in the plant cell walls, the C4 region between 77 and 92 ppm is deconvoluted here using the model of [START_REF] Larsson | A CP/MAS13C NMR investigation of molecular ordering in celluloses[END_REF]. Fig. 5 illustrates this deconvolution, while Table 2 reports the peak assignments with their corresponding values of full width at half height (FWHH) and normalized areas. The crystallinity appears higher for the R6 sample (56%) than for the R1 sample (52%), a slight but significant difference, see [START_REF] Van De Velde | Thermal and mechanical properties of flax fibres as potential composite reinforcement[END_REF]. Although the NMR-derived crystallinity values are lower than those obtained by XRD, the observed trend is in line with the measurements already available on non-retted and retted flax or hemp fibres. These literature measurements were carried out using XRD on fibre bundles, which can make their interpretation more complex. Indeed, during the data processing of bundles, the signal corresponding to the amorphous constituents can be influenced by components outside the cell walls, in particular the residues of middle lamellae which may be present, especially for samples with a low retting level, as shown in Fig. 6. In the case of NMR analysis, this problem does not arise, as the processing of data from C4 celluloserelated signals restricts the scope of investigation solely to the cell wall components.

To supplement the crystallinity data obtained by NMR, the XRD patterns of R1 and R6 samples are presented in Fig. 6. As usual for vegetal samples, we observe that the major crystalline peak occurs at around 22.4°, which corresponds to the crystallographic plane (002) representative of type-1 cellulose. In the case of cellulose II conversion, new peaks attributed to ( 110) and (012) planes could be observed around 20° [START_REF] Ouajai | Composition, structure and thermal degradation of hemp cellulose after chemical treatments[END_REF], but are not present here. The crystallinity index (CI) is calculated according to the Segal method; although this method applies to bundles and not isolated fibres (Thygesen, Oddershede, Lilholt, Thomsen, & Sta, 2005), it is suitable for obtaining rapid measurements and therefore abundant published data are available for purposes of comparison. Estimated CI for R1 and R6 are 31% and 44%; thus, R1 exhibits a low CI compared to R6. As observed by NMR analysis, an increase of the CI is highlighted between R1 and R6 samples, but this latter is more pronounced by XRD analysis. This is probably due to the large amount of residual amorphous components remaining in the middle lamella areas and inducing an error into the calculation of CI, based on the ratio between crystalline and amorphous materials at the bundle scale. For this reason, the CI of R1 sample is probably underestimated. To check this hypothesis, it is possible to study the behaviour of the diffraction peaks at 2 θ = 14.8°a nd 2 θ = 16.4°, corresponding to the (11 ¯0 and (110) crystallographic planes, respectively. The literature shows that these two peaks are more separated when the crystalline cellulose content of the sample is high [START_REF] Tserki | A study of the effect of acetylation and propionylation surface treatments on natural fibres[END_REF]. On the other hand, if the samples contain a significant quantity of amorphous constituents, attributable for flax fibres to pectins, lignin, hemicelluloses or amorphous parts of cellulose, these two peaks appear confused and more difficult to distinguish. The smoothing of this specific area (Fig. 6) confirms that the content of amorphous components is higher in the R1 sample.

Investigation of cell wall biochemical structure using NMR

Table 2 gives the detailed characteristics of the different peaks obtained after deconvolution of the NMR spectra presented in Fig. 4. Two main observations can be drawn from these data; firstly, the inaccessible surface of cellulose decreases significantly with retting, decreasing from 35 to 31% and, at the same time, the fraction of paracrystalline cellulose increases from 32 to 37%. Paracrystalline cellulose is an intermediate form of cellulose, but which is considered as crystalline. It is noteworthy that, at the same time, the proportion of crystalline cellulose (Cr) remains unchanged at 20% and 19%, respectively, for R1 and R6. The same applies to the accessible surface, which remains constant (about 13%) for both levels of retting. These data indicate that the inaccessible fraction of cellulose is the main component impacted by retting, representing a component evolving towards paracrystalline cellulose

Owing to this data set, it is possible to determine morphological parameters related to the structure of cellulose and microfibrils. From the determination of crystallinity, we can estimate the LFD (Lateral Fibril Dimension) and the LFAD (Lateral Fibril Aggregate Dimension) according to the method of Zuckerstätter et al. [START_REF] Zuckerstätter | The elucidation of cellulose supramolecular structure by 13C CP-MAS NMR[END_REF]. The LFD of samples R1 and R6 are 4.1 and 4.5 nm, respectively, while the LFAD slightly decreases during retting from 17.1 nm for R1 to 16.3 nm for R6. Fig. 7 gives a schematic representation of the flax cellulose elementary microfibrils that reflects the differences in structure highlighted by the solid NMR spectra. The basic pattern chosen here with 16 fibrils is compatible with the relative scale of the constituents, the aggregate being approximately 4 times larger than the elementary fibril.

As mentioned above, we observe a slight decrease in LFAD corresponding to a densification in aggregate size, which is well correlated with the decrease in inaccessible surface area (IAS). This is probably caused by a decrease in porosity (as illustrated in Fig. 7) within cell wall polymers, and consequently a compaction of cellulose within the walls in which the cellulose microfibrils are located. The flax cell walls evolve during their maturation, in particular with a transition from the Gn-to the G-layer. This phenomenon is assumed to result in a compaction of the cell walls under the action of galactosidases, responsible for the cutting of the long galactan chains which are then easily entrapped between the cellulose fibrils [START_REF] Gorshkova | Aspen tension wood fibers contain β-(1→4)-galactans and acidic arabinogalactans retained by cellulose microfibrils in gelatinous walls[END_REF][START_REF] Goudenhooft | Investigation of the mechanical properties of flax cell walls during plant development: The relation between performance and cell wall structure[END_REF]. Even if this compaction occurs throughout plant growth, it is not excluded that maturation could be completed during the retting period. Thus, most of the cellulosic components in the R6 sample become truly isolated, being surrounded by densified layers of inaccessible surfaces (IAS). This does not apply in the case of degradation of surrounding layers, which could occur due to over retting [START_REF] Placet | The influence of unintended field retting on the physicochemical and mechanical properties of industrial hemp bast fibres[END_REF] or prolonged thermal exposure (Van de [START_REF] Van De Velde | Thermal and mechanical properties of flax fibres as potential composite reinforcement[END_REF]. The phenomenon of shrinkage described above can be linked to the increase of paracrystalline cellulose; in fact, under the effect of compression, a fraction of amorphous cellulose can evolve towards a paracrystalline form.

The last section of this NMR investigation involves using the twoproton reservoir model to study the evolution of the crystalline and amorphous C4 peaks areas as a function of the contact time. Fig. 8 shows the experimental data obtained for R1 and R6 samples along with model estimates. The associated relaxation time values are summarized in Table 3. We note higher values of T 1 for the R6 sample compared to R1. The increase of these values highlights a drop in the mobility which could be associated with a higher degree of molecular order. As shown on Fig. 8 and Table 3, the amorphous C4 yields lower values than crystalline forms and the increase in T H 1 with retting is reduced, being +56% instead of +80% for crystalline C4. These results indicate that the amorphous components are probably more affected by retting than crystalline components. This suggested decrease in crystalline component mobility is consistent with the hypothesis of compaction of the inaccessible parts of the cell walls. Zhang et al. [START_REF] Zhang | Natural polymer biocomposites produced from processing raw wood flour by severe shear deformation[END_REF] have already shown that the increase in Young's modulus may be related to an increase in T H 1 . This hypothesis, combined with the densification of the walls, allows us to explain the evolution of the mechanical properties of the cell walls as highlighted by AFM PF-QNM (Fig. 9).

Conclusion

This study aims to improve our knowledge of ultrastructural phenomena at the cell wall scale occurring during the field retting of flax. Thus, we make use of AFM PF-QNM, X-RD and NMR techniques to investigate the mechanical and structural modifications during retting.

Determination of the cell wall indentation modulus by AFM PF-QNM allows us to obtain both mechanical and morphological information; the results highlight a significant increase in the indentation modulus with retting time. Moreover, morphological analyses of the samples show a progressive increase in the distance between fibres associated with the disappearance of the middle lamellae, thus illustrating the progressive action of enzymes on the digestion of middle lamellae.

In addition, the degree of crystallinity is studied using both solidstate NMR and XRD. Indeed, these methods of analysis are quite different, as well as their scales of analysis. The global signal obtained on fibre bundles measured by XRD probably overestimates the differences in crystallinity between the retting states. Indeed, the calculation protocol is based on a comparison between amorphous and crystalline components, including the amorphous residues of middle lamellae whose abundance varies greatly between samples. Thus, the modest increase in crystallinity highlighted by NMR is probably more consistent with the physicochemical phenomena involved in retting at the cell wall scale. NMR results also show a significant evolution in the cell Fig. 9. Evolution of the crystalline and amorphous C4 peaks areas according to the contact time. Points showed correspond to experimental values, and curves have been obtained through the use of a two proton reservoir model [START_REF] Kolodziejski | Kinetics of cross-polarization in solid-state NMR: A guide for chemists[END_REF][START_REF] Paris | NMR local range investigations in amorphous starchy substrates: II-Dynamical heterogeneity probed by 1H/13C magnetization transfer and 2D WISE solid state NMR[END_REF].

wall ultrastructure of flax during the retting stages. The improvement in the mechanical performance of plant cell walls can be explained by the compaction of inaccessible zones, combined with an increase in contact time, as demonstrated by modelling with the two-reservoir method showing a decrease in mobility. The original coupling of NMR and XRD for the analysis of ultrastructure provides a significant contribution to the understanding of the mechanisms involved in retting and represents a further step in optimizing the performance of flax fi bres fo r us e as reinforcements for composite materials.

Fig. 1 .

 1 Fig. 1. AFM PF-QNM modulus mapping from R1 to R6 retting degree. The blue lines correspond to the direction and position where the thickness of the interfaces between fibres were measured (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article).

Fig. 2 .

 2 Fig. 2. Evolution of AFM and nanoindentation modulus of flax cell walls with the retting degree.

Fig. 3 .

 3 Fig. 3. Evolution of the distance (A) and indentation modulus (B) between scutched fibres along the retting process.

Fig. 4 .

 4 Fig. 4. Solid state 13 C CP/MAS NMR spectra for R1 and R6 samples.

Fig. 5 .

 5 Fig. 5. Deconvolution of C4 region with crystalline forms Cr (I α ), Cr (I β ) and Cr (I α+β ) (black), para-crystalline form (PCr) (grey), accessible fibril surfaces (AS) (green), and inaccessible fibril surface (IAS) (dark green) signals for R1 (a) and R6 (b) samples (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article).

Fig. 6 .

 6 Fig. 6. SEM images of R1, R3, R4 and R6 samples.

Fig. 7 .

 7 Fig. 7. X-Ray Diffraction patterns of R1 and R6 flax samples.

Fig. 8 .

 8 Fig. 8. Schematic representation of the flax cellulose elementary microfibrils structure. Cr: crystalline cellulose, PCr: paracrystalline cellulose, IAS: inaccessible surface, AS: accessible surface, LFD: lateral fibril dimension and LFAD: lateral fibril aggregate dimension. The surface areas (left part) and the number of points (zoomed views) in the scheme are proportional to the area of the corresponding NMR signals.

Table 1

 1 Statistical analysis with t-test. P values are indicated for each couple of samples. Samples are assumed to be statistically different when P < 0.05. Values in red indicate that samples are statistically similar.

		R1	R2	R3	R4	R5	R6
	R1	-	-	-	-	-	-
	R2	0.00006	-	-	-	-	-
	R3	0.00000	0.00025	-	-	-	-
	R4	0.00001	0.25199	0.03050	-	-	-
	R5	0.00000	0.00053	0.39222	0.01355	-	-
	R6	0.00000	0.00000	0.00000	0.00000	0.00000	-

Table 2

 2 Calculated parameters obtained from the C4 region deconvolution of 13 C CP/ MAS spectra for R1 and R6 samples. δ = chemical shift, FWHH = Full Width at Half Height.

	Peak assignment
	Cr(Iα) Cr(Iα+β) PCr	Cr(Iβ) AS	IAS	AS
	R1 89.37 88.73	88.58 87.83 84.22 83.53 83.16 δ (ppm)
	R6 89.38 88.70	88.54 87.68 84.14 83.68 83.16
	R1 81	60	199	90	98	482	86	FWHH (Hz)
	R6 76	55	197	90	88	484	76
	R1 8	10	32	2	6	35	7	Normalized
	R6 7	10	37	2	7	31		area (%)

Table 3

 3 Relaxation time values extracted from the two proton reservoir model (see text and Fig.8) for the crystalline and amorphous C4 peaks areas of R1 and R6 samples.

		T 1 (ms)		TCH (μs)		T HH (μs)
		R1	R6	R1	R6	R1	R6
	C 4cr	19.3	34.7	15	16	687	947
	C 4am	14.7	23.0	15	16	501	772

Acknowledgements

The authors are grateful to Oseo, Region Bretagne and CNRS for funding this work. The authors acknowledge the BIBS platform of INRA Angers-Nantes for access to NMR facilities and L ucile Nuez and Anthony Magueresse for SEM images.


Anastasiia Tsukanova
email: anastasiia.tsukanova@inria.fr

Benjamin Elie

Yves Laprie

Articulatory Speech Synthesis from Static Context-Aware Articulatory Targets

Keywords: articulatory synthesis, coarticulation, articulatory gestures

published or not. The documents may come   L'archive ouverte pluridisciplinaire

Introduction

Articulatory speech synthesis is a method of synthesizing speech by managing the vocal tract shape on the level of the speech organs, which is an advantage over the state-of-the-art methods that do not usually incorporate any articulatory information. The vocal tract can be modeled with geometric [START_REF] Öhman | Coarticulation in VCV utterances: Spectrographic measurements[END_REF][START_REF] Birkholz | Construction and control of a threedimensional vocal tract model[END_REF][START_REF] Story | Phrase-level speech simulation with an airway modulation model of speech production[END_REF], biomechanical [START_REF] Lloyd | Artisynth: a fast interactive biomechanical modeling toolkit combining multibody and finite element simulation[END_REF][START_REF] Anderson | A comprehensive 3D biomechanically-driven vocal tract model including inverse dynamics for speech research[END_REF] and statistical [START_REF] Maeda | Compensatory articulation during speech: Evidence from the analysis and synthesis of vocal-tract shapes using an articulatory model[END_REF][START_REF] Howard | Modeling the development of pronunciation in infant speech acquisition[END_REF] models. The advantage of statistical models is that they use very few parameters, speeding up the computation time. Their disadvantage is that they follow the data a priori without any guidance and do not have access to the knowledge of what is realistic or physically possible. Because of this, to produce correct configurations, they need to be finely tuned.

We were interested in exploring the potential in using quite little, and yet sufficient, static magnetic resonance imaging (MRI) data and implementing one of the few existing attempts at creating a full-fledged speech synthesizer that would be capable of reproducing the vast diversity of speech sounds.

Building an articulatory speech synthesis system

The system is basically made up of three components: the database with the "building blocks" for articulating utterances, the joint control algorithm for the vocal tract and the glottal source, and acoustic simulation. The primary concern of this work are the first two components.

Dataset

The dataset construction and manipulation were inspired by the work of [START_REF] Birkholz | Modeling consonant-vowel coarticulation for articulatory speech synthesis[END_REF]. We used static MRI data, 97 images collected with a GE Signa 3T machine with an 8-channel neurovascular coil array. The protocol consisted in a 3D volume acquisition of the vocal tract acquired with a custom modified Enhanced Fast Gradient Echo (EFGRE3D, TR 3.12 ms, TE 1.08 ms, matrix 256x256x76, with spatial resolution 1.02x1.02x1.0 mm 3 ). Then we selected the mid-sagittal slice in those images. These data captured articulation without phonation: the speaker was instructed to show the position that he would have to attain to produce a particular sound. For vowels, that is the position when the vowel would be at its clearest if the subject were phonating. For consonant-vowel (CV) syllables, that is the blocked configuration of the vocal tract, as if the subject were about to start pronouncing it. The assumption is that such articulation shows the anticipatory coarticulation effects of the vowel V on the consonant C preceding it. There were 13 vowels, 72 CV syllables and 2 semi-vowels in the final dataset. This covers all main phonemes of the French language, but not in all contexts. Each consonant was recorded in the context of the three cardinal vowels and /y/, which is strongly protruded in French. Some intermediate vocalic contexts were added so as to enable the vowel context expansion algorithm to be checked. Expanding the dataset Since the collected French phonemic dataset was limited, we needed to expand it to cover other contexts as well. We used the notion of the cardinal vowels-/a/, /i/, /u/ and /y/,-assuming that /a/, /i/, /u/ and /y/ represent the most extreme places of vowel articulation, and since then any other vowel articulation can be expanded as a combination of its /a/, /i/ /u/ and /y/ "components". Having captured the C+/a/, C+/i/, C+/u/ and C+/u/context for all consonants C and all non-cardinal vowels V on their own, we were able to estimate the missing C+V samples:

-We projected the vowel V articulatory vector (from R 29 ) onto the convex hull of the /a/, /i/, /u/ and /y/ vectors. -Assuming that the linear relationship between the C+V vector and the C+/a/, C+/i/, C+/u/ and C+/y/ vectors is the same as the one between V and /a/, /i/, /u/ and /y/, we estimated C+V from C+/a/, C+/i/, C+/u/ and C+/y/ using the coefficients from the projection of V onto the convex hull of /a/, /i/, /u/ and /y/.

We also estimated the neutral C configuration, the one without any anticipatory effects, as the average of C+/a/, C+/i/, C+/u/ and C+/y/.

Finally, we assumed that the voiced and unvoiced consonants did not have any differences in the articulation.

Articulatory model After manually annotating the captures as shown in Fig. 1 we applied a principal-component-analysis (PCA)-based model on the articulator contours [START_REF] Laprie | Construction and evaluation of an articulatory model of the vocal tract[END_REF][START_REF] Laprie | Geometric articulatory model adapted to the production of consonants[END_REF][START_REF] Laprie | 2D articulatory velum modeling applied to copy synthesis of sentences containing nasal phonemes[END_REF]. We paid special attention to the interaction between articulators and the relevance of deformation modes. Moreover, articulators other than the jaw, tongue and lips are often neglected and modeled with insufficient precision, whereas they can strongly influence acoustics at certain points in the vocal tract. Here are two examples. The position of the epiglottis, which is essentially a cartilage, is likely to modify the geometry of the lower part of the vocal tract by adding an artificial constriction disturbing all the acoustics. It is therefore important to model its deformation modes and interactions with other articulators correctly. In the same way, the velum plays an important role both in controlling the opening of the velopharyngeal port, and in slightly modifying the oral cavity to obtain resonant cavities that give the expected formants of vowels. The acoustic tests we have carried out show in particular that the velum makes it possible to better control the balance between the two cavities necessary for the realization of /u/ and /i/.

Regarding the tongue, PCA was applied on the contours delineated from images. Deformation modes are likely to be impacted by delineation errors. In the case of the tongue, these errors are marginal, or at least give rise to deformation modes coming after the genuine deformations whose amplitude is bigger. On the other hand, the width of epiglottis and/or velum is small on the images, and the errors of delineation, whether manual or automatic, are of the same order of magnitude as genuine deformations. Consequently, PCA applied without precaution will mix both types of deformation. To prevent the apparition of these spurious deformation components the epiglottis was approximated as a thick curve, and only the centerline of epiglottis was analyzed. As a matter of fact, the centerline was determined after delineation of all the epiglottis contours, and the width was set as the average width of all these contours in the upper part where the two epiglottis edges are clearly visible (see Fig. 3). The height of the upper part (where both contours are visible) is adjusted by hand to fit the contours extracted from images. The centerline is approximated as a B-spline and represented by its control points P l (0 ≤ l < M where M is the number of control points) in the form of a two-coordinate vector, and the reconstruction of the epiglottis from the centerline amounts to draw a line at a distance of half the width from the centerline.

The influence of delineation errors is very similar for the velum, which is a fairly fine structure not always well marked on MRI images because it moves quickly. As for epiglottis we used the centerline and a fairly simple reconstruction algorithm. However, PCA was not applied directly on the control point of the splines because the velum can roll up on itself. This particularity does not lend itself well to the direct use of PCA, which results in the emergence of linear components not appropriate in this case. The centerline is therefore broken down into a series of segments of the same length. Each segment articulates with its predecessor and the first point is fixed. The centerline is then defined as the vector of angles between two consecutive segments. In this way PCA can be applied effectively to velum and gives rise to relevant deformation modes.

The architecture and general organization of the articulatory model are based on the dependency links between the articulators. The main articulator is the jaw which is represented by 3 parameters to get a complete and accurate control. Its geometrical contribution is subtracted from the tongue contours before the application of PCA because tongue is directly attached to the mandible. The tongue is represented by 12 parameters in order to obtain a sufficient precision for the realisation of consonant constrictions. The lips are represented by 3 parameters. Unlike the tongue, the interactions between lips and jaw are more complex. For this reason we subtract the correlation between jaw and lips before applying PCA. The larynx is considered to be independent of the jaw and is represented by 3 parameters to control ist orientation and vertical position. In the same way the velum is considered as an articulator independent of the others. It is analyzed as explained above and is represented by 3 parameters. The epiglottis is the articulator that is subject to the greatest number of influences: the jaw via the tongue, the tongue itself and the larynx. These influences are subtracted by applying a multiple regression to the epiglottis centreline before applying CPA. Analysis of the variance shows that the various influences on the epiglottis account for most of its deformations. Its intrinsic deformation are represented by 3 parameters.

In total these parameters form a vector from R 27 (see Fig. 2 for major parameter contributions to the articulator shape). Since the model uses PCA, the zero configuration should correspond to the central position as identified in the dataset, and small changes in the parameter space within a certain neighborhood of zero should correspond to small changes (in terms of distance and shape, not in terms of the resulting acoustics) in the curves. A clipping algorithm is used to solve problems of collision between articulators, i. e. essentially between the tongue and palate. So the model's behavior is not entirely linear.

Strategies for transitioning between the articulatory targets

The dataset provided static images capturing idealistic, possibly over-articulated, targets for consonants anticipating particular vowels, whereas the goal was to be also able to deal with consonant clusters and consonants that would not anticipate any vowel at all-for example, due to their ultimate position in a rhythmic phrase. So, in our context, to establish a transitioning strategy would mean three things:

-Choose the building blocks: identify the articulatory target for each phoneme in a phrase. It can either be what was captured in the dataset (a vowel or a consonant assuming vocalic anticipation) or an estimation of what the dataset was missing (missing phonemes, such as voiced consonants, missing contexts or the absence of any context). A consonant cannot anticipate multiple phonemes, nor can vowels anticipate anything due to the restrictions of the dataset at our disposal. -Decide when -and whether -the articulatory target should be attained.

-Decide how to generate the articulatory positions between the target ones.

Our basic assumption was that by default, consonants anticipate the next coming vowel. However, it would be unrealistic to assume it happens in all cases. This is why we imposed several restrictions on the anticipatory effect:

-Temporal: no coarticulatory effect if the anticipated phoneme is more than 200 ms ahead; -Spatial: if there is any movement scheduled between the anticipated vowel, the phoneme in question negates the effect. For example, consider such sequence as /lki/: after /l/, the tongue needs to move backward to produce /k/ before coming back forward for /i/. In this situation, our algorithm does not allow the /l/ to anticipate the coming /i/; -Categorical: it is not possible to anticipate a vowel more than 5 phonemes ahead, and this restriction becomes stricter if it applies across syllable boundaries.

For vowels, there is also a model of target undershoot.

Having established the articulatory targets, the question is how to transition between them. We have tested out three strategies for interpolation between the target vectors:

-Linear: the interpolation between the target vectors is linear; -Cosine; -Complex: transitions are done with cosine interpolation, but the timing varies by the articulators. The critical ones reach for their target position faster than the others, while those articulators whose contribution to the resulting sound intelligibility is not as large move slower (for example, the tongue can be in a number of positions for the sound /b/, but the lips have to come into contact). Besides, the articulators composed of heavier tissues (such as the tongue back) move slower than the light and highly mobile ones (such as the lips).

Obtaining the sound

Each vocal tract position was encoded in an area function. They were obtained by the algorithm of [START_REF] Heinz | On the relations between lateral cineradiographs, area functions and acoustic spectra of speech[END_REF] with coefficients adapted by S. Maeda and Y. Laprie. These parameters only depend on the position in the vocal tract between the glottis and the lips. The transition from the sagittal view to the area function has given rise to several works which contradict each other slightly ( [START_REF] Soquet | Mid-sagittal cut to area function tranformations: Direct measurements of mid-sagittal distance and area with MRI[END_REF] and [START_REF] Mcgowan | Analyses of vocal tract cross-distance to area mapping: an investigation of a set of vowel images[END_REF]) and it is therefore clear that the determination of the area function will have to take into account the dynamic position of the articulators in the future.

We used an acoustic simulation system implemented by [START_REF] Elie | Extension of the single-matrix formulation of the vocal tract: Consideration of bilateral channels and connection of self-oscillating models of the vocal folds with a glottal chink[END_REF] to obtain sound from the area functions and supplementary control files: glottal opening and pitch control.

Glottal opening was modeled by using external lighting and sensing photoglottography (ePGG) measurements [START_REF] Honda | Glottal-opening and airflow pattern during production of voiceless fricatives: a new non-invasive instrumentation[END_REF]. Within the model, glottis is at its most closed position when producing vowels, nasals and the liquid sound /l/, and momentarily reaches its most open one when producing voiceless fricatives and stops. Voiced fricatives and stops also create peaks in glottal opening, but not as high.

There was no need to model voicing (high-frequency oscillations of low amplitude superimposed onto the glottal opening waves) since the vocal folds operated by the glottal chink model [START_REF] Elie | Extension of the single-matrix formulation of the vocal tract: Consideration of bilateral channels and connection of self-oscillating models of the vocal folds with a glottal chink[END_REF][START_REF] Elie | A glottal chink model for the synthesis of voiced fricatives[END_REF] are self-oscillating.

Evaluation

Each step in the system was evaluated on its own, and afterwards the synthesis results were evaluated visually, acoustically and perceptually. Since the objective of the work was rather to have a fully functional algorithm that produces reasonably realistic movements and sounds rather than to obtain high-quality speech, a more rigorous evaluation, such as a quantitative comparison to the dynamic data on articulatory trajectories, is still an avenue of future work.

The articulatory model and the trajectories

One peculiarity of the dataset and therefore of the model was the fact that it used only the sagittal section of the speaker's vocal tract. While full threedimensional models can capture the full geometry of the vocal tract with such phenomena as lateral phonemes (e.g. /l/), two-dimensional models get the benefit of faster computation time and overall simplicity, but irreversibly lose the spatial information.

In general, the articulatory model captured vocal tract positions correctly or with no critical errors, and some adjustments could be necessary only at the points of constriction, since on its own the model did not impose much control over them. This is a disadvantage brought by the nature of the articulatory model: choosing to operate at the level of articulators rather than the resulting vocal tract geometry.

As for the movements, for now we can say that they were reasonable and the coarticulation-affected targets guided the articulators to the positions necessary to produce a particular utterance. One key point here is the timing strategy. Rule-based timing strategy seems to be very rigid for the dynamic nature of speech; it would be more natural to follow speech production processes in humans and to guide the synthesis with the elicited sound or the speaker's expectationbased on their experience-on what this sound will be. We plan to evaluate the transitions with new dynamic MRI data.

Glottal opening control

The algorithm for the glottis opening successfully allowed to distinguish between vowels and consonants. Distinguishing between voiced and voiceless consonants, though, stays a point for improvement, as well as well-coordinated control over the glottis and the vocal tract.

The synthesized sound

Vowels and stops were the most identifiable and correct, although sometimes some minor adjustments in the original data were necessary to obtain formants close to the reference values. When compared to human speech, the formant transitions within the suggested strategies sometimes occurred too fast and sometimes too slowly; again, this highlights the utmost importance of realistic timing strategies. Fig. 4 shows an example of the synthesis when it is guided by real timing: /aSa/ as produced by the system and as uttered by a human. The highfrequency contributions in /S/, not appearing in the human sample, are due to the acoustic simulation. The noise of /S/ is at the correct frequencies, but with a bit different energy distribution, probably because of differences in articulation or in the area functions. There is also an acoustic artifact between /S/ and /a/, which means that more work is necessary on liaising the vocal tract and the source control.

Conclusion

Regarding speech as a process of transitioning between context-aware targets is an interesting approach that can be connected with the mental processes of speech production: to allow the others to perceive the necessary acoustic cue, the speaker needs to come close enough to the associated articulatory goal. The important difference between a real speaker and the algorithm is the fact that the algorithm solves a static problem, laid out in full; it needs to hit particular targets in a given order. As for humans, we solve a dynamic problem, and coarticulation is not something we put in its definition; rather, coarticulation is our means to make the problem of reaching too many targets in a too short period of time solvable.

The statistically derived articulatory model encodes complicated shapes of the articulators in only 29 parameters, sometimes struggling at the constrictions because of the inherent-and intentional-lack of control over the resulting geometry of the vocal tract.

Those shapes of the articulators change in time according to the produced trajectories of the vocal tract, and those are phonetically sound. Whether there are any important differences between the produced transitions and the ones in real speech, needs to be verified with actual dynamic data.

After the aspect of how the articulators move we need to consider when. The timing strategies, currently rule-based, apparently need to be extracted from dynamic data, and we can use the approaches by [START_REF] Elie | High spatiotemporal cineMRI films using compressed sensing for acquiring articulatory data[END_REF] for that.

A closer, intertwined interaction with the acoustic simulation unit-such as guidance on how to navigate between the area functions at the level of separate

Fig. 1 .

 1 Fig. 1. An example of dataset image annotation (/a/).

Fig. 2 .

 2 Fig. 2. The PCA-based articulatory model: curve change directions encoded in the first three factors of each articulator (the jaw, the tongue, the lips, the epiglottis, the larynx).

Fig. 3 .

 3 Fig. 3. Epiglottis and velum centerlines reconstructed by the model (solid blue lines). Reconstructed vocal tract is represented by solid red lines. The vocal tract input and output are represented in solid green lines. All these contours are superimposed onto the contours (represented as solid yellow lines) delineated from the image.

Fig. 4 .

 4 Fig.4. An example of a human's utterance of /aSa/ (left) and its synthesis (right) along with the glottal closure control as copied from the EPGG data (below). Phoneme durations are aligned.

acoustic tubes and improved control over the glottal opening-could improve the results for consonants.

Acknowledgments

The data collection for this work benefited from the support of the project ArtSpeech of ANR (Agence Nationale de la Recherche), France.