domain: scco shs info

text:
Clémence Bayard

Laura Machart

Antje Strauss

Silvain Gerber

Vincent Aubanel

Jean-Luc Schwartz
email: jean-luc.schwartz@gipsa-lab.grenoble-inp.fr

Antje Strauß

Cued Speech Enhances Speech-in-Noise Perception

come   L'archive ouverte pluridisciplinaire

Auditory speech perception for deaf or hard-of-hearing persons

In recent years, a large number of deaf people (and in particular many congenitally deaf children) are fitted with a hearing aid (HA) or a cochlear implant (CI). With technological progress, CIs have become the most effective vehicle for helping profoundly deaf people to understand spoken language, to perceive environmental sounds, and, to some extent, to listen to music. The development of the early detection of deafness together with the trend for more and earlier implantation might decrease or minimize the interest for visual cues in oral communication. HAs and CIs immensely help deaf people by providing auditory access to speech. Yet, the auditory input they deliver remains degraded compared to the full auditory signal [START_REF] Shannon | Speech perception with cochlear implants[END_REF][START_REF] Percy | Measuring speech perception abilities in adults with cochlear implants: Comprehension versus speech recognition[END_REF][START_REF] Wolfe | Evaluation of speech recognition of cochlear implant recipients using adaptive, digital remote microphone technology and a speech enhancement sound processing algorithm[END_REF][START_REF] Todorov | Benefits of upgrading to the nucleus® 6 sound processor for a wider clinical population[END_REF]. This is particularly the case for speech perception in noise, which remains a really difficult task for deaf people [START_REF] Revoile | Spectral cues to perception of/d, n, l/ by normal-and impaired-hearing listeners[END_REF][START_REF] Zeng | Amplitude mapping and phoneme recognition in cochlear implant listeners[END_REF][START_REF] Caldwell | Speech perception in noise by children with cochlear implants[END_REF]Srinivasana, Padilla, Shannon, & Landsberge., 2013). HAs and CIs provide inaccurate representations of phonemically relevant spectral structure making the perceptual segregation of that spectral structure from background noise difficult (Baer, Moore, & Gatehouse, 1993;[START_REF] Boothroyd | Effects of spectral smearing on phoneme and word recognition[END_REF][START_REF] Fu | Effects of noise and spectral resolution on vowel and consonant recognition: Acoustic and electric hearing[END_REF][START_REF] Bernstein | Effects of spectral smearing and temporal fine-structure distortion on the fluctuating-masker benefit for speech at a fixed signal-to-noise ratio[END_REF]. [START_REF] Friesen | Speech recognition in noise as a function of the number of spectral channels: Comparison of acoustic hearing and cochlear implants[END_REF] showed that CI users who displayed incremental benefit beyond 4 channels tended to do better in noise than CI users who only showed growth with increase up to 4 channels. As a consequence, they estimate that CI users require a minimum of 8-10 independent spectral channels to perceive speech in noise. In comparison, understanding speech in quiet requires only 4 spectral channels (Shannon, Fu, & Galvin, 2004). Since CIs have typically between 12 and 22 physical electrodes, this might appear sufficient for processing speech in noise, but it appears that CI listeners perform as if information were provided by only 4 to 8 independent information channels [START_REF] Friesen | Speech recognition in noise as a function of the number of spectral channels: Comparison of acoustic hearing and cochlear implants[END_REF][START_REF] Strauß | Narrowed expectancies under degraded speech: revisiting the N400[END_REF]. The lack of fine temporal structure also contributes to the difficulty to segregate a target signal from background noise [START_REF] Lorenzi | Speech perception problems of the hearing impaired reflect inability to use temporal fine structure[END_REF].

The role and potential limitations of lipreading

Understanding speech in noise is of course crucial considering that noisy environments are more likely to occur than clean ones in most situations in real life. Therefore, visual information provided by lipreading remains paramount for Deaf/Hard of Hearing people (D/HH in the following), be they fitted with HA or CI. Starting with pioneering studies done by Norman Erber or others in the 1960s to early 1970's [START_REF] Erber | Auditory-visual perception of speech[END_REF], it is now well known that D/HH listeners spontaneously process lip-reading information to compensate for their auditory deficit [START_REF] Lachs | Use of audiovisual information in speech perception by prelingually deaf children with cochlear implants: A first report[END_REF]Bergeson, Pisoni, & Davis, 2005). As a matter of fact, in spite of a large inter-individual variability, D/HH persons happen to be among the best lip-readers (Bernstein, Demorest, & Tucker, 1998[START_REF] Duchnowski | Development of speechreading supplements based on automatic speech recognition[END_REF]. They can display significant audiovisual fusion after cochlear implantation [START_REF] Schorr | Auditory-visual fusion in speech perception in children with cochlear implants[END_REF][START_REF] Rouger | Evidence that cochlear-implanted deaf patients are better multisensory integrators[END_REF] and in some cases may even be better audiovisual integrators than individuals with typical hearing [START_REF] Rouger | Evidence that cochlear-implanted deaf patients are better multisensory integrators[END_REF].

Still, while speech-reading can help mitigate missing auditory information to some extent, speechreading information alone is likely insufficiently rich to overcome the severe degradation posed by noise to CI deaf participants. Surprisingly, this has actually seldom been tested. As a matter of fact, a number of studies have been done on audiovisual speech perception in noise in unaided D/HH participants (e.g. [START_REF] Erber | Auditory and audiovisual reception of words in low-frequency noise by children with normal hearing and by children with impaired hearing[END_REF]Grant, Walden, & Seitz,1998;[START_REF] Bernstein | Auditory and auditory-visual intelligibility of speech in fluctuating maskers for normal-hearing and hearing-impaired listeners[END_REF]; or on auditoryonly speech perception in noise in aided or cochlear-implanted D/HH participants (e.g. [START_REF] Caldwell | Speech perception in noise by children with cochlear implants[END_REF][START_REF] Tabanez Do Nascimento | Evaluation of speech perception in noise in cochlear implanted adults[END_REF]; or on audiovisual speech perception without noise in aided or cochlear-implanted D/HH participants (e.g. [START_REF] Holt | Assessing multi-modal spoken word-insentence recognition in children with normal hearing and children with cochlear implants[END_REF][START_REF] Liu | Audiovisual Speech Perception at Various Presentation Levels in Mandarin-Speaking Adults with Cochlear Implants[END_REF]. But there are only a small number of studies reporting audiovisual speech perception in noise in aided or cochlear-implanted D/HH participants. [START_REF] Leybaert | Cued speech for enhancing speech perception and first language development of children with cochlear implants[END_REF] report inaccurate audiovisual perception of vowel-consonant-vowel (VCV) sequences embedded in acoustic noise by deaf CI children. [START_REF] Taitelbaum-Swead | Audio-visual speech perception in noise: Implanted children and young adults versus normal hearing peers[END_REF] obtain lower speech perception accuracy for monosyllabic words in white noise at a signal-to-noise ratio (SNR) of 0 dB for CI children and adults as compared to typically hearing participants of the same age, for both auditory and audiovisual presentations.

Cued Speech, a potential crucial complement for speech communication in noise

In this context, it is essential to provide D/HH persons with additional support for communication, particularly in adverse conditions. It has been proposed 50 years ago that the addition of manual cues from the Cued Speech (CS) system could help D/HH individuals to overcome the uncertainty of auditory signals delivered by the CI or HA. Originally, this system was designed to help deaf individuals (without CI) to perceive speech through disambiguating the visual modality [START_REF] Cornett | Cued speech[END_REF]).

The CS system resolves the ambiguity in lip-reading by making each of the phonological contrasts of oral language visible. Each syllable is uttered with a complementary gesture called a manual cue. In its French version, vowels are coded with five different hand placements in relation to the face, and consonants are coded with eight handshapes (see Figure 1). Each manual cue can code several phonemes, but these phonemes differ in their labial visual pattern. Moreover, consonants and vowels sharing the same labial visual pattern are coded by different cues. Therefore, both sources of information (hand and lips) provide complementary information. Cued Speech has been shown to enhance the development of speech perception and language processing in CI children [START_REF] Leybaert | Cued speech for enhancing speech perception and first language development of children with cochlear implants[END_REF] and to constitute, in addition to audition and lip-reading, a common amodal network for language processing in the brain [START_REF] Aparicio | The neural basis of speech perception through lipreading and manual cues: evidence from deaf native users of cued speech[END_REF]. When communicating with CS, a person talks while simultaneously cueing. The combination of visual information, provided by the articulatory labial movements and manual cues, allows deaf individuals to perceive all syllables efficiently [START_REF] Clark | The effect of using cued speech : A follow-up study[END_REF][START_REF] Nicholls | Cued Speech and the reception of spoken language[END_REF][START_REF] Gregory | An investigation of speechreading with and without Cued Speech[END_REF][START_REF] Périer | Evaluation of the effect of prolonged Cued Speech practice upon the reception of spoken language[END_REF][START_REF] Uchanski | Automatic speech recognition to aid the hearing impaired: prospects for the automatic generation of cued speech[END_REF][START_REF] Bratakos | Toward the automatic generation of Cued Speech[END_REF][START_REF] Alegria | Phonological processing in deaf children : when lipreading and cues are incongruent[END_REF]. For example, [START_REF] Nicholls | Cued Speech and the reception of spoken language[END_REF] tested the identification of syllables and keywords within sentences, by 18 deaf children from 9 to 17, with at least 4 years of experience with CS. Syllables comprised all combinations of CV and VC syllables with C one of the 24 consonants and V one of the three /i a u/ vowels in American English. They obtained syllable recognition scores as high as 80% when CS was available, and less than 40% when it was not. Speech reception was higher than 95% for keywords.

Moreover, the CS system enables the perceiver to focus attention in time. Indeed, whereas Cornett (1994) described the CS system as a time-locked system characterized by a synchrony between manual cues and sounds, [START_REF] Attina | A pilot study of temporal organization in Cued Speech production of French syllables: rules for a Cued Speech synthesizer[END_REF] and [START_REF] Attina | Temporal measures of hand and speech coordination during french cued speech production[END_REF] found that the sounds and hands were not synchronous, as it is also the case for sounds and lips in audiovisual speech [START_REF] Schwartz | No, there is no 150 ms lead of visual speech on auditory speech, but a range of audiovisual asynchronies varying from small audio lead to large audio lag[END_REF]. Manual cues naturally precede sound, since the hand reaches its target position and shape up before the vowel target in consonant-vowel syllables (with an advance estimated on French CS speakers to 200 ms by [START_REF] Attina | A pilot study of temporal organization in Cued Speech production of French syllables: rules for a Cued Speech synthesizer[END_REF][START_REF] Gibert | Analysis and synthesis of the three-dimensional movements of the head, face, and hand of a speaker using cued speech[END_REF]; and with a smaller advance of 100 ms used by [START_REF] Duchnowski | Development of speechreading supplements based on automatic speech recognition[END_REF], in their automatic system for American CS synthesis). Interestingly there is still a rather precise temporal coordination between speech and manual cues, but it is in advance of the sound. Indeed, [START_REF] Attina | A pilot study of temporal organization in Cued Speech production of French syllables: rules for a Cued Speech synthesizer[END_REF] observed that the hand reaches its position for a given consonant-vowel syllable precisely at the temporal position of the consonant constriction. Deaf individuals were shown to take advantage of the advance of manual cues relative to lip-reading cues during CS perception [START_REF] Attina | La Langue française Parlée Complétée: production et perception[END_REF][START_REF] Troille | A perceptual desynchronization study of manual and facial information in French Cued Speech[END_REF].

Strikingly, most studies assessing the role of CS were realized in a pure visual environment without sound. The combination of sound, lips and manual cues was only recently explored by Bayard, Colin, & Leybaert (2014) and Bayard, Leybaert, & Colin (2015) who examined syllable perception by CI participants in a paradigm including various cases of congruent or incongruent combinations of auditory and visual speech stimuli. The results showed that, in quiet conditions, CS receivers do combine sound, lip shapes and manual cues into a unitary percept. Still, no study attempted, to our knowledge, to assess the potential benefit provided by CS to improve speech perception in noisy conditions. A D/HH person communicating in a noisy environment and receiving CS information from a partner mastering this system has to solve a complex processing-and-fusion problem. Indeed, adequate reception involves (i) efficient processing of the auditory input degraded by noise, exploiting the benefit of the cochlear implant or hearing aid, and (2) fusion of three sources of information that are sound, lips and hands. Yet it is not known at this stage how D/HH participants can deal with this complex task, the more so in environments where the structure of information may change from time to time (e.g. communicating with partners who either master and use CS or who don't, hence possibly switching between different kinds of fusion situations in the course of communication).

The objective of the present study is to evaluate this capacity in more detail. For this aim, we assessed the comprehension of sentences in noise by a group of D/HH CS users compared to a group of typically hearing (TH) controls, in three conditions: audio only, audiovisual and audiovisual with CS. It is well known that D/HH participants are very heterogeneous concerning their auditory abilities. For this reason, and to facilitate further comparison between groups, we customized the SNR for each participant, ensuring that their level of correct comprehension was about 60% in the audiovisual condition.

With this study, we aimed to answer two basic questions. First, we wanted to evaluate whether there was indeed a difference in the SNRs enabling to achieve similar levels of sentence comprehension across D/HH and TH participants in the audiovisual condition. In fact, there are almost no data in the literature assessing the reception of audiovisual speech in noise in HA or CI D/HH persons with complete sentences, though this is actually a crucial task for assessing their comprehension capacities for speech communication. Second, we wanted to investigate whether CS does provide a gain in the perception of audiovisual speech in noise for D/HH CS users, just as it does when there is no auditory input at all. Particularly, we wanted to check their ability to efficiently integrate the three sources of information (noisy sound, lips and hands), in a paradigm mixing conditions in a single block, imposing the participants to permanently monitor their attention and modulate the fusion process accordingly.

Material and Methods

Participants

The recruitment of D/HH participants fitted with a cochlear implant or a hearing aid and mastering CS is rather complicated and slow. The recruitment of D/HH participants mastering CS in this study was considerably facilitated by the opportunity provided by a week of CS training organized by the French ALPC association. This association supports the use of the French CS version for communication between hearing persons and persons with an auditory handicap (http://alpc.asso.fr/). Twenty D/HH teenagers and adults (nine female; age range 12-21 years, mean = 15.5; see Table 1 for more details) participated in the study. Nineteen participants had a profound deafness and the remaining one a severe deafness. Seventeen of them were cochlear implanted (six bilaterally), the remaining three being fitted with HA.

All participants had been using CS receptively ("decoding") on average since 4.5 years of age, and they learned to cue ("cueing") on average since the age of 5.5 years. The information was provided by the participants, and hence corresponded to a rough self-estimation of these behavioural abilities.

Notice that while in most cases decoding was used before coding, the order could be different in rare cases, particularly for participant 14, probably because this participant had a deaf brother, and a hearing impairment which increased with age in his first years of age. They all communicated with their environment orally, being able to both understand speech from the sound and sight of their interlocutor, and pronounce intelligible speech that their interlocutor could understand. They were integrated in a family and school environment mostly comprised of typically hearing family members or school colleagues with whom they communicated orally without CS. They frequently used CS for decoding language (understanding), typically with parents or school assistants providing on-line CS in school, and particularly in noisy environments. Expressively, they cued while speaking much less frequently.

In one case, the participant cued expressively only in the course of the training sessions organized by ALPC.

To evaluate the speech perception performance of this group of adolescent-to-young-adult D/HH participants, we compared their performance with a reference group of TH adult participants. We could have used as a control a TH group matched in age, but we preferred using TH adults with a completely mature auditory/cognitive system, to provide an optimal baseline enabling to better evaluate the deficit in perception for the group of D/HH participants in adverse conditions. Typically hearing participants were recruited by announcements in the *** and on a national website (http://expesciences.risc.cnrs.fr/pre_formulaire.php), and tested at ***. Fifteen self-reported TH adults (nine female; age range 22-36 years, mean = 29.5; see Table 2 for more details) participated in the study.

Written informed consent was obtained from each TH and D/HH participant together with parental authorization for minors. The experiment was performed in accordance with the ethical standards laid down in the 1964 Declaration of Helsinki and was validated by the *** Ethics Board (***). All participants were French native speakers with normal or corrected-to-normal vision and did not have any declared language or cognitive disorder.

Insert Tables 1,2 here

Stimuli

Sentence material was selected from the Fharvard corpus (Aubanel, Bayard, Strauß, & Schwartz, submitted), a French equivalent of the Harvard sentences [START_REF] Rothauser | IEEE Recommended practice for speech quality measurements[END_REF], which have been used extensively in speech perception research (e.g. [START_REF] Bradlow | Intelligibility of normal speech I: Global and fine-grained acoustic-phonetic talker characteristics[END_REF][START_REF] Cooke | Evaluating the intelligibility benefit of speech modifications in known noise conditions[END_REF]. The corpus consists of 70 phonemically-balanced lists of 10 sentences, where each sentence contains five keywords used for scoring. Each keyword contains one or two syllables and is relatively poorly predictable from context (e.g. "Elle attend le taxi sur la pelouse devant l'hôtel" [engl. "She is waiting for the taxi on the grass in front of the hotel"]; keywords in italics). For this experiment, a subset of 42 different sentences from the Fharvard corpus was presented.

The 42 sentences were spoken by a professional cuer (female, 34 years old, highly experienced in CS production, with a French diploma, "CS professional degree" followed by 8 years of professional practice, enabling her to assist teachers by CS production in a school for D/HH children). Each sentence was recorded two consecutive times: the speaker first produced the sentence with manual cues (AVC) and then without cueing (AV). All productions were checked for accuracy by another professional CS cuer, and some productions were removed when a gesture was inaccurate or seemed ambiguous.

It is well known that there is a trend that speech rate is slower with CS than without CS [START_REF] Attina | A pilot study of temporal organization in Cued Speech production of French syllables: rules for a Cued Speech synthesizer[END_REF]. To be able to compare speech perception in noise with and without CS, we had to control for this difference. If production was slower, it might increase intelligibility in the CS condition. Therefore, we asked the cuer to maintain a stable production rate during recording as much as possible.

Afterwards, we systematically evaluated the duration of recorded sentences, and selected 28 out of 42 sentences, in which both recordings with and without CS production had similar speech rates. The mean duration of selected sentences was 3597 ms for AVC and 3551 ms for AV (3.76 vs. 3.81 syllables/s). The mean difference of 45 ms between AVC and AV productions was not significant (t = 1.65, p > .05). The remaining 14 sentences (pronounced either with or without CS) were used to prepare the Audio-only condition by extracting audio files and dubbing them on a video of the cuer with a neutral face (see Figure 2). Video editing was realized with the Adobe Premiere software. Each participant was presented with all 42 sentences in noise: 14 Audio only (A), 14 Audiovisual (AV), and 14 Audiovisual with Cued Speech (AVC). To control for item effects, we created two orders counterbalancing sentences between conditions AV and AVC (see Table 3). Half of the participants were presented with sentences in order 1 and the other half with sentences in order 2. For example, sentences presented in the AV condition in order 1 were presented in the AVC condition in order 2, and vice-versa. In order 1, half of the sentences in the A condition contained extracted sounds from the AV production and the other half from the AVC production, and conversely in order 2. All 42 stimuli were presented randomly to each participant in one block, hence A, AV and AVC conditions were mixed in an unpredictable order.

Insert Table 3 To generate speech-shaped noise, white noise was filtered by the long-term average speech spectrum taken from the whole recorded corpus. Each sentence was embedded in noise that started 500 ms before and ended 500 ms after the sentence. 24 different signal-to-noise-ratio values (SNR) from +6 dB to -17 dB in 1 dB steps were prepared to enable the selection of the adequate SNR for each listener in a pre-test procedure (see "Procedure").

Procedure

The experiment was conducted in a quiet room with the Presentation ® software (www.neurobs.com).

Instructions and videos were displayed on a laptop at eye level and approximately 70 cm from the participant's head. Sound was played with loudspeakers (Logitech) at a comfortable level around 70 dB SPL (similar for all participants). The D/HH participants used the standard setting of their cochlear implant or hearing aid, with no quantitative evaluation of this setting.

Participants completed pre-testing prior to the main test battery. In the pre-test, the SNR for the main experiment was determined for each participant individually. Mimicking an adaptive tracking procedure, groups of three sentences not used in the main experiment were presented first at a high SNR in the AV condition and participants had to repeat what they had heard. The experimenter calculated the percentage of correct responses and decreased the SNR by 1 dB step for the next sentence group until the number of correctly recognized words fell below 10. A recognition score of 60 % corresponds to 9 out of 15 correctly recognized keywords. The final SNR value for each participant was selected when the participant correctly recognized between 9 and 10 words twice.

In the main experiment, participants were asked to listen to and look at the video and, after each sentence, to repeat aloud what they had heard. The experimenter wrote on a sheet the number of keywords correctly recognized from 0 to 5. The experimenter was facing the participant and could not see the screen. Hence, since all conditions were randomized in a single block, the experimenter did not know the condition being tested while scoring the corresponding comprehension score. The experimenters were trained to the possible speech disfluencies in the production of the D/HH participants, hence the scoring was straightforward for these participants as well as for the TH ones.

The total duration of the experiment, pre-test and main experiment included, was approximately 25 minutes.

Statistical analyses

A first question asked in this study concerned differences in SNR values in the pre-test for TH vs. D/HH participants. The assumption was that SNR values associated to AV speech reception around 60% would be lower for TH than for D/HH participants. Because of the large inter-individual variability in the D/HH group, classical in all studies assessing comprehension performances, it appears that these values were not distributed according to a Gaussian law. Therefore, the difference in SNR values between the two groups was evaluated by a Wilcoxon sign-rank test.

A second set of question concerned differences in speech reception between conditions (A, AV, AVC) and between groups (TH, D/HH). We expected lower reception scores in the A than in the AV condition for both groups, and higher scores in the AVC condition for the D/HH group, and possibly also, to a lesser extent, for the TH group. Indeed, considering that TH participants have no experience with Cued Speech, there might appear in the experiment some learning processes according to which the reception scores would increase along the experiment for TH participants in the AVC condition. Moreover, learning effects could also appear for both groups and all conditions just in relation with task learning.

Hence, in addition to the variables GROUP (TH, D/HH) and CONDITION (A, AV, AVC) we added a variable TRIAL with 14 values corresponding to the number of the trial (from 1 to 14) in the test in a given condition. The dependent variable was the number of correct keywords from 0 to 5, considered as a categorical ordered variable with 6 levels. Participants were considered as a random factor, in a mixed design with GROUP as a between-subject factor and CONDITION and TRIAL as withinsubject factors. The effects of these four factors (PARTICIPANT, GROUP, CONDITION, TRIAL) were assessed by an ordinal regression with random effects [START_REF] Tutz | Random effects in ordinal regression models[END_REF], by using the clmm ordinal package in the R (version 3.2.0) software (R Development Core Team, 2016).

Selection of the appropriate model was based on log-likelihood differences between models, assessed with a Chi-square test with a degree of freedom equal to the difference in the number of parameters, and with the criterion of p-value lower than 0.05. The analysis of reception scores was done in two steps, first for selecting random effects and then for selecting fixed effects. In the first step we tested the need for a PARTICIPANT-CONDITION or a PARTIPANT-GROUP random effect, assessing whether individual variability differed when the participants moved from one modality to another of the CONDITION factor or between TH and D/HH participants. Then we studied the structure of fixed effects by a descendant analysis with the anova function in R. At the end of this process, we checked by graphical inspection of residuals if the model adjusted the data correctly, i.e., if the empirical probabilities were close to the probabilities estimated by the model and if they were within the range of 95% prediction. Finally, multiple comparisons were achieved taking into account that we use an ordinal regression model with random effects. They were realized using the lsmeans function of the lsmeans package of the R software. This method ensures that the risk of type I error does not exceed 0.05.

Results

Differences in AV perception between the two groups

On Fig. 3, we present the individual values of SNR and AV scores for all D/HH and TH participants (see also individual values per participant in Tables 1 and2). As expected, the mean AV score is around 60% for both populations, although the dispersion of scores is actually rather large. Indeed, performance may be rather variable from one sentence to another, and it appears that the initialization phase was insufficient for a perfect estimation of the SNR ensuring a 60% performance for each participant. Still, most participants have their performance in the 40-80% range (with only one TH participant and 3 D/HH participants having scores below 40%). Linear regression lines relating SNR and AV score in each group (in dotted lines on Fig. 3) show that the mean AV score is rather stable, around 55%, along SNRs for both D/HH and TH participants.

The mean SNR for TH controls was -11.2 dB, while the mean SNR for D/HH participants was -0.1 dB (11 dB higher). In more detail, the range of SNRs for TH participants varied from -14 to -9 dB (Table 2). It varied for D/HH participants from -11 to 6 dB with much larger variability, and apart from one single participant with an SNR at -11 dB (that is within the range of values for the TH group) all SNR values were higher than -8 dB for the 19 remaining D/HH participants. The difference between SNR values in the TH vs. D/HH group is highly significant (Wilcoxon sign-rank test, Q=292.5, p=2.10 - 6 ).

Differences in perception between conditions and groups

For each participant and each condition, the percentage of correctly recognized words was calculated.

The mean scores for the two groups and the three conditions are displayed in Figure 4. They were submitted to an ordinal regression test with three fixed factors GROUP, CONDITION, TRIAL and a random factor PARTICIPANT according to the sequence of statistical analyses described previously.

The analysis of random effects displays a significant PARTICIPANT-CONDITION interaction showing that inter-individual variability differs in the three conditions. The corresponding values show that variability is larger in the A condition, intermediate in the AVC condition and smaller in the AV condition, with no significant difference in variability between groups.

The analysis of fixed factors hence incorporated this PARTICIPANT-CONDITION interaction. It displayed no effect of TRIAL alone or in interaction. Hence there is no significant learning effect in the results. The selected model included a highly significant GROUP-CONDITION interaction (c (2)=30.59, p<0.0001).

The multiple comparison analysis confirmed that labial information significantly increased comprehension for both D/HH and TH participants: scores in the A condition are lower than those in the AV condition (15% vs. 55% for D/HH; z_ratio = 9.04; p<0.0001; 13% vs. 53% for TH; z_ratio = 8.06; p<0.0001). Furthermore, the difference between A and AVC was significant in both groups (TH: z_ratio = 5.95; p<0.0001; D/HH: z_ratio = 11.34; p<0.0001). As expected, adding manual information improved scores only for D/HH CS users, for which the performance increased from 55% in the AV condition to 83% in the AVC condition (z_ratio = 9.23; p<0.0001). By contrast, the typically hearing controls showed similar performance in the AV and the AVC condition (53% vs. 52%; z_ratio = 0.40; p = 0.92).

The comparison between groups confirmed that both groups had similar scores in the AV conditions (D/HH: 55% vs. TH: 53%; z_ratio = 0.20; p = 1), close to the targeted 60% value. The scores are also similar across groups in the A condition (D/HH: 15% vs. TH: 13%; z_ratio = 0.08; p = 1), while the scores in the AVC condition are indeed better for D/HH CS users than for TH participants (D/HH: 83% vs. TH: 52%; z_ratio = 5.33; p < 0.0001).

Inter-individual variability in the benefit provided by the visual sources in the D/HH group

In a last stage of analysis, we attempted to explore in more detail the correlates of differences between participants in the D/HH group, known for being highly inhomogeneous. To this aim, we used SNR as a proxy for characterizing the speech reception abilities of the D/HH participants, and assessed the relation of this proxy with performance in the three perceptual conditions. We observe in Figure 5 that there is actually an interesting portrait here. While perception scores in the AV condition are quite stable among SNR, which is not surprising since they were precisely controlled for being more or less constant around 60%, there is a trend that A perception increases and AVC perception decreases with increasing SNR. This suggests that the participants with most perception difficulties (highest SNR) reach a 60% score in the AV condition with already a rather high A score, hence benefit less of the visual modality for this performance. Moreover, they also seem to benefit less of the manual input in the AVC condition. This is confirmed by a statistical test of the relation between SNR (considered to characterize the level of A recovery provided by the cochlear implant or hearing aid) and the gain provided by the two visual sources in the experiment (provided by lips and hands). Normalized gains were used in this analysis, that is the ratio (AVC -A)/(100 -A), with data in percentage, for evaluating the gain AVC vs. A, and the ratio (AVC -AV)/(100 -AV), with data in percentage, for evaluating the gain AVC vs. AV. Pearson correlations are significant for the gain AVC vs. A, evaluating the global benefit provided by the two sources (r 2 = 0.223, t(18) = 2.269, p = 0.036) and for the gain AVC vs. AV, evaluating the specific benefit provided by CS (r 2 = 0.226, t(18) = 2.289, p = 0.034).

It is important to notice that three participants in the D/HH group appear to reach rather low perception scores in the AVC condition, around 50% or even lower while it is higher that 70% for all other participants (see Fig. 5). This is not related to specificities of these participants in terms of age, or age of deafness diagnosis, coding or decoding (see data on these participants, number 2, 5 and 17, on Table 1). More detailed information on these participants provided by the experimenter establishes that two of these participants (2 and 17) are actually rather poor decoders, using decoding only for restricted tasks (participant 2 decodes seldom, and participant 17 decodes only for acquiring novel words). For the third participant (number 5) the performance is more surprising, since this participant does decode regularly, and it is likely that the possibly stressing situation provided by the experimental framework is responsible for the low performance. If we remove these 3 participants from the analysis of the relationship between SNR and CS gain, the Pearson coefficients actually increase (for AVC vs. A, r 2 = 0.336, t(15) = 2.740, p = 0.015); for AVC vs. AV, r 2 = 0.359, t(15) = 2.901, p = 0.011).

Discussion

The present study assessed speech perception in noise with visual information including both lipreading and CS. It provides two main results. Firstly, in the absence of manual cues, there is a large gap in reception of speech in noise between TH and D/HH participants, even with the addition of lipreading. Secondly, CS significantly enhanced speech perception in noise for D/HH CS users. We will analyze these two points in more detail before discussing some clinical implications.

Differences in perception between typically hearing participants and participants with hearing loss fitted with CI or HA

The mean SNR providing around 60% reception in the AV condition, and around 15% in the A condition, is 11 dB higher in the D/HH group than in the TH group (Fig. 3). Since there is actually a large variability in all aspects of this pattern, let us describe it in more detail. The SNR values provide a 40-to-80% reception score in the AV condition for most (14 over 15) TH speakers and are distributed between -14 and -9 dB, while for the same AV score range (for 16 over 20 in the group) D/HH participants display an SNR between -11 and +6 dB. The SNR is actually above 0 dB for 10 D/HH participants (half the group). Finally, for 7 among the 20 D/HH participants (a third of the group) AV scores are actually lower than 60% for these SNR values above 0 dB, which makes intelligibility of speech at these SNRs quite poor.

Coming back to mean values, the average score in the A condition for the D/HH group is at 15% for an average SNR at 0dB. This is in line with previous studies. Indeed, [START_REF] Fu | Effects of noise and spectral resolution on vowel and consonant recognition: Acoustic and electric hearing[END_REF] conducted a series of experiments on the auditory recognition of vowels and consonants embedded in noise by typically hearing persons presented with stimuli simulating cochlear implants. The results show that at an SNR equal to 0 dB, recognition of vowels and consonants is perfect for non-degraded speech, while it decreases to less than 60% for speech spectrally compressed in 8 bands, which is compatible with very low recognition scores for words. Similar values are obtained by [START_REF] Friesen | Speech recognition in noise as a function of the number of spectral channels: Comparison of acoustic hearing and cochlear implants[END_REF]. A number of other studies confirm that with an SNR of 0 dB audio perception is generally quite low in CI persons (see e.g. Caldwell & Nittrouer, 2003;[START_REF] Tabanez Do Nascimento | Evaluation of speech perception in noise in cochlear implanted adults[END_REF].

The comparison of the A and AV conditions showed that the gain associated with lip-reading is similar in the TH and D/HH groups around 40 percentage point (Fig. 4). This confirms that audiovisual fusion does function efficiently in D/HH persons fitted with CI or HA as it has been found in a number of studies (e.g., [START_REF] Tyler | Performance over time of adult patients using the Ineraid or Nucleus cochlear implant[END_REF][START_REF] Lachs | Use of audiovisual information in speech perception by prelingually deaf children with cochlear implants: A first report[END_REF][START_REF] Kaiser | Talker and lexical effects on audiovisual word recognition by adults with cochlear implants[END_REF]Bergeson et al., 2005;[START_REF] Rouger | Evidence that cochlear-implanted deaf patients are better multisensory integrators[END_REF]. It could have been expected that the AV gain would be higher for deaf persons considering that they put more weight on the visual input (Desai et al. 2008) and that they are claimed to be better multisensory integrators (see [START_REF] Rouger | Evidence that cochlear-implanted deaf patients are better multisensory integrators[END_REF]. However, a careful examination of the data in [START_REF] Rouger | Evidence that cochlear-implanted deaf patients are better multisensory integrators[END_REF] shows that, while there is indeed a much higher visual gain for CI deaf participants than for typically hearing participants for spectrally degraded speech ("vocoded speech"), there is much less difference for speech in noise [START_REF] Rouger | Evidence that cochlear-implanted deaf patients are better multisensory integrators[END_REF], see their Fig. 2) in line with the present study.

It is important to note that an SNR of 0 dB consists of noise with similar energy as the signal, which is typically the case in a conversation with several partners. At such an SNR, audio-only word recognition scores across languages usually reach more than 80% for TH persons (for English, see [START_REF] Picard | Revisiting speech interference in classrooms[END_REF], this illustrates the urgent need for complementary means to reach efficient communication for this population.

Benefits of Cued Speech

As argued above, Cued Speech might be an extremely important tool for D/HH persons. In fact, word recognition jumps by 28 percentage points (55% in AV to 83% in AVC) to a level of comprehension that renders communication possible (Fig. 4). In more detail, analysis of Fig. 5 shows that in the AVC condition, 17 among the 20 participants display word recognition scores above 75% at the tested SNR.

Hence, the visual information provided by the combination of lip-reading and Cued Speech enables D/HH to recover comprehension to a level similar to scores displayed by TH participants who have no visual information, as displayed by audio-only word recognition scores in previous studies (see previous sub-section).

Note that in the AVC condition, we cannot separate the contributions of the various sources of information. As mentioned in the Introduction, it is known that lip-reading and manual cues suffice to reach a good level of comprehension (e.g. [START_REF] Nicholls | Cued Speech and the reception of spoken language[END_REF]. Hence, since we did not introduce a V + CS condition without sound, it is unclear to which extent acoustic information was used at all in the AVC condition. Still, since the experiment mixed all conditions within a single block, it is likely that participants keep taking profit of all the information available throughout the task. Indeed, the experimental data clearly show that audition is involved in the A condition, lip-reading does play a role in the AV condition and manual cues do intervene for CS readers in the AVC condition. Moreover, Bayard et al. (2014) have shown that deaf participants do integrate sound, lips and hands into a single percept. Finally, even if fusion per se was not tested in the present study, we can say at this stage that D/HH participants appear to be able to switch efficiently from A, AV to AVC conditions. In fact, this task needs to be solved frequently by D/HH CS users in conversations between typically hearing people and CS users where some interlocutors are not directly visible, some are visible but not cueing, and some are visible and cueing. Importantly, our study shows the importance of the information conveyed by CS in this kind of situation.

The analyses of correlations between SNR and reception scores shed some interesting light on the potential efficiency of CS in communication for D/HH participants. Indeed, we found a significant correlation between SNR and gain in performance associated to the CS input, either alone (AVC-AV)

or in combination with lip-reading (AVC-A). This interestingly shows that a high level of performance in CS decoding (evaluated by the AVC score or the related CS gains) does not impede a good performance without CS, quite on the contrary. It even seems that the best D/HH participants in terms of audiovisual speech reception in noise could be those who benefit most from the CS input. This is actually in line with previous studies by [START_REF] Leybaert | Cued speech for enhancing speech perception and first language development of children with cochlear implants[END_REF] or [START_REF] Aparicio | Early experience of Cued Speech enhances speechreading performance in deaf[END_REF] showing that Cued Speech provides a gain in audiovisual training enabling to improve speech perception in noise in D/HH persons. In any case, the important point is that CS is useful for improving speech perception in noise for D/HH persons mastering this process, and it does not seem to impede efficient A and AV perception without CS for these persons.

We did not find any benefit of CS for word recognition in typically hearing participants (53% for AV and 52% for AVC). We mentioned in Introduction that the CS system might enable the perceiver to focus attention in time. However, it shows that the temporal information leads to no reception benefit for those participants who do not know the phonetic interpretation of the hand positions and shapes.

The reason is probably that there is already a good amount of timing information provided by the lip movements (see e.g. [START_REF] Grant | Auditory-visual speech recognition by hearingimpaired subjects: Consonant recognition, sentence recognition, and auditory-visual integration[END_REF][START_REF] Duchnowski | Development of speechreading supplements based on automatic speech recognition[END_REF][START_REF] Kim | How visual timing and form information affect speech and non-speech processing[END_REF]. Because of this redundancy and in the light of recent results showing a robustness of the temporal information benefit across a range of time delays (Aubanel, Masters, Kim, & Davis, 2017), there is probably no more room for improvement with CS for non CS-users. Follow-up experiments might include acoustic stimuli with noise that are accompanied by CS gestures without visible lips to control for this possibility that was out of the scope of the present study. We are currently investigating this question by means of electrophysiology.

Of course, this does not mean that only D/HH participants using CS should benefit from manual cues in this task. It is expected that typically hearing persons using CS should display typically the same gain between the AV and AVC conditions, unless there exists a specific advantage in the fusion of lip and hand cues in D/HH persons, which to our knowledge has never been tested. The fact that the analysis of score evolution along the experiment did not display any learning effect shows that the experiment was too short, and quite probably too complex, to enable TH participants to detect some specificities of manual cues that could have enabled them to improve their performance. This is in fact unsurprising, considering the long time required for learning the CS system before efficient decoding (see e.g. Clarke & Ling, 1976).

Clinical implications

Due to the technology progress, there is a trend that children with cochlear implants do not consistently look at a speaker's mouth and hands [START_REF] Marthouret | Faut-il proposer la Langue Parlée Complétée à des sourds porteurs d'un implant cochléaire? (Should we propose Cued Speech to deaf children with a cochlear implant?)[END_REF]. The consequence is that some parents may lose their motivation to use Cued Speech, feel discouraged, or simply abandon coding with the hands (Leybaert & Lassaso, 2010). In the light of our results, it appears relevant and important for D/HH persons to maintain Cued Speech decoding abilities. Situations in quiet are rare in real life. Whether this is in the personal or public sphere, background noises are pervasive. Accordingly, it would be important for audiologists, speech therapists, educators, and related service providers to reflect regularly on cueing necessity in certain contexts (e.g. periods when the child is tired, speech perception in noisy situations etc.). Concerning re-education, the major challenge of speech therapists should be to find the right equilibrium between the various sources of information, audio, labial and manual.

Focussing on auditory recovery and speech-reading is important to allow children to take full advantage of their cochlear implant or hearing aid, but including CS in the re-education process might be of importance for achieving efficient communication in specific situations, particularly involving noise and adverse conditions.

Conclusion

The present study confirmed that speech perception in noise remains a challenge for D/HH persons fitted with CI of HA. Importantly, for most D/HH participants, only the combination of audition, lipreading and manual cues enabled them to reach an adequate level of perception in noise (typically above 80% correct words). Speech perception is a multimodal process in which different kinds of information are likely to be merged: phylogenetically inherited phonetic information (provided by lipreading and audition) or recently invented additional relevant information (such as CS cues). Thus for D/HH CS users fitted with cochlear implants, CI and CS could be a successful combination, in particular in noisy environments allowing these persons to further improve their speech comprehension.

Aubanel, V., Masters, C., Kim, J., & Davis, C. (2017).

Figure 1 .

 1 Figure 1. Cues in French Cued Speech: hand shapes for consonants and hand placements for vowels.

Figure 2 .

 2 Figure 2. Illustration of the three experimental conditions: Audio only (A), Audiovisual (AV) and

Figure 3 .

 3 Figure 3. Individual values of selected SNR for a theoretical AV score at 60%, and actual AV score

Figure 4 .

 4 Figure 4. Percentage of correct responses by group and conditions. A = audio only, AV = audiovisual,

Figure 5 .

 5 Figure 5. Variations of recognition scores in percent, in the three conditions (A, AV, AVC), as a

  Contribution of visual rhythmic information to speech perception in noise. Proceedings of the 14th International Conference on Auditory-Visual Speech Processing (AVSP2017), Stockholm, Sweden. Aubanel, V., Bayard, C., Strauß, A., & Schwartz, J.L. (submitted). The Fharvard corpus: A phonemically-balanced French sentence resource for audiology. Baer, T., Moore, B.C., & Gatehouse, S. (1993) Spectral contrast enhancement of speech in noise for listeners with sensorineural hearing impairment: Effects on intelligibility, quality, and response times. Journal of Rehabilitation Research & Development, 30, 49-72. Bayard, C., Colin, C., & Leybaert, J. (2014) How is the McGurk effect modulated by Cued Speech in deaf and hearing adults ? Frontiers in Psychology. 5:416. doi: 10.3389/ fpsyg.2014.00416 Bayard, C., Leybaert, J., & Colin, C. (2015) Integration of auditory, labial and manual signals in cued speech perception by deaf adults : an adaptation of the McGurk paradigm. Proceedings of The 1st Joint Conference on Facial Analysis, Animation and Auditory-Visual Speech Processing FAAVSP-2015, 163-168. Bergeson, T.R., Pisoni, D.B., & Davis, R.A. (2005). Development of audiovisual comprehension skills in prelingually deaf children with cochlear implants. Ear and Hearing, 26(2), 149-64. Bernstein, L. E., Demorest, M. E., & Tucker, P. E. (1998). What makes a good speechreader? First you have to find one. In R. Campbell, B. Dodd, & D. Burnham (Eds.), Hearing by Eye II. The psychology of speechreading and auditory-visual speech (pp. 211-228). East Sussex, U.K: Psychology Press. Bernstein, L. E., Demorest, M. E., & Tucker, P. E. (2000). Speech perception without hearing. Perception & Psychophysics, 62 (2), 233-252.

  [START_REF] Cooke | Evaluating the intelligibility benefit of speech modifications in known noise conditions[END_REF]; for Spanish see[START_REF] Aubanel | The Sharvard corpus: A phonemicallybalanced Spanish sentence resource for audiology[END_REF] for French seeAubanel et al., submitted). Hence, in comparison, the mean audio-only word recognition score of 15% for D/HH participants in our study is extremely low. As mentioned previously, AV comprehension is also quite low and does not ensure efficient comprehension for D/HH persons in noise conditions which are quite frequent in everyday life. Understanding less than 60% of words means that global comprehension is rather degraded, even if D/HH participants benefit from lip-reading. Considering that the ambient SNR in classrooms may be typically as low as -6 dB

Table 3 . Experimental design. Pseudo-randomization procedure.

 3

		Conditions	Number of sentences	Order 1	Order 2
		AV	14	Sent. 1 to 14	Sent. 15 to 28
		AVC	14	Sent. 15 to 28	Sent. 1 to 14
		Sound from AV	7	Sent. 29 to 35	Sent. 36 to 42
	A	Sound from AVC	7	Sent. 36 to 42	Sent. 29 to 35