Anne Lasfargues-Delannoy
email: lasfargues-delannoy.a@chu-toulouse.fr

Kuzma Strelnikov

Olivier Deguine

Mathieu Marx

Pascal Barone

Supra-normal skills in processing of visuo-auditory prosodic information by cochlear-implanted deaf patients

Keywords:

Cochlear implanted (CI) adults with acquired deafness are known to depend on multisensory integration skills (MSI) for speech comprehension through the fusion of speech reading skills and their deficient auditory perception. But, little is known on how CI patients perceive prosodic information relating to speech content. Our study aimed to identify how CI patients use MSI between visual and auditory information to process paralinguistic prosodic information of multimodal speech and the visual strategies employed. A psychophysics assessment was developed, in which CI patients and hearing controls (NH) had to distinguish between a question and a statement. The controls were separated into two age groups (young and aged-matched) to dissociate any effect of aging. In addition, the oculomotor strategies used when facing a speaker in this prosodic decision task were recorded using an eye-tracking device and compared to controls. This study confirmed that prosodic processing is multisensory but it revealed that CI patients showed significant supra-normal audiovisual integration for prosodic information compared to hearing controls irrespective of age. This study clearly showed that CI patients had a visuo-auditory gain more than 3 times larger than that observed in hearing controls. Furthermore, CI participants performed better in the visuo-auditory situation through a specific oculomotor exploration of the face as they significantly fixate the mouth region more than young NH participants who fixate the eyes, whereas the aged-matched controls presented an intermediate exploration pattern equally reported between the eyes and mouth. To conclude, our study demonstrated that CI patients have supra-normal skills MSI when integrating visual and auditory linguistic prosodic information, and a specific adaptive strategy developed as it participates directly in speech content comprehension.

Introduction

Communication in daily life is often defined as being multisensory (MS) as it involves integration between visual and auditory speech and paralinguistic information. Humans perceive speech via the auditory channel but also visually through lipreading. The visuo-auditory congruence of both types of information enhances speech perception in noisy environments due to multisensory integration (MSI), as evidenced by the McGurk effect [START_REF] Mcgurk | Hearing lips and seeing voices[END_REF]. However, speech is not just constructed from linguistic units but also from supra-linguistic characteristics, such as prosody, which in turn depends on changes in ample, a stressed word in a sentence can be identified solely on visual prosodic cues [START_REF] Foxton | Cross-modal facilitation in speech prosody[END_REF]. Secondly, auditory prosodic variations have visual correlates, and certain facial movements are distinctly associated with auditory prosodic cues within a sentence [START_REF] Cvejic | Prosody off the top of the head : prosodic contrasts can be discriminated by head motion[END_REF]. For example, head movements are correlated with variations in the fundamental frequency of speech (pitch) and amplitude [START_REF] Munhall | Visual prosody and speech intelligibility : head movement improves auditory speech perception[END_REF]. Furthermore, visual features of prosodic speech significantly affect auditory prosodic features such as pitch and amplitude [START_REF] Foxton | Cross-modal facilitation in speech prosody[END_REF]. Hence, in bimodal stimulation, visual prosody substantially influences auditory prosodic perception, thereby facilitating the cross-modal perception of prosody ( Strelnikov et al., 2015 ).

The multisensory aspects of speech becomes essential for individuals with severe to profound hearing loss. While, in cases of total deafness, some patients rely on visual lip-reading information for speech comprehension, vision plays an equally important role after hearing recovery with a cochlear implant. Cochlear implantation (CI), provides an electrical stimulation of the inner ear and allows the restoration of activity all along the auditory pathway. Thereby a cochlear implant partially restores auditory functions, including speech comprehension [START_REF] Moore | Beyond cochlear implants : awakening the deafened brain[END_REF] . However, the information provided by the implant is degraded and provides poor spectral information while preserving the temporal structures of sounds [START_REF] Wilson | Better speech recognition with cochlear implants[END_REF]. As a consequence, CI patients rely strongly on multisensory integration to enhance their speech perception and disambiguate the degraded auditory input provided by the implant [START_REF] Strelnikov | Role of speechreading in audio-visual interactions during the recovery of speech comprehension in deaf adults with cochlear implants[END_REF]. While the multisensory benefits in CI users depend on the speech component, [START_REF] Rouger | Evidence that cochlear-implanted deaf patients are better multisensory integrators[END_REF] have clearly demonstrated that CI deaf patients present atypical audio-visual interactions when this was related to word comprehension (for a review see [START_REF] Stevenson | Multisensory integration in cochlear implant recipients[END_REF]. CI patients present supra-normal skills of multisensory integration of speech and high proficiency in audio-visual fusion due to persistent use of visual information derived from lip-reading [START_REF] Rouger | Evidence that cochlear-implanted deaf patients are better multisensory integrators[END_REF]. The lip-reading skills allow CI patients to compensate for the impoverished signal transmitted by the processor.

Unfortunately, the prosodic structures are poorly coded by the implant due to the degraded spectral information ( Marx et al., 2015 ). CI patients' deficits in discriminating the fundamental frequency (F0) induce additional difficulties in the discrimination of auditory speech prosody -more so when the CI patients cannot take advantage of residual hearing ( Marx et al., 2015 ). This results in additional difficulties, including the identification of: gender, vo-cal identity, emotions carried by the voice as well as musical information [START_REF] Belin | Thinking the voice : neural correlates of voice perception[END_REF][START_REF] Massida | Voice discrimination in cochlear-implanted deaf subjects[END_REF]. Because of the multisensory feature of prosodic processing, CI patients could use the visual prosodic components as a compensatory mechanism for the understanding of the paralinguistic auditory information. Inspired by previous research on speech comprehension, this study aimed to analyze how the multisensory integration of prosodic cues can be specific in CI patients in relation to the impoverished information provided by the implant. In addition, deaf participants present an atypical gaze exploration of faces, especially when extracting speech information. Therefore, CI patients' oculomotor pattern during the audiovisual integration of prosodic information will be analyzed. Due to the hearing loss and cochlear implantation, the cochlear implanted patients (CIP) probably have a different gaze strategy compared to normal hearing subject, when retrieving visual prosodic cues. To explore CI deaf patients' multisensory processing, the authors developed an original protocol evaluating the discrimination of questions vs. statements sentences based on visuo-auditory prosodic information.

Materials and methods

This study aimed to evaluate the ability of CIP to process the paralinguistic prosodic information of a speaker based on the multimodal nature of such information. A psychophysics assessment of the patient's ability to discriminate prosodic cues and the oculomotor strategy used when facing the speaker was developed.

Participants

A total of 19 French cochlear implanted patients (CIP) and 36 French normally hearing participants (NH) took part in the behavioral study. Because of the demonstrated impact of cultural origin on prosody perception as well as on oculomotor face exploration [START_REF] Paulmann | Cross-cultural emotional prosody recognition: evidence from Chinese and British listeners[END_REF][START_REF] Zhu | The effects of native prosodic system and segmental context on Cantonese tone perception by Mandarin and Japanese listeners[END_REF] , all participants were French individuals with French as the first language.

The description and individual characteristics of the cochlear implanted participants are provided in Table 1 . The age range of patients was between 20 and 84 years with a mean of 57,8 (SD = 16,4 years), 9 were women, and 10 were male. Two groups of NH participants were recruited. First, a group of young, healthy controls with a mean age range of 23,6 years old ( n = 20, 10 men and 10 women). To dissociate the effect of aging from deafness, the study included a second age-matched group with a mean A trial starts with a 1-second presentation of a fixation cross, followed by a 3 s display of the written sentence, followed by another fixation cross (1 s), then the video stimuli ( ∼3s) and the two choices possible (7s).

age range of 58,1 years ( n = 16, 8 men and 10 women). All NH participants had reported normal or corrected to normal vision. All young NH participants (NHy) had self-reported normal hearing. For age-matched NH individuals (NHam), a hearing screen was undertaken to control for hearing loss. All NHam participants presented a normal or near-normal audiogram (according to the audiology guidelines -between 0-20 dB) with a mean hearing loss of 19.3 dB across speech frequencies (50 0-40 0 0 Hz). All participants who presented with neurological, sensory or motor impairments were excluded from the study. Specifically previous studies have shown the role of the vestibular function in multisensory processing and its associated alteration in deafness [START_REF] Moïn-Darbari | Vestibular status: a missing factor in our understanding of brain reorganization in deaf individuals[END_REF][START_REF] Ponzo | Vestibular modulation of multisensory integration during actual and vicarious tactile stimulation[END_REF]. Care was taken to exclude patients with vestibular dysfunction as it would impact on oculomotor strategies and high level sensori-motor functions (see [START_REF] Moïn-Darbari | Vestibular status: a missing factor in our understanding of brain reorganization in deaf individuals[END_REF] for a recent review).

Cochlear implanted patients were adults with an acquired bilateral and post-lingual profound hearing loss originating from diverse aetiologies. All CI patients included in this study were French speakers, non were bilingual as this was an exclusion criteria. The deafness was acquired in adulthood; prior to the hearing loss all participants communicated orally. A few participants relied on lipreading once the hearing loss arrived/progressed. The duration of sensory deprivation varied according to each participant (range 0-180 months -Table 1 ). Most of the patients (12/19) were unilaterally implanted. The patients were recruited on specific criteria based on our previous study on prosodic cues processing ( Marx et al., 2015 ). Indeed, because residual hearing is involved in pitch information processing since the residual acoustic information provides CI patients with F0 information; all CI patients included had no residual hearing. At the time of the study, all patients were identified as good performers -that is, patients with over 60% of dissyllabic words comprehension in a quiet environment, indicating a good auditory recuperation and adaptation to the cochlear implant [START_REF] Rouger | Evidence that cochlear-implanted deaf patients are better multisensory integrators[END_REF].

Amongst the participants that performed the prosody discrimination test, 9 CI patients and 19 NH participants took part in the eye-tracking analysis looking at oculomotor exploration during the same prosodic discrimination task. All the cochlear implanted patients who took part in the eye-tracking analysis can be identified in Table 1 with an asterisk ( * ). There was no significant difference for age, duration of deafness, exposure to oral language and dura-tion post CI between the CI patients who took part in the behavioral study and those who took part in the eye-tracking experiment ( p > . 1, Wilcoxon rank sum test). Furthermore, there were no significant differences for prosodic discrimination performances in all three conditions ( p > . 3, Wilcoxon rank sum test).

In addition, our group of patients differed according to the implantation (uni-or bilateral). A separate analysis was carried out comparing bilaterally implanted patients with unilaterally implanted patients; no group effect was identified in the data and no interaction between the number of implants and the conditions (LME4 Chi2 = .03 p = .85 and Chi2 = 2.6 p = .11 respectively). Therefore, the data was pooled into one CI patient group for clarity.

For the NHam subset of participants, the mean age was 58,1 years (SD = 4,9 years; 5 female and 4 male). For the NHy group the mean age range was 24,1 years (SD = 2,3 years; 5 female and 5 male) and the CI patients had a mean age range of 56,2 years (SD = 7,1 years; 4 female 5 male). There was no statistical difference in age between the CIP and the NHam groups.

All participants provided full informed consent prior to inclusion in this research protocol. The ethics committee approved this study (n °AFSSAPS: B90183-20).

Stimuli and presentation procedure

The stimuli presented to the participants were 24 videos of unpunctuated spoken sentences, posited to the individual as either a question or a statement using prosodic cues. The mean pitch, mean pitch variation, and mean duration of each sentence are provided in the supplementary materials. The sentences used contained no syntactic, linguistic or other bias and were taken from [START_REF] Patel | Processing Syntactic Relations in Language and Music: An Event-Related Potential Study[END_REF] and previously used in [START_REF] Marx | Speech prosody perception in cochlear implant users with and without residual hearing[END_REF] . Here are some sentence examples: "François est au restaurant" (Francis is at the restaurant), "il est à Paris depuis 3 mois" (he has been in Paris for 3 months) "elle joue de la flute" (she plays the flute).

Two actors, one male, one female, were recorded with a professional digital camera whilst they were pronouncing the sentences. Each sentence was then isolated from the main recording using windows moviemaker®, and the final stimuli were created ( Fig. 1 ).

To avoid a ceiling effect and compare the performances obtained by the CI patients and NH participants, care was taken to deliberately reduced the prosodic features both in the visual and Fig. 2. Movement analysis of prosodic stimuli computed in both actors for all sequences. A systematic frame difference showed that most of the movements that differentiate a question from a statement occur at the level of the eyes and more precisely the right facing one. The difference at the mouth level correspond to the iteration of the sentence. Scale was in AU. auditory modalities. Firstly, the two French actors reduced their facial expressions to diminish the natural visual cues associated with a question and a statement, thereby increasing the task's difficulty. The videos were specifically analyzed to assess where the facial movements (chin, eyebrow, ….) acted as prosodic cues to discriminate a question from a statement. A systematic frame difference through a dense optical flow analysis was implemented in the cv2 Python® package. This movement analysis showed that most of the changes in movements of the face were located at the left eye ( Fig. 2 ). The mouth region also emerged from the analysis and corresponded to the lip movements during the actors' allocution of the sentence ( Fig. 2 ). Excluding the mouth region, it appears that the movements around the left eye carried most of the prosodic information needed to differentiate a question from a statement in the V-only and AV conditions. This could be attributed to the eyes being wide open and to eyebrows inflections.

Secondly, for the NH group, the auditory information was degraded using an 8-channel vocoding system. An 8-channel vocoding system was used to obtain similar performances in the A only condition for the NH participants during the speech comprehension task (see [START_REF] Rouger | Evidence that cochlear-implanted deaf patients are better multisensory integrators[END_REF][START_REF] Strelnikov | Effects of vocoding and intelligibility on the cerebral response to speech[END_REF][START_REF] Marx | Speech prosody perception in cochlear implant users with and without residual hearing[END_REF]. The vocoding simulates the processes that occur in a cochlear implant as the sound's temporal information was preserved, but the fine acoustical structures are degraded [START_REF] Rouger | Evidence that cochlear-implanted deaf patients are better multisensory integrators[END_REF]. The following cut-off frequencies were used: 250,437,709,1104,1676,2507,3713,5462, and 80 0 0 Hz -please refer to the [START_REF] Rouger | Evidence that cochlear-implanted deaf patients are better multisensory integrators[END_REF] article for further details. Our main interest was related to the performance level during the A only condition and how this performance level was enhanced during the AV trials. This reduction in frequency information makes the distinction between a question and a statement more difficult as changes in pitch are associated with linguistic prosodic information (see [START_REF] Marx | Speech prosody perception in cochlear implant users with and without residual hearing[END_REF]. Indeed, the acoustical analysis of the two types of sentences showed that the mean pitch amplitude measured across the sentence increased with questions compared to statements; i.e., questions were subject to greater pitch variations than statements (see supplementary section). Indeed, statement sentences were characterized by a minimum pitch value at the end, while for questions, it was the maximum pitch value that was mainly observed at the end of the sentence.

The reduction of the visual and of the auditory cues for the NH participants, through the use of 8-channel vocoding, enabled a more accurate comparison of the two groups (CI patients and NH participants). Controlling the sensory perception in the NH groups aimed to degrade the auditory perception of NH participants so that it was similar to CI patients. These measures allowed for an accurate comparison of the multisensory gain for linguistic prosodic information in the patient group.

Stimuli presentation

The task was a two alternative-forced-choice task during which the participants had to decide if the sentence (A, V, or AV) was a statement or a question. In order to exclude the language component of the task, especially for the CIP, the unpunctuated written sentence was presented prior to the video (see Fig. 1 ). The participants received no speed instructions since this was not the primary aim of the study.

For each condition, the participants were presented with a fixation cross, followed by the written sentence (unpunctuated to avoid prosodic inference), followed by another fixation cross, and the video according to the condition presented (V only, A only, VA). The delay between the apparition of the first cross and the video was 5 seconds. The recorded videos had durations of ∼2.5 s and were of similar duration between statements and questions (1.5-3 s). At the end of the video, the participants had to make a judgment on the prosodic form of the sentence by pressing a response key on a keyboard. The position was randomly assigned to each individual.

The stimuli were presented through Matlab®, on a portable PC, Dolby® speakers were located on either side of the computer screen, and the volume was calibrated between 55 and 65 dB using a sound meter. The presentation of the sentences according to the modalities were by blocks, but the order of the stimuli within each condition was random. The visual condition was always presented first, to avoid familiarization with the speakers' expressions, followed by either the visuo-auditory or auditory alone conditions, which varied alternatively for each participant.

The oculomotor behavior analysis was recorded and extracted from the eye-tracker Tobii X2-60® system and analyzed with Mat-lab®. This was used to investigate the patients and NH participants facial exploration strategies during the visual and visuoauditory conditions. Prior to beginning each condition, the To-bii® eye-tracker was calibrated to each individual, using a 5-point calibration system. This calibration ensured that participants presented no alteration for eye-muscle movements. All points had to be validated before the start of the eye-tracking experiment.

These participants were asked to keep their head forward and look at the screen for the task's duration. A chin-rest was used for all NH participants but not for CIP due to the cochlear implants magnet. The experimenter was nevertheless controlling that the patients maintained their head in front of the screen.

Data analysis

The behavioral data was collected in terms of hit rates, i.e., the percentage of correct responses was extracted and analyzed using Excel®, Matlab®, and R®. Firstly, a comparative analysis was implemented for the three groups' performances across conditions using a mixed effect linear model (LME4). This model enables to describe the effects (random or fixed) with regards to the data implemented. Secondly, the normalized visuo-auditory gain (VAgain) was calculated between the bimodal condition and the individual's best unimodal channel. This VA gain returns a percentage of enhancement in bimodal stimulation versus unimodal. Then, CIP's normalized VAgain was compared to both NH groups using Kruskal-Wallis non-parametric tests and bootstrap values.

Regarding the eye-tracking analysis, three areas of interest were predefined: the eyes, the nose, and the mouth. The raw fixation data extracted from Tobii® was implemented into Matlab® to calculate the fixation percentages in the predefined visual areas of interest. Comparisons across conditions and groups were carried out using a mixed effect linear model in R®.

Results

This study aimed to evaluate the capacity of cochlear implanted adult patients (CIP) to discriminate a question from a statement based on visual, auditory, and the combined visuo-auditory prosodic information. The CIP group performances were compared to normally hearing counterparts (NH) tested with vocoded auditory information that mimics a cochlear implant degradation to avoid a ceiling effect. Similarly, the visual condition was also con-trolled in such that the actors were required to reduce their facial expressions.

Behavioral analysis across conditions and groups

Each participant had to identify whether the sentence presented in one of the three conditions (V only, A only, and VA) was a question or a statement. No comprehension was required as the written sentence was presented prior to the video or the audio recording. The performances are presented in Fig. 3 . Globally the identification score was optimal across groups for all conditions without attaining a ceiling effect. The NHy and NHam groups presented a mean of 79% (SD = 8%) and 71% (SD = 10%) respectively for the A only vocoded condition. The CIP group evaluated with the original sound presented a mean score of 78% correct (SD = 12%), a relatively significant impairment similar to that reported in our previous study ( Marx et al., 2015 ). The differences between the groups are not significant according to the mixed linear model LME4 ( z -value = 0.382 p = .923 (CIP-NHam); z -value = -1.041 p = .551(CIP -NHy) . The use of vocoding had the desired effect since it impaired NH participants to similar comprehension levels than the cochlear implanted patients.

In real life, the visual prosodic cues are easily distinguishable; but by asking our actors to reduce them during recording, an identification score below the maximum performance level was obtained in all groups. Indeed, for the V only condition, the NHy participants presented a performance level of 79% (SD = 9%) and 77% (SD = 11%) for the NHam group. Unexpectedly, the CIP group obtained a mean performance level for perceiving visual prosody of 69% (SD = 14%), which was significantly worse than both con-trol groups (mixed-effect linear model: z -value = -3.252 p = .004 (NHy); z -value = -2.739 p = .02(NHam)). Consequently, and contrary to our initial assumption, the CIP group was not better at detecting and identifying visual prosodic cues than hearing controls. Furthermore, two patients were at chance level, whilst that was the case for only one aged normal hearing individual.

According to the well-established multi-integration benefits, all groups had higher identification percentages in the bimodal condition than unimodal presentation. Indeed, the CIP group obtained 89% (SD = 9%) of correct prosodic identification, which was higher than the scores obtained in either the visual or the auditory presentations. Similarly, hearing controls presented scores of 85% (SD = 5%) for the NHy group and 82% (SD = 9%) for the NHam group leading to an absence of significance between the groups (mixed linear effect model (LME4): z -value = 1.094 p = .519 (CIP-NHy); z -value = 1.926 p = .136 (CIP-NHam)). Intra groups' analysis revealed that AV performances are systematically higher than the unimodal conditions. When analyzing each group separately, the CIP group improved across all three conditions V < A < VA ( p = .001). Although the NHy and NHam groups did not differ significantly on the V-only and A-only conditions; a significant difference was found between their performances in the unimodal and in the bimodal conditions ( p = .01(NHy); p = .001 (NHam)). Of interest, and as illustrated in Fig. 3 , there was an important variability in the performances of the CIP group and the NHam group in the A and V-only conditions. However, such variability tends to diminish in bimodal stimulation. This result suggests that bimodal stimulation significantly improves performances across all groups and reduces the variability through visuo-auditory integration for both the CIP and NH groups.

Visuo-auditory gain

Our results clearly showed that both the CIP group and the NH group improved their proficiency to discriminate prosodic information in the visuo-auditory condition. In order to compare the visuo-auditory multisensory benefits across groups, a normalized visuo-auditory gain was computed (VAgain, [START_REF] Rouger | Evidence that cochlear-implanted deaf patients are better multisensory integrators[END_REF]. In both NH groups, the unimodal performances were the same irrespective of the presentation modality. Meanwhile, the CIP group were unexpectedly slightly (but significantly) better in the auditory modality than in the visual modality. In order to examine fairly the impact of bimodal presentation on the participant's identification scores, the modality (A only or V only) in which the participant presented their best performance was identified. As shown in Fig. 3 , three quarters (74%) of the CIP group were better in the auditory rather than in the visual condition. Such proportion was more balanced in the normal controls; since 55% of the NHam group and 60% for the NHy group favored the auditory condition. However, this tendency for a lower proportion (26%) of better visual performers in the CIP group was non-significant ( z -test p = .49).

Because of this slight disparity between groups, we decided to compute the VAgain for all participants based on each individual's preferential modality. As you can see from the insert in Fig. 3 , each participant has a preferred modality for identifying linguistic prosodic features. Some participants were better performers in the visual modality and others in the auditory modality. In order to take into account these individual preferences, the VA gain was normalized and calculated for each participant based on their preferred modality. Thus, in the CIP group, it was principally the performances in the A-only modality, which was the unimodal preference (14 cases), same for the controls but less pronounced (12 cases in NHy and 10 cases in NHam). There was't a significant difference between all three groups when comparing the mean performances in the best unimodal modality (CIP 78.82% SD = 11%, NHy 82.89% SD = 6%, and NHam 79.38% SD = 9% Kruskal-Wallis rank sum test for non-parametric data p = .435, H = 1.66, corrected ex aequo p = .428). Consequently, the normalized VAgain was computed in percentage enhancement (see methods) based on the best unimodal performances across groups. The term percentage enhancement refers to: how much the visuo-auditory stimulation enhances the performance of each individual in comparison to their best unimodal score.

As shown in Fig. 4 , the CIP group differed significantly from both groups of hearing participants as they presented a visuoauditory multisensory gain, which was about 4 times greater than that observed in the NHy and NHam groups (13.4% vs. 3.2% vs. 3.5%). Such results clearly demonstrated that the CIP group was much more proficient at fusing the visuo-auditory prosodic cues than hearing participants. Of importance, while at the group level, the performances for the NH group were higher in the bimodal situation, no visuo-auditory multisensory benefit was observed once the MS gain was individually computed. Indeed, based on bootstrap confidence intervals analysis, the VA gain observed in both the NH groups was not significantly different from zero and a limited number of the NH group (11% of the total NH group) present a negative VAgain, thereby being less efficient in the visuo-auditory condition. Note that none of the CI patients presented a negative VAgain.

Overall, these results highly suggest stronger skills for processing visuo-auditory prosodic information in the CIP group compared to both NH groups.

Gaze exploration analysis

An eye-tracking analysis was undertaken to analyze what areas of the faces the participants explored during the visual and visuoauditory conditions. The speakers' face was divided into three areas, namely the eyes, the nose region, and the lower part of the face, including the mouth (see methods). No specific differences in oculomotor behavior were observed in the V and AV modalities (see sup. Material), and the data was pooled together for clarity.

The heat maps presented ( Fig. 5 ) give a global overview of fixation patterns across all three groups. In the CIP group, the gaze was mostly concentrated on the lower half of the face. In contrast, the NHy group have a preferential gaze for the upper half of the face, notably the eyes. The NHam group presented with a mixed oculomotor pattern. The eye tracking quantification clearly showed that the CIP group had a different gazing strategy than the NH group, and that in controls, aging also impacts facial exploration ( Fig. 5 &6 ). The NHy participants spent most of their time (63%) fixating on the speaker's eyes with a slight tendency to target the right facing eye (see Fig. 5 &6 ). NHy participants similarly weakly fixated the other two regions (nose 20% and mouth 17%). Such a pattern was radically different in the deaf CIP group as the patients spent 50% of their fixation-time targeting the mouth region, whereas the rest of the time was roughly distributed between the nose (30%) and eyes (20%). Compared to the CIP and the NHy participants, the NHam group present an intermediate fixation pattern. Indeed, most of the fixations were directed toward the mouth (48% of their fixations) and the eyes-region (38%), while the nose region was only weakly explored (14%) during the prosodic discrimination task.

The statistical analysis confirmed the different oculomotor behavior patterns as the CIP group fixated significantly less on the eyes region than the NHy participants (21% vs. 63%; z -value = -4.241 p = .001). On the other hand, the former participants significantly fixated on the mouth area more than the NHy group (50% vs. 17%; z -value = 3.29 p = .004). At the group level, no differences were found between the CIP and the NHam groups due to individual variability. Nevertheless, there are significant differences between the NHy and the NHam participants. The latter group, NHam, significantly fixated the mouth more than their younger counterparts (48% vs. 17%; z -value = 3.244 p = .005), whereas NHy fixated more on the eyes than the NHam group (63% vs. 38%; zvalue = -2.573 p = .031). This suggests that, while the CIP and the NHam groups presented different levels of performances in visuoauditory prosodic integration, the CIP groups proficiency relies on a quite similar strategy of oculomotor exploration of the speaking face compared to age-matched controls. On the opposite, while aging impacts on the oculomotor behavior of normal hearing partic-ipants, it does not improve their ability to process visuo-auditory prosodic information.

As provided in supplementary data, there were no correlations between the patient's history (duration of deafness, age…) and the behavioral performances or oculomotor behavior.

Discussion

In a previous study, [START_REF] Rouger | Evidence that cochlear-implanted deaf patients are better multisensory integrators[END_REF] demonstrated that CI deaf patients were specifically efficient in fusing visual and auditory information when it concerns speech information. Such profi- ciency for visuo-auditory speech processing was related to CI patients' continuous need to decode the poor signal delivered by the implant, specifically in adverse hearing conditions such as noisy environments. The data presented here is clear evidence that such supra-normal skills can be extended to paralinguistic information. Firstly, this study confirms that the CI processor does not allow the patients to discriminate the subtle modulations of suprasegmental cues (pitch, intensity) that differentiate a question from a statement [START_REF] Chatterjee | Processing F0 with cochlear implants : modulation frequency discrimination and speech intonation recognition[END_REF][START_REF] Peng | Effects of cooperating and conflicting cues on speech intonation recognition by cochlear implant users and normal hearing listeners[END_REF][START_REF] Marx | Speech prosody perception in cochlear implant users with and without residual hearing[END_REF][START_REF] Hay-Mccutcheon | Performance variability on perceptual discrimination tasks in profoundly deaf adults with cochlear implants[END_REF]. However, CI patients presented strong multisensory facilitation when integrating visual and auditory linguistic prosodic information, a benefit that was nearly absent in normal hearing participants. Such proficiency relies on a specific oculomotor strategy when patients integrate the visual prosodic cues of the face.

Prosodic processing is multisensory

Firstly, the data clearly showed that prosodic processing is a visuo-auditory multisensory mechanism, in which visual and auditory cues are complementary and integrated to provide multimodal paralinguistic information. Indeed, all subject groups, deaf and hearing, demonstrated an enhanced performance from unimodal to bimodal stimulation. The visual prosodic cues, when presented congruently with the auditory cues, increased prosodic speech perception. The present results add to those of previous studies suggesting the presence of a temporal correlation between head or eyebrow movements and acoustic changes such as F0 variations [START_REF] Guaïtella | Are eyebrow movements linked to voice variations and turn-taking in dialogue ? an experimental investigation[END_REF][START_REF] Munhall | Visual prosody and speech intelligibility : head movement improves auditory speech perception[END_REF]. In effect, it has been found that the visual prosodic cues can play the role of a rhythmic marker to detect and amplify au-ditory features of prosody, namely pitch and amplitude changes [START_REF] Foxton | Cross-modal facilitation in speech prosody[END_REF]. These rhythmic markers could consequently act as syntactic boundaries and contribute to speech comprehension [START_REF] Cutler | Prosody in the comprehension of spoken language : a literature review[END_REF][START_REF] Munhall | Visual prosody and speech intelligibility : head movement improves auditory speech perception[END_REF][START_REF] Swerts | Facial expression and prosodic prominence : Effects of modality and facial area[END_REF]. At the group level, all groups had significantly higher performance values in the bimodal presentation compared to the unimodal (A or V) conditions. However, when individual visuoauditory benefits were calculated from the best unimodal performance level, only CI patients presented a significant gain. As most of the NH participants were more efficient in the auditory modality, the increase of AV performance compared to Aonly condition can be nevertheless considered a visuo-auditory facilitation.

While it was not the study's purpose, aging did not affect the multisensory benefit for visuo-auditory integration, since aged adults presented prosodic discrimination performances and VA gains, which were similar to the young ones. There are numerous and conflicting results concerning how aging affects multisensory interactions in temporal discrimination tasks unrelated to speech [START_REF] Brooks | Audio-visual temporal perception in aging : the role of multisensory integration and age-related sensory loss[END_REF][START_REF] Deloss | Multisensory integration, aging, and the sound-induced flash illusion[END_REF][START_REF] Laurienti | Enhanced multisensory integration in older adults[END_REF]. When visuo-auditory speech processing was concerned, despite contradictory reports, it appears that elderly participants are more strongly influenced or biased by visual information in McGurk paradigms [START_REF] Mcgurk | Hearing lips and seeing voices[END_REF] . This was probably due to an age-related delayed auditory processing [START_REF] Cienkowski | Auditory-visual speech perception and aging[END_REF][START_REF] Sekiyama | Enhanced audio-visual integration with aging in speech perception : A heightened McGurk effect in older adults[END_REF], while the visual lip-reading performances remain unaffected [START_REF] Huyse | Effects of aging on audio-visual speech integration[END_REF]Tye-Murray et al., 20 07a , 20 07b ). In our study, the NHam group did not present enhanced visual performances, suggesting that VA multisensory prosodic processing, as for audiovisual speech integration, might not be affected by aging.

CI patients are highly efficient for integrating visuo-auditory prosodic information

Our second theoretical issue relates to the notion that CIP are more efficient than the NH participants at fusing visual and auditory prosodic information. They presented a visuo-auditory MS gain, which was up to 3 times greater than observed in hearing groups. First, such higher proficiency was not related to aging, as discussed previously. Second, these results are in accordance with reports on simple RT tasks in elderly and young CI patients showing that multisensory integration was similarly present at all ages [START_REF] Schierholz | Enhanced audio-visual interactions in the auditory cortex of elderly cochlear-implant users[END_REF].

The results of a higher proficiency at fusing visuo-auditory information in CI patiens has been previously demonstrated for speech material [START_REF] Kaiser | Talker and lexical effects on audio-visual word recognition by adults with cochlear implants[END_REF][START_REF] Rouger | Evidence that cochlear-implanted deaf patients are better multisensory integrators[END_REF]. Initially, it was proposed that the higher MS benefits in deaf patients could derive from the inverse effectiveness effect [START_REF] Ross | Do you see what I am saying ? Exploring visual enhancement of speech comprehension in noisy environments[END_REF]. Either the poor auditory processing allowed by the implant or the higher visual skills (lip-reading) developed as a compensatory strategy during deafness allowed CI patients to obtain higher MS benefits. None of these possibilities can explain the present results since de VA multisensory gain was specifically computed from the same unisensory performance level obtained by reducing the available information. At the group level, the CIP and the NH groups presented similar performance levels in visual or auditory modalities. It could be said that for the auditory modality CI patients have access to more information than the NH participants since cochlear implants have between 12-22 electrodes (depending on the manufacturer) coding for the sound. So the CIP group may have an advantage over hearing controls who are perceiving 8-channel vocoded speech. In addition, the MS gain was calculated from the individual's best unimodal performance that put the three groups at the same performance level. Based on these identical unimodal skills, it could strongly be hypothesized that the higher MS benefits in CI patients resulted from a stronger proficiency to integrate visual and auditory paralinguistic information.

Of interest, while CI patients present better lip-reading abilities than hearing controls [START_REF] Calvert | Activation of auditory cortex during silent lip-reading[END_REF][START_REF] Rouger | Evidence that cochlear-implanted deaf patients are better multisensory integrators[END_REF][START_REF] Strelnikov | Role of speechreading in audio-visual interactions during the recovery of speech comprehension in deaf adults with cochlear implants[END_REF], they are surprisingly not better, and even worse at discriminating visual prosodic cues. A similar trend has been reported in CI patients' visual discrimination ability for visual emotional prosodic information [START_REF] Fengler | Multisensory emotion perception in congenitally, early, and late deaf CI users[END_REF]. To our knowledge, there was no evidence of a lower proficiency of CI patients in any visual processing. Thus, it might be possible that the CI patients are only unable to decode and transfer the semantic content from the visual cues, a capacity which was obtained when combined with the auditory information.

In light of previous studies on speech and prosodic cues, it was hypothesized that CI patients would be more proficient at distinguishing multisensory prosodic sentences. This experiment revealed that CIP had a significant normalized VA gain, which was about 3 times larger than that obtained in hearing controls. This result follows on from work done by [START_REF] Rouger | Evidence that cochlear-implanted deaf patients are better multisensory integrators[END_REF] for speech comprehension post cochlear implantation. Nevertheless, in this latter case, the difference was not as large as in the present study. One reason could be that in the present study the patients had a long-term experience with the implant whilst they were evaluated at the time of implant activation in the Rouger et al. study. Habituation to the electric information could explain the increased VA gain. But it could be suggested that the higher proficiency of CI patients for visuo-auditory integration was dependent on the task.

Previous studies have shown that CI patient can integrate multimodal stimuli with perceptual benefits that are characteristic of multisensory integration [START_REF] Stein | Cognitive Neuroscience. The Merging of the Senses[END_REF][START_REF] Stein | Development of multisensory integration from the perspective of the individual neuron[END_REF]. But when it concerns low-level sensory pro-cessing, they are not better than the NH individuals [START_REF] Gilley | The influence of a sensitive period for auditory-visual integration in children with cochlear implants[END_REF][START_REF] Schierholz | Enhanced audio-visual interactions in the auditory cortex of elderly cochlear-implant users[END_REF]. Furthermore, CI proficiency has been shown to impact audio-visual segregation skills [START_REF] Champoux | Visual stimuli can impair auditory processing in cochlear implant users[END_REF][START_REF] Landry | Audiovisual segregation in cochlear implant users[END_REF][START_REF] Tremblay | Audiovisual fusion and cochlear implant proficiency[END_REF]. Indeed, poor performers with the cochlear implant were easily disturbed by visual distractors while highly proficient CI patients were highly impacted by auditory speech stimuli during a visual speechreading task. To distance ourselves from this efficiency constraint, only patients with a good recovery of speech understanding were recruited. In addition, because of the multisensory nature of speech, the audiovisual interactions may be governed by different rules compared to non-speech material [START_REF] Vatakis | Facilitation of multisensory integration by the "unity effect" reveals that speech is special[END_REF]. This could impact on the CI-NH participant comparison. However, the CI patients have been reported as good performers in other forms of multisensory interaction such as for audio-tactile interactions [START_REF] Nava | Audio-tactile integration in congenitally and late deaf cochlear implant users[END_REF]. Indeed, audio-tactile stimulation has been used as an intervention methods in hearing impaired babies and children to provide auditory information relating to speech rhythm. In adults, tactile aids have been shown to improve speech perception, lip-reading and detection thresholds [START_REF] Lakatos | Neuronal oscillations and multisensory interaction in primary auditory cortex[END_REF][START_REF] Rothenberg | Encoding voice fundamental frequency into vibrotactile frequency[END_REF][START_REF] Brooks | Acquisition of a 250-word vocabulary through a tactile vocoder[END_REF]. Electro-tactile stimulation enhanced pitch perception and melody recognition in CI users regardless of musical training [START_REF] Huang | Electro-tactile stimulation enhances cochlear-implant melody recognition: effects of rhythm and musical training[END_REF]. Furthermore, elecro-haptic stimulation on the forearm has recently been shown to improve pitch discrimination in NH participants listening to vocoded speech in quiet and noise [START_REF] Fletcher | Electro-haptic enhancement of speech-in-noise performance in cochlear implant users[END_REF].

It's possible that the CI patients' higher proficiency to fuse visual and auditory cues is specific to speech and speech-related information [START_REF] Kaiser | Talker and lexical effects on audio-visual word recognition by adults with cochlear implants[END_REF][START_REF] Most | Auditory, visual, and auditory-visual perception of emotions by individuals with cochlear implants, hearing aids, and normal hearing[END_REF][START_REF] Rouger | Evidence that cochlear-implanted deaf patients are better multisensory integrators[END_REF]. A recent study analyzing visuo-auditory temporal judgment showed that CI patients presented a shift in temporal acuity specific to speech material that was not present for simple flashbeep presentations [START_REF] Butera | Audio-visual temporal processing in postlingually deafened adults with cochlear implants[END_REF]. CI patients had a greater acuity to detect asynchronies between the auditory and visual speech stimuli, specifically when the visual information arrived first. Prosody is an integral part of communication, it can either be emotional or linguistic. Emotional prosody relates to the state of the speaker and the emotional meaning associated, whereas linguistic prosody allows the speaker to highlighting key speech information and meaning and can be crucial for speech comprehension. For example, linguistic prosodic information is important for word segmentation based on intonational cues. This task is difficult for CI patients in the auditory-only modality but it can be achieved through visual prosodic cues [START_REF] Basirat | Word segmentation in phonemically identical and prosodically different sequences using cochlear implants : a case study[END_REF][START_REF] Guaïtella | Are eyebrow movements linked to voice variations and turn-taking in dialogue ? an experimental investigation[END_REF]. Thus, CI patients may have developed specific skills to combine visual and auditory prosodic cues as they participate directly in speech content comprehension. However, it remains unclear why the discrimination of the emotional content in a sentence is improved in multimodal conditions similarly in CI patients and hearing participants [START_REF] Fengler | Multisensory emotion perception in congenitally, early, and late deaf CI users[END_REF] as it relies on similar acoustic changes.

Oculomotor strategies and prosodic processing

Our study demonstrated a higher ability of CI patients to integrate visuo-auditory linguistic prosody. This higher proficiency is supported by facial oculomotor exploration that differs from normal hearing participants. While CI patients focused more on the mouth region, younger hearing controls gazed preferentially at the eyes. Of interest, aged hearing participants tended to have an intermediate oculomotor strategy with equal fixation of the mouth and eye region. Such results are compatible with previous studies attesting that, with aging, the facial exploration tends to shift toward the lower part of the face [START_REF] Chaby | Gaze behavior consistency among older and younger adults when looking at emotional faces[END_REF][START_REF] Wong | Patterns of visual scanning as predictors of emotion identification in normal aging[END_REF]. This gaze strategy in older participants appears to be nonoptimal, leading to lower performances for extracting facial emo-tional information that could be one of the origins of the inability of aged participants to decode social cues and mental states of others [START_REF] Sullivan | Age differences in emotion recognition skills and the visual scanning of emotion faces[END_REF][START_REF] Phillips | Lifespan aging and belief reasoning : influences of executive function and social cue decoding[END_REF][START_REF] Slessor | Age-related differences in processing face configuration : the importance of the eye region[END_REF][START_REF] Slessor | Specificity of age-related differences in eye-gaze following : evidence from social and nonsocial stimuli[END_REF]. However, such putative relation between discrimination performance and face exploration does not apply to deaf CI patients as they presented a higher VA gain. The main objective was to assess whether CI patients used a different oculomotor strategy in order to enhance prosodic discrimination. Firstly, it could be a combination of the effect of aging (see above) and of the strong lip-reading behavior maintained post-implantation by the deaf patients to compensate for the crude auditory speech information [START_REF] Rouger | Evidence that cochlear-implanted deaf patients are better multisensory integrators[END_REF][START_REF] Tyler | Evaluation of five different cochlear im plant designs : audiologic assessment and predictors of performance[END_REF]. It is possible that CI patients automatically directed their attention toward the mouth region regardless of speech comprehension per se, which was not required for the task; the written sentence being provided to the participants (see methods). However, by fixating the mouth, the face region that provided the visual prosodic information (the eyebrow region, see Fig. 2 ) was consequently located in the peripheral visual field at about 14 °of eccentricity in our experimental set-up. There is evidence showing that profoundly deaf patients developed enhanced visual performances for the peripheral visual field -with enhanced visual skills for processing some aspects of facial attributes [START_REF] Armstrong | Auditory deprivation affects processing of motion, but not color[END_REF][START_REF] Chen | Effects of spatial distribution of attention during inhibition of return (IOR) on flanker interference in hearing and congenitally deaf people[END_REF][START_REF] Chen | Altered spatial distribution of visual attention in near and far space after early deafness[END_REF]Codina, Buckley, et al., 2011 ;Codina, Pascalis, et al., 2011 ;[START_REF] Muir | Perception of sign language and its application to visual communications for deaf people[END_REF][START_REF] Neville | Attention to central and peripheral visual space in a movement detection task : an event-related potential and behavioral study. II. Congenitally deaf adults[END_REF][START_REF] Arnold | Memory for faces and objects by deaf and hearing signers and hearing nonsigners[END_REF][START_REF] Bettger | Enhanced facial discrimination : effects of experience with American sign language[END_REF][START_REF] Mccullough | Face processing by deaf ASL signers : evidence for expertise in distinguishing local features[END_REF]. Unfortunately, very little is known about CI patients' face perception skills besides the presence of a deficit in discriminating emotional facial expression [START_REF] Ambert-Dahan | Emotional recognition of dynamic facial expressions before and after cochlear implantation in adults with progressive deafness[END_REF][START_REF] Wiefferink | Emotion understanding in deaf children with a cochlear implant[END_REF]. This is in agreement with the weaker ability to detect visual prosodic cues observed in CI patients in the V-only condition. Thus, the absence of better visual discrimination of CI patients for facial attributes confirms that their supra-normal skill takes its origin in the capacity to combine specifically the peripheral visual cues with the auditory information.

Alternatively, it is now firmly established that any acoustical change produced by the vocal tract articulators induces a simultaneous facial deformation specifically around the mouth, but not only ( Vatikiotis-Bateson et al., 1998 ). Therefore, it's possible to suggest that by fixating the lower part of the face the CI patients were able to decode the simultaneous unique visuo-auditory changes in information for linguistic prosodic discrimination. This occurs whilst CIP presented a fixation map similar to aged hearing participants who only obtained a weak visuo-auditory gain.

In summary, the present study does not make it possible to conclude whether CI patients' specific oculomotor exploration was devoted to higher decoding of the auditory cues combined either with the peripheral (eyebrow) or the central (mouth centered) facial visual cues, or both.

Visuo-auditory multisensory integration in CI users

CI patients' higher skills to integrate visual and auditory linguistic prosody probably relied on a functional adaptation of the brain network involved in speech and multisensory integration. Numerous studies have shown how the auditory system of a deaf brain is progressively reactivated after cochlear implantation by speech stimuli [START_REF] Green | Auditory cortical activation and speech perception in cochlear implant users : effects of implant experience and duration of deafness[END_REF][START_REF] Mortensen | Restored speech comprehension linked to activity in left inferior prefrontal and right temporal cortices in postlingual deafness[END_REF][START_REF] Pantev | Dynamics of auditory plasticity after cochlear implantation : a longitudinal study[END_REF]. However, differential activation patterns emerge in CI patients reflecting compensatory mechanisms due to the degraded auditory information conveyed by the cochlear implant [START_REF] Giraud | The contribution of visual areas to speech comprehension : a PET study in cochlear implants patients and normal-hearing subjects[END_REF]Giraud et al., 20 0 0 , 20 01 ). To our knowledge, no study has specifically analyzed the brain network involved in auditory prosodic information in the CIP group, while in NH individuals it is supposed to involve an extensive bilateral cortical and subcortical network [START_REF] Hellbernd | Prosody conveys speaker's intentions : acoustic cues for speech act perception[END_REF][START_REF] Sammler | Dorsal and ventral pathways for prosody[END_REF][START_REF] Wildgruber | Distinct frontal regions subserve evaluation of linguistic and emotional aspects of speech intonation[END_REF][START_REF] Witteman | The nature of hemispheric specialization for linguistic and emotional prosodic perception : a meta-analysis of the lesion literature[END_REF]. However, there is now a strong assertion that in CI patients, the processing of visuo-auditory speech information is supported by a different cortical network compared to hearing counterparts ( Strelnikov et al., 2015 ;[START_REF] Stropahl | Auditory cross-modal reorganization in cochlear implant users indicates audio-visual integration[END_REF]. The crossmodal compensation in CI deaf patients is expressed by specific colonization of the auditory areas by visual functions [START_REF] Debener | Source localization of auditory evoked potentials after cochlear implantation[END_REF]Giraud et al., 20 0 0 ;Lee et al., 20 01 ;[START_REF] Stropahl | Auditory cross-modal reorganization in cochlear implant users indicates audio-visual integration[END_REF]. This crossmodal reorganization is specifically related to CI outcomes [START_REF] Doucet | Cross-modal reorganization and speech perception in cochlear implant users[END_REF][START_REF] Lee | Cortical activity at rest predicts cochlear implantation outcome[END_REF][START_REF] Sandmann | Visual activation of auditory cortex reflects maladaptive plasticity in cochlear implant users[END_REF]Strelnikov et al., 2015 ). Of interest, progressively after cochlear implantation, during visuo-auditory speech processing, there's a broader involvement of the middle/posterior temporal gyrus known as a locus of multisensory processing [START_REF] Beauchamp | FMRI-guided transcranial magnetic stimulation reveals that the superior temporal sulcus is a cortical locus of the McGurk effect[END_REF]Belin et al., 20 0 0 , 20 04 ;[START_REF] Belin | Adaptation to speaker's voice in right anterior temporal lobe[END_REF][START_REF] Strelnikov | Role of speechreading in audio-visual interactions during the recovery of speech comprehension in deaf adults with cochlear implants[END_REF]Strelnikov et al., , 2015 ) ). The middle/posterior temporal gyrus is also involved in creating audio-visual verbal concepts by matching auditory and visual speech stimuli information or when the auditory speech information is degraded [START_REF] Hocking | Dissociating verbal and nonverbal audio-visual object processing[END_REF][START_REF] Ozker | A double dissociation between anterior and posterior superior temporal gyrus for pro-A[END_REF][START_REF] Thierry | Dissociating verbal and nonverbal conceptual processing in the human brain[END_REF]. Therefore, an experienced CI patient would mobilize the multimodal STS/STG regions involved in multisensory linguistic and emotional prosody compared to NH individuals in relation to their higher behavioral performances [START_REF] Watson | People-selectivity, audio-visual integration and heteromodality in the superior temporal sulcus[END_REF].

Conclusions

Our results indicate that the technical limitations of the CI processor do not allow the patients to discriminate the subtle modulations of suprasegmental cues (pitch, intensity), which differentiate a question from a statement. Nonetheless, CI patients presented strong visuo-auditory multisensory facilitation with respect to the preferential modality when integrating visual and auditory linguistic prosodic information, a benefit which was nearly absent in normal hearing participants. The eye-tracking results obtained in this study further suggest that this proficiency for visuo-auditory integration relies on a specific oculomotor strategy when patients are integrating the visual prosodic cues of the face .

Authorship statement

Anne Lasfargues-Delannoy: Methodology, Investigation, Formal analysis, Data curation, Writing, Visualization.

Kuzma Strelnikov: Methodology, Formal analysis, Data curation, Writing, Visualization.

Mathieu Marx: Conceptualization, Methodology, Writing, Visualization, Supervision.

Pascal Barone: Conceptualization, Methodology, Funding acquisition, Project administration, Resources, Writing, Visualization, Supervision.

Olivier Deguine: Conceptualization, Methodology, Funding acquisition, Resources, Writing, Visualization Supervision.

Fig. 1 .

 1 Fig. 1. Stimuli presentation.A trial starts with a 1-second presentation of a fixation cross, followed by a 3 s display of the written sentence, followed by another fixation cross (1 s), then the video stimuli ( ∼3s) and the two choices possible (7s).

Fig. 3 .

 3 Fig. 3. Performance in identifying prosodic sentences in three conditions: visual, auditory, and visuo-auditory. The boxplots show that CIP and NH participants present similar A and AV condition performances while the CIP presents a significantly lower performance level in the V-only conditions. Insert presents the proportion of participants being more performant in the A-only (plain color) than in the V-only (hatched color) unimodal conditions.

Fig. 4 .

 4 Fig.4. Box Plot of the normalized visuo-auditory gain according to each individual's preferential unimodal condition. This analysis clearly shows that CIP presented a significantly higher multisensory gain compared to NH participants. In hearing controls, the gain was not significantly different from zero.

Fig. 5 .

 5 Fig. 5. Global heat map, irrelevant of condition for all three groups: retrieved through eye-tracking analysis. CIP (1st image on the left) mostly fixate the lower half of the face, mainly the mouth region. NH young (3rd image from the left) have a fixation pattern centered around the upper half of the face, focusing mainly on the eyes. NH age match participants (2 nd image from the left) demonstrate an intermediate fixation pattern to CIP and NHy as they fixate all three regions of interest).

Fig. 6 .

 6 Fig. 6. Fixation percentages irrespective of condition across three areas of interest. CIP (in blue) significantly fixate the mouth region more than NHy while NHy significantly fixate the eyes more than CIP. NHam participants present a similar pattern to CIP as they significantly fixate the mouth more than NHy. In contrast, NHy significantly fixate the eyes more than NHam . Error bars represent standard deviation.

Table 1

 1 Individual CI patient characteristics. The * represents all the patients that took part in the oculomotor eye-tracking analysis.

	Subject	Age	gender	Unilateral/bilateral CI	Length of exposure to oral	Duration of sensory	Duration Post CI	Dissyllabic word
		(years)			language since birth (months)	deprivation (months)	(months)	restitution (%)
	IC1	27	M	Bilateral CI	320	4	144	70
	IC2	41	F	Bilateral CI	468	24	132	95
	IC3	27	F	Bilateral CI	321	3	60	80
	IC4	73	M	Unilateral left CI	874	1	276	85
	IC5	73	M	Unilateral left CI	875	0	192	75
	IC6 *	60	F	Bilateral CI	540	180	30	90
	IC7 *	54	M	Bilateral CI	528	120	16	90
	IC8	81	F	Unilateral right CI	918	54	132	70
	IC9	66	M	Unilateral right CI	780	12	48	60
	IC10 *	54	F	Unilateral left CI	608	40	72	80
	IC11 *	59	F	Unilateral right CI	NA	Unknown	24	60
	IC12	70	M	Unilateral right CI	744	96	30	80
	IC13 *	61	F	Unilateral left CI	721	11	60	75
	IC14	55	M	Bilateral CI	660	0	30	80
	IC15 *	30	M	Unilateral right CI	359	1	156	80
	IC16	80	M	Unilateral left CI	NA	Unknown	36	80
	IC17 *	59	F	Bilateral CI	670	38	14	95
	IC18 *	60	M	Unilateral right CI	684	36	300	80
	IC19 *	69	F	Unilateral right CI	814	14	48	80

Acknowledgments

We would like to thank the CI patients and NH participants for their participation, S. Oudiz for participation in collecting data, S. De Gucht for technical help in setting the eye-tracking system, ML. Laborde, C. Algans, and M. Tartayre for their help in the recruitment process, and the ENT service for their support. We thanks the actors Caty Rota and Alain Dalmau for the prosodic sentences iteration.

This work was supported by ANR Archicore (ANR-14-CE13-0033-02) ANR VirtualHearing3D (ANR-16-CE17-0016-03), and the recurrent funding of the CNRS.

Supplementary materials

Supplementary material associated with this article can be found, in the online version, at doi: 10.1016/j.heares.2021.108330 .