Frédéric Aman
email: frederic.aman@imag.fr

Michel Vacher
email: michel.vacher@imag.fr

Solange Rossato
email: solange.rossato@imag.fr

François Portet
email: francois.portet@imag.fr

Analysing the Performance of Automatic Speech Recognition for Ageing Voice: Does it Correlate with Dependency Level?

Keywords: Ambient Assisted Living (AAL), Dependency, Elderly speech, Voice command

Ambient Assisted Living aims at providing assistance by allowing people with special needs to perform tasks which they have increasing difficulty with and to provide reassurance through surveillance in order to detect distress and accidental falls. Aged people are among the ones who might benefit from advances in ICT to live as long as possible in their own home. Voice-base smart home is a promising way to provide AAL, but even mature technologies must be evaluated from the perspective of its potential beneficiaries. In this paper, we investigate which characteristics of the ageing voice that challenge a state of the art ASR system. Though in the literature, chronological age is retained as the sole factor predicting decrease in performance, we show that degree of loss of autonomy is even more correlated to ASR performance.

INTRODUCTION

With advances in medicine, life expectancy has increased. However, this phenomenon coupled with a low birthrate has led to an ageing population in industrialised countries. To help elderly people to live as long as possible in their home, solutions have been developed based on robotics, automation, cognitive science, and computer networks. These solutions are being developed to compensate their possible physical or mental decline to keep them with a good degree of autonomy. The aim is to provide assistance by allowing them to perform tasks which they have increasing difficulty with and to provide reassurance through surveillance in order to detect distress and accidental falls. Such a system must allow independence of elderly while facilitating social contact, with a major impact on well-being and health. In addition, it helps caregivers and reassure relatives. However, technological solutions must be able to adapt to the needs and the specific capacities of this population. Indeed, elderly are often confused by complex interfaces of devices. Therefore, the usual interfaces (remote controls, mice or keyboards) must be complemented by more accessible and natural interfaces such as a system of Automatic Speech Recognition (ASR) [START_REF] Portet | Design and evaluation of a smart home voice interface for the elderly -Acceptability and objection aspects[END_REF].

In this context, the CIRDO project 1 wherein the authors take part aims to promote autonomy and support of elderly people by caregivers through a social inclusion product. The objective of the project is to integrate an ASR system into this product to perform detection of distress situations, distress calls and voice commands. Such kind of voice based interaction is an emerging feature of many AAL related research projects [START_REF] Wang | Robust environmental sound recognition for home automation[END_REF][START_REF] Jiang | Hermes: a FP7 funded project towards computer aided memory management via intelligent computations[END_REF][START_REF] Brdiczka | Detecting human behaviour models from multimodal observation in a smart home[END_REF][START_REF] Milhorat | Handsfree speech-sound interactions at home[END_REF][START_REF] Vacher | Speech Analysis for Ambient Assisted Living: Technical and User Design of a Vocal Order System[END_REF] but this remains a very challenging area due to the 1 http://liris.cnrs.fr/cirdo/ atypical nature of the application (distant speech, aged people, noise, uncontrolled area, multi-speaker, etc.) [START_REF] Vacher | Development of Audio Sensing Technology for Ambient Assisted Living: Applications and Challenges[END_REF].

One of the main challenges in this domain is to make sure that the ASR performance will be good enough to deliver a high quality voice order recognition system. This is a fear of the elderly population who are inclined to switch the system off if it has difficulties in understanding them. Most of the deployed ASR systems have reached a very good recognition rate in close, noise free talking, but their performances were rarely assessed with aged or children voice. A few studies compared ageing voice vs. non-ageing voice on ASR performance [START_REF] Baba | Acoustic models of the elderly for largevocabulary continuous speech recognition[END_REF][START_REF] Privat | Etude de l'effet du vieillissement sur les productions langagières et sur les performances en reconnaissance automatique de la parole[END_REF][START_REF] Vipperla | Longitudinal study of ASR performance on ageing voices[END_REF][START_REF] Pellegrini | Impact of Age in ASR for the Elderly: Preliminary Experiments in European Portuguese[END_REF], but their fields was quite far from our topic of home automation commands recognition. Moreover, an issue for our work was the non-existence of a speech corpus in French containing distress signals and automation commands.

The purpose of this study was to determine the impact of ageing voice on the ASR system performance and to find out which people characteristics might serve to predict ASR performance. The method we used is detailed in Section 3 after having discussed the related work in Section 2. Then the results of the evaluation are presented in Section 4 and an outlook on further work is given Section 5.

RELATED WORK

The perception of voice alteration with age has been the subject of many studies [START_REF] Gorham-Rowan | Acousticperceptual correlates of voice quality in elderly men and women[END_REF][START_REF] Benjamin | Frequency variability in the aged voice[END_REF][START_REF] Linville | Elderly listeners' estimates of vocal age in adult females[END_REF][START_REF] Morgan | Variability of voice fundamental frequency in elderly female speakers[END_REF][START_REF] Morris | Age-related differences in speech variability among women[END_REF]. Elderly speakers are characterized by tremors, hesitations, imprecise production of consonants, broken voice, and slower articulation [START_REF] Benjamin | Frequency variability in the aged voice[END_REF]. Regarding women, the changes seem partly due to an increase of the vocal cords mass due to some changing levels of certain hormones [START_REF] Honjo | Laryngoscopic and voice characteristics of aged persons[END_REF]. Regarding men, perception of gasp come from an incomplete closure of the vocal cords that would be compensated by an increasing tension in larynx [START_REF] Ryan | Perceptual and acoustic correlates in the speech of males[END_REF]. From the anatomical point of view, some studies have shown age-related degeneration with atrophy of vocal cords, calcification of laryngeal cartilages, and changes in muscles of larynx [START_REF] Takeda | Aging effects on motor units in the human thyroarytenoid muscle[END_REF][START_REF] Mueller | Acoustic and morphologic study of the senescent voice[END_REF].

Some studies have shown a significant increase in the standard deviation measures of the fundamental frequency of elderly, both men and women [START_REF] Xue | Effect on aging on selected acoustic voice parameters: Preliminary normative data and educational implications[END_REF][START_REF] Morris | Age-related differences in speech variability among women[END_REF][START_REF] Morgan | Variability of voice fundamental frequency in elderly female speakers[END_REF]. Stability of the fundamental frequency (F0) is reduced in elderly voice [START_REF] Gorham-Rowan | Acousticperceptual correlates of voice quality in elderly men and women[END_REF] and is associated with variability in the peak-to-peak amplitude of speech signal. Hesitations and gasping in pathological voices have been associated with increased noise in the speech signal driven by an aperiodic vibration of the vocal cords [START_REF] Eskenazi | Acoustic correlates of vocal quality[END_REF][START_REF] Selby | Perceptual and acoustic evaluation of individuals with laryngopharyngeal reflux pre-and post-treatment[END_REF]. Some measures of the ratio between noise energy and harmonics have quantified this phenomenon by comparing older and younger speakers [START_REF] Xue | Effect on aging on selected acoustic voice parameters: Preliminary normative data and educational implications[END_REF][START_REF] Ferrand | Harmonic-to-noise ratio: An index of vocal aging[END_REF]. Incomplete closure of vocal cords was observed during vocalisation [START_REF] Linville | Elderly listeners' estimates of vocal age in adult females[END_REF]. The study cited above [START_REF] Gorham-Rowan | Acousticperceptual correlates of voice quality in elderly men and women[END_REF] confirms fundamental frequency instabilities and the increasing noise on both sexes for healthy people with an average age of 70. These studies show that aged voice presents a much greater variability than typical voice. Ability of state-of-theart ASR systems to handle this kind of population can thus be questioned.

A more general study of Gorham-Rowan and Laures-Gore [START_REF] Gorham-Rowan | Acousticperceptual correlates of voice quality in elderly men and women[END_REF] highlights the effects of ageing on the speech utterance and the consequences on the speech recognition. The experiments carried out in automatic speech recognition have shown performance degradation for "atypical" population such as children or elderly people [START_REF] Wilpon | A study of speech recognition for children and the elderly[END_REF][START_REF] Vipperla | Longitudinal study of ASR performance on ageing voices[END_REF][START_REF] Gerosa | Towards ageindependent acoustic modeling[END_REF] and have shown the interests of an adaptation to the target populations [START_REF] Anderson | Recognition of elderly speech and voice-driven document retrieval[END_REF][START_REF] Gerosa | Towards ageindependent acoustic modeling[END_REF]. Speech recognition adapted to the voice of elderly people is still an underexplored area. The relevant languages are mainly English [START_REF] Vipperla | Longitudinal study of ASR performance on ageing voices[END_REF] and Asian languages such as Japanese [START_REF] Baba | Acoustic models of the elderly for largevocabulary continuous speech recognition[END_REF]. A very interesting study [START_REF] Vipperla | Longitudinal study of ASR performance on ageing voices[END_REF] used some recordings of speeches delivered in the Supreme Court of the United States over a decade. These recordings are particularly interesting because they were used to study the evolution of recognition performance on the same person depending on his age over 7-8 years. These studies show that the performance of recognition systems decreases steadily with age, and that a special adaptation to each speaker can get closer to the scores obtained from the youngest speakers without adaptation. The implicit consequence is that the recognition system is adapted to a single speaker. To make the system adapted to the person, Renouard et al. [START_REF] Renouard | Vocal interface with a speech memory for dependent people[END_REF] proposed to use the recognized words to adapt online the recognition models. Proposed in the context of home assistance, this research does not appear to have been pursued.

From an applicative point of view in the smart home context, speech recognition has been mainly implemented marginally in the field of voice commands in English. Indeed, most current studies use conventional sensors (presence sensor, door contact, etc.) and tactile interfaces (remote controls, handset, touch-screens) more reliable but less natural, offering fewer opportunities for interaction and comfort (for example: need to walk to reach the remote device). Among the advances in the field of voice controlled devices, a study conducted by Anderson [START_REF] Anderson | Recognition of elderly speech and voice-driven document retrieval[END_REF] showed that a voice interface which is adapted (models acquired on 300 elderly speakers) allows to make voice requests on computer with the same performance than a query typed at the keyboard. This study has also revealed that only 2 of the 37 participants preferred the keyboard compared to the voice interface. In the same field, Kumiko [START_REF] Kumiko | Input support for elderly people using speech recognition[END_REF] proposed a computer voice command interface that takes into account the possible sources of error (duration, intensity, vocabulary) to improve performance and feedback. While Interactive Voice Response is a pervasive component of today's telephone communication, some of which take into account the different voice population [START_REF] Pinto | Development of new telephone speech databases for french : the neologos project[END_REF], voice control in smart home is clearly in its infancy. A large number of issues, such as noisy environment, number of sound sources (for example: several people), vocabulary coverage, coverage of speakers, etc. still need to be addressed [START_REF] Vacher | Development of Audio Sensing Technology for Ambient Assisted Living: Applications and Challenges[END_REF]. Recently, Moir and Filho [START_REF] Moir | From science fiction to science fact: A smart-house interface using speech technology and a photo-realistic avatar[END_REF] proposed a low-coverage system using adaptive filters for a good recognition of keywords. But this research remains still exploratory.

To the best of our knowledge, no application of voice control in smart home has explicitly considered the problem of voice recognition of French elderly speakers, even though major advances in terms of ergonomics, safety and data acquisition with high semantic value can be made by this modality. From this short literature review, it can be emphasized that no study had considered French aged voice in smart home condition. Moreover, most studies considered the chronological age as global explanatory factor while many other effects can also be responsible for ASR performance degradation as raised by [START_REF] Pellegrini | Impact of Age in ASR for the Elderly: Preliminary Experiments in European Portuguese[END_REF]. There is thus no certainty that age can predict the reliability of a voice-based control system. That is why our study includes an evaluation from the dependence perspective.

METHOD

To assess the impact of the ageing voice on ASR performance, we started by acquiring a corpus targeted to the elderly population. From this corpus and a non-aged one, the first task was to identify the most problematic phonemes and to check whether standard adaptation can be employed to reduce the discrepancy between aged and non-aged speakers at phoneme level. Once adapted, the second task was to assess whether measures other than strictly chronological age can explain ASR performance degradation.

Corpus collection

The corpus collection was performed sporadically from 2009 to 2012 in collaboration with a rehabilitation centre, volunteers and a nursing home. Targeted speakers were persons aged of more than 60 years old, able to read and with no mental disorder or pathologies altering the voice. The recording was done with a single microphone positioned about 30 cm from the speaker's mouth. Most speakers were sat, but some were in a wheelchair or laying in a bed. The recording was done using a computer and a home made software to prompt sentences to be read by the speaker and to record the utterances using voice activity detection. Given the targeted application (in-home voice commands and distress calls) the participants were requested to read a list of short distress/home automation and casual sentences such as Aidez-moi (Help me) or Il fait beau (It's sunny). Based on [START_REF] Chaumon | Démarche de développement de technologies ambiantes pour le maintien à domicile des personnes dépendantes : vers une triangulation des méthodes et des approches[END_REF], who interviewed elderly people in nursing homes to identify and describe what situations of distress they could have experienced, we created a list of home automation orders the person could utter during a distress situation to request for assistance. Ten samples of each kind are given in Table 1.

The non-aged corpus was previously recorded in our laboratory in 2004 and was complemented in 2013 with sentences based on [START_REF] Chaumon | Démarche de développement de technologies ambiantes pour le maintien à domicile des personnes dépendantes : vers une triangulation des méthodes et des approches[END_REF]. The procedure was similar to the aged corpus acquisition.

This aged and non-aged corpus is called the AD corpus (Anodin-Détresse: anodin means colloquial and détresse means distress).

Finally, another aged corpus, the ERES38 corpus (Entretiens RESidences 38: Entretiens means interviews) was acquired for model adaptation purpose. This corpus was recorded in 2011 in the living place of the person. During the interviews, we requested each speaker to read a text but they were also asked to talk freely about their life. The text was an article about gardening created by the experimenters in order to target phoneme issues reported in [START_REF] Privat | Etude de l'effet du vieillissement sur les productions langagières et sur les performances en reconnaissance automatique de la parole[END_REF][START_REF] Aman | Etude de la performance des modèles acoustiques pour des voix de personnes âgées en vue de l'adaptation des systèmes de RAP (Assessment of the acoustic models performance in the ageing voice case for ASR system adaptation)[END_REF].

All the corpora were annotated at the sentence level using the Transcriber software.

ASR system

The ASR toolkit chosen in our study was Sphinx3 [START_REF] Seymore | The 1997 CMU Sphinx-3 English broadcast news transcription system[END_REF]. This decoder used a context-dependent acoustic model with 3-state left-to-right HMM. The acoustic vectors are composed of 13 MFCC coefficients, the delta and the delta delta of each coefficient. This HMM-based context-dependent acoustic model was trained on the BREF120 corpus [START_REF] Lamel | BREF, a large vocabulary spoken corpus for french[END_REF] French speakers. We called it the generic acoustic model.

Language model

A general language model (LM) was estimated from the French Gigaword corpus which is a archive of newswire text data that has been acquired over several years by the Linguistic Data Consortium (LDC) at the University of Pennsylvania 2 . It was 1-gram with 11018 words. Moreover, to reduce the linguistic variability, a 3-gram domain language model was learned from the sentences used during the corpus collection described in Section 3.1, with 88 1-gram, 193 2-gram and 223 3-gram models. Finally, the language model was a 3-gram-type which results from the combination of the general language model (with a 10% weight) and the domain one (with 90% weight). This combination has been shown as leading to the best WER for domain specific application [START_REF] Lecouteux | Distant Speech Recognition in a Smart Home: Comparison of Several Multisource ASRs in Realistic Conditions[END_REF]. The interest of such combination is to bias the recognition towards the domain LM but when the speaker deviates from the domain, the general LM makes it possible to correctly recognise the utterances.

Word error rate and phoneme matching

The simplest and most common way to evaluate ASR performances is to compute the Word Error Rate (WER). The WER is computed by first aligning the output (the decoded speech) with the reference (i.e., the ground truth) and then applying W ER = I+D+S N where I, D and S is the number of insertions, deletion and substitution of words and N is the number of words in the reference.

Though this measure was used in many related studies [START_REF] Baba | Acoustic models of the elderly for largevocabulary continuous speech recognition[END_REF][START_REF] Vipperla | Longitudinal study of ASR performance on ageing voices[END_REF][START_REF] Pellegrini | Impact of Age in ASR for the Elderly: Preliminary Experiments in European Portuguese[END_REF], it does not indicate which specific phonemes play a role in the ASR performance degradation. To do so, the annotation should be performed at the phoneme level. However, this is a very laborious and time-consuming task which furthermore requires a good level of expert agreement. That is why we analysed the results of the forced alignments. The forced alignment algorithm that was used is the one of Sphinx3.

Forced alignment consists in finding the boundaries of phonemes in an utterance knowing the uttered sentence. This sentence is mapped in phoneme (using a dictionary) which is used to constrain an optimal alignment between the acoustic model and the speech utterance. The forced alignment scores are for each signal segment within a boundary, the likelihood of belonging to a phoneme model. This score can be interpreted as a proximity to the "standard" pronunciation, modelled by the generic acoustic model. The differences in scores of phoneme categories between the aged group and the non-aged group allow to identify which phonemes are the most difficult for the ASR system. We are not aware of any study having used such method to assess ASR performances. Phonemes were grouped according to their highest level catgories as shown in table 2.

Adaptation with MLLR

Once the phonemes are identified, the most common method to overcome the ASR limitation is to apply speaker adaptation. Speaker adaptation consists in generating a new acoustic model from a generic one and some new annotated speech in limited quantity. One of the most popular technique is to apply the Maximum Likelihood Linear Regression (MLLR) which is particularly adapted when a limited amount of data per class is available. MLLR is an adaptation technique that uses small amounts of data to train a linear transform which warps the Gaussian means so as to maximize the likelihood of the data. The principle is that acoustically close classes are grouped and transformed together.

Assessing the level of autonomy

Despite the acoustic adaptation, there might be a disparity between the WERs of the elderly group even in aged people of the same age category. Therefore, we investigated other criteria and focused on elderly dependence. As reference, we used a French national test which is daily used in assessing the degree of loss of autonomy: the AGGIR (Autonomie Gérontolo-gie Groupes Iso-Ressources) grid 3 . The degree of autonomy loss is evaluated in terms of physical and cognitive decline. According to the result of this test, the person can receive financial support: the Personalized Allocation of Autonomy (APA). The evaluation is done using 17 variables. Ten variables refer to the loss of physical and cognitive autonomy: coherence, orientation, washing, feeding, disposal, transfers (to rise, to lie down, to sit down), internal displacement, external displacement and remote communication. Seven variables refer to the loss of domestic and social autonomy: personal management of budget and possessions, cooking, cleaning, transporting, purchasing, treatment monitoring and past time activities. Each variable is coded with A (independent), B (partially dependent) and C (totally dependent). The GIR (Iso-Ressources Group) score is computed from the variables to classify the person in one of the six groups: GIR 1 (total dependence) to GIR 6 (total autonomy).

RESULTS

Collected Corpus

The AD80 French test corpus

The AD corpus (cf. 3.1) was acquired from 95 speakers (36 men and 59 women) which are divided into two groups: the elderly group composed of 43 speakers (11 men and 32 women), 62 to 94 years old, with 2796 distress and home automation sentences for a duration of 1 hour 5 minutes, and 3006 casual sentences for a duration of 1 hour 6 minutes, and the non-elderly group composed of 52 speakers, 18 to 64 years old, with 3903 distress and home automation sentences for a duration of 1 hour 18 minutes, and 3897 casual sentences for a duration of 1 hour 12 minutes.

We fixed the limit of the non-aged group at 65 years old, but we recorded 2 people aged 62 and 63 years old with autonomy loss, looking very aged physically and living in nursing home. Thus we included this two persons, as exceptions, in the aged group.

For the 43 speakers of the aged AD corpus, a GIR score was obtained after clinicians filled the AGGIR grid.

Finally, the AD corpus is made up of 13,602 annotated sentences, with 4 hours and 42 minutes of recording.

Collection of the training ERES38 corpus

The ERES38 (cf. 3.1) corpus was acquired from 22 elderly people (14 women and 8 men) between 68 and 98 years old. The corpus included 48 minutes of read speeches (around 2 minutes per speaker) and 17 hours of interviews. The speakers lived in specialized institutes, such as nursing homes and were cognitively intact without severe disabilities.

Phoneme distance between aged and non-aged voice

When performing ASR using the generic acoustic model on the distress/home automation sentences of the AD corpus, we obtained an average WER of 9.07% for the non-elderly group, and an average WER of 43.47% for the elderly group. Thus, we observed a significant performance degradation of ASR for elderly speech, with an absolute difference of 34.40%. Figure 1 represents the WER according to the chronological age for both groups. It shows that the WER is globally higher for elderly group as previous studies showed [START_REF] Baba | Acoustic models of the elderly for largevocabulary continuous speech recognition[END_REF][START_REF] Vipperla | Longitudinal study of ASR performance on ageing voices[END_REF][START_REF] Pellegrini | Impact of Age in ASR for the Elderly: Preliminary Experiments in European Portuguese[END_REF]. However, it can also be seen that the variability between speakers also increases 3 http://vosdroits.service-public.fr/F1229.xhtml with the age. For instance, some 83 years old speakers have their WER ranging from 13.6% to 80.2%. Standard deviation is 6% for the non-elderly group and 17.27% for the elderly group. In other words, the WER is far less predictable in the elderly group than in the non-elderly group. Consequently, we have to deal with the fact that a speech recognition with such a system can work very well with some of the elderly speakers, and very badly with others. The forced alignment scores on both AD groups nonelderly and elderly with the generic acoustic model are presented in Figure 2 based on phonemic categories. The relative differences of forced alignment scores observed between both groups led to sort the phoneme categories in descending order of differences: nasal vowels (-100,34%), voiced plosives (-56,55%), unvoiced fricatives (-50,48%), unvoiced plosives (-44,05%), nasal consonants (-41,03%), open vowels (-37,12%), central vowels (-34,80%), voiced fricatives (-31,30%), back vowels (-29,26%), semi-vowels (-29,18%), liquids (-19,99%), and front vowels (-11,89%). The repartition of French phonemes inside the different groups are presented in Table 2.

For the elderly group, the alignment scores are lower than those obtained for the non-elderly group especially for plosives and nasal vowels. Based on the relative differences, the phoneme categories most affected for elderly group are nasal vowels, plosive consonants, unvoiced fricatives and nasal consonants. WER aged group before adaptation (%) WER aged group after adaptation (%) Figure 3: WER on aged group before and after adaptation Figure 3 shows that using the MLLR adapted acoustic model was able to reduce the WER significantly for all speakers of the AD corpus. With the global MLLR adaptation using ERES38, the average WER was 14.52%. Compared to the 43.47% WER without adaptation (see Section 4.2), the absolute difference was -28.95%. Furthermore, the speaker with the worst performance had his error rate reduced from 80.2% to 44.9%, and the speaker with the best performance had his error rate reduced from 9% to 1.8%. Also, the standard deviation was reduced from 17.27% to 10.34%, showing a reduction of the variability between the speakers.

Impact of the acoustic adaptation on ASR performance with aged voice

A comparison between the forced alignment scores obtained for non-elderly without adaptation and for elderly after adaptation using the ERES38 corpus is shown in Figure 4. On the whole, the scores for the elderly after adaptation are better than those of non-elderly with the generic acoustical model. Indeed, the use of an acoustical model adapted to elderly people reduces the mismatching of phonemes. The alignment scores of the adapted model presented in Figure 4 show that the average distance has reduced below the non-aged one for all phonemes except for unvoiced fricatives and nasal vowels.

From an applicative point of view, this test shows that we can use a database of elderly speech in MLLR adaptation with speakers which are different from the test database. Even though the size of the corpus is small, we have a significant improvement of WER. Furthermore, this demonstrates that the voices of ageing people have common characteristics.

Influence of elderly dependence on ASR system

Despite the acoustic adaptation, there is a great variability between the WERs of the elderly group. Therefore, we investigated to establish if the level of elderly dependence can be an indicator of the ASR performance for the elderly group. Figure 5 shows a box-and-whisker diagram of the WER from MLLR adaptation as a function of the elderly dependence. Four speakers were in GIR 2, two speakers were in GIR 3, 21 speakers were in GIR 4, one speaker was in GIR 5 and 15 speakers were in GIR 6. No speaker was represented in GIR 1. Due to the small number of speakers in GIR 2, GIR 3 and GIR 5, we merged GIR 2 with GIR 3 and GIR 4 with GIR 5. From Figure 5 it can be seen that WERs are different according to the GIR category. Indeed, the WER averages for GIR 2-3, GIR 4-5 and GIR 6 are respectively 25.2%, 13.2% and 12.2%, and the WER standard deviations are respectively 16.8%, 8.4% and 7.6%. Then, we performed an ANOVA test on the groups GIR 2-3, GIR 4-5 and GIR 6. From this test, the GIR score have a significant effect on WER (F (2, 40) = 4.3; p < 0.05%).

We conducted a Bonferroni post-hoc analysis to characterize which groups were significantly different from other groups. The post-hoc test highlighted that there was a significant difference between the GIR 2-3 group and both groups GIR 4-5 and GIR 6, while there is no significant difference between GIR 4-5 and GIR-6.

CONCLUSION

The paper presents our study on the behavior of an ASR system with elderly voices. Given the absence of a corpus containing the voice of elderly in French language usable for testing ASR system, we recorded the AD corpus. From this corpus, we observed an increase of the average WER of the ASR system for elderly people, with an absolute difference between non-elderly and elderly voice of 34.4%. With forced alignment, we analyzed which phonemes for elderly speech were posing the most problems to ASR systems. These results allowed us to proceed to the recording of the ERES38 corpus, allowing us to adapt the generic acoustic model to the voice of elderly people through the MLLR adaptation method. The global MLLR adaptation was interesting because with less than one hour of recordings from speakers different from the test speakers, we obtained a WER close to the case of recognition with the generic acoustic model on non-elderly group, with a WER of 14.53%, against 43.47% before adaptation. Moreover, we showed that inside the elderly group, the WER was not correlated with the age but could be correlated with the level of dependence due to a general physical degradation. The continuation of our work would be to show how the different parameters of the AGGIR grid are correlated to the WER. Therefore, predicting the ASR behavior would allow in facilitating the use of these new technologies in the daily life of the dependent elderly people.

ACKNOWLEDGMENT

This study was funded by the National Agency for Research under the project CIRDO -Industrial Research (ANR-2010-TECS-012). The authors would like to thank to Mrs Vézignol and Bonnefond-Jimenez, Mr Debrus, Mrs Aman, Bron, Lalande and Martins of the medical institutions SSR "Les Cadières" and EHPAD "Château de Labahou" for their help in corpus recording. Special thanks to R. Dugheanu, J. Le Grand and Y. Sasa for their active contribution, and to various elderly and caregivers who agreed to participate in the recordings.

Figure 1 :

 1 Figure 1: WER as a function of age for aged and non-aged groups

Figure 2 :

 2 Figure 2: Forced alignment scores by phonemes categories before adaptation

Figure 5 :

 5 Figure 5: WER as a function of levels of dependence

Table 1 :

 1 which is composed of about 100 hours of annotated speech from 120 non-elderly Examples of sentences of the AD corpus

	Sample Distress Sentence	Home Automation Order	Casual Sentence
	1	Aidez-moi !	e-lio appelle le samu !	Bonjour madame !
	2	Au secours !	e-lio appelle les pompiers !	Ça va très bien.
	4	Je me sens mal !	e-lio appelle les secours !	Ce livre est intéressant.
	5	Je suis tombé !	e-lio appelle un docteur !	Il fait soleil.
	3	Du secours s'il vous plaît !	e-lio appelle une ambulance !	J'ai ouvert la porte.
	6	Je ne peux plus bouger !	e-lio appelle une infirmière !	Je dois prendre mon médicament !
	7	Je ne suis pas bien !	e-lio appelle ma fille !	J'allume la lumière !
	8	Je suis blessé !	e-lio appelle mon fils !	Je me suis endormi tout de suite !
	9	Je ne peux pas me relever !	e-lio tu peux téléphoner au samu ? Le café est brûlant !
	10	Ma jambe ne me porte plus ! e-lio il faut appeler les secours !	Où sont mes lunettes ?

Table 2 :

 2 Phoneme categories (IPA symbols)

	2 http://www.ldc.upenn.edu/Catalog/CatalogEntry.jsp?catalogId=
	LDC2006T17


Michel Vacher

Benjamin Lecouteux

Dan Istrate

Thierry Joubert
email: thierry.joubert@theoris.fr

François Portet
email: francois.portet@imag.frdan.istrate@esigetel.fr

Mohamed Sehili
email: mohamed.sehili@esigetel.fr

Pedro Chahuara

Experimental Evaluation of Speech Recognition Technologies for Voice-based Home Automation Control in a Smart Home

Keywords: Real-time audio analysis, experimental in-situ evaluation, Smart Home, Ambient Assisted Living

This paper presents an audio-based interaction technology that lets the user have full control over her home environment and at detecting distress situations for the elderly and frail population. We introduce the PATSH framework which performs real-time recognition of voice commands anywhere in the home and detail its architecture and the state-of-the-art processing technologies it employs. This system was evaluated in a realistic Smart Home with three user groups: seniors, visually impaired people and people with no special needs. Results showed the validity of the PATSH approach and shed light on its usability for people with special needs.

Introduction

Due to the demographic change and ageing in developed countries, the number of older persons is steadily increasing. In this situation, the society must find solutions to allow these people to live in their home as comfortably and safely as possible by assisting them in their daily life. This concept, known as Ambient Assisted Living (AAL) aims at anticipating and responding to the special needs of these persons. In this domain, the development of Smart homes and intelligent companions is seen as a promising way of achieving in-home daily assistance [START_REF] Chan | A review of smart homes-present state and future challenges[END_REF]. However, given the diverse profiles of the senior population (e.g., low/high technical skill, disabilities, etc.), complex interfaces should be avoided. Nowadays, one of the best interfaces seems to be the speech interface, that makes possible interaction using natural language so that the user does not have to learn complex computing procedures or jargon. Moreover, it is well adapted to people with reduced mobility and to some emergency situations because the user doesn't need to be close to a switch ("hands free" system). Despite all this, very few Smart Home projects have seriously considered speech recognition in their design [START_REF] Istrate | Information extraction from sound for medical telemonitoring[END_REF][START_REF] Charalampos | Enabling human status awareness in assistive environments based on advanced sound and motion data classification[END_REF][START_REF] Popescu | An acoustic fall detector system that uses sound height information to reduce the false alarm rate[END_REF][START_REF] Badii | CompanionAble -integrated cognitive assistive & domotic companion robotic systems for ability & security[END_REF][START_REF] Hamill | Development of an automated speech recognition interface for personal emergency response systems[END_REF][START_REF] Filho | From science fiction to science fact: a smart-house interface using speech technology and a photorealistic avatar[END_REF][START_REF] Lecouteux | Distant Speech Recognition in a Smart Home: Comparison of Several Multisource ASRs in Realistic Conditions[END_REF]. Part of this can be attributed to the complexity of setting up this technology in a real environment and to important challenges that still need to be overcome [START_REF] Vacher | Development of Audio Sensing Technology for Ambient Assisted Living: Applications and Challenges[END_REF].

In order to make in home voice control a success and a benefit for people with special needs, we argue that a complete framework for audio analysis in Smart Home must be designed. This framework should be able to provide real-time response, to analyse concurrently several audio channels, to detect audio events, to filter out noise and to perform robust distant speech recognition. Furthermore, in contrast with current triggered-bybutton ASR systems commonly found in smart phone, this voice control should be able to work in an "hand free" manner in case the person is not able to move. Another important aspect is the respect for privacy: the system should not disseminate any raw personal data outside the home without the user's consent. Our approach, called PATSH is a step toward these goals. The originality of the approach is to consider these problems together while they have mostly been studied separately.

To the best of our knowledge, the main trends in audio technology in Smart Homes are related to augmented human machine interaction (e.g., voice command, conversation) and security (mainly fall detection and distress situation recognition). Regarding security, the main application is the fall detection using the signal of a wearable microphone which is often fused with other modalities (e.g., accelerometer) [START_REF] Popescu | An acoustic fall detector system that uses sound height information to reduce the false alarm rate[END_REF][START_REF] Charalampos | Enabling human status awareness in assistive environments based on advanced sound and motion data classification[END_REF]. However, the person is constrained to wear these sensors at all times. To address this constraint, the dialogue system developed by [START_REF] Hamill | Development of an automated speech recognition interface for personal emergency response systems[END_REF] was proposed to replace traditional emergency systems that requires too much change in the lifestyle of the elders. However, the prototype had a limited vocabulary (yes/no dialogue), was not tested with aged users and there is no mention about how the noise was taken into account. Most of the speech related research or industrial projects in AAL are actual highly focused on dialogue to build communicative agent (e.g., see the EU funded Companions or CompanionAble projects or the Semvox system 1 ). These systems are often composed of ASR, NLU, Dialogue management and TTS parts supplying the user the ability to communicate with the system in an interactive fashion. However, it is generally the dialogue module (management, modelling, architecture, personalization, etc.) that is the main focus of these projects (e.g., see Companions, OwlSpeak or Jaspis). Moreover, this setting is different from the Smart Home one as the user must be close to the avatar to speak (i.e., not a distant speech setting). In [START_REF] Filho | From science fiction to science fact: a smart-house interface using speech technology and a photorealistic avatar[END_REF], a communicative avatar was designed to interact with a person in a smart office. In this research, enhanced speech recognition is performed using beamforming and a geometric area of recording. But this promising research is still to be tested in a multiroom and multisource realistic home.

Designing and applying speech interfaces in Smart Home to provide security reassurance and natural man-machine interaction is the aim of the SWEET-HOME 2 project. With respect to this short state-of-the-art, the project addresses the important issues of distant voice command recognition and sound source identification. The outcomes of this research are of high importance to improve the robustness of the systems mentioned above. In this paper, we introduce the PATSH system which perform the real-time identification of the voice command anywhere in the home. Its architecture and the state-of-the-art processing technologies employed are detailed in Section 2. This system was evaluated in a realistic Smart Home with three user groups: people with no special needs, seniors and, visually impaired people. These experiments are summarised in Section 3. PATSH was used on-line (vs. off-line) during the experiment, these results are analysed in Section 4. The paper finishes with a short outlook of future work.

The Audio Analysis System

The SWEET-HOME system is composed of an Intelligent Controller which analyses the streams of data and makes decision based on these. This framework acquires data from sensors and interprets them, by means of IA techniques, to provide contextual information for decision making. The description of this intelligent controller is out of the scope of the paper, the reader is thus referred to [START_REF] Chahuara | Context aware decision system in a smart home : knowledge representation and decision making using uncertain contextual information[END_REF] for further details. This system uses a two-level ontology to represent the different concepts handled during the processing which also contains SWRL instances to automatise some of the reasoning. An important aspect is the relationship between the knowledge representation and the decision process which uses a dedicated Markov Logic Network approach to benefit from the formal logical definition of decision rules as well as the ability to handle uncertain facts inferred from real data. The location of the inhabitant was determined by the intelligent controller that analysed continuously the data stream of the smart-home (not only audio) and made decisions based on the recognized voice commands and this contextual information.

Therefore, the streams of data are composed of all the usual home automation data sensors (switches, lights, blinds, etc.), multimedia control (uPnP), and the audio events processed in real-time by the multi-channel audio analysis system: PATSH. Indeed, this section describes the overall architecture of PATSH, details the sound/speech discrimination and the ASR part.

PATSH framework

The global architecture of PATSH is illustrated in Figure 1. The PATSH framework is developed with the .Net cross platform technology. The main data structure is the Sound object, which contains a segment of the multidimensional audio signal whose interpretation is continuously refined during the processing pipeline. PATSH deals with the distribution of the data among the several plugins that perform the processing to interpret the audio events. The execution can be done, in parallel, synchronously or asynchronously, depending on the settings stored in a simple configuration file. In SWEET-HOME, the plugins were actually developed in C or C++ and PATSH includes the mechanism to transfer sound events from the plugins to the PATSH framework and vice-versa.

In the SWEET-HOME configuration, PATSH runs plugins that perform the following tasks: The PATSH framework was developed to process on-line sound objects continuously detected on the 7 audio channels. However, it exists a bottleneck between the acquisition task and the event processing task. Given that one sound event can be simultaneously detected by several channels, the amount of the sound events in the queue can quickly rise.

Sound Event Detection

The detection of the occurrence of an audio event is based on the change of energy level of the 3 highest frequency coefficients of the Discrete Wavelet Transform (DWT) in a sliding window frame (last 2048 samples without overlapping). Each time the energy on a channel goes beyond a self-adaptive threshold, an audio event is detected until the energy decrease below this level for at least an imposed duration [START_REF] Istrate | Information extraction from sound for medical telemonitoring[END_REF]. At the end of the detection, the Signal to Noise Ratio (SNR) is computed by dividing the energy in the event interval and the previous energy in a window outside this interval. This process is operated on each channel independently.

Sound/Speech Discrimination

Once sound occurrences are detected, the most important task is to distinguish speech from other sounds. In everyday life, there is a large number of different sounds, modelling all of them is irrelevant. For the SWEET-HOME project, distant voice command and distress situation detection, speech is the most important sound class. The method used for speech/sound discrimination is a GMM (Gaussian Mixture Models) classification.

The Sound/Speech Discrimination stage has a very important role: firstly, vocal orders must not be missed, secondly, daily living sounds must not be sent to the ASR because undesirable sentences could be recognized. To recognize only vocal orders and not all sentences uttered in the flat, all sound events shorter than 150 ms and longer than 2.2 seconds were ignored as well as those whose SNR is below 0 dB. These values were chosen after a statistical study on our data bases.

Voice order recognition

In a Smart Home, the microphones are generally set in the ceiling and on the wall. This places the study in a distantspeech context where microphones may be far from the speaker and may record different noise sources. Moreover, the application calls for quick decoding so that voice commands are sent as soon as possible to the intelligent controller. This is why we used the Speeral tool-kit [START_REF] Linarès | The LIA speech recognition system: from 10xRT to 1xRT[END_REF] (Laboratoire d'Informatique d'Avignon). Indeed, its 1xRT configuration allows a decoding time similar to the signal duration. Speeral relies on an A * decoder with HMM-based context-dependent acoustic models and trigram language models. HMMs are classical three-state left-right models and state tying is achieved by using decision trees. Acoustic vectors are composed of 12 PLP (Perceptual Linear Predictive) coefficients, the energy, and the first and second order derivatives of these 13 parameters.

The acoustic models of the ASR system were trained on about 80 hours of annotated speech. Furthermore, acoustic models were adapted to the speech of 23 speakers recorded in the same flat during previous experiments by using Maximum Likelihood Linear Regression (MLLR) [START_REF] Lecouteux | Distant Speech Recognition in a Smart Home: Comparison of Several Multisource ASRs in Realistic Conditions[END_REF]. A 3-gram Language Model (LM) with a 10K lexicon was used. It results from the interpolation of a generic LM (weight 10%) and a domain LM (weight 90%). The generic LM was estimated on about 1000M of words from the French newspapers Le Monde and Gigaword. The domain LM was trained on the sentences generated using the grammar of the application (see Fig. 3). The LM combination biases the decoding towards the domain LM but still allows decoding of out-of-domain sentences. A probabilistic model was preferred over using strictly the grammar because it makes it possible to use uncertain hypotheses in a fusion process for more robustness.

Experiments in real conditions

Experimental flat

Experiments were run in the DOMUS smart home. Figure 2 shows the details of the flat. It is a thirty square meters suite flat including a bathroom, a kitchen, a bedroom and a study, all equipped with 150 (konnex) KNX sensors and actuators. The flat has been equipped with 7 radio microphones set in the ceiling for audio analysis. A specialized communication device, e-lio, from the Technosens company was used to initiate a communication between the user and a relative.

Voice orders

Possible voice orders were defined using a very simple grammar as shown on Figure 3. Each order belongs to one of three categories: initiate command, stop command and emergency call. Except for the emergency call, every command starts with a unique key-word that permits to know whether the person is The grammar was built after a user study that showed that targeted users would prefer precise short sentences over more natural long sentences [START_REF] Portet | Design and evaluation of a smart home voice interface for the elderly: acceptability and objection aspects[END_REF]. In this study, although most of the older people spontaneously controlled the home by uttering sentences, the majority said they wanted to control the home using keywords. They believe that this mode of interaction would be the quickest and the most efficient. This study also showed that they also had tendency to prefer or to accept the 'tu' form (informal in French) to communicate with the system given this system would be their property.

Scenarios and experiments

To validate the system in realistic conditions, we built scenarios in which every participant was asked to perform the following activities: (1) Sleeping; (2) Resting: listening to the radio; (3) Feeding: preparing and having a meal; and (4) Communicating: having a talk with a remote person thanks to e-lio. Therefore, this experiment allowed us to process realistic and representative audio events in conditions which are directly linked to usual daily living activities. Moreover, to evaluate the decision making, some specific situations were planned in the scenarios. For instance, for the decision regarding the activation of the light, given that the bedroom had two lights (the ceiling and the bedside one) as well as the kitchen (above the dinning table and above the sink), the four following situations were planed:

1. Situation 1. The person is having a meal on the kitchen table. The most appropriate light is the one above the table.

2. Situation 2. The person is cleaning up the bedroom. The most appropriate light is the ceiling one.

3. Situation 3. The person is cleaning the sink and doing the dishes. The most appropriate light is the one above the sink.

4. Situation 4. The person has just finished a nap. The most appropriate light is the bedside one.

Each participant had to use vocal orders to make the light on or off, open or close blinds, ask about temperature and ask to call his or her relative. The instruction was given to the participant to repeat the order up to 3 times in case of failure. In case of, a wizard of Oz was used in case of persistent problem.

Sixteen participants (including 7 women) without special needs were asked to perform the scenarios without condition on the duration. A visit, before the experiment, was organized to ensure that the participants will find all the items necessary to perform the scenarios. It was necessary to explain the right way to utter vocal orders and to use the e-lio system. Before the experiment, the participant was asked to read a text of 25 short sentences in order to adapt the acoustic models of the ASR for future experiments. The average age of the participants was 38 years (19-62, min-max) and the experiment lasted between 23min and 48min. The scenario includes at least 15 vocal orders for each participant but more sentences were uttered because of repetitions.

Acquired Corpus

During the experiment, audio data were recorded and saved in two ways. Firstly, the 7-channel raw audio signal was stored for each participant to make subsequent analysis possible. In total, 8h 52min 36s of data was recorded for the 16 participants. Secondly, the individual sound events automatically detected by PATSH were recorded to study the performances of this framework.

Apart from daily living sounds and sentences uttered in the flat by the participant, PATSH also detected the system messages (vocal synthesizer) and the e-lio communications. Overall, 4595 audio events were detected whose 993 were speech and 3503 were other noise occurrences. The number of events corresponding to each category -speech or everyday living sound-is displayed Table 1. In this study, we are only interested in recognizing vocal orders or distress sentences. All other spontaneous sentences and system messages are not irrelevant. Therefore, the global audio records were annotated using Transcriber in order to extract the syntactically correct vocal orders, results are shown in Table 2. The average SNR and duration are 15.8dB and 1s, this SNR value is low compared to studio conditions (SNR≥35dB). As the home automation system needs only one correct sentence to interact, only the less noisy channel was kept. The number of vocal orders is different for each speaker because if a vocal order was not correctly recognized, the requested action was not operated by the intelligent controller (light on or off, curtains up or down. . . ) and thus the speaker often uttered the order two or three times. Thanks to this annotation, an oracle corpus was extracted. The comparison between experimental real-time results with thus obtained with the same ASR on the oracle corpus will allow to analyse the performance of the PATSH system.

Results

Discrimination between speech and sounds

The detection part of the system is not specifically evaluated because of the lack of time to label all the sound events on the 7 channels. However, all the results presented take into account the performances of the detection because the signals are extracted automatically by the system. The sound/speech discrimination misclassified 108 sound and 232 speech occurrences which gives a total error rate of about 7.4% which is in line with other results of the literature [START_REF] Sehili | Sound Environnment Analysis in Smart Home[END_REF]. 23.4% of speech occurrences were classified as sound. Theses poor performances are explained by the fact that PATSH was not successful in selecting the best audio event among the set of simultaneous events and thus the events with low SNR introduced errors and were not properly discriminated. For the sounds, 3.1% of sound occurencies are classified as speech. Sounds such as dishes, water flow or electric motor were often confused with speech. For instance, when certain persons stirred the coffee and chocked the spoon on the cup or when they chocked plates and cutlery, the emitted sounds had resonant frequencies very close to the speech one. This is emphasizing the difficulty of the task and models must be improved to handle these problematic samples.

Home automation order recognition

The global performance of the system is directly related to vocal order recognition. The DER (Domotic Error Rate) is shown in Table 3, the 2 nd and 5 th columns "Expe." indicates the results for the real-time experiment. This error rate is evaluated after filtering at the input of the intelligent controller and includes the global effects of all stages: detection, discrimination between speech and sound, ASR. When the uttered voice orders were not respecting the grammar, for example when a sentence such as "Nestor heure" is uttered instead of the command "Nestor donne l'heure", these utterances were discarded. Moreover, some speakers' utterances exceeded the 2.2s duration threshold because of their hesitation, therefore corresponding vocal orders were not analysed and considered as missed.

In case of music in the room, vocal orders were often longer because of the mixing between speech and music. Consequently future experiments will need to set the threshold to 2.5s and to include a short training step to allow the participant to become familiar with this technology.

The ASR system used generic acoustic models without adaptation to the speaker and then regional or foreigner accent may have an influence: it's in particular the case for S10 (Arabic) and S14 (Alsatian). The participants S07 and S15 show low performance because they were not able to follow the given instructions, the presence of large part of silence mixed with noise between the words is analysed as phoneme and therefore increases the error rate. Part of the errors was due to the way PATSH managed simultaneous detections of one sound event. At this time of the process, the SNR is not known with a sufficient precision and the choice is not perfect. Then, in some cases, a part of the speech signal is missed (beginning or end of the order) and this introduces a bad recognition. Moreover, very often the detection is not perfectly simultaneous and more than one channel is analysed by the ASR. Therefore, some improvement were introduced in PATSH for future experiments: that consisted in making the decision after the end of detection on the 7 threads (each thread corresponding to one channel) thanks to a filtering window of 500ms. The disadvantage is that the system is slowed down with a delay of 500ms but this will avoid the recognition of bad extracted sentences and this is compensated by the analysis of only the signal of the best channel.

An important aspect is the decoding time because the device must be activated with a delay as short as possible. In this experiment, the decoding times reached up to 4 seconds which was a clear obstacle for usage in real condition. Hopeful, this has been reduced.

Preliminary Results from experiments with the aged and visually impaired population

The method has also been applied in the same context but with aged and visually impaired people. The aim was both to validate the technology with this specific population and to perform a user study to assess the adequacy of this technology with the targeted users and to compare with the other user studies of the literature [START_REF] López-Cózar | Multimodal dialogue for ambient intelligence and smart environments[END_REF][START_REF] Portet | Design and evaluation of a smart home voice interface for the elderly: acceptability and objection aspects[END_REF].

Between the two experiments, several corrections were applied to PATSH so that the sound/speech discrimination was greatly improved as well as the speech decoding time. The measured decoding time was 1.47 times the sentence duration; as the average duration of a vocal order was 1.048s, the delay between the end of the utterance and the execution of the order was 1.55s. This is still not a satisfactory delay but this does not prevent usage in real conditions.

Experimental set up

In this experiment, eleven participants either aged (6 women) or visually impaired (2 women, 3 men) were recruited. The average age was 72 years (49-91, min-max). The aged persons were fully autonomous but were living alone. The participants were asked to perform 4 scenarios involving daily living activities and distress or risky situations.

1. The participant is eating her breakfast and is preparing to go out. She asked the house to turn on the light, close the blinds or ask for the temperature while doing these activities.

2. The participant is coming back from shopping and is going to have a nap. She asked the same kind of commands but in this case, a warning situation alerts about the front door not being locked.

3. The participant is going to the study to communicate with one relative through the dedicated e-lio system. After the communication, the participant simulates a sudden weakness and call for help.

4. The participant is waiting in the study for friends going to visit her. She tests various voice orders with the radio, lights and blinds.

During this experiment, 4 hours and 39 minutes of data was collected including the same sensors as the one previously described in Section 3.4.

First feedbacks

All the participants went through a questionnaire and a debriefing after the experiment. We are still in the process of analysing the results but overall, none of the aged or visually impaired persons had any difficulty in performing the experiment. They all appreciated to control the house by voice.

It is worth emphasizing that aged people preferred the manual interaction because this was quicker. However, they liked the voice warning in case of risky situations. Regarding the visually impaired participants, they found that the voice command would be more adequate if it could enable performing more complex or dangerous tasks than controlling blinds or radio. For instance, by enabling them to use the household appliances. Overall, half of the participants found the system adapted for their use.

Discussion

Overall, the performance of the system was still low but the results showed there is room for improvement. Sound/Speech discrimination has been improved since the beginning of the experiment and continue to be improved. The biggest problems were the response time which was unsatisfactory (for 6 participants out of 16) and the mis-understanding of the system which implied to repeat the order (8/16). These technical limitations were reduced when we improved the ASR memory management and reduced the search space. After this improvement, only one participant with special needs complained about the response time. None of the encountered problem challenged the PATSH architecture. That is why we are studying the possibility of releasing the code publicly.

The grammar was not the focus of the project but it has been built to be easily adaptable at the word level (for instance, if someone wants to change "Nestor" for another word). All the 16 participants found the grammar easy to learn. Only four of them found the keyword "Nestor" unnatural while the others found it natural and funny. However, this approach suffers a lack of natural adaptivity to the user's preferences, capacities and culture as any change would require technical intervention.

For instance, [START_REF] Gödde | Study of a speech-based smart home system with older users[END_REF] emphasized that elder Germans tend to utter longer and politer commands than their fellow countrymen which contrast with our findings. Despite longitudinal studies are require to understand human preferences regarding voice orders, methods to adapt on-line the grammar to the user must be developed.

The acquired corpus made it possible to evaluate the performance of the audio analysis software. But interest goes far beyond this experiment because it constitutes a precious resource for future work. Indeed, one of the main problems that impede researches in this domain is the need for a large amount of annotated data (for analysis, machine learning and reference for comparison). The acquisition of such datasets is highly expensive both in terms of material and of human resources. For instance, in a previous experiment involving 21 participants in the DOMUS smart home, the acquisition and the annotation of a 33-hour corpus has cost approximatively 70ke. Thus, making these datasets available to the research community is highly desirable. This is why we are studying the possibility of making part of it available to the society as we did in our previous project [START_REF] Fleury | A french corpus of audio and multimodal interactions in a health smart home[END_REF].

Conclusion

This paper presents the PATSH system, the audio processing module of the voice controlled SWEET-HOME system which performs real-time identification of voice commands in the home for assisted living. In this system, the identified sound events are sent to an intelligent controller for final context-aware decision about the action to make on the house [START_REF] Vacher | The SWEET-HOME project: Audio processing and decision making in smart home to improve well-being and reliance[END_REF]. The experiments made in the Smart Home to evaluate the system showed promising results and validate the approach. This technology can benefit both the disabled and the elderly population that have difficulties in moving or seeing and want security reassurance.

Our application of this technology within a realistic Smart Home, showed that one of the most sensible tasks is the speech/noise discrimination [START_REF] Vacher | Development of Audio Sensing Technology for Ambient Assisted Living: Applications and Challenges[END_REF]. According to the SNR level, the performance can be quite poor, which has side effects on both the ASR and the sound classification (and then on the decision making). Another issue is linked to the lack of handling of simultaneous sound event records. These fill the sound object queue, which is the system bottleneck, and thus slow down the processing while real-time performances are required. To increase the performance and free this bottleneck, we had implemented a filtering strategy to remove low SNR audio events as well as too delayed events. The preliminary results showed a significant increase in performance. In a second step, PATSH will be modified to allow in real-time a multisource ASR thanks to the Driven Decoding Algorithm [START_REF] Vacher | Recognition of voice commands by multisource ASR and noise cancellation in a smart home environment[END_REF].

Although the participants had to repeat, sometimes up to three times, the voice command, they were overall very excited about commanding their own home by voice. We are still in the process of analysing the results of the experiment which included seniors and visually impaired people to get essential feedback from this targeted population. Future work will include improvements of the speech recognition in noisy environment and customisation of the grammar as well as experiments using specialised communication devices to enhance user's communication capacity.

Figure 1 :

 1 Figure 1: The PATSH architecture.

Figure 2 :

 2 Figure 2: Position of the microphones and other sensors inside the DOMUS smart home.

Figure 3 :

 3 Figure 3: Excerpt of the grammar of the voice orders (terminal symbols are in French)

  Presentation, communicating the sound event to the Intelligent Controller. If a vocal order is detected and according to the context (activity and localisation of the user in the flat), a home automation command is generated to make the light up, close the curtains or emit a warning message thanks to a voice synthesizer.

	2. Sound Detection and Extraction, detecting the start and
	end of sound events on each channel in parallel;
	3. Sound/Speech Discrimination, discriminating speech
	from other sounds to extract voice commands;
	4. Sound Classification, recognizing daily living sounds
	(not developed in this paper, see [13] for details);
	5. Automatic Speech Recognition (ASR), applying speech
	recognition to events classified as speech and extracting
	vocal orders; and
	6.
	1. Multichannel data Acquisition through the NI-
	DAQ6220E card. Seven channels are acquired at 16kHz
	(16 bits quantification);

  developed by the LIA

	PATSH								Work
	Capture N Channels Audio source	Object Work	Work Object Work Object Object Work		Sound Event Processing Work Object	Object ASR		Work Object
		Acquisition	Detection		Sound/Speech Discrimination		Sound Classification	Presentation
	transfer	write back	transfer	fill	transfer	write back	transfer	back write	transfer	write back
	Sound Object		Object Sound	FIFO	FIFO	Object Sound	Object Sound		Object Sound	Object Sound
	create		duplicate							destroy

Table 1 :

 1 Number of audio events (speech and sound).

	Speaker Speech Sound Speech	Mis	Mis
	ID	and			classified classified
		sound			speech	sound
	S01	213	184	29	8	1
	S02	285	212	73	10	6
	S03	211	150	61	8	6
	S04	302	211	91	10	11
	S05	247	100	48	11	4
	S06	234	189	45	17	6
	S07	289	216	72	21	6
	S08	249	190	59	25	3
	S09	374	283	91	19	7
	S10	216	163	53	10	4
	S11	211	155	56	18	2
	S12	401	346	55	13	13
	S13	225	184	41	4	7
	S14	235	173	62	9	10
	S15	641	531	111	39	17
	S16	262	216	46	10	5
	ALL	4595	3503	993	232	108

Table 2 :

 2 Number of syntactically correct vocal orders

	Speaker Number SNR Speaker Number SNR
	ID		(dB)	ID		(dB)
	S01	20	17	S02	32	17
	S03	22	19	S04	26	18
	S05	26	12	S06	24	15
	S07	19	25	S08	33	12
	S09	40	20	S10	40	11
	S11	37	14	S12	26	17
	S13	21	14	S14	27	12
	S15	28	14	S16	22	14
	All	443	15.8

Table 3 :

 3 Home automation order error rate (DER)

	Speaker	Expe.	Oracle	Speaker	Expe.	Oracle
	ID	(%)	(%)	ID	(%)	(%)
	S01	35	20	S02	12.5	6.2
	S03	22.7	22.7	S04	23	7.7
	S05	15	3.8	S06	21	8.3
	S07	79	52.6	S08	30	33.3
	S09	40	22.5	S10	67	47.5
	S11	46	27	S12	21	7.7
	S13	43	19	S14	48	29.6
	S15	71	55.5	S16	18	13.6
	Average	38%	23.9%

http://www.semvox.de

http://sweet-home.imag.fr

Acknowledgements

This work is part of the SWEET-HOME project founded by the French National Research Agency (Agence Nationale de la Recherche / ANR-09-VERS-011). The authors would like to thank the participants who accepted to perform the experiments. Thanks are extended to B. Meillon, N. Bonnefond, D. Guerin, C. Fontaine and S. Pons for their support.


Frédéric Aman
email: frederic.aman@imag.fr

Michel Vacher
email: michel.vacher@imag.fr

Solange Rossato
email: solange.rossato@imag.fr

Franc ¸ois Portet
email: francois.portet@imag.fr

In-home detection of distress calls: the case of aged users

Keywords: speech recognition, dependence, elderly, keyword detection

In the context of technologies development aiming at helping aged people to live independently at home, the CIRDO 1 project aims at implementing an ASR system into a social inclusion product designed for elderly people in order to detect distress situations and provide capability to call for help. In this context we present a system able to detect distress and call for help sentences on line.

Introduction

A survey shows that 80% of people above 65 years old would prefer to stay living at home if they lose autonomy [START_REF] Csa | Les franc ¸ais et la dépendance[END_REF] and some of them needs the assistance of someone for regular elementary activities (nursing home services, domestic help, etc.) [START_REF] Tlili | Perspectives démographique et financières de la dépendance[END_REF].

Few projects have seriously considered audio/speech technology in their design [START_REF] Hamill | Development of an automated speech recognition interface for personal emergency response systems[END_REF] [START_REF] López-Cózar | Multimodal dialogue for ambient intelligence and smart environments[END_REF]. Part of this can be attributed to the complexity of setting up this technology in a real environment and to important challenges that still need to be overcome [START_REF] Vacher | Development of audio sensing technology for ambient assisted living: Applications and challenges[END_REF]. However, audio technology is well accepted by the potential users [START_REF] Portet | Design and evaluation of a smart home voice interface for the elderly -Acceptability and objection aspects[END_REF].

In this context, the CIRDO project promotes autonomy and support for elderly people by caregivers through the social inclusion product e-lio 2 . The objective of the project is to integrate an ASR system into this product that will include detection of distress situations and voice commands.

The use of elderly voice can be an issue for performance of speech recognition system. Some authors [START_REF] Baba | Acoustic models of the elderly for large-vocabulary continuous speech recognition[END_REF][START_REF] Vipperla | Longitudinal study of ASR performance on ageing voices[END_REF] have reported that classical ASR systems exhibit poor performances with elderly voice because most acoustic models of ASR systems are acquired from non-aged voice samples. Indeed, ageing voice is characterized by some specific features such as imprecise production of consonants, tremors and slower articulation [START_REF] Ryan | Perceptual and acoustic correlates in the speech of males[END_REF] due to changes in vocal production system [START_REF] Takeda | Aging effects on motor units in the human thyroarytenoid muscle[END_REF][START_REF] Mueller | Acoustic and morphologic study of the senescent voice[END_REF]. Howerver, it is possible to improve the recognition performances thanks to acoustic models adaptation [START_REF] Vacher | Speech-based interaction in an aal context[END_REF].

This paper presents the ASR ability to detect sentences of distress in the case of ageing voice, the corpora and the ASR that we used were presented in [START_REF] Aman | Contribution à l'étude de la variabilité de la voix des personnes âgées en reconnaissance automatique de la parole (assessment of the acoustic models performance in the ageing voice case for asr system adaptation)[END_REF].

Audio processing in line: CirdoX

We developed an application for the recognition of distress situations by analyzing the audio signal to detect sound and voice in real-time in an equipped flat. The system CirdoX will be coupled with an analysis of video scenes developed by the LIRIS laboratory in Lyon. Figure 1 shows the chain of acquisition and processing of the audio signal.

CirdoX is made of two parallel processes communicating by IPC (Inter-Process Communication) signals. This application is designed to be modular, with independent modules. For each module, the user can choose from among several plug-ins corresponding to different techniques.

The audio stream is captured by a microphone and recorded by the Acquisition module. This module can work through the plug-ins using the PortAudio library (audio card), Kinect or a National Instrument card. Then, the Detection module detects the occurrence of a signal (sound or voice). The detection of the occurrence of an audio event is based on the change of energy level of the three highest frequency coefficients of the Discrete Wavelet Transform (DTW) in a floating window frame (last 2048 samples without overlapping). Each time the energy on a channel goes beyond a self-adaptive threshold, an audio event is detected until the energy decrease below this level for at least an imposed duration.

Detection is done by thresholding on the discrete wavelet transform of the signal energy [START_REF] Istrate | Information extraction from sound for medical telemonitoring[END_REF]. Then, a wave file is created by the Wave Creation module. The Discrimination module determines whether the signal is sound or speech. This module can be executed through the plug-ins using a Gaussian Mixture Model (GMM) or decision tree. In the case of sound, the Classification module defines to which sound class the signal belongs. It can work through GMM, decision tree and also Hidden Markov Model (HMM). In the case of speech, the signal is sent to the ASR module in order to output a sentence. Both hypotheses from Classification and ASR module are received by the Data Formatting module that sends a socket containing all the relevant data to the Fusion module. This one also receives data from the recognition of distress by video analysis in order to make a fusion between video and audio data. The Fusion module determines whether or not a distress situation is occurring and sends a socket containing its decision to the social inclusion product, e-lio. Then, e-lio executes an appropriate action, such as calling a doctor.

Distress sentences detection

In this section, we focus now on a part of the Fusion module: the filtering of distress sentences. The filter detects the distress sentences into the ASR hypotheses by using Levenshtein distance [START_REF] Levenshtein | Binary codes capable of correcting deletions, insertions and reversals[END_REF]. Based on the GRePS laboratory's work [START_REF] Chaumon | Démarche de développement de technologies ambiantes pour le maintien à domicile des personnes dépendantes : vers une triangulation des méthodes et des approches[END_REF] that interviewed elderly people in nursing homes to identify and describe what situations of distress they could have experienced, we created a list of sentences that can occur during a distress situation.

The filter calculates the Levenshtein distance between the ASR output hypothesis and all the distress sentences of the list. The sentence from the list with the best Levenshtein distance is selected according to a threshold. The lower the distance (score from 0 to infinity), the better the matching will be. To not be biased by orthography, the distance is calculated on a phonemic level. This approach takes into account some recognition errors such as word endings errors or slight variations. Moreover, in many cases, a miss-decoded word is phonetically close to the correct one due to close pronunciation.

A detected sentence occurs when the Levenshtein distance (normalized by the number of phonemes) is under the threshold, and is well matched if the ASR output hypothesis corresponds to the sentence selected from the list. If a detected sentence is well matched, it is considered as true positive (TP), otherwise a detected sentence not well matched is considered as a false positive (FP). A casual (C) ASR output hypothesis is never correct because of the ASR language model specifically adapted to distress utterances, and will never be well matched with a sentence of the distress sentences list. But if its distance is under the threshold, it is yet detected and becomes a false positive; if above threshold, it is true negative (TN). Also, if a distress (D) ASR output hypothesis is not well matched with the selected sentence and is not detected, it is also true negative. Finally, if a distress ASR output hypothesis is well matched with selected sentence but not detected, it is false negative (FN).

In order to assess the Levenshtein distance filter, we realized a decoding with the elderly speakers from the AD corpus, including 2796 distress sentences (same sentences as in Section 2 and 3) and 3006 casual sentences, for a total duration of 2 hour 12 minutes. The casual utterances were used as disrupters, with some sentences far from the distress ones: for instance Les patates sont cuites (Potatoes are cooked), or closer, for instance Le médecin a appelé (The doctor called). The average WER with the adapted acoustic model was 14.5% for the distress sentences, and was much higher for the casual sentences, 87.5%, due to the adapted-to-distress language model. Then, we drew a ROC curve (Figure 2) representing the True Positive Rate (TPR, sensibility or recall) in function on the False Positive Rate (FPR, 1-specificity) by variating the threshold on the Levenshtein distance, for the 43 elderly speakers. At the point of equal error (cutoff=0.5), sensibility and specificity were equal to 91.8%. For our application, a high sensibility must be privileged in order to not miss some real distress situations. We obtained the positive and negative test showed in table 1 with threshold=0.5. Also, we found a recall, precision and F-mesure equal to 91.8, 89.4 and 90.6%. This is in line with an other study conducted by Vacher et al. [START_REF] Vacher | Recognition of voice commands by multisource ASR and noise cancellation in a smart home environment[END_REF], where they detected domotic orders among casual utterances with a multisource ASR in a smart home environment. On their baseline system (best SNR channel), they found WER, recall, precision, and F-mesure respectively equal to 18.3, 88.0, 90.5 and 89.2%.

In table 2, we evaluated the distress sentences detection for the different levels3 of elderly dependence GIR 2-3; GIR 4-5 and GIR 6 by using recall, precision and F-mesure. We observed that the system performance is quite depending on the level of the elderly dependence. Due to the lower F-mesure found on GIR 2-3 comparing to GIR 4-5 and GIR 6 (-7%), we suggest that such a system could be provided to elderly and usable by them only for score ranges from GIR 4 to GIR 6.

Conclusion

In this paper we presented the CirdoX software, used to filter and detect the distress calls. We evaluated a filter based on Levenshtein distance and showed that the recall and precision reach 91.8 and 89.4%. Moreover, we showed that such a system can not detect distress sentence well enough in the case of elderly in GIR 2-3 because of the high WER in this group.

In a future work, the whole system CirdoX including sound classification, speech recognition and fusion with video scene recognition is going to be evaluated in realistic condition in a real smart home [START_REF] Gallissot | Une plateforme usage pour l'intégration de l'informatique ambiante dans l'habitat : Domus[END_REF]. Professional actors will play some scenarios, including for example falls to the ground because of the foot grasped in the carpet, sudden weakness, etc. We will assess how video can improve distress detection.

Figure 1 :

 1 Figure 1: Diagram of CirdoX.

Figure 2 :

 2 Figure 2: ROC curve representing TPR in function of FPR.

Table 1 :

 1 Positive and negative test.

Table 2 :

 2 Distress sentences detection in function of dependence

		Recall	Precision	F-mesure
	GIR 6	92.6%	91.1%	91.9%
	GIR 4-5	92.0%	90.0%	91.0%
	GIR 2-3	88.4%	80.7%	84.3%

This study was funded by the National Agency for Research under the project CIRDO -Industrial Research (ANR-2010-TECS-012)

http://www.technosens.fr

We used a French national test as reference: the AG-GIR (Autonomie Gérontologie Groupes Iso-Ressources) grid, http://vosdroits.service-public.fr/F1229.xhtml


Michel Vacher

Benjamin Lecouteux

Dan Istrate

Thierry Joubert
email: thierry.joubert@theoris.fr

François Portet
email: francois.portet@imag.frdan.istrate@esigetel.fr

Mohamed Sehili
email: mohamed.sehili@esigetel.fr

Pedro Chahuara

Evaluation of a Real-Time Voice Order Recognition System from Multiple Audio Channels in a Home

Keywords: Real-time audio analysis, experimental in-situ evaluation, Smart Home, Home Automation, AAL

The SWEET-HOME 1 project aims at providing audio-based interaction technology that lets the user have full control over their home environment and at detecting distress situations. This paper presents the audio analysis system PATSH developed for this project and a user experiment in a smart home that evaluates the performances of the first revision of the system regarding vocal order recognition with realistic scenarios.

Introduction

The development of smart homes and intelligent companions is seen as a promising way of achieving in-home daily assistance [START_REF] Chan | A review of smart homes-present state and future challenges[END_REF]. Despite, some smart home projects have considered speech recognition in their design [START_REF] Istrate | Embedded implementation of distress situation identification through sound analysis[END_REF][START_REF] Charalampos | Enabling human status awareness in assistive environments based on advanced sound and motion data classification[END_REF][START_REF] Popescu | An acoustic fall detector system that uses sound height information to reduce the false alarm rate[END_REF][START_REF] Badii | CompanionAble -integrated cognitive assistive & domotic companion robotic systems for ability & security[END_REF][START_REF] Hamill | Development of an automated speech recognition interface for personal emergency response systems[END_REF][START_REF] Filho | From science fiction to science fact: a smart-house interface using speech technology and a photorealistic avatar[END_REF][START_REF] Lecouteux | Distant Speech Recognition in a Smart Home: Comparison of Several Multisource ASRs in Realistic Conditions[END_REF], there are still important challenges to be overcome to set up this technology in a real environment [START_REF] Vacher | Development of Audio Sensing Technology for Ambient Assisted Living: Applications and Challenges[END_REF].

Designing and applying speech interface in Smart Home to provide security reassurance and natural man-machine interaction is the aim of the SWEET-HOME 2 project [START_REF] Vacher | The SWEET-HOME Project: Audio technology in Smart Homes to improve well-being and reliance[END_REF]. The project addresses the important issues of distant voice command recognition and sound source identification to improve the robustness of voice-based home automation control. In this paper, we introduce the PATSH system which performs the real-time identification of the voice command anywhere in the home and sends the result to an intelligent controller in charge of making decision based on these [START_REF] Chahuara | Using Markov Logic Network for On-line Activity Recognition from Non-Visual Home Automation Sensors[END_REF].

The Audio Analysis System

The global architecture of PATSH is illustrated in Figure 1. The PATSH framework is developed with the .Net cross-plateform technology. The main data structure is the Sound object, which contains a segment of the multidimensional audio signal whose interpretation is continuously refined along the processing pipeline. PATSH deals with the distribution of the data among the several plugins that perform the processing to interpret the audio events. The execution can be done, in parallel, synchronously or asynchronously, depending on the settings stored in a simple configuration file. For a complete description of the system the reader is referred to [START_REF] Sehili | Sound Environnment Analysis in Smart Home[END_REF].

In the SWEET-HOME configuration, PATSH runs plugins that performs the following tasks: (1) multichannel data Acquisition through the NI-DAQ6220E card (16bits, 16kHz, 7channels), (2) sound detection, (3) sound/speech discrimination, (4) sound classification, (5) automatic speech recognition (ASR) and extracting vocal orders, and (6) presentation, communicating the sound event to the Intelligent Controller. If a vocal order is detected and according to the context (activity and localisation of the user in the flat), an home automation command is generated to make the light up, close the curtains or emit a warning message thanks to a speech synthesizer.

Experiments in real conditions

Experiments were run in the DOMUS smart home at the Laboratoire d'Informatique de Grenoble [START_REF] Gallissot | Une plateforme usage pour l'intégration de l'informatique ambiante dans l'habitat : Domus[END_REF]. The flat has been equipped with 7 radio microphones set in the ceiling for audio analysis. A specialized communication device, e-lio, from the Technosens company is used to initiate a communication between a senior and a relative [START_REF] Vacher | The SWEET-HOME Project: Audio technology in Smart Homes to improve well-being and reliance[END_REF]. The possible voice orders were defined using a very simple grammar : set an actuator on:

(e.g. Nestor ferme fenêtre) key initiateCommand object stop an actuator:

(e.g. Nestor arrête) key stopCommand [object] emergency call:

(e.g. au secours)

Our previous user study showed that targeted users prefer precise short sentences over more natural long sentences [START_REF] Portet | Design and evaluation of a smart home voice interface for the elderly -Acceptability and objection aspects[END_REF].

Each order belongs to one of three categories: initiate command, stop command and emergency call. Except for the emergency call, every command starts with a unique key-word (here : 'Nestor') that permits to know whether the person is talking to the smart home or not.

To validate the system in realistic conditions, we built scenarios in which every participant was asked to perform the following activities: (1) Sleeping; [START_REF] Istrate | Embedded implementation of distress situation identification through sound analysis[END_REF] Resting: listening to the radio; (3) Feeding: realizing and having a meal; and (4) Communicating: going out of the flat to do shopping and having a talk with a relative thanks to e-lio. Therefore, this experiment allowed us to process realistic and representative audio events in conditions which are directly linked to usual daily living activities. The participant had to use vocal orders to make the light on or off, open or close blinds, ask about temperature and ask to call his or her relative. The instruction was given to the participant to repeat the order up to 3 times in case of failure. A wizard of Oz was used in case of persistent problem.

Sixteen healthy participants (including 7 women) were asked to perform the scenarios without condition on the duration. Before the experiment, the participant was asked to read a text of 25 short sentences in order to adapt the acoustic models of the ASR for future experiments. The average age was 38 years (19-62, min-max) and the experiment lasted between 23min and 48min. While the scenario included 15 vocal orders more sentences were uttered because of repetitions.

During the experiment, audio data were recorded and saved in two ways. Firstly, the 7-channel raw audio signal was stored for each participant to make subsequent analysis possible. In total, 8h 52min 36s of data was recorded for the 16 participant. Secondly, the individual sound events automatically detected by PATSH were recorded to study the performances of this framework. Overall, 4595 audio events were detected whose 993 were speech utterances.

In this study, we are only interested in recognizing vocal orders or distress sentences. All other spontaneous sentences and system messages were discarded. Therefore, only the syntactically correct vocal orders were annotated. The average SNR and duration are 15.8dB and 1s with is far from the recording studio condition. Thanks to this annotation, an oracle corpus corpus was extracted. The comparison between experimental real-time results with thus obtained with the same ASR on the oracle corpus will allow to analyse the performance degradation introduced by the PATSH system.

Results

Discrimination between speech and sounds

All the results presented take into account the performances of the detection because the signals are extracted automatically by the system. The sound/speech discrimination badly classified 108 sound and 232 speech occurrences which gives a total error rate of about 7.4% which is correlated with our previous results [START_REF] Sehili | Sound Environnment Analysis in Smart Home[END_REF]. 23.4% of speech occurrences were missed speech and 3% of the detected speech was actually sounds. Theses poor performances are explained by the fact that PATSH was not successful in selecting the best audio event among the set of simultaneous event and thus the event with low SNR introduced errors and were not properly segmented. Theses performances are explained by the fact that the multichannel information was not used; each channel was processed separately. The channel with low SNR introduced errors. For the sounds, the number of missed sounds is about 6.6% and of speech identified as sound is about 3%. Sounds such as dishes, water flow or electric motor were often confused with speech. For instance, when certain persons stirred the coffee and chocked the spoon on the cup or when they chocked plates and cutlery, the emitted sounds had resonant frequencies very to the speech one. This is emphasiz- ing the difficulty of the task and Models must be improved to handle these problematic samples [START_REF] Vacher | Development of Audio Sensing Technology for Ambient Assisted Living: Applications and Challenges[END_REF].

Home automation order recognition

The global performance of the system is directly related to vocal order recognition. The DER (Domotic Error Rate) is shown in Table 1, the 3 nd and 7 th columns "Expe." indicate the results for the real-time experiment, the columns "Nb." indicate the number of vocal orders for each speaker. This error rate is evaluated after filtering at the input of the intelligent controller and includes the global effects of all stages: detection, discrimination between speech and sound, ASR. When the uttered voice orders were not respecting the grammar, for example when a sentence such as "Nestor heure" is uttered instead of the command "Nestor donne l'heure", these utterrances were discarded.

The error rate is 38% on average and 23.9% for the oracle.

Conclusion

This paper presents the PATSH system which performs realtime identification of voice commands in the home. The architecture is based on a pipeline of sound event detection, speech/noise discrimination and ASR or sound classification modules. Each identified sound event is then sent to an intelligent controller for final decision about the action to make on the house. Our application of this technology within a realistic smart home, showed that one of the most sensible tasks is the speech/noise discrimination [START_REF] Vacher | Development of Audio Sensing Technology for Ambient Assisted Living: Applications and Challenges[END_REF]. According to the SNR level, the performance can be quite poor, which has side effects on both the ASR and the sound classification (and then on the decision making). Another issue is linked to the lack of handling of simultaneous sound event records. These fill the sound object queue, which is the system bottleneck, and thus slow down the processing while real-time performances are required. To increase the performance and free this bottleneck, we had implemented a filtering strategy to remove low SNR audio events as well as too delayed events. The preliminary results showed a significant increase in performance. In a second step, PATSH will be modified to allow in real-time a multisource ASR thanks to the Driven Decoding Algorithm [START_REF] Vacher | Recognition of voice commands by multisource ASR and noise cancellation in a smart home environment[END_REF].

Although the 16 participants had to repeat, sometimes up to three time, the voice commands, they did not complained about the reaction time nor about the syntax. They were overall very excited about commanding their own home by voice. The next experiment, will involve aged participants and a more controlled evaluation of their feeling and reaction about the system in order to compare with our previous user study [START_REF] Portet | Design and evaluation of a smart home voice interface for the elderly -Acceptability and objection aspects[END_REF] and conclude about the potential of the state of the art speech technologies for Ambient Assisted Living.

Figure 1 :

 1 Figure 1: The PATSH architecture.

create transfer write back duplicate fill write back write back transfer transfer transfer transfer destroy write back

	PATSH
	Capture Audio source N Channels	Object Work	Work Object Work Object Object Work		Sound Event Processing Work Object	Work Object		Work Object
		Acquisition	Detection		Sound/Speech Differentiation	ASR or Sound Classification	Presentation
	Sound Object	Object Sound	FIFO	FIFO	Object Sound	Object Sound	Object Sound	Object Sound

1

SWEET-HOME is founded by the French National Research Agency (Agence Nationale de la Recherche / ANR-09-VERS-011)

2

http://sweet-home.imag.fr

Table 1 :

 1 Home automation order error rate (DER)

	Spk.	Nb.	Expe.	Oracle	Spk.	Nb.	Expe.	Oracle
	ID		(%)	(%)	ID		(%)	(%)
	S01	20	35	20	S02	32	12.5	6.2
	S03	22	22.7	22.7	S04	26	23	7.7
	S05	26	15	3.8	S06	24	21	8.3
	S07	19	79	52.6	S08	33	30	33.3
	S09	40	40	22.5	S10	40	67	47.5
	S11	37	46	27	S12	26	21	7.7
	S13	21	43	19	S14	27	48	29.6
	S15	28	71	55.5	S16	22	18	13.6
	Average	28	38%	23.9%