Joëlle Coutaz
email: joelle.coutaz@imag.fr

Emeric Fontaine
email: emeric.fontaine@imag.fr

Nadine Mandran
email: nadine.mandran@imag.fr

Alexandre Demeure
email: alexandre.demeure@inrialpes.fr

About Composing Our Own Smart Home

Keywords: D.2.2 [Software Engineering]: Design Tools and Techniques -User interfaces Design, Experimentation, Human Factors End-User composition, smart artifacts coupling, smart home, ubiquitous computing, service-oriented computing

This paper reports on an empirical study designed as a follow-up of a theoretical model intended to support reasoning about the composition of smart artifacts by end users. We have solicited 17 families and used a combination of interviews and playful cultural probes. Results show that families are willing to couple smart objects to improve their lives, and that the theoretical questions raised by our model are sound.

INTRODUCTION

Ubiquitous computing promises unprecedented empowerment from the flexible and robust combination of software services with the physical world. Software researchers assimilate this promise as system autonomy where users are "conveniently" kept out of the loop. Their hypothesis is that services, such as music playback and calendars, are developed by service providers and pre-assembled by software designers to form new service frontends. Their scientific challenge is then to develop secure, multiscale, multi-layered, virtualized infrastructures that guarantee service front-end continuity. Although service continuity is desirable in many circumstances, end users, with this interpretation of ubiquitous computing, are doomed to behave as mere consumers, just like with conventional desktop computing.

Another interpretation of the promises of ubiquitous computing, is the empowerment of end users with tools that allow them to create and reshape their own interactive spaces. The mashup paradigm incarnates this view for networked knowledge and services.

Mashups, however, are concerned with the digital dimension of our world, not with the combination of the digital with the physical. In this paper, we focus on the composition of smart artifacts by end users, bringing together physicality and digital power. Examples of such artifacts include smart phones, augmented fridges, or information appliances.

Our hypothesis is that end users are willing to shape their own interactive spaces by coupling smart artifacts, building imaginative new functionalities that were not anticipated by system designers. A number of tools and techniques have been developed to support this view including the Jigsaw editor [START_REF] Humble | Playing with the bits" user-configuration of ubiquitous domestic environments[END_REF], CAMP [START_REF] Truong | CAMP: A Magnetic Poetry Interface for End-User Programming of Capture Applications for the Home[END_REF], iCAP [START_REF] Dey | iCAP: Interactive prototyping of context-aware applications[END_REF], or Newman's work on end user composition with OSCAR [START_REF] Newman | Providing an integrated user experience of networked media, devices, and services through end-user composition[END_REF]. The major focus of this prior work is on exploring novel interaction techniques and on technical frameworks. In this paper, we are concerned with the fundamental meaning (and human needs) of building confederation of interoperating smart artifacts.

In [START_REF] Coutaz | End-User Programming and the Intrinsic Complexity of Networked Artefacts[END_REF], we present a chemistry-inspired theoretical model that supports reasoning about the composition of smart artifacts by end users. Whereas scientific knowledge in chemistry is sufficiently advanced to predict the occurrence and results of a reaction, such knowledge is clearly lacking in ubiquitous computing. In particular, we are unable to predict which artifacts of every day life end users would be willing to couple (and decouple) to obtain new services. Is coupling commutative, associative, distributive over some other operation? In other word, can we define an algebra over smart artifacts so that we can generalize the problem and reason at a high level of abstraction in a rigorous manner?

The field study presented in this paper is our first attempt at answering the questions raised by our theoretical model. The experimental study is presented in the next section. Our findings are discussed in the last section.

THE EXPERIMENTAL STUDY 2.1 Participants

Drawing on Davidoff's et al. experiment and conclusions (i.e. "families want more control of their lives" [START_REF] Dey | iCAP: Interactive prototyping of context-aware applications[END_REF]), we focused on "busy" families. The participants have been solicited through bulletin board advertisements, email, as well as from personal relationships.

We have recruited 17 families representing a total of 40 persons (35 adults and 5 children), all living in the area of Grenoble (France). Of the 17 families, 12 were dual income families, 1 was single parent, 2 were house mates, and 2 were retired couples. All families were well educated with medium to high standard of living.

Method

Our method was designed to reconcile the following requirements: to collect meaningful data in a minimum of time while respecting privacy. As presented above, we are interested in determining how far people are ready to envision the interconnection of everyday devices to improve control of their lives, and to which extent coupling objects is commutative, associative and distributive. For so doing, we have used a combination of interview [START_REF] Silverman | Introducing qualitative research[END_REF] (good for clarification), playful cultural probe (appropriate for respecting privacy and for improving subjects involvement [START_REF] Bernhaupt | Playful Probing: Making Probing more Fun[END_REF]). The presence of the experimental team (ourselves, from 1 to 3 persons) was limited to 1h30 per family home. Fieldwork was structured as a four-step process: photographing, interview, game, and debriefing.

Step 1: Photographing. Using digital cameras provided by the experimental team, two volunteer family members were asked to take pictures of 10 objects at the rate of 2 objects per room. For each of the 5 rooms of their choice, they were asked to take the picture of one object that they considered to be necessary in their every day life or that would help them in organizing their lives, as well as the picture one object that they considered to be superfluous but valuable (typically, a painting). The volunteers (in general, the parents) were not supposed to be in the same room at the same time so that they would not know which pictures the other member had taken. Meanwhile, the experimental team would wait sitting at a place indicated by the parents (typically, the living room where they usually meet with friends and visitors).

Step 2: Interview. We then conducted an interview with all the family members, using the pictures as input material. Questions were directed at understanding the reasons for their choices, the value attached to the objects or the services provided in daily use. Special attention was given to the (many) remote controller(s) typically found in the household environment. We progressively oriented our questions towards novel uses of smart artifacts. In particular, we asked which objects of the house (including those on the pictures) they would qualify as "programmable" (e.g., TV's, washing machines, alarm clocks), "communicating" (e.g., computers, mobile phones), or emotional (i.e. carrying intimate value). This was used as a means to elicit routines and exceptional needs as well as to prepare the game developed in Step 3.

Step 3: Association game. The association game drew on people creativity using the pictures as play cards. Pictures were sorted randomly and presented two at a time (then, three at a time) on a tablet PC. Family members were asked to imagine which service(s) and value(s) these two (or three) objects coupled together would provide them with. Random coupling was designed to solicit imagination in unexpected ways, and to get hints about the existence of a "natural" algebra over smart artifacts.

Step 4. Debriefing and informal discussion. The last stage was dedicated to debriefing, including opened friendly discussions.

Overall, we have collected comments and objective data for 349 couplings for a total duration of 25 hours of our presence in the 17 family homes.

Data Analysis

Interviews and debriefings helped us to identify recurring facts between home families such as key moments during weekdays for which families would expect support from a smart home, or attitudes with regard to "programming the home". Data from the association game as well as from the interviews were used to find answers to our theoretical questions. More specifically, we classified the objects that have been photographed into four categories: objects that have been denoted as "programmable" by the subjects, objects that have been declared as "communicating", objects that support both capabilities, and objects that have none of these two properties. Using the Chi-square test, we have been able to find strong significance between the abilities of the subjects to envision (or not) services depending on the capabilities of the assembled objects. In particular, the communication capability allows peoples to more easily imagine new services from the assembled object.

FINDINGS

Our experiment has led to three types of results: recurring facts across families, early answers to our theoretical questions, as well as insights about our method.

Recurring Facts

We found a number of facts that are quite consistent with the results reported in prior literature:

1. "Wake-up" time, "on-the-way-to-home" and "arrivinghome" times are key to people. To save time and improve efficiency, activities are organized into well-polished procedures. As a result, exceptions to these routine tasks are sources of stress. Support for avoiding or for solving exceptions is one class of services expected from a smart home. This includes the management of possessions (laundry to be launched because of a business trip planned in a couple of days, food on the point to be missing, medicine close to expiration date), decision-making (what to buy, what to wear today), reminders (doctor appointment), security (door properly locked), resources consumption and resource sharing among family members (typically, hot water and bathroom occupation in the morning), etc.

2. With regard to programming, attitudes range from "I do not want to be assisted" to "It will work 99% of the time, but it will be hell for the other 1%". Motivation for programming is systematically grounded on a clear straight forward observable benefit.

We believe that our findings related to coupling everyday life objects are original.

About Coupling

Our data from the association game shows two important results:

(1) Family members are prone to envision new services when coupling involves one "communicating" object, or one "programmable" object, at least. [START_REF] Coutaz | End-User Programming and the Intrinsic Complexity of Networked Artefacts[END_REF] The "communicating" capability has more impact than "programmability" on the capacity of family members to imagine new services. However, 78 of the 349 couplings resulted in service finding although none of the objects were programmable or communicating. For example, the couple "bed-shower", whose objects had not been classified as programmable nor communicating, suggested that "getting up from the bed in the morning would turn the shower on in order to provide water at the right temperature when coming back from the toilet". This means that there is a large body of potentiality for novel services based on mundane everyday objects 1 .

The services suggested by our family members fall into four categories. We illustrate them with the most typical examples drawn from our fieldwork.

Service substitution. People have observed that, for the same (sport) events, commentaries on radio broadcasts are richer than those provided by TV. As a result, they would like to replace the TV sound service with that of the radio to improve the overall quality of the informational experience. Another example is User Interface substitution: some people are quite skill at setting up alarms on their mobile phone, but they do not know how to do this for their physical home alarm clock. As a consequence, they would find it quite convenient to replace the user interface of their alarm clock with that of their phone thanks to a convenient opportunistic coupling of the phone with the alarm clock.

Service improvement. Some household appliances such as washing machines and cubboards, do not provide any convenient way to control and monitor their current internal state. Appliances than are not sufficiently equipped could be improved by coupling them with additional input and output facilities such as those of the TV set.

Service chaining. Service chaining is intended to improve comfort, wellbeing as well as resources for the routine, but hectic, activities. For example, "picking up the towel the shower would trigger the coffee machine so that coffee would be ready just in time, at the right temperature, along with the radio turned on in the kitchen broadcasting the news using the appropriate sound level".

Service "starter". We have observed that some appliances serve as triggers for services that are expected to be pre-composed to support routine activities. The towel and the bed mentioned above, play this role, implicitly. Not surprisingly, people also want to have an explicit and reliable control over the home (cf. the worry that 1% of the time, the house would turn into hell). Some people came up with the "morning starter push button" conveniently located close to the bed that would gently "wake up" the house when pushed.

The need for chains of services underpins some form of associativity. For example, one family qualified the "towel-coffee machine" coupling as a "morning package". If the expression towel -coffee denotes the coupling of the towel with the coffee machine, then (towell -coffee), between parenthesis, denotes the notion of package. During the discussion, our family members thought of adding the "morning starter push button" b to get a controllable chain "b -t -c". Here, their mental construction can be formalized as a right associativity of the "coupling" operation:

b -t -c = b -(t -c).
Coupling for service improvement entails some form of distributivity. Typically, the TV set tv has often be mentioned as a way to observe and control the state of a number of appliances such as the washing machine w or the oven o. This can be formalized in the following way: tv

-(w | o) = tv -w | tv -o.
Commutativity is generally satisfied with notable exceptions when there is a causality relationships between the objects. 1 Detailed quantitative data will be presented in a full paper.

About the Experimental Method

The "Snapshots taking" of our fieldwork has multiple advantages:

(1) It serves as an ice breaking between the family members and the experimental team; (2) Family members "reveal their house" naturally while we, the experimenters, do not intrude their private spots. (3) Family members get truly involved (and intrigued by what will come next). ( 4) As opposed to playful probing proposed by R. Berhaupt [START_REF] Bernhaupt | Playful Probing: Making Probing more Fun[END_REF], our game uses images of intimate objects, not of generic entities. This increases the interest and imagination of the participants while improving the meaningfulness of the data collected.

CONCLUSION

A central focus of our work is investigating the fundamental meaning of building confederation of interoperating smart artifacts. Our approach to this problem is theory-driven with the quest for an algebra that would support generalization and prediction. Although additional investigations are necessary, early results from our fieldwork support this approach.

ACKNOWLEDGMENTS

This work has been supported by the ANR CONTINUUM project reference ANR-08-VERS-005.


Michel Vacher
email: michel.vacher@imag.fr

François Portet

Solange Rossato

Frédéric Aman

Caroline Golanski

Remus Dugheanu

Speech-based interaction in an AAL context

Keywords: voice interface, speech recognition, ageing voice, AAL, smart Home

One of the biggest challenges in Ambient Assisted Living is to develop smart homes that anticipate the health needs of their inhabitants while maintaining their safety and comfort. To facilitate interactions with the smart home, systems that respond naturally to voice commands would be the most adequate for disabled and frail people. In this paper we present two studies aiming at investigating the feasibility of such interactive systems. In the first study, the acceptability of a voice interface as part of the smart home was investigated. The second study is related to the adaptation of speech recognition technologies to the senior population; population which is known to challenge standard ASR systems. To this aim, we recorded two specific speech corpora (Voice-Age and ERES38) which were analyzed in a semi automatic manner to reveal the aged-voice characteristics. Some phonemes are more affected by age than others and this study shows that, by adapting acoustic models to seniors, performances increase. Voice interfaces appear to have a great potential to ease daily living for elderly and frail persons and would be better accepted than more intrusive solutions. An interesting finding that came up is their overall acceptance provided the system does not drive them to a lazy lifestyle by taking control of everything. Smart homes must support daily living by giving seniors more ability to control rather than acting in place of people.

INTRODUCTION

The demographic change and ageing in the developed countries imply challenges in the way the population will be cared for in the near future. A popular solution is to develop ICT to make it possible for older person to stay at home. Home support of older adults is related to several constraints: first, the increasing number of older persons who often wish to live independently as long as possible in their own homes; secondly, the cost challenge to society to support people loosing independence and; thirdly, the shortage of places in specialized institutions. Given these demographic trends and using new technologies, Ambient Assisted Living (AAL) research programs intend to develop services and innovative products that improve the quality of life of older people, maintain their independence and quality of life in a normal living environment whereas seniors are more affected by physical or cognitive diseases. Solutions must be developed to compensate the possible physical or mental decline to keep older persons with a good degree of autonomy. The aim is also to provide assistance if necessary through surveillance to detect distress situations. However, in the most general case, these persons are often confused by complex interfaces and technological solutions must be adapted to the needs and the specific capacities of this population. Therefore, as voice is the most natural way of communication, interfaces using a system of Automatic Speech Recognition (ASR) may be more accessible.

The aim of this paper is to expose the possible uses of audio and speech analysis in the AAL context before presenting two studies involving older persons. The first study aims at evaluating how ambient assistive speech technology is received by the targeted population. We report a user evaluation assessing the acceptance and the fear of this new technology. The second study is related to the adaptation of speech recognition technologies to the older population which has not been extensively studied in the literature, even if it is known that system development for other categories of population is not adapted to the senior. This article will terminate by a brief discussion about the results and by a presentation of the perspectives.

AUDIO USE IN AN AMBIENT ASSISTED LIVING CON-

TEXT

Smart Home and AAL

One of the biggest challenges in Ambient Assisted Living (AAL) is to develop smart homes that anticipate and respond to the needs of their inhabitants, this is especially important when they have disabilities. Smart homes are habitations equipped with a home automation system including a set of sensors, automated devices and centralized software which control the increasing amount of household appliances. Given the diverse profile of the elderly population, it is thus essential to facilitate interaction with the smart home through systems that respond naturally to voice commands rather than using tactile interfaces which require physical and visual interaction. Therefore, although speech interaction is rarely considered, it seems more adapted to people who have difficulties in moving or seeing [START_REF] Portet | Design and evaluation of a smart home voice interface for the elderly: acceptability and objection aspects[END_REF] . Chan et al. [START_REF] Chan | A review of smart homes-Present state and future challenges[END_REF] reviewed the projects with a medical perspective and identified the necessary conditions to satisfy, namely: -user needs, acceptability and satisfaction; -viability and efficacy of sensors and software; -standard compliance for information and communication systems; -legacy and ethical constraints; -cost reduction and socio-economical impact. Moreover, several studies were conducted to identify the needs of older people towards a system that can help them in their everyday life [START_REF] Callejas | Designing smart home interfaces for the elderly[END_REF][START_REF] Rialle | What do family caregivers of alzheimer's disease patients desire in smart home technologies? Contrasted results of a wide survey[END_REF] . The proposed systems consider providing assistance in three main areas: -health (tracking the status of the person and the evolution of his loss of autonomy by using physiological sensors, motion sensors, video cameras, home intrusion, etc.); -security (preventing and detecting situations of distress or risks through fall detectors, smoke detectors, etc.);and assistance for home automation systems (compensation of disabilities through better access to domestic appliances). It should be noted that a fourth priority must be added, it is the communication with relatives and with external persons which is essential for the person isolated at home.

Audio sensing technology in Smart Home

Audio analysis can be divided into speech recognition and sound identification. In this context, speech recognition is useful for voice command and dialogue while sound identification gives information about the activity of the person (e.g., closing the door or using the vacuum). Many challenges have to be overcome before these technologies could be usable and deployable in assisted living applications [START_REF] Vacher | Development of audio sensing technology for ambient assisted living: Applications and challenges[END_REF] . Sounds are produced by very varied audio sources and then sound identification is less advanced than speech recognition which benefits from continuous progress since the use of probabilistic models (Hidden Markov Model-HMM) for phoneme modeling [START_REF] Rabiner | A tutorial on hidden Markov models ans selected applications in speech recognition[END_REF] . Therefore, this section focuses on speech recognition application in smart homes.

Extraction of speech in a noisy environment

In real conditions, audio processing is affected by: background noise (TV, devices, traffic…), the room acoustic (reverberation on windows) and the position of the speaker with respect to the microphones set in the room. The most significant problem is related to speech signal mixed with unwanted noises such as music or vacuum. Sophisticated signal processing techniques should be considered to solve this problem (i.e. echo cancellation, blind source separa-tion…).

Voice interface for compensation and comfort

Voice interface is a natural way of providing the ability of driving verbally the home automation system. It's a natural way to enable a physically disabled person (e.g., person in a wheel-chair, blinds…) to keep control of their environment.

Detection of distress situations

Identifying specific sounds (glass breaking or falls) would be of great interest for such situation detection. On the other hand, voice interfaces make it possible to call for help when the person is in a distress situation but remains conscious. Moreover, gerontologists and physicians pointed out the importance of emotion in the voice [START_REF] Cornet | Les gérontechnologies au coeur de l'innovation hospitalière et médico-sociale[END_REF] . Automatic emotion level recognition would be highly helpful to detect an important problem to solve and then assistance is requested.

Evaluation of the ability to communicate

One of the most tragic symptoms of Alzheimer's disease is the progressive loss of vocabulary and communication skills. The changes can be very slow and difficult to detect by caregivers, and an automatic monitoring could allow the detection of important stages of this evolution [START_REF] Taler | Language performance un Alzheimer's disease and mild cognitive impairment : A comparative review[END_REF] .

Speech recognition adapted to the speaker

Several experiments established that automatic speech recognition performances are degraded for atypical population like children or aged persons [START_REF] Gerosa | Towards ageindependent acoustic modelling[END_REF] . As AAL concern a lot older persons, this point will be the matter of a following section.

Privacy and acceptability

Due to the increasing number of sophisticated electronic devices, the question of privacy becomes crucial [START_REF] Sharkey | Granny and the robots: ethical issues in robot care for the elderly[END_REF][START_REF] Van Hoof | Ambiant intelligence, ethics and privacy[END_REF] and speech recognition must respect privacy. Therefore, the Automatic Speech Recognizer (ASR) must be adapted to the application and should not be able to recognize private conversation sentences. Regarding the acceptability point of view, a system will be more accepted if it is regularly used in the daily life such as with a home automation system, than if it is used in rare circumstances (e.g., fall). Acceptability is crucial because it conditions the use of the system by the person. Therefore the next section is devoted to this aspect.

ACCEPTABILITY OF VOCAL ORDERS

Older persons are the most likely persons to develop cognitive and physical decline but that does not imply they are all not self-governing. Thus, the design of new daily living support technologies must take into account studies which showed that the reduction of sense of control in the elderly population may have a significant adverse effect on their health [START_REF] Rodin | Aging and health: effects of the sense of control[END_REF] .

Moreover, given the implication of the relatives in supporting their seniors, caregivers must also be included in the design. To find out what the needs of this target population are, we conducted a user evaluation assessing the acceptance and the fear of the audio technology [START_REF] Portet | Design and evaluation of a smart home voice interface for the elderly: acceptability and objection aspects[END_REF] . In the experiment we assessed the acceptability of such technology. This is the key factor for integrating new technologies in homes, particularly when the users are older persons or low ICT educated persons. In the following we present the experiment we set up to test users' acceptance and a summary of the results.

Experimental set-up

The experience we undertook aimed at testing three important aspects of speech interaction in a smart home: voice orders, communication with the outside world, home automation system interrupting a person's activity. It consisted in Wizard of Oz phases and interview phases in the smart home. The WOZ interaction consisted mainly in the control of the environment. For instance, if the participant said "close the blind", the blind were closed remotely. The participants consisted of 18 persons from the Grenoble area. Among them, 8 were in the senior group, 7 in the relatives group (composed of mature children, grandchildren or friends). The mean age of the elderly group was 79.0 (SD=6.0), and 5 out of 8 were women. These persons were single and perfectly autonomous. In order to acquire another view about the interest and acceptability of the project system, 3 professional caregivers were also recruited to participate in the experiment (2 nurses and one professional elderly assistant).

Fig.1. Each test was composed of an interviewer, a wizard and a couple of one elderly person with a relative (except for one senior who was alone) inside the DOMUS smart home of the LIG Laboratory 1

The co-discovery approach (see Figure 1) was chosen to reassure the senior about the experimental context (new environment, experimenter, etc.) thanks to the presence of their relative. Moreover, it eased the projection of both participants into the new system because they could exchange points of view. Of course, the relationship between the two people can also influence the experiment that is why some short periods were planned during which the participants were interrogated separately. To assess the acceptability of the system which has no standard definition, most of the experiments were conducted to find out whether the potential users would appreciate the new functionalities brought by the system (e.g., "Do you appreciate making the system operate using your voice? Why?"). Moreover, in order to guide the development of the system, aspects of usefulness, usability, interactiveness, proactiveness, intrusiveness, social interaction, and security were investigated. The first phase of the experiment was about the voice command aspect of the project. Both the senior and her relative were present in the room. The senior was asked to control blinds, lights and the coffee machine using her voice without any recommendation about how to do it. This consisted in talking "to the home". The second phase consisted in using technology for communication with the outside such as video conferencing. The senior was left alone in the smart home watching a TV program, when the program interrupted itself to let the face of the relative appear on the screen so that they can start a conversation. The third phase focused on system interruption. The couple and the interviewer were discussing in the smart home when the system interrupted them via a pre-recorded voice played through the speakers, calling for a door to be closed or the cooker to be turned off. After this, questions related to whether being interrupted by the system was acceptable or not. Also, the problem of security in general and how such system could enhance security was discussed with the couple.

Results of the study

From the results of the study, it appears that seniors and their relatives preferred mostly the voice command, the system interventions about safety issues and the video-conferencing. It is interesting to note that, in our study, the "key-word" form for commands is highly accepted (rather than the sentence based command). This highly simplifies the integration of such technology in smart home given that small vocabulary systems are generally performing better in real world applications than large vocabulary ones. As in other related studies [START_REF] Callejas | Designing smart home interfaces for the elderly[END_REF] , all participants found a strong interest in the voice interaction system. It is strongly preferred over a tactile system (or touchscreen) which would necessitate being physically available at the place the command is to be found or would imply to constantly know where the remote controller is. This is in line with other studies concerning personal emergency response systems which showed that push-button based control is not adapted to this population [START_REF] Hamill | Development of an automated speech recognition interface for personal emergency response systems[END_REF] . Although the system was well received, it turned out that some functionality provoked strong objections among the participants. The main fear of the elderly and relatives is the system failure. Another main concern about the system is the fact that too much assistance would increase the dependence of the person by pushing her toward inactivity. Regarding the caregivers, they expressed the concern that such system would tend to gradually replace some of their visits and end up in making the seniors even more isolated. Most of these fears can be addressed by a good design of the system. However, fear about a decrease in autonomy due to a system that can do everything is a subtle one. A system designed for active people in order to improve comfort, security and save time may not be adapted to healthy but aged persons [START_REF] Koskela | Evolution towards smart home environments: empirical evaluation of three user interfaces[END_REF] . While the proposed system can bring more comfort and autonomy to daily life by providing an easy interaction with the home automation elements, the majority of the participants insisted on the security aspects. For instance, the voice interface would be of great use in case of falls. The elderly and their relatives have particularly appreciated that the system spares the elderly actions that can be dangerous and warns them of dangerous situations. This trend is confirmed in almost all user evaluations involving elderly [START_REF] Rantz | Tiger-Place, a State-Academic-Private project to revolutionize traditional Long-Term care[END_REF][START_REF] Callejas | Designing smart home interfaces for the elderly[END_REF][START_REF] Rialle | What do family caregivers of alzheimer's disease patients desire in smart home technologies? Contrasted results of a wide survey[END_REF] . Thus, smart homes for the elderly would be much more accepted if they contain features that can reassure them regarding security more than any other features whatever their initial condition and origin in developed countries are. Overall, the participants mainly stressed the interest of voice command and how this could improve security, autonomy and, to a smaller extend, could fight loneliness. However, they were very careful about privacy and clearly showed that they were very cautious of not accepting systems that would push them into a dependent situation. They want to keep control. Although only a small sample of seniors and relatives in healthy condition was recruited, this qualitative study confirmed the interest of voicebased technology for smart home and uncovered some pitfalls to avoid in its design. In the next section we describe a preliminary study to adapt standard speech recognition to the ageing voice.

AGEING VOICE

Speech of the older persons is characterized by tremor of the voice, an imprecise production of consonants, and a slower articulation [START_REF] Ryan | Perceptual and acoustic correlates in the speech of males[END_REF] . From an anatomical point of view, studies have shown agerelated degeneration with atrophy of the vocal cords, calcification of the laryngeal cartilages, and changes in the muscles of the larynx [START_REF] Taked | Aging effects on motors units in the human thyroarytenoid muscle[END_REF][START_REF] Mueller | Acoustic and morphologic study of the senescent voice[END_REF] . Given that most acoustic models of ASR systems are trained from non-aged voice samples, they are not adequate to deal with more aged population and then classical ASR systems exhibit poor performances [START_REF] Baba | Acoustic models of the elderly for largevocabulary continuous speech recognition[END_REF][START_REF] Vipperla | Longitudinal study of ASR performance on ageing voices[END_REF][START_REF] Privat | Etude de l'effet du vieillissement sur les productions langagières et sur les performances en reconnaissance automatique de la parole[END_REF] . To improve the acoustic-phonetic decoding module in ASR systems and to adapt it to the voice of seniors, we studied the phonemes that were poorly recognized in an aged voice. To do so, we collected corpora, used ASR alignment and analyzed the most difficult phonemes. Then, acoustic model adaptation was done in order to evaluate the ASR. The main steps of the study are summed up in Figure 1.

Fig.1. Main steps of the Ageing Voice study

Speech corpora

Given the absence of oral corpus adapted to the assistance at home, we recorded two new corpora specific to home automation and distress detection: the Anodin-Détresse (AD or Colloquial-Distress) corpus [START_REF] Vacher | Preliminary evaluation of speech/sound recognition for telemedicine application in a real environment[END_REF] and Voice-Age (VA) corpus [START_REF] Aman | Performance des modèles acoustiques pour des voix de personnes âgées en vue de l'adaptation des systèmes de RAP[END_REF] . AD is a corpus made of short sentences read by 21 speakers. Each of the 37 older speakers of VA read the 126 sentences of AD and long sentences extracted from newspapers or magazines. Given the results of our first studies [START_REF] Aman | Performance des modèles acoustiques pour des voix de personnes âgées en vue de l'adaptation des systèmes de RAP[END_REF] , we recorded a new corpus named ERES38 in order to improve the acoustic models for older persons by favoring the problematic phonemes such as plosives and fricatives. The ERES38 corpus is a collection of interviews useful to collect informal and spontaneous speech recorded in specific establishments, such as nursing homes. A reading was also performed during this interview. Plosives and fricatives were introduced in the text in order to be in context /a/, /i/ and /u/. The interviews were conducted with people more or less autonomous, cognitively intact, sometimes with serious mobility problems, but without other severe disabilities. The recordings began to be transcribed, and all readings were transcribed and checked. This corpus is used for acoustic model training.

Corpus

Influence of ageing voice on Automatic Speech Recognition

To compare the influence of the Aged Speech (VA) and the Non-Aged Speech (AD) groups we use the Sphinx3 engine as ASR 20 . Acoustic models were trained using the BREF120 corpus [START_REF] Lamel | BREF, a large vocabulary spoken corpus for French[END_REF] . The language model was trained with the transcript of AD in order to match the context of home automation voice commands. The result is a very small trigram language model with a vocabulary of about 170 words focused on error analysis of the acoustic-phonetic decoding step. A more detailed description of this system is available [START_REF] Aman | Performance des modèles acoustiques pour des voix de personnes âgées en vue de l'adaptation des systèmes de RAP[END_REF] .

The decoding with sphinx3 generates an orthographic transcription of the audio signal of speech.

We Consonants are generally most affected and the absence of voicing is the main factor of degradation followed by the modality of implementation (fricative vs. plosive). Therefore, it is possible that unvoiced consonants for older persons are closer to voiced consonants. These findings are in line with other results [START_REF] Privat | Etude de l'effet du vieillissement sur les productions langagières et sur les performances en reconnaissance automatique de la parole[END_REF] apart for the nasal and liquid consonants and the nasal vowels where their results are poorer.

Acoustic adaptation

The adjustment method of Maximum Likelihood Linear Regression (MLLR) was used to adapt the generic acoustic model trained with BREF120 to the voice of seniors. The adaptation was made globally with all sentences of the ERES38 corpus. The new acoustic model is the MLLR adapted model. The WER was then obtained for the 36 speakers (VA) using the generic model (WER 1 ) and using the MLLR adapted model (WER 2 ).

Results of the study

The speakers are grouped together using k-means clustering method based on observations given by WER 1 and WER 2 . The main characteristics of the 3 groups, number, gender and age, are reported in Table 2.

As shown in Table 3

CONCLUSION AND PERSPECTIVES

In this paper we present two studies aiming at investigating the feasibility of speech-based interactive systems. In the first study, the acceptability of a voice interface as part of the smart home was investigated.

Voice interfaces appear to be better accepted by the seniors than more intrusive solutions such as video cameras. Otherwise, the "key-word" form for commands is highly accepted rather than the sentence based command. An interesting finding that came up is their overall acceptance provided the system does not drive them to a lazy lifestyle by taking control of everything. Smart homes could give seniors more ability to control their daily living.

The second study is related to the adaptation of speech recognition technologies to the senior population. Therefore, we recorded two specific speech corpora (Voice-Age and ERES38) which were analyzed in a semi automatic manner to reveal the aged-voice characteristics. Some phonemes are more affected by age than others, nasal vowels and consonants. Moreover, the absence of voicing is the main factor of degradation. Our current work is to complete this corpus in order to obtain more generic senior models. The CIRDO system 27 will take advantage of theses models to integrate new services for autonomy increase and make easier the support for relatives and caregivers.

Table 1 .

 1 Main characteristics of the 3 speech corpora

		Gender	Age	Duration	Sentence
		M/F	min-		number
			max
	AD	10/11 20-65	38min 2 646
	VA	11/26 62-94	5h19min 9 052
	ERES38	8/16 68-98 17h44min 7 600

Table 2 .

 2 obtained a Word Error Rate (WER) of 7.33% for the Non-Aged Speech group (21 speakers) and a WER of 27.56% for the decoding of the Aged Speech group (36 speakers). Thus we observed a significant performance degradation of ASR for elderly speech, with an absolute difference of 20.23%. A more precise analyze is given by the forced alignment scores by phoneme. Forced alignment scores are likelihood scores belonging to the phoneme normally delivered for the considered signal portion. This score can be interpreted as a proximity to the "standard" pronunciation modeled by the generic acoustic model. The difference of acoustic score for Aged versus Non-Aged is shown in Table2. Difference of acoustic score in forced alignment for Aged Speech vs. Non-Aged Speech

	Phonemic group	Score difference
	Nasal vowels	-117.00%
	Unvoiced fricative consonants	-110.56%
	Unvoiced plosive consonants	-105.72%
	Voiced fricative consonants	-87.86%
	Voiced plosive consonants	-83.29%
	Medium vowels	-63.74%
	Open vowels	-53.21%
	Closed vowels	-45.52%
	Nasal and liquid consonants	-42.65%

Table 3 .

 3 , using the MLLR Adapted Acoustic Model reduces the WER significantly up to 11.95%. Compared to the 27.56% WER without adaptation, the absolute difference is -15.61% (relative difference of -56.65%). From an applicative point of view, this shows that we can use a database of elderly speech to make MLLR adaptation with speakers which are different from the test base. This demonstrates that the voices of ageing people have common characteristics. Nevertheless, a little part of senior people (G03 group) can be characterized with poorer results of speech recognition. This group is not composed of the oldest people. Comparison of WER with the generic acoustic model (WER 1 ) and the MLLR adapted model (WER 2 ) for the 36 Aged Speech group

	Group	Gender	Age	WER1	WER2
		M/F	min-max	(%)	(%)
	G01	4/13	70-92	13.58	5.54
	G02	3/12	63-94	33.32	14.41
	G03	4/0	62-84	65.38	29.97
	All	11/25	62-94	27.56	11.95

Acknowledgments

These studies were funded by the French National Agency (project Sweet-Home 26 ANR-2009-VERS-011 and project CIRDO ANR-2010-TECS-012). The authors would like to thank the participants (seniors, relatives and caregivers) who accepted to perform the experiment and agreed to participate to the speech recordings.


Michel Vacher

Benjamin Lecouteux

Franc ¸ois Portet

RECOGNITION OF VOICE COMMANDS BY MULTISOURCE ASR AND NOISE CANCELLATION IN A SMART HOME ENVIRONMENT

Keywords: Home automation, smart home, distant speech, multisource ASRs, keyword detection

In this paper, we present a multisource ASR system to detect home automation orders in various everyday listening conditions in a realistic home. The system is based on a state of the art echo cancellation stage that feeds recently introduced ASR techniques. The evaluation was conducted on a realistic noisy data set acquired in a smart home where a microphone was placed near the noise source and several other microphones were placed in different rooms. This distant speech corpus was recorded with 23 speakers uttering colloquial or distress sentences as well as home automation orders. Techniques acting at the decoding stage and using a priori knowledge gave the best results in noisy condition compared to the baseline (recall= 93.2% vs 59.2%) reaching good enough performance for a real usage although improvement still need to be made when music is used as background noise.

INTRODUCTION

The demographic change and ageing in the developed countries imply challenges in the way this population will be cared for in the near future. At the same time, evolution in ICT gives many opportunities to enhance in-home quality of life and to support the elderly and disabled persons in living in their own home as autonomously as possible. One of the way to bring this everyday assistance is the development of smart homes which are habitations equipped with a set of sensors, actuators, automated devices and centralised software which control the increasing amount of household appliances. Various interaction methods are being developed in this setting but one of the most promising is the speech interaction. Indeed, voice interfaces are much more adapted to people who have difficulties in moving or seeing than tactile interfaces (e.g., remote control) which require physical and visual interaction [START_REF] Portet | Design and evaluation of a smart home voice interface for the elderly -Acceptability and objection aspects[END_REF]. Moreover, voice command is particularly suited to distress situations when a person, who cannot move after a fall but being conscious, may still have the capacity to call for assistance while a remote control may be unreachable [START_REF] Hamill | Development of an automated speech recognition interface for personal emergency response systems[END_REF]. Furthermore, given the increasing complexity of home appliances, speech interfaces seem much more natural than tactile interfaces [START_REF] Vovos | Speech operated smart-home control system for users with special needs[END_REF].

While the speech interaction is a desirable feature of smart homes, many challenges are to be addressed before transferring this technology from the lab to the home. One of the major issues is the poor performance of Automatic Speech Recognition (ASR) in a noisy environment [START_REF] Vacher | Development of Audio Sensing Technology for Ambient Assisted Living: Applications and Challenges[END_REF]. Indeed, in realistic conditions, the performance of ASR systems decreases significantly as soon as the microphone is 'distant' from the speaker. This deterioration is due to a broad variety of effects including reverberation and presence of undetermined background noise such as TV, radio and devices [START_REF] Wölfel | Distant Speech Recognition[END_REF]. While user's linguistic preferences, dialogues and age dependant voice interfaces have been studied during this decade [START_REF] Vovos | Speech operated smart-home control system for users with special needs[END_REF][START_REF] Hamill | Development of an automated speech recognition interface for personal emergency response systems[END_REF][START_REF] Vipperla | Speech input from older users in smart environments: Challenges and perspectives[END_REF], speech separation from in-home noise received attention very recently within the speech processing community [START_REF] Barker | The PASCAL 'CHiME' Speech Separation and Recognition Challenge[END_REF].

In this paper, we present a system to recognise vocal home automation orders (also called home automation orders) in a noisy multiroom smart home. This work is part of the SWEET-HOME project which is introduced in Section 2 along with the evaluation dataset. The approach is based on a echo canceller stage useful in the restricted case of known noise sources like TV and radio, and a multisource ASR system that uses a priori knowledge to enhance the in-domain home automation orders recognition. This framework is presented in Section 3. The experiments and results are then presented in Section 4 before the conclusion.

CONTEXT OF THE STUDY AND EVALUATION DATASET

This study was done in the context of the SWEET-HOME project (http://sweet-home.imag.fr) which aims at designing a new smart home system based on audio technology to provide assistance via natural man-machine interaction and security reassurance by detecting situations of distress. If these aims are achieved, then the person will be able to pilot, from anywhere in the house, her environment at any time in the most natural way possible. In this study, voice orders were defined using a very simple grammar as shown on Figure 1. Our previous user study showed that targeted users prefer precise short sentences over more natural long sentences [START_REF] Portet | Design and evaluation of a smart home voice interface for the elderly -Acceptability and objection aspects[END_REF]. Each order belongs to one of three categories: initiate command, stop command and emergency call. Except for the emergency call, all command starts with a unique key-word that permits to know whether the person is talking to the smart home or not. In the following, we will use 'Nestor' as key-word: In this project, the targeted environment in which speech recognition must be performed is shown in Figure 2. It is a thirty square meters suite flat set up by the MULTICOM team of the Laboratory of Informatics of Grenoble, which includes a bathroom, a kitchen, a bedroom and a study, all equipped with sensors, switches and actuators. It was complemented with seven RF microphones (set in the ceiling and directed to the floor) whose audio channels are recorded in real-time thanks to a dedicated PC embedding an 8-channel input audio card [START_REF] Vacher | Development of Audio Sensing Technology for Ambient Assisted Living: Applications and Challenges[END_REF]. This places the study in a distant-speech context where microphones may be far from the speaker and may record different noise sources. For the sake of the experiment, an eighth microphone was set in front of a loud speaker that was used to record the noise source. For practical reason, the noise signal was recorded by a microphone and not directly forwarded to the noise canceller stage. This can lead to worse performance because the noise signal can actually contains user's utterances.

Given that, to the best of our knowledge, no dataset of French utterances of voice commands in a noisy multisource home exists, we conducted an experiment to acquire a representative speech corpus composed of utterances of not only home automation orders and distress calls, but also colloquial sentences. In order to get more realistic conditions, two types of background noise were considered while the user 000000000 111111111 0000000 1111111 00000 11111 was speaking: the broadcast news radio and a music (classical) background. These were played in the study through two speakers. Note that this configuration poses much more challenges to classical blind source separation techniques than when speech and noise sources are artificially linearly mixed.

0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1
The protocol was composed of two phases. In Phase 1 the participant was asked to go to the study, to close the door and to read a short text of 285 words. This data was used for the adaptation of the acoustic model of the ASR. In Phase 2, the participant uttered 30 sentences in different rooms in different conditions. The first condition was without noise, the second one was with the radio turned on in the study and the third one was with classical music played in the study. Table 1 summarises the conditions and locations. Each sequence of 30 sentences was composed by a random selection of 21 home automation orders (9 without initiating keyword), 2 distress calls (e.g., " À l'aide"(help), "Appelez un docteur" (call a doctor)) and 7 casual sentences (e.g., "Bonjour" (Hello), "J'ai bien dormi" (I slept well)). No participant uttered the same sequences. The radio and the music were unique and pre-recorded and were started at a random time for each participant.

23 persons (including 9 women) participated to the experiment. The average age of the participants was 35 years (19-64 min-max). No instruction was given to any participant about how they should speak or in which direction. Consequently, no participant emitted sentences directing their voice to a particular microphone. The distance between the speaker and the closest microphone was about 2 meters. The total duration of the experiment was about 5 hours At the end of the experiment, the dataset was composed, for each speaker, of a text of 285 words for Phase 1 (36 minutes for 351 sentences in total for the 23 speakers), and of 240 short sentences for Phase 2 (2 hours and 30 minutes per channel in total for the 23 speakers) with a total of 5520 sentences overall, 2760 of which being instances in noisy conditions (38 minutes of Radio and 37 of music). Each sentence was humanly annotated on the best Signal-to-Noise Ratio (SNR) channel. In clean condition, 1076 voice commands and 348 distress calls were uttered while they were respectively 489 and 192 in radio background noise and 412 and 205 with music. Only the microphone data were used in this study (i.e., no video data). In the following, this corpus is called the SWEET-HOME Home Automation Speech Corpus.

PROPOSED APPROACH FOR ROBUST ASR

To detect voice commands in the SWEET-HOME context, we propose a three-stage approach. The first one detects speech activities in the audio streams, the second one extracts the best utterance hypotheses using an ASR system and the last one recognises a vocal command or a distress situation from the decoded utterances. This paper describes the two last stages. For a description of the first stage, the reader is referred to [START_REF] Vacher | Development of Audio Sensing Technology for Ambient Assisted Living: Applications and Challenges[END_REF].

To address the issues of the SWEET-HOME context (i.e., noise, distant-speech) and to benefit from it (i.e., multiple microphones which are continuously recording), we proposed to test the impact of some state-of-the-art and novel techniques that fuse the streams of information at three independent levels of the speech processing: acoustic signal enhancement, decoding enhancement, and ASRs output combination (see [START_REF] Lecouteux | Distant Speech Recognition in a Smart Home: Comparison of Several Multisource ASRs in Realistic Conditions[END_REF]). Despite a good improvement of voice command recognition, the method did not include a systematic treatment of the background noise. This section focuses on the implemented techniques for noise cancellation in the case of known noise sources and on the method for vocal order recognition with a specific dataset different from that of our previous work [START_REF] Lecouteux | Distant Speech Recognition in a Smart Home: Comparison of Several Multisource ASRs in Realistic Conditions[END_REF].

Known source noise cancellation

Listening to the radio and watching TV are very frequent everyday activities; this can seriously disturb a speech recogniser at two levels: firstly, speech emitted by the person in the flat can be altered by loudspeakers and badly recognised, and secondly, radio and TV sounds and speech will be analysed by the ASR although their information is not relevant. It is thus mandatory to cancel the radio or the TV noise. To do so we used an acoustic echo cancellation technique (AEC).

The AEC processing chain is described in Figure 3. In AEC, the sound emitted by a noise source x(n) (here the Radio loudspeaker in the smart home) is altered by the room acoustics. The resulting noise y b (n) of this alteration can be expressed by a convolution product in the time domain y b (n) = h(n) * x(n), h being the impulse response of the room and n the discrete time. This noise is then mixed with the interesting signal e(n) emitted in the room (here the voice order). The signal recorded by the microphone is then y(n) = e(n) + h(n) * x(n). To cancel the noise, an adaptive filter that estimates the impulse response of the room ĥ(n) is generally used to generate an estimate of the original e(n) following the formula:

ν(n) = e(n)+y b (n)-ŷ(n) = e(n)+h(n) * x(n)-ĥ(n) * x(n)
The noise filter is adapted using the residual ν(n) leading to a tracking system where ν is the feedback. At the begin-

y (n) b ŷ(n) ĥ(n) Habitant Loud speaker x(n) e(n) Noise reference Microphone channel (noised signal) Room acoustic Microphone i v(n) Output signal h(n) y(n) MIC1 MIC2 MIC i

Adaptative filter

Fig. 3. Echo cancellation principle for noise cancellation ning of the process, some time is needed to learn ĥ and decrease the error below a specific threshold: this is the convergence time. When no x(n) signal is present, the filter adaptation tends to diverge because both y b and ŷ are zero. Thus AEC must be applied only when background noise is present. It is also generally required that e stays close to zero otherwise the useful signal (here the domotic order) is seen as an additive noise and the adaptation becomes unstable. This is known as double-talk (i.e., two sources emitting at the same time: here voice order and background noise). To prevent this problem, robust echo cancellers require adjustment of the learning rate to take the presence of double talk in the signal into account. Most echo cancellation algorithms attempt to explicitly detect double-talk but this approach is not very successful, especially in presence of a stationary background noise. In our case, the voice commands used in the experiment are expected to the sufficiently short and separated by silence periods to not disturb the adaptation process. We used the SPEEX library whose AEC stage is based on a multidelay block frequency (MDF) algorithm [START_REF] Valin | A new robust frequency domain echo canceller with closed-loop learning rate adaptation[END_REF]. Noise cancellation was operated separately on the 7 microphone channels.

Voice order recognition

Once noise filtered, the channels feed a multisource ASR system. The ASR system under consideration is the Speeral tool-kit [START_REF] Linarès | The LIA speech recognition system: from 10xRT to 1xRT[END_REF] by the the LIA (Laboratoire d'Informatique d'Avignon). Given the targeted application of SWEET-HOME, and its real-time constraints, the 1xRT Speeral configuration was used (decoding time similar to signal duration). At the decoding level, a novel version of the Driven Decoding Algorithm (DDA) was applied within Speeral. DDA aims to align and correct the a priori transcripts using the speech recognition engine [START_REF] Lecouteux | Generalized driven decoding for speech recognition system combination[END_REF]. This algorithm improves the system performance dramatically by taking advantage of the availability of the predefined transcripts (e.g., automatic speech recognition of a journalist speech using her discourse text as a priori information)

In the smart home context, the system knows the grammar of the voice orders and has multiple sources. Thus, in this DDA version, the a priori transcript is given by decoding a first channel. Then, at each new generated assumption of the ASR system, the current ASR assumption is aligned with this a priori transcript (from the previous decoding pass). Then, a matching score α is computed and integrated with the lan-Fig. 4. DDA 2-level: vocal orders are recognised from the first decoded stream, the result is then used to drive the decoding of the second stream guage model [START_REF] Lecouteux | Generalized driven decoding for speech recognition system combination[END_REF]:

P (w i |w i-1 , w i-2 ) = P 1-α (w i |w i-1 , w i-2 )
where P (w i |w i-1 , w i-2 ) is the updated trigram probability of the word w i knowing the history w i-2 , w i-3 , and P (w i |w i-1 , w i-2 ) is the initial probability of the trigram.

The applied strategy is dynamic and uses, for each utterance to decode, the best channel for the first pass and the second best channel for the last pass. This approach was extended to take into account a priori knowledge about the expected utterances. The ASR system is driven by vocal orders recognised during the first pass : speech segments of the first microphone are projected into the 3 -best vocal orders by using an edit distance and injected via DDA into the ASR system for the fast second pass as presented in Figure 4.

By using DDA, the output of the first microphone drives the output of the second one (cf. Figure 4). This approach presents several benefits: -the second ASR system speed is boosted by the approximated transcript (only 0.1xReal-Time), -DDA merges truly and easily the information from the two streams while other strategies (such as ROVER) do not merge ASR systems outputs, -while a strict usage of the grammar may bias toward voice orders, the projection of the voice orders does not prevent the ASR recognising colloquial sentences.

EXPERIMENTS AND RESULTS

In all experiments, the Phase 1 corpus was used for development and training whereas the Phase 2 corpus served for the evaluation. This section presents the ASR tuning and the experimental results of the proposed approaches.

In the study, the acoustic models were trained on about 80 hours of annotated speech. Furthermore, acoustic models were adapted to each of the 23 speaker by using the Maximum Likelihood Linear Regression (MLLR) on the Phase 1 data. Table 2. Home automation and distress detection in three cases: music, Broadcast News (BN) and noiseless

For the decoding, a 3-gram Language Model (LM) with a 10K lexicon was used. It results from the interpolation of a generic LM (weight 10%) and a domain LM (weight 90%). The generic LM was estimated on about 1000M of words from the French newspapers Le Monde and Gigaword. The domain LM was trained on the sentences generated using the grammar. The LM combination biases the decoding towards the domain LM but still allows decoding of out-of-domain sentences. A probabilistic model was preferred over using strictly the grammar because it makes its possible to use uncertain hypotheses in a fusion process for more robustness.

Results of the approaches are presented in Table 2. In this study we focus on the voice order recognition (classification) stage of each speech event into one of three classes: home automation orders, distress calls and neutral sentences (i.e., sentences that are neither home automation orders nor distress). The recognition is evaluated using recall/precision/F-measure triplet. During the detection, a genuine voice order/distress call is considered as detected only if it completely matches the grammar/distress call sentences. Neutral sentences are considered detected as long as they are not recognized as distress call or voice orders. All other cases are considered as incorrect classification. For each approach, the presented results are the average over the 23 speakers. For the sake of comparison, results of a baseline system (without DDA nor denoised stream) are provided.

The baseline without noise presents a home automation recall of 62% and distress recall of 84%. This better detection can be explained by the lower number of possibilities of distress call expressions than voice orders which reaches about 400. When DDA is used, voice order detection rise to 92.7% and distress call detection is slightly improved (87.2%). Its impact is better for voice command detection because it introduces directly the grammar in the ASR and in the case of distress recall, acts as a combination between two microphones.

In the case of broadcast news, home automation recall falls to 29.3% while distress recall decreases to 74.3%. The introduction of DDA double the home automation detection (57.2%) but has no effect on the distress calls. Using the AEC system both the home automation detection (42.6%) and the distress detection (79.4%) are increased. Finally, the best configuration is obtained by combining the two approaches: the home automation detection is then dramatically increased to 83.5% and distress detection to 81.2%.

In the music context, results are surprising in two points. The music does not seem to impact strongly the ASR as the results with music are just slightly below the results without noise. When the AEC system is used, the performances are overall worst than without it. Thus AEC does not seem adapted to this kind of noise. It must be emphasized that there is only one participant for which the AEC improved the results, and in this case the music was set at a very loud level. Regarding DDA, as in the other cases, results are improved for all classes excepted when it is used in conjunction with the AEC stage.

In all configurations, accuracy of the home automation orders recognition is improved by using DDA: the recognition rate is above 80%. The AEC approach makes a significant difference in case of broadcast news, but does not seem adapted to other background noise.

CONCLUSION

This paper describes an approach for voice command recognition in multi-room smart-homes where audio information is captured by several microphones in a distant speech context. This approach is designed to perform in various known background noise. Given that a fully equipped smart home is likely to have a media network (such as UPnP), we performed experiments considering two noise sources that can be known: broadcast news and music (in contrast with other independent source such as a drilling). The approach acts at two levels of the ASR system by applying an AEC and DDA.

Very good results are obtained. Using the DDA, more than 80% of well detected voice commands and distress calls were obtained in noisy and clean conditions. While the DDA is the most conclusive improvement, the denoising method is effective only in the case of radio broadcasts as it leads to very poor performance in music condition. This might be due to the fact that the AEC system introduces noise and non-linear distortion. We will make some further experiments in order to analyse this aspect. It must be emphasized that the baseline ASR system and the DDA one are far less perturbed by music than by voice. Indeed, when a speaker speaks on radio, her words are decoded by the ASR system, thus introducing many errors. But in the case of music, some of the spectrum is filtered by the acoustic models. The AEC is thus highly relevant for this specific application when voice background noise is present, because, in the context of home automation, it is plausible that the system knows what radio or TV broadcasts the person is listening. The future step of our work is to study the use of classical denoising methods like BSS in our context with real time constraints when interference source in unknown (i.e. vacuum).

Fig. 1 .

 1 Fig. 1. Excerpt of the grammar of the voice orders (terminal symbols are in French)

Table 1 .

 1 Position of the microphones in the DOMUS Home Sentence number as function of the room (Phase 2)

	Main entrance
					1
	Kitchen	M2 M3	M5 Bedroom	M7 Study	000000 111111 Mi Microphone Window
	M1	Bath-room	M4	M6 M8	Door Loud speaker
			00000 11111	00000 11111	000000 111111
	Fig. 2. External noise	Study		Bedroom	Bathroom	Kitchen
	None	30		30	30	30
	Music (radio)	30		30	-	-
	Speech (broadcast news)	30		30	-	-

20th European Signal Processing Conference (EUSIPCO 2012) Bucharest, Romania, August 27 -31, 2012 © EURASIP, 2012 -ISSN 2076-1465

This work is a part of the SWEET-HOME project founded by the French National Research Agency (Agence Nationale de la Recherche / ANR-09-VERS-011).


Pedro Chahuara

François Portet

Michel Vacher

Context Aware Decision System in a Smart Home: Knowledge Representation and Decision Making Using Uncertain Contextual Information

Keywords: Ambient intelligence and pervasive computing, Decision making, Frameworks for formalizing context and context-aware knowledge representation, Reasoning under uncertainty

HAL is

Introduction

As the development of Smart Homes (SH) has gained a growing interest among many communities such as medicine, architecture, computer sciences, etc. two major challenges have emerged in the area of Ambient Intelligence. Firstly, the need for knowledge representation models featuring high readability, modularity and expressibility. Secondly, the requirement to develop decision making methods that can leverage knowledge models to take context the particular situation under which a decision is taken and its uncertainty into account. Indeed, in most cases the information gathered to infer context comes from sources aected by uncertainty and imprecision.

In the literature, logical models, mostly ontologies and logic rules, seem to have reach a consensus due to the high readability and expressibility they oer. This work is part of the Sweet-Home project founded by the French National Research Agency (Agence Nationale de la Recherche / ANR-09VERS-011)

The Open AAL platform [START_REF] Wolf | Soprano -an extensible, open aal platform for elderly people based on semantical contracts[END_REF] uses an ontology that describes in-home entities belonging to low and high abstraction levels. The framework designed around this ontology is appropriate to facilitate the integration of devices from dierent providers, as they share a common taxonomy, and the implementation of computational methods to make context inference. The independence between knowledge representation and inference methods guarantees modularity, however it does not take advantage of the reasoning capacities supported by logical reasoners, as the only purpose of the ontology is to be an artefact of integration. Chen et al. [START_REF] Chen | Ontology-based activity recognition in intelligent pervasive environments[END_REF] have proposed a method to perform activity recognition in home, an important element of context awareness, by using subsumption checking in an ontology, but uncertainty is not supported in this work. A more general approach was designed by Liao [START_REF] Liao | A rdf and owl-based temporal context reasoning model for smart home[END_REF], in which some context elements, such as level of risk, are dened through logic rules using RDF-based events to perform activity recognition. However, uncertainty of the information sources is not considered even if a prior probability of risk is estimated. Answer Set Programming (ASP) is another logic approach for representation and reasoning that has been applied by Mileo et al. [START_REF] Mileo | Reasoning support for risk prediction and prevention in independent living[END_REF] to estimate the evolution of the inhabitant's health state. They present a framework that can properly deal with reasoning under incompleteness and uncertainty. Furthermore, the knowledge encoded in the ASP rules could be integrated into an ontology as well. Although their approach is very relevant for context recognition, they have not developed formal decision models containing essential elements such as utilities, risks and actions. On the side of decision methods for SH dealing with uncertainty, several Bayesian approaches have been suggested, as in the SOCAM project [START_REF] Gu | A service-oriented middleware for building context-aware services[END_REF]. Inuence diagrams [START_REF] Howard | Inuence diagrams. Readings on The Principles and Applications of Decision Analysis[END_REF], which are based on Bayesian networks, have been also applied to model the causal relation among decision actions, uncertain variables, risk, and utilities [START_REF] Nishiyama | Development of agent system based on decision model for creating an ambient space[END_REF][START_REF] De | C@sa: Intelligent home control and simulation[END_REF]. However in these works, the decision process is not supported by a formal knowledge representation that can be exploited in other tasks besides decision.

It seems that there exists a gap between the development of formal models to represent knowledge in pervasive environments and the methods for decision making that must act under uncertain information. We are tackling this problem in the Sweet-Home project, a new smart home system whose main man-machine interaction modality is based on audio processing technology. Our proposed solution involves the representation of concepts by means of ontologies and a set of logical rules. It takes advantage of description logic reasoners and SWRL for situation recognition and obtains a system adaptable to other SH implementations, as well. In the decision stage, a part of the logical rules is employed to construct an inuence diagram based on Markov Logic Networks (MLN), a statistical method that makes probabilistic inference from a model consisting of weighted logic rules. The rest of this paper describes the Sweet-Home framework. Sections 2 presents the project and Section 3 the framework architecture. Section 4 shows the ontologies and how situation recognition is performed. A detailed explanation of our decision making model is given in Section 5. Finally Section 6 concludes with a brief discussion. This research is related to the Sweet-Home project (http://sweet-home.imag. fr), a French national supported research project aiming at designing a new smart home system based on audio technology which focuses on three main aspects: to provide assistance via natural human-machine interaction (voice and tactile command), to ease social interaction and to provide security reassurance by detecting situations of distress. In this project, the SH under consideration is DOMUS, a at lled in with sensor technology which was set up by the Multicom team of the Laboratory of Informatics of Grenoble. This 30m 2 suite at, depicted in Figure 1, is equipped with sensors and actuators such as infra-red presence detectors, contact sensors, video cameras (used only for annotation purpose), etc. This kind of smart home can support daily living by making context-aware decision base on the current situation the user is. To illustrate this support let's consider the following two scenarios: The inhabitant arrives to her apartment at night and goes to the bedroom immediately, forgetting to lock the door. She prepares to sleep and turns all the lights o but the bedside lamp as she usually reads before sleeping. After some minutes, she turns o the lamp and, at this moment, from the sequence of her interactions with the environment, the system recognizes that she is about to sleep, and a relatively dangerous situation is recognized as the main door is not locked. A decision could result in sending a message through a speech synthesizer considering the risk of interrupting her rest to remind her of the state of the door.

Scenario 2

The inhabitant wakes up in the middle of the night and utters the vocal order "Turn on the light". This simple command requires context information (location and activity) to realize which light to turn on and what the appropriate intensity is. In this case, the system decides to turn on the bedside lamp with a middle intensity since the ceiling light could aect her eyes sensitivity at that moment.

From these scenarios it can be noticed that contextual information, such as location and activity, plays a major role to deliver appropriate support to the user. In this paper we dene Location and Activity as follows:

Denition 1 (Location). l(t) ∈ L, where L is the set of predened locations in the SH and t ∈ N is the time, species where the inhabitant is located.

In this work a specic area corresponds to a room and we assume a single inhabitant in the environment.

Denition 2 (Activity). Routine activities performed during daily live; such as, sleeping, cooking, or cleaning. In an instant t the activity might be undetermined; so an activity occurrence, a is dened in an interval of time, A(t begin , t end ). Thus A : t b , t e → a, t b , t e ∈ N and t b < t e Moreover, many more information can be inferred from the raw data such as agitation, communication, etc. They are dened as sources of information: Denition 3 (Source of Information). The system contains a set of variables V that describes the environment. A source of information is a variable V i ∈ V with domain Dom(V i ) representing the information provided by a sensor or a inference process i.

Denition 4 (System state). If

Υ is the set of possible values of V , a system state is an assignment v ∈ Υ making V = {V 1 = v 1 , V 2 = v 2 , ..., V n = v n }
The Situation is dened by: Denition 5 (Situation). A situation S ⊂ Υ is dened by a set of constraints

C = {C k1 1 , C k2 2 , ..., C km m }, where each constraint C ki i establish a set A i ⊂ DOM (V ki )
to constraint the value of a source of information V ki . Thus S = {v/∀C ki i ∈ C, v ki ∈ A i } For example, if we have two sources of information, V 1 and V 2 , corresponding to the the state of the main door and the location of the inhabitant, a situation can be dened by constraints, C 1 1 , C 2 2 , holding the following sets:

A 1 = {open}, A 2 = {study, bedroom}.
Denition 6 (Temporal Situation). Let's consider a temporal sequence of system states δ = (v t1

1 , v t2 2 , ..., v tn n ) where t i is the time of occurrence. A temporal situation R, is dened by a set of constraints T = {T 1 , T 2 , ..., T m }, where each T k denes a pair of situations (S 1 k , S 2 k ) and an interval [a k , b k ] such as R = {(v ti i , v tj j )/∀T k ∈ T, v ti i , v tj j ∈ δ, v i ∈ S 1 k , v j ∈ S 2 k , a k t j -t i b k } Thus, if a temporal constraints T 1 establish an interval [t i , t j ],
a temporal situation will be recognized when two instances of the situations S 1 and S 2 occur with a dierence of time falling into the interval. In the rest of the paper we refer to temporal situations simply as situations.

Based on our study of the context, we dene it as follows:

Denition 7 (Context). Set of informations characterizing the circumstance under which an inference is made.

The main usage of context is disambiguation. When a situation is recognized, context provides the complementary information to evaluate the circumstance in terms of a certain quality Q ∈ {risk, comf ort, saf ety, ...}. Let's a function F Q assigning a value, in the scope of Q, to a situation S. The nal value of F Q depends on the information contained in the context κ : F Q (S|κ). This function in our work is given by the decision model.

3

The Sweet-Home System: an Audio-controlled Smart

Home

The input of the Sweet-Home system is composed of the information from the domotic system transmitted via a local network and information from the microphones transmitted through radio frequency channels. The microphone data is processed by an audio processing chain delivering hypotheses about the sound or the sentences being uttered by the user [START_REF] Vacher | The sweet-home project: Audio technology in smart homes to improve well-being and reliance[END_REF]. All these streams of information (audio and domotic) are captured by an intelligent controller which interprets them to recognize situations and makes decisions. The diagram of this intelligent controller is depicted in Figure 2. The knowledge of the controller is dened using two semantic layers: the low-level and the high-level ontologies which are described in the next section. Besides knowledge representation, another role of the ontologies is to store the events from which inference is carried out. The estimation of the current situation is carried out through the collaboration of several processors, each one being specialized in a specic source of information. All processors share the knowledge specied in both ontologies and use the same repository of facts. Furthermore, the access to the knowledge base is executed under a service oriented approach that allows any processor being registered to be notied only about particular events and to make inferred information available to other processors. This data and knowledge centred approach ensures that all the processors are using the same data structure and that the meaning of each piece of information is clearly dened among all of them.

We have considered that the main aspects for situation recognition are the location of the inhabitant, the current activity and the period of the day. These informations are useful to eliminate ambiguity in the decision making process.

For example, in Scenario 2, when the vocal order Turn on the light is uttered by the inhabitant, in order to decide which light must be activated, the controller infers inhabitant's location. Furthermore, there can be many lights in the same room, so if the command is given in the middle of the night after the inhabitant has interrupted her sleep, knowing the previous activity and time period helps to infer that the best choice of light are the bedside lamps. Other works have also reckoned location and activity as fundamental for context inference [START_REF] Mileo | Reasoning support for risk prediction and prevention in independent living[END_REF][START_REF] Schilit | Context-aware computing applications[END_REF].

In order to perform location and activity inference, two independent modules were developed and integrated in the framework. Due to space limitation the reader is referred to [START_REF] Chahuara | Fusion of Audio and Temporal Multimodal Data by Spreading Activation for Dweller Localisation in a Smart Home[END_REF][START_REF] Chahuara | Méthodes SVM et MLN pour la reconnaissance automatique d'activités humaines dans les habitats perceptifs: tests et perspectives[END_REF] for further details. Ontologies and Rules for Situation Recognition

The intelligent controller performs inference in several stages, from raw input data until the evaluation of situations. Each event is produced by the arrival of a sensor information. These events are considered of low level as they do not require inference. Once they are stored in the facts base, processing modules are executed hierarchically (e.g., location then activity then situation). Thus, each inference corresponding to a high level event is stored in the database and used subsequently by the next module. Within the controller architecture, other inference modules can be added without compromising the processing of the other components. The two ontologies were designed, not only for domain knowledge representation, but also for storing the events resulting from the processing modules. Furthermore, situations are dened within the ontologies allowing description logic reasoners to evaluate if a situation is happening. Consequently, the importance of the ontology transcends the mere description of the environment.

Low and High Level Ontologies

The knowledge of the controller is dened using two semantic layers: the low-level and the high-level ontologies. The former ontology is devoted to the representation of raw data and network information description. State, location, value and URI of switches and actuators are examples of element to be managed at this level. The high level ontology represents concepts being used at the reasoning level. These concepts are organized in 3 main branches: the Abstract Entity, the Physical Entity,and the Event concept that represents the transient observations of one abstract entity involving zero or several physical entities (e.g., at 12:03 the dweller is sleeping). Instances in the high-level ontology are produced by the inference modules (e.g. activity, location, and situations) after treating information coming from sensors. This separation between low and high levels makes possible a higher re-usability of the reasoning layer when the sensor network and the home must be adapted [START_REF] Klein | Ontology-centred design of an ambient middleware for assisted living: The case of soprano[END_REF].

Figure 3 shows some of the concepts and relations of both ontologies. The ABoxes serve as an example of the state of the fact base at a certain moment. Let's refer to the scenario 1 when the inhabitant turns the bedside lamp o to sleep. The controller updates devices states in the low level ontology and it can be inferred, still at a low level, that every light in the room is o. In the high level ontology, the interaction with the switch lamp is stored as a device event having time and room as properties. At this stage, the module on charge of location is requested and it gives a straightforward answer as the switch is placed in the bedroom. Then, the evidences of the inhabitant being in the bedroom, having all lights turned o, and the evening as the period of the day, can be used to infer that the current activity is sleeping. Finally, these inferences provide the context on which situation recognition is applied. Under the same scenario, if the inhabitant forgot to close the main door and a situation was dened for this case, the situation will be labelled as detected in the ontology. Detected situations are treated by the decision module explained in section 5.

Application of SWRL to Situation Recognition

A situation can be seen as a temporal pattern of the system state which is given by the facts base. Ontologies provide an appropriate foundation for situation recognition since they store all the facts and a complete semantic description of the environment as well. Furthermore, temporal representation can be achieved by means of role properties among event concepts dening temporal relations such as previous and next which, through chaining property of OWL2, can generate the after and before relations. Under some restrictions, Datalogs describing situations as logic rules can be transformed in description logic and written on ontologies [START_REF] Hitzler | Ontologies and rules[END_REF]. However the scope of this approach is very limited as it does not allow to specify complex denitions. Even when it is limited to safe rules, it overcomes several restrictions of description logics while having the denitions still as part of the ontology. In addition, SWRL builtin functions further extend the semantics of context denitions. A possible situation denition in SWRL, based on scenario 1, is given below:

DeviceEvent(?d), has_associated_object(?d, door), takes_place_in(?d, kitchen),state_value(?d, open), DeviceEvent(?l), has_associated_object(?l, setLights), takes_place_in(?l, kitchen),state_value(?l, o), temp:after(?l,?d) → current_state(LightsOOpenMainDoor, detected)

We assume that these events reect the current state of the system. Note that a high level, events can be dened by means of sets of devices as well.

Decision Making

The decision making module is the main component of the intelligent controller. When a situation is recognized, this module employs the high level knowledge in order to construct dynamically a decision model that takes into account the context and its degree of uncertainty. In this section we briey describe the base method used for decision making, and give details about our implementation.

Markov Logic Networks (MLN)

MLN [START_REF] Richardson | Markov logic networks[END_REF] combines rst-order logic and Markov Networks, an undirected probabilistic graphical model. A MLN is composed of a set of rst-order formulas each one associated to a weight that expresses a degree of truth. This approach soften the assumption that a logic formula can only be true or false. A formula in which each variable is replaced by a constant is ground and if it consists of a single predicate is a ground atom. A set of ground atoms is a possible world.

All possible worlds in a MLN are true with a certain probability which depends on the number of formulas they agree with and the weights of these formulas.

A MLN, however, can also have hard constraints by giving a innite weight to some formulas, so that worlds violating these formulas have zero probability. Let's consider F a set of rst-order logic formulas, i.e. a knowledge base, w i ∈ R the weight of the formula f i ∈ F , and C a set of constants. During the inference process [START_REF] Richardson | Markov logic networks[END_REF], every MLN predicated is grounded and Markov network M F,C is constructed where each random variable corresponds to a ground atom. The obtained Markov network allows to estimate the probability of a possible world P (X = x) by the equation 1:

P (X = x) = 1 Z exp fi∈F w i n i (x) (1)
where Z = x ∈χ exp fi∈F w i n i (x ) is a normalisation factor, χ the set of possible worlds, and n i (x) is the number of true groundings of the i-th clause in the possible world x. Exact inference in MLN is intractable in most cases, so Markov Chain Monte Carlo methods are applied [START_REF] Richardson | Markov logic networks[END_REF].

Learning an MLN consists of two independent tasks: structure learning and weight learning. Structure can be obtained by applying machine learning methods, such as Inductive Logic Programming, or rules written by human experts. Weight learning is an optimisation problem that requires learning data. The most applied algorithm in the literature is Scaled Conjugate Gradient [START_REF] Lowd | Ecient weight learning for markov logic networks[END_REF].

Inuence Diagrams with MLN

Inuence diagrams [START_REF] Howard | Inuence diagrams. Readings on The Principles and Applications of Decision Analysis[END_REF] are probabilistic models used to represent decision problems. They result from an extension of Bayesian networks composed only of state nodes by the inclusion of two types of node: actions and utilities. An action node is a variable corresponding to a decision choice. The state nodes in the Bayesian network represent how the variables in the problem domain are affected by the actions. Finally, utility nodes are variables that represent the value obtained as consequence of decisions made. Formally, given a set of actions A, an assignment of choices to these actions a, a ∈ A, is taking according to its utility function, U : X → [0, 1], where X is the state of the random variables in the network after the decision is made. The expected utility for the assignment of choices a is computed as: EU (a) = X P (X|a, e)U (X) Where e is the evidence given to the network. The process of nding the optimal decision, i.e. where a decision is made as a response to a vocal order Turn on the light. In this case, the setting of action variables, represented by rectangular nodes, designates which lights devices in the environment are operated and the intensity of the lights. Round nodes are the state nodes aected by the decision. Among the state nodes, information belonging to the context is bound within a dashed area. There are two variables inuencing directly the utility: the comfort of the inhabitant and the suitability of the activated lights location that ideally should be the same of the inhabitant. Note that this location is not easy to determine in some cases since the inhabitant could be moving in the environment while uttering the vocal order. Since a Markov network is a more general probabilistic model than a Bayesian network, Inuence diagrams can also be implemented by means of MLN [START_REF] Nath | A language for relational decision theory[END_REF]. Nath et al. [START_REF] Nath | Ecient belief propagation for utility maximization and repeated inference[END_REF] have proposed an algorithm that evaluates all the choices in a set of actions without executing the whole inference process for each choice resulting in an ecient way to estimate the optimal assignation. We have considered this approach suitable for implementing decision making in our framework for two main reasons: First at all, MLNs are dened through logical rules which can be stored in an ontological representation, using the concepts already established in order to keep a standard vocabulary besides achieving decision model readability. Secondly, it allows to deal with the uncertainty related to context variables and evidence. This model must be constructed dynamically since the probability of context variables, location and activity, can not be known a priori. These variables are provided by the specialised modules of location and activity that supply also a probability for their inference results. These results are uncertain evidences. To introduce them into the MLN model, we have followed an approach similar to the one implemented by Trans et al. [START_REF] Tran | Event modeling and recognition using markov logic networks[END_REF]. Therefore, after the vocal command is received, the context variable values are requested by the decision module, the decision model is constructed using the rules saved in the ontology and decision inference is performed using MLN. Given xed values for the action nodes, Light-Location and Intensity; the inference will give the probability for each grounding of the utility predicates, RightArea and Comfort. Let's consider the case where action nodes are xed as a = (LightLocation(kitchen), Intensity(low)), then for this conguration we obtain the following expected utility: The optimal assignment of actions will be the one having the maximal EU . Dealing with context in pervasive environments involves treating uncertainty, imprecision, and incompleteness; and so far, not a single method can overcome all these problems. Therefore Ambient Intelligence projects must rely on the application of several methods sharing a common base and serving each one a specic purpose. Our proposed framework is an attempt towards this direction.

Activity

EU (a) =
Decision making by means of Markov logic networks seems very promising as it can take the best of logic and probabilistic models: a simple and clear representation in the framework while being able to treat uncertainty through probabilistic inference. However, as most of probabilistic models, MLN learning requires a considerable amount of data to estimate the optimal parameters. Unfortunately, corpora on pervasive environments with annotated data useful for decision making is rarely available. Furthermore, to the best of our knowledge there is no available corpora for decision making from vocal orders. Therefore, we plan, in the short term, to carry out experiments on a real SH platform that will provide us with data to optimize our decision models and to test the complete framework in realistic circumstances.

To further improve our framework, we intend to work on two improvements: the rst one relates a tighter integration of the decision model with the ontology. We consider very interesting the possibility to check for coherence of the decision model rules by means of an ontology reasoner. In general, this integration is not trivial as MLN rules are dened in rst-order logic, while description logic and safe rules are only a subset of rst-order logic. Our second idea consists in extending the semantics of the situation recognition module in order to be able to dene situations in terms of complex events.

  Scenario 1

Fig. 2 .

 2 Fig. 2. The Intelligent Controller Diagram.

4

 4

Fig. 3 .

 3 Fig. 3. The low and high level ontologies.

Fig. 4 .

 4 Fig. 4. Inuence diagram for a decision after a vocal order is recognised.

Figure 4

 4 Figure 4 shows an example of Inuence Diagram, based on the scenario 2,

  x∈{bad,acceptable,good} P (RightArea(x) | a).U (RightArea(x)) + x∈{low,medium,high} P (LightLocation(x) | a).U (LightLocation(x))

6

  Discussion and Future Work

A MLN for the inuence diagram in gure 4 can be dened as follows: